agentops

AgentOps TypeScript SDK

Observability and DevTool platform for AI Agents

A TypeScript implementation of the AgentOps SDK that exports GenAI conventional OpenTelemetry data to standards-compliant OTel collectors. This SDK provides automatic instrumentation for multiple agent frameworks and AI libraries.

Features

• 🔌 Plugin Architecture: Dynamic loading and configuration of instrumentors
• 🤖 GenAI Support: Built-in support for OpenTelemetry GenAI semantic conventions
• 📊 Standards Compliant: Exports to any OpenTelemetry-compatible collector
• 🛠️ Framework Agnostic: Instrument multiple agent frameworks simultaneously
• 🔧 TypeScript First: Full TypeScript support with comprehensive type definitions
• 💸 LLM Cost Management: Track spend with LLM foundation model providers
• 🧪 Agent Benchmarking: Test your agents against 1,000+ evals
• 🔐 Compliance and Security: Detect common prompt injection and data exfiltration exploits

Installation

npm install agentops

Quick Start

1. Set your API key (recommended)

export AGENTOPS_API_KEY=your-api-key

2. Initialize the SDK

import { agentops } from 'agentops';

await agentops.init();

// Your AI agent code here - instrumentation happens automatically!

Alternative: Pass API key explicitly

import { agentops } from 'agentops';

await agentops.init({
  apiKey: 'your-api-key'
});

Building

To build the project from source:

npm install
npm run build

This will compile the TypeScript source code to JavaScript in the dist/ directory.

Running the Example

The repository includes an OpenAI Agents example that demonstrates the SDK in action:

1. First, create a .env file in the examples/openai-agents-example directory:

cd examples/openai-agents-example
cat > .env << EOF
AGENTOPS_API_KEY=your-agentops-api-key
OPENAI_API_KEY=your-openai-api-key
EOF

2. Then run the example:

npm install
npm run dev

The example will:

1. Initialize AgentOps instrumentation
2. Create a weather assistant agent with tool calling capabilities
3. Execute a sample query
4. Export telemetry data to the AgentOps platform

OpenAI Agents Support

AgentOps provides first-class support for the OpenAI Agents SDK, automatically instrumenting:

• Agent Lifecycle: Track agent creation, execution, and completion
• LLM Generation: Capture model requests, responses, and token usage
• Function Calls: Monitor tool usage and function execution
• Audio Processing: Observe speech-to-text and text-to-speech operations
• Handoffs: Track agent-to-agent communication and workflow transitions
• Custom Events: Capture domain-specific agent behaviors

Automatic Instrumentation

Simply initialize AgentOps before using the OpenAI Agents SDK:

import { agentops } from 'agentops';
import { Agent, run } from '@openai/agents';

// Initialize AgentOps first
await agentops.init();

// Create your agent with tools and instructions
const agent = new Agent({
  name: 'My Assistant',
  instructions: 'You are a helpful assistant.',
  tools: [/* your tools */],
});

// Run the agent - instrumentation happens automatically
const result = await run(agent, "Hello, how can you help me?");
console.log(result.finalOutput);

All agent interactions will be automatically captured and exported to your AgentOps dashboard with full OpenTelemetry semantic conventions.

Debug Logging

To see detailed instrumentation and tracing logs:

DEBUG=agentops:* node your-app.js

Why AgentOps? 🤔

Without the right tools, AI agents are slow, expensive, and unreliable. Our mission is to bring your agent from prototype to production. Here's why AgentOps stands out:

• Comprehensive Observability: Track your AI agents' performance, user interactions, and API usage.
• Real-Time Monitoring: Get instant insights with session replays, metrics, and live monitoring tools.
• Cost Control: Monitor and manage your spend on LLM and API calls.
• Failure Detection: Quickly identify and respond to agent failures and multi-agent interaction issues.
• Tool Usage Statistics: Understand how your agents utilize external tools with detailed analytics.
• Session-Wide Metrics: Gain a holistic view of your agents' sessions with comprehensive statistics.

AgentOps is designed to make agent observability, testing, and monitoring easy.

Star History

Check out our growth in the community: