AgentOps
#20 in Observability & EvalsAgentops · since 2023 (gegründet laut infrabase.ai) · 2× · last seen Jun 29, 2026
AgentOps is an observability and evaluation platform built specifically for AI agents and LLM applications, developed by the eponymous startup AgentOps-AI (founded 2023, San Francisco/Delray Beach). With just a few lines of code (Python/TypeScript SDK), it records session traces, LLM calls, tool usage, and multi-agent interactions, visualized in a dashboard with session replay, cost tracking, and guardrails. The platform natively integrates with 400+ LLMs and agent frameworks such as CrewAI, AutoGen/AG2, OpenAI Agents SDK, and LangChain. The SDK/app is open source under the MIT license, while the hosted dashboard is a commercial product with tiered pricing (free to enterprise).
Features
| Deployment (Self-Hosted/Cloud) | Primarily cloud dashboard; self-hosting of dashboard+API backend possible (app+backend can run locally); Enterprise self-hosting on AWS/GCP/Azure |
| License | SDK/app open source under MIT license; hosted dashboard is commercial |
| Open-Source Option | Yes – the AgentOps app (dashboard + API backend) is open-source under the MIT license on GitHub (github.com/AgentOps-AI/agentops); Python SDK: MIT license |
| Platform | Web dashboard (app.agentops.ai) plus Python and TypeScript/JavaScript SDK |
| Price | Free Basic tier (up to 5,000 events/month); Pro from $40/month; Enterprise custom-priced (SOC-2, HIPAA, self-hosting, SSO) |
| Pricing Model | Freemium: free entry tier, paid plans starting at ~$40/month usage-based (billed by number of tracked events), enterprise plans available on request |
| Protocol Compatibility | Proprietary SDK/event model, not consistently built on OpenTelemetry/OpenInference conventions like competing tools |
| Release Date | Company founded 2023; official X/Twitter account active since October 2023 |
| Self-Hosting Possible | Yes – officially supported; full platform (dashboard + API backend) can be run on your own infrastructure via Docker/Docker Compose; setup guide documented in app/README.md |
| Supported Models/Providers | 400+ LLMs and frameworks, incl. OpenAI, CrewAI, AutoGen/AG2, LangChain, LlamaIndex, CamelAI, SwarmZero, Llama Stack |