18 Best Open Source LangSmith Alternatives
A curated collection of the 18 best open source and self-hosted alternatives to LangSmith in 2026.
Looking for a free, privacy-friendly, or self-hosted replacement for LangSmith in 2026? The top open-source alternative is Langfuse — Open source AI agent observability platform.
Other highly rated open-source LangSmith alternatives include Arize Phoenix, OpenLLMetry and Helicone.
Open source AI agent observability platform
Langfuse is an open-source platform designed to help teams trace, evaluate, and improve AI agents. By leveraging production data, it provides deep insights into agent behavior, enabling teams to collaborate on fixes and optimize performance for better quality, reduced costs, and lower latency. - LLM Observability: Monitor and debug AI agent interactions in real-time with detailed traces and logs. - Prompt Management: Centralize and version control prompts for consistent AI outputs. - Evaluation & Metrics: Assess agent performance using built-in evaluation tools and custom metrics. - Collaboration Tools: Share insights and collaborate on fixes with your team. - Self-Hosting Support: Deploy Langfuse on your infrastructure using Docker, Kubernetes, or cloud providers (AWS, GCP, Azure). - Scalability: Handle millions of observations per month with high-performance data processing. - Integration Ready: Works seamlessly with popular AI frameworks like LangChain and OpenAI. - Open Source: Fully open-source with a vibrant community and enterprise-grade features. Langfuse is built with modern technologies and supports multiple deployment options: - Docker Compose: Quick local or cloud deployment. - Kubernetes (Helm): Scalable production deployments. - Cloud Providers: AWS, GCP, and Azure via Terraform. - Database: Powered by ClickHouse for high-performance analytics. Langfuse is trusted by over 100,000 engineers and used by 21 of the Fortune 50 companies.
Open-source AI agent development platform
Phoenix is an open-source platform designed for AI agent development and evaluation. It provides comprehensive observability, tracing, and evaluation tools to help AI engineering teams build, monitor, and improve their AI applications. - AI Agent Development: Build and test AI agents with integrated tracing and debugging tools. - Observability & Monitoring: Gain deep insights into AI agent performance and behavior. - Evaluation & Benchmarking: Measure and compare AI agent capabilities with built-in evaluation frameworks. - Self-Hosting: Deploy Phoenix locally or in your own infrastructure for full control. - Integration with Coding Agents: Works seamlessly with popular coding agents like Claude, Codex, and Cursor. - Open-Source: Fully open-source under Apache 2.0 license. - Enterprise-Grade: Used by Fortune 500 companies for production AI deployments. Phoenix is open-source and can be self-hosted using Docker, Kubernetes, or directly via CLI. It supports Python and TypeScript SDKs for easy integration with existing AI workflows. Documentation and quick-start guides are available for local setup and production deployment.
Open-source observability for LLMs with OpenTelemetry
OpenLLMetry is an open-source observability platform designed specifically for Large Language Model (LLM) applications. It provides comprehensive tracking, monitoring, and debugging capabilities using the OpenTelemetry standard, enabling developers to gain deep insights into their LLM workflows. - LLM-Specific Instrumentation: Automatically captures and traces LLM interactions, including prompts, completions, and tool usage - OpenTelemetry Integration: Seamlessly integrates with existing OpenTelemetry-based observability stacks - Multi-Provider Support: Works with major LLM providers including OpenAI, Anthropic, Mistral AI, Cohere, and more - Multi-Platform Compatibility: Supports Python and TypeScript implementations - Trace Visualization: Provides detailed trace views for debugging complex LLM workflows - Performance Monitoring: Tracks latency, token usage, and error rates across LLM calls - Cost Optimization: Helps identify expensive LLM operations and optimize usage - Extensible Architecture: Modular design allows for custom instrumentation and extensions OpenLLMetry is open-source software that can be self-hosted. The project provides comprehensive documentation for deployment and configuration. It requires an OpenTelemetry-compatible backend for storing and visualizing traces, with support for popular platforms like Instana, Dynatrace, Grafana Tempo, Honeycomb, New Relic, Datadog, and Splunk.
Open-source LLM observability and monitoring platform
Helicone is the LLMOps platform powering the fastest-growing AI companies. It enables developers to route, debug, and analyze their AI applications with real-time observability, monitoring, and analytics. - LLM Observability: Real-time monitoring and debugging of LLM interactions. - AI Gateway: Route requests to multiple LLM providers seamlessly. - Analytics Dashboard: Track performance, usage, and costs across models. - Prompt Management: Store, version, and optimize AI prompts. - Rate Limiting & Alerts: Prevent abuse and get notified of anomalies. - Multi-Provider Support: Integrate with OpenAI, Anthropic, Azure, and more. - HQL (Helicone Query Language): Query and analyze AI interactions efficiently. - Datasets & Playground: Test and refine AI models with interactive tools. Helicone is open-source and can be self-hosted. It supports integration with major LLM providers and offers a cloud-based solution for ease of use. The platform is designed for scalability and reliability, making it suitable for both small projects and enterprise-level AI applications.
Open-source AI agent observability platform
Latitude is an open-source AI agent observability and monitoring platform designed to help teams debug, monitor, and improve their AI agents in production. - Self-Healing Agents: Automatically detects failures in production AI agents and dispatches coding agents (like Claude Code or Cursor) to fix issues with minimal intervention. - Conversation Intelligence: Analyzes completed sessions to extract behavior patterns, escalations, resolutions, and failure modes. - Automatic Issue Discovery: Groups recurring failures, prioritizes them, and provides trend analysis with failing traces attached. - Real-Time Alerts: Sends notifications via Slack, email, or webhooks when new issues are detected or existing ones escalate. - OTEL Compatibility: Integrates seamlessly with OpenTelemetry pipelines without proprietary formats or vendor lock-in. - Custom Signals: Allows defining custom signals for tracking specific failure modes, policies, or quality checks. - Dataset Management: Automatically builds golden datasets from validated production traces for evaluation and regression testing. - Human Annotations: Enables inline feedback on traces, spans, or outputs to improve structured signal collection. - Advanced Filters: Supports semantic search, exact text, and metadata filters for cohort analysis (e.g., "frustrated users on GPT-5.5 in prod after the May release"). - Failure Mode Clustering: Groups similar failing traces into issues with examples, trends, and affected user metrics for efficient triaging. Latitude is open-source and can be self-hosted. It supports integration with existing OpenTelemetry pipelines and provides an MCP server for running the entire monitoring loop within your editor. The platform is designed to scale with production workloads and offers comprehensive documentation for setup and customization.
Open-source workspace for building AI agents
Agenta is an open-source workspace designed for building, improving, and sharing AI agents. Whether you're working solo or with a team, Agenta provides the tools to create agents through natural language, refine them with feedback, and deploy them seamlessly. - Chat-based Agent Building: Create custom agents by describing tasks in plain language, eliminating the need for complex coding. - Feedback Loops: Continuously improve agents by collecting and integrating user feedback directly into their workflows. - Team Collaboration: Share agents with your entire team, enabling collaborative development and deployment. - Automation: Schedule agents to run at specific times or trigger them based on events, ensuring they work even when you're not active. - Multi-Model Support: Use self-hosted models or integrate with popular providers like Claude, ChatGPT, or others without changing your agent's code. - MCP & Tool Integration: Connect agents to over 20 tools and services, including GitHub, Slack, Linear, Notion, PostgreSQL, and more. - Pre-built Templates: Start with ready-to-use templates for engineering, customer support, sales, knowledge management, and operations. - Open Standards: Agents are defined by AGENTS.md files, skills, and MCP tools, allowing easy swapping of harnesses, models, or runtimes. Agenta is fully open-source and can be self-hosted on your own infrastructure. It supports local model runners, making it compatible with existing subscriptions like Claude or ChatGPT. The platform is designed for flexibility, allowing you to deploy agents in the cloud or on-premises based on your needs.
AI agent testing and evaluation platform
AI agent testing and evaluation platform that transforms unpredictable agents into reliable production systems. LangWatch provides simulations, evaluations, observability, and governance to ensure your AI agents perform consistently before and after deployment. - Agentic AI Testing: Run realistic user scenarios against your agent to catch issues before production - LLM Evaluation: Measure response quality and accuracy to ship agents that hold up in production - LLM Observability: Trace every agent step and monitor cost and latency with full production visibility - AI Governance: Govern every model, key, and tool with virtual keys, budgets, routing policies, and audit trails - Prompt Management: Version, deploy, and A/B test prompts as code with full history and GitHub sync - Voice AI Testing: Simulate and test voice AI agents at scale before customer interactions - LLM Red-teaming: Uncover safety and security gaps through simulated attacks - Spec-driven Testing: Automatically turn requirements into agent tests for continuous improvement - Self-improving Loops: Set up automated agent improvement cycles based on production traces - CI/CD Integration: Run scenarios locally and in CI pipelines with no separate setup LangWatch is open source and can be self-hosted in 15 minutes. The platform supports local development, CI/CD integration, and enterprise-grade deployment options. Documentation is available at docs.langwatch.ai. Technologies include: - TypeScript/JavaScript for core platform - PostgreSQL for data storage - Redis for caching and real-time features - Docker for containerization - Kubernetes for orchestration (enterprise deployments) Self-hosting is supported via Docker Compose and Kubernetes Helm charts. Enterprise features include advanced security, SSO, and dedicated support.
Open source AI observability and harness platform
OpenLIT is an open-source platform designed to streamline AI-native development workflows by providing comprehensive observability, evaluation, and management capabilities for LLMs and AI agents. - AI Observability: Trace LLMs and AI agents using OpenTelemetry for real-time monitoring and debugging. - LLM Evaluations: Run automated evaluations to assess model performance and reliability. - Prompt Management: Centralize and version-control prompts for consistent AI outputs. - Model Comparison: Compare different AI models to optimize cost, performance, and accuracy. - API Key Vault: Securely store and manage API keys for AI services. - Self-Hosting: Deploy OpenLIT on-premises or in your cloud for full data control. - Apache 2.0 License: Free and open-source under a permissive license. OpenLIT is built with modern technologies and supports self-hosting for complete data privacy. The platform is designed to integrate seamlessly with existing AI workflows and infrastructure, making it ideal for enterprises and developers prioritizing control and compliance.
Open source LLM observability & monitoring
Spanlens provides open-source observability and monitoring for Large Language Model (LLM) applications. It enables developers to log, trace, and analyze every model call, agent interaction, and tool execution across multiple providers including OpenAI, Anthropic, and Gemini. - Request Logging & Cost Tracking: Monitor spend, tokens, and cache savings per model and project with real-time pricing updates. - Agent Tracing: Visualize nested spans across tools, retrievals, and model calls to identify performance bottlenecks. - Prompt Versioning: Pin and compare prompt versions to assess cost, latency, and quality changes over time. - Caching & Evals: Implement exact-match caching to reduce costs and run automated evaluations in CI pipelines. - Multi-Provider Support: Compatible with OpenAI, Anthropic, Gemini, Groq, Mistral, XAI, OpenRouter, and more. - PII Detection: Automatically mask sensitive data (SSN, credit cards, API keys) without blocking requests. - OpenTelemetry Integration: Seamlessly integrate with existing OTel tracing workflows. - Self-Hosting: Deploy via Docker for unlimited requests, custom retention, and full data control. Spanlens is MIT-licensed and can be self-hosted using a simple Docker command. It supports OpenTelemetry (OTLP/HTTP) for ingestion and export, and offers configurable data retention policies. The platform is designed to minimize latency overhead (p99 under 3ms) and ensures requests complete even if ingestion fails.
Build and deploy agentic workflows with AI
Dify is the platform for production-ready agentic workflows. Build agents, knowledge pipelines, models, and tools on one collaborative workspace, then deploy them on Cloud, in your VPC, or self-hosted. - Visual Workflow Builder: Drag-and-drop interface to design agentic workflows without coding. - Knowledge Pipeline: Prepare searchable knowledge bases for RAG applications. - Model & Tool Integration: Supports multiple AI models and third-party tools. - Multi-Deployment Options: Deploy on Dify Cloud, private VPC, or self-hosted via Docker. - Enterprise-Grade Security: Role-based access control, audit logs, and compliance features. - Marketplace: Explore pre-built tools, models, and integrations. - Scalability: Move from prototype to production without rebuilding your stack. Dify is open-source and self-hostable via Docker. The Community Edition allows full control over deployment, data, and infrastructure. Supports PostgreSQL, Redis, and other standard technologies for reliable operation.
Open-source AI agent observability platform
Laminar is an open-source platform designed to help developers trace, evaluate, and debug AI agents in production. It automatically captures agent interactions, costs, and failures, providing actionable insights to improve reliability. - Automated Tracing: Capture LLM calls, tool invocations, sub-agents, costs, and tokens in a readable transcript view. - Failure Detection: AI-powered Signals analyze every agent run to surface unexpected failure modes without manual configuration. - Signal Clustering: Group similar failures into patterns for high-level insights and trend analysis. - Real-time Alerts: Receive notifications in Slack about new failures and clusters. - Debugging Tools: Use the CLI and MCP to query traces, replay checkpoints, and verify fixes. - Cost & Performance Monitoring: Visualize agent costs and performance with heatmaps and detailed metrics. - Integration-Friendly: Works with popular AI frameworks and tools for seamless adoption. Laminar is open-source (Apache 2.0) and can be self-hosted. The platform supports Docker-based deployments and integrates with existing infrastructure. Documentation and setup guides are available on the official website.
AI observability platform for LLMs and apps
Pydantic Logfire is a production-grade observability platform designed specifically for AI applications, agents, APIs, and databases. Built on OpenTelemetry, it provides comprehensive monitoring across your entire stack, not just LLM calls. - Whole-App Tracing: Trace model calls, agent steps, API requests, and database queries in a single timeline - Multi-Language Support: SDKs for Python, JavaScript/TypeScript, and Rust with OpenTelemetry compatibility for other languages - Evals & Scoring: Built-in evaluation framework that runs alongside your application logic - Cost Tracking: Monitor and optimize AI application costs in real-time - PII Scrubbing: Automatic sensitive data redaction in SDKs - Postgres-Flavored SQL: Query spans directly using familiar SQL syntax - Live Debugging: Real-time monitoring of production services with error rates, latency, and traces - Self-Hosting Option: Deployable on your own infrastructure - OpenTelemetry Foundation: Built on industry-standard observability protocols - Self-Hosted Deployment: Available for on-premises or private cloud deployments - Compliance: SOC 2 Type II, HIPAA, and EU data residency compliant - Integration: Seamless integration with existing monitoring and logging systems
Unified AI gateway for 1,600+ LLMs and observability
Portkey AI Gateway is a production stack for Gen AI builders, offering a unified solution for AI gateway, observability, guardrails, governance, and prompt management. Designed to help AI teams go to production efficiently, it eliminates the complexity of integrating multiple LLMs while providing real-time monitoring and governance capabilities. - Unified AI Gateway: Access 1,600+ LLMs via a single API, eliminating the need for multiple integrations. - Real-time Observability: Monitor LLM behavior, catch anomalies early, and manage usage proactively with a comprehensive dashboard. - Guardrails & Governance: Implement usage policies, rate limiting, and compliance controls to ensure safe and responsible AI usage. - Prompt Management: Centralize prompt management to streamline AI workflows and reduce redundancy. - Caching & Cost Optimization: Reduce costs by caching repeated API calls and optimizing token usage. - Production-ready Workflows: Deploy agent workflows with built-in support for caching, retries, and fallbacks. - Multi-Model Support: Seamlessly switch between different LLMs without changing your codebase. - Enterprise Scalability: Designed for enterprise-scale deployments with high availability and reliability. Portkey AI Gateway is available as a cloud service with enterprise-grade features. While the core gateway functionality is cloud-hosted, Portkey provides tools and integrations for self-hosting components like caching and observability dashboards. The open-source repository (https://github.com/portkey-ai/gateway) offers foundational tools for building custom integrations and extensions.
AI memory layer for persistent agent context
Mem0 provides persistent memory infrastructure for AI agents and applications, enabling them to continuously learn from past interactions and maintain context across sessions. Built for production with enterprise-grade security and compliance. - Persistent Memory: Retains user context and preferences across sessions without pipeline changes - Memory Compression Engine: Automatically condenses chat history into compact memories to reduce tokens and latency - Multi-Signal Retrieval: Advanced retrieval system for accurate context matching - Cross-Domain Adaptation: Configurable for healthcare, education, e-commerce, customer support, and sales - Enterprise Governance: SOC 2, HIPAA, GDPR compliant with BYOK and zero-trust architecture - Portable Deployment: Runs on Kubernetes, private cloud, or air-gapped environments - Full Observability: Complete audit logs for every read/write operation - SDK Integration: Python and Node.js SDKs for easy implementation Mem0 is primarily offered as a cloud service with enterprise-grade infrastructure. The platform is SOC 2 Type 1, HIPAA, and GDPR compliant, ensuring robust security and privacy standards. While the core service is managed, Mem0 provides portable deployment options including Kubernetes and private cloud configurations for organizations requiring on-premises or air-gapped solutions. The open-source components and SDKs allow for integration into existing AI agent frameworks and applications, with the main repository available at https://github.com/mem0ai/mem0.
Unified API for 40+ LLM providers
LLM Gateway simplifies working with multiple LLM providers by offering a single, OpenAI-compatible API interface. Whether you're using OpenAI, Anthropic, Google, or 40+ other providers, LLM Gateway handles routing, cost tracking, and performance monitoring—all in one place. - Unified API Interface: Keep your existing OpenAI SDK code; just change the base URL to switch providers. - Multi-provider Support: Access 40+ providers through one integration with no vendor lock-in. - Performance Monitoring: Compare latency, cost, and quality across models to optimize your use case. - Secure Key Management: Centralize all provider keys in one dashboard to avoid scattered credentials. - Self-hosted or Cloud: Deploy on your own infrastructure or use the managed cloud service. - Cost-aware Analytics: Track requests, tokens, total spend, and average cost per 1K tokens. - Per-model/provider Breakdown: Analyze usage and spend by provider and model to identify outliers. - Errors & Reliability Monitoring: Monitor error rates, cache hit rates, and reliability trends. - Project-level Usage Explorer: Drill into each project's requests, models, errors, cache, and costs. - Enterprise Audit Logs: Maintain compliance with comprehensive audit trails. - LLM Guardrails: Prevent prompt injection, detect PII, and block malicious requests. LLM Gateway is open source and can be self-hosted for full control over your infrastructure. Alternatively, use the managed cloud service for a hassle-free experience. The project is SOC 2 Type II certified, ensuring enterprise-grade security and compliance.
Open-source AI gateway with tracing and PII protection
Grepture is a privacy-focused AI gateway that sits directly on the request path to trace, evaluate, and protect every LLM call your application makes. Designed for teams shipping AI features, it provides full observability, built-in evals, and advanced PII redaction without storing your data. - Hot-path proxy: Processes every request in-flight for PII redaction, threat blocking, and rule enforcement before data leaves your network - Full observability: Detailed request logs, multi-step traces with waterfall views, and full-text search across prompts and responses - Built-in evals: LLM-as-judge scoring runs on live traffic with six templates out of the box - Cost tracking: Real-time token cost tracking with per-model breakdowns - Multi-model gateway: Works with OpenAI, Anthropic, Gemini, Cohere, Mistral, AWS Bedrock, Hugging Face, Groq, Replicate, and more - PII redaction: 80+ detection patterns with reversible tokenized redaction (mask & restore with TTL) - Prompt injection protection: Detects and blocks adversarial inputs - Zero-data mode: Processes requests without writing content to disk; only operational metadata is logged - EU-hosted infrastructure: All services run in Frankfurt & Nuremberg with GDPR compliance - Developer-first SDK: Drop-in TypeScript SDK with zero dependencies, CLI for local dev, and pre-commit scanning Grepture is open-source and can be self-hosted. The official repository provides the proxy implementation, SDK, and documentation for deployment. The cloud version is hosted in EU data centers (Germany & Ireland) with zero US data transfers. The system is designed for high availability with 77.93% uptime over 30 days.
First LLM app development platform
Humanloop was the first platform designed specifically for developing and managing LLM applications. It provided teams with tools to build, evaluate, and deploy AI systems while maintaining safety and reliability. - LLM Application Development: First platform dedicated to building and managing LLM applications - Evaluation & Testing: Tools for evaluating AI model performance and reliability - Safety & Governance: Features to ensure safe AI deployment and compliance - Collaboration: Enabled teams to work together on AI projects - Industry Standards: Helped shape standards for AI management and evaluation Humanloop was a cloud-based platform with no self-hosting option. The service has been sunsetted following its acquisition by Anthropic.
AI agent observability and evaluation platform
Arize AI provides a comprehensive platform for observing, evaluating, and improving AI agents in production. The platform helps teams trace agent workflows, assess performance, and implement continuous learning cycles to enhance agent capabilities. - Agent Observability: Real-time monitoring of agent workflows, inputs, and outputs - Evaluation & Scoring: Automated evaluation of agent responses and performance metrics - Tracing & Debugging: Detailed trace logs for troubleshooting and optimization - Continuous Learning: Feedback loops to improve agent decision-making over time - Multi-Agent Support: Tools for managing and coordinating multiple AI agents - Integration Hub: Connects with popular AI tools like Claude Code, Codex, Cursor, GitHub Copilot, and Gemini - Open-Source Option: Phoenix OSS for self-hosted observability and evaluations - Managed Platform: Arize AX available as a cloud service - Open-Source Core: Phoenix OSS available on GitHub for self-hosting - CLI Tools: Easy setup via npx for quick deployment - Multi-Platform Support: Works across Linux, Windows, and macOS
About LangSmith
LangSmith is a complete AI agent and LLM observability platform designed to help teams debug, monitor, and improve their AI agents in production. It provides real-time tracing, performance tracking, and cost analysis to ensure agents operate efficiently and reliably.
Key Features of LangSmith
- Real-time Monitoring: Track agent behavior, costs, and latency with live dashboards and alerts.
- Tracing & Debugging: Capture detailed traces of agent actions to identify failures and bottlenecks.
- Performance Evaluation: Score and compare agent performance using built-in evaluation tools.
- Multi-Framework Support: Integrate with any agent stack via SDKs for Python, TypeScript, Go, and Java.
- Sandboxed Execution: Safely run agent-generated code in isolated environments.
- LLM Gateway: Control and route model calls to optimize performance and costs.
- Deployment & Scaling: Ship and scale agents in production with ease.
- No-Code Agent Builder: Create and deploy agents without writing code.
Tech / Self-Hosting
LangSmith is a cloud-based platform with enterprise-grade security and compliance. While primarily offered as a managed service, LangSmith supports self-hosting for organizations requiring full data control. It is built to integrate seamlessly with LangChain’s open-source agent frameworks (e.g., LangGraph, DeepAgents) and other AI tools.