8 Best Open Source Galileo Alternatives
A curated collection of the 8 best open source and self-hosted alternatives to Galileo in 2026.
Looking for a free, privacy-friendly, or self-hosted replacement for Galileo in 2026? The top open-source alternative is Langfuse — Open source AI agent observability platform.
Other highly rated open-source Galileo alternatives include Arize Phoenix, OpenLLMetry and Helicone.
Open source AI agent observability platform
Langfuse is an open-source platform designed to help teams trace, evaluate, and improve AI agents. By leveraging production data, it provides deep insights into agent behavior, enabling teams to collaborate on fixes and optimize performance for better quality, reduced costs, and lower latency. - LLM Observability: Monitor and debug AI agent interactions in real-time with detailed traces and logs. - Prompt Management: Centralize and version control prompts for consistent AI outputs. - Evaluation & Metrics: Assess agent performance using built-in evaluation tools and custom metrics. - Collaboration Tools: Share insights and collaborate on fixes with your team. - Self-Hosting Support: Deploy Langfuse on your infrastructure using Docker, Kubernetes, or cloud providers (AWS, GCP, Azure). - Scalability: Handle millions of observations per month with high-performance data processing. - Integration Ready: Works seamlessly with popular AI frameworks like LangChain and OpenAI. - Open Source: Fully open-source with a vibrant community and enterprise-grade features. Langfuse is built with modern technologies and supports multiple deployment options: - Docker Compose: Quick local or cloud deployment. - Kubernetes (Helm): Scalable production deployments. - Cloud Providers: AWS, GCP, and Azure via Terraform. - Database: Powered by ClickHouse for high-performance analytics. Langfuse is trusted by over 100,000 engineers and used by 21 of the Fortune 50 companies.
Open-source AI agent development platform
Phoenix is an open-source platform designed for AI agent development and evaluation. It provides comprehensive observability, tracing, and evaluation tools to help AI engineering teams build, monitor, and improve their AI applications. - AI Agent Development: Build and test AI agents with integrated tracing and debugging tools. - Observability & Monitoring: Gain deep insights into AI agent performance and behavior. - Evaluation & Benchmarking: Measure and compare AI agent capabilities with built-in evaluation frameworks. - Self-Hosting: Deploy Phoenix locally or in your own infrastructure for full control. - Integration with Coding Agents: Works seamlessly with popular coding agents like Claude, Codex, and Cursor. - Open-Source: Fully open-source under Apache 2.0 license. - Enterprise-Grade: Used by Fortune 500 companies for production AI deployments. Phoenix is open-source and can be self-hosted using Docker, Kubernetes, or directly via CLI. It supports Python and TypeScript SDKs for easy integration with existing AI workflows. Documentation and quick-start guides are available for local setup and production deployment.
Open-source observability for LLMs with OpenTelemetry
OpenLLMetry is an open-source observability platform designed specifically for Large Language Model (LLM) applications. It provides comprehensive tracking, monitoring, and debugging capabilities using the OpenTelemetry standard, enabling developers to gain deep insights into their LLM workflows. - LLM-Specific Instrumentation: Automatically captures and traces LLM interactions, including prompts, completions, and tool usage - OpenTelemetry Integration: Seamlessly integrates with existing OpenTelemetry-based observability stacks - Multi-Provider Support: Works with major LLM providers including OpenAI, Anthropic, Mistral AI, Cohere, and more - Multi-Platform Compatibility: Supports Python and TypeScript implementations - Trace Visualization: Provides detailed trace views for debugging complex LLM workflows - Performance Monitoring: Tracks latency, token usage, and error rates across LLM calls - Cost Optimization: Helps identify expensive LLM operations and optimize usage - Extensible Architecture: Modular design allows for custom instrumentation and extensions OpenLLMetry is open-source software that can be self-hosted. The project provides comprehensive documentation for deployment and configuration. It requires an OpenTelemetry-compatible backend for storing and visualizing traces, with support for popular platforms like Instana, Dynatrace, Grafana Tempo, Honeycomb, New Relic, Datadog, and Splunk.
Open-source LLM observability and monitoring platform
Helicone is the LLMOps platform powering the fastest-growing AI companies. It enables developers to route, debug, and analyze their AI applications with real-time observability, monitoring, and analytics. - LLM Observability: Real-time monitoring and debugging of LLM interactions. - AI Gateway: Route requests to multiple LLM providers seamlessly. - Analytics Dashboard: Track performance, usage, and costs across models. - Prompt Management: Store, version, and optimize AI prompts. - Rate Limiting & Alerts: Prevent abuse and get notified of anomalies. - Multi-Provider Support: Integrate with OpenAI, Anthropic, Azure, and more. - HQL (Helicone Query Language): Query and analyze AI interactions efficiently. - Datasets & Playground: Test and refine AI models with interactive tools. Helicone is open-source and can be self-hosted. It supports integration with major LLM providers and offers a cloud-based solution for ease of use. The platform is designed for scalability and reliability, making it suitable for both small projects and enterprise-level AI applications.
Open-source AI agent observability platform
Latitude is an open-source AI agent observability and monitoring platform designed to help teams debug, monitor, and improve their AI agents in production. - Self-Healing Agents: Automatically detects failures in production AI agents and dispatches coding agents (like Claude Code or Cursor) to fix issues with minimal intervention. - Conversation Intelligence: Analyzes completed sessions to extract behavior patterns, escalations, resolutions, and failure modes. - Automatic Issue Discovery: Groups recurring failures, prioritizes them, and provides trend analysis with failing traces attached. - Real-Time Alerts: Sends notifications via Slack, email, or webhooks when new issues are detected or existing ones escalate. - OTEL Compatibility: Integrates seamlessly with OpenTelemetry pipelines without proprietary formats or vendor lock-in. - Custom Signals: Allows defining custom signals for tracking specific failure modes, policies, or quality checks. - Dataset Management: Automatically builds golden datasets from validated production traces for evaluation and regression testing. - Human Annotations: Enables inline feedback on traces, spans, or outputs to improve structured signal collection. - Advanced Filters: Supports semantic search, exact text, and metadata filters for cohort analysis (e.g., "frustrated users on GPT-5.5 in prod after the May release"). - Failure Mode Clustering: Groups similar failing traces into issues with examples, trends, and affected user metrics for efficient triaging. Latitude is open-source and can be self-hosted. It supports integration with existing OpenTelemetry pipelines and provides an MCP server for running the entire monitoring loop within your editor. The platform is designed to scale with production workloads and offers comprehensive documentation for setup and customization.
AI agent testing and evaluation platform
AI agent testing and evaluation platform that transforms unpredictable agents into reliable production systems. LangWatch provides simulations, evaluations, observability, and governance to ensure your AI agents perform consistently before and after deployment. - Agentic AI Testing: Run realistic user scenarios against your agent to catch issues before production - LLM Evaluation: Measure response quality and accuracy to ship agents that hold up in production - LLM Observability: Trace every agent step and monitor cost and latency with full production visibility - AI Governance: Govern every model, key, and tool with virtual keys, budgets, routing policies, and audit trails - Prompt Management: Version, deploy, and A/B test prompts as code with full history and GitHub sync - Voice AI Testing: Simulate and test voice AI agents at scale before customer interactions - LLM Red-teaming: Uncover safety and security gaps through simulated attacks - Spec-driven Testing: Automatically turn requirements into agent tests for continuous improvement - Self-improving Loops: Set up automated agent improvement cycles based on production traces - CI/CD Integration: Run scenarios locally and in CI pipelines with no separate setup LangWatch is open source and can be self-hosted in 15 minutes. The platform supports local development, CI/CD integration, and enterprise-grade deployment options. Documentation is available at docs.langwatch.ai. Technologies include: - TypeScript/JavaScript for core platform - PostgreSQL for data storage - Redis for caching and real-time features - Docker for containerization - Kubernetes for orchestration (enterprise deployments) Self-hosting is supported via Docker Compose and Kubernetes Helm charts. Enterprise features include advanced security, SSO, and dedicated support.
Open source AI observability and harness platform
OpenLIT is an open-source platform designed to streamline AI-native development workflows by providing comprehensive observability, evaluation, and management capabilities for LLMs and AI agents. - AI Observability: Trace LLMs and AI agents using OpenTelemetry for real-time monitoring and debugging. - LLM Evaluations: Run automated evaluations to assess model performance and reliability. - Prompt Management: Centralize and version-control prompts for consistent AI outputs. - Model Comparison: Compare different AI models to optimize cost, performance, and accuracy. - API Key Vault: Securely store and manage API keys for AI services. - Self-Hosting: Deploy OpenLIT on-premises or in your cloud for full data control. - Apache 2.0 License: Free and open-source under a permissive license. OpenLIT is built with modern technologies and supports self-hosting for complete data privacy. The platform is designed to integrate seamlessly with existing AI workflows and infrastructure, making it ideal for enterprises and developers prioritizing control and compliance.
Open source LLM observability & monitoring
Spanlens provides open-source observability and monitoring for Large Language Model (LLM) applications. It enables developers to log, trace, and analyze every model call, agent interaction, and tool execution across multiple providers including OpenAI, Anthropic, and Gemini. - Request Logging & Cost Tracking: Monitor spend, tokens, and cache savings per model and project with real-time pricing updates. - Agent Tracing: Visualize nested spans across tools, retrievals, and model calls to identify performance bottlenecks. - Prompt Versioning: Pin and compare prompt versions to assess cost, latency, and quality changes over time. - Caching & Evals: Implement exact-match caching to reduce costs and run automated evaluations in CI pipelines. - Multi-Provider Support: Compatible with OpenAI, Anthropic, Gemini, Groq, Mistral, XAI, OpenRouter, and more. - PII Detection: Automatically mask sensitive data (SSN, credit cards, API keys) without blocking requests. - OpenTelemetry Integration: Seamlessly integrate with existing OTel tracing workflows. - Self-Hosting: Deploy via Docker for unlimited requests, custom retention, and full data control. Spanlens is MIT-licensed and can be self-hosted using a simple Docker command. It supports OpenTelemetry (OTLP/HTTP) for ingestion and export, and offers configurable data retention policies. The platform is designed to minimize latency overhead (p99 under 3ms) and ensures requests complete even if ingestion fails.
About Galileo
AI observability and evaluation platform empowers AI teams to evaluate, monitor, and protect GenAI applications and agents at enterprise scale.