Envoy AI Gateway
A CNCF Envoy Gateway-based solution for routing and managing AI service requests with enterprise-grade observability and security.
Envoy AI Gateway is an open-source alternative to:
Envoy AI Gateway is an open-source project built on CNCF Envoy Gateway to handle request traffic from application clients to GenAI services. It provides a stable, production-ready control plane API, supports 16+ AI providers, multimodal capabilities, and enterprise observability.
Key Features
- CNCF Envoy Gateway Foundation: Leverages the proven Envoy proxy for high-performance traffic routing
- Multi-Provider Support: Native integration with 16+ LLM providers including OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, and Mistral
- Stable Control Plane API: Committed-stable API for production deployments
- MCP Gateway: Enables Model Context Protocol (MCP) tool calls with per-user identity management
- Multimodal Support: Handles text, image, and other multimodal AI requests
- Enterprise Observability: Built-in monitoring and metrics for AI service performance
- Header Forwarding: Per-user identity management without requiring OAuth infrastructure
- Scalability: Benchmarked to handle 2,000+ routes with low latency
- Community-Driven: Open development with weekly meetings and active GitHub discussions
Tech / Self-Hosting
- Open Source: Fully open-source under the Apache License
- Self-Hostable: Can be deployed on any Kubernetes cluster
- CNCF Project: Part of the Cloud Native Computing Foundation ecosystem
- Documentation: Comprehensive docs available at https://aigateway.envoyproxy.io/docs
- Community Support: Active Slack channel and weekly community meetings