LocalAI
Open-source AI runtime that runs LLMs, vision, voice, and agents on your machine without requiring a GPU. Compatible with OpenAI, Anthropic, and Ollama APIs.
LocalAI is an open-source alternative to:
LocalAI is an open-source AI engine designed to make AI accessible on any hardware, from laptops to GPU clusters. It provides a unified runtime for running text, voice, vision, image, video, 3D, and agent-based models locally, eliminating the need for cloud-based AI services.
Key Features
- Multi-Modal Support: Run text, voice, vision, images, video, 3D, and agents from a single runtime.
- Hardware Agnostic: Works on CPUs, GPUs (CUDA, ROCm, Metal, Vulkan), and ARM64 without requiring specialized hardware.
- OpenAI-Compatible API: Seamlessly integrates with existing clients using OpenAI, Anthropic, Ollama, or ElevenLabs APIs.
- Dynamic Engine Switching: Swap between backends (llama.cpp, vLLM, MLX, etc.) with a single line in the model's config.
- Realtime Capabilities: Enable real-time voice interactions, transcription, and speech synthesis over WebRTC without a GPU.
- Distributed AI: Scale across multiple machines with smart routing, VRAM-aware placement, and failover support.
- Privacy-First: All processing happens locally; no data leaves your machine.
- 1,255+ Pre-Configured Models: One-click installation for a wide range of models.
- 19 Custom Engines: Built from scratch for specific tasks like real-time voice, vision, and 3D reconstruction.
- Agent & MCP Support: Run autonomous agents, RAG, and interactive tools locally.
Tech / Self-Hosting
- Self-Hosted: Fully self-hostable on any hardware, including edge devices.
- Open Source: MIT-licensed, available on GitHub.
- Cross-Platform: Supports x86_64, ARM64, and various GPU frameworks (CUDA, ROCm, Metal, Vulkan).
- Scalable: From a single laptop to a distributed cluster of machines.
- Documentation: Comprehensive docs for setup, configuration, and advanced use cases.