Download
Self-hosted, community-driven, local OpenAI compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU is required. Runs ggml, GPTQ, onnx, TF compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many others. LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer-grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU.
Features
- Local, OpenAI drop-in alternative REST API
- NO GPU required
- Supports multiple models
- Once loaded the first time, it keep models loaded in memory for faster inference
- Doesn’t shell-out, but uses C++ bindings for a faster inference and better performance
- You own your data
Project Samples
License
MIT LicenseFollow LocalAI
Other Useful Business Software
Deploy Apps in Seconds with Cloud Run
Cloud Run is the fastest way to deploy containerized apps. Push your code in Go, Python, Node.js, Java, or any language and Cloud Run builds and deploys it automatically. Get fast autoscaling, pay only when your code runs, and skip the infrastructure headaches. Two million requests free per month. And new customers get $300 in free credit.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of LocalAI!
