Download
API-for-Open-LLM is a lightweight API server designed for deploying and serving open large language models (LLMs), offering a simple way to integrate LLMs into applications.
Features
- Provides a REST API for serving open LLMs
- Supports multiple backends, including Hugging Face models
- Enables GPU and CPU-based inference
- Offers token streaming for real-time responses
- Supports user authentication and request management
- Open-source and customizable for different use cases
Project Samples
License
Apache License V2.0Follow API-for-Open-LLM
Other Useful Business Software
Gemini 3 and 200+ AI Models on One Platform
Build generative AI apps with Vertex AI. Switch between models without switching platforms.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of API-for-Open-LLM!
