Download
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API.
Features
- Web layer and task coordination built with Rust, which offers blazing speed in addition to efficient CPU utilization powered by async I/O
- User interface purely in Python, by which users can serve their models in an ML framework-agnostic manner using the same code as they do for offline testing
- Aggregate requests from different users for batched inference and distribute results back
- Spawn multiple processes for pipelined stages to handle CPU/GPU/IO mixed workloads
- Designed to run in the cloud, with the model warmup, graceful shutdown, and Prometheus monitoring metrics, easily managed by Kubernetes or any container orchestration systems
- Focus on the online serving part, users can pay attention to the model optimization and business logic
Project Samples
License
Apache License V2.0Follow Mosec
Other Useful Business Software
AI-powered service management for IT and enterprise teams
Give your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of Mosec!
