Audience
Developers looking for a solution to deploy and manage AI workloads on GPUs
About NVIDIA Cloud Functions
NVIDIA Cloud Functions (NVCF) is a serverless API designed to deploy and manage AI workloads on GPUs, providing security, scalability, and reliability. It supports HTTP polling, HTTP streaming, and gRPC protocols for accessing workloads. Cloud Functions is primarily suited for shorter-running, preemptable workloads such as inferencing and fine-tuning. It supports two function types: "Container" and "Helm Chart", allowing users to define functions based on their specific requirements. Workloads are ephemeral and preemptable, so users should ensure they save their work appropriately. Models, containers, helm charts, and additional resources are hosted and pulled from the NGC Private Registry. To get started, users can follow the functions quickstart guide, which provides an end-to-end workflow for creating and deploying a container-based function using the fastapi_echo_sample container.
Other Popular Alternatives & Related Software
IronFunctions
IronFunctions is an open source serverless platform, also known as a Functions-as-a-Service (FaaS) platform, that allows developers to write functions in any language and deploy them across various environments, including public, private, and hybrid clouds. It supports AWS Lambda function formats, enabling seamless import and execution of existing Lambda functions. Designed for both developers and operators, IronFunctions simplifies coding by allowing the creation of small, focused functions without the need to manage the underlying infrastructure. Operators benefit from efficient resource utilization, as functions consume resources only during execution, and the platform's scalability is managed by adding more IronFunctions nodes as needed. It is built using Go and leverages container technologies to handle incoming workloads by spinning up new containers, processing the payloads, and returning responses.
Learn more
NVIDIA DGX Cloud Serverless Inference
NVIDIA DGX Cloud Serverless Inference is a high-performance, serverless AI inference solution that accelerates AI innovation with auto-scaling, cost-efficient GPU utilization, multi-cloud flexibility, and seamless scalability. With NVIDIA DGX Cloud Serverless Inference, you can scale down to zero instances during periods of inactivity to optimize resource utilization and reduce costs. There's no extra cost for cold-boot start times, and the system is optimized to minimize them. NVIDIA DGX Cloud Serverless Inference is powered by NVIDIA Cloud Functions (NVCF), which offers robust observability features. It allows you to integrate your preferred monitoring tools, such as Splunk, for comprehensive insights into your AI workloads. NVCF offers flexible deployment options for NIM microservices while allowing you to bring your own containers, models, and Helm charts.
Learn more
AWS Lambda
Run code without thinking about servers. Pay only for the compute time you consume. AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app. AWS Lambda automatically runs your code without requiring you to provision or manage servers. Just write the code and upload it to Lambda. AWS Lambda automatically scales your application by running code in response to each trigger. Your code runs in parallel and processes each trigger individually, scaling precisely with the size of the workload.
Learn more
JFrog Artifactory
The Industry Standard Universal Binary Repository Manager.
Supports all major package types (over 27 and growing) such as Maven, npm, Python, NuGet, Gradle, Go, and Helm including Kubernetes and Docker as well as integration with leading CI servers and DevOps tools that you already use.
Additional functionalities include:
- High Availability that scales to infinity with active/active clustering of your DevOps environment and scales as business grows
- On-Prem, Cloud, Hybrid, or Multi-Cloud Solution
- De Facto Kubernetes Registry managing application packages, operating system’s component dependencies, open source libraries, Docker containers, and Helm charts with full visibility of all dependencies. Compatible with a growing list of Kubernetes cluster providers.
Learn more
Company Information
NVIDIA
Founded: 1993
United States
docs.nvidia.com/cloud-functions/index.html
Videos and Screen Captures
Other Useful Business Software
Auth0 for AI Agents now in GA
Connect your AI agents to apps and data more securely, give users control over the actions AI agents can perform and the data they can access, and enable human confirmation for critical agent actions.
Product Details
Platforms Supported
Cloud
Training
Documentation
Live Online
Webinars
In Person
Videos
Support
Phone Support
24/7 Live Support
Online
