Private Inference API

Private Inference API is a cloud-based offering that provides businesses with access to pre-trained machine learning models, which can be easily integrated into their existing applications and workflows. Nebul offers the deployment of models from its own Private NeoCloud, so businesses can leverage Private Inference API to quickly deploy, scale, and customize AI models to meet specific needs.

Nebul offers a wide range of machine learning models, including but not limited to:​

– Natural Language Processing (NLP) models for text analysis, sentiment analysis, and chatbot functionality​

– Vision models for image recognition, object detection, and video analysis​

– Image models for image generation​

– Code models for code generation​

– Embeddings models for objects in a vector space

How efficient a Private Inference API operated is determined by:​

–  Latency: The time it takes for the model to process input and return an output. In a Private Inference API context, low latency is critical for real-time applications such as fraud detection or autonomous systems​

– Throughput: Measures how many inferences or predictions the model can handle per unit of time. This is critical for understanding how the model scales under load​

– Model Size: The memory footprint of the model, which impacts how easy it is to deploy, especially when resources are limited (e.g., on edge devices)​

– Inference Cost: The cost per inference when using the model via Private Inference API, particularly for cloud-based services where processing costs can vary

The price can differ per model due to several reasons:​

– Complexity (and size) of the model​

– Compute and Storage requirements​

– Model performance and accuracy​

– Industry specific models (fine-tuned)​

Next, an important part of the regulatory and compliancy factors that are applicable to all models.