Skip to content

Inference

Serving model predictions in production — routing, batching and accelerated pipelines.

21 apps, 14 skills and 1 MCP server tagged Inference.

Best Inference AI tools, ranked

Best Inference agent skills, ranked

Apps

All apps

The open-source AI hub: over a million models, hundreds of thousands of datasets, and hosted demo Spaces.

Coding & DevelopmentFreemium
Featured

One API and one bill for hundreds of language models, with automatic failover between providers.

Coding & DevelopmentFreemium

App: Unsloth

Unsloth AI

Featured

Open-source desktop app to run, serve and fine-tune text, image, video and audio models entirely on your own machine.

Coding & DevelopmentFree
Featured

Open-source inference engine that runs LLMs, vision and speech models fully on-device — no API keys, no cloud calls, from Swift to Godot.

Coding & DevelopmentFree

App: oMLX

Jun Kim

Featured

Menu-bar LLM inference server for Apple Silicon, with continuous batching and tiered KV caching that keeps local models fast enough for real coding work.

Coding & DevelopmentFree

Privacy-first AI gateway putting 600+ models from 90+ providers behind one OpenAI-compatible API, with no prompt or output logging.

Coding & DevelopmentPaid

App: Cactus

Cactus Compute, Inc.

8-29 MB automation models that run on phones, wearables and microcontrollers — tool calling, structured extraction and embeddings, fully offline.

Coding & DevelopmentFreemium

App: LocalAI

LocalAI (Ettore Di Giacinto)

New

Open-source, self-hosted AI engine that runs open models for chat, images, speech, video and music on your own CPU or GPU behind an OpenAI- and Anthropic-compatible API.

Coding & DevelopmentFree

Inference and training platform for open models: serverless per-token APIs, dedicated GPU deployments, and managed fine-tuning with SFT, DPO and RFT.

Coding & DevelopmentPaid

NVIDIA GPU cloud for training and inference: self-serve on-demand instances billed by the minute, 1-Click Clusters, and large private superclusters.

Coding & DevelopmentPaid

GPU cloud for AI developers: on-demand GPU pods, serverless GPU endpoints that scale to zero, and multi-node clusters, billed by the second.

Coding & DevelopmentPaid

Simulate how fast a local LLM runs on a given GPU or Mac before you buy the hardware.

Data & AnalyticsFree

Low-latency speech APIs from the team behind the open-source Dia model — text-to-speech and transcription priced well below the incumbents.

Audio & VoicePaid

Open-source local inference server that profiles your hardware, picks models that fit, and points your coding agent at them — free, private and offline.

Coding & DevelopmentFree

A hosted LLM gateway from ngrok: one base URL and one key for public providers, your own keys and self-hosted models, with failover, scoped access and cost analytics.

Coding & DevelopmentPaid

Production inference for open and custom models — fast, autoscaling, and deployable into your own cloud.

Coding & DevelopmentFreemium

Serverless GPUs from a Python decorator — deploy models and batch jobs with no containers or cluster to manage.

Coding & DevelopmentFreemium

Inference, fine-tuning, and GPU clusters for open models — the full stack for teams building on open weights.

Coding & DevelopmentFreemium

A generative media platform tuned for speed — image, video, and audio models served with very low latency.

Image & DesignFreemium

Run thousands of open models with one API call, and deploy your own without touching Kubernetes.

Coding & DevelopmentFreemium

Inference on custom LPU hardware, built for latency — open models served at speeds general-purpose GPUs struggle to match.

Coding & DevelopmentFreemium

Skills

All skills

Trains, distils, quantises, evaluates, exports and runs inference for RT-DETR real-time object detection in NVIDIA TAO, routing training through AutoML hyperparameter search by default.

Runs NVIDIA's NV-Reason-CXR-3B chest X-ray reasoning model through a documented wrapper, locally on GPU or via the public Hugging Face Space, and returns the model's full reasoning trace as engineering evidence.

Replicate's official skill for running models via its API — predictions, polling, Prefer: wait, webhooks with signature validation, streaming and file handling.

Skill: Transformers.js

by Hugging Face

Run machine learning models directly in the browser or Node with Transformers.js — no inference server required.

Authoritative reference for the Claude API and Anthropic SDKs — model IDs, pricing, parameters, streaming, tool use, MCP, agents, prompt caching, and token counting.

MCP servers

All MCP servers
Featured

Replicate's official MCP server: search thousands of hosted models, read their schemas, and run predictions on image, video, audio and language models from inside an agent.

Related tags

Tags that appear alongside this one, ranked by how often.

All tags