Best Inference AI tools
Serving model predictions in production — routing, batching and accelerated pipelines. The 15 highest-ranked of the 17 apps tagged Inference in the Onei AI catalogue.
Everything tagged Inference: apps, skills and MCP servers
-
The open-source AI hub: over a million models, hundreds of thousands of datasets, and hosted demo Spaces.
Also: Developers, Hosting, Open Source · Editor's pick
-
One API and one bill for hundreds of language models, with automatic failover between providers.
Also: API, Developers, LLM Gateway · Editor's pick
-
Unsloth AI
Open-source desktop app to run, serve and fine-tune text, image, video and audio models entirely on your own machine.
Also: Fine-tuning, GPU, Local Models · Editor's pick
-
Open-source inference engine that runs LLMs, vision and speech models fully on-device — no API keys, no cloud calls, from Swift to Godot.
Also: Local Models, Open Source · Editor's pick
-
Jun Kim
Menu-bar LLM inference server for Apple Silicon, with continuous batching and tiered KV caching that keeps local models fast enough for real coding work.
Also: Open Source, Privacy · Editor's pick
-
Privacy-first AI gateway putting 600+ models from 90+ providers behind one OpenAI-compatible API, with no prompt or output logging.
Also: API, Developers, LLM Gateway · Editor's pick
-
Cactus Compute, Inc.
8-29 MB automation models that run on phones, wearables and microcontrollers — tool calling, structured extraction and embeddings, fully offline.
Also: Fine-tuning, Local Models, Open Source
-
Simulate how fast a local LLM runs on a given GPU or Mac before you buy the hardware.
Also: GPU
-
Low-latency speech APIs from the team behind the open-source Dia model — text-to-speech and transcription priced well below the incumbents.
Also: API
-
Open-source local inference server that profiles your hardware, picks models that fit, and points your coding agent at them — free, private and offline.
Also: Local Models, Open Source, Privacy
-
ngrok
A hosted LLM gateway from ngrok: one base URL and one key for public providers, your own keys and self-hosted models, with failover, scoped access and cost analytics.
Also: API, Developers, LLM Gateway
-
Production inference for open and custom models — fast, autoscaling, and deployable into your own cloud.
Also: Developers, GPU, Hosting
-
Serverless GPUs from a Python decorator — deploy models and batch jobs with no containers or cluster to manage.
Also: Developers, GPU, Hosting
-
Inference, fine-tuning, and GPU clusters for open models — the full stack for teams building on open weights.
Also: API, Developers, Fine-tuning
-
A generative media platform tuned for speed — image, video, and audio models served with very low latency.
Also: API, Developers
How this list is ranked
Ranked by editorial weight (0–100), plus 5 points per bookmark up to 10, plus or minus up to 20 for the average review rating; ties go to featured apps, then the most recently updated. Discontinued apps are never ranked.