Skip to content

Best Inference AI tools

Serving model predictions in production — routing, batching and accelerated pipelines. The 15 highest-ranked of the 20 apps tagged Inference in the Onei AI catalogue.

Everything tagged Inference: apps, skills and MCP servers Best Inference agent skills, ranked

  1. The open-source AI hub: over a million models, hundreds of thousands of datasets, and hosted demo Spaces.

    Also: Developers, Hosting, Open Source · Editor's pick

  2. One API and one bill for hundreds of language models, with automatic failover between providers.

    Also: API, Developers · Editor's pick

  3. Unsloth AI

    Open-source desktop app to run, serve and fine-tune text, image, video and audio models entirely on your own machine.

    Also: Fine-tuning, GPU, Local Models · Editor's pick

  4. Open-source inference engine that runs LLMs, vision and speech models fully on-device — no API keys, no cloud calls, from Swift to Godot.

    Also: Local Models, Open Source · Editor's pick

  5. Jun Kim

    Menu-bar LLM inference server for Apple Silicon, with continuous batching and tiered KV caching that keeps local models fast enough for real coding work.

    Also: Open Source · Editor's pick

  6. Privacy-first AI gateway putting 600+ models from 90+ providers behind one OpenAI-compatible API, with no prompt or output logging.

    Also: API, Developers · Editor's pick

  7. Cactus Compute, Inc.

    8-29 MB automation models that run on phones, wearables and microcontrollers — tool calling, structured extraction and embeddings, fully offline.

    Also: Fine-tuning, Local Models, Open Source

  8. Run thousands of open models with one API call, and deploy your own without touching Kubernetes.

    Also: API, Developers, Fine-tuning

  9. NVIDIA GPU cloud for training and inference: self-serve on-demand instances billed by the minute, 1-Click Clusters, and large private superclusters.

    Also: Developers, GPU, Machine Learning

  10. Serverless GPUs from a Python decorator — deploy models and batch jobs with no containers or cluster to manage.

    Also: Developers, Fine-tuning, GPU

  11. Production inference for open and custom models — fast, autoscaling, and deployable into your own cloud.

    Also: Developers, Fine-tuning, GPU

  12. A generative media platform tuned for speed — image, video, and audio models served with very low latency.

    Also: API, Developers, GPU

  13. Inference and training platform for open models: serverless per-token APIs, dedicated GPU deployments, and managed fine-tuning with SFT, DPO and RFT.

    Also: API, Developers, Fine-tuning

  14. GPU cloud for AI developers: on-demand GPU pods, serverless GPU endpoints that scale to zero, and multi-node clusters, billed by the second.

    Also: Developers, Fine-tuning, GPU

  15. Simulate how fast a local LLM runs on a given GPU or Mac before you buy the hardware.

    Also: GPU

How this list is ranked

Ranked by editorial weight (0–100), plus 5 points per bookmark up to 10, plus or minus up to 20 for the average review rating; ties go to featured apps, then the most recently updated. Discontinued apps are never ranked.

More ranked lists