Ollama

Ollama made local language models genuinely easy. ollama run <model> downloads the weights, picks sensible quantisation for your hardware, and drops you into a chat — on macOS, Windows, or Linux, with GPU acceleration when it is available.

What makes it stick

• A real package manager for models. One command pulls a model; another lists what you have; another removes it. Versions and quantisations are handled for you.
• A local API on port 11434. Anything that speaks the OpenAI chat format can point at Ollama instead, which is why so many desktop clients, editor plugins, and agent frameworks support it out of the box.
• Modelfiles. A short, Dockerfile-like description bakes a system prompt, parameters, and adapter into a reusable local model.
• Nothing leaves your machine. For regulated work, offline environments, or anyone who simply prefers it, that is the entire value proposition.

Practical notes

Model quality tracks the open-weight ecosystem, and speed tracks your hardware — a recent Apple Silicon Mac or a discrete GPU makes mid-sized models comfortable. Ollama is free and open source, and it has become the default local runtime that other tools assume you already have.