Local LLM Runners

Compare tools for running AI models locally on your own hardware. From desktop apps to inference frameworks - run LLMs privately without cloud APIs.

Tools

Open Source

CLI Runner

Ollama

CLI RunnerOpen Source

The easiest way to run LLMs locally. Simple CLI with one-command model downloads. Supports GGUF models on Mac, Linux, and Windows.

Dead simpleOne commandOpenAI-compatible API

Pricing: Free (open source)

Desktop App

LM Studio

Desktop App

Desktop app for discovering, downloading, and running local LLMs. Beautiful UI with built-in model search and chat interface.

Beautiful UIModel discoveryChat interface

Pricing: Free

Jan

Desktop AppOpen Source

Open-source ChatGPT alternative that runs 100% offline. Desktop app with local model management and OpenAI-compatible API.

100% offlinePrivacy-firstExtensions

Pricing: Free (open source)

GPT4All

Desktop AppOpen Source

Desktop app for running local LLMs with focus on simplicity. Includes curated model library and document Q&A features.

Simple setupDocument Q&ACurated models

Pricing: Free (open source)

Inference Engine

llama.cpp

Inference EngineOpen Source

The foundational C/C++ inference engine that powers most local LLM tools. Highly optimized for CPU and GPU inference with quantization support.

Maximum performanceGGUF formatCPU + GPU

Pricing: Free (open source)

vLLM

Inference EngineOpen Source

High-throughput LLM serving engine. PagedAttention for efficient memory management. Best for production self-hosting scenarios.

Production servingPagedAttentionHigh throughput

Pricing: Free (open source)

Web Interface

Open WebUI

Web InterfaceOpen Source

Self-hosted web interface for running LLMs. Supports Ollama and OpenAI-compatible backends. Extensible with plugins and tools.

Web-basedMulti-backendPlugin system

Pricing: Free (open source)

text-generation-webui

Web InterfaceOpen Source

Feature-rich web UI for running LLMs. Supports multiple backends (GPTQ, GGUF, ExLlama). Extensive customization options.

Feature-richMultiple backendsExtensions

Pricing: Free (open source)

Frequently Asked Questions

We track 8 local llm runners tools. Ollama is among the most popular options. See our full directory above for pricing, features, and open-source status of each tool.

Yes, several local llm runners offer free tiers or are fully open-source. Check the pricing column in our directory to find free and open-source options.

Consider your specific needs: budget (free vs paid), whether you need open-source, integration requirements, and the specific AI models you want to use. Our directory includes feature highlights to help you compare.

Browse Other Directories

LLM Observability Tools

6 tools