Skip to main content
Drop a repoSign up
NEWvicrodh/qbzjust droppedNEWkowalski-ptr/compass-before-compactjust droppedNEWlastmile-ai/mcp-agentjust droppedREPOxai-org/grok-build+2097 stars 24hREPOmsitarzewski/agency-agents+659 stars 24hREPOsafishamsi/graphify+599 stars 24hREPOstablyai/orca+566 stars 24hREPOjamiepine/voicebox+537 stars 24hNEWvicrodh/qbzjust droppedNEWkowalski-ptr/compass-before-compactjust droppedNEWlastmile-ai/mcp-agentjust droppedREPOxai-org/grok-build+2097 stars 24hREPOmsitarzewski/agency-agents+659 stars 24hREPOsafishamsi/graphify+599 stars 24hREPOstablyai/orca+566 stars 24hREPOjamiepine/voicebox+537 stars 24h
Glossary

What is local LLM?

A local LLM is a language model that runs on your own machine or servers instead of a hosted API — for privacy, offline use, and cost control.

A local LLM runs inference on hardware you control. With quantization and efficient runtimes, capable open models now run on a laptop or a single GPU. Teams choose local for data privacy, offline operation, predictable cost, and no per-token API fees.

The open-source local-LLM ecosystem covers inference engines, model runners with one-command setup, and UIs — the layer that turns raw model weights into something you can actually chat with or build on.

Best local LLM tools

Trending local LLM projects

  1. open-webui/open-webui

    User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

    145.9K+96 · 24hmomentum 31Python
  2. ollama/ollama

    Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

    176.4K+68 · 24hmomentum 30Go
  3. Mintplex-Labs/anything-llm

    Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

    63.5K+52 · 24hmomentum 29JavaScript
  4. sgl-project/sglang

    SGLang is a high-performance serving framework for large language models and multimodal models.

    30.5K+41 · 24hmomentum 28Python
  5. debpalash/OmniVoice-Studio

    Local voice clone, video dubbing, dictation and audiobook maker. The open-source ElevenLabs alternative.

    8.6K+44 · 24hmomentum 25Python
  6. AlexsJones/llmfit

    Hundreds of models & providers. One command to find what runs on your hardware.

    29.6K+33 · 24hmomentum 25Rust

local LLM — FAQ

What is local LLM?
A local LLM is a language model that runs on your own machine or servers instead of a hosted API — for privacy, offline use, and cost control. A local LLM runs inference on hardware you control. With quantization and efficient runtimes, capable open models now run on a laptop or a single GPU. Teams choose local for data privacy, offline operation, predictable cost, and no per-token API fees.