Lum1104/Understand-Anything+606 stars 24hREPO
anthropics/financial-services-plugins+396 stars 24hREPO
rohitg00/ai-engineering-from-scratch+331 stars 24hREPO
addyosmani/agent-skills+327 stars 24hREPO
colbymchenry/codegraph+327 stars 24hREPOLum1104/Understand-Anything+606 stars 24hREPOanthropics/financial-services-plugins+396 stars 24hREPOrohitg00/ai-engineering-from-scratch+331 stars 24hREPOaddyosmani/agent-skills+327 stars 24hREPOcolbymchenry/codegraph+327 stars 24hQuantization shrinks a model by storing its weights at lower numeric precision — for example 4-bit instead of 16-bit — cutting memory and speeding inference with a small quality trade-off.
Quantization is what lets large open models run on modest hardware. By representing weights with fewer bits, a model that needed a server can fit on a consumer GPU or a laptop. Formats like GGUF, GPTQ, and AWQ package quantized weights for different runtimes.
The trade-off is size and speed versus accuracy: aggressive quantization saves the most memory but can degrade quality, so runtimes offer several levels. It is a core enabling technique for the local-LLM ecosystem.
Hundreds of models & providers. One command to find what runs on your hardware.
A powerful model compression framework for LLMs and LVLMs, adapted for NVIDIA GPUs and Huawei Ascend NPUs.
⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.