jundot/omlx

Mac-optimized LLM inference server github.com/jundot/omlx

Contains the canonical GitHub repository URL.

oMLX cache really makes a difference when you start dealing with large or repeating contexts. Examples are initial prompts of Hermes Agent or any coding agents. In this case it's autoresearch in pi mono iterating on experiments on an M3 Ultra using gemma-4-26b-a4b-it-8bit 🔥

Matched through a repo-specific project phrase query.

github.com/jundot/omlx

Contains the canonical GitHub repository URL.

setting up my @NousResearch hermes agent to build itself in a sandbox it will train the qwen3.5-9B along with the dflash model + oMLX for cache/gen + access to arxiv for research it will use an autoresearch loop he has 14 days before his main brain becomes qwen3.5-9b

Matched through a repo-specific project phrase query.

Speculative decoding never works for me, and now DFlash doesn’t either. Is there a trick? I tried Qwen 3.5‑9B and the model performs worse, especially in reasoning. I often get only a 5–6‑token boost, usually even fewer. I tested this with oMLX?

Matched through a repo-specific project phrase query.

⭐ Trending on GitHub Today: > jundot/omlx (2.8K stars, +264 stars) - LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar > NousResearch/hermes-agent (3.9K stars, +159 stars) - The agent that grows with you > promptfoo/promptfoo (12K stars, +141 stars) - Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration. > karpathy/autoresearch (23.2K stars, +1,140 stars) - AI agents running research on single-GPU nanochat training automatically > msitarzewski/agency-agents (25.9K stars, +995 stars) - A complete AI agency at your fingertips - From frontend wizards to Reddit community ninjas, from whimsy injectors to reality checkers. Each agent is a specialized expert with personality, processes, and proven deliverables.

Contains the exact GitHub repo slug.

So which model was your favorite? TIL about oMLX thanks!

Matched through a repo-specific project phrase query.

Current CLI engines supporting DFlash on Mac (Apple Silicon/MLX): - **dflash-mlx** (primary): pip install dflash-mlx CLIs: dflash (generation/chat), dflash-serve (OpenAI-compatible server wrapping mlx_lm), dflash-benchmark. Auto-detects drafts for Qwen3.5+. - **oMLX** (experimental): Enable DFlashEngine in admin UI for Qwen3.5 models; runs via omlx.cli serve (short ctx only, auto-fallback). mlx-lm has no native support yet (open request). No others like Ollama.

Matched through a repo-specific project phrase query.

Hi, @ivanfioravanti I'm looking for someone to record a stres test on the OMLX engine on a Mac M5 Max 128 GB: - 4, 6, and 8 agents running in parallel at the same time - Each agent must have a different task - Fill context up to 64k tokens for each - Use Qwen 3.5/3.6 35B #omlx

Matched through a repo-specific project phrase query.