What is omlx.ai?

omlx is an LLM inference server for macOS that enables local AI on Apple Silicon with a native menu bar app and web dashboard for model management, chat, and real-time metrics. It implements paged SSD key-value caching (safetensors) with a two-tier RAM/SSD LRU policy to persist cache blocks across requests and restarts, reducing recomputation and improving time-to-first-token (TTFT). Continuous batching and a batch generator increase throughput and concurrency, with benchmarked generation speedups (eg. up to ~4.14× at 8× concurrency on M3 Ultra). Multi-model serving supports LLMs, VLMs, embeddings, and rerankers simultaneously with LRU eviction, plus built-in model browsing and Hugging Face/LM Studio cache support. OpenAI- and Anthropic-compatible API endpoints (/v1/chat/completions, /v1/messages) and compatibility with Claude Code, OpenClaw, Cursor enable drop-in backend use and tool calling via JSON, Qwen, Gemma, GLM, and Minimax formats. Hardware requirements. macOS 15+, Apple Silicon (M1+), 16 GB RAM minimum (64 GB+ recommended for larger models); vision-language models and common formats (Qwen, Llama, Mistral, Gemma, Minimax, GLM) are supported.

omlx.ai pricing Freemium

omlx.ai offers a free plan with paid upgrades available.

Verify on the official pricing page.

View plans

omlx.ai user reviews

Would you recommend omlx.ai?

Recommend this tool?

omlx.ai's key features

Native macOS menu bar app and web dashboard for model management, chat, and real-time metrics
Paged SSD key-value caching (safetensors) with a two-tier RAM/SSD LRU policy that persists cache blocks across requests and restarts
Continuous batching and a batch generator to increase throughput and concurrency
Multi-model serving for LLMs, VLMs, embeddings, and rerankers with LRU eviction, built-in model browsing, and Hugging Face/LM Studio cache support
OpenAI- and Anthropic-compatible API endpoints (/v1/chat/completions, /v1/messages) and compatibility with multiple model/tool formats (Claude Code, OpenClaw, Cursor; Qwen, Gemma, GLM, Minimax)

omlx.ai use cases

Run a private, low-latency coding assistant on your Mac using omlx to power in-IDE code completion, refactoring and debugging with OpenAI-compatible APIs, on-device inference and SSD-backed paged caching to support long contexts and near-instant time-to-first-token
Deploy multi-model customer support and knowledge-base bots locally with omlx's menu-bar app and web dashboard, leveraging continuous batching and two-tier LRU/SSD caching to scale throughput for many concurrent requests while keeping sensitive data on-device
Process and summarize large document sets, perform bulk content moderation or batch transcript generation using omlx's continuous batching and paged SSD cache to accelerate throughput and reduce latency for high-volume offline inference workflows