Drex 2.0 runs locally on Apple Silicon, with a 21.2 GB model and 36 GB memory requirement
Nace AI’s Drex 2.0 decision model can run locally on Apple Silicon through Kapitsa or Nace’s modified llama.cpp and Ollama engines; stock versions of llama.cpp and Ollama cannot serve its decision endpoint. The model file is 21.2 GB, and the setup requires a Mac with at least 36 GB of unified memory. In Nace’s separate test, the 4-bit version matched the full-precision model’s top answer on 96.97% of 2,710 questions.
Bottom line — The 4-bit Drex 2.0 scored 80.42% on a 2,710-question subset, against 80.51% for full precision.
Go deeper 5
-
Hugging Face lists median decision latency on an M5 Pro at 113 ms with Kapitsa, 248 ms with llama.cpp and 282 ms with Ollama.
-
The latency figures are local HTTP tests, with 100 requests per cell; Kapitsa’s repeat-read figure benefits from retaining a document’s state.
-
Hugging Face says Kapitsa and llama.cpp selected the same top answer on all 71 questions in a separate comparison.
-
The 4-bit build is a single 21.2 GB GGUF file; the Kapitsa pack takes 24.1 GB, so the download and converted model need additional storage.
-
Kapitsa is an Apple Silicon inference engine that also serves OpenAI- and Anthropic-compatible APIs, according to its GitHub project page.