Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac(github.com)
901 points by gitpusher42 46 days ago | 336 comments
tl;dr: TurboFieldfare is a Swift/Metal runtime that runs Gemma 4 26B-A4B (a 26B-parameter MoE model with ~3.88B active per token) on Apple Silicon Macs using only ~2GB RAM by keeping shared weights and KV cache resident while streaming needed experts from SSD per token. Measured decode speeds are 5.1–6.3 tok/s on an 8GB M2 MacBook Air and 31–35 tok/s on a 24GB M5 Pro. It ships as a library, CLI, native Mac app, and loopback OpenAI-compatible server, requiring macOS 26 and ~14.3GB of storage for the model.
HN Discussion:
  • Praise for solving the impractical 'load entire model into memory' problem
  • Users sharing successful benchmarks on their own hardware confirming the claims
  • ~Questioning how this differs from existing solutions like llama.cpp's mmap
  • Suggesting technical improvements like MTP-based speculative expert prefetching
  • Skepticism about AI-generated marketing language and unexplained performance spread