Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac(github.com)
843 points by gitpusher42 22 hours ago | 299 comments
tl;dr: TurboFieldfare is a Swift/Metal runtime that runs Gemma 4 26B-A4B (a MoE model with ~3.88B active params) in about 2 GB of RAM on Apple Silicon Macs, including 8 GB models, by keeping shared weights and KV cache resident while streaming individual experts from SSD per token. Measured decode speeds range from 5-6 tok/s on an M2 Air to 31-35 tok/s on an M5 Pro. It ships with a native Mac app, CLI, and a loopback OpenAI-compatible server, and requires macOS 26 with Metal 4.
HN Discussion:
  • Excitement about breaking the assumption that entire models must fit in memory
  • Sharing hands-on performance results confirming the project's claims on various Macs
  • ~Questioning how this differs from existing mmap-based approaches like llama.cpp
  • Suggesting technical improvements like MTP-based speculative expert prefetching
  • Questioning the large performance spread between M2 Air and M5 Pro benchmarks