We turned EmbeddingGemma 2 into a speech-to-text model(eg2.haku.sh)
417 points by amrrs 1 day ago | 94 comments
tl;dr: Developers repurposed Google's EmbeddingGemma 2 multimodal embedding model into an on-device speech-to-text system by training a small "reader" that maps audio slices to the nearest of ~229,000 English words in the shared embedding space. On a base M3 MacBook Air it transcribes an hour of audio in 12.6 seconds (288× real-time) using the Neural Engine, roughly twice as fast as IBM Granite Speech 5.0 Turbo. Word error rate is around 5%, versus 2–3% for leading STT models, which they hope to close with better reader training.
HN Discussion:
  • ↑Author shares background clarifying the experiment was about shared vector space, with speed as a bonus
  • ↑Shares related tips about running models fast on Apple Neural Engine via CoreML conversion
  • ↑Expresses amazement at how rapidly local transcription performance has improved
  • ~Points out existing alternatives like Parakeet v3 are already comparably fast on similar hardware
  • ~Questions the WER benchmark used and asks for methodological clarification