Transformers Explained Visually(poloclub.github.io)
599 points by aray07 22 days ago | 87 comments
tl;dr: An interactive visualization tool (Transformer Explainer) runs a live GPT-2 small model in the browser to demonstrate how Transformers work, walking through embedding, multi-head self-attention, MLP layers, and output sampling. Built by Georgia Tech researchers using Svelte and D3.js with ONNX Runtime, it lets users input text, tweak temperature/top-k/top-p parameters, and inspect attention weights and intermediate computations in real time.
HN Discussion:
  • ↑Praises the visualization as well-done and effective for learning
  • ↑Adds technical insight about how attention heads dynamically construct networks
  • •Recommends alternative/complementary visualization resources for learning transformers
  • ~Critiques specific presentation choices like Q/K/V weights placement or 'safety' wording
  • ↓Warns the tool is outdated and misrepresents how modern models actually work