| AMD acquires Taalas to boost inference performance by etching models in silicon(theregister.com) | |
| 923 points by itvision 15 days ago | 693 comments | |
tl;dr: AMD has acquired Toronto-based startup Taalas, which etches AI model weights directly into silicon rather than storing them in HBM, creating what amounts to model-specific integrated circuits. Taalas' first test chip reportedly ran Llama 3.1 8B at ~17,000 tokens/sec (48x faster than Nvidia GPUs), and its upcoming HC2 chip targets 20B parameters per accelerator. The tradeoff is inflexibility—model changes require a chip re-spin (though only two metal layers)—making the tech best suited for stable, high-volume inference workloads from major model providers, likely paired with AMD's Instinct GPUs in disaggregated deployments. | |
HN Discussion:
| |