| Performance per dollar is getting faster and cheaper(wafer.ai) | |
| 354 points by latchkey 49 days ago | 135 comments | |
tl;dr: Wafer benchmarked GLM-5.2 on AMD's MI355X, achieving 213 tok/s single-stream and 2626 tok/s/node aggregate throughput—about 80% of B200 performance at less than half the cost. Getting there required MXFP4 quantization via AMD Quark, switching to sglang, patching two ROCm bugs to enable speculative decoding, and manually tuning the MoE kernel selection to bypass a slow fallback path. The takeaway: closing the gap with NVIDIA on AMD hardware is increasingly a matter of day-0 support rather than fundamental software limitations, and no custom kernels were needed this round. | |
HN Discussion:
| |