DeepSeek V4 Flash on a Single AMD MI300X

	DeepSeek V4 Flash on a Single AMD MI300X(github.com)
	373 points by zhoutong 1 day ago \| 100 comments
	tl;dr: A production configuration for running DeepSeek-V4-Flash (304B params) on a single AMD MI300X, fitting the full 156.67 GiB model in HBM without quantization or offload, achieving 168.6 tok/s single-stream decode and 830 tok/s aggregate across 64 streams. The repo provides Docker Compose stack, SHA-pinned overlays, and AITER GEMM tuning tables that fix MI300X-specific issues including FNUZ vs OCP FP8 format mismatches, MXFP4 routing bitmatrix padding bugs, causal speculative verification, and CPU-KV synchronization gaps not yet in upstream vLLM.
	HN Discussion: ~MI300X hardware is impractical to buy individually; alternative cards like MI350P may work too ↓Missing prior art reference to DwarfStar which runs the same model in less memory ↑Related work on 2xMI300x confirms the value of MI300X's high HBM for this use case ↑The tradeoffs are favorable: full weights preserved, good speed, only reduced context window ~Performance still lags DeepSeek's H800 numbers significantly, suggesting more optimization is needed