Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

	Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s(github.com)
	911 points by snehesht 9 days ago \| 412 comments
	tl;dr: Strata is a free, open-source tool that runs the 125B-parameter Qwen3.8-Flash-Next model locally on consumer GPUs (12GB+ VRAM) by distributing experts across GPU, RAM, and SSD, with speculative decoding for speed. Benchmarks show ~94 tokens/sec on an RTX 5070 (12GB) and ~60 tokens/sec on an RX 9070 XT, with an OpenAI-compatible API at localhost:8080 for integration with coding agents like Claude Code and Codex. Supports NVIDIA RTX 20-50 series and various AMD Radeon cards on Windows/Linux, with multi-GPU and vision capabilities.
	HN Discussion: ↑Confirmed working with impressive speeds on their consumer hardware setups ↓Quality suffers significantly compared to llama.cpp on vision benchmarks ↓Skeptical of aggressive quantization; prefers 4-bit quants on rented pro hardware ↓Hype feels overblown; speed is real but accuracy is underwhelming ↑Excited about progress toward running frontier-quality models locally