From the creator of Redis; run LLM locally with ds4

	From the creator of Redis; run LLM locally with ds4(dwarfstar.sh)
	344 points by fibo 11 days ago \| 98 comments
	tl;dr: DS4 (DwarfStar 4), from Redis creator Salvatore Sanfilippo (antirez), is a narrow C inference engine for running large open-weight MoE models like DeepSeek V4/V4.1, GLM 5.x, and Qwen3.8 locally on high-memory Mac, CUDA, and ROCm machines. It uses asymmetric 2-bit quantization on routed experts, SSD-resident KV cache that survives restarts, and ships a CLI, OpenAI/Anthropic-compatible server, and native coding agent. Reference benchmarks show ~34 tokens/sec generation on an M5 Max with 128 GB running V4 Flash Q2.
	HN Discussion: ↑Developers extending ds4 with bindings and forks for other languages and platforms ↑Users sharing positive hands-on experience running large models on high-end hardware •Questions about performance on lower-spec hardware and SSD streaming viability •Promotion of alternative or inspired inference engines for different hardware targets •Curiosity about the expertise required to build model-specific inference engines