tl;dr: DeepSeek has open-sourced DeepInfer, the inference framework powering its production services, built on vLLM with influences from SGLang and TensorRT-LLM. It supports DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, featuring optimized kernels (FlashMLA, DeepGEMM), speculative decoding with confidence-scheduled verification, batch-invariant and deterministic inference, async scheduling, and structured outputs. The release covers offline inference only—a KV cache system, prefill/decode disaggregation, and load balancing are required but not included for a full serving stack.