tl;dr: Local LLM inference diverges from reference implementations because every step—attention backends (FlashAttention vs Triton), KV-cache quantization, weight quantization schemes, and the specific CUDA/GEMM kernels selected—produces subtly different logits, causing top-token flips that compound over long contexts. Benchmarks on Qwen3.6-27B show INT4 KV-cache and NVFP4 weights hitting ~50% token divergence by 88k tokens and failing tool calls, while W8A16 INT8 actually outperforms official FP8. Takeaway: your setup's "dumbness" is largely a function of stack-specific numerical drift, not just the model or quant level.