| Managing AI Coding Costs at Scale(databricks.com) | |
| 300 points by moonikakiss 14 days ago | 261 comments | |
tl;dr: Databricks outlines how large AI-adopters (including Stripe, Coinbase, Uber, Ramp) are containing runaway agentic coding costs through five main levers: chasing the "efficiency frontier" of cheaper-but-capable models, using meta-harnesses to preserve model flexibility, dynamic request/task routing to lowest-capable models, progressive spend visibility and gating (rather than hard budget cutoffs), and reducing token/context overhead via compaction and cache tuning. These techniques are converging around a new "AI Gateway" infrastructure pattern for centralized model proxying, budget enforcement, and observability—Databricks has open-sourced its own stack (Unity AI Gateway and Omnigent). | |
HN Discussion:
| |