Controlling spend on a fine-tuning platform

The important property of a spend limit is when it is checked. Estimating a job’s cost and reserving it against available headroom before leasing compute means a job that would breach the ceiling never starts. Measuring spend and alerting past a threshold tells you about an overrun once the money is gone.

The reservation must also count open reservations, not just settled spend. Otherwise several jobs authorised simultaneously each pass the check individually and collectively exceed the limit.

Training is bounded — a run ends. Serving is open-ended, and an always-warm endpoint from an experiment three weeks ago bills continuously whether or not anything calls it.

Audit warm endpoints on a schedule. For each, ask whether a human waits on it; if not, it should be scaling to zero.

Deploying an 8B for a task a 1.7B handles identically is a decision you pay for on every request forever. The gate makes finding the smallest sufficient model a measurement rather than an argument: train at the smallest plausible size, gate it, and move up only on a fail.

This is the highest-leverage cost control available and it is almost always skipped, because the larger model is the safer-feeling default.

A per-run cost figure alongside the forecast is what calibrates a team’s intuition. Consistently over-forecasting means your estimates are pessimistic; consistently under means something is running longer than expected and is worth investigating.

Without that feedback loop, cost stays an abstraction until an invoice makes it concrete.