Serving
Scale to zero
What is scale to zero?
Scale to zero means an endpoint releases all its compute when idle and costs nothing until the next request. The trade is a cold start on the first request after an idle period, while capacity is acquired and the model is loaded.
It suits intermittent traffic: internal tools, batch jobs, development environments and anything with pronounced quiet periods. For those, paying for a resident GPU overnight is pure waste.
It does not suit latency-sensitive interactive products. A user waiting through a cold start experiences it as the product being broken, however good the steady-state latency is.
A middle path is a cooldown long enough to span normal gaps in traffic, so the endpoint only cools down during genuinely quiet periods rather than between one request and the next.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.