Serving
Cooldown
What is cooldown?
Cooldown is how long an endpoint stays resident after its last request before releasing capacity. It is the dial that trades cost against the frequency of cold starts, and it should be set from the actual gaps in your traffic.
Too short and normal pauses in a conversation trigger a cold start, which is the worst of both worlds — you pay for the reload and the user waits for it.
Too long and it becomes an always-warm endpoint with extra steps. Look at the distribution of gaps between requests and set cooldown above the typical gap but below the quiet period.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.