Docs
Deployments
How do I serve a fine-tuned model on Onrup?
Create a deployment from a finished run and you get an OpenAI-compatible endpoint. Set min_replicas to zero for scale-to-zero, or one and above for always-warm. That single setting determines both the cost profile and the worst-case latency.
One decision matters more than the rest
A replica floor of zero means the endpoint releases capacity when idle and costs nothing until the next request — at the price of a cold start on the first request after a quiet period. A floor of one or more means no cold starts and a continuous hourly charge.
The rule is simple: if a human is waiting for the response, keep it warm. If a batch job or an internal tool is calling it, let it scale to zero.
{
"run_id": "run_...",
"slug": "support-triage",
"gpu_class": "rtx-4090-24g",
"min_replicas": 0,
"cooldown_seconds": 600,
"max_queue_depth": 100,
"max_wait_seconds": 30
}Cooldown is the dial
Cooldown is how long the endpoint stays resident after its last request. Set it below the typical gap between requests and normal pauses trigger cold starts — the worst of both worlds, since you pay for the reload and the user waits for it. Set it far above and you have an always-warm endpoint with extra steps.
Look at the distribution of gaps in your traffic and set it above the typical gap and below the quiet period.
Queueing behaviour is explicit
A cold endpoint queues incoming requests while capacity is acquired. Both bounds on that queue are yours to set: a depth beyond which requests are refused with a retry hint, and a wait beyond which they time out.
Making these explicit is deliberate. An unbounded queue converts a capacity problem into a mysterious latency problem, and the client never finds out which it is.
Several adapters, one base
Adapters trained against the same base model and version can be co-resident on one deployment, so serving five task variants costs one base model plus a small amount per adapter rather than five full models.
The constraint is exact: same base, same version. A variant trained against a different base needs its own deployment.
Frequently asked questions
How long is a cold start?
It is dominated by loading weights, so it scales with model size — a small model is quick and a large one is not. Loading an adapter against an already-resident base is much faster than loading a whole model.
Can I point my existing OpenAI client at it?
Yes. Change the base URL and the key. Chat completions, streaming and tool calling all use the same shapes.
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.