Tooling · YAML
LiteLLM
How do I use LiteLLM with Onrup?
Add your deployment as a model entry in the LiteLLM config with an openai/ prefix and your base URL. It then participates in routing, fallbacks and budget tracking alongside every other provider.
A proxy and library that presents many model providers behind one OpenAI-compatible interface.
Minimal working example
model_list:
- model_name: support-triage
litellm_params:
model: openai/default
api_base: https://serve.onrup.com/<tenant>/<deployment>/v1
api_key: os.environ/ONRUP_API_KEYThe thing that catches people out
The openai/ prefix is required. Without it LiteLLM tries to infer the provider from the model name, fails, and reports an error that says nothing about the prefix.
Worth knowing
- Useful for A/B routing between a fine-tune and the frontier model it is replacing.
- Its budget tracking is per-proxy and sits alongside, not instead of, the spend limit enforced before compute is leased.
- Fallback chains work well with scale-to-zero: fall back on a cold start rather than making the user wait.
Other tooling
n8n
A workflow automation tool with a visual editor, self-hostable, with language-model nodes built in.
Open WebUI
A self-hosted chat interface that can connect to any OpenAI-compatible backend.
Continue
An open-source coding assistant extension for VS Code and JetBrains that can use any compatible model.
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.