Frameworks · Python
LangChain
How do I use LangChain with Onrup?
Use the ChatOpenAI class with base_url pointed at your deployment. LangChain treats any OpenAI-compatible endpoint as an OpenAI model, so chains, agents and retrievers built against it work unchanged.
A framework for composing language-model calls into chains, agents and retrieval pipelines.
Minimal working example
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
base_url="https://serve.onrup.com/<tenant>/<deployment>/v1",
api_key="onrup_sk_...",
model="default",
temperature=0,
)
print(llm.invoke("Which queue should this ticket go to?").content)The thing that catches people out
LangChain sometimes assumes provider-specific features are present — token log probabilities and certain structured-output modes among them. If a chain depends on one, check it against the endpoint before assuming portability.
Worth knowing
- Use the OpenAI integration class rather than a generic one. It is better maintained and matches the wire format.
- A fine-tuned model usually lets you delete much of the prompt template, since the behaviour is now in the weights.
- Agent loops amplify per-call latency. Measure end to end rather than per call.
Other frameworks
LlamaIndex
A framework for connecting language models to your own data, with a focus on indexing and retrieval.
Vercel AI SDK
A TypeScript toolkit for building streaming language-model interfaces in web applications.
Pydantic AI
A Python agent framework that uses Pydantic models to type and validate model output.
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.