Frameworks · Python

LlamaIndex

How do I use LlamaIndex with Onrup?

Configure the OpenAILike LLM with your deployment’s base URL. LlamaIndex then treats it as any other model for query engines, retrievers and response synthesisers.

A framework for connecting language models to your own data, with a focus on indexing and retrieval.

Minimal working example

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://serve.onrup.com/<tenant>/<deployment>/v1",
    api_key="onrup_sk_...",
    model="default",
    is_chat_model=True,
    temperature=0,
)

The thing that catches people out

Set is_chat_model to True. Left unset, LlamaIndex may send a completion-style request that a chat endpoint will reject, and the resulting error does not obviously point at this setting.

Worth knowing

Fine-tuning complements retrieval rather than replacing it: retrieval supplies the facts, the fine-tune supplies the behaviour.
Train on cases where the retrieved context does not support an answer, or the model will answer anyway.
Embedding models are a separate concern — this endpoint serves chat completions only.

Other frameworks

LangChain

A framework for composing language-model calls into chains, agents and retrieval pipelines.

Vercel AI SDK

A TypeScript toolkit for building streaming language-model interfaces in web applications.

Pydantic AI

A Python agent framework that uses Pydantic models to type and validate model output.

Start with the free tier

A magic link creates your account, your tenant and your first API key. No card until you ask for compute.

Get your API key Read the docs