Frameworks · TypeScript

Vercel AI SDK

How do I use Vercel AI SDK with Onrup?

Create an OpenAI-compatible provider with your deployment’s base URL and use it with streamText or generateText. Streaming to a React interface works exactly as it does with any other provider.

A TypeScript toolkit for building streaming language-model interfaces in web applications.

Minimal working example

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const onrup = createOpenAICompatible({
  name: "onrup",
  baseURL: "https://serve.onrup.com/<tenant>/<deployment>/v1",
  apiKey: process.env.ONRUP_API_KEY,
});

const result = streamText({
  model: onrup("default"),
  prompt: "Draft a reply to this complaint.",
});

The thing that catches people out

Use the openai-compatible provider rather than the OpenAI one. The dedicated OpenAI provider assumes endpoints and behaviours that only OpenAI implements, and the failures are confusing.

Worth knowing

Keep the key on the server. Route requests through an API route rather than calling from the browser.
A scale-to-zero endpoint can make the first request slow. Show a loading state rather than a timeout.
Streaming makes the cold start much less noticeable, because the first token ends the silence.

Other frameworks

LangChain

A framework for composing language-model calls into chains, agents and retrieval pipelines.

LlamaIndex

A framework for connecting language models to your own data, with a focus on indexing and retrieval.

Pydantic AI

A Python agent framework that uses Pydantic models to type and validate model output.

Start with the free tier

A magic link creates your account, your tenant and your first API key. No card until you ask for compute.

Get your API key Read the docs