Serving
Streaming
What is streaming?
Streaming returns output tokens as they are generated rather than waiting for the complete response. It does not make generation faster; it makes the wait feel much shorter by replacing one long silence with immediate, continuous feedback.
The perceptual difference is large. A five-second wait for a complete answer feels slow; the same answer streaming from half a second feels immediate, even though the last token arrives at the same moment.
It complicates anything that needs the whole response before acting — validation, moderation, structured parsing. Those either buffer, giving up the benefit, or operate incrementally.
Related terms
Start with the free tier
A magic link creates your account, your tenant and your first API key. No card until you ask for compute.