Streaming AI responses without a fragile interface
Make long answers feel responsive while handling partial output, disconnects and errors honestly.

Why the first visible word matters
When a model takes several seconds to answer, an empty screen leaves people wondering whether anything happened. Streaming can show output as it arrives. It improves perceived responsiveness, but it does not make the model finish sooner, and support depends on the selected model and endpoint.
Showing the first words quickly tells the user that the request is underway, but the complete answer may still take time. Measure time from request to first output, total completion and interruptions separately. A short answer may benefit little from streaming; a long one can feel less uncertain. Measure the effect in your own workflow rather than assuming a universal improvement.
Treat the answer as a sequence
Read and parse events according to the endpoint documentation, append text in order, and keep the interface in a clear generating state. A network chunk is not necessarily a complete event or a complete word; buffer partial data before parsing. Let the user stop generation and distinguish a cancelled response from a finished one.
When an endpoint uses server-sent events, messages follow a protocol format, while a network read may split one message or combine several. Buffer data until a complete event can be parsed, and account for empty lines, heartbeats and any documented completion marker. Do not parse every network chunk as a complete JSON object. Follow the format documented for the actual endpoint.
A response arrives as a sequence of events and may be interrupted.Plan for a connection that ends early
An interrupted stream may contain useful text but no final usage information or completion marker. Preserve the visible partial answer and explain that it stopped; do not present it as complete. Retrying can create a new response and may incur additional usage, so make the action explicit and avoid automatically duplicating expensive work.
Show a clear generating state and a reachable stop control. Cancellation, a broken connection and successful completion deserve different messages. Keep already displayed text when a stream fails, but label it as potentially incomplete. For screen-reader users, avoid an interruptive announcement for every character; update in sensible chunks while keeping controls keyboard accessible.
Some streams report final state or usage only at the end; an interruption can leave those details missing. Record “text shown” separately from “task confirmed complete.” If the user regenerates an answer, explain that a new request and additional usage may result, and keep fragments from the old and new attempts separate.
Test more than the happy path
Try a slow connection, an empty first event, a disconnect in the middle, cancellation and a provider error. Check keyboard and screen-reader behavior as text updates. Measure time to first output separately from total completion time, and consult the model documentation before relying on streaming-specific fields.
If the server produces content but the browser sees nothing for a long time, inspect the application server, reverse proxy and network intermediaries for response buffering and connection timeouts. Compare when the server starts sending with when the browser starts receiving. Do not disable unrelated security controls merely to make text appear sooner.
Follow the stream from connection to completion
Streaming APIs return a sequence of events, but one network read is not necessarily one complete event. Data can be split across reads or several events can arrive together. Parse the selected protocol correctly and distinguish content deltas, status, errors and the finish signal.
Show new text as it arrives while keeping a clear “generating” state until completion is confirmed. Test the real deployed path as well as the client: an intermediary that buffers the response can make a stream appear all at once. Keep the page readable while content updates, and offer an accessible way to stop.
Handle interruptions before retrying
Distinguish no content, partial content and a confirmed finish. If the connection drops, preserve the partial response and tell the user it is incomplete. Don't silently replace it with a new answer or label it finished.
A retry can start a new generation and incur additional usage. If a tool was involved, its execution state may also be uncertain. Only resume automatically when the API and your application support a safe recovery path. User cancellation and network failure are different events; a closed browser connection does not by itself prove that remote work stopped. Preserve drafts and avoid appending duplicate text.
Measure the experience, not just the demo
Measure time to first content separately from time to a complete answer. Also track interruptions, cancellations and errors; a fast first word does not make a good experience if the page jumps, cannot be stopped or never reports completion.
Run repeatable checks with short and long answers, a slow connection, a dropped connection, navigation away and tool calls. Verify that text isn't duplicated and that the end state is clear. Use non-sensitive examples and check that logs do not inadvertently contain keys or full prompts. Evaluate the experience on slower devices and with keyboard navigation.
A useful interruption test
Start a response, then deliberately close the connection after the first few events in a test environment. The page should retain the text already shown, stop the loading indicator and offer a clear next action. Check that the parser also handles one event split across multiple network chunks. Do not infer a successful completion or final usage when the endpoint has not sent it.
A phone may change networks, lock its screen or enter the background. In a test environment, switch networks or terminate the connection and check that the interface stops waiting, retains visible text and lets the user act again. Reconnection does not automatically mean the original stream can resume; build that behavior only if the endpoint documents it.