Technical guide
Streaming makes an agent feel fast, and it is where the most confusing bugs we have shipped came from: replies that contained a second, unrelated answer; two answers woven together word by word; sentences fused across a tool call; a half-finished reply sent twice. Every one came from how LangChain and LangGraph route tokens to a stream. Here is how that routing works, and the rules we now follow.
When you call graph.stream(..., stream_mode="messages"), LangGraph attaches a callback handler to the run. Every chat model that starts within that run registers with the handler, and every token it generates is pushed to your stream. "Within that run" is decided by configuration inheritance: LangChain keeps the current run's config in a context variable, and any runnable invoked while it is set merges that config into its own - callbacks included.
That design is what makes streaming work with no plumbing: the agent's model call streams because it inherits the handler. It is also the source of the first bug, because a model you call yourself inside a tool inherits it too.
Suppose a tool describes an image by calling a vision model, or summarises a document with a cheaper model, or asks another agent a question. The tool calls model.invoke(...) with no config. LangChain merges in the ambient config anyway, the call registers with the stream handler, and its tokens arrive in your stream exactly like the agent's own. If your consumer appends every text chunk to the reply - most do - the user receives the tool's internal output as part of the answer.
It is easy to miss in testing because the leaked text is often plausible: a description of the image the user sent, a summary of the document they asked about. It only looks wrong when the agent then writes its own answer underneath.
Now let the agent call that tool twice in parallel - a customer sends three photos at once and each gets described. Two model calls stream into the same handler concurrently, and their tokens interleave. The user receives two descriptions shuffled together word by word, which reads as nonsense. We shipped this twice through two different tools before treating it as a rule rather than a bug.
A model called inside the graph is never part of the answer, so it must never reach the stream.
LangGraph's stream handler skips any chat model run tagged nostream - the value of TAG_NOSTREAM in langgraph.constants. Pass config={"tags": ["nostream"]} on the inner call and it never registers. Use the constant, not the literal, or pin the literal with a test: an upstream rename would otherwise bring the leak back silently.
Run the inner call outside the inherited context, so no parent callbacks are merged in at all. Copying the current context and resetting LangChain's child-config context variable before invoking is enough; a top-level caller is unaffected. This also keeps the inner call out of the parent's trace, which you may or may not want.
Each streamed chunk carries metadata: the graph node, the tags and the run's metadata. Drop chunks whose tags contain nostream and chunks the harness marks as internal - LangChain's summarisation middleware, for instance, marks its own calls with lc_source: "summarization", and without the filter the summary of a long conversation can appear in the chat.
We take the first two at every call site where a model is invoked inside the graph, and the third in the one place every stream passes through. Any one of them fixes the bug in isolation; all three mean a future call site that forgets one is still covered.
The messages mode yields text chunks, and nearly every consumer joins them verbatim. But an agent's reply is often several model calls separated by tool calls: some text, a tool call, more text. Joined verbatim, the last sentence before a tool call and the first sentence after it run together with no space. Insert a paragraph break whenever text resumes after a tool boundary. If your interface shows tool activity, record where each text segment falls relative to the tool calls, so the reply can be drawn in the order it happened rather than as one block above a list of tools.
Despite the name, stream_mode="messages" yields every message the graph produces, not only the assistant's tokens: tool calls, tool results and the human input can all appear. Check the chunk's type before treating its content as reply text, and treat a tool result as data. Better still, translate the raw stream once into typed events - text, tool call, tool result - and have every surface consume those, so the filtering lives in one place.
Model calls fail transiently, and retrying is right. Retrying a stream is right only until its first chunk has been delivered. After that, a retry replays the start of the answer to a user who has already seen it, and they receive half an answer followed by the whole answer. The rule is simple and absolute: track whether anything has been yielded, and once it has, surface the error instead of retrying.
Be precise about what to retry, too. The official OpenAI Python SDK, which many LangChain integrations sit on, retries timeouts, conflicts, rate limits and server errors (408, 409, 429 and 5xx) and connection failures - but only for the request it makes. A connection that drops while the response body is being read is outside that retry, and a streamed turn spends nearly all its time reading the body. The resulting exception may have an empty message: str(httpx.ReadError()) is an empty string, so an error reporter that prints only the message shows the user nothing at all. Classify transport errors by their type, not their text.
Retry at the client, around the single model call, not by re-running the graph: re-running a turn re-executes its tools and rewrites its checkpoints. Olano's managed model tier retries a small allowlist of transient failures this way, a few times with jittered back-off, and never once a stream has produced output.
The simplest server design runs the agent inside the HTTP response that streams it. Then the turn's lifetime is the socket's: a phone locking its screen or switching from wifi to mobile data closes the connection, the framework cancels the response, and the cancellation propagates into the graph mid-tool-call. Run the turn as its own task and make each HTTP stream a viewer of it, with a sequence number on every event so a returning client can resume from where it left off. Then make sure an explicit Stop button actually stops the task, since closing the socket no longer does.
LangChain propagates the current run's config, including LangGraph's stream callback handler, to any runnable invoked inside it. A model called from a tool therefore registers with the stream and its tokens arrive like the agent's own. Tag the inner call with "nostream" (TAG_NOSTREAM in langgraph.constants), invoke it with a detached config, or both.
Two model calls inside the graph, usually the same tool called in parallel, are streaming into the same handler at once and their tokens interleave. The fix is the same as for a single leaked call: keep models invoked inside tools out of the stream entirely.
Only if nothing has been delivered yet. Once the first chunk has reached the user, a retry repeats the start of the answer. Track whether the stream has yielded, retry only before that, and retry around the single model call rather than re-running the whole graph, which would re-execute tools.
The messages mode yields every message the graph produces, including tool calls, tool results and human input, not only assistant tokens. Check each chunk's message type before treating it as reply text, or convert the stream into typed text, tool-call and tool-result events in one place.
An agent's reply is often several model calls separated by tool calls, and joining the text chunks verbatim fuses the sentence before a tool call with the one after it. Insert a paragraph break whenever text resumes after a tool boundary.
The rest of the LangChain series, and where to go next.
Which layer of the LangChain stack does what inside an Olano agent, and what Olano adds on top.
Debug mode, LangSmith traces, stream modes, state history, loop limits and fake models.
Thread ids, lost memory after a restart, event-loop errors, Postgres setup, encryption and deletes.
Content blocks one provider writes and another rejects, missing context windows, and retries that should not happen.
What it takes for an agent's work to outlive the device that started it.
Checked 13 October 2026 against LangChain 1.4, langchain-core 1.6, LangGraph 1.2 and Deep Agents 0.7. Olano is an independent company that builds on LangChain's open-source libraries; it is not affiliated with or endorsed by LangChain, Inc. Library behaviour changes between releases, and the LangChain documentation is the primary source. If we have described something wrongly, write to support@olano.ai and we will correct this page.
Send us the symptom. We have met most of the shapes this bug takes on the LangChain stack, and Olano's channels already handle the rest.
