Why AI Types Word by Word

Ask an AI chat for a long answer and watch closely. The words don't appear all at once. They flow — like someone typing very fast on the other side.

Is that a cute animation? A designer's trick?

No. It's the truth about how models work. And once you see it, you can't unsee it.

A model writes one piece at a time

A language model doesn't plan a whole paragraph and then hand it over. It produces one small piece of text (a token — roughly a word or part of a word), then the next, then the next. Each piece depends on everything before it.

So the answer exists piece by piece, whether you show it that way or not.

In the last article, our program waited for all the pieces, then printed them together. Ten seconds of blank screen, then a wall of text. That's what we're fixing.

The one-line change

Save this as stream.py in your agent-lab folder:

import ollama

messages = [
    {"role": "user", "content": "In 3 sentences, explain what an AI agent is."},
]

for chunk in ollama.chat(model="qwen2.5", messages=messages, stream=True):
    print(chunk.message.content, end="", flush=True)
print()

The change is stream=True. Instead of one big reply, chat now gives you a stream of small chunks — and the for loop prints each one the moment it arrives.

Two tiny details make it feel live:

• end="" — don't add a new line after every chunk; keep writing on the same line.
• flush=True — push it to the screen now, don't wait to collect more.

Run it with uv run stream.py. Your terminal now types like ChatGPT. Same model, same question — but it feels alive.

Same speed, different feeling

Here's the part that surprises people: streaming doesn't make the model faster. The full answer takes exactly as long as before.

What changes is when you see the first word.

Engineers call this time to first token, and it's one of the most important numbers in any AI product. People forgive a slow answer. They don't forgive a blank screen.

A fun way to feel it

Try this little experiment. Ask for something long — say, "Write a 200-word story about a robot who's afraid of the dark" — and run it twice: once with stream=True, once without.

Time how long you feel like you're waiting. The second version feels broken, even though the total time is identical.

That's the whole lesson: streaming is a user-experience decision, not a speed decision.

Where this shows up again

Right now we're streaming to a terminal. Later in the series we'll put our agent behind a web API, and the same idea comes back — this time streaming chunks over HTTP to a browser or another program. Same for loop at heart.

There's one honest catch we'll meet then too: streaming plain text is easy, but an agent also uses tools mid-answer. Mixing "I'm calling a tool" events with "here's some text" in one stream needs a bit more design. Good problem to have — it means your agent is doing things.

Three things to keep

1. Models generate text one token at a time — streaming just shows you that.
2. stream=True + print(..., end="", flush=True) is all it takes.
3. Streaming changes when you see the first word, not how long the answer takes.

What's next

So far our model can only talk. Next, we do the thing that starts turning it into an agent: we give it its first tool — and discover that the model never actually runs it.