← All experiments

Experiment

Write like a chatbot

One word at a time, guessing the next. Build it wrong first: always pick the most common next word, and watch it go round in a loop.

From the Short "How Does a Chatbot Actually Write?". ▶ Watch the Short

A chat window filling up with word blocks. Below it, a counting machine (later a small model) with arrows to the words it might pick next, each with a bar for how likely it is, or a spinning wheel with bigger slices for likelier words. the pile the word counter

Pick the most common word

Let's build one. Take a pile of text and count which word follows which. Then always pick the word that most often came next.

The pilethe cat sat on the mat. the cat sat on the rug. the dog sat on the cat. the cat saw the dog. the dog ran to the park.

Keep pressing Add the top word. What goes wrong?

Looks back at1 word
Top guesscat
Words written0
Different next words5

What's going on

A chatbot writes one token at a time. Each step, its model gives every token it knows a score: how likely that token is to come next. One is chosen and added to the text, and the whole lot, your question still in there, goes round again.

The simple version counts which word follows which in a pile of text. Claude Shannon tried this kind of thing by hand in 1948. Always taking the most common next word easily falls into loops, because it only looks one word back. That's step 1.

Looking further back helps, but most long stretches of text never appear twice, so there's nothing to look up. There are too many combinations to count: with a list of 50,000 words there are 2.5 billion possible pairs, and about 6 × 1018 possible strings of four, most of them nonsense. So real chatbots read all the text so far, up to a limit called the context window, and use a model instead: a neural network with billions or more adjustable numbers, called parameters. Training adjusts those numbers, over and over, so its next-token guesses on huge amounts of text get more accurate.

A token is a chunk of text: often a whole common word, sometimes a piece of a longer or rarer word, or a punctuation mark. Each model has its own fixed list of tokens, commonly tens of thousands to a few hundred thousand, and splits words its own way.

To choose, chatbots usually pick at random, weighted by the scores, often after trimming the least likely tokens (settings called temperature and top-p). That's the wheel, and it's why the same question can get different answers. With it turned off, a chatbot takes the top token every time and gives the same reply to the same text, or very nearly. Replies can also differ for other reasons, such as different earlier messages.

Simplified: the pile is five short sentences, not huge amounts of text; when counting, a full stop counts as a word. The 50,000-word list is a rough guide that counts every arrangement, sensible or not. The "model" is a tiny hand-made table of scores for one question and eight answers, standing in for a real model's billions of numbers; it gives every other token a score of zero, where a real model gives each one some score, most of them tiny. Its 24 tokens and the split of "Sunlight" are examples: real splits vary by model. Equal slices is a wrong version, there to show why the weights matter.