Every AI product you have used, from a chat assistant to a coding agent, is built around one component: a large language model. Once you understand what that component actually does, a lot of confusing behaviour makes sense. Why it is brilliant one minute and confidently wrong the next, and why it needs tools to do anything in the real world.

The short version is surprisingly plain. An LLM reads some text and predicts what comes next. That is all it does. The rest of this series is about how that one trick is built, run, and turned into products.

The running example

Throughout this series we follow one scenario. An AI coding agent receives a bug ticket:

Login fails when the email has uppercase letters.

The agent finds the relevant code, fixes it, runs the tests, and opens a pull request. Every article shows a different part of how that works. Here is the code the agent will eventually find:

def login(email, password):
    user = db.find_user_by_email(email)   # emails are stored in lowercase
    if user is None:
        raise AuthError("Invalid email or password")
    return check_password(user, password)

A user who types Maria@Example.com gets rejected, because the database stores maria@example.com. The fix is one line. How a model that "only predicts text" ends up writing that line is what this article is about.

The one job: predict the next token

An LLM takes a sequence of text as input and outputs a guess about what comes next. Not one guess, in fact, but a score for every possible continuation.

The model does not work with whole words. It works with tokens, which are chunks of text. A token is often a whole common word like login, sometimes part of a word, sometimes punctuation like (. Article 2 covers tokens properly. For now, think of a token as roughly three quarters of an English word on average.

So the precise version of the job is this: given a sequence of tokens, produce a probability for every token in the model's vocabulary, saying how likely each one is to come next. A probability is a number between 0 and 1, and all of them together add up to 1.

Imagine the model has just read this:

email = email.strip().

What token comes next? The model might score the options like this.

The bars below show an illustrative probability for each candidate next token. These numbers are made up to show the idea. They are not measured from a real model.

email = email.strip().▮ lower 81% casefold 7% upper 4% replace 3% all others 5%

Figure 1.1: The model's output is a probability for every possible next token. Here "lower" is by far the most likely.

The model has not "decided" to fix the bug in a human sense. It has seen a huge amount of code where .strip(). is followed by lower, especially near the word email, so it gives that continuation a high probability.

From one token to a whole answer

Predicting one token is not very useful on its own. The trick is to do it in a loop:

  1. Feed the text so far into the model.
  2. Get probabilities for the next token.
  3. Pick one token, usually one of the most likely.
  4. Append it to the text.
  5. Go back to step 1 with the slightly longer text.

This is called autoregressive generation. Each new token is predicted from everything before it, including the tokens the model itself just produced. When a chat assistant "types" its answer word by word, you are watching this loop. It stops when the model produces a special end token or hits a length limit.

The diagram shows the generation loop that turns single predictions into a full response.

flowchart TD
    A["Text so far"]:::io --> B["LLM reads all tokens"]:::model
    B --> C["Probability for each token"]:::model
    C --> D["Pick one token"]:::model
    D --> E["Append to text"]:::io
    E -->|"repeat until done"| A
    classDef model fill:#2a2140,stroke:#a78bfa,color:#ece5ff
    classDef io fill:#1f1d1a,stroke:#8a8275,color:#ece5d8

Figure 1.2: Autoregressive generation. One prediction per pass, appended and fed back in, until the answer is complete.

How the model picks from the probabilities matters. Always taking the top token gives predictable output, while adding some randomness gives more varied output. That is what the "temperature" setting controls, covered in Article 8.

A worked example: prediction by counting

To build intuition, let's make the world's smallest language model. It is so simple that it is really a counting exercise, and I am simplifying heavily here. Real LLMs do not count phrases. But the core idea, turning past text into probabilities for what comes next, is the same.

Our "training data" is five lines from a CI log:

the test passed
the test failed
the build passed
the test passed
the build failed

Step 1: count what follows "the". The word test follows the 3 times. The word build follows it 2 times. That is 5 cases in total.

Step 2: turn counts into probabilities. Divide each count by the total:

  • P(test after the) = 3 ÷ 5 = 0.6
  • P(build after the) = 2 ÷ 5 = 0.4

The two numbers add up to 1.0, as probabilities must.

Step 3: go one level deeper. After the test, the word passed appears 2 times and failed once. So:

  • P(passed after the test) = 2 ÷ 3 ≈ 0.67
  • P(failed after the test) = 1 ÷ 3 ≈ 0.33

Step 4: generate. Start with the, pick the most likely next word (test, 0.6), then the most likely after the test (passed, 0.67). Our model has generated "the test passed".

This toy model has two big problems. It can only repeat phrases it has seen exactly, and it only looks at the last word or two. Ask it about the deploy and it has nothing to say.

A real LLM fixes both. Instead of a lookup table of counts, it has billions of adjustable numbers, called parameters or weights, that learn general patterns. It can handle sentences it has never seen, because it has learned that deploy, build and test behave similarly. And it looks at the entire input at once, which might be hundreds of pages, not just the last two words. Articles 3 to 5 explain how. The "large" in large language model refers to this scale. Meta, for example, has published that its Llama 3.1 family goes up to 405 billion parameters and was trained on over 15 trillion tokens of text.

Why predicting text produces useful behaviour

This is the part that surprises people. How does "guess the next token" turn into fixing bugs, writing contracts and answering questions?

The answer is that predicting text well is a very hard task, and getting good at it forces the model to learn a lot about the world the text describes.

Consider what a model needs in order to predict the next token accurately in different situations:

  • To continue The capital of France is, it needs to have absorbed the fact.
  • To continue a Python function, it needs to have absorbed Python's syntax and common patterns.
  • To continue a bug report followed by The fix is to, it needs to connect the symptom to the code.

During training (Article 6), the model sees trillions of tokens and is adjusted every time its prediction is wrong. The best way to be less wrong across all that text is to learn the underlying patterns: grammar, facts, the structure of code, common reasoning. Nobody programs these in. They emerge because they help with prediction.

A model trained this way is called a base model. It is good at continuing text but not at following instructions. Ask it a question and it might continue with three more questions, because that often follows a question on the internet. Turning it into a helpful assistant takes further training (Article 7), but the mechanism never changes. Even the most capable assistant still predicts one token at a time.

Where this shows up in our bug-fix agent

The agent looks like one smart system, but it is an LLM inside a regular program. That program, often called the agent harness, does everything the model cannot: reading files, running tests, calling the GitHub API.

The diagram shows where the LLM sits inside the bug-fix agent, and what it does and does not do.

flowchart TD
    T["Bug ticket"]:::io --> H["Agent harness"]:::infra
    H <-->|"prompt / text"| M["LLM"]:::model
    H <-->|"actions / results"| X["Tools: read, edit, test"]:::infra
    M -.- N["Outputs text only"]:::warn
    X --> PR["Pull request"]:::io
    classDef model fill:#2a2140,stroke:#a78bfa,color:#ece5ff
    classDef infra fill:#12302d,stroke:#2dd4bf,color:#d5fbf5
    classDef io fill:#1f1d1a,stroke:#8a8275,color:#ece5d8
    classDef warn fill:#3a1d18,stroke:#fb7a5e,color:#ffe1d9

Figure 1.3: The LLM never touches files or runs tests. It produces text, and the harness turns that text into actions.

Every step of the agent's work is next-token prediction:

  • Understanding the ticket. The harness puts the ticket text into the prompt. The model predicts a response that starts with something like "I'll search for the login function."
  • Choosing an action. The model generates text in a format the harness understands, for example a request to search the codebase for find_user_by_email. The harness runs the search and pastes the results back into the text. Article 16 explains this tool-use loop.
  • Writing the fix. With the login function in its input, the model generates the corrected line, token by token: email, =, email, .strip, ()., lower, (). Exactly how the text splits into tokens depends on the model, which is Article 2's topic.
  • Reacting to tests. The harness adds the test output to the text. If a test failed, the likely continuation is an explanation and another attempt.

The intelligence you see is next-token prediction, repeated thousands of times, plus a harness that turns text into actions.

Common misconceptions

"It looks up answers in a database." A plain LLM has no database and no search engine. What it knows is stored indirectly in its parameters, learned during training. This is why it can be wrong about facts and why it does not know about events after its training data was collected. Products often add search or document retrieval on top, which Article 14 covers.

"It's just autocomplete, so it can't be useful." The mechanism is autocomplete, but scale changes what it can do. Predicting the next token of a bug fix well requires modelling the code. Judge it by what it does, not by how simple the job sounds.

"It plans the whole answer, then writes it out." Tokens are produced one at a time, and once a token is generated it is not revised. That said, this is a simplification. Interpretability research has found evidence that models internally represent some information about words further ahead, such as planning a rhyme before reaching the end of a line. The output is still produced one token at a time.

Key takeaways

  • An LLM does one job: given a sequence of tokens, it gives a probability for every possible next token.
  • Full answers come from a loop that predicts one token, appends it, and repeats. This is autoregressive generation.
  • Getting good at predicting text forces the model to learn grammar, facts, code structure and patterns of reasoning, even though nobody programs those in.
  • The model only produces text. Agents work because a regular program around the model turns that text into actions and feeds the results back.
  • What a model knows comes from its training data, stored in its parameters, not from a database it can look things up in.

Glossary

  • LLM (large language model): a model trained on very large amounts of text to predict the next token. "Large" refers to the number of parameters and the amount of training data.
  • Token: a chunk of text the model works with. It can be a word, part of a word, or punctuation.
  • Vocabulary: the full set of tokens a model knows.
  • Probability: a number between 0 and 1 for how likely something is.
  • Next-token prediction: the model's core task of scoring every possible next token given the text so far.
  • Autoregressive generation: producing text one token at a time, with each new token predicted from all the text before it.
  • Parameters (weights): the adjustable numbers inside a model that are learned during training. Modern LLMs have billions of them.
  • Training: the process of adjusting a model's parameters so its predictions get better.
  • Base model: a model after its initial training on raw text, good at continuing text but not tuned to follow instructions.
  • Agent harness: the regular software around a model that sends it prompts, interprets its output, and runs actions such as reading files or running tests.

Next up

We kept saying "token" and promising to explain it. In Article 2: Tokens and tokenization, we look at how text is actually chopped up before the model sees it, why Maria@Example.com splits into more pieces than you might expect, and why code and non-English text cost more to process than plain English.