"70B", "8B", "405B": model names are full of parameter counts, and those numbers decide what hardware you need, how fast responses come back and what you pay to run the model. They sound abstract until you see what a single parameter actually does. It turns out to be ordinary multiplication and addition, repeated an enormous number of times.

A neuron is a small calculation

A neuron in a language model is a tiny, fixed recipe:

  1. Take a list of input numbers.
  2. Multiply each input by its own weight.
  3. Add the results together, then add one more number called the bias.
  4. Pass the total through an activation function, a simple rule that decides what comes out.

That's it. No magic, no memory, just arithmetic. The weights and the bias are the neuron's parameters. They are the numbers that training adjusts (Article 6), and they are what a model file mostly consists of.

The name comes from a loose analogy with brain cells, which receive signals and "fire" when the signals are strong enough. I'm simplifying the biology away entirely here. An artificial neuron is a calculation, not a model of a real cell.

A common activation function is ReLU, short for rectified linear unit. Its rule is: if the total is positive, pass it through unchanged. If it's negative, output 0. So ReLU(0.6) = 0.6 and ReLU(-0.2) = 0. Many modern LLMs use smoother variants of this idea, but the effect is similar: weak or negative signals get suppressed, strong ones pass through.

A worked example: a neuron that notices uppercase emails

Let's build a neuron that reacts to text containing an email address with capital letters. Our neuron gets three inputs, each either 1 (yes) or 0 (no):

  • x1: does the text contain an @?
  • x2: does it contain a capital letter?
  • x3: is it about colours or styling?

We give it these weights and bias. In a real model these values would come from training. Here I've picked them by hand so the example is easy to follow.

  • weight for @: 0.5
  • weight for capital letter: 0.8
  • weight for styling: -0.6
  • bias: -0.7

The diagram shows this neuron working on the text Maria@Example.com.

@: 1 capital: 1 colour: 0 ×0.5 ×0.8 ×-0.6 Σ bias -0.7 1.3 - 0.7 = 0.6 ReLU: 0.6

Figure 4.1: One neuron. Each input is multiplied by its weight, the results and the bias are added, and ReLU decides the output.

Now the arithmetic.

Case 1: Maria@Example.com. Inputs are x1 = 1, x2 = 1, x3 = 0.

  • Weighted inputs: (1 × 0.5) + (1 × 0.8) + (0 × -0.6) = 0.5 + 0.8 + 0 = 1.3
  • Add the bias: 1.3 + (-0.7) = 0.6
  • ReLU: 0.6 is positive, so the output is 0.6. The neuron "fires".

Case 2: maria@example.com. Inputs are x1 = 1, x2 = 0, x3 = 0.

  • Weighted inputs: 0.5 + 0 + 0 = 0.5
  • Add the bias: 0.5 + (-0.7) = -0.2
  • ReLU: -0.2 is negative, so the output is 0. The neuron stays quiet.

Notice what the bias did. Without it, any text with an @ would have produced a positive output. The bias sets a threshold, so the neuron only fires when there is enough evidence.

One honest caveat. In a real model, neurons almost never have a clean, nameable job like "detect uppercase emails". Each concept is spread across many neurons, and each neuron takes part in many concepts. The calculation is exactly the one above, but the meaning is distributed.

From one neuron to a layer

One neuron is not much use. A layer is many neurons that all read the same inputs, each with its own weights. If a layer reads a vector of 4,096 numbers (an embedding from Article 3, for example) and has 4,096 neurons, then:

  • each neuron has 4,096 weights plus 1 bias
  • weights: 4,096 × 4,096 = 16,777,216
  • biases: 4,096
  • total: about 16.8 million parameters, for one layer

Computing a whole layer at once is a matrix multiplication: a big grid of weights multiplied by the input vector. That single operation is what GPUs are built to do fast, which is the subject of Article 9.

Layers are stacked. The output of one layer becomes the input of the next. An LLM has dozens of these stages, each containing several layers, and each refining the representation a little more.

The diagram shows how vectors flow through stacked layers to produce the next-token scores.

flowchart TD
    A["Token vectors in"]:::io --> B["Layer 1"]:::model
    B --> C["Layer 2"]:::model
    C --> D["Dozens more layers"]:::model
    D --> E["Final layer"]:::model
    E --> F["Next-token scores"]:::io
    classDef model fill:#2a2140,stroke:#a78bfa,color:#ece5ff
    classDef infra fill:#12302d,stroke:#2dd4bf,color:#d5fbf5
    classDef io fill:#1f1d1a,stroke:#8a8275,color:#ece5d8
    classDef warn fill:#3a1d18,stroke:#fb7a5e,color:#ffe1d9

Figure 4.2: Layers are stacked. Each one takes vectors in, applies its weights, and passes new vectors to the next.

Article 5 explains what is special about the layers in a transformer. For now, the key point is that every one of them is built from the same multiply, add and activate steps.

What "70B parameters" means

A 70B model has about 70,000,000,000 parameters: weights and biases spread across its embedding table and all of its layers. Every one of them is a number that has to be stored somewhere and used in calculations.

Memory. Each parameter is stored in a fixed number of bits. A common format for running models is 16-bit, which takes 2 bytes per number.

  • 70,000,000,000 parameters × 2 bytes = 140,000,000,000 bytes
  • That is 140 GB, just to hold the weights.

At 32-bit precision (4 bytes each) it would be 280 GB. Article 12 covers quantization, which stores weights in fewer bits, such as 8-bit (1 byte each, 70 GB) or 4-bit (half a byte each, 35 GB), trading a little quality for a lot less memory.

The chart compares the memory needed for 70 billion weights at different precisions with the memory of one 80 GB GPU.

16-bit 140 GB 8-bit 70 GB 4-bit 35 GB one 80 GB GPU

Figure 4.3: The weights of a 70B model at 16-bit do not fit on a single 80 GB GPU. At 8-bit or 4-bit they do, before counting any working memory.

The 80 GB line is a real reference point: common data-centre GPUs, such as NVIDIA's H100, come with 80 GB of memory. And the weights are not the only thing in memory. Running the model also needs working space, especially the KV cache that Article 8 explains, and that grows with the length of the text.

Compute. A useful rough rule is that generating one token with a standard model takes about two arithmetic operations per parameter, one multiply and one add. For a 70B model that is roughly 2 × 70 billion = 140 billion operations for every single token. I'm simplifying, because the exact number depends on the architecture and the length of the input, and Article 10 covers mixture-of-experts models, which only use part of their parameters per token. But the rule explains why big models need big hardware.

Where this shows up in our bug-fix agent

Say our agent runs on a 70B open-weight model hosted on the team's own servers.

When the harness sends auth/login.py to the model, every token of that file passes through every layer. Every one of the 70 billion parameters takes part in processing each token. If the model reads 200 tokens of code, that's roughly 200 × 140 billion = 28 trillion operations before it writes a single character of the fix. That is why model choice is an infrastructure decision, not just a quality one.

It also explains the hardware plan. At 16-bit, the weights alone need 140 GB, so the team needs at least two 80 GB GPUs, or a quantized version of the model on one. Articles 9 and 12 work through that sizing properly.

And at the level of a single neuron, the bug is visible too. Some combination of neurons, spread across many layers, responds differently to Maria@Example.com and maria@example.com. Those learned patterns are part of how the model connects the ticket to the missing .lower() call.

Common misconceptions

"More parameters always means a better model." Parameter count is capacity, not quality. Training data, training method and later tuning matter just as much. Smaller, newer models regularly match or beat larger, older ones on many tasks.

"The weights store the training data, like a compressed database." Models can memorise some passages, especially ones repeated many times in training. But mostly the weights store patterns, not copies. There is no way to look up a specific document inside them.

"Each neuron has a specific job you can read off." A few neurons in some models do seem to track recognisable things, but most meaning is spread across many neurons at once. Researchers studying this, in a field called interpretability, need special techniques to find what the model represents.

Key takeaways

  • A neuron multiplies its inputs by weights, adds a bias and applies an activation function such as ReLU. It's ordinary arithmetic.
  • Weights and biases are the model's parameters. They are learned during training and make up almost all of a model file.
  • A layer is many neurons reading the same input, computed together as a matrix multiplication. Models stack dozens of stages of layers.
  • A 70B model has 70 billion parameters. At 16-bit that is 140 GB just for the weights, more than one 80 GB GPU can hold.
  • Generating each token takes roughly two operations per parameter, so bigger models need more compute for every token.

Glossary

  • Neuron: a small calculation that multiplies inputs by weights, adds a bias and applies an activation function.
  • Weight: a learned number that says how much one input matters to a neuron.
  • Bias: a learned number added to a neuron's total, which shifts its threshold for firing.
  • Activation function: the rule applied to a neuron's total to decide its output.
  • ReLU (rectified linear unit): an activation function that passes positive numbers through and turns negative numbers into 0.
  • Layer: a group of neurons that read the same inputs, computed together.
  • Matrix multiplication: the operation that computes a whole layer at once by multiplying a grid of weights by an input vector.
  • Precision: how many bits are used to store each number, such as 16-bit or 8-bit.
  • Quantization: storing weights with fewer bits to save memory. Article 12 covers it.

Next up

Stacked layers of neurons have existed for decades. What made modern LLMs possible was a particular design for those layers. In Article 5: The transformer and attention, we look at how each token gets to "look at" every other token, work through a tiny attention calculation, and see why what you put in the context matters so much.