Series · five parts · reference model Qwen2.5-0.5B

Inside an LLM.

What happens between a prompt and the next token: embeddings, transformer layers, attention, the KV cache, logits and sampling. Every part uses Qwen2.5-0.5B as the reference model, so the shapes and counts stay consistent: 24 layers, d_model 896 and a vocabulary of 151,936.

Published

Part 1 of 5

How an LLM generates one token

One token from input to output: tokenization, embeddings, twenty-four transformer layers, the LM head, temperature and sampling. CPU work is blue, GPU work is green, and two 3-D benches show the tensor moving through the pipeline.

Part 2 of 5

Making the sentence stand up

Type a sentence and watch it stand up into a tensor: one row per token, through the ids, the embedding lookup and all 24 layers, until only the last row is left.

Coming soon

Parts 3–5 are drafted and under technical review.

part 3The decode phase Prefill versus decode, every tensor shape inside one layer, and what a step costs in FLOPs and bandwidth.
part 4Attention in detail Q, K and V, 14 query heads against 2 KV heads, RoPE, and what the cache holds byte by byte.
part 5The LM head, magnified 151,936 dot products, what a logit is, temperature as a division, top-k, top-p and the draw.