What a large language model does with a sentence, from the text it reads to the number it predicts, taken from the original papers and checked by script.
| # | Chapter | Status | Main source |
|---|---|---|---|
| 1 | Tokens: the unit a model actually reads | draft | Sennrich, Haddow, Birch 2015 (byte pair encoding) |
| 2 | Embeddings: words as points | planned | Mikolov et al. 2013 |
| 3 | Attention: who looks at whom | planned | Vaswani et al. 2017, §3.2 |
| 4 | The transformer block | planned | Vaswani et al. 2017, §3 and Table 1 |
| 5 | Training: loss, gradients and scale | planned | Books XVII and XVIII; scaling-law papers (unread) |
| 6 | What is not understood | planned | to be chosen after the earlier chapters |
Read for this draft: the byte-pair-encoding paper (Sennrich et al., arXiv 1508.07909, §3.2) and the formulas, hyperparameters, complexity table and results of the transformer paper (Vaswani et al., arXiv 1706.03762, version 7, §§3–6). Not read: the word-embedding paper beyond its abstract, any scaling-law or alignment paper, and the neural-network chapters of a current textbook. The Jurafsky–Martin text on the desk is the 2007 edition, which predates neural language models, so it cannot support chapters 2 to 6. Those chapters are marked planned, not drafted.
Book XVII (mathematics of machine learning) · Book XVIII (chain rule and backpropagation) · Book XXIV (linear algebra) · Book XIV (language of mathematics, parsing machines) · Book XXV (languages side by side, which will supply the multilingual tokenization test in chapter 1).
© 2026 Pablo Nogueira Grossi / G6 LLC · Newark, New Jersey · CC BY-NC-ND 4.0 · Independent teaching text; not peer-reviewed.