Pick anything extreme on one measurement and measure it again: on average it comes back less extreme. Nothing has to happen in between. Regression to the mean is a property of imperfect correlation, not a force, and most of the mistakes made with it come from treating the drift back as an effect.
Francis Galton collected the heights of 928 adult children born to 205 pairs of parents (p. 254). He averaged each pair into a “mid-parent”, scaling women's heights by 1.08 so that they could be averaged with men's. He then measured everyone as a deviation from the population's “level of mediocrity” of about 68¼ inches. On p. 252 he states what he found in one sentence:
Parents three inches above average have children, on average, two inches above. Parents three inches below have children two inches below. Both groups move toward the middle. Galton called it regression toward mediocrity, and the word regression has named the whole statistical method ever since. The textbook this volume reads, Evans and Rosenthal, uses the method in §10.3 but never names the phenomenon. The script searches for four names for it and finds none. That is why this chapter is written from Galton.
If every generation were pulled two-thirds of the way to the middle, heights would bunch together within a few generations. They do not, and Galton saw the objection. His answer (p. 256) is a balance: children of similar mid-parents also scatter around their family's centre. The pull inward and the scatter outward cancel, and he writes the balance as an equation connecting three spreads. His own numbers satisfy it:
| quantity | value | where Galton prints it |
|---|---|---|
| regression, v | 2/3 | p. 252 |
| spread of mid-parents, p | 1.22 in | p. 263 |
| spread within a family, f | 1.5 in | p. 252 |
| spread of the population, P — computed | 1.706 in | p. 252 prints 1.7 |
The four numbers agree COMPUTED. Galton's image for it is mechanical: “It acts like a spring against a weight”. The distribution of heights holds steady because regression and scatter balance, and each is needed to explain the other.
The next thing Galton says is the most often forgotten (p. 253): “The converse of this law is very far from being its numerical opposite.” If children deviate two-thirds as much as their mid-parents, you might expect mid-parents to deviate three-halves as much as their children. They do not. Reading his table by columns instead of rows, the mid-parent of an exceptional person deviates only one-third as much as that person (p. 254).
Both directions regress, and the script derives Galton's one-third from the spreads above. The slope in the reverse direction is the forward slope scaled by the ratio of the two variances:
His “four and a half times smaller” is (3/2) / (1/3); with the measured spreads it comes out at 4.40 COMPUTED. The practical reading is his own: “an exceptional man is the somewhat exceptional son of rather mediocre parents” far more often than the average son of exceptional ones. There are simply many more mediocre parents to be born to.
Galton explained regression by ancestry: the child inherits partly from a long line whose average is the average of the race. That explanation turned out to be unnecessary. The effect appears whenever two measurements share part of their variation and not all of it, whether or not one causes the other.
The script builds the plainest possible case. It takes 100,000 things, each with a fixed true value, and measures each twice with independent noise as large as the true spread. It then selects the top 5% on the first measurement and looks at the same things on the second.
| group, chosen on measurement 1 | mean on measurement 1 | mean on measurement 2 | ratio |
|---|---|---|---|
| top 5% | +2.91 | +1.44 | 0.50 |
| bottom 5% | −2.93 | −1.46 | 0.50 |
Nothing happened to any of them between the two measurements MODEL. The top group “declined” by half and the bottom group “improved” by half, exactly the share of the first measurement that was noise. Put a treatment between the two measurements and give it to the bottom group, and it will appear to work. Give a warning to the top group and it will appear to work too. This is how regression becomes a finding: the drift back gets credited to whatever was done in between.
R19 in this repository's rules says that a number already in the ledger is settled, and that “if it looks wrong, say which row and why, then re-run that one thing.” That is the right economy, and this chapter adds one warning to it. A number is chosen for a re-run because it looked extreme, and a number chosen for being extreme will, on a noisy measurement, tend to come back less extreme. If the instrument has any noise in it — a sampled audit, a similarity score, a timing, an OCR count — the re-run will usually look like a correction, whether or not anything was wrong.
The fix is Chapter 2's: decide the comparison before the re-run. Re-run a few ordinary rows alongside the one that looked wrong, and compare how far the suspicious row moved with how far the ordinary rows moved. OPEN — no re-run in the ledger has been checked for this. Most of this corpus's instruments are deterministic scripts, where the second run equals the first and none of this applies. The warning is for the ones that are not.