Random doesn't look random

Your brain runs a pattern detector that never sleeps and never asks permission. It is the reason you can read a friend's handwriting and hear your name across a loud room. It is also the reason you will look at a scatter of dots and feel, with real conviction, that something arranged them.

The trouble is that randomness is lumpy. Genuine chance produces clusters, streaks, runs and coincidences at a rate that feels wrong to almost everybody, including people who have studied it for years. Meanwhile the thing we picture when we say "random" is something closer to evenly spread, well shuffled, nicely mixed. That picture is not random at all. It is the signature of a process that is actively avoiding repetition, which is to say, a designed one.

Below are six demonstrations. Each one runs live in your browser, on numbers generated at the moment you press the button. Nothing is pre-baked and there is no trick in the code. The only trick is the one your visual cortex is going to play on you.

Two fields of three hundred dots. One was placed by a random number generator. The other was placed by an algorithm that deliberately keeps every dot away from its neighbours. Pick the random one.

Most people choose the even field, and the reason is that the random one contains clumps and voids that look deliberate. They are not. Each dot in the random field was placed with no knowledge of any other dot, which is exactly why some of them landed on top of each other. Independence does not mean tidiness. It means no coordination, and no coordination means collisions.

There is a well documented case of this. During 1944 south London was hit by V-1 flying bombs, and it became common knowledge in the city that the strikes were clustering, that certain districts were being singled out, and that the pattern implied German precision or informers on the ground. In 1946 an actuary at the Prudential named R. D. Clarke divided the affected area into 576 squares of a quarter square kilometre each and counted the 537 hits. The distribution matched a Poisson process, which is the mathematical description of things landing completely at random. The clusters were real. The targeting was not.

Tap heads and tails sixty times. Do not use a coin, a die or a random number generator. Produce a sequence from your own head that you think would pass as the record of a real fair coin. When you are done, this page will compare what you wrote against twenty thousand simulated runs of a genuine coin.

Two things give people away almost every time. The first is the switch rate. A fair coin changes its mind on about half of all consecutive flips. People inventing a sequence switch closer to six times in ten, because staying on heads feels like it is breaking the rules, and every extra head feels like it owes a tail back.

The second is the longest run. In sixty flips of a real coin you should expect to see five or six of the same face in a row, and a run of seven or eight is entirely ordinary. Almost nobody writes that down voluntarily. A hand written run of six looks so much like a mistake that people delete it before they finish typing.

Here is a strip of five hundred fair coin flips. The longest run is marked. Redraw it as many times as you like and you will never get a strip without a conspicuous streak somewhere in it.

In 1985 Gilovich, Vallone and Tversky went through the shooting records of the Philadelphia 76ers looking for the hot hand, the widely held belief that a player who has just scored is more likely to score again. They found nothing. A hit after a hit was no more likely than a hit after a miss. The finding became one of the most cited examples of humans inventing structure inside noise, and it stayed that way for thirty years.

Then in 2015 Joshua Miller and Adam Sanjurjo noticed something about the method. If you take a finite sequence of coin flips, find every flip that followed a heads, and average the proportion of those that were themselves heads, you do not get a half. You get less than a half. The bias is a property of the counting procedure, not of the coin. It is small, it is completely counterintuitive, and it is large enough to have hidden a real hot hand effect in the original data.

Sit with that for a moment. Three of the most careful researchers in the field built a landmark result on a measurement that quietly tilts downwards, and the error survived peer review, thirty years of citation and a place in every undergraduate reading list. The lesson is not that the original team were careless. It is that intuition about randomness fails even when you are actively hunting for the ways intuition about randomness fails.

Below is a county. Five thousand households, spread evenly, and one illness that strikes nine homes in a thousand. Every household carries exactly the same risk. There is no pollution, no transmitter mast, no contaminated water and no cause of any kind beyond chance. Press the button and watch the local paper write itself.

The circle is drawn once the cases are on the map, which is what makes it dishonest. Anywhere you look, given enough places to look, something is denser than average. Pick the densest spot afterwards and you can always produce a number that sounds alarming, because you chose where to point the microscope by looking at where the answer already was. Statisticians call this the Texas sharpshooter, after the man who fires at a barn and then paints the target around the tightest group of holes.

This is not an abstract concern. American state health departments spent decades investigating reported cancer clusters, thousands of them, and confirmed a specific local environmental cause in a vanishingly small number of cases. Atul Gawande wrote about the pattern in 1999. The investigations were expensive, they were slow, and the families who raised the alarm were not being irrational. They were doing exactly what a working pattern detector does. The problem is that a working pattern detector, pointed at a country, will find hot spots in a country where nothing at all is happening.

Take a large collection of real world numbers spanning several orders of magnitude, and look only at the leading digit. You might expect each digit from one to nine to turn up about eleven per cent of the time. Instead about thirty per cent of them begin with a one, and barely five per cent begin with a nine. Choose a source and see.

The effect falls out of scale invariance. If a quantity grows by multiplication rather than addition, it spends longer passing through the stretch between one and two than it does crossing from nine to ten, because that first stretch is a doubling and the last one is an eleven per cent nudge. Anything that compounds inherits the pattern, which covers populations, river lengths, share prices, invoice totals and street addresses. Notice that the uniform source in the list above does not follow the curve at all. The law needs data that spans orders of magnitude, and it is misapplied constantly by people who skip that part.

Forensic accountants use this. Numbers that a human invented to fill a gap in a ledger tend to be spread far too evenly across the leading digits, because the inventor is doing the same thing you did with the coin flips: producing what randomness feels like rather than what it is. It is a screening tool rather than proof, and it has been stretched well past its evidential weight in some high profile election disputes, but as a first pass over a set of books it is genuinely useful.

The classic. Drag the slider and watch the probability that at least two people in the room were born on the same day of the year.

Twenty three people gets you past even odds, which feels absurd until you count the pairs rather than the people. Twenty three people form two hundred and fifty three pairs, and every one of those pairs is a fresh chance to match. The intuition fails because we instinctively ask how likely someone is to share our own birthday, which is a question about twenty two pairs and has a completely different answer.

Diaconis and Mosteller gave this its general form. If something has a one in a million chance of happening to a given person on a given day, then in a country of sixty eight million people it happens to sixty eight people every day, roughly twenty five thousand times a year. Every one of those people will experience it as a miracle, tell their friends, and be entirely correct that it was staggeringly unlikely. It was also completely inevitable that it happened to somebody.