Two Standard Recompressions

Search engines gave us top web page results. For every key phrase, a top hit. No use going down the list, the answer’s surely in the first link.

World knowledge became compressed. All of a sudden, there is no doubt. The standard answer created a groove, and all similar answers tended to fall into the new groove. World Wide Standards.

LLMs gave compress knowledge even more. Feeding on the compression that search engines encouraged, LLMs hurried to standardize all of the answers. No need for effort now, because the full answer, adapted to you no less, is a conversation away.

If the world’s answers become compressed into new standards, if the grooves of the standards become deeper, there’s going to be a new kind of consequence.

This double compression will have an effect on creativity. It will soon leave a lot of room for new kinds of creativity. You might even have started doing something about it already.

—

In the world of statistics, the distribution of a random process ends up looking like a curve. A normal curve is symmetrical, a gaussian distribution, the bell curve. Other distributions are elongated in one direction, forming a spike and then a long tail, like the lognormal.

LLMs took a trick from the search engines. They use a web crawler too, reading and following all the links they can find, and they amass a corpus, on which they train. When an idea gets a lot of mentions, it becomes a standard, the top of the distribution curve, inscribed in the weights of the model. Other ideas, also popular, contribute to the curve too. There are a lot of other ideas, and at some point, the crawler must stop. The weird, less popular ideas, those don’t get learned properly. The weird ideas are too far away from the curve top. The crawler chops the curve at the edges.

If we creatives rely on the top hits, the central ideas most indexed by LLMs, if our output resembles the output of LLMs, why are we even needed?

—

Over time, of course, we won’t reinforce the compression of standards. Not completely. Certainly not you, who craves creativity.

The looping, the reinforcement and the compression of standards, that will leave you in bad taste. Highly refined foods leave you craving for goodness, and you will set out exploring.

You will proceed where no one has dabbled. You will want to invent something weird in your spare time. You will want to put out something creative to kill the boredom.

You will want to add deviations back into the distribution curve. You will create where it’s two deviations away. You might even feel like it’s a moral obligation.

You will resist merely training. That’s the process that creates LLMs. You will have personality instead.

Creatives will be here to stay, because creatives exert effort.