How it works

Few training examples at the ends means timid predictions there

If the training data had few very low or very high examples, the model has little to go on at the extremes and retreats toward the centre.

3 min readHow it works

Most rating scales are not filled evenly by real submissions. People tend to score most images somewhere in the middle of a rubric, which means a training set built from real labelled photos has plenty of 5s and 6s and very few 1s or 10s. That imbalance shapes what the model can learn, before anything about architecture or optimisation enters the picture.

Why the ends starve first

A model learns a category, or a region of a scale, from the examples it was shown of it. Give it a thousand examples of "around average" and ten examples of "extremely high," and the gradient signal pushing it to recognise the extreme case is a tiny fraction of the signal pushing it toward the average.

The model is not being lazy. It is doing exactly what the data told it to do: minimising error across the whole training set, and the whole training set is overwhelmingly made of middling examples. An occasional genuine 9 or 10 in training barely moves the average loss, so there is little pressure to get those rare cases right specifically.

What this looks like at inference time

Ask the finished model to score something genuinely at the top of the distribution, and it will often undershoot, landing at a 7 or 8 rather than committing to the top of the scale. The same happens at the bottom. The model has seen so few examples out there that its safest bet - safest in the sense of minimising expected error, which is what it was trained to do - is to hedge toward where most of its training mass sat.

This is a training-data effect specifically, distinct from the several other reasons scores cluster near the middle of a scale, which is its own subject covering the full list of causes including scale design and label caution. It also compounds with ceiling and floor effects, where a model that has reached the top of what it can express treats two different exceptional inputs identically.

What fixes it, and what does not

Collecting more examples at the extremes is the direct fix, but it is expensive precisely because extremes are rare in the wild by definition. Some teams reweight the loss function so rare-score examples count for more during training, which helps without needing more data, though it can make the model noisier at the extremes if pushed too far. Oversampling the rare examples - showing them to the model more often per epoch - is the cheaper cousin of the same idea. A tool that reports separate axes rather than one blended figure, the way Rate Cock does, at least lets you see which axis is doing the hedging, rather than one total quietly absorbing it.

None of this is unique to body-image scoring. Any model trained on a naturally lopsided distribution of labels shows the same retreat toward the centre, which is why measuremycock.com's coverage of the data side treats sample composition as a first-order concern rather than a footnote. A result that lands at the extreme end of a tool's scale is, for this reason, worth a second look rather than automatic trust, a habit penisrater.com recommends for reading any single number. A human reviewer does not have this particular failure mode, since a person is not limited to whatever the training set happened to contain, which is one of the structural differences a commissioned review offers.

Read next

Full archive