Features are not the things you would name

When someone describes a vision model as "looking at" the shape or the texture, that word is doing a lot of unearned work. A feature, in the technical sense a model actually uses, is an activation pattern - a detector that fires more or less strongly depending on what is in front of it - and it was never given a name, a definition, or a boundary that matches anything a person would list.

Where the word comes from

A trained network is built out of layers, and each layer contains many small detectors, each tuned during training to respond to some pattern in its input. A "feature" is just the output of one of these detectors: a number, or a small grid of numbers, indicating how strongly that pattern was present. Nobody writes down what pattern a detector is looking for in advance. Training adjusts the detectors until the network's final output matches the labels it was shown, and whatever patterns turn out to be useful for that goal are the ones that survive - the process has no obligation to produce detectors a person would recognise or could put a word to.

The hierarchy, briefly

Features are not all the same kind of thing, and where a detector sits in the network changes what it responds to. Early layers, closest to the raw pixels, respond to simple local structure - an edge at a particular angle, a patch of high contrast, a boundary between two colours. Middle layers combine several early detectors into something more complex - a corner, a repeating texture, a curve. Later layers combine those again into patterns complex enough that they start to correlate, loosely, with things a person might call a part or an object.

That correlation is the source of most of the confusion. It is tempting to say a late-layer detector has "learned what a hand looks like," and in a rough sense that can be true - but the detector was never told what a hand is, has no concept that a hand is jointed or attached to an arm, and will happily fire on something that merely shares its statistical signature. This full progression, layer by layer, from raw pixels through to a compressed representation, is covered in more depth in what an image becomes before it is scored; this post is about what any one step in that chain actually consists of.

Why the mismatch matters

The practical consequence is that a model can be responding strongly to something in a photo that no person filling out a description of the image would think to mention - a particular lighting gradient, a texture that resembles one seen often in training, an edge configuration that happens to correlate with a rating in the training data for reasons nobody chose on purpose. None of that is visible from outside. The interface shows a score or a written verdict; it does not show which detectors fired, unless the tool has specifically built a way to surface that, which is its own separate mechanism covered in attention maps and what the heatmap really shows.

This is also why two photos that look, to a person, like straightforward variations of the same thing can be read quite differently by the model: if the variation happens to land on a feature the model is sensitive to - a particular contrast pattern, a particular texture at a particular scale - the resulting shift in output can be larger than the visual change seems to warrant to a human eye. Framing and lighting changes are a common practical source of exactly this kind of feature-level shift, which is one reason the guide to taking a photo suited to an accurate reading exists as its own resource on Rate Cock rather than being reconstructed here.

Not the same as bias, not the same as noise

It is worth separating this from two adjacent ideas. A feature being alien to human vocabulary is not the same claim as the model being biased - bias is about what the training data over- or under-represented, a different subject covered under dataset bias. It is also not the same as randomness - a given detector fires consistently on a given input; the strangeness is in what it responds to, not in whether it is repeatable.

What this means for reading a result

None of this makes a score meaningless, and it is not a reason to distrust a tool on principle. It is a reason to hold the score loosely as "what a set of learned pattern-detectors produced," rather than as a checklist match against the properties a person would name. Penis Rater's notes on reading a score honestly cover the user-facing side of that same caution - what a total number does and does not license you to conclude. A human reviewer, by contrast, genuinely does reason in terms a person would name, which is one of the structural differences between the two approaches; what a person reads that a model structurally cannot is worth knowing if you are deciding which kind of feedback you actually want. Where a rubric splits a result across several named axes rather than one figure, each axis is still, underneath, being read off features with no name of their own - the naming happens at the label, not inside the model - which is part of why a rubric beats a single score without ever making the underlying read-out any less alien. If you want a number with none of this ambiguity attached to it, Measure My Cock's method starts from a tape rather than a feature detector, which sidesteps the whole question.