Classifier, regressor or language model: three scorers, three kinds of answer

An image scorer is built one of three ways: a classifier that picks a bucket, a regressor that outputs a continuous value, or a vision-language model that writes a number as text. All three can return "7" for the same photo, but the 7 means something different in each, and so does the failure when it is wrong.

Why the build matters more than the number

A score on a screen carries no label saying how it was produced. Two tools can show the same figure with the same decimal places and be doing entirely different things underneath. One is looking up which of ten bins the photo most resembles. Another is interpolating along a line. A third is a language model that has decided, after reading the picture, that "seven" is a plausible thing to say next.

This is the site's recurring point in a concrete form: the number looks like a measurement, and the architecture shows it is an inference of a particular kind. Knowing which kind tells you what the number can be trusted for, how it will wobble, and what questions are even sensible to ask of it. It also tells you what the tool is able to explain, which is usually less than its interface suggests.

The comparison below is a methodology question, not a ranking. None of the three is the best design. Each trades something away to gain something else, and the trades are what you want to understand.

Design one: the classifier

A classifier looks at the image and picks one of a fixed set of classes. For scoring, the classes are the score levels, or coarser groups of them such as low, middle and high. The final layer produces one raw value per class, and a softmax turns those into probabilities that sum to one.

The usual output is the class with the highest probability. The more interesting output is the whole distribution, because it says how sure the model was. A photo that gets 0.9 on "seven" and a photo that gets 0.35 on "seven" with 0.30 on "six" both display as a 7, but they are very different situations. Many tools then collapse the distribution to its expected value, so that probability spread across six, seven and eight becomes a figure near seven with a decimal.

What a classifier can explain is limited but honest. It can tell you which class won and how close the runner-up was. What it cannot do natively is express that six is closer to seven than to two, because the classes are treated as unrelated labels. Ordinal structure has to be bolted on, and when it is not, a model can make a confident mistake that is far from the truth rather than a near miss.

Classifiers fail in recognisable ways. They are sensitive to class imbalance, so rare extreme scores are underpredicted, a problem covered in why extreme scores are rarely given. They fail abruptly at bucket boundaries, where a tiny pixel change can flip the winning class and move the displayed score a whole step. And they are only as fine-grained as their buckets: a ten-class model cannot say 6.4 unless something averages its outputs.

Design two: the regressor

A regression head outputs a single continuous number directly. The training loss punishes distance, so predicting 6 for a photo labelled 7 costs less than predicting 2. That solves the ordering problem the classifier has.

The regressor's strength is smoothness. Small changes to the input tend to produce small changes in the output, which makes before-and-after comparisons more meaningful. Its weakness is a pull toward the middle. Because the loss is lowest for safe guesses when the data is noisy, regressors tend to shrink their outputs toward the average, so extremes are compressed. The numbers also look more precise than they are, since a continuous value invites a decimal point whether or not the model has any real information at that resolution.

A regressor offers no built-in sense of confidence. It returns one value and nothing about how sure it was, unless the design adds a second output for uncertainty or the tool runs it several times. The head-level trade-off between the two is worked through separately; the point here is that the choice of head is one of the three big architectural forks, not a detail.

Design three: the vision-language model

A vision-language model, or VLM, does not have a score output at all. It reads the image and generates text, and one of the things it can be asked to generate is a number. The "score" is a token sequence chosen by the same process that writes sentences.

That changes almost everything. The model can describe what it saw before committing to a number, and the order matters, as describe-then-score shows. It can be asked to rate on a rubric written in plain language, which is far more flexible than retraining a head for each new rubric. And it can give a reason, which neither of the other designs can do natively.

The reason, though, is generated alongside the number rather than being a readout of why the number came out as it did. It is a plausible account, not a trace of the computation. Treat it as commentary.

VLM scores are also sampled. Unless the sampling is pinned down, the same photo can return different numbers on different runs, a property dissected in the temperature setting. Their outputs cluster on a few favourite numbers, because language models have habits about which digits sound reasonable. And they respond to the wording of the instructions, so a rubric edit is a model edit.

A related and older design sits between the camera and the language model: contrastive image-text models, which Radford and colleagues at OpenAI (2021) trained on 400 million image-text pairs to match pictures with captions. Scoring by similarity to a described target is a fourth route that borrows from both worlds, discussed in CLIP-style scoring.

The three side by side

The last row deserves a second look. Fluent explanation is not the same as faithful explanation. The designs that explain least are the ones whose behaviour is easiest to probe from outside, because their outputs are deterministic and their failure modes are mechanical.

How each one fails when the photo is unusual

Every scorer is trained on a distribution of photos, and an unusual input sits at its edge. What happens there depends on the design.

A classifier will still pick a class, often with spread-out probabilities that signal doubt if you can see them. A regressor will return a number with no flag at all, usually one near the middle. A VLM may describe the oddity accurately and then score it anyway, or it may refuse, or it may produce a number whose relation to the description is thin.

None of these failures announces itself in the final figure. That is why the question "what happens on an odd photo" cannot be answered by looking at ordinary results, and why cheap checks in front of the model, such as quality gates, exist.

What the training data does to each

All three designs inherit their labels from people, and the labelling problem sits upstream of the architecture. Who taught the model a 7 applies equally to a classifier trained on buckets, a regressor trained on averages, and a VLM whose instruction tuning leaned on human preference data.

The difference is where the label's imprecision lands. A classifier is hurt when the labellers disagree, because the buckets are forced to take a side. A regressor averages the disagreement into a mean and loses the fact that it existed. A VLM absorbs it into a general sense of what sounds reasonable, with no record of which humans it came from. Disagreement among raters is a property of the task, and each architecture simply hides it differently.

Many tools use more than one

Real systems rarely commit to a single design. A common arrangement runs a fast regressor for a base number and a language model for a written comment. Another runs several models and averages them, which the ensemble post covers, smoothing noise at the cost of blurring which model is responsible for what. A pipeline can also be layered, with one model deciding whether a photo is usable and another scoring the survivors.

When a tool does this, the right question is which component produces the number you are reading. Rate Cock's scan is a specific, documented case of this kind of layering, and its own page is the place for the step-by-step; the general lesson here is that a single displayed score is usually the end of a chain.

Questions worth asking of any scorer

You can learn a lot about a tool without seeing its code. These questions map directly onto the three designs.

• Does the same photo return the same number every time? If not, a sampled model is probably involved.
• Do the results cluster on a few values, or does every decimal appear? Clustering suggests buckets or a language model.
• Are extreme scores ever given? If almost nothing is rated at either end, the head or the data is compressing the range.
• Does the tool give a reason? If so, treat the reason as commentary about the number rather than a readout of it.
• If you change only the wording of a prompt-like setting, does the score move? That points to a language model.

Asking these is a methodology exercise. It is the same discipline that applies when penisrater.com explains what a total can and cannot support from the user's side, only pointed at the machinery.

Where to go next

If the number came from a language model, start with describe-then-score and the sampling post, because they explain most of the wobble you will see. If it came from a head, read the regression-versus-classification comparison for the loss-level detail this piece skipped. For how any of them are trained and where their labels come from, the end-to-end training walk-through fills in the part that sits before the architecture.

When the question shifts from how a score is built to how it compares with a person, the reference is an actual panel, and ratepenis.com covers what separate judges bring to that comparison. For the physical dimensions of anything, none of these three designs measures them, and measuremycock.com is the property that owns method. Whichever design is under the hood, it is an estimator trained on opinions, and the useful thing to carry away is a habit: before reading a number, work out which of the three machines produced it.