A model trained on engagement learns what got clicks

A label is whatever a model was told counted as the right answer during training. Two very different things get used as labels in image scoring, and they produce two very different models even when the output looks identical: a number between one and ten.

Two ways to get a training label

The deliberate route is a person looking at an image and assigning a score against a rubric, usually with instructions about what each point on the scale should mean. This is slow and expensive, because it needs trained labellers and takes their time per image.

The other route is to skip the person and use signals the platform already has. A photo that got more likes, more views, or a longer dwell time is treated as if it scored higher, and one that got ignored is treated as if it scored lower. No one ever assigned it a number. The number is manufactured from behaviour.

What each label actually encodes

A judged label encodes a rubric. Whatever the labeller was told to look for is what ends up in the model, for better or worse.

An engagement label encodes something else entirely: what an audience, on a specific platform, at a specific time, chose to click on. That includes the subject, but it also includes the thumbnail, the caption, the time of day it was posted, who follows whom, and the platform's own recommendation algorithm deciding who saw it in the first place. A photo can get few likes because it is a weak subject, or because it was posted at 3am to nobody's feed. The label cannot tell those apart, and neither can a model trained on it.

Popularity bias

A model trained on engagement is not learning "what looks good." It is learning "what this audience, under this platform's distribution mechanics, engaged with," which is a narrower and stranger thing.

That distinction shows up as popularity bias: the scorer will systematically favour whatever category of image already got algorithmic reach, independent of the property it is supposed to be rating. An account with more followers gets more engagement on a mediocre post than a new account gets on a strong one, and a model that treats engagement as ground truth inherits that gap as if it were a fact about the photo. This is a known failure mode in recommendation research more broadly - the label reflects exposure as much as quality, and exposure is unevenly distributed before any scoring happens.

Why this matters for a rating tool

If a tool never says where its labels came from, engagement is a plausible source, because it is cheap and already sitting in a platform's database, unlike judged labels which have to be commissioned. A tool built on engagement data will tend to reward images that resemble whatever got attention on the platform it scraped, which is not the same target as the rubric the tool claims to score against.

This is worth checking for the same reason it is worth checking where any scorer's labels came from at all: the labelling method sets a ceiling on what the finished model can honestly claim to measure, and an engagement-trained model's ceiling is attention, not quality. Rate Cock scores against a defined rubric rather than platform engagement, which is a distinction worth knowing to look for in any tool, not just this one.

What to look for

A model built on judged labels can still be biased, inconsistent, or badly calibrated, but at least the target it was trained toward is the one it claims. A model built on engagement has a target mismatch baked in before any of that, and no amount of downstream calibration fixes a model that was pointed at the wrong thing from the start.

The practical tell is usually in what the tool rewards. An engagement-trained scorer tends to favour whatever composition, lighting and framing already photographs well for social sharing, independent of the rubric it claims to score, because that is the pattern its labels actually encoded. A judged-label scorer has no reason to reward shareability specifically, since its labels were never about what an audience did with the image afterward. That same confound shows up more broadly whenever a model's sense of "typical" comes from whatever happened to fill its training set, which is the general shape of dataset bias - engagement labels are simply one particularly clear route into it.

Collecting judged labels at scale, with real people looking at real images against a real rubric, is the harder and more expensive path, which measuremycock.com covers from the data-collection side. Reading a result with this distinction in mind - asking what the number is actually predicting, popularity or a rubric - is part of reading a score honestly, and a human reviewer sidesteps the whole question by never training on engagement at all, which is one of the actual differences a commissioned review offers.