Relative judgements are easier than absolute ones

Ask a model "which of these two photos is better" and it will usually give a consistent answer. Ask the same model "what score, out of ten, is this photo" and the answer will wobble more from run to run. This is not a quirk of any particular tool - ordering two things reliably is a genuinely easier problem than assigning a stable absolute number to one thing, and the gap between them shows up in almost every scoring system built this way.

Why ranking is the easier problem

An absolute score requires the model to place a single input on a fixed scale with no other input to compare it against - it has to recall, implicitly, everything else it was ever trained on and locate this one photo among all of it, using only the number line as a place to put the answer. A ranking task hands the model both items at once and asks for one bit of information: which one, of these two, is better. The model can use direct contrast - texture, symmetry, framing, whatever the axis is - between two things actually in front of it, rather than reconstructing an internal reference scale from memory. This is also closer to how humans behave: people are notoriously inconsistent giving absolute ratings and much more reliable picking between two things, which is why so much of the training data behind these systems was collected as comparisons in the first place rather than as scores.

What this does to the numbers you see

A 1-10 score is usually the output of a system that was better at comparisons than it is at the absolute scale it now has to report on. Somewhere in the pipeline, comparative judgements get converted into a fixed scale - a process that introduces its own error, since a scale has to decide not just which of two things is better but by how much, which a pure ranking model was never actually trained to answer. This is one reason absolute scores cluster and wobble more than the underlying comparative judgement would suggest: the conversion step is doing extra work the ranking itself did not need to do. A model can be extremely reliable at "A beats B" while being noticeably less reliable at "A is a 7.2 and B is a 6.8," even though the second claim looks like it follows from the first.

Where this matters practically

If what you actually want to know is whether a retake is better than the original, a ranking-style comparison is the more trustworthy way to ask the question, because it plays to the part of the system that was built to be good at exactly that. Running a controlled comparison already assumes this implicitly - change one variable, compare the two results - and the method works better than trusting two absolute scores taken separately precisely because it is leaning on the model's stronger skill. Ceiling and floor effects are also a symptom of the same underlying gap: near the top and bottom of an absolute scale, the model has the least room to express a relative judgement as a numeric difference, and two genuinely different inputs can collapse onto the same top or bottom number even though a head-to-head comparison between them would still separate them cleanly.

The limits of leaning on ranking

Ranking is not a free upgrade to accuracy - it answers a narrower question and cannot be stretched to answer a broader one. It tells you which of two specific things is better, not by how much, and not where either one sits on any absolute or externally meaningful scale. A long chain of pairwise comparisons can be used to reconstruct an implied ranking across many items, but reconstructing a full ordering this way takes far more comparisons than most tools ever run, and the result is still an ordering, not a scale with fixed, comparable units. Whether a scale is even the right kind of thing to ask a model to produce is close to the broader question of what accuracy could mean here at all, since a ranking sidesteps the ground-truth problem for the absolute number without actually solving it.

What this means for reading a result

Treat a single absolute score with the understanding that it was likely built on top of a system better suited to comparisons than to standalone numbers, and treat a head-to-head result - this photo against that one - as the more load-bearing claim. Rate Cock shows a breakdown by axis rather than only a total, which makes it easier to compare two of your own submissions axis by axis, closer to the comparative task the underlying system is actually good at, rather than trusting the gap between two absolute totals. Comparing your own results over time is exactly the practice penisrater.com treats as the useful unit of interpretation, for the same underlying reason. A human panel does not have this particular asymmetry in the same form, since a person giving a score and a person picking a winner are drawing on the same judgement either way, which is one of the differences a commissioned human review brings to the comparison that a model-based one does not. None of this applies to a measured figure, where the number is not reconstructed from comparisons at all but read directly, which is the different kind of accuracy a physical method provides.