A 92% is a normalised number, not a belief

When a rating tool shows something like "92% confidence," the number underneath is very often a softmax output, and a softmax output is not a measurement of how sure the model is about anything. It is a normalisation step, built to make a set of raw scores add up to exactly one so they can be read as if they were probabilities.

What softmax actually does

A classification head, of the kind described in regression head or class buckets, produces one raw number per possible category before any confidence figure exists - an unbounded value called a logit, which can be any size and does not naturally sum to anything meaningful across categories. Softmax takes that whole set of logits, exponentiates each one, and divides by the total, guaranteeing the results land between 0 and 1 and add up to exactly 1 no matter what the raw logits looked like.

This is a mathematical convenience, not a claim about certainty. If a model has two categories and genuinely has no idea which is correct, softmax over two similar logits will still hand back numbers like 51% and 49% - and if the logits happen to differ more than that even under real uncertainty, the softmax step can amplify a small raw gap into a large-looking percentage gap, purely as an artefact of the exponential in the formula.

Why high confidence and being wrong coexist

Neural networks trained the ordinary way are well documented to be overconfident: their top softmax value is, on average, higher than their actual accuracy would justify. This was studied directly by Guo and colleagues in their widely cited 2017 paper "On Calibration of Modern Neural Networks," which found that many modern architectures, despite being more accurate than older ones, are also more miscalibrated - their confidence numbers overstate how often they are actually right. A 92% figure, in other words, does not mean the model is right 92% of the time it says that; it means the softmax output for the top category happened to be 0.92, which is a different and much weaker claim.

There is also nothing structurally stopping a model from being both confident and wrong on a specific input. Confidence, as softmax computes it, is a function of how spread out the raw logits were - a very peaked set of logits produces a high softmax value regardless of whether the peak landed on the right category. A model can be extremely peaked, and extremely wrong, on an image that happens to resemble the wrong category strongly along whatever features it is sensitive to.

What this means for a percentage on screen

None of this means a displayed confidence number is meaningless - a genuinely low softmax value is a real signal that the model was comparing closely matched options and is worth treating with more caution than a high one. The caution is specifically about treating the number as calibrated - as literally tracking the frequency of being correct - rather than as one more relative signal among several. Whether a system is designed to decline answering when its own confidence is low, rather than reporting a number regardless, is a design choice covered separately in should a scorer abstain when it is unsure?, and calibration techniques that attempt to correct softmax's overconfidence, when a tool bothers to apply them, are their own subject in calibrating a scoring model.

Rate Cock does not surface a raw confidence percentage as its headline output, reporting axis scores instead, which sidesteps this specific source of confusion even though the underlying softmax mechanics, where classification is used anywhere in the pipeline, are the same as in any other tool. A human reviewer expressing confidence in a judgement is doing something structurally different - reasoning about their own certainty rather than reading off a forced normalisation - and Rate Penis's notes on what a human review actually contains cover that distinction from the judging side. If your interest is a figure where "confidence" is not even the right concept because the number comes from a tape rather than a trained classifier, Measure My Cock's method is the relevant page, and Penis Rater's tool notes cover what to make of any confidence figure a given service shows you in practice.