The second upload may not be scored at all

You upload a photo, get a score, upload the exact same file again a minute later, and get the exact same score. That looks like proof the tool is perfectly reliable. It might just be proof that the tool remembered the file and never scored it twice.

Why services cache

Running a model costs compute, and an identical file arriving twice is a request most systems can answer for free the second time by checking whether they have seen those exact bytes before. The usual mechanism is a hash of the file - a short fingerprint computed from its contents - stored alongside the result the first time it was scored, the same technique covered in more detail elsewhere for how it works and what it is used for beyond caching, such as recognising near-duplicate uploads. When the identical hash shows up again, the service can return the cached result instantly instead of running the model, and there is nothing dishonest about doing this - it is ordinary engineering, and most users benefit from the speed without ever noticing.

Why this ruins a reliability test

If you are deliberately testing whether a tool's scores hold steady on repeat runs, a cache defeats the entire point. You are no longer measuring the model's behaviour on the second and third attempts - you are measuring how well the cache remembers the first one, which will always be a perfect match, because it is the same stored value. A reliability check that only ever returns the identical number is not evidence of a tight distribution; it can just as easily be evidence that no second measurement was ever taken.

How to defeat it, if you want a real test

The fix is to make sure each upload is a genuinely different file at the byte level, even though the image looks identical to a person. Re-encoding the same photo through even a lossless conversion changes its bytes and its hash. Changing a single pixel, adjusting the file's metadata, or re-saving it through different software all produce a new hash while leaving the visible content effectively unchanged. Any of these breaks the cache and forces a genuine re-score, which is what a real repeat-reliability test needs.

A related trap: order effects

A separate issue worth knowing about is order sensitivity - some pipelines that batch or route requests can behave slightly differently depending on what else is being processed alongside your upload, which is a much smaller effect than caching but worth ruling out if your results still look suspiciously patterned after you have defeated the hash cache. If scores seem to depend on time of day or how quickly you submit them, that is a different investigation from the one this note covers.

The takeaway

Before concluding a tool is remarkably consistent, check whether you actually ran it twice - taking a fresh photo each time rather than reusing the same file is the simplest way to sidestep the question of caching entirely. How many genuinely fresh repeats you need only applies once each one is a real, distinct request. The equivalent trap does not really exist for a physical measurement - a tape measure has no memory of the last reading to fall back on - which makes this a distinctly digital failure mode. It is worth ruling out before trusting any self-run consistency check on your own uploads, and a human reviewer sidesteps it entirely, since a person looking at the same submission twice is never simply retrieving a cached answer.