Accuracy

Computational photography is a hidden preprocessing step

Modern phones sharpen, smooth, tone-map and merge frames automatically, so the file the model receives is already an interpretation.

3 min readAccuracy

The photo that leaves a modern phone is not what the sensor recorded. It is the output of a processing pipeline that ran automatically, merged several exposures, and made a series of decisions about sharpness, smoothness and tone before the file was ever saved. A scoring model never sees the raw capture, only the result of that pipeline, and the pipeline is doing more than most people assume.

HDR merging

Most phone cameras now capture a burst of frames at different exposures and merge them into a single image, a process generally called HDR. The goal is to preserve detail in both bright and dark parts of a scene that a single exposure would lose, and the merge is computed automatically, with no user-visible step. The output is a synthesis, a single image built from several, and the merge algorithm makes choices - how to weight each frame, how to blend the seams - that a raw single-exposure capture would not have involved. Different phones merge differently, so identical scenes shot on different devices can produce meaningfully different files even before anything about editing or filters enters the picture.

Automatic smoothing and sharpening

Phone image pipelines commonly apply noise reduction, which smooths fine texture, and then apply sharpening to compensate, which can exaggerate edges the smoothing softened. Some phones go further and apply semantic-aware smoothing, in effect a mild automatic beautification, that specifically softens things the pipeline has recognised as skin - a decision baked into the camera app, made before you touch anything, and not something you opted into deliberately the way you would with a beauty filter you chose. Texture is one of the properties a scoring model reads most directly - it's close to what surface-condition axes are trained to respond to - so smoothing applied silently by the camera changes exactly the kind of signal the model treats as informative.

Tone mapping

HDR merging and general processing also involve tone mapping: compressing a wider range of brightness values into the range a screen and a JPEG can display. This changes local contrast, which is another cue the model reads as shape, related to but distinct from the lighting-direction effect covered elsewhere - tone mapping happens after the light was captured, reshaping the gradient information rather than being the cause of it. Two photos taken under identical physical light, on phones with different tone-mapping behaviour, can present different-looking shading to a model even though nothing about the actual lighting setup differed. Frame merging can also introduce faint ghosting or blending artefacts around anything that moved slightly between exposures, a small extra source of texture noise layered on top of everything else the pipeline already changed.

Why this matters for a score

None of this is a choice the person taking the photo made. It happens inside the camera app, by default, and most people have no visible control over it beyond turning the whole camera app's processing off, which few phones let you do easily. The consequence is that a model is never really scoring "the photo you took" in a raw sense - it is scoring the output of an image-processing pipeline that already interpreted the scene once, an additional layer on top of everything about why the same subject spans a range of scores rather than the same one every time.

This is a different mechanism from JPEG compression, which happens at the file-encoding stage after the image is already finished - that's covered separately - and it is a different mechanism from a filter applied deliberately after the fact, which is its own subject. Understanding which stage introduced a given change is useful mainly for anyone running an actually controlled comparison, where switching phones between shots quietly swaps out the processing pipeline along with everything else. A tool worth trusting with a comparison is one you keep the same device and settings across, precisely because this stage is invisible and easy to forget about. None of it is something Rate Cock or any scoring tool can see past or correct for after the fact, since by the time the file arrives, the interpretation has already happened - a limit worth remembering the next time a human reviewer's take differs from what the model reported on the same photo.

Read next

Full archive