Accuracy

Where the light comes from matters more than how much

Side light, top light and flash change contours and shadows the model reads as shape, so direction is a bigger variable than brightness.

3 min readAccuracy

Two photos of the same subject at the same brightness can produce noticeably different scores if the light comes from a different direction in each. That is not a brightness effect. It is a shape effect, because direction is what determines where the shadows and gradients fall, and shape is exactly what a vision model is reading.

Shading is a proxy for form

A single photograph has no depth channel, and models have no way to directly perceive three-dimensional structure from one image. What they get instead is shading: the gradual change in brightness across a curved surface, which the human visual system - and, having been trained on human-labelled images, the model by extension - uses as one of the strongest available cues to inferred shape. Overhead light produces one gradient pattern; light from the side produces a different one across the same physical surface; flash, coming from close to the camera, produces almost no gradient at all because the light source and the viewing angle are nearly the same. The physical subject has not changed between these three photos. The gradient the model reads off the pixels has changed substantially, and gradient is one of the features closest to what a shape-sensitive axis is trained to respond to.

Why direction outweighs amount

Increasing or decreasing overall brightness mostly rescales pixel values uniformly, which is close to the kind of variation models are explicitly trained to be robust to through data augmentation - what augmentation teaches a model to ignore usually includes exactly this. Changing the direction of the light changes the spatial pattern of the shading itself, which is structural information the model was never told to discount, because there is no equivalent "ignore this" augmentation for the actual shape cue a photo carries. This is why a slightly darker photo lit well can score more consistently than a brighter photo lit from an unflattering angle - the model was built to shrug off exposure, not off geometry.

Side lighting exaggerates surface contour by casting a visible shadow along one edge, which tends to read as more defined shape. Flat, frontal light - flash being the extreme case - collapses most of that gradient, leaving an image that looks evenly lit but supplies the model with comparatively little shape information to work from, an effect covered in more detail in how flash specifically flattens contour. Overhead light casts shadows downward in a way that can distort where an edge appears to sit relative to the rest of the frame. None of these are the model being wrong so much as the model doing exactly what it was built to do with whatever gradient pattern it was handed.

What this is not

This is not a statement about exposure or skin tone rendering, which is a related but distinct question covered in how exposure and skin tone interact with model bias. It is also distinct from the light's natural source changing across a day rather than a room, since time of day and physiological variation covers a subject-side cause the camera is not responsible for. It is also not a how-to for lighting a photo well - that's a practical guide Rate Cock owns, written for someone about to take a photo rather than someone trying to understand why two photos differed. This piece is one entry in the wider fact that the same subject can produce a surprisingly wide spread of scores, and lighting direction is one of the more controllable variables inside that spread, alongside camera angle and lens distance.

Where it fits alongside other variables

Light direction interacts with angle and distance rather than acting alone - a side-lit photo taken close with a wide lens compounds a perspective effect with a shading effect, and separating the two after the fact from a single photo is not generally possible. Controlled testing, the kind measurement-focused sites take seriously, changes one variable at a time for exactly this reason. A human reviewer looking at the same photo brings context that partly compensates for lighting the way a model cannot - what a human judge reads that a model cannot includes exactly this kind of inference across an unusual shadow. Reading a spread of scores with lighting in mind, rather than treating each number as independent, is the more useful habit, and it is one worth carrying into any comparison of results across a session.

Read next

Full archive