Two ways to build a rubric scorer, and how they fail differently

Most production rubric scorers use one shared model with six outputs rather than one model per axis, because it is far cheaper to train and run. The cost is leakage: a shared representation lets axes influence each other, while separate models stay independent but are slower, pricier and noisier on thin data.

Both can present identical interfaces, so the choice is invisible from outside.

Shared backbone, multiple heads

The common approach is a single vision encoder that turns a photo into one embedding, followed by several small output heads, one per axis, each reading from that same shared embedding and producing its own number. This is a multi-task model in the standard machine-learning sense: one representation, several predictions drawn from it.

It is cheaper to train, since the expensive part - the encoder that actually learns to understand images - is trained once and shared, rather than duplicated six times. It is cheaper to run in production too, since scoring a photo means one forward pass through the shared encoder plus six lightweight heads, rather than six full model evaluations.

Separate models per axis

The alternative trains a fully independent model for each axis, each with its own encoder, learning its own representation of the photo tuned only to what that one axis needs. Nothing is shared between them except the training methodology.

This is more expensive on every axis: more compute to train six full models instead of one shared one, more compute to run six full inferences per photo instead of one, and six times the labelled data requirement if each model needs its own full training set rather than sharing labels across a joint training run.

Where the shared approach leaks

A shared encoder learns one representation optimised to be useful across all six axes at once, which means each axis's prediction is drawn from an embedding shaped partly by the other five axes' training signal. If the embedding ends up encoding information useful mainly for one or two axes, the heads for the remaining axes are drawing on a representation that was not built with them specifically in mind, and their accuracy on those axes can suffer relative to a model trained only for that one purpose.

More subtly, gradient updates from all six axes flow back into the same shared encoder during training, which means the encoder's development is influenced by every axis at once, and it is a documented pattern in multi-task learning that a task with more or noisier training examples can dominate what the shared representation ends up encoding, at the expense of tasks with less signal. Chen and colleagues (2018) describe deep multitask networks as "challenging to train properly", and their method, GradNorm, balances training by dynamically tuning gradient magnitudes across tasks rather than leaving one task to set the pace. This produces a specific version of the leakage correlated axes and the halo effect can cause independently: even with clean, blind labels and genuinely uncorrelated underlying properties, a shared architecture can still manufacture some cross-axis influence purely through the shared representation, on top of whatever the labels already carry.

Where independent models fail differently

Separate models have no such leakage between axes by construction, since nothing is shared. Their failure mode is instead that each one is trained on a smaller effective signal - fewer parameters justified by fewer labels per axis in practice, since teams rarely have the budget to collect a full independent dataset six times over - and a model with less to learn from tends to be noisier, not more accurate, than the same axis trained as part of a well-designed shared model with enough data behind it.

Independent models are also six times the maintenance burden going forward: six things to monitor for drift after retraining, six things to validate, six deployment pipelines instead of one.

Which one a rubric ends up using

In practice, most production rubric scorers use a shared backbone with multiple heads, because the cost difference is large and the leakage, while real, is usually smaller in effect than the accuracy lost from splitting a limited labelled dataset six ways. A shared model with genuinely separable, well-defined axes - the kind worth designing carefully in the first place - keeps leakage to a manageable level; a shared model with axes that were poorly separated to begin with will show leakage that looks a great deal like the halo effect, whatever labelling discipline produced the training data.

This is an architecture choice with real compute costs behind it, and compute is the kind of variable measuremycock.com's coverage of gear treats seriously when it comes to what running a model at any scale actually requires. Rate Cock reports six axes from a single result, which is consistent with either architecture from a user's vantage point, and penisrater.com's guidance on evaluating tools is largely about judging a result by its consistency rather than by guessing at what is underneath it. A human reviewer has no architecture question at all here, since a person forming six separate judgements is not drawing them from one shared internal representation the way a multi-task model is, which is a genuine structural difference rather than a stylistic one.