A generic marker is not good enough
Two teachers can mark the same script and write different comments. One awards the method mark when the working is close. The other withholds it. One writes three sentences of feedback. The other writes eight words.
Neither teacher is wrong. Each department holds a standard, and that standard is the product of years of moderation. An AI that ignores it produces work the teacher must rewrite.
So Ren learns the standard. It does this two ways, and the two work on different timescales.
- The style guide builds slowly from a teacher's marking across every paper.
- Calibration runs quickly on one paper, before the full batch is marked.
This post explains both.
How the style guide learns
Ren marks a submission. The teacher then reviews it and finalises it. At that moment Ren compares what the AI suggested against what the teacher kept.
Five things count as a signal:
- The teacher edited the AI's comment.
- The teacher changed the AI's score.
- The teacher changed the AI's tags.
- The teacher rejected the suggestion.
- The teacher wrote a comment where the AI offered none.
A background job collects these signals for the teacher. It does not act on the first one. The job needs at least three signals before it rewrites anything, because one edit is not a pattern.
Ren keeps two guides per teacher: one for structured question papers and one for essay papers. The preferences differ, so the guides must differ. A rule about partial credit on a three-mark question does not belong in an essay guide.
The job then rewrites the guide from scratch. It reads the current guide and the new signals together. It drops any rule the new signals contradict. This matters more than it sounds — a guide that only grows becomes a pile of stale rules that argue with each other.
Why the guide is short
The guide is capped hard: 600 words, six bullets per section, three examples. Four sections only — General Preferences, Scoring Rules, Feedback Style, Examples.
The cap is not a cosmetic choice. The guide is injected into the marking prompt, next to the answer scheme. Prompt space is not free.
We measured this. In a controlled test, we appended 573 characters of meaningless text to a marking prompt. The AI's total for the paper moved by 1.2 marks. The text carried no instruction at all. Length alone moved the score.
A style guide that grows without a limit crowds out the answer scheme it is supposed to support. So the guide stays tight, and every rule must earn its place. The job is also told to omit any rule that is obvious common sense.
One more constraint: the guide must stay subject-agnostic. It states principles, not topics. It may say "award the mark when the method is correct and the arithmetic is not". It may not name a formula or a specific compound. A teacher's marking standard should carry across their whole subject, not stick to the paper that produced it.
Calibration: agree on the scheme before the batch runs
The style guide is slow and general. It learns your voice across many papers.
Calibration is the opposite. It is fast and specific, and it fixes one paper before that paper is marked at scale.
The flow is simple. A teacher elects to calibrate an assignment. Ren asks them to pick up to three submissions. The teacher marks those by hand, the normal way.
Ren then compares the AI and the teacher on every question in those submissions. For each question it examines the score, the comment, the tags, and the component breakdown. From those differences it proposes corrections to the paper's answer scheme.
A proposal can change the marking criteria for a question, the total points, the marking notes, the essay grading guide, or the component definitions. Each proposal carries a rationale and the evidence behind it.
The teacher approves every change
This is the part that matters most, so we will state it plainly.
The calibration analysis never edits the live answer scheme. It writes proposals. Every proposal starts as pending. The teacher opens each one, reads the rationale, and approves or rejects it. Only then does the scheme change, and only then does the rest of the batch get marked.
An assignment moves through explicit states: awaiting, marking, analysing, review, released. A teacher can also skip calibration entirely. The state is visible, so nobody has to guess whether the scheme they are looking at is the one the AI will use.
A run that produces no proposals still stops at review. Ren shows a "nothing to change — release?" state. Zero proposals is a result, and the teacher should see it and confirm it.
What happens when the analysis fails
One design decision here is worth describing, because it shows the standard we hold.
A calibration run makes several independent AI calls, one per question. Some can fail. If every call fails, the run would end with zero proposals — and the review screen would say "nothing to change".
That message would be a lie. It reads as "your marking matched the AI", when the truth is "we never compared them".
So a run that suffers any analysis failure does not advance. It fails the job instead, rolls back, and retries. A teacher never sees a clean result that came from a broken run.
Which rule wins
A single mark can be governed by four layers of instruction. Ren applies them in a fixed order, and the order is deliberate:
- Subject instructions — the broadest layer, shared across the subject.
- The teacher style guide — this teacher's learned preferences.
- Additional instructions — free text the teacher wrote for their own marking.
- Marking instructions for this paper — rules the teacher set for this specific assignment.
The paper-level instructions come last, so they outrank everything above them. A teacher who writes "on this paper, accept either notation" should not be overruled by a general preference they set three months ago.
Paper-wide marking instructions are a recent addition. A teacher writes them once, and the AI applies them to every question on that paper, alongside each question's own criteria.
The teacher stays the author
None of this makes the AI the authority. Read it in the other direction.
The style guide is built from a teacher's own edits. Calibration is built from a teacher's own marking, on scripts they chose, and every change waits for their approval. The paper instructions are typed by the teacher.
Ren's job is to apply that standard consistently across 700 scripts without becoming tired at script 400. The standard itself stays where it belongs.
We explain why we hold this line in Why AI Marking Still Needs Human Oversight. We also tested whether the AI knows when it is wrong. Those results are in Confidence Scores Are Not a Safety Net.