After a quiz, investigate the question that stronger scorers missed

If learners who scored highly on the rest of a quiz mostly miss one question, while lower scorers mostly choose the keyed answer, investigate the question before drawing conclusions. Check the answer key, wording, alternatives and access arrangements. The pattern may reveal a problem, but it may also reflect the small group or the knowledge being tested.

Here is a practical way to review the pattern without assuming that either the question or the learners are at fault.

Start with the response pattern

Imagine 20 learners complete a quiz. An educator ranks them using their scores on all the other questions, leaving the question under review out of the ranking. They compare the five highest and five lowest scorers on the rest of the quiz.

For this fictional question, option A is the keyed answer:

The higher-scoring group selected the key less often: one out of five, compared with four out of five. The simple difference between those proportions is 20% minus 80%, or −0.60. This is a descriptive comparison of two small groups, not a point-biserial correlation or proof that the question is defective. We do not know from this table how the other ten learners responded.

This reversal is related to what assessment teams call negative item discrimination: learners doing better on the wider test are less likely to receive credit for the item. The University of Washington’s item-analysis guidance recommends examining negative values, including checking for a mis-keyed answer. A five-person group is especially sensitive to individual responses: each learner changes its proportion by 20 percentage points. ([washington.edu](https://www.washington.edu/assessment/scanning-scoring/scoring/reports/item-analysis/))

Check what the learners actually saw

Before interpreting the numbers, preserve the question and answer options as presented, the published key and the response data. Check which attempt and version you are reviewing. If scores on the other questions are tied at a group boundary, agree and record a consistent way to handle the tie rather than choosing learners to produce a particular pattern.

Where available, look at responses from all 20 learners, including omissions. Did a particular alternative attract many responses? Were there reports of a display or access problem? A pattern across answer options can suggest where to look, but it cannot establish which answer is correct. The NBME’s item-analysis guide describes reviewing answer options as well as discrimination. ([nbme.org](https://www.nbme.org/sites/default/files/2021-02/NBME_Item%20Writing%20Guide_R_6.pdf?utm_source=openai))

Work through four checks

1. Verify the key independently

Ask a subject specialist to answer the question without looking at the stored key, then compare their reasoning with the intended learning objective and any source material. Check that the answer entered in the quiz matches the approved answer and the version learners saw.

In our fictional quiz, the question says: “A £200 fee rises by 15%. What is the increase?” Option A is £230 and option B is £30. The increase is £30; £230 is the new fee. If A was entered as the key, the response pattern is consistent with a keying error. The arithmetic and the question wording—not the pattern alone—establish why B should have been the intended answer.

2. Read the wording as a learner might

Check whether the question asks for the increase or the new total, whether essential information is missing, and whether a negative word such as “not” is easy to overlook. Could a reasonable reader understand the task in more than one way? Compare the wording with the skill the item is meant to assess rather than rewriting it merely to make it easier.

3. Test every alternative

Work through each answer, not just the key. Is there another defensible answer under the stated conditions? Is an option correct only if a learner supplies an unstated assumption? In the fee example, £230 is a plausible answer to a different question: the new total. That distinction is useful when deciding whether the key, the wording or both need attention.

4. Check access and delivery

Review the actual presentation: for example, whether all options displayed, a table or image was readable, and any required alternative format conveyed the same information. Consider whether timing or a reported technical issue could have affected access. Do not infer an individual learner’s needs from their score. Ofqual’s guidance for awarding organisations discusses avoiding unnecessary barriers while retaining what the assessment is intended to measure. ([gov.uk](https://www.gov.uk/guidance/ofqual-handbook/section-d-general-requirements-for-regulated-qualifications?utm_source=openai))

Decide what happens next

Record the evidence, the reviewer’s reasoning and the decision. The next step depends on what the review finds:

• Key error confirmed: escalate it and identify everyone whose result may be affected, not just the ten learners in the comparison. In the example, the team would check the consequences of A being keyed instead of B.
• Wording or access issue found: document the issue and seek the appropriate review of the current attempt. Draft and check a revised version for future use.
• No defect established: retain or monitor the question, and consider whether the responses point to content worth revisiting in teaching. A small sample cannot settle the question on its own.

For assessed work, do not independently change marks, remove an item or alter released results simply because the pattern looks unusual. Follow local and, where applicable, awarding-body procedures for approval, recalculation, communication and record keeping. Changes for future use are a separate decision from any remedy for learners who have already taken the quiz.

A short review record

Use this prompt in your team’s existing process:

Question and version: …
Observed pattern and group sizes: …
Key and subject check: …
Wording, alternatives and access checks: …
Decision, reviewer and date: …
Action for this attempt and for future use: …

EduMark Studio is a workspace for reviewing learner work alongside criteria and feedback. It is not presented here as an item-analysis tool: use your available quiz response data for the comparison, and apply human judgement to the review. ([edumark.ai](https://www.edumark.ai/marking-studio?utm_source=openai))