The Final Invitation That Never Came

Medium

· Backend

Incident

SEV-1

BUG-NHSBSP-0518

Priority: Critical · reported by screening programme

Time to fix: 20 min

Failing checks on the shipped build

  • a 70-year-old gets her final invitation

    aged 70 is still in the programme: expected ["w4"] but got []

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

NHS England · 2018

Modelled on the NHS England breast screening failure announced in May 2018: the Health Secretary told Parliament that a computer algorithm failure dating back to 2009 meant an estimated 450,000 women aged 68 to 71 had not been invited to their final breast screening. (A later independent review found the causes were more tangled than a single coding error.)

This project is a reconstruction of the simplest form of that failure — an age-band boundary. screening.py decides who is in the programme with an exclusive upper bound on completed years of age, so the oldest women in the programme silently drop out of the invitation run.

Fix in_programme so the band matches the policy exactly.

Code map

dates = bug_require("./dates.py")
policy = bug_require("./policy.py")


def in_programme(dob, today):
    age = dates.age_years(dob, today)
    return policy.FIRST_AGE <= age < policy.LAST_AGE


def select_for_invite(women, today_iso):
    today = dates.parse(today_iso)
    chosen = []
    for w in women:
        if not in_programme(dates.parse(w["dob"]), today):
            continue
        last = w.get("last")
        if last is not None and today < dates.add_years(dates.parse(last), policy.INTERVAL_YEARS):
            continue
        chosen.append(w["id"])
    return sorted(chosen)

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

The Final Invitation That Never Came — Python Bug Hunt

Modelled on the NHS England breast screening failure announced in May 2018: the Health Secretary told Parliament that a computer algorithm failure dating…

  • Language: Python
  • Layer: Backend
  • Difficulty: Medium
  • Concepts: Time, Validation
  • Modelled on: NHS England · 2018
  • Visible tests: a routine run picks the due women in the band; a 70-year-old gets her final invitation
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-1 (priority: Critical)
  • Ticket: BUG-NHSBSP-0518
  • Reported by: screening programme
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • a 70-year-old gets her final invitation — aged 70 is still in the programme: expected ["w4"] but got []

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Modelled on the NHS England breast screening failure announced in May 2018: the Health Secretary told Parliament that a computer algorithm failure dating back to 2009 meant an estimated 450,000 women aged 68 to 71 had not been invited to their final breast screening. (A later independent review found the causes were more tangled than a single coding error.)

This project is a reconstruction of the simplest form of that failure — an age-band boundary. screening.py decides who is in the programme with an exclusive upper bound on completed years of age, so the oldest women in the programme silently drop out of the invitation run.

Fix in_programme so the band matches the policy exactly.

Bug report

BUG-NHSBSP-0518 · Priority: Critical · Reported by: screening programme

in_programme(dob, today) — both datetime.date — is True from the day of a woman's 50th birthday up to AND INCLUDING the day of her 71st birthday, and False outside that. Birthdays come from dates.add_years(dob, n) (29 February falls on 1 March in non-leap years).

select_for_invite(women, today_iso) returns the sorted ids of women who are in_programme on that day AND are due: never screened (last is None), or today >= add_years(last screening date, 3).

Observed: women aged 70, and women on their 71st birthday, are never selected.

Logs

[bso-batch] run 2018-05-01 selected=18,204 eligible_by_age=50..69
[audit] 0 invitations issued to women aged 70 since 2009

The code as shipped

src/screening/screening.py (editable)

dates = bug_require("./dates.py")
policy = bug_require("./policy.py")


def in_programme(dob, today):
    age = dates.age_years(dob, today)
    return policy.FIRST_AGE <= age < policy.LAST_AGE


def select_for_invite(women, today_iso):
    today = dates.parse(today_iso)
    chosen = []
    for w in women:
        if not in_programme(dates.parse(w["dob"]), today):
            continue
        last = w.get("last")
        if last is not None and today < dates.add_years(dates.parse(last), policy.INTERVAL_YEARS):
            continue
        chosen.append(w["id"])
    return sorted(chosen)

Read-only context: src/screening/dates.py, src/screening/policy.py.

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Python bug hunts.