Debts Averaged Out of Thin Air

Medium

· Backend

Incident

SEV-1

BUG-OCI

Priority: Critical · reported by ombudsman referral

Time to fix: 20 min

Failing checks on the shipped build

  • uneven earnings raise no debt when every fortnight was right

    averaging invents a debt here: expected {"debt": 0, "review": false} but got {"debt": 4550, "review": false}

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Robodebt (Australia) · 2016–2019

Modelled on Robodebt, the Australian government's automated welfare debt-recovery scheme (2016–2019): it averaged a person's annual income from tax records evenly across the year's fortnights and compared that average with the income they had reported fortnight by fortnight. Anyone whose earnings were uneven — casual or seasonal work — could be issued a debt they did not owe. In 2019 the Federal Court found a debt based on averaging unlawful, and a Royal Commission reported on the scheme in 2023.

This project is a reconstruction. assess.py computes a debt by spreading the annual figure across 26 fortnights.

Fix assess so each fortnight is judged on that fortnight's own income, and a mismatch with the annual figure is flagged for review — never turned into a debt.

Code map

rules = bug_require("./rules.py")


def assess(annual_income, fortnights):
    fortnightly = annual_income // rules.FORTNIGHTS_PER_YEAR
    debt = 0
    for f in fortnights:
        owed = f["paid"] - rules.entitlement(fortnightly)
        if owed > 0:
            debt += owed
    used_total = fortnightly * len(fortnights)
    review = abs(annual_income - used_total) > rules.REVIEW_TOLERANCE
    return {"debt": debt, "review": review}

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

Debts Averaged Out of Thin Air — Python Bug Hunt

Modelled on Robodebt, the Australian government's automated welfare debt-recovery scheme (2016–2019): it averaged a person's annual income from tax records…

  • Language: Python
  • Layer: Backend
  • Difficulty: Medium
  • Concepts: Money, Validation
  • Modelled on: Robodebt (Australia) · 2016–2019
  • Visible tests: steady earnings: a real over-payment is found; uneven earnings raise no debt when every fortnight was right
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-1 (priority: Critical)
  • Ticket: BUG-OCI
  • Reported by: ombudsman referral
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • uneven earnings raise no debt when every fortnight was right — averaging invents a debt here: expected {"debt": 0, "review": false} but got {"debt": 4550, "review": false}

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Modelled on Robodebt, the Australian government's automated welfare debt-recovery scheme (2016–2019): it averaged a person's annual income from tax records evenly across the year's fortnights and compared that average with the income they had reported fortnight by fortnight. Anyone whose earnings were uneven — casual or seasonal work — could be issued a debt they did not owe. In 2019 the Federal Court found a debt based on averaging unlawful, and a Royal Commission reported on the scheme in 2023.

This project is a reconstruction. assess.py computes a debt by spreading the annual figure across 26 fortnights.

Fix assess so each fortnight is judged on that fortnight's own income, and a mismatch with the annual figure is flagged for review — never turned into a debt.

Bug report

BUG-OCI · Priority: Critical · Reported by: ombudsman referral

assess(annual_income, fortnights) — each fortnight is {"reported": int, "actual": int or None, "paid": int}.

  • the income for a fortnight is "actual" (payslip evidence) when it is not None, otherwise the "reported" figure; the annual figure is NEVER divided across fortnights
  • debt = sum over fortnights of max(0, paid - rules.entitlement(income))
  • review = abs(annual_income - sum of the incomes used) > rules.REVIEW_TOLERANCE

Returns {"debt": int, "review": bool}.

Observed: a casual worker who earned everything in half the year and claimed benefits correctly in the other half was issued a debt of 4550.

Logs

[oci] client 88231 annual=26000 averaged=1000/fn
[oci] client 88231 raised debt=4550 (13 fortnights over-paid)

The code as shipped

src/compliance/assess.py (editable)

rules = bug_require("./rules.py")


def assess(annual_income, fortnights):
    fortnightly = annual_income // rules.FORTNIGHTS_PER_YEAR
    debt = 0
    for f in fortnights:
        owed = f["paid"] - rules.entitlement(fortnightly)
        if owed > 0:
            debt += owed
    used_total = fortnightly * len(fortnights)
    review = abs(annual_income - used_total) > rules.REVIEW_TOLERANCE
    return {"debt": debt, "review": review}

Read-only context: src/compliance/rules.py.

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Python bug hunts.