The February 29th Certificate

Medium

· Backend

Incident

SEV-1

BUG-0229-AZ

Priority: Critical · reported by platform security

Time to fix: 20 min

Failing checks on the shipped build

  • leap day clamps to February 28th

    expected [2013, 2, 28] but got [2013, 2, 29]

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Microsoft Azure · 2012

Inspired by Azure's 2012 leap-day outage: certificates minted on February 29th were given a validity of "same date next year" — a date that does not exist. Cert creation failed, and the failure cascaded into a global outage.

CertIssuer.java adds one year to an issue date. Naively.

Code map

class CertIssuer {
    static boolean isLeap(int year) {
        return year % 4 == 0 && (year % 100 != 0 || year % 400 == 0);
    }

    static int[] expiryDate(int year, int month, int day) {
        return new int[] { year + 1, month, day };
    }
}

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

The February 29th Certificate — Java Bug Hunt

Inspired by Azure's 2012 leap-day outage: certificates minted on February 29th were given a validity of "same date next year" — a date that does not exist.

  • Language: Java
  • Layer: Backend
  • Difficulty: Medium
  • Concepts: Dates, Leap Years
  • Modelled on: Microsoft Azure · 2012
  • Visible tests: ordinary dates keep month and day; leap day clamps to February 28th
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-1 (priority: Critical)
  • Ticket: BUG-0229-AZ
  • Reported by: platform security
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • leap day clamps to February 28th — expected [2013, 2, 28] but got [2013, 2, 29]

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Inspired by Azure's 2012 leap-day outage: certificates minted on February 29th were given a validity of "same date next year" — a date that does not exist. Cert creation failed, and the failure cascaded into a global outage.

CertIssuer.java adds one year to an issue date. Naively.

Bug report

BUG-0229-AZ · Priority: Critical · Reported by: platform security

expiryDate(year, month, day) = issue date + 1 year:

  • Feb 29 -> Feb 28 of the next year (clamp; next year is never leap+1)
  • all other dates keep month/day

Observed: [2012, 2, 29] -> [2013, 2, 29], which downstream validation rejects, which takes cert issuance down entirely.

Logs

[pki] mint 2012-02-29 -> expiry 2013-02-29 INVALID DATE
[pki] issuance pipeline halted

The code as shipped

CertIssuer.java (editable)

class CertIssuer {
    static boolean isLeap(int year) {
        return year % 4 == 0 && (year % 100 != 0 || year % 400 == 0);
    }

    static int[] expiryDate(int year, int month, int day) {
        return new int[] { year + 1, month, day };
    }
}

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Java bug hunts.