Two Spikes, 1.2 Seconds Apart — JavaScript Bug Hunt
Modelled on Qantas Flight 72 (7 October 2008), an Airbus A330 flying from Singapore to Perth that made two sudden, uncommanded pitch-down manoeuvres,…
- Language: JavaScript
- Layer: Backend
- Difficulty: Hard
- Concepts: Sensors, State, Time
- Modelled on: Qantas Flight 72 · 2008
- Visible tests: clean data passes straight through; a single spike is held out; two spikes exactly one hold apart never reach the control laws
- Reward: 50 XP for a complete fix
Incident
- Severity: SEV-1 (priority: Critical (flight controls))
- Ticket: BUG-QF72
- Reported by: flight test
- Time to fix: 35 minutes, the target for a hard incident
Failing checks on the shipped build
- two spikes exactly one hold apart never reach the control laws — a spike got through: expected -1 but got 22
How to work this incident
- Read the report: write down the expected behaviour and the actual one in a sentence each.
- Reproduce it: run the visible tests and find the failing check that matches the report.
- Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
- Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
- Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.
Briefing
Modelled on Qantas Flight 72 (7 October 2008), an Airbus A330 flying from Singapore to Perth that made two sudden, uncommanded pitch-down manoeuvres, injuring many of those on board; it diverted to Learmonth. The Australian Transport Safety Bureau found that one air data inertial reference unit had begun emitting intermittent data spikes, including in angle of attack. The flight control computers' algorithm was designed to hold the last good value for 1.2 seconds when it detected a spike — but in a very specific pattern, a second spike arriving 1.2 seconds after the first, a spike got through and the computers commanded nose-down.
aoaFilter.js is a reconstruction of that spike filter.
Fix sample so a hold never lets an unchecked value through.
Bug report
BUG-QF72 · Priority: Critical (flight controls) · Reported by: flight test
createFilter({ initial, threshold, holdMs }).sample(t, value) returns the angle of attack the control laws use; t is in ms and strictly increasing.
- not holding, |value - lastGood| <= threshold: accept it (lastGood = value) and return it
- not holding, deviation > threshold: this is a spike — start a hold at t and return lastGood
- holding and t - holdStart < holdMs: return lastGood; the value is ignored
- the first sample with t - holdStart >= holdMs ends the hold and is then judged exactly like a sample that is not holding (so a spike at that very instant starts a new hold)
Nothing above threshold away from lastGood is ever returned.
Observed: spikes at t = 1000 and t = 2200 with holdMs = 1200 — the second spike is returned as the angle of attack, and protection commands nose-down.
Logs
[adiru-1] AOA spike 50.6 deg at t=1000 -> hold
[fcpc] AOA used=50.6 deg at t=2200
[fcpc] high-AOA protection: PITCH DOWNThe code as shipped
src/fcpc/aoaFilter.js (editable)
// Rejects short spikes in the angle-of-attack signal by holding the last
// good value for holdMs.
exports.createFilter = function (opts) {
var lastGood = opts.initial;
var holdStart = null;
return {
sample: function (t, value) {
if (holdStart !== null) {
if (t - holdStart < opts.holdMs) return lastGood;
holdStart = null;
lastGood = value;
return value;
}
if (Math.abs(value - lastGood) > opts.threshold) {
holdStart = t;
return lastGood;
}
lastGood = value;
return value;
}
};
};
Read-only context: src/fcpc/protection.js.
Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More JavaScript bug hunts.