The Opening Cross That Never Closed — JavaScript Bug Hunt
Modelled on the Nasdaq Facebook IPO (18 May 2012): the opening cross recalculated the opening price, but orders cancelled during the calculation were fed…
- Language: JavaScript
- Layer: Backend
- Difficulty: Hard
- Concepts: Concurrency, Convergence
- Modelled on: Nasdaq · Facebook IPO 2012
- Visible tests: an empty book settles immediately; a book with cancellations converges quickly
- Reward: 50 XP for a complete fix
Incident
- Severity: SEV-1 (priority: Critical)
- Ticket: BUG-FBIPO
- Reported by: market operations
- Time to fix: 35 minutes, the target for a hard incident
Failing checks on the shipped build
- an empty book settles immediately — Killed
- a book with cancellations converges quickly — Killed
How to work this incident
- Read the report: write down the expected behaviour and the actual one in a sentence each.
- Reproduce it: run the visible tests and find the failing check that matches the report.
- Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
- Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
- Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.
Briefing
Modelled on the Nasdaq Facebook IPO (18 May 2012): the opening cross recalculated the opening price, but orders cancelled during the calculation were fed back in and forced another pass. The loop never converged, the IPO opened 30 minutes late, and confirmations were delayed for hours.
cross.js recomputes until nothing changed, but it re-queues the very events it generates.
Fix runCross so it converges: only events that arrived before the pass began may trigger another one, and the loop is bounded.
Bug report
BUG-FBIPO · Priority: Critical · Reported by: market operations
runCross(book) must:
- repeatedly recompute the price while pending cancellations remain
- process each pass against a SNAPSHOT of the pending list taken at the start of that pass
- return { price, passes } — passes must be small and bounded
Observed: each pass appends its own bookkeeping event back onto the pending list, so the loop runs until the 100000-iteration guard trips.
Logs
[cross] pass 99998 pending=1 price=42
[cross] convergence guard tripped; opening delayedThe code as shipped
src/market/cross.js (editable)
// Recomputes the opening price until the book settles.
exports.runCross = function (book) {
var passes = 0;
var price = book.reference;
while (book.pending.length > 0) {
passes++;
if (passes > 100000) return { price: price, passes: passes };
for (var i = 0; i < book.pending.length; i++) {
price = price + book.pending[i].delta;
// Every processed cancellation logs a re-quote back into the queue.
book.pending.push({ delta: 0, requote: true });
}
book.pending = book.pending.slice(1);
}
return { price: price, passes: passes };
};
Read-only context: src/market/RULES.js.
Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More JavaScript bug hunts.