The Extra Second That Pegged Every CPU — JavaScript Bug Hunt

Briefing

Modelled on the leap second of 30 June 2012: inserting a 61st second left kernel timers with a deadline that had already passed. The timer loop re-armed a task that was instantly due again, so it spun without progressing. Reddit, Mozilla, LinkedIn and Qantas all went down at the same instant.

scheduler.js re-arms timers by adding an interval to the deadline, which never catches up once the clock jumps backwards.

Fix runDueTimers so it always makes progress, even when the clock steps.

Bug report

BUG-LEAPSEC · Priority: Critical · Reported by: kernel team

runDueTimers(now, timers) must:

• fire every timer whose deadline is at or before now
• re-arm each fired timer strictly AFTER now
• return { fired, timers } and terminate in bounded time

Observed: when the clock steps backwards, a re-armed timer is immediately due again and the loop spins until the iteration guard trips.

Logs

[timer] iterations=1000000 fired=1000000 pending=1
[timer] all cores at 100%; ntp step -1s

The code as shipped

src/kernel/scheduler.js (editable)

// Fires every timer that is due, re-arming it for its next interval.
exports.runDueTimers = function (now, timers) {
  var fired = 0;
  var guard = 0;
  var due = true;
  while (due) {
    due = false;
    for (var i = 0; i < timers.length; i++) {
      if (timers[i].deadline <= now) {
        fired++;
        // Re-arm relative to the old deadline.
        timers[i].deadline = timers[i].deadline + timers[i].interval;
        due = true;
      }
    }
    guard++;
    if (guard > 1000000) break;
  }
  return { fired: fired, timers: timers };
};

Read-only context: src/kernel/TIMERS.js.