The Filter That Held Back the Flood — JavaScript Bug Hunt

Briefing

Modelled on the Rogers outage of 8 July 2022: a maintenance step removed a routing filter from the core network. Without it the routers received far more routing information than they could process, the IP core collapsed, and a third of Canada's internet — including 911 service — was down for around 15 hours.

distribution.js distributes routes to the core and treats an empty filter list as "allow everything".

Fix distribute so an empty filter list fails closed, and so a batch that exceeds the core's processing budget is refused.

Bug report

BUG-ROGERS0708 · Priority: Critical · Reported by: IP core

distribute(routes, filters, budget) must return { sent, blocked }:

• an EMPTY filters array means nothing is explicitly permitted, so nothing is sent — fail closed, not open
• otherwise send only routes whose prefix starts with one of the filters
• if the number of routes that would be sent exceeds budget, send none and report them all as blocked

Observed: removing the last filter turns the guard into a pass-through and the entire global table is pushed at the core.

Logs

[core] filters=[] sent=1043221 routes
[core] control plane unresponsive across all regions

The code as shipped

src/net/distribution.js (editable)

// Distributes routes into the core network.
exports.distribute = function (routes, filters, budget) {
  var sent = [];
  var blocked = [];
  for (var i = 0; i < routes.length; i++) {
    var allowed = filters.length === 0;
    for (var j = 0; j < filters.length; j++) {
      if (routes[i].indexOf(filters[j]) === 0) allowed = true;
    }
    if (allowed) sent.push(routes[i]);
    else blocked.push(routes[i]);
  }
  return { sent: sent, blocked: blocked };
};

Read-only context: src/net/GUARD.js.