The Feature File That Doubled Overnight — JavaScript Bug Hunt

Briefing

Modelled on the Cloudflare outage of 18 November 2025: a change to database permissions made a metadata query — one that did not filter by database name — return every column twice. The Bot Management feature file generated from it doubled in size, went past the proxy's hard limit of 200 features, and the proxy code that loaded it failed with an unhandled error instead of rejecting the file. Cloudflare's network served 5xx errors for hours.

In this reconstruction, loader.js turns the metadata rows into the feature list the proxy runs on. It takes every row at face value and treats an oversize file as fatal.

Fix loadFeatureFile so duplicate rows collapse and a bad file is refused while the last good configuration keeps serving.

Bug report

BUG-BOTFEAT · Priority: Critical (global 5xx) · Reported by: edge SRE

loadFeatureFile(rows, live) builds the feature config from metadata rows ({ database, name, type }). live is the config currently serving ({ features: [...] }) or null.

• features are the DISTINCT row names, in first-seen order — the same column reported by two databases is one feature
• if the distinct count is within limits.MAX_FEATURES (200, inclusive), return { features: <list>, status: "loaded" }
• if it exceeds the limit the file is bad. Never throw: return { features: live.features, status: "kept-last-good" }, or { features: [], status: "rejected" } when there is no live config

Observed: after the permissions change every row came back twice, the feature count doubled, and the loader threw — taking the proxy down.

Logs

[featgen] wrote bot feature file: 2 x rows (default, r0)
[proxy] loadFeatureFile: Error: feature count 240 exceeds limit 200
[proxy] worker panicked; serving 5xx

The code as shipped

src/bots/loader.js (editable)

var limits = require("./limits");

// Builds the bot-management feature config from column metadata rows.
exports.loadFeatureFile = function (rows, live) {
  var features = rows.map(function (row) { return row.name; });
  if (features.length > limits.MAX_FEATURES) {
    throw new Error("feature count " + features.length + " exceeds limit " + limits.MAX_FEATURES);
  }
  return { features: features, status: "loaded" };
};

Read-only context: src/bots/limits.js, src/bots/QUERY.js.