Schrödinger's Follower Count

Medium

· Backend

Incident

SEV-3

Time to fix: 20 min

Failing checks on the shipped build

  • lagging replicas never lower the number

    monotonic display: expected 10204 but got 10201

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Social counters

Inspired by the follower counters of large social networks, which read from whichever replica answers first — so refreshing your profile makes the number bounce down and back up. Displayed counters should be monotonic: never show fewer than the user has already seen.

counter.js picks what to display.

Code map

// Chooses the follower count to display.
exports.nextDisplay = function (previousShown, freshReads) {
  return freshReads[freshReads.length - 1];
};

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

Schrödinger's Follower Count — JavaScript Bug Hunt

Inspired by the follower counters of large social networks, which read from whichever replica answers first — so refreshing your profile makes the number…

  • Language: JavaScript
  • Layer: Backend
  • Difficulty: Medium
  • Concepts: Consistency, Caching
  • Modelled on: Social counters
  • Visible tests: lagging replicas never lower the number; genuinely higher counts show
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-3
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • lagging replicas never lower the number — monotonic display: expected 10204 but got 10201

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Inspired by the follower counters of large social networks, which read from whichever replica answers first — so refreshing your profile makes the number bounce down and back up. Displayed counters should be monotonic: never show fewer than the user has already seen.

counter.js picks what to display.

Bug report

BUG-BOUNCE · Reported by: creator support (screenshots of "losing" followers)

nextDisplay(previousShown, freshReads):

  • take the max of the fresh replica reads
  • never display LESS than previousShown (replicas lag; going down is noise)
  • returns the new displayed value

Observed: the counter renders whatever the slowest replica said.

Logs

[profile] shown 10,204 -> refresh -> 10,197 -> refresh -> 10,205

The code as shipped

src/social/counter.js (editable)

// Chooses the follower count to display.
exports.nextDisplay = function (previousShown, freshReads) {
  return freshReads[freshReads.length - 1];
};

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More JavaScript bug hunts.