Shuffle Isn't Random

Medium

· Frontend

Incident

SEV-3

Time to fix: 20 min

Failing checks on the shipped build

  • rng pinned high keeps the order (j === i each step)

    expected [1,2,3,4] but got [1,3,4,2]

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the value the screen shows back through the state and the pure helpers that compute it; the view is usually only the messenger.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Spotify

Inspired by Spotify's famous shuffle complaints — and the classic broken shuffle behind them. A proper Fisher–Yates shuffle draws index j uniformly from 0..i at each step; the shipped version draws from the whole array every time, biasing the result.

The RNG harness is locked and deterministic, so the correct permutations are exactly checkable.

Code map

// Fisher–Yates shuffle over a copy of the queue.
// rng() returns a float in [0, 1).
exports.shuffle = function (queue, rng) {
  var arr = queue.slice();
  for (var i = arr.length - 1; i > 0; i--) {
    var j = Math.floor(rng() * arr.length);
    var t = arr[i];
    arr[i] = arr[j];
    arr[j] = t;
  }
  return arr;
};

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

Shuffle Isn't Random — JavaScript Bug Hunt

Inspired by Spotify's famous shuffle complaints — and the classic broken shuffle behind them.

  • Language: JavaScript
  • Layer: Frontend
  • Difficulty: Medium
  • Concepts: Algorithms, Randomness
  • Modelled on: Spotify
  • Visible tests: rng pinned high keeps the order (j === i each step); rng pinned low rotates deterministically; every track is preserved
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-3
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • rng pinned high keeps the order (j === i each step) — expected [1,2,3,4] but got [1,3,4,2]

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the value the screen shows back through the state and the pure helpers that compute it; the view is usually only the messenger.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Inspired by Spotify's famous shuffle complaints — and the classic broken shuffle behind them. A proper Fisher–Yates shuffle draws index j uniformly from 0..i at each step; the shipped version draws from the whole array every time, biasing the result.

The RNG harness is locked and deterministic, so the correct permutations are exactly checkable.

Bug report

BUG-4096 · Reported by: playback QA

With the seeded RNG that always returns 0.99, a correct Fisher–Yates pass must leave the queue in its original order (j always equals i). Ours reorders it.

Statistical audit: the naive draw over the full length makes some permutations ~30% more likely than others.

Logs

[shuffle] rng=const(0.99) input=[1,2,3,4] -> [1,3,4,2] (expected [1,2,3,4])

The code as shipped

src/player/shuffle.js (editable)

// Fisher–Yates shuffle over a copy of the queue.
// rng() returns a float in [0, 1).
exports.shuffle = function (queue, rng) {
  var arr = queue.slice();
  for (var i = arr.length - 1; i > 0; i--) {
    var j = Math.floor(rng() * arr.length);
    var t = arr[i];
    arr[i] = arr[j];
    arr[j] = t;
  }
  return arr;
};

Read-only context: src/player/rng.js.

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More JavaScript bug hunts.