The Gift Card Paid Twice

Hard

· Database

Incident

SEV-1

BUG-SBUX-0515

Priority: Critical (money creation) · reported by responsible disclosure

Time to fix: 35 min

Failing checks on the shipped build

  • two concurrent full-balance transfers pay out once

    only one may succeed: expected ["insufficient","ok"] but got ["ok","ok"]

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the data through the query or the repository code; compare the rows it returns with the rows the report expects.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Starbucks · 2015

Modelled on Egor Homakov's 2015 Starbucks report: he found that transferring balance between Starbucks gift cards was vulnerable to a race condition — by sending the same transfer request several times at once, the balance was credited more than once while the source card was debited only once, creating money that never existed.

transfer.js builds a transfer as a sequence of steps; scheduler.js runs steps from several transfers in a chosen interleaving, each step atomically. The transfer reads the balance in one step, credits in the next and writes the debit back in a third — so two transfers can both spend the same balance.

Fix makeTransfer so the balance check and the debit happen together, before anything is credited.

Code map

// Moves `amount` from one gift card to another as a sequence of steps.
exports.makeTransfer = function (store, from, to, amount) {
  var t = { steps: [], pc: 0, result: null };
  var balance = 0;
  t.steps.push(function () {
    balance = store.get(from);
  });
  t.steps.push(function () {
    if (balance < amount) {
      t.result = "insufficient";
      return;
    }
    store.set(to, store.get(to) + amount);
  });
  t.steps.push(function () {
    store.set(from, balance - amount);
    t.result = "ok";
  });
  return t;
};

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

The Gift Card Paid Twice — JavaScript Bug Hunt

Modelled on Egor Homakov's 2015 Starbucks report: he found that transferring balance between Starbucks gift cards was vulnerable to a race condition — by…

  • Language: JavaScript
  • Layer: Database
  • Difficulty: Hard
  • Concepts: Concurrency, Money
  • Modelled on: Starbucks · 2015
  • Visible tests: a single transfer moves the money; two concurrent full-balance transfers pay out once
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-1 (priority: Critical (money creation))
  • Ticket: BUG-SBUX-0515
  • Reported by: responsible disclosure
  • Time to fix: 35 minutes, the target for a hard incident

Failing checks on the shipped build

  • two concurrent full-balance transfers pay out once — only one may succeed: expected ["insufficient","ok"] but got ["ok","ok"]

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the data through the query or the repository code; compare the rows it returns with the rows the report expects.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Modelled on Egor Homakov's 2015 Starbucks report: he found that transferring balance between Starbucks gift cards was vulnerable to a race condition — by sending the same transfer request several times at once, the balance was credited more than once while the source card was debited only once, creating money that never existed.

transfer.js builds a transfer as a sequence of steps; scheduler.js runs steps from several transfers in a chosen interleaving, each step atomically. The transfer reads the balance in one step, credits in the next and writes the debit back in a third — so two transfers can both spend the same balance.

Fix makeTransfer so the balance check and the debit happen together, before anything is credited.

Bug report

BUG-SBUX-0515 · Priority: Critical (money creation) · Reported by: responsible disclosure

makeTransfer(store, from, to, amount) returns { steps, pc: 0, result: null }. scheduler.run(transfers, order) runs each step atomically, in any interleaving.

Whatever the interleaving:

  • a transfer succeeds (result "ok") only if the source card held at least amount at the moment it was debited; the destination is credited exactly amount
  • otherwise result is "insufficient" and neither card changes
  • money is conserved: the sum of all balances never changes
  • a card's balance never goes below zero

store.debitIfAtLeast(card, amount) is an atomic check-and-debit.

Observed: two simultaneous transfers of a card's full balance both report "ok" and the destination receives it twice.

Logs

[cards] transfer 7713->2204 amount=10.00 ok
[cards] transfer 7713->2204 amount=10.00 ok
[ledger] reconciliation: card 2204 balance 20.00 exceeds funded total 10.00

The code as shipped

src/cards/transfer.js (editable)

// Moves `amount` from one gift card to another as a sequence of steps.
exports.makeTransfer = function (store, from, to, amount) {
  var t = { steps: [], pc: 0, result: null };
  var balance = 0;
  t.steps.push(function () {
    balance = store.get(from);
  });
  t.steps.push(function () {
    if (balance < amount) {
      t.result = "insufficient";
      return;
    }
    store.set(to, store.get(to) + amount);
  });
  t.steps.push(function () {
    store.set(from, balance - amount);
    t.result = "ok";
  });
  return t;
};

Read-only context: src/cards/scheduler.js, src/cards/store.js.

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More JavaScript bug hunts.