The Player That Wore Out Disks — JavaScript Bug Hunt

Briefing

Modelled on the Spotify desktop client disk-write reports of November 2016: users noticed the app writing tens to hundreds of gigabytes a day to disk even while idle — a real concern for SSD lifespan. The writes were traced to the client's local database being rewritten over and over; Spotify shipped a fix in an update later that month.

store.js is a reconstruction: a small persistent key/value cache that rewrites its entire snapshot file on every change. The file system is injected (fakefs.js counts full rewrites and appends), so you can see exactly what it does.

Rewrite store.js to log changes by appending and only compact into a snapshot after enough changes have accumulated, exactly as the spec describes.

Bug report

BUG-SSD-WRITES · Priority: High · Reported by: community forum

open(fs) -> { get, set }. Paths and limits are in config.js.

Loading: state = JSON.parse(fs.readFile(SNAPSHOT)) if it exists (else {}), then every non-empty line of fs.readFile(LOG) (if any) is a JSON [key, value] applied in order. The count of changes since the last compaction starts at the number of log lines replayed.

get(key) -> the value, or null if never set.

set(key, value) (values are strings, numbers or booleans):

• if state already holds exactly that value (===) -> no I/O at all
• otherwise update state and fs.appendFile(LOG, JSON.stringify([key, value]) + "\n") and count one change
• when the count reaches COMPACT_AFTER, compact: fs.writeFile(SNAPSHOT, JSON.stringify(state)) then fs.writeFile(LOG, "") and reset the count to 0

So 50 changes = 50 appends and 0 rewrites; 100 changes = 100 appends and 2 rewrites (snapshot + log truncate); 250 changes = 4 rewrites.

Observed: every single set rewrites the full snapshot.

Logs

[storage] state.json rewritten (4.1 MB) reason=set
[storage] state.json rewritten (4.1 MB) reason=set
[disk] 81 GB written in the last 24h by the desktop client

The code as shipped

src/cache/store.js (editable)

var config = require("./config");

exports.open = function (fs) {
  var state = {};
  var snap = fs.readFile(config.SNAPSHOT);
  if (snap) state = JSON.parse(snap);

  function persist() {
    fs.writeFile(config.SNAPSHOT, JSON.stringify(state));
  }

  return {
    get: function (key) {
      return Object.prototype.hasOwnProperty.call(state, key) ? state[key] : null;
    },
    set: function (key, value) {
      state[key] = value;
      persist();
    }
  };
};

Read-only context: src/cache/config.js, src/cache/fakefs.js.