False Positives at the Signup Gate

Medium

· Frontend

Incident

SEV-2

BUG-2266

Priority: High · reported by growth team (signup conversion -12%)

Time to fix: 20 min

Failing checks on the shipped build

  • age exactly at the minimum is accepted

    18 satisfies min:18 — got [{"field":"age","message":"age must be at least 18"}]: expected 0 but got 1

  • zero satisfies a required field

    0 is present, required must pass — got [{"field":"frequency","message":"frequency is required"}]: expected 0 but got 1

  • an email without a domain dot is rejected

    expected 1 but got 0

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the value the screen shows back through the state and the pure helpers that compute it; the view is usually only the messenger.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Signup-form lore

The signup form on Northwind Cloud rejects perfectly valid users and lets some invalid ones through. The rule schema and error messages are locked; the interpreter in validate.js is yours to fix.

Three symptoms, one file.

Code map

// Interprets the rule schema against submitted values.
// Returns an array of { field, message } errors (empty = valid).
var messages = require("./messages");

var EMAIL_RE = /\S+@\S+/;

exports.validate = function (rules, values) {
  var errors = [];

  for (var field in rules) {
    var rule = rules[field];
    var value = values[field];

    if (rule.required && !value) {
      errors.push({ field: field, message: messages.required(field) });
      continue;
    }

    if (rule.email && value !== undefined && !EMAIL_RE.test(value)) {
      errors.push({ field: field, message: messages.email(field) });
    }

    if (rule.min !== undefined && typeof value === "number" && value <= rule.min) {
      errors.push({ field: field, message: messages.min(field, rule.min) });
    }
  }

  return errors;
};

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

False Positives at the Signup Gate — JavaScript Bug Hunt

The signup form on Northwind Cloud rejects perfectly valid users and lets some invalid ones through.

  • Language: JavaScript
  • Layer: Frontend
  • Difficulty: Medium
  • Modelled on: Signup-form lore
  • Visible tests: a fully valid submission has no errors; age exactly at the minimum is accepted; zero satisfies a required field; an email without a domain dot is rejected
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-2 (priority: High)
  • Ticket: BUG-2266
  • Reported by: growth team (signup conversion -12%)
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • age exactly at the minimum is accepted — 18 satisfies min:18 — got [{"field":"age","message":"age must be at least 18"}]: expected 0 but got 1
  • zero satisfies a required field — 0 is present, required must pass — got [{"field":"frequency","message":"frequency is required"}]: expected 0 but got 1
  • an email without a domain dot is rejected — expected 1 but got 0

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the value the screen shows back through the state and the pure helpers that compute it; the view is usually only the messenger.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

The signup form on Northwind Cloud rejects perfectly valid users and lets some invalid ones through. The rule schema and error messages are locked; the interpreter in validate.js is yours to fix.

Three symptoms, one file.

Bug report

BUG-2266 · Priority: High · Reported by: growth team (signup conversion -12%)

  1. A user aged exactly 18 is rejected ("must be at least 18") — 18 IS the minimum.
  2. Entering 0 for "newsletter frequency" says "this field is required" — 0 is a valid choice per the schema (required means present, not truthy).
  3. The email "sales@acme" passes validation and bounces later — the rule says a dot-separated domain is required.

Logs

[signup] reject { field: "age", value: 18, rule: "min:18" }
[signup] reject { field: "frequency", value: 0, rule: "required" }
[signup] accept { field: "email", value: "sales@acme" }

The code as shipped

src/forms/validate.js (editable)

// Interprets the rule schema against submitted values.
// Returns an array of { field, message } errors (empty = valid).
var messages = require("./messages");

var EMAIL_RE = /\S+@\S+/;

exports.validate = function (rules, values) {
  var errors = [];

  for (var field in rules) {
    var rule = rules[field];
    var value = values[field];

    if (rule.required && !value) {
      errors.push({ field: field, message: messages.required(field) });
      continue;
    }

    if (rule.email && value !== undefined && !EMAIL_RE.test(value)) {
      errors.push({ field: field, message: messages.email(field) });
    }

    if (rule.min !== undefined && typeof value === "number" && value <= rule.min) {
      errors.push({ field: field, message: messages.min(field, rule.min) });
    }
  }

  return errors;
};

Read-only context: src/forms/messages.js, src/forms/rules.js.

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More JavaScript bug hunts.