The Replayed Webhook

Medium

· Backend

Incident

SEV-1

BUG-HOOK-2X

Priority: Critical (security) · reported by pentest

Time to fix: 20 min

Failing checks on the shipped build

  • the same event id is a replay

    expected "replay" but got "ok"

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Inspired by

Stripe webhooks

Inspired by the webhook-replay attacks every payment integration guide warns about: capture one signed delivery, resend it later (or pre-date it into the future), and trigger the fulfilment twice.

webhook.py validates deliveries. Its replay window only looks one way, and its dedupe records the event before checking it.

Code map

# Validates an incoming webhook delivery.

TOLERANCE_SECONDS = 300

def accept(event_id, timestamp, now, seen):
    seen.add(event_id)
    if now - timestamp > TOLERANCE_SECONDS:
        return "stale"
    if event_id in seen:
        return "ok"
    return "ok"

Sign in to edit the file, run the tests and submit a fix

A free account — no card, no trial clock. You come straight back here.

The Replayed Webhook — Python Bug Hunt

Inspired by the webhook-replay attacks every payment integration guide warns about: capture one signed delivery, resend it later (or pre-date it into the…

  • Language: Python
  • Layer: Backend
  • Difficulty: Medium
  • Concepts: Security, Time
  • Modelled on: Stripe webhooks
  • Visible tests: a fresh event is accepted once; the same event id is a replay; old deliveries are stale
  • Reward: 50 XP for a complete fix

Incident

  • Severity: SEV-1 (priority: Critical (security))
  • Ticket: BUG-HOOK-2X
  • Reported by: pentest
  • Time to fix: 20 minutes, the target for a medium incident

Failing checks on the shipped build

  • the same event id is a replay — expected "replay" but got "ok"

How to work this incident

  1. Read the report: write down the expected behaviour and the actual one in a sentence each.
  2. Reproduce it: run the visible tests and find the failing check that matches the report.
  3. Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
  4. Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
  5. Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.

Briefing

Inspired by the webhook-replay attacks every payment integration guide warns about: capture one signed delivery, resend it later (or pre-date it into the future), and trigger the fulfilment twice.

webhook.py validates deliveries. Its replay window only looks one way, and its dedupe records the event before checking it.

Bug report

BUG-HOOK-2X · Priority: Critical (security) · Reported by: pentest

accept(event_id, timestamp, now, seen) rules (300s tolerance):

  • reject if the event id was already accepted -> "replay"
  • reject if |now - timestamp| > 300 -> "stale"
  • otherwise record the id in seen and return "ok"

Observed: the same event id is accepted twice, and a timestamp 2 hours in the FUTURE sails through.

Logs

[hook] evt_9f2 accepted
[hook] evt_9f2 accepted   <- replay!

The code as shipped

src/hooks/webhook.py (editable)

# Validates an incoming webhook delivery.

TOLERANCE_SECONDS = 300

def accept(event_id, timestamp, now, seen):
    seen.add(event_id)
    if now - timestamp > TOLERANCE_SECONDS:
        return "stale"
    if event_id in seen:
        return "ok"
    return "ok"

Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Python bug hunts.