The Runaway Retry — Python Bug Hunt

Briefing

Inspired by the retry storms that finish off every wounded database: a hiccup triggers retries, uncapped exponential delays overflow into instant retries — and the "backoff" becomes a battering ram.

backoff.py: n retry delays, doubling from 1s, capped at 60s.

Bug report

BUG-STAMPEDE · Priority: High · Reported by: SRE

delays(n):

• [1, 2, 4, 8, 16, 32, 60, 60, ...] — doubling, capped at 60, length n

Observed: the list starts at 0 (an INSTANT retry), never caps, and returns one extra element — retry #31 asks to wait 2^31 seconds.

Logs

[retry] delays(3) -> [0, 1, 2, 4]

The code as shipped

src/db/backoff.py (editable)

# Exponential backoff schedule for reconnect attempts.

CAP_SECONDS = 60

def delays(n):
    return [2 ** i // 2 for i in range(n + 1)]