Read Your Own Writes — Java Bug Hunt
Inspired by the "my comment vanished!" bug of every replicated database: a user writes to the primary, the very next read hits a lagging replica, and their…
- Language: Java
- Layer: Database
- Difficulty: Easy
- Concepts: Replication, Consistency
- Modelled on: Replicated databases
- Visible tests: fresh writers read from the primary; old writers read from replicas
- Reward: 50 XP for a complete fix
Incident
- Severity: SEV-2 (priority: High)
- Ticket: BUG-RYW
- Reported by: community team
- Time to fix: 10 minutes, the target for an easy incident
Failing checks on the shipped build
- fresh writers read from the primary — expected primary but got replica
How to work this incident
- Read the report: write down the expected behaviour and the actual one in a sentence each.
- Reproduce it: run the visible tests and find the failing check that matches the report.
- Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
- Follow the data through the query or the repository code; compare the rows it returns with the rows the report expects.
- Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.
Briefing
Inspired by the "my comment vanished!" bug of every replicated database: a user writes to the primary, the very next read hits a lagging replica, and their data is briefly gone. The classic fix: pin reads to the primary for a short window after your own write.
ReadRouter.java ignores the window.
Bug report
BUG-RYW · Priority: High · Reported by: community team
routeRead(msSinceOwnWrite) with a 500ms pin window:
- writes newer than 500ms (including clock-skew negatives) -> "primary"
- otherwise -> "replica"
Observed: everything goes to replicas; users see their posts flicker out of existence right after submitting.
Logs
[router] msSinceOwnWrite=40 -> replica (lag 180ms) -> row not foundThe code as shipped
ReadRouter.java (editable)
class ReadRouter {
static final long PIN_WINDOW_MS = 500;
static String routeRead(long msSinceOwnWrite) {
return "replica";
}
}Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Java bug hunts.