Orders Sent to the Unknown Queue — Java Bug Hunt
Modelled on the VA's Oracle Cerner EHR rollout, reviewed by the VA Office of Inspector General in 2022: at the first site to go live, the Mann-Grandstaff VA…
- Language: Java
- Layer: Backend
- Difficulty: Medium
- Concepts: Validation, Routing
- Modelled on: VA / Oracle Cerner · 2022
- Visible tests: a mapped order reaches its clinic queue; an unmapped order goes to the monitored triage queue
- Reward: 50 XP for a complete fix
Incident
- Severity: SEV-1 (priority: Critical (patient safety))
- Ticket: BUG-OIG-UNKNOWN
- Reported by: clinical informatics
- Time to fix: 20 minutes, the target for a medium incident
Failing checks on the shipped build
- an unmapped order goes to the monitored triage queue — expected ORDER_TRIAGE but got UNKNOWN
How to work this incident
- Read the report: write down the expected behaviour and the actual one in a sentence each.
- Reproduce it: run the visible tests and find the failing check that matches the report.
- Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
- Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
- Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.
Briefing
Modelled on the VA's Oracle Cerner EHR rollout, reviewed by the VA Office of Inspector General in 2022: at the first site to go live, the Mann-Grandstaff VA Medical Center in Spokane, the new system sent orders that it could not match to a known location to an "unknown queue" that clinicians did not know existed. Thousands of orders sat there without anyone acting on them, and the OIG linked the problem to patient harm.
This reconstruction routes orders to clinic queues by location code; anything unmapped is parked in a queue nobody works.
Fix OrderRouter.route so no order is ever silently parked.
Bug report
BUG-OIG-UNKNOWN · Priority: Critical (patient safety) · Reported by: clinical informatics
route(orderId, location, orderedBy, store) returns the queue the order was put in. The location code is normalised first: trimmed and upper-cased (null -> "").
- mapped location: enqueue in LocationMap.queueFor(code), notify nobody
- unmapped location: enqueue in QueueStore.TRIAGE (a monitored queue) AND notify the ordering clinician with exactly store.notifyUser(orderedBy, "order <orderId> not routed: location '<code>' is not mapped")
- every order must end up in a queue listed in QueueStore.MONITORED; there must never be a queue named "UNKNOWN"
Observed: orders for unmapped locations are put in "UNKNOWN", which no worklist shows, and the ordering clinician is never told.
Logs
[orders] O-88121 location=SPK-PT -> UNKNOWN
[orders] UNKNOWN queue depth 11,204 (no subscribers)The code as shipped
src/orders/OrderRouter.java (editable)
class OrderRouter {
static String route(String orderId, String location, String orderedBy, QueueStore store) {
String code = location == null ? "" : location.trim().toUpperCase();
String queue = LocationMap.queueFor(code);
if (queue == null) {
queue = "UNKNOWN";
}
store.enqueue(queue, orderId);
return queue;
}
}Read-only context: src/orders/LocationMap.java, src/orders/QueueStore.java.
Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Java bug hunts.