The Track That Forgot Its Past — Python Bug Hunt
Modelled on the Uber ATG test-vehicle crash in Tempe, Arizona, March 2018.
- Language: Python
- Layer: Backend
- Difficulty: Hard
- Concepts: State, Safety
- Modelled on: Uber ATG · 2018
- Visible tests: a steadily labelled object gets a prediction; reclassification keeps the history
- Reward: 50 XP for a complete fix
Incident
- Severity: SEV-1 (priority: Critical (safety))
- Ticket: BUG-TRACK-HIST
- Reported by: perception review
- Time to fix: 35 minutes, the target for a hard incident
Failing checks on the shipped build
- reclassification keeps the history — moving 2 m/s towards the lane: expected [0.0, 0.0] but got null
How to work this incident
- Read the report: write down the expected behaviour and the actual one in a sentence each.
- Reproduce it: run the visible tests and find the failing check that matches the report.
- Read the logs and the failing checks for the first value that is wrong — the earliest wrong value is nearer the cause than the loudest error.
- Follow the request through the handler, the validation and the service it calls; find the first step whose output disagrees with the report.
- Fix the cause rather than the symptom, run the visible tests again, and submit against the hidden ones — then write the root cause down.
Briefing
Modelled on the Uber ATG test-vehicle crash in Tempe, Arizona, March 2018. The NTSB's investigation found that the automated driving system detected the pedestrian several seconds before impact but repeatedly changed its classification of her — between vehicle, bicycle and "other". Each time the classification changed, the system did not use the object's previous tracking history, so it could not form a path prediction for her in time.
This reconstruction's tracker starts a fresh, empty history whenever an object's label changes, even though it is the same track id. An object that keeps being relabelled never accumulates the two observations a motion estimate needs.
Fix Tracker so an object's history survives reclassification.
Bug report
BUG-TRACK-HIST · Priority: Critical (safety) · Reported by: perception review
Tracker, per track_id:
- observe(track_id, label, t, x, y) appends (t, x, y) to that track's history and sets its label to the latest one. A change of label is a reclassification of the SAME object: the history is kept.
- label(track_id) -> the latest label, or None for an unknown id
- predict(track_id, dt) -> [x, y] extrapolated dt seconds past the last observation using motion.velocity(history) (first to last observation), each coordinate rounded to 3 places; None for an unknown id or when velocity is None (fewer than two observations)
- tracks are independent: nothing about one id affects another
Observed: a pedestrian observed for three frames as vehicle, bicycle, other has no prediction at all.
Logs
[perception] track 17 label vehicle -> bicycle (history reset)
[perception] track 17 label bicycle -> other (history reset)
[planner] track 17: no predicted pathThe code as shipped
src/perception/tracker.py (editable)
motion = bug_require("./motion.py")
class Tracker:
def __init__(self):
self.tracks = {}
def observe(self, track_id, label, t, x, y):
track = self.tracks.get(track_id)
if track is None or track["label"] != label:
track = {"label": label, "history": []}
self.tracks[track_id] = track
track["history"].append((t, x, y))
def label(self, track_id):
track = self.tracks.get(track_id)
return track["label"] if track else None
def predict(self, track_id, dt):
track = self.tracks.get(track_id)
if track is None:
return None
v = motion.velocity(track["history"])
if v is None:
return None
t, x, y = track["history"][-1]
return [round(x + v[0] * dt, 3), round(y + v[1] * dt, 3)]
Read-only context: src/perception/motion.py.
Open the hunt to start the incident clock, edit the files, run the visible tests and submit against the hidden ones, then write the root cause down. More Python bug hunts.