Hindsight is an interactive visualization tool for the engineers and operators of NASA's Mars 2020 Perseverance rover mission. Every decision for the rover is high-stakes and must be rigorously justified. Operators chart the vehicle's daily path by investigating past drive faults and proving future plans are safe — a workflow that relies heavily on institutional memory, the collective and often unwritten knowledge of the operations team. Hindsight replaces that fragile process with a similarity search engine that lets operators ask "find me drives that look like this" instead of writing complex database queries.
Design Research
The original brief was to build a database with standard parametric queries. Instead, we spent five weeks using design probes to uncover how operators actually think. One probe was a projection-mapped model of Mars's Jezero Crater — a 3D-printed terrain map with rover paths and data projected directly onto its surface, revealing how spatial understanding helps operators build intuition and collaborate on planning.
The projection-mapped model was created with support from ArtCenter Fellow Adam Xu.
A 3D projection-mapped model of Mars's Jezero Crater, used as a design probe.
Sketches from the "draw your most memorable drive" design probe (top). Usability testing (bottom).
Another probe asked operators to "draw their most memorable drive." The results showed that operator intuition isn't built from isolated data points, but from the holistic character of an entire drive. Their real goal was to find analogous past situations to inform present decisions. This need to evaluate similarity — not filter parameters — became the guiding principle of our design.
The drive details view showing panels for the orbital map, imagery, telemetry charts, and event records.
The Tool
Hindsight's similarity search uses Dynamic Time Warping to compare the "shape" of time-series data — wheel slip, motor current, terrain type — across drives. It identifies and ranks past drives that are functionally analogous to a reference, even if they differ in length or location. Results are presented in a side-by-side layout for quick visual comparison of orbital paths, telemetry, and event records.
Timeline synchronization across multiple panels.
Synchronized Analysis
For deep-dives on a single drive, a synchronized playhead scrubs across the timeline, simultaneously updating the rover's position on the orbital map, time-series chart values, software event records, and corresponding rover imagery — correlating disparate data sources to show exactly what happened at any given moment.