A simulated street scene: the vehicle, its planned path, and surrounding traffic tracked with speed and distance readouts

Synthetic Scenario Generation

Synthetic Scenario Generation produces driving scenarios by the million, preparing autonomous vehicles for rare road conditions and interactions. An autonomous vehicle has to handle edge cases correctly the first time it encounters them. Some of the more interesting cases Waymo encountered during my time there include: a mattress falling off the back of a truck at highway speed, a five-way intersection with the markings worn away, a deep puddle in a construction zone. Collecting enough data from real driving for each of these cases would take years, so they get built in simulation instead. At Waymo, the operations team designs these simulated driving scenarios to aid in development and testing of the core software.

A simulated reconstruction of a San Francisco street above the camera footage it was built from
A driving scenario in simulation (top), against the original camera footage it was derived from (bottom).

Leveraging my team's core simulation tech, I designed and built a software stack that could traverse a mapped city and script interactions at every qualifying location. For example, the system made it possible to generate scenarios with a motorcycle running a red light at every left turn in San Francisco, or generate thousands of examples of different objects — mattresses, barbecues, tires, oh my! — falling off trucks in every lane of I-80. I led technical development (and took on much of the UX design) over the course of a year. At launch, our system could generate millions of scenarios in a matter of hours.

Luke Fiorante beside a Waymo prototype vehicle
At Waymo HQ, beside the first Waymo Ojai prototype.