How Do Self-driving Cars See The World?

Author: Brandon Butrick |


Recently, Photon Creative worked on a set of videos for an event about cognition for autonomous cars using 6D localization for Civil Maps.

“Civil Maps provides cognition for autonomous vehicles, enabling them to crowdsource continental scale, 3D semantic maps for safe driving. With a highly scalable approach, we are creating a new generation of maps that enable fully self-driving cars to traverse any road safely and smoothly without any human intervention.”

Here at Photon, we get excited about tech that actually feels like we’re living in 2017, for instance: self-driving cars. When we learned about Civil Maps’ localization technology, it definitely gave us that futuristic feeling that pushes us forward. It also let us know what really sets them apart from other autonomous car technology. The localization in this instance is referring to localizing a vehicle in six degrees of freedom. This is the movement axes (x, y, z) and also rotational axes (roll, pitch, yaw). Utilizing these 6 degrees of motion, along with the 3D semantic maps created by Civil Maps, this video shows how the car sees in 3D projected onto a real environment. Localization in the 6 degrees of freedom allows the car to use selective attention and use less bandwidth for computations. This results in more accurate results with shorter data transfer times – a win, win. To create a product video that shows the vision of an automated vehicle, we created a full 3D camera track of each scene provided. This allowed us to realistically add in street layers and lane markers, with the proper perspective. We were able to achieve this natively inside of Adobe After Effects with the 3D camera tracker tool. For all of the miscellaneous objects, such as street signs and stop signs, we created housings for those in order to bring it all together. The end result is how a self-driving car sees the world. Our video was featured on TechCrunch with a write up by Darrell Etherington and is just passing the 60k view mark on youtube.