Autonomous vision: day and night
- Timofey Uvarov
- Jan 8
- 2 min read
Updated: Apr 14
camera for traffic light detection (developed for Pony.ai)
Driving through a traffic light control intersection under a red light is incredibly dangerous. To reduce the probability of such an event to zero, Pony.ai has developed an AI-trained camera. We analyze traffic light data from millions of miles driven and all types of corner cases, such as broken, partially covered, dirty, and incorrectly directed traffic lights. We address all these scenarios in the camera design and tuning process.
Camera shutter speed, often called integration or exposure time, controls the time the camera sensor uses to capture the scene. Camera exposure control is autonomous and doesn’t depend on the car’s speed. It only decides the integration time based on the scene’s brightness, making it extremely hard for the conventional camera to capture LED lights with flickering.
The camera is designed to see traffic lights regardless of day or night, whether they are modern bright LED lights with flickering or old-style traffic lights using an Edison-type light bulb.
Pony.ai’s in-house-designed camera (PiDC) always uses the same exposure time of 11ms to achieve that effect. To compensate for such a highly sensitive sensor setting, we use a neutral density filter, equivalent to multiple pairs of sunglasses stacked on top of each other, to avoid oversaturation. And, yes, we use this filter at night, too!
Here is how our camera sees the world when we slightly adjust its tuning for human vision:
Mt. View Caltrain station, Pony.ai camera ISO_low 0.011s
The sun is in the center of our visual field. Usually, it would be very challenging for an autonomous vehicle camera to see traffic lights when the sun is behind it due to the enormous difference between the sun and the traffic light. Still, Pony’s camera can clearly identify traffic light signals and reproduce the street sign backed by the sun disc, which is much darker than the traffic light.
Shoreline Blvd, Pony.ai camera: 0.011s 2x-enlarged
Mountain View City Hall, Pony.ai camera: 0.011s. The photo is 3x enlarged.
Pony added the camera to its fleet and trained its traffic lights model. In August 2021, the number of monthly issues caused by traffic lights has been reduced to zero.
Below, you can watch in-depth technical presentation of tuning this camera:
