That screen on your Toyota Prius may someday do more than show your mileage in a fancy graph. It could show you what's hiding ahead in the mist.
There was a time before Xbox 360s and quad-core gaming systems when adding fog-like visual effects to video games was considered a big deal. Removing fog might not seem as intriguing, but what if you could remove fog from a real-life scenario, such as driving on a foggy (or snowy, or rainy) road?
Computer Vision Systems
MAJOR INNOVATION Real-time display of obscured objects.
WHY IT MATTERS Could help car drivers and airplane pilots see through fog, and submarines explore under the sea. Could provide safety features for future intelligent transportation systems.
ESTIMATED ARRIVAL Under-water in a year or two; in buses, trains, and planes in five years; much later for the car companies.
We're not talking about weather modification. "You don't take the fog away for real," says Srinivasa Narasimhan, an assistant professor at Carnegie Mellon University, where he teaches computer vision and graphics systems. "You take the fog from the images." He readily admits that "the two problems [adding fog to computer images and removing fog from real-world images] are not of equal complexity." Adding fog is easier, but he is already taking cloudy, obscured images and making them clear.
The key is in the illumination. "Light up a foggy scene with a flood light. That's the worst thing you can do because it's backscattered [reflected] and all contrast is lost," says Narasimhan. Fog, mist, and water are considered "scattering mediums" for visual purposes. He explains that you have to scan light across a scene instead, to minimize backscattering. A camera can then take the results to make a clear image.
In a car, for example, you can't drive in deep fog with your high beams on; the light just saturates the fog, obscuring everything. That's why fog lights are usually low and close to the road, somewhat below the fog level. The journal Nature reported a decade ago that drivers tend to speed up while driving in fog, because the lack of visual cues makes them perceive their movement as slower than it is. Visibility naturally makes a difference in driver safety, but it's not as if drivers can always wait out fog or rain.
Narasimhan has made intelligent transportation systems in more than just cars a primary focus of his research. Exploration and safety will be enhanced both in flight and under the sea. Pilots of jets and submarines will get a visual leg up that's far more useful than sonar. And handheld vision systems for firefighters and miners (or anyone entering an area obscured by more than just darkness) are possible. The technology could also be useful for filmmakers who find their location shoots plagued by inclement weather. The professor's Web site (www.cs.cmu.edu/~srinivas) includes a scene from Forrest Gump in which a rainstorm is removed from the footage.
Narasimhan's group is working on the programming to make this happen. For lights, they currently use projectors, which they can easily program to show different patterns. In the real world, a specialized LED would be the simplest light to use, but a laser could work as well.
One of the offshoots of Narasimhan's research is scene reconstruction. Rather than just providing a glimpse of what's ahead, the scanning lights and cameras can capture a full 3D representation of what is obscured. Another spin-off is turning the camera into a weather meter. In the same way a person can glance at a street lamp in the rain and know it's raining outside based on the halo around the light, a vision system could immediately determine the exact amount of precipitation.
Commercialization of the technology is limited not by the technology but by the marketing, says Narasimhan. For example, car headlights that work with an intelligent transportation system may be ready in five years, but that doesn't mean the car companies—which have fog lamps to sell—will jump on it. "Fog lamps help psychologically, but they're not as safe," he says.
With sponsors like DARPA, the Office of Naval Research, and the National Science Foundation, however, he's safe in knowing his research is in the clear for the foreseeable future.
