Research project to use photons to see round corners

Conventional optical imaging systems today largely limit themselves to the measurement of light intensity, providing two-dimensional renderings of three-dimensional scenes and ignoring significant amounts of additional information that may be carried by captured light. For example, many photons traverse complex paths punctuated by multiple bounces prior to entering the aperture of a camera or other imager, picking up information about their surroundings on the way.

Beyond such directional variability, light enjoys other aspects or degrees of freedom - including variations in propagation time, polarisation state and spectral content, as well as wave-related properties such as coherence, diffraction and interference - all of which provide potential mechanisms by which light can acquire and convey information. Most of this information remains untapped today.

DARPA’s Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) programme seeks to unlock information in photons that current imaging systems discard. The REVEAL programme aims to develop a comprehensive theoretical framework to enable maximum information extraction from complex scenes by using all the photon pathways of captured light and leveraging light’s multiple degrees of freedom.

The goal is for this framework to guide the development of imaging hardware and software technologies. Furthermore, the programme will test the bounds of the developed framework and the functionality of the new imaging technologies via a challenge problem that calls for full 3D scene reconstruction from a single viewpoint. By contrast, current light-capturing methods require multiple viewpoints for rendering a scene in 3D.

“There are some current limited efforts attempting to exploit some of light’s multiple degrees of freedom, but REVEAL aims to make a revolutionary leap forward by simultaneously addressing all aspects of light,” said Predrag Milojkovic, Programme Manager, Defense Sciences Office, DARPA. “In effect, we want to use mathematical methods to coax from photons a little more of a story about where they’ve been and what they’ve seen.”

An ability to interpret the information that light may be carrying in degraded form could enhance situational awareness for troops - potentially allowing them to reconstruct, from a single vantage point, a complex scene including objects or people not visible by line-of-sight viewing.

Imagine, for example, squad members patrolling a street in a deployed urban environment, and an armed assailant crouching behind a car or a concrete barrier. Without the benefit of different vantage points (from the air, for example), the squad could be blind to the hidden threat. If by chance a glass storefront window were behind the assailant, the squad might spot the assailant’s reflection in the window. But if the backdrop were a brick wall, there would be no visible reflection. By exploiting currently untapped aspects of light and the varied paths of photons bouncing off the brick wall, troops using hardware based on the theoretical foundations provided by REVEAL might someday be able to detect the otherwise hidden assailant.

Another potential application could be determining an unknown material’s composition and other properties from a safe distance, avoiding the potential danger associated with close proximity and physical examination. Based on information carried by the photons interacting with the material, it may be possible for troops in the future to identify radioactive, biological or chemical threats and camouflaged targets from much farther away than currently possible.

“Ultimately, collecting all pertinent information about a scene could enable computational generation of arbitrarily located virtual viewpoints and effectively allow ‘flying through the scene’ without changing one’s physical location,” Milojkovic added.

The REVEAL programme is structured in two 24-month phases. The first phase seeks to determine fundamental limits of single-viewpoint scene reconstruction through laboratory experimentation to validate critical concepts and approaches. Phase 2 intends to test and evaluate full 3D scene reconstruction under realistic illumination conditions and develop a general theoretical framework for exploiting light’s multiple degrees of freedom. As a basic research effort, REVEAL will not develop fieldable hardware, software or imaging systems.

Article & image courtesy of DARPA.