-
Viewpoint on 'IONS'
Viewpoint on 'Scientific Literacy'
- Proudly sponsored by
-
-
Towards Filming Chemical Reactions
High-speed cameras can film a bullet passing through an apple. Chemical reactions, however, are far too fast even for the best cameras. Attosecond physics promises new approaches to study and manipulate them.
-
Forensic Optics: Transforming a Hair into a Travel Log
Solving crimes requires and complicated investigations in order to find clear evidence. Now, it is possible to obtain information about a suspect's travel history by analyzing their hair.
-
Why We Should All Care about Science Writing
Science reporters play a crucial role in developing a public that is literate about science so that people are able to make wise choices about issues with scientific underpinnings.
by Lulu Rodriguez
Volume 18 Story 4 - 21/11/2012
Now also in Spanish
Espiar a la vuelta de la esquina
and Portuguese
Visão através das paredes,
brought to you by
Optics and Photonics Latin America
If you have ever played a first-person-shooter video game, you know the feeling when your character is hiding behind a corner and your heart races, not knowing what kinds of grotesque aliens or heavily-armed bad guys are waiting for you to step out into the open. But what if you could turn on a special camera that lets you peek around the corner without exposing yourself to an attack? Recent research from a group at the MIT media lab (Boston, USA) suggests that seeing around corners is not just a cool superpower for a video game, but it might actually be possible. And imaging around corners could have applications in everything from search and rescue to new types of endoscopes for studying hard-to-reach areas inside the body.
In a typical camera, the image that we capture is an average image: it represents the amount of light coming from each point in the scene during a small time interval, usually in the order of some milliseconds. During an exposure, each pixel accumulates energy from the light hitting its surface. If we imagine the light from the scene as a collection of rays, then each pixel collects rays like a bucket collects raindrops. At the end of the exposure, the pixel value is a measure of the total energy collected in the “light bucket.” The light from a camera flash travels so fast — about a fraction of a meter per nanosecond — that a conventional camera cannot distinguish between the first light to reach the camera and the last light to reach the camera. A disadvantage of the light-bucket measurement is that we lose information about the paths of the individual light rays. Did the raindrop fall out of the sky and directly into the bucket, or did it bounce off of a few leaves on its way down?
In the past decade, advances in ultra-high-speed light sources and detectors have led to cameras that can measure the travel time of light rays, as well as to algorithms for using travel-time measurements to recover the 3D properties of a scene. But what if we want to know what’s happening beyond the camera’s direct line of sight?
Imaging an object beyond the line-of-sight is much like normal imaging — incident light bounces off the object and is collected by a camera. The main difference is in the number of bounces. To image around a corner, we are mainly interested in the light that first bounces off an intermediate surface, then off the object, then again off the intermediate surface, and that finally reaches the camera indirectly. By measuring the travel time of the multi-bounce light, it is possible to recover information about the hidden object.
Andreas Velten and his group at the MIT media lab set up an experiment to test the multi-bounce imaging system. In their experiment, they scanned laser light across an intermediate surface and collected the multi-bounce light from a hidden object using a special high-speed camera. At each point in the laser scan, the distribution of the multi-bounce light captured by the camera is different. By appropriately combining the data acquired during the laser scan, they found that they could recover information about the shape of the object hidden behind the obscuring wall.
Simple, right? Not so fast! As usual in experiments, things are not as straightforward as they sound and they present various technical challenges. In addition to building a robust experimental setup to acquire images at a trillion frames per second, the critical issue for Velten’s group was to find an appropriate algorithm for computing the final image of the hidden object. "For quite a while, we had what we thought was good data, and we had an algorithm that worked in simulation, but not on the actual data," says Velten. There was a need for experimental-oriented and also for theory-oriented people to work together to understand the limitations of the data and how to best make use of the information.
Much of the fun surrounding a new discovery like imaging around corners is imagining all of the potential applications of the technology. According to Velten, "[Possible applications] range from rescue and law enforcement operations where one needs to check the inside of a room or building, to helping solve problems in endoscopy and robotic surgery where it is crucial to get a good idea of the 3D structure of a cavity inside the body from very limited camera positions." As with other young technologies, it can be difficult — and restricting — to focus on one specific application. In fact, according to Kyros Kutulakos at the University of Toronto in Canada, "It is quite possible that its most important applications are yet to be recognized."
Peeking Around Corners
Until now cameras have only been able to see what is in front of them. Thanks to recent developments in ultra-fast imaging and computational photography, they may also soon be able to see objects out of the line of sight.Computed image of a hidden object. By carefully combining the multi-bounce image data at each point in the laser scan, Velten and colleagues were able to recover an image of the hidden object.
Espiar a la vuelta de la esquina
and Portuguese
Visão através das paredes,
brought to you by
Optics and Photonics Latin America
If you have ever played a first-person-shooter video game, you know the feeling when your character is hiding behind a corner and your heart races, not knowing what kinds of grotesque aliens or heavily-armed bad guys are waiting for you to step out into the open. But what if you could turn on a special camera that lets you peek around the corner without exposing yourself to an attack? Recent research from a group at the MIT media lab (Boston, USA) suggests that seeing around corners is not just a cool superpower for a video game, but it might actually be possible. And imaging around corners could have applications in everything from search and rescue to new types of endoscopes for studying hard-to-reach areas inside the body.
In a typical camera, the image that we capture is an average image: it represents the amount of light coming from each point in the scene during a small time interval, usually in the order of some milliseconds. During an exposure, each pixel accumulates energy from the light hitting its surface. If we imagine the light from the scene as a collection of rays, then each pixel collects rays like a bucket collects raindrops. At the end of the exposure, the pixel value is a measure of the total energy collected in the “light bucket.” The light from a camera flash travels so fast — about a fraction of a meter per nanosecond — that a conventional camera cannot distinguish between the first light to reach the camera and the last light to reach the camera. A disadvantage of the light-bucket measurement is that we lose information about the paths of the individual light rays. Did the raindrop fall out of the sky and directly into the bucket, or did it bounce off of a few leaves on its way down?
Imaging around a corner. To see beyond the line of sight, a camera must be able to discriminate between single- and multi-bounce light.
Imaging an object beyond the line-of-sight is much like normal imaging — incident light bounces off the object and is collected by a camera. The main difference is in the number of bounces. To image around a corner, we are mainly interested in the light that first bounces off an intermediate surface, then off the object, then again off the intermediate surface, and that finally reaches the camera indirectly. By measuring the travel time of the multi-bounce light, it is possible to recover information about the hidden object.
Andreas Velten and his group at the MIT media lab set up an experiment to test the multi-bounce imaging system. In their experiment, they scanned laser light across an intermediate surface and collected the multi-bounce light from a hidden object using a special high-speed camera. At each point in the laser scan, the distribution of the multi-bounce light captured by the camera is different. By appropriately combining the data acquired during the laser scan, they found that they could recover information about the shape of the object hidden behind the obscuring wall.
Velten's experiment at the MIT Media Lab. Laser light is scanned across a wall, and the multi-bounce light is captured by a high-speed streak camera.
Much of the fun surrounding a new discovery like imaging around corners is imagining all of the potential applications of the technology. According to Velten, "[Possible applications] range from rescue and law enforcement operations where one needs to check the inside of a room or building, to helping solve problems in endoscopy and robotic surgery where it is crucial to get a good idea of the 3D structure of a cavity inside the body from very limited camera positions." As with other young technologies, it can be difficult — and restricting — to focus on one specific application. In fact, according to Kyros Kutulakos at the University of Toronto in Canada, "It is quite possible that its most important applications are yet to be recognized."
Stefano Young
2012 © Optics & Photonics Focus
SY holds a PhD from the University of Arizona's College of Optical Sciences and is currently based in Los Angeles, CA.
Andreas Velten, Thomas Willwacher, Otkrist Gupta, Ashok Veeraraghavan3, moungi G. Bawendi & Ramesh Raskar, Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging, Nature Communications (2012) 3, 745 (link).