Volume 17 Story 3 - 7/6/2012

A Seriously Defocused Spider

A jumping spider about to pounce on its prey needs to be able to accurately measure its distance from its target in order to be able to hit it with precision. This is possible thanks to a visual approach previously unknown to exist in nature.

Jumping spider and its 4 eyes. A jumping spider (<i>H. adansoni</i>) has well-developed camera-type eyes, seemingly like human eyes. This is in striking contrast to the compound eyes — eyes made of many small and simple photoreceptors — of other arthropods such as shrimps and insects.

Jumping spider and its 4 eyes. A jumping spider (H. adansoni) has well-developed camera-type eyes, seemingly like human eyes. This is in striking contrast to the compound eyes — eyes made of many small and simple photoreceptors — of other arthropods such as shrimps and insects. Picture credits: Akihisa Terakita, Mitsumasa Koyanagi and Takashi Nagata.

A jumping spider sets up an ambush
on a blissfully unaware fly.
It jumpsÖ
Ö but misses its prey.
Unheard of!
Thatís never happened before!
That red light is really getting on its nerves!
It was so much better when the light was green!

Catching a ball, driving a car, and jumping on prey are all examples of daily tasks that critically rely on our being able to see the world in 3D. Both of our eyes working together, in unison, afford us stereoscopic vision. However, some other animals use different mechanisms — accommodation and motion parallax — to perceive the absolute distance of objects with one eye alone. Now, a group of Japanese researchers has discovered that some spiders make use of an entirely different approach — depth from defocus — to decide how far they must jump to hit their prey.

The jumping spider — the Hasarius adansoni to be precise — has been the subject of experiments performed by Akihisa Terakita, Mitsumasa Koyanagi and Takashi Nagata and coworkers at Osaka City University, in Japan. The task of the spider was to look at some prey and try to target it with precision under various illumination conditions. Of course, in order to accomplish this, the jumping spider must accurately measure its distance from its potential prey.

Nature has developed various approaches for 3D vision. The one we are most familiar with is our own: stereoscopic vision. It is very powerful, but it requires two eyes working in unison. Our brainís visual system combines the slightly displaced images from our right and left eye to infer the position of an object. The jumping spider has not two, but four eyes, potentially allowing it an even more powerful quadri-stereoscopic vision. However, the spider is able to correctly jump on the prey even if all of its eyes but one are covered; so it must adopt some other mechanism that makes use of one eye alone.

Depth from defocus. Theoretically, the amount of image defocus (i.e., how much an image is defocused) can be used as an absolute depth cue by comparing a defocused image with one or more additional images in which the same object is sharply focused or differently defocused.

Depth from defocus. Theoretically, the amount of image defocus (i.e., how much an image is defocused) can be used as an absolute depth cue by comparing a defocused image with one or more additional images in which the same object is sharply focused or differently defocused. Picture credits: Akihisa Terakita, Mitsumasa Koyanagi and Takashi Nagata.

Evolution has provided some animals with alternative approaches to depth estimation, in which only one eye is enough. One possible way is to adjust the shape of the eye — a mechanism known as accommodation; this is, for example, the mechanism used by chameleons. Another common mechanism, especially amongst insects, is motion parallax, where by moving their head side-to-side they can estimate the distance from the apparent movement of an object against the background.

The surprising result of our researchers was that the jumping spider uses a mechanism different from all of the above — something never seen before in nature. "In this work, we found that jumping spiders use defocused images to measure the distance from their prey," Terakita, Koyanagi and Nagata explain. "This is the first example of the depth perception from defocus in living animals, a new way of perceiving depth." This mechanism is called depth from defocus. It is somewhat counterintuitive. Indeed, typically, one tries to focus an image to see it properly. Defocusing it might seem counterproductive. However, the comparison of two differently defocused images can be used as an absolute depth cue. "Defocus is basically an unfavorable factor for accurate vision," Terakita, Koyanagi and Nagata remark. "Therefore, we were quite surprised to find that jumping spiders can take advantage of image defocus to accurately estimate distance."

Terakita, Koyanagi, Nagata and coworkers noticed that the eyes of a jumping spider have various layers of photoreceptors. The second layer, in particular, always captures defocused images. This result was confusing because a defocused image did not seem to be useful for accurate vision. "Finally, one of us came up with the idea that the amount of defocus contained distance information." In their hypothesis, the same image is projected on two layers of photodetectors and the spider is able to reconstruct the distance of the object from the different level of defocusing of the two images.

The researchers, therefore, went to the lab and tested their spiders with the assumption that the jumping insects should perform best under green light illumination. In fact, their detector layers have evolved to be sensitive mainly to green light, where sunlight peaks. The performance of these detectors with different colors, therefore, should be substantially reduced, because the defocusing also depends on the light color. As expected, under green light, the jumping spiders were able to catch their prey with impeccable precision, but they seemed to misjudge the distance and, as a result, underperform when red was chosen as illumination light. This fact was taken as a behavioral confirmation of the researchersí hypothesis; however, Terakita, Koyanagi, and Nagata say that "how the defocus amount is translated to the depth information at the neural level in a jumping spider, remains an open question."

With the work of Terakita, Koyanagi, Nagata and coworkers, we have had yet another peek into the workings of nature. Although image defocusing in animal vision has not traditionally been thought of as very useful, the eyes of these tiny spiders point to the contrary conclusion. "We are still learning many things from animals." Furthermore, there is also a technological pitch to it. Indeed, 3D vision is quickly taking center stage in our technological world: 3D movies are taking theaters by storm and are coming into our homes; 3D displays are appearing in airports and shopping centers; 3D vision technology is acquiring an ever-increasing role in medicine and security. In this context, "the concept depth from defocus is of particular interest in the field of computer vision," Terakita, Koyanagi and Nagata envision. "The jumping spider system could help develop optical apparatus for depth from defocus."

Giovanni Volpe

2012 © Optics & Photonics Focus

GV is Assistant Professor at Bilkent University in Ankara (Turkey); his research focuses on optics, statistical physics and soft matter (http://softmatter.bilkent.edu.tr/).


Download pdf for A Seriously Defocused Spider

Takashi Nagata, Mitsumasa Koyanagi, Hisao Tsukamoto, Shinjiro Saeki, Kunio Isono, Yoshinori Shichida, Fumio Tokunaga, Michiyo Kinoshita, Kentaro Arikawa & Akihisa Terakita, Depth Perception from Image Defocus in a Jumping Spider, Science (2012) 335, 469-471 (link).