Researchers from Tel Aviv University have developed a computational photography process based on an optical element that encodes motion information and a corresponding digital image processing algorithm, enabling clear, sharp photography of moving objects without motion blur, i.e. avoiding the movement being “smeared” over the picture.
This integrated processing method was developed by Ph.D. student Shay Elmalem from the School of Electrical Engineering in the Iby and Aladar Fleischman Faculty of Engineering, under the joint guidance of Prof. Emanuel Marom and Dr. Raja Giryes. The results of the study have been published in the prestigious Optica Journal (Motion deblurring using spatiotemporal phase aperture coding).
“The term ‘long exposure’ always refers to the velocity of the photographed object”, explains Shay Elmalem. “If you photograph a racing car, even an exposure of a tenth of a second could be too long, and if you’re photographing a person walking, long exposure could be a second or longer. According to the conventional camera design approach, the lens is designed to produce the best possible image, i.e., the most similar to what the human eye sees, and thereafter digital image processing algorithms are applied to remove the optical distortions. However, as anyone with a camera in their phone knows, this isn’t always effective; hence, it is still very difficult to photograph moving objects.”
Through integrated design of the optical components and image post-processing algorithms, Elmalem and his colleagues have encoded motion information cues in the raw optical image; these cues are in turn decoded by the image processing algorithm which utilizes them for motion deblurring. The cues have been encoded using two optical components integrated in a conventional lens: a clear phase plate developed by the researchers, and a commercial electronic focusing lens. The phase plate contains a micro-optical structure designed to introduce a color-focus dependency, whereas the focusing lens is synchronized to make a gradual focus change during the image exposure. As a result, moving objects are colored with various colors as they move. Encoding the colors enables the algorithm to decode the direction and velocity of the object’s movement, which enables it to correct the motion blur and restore the image sharpness.
“In every split second of exposure, our lens generates a bit different image,” Elmalem explains; “thus, the blur of a moving object will not be uniform, but rather change gradually with its movement. To understand where and how fast the object in the image is going, we use color. Thus, for example, a white ball suddenly thrown into the frame will be colored with different colors throughout its movement, like passing light through a prism. According to these colors, our algorithm knows where the ball has been thrown from and at what velocity. It will thus know how to correct the blur. With a regular camera, we’d see a white wake that would compromise the sharpness of the whole picture, whereas with our camera the final image will be a clear focused white ball.”
According to Elmalem, the computational image technique they developed can enhance any camera – and at minimum cost. “The potential is very broad: from basic uses like smartphone cameras to research, medical and industrial uses such as for production line controllers, microscopes and telescopes. They all suffer from the same smearing problem, and we offer a systemic solution to it.”
Ramot, the Technology Transfer Company of Tel-Aviv University, has filed several patent applications covering this breakthrough technology, which is generating great interest among industry players.
Prof. Marom passed away during the study, and the paper has been published in his memory.
The late Prof. Marom was among the founders of the Faculty of Engineering at Tel Aviv University, served as its Dean in 1980-1983 and Vice President of Tel Aviv University in 1992-1997. After his retirement, Prof. Marom continued dealing in active research and advising graduate students, until his very last day.