The New 2D to 3D Converter by MIT is supposed to be Better
Researchers at MIT and the Qatar Computing Research Institute (QCRI) presented the new 2D-3D conversion system in November 2015 at the Association for Computing Machinery’s Multimedia conference.
On the plus side, the unconventional method provides a very convincing stereoscopic 3D effect, with no visual artifacts. The researchers conducted a user study in which the majority of subjects gave the -D effect a rating of 5 (“excellent”) on a five-point (“bad” to “excellent”) scale; the average score was between 4 (“good”) and 5. On the minus side, the method works only for soccer matches...
Video Conversion Results (in 3D)
How It Works
Constraints of a football field environment makes the 3D conversion task easier but the new algorithm is truly original: it is based on a large database of exisiting football pictures and their associated depth taken from Microsft "FIFA 13" video game.
For every frame of 2-D video of an actual soccer game, the system looks for the 10 or so screen shots in the database that best correspond to it. Then it decomposes all those images, looking for the best matches between smaller regions of the video feed and smaller regions of the screen shots. Once it’s found those matches, it superimposes the depth information from the screen shots on the corresponding sections of the video feed. Finally, it stitches the pieces back together.
The system is one result of a collaboration between QCRI and MIT’s Computer Science and Artificial Intelligence Laboratory. Joining Wojciech Matusik, an associate professor of electrical engineering and computer science at MIT on the conference paper are Kiana Calagari, a research associate at QCRI and first author; Alexandre Kaspar, an MIT graduate student in electrical engineering and computer science; Piotr Didyk, who was a postdoc in Matusik’s group and is now a researcher at the Max Planck Institute for Informatics; Mohamed Hefeeda, a principal scientist at QCRI; and Mohamed Elgharib, a QCRI postdoc. QCRI also helped fund the project.
Read the full paper here.
Source : CSail.