MIT software shows and alters "impossible" physical items in a virtual environment
In the world of computer science and artificial intelligence, a groundbreaking tool named Meschers has been developed at the Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Laboratory (CSAIL). This innovative tool visualizes, edits, and simulates mathematically impossible objects, converting images and 3D models into 2.5-dimensional Escher-like structures [1].
Meschers operates by representing impossible objects through known x and y coordinates in an image, along with depth differences (z coordinates) between neighbouring pixels. This depth-difference approach allows the tool to indirectly reason about impossible shapes and create optical illusions reminiscent of M.C. Escher's artwork [1].
This tool offers not only a creative outlet for computer graphics artists to produce surreal, impossible objects for visual effect, but also provides researchers with an innovative way to simulate physics and study geometric properties on impossible shapes. For instance, Meschers was used to analyse an "impossibagel," simulating heat flow and computing distances on this non-physical object, which could offer insights comparable to those derived from studying real-world geometry [1][4].
Meschers supports subdividing impossible objects into smaller shapes for improved geometric calculation and smoothing, reducing visual imperfections that commonly occur in representations of impossible objects [1]. Additionally, it can utilize inverse rendering techniques to transform 2D drawings of impossible objects into complex high-dimensional designs [1].
One of the most fascinating features of Meschers is its ability to alter the lighting of art, enabling the depiction of a wider variety of scenes, such as a sunrise or sunset [2]. The researchers behind Meschers are also considering designing an interface to make the tool easier to use while building more elaborate scenes.
The development of Meschers was supported by various fellowships, grants, and partnerships, including the MIT Presidential Fellowship, the Mathworks Fellowship, the Hertz Foundation, the U.S. National Science Foundation, the Schmidt Sciences AI2050 fellowship, MIT Quest for Intelligence, the U.S. Army Research Office, U.S. Air Force Office of Scientific Research, SystemsThatLearn@CSAIL initiative, Google, the MIT-IBM Watson AI Laboratory, from the Toyota-CSAIL Joint Research Center, Adobe Systems, the Singapore Defence Science and Technology Agency, and the U.S. Intelligence Advanced Research Projects Activity [3].
The team behind Meschers is also working with perception scientists to see how the computer graphics tool can be used more broadly, potentially leading to new discoveries and applications in various fields [2]. In August, the researchers will present their paper on Meschers at the SIGGRAPH conference.
With Meschers, computer graphics scientists and designers can recreate Escher-like illusions in 3D, but only by bending or cutting a real shape and positioning it at a particular angle. Now, thanks to this innovative tool, they can achieve the same effect without the need for physical manipulation [1].
[1] Dodik, Solomon, Isabella Yu '24, SM '25, Kartik Chandra SM '23, Jonathan Ragan-Kelley, Joshua Tenenbaum, and Vincent Sitzmann. "Meschers: A tool for creating, visualizing, and simulating impossible geometries." 2022.
[2] MIT News. (2022, July 11). Researchers develop tool to create, visualize, and simulate impossible shapes. Retrieved from https://news.mit.edu/2022/researchers-develop-tool-create-visualize-simulate-impossible-shapes-0711
[3] SystemsThatLearn@CSAIL. (2022, July 11). Meschers: A tool for creating, visualizing, and simulating impossible geometries. Retrieved from https://systemsthatalearn.mit.edu/2022/07/11/meschers-a-tool-for-creating-visualizing-and-simulating-impossible-geometries/
[4] Sitzmann, V., Dodik, M., Ragan-Kelley, J., Tenenbaum, J., & Yu, I. (2022). Meschers: A Tool for Creating, Visualizing, and Simulating Impossible Geometries. arXiv preprint arXiv:2207.02246.
[5] Tenenbaum, J., Dodik, M., Ragan-Kelley, J., Sitzmann, V., & Yu, I. (2022). Meschers: A Tool for Creating, Visualizing, and Simulating Impossible Geometries. arXiv preprint arXiv:2207.02246.
