Hertz Fellow Avideh Zakhor Maps Indoor Environments One Step at a Time

As anyone who has used Google Street View with a virtual reality headset or played any of the latest first-person video games can attest, immersive 3D environments can blur the lines between the real world and the computer-generated one.

The technology making this possible owes a debt of gratitude to the work of Hertz Fellow Avideh Zakhor, a professor of electrical engineering and computer science at UC Berkeley, who also heads the university’s Video and Image Processing Lab.

Zakhor is currently on leave from academia to focus on her role as CEO of the startup, Indoor Reality, a Berkeley based company that specializes in fast scanning “reality capture” of indoor environments, just as Google does with the outdoors. Using specially designed backpacks and handheld devices, Indoor Reality creates auto-generated 3D blueprints, virtual walkthroughs and maps of buildings, several orders of magnitude faster than traditional scanning. The company recently had its first paying customer, and the technology has garnered interest from the architecture, engineering and construction industries.

“The speed of data acquisition is very high, which most people are pleasantly surprised by,” Zakhor said. “Traditional methods take a really long time, and that limits the use of it. Because we’re so fast, it allows (these industries) to do things they were never able to do before. It’s an ‘a-ha!’ moment.”

Instead of taking days or weeks to gather data by outdated means, such as pushcarts, an operator wearing the 30-pound backpack need only to walk through a building at a regular speed while the device records data. The backpack, equipped with fisheye cameras, sensors, an inertial measurement unit and laser range finders, uses simultaneous localization and mapping (SLAM) algorithms to make 3D point clouds and create photo-realistic models. A 50,000 square foot building can be fully mapped in about 90 minutes.

“Indoor mapping and modeling is harder than outdoors because when you’re outdoors, you have GPS to help you correct errors,” Zakhor said. “When you’re indoors, you have to play a lot of games and tricks to localize the operator with the backpack. That’s a hard technical problem.”

Stemming from a request for proposal by the Department of Defense for a means to map the interior of buildings for reconnaissance purposes, the backpack has applications for first responders, commercial real estate, remodeling, facility management, and even energy efficiency, by analyzing thermal anomalies, water leaks and insulation.

“The beauty of this system is that it’s not only faster than traditional methods for generating the virtual models, but also it’s not just pretty pictures,” Zakhor said. “I like to think of the backpack as a hardware and software platform that allows different stakeholders to communicate, plan and coordinate through the life cycle of the building.”

Zakhor’s interest in engineering has roots in her childhood in Iran. Her father started the first button-manufacturing factory in Tehran in the basement of the building she grew up in, and the process fascinated young Avideh. At 15, she received a scholarship from the Iranian government to study in Wales, where she was awed by electronics. When the Shah of Iran was overthrown in 1979, her family fled to Los Angeles.

After finishing her schooling in Wales, Zakhor joined her family in the U.S. and attended the California Institute of Technology, where she received her bachelor’s degree in electrical engineering. As a Hertz Fellow, she continued on to graduate school at MIT, where she got her masters and PhD, also in electrical engineering. Her thesis focused on signal processing.

In 1988, Zakhor joined the faculty at Berkeley, and began doing research into optical proximity correction, a photolithography technique used to increase resolution and improve pattern transfer in the manufacturing of integrated circuits. She and a Phd student founded her first startup commercializing the technology, which was sold to Mentor Graphics in 1998.

In the early 2000s, Zakhor essentially created the precursor to Google Street View by loading a car with sensors and cameras and driving it around Berkeley. When combined with aerial photography and Lidar, her team created a complete 3D model of an entire city block. The Defense Advanced Research Projects Agency (DARPA) got wind of the tests and launched the UrbanScape program to commercialize it for real-time 3D reconstruction of urban environments. Google acquired the startup she founded with a postdoc in 2007.

Development of the first iteration of the indoor 3D modeling backpack began at Berkeley soon afterward. Now on their third generation of the device, a lighter version with infrared cameras and portable enough to carry-on airplanes, Indoor Reality is receiving funding from the Advanced Research Projects Agency-Energy (ARPA-E) to demonstrate it for energy modeling in buildings.

“To get a PhD thesis as a student you have to show that something works once, but when you’re productizing, then you have to show that it works every time, it can’t even fail once,” Zakhor said. “You have to figure out if anyone is willing to buy and use it, and how much are they willing to pay for it. That’s an issue that at the university, you never have to worry about.”

Zakhor said she plans to return to teaching at Berkeley after she raises venture capital dollars and turns Indoor Reality over to a professional CEO.