SCIENCE & ENGINEERING NEWS

Philadelphia, PA. — By marrying telecommunications and technology similar to that used in 3D movies, computer scientists have orchestrated a session where participants sitting in different states feel as if they’re chatting in the same room. This first successful demonstration of this technique, known as “tele-immersion,” was accomplished by researchers at the University of Pennsylvania, the University of North Carolina at Chapel Hill, Brown University and Advanced Network and Services, a non-profit firm in Armonk, N.Y.

Kostas Daniilidis, Ph.D., Penn’s group leader on the National Tele-Immersion Initiative, said that unlike conventional videoconferencing, tele-immersion all but teleports faraway places into the here and now.

“While videoconferencing results in two-dimensional images on a screen, in tele-immersion the screen becomes a window allowing access to a site far away,” said Daniilidis, an assistant professor of computer and information science at Penn. “The person with whom you’re speaking is projected life-size in three dimensions – you can even peer behind him or her.”

Daniilidis and his NTII colleagues say tele-immersion has the potential to revolutionize the way people communicate, allowing people on opposite ends of the country or world to feel temporarily as if they’re in each other’s presence. Expert surgeons thousands of miles away could be reassuringly present in an operating room to offer counsel, actors in New York and Los Angeles could rehearse together and distance learning could become as real and engaging as a traditional classroom experience. Executives or researchers on different continents could hold face-to-face meetings without ever boarding a jet.

Key to tele-immersion’s realistic feel are a hemispherical bank of digital cameras to capture participants from a variety of angles and tracking gear worn on their heads. Combined with polarized glasses much like those worn at 3D movies, the setup creates subtly different images in each eye – much as our eyes do in daily life. Working from this sensory input, the brain is able to recreate the three-dimensional “telepresence” of a person actually sitting in a distant studio.

“Tele-immersion essentially takes comprehensive, real-time measures of a person and his surroundings,” Daniilidis said, “and conveys that information directly to the senses of a person far away.”

The result is that when a participant moves his or her head, the view of the others shifts almost as seamlessly as if the meeting were occurring face-to-face. When participants lean forward, their peers appear larger; when they recline, their virtual friends shrink in size. The Penn group’s contribution is the technology for three-dimensional scanning of the environment and moving persons in real time using only conventional videocameras.

In the handful of tele-immersion sessions achieved so far, screens were mounted at right angles to a desk in the corner of a room, with each displaying one of the remote locations. While the screens are solid and flat, once a session begins they appear more like a window. Even the tabletops at the distant locations are perfectly matched to augment the feeling of shared space.

“It’s somewhat like the Starship Enterprise’s ëHolodeck’ on Star Trek,” Daniilidis said. “It allows us to interact with flat images as if they were living, breathing people right there in front of us.”

As with any brand-new technology, the sessions have been far from seamless: transmission quality has been a tad shaky, and at this point only one participant in any given conference is able to see his distant colleagues. Daniilidis and his colleagues from the NTII plan to address the latter shortcoming soon, with two-way sessions in which all participants can see each other.

Daniilidis said that tele-immersion’s potential to revolutionize communications remains a driving force behind the development of Internet2, the collaborative effort to create the high-bandwidth web of the future. While most online applications use only a tiny fraction of Internet2’s massive bandwidth, tele-immersion is one of the few that requires moving far greater quantities of data than today’s Internet can handle. A single tele-immersion session occurring on campus has temporarily boosted web traffic at Penn and other participating institutions to four times its normal volume.

The NTII, whose Penn participants include Daniilidis and colleagues Ruzena Bajcsy, Ph.D., and Jane Mulligan, Ph.D., is largely supported by Advanced Network and Services. Funding from the firm, founded by virtual reality pioneer Jaron Lanier, is augmented by support from the National Science Foundation, the Defense Advanced Research Projects Administration, the Department of Energy and the Intel Corporation.

============================================================