Emerging Technologies: SIGGRAPH 2006

An interactive game that helps players sense unseen existences through indirect information such as their shadows, sounds, and weights. In this game, "invisible" goblins sneak around, but you can only see their shadows. Your mission is to chase their shadows and capture them with a special vacuum.

Enhanced Life
This project presents an object's existence in the real world using multimodal perceptions, a concept that may be useful for supporting barrier-free services for handicapped people in public and entertainment spaces.

Goals
The goal of this project is to help users feel the existence of specific objects through indirect information and to show that the approach can be effectively implemented as a simple virtual reality system.

Innovations
The interactive system allows users to feel the presence of 3D virtual objects without stereoscopic visual information. It includes:

1. Capturing 3D information (position and orientation) from the projector. For natural projection of the shadow shapes, the 3D position and orientation of the projector are obtained in real time with a simple method:

Two infrared LEDs and a 3DOF (degrees of freedom) tracker are attached on the projecting device.
Infrared LEDs are used to obtain the distance from a camera with an IR transmission filter, using the known geometry of the LEDs.
Information on pitch, yaw, and roll is obtained from from a 3DOF tracker.

The observed pattern of the LEDs is corrected by the orientation of the gyro, so the size of the LED pattern shows the distance from the camera. This combination of the 3DOF tracker and the simple LEDs arrangement enables high-speed tracking and positioning of the projector device.

2. A system to present indirect information (shadows, sounds). To make players feel the goblin's existence, the system provides indirect information such as the shadows and sounds of goblins instead of direct images.

Shadow projector
Each goblin's position is controlled and placed in a virtual space that modeled on a real room. To project a shadow image based on the 3D position of the projector, the rendering viewpoint is set to the projector in the real world. To render the goblin's shadow, the material color of the goblin is set to black, and reflectivity is set to none. The rendered image is masked to create an actual spotlight shape. Dynamic image is calculated by the projector, and the goblin's shadow is shown properly in the real world.
Sounds
When the goblins move, players can hear their footsteps. The volume of the sound changes depending on the goblins' position on the floor. When players capture goblins, they hear the goblins' scream and vacuuming sounds.

3. A capture device. The responses of the goblins are very important. Players feel a sense of capture through vibration weight change. A number of small vibrating motors are fixed in the hose of the device. When players capture the goblin, these motors vibrate sequentially from the nozzle toward the handle. Then a large vibrating motor in the backpack presents a sense that the captured goblin is struggling. At the same time, water is moved from a tank on the ground to another in the backpack, so players feel the weight of the captured goblins.

Vision
When users experience INVISIBLE, they feel that fantasy objects actually exist, as children imagine that their fantasies are real as they listen to fairy tales. The projector displays a 3D object's existence and behavior. Simple location techniques monitor users' intentions and react to them.

The system demonstrates a new technique for integrating the real world and the virtual world. The next generation of this work will create a world in which many people clearly sense the existence of virtual objects and freely interact with and control them.

Contact
Yoshinobu Nakano
Nara Institute of Science and Technology
shadow (at) is.naist.jp

Contributors
Kunihiro Chihara
Masako Fukushima
Kenji Kawashima
Hitomi Komura
Haruka Ouwa
Masayuki Sagano
Taku Shirazawa
Soushi Tujimura
Daisuke Watanabe
Masayuki Yamazaki
Yasuhiro Yoshimura
Nara Institute of Science and Technology

INVISIBLE ~The Shadow Chaser~