SIGGRAPH 2008 > For Attendees > New Tech Demos > FingerSight

Browse Program

By Format

FingerSight: Fingertip Control and Haptic Sensing of the Visual Environment

Theme: Future History
Hall H

FingerSight is the underlying concept for a visual sensing device with haptic feedback that allows users to manipulate remote objects by gesture. The current model utilizes a small finger-mounted video camera to track graphical controls on a computer screen. Vibro-tactile feedback alerts users when they have locked onto a control.

Enhanced Life
Because visual information is relayed through the sense of touch, FingerSight has potential application for the visually impaired. Haptic feedback allows users to receive additional information about their surroundings without interfering with auditory cues.

The basic goal is to create a novel application of machine vision and haptic feedback to enhance sensing and remote control of the environment by blind and sighted individuals. Technical goals include recognizing gestures over arbitrary visual fields, simultaneously maximizing information bandwidth and minimizing intrusiveness of haptic feedback, and incorporating multiple cameras.

FingerSight allows users to sense and manipulate completely virtual objects with a single finger. The system can be extended to multiple cameras, one per finger, each with different specifications, allowing synergistic merger of information. This supports capabilities such as 3D reconstruction, color coding, and simultaneous wide-range and focal views.

FingerSight could empower the low-vision population to interact with the distal environment in a new way. Sighted users can manipulate objects beyond their reach, so, for example, they can safely interact with remote targets in sterile or hazardous environments. Educators and gamers can use the technology in training simulations and virtual-reality systems.

John Galeotti
Roberta Klatzky
Mel Siegel
Carnegie Mellon University

Samantha Horvath
Brock Nichol
George Stetten
University of Pittsburgh