SIGGRAPH 2004 - The 31st international conference on computer graphics and interactive techniques
Conferenece Exhibition Call For Participation Presenters Media Registration spacer
line


Non-Photorealistic Camera: Automatic Stylization With Multi-Flash Imaging
This is a multi-flash camera that can automatically generate stylized images and videos. Strategically positioned flashes cast shadows along silhouettes in the scene. The detected silhouettes are rendered in cartoon style or as technical illustrations. This overcomes the need for per-frame photo editing or 3D scanning of environments.

Life Enhancement
While photographs are the de facto visual medium for depicting reality, for some scenarios it is hard to produce pictures that convey clearly the 3D structure of a scene to the human eye. Consider imaging a white piece of paper with a white background. A traditional camera will record a mostly white image, and the shape of the paper will be lost or difficult to perceive. Our goal is to create enhanced images and video that make it easy for the viewer to understand the relative depth of the objects in the scenes depicted. This non-photorealistic camera is inspired by techniques used by skilled artists and digital illustrators to make images comprehensible: accentuating important features and reducing visual clutter.

Vision
Detecting silhouettes and depth edges in a real scene is a very challenging task. Intensity edges are different from depth edges, so a simple intensity-edge detection in photo editors will not create the same quality of shape boundaries that we generate. We are showing applications in generating stylized images, but the captured depth edges can be used in many other applications. Effects such as depth-of-field effect during post-processing and synthetic aperture using camera array and screen matting for virtual sets (with arbitrary background) require high-quality signed (local foreground vs. background) depth edges.

Many artists have shown a wonderful array of work in stylizing images of videos using photo editors or rotoscoping ("Waking Life," "Avenue Amy"). Our goal is to supply digital artists with powerful scene-feature detection tools, so that meticulous pixel marking can be reduced or eliminated.

Goals
We believe that our camera will be a useful tool for professional artists and photographers, and we expect that it will also enable the average user to easily create stylized imagery. We intend to automate tasks for stylized rendering where meticulous manual operation was originally required, such as image editing or rotoscoping. We also aim to demonstrate and create awareness of the use of silhouettes beyond stylized imaging in other applications.

Our goal is to create stylized images that facilitate viewer comprehension of the shape of the objects depicted.

Innovations
This project shows that it is possible to bypass geometry acquisition and directly create stylized renderings from images and video. Instead of expensive, elaborate equipment for geometry acquisition, we propose using a camera with a simple extension: multiple strategically positioned flashes. Rather than estimating the full 3D coordinates of points in the scene and looking for depth discontinuities, our technique reduces the general 3D problem of depth-edge recovery to one of 2D image processing.

Our approach is based on taking successive photos of a scene, each with a different light source close to and around the camera's center of projection. We use the location of the shadows abutting depth discontinuities as a robust cue to create a depth-edge map in both static and dynamic scenes. We produce enhanced images and videos that more clearly convey the 3D structure of the imaged scene. The depth-edge map can also be used to produce other types of non-photorealistic or artistic renderings.

Presentation
Monday, 9 August
10:30 - 12:15 pm
Room 404AB

Contact
Ramesh Raskar
Mitsubishi Electric Research Laboratories (MERL)
raskar@merl.com

Contributors
Karhan Tan
Mitsubishi Electric Research Laboratories (MERL)

Rogerio Feris
University of California, Santa Barbara

Jingyi Yu
Massachusetts Institute of Technology

Matthew Turk
University of California, Santa Barbara

 
presentations
emerging technologies jury and committee
   
   
REFER A FRIEND
 > share the SIGGRAPH 2004 web site
   
   
 
Conference 8-12 August, Exhibition 10-12 August.  In Los Angeles, CA