Siggraph 2005 logo
Conference Exhibition Call For Participation Presenters Media Registration

emerging technologies

Ubiquitous Graphics

This system supports collaboration and interaction with very large computer graphics images. Position-aware handheld displays function as "magic lenses" that allow seamless access to both overview and detail on a large display.

In-place annotations and modifications can be added easily and are propagated throughout the system to facilitate multi-user collaboration.

Art and Science

The Ubiquitous Graphics system is an example of how computer graphics is becoming more and more integrated with the physical world.

Traditionally, we are used to graphics confined to a fixed "box": a stationary computer, a television display, a cinema screen, and so on. The increased capabilities of mobile devices (handheld games, PDAs, mobile phones) are blurring that distinction by literally putting advanced computer graphics into everyone's pocket. But the graphic display still stays in its "box" and has little interaction with the world outside it. A mobile phone is as graphically advanced as a PC was only a few years ago, but it also shares the same limitations: it knows very little about the world around it.

In this project, we show how the addition of position-sensing and networking can break the boundaries between devices, effectively creating a single large, shared graphics surface that is accessible to many users. By mixing various display types of different size and resolutions, we can create a more fluid computer graphics surface and allow for different styles of interaction and collaboration. As the required technology (position-aware handheld devices, large displays, etc.) becomes cheaper and more accessible, there is great potential for creating new applications inspired by this paradigm. While this demonstration does not directly address the question of artistic expression, it points the way to future possibilities in artistic visualization and exploration, as well as educational and scientific applications.


1. Development of novel techniques that allow handheld computers to interact with portions of a large workspace, using a combination of the "peephole" interaction technique and focus-plus context displays. The system should support several users working together, as well as displays of any size and resolution.

2. Creation of a system that can actually be used in a real work environment. Ubiquitous Graphics uses only standard off-the-shelf components.


This project is being shown publicly for the first time at SIGGRAPH 2005.

The fluid combination of many different types of displays is novel. Since the system uses a cross-platform development environment, most portable displays fitted with an off-the-shelf ultrasonic transmitter can be used with it, including: tablet PCs, pocket PCs, Palm devices, smartphones, and other mobile phones. Users can switch between displays at any time, and several people can work in parallel using several displays on the same large-resolution workspace.

Users can add annotations and animated objects that are shared over all devices and anchored in a certain position on a larger projected image. While several systems allow users to simultaneously share the same work area, only this one allows users to add and access annotations directly and physically on a large graphics image. This makes it much easier to work collaboratively on a large image, for instance to work out changes in a plan or to explore an interesting artwork.

To facilitate high-performance graphics on handheld devices we have developed the GapiDraw platform, which provides real-time scaling, rotation, and filtering of images at more than 30 fps on most handheld computers and smartphones. GapiDraw is a current de-facto standard for handheld gaming and has been used in over 150 commercial products.

The system allows multiple users to add annotations and other graphics to the workspace that are propagated across all displays. Users interacting with a handheld display can do that either by using a two-handed navigation technique, or they can work away from the projection and use the GUI of the handheld display to change the display offset on the large workspace. The position of each user's display and the annotations are always shown on the projected background picture and on the other displays.

To enable high-performance networking (to distribute annotations and objects between devices), the system uses the OpenTrek platform, which supports high-performance UDP transfers on portable computers (PDAs or tablet PCs), and will automatically form multicast networks and send guaranteed and secure packets over any wireless connection.


Mobile and handheld devices have become ubiquitous and are now a natural part of daily life. The same can be said for large information displays, which are now seen in most public places. Large plasma displays and projectors are used for a wide variety of purposes, from showing today's menu at a restaurant to presenting an overview of the subway network. This project integrates these two common technologies to create a system that can be used on any networked mobile device together with any large display, in any environment.

New mobile devices are constantly gaining more features; the latest mobile phones even have built-in GPS navigation and wireless LAN for short-range communication. Hardware for ultrasonic positioning might be one possible feature for mobile phones, if sufficiently interesting applications exist. Ubiquitous Graphics might be one such application, and could encourage hardware manufacturers to add positioning hardware to future mobile devices.


Johan Sanneblad
Viktoria Institute
johans (at)


Lars Erik Holmquist
Viktoria Institute