Vol.32 No.2 May 1998
Introduction to Collaborative Visualization
As the general state of the world's wide area computer networks improves, resulting in higher bandwidth and lower latency connections, effective modes of computer-supported cooperative work (CSCW) are approaching realization . The term “collaborative visualization” refers to a subset of CSCW applications in which control over parameters or products of the scientific visualization process is shared. Examples of the potential of these systems are extensive. Consider a molecular biologist working with a researcher at a remote pharmaceuticals company to find potential docking sites for a new drug, by cooperatively studying a shared 3D representation of the target protein molecule . Or perhaps a pair of radiologists compare their findings by cooperatively controlling the view and tissue types displayed in a volume rendering of an ultrasonic scan of the patient .
The development of collaborative visualization systems is complicated by challenges inherent in the multi-user nature of these applications. Some of these challenges match issues shared by programmers of multiprocessor computers. In addition, part of the burden of a collaborative system is shouldered by the user, who must adjust work habits to accommodate this new medium. Due to the complexity involved, compelling reasons must exist that overwhelm the additional development and deployment effort required.
TeleInViVoTM: Towards Collaborative Volume Visualization Environments, by J. Coleman et al, offers reasons why collaborative visualization is compelling .
In this column, I will give an overview of a handful of collaborative visualization applications. Each is described in the context of its solutions to a few of the issues facing both the users and the developers of collaborative systems. In the process, multiple interpretations of the term “collaborative visualization” will be presented. Equally valid, these interpretations differ primarily in the nature of the visualization component(s) shared.
Traditionally, the visualization process consists of a cyclic progression of filtering raw data to select the desired resolution and region of interest, mapping the result into a graphical form and producing an image, animation or other product. The result is evaluated, the visualization parameters tweaked and the process run again. Applications exist today that provide visualization capabilities in a collaborative context. These systems can be categorized by the level of shared control they provide over the visualization process.
|Figure 1: A simple collaborative visualization system in which an image product is viewed by multiple participants.||
In its simplest form, a collaborative visualization application consists of image data rebroadcast to all participants, as shown in Figure 1. Only the individual creating the image(s) has direct interaction with the visualization process. The other participants are limited to passive viewing of the results. Feedback might be exchanged between participants via a telephone conference call, or by means of audio, video and whiteboard teleconferencing software.
Local Control with Shared Data
A more complex variation is represented by applications in which participants can share data from any step in the visualization process. Direct interaction and control over the visualization process occurs locally, but raw, partially or fully processed data can be shared.
|Figure 2: Colormap data are intercepted in an Application Visualization System (AVS) network by the collaborative module share colormap, which propagates the data to the respective share module in the AVS networks of other participants.||
This level of cooperative visualization is one aspect of the collaborative Application Visualization System (AVS) modules under development at the San Diego Supercomputer Center (SDSC) . These modules are designed to augment the capabilities of AVS to support collaborative behavior in a flexible manner. One characteristic of the modules is their ability to intercept data at key points within an AVS network as shown in Figure 2, and broadcast the data to the respective modules in the AVS networks on the hosts of the participants.
Limited Shared Control
Another category is represented by applications in which participants can share the viewpoints of others, and insert items into this shared view. Cooperative control of the visualization process is primarily limited to annotation of the resulting visual elements, and control of the view position. Systems of this type include CSpray from the University of California, Santa Cruz and Hewlett-Packard Labs, and the prototype of the Molecular Interactive Collaborative Environment (MICE) from the San Diego Supercomputer Center [2, 6].
CSpray is a collaborative version of the Spray visualization system, which uses a “spray paint” can metaphor to insert “smart particles” into a data volume workspace to highlight regions of interest. When activated, these particles take on the form of graphics primitives such as polygons, spheres and streamlines. CSpray allows participants to share the view of the workspace from the perspective of any other participant, control the visibility of visualization primitives within the workspace and provide annotation of these elements in the form of arrows, labels and sound bites.
The MICE prototype includes a customized VRML browser in which collaborative actions can be attached to objects in a scene by means of a behavioral scripting system. The MICE prototype allows participants to view molecular models in a shared scene, control the view position of others and annotate scene elements with colors, shared pointers and HTML documentation.
Fully Shared Control
Other collaborative visualization applications are those that provide shared control over the parameters associated with a given visualization. These parameters may affect how a dataset is filtered, mapped to graphical elements, viewed and aspects of the products output. Thus the work of steering any aspect of the visualization process can be a shared activity.
|Figure 3: The collaborative AVS module share geometry shares the value of the iso-level dial widget in the menu of the isosurface module connected to it, with the respective isosurface module in the AVS networks of other participants.||
Cooperative interaction of this type is another mode in which SDSC's collaborative AVS modules can operate. In this context, a change in the value of any parameter in the module directly upstream of a collaborative module (see Figure 3) is broadcast to the respective modules in the AVS networks on the hosts of the participants. There it is used to update the respective upstream parameter. Control over most parameters in any AVS module can effectively be shared across all participants.
A related application is the Shastra collaborative multimedia environment from Purdue University . Shastra includes components for managing distributed applications, initiating and maintaining collaborative sessions, supporting audio and video teleconferencing and performing scientific visualization. The Poly volume visualization tool of the Shastra environment provides users with shared control over both the view position and the parameters that control the volume rendering process.
Challenges Facing the User
Collaborative visualization systems can be further categorized by the way in which they evolved. Some began life as visualization packages, and were later extended to support collaborative behavior, such as CSpray, SDSC's collaborative AVS modules and TeleInViVo. Developmental issues including limited access to the underlying event model of the original visualization software can restrict the scope of the collaborative behavior available in these systems.
Other applications, including MICE and Shastra, were built from scratch with collaboration in mind. This type of system often provides focused visualization capabilities as a result of limited development resources, or targeting a specific audience. Thus one challenge to the user is to find a system with suitable collaborative and visualization capabilities.
Further, fundamental collaborative capabilities are often insufficient to enable participants to carry on a natural level of interaction with one another. In many cases traditional audio, video and whiteboard teleconferencing software provide an indispensable component of the collaborative session.
Challenges Facing the Developer
Developers of CSCW systems are confronted by challenges resulting from necessarily coordinating the activities of multiple simultaneous users. Throughout development, all instances of the application in a given collaborative session must be treated as a coherent whole. For this reason, many of these challenges are akin to those faced by programmers of multiprocessor computers. Three of the most significant include maintaining the coherency of shared data across multiple users, synchronizing the activities of these users and optimizing for network bandwidth and latency.
Shared memory multiprocessor computers are systems in which main memory is globally addressable by all processors. Often each CPU is also equipped with a small amount of local cache memory. Thus one issue is how a given processor can guarantee that a cached value of a global variable is current, since at any time another processor could update this variable in main memory. In other words, the problem is how to guarantee that everyone has the same value of a global variable.
This issue is similar to the problem of maintaining the coherency of data shared across multiple participants in a collaborative session. Many systems handle this by tracking the current state of the shared data, and initializing new participants with this state. An order of execution is then enforced that prevents an update from overwriting the current state until previous updates have been propagated to all participants.
For example, when a new user joins a CSpray session, a session manager updates the user with a list of current participants, associated view positions and visualization primitives visible in the shared workspace. Similarly, when a new instance of a collaborative AVS module contacts a server, it is automatically sent a copy of the most recently broadcast message. Further, a change in a given AVS network that modifies the value of a shared item does not immediately affect that network. Rather the change takes effect locally only after the server indicates its successful broadcast to other participants.
The actions of multiple processors in a parallel computer must be coordinated to avoid contention for shared resources. So too, developers of collaborative systems must synchronize the activities of multiple participants to avoid contention for shared items.
Synchronization can take the form of explicitly regulated access to shared parameters and data. Often visual cues are provided to each participant to denote local control, or lack thereof, over a shared item. The MICE prototype employs explicit synchronization to share control over the view position in a coordinated manner. Keyboard commands are used to gain and release control of the view. Only the participant currently in control can release the view. All participants are notified of changes in the state of their access by the color of a camera-shaped icon.
Synchronization can also occur implicitly on a first-come first-served basis, relying on the assumption that participants will conduct themselves in a civilized manner. Control over visualization parameters shared with SDSC's collaborative AVS modules is implicitly synchronized. Parameter changes are sequentially serviced by the collaboration server in the order in which they are received.
Optimization for Bandwidth and Latency
Just as the overhead of interprocessor communication can reduce the efficiency of applications on parallel computers, so network bandwidth and latency limitations reduce the ability of physically remote users to interoperate effectively. Responsiveness in a collaborative system is often no better than the slowest participant's link to the collaboration server.
The developer can do little directly to improve bandwidth. However the effects of limited bandwidth can be ameliorated by carefully selecting the type of data to share between collaborators. For example, broadcasting changes in the parameters that control how a dataset is visualized generally requires considerably less bandwidth than broadcasting changes to the actual dataset. As described previously, SDSC's collaborative AVS modules provide the ability to share the values of visualization parameters.
Alternatively, bandwidth requirements can be reduced by compressing, downsampling or cropping the shared dataset. TeleInViVo, a collaborative volume visualization system oriented towards medical applications, has the ability to compress volume data for transit between participants . Further, this application can initially present a participant with a downsampled version of a dataset of interest, allowing the user to select a subvolume which is then shared at higher resolution.
Finally, prediction and culling schemes can help to reduce the aggregate number of messages sent between collaborative servers and clients. In the case of prediction schemes, the sending client attempts to predict the value of a rapidly changing variable (based on the range of values, rate and number of changes), periodically updating the current state of shared data based on these predictions.
Greg Johnson is a Senior Programmer/ Analyst at the San Diego Supercomputer Center, and holds a M.S. in Computer Science with a computer graphics emphasis. His interests include interactive visualization on HPC systems, collaborative visualization and parallel volume rendering.
T.Todd Elvins is a Staff Scientist at San Diego Supercomputer Center. He recently finished his Computer Engineering Ph.D. at the University of California, San Diego, and his research interests include perceptually based user interface design, data and information visualization, Web-based imaging and computer graphics. He can be contacted at:
In the case of culling, either the sending client or the server itself may regulate the propagation of a given message based on the degree to which it changes the current state of shared data. When a participant in a TeleInViVo collaboration makes a change to the local user interface, the scope of the change is examined. Based on this analysis, no message, a small message containing an incremental update or a larger message containing a full update of a shared entity is sent to the session manager for rebroadcast.
The similarities between the issues associated with developing collaborative systems and programming multiprocessor systems suggest that innovative solutions might be borrowed from the body of work already done in the area of parallel computation. Suggested reading includes an excellent text by Kai Hwang which covers the memory coherency and synchronization problems, common solutions and latency hiding techniques, and is representative of the efforts in this field .
The challenges facing the users and developers of collaborative visualization systems often conspire to prevent them from becoming widely used. Nonetheless, the considerable potential for applications of this type will drive continuing study in this area. Additional development effort should be focused on extending the capabilities of widely used visualization systems to support collaborative behavior. With these applications to provide a bridge between conventional visualization systems and advanced collaboratories, scientists will have a means by which to evaluate the benefits of CSCW in the context of the familiar, and a direction in which to head as additional collaborative capabilities are required.