VISFILES
Vol.32 No.3 August 1998
ACM SIGGRAPH
VisAD: Connecting People to Computations and People to People
T.Todd Elvins
San Diego Supercomputer Center
Bill Hibbard
Space Science and Engineering Center
University of Wisconsin - Madison
August 98 Columns
T.Todd Elvins
In our last Computer Graphics issue, Greg Johnson contributed an interesting survey article on collaborative visualization. Another perspective on this hard problem is discussed by Bill Hibbard in the following article. Most of you in the visualization community know Bill from his pioneering work on Vis5D and more recently VisAD. As always, I welcome your comments and thoughts on this column.
- T. Todd Elvins
Progress in visualization is driven by physical progress with computers. A key milestone of this progress occurred when shaded graphics could be generated quickly enough to be perceived as smooth motion. This was the basis of a burst of new interactive techniques and systems that focused on visualization as two-way communication between people and computations [1, 4, 5, 6, 8, 10]. Now we have an even more important kind of physical progress with computers: they are virtually all connected together via the Internet. This will be the basis of a burst of collaborative techniques and systems that focus on visualization as two-way communication between people and other people.
Sun's slogan is "The Network is the Computer" and their Java programming language is designed for writing programs that run on the network rather than on any particular computer. We have rewritten the VisAD system [7] as a Java class library for building applications that provide many users with interactive visualizations of a shared set of numerical data and computations. Such applications define communication media between people, specialized for numerical data and computations. A VisAD application is an event-driven network of:
- Data objects: these may be simple real number values, text strings, vectors of real numbers, arrays such as images or grids or complex hierarchies of data.
- Display objects: these generate interactive 3D depictions of data objects that update whenever data values change.
- User interface (UI) objects: these are linked to data objects. For example, sliders are linked to real number data objects while color table widgets are linked to arrays.
- Computational objects: these are computations that are invoked whenever their input data objects change (like the computations in spread sheet cells).
These data, display, UI and computational objects may be linked together independent of their location on the network. VisAD is freely available, including source code, developers guide and example applications, at this website.
Many of the interactive visualization systems developed during the 1988-1992 period include support for distributed computing, some via remote X displays, some as customized client/server architectures, and some, recently, as Java front ends for access to visualization servers from Web applets. In addition to VisAD, other pure Java visualization systems include VizWiz from SDSC, Horizon from NCSA, Visualize from Visualize Inc. and WebWinds from JPL. Habanero from NCSA provides general support for building collaborative applications, including visualization applications.
An Application Example
Figure 1: GoesCollaboration Screen shot.
Figure 1 is a screen shot of the GoesCollaboration application - a spread sheet for experimenting with the way the GOES satellite senses vertical columns of atmosphere using multiple spectral bands. This application is a linked network that includes four display objects (the four graphics windows), four active user interface objects (the four sliders), five computational objects and 16 data objects. The colored graphs in the lower left window are depictions of pressure, temperature, water vapor and ozone in a vertical column of atmosphere. Their displays support direct manipulation, so that users can change the atmosphere profile data objects by redrawing their graphs. This triggers computational objects to recompute the satellite response function depicted in the upper left window, and similarly for the error function seen in the lower right window. The function in the upper right window can be used to visualize changes in the satellite response function corresponding to small changes in the atmospheric profiles, or small changes in solar zenith angle and surface temperature controlled by the sliders on the left.
When UI or display objects on multiple computers are linked to a shared data object, they all maintain a consistent depiction of the data values. The user interface shown in Figure 1 can be replicated on any number of workstation screens and linked to the same set of data and computational objects, allowing scientists at different universities and labs to see the consequences of changes made by each other in the atmosphere profiles and sliders. The GoesCollaboration application is thus a two-way medium of communication between these scientists, focused on the specialized subject matter of the GOES satellite.
General Numerical Data and Display Models
Of course, GoesCollaboration is a simple application with a very task-specific user interface. More generic applications can be developed that allow users to locate data objects via Web browsers, drag-and-drop them into displays, link them into data analysis algorithms and share visualizations of the results with colleagues at other locations. We can even imagine that an atmospheric chemist may use a Web browser to locate a running weather model and connect it to her chemistry model, dynamically creating a distributed and coupled earth system model. She and her weather-modeling colleague may then share visualizations of the coupled model so that they can explore together how weather and chemistry interact.
Such flexibility for sharing access to numerical data and computations on the Internet requires a data model [3, 9] that:
- Can be applied to numerical data from a variety of data sources, from a variety of disciplines (e.g., meteorology, oceanography, atmospheric chemistry) and in a variety of formats.
- Documents how data relate to the physical world so that data in different units, in different coordinate systems and with different samplings can be combined in computations and visualizations.
The VisAD system is designed around its flexible data model and metadata integration. The central metadata of each data object is its mathematical type, which is a kind of data schema. The type defines names for primitive numerical and text values, groupings of these values and functional relations between values. For example, an earth image may have the type:
- ((latitude, longitude) -> radiance)
which says that an image is a functional relation from (latitude, longitude) pairs to radiances. Other metadata associated with an earth image data object may include sampling geometry and topology for (latitude, longitude) pairs, units for latitude, longitude and radiance, a coordinate transformation relating (latitude, longitude) to some reference coordinate system (e.g., a standard Mercator map), missing data indicators for radiance values and error estimates attached to latitude, longitude and radiance values. A time sequence of earth images may have the type:
- (time -> ((latitude, longitude) -> radiance))
with additional metadata for time units and sampling.
To match the variety of data definable via mathematical type schemas, data displays are defined by mappings from primitive numerical types occurring in the schemas to primitive display types. For example, a traditional display of a time sequence of images can be defined by:
- time -> animation
- latitude -> y axis
- longitude -> x axis
- radiance -> RGB (pseudo color mapping)
In VisAD these are called scalar maps and may be regarded as a kind of display metadata. They provide a way to parameterize display designs. The list of primitive display types includes coordinates in a variety of spatial and color coordinate systems, spatial offsets, animation, iso-contour, flow components, data selectors and shape indices. This list is extensible by applications.
Allowing data and displays to be expressed in terms of their primitive elements is the key to achieving the necessary flexibility for widespread sharing of numerical data and computations across the Internet. We are developing packages to adapt data in a number of common file formats to the VisAD data model, including FITS, netCDF, HDF-EOS, Vis5D, GIF and JPEG. In addition to hiding storage format from applications, the format adapters can hide data movement between disk and memory, and data movement across the network.
Bill Hibbard's research interests are interaction techniques, data models and distributed architectures for numerical visualization. He leads the SSEC Visualization Project and is primary author of the Vis5D and VisAD systems. He has degrees in Mathematics and Computer Science from the University of Wisconsin - Madison.
Bill Hibbard
San Space Science and Engineering Center
1225 W. Dayton Street
Madison, WI
53706, USA
Tel: +1-608-263-4427
Fax: +1-608-263-6738
Website
T.Todd Elvins is a Staff Scientist at San Diego Supercomputer Center. He recently finished his Computer Engineering Ph.D. at the University of California, San Diego, and his research interests include perceptually based user interface design, data and information visualization, Web-based imaging and computer graphics. He can be contacted at:
T.Todd Elvins
San Diego Supercomputer Center
University of California
San Diego
MC 0505
La Jolla, CA
92093-0505, USA
Web site
The copyright of articles and images printed remains with the author unless otherwise indicated.
A Toolkit for Interaction and Collaboration Techniques
Event-driven networks of data, display, UI and computational objects define a medium for expressing a wide variety of interaction and collaboration techniques. The basic elements of these techniques are:
- Computational steering feedback loops:
UI or Display -> Data -> Computation -> Data -> UI or Display
- Direct manipulation rendering:
UI or Display <--> Data
- Collaboration:
UI or Display <--> Data <--> UI or Display
- Display design and control:
UI or Display -> Data -> Display Parameters
By display parameters we mean simple display parameters such as 3D view point, animation, iso-levels of iso-surfaces and pseudo color lookup tables, as well as more sophisticated display parameters such as the scalar maps described in the previous section. Scalar maps provide a way for users to interactively explore the design space for visualizations [2].
These four basic elements can be combined in limitless ways to define interaction and collaboration techniques. For example, data objects displayed with direct manipulation can be linked to computations and then back to display parameters, in order to define embedded display controls. And, of course, these controls can be shared among many users.
The next step for VisAD is to define a set of JavaBean components for data, display, UI and computational objects. This will enable scientists to use commercial and freeware graphical Java programming tools for interactively building networks of VisAD objects.
Acknowledgments
My co-authors of the VisAD system are Curtis Rueden, Tom Rink, Dave Glowacki, Steve Emmerson and Nick Rasmussen. John Anderson and Dave Fulker made significant contributions to the design. My co-authors of the GoesCollaboration application are Paolo Antonelli, Bob Aune and Allen Huang. I wish to thank Joe Bredekamp of NASA (NAG5-3943) and Larry Smarr of NCSA for their support.
References
- Bancroft, G. V., F. J. Merrit, T. C. Plessel, P. G. Kelaita, R. K. McCabe and A. Globus. "FAST: a multi-processed environment for visualization of computational fluid dynamics," Proc. Visualization '90, IEEE, 1990, p. 14-27.
- Card, S. K. and J. Mackinlay. "The Structure of the Information Visualization Design Space," Proc. Information Visualization '97, IEEE, 1997.
- Haber, R. B., B. Lucas and N. Collins. "A data model for scientific visualization with provisions for regular and irregular grids," Proc. Visualization 91, IEEE, 1991, p. 298-305.
- Haberli, P. "ConMan: A visual programming language for interactive graphics," Computer Graphics 22(4), 1988, p. 103-111.
- Hibbard, W. and D. Santek. "Interactivity is the key," Chapel Hill Workshop on Volume Visualization, University of North Carolina, Chapel Hill, 1989, p. 39-43.
- Hibbard, W. and D. Santek. "The Vis5D system for easy interactive visualization," Proc. Visualization '90, IEEE, 1990, p. 28-35.
- Hibbard, W., C. R. Dyer and B. E. Paul. "Display of scientific data structures for algorithm visualization," Proc. Visualization '92, IEEE, 1992, p. 139-146.
- Lucas, B., G. D. Abrams, N. S. Collins, D. A. Epstein, D. L. Gresh and K. P. McAuliffe. "An architecture for a scientific visualization system," Proc. IEEE Visualization '92, 1992, p. 107-114.
- Treinish, L. A. "SIGGRAPH '90 workshop report: data structure and access software for scientific visualization," Computer Graphics 25(2), 1991, p. 104-118.
- Upson, C., T. Faulhaber, Jr., D. Kamins, D. Laidlaw, D. Schlegel, J. Vroom, R. Gurwitz, A. van Dam. "The application visualization system: A computational environment for scientific visualization," Computer Graphics and Applications, 9(4), 1989, p. 30-42.
VisAD: Connecting People to Computations and People to People
T.Todd Elvins
San Diego Supercomputer Center
Bill Hibbard
Space Science and Engineering Center
University of Wisconsin - Madison
T.Todd Elvins
In our last Computer Graphics issue, Greg Johnson contributed an interesting survey article on collaborative visualization. Another perspective on this hard problem is discussed by Bill Hibbard in the following article. Most of you in the visualization community know Bill from his pioneering work on Vis5D and more recently VisAD. As always, I welcome your comments and thoughts on this column.
- T. Todd Elvins
Progress in visualization is driven by physical progress with computers. A key milestone of this progress occurred when shaded graphics could be generated quickly enough to be perceived as smooth motion. This was the basis of a burst of new interactive techniques and systems that focused on visualization as two-way communication between people and computations [1, 4, 5, 6, 8, 10]. Now we have an even more important kind of physical progress with computers: they are virtually all connected together via the Internet. This will be the basis of a burst of collaborative techniques and systems that focus on visualization as two-way communication between people and other people.
Sun's slogan is "The Network is the Computer" and their Java programming language is designed for writing programs that run on the network rather than on any particular computer. We have rewritten the VisAD system [7] as a Java class library for building applications that provide many users with interactive visualizations of a shared set of numerical data and computations. Such applications define communication media between people, specialized for numerical data and computations. A VisAD application is an event-driven network of:
- Data objects: these may be simple real number values, text strings, vectors of real numbers, arrays such as images or grids or complex hierarchies of data.
- Display objects: these generate interactive 3D depictions of data objects that update whenever data values change.
- User interface (UI) objects: these are linked to data objects. For example, sliders are linked to real number data objects while color table widgets are linked to arrays.
- Computational objects: these are computations that are invoked whenever their input data objects change (like the computations in spread sheet cells).
These data, display, UI and computational objects may be linked together independent of their location on the network. VisAD is freely available, including source code, developers guide and example applications, at this website.
Many of the interactive visualization systems developed during the 1988-1992 period include support for distributed computing, some via remote X displays, some as customized client/server architectures, and some, recently, as Java front ends for access to visualization servers from Web applets. In addition to VisAD, other pure Java visualization systems include VizWiz from SDSC, Horizon from NCSA, Visualize from Visualize Inc. and WebWinds from JPL. Habanero from NCSA provides general support for building collaborative applications, including visualization applications.
An Application Example
Figure 1: GoesCollaboration Screen shot.
Figure 1 is a screen shot of the GoesCollaboration application - a spread sheet for experimenting with the way the GOES satellite senses vertical columns of atmosphere using multiple spectral bands. This application is a linked network that includes four display objects (the four graphics windows), four active user interface objects (the four sliders), five computational objects and 16 data objects. The colored graphs in the lower left window are depictions of pressure, temperature, water vapor and ozone in a vertical column of atmosphere. Their displays support direct manipulation, so that users can change the atmosphere profile data objects by redrawing their graphs. This triggers computational objects to recompute the satellite response function depicted in the upper left window, and similarly for the error function seen in the lower right window. The function in the upper right window can be used to visualize changes in the satellite response function corresponding to small changes in the atmospheric profiles, or small changes in solar zenith angle and surface temperature controlled by the sliders on the left.
When UI or display objects on multiple computers are linked to a shared data object, they all maintain a consistent depiction of the data values. The user interface shown in Figure 1 can be replicated on any number of workstation screens and linked to the same set of data and computational objects, allowing scientists at different universities and labs to see the consequences of changes made by each other in the atmosphere profiles and sliders. The GoesCollaboration application is thus a two-way medium of communication between these scientists, focused on the specialized subject matter of the GOES satellite.
General Numerical Data and Display Models
Of course, GoesCollaboration is a simple application with a very task-specific user interface. More generic applications can be developed that allow users to locate data objects via Web browsers, drag-and-drop them into displays, link them into data analysis algorithms and share visualizations of the results with colleagues at other locations. We can even imagine that an atmospheric chemist may use a Web browser to locate a running weather model and connect it to her chemistry model, dynamically creating a distributed and coupled earth system model. She and her weather-modeling colleague may then share visualizations of the coupled model so that they can explore together how weather and chemistry interact.
Such flexibility for sharing access to numerical data and computations on the Internet requires a data model [3, 9] that:
- Can be applied to numerical data from a variety of data sources, from a variety of disciplines (e.g., meteorology, oceanography, atmospheric chemistry) and in a variety of formats.
- Documents how data relate to the physical world so that data in different units, in different coordinate systems and with different samplings can be combined in computations and visualizations.
The VisAD system is designed around its flexible data model and metadata integration. The central metadata of each data object is its mathematical type, which is a kind of data schema. The type defines names for primitive numerical and text values, groupings of these values and functional relations between values. For example, an earth image may have the type:
- ((latitude, longitude) -> radiance)
which says that an image is a functional relation from (latitude, longitude) pairs to radiances. Other metadata associated with an earth image data object may include sampling geometry and topology for (latitude, longitude) pairs, units for latitude, longitude and radiance, a coordinate transformation relating (latitude, longitude) to some reference coordinate system (e.g., a standard Mercator map), missing data indicators for radiance values and error estimates attached to latitude, longitude and radiance values. A time sequence of earth images may have the type:
- (time -> ((latitude, longitude) -> radiance))
with additional metadata for time units and sampling.
To match the variety of data definable via mathematical type schemas, data displays are defined by mappings from primitive numerical types occurring in the schemas to primitive display types. For example, a traditional display of a time sequence of images can be defined by:
- time -> animation
- latitude -> y axis
- longitude -> x axis
- radiance -> RGB (pseudo color mapping)
In VisAD these are called scalar maps and may be regarded as a kind of display metadata. They provide a way to parameterize display designs. The list of primitive display types includes coordinates in a variety of spatial and color coordinate systems, spatial offsets, animation, iso-contour, flow components, data selectors and shape indices. This list is extensible by applications.
Allowing data and displays to be expressed in terms of their primitive elements is the key to achieving the necessary flexibility for widespread sharing of numerical data and computations across the Internet. We are developing packages to adapt data in a number of common file formats to the VisAD data model, including FITS, netCDF, HDF-EOS, Vis5D, GIF and JPEG. In addition to hiding storage format from applications, the format adapters can hide data movement between disk and memory, and data movement across the network.
Bill Hibbard's research interests are interaction techniques, data models and distributed architectures for numerical visualization. He leads the SSEC Visualization Project and is primary author of the Vis5D and VisAD systems. He has degrees in Mathematics and Computer Science from the University of Wisconsin - Madison.
Bill Hibbard
San Space Science and Engineering Center
1225 W. Dayton Street
Madison, WI
53706, USA
Tel: +1-608-263-4427
Fax: +1-608-263-6738
Website
T.Todd Elvins is a Staff Scientist at San Diego Supercomputer Center. He recently finished his Computer Engineering Ph.D. at the University of California, San Diego, and his research interests include perceptually based user interface design, data and information visualization, Web-based imaging and computer graphics. He can be contacted at:
T.Todd Elvins
San Diego Supercomputer Center
University of California
San Diego
MC 0505
La Jolla, CA
92093-0505, USA
Web site
The copyright of articles and images printed remains with the author unless otherwise indicated.
A Toolkit for Interaction and Collaboration Techniques
Event-driven networks of data, display, UI and computational objects define a medium for expressing a wide variety of interaction and collaboration techniques. The basic elements of these techniques are:
- Computational steering feedback loops:
UI or Display -> Data -> Computation -> Data -> UI or Display - Direct manipulation rendering:
UI or Display <--> Data - Collaboration:
UI or Display <--> Data <--> UI or Display - Display design and control:
UI or Display -> Data -> Display Parameters
By display parameters we mean simple display parameters such as 3D view point, animation, iso-levels of iso-surfaces and pseudo color lookup tables, as well as more sophisticated display parameters such as the scalar maps described in the previous section. Scalar maps provide a way for users to interactively explore the design space for visualizations [2].
These four basic elements can be combined in limitless ways to define interaction and collaboration techniques. For example, data objects displayed with direct manipulation can be linked to computations and then back to display parameters, in order to define embedded display controls. And, of course, these controls can be shared among many users.
The next step for VisAD is to define a set of JavaBean components for data, display, UI and computational objects. This will enable scientists to use commercial and freeware graphical Java programming tools for interactively building networks of VisAD objects.
Acknowledgments
My co-authors of the VisAD system are Curtis Rueden, Tom Rink, Dave Glowacki, Steve Emmerson and Nick Rasmussen. John Anderson and Dave Fulker made significant contributions to the design. My co-authors of the GoesCollaboration application are Paolo Antonelli, Bob Aune and Allen Huang. I wish to thank Joe Bredekamp of NASA (NAG5-3943) and Larry Smarr of NCSA for their support.
References
- Bancroft, G. V., F. J. Merrit, T. C. Plessel, P. G. Kelaita, R. K. McCabe and A. Globus. "FAST: a multi-processed environment for visualization of computational fluid dynamics," Proc. Visualization '90, IEEE, 1990, p. 14-27.
- Card, S. K. and J. Mackinlay. "The Structure of the Information Visualization Design Space," Proc. Information Visualization '97, IEEE, 1997.
- Haber, R. B., B. Lucas and N. Collins. "A data model for scientific visualization with provisions for regular and irregular grids," Proc. Visualization 91, IEEE, 1991, p. 298-305.
- Haberli, P. "ConMan: A visual programming language for interactive graphics," Computer Graphics 22(4), 1988, p. 103-111.
- Hibbard, W. and D. Santek. "Interactivity is the key," Chapel Hill Workshop on Volume Visualization, University of North Carolina, Chapel Hill, 1989, p. 39-43.
- Hibbard, W. and D. Santek. "The Vis5D system for easy interactive visualization," Proc. Visualization '90, IEEE, 1990, p. 28-35.
- Hibbard, W., C. R. Dyer and B. E. Paul. "Display of scientific data structures for algorithm visualization," Proc. Visualization '92, IEEE, 1992, p. 139-146.
- Lucas, B., G. D. Abrams, N. S. Collins, D. A. Epstein, D. L. Gresh and K. P. McAuliffe. "An architecture for a scientific visualization system," Proc. IEEE Visualization '92, 1992, p. 107-114.
- Treinish, L. A. "SIGGRAPH '90 workshop report: data structure and access software for scientific visualization," Computer Graphics 25(2), 1991, p. 104-118.
- Upson, C., T. Faulhaber, Jr., D. Kamins, D. Laidlaw, D. Schlegel, J. Vroom, R. Gurwitz, A. van Dam. "The application visualization system: A computational environment for scientific visualization," Computer Graphics and Applications, 9(4), 1989, p. 30-42.