Vol.34 No.2 May 2000
Teaching Computer Graphics Visual Literacy to Art and Computer Science Students: Advantages, Resources and Opportunities
AbstractAlthough it is a requisite skill for success in industry, visual literacy in graphics is intimidating to computer science and art students. Computer science majors are uneasy about using their eyes to examine images while art students may not have much background in the technical terminology. This column is the second in a two-part series that discusses an interdisciplinary teaching technique that overcomes these obstacles. Part one was published in Computer Graphics 34(1) February 2000, pp. 24-26. With this approach students become more familiar with the limits and possibilities of the medium of computer graphics, learn how to analyze and talk about what visual images might mean and develop a deeper understanding of time constraints. In addition, they gain confidence with technological terminology and the idea of suggesting alternative algorithms to create a desired visual "look." As a result both computer science and art students become more able to communicate effectively about and with visual imagery.
As mentioned in the previous column, visual literacy is important for both computer science and art students as it is a highly prized skill in the computer graphics industry. The workplace awaiting graduates is an interdisciplinary, team-oriented environment where people skills and empathy are essential for successful collaboration. Visual literacy forms a basis for shared vision and communication. For art students, visual literacy complements their visual knowledge by imparting an understanding of technical terminology and requirements. It gives them a standard and consistent means to communicate to different audiences. Visual literacy also enhances the technical abilities of computer science students because it gives them experience in looking at an image and analyzing what they perceive. It allows them to build a better understanding and appreciation of the visual aspects of computer graphics.
In order to impart a visual literacy to graphics students, the authors use an interdisciplinary approach called visual analysis. Visual analysis is the process of discovering visual cues in an image and identifying the algorithms associated with those cues. The approach draws upon critical analysis ; but instead of emphasizing design, concept and media, it teaches students to recognize a small number of visual cues. These include:
These cues are usually sufficient to identify commonly used rendering algorithms.
In a classroom setting, an instructor gradually introduces cues over a period of three to six weeks, starting with two or three that appear starkly different. For example, a good beginning set of cues encompasses the difference between outlined and filled in polygons and the presence or absence of reflection and refraction. In this stage instructors need to focus on the characteristic visual behavior of rendering techniques to give students the time needed to build their skills. Once students become confident with identifying rendering algorithms based on typical visual behavior, they are ready to tackle visual equivalencies.
Visual Equivalencies and Time
Under certain circumstances, different rendering algorithms can create similar results. This is the phenomenon of visual equivalency. For example, it is possible to use a z-buffer surface algorithm to produce shadows (via a shadow buffer) and reflections (via environment mapping), and the resulting image may appear as if it had been ray traced. When an object has a dull surface, it can be virtually impossible to distinguish Gouraud and Phong shading. Another example is the simulation of a Phong highlight with Gouraud shading and a texture map depicting a highlight. As instructors introduce visual equivalencies, they can also broach the subject of algorithm performance.
Visual equivalencies can motivate discussions on the amount of time needed to render an image. All graphics students need to be able to estimate the amount of time required for image creation. Computer science students often learn about the basic differences in algorithm performance through informal discussions of an algorithm’s time complexity or through examination of bench mark data showing the time required for various algorithms to rendering a given scene.
Another way to understand algorithm performance is through empirically gained knowledge. Art students learn to estimate the rendering time for a single image or frame by rendering a low-resolution version and then estimate the time required for completing a high-resolution image. From there, they can estimate the rendering time for animations of various lengths and frame rates. They are shocked to discover the time requirements for rendering a five-minute ray traced animation.
For students, it is often disappointing to find that a technique capable of producing a desired visual effect is simply not practical for the chosen application. A discussion of possible alternatives based on visual equivalency can follow naturally from this discovery.
Figure 1: A sample TERA session.
Figure 2: TERA in “Self Quiz” mode.
Figure 3: By Lana Lazebnik, DePaul University. Implemented in C and OpenGL.
Figure 4: From Liberation by Hunter Grant, Bowling Green State University.
Figure 5: From Liberation by Hunter Grant, Bowling Green State University.
Figure 6: From Liberation by Hunter Grant, Bowling Green State University.
Students need to practice visual analysis outside of the classroom. While posting images on the Web or pointing out examples in textbooks is a start, students say that they benefit more from the question-and-answer sessions held at the end of a lecture. Simply looking at images does not promote active learning. In fact, students learn best by taking part in the experience, in other words, by doing .
Having students learn a rendering package helps a bit because students can choose parameters and view the resulting images. However it takes a significant amount of time to learn a package, and since the students have a priori knowledge of the surface and shading algorithms, this approach does not provide a student with a means of self assessment. A better approach is to provide students with an easy-to-use interactive tool that can demonstrate any rendering or shading algorithm while encouraging active learning. One such tool is TERA (Tool for Exploring Rendering Algorithms), an interactive program that facilitates comparative study of the visual effects of rendering algorithms . See Figure 1 for an annotated screen dump of TERA.
A student can choose a scene and select a rendering algorithm for any object in the scene. Students can practice visual analysis using TERA. In "Self Quiz" mode, students select a scene, and TERA presents it with each object rendered by a random algorithm. Students then guess the rendering algorithm for each object in the scene. TERA responds with "Correct," "Try Again" or "Close Enough."
The "Close Enough" response is for those cases where multiple algorithms produce similar visual effects. For instance, Gouraud and Phong shading produce similar effects when no highlight is present.
Always available is the "Tell me more" button, which provides specific feedback about a student’s last algorithm selection. When a student is in "Explore" mode, the "Tell Me More" button will activate a pop-up window describing the relevant visual cues. In "Self Quiz" mode, when students get a response of "Close Enough," they can click on "Tell Me More" for an explanation. A pop-up window will list the algorithm they picked and the actual algorithm in addition to a specific explanation of why the two algorithms produced effects that are visually equivalent. For example, if a flat surface is very evenly lit, then Gouraud shading may produce variations in shade that are so subtle that the result looks like constant shading. In this situation, if a student picks "constant" and receives the "Close Enough" response, the "Tell Me More" button will activate a pop-up window containing the relevant explanation. See Figure 2.
The new version of TERA is capable of creating nearly a million images for students to analyze. The images cover surface algorithms, z-buffer shaders, ray tracing shaders, texture mapping, bump mapping, lighting and visual equivalencies.
Along with TERA, the students can do an in-class interactive exercise by taking the terminology from TERA and applying it to actual works of art from well established digital art exhibitions such as the SIGGRAPH Fine Arts Gallery, the SIGGRAPH Electronic Theater or the New York Digital Salon. After they apply terms or describe what they see, they will formally connect the visual cues, interpret them and finally assess the work. During critiques, the students will apply a similar approach to describing their own work and that of their classmates.
Benefits to Students
For computer science students, visual analysis adds excitement and a sense of the "big picture" to introductory graphics courses. Students may not be able to implement every rendering algorithm when they leave the course, but they will be able to recognize them and know their names, which provides a starting point for further investigation. They develop an awareness and a deeper appreciation of computer generated imagery in addition to a sensibility for the technical requirements of the animations that they see in movies and on television.
A visual sensibility, fostered by visual analysis, enhances a depth of knowledge of those rendering algorithms that students do implement. Because they are already familiar with the visual behavior of the algorithm, they have expectations of how an image should appear and can perform more of their debugging on their own without outside help. Students no longer ask, "Is it right?" They state, "There’s something wrong with my highlight."
A greater awareness of visual possibilities serves as an effective motivator. Figure 3 is an example of student’s final project in a programming course where the emphasis was on developing a z-buffer based renderer using OpenGL. The student wanted reflections and shadows in her image, so she took the time to implement a special case of environment mapping and a shadow buffer.
For art students, visual analysis imparts an awareness of the possibilities in visual communication with digital media. Even if students cannot create a large-scale virtual environment or full feature animation, they have a better appreciation of future possibilities. In addition to comprehending the possibilities of the medium, the students will become aware of the limitations. In understanding the limitations, they will find creative solutions to express similar ideas in a different way, thus pushing the digital medium. Not only will the students learn to communicate their ideas verbally, but with practice they will develop and understand their visual intentions. This is demonstrated in Figures 4-6, which depict scenes from recent student work depicting the harsh treatment of those jailed in Tibet.
Images are seen through the eyes, but the perception of them is something that happens in concert with the mind and other body memories. Without background in visual literacy, students often do not express perception and they do not understand why they chose certain formal aspects of their artwork, or even what they are trying to say. After learning about visual literacy, students show an understanding of how their choices impact the perception of the work. When asked about their art, students describe their work through a formal analysis coupled with ideas or meaning instead of, "Well, it looks cool."
Especially helpful for art students would be to develop tools similar to TERA for leaning terminology for a larger scope of digital media including 2D imagery, digital sculpture and time-based works such as 2D and 3D animation, interactivity and non-linear video editing. Such tools would also be enormously useful to computer science students taking courses in multimedia, human-computer interaction and Web development. An additional goal is to establish a taxonomy for critically analyzing digital art work which includes a discussion of meaning, formal qualities and the success of communicating ideas. This would be similar to that of Feldman’s approach , but would incorporate terminology and ideas relevant to digital technology.
Rosalee Wolfe obtained a masters of music from Indiana University before changing majors to earn a Ph.D. in computer science. She is a NASA Fellow, was SIGGRAPH Technical Slides Editor in 1993 and 1995-97 and edited Seminal Graphics for SIGGRAPH 98.
She also authored the 1997 education slide set on mapping techniques, co-created the first B.S. in human-computer interaction (at DePaul University) and is currently Director of the Division of Graphics and Human-Computer Interaction in the School of Computer Science, Telecommunications and Information Systems at DePaul University.
Rosalee J. Wolfe