DETAILS


COLUMNS


CONTRIBUTIONS

a a a

Non-Photorealistic Rendering

Vol.32 No.1 February 1999
ACM SIGGRAPH

Intentional Non-Photorealistic Rendering



Dan Goldstein
Proletariat Entertainment, Inc.

Non-photorealistic rendering (NPR) is a subject that has been generating quite a bit of interest in the computer graphics community lately. This is partly because any advance in the area is so rapidly picked up by authors who create actual content. The term NPR covers any computer-generated imagery explicitly rendered using techniques designed not to mimic physical reality. Most frequently, NPR rendering is performed by a program that takes images or three-dimensional geometry as input and creates a new output image within the bounds of a given artistic style.

Successful Uses of NPR

Feature films have been using NPR techniques successfully to integrate computer art with various film media, allowing directors to utilize effects never before possible. Even neophyte artists can now quickly and painlessly convert their images to some approximation of a variety of standard artistic media or styles, rendering accessible everything from watercolors to pen and ink to impressionist oil paints. Practically every 3D rendering package now has a cartoon shading plug-in, allowing anyone to display their geometry with a cel-painted look.

Clearly, NPR has empowered many people to create much more than they were once able to. The question is, can it remove more limitations for authors than just those related to style? I believe that it can, and that it is uniquely positioned to do so. Some of the current strengths of NPR include its potential to decrease rendering time by alleviating the burden of extreme detail, its ability to render images or geometry into a certain style potentially matching other works of art and its aptitude at stressing only the important details within the input. These are important characteristics that, when viewed as a whole, point to the possibility that NPR could have a vast variety of new uses in the near future.

Adapting to Artists

At the moment, most NPR rendering programs are analogous to the filters found in any popular image-editing package. They operate on the supplied image or geometry and generate a stylized output image. Although this is a natural choice from a programmerís point of view, when compared to how human artists would generate a painting it seems positively strange. In order to create a work of art, these artists would have to understand their subject matter so that they could include their own interpretation of the important details in their rendering. Computers have never been very good at interpreting meaning, and although a 3D model certainly contains more information than a 2D image, it still doesnít help the computer generate descriptive information about the subject matter.

The natural solution then is not to try to compute a description of the subject, but to start with that description in hand. This does not mean retrofitting externally acquired input materials with descriptive data. It means starting with a description of the artistís intent, generating annotated geometry and then rendering this geometry into an image. In effect, we would like to infect the entire rendering pipeline with knowledge of the artistís intent and the ability to transform data using that knowledge.

Intentional NPR

This may be starting to sound a lot like procedural modeling, which has been around for a very long time and can be a useful tool for generating unique and interesting geometry. Indeed, one could use ďintentional NPRĒ in similar fashion. However, it has some subtle differences that make it different at a fundamental level. With intentional NPR, different geometry would be created at every frame to represent only the parts of the underlying model description currently considered important for the final rendering. This is more than just animated procedural modeling because the geometry generated for each frame would be based on the geometry generated in previous frames, meaning that there would be no single underlying geometry representing the model. Additionally, intentional NPR would not necessarily ever need to generate a set of traditionally specified three-dimensional geometry for rendering, instead creating potentially simpler representations and relying on other intentional components to composite these and render them into a final image.

An important question is why we need non-photorealism in this process of intentional rendering. In order to answer this, we must explore the differences between seeking realism and believability. The main difficulty inherent in striving for realism is that the physical world is a tremendously complex place. In order to render a model realistically, the model itself must approximate its real-world counterpart to some high degree of accuracy, often resulting in extremely complex geometry. NPR attempts to render models believably, rather than realistically. Believability is much simpler to achieve than realism, because a believable model needs only to include those details representative of the intent behind the model. It stands to reason that it should be simpler to render the intent of an intent-based model than the intent of a model based on complex geometric measurements, since for the latter the computer would need to first infer the intent from the geometry itself.

The end result of an intent-based rendering process would hopefully be that the user/audience would be presented with art that focuses on user experience rather than physical correctness. Such a goal is very attractive, especially considering that most of the digitally created worlds presented to users today are not and do not even try to be physically accurate.

The Search for New Models

In our search for new models we must focus on abstracting the intent from the geometry of our scenes. Some current NPR points to this, using silhouettes to designate figures and simple shading strokes to indicate texture, curvature or lighting conditions. Certain procedural models do so as well. Spline and patch representations accomplish many of the goals of intentional NPR, allowing the author to specify the intent to display a certain curvature, which is then translated into simple line segments or polygons at render time. Procedural textures, too, allow the author to specify intent, this time related to the type of material used for an object in a scene. While none of these examples fulfills all of the requirements of an intentional NPR model, together they show us that we can exploit the fundamental assumptions behind current methods to create a rich framework for intentional NPR.

One of the most important of these assumptions is that computers can often effectively represent a family of conceptual looks with a simple set of parameters. From this we can imagine a framework for intentional NPR in which the author would create parameterized entities to place in his or her world. These entities would for every frame of animation calculate a new geometric representation for themselves, annotated with information about intent. This representation would be passed on to a higher level entity, and the process would continue until a final image was produced by the root entity of the tree. An example would be a fur entity placed on a wolf, put in a pack and placed into the world.

An exciting possibility is that a large library of such intent programs could be built by people around the world and shared among authors. The result of this would be that anyone could make parameterized works of art in a variety of styles, tailored individually to the authorís tastes. Although the authors might be using the same intent programs, this would not limit their creative abilities or make their works look similar to those created by anyone else. The parameterized nature of the programs along with layering of multiple programs would allow for nearly infinite possibilities, and provide a means through which oneís work could achieve true individuality while retaining its ease of creation. Were such a system to be usable for creating real-time interactive scenes, animations or even just static clip-art, users would gain the ability to make art significantly more quickly and easily than ever before.

Dan Goldstein has worked at the Brown University Graphics Research Group for several years doing research on new methods for real-time rendering and behavior simulation. He was a co-author of the first SIGGRAPH paper on real-time NPR, and has consulted for Microsoft on such topics as real-time behavior specification and simulation. He recently founded the new game company, Proletariat Entertainment; the demo for their first game should be publicly available by press time.

Dan Goldstein
Proletariat Entertainment, Inc.
Website

The copyright of articles and images printed remains with the author unless otherwise indicated.

Focus on New Techniques

Although decreased content creation costs, increased productivity and computational efficiency are all important goals of intentional NPR, possibly the most important point of all is that it can give artists completely new techniques, allowing them to create things that were never before possible. One important example of this is the human figure. Attempts to model the complexities of the human body with computers in a visually realistic way have largely failed. However, the human body is probably the single entity we would most like to place into our virtual worlds. Professional animators can sketch out a believable non-photorealistic picture of a person in seconds, given an idea of the personís general characteristics. With intentional NPR, computers will be able to do this too, giving artists easy access to characters and character animation that will change the way we use our computers forever.

Intentional NPR is a technology whose time has come. Research has focused on simulation for too long, and it has been at the expense of usability. A new form of image generation technology is necessary, and this technology will integrate the previously separated modeling and rendering processes into a cohesive whole. These new models will be self-contained artistic units, able to transform themselves into images in a series of steps by relying on intentional information and working together with the other units in a scene.

But before this can happen, a lot of research must still be accomplished. We must explore the field of intentional modeling, expand the gamut of NPR techniques by studying traditional art forms and methods and create object-oriented algorithms for synergistic component-based rendering. Hopefully, research in these areas will eventually culminate in the creation of a unified and highly functional framework for intentional non-photorealistic rendering.