Author - Mark Chavez, Assistant Professor, Nanyang Technological University - School of Art, Design and Media

It seems like it was practically just a few years ago that I was living Los Angeles, California, commuting on the freeway and working on "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe". I left that project upon the completion of my tasks to join in setting up an art school at a technological university in Singapore.

It was a good respite from years of production work in “Hollywood.” Not that I disliked working in the visual effects and animation indus­try, quite the contrary. After working in computer animation for the past 25 years, first in Los Angeles, then overseas in Tokyo, in New York and finally returning to Los Angeles in 1995, entering corporate animation production (and salary) was a great opportu­nity to do interesting work with teams of talented people from many different backgrounds: artists, animators, software developers, research scientists and of course experienced and capable pro­ducers. Though production work was engaging particularly in situations where we were developing pipelines and techniques, there is a limit to how many times I could do the same or similar thing and find it interesting. Though I did live and work in Tokyo years earlier I had not considered the possibility pursuing my interests in South East Asia until I looked at the broader scope of career opportunities in an academic and research environment.


Working on Stone Mountain animation at Lasermedia in 1983

The Road to Art, New Media and Animation

My grandparents moved to the United States from Chihuahua, Mexico at around the time of the Mexican Revolution, the time of Pancho Villa and Zapata. I was born in El Paso, Texas as were my parents. My family moved to southwestern Arizona when my father took a job at the Yuma Proving Grounds. He worked on missile testing. I remember an interesting test he did in­volved setting up an array of cameras to photograph a missile launch in slow motion. It looked like a technique currently called time splice photography. My mother was an art and elementary school teacher. She was an avid painter. I completed high school in Yuma and went on to study at Ari­zona State University getting a Bachelors of Fine Arts degree specialising in Drawing. Afterwards, I moved to California to get a Masters of Fine Arts in Animation from the University of Califor­nia, Los Angeles.  I studied under John Whitney Sr., Shirley Clarke and Dan Mclaughlin. While pursuing my MFA and leveraging the advantages of living and studying in Los Angeles, I entered the animation industry doing a part time work while pursuing my studies and in 1980 I started working at a small company that specialized in projected laser light shows.  I was pretty new at animation as were those who joined me with most of us in our early twenties this was the early 1980’s. We working in a new media with computers to make art; a team of artists and software developers creating animation and input techniques to aid in the projection of hand drawn anima­tion and artwork. With this media we were able to project in laser light displays of hand drawn and computer aided animation to audiences of more than 100,000. Generally speaking the venues were comprised of rock and roll performances and other spectacle projected onto unusual surfaces like buildings on the Sunset Strip in Hol­lywood and a mountain of granite in Stone Moun­tain, Georgia. There was nobody to show us how to do this stuff. We had to invent it. One of the notable guys I worked with at Lasermedia was David Silverman. Among other directorial feats he went on to direct The Simpsons for more than 20 years.


MFA thesis project projected onto the Federal Building in Westwood, California during the 1984 Summer Olympics

I worked at Lasermedia for six years until my mid-twenties. Unfortunately because laser animation could not be easily filmed, videotaped or reproduced I was unable to record the mass of work I did during that time. The only thing I was able to bring out of it was my MFA thesis film, of which I did as a independent project during after hours. To record onto film, though I had sought and gained permission, I had to surreptitiously go in late at night without anyone around or knowl­edge of my doing so with a camera borrowed from UCLA. It was guerilla film making in progress. I projected onto a wall and shot the work frame by frame. The only thing that gave me away was the evening I filled the room up with smoke and  shot the projected image onto fabric. The short, projected in laser-light was based on Mexican iconography and played as though in real-time. Be­cause the company had not actually commissioned a short subject project to be made or thought that this would be of interest until my short, my student experiment was used as a marketing tool and showcased at venues including the side of the Federal Building in Westwood, California dur­ing the 1984 Olympics.

The media of laser animation never reached it potential. It was weighed down by the cost of the lasers and the type of venues it was expected to be presented in, the occasional motion bill­board, discos, rock concerts and trade shows. The media was well on its way to the kitsch was largely relegated to planetarium light-shows and scored to 70’s psychedelic music. Though I have seen dazzling light shows in unexpected places in China and other remote parts the world, as a developing media for animation it has largely faded away.

The Emergence of Digital Media and Computer Graphics

Laser animation was my first taste of computer animation in the world of emerging new me­dia. I was an early adopter of digital techniques. This was at that time prior to desktop publish­ing. There were only a few places working in digital media for animation in Los Angeles, primarily Digital Productions and Robert Abel and Associates. At about the same time the movie Tron was in production. From these early career experiences what drew me to digital animation was the fact that in doing this kind of work I needed to always improve my artistic skills both technically and aesthetically. It necessitated that I keep up with technology but at the same time keep my artistic perceptive abilities honed. This constant honing of skills spurred me to look for venues to explore research more directly. This is one of the things I mention to students here in Singapore, you never stop having to recreate yourself and your skillsets. It is an absolute necessity to study and improve.


Still from a never completed short film "Beanville", 1991

In about 1987\8 I received a grant to do a short film with a Symbolics Computer Graphics system. This was just after they created Stella and Stanley. Animation at that time was created by animating hierarchies. There were other artists working with computers to make animation at that time, some more successful than others in the long run. I had a story idea with a storyboard and letters of recommendation and  pitched my idea to do a short film to the Symbolics Graphics Divi­sion. They provided me with a system. It was and old 3600 called Orange, as large as a big refrigerator, hot and extremely power consump­tive, quite a beast. Notable however is that it had an email system and an integrated graph­ics package.

I worked with this system for a number of years in a little closet at UCLA developing a short film. At about the same time I worked in the 2D animation industry on Bill Kroyer’s Fern Gully and other projects. Unfortunately Symbol­ics Graphics Division and the company itself did not survive the competitive times during the late 80’s and I was unable to complete the project.

The last bastion for Symbolics however was Japan. With the help of Nichimen Graphics and people like Mr. Kobayashii the manager of the group, I worked at Tokyo Broadcasting System Vi­sion with a group of great people. I was the only foreigner in the entire large company and had to learn how to communicate quickly. While taking intensive Japanese courses and us­ing drawings and Pentel color cards I was able to create story pitches and other ideas for animation used in game shows and commercials as well as other media. In the evenings and during down times I worked on my short film. Ultimately though I had three minutes completed in 3D layout, I was only able to render about a half minute of the project.


Working in Japan on a Symbolics Ivory XL1200, 1991

Growing Hand in Hand  with Computer Animation

After two years in Tokyo just as the early 1990’s financial bubble popped and work appeared to start to dry up I moved back to the states and took work in a game company that did some of the first and best motion capture at the time, Acclaim Entertainment Inc., a now defunct game company in Long Island, New York. As part of the Advanced Technologies Group we pioneered many motion-capture techniques with real-time toolsets on one of the first Playstation 2 games to use the technique. Again this was all new to us... we were creating methods and techniques to utilize optical markers to move characters around. The limits seemed endless. Artistically, what seemed possible appeared to be limitless. A new way to mimic human motion, in the least to the animator in me seemed to be a great way to study motion. Our contribution to the Electronic Thea­tre at SIGGRAPH Orlando was a stunning success in that people could not tell if the characters moving around on the screen were photographed or not. In retrospect looking at the imagery we created they were obviously captured and rendered, it was a good start.

Afterwards some of the people I worked with went on to achieve greatness in the motion cap­ture business. Notably, Remmington Scott went on to work and innovate on a number of projects including “Final Fantasy: The Spirits Within,” “Spiderman3” and “The Watchmen”.

I moved on from Acclaim and the east coast of the United States to return to the west and California to join an exciting new start-up company in 1995, Dreamworks SKG. I worked on their four first 2 1/2 D feature films. My most of my work was in creating crowds and other shots with 3D that looked like hand-drawn artwork. These films included “The Prince of Egypt” and “El Dorado” among others.


A crowd scene I worked on for “Prince of Egypt”
(courtesy Dreamworks Feature Animation)

This was a fun time. Dreamworks was looking for their first big runaway hit, In development a movie called Shrek was being batted around with experiments being done by motion capture advocates and small elite groups of dig­ital artists. Everybody involved at the beginning, when Dreamworks was up in the Lakeside stu­dio building next to the Universal back lot, from artist to engineer to production coordinators, was tightly focused on the goal of creating a new animation studio with a unique vision and approach towards the art. With many visionary computer scientists like Lance Williams among others as well as veteran Disney animators like Don Paul and Dan Phillips, we embraced technology and tackled the challenges with the full force of corporate funding aided by the magic of Dreamworks with their founders Steven Spielberg, Jeffrey Katzenberg, and David Geffen fueling our visions.

I had worked at Dreamworks for eight years... a long time to work in one place doing this kind of work. I moved over to live-action visual effects. The new place was another large company, Rhythm and Hues Studios, Los Angeles. At this long established and venerable company though working there for only two and a half years, I was able to chalk up 7 feature films to my credit list from Daredevil to Elf and Narnia. There I was able to work with many veteran digital artists some who had worked in the same company for their entire career. The experience was great, the com­panies work-flow had long ago been established and worked well. Their persistent drive to create better special effects for live-action feature films given the competitive nature of the industry is inspiring.

Exploring Realms of Virtual Reality

Sometime during the middle of this period, before leaving Dreamworks at around 2000, I began toying with a real-time 3D character animation pipeline and display software created by a friend’s company a few years earlier. The software allowed me to create real-time 3D characters for the web that when integrated with chat bot software could talk with the visitor to the web site in a close and familiar way. The system was designed to display a talking and emoting head only, though bodies could be added they were difficult to animate. As well there were limited toolsets for moving the body and no camera controls. Most of the work created during this period was largely influenced by the commercial nature of the work I was doing in Hollywood.

The characters that I created took on sensuous volumes that are so prevalent in the depic­tion of human characters in corporate animation. In the animation industry the term appeal is most commonly used, “...a quality of charm, pleasing design, simplicity, communication, and magnet­ism.” (Frank Thomas & Ollie Johnston 1981) I conceded that the character must be attractive.


3D Chatbot from CLONE3D.net

Though I had considered creating a talking dog, instead I used a familiar characteristic form based on a female human. Designing appealing humanoid shapes and in particular female char­acters that can be visited by anyone on the internet, inevitably will draw criticism. I recall at one conference I attended someone complained that the characters mouth is often slightly open. Admittedly, the character that I put online drew a fan base whose discourse was leaned toward provocative tones, the responses that I wrote however though anticipating this kind of discourse was not designed to encourage it. Mostly the character told bad web jokes and taunted the person visiting the site.


Text from online conversation:

Human: hello
Nadia: Hi. Can you see me in the window to the left?
Human: yes i was
Nadia: So, who opened that first ‘oyster’ and said “My, my, my. Now doesn’t ‘this’ look yummy!”.
Human: you’re very beautiful
Nadia: You are persistent now aren’t you?
Human: i like you
Nadia: I like to make new friends.
Human: i’ll be your friend
Nadia: You really want a virtual doll as a friend?
Human: yes i do
Nadia: Psshaa! Really.
Human: yes really
Nadia: How about that. For real.
Human: would you like to know anything about me?
Nadia: Yes, please tell me about it.
Human: what would you like to know?
Nadia: I put tape on the mirrors in my house so I don’t accidentally walk through into another dimension.
Human: may i kiss you?
Nadia: You might get an electrical shock if you tried that.
Human: i love you
Nadia: How do you make an egg laugh? Tell it a yolk.

The unique thing about the way this character was presented is that it used an “Emotion En­gine” to animate from sub-pose to sub-pose. The engine enabled the character to switch through a number of base expressions creating the illusion of process thinking. Using chaos theory and parametric animation techniques as a basis for motion, the software developed by Haptek Inc. cre­ated a animated character whose motion design resulted in ever changing character performances that could be shifted to express particular emotive characteristics, like confused/happy, confused/angry and other more generic emotive types while performing a prerecorded audio track or read­ing via text to speech. This experimentation led me to plan forward to where animation technol­ogy could potentially go in a few years. I decided to pursue these interests directly, focusing my personal creative goals, leveraging the MFA I had received from the UCLA Film School to take a position in a research university. Once I joined academia this field test of sorts, putting a charac­ter online strangers for to interaction with, led to exhibition works and a number of papers and my current research thread. How design motivates emotive reactions in the viewer and how these responses can be manipulated with time based media is a topic that currently fascinates me.

Entering Academia and Research

I came to Nanyang Technological University (NTU) in March of 2005. NTU is Singapore’s sci­ence and technology university, rated in the top 100 of the world’s universities in the latest Times Higher Education-QS World University Rankings. The opportunity to work in a  technological university whose primary focus is research and the support of teaching through research, is largely what drew me here, this and the ability to work with the best and brightest in the region, creating something new in my area of focus. Interactive Digital Media (IDM) is an area of strategic focus in Singapore with the goal of enhancing its creative industries to world class standards. Their plan is to develop research with a facility to produce an industry that can sustain aspects of new media in the service of Education; Animation, Games & Effects; Media Intermediary (“Mediary”) Services; and “On-the-Move” Technologies. The government has committed hundreds of millions of dollars to research to this area with a larger national R&D spending target of at least 3% of GDP by 2010.


In Kuala Lumpur, Malaysia with the Singapore
SIGGRAPH local chapter

Upon my arrival, I was quickly im­mersed into the construction of a new school, writing curricula and research proposals, teaching and giving presen­tations. Though there was a degree of activity in the region with a couple of 3D feature animated films in production and more interesting commercial work, the quality level was far below what I was used to. At about this time I traveled throughout the region with the local Singapore SIGGRAPH Chapter. We visited Malaysia, Thailand and Indonesia where we made presentations on research ac­tivities and goals as a local SIGGRAPH chapter. These activities ultimately led to me chairing the SIGGRAPH Asia Educators Program in 2008 a truly rewarding experience.

Nanyang Technological University - School of Art, Design and Media

Currently instruction in the School of Art, Design and Media at NTU offers a Bachelor of Fine Arts in 6 areas of specialization. Digital Animation offers an advanced programme focusing on animation in all its forms; Digital Filmmaking offers a comprehensive education in the theory and practice of narrative film production; Interactive Media offers a comprehensive curriculum in the theory and practice of the interactive world; Photography and Digital Imaging as a programme offers an in-depth exploration of traditional, experimental, and technological genres of imaging through the photographic medium; Product Design offers a curriculum in design methodologies and an environment conducive to innovative thinking; Visual Communication presents a com­prehensive curriculum, preparing students for the challenges of an increasingly amorphous field.


The Courtyard
School of Art, Design and Media, Nanyang Technological University

The area of my specialization, animation includes stop-motion and traditional animation as well as visual effects, 3D and other con­temporary animation approaches. Although just having graduated our first batch of undergraduate stu­dents they are winning awards in festivals internationally. The school is expanding as well as we move forward and begin to do the work necessary to establish a graduate school. We are well on our way to being a significant voice in the arts regionally and internationally. NTU’s mandate toward academic pursuits is largely focused on research.


With students from the first graduating class of the School of Art, Design and Media, Nanyang Technological University (photo courtesy Ang Yixin)

Having worked in start-ups that developed new animation techniques and approaches during most of my career and having to deal with research in working situations the opportunity to pursue my own research interests was a strong motivating factor that led to my coming to Singapore and an academic career. The Singaporean research environment offers many funding opportunities. I was fortunate to have become a recipient of a National Research Foundation grant.

My current research project is called Cinematics and Narratives (CaN). Our focus is on creat­ing a real-time professional movie authoring tool set, a system that changes the content of a narra­tive stream to manipulate the emotions of an audience. It is essentially a real-time animation movie system that adapts the aesthetic design match an emotional design of a beat-board… that uses sensors to estimate the viewer state of being, whether they are feeling the same feelings as the board prescribes. If the detected state of being is off target then the design changes to influence the viewer to have the planned response. The research is design centric and will result in both new media exhibitions and fixed cinematic content. The system is open-ended and will extend to other applications.


Screen capture from Prototype1.1

Currently at one year into the project with a small staff of artist and engineers, we have de­veloped a solid DirectX10 graphics engine running on the latest consumer hardware, a solid con­cept and a storyboard, design and a pipeline of sorts. Currently as we port up to a solid DirectX11 graphics engine we are planning our level editing system, character behaviour editor, a procedural animation system and developing artwork. Though the project is limited in scope; we’re not devel­oping a game or a feature film but rather a kind of dynamic animated short. The research goal is to define how design influences cinematic communication. I anticipate from the materials that are generated by my artists and the development provided by the software engineers we will create a number of cinematic experiences and artistic installations. We will have a system that will allow for abstract experimental film-making as well as dramatic storytelling however each viewing within the system will seem like a new experience.



Concept development artwork by JC Wong

Using resources and talent provided within a research university is a clear advantage and leading a team of software developers and artists towards a research goal is a pleasant challenge. With hard work the system will have the necessary visual and behavioural assets needed to result in a compelling experience.

The project is funded to run another two years. We’re making both a narrative short subject pipeline and a non-narrative visual hotbed for experimentation and data collection. I have a team of 17 people including my Collaborators and co-Primary Investigators in other schools and univer­sities in the region. I hope to Open Source some of the engine for others to explore.

About the Author:

Mark Chavez

is an animation industry expert, academic and researcher. Mark's interests are in emergent computer animation techniques including synthetic sculpture, motion and related forms in popular culture. His research interests are in characterization and storytelling with real-time and rendered imagery exploring visual and behavioral representation in the animated form; including the creation of intelligent animated forms with a richness in personality and emotive evocative states that are flexible enough to respond to the viewer within a predetermined simulated performance. With a focus on visual aspects of artistic research, he is currently conducting basal studies in the emotive impact of non-objective imagery. He plans to leverage his findings into more aspects of visual narrative and storytelling.

Contact information:

Address:
School of Art, Design and Media Nanyang Technological University
ACAD 4-1, 81 Nanyang Drive, Singapore 637458

Tel:
+65 6316-8827 GMT+8h

Fax:
+65 6795-3140

Email: mchavez@ntu.edu.sg
Web: www.ntu.edu.sg