SIGGRAPH 2004 - The 31st international conference on computer graphics and interactive techniques
Conferenece Exhibition Call For Participation Presenters Media Registration spacer

Imagination Environment
Everything is linked. Thought to words. Words to ideas. Ideas to images. Each leads to the other in a mosaic of connectivity.

The online world is a reflection of these links, this connectivity, and the shared culture that built them. While the web plays many roles in our lives, one of the more interesting, yet unexploited, is this role as a storehouse of cultural connections.

The Imagination Environment now reflects these links back to us: out of the virtual world into the real. Using any video stream as its starting point, it discovers images linked to the words being said, and shows us the flow of connections between ideas and images that we ourselves crafted. Exploiting the connectivity of the web and the core technologies of information retrieval, it opens a window to our world that is a machine's "imagination" of who and what we are.

Life Enhancements
Search engines, blogs, web portals, and individual web sites are reflections of our cultural reality. They represent a set of created systems that expose and heighten the connections we use, but rarely see, both in our minds and in the online world. By exposing both their results and processes, these systems reflect and reuse the mundane, the available, and the purely popular as art. In doing so, the system itself is an artistic agent, gathering, sifting, and presenting our own reality back to us as it moves through the web, seeking information.

Imagination Environment uses video and images to construct a new sort of media object that immediately exposes the cultural connections that exist between the video and the other information that surrounds us. We are creating several new installations, each with its own unique view of the world of information, from digital improvisers to large-scale distributed narrative videos.

This project creates a new area that we call "network arts." At the core of network arts are technological advancements in information retrieval, social networks, and semantics, and a new cultural understanding of meaning, impact, and artistic portrayal. It is important for the portrayal to be meaningful to the culture it represents and not esoterically complex. Our goal is that in this new form of art and technology, we introduce the machine in art: a role in which the machine is used to expose the world of communication and cultural connections that are linked together and within the grasp of online systems. Doing this creates a new breed of artists who are able to harness the power of these interconnections to not only create art with the machine, but also create artistic agents that themselves are active in the creative process.

Imagination Environment uses advanced information n-retrieval techniques on media streams that are invisible to us. When we "watch" TV, the TV receiver is reading (actually decoding) the closed-captioning stream and using it to identify what is being said. Then, by exploiting indexing mechanisms within search engines, it finds distinct images and displays them as juxtaposition, to externalize either the canonical or the popular culture.

The driving system is an autonomous agent. Each performance is different in both pace and content. The speed of a slow dramatic movie monologue does not match that of a live speech or a fast hip-hop video. Imagination Environment balances its rate of image presentation based on the pace of the content and the available presentation space (number of available monitors). Our introductory work in this area creates a model of presentation complementary to the source media. As a result, an effective flow state for the overall installation is automatically achieved.

The actual accounting method varies depending on the structure of the source. For DVD closed-captioning information, Imagination Environment looks at how many words are in a caption and how many captions are on the screen at once, since each line counts as a caption. It then determines salient words by removing stop words, recognizing characters' names, and so on. Once it determines the set of terms to display, it looks at the number of available monitors and loads new images over the screens that no longer apply to the current video's context. The rate at which this happens is synchronized with the speed at which the captions are sent in the video stream. To keep the flow state engaging, thresholds are set to keep the images from changing too fast or too slow, which prevents the audience from being overwhelmed or becoming bored. This is a new application of software agents and artificial intelligence technologies.

Wednesday, 11 August
1:45 - 3:30 pm
Room 404AB

David A. Shamma
Northwestern University

Kristian Hammond
Northwestern University

emerging technologies jury and committee
 > share the SIGGRAPH 2004 web site
Conference 8-12 August, Exhibition 10-12 August.  In Los Angeles, CA