Video Agents

In this interactive cyberspace environment, users interact with autonomous agents generated from video images of real-world creatures. Each agent has autonomy, personality traits, and behaviors that reflect the results of various interactions, which are determined by an emotional model.


Our primary goal is to provide users with an entertaining cyberspace environment that is also educational. The system's entertainment value is derived from allowing users to easily construct interactive environments that include many of their favorite creatures as autonomous video agents. Users can create video agents based on their real-world pets and allow the video agents to live in cyberspace forever, even after the real-world pet is gone. The system's educational value stems from the knowledge users gain through interactive experiences.

Another goal of this project is to produce a system that facilitates movie or cinema creation. This system can be a powerful tool for generating background scenes of animals or humans (extras) moving or walking.

With this project, we are also creating an ecosystem in cyberspace for science, education, and relaxation. A cyberspace ecosystem can be based on a real environment; however, our ecosystem is interactive, and each creature that autonomously exists in it has an individual personality. Such a novel approach to interactive ecosystem simulation carefully addresses the fragile balance and tradeoff between the autonomy of the simulated ecosystem and freedom of user interaction. This project will be useful for future computer simulations of natural ecological systems (not only zoological but botanical environments) for science, education, and relaxation.


The most important feature of our work is that agents are generated from live video taken from the real world. Our work successfully integrates such diverse technologies as image processing, computer vision, database, artificial intelligence, and post production. Based on these technologies, a graphical user interface allows users to create content with video agents without creating program code. The following are the core technical innovations:
  • Image Processing: extraction of a target creature in each frame of video, segmentation of the video sequence based on feature values of the extracted target creatures, and establishment of links among video segments.

  • Behavior Generation: fuzzy logic is used to determine the behavior of each autonomous agent by emotional and physical parameters, and personality is implemented based on Big Five factors.

  • Video Database: automatic generation of a video database and automatic editing of retrieved video segments based on generated behavior.

  • User Interface: a graphical tool to create video agents and an authoring tool to create an interactive cyberspace in which autonomous video agents move.


Kazuhiro Asai
Osaka University
asai (at)


Yoshifumi Kitamura
Takashi Nishimoto
Yoshinori Hirano
Emiko Hama
Fumio Kishino
Osaka University