Centre for Human-Computer Interface Design
Department of Business Computing
School of Informatics
City University
Northampton Square
London EC1V 0HB
Tel: 0171 477 8416
Fax: 0171 477 8586
We introduce and examine the role and function of drama in constructing interactive virtual worlds and consider the advantages of using live performers as virtual puppeteers over the use of intelligent or virtual software agents.
In order to approach the design of virtual worlds in which people can interact, it is helpful to use a form of structured method that can assess and incorporate the interaction and behaviour of participants in terms of systems analysis and design. This does not imply that such a method should be rigidly structured or based on a rigid requirement set. When approaching a system intended to describe interaction in a virtual world, we define the live participants in the system as the players and any computer-based elements that emulate players as agents. Together, these comprise the characters in a structured design we have described as an envisionment "a network of all possible states and transitions" (Rich and Knight, 1991) resulting in a qualitative environment (Cooper & Benjamin, 1994; see Fig. 1 below). We base the construction of an envisionment on the components we understand as necessary for interactive drama, derived from research in educational drama and experimental theatre, object-oriented systems design and human-computer interaction. Our aim is to inform and improve the construction of virtual environments that involve human-human interaction or its simulation, for entertainment, education, training or other purposes. The goal of this paper is to introduce and elaborate drama as interaction for virtual worlds.
As the teaching of drama becomes ever more marginalised in the curriculum, it becomes increasingly important to place its value clearly before an audience who may be unfamiliar with both its theory and practice. When introducing ideas from drama, it is important to dispel the common confusion between drama and theatre, a confusion best untangled by examining the semantic roots; drama, directly adopted from the Greek, meaning "the doing thing"; theatre, from the Greek theatron, meaning "the seeing place". As a noun, "the drama" is what is performed in a theatre, for a camera, in a classroom; as a verb dramatising drama is a fundamental human activity derived from, and considered to be a key element of socialisation in children's play (Moyles, 1989; Neelands, 1992; and O'Toole, 1992). Drama is honed through practice and, though often trivialised or sublimated as "entertainment", it provides adults with the vital facility to build active models of human behaviour (O'Toole, 1992). The "theatre" is the place where the drama is watched or enacted. Of course, today the drama may be enacted and recorded on a film set and be watched later in a cinema. This separation of actors from audience may be considered as an economy of scale that allows the drama to be viewed by a larger number of people, just as repeated performances of a play by the players allow an audience to partake passively - they can pay the players to play, rather than playing themselves. In many senses, a virtual world is analogous to a theatre a place where something will happen, and which must be designed and staged and peopled to suit that which will take place there. The design and construction of the inanimate elements of virtual worlds is then analogous to the design and construction of sets, scenery, properties; to the location and contents of an envisionment. What is missing at the moment is the performance and the players.
In drama, performance is typically taken to mean the action of theatre and rehearsal is the preparation for performance by the actors and technicians who will take part. This is overseen by those who will take a creative and technical responsibility directors, writers, designers; the same functions are also required in virtual world design. But rehearsal carries with it some clues for our understanding of performance. In French, rehearsal is repetition, which is intended to indicate its primary activity; the perfection of a repeatable event. Until the twentieth century, the result of rehearsal performance was almost always taken to be a linear and repeatable event, either based upon a text, or providing a text for further use. The demand of the marketplace for adult drama is for repeatable products, and the demand of academia is for fixed historical events. It is no wonder that the most acceptable record of performance still tends to be a script, which properly should be recognised as merely a blueprint; the code for the performance. Virtual worlds are analogous to theatres; they could certainly be used to view pre-recorded performances in three dimensions. But virtual worlds are clearly not merely alternatives to traditional performance venues for viewing linear narratives. So what is it that can be performed in virtual worlds, and how does this differ from conventional performance, or from other forms of computer-based dramatic interaction?
The fundamental differences between performance in a virtual world and in a theatre (or as seen and heard on a recorded media) are that the virtual performance event has the potential for spontaneous interactivity between characters, and can be more-or-less non-linear. The "interactor" (attrib. Brenda Laurel, 1991) is not an invisible observer or a traditional audience member, but a participant, another performer. The story, if there is one, is directly mediated by the players and only indirectly mediated by writers and directors, the traditional event controllers. This makes virtual performance fundamentally different to conventional theatre or film. There are past and present examples of interactive drama; from medieval Italian commedia-del-arté to modern-day improvisation in the classroom or rehearsal room, where improvisation around text or incident is often used to help flesh out an actor's understanding of their character. The experimental drama of Augusto Boal in South America seeks to make the actions and experiences of the audience the centre of the dramatic event, rather than the writer's text. Boal's techniques for structured audience participation include:
(Boal, 1979 pp. 131-139)
In conventional theatre, many professional actors are skilled at improvisation, but tend to see it as a technique for improving performance, seldom as performance itself. Improvisation has also been popularised as the basis for stand-up comedy (often referred to incorrectly as "forum theatre"), a topical example being the UK television series "What's My Line?" Nevertheless, these are marginal examples compared with mainstream, rehearsed linear dramas. We believe that the drive to "people" virtual worlds is likely to change this emphasis on linear performance and the requirement for repeatability, and place a far greater emphasis on improvisation. So who will people our virtual worlds? Current auditioning for the roles of virtual inhabitants are both live actors and computer-based agents virtual actors.
Virtual actors (V-actors), virtual puppets, or synthespians are buzzwords for animated or digitised characters created and driven by software. Nadia and Daniel Thalmann's work on 3D animations of Marilyn and Bogart are sophisticated simulcrae of their late human counterparts, a potential beginning for an "era of real computer-generated films produced in a virtual world and directed by real human directors." (Thalmann, 1994). The Oz Project, headed by Joseph Bates at Carnegie Mellon University, aims its research at the development of "broad-based dramatic agents" software simulcrae of behaviour that can act as believable creatures within the imaginative constraints of a drama. These are intended to provide the basis for the "willing suspension of disbelief" for virtual reality systems and other media designs requiring acceptable, limited imitations of character and behaviour (Bates, 1991, 1994; Kelso, 1991). Broad-based, shallow agents are intended to "foster the illusion of reality", to enable the user to "suspend disbelief" by employing a mixture of goals, goal-directed states and behaviours, emotional states and behaviours, and limited natural language abilities, memory and interface functions (Loyall, 1992). Jonathan Waldern of Virtuality PLC has usefully outlined his requirements for V-Actors; "software constructs or entities with whom the participant can interact in a contextually meaningful manner", which he subdivides into a range of more-or-less sophisticated artificial entities:
Waldern broadly suggests implementing such anthropomorphic entities through the use of fuzzy logic, neural nets and genetic algorithms (Waldern 1994). Anthropomorphic systems can be very successful; Weizenbaum's program ELIZA (Weizenbaum, 1976) could convince users briefly that they were conversing with a real person. MICKIE (an ELIZA derivative Bevan et al, 1981) was used successfully to obtain information from medical patients. But these programs are entirely dependent on context to secure the user's suspension of disbelief. They cannot necessarily be trusted to deal convincingly with interaction outside their designed context. There is a general acceptance of the limitations of AI in dealing with the complexity of human interaction; "there remains a practical concern of constructing agents of increasing degrees of believability in the short term" (Bates, 1994, p.6). Moreover, Paul Booth suggests that the very success of initiating suspension of disbelief is likely to draw naive users into inferring more intelligence and breadth of potential in the agents than actually exists, and provoking usage for which the agents are not designed (Benjamin, 1993).
An alternative to generating dramatic interaction with computer-based agents is to use live performers as virtual puppeteers, remotely controlling their animated characters. The technology for body mapping and 3D-graphics was pioneered in pre-recorded special effects for films. Mixing live and animated footage using blue-screen is a technique with a fifty-year track record. More recently, techniques have become more sophisticated, with the use of performance animation characters, or "digital puppets", where the actor controls the character's movements and expressions as well as providing a voice, much as a puppeteer might. The demands of pre-recorded effects are not as high as for real-time graphics; it is only recently that live virtual actors have made their debut, but the technology to map body movement and facial expression is continually improving, allowing animators to create realistic "humanoid" characters rapidly (Robertson, 1993). The step from pre-recorded media to live media is a matter of scale, speed and improvement in the underlying technology. Ascension sell their "Flock of Birds" system as an adapted motion detection suit which is used by a number of Hollywood special effects firms. In fact, this technology has been around for well over ten years, developed in sports science and medicine for the capture and analysis of movement. Prior to optical and electronic motion capture, mechanical waldos assisted in the replication of movement for animation. Facial waldos have been in use since the mid-1980s to capture expression and lip-synch. In the last three years, facial waldo technology has been extended to provide real-time virtual puppets like Ratz the Cat (Broadsword Productions, UK) for use in both UK and US children's television. Steve Glenn, Vice President for New Business Development, SimGraphics (CA, USA), comments on their own version of virtual actors:
"We named it VActor, because you need an actor to bring the character
to life....
VActors are lifeless unless they are connected to real people."
(Steve Glenn, quoted in: Robertson, 1993, pp. 10-11)
VActor performance software is used by Broadsword to animate Ratz the Cat; it is also used in the Nintendo Super Mario Brothers live presentations. Both rely on the skill of their actors, who themselves find what might be thought a restrictive medium strangely liberating:
"... when I realized how well it works, all of a sudden I became the
character.
I wasn't controlling a puppet, I was the puppet."
(Charles Martinet, quoted in: Robertson, 1993, p.11)
"What we like is that we can put a performer in the suit [full-body
waldo] and let them be spontaneous, improvisational."
(Carl Rosendahl, quoted in: Robertson, 1993, p.12)
It should be clear that the underlying technology to link professional performers, players and animated agents in virtual environments is falling into place. In our opinion, the immediate and medium-term future of dramatic interaction in virtual environments rests on the use of skilled live performers. With a view to their use in virtual worlds, the Oz Project and Brenda Laurel have experimented with live actors in improvised situations. Using a remote director to control actors via radio, they observed a player (interactor) engaged in an improvisation with an unspecified outcome the actors were instructed and updated by radio as to their best actions to improvise their way to one of several desired outcomes (Laurel 1991 and Kelso, 1991). No such work would have been possible with software agents.
In an informal study examining the problems of such an approach in actors' workshops at the 1994 National Student Drama Festival, we found that specifically limiting the players' interactions by attempting to maintain a single overall set of goals, or a single storyline, tended to limit participation overall; the stress between suspension of disbelief and maintaining plot was too great. Conversely, the maintenance of limited or specific objectives by individual players improved suspension of disbelief, even when players' goals conflicted. The players' ingenuity and creativity were more engaged by apparently open-ended situations, where conclusions were not and could not be foretold. In effect, what becomes important in a virtual world is not the story, but the potential for stories. An interesting side-issue to this work was the discovery that improvisation is an activity that requires the application of a conscious blend of skill, emotional commitment, and thought; it is likely to create an active, questioning mindset that is entirely unlike the kind of mind-numbed passivity described by those who demonise virtual reality in the popular press.
It is apparent that the design of virtual worlds must include beings who offer us more possibilities for interaction than the mute examination of their beautiful texture mapping, or target practice on hapless aliens. In our opinion, the use of software-based agents can only be useful for short or tightly constrained interaction, suited to the limited "weak" interactivity of current CD-ROM-based interactive movies or to some computer games, but not necessarily to the requirements of virtual worlds. An increasing number of computer games make use of virtual puppets and virtual actors; again, their limitations become apparent if the encounters are not simple and highly structured. The behaviour of alien foes can be reduced to percentages of active aggression and accessed as a data table. If you want to talk to them, then the kind of broad-based agents under examination in the Oz Project may convince briefly, like Weizenbaum's Eliza. But these are no substitute for a skilled actor on-line in a networked virtual world, creating, improvising and adapting directly. And if skilled performers are to take their place in this new medium, then virtual world design will have to take account of them, and of the integration of the themes and elements that contribute to good drama. The structure of envisionments we have proposed offer this kind of integration, but it is the spirited collaboration of artists and technicians that will ultimately produce convincing and exciting virtual worlds. Disney's animated films are developed in a remarkable collaboration between writers, actors, artists, technicians and musicians who learn and become aware of each others' dynamics, constraints and potentials virtual world-building could flourish in a similar atmosphere.
Our emphasis in this paper has been on the use of live performers and improvisation as the heart of interactive drama in virtual worlds. We do not intend to discount the use of animated agents; we have already described them as an important feature of an envisionment (Cooper & Benjamin, 1994). They are part of a continuum of levels of autonomy classified so well by Jonathan Waldern, a continuum that will help create a depth of experience unique to virtual reality. But if we are to avoid being seduced by the technology itself, then we must look to augment human-human interaction in virtual worlds, not replace it.
Bates, J., Loyall, B. & Scott Reilly, W. [1992] "Broad Agents" in Proceedings of the AAAI Spring Symposium on Integrated Intelligent Architectures, Stanford University, March 1991, published in SIGART Bulletin, Vol. 2, No. 4, August 1992.
Bates J. [1991] "Virtual Reality, Art and Entertainment" in Presence Vol. 1 No. 1, 1992, pp. 133-138 (MIT Press)
Bates J. [1994] "The Role of Emotion in Believable Agents", Communications of the ACM, Special Issue on Agents, July 1994 and Carnegie Mellon University paper: CMU-CS-94-136.
Benjamin, I. [1993] Review of "Animated Agents" lecture by Paul Booth, in British Computer Society HCI Group Newsletter Issue 21, April 1993.
Bevan, N., Pobgee, P. & Somerville, S. [1981] "MICKIE - a microcomputer for medical interviewing", in International Journal of Man-Machine Studies, Issue 14, pp. 39-47.
Boal, A. [1979] Theatre of the Oppressed, (Pluto Press, London UK)
Cooper M. & Benjamin I. [1994] "Envisionments - Constructing Dramatic Virtual Worlds" in Proceedings of Virtual Reality Software and Technology '94, Singapore, August 1994, pp. 251-265 (World Scientific, Singapore).
Kelso M.T., Weyrhauch P. & Bates J. [1991] "Dramatic Presence" in Presence Vol 2. No. 1 (MIT Press) and Carnegie Mellon University paper: CMU-CS-92-195.
Laurel, B. [1991] Computers As Theatre (Addison-Wesley, USA).
Moyles, J.R. [1989] Just Playing: The Role and Status of Play in Early Childhood Education (Open University, Buckingham, UK)
Neelands, J. [1992] Learning Through Imagined Experience, (Hodder & Stoughton, London, UK).
O'Toole, J. [1992] The Process of Drama (Routledge, London, UK).
Robertson, B., [1993] "Motion Capture Meets 3D Animation" in: Ringo, T. [1993] On the Cutting Edge of Technology, pp. 1-14, (Sams Publishing, Carmel, IN, USA).
Thalmann, D. [1994] "Automatic Control and Behaviour of Virtual Actors" in: McDonald, L. &
Vince, J. (eds.) Interacting with Virtual Environments, pp. 217-228, (John Wiley, Chichester, UK)
Waldern, J. D. [1994] "Software Design of Virtual Teammates and Virtual Opponents" in Proceedings of VR '94,The London Virtual Reality Expo, February 1994, pp. 120-125 (Mecklermedia, London, UK)
Weizenbaum, J. [1976] Computer Power and Human Reason (W.H.Freeman, San Francisco, CA, USA)
Maggie Cooper is a linguist specialising in artificial intelligence and natural language processing. Ivor Benjamin is a professional theatre director and a recently qualified systems analyst. We are addressing virtual reality from our joint arts and science backgrounds, as we feel that modelling virtual drama calls on both. This paper is part of ongoing research at City University by Ivor Benjamin into the application of drama to human-computer interaction, partly funded by the Engineering and Physical Research Council.
Back to PUBLICATIONS
HOMEPAGE