I Andrew Marriott - Curtin University of Technology, Project InterFace

Simon Beard - Curtin University of Technology, Project InterFace

Recently, the increase in desktop computing functionality and power has signalled a dramatic change in the design of user interfaces. Multi and hyper media technology allows for interfaces that incorporate computer generated speech along with realistically rendered computer animated human heads. Users may, with plain English queries, interact with a lifelike computer generated image that responds to them with computer generated speech using textual information coming from a knowledge base. Conversational agents such as avatars, personal desktop assistants, etc are also becoming increasingly available to help users. Typically these are Virtual Humans or Talking Heads.

To be really useful, however, these interfaces have to be functional and behave in a believable and natural manner. There are many factors that affect the functionality and believability of these interfaces.

• It is possible to give the Virtual Human a visual and audible appearance and a personality that makes the interaction more effective. People are used to dealing with other people and hence expect the appropriate gestural and emotional behaviour when dealing with a Virtual Human. Believability and user trust are increased with the appropriate "type" of Virtual Human.
• In a similar way, the responsiveness of the Virtual Human adds to believability - quick accurate responses to enquiries, fast and realistic rendering on a variety of platforms, low bandwidth transmission of data from any server to the client. An efficient Dialogue Manager using a flexible knowledge base will increase the users’ confidence in the Virtual Human’s responses.

To make the interaction with the user more effective and engaging, it is possible for these interactive Virtual Humans to be "directed" by marking up the text that they speak. This markup language must make the interaction more believable and realistic. The Virtual Human Markup Language (VHML) (www.vhml.org ) uses / builds on existing (de facto) standards such as those specified by the W3C Voice Browser Activity, and adds new tags to accommodate functionality that is not catered for. The language is XML/XSL based and consists of the following sub-systems:

• DMML Dialogue Manager Markup Language
• FAML Facial Animation Markup Language
• BAML Body Animation Markup Language
• SML Speech Markup Language
• EML Emotion Markup Language
• GML Gesture Markup Language

The intent of VHML is to facilitate the realistic and natural interaction of a Virtual Human with a user.

Initiated by the IST InterFace Project, the purpose of the workshop is to discuss the current state of VHML, to demonstrate its effectiveness and believability in creating Talking Head interfaces, and to discuss VHML’s future as a specification language through the VHML Architectural Review Board.

.

Topics of interest include VHML’s impact on:

• Virtual Human (VH) directing - controlling of emotions, speech intonation, facial behaviour
• conversational skills of VHs - natural language parsing, knowledge bases, Dialogue Managers
• personality models for Virtual Humans
• factors which influence user acceptance, i.e. the use of cartoon-like characters vs. photorealism
• VH applications - shop assistants, presentation agents for the WWW, pedagogical agents etc.
• Text to Speech systems for Virtual Humans

Workshop Format:

The workshop, initiated by the IST InterFace project, will take place as part of the IST Concertation meeting in Bruxelles, 5^th-6^th February 2001, As such, it will provide an important opportunity to researchers interested in creating believable interactive computer generated characters. The 3 hour Workshop will consist of three parts - Talking Head Demonstrations, VHML presentation and Panel Discussion.

1. Primary/secondary audience

Researchers building Virtual Humans, people interested in Virtual Humans and their applications.

Workshop Organizers:

Andrew Marriott, Project InterFace, Curtin University of Technology, Australia raytrace@talkingheads.computing.edu.au

Simon Beard, Project InterFace, Curtin University of Technology, Australia beardsw@cs.curtin.edu.au

2. Further Information

For more information, please contact raytrace@talkingheads.computing.edu.au .

Workshop Description:

The workshop will be concerned with the Virtual Human Markup Language ( www.vhml.org ) and the sub-topics of HCI, Text to Speech Synthesis, natural language parsing, knowledge bases, dialogue management, user acceptance, development and maintenance of Virtual Human applications, and Virtual Human personality modelling.

At the end of the workshop, participants should have a broad understanding of the differing technologies involved in the production, development and directing of Virtual Human applications through the use of VHML. They will have seen several Talking Head demonstrations and will understand the problems associated with the technology. They will have contributed, through discussion, to the future development of VHML.

The contributions of the workshop will be the demonstration and discussion of current and future VHML technology within the global context. This will foster a network for researchers and practitioners as well provide a focal point for future research / discussion in this emerging area. See also Outcomes above.

It is intended that the workshop be run in 3 parts:

• Demonstration session: similar to a tutorial to introduce real examples to the attendees and to explain nomenclature and concepts of Virtual Human Technology and VHML.
• Presentation: presentation of current research and possibilities of VHML, associated tools, etc
• Discussion/Question session: what are the trends, what applications, what opportunities for exploitation, where to from here, etc.

The Workshop Organizers will demonstrate and explain the components of various VHML based Talking Head applications :

• Simple MPEG-4 compliant FAP players with different head models,
• Interactive Story Telling Talking Heads with personalities,
• A client-server-server adventure game marked up in VHML
• A Talking Head client connected to a Natural Language Processing Mentoring system,
• A Java MPEG-4 2 ½ D Talking Head front end to information.

A presentation showing the results of current research in the area of VHML, its formal specification, its effectiveness and finally, its future..

This will include the areas of HCI, Text to Speech Synthesis, natural language parsing, knowledge bases, Talking Head control languages, modelling / rendering, user acceptance, applications, personality modelling and Talking Heads on the Web.

Areas to discuss/ questions to ask:

• VHML –like languages are relatively new and little has been done in evaluating the effectiveness of using them to direct a Talking Head as an interface to various applications / information sources. How do we evaluate a globally used Talking Head? What factors affect the believability of a Talking Head? Is VHML enough?
• Many proprietary Talking Heads are being developed but should they use a single standard for modelling? For rendering? For marking up of text?
• How does a new researcher start? What components exist already? What public domain packages exist that can be used?
• Where do we go to from here? Future workshops? Who wants to work with whom?

Given the nature of Virtual Human Technology, it is suggested that the workshop information (Power point presentation, movies, etc) be hosted on a stable web site. The organizers can provide this via the URL http://www.vhml.org/ . Audio and Video examples could be stored there and referenced from the main IST Concertation web page.

Andrew Marriott is a senior lecturer in the School of Computing at the Curtin University of Technology. In 1988 he formed the Computer Animation Negus(CAN), a research and development group at Curtin whose aim is to provide a sophisticated environment for animation work at the undergraduate, post-graduate and commercial level.

He is a full partner in a European Union 5th Framework project called InterFace:

Multimodal Analysis/Synthesis System for Human Interaction to Virtual and Augmented Environments.

He was co-organiser of the Talking Head Technology Workshop at OZCHI 2001, Perth Australia and is co-organiser of the proposed workshop entitled Representation Formats for Expressive Embodied Conversational Characters for AAMAS 2002 in Bologna, Italy.

Simon Beard is a PhD student in the School of Computing at Curtin University of Technology.

His research draws upon areas such as Metaphors and HCI, chatterbots, interface agents and talking heads, search engines, vortals, and portals as well as technologies and standards such as XML and MPEG-7. The culmination of research in these areas is the MetaFace project: designed to bring a new metaphor and metaphor-enabling framework to the Internet. The aim is to reduce complexity and make finding information easier, as interaction is based on anthropomorphic (human-like) concepts.

He was co-organiser of the Talking Head Technology Workshop at OZCHI 2001, Perth Australia and is on the Program Committee for the proposed workshop entitled Representation Formats for Expressive Embodied Conversational Characters for AAMAS 2002 in Bologna, Italy.

Selected Papers:

Beard, S., Crossman, B., Cechner, P. & Marriott, A. "FAQBot". Proc. of Pan Sydney Area Workshop on Visual Information Processing. (Nov, 10, 1999) University of Sydney, Australia.

Marriott, A., Ambrosini, L. & Lavagetto, F. "Virtual Salesperson". Proc. of Aust. Workshop on AI in Electronic Commerce, Sydney, Australia. (6 Dec, 1999).

Beard, S., Marriott, A., & Pockaj, R. "A Humane Interface". OZCHI 2000 Conference on Human-Computer Interaction: Interfacing reality in the new Millenium. (4-8 Dec, 2000) Sydney Australia.

Marriott, A., Pockaj, R. & Parker, C. "The Face of E-commerce". In Internet Commerce and Software Agents - Cases, Technologies and Opportunities". Editors Rahman, S & Bignall, R. Idea Group Publishing. 2000.

Marriott, A., Beard, S., Stallo, J. & Huynh, Q.. "VHML - Directing a Talking Head." In Active Media Technology. Editors: Jiming Liu, Pong C Yuen, Chun-hung Li, Jospeh Ng and Toru Ishida. LNCS 2252 Springer, 2001.

Marriott, A., Beard, S., Haddad, H., Pockaj, R., Stallo, J., Huynh, Q. & Tschirren, B. "The Face of the Future." Journal of Research and Practice in Information Technology, Vol 32, No.3/4, Aug / Nov 2001

Beard, S., Reid, D & Shepherdson, R.. "Believable and interactive Talking Heads for Websites: MetaFace and MPEG-4" In Active Media Technology. Editors: Jiming Liu, Pong C Yuen, Chun-hung Li, Jospeh Ng and Toru Ishida. LNCS 2252 Springer, 2001.