Multimodal plan representation for adaptable BML s

Authors: Herwin van Welbergen Dennis Reidsma Job Zwiers

Publish Date: 2013/01/13

Volume: 27, Issue: 2, Pages: 305-327

PDF Link

Abstract

Natural human interaction is characterized by interpersonal coordination interlocutors converge in their speech rates smoothly switch speaking turns with virtually no delay provide their interlocutors with verbal and nonverbal backchannel feedback wait for and react to such feedback execute physical tasks in tight synchrony etc If virtual humans are to achieve such interpersonal coordination they require very flexible behavior plans that are adjustable onthefly In this paper we discuss how such plans are represented maintained and constructed in our BML realizer Elckerlyc We argue that behavior scheduling for Virtual Humans can be viewed as a constraint satisfaction problem and show how Elckerlyc uses this view in its flexible behavior plan representation that allows one to make onthefly adjustments to behaviors while keeping the specified constraints between them intactA sync ref consists of either an offset from the start of the BML block or a pair s o where s is a sync point defined by the pair btext sync text id and o is a time offset in seconds from the time of the sync id b is defined as text block text id text behavior text idFor ease of specification and without loss of generality we define each time constraint as acting between two sync refs A constraint is an absolute constraint if one of the sync refs is an offset from the start of the BML block A constraint is a relative constraint if both sync refs are triples of behavior id sync id and offset timeThe function fmathbf s rightarrow mathbf t maps a sync point s to global time t The goal of scheduling is to find such a mapping for all sync points in all behaviors in the block in such a way that all constraints are satisfied The function text blockstartmathbf u rightarrow mathbf t maps the block id u to its global start time t In Sect 35 we show how the text blockstart is defined given the composition attribute of the BML block

Keywords: