language: en

Generating Referring Expressions in Multimodal Contexts

Susanne Salmon-Alt
Loria ? UMR 7503
Campus Scientifique, B.P.239 54506 Vandoeuvre-lès-Nancy, France
Susanne.Alt@loria.fr
Laurent Romary
Loria ? UMR 7503
Campus Scientifique, B.P.239 54506 Vandoeuvre-lès-Nancy, France
Laurent.Romary@loria.fr

Abstract This paper adresses the need of structuring the global context set into subsets or domains in order to explain adequately the use of referring expressions in a multimodal corpus. We underline, in particular, the importance of taking into account not only the discourse, but also perception and gestures for the construction of these domains. We propose a unified context model where the context is built up dynamically from different information sources and show that this way of context modelling predicts correctly the use of referring expressions in a corpus of instructional dialogues.

1 Need of a multimodal context model