language: en

M.I.T Media Laboratory Perceptual Computing Section
Technical Report No. 307

[Appears: International Conference on Computer Vision '95, Cambridge, MA, June 20-23, 1995]

Facial Expression Recognition using a Dynamic Model and Motion Energy

Irfan A. Essa and Alex P. Pentland
Perceptual Computing Group, The Media Laboratory, Massachusetts Institute of Technology
Cambridge, MA 02139, U.S.A.

Abstract Previous efforts at facial expression recognition have been based on the Facial Action Coding System (FACS), a representation developed in order to allow human psychologists to code expression from static facial mugshots. In this paper we develop new, more accurate representations for facial expression by building a video database of facial expressions and then probabilistically characterizing the facial muscle activation associated with each expression using a detailed physical model of the skin and muscles. This produces a muscle-based representation of facial motion, which is then used to recognize facial expressions in two different ways. The first method uses the physics-based model directly, by recognizing expressions through comparison of estimated muscle activations. The second method uses the physics-based model to generate spatio-temporal motion-energy templates of the whole face for each different expression. These simple, biologically-plausible motion energy templates are then used for recognition. Both methods show substantially greater accuracy at expression recognition than has been previously achieved.

1 Introduction