Talking Heads: Articulators

An interview with Christian Benoît

Christian Benoît died in April 1998. He was 42 years old. This interview was conducted several weeks before his death.

Q: Please provide a brief overview of your audiovisual research.

A: My research is involved with audio-visual speech processing by humans and by machines, including:
- the influence of vision of the speaker's face on the perception of speech;
- lip analysis and modeling for automatic liptracking and for lip synthesis;
- 3D animation of speaking faces;
- text-to-visual-speech synthesis, and text-to-cued-speech synthesis;
- automatic recognition of audio-visual speech (isolated words).

Q: What drives your work, theoretically?

A: The mystery of bimodal integration by humans, and the quest for anthropomorphic machines.

Q: What are your visions for the future of this sort of research?

A: Auditory visions, for sure!
We must learn from the human brain activity how auditory and visual speech are integrated, and what processes are involved. Brain imagery will help greatly. Pathological studies, as well as highly controlled stimuli should teach us part of the mystery. On the machine side, I anticipate that AV speech (synthesis and recognition) will have a few years of peace (and love) where human modeling will be the basis of our models -- as it was the case in the early age of acoustic speech -- before stochastic approaches associated with huge databases take the lead. Also, I anticipate that the new age of multimedia and of CAL will quickly integrate our speaking toys in the education of foreign languages learners as well as in re-education of disabled speakers.

Q: Do you have any comments on related work by others that you consider to be exciting?

A: AV speech is so immature yet that any insight from other researchers will be informative. The Bonas (95) ASI and the AVSP '97 workshops clearly showed that psychologists, neurologists, phoneticians, speech therapists, pragmaticians, have a lot to give, receive and share with engineers and experts in image or signal processing as well as in image synthesis.

Texture mapped kiss (QuickTime, 3.3 Mbytes).

Christian Benoît worked at the Institut de la Communication Parlee, in Grenoble, France.