Metrics details. We describe here the control, shape and appearance models that are built using an original photogrammetric method to capture characteristics of speaker-specific facial articulation, anatomy, and texture. Two original contributions are put forward here: the trainable trajectory formation model that predicts articulatory trajectories of a talking face from phonetic input and the texture model that computes a texture for each 3D facial shape according to articulation.
Using motion capture data from different speakers and module-specific evaluation procedures, we show here that this cloning system restores detailed idiosyncrasies and the global coherence of visible articulation. Results of a subjective evaluation of the global system with competing trajectory formation models are further presented and commented. Embodied conversational agents ECAs βvirtual characters as well as anthropoid robotsβshould be able to talk with their human interlocutors.
They should generate facial movements from symbolic input. Given history of the conversation and thanks to a model of the target language, dialog managers and linguistic front-ends of text-to-speech systems compute a phonetic string with phoneme durations.
This minimal information can be enriched with details of the underlying phonological and informational structure of the message, with facial expressions, or with paralinguistic information mental or emotional state that all have an impact on speech articulation.
A trajectory formation modelβcalled also indifferently articulation or control modelβhas thus to be built that computes control parameters from such a symbolic specification of the speech task. These control parameters will then drive the talking head the shape and appearance models of a talking face or the proximal degrees-of-freedom of the robot.
