Summary of the paper
| Title | Building a synchronous corpus of acoustic and 3D facial marker data for adaptive audio-visual speech synthesis |
|---|---|
| Authors | Dietmar Schabus, Michael Pucher and Gregor Hofer |
| Abstract | We have created a synchronous corpus of acoustic and 3D facial marker data from multiple speakers for adaptive audio-visual text-to-speech synthesis. The corpus contains data from one female and two male speakers and amounts to 223 Austrian German sentences each. In this paper, we first describe the recording process, using professional audio equipment and a marker-based 3D facial motion capturing system for the audio-visual recordings. We then turn to post-processing, which incorporates forced alignment, principal component analysis (PCA) on the visual data, and some manual checking and corrections. Finally, we describe the resulting corpus, which will be released under a research license at the end of our project. We show that the standard PCA based feature extraction approach also works on a multi-speaker database in the adaptation scenario, where there is no data from the target speaker available in the PCA step. |
| Topics | Corpus (creation, annotation, etc.), Speech Synthesis, Speech resource/database |
Full paper  |
Building a synchronous corpus of acoustic and 3D facial marker data for adaptive audio-visual speech synthesis |
| Bibtex | @InProceedings{SCHABUS12.302, author = {Dietmar Schabus and Michael Pucher and Gregor Hofer}, title = {Building a synchronous corpus of acoustic and 3D facial marker data for adaptive audio-visual speech synthesis}, booktitle = {Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12)}, year = {2012}, month = {may}, date = {23-25}, address = {Istanbul, Turkey}, editor = {Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Mehmet Uğur Doğan and Bente Maegaard and Joseph Mariani and Asuncion Moreno and Jan Odijk and Stelios Piperidis}, publisher = {European Language Resources Association (ELRA)}, isbn = {978-2-9517408-7-7}, language = {english} } |