Image-based facial animation system for interactive services


Golden Lips Award for Audiovisual Consistency:

    This excellent research work on facial animation received the Golden Lips Award for Audiovisual Consistency in the first visual speech synthesis challenge: LIPS 2008, Sept. 22-26, 2008 in Brisbane, Australia.

Abstract:

    At the Institut fuer Informationsverarbeitung a photo-realistic facial animation system is developed, which uses an image-based rendering technique. The image-based facial animation system consists of two parts. One is the off-line audio-visual analysis of recorded human subjects and the other is the on-line synthesis of facial animations.



  1. Applications

      Today, human-machine communication is dominated by the text input and mouse clicks of the user, while the machine returns text and graphics. In the future, the machine may return synthesized speech with facial animation. Especially dialog systems, as used in e-commerce and e-care, can integrate facial animations with synthesized speech in web sites to improve human-machine communication. For instance, a virtual, personal adviser can navigate and assist a customer through web sites. Subjective tests indicated that e-commerce web sites with an integrated facial animation and spoken output achieve higher customer satisfaction.

  1. Details and Demos (If the codec is required by your windows media player software, please install the codec package!)

      1. Mouth animation (published in MMSP 2005, ICME 2006)

      2. Eye animation (published in EUROGRAPHICS 2009)

      3. Original and synthesized video pairs for subjective tests (published in Interspeech 2008)

      4. Minimized database for visual speech synthesis ( published in CAIP2009)

      5. Facial expression synthesis (accepted in ICME 2011)

      6. Head motion synthesis (published in FG 2011)

      7. Evaluation of talking head (accepted in CASA 2011)

      8. Expressive database


  1. Talking head stand-alone software download (executable version)

    wintnt 1.0

    Download now!


    Description:
    This is a test software for talking head. Please extract downloaded archived rar files and execute the "wintnt.exe" program. The corresponding plugin of the program is also available.
    License:
    Test software
    Operate System:
    Win XP/Win 2000/Win 98/Win Me
    Datasize:
    37.35MB
    TTS language:
    English
    Released:
    19.04.2009


  1. Publications -- under constrution

    1. Axel Weissenfeld, Kang Liu, Sven Klomp, Joern Ostermann.

      Personalized Unit Selection for an Image-based Facial Animation System,

      MMSP 2005, Shanghai, China, 2005.

    2. Kang Liu, Axel Weissenfeld, Joern Ostermann.

      Parameterization of Mouth Images by LLE and PCA for Image-based Facial Animation,

      ICASSP 2006, pp. 461-464, Toulouse, France, May 2006

    3. Axel Weissenfeld, Onay Urfalioglu, Kang Liu, Joern Ostermann.

      Robust Rigid Head Motion Estimation Based on Differential Evolution,

      ICME 2006, pp. 225-228, Toronto, Canada, July 2006

    4. Kang Liu, Axel Weissenfeld, Joern Ostermann, Xinghan Luo.

      Robust AAM Building for Morphing in an Image-based Facial Animation System,

      ICME 2008, Hannover, June, 2008.

    5. Kang Liu, Joern Ostermann.

      Realistic Facial Animation System for Interactive Services,

      LIPS 2008: Visual Speech Synthesis Challenge, Special Session in Interspeech 2008. Brisbane, September, 2008

    6. Axel Weissenfeld, Kang Liu, Joern Ostermann.

      Video-Realistic Image-based Eye Animation System,

      EUROGRAPHICS 2009 (Short Paper), Munich, April 2009.

    7. Kang Liu, Joern Ostermann.

      Minimized Database of Unit Selection in Visual Speech Synthesis Without Loss of Naturalness,

      CAIP 2009, Münster, Germany, September 2009.

    8. Kang Liu, Joern Ostermann.

      An Image-based Talking Head System,

      LIPS 2009 Special Session in AVSP 2009, Norwich,UK, September 2009.

    9. Kang Liu, Joern Ostermann.

      Optimization of An Image-based Talking Head System,

      EURASIP Journal on Audio, Speech, and Music Processing, September 2009.

    10. Axel Weissenfeld, Kang Liu, Joern Ostermann.

      Video-realistic image-based eye animation via statistically driven state machines,

      The Visual Computer, International Journal of Computer Graphics, November 2009.




  1. Contact

    Prof. Dr.-Ing. Jörn Ostermann

    M.Sc. Kang Liu

    Leibniz Universität Hannover

    Institut für Informationsverarbeitung

    Appelstr. 9A, D-30167 Hannover

    Tel. +49(0)511 7625316

    Fax +49(0)511 7625333

    Email: kang@tnt.uni-hannover.de



Leibniz Universitaet Hannover

  

TNT Home | LUH | Search | Administrator
Updated 18/02/2009
Webpage design M.Sc. Kang Liu