3DTV - Integrated Three-Dimensional Television - Capture, Transmission, and Display

TNT members involved in this project:
Nobody is listed for this project right now!
Show all

It is believed that the 3DTV concept is well understood by a large majority of the public. However, it is perceived as a highly futuristic mode of real-time visual communications. There are many examples of it in science-fiction type movies. Therefore, it might not be wrong to assume that the current image of 3DTV in the public perception is a “highly desirable” but “probably impossible to achieve in the foreseeable future” technological wonder. Therefore, the reasons, which prevented such a highly desirable mode of communications from becoming a reality, should be essentially technological deficiencies. It is not too difficult to identify these missing technological building blocks; for example, lack of practical fully electronic means of 3D scene capture and 3D scene display units is probably the main complicating factor. A more careful analysis would reveal that there are many other technological components missing, and therefore, 3DTV is still not a common tool of daily life as in the case of conventional TV. However, it is also not too difficult to see that the technological components, which are necessary to bring 3DTV into reality, have been significantly matured over the past two decades. For example, image processing algorithms have evolved to handle video data from multiple synchronous sources and can extract and match feature points from each such source; this paves the road to a successful capture of accurate 3D scene information. Another mode of 3D scene information capture is to utilize holographic cameras which directly record information carrying interference fringes; again the technology has evolved to provide the large volume high-density electronic cells in such electronic sensing chips. Computer graphics technology has matured to provide almost all necessary tools for abstract 3D scene representation, like deformable meshes and other generic 3D motion object representation tools.Another main building block is the digital TV technology in broad sense. The last decade has witnessed many important technological jumps in that regard: major technological advances and standardization activities first gave us videoconferencing and videophone, and then made MPEG-1 (VCD) and MPEG-2 (digital TV and DVD) a reality, and eventually evolved to the most complicated intellectual property developed in the history, the MPEG-4. The developed technological treasure is now ready to be adapted to 3D technology. Telecommunications, in general, and internet protocols in particular, paved the way to easy-to-generate/use video content which can be delivered digitally to our monitors. Streaming video know-how has developed significantly. In the meantime, major technological breakthroughs in optical display technologies have been witnessed: spatial light modulators (SLM), digital micromirror devices (DMD), acousto-optic technology and similar approaches have hinted successful electronic holographic displays. Signal processing tools are mature enough to tackle all associated fast signal conversion steps needed during the operational phases as signals are captured, processed and directed to next components of the 3DTV chain up to the display end. Of course, the fundamental underlying technological infrastructure is the fast electronic circuitry and associated computer technology that converted computational power from a research lab luxury to a routine embedded component of all service segments and consumer appliances in multimedia communications. Looking at this picture collectively, it can be concluded that the scientific and technological environment is ripe for the important enabling step towards 3DTV. This observation is also confirmed by various already established research activities in 3DTV field in Europe (for example ATTEST), Japan and USA; an important indicator is the recent activity in MPEG-4 standardization group towards incorporating 3D objects into object-based video technology.

The primary objective of this project is to align European researchers with diverse experience and activity in distinct, yet complementary, areas so that an effective network for achieving full scale 3D video capabilities integrated seemlessly to a more general information technology base (like internet) is established and kept functional for a long time. The project will create a highly needed synergy among the European partners, at a critical time since 3DTV related research has been significantly accelerating throughout the world, and therefore will boost the European competitiveness. Potential application areas and social impact of 3DTV will also be investigated. Capturing three-dimensional visual information of a real-life scene and creating an exact (except the scale) optical duplicate of it at a remote site instantaneously, or at a later time, are ultimate goals in visual communications. All core and peripheral components related to this goal are collectively referred as “Three-Dimensional Television (3DTV)”. Main functional components of 3DTV are, “capture and representation of 3D scene information”, “complete definition of digital 3DTV signal”, “storage and transmission of this signal”, and finally the “display of the reproduced 3D scene”. For a successful consumer accepted operation of 3DTV, all these functional components must be carefully designed in an integrated fashion by considering the harmonious interaction among them. This kind of large-scale integration naturally involves a large group of researchers with diverse backgrounds, and therefore, has a highly multidisciplinary nature.

Show recent publications only
  • Conference Contributions
    • Kang Liu, Joern Ostermann
      Realistic Facial Expression Synthesis for an Image-based Talking Head
      IEEE Conference on Multimedia and Expo, ICME2011 , p. 6, Barcelona, Spain, July 2011
    • Kang Liu, Joern Ostermann
      Realistic Head Motion Synthesis for an Image-based Talking Head
      FG 2011, The 9th IEEE Conference on Automatic Face and Gesture Recognition , p. 6, Santa Barbara, CA, March 2011
    • Kang Liu, Joern Ostermann
      Image-based Talking Head: Analysis and Synthesis
      DAGA 2010, 36. International Conference on Acoustics, Deutschen Gesellschaft für Akustik, pp. 87-88, Berlin, March 2010
    • Kang Liu, Joern Ostermann
      Minimized Database of Unit Selection in Visual Speech Synthesis Without Loss of Naturalness
      The 13th International Conference on Computer Analysis of Images and Patterns CAIP2009, Springer-Verlag Berlin Heidelberg, pp. 1212-1219, Münster, Germany, September 2009, edited by X. Jiang and N. Petkov
    • Kang Liu, Joern Ostermann
      An Image-based Talking Head System
      LIPS 2009 Special Session in AVSP 2009, Norwich, UK, September 2009
    • Nikolce Stefanoski, Jörn Ostermann
      Spatially and Temporally Scalable Compression of Animated 3D Meshes with MPEG-4/FAMC
      ICIP '08 - IEEE International Conference on Image Processing, San Diego, USA, October 2008
    • Kang Liu, Joern Ostermann
      Realistic Facial Animation System for Interactive Services
      Interspeech 2008, LIPS 2008: Visual Speech Synthesis Challenge, Brisbane, Australia, September 2008
    • Kang Liu, Joern Ostermann
      Realistic Talking Head for Human-Car-Entertainment Services
      IMA 2008 Informationssysteme für mobile Anwendungen, GZVB e.V. (Hrsg.), pp. 108-118, Braunschweig, Germany, September 2008
    • Kang Liu, Axel Weissenfeld, Joern Ostermann, Xinghan Luo
      Robust AAM Building for Morphing in an Image-based Facial Animation System
      IEEE Multimedia and Expo, 2008 IEEE International Conference on , Hannover, Germany, June 2008
    • Nikolce Stefanoski, Libor Vasa, Jörn Ostermann
      Tutorial on Compression of 3D Meshes - Applications, Approaches, Standards
      3DTV-CON 08, Istanbul, May 2008
    • Nikolce Stefanoski, Patrick Klie, Xiaoliang Liu, Jörn Ostermann
      Layered Predictive Coding of Time-Consistent Dynamic 3D Meshes Using a Non-Linear Predictor
      ICIP '07 - IEEE International Conference on Image Processing, San Antonio, September 2007
    • Nikolce Stefanoski, Xiaoliang Liu, Patrick Klie, Jörn Ostermann
      Scalable Linear Predictive Coding of Time-Consistent 3D Mesh Sequences
      3DTV-CON, The True Vision - Capture, Transmission and Display of 3D Video, Kos Island, Greece, May 2007
    • Patrick Klie, Eugen Okon, Nikolce Stefanoski, Jörn Ostermann
      A Framework for Scene-Flow Driven Creation of Time Consistent Dynamic 3D Objects Using Mesh Parametrizations
      3DTV-CON, The True Vision - Capture, Transmission and Display of 3D Video, Kos Island, Greece, May 2007
    • Axel Weissenfeld, Kang Liu, Wei Liu, Joern Ostermann
      Image-based Head Animation System
      1. Kongress Multimediatechnik 2006, Institut für Multimediatechnik GmbH -IFM, pp. 67-72, Wismar, November 2006
    • Nikolce Stefanoski, Joern Ostermann
      Connectivity-Guided Predictive Compression of Dynamic 3D Meshes
      Proc. of ICIP '06 - IEEE International Conference on Image Processing, Atlanta, October 2006
    • Axel Weissenfeld, Onay Urfalioglu, Kang Liu, Joern Ostermann
      Robust Rigid Head Motion Estimation based on Differential Evolution
      IEEE International Conference on Multimedia & Expo 2006, IEEE Multimedia and Expo, 2006 IEEE International Conference on, pp. 225 - 228, Toronto, CN, July 2006
    • Kang Liu, Axel Weissenfeld, Joern Ostermann
      Parameterization of Mouth Images by LLE and PCA for Image-based Facial Animation
      ICASSP06,Toulouse, France IEEE Proceedings, IEEE, Vol. 5, pp. 461-464, May 2006
    • Levent Onural, Thomas Sikora, Jörn Ostermann, A. Smolic, R. Civanlar, J. Watson
      An assessment of 3DTV technologies
      NAB Broadcast Engineering Conference 2006, NAB Services, pp. 456-467, Washington, DC, April 2006
    • Axel Weissenfeld, Kang Liu, Sven Klomp, Joern Ostermann
      Personalized Unit Selection for an Image-based Facial Animation System
      IEEE MMSP 2005, Shanghai/China, IEEE, November 2005
    • Axel Weissenfeld, Nikolce Stefanoski, Shen Qiuqiong, Joern Ostermann
      Adaptation of a Generic Face Model to a 3D Scan, Berlin, Germany
      ICOB 2005 - Workshop On Immersive Communication And Broadcast Systems, ICOB, 2005
  • Journals
    • Axel Weissenfeld, Kang Liu, Jörn Ostermann
      Video-realistic image-based eye animation via statistically driven state machines
      The Visual Computer, Springer Berlin / Heidelberg, November 2009
    • Kang Liu, Joern Ostermann
      Optimization of An Image-based Talking Head System
      Special issue on animating virtual speakers or singers from audio: Lip-synching facial animation, EURASIP Journal on Audio, Speech, and Music Processing, Hindawi Publishing Corporation, Vol. 2009, September 2009
    • A. Smolic, K. Mueller, N. Stefanoski, J. Ostermann, A. Gotchev, G.B. Akar, G. Triantafyllidis, A. Koz
      Coding Algorithms for 3DTV - A Survey
      IEEE Transactions on Circuits and Systems for Video Technology, Vol. 17, No. 11, pp. 1606 - 1621, November 2007
  • Books
    • A. Smolic, R. Sondershaus, N. Stefanoski, L. Vasa, K. Mueller, J. Ostermann, T. Wiegand
      A survey on coding of static and dynamic 3D meshes
      Book chapter in "Haldun M. Ozaktas and Levent Onural, Three-Dimensional Television: Capture, Transmission, Display", Springer Verlag, Heidelberg, December 2007
  • Standardisation Contributions
    • Nikolce Stefanoski, Jörn Ostermann
      Scalable Compression of Dynamic 3D Meshes
      ISO/IEC JTC1/SC29/WG11, Doc. M14363, San Jose, USA, April 2007