Region of Interest (ROI)-based Coding

TNT members involved in this project:
Dr.-Ing. Marco Munderloh
Prof. Dr.-Ing. Jörn Ostermann
Show all

For video surveillance, observed areas shall be presented in high resolution and have to be available in real-time to the human observer on the ground. Therefore it is necessary to transmit high resolution HDTV video (1920x1080, PCM data rate 622 Mbit/s) over channels with very small bandwidth, typically about 1 Mbit/s. Standardized video codecs like AVC (Advanced Video Coding) can achieve bit rate reduction up to 10 Mbit/s without noticeable differences. However, further bit data reduction is needed.

The goal is to reduce the data rate of HDTV airborne surveillance video by an additional factor of ten compared to standardized AVC coding.

Instead of retransmitting areas of the video sequence which have already been transmitted in one of the preceding frames, only "new areas" are detected and transmitted. Additional ROI detectors can be connected to the system to detect for instance moving objects and transmit them additionally. A block diagram is shown in Figure 1.
Block diagram of a ROI coder

Figure 1: Block diagram of Region of Interest based coding system

 


ROI decoder principle

 

Figure 2: Principle of picture reconstruction on decoder-side. In this example a picture is created out of new areas.


On decoder-side (Figure 2) a complete frame is reconstructed employing an global motion compensation. New areas and any additional data like moving objects are inserted at decoder-side to reconstruct the original frame.
With this approach HDTV video sequences can be currently transmitted with a bit rate of 3-4 Mbit/s.

We created several self-recorded aerial test sequences with a moving camera for the evaluation of algorithms for aerial (surveillance) video processing.
It can be downloaded here: TNT Aerial Video Testset (TAVT)

Show recent publications only
  • Conference Contributions
    • Holger Meuel
      Application of the Rate-Distortion Theory for Affine Motion Compensation in Video Coding
      Proceedings of the 5th Summer School on Video Compression and Processing (SVCP) 2019, Universität Konstanz, Workgroup Multimedia Signal Processing, Konstanz, Germany, June 2019, edited by Saupe, Dietmar; Kaup, André; Ohm, Jens-Rainer
    • Holger Meuel, Stephan Ferenz, Yiqun Liu, Jörn Ostermann
      Rate-Distortion Theory for Simplified Affine Motion Compensation Used in Video Coding
      Proceedings of the IEEE International Conference on Visual Communications and Image Processing (VCIP), Taichung, Taiwan, December 2018
    • Holger Meuel, Stephan Ferenz, Yiqun Liu, Jörn Ostermann
      Rate-Distortion Theory for Affine Global Motion Compensation in Video Coding
      Proceedings of the 25th IEEE International Conference on Image Processing (ICIP), pp. 3593-3597, Athens, Greece, October 2018
    • Holger Meuel
      Rate-Distortion Theory for Affine (Global) Motion Compensation in Video Coding
      Proceedings of the 4th Summer School on Video Compression and Processing (SVCP) 2018, Leibniz Universität Hannover, Institut für Informationsverarbeitung, p. 6, Hannover, Germany, July 2018, edited by Voges, Jan
    • Holger Meuel, Stephan Ferenz, Florian Kluger, Jörn Ostermann
      Robust Long-Term Aerial Video Mosaicking by Weighted Feature-based Global Motion Estimation
      17th International Conference on Computer Analysis of Images and Patterns (CAIP), Ystad, Sweden, Proceedings, Part I, Springer International Publishing, Vol. 1, pp. 135-147, Cham, August 2017, edited by Felsberg, Michael and Heyden, Anders and Krüger, Norbert
    • Holger Meuel, Luis Angerstein, Roberto Henschel, Bodo Rosenhahn, Jörn Ostermann
      Moving Object Tracking for Aerial Video Coding using Linear Motion Prediction and Block Matching
      Proceedings of the 32nd Picture Coding Symposium (PCS), pp. 1-5, Nuremberg, Germany, December 2016
    • Holger Meuel, Stephan Ferenz, Marco Munderloh, Hanno Ackermann, Jörn Ostermann
      In-loop Radial Distortion Compensation for Long-term Mosaicking of Aerial Videos
      Proc. of the 23rd IEEE International Conference on Image Processing (ICIP), pp. 2961-2965, Phoenix, Arizona, USA, September 2016
    • Holger Meuel, Marco Munderloh, Florian Kluger, Jörn Ostermann
      Codec Independent Region of Interest Video Coding using a Joint Pre- and Postprocessing Framework
      Proc. of the IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, Seattle, Washington, USA, July 2016
    • Holger Meuel, Florian Kluger, Jörn Ostermann
      Illumination Change Robust, Codec Independent Low Bit Rate Coding of Stereo from Singleview Aerial Video
      Proc. of the 10th IEEE 3DTV Conference (3DTV-CON), pp. 1-4, Hamburg, Germany, July 2016
    • Holger Meuel, Marco Munderloh, Jörn Ostermann
      Stereo Mosaicking and 3D-Video for Singleview HDTV Aerial Sequences using a Low Bit Rate ROI Coding Framework
      Proceedings of the 12th IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS), pp. 1-6, Karlsruhe, Germany, August 2015
    • Holger Meuel, Marco Munderloh, Matthias Reso, Jörn Ostermann
      Optical Flow Cluster Filtering for ROI Coding
      Proceedings of 30th Picture Coding Symposium (PCS), pp. 129-132, San Jose, California, USA, December 2013
    • Holger Meuel, Matthias Reso, Jörn Jachalsky, Jörn Ostermann
      Superpixel-based Segmentation of Moving Objects for Low Bitrate ROI Coding Systems
      IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 395-400, Krakow, Poland, August 2013
    • Marco Munderloh, Holger Meuel, Joern Ostermann
      Mesh-based Global Motion Compensation for Robust Mosaicking and Detection of Moving Objects in Aerial Surveillance
      IEEE CVPR 2011, 1st Workshop of Aerial Video Processing (WAVP), pp. 1-6, Colorado Springs, Colorado, USA, June 2011
    • Holger Meuel, Marco Munderloh, Joern Ostermann
      Low Bit Rate ROI Based Video Coding for HDTV Aerial Surveillance Video Sequences
      IEEE CVPR 2011, 1st Workshop of Aerial Video Processing (WAVP), pp. 13-20, Colorado Springs, Colorado, USA, June 2011
  • Journals
    • Holger Meuel, Marco Munderloh, Matthias Reso, Jörn Ostermann
      Mesh-based Piecewise Planar Motion Compensation and Optical Flow Clustering for ROI Coding
      APSIPA Transactions on Signal and Information Processing, Vol. 4, October 2015
  • Books
    • Holger Meuel
      Analysis of Affine Motion-Compensated Prediction and its Application in Aerial Video Coding
      Fortschritt-Berichte VDI, VDI Verlag GmbH, Vol. 10, No. 865, Düsseldorf, December 2019
  • Book Chapters
    • Holger Meuel, Julia Schmidt, Marco Munderloh, Jörn Ostermann
      Advanced Video Coding for Next-Generation Multimedia Services - Chapter 3: Region of Interest Coding for Aerial Video Sequences Using Landscape Models
      Advanced Video Coding for Next-Generation Multimedia Services, Intech, pp. 51-78, January 2013, edited by Yo-Sung Ho
  • Technical Report
    • Holger Meuel, Florian Kluger, Jörn Ostermann
      Region of Interest Coding for Aerial Surveillance Video Using AVC & HEVC
      arXiv Technical Report, Institut für Informationsverarbeitung, October 2015