Video Surveillance for Helpful Purposes

TNT members involved in this project:
Prof. Dr.-Ing. Jörn Ostermann
Show all

Surveillance systems have been widely deployed in public places, for example to maintain order in a train station with stong people stream, to detect potential dangerous object in airport, to recognize a theft in a store, etc. Traditional way in which the surveillance videos are watched by a man sitting before the monitors is unreliable, low efficient and costly. Ideally, we would like the system to automatically analyze the surveillance videos for reporting the speciall situation.


Objects and person of interest are detected by a deep learning-based method, such as Faster RCNN. Each detected object is labeled with an owner. A background model is utilized to find static objects. If the object's ower disappears from the surveillance scene, an alarm for abandonment is triggered. Further events around the abandoned object are analyzed. If anyone attempts to do anything on the under watched object, the person is verified whether he is the owner. If not, a warning for an un-owner moving the object is triggered.  When the person is going to leave the surveillance scene but the missing object is not detected within the scene again, the behavior is recognized as stolen.




In the following sequences, different application of surveillance system are shown.


We provide the Security Event Recognition Dataset (SERD) for research purposes. It is collected and manually labeled by the Institut für Informationsverarbeitung. For more information please contact the authors.  Wentong Liao, M.Sc.


  • E. Nowak, F. Jurie, and B. Triggs, "Sampling strategies for bag-of-features image classification", Proc. ECCV, 2006.
  • David Lowe: "Distinctive Image Features from Scale-Invariant Keypoints", IJCV, 2004.
  • Herbert Bay, Tinne Tuytelaars and Luc Van Gool: "SURF: Speeded Up Robust Features", ECCV, 2006.
  • Krystian Mikolajczyk and Cordelia Schmid: A performance evaluation of local descriptors, TPAMI, 2005.

Show all publications
  • Wentong Liao, Chun Yang, Michael Ying Yang, Bodo Rosenhahn
    Security Event Recognition for Visual Surveillance
    ISPRS Annals of Photogrammetry, Remote Sensing \& Spatial Information Sciences, Vol. 4, June 2017
  • Wentong Liao, Yang Michael, Bodo Rosenhahn
    Video Event Recognition by Combining HDP and Gaussian Process
    IEEE International Conference on Computer Vision (ICCV) Workshop, pp. 19-27, Santiago, Chile, December 2015
  • Wentong Liao, Bodo Rosenhahn, Yang Michael
    Gaussian Process for Activity Modeling and Anomaly Detection
    International Society for Photogrammetry and Remote Sensing ISA workshop, La Grande Motte, France, September 2015