Human Action Recognition using Random Forest

TNT members involved in this project:
Dr.-Ing. Florian Baumann
Wentong Liao, M.Sc.
Show active Staff only

Human action recognition is a complex area of computer vision since static object characteristics, motion and time information have to be taken into account. Furthermore, actions are divided into human actions (walking, running, jogging), human-human interactions (handshaking, kissing, punching), human-object interactions (calling, writing, driving car) and group activities (football, soccer, group stealing). Due to environment variations such as moving backgrounds, different view points or occlusions the detection and classification of actions is even more difficult. Additionally, each actor has its own style of performing an action, leading to many variations in the subject's movement and a large intra-class variation.

  • Environment variations such as moving background, occlusions, different view points.
  • Variations of actors movement, each person has own style of executing an action.
  • Various types of activity: gestures, actions, interactions, group activities.
  • Insufficient amount of training videos.

A Random Forest consists of CART-like decision trees that are independently constructed on a bootstrap sample. Compared to other ensemble learning algorithms, i.e. boosting that build a flat tree structure of decision stumps, a Random Forest uses an ensemble of decision trees and is multi-class capable.
A tree is grown using the following algorithm:

  • Choose n samples with m variables from N training samples at random.
  • The remaining samples are used to calculate the out-of-bag error (OOB-error).
  • At each node specify m_try << M variables at random based on best split.
  • Completely grow the tree without pruning.

A completed classifier consists of several trees in which the class probabilities, estimated by majority voting, are used to calculate the sample's label.

Hollywood dataset:


 


UCF Sports dataset:


 


KTH dataset:


 


TNT Action Recognition using Motion Binary Patterns:


 


Supervised theses:

  • Evaluation of different LBP Computation Strategies for Action Recognition
  • Kinect-based Action Recognition
  • Evaluation of HOG3D and Random Forest for Action Recognition
  • Acceleration-based Action Recognition
  • An Augmented Reality Game for Mobile Devices
  • Slow-Feature Analysis for Action Recognition
  • A Spacetime Feature for Action Recognition
  • Action Bank for Human Action Recognition

 

Open theses:

 

  • XSens accelerations sensors
  • Eye tracking on mobile devices (iOS and Android)
  • Kinect Action Recognition
  • ToF Action Recognition (with Time of Flight camera)
  • Modify existing framework for Action Recognition with Volume Local Binary Patterns
  • Implement interactive games on mobile devices (iOS and Android) using object detection frameworks

 

 


 

 

Projects: Action Recognition in Thermal Images:


 

 


 

 

Projects: Action Recognition with Inertial Sensors:




Acceleration-based with inertial sensor by http://www.xsens.com

 

 


 

 

Projects: Recognize head movements and implement interactive games:


 

 


 

 

ERC Starting Grants

This project has been partially funded by the ERC within the starting grant Dynamic MinVIP.
 

Show all publications
  • Wentong Liao, Yang Michael, Bodo Rosenhahn
    Video Event Recognition by Combining HDP and Gaussian Process
    IEEE International Conference on Computer Vision (ICCV) Workshop, pp. 19-27, Santiago, Chile, December 2015
  • Christoph Reinders, Florian Baumann, Björn Scheuermann, Arne Ehlers, Nicole Mühlpforte, Alfred O. Effenberg, Bodo Rosenhahn
    On-The-Fly Handwriting Recognition using a High-Level Representation
    The 16th International Conference on Computer Analysis of Images and Patterns (CAIP), Valetta, Malta, September 2015
  • Wentong Liao, Bodo Rosenhahn, Yang Michael
    Gaussian Process for Activity Modeling and Anomaly Detection
    International Society for Photogrammetry and Remote Sensing ISA workshop, La Grande Motte, France, September 2015