Hand pose estimation and hand gesture recognition using range sensors

TNT members involved in this project:
Dr.-Ing. Alina Kuznetsova
Show active Staff only

Hand pose estimation, hand tracking and recognition of hand signs are very important for human-computer interaction (HCI), understanding human grasping and robotics.

A decade ago the tasks seemed to be almost unsolvable with the data provided by a single RGB camera. Due to recent advances in sensing technologies and appearance of range cameras, there are new data sources available, making the solution for the mentioned above problems much more feasible.

There are several approaches possible, which can be divided into two main types: model-based approaches and machine learning approaches, depending on the requirements of a concrete application. Moreover, combining them should lead to more stable solutions.

American Sign Language fingerspelling alphabet dataset

Three persons performing static signs from the ASL alphabet were recorded using Intel Creative Gesture Camera. For each subject, RGB images, depth images and confidence maps were recorded. Additionally, binary masks are provided for a hand performing a sign on depth images.

When using the dataset, please cite:
Alina Kuznetsova, Laura Leal-Taixé, Bodo Rosenhahn "Real-time sign language recognition using a consumer depth camera" IEEE International Conference on Computer Vision Workshops (ICCVW), 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), December 2013

Download: part 1 , part 2 , part 3 , README

confidence map depth map binary mask color image

Example of images from the dataset

This project has been partially funded by
the ERC within the starting grant Dynamic MinVIP.

ERC Starting Grants

  • Conference Contributions
    • Alina Kuznetsova, Bodo Rosenhahn
      On calibration of a low-cost time-of-flight camera
      IEEE European Conference on Computer Vision Workshops (ECCVW), 4rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), September 2014
    • Alina Kuznetsova, Laura Leal-Taixé, Bodo Rosenhahn
      Real-time sign language recognition using a consumer depth camera
      IEEE International Conference on Computer Vision Workshops (ICCVW), 3rd Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), December 2013
    • Alina Kuznetsova, Bodo Rosenhahn
      Hand pose estimation from a single RGB-D image
      9th International Symposium on Visual Computing (ISVC), July 2013