Markerless Motion Capture

Background

Motion Capture means to record and analyse movements of objects or humans from video data. It is used for animation, sports science or medical applications.

Goal

The goal of this project is to perform markerless motion capture. So instead of using artificial markers, attached to the body we are interested to track humans from multi view video streams without special preparation of the subject. This is even more challenging in the context of outdoor scenes, clothed people and people interaction.
( Some of the Video data is available here)

Approach

Our approach assumes as input calibrated multi-view video data and a rigged 3D mesh of the person. (For this we use a multi-camera studio and a body laser scanner). Then we perform segmentation (based on Levelsetfunctions), registration (e.g. ICP, Chamfer Distance Matching or OF) and pose estimation (by exploiting the exponential mapping of rigid body motions). Additional projects are dealing with
( Click on the images for some example videos ! )

Outdoor Motion Capture (pdf)
Textured models (pdf)
Cloth draping and simulation (pdf) (pdf)
A morphable body and pose model (pdf)
Geometric priors and motion restrictions (pdf)
Statistical learning (pdf)
Motion Capture with Moving Unsynchronized Cameras (pdf)
Learning Skeletons for Shape and Pose (pdf)

References

[1] B. Rosenhahn, T. Brox, H.-P. Seidel
Scaled Motion Dynamics for Markerless Motion Capture
IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, Minnesota, USA., 2007(pdf)
[2] J. Gall, B. Rosenhahn, H.-P. Seidel
Drift-free Tracking of Rigid and Articulated Objects
IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, 2008(pdf)
[3] B. Rosenhahn, Uwe G. Kersting, K. Powell, T. Brox, Hans-Peter Seidel
Tracking Clothed People
Human Motion - Understanding, Modelling, Capture and Animation, Springer Verlag, Dordrecht, The Netherlands, Vol. 36, pp. 295-317, 2007, edited by Rosenhahn B.; Klette R.; Metaxas D.(pdf)
[4] B. Rosenhahn, U. Kersting, K. Powell, R. Klette, H.-P. Seidel
A system for articulated tracking incorporating a clothing model
Machine Vision and Applications, Springer Verlag, Berlin-Heidelberg, Vol. 18, Nr. 1, pp. 25-40, February 2007(pdf)
[5] N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, H.-P. Seidel
A Statistical Model of Human Pose and Body Shape
Computer Graphics Forum (Proc. Eurographics 2009), Munich, Germany, 2009(pdf)
[6] B. Rosenhahn, C. Schmaltz, T. Brox, J. Weickert, D. Cremers, H.-P. Seidel
Markerless Motion Capture of Man-Machine Interaction
IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, 2008(pdf)
[7] T. Brox, B. Rosenhahn, D. Cremers, H.-P. Seidel
Nonparametric Density Estimation with Adaptive Anisotropic Kemels for Human Motion Tracking
2nd. Workshop on Human Motion, Springer-Verlag, Berlin Heidelberg, pp. 152-165, 2007, edited by Elgammal, A.; Rosenhahn, B. ; Klette, R.(pdf)
[8] N. Hasler, C. Stoll, B. Rosenhahn, T. Thormählen, H.-P. Seidel: Estimating Body Shape of Dressed Humans, Shape Modeling International (SMI 2009), Beijing, China, June 2009.(pdf)
[9] N. Hasler, B. Rosenhahn, T. Thormählen, M. Wand, J. Gall, H.-P. Seidel: Markerless Motion Capture with Unsynchronized Moving Cameras, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami Beach, Florida, June 2009.(pdf)
[10] N. Hasler, T. Thormählen, B. Rosenhahn, H.-P. Seidel: Learning Skeletons for Shape and Pose, ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Washington DC, USA, 2010. (pdf)