3D Reconstruction and Camera Motion Estimation

Ph.D. Hanno Ackermann

Dipl. Math. Kai Cordes

Prof. Dr.-Ing. Bodo Rosenhahn

Background

When a picture is taken by a camera, the depth of the observed objects is lost. 3D-reconstruction is to recover the missing 3D information of the scene, and to estimate position and orientation of all cameras at the same time as shown in the following example:

This an important step in applications like augmented reality (e.g. integrating virtual objects into real image sequences), movie production, robot or car navigation, or reconstruction of historical buildings or other architectural purposes.
Or for reconstructing Cuxi.

Problem

We extract relevant information such as feature points or line structures from one or more input images. Given this information, we wish to estimate the positions and orientations of the cameras which observed the images, and the positions of the features within the 3D-scene (cf. to the image).

Approach

We pursue several different approaches in this institute: a traditional method is to estimate a tentative reconstruction from very few images (epipolar geometry), and initialize a nonlinear optimization with this input (bundle adjustment).
A different algorithm is the so-called factorization method. While it converges robustly, such an estimation is susceptible to missing data. In our group, we investigate the mathematical structure of this problem. We were able to develop surprisingly simple solutions which also apply to more general problems such as PCA.

Figure: Detected feature points, trajectories, reconstructed scene and camera path, and the application of integrating a virtual object ( Click on the images for some example videos ! )

Occlusion Handling by Retrieval of Discontinued Trajectories (ISVC'11, VISAPP'12,)
Increasing the Accuracy of Feature Evaluation Benchmarks (SSCI'11, Project Page)
A Linear Solution to 1-Dimensional Subspace Fitting under Incomplete Data (ACCV'10)
Multilinear Pose and Body Shape Estimation of Dressed Subjects from Image Sets (CVPR'10)
Scale Invariant Feature Detection based on Shape Models (CVPRw'09,ISVC'10,ICIAR'11)
Affine Structure-from-Motion by Global and Local Constraints (CVPR'09)
Robust Registration of 3D-Point Data and a Triangle Mesh (ACIVS'06, CVMP'09)
Keyframe Selection for Camera Motion and Structure Estimation (ECCV'04)
User-Friendly Integration of Virtual Objects into Image Sequences with Mosaics (VI'03)
Robust Estimation of Camera Parameters for Integration of Virtual Objects into Video Sequences (IWSNHC3DI'99 (.ps))

References

Camera Tracking Software and Example Image Sequences here:

• http://www.digilab.uni-hannover.de
• Affine Structure-from-Motion