Predicting 3D People from 2D Pictures

5 February 2012

00:14

Predicting 3D People from 2D Pictures, (Best Paper Award). Sigal, L., Black M. J. AMDO 2006 - IV Conference on Articulated Motion and Deformable Objects, Mallorca, Spain, LNCS vol. 4069, pp. 185-195, July 2006. http://www.cs.brown.edu/~black/Papers/amdo2006sigal.pdf Abstract: We propose a hierarchical process for inferring the 3D pose of a person from monocular images. First we infer a learned view-based 2D body model from a single image using non-parametric belief propagation. This approach integrates information from bottom-up body-part proposal processes and deals with self-occlusion to compute distributions over limb poses. Then, we exploit a learned Mixture of Experts model to infer a distribution of 3D poses conditioned on 2D poses. This approach is more general than recent work on inferring 3D pose directly from silhouettes since the 2D body model provides a richer representation that includes the 2D joint angles and the poses of limbs that may be unobserved in the silhouette. We demonstrate the method in a laboratory setting where we evaluate the accuracy of the 3D poses against ground truth data. We also estimate 3D body pose in a monocular image sequence. The resulting 3D estimates are sufficiently accurate to serve as proposals for the Bayesian inference of 3D human motion over time.