Shining a light on human pose: On shadows, shading and the estimation of pose and shape

11 June 2010

00:46

Strong lighting is common in natural scenes yet is often viewed as a nuisance for object pose estimation and tracking. In human shape and pose estimation, cast shadows can be confused with foreground structure while self shadowing and shading variation on the body cause the appearance of the person to change with pose. Rather than attempt to minimize the effects of lighting and shadows, we show that strong lighting in a scene actually makes pose and shape estimation more robust. Additionally, by recovering multiple body poses we are able to automatically estimate the lighting in the scene and the albedo of the body. Our approach makes use of a detailed 3D body model, the parameters of which are directly recovered from image data. We provide a thorough exploration of human pose estimation under strong lighting conditions and show: 1.~the estimation of the light source from cast shadows; 2.~the estimation of the light source and the albedo of the body from multiple body poses; 3.~that a point light and cast shadows on the ground plane can be treated as an additional ``shadow camera'' that improves pose and shape recovery, particularly in monocular scenes. Additionally we introduce the notion of albedo constancy which employs lighting normalized image data for matching. Our experiments with multiple subjects show that rather than causing problems, strong lighting improves human pose and shape estimation. Shining a light on human pose: On shadows, shading and the estimation of pose and shape, Balan, A., Black, M. J., Haussecker, H., Sigal, L., Int. Conf. on Computer Vision, ICCV, Rio de Janeiro, Brazil, October 2007.