Body shape and movement are related to human health. For example, our shape tells us about our body fat and our movement tells us something about the health of our motor system. Using our 3D models of body shape we analyze movement and shape to create non-invasive and deployable methods of analyzing human health.
For example, if Cerebral Palsy (CP) is detected early, there are effective therapies to minimize the impact in later life. CP can be diagnosed in infants based on their spontaneous, undirected movements. Unfortunately, this requires expert training that is not widely available. If we can automatically track infant movement, we may automate the early detection of CP. The vision community has made great progress on 3D tracking of adults. Infants have a very different body shape from adults (see figure), which makes it difficult to directly extend prior work to infants. To address this, we learn a model of infant body shape [ ] and use it to track 3D movement in RGB-D sequences. Previous models of 3D humans [ ] were learned from thousands of high quality 3D scans, which is not practical with infants. Consequently we developed a novel method that learns infant body shape directly from low quality, incomplete, RGB-D scan sequences and deployed this in hospitals where we scanned over 30 infants.
Another example involves the distribution of adipose tissue in the body. Not all fat is the same. Visceral adipose tissue (around the organs) is highly correlated with diabetes and cardiovascular disease. In contrast, sub-cutaneous adipose tissue (fat under the skin) is relatively benign. Today an analysis of this fat distribution requires an MRI scan to reveal where fat is stored. We are developing methods to estimate this fat distribution purely from the surface shape of the body. To that end, we fit our 3D body models to full-body MRI scans [ ] to model both the external surface and the subcutaneous fat layer. We are collecting a dataset of matched MRI data and 3D surface shape and our ongoing work is focused on predicting what is inside solely from the surface.