In this talk I will consider the problem of scene-level inverse rendering to recover shape, reflectance and lighting from a single, uncontrolled, outdoor image. This task is highly ill-posed, but we show that multiview self-supervision, a natural lighting prior and implicit lighting estimation allow an image-to-image CNN to solve the task, seemingly learning some general principles of shape-from-shading along the way. Adding a neural renderer and sky generator GAN, our approach allows us to synthesise photorealistic relit images under widely varying illumination. I will finish by briefly describing recent work in which some of these ideas have been combined with deep face model fitting replacing parameter regression with correspondence prediction enabling fully unsupervised training.
Biography: Will Smith is a Reader (Associate Professor) in the Computer Vision and Pattern Recognition research group in the Department of Computer Science, University of York. He is also currently a Royal Academy of Engineering/The Leverhulme Trust Senior Research Fellow. He is Associate Editor of the journal Pattern Recognition and a Programme Chair for BMVC 2020. His research interests include physics-based and 3D computer vision, shape and appearance modelling and the application of statistics and machine learning to these areas. He currently leads a team of six PhD students and one postdoc.