Neural Graphics in a Generative World (Talk)
Recent years have seen significant advancements in deep learning, which has led to a growing belief that Moore's law, which traditionally pertained to the packing of transistors, is now transitioning towards the improvement of photo-realistic 3D graphics. The advancements in this research field can be broadly categorized into two areas: neural fields, which are capable of modeling photo-realistic 3D representations, and diffusion models, which are able to generalize to large scale data and produce photo-realistic images. To combine these technologies for large scale 3D generative modeling, methods such as DreamFusion and Magic3D have been developed. Despite these advancements, several key questions remain regarding the animation of these worlds, interaction with them, and their potential use within the current ecosystem of computer graphics and gaming. In this talk, we will delve into learning representations for generating animatable 3D worlds. More closely, I will introduce Neuman, which enables photorealistic 3D view synthesis of animatable humans embedded in a scene. Additionally, we will examine the challenges in making neural networks more efficient for deployment in the existing ecosystem and introduce Mobileone, the fastest neural network on the iPhone. Through this talk, I aim to discuss the importance of both achieving animatable photorealistic 3D worlds and challenges with their deployment in the existing ecosystem of VFX, graphics, and gaming.
Biography: Anurag is a researcher at Apple Machine Intelligence. His interests lie at the intersection of deep learning, computer vision and 3D geometry. He did his PhD at Max Planck Institute for Intelligent Systems with Michael Black with a thesis on Geometric Understanding of Motion. His work links motion of objects in videos to structure and geometry in the world using deep learning paradigms and self-supervised learning. He received his Masters degree from the Computer Science Department at The University of British Columbia, Vancouver. He was a research assistant in the Sensorimotor Systems Lab at UBC and he worked with Dinesh Pai on understanding motion of Eyes and Upper Faces. He completed his undergraduate studies at National Institute of Technology Karnataka in the Electronics and Communications Department.
