Perceiving Systems: 2016-2021

17 February 2022

23:24

Every few years our institute (the Max Planck Institute for Intelligent Systems) is evaluated by our Scientific Advisory Board (SAB). I gave this talk at our recent evaluation, which covered the research in the Perceiving Systems department over the last 6 years. Since it summarizes our research in about 20 minutes, I thought It might be interesting to a broader audience. I couldn't cover everything and everyone who was part of our journey so please visit our website for a full description of our research: https://ps.is.mpg.de/research All the publications mentioned are available here: https://ps.is.mpg.de/publications The code and data is here: https://ps.is.mpg.de/code Full citations of the work I talk about: SMPL: Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M. J., “SMPL: A Skinned Multi-Person Linear Model”, ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1-248:16, Oct. 2015 SMPLify: Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., Black, M. J., “Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image,” In Computer Vision – ECCV 2016, pages: 561-578, Oct. 2016 HMR: Kanazawa, A., Black, M. J., Jacobs, D. W., Malik, J., “End-to-end Recovery of Human Shape and Pose,” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages: 7122-7131, 2018 3DPW: von Marcard, T.., Henschel, R., Black, M. J., Rosenhahn, B., Pons-Moll, G., Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera,” In European Conference on Computer Vision (ECCV), Sept. 2018 SPIN: Kolotouros, N., Pavlakos, G., Black, M. J., Daniilidis, K., “Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop,” Proceedings International Conference on Computer Vision (ICCV), pages: 2252-2261, Oct. 2019 SMPL-X: Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A. A., Tzionas, D., Black, M. J., “Expressive Body Capture: 3D Hands, Face, and Body from a Single Image,” In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages: 10975-10985, June 2019 VIBE: Kocabas, M., Athanasiou, N., Black, M. J., “VIBE: Video Inference for Human Body Pose and Shape Estimation,” In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages: 5252-5262, June 2020 AMASS: Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., Black, M. J., “AMASS: Archive of Motion Capture as Surface Shapes,” Proceedings International Conference on Computer Vision, pages: 5442-5451, October 2019 MANO: Romero, J., Tzionas, D., Black, M. J., “Embodied Hands: Modeling and Capturing Hands and Bodies Together,” ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 36(6):245:1-245:17, 245:1–245:17, ACM, Nov. 2017 FLAME: Li, T., Bolkart, T., Black, M. J., Li, H., Romero, J., “Learning a model of facial shape and expression from 4D scans,” ACM Transactions on Graphics, 36(6):194:1-194:17, Nov. 2017 PROX: Hassan, M., Choutas, V., Tzionas, D., Black, M. J., “Resolving 3D Human Pose Ambiguities with 3D Scene Constraints,” In Proceedings International Conference on Computer Vision, pages: 2282-2292, Oct. 2019 eXpose: Choutas, V., Pavlakos, G., Bolkart, T., Tzionas, D., Black, M. J., “Monocular Expressive Body Regression through Body-Driven Attention,” In Computer Vision – ECCV 2020, LNCS 12355, pages: 20-40, Aug. 2020 PIXIE: Feng, Y., Choutas, V., Bolkart, T., Tzionas, D., Black, M., “Collaborative Regression of Expressive Bodies using Moderation,” International Conference on 3D Vision (3DV), December 2021 BABEL: Punnakkal, A. R., Chandrasekaran, A., Athanasiou, N., Quiros-Ramirez, A., Black, M. J., “BABEL: Bodies, Action and Behavior with English Labels,” In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages: 722-731, June 2021 GRAB: Taheri, O., Ghorbani, N., Black, M. J., Tzionas, D., “GRAB: A Dataset of Whole-Body Human Grasping of Objects,” In Computer Vision – ECCV 2020, LNCS 12355, pages: 581-600, Aug. 2020 ObMan: Hasson, Y., Varol, G., Tzionas, D., Kalevatykh, I., Black, M. J., Laptev, I., Schmid, C., “Learning Joint Reconstruction of Hands and Manipulated Objects,” In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages: 11807-11816, June 2019 POSA: Hassan, M., Ghosh, P., Tesch, J., Tzionas, D., Black, M. J., “Populating 3D Scenes by Learning Human-Scene Interaction,” In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages: 14708-14718, June 2021 SAMP: Hassan, M., Ceylan, D., Villegas, R., Saito, J., Yang, J., Zhou, Y., Black, M., Stochastic Scene-Aware Motion Prediction,” In Proc. International Conference on Computer Vision (ICCV), pages: 11374-11384, October 2021 SCANimate: Saito, S., Yang, J., Ma, Q., Black, M. J., “SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks,” In Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pages: 2886-2897, June 2021