TEMOS: Generating diverse human motions from textual descriptions
2022
Conference Paper
ps
We address the problem of generating diverse 3D human motions from textual descriptions. This challenging task requires joint modeling of both modalities: understanding and extracting useful human-centric information from the text, and then generating plausible and realistic sequences of human poses. In contrast to most previous work which focuses on generating a single, deterministic, motion from a textual description, we design a variational approach that can produce multiple diverse human motions. We propose TEMOS, a text-conditioned generative model leveraging variational autoencoder (VAE) training with human motion data, in combination with a text encoder that produces distribution parameters compatible with the VAE latent space. We show the TEMOS framework can produce both skeleton-based animations as in prior work, as well more expressive SMPL body motions. We evaluate our approach on the KIT Motion-Language benchmark and, despite being relatively straightforward, demonstrate significant improvements over the state of the art. Code and models are available on our webpage.
Author(s): | Mathis Petrovich and Michael J. Black and Gül Varol |
Book Title: | European Conference on Computer Vision (ECCV 2022) |
Pages: | 48--497 |
Year: | 2022 |
Month: | October |
Publisher: | Springer International Publishing |
Department(s): | Perceiving Systems |
Bibtex Type: | Conference Paper (inproceedings) |
Paper Type: | Conference |
DOI: | 10.1007/978-3-031-20047-2_28 |
Event Place: | Tel Aviv, Israel |
ISBN: | 978-3-031-20046-5 |
State: | Published |
Talk Type: | Oral |
URL: | https://link.springer.com/chapter/10.1007/978-3-031-20047-2_28 |
Links: |
website
code paper-arxiv video |
Video: | |
BibTex @inproceedings{TEMOS, title = {TEMOS: Generating diverse human motions from textual descriptions}, author = {Petrovich, Mathis and Black, Michael J. and Varol, G\"{u}l}, booktitle = {European Conference on Computer Vision (ECCV 2022)}, pages = {48--497}, publisher = {Springer International Publishing}, month = oct, year = {2022}, doi = {10.1007/978-3-031-20047-2_28}, url = {https://link.springer.com/chapter/10.1007/978-3-031-20047-2_28}, month_numeric = {10} } |