We are very happy to work with students and would like to briefly introduce you to the subject areas in which we would particularly appreciate your support. Feel free to drop by with your ideas and concepts and together we will look for ways in which you can complete your final theses, seminar papers, research and project modules with us. We are always available to answer your questions.
Language-motion models for 3D human motion generation, captioning and prediction
Language-coupled human 3D full-body motion generation has many applications in robotics, artificial intelligence, virtual reality and other domains. Existing methods, such as MotionGPT, are able to generate short sequences of human poses from text description, model interaction with other people and with the static environment, describe the observed motion with language captions, and make predictions about the future motion. Their application, however, is limited to typical activities, and may not transfer well to human motion in specialized environments, such as production and healthcare. Their capability to reason about timings and distances, as well as to generate meaningful motion sequences beyond the short-term, are similarly in question.
In this thesis project we will investigate the capabilities of existing language-motion models to generate and describe motion in specialized environments. The candidate will research the existing models for generating motion, apply them in novel scenes and for custom inputs, and eventually investigate the possibility to fine-tine them with a small amount of specialized data to achieve high performance in target environments.
Prerequisites
- Strong machine learning and deep learning background, experience with foundation models is recommended
- Background in 3D perception, basics of human detection and tracking is a plus
- Excellent programming skills in Python
- Enrollment in Computer Science, Applied Computer Science, Data Science, or similar studies
- Independent and self-motivated working
This thesis is part of a collaboration with TU Munich (Dr. Andrey Rudenko). Required materials and hardware are offered through both labs.