Project A4 - Crossmodal representation facilitating robust robot behaviour

PIs: Prof. Dr. Changshui Zhang, Prof. Dr. Yizhou Wang, Prof. Dr. Jianwei Zhang

Project A4 approaches the “dynamics of crossmodal adaption” topic from a computer science and robotics perspective, with the goal to learn crossmodal representations for robust robot behaviour. A4 proposes to build a novel crossmodal simulation platform that will provide near-realistic physical properties and integrate novel tactile sensors, deformable objects, and robot models. Both meta-learning and multi-agent collaborative techniques, which respectively focus on shared and dynamic crossmodal representation, will be studied in a challenging clothes-hanging scenario. The learned models will be tested on simulated and real humanoid platforms using transfer strategies. The developed datasets and simulation environment will result in standardized robot benchmarking scenarios.