7 May 2020
On May 7, 2020, the first phase of the Sino-German international collaborative project "Crossmodal Learning: Adaptivity, Prediction and Interaction" (CML) was evaluated via an on-line defense meeting. A total of 45 people participated in the defense meeting, including Prof. Hao, Academician and the Director of the Information Science Division, and Deputy Director Jianjun Li; the Director of the Artificial Intelligence Department Guozheng Wu, Ms. Yingjie Fan, European Director of of the Bureau of International Cooperation; and participants from Hamburg University, Tsinghua University, the Institute of Psychology of the Chinese Academy of Sciences, Peking University and Beijing Normal University.
This major international cooperation project is jointly funded by the National Natural Science Foundation of China (NSFC 61621136008) and the German Science Foundation (DFG SFB/TRR 169). CML is a fundamental theoretical research project in the interdisciplinary fields of artificial intelligence, brain and neuroscience, and cognitive psychology. It combines computer science and statistical knowledge, and jointly explores how to develop new theories for artificial intelligence based on the progress of scientific fields such as neuro-cognition. The project brings together experts from fields such as artificial intelligence, psychology, and neuroscience, and uses cross-modal learning as a new perspective to integrate the latest developments in these fields and establish a Sino-German cooperative research center. Cross-modal learning refers to the synthesis of multiple sensory information, mainly focusing on the interactive fusion of different information modalities. I.e., it deals with how the information of one modality affects the other and aims to use information from other sensory modalities to improve the learning effect of a single sense.
Professor Jianwei Zhang from the Universität Hamburg, Germany and Professor Fuchun Sun from Tsinghua University coordinate this project. In the first phase, the consortium consisted of 14 German and 15 Chinese researchers. Over a period of 4 years, the project has researched the theme “Crossmodality” with the efforts of various institutes, fully utilizing the complementary advantages of each team. It strengthened interdisciplinary, cross-regional and cross-cultural cooperation, bringing about several ground-breaking results in the field. Finally, it promoted interdisciplinary research on intelligent robots, psychology, and brain-like computing in related fields from the perspective of crossmodal learning.
During the evaluation meeting, Professor Fuchun Sun first introduced the research goals and research results of the project, followed by a 20-minute project summary. He focused on the dynamic adaptation mechanism of cross-modality, generalization and prediction of cross-modality, and crossmodal human-computer interaction. Professor Sun summarized the more than ten sub-projects of the project that have made important progress in recent years and have attained some breakthroughs in fundamental research. These include the publication and co-publishing of more than 400 high-quality papers, covering almost all the top journals in this field including Nature Neuroscience, PNAS, IEEE PAMI and top international conferences such as CVPR, NIPS, ICML, AAAI, etc. After this, four principal investigators from neuroscience, psychology and AI gave detailed introductions to their research results.
The review experts have concluded in their evaluation meeting: the consortium has not only achieved excellent results in publishing scientific research papers, but also with the support by the theories and algorithms developed by the project, they ranked among the top in many international robotic and artificial intelligence competitions. This includes winning first place in the robotic grasping and manipulation challenges, winning the global AI challenge on image captioning in a Chinese competition, and winning first place in all three competitions in the NIPS 2017 competition for adversarial sample attack and defense challenges. In addition, they funded international conferences and international journals in this field and partially established an online knowledgebase for cross-modal machine learning. With the joint efforts of various institutes, the research goals proposed by this project have been successfully attained, and the first phase of the defense has been completed. Based on the platform established in the first phase of the project, each institute will further deepen the cooperation between projects, disciplines, and China and Germany, to truly implement our cooperation and bear more fruit.