Project C5 - Combining active vision with multimodal speech comprehension
PIs: Prof. Dr. Xingshan Li, Prof. Dr. Wolfgang Menzel, Prof. Dr. Qingqing Qu
Project C5 uses eye tracking to investigate how humans integrate visual clues from the environment while they simultaneously hear speech that refers to that environment. The goal is to use insights from this investigation not only to improve existing computational models of visually informed language comprehension, but also to increase the effectiveness of crossmodal learning in humans. Experiments will explore how eye movements are modulated by cognitive activity and influenced by culture.