Publications - Thematic Area C
Publications project C1 - Crossmodal speech representations and their implications in speech learning
Publications of the current project
Si X, Zhou W, Hong B*. (2017).
Cooperative cortical network for categorical processing of Chinese lexical tone.
Proceedings of the National Academy of Sciences. 114(46):12303-12308.
Zhang D*, Hong B, Gao S, Röder B. (2017).
Exploring the temporal dynamics of sustained and transient spatial attention
using steady-state visual evoked potentials.
Experimental Brain Research. 2017.
Zhang Q, Hu X*, Hong B, Zhang B. (in revision).
A hierarchical sparse coding model predicts acoustic feature encoding in both auditory midbrain and cortex.
PLoS Computational Biology.
Relevant previous work
Ewald, A., Aristei, S., Nolte, G., & Rahman, R. A. (2012). Brain oscillations and functional connectivity during overt language production. Frontiers in psychology, 3.
Guo, J., Gao, S., & Hong, B. (2010). An auditory brain–computer interface using active mental response. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(3), 230-235.
Hong, B., Guo, F., Liu, T., Gao, X., & Gao, S. (2009). N200-speller using motion-onset visual response. Clinical neurophysiology, 120(9), 1658-1666.
Lou, B., Hong, B., Gao, X., & Gao, S. (2008). Bipolar electrode selection for a motor imagery based brain–computer interface. Journal of Neural Engineering, 5(3), 342.
Marzetti, L., Del Gratta, C., & Nolte, G. (2008). Understanding brain connectivity from EEG data by identifying systems composed of interacting sources. Neuroimage, 42(1), 87-98.
Nolte, G., Bai, O., Wheaton, L., Mari, Z., Vorbach, S., & Hallett, M. (2004). Identifying true brain interaction from EEG data using the imaginary part of coherency. Clinical neurophysiology, 115(10), 2292-2307.
Nolte, G., Ziehe, A., Nikulin, V. V., Schlögl, A., Krämer, N., Brismar, T., & Müller, K. R. (2008). Robustly estimating the flow direction of information in complex physical systems. Physical review letters, 100(23), 234101.
Qian, T., Zhou, W., Ling, Z., Gao, S., Liu, H., & Hong, B. (2013). Fast presurgical functional mapping using task-related intracranial high gamma activity: Laboratory investigation. Journal of neurosurgery, 119(1), 26-36.
Zhang, D., Gong, E., Wu, W., Lin, J., Zhou, W., & Hong, B. (2012, August). Spoken sentences decoding based on intracranial high gamma response using dynamic time warping. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE (pp. 3292-3295). IEEE.
Zhang, D., Song, H., Xu, R., Zhou, W., Ling, Z., & Hong, B. (2013). Toward a minimally invasive brain–computer interface using a single subdural channel: a visual speller study. Neuroimage, 71, 30-41.
Guo, J., Gao, S., & Hong, B. (2010). An auditory brain–computer interface using active mental response. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(3), 230-235.
Hong, B., Guo, F., Liu, T., Gao, X., & Gao, S. (2009). N200-speller using motion-onset visual response. Clinical neurophysiology, 120(9), 1658-1666.
Lou, B., Hong, B., Gao, X., & Gao, S. (2008). Bipolar electrode selection for a motor imagery based brain–computer interface. Journal of Neural Engineering, 5(3), 342.
Marzetti, L., Del Gratta, C., & Nolte, G. (2008). Understanding brain connectivity from EEG data by identifying systems composed of interacting sources. Neuroimage, 42(1), 87-98.
Nolte, G., Bai, O., Wheaton, L., Mari, Z., Vorbach, S., & Hallett, M. (2004). Identifying true brain interaction from EEG data using the imaginary part of coherency. Clinical neurophysiology, 115(10), 2292-2307.
Nolte, G., Ziehe, A., Nikulin, V. V., Schlögl, A., Krämer, N., Brismar, T., & Müller, K. R. (2008). Robustly estimating the flow direction of information in complex physical systems. Physical review letters, 100(23), 234101.
Qian, T., Zhou, W., Ling, Z., Gao, S., Liu, H., & Hong, B. (2013). Fast presurgical functional mapping using task-related intracranial high gamma activity: Laboratory investigation. Journal of neurosurgery, 119(1), 26-36.
Zhang, D., Gong, E., Wu, W., Lin, J., Zhou, W., & Hong, B. (2012, August). Spoken sentences decoding based on intracranial high gamma response using dynamic time warping. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE (pp. 3292-3295). IEEE.
Zhang, D., Song, H., Xu, R., Zhou, W., Ling, Z., & Hong, B. (2013). Toward a minimally invasive brain–computer interface using a single subdural channel: a visual speller study. Neuroimage, 71, 30-41.
Publications project C2 - Crossmodal learning of language – mechanisms and facilitation
Publications of the current project
Relevant previous work
He, Q., Xue, G., Chen, C., Chen, C., Lu, Z. L., & Dong, Q. (2013). Decoding the neuroanatomical basis of reading ability: A multimoxel morphometric study. Journal of Neuroscience, 33, 12835-12843.
Helfrich, R. F., Knepper, H., Nolte, G., Strüber, D., Rach, S., Herrmann, C. S., Schneider, T. R., & Engel, A. K. (2014). Selective modulation of interhemispheric functional connectivity by HD-tACS shapes perception. PLoS Biology, 12, e1002031.
Hipp, J. F., Engel, A. K., & Siegel, M. (2011). Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron, 69, 387-396.
Xue, G., Chen, C., Zhen, J., & Dong, Q. (2006). Language experience shapes fusiform activation in processing a logographic artificial language: an fMRI Training Study. Neuroimage, 31, 1315-1326.
Xue, G., Dong, Q., Chen, C., Lu, Z. L., Mumford, J. A., & Poldrack, R. A. (2013). Complementary role of frontoparietal activity and cortical pattern similarity in successful episodic memory encoding. Cerebral Cortex, 23, 1562-1571.
Xue, G., Dong, Q., Chen, C., Lu, Z. L., & Poldrack, R. A. (2010). Greater neural pattern similarity across repetitions is associated with better later memory. Science, 330, 97-101.
Xue, G., Mei, L., Chen, C., Lu, Z. L., Poldrack, R. A., & Dong, Q. (2010). Facilitating memory for novel characters by reducing the neural repetition suppression in the left fusiform cortex. PLoS One, 5, e13204.
Xue, G., Juan, C. H., Chang, C. F., Lu, Z. L., & Dong, Q. (2012). Lateral prefrontal cortex contributes to maladaptive decisions. Proceedings of the National Academy of Sciences U. S. A., 109, 4401-4406.
Lu, Y., Wang, C., Chen, C., & Xue, G. (2015). Spatiotemporal neural pattern similarity supports episodic memory. Current Biology, 25, 780-785.
Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., & Fries, P. (2007). Modulation of neuronal interactions through neuronal synchronization. Science, 316, 1609-1612.
Helfrich, R. F., Knepper, H., Nolte, G., Strüber, D., Rach, S., Herrmann, C. S., Schneider, T. R., & Engel, A. K. (2014). Selective modulation of interhemispheric functional connectivity by HD-tACS shapes perception. PLoS Biology, 12, e1002031.
Hipp, J. F., Engel, A. K., & Siegel, M. (2011). Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron, 69, 387-396.
Xue, G., Chen, C., Zhen, J., & Dong, Q. (2006). Language experience shapes fusiform activation in processing a logographic artificial language: an fMRI Training Study. Neuroimage, 31, 1315-1326.
Xue, G., Dong, Q., Chen, C., Lu, Z. L., Mumford, J. A., & Poldrack, R. A. (2013). Complementary role of frontoparietal activity and cortical pattern similarity in successful episodic memory encoding. Cerebral Cortex, 23, 1562-1571.
Xue, G., Dong, Q., Chen, C., Lu, Z. L., & Poldrack, R. A. (2010). Greater neural pattern similarity across repetitions is associated with better later memory. Science, 330, 97-101.
Xue, G., Mei, L., Chen, C., Lu, Z. L., Poldrack, R. A., & Dong, Q. (2010). Facilitating memory for novel characters by reducing the neural repetition suppression in the left fusiform cortex. PLoS One, 5, e13204.
Xue, G., Juan, C. H., Chang, C. F., Lu, Z. L., & Dong, Q. (2012). Lateral prefrontal cortex contributes to maladaptive decisions. Proceedings of the National Academy of Sciences U. S. A., 109, 4401-4406.
Lu, Y., Wang, C., Chen, C., & Xue, G. (2015). Spatiotemporal neural pattern similarity supports episodic memory. Current Biology, 25, 780-785.
Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., & Fries, P. (2007). Modulation of neuronal interactions through neuronal synchronization. Science, 316, 1609-1612.
Publications project C4 - Vision- and action-embodied language learning
Publications of the current project
Wieser, I., Barros, P., Heinrich, S., Wermter, S. (2018, accepted)
Understanding Auditory Representations of Emotional Expressions with Neural Networks.
Springer Journal Neural Computing and Applications.
Heinrich, S., Springstübe, P., Knöppler, T., Kerzel, M., Wermter. S. (2018, accepted)
Continuous Convolutional Object Tracking in Developmental Robot Scenarios.
Journal Neurocomputing.
Heinrich, S., Alpay, T., Wermter, S. (2018).
Adaptive and Variational Continuous Time Recurrent Neural Networks.
Proc. IEEE ICDL-EpiRob 2018.
Heinrich, S., Kerzel, M., Strahl, E., Wermter, S. (2018).
Embodied Multi-modal Interaction in Language learning: the EMIL data collection.
Proc. ICDL-EpiRob Workshop on Active Vision, Attention, and Learning 2018.
Springstübe, P., Heinrich, S., Wermter, S. (2018).
Continuous convolutional object tracking.
Proc. ESANN, 2018.
Springenberg, S., Lakomkin, E., Weber, C., Wermter, S. (2018).
Image-to-Text Transduction with Spatial Self-Attention.
Proc. ESANN, 2018.
Heinrich, S., Wermter, S. (2018).
Interactive Natural Language Acquisition in a Multi-modal Recurrent Neural Architecture.
Connection Science 30(1), pp. 99-133, 2018.
Kerzel, M., Strahl, E., Heinrich, Navarro-Guerrero, Heinrich, Wermter. (2017).
NICO - Neuro-Inspired COmpanion:
A Developmental Humanoid Robot Platform for Multimodal Interaction.
Proc. IEEE RO-MAN, pp 113-120, 2017
Ng, H.G. et al., (2017).
Hey Robot, Why Don't You Talk to Me?
Proc. 26th IEEE International Symposium on Robot and Human Interactive Communication
(RO-MAN 2017).
Xie, R., Liu Z., Luan H., Sun M. (2017).
Image-embodied Knowledge Representation Learning.
Proceedings of International Joint Conference on Artificial Intelligence.
Lin Y., Liu Z., Sun M. (2017).
Neural Relation Extraction with Multi-lingual Attention.
The 55th Annual Meeting of the Association for Computational Linguistics (ACL). 34-43.
Niu Y., Xie R., Liu Z., Sun M. (2017).
Improved Word Representation Learning with Sememes.
The 55th Annual Meeting of the Association for Computational Linguistics (ACL). 2049-2058.
Heinrich, S., Weber, C., Wermter, S., Xie, R., Lin, Y., Liu, Z. (2016).
Crossmodal language grounding, learning, and teaching.
NIPS2016 Workshop on Cognitive Computation, pp. 62-68, Barcelona, Spain.
Relevant previous work
Chen, X., Liu, Z., & Sun, M. (2014). A Unified Model for Word Sense Representation and Disambiguation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP'14) (pp.1025-1035). Association for Computational Linguistics.
Heinrich, S., Magg, S., & Wermter, S. (2015). Analysing the Multiple Timescale Recurrent Neural Network for Embodied Language Understanding. In Artificial Neural Networks - Methods and Applications (pp. 149-174). Springer International Publishing.
Heinrich, S., & Wermter, S. (2014). Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. In Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN 2014) (pp. 193-200), Springer International Publishing.
Kleesiek, J., Engel, A. K., Weber, C., & Wermter, S. (2011, August). Reward-driven learning of sensorimotor laws and visual features. In Proceedings of the International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) (pp. 1-6). IEEE Xpress.
Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015, February). Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of the29th AAAI Conference on Artificial Intelligence (AAAI'15) (pp. 2181-2187). Association for the Advancement of Artificial Intelligence.
Saeb, S., Weber, C., & Triesch, J. (2009). Goal-directed learning of features and forward models. Neural Networks, 22(5-6), 586-592, Elsevier
Weber, C., Wermter, S., & Elshaw, M. (2006). A hybrid generative and predictive model of the motor cortex. Neural Networks, 19(4), 339-353.
Wermter, S., Weber, C., Elshaw, M., Gallese, V., & Pulvermüller, F. (2005). Grounding neural robot language in action. In Wermter S, Palm G, & Elshaw M. (Eds.), Biomimetic Neural Learning for Intelligent Robots (pp.162-181). Springer Berlin Heidelberg.
Zhao, Y., Liu, Z., & Sun, M. (2015, February). Phrase Type Sensitive Tensor Indexing Model for Semantic Composition. In Proceedings of the 29th Conference on Artificial Intelligence (AAAI 2015) (pp. 2195-2201).
Zhong, J., Weber, C., & Wermter, S. (2012). A predictive network architecture for a robust and smooth robot docking behavior. Paladyn. Journal of Behavioral Robotics, 3(4), 172-180.
Heinrich, S., Magg, S., & Wermter, S. (2015). Analysing the Multiple Timescale Recurrent Neural Network for Embodied Language Understanding. In Artificial Neural Networks - Methods and Applications (pp. 149-174). Springer International Publishing.
Heinrich, S., & Wermter, S. (2014). Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. In Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN 2014) (pp. 193-200), Springer International Publishing.
Kleesiek, J., Engel, A. K., Weber, C., & Wermter, S. (2011, August). Reward-driven learning of sensorimotor laws and visual features. In Proceedings of the International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) (pp. 1-6). IEEE Xpress.
Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015, February). Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of the29th AAAI Conference on Artificial Intelligence (AAAI'15) (pp. 2181-2187). Association for the Advancement of Artificial Intelligence.
Saeb, S., Weber, C., & Triesch, J. (2009). Goal-directed learning of features and forward models. Neural Networks, 22(5-6), 586-592, Elsevier
Weber, C., Wermter, S., & Elshaw, M. (2006). A hybrid generative and predictive model of the motor cortex. Neural Networks, 19(4), 339-353.
Wermter, S., Weber, C., Elshaw, M., Gallese, V., & Pulvermüller, F. (2005). Grounding neural robot language in action. In Wermter S, Palm G, & Elshaw M. (Eds.), Biomimetic Neural Learning for Intelligent Robots (pp.162-181). Springer Berlin Heidelberg.
Zhao, Y., Liu, Z., & Sun, M. (2015, February). Phrase Type Sensitive Tensor Indexing Model for Semantic Composition. In Proceedings of the 29th Conference on Artificial Intelligence (AAAI 2015) (pp. 2195-2201).
Zhong, J., Weber, C., & Wermter, S. (2012). A predictive network architecture for a robust and smooth robot docking behavior. Paladyn. Journal of Behavioral Robotics, 3(4), 172-180.
Publications project C5 - Combining active vision with multimodal speech comprehension
Publications of the current project
Salama, A. R., Alaçam, Ö and Menzel, W. (2018)
Text Completion using Context-Integrated Dependency Parsing.
In the Proceedings of 3rd Workshop on Representation Learning for NLP, ACL 2018.
Qu, Q., Cui, Z. L., and Damian, M. F.
Orthographic effects in second-language spoken word recognition.
Journal of Experimental Psychology: Learning, Memory, and Cognition. In press.
Alaçam, Ö., Staron, T., and Menzel, W. (2018). A multi-modal data-set for systematic analyses of linguistic ambiguities in situated contexts. In Juan Antonio Lossio-Ventura et al., editors, Information Mangement and Big Data, Communications in Computer and Information Science. Extended version of Alaçam et al. (2017). In press.
Alaçam, Ö., and Habel, C. (2018). What does graph segmentation and description tell us about speech-gesture-referent relation? In Proceedings of the 8th Conference of the International Society for Gesture Studies. Accepted.
Staron, T., Alaçam, Ö., and Menzel, W. (2018). Incorporating Contextual Information for Language-Independent, Dynamic Disambiguation Tasks. In LREC. In press.
Alaçam, Ö., Staron, T. and Menzel, W. (2017). Towards a Systematic Analysis of Linguistic and Visual Complexity in Disambiguation and Structural Prediction. In the Proceedings of the 5th International Conference on Information Management and Big Data 4-6 September 2017. Lima, Peru.
Alaçam, Ö. and Habel, C. (2017). Negation in Gesture Accompanying Speech: A Cross-Modal Study. Journal of Multimodal Communication Studies. Vol. 4, issue 1-2.
Shen, W., Qu, Q., Ni, A., Zhou, J., & Li, X. (2017). The time course of morphological processing during spoken word recognition in Chinese. Psychonomic Bulletin & Review, 1-7. doi:10.3758/s13423-017-1274-z.
Qu, Q. Q., & Damian, M. F. (2016). Orthographic effects in spoken word recognition: Evidence from Chinese, Psychonomic Bulletin & Review, 1-6.
Shen, W., Qu, Q., & Li, X. (2016). Semantic information mediates visual attention during spoken word recognition in Chinese: Evidence from the printed-word version of visual-world paradigm. Attention, Perception, & Psychophysics., 78: 1267. doi:10.3758/s13414-016-1081-z.
Relevant previous work
Baumgärtner, C., Beuck, N., & Menzel, W. (2012). An Architecture for Incremental Information Fusion of Cross-Modal Representations. In Proc. IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI-2012 (pp. 498-503). IEEE
Beuck, N., & Menzel, W. (2013). Structural prediction in Incremental Dependency Parsing, In Proc. 14th Int. Conf. in Intelligent Text Processing and Computational Linguistics, CICLing-2013 (pp. 245-257). Springer
Foth, K. A., & Menzel, W. (2006). Hybrid Parsing: Using Probabilistic Models as Predictors for a Symbolic Parser, In Proc. 21st Int. Conf. on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Coling-ACL-2006 (pp. 321-328). IEEE.
Köhn, A., & Menzel, W. (2014). Incremental Predictive Parsing with TurboParser, In Proc. 52nd Annual Meeting of the Association for Computational Linguistics (pp. 803-808).
Li, X., Bicknell, K., Liu, P., Wei, W., & Rayner, K. (2014). Reading is fundamentally similar across disparate writing systems: A systematic characterization of how words and characters influence eye movements in Chinese
reading. Journal of Experimental Psychology: General, 143, 895-913.
Li, X., Gu, J., Liu, P., & Rayner, K. (2013). The advantage of word-based processing in Chinese reading: Evidence from eye movements. Journal of Experimental Psychology: Learning, Memory and Cognition, 39, 879-889.
Li, X., & Logan, G. D. (2008). Object-based attention in Chinese readers of Chinese words: Beyond Gestalt principles. Psychonomic Bulletin & Review, 15, 945-949.
McCrae, P., Menzel, W., & Sun, M. (2011). A computational model of concept generalization in cross-modal reference. Tsinghua Science & Technology, 16, 113-120.
Menzel, W. (2009). Towards radically incremental parsing of natural language. In Nicolov, N., & Angelova, G. (Eds.), Proc. 5th Int. Conf. on Recent Advances in Natural Language Processing (pp. 41-56). Amsterdam &
Philadelphia: John Benjamin's Publishers.
Rayner, K., Li, X., & Pollatsek, A. (2007). Extending the E-Z Reader model of eye movement control to Chinese reading. Cognitive Science, 31, 1021-1033.
Beuck, N., & Menzel, W. (2013). Structural prediction in Incremental Dependency Parsing, In Proc. 14th Int. Conf. in Intelligent Text Processing and Computational Linguistics, CICLing-2013 (pp. 245-257). Springer
Foth, K. A., & Menzel, W. (2006). Hybrid Parsing: Using Probabilistic Models as Predictors for a Symbolic Parser, In Proc. 21st Int. Conf. on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Coling-ACL-2006 (pp. 321-328). IEEE.
Köhn, A., & Menzel, W. (2014). Incremental Predictive Parsing with TurboParser, In Proc. 52nd Annual Meeting of the Association for Computational Linguistics (pp. 803-808).
Li, X., Bicknell, K., Liu, P., Wei, W., & Rayner, K. (2014). Reading is fundamentally similar across disparate writing systems: A systematic characterization of how words and characters influence eye movements in Chinese
reading. Journal of Experimental Psychology: General, 143, 895-913.
Li, X., Gu, J., Liu, P., & Rayner, K. (2013). The advantage of word-based processing in Chinese reading: Evidence from eye movements. Journal of Experimental Psychology: Learning, Memory and Cognition, 39, 879-889.
Li, X., & Logan, G. D. (2008). Object-based attention in Chinese readers of Chinese words: Beyond Gestalt principles. Psychonomic Bulletin & Review, 15, 945-949.
McCrae, P., Menzel, W., & Sun, M. (2011). A computational model of concept generalization in cross-modal reference. Tsinghua Science & Technology, 16, 113-120.
Menzel, W. (2009). Towards radically incremental parsing of natural language. In Nicolov, N., & Angelova, G. (Eds.), Proc. 5th Int. Conf. on Recent Advances in Natural Language Processing (pp. 41-56). Amsterdam &
Philadelphia: John Benjamin's Publishers.
Rayner, K., Li, X., & Pollatsek, A. (2007). Extending the E-Z Reader model of eye movement control to Chinese reading. Cognitive Science, 31, 1021-1033.
Publications project C6 - Multisensory perception, learning and action for crossmodal tele-operations
D. Krupke, F. Steinicke, P. Lubos, Y. Jonetzko, M. Görner and J.Zhang (2018).
Comparison of Multimodal Heading and Pointing Gestures for Co-Located
Mixed Reality Human-Robot Interaction,
Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018).
Chen, L.*, Zhou, X., Müller, H.J., Shi, Z. (in press)
What you see depends on what you hear: temporal averaging and crossmodal integration,
Journal of Experimental Psychology: General.
Zheng,W.* & Chen, L.* (2018).
The Roles of Attentional Shifts and Attentional Reengagement
in Resolving The Spatial Compatibility Effect in Tactile Simon-like Tasks.
Scientific Reports, 8:8760.
doi:10.1038/s41598-018-27114-9.
Wan, Y. & Chen, L.* (2018).
Temporal Reference, Attentional Modulation, and Crossmodal Assimilation.
Front. Comput. Neurosci. 12:39.
doi: 10.3389/fncom.2018.00039.
Na R., Bi T., Tjan B.S., Liu Z. and Fang F. (2018).
Effect of task difficulty on blood-oxygen-level-dependent signal:
a functional magnetic resonance imaging study in a motion discrimination task.
PLoS ONE 13(6): e0199440.
Wang Y., Zhu Z., Chen B. and Fang F. (2018)
Perceptual learning and recognition confusion reveal the underlying relationships
among the six basic emotions.
Cognition and Emotion.
https://doi.org/10.1080/02699931.2018.1491831
Yiltiz, H. & Chen, L.*（2018）.
Emotional cues and social anxiety resolve ambiguous perception of biological motion.
Exp Brain Res. 236：1409-1420
Tian,Y. & Chen, L.* (2018).
Cross-modal attention modulates tactile subitizing but not tactile numerosity estimation.
Attention, Perception, & Psychophysics, https://doi.org/10.3758/s13414-018-1507-x
Mo C.*, He D.* and Fang F. (2018).
Attention priority map of face images in human early visual cortex.
Journal of Neuroscience. 38(1), 149-157
Chen N., Lu J., Shao H., Weng X. and Fang F. (2017).
Neural mechanisms of motion perceptual learning in noise.
Human Brain Mapping. 38(12), 6029-6042
N. Katzakis, F. Steinicke,
Excuse me! Perception of Abrupt Direction Changes Using Body Cues and Paths on Mixed Reality Avatars.
Proceedings of ACM SIGCHI Conference on Human Robot Interaction, Late Breaking Reports March 4-8, 2018. Chicago, IL.
https://doi.org/10.1145/3173386.3177040
Jingxin Zhang, Eike Langbehn, Dennis Krupke and Frank Steinicke.
A 360° Video-based Robot Platform for Telepresent Redirected Walking.
Proceedings of ACM SIGCHI Conference on Human Robot Interaction Workshop on Virtual, Augmented and Mixed Reality for Human-Robot Interaction.
O. Ariza, N. Katzakis, F. Steinicke, G. Bruder,
Analysis of Proximity-Based Multimodal Feedback for 3D Selection in Immersive Virtual Environments,
25th IEEE Conference on Virtual Reality and 3D User Interfaces, March 18-22, 2018 Reutlingen, Germany. 2018.
J. Zhang, E. Langbehn, D. Krupke, N. Katzakis, F. Steinicke,
Detection Thresholds for Rotation and Translation Gains in 360° Video-based Telepresence System,
IEEE Transactions on Visualization and Computer Graphics,
https://doi.org/10.1109/TVCG.2018.2793679. 2018.
N. Katzakis, J. Tong, O. Ariza, L. Chen, G. Klinker, B. Röder,and F. Steinicke.
Stylo and Handifact: Modulating Haptic Perception throughVisualizations for PostureTraining in Augmented Reality.
Proceedings of ACM SIGCHI Conference on Spatial User Interaction, Brighton, UK, October 2017.
https://doi.org/10.1145/3131277.3132181; Best Paper Award
Krupke, D., Starke, S., Einig, L., Steinicke, F., Zhang, J.,
Prototyping of Immersive HRI Scenarios,
Human-Centric Robotics, Proceedings of CLAWAR 2017: 20th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines, p. 537-544, 2017.
https://doi.org/10.1142/9789813231047_0065; Industrial Robot Innovation Award
Chen, L., Guo, L., Bao,M. (2017). "Sleep-dependent consolidation benefits fast transfer of time interval training". Experimental Brain Research, 235(3),661-672.
Guo,L., Bao, M., Guan, L., Chen, L.. (2017). "Cognitive Styles Differentiate Crossmodal Correspondences Between Pitch Glide and Visual Apparent Motion", Multisensory Research.30,363-385.
He D., Mo C. and Fang F. (2017) "Predictive feature remapping before saccadic eye movements". Journal of Vision 17(5):14, 1-12.
Ariza, O., Lange, M., Steinicke, F., and Bruder, G. (2016) , Vibrotactile assistance for user guidance towards selection targets in VR and the cognitive resources involved, 2017 IEEE Symposium on 3D User Interfaces (3DUI), Los Angeles, CA, 2017, pp. 95-98. doi: doi.org/10.1109/3DUI.2017.7893323
Yu Q., Zhang P., Qiu J. and Fang F. (2016). Perceptual learning of contrast detection in the human lateral geniculate nucleus. Current Biology. 26(23), 3176-3182.
J. Mayer and N. Katzakis. 2016. A Metric for Short-Term Hand Comfort and Discomfort: Exploring Hand Posture Evaluation. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16). ACM, New York, NY, USA, 131-134. DOI: doi.org/10.1145/2983310.2985752
Oscar Ariza, Frank Steinicke, Lihan Chen, Fang Fang, Visual-Haptic Distance Perception in the Central Fovea and Periphery of the Peripersonal Space, International Multisensory Research Forum (IMRF), 2016.
Relevant previous work
Bruder, G., Steinicke, F., & Sturzlinger, W. (2013, July). To touch or not to touch?: comparing 2D touch and 3D mid-air interaction on stereoscopic tabletop surfaces. In Proceedings of the 1st symposium on Spatial user interaction (pp. 9-16). ACM.
Bruder, G., Lubos, P., & Steinicke, F. (2015). Cognitive Resource Demands of Redirected Walking. IEEE Trans. on Visualization and Computer Graphics (TVCG), 4, 539-544.
Chen, L., Wang, Q., & Bao, M. (2014). Spatial references and audio-tactile interaction in cross-modal dynamic capture. Multisensory Research, 27, 55-70.
Chen, L., & Vroomen, J (2013). Intersensory Binding across Space and Time: A Tutorial Review. Attention, Perception, & Psychophysics, 75, 790-811.
Chen, L., Shi, Z., & Müller, H. J. (2011). Interaction of Perceptual Grouping and Crossmodal Temporal Capture in Tactile Apparent-Motion. PLoS ONE, 6(2), e17130.
Chen, J., He, Y., Zhu, Z., Zhou, T., Peng, Y., Zhang, X., & Fang, F. (2014). Attention-dependent early Cortical Suppression contributes to Crowding. Journal of Neuroscience. 34(32), 10465-10474.
Daiber, F., Valkov, D., Steinicke, F., Hinrich, K., & Krüger, A. (2012). Towards Object Prediction based on Hand Postures for Reach to Grasp Interaction. In Proceedings of the ACM, the 3rd Dimension of CHI (3DCHI) (pp.99-106). ACM.
Fang, F., & He, S. (2005). Cortical Responses to Invisible Objects in Human Dorsal and Ventral Pathways. Nature Neuroscience. 8(10), 1380-1385.
Steinicke, F., Bruder, G., Jerald, J., Frenz, H., & Lappe, M. (2010). Estimation of Detection Thresholds for Redirected Walking Techniques. IEEE Trans. on Visualization and Computer Graphics (TVCG), 16(1), 17-27
Zhang, X., Zhaoping, L., Zhou, T., & Fang, F. (2012). Neural Activities in V1 create a Bottom-Up Saliency Map. Neuron 73(1), 183-192.
Bruder, G., Lubos, P., & Steinicke, F. (2015). Cognitive Resource Demands of Redirected Walking. IEEE Trans. on Visualization and Computer Graphics (TVCG), 4, 539-544.
Chen, L., Wang, Q., & Bao, M. (2014). Spatial references and audio-tactile interaction in cross-modal dynamic capture. Multisensory Research, 27, 55-70.
Chen, L., & Vroomen, J (2013). Intersensory Binding across Space and Time: A Tutorial Review. Attention, Perception, & Psychophysics, 75, 790-811.
Chen, L., Shi, Z., & Müller, H. J. (2011). Interaction of Perceptual Grouping and Crossmodal Temporal Capture in Tactile Apparent-Motion. PLoS ONE, 6(2), e17130.
Chen, J., He, Y., Zhu, Z., Zhou, T., Peng, Y., Zhang, X., & Fang, F. (2014). Attention-dependent early Cortical Suppression contributes to Crowding. Journal of Neuroscience. 34(32), 10465-10474.
Daiber, F., Valkov, D., Steinicke, F., Hinrich, K., & Krüger, A. (2012). Towards Object Prediction based on Hand Postures for Reach to Grasp Interaction. In Proceedings of the ACM, the 3rd Dimension of CHI (3DCHI) (pp.99-106). ACM.
Fang, F., & He, S. (2005). Cortical Responses to Invisible Objects in Human Dorsal and Ventral Pathways. Nature Neuroscience. 8(10), 1380-1385.
Steinicke, F., Bruder, G., Jerald, J., Frenz, H., & Lappe, M. (2010). Estimation of Detection Thresholds for Redirected Walking Techniques. IEEE Trans. on Visualization and Computer Graphics (TVCG), 16(1), 17-27
Zhang, X., Zhaoping, L., Zhou, T., & Fang, F. (2012). Neural Activities in V1 create a Bottom-Up Saliency Map. Neuron 73(1), 183-192.