Publications - Thematic Area B
Publications project B1 - Neural dynamics in crossmodal prediction
Publications of the current project
Maye A, Wang P, Daume J, Hu X, Engel AK (2019) An oscillator ensemble model of sequence learning. Frontiers in Integrative Neuroscience 13: 43.
Wang P*, Göschl F*, Friese U, König P, Engel AK (2019) Long-range functional coupling predicts performance: oscillatory EEG networks in multisensory processing. Neuroimage 196: 114-125.
Rimmele JM, Gudi-Mindermann H, Nolte G, Röder B, Engel AK (2019) Working memory training integrates visual cortex into beta-band networks in congenitally blind individuals. Neuroimage 194: 259-271.
Chen J, Li Z, Hong B, Maye A, Engel AK, Zhang D (2019) A single-stimulus, multi-target BCI based on retinotopic mapping of motion-onset VEPs. IEEE Transactions on Biomedical Engineering 66: 464-470.
Nolte G, Aburidi M, Engel AK (2019) Robust calculation of slopes in detrended fluctuation analysis and its application to envelopes of human alpha rhythm. Scientific Reports 9: 6339.
Shahbazi Avarvand F, Bartz S, Andreou C, Samek W, Leicht G, Mulert C, Engel AK, Nolte G (2018).
Localizing bicoherence from EEG and MEG. Neuroimage 174: 352-363.
Chen, J., Zhang, D., Engel, A. K., Gong, Q., and Maye, A. (2017).
Application of a single-flicker online SSVEP BCI for spatial navigation,
PloS one, 12(5), e0178385.
Maye A, Zhang D, Engel AK (2017).
Utilizing retinotopic mapping for a multi-target SSVEP BCI with a single flicker frequency.
IEEE Transactions on Neural Systems and Rehabilitation Engineering 25: 1026–1036.
Relevant previous work
Engel, A. K., Fries, P., & Singer, W. (2001). Dynamic predictions: oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2, 704-716.
Engel, A. K., Gerloff, C., Hilgetag, C. C., & Nolte, G. (2013). Intrinsic coupling modes: multiscale interactions in ongoing brain activity. Neuron, 80, 867-886.
Hipp, J. F., Engel, A. K., & Siegel, M. (2011). Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron, 69, 387-396.
Hipp, J. F., Hawellek, D., Corbetta, M., Siegel, M., & Engel, A. K. (2012). Large-scale cortical correlation structure of spontaneous oscillatory activity. Nature Neuroscience, 15, 884-890.
Höfle, M., Pomper, U., Hauck, M., Engel, A. K., & Senkowski, D. (2013). Spectral signatures of viewing a needle approaching one' own body when anticipating pain. European Journal of Neuroscience, 7, 3089-3098.
Hu, X., Sun, C., & Zhang, B. (2010). Design of recurrent neural networks for solving constrained least absolute deviation problems. IEEE Transactions on Neural Networks, 21, 1073-1086.
Liang, M., & Hu, X. (2015). Recurrent convolutional neural network for object recognition. Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3367-3375.
Schepers, I. M., Hipp, J. F., Schneider, T. R., Röder, B., & Engel, A. K. (2012). Functionally specific oscillatory activity correlates between visual and auditory cortex in the blind. Brain, 135, 922-934.
Siegel, M., Donner, T. H., Oostenveld, R., Fries, P., & Engel, A. K. (2008). Neuronal synchronization along the dorsal visual pathway reflects the focus of spatial attention. Neuron, 60, 709-719.
Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., & Fries, P. (2007). Modulation of neuronal interactions through neuronal synchronization. Science, 316, 1609-1612.
Engel, A. K., Gerloff, C., Hilgetag, C. C., & Nolte, G. (2013). Intrinsic coupling modes: multiscale interactions in ongoing brain activity. Neuron, 80, 867-886.
Hipp, J. F., Engel, A. K., & Siegel, M. (2011). Oscillatory synchronization in large-scale cortical networks predicts perception. Neuron, 69, 387-396.
Hipp, J. F., Hawellek, D., Corbetta, M., Siegel, M., & Engel, A. K. (2012). Large-scale cortical correlation structure of spontaneous oscillatory activity. Nature Neuroscience, 15, 884-890.
Höfle, M., Pomper, U., Hauck, M., Engel, A. K., & Senkowski, D. (2013). Spectral signatures of viewing a needle approaching one' own body when anticipating pain. European Journal of Neuroscience, 7, 3089-3098.
Hu, X., Sun, C., & Zhang, B. (2010). Design of recurrent neural networks for solving constrained least absolute deviation problems. IEEE Transactions on Neural Networks, 21, 1073-1086.
Liang, M., & Hu, X. (2015). Recurrent convolutional neural network for object recognition. Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3367-3375.
Schepers, I. M., Hipp, J. F., Schneider, T. R., Röder, B., & Engel, A. K. (2012). Functionally specific oscillatory activity correlates between visual and auditory cortex in the blind. Brain, 135, 922-934.
Siegel, M., Donner, T. H., Oostenveld, R., Fries, P., & Engel, A. K. (2008). Neuronal synchronization along the dorsal visual pathway reflects the focus of spatial attention. Neuron, 60, 709-719.
Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W., Desimone, R., Engel, A. K., & Fries, P. (2007). Modulation of neuronal interactions through neuronal synchronization. Science, 316, 1609-1612.
Publications project B2 - Bayesian analysis of the interaction of learning, semantics and social influence
with crossmodal integration
Publications of the current project
Oganian, Y., Heekeren, H. R., & Korn, C. W. (2019). Low foreign language proficiency reduces optimism about the personal future. Quarterly Journal of Experimental Psychology, 72(1), 60-75.
Korn, C. W., Heekeren, H. R., & Oganian, Y. (2019). The framing effect in a monetary gambling task is robust in minimally verbal language switching contexts. Quarterly Journal of Experimental Psychology, 72(1), 52-59.
Liu, C., Zhuo, J., & Zhu, J. (2019). Understanding MCMC Dynamics as Flows on the Wasserstein Space. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
Liu, C., Zhuo, J., Chen P., Zhang, R., & Zhu, J. (2019). 135 Understand and Accelerate Particle-based Variational Inference. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
Shi, J., Khan, M. E., & Zhu, J. (2019). Scalable Training of Inference Networks for Gaussian-Process Models. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
Pang, T., Xu, K., Du, C., Chen, N., & Zhu, J. (2019). Improving Adversarial Robustness via Promoting Ensemble Diversity. In International Conference on Machine Learning (ICML), Long Beach, CA, USA.
Wang, Z., Ren, T., Zhu, J., & Zhang, B. (2019). Function space particle optimization for Bayesian neural networks. In International Conference on Learning Representations (ICLR), New Orleans, Louisiana, USA.
Dong, Y., Pang, T., Su, H., & Zhu, J. (2019). Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4312-4321.
Dong, Y., Su, H., Wu, B., Li, Z., Liu, W., Zhang, T., & Zhu, J. (2019). Efficient Decision-based Black-box Adversarial Attacks on Face Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7714-7722.
Wei, X., Zhu, J., & Su, H. (2019). Sparse adversarial perturbations for videos. In the 33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, Hawaii, USA.
Liu, C., & Zhu, J. (2018). Riemannian Stein variational gradient descent for Bayesian inference. Thirty-Second AAAI Conference on Artificial Intelligence.
Chen, J., Zhu, J., Teh, Y. W., & Zhang, T. (2018). Stochastic Expectation Maximization with variance reduction. Advances in Neural Information Processing Systems(NeurIPS), 7967-7977.
Shi, J., Sun, S., & Zhu, J. (2018). A spectral approach to gradient estimation for implicit distributions. Proc. of the 35th International Conference on Machine Learning (ICML), arXiv preprint arXiv:1806.02925.
Tian, T., Zhou, Y., & Zhu, J. (2018). Selective Verification Strategy for Learning from Crowds. In Thirty-Second AAAI Conference on Artificial Intelligence.
Du, C., Li, C., Zheng, Y., Zhu, J., & Zhang, B. (2018). Collaborative filtering with user-item co-autoregressive models. In Thirty-Second AAAI Conference on Artificial Intelligence.
Bayer, J., Gläscher, J., Finsterbusch, J., Schulte, L. H., & Sommer, T. (2018). Linear and inverted U-shaped dose-response functions describe estrogen effects on hippocampal activity in young women. Nature Communications, 9(1), 1220.
Korn, C. W., & Bach, D. R. (2018). Heuristic and optimal policy computations in the human brain during sequential decision-making. Nature Communications, 9(1), 325.
Rosenblau, G., Korn, C. W., & Pelphrey, K. A. (2018). A computational account of optimizing social predictions reveals that adolescents are conservative learners in social contexts. Journal of Neuroscience, 38(4), 974-988.
Pang, T., Du, C., Dong, Y., & Zhu, J. (2018). Towards robust detection of adversarial examples. In Proc. of Advances in Neural Information Processing Systems (NeurIPS), Montreal, Canada, 4579-4589. (Spotlight, NVIDIA Pioneering Research Award)
Li, C., Welling, M., Zhu, J., & Zhang, B. (2018). Graphical Generative Adversarial Networks, In Proc. of Advances in Neural Information Processing Systems (NeurIPS), Montreal, Canada
Luo, Y., Tian, T., Shi, J., Zhu, J., & Zhang, B. (2018). Semi-crowdsourced clustering with deep generative models. In Proc. of Advances in Neural Information Processing Systems (NeurIPS), Montreal, Canada, 3212-3222.
Li, J., Su, H., Zhu, J., & Zhang, B. (2018). Essay-Anchor Attentive Multi-Modal Bilinear Pooling for Textbook Question Answering. In 2018 IEEE International Conference on Multimedia and Expo (ICME), 1-6.
Wei, X., Zhu, J., Feng, S., & Su, H. (2018). Video-to-video translation with global temporal consistency. In 2018 ACM Multimedia Conference on Multimedia Conference, Seoul, Korea, 18-25.
Shi, J., Sun, S., & Zhu, J. (2017). Kernel implicit variational inference. Proc. of the 6th International Conference on Learning Representations (ICLR), arXiv preprint arXiv:1705.10119.
Zhuo, J., Liu, C., Shi, J., Zhu, J., Chen, N., & Zhang, B. (2017). Message passing stein variational gradient descent. Proc. of the 35th International Conference on Machine Learning (ICML), arXiv preprint arXiv:1711.04425.
Li, C., Zhu, J., & Zhang, B. (2017). Max-margin deep generative models for (semi-) supervised learning. IEEE transactions on pattern analysis and machine intelligence, 40(11), 2762-2775.
Ren, Y., Wang, Y., & Zhu, J. (2017). Spectral learning for supervised topic models. IEEE transactions on pattern analysis and machine intelligence, 40(3), 726-739.
Li, C., Xu, K., Zhu, J., & Zhang, B. (2017). Triple generative adversarial nets. 31st Conference on Neural Information Processing Systems(NeurIPS), Long Beach, CA, USA, 4088-4098.
Liu, M., Shi, J., Cao, K., Zhu, J., & Liu, S. (2017). Analyzing the training processes of deep generative models. IEEE transactions on visualization and computer graphics, 24(1), 77-87.
Deng, Z., Zhang, H., Liang, X., Yang, L., Xu, S., Zhu, J., & Xing, E. P. (2017). Structured generative adversarial networks. Proc. of Advances in Neural Information Processing Systems(NeurIPS), 3899-3909.
Zhou, Y., Li, J., & Zhu, J. (2017). Identify the Nash Equilibrium in static games with random payoffs. In Proceedings of the 34th International Conference on Machine Learning (ICML), 4160-4169.
Liu, S., Xiao, J., Liu, J., Wang, X., Wu, J., & Zhu, J. (2017). Visual diagnosis of tree boosting methods. IEEE transactions on visualization and computer graphics, 24(1), 163-173.
Dong, Y., Ni, R., Li, J., Chen, Y., Zhu, J., & Su, H. (2017). Learning accurate low-bit deep neural networks with stochastic quantization. The 28th British Machine Vision Conference (BMVC), arXiv preprint arXiv:1708.01001.
Ren, Y., & Zhu, J. (2017). Distributed Accelerated Proximal Coordinate Gradient Methods. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2655-2661.
Hu, W., Zhu, J., Su, H., Zhuo, J., & Zhang, B. (2017). Semi-supervised Max-margin Topic Model with Manifold Posterior Regularization. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 1865-1871.
Liu, M., Jiang, L., Liu, J., Wang, X., Zhu, J., & Liu, S. (2017). Improving Learning-from-Crowds through Expert Validation. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2329-2336.
Dong, Y., Su, H., Zhu, J., & Zhang, B. (2017). Improving interpretability of deep neural networks with semantic information. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4306-4314.
Tian, T., Chen, N., & Zhu, J. (2017). Learning attributes from the crowdsourced relative labels. Thirty-First AAAI Conference on Artificial Intelligence.
Korn, C. W., Zaiser, J., Schalk, L., Oganian, Y., & Saalbach, H. (2017). Hard-to-read fonts do not influence the framing effect. Psychonom Bull & Rev 25:2:696-703.
Chen, J., Zhu, J., & Song, L. (2017). Stochastic training of graph convolutional networks with variance reduction. In Proc. of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden.
Relevant previous work
Gläscher, J., & Büchel, C. (2005). Formal learning theory dissociates brain regions with different temporal integration. Neuron, 47, 295-306.
Gläscher, J., Hampton, A. N., & O’Doherty, J. (2009). Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex, 19, 483-495.
Gläscher, J., & O’Doherty, J. (2010). Model-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data. In Sanfey AG, Nadel L (Eds.), Wiley Interdisciplinary Reviews: Cognitive Science, 1, 501-510.
Gläscher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585-595.
Gläscher, J., Adolphs, R., Damasio, H., Bechara, A., Rudrauf, D., Calamia, M., Paul, L. K., & Tranel, D. (2012). Lesion map-ping of cognitive control and value-based decision-making in the prefrontal cortex. Proc Natl Acad Sci USA, 109, 14681-14686.
Chen, N., Zhu, J., Sun, F., & Xing, E. P. (2012). Large-margin Subspace Learning for Multi-view Data Analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence, 34, 2365-2378.
Zhu, J., Ahmed, A., & Xing, E. P. (2012). MedLDA: Maximum Margin Supervised Topic Models. Journal of Machine Learning Research, 13, 2237-2278.
Zhu, J., & Xing, E. P. (2009). Maximum Entropy Discrimination Markov Networks. Journal of Machine Learning Research, 10, 2531-2569.
Zhu, J., Chen, N., & Xing, E. P. (2014a) Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs. Journal of Machine Learning Research, 15, 1799-1847.
Zhu, J., Chen, N., Perkins, H., & Zhang, B. (2014b) Gibbs Max-margin Topic Models with Data Augmentation. Journal of Machine Learning Research, 15, 1073-1110.
Gläscher, J., Hampton, A. N., & O’Doherty, J. (2009). Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex, 19, 483-495.
Gläscher, J., & O’Doherty, J. (2010). Model-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data. In Sanfey AG, Nadel L (Eds.), Wiley Interdisciplinary Reviews: Cognitive Science, 1, 501-510.
Gläscher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585-595.
Gläscher, J., Adolphs, R., Damasio, H., Bechara, A., Rudrauf, D., Calamia, M., Paul, L. K., & Tranel, D. (2012). Lesion map-ping of cognitive control and value-based decision-making in the prefrontal cortex. Proc Natl Acad Sci USA, 109, 14681-14686.
Chen, N., Zhu, J., Sun, F., & Xing, E. P. (2012). Large-margin Subspace Learning for Multi-view Data Analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence, 34, 2365-2378.
Zhu, J., Ahmed, A., & Xing, E. P. (2012). MedLDA: Maximum Margin Supervised Topic Models. Journal of Machine Learning Research, 13, 2237-2278.
Zhu, J., & Xing, E. P. (2009). Maximum Entropy Discrimination Markov Networks. Journal of Machine Learning Research, 10, 2531-2569.
Zhu, J., Chen, N., & Xing, E. P. (2014a) Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs. Journal of Machine Learning Research, 15, 1799-1847.
Zhu, J., Chen, N., Perkins, H., & Zhang, B. (2014b) Gibbs Max-margin Topic Models with Data Augmentation. Journal of Machine Learning Research, 15, 1073-1110.
Publications project B3 - Neurocognitive mechanisms for implicit learning of crossmodal predictions
Publications of the current project
Taesler, P., Jablonowski, J., Fu, Q., & Rose, M. (2019). Modeling implicit learning in a cross-modal audio-visual serial reaction time task. Cognitive Systems Research, 54, 154-164.
Fu, Q., Sun, H., Dienes, Z., & Fu, X. (2019). Dataset of implicit sequence learning of chunking and abstract structures. Data in brief, 22, 72-75.
Zhou, X., Fu, Q., Rose, M. R., & Sun, Y. (2019). Which Matters More in Incidental Category Learning: Edge-based vs. Surface-based Features. Frontiers in Psychology, 10, 183.
Sun, X, Sun, Y., & Fu, Q. (2019). Cross-modal learning and its cognitive and neural mechanisms. Progress in Biochemistry and Biophysics, 46(6), 565-577. (in Chinese)
Wu, J., Fu*, Q., Zhou, X., & Sun, X. (2018). The effect of presenting mode of different features on the acquisition of rule-based and similarity-based knowledge in category learning. Journal of Psychological Science, 41(5), 1–6. (in Chinese).
Jablonowski, J., Taesler, P., Fu, Q., & Rose, M. (2018). Implicit acoustic sequence learning recruits the hippocampus. PloS one, 13(12), e0209590.
Fu, Q., Sun, H., Dienes, Z., & Fu, X. (2018). Implicit sequence learning of chunking and abstract structures. Consciousness and Cognition, 62, 42-56.
Clos, M., Sommer, T., Schneider, S. L., & Rose, M. (2018). Enhanced transformation of incidentally learned knowledge into explicit memory by dopaminergic modulation. PloS one, 13(6), e0199013.
Fu, Q., Liu, Y. J., Dienes, Z., Wu, J., Chen, W., & Fu, X. (2017). Neural correlates of subjective awareness for natural scene categorization of color photographs and line-drawings. Frontiers in psychology, 8, 210.
Taesler, P., & Rose, M. (2017). Psychophysically-anchored, Robust Thresholding in Studying Pain-related Lateralization of Oscillatory Prestimulus Activity. Journal of Visualized Experiments, (119), e55228.
Taesler, P. & Rose, M. (2016). Psychophysically Anchored, Robust Thresholding in Studying Pain Related Lateralization of Oscillatory Pre-Stimulus Activity. Journal of Visualized Experiments, e55228.
Relevant previous work
Fu, Q., Bin G, Dienes, Z., Fu, X., & Gao, X. (2013a). Learning without consciously knowing: Evidence from event-related potentials in sequence learning. Conscious Cogn, 22, 22–34.
Fu, Q., Dienes Z, & Fu, X. (2010). Can unconscious knowledge allow control in sequence learning? Conscious Cogn, 19, 462–474.
Fu, Q., Dienes, Z, Shang, J., & Fu, X. (2013b). Who learns more? Cultural differences in implicit sequence learning. PLoS One, 8, e71625.
Fu, Q., Fu, X., & Dienes, Z. (2008). Implicit sequence learning and conscious awareness. Conscious Cogn, 17, 185–202.
Gao, X., Xu, D., Cheng, M., & Gao, S. (2003). A BCI-based environmental controller for the motion-disabled. IEEE Trans Neural Syst Rehabil Eng, 11, 137-140.
Haider, H., Eberhardt, K., Kunde, A., & Rose, M. (2013). Implicit visual learning and the expression of learning. Conscious Cogn, 22, 82-98.
Rose, M., Haider, H., & Büchel, C. (2010). The emergence of explicit memory during learning. Cerebral Cortex, 12, 2787-2797.
Rose, M., Haider, H., Salari, N. & Büchel, C. (2011). Functional dissociation of hippocampal mechanism during implicit learning based on the domain of associations. The Journal of Neuroscience, 31, 13739-13745.
Rose, M., Haider, H., Weiller, C., & Büchel, C. (2002). The role of medial temporal lobe structures in implicit learning: an event-related fMRI study. Neuron, 36, 1221-1231.
Schmid, C., Büchel, C., & Rose, M. (2011). The neural basis of visual dominance in the context of audio-visual object processing. Neuroimage, 55, 304-311.
Fu, Q., Dienes Z, & Fu, X. (2010). Can unconscious knowledge allow control in sequence learning? Conscious Cogn, 19, 462–474.
Fu, Q., Dienes, Z, Shang, J., & Fu, X. (2013b). Who learns more? Cultural differences in implicit sequence learning. PLoS One, 8, e71625.
Fu, Q., Fu, X., & Dienes, Z. (2008). Implicit sequence learning and conscious awareness. Conscious Cogn, 17, 185–202.
Gao, X., Xu, D., Cheng, M., & Gao, S. (2003). A BCI-based environmental controller for the motion-disabled. IEEE Trans Neural Syst Rehabil Eng, 11, 137-140.
Haider, H., Eberhardt, K., Kunde, A., & Rose, M. (2013). Implicit visual learning and the expression of learning. Conscious Cogn, 22, 82-98.
Rose, M., Haider, H., & Büchel, C. (2010). The emergence of explicit memory during learning. Cerebral Cortex, 12, 2787-2797.
Rose, M., Haider, H., Salari, N. & Büchel, C. (2011). Functional dissociation of hippocampal mechanism during implicit learning based on the domain of associations. The Journal of Neuroscience, 31, 13739-13745.
Rose, M., Haider, H., Weiller, C., & Büchel, C. (2002). The role of medial temporal lobe structures in implicit learning: an event-related fMRI study. Neuron, 36, 1221-1231.
Schmid, C., Büchel, C., & Rose, M. (2011). The neural basis of visual dominance in the context of audio-visual object processing. Neuroimage, 55, 304-311.
Publications project B4 - Brain dynamics of crossmodal learning and conflict processing
Publications of the current project
Avarvand, F. S., Bartz, S., Andreou, C., Samek, W., Leicht, G., Mulert, C., ... & Nolte, G. (2018). Localizing bicoherence from EEG and MEG. Neuroimage, 174, 352-363.
Daume, J., Graetz, S., Gruber, T., Engel, A. K., & Friese, U. (2017). Cognitive control during audiovisual working memory engages frontotemporal theta-band interactions. Scientific Reports, 7(1), 12585.
Li, Q.*, Yang, G.*, Li, Z., Qi, Y., Cole, M. W., & Liu, X. (2017). Conflict detection and resolution rely on a combination of common and distinct cognitive control networks. Neuroscience & Biobehavioral Reviews, 83, 123-131 (* contributed equally).
Li, Q., Wang, K., Nan, W., Zheng, Y., Wu, H., Wang, H., & Liu, X. (2015). Electrophysiological dynamics reveal distinct processing of stimulus-stimulus and stimulus-response conflicts. Psychophysiology, 52(4), 562-571.
Li, Z, Yang, G., Nan, W., Li, Q., & Liu, X. (2018). Attentional regulation mechanisms of cognitive control in conflict resolution.
Advances in Psychological Science, 26(6), 966-974.
Nolte, G., Aburidi, M., & Engel, A. K. (2019) Robust calculation of slopes in detrended fluctuation analysis and its application to envelopes of human alpha rhythms. Scientific Reports, 9 (1), 6339.
Wang, P.*, Göschl, F.*, Friese, U., König, P., & Engel, A.K. (2019). Long-range functional coupling predicts performance: Oscillatory EEG networks in multisensory processing. NeuroImage, 196, 114-125 (* contributed equally).
Yang, G., Li, Z., Wu, H., & Liu, X. (2019). Generality and specificity of cognitive control: research logics and debates. Acta Physiologica Sinica, 71(01), 140-148.
Yang, G., Nan, W., Zheng, Y., Wu, H., Li, Q., & Liu, X. (2017). Distinct cognitive control mechanisms as revealed by modality-specific conflict adaptation effects. Journal of experimental psychology: human perception and performance, 43(4), 807-818.
Parisi, G., Barros, P., Fu, D., Magg, S., Wu, H., Liu, X., & Wermter, S. (2019). A Neurorobotic Experiment for Crossmodal Conflict Resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 33-36.
Zamani, M. A., Magg, S., Weber, C., Wermter, S., & Fu, D. (2018). Deep reinforcement learning using compositional representations for performing instructions. Paladyn Journal of Behavioral Robotics, 9(1), 358-373.
Barros, P., Parisi, G.I., Fu, D., Liu, X., Wermter, S. (2017) Adaptive crossmodal stimuli association using expectation learning. European Society for Cognitive Systems, Zurich, Switzerland.
杨国春, 李政汉, 伍海燕, & 刘勋. (2019). 认知控制的一般性/特异性机制: 研究逻辑和争论. 生理学报, (1), 14.
李政汉, 杨国春, 南威治, 李琦, & 刘勋. (2018). 冲突解决过程中认知控制的注意调节机制. 心理科学进展, 26(6), 966-974.
Relevant previous work
Ewald, A., Marzetti, L., Zappasodi, F., Meinecke, F. C., & Nolte, G. (2012). Estimating true brain connectivity from EEG/MEG data invariant to linear and static transformations in sensor space. NeuroImage, 60(1), 476-488.
Li, Q., Wang, K., Nan, W., Zheng, Y., Wu, H., Wang, H., & Liu, X. (2015). Electrophysiological dynamics reveal distinct processing of stimulus-stimulus and stimulus-response conflicts. Psychophysiology, 52(4), 562-571.
Liu, X., Banich, M. T., Jacobson, B. L., & Tanabe, J. L. (2004). Common and distinct neural substrates of attentional control in an integrated Simon and spatial Stroop task as assessed by event-related fMRI. Neuroimage, 22(3), 1097-1106.
Liu, X., Banich, M. T., Jacobson, B. L., & Tanabe, J. L. (2006). Functional dissociation of attentional selection within PFC: response and non-response related aspects of attentional selection as ascertained by fMRI. Cerebral Cortex, 16(6), 827-834.
Liu, X., Park, Y., Gu, X., & Fan, J. (2010). Dimensional overlap accounts for independence and integration of stimulus—response compatibility effects.Attention, Perception, & Psychophysics, 72(6), 1710-1720.
Luft, C. D. B., Nolte, G., & Bhattacharya, J. (2013). High-learners present larger mid-frontal theta power and connectivity in response to incorrect performance feedback. The Journal of Neuroscience, 33(5), 2029-2038.
Nolte, G. (2003). The magnetic lead field theorem in the quasi-static approximation and its use for magnetoencephalography forward calculation in realistic volume conductors. Physics in medicine and biology, 48(22), 3637.
Nolte, G., Bai, O., Wheaton, L., Mari, Z., Vorbach, S., & Hallett, M. (2004). Identifying true brain interaction from EEG data using the imaginary part of coherency. Clinical neurophysiology, 115(10), 2292-2307.
Nolte, G., Ziehe, A., Nikulin, V. V., Schlögl, A., Krämer, N., Brismar, T., & Müller, K. R. (2008). Robustly estimating the flow direction of information in complex physical systems. Physical review letters, 100(23), 234101.
Wang, K., Li, Q., Zheng, Y., Wang, H., & Liu, X. (2014). Temporal and spectral profiles of stimulus–stimulus and stimulus–response conflict processing. Neuroimage, 89, 280-288.
Li, Q., Wang, K., Nan, W., Zheng, Y., Wu, H., Wang, H., & Liu, X. (2015). Electrophysiological dynamics reveal distinct processing of stimulus-stimulus and stimulus-response conflicts. Psychophysiology, 52(4), 562-571.
Liu, X., Banich, M. T., Jacobson, B. L., & Tanabe, J. L. (2004). Common and distinct neural substrates of attentional control in an integrated Simon and spatial Stroop task as assessed by event-related fMRI. Neuroimage, 22(3), 1097-1106.
Liu, X., Banich, M. T., Jacobson, B. L., & Tanabe, J. L. (2006). Functional dissociation of attentional selection within PFC: response and non-response related aspects of attentional selection as ascertained by fMRI. Cerebral Cortex, 16(6), 827-834.
Liu, X., Park, Y., Gu, X., & Fan, J. (2010). Dimensional overlap accounts for independence and integration of stimulus—response compatibility effects.Attention, Perception, & Psychophysics, 72(6), 1710-1720.
Luft, C. D. B., Nolte, G., & Bhattacharya, J. (2013). High-learners present larger mid-frontal theta power and connectivity in response to incorrect performance feedback. The Journal of Neuroscience, 33(5), 2029-2038.
Nolte, G. (2003). The magnetic lead field theorem in the quasi-static approximation and its use for magnetoencephalography forward calculation in realistic volume conductors. Physics in medicine and biology, 48(22), 3637.
Nolte, G., Bai, O., Wheaton, L., Mari, Z., Vorbach, S., & Hallett, M. (2004). Identifying true brain interaction from EEG data using the imaginary part of coherency. Clinical neurophysiology, 115(10), 2292-2307.
Nolte, G., Ziehe, A., Nikulin, V. V., Schlögl, A., Krämer, N., Brismar, T., & Müller, K. R. (2008). Robustly estimating the flow direction of information in complex physical systems. Physical review letters, 100(23), 234101.
Wang, K., Li, Q., Zheng, Y., Wang, H., & Liu, X. (2014). Temporal and spectral profiles of stimulus–stimulus and stimulus–response conflict processing. Neuroimage, 89, 280-288.
Publications project B5 - Crossmodal fusion for dexterous manipulation in proactive human-robot collaboration
Publications from phase 1
Hongzhuo Liang*, Xiaojian Ma*, Shuang Li, Michael Görner, Song Tang, Bin Fang, Fuchun Sun, and Jianwei Zhang (2019).
PointNetGPD: Detecting Grasp Configurations from Point Sets,
IEEE International Conference on Robotics and Automation, ICRA 2019, Montreal, Canada.
(pdf)
Shuang Li*, Xiaojian Ma*, Hongzhuo Liang, Michael Görner, Philipp Ruppel, Bin Fang, Fuchun Sun, and Jianwei Zhang (2019).
Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network,
IEEE International Conference on Robotics and Automation, ICRA 2019, Montreal, Canada.
(pdf)
Zhen Deng, Ge Gao, Simone Frintrop, Fuchun Sun, Changshui Zhang, Jianwei Zhang. (2019).
Attention based visual analysis for fast grasp planning with a multi-fingered robotic hand.
Frontiers in Neurorobotics, 2019.
DOI: 10.3389/fnbot.2019.00060.
Liu, C., Fang, B., Sun, F., Li, X., & Huang, W. (2019). Learning to Grasp Familiar Objects Based on Experience and Objects' Shape Affordance. IEEE Transactions on Systems, Man, and Cybernetics: Systems (early access).
Wang, T., Yang, C., Kirchner, F., Du, P., Sun, F., & Fang, B. (2019). Multimodal grasp data set: A novel visual–tactile data set for robotic manipulation. International Journal of Advanced Robotic Systems, 16(1), 1729881418821571.
Fuchun, Sun, and Liu Huaping (2019). A Novel Multi-modal Tactile Sensor Design using Thermochromic Material. SCIENCE CHINA Information Sciences.
Fang, B., Wei, X., Sun, F., Huang, H., Yu, Y., & Liu, H. (2019). Skill learning for human-robot interaction using wearable device. Tsinghua Science and Technology, 24(6), 654-662.
Deng, Z., Guan, H., Huang, R., Liang, H., Zhang, L., & Zhang, J. (2019). Combining Model-Based Q-Learning With Structural Knowledge Transfer for Robot Skill Learning. IEEE Transactions on Cognitive and Developmental Systems, 11(1), 26-35.
Starke, S., Hendrich, N., & Zhang, J. (2019). Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation. IEEE Transactions on Evolutionary Computation, 23(3), 406-420.
Deng, Z., Gao, G., Frintrop, S., Zhang, J. (2019). Attention based visual analysis for fast grasp planning with multi-fingered robotic hand. Frontiers in Neurorobotics. doi: 10.3389/fnbot.2019.00060. (joint paper)
Jing, M., Ma, X., Huang, W., Sun, F., & Liu, H. (2018). Task Transfer by Preference-Based Cost Learning. arXiv preprint arXiv:1805.04686.
Deng, Z., Zheng, X., Zhang, L., & Zhang, J. (2018). A learning framework for semantic reach-to-grasp tasks integrating machine learning and optimization. Robotics and Autonomous Systems, 108, 140-152.
Starke, S., Hendrich, N., Zhang, J. (2018).
A Forward Kinematics Data Structure for Efficient Evolutionary Inverse Kinematics,
in S. Zeghloul et al. (eds.), Computational Kinematics, Mechanisms and Machine Science 50, Springer 2018.
DOI: 10.1007/978-3-319-60867-9_64
Fang, B., Sun, F., Yang, C., Xue, H., Chen, W., Zhang, C., Guo, D., Liu, H.
(2018).
A Dual-Modal Vision-based Tactile Sensor for Robotic Hand Grasping,
IEEE International Conference on Robotics and Automation, (ICRA-2018), Brisbane, Australia.
Gao, G., Lauri, M., Zhang, J., and Frintrop, S. (2018).
Occlusion Resistant Object Rotation Regression from Point Cloud Segments,
Proceedings of the ECCV workshop on Recovering 6D Object Pose, 2018.
https://arxiv.org/abs/1808.05498.
Tao Kong, Fuchun Sun, et al. (2018).
Deep Feature Pyramid Reconfiguration for Object Dectection.
ECCV, 2018.
Liang, H., Li, S., Görner, M., and Zhang, J. (2018).
Generating Robust Grasps for Unknown Objects in Clutter Using Point Cloud Data.
Shanghai International Symposium on Human-Centered Robotics (HCR), 299-303, 2018.
Liang, H., and Zhao, Q. (2018).
Multi-View CNNs for 3D Hand Pose Estimation,
Shanghai International Symposium on Human-Centered Robotics (HCR), 255-259, 2018.
Sebastian Starke, Norman Hendrich, and Jianwei Zhang (2018).
Memetic Evolution for Generic Full-Body Inverse Kinematics in Robotics and Animation,
IEEE Transactions on Evolutionary Computation, 2018.
doi: 10.1109/TEVC.2018.2867601
Jing M, Ma X, Sun F, et al. (2018).
Learning and Inference Movement with Deep Generative Model.
arXiv preprint arXiv:1805.07252, 2018.
Ma X, Jing M, Sun F, et al. (2018).
Adversarial Task Transfer from Preference. arXiv preprint arXiv:1805.04686, 2018
Fuchun Sun, Wenchang Zhang, et al. (2018).
Fused Fuzzy Petri Nets: A Shared Control Method for Brain Computer Interface Systems,
IEEE Trans. On Cognitive and Developmental Systems, 2018.
Ruppel, P., Hendrich, N., Starke, S., Zhang, J. (2018).
Cost Functions to Specify Full-Body Motion and Multi-Goal Manipulation Tasks,
IEEE International Conference on Robotics and Automation, (ICRA-2018), Brisbane, Australia.
Han, D., Nie, H., Chen, J., Chen, M., Deng, Z., Zhang, J. (2018).
Multi-modal haptic image recognition based on deep learning.
Sensor Review.
Tao Kong, Fuchun Sun, (2017).
RON: Reverse Connection with Objectness Prior Networks for Object Detection,
CVPR 2017.
Huang, W., Harandi, M., Zhang, T., Fan, L., Sun, F., & Huang, J. (2017). Efficient optimization for linear dynamical systems with applications to clustering and sparse coding. Proc. of Advances in Neural Information Processing Systems(NeurIPS), 3444-3454.
Liu, C., Sun, F., Wang, C., Wang, F., & Yuille, A. (2017). MAT: A multimodal attentive translator for image captioning. Proc. of International Joint Conference on Artificial Intelligence (IJCAI), arXiv preprint arXiv:1702.05658.
Huang, Z., Sun, F., Min, H., Fang, B., Zhang, W., & Hu, X. (2017, December). A novel wearable tactile sensor array designed for fingertip motion recognition. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO) (pp. 165-170)
Li, L., Sun, F., Fang, B., Huang, Z. Yang, C., & Jing, M. (2017, December). Learning to detect slip for stable grasping. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).
Wasserfall, F., Hendrich, N., & Zhang, J. (2017, August). Adaptive slicing for the FDM process revisited. In 2017 13th IEEE Conference on Automation Science and Engineering (CASE) (pp. 49-54)
Starke, S., Hendrich, N., & Zhang, J. (2017, June). A memetic evolutionary algorithm for real-time articulated kinematic motion. In 2017 IEEE Congress on Evolutionary Computation (CEC) (pp. 2473-2479).
Starke, S., Hendrich, N., Krupke, D., & Zhang, J. (2017). Evolutionary multi-objective inverse kinematics on highly articulated and humanoid robots. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 6959-6966).
Bestmann, M., Wasserfall, F., Hendrich, N., & Zhang, J. (2017, December). Replacing cables on robotic arms by using serial via Bluetooth. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO) (pp. 189-195).
Huaping Liu, Fuchun Sun, Di Guo, Bin Fang, (2017).
Structured output-associated dictionary learning for haptic understanding,
IEEE Transactions on Systems, Man and Cybernetics: Systems, vol.47, no.7, 2017, pp.1564-1574
Deng, Z., Guan, H., Huang, R., Liang, H., Zhang, L., Zhang, J. (2017).
Combining Model-based Q-learning with Structural Knowledge Transfer for Robot Skill Learning.
IEEE Transactions on Cognitive and Developmental Systems.
Huang, T., Sun, F., Fang, B., (2017).
A novel wearable tactile sensor for fingertip motion recognition.
IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
Li, X., Sun, F., Fang, B.,(2017).
Learning to detect slip for stable grasping.
IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
Wei, X., Sun, F., Fang, B., (2017).
Robotic skills learning using dynamical movement primitives from a wearable device.
IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
Bestmann, M., Wasserfall, F., Hendrich, N., Zhang, J. (2017).
Replacing Cables on Robotic Arms by Using Serial via Bluetooth,
IEEE Intl. Conference on Robotics and Biomimetics, (ROBIO-2017), Macau, China.
DOI: 10.1109/ROBIO.2017.8324416
Starke, S., Hendrich, N., Zhang, J. (2017).
Multi-Objective Evolutionary Optimisation for Inverse Kinematics on Highly Articulated and Humanoid Robots,
IEEE Intl. Conference on Intelligent Robots and Systems, (IROS-2017), Vancouver, Canada.
DOI: 10.1109/IROS.2017.8206620
Wasserfall, F., Hendrich, N., Fiedler, F., Zhang, J. (2017).
3D-Printed Low-Cost Modular Force Sensors,
20th Intl. Conference on Climbing and Walking Robots (CLAWAR-2017).
In "Human-Centric Robotics", ISBN: 978-981-3231-03-0 (hardcover). ISBN: 978-981-3231-05-4 (ebook).
Wasserfall, F., Hendrich, N., Zhang, J. (2017).
Adaptive Slicing for the FDM Process Revisited,
13th IEEE Conference on Automation Science and Engineering (CASE-2017).
DOI: 10.1109/COASE.2017.8256074
Starke, S., Hendrich, N., Zhang, J. (2017).
A Memetic Evolutionary Algorithm for Real-Time Articulated Kinematic Motion,
IEEE Congress on Evolutionary Computation (CEC 2017), p 2473-2479, Sán Sebastian, Spain, 2017.
DOI: 10.1109/CEC.2017.7969605
Starke, S., Hendrich, N., Magg, S., Zhang, J. (2016).
An Efficient Hybridization of Genetic Algorithms and Particle Swarm Optimization for Inverse Kinematics,
IEEE Intl. Conference on Robotics and Biomimetics (ROBIO 2016), 3-7 Dec. 2016, Qingdao, China.
DOI: 10.1109/ROBIO.2016.7866587
Relevant previous work
Bernardino, A., Henriques, M., Hendrich, N., & Zhang, J. (2013, December). Precision grasp synergies for dexterous robotic hands. In Proc. IEEE Conference on Robotics and Biomimetics, ROBIO 2013 (pp. 62-67). IEEE. (Best Conference Paper Award).
Chen, N., Zhu, J., Sun, F., & Xing, E. P. (2012). Large-margin predictive latent subspace learning for multiview data analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(12), 2365-2378.
Cheng, G., Hendrich, N., & Zhang, J. (2012, October). Action gist based automatic segmentation for periodic in-hand manipulation movement learning. In Proc. IEEE Conference on Intelligent Robots and Systems, IROS 2012 pp. 4768-4775). IEEE.
Guo, D., Sun, F., Zhang, J., & Liu, H. (2013, December). A grasp synthesis and grasp synergy analysis for anthropomorphic hand. Proc IEEE Conference on Robotics and Biomimetics, ROBIO 2013 (pp. 1617-1622). IEEE.
Hendrich, N., Klimentjew, D., & Zhang, J. (2010, September). Multi-sensor based segmentation of human manipulation tasks. Proc. IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI 2010. (pp. 223-229). IEEE.
Chen, N., Zhu, J., Sun, F., & Xing, E. P. (2012). Large-margin predictive latent subspace learning for multiview data analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(12), 2365-2378.
Cheng, G., Hendrich, N., & Zhang, J. (2012, October). Action gist based automatic segmentation for periodic in-hand manipulation movement learning. In Proc. IEEE Conference on Intelligent Robots and Systems, IROS 2012 pp. 4768-4775). IEEE.
Guo, D., Sun, F., Zhang, J., & Liu, H. (2013, December). A grasp synthesis and grasp synergy analysis for anthropomorphic hand. Proc IEEE Conference on Robotics and Biomimetics, ROBIO 2013 (pp. 1617-1622). IEEE.
Hendrich, N., Klimentjew, D., & Zhang, J. (2010, September). Multi-sensor based segmentation of human manipulation tasks. Proc. IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI 2010. (pp. 223-229). IEEE.
Hueser, M., Baier, T., & Zhang, J. (2006, May). Learning of demonstrated grasping skills by stereoscopic tracking of human head configuration. Proc. IEEE Conference on Robotics and Automation, ICRA 2006 (pp. 2795-2800). IEEE.
Liu, H., & Sun, F. (2012). Fusion tracking in color and infrared images using joint sparse representation. In Science China Information Sciences, 55(3), 590-599.
Pei, D., Liu, H., Liu, Y., & Sun, F. (2013, August). Unsupervised multimodal feature learning for semantic image segmentation. In Proc IEEE Conference on Neural Networks, IJCNN 2013 (pp. 1-6). IEEE.
Wang, H. Q., Sun, F. C., Cai, Y. N., Ding, L. G., & Chen, N. (2010). An unbiased LSSVM model for classification and regression. Soft Computing, 14(2), 171-180.
Zhang, J., & Rössler, B. (2004). Self-valuing learning and generalization with application in visually guided grasping of complex objects. Robotics and Autonomous systems, 47(2), 117-127.
Liu, H., & Sun, F. (2012). Fusion tracking in color and infrared images using joint sparse representation. In Science China Information Sciences, 55(3), 590-599.
Pei, D., Liu, H., Liu, Y., & Sun, F. (2013, August). Unsupervised multimodal feature learning for semantic image segmentation. In Proc IEEE Conference on Neural Networks, IJCNN 2013 (pp. 1-6). IEEE.
Wang, H. Q., Sun, F. C., Cai, Y. N., Ding, L. G., & Chen, N. (2010). An unbiased LSSVM model for classification and regression. Soft Computing, 14(2), 171-180.
Zhang, J., & Rössler, B. (2004). Self-valuing learning and generalization with application in visually guided grasping of complex objects. Robotics and Autonomous systems, 47(2), 117-127.