Xianyu Chen

Biography [Resume] [Curriculum Vitae]

I am a sixth-year Ph.D. candidate at University of Minnesota - Twin Cities supervised by Prof. Catherine Qi Zhao in Computer Science from 2019 fall. My research interests include vision & language, human attention and few-shot learning, with the goal of developing human-centered AI systems.

I used to be an intern in Multimedia Research Center, Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences supervised by Prof. Yali Wang and Prof. Yu Qiao from August 2018 to January 2019. I focus on object detection, especially, low shots detection and life-long learning.

I have graduated and gotten my master degree in July 2018 in School of Electronics and Information Technology, Sun Yat-sen University, supervised by Prof. Ming Jiang. Previously, I received the B. Eng degree from School of Information Science and Technology in Sun Yat-sen University in 2015, under the supervision of Prof. Ming Jiang.

News

[08/2024] Our GazeXplain was selected as an oral paper by ECCV 2024! Thanks Ming and Qi.

[07/2024] Our paper was accepted by ECCV 2024! Thanks Ming and Qi.

[02/2024] Our paper was accepted by CVPR 2024! Thanks Ming and Qi.

[01/2024] Our paper was accepted by TPAMI (Impact Factor: 23.6)! Thanks James, Ming, Shi, Louis and Qi.

[03/2022] Our paper was accepted by CVPR 2022! Thanks James, Ming, Shi, Louis and Qi.

[04/2021] Our paper was accepted by IJCAI 2021! Thanks Qi and Ming.

[03/2021] Our paper was accepted by CVPR 2021! Thanks Qi and Ming.

[08/2019] Finish my two-month internship at Tencent AI Lab.

[02/2019] I am happy to accept the Ph.D. offer from University of Minnesota - Twin Cities and awarded a two year CSE Graduate Fellowship!

Selected Publication [Google Scholar]

Understanding and Solving the Real World Tasks

1. Xianyu Chen, Jinhui Yang, Shi Chen, Louis Wang, Ming Jiang, and Qi Zhao, "Every Problem, Every Step, All In Focus: Learning to Solve Real-World Problems with Integrated Attention" in IEEE Transactions on Pattern Analysis and Machine Intelligence Jan. 2024. [Code]
2. Jinhui Yang*, Xianyu Chen*, Ming Jiang, Shi Chen, Louis Wang and Qi Zhao, "VisualHow: Multimodal Problem Solving" in Proceedings of the 2022 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022). (*Co-first authors/Equal contribution) [Project Page] [Paper] [Code]

Understanding Human Attention in Performing General Tasks

1. Xianyu Chen, Ming Jiang and Qi Zhao, "GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths" in Proceedings of the 2024 European Conference on Computer Vision (ECCV 2024). (Oral: 2.3% [200/8585]). [Paper] [Code] [Video]
2. Xianyu Chen, Ming Jiang and Qi Zhao, "Beyond Average: Individualized Visual Scanpath Prediction" in Proceedings of the 2024 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2024). [Paper] [Code] [Video]
3. Xianyu Chen, Ming Jiang and Qi Zhao, "Predicting Human Scanpaths in Visual Question Answering" in Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021). [Project Page] [Paper] [Code] [Video]
4. Xianyu Chen, Ming Jiang and Qi Zhao, "Leveraging Human Attention in Novel Object Captioning" in Proceedings of the 2021 International Joint Conference on Artificial Intelligence (IJCAI 2021). [Paper] [Code] [2min Video] [15min Video]

Applications Related to Few-Shot Learning

1. Xianyu Chen, Ming Jiang and Qi Zhao, "Self-Distillation for Few-Shot Image Captioning" in Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV 2021). [Paper] [Code] [Video]
2. Ze Yang, Yali Wang, Xianyu Chen, Jianzhuang Liu and Yu Qiao, "Context-transformer: tackling object confusion for few-shot detection" in The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020), Hilton New York Midtown, New York, USA, 7-12 Feb. 2020. [PDF]
3. Xianyu Chen, Yali Wang, Jianzhuang Liu and Yu Qiao, "DID: Disentangling-Imprinting-Distilling for Continuous Low-Shot Detection" in IEEE Transactions on Image Processing, vol. 29, pp. 7765-7778, July 2020.
4. Xianyu Chen, Ming Jiang, and Qi Zhao, "Leveraging Bottom-Up and Top-Down Attention for Few-Shot Object Detection" in Arxiv 2020.

Skills

• Programming Language: Python, Matlab, JavaScript, HTML, C/C++, Linux shell
• Tools: Pytorch, Tensorflow, Keras, Opencv, Unix/Linux, Git, Scikit-Learn, LaTeX
• Mathematics: Matrix Theory, Convex Optimization, Probability Theory, Information Theory, Numerical Computation, Stochastic Processes and Introduction to Modern Mathematics

Achievements

Honors

Teaching Assistant

Academic Services

Journal Reviews:

• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2023-now)
• IEEE Transactions on Multimedia (TMM) (2023-now)
• IEEE Transactions on Neural Networks and Learning Systems (TNNLS) (2023-now)
• Journal of Visual Communication and Image Representation (2021-now)

Conference Reviews:

• International Conference on Computer Vision (ICCV'23)
• European Conference on Computer Vision (ECCV'22, ECCV'24)
• Conference on Computer Vision and Pattern Recognition (CVPR'22, CVPR'23)
• Winter Conference on Applications of Computer Vision (WACV'22, WACV'23, WACV'24, WACV'25)
• AAAI Conference on Artificial Intelligence (AAAI'23, AAAI'24, AAAI'25)