Knowledge Organization and Visualization for Text-Dominant Multimodal Documents

Xiaoyu Zhang
City University of Hong Kong
报告摘要:The management, integration, and presentation of knowledge from documents, human expertise, and embodied practice remain a big challenge in knowledge engineering. In this talk, I will share our research into structuring and visualizing such knowledge across science, education, and cultural heritage, focusing specifically on text-dominant multimodal data. Central to this approach is how we can leverage and extend standard ontologies to convert unstructured documents into coherent, interactive learning experiences where visualization serves as the primary bridge between raw data and structured cognitive spaces. In particular, I will discuss our work in document analysis, showing how domain ontologies map text structures and enable customizable summaries, as well as educational visualization, illustrating how knowledge-centric aids can be carefully moderated to enhance textbook reading and video lectures. I will also show how these semantic and interactive techniques scale to the digital humanities through our ongoing efforts to preserve and visualize the Hong Kong Martial Arts Archives. This talk offers a glimpse into my vision where visualization doesn't just display information, but actively transforms static data into living, dynamic experiences for complex human heritage.
讲者简介:Xiaoyu Zhang is a tenure-track Assistant Professor at School of Creative Media, City University of Hong Kong. Her research interests lies broadly in data visualization, education technology, human-computer interaction (HCI), human-AI collaboration, and explainable AI. She particularly focuses on integrating data visualization and artificial intelligence (AI), especially natural language processing (NLP), to explore and exploit knowledge for varies applications in education, smart manufacturing, and human well-being. Xiaoyu directs the KnowVis lab at CityU and her work has been published at top-tier journals and conferences including International Journal of Artificial Intelligence in Education, IEEE TVCG, IEEE VIS, ACM CHI, ACM IUI, IEEE PacificVis, ACL. She also received the Honorable Mention Best Paper Award at IEEE VIS 2022 and has been granted multiple US patents. Xiaoyu also served on the organization committee of AI and the Future Factory at AI House Davos 2024, IEEE VIS 2023 (Workshop), ACM DIS 2026 & 2027 . Before joining CityU, Xiaoyu was a Postdoctoral Fellow at the ETH AI Center, ETH Zurich. She received her Ph.D. degree in Computer Science from University of California, Davis. In addition to her academic pursuits, Xiaoyu has profound industrial internship and collaboration experience with leading organizations such as Meta, Microsoft Research Asia, Bosch Research, Netease Games, EthonAI, and the US National Institute of Standards and Technology (NIST).