专题10:前沿论文专题 1


主持人:刘乐,副教授,西北工业大学


人:杨维铠,香港科技大学(广州),助理教授
报告题目:Dynamic Color Assignment for Hierarchical Data
论文作者:Jiashu Chen, Weikai Yang, Zelin Jia, Lanxi Xiao, Shixia Liu

发表会议:IEEE VIS 2024

报告摘要:Assigning discriminable and harmonic colors to samples according to their class labels and spatial distribution can generate attractive visualizations and facilitate data exploration. However, as the number of classes increases, it is challenging to generate a high-quality color assignment result that accommodates all classes simultaneously. A practical solution is to organize classes into a hierarchy and then dynamically assign colors during exploration. However, existing color assignment methods fall short in generating high-quality color assignment results and dynamically aligning them with hierarchical structures. To address this issue, we develop a dynamic color assignment method for hierarchical data, which is formulated as a multi-objective optimization problem. This method simultaneously considers color discriminability, color harmony, and spatial distribution at each hierarchical level. By using the colors of parent classes to guide the color assignment of their child classes, our method further promotes both consistency and clarity across hierarchical levels. We demonstrate the effectiveness of our method in generating dynamic color assignment results with quantitative experiments and a user study.

个人简历:杨维铠,香港科技大学(广州)信息枢纽数据科学与分析学域(DSA)和计算媒体与艺术(CMA)助理教授。2024年于清华大学获得博士学位。主要研究方向为可视分析与机器学习,已在可视分析和机器学习领域的顶级会议和期刊(IEEE VIS/IEEE TVCG/ACM MM)等发表论文10余篇,研究成果落地应用于上海数字交易所、北京冬奥组委、快手等机构和企业的解决方案中。他同时担任IEEE VIS、IEEE TVCG、PacificVis、ACM IUI、AAAI等国际学术期刊会议的程序委员会委员与审稿人。

报告人:朱倩,香港科技大学,博士生
报告题目:CompositingVis: Exploring Interaction for Creating Composite Visualizations in Immersive Environments
论文作者:Qian Zhu, Tao Lu, Shunan Guo, Xiaojuan Ma, Yalong Yang

发表会议:IEEE VIS 2024

报告摘要:Composite visualization represents a widely embraced design that combines multiple visual representations to create an integrated view. However, the traditional approach of creating composite visualizations in immersive environments typically occurs asynchronously outside of the immersive space and is carried out by experienced experts. In this work, we take the first step to empower users to participate in the creation of composite visualization within immersive environments through embodied interactions. This could provide a flexible and fluid experience for data exploration and facilitate a deep understanding of the relationship between data visualizations. We begin with forming a design space of embodied interactions to create various types of composite visualizations with the consideration of data relationships. Drawing inspiration from people's natural experience of manipulating physical objects, we design interactions to directly assemble composite visualizations in immersive environments. Building upon the design space, we present a series of case studies showcasing the interactive method to create different kinds of composite visualizations in Virtual Reality (VR). Subsequently, we conduct a user study to evaluate the usability of the derived interaction techniques and user experience of embodiedly creating composite visualizations. We find that empowering users to participate in composite visualizations through embodied interactions enables them to flexibly leverage different visualization representations for understanding and communicating the relationships between different views, which underscores the potential for a set of application scenarios in the future.

个人简介:朱倩,香港科技大学博士生,师从麻晓娟教授。她的主要研究方向包括人机交互,数据可视化和混合现实。她的研究关注混合现实中面向数据可视化的空间、多模态交互和智能系统设计,相关研究成果以第一作者身份发表在 IEEE VIS,ACM CHI,ACM CSCW,ACM DIS和人机交互A类期刊IJHCS (International Journal of Human-Computer Studies)上。个人主页: https://zhuqian.org/

报告人:曾星辰,香港科技大学(广州),博士生
报告题目:Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuning
论文作者:Xingchen Zeng, Haichuan Lin, Yilin Ye, Wei Zeng

发表会议:IEEE VIS 2024

报告摘要:Emerging multimodal large language models (MLLMs) exhibit great potential for chart question answering (CQA). Recent efforts primarily focus on scaling up training datasets (charts, data tables, and question-answer (QA) pairs) through data collection and synthesis. However, our empirical study on existing MLLMs and CQA datasets reveals notable gaps. First, current data collection and synthesis focus on data volume and lack consideration of fine-grained visual encodings and QA tasks, resulting in unbalanced data distribution divergent from practical CQA scenarios. Second, existing work follows the training recipe of the base MLLMs initially designed for natural images, under-exploring the adaptation to unique chart characteristics, such as rich text elements. To fill the gap, we propose a visualization-referenced instruction tuning approach to guide the training dataset enhancement and model development. Specifically, we propose a novel data engine to effectively filter diverse and high-quality data from existing datasets and subsequently refine and augment the data using LLM-based generation techniques to better align with practical QA tasks and visual encodings. Then, to facilitate the adaptation to chart characteristics, we utilize the enriched data to train an MLLM by unfreezing the vision encoder and incorporating a mixture-of-resolution adaptation strategy for enhanced fine-grained recognition. Experimental results validate the effectiveness of our approach. Even with fewer training examples, our model consistently outperforms state-of-the-art CQA models on established benchmarks. We also contribute a dataset split as a benchmark for future research.

个人简介:曾星辰,香港科技大学(广州)数据科学与分析学域一年级博士生,研究兴趣包括基座模型在可视化场景下的增强及高维数据可视化,曾在VIS,CHI,JoV上第一作者身份发表论文,个人主页是https://zengxingchen.github.io/

报告人:叶依林,香港科技大学(广州),博士生
报告题目:ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion Map
论文作者:Yilin Ye, Shishi Xiao, Xingchen Zeng, Wei Zeng

发表会议:IEEE VIS 2024

报告摘要:Multi-modal embeddings form the foundation for vision-language models, such as CLIP embeddings, the most widely used text-image embeddings. However, these embeddings are vulnerable to subtle misalignment of cross-modal features, resulting in decreased model performance and diminished generalization. To address this problem, we design ModalChorus, an interactive system for visual probing and alignment of multi-modal embeddings. ModalChorus primarily offers a two-stage process: 1) embedding probing with Modal Fusion Map (MFM), a novel parametric dimensionality reduction method that integrates both metric and nonmetric objectives to enhance modality fusion; and 2) embedding alignment that allows users to interactively articulate intentions for both point-set and set-set alignments. Quantitative and qualitative comparisons for CLIP embeddings with existing dimensionality reduction (e.g., t-SNE and MDS) and data fusion (e.g., data context map) methods demonstrate the advantages of MFM in showcasing cross-modal features over common vision-language datasets. Case studies reveal that ModalChorus can facilitate intuitive discovery of misalignment and efficient re-alignment in scenarios ranging from zero-shot classification to cross-modal retrieval and generation.

个人简介:叶依林是香港科技大学跨学科研究学院和香港科技大学(广州)信息枢纽计算媒体与艺术方向的博士生。他的研究方向是人工智能、可视化和人机交互的交叉领域,旨在通过基于高维数据嵌入的可视化和信息检索技术,帮助用户探索、理解并与多模态数据交互。他在TVCG、CSCW和CHI等期刊和会议上发表了多篇论文。

报告人:刘予,爱丁堡大学,研究生
报告题目:"I Came Across a Junk": Understanding Design Flaws of Data Visualization from the Public's Perspective
论文作者:蓝星宇、刘予

发表会议:IEEE VIS 2024

报告摘要:The visualization community has a rich history of reflecting upon flaws of visualization design, and research in this direction has remained lively until now. However, three main gaps still exist. First, most existing work characterizes design flaws from the perspective of researchers rather than the perspective of general users. Second, little work has been done to infer why these design flaws occur. Third, due to problems such as unclear terminology and ambiguous research scope, a better framework that systematically outlines various design flaws and helps distinguish different types of flaws is desired. To address the above gaps, this work investigated visualization design flaws through the lens of the public, constructed a framework to summarize and categorize the identified flaws, and explored why these flaws occur. Specifically, we analyzed 2227 flawed data visualizations collected from an online gallery and derived a design task-associated taxonomy containing 76 specific design flaws. These flaws were further classified into three high-level categories (i.e., misinformation, uninformativeness, unsociableness) and ten subcategories (e.g., inaccuracy, unfairness, ambiguity). Next, we organized five focus groups to explore why these design flaws occur and identified seven causes of the flaws. Finally, we proposed a set of reflections and implications arising from the research.

个人简介:刘予,爱丁堡大学信息学院信息设计研究生,本科毕业于华南师范大学心理学院。主要研究方向为信息设计、数据叙事、用户体验等。个人主页:https://coralineee.notion.site/

报告人:Yutian Zhang,中山大学,硕士生
报告题目:CSLens: Towards Better Deploying Charging Stations via Visual Analytics —— A Coupled Networks Perspective
论文作者:Yutian Zhang, Liwen Xu, Shaocong Tao, Quanxue Guan, Quan Li, Haipeng Zeng

发表会议:IEEE VIS 2024

报告摘要:In recent years, the global adoption of electric vehicles (EVs) has surged, prompting a corresponding rise in the installation of charging stations. This proliferation has underscored the importance of expediting the deployment of charging infrastructure. Both academia and industry have thus devoted to addressing the charging station location problem (CSLP) to streamline this process. However, prevailing algorithms addressing CSLP are hampered by restrictive assumptions and computational overhead, leading to a dearth of comprehensive evaluations in the spatiotemporal dimensions. Consequently, their practical viability is restricted. Moreover, the placement of charging stations exerts a significant impact on both the road network and the power grid, which necessitates the evaluation of the potential post-deployment impacts on these interconnected networks holistically. In this study, we propose CSLens, a visual analytics system designed to inform charging station deployment decisions through the lens of coupled transportation and power networks. CSLens offers multiple visualizations and interactive features, empowering users to delve into the existing charging station layout, explore alternative deployment solutions, and assess the ensuring impact. To validate the efficacy of CSLens, we conducted two case studies and engaged in interviews with domain experts. Through these efforts, we substantiated the usability and practical utility of CSLens in enhancing the decision-making process surrounding charging station deployment. Our findings underscore CSLens’s potential to serve as a valuable asset in navigating the complexities of charging infrastructure planning.

个人简介:Yutian Zhang is currently working towards the M.S. degree in the School of Intelligent Systems Engineering at Sun Yat-sen University, supervised by Prof. Haipeng Zeng. He received a B.S. degree in transportation engineering from Sun Yat-Sen University. His research interests include visual analytics, interpretable machine learning, and transportation big data. Email: zhangyt85@mail2.sysu.edu.cn

报告人:俞凌云,西交利物浦大学,副教授
报告题目:Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed Reality
论文作者:Shuqi He, Haonan Yao, Luyan Jiang, Kaiwen Li, Nan Xiang, Yue Li, Hai-Ning Liang, Lingyun Yu

发表会议:ACM CHI 2024

报告摘要:Tangible interfaces in mixed reality (MR) environments allow for intuitive data interactions. Tangible cubes, with their rich interaction affordances, high maneuverability, and stable structure, are particularly well-suited for exploring multi-dimensional data types. However, the design potential of these cubes is underexplored. This study introduces a design space for tangible cubes in MR, focusing on interaction space, visualization space, sizes, and multiplicity. Using spatio-temporal data, we explored the interaction affordances of these cubes in a workshop (N=24). We identified unique interactions like rotating, tapping, and stacking, which are linked to augmented reality (AR) visualization commands. Integrating user-identified interactions, we created a design space for tangible-cube interactions and visualization. A prototype visualizing global health spending with small cubes was developed and evaluated, supporting both individual and combined cube manipulation. This research enhances our grasp of tangible interaction in MR, offering insights for future design and application in diverse data contexts.

个人简介:俞凌云,西交利物浦大学智能工程学院计算机系副教授,博士生导师,人机交互硕士专业主任。研究方向包括数据可视化,空间交互技术、虚拟/增强/混合现实环境中的可视化与交互。发表可视化和人机交互相关领域论文60余篇,主持国家自然科学青年基金和面上项目各1项。近年来多次在虚拟现实领域顶会IEEE VR和ISMAR中组织沉浸式可视化方向相关的workshop,担任IEEE VIS 2024 workshop chair。个人主页:https://yulingyun.com/

报告人:杨维铠,香港科技大学(广州),助理教授
报告题目:Interactive Reweighting for Mitigating Label Quality Issues
论文作者:Weikai Yang, Yukai Guo, Jing Wu, Zheng Wang, Lan-Zhe Guo, Yu-Feng Li, Shixia Liu

发表期刊:IEEE TVCG

报告摘要:Label quality issues, such as noisy labels and imbalanced class distributions, have negative effects on model performance. Automatic reweighting methods identify problematic samples with label quality issues by recognizing their negative effects on validation samples and assigning lower weights to them. However, these methods fail to achieve satisfactory performance when the validation samples are of low quality. To tackle this, we develop Reweighter, a visual analysis tool for sample reweighting. The reweighting relationships between validation samples and training samples are modeled as a bipartite graph. Based on this graph, a validation sample improvement method is developed to improve the quality of validation samples. Since the automatic improvement may not always be perfect, a co-cluster-based bipartite graph visualization is developed to illustrate the reweighting relationships and support the interactive adjustments to validation samples and reweighting results. The adjustments are converted into the constraints of the validation sample improvement method to further improve validation samples. We demonstrate the effectiveness of Reweighter in improving reweighting results through quantitative evaluation and two case studies.

个人简介:杨维铠,香港科技大学(广州)信息枢纽数据科学与分析学域(DSA)和计算媒体与艺术(CMA)助理教授。2024年于清华大学获得博士学位。主要研究方向为可视分析与机器学习,已在可视分析和机器学习领域的顶级会议和期刊(IEEE VIS/IEEE TVCG/ACM MM)等发表论文10余篇。

报告人:邵汉宁,北京大学,博士生
报告题目:CataAnno: An Ancient Catalog Annotator for Annotation Cleaning by Recommendation
论文作者:Hanning Shao, Xiaoru Yuan

发表期刊:IEEE VIS 2024

报告摘要:Classical bibliography, by researching preserved catalogs from both official archives and personal collections of accumulated books, examines the books throughout history, thereby revealing cultural development across historical periods. In this work, we collaborate with domain experts to accomplish the task of data annotation concerning Chinese ancient catalogs. We introduce the CataAnno system that facilitates users in completing annotations more efficiently through cross-linked views, recommendation methods and convenient annotation interactions. The recommendation method can learn the background knowledge and annotation patterns that experts subconsciously integrate into the data during prior annotation processes. CataAnno searches for the most relevant examples previously annotated and recommends to the user. Meanwhile, the cross-linked views assist users in comprehending the correlations between entries and offer explanations for these recommendations. Evaluation and expert feedback confirm that the CataAnno system, by offering high-quality recommendations and visualizing the relationships between entries, can mitigate the necessity for specialized knowledge during the annotation process. This results in enhanced accuracy and consistency in annotations, thereby enhancing the overall efficiency.

个人简介:邵汉宁,北京大学智能学院博士研究生,导师为袁晓如研究员。研究兴趣主要涉及面向数字人文领域的可视化方法,包括古籍标注、地图可视化等,探索可视化与可视分析技术应用于人文学科,处理领域特有挑战时的新技术与新可能。

报告人:时磊,北京航空航天大学,教授
报告题目:How "Applied" is Fifteen Years of VAST conference?
论文作者:Lei Shi, Lei Xia, Zipeng Liu, Ye Sun, Huijie Guo and Klaus Mueller

发表期刊:IEEE VIS 2023

报告摘要:Visual analytics (VA) science and technology emerge as a promising methodology in visualization and data science in the new century. Application-driven research continues to contribute significantly to the development of VA, as well as in a broader scope of VIS. However, existing studies on the trend and impact of VA/VIS application research stay at a commentary and subjective level, using methods such as panel discussions and expert interviews. On the contrary, this work presents a first study on VA application research using data-driven methodology with cutting-edge machine learning algorithms, achieving both objective and scalable goals. Experiment results demonstrate the validity of our method with high F1 scores up to 0.89 for the inference of VA application papers on both the expert-labeled benchmark dataset and two external validation data sources. Inference results on 15 years of VAST conference papers also narrate interesting patterns in VA application research’s origin, trend, and constitution.

个人简介:时磊,北航计算机学院教授。本硕博毕业于清华大学计算机系。研究方向为数据挖掘、数据可视化、人工智能。曾在KDD、VIS、TVCG、TKDE、ICDE等国际顶尖会议及期刊上发表近百余篇研究论文。四次荣获IEEE可视分析大会挑战赛优胜奖及IBM研究机构可视分析贡献奖。2016年入选中国科学院青年创新促进会,2017年入选中国科学院软件研究所杰出青年人才发展专项计划,2019年入选北航青年拔尖人才支持计划,2022年获中国计算机学会技术发明一等奖(排名第二)。

专题11:前沿论文专题 2


主持人:马昱欣,副教授,南方科技大学

报告人:陈龙飞,上海科技大学,博士生
报告题目:FMLens: Towards Better Scaffolding the Process of Fund Manager Selection in Fund Investments
论文作者:Longfei Chen, Chen Cheng, He Wang, Xiyuan Wang, Yun Tian, Xuanwu Yue, Wong Kam-Kwai, Haipeng Zhang, Suting Hong, and Quan Li

发表期刊:IEEE TVCG

报告摘要:The fund investment industry heavily relies on the expertise of fund managers, who bear the responsibility of managing portfolios on behalf of clients. With their investment knowledge and professional skills, fund managers gain a competitive advantage over the average investor in the market. Consequently, investors prefer entrusting their investments to fund managers rather than directly investing in funds. For these investors, the primary concern is selecting a suitable fund manager. While previous studies have employed quantitative or qualitative methods to analyze various aspects of fund managers, such as performance metrics, personal characteristics, and performance persistence, they often face challenges when dealing with a large candidate space. Moreover, distinguishing whether a fund manager's performance stems from skill or luck poses a challenge, making it difficult to align with investors' preferences in the selection process. To address these challenges, this study characterizes the requirements of investors in selecting suitable fund managers and proposes an interactive visual analytics system called FMLens. This system streamlines the fund manager selection process, allowing investors to efficiently assess and deconstruct fund managers' investment styles and abilities across multiple dimensions. Additionally, the system empowers investors to scrutinize and compare fund managers' performances. The effectiveness of the approach is demonstrated through two case studies and a qualitative user study. Feedback from domain experts indicates that the system excels in analyzing fund managers from diverse perspectives, enhancing the efficiency of fund manager evaluation and selection.

个人简介:陈龙飞 (https://chenlf126.github.io/),就读于上海科技大学,师从李权教授,目前正在ViSeer Lab课题组中攻读博士学位。他的学术研究主要围绕城市数据可视化、可解释人工智能和金融数据可视化等多个领域展开,重点是通过利用可扩展的可视化界面和可解释的方法,实现对数据和模型的有效理解和交流。目前已在TVCG等期刊上以第一作者身份发表多篇工作。

报告人:姜浩然,上海科技大学,硕士生
报告题目:SLInterpreter: An Exploratory and Iterative Human-AI Collaborative System for GNN-based Synthetic Lethal Prediction
论文作者:Haoran Jiang, Shaohan Shi, Shuhao Zhang, Jie Zheng, Quan Li

发表会议:IEEE VIS 2024

报告摘要:Synthetic Lethal (SL) relationships, although rare among the vast array of gene combinations, hold substantial promise for targeted cancer therapy. Despite advancements in AI model accuracy, there remains a persistent need among domain experts for interpretive paths and mechanism explorations that better harmonize with domain-specific knowledge, particularly due to the significant costs involved in experimentation. To address this gap, we propose an iterative Human-AI collaborative framework comprising two key components: 1)Human-Engaged Knowledge Graph Refinement based on Metapath Strategies, which leverages insights from interpretive paths and domain expertise to refine the knowledge graph through metapath strategies with appropriate granularity. 2)Cross-Granularity SL Interpretation Enhancement and Mechanism Analysis, which aids domain experts in organizing and comparing prediction results and interpretive paths across different granularities, thereby uncovering new SL relationships, enhancing result interpretation, and elucidating potential mechanisms inferred by Graph Neural Network (GNN) models. These components cyclically optimize model predictions and mechanism explorations, thereby enhancing expert involvement and intervention to build trust. This framework, facilitated by SLInterpreter, ensures that newly generated interpretive paths increasingly align with domain knowledge and adhere more closely to real-world biological principles through iterative Human-AI collaboration. Subsequently, we evaluate the efficacy of the framework through a case study and expert interviews.

个人简介:姜浩然,上海科技大学信息学院ViSeer Lab一年级硕士研究生,导师为李权教授。目前的研究方向主要关注人与AI协同过程中的用户参与及认知去偏,包括生物医学模型的可解释性增强及人类决策活动中的认知偏差消除。

报告人:高琳,复旦大学,硕士生
报告题目:Fine-Tuned Large Language Model for Visualization System: A Study on Self-Regulated Learning in Education
论文作者:Lin Gao, Jing Lu, Zekai Shao, Ziyue Lin, Shengbin Yue, Chiokit Ieong, Yi Sun, Rory James Zaunar, Zhongyu Wei and Siming Chen

发表会议:IEEE VIS 2024

报告摘要:We propose a framework and workflow to guide the application of fine-tuned LLMs in enhancing visual interactions for domain-specific tasks. Applied to education, this framework introduces Tailor-Mind, an interactive visualization system for self-regulated learning. Insights from a preliminary study help identify learning tasks and fine-tuning objectives to guide visualization design and data construction. By aligning visualization and interactions with fine-tuned LLMs, Tailor-Mind acts as a personalized tutor, offering interactive recommendations to aid beginners in achieving their learning goals. Model performance evaluations and user studies confirm that Tailor-Mind significantly enhances the self-regulated learning experience, validating the proposed framework.

个人简介:高琳,复旦大学大数据学院硕士研究生一年级。研究方向为人机交互、可视分析以及智能教育。

报告人:卜佳,华东师范大学,硕士生
报告题目:DoodleTunes: Interactive Visual Analysis of Music-Inspired Children Doodles with Automated Feature Annotation
论文作者:Shuqi Liu, Jia Bu, Huayuan Ye, Juntong Chen, Shiqi Jiang, Mingtian Tao, Liping Guo, Changbo Wang, Chenhui Li

发表期刊:ACM CHI 2024

报告摘要:Music and visual arts are essential in children’s arts education, and their integration has garnered significant attention. Existing data analysis methods for exploring audio-visual correlations are limited. Yet, relevant research is necessary for innovating and promoting arts integration courses. In our work, we collected substantial volumes of music-inspired doodles created by children and interviewed education experts to comprehend the challenges they encountered in the relevant analysis. Based on the insights we obtained, we designed and constructed an interactive visualization system DoodleTunes. DoodleTunes integrates deep learning-driven methods for automatically annotating several types of data features. The visual designs of the system are based on a four-level analysis structure to construct a progressive workflow, facilitating data exploration and insight discovery between doodle images and corresponding music pieces. We evaluated the accuracy of our feature prediction results and collected usage feedback on DoodleTunes from five domain experts.

个人简介:卜佳,华东师范大学硕士研究生,导师为李晨辉副教授。她的研究兴趣包括可视化叙事、金融数据可视化、人机交互等。她曾在 CHI 会议上发表学术论文 1 篇,曾获华东师范大学一等奖学金、优秀毕业论文等荣誉。

报告人:陈俊潼,华东师范大学,硕士生
报告题目:SenseMap: Urban Performance Visualization and Analytics via Semantic Textual Similarity
论文作者:Juntong Chen, Qiaoyun Huang, Changbo Wang, and Chenhui Li

发表期刊:IEEE TVCG

报告摘要:As urban populations grow, effectively accessing urban performance measures such as livability and comfort becomes increasingly important due to their significant socioeconomic impacts. While Point of Interest (POI) data has been utilized for various applications in location-based services, its potential for urban performance analytics remains unexplored. In this paper, we present SenseMap, a novel approach for analyzing urban performance by leveraging POI data as a semantic representation of urban functions. We quantify the contribution of POIs to different urban performance measures by calculating semantic textual similarities on our constructed corpus. We propose Semantic-adaptive Kernel Density Estimation which takes into account POIs' influential areas across different Traffic Analysis Zones and semantic contributions to generate semantic density maps for measures. We design and implement a feature-rich, real-time visual analytics system for users to explore the urban performance of their surroundings. Evaluations with human judgment and reference data demonstrate the feasibility and validity of our method. Usage scenarios and user studies demonstrate the capability, usability and explainability of our system.

个人简介:陈俊潼,华东师范大学硕士生,导师为王长波教授和李晨辉副教授。他的研究兴趣包括时空数据分析、大规模数据可视化、人机交互等。他于 2022 年获华东师范大学学士学位,硕士期间累计在 TVCG、CHI 等期刊/会议上发表论文 5 篇。他曾获 Apple WWDC Scholarship、华东师范大学优秀毕业生等荣誉。

报告人:刘冬煜(Dongyu Liu),University of California, Davis,Assistant Professor
报告题目:Towards Dataset-scale and Feature-oriented Evaluation of Text Summarization in Large Language Model Prompts
论文作者:Sam Yu-Te Lee, Aryaman Bahukhandi, Dongyu Liu, Kwan-Liu Ma

发表会议:IEEE VIS 2024

报告摘要:Recent advancements in Large Language Models (LLMs) and Prompt Engineering have made chatbot customization more accessible, significantly reducing barriers to tasks that previously required programming skills. However, prompt evaluation, especially at the dataset scale, remains complex due to the need to assess prompts across thousands of test instances within a dataset. Our study, based on a comprehensive literature review and pilot study, summarized five critical challenges in prompt evaluation. In response, we introduce a feature-oriented workflow for systematic prompt evaluation. In the context of text summarization, our workflow advocates evaluation with summary characteristics (feature metrics) such as complexity, formality, or naturalness, instead of using traditional quality metrics like ROUGE. This design choice enables a more user-friendly evaluation of prompts, as it guides users in sorting through the ambiguity inherent in natural language. To support this workflow, we introduce Awesum, a visual analytics system that facilitates identifying optimal prompt refinements for text summarization through interactive visualizations, featuring a novel Prompt Comparator design that employs a BubbleSet-inspired design enhanced by dimensionality reduction techniques. We evaluate the effectiveness and general applicability of the system with practitioners from various domains and found that (1) our design helps overcome the learning curve for non-technical people to conduct a systematic evaluation of summarization prompts, and (2) our feature-oriented workflow has the potential to generalize to other NLG and image-generation tasks. For future works, we advocate moving towards feature-oriented evaluation of LLM prompts and discuss unsolved challenges in terms of human-agent interaction.

个人简介:Dongyu Liu is an Assistant Professor in the Department of Computer Science at the University of California, Davis, where he directs the Visualization and Intelligence Augmentation (VIA) Lab. His research focuses on developing visualization-empowered human-AI teaming systems for decision-making, with an emphasis on sustainability and healthcare. He was a Postdoctoral Associate at MIT and received his Ph.D. from the Hong Kong University of Science and Technology. His research has been published in leading computer science venues such as TVCG, VIS, CHI, CSCW, SIGMOD, earning awards including a best paper award at VIS’18 and an honorable mention at VIS’21.

报告人:黄泽宇,香港科技大学,博士生
报告题目:Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic Chills
论文作者:Zeyu Huang, Xinyi Cao, Yuanhao Zhang, and Xiaojuan Ma

发表会议:ACM CHI 2024

报告摘要:On online video platforms, viewers often lack a channel to sense others’ and express their affective state on the fly compared to co-located group-viewing. This study explored the design of complementary affective communication specifically for effortless, spontaneous sharing of frissons during video watching. Also known as aesthetic chills, frissons are instant psycho-physiological reactions like goosebumps and shivers to arousing stimuli. We proposed an approach that unobtrusively detects viewers’ frissons using skin electrodermal activity sensors and presents the aggregated data alongside online videos. Following a design process of brainstorming, focus group interview (N=7), and design iterations, we proposed three different designs to encode viewers’ frisson experiences, namely, ambient light, icon, and vibration. A mixed-methods within-subject study (N=48) suggested that our approach offers a non-intrusive and efficient way to share viewers’ frisson moments, increases the social presence of others as if watching together, and can create affective contagion among viewers.

个人简介:黄泽宇,香港科技大学人机交互方向博士生。他的主要研究兴趣是情感与社交导向的交互设计,包括探究情感信息的呈现与分享,拓展感性表达的可能性,建立社群中的人文联结,以及促进情绪能力的自我觉察与提升。

报告人:陈子昕,香港科技大学,博士生
报告题目:StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions
论文作者:Zixin Chen, Jiachen Wang, Meng Xia, Kento Shigyo, Dingdong Liu, Rong Zhang, Huamin Qu

发表会议:IEEE VIS 2024

报告摘要:The integration of Large Language Models (LLMs), especially ChatGPT, into education is poised to revolutionize students’ learning experiences by introducing innovative conversational learning methodologies. To empower students to fully leverage the capabilities of ChatGPT in educational scenarios, understanding students’ interaction patterns with ChatGPT is crucial for instructors. However, this endeavor is challenging due to the absence of datasets focused on student-ChatGPT conversations and the complexities in identifying and analyzing the evolutional interaction patterns within conversations. To address these challenges, we collected conversational data from 48 students interacting with ChatGPT in a master’s level data visualization course over one semester. We then developed a coding scheme, grounded in the literature on cognitive levels and thematic analysis, to categorize students’ interaction patterns with ChatGPT. Furthermore, we present a visual analytics system, StuGPTViz, that tracks and compares temporal patterns in student prompts and the quality of ChatGPT’s responses at multiple scales, revealing significant pedagogical insights for instructors. We validated the system’s effectiveness through expert interviews with six data visualization instructors and three case studies. The results confirmed StuGPTViz’s capacity to enhance educators’ insights into the pedagogical value of ChatGPT. We also discussed the potential research opportunities of applying visual analytics in education and developing AI-driven personalized learning solutions.

个人简介:陈子昕,香港科技大学博士二年级学生,本科毕业于香港科技大学数据科学专业。主要研究方向是可视分析与人工智能在教育领域的应用。在IEEE VIS、ACM CSCW、ACM DIS等A类国际会议上均有论文发表。个人主页: https://cinderd.github.io/

报告人:郭宇涵,北京大学,博士生
报告题目:PrompTHis: Visualizing the Process and Influence of Prompt Editing during Text-to-Image Creation
论文作者:Yuhan Guo, Hanning Shao, Can Liu, Kai Xu, Xiaoru Yuan

发表期刊:IEEE TVCG

报告摘要:Generative text-to-image models, which allow users to create appealing images through a text prompt, have seen a dramatic increase in popularity in recent years. However, most users have a limited understanding of how such models work and often rely on trial and error strategies to achieve satisfactory results. The prompt history contains a wealth of information that could provide users with insights into what has been explored and how the prompt changes impact the output image, yet little research attention has been paid to the visual analysis of such process to support users. We propose the Image Variant Graph, a novel visual representation designed to support comparing prompt-image pairs and exploring the editing history. The Image Variant Graph models prompt differences as edges between corresponding images and presents the distances between images through projection. Based on the graph, we developed the PrompTHis system through co-design with artists. Based on the review and analysis of the prompting history, users can better understand the impact of prompt changes and have a more effective control of image generation. A quantitative user study and qualitative interviews demonstrate that PrompTHis can help users review the prompt history, make sense of the model, and plan their creative process.

个人简介:郭宇涵,北京大学智能学院一年级博士研究生,导师为袁晓如研究员。研究兴趣涉及文本可视化、面向人文领域的可视化,发表数篇论文于 IEEE TVCG, IEEE VIS 等期刊会议。


<!–End of footer Section–> <!– START SCROLL TO TOP –>