The ASGAARD lab develops AI-powered techniques for detecting and understanding software defects, with a focus on visual quality assurance in interactive systems such as video games, web applications, and AI-driven software.
Current themes:
Intelligent Quality Assurance for Video Games (2021 – present)
As video games grow in complexity, traditional testing methods often fail to capture the dynamic and visual nature of gameplay. This project investigates how to automate the detection of glitches and bugs by leveraging Large Multimodal Models (LMMs) and Vision-Language Models (VLMs). We develop benchmarks like VideoGameQA-Bench and GlitchBench to evaluate model performance on real-world QA tasks, including visual regression testing, “needle-in-a-haystack” bug detection, and automated bug report generation from gameplay videos. We collaborate(d) with several industrial partners on this research topic, including (in alphabetical order) EA, Prodigy Education, Sony Interactive Entertainment and Ubisoft La Forge.
Intelligent Quality Assurance for Web Applications and Browsers (2021 – present)
Modern web applications rely on complex rendering engines and dynamic front-end frameworks, making visual inconsistencies difficult to detect automatically. This project investigates how AI and computer vision techniques can be used to detect graphical inconsistencies across browsers and devices. Our work studies cross-browser rendering issues, visual layout inconsistencies, and GUI defects, and develops automated approaches to detect these problems at scale. We collaborate with industry partners such as Mozilla and other organizations interested in improving the reliability of web interfaces.
Software Engineering for AI & Foundation Models (SE4AI / FM4SE) (2023 – present)
This research explores the bidirectional relationship between software engineering and AI. We investigate “FM4SE” (using Foundation Models to automate SE tasks like code understanding) and “SE4FM” (applying engineering rigor to the lifecycle of AI models). Our work includes analyzing practitioner perspectives and developing techniques to improve model documentation.
Earlier research:
Mining Gameplay Videos for Bug Discovery (2018 – 2022)
Gameplay videos shared on platforms such as YouTube contain a large amount of information about real-world software bugs in video games. This project investigates how these videos can be used as software engineering artifacts for bug discovery and debugging. We developed techniques to automatically identify videos that showcase game bugs, enabling developers to mine large video repositories for evidence of glitches and unexpected behavior.
Empirical Performance Engineering & Microbenchmarking (2015 – 2025)
This research investigates how to detect and diagnose performance regressions in large software systems. By analyzing performance metrics, system logs, and architectural signals, we develop techniques that help developers identify performance problems early and understand their root causes.
Improving Post-Release Support (2017 – 2023)
Post-release support is essential for maintaining user satisfaction and loyalty. We study how developers handle user feedback, bug reports, and online reviews after a software release. For example, we investigated how developer responses to app store reviews influence user ratings, and developed techniques to automatically identify videos on platforms such as YouTube that demonstrate software bugs.
Studying the Impact of Release Strategies (2015 – 2019)
Modern development practices such as DevOps enable rapid and frequent software releases. In this project, we study how release strategies affect user-perceived software quality. For example, we examined urgent game updates and found that frequent release strategies are associated with a higher number of zero-day urgent patches. We also analyzed why some mobile app releases are perceived negatively by users.
