Schedule

Mark Riedl

Reasoning, Alignment, and Explainability

Abstract: In this talk, I will present a vision for human-centered artificial intelligence that emphasizes two (non-exhaustive) properties: AI that understands humans, and AI that helps people understand AI systems. Alignment is often cast as the process of making AI systems helpful, harmless, and honest; we can also think of these principles as AI system needing to understand what behaviors human users view as helpful and harmful. Explainable AI (XAI) is one path toward AI that helps its users understand its behaviors. We explore how reasoning can act as a framework for unifying perspectives on alignment and explainability. I will present work from my lab on alignment, explainability, and reasoning as partial glimpses at what might be possible. 

Isabela Albuquerque

From Pixels to Actions: A Roadmap for Evaluating Safety in Multimodal Agents

Abstract: As generative models evolve into autonomous agents, evaluation must shift from static "Pixel Parity" to dynamic "Procedural Parity". While traditional safety frameworks audit a single output, agentic workflows introduce sequential risks like behavioral drift over long horizons. This talk outlines a roadmap toward a formal Evaluation Science, focusing on critical safety aspects like fairness and alignment in multimodal agents. We bridge foundational statistical baselines used to measure multimodal alignment and distributional skew with Equality of Service, assessing whether quality remain uniform across demographic intersections. By investigating how agents resolve query ambiguity, we highlight the importance of evaluating the equity of the decision-making process itself. Finally, we describe a statistical method that unifies human and automated signals, providing a scalable path to help realize the ultimate goal of treating responsible AI as a measurable, predictable property of autonomous systems. 

Furong Huang

Rethinking Test-Time Thinking: From Token-Level Rewards to Robust Generative Agents

Abstract: We present a unified perspective on test-time thinking as a lens for improving generative AI agents through finer-grained reward modeling, data-centric reasoning, and robust alignment. Beginning with GenARM, we introduce an inductive bias for denser, token-level reward modeling that guides generation during decoding, enabling token-level alignment without retraining. While GenARM targets reward design, ThinkLite-VL focuses on the data side of reasoning. It proposes a self-improvement framework that selects the most informative samples via MCTS-guided search, yielding stronger visual reasoning with fewer labels. Taking this a step further, MORSE-500 moves beyond selection to creation: it programmatically generates targeted, controllable multimodal data to systematically probe and stress-test models’ reasoning abilities. We then interrogate a central assumption in inference-time alignment: Does Thinking More Always Help? Our findings reveal that increased reasoning steps can degrade performance--not due to better or worse reasoning per se, but due to rising variance in outputs, challenging the naive scaling paradigm. Finally, AegisLLM applies test-time thinking in the service of security, using an agentic, multi-perspective framework to defend against jailbreaks, prompt injections, and unlearning attacks--all at inference time. Together, these works chart a path toward generative agents that are not only more capable, but more data-efficient, introspective, and robust in real-world deployment.

Seraphina Goldfarb-Tarrant

Title: Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations

Edie Pearman

Title: Mechanics of Bias and Reasoning: Interpreting the Impact of Chain-of-Thought Prompting on Gender Bias in LLMs

Canyu Chen

Title: Fairness Failure Modes of Multimodal LLMs