WHAT: TWed Talk: "Detecting Ambiguity in Question Answering over Financial Documents using LLMs"
WHO: Clare Arrington & Kelsey Rook
WHEN: 4p, Weds, 28 Jan 2026
WHERE: Winslow 1140
WEBEX: https://rensselaer.webex.com/meet/erickj4
VIDEO: TBD
EVENT PAGE: https://bit.ly/4raNtK6
Please join us on Weds, 28 Jan as Clare Arrington and Kelsey Rook lead us in a discussion of their work investigating how LLMs handle ambiguity, particularly in the financial document QA setting over tabular data. Pizza arrives approx. 3:30p, the talk begins at 4p.
DESCRIPTION: LMs have demonstrated impressive performance in question answering (QA) over general knowledge, due in part to training on progressively larger amounts of data. However, queries can contain ambiguous language making it difficult to understand and answer correctly. Users may leave out important context or use domain-specific jargon, an LLM may need to rely on assumptions if it lacks necessary information, and an answer may be stated confidently by an LLM rather than asking clarifying questions. In this study, we investigate how LLMs handle ambiguity, particularly in the financial document QA setting over tabular data. We analyze how existing financial QA datasets align with taxonomies of ambiguity created for common sense and trivia QA, and evaluate open-source and frontier models on their ability to detect various categories of ambiguities.
BIOGRAPHIES:
Clare Arrington is a PhD candidate in Computer Science, advised by Dr. Sibel Adalı. Her research focus is in semantics, computational linguistics and social science. Clare predominantly studies word senses, including sense modeling, semantic shift detection and community analysis via sense variation. She has also worked on bias detection within language models, analysis of consumer reasoning when identifying fake news and financial document understanding with LLMs.
Kelsey Rook is a PhD student in Computer Science at RPI, and is advised by Dr. Deborah L. McGuinness. Kelsey's research interests include conversational AI, and knowledge graph and ontology construction. Currently, she is studying how semantic ontologies can improve how LLMs understand dialogue. Additionally, she has worked with ontologies for clinical psychiatry, knowledge-graph grounded dialogue systems, and financial document understanding with LLMs.