WHAT: TWed Talk: "ChatBS-NexGen: A Platform for Automated KG-based LLM Fact Checking"
WHEN: 4p, Weds, 23 Apr
WHERE: Winslow 1140
VIDEO: https://youtu.be/DiXwEkfUw0o
EVENT PAGE: https://bit.ly/44t21MX
TWC grad students Nipun Deelaka and Devin De Silva discuss and demonstrate ChatBS-NexGen, their impressive evolution of TWC's ChatBS LLM experimentation sandbox.
DESCRIPTION: ChatBS-NextGen is a user-friendly application designed to assess the factual accuracy of AI-generated responses using semantic knowledge graphs. By extracting entities from LLM outputs and mapping them to authoritative knowledge sources, it streamlines fact verification, enabling seamless factoid extraction through simple text input.
The platform supports multiple LLMs, multi-query workflows, and iterative experiment pipelines, generating structured, downloadable reports for each run. This makes it an ideal solution for organizations, researchers, and individuals who require reliable verification of AI-generated content.
ChatBS-NextGen automates the complex yet repetitive process of answer generation across multiple LLMs using custom datasets. To ensure answer consistency, it offers a unique feature that allows the same query to be run multiple times. The results are compiled into a structured yet easily interpretable report, detailing both the generated responses and experiment logs.
For factual verification, the application follows a structured factoid extraction process. It first identifies the information that needs verification, queries a designated authoritative knowledge graph (KG) for validated facts, and then presents the user with a side-by-side comparison of extracted facts from both the AI-generated response and the KG.