2023 ā Present
Team Members: Yu Fu, Shunan Guo, Jane Hoffswell, Victor S. Bursztyn, Ryan Rossi, John Stasko
Aletheia is an automated fact-checking prototype that facilitates data claims verification and improves how data evidence is communicated to readers.
Fact-checking quantitative claims often involves retrieving and analyzing structured data evidenceāa process that can be tedious and intractable when done manually.
Aletheia leverages a pre-trained large language model (LLM) to parse claim semantics and identify relevant data evidence from tables and datasets. It then generates two complementary forms of evidence presentationādata tables and visualizationsātailored to different types of data facts. Interactive views enable users to explore these representations and assess the credibility of claims in realistic scenarios.
A user study with 20 participants and a curated dataset of 400 claims evaluated the system's NLP pipeline and evidence presentation modalities. The results reveal the feasibility and limitations of using LLMs for data fact-checking, and provide design recommendations for improving evidence communication and user trust.