Over a month-long study, participants’ unassisted accuracy dropped by 15.3% after using AI help, despite immediate gains of 21%. The findings suggest AI assistance may actually impair users’ long-term ability to detect fake news on their own, rather than improving it.
Misinformation, AI Intervention, and the Learning Gap
The rapid spread of AI-generated misinformation, such as the 2023 fake Pentagon explosion report, underscores the urgent need for scalable detection interventions. While AI assistants like chat generative pre-trained transformer (ChatGPT) are increasingly used to evaluate online content, traditional fact-checking and machine-learning methods struggle with the scale and sophistication of modern misinformation.
Psychological barriers, such as the continued influence effect, make post hoc corrections insufficient. Recent work shows AI-driven persuasive dialogues can reduce conspiracy beliefs, yet it remains unclear whether these interactions build lasting independent skills or foster overreliance on AI systems.
Prior interventions, such as fact-checks, prebunking, and accuracy nudges, offer one-shot corrections without engaging users' own reasoning. This paper addressed this critical gap by longitudinally measuring both AI-assisted and unassisted misinformation detection, examining whether persuasive AI dialogues cultivate genuine discernment skills or merely create dependency.
System, Data, and Study Design
The researchers designed a web-based AI chatbot to investigate whether persuasive dialogues can build lasting misinformation detection skills. The system architecture integrates a React front-end for presenting news headline-image pairs and collecting participant ratings, OpenAI's GPT-4o for generating persuasive dialogue, and Google Custom Search application programming interface (API) for real-time factual retrieval.
Following established persuasive dialogue approaches, the chatbot acknowledges participants' reasoning, presents relevant evidence, and guides users toward accurate assessments of authenticity across up to 9 conversational rounds per item.
For data curation, 55 news items were randomly selected from the MiRAGeNews dataset, comprising both real and AI-generated headline-image pairs with ground truth labels. After removing six items that the system misclassified, 49 items were distributed across three study phases, spanning diverse topics such as elections, protests, and health matters.
A systematic evaluation of prompt variations was conducted before deployment, testing combinations of web search, image analysis, and persuasive strategies. The combined prompt integrating forensic expertise with persuasion, alongside Google search and image analysis, achieved 100% accuracy and 0% rejection rate, leading to its selection.
The study followed a longitudinal design with 67 participants completing sessions at weeks 0, 2, and 4. Each session comprised three phases, namely, unaided pre-assessment, AI-assisted evaluation of the same items, and independent assessment of novel items.
Accuracy served as the primary dependent variable, analyzed using linear probability models with participant fixed effects. The analysis specifically contrasted within-session AI gains against unassisted learning on new items, examined decay trends over four weeks, and applied large language model (LLM)-as-a-judge methodology to classify conversational strategies into 21 categories to identify which dialogue behaviours promoted skill development versus dependency.
The Dependency Paradox
While AI assistance produced substantial immediate accuracy gains averaging +21.3 percentage points across all weeks, these improvements did not translate into lasting skill development. Unassisted accuracy after AI interaction declined significantly over time, dropping 15.3 percentage points by Week 4 compared to Week 0, with this deterioration driven almost entirely by a reduced ability to detect fake content.
Baseline accuracy before AI interactions showed only modest, non-significant declines, suggesting the sharper post-AI drops stemmed from short-term dependency or cognitive fatigue effects within sessions. Guided questioning and deep probing were negatively associated with performance during AI interaction but positively predicted independent detection afterwards, functioning as pedagogical scaffolding.
Conversely, confidence calibration and devil's advocate roleplay consistently undermined unassisted performance, fostering dependence. Longitudinal conversation analysis revealed increasing human agreement with AI over time alongside persistently low independent thinking, reinforcing the overreliance interpretation.
Download the PDF of this page here
Qualitative analysis of participant trajectories identified five distinct patterns: progressive learners growing to trust AI; consistently collaborative learners developing dependence on AI validation; growing skeptics reasserting independence; persistent self-reliant learners treating AI as a secondary tool; and dependency developers shifting toward passive acceptance of AI guidance, explicitly acknowledging cognitive offloading.
Beyond Immediate Accuracy
This study discovered a paradox at the core of AI-assisted misinformation detection. While persuasive AI systems deliver substantial immediate accuracy gains of 21%, they simultaneously erode long-term independent judgment, with unassisted performance declining by 15.3% after four weeks.
Critically, this deterioration was driven almost entirely by a reduced ability to detect fake content, while recognition of real news remained stable. These findings challenge existing assumptions about AI’s educational potential, demonstrating that current approaches prioritise belief correction over genuine skill development, inadvertently fostering cognitive dependency.
The qualitative trajectory analysis further revealed that many participants shifted toward passive acceptance of AI guidance, explicitly acknowledging their own cognitive offloading. As AI systems become increasingly sophisticated and persuasive, ensuring they build rather than undermine critical thinking capabilities is essential for preserving public resilience against misinformation.
Future systems must prioritise pedagogical strategies, such as guided questioning, that scaffold independent reasoning rather than replace it.
Journal Reference
Rani, A., Danry, V., Liang, P. P., Lippman, A., & Maes, P. (2026). Dialogues with AI Reduce Beliefs in Misinformation but Build No Lasting Discernment Skills. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 1–26. DOI:10.1145/3772318.3790656, https://dl.acm.org/doi/10.1145/3772318.3790656
Disclaimer: The views expressed here are those of the author expressed in their private capacity and do not necessarily represent the views of AZoM.com Limited T/A AZoNetwork the owner and operator of this website. This disclaimer forms part of the Terms and conditions of use of this website.