AI research tools can distort findings and misattribute authors, CQUniversity researcher warns
A CQUniversity researcher is urging academics to verify AI-generated research summaries after tests found missing author attribution, inaccurate findings and incomplete references that made it difficult to trace evidence back to original studies.
As generative AI becomes more widely used in research, CQUniversity's Dr Mahmoud El Khodr is warning that researchers need to be alert not only to what they share with AI tools, but also to how those tools interpret and represent other people's work.
Dr El Khodr, Senior Lecturer in CQUniversity’s School of Engineering and Technology, has discussed these risks in an open-access commentary and conducted illustrative tests of AI research tools, including a Gemini Deep Research test.
In one test, Google's Gemini Deep Research reviewed more than 100 source entries while preparing a literature review on generative AI in university networking courses. However, the exported report contained only seven reference entries, including two links to the same paper, making it difficult to trace some findings back to their original sources.
Checking the report against the original papers also revealed multiple discrepancies.
One paper co-authored by Dr El Khodr was attributed to the publisher rather than its four authors. The AI-generated summary also changed the context of the study, presenting findings from design-based student tasks as evidence from networking tutorials.
It further misstated a finding about students using ChatGPT. The original paper found that students who used ChatGPT achieved higher scores for functionality and user-flow tasks, with average scores increasing from 1.8 to 3.2 out of five. However, the AI-generated summary stated that students performed similarly to those who did not use ChatGPT.
In another example, a paper proposing an AI troubleshooting tutor was described as having validated the AI system, even though the researchers had tested the approach using a human mentor and stated that implementing and testing the AI system remained future work.
Dr El Khodr said these examples highlighted the importance of checking AI-generated summaries against original research rather than assuming polished, authoritative-sounding output was accurate.
"When AI summaries omit authors, alter context or misrepresent findings, readers can lose sight of who conducted the research and what the evidence actually shows," Dr El Khodr said.
"These tools can save time, but they should not replace critical evaluation. Researchers still need to verify sources, check citations and confirm findings against the original work."
The findings come amid rapid growth in the use of generative AI across Australia's research sector.
Science & Technology Australia's National AI in Research Survey, released in October, found that nearly three-quarters of respondents reported using AI, while 54 per cent considered their organisation's AI training and skills-building opportunities inadequate.
Dr El Khodr said researchers should routinely verify AI-generated citations against original papers, check that authors and findings are accurately represented, and avoid uploading unpublished research, confidential manuscripts or sensitive information without appropriate safeguards.
He also recommended checking how AI services handle submitted information, including whether inputs may be retained or used for model training.
Dr El Khodr and Professor Ergun Gide have developed SAGE-R, a research-informed framework and practical resources for researchers.
It is designed to support responsible AI use while preserving independent judgement and research ownership. SAGE central principle is that AI may assist the research, while researchers remain responsible for their reasoning, decisions and findings.
"Generative AI can be a valuable research assistant, but researchers remain responsible for verifying evidence, acknowledging authors and maintaining research integrity," Dr El Khodr said.
