AI vs. Medical Students: Divergent Psychedelic Harm Ratings in Clinical Education

AI and Medical Students Differ on Psychedelic Drug Harm Assessments

A 2026 comparative analysis published via OpenAlex (W7212365790) reveals that while final-year medical students and ChatGPT agree on the most harmful substances, their perceptions diverge sharply for less harmful drugs, notably psychedelics. Both groups rated heroin, crack cocaine, and fentanyl as the most dangerous substances to users and society. However, students identified snus (smokeless tobacco), laughing gas (nitrous oxide), and cannabis as least harmful, whereas ChatGPT consistently rated magic mushrooms (psilocybin), DMT/ayahuasca, and LSD as posing the lowest risk. Notably, ChatGPT’s ratings for psychedelics were more closely aligned with those of established expert panels than with student perceptions.

Mechanisms Behind Divergence: Social, Cultural, and Educational Influences

The divergence in harm ratings between AI and medical students reflects the complex interplay of social, cultural, and educational factors that shape perceptions of drug risk. Medical students’ harm assessments are influenced by their clinical exposure, local drug use patterns, and the content of their formal training. The study found that students rated their own knowledge of drug harm at 5.8 out of 10 and their education on the subject at 5.3, indicating moderate confidence and possible curricular gaps. In contrast, ChatGPT self-rated its knowledge at 8.3, drawing from a broad, literature-based evidence base that mirrors expert consensus but may lack sensitivity to local context or recent clinical trends.

One underappreciated mechanism is the tendency for AI models to reflect the prevailing scientific literature, which itself may be shaped by regulatory priorities, publication bias, and the slow pace of updating systematic reviews. As a result, ChatGPT’s alignment with expert panels may not always equate to real-time clinical relevance, especially in rapidly evolving or contested domains like psychedelic medicine.

Policy and Educational Implications for AI in Drug Harm Assessment

The findings have significant implications for educators, clinicians, and policymakers considering the integration of AI tools into substance harm assessment and medical training. While AI models like ChatGPT can provide evidence-based perspectives that align with expert panels, their outputs may diverge from the lived experience and local knowledge of clinicians-in-training. This raises questions about the reliability and appropriateness of using AI-generated harm assessments as a basis for clinical decision-making or public health policy, particularly for substances where the evidence base is sparse, contested, or rapidly changing.

For medical educators, the study highlights a need to strengthen curricula around critical appraisal of both traditional and AI-generated evidence. Students’ relatively low self-assessed confidence in drug harm knowledge suggests that current training may not adequately prepare them to interpret or challenge AI outputs, especially in nuanced cases. Policymakers should be cautious about over-relying on AI-generated evidence for regulatory or scheduling decisions, particularly for emerging substances like psychedelics where scientific consensus is still forming.

Risks, Unknowns, and Real-World Failure Modes

AI-generated drug harm assessments carry risks of both overconfidence and misalignment with frontline clinical realities. The study found that ChatGPT’s harm ratings were less variable and more confident than those of students, potentially masking underlying uncertainties or gaps in the evidence. In practice, this could lead to inappropriate risk stratification if AI outputs are accepted uncritically. Furthermore, the lack of transparency in how large language models weigh conflicting data points makes it difficult for clinicians or policymakers to audit or challenge specific harm ratings, a critical limitation in contested domains like psychedelic medicine.

A concrete failure mode, not widely discussed in competing literature, is the potential for AI-generated harm assessments to be used in legal or regulatory settings without adequate scrutiny. For instance, if a court or regulator were to cite ChatGPT’s low harm rating for psilocybin in a policy decision, this could outpace both clinical practice and societal readiness, leading to unintended consequences. The study underscores the importance of maintaining human oversight and critical appraisal when deploying AI in high-stakes contexts.

Looking Forward: Integrating AI Responsibly in Psychedelic Harm Evaluation

As AI systems become more integrated into medical education and policy, stakeholders must balance the strengths of evidence-based outputs with the limitations of context-blind algorithms. Future research should focus on developing transparent, auditable AI models and on training clinicians to critically engage with both AI and traditional sources of evidence. For the psychedelic field in particular, ongoing dialogue between clinicians, educators, regulators, and AI developers will be essential to ensure that harm assessments reflect both scientific consensus and real-world complexity.

How we research: This article was written and reviewed by Dr. Jamie L. Carter, MD, MSc (psychiatry, clinical educator). Reviewed on 2026-09-15. Sources include the original OpenAlex publication and direct review of cited expert panel methodologies.