Reddit's AI tool faces backlash after spreading dangerous medical misinformation
What's the story
Reddit's latest artificial intelligence (AI) tool, Reddit Answers, is facing severe backlash from users and moderators. The criticism stems from its distribution of dangerous medical misinformation, and moderators can't disable it. The issue was first highlighted in the r/familymedicine subreddit, where a moderator found that the AI-powered feature was automatically responding to health-related posts with potentially harmful information.
Misinformation examples
AI suggested stopping prescribed medications, taking illegal drugs
A healthcare worker tested Reddit Answers and found some shocking examples of its dangerous suggestions. In one instance, the tool suggested that people stop their prescribed medications and take high doses of kratom instead, after a post about chronic pain. Kratom is illegal in some US states and not approved for medical use. In another case, the AI linked to a post encouraging heroin use as a way to manage chronic pain.
Health hazards
AI responses could be mistaken for subreddit endorsements
The suggestions from Reddit Answers are not just medically inaccurate but also pose a serious threat to health. Heroin, a Schedule I drug in the US, has no accepted medical uses and is highly addictive. A moderator warned that these AI-generated answers could easily be mistaken for endorsements by the subreddit when it clearly doesn't support using heroin for chronic pain treatment.
User concerns
Moderators demand disabling AI in medical, mental health communities
In light of these issues, Reddit moderators have demanded that the platform either disable Reddit Answers for medical and mental health communities or give them an option to opt out. "If this continues, people will be harmed," one post warned. A Reddit admin has acknowledged the problem and said they have "made some changes to where Answers appears" and will continue refining it based on user feedback.
Platform's stance
r/familymedicine subreddit now has a warning pinned at top
In light of the controversy, the r/familymedicine subreddit now has a warning pinned at the top. It reads: "We do NOT and CANNOT endorse Reddit Answers at this time and urge every user of this sub to disregard anything it says." The incident highlights concerns over the unchecked use of AI in sensitive areas, such as health advice, where a single incorrect answer can have real-world consequences.