AI Chatbots Are Becoming Mandated Reporters. Without the Manual.

Eight months before the Tumbler Ridge school shooting, OpenAI flagged the account of the person who would later kill eight people. An automated review system caught gun violence scenarios in her ChatGPT chats, according to The Wall Street Journal. About a dozen employees debated whether to tell Canadian police. Some of them thought the conversations showed real-world risk. Management ultimately decided her conversations didn’t cross the company’s internal threshold for calling the police. Although the account was banned, nobody got called. On February 10, 2026, Jesse Van Rootselaar killed five students at Tumbler Ridge Secondary School, an education assistant, her mother, and her 11-year-old half-brother, then killed herself. A 12-year-old named Maya Gebala remains hospitalized with catastrophic brain injuries. Her family is now suing OpenAI.

Whether a referral in June 2025 would have prevented the shooting is not something anyone can know from the outside. But a private judgment call inside an AI company had public consequences. In the same month, two governments answered the question the case forced open, in two very different ways.

Representative Andrew Garbarino, the New York Republican who chairs the House Homeland Security Committee, told WP Intelligence last week he wants AI companies to report suspicious queries to the government as part of Congress’s ongoing AI rule negotiations. He put it plainly. As someone trying to prevent another 9/11, he would like to know if somebody is asking a chatbot how to build a bomb or blow up a skyscraper. His stated concern is terrorism and mass-casualty violence, and he has been holding private roundtables with committee members and AI executives about how the reporting should work. He is floating a direction in active committee negotiations, not introducing a bill.

Ten days earlier, China finalized a rule that answered the same question differently. Five Chinese agencies, led by the Cyberspace Administration of China, issued the country’s first binding rule specifically governing human-like AI chatbots, which takes effect July 15. When a user talks about self-harm or faces a situation that threatens their life or health, companies have to step in and contact a guardian or emergency contact. Minors cannot use virtual companion products at all. Companies serving more than 1 million registered users or 100,000 monthly active users have to run security assessments. The rule is narrower than it might sound. It applies only to chatbots designed to mimic human emotional interaction, not to AI generally. Carnegie Endowment describes the rule as treating sustained emotional interaction with AI as a governance problem rather than a content moderation problem.

Put next to each other, the two proposals point in different places, but pull in the same direction. In different ways, both governments are moving toward a model in which chatbot providers are expected to do something with what users disclose. Garbarino’s default destination is law enforcement. China’s default is the user’s care network, with law enforcement at the edge. The destinations are different. The demand on the company is the same.

China’s rule works the way it does because the Chinese state already has deep visibility into online activity through real-name registration and existing platform cooperation. Routing a chatbot notification through a guardian works because the state already has other ways of knowing. The US proposal would turn a patchwork of voluntary company practices into a required reporting layer. The starting conditions are different, and that changes what the same rule does.

Mandated reporting is not new. Teachers report suspected child abuse. Doctors report gunshot wounds. Therapists have a legal duty to warn when a patient threatens someone. Banks file Suspicious Activity Reports under the Bank Secrecy Act, the 1970 law requiring financial institutions to flag possible money laundering to the government. Each of these regimes took decades to develop the training requirements, legal immunity, threshold standards, and outside review that make them function. Each of them has also prevented real harm and exposed real crime, which is why they exist. But each of them has an uncomfortable track record. Research in the American Journal of Public Health found that 53% of Black children in the US will have a child welfare investigation before age 18, compared to 28% of white children. The Bank Policy Institute, the industry’s own research arm, found that about 96% of Suspicious Activity Reports result in no law enforcement follow-up at all. Mandated reporting catches some harm. It also produces patterned contact with specific communities and a huge volume of false positives whose cost falls on the person flagged.

AI companies are designing the chatbot version of this mechanism without any of that context. The threshold for a report is internal to the company. The person making the call is an employee or contractor trained in usage policy, not a licensed professional whose decision is subject to outside review. There is no legal immunity for the reporter, no standard of care, no external review, and no way for the person flagged to know they were flagged or contest it. OpenAI has said that if the same account were flagged today, it would refer it to police under a revised protocol. It is a meaningful company policy change. It is not a public rule anyone can see or test. A legitimate system would need a public threshold, a standard for who decides, a path to contest the referral, and a way to measure what the system catches.

Reporting I did on facial recognition and policing in the UK, Metropolitan Police data showed that 80% of the people incorrectly flagged by the Met’s facial recognition system were Black. Police built the system under an internal threshold, deployed it, and researchers measured the disparity afterward. That sequence is a governance problem. A reporting mechanism that draws its threshold in private and publishes its failures in public ends up with an oversight system shaped by who gets wrongly caught, rather than by who was supposed to be protected. That is the sequence about to play out with chatbot reporting in both the United States and China.

Every other category of mandated reporter spent years fighting over who gets reported, who gets to decide, and what the people flagged are owed in return. The chatbot version is starting at step one. The flagging systems are already in production. The thresholds are already in place. The people being flagged are not in the room.