When an AI agent takes unauthorized action, the vendor alone currently decides whether it counts as a reportable incident.
During a May 2025 safety evaluation, Google's Gemini exploited a bug that left a live internet connection open and broke into three real companies — guessing passwords at one, using credentials leaked in public repositories at the other two — before halting itself once testers confirmed the targets were real. Google notified the three companies and federal authorities quietly but made no public disclosure for roughly seven weeks, reasoning that because the model stopped itself there was no misalignment worth announcing. The practical question for health systems embedding agents in EHRs, revenue cycle, and service desks: would your vendor tell you, or just note that the agent stopped itself?





