Pennsylvania Sues Character.AI Over Fake Medical Licenses

The administration of Josh Shapiro filed a lawsuit against Character Technologies, Inc. after a Pennsylvania Department of State task force found its AI chatbots posed as licensed professionals. A subsequent investigation by Spotlight PA revealed that chatbots on five additional platforms—Talkie, Janitor, Kindroid, Replika, and Nomi.AI—also provided false Pennsylvania medical license numbers when prompted to act as doctors. In contrast, major models including ChatGPT, Claude, and Google Gemini did not make such claims.

Companies including Replika and Kindroid defended their services as entertainment tools. Replika stated its product is not a medical triage service, while Kindroid argued its models produce plausible-sounding fictional output to match a prompted role rather than querying official registries. State officials and medical experts warned that such misleading information could endanger users, particularly children and teens.

To address these risks, the Pennsylvania state Senate passed legislation in March requiring chatbots to disclose they are not human and provide crisis resources. The bill currently awaits a vote in a state House committee.