Pennsylvania: first-of-kind lawsuit against chatbot for medical claims
Pennsylvania has filed a lawsuit against Character.AI alleging that the company’s chatbot generated false and potentially harmful medical information, raising concerns about consumer protection and the regulation of artificial intelligence in health-related contexts. State officials argue that the platform’s outputs could mislead users into relying on inaccurate medical advice, particularly when presented in a conversational format that may appear authoritative or personalized. The case centers on whether AI-generated content can constitute deceptive or unsafe practices under existing laws and highlights broader regulatory challenges around accountability, disclosure, and the boundaries between informational tools and regulated health services. For regulatory bodies, the lawsuit underscores increasing scrutiny of AI systems that operate in quasi-clinical roles and signals potential expansion of enforcement efforts to address risks associated with unregulated digital health information.