OpenAI recently announced that it is tracking signs of user distress across sessions, rather than only within them, and using that information to inform model responses. The announcement follows a wave of new safeguards for mental health and well-being conversations with artificial intelligence at OpenAI and elsewhere, such as parental controls for teens, usage limits and break reminders, and crisis detection and referral. Policymakers have also moved to address associated risks, with a dozen states now requiring companion chatbot operators to disclose to users that their tools are not human, and to institute crisis referral procedures.
This attention, within AI companies and legislatures, is important and urgent. But it is directed almost entirely at acute harm from interacting with AI, rather than the cumulative harm that develops over time, such as the gradual replacement of one’s connections with friends and family by AI interaction.
