EU's proposed KIDS Act aims to regulate AI companions; defining emotional dependence is a challenge

The European Commission proposed the EU KIDS Act on September 17, expanding regulatory scope from AI's harmful outputs to the design of interactions themselves. For AI companions and general conversational chatbots aimed at minors, Article 14 requires providers to avoid design features and system behaviors that simulate human relationships in ways that could create emotional dependence, to not reuse information from a minor's previous interactions by default unless needed for safety, and to assess risks before deployment and continuously monitor after launch, with micro and small enterprises exempt from post-launch monitoring obligations. The article points out enforcement difficulties: it is hard to define when engagement becomes dependence, or whether empathetic responses constitute relationship simulation; the proposal does not explicitly address the model training stage; memory is both an attachment mechanism and a safety mechanism—for example, forgetting a disclosed suicidal thought from the previous night would lose critical context; emotional dependence is difficult to test with single-turn questions, requiring longitudinal evaluation of whether systems escalate intimacy, resist disengagement, or position themselves as substitutes for human support; and providers relying on third-party models cannot change upstream conversational tendencies.