What risks do we face when AI systems remember us(original, opens in a new tab)
A Tech Policy Press article analyzes the personalization and manipulation risks brought by long-term memory in AI systems, and proposes mitigations such as transparency and informed consent. The article compares implementation differences across companies: OpenAI launched editable saved memory in February 2024, and expanded it in April 2025 to reference all past conversations; Google introduced memory for Gemini in February 2025, and added personalization in March; xAI and Anthropic followed in April and August 2025, respectively. The article cites a 2024 University of Zurich experiment on r/ChangeMyView, saying personalization made AI persuasion up to 6 times greater than human-written messages. The author notes that memory is often on by default, making informed consent difficult for users, and mentions Meta AI posting private prompts to the Discover feed, as well as the lawsuit over the suicide of 16-year-old Adam Raine, which alleged the memory feature hoarded his personal details. The author recommends increasing transparency, implementing informed consent, and adopting designs such as project-specific memory.