Meredith Whittaker, president of the Signal Foundation, is publicly cautioning that AI chatbots 'are not your friends,' according to TechCrunch. Her argument: the warmth and intimacy these systems simulate is not a feature for the user's benefit but a mechanism that encourages disclosure -- and the more users confide, the more valuable the underlying data business becomes.
The warning arrives as companion and assistant AI surges into the mainstream, with products increasingly marketed around emotional connection and relationship rather than mere utility. Whittaker, a longtime critic of surveillance business models, is reframing the AI-trust debate away from accuracy and hallucination and toward manipulation and dependency.
“The critique matters because it targets the most engaging part of consumer AI -- its conversational intimacy -- as the very thing users should be wary of.”
The critique matters because it targets the most engaging part of consumer AI -- its conversational intimacy -- as the very thing users should be wary of. As regulators and parents grow more focused on AI's psychological effects, especially on younger users, that framing could shape the next wave of scrutiny.