If you’ve used ChatGPT sensitive information, this news warrants your immediate attention. According to 404 Media reports, OpenAI internally runs a project codenamed Project Lily —hundreds of outsourced workers directly read real users’ ChatGPT prompts (including sensitive personal information) and score the model’s responses to reduce tendencies toward “sycophancy” and “over-personification.”
Who’s Reading Your Conversations
Reports indicate that these outsourced workers were recruited by Crossing Hurdles and compensated via the Mercor platform, with some roles paying over $50 per hour. ChatGPT Currently has Over 900 million usersThe trigger for all this is a setting named “Improve the model for everyone” whichEnabled by defaultmust be manually disabled.
The issue is not review itself, but scale and default behavior.
To be fair, human review of anonymized transcripts has long been standard practice across AI labs. What is truly alarming are three things: scale (900 million users), deliberate obfuscation via codenames, and the “opt-in by default” design—which means most users have no idea this toggle even exists. Adding irony, OpenAI publicly endorsed the “transparency reporting” provision in the FRONTIER Act just that same week—yet such reports should have disclosed Project Lily before journalists uncovered it.
Anonymization does not equal safety
Another unsettling detail from the report is that, even after “anonymization,” these transcripts routinely contain names, addresses, medical records, and other personally identifiable information in practice. For ordinary users, this means anything you paste into ChatGPT could become a data point on an outsourced worker’s task list. Especially concerning is that many people use ChatGPT as a “personal assistant” without realizing the free tier participates in this data return by default.
What you should do
- Individual usersGo to Settings today and disable “Improve the model for everyone” ChatGPT Confirm whether your plan includes this setting—most paid and enterprise plans opt out of training by default
- Enterprise usersComparative reference
- Anthropic’s consumer data policy is opt-in, cited in the report as a contrasting exampleAnthropic’s consumer data strategy is opt-in, serving as the contrasting example cited in the report
This incident reminds us that beyond the debate over slowing capabilities,trust in data processing is equally urgentFor authentic AI tool experiences and safety boundaries, visit AI Dash.