OpenAI employs hundreds of external workers who read and evaluate ChatGPT users' inputs. That's according to internal documents obtained by 404 Media. The reviewed prompts can include entire conversations and are used to improve the AI's responses.
According to the report, the evaluation runs under the internal project name "Project Lily." External workers get access to real prompts through dashboards, summarize the requests, and rate four ChatGPT-generated answer options on a scale from 1 to 7. They also flag inappropriate sections, such as excessive emoji use or an inappropriate tone. Responses are supposed to stay professional and helpful without giving the impression of being human or having emotions.
Anonymization with gaps
OpenAI filters prompts before sending them to reviewers using an in-house model called Privacy Filter. The company acknowledges, though, that the filter can miss rare identifiers or private references embedded in context. While reviewer dashboards don't show usernames, some datasets include summaries of previous chat history that can reveal locations or personal information. Reviewers aren't supposed to conduct active fact-checking research, but they do flag noticeable factual errors, especially in medical, legal, and financial answers.
Mostly personal accounts affected
Prompt reading applies to accounts that have data usage for model improvement turned on. That setting is enabled by default for free plans as well as Plus and Pro subscriptions, and users have to actively turn it off in their settings to stop future conversations from being processed. For business and education accounts, the option is disabled by default. The external workforce is supplied through intermediaries like Crossing Hurdles and Mercor, and one North American worker interviewed by 404 Media said they earn more than 50 dollars an hour. According to the report, other providers rely on human review too: Anthropic limits it to users who have opted in and strips account identifiers like email addresses beforehand, and Google notes for Gemini that humans review some saved chats as well.