Meta archnemesis turns his attention to OpenAI’s ‘hallucinations’
Generative AI’s tendency to make up information (known in the industry as “hallucination”) can have negative consequences when the information is about a real person. In Europe, these hallucinations may also create some fallout for AI companies deploying the technology—starting with OpenAI.
Noyb, the privacy nonprofit founded by prominent Austrian activist-lawyer Max Schrems, said this morning that it had filed a complaint with the Austrian privacy regulator, alleging that ChatGPT’s hallucinations had broken the EU’s General Data Protection Regulation (GDPR) in multiple ways.
The GDPR gives Europeans the right to complain about companies holding incorrect data on them, and to force companies to correct it. According to Noyb’s complaint—filed on behalf of an unnamed “public figure,” though Schrems confirmed to me that it’s him—OpenAI claims it’s unable to stop ChatGPT from generating inaccurate information, or to correct it. As an example, the complaint says Schrems asked OpenAI to stop ChatGPT generating an incorrect date of birth for him, and was told this was impossible. It also says OpenAI didn’t disclose what data led ChatGPT to emit a false birth date, or where that data came from, again breaching the GDPR.
OpenAI’s reported claim sounds about right, given that: a) gen AI chatbots just predict the words that are most likely to fit in a response to a query, and have no concept of truth; b) the process through which training data influences those responses is super-opaque; and c) once a model is trained on data, it can’t just forget it. (OpenAI had not responded to a request for comment at the time of publication.) But even if it’s correctly describing the limitations of its tech, that doesn’t give the company a valid defense under the GDPR, Noyb lawyer Maartje de Graaf said in a statement.
“It’s clear that companies are currently unable to make chatbots like ChatGPT comply with EU law, when processing data about individuals,” she said. “If a system cannot produce accurate and transparent results, it cannot be used to generate data about individuals. The technology has to follow the legal requirements, not the other way around.”
This isn’t the first complaint of its kind; last year, Polish cybersecurity researcher Lukasz Olejnik also filed one about OpenAI with that country’s privacy regulator. “I’m happy that others also feel legitimate concerns about technical-legal circumstances,” Olejnik told me today, regarding Schrems’s complaint.
The Italian data-protection authority also warned earlier this year that OpenAI was breaching the GDPR, though it didn’t provide specifics. Outside the EU, the U.S. Federal Trade Commission is probing the capacity of ChatGPT’s hallucinations for “reputational harm,” though under current U.S. law that’s a consumer-protection rather than privacy matter.
But Schrems is a formidable opponent—his lengthy crusade against Meta has repeatedly threatened (and still threatens) the ability of U.S. tech firms to process the personal data of European users without breaking European law. As someone who began that crusade while a law student and who went on to build his career around it, Schrems has a strong track record of succeeding in the courts, and OpenAI should be very concerned that it’s fallen into his sights.
On the plus side for OpenAI, the wheels of GDPR justice are slow to turn. But in time, it does seem quite possible that the EU’s privacy regulators will force the Microsoft-sponsored AI darling to limit what its models say about people in Europe, and perhaps to address inaccuracies in its training data. If worst comes to worst, OpenAI might even have to pay a fine of up to 4% of its global revenues—and if that happens, I’m pretty sure Redmond will be thankful that OpenAI isn’t a Microsoft subsidiary.
More news below.
David Meyer
Want to send thoughts or suggestions to Data Sheet? Drop a line here.
This story was originally featured on Fortune.com
