cross-posted from: https://scribe.disroot.org/post/11979760
Google’s AI search told users that an Aachen doctor had been convicted of a sex crime. The conviction belonged to a different man, and on September 24 the Aachen Regional Court [in Germany] banned Google from repeating it, ruling that the company answers for what its AI Mode says. It is the second German court this year to treat Google’s AI answers as Google’s own words, and the first ruling to say so has now been settled rather than tested on appeal.
…
The doctor asked Google to delete the text on August 28 and followed up with a formal legal warning on September 15. Neither worked. The court’s 8th civil chamber then acted without hearing Google first, citing the urgency of false claims that were “publicly visible” to an unlimited audience. Google Ireland, which runs Google Search in the EU, now faces a fine of up to €250,000 for each breach, or detention of its managers, if it repeats three versions of the text. Google can still challenge the order.
The court’s reasoning is blunt: Google “provides AI-generated search and is therefore also responsible for the answers generated in this context.” It added that checking would have been easy: had Google “confronted its own AI” with the doctor’s objections, it would have found the statements were false.
…
That reasoning comes almost word for word from Munich. On May 28 the Munich Regional Court I ruled (case 26 O 869/26) on AI Overviews that told searchers two Munich publishing companies were “known for dubious business practices”, ran subscription traps and kept changing their names to avoid being traced. None of the linked sources said that; the first one cited did not mention the companies at all.
Google argued that it was only summarising other people’s pages, the same defence that has long protected search engines in Germany. The court disagreed. AI Overviews, it said, do not just show links but make “independent, new and substantive statements” built by evaluating and combining third-party sites, so Google must treat them as its own, including as the provider of an AI system under the EU’s AI Act. “Only it has influence over the AI,” the judges wrote.
Google said at the time it would appeal. That appeal has ended in a settlement, The Decoder reported on Saturday; the terms are not known and neither Google nor the publishers would comment. The upshot is that no higher court has ruled, so the Munich judgment is not binding precedent, but lower courts such as Aachen are already following it.
…
This seems like a pretty obvious ruling, hopefully it is more broadly applied.
The ‘oopsie my AI did it’ excuse is wearing thin when ‘it’ is things like this, or encouraging teens to commit suicide. The company is responsible for what communications they send and should be liable for the damage that they cause.
I really dislike how much hype i read in the really pro ai spaces who think law is the next big field after medicine. We already have errors with alrps and we are already trying to introduce another way the justice system can introduce more errors. Lawyers are already getting in trouble with ai generated work that has obvious errors in it to things that don’t exist. If that happens the case should just get thrown out of court, and the lawyers punished heavily.
Like seriously are we going to make ai the judge, the jury, and the executioner too?
People could argue that a person has a final say over it so that makes it okay. There’s a soldier in Israel that reads an ai generated report that assures him that there is a terrorist in a building that will murder his family, he presses a button and bombs the building, and he has the final say over that too. Just numbers on a screen and they where probably bad people anyways, just following the book/ai.
We are getting to the point where we are putting peoples lives at risk, or the very least a traumatic time with guns pointing a gun at your face accusing you of stealing your own car and a few days in the holding cell. Its crazy how much here in the united states ai gets free leeway to break anylaw possible, and some how its okay because an ai did it. They’re rigging your rent prices, and getting away with manipulating prices uber drivers get in a way that is highly illegal if a human was involved yet its completely fine. Predictive markets are some how not gambling because ai. There needs to be a line, it is absolute insanity that trump made it illegal (essentially) for the states to make any law regulating ai in any shape or form. We literally are using the same tech they use to target and genocide kids its crazy.
And the penalty needs to be steep enough that it’s not just an operating cost for these companies.
Google might just budget a few million a year for paying the fine every time a court rules they pushed someone to kill themselves if it’s profitable enough.
EU countries have the right idea for fines, companies get fined as a percentage of their global gross revenue. 10% in some cases (and 20% if there is a repeat offense in 8 years).
Having a few million dollar fine isn’t going to create a proper deterrence.
Google’s like “Whaaaaat come on we even have that little text at the bottom:”
“~Results are probably false.~”
“How could this happen??? Not our fault.”
I don’t know about Germany but in the UK repeating a libel is also libel, so “I was just repeating what someone else wrote” isn’t a defence.
I could be wrong, but this appears to be a hallucination, not repeating information. I can’t comprehend how this could be anything other than a massive liability for Google.
Google argued that it was only summarising other people’s pages
So they’re admitting to plagiarizing? Either they are plagiarizing or they need to take responsibility. Pick one.
Technically summarizing isn’t plagiarizing. Otherwise every English class essay would be plagiarizing.
I’m glad countries in EU take action like this, they still have balls. In U.S.A. Google instructs the court to ignore these things and the courts obey the orders from their masters.
I hate that one has to travel halfway across the world from the US to see some semblance of a justice system doing the obviously correct thing.
Well well well.
