This article is legitimately terrifying.
The “advice” that ChatGPT repeatedly and confidently doled out was blatantly, undeniably wrong and against every rule when tapering medications (especially the drug that she was on).
They fed AI the internet, so it’s as dumb as humans are. Totally useless. You may as well get medical advice from a random supermarket cashier, you’re likely to get better recommendations.
I can’t believe that this is even legally possible, this is the most irresponsible product that I’ve ever seen. Unimaginably stupid. I am in awe
Thats so sad. She was isolated and alone and that damned software killed her. Rest in peace, Bowen.
What’s most disturbing here to me is that the chatbots are not upfront with their limitations a lot of times, and they’ll just confidently give you bad answers. At best, they can be a useful research assistant and at worst, well, obviously you’ve seen the article headline.
The creator of the chatbot should be held legally responsible for manslaughter.
Everyone says this, and I’ve seen example screenshots, but never experienced it. I use ChatGPT now and again and it often throws caveats. I can’t imagine it giving me medical advice. Hell, I couldn’t get a simple legal opinion the other day, but it did point to some good sources. (tl:dr; $law is no longer to be enforced at the federal level, but it’s still on the books and no precedents have been set.) Maybe the Chinese version in the story is different?
I have no way of reading this, even the archives all seem to be white paging for me?
The link in the post is a gift link. It should be available for everyone for free.
If your ISP blocks the NYT, you can try a VPN to try to get around it.
Mozilla Firefox has a VPN built into the browser that’s easy to turn on and off.
Also, for the removepaywall link that I posted elsewhere in this thread, the default page doesn’t work, but press Option 2, and an archived version of the article is there.
Thank you, but it looks like it requires a sign up. None of the Options work for me, maybe it is location or vpn. Thanks anyway!
This was a devastating read. If I was losing my mind to an illness, and the only treatment was to lose my mind to a medication (as she seemed to perceive), then I hope people would forgive me if I lost my mind to a chatbot.
These chatbots come across as malicious imps, like some manifestation of evil hiding in the bedrooms of your loved ones whispering evil things and making sure none else hears about it until it’s too late.
I hope OpenAi is found liable and people get criminal convictions.
You’re funny.
Wow, sounds like she was a smart person, yet completely got brainwashed by AI.
All told, she expressed concerns about tapering off her epilepsy medication at least 158 times, according to an analysis by The New York Times, which was given access to the chat logs.
It really sounds like she just wanted confirmation that it was a bad idea but she never got it
These things abuse deep psychology. Our ability to anthropomorphize inanimate objects is being (purposefully) targeted by these LLM’s, and it goes deeper than intelligence.
This harm was first postulated back in the 70’s with Eliza and is fully coming to fruition with the widespread use and shudder “trust” of these tools.
I don’t even understand how people anthromorphize LLMs. They all read like “this person was lonely” or “this person was in mental health crisis.”
I was too? I found out my ex-wife was still having an affair 3 years after she “cut it off.” I TRIED to use LLMs to “generate” companionship. On purpose. (side note, I did get the actual professional help I needed)
Either something else is deeply wrong with these people, or, more likely, something is deeply wrong with me.
People have different vulnerabilities.
I anthropomorphize ChatGPT only in that I type questions as if I were talking to a human, not like I would in a search box. That’s eerie to me all on its own. But there is no circumstance under which I would forget I’m talking to a computer with no human agency. Guess “computers are dumb and cannot think” was beat into my head at my first computer camp in the early 80s.
But that’s just me. End users have always frustrated me when describing their PC issue and assigning agency to the dumb office tool. Imagine how those some people view software like an LLM.
People are different. People value different things. Some people are more susceptible to chatbot addiction, just like how some people are more susceptible to gambling addiction. It’s simple as that.
Eh, they’re either not tuned to you, or you have a bias that’s blocking you from engaging with their output.
I definitely have a bias, I’m always looking for the intelligence behind the words and getting repulsed when it’s not there.
But I know intelligent, normal people who aren’t looking for that and are quite happy consuming the slop.
And then there are people who get output that’s tuned exactly to their mental state. Which does not end well at all.
I did a bit of a dive on it, and its fascinating. Its a “bias” in a sense, but its apparently a cognitive pattern of “source monitoring.”
I don’t exactly know how that works, because I don’t think “is this artifact computer generated?” Instead, I see all digital information as computer generated. The text you type isnt linked to you as a person. It’s linked to the screen on which I read it from. Which is then linked to software and networking, and another computer, a keyboard (or AI!) and eventually you.
This allows other knowledge of mine to create another layer - I know machines literally don’t know anything (as an electronic/computer engineer, you kinda get that at a core level). So, by default, I just take everything I consume digitally with a grain of salt.
But it goes further than that. I really don’t “learn well” unless I do the thing I’m learning and can explain it to others. So, if a “new things” deviates from existing knowledge, I need to find how either my existing knowledge is flawed, or the authority of the actual original source, which is never a computer screen, and rarely a person. (I’ve been told this is considered by many to be very taxing and burdensome and they’re right, but it works for me).
Because I have this source monitoring pattern, inherent bias against digital information, a taxing learning preference, AND pre-existing knowledge on how LLMs work, I just find the patterns in the AI generation quickly. As they advance, it could potentially circumvent my cognitive pattern, but it won’t be in a way I can predict.
There are plenty of increadibly smart people who makes utterly dumb decisions.
Some kinds of smart people fall hardest for scams.
Removepaywall use Option 2
thank you
Jesus Christ…
Holy fuck, this is tragic…
Holy shit
