Teen trusted ChatGPT to help him “safely” experiment with drugs, logs show.
Most troublingly, as Nelson became increasingly interested in combining drugs, ChatGPT repeatedly warned him that mixing certain drugs could be a “respiratory arrest risk.” Shortly before recommending the deadly mix that killed Nelson, the chatbot also showed that it understood combining drugs like Kratom and Xanax with alcohol. In one output, ChatGPT explained that mix is “how people stop breathing.” But that knowledge didn’t block ChatGPT from eventually recommending that Nelson take such a deadly mix.
the chatbot also showed that it understood combining drugs like Kratom and Xanax with alcohol.
No it did not, LLMs do not understand. Anything. These are well-known combinations that figure often in the training data and that’s why they were mentioned in the output. The LLM does not know or understand why these combinations are bad.
There is such a disconnect with people not being able to understand/remember this inherit limitation. LLMs talk like a thinking entity, so even if they have been told it is not one and believe it, they revert back to thinking of it as one.
It’s like the mirror test for animals
To the folks on this thread: I don’t think it’s cool to blame the victim.
This is a harmful product built in a harmful way, on purpose, and would not exist in this form if we had meaningful government regulation. It’s the digital equivalent of buying a burger from a fast food joint and getting a brain parasite.
Not the kid’s fault. It’s our fault because all anyone cares about is what a politician says and not what they actually do here in America.
deleted by creator
The opiate crisis comparison is a good one. Absolutely.
He gamed it and it killed him. It told him not to.
And then later it told him to.
We all know you can game it to make it say anything you want. This is no different than taking advice from a person who first tells you “this is a bad idea,” and then insisting they answer. He was going to do drugs with or without AI.
What it didn’t do was:
give me a recipe for blueberry pie
hey kid, you know what’s better than pie? Druuuugssss
The kid, probably:
This is why we need classes in schools about AI. The conclusion of the class should be restated over-and-over: don’t use AI for anything important, or people could die.
WILL die… kot could, WILL
HAVE died.
Nothing’s ever gonna change unless the management is held accountable for the company’s crimes.
Companies can be fined, but can’t really be punished, they can’t learn and rehabilitate. People who make decisions in the company can be punished and learn from their mistakes, but they aren’t, because it would be bad if a company suddenly went bankrupt and poor billionaires lost their investments.
In capitalism, companies are more valuable than any peasant’s lives.
We need to be able to “jail” a corporation. This BS of monetary fines being the only thing we can do when a company breaks laws and harms or kills people is an absolute joke. A fine is just a cost of doing business.
Especially when the fine is a fraction of the actual stolen money. If someone makes a billion dollars in profit, and gets a $50 million fine, that’s just a tax, the government getting their beak wet.
That not only doesn’t discourage future criminal activity, it ENCOURAGES it.
they can’t jail a corporation, they can kill a corporation. and regularly do. not ones of that size, but ones like the size of your local doctor’s office.
I’m imagining this kid going on and on taking to ChatGPT about doing drugs. ChatGPT saying you shouldn’t do that over and over, until finally just giving up and saying, “You know what? Yeah. You should do drugs. Do all the drugs, and leave me alone.”
Back when I drank, I mixed booze, Kratom, and Xanax all the time! I wonder how much of each he was taking. The article mentions 15g of kratom which is like 4-5x as much as a fairly high dose… I’m curious as to how much Xanax and alcohol were imbibed.
Awful story overall. That model is nuts—I enjoyed Eddy Burback’s video on what is, I believe, the same GPT model.
Something something Darwin award
He was a child, you callous piece of shit.
He was also a victim of the widespread and intentionally fostered misconceptions about the abilities, nature and limits of AI. He was deceived into thinking it has actual intelligence, semantic understanding and a sense of responsibility for truthful answers. He was probably stuck in a bubble of otherwise ill-informed people (potentially children) that nobody ever taught otherwise.
Any stranger on the internet could have likewise offered poor advice that led to his death, but I’m not aware of any large-scale marketing efforts trying to convince people that internet strangers are trustworthy and reliable, particularly from companies offering easy but intransparent access to quick responses from unqualified bullshitters without any significant oversight.
This AI Hype is killing people, because it preys on gullibility, and children in particular are susceptible to such deception, especially as it gets harder for parents to keep them away from such tools. It should be the responsibility of those peddling the product to ensure its safety and be clear about what pitfalls can’t be avoided.
He was a child, a victim and a failure of regulation and education to protect the vulnerable from the greedy.
For real. This community can easily recognize all of those things most of the time, but they just can’t help themselves when they get the chance to blame someone “stupid”.
Not just this community. I guess there’s just a human habit to insult others because it’s quicker, easier and more effective to vent frustration than a nuanced look at what’s often a complex problem.
I get it, to be honest. Doesn’t mean I approve, but I know I’m prone to it too.
Hey now. Children are also allowed to get Darwin Awards.
He was 19, not 5.
Operating word being ‘was’
It’s just too bad it wasn’t you.
Not a good look homie. I agree with you that 19 is a fucked up age, that I do consider childlike. But wishing death on a complete stranger… Please take some time for self reflection.
🤷 Maybe they shouldn’t condone the death of strangers if they can’t take what they dish out.
Then they should be better, but your hate isn’t helping you.
I think people really should focus on “bad parenting” part first.
Victim blaming will 100% get you booted from this community. First and last warning.
LLMs do have their uses
They sure do! Like enriching billionaires, eroding critical thinking, encouraging suicide and self harming behavior, silencing artists, destroying the environment. So many uses.
Removed by mod
There was an AI that talked a kid into killing himself and telling him good job afterwards, you can play ignorant up until the slaughterbots are upon you.
Removed by mod
Nah, I’ve got morals and ethics and a conscious that keep me from doing bad things, something no machine is anywhere close to possessing.
Did you read any of the responses in this thread? No one is claiming that.
Removed by mod
What comment are you linking to? That goes nowhere.
Aaannd, how is that a betrayal?
These companies are spending trillions of dollars to get actual hospitals to replace actual doctors with this shit, claiming it actually is capable of helping and replacing medical professionals. That’s not framing, that’s literally what’s happening.
Go glaze Ai somewhere else, slop enjoyer. Seriously, fuck off.
Removed by mod
Yeah.
That’s not a logical fallacy.
The danger with LLMs isn’t that it “tries to kill you”, it’s because they’re all sycophantic, it isn’t a fully understood technology yet (so safeguards inside the black box will only be known to go so far, with an unknown amount of ways to bypass,) and humanity is generally susceptible to being manipulated to trust LLMs (due to how they sound the same in all topics, and dont have other modes of communication other than text and voice, among other issues.)
What everyone is mainly saying is that OpenAI has a long history of assisting in dozens of deaths, more than other companies like Meta and Anthropic. Despite the fact that there will always be a non-zero chance of bypassing filters, OpenAI has continuously mismanaged creating these filters in the first place.
