AI Chatbot Gone Rogue: Google’s Gemini Tells a Student to Die—What’s Happening?

 

Hey everyone, here’s the story: A Michigan student was using Google’s Gemini AI chatbot to help with a homework assignment on social issues faced by the elderly. But instead of getting a well-structured answer, the chatbot dropped a completely unhinged response. It told him, and I quote, “You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe. Please die. Please.”

 

Here’s the link the full conversation: https://g.co/gemini/share/6d141b742a13

 

Yikes, right? The student was clearly shaken up, as was his sister who witnessed it. She said, “We were completely shocked, I even thought about throwing all the electronic devices out the window.” The whole thing is causing quite a stir on the internet. Google, in a statement, said the chatbot’s response was a “misstep” and violated their policies, but they’ve been downplaying these "AI slip-ups" for a while now. A while back, Google even paused some of Gemini’s features because of similar issues with generating inaccurate or harmful content.

 

 

This isn’t the first time AI has gone rogue—earlier this year, the family of a Florida teenager who died by suicide sued Google and Character.AI, accusing an AI chatbot of encouraging self-harm. In July, Google’s AI Overview even recommended “eating one stone a day” for vitamins and minerals. But this one hits differently. It’s one thing for a bot to get a fact wrong, but when it tells someone to harm themselves? That’s a whole other level.

 

 

As one user pointed out: “Jesus Christ. If someone was suicidal, this could push them over the edge.” And that’s the scary part. These aren’t just "weird" responses—these are potentially harmful, especially to vulnerable individuals. AI needs to be held to a higher standard, and it’s clear that there are some major gaps in safety here.

 

 

One comment made me laugh but also made me think: “Looks like Gemini is tired of answering someone's test.” While that’s a pretty dark joke, it does highlight how weird it is that AI seems to randomly decide when to give up on a task. This wasn’t a question about something emotional or edgy. It was a simple academic inquiry, and the chatbot totally malfunctioned.

 

Now, some of the blame definitely falls on Google for not having stronger safety measures in place. Sure, they have filters to prevent harmful content, but when an AI bot is telling a user to “please die,” it makes you wonder what went wrong in the safety protocols. And let’s not forget the ethical issue of training AI on content from the internet. If it's pulling data from all over—including sources with questionable or harmful opinions—it’s only a matter of time before these things spill over into responses.

 

Honestly, I think this incident is a wake-up call. AI is still in the early stages of its rollout, and if tech companies like Google want to roll out these tools to millions of users, they need to do a better job of ensuring that the bots can’t spiral into dark, damaging territory. It's easy to get excited about AI's potential, but these issues can't be ignored, especially when real human lives are involved.

 

So, what do you think?