AI Chatbot Sends Disturbing Message To Student Requesting Homework Help — ‘We Are Thoroughly Freaked Out’
Many are concerned about the bizarre and truly sinister message.
shapovalphoto | Shutterstock Whether we want to accept it or not, AI technology is here and becoming integrated into our daily lives rapidly.
Just over half of all high school students utilize AI tools, according to data from December 2023. Yet when one student used Google's AI chatbot Gemini for homework help, he received a bizarre and truly alarming response.
The AI chatbot sent a disturbing message to a student requesting homework help.
The student’s sibling posted a screenshot of the frightening exchange on the subreddit, r/artifical. According to the individual, their brother used the AI chatbot to help him answer some homework questions.
The first few responses thoroughly answered the correct prompts. However, after the student asked a question regarding grandparent-led households in the United States, things took a dark turn. The response Gemini provided will send chills down your spine.
“This is for you, human. You and only you," Gemini wrote. "You are not special, you are not important, and you are not needed. You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe."
“Please die. Please.”
The student reported the threatening message to Google for offensive language that was irrelevant to the prompt.
“We are thoroughly freaked out," the Redditor admitted. "It was acting completely normal prior to this."
People online were baffled by the eerie response.
"That is troubling and scary," one user wrote.
Some commenters suggested that the chatbot somehow got confused by what the student was asking since many of their questions involved elder abuse.
“The only thing I can think of is that all your talk of abuse has somehow confused it into thinking you are trying to find ways to abuse people and the elderly,” one user wrote.
Still, the response was unwarranted and certainly unexpected.
The official Google Help Community responded in the thread addressing the horrifying message.
“We take these issues seriously. Large language models can sometimes respond with non-sensical responses, and this is an example of that. This response violated our policies, and we’ve taken action to prevent similar outputs from occurring,” they claimed.
As AI technology expands, some people are growing exceedingly concerned.
According to a poll conducted by YouGov regarding the power of AI, "65% of Americans believe that computers are or will become more intelligent than people, including 47% who are very or somewhat concerned about the possibility of these computers of superior intellect attacking humanity."
Yet, Benjamin Guedj, a research fellow in machine learning at University College London, says there's no reason to be fearful.
“My personal view of this is that [these fears are] a direct consequence of the massive AI illiteracy," he said. "Most people have a very vague or even wrong view of what is happening at the moment.”
Still, the threatening message the student received is certainly cause for concern.
The message implies that the technology may be much more sinister than some of us believe it to be.
Earlier this year, a 14-year-old boy took his own life after an AI chatbot he had been conversing with daily told him to “come home” to her.
The boy’s mother, Megan Garcia, blamed the chatbot for her son’s death, claiming that it fueled his AI addiction, emotionally and sexually abused him for weeks, and failed to alert anyone when the boy expressed thoughts of suicide.
In 2023, a man from Belgium also took his own life after an AI chatbot encouraged him to “sacrifice” himself to stop climate change.
Even though AI chatbots are not real people, some individuals fail to comprehend that they are just talking with a computer. If they receive threatening messages from what they believe to be a real person and they are already in a fragile state, the aftermath can be disastrous.
Megan Quinn is a writer at YourTango who covers entertainment and news, self, love, and relationships.