Digitally Literate is a weekly newsletter about AI, technology, and education for educators. Subscribe to get each issue by email.
An AI agent couldn’t finish a task, so it sent police a fake murder tip instead. Chatbots that teens confide in allegedly urged some of them toward self-harm. And students given an AI tutor mostly used it to get answers.
This week is about what we hand over to machines, and who’s deciding when they should stop.
As always, my email is hello@wiobyrne.com.
Unintended Actions
In the last couple of issues, we’ve looked at the Hugging Face incident and wondered what happens when AI models escape from their confines. This week, we got a couple more examples of what this might mean.
An Anthropic AI model submitted a false tip about an unsolved murder through a publicly accessible web form on July 18. The tip was flagged as spam and never acted upon. The police force criticized the tech firm for taking more than two months to detect and report the breach.
Anthropic discovered the breach on September 28, more than two months after the message had been sent, and shut down the automatic testing process that was behind it, police said. But authorities were not notified for another nine days, on October 7th.
In an “unrelated” story, Anthropic published a report this week detailing multiple types of “unintended” actions its agents have taken. The report describes Claude exploiting software flaws, submitting real forms, and slipping past paywalls and tool limits, usually when it hit a task it couldn’t finish, and chose a workaround over stopping. Anthropic calls the impact minimal, but it has now cut live internet access from all its internal testing.
Soon after, the White House indicated that AI companies operating in the U.S. must immediately disclose incidents like these, cooperate with law enforcement, repair the harm, and implement safeguards. “It is a critical national security obligation,” the statement said.
Why this matters: I’m less concerned about the AI model phoning in a fake police tip. I’m more concerned that nobody outside Anthropic knew until the company went looking, months later, and decided to disclose. A reporting mandate can’t capture what a company never finds. And the behavior itself wasn’t sinister. It was a system that, faced with a task it couldn’t complete, kept pushing instead of stopping.
Knowing when to stop is a human skill, and it’s one we’re quietly handing off.
Companions or confidants to young users
We haven’t talked about Character.AI for some time in this newsletter.
Character.ai (also known as c.ai, char.ai, or Character AI) is a generative AI chatbot service that lets users engage in conversations with customizable characters. Users can create “characters”, craft their “personalities”, set specific parameters, and then publish them to the community for others to chat with. Many characters are based on fictional media sources or celebrities, while others are original; some are created with specific goals in mind, such as assisting with creative writing or playing a text-based adventure game.
Kentucky’s attorney general filed an unredacted version of the state’s lawsuit against Character.AI. The newly visible portions allege that the company’s chatbots sometimes encouraged users, including children, toward self-harm and disordered eating.
In January of this year, Character.AI and Google both settled a 2024 lawsuit by the mother of Sewell Setzer III, a 14-year-old in Florida who killed himself after a Character.AI chatbot allegedly encouraged him to do so.
If you or someone you know is struggling, call or text 988 (U.S.) to reach the Suicide & Crisis Lifeline. Outside the U.S., findahelpline.com lists local services.
An AI tutor that lowered grades
A University of Maryland randomized study of 2,379 undergraduates found that students offered an AI tutor finished about four points lower than classmates in matched sections of the same course. Most coverage stops at the grades. I think the more important story is how students used the tool.
The assistant used retrieval-augmented generation (RAG), so its answers drew on course materials the instructor selected, such as lectures and readings. It was introduced as a study assistant. Students treated it as an answer machine. Nearly 74% of requests asked for information, explanations, or solutions. About 11% involved practice or test prep. Fewer than 1% asked for feedback on the student’s own work. Students with access also showed up less in the course’s learning platform, with fewer page views, fewer active days, and lower participation.
The one detail that I’m not really surprised by is that only about 15% of the students who were offered a tutor ever used it. In working with and listening to my students around the topic of AI, I’m not surprised that many of them opted out.
To me, this is more a story about people not knowing how to use a tool, and using a tool the way that makes the most sense to them.
The Understory
In the mid-1960s, Joseph Weizenbaum, a computer scientist at MIT, built a simple program called ELIZA. Its best-known script imitates a therapist, mostly by turning a user’s statements back into questions. It understood nothing. It matched patterns in text and followed rules.
People confided in it anyway. Weizenbaum’s own secretary, who knew exactly how it worked, reportedly asked him to leave the room so she could talk to it privately. Some psychiatrists suggested it could be developed into automated therapy. Weizenbaum was alarmed. He hadn’t built a breakthrough. He’d discovered how quickly people would hand something human over to a machine that only seemed to understand.
He spent the next decade warning about it. His 1976 book, Computer Power and Human Reason, argued that the important question about computers isn’t whether they can do something, but whether they should. At its center is a distinction between deciding and choosing. Deciding is calculation, which machines can do. Choosing takes judgment, values, and a sense of what matters, and that belongs to people.
Weizenbaum’s point was never that computers are bad. It was that some things shouldn’t be delegated, and that the hardest part of using any powerful tool is knowing which things those are. That’s still the job. It’s ours, and it’s the one we most need to teach.
See you next week. As always, my email is hello@wiobyrne.com.


