Even by the dizzying pace that AI development has already set, this spring has been… a lot. There’s the headline news, of course, like OpenAI founder Sam Altman warning Congress, of the potential existential harms AI might pose, or yesterday’s open letter saying that AI should be treated with the risk profile of “pandemics and nuclear war.” But there’s also the near-constant drumbeat of weird, embarrassing and disorienting AI news — not the stuff of a techno-thriller plot, but just as important to keep an eye on as the technology rapidly percolates through society. “There are equally dangerous problems that are far less speculative because they are already here,” said Louis Rosenberg, a computer scientist who published an academic paper earlier this year on “Conversational AI as a Threat to Epistemic Agency.” “You don't need a sentient AI to wreak havoc, you just need a few sentient humans controlling current AI technologies.” You could call it the (early-days) AI Hall of Shame. Even AI optimists need to think hard about what these incidents mean — and how they suggest what tools we might need to actually deal with this disruptive technology. CheatGPT Jared Mumm’s students had a bit of a rough end to their spring semester. A few weeks ago, the animal science professor at Texas A&M University at Commerce emailed his class to inform them that he had run a check by ChatGPT to analyze their essays and see whether they had been composed… by ChatGPT. Which, the bot dutifully reported, they were, and therefore every student in the class would be receiving a grade of “incomplete,” potentially endangering their diplomas. Except they weren’t. After one student proved via timestamps in Google Docs that she composed her essay herself, Mumm gave his students the opportunity to submit an alternate assignment, and a university spokesman noted to the Washington Post that “several students have been exonerated and their grades have been issued, while one student has come forward admitting his use of [ChatGPT] in the course.” Whatever the final outcome for those harried students, this example is perhaps the most straightforward one yet of how blind human trust in AI-generated content can lead to disaster. AI gets many things wrong. (Plagiarism detection is especially difficult.) For Mumm’s students, that meant a fraught end to their semester, to say the very least, and it could have far more serious repercussions in scenarios with less margin for error. An AI “flight” of fancy … Like, for example, a lawsuit that makes it to federal court. As the New York Times reported over the weekend, a federal judge in Manhattan is threatening to sanction a lawyer who created a 10-page brief filled with references to imaginary decisions and precedent — all invented by ChatGPT. The lawyer, Steven A. Schwartz of the firm Levidow, Levidow & Oberman, insisted he had no intent of defrauding the court in citing entirely made-up cases like “Varghese v. China Southern Airlines” as part of his client’s personal injury lawsuit against the airline Avianca. As the Times notes, he even said that he “asked the program to verify that the cases were real,” which of course it dutifully verified. Good enough, right? Not for the judge, who’s scheduled Schwartz for a hearing on June 8 to “discuss potential sanctions.” We’re moving up the risk ladder: the law has significantly less room for leniency than a classroom does, and to be overly credulous to AI could not only threaten the credibility of a given case, but that of the attorneys (and the legal system) itself. An unexpected blast radius And then there are the real catastrophes. Well, fake real catastrophes — that have real consequences, despite the lack of any “real” damage or danger. Washington was rocked last week by a faked video that circulated on social media claiming to show an explosion near the Pentagon, most prominently shared by a popular national security Twitter account with more than 300,000 followers. There was no explosion. But the video sent very real shockwaves across the country: The S&P 500 briefly dipped by a quarter of a percent. The White House press shop went into full crisis preparation mode, as West Wing Playbook reported this morning. Twitter announced it would expand its “community notes” crowd-sourced fact-checking feature to include images. This is already pretty bad, and it doesn’t include any of the additional scenarios — mass blackmail, propaganda, targeted financial fraud — helpfully outlined in a 2022 Department of Homeland Security memo. How should regulators know where to start when it comes to AI-proofing our most vulnerable systems? Viewed through the lens of human error or gullibility causing most current AI harms, the European Union’s risk-based framework outlined in the draft text of its AI Act begins to look fairly sensible — the more sensitive the system, the more legal restrictions are placed on the use of AI systems there. “The AI Act from the EU is a good step towards controlling many of the risks of AI,” Rosenberg said, pointing out that it could be quite useful in regulating potential harms of institutional AI deployment, like in parole, hiring, or lending decisions. But outside those institutions there’s still a Wild West of human error, laziness, and advantage-taking, and guarding against that will take a lot more than federal regulatory strictures. Regulators “need to be focused on the problems that are about to hit because AI capabilities are moving so quickly,” Rosenberg said. “The EU proposal is very good, but it needs to look a little further ahead. By this time next year, we will all be talking to AI systems on a regular basis, engaging interactively, and we're not ready for those dangers.”
| 
