Due to lawyers’ excessive dependence on all things Microsoft, the rule of law is taking hits from this week’s Outlook outage. (I was not affected—I use Fastmail.) But weird cybersecurity events are becoming more common. For instance—last week, Vanderbilt University was hacked. And last month, commercial refrigerators in France were too.
AI is indisputably revealing computer-security flaws that humans have long overlooked—for instance, last month AI discovered a critical vulnerability in PHP, a programming language used on 70% of websites, that had been lurking for 10+ years.
Still, on a net basis, is AI improving computer security? That’s going to be a thornier question, because AI’s ability to reveal security flaws is counteracted by two other AI effects:
Hackers—some state-affiliated—are using AI tools to find and exploit weaknesses in systems before they can be detected or fixed, at lower cost than ever.
The rise of AI-powered coding tools within software organizations is quietly introducing weird new vulnerabilities that elude human detection.
When we think of AI-powered mischief, we may imagine hackers cinematically targeting banks and secret government servers. But these systems are operated by entities with big budgets who can afford high-end security and rebuff casual attacks. The greater risk is that AI-powered hacking will disproportionately accelerate against softer targets with smaller security budgets—especially schools, hospitals, and critical infrastructure like water and power. It’s already happening.
Was the Microsoft outage AI-related? No confirmation yet. Though just last month, Microsoft described an AI-driven cyberattack with similar characteristics. In general, I expect that AI-powered hacks and outages will become a more regular feature of American life, and we will have to navigate around the failure of systems so far notable for their mundane reliability.
As certain AI models are used for attacks and certain others for defense, we can foresee a future where the severest cybersecurity vulnerabilities—and their fixes—will both be AI generated, and too complex for humans to comprehend. Once we pass that threshold, we will be observing the first largely autonomous AI-vs.-AI conflict, where humans are incapable of understanding the nature of the battle, yet will be enduring severe consequences.
Literally minutes after I finished this piece, I saw that the FAA has today imposed a nationwide ground stop due to “communications problems” with certain ATC systems. AI is not officially the culprit, but Gary Marcus—stuck at EWR—notes that the FAA recently rolled out nationwide its new AI air-traffic routing system called—smh—SMART.
The Microsoft outage sure sounds like the result of an AI-generated software patch being pushed onto live servers without being fully tested or understood. This is the same company that recently offered buyouts to 27% of its workforce because of the expected efficiencies of AI. The path from here leads only one direction. Microsoft software quality has been a punchline for decades. But in the desktop-software era, one could always refuse to install an update that was reported to be buggy. These days, with desktop software becoming increasingly a wrapper around cloud services, this avoidance technique becomes impossible: if the cloud breaks, so does everything that depends on it. It is gobsmacking that a public company that’s already squandered its once considerable lead in AI is now seeking to throw its existing, stable businesses onto that fire, along with its workforce. But the effect on Microsoft software quality is easy to predict: more outages, lasting longer. (GitHub users have already witnessed this downward spiral for months.) Furthermore, this week’s outage revealed that many Microsoft customers have no alternative email systems.
A JPMorgan Chase software bug is impairing banking transactions. Separately, United Airlines experiences its own system outage.
Apple is experiencing an outage affecting many cloud services. Separately, an NSA communications network is down. A US Senator says “If this is all a coincidence … it’s like the greatest coincidence in the history of the world.”
I enjoyed The Decoherence Trap, a piece that describes a specific kind of rot that AI-generated code will be susceptible to.