Sign of the cyber times
The CrowdStrike outage could be a harbinger of future disasters.
It is ironic that software meant to protect millions of computers could also cause them to be completely inaccessible. In this digital age of infinite complexity and total connectivity, consequences like those seen in the widespread computer outage on July 19 — that happened as a result of the CrowdStrike configuration update causing a Windows system crash — are now felt across the globe, including Minnesota.
Throughout the state, hospitals and health systems had issues with medical equipment and record systems. Hundreds of flights at Minneapolis-St. Paul International Airport (MSP) were canceled and delayed. Even state government agencies, counties, and cities could not escape the totality of this cyber event. It will take some time to understand how costly these effects were, but initial loss reports of $1.5 million by Sun Country Airlines and a whopping $500 million by Delta give some idea. CrowdStrike, the company at the center of this story, estimates that 99 percent of computers with this particular security software were online as of July 31. With most of the bleeding stopped, it is time for a postmortem.
Such a massively disruptive event is unsurprising when the functioning of society is dependent upon interconnectedness to the internet — a risky endeavor. With all the cyber threats that exist it is surprising that similar events are not occurring all the time. While there is no turning back from this new internet-powered world, there are steps that can be taken to avoid similar disasters in the future.
CrowdStrike Falcon is a platform that encompasses many different cybersecurity technologies. Part of the platform includes a sensor that can be installed on devices to secure them against cyberattacks. CrowdStrike sends out content configuration updates to these sensors through Channel Files so they can detect and stop new and emerging threats. A particular Channel File sent out on July 19 contained “problematic content” that caused the program to error. The issues that arise from errors like these are usually contained within the program to limit the damage to the entire machine. However, security software must run at the operating system level of a computer to properly catch cyberattacks, and in this case, the error meant the operating system itself stopped working.
CrowdStrike should have caught the error before pushing the file out. As one of the leading providers of endpoint security according to market share, CrowdStrike has a profound responsibility to effectively and consistently test its software pushes for potential issues. The company is blaming a bug in the “Content Validator that performs validation checks on the content before it is published.” However, in the world of cybersecurity there is no room for single points of failure.
This Channel File was only pushed to sensors on Windows operating systems, so Mac and Linux machines were unaffected. How could Microsoft allow a buggy file to run on the most sensitive layer of its product? Microsoft is blaming a 2009 anti-competition ruling by the EU that decided Microsoft must let third-party software run on “equal footing” with its other products.
Holding responsible parties accountable is important, but the nature of this outage is even more critical. Cybersecurity is used in part to ensure that systems are accessible — so it must not be ignored that errors and bugs will always be present in human-developed code. Imagine if instead of a buggy file, ransomware had been pushed out by a disgruntled employee or a sophisticated threat actor with access to CrowdStrike’s systems.
After-action reports must be produced in the wake of a cyber incident so that steps can be taken to strengthen systems against future incidents. CrowdStrike is doing exactly that by implementing more software testing and validation checks and creating a more robust content deployment process. Minnesota-based companies and governments should take a hard look at their disaster recovery plans, particularly their backup strategy for endpoints.
Alongside the technical improvements, there must be a reconsideration of how Big Tech regulations are approached. Did the EU take cybersecurity into account when imposing its ruling on Microsoft? Perhaps the ruling was too broad in scope and introduced the possibility of faulty code being injected into Windows. However, without sufficient access, these security tools that many organizations rely on to defend their systems are substantially disadvantaged.
In a world where reliable computer systems stand between a well-functioning society and total chaos, cybersecurity deserves a seat at the table when decisions are made that could affect the most sensitive layers of computer software. Minnesota, along with America at large, must get deadly serious about improving cybersecurity before the fallout from a future cyber incident does.
The presence of computing machines is everywhere, and the reliance upon them is only going to increase. People are waking up to their new reality of interconnection, so high publicity events like the CrowdStrike outage are key to raising awareness of the stakes. From cybercriminals to coding mistakes, there is no shortage of ways for things to fall apart.
Caleb Larson is a member of American Experiment’s Young Leaders Council.