When the Los Angeles Department of Water and Electricity was hacked in 2018, it only took six hours. Earlier this year, an intruder was hiding in hundreds of computers linked to water supply systems across the United States. In Portland, Oregon, burglars installed malicious computers on a network feeding part of the Northwest.
Two of those cases – LA and Portland – were tests. The threat of water was real, discovered by cybersecurity firm Dragos.
All three bring to mind a point that has long been known but, until recently, little appreciated: the digital security of American computer networks controlling the machines that produce and distribute water and electricity is woefully inadequate, a low priority for operators and regulators, posing a terrifying national threat. .
“If we have a new world war tomorrow and we have to worry about protecting infrastructure against a cyberattack from Russia or China, then no, I don’t think we are where we would like to be,” said Andrea Carcano, co-founder of Nozomi Networks, a control systems security company.
For-profit and spy hackers have long been a threat to US information systems. But over the past six months, they’ve targeted companies running operational networks like the Colonial Pipeline fuel system, with greater persistence. These are the systems where water can be contaminated, a gas line can cause a leak, or a substation can explode.
The threat has been around for at least a decade – and fears about it for a generation – but cost and indifference have stood in the way of action.
It’s not entirely clear why ransomware hackers – those who use malware to block access to a computer system until some money has been paid – have recently moved on from universities. , from banks and small-scale local governments to energy companies, meat packing plants. and utilities. Experts suspect increased competition and larger payments as well as involvement of foreign governments. The change is finally drawing serious attention to the problem.
The US government began taking small steps to defend cybersecurity in 1998, when the Clinton administration identified 14 private sectors as critical infrastructure, including chemicals, defense, energy, and financial services. This triggered regulation in finance and power. Other industries have been slower to protect their computers, including the oil and gas industry, said Rob Lee, founder of Dragos.
One of the reasons is the operational and financial burden of interrupting production and installing new tools.
Much of the infrastructure running tech systems is too old for sophisticated cybersecurity tools. Removing and replacing hardware is costly, as are downtime. Network administrators are concerned that piecemeal work will be worse because it can increase a network’s exposure to hackers, Nozomi’s Carcano said.
While the Biden administration’s budget includes $ 20 billion to modernize the nation’s grid, it comes after a history of shrugging from federal and local authorities. Even when companies in under-regulated industries like oil and gas prioritized cybersecurity, they received little support.
Take the case of ONE Gas Inc. in Tulsa, Oklahoma.
Niyo Little Thunder Pearson was overseeing cybersecurity there in January 2020 when his team was alerted to malware attempting to enter his operating system, the side that controls natural gas traffic through Oklahoma, Kansas and Texas. .
For two days, his team battled with the hackers who moved sideways across the network. Eventually, Pearson’s team managed to evict the intruders.
When Richard Robinson of Cynalytica introduced the corrupted files into his own identification program, ONE Gas learned that they were malware capable of running ransomware, exploiting industrial control systems and collecting data. user credentials. At its base were digital fingerprints found in some of the most malicious code of the past decade.
Pearson tried to pass the data to the Federal Bureau of Investigation, but the Federal Bureau of Investigation would only accept it on a compact disc, he said. His system could not burn the data to a CD. When he alerted the Department of Homeland Security and sent it through a secure portal, he never got a response.
Robinson of Cynalytica was convinced that a nation-state operator had attacked a regional natural gas supplier. So he made a presentation to DHS, the Departments of Energy and Defense, and the intelligence community on a conference call. He never got an answer either.
“We got zero, and that’s what was really surprising,” he said. “Not a single individual came back to find out more about what happened to ONE Gas.”
Agencies did not respond to requests for comment.
Such official indifference, even hostility, is not uncommon.
Another example is the 2018 burglary in the LA water and power system.
They were not criminals, but hackers paid to break into the system to help improve security.
After the initial intrusion, the city’s security team asked hackers to assume that the original source of the compromise had been corrected (it was not) when researching a new one. They found a lot of them.
Between late 2018 and most of 2019, hired hackers discovered 33 compromised paths, according to a person familiar with the test who was not authorized to speak in public. Bloomberg News has reviewed a report produced by hackers for Mayor Eric Garcetti’s office.
It described 10 vulnerabilities discovered in their own testing, as well as 23 issues that researchers discovered as early as 2008. (Bloomberg News will not release information that hackers could use to attack the utility.) Operation discovered that few, if any, of the 33 security vulnerabilities have been addressed since the report was submitted in September 2019.
It’s getting worse.
Shortly after the hackers produced the report, Garcetti terminated their contract, according to a preliminary legal complaint filed by hackers hired by Ardent Technology Solutions in March 2020. The company alleges the mayor fired the hackers as ” retaliation “for the scathing report. .
Ellen Cheng, a utility spokeswoman, admitted that Ardent’s contract was terminated but said it had nothing to do with the substance of the report. She said the utility frequently partners with public agencies to improve security, including analyzing potential cyber threats.
“We want to assure our customers and stakeholders that cybersecurity is of the utmost importance to LADWP and that appropriate steps have been taken to ensure that our cybersecurity complies with all applicable security laws and standards,” said Cheng in a press release.
Garcetti’s office did not respond to a request for comment.
The case of the Oregon grid – the Bonneville Power Administration – is not more encouraging.
The tests lasted for years starting in 2014 and involved an almost shocking level of intrusion followed by two public reports. A published in 2017 berated the agency for repeatedly failing to take action.
As of 2020, two-thirds of the more than 100 flaws identified by the Energy Ministry and the utility’s own security team remained unresolved, according to interviews with more than a dozen former and current operatives. Bonneville’s security and contractors cyber team at the Department of Energy, in addition to documents, some of which were accessed via the Freedom of Information Act request.
Bonneville spokesperson Doug Johnson did not respond to requests for comment on how to resolve the vulnerabilities, including some detailed in documents reviewed by Bloomberg in 2020.
Dragos estimated in its Cyber Security 2020 report that 90% of its new customers have “extremely limited or no visibility” inside their industrial control systems. This means that once inside, hackers have free rein to collect sensitive data, investigate system configurations, and choose the right time to launch an attack.
The industry is finally focusing on the response.
“If the bad guys are chasing us, there has to be an eye for an eye, or better,” Tom Fanning, CEO of Southern Co., observed at a conference this week. “We have to make sure the bad guys understand that there will be consequences.”