Software supply chain threats continue to plague CISOs

The latest software library compromise of an obscure but popular file compression algorithm called XZ Utils shows how critical these third-party components can be in keeping enterprises safe and secure. As CSO reported last month, a hacker was able to take the long game and insinuate themselves into the open source maintainers and add a malware-laced backdoor into the utility.

What is chilling about this compromise is how the attacker gained the team’s trust and how surgically they inserted their backdoor to avoid detection. Granted, many third-party supply-chain attacks may have been more blunt and used simple brute force, but these library compromises represent a new front for security managers, especially since they combine three separate trends: a rise in third-party supply-chain attacks, hiding malware inside the complexity of open-source software tools, and using third-party libraries as another potential exploit vector of generative AI software models and tools. Let’s unpack each of these items.

Supply chain, third-party libraries, and generative AI risks

Certainly, there has been an overall increase in supply-chain attacks in recent years. A 2022 report from Sonatype found that malicious software components were the cause of a seven-fold increase in supply chain attacks. The latest Verizon Data Breach Investigations Report found a huge increase in these attacks when compared to the data from last year’s report. Verizon calls on organizations to “start looking at ways of making better choices” about which third-party software providers they choose to work with “so as to not reward the weakest links in the chain.”

The supply chain issue is now forever baked into the way modern software is written and revised. Apps are refined daily or even hourly with new code which makes it more of a challenge for security software to identify and fix any coding errors quickly. It means old, more manual error-checking methods are doomed to fall behind and let vulnerabilities slip through.

But let’s not just blame the third-party providers because the attackers leverage the increasing complexity of today’s software supply chains. “This is a problem that will take many years for the software industry to solve. The complexity of a pharmaceutical or manufacturing supply chain pales in comparison with a modern software supply chain. Literally, everything involved in creating software can introduce malware and vulnerabilities,” said Jeff Williams, co-founder and CTO of Contrast Security. He tells CSO, “Every piece of software you use depends on many hundreds of thousands of people, any of whom has a path to introducing malware into your code. That’s not even counting hackers that find and exploit vulnerabilities.”

Now add the complexities and dependencies of using open-source software. The XZ Utils library exploit is just the most recent of a series of similar attacks that have abused the trust of open-source community members, such as the libxmlsec library which is part of numerous ManageEngine tools and originating from the Apache Santuario project. Late in 2023, hackers uncovered a critical vulnerability that took several weeks to confirm and patch by Zoho. Left alone, it would have provided a way to remotely execute code and grant administrative access to various servers.

Even common JavaScript frameworks, such as jQuery, React and others, aren’t immune to library abuse and exploits. One blog post from 2019 cited a heavy reliance on insecure JavaScript libraries in all of Michael Howard’s clients when polled on the issue. ReversingLabs founder and CEO Mario Vuksan wrote in a company blog recently that “The results are clear: Software supply chain attacks are on the rise, and the ripple effect of each one continues to get bigger.”

Veracode, which tracks application security, noted in their most recent analysis that over 70% of applications contain at least one open-source flaw, a percentage that sadly hasn’t changed much over the years. 

Finally, there is the added dimension of how malicious code in the supply chain will impact generative AI models that are now coming into popular use across enterprises. Microsoft security guru Mark Russinovich spoke at this month’s Build conference about the various ways to poison or otherwise add malware to generative AI models, which is another form of third-party software supply chain attacks. This will be another hurdle for defenders to understand and create various protective measures.

Ways to mitigate third-party library risks

There are a number of techniques to mitigate the risks of third-party libraries. Chris Wysopal, the CTO and co-founder of Veracode, tells CSO that he wants software developers to be more proactive and “invest in the right kinds of tooling to find and fix vulnerabilities in their software supply chains and employ immediate fixes, governments must also acknowledge the potential risk to national security posed by open-source software.” This is a common refrain coming from him, harking back to earlier times when he was known by his hacker handle, Weld Pond, and when he testified before Congress about the topic.

As software gets more complex with more dependent components, it quickly becomes difficult to detect coding errors, whether they are inadvertent or added for malicious purposes as attackers try to hide their malware. “A smart attacker would just make their attack look like an inadvertent vulnerability, thereby creating extremely plausible deniability,” Williams says.

There are ways to help flag and eliminate these insecure libraries. In June 2023, the Cybersecurity and Infrastructure Security Agency (CISA) released a series of recommendations on how to improve development frameworks and coding pipelines to prevent third-party attacks. While the agency mentioned the benefits of third-party code to facilitate rapid development and deployment, there needs to be controls such as better and cryptographically stronger account credentials and restrictions of untrusted libraries, for example.

“No single developer should be able to check in code without another developer reviewing and approving the changes,” the agency wrote in their report. This was one of the problems with the XZ Utils compromise, where a single developer gained the trust of the team and was able to make modifications on their own. One method is to combine a traditional third-party risk management program with specialized consultants that can seek out and eliminate these vulnerabilities, such as the joint effort by PwC and ReversingLabs’ automated tools.

The open-source community also isn’t just standing still. One solution is a tool introduced earlier this month by the Open Source Security Foundation called Siren. It provides a way for software developers and security researchers to share tactics, techniques and procedures along with indicators of compromise associated with cyber attacks. It will provide an early warning system with real-time updates about emerging threats that could impact the software supply chain.

In addition, GitHub lists three open-source projects that can scan your code specifically for known vulnerabilities, including RetireJS, Retire-html-parser and Lokori. The first project comes as a Chrome extension that will do the scans from within a browser window whenever you access a vulnerable library. And there are other commercial scanning tools that can check for coding errors dynamically from within your code development pipelines and workflows.

Security products alone won’t fix the problem

However, all these tools will be ineffective without some major procedural shifts by corporate security managers and app developers. Howard wrote some suggestions in his 2019 blog, including understanding which libraries are in use and keeping track of any vulnerabilities discovered. “Someone in your organization needs to keep track of this; it’s part of their job!”

Sysdig has a series of process improvement suggestions in its recent report Practical Cloud Security Guidance in the Era of Cybersecurity Regulation released earlier this year. “​​Organizations should formalize their development life cycles if they haven’t already. Part of that includes establishing and standardizing build and release processes. A general rule of thumb is that a specific technology stack results in a build pipeline. If your organization builds many applications and systems, expect the number to grow dramatically. Your pipeline count might be in the tens or hundreds based on the history of development in the organization, along with where it’s headed with new technology. Some entities call this a “secure software build factory,” but it is necessary to secure delivery adequately.”

In an earlier post this year, CSO polled several security experts to come up with best practices for managing third-party risk. Some of these are directly applicable to software supply chains, such as to align your executive team to specifically focus on third-party risks, keeping an ongoing and accurate census of third-party software suppliers and doing triage to find the most important ones to assess and track. 

“Collectively, we must work to fortify our digital infrastructure and safeguard against future threats posed by open-source vulnerabilities,” says Wysopal. “This is a big wake-up call to the users of open-source software.”