Modern societies are dependent upon highly connected, complex digital systems in all areas of society — for example banking, health care, industrial automation, cybersecurity etc., and many others which all depend upon software and/or pre-trained AI models. Yet despite this connection, these same systems have exposed a major weakness; when one link or dependency within a system fails, it can potentially cause a domino effect in other systems and thus create a global security crisis.
SolarWinds (2020), Log4j (2021), and the Hugging Face credential breach (2023) demonstrated to us that code, models, and data sets' integrity is a national security risk. To mitigate this type of systemic vulnerability, we need to move beyond reactive measures such as patching, to proactively governing our use of software and AI through governance.
This article will present eight strategic controls to provide enhanced protection to the software and AI supply chains.
1. Provenance and cryptographic signing
In the case of the SolarWinds hack, it was shown that if an attacker was able to infiltrate a trusted software development environment (build process), they would have been able to create a global backdoor from a trusted piece of software with a software update. The hackers were able to insert malware in SolarWinds' build pipeline that created a backdoor in Orion software updates; those software updates were signed and sent out as legitimate. Thousands of public and private organizations installed the backdoor without knowing about it.To avoid such catastrophic failures, all digital artifacts – including the source code and compiled binary versions of software, the weights used by Artificial Intelligence models and the data sets that are used by those models – need to include cryptographic signatures and provenance metadata. Governments must provide baseline standards on artifact signing similar to that of EU's NIS2 Directive and the U.S. National Institute of Standards and Technology (NIST) framework.
Software repositories such as npm, PyPI, and Hugging Face must refuse to accept unsigned artifacts for upload and organizations using software must configure their pipelines so that artifacts are verified with signatures automatically. With signed provenance, you will have the ability to track an entire ecosystem when someone makes a modification to an artifact or tampers with one, thereby creating immediate detection for modification or tampering.
2. Software bill of materials (SBOM) for AI and code
In addition to the log4j example demonstrating how very few organizations have knowledge about their dependence on a particular library in a software application, many other enterprise applications utilising pre-trained artificial intelligence models may be unable to provide an accurate list of the various components such as tokenisers, training data sets and optimisation libraries.An organisation can create an auditable document listing each component including AI model metadata known as a Software Bill of Materials (SBOM) which allows them to track where vulnerabilities are located in the system.
3. Continuous dependency graph monitoring
As shown by the 2018 npm event-stream attack, it is often transitive dependencies which carry the most significant risk in that they may contain an unknown malicious sub-package. This was the case for flatmap-stream, which slipped past all the usual checks and was then introduced to production wallets and fintech applications.As new versions of packages are released, or older packages are deprecated, these dependency graphs can be considered dynamic networks; therefore graph analysis and monitoring techniques will enable enterprises and governments to identify suspicious activity such as abnormal version changes, inactive maintainers or unexpected ownership transfers.
Government agencies (such as CISA in the United States and ENISA in Europe) along with large enterprise organizations should deploy graph-based threat intelligence solutions to better understand their dependency exposure. Through public-private cooperation dependency visibility can be transitioned from being fragmented and disjointed views of individual environments into a single unified shared risk radar.
4. Centralized vulnerability intelligence for AI systems
AI ecosystems are lacking a common registry of vulnerabilities that is typical of other types of software. It is because when poisoned or backdoored models were found on Hugging Face and TensorFlow Hub, it had no universal process for tracking, classifying, or reporting those types of threats. Read More
The global community (MITRE, OpenSSF and the AI Incident Database) should extend their existing taxonomy to include AI-specific categories of vulnerability: model poisoning, dataset contamination and tokeniser hijacking, to help address this problem.
A centralized data source for AI vulnerability intelligence will enable a shared situational awareness and will provide an accelerated response time. The open repositories of AI vulnerability intelligence should be enabled to automatically scan for malicious payloads.
Also Read: How can companies secure their future with rise in agentic AI adoption
5. Securing build and training pipelines
The Codecov breach (2021) demonstrated how vulnerable build environments are today. A single edit to the CI/CD build process can lead to large scale credential theft which can compromise thousands of dependent systems. Attacks that are equivalent on an AI pipeline will be able to modify your model checkpoint(s), modify your training dataset(s), or even modify the fine-tuning script(s).Build and training environments must be treated as critical infrastructure. Security measures can include signed commits, isolated build runners, deterministic builds and integrity checks for all datasets and pre-trained weights.
To encourage organizations to adopt secure training pipelines, cloud service providers (AWS, Azure, Google Cloud) must embed these controls by default into "secure-by-design" ML training pipelines. Governments can create compliance frameworks and procurement requirements that will incentivize adoption.
6. Managing High-Centrality Dependencies
The use of structural choke points within the software and AI ecosystems exists. Libraries such as numpy, requests, and colors support tens of thousands of software applications, while large-scale datasets (e.g., LAION-5B) and model families (e.g., transformers) form a foundation for entire sectors of the AI ecosystem.In 2022, when two developers deliberately sabotaged their own libraries ("colors" and "faker.js") on the npm registry, thousands of systems were impacted overnight. The impact was an example of how a single node in a dependency tree could produce global disruptions.
Therefore, policymakers and industry coalitions have a responsibility to find and protect critical dependencies with high centrality through regulatory and/or oversight mechanisms. Open-source software and AI components that are important to multiple industries should be supported by funded maintenance and formal governance and continuously monitored for security vulnerabilities. Insurers and regulatory agencies also have the ability to include dependency centrality metrics into risk assessments to ensure that those with high centrality receive priority consideration for oversight and funding.
7. Contributor identity and repository governance
The attacker is able to gain access to the developer account by stealing the developers’ credentials in order to upload malicious code into the repository. Multiple breaches of both PyPI and npm have occurred due to an attacker gaining access to a developer's account and then uploading malicious updates.Repositories need to implement multi-factor authentication (MFA) and verify the identity of all maintainers on high impact models and datasets and require regular key rotation. GitHub has started requiring MFA for maintainers of top repositories, and it is recommended that all model and data set providers do the same. OpenSSF and The Linux Foundation can create lists of "high risk" maintainers currently being actively protected, and governments and insurance companies can provide funding to support resiliency efforts similar to "critical maintainer subsidies."
8. Economic and insurance incentives
Security isn’t simply a technical issue – it is also an economic one. Insurers are able to encourage improvements across systems by matching their premiums with organizations’ actual security posture as verified by third-party audits.Organizations that have an active signed SBOM, who enforce provenance and who secure their CI/CD environment should be paying less than those that do not. There are examples of this model in use today. The Beazley Cyber CAT Bond (2023) showed that financial markets were capable of absorbing systemic cyber risks. Similar models can be created by cyber-insurers using tiered coverage, rewarding organizations for their proactive supply chain security measures.
Governments can support such initiatives with tax incentives, procurement preferences and disclosure requirements for organizations that meet supply chain security benchmarks. Aligning economics with security will allow it to become a competitive advantage for organizations, and not simply another compliance requirement.
From Reactive Defense to Structural Resilience
Both the SolarWinds and Log4j breaches have been a wake-up call for the technology community as it has come to realize that neither software nor artificial intelligence (AI) supply chains are a linear manufacturing process, but rather a complex web of dynamic interdependencies.Vulnerabilities spread like diseases throughout the dependency graph and do not become apparent until they reach a critical mass within the network.
Traditional security tools were designed to protect endpoints and the perimeter of systems, however today's risks lie further upstream in the hidden layers of dependencies, tokenisers, datasets, and CI/CD scripts. It is impossible to identify each and every vulnerable link; instead we must ensure that should one fail, the system will be able to absorb the resulting shock without collapsing.
To achieve this type of resilience, we need a multi-layered governance model which includes:
1. Regulatory standards - Government establishes baseline security and provenance standards for all vendors, requiring SBOMs and signatures
2. Platform custodianship - All repository and model hub operators continuously verify their assets' identity, manage their identities, and perform automated scans
3. Financial incentives - Insurers, investors, and rating agencies offer financial incentives based on a vendor's demonstrated security maturity
4. Transparency among collaborators - Vendors, researchers, and regulatory bodies collaborate to share vulnerability intelligence with no fear of liability
Operationalizing these layers of governance will enable societies to move from a reactive approach to a proactive approach by designing systemic resiliency and enabling digital ecosystems to recover and evolve while being attacked.
Ranjan Pal (MIT Sloan School of Management, USA)Canon Robins (MIT EECS, USA)Bodhibrata Nag (Indian Institute of Management Calcutta)