How to manage cyber risk in AI LLM-driven pharmaceutical supply chains

Also read: Indian pharma catching up to Ozempic, Mounjaro demand

Data Analytics-Driven Service Automation – LLMs and GenAI technology will allow a client of a pharmaceutical company to ask service-related questions (both for the customer and the enterprise) such as "Which medications are the most frequently used, and their resulting cost?" and get an answer via AI-powered chatbots. LLMs could help facilitate, for a business enterprise, the automation of complex hierarchical and planning tasks, such as helping triage messages (e.g., medication dosage, potential side effects, interactions with other drugs) to the correct recipients. Moreover, LLMs will generate synthetic datasets for business staff training and onboarding that will reflect automated training and test scenarios that need to be worked out and graded for effective and quality training (including overseeing critical pharma compositions). Examples include generating synthetic (and PII private) datasets that can include comprehensive medical histories, presenting symptoms, and potential treatment responses, enabling sales teams to practice discussing different medications and interacting with healthcare professionals in a realistic setting. 

Resilience Management Simulations – AI-driven LLMs can be deployed to improve the resilience of pharmaceutical supply chains by designing and analysing service disruption scenarios as part of generating and improving business risk management plans. For instance, a simulation could analyse the scenario where a critical supplier's manufacturing facility is forced to shut down due to unforeseen circumstances such as a natural disaster (or a pandemic) or regulatory non-compliance. The LLM would analyse historical data pertaining to the supplier's production capacity, identify potential alternative suppliers, assess current inventory levels of the affected raw material, and project potential delays in the pharmaceutical company's production and delivery schedules. This simulation provides invaluable insights into the company's vulnerabilities. It allows for the development of proactive contingency plans, such as identifying and pre-qualifying alternative sources of supply or strategically increasing safety stock levels for critical materials. LLM-driven simulations can be employed to analyse new or impending regulatory changes related to pharmaceutical manufacturing processes, distribution protocols, or import and export regulations. These models can simulate the potential impact of these changes on the pharmaceutical company's supply chain, assessing the time and resources required for compliance, identifying potential disruptions to existing operational processes, and evaluating the overall economic impact on the company's business. 

Knowledge Management - Knowledge management has always been an uphill challenge in pharmaceutical enterprises and their supply chains. The status quo in this regard is mostly paper-based, with a lot of manual and time-consuming effort going into mining data from document piles and subject to human biases (across enterprise divisions/groups) and errors in data interpretation. Simply put, knowledge management is highly inefficient. Pharmaceutical companies possess vast internal documentation, including research reports, scientific publications, standard operating procedures (SOPs), regulatory guidelines, and training materials. LLMs can be deployed to create intelligent search engines that enable employees to efficiently locate specific information within these extensive repositories in a scalable manner. In addition, LLMs enable researchers, regulatory affairs personnel, and other employees to quickly grasp the main points of these essential documents without needing to read them in their entirety. Overall, LLMs promote scalability, reduce interpretation biases between enterprise divisions and reduce time and budgetary costs of knowledge management.

Cyber Issues of LLMs in Pharma Supply Chains

While the use of LLMs comes with a plethora of management (and technological) benefits for pharma supply chains (as characterised by applications mentioned above), its use is loaded with many cyber vulnerabilities that can impact training data and models in a manner that leads to biased and erroneous outputs, cybersecurity/privacy breaches, and industry system failures. Three major LLM deployment vulnerabilities are:

1. Third-Party Package Vulnerabilities – The overarching technology-independent cyber issue in (pharma) software supply chains is its reliance on weakly regulated (open source) software. Most (around 80 percent) of the software packages and/or AI models used for developing LLM applications are from third parties and prone to security vulnerabilities (these packages are either new or too deprecated to have commercial patching available). These security vulnerabilities will adversely affect pharma applications concerning the consumer service quality of pharma businesses and their ecosystem and the privacy and security of sensitive pharma data of consumers and businesses, similar to other business sectors.  
2. Vulnerabilities in Pre-Trained AI Models – Most AI models used for developing LLM applications are from model repositories (e.g., Hugging Face). These models are black boxes that are officially not screened in through evaluations and are hard to look into for finding and fixing security loopholes, hidden biases, and backdoors. Businesses in pharma supply chains commonly use these models, which can result in adverse large-scale chain cyber impact. Moreover, many such models are often merged (e.g., via collaborative business efforts) to support a single business application – increasing the cyber-risk terrain for the application.
3. Vulnerabilities in LORA and PEFT Adaptors – Most of the AI models in repositories are fine-tuned using the popular adaptor application LORA (Low-Rank Adaptation) and PEFT (Parameter Efficient Fine Tuning) bolting on pre-trained LLMs to increase model efficiency. These adaptor methods are susceptible to malicious compromises affecting multiple enterprises in pharma supply chains whose services rely on LLMs. Currently, model and adaptor provenance (using Model Cards and AIBOMs) is quite weak not to be able to offer guarantees on model origin so that governance cannot track down perpetrators whose actions are most liable for a pharma supply chain cyber-attack. 

Managerial Action Items 

The following action items are proposed to mitigate the effect of LLM security vulnerabilities on the pharmaceutical industry and its supply chain ecosystem. 

Action Item #1—Business managers must ensure AI model encryption with necessary security checks to avoid model poisoning and tampering. One way of complying with these security checks is by performing vendor attestations of AI LLM models used for pharmaceutical applications before introducing them into applications and supply chains. 

Action Item #2—Business decision-makers should invest in sufficient AI Red Teaming solutions (e.g., Decoding Trust) to select and evaluate an AI model and make such solutions part of MLOps and LLM pipelines. These solutions comprise anomaly detection and adversarial robustness testing mechanisms to detect model tampering and poisoning. The consequent learning and relevant AI CTI information should be shared (in a private manner) within a supply chain network. 

Action Item #3 - Enterprise management must use SBOMs and AIBOMs to maintain an inventory of audited components in AI models and data sources used for their training; check the licensing compliances of such models and training data sources; and use such information to train AI Red Teaming staff and solutions to detect outlier elements. Concerning SBOM and AIBOM analysis, cost-effective operations solutions should be used to identify the most critical (and vulnerable) elements in BOMs populated with many components. 

Action Item #4 - Business managers must collaborate with their technical team and invest in state-of-the-art and relevant technology (e.g., HuggingFace SF_Convertbot Scanner) to detect abuse specific to AI abuse – and avoid using generic solutions. The technical team should invest in strict sandboxing solutions to limit model exposure to unverified training data sources. AI-driven LLM solutions should also be integrated with Retrieval-Augmented Generation (RAG) and grounding solutions to prevent hallucinations on related outputs across any pharmaceutical business enterprise's design, research, and analysis phases of drug design and discovery and knowledge management activities. 

Ranjan Pal (MIT Sloan School of Management, USA), Bodhibrata Nag (Indian Institute of Management Calcutta)