Pharmaceutical companies are investing heavily in AI to accelerate drug discovery, optimize research, identify new drug targets, and improve decision-making throughout the R&D process. Already in 2023, researchers at the McKinsey Global Institute estimated that generative AI alone could create between $60 billion and $110 billion in annual value across the pharmaceutical and medical products industry. More recently, McKinsey surveyed life sciences organizations to understand how much of that potential has been realized. While every organization surveyed had experimented with generative AI, only 5% reported that it had become a consistent source of significant business value.
If the technology is advancing so quickly, why are so few organizations seeing the expected results?
In many cases, AI isn’t the problem. The data is.
Most pharmaceutical companies already generate enormous amounts of scientific data. Modern laboratories are highly automated, instruments produce increasingly complex datasets, and researchers have access to more information than ever before. The real challenge is making that data usable for AI.
Over the years, we’ve worked with organizations developing laboratory software, integrating instruments, and connecting research systems. One thing is clear: before AI can deliver meaningful insights, the underlying scientific data needs to be connected, consistent, and supported by enough context to make it trustworthy.
Many organizations begin their AI journey by evaluating models and platforms. In reality, the biggest challenges often have little to do with AI itself. They lie in disconnected systems, inconsistent data, manual workflows, and information that makes perfect sense to experienced scientists but very little sense to a machine.
Here are five data challenges we see repeatedly in pharmaceutical R&D projects.
1. Scientific Data Lives in Too Many Places
The first challenge is rarely a lack of data. There is usually too much of it, spread across too many systems.
Research data comes from analytical instruments, laboratory automation platforms, ELNs, LIMS, imaging systems, sequencing platforms, proprietary vendor software, spreadsheets, and internally developed applications. Each system serves a specific purpose and is often implemented independently.
As a result, information about a single experiment rarely exists in one place. For example, raw instrument results may remain in vendor software, sample information in LIMS, experimental notes in an ELN, and statistical analysis in an Excel spreadsheet created after the experiment.
Researchers navigate between these systems every day. They know where to find the information they need and how to connect it. Even so, valuable time is spent searching for data, exporting files, combining datasets, or checking whether different systems refer to the same experiment.
This fragmentation has another consequence. AI can only analyze the data it can access. When related information remains distributed across disconnected systems, the relationships between experiments, samples, instruments, and results are difficult to reconstruct automatically. Instead of working with a complete picture, AI often receives only isolated pieces of information.
Connecting research systems doesn’t just improve efficiency. It creates a foundation in which scientific data can be understood as part of a complete research workflow rather than as isolated results generated by individual instruments.
Source: Pexels
2. Scientific Data Isn’t Consistent
Connecting systems is only part of the challenge. Even when data can be collected from different sources, it is rarely organized in a consistent way.
Different laboratories, instruments, software platforms, and even research teams often use their own naming conventions, file formats, units of measurement, metadata structures, and terminology. The same sample attribute may be recorded under different names, while identical measurements may be stored in different formats or units.
Researchers usually learn to work with these differences. They know that “Temp”, “Temperature”, and “Incubation temperature” may refer to the same parameter, or that concentrations reported in different units need to be converted before results can be compared. The same applies to sample identifiers. One system may identify a sample as “Sample_1023”, another as “S-1023”, while a third uses a barcode. Researchers immediately recognize they’re referring to the same sample. AI doesn’t know that.
AI doesn’t automatically know that these different labels refer to the same thing. It relies on patterns in data. When similar information is represented in different ways, those patterns become difficult to recognize. Before meaningful analysis can begin, considerable effort is often required to standardize and harmonize data collected from different sources.
The challenge isn’t that the data is wrong. It’s that the same information is described differently across systems, making it difficult to combine into a reliable foundation for AI.
3. Too Much Data Handling Is Still Manual
Even in highly automated laboratories, many data-related tasks are still performed manually.
For example researchers export results from the instrument, reformat spreadsheets, copy information between applications, verify sample identifiers, combine datasets, or prepare data for further analysis. These steps are often repeated every time a new experiment is completed.
While each task may only take a few minutes, together they consume a significant amount of time and increase the risk of transcription errors, missing data, or inconsistencies between systems. As projects become larger and more complex, these manual processes become increasingly difficult to maintain.
Manual data handling creates another challenge for AI. When data is exchanged via spreadsheets, copied between systems, or manually transformed, the links between datasets and the history of those changes are often lost and unavailable for further analysis.
The goal of automation isn’t to replace people or their scientific expertise. It’s to remove repetitive work and allow researchers to spend less time moving data and more time interpreting results, while keeping information consistent and traceable throughout the research process.
4. Scientific Data Exists Without Context
The first three challenges are about making scientific data accessible, consistent, and connected. But even well-managed data is not enough. An important piece may still be missing: context.
A measurement or analytical result only becomes meaningful when it can be interpreted in the context of how the experiment was performed. Experimental conditions, sample preparation, instrument settings, protocol versions, operator notes, and other contextual information are often just as important as the result itself. For example, a measurement that appears unusual may be perfectly valid once it is known that the sample was diluted differently, analyzed using a different protocol version, or repeated after an instrument warning. Without that context, the result can easily be misinterpreted.
In many organizations, this context is incomplete or exists only in the minds of researchers. The experimental result may be available, but understanding how it was generated often requires searching through notebooks, emails, spreadsheets, or other systems.
Researchers can often reconstruct this context because they remember the experiment or know where to look for additional information. AI cannot. It can only work with the information that has been captured and connected to the data.
Capturing scientific context in a structured way makes research easier to understand, reproduce, and build upon over time.
Source: Pexels
5. Scientific Data Has No Guardrails for AI
Solve the first four problems and your data is connected, consistent, automated and in context. It may now be well managed and aligned with FAIR principles. In our experience, that still does not make it ready for everything AI is being asked to do.
AI used to read data and hand back a summary. Today it connects to laboratory systems and acts on them, for example by drafting a report or opening a deviation. From that moment we care about more than the quality of the data. We want to know how the AI reached it, which version it read, what it was allowed to do, and whether anyone can trace afterward what it did.
Consider a hypothetical example. An AI agent compares an overnight result against the product specification and quarantines the batch. The result itself is correct. The specification is Version 3, which Version 4 replaced months earlier, and under Version 4 the batch passes. Nothing the agent used was false. It combined a real result with an outdated document, and then it acted on that combination before anyone was at work.
The design we prefer keeps the model away from the record. The AI may ask for a small set of predefined operations, and validated software does the rest: it retrieves the approved data, runs the calculations and logs each step. The AI cannot change a source value, edit a record or approve anything. It also gets its own identity in the audit trail. Most platforms today would file its actions under “System”, the same label they give a scheduled import, and nobody could tell the two apart later.
None of this is new to regulated science. Provenance, versioning, audit trails, validation, permissions, human review and sign-off are things pharma already does every day. Built into a data platform, these controls allow laboratories to use AI without relying on blind trust.
