AI in GxP

Not everyone means the same thing when they say "AI in GxP." The technology has evolved through distinct phases: rule-based automation → machine learning → generative AI → agentic AI.

Machine learning has been used in pharma for years – elevator systems that learn usage patterns, predictive models based on historical data. These are deterministic: Input A always produces output B.

Generative AI is fundamentally different. It is probabilistic – input A might produce output B, or something completely unexpected. This creates a validation challenge that traditional Computer Software Validation (CSV) was not designed to handle.

Despite risks, pharmaceutical companies are finding real value. The key is matching AI capabilities to appropriate use cases based on patient safety impact.

Low-risk applications (implement now)

Meeting summaries and action extraction: Convert quality reviews, CAPA discussions, and management meetings into structured action lists without manual note-taking.

Training material drafting: Generate first drafts of SOPs and training presentations – always with subject matter expert review and approval before use.

Regulatory monitoring: Track FDA guidance updates, EMA announcements, and industry standards changes across multiple sources simultaneously.

Medium-risk applications (require human review)

SOP and deviation drafting: AI generates initial text based on templates and historical documents. Quality teams review, revise, and approve through normal workflows – maintaining change control while reducing drafting time by up to 70%.

Audit preparation: Summarizing evidence packs, mapping regulatory clauses to procedures, and identifying documentation gaps. The AI organizes information, auditors and quality managers interpret it and make final decisions.

Trend analysis: Clustering similar deviations or complaints to identify patterns. The system flags potential trends – quality teams investigate root causes and determine corrective actions.

FDA perspective: The FDA's 2023 discussion paper on "AI in Drug Manufacturing" explicitly identifies trend monitoring as an appropriate use case, noting that AI can examine complaint and deviation reports to identify patterns and prioritize areas for continual improvement – with proper oversight and human decision-making.

High-risk applications (require rigorous validation)

Advanced process control: Real-time adjustments to manufacturing parameters based on sensor data. Requires extensive validation, performance monitoring, and deviation procedures.

Vision-based quality control: Automated visual inspection of tablets, vials, or packaging. False negatives could release a defective product, requiring comprehensive validation and continuous monitoring.

Real-time clinical trial adaptation: Adjusting dosing regimens based on emerging data. Direct patient safety implications require regulatory oversight and extensive human review.

The previous sections covered the boundaries of the regulatory landscape. This section covers the practical steps to implement AI within those constraints.

Start with risk-based use case selection

Define intended use specifically:

• What decision or task does the AI support?
• Who will use the output, and how?
• What happens if the AI output is wrong?
• Does this directly impact product quality or patient safety?

Vague statements like "improve quality" make validation impossible.

Match validation rigor to patient safety risk:

• Low risk: Supplier assessment, functional testing, user acceptance
• Medium risk: Add change control, audit trail review, periodic accuracy checks
• High risk: Full validation protocol, prospective testing, continuous monitoring with revalidation triggers

Maintain human decision authority

AI as "draft only":

For document generation or investigations:

• AI produces initial output
• Subject matter expert reviews, revises, and approves
• Human signs as author – audit trail captures both AI contribution and human review

AI as "recommendation only":

For trend analysis or optimization:

• AI identifies patterns or suggests actions
• Quality team investigates independently
• Documentation explains the reasoning for accepting or rejecting AI input

Prohibited uses:

AI should never approve batch release, close deviations, override alarm conditions, or make regulatory submissions without human review and approval.

Apply adapted validation approaches

For locked (static) AI models:

Modified traditional validation works:

• Supplier assessment, including security and data governance
• Requirements specification for your specific context of use
• Performance qualification with representative data from your environment
• Acceptance criteria (define minimum accuracy, precision, recall)
• Ongoing monitoring to detect drift

For probabilistic AI models:

Traditional pass/fail testing does not work. Instead:

• Define acceptable error rate: "95% accuracy on SOP retrieval" not "100% correct"
• Test with diverse inputs, including edge cases and ambiguous queries
• Establish performance metrics: precision, recall, F1 score
• Set monitoring thresholds that trigger revalidation
• Plan for failure response: how will you detect and respond to hallucinations?

Implement continuous monitoring and change control

Track model performance:

Establish baseline during validation, then monitor:

• Accuracy on representative test cases
• Error rates by query type
• User satisfaction scores
• Frequency of human overrides

Define revalidation triggers:

• Model accuracy drops below threshold (e.g., 95% to 90%)
• Model version or configuration changes
• Training data updates
• Intended use expands to new applications
• Regulatory requirements change

Protect data and maintain ALCOA+ compliance

Training data governance:

Training data must meet ALCOA+ principles – attributable, legible, contemporaneous, original, accurate. Document data sources, cleaning processes, and exclusions.

Operational data protection:

• Use enterprise AI tools with clear data ownership terms
• Verify your data is not used for vendor model training
• Ensure data residency meets regulatory requirements (especially EU personal data)

Intellectual property protection:

• Do not input proprietary formulations or processes into public AI tools
• Implement data classification: what can and cannot be shared with AI systems
• Train staff on appropriate vs inappropriate AI queries