AI in Biopharma Manufacturing: Are We There Yet?

Fact check — Context of Use: What Decision Is the Model Making?

Claims were extracted from the episode script and verified against the primary documents by AI agents that saw only the claim and the document. This report is itself AI output; it can be wrong. Corrections: jack@jackprior.ai.

Fact check — Context of Use: What Decision Is the Model Making?

This is the fact check of the episode as published. Each factual claim was extracted from the script and verified by an AI agent that saw only the claim and the primary document (the PDFs in the corpus). Verdicts: SUPPORTED (the document says it), PARTIAL (supported with a difference, noted), NOT-IN-CORPUS (the source is not among the primary documents on disk), NOT-CHECKABLE (an estimate or a characterisation). Opinions voiced by the hosts are listed but not verified. This report is itself AI output and can be wrong; corrections: jack@jackprior.ai.

Episode as published	Claims
Claims checked	74
Supported by the primary text	70
Confirmed by official web sources	4
Resting on secondary sources or hedged as such	0
Corrected before release	0
Open	0
Host opinions (not verified)	16

Claims resting on secondary sources

These statements stand, but the primary documents on disk do not themselves confirm them: they come from the landscape reference (a working index whose rows are checked against the web), are estimates, or are hedged in the episode as such.

Line	Speaker	Claim	Verdict	Why it stands	Evidence
7	SARAH	The FDA draft is issued by CDER together with CBER, CDRH and CVM.	PARTIAL	Footnote 1 also names the Oncology Center of Excellence, the Office of Inspections and Investigations and the Office of Combination Products; the point spoken (it speaks for the agency, not one office) holds	Footnote 1: prepared by CDER 'in collaboration with' CBER, CDRH, CVM, the Oncology Center of Excellence, the Office of Inspections and Investigations, and the Office of Combination Products
7	SARAH	The FDA draft's comment docket closed in April 2025 with about 120 comments.	NOT-IN-CORPUS	Landscape row marks the docket as confirmed (FDA-2024-D-4689, 119 comments, closed 7 Apr 2025); the draft text says nothing about its docket	Landscape row: 'docket confirmed — FDA-2024-D-4689, 119 comments, closed 7 Apr 2025'
7	SARAH	The FDA draft is still a draft as of summer 2026.	NOT-CHECKABLE	Status statement; the copy on disk is the draft and nothing later is on disk	Every page header of the copy on disk: 'Draft — Not for Implementation'; no later status on disk
82	HOST	The ICH implementation guide (Points to Consider) dates from 2011.	PARTIAL	Reference number EMA/CHMP/ICH/902964/2011; the copy on disk is the EMA transmission edition dated February 2012	Header: 'February 2012, EMA/CHMP/ICH/902964/2011' (EMA transmission edition)

All claims, by document

EMA-2024-09-AI-Medicinal-Product-Lifecycle-Reflection-Paper.pdf

Line	Speaker	Claim	Verdict	Evidence
9	SARAH	EMA's reflection paper on AI across the medicinal product lifecycle is final since September 2024.	SUPPORTED	Cover: '9 September 2024 … Final version adopted by CHMP 9 September 2024'
49	SARAH	EMA's reflection paper uses the terms high patient risk (systems affecting patient safety) and high regulatory impact (substantial impact on regulatory decision-making), rated separately.	SUPPORTED	EMA 2.2 lines 157-159: 'high patient risk for systems affecting patient safety … high regulatory impact … where impact on regulatory decision-making is substantial'
49	SARAH	EMA's paper says the level of scrutiny and the expectation of early interaction depend on both patient risk and regulatory impact.	SUPPORTED	EMA 2.2 lines 171-173: 'The level of scrutiny depends on the level of risk and regulatory impact posed by the system'; 2.4 line 392 on timing of interactions
49	SARAH	EMA's paper says the degree of risk depends not only on the AI technology and data quality but also on the context of use and the degree of influence the AI technology exerts.	SUPPORTED	EMA 2.2 lines 163-164, verbatim
51	SARAH	EMA's paper says that if an AI system is expected to impact, even potentially, the benefit-risk balance of a medicinal product, early regulatory interaction is advised, and the level of scrutiny depends on the level of risk and regulatory impact.	SUPPORTED	EMA 2.2 lines 170-173: 'expected to impact, even potentially, on the benefit-risk balance … early regulatory interaction is advised'
51	SARAH	EMA's paper says it is the responsibility of the applicant or manufacturer to ensure that all algorithms, models, datasets and data processing pipelines are fit for purpose.	SUPPORTED	EMA 2.2 lines 175-177: 'responsibility of the clinical trial sponsor, marketing authorisation applicant/holder or manufacturer to ensure that all algorithms, models, datasets, and data processing pipelines used are fit for purpose'
51	SARAH	EMA's paper says 'these requirements may in some respects be stricter than what is considered standard practice in the field of data science'.	SUPPORTED	EMA 2.2 lines 178-180, verbatim
55	SARAH	EMA section 2.3.6 says AI in manufacturing, including process design, scale-up, optimisation, in-process control and batch release, is expected to increase.	SUPPORTED	EMA 2.3.6 lines 339-342: 'process design and scale up, process optimisation, in-process quality control and batch release is expected to increase'
55	SARAH	EMA section 2.3.6 says model development, performance assessment and lifecycle management should follow quality risk management principles considering patient safety, data integrity and product quality, and that ICH Q8, Q9 and Q10 should be considered.	SUPPORTED	EMA 2.3.6 lines 342-345 ('For human medicines the principles of ICH Q8, Q9 and Q10 should be considered')
55	SARAH	EMA section 2.3.6 uses the words 'awaiting revision of current regulatory requirements and GMP standards'.	SUPPORTED	EMA 2.3.6 lines 345-346, verbatim

FDA-2023-11-Credibility-Computational-Modeling.pdf

Line	Speaker	Claim	Verdict	Evidence
4	SARAH	Context of use as a concept came from engineering standards for computer models of medical devices.	SUPPORTED	FDA 2023, Section IV Definitions p.9: COU definition 'Reprinted with permission from ASME V&V 40-2018'; Section II: ASME V&V 40 'provides a risk-informed framework … for computational modeling of medical devices'
9	SARAH	FDA's guidance on assessing the credibility of computational modelling in medical device submissions is final and dated November 2023.	SUPPORTED	Title page: 'Document issued on November 17, 2023. The draft of this document was issued on December 23, 2021.'
21	SARAH	ASME published standard V&V 40 on the credibility of computational models for medical devices in 2018.	SUPPORTED	FDA 2023 Section II: 'ASME V&V 40-2018' throughout
21	SARAH	FDA's device centre published a final guidance in November 2023 on assessing the credibility of computational modelling in device submissions.	SUPPORTED	FDA 2023 title page: CDRH, 'Document issued on November 17, 2023'
21	SARAH	The FDA 2023 guidance glossary defines context of use as 'a statement that defines the specific role and scope of the computational model used to address the question of interest'.	SUPPORTED	FDA 2023 Section IV Definitions p.9, verbatim (footnote 19: 'Reprinted with permission from ASME V&V 40-2018')
21	SARAH	The FDA 2023 guidance says it uses key concepts of the ASME standard but provides a more general framework.	SUPPORTED	FDA 2023 Section II p.7: 'uses key concepts of ASME V&V 40-2018 but provides a more general framework', verbatim
23	SARAH	The FDA 2025 draft's definitions of context of use, model influence, decision consequence and model risk are nearly word for word those of the FDA 2023 device guidance, with 'computational model' replaced by 'AI model'.	SUPPORTED	FDA 2025 lines 176, 213, 218, 223 against FDA 2023 Section IV p.9: context of use and decision consequence identical; model influence and model risk differ only in the noun phrase

FDA-2025-01-AI-Regulatory-Decision-Making-Draft.pdf

Line	Speaker	Claim	Verdict	Evidence
4	SARAH	FDA has adopted context of use for AI in drugs and biologics.	SUPPORTED	FDA 2025, Intro lines 16-22: 'credibility of an AI model for a particular context of use (COU)'
7	SARAH	FDA's draft guidance is dated January 2025 and titled Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products.	SUPPORTED	Title page: 'Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products … DRAFT GUIDANCE … January 2025'
11	SARAH	The FDA draft lays out a seven-step framework.	SUPPORTED	IV.A lines 117-119: 'the following 7-step process'
11	SARAH	The first three steps of the FDA framework do not mention model architecture or accuracy.	SUPPORTED	IV.A.1–3 (lines 150-257) read in full: 'architecture' first appears in step 4 (line 312), 'accuracy' in step 4.a.iii (line 401)
11	SARAH	The FDA draft defines the question of interest as 'the specific question, decision, or concern being addressed by the AI model'.	SUPPORTED	IV.A.1 line 152-153, verbatim
11	SARAH	The FDA draft defines context of use as 'the specific role and scope of the AI model used to address a question of interest'.	SUPPORTED	IV.A.2 line 183-184, verbatim
11	SARAH	The FDA draft says the context of use should state whether other information will be used alongside the model output to answer the question.	SUPPORTED	IV.A.2 line 186-187: 'a statement on whether other information … will be used in conjunction with the model output to answer the question of interest'
13	SARAH	The FDA draft defines model risk as a combination of two factors, model influence and decision consequence.	SUPPORTED	IV.A.3 lines 211-217: 'Model risk is a combination of two factors: (a) model influence … and (b) decision consequence'
13	SARAH	The FDA draft defines model influence as 'the contribution of the evidence derived from the AI model relative to other contributing evidence'.	SUPPORTED	IV.A.3 line 212-216, verbatim (sentence continues 'used to inform the question of interest')
13	SARAH	The FDA draft defines decision consequence as 'the significance of an adverse outcome resulting from an incorrect decision'.	SUPPORTED	IV.A.3 line 216-220 and footnote 22, verbatim
13	SARAH	The FDA draft combines influence and consequence in a matrix shown as Figure 1, with model risk rising as either rises.	SUPPORTED	Figure 1 caption: 'The model risk moves from low to high as decision consequence or model influence increases. The ratings … are independently determined.'
15	SARAH	A footnote in the FDA draft says decision consequence is the potential outcome of the overall decision, outside the scope of the AI model and irrespective of how modelling is used.	SUPPORTED	Footnote 22: 'the potential outcome of the overall decision that is made by answering the question of interest, outside of the scope of the AI model and irrespective of how modeling is used'
17	SARAH	The FDA draft says model risk is 'the possibility that the AI model output may lead to an incorrect decision that could result in an adverse outcome' and 'not risk intrinsic to the model'.	SUPPORTED	IV.A.3 lines 223-227, verbatim
17	SARAH	In the FDA draft, data quality, drift and the model's limitations are addressed in step four (the credibility assessment plan), not as inputs to the model-risk grade.	SUPPORTED	Step 4.a.ii data fit for the COU; 4.b 'data drift'; 4.b line 466 'limitations of the modeling approach, including potential biases'
19	SARAH	The FDA draft's clinical example concerns Drug A with a life-threatening adverse reaction and an AI model to decide which trial participants can skip 24-hour inpatient monitoring.	SUPPORTED	IV.A.1 lines 155-161: 'Drug A … associated with a life-threatening drug-related adverse reaction … 24-hour inpatient monitoring'
19	SARAH	In the clinical example the draft's scope says only the AI model will be used to determine who is low risk; influence high, consequence high, model risk high.	SUPPORTED	Lines 195-197: 'only the AI model will be used to determine whether the participant is considered low risk'; lines 239-241: 'the model risk for this COU is high'
21	SARAH	ASME V&V 40 sections 2, 3 and 4 define question of interest, context of use, and model risk as influence combined with consequence.	SUPPORTED	FDA 2025 footnote 13: question of interest, COU and 'assessment of model risk … outlined in sections 2, 3, and 4 of the ASME V&V40 standard'; FDA 2023 p.17: 'following ASME V&V 40, which considers model risk as a combination of two factors, model influence and decision consequence'
23	SARAH	A footnote in the FDA 2025 draft says the high-level concepts of steps 1 through 3 were informed by ASME V&V 40 sections 2, 3 and 4, applied to AI.	SUPPORTED	FDA 2025 footnote 13: concepts of 'sections IV.A.1 through A.3 … were informed by … (ASME V&V40) … sections 2, 3, and 4'
31	SARAH	The FDA draft gives two worked examples, one of which is manufacturing.	SUPPORTED	IV.A lines 138-141: 'two examples … One example involves AI use in clinical development and the other involves AI use in manufacturing'
31	SARAH	In the manufacturing example, Drug B is a parenteral in a multidose vial and fill volume is a critical quality attribute for release.	SUPPORTED	IV.A.1 lines 166-168: 'Drug B is a parenteral injectable dispensed in a multidose vial. The volume is a critical quality attribute for the release'
31	SARAH	The manufacturer proposes an AI-based visual analysis system doing 100 percent automated assessment of fill level to identify deviations.	SUPPORTED	IV.A.1 lines 168-170, verbatim
31	SARAH	The example's question of interest is, in the draft's words, 'Do vials of Drug B meet established fill volume specifications?'	SUPPORTED	IV.A.1 lines 170-171, verbatim
33	SARAH	In the example the model analyses images of vials to determine whether a volume deviation has occurred, and independent verification on a representative sample of each batch means the model is not the sole determinant for release.	SUPPORTED	IV.A.2 lines 201-206, verbatim
35	SARAH	The draft says releasing vials that do not meet fill volume could lead to medication errors, mentioning inability to withdraw the labelled content or pooling vials to make a dose.	SUPPORTED	IV.A.3 lines 244-247: 'inability to withdraw labeled content or pooling of vials to obtain a single dose'
35	SARAH	In the example decision consequence is high, model influence is low because release testing measures fill volume on a sample, and model risk is medium.	SUPPORTED	IV.A.3 lines 247-253: consequence high, influence low, 'the model risk for this COU is medium'
39	SARAH	In step seven, when credibility is not sufficiently established for the model risk, the draft lists five outcomes, the first being that the sponsor may downgrade the model influence by incorporating additional types of evidence.	SUPPORTED	IV.A.7 lines 500-508: outcomes (1)–(5); '(1) the sponsor may downgrade the model influence by incorporating additional types of evidence'
41	SARAH	Step four is to develop a credibility assessment plan describing the model, the data and training, and how the model will be evaluated.	SUPPORTED	IV.A.4 lines 262-263 and subsection headings a.i–iii, b
41	SARAH	Step five is to execute the plan, ideally after discussing it with FDA.	SUPPORTED	IV.A.5 lines 474-477: 'discussing the plan with FDA prior to execution may help'
41	SARAH	Step six is to document results in a credibility assessment report including deviations from the plan; the report may go into a submission or be held and made available on request, for instance during an inspection.	SUPPORTED	IV.A.6 lines 486-495: 'credibility assessment report … any deviations … (2) held and made available to FDA on request (e.g., during an inspection)'
41	SARAH	Step seven is to determine whether the model is adequate for its context of use, with five outcomes if it is not.	SUPPORTED	IV.A.7 lines 497-508
43	SARAH	The draft says the two examples do not extend beyond step three because the step-four activities are a general list and the appropriate activities depend on programme specifics a hypothetical cannot capture.	SUPPORTED	IV.A lines 141-145: 'do not extend beyond step 3 because the credibility assessment activities listed in step 4 are intended to provide a general list'
45	SARAH	Early engagement is its own section of the FDA draft.	SUPPORTED	Section IV.C 'Early Engagement' (line 569)
45	SARAH	For manufacturing, the draft names CDER's Emerging Technology Program and CBER's Advanced Technologies Team as early-engagement routes.	SUPPORTED	Table 1, IV.C p.19: 'CDER's Emerging Technology Program (ETP) and CBER's Advanced Technologies Team (CATT)'
45	SARAH	The draft says early engagement is highly encouraged before submitting a regulatory application or implementing an AI technology for drug or biological product manufacturing.	SUPPORTED	Table 1, p.19: 'Early engagement with the ETP or CATT is highly encouraged before submitting a regulatory application or implementing an AI technology for drug or biological product manufacturing'
47	SARAH	A footnote on the manufacturing example says AI in production and process controls must be implemented in accordance with CGMP and that the quality control unit's responsibilities under 21 CFR 211.22 and 211.68 apply.	SUPPORTED	Footnote 21: 'must be implemented in accordance with current good manufacturing practice … the responsibilities of the quality control unit described in 21 CFR 211.22 and 211.68 are applicable'
60	SARAH	In step one the draft says a variety of evidentiary sources may be used to answer the question of interest, including manufacturing process validation studies, and that these sources should be stated in the context of use in step two and are relevant to model influence in step three.	SUPPORTED	IV.A.1 lines 173-178: 'manufacturing process validation studies … should be stated when describing the AI model's COU in step 2 and are relevant when determining model influence as assessed in step 3'
66	SARAH	The draft's scope carve-out excludes AI used for operational efficiencies that do not impact drug quality.	SUPPORTED	Section II lines 47-50: 'operational efficiencies … that do not impact patient safety, drug quality, or the reliability of results from a nonclinical or clinical study'
68	SARAH	In step four the draft says that if a human is in the loop, the evaluation should address the human-AI team and not the model alone.	SUPPORTED	IV.A.4.b lines 446-449: 'consider the performance of the human-AI team, rather than just the performance of the model in isolation'
73	SARAH	The draft says credibility activities should be commensurate with model risk and tailored to the specific context of use.	SUPPORTED	IV.A.3 lines 255-257: 'commensurate with the AI model risk and tailored to the specific COU'
75	HOST	The draft says to engage before implementing, for manufacturing, and the emerging technology programs exist for it.	SUPPORTED	Table 1, p.19, as above
76	SARAH	The draft defines fit-for-use data as relevant (including key data elements and sufficient data representative of the manufacturing process or operation) and reliable (accurate, complete, and traceable).	SUPPORTED	IV.A.4.a.ii lines 336-339: 'relevant (e.g., includes key data elements and … sufficient data that is representative of the manufacturing process or operation) and reliable (i.e., accurate, complete, and traceable)'
77	SARAH	The draft names the quality unit as ultimately responsible, points the life cycle plan at the site's quality system, and routes early engagement through the emerging technology programs.	SUPPORTED	Footnote 21: 'The quality control unit is ultimately responsible'; IV.B lines 552-555: 'component of the manufacturing site's pharmaceutical quality system'; Table 1 ETP/CATT
81	SARAH	The vocabulary came from ASME's device standard through FDA's 2023 device guidance into the 2025 draft, and ICH M15 arrived at the same terms.	SUPPORTED	FDA 2025 footnote 13 (ASME V&V40) and footnote 22 (cites the November 2023 device guidance); M15 2.1 lists question of interest, context of use, model influence, consequence of wrong decision, model risk

FDA-2025-01-FR-Notice-AI-Regulatory-Decision-Making-Draft.pdf

Line	Speaker	Claim	Verdict	Evidence
7	SARAH	The FDA draft is issued by CDER together with CBER, CDRH and CVM.	PARTIAL	Footnote 1: prepared by CDER 'in collaboration with' CBER, CDRH, CVM, the Oncology Center of Excellence, the Office of Inspections and Investigations, and the Office of Combination Products
7	SARAH	The FDA draft's comment docket closed in April 2025 with about 120 comments.	NOT-IN-CORPUS	Landscape row: 'docket confirmed — FDA-2024-D-4689, 119 comments, closed 7 Apr 2025'

ICH-2011-Q8Q9Q10-Points-to-Consider.pdf

Line	Speaker	Claim	Verdict	Evidence
38	HOST	The ICH Q8/Q9/Q10 Points to Consider describes parallel testing against the reference method as a control alongside models.	SUPPORTED	PtC 5.3 (p.12, line 385): 'verify the prediction accuracy of the model by parallel testing with the reference method'
82	HOST	The ICH implementation guide (Points to Consider) dates from 2011.	PARTIAL	Header: 'February 2012, EMA/CHMP/ICH/902964/2011' (EMA transmission edition)

ICH-2026-02-M15-MIDD.pdf

Line	Speaker	Claim	Verdict	Evidence
4	SARAH	ICH has adopted context of use for model-informed drug development.	SUPPORTED	M15 2.1.2 (p.7): 'The context of use should be outlined as a concise, clear, and explicit description of the role and scope of the model(s)'
9	SARAH	ICH M15 on model-informed drug development was adopted at Step 4 in January 2026.	SUPPORTED	Document history: 'Adoption … under Step 4 — 29 January 2026'
25	SARAH	ICH M15 was adopted in January 2026 and concerns model-informed drug development, where the models are mostly pharmacokinetic and exposure-response, though machine learning is named in its scope.	SUPPORTED	M15 1.2 (p.6): population PK/PD, PBPK, exposure-response … 'and artificial intelligence/machine learning'
25	SARAH	ICH M15 section 2 defines model influence as 'the intended weight of the model outcomes in decision-making considering the contribution of additional data or evidence'.	SUPPORTED	M15 2.1.3 (p.8), verbatim
25	SARAH	ICH M15 rates consequence of wrong decision on severity and likelihood together.	SUPPORTED	M15 2.1.4 (p.8): 'both the severity of potential negative effects as well as the likelihood … combined to generate a rating for consequence of wrong decision'
25	SARAH	ICH M15 says model risk combines influence and consequence, and that when the two ratings differ the risk may be driven by the more influential one.	SUPPORTED	M15 2.1.5 (p.8): 'derived by combining model influence and consequence of wrong decision … may be driven by the most influential of the two items'
27	SARAH	ICH M15 says model risk is not to be perceived as a risk intrinsic to modelling and simulation.	SUPPORTED	M15 2.1.5 (p.8): 'is not to be perceived as a risk intrinsic to MIDD or M&S'
27	SARAH	ICH M15 says model risk is key for determining the requirements for model evaluation, and that evaluation should be commensurate with the model risk.	SUPPORTED	M15 2.1.5 (p.8): 'Model risk is key for determining the requirements for model evaluation … be commensurate with the model risk'
29	SARAH	ICH M15 defines model impact as how far the modelling strategy departs from regulatory standards, distinct from model risk.	SUPPORTED	M15 2.1.6 (p.9): 'Model impact reflects the extent to which the proposed MIDD strategy varies from regulatory standards'

Not among the primary documents on disk

Line	Speaker	Claim	Verdict	Evidence
7	SARAH	The FDA draft is still a draft as of summer 2026. [FDA guidance status, not in corpus]	NOT-CHECKABLE	Every page header of the copy on disk: 'Draft — Not for Implementation'; no later status on disk

Host opinions (listed, not verified)

line 55, SARAH: The EMA clause about awaiting revision of GMP standards points at what became draft Annex 22.
line 18, HOST: The framework is usable on a plant floor because a decision can be graded in one meeting.
line 38, HOST: Keeping an orthogonal control in place is how you buy down model risk; the cheapest validation strategy is often a sample test.
line 53, SARAH: Illustrative EMA-axis ratings for a soft sensor (high patient risk, low regulatory impact) and a stability model (low patient risk, high regulatory impact).
line 56, HOST: EMA's patient-risk axis corresponds to FDA's decision consequence, and regulatory impact to FDA's scope question.
line 58, SARAH: Illustrative grading of the soft sensor (automated: high; advisory with orthogonal check: medium).
line 61, SARAH: Illustrative grading of the batch monitoring model (alert: low to medium; release support: high).
line 64, SARAH: Illustrative context of use for the agentic assistant.
line 66, SARAH: A deviation investigation is inside the FDA framework because it shapes a quality outcome.
line 68, SARAH: If the engineer signs the agent's draft, influence is high whatever the org chart says.
line 70, SARAH: Illustrative grading of the agent (medium if human review is real, high if not; EMA: high patient risk, low regulatory impact).
line 71, HOST: The agent's context of use must be written so the human is the determinant and the site can show it.
line 72, HOST: Define the context of use before touching the model.
line 74, HOST: Make quality, manufacturing science and regulatory sign the same sentence; treat orthogonal controls as a design decision.
line 75, HOST: Grade the agent high if the human review cannot be designed; fix the data under the decision.
line 77, SARAH: Quality, manufacturing science and regulatory CMC should be in the room when the sentence is written.