This is the fact check of the episode as published. Each factual claim was extracted from the script and verified by an AI agent that saw only the claim and the primary document (the PDFs in the corpus). Verdicts: SUPPORTED (the document says it), PARTIAL (supported with a difference, noted), NOT-IN-CORPUS (the source is not among the primary documents on disk), NOT-CHECKABLE (an estimate or a characterisation). Opinions voiced by the hosts are listed but not verified. This report is itself AI output and can be wrong; corrections: jack@jackprior.ai.
These statements stand, but the primary documents on disk do not themselves confirm them: they come from the landscape reference (a working index whose rows are checked against the web), are estimates, or are hedged in the episode as such.
| Line | Speaker | Claim | Verdict | Why it stands | Evidence |
| 8 | SARAH | FDA published ICH Q12 as guidance in May 2021. | NOT-IN-CORPUS | Landscape/general knowledge; the ICH text confirms Step 4 in 2019 | FDA's own publication of Q12 is not in the corpus |
| 34 | SARAH | ICH Q12's decision tree: if a parameter must be controlled to ensure quality it is an EC and the risk if it changes sets the category (high prior approval; moderate or low notification); if not, not an EC and not reported. | PARTIAL | The two-step logic is in 3.2.3.1; Figure 1's box labels could not be read from the text | 3.2.3.1 text: parameters that need to be controlled 'should be considered ECs'; risk of change 'informs the reporting category … from high to low risk'; Figure 1 is an image not present in the text extract |
| 90 | HOST | An industry proposal is circulating that asks for a repeatable AI PACMP extension. | NOT-CHECKABLE | Excluded source; no content spoken | Excluded source; the script deliberately says nothing of its content |
| Line | Speaker | Claim | Verdict | Evidence |
| 18 | SARAH | No document treats a third-party model or platform change as a category; the nearest hook is the Annex 11 revision's periodic review of vendor contracts and SLAs. | SUPPORTED | 14.2 vii 'support contracts and service level agreements (SLA)'; viii 'Contracts and key performance indicators (KPI) with vendors and service providers' |
| 24 | SARAH | Annex 11 revision's quality system clause: any change to a computerised system, including configuration, components, platform and operating system, is made in a controlled manner; any significant change that may impact quality, safety or data integrity is subject to re-qualification and validation. | SUPPORTED | 3.1 ii, near-verbatim |
| 26 | SARAH | Annex 11 chapter 14 periodic review asks whether the system remains fit for intended use and in a validated state or needs re-validation in whole or in part; scope includes changes to components, configuration, platform and infrastructure, 'the combined effect of multiple changes in this, and in other systems', undocumented changes found by configuration auditing, vendor contracts and performance indicators, and changes to regulatory requirements. | SUPPORTED | 14.1 and 14.2 (i, iii, vii, viii, xii); 'the combined effect of multiple changes in this, and in other systems' verbatim (14.2 iii) |
| 68 | SARAH | Annex 11 revision principle 2.6: when using outsourced activities the regulated user remains fully responsible for adherence to requirements and for the evidence. | SUPPORTED | 2.6, verbatim substance |
| 68 | SARAH | Annex 11 periodic review includes support contracts, service level agreements and vendor performance indicators. | SUPPORTED | 14.2 vii, viii |
| 73 | SARAH | The Annex 11 revision states as a principle that the regulated user remains fully responsible for outsourced activities. | SUPPORTED | 2.6 Outsourced activities: 'When using outsourced activities, the regulated user remains fully responsible for adherence to the requirements included in this document' (p. 2) |
| 73 | SARAH | The Annex 11 revision's periodic review covers contracts with vendors and service providers. | SUPPORTED | 14.2 Scope of review, viii: 'Contracts and key performance indicators (KPI) with vendors and service providers' (p. 11) |
| 73 | SARAH | The Annex 11 revision has a chapter on supplier and service management. | SUPPORTED | Chapter 7 heading 'Supplier and Service Management' (p. 5; also in the document map, p. 1) |
| 73 | SARAH | The Annex 11 revision states the principle that the regulated user stays fully responsible for outsourced activities. | SUPPORTED | 2.6 Outsourced activities (p.2): 'When using outsourced activities, the regulated user remains fully responsible for adherence to the requirements included in this document, for maintaining the evidence for it, and for providing it for regulatory review.' |
| 73 | SARAH | The Annex 11 revision includes periodic review of vendor contracts. | SUPPORTED | 14.2 Scope of review (p.12): 'periodic reviews should include, but may not be limited to: ... viii. Contracts and key performance indicators (KPI) with vendors and service providers.' |
| 73 | SARAH | The Annex 11 revision has a chapter on supplier and service management. | SUPPORTED | Section 7 'Supplier and Service Management' (p.5), clauses 7.1-7.5: Responsibility, Audit, Oversight, Documentation availability, Contracts |
| 84 | SARAH | Annex 11 14.1 covers re-validation in parts; Annex 22 10.5 human-review records. | SUPPORTED | 14.1 're-validation (complete or in parts)'; Annex 22 10.5 human review records |
| Line | Speaker | Claim | Verdict | Evidence |
| 12 | SARAH | Annex 22 10.1 names any change to the model, the system, or the process, 'including any change to physical objects the model is using as input'. | SUPPORTED | 10.1, verbatim |
| 16 | SARAH | Annex 22 calls self-changing models dynamic and says they should not be used in critical GMP applications. | SUPPORTED | Scope lines 12-15: dynamic models 'not covered by this document, and should not be used in critical GMP applications' |
| 28 | SARAH | Annex 22 10.1: a tested model, its system and the process it automates or assists under change control before deployment; any change to any of them including physical inputs documented and evaluated for retest; 'any decision not to conduct such retest should be fully justified'. 10.2 configuration control with measures to detect unauthorised change. 10.3 performance monitored against metrics, example a change of lighting. 10.4 input sample space monitoring with drift metrics. | SUPPORTED | 10.1–10.4, verbatim phrases |
| 68 | SARAH | Annex 22 2.2: documentation available to and reviewed by the regulated user whether the model is trained, validated and tested in-house or by a supplier. | SUPPORTED | 2.2, verbatim |
| 73 | SARAH | Draft Annex 22 does not describe tiering supplier or vendor change notifications by potential impact so that assessment effort goes where the risk is. | SUPPORTED | Absence confirmed: the only supplier reference is 2.2 (documentation review 'whether the model is trained, validated and tested in-house or by a supplier'); the only change provision is 10.1 Change control, generic, with no notification or tiering |
| 73 | SARAH | The AI documents lack the habit of tiering supplier change notifications by potential impact. | SUPPORTED | Annex 22 10.1 Change control (p.5): 'Any change to the model itself, the system, or the process in which it is used ... should be documented and evaluated to determine if the model needs to be retested.' 'Supplier' occurs once (2.2); 'vendor', 'notification' and 'tier' do not occur |
| 74 | SARAH | Annex 22 requires a justification whenever a retest is not done after any change; Q12 requires a submission only when an EC changes; FDA's draft a report only when the change impacts model performance. | SUPPORTED | 10.1: 'Any decision not to conduct such retest should be fully justified' |
| Line | Speaker | Claim | Verdict | Evidence |
| 16 | SARAH | The PCCP guidance calls automatic modification continuous learning and writes a plan for it. | SUPPORTED | Scope p.5: 'implemented automatically by software, also known as continuous learning'; VI.B p.21 |
| 49 | SARAH | The PCCP guidance is titled Marketing Submission Recommendations for a Predetermined Change Control Plan for AI-Enabled Device Software Functions; from CDRH with CBER and CDER on the cover; originally issued 4 December 2024, reissued 18 August 2025; based on a statutory section Congress added in December 2022. | SUPPORTED | Cover (also names the Office of Combination Products; full title 'Artificial Intelligence-Enabled'); p.4: FDORA enacted 29 December 2022 added section 515C |
| 49 | SARAH | PCCP scope: device software the manufacturer intends to modify over time, automatic (continuous learning), manual, or both; reaches the device constituent of a combination product and stops there. | SUPPORTED | Section III Scope pp.5-6 |
| 51 | SARAH | PCCP Description of Modifications: specific planned modifications with performance specifications after each; automatic or manual; for automatic ones 'boundaries or guardrails that define the range of automatic updates'; expected frequency from annual to continuous; global or local. | SUPPORTED | VI.B p.21, verbatim; frequency 'annually … to continuously'; VI.A; global/local p.21 |
| 51 | SARAH | PCCP Modification Protocol has four parts: data management; re-training practices including triggers such as new data reaching a certain size or observed drift; performance evaluation against both the original and the last version; update procedures including user communication and real-world monitoring. Impact Assessment: benefits and risks of each modification, how one affects another, cumulative impact. | SUPPORTED | VII.A p.24 (four components); p.27 triggers and comparison to original and last version; VIII p.30 impact assessment items |
| 53 | SARAH | An authorised PCCP is 'a technological characteristic of the authorized device'; modifications specified and implemented per protocol do not trigger a new marketing submission; an unresolvable performance-evaluation failure is recorded and the modification not implemented; a deviation from the PCCP (data management or re-training failure, failure to meet criteria) would generally make the device adulterated and misbranded; every modification goes through the quality system. | SUPPORTED | IV.C p.9; p.10-12 (no new submission; deviations 'adulterated and misbranded'; quality system); p.27 unresolvable failure |
| 53 | SARAH | PCCP's eligible modifications include changes to device inputs and compatibility, such as different makes of acquisition system, updated operating systems, or 'updated cloud infrastructure'. | SUPPORTED | VI.C p.22, verbatim |
| 74 | SARAH | The PCCP guidance treats changes to device inputs and compatibility as modifications for the plan. | SUPPORTED | VI.C p.22 type (ii) |
| Line | Speaker | Claim | Verdict | Evidence |
| 12 | SARAH | FDA's draft names 'changes in manufacturing that may impact the performance of the AI model'. | SUPPORTED | IV.B lines 539-542, verbatim |
| 14 | SARAH | FDA's draft calls deliberate model changes 'new signals requiring manual changes in the model'; the PCCP guidance calls them manual modifications. | SUPPORTED | IV.B lines 542-544, verbatim; 'deliberate' is the script's gloss on IV.B line 514 'incidentally or deliberately' |
| 16 | SARAH | FDA's draft says AI models can be self-evolving, capable of autonomously adapting without any human intervention, and tells sponsors to anticipate inherent, model-directed changes. | SUPPORTED | IV.B lines 528-535, verbatim |
| 43 | SARAH | FDA IV.B lifecycle maintenance is aimed explicitly at pharmaceutical manufacturing; defines it as management of changes to AI models 'whether incidentally or deliberately, to ensure the model remains fit for use over the drug product life cycle for its context of use'; changes evaluated by the manufacturer's change management system within the PQS with three examples: newly available manufacturing data, new signals requiring manual changes, model-directed changes. | SUPPORTED | IV.B lines 513-515 ('for its COU'), 539-544; footnote 32 frames manufacturing 'as an example' |
| 45 | SARAH | FDA: the impact of a change is determined by model risk and by the change in model performance; credibility steps may need re-execution including retraining and retesting; if the change impacts performance it should be reported in accordance with regulatory requirements; a footnote ties the notification mechanism to impact on performance and product quality; detailed plans (metrics, monitoring frequency, retest triggers) in the site's PQS with a summary in the application at a level commensurate with model risk. | SUPPORTED | IV.B lines 544-557; footnote 35 |
| 47 | SARAH | FDA: sponsors may use Q12 tools, established conditions and comparability protocols 'referred to as postapproval change management plans'; may propose 'model-related elements to be considered established conditions, along with a plan to manage changes to these established conditions'; by including such plans sponsors may prospectively obtain Agency input including which changes would not require submission prior to modification. | SUPPORTED | IV.B lines 559-567, verbatim |
| 73 | SARAH | FDA's January 2025 draft on AI for regulatory decision-making does not describe tiering supplier or vendor change notifications by potential impact so that assessment effort goes where the risk is. | SUPPORTED | Absence confirmed: 'supplier', 'vendor', 'third-party' do not occur; the only risk tiering is the sponsor's own credibility assessment ('The level of oversight for a model over its life cycle should be risk-based', l. 945) |
| 73 | SARAH | The AI documents lack the habit of tiering change notifications by potential impact (FDA draft). | PARTIAL | Footnote 35 (p.17): 'The mechanism for postapproval notification of changes to models can be determined on the basis of the following two factors: (1) impact of the change on model's performance and (2) impact of the change on product quality.' 'Vendor', 'supplier' and 'tier' do not appear |
| Line | Speaker | Claim | Verdict | Evidence |
| 20 | SARAH | ICH Q10 has been final since 2008. | SUPPORTED | Cover: 'Current Step 4 version dated 4 June 2008' |
| 20 | SARAH | ICH Q10 section 3.2.3 change management: QRM evaluates proposed changes with formality commensurate with risk; changes evaluated against the marketing authorisation with 'an assessment to determine whether a change to the regulatory filing is required'; expert teams from development, manufacturing, quality and regulatory evaluate against pre-set criteria; post-implementation evaluation confirms objectives met with no deleterious impact. | SUPPORTED | 3.2.3 (a)–(d) (expert teams list also includes 'Medical') |
| 22 | SARAH | ICH Q10: working within the design space is not considered a change from a regulatory filing perspective; however, from a quality system standpoint, all changes should be evaluated by the change management system. | SUPPORTED | 3.2.3(b): 'working within the design space is not considered a change (from a regulatory filing perspective). However, from a pharmaceutical quality system standpoint, all changes should be evaluated by a company's change management system' |
| 73 | SARAH | ICH Q10 includes a change management system as an element of the pharmaceutical quality system. | SUPPORTED | 3.2 lists 'Change management system' among the PQS elements; 3.2.3 'Change Management System' |
| 73 | SARAH | ICH Q10 has a change management system. | SUPPORTED | 3.2 (p.7): 'These four elements are: Process performance and product quality monitoring system; Corrective action and preventive action (CAPA) system; Change management system; Management review'; 3.2.3 'Change Management System' |
| Line | Speaker | Claim | Verdict | Evidence |
| 8 | SARAH | ICH Q12 is titled Technical and Regulatory Considerations for Pharmaceutical Product Lifecycle Management, adopted at Step 4 on 20 November 2019. | SUPPORTED | Cover: 'Final version … Adopted on 20 November 2019'; Document History Step 4 20 November 2019 |
| 8 | SARAH | ICH Q12 never mentions a model as a thing that changes. | SUPPORTED | 'model' occurs twice (3.2.3.1 'e.g., models, Process Analytical Technology'; ch.9 kinetic models); in neither is a model the object of a change or an EC |
| 8 | SARAH | Three of ICH Q12's tools are risk-based reporting categories, Established Conditions, and the post-approval change management protocol (PACMP). | SUPPORTED | 1.3 lists eight 'regulatory tools and enablers': categorisation of changes (ch.2), ECs (ch.3), PACMP (ch.4), PLCM document (ch.5), PQS/change management (ch.6), assessment/inspection (ch.7), structured approaches (ch.8), stability data approaches (ch.9) |
| 30 | SARAH | ICH Q12 chapter 2 describes three reporting levels: prior approval (sufficient risk to require review and approval before implementation); notification (moderate to low risk, communicated formally before or after implementation per regional rules); and changes not required to be reported, 'only managed and documented within the PQS, but may be verified during routine or other inspection'. | SUPPORTED | Chapter 2 (p.9), verbatim |
| 32 | SARAH | ICH Q12 chapter 3: Established Conditions are legally binding information considered necessary to assure product quality; any change to an EC necessitates a submission; other dossier content is supportive information; the company proposes ECs and reporting categories with justification and the regulator approves. | SUPPORTED | 3.2.1, 3.2.2 ('MAH' rather than 'company'; approval 'where appropriate') |
| 34 | SARAH | ICH Q12's decision tree: if a parameter must be controlled to ensure quality it is an EC and the risk if it changes sets the category (high prior approval; moderate or low notification); if not, not an EC and not reported. | PARTIAL | 3.2.3.1 text: parameters that need to be controlled 'should be considered ECs'; risk of change 'informs the reporting category … from high to low risk'; Figure 1 is an image not present in the text extract |
| 34 | SARAH | ICH Q12 parameter-based vs performance-based approaches; performance-based: 'ECs could be primarily focused on control of process outputs rather than process inputs', enabled by a data-rich environment and enhanced control strategy, examples models and Process Analytical Technology, with 'feedback controls or optimisation algorithms to achieve the relevant targets'; the enhanced control strategy 'may remove the need for certain process parameters to be ECs'. | SUPPORTED | 3.2.3.1 (pp.12-13), three quoted fragments verbatim |
| 36 | SARAH | ICH Q12 chapter 4: a PACMP provides predictability and transparency; the approved protocol is an agreement between the MAH and the regulator; describes the intended change, how it would be prepared and verified including an impact assessment, and the suggested reporting category, 'a lower reporting category and/or shortened review period as compared to a similar change without an approved PACMP'; sets conditions and acceptance criteria. | SUPPORTED | 4.1, near-verbatim (text: 'as compared to similar change procedure without an approved PACMP') |
| 38 | SARAH | ICH Q12 PACMP two steps: submit the protocol (proposed changes, rationale, risk management, studies, acceptance criteria, proposed reporting category) for approval before execution; perform studies and submit under the agreed category if criteria are met; if acceptance criteria are not met 'the change cannot be implemented using this approach'; if new information shows increased risk 'the previously approved reporting category should no longer be considered appropriate'. | SUPPORTED | 4.2 Step 1 and Step 2; 'the change cannot be implemented using this approach' verbatim; increased-risk sentence in 4.1 |
| 40 | SARAH | ICH Q12 4.5 types of PACMPs: one or more changes for a single product; and 'a PACMP can also be designed to be used repeatedly to make a specified type of CMC change over the lifecycle of a product, applying the same principles'; examples of broader protocols are a stopper change across products and an analytical method change across sites. | SUPPORTED | 4.5, verbatim (also a third example: manufacturing site change across products) |
| 55 | SARAH | Q12 is silent on a change the product makes to itself (automatic modification). | SUPPORTED | No 'automatic', 'self', 'continuous', 'adaptive', 'retrain' in Q12; nearest is 'feedback controls or optimisation algorithms' (3.2.3.1) |
| 59 | SARAH | Q12 sections 4.1, 4.2 and 4.5 describe the PACMP definition, the two-step process and the types. | SUPPORTED | 4.1 Definition; 4.2 Application (the two steps); 4.3 Elements; 4.5 Types |
| 91 | SARAH | Q12 puts maintenance of the authorisation on the MAH and change management on the PQS. | SUPPORTED | 3.3: 'The management of all changes to, and maintenance of, the approved marketing authorisation is the responsibility of the MAH'; 1.3 and 6.1 on the PQS |