AI in Biopharma Manufacturing: Are We There Yet?
This is the script the AI voices read, so it matches the audio word for word; times are from the render. Researched, scripted and voiced by AI systems under Jack Prior's direction. Sam and Sarah are AI characters; nothing in the episode is Jack speaking, and none of it is a statement of his views or his employer's. Generative AI can be confidently wrong — check the sources. Corrections: jack@jackprior.ai.
00:00 Narrator This is Are We There Yet — a podcast on the evolution of AI in biopharma manufacturing, directed by Jack Prior. A word about how it's made. This episode was researched, scripted and voiced by AI systems. Jack sets the questions and the frames; the AI reads the documents and writes the conversation you're about to hear, between two AI characters: Sam, who plays a manufacturing-science practitioner, and Sarah, who has read the documents. Nothing you hear is Jack speaking, and none of it is a statement of his views or his employer's. Like any generative AI output, it can be wrong — confidently wrong, or missing a nuance — which are exactly the risks this industry is working to mitigate, and exactly what this season is about. Check the sources before you rely on anything. Corrections are welcome at jackprior dot A I. Now, the episode.
00:53 Sam Tuesday morning, and the fill-volume vision system's reject rate has fallen by a third overnight. Same line, same product, same lighting. QA opens an investigation and finds nothing in change control, because nothing at the site changed. The vendor pushed a new model version through the maintenance connection at two a.m. Their release notes call it a performance improvement.
01:15 Sarah And the same week?
01:17 Sam Same week, regulatory gets an email from the company that provides the language model under the deviation assistant. The version the site validated in the spring is deprecated in ninety days; please migrate. Two changes to two validated systems, neither initiated by the site, and the question on the table is the oldest one in GMP: what does the change control say, and what does the filing say?
01:41 Sarah Tonight's frame, Sam. Change. What happens when the model, or the process, changes: post-approval change management protocols, predetermined change control plans, Established Conditions, and the vendor update nobody planned for.
01:56 Sam Let's read.
01:57 Sam Sarah, the question and the documents.
01:59 Sarah When the model, or the process around it, changes, which instrument governs the change and what does the regulator need to hear? Eight documents, three layers. From harmonisation: ICH Q twelve, tonight's anchor; ICH Q ten, whose change management system everything else assumes; and ICH's October twenty twenty-five reflection paper on advanced manufacturing. From regulator guidance: the lifecycle section of FDA's January twenty twenty-five draft; FDA's device-side guidance on predetermined change control plans; and CDER's twenty twenty-three discussion paper. From the EU's binding track: the draft revision of Annex Eleven, and draft Annex Twenty-two's operation clause.
02:45 Sam Provenance and maturity of the anchor.
02:47 Sarah ICH Q twelve, Technical and Regulatory Considerations for Pharmaceutical Product Lifecycle Management. An ICH harmonised guideline, adopted at Step Four on the twentieth of November twenty nineteen; FDA published it as guidance in May twenty twenty-one. It is final, it is the post-approval change framework in every ICH region, and it never mentions a model as a thing that changes. Three of its tools: risk-based reporting categories, Established Conditions, and the post-approval change management protocol, the PACMP. Tonight's job is to see how far those reach when the thing changing is a model.
03:27 Sam Why now, briefly, because episode one said why this season exists. The soft sensor and the batch model have changed slowly for twenty years, by human hands, on a schedule, with a change request. The two systems in the cold open change under you. The vendor's release notes are now your change record, and the language model under the agent has a lifecycle that belongs to a company that has never heard of your site. The rulebook was written for changes a site initiates. Nobody wrote the chapter for changes a site receives.
04:01 Sarah And last episode ended on the signature: a person signs, and could that person have known the model was wrong. Tonight asks what happens when what they signed for changes underneath them.
04:12 Sam Start by splitting the word change, because the documents disagree mostly because they picture different changes.
04:19 Sarah Four kinds. First, the environment changes and the model does not: a new probe, a new raw material source, a new lighting fixture. Draft Annex Twenty-two's operation clause, ten point one, is the only GMP text that names this: any change to the model, the system, or the process, quoting, including any change to physical objects the model is using as input. FDA's draft names it too: changes in manufacturing that may impact the performance of the AI model. The model is untouched; its inputs have drifted from what it learned.
04:59 Sam Call that environment change. Second?
05:01 Sarah Second, the owner changes the model on purpose: retraining, recalibrating, moving a threshold, changing the architecture. FDA's draft calls these new signals requiring manual changes in the model; the device guidance calls them manual modifications. And BioPhorum's June twenty twenty-six guidance, from last episode, adds one nobody else lists: a change to the level of autonomy or the human-oversight configuration is a model change.
05:32 Sam Which follows from episode seven: if the oversight setting is worth a grade, changing it is a change. Third.
05:39 Sarah Third, the model changes itself. FDA's draft says AI models can be self-evolving, capable of autonomously adapting without any human intervention, and tells sponsors to anticipate inherent, model-directed changes. The device guidance calls it continuous learning and writes a plan for it. Draft Annex Twenty-two calls these dynamic models and says they should not be used in critical GMP applications; EMA's reflection paper, from episode four, will not accept incremental learning in a pivotal trial.
06:14 Sam Episode four's divergence, in change-control terms. And fourth.
06:18 Sarah Fourth, a third party changes the model, or the platform under it, outside your change control: a vendor's new version, a foundation model deprecated, a cloud runtime updated. No document tonight treats that as a category. The nearest hook is the Annex Eleven revision's periodic review of vendor contracts and service level agreements. I'll come back to it, because it is the gap.
06:43 Sam Environment, deliberate, self-directed, third-party. Hold those four. Now the instruments, inside out, starting with the ones every site already has.
06:53 Sarah ICH Q ten, the pharmaceutical quality system guideline, final since two thousand eight. Section three point two point three, change management, in four parts. Quality risk management evaluates proposed changes, with formality commensurate with risk. Changes are evaluated against the marketing authorisation, and, quoting, there should be an assessment to determine whether a change to the regulatory filing is required. Expert teams from development, manufacturing, quality and regulatory evaluate the change against criteria set in advance. After implementation, an evaluation confirms the objectives were met with no deleterious impact on quality. EU Annex Fifteen carries the same expectation into GMP.
07:46 Sam And the sentence about design space.
07:48 Sarah Working within the design space is not considered a change from a regulatory filing perspective; however, from a quality system standpoint, all changes should be evaluated by the change management system. Two verdicts for one change, the filing's and the PQS's. That split is the architecture of tonight, and it dates from two thousand eight.
08:10 Sam Annex Eleven.
08:11 Sarah The Annex Eleven revision, draft for consultation July to October twenty twenty-five, on the binding track, final expected around the end of this year as I understand it. Its quality system clause: any change to a computerised system, including configuration, components, platform and operating system, is made in a controlled manner; any significant change that may impact quality, safety or data integrity is subject to re-qualification and validation.
08:41 Sam And its periodic review.
08:43 Sarah Chapter fourteen: does the system remain fit for intended use and in a validated state, or is re-validation, complete or in parts, required. The scope includes changes to components, configuration, platform and infrastructure; quoting, the combined effect of multiple changes in this, and in other systems; undocumented changes found by configuration auditing; vendor contracts and performance indicators; and changes to regulatory requirements.
09:12 Sam Combined effect of multiple changes is the line I'd underline. Five small retrains and two sensor swaps are one change nobody assessed. Annex Twenty-two.
09:22 Sarah Draft Annex Twenty-two, same consultation, same track, clause ten. Ten point one: a tested model, the system it runs in, and the whole process it automates or assists, under change control before deployment; any change to any of them, including the physical inputs, documented and evaluated to determine if the model needs to be retested; and, quoting, any decision not to conduct such retest should be fully justified. Ten point two: configuration control, with effective measures to detect any unauthorised change. Ten point three: performance monitored against its metrics, the example being a change of lighting. Ten point four: monitor whether inputs are still within the model sample space, with metrics for drift.
10:12 Sam Read that against the cold open. The vendor's push is a change to the model. Ten point two says you should have detected it. Ten point one says retest, or write down why not. The annex doesn't know the vendor exists, and it still catches the vendor. That's the PQS layer: Q ten says evaluate everything and ask whether the filing moves; Annex Eleven says review the pile; Annex Twenty-two says model, system, process and inputs, retest or justify. Now the filing.
10:42 Sarah Q twelve's first tool is the reporting category. Chapter two describes three levels. Prior approval: changes with sufficient risk to require review and approval before implementation. Notification: moderate to low risk changes communicated formally, before or after implementation according to regional rules. And a third level in one sentence: changes that are not required to be reported to regulators are only managed and documented within the PQS, but may be verified during routine or other inspection.
11:16 Sam So PQS-only is a reporting category, not the absence of one. The inspector can still open the file.
11:22 Sarah Yes. The second tool is the Established Condition, chapter three. One line: Established Conditions are legally binding information considered necessary to assure product quality; as a consequence, any change to an EC necessitates a submission to the regulatory authority. Everything else in the dossier is supportive information. The company proposes which elements are ECs and the reporting category for each, with a justification, and the regulator approves.
11:54 Sam How does Q twelve decide what's an EC for a process?
11:57 Sarah A decision tree. Does the parameter need to be controlled to ensure product quality? If yes, it is an EC, and the risk if it changes sets the category: high, prior approval; moderate or low, notification. If no, not an EC, not reported. Then two approaches. Parameter-based, registering inputs and outputs. And performance-based, where, quoting, ECs could be primarily focused on control of process outputs rather than process inputs, enabled by a data-rich environment and an enhanced control strategy, its examples being models and Process Analytical Technology, with, quoting again, feedback controls or optimisation algorithms to achieve the relevant targets. And a warning: the enhanced control strategy may remove the need for certain process parameters to be ECs.
12:54 Sam Stop there, because that's the soft sensor's future in a paragraph. A performance-based EC is a promise about the output, and the thing keeping the promise is the model. Q twelve lets you register the output and makes the model an enabler. It does not say the model itself is an EC. Which is exactly the question FDA asked in twenty twenty-three, and we'll get there.
13:18 Sarah Chapter four. A PACMP provides predictability and transparency: the approved protocol is an agreement between the marketing authorisation holder and the regulator. It describes the change the holder intends to make, how it would be prepared and verified including an impact assessment, and the suggested reporting category, which is, quoting, a lower reporting category and/or shortened review period as compared to a similar change without an approved PACMP. It sets specific conditions and acceptance criteria.
13:53 Sam Two steps.
13:55 Sarah Step one: submit the protocol, with the proposed changes, rationale, risk management, studies and acceptance criteria, and the proposed reporting category; the regulator approves it before execution. Step two: perform the studies; if the results meet the criteria, submit under the agreed category. Then the two sentences that make it a real instrument. If the acceptance criteria are not met, quoting, the change cannot be implemented using this approach. And if new information after approval shows an increased level of risk, the previously approved reporting category should no longer be considered appropriate.
14:37 Sam So a PACMP is a bet you place with the regulator: here's what I'll change, here's what I'll measure, here's what passing looks like; if it passes, I report it lightly; if it fails, I don't do it. Now the question I care about. A retrain is not one change. It's the same change, every quarter. Can a PACMP be written for a change that repeats?
14:59 Sarah Section four point five, types of PACMPs. First type: one or more changes for a single product. And then, quoting in full: a PACMP can also be designed to be used repeatedly to make a specified type of CMC change over the lifecycle of a product, applying the same principles. One sentence, but it says it. Its own examples of broader protocols are a stopper change across products and an analytical method change across sites.
15:28 Sam That sentence is worth the episode. The repeatable protocol is not a wish; it's been in the text since twenty nineteen. What's missing is that nobody has written one whose specified type of change is retrain the model, and Q twelve has never seen a model that changes itself. Hold that too.
15:43 Sam Checkpoint, a third in. Four kinds of change. Inside the PQS: Q ten's two verdicts, Annex Eleven's periodic review, Annex Twenty-two's retest or justify. In the filing: Q twelve's three reporting levels, Established Conditions, and a PACMP that can be written to repeat. Now the one document that ties the layers together for AI, and it's a draft.
16:08 Sarah FDA's January twenty twenty-five draft, still a draft as of this summer. Section four B, lifecycle maintenance, aimed explicitly at pharmaceutical manufacturing. Its definition: the management of changes to AI models, quoting, whether incidentally or deliberately, to ensure the model remains fit for use over the drug product life cycle for its context of use. Incidentally covers our first and third kinds; deliberately, the second. Then the change management sentence: changes to the AI model, or changes in manufacturing that may impact its performance, should be evaluated by the manufacturer's change management system within their pharmaceutical quality system, with three examples: newly available manufacturing data, new signals requiring manual changes, and model-directed changes.
17:04 Sam And when does FDA hear about it?
17:06 Sarah The impact of a change is determined by model risk and by the change in model performance; steps of the credibility assessment may need to be re-executed, including retraining and retesting. And: if the model change impacts model performance, it should be reported to the Agency in accordance with regulatory requirements; a footnote ties the notification mechanism to impact on performance and on product quality. Detailed plans, with performance metrics, the risk-based monitoring frequency, and triggers for retesting, should be available for review as a component of the manufacturing site's pharmaceutical quality system, with a summary in the marketing application for product or process-specific models, at a level of detail commensurate with model risk.
17:53 Sam So the trigger is performance, not the fact of a change; detail in the PQS, summary in the filing. And the Q twelve paragraph.
18:01 Sarah Sponsors may use Q twelve's tools, established conditions and comparability protocols, which the draft says are referred to as postapproval change management plans. Sponsors may propose, quoting, model-related elements to be considered established conditions, along with a plan to manage changes to these established conditions; and by including such plans in the application, sponsors may prospectively obtain input from the Agency, including which changes would not require submission prior to making modifications.
18:33 Sam That's the hinge of the episode. FDA says: make model elements Established Conditions, attach a Q twelve protocol, and the point of doing so is to learn in advance which changes don't need a submission. What it doesn't give you is a reporting category or a template. For the template, you cross the street to devices.
18:53 Sarah Marketing Submission Recommendations for a Predetermined Change Control Plan for AI-Enabled Device Software Functions. Final guidance from the device centre, with the biologics and drug centres on the cover; originally issued the fourth of December twenty twenty-four, reissued the eighteenth of August twenty twenty-five, on a statutory section Congress added in December twenty twenty-two. Scope: device software the manufacturer intends to modify over time, whether the modifications are implemented automatically by software, also known as continuous learning, manually, or both. It reaches the device constituent of a combination product and stops there.
19:35 Sam Three components.
19:36 Sarah A Description of Modifications: the specific, planned modifications, with performance specifications after each; whether each is automatic or manual; for automatic ones, quoting, boundaries or guardrails that define the range of automatic updates; the expected frequency, from annual to continuous; global or local. A Modification Protocol, with pre-defined acceptance criteria, in four parts: data management; re-training practices, including triggers such as when the quantity of new data reaches a certain size or when a drift in data is observed; performance evaluation against both the original and the last version; and update procedures, including communication to users and real-world monitoring. And an Impact Assessment: benefits and risks of each modification, how one affects another, and their cumulative impact.
20:33 Sam And what you get for it.
20:35 Sarah An authorised PCCP is, quoting, a technological characteristic of the authorized device. Modifications specified in it and implemented per the protocol can be made without triggering the need for a new marketing submission. Two guard-rails. If there is an unresolvable failure in performance evaluation, the failure is recorded and the modification is not implemented. And a deviation from the PCCP, its examples being data management or re-training failure, or failure to meet pre-specified performance criteria, would generally make the device adulterated and misbranded. Every modification still goes through the quality system. One more line, from its list of eligible modifications: changes to device inputs and compatibility, including different makes of acquisition system, updated operating systems, or, quoting, updated cloud infrastructure.
21:32 Sam The device world already counts a cloud update as a model change worth planning for. Keep that for the fourth kind. Now put the two side by side, because these are cousins and nobody has introduced them.
21:43 Sarah The Description of Modifications matches Q twelve's detailed description of the change with a rationale. The Modification Protocol matches Q twelve's tests and studies with acceptance criteria. The Impact Assessment matches Q twelve's risk assessment. Both say the same about failure: criteria not met, change not implemented. Both keep every change inside the quality system. Two differences. A PCCP is a standing authorisation attached to the product; a PACMP is a protocol, approved, executed, reported. And the PCCP covers automatic modifications in so many words; Q twelve is silent on a change the product makes to itself.
22:30 Sam And the first difference is smaller than it looks, because four point five lets a PACMP be written for a specified type of change used repeatedly. So the device guidance supplies the structure, Q twelve supplies the permission, and the gap is only the automatic case, which Annex Twenty-two forbids in critical use anyway.
22:51 Sarah So what would you write?
22:53 Sam The character's synthesis. An AI PACMP whose Description of Modifications is the retraining envelope: which data may feed a retrain; which changes are allowed, new weights, a threshold within a stated range; which are prohibited, a new architecture, a new intended use, a new input variable; and the input-space bounds the model may operate in. Whose protocol borrows the device guidance's four parts, with episode five's independent test set and Annex Twenty-two's no-decrease comparison. And which says what happens on a failed retrain: revert, quarantine the outputs since, investigate. Executed under the agreed category, each time.
23:34 Sarah What the documents support in that: Q twelve four point one, four point two and four point five; FDA's draft on Established Conditions and prospective input; the device guidance's three components and its failure clause. What none of them says is that a retrain is the specified type of change. That is the step you're adding.
23:55 Sarah The two questions the documents leave open were asked by FDA itself. The March twenty twenty-three discussion paper on AI in drug manufacturing, from CDER's quality office, area five, continuously learning AI systems. Models in manufacturing, it says, are updated through the change control process within the pharmaceutical quality system. Then, quoting: it may be challenging to determine when an AI model can be considered an established condition of a process. And: it may also be challenging to determine the criteria for regulatory notification of changes to the model. It asks how FDA will examine continuously updated AI control models during an inspection, and about product comparability after changes introduced by the model, especially for biologics.
24:45 Sam Three and a half years ago. And the harmonisation body?
24:48 Sarah ICH's reflection paper on advanced pharmaceutical manufacturing, endorsed by the ICH Assembly on the eighth of October twenty twenty-five. On process models: there is a need for global harmonisation on terminology, model risk framework, basis for oversight and data requirements. Then, quoting: manufacturers require guidance on regulatory notification of model updates considering model risk and maturity of a site's quality system. The Points to Consider, it says, does not address linking model risk to validation and lifecycle management, and did not explicitly foresee these new types of AI models. A new ICH guideline on process models could provide a comprehensive framework. No topic has been adopted yet.
25:35 Sam So the harmonisers have written the gap down and not filled it. Until they do, here's the rule I'd run, using episode three's four tiers, as the character's synthesis. Tier one, advisory: the model lives entirely in the PQS, under Annex Eleven change control and Annex Twenty-two clause ten, never an Established Condition; a change is PQS-only, verifiable at inspection. Tier two, contributing control: the model supports an EC, an alert limit, a feed-forward set point; changes that move the registered output are a notification, or PQS-only if justified; FDA's lifecycle summary goes in the application; and this is where the AI PACMP earns its keep, so routine retrains run under a pre-agreed category.
26:28 Sarah And tier three?
26:29 Sam Determinative: the model is the EC. Treat it like a registered analytical method: performance-affecting changes are prior approval or a PACMP with comparability to the reference method, and parallel testing against that reference through the lifecycle, which the twenty eleven Points to Consider already suggested.
26:48 Sarah One precedent for the middle tier that no drug-side document cites: FDA's Computer Software Assurance guidance for devices, final September twenty twenty-five, updated February twenty twenty-six. For devices under a premarket approval, changes to the manufacturing method, production software included, that do not affect safety or effectiveness go in an annual report; changes that do go in a thirty-day notice; its example is a manufacturing execution system. A working notification tier for manufacturing software, sitting in the device rulebook.
27:24 Sam Second checkpoint, two-thirds. Four kinds; the PQS layer; Q twelve's ECs and a repeatable PACMP; FDA's lifecycle plan and its Q twelve hook; the device PCCP; the AI PACMP sketch; two open questions and a tier rule. Now the fourth kind, the one that opened the show.
27:43 Sarah No instrument tonight treats a supplier's update to a model or a foundation model as a change to the user's validated state. What exists is oblique. The Annex Eleven revision's principle two point six: when using outsourced activities, the regulated user remains fully responsible for adherence to the requirements and for the evidence. Its periodic review: support contracts, service level agreements, and vendor performance indicators. Draft Annex Twenty-two's two point two: documentation should be available and reviewed by the regulated user irrespective of whether a model is trained, validated and tested in-house or provided by a supplier. And from the device side, the software assurance guidance's list of what to review about a vendor.
28:34 Sam Which adds up to: you are responsible, review the contract once a year, and read the vendor's paperwork. None of that stops a push at two in the morning. So the minimum practice today, as I'd write it. One: pin the version; if the system can receive a model you haven't tested, Annex Twenty-two's configuration control clause is already unmet. Two: put notice of model changes in the contract, with a lead time and the right to stay on the validated version; and ask whether the version string means anything, because the silent update behind the same name is the one to fear.
29:11 Sarah Three?
29:12 Sam Three: keep a regression test set, episode five's independent set, and run it on every vendor version before it touches production; that run is also your answer to ten point one, because you did retest. Four: for the agent, the model version is part of the intended use. A different foundation model is a different system.
29:33 Sam And none of this is new to a plant, which is the part that reassures me. We've managed supplier changes to raw materials for thirty years. The film supplier changes an additive in the resin of a single-use bag. The media vendor changes a sub-component that comes from their own supplier, two tiers down. Sometimes it matters — a leachable that dents cell growth, a lot that behaves differently in the bioreactor — and mostly it's a change notification that lands in change control, somebody assesses it, and checks the box. Both failure modes are real: the change you never heard about, and the burden of assessing a hundred notifications that don't matter, which is how the one that does slips through. The foundation model under the agent is a raw material from a supplier who has a sub-supplier. Treat it like one: a notification clause, an impact assessment tiered by what could actually change, a real test for the ones that could matter — the regression set — and a documented "no impact" for the rest.
30:33 Sarah The documents support the analogy better than they support the model. Supplier and material change is an established discipline: Q10's change management system, the Annex Eleven revision's principle that the regulated user stays fully responsible for outsourced activities, its periodic review of vendor contracts, and its chapter on supplier and service management. What the material world adds, and the AI documents lack, is the habit of tiering notifications by potential impact so the assessment effort goes where the risk is. The gap is contractual. A resin change arrives with a certificate and a notification because the supply agreement says it must; a model version arrives with release notes because the vendor felt like writing them. The contract is the instrument — and few quality agreements with a model vendor say that yet.
31:25 Sarah The device guidance would call each of those a modification to device inputs and compatibility, and put it in the plan. The drug side has no plan to put it in. Two asymmetries to design around. First, retest and reporting use different tests. Annex Twenty-two requires a justification whenever you do not retest after any change. Q twelve requires a submission only when an Established Condition changes, and FDA's draft a report only when the change impacts model performance. Every change produces two decisions with two triggers, and nothing says they must rest on the same evidence.
32:07 Sam Make them. The lifecycle plan FDA asks for, the metrics, the monitoring frequency, the retest triggers, is the one evidence base both decisions should cite. One record, two verdicts, the way Q ten always had it.
32:20 Sarah Second, the transatlantic split on the third kind. Annex Twenty-two removes model-directed change from critical use by prohibition; FDA manages it by anticipation and monitoring. A global site with an EU inspector will not run a self-updating model in a critical step. It will freeze the model and retrain on a schedule, which converts the third kind of change into the second.
32:45 Sam And forfeits the benefit that justified the adaptive model in the first place. Not a criticism of either regulator; it's the practical consequence, and the sentence to put in front of anyone proposing continuous learning for a critical step in a plant that ships to Europe. Now the four systems.
33:05 Sarah The bioreactor soft sensor. A probe is replaced: environment change. Ten point one: documented, evaluated, retested or justified; ten point four's input monitoring should show whether the new probe reads inside the sample space. If the sensor's alert limit is a registered Established Condition and the replacement moves it, a notification. A quarterly retrain: deliberate change. Today, a PQS change plus, if the output supports an EC, possibly a filing. With an AI PACMP, PQS-only inside the envelope, reported under the agreed category.
33:43 Sam The batch model.
33:44 Sarah The multivariate batch model. A new product on the line: environment change, and more, because the golden-batch set that defines normal changes with it; every batch added or excluded is an exclusion decision, which episode six said Annex Twenty-two and FDA's data integrity questions both want documented and justified. A re-architecture, say from a linear projection to a neural network: deliberate change; if the model is tier three, a real-time-release surrogate, that is prior approval, or the PACMP would have had to name it as allowed, which it shouldn't have.
34:23 Sam The vision system.
34:24 Sarah The fill-volume vision system, from the cold open. The vendor pushes a new model version: third-party change. Three questions, in order. Who noticed: ten point two says configuration control should have. What regression set ran: ten point one says retest or justify. What does the filing say: FDA's own example kept the sample fill-volume test at release, so the model is not the sole determinant; a change that does not move the registered control is PQS-only under Q twelve, but it goes in the periodic review, and if the regression run shows a change in performance, FDA's draft says report it.
35:06 Sam And the agent.
35:07 Sarah The agentic assistant. The language-model provider deprecates the version you validated: third-party change, and there is no instrument. Under draft Annex Twenty-two the assistant is non-critical use only, with a human in the loop, so there is no Established Condition and no filing; the change is entirely inside the PQS: Annex Eleven change control, re-validation in parts under fourteen point one, the human-review records from ten point five carried across. And if the assistant learns from your corrections, that is model-directed change, and it is out of critical use in the EU regardless of what any plan says.
35:49 Sam For the agent, the regression set is an adjudicated set of historical deviations, the only honest test set for text, per episode five, and you run the new model version against it before the migration. Ninety days' notice is a change window; use it.
36:05 Sarah And the sketch you promised, for the soft sensor.
36:08 Sam Its Description of Modifications, in five lines. Permitted data: released batches of this product on this train, with the exclusions procedure applied. Allowed changes: re-estimation of the model weights; recalibration of the offset; the alert threshold within a stated range. Prohibited: any change to the input variables, the model form, or the context of use. Bounds: inputs within the characterised sample space; a retrain triggered by the drift metric crossing its limit, or by a stated number of new batches. On failure: revert, quarantine, investigate. The protocol underneath is the device guidance's four parts with Annex Twenty-two's test-data rules, and the reporting category is the one you agreed in step one.
36:59 Sam So what for biomanufacturing, opinionated and labelled. For every AI use case, record three things now: its tier; the change types you expect, environment, deliberate, self-directed, third-party; and the instrument for each. One page per system; it is the page an inspector reading clause ten will ask for and the summary FDA's draft wants in the application. Make the lifecycle plan's monitoring metrics the single evidence base for the retest decision and the reporting decision. Go to your vendors, in the contract, for notice of model changes and a pinned version.
37:35 Sarah And the protocol?
37:36 Sam If you have a tier-two model you retrain on a schedule, write the AI PACMP. The template exists in the device guidance, Q twelve already allows a repeatable protocol, and an industry proposal is circulating that asks for exactly this extension. The plant that files the first one sets the expectation for everyone else.
37:57 Sarah Who has to be in the room, from the documents. Q ten's expert teams: development, manufacturing, quality, regulatory. Q twelve puts maintenance of the authorisation on the marketing authorisation holder and change management on the PQS; FDA's draft puts the detailed plan with the site and the summary with the sponsor. So regulatory CMC owns the Established Conditions and the protocol; QA owns clause ten; MSAT owns the metrics both of them cite; and whoever signs the vendor contract owns the sentence about notice.
38:33 Sam And the data point under every episode: a retest decision is only as good as the regression set, and a drift metric is only as good as the historian it reads. If the basement isn't dry, you cannot tell that the model changed, let alone whether it matters. The season's refrain, and it's mine.
38:50 Sam Three things. Sarah.
38:51 Sarah First, change is four things: the environment drifts, the owner retrains, the model learns, the vendor pushes. The documents disagree mostly because each pictures a different one, and only the fourth has no instrument at all.
39:06 Sarah Second, the instruments stack inside out. Q ten and Annex Twenty-two in the PQS: evaluate everything, retest or justify. Q twelve in the filing: Established Conditions, three reporting levels, and a PACMP that can be written to repeat. FDA's draft ties them: detail in the PQS, summary in the application, model elements as Established Conditions with a plan. The device PCCP is the template for that plan.
39:36 Sam Third, mine: the tier decides the instrument. Advisory models live in the PQS. Contributing models support an EC and want an AI PACMP for retrains. Determinative models are the EC and are handled like a registered method. For the vendor: pin, contract, regress. Nobody has written the AI PACMP; the sentence permitting it is in Q twelve, and the plant that writes it first sets the bar.
40:03 Sam Next time, the last episode of the season: convergence. Where is this going, and are we there yet? The joint EMA and FDA principles, ICH's reflection paper as a destination, CDER's promised manufacturing guidance, the sandboxes, and the open questions as the season found them. Then the verdict. We know what changes and who to tell. Next we ask whether the rulebook is arriving faster than the plants can read it.