AI-Assisted Triage: Accelerating Post-Market Surveillance While Eliminating Reporting Blind Spots

The triage problem nobody planned for

Complaint handling was designed for a world where complaints arrived slowly and looked alike. That world is gone. A single high-volume product line can generate thousands of records a month, and they arrive from everywhere: call centers, field service reports, distributor emails, hospital portals, sales reps, and social media. Some are detailed. Many are two vague lines written by someone who never saw the device fail.

Every one of those records has to be assessed. Is it a complaint? Does it need investigation? Is it reportable to FDA? The timelines are not negotiable. Under 21 CFR Part 803, a manufacturer must file a Medical Device Report within 30 calendar days of becoming aware of an event that reasonably suggests a death, serious injury, or a malfunction likely to cause one if it recurs. Certain events that require remedial action to prevent an unreasonable risk of substantial harm to public health must be reported within five work days. The clock starts when any employee becomes aware of the event, not when the complaint file is finally opened.

Manual triage cannot keep up with this volume. Senior reviewers read everything, backlogs build, and consistency drifts. Two experienced reviewers can look at the same vague complaint and score it differently, and the same reviewer can score it differently on a Friday afternoon than on a Monday morning. Triage was already the weakest link in post-market surveillance before volumes exploded. AI arrived as the obvious fix.

What AI genuinely does well

Used properly, AI removes blind spots that manual triage has always had.

The clearest gain is reading free text at scale. Natural language models can process every incoming record the day it arrives, normalize a dozen formats into one structure, translate foreign-language reports, and suggest initial event codes. Nothing sits unread. For a process where the regulatory clock starts before the file is opened, that alone is worth the effort.

The second gain is memory. A reviewer reading sixty complaints a day will not connect complaint number fourteen on Tuesday with a similar complaint from another region three months ago. A model will. Pattern detection across regions, lots, and time is where algorithms beat people convincingly, and it is exactly the signal detection work that post-market surveillance regulations expect. A cluster that would take a human team a quarter to notice can surface in days.

The third gain is consistency. A model applies the same logic to the ten thousandth record as it did to the first. It does not get tired, and its judgment does not change on a Friday afternoon.

These are real blind spots, eliminated. If the story ended here, every company would automate triage tomorrow. It does not end here.

The blind spot AI can create

Human mistakes and model mistakes fail differently, and the difference matters more in complaint handling than almost anywhere else in the quality system.

When a human reviewer misjudges a complaint, the error is usually random. Different reviewers catch each other, second looks happen, and the process self-corrects. When a model misjudges a complaint, the error is systematic. If it has learned a pattern that scores a certain type of event as low priority, it will score every instance of that event as low priority, quietly, at scale, until someone notices. Nobody notices quickly, because the records the model buries are exactly the records nobody is reading anymore.

Models also drift. Products change, new failure modes appear, and users invent new language to describe them. A model trained on last year’s complaints can be blind to this year’s problem. And people drift too. Reviewers who watch an algorithm sort the queue correctly for months start trusting the queue order itself. That habit has a name, automation bias, and it means the human check weakens as the team relaxes.

Here is how this can play out. A model trained mostly on infusion pump alarm complaints learns that the word alarm usually means routine noise. Then a lot ships with a fault where the alarm fails silently while the pump keeps running. Complaints mention the silent alarm in passing, the model files each one as routine, and the cluster sits in the low-priority queue for six weeks. Nobody acted badly. The system worked exactly as trained.

This is exactly how the worst case happens. A reportable event lands in a low-priority bucket, ages past 30 days, and surfaces during a routine file review. Now the company is filing a late MDR and explaining to an investigator why its triage system hid the event. The time the tool saved is not worth that conversation.

The regulatory frame already exists

A common mistake is waiting for an AI regulation before governing these tools. The frame already exists, and FDA is already inspecting inside it.

An AI triage tool is software used within the quality management system. Under the Quality Management System Regulation, which incorporates ISO 13485:2016, the requirement in clause 4.1.6 applies: software used in the QMS must be validated before use, with rigor proportionate to the risk it carries. A tool that influences which complaints get read first, and which reportability calls get made when, carries about as much risk as QMS software can carry. Validation here is not a checkbox. It means defining what the tool is allowed to decide, testing it against records with known outcomes, and setting acceptance criteria before it touches live data.

Inspection practice makes this concrete. Under FDA’s current compliance program for device inspections, Medical Device Reporting is one of the areas reviewed in essentially every inspection, and investigators are expected to use complaint trends and reporting history to decide where to look more closely. If an investigator asks how complaints are prioritized and the answer is an algorithm, the next questions are predictable. Show me the validation. Show me how you monitor it. Show me who decided this event was not reportable.

The logic is not unique to FDA. The EU Medical Device Regulation raised post-market surveillance expectations across the board, and notified bodies ask the same questions in their own way: how was the system validated, and who owns its output.

That last question points at the audit trail. Records must show what the model suggested, what the human decided, and who owned the decision. If an override has no name attached, an auditor will write it up as a finding.

Drawing the line: automate the work, keep the judgment

The practical question is never whether to use AI in complaint handling. It is where to draw the line between the work you automate and the judgment you keep. Table 1 shows a line that holds up.

Table 1. Complaint handling tasks: safe to automate vs. human-owned

The principle behind the table is simple. The algorithm may order the queue. It may never decide what the items in the queue are.

The second half of a defensible program is monitoring, and the healthiest way to think about it is to treat the model like any other process. Sample its output: pull a monthly sample of records the model scored as low priority and have a qualified reviewer re-read them without seeing the model’s score. Track its performance: measure how reliably it flags records that were ultimately reportable, and watch that number over time. Define retraining triggers: a new product launch, a design change, a shift in complaint language, or a drop in sampling performance should each force a revalidation, the same way a process change forces one on the production floor. And set the acceptance bar before go-live, not after. If the tool must catch every known reportable case in a challenge set before it touches live data, write that down and keep the test set. None of this is exotic. Quality teams already know how to trend a process. The model is a process.

A defensible line, written down

The companies getting this right share one habit: the line between automation and judgment is written down, in a procedure, with names attached. Their validation file says what the tool does. Their monitoring data says it still works. Their records show a human owning every reportability call.

When an investigator asks about AI, that is the whole conversation. Not whether the company uses it, but whether the company can show how it knows the tool works, and who decided the event in question was not reportable. Teams that can answer both get the speed, the pattern detection, and a queue where every record is read on arrival, without inheriting the silent failure mode. Teams that cannot answer will find their blind spot eventually, usually at the worst possible moment.

References

1. U.S. Food and Drug Administration, 21 CFR Part 803, Medical Device Reporting. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-H/part-803
2. U.S. Food and Drug Administration, Medical Devices; Quality System Regulation Amendments (Quality Management System Regulation), 89 FR 7496, final rule effective February 2, 2026. https://www.fda.gov/medical-devices/postmarket-requirements-devices/quality-management-system-regulation-qmsr
3. U.S. Food and Drug Administration, Compliance Program 7382.850, Inspection of Medical Device Manufacturers, issued January 30, 2026, implemented February 2, 2026. https://www.fda.gov/media/80195/download
4. ISO 13485:2016, Medical devices. Quality management systems. Requirements for regulatory purposes, clause 4.1.6.
5. U.S. Food and Drug Administration, 21 CFR Part 11, Electronic Records; Electronic Signatures. https://www.ecfr.gov/current/title-21/chapter-I/subchapter-A/part-11

The post AI-Assisted Triage: Accelerating Post-Market Surveillance While Eliminating Reporting Blind Spots appeared first on MedTech Intelligence.

🔗 Read the full article on the original source