What to Ask a Vendor About Training AI on Your Patient Data

Written by:Zain QayyumZain QayyumLast Updated: Nov 13, 2026
8 min read

Almost no healthcare artificial intelligence (AI) vendor states publicly whether your patient data trains its models, and a standard business associate agreement (BAA) usually does not cover the question. Among nine vendors read on 12 November 2026, Medsender and eight competitors, none addresses model training on a public page.

That silence is why this is a question set rather than a comparison. Nobody has published enough to rank.

TL;DR

The distinction that matters is between running a model on your data and learning from it.

A standard business associate agreement governs use and disclosure, and rarely addresses training.

De-identified data falls outside the Health Insurance Portability and Accountability Act (HIPAA), so the word does a great deal of work.

A model provider handles protected health information whether or not it trains on it.

Get the answer as a contract term rather than an assurance in a demo.

Not one vendor read for this page names its model providers.

Why the Business Associate Agreement Does Not Answer This

Buyers reasonably assume the contract covers it. In most cases the contract was written before the question existed.

It Governs Use and Disclosure, Not Learning

A business associate agreement sets permitted uses, required safeguards, breach reporting duties, subcontractor obligations and what happens at termination. Whether a model learns from the data sits awkwardly across those categories rather than inside any of them, so a standard template is silent unless somebody adds a clause.

Ask for It as a Contract Term

A verbal assurance during a demo is not something you can enforce two years later when the account team has changed. If a vendor is willing to commit, it will put the commitment in writing. If it is not willing, that reluctance is itself the answer and it is worth knowing early.

Training Versus Inference, and Why the Difference Decides This

Training and inference get used interchangeably in sales conversations, and they describe very different exposures.

Inference Uses the Model

Your document goes to a model, a result comes back, and nothing is retained for learning. That is what most healthcare AI does day to day, and it is the lower-exposure pattern because the data's influence ends when the response does.

Training Changes the Model

Your data becomes part of what the system knows, and that is effectively irreversible. A vendor can truthfully say it does not sell your data while still training on it, so ask the narrower question rather than the comfortable one.

Where De-identified Does the Heavy Lifting

The word appears in nearly every answer a vendor gives on this topic, and it is doing more work than most buyers realise.

A Legitimate Category, and a Convenient One

De-identified data falls outside HIPAA, so training on it is permitted and common. That is lawful and unremarkable. The question is what method produced the de-identification and who verified that it worked.

Two Follow-Ups That Separate Answers

Ask which standard was applied, Safe Harbor or Expert Determination. Then ask who performed it and whether the determination is documented. A vendor that can answer both has thought about this. A vendor that uses the word without being able to say how is repeating something from a sales deck.

The Model Provider Is a Subprocessor Either Way

Training is not the only exposure, and focusing on it exclusively misses a simpler risk.

Processing Is Exposure Even Without Training

If protected health information reaches a third-party model provider, that provider handles protected health information regardless of whether it learns from it. Ask who the providers are and whether each is covered by a downstream agreement.

Vendors Rarely Name Them

No vendor among the nine read on 12 November 2026 names its model providers publicly. Treat unwillingness to name subprocessors as a data point rather than a disqualification, since commercial sensitivity is a real constraint, and focus on what the vendor will commit to contractually instead.

The Six Questions, and What Good Answers Look Like

Send these in writing and compare the responses side by side. The variation between vendors is larger than you would expect.

Does our patient data train your models, in any form, including de-identified

If yes, is there an opt-out, and is it enabled by default

Which model providers process our data, and is each covered by an agreement

Is your answer to the first question a contract term or a policy you can change

How long is our data retained, by you and by any provider

What happens to it, and to anything derived from it, when we leave

How to Decide What You Can Accept

Decide your own position before asking, because the questions are only useful if you know which answers you can live with. Some practices will accept training on de-identified data from a vendor that documents its method. Others will not accept it in any form. Both positions are defensible, and neither can be worked out mid-negotiation.

Then insist the agreed position lands in the contract rather than in an email. Have your own counsel review the wording, since this is a contractual question rather than a technical one, and the person best placed to judge the drafting is not the person evaluating the software. Our guide tohow HIPAA applies to AI processing of patient recordscovers the regulatory frame,vetting a healthcare AI vendorcovers the wider paperwork, andwhat happens to patient data when you leave a vendorcovers the termination side.

FAQs

Common questions about model training on patient data.

Do Healthcare AI Vendors Train Models on Patient Data?

Some do and some do not, and among nine vendors read on 12 November 2026 none stated a position publicly. The absence of a published answer is not evidence either way. It means the question has to be asked directly and the answer secured contractually.

Does a Business Associate Agreement Stop a Vendor Training on Your Data?

Usually not. Standard agreements govern permitted uses, safeguards, breach reporting and termination, and most templates predate the question entirely. Unless a specific clause addresses training, the agreement is silent on it, and silence should not be read as prohibition. Add the clause rather than assuming it is covered.

What Is the Difference Between Training and Inference?

Inference sends your data to a model and returns a result, retaining nothing for learning. Training incorporates your data into what the model knows, which is effectively permanent. Most healthcare AI performs inference day to day, but only the vendor can confirm which applies.

Can a Vendor Train on De-identified Patient Data?

Yes, lawfully, because de-identified data falls outside HIPAA once it meets the standard. The questions worth asking are which standard was applied, Safe Harbor or Expert Determination, who performed the work, and whether that determination is documented somewhere you could review if you asked for it.

Should You Ask Which AI Model a Vendor Uses?

Ask who processes the data rather than which model is used. A provider handling protected health information is a subprocessor whether or not it trains, so coverage by a downstream agreement matters more than the model name. Many vendors treat the specific model as commercially sensitive.

What Should a Contract Say About Model Training?

Whether your data trains models in any form including de-identified, whether an opt-out exists and its default state, which providers process the data, retention periods, and what happens to derived artefacts at termination. Have your own counsel review the wording before signing.

ShareLinkedInX
Zain Qayyum

Zain Qayyum

Co-Founder and Co-CEO

Zain Qayyum is co-founder and co-CEO of Medsender. His work centres on the front-office problems that decide whether a referred patient is ever seen: inbound fax, referral backlogs, and phones nobody has time to answer. He argues that healthcare's hardest problems are operational rather than technical, and speaks and writes on where AI genuinely helps a practice and where it overpromises. He is based in New York.

Related Signals