The AI in Your Stack That Nobody Bought

Category: AI & Automation Tags: vendor AI, embedded AI, shadow AI, subprocessors, fourth-party risk, AI inventory, model concentration

The most consequential AI deployment I have ever reviewed did not come through procurement, and nobody in the company decided to make it. A collaboration platform we had assessed two years earlier shipped an AI assistant into the product mid-contract, on by default, with a new model provider quietly added to the subprocessor list to power it. No purchase order, no security review, no intake ticket, because nothing was bought. Customer content started flowing to a company that did not exist on our data-flow diagrams, and we found out from a changelog. The part that should bother you is that the vendor had not broken the contract. The contract had simply been written for a product that no longer existed.

That is the actual shape of most companies' AI estates today. The majority of the AI touching your data was never purchased as AI. It arrived as a feature inside something you already owned: an assistant in the CRM, a summarizer in the ticketing tool, a copilot in the HR platform, a transcription layer in the meeting software. Your vendor risk process is aimed at the front door, and the AI came in through the walls.

Procurement Was Never the Front Door

Vendor risk programs are built around a trigger, and the trigger is a purchase. A new tool gets an intake review, a questionnaire, a data classification, maybe a penetration test summary if someone is feeling thorough. The whole apparatus assumes that risk enters the company when money changes hands. But vendor AI does not enter that way. It enters through release notes, through a toggle that defaults to on, through a "new: ask AI anything" banner that appears one Tuesday inside a tool that cleared security review years ago. No procurement event fires, so no review fires, and the feature inherits the trust of the product it landed in.

When people say shadow AI, they usually mean employees pasting company data into consumer chatbots, and that problem is real. But the bigger channel is this one, and it is worse precisely because it looks legitimate. An employee using an unapproved tool knows, at some level, that they are around the edge of policy. A team using the AI assistant inside an approved platform has no reason to think anything changed. The tool was reviewed. The logo is familiar. Nobody experiences turning on a feature as adopting a new data processor, even when that is exactly what it is.

The Assessment Describes a Product That No Longer Exists

The deeper problem is temporal. A vendor assessment is a photograph, and vendor AI is a moving subject. Whatever you concluded about a platform at intake, the conclusion was about the product as it existed that quarter: these data flows, these subprocessors, these failure modes. When the vendor ships an AI feature mid-contract, every one of those facts can change without a single field changing in your vendor register. There is a new data flow to a model provider. There are new failure modes, because a system that summarizes and drafts inside your workflow can now be confidently wrong inside your workflow. And in the worst cases there are new rights, because somewhere in the updated terms is a sentence about using customer content to improve the services.

I have argued before that vendor reassessment cadence should be earned, not calendared, with defined events that reset the clock regardless of the schedule. A vendor launching an AI capability into a product you use is exactly such an event, and it is the one almost no reassessment trigger list includes. The annual cycle will get to that vendor eventually, months after the feature shipped and long after your data started moving. If your triggers do not include "the product materially changed what it does with our data," your cadence is measuring the calendar, not the risk.

Read the Subprocessor List Like a Register

The highest-signal document in this whole domain is one almost nobody reads: the subprocessor notification. Vendors are generally obligated to disclose the parties that process your data on their behalf, and most will notify you of additions if you subscribe to updates. When a model provider appears on that list, that is the announcement. It usually precedes or accompanies the feature launch, it is legally meaningful in a way a marketing email is not, and it names the exact party your data will now reach. Treating subprocessor updates as compliance noise, something legal skims and files, discards the best early-warning feed you have.

When the notification arrives, the questions that matter are narrow and concrete, which is exactly why generic questionnaires miss them. Is the feature on by default, and can it be disabled per tenant rather than per user? Is customer content used to train or improve models, under what definition of improve, and is the commitment contractual or just a settings page? Where does inference run, and does it change the data residency story you have given your own customers? What is retained, for how long, and is the retention the vendor's or the model provider's? I wrote in scoring the questionnaire, not the vendor that assessments fail when they measure form-filling instead of the risk actually carried. Vendor AI is where that failure gets expensive, because the standard questionnaire was written before these questions existed.

Your Fourth Party Is a Model Provider Now

Zoom out from any single vendor and a second, structural risk appears. Your vendors did not each build their own frontier model. They built their features on a small pool of foundation model providers, the same pool everyone else's vendors chose from. Which means the AI features scattered across your stack, in the CRM and the ticketing system and the meeting software, are not forty independent capabilities. They are forty doors into a shared dependency. A policy change, a regional outage, a deprecation, or an incident at one model provider propagates through vendors that have nothing else in common, and none of your vendor-level assessments will show it, because each one looks at one vendor at a time.

This is the same lesson I took from fourth-party risk and concentration: the register that lists vendors individually can be accurate on every row and still miss the only fact that matters, which is that the rows share a single point of failure one layer down. The fix is the same too. Record the model provider behind each vendor AI feature as a first-class attribute, then group by it. The concentration is invisible vendor by vendor and obvious the moment you pivot the table.

Rows in Your Register, Clauses in Your Contract

So what does governing this actually look like? Not a new committee and not a moratorium, which fails for the reasons every blanket ban fails: it converts visible usage into invisible usage. The answer is the same machinery I described in AI governance is a risk register, not a committee, extended to cover AI you did not build or buy. Every vendor AI feature that touches your data becomes a row in the AI use-case register, scored on the same rule as everything else: autonomy times the greater of data sensitivity or decision impact. A default-on meeting transcriber that touches every conversation in the company scores differently than a code-completion tool, and both belong in the same table as your internal use cases, because your auditor, your customers, and your incident commander will not care which door the AI came through.

Then move the control upstream into the paper. The contract is the one moment you have real leverage over a vendor's roadmap, so use renewals to add the clauses the last contract did not know it needed: advance notice of AI capabilities being added to the product, subprocessor additions flagged rather than buried, a no-training commitment on customer content that lives in the agreement rather than in a settings page the vendor can redesign, and defaults set to off at the tenant level until you turn them on. None of this is exotic. Vendors are hearing these asks from every serious customer now, and the ones that resist are telling you something useful about the row they occupy in your register.

The moment of scrutiny here is not hypothetical, because it is already a standard question in every enterprise sales cycle and every regulator conversation: which of your vendors run AI over our data, and do any of them train on it? There are two possible answers. One is a lookup, because the register has a row for every vendor AI feature, a score, a model provider, and a training commitment with a contract reference. The other is a scramble through forty trust portals under deadline. The difference between those answers is not effort on the day. It is whether the inventory existed before the question arrived.

We kept the collaboration platform, in the end. The assistant stayed off at the tenant level until it earned its way on: a row in the register, a score, a subprocessor review, and a no-training amendment signed at renewal. The vendor never asked permission to change the product, and that is fine. Vendors will keep shipping AI into the tools you already own, faster every quarter, and no contract will ever make them stop. The register means they do not have to ask. It means that when the changelog surprises you, it is the last surprise in the chain, not the first.