AI Governance Without a Model Inventory Is Guesswork

If I asked you today to list every AI model running inside your organisation — who built it, what data it uses, what decisions it influences, and who is accountable for its output — could you answer?

In my experience, most enterprises cannot. They have AI models in production that nobody outside the original team knows about. Some are vendor-provided, embedded inside SaaS products. Some were built by data science teams that have since moved on. Some are spreadsheet macros with statistical functions that nobody thinks of as AI but that make decisions about customers.

You cannot govern what you have not catalogued. An AI governance framework without a model inventory is a policy document with no enforcement mechanism.

This piece is for CIOs, CISOs, data protection officers and heads of AI who are building or reviewing their AI governance approach.

The short version

AI governance starts with knowing what AI models you have. Build a model inventory before writing policies. The inventory is the enforcement mechanism. Without it, governance is aspirational.

• Most enterprises undercount their AI models by a factor of three to five.
• Vendor-embedded AI models count as yours for governance purposes.
• Every model entry needs an owner, a data source, an output description and a risk tier.
• The inventory must include models that influence decisions about people.
• Review the inventory quarterly; AI models change faster than policies.

This article sits inside the technology governance cluster, where the wider argument is set out in full.

Why enterprises undercount their models

When most teams think of AI models, they think of the ones the data science team built: a churn prediction model, a demand forecaster, a fraud detector. These are visible because somebody ran a project to create them.

But the actual count is higher because of three categories that fly under the radar:

Vendor-embedded models. Your CRM uses AI to score leads. Your HRMS uses AI to rank resumes. Your customer support platform uses AI to route tickets. These models were not built by your team, but they make decisions using your data, about your customers or your employees. Under DPDP and most emerging AI governance frameworks, you are responsible for those decisions.

Departmental models. A credit analyst builds a logistic regression model in Python to pre-screen loan applications. A marketing team uses a clustering model in a BI tool to segment customers. These are not in any central repository because they were never part of a formal AI programme.

Automated decision rules with statistical components. A pricing engine that adjusts based on demand patterns. A collections priority score calculated from customer attributes. These may not call themselves AI, but they make automated decisions about people, and that is the regulatory test that matters.

The first version of a model inventory is always surprising. I have seen organisations that believed they had fifteen AI models discover they had sixty, once vendor-embedded and departmental models were included.

Should your organisation do this now?

• Yes, if you have an AI governance policy but no model inventory.
• Not yet, if you have no AI models in production at all.
• Instead, first: check whether vendor-embedded AI counts in your use.
• Measure it by: number of models discovered versus initially expected.

What the inventory needs to capture

Each model entry should answer:

What does it do? A plain language description of the model's function and the business process it supports. Not the algorithm type — the business outcome.

What data does it use? Source systems, data types, whether it includes personal data, and whether it uses data that could serve as a proxy for protected attributes.

What decisions does it influence? Loan approvals, hiring shortlists, customer risk scores, pricing — and whether the model's output is advisory or automated.

Who owns it? The person accountable for the model's performance, fairness and compliance. Not the data scientist who built it — the business owner who uses its output.

What is the risk tier? Based on the consequence of the model being wrong. A product recommendation engine being wrong is low risk. A credit scoring model being wrong affects livelihoods. Tier accordingly.

When was it last validated? Models degrade over time as the data they were trained on becomes less representative. When was performance last checked against current data?

What are its known limitations? Every model has conditions under which it performs poorly. These should be documented, not discovered in production.

Risk tiering in practice

Not every AI model needs the same level of governance. A model that recommends products on an e-commerce site and a model that decides whether someone gets a loan require fundamentally different oversight.

I use three tiers:

Tier 1 — High risk. Models that make or directly influence decisions about people's access to financial services, employment, insurance, housing or essential services. These need regular bias testing, performance monitoring, explainability documentation and human oversight of edge cases.

Tier 2 — Medium risk. Models that affect business operations and customer experience but do not directly gate access to essential services. Demand forecasting, inventory optimisation, marketing segmentation. These need performance monitoring and periodic validation.

Tier 3 — Low risk. Models that support internal operations without customer-facing or people-affecting decisions. Internal search ranking, content tagging, document classification. These need basic documentation and annual review.

The tier determines the governance requirements: how often the model is reviewed, what testing is required, and what level of human oversight is needed.

Should your organisation do this now?

• Yes, if you treat all AI models with the same level of oversight, or none at all.
• Not yet, if your inventory is not yet complete.
• Instead, first: complete the inventory before assigning tiers.
• Measure it by: percentage of Tier 1 models with current bias testing and performance monitoring.

The vendor-embedded model problem

When you buy a SaaS product with embedded AI, you are deploying someone else's model on your data. This creates a governance gap that most organisations do not address.

The gap exists because:

• The vendor controls the model. You cannot inspect its training data, its architecture or its update schedule.
• The vendor updates the model without telling you. A model that worked well last month may behave differently after a silent update.
• Your data goes into the model, but you may not know how. Is it used for training? For improving the product for other customers? The contract should answer this, but often does not.
• The model's decisions are your responsibility. If a vendor's AI model makes a biased hiring recommendation using your candidate data, the regulatory obligation falls on you, not the vendor.

For every SaaS product with embedded AI, the inventory should capture the same fields as an internal model, plus three vendor-specific questions:

1. Can we audit or test the model's output? If not, how do we know it is fair? 2. Is our data used for model training? If so, under what terms? 3. How are we notified of model changes? If we are not, how do we detect performance shifts?

Before you build the inventory

Checklist:

• Named owner for the AI model inventory with cross-functional authority.
• Survey template sent to every department and product team.
• SaaS vendor list reviewed for embedded AI features.
• Risk tiering framework agreed with legal, compliance and business leads.
• Quarterly review cycle scheduled.
• Clear rule: no new AI model deployed without an inventory entry.

Questions to ask:

• Your data science team: how many models are in production, and when was each last validated?
• Your procurement team: which SaaS vendors include AI features, and what do those features do?
• Your operations teams: does anyone use a statistical model or scoring system that was not built by the data science team?
• Your legal team: which of our AI models make or influence decisions about people?
• Your vendors: is our data used to train your AI models, and how are we notified of model changes?

How to measure it

• Inventory completeness. Percentage of known AI models with full inventory entries. Baseline: first census. Owner: AI governance lead. Review: quarterly. Leading.
• Tier 1 compliance. Percentage of high-risk models with current bias testing and performance monitoring. Baseline: first assessment. Owner: AI governance lead. Review: quarterly. Leading.
• Vendor AI coverage. Percentage of SaaS vendors with embedded AI that have governance entries. Baseline: first vendor review. Owner: procurement with AI governance. Review: semi-annual. Leading.
• Model validation currency. Percentage of production models validated within the last six months. Baseline: first inventory. Owner: data science lead. Review: quarterly. Leading.
• AI-related incidents. Complaints, errors or bias concerns traced to AI model output. Baseline: last twelve months. Owner: risk management. Review: quarterly. Lagging.

Mistakes that cost the most

Governing only the models you built

Vendor-embedded models are excluded from governance because you did not build them.

• Why it happens: the governance team focuses on the data science team's output.
• Prevention: include all AI that touches your data or your customers in the inventory.
• Early warning: no SaaS vendor has been asked about their AI features.

No business owner

The data scientist is listed as the owner, but they do not own the business outcome.

• Why it happens: the model was a data science project, not a business initiative.
• Prevention: require a business owner who is accountable for the model's real-world impact.
• Early warning: the model's performance is reviewed by the data team, not the business team.

Static inventory

The inventory was built once and never updated.

• Why it happens: building the inventory was a project; maintaining it is nobody's job.
• Prevention: schedule quarterly reviews and tie them to deployment approvals.
• Early warning: new models are in production that are not in the inventory.

What to do next

Send a one-page survey to every department asking three questions: do you use any automated scoring, prediction or classification tools? Where does the data come from? What decisions does the output influence? Compile the results into your first model inventory. Then read the related piece on shadow AI as a data processor and the technology governance topic page.

Last reviewed: 14 October 2026.

Views are my own and do not represent my employer.