Before you deploy AI agents, five things need to be in place: agreed definitions for your key metrics, a named owner for each definition, visibility into the quality of the data the agent reads, clear rules on which data an agent can use, and a way to report errors. Most organisations start with the model and access. Start with the definitions. An agent inherits what you have, including the contradictions.
Why an AI agent makes your definition problem bigger
The common assumption is that AI agents are a technology project: choose a platform, set up access, launch a pilot. That is only half the story. The biggest risk lies not in the model, but in the figures it reads.
For example, your organisation has two reports showing "revenue". One includes credit notes; the other does not. Your financial controller knows this and chooses the right one. An agent does not. It picks one and gives a fluent answer, without any caveats.
In practice, people bridge these gaps using context that is never written down. An agent does not have that context. If it is not documented, it does not exist as far as the agent is concerned.
What a colleague knows and what an agent needs
| Question | How a colleague handles it | What an agent needs |
|---|---|---|
| What does this metric mean? | Knows from experience, or asks | A written definition for each KPI |
| Which version is correct? | Knows which report is certified | One agreed source per definition |
| Who decides? | Knows the financial controller by name | A named owner in the metadata |
| Is today's data reliable? | Hears about it in passing | A quality status it can read |
Which definitions need to be ready for an AI agent?
You do not need to define everything. Start with the metrics the agent will be asked about: revenue, margin, customer numbers, inventory, absenteeism. These are also the metrics your executive committee debates.
Write one sentence in plain language for each KPI. Specify which source is correct. And resolve differences between departments before the agent goes live, not afterwards. An agent that sees two definitions chooses arbitrarily.
Who owns the metric when an agent gives the wrong figure?
Someone needs to be able to say whether a definition is correct. That is the metric's owner, usually someone from the business, not IT.
Record that owner alongside the definition. If an agent gives an odd answer, you immediately know who needs to check it. Without an owner, every error ends up with the data team, which does not know the business well enough to resolve it.
Which data can an agent use?
This is the part most CIOs are already working on, and rightly so. For each agent, determine which sources it can read, whose permissions it operates under, and which data is off limits, such as personal data or confidential financial figures.
Build on what you already have: your existing access management and sensitive data classification. Ask your legal team to assess which obligations under the EU AI Act apply to your use case. For AI risk management, the NIST AI Risk Management Framework (2023) is a useful, publicly available starting point.
How do you know whether your organisation is ready?
Run a simple test. Choose a handful of questions your leadership team often asks, such as "what was the margin last quarter?". Ask two people from different departments.
Do you get the same answer, using the same source? Then that definition is ready for an agent. If you get two answers, you know where to start. An agent would produce exactly the same discrepancy, just faster.
A CIO checklist: Definitions, Owners, Quality, Access, Feedback
- Definitions: one written definition for each KPI the agent will be asked about, with one agreed source.
- Owners: a named business owner for each definition, responsible for reviewing errors.
- Quality: a quality status for each dataset, so the agent and the reader know whether today's data is fit for use.
- Access: a record of which sources each agent can read and which data is off limits.
- Feedback: an established route for reporting incorrect answers that leads to the definition's owner.
Where dScribe fits
dScribe is where you document definitions, owners and data quality before an agent needs them. The same definition your readers see in a Power BI report can also serve as the source for your AI agents. dScribe does not control which data an agent can read. That remains part of your existing access management.
Final thoughts
An AI agent is only as reliable as the definitions it reads. Choose the model first and sort out the definitions later, and you get fast answers that nobody trusts. Start with the metrics the agent will be asked about. Agree on what they mean, who decides and whether the data is reliable. Only then does an agent have something to build on.
Frequently asked questions
What does a CIO need to put in place before AI agents go live?
Agreed definitions for the key KPIs, an owner for each definition, visibility into data quality, access rules for each agent and a way to report errors. Most IT teams are already familiar with the last three. The first two are the ones most often skipped.
Is our data ready for AI agents?
Ask two people from different departments a few questions your leadership team frequently raises. If you get different answers, an agent will inherit the same confusion. Start by clearing that up.
Why does our AI assistant give different figures from our Power BI reports?
Usually because the assistant uses a different source or definition from the report. It does not know which version is certified unless you explicitly tell it. One agreed definition per KPI, shared by the report and the agent, resolves this.
Who is responsible when an AI agent uses the wrong figure?
The organisation remains responsible for the decision. In practice, each definition needs an owner who can say whether the figure is correct. Without that owner, every error ends up with IT.
Do you need a data catalogue before deploying AI agents?
You need a maintained list of definitions and owners that the agent can read. Whether it lives in a catalogue or elsewhere matters less than keeping it up to date and agreed upon.