Which AI metrics actually tell you something, and how do you read them?

The monthly review opens with a screen full of numbers. Fourteen AI tools in use, three of them new. 312 detections of sensitive data. AI adoption: 62 of 90 workstations. The director nods, then asks the question that matters: is this good or bad? The honest answer is that none of those numbers tells you that on its own. An AI metric only gains meaning once you know what you are comparing it with.

#Which figures show how AI fits into the work?

Start with usage, because that is where the good news is. How many workstations use AI, how often (daily, weekly, occasionally) and for what: summarising, translating, writing communications, coding. Those figures show where AI has already earned its place in the work.

Read them as a question about support, not as a score. If most of the use is summarising, you know where good examples and clear rules pay off most. If one department barely uses AI, that may be a deliberate choice, or a sign that people are unsure what is allowed.

One thing these figures do not tell you is what AI delivers. Time saved or better work does not show up here. If you want to know, ask the teams.

#Which figures show whether your AI rules are working?

This is where the shares come in, and they tell you more than the totals. The most important: what proportion of AI use goes through tools you have allowed? Alongside that: how many tools are in use without a decision, and how often are tools that are not allowed being visited?

80 per cent through allowed tools sounds good, but the comparison tells the story. If it was 65 per cent last month, something is working. If it was 90, something has changed that you will want to understand, such as a new tool that caught on quickly.

With notices, the counts matter less than what happened next. Did the employee open the alternative, or carry on to the original tool? If lots of people carry on to the same tool after a notice, that often says something about the alternative: perhaps it cannot quite do what people need.

#What does the number of detections tell you?

312 detections sounds like a lot. But if usage doubled over the same period, a rise is to be expected, and even a sign that detection is doing its job. The total is mostly context.

Look at two other figures instead. What share of critical detections, such as a national ID number or bank account number, was adjusted before sending? And which data types come up most often? If customer numbers top the list, there is a specific conversation to have: perhaps customer numbers belong in a separate workflow, or in a tool you have approved for them.

Leaving a standard highlight in place is not automatically a mistake. The employee knows the context; a name in a meeting invite is not the same as a name in a medical record. The bar is higher for critical data, so that is where you look hardest.

#Which figures should you not read on their own?

Three pitfalls come up again and again.

• A count without a denominator. "Fourteen tools" says little until you know how many of them have a decision.
• Two kinds of figures added together. Seeing which tools are in use covers hundreds of services. Help with sensitive data is only available in supported apps. Blend them into one score and you lose the difference.
• A missing figure read as zero. If something was not measured, for instance because a feature was off, that is not a good result but an open question.

#How do you turn a figure into a question?

The most useful thing you can do with AI metrics is turn each one into a single follow-up question:

In BeeSensible these figures appear on the Usage, Compliance and Sensitive data screens, as counts without text and without names. This is what the sensitive data overview looks like: