This is the case of the conflicting KPI, the fourth file in our Data Detectives series. Nobody in the room is lying, which is exactly what makes it such a puzzling case.

The evidence

  • Each team calculates the number from a different system, using different filters and business rules
  • The definition changed at some point without anyone recording when
  • Neither figure can be traced back to its source

The usual suspects

Finance and Sales each suspect the other, while the BI tool and the data team come in for their share of the blame. All of them have an alibi, because the data was sound in both systems. What was missing was an agreed meaning.

The root cause is that nobody owned the definition. “Active customer” meant one thing to Finance and another to Sales, and nobody had the job of deciding which was right, or of recording when it changed.

This matters even more with AI. Organizations carry definitions, relationships, rules and context behind their data, and when those aren’t captured and governed, a model asked about “active customers” will either pick a definition or improvise one. Two conflicting dashboards are an irritation. An AI answering with the wrong definition, confidently and at scale, is a much bigger problem.

A lead worth following

A leading not-for-profit authority in information security and risk management wanted predictive analytics to improve member engagement and decision-making. Its data sat in silos across various systems, which made it hard to generate meaningful insight.

Amplifi designed a two-year data strategy to build a scalable foundation. Its key elements included a cloud-based data ecosystem (Microsoft Fabric) to unify data access and governance, standardized data definitions to ensure consistency and reliability, monitoring mechanisms to manage data quality proactively, and tailored training so internal teams could manage data processes themselves. By the end, the organization could integrate predictive analytics into its daily operations. Read the not-for-profit case study.

The order matters. Standard definitions came before the analytics, because analytics built on disputed definitions only produce better-looking disputes.

Definitions also stick better when the people who use them help write them. In our work with a major financial services institution on an AI governance framework, policy terms were shaped by contributions from Governance, IT, Security, Legal and Risk teams, building shared ownership and cross-functional clarity from the outset. Roles and ownership were defined clearly too. It’s a different kind of agreement, but the principle carries over to a KPI: if Finance and Sales both help write the definition, both are far more likely to use it. Read the AI governance case study.

Your first lines of inquiry

Every good investigator starts by finding the quarrel at the heart of the case. Three moves to start:

  1. Find the quarrel. Take the one metric that causes the most arguments, write a single definition including its filters and exclusions, and name who approves any change.
  2. Follow both trails. Trace both versions of the number back to source, so you can see exactly where the logic diverges. The gap is usually smaller, and more specific, than the argument suggests.
  3. Resist the tempting suspect. Resist buying a data catalog first. Tools bought before anyone has agreed who will use them and who owns what are the ones that tend to stall.

Case closed? Almost

A KPI stops being contested the moment it has an owner, a written definition and a trail back to source. After that, the meeting can get on with the decision.

Our guide, Data Detectives: The Case of Trusted Data Foundations, follows five common cases from incident to root cause, with three practical moves for each and a self-check to test the strength of your own evidence.

Download the guide

If you’d like help finding where your own definitions diverge, our Data Capability Assessment measures your data capabilities across six core areas and shows how ready your organization is to deliver on its data and AI ambitions. You get an evidence-based baseline, clear recommendations and a prioritized action plan, not a slide deck.

Learn more about the Data Capability Assessment

Or talk to a data expert about getting your numbers to agree.