When the numbers don't match
A weekly leadership report exposed a material difference between two related performance measures. I traced the variance back to transaction population logic rather than forcing the totals to agree.
Signal
Two measures that should have reconciled did not.
The weekly reporting package compared current performance with prior periods and other trusted reporting. During a period when specific transaction types were not supposed to affect the reporting population, two related measures should have aligned.
They did not, so I treated the difference as a population problem first rather than a formula problem.
Investigation
I worked through the populations one by one.
I compared the treatment of test activity, returns, exchanges, cancellations and other reporting populations across the two methodologies. That narrowed the difference to roughly $150K of exchange-related activity.
The issue was upstream: those transactions were being included contrary to the business rule that applied to that reporting period.
Resolution
The source issue was corrected, then the methodology was clarified.
I documented the finding and worked with the data team responsible for the upstream logic. After the correction, I standardized how the reporting methodology treated the relevant transaction populations and kept the distinction between the two related measures explicit.
What changed
The reconciliation became a definition, not a one-off fix.
The work gave the reporting process a clearer set of rules for which records belong in each measure and a repeatable way to investigate future differences.