Field note
Sampling, filters, and the reports leadership still quotes
A regional manager in Selangor had been reporting a 22 percent lift in engaged sessions after a site redesign. The exploration she used applied a filter for a campaign UTM that the redesign pages did not carry. The lift was real for that campaign. It was not a description of the redesign.
This pattern is ordinary. Analytics applications make it easy to save an exploration, pin it, and forget the filter. Sampling then quietly changes the figure when the date range grows. A calculated metric that once excluded internal traffic starts including a new office IP.
Our reporting-integrity work begins with the slide, not the property. We ask for the last three packs leadership saw. We open the exact report, note the date range, comparison, filters, and sampling notice, and rebuild the number from an unfiltered extract. Where we cannot rebuild it, we say so.
The register we leave behind is unglamorous: report name, owner, last used, verdict. Keep, fix, or withdraw. The 22 percent slide was withdrawn. A slower, unfiltered sessions chart took its place. The redesign is now discussed with a number that describes the redesign.
If a similar mismatch is sitting on your desk, request a briefing.