Notes / 21 November 2025

Sampling bias in app telemetry

Usage trends are often stories about the people your SDK bothered to count. Vendor sampling, client-side rate limits, and “do not track” style OS settings do not remove a random 10% of sessions. They remove particular sessions: older devices, logged-out browsers inside a WebView, and users who open the app once from a widget and vanish.

In week five of the atlas we write a coverage footnote before we compute a delta. The footnote names the share of sessions with a stable user key, the share on tablets, and whether the peer’s public numbers (if any) could plausibly have the same holes. If you cannot estimate those shares, the delta does not go on the one-pager.

Logged-out traffic is the usual missing mass for media apps. A weekday commute reader who never signs in may be your actual habit user and your telemetry’s ghost. Comparing signed-in minutes to a peer who reports all openers is not Benchmarking App Usage Trends. It is two different populations wearing the same chart colours.

Softmatrixhub still will not ingest your raw stream. We will ask you to count keys and devices in the same week you count minutes. That extra query is less glamorous than a peer table. It is the reason a Highland studio keeps marking decks that skip it.