Real-Time Anomaly Detection for Web Analytics

The expensive failures in web analytics are rarely subtle. A tag gets removed in a deploy and conversions stop recording. A consent banner update blocks the script for a third of the audience. A robots.txt change takes a section out of the index. Each of those is obvious in hindsight and invisible for days, because nobody looks at a dashboard on Saturday.

Anomaly detection is the practice of letting the data raise its hand. It is not machine learning for its own sake — at small scale it is a handful of rules applied to a handful of metrics.

What the tools already do

Both major platforms ship with detection built in, and it helps to know what they are actually doing.

Google Analytics 4 applies a Bayesian model to a metric’s history, forms an expected range for the current period, and flags points outside it. Training windows differ by granularity — daily anomalies use several weeks of history, hourly ones a shorter window. Detection appears in the Insights area and can be sent by email.

Adobe Analytics uses a similar idea with a selectable statistical method and exposes the expected range in the report itself, so a spike shows up against a shaded corridor rather than as a bare label.

Self-hosted options generally do not include detection, which is not a disadvantage if you have direct access to the database — a query on a schedule is easier to reason about than a black box.

All of them share two weaknesses. They know nothing about your deployments, so they cannot tell a real drop from an instrumentation change. And they compare against your own history, so a metric that has been broken for a month becomes the new normal.

The metrics worth watching

Alerting on everything guarantees that alerts get ignored. Four metrics cover most of the damage.

Notice what is missing: bounce rate, average session duration, and anything else easily moved by a single unusual referral. Alert on what breaks a business process, not on what wobbles.

Thresholds that survive contact with reality

The first version of every alerting setup fires too often, and the second fires too late. Three habits get to a workable middle.

Compare like with like. Traffic on Sunday is not traffic on Tuesday. Compare each hour or day against the same slot in previous weeks, not against yesterday.

Require duration. A single hour of zero conversions at 3 a.m. is noise on most sites. Two consecutive periods below the floor, or a rolling window, removes the majority of false alarms.

Set a volume floor. Percentage rules are meaningless on small numbers: four purchases down to two is a 50% collapse and a normal Tuesday. Suppress alerts below a minimum expected count.

Where seasonality is strong, a fixed threshold beats a clever one. A store that does a third of its revenue in December will get better results from “conversions below 30 in an hour on a weekday” than from any model trained on eleven quiet months.

Building it without a platform

If you have database access, the whole thing is a scheduled query and a message. The shape that works:

-- current hour against the same hour over the previous four weeks
WITH now_h AS (
  SELECT count(*) AS c FROM events
  WHERE name = 'purchase' AND ts >= now() - interval '1 hour'
),
base AS (
  SELECT count(*) / 4.0 AS c FROM events
  WHERE name = 'purchase'
    AND extract(dow from ts) = extract(dow from now())
    AND extract(hour from ts) = extract(hour from now())
    AND ts >= now() - interval '28 days'
)
SELECT (SELECT c FROM now_h) AS current, (SELECT c FROM base) AS expected;

Run it every fifteen minutes, compare the two numbers, and send a message when the current value falls below half the expected one and the expected one is above your floor. That is ninety per cent of the value of a commercial detector, and you can read the code.

What to check when one fires

A useful alert names the next action. The order that resolves fastest:

Is the site up? Load it from outside your network. Half of all “analytics anomalies” are an outage.

Did anything ship? Cross-reference the timestamp with your deployment log before touching analytics at all.

Is the tag firing? Open a page with the browser console and confirm the request goes out. A tag manager container published without a trigger looks exactly like a traffic drop.

Did the source change? Break the metric by channel. A drop confined to one channel is a campaign or a referrer problem, not a site problem.

Did the definition change? Renamed events, new consent categories and altered filters all produce clean cliffs in a chart.

Keep a short log of each alert and its cause. After a few months the log tells you which rule earns its place and which one only ever fires on Black Friday.

The failure this catches best

Silent tracking loss is the reason to bother. A ranking drop announces itself in Search Console eventually; a broken purchase event does not announce itself at all, and the data lost during the gap cannot be recovered. One rule — purchases at zero for two hours — pays for the whole setup the first time it fires.