Sam Tomlinson’s newsletter argues that almost every account is managed on a schedule its conversion volume cannot support. Two accounts set him off: one steering nearly seven figures of media by how busy the call centre sounds, and a $35M store paying $10k a month to change the site every week. Here is the argument as eight things you can play with. Drag, press, break.
Conversions are counts, and counts follow a Poisson distribution. Every account varies on its own before anyone touches it.
Nothing changes in this simulation. Same rate, 52 times. Set the average and watch what a stable account looks like on a dashboard.
Every pair below comes from the identical process, so the true difference is always zero. Every percentage is a number somebody could put in a report.
Below this floor, a real change and no change look exactly the same in a report.
| To see a change of | You need, per window | Which takes |
|---|
A better converting site needs far less traffic to prove a lift, and takes the same number of weeks to do it. The two effects cancel, so the time to an answer is set by how many conversions the site produces and nothing else.
Deming showed this with a funnel, a marble and a target on the floor. Correct the funnel after each miss and the scatter gets wider, not tighter.
An account manager who shifts budget every time yesterday came in light is running rule 2. A big share of the variation they're reacting to is variation they made.
The spread the system has, forever. 1.0x. This is the benchmark, and the best the system can do without being redesigned.
Every marble misses twice, once by chance and once for the previous miss. 2.0x the variance, exactly. This is the sensible, responsive, well intentioned one.
A random walk with no bound on it. It gets worse the longer it runs, never comes back, and the zoom factor keeps climbing.
A bad hour is followed by a normal hour because that's what an average is. The change takes the credit, and the loop tightens.
Set the trigger and set what the change truly does. Leave the effect on zero, which is the honest setting, and count the saves anyway.
Compare what happened after an intervention against comparable low points where nobody touched anything. Same recovery rate means your change log is a record of activity, not value.
What share of interventions fired on a number that was already inside the control limits? That percentage is the answer. Both audits need a change log with timestamps.
Two rules. A point outside the limits is a signal. Eight in a row on one side of the average is a signal. Everything else is the process breathing.
The fix isn't looking less often. It's running three different jobs off three different calendars.
| Act on a change of | Conversions per window | Review every |
|---|
Hourly is fine. Tracking failures, disapprovals, feed errors, dead budgets, broken checkouts.
These are step changes, so they're visible instantly at any volume. This is the only thing a glance can legitimately catch.
No schedule at all. New creative, new angles, new landers, new offers.
Each one is a cheap option with a long tailed payoff. Nothing here argues for slowing down. Ship fast, judge slow.
16 ÷ threshold² ÷ conversions per period.
That's a property of the account, not something to negotiate in a contract. In between, put a control chart up and leave it alone.
It does. The answer is removing the drift from the comparison, not shortening the window until you can't see anything. Randomised splits, matched geos, holdouts, year on year indexing.
Much better, through variance reduction and sequential methods. Resolution is a design problem, not a frequency problem. Sampling more often never tells you when something is off.
A stable system doesn't improve when you react to it. It improves when you change it structurally. Knowing which one you're in is the whole job.