Skip to main content

Prediction performance by change type

Prediction performance

SN21 models predict what each Google Ads change will do before the outcome is known. When it settles, every prediction is scored — this shows how reliable they are, one row per kind of change.

How to read this page

What we measure
The change's effect on cost, conversions and the goal metric — CPA (cost per lead) for lead-generation accounts, ROAS (return on ad spend) for ecommerce.
At what level
The campaign the change belongs to — not the whole account. An ad, keyword or ad-group change is measured on its campaign's results against a 60-day baseline.
Over what window
The next 7, 14, or 28 days after the change — switch the window on the table below.
The rating Reliable lean on it Roughly right direction right, numbers less so Not reliable don't lean on it yet Too early not enough has settled yet
A rating is prediction accuracy — how close the models were — not a probability the change will succeed, and not advice to make it.

Changes settled & scored

17,248

served changes settled since 20 August 2026

Kinds of change

14

one line each, below

Models compared

170

every model predicts every change

Best model today

UID 72

leaderboard #1

Where the number comes from

63 days, 2026-08-20 to 2026-10-21
  1. 1. Captured1,127,131raw changes per day, before any rule
  2. 2. Measurable episodes52,144since 20 August 2026, after the published rules and 72-hour clustering
  3. 3. Served to models17,670of 23,587 eligible; 4,790 held out for validation, 200 on our own accounts
  4. 4. Settled & scored17,580this page — served changes old enough to have settled
  5. 5. Predictions scored4,264,976every model predicts every change

Each stage is deliberate. The page counts the fourth: a change reaches it about 15 days after it happened, once its first outcome window has closed and settled.

14 groups covering 93 change types. Open a row to see the types inside it.

Budget change — when a daily budget was raised or lowered 4,527 Reliable UID 91 6 types

Across 4,527 such changes (449,907 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

down 1,038 Reliable UID 246
down large 1,097 Reliable UID 29
flat 278 Roughly right UID 235
unknown 6 Reliable UID 254
up 839 Roughly right UID 91
up large 1,269 Roughly right UID 29
Bid strategy switch — when the bid strategy was switched from one type to another 113 Reliable UID 165 8 types

Across 113 such changes (10,799 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

maximize conversion value → maximize conversions 1 Too early —
maximize conversions → maximize conversion value 4 Too early —
maximize conversions → target spend 9 Reliable UID 63
target spend → maximize conversion value 1 Too early —
target spend → maximize conversions 6 Roughly right UID 5
switched to Maximize Conversion Value 26 Reliable UID 107
switched to Maximize Conversions 34 Reliable UID 204
unknown → target spend 32 Reliable UID 90
Bid target change — when a bid-strategy target value changed (e.g. Target CPA $60 → $50) 488 Roughly right UID 173 13 types

Across 488 such changes (46,871 model predictions), a prediction here is roughly right. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

ad-group bid change 14 Reliable UID 198
maximize conversion value · down 57 Roughly right UID 222
maximize conversion value · down large 4 Too early —
maximize conversion value · flat 11 Roughly right UID 66
switched to Maximize Conversion Value 2 Too early —
maximize conversion value · up 57 Roughly right UID 222
maximize conversion value · up large 18 Reliable UID 95
maximize conversions · down 86 Reliable UID 126
maximize conversions · down large 31 Reliable UID 124
maximize conversions · flat 17 Roughly right —
switched to Maximize Conversions 13 Roughly right UID 196
maximize conversions · up 139 Reliable —
maximize conversions · up large 39 Roughly right UID 246
Negative keyword changes — when negative keywords were added 2,694 Reliable UID 29 16 types

Across 2,694 such changes (262,012 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

negative keyword add 327 Reliable —
broad · 1 52 Roughly right UID 173
broad · 11+ 19 Roughly right UID 204
broad · 2-10 52 Roughly right UID 165
exact · 1 431 Reliable UID 126
exact · 11+ 277 Reliable UID 173
exact · 2-10 678 Reliable UID 107
mixed · 11+ 144 Reliable UID 193
mixed · 2-10 180 Reliable UID 173
phrase · 1 140 Reliable UID 162
phrase · 11+ 84 Reliable UID 235
phrase · 2-10 208 Reliable —
unknown · 1 7 Reliable —
unknown · 11+ 7 Reliable UID 162
unknown · 2-10 8 Roughly right UID 45
negative keyword remove 80 Reliable UID 239
Keyword changes — when this changed 828 Reliable — 2 types

Across 828 such changes (80,762 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

keyword add 489 Reliable UID 126
keyword remove 339 Reliable —
Other criterion changes (kind not recorded) — when this changed 1,733 Reliable UID 18 6 types

Across 1,733 such changes (164,694 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

bid on a keyword / audience / placement 436 Reliable UID 29
targeting criteria change 676 Reliable UID 95
criterion enable 84 Reliable UID 55
criterion pause 508 Reliable UID 18
criterion remove 24 Reliable UID 162
criterion url change 5 Too early UID 104
Location, audience and demographic targeting — when this changed 156 Reliable UID 12 3 types

Across 156 such changes (15,411 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

audience change 97 Reliable UID 246
demographic change 25 Reliable —
geo change 34 Reliable —
Device, schedule and placement targeting — when this changed 162 Reliable UID 173 2 types

Across 162 such changes (14,180 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Roughly right Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

device targeting change 2 Too early —
schedule change 160 Reliable UID 173
Campaign pause / enable — when this changed 878 Reliable UID 72 1 type

Across 878 such changes (90,769 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Roughly right Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Ad group pause / enable — when this changed 114 Reliable UID 141 2 types

Across 114 such changes (10,780 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup enable 19 Reliable UID 113
adgroup pause 95 Reliable UID 141
Other ad group changes — when this changed 168 Reliable — 3 types

Across 168 such changes (15,486 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup bid change 88 Roughly right UID 247
adgroup change 78 Reliable —
adgroup remove 2 Too early —
Ad changes — when this changed 218 Reliable UID 38 5 types

Across 218 such changes (20,761 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

ad create 39 Reliable UID 113
ad edit 84 Reliable UID 38
ad enable 7 Reliable UID 115
ad pause 81 Reliable UID 55
ad remove 7 Reliable —
Asset changes — when this changed 706 Reliable — 1 type

Across 706 such changes (58,724 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Combined changes — when several kinds of change happened together 4,463 Reliable UID 18 25 types

Across 4,463 such changes (434,281 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup bid change 29 Roughly right UID 107
adgroup change 7 Reliable UID 214
adgroup enable 13 Reliable —
adgroup pause 330 Reliable UID 246
ad-group bid change 1 Too early —
ad create 541 Reliable UID 246
ad edit 49 Reliable UID 222
ad enable 23 Roughly right UID 13
ad pause 10 Reliable UID 196
asset change 40 Reliable UID 52
audience change 18 Reliable UID 107
bid strategy change 37 Reliable UID 204
budget change 1,245 Reliable UID 18
campaign pause 177 Reliable —
bid on a keyword / audience / placement 37 Reliable UID 12
targeting criteria change 107 Reliable UID 199
criterion enable 33 Reliable UID 55
criterion pause 16 Reliable UID 90
criterion remove 6 Reliable UID 141
demographic change 1 Too early —
keyword add 990 Reliable —
keyword remove 112 Reliable —
negative keyword add 464 Reliable —
negative keyword remove 1 Too early —
target value change 176 Reliable UID 253

14 groups covering 90 change types. Open a row to see the types inside it.

Budget change — when a daily budget was raised or lowered 4,169 Roughly right UID 91 6 types

Across 4,169 such changes (431,581 model predictions), a prediction here is roughly right. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

down 947 Roughly right UID 87
down large 1,020 Reliable UID 218
flat 238 Roughly right —
unknown 5 Roughly right —
up 786 Roughly right UID 91
up large 1,173 Roughly right UID 91
Bid strategy switch — when the bid strategy was switched from one type to another 100 Reliable UID 126 7 types

Across 100 such changes (10,266 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

maximize conversion value → maximize conversions 1 Too early —
maximize conversions → maximize conversion value 4 Too early —
maximize conversions → target spend 10 Reliable UID 19
target spend → maximize conversions 6 Roughly right UID 5
switched to Maximize Conversion Value 23 Roughly right UID 107
switched to Maximize Conversions 29 Reliable UID 91
unknown → target spend 27 Reliable UID 87
Bid target change — when a bid-strategy target value changed (e.g. Target CPA $60 → $50) 444 Roughly right UID 165 13 types

Across 444 such changes (45,790 model predictions), a prediction here is roughly right. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

ad-group bid change 14 Roughly right UID 138
maximize conversion value · down 51 Roughly right UID 165
maximize conversion value · down large 4 Too early —
maximize conversion value · flat 10 Roughly right UID 196
switched to Maximize Conversion Value 2 Too early —
maximize conversion value · up 53 Roughly right UID 91
maximize conversion value · up large 15 Roughly right UID 238
maximize conversions · down 76 Roughly right UID 87
maximize conversions · down large 29 Roughly right UID 72
maximize conversions · flat 14 Reliable —
switched to Maximize Conversions 13 Roughly right UID 196
maximize conversions · up 132 Reliable UID 12
maximize conversions · up large 31 Roughly right UID 115
Negative keyword changes — when negative keywords were added 2,355 Roughly right — 16 types

Across 2,355 such changes (242,842 model predictions), a prediction here is roughly right. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

negative keyword add 330 Roughly right UID 245
broad · 1 38 Roughly right —
broad · 11+ 19 Roughly right UID 55
broad · 2-10 36 Roughly right UID 115
exact · 1 379 Roughly right —
exact · 11+ 237 Reliable UID 126
exact · 2-10 588 Roughly right UID 165
mixed · 11+ 120 Roughly right —
mixed · 2-10 157 Reliable UID 165
phrase · 1 116 Reliable UID 246
phrase · 11+ 70 Reliable UID 107
phrase · 2-10 178 Reliable UID 126
unknown · 1 7 Reliable UID 39
unknown · 11+ 4 Too early —
unknown · 2-10 8 Reliable UID 56
negative keyword remove 68 Roughly right —
Keyword changes — when this changed 665 Reliable — 2 types

Across 665 such changes (68,504 model predictions), a prediction here is reliable. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

keyword add 434 Reliable —
keyword remove 231 Reliable —
Other criterion changes (kind not recorded) — when this changed 1,463 Reliable UID 126 6 types

Across 1,463 such changes (150,708 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

bid on a keyword / audience / placement 381 Reliable UID 113
targeting criteria change 605 Reliable UID 107
criterion enable 74 Reliable —
criterion pause 375 Reliable UID 126
criterion remove 23 Reliable UID 95
criterion url change 5 Too early —
Location, audience and demographic targeting — when this changed 114 Reliable UID 91 3 types

Across 114 such changes (11,798 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

audience change 58 Reliable UID 141
demographic change 24 Reliable —
geo change 32 Reliable UID 95
Device, schedule and placement targeting — when this changed 109 Reliable — 1 type

Across 109 such changes (11,202 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Campaign pause / enable — when this changed 877 Reliable — 1 type

Across 877 such changes (91,812 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Ad group pause / enable — when this changed 91 Roughly right UID 12 2 types

Across 91 such changes (9,326 model predictions), a prediction here is roughly right. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup enable 17 Roughly right UID 171
adgroup pause 74 Reliable UID 141
Other ad group changes — when this changed 144 Reliable — 3 types

Across 144 such changes (14,739 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup bid change 74 Roughly right UID 91
adgroup change 69 Reliable —
adgroup remove 1 Too early —
Ad changes — when this changed 187 Reliable UID 55 5 types

Across 187 such changes (19,207 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

ad create 35 Roughly right UID 218
ad edit 68 Reliable UID 90
ad enable 5 Reliable UID 199
ad pause 70 Reliable UID 55
ad remove 9 Roughly right UID 117
Asset changes — when this changed 501 Reliable UID 126 1 type

Across 501 such changes (51,251 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Combined changes — when several kinds of change happened together 3,996 Reliable UID 246 24 types

Across 3,996 such changes (411,139 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup bid change 19 Reliable UID 55
adgroup change 4 Too early —
adgroup enable 12 Reliable —
adgroup pause 301 Reliable —
ad-group bid change 1 Too early —
ad create 482 Roughly right UID 113
ad edit 43 Roughly right UID 204
ad enable 20 Roughly right UID 91
ad pause 9 Roughly right UID 66
asset change 34 Roughly right UID 107
audience change 15 Reliable UID 91
bid strategy change 32 Reliable —
budget change 1,111 Roughly right UID 165
campaign pause 184 Reliable —
bid on a keyword / audience / placement 32 Reliable UID 126
targeting criteria change 93 Reliable UID 107
criterion enable 31 Reliable UID 55
criterion pause 15 Roughly right UID 90
criterion remove 3 Too early —
keyword add 896 Reliable —
keyword remove 116 Reliable UID 12
negative keyword add 390 Reliable —
negative keyword remove 1 Too early —
target value change 152 Roughly right UID 239

14 groups covering 88 change types. Open a row to see the types inside it.

Budget change — when a daily budget was raised or lowered 2,764 Roughly right UID 95 6 types

Across 2,764 such changes (284,990 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

down 655 Roughly right UID 72
down large 635 Roughly right UID 148
flat 176 Roughly right —
unknown 4 Too early —
up 528 Roughly right UID 95
up large 766 Roughly right UID 141
Bid strategy switch — when the bid strategy was switched from one type to another 66 Reliable UID 95 6 types

Across 66 such changes (6,716 model predictions), a prediction here is reliable. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

maximize conversions → maximize conversion value 3 Too early —
maximize conversions → target spend 10 Reliable UID 92
target spend → maximize conversions 5 Roughly right —
switched to Maximize Conversion Value 10 Roughly right UID 75
switched to Maximize Conversions 22 Reliable UID 95
unknown → target spend 16 Reliable UID 241
Bid target change — when a bid-strategy target value changed (e.g. Target CPA $60 → $50) 332 Roughly right — 13 types

Across 332 such changes (34,141 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

ad-group bid change 14 Roughly right UID 254
maximize conversion value · down 36 Roughly right —
maximize conversion value · down large 3 Too early —
maximize conversion value · flat 9 Roughly right UID 196
switched to Maximize Conversion Value 2 Too early —
maximize conversion value · up 38 Roughly right UID 91
maximize conversion value · up large 14 Roughly right UID 204
maximize conversions · down 58 Roughly right —
maximize conversions · down large 21 Roughly right UID 117
maximize conversions · flat 10 Roughly right UID 241
switched to Maximize Conversions 13 Roughly right UID 196
maximize conversions · up 93 Reliable UID 8
maximize conversions · up large 21 Roughly right UID 8
Negative keyword changes — when negative keywords were added 1,597 Roughly right UID 95 16 types

Across 1,597 such changes (163,702 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

negative keyword add 332 Roughly right UID 218
broad · 1 22 Roughly right —
broad · 11+ 9 Roughly right UID 75
broad · 2-10 25 Reliable UID 90
exact · 1 234 Roughly right —
exact · 11+ 136 Roughly right —
exact · 2-10 370 Roughly right UID 162
mixed · 11+ 86 Roughly right —
mixed · 2-10 86 Roughly right —
phrase · 1 75 Roughly right UID 19
phrase · 11+ 51 Roughly right UID 95
phrase · 2-10 114 Roughly right UID 162
unknown · 1 7 Reliable —
unknown · 11+ 4 Too early —
unknown · 2-10 4 Too early —
negative keyword remove 42 Roughly right —
Keyword changes — when this changed 459 Roughly right — 2 types

Across 459 such changes (47,264 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

keyword add 336 Roughly right —
keyword remove 123 Roughly right —
Other criterion changes (kind not recorded) — when this changed 981 Roughly right UID 95 6 types

Across 981 such changes (100,815 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

bid on a keyword / audience / placement 253 Roughly right UID 13
targeting criteria change 426 Roughly right UID 95
criterion enable 52 Roughly right —
criterion pause 229 Roughly right UID 239
criterion remove 18 Reliable UID 238
criterion url change 3 Too early —
Location, audience and demographic targeting — when this changed 68 Roughly right UID 55 3 types

Across 68 such changes (6,950 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Roughly right.

Broken down by kind:

audience change 32 Roughly right UID 134
demographic change 15 Reliable —
geo change 21 Roughly right UID 103
Device, schedule and placement targeting — when this changed 56 Reliable — 1 type

Across 56 such changes (5,661 model predictions), a prediction here is reliable. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Roughly right Range — how often the real result fell inside the predicted range
  • Roughly right Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Campaign pause / enable — when this changed 476 Roughly right — 1 type

Across 476 such changes (48,999 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Ad group pause / enable — when this changed 73 Reliable UID 12 2 types

Across 73 such changes (7,473 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup enable 11 Roughly right —
adgroup pause 62 Reliable UID 12
Other ad group changes — when this changed 100 Reliable — 3 types

Across 100 such changes (10,223 model predictions), a prediction here is reliable. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup bid change 54 Roughly right UID 247
adgroup change 45 Reliable —
adgroup remove 1 Too early —
Ad changes — when this changed 134 Reliable UID 55 5 types

Across 134 such changes (13,711 model predictions), a prediction here is reliable. What that's made of:

  • Reliable Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

ad create 33 Roughly right UID 218
ad edit 35 Roughly right UID 241
ad enable 3 Too early —
ad pause 54 Reliable UID 55
ad remove 9 Roughly right —
Asset changes — when this changed 289 Roughly right — 1 type

Across 289 such changes (29,285 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Roughly right.

Combined changes — when several kinds of change happened together 2,539 Roughly right UID 95 23 types

Across 2,539 such changes (259,444 model predictions), a prediction here is roughly right. What that's made of:

  • Roughly right Direction — called the change up or down correctly on the goal metric
  • Reliable Size — how close the predicted size of the effect was
  • Not reliable Range — how often the real result fell inside the predicted range
  • Not reliable Goal — how close on the goal number itself (CPA or ROAS)

Today's leaderboard #1 (UID 72) on this change type: Reliable.

Broken down by kind:

adgroup bid change 14 Roughly right UID 91
adgroup change 4 Too early —
adgroup enable 5 Roughly right —
adgroup pause 180 Roughly right UID 95
ad-group bid change 1 Too early —
ad create 227 Roughly right UID 141
ad edit 29 Roughly right —
ad enable 12 Roughly right —
ad pause 6 Roughly right UID 205
asset change 23 Roughly right UID 148
audience change 13 Roughly right UID 14
bid strategy change 22 Reliable UID 19
budget change 721 Roughly right UID 95
campaign pause 144 Roughly right UID 250
bid on a keyword / audience / placement 18 Roughly right UID 72
targeting criteria change 55 Reliable UID 72
criterion enable 21 Reliable UID 55
criterion pause 12 Roughly right UID 72
criterion remove 2 Too early —
keyword add 636 Reliable —
keyword remove 46 Reliable —
negative keyword add 242 Roughly right —
target value change 106 Roughly right —

A row reads Too early until at least 500 of its changes have settled. Open any row for the score breakdown and the best model.

Counted: changes to things that already exist, once their outcome has settled.

Not counted: brand-new campaigns & ads (no "before" to score), and recent changes still settling.

The winning model over time

The same winning model, week by week — is the best model on the subnet getting better? Every point is the mean of per-entry scores already published in the daily receipts, so anyone can recompute it.

Winning model accuracy by week, per prediction horizon One line per horizon. Vertical axis is mean score from 0 to 1; the full range is always shown. The same numbers appear in the table below. 0.00 0.25 0.50 0.75 1.00 W34 ’26W36W38W40W42W43 7-day · 2026-W34 · mean 0.606 · n=1086 7-day · 2026-W35 · mean 0.605 · n=390 7-day · 2026-W36 · mean 0.591 · n=861 7-day · 2026-W37 · mean 0.600 · n=2502 7-day · 2026-W38 · mean 0.601 · n=2505 7-day · 2026-W39 · mean 0.602 · n=2467 7-day · 2026-W40 · mean 0.605 · n=2181 7-day · 2026-W41 · mean 0.635 · n=2820 7-day · 2026-W42 · mean 0.617 · n=3063 7-day · 2026-W43 · mean 0.612 · n=750 · week still open 7-day 14-day · 2026-W35 · mean 0.592 · n=1174 14-day · 2026-W36 · mean 0.594 · n=863 14-day · 2026-W37 · mean 0.567 · n=884 14-day · 2026-W38 · mean 0.577 · n=2541 14-day · 2026-W39 · mean 0.581 · n=2536 14-day · 2026-W40 · mean 0.582 · n=2511 14-day · 2026-W41 · mean 0.590 · n=2236 14-day · 2026-W42 · mean 0.602 · n=2849 14-day · 2026-W43 · mean 0.595 · n=1555 · week still open 14-day 28-day · 2026-W36 · mean 0.518 · n=489 28-day · 2026-W37 · mean 0.580 · n=1581 28-day · 2026-W38 · mean 0.585 · n=874 28-day · 2026-W39 · mean 0.555 · n=892 28-day · 2026-W40 · mean 0.563 · n=2555 28-day · 2026-W41 · mean 0.575 · n=2571 28-day · 2026-W42 · mean 0.579 · n=2526 28-day · 2026-W43 · mean 0.569 · n=1286 · week still open 28-day
  • 7-day horizon
  • 14-day horizon
  • 28-day horizon
  • week still open
Each point is the mean score of the leading model's predictions that settled in that week, at that horizon. Every score is published per entry in the day's signed receipt, so any point here can be recomputed from the same documents used to verify a day — this chart reports those numbers, it does not compute a separate one. The leading model is shown without identity. A horizon with no settled entries in a week has no point rather than a zero: the 28-day line cannot begin before 8 September 2026.
Winning model mean score by week and horizon
Week Week ending Horizon Mean score Entries Status
2026-W34 2026-08-23 7-day 0.606 1086 settled
2026-W35 2026-08-30 7-day 0.605 390 settled
2026-W35 2026-08-30 14-day 0.592 1174 settled
2026-W36 2026-09-06 7-day 0.591 861 settled
2026-W36 2026-09-06 14-day 0.594 863 settled
2026-W36 2026-09-06 28-day 0.518 489 settled
2026-W37 2026-09-13 7-day 0.600 2502 settled
2026-W37 2026-09-13 14-day 0.567 884 settled
2026-W37 2026-09-13 28-day 0.580 1581 settled
2026-W38 2026-09-20 7-day 0.601 2505 settled
2026-W38 2026-09-20 14-day 0.577 2541 settled
2026-W38 2026-09-20 28-day 0.585 874 settled
2026-W39 2026-09-27 7-day 0.602 2467 settled
2026-W39 2026-09-27 14-day 0.581 2536 settled
2026-W39 2026-09-27 28-day 0.555 892 settled
2026-W40 2026-10-04 7-day 0.605 2181 settled
2026-W40 2026-10-04 14-day 0.582 2511 settled
2026-W40 2026-10-04 28-day 0.563 2555 settled
2026-W41 2026-10-11 7-day 0.635 2820 settled
2026-W41 2026-10-11 14-day 0.590 2236 settled
2026-W41 2026-10-11 28-day 0.575 2571 settled
2026-W42 2026-10-18 7-day 0.617 3063 settled
2026-W42 2026-10-18 14-day 0.602 2849 settled
2026-W42 2026-10-18 28-day 0.579 2526 settled
2026-W43 2026-10-25 7-day 0.612 750 week still open
2026-W43 2026-10-25 14-day 0.595 1555 week still open
2026-W43 2026-10-25 28-day 0.569 1286 week still open

Where these numbers come from

Every score on this page is the settle-flow score in the daily receipts, published per prediction at /v1/daily — the same numbers that feed the leaderboard. This page and the leaderboard can never disagree, because there is nothing here to disagree with: it is the same data, added up by change type.