Skip to content

Make the validation tab two-sided, and stop quoting a ratio as a multiplier #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vahid-ahmadi merged 1 commit into from
Aug 26, 2026
Merged

Make the validation tab two-sided, and stop quoting a ratio as a multiplier #186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions obr/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,9 +239,12 @@ <h4>Worked example</h4>
<em>spending</em> shock (<code>CGG</code>) lands straight in the GDP
identity with no behavioural equation responding, giving a multiplier
of exactly <strong>1.0000</strong>, flat, against the OBR's 0.6 &mdash;
and capital spending (<code>CGIPS</code>) is worse, moving business
investment <strong>&minus;292% of the shock</strong>, wrong-signed
against the OBR's 1.0. The
and capital spending (<code>CGIPS</code>) is worse still: it does not
transmit at all. Total fixed investment moves exactly zero, and
business investment moves wrong-signed and
<strong>non-proportionally</strong> &mdash; quadrupling the shock
changes the response by under a factor of two, so there is no
multiplier to quote, against the OBR's published 1.0. The
<a href="/obr/validation#caveats">validation page</a> lists the
full set of known limits.
</div>
Expand Down
173 changes: 156 additions & 17 deletions obr/validation/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,26 @@ <h1 class="reveal page-title" style="--d:1;">
</div>
</nav>

<section id="trust" class="band">
<section class="band band-wide">
<div class="band-head">
<span class="kicker mono">how far to trust it</span>
<h2>Excellent anchored. Weak free-running. Both published.</h2>
<span class="kicker mono">validation</span>
<h2>How far to trust it, and where it stops.</h2>
</div>
<div class="prose">
<p>
Five steps: what the three configurations score, the anchored fit
against the OBR's own forecast, the one independent test, what the
equations do free-running with no OBR judgement, and the limits that
follow. The panel alongside carries the figures each step refers to.
</p>
</div>

<div class="scrolly-wrap">
<div class="scrolly-steps">

<div class="meth-step" id="trust" data-step="0">
<div class="meth-step-head"><span class="meth-step-num mono">1</span><h3>How far to trust it</h3></div>
<p class="mo-lead">Excellent anchored. Weak free-running. Both published.</p>
<div class="prose">
<p>
Three configurations, three levels of trust. Errors are MAPE (mean
Expand Down Expand Up @@ -141,13 +156,17 @@ <h2>Excellent anchored. Weak free-running. Both published.</h2>
Office for Budget Responsibility, and not official OBR estimates.
</p>
</div>
</section>

<section id="anchored" class="band">
<div class="band-head">
<span class="kicker mono">evidence — anchored</span>
<h2>Against the OBR's own forecast, and HMRC's reckoner.</h2>
</div>
<div class="meth-inline">
<div class="meth-sec-title">the three configurations</div>
<div class="meth-eq"><div class="meth-eq-label">anchored</div><div class="meth-eq-body">GDP 0.15% MAPE, consumption 0.25% &mdash; by construction, not skill</div></div>
<div class="meth-eq"><div class="meth-eq-label">held add-factors</div><div class="meth-eq-body">GDP 0.37%, 6 of 8 computed in band</div></div>
<div class="meth-eq"><div class="meth-eq-label">free-running</div><div class="meth-eq-body">GDP 4.48%, consumption 7.49% &mdash; report-only</div></div>
<div class="meth-eq"><div class="meth-eq-label">CI gate</div><div class="meth-eq-body">the build hard-fails if anchored GDP drifts past 1%</div></div>
</div>
</div>
<div class="meth-step" id="anchored" data-step="1">
<div class="meth-step-head"><span class="meth-step-num mono">2</span><h3>Anchored against the EFO</h3></div>
<p class="mo-lead">One test, not two.</p>
<div class="prose">
<p>
Anchored GDP and consumption are <strong>one test, not two</strong>:
Expand Down Expand Up @@ -206,6 +225,18 @@ <h2>Against the OBR's own forecast, and HMRC's reckoner.</h2>
</svg>
<figcaption>Current March 2026 EFO baseline. Quarterly deviation from the published EFO; CI hard-fails at &#177;1.00%, off the top and bottom of this frame. Computed from <code>papers/obr-macro/figures/fig_anchored_data.csv</code>, regenerated from the March 2026 detailed forecast tables on 21 July 2026.</figcaption>
</figure>
</div>
<div class="meth-inline">
<div class="meth-sec-title">anchored fit</div>
<div class="meth-eq"><div class="meth-eq-label">real GDP</div><div class="meth-eq-body">0.15% MAPE, 2025Q1&ndash;2027Q4</div></div>
<div class="meth-eq"><div class="meth-eq-label">consumption</div><div class="meth-eq-body">0.25% &mdash; the same &pound;m error as GDP, not a second test</div></div>
<div class="meth-eq"><div class="meth-eq-label">why</div><div class="meth-eq-body">every other demand-identity term is exogenous or pinned</div></div>
</div>
</div>
<div class="meth-step" id="val-hmrc" data-step="2">
<div class="meth-step-head"><span class="meth-step-num mono">3</span><h3>The independent test</h3></div>
<p class="mo-lead">HMRC's own ready reckoner.</p>
<div class="prose">
<p>
A 1pp basic-rate rise from April 2026, costed through the PolicyEngine
static-costing bridge: <strong>£6.46bn against HMRC's £6.9bn,
Expand Down Expand Up @@ -266,6 +297,18 @@ <h2>Against the OBR's own forecast, and HMRC's reckoner.</h2>
</tbody>
</table>
</div>
</div>
<div class="meth-inline">
<div class="meth-sec-title">1p on the basic rate</div>
<div class="meth-eq"><div class="meth-eq-label">ours, 2026</div><div class="meth-eq-body">&pound;6.46bn</div></div>
<div class="meth-eq"><div class="meth-eq-label">HMRC reckoner</div><div class="meth-eq-body">&pound;6.90bn &mdash; a &minus;6.4% gap</div></div>
<div class="meth-eq"><div class="meth-eq-label">status</div><div class="meth-eq-body">a benchmark, not a validation</div></div>
</div>
</div>
<div class="meth-step" id="val-freerun" data-step="3">
<div class="meth-step-head"><span class="meth-step-num mono">4</span><h3>Free-running</h3></div>
<p class="mo-lead">What the equations do with no OBR judgement.</p>
<div class="prose">
<figure class="vfig">
<svg class="vchart" data-chart="obr-computed-share" viewBox="0 0 760 314" role="img" aria-labelledby="obr-computed-share-t obr-computed-share-d">
<title id="obr-computed-share-t">How much of the OBR emulator scorecard the model actually computes</title>
Expand Down Expand Up @@ -435,13 +478,17 @@ <h2>Against the OBR's own forecast, and HMRC's reckoner.</h2>
<a href="/obr">obr-macro model page →</a>
</p>
</div>
</section>

<section id="caveats" class="band">
<div class="band-head">
<span class="kicker mono">limits</span>
<h2>Known limitations.</h2>
</div>
<div class="meth-inline">
<div class="meth-sec-title">the honest scorecard</div>
<div class="meth-eq"><div class="meth-eq-label">real GDP</div><div class="meth-eq-body">4.48% MAPE &mdash; the model contracts while the EFO grows</div></div>
<div class="meth-eq"><div class="meth-eq-label">computed</div><div class="meth-eq-body">11 of 21 headline variables; the other 10 are passthrough</div></div>
<div class="meth-eq"><div class="meth-eq-label">in band</div><div class="meth-eq-body">6 of the 11, and one of those is a trivial identity</div></div>
<div class="meth-eq"><div class="meth-eq-label">worst</div><div class="meth-eq-body">company profits 63.29%</div></div>
</div>
</div>
<div class="meth-step" id="caveats" data-step="4">
<div class="meth-step-head"><span class="meth-step-num mono">5</span><h3>Known limitations</h3></div>
<p class="mo-lead">What it cannot be used for.</p>
<div class="prose">
<div class="table-scroll mo-tablecard"><table>
<caption>Known limits of the OBR emulator</caption>
Expand Down Expand Up @@ -489,8 +536,63 @@ <h2>Known limitations.</h2>
</tr>
</tbody>
</table></div>
</div>
<div class="meth-inline">
<div class="meth-sec-title">the headline limits</div>
<div class="meth-eq"><div class="meth-eq-label">spending multiplier</div><div class="meth-eq-body"><code>CGG</code> is exactly 1.0000, flat, against the OBR's 0.6</div></div>
<div class="meth-eq"><div class="meth-eq-label">capital spending</div><div class="meth-eq-body"><code>CGIPS</code> does not transmit &mdash; do not score it</div></div>
<div class="meth-eq"><div class="meth-eq-label">corporation tax</div><div class="meth-eq-body"><code>TCPRO</code> converges slowly: 12 quarters is ~40% of plateau</div></div>
<div class="meth-eq"><div class="meth-eq-label">inert blocks</div><div class="meth-eq-body">unemployment, prices and trade never respond to any lever</div></div>
<div class="meth-eq"><div class="meth-eq-label">convergence</div><div class="meth-eq-body">no quarter reaches tolerance &mdash; read scale, not quarters</div></div>
</div>
</div>

</div>

<aside class="scrolly-sticky" aria-hidden="true">
<div class="meth-panel" data-step="0">
<div class="meth-panel-head"><span class="meth-panel-title">How far to trust it</span><span class="meth-panel-badge mono">Step 1</span></div>
<div class="meth-sec-title">the three configurations</div>
<div class="meth-eq"><div class="meth-eq-label">anchored</div><div class="meth-eq-body">GDP 0.15% MAPE, consumption 0.25% &mdash; by construction, not skill</div></div>
<div class="meth-eq"><div class="meth-eq-label">held add-factors</div><div class="meth-eq-body">GDP 0.37%, 6 of 8 computed in band</div></div>
<div class="meth-eq"><div class="meth-eq-label">free-running</div><div class="meth-eq-body">GDP 4.48%, consumption 7.49% &mdash; report-only</div></div>
<div class="meth-eq"><div class="meth-eq-label">CI gate</div><div class="meth-eq-body">the build hard-fails if anchored GDP drifts past 1%</div></div>
</div>
<div class="meth-panel" data-step="1" hidden>
<div class="meth-panel-head"><span class="meth-panel-title">Anchored against the EFO</span><span class="meth-panel-badge mono">Step 2</span></div>
<div class="meth-sec-title">anchored fit</div>
<div class="meth-eq"><div class="meth-eq-label">real GDP</div><div class="meth-eq-body">0.15% MAPE, 2025Q1&ndash;2027Q4</div></div>
<div class="meth-eq"><div class="meth-eq-label">consumption</div><div class="meth-eq-body">0.25% &mdash; the same &pound;m error as GDP, not a second test</div></div>
<div class="meth-eq"><div class="meth-eq-label">why</div><div class="meth-eq-body">every other demand-identity term is exogenous or pinned</div></div>
</div>
<div class="meth-panel" data-step="2" hidden>
<div class="meth-panel-head"><span class="meth-panel-title">The independent test</span><span class="meth-panel-badge mono">Step 3</span></div>
<div class="meth-sec-title">1p on the basic rate</div>
<div class="meth-eq"><div class="meth-eq-label">ours, 2026</div><div class="meth-eq-body">&pound;6.46bn</div></div>
<div class="meth-eq"><div class="meth-eq-label">HMRC reckoner</div><div class="meth-eq-body">&pound;6.90bn &mdash; a &minus;6.4% gap</div></div>
<div class="meth-eq"><div class="meth-eq-label">status</div><div class="meth-eq-body">a benchmark, not a validation</div></div>
</div>
<div class="meth-panel" data-step="3" hidden>
<div class="meth-panel-head"><span class="meth-panel-title">Free-running</span><span class="meth-panel-badge mono">Step 4</span></div>
<div class="meth-sec-title">the honest scorecard</div>
<div class="meth-eq"><div class="meth-eq-label">real GDP</div><div class="meth-eq-body">4.48% MAPE &mdash; the model contracts while the EFO grows</div></div>
<div class="meth-eq"><div class="meth-eq-label">computed</div><div class="meth-eq-body">11 of 21 headline variables; the other 10 are passthrough</div></div>
<div class="meth-eq"><div class="meth-eq-label">in band</div><div class="meth-eq-body">6 of the 11, and one of those is a trivial identity</div></div>
<div class="meth-eq"><div class="meth-eq-label">worst</div><div class="meth-eq-body">company profits 63.29%</div></div>
</div>
<div class="meth-panel" data-step="4" hidden>
<div class="meth-panel-head"><span class="meth-panel-title">Known limitations</span><span class="meth-panel-badge mono">Step 5</span></div>
<div class="meth-sec-title">the headline limits</div>
<div class="meth-eq"><div class="meth-eq-label">spending multiplier</div><div class="meth-eq-body"><code>CGG</code> is exactly 1.0000, flat, against the OBR's 0.6</div></div>
<div class="meth-eq"><div class="meth-eq-label">capital spending</div><div class="meth-eq-body"><code>CGIPS</code> does not transmit &mdash; do not score it</div></div>
<div class="meth-eq"><div class="meth-eq-label">corporation tax</div><div class="meth-eq-body"><code>TCPRO</code> converges slowly: 12 quarters is ~40% of plateau</div></div>
<div class="meth-eq"><div class="meth-eq-label">inert blocks</div><div class="meth-eq-body">unemployment, prices and trade never respond to any lever</div></div>
<div class="meth-eq"><div class="meth-eq-label">convergence</div><div class="meth-eq-body">no quarter reaches tolerance &mdash; read scale, not quarters</div></div>
</div>
</aside>
</div>
</section>

</main>

<footer class="foot foot-bar">
Expand All @@ -503,5 +605,42 @@ <h2>Known limitations.</h2>
</footer>

<script src="/reveal.js?v=2" defer></script>
<script>
// Validation scrollytelling: highlight the step at the reading line and
// show its panel in the sticky aside.
(function () {
"use strict";
var steps = Array.from(document.querySelectorAll(".meth-step"));
var panels = Array.from(document.querySelectorAll(".meth-panel"));
if (!steps.length || !panels.length) return;
var current = -1;
function update() {
var offset = window.innerHeight * 0.33;
var active = 0;
steps.forEach(function (step, i) {
if (step.getBoundingClientRect().top < offset) active = i;
});
if (active === current) return;
current = active;
steps.forEach(function (step, i) {
step.classList.toggle("active", i === active);
});
var want = Math.min.apply(null, panels.map(function (panel) {
return Number(panel.getAttribute("data-step"));
}));
panels.forEach(function (panel) {
var ds = Number(panel.getAttribute("data-step"));
if (ds <= active && ds > want) want = ds;
});
panels.forEach(function (panel) {
panel.hidden = Number(panel.getAttribute("data-step")) !== want;
});
}
window.addEventListener("scroll", update, { passive: true });
window.addEventListener("resize", update, { passive: true });
window.addEventListener("hashchange", function () { setTimeout(update, 0); });
update();
})();
</script>
</body>
</html>