feat(Notebooks): Add backtest-slippage use-case notebook

[szemyd]

Summary

Adds notebooks/backtest-slippage.py, implementing the backtest-slippage outline from #34 (use case: Improve your backtest).

Builds a microstructure transaction-cost model — half-spread plus market impact — from Aperiodic's l1_price, impact and slippage metrics, and shows how different cost assumptions reshape a simple momentum backtest.

• Binance BTC perpetuals, 5-minute, May 2025 preview slice; runs end-to-end on DEMO-KEY.
• Fees are excluded by design and disclaimed prominently (a Binance taker fee ~5 bps one-way is cited as the dominant real cost); the flat benchmark is a comparable ~1 bps so the equity chart stays readable.
• Section 6 checks the impact model against measured per-trade slippage_bps_mean at the actual average trade size (volume_notional / taker trade count). The result is reported straight: the model under-predicts measured slippage by roughly an order of magnitude at the average trade size, with ~zero per-bar correlation — impact_per_notional (net-flow-based) is an order-of-magnitude proxy, not a per-trade predictor. The notebook prints the notional at which the model mean would equal measured slippage to make the size dependence explicit.
• Reuses the intro-notebook theme/helpers and the shared run_position_backtest.

.ipynb and HTML are generated by CI. Adds one README row and one preview-notebooks.json slug (also touched by the sibling notebook PRs, so whichever merges last needs a trivial rebase).

Part of splitting #35 into one PR per notebook. Implements #34.

🤖 Generated with Claude Code

[szemyd]

Quant review of notebooks/backtest-slippage.py

Verdict: REQUEST CHANGES — posted as a comment review because GitHub blocks formal change-requests from the PR author's own identity. Treat the blocking items below as merge-blockers; this review is written so a follow-up agent can implement it directly.

Reviewed the .py source against the executed .ipynb outputs on this branch. The scaffolding is genuinely good — schema audit that fails loudly, documented key resolution, correct signal/return timing (no look-ahead), honest "one month is an illustration" disclaimers, and consistency with the house run_position_backtest. But the notebook's two central quantitative claims don't survive scrutiny yet. Inline comments carry the details and concrete fixes; summary below.

Blocking

1. Fees are absent from the "realistic" cost model (inline at the scenario cell). Modelled dynamic cost @ $50k averages 0.21 bps; Binance USDT-M taker fees are ~1.7–5 bps. As written, the notebook teaches that a flat 5 bps assumption is "too harsh" when it is in fact a fair proxy for all-in taker cost, and that a properly modelled cost is ~negligible. Add a fee term or reframe the model explicitly as the microstructure-only component.
2. The calibration story is circular (inline at the calibration cell). The "slippage-implied coefficient" (3.80 bps/$1M) is (mean slippage − mean half-spread) / BACKTEST_NOTIONAL — inversely proportional to an assumed $50k that is never justified against the definition of slippage_bps_mean. Additionally .clip(lower=0) biases the implied coefficient upward, and the executed MAE (0.296 bps) exceeds the mean measured level (0.195 bps), i.e. the model has no pointwise skill in-sample — only the means agree. State the metric definition, match the notional to it, drop or justify the clip, and report corr/R² alongside MAE.
3. impact_per_notional units are guessed, not verified (inline). Everything downstream — including the sibling execution-liquidity-timing notebook that imports this convention — inherits the ×1e4 assumption. Verify against the catalogue docs and cite.

Important

4. Position/notional convention unstated; sign flips under-charged ~2× on the impact term under the model's own linearity (inline). One sentence + a printed turnover stat fixes it.

Minor (no inline comment)

• Section numbering jumps ## 2 → ## 3 → ## 6 (template remnant). Renumber.
• The equity chart is unreadable around the interesting comparison because the −70% flat-cost curve compresses the zero-vs-dynamic gap (~4pp) to nothing; log scale, separate panel, or a saner flat benchmark all work.
• Chart 2's violin mixes slippage_bps_std (a dispersion statistic) in with level statistics (mean/p95/vwap) on one "bps" axis — visually implies they're comparable quantities.
• amihud_like / kyle_like_lambda are fetched and audited but never used; either use them (they'd make a nice alternative impact proxy comparison) or drop them from the fetch.
• Annualized Sharpe of −36.5 from one month is technically what the arithmetic yields, but printing it invites ridicule; consider omitting Sharpe for the strawman scenario or footnoting it.

Suggested acceptance criteria for the revision

• Cost scenarios include (or explicitly disclaim) exchange fees, and the Takeaways no longer assert a flat bps assumption is "too harsh" for taker flow.
• The calibration section states the catalogue definition of slippage_bps_mean, prices the model at a notional consistent with that definition, and reports a pointwise fit statistic next to the MAE.
• The impact_per_notional unit is cited from documentation, not inferred.
• Re-executed .ipynb/HTML committed by CI so outputs match the revised source.

---

Generated by Claude Code

[szemyd]

Thanks for the thorough review — all items addressed in the latest commit (CI re-executed against real May 2025 data and is green). Point by point:

Blocking

• Fees. Scoped the model explicitly as microstructure-only with a prominent disclaimer (Binance VIP0 taker ~5 bps cited as excluded and dominant, not modelled). Dropped the "flat is too harsh" framing, set the flat benchmark to a comparable 1 bps, and the equity chart is now readable.
• Circular calibration. slippage_bps_mean is per the catalogue a per-trade, beyond-the-touch cost (excludes half-spread), so the model's impact term is now priced at the actual average executed trade notional (volume_notional / taker trade count) and compared to measured slippage — with corr and MAE reported, the .clip(lower=0) removed, and the notional at which model-mean = measured printed. On real data that lands at ~$45.4k, which makes the old $50k "agreement" visibly a coincidence of the chosen size; corr ≈ 0 and MAE ≈ the signal level, so it's stated as an order-of-magnitude proxy, not a calibrated predictor.
• impact_per_notional units. The catalogue defines it as |interval return| / |signed notional flow| = a price fraction per USD, so ×1e4 → bps/USD is correct; cited in a comment, redundant .abs() dropped (non-negative by construction), zero-flow inf guarded.

Important / minor: notional convention stated (unit = BACKTEST_NOTIONAL, flips as two one-way clips) with turnover printed; slippage_bps_std split out of the level violin; impact coef shown in bps/$1M; unused amihud_like/kyle_like_lambda dropped from the fetch; sections renumbered; one-month Sharpe footnoted.

---

Generated by Claude Code

[szemyd]

Quant re-review (round 2) of notebooks/backtest-slippage.py @ 9262367

Verdict: no remaining merge-blockers in the notebook. One required follow-up outside the code (stale PR description, below) and two inline nits. Posted as a comment review (GitHub blocks formal states from the PR author's identity).

All four round-1 blocking/important items are fixed, and I verified each against the re-executed .ipynb:

1. Fees — resolved. Scope note up top (title now says microstructure cost), REFERENCE_TAKER_FEE_BPS = 5.0 documented and referenced in the scenario markdown and Takeaways ("Fees dominate for takers"); the misleading "flat is too harsh" claim is gone; flat benchmark lowered to 1 bps so the chart is readable (zero +2.47% / flat-1bps −19.9% / dynamic −1.46%).
2. Calibration circularity — resolved, and honestly. The comparison is now impact-only at the bar's actual average trade size (median $5.2k, computed from volume_notional / taker trade count) vs measured per-trade slippage, with pointwise stats printed in the chart title and table: corr −0.01, MAE 0.195 bps, mean model 0.020 vs measured 0.195 bps. The old $50k "agreement" is exposed explicitly via "Notional where model mean = measured ($45,385)". The clip(lower=0) is gone. This is exactly the right shape: a negative validation result reported straight.
3. Units — addressed via quoted catalogue definitions for impact_per_notional and slippage_bps_mean, which now justify the ×1e4 conversion and the impact-only comparison. Caveat: the metrics docs are auth-walled from here, so I could not independently confirm the quotes — see inline note asking for an exact doc link (the cites are now load-bearing).
4. Notional convention — documented (1 unit = BACKTEST_NOTIONAL = 100% of equity; flips priced as two one-way clips, stated as deliberately under-charging vs a single $100k clip), plus printed turnover diagnostics (1,234 trades ≈ $123M on $50k equity) and the Sharpe-comparability footnote.

Minor round-1 items also done: section numbering, violin split (levels vs dispersion), unused fetches dropped, impact-coef display in bps/$1M.

Remaining items

• [Required] Update the PR description. It still claims "the modelled impact coefficient (4.18 bps/$1M) calibrates closely against measured slippage (implied 3.80 bps/$1M)" — the revised notebook explicitly repudiates that framing (coincidence of the $50k size assumption; corr ≈ 0). Whoever merges will otherwise ship a summary the notebook itself contradicts. Suggested replacement: "checks the impact model against measured per-trade slippage at the actual average trade size; the honest result is weak per-bar tracking (corr ≈ 0) and a ~10× level gap, discussed in Section 6."
• [Nit, inline] Non-negativity assertion on the impact coefficient (self-enforcing units tripwire).
• [Nit, inline] One sentence stating the 10× level miss as plainly as the correlation.

---

Generated by Claude Code

[szemyd]

Round-2 items done:

• Non-negativity tripwire added on impact_coef_bps_per_usd (your exact assert).
• Level miss stated plainly — the measured/model ratio is now printed in the calibration table and the interpretation says it in words (roughly an order of magnitude under, ~zero per-bar correlation, "order-of-magnitude proxy" meant literally).
• Verifiable cite — the impact_per_notional and slippage_bps_mean definitions now link the trade-metrics catalogue (the public metrics page; the definitions were confirmed against the catalogue JSON).
• PR description rewritten to drop the stale "calibrates closely (4.18 vs 3.80)" claim and describe the honest negative validation instead.

---

Generated by Claude Code

[almostintuitive]

please change the y axis starting position in 4. Chart 1 — Spread distribution by hour
so it doesn't start from 0 but from the minimum value

[szemyd]

Done in 4e53cc0. Chart 1 (Section 4, spread distribution by hour) now starts the y-axis at the minimum observed spread instead of 0:

ax.set_ylim(df["spread_bps"].min(), df["spread_bps"].quantile(0.99))

so the hourly boxplots fill the panel rather than sitting compressed near the top. CI will re-execute the notebook and regenerate the .ipynb and HTML from this .py.

---

Generated by Claude Code

[almostintuitive]

👍