ForexBot2/backtest/research/replay_recording_2026-09-09.md

Replay recording campaign — 2026-09-09 (2-year Oanda window, M5 + M15, $0.65)

Purpose and owner decisions

The owner asked for a 2-year replay of the M5 and M15 timeframes that writes down, for every trade setup the system generates, the indicator weights and values the decision was made with and the economic-calendar events around it, in a form that a later study can use to decide whether indicator weights should change, or indicators be dropped or replaced. Owner decisions (2026-09-09): also record one compact row per ANALYZED bar (not only the bars that became trades), and restore the Forex Factory calendar scraper to scripts/.

Nothing in the signal core changed. The recorder is opt-in (--record DIR, --record-bars) and read-only with respect to the signal path; the trade book with recording on is byte-identical to the book with it off (MD5 below).

Harness

backtest/gpu_backtest.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 --spread 0.65 --workers 12 --primary-tf {5min|15min}, pair-factory config (no --set/--quality/--strategies/ --confirm-tfs; confirmation TFs from the factory map: M5→15min,1h and M15→5min,1h), Backend: NumPy CPU fallback (CuPy cannot find CUDA on this box). Code at git 28b591d plus the recorder (this campaign). Bars fetched once from Oanda into backtest/cache/oanda_XAU_USD_{5min,15min,1h}_20240705_20260905.parquet (62-day warm-up pad) and read from the cache by every later run. Each timeframe was run twice: a CONTROL run (--export-csv book_control.csv) and a RECORDED run (--export-csv book.csv --record DIR --record-bars). Outputs in backtest/research/replay_2026-09/{m5,m15}/ (gitignored).

Books (identical for control and recorded — that is the point)

book	trades	total R	WR	PF	max DD
M5 whole window	781	+146.07	52.9%	1.52	10.1
first half	189	+28.01	49.7%	1.41	8.4
second half	592	+118.06	53.9%	1.56	10.1
M15 whole window	793	+117.62	60.4%	1.41	14.8
first half	383	+38.22	58.5%	1.26	14.8
second half	410	+79.40	62.2%	1.55	9.9

Halves split the entry-time range in two; M5 trades cluster in the second year.

Behaviour-neutrality check

TF	control MD5	recorded MD5	manifest book_csv_md5
M5	29120a1ee32979431d062f210d41a9d1	29120a1ee32979431d062f210d41a9d1	same
M15	e62d3ded56f35572f52a96f604a99cf4	e62d3ded56f35572f52a96f604a99cf4	same

Replay time M5 114 s / M15 35 s on 12 workers (control runs 110 s / 35 s — recording costs seconds). backtest/replay_scorecard.py DIR --check passes all eight self-consistency invariants on both runs: trade counts match the manifest; every trade has its primary block and both confirmation blocks; base weight × regime multiplier = effective weight and contribution = vote × effective weight × confidence/100 on every one of the 17,869 (M5) / 18,070 (M15) indicator rows; the composite score is reproduced from the stored rows on every trade and timeframe; every regime multiplier equals the table in the manifest; the bar table has exactly one row per analyzed bar and contains every trade's bar.

Cross-check against the owner's 3-year baseline books (same window, pre-2026-09-07 code)

TF	2-yr run	baseline sizing_2026-09/base_*_3yr_065.csv cut to the window	shared entries
M5	781 / +146.07R	769 / +153.58R	757 (24 only new, 12 only baseline)
M15	793 / +117.62R	788 / +115.78R	774 (19 only new, 14 only baseline)

The baseline files are dated 2026-09-05 and predate the 2026-09-07 outlier-filter change (which moved the 3-yr M5 book 857→870 trades, +161.3→+153.6R), so the differences are the expected ones; byte identity was never expected here. The behaviour-neutrality proof is the MD5 pair above, on identical code.

What was recorded (storage decision: Parquet, one folder per run)

Parquet keeps exact typed numbers and UTC timestamps, compresses well (M5: 20 MB for 86k bars × 172 columns), and pandas reads a file in one line; the repo already uses it for the bar cache. SQLite would need JSON blobs or many tables with string timestamps; CSV loses types and is several times larger. DuckDB can query the parquet files directly if SQL is ever wanted.

file	grain	M5 rows	M15 rows
manifest.json	CLI, window, spread, TFs, engine + gate fingerprints, git SHA, backend, regime weight table in force, observed base weights, composite thresholds, bar counts + skip reasons, cache file names, calendar fingerprint, table row counts, book MD5	1	1
trades.parquet	one row per ACCEPTED trade: the 63 book columns + per-timeframe regime / composite / confidence / ATR / vote sums, penalty tags, setup fields, all warnings and reasons, fingerprints, calendar scalars	781	793
indicators.parquet	one row per trade × timeframe × indicator that voted: vote, value, confidence, base weight, regime multiplier, effective weight, contribution, agree/oppose/neutral, regime	17,869	18,070
events.parquet	one row per trade × calendar event from 4 h before entry to exit (or 4 h after entry), plus all-day rows on the entry's New-York date, with phase before / during_hold / after_exit / all_day	7,803	12,932
bars.parquet	one row per ANALYZED primary bar (after weekend/session filters, before the setup gates): final score/signal/quality, has_setup, per-timeframe regime/composite and per-indicator vote/value/confidence/effective weight/contribution as float32, plus forward moves 1 h / 4 h / 24 h and MFE/MAE in ATRs on the raw bars	86,192	29,917

Candidates before the 5-bar cooldown: M5 1,279 → 781 accepted; M15 1,535 → 793. An indicator that did not fire (divergence, candlestick) has NO row / NaN — never a zero vote. Reader: backtest.replay_record.load_run(dir). First analysis pass: backtest/replay_scorecard.py DIR --what-if --bars (full output saved as scorecard.txt in each run folder).

First look (descriptive statistics only — NOT evidence for a change)

Everything below is in-book: it cannot see trades a change would have created, and it ignores the MTF/penalty/gate chain that sits between the composite and the entry. The 2-year window is a strong gold up-trend, so the "no vote" baseline itself drifts up about +0.37 ATR per 4 h and +1.9 ATR per 24 h on M5 (+0.15 / +0.9 on M15); any BUY-vote number must be read against that baseline, not against zero.

Trade-level (primary timeframe):

• Bollinger opposes the entry on 731 of 781 M5 trades (94%): a mean-reversion reading inside a trend-following entry — the "constant bias" seen in the 2026-08-04 audit — and those trades carry +152R. RSI is neutral on 73% of M5 entries.
• M5 trades where divergence OPPOSED the entry did best (158 trades, +0.37R mean, 63% wins) and trades where it was absent did worst (546, +0.13R); the same "inverted" pattern as the 2026-08-04 in-book audit, which replayed WORSE when "fixed" — the reminder that this table is a hypothesis generator.
• What-if (M5): removing ema_trend would drop 206 accepted trades carrying +36.8R below the entry band; removing MACD 211 / +21.2R; ADX 127 / +22.6R. On M15 hardly any trade loses direction from a single-indicator removal (the entries sit further above the band); the changes show up as strong/weak band moves instead.
• Calendar: M5 trades held through at least one high-impact USD release did better (69 trades, +0.42R mean vs +0.19 overall; M15 224 trades, +0.23 vs +0.15) — the same direction as the 2026-09-06 addendum. Entries within 30 min after a release are slightly negative on both TFs (40 and 35 trades — too few to act on).

Bar-level (all analyzed bars, drift NOT yet removed):

• Every indicator's SELL-side votes were followed, on average, by a move AGAINST the vote on both timeframes (e.g. M5 ema_trend −0.27 ATR / 4 h, divergence −0.63); this is the up-drift plus the known "SELL negative everywhere" finding. The next analysis step is to subtract the per-bar baseline (or split by regime) before scoring votes.
• Above-baseline BUY callers on M5 at 4 h: ADX +0.50, keltner +0.50, RSI +0.45, candlestick +0.46, ROC +0.46, ema_trend (+0.67 vs baseline +0.32 among its own neutral bars); below baseline: divergence +0.30, Bollinger +0.36.
• Bars what-if (M5): dropping divergence would newly qualify 2,268 bars whose later move averaged +0.26 ATR / 4 h and drop 3,611 bars averaging −0.26 — the one single-indicator removal where both sides point the same way. Dropping ema_trend or ADX points the other way (the bars they add did worse, the bars they remove did better).

Pre-registered rule for anything that comes out of these files

A candidate change (weight, regime multiplier, dropped or replaced indicator) is a hypothesis until it passes CLAUDE.md rule 1 on a full rerun through analyze_from_dataframes: positive total R in BOTH halves of a ≥12-month window, PF ≥ 1.3 at $0.65, trailing exits, next-open fills, plus the 2024-06→2025-03 out-of-sample window for marginal edges, compared with a CONTROL produced on the same harness. Nothing in this campaign changes any number.

Suggested next analysis (not done here)

1. Drift-adjust the bar table (subtract the neutral/absent baseline per regime and hour) and re-score each indicator's standalone hit rate by timeframe role.
2. For the 2–3 most suspect indicators (divergence, Bollinger, RSI on M5), run the real rule-1 rerun with --set technical.regime_weights_override=... or disabled_indicators and compare against the control books in this folder.
3. Re-run this recording after any config change so the folders stay comparable (the manifest carries the engine fingerprint and the regime table for exactly that purpose).

Tests: Module 33 "backtest replay recording" (12 tests); suite green.