Backtest reporting checklist — ForexBot2 (XAU/USD) ================================================== Every backtest report must record: - The exact command line (window, --primary-tf, --spread, and any --set / --quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line. - Trades, total R, win rate, profit factor and max drawdown for the whole window AND for both halves, at both cost conventions ($0.65 measured-Oanda headline, $0.30 cross-book comparability) — CLAUDE.md rule 1. - Per trade (the exported CSV already carries these): entry/exit time and price, exit reason, R multiple, MFE/MAE, entry reasons, and every indicator's signal / value / agreement at decision time. - Breakdowns by session/hour, weekday, regime and volatility regime, and, for any candidate change, the removed/added/shifted trade diff against the control book (the 5-bar cooldown displaces entries, so judge run-level numbers, never the added-trade bucket alone). - The md5 of the exported book whenever the run is a parity or regression check. Replays go through the real signal code (analyze_from_dataframes) — never a re-implementation. --- 2026-09-06: decision-time feature export --- - `--export-features` appends stake_multiplier, r_sized and the 65 `f_*` decision-time features (position_sizing.FEATURE_ORDER) AFTER the 63 pinned columns; default export is byte-identical to the pinned books. - Any sizing report must show the FLAT book and the SIZED book side by side and the label-shuffle null; the 2026-09-06 campaign verdict (nothing ships, backtest/research/sizing_campaign_2026-09.md) is the reference. --- 2026-09-09: replay recording (--record DIR [--record-bars]) --- - `--record DIR` writes manifest.json + trades/indicators/events parquet for every ACCEPTED trade: every indicator's signal / value / confidence / BASE weight / regime multiplier / EFFECTIVE weight / contribution on the primary AND both confirmation timeframes, penalties, the setup, the outcome, and the Forex Factory calendar events around the trade (backtest/cache/calendar.parquet, refreshed with scripts/ff_calendar_scraper.py). `--record-bars` adds one row per ANALYZED primary bar with the same votes plus forward moves (1h/4h/24h in ATR units). - Recording is read-only w.r.t. the signal path: any recorded run must be paired with a control run (same CLI without --record) and the report must show the two books' MD5s equal (manifest.book_csv_md5 is the md5 of the 63-column book). - Scorecards and "what-if" tables from these files (backtest/replay_scorecard.py) are IN-BOOK descriptive statistics: they cannot add trades a change would create and ignore the gates. Any weight / indicator change they suggest is a hypothesis for a full rule-1 rerun, never a decision.