Harwinder Sekhon
daa2369ccb
Adds an opt-in recorder to the backtester so a replay keeps the evidence each trade setup was decided on: every indicator's vote, value, confidence, base weight, regime multiplier, effective weight and contribution on the primary and both confirmation timeframes, penalties, the setup and outcome, and the Forex Factory calendar events around the trade (4 h before entry to exit, all-day rows on the New-York date). --record-bars adds one float32 row per analyzed bar with the same votes plus 1 h/4 h/24 h forward moves in ATRs. Output is Parquet in a folder per run with a manifest carrying the engine/gate fingerprints, the regime weight table in force and the book MD5. Nothing in the signal core changes: the trade book is byte-identical with recording on or off (control-vs-recorded MD5 equal on both 2-year runs, M5 781 / +146.07R / PF 1.52, M15 793 / +117.62R / PF 1.41 at $0.65). Module 33 adds 12 tests (498 total): snapshot reproduces the engine's composite, absence is no vote, calendar join edges/phases/holiday, write/load round-trip, _replay_span parity with recording on, CLI flags, import and source guards. Also: backtest/replay_scorecard.py (first-look scorecards, calendar buckets, in-book and per-bar what-ifs, --check invariants), scripts/ff_calendar_scraper.py restored from git history (the only way to regenerate the gitignored calendar file), report in backtest/research/replay_recording_2026-09-09.md. Claude-Session: https://claude.ai/code/session_01Nc9Ninzyi18QkWezrxeBvB
41 lines
2.7 KiB
Plaintext
41 lines
2.7 KiB
Plaintext
Backtest reporting checklist — ForexBot2 (XAU/USD)
|
|
==================================================
|
|
Every backtest report must record:
|
|
- The exact command line (window, --primary-tf, --spread, and any --set /
|
|
--quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
|
|
- Trades, total R, win rate, profit factor and max drawdown for the whole
|
|
window AND for both halves, at both cost conventions ($0.65 measured-Oanda
|
|
headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
|
|
- Per trade (the exported CSV already carries these): entry/exit time and
|
|
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
|
indicator's signal / value / agreement at decision time.
|
|
- Breakdowns by session/hour, weekday, regime and volatility regime, and,
|
|
for any candidate change, the removed/added/shifted trade diff against the
|
|
control book (the 5-bar cooldown displaces entries, so judge run-level
|
|
numbers, never the added-trade bucket alone).
|
|
- The md5 of the exported book whenever the run is a parity or regression check.
|
|
Replays go through the real signal code (analyze_from_dataframes) — never a
|
|
re-implementation.
|
|
|
|
--- 2026-09-06: decision-time feature export ---
|
|
- `--export-features` appends stake_multiplier, r_sized and the 65 `f_*` decision-time
|
|
features (position_sizing.FEATURE_ORDER) AFTER the 63 pinned columns; default export is
|
|
byte-identical to the pinned books.
|
|
- Any sizing report must show the FLAT book and the SIZED book side by side and the
|
|
label-shuffle null; the 2026-09-06 campaign verdict (nothing ships,
|
|
backtest/research/sizing_campaign_2026-09.md) is the reference.
|
|
|
|
--- 2026-09-09: replay recording (--record DIR [--record-bars]) ---
|
|
- `--record DIR` writes manifest.json + trades/indicators/events parquet for every ACCEPTED trade:
|
|
every indicator's signal / value / confidence / BASE weight / regime multiplier / EFFECTIVE
|
|
weight / contribution on the primary AND both confirmation timeframes, penalties, the setup, the
|
|
outcome, and the Forex Factory calendar events around the trade (backtest/cache/calendar.parquet,
|
|
refreshed with scripts/ff_calendar_scraper.py). `--record-bars` adds one row per ANALYZED primary
|
|
bar with the same votes plus forward moves (1h/4h/24h in ATR units).
|
|
- Recording is read-only w.r.t. the signal path: any recorded run must be paired with a control
|
|
run (same CLI without --record) and the report must show the two books' MD5s equal
|
|
(manifest.book_csv_md5 is the md5 of the 63-column book).
|
|
- Scorecards and "what-if" tables from these files (backtest/replay_scorecard.py) are IN-BOOK
|
|
descriptive statistics: they cannot add trades a change would create and ignore the gates.
|
|
Any weight / indicator change they suggest is a hypothesis for a full rule-1 rerun, never a decision.
|