Harwinder Sekhon
02589cbb88
CuPy from pip had never compiled a kernel on the dev box (no CUDA_PATH for the splayed CUDA-12 wheel layout); the probe swallowed the error and every book to date was built on the NumPy path under a "GPU backtester" name. - backtest/gpu_kernels.py (new): lazily probed, never-silent BACKEND state (CUDA_PATH pointed at the nvidia-cuda-runtime-cu12 header wheel; the probe compiles a kernel and keeps the failure reason), --backend auto|gpu|cpu and FB2_BACKEND, and three CUDA kernels compiled with --fmad=false: ewm_batch (the _gpu_ewm_alpha recurrence, one thread per series), roll_mean_var (pandas 3.0.2 roll_mean/roll_var replica, Kahan + Welford + unstable recompute), resolve_trades (trailing and static exits, one thread per trade). - gpu_backtest.py: precompute_indicators and simulate_trades dispatch to the GPU twins or to the untouched NumPy paths; caches are host NumPy and device memory is released before the replay workers fork; the Backend: line names the device or the fallback reason. - test_all.py Module 36 (7 tests): a visible CUDA device must never fall back silently; kernels vs their CPU references on synthetic edge cases; backend selection; import guards (cupy banned from the live roots). Suite 536. - CLAUDE.md, instructions_for_backtest.txt, research note with the evidence. Verification: all five reference MD5s reproduced on the GPU and with --backend cpu (ten runs, GPU cache fallbacks 0); precompute GPU == CPU on all 42 cached windows (2.34 M bars); suite green with GPU, with CUDA_VISIBLE_DEVICES="" and with FB2_BACKEND=cpu; --workers 1 and --record pairs equal. The per-bar replay stays the live Python signal code on CPU workers by design; wall time is roughly unchanged (precompute ~3x faster).
49 lines
3.4 KiB
Plaintext
49 lines
3.4 KiB
Plaintext
Backtest reporting checklist — ForexBot2 (XAU/USD)
|
|
==================================================
|
|
Every backtest report must record:
|
|
- The exact command line (window, --primary-tf, --spread, and any --set /
|
|
--quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
|
|
- Trades, total R, win rate, profit factor and max drawdown for the whole
|
|
window AND for both halves, at both cost conventions ($0.65 measured-Oanda
|
|
headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
|
|
- Per trade (the exported CSV already carries these): entry/exit time and
|
|
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
|
indicator's signal / value / agreement at decision time.
|
|
- Breakdowns by session/hour, weekday, regime and volatility regime, and,
|
|
for any candidate change, the removed/added/shifted trade diff against the
|
|
control book (the 5-bar cooldown displaces entries, so judge run-level
|
|
numbers, never the added-trade bucket alone).
|
|
- The md5 of the exported book whenever the run is a parity or regression check.
|
|
Replays go through the real signal code (analyze_from_dataframes) — never a
|
|
re-implementation.
|
|
|
|
--- 2026-09-06: decision-time feature export ---
|
|
- `--export-features` appends stake_multiplier, r_sized and the 65 `f_*` decision-time
|
|
features (position_sizing.FEATURE_ORDER) AFTER the 63 pinned columns; default export is
|
|
byte-identical to the pinned books.
|
|
- Any sizing report must show the FLAT book and the SIZED book side by side and the
|
|
label-shuffle null; the 2026-09-06 campaign verdict (nothing ships,
|
|
backtest/research/sizing_campaign_2026-09.md) is the reference.
|
|
|
|
--- 2026-09-09: replay recording (--record DIR [--record-bars]) ---
|
|
- `--record DIR` writes manifest.json + trades/indicators/events parquet for every ACCEPTED trade:
|
|
every indicator's signal / value / confidence / BASE weight / regime multiplier / EFFECTIVE
|
|
weight / contribution on the primary AND both confirmation timeframes, penalties, the setup, the
|
|
outcome, and the Forex Factory calendar events around the trade (backtest/cache/calendar.parquet,
|
|
refreshed with scripts/ff_calendar_scraper.py). `--record-bars` adds one row per ANALYZED primary
|
|
bar with the same votes plus forward moves (1h/4h/24h in ATR units).
|
|
- Recording is read-only w.r.t. the signal path: any recorded run must be paired with a control
|
|
run (same CLI without --record) and the report must show the two books' MD5s equal
|
|
(manifest.book_csv_md5 is the md5 of the 63-column book).
|
|
- Scorecards and "what-if" tables from these files (backtest/replay_scorecard.py) are IN-BOOK
|
|
descriptive statistics: they cannot add trades a change would create and ignore the gates.
|
|
Any weight / indicator change they suggest is a hypothesis for a full rule-1 rerun, never a decision.
|
|
|
|
--- 2026-09-18: GPU backend (backtest/gpu_kernels.py) ---
|
|
- The "Backend:" line now names the CUDA device ("CuPy GPU (<device>, cupy <version>)") or the
|
|
reason for the NumPy fallback; quote it verbatim. `--backend cpu` forces the NumPy twins, `gpu`
|
|
refuses to fall back. Books are byte-identical on both backends (reference MD5s, CLAUDE.md §9).
|
|
- Any change to a kernel or to its CPU twin is a parity event: re-run Module 36, regenerate the
|
|
five reference books on BOTH backends and show the ten MD5s unchanged. Never replace a kernel
|
|
with a differently ordered algorithm (parallel scan, two-pass variance) even if it is faster.
|