harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

backtest: full-GPU research twin (cudaBasedBacktesting.py + cuda_signal_kernels.py); books byte-identical to gpu_backtest.py (2026-09-18)

Browse Source 
Owner decisions: research only (live scans stay on the CPU code), gpu_backtest.py
stays the base reference, every research flag supported, the one exemption from
instructions item 6 recorded in backtest/instructions_for_backtest.txt.

- backtest/cuda_signal_kernels.py: the CUDA decision stage. K1 = engine.py + the
  GPUTechnicalEngine overrides (one thread per bar), K2 = analyze_from_dataframes
  end to end incl. gates 6b-6i, the four setup generators and the regime filter
  (one thread per primary bar), K3 = position_sizing.build_features, K4 cooldown,
  K5 prefix median, K6 recorder bar rows. Config packer with generated #defines;
  leaf module; --fmad=false; Python evaluation order kept (ternary min/max,
  pairwise sums, Neumaier sum(), exact round(x, 2)).
- backtest/cudaBasedBacktesting.py: same CLI, cache files and writers as the
  reference; the host turns kernel codes into the exact strings; no CPU path.
- test_all.py Module 37 (16 tests): every stage pinned against the live code;
  suite 552 / 552 on the GPU, without CUDA and with FB2_BACKEND=cpu.
- Proof: five reference MD5s, four 6-month acceptance books (M5/M15, $0.65/$0.30),
  M15 OOS --export-features and --record --record-bars tables, ten research-flag
  variants, and the owner's five-year M5 and M15 books, all byte-identical to
  gpu_backtest.py. Wall time 0.8-6.6 s per book against 13-353 s.
- CLAUDE.md sections 1, 3, 6, 7, 9, 10, 11 and the instructions addendum: any
  signal change is now a parity event for the twin.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
2026-09-18 16:11:47 -04:00
parent bac11d81eb
commit 1e8110ea5e
8 changed files with 5602 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backtest/research/cuda_backtester_2026-09-18.md
+215
View File
@@ -0,0 +1,215 @@
# Full-GPU research twin of the backtester — 2026-09-18
Behaviour-neutral by construction and by proof. `backtest/cudaBasedBacktesting.py` runs every stage
of a backtest on the GPU, including the per-bar decision that `gpu_backtest.py` still runs through
the live Python signal code. Nothing in the signal logic, the weights, the gates or the exit rules
moved; the proof is that both scripts export byte-identical books (same CSV MD5) on every window
and every research flag tried, listed in section 3.
## 1. Owner decisions
- Research and backtesting only. Live scans stay on the CPU signal code; the Telegram bot, the CLI
and the dashboard never import the new files (Module 37 sweeps the live roots).
- `gpu_backtest.py` stays in the tree as the **base reference**. The twin must reproduce its
exported book byte for byte for the same arguments; the MD5 of the CSV is the acceptance gate.
- `--record`, `--record-bars`, `--export-features` and `--fixed-rr-tp` all work on the GPU path.
- Acceptance: 6-month M5 and M15 books, window 2026-03-18 → 2026-09-18, both scripts, identical
CSV MD5 at $0.65 (headline) and $0.30 (comparability). That window was not cached; the first
control run fetched M5/M15/H1 from Oanda (padded 62 days, `oanda_XAU_USD_<tf>_20260115_20260918`)
and both scripts read the same files afterwards.
- `backtest/instructions_for_backtest.txt` item 6 ("replays go through the real signal code, never
a re-implementation") is waived for this ONE research script; the dated addendum in that file
records the exemption and the new obligation: any signal change is a parity event, re-prove the
MD5 with both scripts before a number from the twin counts as evidence.
## 2. What shipped
| File | Role |
|---|---|
| `backtest/cudaBasedBacktesting.py` (1,028 lines) | Same CLI as the reference (it reuses `_parse_args`), same cache files, same report and CSV writers. Reuses by import everything that is not the replay: the loader and its cleaning, `precompute_indicators` (GPU), the trade resolver (GPU), `_reanchor_entries_to_raw`, `_apply_fixed_rr_tp`, `compute_metrics`, `print_report`, `trades_to_dataframe`, `replay_record.write_run`. Replaces `_replay_span` + the worker pool + `_apply_cooldown` with kernel launches and host builders that turn the kernels' numeric codes into the exact strings (indicator descriptions, warnings, reasons) with the same f-string templates as `engine.py` / `signal_aggregator.py`, then call the real `SignalAggregator._clean_accepted_cp_warnings` and `_drop_stale_spread_gate_vetoes`. No CPU path: `--backend cpu`, `FB2_BACKEND=cpu` or a failed probe exit with a message that points to `gpu_backtest.py`; configurations beyond the compile-time limits (more than 4 confirmation frames, divergence lookback ≥ 64, swing lookback ≥ 256, `cp_lookback` ≥ 128, a big-candle window over 500 bars) are refused the same way. |
| `backtest/cuda_signal_kernels.py` (2,914 lines, most of it CUDA C in strings) | K1 `fb2_analyze_tf` (one thread per bar of one timeframe; the nine `GPUTechnicalEngine` overrides plus the live engine code they do not cover: divergence, structure S/R, candlestick, regime, regime-aware RSI, composite, the `len < 30` empty analysis, `ema_mode="2ema"`). K2 `fb2_aggregate` (one thread per primary bar; `analyze_from_dataframes` end to end: the multi-timeframe merge, bonuses, penalties in the post-sort decay order with the 30 cap, exhaustion block, signal, grade, gates 6b–6i, the XAU_ATR / GENERIC / XAU_PB / XAU_CP generators, the NEUTRAL-at-grade-C direction, `_apply_regime_filter_live`, plus the replay's quality / strategy filters, next-open entry and distance-preserving re-anchor). K3 `fb2_features` (the 65 `position_sizing.build_features` columns per accepted trade). K4 cooldown (single thread, the only cross-bar rule). K5 prefix median (the snapshot's `spread_estimate`). K6 recorder bar rows (the 160-column float32 row). The config packer (`PARAMS_F` / `PARAMS_I` name lists → generated `#define`s, so a name cannot drift), the classification table of every config field the kernels do not read, and thin test launches of the device helpers. Compiled with `--fmad=false`; imports only numpy, `gpu_kernels`, `config`, `chart_patterns` (pattern priority order) and `position_sizing` constants. |
| `test_all.py` Module 37 (16 tests) | Section 4. |
| `backtest/instructions_for_backtest.txt` | The 2026-09-18 CUDA-twin addendum. |
What stays on the host, on purpose: loading and cleaning the parquet cache; the calendar and
session flags (pandas time-zone arithmetic over the whole index: NY weekend, UTC weekday, blocked
close hours, and the three clocks the gates use: bar label for gate 6b, bar close for gate 6g3 and
XAU_CP, decision time for the replay skip); the text; the writers.
Rules that keep the twins bit-identical (they cost most of the debugging time and are now pinned
by tests): doubles everywhere; Python `min()` / `max()` as argument-order ternaries, never
`fmin` / `fmax`; every sum in Python's iteration order; numpy's pairwise summation (8 accumulators,
128-blocks) for `np.mean` / `np.nanmean`; Python 3.12's Neumaier-compensated builtin `sum()` for
the multi-timeframe score mean; pandas' online rolling mean and EWM recurrences copied from
`gpu_kernels.py` for the restarted tails the features use; `round(x, 2)` reproduced exactly with an
error-free product (the host additionally re-checks Python's `round()` on the device doubles of
every accepted trade); `atr_percentile = count / 50 * 100` (not `count * 2`: 15/50·100 =
30.000000000000004 flips "low" to "normal"); the as-of alignment compares int64 nanoseconds.
## 3. Evidence
Environment: NVIDIA GeForce RTX 4070 SUPER, CuPy 14.0.1 (`cupy-cuda12x` with the
`nvidia-cuda-runtime-cu12` header wheel), NumPy 2.2.6, pandas 3.0.2, Python 3.12, 16 CPUs. Code:
this working tree on branch `sql-integration` on top of `bac11d8`. `Backend:` lines verbatim:
reference `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, twin
`CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`; both log `GPU cache fallbacks: 0` (the
twin's line adds "no per-bar recompute path in cudaBasedBacktesting").
### 3.1 Acceptance: 6 months, both scripts, both costs
Harness: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --primary-tf <tf> --start
2026-03-18 --end 2026-09-18 --spread <sp> --workers 12 --export-csv <out>` for the control and
`venv/bin/python backtest/cudaBasedBacktesting.py` with the same arguments (no `--workers`) for the
twin; factory config, no overrides.
| Primary | Cost | Control MD5 | Twin MD5 | Equal | Control wall | Twin wall |
|---|---|---|---|---|---|---|
| M5 | $0.65 | `86111c36c1a07e543a65f0cd09d56ebf` | `86111c36c1a07e543a65f0cd09d56ebf` | ✓ | 51.8 s (replay 43.8 s) | 1.27 s |
| M5 | $0.30 | `e70d025205844ecedc2b9e9c08f38f77` | `e70d025205844ecedc2b9e9c08f38f77` | ✓ | 47.1 s (replay 45.9 s) | 1.65 s |
| M15 | $0.65 | `b8c980833f1a32bd59930bd037c75b89` | `b8c980833f1a32bd59930bd037c75b89` | ✓ | 20.4 s (replay 18.7 s) | 1.24 s |
| M15 | $0.30 | `e66b5fbdaace1d3fda4a76a0e2d47d69` | `e66b5fbdaace1d3fda4a76a0e2d47d69` | ✓ | 14.8 s (replay 13.5 s) | 1.50 s |
Because the CSVs are identical the figures below are the same from either script (`NO_DATA` rows
excluded; halves split at 2026-06-18). They are reported because the instructions require them,
not as a validation: nothing changed, and the window is 6 months (rule 1 needs 12).
| Primary | Cost | Segment | Trades | Total R | Win rate | PF | Max DD (R) |
|---|---|---|---|---|---|---|---|
| M5 | $0.65 | whole | 283 | +41.67 | 56.5% | 1.42 | 10.1 |
| M5 | $0.65 | first half | 146 | +29.58 | 58.9% | 1.64 | 6.6 |
| M5 | $0.65 | second half | 137 | +12.09 | 54.0% | 1.23 | 8.3 |
| M5 | $0.30 | whole | 283 | +46.38 | 57.2% | 1.48 | 8.4 |
| M5 | $0.30 | first half | 146 | +32.07 | 60.3% | 1.71 | 6.4 |
| M5 | $0.30 | second half | 137 | +14.30 | 54.0% | 1.28 | 7.4 |
| M15 | $0.65 | whole | 238 | +16.35 | 60.1% | 1.19 | 14.9 |
| M15 | $0.65 | first half | 120 | +14.09 | 63.3% | 1.37 | 7.6 |
| M15 | $0.65 | second half | 118 | +2.26 | 56.8% | 1.05 | 14.4 |
| M15 | $0.30 | whole | 238 | +19.20 | 60.9% | 1.23 | 13.8 |
| M15 | $0.30 | first half | 120 | +15.37 | 65.0% | 1.41 | 7.1 |
| M15 | $0.30 | second half | 118 | +3.82 | 56.8% | 1.08 | 13.8 |
By generator at $0.65: M5 GENERIC 163 trades +42.4R, XAU_CP 104 −2.7R, XAU_ATR 16 +2.0R (BUY 244
+25.7R, SELL 39 +16.0R); M15 GENERIC 149 +26.3R, XAU_ATR 89 −10.0R (BUY 144 +9.7R, SELL 94
+6.7R). The M15 second half is the same weak stretch the 2026-09-17 live review saw; no decision
follows from a 6-month window.
### 3.2 The five reference books (kept cache, `--spread 0.65`, factory config)
| Book | Window | Twin MD5 | Pinned (CLAUDE.md §9) | Twin wall |
|---|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | `49f8e602fc6ae769713c94ef7b257810` | ✓ | 2.4 s (reference 114.8 s) |
| M15 W2 | 2024-09-05 → 2026-09-05 | `b4aa07e33e17e23774c486a6900d673b` | ✓ | 2.3 s (37.2 s) |
| M15 OOS | 2024-06-01 → 2025-03-30 | `9fd624fb387c36052122bd1830cf9c32` | ✓ | 1.5 s (15.6 s) |
| H1 W3 | 2023-09-05 → 2026-09-05 | `7b848c917dcb5c371ecbb9f8f7292278` | ✓ | 1.3 s (13.0 s) |
| H4 W3 | 2023-09-05 → 2026-09-05 | `186e677fbbd53cca735c94094c23ce05` | ✓ | 0.8 s (21.2 s) |
Reference wall times are the GPU-backend rows of `gpu_backend_2026-09-18.md`. On the twin the
decision stage itself is milliseconds (M5 W2: K1 218,088 bars of three timeframes in 0.020 s, K2
141,819 primary bars in 0.002 s); what remains is loading and cleaning the parquet files, the
indicator precompute (1.3 s on the two-year M5 window), the host formatting of the accepted trades
and the CSV write.
### 3.3 Research flags and knobs (M15 OOS window unless stated; both scripts, `--spread 0.65`)
| Variant | Rows | Twin MD5 (prefix) | Equal |
|---|---|---|---|
| `--export-features` (130 columns) | 355 | `6239d197` | ✓ |
| `--record DIR --record-bars` | 355 | book `9fd624fb`; `trades` (355 × 138), `indicators` (8,120 × 15), `events` (6,902 × 14), `bars` (12,315 × 172) parquet tables equal via `DataFrame.equals`; manifests differ only in `cli`, `created_at`, `replay_seconds`, `workers` | ✓ |
| `--fixed-rr-tp 2.0` | 355 | `95a7d2ab` | ✓ |
| `--quality A+,A` | 249 | `1883c346` | ✓ |
| `--strategies XAU` | 3 | `c07a4025` | ✓ |
| `--cp` (all six chart patterns on M15) | 396 | `7ff51617` | ✓ |
| `--big-candle 3.0` | 104 | `76f76f3b` | ✓ |
| `--set backtest.regime_filter=true` (H4 ADX gate, a fourth frame) | 179 | `0a65af6b` | ✓ |
| `--set technical.ema_mode=2ema` | 368 | `faaa7e86` | ✓ |
| M5, `--set xau_strategy.m5_pullback_enabled=true --quality A+,A,B,C,D` | 279 | `1811117a` | ✓ |
| `--trail-arm 0.5 --trail-dist 0.9 --cooldown-bars 3 --max-hold-bars 150` | 414 | `9665d753` | ✓ |
| H1, `--swing-sr-lookback 80 --warmup-bars 300` | 75 | `f8ef2b9f` | ✓ |
One variant could not run on either script: H1 with `--confirm-tfs 15min,4h,1d` on the OOS
window needs a D1 file padded 1,400 days (2020-08-01) that is not cached, and Oanda returned an
empty page for that start; both scripts fail identically in `fetch_range`. Not a parity issue.
### 3.4 Test suite, three ways (final tree)
| Run | Result | Wall | Module 37 note |
|---|---|---|---|
| default (GPU) | `RESULTS: 552 passed, 0 failed, 552 total` | 116 s | every device test exercised |
| `CUDA_VISIBLE_DEVICES=""` | `RESULTS: 552 passed, 0 failed, 552 total` | 30 s | `CUDA unavailable here: NumPy CPU fallback (CUDARuntimeError: cudaErrorNoDevice: no CUDA-capable device is detected)` printed; device tests print a notice |
| `FB2_BACKEND=cpu` | `RESULTS: 552 passed, 0 failed, 552 total` | 31 s | `NumPy CPU fallback (forced: --backend cpu / FB2_BACKEND=cpu)` printed; the twin's refusal of a cpu-forced backend is exercised for real |
536 tests before this change, 16 added by Module 37.
## 4. Module 37 (what each test pins)
1. `fb2_round2` / `round1` against Python `round(x, 2)` / `round(x, 1)` on 400k doubles including
every `.xx5` half in ±300 and signed zero; without CUDA the rule is checked in exact arithmetic.
2. Parameter packing round-trips every decision leaf of the four factories (M5 blocked hours with
9 and without 21, M15 SELL hours 15/17 and the RSI × 0.5 table, H1 empty SELL quality, M15
`tp_floor_rr` 1.0, the six-regime × nine-indicator table, `FB2_EMA_LO` read once with the same
fallback); every field of the four config dataclasses is either packed or classified host-only,
so a new field fails the suite until it is placed; the `#define`s are generated from the lists.
3. K1 against `GPUTechnicalEngine.analyze` on every bar of a 600-bar fixture (doji, hammer,
shooting star, engulfing, marubozu, a flat stretch, plateaus) for 5min/15min/1h/4h under their
factories, partial-EMA off, `2ema`, classic RSI, disabled indicators and a 150-bar frame: valid
flag, composite, regime, volatility, every indicator's value / signal / confidence / description
/ effective weight / weighted score, key levels, pattern signals.
4. The arithmetic twins: numpy pairwise sums and `Series.mean` for n = 1…600, `nanmean` with NaNs,
EWM and rolling means restarted on tails, Wilder ATR on tails, `_ext_vs_sma`, the prefix median
with ties, CPython float floor division; and the rolling helper text is the prefix of
`gpu_kernels._ROLL_SRC`.
5. `align_cut` against the replay's `searchsorted` rule on gapped, late-starting M15/H1/H4/D1
frames for M5 and H1 primaries (`cut == 0` included).
6. `host_flags` against `_is_xau_blocked_by_calendar` plus the session block bar by bar across
both 2026 DST shifts, an H4 primary (label hour ≠ close hour), the day filter, and each filter
switched off.
7. K2 against `analyze_from_dataframes` on every analysed bar: the M5 factory over a 21-day
synthetic fixture (4,046 bars: all six grades, GENERIC and XAU_CP setups, both directions, the
mixed `trending` regime with both score signs, 23 distinct warnings); the M5 research knobs
(all six chart patterns, big-candle 3.0, the H4 regime filter, `FB2_EMA_LO=0.9`, grades to D;
every second bar); the M15 factory (1,182 bars, 62 candidates, XAU_ATR and GENERIC, the SELL,
exhaustion-block and spread gates); the H1 factory (shorts blocked) and the H4 factory with a
D1 confirmation shorter than 30 bars on a 30-day fixture. Compared per bar: score, confidence,
signal, grade, agreement, exhaustion count, momentum fading, penalty total and tags, the full
warning and cleaned-warning lists, both reason lists, price, spread estimate, regime,
volatility, the setup (strategy, direction, entry, stop, targets, rounded RR, pattern) and the
replay's candidate fields.
8. The device chart-pattern detector against `detect_chart_pattern` with all six patterns on the
fixture and on its mirror image (bear patterns), zero ATR included; the device big-candle
detector against `_detect_big_candle` (index and ratio) with injected abnormal bars.
9. K4 against `_apply_cooldown` (cooldown 0 / 1 / 5 / 17, empty, a dense run).
10. `Trade` rows, the 65 features, the recorder snapshots and the per-bar float32 matrix against a
real `_replay_span` over the whole M5 fixture with the pullback path re-enabled (XAU_PB, XAU_CP
and GENERIC trades; `dataclasses.asdict` field by field, exact floats, tz-aware entry times, the
CSV frame, dtypes and bytes with the research columns).
11. End to end: `cudaBasedBacktesting.main` against `gpu_backtest.main` on the fixture written to a
temporary cache directory (M15, two weeks): plain book MD5 equal; `--fixed-rr-tp 2.0
--export-features --record DIR --record-bars` MD5 equal and the four parquet tables equal.
12. Refusals: `--backend cpu`, a cpu-forced backend and a failed probe exit before the cache is
touched; each compile-time limit is refused with a message naming the knob and pointing to
`gpu_backtest.py`, the largest supported values pass; the parser is the reference script's;
`require_gpu_or_exit("gpu")` and the `Backend:` text on a working box.
13. Source guards: the script never calls `analyze_from_dataframes` or `_replay_span`, never
imports cupy or a live writer; the kernel module stays a leaf (no `engine`,
`signal_aggregator`, `gpu_backtest`, `replay_record`, live writer or ML import at module level),
pins `options=KERNEL_OPTIONS` (`--fmad=false`) and contains no `fmin(` / `fmax(`; the shared
code tables equal the recorder's; the live roots import neither new module.
On the GPU the module takes about 85 s, almost all of it the CPU references (about 5 ms per bar
through the live signal code). Without CUDA the device tests print a notice and the module takes
about a second.
## 5. Honest notes
- The twin is a second implementation of the decision stage. The MD5 gate proves equality on the
windows and flags tried, not in general; a future signal change that is not mirrored in the
kernels will make the twin drift silently. That is why the instructions addendum and CLAUDE.md
section 7 make every signal change a parity event, and why the reference script stays.
- Speed comes from the batch, not from the per-bar work: the decision kernels run one thread per
bar over the whole window at once. The remaining wall time (about a second per book) is parquet
loading, cleaning, the precompute and the host formatting of the accepted trades.
- `--workers` is accepted and ignored (logged); the twin never forks.
- Nothing in `engine.py`, `signal_aggregator.py`, `chart_patterns.py`, `position_sizing.py`,
`replay_record.py`, `gpu_backtest.py` or `gpu_kernels.py` changed. `git diff --stat` touches
`test_all.py`, `CLAUDE.md` and `backtest/instructions_for_backtest.txt` and adds the two new
modules and this note.