From b357ab1e87c2411d414f8b7b0bc5026962097cbe Mon Sep 17 00:00:00 2001 From: Harwinder Sekhon Date: Fri, 7 Aug 2026 18:21:39 -0400 Subject: [PATCH] backtest: walk-forward mode + extended-range data safety MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --walk-forward slices ONE continuous replay into rolling train/test folds by entry time (default 12mo/3mo/3mo; no per-fold re-optimization) and reports per-fold PF / expectancy / max DD plus aggregated OOS stats, flagging test folds with PF < 1. Off by default — default runs are byte-identical (scripts/compare_baseline.py 32/32 PASS with this diff). Data safety for deep-history fetches: fetch_history refuses to recreate any baseline-pinned parquet filename and records the sha256 of every freshly written parquet in backtest/cache/data_manifest.json. Tests: Module 23 (fold builder, slicing, flagging, OOS dedup, CLI defaults, pin guard, manifest recording) — 367 total. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_017FBjfomAipxvPWwRJEDDyJ --- CLAUDE.md | 8 +- backtest/gpu_backtest.py | 253 +++++++++++++++++++++++++++++++++++++++ test_all.py | 241 +++++++++++++++++++++++++++++++++++++ 3 files changed, 500 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 85748c9..dc43601 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest ## Commands ```bash -python test_all.py # QA suite: 359 tests, 22 modules, no network (mocked) +python test_all.py # QA suite: 367 tests, 23 modules, no network (mocked) python main.py --pair XAUUSD --timeframe H1 --detail # CLI signal report (--json for full dump) python telegram_bot.py # the bot (refuses to boot without credentials) python oanda_daemon.py # auto-execution daemon on Oanda practice account @@ -26,6 +26,10 @@ python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-0 # --warmup-bars --cooldown-bars --no-cache --export-csv out.csv --detail # --fixed-rr-tp N (research-only: hard TP at N×risk, trail off; verdict # 2026-08-05: NOT shipped — backtest/research/fixedtp_2026-08-05.md) +# --walk-forward [--wf-train-months 12 --wf-test-months 3 --wf-step-months 3] +# rolling-fold OOS report over the --start/--end window (one continuous +# replay sliced by entry time; fixed config, no re-optimization; test +# folds with PF<1 flagged). Off by default — default runs byte-identical. ``` **`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. When patching weekend logic in tests, patch `providers.base._is_forex_weekend` — but know there are THREE independent weekend implementations (`providers/base.py`, `signal_aggregator.py`, and an inline copy in `backtest/gpu_backtest.py:_is_xau_blocked_by_calendar`); patching the provider one does not reach the other two. @@ -62,7 +66,7 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz - **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups. - **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`). - **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. News/calendar rows are clickable (detail dialog); headlines carry `id` = sha1(headline) in news.json, and exactly one on-demand endpoint exists: `static/news_summary.php` (installed by `install_static`), which serves a DeepSeek gold-impact summary per headline — whitelist-gated against news.json ids, cache-forever in `ai/`, own fail-closed daily cap (150/day, file-based, separate from the publisher's), key at `/etc/gold-dashboard/deepseek.key`, never reads repo config. Deployment checklist: `deploy/README-dashboard.md`. -- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record. +- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window; `fetch_history` refuses to (re)write any filename pinned by a `baseline/*/manifest.json` (a cache miss there means the reference file was deleted — restore it, don't refetch) and records the sha256 of every freshly fetched parquet in `backtest/cache/data_manifest.json` (same `cache_files` format as the baseline manifest). `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record. ## Rules for making changes (read before editing signal logic) diff --git a/backtest/gpu_backtest.py b/backtest/gpu_backtest.py index 213144d..d749422 100644 --- a/backtest/gpu_backtest.py +++ b/backtest/gpu_backtest.py @@ -25,11 +25,14 @@ Usage: python backtest/gpu_backtest.py --pair XAUUSD --days 365 python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-01 python backtest/gpu_backtest.py --pair XAUUSD --days 180 --no-cache + python backtest/gpu_backtest.py --pair XAUUSD --start 2021-07-30 --end 2026-07-30 \ + --walk-forward # rolling 12mo-train/3mo-test folds, step 3mo """ from __future__ import annotations import argparse +import hashlib import json import logging import multiprocessing as mp @@ -972,6 +975,65 @@ class GPUTechnicalEngine(TechnicalEngine): CACHE_DIR = Path(__file__).resolve().parent / "cache" +#: Root of the byte-for-byte reference set (scripts/capture_baseline.py). +BASELINE_ROOT = PROJECT_ROOT / "baseline" + + +def _sha256_file(path: Path) -> str: + h = hashlib.sha256() + with open(path, "rb") as f: + for chunk in iter(lambda: f.read(1 << 20), b""): + h.update(chunk) + return h.hexdigest() + + +def _baseline_pinned_names() -> set: + """Parquet filenames claimed by any baseline//manifest.json. + + These are the pinned inputs of the byte-for-byte reference runs + (scripts/compare_baseline.py). fetch_history must never write one: + the write path only runs on a cache miss, so reaching it for a pinned + name means the original file was deleted — and recreating it from a + live fetch would put NEW bars under the baseline's filename, silently + poisoning the reference set. Extended-range windows always resolve to + different filenames (the cache key embeds start/end), so they land in + separate parquets and never collide with the pinned set. + """ + names: set = set() + if not BASELINE_ROOT.is_dir(): + return names + for manifest in BASELINE_ROOT.glob("*/manifest.json"): + try: + names.update(json.loads(manifest.read_text()).get("cache_files", {})) + except (OSError, json.JSONDecodeError): + continue + return names + + +def _record_in_data_manifest(path: Path) -> None: + """Record a freshly written parquet's sha256 in cache/data_manifest.json. + + Uses the baseline manifest's ``cache_files`` mapping format + ({filename: sha256}) so extended-range data files carry the same + provenance as the pinned baseline set. Best-effort: a manifest write + failure must not fail the fetch that produced the data. + """ + manifest_path = CACHE_DIR / "data_manifest.json" + try: + manifest = json.loads(manifest_path.read_text()) + except (OSError, json.JSONDecodeError): + manifest = {} + files = manifest.get("cache_files", {}) + files[path.name] = _sha256_file(path) + manifest["cache_files"] = dict(sorted(files.items())) + manifest["updated_utc"] = datetime.now(timezone.utc).strftime( + "%Y-%m-%dT%H:%M:%SZ") + try: + manifest_path.write_text( + json.dumps(manifest, indent=2, sort_keys=True) + "\n") + except OSError as exc: + logger.warning(f"data_manifest.json not updated: {exc}") + def _cache_path( symbol: str, tf: Timeframe, start: datetime, end: datetime, @@ -1031,7 +1093,15 @@ def fetch_history( df = provider.fetch_range(symbol, timeframe, start, end) if use_cache: + if path.name in _baseline_pinned_names(): + raise RuntimeError( + f"refusing to write {path.name}: this filename is pinned by a " + f"baseline manifest under baseline/ and freshly fetched bars " + f"are not the baseline's bars. Restore the original parquet " + f"(git/backup) instead of refetching, or use a different " + f"--start/--end window.") df.to_parquet(path) + _record_in_data_manifest(path) logger.info(f"Cached {len(df)} raw bars to {path.name}") return provider.validate_and_clean(df, symbol, timeframe) if clean else df @@ -2157,6 +2227,154 @@ def print_trade_reasoning(trades: List[Trade], limit: int = 5) -> None: print(f" MFE / MAE: {t.mfe_r:.2f}R / {t.mae_r:.2f}R") +# ═════════════════════════════════════════════════════════════════════ +# WALK-FORWARD ANALYSIS (--walk-forward) +# ═════════════════════════════════════════════════════════════════════ +# +# Rolling-fold out-of-sample evaluation of ONE continuous replay: the +# window is replayed exactly once with the run's fixed config (there is +# no per-fold re-optimization — nothing here retunes anything), then the +# trade book is sliced into rolling train/test folds by ENTRY time. +# Slicing a single replay keeps the book identical to a normal run of the +# same window: warmup happens once and the entry cooldown crosses fold +# boundaries exactly as it would live. Train columns are in-sample +# context; the aggregated stats over all test folds are the OOS read. + +def _add_months(dt: datetime, months: int) -> datetime: + """Calendar-month arithmetic (tz preserved; clamps at month end).""" + return (pd.Timestamp(dt) + pd.DateOffset(months=months)).to_pydatetime() + + +def build_walkforward_folds( + start: datetime, end: datetime, + train_months: int, test_months: int, step_months: int, +) -> List[Tuple[datetime, datetime, datetime, datetime]]: + """Rolling (train_start, train_end, test_start, test_end) folds. + + test_start == train_end; a fold is emitted only if it fits entirely + inside [start, end]. Returns [] when the window is too short. + """ + if min(train_months, test_months, step_months) <= 0: + raise ValueError("train/test/step months must all be > 0") + folds = [] + i = 0 + while True: + tr_s = _add_months(start, i * step_months) + tr_e = _add_months(tr_s, train_months) + te_e = _add_months(tr_e, test_months) + if te_e > end: + break + folds.append((tr_s, tr_e, tr_e, te_e)) + i += 1 + return folds + + +def walkforward_stats( + trades: List[Trade], + folds: List[Tuple[datetime, datetime, datetime, datetime]], +) -> Dict: + """Per-fold train/test Metrics + deduped aggregated OOS stats. + + Fold membership is by entry time (half-open [lo, hi)); a trade's + resolution may extend past its fold end, matching how a live account + would hold through the boundary. With step < test length the test + windows overlap — the OOS aggregate counts each trade exactly once, + in chronological order. + """ + fold_stats: List[Dict] = [] + oos_trades: List[Trade] = [] + seen_ids: set = set() + for k, (tr_s, tr_e, te_s, te_e) in enumerate(folds, 1): + lo_tr, hi_tr = pd.Timestamp(tr_s), pd.Timestamp(tr_e) + lo_te, hi_te = pd.Timestamp(te_s), pd.Timestamp(te_e) + test_slice = [t for t in trades if lo_te <= t.entry_time < hi_te] + train_m = compute_metrics( + [t for t in trades if lo_tr <= t.entry_time < hi_tr]) + test_m = compute_metrics(test_slice) + fold_stats.append({ + "fold": k, + "train_start": tr_s, "train_end": tr_e, + "test_start": te_s, "test_end": te_e, + "train": train_m, "test": test_m, + # PF < 1.0 on the fold's OOS slice; empty folds are not flagged + # (no evidence either way) but are visible as n=0 in the table. + "flagged": test_m.n_trades > 0 and test_m.profit_factor < 1.0, + }) + for t in test_slice: + if id(t) not in seen_ids: + seen_ids.add(id(t)) + oos_trades.append(t) + oos_trades.sort(key=lambda t: t.entry_time) + return { + "folds": fold_stats, + "oos": compute_metrics(oos_trades), + "n_flagged": sum(1 for f in fold_stats if f["flagged"]), + } + + +def _fmt_pf(m: Metrics) -> str: + if m.n_trades == 0: + return "—" + return "inf" if np.isinf(m.profit_factor) else f"{m.profit_factor:.2f}" + + +def print_walkforward_report( + bt_cfg: BacktestConfig, stats: Dict, + train_months: int, test_months: int, step_months: int, +) -> None: + bar = "═" * 78 + folds = stats["folds"] + print(f"\n{bar}") + print(f" WALK-FORWARD ANALYSIS " + f"({bt_cfg.start.date()} → {bt_cfg.end.date()})") + print(f" Train {train_months}mo / Test {test_months}mo / " + f"Step {step_months}mo → {len(folds)} folds " + f"(fixed config — no per-fold re-optimization)") + print(f"{bar}") + + rows = [] + for f in folds: + tr, te = f["train"], f["test"] + rows.append([ + str(f["fold"]), + f"{f['train_start']:%Y-%m-%d}→{f['train_end']:%Y-%m-%d}", + str(tr.n_trades), _fmt_pf(tr), + f"{tr.avg_r:+.3f}" if tr.n_trades else "—", + f"{tr.max_drawdown_r:.2f}" if tr.n_trades else "—", + f"{f['test_start']:%Y-%m-%d}→{f['test_end']:%Y-%m-%d}", + str(te.n_trades), _fmt_pf(te), + f"{te.avg_r:+.3f}" if te.n_trades else "—", + f"{te.max_drawdown_r:.2f}" if te.n_trades else "—", + "⚠ PF<1" if f["flagged"] else "", + ]) + print("\n ── Rolling folds (expR = expectancy per trade, " + "DD = max drawdown in R) ──") + print(_fmt_table( + ["Fold", "Train window", "n", "PF", "expR", "DD", + "Test window", "n", "PF", "expR", "DD", "Flag"], rows)) + + o = stats["oos"] + print("\n ── Aggregated out-of-sample (union of test folds, " + "each trade once) ──") + if o.n_trades == 0: + print(" No trades in any test fold.") + else: + print(_fmt_table(["Metric", "Value"], [ + ["OOS trades", f"{o.n_trades}"], + ["Win rate", f"{o.win_rate:.1f}%"], + ["Total R", f"{o.total_r:+.2f}"], + ["Expectancy / trade", f"{o.avg_r:+.3f}"], + ["Profit factor", _fmt_pf(o)], + ["Max DD (R)", f"{o.max_drawdown_r:.2f}"], + ["Folds flagged PF<1", f"{stats['n_flagged']} / {len(folds)}"], + ])) + if stats["n_flagged"]: + flagged = [str(f["fold"]) for f in folds if f["flagged"]] + print(f"\n ⚠ {stats['n_flagged']} fold(s) with out-of-sample " + f"PF < 1.0: fold {', '.join(flagged)}") + print(f"{bar}\n") + + def trades_to_dataframe(trades: List[Trade]) -> pd.DataFrame: """Export trades for downstream analysis (e.g. parquet/CSV). @@ -2217,6 +2435,19 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace: p.add_argument("--days", type=int, default=180, help="Days of history (ignored if --start/--end set)") p.add_argument("--start", help="ISO date YYYY-MM-DD (overrides --days)") p.add_argument("--end", help="ISO date YYYY-MM-DD (defaults to today)") + p.add_argument("--walk-forward", action="store_true", + help="Walk-forward analysis: replay the window once with " + "the run's fixed config, then slice the book into " + "rolling train/test folds and report per-fold PF / " + "expectancy / max DD / trade count plus aggregated " + "out-of-sample stats (folds with test PF < 1.0 are " + "flagged). No per-fold re-optimization.") + p.add_argument("--wf-train-months", type=int, default=12, + help="Walk-forward: train window length (calendar months)") + p.add_argument("--wf-test-months", type=int, default=3, + help="Walk-forward: test window length (calendar months)") + p.add_argument("--wf-step-months", type=int, default=3, + help="Walk-forward: step between fold starts (calendar months)") p.add_argument("--primary-tf", default="15min", help="Primary TF (1min/5min/15min/1h/4h)") p.add_argument("--confirm-tfs", default=None, help="Comma-separated confirmation TFs. Default: the pair " @@ -2398,6 +2629,24 @@ def main(argv: Optional[List[str]] = None) -> int: else: start = end - timedelta(days=args.days) + # Fail fast on an unusable walk-forward request — before any data + # loading. With walk-forward off nothing below changes. + wf_folds = None + if args.walk_forward: + try: + wf_folds = build_walkforward_folds( + start, end, args.wf_train_months, + args.wf_test_months, args.wf_step_months) + except ValueError as exc: + raise SystemExit(f"--walk-forward: {exc}") + if not wf_folds: + raise SystemExit( + f"--walk-forward: window {start.date()} → {end.date()} " + f"({(end - start).days} days) is shorter than one fold " + f"(train {args.wf_train_months}mo + test " + f"{args.wf_test_months}mo); pass a longer --start/--end " + f"range.") + spread = (args.spread if args.spread is not None else PAIR_INFO[pair].get("default_spread", 0.30)) @@ -2427,6 +2676,10 @@ def main(argv: Optional[List[str]] = None) -> int: trades, _ = run_backtest(bt_cfg, platform_cfg=platform_cfg) metrics = compute_metrics(trades) print_report(pair, bt_cfg, metrics, trades=trades, detail=args.detail) + if wf_folds: + print_walkforward_report( + bt_cfg, walkforward_stats(trades, wf_folds), + args.wf_train_months, args.wf_test_months, args.wf_step_months) print_trade_reasoning(trades, limit=args.reasoning_samples) if args.export_csv: diff --git a/test_all.py b/test_all.py index acb562a..27af25b 100644 --- a/test_all.py +++ b/test_all.py @@ -6943,6 +6943,247 @@ def _(): assert "must be > 0" in str(e) +# ══════════════════════════════════════════════════════════════════════ +# MODULE 23: walk-forward mode + extended-range data safety +# ══════════════════════════════════════════════════════════════════════ +print("\n═══ Module 23: walk-forward mode + extended-range data safety ═══") + + +def _wf_trade(ts, r): + """Resolved trade at ts (UTC) with realized R — for fold slicing tests.""" + t = _m21_sim_trade(ts=ts) + t.exit_reason = "TP1" if r > 0 else "SL" + t.exit_time = t.entry_time + t.r_multiple = r + t.bars_held = 3 + return t + + +@test("walk-forward: fold builder — 12/3/3 over 24mo = 4 contiguous folds") +def _(): + import gpu_backtest as _gb + start = datetime(2021, 1, 1, tzinfo=timezone.utc) + end = datetime(2023, 1, 1, tzinfo=timezone.utc) + folds = _gb.build_walkforward_folds(start, end, 12, 3, 3) + assert len(folds) == 4, f"expected 4 folds, got {len(folds)}" + for i, (tr_s, tr_e, te_s, te_e) in enumerate(folds): + assert te_s == tr_e, "test must start exactly at train end" + assert tr_s == datetime(2021, 1 + 3 * i, 1, tzinfo=timezone.utc) + assert folds[0][3] == datetime(2022, 4, 1, tzinfo=timezone.utc) + assert folds[-1][3] == end, "last fold must end exactly at window end" + + +@test("walk-forward: fold builder — month-end clamp, short window, bad months") +def _(): + import gpu_backtest as _gb + # Jan 31 + 1 month clamps to Feb 28 (calendar-month arithmetic). + d = _gb._add_months(datetime(2021, 1, 31, tzinfo=timezone.utc), 1) + assert (d.year, d.month, d.day) == (2021, 2, 28) + # 14mo window can't fit train 12 + test 3 → no folds, no exception. + start = datetime(2021, 1, 1, tzinfo=timezone.utc) + assert _gb.build_walkforward_folds( + start, datetime(2022, 3, 1, tzinfo=timezone.utc), 12, 3, 3) == [] + # 18mo window fits exactly two stepped folds. + assert len(_gb.build_walkforward_folds( + start, datetime(2022, 7, 1, tzinfo=timezone.utc), 12, 3, 3)) == 2 + for bad in ((0, 3, 3), (12, 0, 3), (12, 3, -1)): + try: + _gb.build_walkforward_folds(start, start, *bad) + assert False, f"months {bad} must raise ValueError" + except ValueError as e: + assert "must all be > 0" in str(e) + + +@test("walk-forward: fold slicing — half-open boundaries, per-fold metrics") +def _(): + import gpu_backtest as _gb + fold = (datetime(2024, 1, 1, tzinfo=timezone.utc), + datetime(2025, 1, 1, tzinfo=timezone.utc), + datetime(2025, 1, 1, tzinfo=timezone.utc), + datetime(2025, 4, 1, tzinfo=timezone.utc)) + trades = [ + _wf_trade("2024-02-01 10:00", +1.0), # train + _wf_trade("2024-03-01 10:00", -1.0), # train + _wf_trade("2024-06-01 10:00", +2.0), # train + _wf_trade("2025-01-01 00:00", -1.0), # exactly test_start → test + _wf_trade("2025-02-15 10:00", +0.5), # test + _wf_trade("2025-04-01 00:00", +9.0), # exactly test_end → excluded + ] + unresolved = _wf_trade("2025-02-20 10:00", +1.0) + unresolved.exit_reason = "" # never scored + trades.append(unresolved) + stats = _gb.walkforward_stats(trades, [fold]) + f = stats["folds"][0] + tr, te = f["train"], f["test"] + assert tr.n_trades == 3 and te.n_trades == 2 + assert abs(tr.profit_factor - 3.0) < 1e-9 + assert abs(tr.avg_r - 2.0 / 3.0) < 1e-9 + assert abs(tr.max_drawdown_r - 1.0) < 1e-9 + assert abs(te.profit_factor - 0.5) < 1e-9 + assert abs(te.avg_r - (-0.25)) < 1e-9 + assert abs(te.max_drawdown_r - 1.0) < 1e-9 + assert f["flagged"], "test PF 0.5 < 1.0 must be flagged" + + +@test("walk-forward: flagging — PF≥1 and zero-trade folds are not flagged") +def _(): + import gpu_backtest as _gb + good = (datetime(2024, 1, 1, tzinfo=timezone.utc), + datetime(2025, 1, 1, tzinfo=timezone.utc), + datetime(2025, 1, 1, tzinfo=timezone.utc), + datetime(2025, 4, 1, tzinfo=timezone.utc)) + empty = (datetime(2024, 4, 1, tzinfo=timezone.utc), + datetime(2025, 4, 1, tzinfo=timezone.utc), + datetime(2025, 4, 1, tzinfo=timezone.utc), + datetime(2025, 7, 1, tzinfo=timezone.utc)) + trades = [_wf_trade("2025-02-01 10:00", +1.0), + _wf_trade("2025-03-01 10:00", -0.5)] + stats = _gb.walkforward_stats(trades, [good, empty]) + assert not stats["folds"][0]["flagged"], "PF 2.0 must not be flagged" + assert stats["folds"][1]["test"].n_trades == 0 + assert not stats["folds"][1]["flagged"], "empty fold must not be flagged" + assert stats["n_flagged"] == 0 + assert _gb._fmt_pf(stats["folds"][1]["test"]) == "—" + + +@test("walk-forward: aggregated OOS dedupes overlapping test folds") +def _(): + import gpu_backtest as _gb + f1 = (datetime(2024, 1, 1, tzinfo=timezone.utc), + datetime(2025, 1, 1, tzinfo=timezone.utc), + datetime(2025, 1, 1, tzinfo=timezone.utc), + datetime(2025, 4, 1, tzinfo=timezone.utc)) + f2 = (datetime(2024, 2, 1, tzinfo=timezone.utc), + datetime(2025, 2, 1, tzinfo=timezone.utc), + datetime(2025, 2, 1, tzinfo=timezone.utc), + datetime(2025, 5, 1, tzinfo=timezone.utc)) + trades = [ + _wf_trade("2025-01-05 10:00", -1.0), # f1 test only + _wf_trade("2025-02-15 10:00", +0.5), # BOTH test windows + _wf_trade("2025-04-15 10:00", +2.0), # f2 test only + ] + stats = _gb.walkforward_stats(trades, [f1, f2]) + o = stats["oos"] + assert o.n_trades == 3, "overlapping-window trade must count exactly once" + assert abs(o.total_r - 1.5) < 1e-9 + assert abs(o.max_drawdown_r - 1.0) < 1e-9 # chronological OOS equity + assert stats["n_flagged"] == 1 # f1 PF 0.5; f2 has no losses + + +@test("walk-forward: CLI — off by default, 12/3/3 defaults, short window refused") +def _(): + import gpu_backtest as _gb + a = _gb._parse_args(["--pair", "XAUUSD"]) + assert a.walk_forward is False + assert (a.wf_train_months, a.wf_test_months, a.wf_step_months) == (12, 3, 3) + a = _gb._parse_args(["--pair", "XAUUSD", "--walk-forward", + "--wf-train-months", "6", "--wf-test-months", "2", + "--wf-step-months", "1"]) + assert a.walk_forward is True + assert (a.wf_train_months, a.wf_test_months, a.wf_step_months) == (6, 2, 1) + # Default --days 180 can't fit train 12 + test 3 → refuse before any + # data loading. + try: + _gb.main(["--pair", "XAUUSD", "--walk-forward"]) + assert False, "short window must raise SystemExit" + except SystemExit as e: + assert "shorter than one fold" in str(e) + try: + _gb.main(["--pair", "XAUUSD", "--walk-forward", + "--wf-train-months", "0"]) + assert False, "zero train months must raise SystemExit" + except SystemExit as e: + assert "must all be > 0" in str(e) + + +class _WFMockProvider: + """Offline provider stand-in for fetch_history cache-write tests.""" + name = "mockprov" + + def __init__(self, df): + self.df = df + self.fetches = 0 + + def fetch_range(self, symbol, tf, start, end): + self.fetches += 1 + return self.df + + def validate_and_clean(self, df, symbol, tf): + return df + + +@test("fetch_history: refuses to recreate a baseline-pinned parquet") +def _(): + import tempfile + from pathlib import Path as _Path + import gpu_backtest as _gb + old_cache, old_base = _gb.CACHE_DIR, _gb.BASELINE_ROOT + with tempfile.TemporaryDirectory() as td: + _gb.CACHE_DIR = _Path(td) / "cache" + _gb.BASELINE_ROOT = _Path(td) / "baseline" + try: + start = datetime(2020, 1, 1, tzinfo=timezone.utc) + end = datetime(2020, 6, 1, tzinfo=timezone.utc) + pinned = _gb._cache_path("XAU/USD", Timeframe.H1, start, end, + "mockprov").name + mdir = _gb.BASELINE_ROOT / "abc1234" + mdir.mkdir(parents=True) + (mdir / "manifest.json").write_text( + json.dumps({"cache_files": {pinned: "0" * 64}})) + prov = _WFMockProvider(make_ohlcv(30, seed=7)) + try: + _gb.fetch_history(prov, "XAU/USD", Timeframe.H1, start, end) + assert False, "must refuse to write a baseline-pinned name" + except RuntimeError as e: + assert "baseline" in str(e) + assert not (_gb.CACHE_DIR / pinned).exists(), \ + "refused write must leave nothing behind" + # A different window is a different filename → allowed. + end2 = datetime(2020, 7, 1, tzinfo=timezone.utc) + out = _gb.fetch_history(prov, "XAU/USD", Timeframe.H1, start, end2) + assert len(out) == 30 + other = _gb._cache_path("XAU/USD", Timeframe.H1, start, end2, + "mockprov").name + assert (_gb.CACHE_DIR / other).exists() + finally: + _gb.CACHE_DIR, _gb.BASELINE_ROOT = old_cache, old_base + + +@test("fetch_history: new parquet sha256 lands in data_manifest.json " + "(baseline cache_files format); cache hits never refetch or rehash") +def _(): + import hashlib + import tempfile + from pathlib import Path as _Path + import gpu_backtest as _gb + old_cache, old_base = _gb.CACHE_DIR, _gb.BASELINE_ROOT + with tempfile.TemporaryDirectory() as td: + _gb.CACHE_DIR = _Path(td) / "cache" + _gb.BASELINE_ROOT = _Path(td) / "baseline" # no manifests → no pins + try: + start = datetime(2020, 1, 1, tzinfo=timezone.utc) + end = datetime(2020, 6, 1, tzinfo=timezone.utc) + prov = _WFMockProvider(make_ohlcv(30, seed=8)) + _gb.fetch_history(prov, "XAU/USD", Timeframe.H1, start, end) + name = _gb._cache_path("XAU/USD", Timeframe.H1, start, end, + "mockprov").name + mpath = _gb.CACHE_DIR / "data_manifest.json" + manifest = json.loads(mpath.read_text()) + assert "cache_files" in manifest and "updated_utc" in manifest + want = hashlib.sha256( + (_gb.CACHE_DIR / name).read_bytes()).hexdigest() + assert manifest["cache_files"] == {name: want}, \ + "manifest must map filename → sha256 like the baseline format" + # Second call: cache hit — no refetch, manifest untouched. + before = mpath.read_text() + _gb.fetch_history(prov, "XAU/USD", Timeframe.H1, start, end) + assert prov.fetches == 1, "cache hit must not refetch" + assert mpath.read_text() == before, \ + "cache hit must not rewrite the data manifest" + finally: + _gb.CACHE_DIR, _gb.BASELINE_ROOT = old_cache, old_base + + # ══════════════════════════════════════════════════════════════════════ # SUMMARY # ══════════════════════════════════════════════════════════════════════