backtest: walk-forward mode + extended-range data safety
--walk-forward slices ONE continuous replay into rolling train/test folds by entry time (default 12mo/3mo/3mo; no per-fold re-optimization) and reports per-fold PF / expectancy / max DD plus aggregated OOS stats, flagging test folds with PF < 1. Off by default — default runs are byte-identical (scripts/compare_baseline.py 32/32 PASS with this diff). Data safety for deep-history fetches: fetch_history refuses to recreate any baseline-pinned parquet filename and records the sha256 of every freshly written parquet in backtest/cache/data_manifest.json. Tests: Module 23 (fold builder, slicing, flagging, OOS dedup, CLI defaults, pin guard, manifest recording) — 367 total. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017FBjfomAipxvPWwRJEDDyJ
This commit is contained in:
@@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
python test_all.py # QA suite: 359 tests, 22 modules, no network (mocked)
|
||||
python test_all.py # QA suite: 367 tests, 23 modules, no network (mocked)
|
||||
python main.py --pair XAUUSD --timeframe H1 --detail # CLI signal report (--json for full dump)
|
||||
python telegram_bot.py # the bot (refuses to boot without credentials)
|
||||
python oanda_daemon.py # auto-execution daemon on Oanda practice account
|
||||
@@ -26,6 +26,10 @@ python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-0
|
||||
# --warmup-bars --cooldown-bars --no-cache --export-csv out.csv --detail
|
||||
# --fixed-rr-tp N (research-only: hard TP at N×risk, trail off; verdict
|
||||
# 2026-08-05: NOT shipped — backtest/research/fixedtp_2026-08-05.md)
|
||||
# --walk-forward [--wf-train-months 12 --wf-test-months 3 --wf-step-months 3]
|
||||
# rolling-fold OOS report over the --start/--end window (one continuous
|
||||
# replay sliced by entry time; fixed config, no re-optimization; test
|
||||
# folds with PF<1 flagged). Off by default — default runs byte-identical.
|
||||
```
|
||||
|
||||
**`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. When patching weekend logic in tests, patch `providers.base._is_forex_weekend` — but know there are THREE independent weekend implementations (`providers/base.py`, `signal_aggregator.py`, and an inline copy in `backtest/gpu_backtest.py:_is_xau_blocked_by_calendar`); patching the provider one does not reach the other two.
|
||||
@@ -62,7 +66,7 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
|
||||
- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
|
||||
- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`).
|
||||
- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. News/calendar rows are clickable (detail dialog); headlines carry `id` = sha1(headline) in news.json, and exactly one on-demand endpoint exists: `static/news_summary.php` (installed by `install_static`), which serves a DeepSeek gold-impact summary per headline — whitelist-gated against news.json ids, cache-forever in `ai/`, own fail-closed daily cap (150/day, file-based, separate from the publisher's), key at `/etc/gold-dashboard/deepseek.key`, never reads repo config. Deployment checklist: `deploy/README-dashboard.md`.
|
||||
- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
|
||||
- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window; `fetch_history` refuses to (re)write any filename pinned by a `baseline/*/manifest.json` (a cache miss there means the reference file was deleted — restore it, don't refetch) and records the sha256 of every freshly fetched parquet in `backtest/cache/data_manifest.json` (same `cache_files` format as the baseline manifest). `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
|
||||
|
||||
## Rules for making changes (read before editing signal logic)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user