Harwinder Sekhon
2cc5915b00
Two bodies of work that touch the same files, so one commit. Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule, written before the runs): - SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto "both"). A finished setup against a clear daily trend (closed D1 bars, Sunday stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier) is removed as the last step of the core. One definition (signal_aggregator.daily_trend_series) for live, the bar replay and the twin's host side; D1 is a required frame of the M15 scan; mirrored in cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R. - FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1 (−9.7R), buy-only and sell-only on M15, the wide-stop cap (xau_strategy.wide_stop_cap_atr stays 0.0, a research knob). - New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15 pins byte for byte. Twin and reference script MD5-equal on every window; recorder and feature-export parity re-proved; base-rate table rebuilt. - DISPLAY (books byte-identical): exposure.py — the lot size for trade_size.risk_per_trade (new config.json block, off by default, private to the owner) and the count of alerted same-direction plans still open (signals.alerted_at, read-only open_alerted_setups); the daily trend in words on Telegram, the CLI and the dashboard; signals.daily_trend logged. - providers/oanda: a date window with no bars is an empty page, not an error (a D1 pad starting on a Saturday could not be fetched). - Reports only, nothing shipped: seven context screens (no state nominated) and eight outside data sources (none adds a forecast to gold's own daily trend). - Independent verification (report section 5.13): every figure reproduced on both scripts, scoring identical to the previous commit on all four timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022; the neighbour clause is thin (a simple 100-day average misses the R line by 1.2R); "never adds a trade" is an observation, not a guarantee. Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects elsewhere, plus 4 found by an independent review of the fixes; all fixed, each with a test; no trade book changed: - telegram_bot: one _deliver_alert for the candle scan and the level break. The row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and un-stamped when no chat received it or the task is cancelled mid-stamp, so two alerts at one candle close count each other and no SQLite write runs on the event loop. The 5-bar cooldown counts bars of the scan's own frame (== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after the settlement pause) and is restored from signals.db at start-up. The lot-size line goes to the owner's own chat only (a group the owner ran /autoscan in used to receive it). /status gains a news-model line and escapes failure details. - signal_log: mark_alerted returns the stamp it wrote; unmark_alerted; last_alerted_bars (read-only). - advisory_outcome: candle continuity is judged bar by bar inside the walk; a hole after a trade's exit no longer voids it. The stored-candle replay resolves 108 of 111 logged setups instead of 99, all equal to the live resolver's R. - providers/oanda: an empty date window is accepted only when it is 5 days or shorter and no malformed candle was skipped; a longer one is retried, then refused, instead of being cached as history with a hole. - news_engine: blank RSS titles skipped; one unreadable headline no longer costs the local model its whole batch (checked on the real bundle); model health recorded; no DeepSeek audit call in a process that does not archive; event-direction cache keyed per release; peek_week_events never fetches. - llm_client: chat_json never raises and never logs the API key (requests quotes the Authorization value in InvalidHeader; error bodies are redacted). - signal_aggregator: the scan's news block is guarded; the replay path (use_news=False) assigns exactly the values it did before. - agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input (event timing over the cached calendar week, headlines newest first). scripts/panel_replay.py replays the logged payload per persona; scripts/label_scan_outcomes.py counts with an ISO cutoff. - test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34); tests that write Parquet print a notice and return without pyarrow. Verification after the last code change: suite 632 passed, 0 failed (also with both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37 green; static Python 3.9 / SQLite 3.34 check clean on every touched file. Evidence: backtest/research/daily_trend_campaign_2026-09-30.md, context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6, 8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7).
91 lines
6.7 KiB
Plaintext
91 lines
6.7 KiB
Plaintext
Backtest reporting checklist — ForexBot2 (XAU/USD)
|
|
==================================================
|
|
Every backtest report must record:
|
|
- The exact command line (window, --primary-tf, --spread, and any --set /
|
|
--quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
|
|
- Trades, total R, win rate, profit factor and max drawdown for the whole
|
|
window AND for both halves, at both cost conventions ($0.65 measured-Oanda
|
|
headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
|
|
- Per trade (the exported CSV already carries these): entry/exit time and
|
|
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
|
indicator's signal / value / agreement at decision time.
|
|
- Breakdowns by session/hour, weekday, regime and volatility regime, and,
|
|
for any candidate change, the removed/added/shifted trade diff against the
|
|
control book (the 5-bar cooldown displaces entries, so judge run-level
|
|
numbers, never the added-trade bucket alone).
|
|
- The md5 of the exported book whenever the run is a parity or regression check.
|
|
Replays go through the real signal code (analyze_from_dataframes) — never a
|
|
re-implementation.
|
|
|
|
--- 2026-09-06: decision-time feature export ---
|
|
- `--export-features` appends stake_multiplier, r_sized and the 65 `f_*` decision-time
|
|
features (position_sizing.FEATURE_ORDER) AFTER the 63 pinned columns; default export is
|
|
byte-identical to the pinned books.
|
|
- Any sizing report must show the FLAT book and the SIZED book side by side and the
|
|
label-shuffle null; the 2026-09-06 campaign verdict (nothing ships,
|
|
backtest/research/sizing_campaign_2026-09.md) is the reference.
|
|
|
|
--- 2026-09-09: replay recording (--record DIR [--record-bars]) ---
|
|
- `--record DIR` writes manifest.json + trades/indicators/events parquet for every ACCEPTED trade:
|
|
every indicator's signal / value / confidence / BASE weight / regime multiplier / EFFECTIVE
|
|
weight / contribution on the primary AND both confirmation timeframes, penalties, the setup, the
|
|
outcome, and the Forex Factory calendar events around the trade (backtest/cache/calendar.parquet,
|
|
refreshed with scripts/ff_calendar_scraper.py). `--record-bars` adds one row per ANALYZED primary
|
|
bar with the same votes plus forward moves (1h/4h/24h in ATR units).
|
|
- Recording is read-only w.r.t. the signal path: any recorded run must be paired with a control
|
|
run (same CLI without --record) and the report must show the two books' MD5s equal
|
|
(manifest.book_csv_md5 is the md5 of the 63-column book).
|
|
- Scorecards and "what-if" tables from these files (backtest/replay_scorecard.py) are IN-BOOK
|
|
descriptive statistics: they cannot add trades a change would create and ignore the gates.
|
|
Any weight / indicator change they suggest is a hypothesis for a full rule-1 rerun, never a decision.
|
|
|
|
--- 2026-09-18: GPU backend (backtest/gpu_kernels.py) ---
|
|
- The "Backend:" line now names the CUDA device ("CuPy GPU (<device>, cupy <version>)") or the
|
|
reason for the NumPy fallback; quote it verbatim. `--backend cpu` forces the NumPy twins, `gpu`
|
|
refuses to fall back. Books are byte-identical on both backends (reference MD5s, CLAUDE.md §9).
|
|
- Any change to a kernel or to its CPU twin is a parity event: re-run Module 36, regenerate the
|
|
five reference books on BOTH backends and show the ten MD5s unchanged. Never replace a kernel
|
|
with a differently ordered algorithm (parallel scan, two-pass variance) even if it is faster.
|
|
|
|
--- 2026-09-18: CUDA research twin (backtest/cudaBasedBacktesting.py) ---
|
|
- Exemption from "replays go through the real signal code" above, granted by the owner for this
|
|
ONE research script: its per-bar decision is a CUDA re-implementation of engine.py,
|
|
signal_aggregator.py and chart_patterns.py (backtest/cuda_signal_kernels.py). gpu_backtest.py
|
|
stays the base reference and live scans never use the twin.
|
|
- Its numbers count as evidence only while parity holds. After ANY change to engine.py,
|
|
signal_aggregator.py, chart_patterns.py, position_sizing.py, the recorder's row layout or the
|
|
GPUTechnicalEngine overrides: mirror the change in cuda_signal_kernels.py in the same commit,
|
|
keep test_all.py Module 37 green, and regenerate at least the M15 OOS book (2024-06-01 ->
|
|
2025-03-30, $0.65) with BOTH scripts and show equal MD5s before quoting a CUDA-script result.
|
|
- A report built from the twin quotes both scripts' "Backend:" lines (the twin's necessarily
|
|
starts with "CuPy GPU (") and the paired MD5 of at least one book from the reference script on
|
|
the same code and cache. Whole-window and half-window figures, costs and breakdowns are then
|
|
read from the twin's book exactly as from the reference's; the CSVs are identical.
|
|
- The twin has no CPU path (--backend cpu / FB2_BACKEND=cpu are refused), --workers is ignored,
|
|
and it refuses configurations beyond its compile-time limits (more than 4 confirmation
|
|
timeframes, divergence lookback >= 64, swing lookback >= 256, cp_lookback >= 128) with a
|
|
message that points back to gpu_backtest.py.
|
|
|
|
--- 2026-09-30: daily-trend veto (M15) and the D1 frame ---
|
|
- The M15 factory config carries xau_strategy.daily_trend_veto = "both" (M15 override 7). Both
|
|
backtesters then load the D1 frame as an EXTRA frame next to the primary and confirmation
|
|
frames (the "Confirm TFs:" line does not list it), cached under its own pad:
|
|
backtest/cache/oanda_XAU_USD_1d_<start minus 1400 days>_<end>.parquet. A new M15 window therefore
|
|
needs that file (one network fetch the first time; "GPU cache fallbacks: 0" is unaffected).
|
|
- The veto reads CLOSED daily bars only (a bar labelled D counts from D + 1 day <= the primary
|
|
bar's close), drops the provider's Sunday stub bars, and removes a finished setup that goes
|
|
against a non-zero state. Definition and constants: signal_aggregator.daily_trend_series.
|
|
- Research switches, both default-neutral elsewhere: --set xau_strategy.daily_trend_veto=
|
|
off|both|buy|sell and --set xau_strategy.wide_stop_cap_atr=K (K = 0 is off; the cap FAILED its
|
|
rule on 2026-09-30 and ships nowhere). A control for an M15 candidate is the factory config,
|
|
i.e. WITH the veto; to reproduce a pre-2026-09-30 M15 book pass --set
|
|
xau_strategy.daily_trend_veto=off.
|
|
- A candidate that is a FINAL veto (it only removes finished setups) may be screened on a research
|
|
harness (K1 + K2 of the twin, a mask on the final candidates before the cooldown, the shared
|
|
resolver) once that harness is shown to reproduce a shipped book byte for byte; the formal
|
|
numbers still come from the real code path with the paired gpu_backtest.py MD5.
|
|
- The 2026-09-30 report (backtest/research/daily_trend_campaign_2026-09-30.md) is the template for
|
|
a risk-first rule: rules, windows and neighbours written before the runs; the long window
|
|
(2019-01-01 -> 2026-08-07) and an unseen holdout as clauses; the rising-market cost reported,
|
|
not hidden.
|