harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove EUR/USD support — XAU/USD-only codebase

Browse Source 
EUR/USD never had a validated edge (2026-07-11/12 verdict: no config
passed validation on M5/M15/H1 at 1-pip cost; cost-to-volatility root
cause). Removed: EurUsdStrategyConfig, pair_signals/eurusd.py, EUR
gates 6h + weekend/day filters + strategy marker + trade geometry in
signal_aggregator, EUR pair registry/aliases in main.py and
gpu_backtest.py, EUR/USD news pair mappings, test_all.py Module 10.
Generic multi-currency news keyword infra kept (GBP/JPY/EUR keyword
table is not pair code). Full EUR history remains at 7d3fd95.

Suite: 162/162 green (was 187; 25 EUR-specific tests removed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
This commit is contained in:
Harwinder Sekhon
2026-07-16 14:06:27 -04:00
parent 7d3fd9502e
commit 68a62b0a53
13 changed files with 75 additions and 919 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+31 -39
View File
@@ -1,6 +1,6 @@
# ForexBot2 — Codebase Reference
Signal-intelligence platform for **XAU/USD (gold — live, validated)** and **EUR/USD (functional but OFF by default — no validated edge, see below)**. It fetches OHLCV from Twelve Data, scores 9 technical indicators per timeframe with regime-adaptive weights, checks multi-timeframe agreement and news/calendar risk, grades the signal (A+…F), and generates a full trade setup (entry, SL, TP1/TP2, trailing plan). Delivery: Telegram bot (auto-scanner + subscriber channel) and a CLI. A realism-focused backtester replays history through the *exact same* signal code.
Signal-intelligence platform for **XAU/USD (gold — live, validated)**. It fetches OHLCV from Twelve Data, scores 9 technical indicators per timeframe with regime-adaptive weights, checks multi-timeframe agreement and news/calendar risk, grades the signal (A+…F), and generates a full trade setup (entry, SL, TP1/TP2, trailing plan). Delivery: Telegram bot (auto-scanner + subscriber channel) and a CLI. A realism-focused backtester replays history through the *exact same* signal code.
This file is the authoritative developer/agent reference. The human-facing docs live in `docs/` (`user_guide.html` + `beginner_guide.html`, and their PDF builds) — keep them in sync when signal behavior changes. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
@@ -20,23 +20,22 @@ This file is the authoritative developer/agent reference. The human-facing docs
| `telegram_bot.py` | ~1259 | Async Telegram bot: candle-boundary auto-scanner, WebSocket level-breach monitor, owner lock, subscriber channel |
| `pair_signals/__init__.py` | 52 | `PAIR_PRODUCERS` registry + `produce_signal()` |
| `pair_signals/xauusd.py` | 71 | XAU config factory + per-timeframe overrides (docstring = validation record) |
| `pair_signals/eurusd.py` | 158 | EUR config factory + EUR regime weights (docstring = no-edge verdict + tuning history) |
| `main.py` | ~348 | CLI: `python main.py --pair XAUUSD --timeframe H1 --detail/--json` |
| `test_all.py` | ~2570 | QA suite: **187 tests in 11 modules**, no live API needed. Must stay green. |
| `test_all.py` | ~2100 | QA suite: **162 tests in 10 modules**, no live API needed. Must stay green. |
| `backtest/gpu_backtest.py` | ~2231 | Bar-replay backtester (CuPy→NumPy fallback) using the real `SignalAggregator` |
| `backtest/diagnose_filters.py` | 609 | Filter-diagnosis tooling |
| `backtest/research/` | — | Calendar research infra (`ff_calendar_scraper.py`, `calendar_store.py`) + result CSVs |
| `backtest/*.csv` | — | Verification trade books from past tuning campaigns (`xau_*.csv`, `eur_verify.csv`, …). **Gitignored, local-only** — used for manual before/after comparison, not by tests. `eth/btc/gbp_verify.csv` come from research whose pair code is NOT on main |
| `backtest/*.csv` | — | Verification trade books from past tuning campaigns (`xau_*.csv`, …). **Gitignored, local-only** — used for manual before/after comparison, not by tests |
| `backtest/cache/` | — | Parquet OHLCV cache (raw bars) |
| `test_ws_dual.py` | 108 | Standalone WebSocket dual-connection probe |
| `logs/telegram.log` | — | Bot log (midnight rotation, 30 backups) |
---
## Pair status (do not confuse the two)
## Pair status
- **XAU/USD**: the tuned, validated product. It is the only entry in `PlatformConfig.scan_pairs`. All defaults in `PlatformConfig`/`XauStrategyConfig` are its tuned values.
- **EUR/USD**: fully wired (factory, gates, tests) but **not in `scan_pairs`** and its signals are informational only. Verdict (2026-07-11/12, recorded in `pair_signals/eurusd.py` docstring): no configuration passed validation on M5, M15, or H1 at a 1-pip round-trip cost; root cause is cost-to-volatility (EUR M15 ATR ≈ 10 pips → 1-pip spread taxes ~10% of ATR; gold's ATR/spread ratio is several times larger). Calendar-surprise and H1-hybrid avenues were also tested and closed. Do not re-open EUR tuning without new data (tick fills or H4/D1 horizons).
- **XAU/USD**: the tuned, validated product — and the ONLY supported pair. All defaults in `PlatformConfig`/`XauStrategyConfig` are its tuned values.
- **EUR/USD**: REMOVED from the codebase (2026-07-16, branch `xau-only-chart-patterns`). It never had a validated edge: the 2026-07-11/12 verdict found no configuration passing validation on M5, M15, or H1 at a 1-pip round-trip cost; root cause was cost-to-volatility (EUR M15 ATR ≈ 10 pips → 1-pip spread taxes ~10% of ATR; gold's ATR/spread ratio is several times larger). Calendar-surprise and H1-hybrid avenues were also tested and closed. The full EUR implementation and verdict history live in git history (last commit with EUR support: `7d3fd95`). Do not re-add EUR without new data (tick fills or H4/D1 horizons).
---
@@ -48,7 +47,7 @@ Twelve Data REST (+ WebSocket ticks in the bot)
→ engine.TechnicalEngine.analyze() per timeframe: 9 scored indicators + ATR context → regime → composite score
→ signal_aggregator.analyze_pair() live entry point (weekend/day filters, fetch, staleness)
└─ analyze_from_dataframes() replayable core: MTF agreement → news → bonuses/penalties →
final signal + quality grade → gates (session, SELL, M5, EUR, regime) →
final signal + quality grade → gates (session, SELL, M5, regime) →
trade setup (3 generators) → reasons
→ SignalReport consumed by telegram_bot.py, main.py, backtest/gpu_backtest.py
```
@@ -59,17 +58,17 @@ Signal currency everywhere is the `SignalStrength` enum: `STRONG_BUY=2, BUY=1, N
## config.py — every knob
### PAIR_INFO (market constants; keys are the canonical symbols `"XAU/USD"`, `"EUR/USD"`)
| Field | XAU/USD | EUR/USD | Meaning |
|---|---|---|---|
| `pip_value` | 0.10 | 0.0001 | price units per pip |
| `price_precision` | 2 | 5 | decimals for SL/TP rounding & display |
| `default_spread` | 0.30 | 0.0001 | round-trip cost charged by the backtester when `--spread` not given |
### PAIR_INFO (market constants; key is the canonical symbol `"XAU/USD"`)
| Field | XAU/USD | Meaning |
|---|---|---|
| `pip_value` | 0.10 | price units per pip |
| `price_precision` | 2 | decimals for SL/TP rounding & display |
| `default_spread` | 0.30 | round-trip cost charged by the backtester when `--spread` not given |
### TechnicalConfig (indicator parameters)
EMA 9/21/50/200 (`ema_mode`: `"4ema"` default or `"2ema"` = 50/200 only); RSI 14 (OB 70 / OS 30 / neutral zone 40–60; `rsi_mode`: `"regime_aware"` default or `"classic"`); MACD 12/26/9 (`macd_as_pillar=True` — MACD is a scored indicator, not just divergence input); BB 20/2.0σ; ATR 14; ADX 14 (strong ≥25, very strong ≥50); divergence lookback 40; swing-S/R lookback 50; ROC period 10, threshold 1.0%; Keltner: EMA 20 mid ± 1.5 × ATR(10); ADX slope lookback 5, "emerging" threshold 3.0; `disabled_indicators` list (removes the pillar from scoring; for MACD the histogram is still computed — divergence needs it); `regime_weights_override` dict (used by EUR).
EMA 9/21/50/200 (`ema_mode`: `"4ema"` default or `"2ema"` = 50/200 only); RSI 14 (OB 70 / OS 30 / neutral zone 40–60; `rsi_mode`: `"regime_aware"` default or `"classic"`); MACD 12/26/9 (`macd_as_pillar=True` — MACD is a scored indicator, not just divergence input); BB 20/2.0σ; ATR 14; ADX 14 (strong ≥25, very strong ≥50); divergence lookback 40; swing-S/R lookback 50; ROC period 10, threshold 1.0%; Keltner: EMA 20 mid ± 1.5 × ATR(10); ADX slope lookback 5, "emerging" threshold 3.0; `disabled_indicators` list (removes the pillar from scoring; for MACD the histogram is still computed — divergence needs it); `regime_weights_override` dict (replaces the gold regime-multiplier table for experiments).
### SignalConfig (scoring & trade geometry — XAU-calibrated; EUR uses its own copies)
### SignalConfig (scoring & trade geometry — XAU-calibrated)
| Field | Default | Meaning |
|---|---|---|
| `mtf_agreement_threshold` | 2 | timeframes (of primary + confirmations) that must agree |
@@ -84,7 +83,7 @@ EMA 9/21/50/200 (`ema_mode`: `"4ema"` default or `"2ema"` = 50/200 only); RSI 14
| `weights`, `min_confidence`, `cooldown_bars`, `default_risk_pct`, `default_rr_ratio`, `atr_sl_multiplier` | — | **LEGACY / UNUSED** outside tests. The engine has its own weights; backtest cooldown lives in `BacktestConfig` |
### NewsConfig
Tracked currencies `["USD","XAU","EUR"]`; pre-event buffer 30 min, post-event 15 min (45 min after a *surprise* print); `event_risk_caps_signals=False` — **XAU default: event risk warns but does NOT cap confidence** (2026-07-11 audit: the caps suppressed trades worth +4.6..+7.2R/yr; XAU trades near USD releases outperform). EUR factory turns the cap back ON. Context cache TTL 600 s (must stay < pre-event buffer). 4 RSS feeds (investinglive, FXStreet, Google-News gold query, CNBC economy).
Tracked currencies `["USD","XAU"]`; pre-event buffer 30 min, post-event 15 min (45 min after a *surprise* print); `event_risk_caps_signals=False` — **XAU default: event risk warns but does NOT cap confidence** (2026-07-11 audit: the caps suppressed trades worth +4.6..+7.2R/yr; XAU trades near USD releases outperform). Context cache TTL 600 s (must stay < pre-event buffer). 4 RSS feeds (investinglive, FXStreet, Google-News gold query, CNBC economy).
### XauStrategyConfig (all XAU gates; `enabled=True`)
| Field | Default | Notes |
@@ -103,9 +102,6 @@ Tracked currencies `["USD","XAU","EUR"]`; pre-event buffer 30 min, post-event 15
Legacy fields kept for config-shape compatibility: `rsi_entry_threshold`, `min_ema200_tfs`, `allow_generic_trades`, `buy_threshold_boost`, `m5_confirmation_bars`, `h4_rsi_gate=False`, `h1_macd_gate=False`, `rsi_zone` — not wired to current gates.
### EurUsdStrategyConfig (EUR-local; nothing gold-calibrated leaks in)
Blocked hours [22,23,0,1,2,3]; weekdays Mon–Fri (M5 factory: Mon–Thu, Fridays bled −17R); `min_atr_threshold=0.0003`; SL: swing-buffer ×1.5 ATR, floor ×1.2 ATR (M5: 0.3/0.7), min 8 pips; `min_rr_ratio=1.0`; `mature_move_threshold=35` (|score|>35 → tighter TP 1.2 ATR, else 1.8 ATR); `blocked_regimes=["ranging","transitioning"]`; `min_quality_grades=["A+","A","B"]`; `min_entry_score=38` / `min_entry_confidence=60` (M5 factory: 35/45); `block_rsi_extended=True` (no BUY at RSI>70, no SELL at RSI<30); costs: `round_trip_cost=0.0001`, `min_tp_cost_multiple=10` (10-pip TP minimum), `min_sl_pips=8`. `be_trigger_r`/`be_buffer_r` are legacy (superseded by trailing exits).
### FilterConfig (`config.backtest` — despite the name it applies LIVE too)
`session_filter=True`, window 03:00–23:00 UTC, `blocked_hours_utc` mirrors XAU's list (non-XAU pairs read this copy), `allowed_weekdays` Mon–Fri+Sun. `regime_filter=False` (optional H4-ADX≥25 gate, off).
@@ -155,7 +151,7 @@ Structure S/R (not scored): swing highs/lows (window 3) over `swing_sr_lookback`
### Composite score
For each scored indicator: `effective_weight = base_weight × regime_multiplier`; contribution = `signal.value × effective_weight × confidence/100`. Composite = `(Σ contributions / Σ effective_weights) × 50`, clamped to ±100. Confidence = weight-averaged indicator confidence. Composite signal: ≥40 STRONG_BUY, ≥15 BUY, ≤−40 STRONG_SELL, ≤−15 SELL, else NEUTRAL.
**Gold regime multipliers** (defaults in `_compute_composite`; `technical.regime_weights_override` replaces the whole dict — EUR does this):
**Gold regime multipliers** (defaults in `_compute_composite`; `technical.regime_weights_override` replaces the whole dict):
| Indicator | trending_up/down/trending | ranging | transitioning |
|---|---|---|---|
@@ -174,7 +170,7 @@ For each scored indicator: `effective_weight = base_weight × regime_multiplier`
## signal_aggregator.py — SignalAggregator
Two entry points:
- **`analyze_pair(symbol)`** — LIVE: applies XAU/EUR weekend + allowed-weekday filters on wall clock, fetches all TFs (primary + confirmations), flags stale data (primary stale → signal suppressed to NEUTRAL/F), delegates to `analyze_from_dataframes`, then applies the live session check on the **bar-CLOSE time** (bar timestamp + candle duration): outside 03:00–23:00 or in blocked hours → NO TRADE. This is the exact predicate the backtester replays, so live == backtest.
- **`analyze_pair(symbol)`** — LIVE: applies XAU weekend + allowed-weekday filters on wall clock, fetches all TFs (primary + confirmations), flags stale data (primary stale → signal suppressed to NEUTRAL/F), delegates to `analyze_from_dataframes`, then applies the live session check on the **bar-CLOSE time** (bar timestamp + candle duration): outside 03:00–23:00 or in blocked hours → NO TRADE. This is the exact predicate the backtester replays, so live == backtest.
- **`analyze_from_dataframes(symbol, dfs, context, use_news)`** — replayable core; treats each df as the visible history at decision time (caller must slice; no weekend filter here — the backtester applies it).
### Pipeline inside `analyze_from_dataframes` (numbered as in the source)
@@ -185,7 +181,7 @@ Two entry points:
5. Spread estimate = median(High−Low) of primary df.
6. **`_compute_final_signal`** (bonuses → event cap → penalties → thresholds → grade):
- Bonuses: MTF agreement → +10 conf and `score = 0.6×primary + 0.4×mean(all TF scores)`; news sentiment aligned (|sent|>0.2) → +5; in a *directional* trend, RSI in pullback zone → +5, BB position favorable → +5 (mixed `trending` regime gets NO directional bonus).
- Event risk `danger`: warning always; confidence hard-capped at 30 ONLY if `news.event_risk_caps_signals` (False for XAU, True for EUR).
- Event risk `danger`: warning always; confidence hard-capped at 30 ONLY if `news.event_risk_caps_signals` (False by default — XAU-tuned).
- **Penalty system (diminishing, replaces old additive stacking):** collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: `mtf_disagreement` 15; `news_conflict` 10; `event_caution` 15 (config-gated); `counter_trend` 15; `extreme_vol` 10; `ema200_near` 5 (price within 0.5% of EMA200 — env `FB2_EMA_LO` overrides); `ema200_far` 5 (>2.5%); `exhaustion` 8 (price >3.5 ATR beyond nearest swing); `no_pullback` 8 (>2.5 ATR from EMA21 in trend) / `weak_pullback` 5 (>2.0); `rsi_extended` 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed `trending`); `bb_overextended` 8 (BB pos >0.85 BUY / <0.15 SELL).
- **Exhaustion hard block**: ≥2 exhaustion-tagged penalties ({exhaustion, no_pullback, rsi_extended, bb_overextended} → `report.exhaustion_count`) AND momentum fading (ROC sign against score, or MACD signal against score) → confidence = 0.
- Confidence clamped [0, 95]. **Final signal**: score ≥35 & conf ≥55 STRONG_BUY; ≥25 & ≥50 BUY; mirrored sells; else NEUTRAL.
@@ -199,10 +195,9 @@ Two entry points:
- **6g XAU SELL volatility gate**: sell-direction & vol regime not in {high, extreme} → D.
- **6g2 XAU regime gates**: regime in `blocked_regimes` (M15: mixed `trending`) → D; sell-direction & regime in `sell_blocked_regimes` → D.
- **6g3 XAU SELL hour gate**: sell-direction & bar-CLOSE hour (label + candle duration — the entry hour under next-open fills) in `sell_blocked_hours_utc` → D. Per-TF opt-in; M15 factory blocks 15/17 UTC.
- **6h EUR gates** (EUR only): blocked regimes (ranging/transitioning), quality < B, exhaustion count, session hours, |score| < min_entry_score, conf < min_entry_confidence, blocked vol regimes, indicator-agreement floor, RSI-extended block → each sets grade F with a warning.
- **6i XAU M5 noise gates** (`_apply_m5_filters`, primary TF = M5 only): (1) |score| ≥ 30; (2) MTF agreement required; (3) ≥1 confirmation TF with score ≥ ±10 in trade direction (NEUTRAL confirms don't count); (4) SELL grade must be B/C (A-grade M5 shorts lose −0.017R/trade); (5) vol regime `high` blocked; (6) SELL blocked when RSI < 25. Each failure → D, setup killed.
8. **Trade setup**: XAU + grade A+/A → `_generate_xau_atr_trade_setup` first; any pair + grade A+/A/B/C and no setup yet → `_generate_trade_setup` (swing-based).
9. Reasons: every non-neutral indicator description → `bullish_reasons`/`bearish_reasons`; EUR setups get an `"EUR STRATEGY: …"` marker inserted at reasons[0] (the backtester tags strategy from reasons[0]).
9. Reasons: every non-neutral indicator description → `bullish_reasons`/`bearish_reasons` (the backtester tags strategy from reasons[0] markers).
10. **XAU_PB pullback** (M5 XAU only, `m5_pullback_enabled`, no setup yet, grade ≠ F): `_try_m5_pullback` — see below.
### Trade setup generators (exact geometry)
@@ -210,10 +205,10 @@ Two entry points:
**A. `_generate_xau_atr_trade_setup`** (XAU, grade A+/A, ATR>0): TP quality scale A+ 2.0 / A 1.5 (A with exhaustion≥2 → 1.0). TP1 = entry ± `tp_atr_multiplier`(1.0)×scale×ATR; TP2 = 1.5×that; SL = entry ∓ 2.0×ATR. Gates: RR ≥ 0.5; TP1 distance ≥ $9 (spread gate). Prices rounded to 2 dp; pip 0.10 hardcoded — **this generator is XAU-only by construction**.
**B. `_generate_trade_setup`** (swing-based, any pair, grade A+..C): XAU requires ATR ≥ $1 (LOW ATR FILTER). Per-side:
- SL anchor = nearest swing low below price (BUY) / swing high above (SELL) ∓/± `sl_swing_buffer`×ATR (XAU 1.0, EUR = `eur.sl_atr_multiplier`); fallback (no swing) = (1+buffer)×ATR. SL distance floored at max(ATR × sl_atr_floor, min_sl_pips×pip) — XAU: max(2.0×ATR, $8); EUR: max(1.2×ATR, 8 pips) (M5 EUR: 0.7×ATR).
- TP1 = nearest swing beyond entry ± 0.3 ATR if it's within `tp_swing_cap` ATR (XAU 2.5) AND ≥ SL×RR-floor; else entry ± max(`tp_mult`×ATR, SL×RR-floor). XAU: tp_mult 1.5, RR-floor = max(min_rr 0.5, tp_floor_rr — 1.0 on M15). EUR: tp_mult 1.2 (mature move, |score|>35) or 1.8, RR-floor 1.0.
- SL anchor = nearest swing low below price (BUY) / swing high above (SELL) ∓/± `sl_swing_buffer`×ATR (1.0); fallback (no swing) = (1+buffer)×ATR. SL distance floored at max(ATR × sl_atr_floor, min_sl_pips×pip) — XAU: max(2.0×ATR, $8).
- TP1 = nearest swing beyond entry ± 0.3 ATR if it's within `tp_swing_cap` ATR (2.5) AND ≥ SL×RR-floor; else entry ± max(`tp_mult`×ATR, SL×RR-floor). tp_mult 1.5, RR-floor = max(min_rr 0.5, tp_floor_rr — 1.0 on M15).
- TP2 = second swing if within 5 ATR, else entry ± max((tp_mult+1.5)×ATR, SL×floor×1.5); forced beyond TP1 by ≥1 ATR.
- Gates: RR ≥ min_rr; spread gate TP1 distance ≥ cost×multiple (XAU $9; EUR 10 pips).
- Gates: RR ≥ min_rr; spread gate TP1 distance ≥ cost×multiple ($9 on XAU).
**C. `_try_m5_pullback` (XAU_PB)**: detection = EMA alignment ≥3 of 4 checks (9>21, 21>50, 50>200, close>200 — mirrored for SELL), ADX ≥ 20, price touched max(EMA21, KC-mid) within last 3 bars, resumption bar closed back in trend direction beyond EMA9, retrace depth 0.8–6.0 ATR from the 40-bar extreme, RSI in pullback zone (BUY 35–62 / SELL 38–65). Then gates: directional score ≥ 15; ≥1 confirmation TF ≥ ±10; not against MTF consensus; vol regime gates (same as breakout path). Geometry: SL = pullback 8-bar extreme ∓ 0.5 ATR (min $8); TP1 = 1.5×risk, TP2 = 2.5×risk; spread gate $9. Sets `strategy="XAU_PB"`, grade min B, marker `"XAU PULLBACK STRATEGY: …"` at reasons[0]. Rejection funnel counters in class attr `SignalAggregator.pb_funnel`.
@@ -245,11 +240,11 @@ Two entry points:
## pair_signals/ — per-pair factories (ALWAYS use these, never bare PlatformConfig for a pair)
`PAIR_PRODUCERS: Dict[symbol, factory]`; factory signature `make_config(primary_tf, confirmation_tfs=None) -> PlatformConfig`. `produce_signal(symbol, tf, data_provider)` builds config → aggregator → `analyze_pair`. Registered: XAU/USD, EUR/USD.
`PAIR_PRODUCERS: Dict[symbol, factory]`; factory signature `make_config(primary_tf, confirmation_tfs=None) -> PlatformConfig`. `produce_signal(symbol, tf, data_provider)` builds config → aggregator → `analyze_pair`. Registered: XAU/USD.
**A bare `PlatformConfig()` silently runs any pair with XAU defaults — always go through the factory** (the bot, CLI, and backtester all do).
Default confirmation map (same in both factories, the bot, and backtest usage): M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1]. (Exception: `main.py`'s CLI map uses M5→[M1,M15] — a CLI-only quirk.)
Default confirmation map (same in the factory, the bot, and backtest usage): M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1]. (Exception: `main.py`'s CLI map uses M5→[M1,M15] — a CLI-only quirk.)
**xauusd.py**: `PlatformConfig()` defaults + per-TF overrides (each validated; full evidence in the module docstring):
- M5: append hour 9 UTC to blocked hours (6-yr replay: +207.1R vs +178.3R; M15-and-up keep hour 9 — it's positive there).
@@ -257,8 +252,6 @@ Default confirmation map (same in both factories, the bot, and backtest usage):
- M15: `xau_strategy.sell_blocked_hours_utc = [15, 17]` (SELL-only hour block, validated 2026-07-16 on the 5-yr replay: +176.7R PF 1.31 vs +144.9R PF 1.22, better in all 6 years, both 12-mo halves, and 2024-06→2025-03 OOS; M5 cross-TF pocket confirms; BUYs at those hours are +34.3R so `blocked_hours_utc` would be wrong).
- The factory must stay equivalent to inline `PlatformConfig(symbol=XAUUSD, …)` + overrides — **enforced by a test in test_all.py**.
**eurusd.py**: sets symbol/display/pip, `xau_strategy.enabled = False` (belt-and-braces; XAU gates are also display-gated), `news.event_risk_caps_signals = True`, EUR session (7–20 h window, blocked [22,23,0-3], Mon–Fri), EUR regime-weight overrides (`EUR_REGIME_WEIGHTS`, and `M5_REGIME_WEIGHTS` + tighter trailing 0.45/0.5 + Mon–Thu + extra blocked hours [8,12,13,18,19] + swing lookback 20 on M5). Docstring = the complete no-edge verdict and closed research avenues.
---
## telegram_bot.py — the bot
@@ -278,7 +271,7 @@ Run: `python telegram_bot.py` (env `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGR
## main.py — CLI
`python main.py [--pair XAUUSD|EURUSD] [--timeframe M5|M15|M30|H1|H4|D1] [--detail] [--json] [--verbose]`. Uses the pair factory, renders a compact terminal report (decision, setup, context, top-3 reasons per side, warnings) or full JSON via `report_to_dict`.
`python main.py [--pair XAUUSD] [--timeframe M5|M15|M30|H1|H4|D1] [--detail] [--json] [--verbose]`. Uses the pair factory, renders a compact terminal report (decision, setup, context, top-3 reasons per side, warnings) or full JSON via `report_to_dict`.
---
@@ -288,7 +281,7 @@ Replays history bar-by-bar through the **real `SignalAggregator.analyze_from_dat
**Realism model (all validated fixes — do not regress):**
- Entry at the **next bar's open** after the signal bar; SL/TP distances preserved relative to actual entry. Entries are **re-anchored to the raw price path** (`_reanchor_entries_to_raw`): if cleaning dropped a >10σ spike bar right after the signal bar, the trade enters at the spike bar's open, not the bar after it — skipping it was a favorable bias.
- Round-trip `--spread` subtracted from every trade's PnL before computing R (default = pair's `default_spread`: XAU $0.30, EUR 1 pip). Results before 2026-07-08 (+59.5R baseline era) charged no spread — not comparable.
- Round-trip `--spread` subtracted from every trade's PnL before computing R (default = pair's `default_spread`: XAU $0.30). Results before 2026-07-08 (+59.5R baseline era) charged no spread — not comparable.
- Trade resolution runs against the **RAW price path** (`fetch_history(clean=False)`): the parquet cache stores raw bars; cleaning (which drops >10σ news spikes that hit stops in reality) is applied only to the signal-side data. Caches written before this fix contain cleaned data — refetch to restore spike bars.
- Same-bar SL+TP tie → SL first (conservative). Gap through SL/TP → filled at the bar open.
- **Trailing exits** (default on, from `signals.trail_arm_r/trail_dist_r`; XAU_PB trades carry per-trade 0.8/0.75 overrides): stop ratchets at bar close only, so a new stop can only fill from the next bar (conservative). Exit reasons: SL / TRAIL / TP1 (static mode) / TIMEOUT / NO_DATA.
@@ -297,13 +290,12 @@ Replays history bar-by-bar through the **real `SignalAggregator.analyze_from_dat
**Performance:** `precompute_indicators` builds every indicator series once per TF (CuPy on GPU or NumPy — check the log line `Backend: CuPy GPU` vs `NumPy CPU fallback`; CuPy needs the `nvidia-*-cu12` pip wheels or it silently falls back). `GPUTechnicalEngine` subclasses the live engine and overrides only the `_compute_*` methods with cache lookups — outputs are byte-identical to the live engine (regression-tested). Bar replay: pure per-bar evaluation fans out over forked worker processes (`--workers`, 0 = auto: cores−2 capped 12; <5000 bars forces 1); the entry **cooldown (5 bars) is the only sequential rule**, applied in an ordered pass afterwards — parallel and sequential books are identical. ~300 bars/s/core, ~1700 bars/s at 10 workers.
**Strategy tags** (from reasons[0] markers): `XAU_PB`, `EUR`, `GENERIC` (plus legacy `XAU`, `GBP_PB` recognized). `--strategies` filters the book by tag.
**Strategy tags** (from reasons[0] markers): `XAU_PB`, `GENERIC` (plus legacy `XAU` recognized). `--strategies` filters the book by tag.
**CLI:**
```bash
python backtest/gpu_backtest.py --pair XAUUSD --days 365
python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-01 --primary-tf 15min --confirm-tfs 1h,4h
python backtest/gpu_backtest.py --pair EURUSD --primary-tf 5min --confirm-tfs 15min,1h --days 365
# key flags: --spread --quality A+,A,B,C --strategies --workers --max-hold-bars 300
# --warmup-bars 250 --cooldown-bars 5 --no-cache --export-csv out.csv --detail
```
@@ -317,7 +309,7 @@ Data: paginated TD fetches (5000-bar chunks) cached as parquet in `backtest/cach
## test_all.py — QA suite
`python test_all.py` — **187 tests, 11 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, EUR/USD support, 2026-07-16 audit-fix regressions). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: pair-isolation tests (no gold logic on EUR configs and vice versa), the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), and a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11). **Any signal-logic change must keep this green and should add tests.**
`python test_all.py` — **162 tests, 10 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), and a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11). **Any signal-logic change must keep this green and should add tests.**
---
@@ -348,9 +340,9 @@ Dependencies (`requirements.txt`): numpy, pandas, requests, twelvedata, python-t
## Rules for making changes (READ BEFORE EDITING SIGNAL LOGIC)
1. **Never change a tuned value without re-validation.** The acceptance rule used throughout: positive total R in BOTH the train and validation halves of a ≥12-month window, PF ≥ 1.3, at realistic cost (XAU $0.30 / EUR 1 pip round trip), with trailing exits and next-open fills. For EUR (and any marginal edge) additionally re-test on the 2024-06→2025-03 out-of-sample window — within-year splits alone have approved configs that failed OOS.
1. **Never change a tuned value without re-validation.** The acceptance rule used throughout: positive total R in BOTH the train and validation halves of a ≥12-month window, PF ≥ 1.3, at realistic cost (XAU $0.30 round trip), with trailing exits and next-open fills. For any marginal edge additionally re-test on the 2024-06→2025-03 out-of-sample window — within-year splits alone have approved configs that failed OOS.
2. **Per-timeframe overrides live in the pair factory** (`pair_signals/*.py`), never in global defaults — the same knob has opposite effects on different TFs (proven twice: `tp_floor_rr`, hour-9 block). Document the validation evidence in the factory docstring.
3. **Pair isolation is a hard invariant.** XAU gates in the aggregator are display-gated (`report.display_name == "XAU/USD"`); EUR reads only `EurUsdStrategyConfig` for costs/geometry. Tests enforce zero XAU drift when EUR changes and vice versa.
3. **Pair isolation is a hard invariant.** XAU gates in the aggregator are display-gated (`report.display_name == "XAU/USD"`) so a future pair port cannot inherit gold-calibrated logic by accident.
4. **Live == backtest parity**: session/weekend/hour filters must evaluate bar-CLOSE time in both paths; `xauusd.make_config` must stay equivalent to the inline config the backtester/bot would build (test-enforced). If you add a live gate, add it to the replay path (or `analyze_from_dataframes`) in the same commit.
5. **ROC stays weighted** (see engine table) unless you retune composite thresholds and re-validate — removing it lost 13R.
6. **Backtester realism knobs are results-critical**: raw-path resolution, spread charging, next-open entry, SL-first ties, close-only trail ratchet. Any "optimization" that touches these invalidates comparability with the pinned books (`backtest/*.csv`, memory notes).