Docs: rewrite CLAUDE.md as a complete codebase reference

Replaces the summary-level doc with a full spec derived from a read of
every core module, so contributors (and low-context models) don't have
to infer behavior or rediscover tuned values:

- repo map with line counts; flags DOCUMENTATION.md/USAGE.md as stale
- every config knob with its real default and the backtest evidence
  behind it; marks the legacy/unused fields (SignalConfig.weights,
  min_confidence, cooldown_bars, XAU h4/h1 gates) so nothing new is
  built on dead knobs
- exact indicator logic for all 9 scored indicators + ATR context,
  regime classification rules, regime-aware RSI zone tables, and the
  regime-multiplier table copied from _compute_composite (the old
  table did not match the code)
- aggregator pipeline in source order: MTF agreement, the 12 penalties
  and the diminishing cap, exhaustion block, signal/grade thresholds,
  and gates 6b-6i with exact numbers
- all three trade-setup generators with SL/TP geometry, spread gates,
  and the trailing-exit model
- bot/CLI/news/data-provider/backtester mechanics incl. realism model
  (raw-path resolution, next-open entry, SL-first ties, parallel replay)
- "Rules for making changes": validation acceptance rule, pair
  isolation, live==backtest parity, ROC's load-bearing weight

Also corrects details the old doc got wrong: test count is 179 (not
~176), EUR ships off by default, backtest/*.csv are gitignored local
artifacts rather than test-enforced pins.

CLAUDE.md

@@ -1,130 +1,355 @@
-# ForexBot2
+# ForexBot2 — Codebase Reference
 
-Multi-pair signal intelligence platform (XAU/USD + EUR/USD). Analyzes each pair across multiple timeframes using 9 technical indicators with regime-adaptive weights, news sentiment, and economic calendar data. Delivers signals via Telegram with full trade setups (entry, SL, TP).
+Signal-intelligence platform for **XAU/USD (gold — live, validated)** and **EUR/USD (functional but OFF by default — no validated edge, see below)**. It fetches OHLCV from Twelve Data, scores 9 technical indicators per timeframe with regime-adaptive weights, checks multi-timeframe agreement and news/calendar risk, grades the signal (A+…F), and generates a full trade setup (entry, SL, TP1/TP2, trailing plan). Delivery: Telegram bot (auto-scanner + subscriber channel) and a CLI. A realism-focused backtester replays history through the *exact same* signal code.
 
-## Architecture
+This file is the authoritative documentation. `DOCUMENTATION.md` and `USAGE.md` are stale (May 2026) — do not trust them. `PULLBACK_ENTRY_PLAN.txt` documents the XAU_PB pullback strategy design. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
+
+**Read this whole file before changing signal logic.** Every numeric threshold below was either backtested or is load-bearing for a backtested result. Rules for changes are at the bottom ("Rules for making changes").
+
+---
+
+## Repository map
+
+| Path | Lines | Purpose |
+|------|-------|---------|
+| `config.py` | ~374 | All config dataclasses, `PAIR_INFO` registry, per-pair strategy configs. **`PlatformConfig` defaults ARE the tuned XAU config.** |
+| `data_provider.py` | ~347 | Twelve Data SDK wrapper: fetch, rate-limit (55 credits/min), in-memory cache, validation/cleaning, freshness checks |
+| `engine.py` | ~1173 | `TechnicalEngine`: computes all indicators on one DataFrame → `TechnicalAnalysis` (composite score, regime, key levels) |
+| `signal_aggregator.py` | ~1447 | `SignalAggregator`: MTF analysis, news integration, gates/penalties, quality grade, trade-setup generation → `SignalReport` |
+| `news_engine.py` | ~680 | Economic calendar (Forex Factory JSON + MyFXBook RSS fallback) + keyword RSS sentiment |
+| `telegram_bot.py` | ~1259 | Async Telegram bot: candle-boundary auto-scanner, WebSocket level-breach monitor, owner lock, subscriber channel |
+| `pair_signals/__init__.py` | 52 | `PAIR_PRODUCERS` registry + `produce_signal()` |
+| `pair_signals/xauusd.py` | 71 | XAU config factory + per-timeframe overrides (docstring = validation record) |
+| `pair_signals/eurusd.py` | 158 | EUR config factory + EUR regime weights (docstring = no-edge verdict + tuning history) |
+| `main.py` | ~348 | CLI: `python main.py --pair XAUUSD --timeframe H1 --detail/--json` |
+| `test_all.py` | ~2361 | QA suite: **179 tests in 10 modules**, no live API needed. Must stay green. |
+| `backtest/gpu_backtest.py` | ~2231 | Bar-replay backtester (CuPy→NumPy fallback) using the real `SignalAggregator` |
+| `backtest/diagnose_filters.py` | 609 | Filter-diagnosis tooling |
+| `backtest/research/` | — | Calendar research infra (`ff_calendar_scraper.py`, `calendar_store.py`) + result CSVs |
+| `backtest/*.csv` | — | Verification trade books from past tuning campaigns (`xau_*.csv`, `eur_verify.csv`, …). **Gitignored, local-only** — used for manual before/after comparison, not by tests. `eth/btc/gbp_verify.csv` come from research whose pair code is NOT on main |
+| `backtest/cache/` | — | Parquet OHLCV cache (raw bars) |
+| `test_ws_dual.py` | 108 | Standalone WebSocket dual-connection probe |
+| `logs/telegram.log` | — | Bot log (midnight rotation, 30 backups) |
+
+---
+
+## Pair status (do not confuse the two)
+
+- **XAU/USD**: the tuned, validated product. It is the only entry in `PlatformConfig.scan_pairs`. All defaults in `PlatformConfig`/`XauStrategyConfig` are its tuned values.
+- **EUR/USD**: fully wired (factory, gates, tests) but **not in `scan_pairs`** and its signals are informational only. Verdict (2026-07-11/12, recorded in `pair_signals/eurusd.py` docstring): no configuration passed validation on M5, M15, or H1 at a 1-pip round-trip cost; root cause is cost-to-volatility (EUR M15 ATR ≈ 10 pips → 1-pip spread taxes ~10% of ATR; gold's ATR/spread ratio is several times larger). Calendar-surprise and H1-hybrid avenues were also tested and closed. Do not re-open EUR tuning without new data (tick fills or H4/D1 horizons).
+
+---
+
+## End-to-end pipeline (what happens on every scan)
 
 ```
-Twelve Data REST API + WebSocket
-        |
-  data_provider.py          (fetch, cache, validate OHLCV; rate-limited 55 calls/min)
-        |
-    engine.py                (9 indicators per timeframe → composite score)
-        |
-  signal_aggregator.py       (MTF agreement + news + final signal + trade setup)
-        |
-  pair_signals/              (per-pair config factories: xauusd.py, eurusd.py)
-        |
-  +-----+--------+
-  |              |
-telegram_bot.py  main.py
-(Telegram alerts, (CLI per-pair
- auto-scanner,    analysis)
- WS monitor)
+Twelve Data REST (+ WebSocket ticks in the bot)
+  → data_provider.fetch_ohlcv()        per timeframe: throttle, fetch, drop forming candle, validate/clean, cache
+  → engine.TechnicalEngine.analyze()   per timeframe: 9 scored indicators + ATR context → regime → composite score
+  → signal_aggregator.analyze_pair()   live entry point (weekend/day filters, fetch, staleness)
+      └─ analyze_from_dataframes()     replayable core: MTF agreement → news → bonuses/penalties →
+                                       final signal + quality grade → gates (session, SELL, M5, EUR, regime) →
+                                       trade setup (3 generators) → reasons
+  → SignalReport                       consumed by telegram_bot.py, main.py, backtest/gpu_backtest.py
 ```
 
-**Per-pair modularity:** each pair registers a `make_config(primary_tf, confirmation_tfs) -> PlatformConfig` factory in `pair_signals/`. `PlatformConfig.scan_pairs` is the single switch for which pairs the bot scans (single- or multi-pair). Market constants (pip value, price precision, backtest default spread) live in `PAIR_INFO`; strategy knobs live in per-pair dataclasses (`XauStrategyConfig`, `EurUsdStrategyConfig`). All XAU-specific gates in the aggregator are display-gated — nothing gold-calibrated touches the EUR path, and vice versa (enforced by pair-isolation tests + a pinned-window XAU regression CSV).
+Signal currency everywhere is the `SignalStrength` enum: `STRONG_BUY=2, BUY=1, NEUTRAL=0, SELL=-1, STRONG_SELL=-2`.
 
-## Core Files
+---
 
-| File | Purpose |
-|------|---------|
-| `config.py` | All configuration dataclasses, `PAIR_INFO` registry, per-pair strategy configs |
-| `data_provider.py` | Twelve Data SDK wrapper with in-memory cache and rate limiting |
-| `engine.py` | `TechnicalEngine` — computes EMA, RSI, BB, ADX, ATR, MACD, ROC, Keltner, divergences, candlestick patterns, swing S/R |
-| `signal_aggregator.py` | `SignalAggregator` — MTF analysis, news integration, diminishing penalty cap, trade setup generation |
-| `news_engine.py` | Economic calendar (Forex Factory JSON) + RSS sentiment (keyword-based) |
-| `telegram_bot.py` | Async Telegram bot with auto-scanner (M5/M15/H1/H4), WebSocket price monitor |
-| `pair_signals/` | Per-pair producer registry (`PAIR_PRODUCERS`) + config factories |
-| `main.py` | CLI entry point (`--pair XAUUSD\|EURUSD`) |
-| `test_all.py` | Comprehensive QA suite (~176 tests, no live API needed) |
+## config.py — every knob
 
-## Key Concepts
+### PAIR_INFO (market constants; keys are the canonical symbols `"XAU/USD"`, `"EUR/USD"`)
+| Field | XAU/USD | EUR/USD | Meaning |
+|---|---|---|---|
+| `pip_value` | 0.10 | 0.0001 | price units per pip |
+| `price_precision` | 2 | 5 | decimals for SL/TP rounding & display |
+| `default_spread` | 0.30 | 0.0001 | round-trip cost charged by the backtester when `--spread` not given |
 
-**Signal Pipeline:** OHLCV → indicators → regime classification → regime-weighted composite score → MTF agreement check → news/event adjustment → diminishing penalty cap → final signal (STRONG_BUY/BUY/NEUTRAL/SELL/STRONG_SELL) → quality grade (A+/A/B/C/D/F) → trade setup if grade C+.
+### TechnicalConfig (indicator parameters)
+EMA 9/21/50/200 (`ema_mode`: `"4ema"` default or `"2ema"` = 50/200 only); RSI 14 (OB 70 / OS 30 / neutral zone 40–60; `rsi_mode`: `"regime_aware"` default or `"classic"`); MACD 12/26/9 (`macd_as_pillar=True` — MACD is a scored indicator, not just divergence input); BB 20/2.0σ; ATR 14; ADX 14 (strong ≥25, very strong ≥50); divergence lookback 40; swing-S/R lookback 50; ROC period 10, threshold 1.0%; Keltner: EMA 20 mid ± 1.5 × ATR(10); ADX slope lookback 5, "emerging" threshold 3.0; `disabled_indicators` list; `regime_weights_override` dict (used by EUR).
 
-**Regime-Adaptive Weights:** Market classified as `trending_up`, `trending_down`, `ranging`, or `transitioning` via ADX + EMA + ADX slope. Gold-specific weights per regime:
-- **Trending:** MACD (2.0), EMA (2.0), Keltner (1.5), ROC (1.5), Divergence (1.0)
-- **Ranging:** Divergence (2.5), Keltner (2.0), RSI (1.5), BB (1.5), MACD (1.5)
-- **Transitioning:** Keltner (2.5), MACD (2.5), ROC (2.0), Divergence (2.0), EMA (1.5)
+### SignalConfig (scoring & trade geometry — XAU-calibrated; EUR uses its own copies)
+| Field | Default | Meaning |
+|---|---|---|
+| `mtf_agreement_threshold` | 2 | timeframes (of primary + confirmations) that must agree |
+| `min_rr_ratio` | 0.5 | reject setups below this RR |
+| `tp_tightening_factor` | 1.0 | TP scale factor. **Leave at 1.0**: 0.8 was re-tested 2026-07-08 with spread+gap fills and nearly halved total R |
+| `round_trip_cost` | 0.30 | XAU $ round trip |
+| `min_tp_cost_multiple` | 30.0 | spread gate: TP1 distance must be ≥ cost × this (= $9 on XAU). Validated: sub-$10 targets lose |
+| `min_sl_pips` | 80.0 | SL floor in pips ($8 on XAU) |
+| `tp_floor_rr` | 0.5 | TP1 floor as fraction of SL distance (M15 factory overrides to 1.0) |
+| `trail_arm_r` / `trail_dist_r` | 0.6 / 0.75 | trailing exit: once favorable excursion ≥ arm×risk, stop trails dist×risk behind best price (bar-close updates). TP1 becomes advisory. Validated on 12mo M5: +105R vs +48R static. 0 disables |
+| `pb_trail_arm_r` / `pb_trail_dist_r` | 0.8 / 0.75 | trailing params for XAU_PB pullback trades (tighter stops → arm later) |
+| `weights`, `min_confidence`, `cooldown_bars`, `default_risk_pct`, `default_rr_ratio`, `atr_sl_multiplier` | — | **LEGACY / UNUSED** outside tests. The engine has its own weights; backtest cooldown lives in `BacktestConfig` |
 
-ROC is directionally redundant with MACD/Keltner/EMA (0.8–0.94 correlation) but its weight is load-bearing: composite thresholds are calibrated with it — removing it backtested 13R worse. Don't zero it without retuning thresholds.
+### NewsConfig
+Tracked currencies `["USD","XAU","EUR"]`; pre-event buffer 30 min, post-event 15 min (45 min after a *surprise* print); `event_risk_caps_signals=False` — **XAU default: event risk warns but does NOT cap confidence** (2026-07-11 audit: the caps suppressed trades worth +4.6..+7.2R/yr; XAU trades near USD releases outperform). EUR factory turns the cap back ON. Context cache TTL 600 s (must stay < pre-event buffer). 4 RSS feeds (investinglive, FXStreet, Google-News gold query, CNBC economy).
 
-**RSI (regime-aware, default):** In trends RSI buys pullback zones instead of fading strength (classic mode backtested worse for gold). In `transitioning` only extremes (≥70/≤30) signal.
+### XauStrategyConfig (all XAU gates; `enabled=True`)
+| Field | Default | Notes |
+|---|---|---|
+| `blocked_hours_utc` | [0,1,2,10,11,13,16,21] | bar-close hours never traded. M5 factory appends 9 (post-London-open whipsaw, −28.9R/143 over 6 yr) |
+| `allowed_weekdays` | [0,1,2,3,4,6] | Mon–Fri + Sun (Sydney open). `weekend_filter=True`: Fri 22:00 → Sun 22:00 UTC closed |
+| `min_atr_threshold` | 1.0 | $1 ATR floor for swing-based setups |
+| `v2_sl_atr_multiplier` | 2.0 | SL ATR floor in the generic setup generator |
+| `sl_atr_multiplier` / `tp_atr_multiplier` | 2.0 / 1.0 | used by the XAU ATR setup generator (A+/A only) |
+| `sell_min_quality` | ["A+","A","B"] | SELLs below B are blocked |
+| `sell_vol_regimes` | ["high","extreme"] | SELLs only in these volatility regimes (quiet shorts fade vs gold's macro uptrend) |
+| `blocked_regimes` / `sell_blocked_regimes` | [] / [] | per-TF factory opt-ins. M15 factory sets `blocked_regimes=["trending"]` (mixed regime: strong ADX, EMAs unaligned — lost −6.8R/24) |
+| M5 gates | | `m5_require_mtf=True`; `m5_min_abs_score=30`; `m5_min_confirmation_score=10`; `m5_sell_max_quality=["B","C"]` (A-grade M5 SELLs enter at dip extremes and lose); `m5_blocked_vol_regimes=["high"]` (chop transition — 61% WR, negative total); `m5_sell_min_rsi=25` |
+| M5 pullback (XAU_PB) | | `m5_pullback_enabled=True`; min score 15 (lower than gate 1 BY DESIGN — mid-pullback scores weaker); ADX ≥ 20; RSI zones BUY (35,62) / SELL (38,65); retrace depth 0.8–6.0 ATR from the 40-bar swing extreme; band touch within 3 bars. Validated +38.3R/254 trades on 12mo |
 
-**Divergence:** 40-bar lookback; price and indicator swings must align within ±2 bars to count as one divergence observation.
+Legacy fields kept for config-shape compatibility: `rsi_entry_threshold`, `min_ema200_tfs`, `allow_generic_trades`, `buy_threshold_boost`, `m5_confirmation_bars`, `h4_rsi_gate=False`, `h1_macd_gate=False`, `rsi_zone` — not wired to current gates.
 
-**Transitioning Regime:** ADX < 25 but ADX slope > 3.0 = trend emerging. Catches trend birth 5-10 bars before ADX crosses 25. Boosts quality grade for early entries.
+### EurUsdStrategyConfig (EUR-local; nothing gold-calibrated leaks in)
+Blocked hours [22,23,0,1,2,3]; weekdays Mon–Fri (M5 factory: Mon–Thu, Fridays bled −17R); `min_atr_threshold=0.0003`; SL: swing-buffer ×1.5 ATR, floor ×1.2 ATR (M5: 0.3/0.7), min 8 pips; `min_rr_ratio=1.0`; `mature_move_threshold=35` (|score|>35 → tighter TP 1.2 ATR, else 1.8 ATR); `blocked_regimes=["ranging","transitioning"]`; `min_quality_grades=["A+","A","B"]`; `min_entry_score=38` / `min_entry_confidence=60` (M5 factory: 35/45); `block_rsi_extended=True` (no BUY at RSI>70, no SELL at RSI<30); costs: `round_trip_cost=0.0001`, `min_tp_cost_multiple=10` (10-pip TP minimum), `min_sl_pips=8`. `be_trigger_r`/`be_buffer_r` are legacy (superseded by trailing exits).
 
-**Diminishing Penalty Cap:** Penalties sorted by magnitude, applied with exponential decay (1st at 100%, 2nd at 50%, 3rd at 25%). Hard cap at 30% total. Replaces old additive stacking that could gut 30-50% confidence.
+### FilterConfig (`config.backtest` — despite the name it applies LIVE too)
+`session_filter=True`, window 03:00–23:00 UTC, `blocked_hours_utc` mirrors XAU's list (non-XAU pairs read this copy), `allowed_weekdays` Mon–Fri+Sun. `regime_filter=False` (optional H4-ADX≥25 gate, off).
 
-**Indicators (9 scored):** EMA Trend, RSI, Bollinger Bands, MACD, ADX, Keltner Channels (TTM Squeeze), ROC (Rate of Change), Divergence, Candlestick Patterns.
+### PlatformConfig (top level)
+`symbol="XAU/USD"`, primary TF H1, confirmations [M15, H4], `data_lookback_days=60`, `scan_pairs=[XAUUSD]` (**the single switch for what the bot scans**), `scan_timeframes=[M5,M15,H1,H4]`, `candle_buffer_seconds=10`, `ws_cooldown_seconds=300`, hardcoded `twelve_data_api_key` fallback.
 
-**Trade Setup:** SL = nearest swing low/high ± 1×ATR buffer (min 1.5×ATR). TP = nearest swing level if RR≥1.2, else calculated from ATR. Minimum RR 0.5:1.
+---
 
-## How to Run
+## engine.py — TechnicalEngine
 
+**Input contract:** pandas DataFrame, UTC tz-aware DatetimeIndex, oldest-first, columns `Open High Low Close Volume` (capitalized). `< 30` bars → empty NEUTRAL result. EMA200 needs ≥ 200 bars (otherwise EMA-trend indicator degrades gracefully).
+
+**Compute order matters:** ATR first (EMA-ribbon and candlestick proximity use it) → EMA → RSI → BB → ADX → MACD (always computed, even if not a pillar — divergence needs the histogram) → ROC → Keltner (needs BB levels + MACD histogram) → divergence → structure S/R → candlesticks → regime classification → regime-aware RSI reinterpretation → composite.
+
+Results land in `TechnicalAnalysis.indicators: Dict[str, IndicatorSignal]` (fields: name, value, signal, confidence 0-100, description, weight). Private series ride on the result object as underscore attrs (`_rsi_series`, `_macd_histogram`, `_adx_slope`, `_kc_squeeze`, `_kc_squeeze_released`) and are skipped by the composite scorer and all formatters (skip rule: name starts with `_` or `weight == 0`).
+
+### The 9 scored indicators (+ ATR as context)
+
+| Key | Base weight | Signal logic (exact) |
+|---|---|---|
+| `ema_trend` | 1.0 | 4-EMA mode: alignment score over checks (price>EMA9, 9>21, 21>50, 50>200), normalized to [-1,1]. >0.6 STRONG_BUY (conf ≤90), >0.2 BUY, mirrored sells, else NEUTRAL (40). Fast/medium cross ±10 conf. Ribbon (9/21/50 spread < 1 ATR) = "COMPRESSED"; compressed→expanded = +10 conf. Emits `ema_fast/medium/slow/trend` key levels |
+| `rsi` | 1.5 | Wilder RSI (ewm α=1/14). Classic pass first: ≥70 SELL, ≤30 BUY, ≥60 BUY-momentum (45), ≤40 SELL-momentum (45), else NEUTRAL (30). **Then overwritten by regime reinterpretation** (below) when `rsi_mode="regime_aware"` |
+| `macd` | 1.5 | Histogram = MACD line − signal line. Cross above 0 → BUY (conf 55+); accelerating (hist>prev>0) → STRONG_BUY; mirrored down; positive-but-fading / negative-but-fading → NEUTRAL (35); flat → NEUTRAL (30). Confidence scaled by |hist| vs its 20-bar mean |
+| `bollinger` | 1.5 | Position = (close−lower)/(upper−lower). >1.0 SELL (65), <0.0 BUY (65), >0.8 SELL (45), <0.2 BUY (45), else NEUTRAL (30). Squeeze = width < 0.75 × 20-bar avg width → +10 conf. Emits `bb_upper/lower/mid` |
+| `adx` | 0.5 | Wilder ADX/DI (ties zero BOTH DMs — fixes a systematic bearish nudge). ADX ≥ 25 → BUY/SELL by DI dominance, else NEUTRAL. Conf 40/65/80 by ADX band. ADX slope (5-bar diff): <25 with slope > 3.0 → "TREND EMERGING" +15 conf; also strengthening/weakening adjustments. Sets `_adx_slope` |
+| `roc` | 1.5 | 10-period %change. Positive & rising → BUY (STRONG if >2× threshold=2%); negative & falling → SELL; zero-cross → BUY/SELL (65); else NEUTRAL. **Directionally redundant with MACD/Keltner/EMA (0.8–0.94 corr) but its weight is load-bearing: zeroing it backtested 13R WORSE — composite thresholds are calibrated with it. Do not remove without retuning thresholds** |
+| `keltner` | 1.5 | EMA20 mid ± 1.5×ATR(10). TTM squeeze = BB fully inside KC. Squeeze released (was→not) + MACD hist sign → STRONG_BUY/STRONG_SELL (80); squeeze active → NEUTRAL (60); close above/below channel → BUY/SELL continuation (65); inside → NEUTRAL (30). Emits `kc_upper/lower/mid` |
+| `divergence` | 1.5 | RSI + MACD-hist vs price over 40 bars. Swing points (window 3) must align within ±2 bars to pair up; last two pairs: price LL + indicator HL = bullish, price HH + indicator LH = bearish. Any found → BUY/SELL/NEUTRAL-if-both, conf 70, value = count |
+| `candlestick` | 1.0 | Last bar: Doji (body<10% range), Hammer/Hanging Man (lower wick>2×body), Shooting Star/Inverted Hammer, Engulfing (vs prev bar), Marubozu (body>85%). Majority vote → signal, conf 50; +20 if pattern sits within 0.5 ATR of a swing S/R level |
+| `atr` | **0.0** (context) | Wilder ATR. Volatility regime from percentile vs last 50 ATR values: >80 extreme, >60 high, >30 normal, else low. Emits `atr`, `atr_pct` |
+
+Structure S/R (not scored): swing highs/lows (window 3) over `swing_sr_lookback` (min 120 bars on 1min/5min TFs); up to 3 above and 3 below stored as `swing_high_1..3` / `swing_low_1..3` key levels.
+
+### Market regime classification (exact rules)
+- ADX ≥ 25 **and** EMA-trend signal bullish → `trending_up`; bearish → `trending_down`; EMA mixed → **`trending`** (the "mixed" regime — momentum without directional structure; treated as direction-UNKNOWN everywhere; a past bullish-bias bug here cost −6.8R and is fixed)
+- else ADX slope > 3.0 → `transitioning` (trend birth, ~5–10 bars before ADX crosses 25)
+- else → `ranging`
+
+### Regime-aware RSI reinterpretation (`rsi_mode="regime_aware"`, the default — classic mode backtested worse for gold)
+| Regime | RSI zones |
+|---|---|
+| trending_up | 38–55 STRONG_BUY 75 ("pullback zone, ideal entry"); 30–38 BUY 60; 55–65 BUY 50; >65 NEUTRAL ("extended, wait"); <30 NEUTRAL ("possible reversal") |
+| trending_down | mirrored: 45–62 STRONG_SELL 75; 62–70 SELL 60; 35–45 SELL 50; <35 / >70 NEUTRAL |
+| transitioning / trending (mixed) | ONLY extremes: ≥70 SELL 50, ≤30 BUY 50, else NEUTRAL 30 |
+| ranging | ≥70 SELL 70, ≤30 BUY 70, ≥60 SELL 45, ≤40 BUY 45, else NEUTRAL |
+
+### Composite score
+For each scored indicator: `effective_weight = base_weight × regime_multiplier`; contribution = `signal.value × effective_weight × confidence/100`. Composite = `(Σ contributions / Σ effective_weights) × 50`, clamped to ±100. Confidence = weight-averaged indicator confidence. Composite signal: ≥40 STRONG_BUY, ≥15 BUY, ≤−40 STRONG_SELL, ≤−15 SELL, else NEUTRAL.
+
+**Gold regime multipliers** (defaults in `_compute_composite`; `technical.regime_weights_override` replaces the whole dict — EUR does this):
+
+| Indicator | trending_up/down/trending | ranging | transitioning |
+|---|---|---|---|
+| ema_trend | 2.0 | 0.3 | 1.5 |
+| rsi | 1.0 | 1.5 | 1.0 |
+| macd | 2.0 | 1.5 | 2.5 |
+| bollinger | 0.5 | 1.5 | 1.0 |
+| keltner | 1.5 | 2.0 | 2.5 |
+| roc | 1.5 | 0.5 | 2.0 |
+| candlestick | 0.8 | 1.0 | 1.0 |
+| divergence | 1.0 | 2.5 | 2.0 |
+| adx | 0.8 | 0.5 | 1.0 |
+
+---
+
+## signal_aggregator.py — SignalAggregator
+
+Two entry points:
+- **`analyze_pair(symbol)`** — LIVE: applies XAU/EUR weekend + allowed-weekday filters on wall clock, fetches all TFs (primary + confirmations), flags stale data (primary stale → signal suppressed to NEUTRAL/F), delegates to `analyze_from_dataframes`, then applies the live session check on the **bar-CLOSE time** (bar timestamp + candle duration): outside 03:00–23:00 or in blocked hours → NO TRADE. This is the exact predicate the backtester replays, so live == backtest.
+- **`analyze_from_dataframes(symbol, dfs, context, use_news)`** — replayable core; treats each df as the visible history at decision time (caller must slice; no weekend filter here — the backtester applies it).
+
+### Pipeline inside `analyze_from_dataframes` (numbered as in the source)
+1. Primary TF analysis → seeds report (score, confidence, regime, vol regime, price, timestamp = last bar's index label).
+2. Confirmation TF analyses → `mtf_signals` / `mtf_scores` keyed by TF string ("5min", "1h", …).
+3. **MTF agreement**: count BUY-ish vs SELL-ish across ALL TFs (primary included); agreement = count ≥ 2 **in the primary's direction** (two bullish confirms cannot "confirm" a bearish primary).
+4. News (if `use_news`): event risk (`clear`/`caution`/`danger`) + pair sentiment (−1..+1). Backtests pass `use_news=False`.
+5. Spread estimate = median(High−Low) of primary df.
+6. **`_compute_final_signal`** (bonuses → event cap → penalties → thresholds → grade):
+   - Bonuses: MTF agreement → +10 conf and `score = 0.6×primary + 0.4×mean(all TF scores)`; news sentiment aligned (|sent|>0.2) → +5; in a *directional* trend, RSI in pullback zone → +5, BB position favorable → +5 (mixed `trending` regime gets NO directional bonus).
+   - Event risk `danger`: warning always; confidence hard-capped at 30 ONLY if `news.event_risk_caps_signals` (False for XAU, True for EUR).
+   - **Penalty system (diminishing, replaces old additive stacking):** collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: `mtf_disagreement` 15; `news_conflict` 10; `event_caution` 15 (config-gated); `counter_trend` 15; `extreme_vol` 10; `ema200_near` 5 (price within 0.5% of EMA200 — env `FB2_EMA_LO` overrides); `ema200_far` 5 (>2.5%); `exhaustion` 8 (price >3.5 ATR beyond nearest swing); `no_pullback` 8 (>2.5 ATR from EMA21 in trend) / `weak_pullback` 5 (>2.0); `rsi_extended` 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed `trending`); `bb_overextended` 8 (BB pos >0.85 BUY / <0.15 SELL).
+   - **Exhaustion hard block**: ≥2 exhaustion-tagged penalties ({exhaustion, no_pullback, rsi_extended, bb_overextended} → `report.exhaustion_count`) AND momentum fading (ROC sign against score, or MACD signal against score) → confidence = 0.
+   - Confidence clamped [0, 95]. **Final signal**: score ≥35 & conf ≥55 STRONG_BUY; ≥25 & ≥50 BUY; mirrored sells; else NEUTRAL.
+   - **Quality grade**: |score|≥40 & conf≥65 & MTF → **A+**; ≥33 & ≥55 & MTF → **A**; ≥28 & ≥50 → **B**; ≥25 & ≥45 → **C**; ≥15 → **D**; else **F**.
+7. Post-grade gates, in order:
+   - **6b session filter** (bar-label hour + weekday; XAU reads `xau_strategy.blocked_hours_utc`/`allowed_weekdays`, others read `config.backtest.*`) → NEUTRAL/F.
+   - **6c transitioning boost**: regime `transitioning` and grade B/C/D → A if |score|≥30 & conf≥50 & MTF; else B if |score|≥25 & conf≥45 (early trend entries).
+   - **6d exhausted-trend cap**: trending regime + exhaustion_count ≥2 + grade C/D → D, setup killed.
+   - **6e XAU SELL quality gate**: sell-direction & grade not in `sell_min_quality` → D.
+   - **6f MTF opposition**: ≥2 TFs pointing against the trade direction → D (never trade against consensus).
+   - **6g XAU SELL volatility gate**: sell-direction & vol regime not in {high, extreme} → D.
+   - **6g2 XAU regime gates**: regime in `blocked_regimes` (M15: mixed `trending`) → D; sell-direction & regime in `sell_blocked_regimes` → D.
+   - **6h EUR gates** (EUR only): blocked regimes (ranging/transitioning), quality < B, exhaustion count, session hours, |score| < min_entry_score, conf < min_entry_confidence, blocked vol regimes, indicator-agreement floor, RSI-extended block → each sets grade F with a warning.
+   - **6i XAU M5 noise gates** (`_apply_m5_filters`, primary TF = M5 only): (1) |score| ≥ 30; (2) MTF agreement required; (3) ≥1 confirmation TF with score ≥ ±10 in trade direction (NEUTRAL confirms don't count); (4) SELL grade must be B/C (A-grade M5 shorts lose −0.017R/trade); (5) vol regime `high` blocked; (6) SELL blocked when RSI < 25. Each failure → D, setup killed.
+8. **Trade setup**: XAU + grade A+/A → `_generate_xau_atr_trade_setup` first; any pair + grade A+/A/B/C and no setup yet → `_generate_trade_setup` (swing-based).
+9. Reasons: every non-neutral indicator description → `bullish_reasons`/`bearish_reasons`; EUR setups get an `"EUR STRATEGY: …"` marker inserted at reasons[0] (the backtester tags strategy from reasons[0]).
+10. **XAU_PB pullback** (M5 XAU only, `m5_pullback_enabled`, no setup yet, grade ≠ F): `_try_m5_pullback` — see below.
+
+### Trade setup generators (exact geometry)
+
+**A. `_generate_xau_atr_trade_setup`** (XAU, grade A+/A, ATR>0): TP quality scale A+ 2.0 / A 1.5 (A with exhaustion≥2 → 1.0). TP1 = entry ± `tp_atr_multiplier`(1.0)×scale×ATR; TP2 = 1.5×that; SL = entry ∓ 2.0×ATR. Gates: RR ≥ 0.5; TP1 distance ≥ $9 (spread gate). Prices rounded to 2 dp; pip 0.10 hardcoded — **this generator is XAU-only by construction**.
+
+**B. `_generate_trade_setup`** (swing-based, any pair, grade A+..C): XAU requires ATR ≥ $1 (LOW ATR FILTER). Per-side:
+- SL anchor = nearest swing low below price (BUY) / swing high above (SELL) ∓/± `sl_swing_buffer`×ATR (XAU 1.0, EUR = `eur.sl_atr_multiplier`); fallback (no swing) = (1+buffer)×ATR. SL distance floored at max(ATR × sl_atr_floor, min_sl_pips×pip) — XAU: max(2.0×ATR, $8); EUR: max(1.2×ATR, 8 pips) (M5 EUR: 0.7×ATR).
+- TP1 = nearest swing beyond entry ± 0.3 ATR if it's within `tp_swing_cap` ATR (XAU 2.5) AND ≥ SL×RR-floor; else entry ± max(`tp_mult`×ATR, SL×RR-floor). XAU: tp_mult 1.5, RR-floor = max(min_rr 0.5, tp_floor_rr — 1.0 on M15). EUR: tp_mult 1.2 (mature move, |score|>35) or 1.8, RR-floor 1.0.
+- TP2 = second swing if within 5 ATR, else entry ± max((tp_mult+1.5)×ATR, SL×floor×1.5); forced beyond TP1 by ≥1 ATR.
+- Gates: RR ≥ min_rr; spread gate TP1 distance ≥ cost×multiple (XAU $9; EUR 10 pips).
+
+**C. `_try_m5_pullback` (XAU_PB)**: detection = EMA alignment ≥3 of 4 checks (9>21, 21>50, 50>200, close>200 — mirrored for SELL), ADX ≥ 20, price touched max(EMA21, KC-mid) within last 3 bars, resumption bar closed back in trend direction beyond EMA9, retrace depth 0.8–6.0 ATR from the 40-bar extreme, RSI in pullback zone (BUY 35–62 / SELL 38–65). Then gates: directional score ≥ 15; ≥1 confirmation TF ≥ ±10; not against MTF consensus; vol regime gates (same as breakout path). Geometry: SL = pullback 8-bar extreme ∓ 0.5 ATR (min $8); TP1 = 1.5×risk, TP2 = 2.5×risk; spread gate $9. Sets `strategy="XAU_PB"`, grade min B, marker `"XAU PULLBACK STRATEGY: …"` at reasons[0]. Rejection funnel counters in class attr `SignalAggregator.pb_funnel`.
+
+`TradeSetup` fields: pair, direction, entry, SL, TP1, TP2, RR, risk/reward pips, pip_value, `strategy` ("GENERIC" or "XAU_PB" — selects trailing params).
+
+---
+
+## news_engine.py — NewsEngine
+
+- **Calendar**: Forex Factory free JSON (`nfs.faireconomy.media/ff_calendar_thisweek.json`), 3 retries, honors 429 Retry-After with class-level backoff (≤600 s); fallback MyFXBook RSS. Never fabricates events — unavailable calendar = risk checks skipped that cycle.
+- **Caches are class-level** (a fresh NewsEngine is built per scan): calendar + headlines, TTL = 600 s on success, 60 s on empty result; lock held across fetch so concurrent scans share one request.
+- **Event risk** (`get_event_risk_level`): high-impact event for base/quote within 30 min → `danger`; within 60 min → `caution`; already-released high-impact → `caution` for 15 min after (45 min if actual ≠ forecast — "surprise"). `EconomicEvent.surprise` compares parsed numeric actual vs forecast (handles "%", "K/M/B" suffixes); direction is numeric-only (doesn't know if higher is bullish).
+- **Sentiment**: keyword sets over headlines from 4 RSS feeds, deduped by title (Google News " - Publisher" suffix stripped), last 24 h only. Per-currency scores; **`GOLD_INVERSE_KEYWORDS`** (rate hike/hawkish/tightening/rate cut/dovish/easing/stimulus) flip sign for XAU (hawkish = dollar-bullish = gold-bearish). Currency detection via compiled word-boundary-ish regexes (bare "us"/"eu"/"uk" deliberately excluded). Pair sentiment = base − quote.
+
+---
+
+## data_provider.py — ForexDataProvider
+
+- Twelve Data SDK (`TDClient.time_series(...).as_pandas()`), symbols normalized to `"XAU/USD"` form; renames lowercase columns → capitalized; sorts oldest-first; localizes UTC.
+- **Rate limit**: 55 credits/min sliding window (`_throttle`), blocks with sleep when exhausted.
+- **In-memory cache**: key `{symbol}_{tf}`, TTL = candle duration (M5 300 s … H4 14400 s); stale cached data (freshness check) is evicted and refetched.
+- **`_drop_forming_candle`**: Twelve Data includes the current partial candle; the scanner fires seconds after boundary, so the last bar is dropped if its close time is in the future. Without this every indicator would read a seconds-old stub and signals would repaint.
+- **Freshness** (`_is_data_fresh`): last bar older than 3× candle duration → stale (warns; live path suppresses signal if the PRIMARY TF is stale). Weekends exempt (Fri 22:00→Sun 22:00 UTC).
+- **`_validate_and_clean`**: drop all-NaN rows → ffill (limit 3) → drop NaN closes → sort + dedupe (keep last) → drop future timestamps (>now+5 min) → repair OHLC inconsistencies (High=max, Low=min of O/C/existing) → remove >10σ return outliers → warn if < ~210 bars.
+- `outputsize = max(300, min(days×bars_per_day+20, 5000))`; TD caps at 5000/call (the backtester paginates by date).
+- `get_latest_price` / `get_spread_estimate` (median M5 High−Low) helpers.
+
+---
+
+## pair_signals/ — per-pair factories (ALWAYS use these, never bare PlatformConfig for a pair)
+
+`PAIR_PRODUCERS: Dict[symbol, factory]`; factory signature `make_config(primary_tf, confirmation_tfs=None) -> PlatformConfig`. `produce_signal(symbol, tf, data_provider)` builds config → aggregator → `analyze_pair`. Registered: XAU/USD, EUR/USD.
+
+**A bare `PlatformConfig()` silently runs any pair with XAU defaults — always go through the factory** (the bot, CLI, and backtester all do).
+
+Default confirmation map (same in both factories, the bot, and backtest usage): M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1]. (Exception: `main.py`'s CLI map uses M5→[M1,M15] — a CLI-only quirk.)
+
+**xauusd.py**: `PlatformConfig()` defaults + per-TF overrides (each validated; full evidence in the module docstring):
+- M5: append hour 9 UTC to blocked hours (6-yr replay: +207.1R vs +178.3R; M15-and-up keep hour 9 — it's positive there).
+- M15: `signals.tp_floor_rr = 1.0` (1R minimum targets: +116.6R vs +105.4R, better in train/val and 2024 OOS; **hurts M5** — per-TF only) and `xau_strategy.blocked_regimes = ["trending"]` (mixed regime block: +65.7R vs +59.0R). A transitioning-SELL block was tested and REJECTED (bull-market artifact — failed 2024 OOS).
+- The factory must stay equivalent to inline `PlatformConfig(symbol=XAUUSD, …)` + overrides — **enforced by a test in test_all.py**.
+
+**eurusd.py**: sets symbol/display/pip, `xau_strategy.enabled = False` (belt-and-braces; XAU gates are also display-gated), `news.event_risk_caps_signals = True`, EUR session (7–20 h window, blocked [22,23,0-3], Mon–Fri), EUR regime-weight overrides (`EUR_REGIME_WEIGHTS`, and `M5_REGIME_WEIGHTS` + tighter trailing 0.45/0.5 + Mon–Thu + extra blocked hours [8,12,13,18,19] + swing lookback 20 on M5). Docstring = the complete no-edge verdict and closed research avenues.
+
+---
+
+## telegram_bot.py — the bot
+
+Run: `python telegram_bot.py` (env `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`; hardcoded fallbacks exist in the file).
+
+- **Owner lock**: every command/button is wrapped in `_owner_only` — only `TELEGRAM_CHAT_ID` gets responses; others get a refusal. The bot is the owner's control panel; the product is the channel.
+- **Subscriber channel** (`TELEGRAM_CHANNEL_ID`, `-100…`; `""` disables): auto-scan signal alerts + breach alerts also post there (bot needs channel-admin). Monetization = Telegram Stars subscription invite links; Telegram handles membership, the bot has zero payment code. The channel feed keeps running even when the owner toggles DM auto-scan off (`_feed_active()` = DM chats OR channel).
+- **Auto-scan**: boot-starts if `TELEGRAM_CHAT_ID` set (initial M15 scan populates the level cache first). Candle-boundary loops for M5/M15/H1/H4, firing at boundary + 10 s. Each scan runs all `scan_pairs` concurrently, updates the level cache, then broadcasts.
+- **Alert gating**: quality threshold (`_alert_min_quality`, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle.
+- **WebSocket monitor**: subscribes every `scan_pairs` symbol to TD quotes WS; 10 s heartbeat; exponential reconnect (≤60 s). On tick crossing any cached key level (`swing_high/low_1..3`, `ema_trend`, `bb_upper/lower`, `kc_upper/lower`; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown.
+- **Commands**: `/start` (menu), `/scan` (buttons: per pair — M5/M15/H1/H4 + Scan All TFs), `/autoscan` (toggle DM copy), `/status` (loops, WS, live prices, level cache, cooldowns), `/cache` (age per pair/TF), `/clearcache`, `/help`.
+- Signal messages include the full indicator breakdown, warnings, trade setup, **trailing-stop management plan** (arm price + trail distance, from Signal/pb params by `strategy`), and a risk disclaimer on every outbound message.
+- Blocking scan work runs in `ThreadPoolExecutor(max_workers=4)`; one shared `ForexDataProvider` across all scans (cache reuse). Logs to `logs/telegram.log`.
+
+---
+
+## main.py — CLI
+
+`python main.py [--pair XAUUSD|EURUSD] [--timeframe M5|M15|M30|H1|H4|D1] [--detail] [--json] [--verbose]`. Uses the pair factory, renders a compact terminal report (decision, setup, context, top-3 reasons per side, warnings) or full JSON via `report_to_dict`.
+
+---
+
+## backtest/gpu_backtest.py — the backtester
+
+Replays history bar-by-bar through the **real `SignalAggregator.analyze_from_dataframes`** — same gates, same setup math. News is disabled (`use_news=False`): backtests are pure technical.
+
+**Realism model (all validated fixes — do not regress):**
+- Entry at the **next bar's open** after the signal bar; SL/TP distances preserved relative to actual entry.
+- Round-trip `--spread` subtracted from every trade's PnL before computing R (default = pair's `default_spread`: XAU $0.30, EUR 1 pip). Results before 2026-07-08 (+59.5R baseline era) charged no spread — not comparable.
+- Trade resolution runs against the **RAW price path** (`fetch_history(clean=False)`): the parquet cache stores raw bars; cleaning (which drops >10σ news spikes that hit stops in reality) is applied only to the signal-side data. Caches written before this fix contain cleaned data — refetch to restore spike bars.
+- Same-bar SL+TP tie → SL first (conservative). Gap through SL/TP → filled at the bar open.
+- **Trailing exits** (default on, from `signals.trail_arm_r/trail_dist_r`; XAU_PB trades carry per-trade 0.8/0.75 overrides): stop ratchets at bar close only, so a new stop can only fill from the next bar (conservative). Exit reasons: SL / TRAIL / TP1 (static mode) / TIMEOUT / NO_DATA.
+- Session + weekend + weekday filters evaluate the **decision time = bar close** (`ts + primary_duration`) — exactly matching the live bot's wall-clock check.
+- Confirmation TFs are sliced to bars **fully closed** by decision time (no look-ahead).
+
+**Performance:** `precompute_indicators` builds every indicator series once per TF (CuPy on GPU or NumPy — check the log line `Backend: CuPy GPU` vs `NumPy CPU fallback`; CuPy needs the `nvidia-*-cu12` pip wheels or it silently falls back). `GPUTechnicalEngine` subclasses the live engine and overrides only the `_compute_*` methods with cache lookups — outputs are byte-identical to the live engine (regression-tested). Bar replay: pure per-bar evaluation fans out over forked worker processes (`--workers`, 0 = auto: cores−2 capped 12; <5000 bars forces 1); the entry **cooldown (5 bars) is the only sequential rule**, applied in an ordered pass afterwards — parallel and sequential books are identical. ~300 bars/s/core, ~1700 bars/s at 10 workers.
+
+**Strategy tags** (from reasons[0] markers): `XAU_PB`, `EUR`, `GENERIC` (plus legacy `XAU`, `GBP_PB` recognized). `--strategies` filters the book by tag.
+
+**CLI:**
 ```bash
-# CLI
-python main.py                    # XAU/USD H1 default
-python main.py --pair EURUSD --timeframe M5 --detail
-
-# Telegram bot (auto-scanner)
-TELEGRAM_TOKEN=<token> TELEGRAM_CHAT_ID=<id> python telegram_bot.py
-
-# Tests
-python test_all.py
-
-# Backtest (spread defaults per pair: XAU $0.30, EUR 0.0001 — see PAIR_INFO)
 python backtest/gpu_backtest.py --pair XAUUSD --days 365
+python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-01 --primary-tf 15min --confirm-tfs 1h,4h
 python backtest/gpu_backtest.py --pair EURUSD --primary-tf 5min --confirm-tfs 15min,1h --days 365
+# key flags: --spread --quality A+,A,B,C --strategies --workers --max-hold-bars 300
+#            --warmup-bars 250 --cooldown-bars 5 --no-cache --export-csv out.csv --detail
 ```
+Report: summary metrics (total R, PF, Sharpe on R, max DD, MFE/MAE), breakdowns by quality/strategy/regime, trade log, and per-trade reasoning dumps (indicator-by-indicator agreement at entry — required by `instructions_for_backtest.txt`). `--export-csv` flattens every indicator's signal/value/agreement into columns.
 
-## Environment Variables
+Data: paginated TD fetches (5000-bar chunks) cached as parquet in `backtest/cache/` keyed by symbol/TF/window; warmup pad ≥60 days before the window for EMA200.
+
+---
+
+## test_all.py — QA suite
+
+`python test_all.py` — **179 tests, 10 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, EUR/USD support). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: pair-isolation tests (no gold logic on EUR configs and vice versa), the xauusd-factory ≡ inline-config equivalence test, and engine bug-fix regression tests (Module 9b). **Any signal-logic change must keep this green and should add tests.**
+
+---
+
+## Environment variables
 
 | Variable | Purpose | Default |
-|----------|---------|---------|
-| `TWELVE_DATA_API_KEY` | Market data API key | Hardcoded fallback in config |
-| `TELEGRAM_TOKEN` | Telegram bot token | Hardcoded fallback |
-| `TELEGRAM_CHAT_ID` | Owner chat ID (enables boot auto-scan; only this user can drive the bot) | Hardcoded fallback |
-| `TELEGRAM_CHANNEL_ID` | Subscriber channel ID (`-100…`); signal + breach alerts mirror there; `""` disables | Hardcoded fallback |
+|---|---|---|
+| `TWELVE_DATA_API_KEY` | market data key | hardcoded fallback in `config.py` |
+| `TELEGRAM_TOKEN` | bot token | hardcoded fallback in `telegram_bot.py` |
+| `TELEGRAM_CHAT_ID` | owner chat id (boot auto-scan + owner lock) | hardcoded fallback |
+| `TELEGRAM_CHANNEL_ID` | subscriber channel (`-100…`); `""` disables the feed | hardcoded fallback |
+| `FB2_EMA_LO` | EMA200 "near" penalty threshold in % | `0.5` |
 
-## Dependencies
+Dependencies (`requirements.txt`): numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets. Python 3.12. A `venv/` exists in-repo.
 
-`numpy`, `pandas`, `requests`, `twelvedata`, `python-telegram-bot`, `websockets` — see `requirements.txt`.
+---
 
-## Data Provider
+## Code conventions
 
-- **Source:** Twelve Data REST API via `twelvedata` SDK
-- **Rate limit:** 55 credits/min (Grow plan), enforced by sliding-window throttle
-- **Cache:** In-memory dict, TTL = candle duration of the timeframe
-- **Validation:** Drops NaN rows, forward-fills gaps (limit 3), fixes OHLC inconsistencies, removes outliers (>10σ), strips future timestamps, deduplicates
-- **WebSocket:** subscribes every pair in `scan_pairs`
+- Type hints on public methods; dataclasses for all config and outputs.
+- `SignalStrength` (−2..+2) is the universal signal currency.
+- Key levels dict keys: `swing_high_*`, `swing_low_*`, `ema_*`, `bb_*`, `kc_*`, `atr`, `atr_pct`.
+- Private per-indicator series ride on `TechnicalAnalysis` as `_underscore` attributes; anything `_`-prefixed or weight-0 is skipped by scorers/formatters.
+- UTC everywhere; DataFrames indexed by tz-aware `Datetime`, oldest-first, capitalized OHLCV columns.
+- Logging via module loggers; the bot adds a rotating file handler.
 
-## Telegram Bot
+---
 
-- **Auto-scan:** M5, M15, H1, H4 at candle boundaries + 10s buffer (all timeframes alert)
-- **Manual scan:** Buttons for M15/H1/H4 individual scans + "Scan All TFs" multi-timeframe summary
-- **WebSocket:** Streams live ticks for XAU/USD; triggers immediate M15 scan on key level breach (swing S/R, BB, KC, EMA200) with 5-min cooldown
-- **Commands:** `/start`, `/scan`, `/autoscan`, `/status`, `/cache`, `/clearcache`, `/help`
-- **Owner lock:** all commands/buttons respond only to `TELEGRAM_CHAT_ID`; others get a refusal message
-- **Subscriber channel:** with `TELEGRAM_CHANNEL_ID` set, auto-scan + breach alerts also post to the channel (bot must be channel admin). Monetized via Telegram Stars subscription invite links — Telegram handles membership; the bot has no payment code. Channel feed keeps running even if the owner's DM auto-scan is off
+## Rules for making changes (READ BEFORE EDITING SIGNAL LOGIC)
 
-## Backtesting
-
-The `backtest/` directory contains `gpu_backtest.py` — a GPU-accelerated backtester (CuPy with NumPy fallback) that replays historical OHLCV through the real SignalAggregator. Pre-computes all indicator series including ROC, Keltner, ADX slope for 100x+ speedup. Uses parquet caching for data persistence.
-
-Fill realism: charges a round-trip spread per trade (`--spread`; defaults to the pair's `default_spread` in PAIR_INFO), fills gap-through SL/TP at the bar open, and resolves trades against the RAW price path (the parquet cache stores raw bars; validation/cleaning runs on load — signals see cleaned data, trade resolution sees real spikes). Entry is at the next bar's open after the signal bar. Weekend/session filters evaluate bar-close (decision) time to match the live bot's wall clock.
-
-GPU note: CuPy needs the NVIDIA runtime pip wheels (`nvidia-cuda-runtime-cu12`, `nvidia-cuda-nvrtc-cu12`, `nvidia-curand-cu12`, `nvidia-nvjitlink-cu12`) or it silently falls back to NumPy — check the "Backend:" line in the run log. GPU accelerates indicator precompute and SL/TP scanning; the bar-replay loop is Python (~300 bars/s/core) but fans out across forked worker processes (`--workers`, default auto = cores−2 capped at 12, ~1700 bars/s at 10). Per-bar signal evaluation is pure, so parallel and sequential books are identical (the entry cooldown is applied in an ordered pass afterwards); `--workers 1` forces the sequential path.
-
-## Code Conventions
-
-- Python 3.12, type hints on public methods
-- Dataclasses for all config and output structures
-- `SignalStrength` enum (-2 to +2) is the universal signal currency
-- Indicators stored as `Dict[str, IndicatorSignal]` in `TechnicalAnalysis`
-- Key levels stored as `Dict[str, float]` — prefixed `swing_high_`, `swing_low_`, `ema_`, `bb_`, `kc_`, `atr`
-- Private indicator data attached as `result._rsi_series`, `result._macd_histogram`, `result._adx_slope`, `result._kc_squeeze` (underscore prefix, skipped by composite scorer)
-- Blocking signal computation runs in `ThreadPoolExecutor(max_workers=4)` to avoid blocking the async Telegram event loop
-- Shared `ForexDataProvider` instance across all scans for cache reuse
+1. **Never change a tuned value without re-validation.** The acceptance rule used throughout: positive total R in BOTH the train and validation halves of a ≥12-month window, PF ≥ 1.3, at realistic cost (XAU $0.30 / EUR 1 pip round trip), with trailing exits and next-open fills. For EUR (and any marginal edge) additionally re-test on the 2024-06→2025-03 out-of-sample window — within-year splits alone have approved configs that failed OOS.
+2. **Per-timeframe overrides live in the pair factory** (`pair_signals/*.py`), never in global defaults — the same knob has opposite effects on different TFs (proven twice: `tp_floor_rr`, hour-9 block). Document the validation evidence in the factory docstring.
+3. **Pair isolation is a hard invariant.** XAU gates in the aggregator are display-gated (`report.display_name == "XAU/USD"`); EUR reads only `EurUsdStrategyConfig` for costs/geometry. Tests enforce zero XAU drift when EUR changes and vice versa.
+4. **Live == backtest parity**: session/weekend/hour filters must evaluate bar-CLOSE time in both paths; `xauusd.make_config` must stay equivalent to the inline config the backtester/bot would build (test-enforced). If you add a live gate, add it to the replay path (or `analyze_from_dataframes`) in the same commit.
+5. **ROC stays weighted** (see engine table) unless you retune composite thresholds and re-validate — removing it lost 13R.
+6. **Backtester realism knobs are results-critical**: raw-path resolution, spread charging, next-open entry, SL-first ties, close-only trail ratchet. Any "optimization" that touches these invalidates comparability with the pinned books (`backtest/*.csv`, memory notes).
+7. **Don't zero/disable indicators via config in production paths** — `disabled_indicators` exists for experiments; composite thresholds assume the full set.
+8. **The forming candle must stay dropped** and analysis must only ever see closed bars, or live signals repaint.
+9. Keep `test_all.py` green (`python test_all.py`); extend it for every new gate/knob.
+10. Legacy/no-op config fields exist (flagged above) — do not build new logic on them; delete only with test updates.