Switch active data source to Oanda v20 (practice feed)

config.json.example: data_providers.active = "oanda" (credentials stay
in env/.env or the top-level oanda block; TD and Tiingo remain
registered fallbacks). Backtest parquet cache is now keyed per provider
— Oanda and TD bars diverge (2026-07-24 study), so a feed switch must
never serve another feed's cached bars; twelve_data keeps the legacy
un-prefixed filename so pinned-book caches stay valid. 254 tests green;
live smoke-tested against the practice API (66 contract checks: all
scanned TFs, boundary/freshness guarantees, real spread $0.71,
fetch_range pagination, typed error paths, CLI + backtest end-to-end).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mc7XKYyPySVXiUpcN82QEK

.gitignore

@@ -1,6 +1,10 @@
 # Secrets
 .env
 .env.local
+config.json
+
+# Oanda daemon state
+oanda_state.json
 
 # Editor / tooling
 .claude/
@@ -24,4 +28,10 @@ Thumbs.db
 # Backtesting data (keep scripts, ignore generated files)
 backtest/cache/
 backtest/*.csv
+backtest/research/*.parquet
+backtest/research/duka_m5_boundary_spreads.csv
+backtest/trailgrid/
+backtest/trailgrid5m/
 backtest/__pycache__/
+.env.*
+logs/

CLAUDE.md

@@ -1,6 +1,6 @@
 # ForexBot2 — Codebase Reference
 
-Signal-intelligence platform for **XAU/USD (gold — live, validated)**. It fetches OHLCV from Twelve Data, scores 9 technical indicators per timeframe with regime-adaptive weights, checks multi-timeframe agreement and news/calendar risk, grades the signal (A+…F), and generates a full trade setup (entry, SL, TP1/TP2, trailing plan). Delivery: Telegram bot (auto-scanner + subscriber channel) and a CLI. A realism-focused backtester replays history through the *exact same* signal code.
+Signal-intelligence platform for **XAU/USD (gold — live, validated)**. It fetches OHLCV from the active data provider (**Oanda v20 since 2026-07-28**; Twelve Data and Tiingo stay registered as fallbacks — config.json `data_providers.active` switches), scores 9 technical indicators per timeframe with regime-adaptive weights, checks multi-timeframe agreement and news/calendar risk, grades the signal (A+…F), and generates a full trade setup (entry, SL, TP1/TP2, trailing plan). Delivery: Telegram bot (auto-scanner + subscriber channel) and a CLI. A realism-focused backtester replays history through the *exact same* signal code.
 
 This file is the authoritative developer/agent reference. The human-facing docs live in `docs/` (`user_guide.html` + `beginner_guide.html`, and their PDF builds) — keep them in sync when signal behavior changes. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
 
@@ -13,7 +13,14 @@ This file is the authoritative developer/agent reference. The human-facing docs
 | Path | Lines | Purpose |
 |------|-------|---------|
 | `config.py` | ~374 | All config dataclasses, `PAIR_INFO` registry, per-pair strategy configs. **`PlatformConfig` defaults ARE the tuned XAU config.** |
-| `data_provider.py` | ~347 | Twelve Data SDK wrapper: fetch, rate-limit (55 credits/min), in-memory cache, validation/cleaning, freshness checks |
+| `app_config.py` | ~290 | config.json loader/validation, env-precedence resolvers, repo-root `.env` loader (setdefault at import), `apply_pair_overrides` deployment overlay |
+| `config.json.example` | 24 | committed schema template; real gitignored config.json holds secrets + deployment settings |
+| `providers/base.py` | ~400 | `BaseDataProvider`: shared cache, validation/cleaning, freshness + boundary guarantees, `fetch_range` pagination, streaming stubs. Subclasses implement only `_fetch_raw` |
+| `providers/twelve_data.py` | ~240 | `TwelveDataProvider` (alias `ForexDataProvider`): TD SDK fetch, rate-limit (55 credits/min), WS quote streaming |
+| `providers/oanda.py` | ~190 | `OandaDataProvider`: v20 REST MBA candles, real bid/ask spread, date-window fetch |
+| `providers/tiingo.py` | ~200 | `TiingoDataProvider`: FX endpoint mid-price candles (no volume, no spread — see Tiingo note), named errors for every failure shape |
+| `providers/__init__.py` | ~55 | `PROVIDER_REGISTRY` + `make_data_provider` (config.json `data_providers.active` selects) |
+| `data_provider.py` / `oanda_provider.py` | ~25 | back-compat shims re-exporting from `providers/` |
 | `engine.py` | ~1173 | `TechnicalEngine`: computes all indicators on one DataFrame → `TechnicalAnalysis` (composite score, regime, key levels) |
 | `chart_patterns.py` | ~280 | Chart-pattern detectors for the XAU_CP archetype (pure functions, ATR-unit geometry, first-close-beyond triggers) |
 | `signal_aggregator.py` | ~1447 | `SignalAggregator`: MTF analysis, news integration, gates/penalties, quality grade, trade-setup generation → `SignalReport` |
@@ -22,14 +29,13 @@ This file is the authoritative developer/agent reference. The human-facing docs
 | `pair_signals/__init__.py` | 52 | `PAIR_PRODUCERS` registry + `produce_signal()` |
 | `pair_signals/xauusd.py` | 71 | XAU config factory + per-timeframe overrides (docstring = validation record) |
 | `main.py` | ~348 | CLI: `python main.py --pair XAUUSD --timeframe H1 --detail/--json` |
-| `test_all.py` | ~2100 | QA suite: **184 tests in 12 modules**, no live API needed. Must stay green. |
+| `test_all.py` | ~4100 | QA suite: **253 tests in 16 modules**, no live API needed. Must stay green. |
 | `backtest/gpu_backtest.py` | ~2231 | Bar-replay backtester (CuPy→NumPy fallback) using the real `SignalAggregator` |
 | `backtest/diagnose_filters.py` | 609 | Filter-diagnosis tooling |
 | `backtest/research/` | — | Calendar research infra (`ff_calendar_scraper.py`, `calendar_store.py`) + result CSVs |
 | `backtest/*.csv` | — | Verification trade books from past tuning campaigns (`xau_*.csv`, …). **Gitignored, local-only** — used for manual before/after comparison, not by tests |
 | `backtest/cache/` | — | Parquet OHLCV cache (raw bars) |
-| `test_ws_dual.py` | 108 | Standalone WebSocket dual-connection probe |
-| `logs/telegram.log` | — | Bot log (midnight rotation, 30 backups) |
+| `logs/telegram.log` | — | Bot log (3-hour rotation × 240 backups = 30-day retention) |
 
 ---
 
@@ -79,9 +85,8 @@ EMA 9/21/50/200 (`ema_mode`: `"4ema"` default or `"2ema"` = 50/200 only); RSI 14
 | `min_tp_cost_multiple` | 30.0 | spread gate: TP1 distance must be ≥ cost × this (= $9 on XAU). Validated: sub-$10 targets lose |
 | `min_sl_pips` | 80.0 | SL floor in pips ($8 on XAU) |
 | `tp_floor_rr` | 0.5 | TP1 floor as fraction of SL distance (M15 factory overrides to 1.0) |
-| `trail_arm_r` / `trail_dist_r` | 0.6 / 0.75 | trailing exit: once favorable excursion ≥ arm×risk, stop trails dist×risk behind best price (bar-close updates). TP1 becomes advisory. Validated on 12mo M5: +105R vs +48R static. 0 disables |
+| `trail_arm_r` / `trail_dist_r` | 0.6 / 0.75 | trailing exit: once favorable excursion ≥ arm×risk, stop trails dist×risk behind best price (bar-close updates). TP1 becomes advisory. Validated on 12mo M5: +105R vs +48R static. 0 disables. **M15 factory overrides arm to 0.70** (2026-07-21 grid: +210.7R vs +175.7R/5yr, OOS +57.4 vs +45.4; DD clause waived by owner); **M5 factory overrides dist to 0.85** (same-day grid: +231.2R vs +218.0R/5yr, OOS +14.5 vs +11.1, DD flat; dist 1.0 rejected at the OOS gate — see factory docstring) |
 | `pb_trail_arm_r` / `pb_trail_dist_r` | 0.8 / 0.75 | trailing params for XAU_PB pullback trades (tighter stops → arm later) |
-| `weights`, `min_confidence`, `cooldown_bars`, `default_risk_pct`, `default_rr_ratio`, `atr_sl_multiplier` | — | **LEGACY / UNUSED** outside tests. The engine has its own weights; backtest cooldown lives in `BacktestConfig` |
 
 ### NewsConfig
 Tracked currencies `["USD","XAU"]`; pre-event buffer 30 min, post-event 15 min (45 min after a *surprise* print); `event_risk_caps_signals=False` — **XAU default: event risk warns but does NOT cap confidence** (2026-07-11 audit: the caps suppressed trades worth +4.6..+7.2R/yr; XAU trades near USD releases outperform); `sentiment_affects_signals=False` — **headline sentiment is informational only** (warnings still emitted; the +5 alignment bonus and 10-pt `news_conflict` penalty are gated off, so live decisions are pure technical, matching backtests). Context cache TTL 600 s (must stay < pre-event buffer). 4 RSS feeds (investinglive, FXStreet, Google-News gold query, CNBC economy).
@@ -102,13 +107,13 @@ Tracked currencies `["USD","XAU"]`; pre-event buffer 30 min, post-event 15 min (
 | M5 pullback (XAU_PB) | | `m5_pullback_enabled=True`; min score 15 (lower than gate 1 BY DESIGN — mid-pullback scores weaker); ADX ≥ 20; RSI zones BUY (35,62) / SELL (38,65); retrace depth 0.8–6.0 ATR from the 40-bar swing extreme; band touch within 3 bars. Validated +38.3R/254 trades on 12mo |
 | Chart patterns (XAU_CP) | `cp_enabled=False` | `cp_*` fields: pattern list, lookback 72, swing window 3, recency 20, double tolerance 0.5 ATR, height 1.5–8 ATR, H&S head ≥0.5 ATR / shoulder tol 1.0 ATR, flag pole ≥3 ATR in ≤12 bars, flag 3–12 bars retracing 10–50%. Textbook defaults, NOT tuned. **Default OFF** — M5 factory enables BUY-only (`inv_hs`, `double_bottom`, `bull_flag`); M15/H1/H4 remain disabled (all failed OOS validation 2026-07-16). Bearish patterns (`hs_top`, `double_top`, `bear_flag`) dropped: SELL CP lost −3.21R on OOS, −4.52R on M15 5yr |
 
-Legacy fields kept for config-shape compatibility: `rsi_entry_threshold`, `min_ema200_tfs`, `allow_generic_trades`, `buy_threshold_boost`, `m5_confirmation_bars`, `h4_rsi_gate=False`, `h1_macd_gate=False`, `rsi_zone` — not wired to current gates.
-
 ### FilterConfig (`config.backtest` — despite the name it applies LIVE too)
 `session_filter=True`, window 03:00–23:00 UTC, `blocked_hours_utc` mirrors XAU's list (non-XAU pairs read this copy), `allowed_weekdays` Mon–Fri+Sun. `regime_filter=False` (optional H4-ADX≥25 gate, off).
 
 ### PlatformConfig (top level)
-`symbol="XAU/USD"`, primary TF H1, confirmations [M15, H4], `data_lookback_days=60`, `scan_pairs=[XAUUSD]` (**the single switch for what the bot scans**), `scan_timeframes=[M5,M15,H1,H4]`, `candle_buffer_seconds=10`, `ws_cooldown_seconds=300`, hardcoded `twelve_data_api_key` fallback.
+`symbol="XAU/USD"`, primary TF H1, confirmations [M15, H4], `data_lookback_days=60`, `scan_pairs=[XAUUSD]` (mirrors config.json `scanning.pairs` — see below), `scan_timeframes=[M5,M15,H1,H4]`, `ws_cooldown_seconds=300`.
+
+Per-pair base blocked hours plus `scan_pairs`/`scan_timeframes` overlay from config.json via `app_config.apply_pair_overrides`, applied in the pair factory BEFORE per-TF deltas. Credentials, alert quality, and log level never touch `PlatformConfig` — `telegram_bot.py` and `data_provider.py` read them directly from `app_config.get_app_config()` and its resolvers. `scanning.pairs` in config.json is the deployment switch for what the bot scans — `PlatformConfig.scan_pairs` just carries the resolved value.
 
 ---
 
@@ -129,7 +134,7 @@ Results land in `TechnicalAnalysis.indicators: Dict[str, IndicatorSignal]` (fiel
 | `macd` | 1.5 | Histogram = MACD line − signal line. Cross above 0 → BUY (conf 55+); accelerating (hist>prev>0) → STRONG_BUY; mirrored down; positive-but-fading / negative-but-fading → NEUTRAL (35); flat → NEUTRAL (30). Confidence scaled by |hist| vs its 20-bar mean |
 | `bollinger` | 1.5 | Position = (close−lower)/(upper−lower). >1.0 SELL (65), <0.0 BUY (65), >0.8 SELL (45), <0.2 BUY (45), else NEUTRAL (30). Squeeze = width < 0.75 × 20-bar avg width → +10 conf. Emits `bb_upper/lower/mid` |
 | `adx` | 0.5 | Wilder ADX/DI (ties zero BOTH DMs — fixes a systematic bearish nudge). ADX ≥ 25 → BUY/SELL by DI dominance, else NEUTRAL. Conf 40/65/80 by ADX band. ADX slope (5-bar diff): <25 with slope > 3.0 → "TREND EMERGING" +15 conf; also strengthening/weakening adjustments. Sets `_adx_slope` |
-| `roc` | 1.5 | 10-period %change. Positive & rising → BUY (STRONG if >2× threshold=2%); negative & falling → SELL; zero-cross → BUY/SELL (65); else NEUTRAL. **Directionally redundant with MACD/Keltner/EMA (0.8–0.94 corr) but its weight is load-bearing: zeroing it backtested 13R WORSE — composite thresholds are calibrated with it. Do not remove without retuning thresholds** |
+| `roc` | 1.5 | 10-period %change. Positive & rising → BUY (STRONG if >2× threshold=2%); negative & falling → SELL (zero-crosses land in these branches by definition — a separate cross branch was provably dead and removed 2026-07-23); else NEUTRAL. **Directionally redundant with MACD/Keltner/EMA (0.8–0.94 corr) but its weight is load-bearing: zeroing it backtested 13R WORSE — composite thresholds are calibrated with it. Do not remove without retuning thresholds** |
 | `keltner` | 1.5 | EMA20 mid ± 1.5×ATR(10). TTM squeeze = BB fully inside KC. Squeeze released (was→not) + MACD hist sign → STRONG_BUY/STRONG_SELL (80); squeeze active → NEUTRAL (60); close above/below channel → BUY/SELL continuation (65); inside → NEUTRAL (30). Emits `kc_upper/lower/mid` |
 | `divergence` | 1.5 | RSI + MACD-hist vs price over 40 bars. Swing points (window 3) must align within ±2 bars to pair up; last two pairs: price LL + indicator HL = bullish, price HH + indicator LH = bearish. Any found → BUY/SELL/NEUTRAL-if-both, conf 70, value = count |
 | `candlestick` | 1.0 | Last bar: Doji (body<10% range), Hammer/Hanging Man (lower wick>2×body), Shooting Star/Inverted Hammer, Engulfing (vs prev bar), Marubozu (body>85%). Majority vote → signal, conf 50; +20 if pattern sits within 0.5 ATR of a swing S/R level |
@@ -184,7 +189,7 @@ Two entry points:
 6. **`_compute_final_signal`** (bonuses → event cap → penalties → thresholds → grade):
    - Bonuses: MTF agreement → +10 conf and `score = 0.6×primary + 0.4×mean(all TF scores)`; news sentiment aligned (|sent|>0.2) → +5 (gated by `news.sentiment_affects_signals`, False by default); in a *directional* trend, RSI in pullback zone → +5, BB position favorable → +5 (mixed `trending` regime gets NO directional bonus).
    - Event risk `danger`: warning always; confidence hard-capped at 30 ONLY if `news.event_risk_caps_signals` (False by default — XAU-tuned).
-   - **Penalty system (diminishing, replaces old additive stacking):** collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: `mtf_disagreement` 15; `news_conflict` 10 (gated by `sentiment_affects_signals`, off by default — warning still emitted); `event_caution` 15 (config-gated); `counter_trend` 15; `extreme_vol` 10; `ema200_near` 5 (price within 0.5% of EMA200 — env `FB2_EMA_LO` overrides); `ema200_far` 5 (>2.5%); `exhaustion` 8 (price >3.5 ATR beyond nearest swing); `no_pullback` 8 (>2.5 ATR from EMA21 in trend) / `weak_pullback` 5 (>2.0); `rsi_extended` 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed `trending`); `bb_overextended` 8 (BB pos >0.85 BUY / <0.15 SELL).
+   - **Penalty system (diminishing, replaces old additive stacking):** collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: `mtf_disagreement` 15; `news_conflict` 10 (gated by `sentiment_affects_signals`, off by default — warning still emitted); `event_caution` 15 (config-gated); `counter_trend` 15; `extreme_vol` 10; `ema200_near` 5 (price within 0.5% of EMA200 — env `FB2_EMA_LO` overrides); `ema200_far` 5 (>2.5%); `exhaustion` 8 (price >3.5 ATR beyond the FARTHEST of the 3 tracked swing levels — fires earlier than a nearest-swing rule would; calibrated as-is in every validated book); `no_pullback` 8 (>2.5 ATR from EMA21 in trend) / `weak_pullback` 5 (>2.0); `rsi_extended` 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed `trending`); `bb_overextended` 8 (BB pos >0.85 BUY / <0.15 SELL).
    - **Exhaustion hard block**: ≥2 exhaustion-tagged penalties ({exhaustion, no_pullback, rsi_extended, bb_overextended} → `report.exhaustion_count`) AND momentum fading (ROC sign against score, or MACD signal against score) → confidence = 0.
    - Confidence clamped [0, 95]. **Final signal**: score ≥35 & conf ≥55 STRONG_BUY; ≥25 & ≥50 BUY; mirrored sells; else NEUTRAL.
    - **Quality grade**: |score|≥40 & conf≥65 & MTF → **A+**; ≥33 & ≥55 & MTF → **A**; ≥28 & ≥50 → **B**; ≥25 & ≥45 → **C**; ≥15 → **D**; else **F**.
@@ -217,7 +222,7 @@ Two entry points:
 
 **D. `_try_chart_pattern` (XAU_CP)**: detection in `chart_patterns.py` — 6 classical patterns (double_bottom/double_top, inv_hs/hs_top, bull_flag/bear_flag), swing-point based (strict local extrema, window `cp_swing_window`=3), all thresholds in ATR units, scanned over the last `cp_lookback`=72 closed bars. Trigger = FIRST close beyond the neckline/boundary since the pattern formed (one breakout → one signal, no repaint); reversal patterns are invalidated by a new extreme beyond the pattern. No composite-score gate BY DESIGN. Gates: MTF non-opposition (6f mirror); SELL patterns respect `sell_vol_regimes`, `sell_blocked_hours_utc` (bar-close hour), M5 RSI floor; M5 blocks `m5_blocked_vol_regimes` both ways; ATR ≥ $1. Geometry: SL = pattern anchor ∓ 0.5 ATR (min $8); TP1 = measured move (pattern height), TP2 = 1.5×height; RR ≥ 0.5; spread gate $9. Sets `strategy="XAU_CP"` (global trailing params), grade min B, marker `"XAU CHART PATTERN: …"` at reasons[0]. Funnel in `SignalAggregator.cp_funnel`.
 
-`TradeSetup` fields: pair, direction, entry, SL, TP1, TP2, RR, risk/reward pips, pip_value, `strategy` ("GENERIC", "XAU_PB", or "XAU_CP" — selects trailing params; XAU_CP uses the global pair).
+`TradeSetup` fields: pair, direction, entry, SL, TP1, TP2, RR, risk/reward pips, pip_value, `strategy` ("GENERIC", "XAU_PB", or "XAU_CP" — selects trailing params; XAU_CP uses the global pair), `trail_arm_r`/`trail_dist_r` (stamped from the PRODUCING config at generation so per-TF factory overrides reach formatters; `None` = legacy unstamped → display falls back to platform defaults).
 
 ---
 
@@ -230,16 +235,25 @@ Two entry points:
 
 ---
 
-## data_provider.py — ForexDataProvider
+## providers/ — data-provider abstraction (refactored 2026-07-25)
 
-- Twelve Data SDK (`TDClient.time_series(...).as_pandas()`), symbols normalized to `"XAU/USD"` form; renames lowercase columns → capitalized; sorts oldest-first; localizes UTC.
-- **Rate limit**: 55 credits/min sliding window (`_throttle`), blocks with sleep when exhausted.
-- **In-memory cache**: key `{symbol}_{tf}`, TTL = candle duration (M5 300 s … H4 14400 s); a cached frame is evicted and refetched if it fails the freshness check **or the boundary check** (`_has_latest_closed_bar`: the frame must contain the most recent fully closed bar, weekend-exempt — a wall-clock TTL alone would serve mid-candle snapshots at the next boundary and scans would run one bar behind). Cache and throttle are lock-protected (one provider is shared across the bot's thread pool).
-- **`_drop_forming_candle`**: Twelve Data includes the current partial candle; the scanner fires seconds after boundary, so the last bar is dropped if its close time is in the future. Without this every indicator would read a seconds-old stub and signals would repaint.
-- **Freshness** (`_is_data_fresh`): last bar older than 3× candle duration → stale (warns; live path suppresses signal if the PRIMARY TF is stale). Weekends exempt (Fri 22:00→Sun 22:00 UTC).
-- **`_validate_and_clean`**: drop all-NaN rows → ffill (limit 3) → drop NaN closes → sort + dedupe (keep last) → drop future timestamps (>now+5 min) → repair OHLC inconsistencies (High=max, Low=min of O/C/existing) → remove >10σ return outliers (spike bar only — the opposite-direction snap-back bar right after it is kept, else every spike punched a two-bar hole) → warn if < ~210 bars.
-- `outputsize = max(300, min(days×bars_per_day+20, 5000))`; TD caps at 5000/call (the backtester paginates by date).
-- `get_latest_price` / `get_spread_estimate` (median M5 High−Low) helpers.
+One class per provider, all subclassing **`providers/base.py BaseDataProvider`**, which owns every behavior the signal pipeline depends on — defined ONCE (a test enforces that neither subclass overrides shared methods):
+
+- **In-memory cache**: key `{symbol}_{tf}`, TTL = candle duration (M5 300 s … H4 14400 s); a cached frame is evicted and refetched if it fails the freshness check **or the boundary check** (`_has_latest_closed_bar`: the frame must contain the most recent fully closed bar, weekend-exempt — a wall-clock TTL alone would serve mid-candle snapshots at the next boundary and scans would run one bar behind). Fresh fetches enforce the same boundary guarantee: a response missing the just-closed bar (feed latency at the boundary) gets one grace retry after `FRESH_FETCH_RETRY_SECONDS` (3 s), then proceeds with a loud warning. Cache is lock-protected (one provider is shared across the bot's thread pool).
+- **`_drop_forming_candle`**: some feeds include the current partial candle; the scanner fires seconds after boundary, so the last bar is dropped if its close time is in the future. Without this every indicator would read a seconds-old stub and signals would repaint. (No-op on feeds that only return complete candles, e.g. Oanda.)
+- **Freshness** (`is_data_fresh`, public): last bar older than 3× candle duration → stale (warns; live path suppresses signal if the PRIMARY TF is stale). Weekends exempt (Fri 22:00→Sun 22:00 UTC).
+- **`_validate_and_clean`** (public wrapper `validate_and_clean`): drop all-NaN rows → ffill (limit 3) → drop NaN closes → sort + dedupe (keep last) → drop future timestamps (>now+5 min) → repair OHLC inconsistencies (High=max, Low=min of O/C/existing) → remove >10σ return outliers (spike bar only — the opposite-direction snap-back bar right after it is kept, else every spike punched a two-bar hole) → warn if < ~210 bars.
+- **`fetch_range(symbol, tf, start, end)`**: paginated RAW date-range fetch under `MAX_BARS_PER_REQUEST` (5000) — the backtester's `fetch_history` calls this, so ANY registered provider can feed the backtester.
+- `outputsize = max(300, min(days×bars_per_day+20, 5000))`; `CANDLE_SECONDS`/`BARS_PER_DAY` tables live on the base class (the aggregator's bar-close clock reads `BaseDataProvider.CANDLE_SECONDS`).
+- **Streaming capability (optional)**: `supports_streaming` flag + async generator `iter_ticks(symbols, on_connect)` yielding `(canonical_symbol, price)` for one connection's lifetime; the caller owns reconnect/backoff. Errors: each provider raises its own `DATA_ERROR`, always a subclass of `DataIntegrityError` — catch the base type provider-agnostically.
+
+**A subclass implements only `_fetch_raw(symbol, tf, outputsize, start=None, end=None)`** (normalized frame: capitalized OHLCV, UTC tz-aware index named "Datetime", oldest-first) plus symbol/interval maps and optional capabilities. To add a provider: one module in `providers/`, one `PROVIDER_REGISTRY` entry in `providers/__init__.py`, one config.json `data_providers` block. The live feed is selected by config.json `data_providers.active` (registration alone changes nothing).
+
+- **`providers/twelve_data.py` `TwelveDataProvider`** (alias `ForexDataProvider`): TD SDK (`TDClient.time_series(...).as_pandas()`), symbols normalized to `"XAU/USD"` form; lowercase→capitalized columns, newest-first→oldest-first; **rate limit** 55 credits/min sliding window (`_throttle`, lock-protected, blocks with sleep). Implements streaming (`ws.twelvedata.com` quotes WS, 10 s heartbeat, `_parse_ws_price`).
+- **`providers/oanda.py` `OandaDataProvider`**: v20 REST MBA candles (complete-only), real bid/ask spread via `get_spread` override; credentials default to the config.json `oanda` block so the registry can build it config-only (explicit `api_token`/`base_url` args still accepted — the daemon passes them). Date-window fetches send `from`/`to` and omit `count` (v20 rejects the combination).
+- **`providers/tiingo.py` `TiingoDataProvider`**: `/tiingo/fx/{ticker}/prices` resampled mid-price candles; no volume (filled 0.0), W1 unmapped (raises). Tiingo takes date windows, not bar counts — live fetches derive `startDate` from the requested count × candle duration × 1.6 weekend pad. Auth in the header (key never in URLs/logs). Every failure shape raises `TiingoDataError` with a named cause (429 quota, 401/403 auth, empty response, malformed candle, missing key). **Candles only** — `get_spread` stays on the base default; the free quote endpoint's indicative spread must not be re-added (see Tiingo spread-monitor removal note).
+- **`data_provider.py` / `oanda_provider.py` are back-compat shims** re-exporting from `providers/`; `ForexDataProvider` is the historical name of the TD provider.
+- **No silent failures**: provider errors all subclass `DataIntegrityError` and carry a named cause; the aggregator logs per-TF fetch failures and degrades the report (missing primary → NEUTRAL/F "data unavailable"); `make_data_provider` raises on an unknown `active` name; keyless providers warn at construction and raise at fetch.
 
 ---
 
@@ -247,30 +261,36 @@ Two entry points:
 
 `PAIR_PRODUCERS: Dict[symbol, factory]`; factory signature `make_config(primary_tf, confirmation_tfs=None) -> PlatformConfig`. `produce_signal(symbol, tf, data_provider)` builds config → aggregator → `analyze_pair`. Registered: XAU/USD.
 
+Inside `make_config`, call order is: `PlatformConfig()` defaults → `app_config.apply_pair_overrides(config)` (stamps config.json's non-credential deployment settings onto the config: base `xau_strategy`/`backtest` `blocked_hours_utc`, `scan_pairs`, `scan_timeframes`) → the validated per-TF deltas below. The overlay runs first so e.g. the M5 hour-9 append lands on top of the JSON base, not the other way around.
+
 **A bare `PlatformConfig()` silently runs any pair with XAU defaults — always go through the factory** (the bot, CLI, and backtester all do).
 
 Default confirmation map (same in the factory, the bot, and backtest usage): M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1]. (Exception: `main.py`'s CLI map uses M5→[M1,M15] — a CLI-only quirk.)
 
 **xauusd.py**: `PlatformConfig()` defaults + per-TF overrides (each validated; full evidence in the module docstring):
 - M5: append hour 9 UTC to blocked hours (6-yr replay: +207.1R vs +178.3R; M15-and-up keep hour 9 — it's positive there). Enable BUY-only chart patterns (`cp_enabled=True`, `cp_patterns=["inv_hs","double_bottom","bull_flag"]`): 12mo +41.8R PF 1.60 (train/val both positive), 5yr +65.2R PF 1.48; bearish CP dropped (OOS −3.21R). M15/H1 CP remain disabled (all failed OOS).
+- M5: `signals.trail_dist_r = 0.85` (wider trail leash, arm stays 0.60; validated 2026-07-21 5-yr grid: +231.2R PF 1.30 vs +218.0R PF 1.29, halves and OOS better, DD flat; dist 1.0 had the best in-sample total but FAILED the OOS gate. XAU_PB keeps pb_trail 0.8/0.75; opposite finding vs M15's later-arm — per-TF divergence again).
 - M15: `signals.tp_floor_rr = 1.0` (1R minimum targets: +116.6R vs +105.4R, better in train/val and 2024 OOS; **hurts M5** — per-TF only) and `xau_strategy.blocked_regimes = ["trending"]` (mixed regime block: +65.7R vs +59.0R). A transitioning-SELL block was tested and REJECTED (bull-market artifact — failed 2024 OOS).
 - M15: `xau_strategy.sell_blocked_hours_utc = [15, 17]` (SELL-only hour block, validated 2026-07-16 on the 5-yr replay: +176.7R PF 1.31 vs +144.9R PF 1.22, better in all 6 years, both 12-mo halves, and 2024-06→2025-03 OOS; M5 cross-TF pocket confirms; BUYs at those hours are +34.3R so `blocked_hours_utc` would be wrong).
+- M15: `signals.trail_arm_r = 0.70` (trail arms later; validated 2026-07-21, 5-yr grid arm 0.3-0.9 × dist 0.5-1.0: +210.7R PF 1.35 WR 57.8% vs +175.7R PF 1.30 WR 53.9% at the 0.6 default; both halves and OOS better; 5yr max-DD clause 33.9R vs 29.1R waived by owner. Earlier-arm cells uniformly worse. M5 keeps 0.6).
 - The factory must stay equivalent to inline `PlatformConfig(symbol=XAUUSD, …)` + overrides — **enforced by a test in test_all.py**.
 
 ---
 
 ## telegram_bot.py — the bot
 
-Run: `python telegram_bot.py` (env `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`; hardcoded fallbacks exist in the file).
+Run: `python telegram_bot.py` (credentials resolve env `TELEGRAM_TOKEN`/`TELEGRAM_CHAT_ID`/`TELEGRAM_CHANNEL_ID` → config.json `telegram.*` → refuses to boot; see Environment variables).
 
 - **Owner lock**: every command/button is wrapped in `_owner_only` — only `TELEGRAM_CHAT_ID` gets responses; others get a refusal. The bot is the owner's control panel; the product is the channel.
 - **Subscriber channel** (`TELEGRAM_CHANNEL_ID`, `-100…`; `""` disables): auto-scan signal alerts + breach alerts also post there (bot needs channel-admin). Monetization = Telegram Stars subscription invite links; Telegram handles membership, the bot has zero payment code. The channel feed keeps running even when the owner toggles DM auto-scan off (`_feed_active()` = DM chats OR channel).
 - **Auto-scan**: boot-starts if `TELEGRAM_CHAT_ID` set (initial M15 scan populates the level cache first). Candle-boundary loops for M5/M15/H1/H4, firing at boundary + 10 s. Each scan runs all `scan_pairs` concurrently, updates the level cache, then broadcasts.
-- **Alert gating**: quality threshold (`_alert_min_quality`, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction, entry bucket) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle, but a setup that moved ≥ half its risk distance (new pullback/swing anchor) re-alerts. The fingerprint is committed only **after** a message reaches ≥1 target (`_commit_alert`), so a failed Telegram send doesn't consume the alert.
-- **WebSocket monitor**: subscribes every `scan_pairs` symbol to TD quotes WS; 10 s heartbeat; exponential reconnect (≤60 s). On tick crossing any cached key level (`swing_high/low_1..3`, `ema_trend`, `bb_upper/lower`, `kc_upper/lower`; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown.
+- **Alert gating**: quality threshold (`_alert_min_quality`, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle, but a setup whose entry moved ≥ half the last-ALERTED risk distance (new pullback/swing anchor; compared against the stored last-alert entry/risk, a stable reference immune to per-candle ATR drift) re-alerts. The fingerprint is committed only **after** a message reaches ≥1 target (`_commit_alert`), so a failed Telegram send doesn't consume the alert.
+- **WebSocket monitor**: the provider owns the connection/protocol (`BaseDataProvider.iter_ticks` streaming capability — TD implements it; heartbeat + message parsing live in `providers/twelve_data.py`); the bot owns exponential reconnect (≤60 s) + breach dispatch, and disables the monitor with a log line if the active provider has `supports_streaming=False`. On tick crossing any cached key level (`swing_high/low_1..3`, `ema_trend`, `bb_upper/lower`, `kc_upper/lower`; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown.
 - **Commands**: `/start` (menu), `/scan` (buttons: per pair — M5/M15/H1/H4 + Scan All TFs), `/autoscan` (toggle DM copy), `/status` (loops, WS, live prices, level cache, cooldowns), `/cache` (age per pair/TF), `/clearcache`, `/help`.
-- Signal messages include the full indicator breakdown, warnings, trade setup, **trailing-stop management plan** (arm price + trail distance, from Signal/pb params by `strategy`), and a risk disclaimer on every outbound message.
-- Blocking scan work runs in `ThreadPoolExecutor(max_workers=4)`; one shared `ForexDataProvider` across all scans (cache reuse). Logs to `logs/telegram.log`.
+- **Tiingo spread monitor: REMOVED 2026-07-23** (shipped 2026-07-22, removed next day). Tiingo's free `/tiingo/fx/top` publishes a two-tier *indicative* spread for gold — exactly $0.10 in liquid hours and exactly $0.70 in the 21:00–00:00 UTC rollover window / news moments (verified: 273-sample production log + live probes; only those two values ever occur). Quotes are mid ± a posted increment, not executable depth, so the feature could never validate the $0.30 backtest cost assumption. Do not re-add a spread display from Tiingo's free FX feed.
+- Signal messages (redesigned 2026-07-24, `format_signal_message`): verdict header first, then trade setup, then a condensed one-line indicator summary (`EMA↑ RSI→ …`, `_indicator_summary`) + warnings. The setup block includes an **informational 100-pip TP line** (`_INFO_TP_PIPS`, display-only — entry ± 100 pips, ≈R vs stop; NOT part of the backtested exit model) and the **trailing-stop management plan** (arm price + trail distance, read from the `trail_arm_r`/`trail_dist_r` stamped on the setup by the producing per-TF config). Risk disclaimer on every outbound message.
+- Blocking scan work runs in `ThreadPoolExecutor(max_workers=4)`; one shared `ForexDataProvider` across all scans (cache reuse).
+- **Logging (hardened 2026-07-22)**: root-logger handlers (`force=True`) → console + `logs/telegram.log` (3-hour rotation, 240 backups = 30 days), full-date timestamps; every module logger (data_provider, news_engine, websockets, telegram.ext) lands in the same file. Failure surfaces all covered: candle loops catch-and-continue per cycle (one bad scan can't kill a loop), breach tasks log exceptions via done-callback, PTB `add_error_handler` registered, and `sys.excepthook`/`threading.excepthook` (installed in `main()`) route uncaught crashes to the log. WS monitor keeps its exponential-backoff reconnect + `logger.exception`.
 
 ---
 
@@ -291,6 +311,7 @@ Replays history bar-by-bar through the **real `SignalAggregator.analyze_from_dat
 - Same-bar SL+TP tie → SL first (conservative). Gap through SL/TP → filled at the bar open.
 - **Trailing exits** (default on, from `signals.trail_arm_r/trail_dist_r`; XAU_PB trades carry per-trade 0.8/0.75 overrides): stop ratchets at bar close only, so a new stop can only fill from the next bar (conservative). Exit reasons: SL / TRAIL / TP1 (static mode) / TIMEOUT / NO_DATA.
 - Session + weekend + weekday filters evaluate the **decision time = bar close** (`ts + primary_duration`) — exactly matching the live bot's wall-clock check.
+- The optional H4-ADX regime filter (`config.backtest.regime_filter`, default off) is applied in the replay by calling the live `_apply_regime_filter_live` method itself under the same guard, and the replay loads/slices the regime-filter TF whenever the gate is on (like live `analyze_pair`) — previously M5/M15 primaries silently no-opped the filter because H4 wasn't in their confirm maps. Live and backtest cannot diverge if it's enabled (2026-07-21 parity fix; the unregistered-pair config fallback also applies the config.json overlay now).
 - Confirmation TFs are sliced to bars **fully closed** by decision time (no look-ahead).
 
 **Performance:** `precompute_indicators` builds every indicator series once per TF (CuPy on GPU or NumPy — check the log line `Backend: CuPy GPU` vs `NumPy CPU fallback`; CuPy needs the `nvidia-*-cu12` pip wheels or it silently falls back). `GPUTechnicalEngine` subclasses the live engine and overrides only the `_compute_*` methods with cache lookups — outputs are byte-identical to the live engine (regression-tested). Bar replay: pure per-bar evaluation fans out over forked worker processes (`--workers`, 0 = auto: cores−2 capped 12; <5000 bars forces 1); the entry **cooldown (5 bars) is the only sequential rule**, applied in an ordered pass afterwards — parallel and sequential books are identical. ~300 bars/s/core, ~1700 bars/s at 10 workers.
@@ -308,24 +329,27 @@ Report: summary metrics (total R, PF, Sharpe on R, max DD, MFE/MAE), breakdowns
 
 **Comparability note (2026-07-16):** the parity/realism fixes of that date (EMA-ribbon port to the GPU engine, exact-pandas Bollinger precompute, raw-path entry re-anchoring, per-TF D1 warmup, live confirm-map CLI default, outlier snap-back retention) all shift books slightly vs earlier runs. Books produced before these fixes are not directly comparable — re-run the baseline before any before/after tuning comparison.
 
-Data: paginated TD fetches (5000-bar chunks) cached as parquet in `backtest/cache/` keyed by symbol/TF/window; warmup pad ≥60 days before the window for EMA200, extended per-TF so slow confirmation TFs get ≥210 bars at window start (D1 gets ~299 days — 62 days gave only ~61 bars: no EMA200, unlike live's 300-bar outputsize floor). `--confirm-tfs` defaults to the pair factory's live confirmation map (pass it explicitly to override); a hardcoded default here previously made the standard invocation backtest a different MTF setup than the bot scans.
+Data: paginated fetches via `provider.fetch_range` (5000-bar chunks; works with any registered provider) cached as parquet in `backtest/cache/` keyed by **provider**/symbol/TF/window (bars differ between feeds — Oanda/TD divergence study; `twelve_data` keeps the legacy un-prefixed filename so pinned-book caches stay valid); warmup pad ≥60 days before the window for EMA200, extended per-TF so slow confirmation TFs get ≥210 bars at window start (D1 gets ~299 days — 62 days gave only ~61 bars: no EMA200, unlike live's 300-bar outputsize floor). `--confirm-tfs` defaults to the pair factory's live confirmation map (pass it explicitly to override); a hardcoded default here previously made the standard invocation backtest a different MTF setup than the bot scans.
 
 ---
 
 ## test_all.py — QA suite
 
-`python test_all.py` — **184 tests, 12 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions, chart patterns). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), and a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11). **Any signal-logic change must keep this green and should add tests.**
+`python test_all.py` — **253 tests, 16 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions, chart patterns, app_config, .env loader + bot resilience, Oanda provider/executor/daemon, provider abstraction). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11), config.json loading/precedence/validation coverage (Module 13), and `.env` loader + bot-resilience coverage (Module 14). **Any signal-logic change must keep this green and should add tests.**
 
 ---
 
 ## Environment variables
 
+Precedence for every credential/deployment setting: **env var → config.json → none**. A repo-root `.env` file (gitignored) is loaded at `app_config` import via `os.environ.setdefault` — it behaves like real env vars except a real shell export always wins. Hardcoded fallbacks (previously in `config.py` / `telegram_bot.py`) were **REMOVED 2026-07-21**; the bot now refuses to boot (`telegram_bot.main`) and data fetches fail with a named error (`data_provider`) if a secret resolves to neither source. See `app_config.py` and `config.json.example`.
+
 | Variable | Purpose | Default |
 |---|---|---|
-| `TWELVE_DATA_API_KEY` | market data key | hardcoded fallback in `config.py` |
-| `TELEGRAM_TOKEN` | bot token | hardcoded fallback in `telegram_bot.py` |
-| `TELEGRAM_CHAT_ID` | owner chat id (boot auto-scan + owner lock) | hardcoded fallback |
-| `TELEGRAM_CHANNEL_ID` | subscriber channel (`-100…`); `""` disables the feed | hardcoded fallback |
+| `TWELVE_DATA_API_KEY` | market data key (twelve_data only) | none — falls through to config.json `data_providers.twelve_data.api_key`, then fails fast |
+| `TIINGO_API_KEY` | market data key (tiingo only) | none — falls through to config.json `data_providers.tiingo.api_key`, then fails fast |
+| `TELEGRAM_TOKEN` | bot token | none — falls through to config.json `telegram.token`, then refuses to boot |
+| `TELEGRAM_CHAT_ID` | owner chat id (boot auto-scan + owner lock) | none — falls through to config.json `telegram.chat_id`, then refuses to boot |
+| `TELEGRAM_CHANNEL_ID` | subscriber channel (`-100…`); `""` disables the feed | none — falls through to config.json `telegram.channel_id` |
 | `FB2_EMA_LO` | EMA200 "near" penalty threshold in % | `0.5` |
 
 Dependencies (`requirements.txt`): numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets. Python 3.12. A `venv/` exists in-repo.
@@ -354,4 +378,4 @@ Dependencies (`requirements.txt`): numpy, pandas, requests, twelvedata, python-t
 7. **Don't zero/disable indicators via config in production paths** — `disabled_indicators` exists for experiments; composite thresholds assume the full set.
 8. **The forming candle must stay dropped** and analysis must only ever see closed bars, or live signals repaint.
 9. Keep `test_all.py` green (`python test_all.py`); extend it for every new gate/knob.
-10. Legacy/no-op config fields exist (flagged above) — do not build new logic on them; delete only with test updates.
+10. Dead/legacy config fields were removed 2026-07-21 (`MarketSession`, `is_jpy_pair`, `candle_buffer_seconds`, the unused `SignalConfig`/`XauStrategyConfig` legacy knobs). Do not add config fields that nothing reads; delete dead fields only with test updates.

OANDA_MIGRATION_ROADMAP.md

@@ -0,0 +1,85 @@
+# Full Migration to Oanda v20 as Market-Data Source — Roadmap
+
+**Status: reference roadmap, not scheduled for implementation.** Owner decision 2026-07-24: document only; when executed, Twelve Data stays as a **permanent fallback** (no removal phase).
+
+## Context
+
+The `oanda_demo` branch already executes trades on an Oanda practice account, but the signal feed (Telegram bot, CLI, backtester) still runs on Twelve Data. Oanda's API is free with no metering (120 req/s vs TD's 55 credits/min), serves real bid/ask spreads, and matches the actual trading calendar. The 2026-07-24 divergence study (`backtest/research/oanda_td_divergence.py`) proved the switch is **not plumbing-only**: steady-state bars differ by median $0.69 (11% of M15 ATR), bar direction agrees only 89.9%, TD prints ~4,000 phantom Saturday bars/yr + 21:00 UTC rollover bars that contaminate Monday indicator windows, and Oanda's real spread (~$0.72 liquid / $0.84 rollover) is ~2.4× the $0.30 the validated books assume. So the migration = plumbing (Phases 1–3, inert) + shadow evidence (4) + book re-validation with owner gates (5) + reversible cutover (6). CLAUDE.md rules 1 (no tuned-value change without re-validation) and 4 (live == backtest parity) govern throughout.
+
+**Master sequencing rule:** Phases 1–3 land inert behind `data_providers.active == "twelve_data"`. Nothing changes live behavior until the Phase 6 config flip, which stays reversible forever (TD is a permanent fallback).
+
+## Phase 1 — Provider plumbing (inert)
+
+Files: `oanda_provider.py`, `data_provider.py`, `config.py`, `config.json.example`.
+
+1. **Constructor self-resolution**: `OandaDataProvider.__init__(config, api_token=None, base_url=None)` — defaults resolve via existing `app_config.oanda_api_token()` / `oanda_base_url()`; explicit args keep `oanda_daemon.py:180` working. Fail lazily at first fetch (mirror ForexDataProvider's `_td is None` pattern) so credential-less construction doesn't break the hermetic test header.
+2. **Registry**: add `"oanda": OandaDataProvider` to `PROVIDER_REGISTRY` (`data_provider.py:392`); import at bottom of `data_provider.py` (one-direction edge, no cycle — `oanda_provider` must never import from `data_provider`). Add `PROVIDER_NAME` class attr to both providers. Cutover config needs an `"oanda": {}` block under `data_providers` (creds stay in the existing top-level `oanda` section / env).
+3. **Dedupe shared constants**: move `CANDLE_SECONDS` / `BARS_PER_DAY` to `config.py`, alias as class attrs in both providers — all existing call sites (`signal_aggregator.py:18,217,640,942`, `telegram_bot.py:1062`, `gpu_backtest.py:975,1706`, tests) keep working unchanged. Also move `_is_forex_weekend` to `config.py` eventually (currently duplicated + sync-tested).
+4. **`_fetch_window(symbol, tf, start, end)`** on both providers — the backtester's common interface. Contract: raw closed bars labeled in [start, end], oldest-first, capitalized OHLCV, UTC index; empty DataFrame (not exception) for empty windows; caller keeps windows ≤ ~4500 bars. TD: thin wrapper over existing `_fetch_twelve_data(start_date=, end_date=)` (keep its exception behavior byte-identical). Oanda: new `from`/`to` RFC3339 request (`price=MBA, dailyAlignment=0, alignmentTimezone=Etc/UTC`); refactor `_fetch_oanda_candles`'s parsing body (complete-filter, mid extraction, spread cache) into a shared `_parse_candles()`. The from+count quirk (Oanda's `count` includes the boundary bar; see `backtest/research/fetch_oanda_m15.py`) does not apply to from/to windows. Add a no-op-by-construction `_drop_forming_candle` to OandaDataProvider (`gpu_backtest.py:1025` calls it unconditionally). No throttle needed for Oanda.
+5. **Tests (extend Module 15)**: registry selection; constructor self-resolution + explicit-args-win; `_fetch_window` both providers (mock `_session.get` / `_fetch_twelve_data`); CANDLE_SECONDS/BARS_PER_DAY cross-provider sync; provider-contract hasattr test covering everything callers reach for (`fetch_ohlcv, get_pair_info, clear_cache, _is_data_fresh, _has_latest_closed_bar, _validate_and_clean, _drop_forming_candle, _fetch_window, CANDLE_SECONDS, BARS_PER_DAY, _cache, _cache_timestamps, PROVIDER_NAME`); port Module 11's two cache/boundary tests (`test_all.py:2278,2313`) to Oanda variants patching `_fetch_oanda_candles`. Tests that swap `set_app_config` must restore `_TEST_APP` in `finally`.
+
+## Phase 2 — Backtester provider-agnosticism
+
+File: `backtest/gpu_backtest.py` (`fetch_history`, lines 943–1036).
+
+1. Line 984: `provider._fetch_twelve_data(...)` → `provider._fetch_window(...)`. Keep the backward-stepping chunk loop, `empty_streak` bail, cursor stepping byte-identical (rule 6 — this loop produced every pinned cache).
+2. Lines 975/1706: `ForexDataProvider.BARS_PER_DAY` → `provider.BARS_PER_DAY`.
+3. **Provider-namespaced cache keys** in `_cache_path` (line 938): TD keeps legacy names (pinned parquets stay loadable, needed for A/B), Oanda gets `oanda_` prefix (`oanda_XAU_USD_15min_....parquet`) — no collisions. Comment the asymmetry.
+4. `clean=True/False` raw-path semantics untouched.
+5. **Verification before merge**: one M15 12-month TD replay pre/post commit — trade books must be identical. **Alignment audit**: fetch one month of Oanda H4 + D1 via `_fetch_window`, assert label sets match TD's cached labels (H4 at 00/04/08 UTC, D1 at 00:00 UTC) — a mismatch would silently shift every MTF confirmation.
+6. Tests: `fetch_history` dispatch with a stub provider (concat/dedupe/trim, clean=False skips cleaning); `_cache_path` namespacing regression.
+
+## Phase 3 — Telegram bot: TD WebSocket → Oanda pricing stream
+
+File: `telegram_bot.py`; new resolver in `app_config.py`.
+
+1. New `oanda_stream_url()` resolver: env `OANDA_STREAM_URL` → config.json `oanda.stream_url` (add to `_KNOWN_OANDA`) → derived `oanda_base_url().replace("api-", "stream-", 1)`.
+2. Pure parser `_parse_stream_line(line) -> Optional[(pair_symbol, mid_price)]`: type==PRICE → mid from closeoutBid/Ask (fallback bids[0]/asks[0]); HEARTBEAT/garbage → None. `_STREAM_TO_PAIR` from `OandaDataProvider.OANDA_INSTRUMENT` (replaces TD-display-keyed `_WS_TO_PAIR`).
+3. Reader thread (no new deps): `requests.get(.../v3/accounts/{id}/pricing/stream?instruments=..., stream=True, timeout=(10,30))`, iterate `iter_lines(chunk_size=1)` — **chunk_size=1 is load-bearing** (urllib3 buffering vs 5s server heartbeats); push lines to asyncio queue via `loop.call_soon_threadsafe`; interrupt a blocked read with `resp.close()` from the async side.
+4. Async monitor `_oanda_stream_monitor`: dedicated thread (NOT the 4-worker scan executor), `await asyncio.wait_for(queue.get(), timeout=15)` as staleness watchdog (3× server heartbeat) → reconnect with existing exponential-backoff pattern. PRICE lines feed the identical existing body: `_check_level_breach` → `_handle_breach` → M15 rescan, 5-min cooldown. No client heartbeat.
+5. **Provider-gated selection** in `_start_scheduler` (line 889): `active_provider == "oanda"` → stream monitor, else existing `_ws_monitor`. This is the rollback hinge and stays permanently (TD fallback). `_stop_scheduler` must stop the reader thread. Generalize `/status` labels (`_status_text` iterates `SCAN_PAIRS`, not `_WS_TO_PAIR`).
+6. Tests: `_parse_stream_line` variants (PRICE/closeout, PRICE/arrays, HEARTBEAT, malformed, unknown instrument); `_STREAM_TO_PAIR` round-trip; monitor selection under both `active` values; watchdog-reconnect (structure one connection lifecycle as a testable inner coroutine).
+
+## Phase 4 — Shadow / parity evidence (3–4 weeks, no hot-path code)
+
+The daemon already runs the full signal stack on Oanda bars. Formalize evidence before any cutover:
+
+1. **Code parity (exact)**: replay the daemon's shadow window through the backtester on Oanda data (Phase 2 enables this); diff against daemon-logged decisions per (tf, bar_close_ts) — signal, quality, direction, entry/SL/TP. Acceptance: 100% modulo documented daemon divergences (single-position rule, quality gate). Any mismatch is a bug, not feed noise.
+2. **Feed divergence (measured, not zero-gated)**: `backtest/research/shadow_compare.py` parses `logs/telegram.log` + daemon log into per-scan rows: signal agreement rate, direction flips, grade drift, clustering by hour/weekday. Expect high-single-digit % divergence concentrated near grade thresholds and Monday sessions (TD Saturday/rollover bars). Deliverable: written note attributing every divergence class.
+3. **Operational telemetry**: boundary grace-retry frequency (is Oanda's just-closed bar present at boundary+10s?), `get_spread()` time series (live input to the Phase 5 cost decision), stream disconnect counts, Monday-open indicator snapshots TD-vs-Oanda.
+4. **Owner note**: at cutover the data feed host family should match the execution host (practice vs live pricing can micro-differ).
+
+## Phase 5 — Re-validation campaign (rule 1, owner gates)
+
+Fetch Oanda history (M5/M15/H1/H4/D1, 2020-07 → present; Oanda XAU reaches ~2005). Run matrix per primary TF (M5/M15/H1, current factory configs): 12-mo train/val halves, 5-yr, OOS 2024-06→2025-03 — each at `--spread 0.30` (pure feed-effect A/B vs pinned TD books), `0.70` (honest Oanda cost), `0.85` (rollover stress). Caution: Jul–Nov 2025 is provider-sensitive (weekly offsets −$6..+$18.5) — lean on 5-yr + OOS for decisions. Acceptance unchanged: positive both halves, PF ≥ 1.3, trailing exits, next-open fills, OOS positive.
+
+**Owner decision gates (never silent):**
+1. **Canonical cost** — (a) `round_trip_cost=0.70` everywhere (moves the live TP1 spread gate $9→$21 — a strategy re-tune); (b) `default_spread=0.70` for backtest PnL but retune `min_tp_cost_multiple` 30→~13 to preserve the validated $9 gate; (c) keep $0.30 with documented optimism (only if $0.70 books pass anyway). Full acceptance matrix at chosen values.
+2. **Degradation handling** per failing TF: re-run tuning grids on Oanda data / drop TF from `scanning.timeframes` / keep Oanda execution-only (status quo).
+3. **Re-pin**: passing books become the new pinned CSVs + factory-docstring evidence; CLAUDE.md tuned-value annotations update in the cutover commit, not before.
+
+Files (per decisions): `pair_signals/xauusd.py`, `config.py` (`PAIR_INFO.default_spread`, `SignalConfig.round_trip_cost`/`min_tp_cost_multiple`), `test_all.py` tuned-constant assertions, CLAUDE.md.
+
+## Phase 6 — Cutover (reversible forever; no TD removal)
+
+1. config.json: `data_providers.active: "oanda"` + `"oanda": {}` block; restart bot. Everything flips through two seams: `make_data_provider` (bot shared dp, CLI, backtester) and the Phase 3 monitor selector. Any Phase 5 tuned-value changes land in this commit with their validation records.
+2. **Bake ≥1 week** (full weekly cycle incl. Monday open): grace-retry rate, stream stability, alert plausibility vs daemon, `/cache` freshness.
+3. **Rollback = flip `active` back** — TD code, key, caches, and WS monitor all stay maintained indefinitely (owner decision). Dual-provider posture: cross-provider sync tests (Phase 1) keep both honest; TD subscription/key must stay valid to remain a real fallback.
+4. Docs: CLAUDE.md (pipeline header, data_provider section, env-var table + `OANDA_STREAM_URL`, backtester cache note, comparability note: "pre-Oanda books are TD-feed books"), `docs/` guides if they name the data source.
+
+## Key risks
+
+1. **Cost-gate coupling**: naively setting `round_trip_cost=0.70` silently moves the live $9 TP1 gate to $21 — owner gate 5.1 exists precisely for this.
+2. **Cutover before re-pinning** — feed divergence is material (11% of ATR, 10% direction flips); sequencing rule prevents it.
+3. **Hermetic test header** — `_TEST_APP` pins twelve_data; provider-swapping tests must restore it or later modules fail mysteriously.
+4. **Stream liveness triad** — `chunk_size=1`, `resp.close()` interruption, 15s watchdog.
+5. **H4/D1 alignment** — audit before trusting any Oanda backtest.
+6. **Monday/rollover divergence class** — attribute in Phase 4 evidence; do not "fix".
+
+## Verification (per phase)
+
+- Phases 1–3: `python test_all.py` green (235 + new); live behavior provably unchanged (TD active).
+- Phase 2: pre/post TD book identity diff; Oanda H4/D1 alignment audit.
+- Phase 3: soak a second bot instance with `active=oanda` scratch config; observe stream reconnects in log.
+- Phase 5: full acceptance matrix, owner sign-offs recorded in factory docstrings.
+- Phase 6: 1-week bake checklist above; rollback drill (flip back, confirm TD scan cycle) once during bake.

app_config.py

@@ -0,0 +1,349 @@
+"""
+app_config.py — deployment-settings overlay (config.json)
+=========================================================
+
+Loads the gitignored config.json at the repo root ONCE per process.
+Deployment surface ONLY: Telegram credentials, data-provider selection
+and API keys, scanned pairs/timeframes, per-pair base blocked hours,
+alert-quality boot default, log level. Signal logic, weights,
+thresholds and gates stay in config.py / pair_signals.
+
+Precedence for every setting: env var → config.json → code default.
+A repo-root .env file (gitignored) is loaded at import via setdefault,
+so it behaves exactly like real environment variables except that a
+real shell export always wins. Secrets have NO code default — with
+neither env nor config.json the bot refuses to start
+(telegram_bot.main) and data fetches fail with a named error
+(data_provider).
+
+Parity contract (test-enforced): with no config.json, or one holding
+the config.json.example defaults, every produced PlatformConfig is
+byte-identical to the pure-code defaults. apply_pair_overrides() runs
+in the pair factory BEFORE the validated per-timeframe deltas, so the
+bot, CLI and backtester all see the same overlay (live == backtest).
+
+Spec: docs/superpowers/specs/2026-07-21-centralized-config-design.md
+"""
+
+import json
+import logging
+import os
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional
+
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+logger = logging.getLogger(__name__)
+
+CONFIG_PATH = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "config.json")
+ENV_PATH = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), ".env")
+
+
+def load_dotenv(path: str = ENV_PATH) -> None:
+    """Minimal .env loader (no python-dotenv dependency).
+
+    KEY=VALUE lines land in os.environ via setdefault — a variable
+    already exported in the shell always wins, keeping the
+    env → config.json precedence intact (.env is just a convenient
+    place to put env vars). Comments, blank lines, ``export `` prefixes
+    and single/double quotes are handled; anything else is skipped.
+    """
+    try:
+        with open(path, encoding="utf-8") as fh:
+            lines = fh.read().splitlines()
+    except OSError:
+        return
+    for line in lines:
+        line = line.strip()
+        if not line or line.startswith("#") or "=" not in line:
+            continue
+        if line.startswith("export "):
+            line = line[len("export "):]
+        key, _, value = line.partition("=")
+        key, value = key.strip(), value.strip()
+        if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
+            value = value[1:-1]
+        elif value[:1] not in "\"'":
+            # Unquoted value: whitespace-then-# starts an inline comment
+            # (KEY=abc  # prod). Quote the value to keep a literal " #".
+            for sep in (" #", "\t#"):
+                cut = value.find(sep)
+                if cut != -1:
+                    value = value[:cut].rstrip()
+        if key:
+            os.environ.setdefault(key, value)
+
+
+# Module import == process boot, before any resolver can run.
+load_dotenv()
+
+_VALID_QUALITIES = ("A+", "A", "B", "C")
+_VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
+# Timeframes the bot's candle-boundary scheduler supports (must have
+# AUTOSCAN_CONFIRMATIONS entries in telegram_bot.py).
+_SCANNABLE_TFS = ("M5", "M15", "H1", "H4")
+
+_KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level", "oanda"}
+_KNOWN_OANDA = {
+    "account_id", "api_token", "base_url", "lot_size",
+    "quality_gate", "max_hold_bars",
+}
+_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
+_KNOWN_SCANNING = {"pairs", "timeframes"}
+_KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
+
+
+class AppConfigError(Exception):
+    """config.json is present but invalid — fail fast at boot."""
+
+
+@dataclass
+class AppConfig:
+    telegram_token: str = ""
+    telegram_chat_id: str = ""
+    telegram_channel_id: str = ""          # "" = channel feed disabled
+    alert_min_quality: str = "C"
+    active_provider: str = "twelve_data"
+    provider_settings: Dict[str, Dict[str, Any]] = field(default_factory=dict)
+    scan_pairs: List[str] = field(default_factory=lambda: [XAUUSD])
+    scan_timeframes: List[Timeframe] = field(default_factory=lambda: [
+        Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4])
+    pair_overrides: Dict[str, Dict[str, Any]] = field(default_factory=dict)
+    oanda_settings: Dict[str, Any] = field(default_factory=dict)
+    log_level: str = "INFO"
+
+
+def _check_keys(section: Dict[str, Any], known: set, where: str) -> None:
+    if not isinstance(section, dict):
+        raise AppConfigError(f"config.json: {where} must be a JSON object")
+    unknown = sorted(set(section) - known)
+    if unknown:
+        raise AppConfigError(
+            f"config.json: unknown key(s) {unknown} in {where}; "
+            f"known keys: {sorted(known)}")
+
+
+def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
+    """Parse + validate config.json. Missing file → pure defaults."""
+    app = AppConfig()
+    if not os.path.exists(path):
+        logger.warning(
+            "config.json not found at %s — running on code defaults "
+            "(secrets must come from environment variables)", path)
+        return app
+    try:
+        with open(path, encoding="utf-8") as fh:
+            raw = json.load(fh)
+    except json.JSONDecodeError as e:
+        raise AppConfigError(f"config.json is not valid JSON: {e}") from e
+    if not isinstance(raw, dict):
+        raise AppConfigError("config.json top level must be a JSON object")
+    _check_keys(raw, _KNOWN_TOP, "top level")
+
+    tg = raw.get("telegram", {})
+    _check_keys(tg, _KNOWN_TELEGRAM, '"telegram"')
+    app.telegram_token = str(tg.get("token", app.telegram_token))
+    app.telegram_chat_id = str(tg.get("chat_id", app.telegram_chat_id))
+    app.telegram_channel_id = str(
+        tg.get("channel_id", app.telegram_channel_id))
+    quality = tg.get("alert_min_quality", app.alert_min_quality)
+    if quality not in _VALID_QUALITIES:
+        raise AppConfigError(
+            f'config.json: telegram.alert_min_quality "{quality}" '
+            f"not in {_VALID_QUALITIES}")
+    app.alert_min_quality = quality
+
+    dp = raw.get("data_providers")
+    if dp is not None:
+        if not isinstance(dp, dict):
+            raise AppConfigError('config.json: "data_providers" must be an object')
+        app.active_provider = str(dp.get("active", app.active_provider))
+        app.provider_settings = {}
+        for name, block in dp.items():
+            if name == "active":
+                continue
+            if not isinstance(block, dict):
+                raise AppConfigError(
+                    f'config.json: data_providers.{name} must be an object')
+            app.provider_settings[name] = dict(block)
+        if app.active_provider not in app.provider_settings:
+            raise AppConfigError(
+                f'config.json: data_providers.active "{app.active_provider}" '
+                "has no matching provider block")
+
+    scanning = raw.get("scanning", {})
+    _check_keys(scanning, _KNOWN_SCANNING, '"scanning"')
+    if "pairs" in scanning:
+        pairs = scanning["pairs"]
+        if not isinstance(pairs, list):
+            raise AppConfigError('config.json: scanning.pairs must be a list')
+        bad = [p for p in pairs if p not in PAIR_INFO]
+        if bad:
+            raise AppConfigError(
+                f"config.json: unknown pair(s) {bad} in scanning.pairs; "
+                f"known: {sorted(PAIR_INFO)}")
+        app.scan_pairs = list(pairs)
+    if "timeframes" in scanning:
+        names = scanning["timeframes"]
+        if not isinstance(names, list):
+            raise AppConfigError(
+                'config.json: scanning.timeframes must be a list')
+        bad = [n for n in names if n not in _SCANNABLE_TFS]
+        if bad:
+            raise AppConfigError(
+                f"config.json: unsupported timeframe(s) {bad} in "
+                f"scanning.timeframes; supported: {list(_SCANNABLE_TFS)}")
+        app.scan_timeframes = [Timeframe[n] for n in names]
+
+    pair_section = raw.get("pairs", {})
+    if not isinstance(pair_section, dict):
+        raise AppConfigError('config.json: "pairs" must be a JSON object')
+    for symbol, overrides in pair_section.items():
+        if symbol not in PAIR_INFO:
+            raise AppConfigError(
+                f'config.json: unknown pair "{symbol}" in pairs; '
+                f"known: {sorted(PAIR_INFO)}")
+        _check_keys(overrides, _KNOWN_PAIR_KEYS, f'pairs."{symbol}"')
+        if "blocked_hours_utc" in overrides:
+            hours = overrides["blocked_hours_utc"]
+            valid = isinstance(hours, list) and all(
+                isinstance(h, int) and not isinstance(h, bool)
+                and 0 <= h <= 23 for h in hours)
+            if not valid:
+                raise AppConfigError(
+                    f'config.json: pairs."{symbol}".blocked_hours_utc must '
+                    "be a list of integers in 0-23")
+    app.pair_overrides = {s: dict(o) for s, o in pair_section.items()}
+
+    oanda = raw.get("oanda", {})
+    _check_keys(oanda, _KNOWN_OANDA, '"oanda"')
+    if "lot_size" in oanda:
+        ls = oanda["lot_size"]
+        if not isinstance(ls, int) or isinstance(ls, bool) or ls < 1:
+            raise AppConfigError(
+                "config.json: oanda.lot_size must be a positive integer")
+    if "quality_gate" in oanda and oanda["quality_gate"] not in _VALID_QUALITIES:
+        raise AppConfigError(
+            f'config.json: oanda.quality_gate "{oanda["quality_gate"]}" '
+            f"not in {_VALID_QUALITIES}")
+    if "max_hold_bars" in oanda:
+        mhb = oanda["max_hold_bars"]
+        if not isinstance(mhb, int) or isinstance(mhb, bool) or mhb < 1:
+            raise AppConfigError(
+                "config.json: oanda.max_hold_bars must be a positive integer")
+    app.oanda_settings = dict(oanda)
+
+    level = raw.get("log_level", app.log_level)
+    if level not in _VALID_LOG_LEVELS:
+        raise AppConfigError(
+            f'config.json: log_level "{level}" not in {_VALID_LOG_LEVELS}')
+    app.log_level = level
+    return app
+
+
+# ── Process-wide singleton ────────────────────────────────────────────
+
+_cached: Optional[AppConfig] = None
+
+
+def get_app_config() -> AppConfig:
+    global _cached
+    if _cached is None:
+        _cached = load_app_config()
+    return _cached
+
+
+def set_app_config(app: Optional[AppConfig]) -> None:
+    """Override the process-wide AppConfig (tests/experiments).
+    Pass None to re-enable lazy loading from CONFIG_PATH."""
+    global _cached
+    _cached = app
+
+
+# ── Resolution helpers: env var → config.json → default ───────────────
+
+def telegram_token(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("TELEGRAM_TOKEN") or app.telegram_token
+
+
+def telegram_chat_id(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("TELEGRAM_CHAT_ID") or app.telegram_chat_id
+
+
+def telegram_channel_id(app: Optional[AppConfig] = None) -> str:
+    """Env set-but-empty ("") means "disable the feed" and beats JSON,
+    so this checks None-ness, not truthiness."""
+    app = app or get_app_config()
+    env = os.environ.get("TELEGRAM_CHANNEL_ID")
+    return env if env is not None else app.telegram_channel_id
+
+
+# Per-provider env-var overrides for API keys (env → config.json → "").
+_PROVIDER_ENV_KEYS = {
+    "twelve_data": "TWELVE_DATA_API_KEY",
+    "tiingo": "TIINGO_API_KEY",
+}
+
+
+def provider_api_key(
+    app: Optional[AppConfig] = None, provider: Optional[str] = None
+) -> str:
+    """API key for `provider` (default: the ACTIVE provider). The env
+    override is per-provider (TWELVE_DATA_API_KEY applies only to
+    twelve_data, TIINGO_API_KEY only to tiingo)."""
+    app = app or get_app_config()
+    name = provider or app.active_provider
+    env_key = _PROVIDER_ENV_KEYS.get(name)
+    if env_key:
+        env = os.environ.get(env_key)
+        if env:
+            return env
+    block = app.provider_settings.get(name, {})
+    return str(block.get("api_key", ""))
+
+
+# ── Oanda resolution helpers (env → config.json → default) ───────────
+
+def oanda_account_id(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("OANDA_ACCOUNT_ID") or str(
+        app.oanda_settings.get("account_id", ""))
+
+
+def oanda_api_token(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("OANDA_API_TOKEN") or str(
+        app.oanda_settings.get("api_token", ""))
+
+
+def oanda_base_url(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("OANDA_BASE_URL") or str(
+        app.oanda_settings.get("base_url", "https://api-fxpractice.oanda.com"))
+
+
+# ── PlatformConfig overlay ────────────────────────────────────────────
+
+def apply_pair_overrides(
+    cfg: PlatformConfig, app: Optional[AppConfig] = None
+) -> PlatformConfig:
+    """Stamp deployment settings from config.json onto a PlatformConfig.
+
+    Called by the pair factories BEFORE their validated per-timeframe
+    deltas, so e.g. the M5 hour-9 append lands on top of the JSON base.
+    The pair's base blocked_hours_utc is set on BOTH mirrored copies
+    (xau_strategy + backtest) so they cannot drift.
+    """
+    app = app or get_app_config()
+    overrides = app.pair_overrides.get(cfg.symbol, {})
+    if "blocked_hours_utc" in overrides:
+        hours = list(overrides["blocked_hours_utc"])
+        cfg.xau_strategy.blocked_hours_utc = hours
+        cfg.backtest.blocked_hours_utc = list(hours)
+    cfg.scan_pairs = list(app.scan_pairs)
+    cfg.scan_timeframes = list(app.scan_timeframes)
+    return cfg

backtest/diagnose_filters.py

@@ -51,7 +51,7 @@ from gpu_backtest import (  # noqa: E402
     precompute_indicators,
     _is_xau_blocked_by_calendar,
 )
-from data_provider import ForexDataProvider  # noqa: E402
+from providers import make_data_provider  # noqa: E402
 
 logger = logging.getLogger("diagnose_filters")
 
@@ -206,7 +206,7 @@ def run_diagnosis(
         confirmation_timeframes=list(confirm_tfs),
     )
 
-    provider = ForexDataProvider(platform_cfg)
+    provider = make_data_provider(platform_cfg)
 
     # ── Load data ──
     timeframes = [primary_tf] + list(confirm_tfs)

backtest/gpu_backtest.py

@@ -55,7 +55,7 @@ from config import (  # noqa: E402
     Timeframe,
     XAUUSD,
 )
-from data_provider import ForexDataProvider  # noqa: E402
+from providers import BaseDataProvider, make_data_provider  # noqa: E402
 from engine import IndicatorSignal, TechnicalAnalysis, TechnicalEngine  # noqa: E402
 from signal_aggregator import SignalAggregator, SignalReport, TradeSetup  # noqa: E402
 
@@ -851,15 +851,10 @@ class GPUTechnicalEngine(TechnicalEngine):
                 signal = SignalStrength.SELL
                 conf = min(70, 50 + roc_abs * 10)
                 desc = f"ROC {roc_val:.2f}% — Falling momentum"
-        elif roc_prev <= 0 < roc_val:
-            signal = SignalStrength.BUY
-            conf = 65
-            desc = f"ROC crossed above zero (+{roc_val:.2f}%)"
-        elif roc_prev >= 0 > roc_val:
-            signal = SignalStrength.SELL
-            conf = 65
-            desc = f"ROC crossed below zero ({roc_val:.2f}%)"
         else:
+            # Zero-crosses land in the branches above by definition (a
+            # cross up is positive and rising, a cross down negative and
+            # falling) — only flat/fading momentum reaches here.
             signal = SignalStrength.NEUTRAL
             conf = 30
             desc = f"ROC {roc_val:+.2f}% — No clear momentum"
@@ -940,13 +935,21 @@ class GPUTechnicalEngine(TechnicalEngine):
 CACHE_DIR = Path(__file__).resolve().parent / "cache"
 
 
-def _cache_path(symbol: str, tf: Timeframe, start: datetime, end: datetime) -> Path:
+def _cache_path(
+    symbol: str, tf: Timeframe, start: datetime, end: datetime,
+    provider_name: str = "twelve_data",
+) -> Path:
     safe_sym = symbol.replace("/", "_").replace("=X", "")
-    return CACHE_DIR / f"{safe_sym}_{tf.value}_{start:%Y%m%d}_{end:%Y%m%d}.parquet"
+    # Bars differ between feeds (Oanda vs TD divergence study 2026-07-24),
+    # so the cache is keyed per provider. twelve_data keeps the legacy
+    # un-prefixed filename so pinned-book caches stay valid.
+    prefix = "" if provider_name == "twelve_data" else f"{provider_name}_"
+    return CACHE_DIR / (
+        f"{prefix}{safe_sym}_{tf.value}_{start:%Y%m%d}_{end:%Y%m%d}.parquet")
 
 
 def fetch_history(
-    provider: ForexDataProvider,
+    provider: BaseDataProvider,
     symbol: str,
     timeframe: Timeframe,
     start: datetime,
@@ -954,91 +957,35 @@ def fetch_history(
     use_cache: bool = True,
     clean: bool = True,
 ) -> pd.DataFrame:
-    """Fetch OHLCV between start/end with parquet caching and pagination.
+    """Fetch OHLCV between start/end with parquet caching.
 
-    Twelve Data caps responses at 5000 bars; we paginate by date range so
-    multi-month M5/M15 backtests work.
+    Pagination under the provider's per-request bar cap lives in
+    `BaseDataProvider.fetch_range`, so any registered provider can feed
+    the backtester.
 
-    The parquet cache stores the RAW series; `_validate_and_clean` runs on
+    The parquet cache stores the RAW series; `validate_and_clean` runs on
     load. Pass clean=False to get the raw price path — trade resolution must
     use it, because cleaning drops >10σ spike bars (news candles) that hit
     stops in reality. (Caches written before this change already contain
     cleaned data; a refetch is needed to restore dropped spike bars.)
     """
     CACHE_DIR.mkdir(parents=True, exist_ok=True)
-    path = _cache_path(symbol, timeframe, start, end)
+    path = _cache_path(symbol, timeframe, start, end, provider.name)
     if use_cache and path.exists():
         logger.info(f"Loading cached {symbol} {timeframe.value}: {path.name}")
         df = pd.read_parquet(path)
         if df.index.tz is None:
             df.index = df.index.tz_localize("UTC")
-        return provider._validate_and_clean(df, symbol, timeframe) if clean else df
+        return provider.validate_and_clean(df, symbol, timeframe) if clean else df
 
     logger.info(f"Fetching {symbol} {timeframe.value} from {start.date()} to {end.date()}")
 
-    # Bars-per-call estimate to size each window safely under the 5000 cap.
-    bars_per_day = ForexDataProvider.BARS_PER_DAY[timeframe]
-    days_per_chunk = max(1, int(4500 / bars_per_day))
-
-    chunks: List[pd.DataFrame] = []
-    cursor_end = end
-    empty_streak = 0
-    while cursor_end > start:
-        cursor_start = max(start, cursor_end - timedelta(days=days_per_chunk))
-        try:
-            chunk = provider._fetch_twelve_data(
-                symbol=symbol,
-                timeframe=timeframe,
-                outputsize=5000,
-                start_date=cursor_start.strftime("%Y-%m-%d %H:%M:%S"),
-                end_date=cursor_end.strftime("%Y-%m-%d %H:%M:%S"),
-            )
-        except Exception as e:
-            # Twelve Data raises BadRequest for windows with no data (weekends,
-            # holidays, asset gaps). Skip the window, step back, retry — but
-            # bail out after a few consecutive empties to avoid spinning.
-            logger.warning(
-                f"Chunk {cursor_start.date()}–{cursor_end.date()} failed: {e}"
-            )
-            empty_streak += 1
-            if empty_streak >= 3:
-                break
-            cursor_end = cursor_start - timedelta(seconds=1)
-            continue
-
-        if chunk is None or chunk.empty:
-            empty_streak += 1
-            if empty_streak >= 3:
-                break
-            cursor_end = cursor_start - timedelta(seconds=1)
-            continue
-
-        empty_streak = 0
-        chunks.append(chunk)
-        # Step back; dedup on concat.
-        cursor_end = chunk.index[0] - timedelta(seconds=1)
-        if len(chunk) < 2:
-            break
-
-    if not chunks:
-        raise RuntimeError(f"No data fetched for {symbol} {timeframe.value}")
-
-    df = pd.concat(chunks).sort_index()
-    df = df[~df.index.duplicated(keep="last")]
-    # Fetches ending "now" include the still-forming candle — drop it so the
-    # cache only ever contains closed bars.
-    df = provider._drop_forming_candle(df, timeframe)
-    # Trim to requested window. `start`/`end` are already tz-aware UTC
-    # datetimes — wrapping with pd.Timestamp(..., tz=...) would raise on
-    # already-localized inputs.
-    end_ts = pd.Timestamp(end)
-    start_ts = pd.Timestamp(start)
-    df = df[(df.index >= start_ts) & (df.index <= end_ts)]
+    df = provider.fetch_range(symbol, timeframe, start, end)
 
     if use_cache:
         df.to_parquet(path)
         logger.info(f"Cached {len(df)} raw bars to {path.name}")
-    return provider._validate_and_clean(df, symbol, timeframe) if clean else df
+    return provider.validate_and_clean(df, symbol, timeframe) if clean else df
 
 
 # ═════════════════════════════════════════════════════════════════════
@@ -1530,6 +1477,13 @@ def _replay_span(span: Tuple[int, int]):
             use_news=False,
         )
 
+        # Live parity: analyze_pair applies the optional higher-TF ADX
+        # regime filter after delegation, under this exact guard. Calling
+        # the SAME method on the SAME sliced dfs keeps the two paths
+        # identical if the (default-off) filter is ever enabled.
+        if platform_cfg.backtest.regime_filter:
+            aggregator._apply_regime_filter_live(report, slice_dfs)
+
         if report.trade_setup is None:
             continue
         if report.signal_quality not in bt_cfg.quality_filter:
@@ -1679,10 +1633,17 @@ def run_backtest(
             f"vs PlatformConfig={platform_cfg.primary_timeframe.value}"
         )
 
-    provider = ForexDataProvider(platform_cfg)
+    provider = make_data_provider(platform_cfg)
 
     # ── Load all required timeframes ──
     timeframes = [bt_cfg.primary_tf] + list(bt_cfg.confirmation_tfs)
+    # Live parity: analyze_pair also fetches the regime-filter TF when the
+    # optional H4-ADX gate is enabled. Without it here, M5/M15 primaries
+    # (whose confirm maps lack H4) would make _apply_regime_filter_live
+    # silently no-op in the replay while live applies the gate.
+    flt = platform_cfg.backtest
+    if flt.regime_filter and flt.regime_filter_timeframe not in timeframes:
+        timeframes.append(flt.regime_filter_timeframe)
     # Pad lookback so earliest TF has enough warmup for EMA200, ADX, etc.
     pad_days = max(60, bt_cfg.warmup_bars // 4)
     fetch_start = bt_cfg.start - timedelta(days=pad_days)
@@ -1694,7 +1655,7 @@ def run_backtest(
         # ≥210 bars (outputsize floor). Target data_provider's min_required
         # (210 bars); ×7/5 converts trading days to calendar days. Intraday
         # TFs stay on the base pad, so their books are unchanged.
-        bars_per_day = ForexDataProvider.BARS_PER_DAY[tf]
+        bars_per_day = BaseDataProvider.BARS_PER_DAY[tf]
         min_pad_days = int(np.ceil(210 / bars_per_day * 7 / 5)) + 5
         tf_start = bt_cfg.start - timedelta(days=max(pad_days, min_pad_days))
         df = fetch_history(
@@ -1872,7 +1833,9 @@ def compute_metrics(trades: List[Trade]) -> Metrics:
     gross_loss = float(-rs[losses_mask].sum()) if losses_mask.any() else 0.0
     pf = gross_profit / gross_loss if gross_loss > 0 else float("inf")
 
-    cum = np.cumsum(rs)
+    # Prepend the flat starting equity so an opening losing streak counts
+    # as drawdown from 0, not from the first trade's already-negative cum.
+    cum = np.concatenate(([0.0], np.cumsum(rs)))
     high_water = np.maximum.accumulate(cum)
     drawdown = high_water - cum
     max_dd = float(drawdown.max()) if len(drawdown) else 0.0
@@ -2141,6 +2104,10 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
     p.add_argument("--max-hold-bars", type=int, default=300)
     p.add_argument("--swing-sr-lookback", type=int, default=0,
                    help="Override TechnicalConfig.swing_sr_lookback (0 = use config default)")
+    p.add_argument("--trail-arm", type=float, default=None,
+                   help="Override signals.trail_arm_r for this run (research knob)")
+    p.add_argument("--trail-dist", type=float, default=None,
+                   help="Override signals.trail_dist_r for this run (research knob)")
     p.add_argument("--quality", default="A+,A,B,C", help="Comma-separated allowed grades")
     p.add_argument("--strategies", default="",
                    help="Comma-separated strategy tags to include (XAU,XAU_PB,XAU_CP,GENERIC). Empty = all.")
@@ -2214,8 +2181,16 @@ def main(argv: Optional[List[str]] = None) -> int:
             primary_timeframe=primary_tf,
             confirmation_timeframes=confirm_tfs or [Timeframe.H1, Timeframe.H4],
         )
+        # The factory path applies the config.json overlay inside
+        # make_config; the fallback must match (live == backtest).
+        from app_config import apply_pair_overrides  # noqa: E402
+        apply_pair_overrides(platform_cfg)
     if args.swing_sr_lookback > 0:
         platform_cfg.technical.swing_sr_lookback = args.swing_sr_lookback
+    if args.trail_arm is not None:
+        platform_cfg.signals.trail_arm_r = args.trail_arm
+    if args.trail_dist is not None:
+        platform_cfg.signals.trail_dist_r = args.trail_dist
     if args.cp:
         platform_cfg.xau_strategy.cp_enabled = True
         if args.cp_patterns:

backtest/research/duka_spread.py

@@ -0,0 +1,194 @@
+"""Dukascopy XAU/USD tick download + bid/ask spread study.
+
+Validates ForexBot2's $0.30 round-trip cost assumption (config:
+SignalConfig.round_trip_cost / PAIR_INFO default_spread). Round-trip
+cost of a market entry+exit = one ask-bid spread width (buy at ask,
+sell at bid), so the assumption holds iff typical spread ~ $0.30.
+
+bi5 format: LZMA 'alone' stream of 20-byte big-endian records
+(ms_in_hour uint32, ask uint32, bid uint32, ask_vol f32, bid_vol f32);
+XAUUSD prices scaled x1000. URL months are 0-indexed.
+"""
+import lzma
+import struct
+import sys
+from concurrent.futures import ThreadPoolExecutor
+from datetime import date, timedelta
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+import requests
+
+SCALE = 1000.0
+CACHE = Path(__file__).parent / "tick_cache"
+CACHE.mkdir(exist_ok=True)
+URL = "https://datafeed.dukascopy.com/datafeed/XAUUSD/{y}/{m:02d}/{d:02d}/{h:02d}h_ticks.bi5"
+
+# Bot's tradable bar-close hours: session 03:00-23:00 minus XAU blocked
+# hours [0,1,2,10,11,13,16,21] (M5 additionally blocks 9 — reported
+# separately below).
+BLOCKED = {0, 1, 2, 10, 11, 13, 16, 21}
+TRADABLE = sorted(set(range(3, 23)) - BLOCKED)
+
+# ── Day sample ────────────────────────────────────────────────────────
+recent = [date(2026, 7, 6) + timedelta(days=i) for i in range(17)]
+recent = [d for d in recent if d.weekday() < 5]              # Mon-Fri Jul 6-22
+monthly = [date(2025, 8, 13), date(2025, 9, 10), date(2025, 10, 15),
+           date(2025, 11, 12), date(2025, 12, 10), date(2026, 1, 14),
+           date(2026, 2, 11), date(2026, 3, 11), date(2026, 4, 15),
+           date(2026, 5, 13), date(2026, 6, 10)]             # mid-month Wednesdays
+nfp = [date(2026, 6, 5), date(2026, 7, 3)]                   # NFP Fridays
+sundays = [date(2026, 7, 12), date(2026, 7, 19)]             # Sun 22-24 open
+DAYS = sorted(set(recent + monthly + nfp + sundays))
+assert all(d.weekday() < 5 for d in recent + monthly + nfp)
+
+
+_tls = __import__("threading").local()
+
+
+def _session() -> requests.Session:
+    if not hasattr(_tls, "s"):
+        _tls.s = requests.Session()
+        _tls.s.headers["User-Agent"] = "Mozilla/5.0 (research; spread-study)"
+    return _tls.s
+
+
+def fetch(day: date, hour: int) -> Path:
+    p = CACHE / f"{day.isoformat()}_{hour:02d}.bi5"
+    if p.exists():
+        return p
+    u = URL.format(y=day.year, m=day.month - 1, d=day.day, h=hour)
+    last = None
+    for attempt in range(4):
+        try:
+            r = _session().get(u, timeout=30)
+            if r.status_code != 200:
+                raise RuntimeError(f"{u} -> HTTP {r.status_code}")
+            p.write_bytes(r.content)
+            return p
+        except (requests.ConnectionError, requests.Timeout, RuntimeError) as exc:
+            last = exc
+            __import__("time").sleep(1.5 * (attempt + 1))
+            if hasattr(_tls, "s"):
+                _tls.s.close()
+                del _tls.s
+    raise RuntimeError(f"giving up on {u}: {last}")
+
+
+def decode(p: Path, day: date, hour: int) -> pd.DataFrame:
+    blob = p.read_bytes()
+    if not blob:
+        return pd.DataFrame()
+    raw = lzma.decompress(blob)
+    n = len(raw) // 20
+    a = np.frombuffer(raw[: n * 20], dtype=">u4").reshape(n, 5)
+    f = np.frombuffer(raw[: n * 20], dtype=">f4").reshape(n, 5)
+    df = pd.DataFrame({
+        "ms": a[:, 0].astype(np.int64),
+        "ask": a[:, 1] / SCALE,
+        "bid": a[:, 2] / SCALE,
+    })
+    df["spread"] = df["ask"] - df["bid"]
+    df["date"], df["hour"] = day.isoformat(), hour
+    df["weekday"] = day.weekday()
+    return df
+
+
+def load_all() -> pd.DataFrame:
+    jobs = [(d, h) for d in DAYS for h in range(24)]
+    with ThreadPoolExecutor(max_workers=3) as ex:
+        paths = list(ex.map(lambda j: fetch(*j), jobs))
+    frames = [decode(p, d, h) for p, (d, h) in zip(paths, jobs)]
+    df = pd.concat([f for f in frames if not f.empty], ignore_index=True)
+    return df
+
+
+def wstats(g: pd.DataFrame) -> pd.Series:
+    """Tick stats + time-weighted mean (each tick's spread held until the
+    next tick; last tick held to hour end)."""
+    tw_num = tw_den = 0.0
+    keys = [k for k in ("date", "hour") if k in g.columns]
+    for _, grp in g.groupby(keys, sort=False):
+        ms = grp["ms"].to_numpy()
+        sp = grp["spread"].to_numpy()
+        w = np.diff(np.append(ms, 3_600_000)).clip(min=0)
+        tw_num += float((sp * w).sum())
+        tw_den += float(w.sum())
+    s = g["spread"]
+    return pd.Series({
+        "ticks": len(s),
+        "tw_mean": tw_num / tw_den if tw_den else np.nan,
+        "mean": s.mean(), "median": s.median(),
+        "p75": s.quantile(0.75), "p90": s.quantile(0.90),
+        "p99": s.quantile(0.99), "max": s.max(),
+        "pct_le_030": (s <= 0.30).mean() * 100,
+        "pct_ge_060": (s >= 0.60).mean() * 100,
+    })
+
+
+def m5_boundary_spreads(df: pd.DataFrame) -> pd.DataFrame:
+    """Spread of the FIRST tick at/after each M5 boundary — what a
+    next-open fill actually pays."""
+    rows = []
+    for (day, hour), grp in df.groupby(["date", "hour"], sort=False):
+        ms = grp["ms"].to_numpy()
+        sp = grp["spread"].to_numpy()
+        wd = grp["weekday"].iat[0]
+        for b in range(0, 3_600_000, 300_000):
+            i = np.searchsorted(ms, b)
+            if i < len(ms) and ms[i] - b <= 30_000:   # tick within 30 s
+                rows.append((day, wd, hour, b // 60_000, sp[i]))
+    return pd.DataFrame(rows, columns=["date", "weekday", "hour", "minute", "spread"])
+
+
+def main() -> None:
+    df = load_all()
+    print(f"days={df['date'].nunique()}  ticks={len(df):,}  "
+          f"price range {df['bid'].min():.0f}-{df['ask'].max():.0f}")
+
+    print("\n── Overall (all sampled hours) ──")
+    print(wstats(df).round(3).to_string())
+
+    trad = df[df["hour"].isin(TRADABLE) & (df["weekday"] < 5)]
+    print("\n── Bot-tradable hours (session 03-23 minus blocked, Mon-Fri) ──")
+    print(wstats(trad).round(3).to_string())
+
+    roll = df[df["hour"].isin([21, 22, 23])]
+    print("\n── Rollover window 21-24 UTC ──")
+    print(wstats(roll).round(3).to_string())
+
+    print("\n── Per UTC hour ──")
+    per_hour = df[df["weekday"] < 5].groupby("hour").apply(wstats, include_groups=False).round(3)
+    per_hour["tradable"] = ["Y" if h in TRADABLE else "-" for h in per_hour.index]
+    print(per_hour.to_string())
+    per_hour.to_csv(Path(__file__).parent / "duka_spread_by_hour.csv")
+
+    print("\n── M5 next-open boundary fills ──")
+    mb = m5_boundary_spreads(df)
+    mb_trad = mb[mb["hour"].isin(TRADABLE) & (mb["weekday"] < 5)]
+    for name, d in [("all hours", mb), ("tradable hours", mb_trad)]:
+        s = d["spread"]
+        print(f"{name}: n={len(s):,} mean={s.mean():.3f} median={s.median():.3f} "
+              f"p90={s.quantile(0.9):.3f} p99={s.quantile(0.99):.3f} "
+              f"%<=0.30={(s <= 0.30).mean()*100:.1f} %>=0.60={(s >= 0.60).mean()*100:.1f}")
+    mb.to_csv(Path(__file__).parent / "duka_m5_boundary_spreads.csv", index=False)
+
+    print("\n── By month (tradable hours, tick median / tw_mean) ──")
+    trad2 = trad.copy()
+    trad2["month"] = trad2["date"].str[:7]
+    bym = trad2.groupby("month").apply(wstats, include_groups=False).round(3)
+    print(bym[["ticks", "tw_mean", "median", "p90", "pct_le_030", "pct_ge_060"]].to_string())
+
+    print("\n── NFP Fridays 12:00-15:00 UTC vs same hours other days ──")
+    nfp_days = {"2026-06-05", "2026-07-03"}
+    win = df[df["hour"].isin([12, 13, 14])]
+    for name, d in [("NFP", win[win["date"].isin(nfp_days)]),
+                    ("non-NFP", win[~win["date"].isin(nfp_days) & (win["weekday"] < 5)])]:
+        s = d["spread"]
+        print(f"{name}: n={len(s):,} tick-median={s.median():.3f} p90={s.quantile(0.9):.3f} "
+              f"p99={s.quantile(0.99):.3f} max={s.max():.2f}")
+
+
+if __name__ == "__main__":
+    sys.exit(main())

backtest/research/duka_spread_by_hour.csv

@@ -0,0 +1,25 @@
+hour,ticks,tw_mean,mean,median,p75,p90,p99,max,pct_le_030,pct_ge_060,tradable
+0,362893.0,0.749,0.768,0.79,0.82,0.86,1.1,7.69,0.053,94.561,-
+1,540182.0,0.735,0.742,0.74,0.81,0.87,1.0,5.59,0.073,90.449,-
+2,431589.0,0.716,0.722,0.73,0.8,0.86,0.95,2.96,0.1,85.27,-
+3,343952.0,0.708,0.711,0.71,0.79,0.84,0.95,1.67,0.081,82.939,Y
+4,246262.0,0.704,0.702,0.71,0.78,0.82,0.94,1.86,0.098,82.627,Y
+5,388365.0,0.712,0.716,0.72,0.79,0.82,0.95,2.17,0.076,86.745,Y
+6,368999.0,0.65,0.651,0.65,0.69,0.74,0.94,2.32,0.079,71.668,Y
+7,313149.0,0.601,0.598,0.59,0.65,0.69,0.994,3.08,0.239,42.904,Y
+8,347276.0,0.612,0.616,0.6,0.66,0.74,1.0,3.04,0.308,48.349,Y
+9,321619.0,0.612,0.615,0.6,0.66,0.72,1.0,5.95,0.288,50.644,Y
+10,295271.0,0.611,0.618,0.6,0.66,0.71,1.0,9.36,0.164,49.253,-
+11,336963.0,0.619,0.624,0.6,0.67,0.79,1.0,4.97,0.203,50.178,-
+12,476773.0,0.66,0.697,0.63,0.72,0.877,1.367,14.98,0.257,59.633,Y
+13,651425.0,0.641,0.648,0.63,0.69,0.787,1.487,11.407,0.296,60.297,-
+14,673391.0,0.625,0.625,0.63,0.68,0.77,0.9,11.98,0.458,59.085,Y
+15,598398.0,0.636,0.638,0.64,0.7,0.78,0.91,4.49,0.433,66.296,Y
+16,485023.0,0.648,0.644,0.64,0.7,0.757,0.917,4.09,0.118,68.496,-
+17,450119.0,0.626,0.628,0.627,0.7,0.726,0.84,4.08,0.154,63.337,Y
+18,411538.0,0.626,0.63,0.637,0.7,0.71,0.79,4.16,0.107,67.734,Y
+19,435396.0,0.638,0.643,0.65,0.7,0.72,0.896,5.804,0.11,72.007,Y
+20,212654.0,0.684,0.665,0.66,0.7,0.76,1.217,7.15,0.226,70.215,Y
+21,31319.0,0.627,0.622,0.61,0.68,0.75,1.02,2.08,0.089,53.153,-
+22,108879.0,0.829,0.834,0.797,0.84,0.98,2.05,10.11,0.056,95.434,Y
+23,161079.0,1.032,0.913,0.8,0.89,1.307,2.77,13.174,0.092,95.41,-

backtest/research/fetch_oanda_m15.py

@@ -0,0 +1,158 @@
+"""Fetch XAU/USD M15 candles from the Oanda v20 REST API.
+
+Mechanical data-fetching infrastructure only — no analysis here.
+Paginates GET /v3/instruments/XAU_USD/candles from a start cursor
+through "now" and writes the result to a parquet file.
+
+Run from the repo root:
+    venv/bin/python backtest/research/fetch_oanda_m15.py
+"""
+import sys
+import time
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+import pandas as pd
+import requests
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
+
+from app_config import oanda_api_token, oanda_base_url  # noqa: E402
+
+INSTRUMENT = "XAU_USD"
+GRANULARITY = "M15"
+START = datetime(2025, 7, 1, 0, 0, 0, tzinfo=timezone.utc)
+OUT_PATH = Path(__file__).parent / "oanda_xau_m15_20250701_now.parquet"
+COUNT = 5000
+REQUEST_SLEEP_S = 0.5
+FRESHNESS_WINDOW = timedelta(minutes=30)
+
+
+def _rfc3339(dt: datetime) -> str:
+    return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
+
+
+def fetch_all_candles():
+    token = oanda_api_token()
+    base_url = oanda_base_url()
+    if not token:
+        raise RuntimeError(
+            "No Oanda API token resolved (env OANDA_API_TOKEN / "
+            "config.json oanda.api_token). Aborting.")
+
+    url = f"{base_url}/v3/instruments/{INSTRUMENT}/candles"
+    headers = {
+        "Authorization": f"Bearer {token}",
+        "Accept-Datetime-Format": "RFC3339",
+    }
+
+    cursor = START
+    include_first = True
+    all_rows = []
+    n_requests = 0
+
+    while True:
+        params = {
+            "granularity": GRANULARITY,
+            "price": "MBA",
+            "count": COUNT,
+            "dailyAlignment": 0,
+            "alignmentTimezone": "Etc/UTC",
+            "from": _rfc3339(cursor),
+            "includeFirst": "true" if include_first else "false",
+        }
+
+        resp = requests.get(url, headers=headers, params=params, timeout=30)
+        n_requests += 1
+        if resp.status_code != 200:
+            raise RuntimeError(
+                f"Oanda API request failed: status={resp.status_code}\n"
+                f"URL: {resp.url}\n"
+                f"Response body: {resp.text}")
+
+        payload = resp.json()
+        candles = payload.get("candles", [])
+
+        for c in candles:
+            if not c.get("complete", False):
+                continue
+            mid = c["mid"]
+            bid = c["bid"]
+            ask = c["ask"]
+            all_rows.append({
+                "Datetime": pd.Timestamp(c["time"]).tz_convert("UTC")
+                if pd.Timestamp(c["time"]).tzinfo is not None
+                else pd.Timestamp(c["time"]).tz_localize("UTC"),
+                "Open": float(mid["o"]),
+                "High": float(mid["h"]),
+                "Low": float(mid["l"]),
+                "Close": float(mid["c"]),
+                "Volume": int(c["volume"]),
+                "bid_o": float(bid["o"]),
+                "bid_h": float(bid["h"]),
+                "bid_l": float(bid["l"]),
+                "bid_c": float(bid["c"]),
+                "ask_o": float(ask["o"]),
+                "ask_h": float(ask["h"]),
+                "ask_l": float(ask["l"]),
+                "ask_c": float(ask["c"]),
+            })
+
+        print(f"  request {n_requests}: from={params['from']} "
+              f"got {len(candles)} candles (complete kept so far: "
+              f"{len(all_rows)})", file=sys.stderr)
+
+        if not candles:
+            break
+
+        last_time = pd.Timestamp(candles[-1]["time"])
+        if last_time.tzinfo is None:
+            last_time = last_time.tz_localize("UTC")
+        else:
+            last_time = last_time.tz_convert("UTC")
+
+        now = datetime.now(timezone.utc)
+        # Oanda's "count" includes the "from" boundary bar itself, so with
+        # includeFirst=false every full page comes back as count-1, not
+        # count (verified empirically: count=N, includeFirst=false always
+        # yields N-1 candles when a full page of data exists ahead of the
+        # cursor). Compare against the count actually expected for this
+        # request's includeFirst setting, not the raw requested count.
+        expected_full_page = COUNT if include_first else COUNT - 1
+        got_fewer_than_requested = len(candles) < expected_full_page
+        near_now = (now - last_time.to_pydatetime()) < FRESHNESS_WINDOW
+
+        if got_fewer_than_requested or near_now:
+            break
+
+        cursor = last_time.to_pydatetime()
+        include_first = False
+        time.sleep(REQUEST_SLEEP_S)
+
+    return all_rows, n_requests
+
+
+def main():
+    print(f"Fetching {INSTRUMENT} {GRANULARITY} candles from "
+          f"{_rfc3339(START)} through now...", file=sys.stderr)
+
+    rows, n_requests = fetch_all_candles()
+
+    if not rows:
+        raise RuntimeError("No complete candles were fetched.")
+
+    df = pd.DataFrame(rows)
+    df = df.set_index("Datetime")
+    df = df.sort_index()
+    df = df[~df.index.duplicated(keep="last")]
+
+    df.to_parquet(OUT_PATH)
+
+    print(f"\nSaved {len(df)} rows to {OUT_PATH}", file=sys.stderr)
+    print(f"Requests made: {n_requests}", file=sys.stderr)
+    print(f"First timestamp: {df.index[0]}", file=sys.stderr)
+    print(f"Last timestamp: {df.index[-1]}", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

backtest/research/oanda_td_divergence.py

@@ -0,0 +1,199 @@
+"""
+Oanda vs Twelve Data bar-divergence study — XAU/USD M15, 12 months.
+====================================================================
+Compares the mid-price OHLC bars the signal engine would see from each
+provider over the same window. Purpose: quantify whether migrating the
+signal feed from Twelve Data to Oanda is a formality (bars ~identical)
+or a recalibration (bars differ at a scale that moves indicators).
+
+Inputs:
+  --td      Twelve Data parquet (raw bars from backtest/cache/)
+  --oanda   Oanda parquet (from fetch_oanda_m5.py / fetch_oanda_m15.py)
+  --start / --end  analysis window (UTC dates)
+
+Outputs:
+  printed report
+  backtest/research/oanda_td_divergence_by_hour.csv
+  backtest/research/oanda_td_bar_diffs.parquet  (per-bar diffs, common index)
+"""
+
+import argparse
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+
+RESEARCH_DIR = Path(__file__).parent
+PIP = 0.10  # XAU pip in $
+
+
+def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
+    hl = df["High"] - df["Low"]
+    hc = (df["High"] - df["Close"].shift(1)).abs()
+    lc = (df["Low"] - df["Close"].shift(1)).abs()
+    tr = pd.concat([hl, hc, lc], axis=1).max(axis=1)
+    return tr.ewm(alpha=1.0 / period, adjust=False).mean()
+
+
+def pct_stats(s: pd.Series) -> dict:
+    a = s.abs()
+    return {
+        "mean": a.mean(), "median": a.median(),
+        "p90": a.quantile(0.90), "p95": a.quantile(0.95),
+        "p99": a.quantile(0.99), "max": a.max(),
+        "bias(mean signed)": s.mean(),
+    }
+
+
+def fmt_stats(name: str, s: pd.Series) -> str:
+    st = pct_stats(s)
+    return (f"  {name:<6} mean ${st['mean']:.3f}  med ${st['median']:.3f}  "
+            f"p95 ${st['p95']:.3f}  p99 ${st['p99']:.3f}  max ${st['max']:.2f}  "
+            f"bias {st['bias(mean signed)']:+.4f}")
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--td", default=str(RESEARCH_DIR.parent / "cache" /
+                                        "XAU_USD_15min_20240522_20260723.parquet"))
+    ap.add_argument("--oanda", default=str(RESEARCH_DIR /
+                                           "oanda_xau_m15_20250701_now.parquet"))
+    ap.add_argument("--start", default="2025-07-24")
+    ap.add_argument("--end", default="2026-07-23")
+    args = ap.parse_args()
+
+    td = pd.read_parquet(args.td)
+    oa = pd.read_parquet(args.oanda)
+
+    start = pd.Timestamp(args.start, tz="UTC")
+    end = pd.Timestamp(args.end, tz="UTC") + pd.Timedelta(days=1)
+    start = max(start, td.index[0], oa.index[0])
+    end = min(end, td.index[-1], oa.index[-1])
+    td = td.loc[(td.index >= start) & (td.index <= end)]
+    oa = oa.loc[(oa.index >= start) & (oa.index <= end)]
+
+    print("=" * 78)
+    print("OANDA vs TWELVE DATA — XAU/USD M15 bar divergence")
+    print(f"Window: {start} -> {end}  ({(end - start).days} days)")
+    print("=" * 78)
+
+    # ── Coverage ─────────────────────────────────────────────────────
+    common = td.index.intersection(oa.index)
+    td_only = td.index.difference(oa.index)
+    oa_only = oa.index.difference(td.index)
+    print(f"\nCOVERAGE")
+    print(f"  Twelve Data bars : {len(td):>7,}")
+    print(f"  Oanda bars       : {len(oa):>7,}")
+    print(f"  common           : {len(common):>7,}")
+    print(f"  TD-only          : {len(td_only):>7,}  "
+          f"({100 * len(td_only) / max(len(td), 1):.2f}% of TD)")
+    print(f"  Oanda-only       : {len(oa_only):>7,}  "
+          f"({100 * len(oa_only) / max(len(oa), 1):.2f}% of Oanda)")
+    for label, idx in (("TD-only", td_only), ("Oanda-only", oa_only)):
+        if len(idx):
+            by_date = pd.Series(1, index=idx).groupby(idx.date).sum()
+            top = by_date.sort_values(ascending=False).head(8)
+            print(f"  {label} bars cluster on: "
+                  + ", ".join(f"{d} ({n})" for d, n in top.items()))
+            by_hour = pd.Series(1, index=idx).groupby(idx.hour).sum()
+            top_h = by_hour.sort_values(ascending=False).head(6)
+            print(f"  {label} by UTC hour: "
+                  + ", ".join(f"{h:02d}h ({n})" for h, n in top_h.items()))
+
+    t = td.loc[common]
+    o = oa.loc[common]
+
+    # ── Scale references ─────────────────────────────────────────────
+    atr = wilder_atr(t).dropna()
+    mean_price = t["Close"].mean()
+    print(f"\nSCALE REFERENCES (Twelve Data feed)")
+    print(f"  mean close      : ${mean_price:,.2f}")
+    print(f"  mean bar range  : ${(t['High'] - t['Low']).mean():.3f}")
+    print(f"  mean ATR14      : ${atr.mean():.3f}   median ${atr.median():.3f}")
+
+    # ── Per-field diffs (TD − Oanda mid) ─────────────────────────────
+    print(f"\nPER-BAR DIFFS  (TD − Oanda, $; pip = $0.10)")
+    diffs = {}
+    for col in ("Open", "High", "Low", "Close"):
+        d = t[col] - o[col]
+        diffs[col] = d
+        print(fmt_stats(col, d))
+
+    dclose = diffs["Close"].abs()
+    print(f"\n  |ΔClose| as % of mean ATR14 : "
+          f"mean {100 * dclose.mean() / atr.mean():.1f}%   "
+          f"p95 {100 * dclose.quantile(0.95) / atr.mean():.1f}%")
+
+    print(f"\nTHRESHOLD EXCEEDANCE  (|ΔClose|)")
+    for thr in (0.10, 0.30, 0.65, 1.00, 2.00, 5.00):
+        pct = 100 * (dclose > thr).mean()
+        print(f"  > ${thr:<5.2f}: {pct:6.2f}% of bars  ({(dclose > thr).sum():,})")
+
+    # ── Direction / return agreement ─────────────────────────────────
+    t_ret = t["Close"].pct_change()
+    o_ret = o["Close"].pct_change()
+    # only consecutive common bars (gap = one M15 step)
+    step_ok = pd.Series(common, index=common).diff().eq(pd.Timedelta(minutes=15))
+    tr, orr = t_ret[step_ok], o_ret[step_ok]
+    corr = tr.corr(orr)
+    t_dir = np.sign(t["Close"] - t["Open"])
+    o_dir = np.sign(o["Close"] - o["Open"])
+    both_nonflat = (t_dir != 0) & (o_dir != 0)
+    dir_agree = 100 * (t_dir[both_nonflat] == o_dir[both_nonflat]).mean()
+    print(f"\nAGREEMENT")
+    print(f"  close-return correlation (consecutive bars): {corr:.5f}  "
+          f"(n={step_ok.sum():,})")
+    print(f"  bar direction (sign of close−open) agree   : {dir_agree:.2f}%")
+
+    # ── By-hour breakdown ────────────────────────────────────────────
+    hours = common.hour
+    rows = []
+    for h in range(24):
+        m = hours == h
+        if not m.any():
+            continue
+        row = {"hour_utc": h, "bars": int(m.sum()),
+               "mean_abs_dclose": dclose[m].mean(),
+               "p95_abs_dclose": dclose[m].quantile(0.95),
+               "mean_abs_dhigh": diffs["High"][m].abs().mean(),
+               "mean_abs_dlow": diffs["Low"][m].abs().mean()}
+        if "bid_c" in o.columns:
+            row["oanda_median_spread"] = (
+                (o["ask_c"] - o["bid_c"])[m].median())
+        rows.append(row)
+    by_hour = pd.DataFrame(rows)
+    out_hour = RESEARCH_DIR / "oanda_td_divergence_by_hour.csv"
+    by_hour.to_csv(out_hour, index=False, float_format="%.4f")
+    print(f"\nBY UTC HOUR  (written to {out_hour.name})")
+    print(by_hour.to_string(index=False,
+                            float_format=lambda x: f"{x:.3f}"))
+
+    # ── By-month drift ───────────────────────────────────────────────
+    print(f"\nBY MONTH  (mean |ΔClose|, $)")
+    by_month = dclose.groupby(common.to_period("M")).mean()
+    for p, v in by_month.items():
+        print(f"  {p}: {v:.3f}")
+
+    # ── Oanda real spread (bonus, ties to Dukascopy study) ───────────
+    if "bid_c" in o.columns:
+        spread = o["ask_c"] - o["bid_c"]
+        liquid = ~o.index.hour.isin([21, 22, 23, 0])
+        print(f"\nOANDA REAL SPREAD (close bid/ask, $)")
+        print(f"  all bars      : median {spread.median():.3f}  "
+              f"p25 {spread.quantile(0.25):.3f}  p75 {spread.quantile(0.75):.3f}")
+        print(f"  liquid hours  : median {spread[liquid].median():.3f}")
+        print(f"  21-00 UTC     : median {spread[~liquid].median():.3f}")
+
+    # ── Persist per-bar diffs ────────────────────────────────────────
+    out = pd.DataFrame({
+        "td_close": t["Close"], "oa_close": o["Close"],
+        "d_open": diffs["Open"], "d_high": diffs["High"],
+        "d_low": diffs["Low"], "d_close": diffs["Close"],
+    })
+    out_path = RESEARCH_DIR / "oanda_td_bar_diffs.parquet"
+    out.to_parquet(out_path)
+    print(f"\nPer-bar diffs written to {out_path}")
+
+
+if __name__ == "__main__":
+    main()

backtest/research/oanda_td_divergence_by_hour.csv

@@ -0,0 +1,25 @@
+hour_utc,bars,mean_abs_dclose,p95_abs_dclose,mean_abs_dhigh,mean_abs_dlow,oanda_median_spread
+0,1029,3.3483,13.9460,3.2906,3.4310,0.7900
+1,1028,3.3483,12.8292,3.3477,3.6398,0.7700
+2,1028,3.0288,10.8960,2.9469,3.1317,0.7700
+3,1028,2.8783,11.1300,2.8419,2.9992,0.7600
+4,1028,2.8232,10.4710,2.8165,2.8455,0.7400
+5,1028,2.9268,10.6027,2.9234,3.0038,0.7400
+6,1028,3.0466,10.9265,2.9487,3.0796,0.7200
+7,1028,3.0286,11.3512,3.0301,3.0797,0.6700
+8,1028,3.0223,11.1483,2.8989,3.0801,0.6600
+9,1028,3.0460,11.6345,2.9307,3.0609,0.6600
+10,1028,2.9280,10.9015,2.8647,3.0029,0.6600
+11,1028,3.0340,11.4712,2.9591,2.9472,0.6600
+12,1028,3.0986,11.3282,2.9412,3.1407,0.7000
+13,1028,3.4014,12.3343,3.1077,3.4228,0.7300
+14,1028,3.5074,11.3628,3.3960,3.6569,0.6900
+15,1028,3.4514,11.8982,3.4124,3.6733,0.6700
+16,1028,3.1275,12.5155,3.2038,3.3061,0.6700
+17,1020,2.9826,12.6435,3.0064,3.1008,0.6900
+18,1015,2.8721,12.5480,2.9101,2.9996,0.6900
+19,1004,2.8637,12.1335,2.8824,2.9454,0.6900
+20,996,2.8006,11.9325,2.8486,2.8577,0.7600
+21,335,1.1085,4.8600,1.1212,1.2034,0.9400
+22,676,4.0827,13.6787,4.8422,4.6761,0.8300
+23,1027,3.2722,12.5725,3.5617,3.7747,0.8700

chart_patterns.py

@@ -12,9 +12,10 @@ Chart Patterns*) that are unambiguous from OHLC at bar close:
                                       the interim peak (neckline)
     double_top     bearish reversal   mirrored
     inv_hs         bullish reversal   inverse head & shoulders, break of
-                                      the neckline (max of the two peaks
-                                      flanking the head — conservative
-                                      horizontal line, no sloped fits)
+                                      the neckline (the LOWER of the two
+                                      peaks flanking the head — the earlier
+                                      horizontal trigger, no sloped fits;
+                                      validated as-is, see detector note)
     hs_top         bearish reversal   head & shoulders top, mirrored
     bull_flag      bullish continuation  strong pole, shallow ordered
                                       retrace, close at a new high
@@ -166,8 +167,10 @@ def _detect_hs(o, h, l, c, atr, cfg, bullish: bool) -> Optional[Dict]:
             return None
         if abs(s1 - s2) > cfg.cp_hs_shoulder_tol_atr * atr:
             return None
-        # Neckline: the LOWER of the two interim peaks — the price must
-        # clear both, so this is the conservative horizontal trigger.
+        # Neckline: the LOWER of the two interim peaks — the EARLIER of
+        # the two horizontal triggers (max() would demand clearing both
+        # peaks). The whole validated CP book (M5 BUY-only) ran on this
+        # trigger — do not flip to max() without full re-validation.
         p1 = float(h[i1 : i2 + 1].max())
         p2 = float(h[i2 : i3 + 1].max())
         neckline = min(p1, p2)

config.json.example

@@ -0,0 +1,36 @@
+{
+  "telegram": {
+    "token": "123456789:REPLACE_WITH_YOUR_BOT_TOKEN",
+    "chat_id": "REPLACE_WITH_OWNER_CHAT_ID",
+    "channel_id": "-100REPLACE_WITH_CHANNEL_ID_OR_EMPTY_STRING",
+    "alert_min_quality": "C"
+  },
+  "data_providers": {
+    "active": "oanda",
+    "twelve_data": {
+      "api_key": "REPLACE_WITH_YOUR_TWELVE_DATA_API_KEY"
+    },
+    "oanda": {},
+    "tiingo": {
+      "api_key": "REPLACE_WITH_YOUR_TIINGO_API_KEY_OR_EMPTY"
+    }
+  },
+  "scanning": {
+    "pairs": ["XAU/USD"],
+    "timeframes": ["M5", "M15", "H1", "H4"]
+  },
+  "pairs": {
+    "XAU/USD": {
+      "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21]
+    }
+  },
+  "oanda": {
+    "account_id": "REPLACE_WITH_ACCOUNT_ID",
+    "api_token": "REPLACE_WITH_API_TOKEN",
+    "base_url": "https://api-fxpractice.oanda.com",
+    "lot_size": 1,
+    "quality_gate": "C",
+    "max_hold_bars": 300
+  },
+  "log_level": "INFO"
+}

config.py

@@ -27,14 +27,6 @@ class SignalStrength(Enum):
     STRONG_SELL = -2
 
 
-class MarketSession(Enum):
-    SYDNEY = "sydney"
-    TOKYO = "tokyo"
-    LONDON = "london"
-    NEW_YORK = "new_york"
-    OVERLAP_LONDON_NY = "london_ny_overlap"
-
-
 # ── Supported pairs ──────────────────────────────────────────────
 XAUUSD = "XAU/USD"
 
@@ -42,7 +34,7 @@ XAUUSD = "XAU/USD"
 # default_spread: round-trip cost per trade in price units, charged by
 # the backtester when --spread is not given.
 PAIR_INFO = {
-    XAUUSD: {"display": "XAU/USD", "pip_value": 0.10, "is_jpy_pair": False,
+    XAUUSD: {"display": "XAU/USD", "pip_value": 0.10,
              "price_precision": 2, "default_spread": 0.30},
 }
 
@@ -102,25 +94,8 @@ class TechnicalConfig:
 @dataclass
 class SignalConfig:
     """Signal scoring & filtering."""
-    min_confidence: float = 55.0
-
-    weights: Dict[str, float] = field(default_factory=lambda: {
-        "ema_trend": 1.0,
-        "rsi": 1.5,
-        "bollinger": 1.5,
-        "macd": 1.5,
-        "keltner": 1.5,
-        "roc": 1.5,
-        "candlestick": 1.0,
-        "divergence": 1.5,
-    })
-
     mtf_agreement_threshold: int = 2  # of 3 timeframes
-    cooldown_bars: int = 5
 
-    default_risk_pct: float = 1.0
-    default_rr_ratio: float = 2.0
-    atr_sl_multiplier: float = 1.5
     min_rr_ratio: float = 0.5
     # TP tightening (May 2026) was tuned on the pre-spread backtester.
     # Re-tested 2026-07-08 with $0.30 spread + gap fills: 0.8 nearly
@@ -199,25 +174,14 @@ class NewsConfig:
 
 @dataclass
 class XauStrategyConfig:
-    """XAU/USD strategy config.
-    Legacy gates (H4 RSI, H1 MACD, M15 score) removed in favor of
-    new momentum indicators (ROC, Keltner squeeze, ADX slope).
-    """
+    """XAU/USD strategy config."""
     enabled: bool = True
-    rsi_entry_threshold: float = 45.0
     tp_atr_multiplier: float = 1.0
     sl_atr_multiplier: float = 2.0
-    min_ema200_tfs: int = 2
-    allow_generic_trades: bool = True
     blocked_hours_utc: List[int] = field(default_factory=lambda: [0, 1, 2, 10, 11, 13, 16, 21])
     weekend_filter: bool = True
     min_atr_threshold: float = 1.0
     v2_sl_atr_multiplier: float = 2.0
-    buy_threshold_boost: int = 0
-    m5_confirmation_bars: int = 0
-    h4_rsi_gate: bool = False
-    h1_macd_gate: bool = False
-    rsi_zone: tuple = (40.0, 65.0)
     allowed_weekdays: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 6])  # Mon-Fri + Sun (Sydney open)
     sell_min_quality: List[str] = field(default_factory=lambda: ["A+", "A", "B"])
     # Volatility regimes in which SELL setups are allowed (low/normal-vol
@@ -343,8 +307,6 @@ class PlatformConfig:
 
     data_lookback_days: int = 60
 
-    twelve_data_api_key: str = "f563b22ffdfd4924955decbe1a8c8701"
-
     refresh_interval: int = 300
 
     scan_pairs: List[str] = field(
@@ -353,7 +315,6 @@ class PlatformConfig:
     scan_timeframes: List[Timeframe] = field(
         default_factory=lambda: [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4]
     )
-    candle_buffer_seconds: int = 10
     ws_cooldown_seconds: int = 300
 
     log_level: str = "INFO"

data_provider.py

@@ -1,386 +1,24 @@
 """
-ForexBot2 — Data Provider (Twelve Data SDK)
-==========================================================
-Fetches and validates OHLCV via the official `twelvedata` Python SDK.
+ForexBot2 — Data Provider (back-compat shim)
+=============================================
+The provider implementations live in the `providers/` package:
+
+    providers/base.py         BaseDataProvider (shared contract + behavior)
+    providers/twelve_data.py  TwelveDataProvider (SDK fetch, throttle, WS)
+    providers/oanda.py        OandaDataProvider (v20 REST, real spread)
+
+`ForexDataProvider` is the historical name of the Twelve Data provider and
+is kept as an alias so existing imports keep working.
 """
 
-import logging
-import os
-import threading
-import time
-from collections import deque
-from typing import Dict, Optional
+from providers import (  # noqa: F401
+    BaseDataProvider,
+    DataIntegrityError,
+    PROVIDER_REGISTRY,
+    TwelveDataProvider,
+    make_data_provider,
+    _is_forex_weekend,
+)
 
-import pandas as pd
-from twelvedata import TDClient
-
-from config import PlatformConfig, Timeframe, XAUUSD, PAIR_INFO
-
-logger = logging.getLogger(__name__)
-
-
-def _is_forex_weekend(now=None) -> bool:
-    """Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC."""
-    if now is None:
-        now = pd.Timestamp.now(tz="UTC")
-    wd = now.weekday()
-    hour = now.hour
-    if wd == 4 and hour >= 22:
-        return True
-    if wd == 5:
-        return True
-    if wd == 6 and hour < 22:
-        return True
-    return False
-
-
-class DataIntegrityError(Exception):
-    """Raised when data fails validation checks."""
-    pass
-
-
-class ForexDataProvider:
-    """Twelve Data SDK client + on-disk parquet cache."""
-
-    TD_INTERVALS = {
-        Timeframe.M1:  "1min",
-        Timeframe.M5:  "5min",
-        Timeframe.M15: "15min",
-        Timeframe.M30: "30min",
-        Timeframe.H1:  "1h",
-        Timeframe.H4:  "4h",
-        Timeframe.D1:  "1day",
-        Timeframe.W1:  "1week",
-    }
-
-    BARS_PER_DAY = {
-        Timeframe.M1: 1440, Timeframe.M5: 288, Timeframe.M15: 96,
-        Timeframe.M30: 48, Timeframe.H1: 24, Timeframe.H4: 6,
-        Timeframe.D1: 1, Timeframe.W1: 0.143,
-    }
-
-    CANDLE_SECONDS = {
-        Timeframe.M1: 60, Timeframe.M5: 300, Timeframe.M15: 900,
-        Timeframe.M30: 1800, Timeframe.H1: 3600, Timeframe.H4: 14400,
-        Timeframe.D1: 86400, Timeframe.W1: 604800,
-    }
-
-    RATE_LIMIT = 55
-    RATE_WINDOW = 60.0
-
-    def __init__(self, config: PlatformConfig):
-        self.config = config
-        self._api_key = (
-            os.environ.get("TWELVE_DATA_API_KEY")
-            or config.twelve_data_api_key
-        )
-        self._cache: Dict[str, pd.DataFrame] = {}
-        self._cache_timestamps: Dict[str, float] = {}
-        # The bot shares one provider across a thread pool; cache eviction
-        # and the throttle deque are check-then-act and need locking.
-        self._cache_lock = threading.Lock()
-        self._throttle_lock = threading.Lock()
-        self._request_times: deque = deque()
-        self._td: Optional[TDClient] = (
-            TDClient(apikey=self._api_key) if self._api_key else None
-        )
-
-    @staticmethod
-    def _to_td_symbol(symbol: str) -> str:
-        if "/" in symbol:
-            return symbol
-        clean = symbol.replace("=X", "")
-        return f"{clean[:3]}/{clean[3:]}"
-
-    # ── Public API ─────────────────────────────────────────────────
-
-    def fetch_ohlcv(
-        self,
-        symbol: str = XAUUSD,
-        timeframe: Timeframe = Timeframe.H1,
-        lookback_days: Optional[int] = None,
-    ) -> pd.DataFrame:
-        """Fetch OHLCV with in-memory caching for live mode."""
-        if lookback_days is None:
-            lookback_days = self.config.data_lookback_days
-
-        cache_key = f"{symbol}_{timeframe.value}"
-        now = time.time()
-
-        cache_ttl = self.CANDLE_SECONDS.get(timeframe, self.config.refresh_interval)
-        with self._cache_lock:
-            cached = self._cache.get(cache_key)
-            cached_at = self._cache_timestamps.get(cache_key)
-        if (
-            cached is not None
-            and cached_at is not None
-            and (now - cached_at) < cache_ttl
-            and not cached.empty
-        ):
-            if (self._is_data_fresh(cached, symbol, timeframe)
-                    and self._has_latest_closed_bar(cached, timeframe)):
-                return cached
-            logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
-            with self._cache_lock:
-                self._cache.pop(cache_key, None)
-                self._cache_timestamps.pop(cache_key, None)
-
-        outputsize = max(
-            300,
-            min(int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20, 5000),
-        )
-        df = self._fetch_twelve_data(symbol, timeframe, outputsize)
-        df = self._drop_forming_candle(df, timeframe)
-        df = self._validate_and_clean(df, symbol, timeframe)
-
-        if not self._is_data_fresh(df, symbol, timeframe):
-            logger.warning(
-                f"STALE DATA from API: {symbol} {timeframe.value} — "
-                f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
-            )
-
-        with self._cache_lock:
-            self._cache[cache_key] = df
-            self._cache_timestamps[cache_key] = now
-        return df
-
-    def _has_latest_closed_bar(self, df: pd.DataFrame, timeframe: Timeframe) -> bool:
-        """Cached data must include the most recent fully closed bar.
-
-        The TTL alone is wall-clock based: a snapshot fetched mid-candle is
-        still within TTL at the next candle boundary but is missing the bar
-        that just closed, so every boundary scan would analyze — and run the
-        bar-close session check against — the previous bar.
-        """
-        now = pd.Timestamp.now(tz="UTC")
-        if _is_forex_weekend(now):
-            return True
-        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
-        dur = pd.Timedelta(seconds=candle_seconds)
-        expected_label = now.floor(dur) - dur
-        return df.index[-1] >= expected_label
-
-    def _drop_forming_candle(self, df: pd.DataFrame, timeframe: Timeframe) -> pd.DataFrame:
-        """Drop the still-forming last candle so analysis only sees closed bars.
-
-        Twelve Data's time_series includes the current partial candle; the
-        scanner fires seconds after the boundary, so without this guard every
-        indicator reads a seconds-old stub bar and signals repaint.
-        """
-        if df is None or df.empty:
-            return df
-        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
-        now = pd.Timestamp.now(tz="UTC")
-        if df.index[-1] + pd.Timedelta(seconds=candle_seconds) > now:
-            df = df.iloc[:-1]
-        return df
-
-    def _is_data_fresh(self, df: pd.DataFrame, symbol: str, timeframe: Timeframe) -> bool:
-        """Check if the last candle is within acceptable age for this timeframe."""
-        if df is None or df.empty:
-            return False
-
-        now = pd.Timestamp.now(tz="UTC")
-
-        if _is_forex_weekend(now):
-            return True
-
-        last_bar = df.index[-1]
-        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
-        max_age_seconds = candle_seconds * 3
-        age_seconds = (now - last_bar).total_seconds()
-
-        if age_seconds > max_age_seconds:
-            logger.warning(
-                f"{symbol} {timeframe.value}: last bar {last_bar.strftime('%Y-%m-%d %H:%M')} "
-                f"is {age_seconds / 60:.0f}m old (limit: {max_age_seconds / 60:.0f}m)"
-            )
-            return False
-
-        return True
-
-    def clear_cache(self) -> int:
-        """Wipe all cached data. Returns number of entries cleared."""
-        count = len(self._cache)
-        self._cache.clear()
-        self._cache_timestamps.clear()
-        logger.info(f"Cache cleared: {count} entries wiped")
-        return count
-
-    def get_latest_price(self, symbol: str = XAUUSD) -> Optional[float]:
-        try:
-            df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
-            if df is not None and not df.empty:
-                return float(df["Close"].iloc[-1])
-        except Exception:
-            pass
-        return None
-
-    def get_spread_estimate(self, symbol: str = XAUUSD) -> float:
-        try:
-            df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
-            if df is not None and not df.empty:
-                return float((df["High"] - df["Low"]).median())
-        except Exception:
-            pass
-        info = PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])
-        return info["pip_value"]
-
-    def get_pair_info(self, symbol: str = XAUUSD) -> Dict:
-        info = PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])
-        return {"symbol": symbol, **info}
-
-    # ── Private ────────────────────────────────────────────────────
-
-    def _throttle(self, credits: int = 1) -> None:
-        """Pro tier: 55 credits / minute. Block until budget allows."""
-        while True:
-            with self._throttle_lock:
-                now = time.time()
-                while self._request_times and now - self._request_times[0] >= self.RATE_WINDOW:
-                    self._request_times.popleft()
-
-                available = self.RATE_LIMIT - len(self._request_times)
-                if available >= credits:
-                    for _ in range(credits):
-                        self._request_times.append(time.time())
-                    return
-
-                used = len(self._request_times)
-                wait = (self._request_times[0] + self.RATE_WINDOW) - now + 0.2
-            logger.info(
-                f"Rate limit: {used}/{self.RATE_LIMIT} credits used "
-                f"— waiting {wait:.1f}s..."
-            )
-            time.sleep(wait)
-
-    def _fetch_twelve_data(
-        self,
-        symbol: str,
-        timeframe: Timeframe,
-        outputsize: int,
-        start_date: Optional[str] = None,
-        end_date: Optional[str] = None,
-    ) -> pd.DataFrame:
-        if self._td is None:
-            raise DataIntegrityError(
-                "No Twelve Data API key. Set TWELVE_DATA_API_KEY env var "
-                "or config.twelve_data_api_key."
-            )
-
-        td_symbol = self._to_td_symbol(symbol)
-        interval = self.TD_INTERVALS[timeframe]
-
-        self._throttle(credits=1)
-
-        kwargs = {
-            "symbol": td_symbol,
-            "interval": interval,
-            "outputsize": min(outputsize, 5000),
-            "timezone": "UTC",
-        }
-        if start_date:
-            kwargs["start_date"] = start_date
-        if end_date:
-            kwargs["end_date"] = end_date
-
-        try:
-            df = self._td.time_series(**kwargs).as_pandas()
-        except Exception as e:
-            raise DataIntegrityError(
-                f"Twelve Data fetch failed for {td_symbol}: {e}"
-            ) from e
-
-        if df is None or df.empty:
-            raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
-
-        # SDK returns lowercase columns; engine expects capitalised.
-        df = df.rename(columns={
-            "open": "Open", "high": "High", "low": "Low",
-            "close": "Close", "volume": "Volume",
-        })
-        for col in ("Open", "High", "Low", "Close", "Volume"):
-            if col in df.columns:
-                df[col] = pd.to_numeric(df[col], errors="coerce")
-        if "Volume" not in df.columns:
-            df["Volume"] = 0.0
-
-        # SDK returns newest-first; engine + indicators want oldest-first.
-        df = df.sort_index()
-        if df.index.tz is None:
-            df.index = df.index.tz_localize("UTC")
-        df.index.name = "Datetime"
-
-        logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
-        return df
-
-    def _validate_and_clean(
-        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
-    ) -> pd.DataFrame:
-        if df is None or df.empty:
-            raise DataIntegrityError(f"Empty dataset for {symbol}")
-
-        original_len = len(df)
-
-        df = df.dropna(subset=["Open", "High", "Low", "Close"], how="all")
-        df = df.ffill(limit=3)
-        df = df.dropna(subset=["Close"])
-
-        df = df.sort_index()
-        df = df[~df.index.duplicated(keep="last")]
-
-        now = pd.Timestamp.utcnow()
-        future_mask = df.index > now + pd.Timedelta(minutes=5)
-        if future_mask.any():
-            logger.warning(f"{symbol}: Dropping {future_mask.sum()} future timestamps")
-            df = df[~future_mask]
-
-        invalid_ohlc = (
-            (df["High"] < df["Low"]) |
-            (df["High"] < df["Open"]) |
-            (df["High"] < df["Close"]) |
-            (df["Low"] > df["Open"]) |
-            (df["Low"] > df["Close"])
-        )
-        if invalid_ohlc.any():
-            logger.warning(f"{symbol}: Fixing {invalid_ohlc.sum()} OHLC-inconsistent bars")
-            for idx in df[invalid_ohlc].index:
-                vals = [df.loc[idx, "Open"], df.loc[idx, "Close"]]
-                df.loc[idx, "High"] = max(vals + [df.loc[idx, "High"]])
-                df.loc[idx, "Low"]  = min(vals + [df.loc[idx, "Low"]])
-
-        returns = df["Close"].pct_change()
-        std_ret = returns.std()
-        if std_ret > 0:
-            dev = returns - returns.mean()
-            outlier_mask = dev.abs() > 10 * std_ret
-            # A spike bar makes the NEXT bar's return look extreme too (it is
-            # measured against the spike). If the previous bar is flagged and
-            # this bar reverses direction, it is the legitimate snap-back —
-            # keep it, or every spike would punch a two-bar hole in the data.
-            recovery = (
-                outlier_mask
-                & outlier_mask.shift(1, fill_value=False)
-                & (dev * dev.shift(1) < 0)
-            )
-            outlier_mask = outlier_mask & ~recovery
-            if outlier_mask.any():
-                logger.warning(f"{symbol}: Removing {outlier_mask.sum()} extreme outlier bars")
-                df = df[~outlier_mask]
-
-        min_required = max(
-            self.config.technical.ema_trend,
-            200,
-        ) + 10
-
-        if len(df) < min_required:
-            logger.warning(
-                f"{symbol} {timeframe.value}: Only {len(df)} bars "
-                f"(need {min_required}). Some indicators will be partial."
-            )
-
-        cleaned = original_len - len(df)
-        if cleaned > 0:
-            logger.info(f"{symbol}: Cleaned {cleaned}/{original_len} bars")
-
-        return df
+# Historical name for the Twelve Data provider.
+ForexDataProvider = TwelveDataProvider

docs/superpowers/plans/2026-07-21-centralized-config.md

@@ -0,0 +1,980 @@
+# Centralized config.json Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Move deployment settings (Telegram credentials, data-provider API keys/selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert quality boot default, log level) into a gitignored `config.json` loaded once at startup, and delete the hardcoded credentials from source.
+
+**Architecture:** A new `app_config.py` module loads and validates `config.json` into an `AppConfig` dataclass (cached singleton), with resolver functions applying the precedence **env var → config.json → code default**. The pair factory calls `apply_pair_overrides()` on every fresh `PlatformConfig` *before* its validated per-timeframe deltas, so the bot, CLI, and backtester all pick up the overlay identically (live == backtest parity by construction). Spec: `docs/superpowers/specs/2026-07-21-centralized-config-design.md`.
+
+**Tech Stack:** Python 3.12 stdlib only (`json`, `os`, `dataclasses`). No new dependencies.
+
+## Global Constraints
+
+- **Zero signal-logic change.** No indicator weight, threshold, gate, or geometry value may change. With no `config.json`, or one holding the example defaults, every produced `PlatformConfig` must be byte-identical (`dataclasses.asdict` equality) to today's factory output — pinned by a test.
+- The test suite is `python test_all.py` — a custom harness (`@test("name")` decorator, tests execute at import time, exit code non-zero on failure). It is NOT pytest; there is no single-test selection. "Run the test" always means `python test_all.py` and checking the named line in the output. Baseline before this work: **184 passed, 0 failed**.
+- The suite must be hermetic: it must pass identically whether or not the developer's machine has a `config.json`, and regardless of `TELEGRAM_*`/`TWELVE_DATA_API_KEY` env vars.
+- Secrets get **no code default**. After Task 4 the only places credentials exist are `config.json` (gitignored) and env vars.
+- `config.json.example` (already committed at repo root) is the schema reference. `config.json` is already in `.gitignore` — never commit it.
+- Repo root: `/home/harwinder/Development/ForexBot2`. Use `venv/bin/python` if bare `python` lacks pandas/numpy.
+- Every commit message ends with:
+  ```
+  Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
+  Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
+  ```
+- Work on branch `centralized-config` (create from `main` at start of Task 1: `git checkout -b centralized-config`).
+
+---
+
+### Task 1: `app_config.py` — loader, validation, resolvers
+
+**Files:**
+- Create: `app_config.py`
+- Modify: `test_all.py` (new Module 13 before the SUMMARY block, ~line 2495; hermetic header after line 19)
+- Test: `test_all.py` (same file — this repo keeps all tests in one suite)
+
+**Interfaces:**
+- Consumes: `config.PAIR_INFO`, `config.Timeframe`, `config.XAUUSD`, `config.PlatformConfig` (all existing).
+- Produces (later tasks rely on these exact names):
+  - `class AppConfigError(Exception)`
+  - `@dataclass class AppConfig` — fields: `telegram_token: str = ""`, `telegram_chat_id: str = ""`, `telegram_channel_id: str = ""`, `alert_min_quality: str = "C"`, `active_provider: str = "twelve_data"`, `provider_settings: Dict[str, Dict[str, Any]] = {}`, `scan_pairs: List[str] = [XAUUSD]`, `scan_timeframes: List[Timeframe] = [M5, M15, H1, H4]`, `pair_overrides: Dict[str, Dict[str, Any]] = {}`, `log_level: str = "INFO"`
+  - `load_app_config(path: str = CONFIG_PATH) -> AppConfig`
+  - `get_app_config() -> AppConfig` (cached), `set_app_config(app: Optional[AppConfig]) -> None` (test hook; `None` re-enables lazy load)
+  - `telegram_token(app=None) -> str`, `telegram_chat_id(app=None) -> str`, `telegram_channel_id(app=None) -> str`, `provider_api_key(app=None) -> str`
+  - `apply_pair_overrides(cfg: PlatformConfig, app: Optional[AppConfig] = None) -> PlatformConfig` (implemented here, wired in Task 2)
+
+- [ ] **Step 1: Create branch**
+
+```bash
+cd /home/harwinder/Development/ForexBot2 && git checkout -b centralized-config
+```
+
+- [ ] **Step 2: Add the hermetic test header and Module 13 failing tests to `test_all.py`**
+
+In `test_all.py`, directly after `sys.path.insert(0, os.path.dirname(__file__))` (line 19), add:
+
+```python
+# ── Hermetic app-config: the suite must not depend on the developer's
+#    local config.json or TELEGRAM_*/TWELVE_DATA_API_KEY env vars. The
+#    dummy TD key mirrors the pre-config.json behavior where a key was
+#    always present (TDClient built, no network at construction).
+import json
+import app_config
+for _k in ("TELEGRAM_TOKEN", "TELEGRAM_CHAT_ID", "TELEGRAM_CHANNEL_ID",
+           "TWELVE_DATA_API_KEY"):
+    os.environ.pop(_k, None)
+_TEST_APP = app_config.AppConfig(
+    provider_settings={"twelve_data": {"api_key": "test-key"}}
+)
+app_config.set_app_config(_TEST_APP)
+```
+
+Then insert Module 13 immediately BEFORE the `#  SUMMARY` comment block near the end of the file (search for `SUMMARY`):
+
+```python
+# ══════════════════════════════════════════════════════════════════════
+#  MODULE 13: app_config.py (config.json overlay)
+# ══════════════════════════════════════════════════════════════════════
+print("\n═══ Module 13: app_config.py ═══")
+
+import tempfile
+from app_config import (
+    AppConfig, AppConfigError, load_app_config, set_app_config,
+    telegram_token, telegram_chat_id, telegram_channel_id,
+    provider_api_key, apply_pair_overrides,
+)
+
+
+def _write_json(payload) -> str:
+    fh = tempfile.NamedTemporaryFile(
+        "w", suffix=".json", delete=False, encoding="utf-8")
+    if isinstance(payload, str):
+        fh.write(payload)
+    else:
+        json.dump(payload, fh)
+    fh.close()
+    return fh.name
+
+
+_FULL_CFG = {
+    "telegram": {"token": "t0k", "chat_id": "111", "channel_id": "-100222",
+                 "alert_min_quality": "B"},
+    "data_providers": {"active": "twelve_data",
+                       "twelve_data": {"api_key": "json-key"}},
+    "scanning": {"pairs": ["XAU/USD"], "timeframes": ["M15", "H1"]},
+    "pairs": {"XAU/USD": {"blocked_hours_utc": [3, 4]}},
+    "log_level": "WARNING",
+}
+
+
+@test("app_config: valid config.json maps every field")
+def _():
+    app = load_app_config(_write_json(_FULL_CFG))
+    assert app.telegram_token == "t0k"
+    assert app.telegram_chat_id == "111"
+    assert app.telegram_channel_id == "-100222"
+    assert app.alert_min_quality == "B"
+    assert app.active_provider == "twelve_data"
+    assert app.provider_settings["twelve_data"]["api_key"] == "json-key"
+    assert app.scan_pairs == [XAUUSD]
+    assert app.scan_timeframes == [Timeframe.M15, Timeframe.H1]
+    assert app.pair_overrides[XAUUSD]["blocked_hours_utc"] == [3, 4]
+    assert app.log_level == "WARNING"
+
+
+@test("app_config: missing file returns pure defaults")
+def _():
+    app = load_app_config("/nonexistent/config.json")
+    assert app == AppConfig(), "absent config.json must yield pure defaults"
+
+
+@test("app_config: malformed JSON raises AppConfigError")
+def _():
+    path = _write_json("{not json")
+    try:
+        load_app_config(path)
+        assert False, "should have raised"
+    except AppConfigError as e:
+        assert "not valid JSON" in str(e)
+
+
+@test("app_config: unknown keys raise AppConfigError")
+def _():
+    for payload in (
+        {"telgram": {}},                                   # top-level typo
+        {"telegram": {"tokn": "x"}},                       # section typo
+        {"pairs": {"XAU/USD": {"blocked_hours": [1]}}},    # pair-block typo
+        {"scanning": {"pair": ["XAU/USD"]}},               # scanning typo
+    ):
+        try:
+            load_app_config(_write_json(payload))
+            assert False, f"should have raised for {payload}"
+        except AppConfigError as e:
+            assert "unknown key" in str(e).lower()
+
+
+@test("app_config: invalid values raise AppConfigError")
+def _():
+    bad_payloads = [
+        {"pairs": {"XAU/USD": {"blocked_hours_utc": [24]}}},      # hour range
+        {"pairs": {"XAU/USD": {"blocked_hours_utc": "3"}}},       # not a list
+        {"pairs": {"EUR/USD": {"blocked_hours_utc": [3]}}},       # unknown pair
+        {"scanning": {"pairs": ["EUR/USD"]}},                     # unknown pair
+        {"scanning": {"timeframes": ["M7"]}},                     # bad TF name
+        {"scanning": {"timeframes": ["W1"]}},                     # unscannable TF
+        {"data_providers": {"active": "polygon",
+                            "twelve_data": {"api_key": "k"}}},    # no block
+        {"telegram": {"alert_min_quality": "Z"}},                 # bad quality
+        {"log_level": "CHATTY"},                                  # bad level
+    ]
+    for payload in bad_payloads:
+        try:
+            load_app_config(_write_json(payload))
+            assert False, f"should have raised for {payload}"
+        except AppConfigError:
+            pass
+
+
+@test("app_config: env precedence env → config.json → default")
+def _():
+    app = load_app_config(_write_json(_FULL_CFG))
+    with patch.dict(os.environ, {}, clear=True):
+        assert telegram_token(app) == "t0k"
+        assert telegram_chat_id(app) == "111"
+        assert telegram_channel_id(app) == "-100222"
+        assert provider_api_key(app) == "json-key"
+    with patch.dict(os.environ, {"TELEGRAM_TOKEN": "env-tok",
+                                 "TELEGRAM_CHAT_ID": "999",
+                                 "TWELVE_DATA_API_KEY": "env-key"}):
+        assert telegram_token(app) == "env-tok"
+        assert telegram_chat_id(app) == "999"
+        assert provider_api_key(app) == "env-key"
+    # channel: env set-but-empty means "disable feed" and must beat JSON
+    with patch.dict(os.environ, {"TELEGRAM_CHANNEL_ID": ""}):
+        assert telegram_channel_id(app) == ""
+```
+
+- [ ] **Step 3: Run to verify the new tests fail (import error)**
+
+```bash
+python test_all.py 2>&1 | head -5
+```
+
+Expected: `ModuleNotFoundError: No module named 'app_config'` (the hermetic header imports it before any test runs).
+
+- [ ] **Step 4: Write `app_config.py`**
+
+Create `app_config.py` with exactly this content:
+
+```python
+"""
+app_config.py — deployment-settings overlay (config.json)
+=========================================================
+
+Loads the gitignored config.json at the repo root ONCE per process.
+Deployment surface ONLY: Telegram credentials, data-provider selection
+and API keys, scanned pairs/timeframes, per-pair base blocked hours,
+alert-quality boot default, log level. Signal logic, weights,
+thresholds and gates stay in config.py / pair_signals.
+
+Precedence for every setting: env var → config.json → code default.
+Secrets have NO code default — with neither env nor config.json the
+bot refuses to start (telegram_bot.main) and data fetches fail with a
+named error (data_provider).
+
+Parity contract (test-enforced): with no config.json, or one holding
+the config.json.example defaults, every produced PlatformConfig is
+byte-identical to the pure-code defaults. apply_pair_overrides() runs
+in the pair factory BEFORE the validated per-timeframe deltas, so the
+bot, CLI and backtester all see the same overlay (live == backtest).
+
+Spec: docs/superpowers/specs/2026-07-21-centralized-config-design.md
+"""
+
+import json
+import logging
+import os
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional
+
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+logger = logging.getLogger(__name__)
+
+CONFIG_PATH = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "config.json")
+
+_VALID_QUALITIES = ("A+", "A", "B", "C")
+_VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
+# Timeframes the bot's candle-boundary scheduler can run (W1 has no loop).
+_SCANNABLE_TFS = ("M5", "M15", "M30", "H1", "H4", "D1")
+
+_KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level"}
+_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
+_KNOWN_SCANNING = {"pairs", "timeframes"}
+_KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
+
+
+class AppConfigError(Exception):
+    """config.json is present but invalid — fail fast at boot."""
+
+
+@dataclass
+class AppConfig:
+    telegram_token: str = ""
+    telegram_chat_id: str = ""
+    telegram_channel_id: str = ""          # "" = channel feed disabled
+    alert_min_quality: str = "C"
+    active_provider: str = "twelve_data"
+    provider_settings: Dict[str, Dict[str, Any]] = field(default_factory=dict)
+    scan_pairs: List[str] = field(default_factory=lambda: [XAUUSD])
+    scan_timeframes: List[Timeframe] = field(default_factory=lambda: [
+        Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4])
+    pair_overrides: Dict[str, Dict[str, Any]] = field(default_factory=dict)
+    log_level: str = "INFO"
+
+
+def _check_keys(section: Dict[str, Any], known: set, where: str) -> None:
+    unknown = sorted(set(section) - known)
+    if unknown:
+        raise AppConfigError(
+            f"config.json: unknown key(s) {unknown} in {where}; "
+            f"known keys: {sorted(known)}")
+
+
+def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
+    """Parse + validate config.json. Missing file → pure defaults."""
+    app = AppConfig()
+    if not os.path.exists(path):
+        logger.warning(
+            "config.json not found at %s — running on code defaults "
+            "(secrets must come from environment variables)", path)
+        return app
+    try:
+        with open(path, encoding="utf-8") as fh:
+            raw = json.load(fh)
+    except json.JSONDecodeError as e:
+        raise AppConfigError(f"config.json is not valid JSON: {e}") from e
+    if not isinstance(raw, dict):
+        raise AppConfigError("config.json top level must be a JSON object")
+    _check_keys(raw, _KNOWN_TOP, "top level")
+
+    tg = raw.get("telegram", {})
+    _check_keys(tg, _KNOWN_TELEGRAM, '"telegram"')
+    app.telegram_token = str(tg.get("token", app.telegram_token))
+    app.telegram_chat_id = str(tg.get("chat_id", app.telegram_chat_id))
+    app.telegram_channel_id = str(
+        tg.get("channel_id", app.telegram_channel_id))
+    quality = tg.get("alert_min_quality", app.alert_min_quality)
+    if quality not in _VALID_QUALITIES:
+        raise AppConfigError(
+            f'config.json: telegram.alert_min_quality "{quality}" '
+            f"not in {_VALID_QUALITIES}")
+    app.alert_min_quality = quality
+
+    dp = raw.get("data_providers")
+    if dp is not None:
+        if not isinstance(dp, dict):
+            raise AppConfigError('config.json: "data_providers" must be an object')
+        app.active_provider = str(dp.get("active", app.active_provider))
+        app.provider_settings = {}
+        for name, block in dp.items():
+            if name == "active":
+                continue
+            if not isinstance(block, dict):
+                raise AppConfigError(
+                    f'config.json: data_providers.{name} must be an object')
+            app.provider_settings[name] = dict(block)
+        if app.active_provider not in app.provider_settings:
+            raise AppConfigError(
+                f'config.json: data_providers.active "{app.active_provider}" '
+                "has no matching provider block")
+
+    scanning = raw.get("scanning", {})
+    _check_keys(scanning, _KNOWN_SCANNING, '"scanning"')
+    if "pairs" in scanning:
+        pairs = scanning["pairs"]
+        bad = [p for p in pairs if p not in PAIR_INFO]
+        if bad:
+            raise AppConfigError(
+                f"config.json: unknown pair(s) {bad} in scanning.pairs; "
+                f"known: {sorted(PAIR_INFO)}")
+        app.scan_pairs = list(pairs)
+    if "timeframes" in scanning:
+        names = scanning["timeframes"]
+        bad = [n for n in names if n not in _SCANNABLE_TFS]
+        if bad:
+            raise AppConfigError(
+                f"config.json: unsupported timeframe(s) {bad} in "
+                f"scanning.timeframes; supported: {list(_SCANNABLE_TFS)}")
+        app.scan_timeframes = [Timeframe[n] for n in names]
+
+    pair_section = raw.get("pairs", {})
+    for symbol, overrides in pair_section.items():
+        if symbol not in PAIR_INFO:
+            raise AppConfigError(
+                f'config.json: unknown pair "{symbol}" in pairs; '
+                f"known: {sorted(PAIR_INFO)}")
+        _check_keys(overrides, _KNOWN_PAIR_KEYS, f'pairs."{symbol}"')
+        if "blocked_hours_utc" in overrides:
+            hours = overrides["blocked_hours_utc"]
+            valid = isinstance(hours, list) and all(
+                isinstance(h, int) and not isinstance(h, bool)
+                and 0 <= h <= 23 for h in hours)
+            if not valid:
+                raise AppConfigError(
+                    f'config.json: pairs."{symbol}".blocked_hours_utc must '
+                    "be a list of integers in 0-23")
+    app.pair_overrides = {s: dict(o) for s, o in pair_section.items()}
+
+    level = raw.get("log_level", app.log_level)
+    if level not in _VALID_LOG_LEVELS:
+        raise AppConfigError(
+            f'config.json: log_level "{level}" not in {_VALID_LOG_LEVELS}')
+    app.log_level = level
+    return app
+
+
+# ── Process-wide singleton ────────────────────────────────────────────
+
+_cached: Optional[AppConfig] = None
+
+
+def get_app_config() -> AppConfig:
+    global _cached
+    if _cached is None:
+        _cached = load_app_config()
+    return _cached
+
+
+def set_app_config(app: Optional[AppConfig]) -> None:
+    """Override the process-wide AppConfig (tests/experiments).
+    Pass None to re-enable lazy loading from CONFIG_PATH."""
+    global _cached
+    _cached = app
+
+
+# ── Resolution helpers: env var → config.json → default ───────────────
+
+def telegram_token(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("TELEGRAM_TOKEN") or app.telegram_token
+
+
+def telegram_chat_id(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("TELEGRAM_CHAT_ID") or app.telegram_chat_id
+
+
+def telegram_channel_id(app: Optional[AppConfig] = None) -> str:
+    """Env set-but-empty ("") means "disable the feed" and beats JSON,
+    so this checks None-ness, not truthiness."""
+    app = app or get_app_config()
+    env = os.environ.get("TELEGRAM_CHANNEL_ID")
+    return env if env is not None else app.telegram_channel_id
+
+
+def provider_api_key(app: Optional[AppConfig] = None) -> str:
+    """API key for the ACTIVE provider. The env override is per-provider
+    (TWELVE_DATA_API_KEY applies only when twelve_data is active)."""
+    app = app or get_app_config()
+    if app.active_provider == "twelve_data":
+        env = os.environ.get("TWELVE_DATA_API_KEY")
+        if env:
+            return env
+    block = app.provider_settings.get(app.active_provider, {})
+    return str(block.get("api_key", ""))
+
+
+# ── PlatformConfig overlay ────────────────────────────────────────────
+
+def apply_pair_overrides(
+    cfg: PlatformConfig, app: Optional[AppConfig] = None
+) -> PlatformConfig:
+    """Stamp deployment settings from config.json onto a PlatformConfig.
+
+    Called by the pair factories BEFORE their validated per-timeframe
+    deltas, so e.g. the M5 hour-9 append lands on top of the JSON base.
+    The pair's base blocked_hours_utc is set on BOTH mirrored copies
+    (xau_strategy + backtest) so they cannot drift.
+    """
+    app = app or get_app_config()
+    overrides = app.pair_overrides.get(cfg.symbol, {})
+    if "blocked_hours_utc" in overrides:
+        hours = list(overrides["blocked_hours_utc"])
+        cfg.xau_strategy.blocked_hours_utc = hours
+        cfg.backtest.blocked_hours_utc = list(hours)
+    cfg.scan_pairs = list(app.scan_pairs)
+    cfg.scan_timeframes = list(app.scan_timeframes)
+    return cfg
+```
+
+- [ ] **Step 5: Run tests, verify Module 13 passes and total is 190**
+
+```bash
+python test_all.py 2>&1 | tail -25
+```
+
+Expected: `RESULTS: 190 passed, 0 failed, 190 total` (184 baseline + 6 new). All six `app_config:` lines show `✓`.
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add app_config.py test_all.py
+git commit -m "Add app_config.py: config.json loader, validation, env-precedence resolvers"
+```
+
+---
+
+### Task 2: Factory overlay wiring + parity pins
+
+**Files:**
+- Modify: `pair_signals/xauusd.py:81-90` (make_config), `test_all.py` (Module 9 equivalence test ~line 1851; Module 13 additions)
+
+**Interfaces:**
+- Consumes: `app_config.apply_pair_overrides`, `app_config.set_app_config`, `app_config.AppConfig`, `app_config.load_app_config` (Task 1).
+- Produces: `make_config` now applies the overlay — every consumer (bot, CLI, backtester) inherits it with no further change. Test symbol `_TEST_APP` (Task 1) is the restore target for any test that swaps the singleton.
+
+- [ ] **Step 1: Add failing tests (Module 13 additions)**
+
+Append to Module 13 in `test_all.py`:
+
+```python
+@test("app_config: apply_pair_overrides stamps both blocked-hours mirrors")
+def _():
+    app = AppConfig(pair_overrides={XAUUSD: {"blocked_hours_utc": [3, 4]}},
+                    scan_pairs=[XAUUSD],
+                    scan_timeframes=[Timeframe.M15])
+    cfg = apply_pair_overrides(PlatformConfig(), app)
+    assert cfg.xau_strategy.blocked_hours_utc == [3, 4]
+    assert cfg.backtest.blocked_hours_utc == [3, 4]
+    assert cfg.scan_timeframes == [Timeframe.M15]
+    # the two mirrors must be independent lists, not aliases
+    cfg.xau_strategy.blocked_hours_utc.append(5)
+    assert cfg.backtest.blocked_hours_utc == [3, 4]
+
+
+@test("app_config: M5 factory appends hour 9 on top of a JSON base")
+def _():
+    from pair_signals import PAIR_PRODUCERS
+    app = AppConfig(pair_overrides={XAUUSD: {"blocked_hours_utc": [3, 4]}},
+                    provider_settings={"twelve_data": {"api_key": "test-key"}})
+    set_app_config(app)
+    try:
+        cfg = PAIR_PRODUCERS[XAUUSD](Timeframe.M5, None)
+    finally:
+        set_app_config(_TEST_APP)
+    assert cfg.xau_strategy.blocked_hours_utc == [3, 4, 9], \
+        "validated M5 hour-9 delta must land on top of the JSON base"
+    assert cfg.backtest.blocked_hours_utc == [3, 4], \
+        "mirror carries the base only (per-TF delta is XAU-strategy-side)"
+
+
+@test("app_config: config.json.example is valid and behavior-neutral")
+def _():
+    import dataclasses
+    from pair_signals import PAIR_PRODUCERS
+    example = load_app_config(
+        os.path.join(os.path.dirname(__file__), "config.json.example"))
+    for tf in (Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4):
+        baseline = PAIR_PRODUCERS[XAUUSD](tf, None)   # under _TEST_APP
+        set_app_config(example)
+        try:
+            with_example = PAIR_PRODUCERS[XAUUSD](tf, None)
+        finally:
+            set_app_config(_TEST_APP)
+        assert dataclasses.asdict(baseline) == dataclasses.asdict(with_example), \
+            f"example config.json must be byte-identical to defaults on {tf}"
+```
+
+- [ ] **Step 2: Run to verify the new tests fail**
+
+```bash
+python test_all.py 2>&1 | grep "app_config: M5 factory"
+```
+
+Expected: `✗ app_config: M5 factory appends hour 9 on top of a JSON base` (the factory doesn't apply the overlay yet, so blocked hours are the defaults `[0,1,2,10,11,13,16,21,9]`, not `[3,4,9]`).
+
+- [ ] **Step 3: Wire the overlay into the factory**
+
+In `pair_signals/xauusd.py`, change the import line and the top of `make_config`:
+
+```python
+from typing import Optional, List
+from config import PlatformConfig, Timeframe
+from app_config import apply_pair_overrides
+```
+
+and in `make_config`, immediately after `config = PlatformConfig()`:
+
+```python
+    config = PlatformConfig()
+    # Deployment overlay (config.json) BEFORE the validated per-TF
+    # deltas below, so e.g. the M5 hour-9 append lands on the JSON base.
+    apply_pair_overrides(config)
+    config.primary_timeframe = primary_tf
+```
+
+- [ ] **Step 4: Update the Module 9 equivalence test**
+
+In `test_all.py` (~line 1851, `"xauusd.make_config equals inline PlatformConfig construction"`), the inline construction must apply the same overlay in the same order. After the `inline_cfg = PlatformConfig(...)` construction and BEFORE the per-TF override lines, add:
+
+```python
+        app_config.apply_pair_overrides(inline_cfg)
+```
+
+(`app_config` is already imported at the top of `test_all.py` from Task 1. Under `_TEST_APP` this is a no-op on blocked hours, but it keeps the equivalence definition exact: factory ≡ inline + overlay + per-TF deltas, whatever the active AppConfig.)
+
+- [ ] **Step 5: Run tests, verify 193 passing**
+
+```bash
+python test_all.py 2>&1 | tail -25
+```
+
+Expected: `RESULTS: 193 passed, 0 failed, 193 total`. Watch specifically for `✓ xauusd.make_config equals inline PlatformConfig construction` (Module 9) and the three new Module 13 lines.
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add pair_signals/xauusd.py test_all.py
+git commit -m "Wire config.json overlay into the pair factory (before per-TF deltas)"
+```
+
+---
+
+### Task 3: Data provider — key from app_config, registry, delete hardcoded key
+
+**Files:**
+- Modify: `config.py:346` (delete field), `data_provider.py:72-76` (key resolution), `data_provider.py:265-269` (error message), `data_provider.py` end (registry), `signal_aggregator.py:107`, `backtest/gpu_backtest.py:1682`, `test_all.py` (Module 13 additions)
+
+**Interfaces:**
+- Consumes: `app_config.provider_api_key`, `app_config.get_app_config`, `app_config.AppConfigError` (Task 1).
+- Produces: `data_provider.PROVIDER_REGISTRY: Dict[str, type]` and `data_provider.make_data_provider(config: PlatformConfig) -> ForexDataProvider` — Task 4 uses `make_data_provider` in `telegram_bot.py`.
+
+- [ ] **Step 1: Add failing tests (Module 13 additions)**
+
+```python
+@test("data_provider: api key resolves env → config.json (no code default)")
+def _():
+    from data_provider import ForexDataProvider
+    app = AppConfig(provider_settings={"twelve_data": {"api_key": "json-key"}})
+    set_app_config(app)
+    try:
+        with patch.dict(os.environ, {}, clear=True):
+            dp = ForexDataProvider(PlatformConfig())
+            assert dp._api_key == "json-key"
+        with patch.dict(os.environ, {"TWELVE_DATA_API_KEY": "env-key"}):
+            dp = ForexDataProvider(PlatformConfig())
+            assert dp._api_key == "env-key"
+        set_app_config(AppConfig())        # no key anywhere
+        with patch.dict(os.environ, {}, clear=True):
+            dp = ForexDataProvider(PlatformConfig())
+            assert dp._api_key is None and dp._td is None
+    finally:
+        set_app_config(_TEST_APP)
+
+
+@test("data_provider: make_data_provider dispatches on active provider")
+def _():
+    from data_provider import ForexDataProvider, make_data_provider
+    assert isinstance(make_data_provider(PlatformConfig()), ForexDataProvider)
+    set_app_config(AppConfig(active_provider="polygon",
+                             provider_settings={"polygon": {"api_key": "k"}}))
+    try:
+        make_data_provider(PlatformConfig())
+        assert False, "unregistered provider must raise"
+    except AppConfigError as e:
+        assert "polygon" in str(e)
+    finally:
+        set_app_config(_TEST_APP)
+
+
+@test("config.py: hardcoded twelve_data_api_key field is gone")
+def _():
+    assert not hasattr(PlatformConfig(), "twelve_data_api_key"), \
+        "credentials must not live in committed source"
+```
+
+- [ ] **Step 2: Run to verify they fail**
+
+```bash
+python test_all.py 2>&1 | grep -A1 "hardcoded twelve_data"
+```
+
+Expected: `✗ config.py: hardcoded twelve_data_api_key field is gone` (plus the make_data_provider import failure).
+
+- [ ] **Step 3: Implement**
+
+`config.py`: delete line 346 (`twelve_data_api_key: str = "<TWELVE_DATA_API_KEY>"`).
+
+`data_provider.py` — add to the local imports:
+
+```python
+from app_config import AppConfigError, get_app_config, provider_api_key
+```
+
+In `ForexDataProvider.__init__` replace the key resolution:
+
+```python
+        self.config = config
+        self._api_key = provider_api_key() or None
+```
+
+In `_fetch_twelve_data`, update the no-key error:
+
+```python
+        if self._td is None:
+            raise DataIntegrityError(
+                "No data-provider API key. Set the TWELVE_DATA_API_KEY env "
+                "var or data_providers.twelve_data.api_key in config.json."
+            )
+```
+
+At the bottom of `data_provider.py` (after the class):
+
+```python
+# ── Provider registry ─────────────────────────────────────────────────
+# config.json data_providers.active selects from here. Adding a provider
+# later = one adapter class + one entry + one config block.
+PROVIDER_REGISTRY: Dict[str, type] = {"twelve_data": ForexDataProvider}
+
+
+def make_data_provider(config: PlatformConfig) -> ForexDataProvider:
+    """Build the ACTIVE data provider (config.json: data_providers.active)."""
+    name = get_app_config().active_provider
+    cls = PROVIDER_REGISTRY.get(name)
+    if cls is None:
+        raise AppConfigError(
+            f'Unknown data provider "{name}"; '
+            f"registered: {sorted(PROVIDER_REGISTRY)}")
+    return cls(config)
+```
+
+`signal_aggregator.py` line 18 + 107 — import and use the registry:
+
+```python
+from data_provider import ForexDataProvider, make_data_provider
+```
+```python
+        self.data_provider = make_data_provider(config)
+```
+
+`backtest/gpu_backtest.py` line 1682 (and its import of `ForexDataProvider`):
+
+```python
+    provider = make_data_provider(platform_cfg)
+```
+
+- [ ] **Step 4: Run tests, verify 196 passing**
+
+```bash
+python test_all.py 2>&1 | tail -25
+```
+
+Expected: `RESULTS: 196 passed, 0 failed, 196 total`. If any Module 2 test fails on a missing key, it means it constructed `ForexDataProvider` outside `_TEST_APP` scope — fix the test to run under `_TEST_APP` (which supplies the dummy key), never by re-adding a code default.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add config.py data_provider.py signal_aggregator.py backtest/gpu_backtest.py test_all.py
+git commit -m "Data provider: key from config.json/env, provider registry, drop hardcoded key"
+```
+
+---
+
+### Task 4: telegram_bot wiring + startup guard
+
+**Files:**
+- Modify: `telegram_bot.py` (credentials block ~93-105, logging ~88, SCAN_PAIRS ~113, alert quality ~478, WS key ~611-614, `_AUTOSCAN_LOOPS` ~797-802, scheduler log ~817, `_get_shared_dp` ~195, `main()` ~1261), `test_all.py` (Module 13 addition)
+
+**Interfaces:**
+- Consumes: `app_config.get_app_config`, resolvers, `data_provider.make_data_provider` (Tasks 1+3).
+- Produces: nothing consumed later; this is the final wiring task.
+
+- [ ] **Step 1: Add failing test (Module 13 addition)**
+
+```python
+@test("telegram_bot: credentials, quality, pairs, loops from app_config")
+def _():
+    import importlib
+    import telegram_bot
+    app = AppConfig(telegram_token="json-tok", telegram_chat_id="42",
+                    telegram_channel_id="", alert_min_quality="B",
+                    scan_timeframes=[Timeframe.M15, Timeframe.H1],
+                    provider_settings={"twelve_data": {"api_key": "k"}})
+    set_app_config(app)
+    try:
+        with patch.dict(os.environ, {}, clear=True):
+            tb = importlib.reload(telegram_bot)
+            assert tb.TOKEN == "json-tok"
+            assert tb.OWNER_ID == 42
+            assert tb.CHANNEL_ID is None, '"" channel id must disable the feed'
+            assert tb._alert_min_quality == "B"
+            assert [tf for tf, _ in tb._AUTOSCAN_LOOPS] == \
+                [Timeframe.M15, Timeframe.H1]
+            assert tb._AUTOSCAN_LOOPS == [(Timeframe.M15, 15),
+                                          (Timeframe.H1, 60)]
+    finally:
+        set_app_config(_TEST_APP)
+        importlib.reload(telegram_bot)
+
+
+@test("telegram_bot: main() refuses to start without credentials")
+def _():
+    import telegram_bot
+    if telegram_bot.TOKEN:      # only meaningful in the hermetic no-cred env
+        return
+    try:
+        telegram_bot.main()
+        assert False, "main() must refuse to start without token/chat_id"
+    except SystemExit as e:
+        assert "credentials" in str(e)
+```
+
+- [ ] **Step 2: Run to verify failure**
+
+```bash
+python test_all.py 2>&1 | grep "telegram_bot: credentials"
+```
+
+Expected: `✗ telegram_bot: credentials, quality, pairs, loops from app_config` (TOKEN still comes from the hardcoded fallback, not "json-tok").
+
+- [ ] **Step 3: Implement the wiring**
+
+`telegram_bot.py` — add to the local imports (after the `config` import block):
+
+```python
+from app_config import (
+    get_app_config, telegram_token, telegram_chat_id, telegram_channel_id,
+)
+from data_provider import ForexDataProvider, make_data_provider
+```
+
+Immediately after the imports (before the logging block), add:
+
+```python
+_APP = get_app_config()
+```
+
+Change the `logging.basicConfig` line (~88) to honor the configured level:
+
+```python
+logging.basicConfig(level=getattr(logging, _APP.log_level, logging.INFO),
+                    handlers=[_console_handler, _file_handler])
+```
+
+Replace the credentials block (lines ~93-105) — keep the existing comments about the owner lock and subscriber channel, change only the assignments:
+
+```python
+TOKEN = telegram_token()
+CHAT_ID = telegram_chat_id()
+
+# ... (owner-lock comment unchanged)
+OWNER_ID: Optional[int] = int(CHAT_ID) if CHAT_ID else None
+
+# ... (subscriber-channel comment unchanged)
+_CHANNEL_ID_RAW = telegram_channel_id()
+CHANNEL_ID: Optional[int] = int(_CHANNEL_ID_RAW) if _CHANNEL_ID_RAW else None
+```
+
+`SCAN_PAIRS` (~113):
+
+```python
+SCAN_PAIRS = list(_APP.scan_pairs)
+```
+
+`_get_shared_dp` (~195):
+
+```python
+        _shared_dp = make_data_provider(PlatformConfig())
+```
+
+Alert quality boot default (~478):
+
+```python
+_alert_min_quality = _APP.alert_min_quality
+```
+
+WS key (~611-614) — `provider_api_key` needs adding to the `app_config` import list:
+
+```python
+_cfg = PlatformConfig()
+_WS_API_KEY = provider_api_key()
+_WS_URL = f"wss://ws.twelvedata.com/v1/quotes/price?apikey={_WS_API_KEY}"
+_WS_COOLDOWN = _cfg.ws_cooldown_seconds
+```
+
+`_AUTOSCAN_LOOPS` (~797-802) — derive from configured timeframes:
+
+```python
+_TF_INTERVAL_MINUTES = {
+    Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.M30: 30,
+    Timeframe.H1: 60, Timeframe.H4: 240, Timeframe.D1: 1440,
+}
+_AUTOSCAN_LOOPS = [
+    (tf, _TF_INTERVAL_MINUTES[tf]) for tf in _APP.scan_timeframes
+]
+```
+
+Scheduler log line (~817):
+
+```python
+        logger.info("Candle-boundary scheduler started (%s)",
+                    "/".join(tf.name for tf, _ in _AUTOSCAN_LOOPS))
+```
+
+(The manual `/scan` buttons — `_TF_BUTTON_LABELS` — deliberately stay hardcoded M5/M15/H1/H4: they are the owner's on-demand UI, not the automated feed. Documented decision, not an omission.)
+
+`main()` (~1261) — first lines of the function body:
+
+```python
+def main() -> None:
+    if not TOKEN or not CHAT_ID:
+        raise SystemExit(
+            "telegram_bot: missing Telegram credentials. Set telegram.token "
+            "and telegram.chat_id in config.json, or the TELEGRAM_TOKEN and "
+            "TELEGRAM_CHAT_ID environment variables.")
+```
+
+- [ ] **Step 4: Run tests, verify 198 passing**
+
+```bash
+python test_all.py 2>&1 | tail -25
+```
+
+Expected: `RESULTS: 198 passed, 0 failed, 198 total`. If Module 7 tests break on `OWNER_ID` being `None`, they were relying on the hardcoded chat id — fix by having those tests set `telegram_bot.OWNER_ID`/`CHAT_ID` explicitly in their setup, never by re-adding fallbacks.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add telegram_bot.py test_all.py
+git commit -m "telegram_bot: credentials/quality/scan-loops from config.json, startup guard"
+```
+
+---
+
+### Task 5: Migration, docs, final verification
+
+**Files:**
+- Create: `config.json` (LOCAL ONLY — gitignored, never committed)
+- Modify: `CLAUDE.md` (repo map, env-var table, config section, test count)
+
+**Interfaces:**
+- Consumes: everything above.
+- Produces: the deployed machine's real `config.json`; updated authoritative docs.
+
+- [ ] **Step 1: Create the local config.json with the current live values**
+
+These are the exact values being deleted from source (from pre-change `telegram_bot.py:93-105` and `config.py:346`), so the running bot behaves identically:
+
+```bash
+cat > /home/harwinder/Development/ForexBot2/config.json <<'EOF'
+{
+  "telegram": {
+    "token": "<TELEGRAM_BOT_TOKEN>",
+    "chat_id": "<OWNER_CHAT_ID>",
+    "channel_id": "<CHANNEL_ID>",
+    "alert_min_quality": "C"
+  },
+  "data_providers": {
+    "active": "twelve_data",
+    "twelve_data": { "api_key": "<TWELVE_DATA_API_KEY>" }
+  },
+  "scanning": {
+    "pairs": ["XAU/USD"],
+    "timeframes": ["M5", "M15", "H1", "H4"]
+  },
+  "pairs": {
+    "XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
+  },
+  "log_level": "INFO"
+}
+EOF
+git status --short   # config.json must NOT appear (gitignored)
+```
+
+(Real values were scrubbed from this doc 2026-07-21 — take them from the previously deployed telegram_bot.py/config.py in git history, and rotate both credentials after merge.)
+
+- [ ] **Step 2: Boot smoke test (no network needed)**
+
+```bash
+python - <<'EOF'
+from app_config import get_app_config, telegram_token, provider_api_key, apply_pair_overrides
+from pair_signals import PAIR_PRODUCERS
+from config import Timeframe, XAUUSD
+app = get_app_config()
+assert telegram_token(app), "token missing"
+assert provider_api_key(app), "api key missing"
+cfg = PAIR_PRODUCERS[XAUUSD](Timeframe.M5, None)
+assert cfg.xau_strategy.blocked_hours_utc == [0, 1, 2, 10, 11, 13, 16, 21, 9]
+print("boot smoke OK — live config.json loads and factory output unchanged")
+EOF
+```
+
+Expected: `boot smoke OK — live config.json loads and factory output unchanged`.
+
+- [ ] **Step 3: Update CLAUDE.md**
+
+- Repository-map table: add rows for `app_config.py` ("config.json loader/validation, env-precedence resolvers, `apply_pair_overrides` deployment overlay") and `config.json.example` ("committed schema template; real gitignored config.json holds secrets + deployment settings").
+- Environment-variables table: note precedence "env var → config.json → none"; state that hardcoded fallbacks were REMOVED 2026-07-21 and the bot refuses to boot without credentials.
+- `PlatformConfig` section: delete the `twelve_data_api_key` mention ("hardcoded fallback"); add one line: "Deployment settings (credentials, scan pairs/TFs, per-pair base blocked hours, alert quality, log level) overlay from config.json via `app_config.apply_pair_overrides`, applied in the pair factory BEFORE per-TF deltas."
+- `pair_signals` section: mention the overlay call order in `make_config`.
+- Test-suite line: update "184 tests in 12 modules" → "198 tests in 13 modules" (verify against the actual final count from Step 4 before writing).
+
+- [ ] **Step 4: Full-suite final verification**
+
+```bash
+python test_all.py 2>&1 | tail -15
+```
+
+Expected: `RESULTS: 198 passed, 0 failed, 198 total` (use the real number if it differs, and make CLAUDE.md match).
+
+- [ ] **Step 5: Commit docs**
+
+```bash
+git add CLAUDE.md
+git commit -m "Docs: config.json deployment overlay (CLAUDE.md repo map, env table, test count)"
+```
+
+- [ ] **Step 6: Merge/PR decision**
+
+Implementation complete → follow superpowers:finishing-a-development-branch (options: merge `centralized-config` to main, or push the branch to `origin`/`gitea` for review). After merge, remind the user: **rotate the Telegram token and Twelve Data key** — the old values remain in git history on both remotes.

docs/superpowers/specs/2026-07-21-centralized-config-design.md

@@ -0,0 +1,154 @@
+# Centralized Configuration (config.json) — Design
+
+**Date:** 2026-07-21
+**Status:** Approved (user confirmed all four scoping decisions)
+
+## Goal
+
+Move deployment settings — Telegram credentials, data-provider API keys and
+selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert
+quality boot default, log level — into a single gitignored `config.json` at the
+repo root, loaded once at startup.
+
+**Hard constraint:** zero change to signal logic, indicator weights, thresholds,
+or gates. With an unedited `config.json` (or none at all), every produced
+`PlatformConfig` must be byte-identical to today's factory output. This is
+test-enforced (see Testing).
+
+## Scoping decisions (user-confirmed 2026-07-21)
+
+1. **Secrets live in config.json** (gitignored). A committed
+   `config.json.example` carries placeholders. Env vars still override.
+   The hardcoded credential fallbacks in `config.py` and `telegram_bot.py`
+   are **deleted** — on a machine with neither config.json nor env vars, the
+   bot refuses to start with an explicit error instead of silently using
+   committed credentials. (Recommend rotating the Telegram token and Twelve
+   Data key after merge; both are exposed in git history.)
+2. **Multi-provider is structure only**: a `data_providers` section with named
+   provider blocks and an `active` selector. Only Twelve Data is implemented.
+   No failover logic (nothing to fail over to yet).
+3. **Blocked hours: base list per pair only.** The validated per-timeframe
+   deltas (M5 appends hour 9; M15 SELL-block 15/17 UTC) stay in
+   `pair_signals/xauusd.py` next to their validation evidence. The overlay is
+   applied **before** those deltas and updates **both** mirrored copies
+   (`xau_strategy.blocked_hours_utc` and `backtest.blocked_hours_utc`) so they
+   cannot drift.
+4. **Extra keys included:** scan timeframes, alert min quality (boot default),
+   log level. Everything else (indicator params, RR geometry, M5 gates,
+   CP knobs, news config) deliberately stays code-only.
+
+## File schema
+
+`config.json` (gitignored; `config.json.example` committed with placeholders
+for secrets and today's tuned values for everything else):
+
+```json
+{
+  "telegram": {
+    "token": "<bot token>",
+    "chat_id": "<owner chat id>",
+    "channel_id": "<-100… or empty string to disable the channel feed>",
+    "alert_min_quality": "C"
+  },
+  "data_providers": {
+    "active": "twelve_data",
+    "twelve_data": { "api_key": "<key>" }
+  },
+  "scanning": {
+    "pairs": ["XAU/USD"],
+    "timeframes": ["M5", "M15", "H1", "H4"]
+  },
+  "pairs": {
+    "XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
+  },
+  "log_level": "INFO"
+}
+```
+
+Timeframe strings use the `Timeframe` enum names (`M5`, `M15`, `H1`, `H4`, …).
+Pair keys use canonical symbols (`XAU/USD`).
+
+## Architecture
+
+### New module: `app_config.py`
+
+Single responsibility: read + validate `config.json`, expose typed accessors,
+apply the pair-level overlay. No signal knowledge.
+
+- `AppConfig` dataclass: `telegram` (token, chat_id, channel_id,
+  alert_min_quality), `data_providers` (active name + per-provider dicts),
+  `scan_pairs`, `scan_timeframes`, `pair_overrides`
+  (symbol → {blocked_hours_utc}), `log_level`.
+- `load_app_config(path=REPO_ROOT/"config.json") -> AppConfig` — module-level
+  cached (loaded once per process). Missing file → `AppConfig` of pure
+  defaults + one warning log. Malformed JSON, unknown top-level/section keys,
+  unknown pair symbols, hours outside 0–23, unknown timeframe names, unknown
+  `active` provider → raise `AppConfigError` with a named message (fail fast
+  at boot).
+- Precedence, applied inside the accessors:
+  **env var → config.json → code default**. Secrets (`TELEGRAM_TOKEN`,
+  `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`) have **no
+  code default** after this change.
+- `apply_pair_overrides(cfg: PlatformConfig, app: AppConfig) -> PlatformConfig`
+  — sets the pair's base `blocked_hours_utc` on **both**
+  `cfg.xau_strategy.blocked_hours_utc` and `cfg.backtest.blocked_hours_utc`,
+  plus `cfg.scan_pairs` / `cfg.scan_timeframes`. Called by the pair factory
+  (see below). A no-op when the pair has no override block.
+
+### Wiring points (all consumers)
+
+| Consumer | Change |
+|---|---|
+| `pair_signals/xauusd.py` | `make_config` calls `apply_pair_overrides` **before** the per-TF deltas (so M5 still appends hour 9 to whatever base the JSON sets, and mirrors stay in sync). |
+| `data_provider.py` | API key resolution becomes `app_config` accessor for the **active** provider (env `TWELVE_DATA_API_KEY` still wins). `config.twelve_data_api_key` field and its hardcoded default are deleted. Missing key keeps today's behavior: provider constructed with `_td=None`, errors on first fetch. |
+| `telegram_bot.py` | `TOKEN`, `CHAT_ID`, `CHANNEL_ID` come from `app_config` (hardcoded fallbacks deleted; missing token/chat_id → explicit startup error). `_alert_min_quality` boot default and `logging` level come from `app_config`. `SCAN_PAIRS` (today built from a bare `PlatformConfig()`) reads `app_config.scan_pairs` directly; `apply_pair_overrides` also stamps `cfg.scan_pairs`/`cfg.scan_timeframes` so the dataclass field stays truthful for any other reader. |
+| `main.py` | No structural change — it uses the factory, which now applies the overlay. |
+| `backtest/gpu_backtest.py` | No structural change — it uses the factory too, so an edited base blocked-hours list flows into replays identically to live (parity by construction). Data fetches pick up the provider key via `data_provider.py`. |
+| `config.py` | `twelve_data_api_key` field removed. Everything else untouched. |
+
+### Multi-provider structure
+
+`data_provider.py` gains a minimal registry:
+`PROVIDER_REGISTRY = {"twelve_data": ForexDataProvider}`. `active` selects
+from it; an unknown name fails at load. Adding a provider later = one config
+block + one adapter class registered here. No shared base class extracted now
+(YAGNI — with one provider there is nothing to abstract over yet).
+
+## Error handling summary
+
+| Condition | Behavior |
+|---|---|
+| No config.json | Defaults + warning. Secrets must come from env or startup fails (bot) / fetch fails (provider). |
+| Malformed JSON / bad values / unknown keys | `AppConfigError` at load, process exits with named message. |
+| Missing Telegram token or chat_id at bot boot | Explicit startup error naming the missing key and both places it can be set. |
+| Missing data-provider key | Same as today: provider builds, first fetch errors. |
+
+## Migration
+
+- Create the local `config.json` populated with the **current live values**
+  (the ones being deleted from source) so the running bot keeps working
+  unchanged. `.gitignore` gains `config.json`.
+- Commit `config.json.example` with placeholders.
+- CLAUDE.md: update the env-var table, repository map, and config section.
+- Post-merge recommendation (user action): rotate the Telegram token and
+  Twelve Data key, since both are in git history.
+
+## Testing (`test_all.py`, new module)
+
+1. **Parity pin (the critical one):** a config.json containing the example
+   defaults, and no config.json at all, both produce factory output identical
+   to today's (full dataclass equality across all four TFs).
+2. Overlay mapping: JSON blocked hours land on **both** mirrored fields; M5
+   factory still appends hour 9 on top of a custom base.
+3. Precedence: env var beats config.json beats default, per key.
+4. Validation: malformed JSON, bad hour, unknown pair/timeframe/provider each
+   raise `AppConfigError`.
+5. Missing-file behavior: defaults + no crash.
+6. Existing factory-equivalence test stays green (equivalence is now defined
+   over factory-with-overlay vs inline-config-with-overlay).
+
+## Out of scope
+
+Hot reload (config is read once at boot), a second concrete provider,
+failover, moving any tuned strategy value beyond the base blocked-hours list,
+news/indicator/geometry settings in JSON.

engine.py

@@ -13,7 +13,7 @@ Design philosophy:
 
 import logging
 from dataclasses import dataclass, field
-from typing import Dict, List, Optional, Tuple
+from typing import Dict, List, Tuple
 
 import numpy as np
 import pandas as pd
@@ -638,15 +638,10 @@ class TechnicalEngine:
                 signal = SignalStrength.SELL
                 conf = min(70, 50 + roc_abs * 10)
                 desc = f"ROC {roc_val:.2f}% — Falling momentum"
-        elif roc_prev <= 0 < roc_val:
-            signal = SignalStrength.BUY
-            conf = 65
-            desc = f"ROC crossed above zero (+{roc_val:.2f}%)"
-        elif roc_prev >= 0 > roc_val:
-            signal = SignalStrength.SELL
-            conf = 65
-            desc = f"ROC crossed below zero ({roc_val:.2f}%)"
         else:
+            # Zero-crosses land in the branches above by definition (a
+            # cross up is positive and rising, a cross down negative and
+            # falling) — only flat/fading momentum reaches here.
             signal = SignalStrength.NEUTRAL
             conf = 30
             desc = f"ROC {roc_val:+.2f}% — No clear momentum"

logs/telegram.log.2026-07-14

@@ -1,21 +0,0 @@
-01:51:29 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 05:51 is 4320m old (limit: 180m)
-01:52:02 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 05:52 is 4320m old (limit: 180m)
-02:42:59 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 06:42 is 4320m old (limit: 180m)
-03:30:24 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 07:30 is 4320m old (limit: 180m)
-03:31:32 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 07:31 is 4320m old (limit: 180m)
-11:30:20 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 15:30 is 4320m old (limit: 180m)
-11:41:34 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 15:41 is 4320m old (limit: 180m)
-11:45:46 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 15:45 is 4320m old (limit: 180m)
-11:56:38 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 15:56 is 4320m old (limit: 180m)
-11:57:11 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 15:57 is 4320m old (limit: 180m)
-11:59:52 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 15:59 is 4320m old (limit: 180m)
-12:10:05 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 16:10 is 4320m old (limit: 180m)
-13:05:37 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 17:05 is 4320m old (limit: 180m)
-13:14:25 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 17:14 is 4320m old (limit: 180m)
-13:38:50 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 17:38 is 4320m old (limit: 180m)
-13:45:14 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 17:45 is 4320m old (limit: 180m)
-13:51:33 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 17:51 is 4320m old (limit: 180m)
-15:04:41 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 19:04 is 4320m old (limit: 180m)
-15:04:50 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 19:04 is 4320m old (limit: 180m)
-15:05:17 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 19:05 is 4320m old (limit: 180m)
-15:06:10 [WARNING] data_provider: BTC/USD 1h: last bar 2026-07-11 19:06 is 4320m old (limit: 180m)

main.py

@@ -13,7 +13,6 @@ import json
 import logging
 import sys
 from datetime import datetime, timezone
-from typing import Dict, List
 
 # Force UTF-8 output on Windows so box-drawing chars render
 if hasattr(sys.stdout, "reconfigure"):
@@ -222,6 +221,9 @@ def report_to_dict(report: SignalReport) -> dict:
             "risk_reward": ts.risk_reward,
             "risk_pips": round(ts.risk_pips, 1),
             "reward_pips": round(ts.reward_pips, 1),
+            "strategy": ts.strategy,
+            "trail_arm_r": ts.trail_arm_r,
+            "trail_dist_r": ts.trail_dist_r,
         }
     if report.primary_analysis:
         d["indicators"] = {
@@ -231,7 +233,7 @@ def report_to_dict(report: SignalReport) -> dict:
                 "description": ind.description,
             }
             for name, ind in report.primary_analysis.indicators.items()
-            if not name.startswith("_")
+            if not name.startswith("_") and ind.weight != 0
         }
         d["key_levels"] = {
             k: round(v, 5) for k, v in report.primary_analysis.key_levels.items()

news_engine.py

@@ -14,7 +14,8 @@ import threading
 import time
 import xml.etree.ElementTree as ET
 from dataclasses import dataclass, field
-from datetime import datetime, timedelta, timezone
+from datetime import datetime, timezone
+from email.utils import parsedate_to_datetime
 from typing import Dict, List, Optional, Tuple
 
 import requests
@@ -281,14 +282,20 @@ class NewsEngine:
         """
         base, quote = self._parse_pair(pair_symbol)
 
-        # Check upcoming high-impact events
+        # Check upcoming high-impact events. Grade from the SOONEST
+        # matching event, not the first listed — an unsorted feed could
+        # otherwise report "caution" for a far event while a nearer one
+        # warrants "danger".
+        soonest = None
         for event in context.events_upcoming_1h:
             if event.is_high_impact and event.currency in (base, quote):
-                mins = event.minutes_until
+                if soonest is None or event.minutes_until < soonest.minutes_until:
+                    soonest = event
+        if soonest is not None:
+            mins = soonest.minutes_until
             if mins <= self.config.pre_event_buffer_minutes:
-                    return "danger", f"{event.currency} {event.event} in {mins:.0f}min"
-                else:
-                    return "caution", f"{event.currency} {event.event} in {mins:.0f}min"
+                return "danger", f"{soonest.currency} {soonest.event} in {mins:.0f}min"
+            return "caution", f"{soonest.currency} {soonest.event} in {mins:.0f}min"
 
         # Check for recent high-impact releases. A surprise print (actual
         # away from forecast) keeps the market unstable longer than an
@@ -650,11 +657,20 @@ class NewsEngine:
         if not date_str:
             return None
 
+        # RFC 822/2822 forms, with or without seconds, numeric offsets AND
+        # named zones (GMT, EST/EDT, PST…). The old strptime "%Z" path
+        # accepted ANY abbreviation but attached no offset, so e.g. an EST
+        # pubDate was silently stamped UTC — a 5-hour error.
+        try:
+            dt = parsedate_to_datetime(date_str.strip())
+            if dt is not None:
+                if dt.tzinfo is None:   # unknown zone name — assume UTC
+                    return dt.replace(tzinfo=timezone.utc)
+                return dt.astimezone(timezone.utc)
+        except (TypeError, ValueError):
+            pass
+
         formats = [
-            "%a, %d %b %Y %H:%M:%S %z",   # RFC 822 with numeric tz offset
-            "%a, %d %b %Y %H:%M:%S %Z",   # RFC 822 with tz abbreviation (GMT/UTC)
-            "%a, %d %b %Y %H:%M %Z",      # RFC 822 without seconds (MyFXBook)
-            "%a, %d %b %Y %H:%M %z",      # RFC 822 without seconds, numeric offset
             "%Y-%m-%dT%H:%M:%S%z",         # ISO 8601 with offset
             "%Y-%m-%d %H:%M:%S",           # ISO 8601 naive (assume UTC)
         ]

oanda_daemon.py

@@ -0,0 +1,775 @@
+"""
+ForexBot2 — Oanda Automated Trading Daemon
+============================================
+Autonomous trading process for XAU/USD on an Oanda practice account.
+Runs candle-boundary scan loops (M5/M15/H1/H4), executes trades via
+the Oanda v20 API, and manages trailing stops at bar close — matching
+the backtested bar-close ratchet model.
+
+Book-parity notes (matching backtest/gpu_backtest.py):
+- No broker-side take-profit: in the trailing model TP1/TP2 are
+  advisory; the only exits are SL (broker), TRAIL (daemon ratchet),
+  and TIMEOUT (max_hold_bars). A hard TP would cap winners the trail
+  lets run and diverge the live book from the backtest.
+- Entry cooldown is 5 bars per timeframe (the backtester's only
+  sequential rule) — not a fingerprint block.
+- The trail ratchets at bar CLOSE only, and each closed bar is
+  processed exactly once (last_bar_ts guard), so weekends and feed
+  lag can't double-count bars or inflate the hold counter.
+
+Divergence from the backtest book (accepted for demo safety): one
+position total. While a trade is open, other signals are logged and
+skipped, so skipped entries can be compared against the book later.
+
+Usage:  python oanda_daemon.py
+Config: env vars OANDA_ACCOUNT_ID, OANDA_API_TOKEN, OANDA_BASE_URL
+        or config.json "oanda" section (env wins).
+"""
+
+import asyncio
+import json
+import logging
+import os
+import signal
+import sys
+import threading
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime, timezone
+from logging.handlers import TimedRotatingFileHandler
+from typing import Dict, Optional
+
+import pandas as pd
+
+from app_config import (
+    get_app_config, oanda_account_id, oanda_api_token, oanda_base_url,
+)
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+from signal_aggregator import SignalAggregator, SignalReport, SignalStrength
+from pair_signals import PAIR_PRODUCERS
+from providers.oanda import OandaDataProvider
+from oanda_executor import OandaExecutor, OandaAPIError
+
+logger = logging.getLogger(__name__)
+
+# ── Constants ──────────────────────────────────────────────────────────
+
+SYMBOL = XAUUSD
+INSTRUMENT = "XAU_USD"
+
+SCAN_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+TF_INTERVAL_MINUTES = {
+    Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.M30: 30,
+    Timeframe.H1: 60, Timeframe.H4: 240, Timeframe.D1: 1440,
+}
+
+SCAN_TIMEFRAMES = [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4]
+
+QUALITY_RANK = {"A+": 4, "A": 3, "B": 2, "C": 1, "D": 0, "F": -1}
+
+# Matches the backtester's --cooldown-bars default (the only sequential
+# rule in the replay): no new entry within 5 bars of the previous entry
+# on the same timeframe.
+COOLDOWN_BARS = 5
+
+# How often the heartbeat confirms the process is alive and the account
+# is still reachable — independent of the candle-boundary loops, so a
+# wedged scan or mid-run credential expiry still surfaces in the log.
+HEARTBEAT_SECONDS = 3600
+
+STATE_FILE = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "oanda_state.json")
+
+# ── Config resolution (env → config.json → default) ──────────────────
+
+def _oanda_setting_int(env_key: str, json_key: str, default: int) -> int:
+    env = os.environ.get(env_key)
+    if env:
+        return int(env)
+    return int(get_app_config().oanda_settings.get(json_key, default))
+
+
+def _oanda_setting_str(env_key: str, json_key: str, default: str) -> str:
+    return (os.environ.get(env_key)
+            or str(get_app_config().oanda_settings.get(json_key, default)))
+
+# ── Logging setup ──────────────────────────────────────────────────────
+
+def _setup_logging(level: str = "INFO") -> None:
+    log_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "logs")
+    os.makedirs(log_dir, exist_ok=True)
+    log_file = os.path.join(log_dir, "daemon.log")
+
+    fmt = logging.Formatter(
+        "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S")
+
+    file_handler = TimedRotatingFileHandler(
+        log_file, when="h", interval=3, backupCount=240, utc=True)
+    file_handler.setFormatter(fmt)
+
+    console_handler = logging.StreamHandler(sys.stdout)
+    console_handler.setFormatter(fmt)
+
+    logging.basicConfig(
+        level=getattr(logging, level.upper(), logging.INFO),
+        handlers=[file_handler, console_handler],
+        force=True)
+
+# ── Boundary timing (same as telegram_bot.py) ─────────────────────────
+
+def _seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
+    now = datetime.now(timezone.utc)
+    total_minutes = now.hour * 60 + now.minute
+    elapsed_minutes = total_minutes % interval_minutes
+    elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
+    interval_seconds = interval_minutes * 60
+    remaining = interval_seconds - elapsed_seconds + buffer
+    if remaining <= buffer:
+        remaining += interval_seconds
+    return remaining
+
+# ── Trade state persistence ───────────────────────────────────────────
+
+def _load_state() -> Dict:
+    if os.path.exists(STATE_FILE):
+        try:
+            with open(STATE_FILE, encoding="utf-8") as fh:
+                return json.load(fh)
+        except (OSError, json.JSONDecodeError):
+            logger.warning("Could not load state file, starting fresh")
+    return {"trades": {}, "last_entries": {}}
+
+
+def _save_state(state: Dict) -> None:
+    try:
+        tmp = STATE_FILE + ".tmp"
+        with open(tmp, "w", encoding="utf-8") as fh:
+            json.dump(state, fh, indent=2)
+        os.replace(tmp, STATE_FILE)
+    except OSError:
+        logger.exception("Failed to save state file")
+
+# ── Trading Daemon ─────────────────────────────────────────────────────
+
+class TradingDaemon:
+
+    def __init__(self):
+        account_id = oanda_account_id()
+        api_token = oanda_api_token()
+        base_url = oanda_base_url()
+        lot_size = _oanda_setting_int("OANDA_LOT_SIZE", "lot_size", 1)
+        self._quality_gate = _oanda_setting_str(
+            "OANDA_QUALITY_GATE", "quality_gate", "C")
+        self._max_hold_bars = _oanda_setting_int(
+            "OANDA_MAX_HOLD_BARS", "max_hold_bars", 300)
+        self._price_precision = PAIR_INFO[SYMBOL]["price_precision"]
+
+        if not account_id or not api_token:
+            msg = (
+                "Missing Oanda credentials. Set OANDA_ACCOUNT_ID and "
+                "OANDA_API_TOKEN env vars or config.json oanda section.")
+            logger.critical(msg)
+            raise SystemExit(msg)
+
+        self._provider = OandaDataProvider(
+            PlatformConfig(), api_token, base_url)
+        self._executor = OandaExecutor(
+            account_id, api_token, base_url, lot_size,
+            price_precision=self._price_precision)
+        self._thread_pool = ThreadPoolExecutor(max_workers=4)
+
+        self._state = _load_state()
+        self._state.setdefault("trades", {})
+        self._state.setdefault("last_entries", {})
+        self._state_lock = threading.Lock()
+        # Serializes the position-check + order-placement critical section
+        # across concurrent TF loops (all four fire at aligned boundaries).
+        self._entry_lock = threading.Lock()
+        self._running = True
+
+    # ── Main entry point ───────────────────────────────────────────
+
+    async def run(self) -> None:
+        loop = asyncio.get_running_loop()
+        loop.set_exception_handler(self._asyncio_exception_handler)
+        for sig in (signal.SIGINT, signal.SIGTERM):
+            loop.add_signal_handler(sig, self._request_shutdown)
+
+        logger.info("=== Oanda Trading Daemon starting ===")
+        logger.info(
+            "Quality gate: %s | Max hold: %d bars | Cooldown: %d bars | "
+            "Timeframes: %s",
+            self._quality_gate, self._max_hold_bars, COOLDOWN_BARS,
+            "/".join(tf.name for tf in SCAN_TIMEFRAMES))
+
+        # Credentials first: with a bad token every later call fails, so
+        # refuse to start before touching positions or spawning loops.
+        self._verify_account_access()
+
+        self._discover_existing_positions()
+
+        tasks = []
+        for tf in SCAN_TIMEFRAMES:
+            interval = TF_INTERVAL_MINUTES[tf]
+            tasks.append(asyncio.create_task(
+                self._candle_loop(tf, interval)))
+        tasks.append(asyncio.create_task(self._heartbeat_loop()))
+
+        logger.info("Candle-boundary loops started")
+        try:
+            await asyncio.gather(*tasks)
+        except asyncio.CancelledError:
+            pass
+        finally:
+            self._shutdown()
+
+    def _verify_account_access(self) -> None:
+        """Boot-time credential check. A 4xx here means the token/account
+        id is wrong — that never fixes itself, so refuse to start rather
+        than run a daemon that can't place or manage trades. A connection
+        failure or 5xx is a transient outage: log it and continue, since
+        broker-side stops already protect any open position and the
+        hourly heartbeat will keep reporting account health."""
+        try:
+            self._executor.get_account_summary()
+        except OandaAPIError as e:
+            if 400 <= e.status_code < 500:
+                logger.critical(
+                    "Oanda rejected the account credentials (HTTP %d): "
+                    "%s — refusing to start", e.status_code, e)
+                raise SystemExit(
+                    f"Oanda rejected the account credentials "
+                    f"(HTTP {e.status_code}): {e}")
+            logger.error(
+                "Failed to fetch account summary at boot (HTTP %d): %s",
+                e.status_code, e)
+
+    def _asyncio_exception_handler(self, loop, context) -> None:
+        exc = context.get("exception")
+        if exc is not None:
+            logger.critical(
+                "Uncaught asyncio exception: %s",
+                context.get("message", ""), exc_info=exc)
+        else:
+            logger.critical(
+                "Uncaught asyncio error: %s", context.get("message", ""))
+
+    def _request_shutdown(self) -> None:
+        logger.info("Shutdown requested")
+        self._running = False
+        for task in asyncio.all_tasks():
+            task.cancel()
+
+    def _shutdown(self) -> None:
+        with self._state_lock:
+            _save_state(self._state)
+        self._thread_pool.shutdown(wait=False)
+        logger.info("=== Oanda Trading Daemon stopped ===")
+
+    # ── Position discovery (restart recovery) ──────────────────────
+
+    def _discover_existing_positions(self) -> None:
+        try:
+            open_trades = self._executor.get_open_trades(INSTRUMENT)
+        except OandaAPIError:
+            logger.exception("Failed to query open trades at boot")
+            return
+
+        saved = self._state.get("trades", {})
+        live_ids = {t["id"] for t in open_trades}
+
+        stale = [tid for tid in saved if tid not in live_ids]
+        for tid in stale:
+            logger.info(
+                "Stale trade %s in state file (closed on Oanda), removing", tid)
+            del saved[tid]
+
+        for trade in open_trades:
+            tid = trade["id"]
+            if tid in saved:
+                logger.info(
+                    "Recovered trade %s from state: %s %s @ %.2f, SL %.2f",
+                    tid, saved[tid].get("direction"), INSTRUMENT,
+                    saved[tid].get("entry"), saved[tid].get("current_sl"))
+                continue
+
+            direction = "BUY" if float(trade.get("currentUnits", 0)) > 0 else "SELL"
+            entry = float(trade.get("price", 0))
+            sl_order = trade.get("stopLossOrder", {})
+            sl = float(sl_order.get("price", entry))
+            risk = abs(entry - sl) if sl else 1.0
+
+            saved[tid] = {
+                "tf": "H1",
+                "direction": direction,
+                "entry": entry,
+                "sl_initial": sl,
+                "current_sl": sl,
+                "risk": risk,
+                "peak": entry,
+                "trail_armed": False,
+                "trail_arm_r": 0.6,
+                "trail_dist_r": 0.75,
+                "bars_held": 0,
+                "last_bar_ts": None,
+                "signal_bar_ts": None,
+            }
+            logger.info(
+                "Adopted unknown trade %s: %s @ %.2f, SL %.2f, risk %.2f "
+                "(conservative defaults)",
+                tid, direction, entry, sl, risk)
+
+        self._state["trades"] = saved
+        _save_state(self._state)
+        logger.info(
+            "Position discovery: %d active trade(s)", len(saved))
+
+    # ── Candle-boundary loop ───────────────────────────────────────
+
+    async def _candle_loop(self, tf: Timeframe, interval_minutes: int) -> None:
+        label = tf.value
+        try:
+            while self._running:
+                wait = _seconds_until_boundary(interval_minutes, buffer=10)
+                logger.info("%s loop: next scan in %.0fs", label, wait)
+                await asyncio.sleep(wait)
+                if not self._running:
+                    break
+                logger.info("%s loop: firing scan", label)
+                try:
+                    await self._scan_and_manage(tf)
+                except Exception:
+                    logger.exception("%s loop: cycle failed", label)
+        except asyncio.CancelledError:
+            pass
+        finally:
+            logger.info("%s loop stopped", label)
+
+    async def _scan_and_manage(self, tf: Timeframe) -> None:
+        loop = asyncio.get_running_loop()
+
+        # Trade management FIRST: the SL ratchet must not wait on (or be
+        # starved by) the signal scan — news fetches can take seconds and
+        # a failed scan must never cost an open trade its trail update.
+        try:
+            await loop.run_in_executor(
+                self._thread_pool, self._manage_open_trades, tf)
+        except Exception:
+            logger.exception("%s: trade management failed", tf.value)
+
+        report = await loop.run_in_executor(
+            self._thread_pool, self._run_signal, tf)
+
+        if not self._should_execute(report, tf):
+            return
+
+        await loop.run_in_executor(
+            self._thread_pool, self._execute_trade, report, tf)
+
+    def _manage_open_trades(self, tf: Timeframe) -> None:
+        self._check_closed_trades()
+        self._manage_trailing(tf)
+
+    # ── Heartbeat ─────────────────────────────────────────────────
+
+    def _heartbeat_once(self) -> None:
+        with self._state_lock:
+            n = len(self._state.get("trades", {}))
+        try:
+            self._executor.get_account_summary()
+        except OandaAPIError as e:
+            logger.error("HEARTBEAT: account summary failed: %s", e)
+        # Logged unconditionally — even on a failed summary call the
+        # heartbeat line itself must reach the log, so a wedged process
+        # doesn't look identical to a healthy but silent one.
+        logger.info("HEARTBEAT: alive | tracked trades: %d", n)
+
+    async def _heartbeat_loop(self) -> None:
+        loop = asyncio.get_running_loop()
+        try:
+            while self._running:
+                await asyncio.sleep(HEARTBEAT_SECONDS)
+                if not self._running:
+                    break
+                try:
+                    await loop.run_in_executor(
+                        self._thread_pool, self._heartbeat_once)
+                except Exception:
+                    logger.exception("heartbeat loop: cycle failed")
+        except asyncio.CancelledError:
+            pass
+        finally:
+            logger.info("heartbeat loop stopped")
+
+    # ── Signal pipeline ────────────────────────────────────────────
+
+    def _run_signal(self, tf: Timeframe) -> SignalReport:
+        conf_tfs = SCAN_CONFIRMATIONS.get(
+            tf, [Timeframe.M15, Timeframe.H4])
+        factory = PAIR_PRODUCERS.get(SYMBOL)
+        if factory is not None:
+            config = factory(tf, conf_tfs)
+        else:
+            config = PlatformConfig()
+            config.primary_timeframe = tf
+            config.confirmation_timeframes = conf_tfs
+
+        aggregator = SignalAggregator(config)
+        aggregator.data_provider = self._provider
+        report = aggregator.analyze_pair(symbol=SYMBOL)
+
+        logger.info(
+            "%s scan: signal=%s score=%.1f conf=%.0f quality=%s "
+            "regime=%s vol=%s price=%.2f spread=%.4f",
+            tf.value,
+            report.final_signal.name, report.final_score,
+            report.final_confidence, report.signal_quality,
+            report.market_regime, report.volatility_regime,
+            report.current_price,
+            self._provider.get_spread(SYMBOL))
+
+        # Stale-data/suppression warnings must be visible at WARNING;
+        # routine session-filter notices go to DEBUG — the M5 loop fires
+        # 288x/day and would drown the log at WARNING otherwise.
+        for w in report.warnings:
+            if "STALE" in w.upper() or "FAIL" in w.upper():
+                logger.warning("%s scan: %s", tf.value, w)
+            else:
+                logger.debug("%s scan: %s", tf.value, w)
+
+        return report
+
+    # ── Execution decision ─────────────────────────────────────────
+
+    def _should_execute(self, report: SignalReport, tf: Timeframe) -> bool:
+        if report.final_signal == SignalStrength.NEUTRAL:
+            return False
+
+        if report.trade_setup is None:
+            return False
+
+        quality_rank = QUALITY_RANK.get(report.signal_quality, -1)
+        gate_rank = QUALITY_RANK.get(self._quality_gate, 1)
+        if quality_rank < gate_rank:
+            return False
+
+        if self._in_cooldown(tf):
+            logger.info(
+                "%s: actionable signal (%s %s) but inside %d-bar entry "
+                "cooldown, skipping",
+                tf.value, report.trade_setup.direction,
+                report.signal_quality, COOLDOWN_BARS)
+            return False
+
+        with self._state_lock:
+            if self._state.get("trades"):
+                logger.info(
+                    "%s: actionable signal (%s %s) but position already "
+                    "open, skipping",
+                    tf.value, report.trade_setup.direction,
+                    report.signal_quality)
+                return False
+
+        return True
+
+    def _in_cooldown(self, tf: Timeframe) -> bool:
+        """Backtest parity: no entry within COOLDOWN_BARS bars of the
+        previous entry on the same timeframe (wall-clock approximation)."""
+        with self._state_lock:
+            last = self._state.get("last_entries", {}).get(tf.value)
+        if not last:
+            return False
+        try:
+            last_ts = datetime.fromisoformat(last)
+        except ValueError:
+            return False
+        interval_s = TF_INTERVAL_MINUTES[tf] * 60
+        elapsed = (datetime.now(timezone.utc) - last_ts).total_seconds()
+        return elapsed < COOLDOWN_BARS * interval_s
+
+    # ── Trade execution ────────────────────────────────────────────
+
+    def _execute_trade(self, report: SignalReport, tf: Timeframe) -> None:
+        ts = report.trade_setup
+        if ts is None:
+            return
+
+        # All four TF loops can fire at an aligned boundary (e.g. :00) and
+        # pass _should_execute before any of them has recorded a fill —
+        # re-check under the entry lock so only one order goes out.
+        with self._entry_lock:
+            with self._state_lock:
+                if self._state.get("trades"):
+                    logger.info(
+                        "%s: position opened by a concurrent scan, skipping",
+                        tf.value)
+                    return
+
+            # NO take-profit on the order: the backtested trailing model
+            # exits via SL/TRAIL/TIMEOUT only — a hard TP would cap
+            # winners and diverge the live book from the backtest.
+            try:
+                result = self._executor.place_market_order(
+                    instrument=INSTRUMENT,
+                    direction=ts.direction,
+                    stop_loss=ts.stop_loss,
+                )
+            except OandaAPIError:
+                logger.exception(
+                    "Failed to place %s order for %s %s",
+                    ts.direction, SYMBOL, tf.value)
+                return
+
+            fill_tx = result.get("orderFillTransaction")
+            if not fill_tx:
+                logger.warning(
+                    "Order not filled for %s %s", SYMBOL, tf.value)
+                return
+
+            trade_opened = fill_tx.get("tradeOpened", {})
+            trade_id = trade_opened.get("tradeID")
+            if not trade_id:
+                logger.warning("No trade ID in fill response")
+                return
+
+            fill_price = float(fill_tx.get("price", ts.entry_price))
+            risk = abs(fill_price - ts.stop_loss)
+
+            trail_arm = ts.trail_arm_r if ts.trail_arm_r is not None else 0.6
+            trail_dist = ts.trail_dist_r if ts.trail_dist_r is not None else 0.75
+
+            now_iso = datetime.now(timezone.utc).isoformat()
+            trade_state = {
+                "tf": tf.value,
+                "direction": ts.direction,
+                "entry": fill_price,
+                "sl_initial": ts.stop_loss,
+                "current_sl": ts.stop_loss,
+                "tp1": ts.take_profit_1,
+                "tp2": ts.take_profit_2,
+                "risk": risk,
+                "peak": fill_price,
+                "trail_armed": False,
+                "trail_arm_r": trail_arm,
+                "trail_dist_r": trail_dist,
+                "bars_held": 0,
+                # The signal bar is already closed; the first bar this
+                # trade lives through is the one that just started. Guard
+                # so a lagging feed can't trail on the signal bar itself.
+                "signal_bar_ts": report.timestamp,
+                "last_bar_ts": None,
+                "strategy": ts.strategy,
+                "quality": report.signal_quality,
+                "rr": ts.risk_reward,
+                "opened_at": now_iso,
+            }
+
+            with self._state_lock:
+                self._state["trades"][trade_id] = trade_state
+                self._state["last_entries"][tf.value] = now_iso
+                _save_state(self._state)
+
+        logger.info(
+            "TRADE OPENED: %s %s @ %.2f | SL %.2f | TP1 %.2f (advisory) | "
+            "risk $%.2f | RR %.2f | quality %s | strategy %s | TF %s | "
+            "trail arm %.2fR dist %.2fR",
+            ts.direction, SYMBOL, fill_price, ts.stop_loss,
+            ts.take_profit_1,
+            risk, ts.risk_reward, report.signal_quality,
+            ts.strategy, tf.value, trail_arm, trail_dist)
+
+    # ── Trailing stop management ───────────────────────────────────
+
+    def _manage_trailing(self, tf: Timeframe) -> None:
+        with self._state_lock:
+            trades = dict(self._state.get("trades", {}))
+
+        for trade_id, info in trades.items():
+            if info.get("tf") != tf.value:
+                continue
+
+            try:
+                df = self._provider.fetch_ohlcv(SYMBOL, tf)
+            except Exception:
+                logger.exception(
+                    "Failed to fetch candle data for trail update")
+                continue
+
+            if df.empty:
+                logger.warning(
+                    "%s: no candle data for trail update of trade %s",
+                    tf.value, trade_id)
+                continue
+
+            bar_ts = df.index[-1].isoformat()
+
+            # Process each closed bar exactly once. Weekends and feed lag
+            # re-serve the same last bar; without this guard bars_held
+            # inflates (an M5 trade would "time out" over a weekend) and
+            # the peak gets double-counted.
+            if bar_ts == info.get("last_bar_ts"):
+                continue
+            sig_ts = info.get("signal_bar_ts")
+            if sig_ts:
+                try:
+                    if df.index[-1] <= pd.Timestamp(sig_ts):
+                        continue
+                except (ValueError, TypeError):
+                    pass
+            info["last_bar_ts"] = bar_ts
+
+            info["bars_held"] = info.get("bars_held", 0) + 1
+
+            if info["bars_held"] >= self._max_hold_bars:
+                logger.info(
+                    "TIMEOUT: trade %s held %d bars, closing",
+                    trade_id, info["bars_held"])
+                try:
+                    self._executor.close_trade(trade_id)
+                except OandaAPIError:
+                    # Keep tracking — the trade is still live on the
+                    # broker; we retry the close at the next bar.
+                    logger.exception(
+                        "Failed to close timed-out trade %s — will retry",
+                        trade_id)
+                    with self._state_lock:
+                        self._state["trades"][trade_id] = info
+                        _save_state(self._state)
+                    continue
+                with self._state_lock:
+                    self._state["trades"].pop(trade_id, None)
+                    _save_state(self._state)
+                continue
+
+            last_bar = df.iloc[-1]
+            bar_high = float(last_bar["High"])
+            bar_low = float(last_bar["Low"])
+
+            new_sl = self._compute_trail(info, bar_high, bar_low)
+
+            if new_sl is not None:
+                # Round to instrument precision and track exactly what the
+                # broker holds, so tracked vs broker SL can never drift.
+                new_sl = round(new_sl, self._price_precision)
+                if new_sl != info["current_sl"]:
+                    try:
+                        self._executor.modify_trade_sl(trade_id, new_sl)
+                        info["current_sl"] = new_sl
+                    except OandaAPIError:
+                        logger.exception(
+                            "Failed to update trailing SL for trade %s",
+                            trade_id)
+
+            with self._state_lock:
+                self._state["trades"][trade_id] = info
+                _save_state(self._state)
+
+    def _compute_trail(
+        self, info: Dict, bar_high: float, bar_low: float
+    ) -> Optional[float]:
+        buy = info["direction"] == "BUY"
+        risk = info["risk"]
+        if risk <= 0:
+            return None
+        arm = info.get("trail_arm_r", 0.6)
+        dist = info.get("trail_dist_r", 0.75)
+        if arm <= 0:
+            return None
+
+        fav = bar_high if buy else bar_low
+        if buy:
+            info["peak"] = max(info.get("peak", info["entry"]), fav)
+        else:
+            info["peak"] = min(info.get("peak", info["entry"]), fav)
+
+        exc = ((info["peak"] - info["entry"]) if buy
+               else (info["entry"] - info["peak"])) / risk
+
+        if exc >= arm:
+            if buy:
+                lvl = info["peak"] - dist * risk
+                cur_sl = info.get("current_sl", info["sl_initial"])
+                if lvl > cur_sl:
+                    info["trail_armed"] = True
+                    return lvl
+            else:
+                lvl = info["peak"] + dist * risk
+                cur_sl = info.get("current_sl", info["sl_initial"])
+                if lvl < cur_sl:
+                    info["trail_armed"] = True
+                    return lvl
+
+        return None
+
+    # ── Closed-trade detection ─────────────────────────────────────
+
+    def _check_closed_trades(self) -> None:
+        with self._state_lock:
+            tracked = set(self._state.get("trades", {}).keys())
+
+        if not tracked:
+            return
+
+        try:
+            open_trades = self._executor.get_open_trades(INSTRUMENT)
+        except OandaAPIError:
+            logger.exception("Failed to query open trades")
+            return
+
+        live_ids = {t["id"] for t in open_trades}
+        closed = tracked - live_ids
+
+        for tid in closed:
+            with self._state_lock:
+                info = self._state["trades"].pop(tid, {})
+            entry = info.get("entry", 0)
+            direction = info.get("direction", "?")
+            risk = info.get("risk", 1)
+            sl = info.get("current_sl", 0)
+            was_trailed = info.get("trail_armed", False)
+
+            logger.info(
+                "TRADE CLOSED BY BROKER: %s %s %s | entry %.2f | "
+                "SL was %.2f | risk $%.2f | trailed=%s | bars_held=%d",
+                tid, direction, SYMBOL, entry, sl, risk,
+                was_trailed, info.get("bars_held", 0))
+
+        if closed:
+            with self._state_lock:
+                _save_state(self._state)
+
+# ── Uncaught exception handlers ───────────────────────────────────────
+
+def _excepthook(exc_type, exc_value, exc_tb):
+    if issubclass(exc_type, KeyboardInterrupt):
+        sys.__excepthook__(exc_type, exc_value, exc_tb)
+        return
+    logger.critical(
+        "Uncaught exception", exc_info=(exc_type, exc_value, exc_tb))
+
+def _threading_excepthook(args):
+    logger.critical(
+        "Uncaught exception in thread %s",
+        args.thread.name if args.thread else "?",
+        exc_info=(args.exc_type, args.exc_value, args.exc_traceback))
+
+# ── Entry point ────────────────────────────────────────────────────────
+
+def main() -> None:
+    _setup_logging(
+        os.environ.get("OANDA_LOG_LEVEL") or get_app_config().log_level)
+    sys.excepthook = _excepthook
+    threading.excepthook = _threading_excepthook
+
+    daemon = TradingDaemon()
+    asyncio.run(daemon.run())
+
+
+if __name__ == "__main__":
+    main()

oanda_executor.py

@@ -0,0 +1,212 @@
+"""
+ForexBot2 — Oanda v20 Order Executor
+======================================
+Handles all Oanda account operations: market orders, trade queries,
+SL/TP modification, and trade closure.  Uses raw ``requests`` against
+the Oanda v20 REST API — no extra dependencies.
+"""
+
+import logging
+import time
+from typing import Dict, List, Optional
+
+import requests
+
+logger = logging.getLogger(__name__)
+
+
+class OandaAPIError(Exception):
+    """Raised when an Oanda API call fails."""
+
+    def __init__(self, message: str, status_code: int = 0, body: str = ""):
+        super().__init__(message)
+        self.status_code = status_code
+        self.body = body
+
+
+class OandaExecutor:
+    """Oanda v20 order execution and trade management."""
+
+    MAX_RETRIES = 1
+    RETRY_DELAY = 2.0
+
+    def __init__(
+        self,
+        account_id: str,
+        api_token: str,
+        base_url: str = "https://api-fxpractice.oanda.com",
+        lot_size: int = 1,
+        price_precision: int = 2,
+    ):
+        self._account_id = account_id
+        self._base_url = base_url.rstrip("/")
+        self._lot_size = lot_size
+        self._price_precision = price_precision
+        self._session = requests.Session()
+        self._session.headers.update({
+            "Authorization": f"Bearer {api_token}",
+            "Accept-Datetime-Format": "RFC3339",
+            "Content-Type": "application/json",
+        })
+
+    # ── Order Placement ────────────────────────────────────────────
+
+    def place_market_order(
+        self,
+        instrument: str,
+        direction: str,
+        stop_loss: float,
+        take_profit: Optional[float] = None,
+        units: Optional[int] = None,
+    ) -> Dict:
+        if units is None:
+            units = self._lot_size
+        signed_units = str(units) if direction.upper() == "BUY" else str(-units)
+
+        order_body = {
+            "order": {
+                "type": "MARKET",
+                "instrument": instrument,
+                "units": signed_units,
+                "timeInForce": "FOK",
+                "stopLossOnFill": {
+                    "price": self._fmt_price(stop_loss),
+                },
+            }
+        }
+        if take_profit is not None:
+            order_body["order"]["takeProfitOnFill"] = {
+                "price": self._fmt_price(take_profit),
+            }
+
+        url = f"{self._base_url}/v3/accounts/{self._account_id}/orders"
+        # Never retry order placement: a timeout after the server already
+        # filled would open a SECOND position on retry. Fail loudly instead;
+        # the next candle boundary re-evaluates.
+        resp = self._request_with_retry("POST", url, json=order_body, retries=0)
+        data = self._parse_json(resp)
+
+        fill_tx = data.get("orderFillTransaction")
+        if fill_tx:
+            trade_id = fill_tx.get("tradeOpened", {}).get("tradeID", "unknown")
+            fill_price = float(fill_tx.get("price", 0))
+            logger.info(
+                "ORDER FILLED: %s %s %s units @ %.2f — trade %s, SL %.2f%s",
+                direction.upper(), instrument, signed_units, fill_price,
+                trade_id, stop_loss,
+                f", TP {take_profit:.2f}" if take_profit else "")
+        else:
+            cancel_tx = data.get("orderCancelTransaction", {})
+            reason = cancel_tx.get("reason", "unknown")
+            logger.warning(
+                "ORDER NOT FILLED: %s %s — reason: %s",
+                direction.upper(), instrument, reason)
+
+        return data
+
+    # ── Trade Queries ──────────────────────────────────────────────
+
+    def get_open_trades(self, instrument: Optional[str] = None) -> List[Dict]:
+        url = f"{self._base_url}/v3/accounts/{self._account_id}/openTrades"
+        resp = self._request_with_retry("GET", url)
+        trades = self._parse_json(resp).get("trades", [])
+        if instrument:
+            trades = [t for t in trades if t.get("instrument") == instrument]
+        return trades
+
+    def get_trade(self, trade_id: str) -> Dict:
+        url = (f"{self._base_url}/v3/accounts/{self._account_id}"
+               f"/trades/{trade_id}")
+        resp = self._request_with_retry("GET", url)
+        return self._parse_json(resp).get("trade", {})
+
+    def get_account_summary(self) -> Dict:
+        url = f"{self._base_url}/v3/accounts/{self._account_id}/summary"
+        resp = self._request_with_retry("GET", url)
+        account = self._parse_json(resp).get("account", {})
+        logger.info(
+            "Account: balance=%.2f, NAV=%.2f, unrealizedPL=%.2f, openTrades=%s",
+            float(account.get("balance", 0)),
+            float(account.get("NAV", 0)),
+            float(account.get("unrealizedPL", 0)),
+            account.get("openTradeCount", 0))
+        return account
+
+    # ── Trade Modification ─────────────────────────────────────────
+
+    def modify_trade_sl(self, trade_id: str, new_sl: float) -> Dict:
+        url = (f"{self._base_url}/v3/accounts/{self._account_id}"
+               f"/trades/{trade_id}/orders")
+        body = {"stopLoss": {"price": self._fmt_price(new_sl)}}
+        resp = self._request_with_retry("PUT", url, json=body)
+        logger.info(
+            "TRAIL UPDATE: trade %s — new SL %.2f", trade_id, new_sl)
+        return self._parse_json(resp)
+
+    def modify_trade_tp(self, trade_id: str, new_tp: float) -> Dict:
+        url = (f"{self._base_url}/v3/accounts/{self._account_id}"
+               f"/trades/{trade_id}/orders")
+        body = {"takeProfit": {"price": self._fmt_price(new_tp)}}
+        resp = self._request_with_retry("PUT", url, json=body)
+        logger.info("TP UPDATE: trade %s — new TP %.2f", trade_id, new_tp)
+        return self._parse_json(resp)
+
+    def close_trade(self, trade_id: str) -> Dict:
+        url = (f"{self._base_url}/v3/accounts/{self._account_id}"
+               f"/trades/{trade_id}/close")
+        resp = self._request_with_retry("PUT", url, json={"units": "ALL"})
+        data = self._parse_json(resp)
+        close_tx = data.get("orderFillTransaction", {})
+        close_price = float(close_tx.get("price", 0))
+        pl = float(close_tx.get("pl", 0))
+        logger.info(
+            "TRADE CLOSED: trade %s @ %.2f — PL %.2f",
+            trade_id, close_price, pl)
+        return data
+
+    # ── Internals ──────────────────────────────────────────────────
+
+    @staticmethod
+    def _parse_json(resp: requests.Response) -> Dict:
+        try:
+            return resp.json()
+        except ValueError as e:
+            raise OandaAPIError(
+                f"Oanda returned a non-JSON response (HTTP {resp.status_code})",
+                resp.status_code, resp.text[:300]) from e
+
+    def _fmt_price(self, price: float) -> str:
+        # Oanda rejects prices with more decimals than the instrument's
+        # displayPrecision (XAU_USD: sub-cent precision is invalid).
+        return f"{price:.{self._price_precision}f}"
+
+    def _request_with_retry(
+        self, method: str, url: str, retries: Optional[int] = None, **kwargs
+    ) -> requests.Response:
+        if retries is None:
+            retries = self.MAX_RETRIES
+        last_err = None
+        for attempt in range(1 + retries):
+            try:
+                resp = self._session.request(method, url, timeout=15, **kwargs)
+                if resp.status_code >= 500:
+                    raise OandaAPIError(
+                        f"Oanda server error {resp.status_code}",
+                        resp.status_code, resp.text)
+                if resp.status_code >= 400:
+                    raise OandaAPIError(
+                        f"Oanda API error {resp.status_code}: {resp.text}",
+                        resp.status_code, resp.text)
+                return resp
+            except (requests.ConnectionError, requests.Timeout,
+                    OandaAPIError) as e:
+                last_err = e
+                if isinstance(e, OandaAPIError) and 400 <= e.status_code < 500:
+                    raise
+                if attempt < retries:
+                    logger.warning(
+                        "Oanda request failed (attempt %d/%d): %s — retrying in %.0fs",
+                        attempt + 1, 1 + retries, e,
+                        self.RETRY_DELAY)
+                    time.sleep(self.RETRY_DELAY)
+        raise OandaAPIError(f"Oanda request failed after retries: {last_err}")

oanda_provider.py

@@ -0,0 +1,9 @@
+"""
+ForexBot2 — Oanda provider (back-compat shim)
+==============================================
+The implementation lives in `providers/oanda.py`; shared behavior is in
+`providers/base.py`. This module keeps the historical import path working.
+"""
+
+from providers.base import _is_forex_weekend  # noqa: F401
+from providers.oanda import OandaDataError, OandaDataProvider  # noqa: F401

pair_signals/__init__.py

@@ -12,7 +12,7 @@ from typing import Optional, List, Dict, Callable
 
 from config import PlatformConfig, Timeframe
 from signal_aggregator import SignalAggregator, SignalReport
-from data_provider import ForexDataProvider
+from providers import BaseDataProvider
 
 PAIR_PRODUCERS: Dict[str, Callable] = {}
 
@@ -28,7 +28,7 @@ def get_supported_pairs() -> List[str]:
 def produce_signal(
     symbol: str,
     primary_tf: Timeframe,
-    data_provider: ForexDataProvider,
+    data_provider: BaseDataProvider,
     confirmation_tfs: Optional[List[Timeframe]] = None,
 ) -> SignalReport:
     config_factory = PAIR_PRODUCERS.get(symbol)

pair_signals/xauusd.py

@@ -56,6 +56,40 @@ positive, val half -1.89R — marginal but sign-correct in aggregate).
 Gold's structural uptrend makes bullish patterns fundamentally more
 reliable. M15/H1 fail on all windows; M5-only enablement.
 
+M5 override 3 (validated 2026-07-21, 5-year replay 2021-07-21 ->
+2026-07-21, $0.30 spread, next-open fills): trailing distance widened
+to 0.85R (trail_dist_r 0.85, arm unchanged 0.60). Same 30-cell grid
+methodology as the M15 campaign, opposite finding — on M5 the arm
+point barely matters; the winners need a looser leash. Plateau arm
+0.5-0.7 x dist 0.85-1.0 (+227..+234R vs +218.0R baseline). Chosen
+0.60/0.85 (OOS-preferred end of the plateau): +231.2R/2028 PF 1.30 WR
+57.1% vs +218.0R PF 1.29 WR 58.3%; both last-12mo halves better
+(+56.8/+115.1 vs +54.0/+104.6); 5yr max DD 19.2R vs 19.1R (noise);
+OOS 2024-06-01 -> 2025-03-30 +14.5R DD 15.1 vs +11.1R DD 15.4.
+dist 1.0 was REJECTED at the OOS gate (+10.9R < baseline +11.1R)
+despite the best in-sample total (+233.3R) — the OOS ordering picks
+the conservative plateau edge. XAU_PB trades are unaffected (they
+carry their own validated pb_trail 0.8/0.75 per-trade overrides);
+XAU_CP trades share the global params and were validated inside this
+book. M15 arms at 0.70 (its own campaign, same day); the divergence
+is the per-TF pattern yet again.
+
+M15 override 4 (validated 2026-07-21, 5-year replay 2021-07-21 ->
+2026-07-21, $0.30 spread, next-open fills): trailing-stop arm raised to
+0.70R (trail_arm_r 0.70, dist unchanged 0.75). Full grid arm 0.3-0.9 x
+dist 0.5-1.0 (30 cells): every earlier-arm cell (0.3/0.45) is WORSE —
+the "arm earlier to save fading trades" hypothesis is refuted; the
+plateau is arm 0.70-0.80 x dist 0.70-0.85 (all >= +190R). Chosen
+plateau center 0.70/0.75: +210.7R/1629 PF 1.35 WR 57.8% vs baseline
+0.60/0.75 +175.7R PF 1.30 WR 53.9%; both last-12mo halves improve
+(+49.7/+50.0 vs +36.4/+46.0); per-year better 4/6 (2022 -0.9R, 2023
+-1.4R, both within tolerance); OOS 2024-06-01 -> 2025-03-30 +57.4R
+PF 1.57 DD 17.0R vs +45.4R PF 1.47 DD 17.8R — better on every OOS
+metric. One clause waived BY OWNER DECISION 2026-07-21: 5yr max DD is
+33.9R vs 29.1R baseline (+4.8R deeper worst episode, 2022-23 chop),
+accepted in exchange for +20% total R. M5 keeps the global 0.6 arm
+(same per-TF divergence pattern as tp_floor_rr and the hour-9 block).
+
 M5 override (validated 2026-07-11, full-history replay 2020-03-17 ->
 2026-07-11 + 12mo book, $0.30 spread): block hour 9 UTC (post-London-
 open whipsaw). Hour-9 entries lost -28.9R/143 over 6 years, negative
@@ -69,6 +103,7 @@ M5-only, same per-timeframe pattern as the M15 overrides.
 
 from typing import Optional, List
 from config import PlatformConfig, Timeframe
+from app_config import apply_pair_overrides
 
 _CONFIRMATIONS = {
     Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
@@ -83,6 +118,9 @@ def make_config(
     confirmation_tfs: Optional[List[Timeframe]] = None,
 ) -> PlatformConfig:
     config = PlatformConfig()
+    # Deployment overlay (config.json) BEFORE the validated per-TF
+    # deltas below, so e.g. the M5 hour-9 append lands on the JSON base.
+    apply_pair_overrides(config)
     config.primary_timeframe = primary_tf
     config.confirmation_timeframes = (
         confirmation_tfs or _CONFIRMATIONS.get(primary_tf, [Timeframe.M15, Timeframe.H4])
@@ -95,6 +133,9 @@ def make_config(
         # BUY-only chart patterns on M5 — see module docstring.
         config.xau_strategy.cp_enabled = True
         config.xau_strategy.cp_patterns = ["inv_hs", "double_bottom", "bull_flag"]
+        # Wider trailing distance on M5 only — see module docstring.
+        config.signals.trail_arm_r = 0.60
+        config.signals.trail_dist_r = 0.85
     if primary_tf == Timeframe.M15:
         # 1R minimum target on M15 only — see module docstring.
         config.signals.tp_floor_rr = 1.0
@@ -102,4 +143,7 @@ def make_config(
         config.xau_strategy.blocked_regimes = ["trending"]
         # Block SELL entries at 15:00/17:00 UTC on M15 only — see docstring.
         config.xau_strategy.sell_blocked_hours_utc = [15, 17]
+        # Later trailing-stop arm on M15 only — see module docstring.
+        config.signals.trail_arm_r = 0.70
+        config.signals.trail_dist_r = 0.75
     return config

providers/__init__.py

@@ -0,0 +1,54 @@
+"""
+ForexBot2 — Data-provider package
+==================================
+One class per provider, all subclassing `BaseDataProvider` (which owns
+caching, validation, boundary guarantees, and range pagination — the
+behavior the signal pipeline depends on).
+
+Adding a provider = one module implementing `_fetch_raw` + one registry
+entry + one config.json `data_providers` block. The live feed is selected
+by config.json `data_providers.active`; registration alone changes
+nothing.
+"""
+
+from typing import Dict
+
+from app_config import AppConfigError, get_app_config
+from config import PlatformConfig
+
+from providers.base import BaseDataProvider, DataIntegrityError, _is_forex_weekend
+from providers.twelve_data import TwelveDataProvider
+from providers.oanda import OandaDataError, OandaDataProvider
+from providers.tiingo import TiingoDataError, TiingoDataProvider
+
+#: config.json data_providers.active selects from here.
+PROVIDER_REGISTRY: Dict[str, type] = {
+    "twelve_data": TwelveDataProvider,
+    "oanda": OandaDataProvider,
+    "tiingo": TiingoDataProvider,
+}
+
+
+def make_data_provider(config: PlatformConfig) -> BaseDataProvider:
+    """Build the ACTIVE data provider (config.json: data_providers.active)."""
+    name = get_app_config().active_provider
+    cls = PROVIDER_REGISTRY.get(name)
+    if cls is None:
+        raise AppConfigError(
+            f'Unknown data provider "{name}"; '
+            f"registered: {sorted(PROVIDER_REGISTRY)}")
+    return cls(config)
+
+
+__all__ = [
+    "BaseDataProvider",
+    "DataIntegrityError",
+    "OandaDataError",
+    "OandaDataProvider",
+    "TiingoDataError",
+    "TiingoDataProvider",
+    "TwelveDataProvider",
+    "PROVIDER_REGISTRY",
+    "make_data_provider",
+    "_is_forex_weekend",
+]

providers/base.py

@@ -0,0 +1,423 @@
+"""
+ForexBot2 — Provider base class
+================================
+`BaseDataProvider` owns everything that must behave identically no matter
+where the bars come from: in-memory caching with candle-duration TTL, the
+closed-bar boundary guarantee, freshness checks, validation/cleaning, the
+forming-candle drop, and paginated date-range fetching for the backtester.
+
+A concrete provider implements only:
+    _fetch_raw(symbol, timeframe, outputsize, start, end) -> pd.DataFrame
+returning a normalized frame (capitalized OHLCV columns, UTC tz-aware
+DatetimeIndex named "Datetime", oldest-first), plus optionally
+`get_spread` and the streaming-quote capability (`supports_streaming` +
+`iter_ticks`).
+
+The signal pipeline (engine + aggregator + backtester) consumes ONLY the
+public contract defined here, so swapping providers never touches signal
+code.
+"""
+
+import logging
+import threading
+import time
+from abc import ABC, abstractmethod
+from datetime import datetime, timedelta
+from typing import AsyncIterator, Callable, Dict, Optional, Sequence, Tuple
+
+import pandas as pd
+
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+logger = logging.getLogger(__name__)
+
+
+def _is_forex_weekend(now=None) -> bool:
+    """Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC."""
+    if now is None:
+        now = pd.Timestamp.now(tz="UTC")
+    wd = now.weekday()
+    hour = now.hour
+    if wd == 4 and hour >= 22:
+        return True
+    if wd == 5:
+        return True
+    if wd == 6 and hour < 22:
+        return True
+    return False
+
+
+class DataIntegrityError(Exception):
+    """Raised when data fails validation checks."""
+    pass
+
+
+class BaseDataProvider(ABC):
+    """Shared OHLCV provider behavior; subclasses supply the raw fetch."""
+
+    #: Registry name; overridden per provider.
+    name: str = "base"
+
+    #: Exception type this provider raises for fetch/validation failures.
+    #: Provider-specific errors subclass DataIntegrityError so consumers
+    #: can catch one type regardless of the active provider.
+    DATA_ERROR = DataIntegrityError
+
+    BARS_PER_DAY = {
+        Timeframe.M1: 1440, Timeframe.M5: 288, Timeframe.M15: 96,
+        Timeframe.M30: 48, Timeframe.H1: 24, Timeframe.H4: 6,
+        Timeframe.D1: 1, Timeframe.W1: 0.143,
+    }
+
+    CANDLE_SECONDS = {
+        Timeframe.M1: 60, Timeframe.M5: 300, Timeframe.M15: 900,
+        Timeframe.M30: 1800, Timeframe.H1: 3600, Timeframe.H4: 14400,
+        Timeframe.D1: 86400, Timeframe.W1: 604800,
+    }
+
+    #: Hard per-request bar cap shared by Twelve Data and Oanda.
+    MAX_BARS_PER_REQUEST = 5000
+
+    # Grace wait before the one retry when a fresh response at a candle
+    # boundary is missing the just-closed bar (feed latency).
+    FRESH_FETCH_RETRY_SECONDS = 3.0
+
+    def __init__(self, config: PlatformConfig):
+        self.config = config
+        self._cache: Dict[str, pd.DataFrame] = {}
+        self._cache_timestamps: Dict[str, float] = {}
+        # The bot shares one provider across a thread pool; cache eviction
+        # is check-then-act and needs locking.
+        self._cache_lock = threading.Lock()
+
+    # ── Abstract: the only provider-specific piece ─────────────────
+
+    @abstractmethod
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        """Fetch up to `outputsize` bars, optionally bounded by start/end
+        (tz-aware UTC datetimes). Must return a normalized frame:
+        capitalized OHLCV columns, UTC DatetimeIndex named "Datetime",
+        oldest-first. Raise self.DATA_ERROR on failure."""
+
+    # ── Public API ─────────────────────────────────────────────────
+
+    def fetch_ohlcv(
+        self,
+        symbol: str = XAUUSD,
+        timeframe: Timeframe = Timeframe.H1,
+        lookback_days: Optional[int] = None,
+    ) -> pd.DataFrame:
+        """Fetch OHLCV with in-memory caching for live mode."""
+        if lookback_days is None:
+            lookback_days = self.config.data_lookback_days
+
+        cache_key = f"{symbol}_{timeframe.value}"
+        now = time.time()
+
+        cache_ttl = self.CANDLE_SECONDS.get(timeframe, self.config.refresh_interval)
+        with self._cache_lock:
+            cached = self._cache.get(cache_key)
+            cached_at = self._cache_timestamps.get(cache_key)
+        if (
+            cached is not None
+            and cached_at is not None
+            and (now - cached_at) < cache_ttl
+            and not cached.empty
+        ):
+            if (self._is_data_fresh(cached, symbol, timeframe)
+                    and self._has_latest_closed_bar(cached, timeframe)):
+                return cached
+            logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
+            with self._cache_lock:
+                self._cache.pop(cache_key, None)
+                self._cache_timestamps.pop(cache_key, None)
+
+        outputsize = max(
+            300,
+            min(int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20,
+                self.MAX_BARS_PER_REQUEST),
+        )
+        df = self._fetch_raw(symbol, timeframe, outputsize)
+        df = self._drop_forming_candle(df, timeframe)
+        df = self._validate_and_clean(df, symbol, timeframe)
+
+        # The closed-bar guarantee must hold for FRESH responses too, not
+        # just cache hits: the scanner fires seconds after the boundary,
+        # and a lagging feed can answer without the just-closed bar — the
+        # scan would silently analyze (and session-check) the previous
+        # bar. Give the feed one short grace retry, then proceed loudly.
+        if not df.empty and not self._has_latest_closed_bar(df, timeframe):
+            time.sleep(self.FRESH_FETCH_RETRY_SECONDS)
+            try:
+                retry = self._fetch_raw(symbol, timeframe, outputsize)
+                retry = self._drop_forming_candle(retry, timeframe)
+                retry = self._validate_and_clean(retry, symbol, timeframe)
+                if retry is not None and not retry.empty:
+                    df = retry
+            except DataIntegrityError as e:
+                logger.warning(
+                    f"{symbol} {timeframe.value}: boundary retry refetch "
+                    f"failed ({e}) — keeping original data")
+            if not self._has_latest_closed_bar(df, timeframe):
+                logger.warning(
+                    f"{symbol} {timeframe.value}: feed lagging — just-closed bar "
+                    f"still missing after retry; this scan may run one bar behind"
+                )
+
+        if not self._is_data_fresh(df, symbol, timeframe):
+            logger.warning(
+                f"STALE DATA from API: {symbol} {timeframe.value} — "
+                f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
+            )
+
+        with self._cache_lock:
+            self._cache[cache_key] = df
+            self._cache_timestamps[cache_key] = now
+        return df
+
+    def fetch_range(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        start: datetime,
+        end: datetime,
+    ) -> pd.DataFrame:
+        """Fetch the RAW closed-bar series between start/end, paginating
+        under the per-request bar cap. Returns uncleaned bars (the
+        backtester resolves trades on the raw price path); callers wanting
+        the signal-side series run `validate_and_clean` on the result."""
+        bars_per_day = self.BARS_PER_DAY[timeframe]
+        days_per_chunk = max(1, int(4500 / bars_per_day))
+
+        chunks = []
+        cursor_end = end
+        empty_streak = 0
+        while cursor_end > start:
+            cursor_start = max(start, cursor_end - timedelta(days=days_per_chunk))
+            try:
+                chunk = self._fetch_raw(
+                    symbol, timeframe, self.MAX_BARS_PER_REQUEST,
+                    start=cursor_start, end=cursor_end,
+                )
+            except Exception as e:
+                # Providers raise for windows with no data (weekends,
+                # holidays, asset gaps). Skip the window, step back, retry
+                # — but bail after a few consecutive empties.
+                logger.warning(
+                    f"Chunk {cursor_start.date()}–{cursor_end.date()} failed: {e}"
+                )
+                empty_streak += 1
+                if empty_streak >= 3:
+                    break
+                cursor_end = cursor_start - timedelta(seconds=1)
+                continue
+
+            if chunk is None or chunk.empty:
+                empty_streak += 1
+                if empty_streak >= 3:
+                    break
+                cursor_end = cursor_start - timedelta(seconds=1)
+                continue
+
+            empty_streak = 0
+            chunks.append(chunk)
+            # Step back; dedup on concat.
+            cursor_end = chunk.index[0] - timedelta(seconds=1)
+            if len(chunk) < 2:
+                break
+
+        if not chunks:
+            raise RuntimeError(f"No data fetched for {symbol} {timeframe.value}")
+
+        df = pd.concat(chunks).sort_index()
+        df = df[~df.index.duplicated(keep="last")]
+        # Fetches ending "now" include the still-forming candle — drop it
+        # so callers only ever see closed bars.
+        df = self._drop_forming_candle(df, timeframe)
+        start_ts = pd.Timestamp(start)
+        end_ts = pd.Timestamp(end)
+        return df[(df.index >= start_ts) & (df.index <= end_ts)]
+
+    def is_data_fresh(
+        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
+    ) -> bool:
+        """Public freshness check (last bar within 3 candle durations)."""
+        return self._is_data_fresh(df, symbol, timeframe)
+
+    def validate_and_clean(
+        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
+    ) -> pd.DataFrame:
+        """Public entry to the shared cleaning pipeline."""
+        return self._validate_and_clean(df, symbol, timeframe)
+
+    def clear_cache(self) -> int:
+        """Wipe all cached data. Returns number of entries cleared."""
+        with self._cache_lock:
+            count = len(self._cache)
+            self._cache.clear()
+            self._cache_timestamps.clear()
+        logger.info(f"Cache cleared: {count} entries wiped")
+        return count
+
+    def get_pair_info(self, symbol: str = XAUUSD) -> Dict:
+        info = PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])
+        return {"symbol": symbol, **info}
+
+    def get_spread(self, symbol: str = XAUUSD) -> float:
+        """Latest observed round-trip spread; providers without live
+        bid/ask fall back to the pair's assumed default."""
+        return PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])["default_spread"]
+
+    # ── Streaming-quote capability (optional) ──────────────────────
+
+    #: True when the provider can stream live ticks via `iter_ticks`.
+    supports_streaming: bool = False
+
+    def iter_ticks(
+        self,
+        symbols: Sequence[str],
+        on_connect: Optional[Callable[[], None]] = None,
+    ) -> AsyncIterator[Tuple[str, float]]:
+        """Async generator yielding (canonical_symbol, price) ticks for one
+        connection's lifetime; the caller owns reconnect/backoff. Providers
+        that support it set `supports_streaming = True` and override."""
+        raise NotImplementedError(
+            f"{self.name} provider does not support streaming quotes")
+
+    # ── Shared internals ───────────────────────────────────────────
+
+    def _has_latest_closed_bar(self, df: pd.DataFrame, timeframe: Timeframe) -> bool:
+        """Cached data must include the most recent fully closed bar.
+
+        The TTL alone is wall-clock based: a snapshot fetched mid-candle is
+        still within TTL at the next candle boundary but is missing the bar
+        that just closed, so every boundary scan would analyze — and run the
+        bar-close session check against — the previous bar.
+        """
+        if df is None or df.empty:
+            return False
+        now = pd.Timestamp.now(tz="UTC")
+        if _is_forex_weekend(now):
+            return True
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        dur = pd.Timedelta(seconds=candle_seconds)
+        expected_label = now.floor(dur) - dur
+        return df.index[-1] >= expected_label
+
+    def _drop_forming_candle(self, df: pd.DataFrame, timeframe: Timeframe) -> pd.DataFrame:
+        """Drop the still-forming last candle so analysis only sees closed bars.
+
+        Some feeds include the current partial candle; the scanner fires
+        seconds after the boundary, so without this guard every indicator
+        reads a seconds-old stub bar and signals repaint. Feeds that only
+        return complete candles pass through unchanged.
+        """
+        if df is None or df.empty:
+            return df
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        now = pd.Timestamp.now(tz="UTC")
+        if df.index[-1] + pd.Timedelta(seconds=candle_seconds) > now:
+            df = df.iloc[:-1]
+        return df
+
+    def _is_data_fresh(self, df: pd.DataFrame, symbol: str, timeframe: Timeframe) -> bool:
+        """Check if the last candle is within acceptable age for this timeframe."""
+        if df is None or df.empty:
+            return False
+
+        now = pd.Timestamp.now(tz="UTC")
+
+        if _is_forex_weekend(now):
+            return True
+
+        last_bar = df.index[-1]
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        max_age_seconds = candle_seconds * 3
+        age_seconds = (now - last_bar).total_seconds()
+
+        if age_seconds > max_age_seconds:
+            logger.warning(
+                f"{symbol} {timeframe.value}: last bar {last_bar.strftime('%Y-%m-%d %H:%M')} "
+                f"is {age_seconds / 60:.0f}m old (limit: {max_age_seconds / 60:.0f}m)"
+            )
+            return False
+
+        return True
+
+    def _validate_and_clean(
+        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
+    ) -> pd.DataFrame:
+        if df is None or df.empty:
+            raise self.DATA_ERROR(f"Empty dataset for {symbol}")
+
+        original_len = len(df)
+
+        df = df.dropna(subset=["Open", "High", "Low", "Close"], how="all")
+        df = df.ffill(limit=3)
+        df = df.dropna(subset=["Close"])
+
+        df = df.sort_index()
+        df = df[~df.index.duplicated(keep="last")]
+
+        now = pd.Timestamp.utcnow()
+        future_mask = df.index > now + pd.Timedelta(minutes=5)
+        if future_mask.any():
+            logger.warning(f"{symbol}: Dropping {future_mask.sum()} future timestamps")
+            df = df[~future_mask]
+
+        invalid_ohlc = (
+            (df["High"] < df["Low"]) |
+            (df["High"] < df["Open"]) |
+            (df["High"] < df["Close"]) |
+            (df["Low"] > df["Open"]) |
+            (df["Low"] > df["Close"])
+        )
+        if invalid_ohlc.any():
+            logger.warning(f"{symbol}: Fixing {invalid_ohlc.sum()} OHLC-inconsistent bars")
+            for idx in df[invalid_ohlc].index:
+                vals = [df.loc[idx, "Open"], df.loc[idx, "Close"]]
+                df.loc[idx, "High"] = max(vals + [df.loc[idx, "High"]])
+                df.loc[idx, "Low"]  = min(vals + [df.loc[idx, "Low"]])
+
+        returns = df["Close"].pct_change()
+        std_ret = returns.std()
+        if std_ret > 0:
+            dev = returns - returns.mean()
+            outlier_mask = dev.abs() > 10 * std_ret
+            # A spike bar makes the NEXT bar's return look extreme too (it is
+            # measured against the spike). If the previous bar is flagged and
+            # this bar reverses direction, it is the legitimate snap-back —
+            # keep it, or every spike would punch a two-bar hole in the data.
+            recovery = (
+                outlier_mask
+                & outlier_mask.shift(1, fill_value=False)
+                & (dev * dev.shift(1) < 0)
+            )
+            outlier_mask = outlier_mask & ~recovery
+            if outlier_mask.any():
+                logger.warning(f"{symbol}: Removing {outlier_mask.sum()} extreme outlier bars")
+                df = df[~outlier_mask]
+
+        min_required = max(
+            self.config.technical.ema_trend,
+            200,
+        ) + 10
+
+        if len(df) < min_required:
+            logger.warning(
+                f"{symbol} {timeframe.value}: Only {len(df)} bars "
+                f"(need {min_required}). Some indicators will be partial."
+            )
+
+        cleaned = original_len - len(df)
+        if cleaned > 0:
+            logger.info(f"{symbol}: Cleaned {cleaned}/{original_len} bars")
+
+        return df

providers/oanda.py

@@ -0,0 +1,192 @@
+"""
+ForexBot2 — Oanda v20 data provider
+====================================
+OHLCV candles from the Oanda v20 REST API. All shared behavior (cache,
+validation, boundary guarantees, range pagination) lives in
+`BaseDataProvider`; this module only maps symbols/granularities, parses
+Oanda's candle JSON, and exposes the real bid/ask spread from MBA candles.
+"""
+
+import logging
+from datetime import datetime
+from typing import Dict, Optional
+
+import pandas as pd
+import requests
+
+from app_config import oanda_api_token, oanda_base_url
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+from providers.base import BaseDataProvider, DataIntegrityError
+
+logger = logging.getLogger(__name__)
+
+
+class OandaDataError(DataIntegrityError):
+    """Raised when an Oanda data fetch fails or returns invalid data."""
+
+
+class OandaDataProvider(BaseDataProvider):
+    """Oanda v20 REST candle provider."""
+
+    name = "oanda"
+    DATA_ERROR = OandaDataError
+
+    OANDA_GRANULARITY = {
+        Timeframe.M1: "M1",
+        Timeframe.M5: "M5",
+        Timeframe.M15: "M15",
+        Timeframe.M30: "M30",
+        Timeframe.H1: "H1",
+        Timeframe.H4: "H4",
+        Timeframe.D1: "D",
+        Timeframe.W1: "W",
+    }
+
+    OANDA_INSTRUMENT = {
+        "XAU/USD": "XAU_USD",
+    }
+
+    def __init__(
+        self,
+        config: PlatformConfig,
+        api_token: Optional[str] = None,
+        base_url: Optional[str] = None,
+    ):
+        super().__init__(config)
+        # Credentials default to the config.json `oanda` block / env vars
+        # so the provider registry can build this class with config only.
+        if api_token is None:
+            api_token = oanda_api_token()
+        if base_url is None:
+            base_url = oanda_base_url()
+        self._base_url = base_url.rstrip("/")
+        self._session = requests.Session()
+        self._session.headers.update({
+            "Authorization": f"Bearer {api_token}",
+            "Accept-Datetime-Format": "RFC3339",
+            "Content-Type": "application/json",
+        })
+        self._spread_cache: Dict[str, float] = {}
+
+    # ── Public extras ──────────────────────────────────────────────
+
+    def get_spread(self, symbol: str = XAUUSD) -> float:
+        return self._spread_cache.get(
+            symbol, PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])["default_spread"])
+
+    # ── Raw fetch ──────────────────────────────────────────────────
+
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        if start is not None or end is not None:
+            return self._fetch_oanda_candles(
+                symbol, timeframe, outputsize, start, end)
+        return self._fetch_oanda_candles(symbol, timeframe, outputsize)
+
+    def _to_oanda_instrument(self, symbol: str) -> str:
+        if symbol in self.OANDA_INSTRUMENT:
+            return self.OANDA_INSTRUMENT[symbol]
+        return symbol.replace("/", "_")
+
+    def _fetch_oanda_candles(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        count: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        instrument = self._to_oanda_instrument(symbol)
+        granularity = self.OANDA_GRANULARITY.get(timeframe)
+        if granularity is None:
+            raise OandaDataError(
+                f"Unsupported timeframe {timeframe} for Oanda")
+
+        url = f"{self._base_url}/v3/instruments/{instrument}/candles"
+        params = {
+            "granularity": granularity,
+            "price": "MBA",
+            "dailyAlignment": 0,
+            "alignmentTimezone": "Etc/UTC",
+        }
+        # v20 rejects `count` combined with both `from` and `to`; a date
+        # window replaces it (used by fetch_range pagination).
+        if start is not None and end is not None:
+            params["from"] = pd.Timestamp(start).isoformat()
+            params["to"] = pd.Timestamp(end).isoformat()
+        else:
+            params["count"] = min(count, self.MAX_BARS_PER_REQUEST)
+
+        try:
+            resp = self._session.get(url, params=params, timeout=30)
+        except requests.RequestException as e:
+            raise OandaDataError(
+                f"Oanda candle fetch failed for {instrument}: {e}") from e
+
+        if resp.status_code != 200:
+            raise OandaDataError(
+                f"Oanda candle fetch failed for {instrument}: "
+                f"HTTP {resp.status_code}: {resp.text[:300]}")
+
+        try:
+            data = resp.json()
+        except ValueError as e:
+            raise OandaDataError(
+                f"Oanda candle response for {instrument} is not valid "
+                f"JSON: {e}") from e
+        candles = data.get("candles", [])
+        if not candles:
+            raise OandaDataError(
+                f"Empty candle response from Oanda for {instrument}")
+
+        rows = []
+        last_bid_close = None
+        last_ask_close = None
+        for c in candles:
+            if not c.get("complete", False):
+                continue
+            mid = c.get("mid", {})
+            bid = c.get("bid", {})
+            ask = c.get("ask", {})
+            rows.append({
+                "Datetime": pd.Timestamp(c["time"]).tz_convert("UTC"),
+                "Open": float(mid.get("o", 0)),
+                "High": float(mid.get("h", 0)),
+                "Low": float(mid.get("l", 0)),
+                "Close": float(mid.get("c", 0)),
+                "Volume": int(c.get("volume", 0)),
+            })
+            if bid and ask:
+                last_bid_close = float(bid.get("c", 0))
+                last_ask_close = float(ask.get("c", 0))
+
+        if not rows:
+            raise OandaDataError(
+                f"No complete candles from Oanda for {instrument}")
+
+        if last_bid_close and last_ask_close:
+            self._spread_cache[symbol] = round(
+                last_ask_close - last_bid_close, 4)
+
+        df = pd.DataFrame(rows)
+        df = df.set_index("Datetime")
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        df.index.name = "Datetime"
+
+        for col in ("Open", "High", "Low", "Close"):
+            df[col] = pd.to_numeric(df[col], errors="coerce")
+        df["Volume"] = pd.to_numeric(df["Volume"], errors="coerce").fillna(0).astype(float)
+
+        logger.debug(
+            "Fetched %d bars for %s @ %s from Oanda",
+            len(df), instrument, granularity)
+        return df

providers/tiingo.py

@@ -0,0 +1,200 @@
+"""
+ForexBot2 — Tiingo data provider
+=================================
+OHLCV candles from Tiingo's FX endpoint
+(`/tiingo/fx/{ticker}/prices`, resampled mid-price bars, no volume).
+All shared behavior (cache, validation, boundary guarantees, range
+pagination) lives in `BaseDataProvider`; this module only maps
+symbols/resample frequencies, sizes the request window, and parses
+Tiingo's JSON into the normalized frame.
+
+Every failure mode raises `TiingoDataError` (a `DataIntegrityError`)
+with a specific message — the scan loops log it and carry on; nothing
+fails silently. NOTE: this is the CANDLE feed only. Tiingo's free
+`/tiingo/fx/top` quote endpoint publishes indicative two-tier spreads
+and must not be used for spread display (removed 2026-07-23) —
+`get_spread` deliberately stays on the base-class default.
+"""
+
+import logging
+from datetime import datetime
+from typing import Optional
+
+import pandas as pd
+import requests
+
+from app_config import provider_api_key
+from config import PlatformConfig, Timeframe
+
+from providers.base import BaseDataProvider, DataIntegrityError
+
+logger = logging.getLogger(__name__)
+
+
+class TiingoDataError(DataIntegrityError):
+    """Raised when a Tiingo data fetch fails or returns invalid data."""
+
+
+class TiingoDataProvider(BaseDataProvider):
+    """Tiingo FX REST candle provider."""
+
+    name = "tiingo"
+    DATA_ERROR = TiingoDataError
+
+    BASE_URL = "https://api.tiingo.com"
+
+    TIINGO_RESAMPLE = {
+        Timeframe.M1: "1min",
+        Timeframe.M5: "5min",
+        Timeframe.M15: "15min",
+        Timeframe.M30: "30min",
+        Timeframe.H1: "1hour",
+        Timeframe.H4: "4hour",
+        Timeframe.D1: "1day",
+        # W1 intentionally unmapped: the FX endpoint has no weekly
+        # resample — requesting it raises loudly instead of guessing.
+    }
+
+    TIINGO_TICKER = {
+        "XAU/USD": "xauusd",
+    }
+
+    # Window padding over the theoretical bar span: FX is closed ~2/7 of
+    # the week (Fri 22:00 → Sun 22:00 UTC), so a calendar window must be
+    # wider than `bars × candle_seconds` to yield the requested count.
+    WINDOW_PAD_FACTOR = 1.6
+
+    def __init__(self, config: PlatformConfig, api_key: Optional[str] = None):
+        super().__init__(config)
+        # Credentials default to env TIINGO_API_KEY → config.json
+        # data_providers.tiingo.api_key, so the provider registry can
+        # build this class with config only.
+        if api_key is None:
+            api_key = provider_api_key(provider="tiingo")
+        self._api_key = api_key or None
+        if self._api_key is None:
+            logger.warning(
+                "Tiingo provider built WITHOUT an API key — every fetch "
+                "will fail with a named error. Set the TIINGO_API_KEY env "
+                "var or data_providers.tiingo.api_key in config.json.")
+        self._session = requests.Session()
+        self._session.headers.update({
+            "Content-Type": "application/json",
+            # Header auth keeps the key out of URLs (and out of logs).
+            "Authorization": f"Token {api_key}" if api_key else "",
+        })
+
+    def _to_tiingo_ticker(self, symbol: str) -> str:
+        if symbol in self.TIINGO_TICKER:
+            return self.TIINGO_TICKER[symbol]
+        return symbol.replace("/", "").replace("=X", "").lower()
+
+    # ── Raw fetch ──────────────────────────────────────────────────
+
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        if self._api_key is None:
+            raise TiingoDataError(
+                "No Tiingo API key. Set the TIINGO_API_KEY env var or "
+                "data_providers.tiingo.api_key in config.json.")
+
+        resample = self.TIINGO_RESAMPLE.get(timeframe)
+        if resample is None:
+            raise TiingoDataError(
+                f"Unsupported timeframe {timeframe} for Tiingo "
+                f"(no FX resample frequency)")
+
+        ticker = self._to_tiingo_ticker(symbol)
+
+        # Tiingo takes date windows, not bar counts: derive the start
+        # from the requested count for live fetches.
+        if start is None:
+            span = self.CANDLE_SECONDS[timeframe] * (outputsize + 10)
+            start_ts = (pd.Timestamp.now(tz="UTC")
+                        - pd.Timedelta(seconds=span * self.WINDOW_PAD_FACTOR))
+        else:
+            start_ts = pd.Timestamp(start)
+
+        params = {
+            "resampleFreq": resample,
+            "startDate": start_ts.strftime("%Y-%m-%dT%H:%M:%S"),
+        }
+        if end is not None:
+            params["endDate"] = pd.Timestamp(end).strftime("%Y-%m-%dT%H:%M:%S")
+
+        url = f"{self.BASE_URL}/tiingo/fx/{ticker}/prices"
+        try:
+            resp = self._session.get(url, params=params, timeout=30)
+        except requests.RequestException as e:
+            raise TiingoDataError(
+                f"Tiingo candle fetch failed for {ticker}: {e}") from e
+
+        if resp.status_code == 429:
+            raise TiingoDataError(
+                f"Tiingo rate limit hit for {ticker} (HTTP 429) — "
+                f"plan quota exhausted; scan will retry next boundary")
+        if resp.status_code in (401, 403):
+            raise TiingoDataError(
+                f"Tiingo auth failed for {ticker} (HTTP {resp.status_code}) "
+                f"— check TIINGO_API_KEY / config.json api_key")
+        if resp.status_code != 200:
+            raise TiingoDataError(
+                f"Tiingo candle fetch failed for {ticker}: "
+                f"HTTP {resp.status_code}: {resp.text[:300]}")
+
+        try:
+            data = resp.json()
+        except ValueError as e:
+            raise TiingoDataError(
+                f"Tiingo candle response for {ticker} is not valid "
+                f"JSON: {e}") from e
+
+        if not isinstance(data, list) or not data:
+            raise TiingoDataError(
+                f"Empty candle response from Tiingo for {ticker} "
+                f"(window {params['startDate']} → "
+                f"{params.get('endDate', 'now')}; check ticker and plan "
+                f"history limits)")
+
+        rows = []
+        for item in data:
+            if "date" not in item:
+                raise TiingoDataError(
+                    f"Malformed Tiingo candle for {ticker}: missing "
+                    f"'date' in {str(item)[:120]}")
+            rows.append({
+                "Datetime": pd.Timestamp(item["date"]),
+                "Open": item.get("open"),
+                "High": item.get("high"),
+                "Low": item.get("low"),
+                "Close": item.get("close"),
+                # FX endpoint has no volume; 0.0 keeps the engine contract.
+                "Volume": 0.0,
+            })
+
+        df = pd.DataFrame(rows)
+        df = df.set_index("Datetime")
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        else:
+            df.index = df.index.tz_convert("UTC")
+        df.index.name = "Datetime"
+
+        for col in ("Open", "High", "Low", "Close"):
+            df[col] = pd.to_numeric(df[col], errors="coerce")
+        if df["Close"].isna().all():
+            raise TiingoDataError(
+                f"Tiingo response for {ticker} has no numeric closes — "
+                f"feed format changed?")
+
+        logger.debug(
+            "Fetched %d bars for %s @ %s from Tiingo",
+            len(df), ticker, resample)
+        return df

providers/twelve_data.py

@@ -0,0 +1,238 @@
+"""
+ForexBot2 — Twelve Data provider
+=================================
+OHLCV via the official `twelvedata` SDK plus streaming quotes over the
+Twelve Data WebSocket. All shared behavior (cache, validation, boundary
+guarantees, range pagination) lives in `BaseDataProvider`.
+"""
+
+import asyncio
+import json
+import logging
+import threading
+import time
+from collections import deque
+from datetime import datetime
+from typing import AsyncIterator, Callable, Optional, Sequence, Tuple
+
+import pandas as pd
+from twelvedata import TDClient
+
+from app_config import provider_api_key
+from config import PAIR_INFO, PlatformConfig, Timeframe
+
+from providers.base import BaseDataProvider, DataIntegrityError
+
+logger = logging.getLogger(__name__)
+
+
+class TwelveDataProvider(BaseDataProvider):
+    """Twelve Data SDK client with credit throttling and WS quotes."""
+
+    name = "twelve_data"
+
+    TD_INTERVALS = {
+        Timeframe.M1:  "1min",
+        Timeframe.M5:  "5min",
+        Timeframe.M15: "15min",
+        Timeframe.M30: "30min",
+        Timeframe.H1:  "1h",
+        Timeframe.H4:  "4h",
+        Timeframe.D1:  "1day",
+        Timeframe.W1:  "1week",
+    }
+
+    RATE_LIMIT = 55
+    RATE_WINDOW = 60.0
+
+    WS_URL = "wss://ws.twelvedata.com/v1/quotes/price?apikey={api_key}"
+
+    def __init__(self, config: PlatformConfig):
+        super().__init__(config)
+        self._api_key = provider_api_key(provider="twelve_data") or None
+        self._throttle_lock = threading.Lock()
+        self._request_times: deque = deque()
+        self._td: Optional[TDClient] = (
+            TDClient(apikey=self._api_key) if self._api_key else None
+        )
+
+    @staticmethod
+    def _to_td_symbol(symbol: str) -> str:
+        if "/" in symbol:
+            return symbol
+        clean = symbol.replace("=X", "")
+        return f"{clean[:3]}/{clean[3:]}"
+
+    # ── Raw fetch ──────────────────────────────────────────────────
+
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        return self._fetch_twelve_data(
+            symbol, timeframe, outputsize,
+            start_date=start.strftime("%Y-%m-%d %H:%M:%S") if start else None,
+            end_date=end.strftime("%Y-%m-%d %H:%M:%S") if end else None,
+        )
+
+    def _throttle(self, credits: int = 1) -> None:
+        """Pro tier: 55 credits / minute. Block until budget allows."""
+        while True:
+            with self._throttle_lock:
+                now = time.time()
+                while self._request_times and now - self._request_times[0] >= self.RATE_WINDOW:
+                    self._request_times.popleft()
+
+                available = self.RATE_LIMIT - len(self._request_times)
+                if available >= credits:
+                    for _ in range(credits):
+                        self._request_times.append(time.time())
+                    return
+
+                used = len(self._request_times)
+                wait = (self._request_times[0] + self.RATE_WINDOW) - now + 0.2
+            logger.info(
+                f"Rate limit: {used}/{self.RATE_LIMIT} credits used "
+                f"— waiting {wait:.1f}s..."
+            )
+            time.sleep(wait)
+
+    def _fetch_twelve_data(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start_date: Optional[str] = None,
+        end_date: Optional[str] = None,
+    ) -> pd.DataFrame:
+        if self._td is None:
+            raise DataIntegrityError(
+                "No data-provider API key. Set the TWELVE_DATA_API_KEY env "
+                "var or data_providers.twelve_data.api_key in config.json."
+            )
+
+        td_symbol = self._to_td_symbol(symbol)
+        interval = self.TD_INTERVALS[timeframe]
+
+        self._throttle(credits=1)
+
+        kwargs = {
+            "symbol": td_symbol,
+            "interval": interval,
+            "outputsize": min(outputsize, self.MAX_BARS_PER_REQUEST),
+            "timezone": "UTC",
+        }
+        if start_date:
+            kwargs["start_date"] = start_date
+        if end_date:
+            kwargs["end_date"] = end_date
+
+        try:
+            df = self._td.time_series(**kwargs).as_pandas()
+        except Exception as e:
+            raise DataIntegrityError(
+                f"Twelve Data fetch failed for {td_symbol}: {e}"
+            ) from e
+
+        if df is None or df.empty:
+            raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
+
+        # SDK returns lowercase columns; engine expects capitalised.
+        df = df.rename(columns={
+            "open": "Open", "high": "High", "low": "Low",
+            "close": "Close", "volume": "Volume",
+        })
+        for col in ("Open", "High", "Low", "Close", "Volume"):
+            if col in df.columns:
+                df[col] = pd.to_numeric(df[col], errors="coerce")
+        if "Volume" not in df.columns:
+            df["Volume"] = 0.0
+
+        # SDK returns newest-first; engine + indicators want oldest-first.
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        df.index.name = "Datetime"
+
+        logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
+        return df
+
+    # ── Streaming quotes ───────────────────────────────────────────
+
+    supports_streaming = True
+
+    @staticmethod
+    def _parse_ws_price(raw: str) -> Optional[Tuple[str, float]]:
+        """Parse one WS message; return (ws_symbol, price) for price
+        events, None for everything else (logged where noteworthy)."""
+        try:
+            msg = json.loads(raw)
+        except json.JSONDecodeError:
+            logger.warning("WebSocket: malformed JSON, skipping")
+            return None
+
+        event = msg.get("event")
+        if event == "subscribe-status":
+            fails = msg.get("fails") or []
+            if fails:
+                logger.warning(f"WebSocket: failed to subscribe: {fails}")
+            return None
+        if event != "price":
+            return None
+
+        ws_symbol = msg.get("symbol")
+        try:
+            price = float(msg["price"])
+        except (KeyError, ValueError, TypeError) as e:
+            logger.warning(f"WebSocket: bad price data for {ws_symbol}: {e}")
+            return None
+        return ws_symbol, price
+
+    @staticmethod
+    async def _ws_heartbeat(ws) -> None:
+        """Twelve Data drops idle WS connections; heartbeat every 10s keeps
+        it alive through quiet periods (weekends, low-liquidity hours)."""
+        while True:
+            await asyncio.sleep(10)
+            await ws.send(json.dumps({"action": "heartbeat"}))
+
+    async def iter_ticks(
+        self,
+        symbols: Sequence[str],
+        on_connect: Optional[Callable[[], None]] = None,
+    ) -> AsyncIterator[Tuple[str, float]]:
+        """One connection's lifetime of (canonical_symbol, price) ticks.
+
+        The TD WS speaks display symbols ("XAU/USD"); ticks are mapped back
+        to the canonical pair symbols passed in. The caller owns the
+        reconnect/backoff loop around this generator.
+        """
+        import websockets
+
+        ws_to_pair = {PAIR_INFO[s]["display"]: s for s in symbols}
+        url = self.WS_URL.format(api_key=self._api_key or "")
+        async with websockets.connect(url) as ws:
+            subscribe = {
+                "action": "subscribe",
+                "params": {"symbols": ",".join(ws_to_pair)},
+            }
+            await ws.send(json.dumps(subscribe))
+            logger.info(f"WebSocket connected, subscribed to {len(ws_to_pair)} symbols")
+            if on_connect is not None:
+                on_connect()
+            hb_task = asyncio.create_task(self._ws_heartbeat(ws))
+            try:
+                async for raw in ws:
+                    parsed = self._parse_ws_price(raw)
+                    if parsed is None:
+                        continue
+                    pair_symbol = ws_to_pair.get(parsed[0])
+                    if pair_symbol is None:
+                        continue
+                    yield pair_symbol, parsed[1]
+            finally:
+                hb_task.cancel()

signal_aggregator.py

@@ -15,7 +15,7 @@ import pandas as pd
 
 from chart_patterns import detect_chart_pattern
 from config import PlatformConfig, SignalStrength, Timeframe
-from data_provider import ForexDataProvider
+from providers import BaseDataProvider, make_data_provider
 from engine import TechnicalEngine, TechnicalAnalysis
 from news_engine import NewsEngine, FundamentalContext
 
@@ -50,6 +50,12 @@ class TradeSetup:
     pip_value: float
     # Entry archetype ("GENERIC", "XAU_PB") — selects trail params.
     strategy: str = "GENERIC"
+    # Trailing exit params stamped from the PRODUCING config, so per-TF
+    # factory overrides (e.g. M15 arm 0.70, M5 dist 0.85) reach every
+    # consumer. None = legacy/unstamped setup — formatters fall back to
+    # platform defaults; 0.0 means trailing is genuinely disabled.
+    trail_arm_r: Optional[float] = None
+    trail_dist_r: Optional[float] = None
 
 
 @dataclass
@@ -104,7 +110,7 @@ class SignalAggregator:
 
     def __init__(self, config: PlatformConfig):
         self.config = config
-        self.data_provider = ForexDataProvider(config)
+        self.data_provider = make_data_provider(config)
         self.tech_engine = TechnicalEngine(config.technical)
         self._news_engine: Optional[NewsEngine] = None  # lazy
 
@@ -174,7 +180,7 @@ class SignalAggregator:
             try:
                 df = self.data_provider.fetch_ohlcv(symbol, tf)
                 dfs[tf] = df
-                if not self.data_provider._is_data_fresh(df, symbol, tf):
+                if not self.data_provider.is_data_fresh(df, symbol, tf):
                     stale_tfs.append(tf.value)
             except Exception as e:
                 logger.warning(f"Fetch failed for {symbol} {tf.value}: {e}")
@@ -208,7 +214,7 @@ class SignalAggregator:
                 bar_close = (
                     pd.Timestamp(report.timestamp)
                     + pd.Timedelta(
-                        seconds=ForexDataProvider.CANDLE_SECONDS[
+                        seconds=BaseDataProvider.CANDLE_SECONDS[
                             self.config.primary_timeframe
                         ]
                     )
@@ -228,7 +234,12 @@ class SignalAggregator:
                     report.action_recommendation = "NO TRADE — Outside tradeable session"
                     report.trade_setup = None
             except Exception:
-                pass
+                # Fail-open by design (an unparseable timestamp shouldn't
+                # kill the scan) but never silently: this filter guards
+                # real money hours.
+                logger.exception(
+                    f"{symbol}: live session filter errored — filter SKIPPED this scan"
+                )
 
         return report
 
@@ -538,6 +549,7 @@ class SignalAggregator:
         min_tp = (self.config.signals.round_trip_cost
                   * self.config.signals.min_tp_cost_multiple)
         if 1.5 * sl_dist < min_tp:
+            _reject("spread_gate")
             return
 
         report.trade_setup = TradeSetup(
@@ -553,7 +565,10 @@ class SignalAggregator:
             reward_pips=1.5 * sl_dist / pip_value,
             pip_value=pip_value,
             strategy="XAU_PB",
+            trail_arm_r=self.config.signals.pb_trail_arm_r,
+            trail_dist_r=self.config.signals.pb_trail_dist_r,
         )
+        _reject("accepted")   # funnel tick, mirrors the CP path
         if report.signal_quality not in ("A+", "A", "B"):
             report.signal_quality = "B"
         report.final_signal = (SignalStrength.BUY if is_buy
@@ -622,13 +637,17 @@ class SignalAggregator:
                     close_h = (
                         pd.Timestamp(report.timestamp)
                         + pd.Timedelta(
-                            seconds=ForexDataProvider.CANDLE_SECONDS[
+                            seconds=BaseDataProvider.CANDLE_SECONDS[
                                 self.config.primary_timeframe
                             ]
                         )
                     ).hour
                 except Exception:
                     close_h = None
+                    logger.warning(
+                        f"{report.symbol}: bar-close hour underivable — "
+                        f"SELL hour gate skipped"
+                    )
                 if close_h in cfg.sell_blocked_hours_utc:
                     _reject("sell_hour")
                     return
@@ -683,6 +702,8 @@ class SignalAggregator:
             reward_pips=height / pip_value,
             pip_value=pip_value,
             strategy="XAU_CP",
+            trail_arm_r=self.config.signals.trail_arm_r,
+            trail_dist_r=self.config.signals.trail_dist_r,
         )
         if report.signal_quality not in ("A+", "A", "B"):
             report.signal_quality = "B"
@@ -814,7 +835,10 @@ class SignalAggregator:
                         f"DAY FILTER: {day_names.get(bar_wd, '?')} not in allowed days"
                     )
             except Exception:
-                pass
+                # Fail-open by design, but never silently (see live wrapper).
+                logger.exception(
+                    f"{symbol}: session/day filter errored — filter SKIPPED"
+                )
 
         # ── 6c. Transitioning regime quality boost ──────────────────
         if report.market_regime == "transitioning" and report.signal_quality in ("B", "C", "D"):
@@ -915,13 +939,17 @@ class SignalAggregator:
                     close_h = (
                         pd.Timestamp(report.timestamp)
                         + pd.Timedelta(
-                            seconds=ForexDataProvider.CANDLE_SECONDS[
+                            seconds=BaseDataProvider.CANDLE_SECONDS[
                                 self.config.primary_timeframe
                             ]
                         )
                     ).hour
                 except Exception:
                     close_h = None
+                    logger.warning(
+                        f"{report.symbol}: bar-close hour underivable — "
+                        f"SELL hour gate skipped"
+                    )
                 if close_h in self.config.xau_strategy.sell_blocked_hours_utc:
                     report.warnings.append(
                         f"SELL FILTER: {close_h:02}:00 UTC entry blocked for shorts"
@@ -1048,6 +1076,8 @@ class SignalAggregator:
             risk_pips=round(sl_distance / pip_value, 1),
             reward_pips=round(tp_distance / pip_value, 1),
             pip_value=pip_value,
+            trail_arm_r=self.config.signals.trail_arm_r,
+            trail_dist_r=self.config.signals.trail_dist_r,
         )
 
     def _check_mtf_agreement(self, report: SignalReport) -> bool:
@@ -1165,6 +1195,11 @@ class SignalAggregator:
 
             atr = primary.key_levels.get("atr", 0)
             if atr > 0 and score != 0:
+                # Distance is measured from the FARTHEST of the 3 tracked
+                # swings (sorted()[0] = deepest low / highest high), so the
+                # penalty fires earlier than a nearest-swing rule would.
+                # Calibrated as-is in every validated book — changing the
+                # anchor is a signal-logic change requiring re-validation.
                 if score > 0:
                     swing_lows = sorted(
                         v for k, v in primary.key_levels.items() if k.startswith("swing_low_")
@@ -1445,6 +1480,8 @@ class SignalAggregator:
             risk_pips=risk_pips,
             reward_pips=reward_pips,
             pip_value=pip_value,
+            trail_arm_r=self.config.signals.trail_arm_r,
+            trail_dist_r=self.config.signals.trail_dist_r,
         )
 
     def _collect_reasons(self, report: SignalReport):

telegram_bot.py

@@ -38,19 +38,16 @@ Usage:
 
 import asyncio
 import functools
-import json
 import logging
 import logging.handlers
 import os
-import signal
 import sys
+import threading
 import time
 from concurrent.futures import ThreadPoolExecutor
 from datetime import datetime, timezone
 from typing import Dict, List, Optional, Set
 
-import websockets
-
 from telegram import InlineKeyboardButton, InlineKeyboardMarkup, Update
 from telegram.constants import ParseMode
 from telegram.error import BadRequest
@@ -65,10 +62,16 @@ from config import (
     PlatformConfig, SignalStrength, Timeframe,
     XAUUSD, PAIR_INFO,
 )
-from data_provider import ForexDataProvider
+from app_config import (
+    get_app_config, telegram_token, telegram_chat_id,
+    telegram_channel_id,
+)
+from providers import BaseDataProvider, make_data_provider
 from pair_signals import PAIR_PRODUCERS
 from signal_aggregator import SignalAggregator, SignalReport
 
+_APP = get_app_config()
+
 # ── Force UTF-8 on Windows ────────────────────────────────────────────
 if hasattr(sys.stdout, "reconfigure"):
     sys.stdout.reconfigure(encoding="utf-8")
@@ -76,22 +79,29 @@ if hasattr(sys.stdout, "reconfigure"):
 _LOG_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "logs")
 os.makedirs(_LOG_DIR, exist_ok=True)
 _log_formatter = logging.Formatter(
-    "%(asctime)s [%(levelname)s] %(name)s: %(message)s", datefmt="%H:%M:%S"
+    "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+    datefmt="%Y-%m-%d %H:%M:%S",
 )
 _console_handler = logging.StreamHandler()
 _console_handler.setFormatter(_log_formatter)
+# 3-hour rotation × 240 backups = 30 days retention. Handlers sit on the
+# ROOT logger (force=True guarantees ownership even if something logged
+# first), so every module logger — data_provider, news_engine,
+# signal_aggregator, websockets, telegram.ext — lands in the same file.
 _file_handler = logging.handlers.TimedRotatingFileHandler(
     os.path.join(_LOG_DIR, "telegram.log"),
-    when="midnight", backupCount=30, encoding="utf-8",
+    when="h", interval=3, backupCount=240, encoding="utf-8",
 )
 _file_handler.setFormatter(_log_formatter)
-logging.basicConfig(level=logging.INFO, handlers=[_console_handler, _file_handler])
+logging.basicConfig(level=getattr(logging, _APP.log_level, logging.INFO),
+                    handlers=[_console_handler, _file_handler],
+                    force=True)
 logging.getLogger("httpx").setLevel(logging.WARNING)
 logging.getLogger("httpcore").setLevel(logging.WARNING)
 logger = logging.getLogger("telegram_bot")
 
-TOKEN = os.environ.get("TELEGRAM_TOKEN", "8228251480:AAFW2OAyQ2idMicql2t_CIXk1r9-qfRTfNE")
-CHAT_ID = os.environ.get("TELEGRAM_CHAT_ID", "6025899022")
+TOKEN = telegram_token()
+CHAT_ID = telegram_chat_id()
 
 # Owner lock: only this user/chat may drive the bot (commands + buttons).
 # The bot is the product's signal source — without the lock, anyone who
@@ -102,21 +112,18 @@ OWNER_ID: Optional[int] = int(CHAT_ID) if CHAT_ID else None
 # Telegram). When set, auto-scan signal alerts and level-breach alerts
 # are also posted there. Manual scans, status, cache and boot messages
 # stay owner-DM only. Set TELEGRAM_CHANNEL_ID="" to disable the feed.
-_CHANNEL_ID_RAW = os.environ.get("TELEGRAM_CHANNEL_ID", "-1003942265371")
+_CHANNEL_ID_RAW = telegram_channel_id()
 CHANNEL_ID: Optional[int] = int(_CHANNEL_ID_RAW) if _CHANNEL_ID_RAW else None
 
 # ── Pairs & buttons ───────────────────────────────────────────────────
-# The scanned pair list is config-driven: PlatformConfig.scan_pairs is
-# the single switch for single- vs multi-pair operation. Buttons, WS
-# subscriptions, and scan loops all derive from it.
+# The scanned pair list is config.json-driven (`scanning.pairs`, read via
+# app_config) and is the single switch for single- vs multi-pair operation.
+# Buttons, WS subscriptions, and scan loops all derive from it.
 
-SCAN_PAIRS = list(PlatformConfig().scan_pairs)
+SCAN_PAIRS = list(_APP.scan_pairs)
 
 _PAIRS_LABEL = " + ".join(PAIR_INFO[p]["display"] for p in SCAN_PAIRS)
 
-_WS_TO_PAIR = {PAIR_INFO[p]["display"]: p for p in SCAN_PAIRS}
-_WS_SYMBOLS = list(_WS_TO_PAIR.keys())
-
 _LEVEL_KEYS = (
     "swing_high_1", "swing_high_2", "swing_high_3",
     "swing_low_1", "swing_low_2", "swing_low_3",
@@ -186,13 +193,13 @@ def _fmt_price(price: float) -> str:
 
 # ── Shared data provider (single cache across all scans) ─────────────
 
-_shared_dp: Optional[ForexDataProvider] = None
+_shared_dp: Optional[BaseDataProvider] = None
 
 
-def _get_shared_dp() -> ForexDataProvider:
+def _get_shared_dp() -> BaseDataProvider:
     global _shared_dp
     if _shared_dp is None:
-        _shared_dp = ForexDataProvider(PlatformConfig())
+        _shared_dp = make_data_provider(PlatformConfig())
     return _shared_dp
 
 
@@ -210,15 +217,6 @@ _QUALITY_EMOJI = {
     "A+": "🏆", "A": "🥇", "B": "🥈", "C": "🥉", "D": "⚠️", "F": "❌",
 }
 
-_SIGNAL_TAG = {
-    SignalStrength.STRONG_BUY:  "Strong Buy",
-    SignalStrength.BUY:         "Bullish",
-    SignalStrength.NEUTRAL:     "Neutral",
-    SignalStrength.SELL:        "Bearish",
-    SignalStrength.STRONG_SELL: "Strong Sell",
-}
-
-
 def _signal_label(sig: SignalStrength) -> str:
     return sig.name.replace("_", " ")
 
@@ -233,10 +231,36 @@ def _mtf_line(report: SignalReport) -> str:
     return " | ".join(parts)
 
 
-def _indicator_line(name: str, ind) -> str:
-    tag = _SIGNAL_TAG.get(ind.signal, "Neutral")
-    short = ind.description.split(" | ")[0] if " | " in ind.description else ind.description
-    return f"[{tag}] *{_escape_md(ind.name)}*: {_escape_md(short)}"
+# Informational fixed-pip target shown on every trade setup (display-only,
+# not part of the backtested exit model).
+_INFO_TP_PIPS = 100
+
+_IND_SHORT = {
+    "ema_trend": "EMA", "rsi": "RSI", "macd": "MACD", "bollinger": "BB",
+    "adx": "ADX", "roc": "ROC", "keltner": "KC", "divergence": "Div",
+    "candlestick": "Cndl",
+}
+
+_IND_ARROW = {
+    SignalStrength.STRONG_BUY:  "⇈",
+    SignalStrength.BUY:         "↑",
+    SignalStrength.NEUTRAL:     "→",
+    SignalStrength.SELL:        "↓",
+    SignalStrength.STRONG_SELL: "⇊",
+}
+
+
+def _indicator_summary(report: SignalReport) -> str:
+    """One-line condensed indicator readout: 'EMA↑ RSI↑ MACD→ …'."""
+    if not report.primary_analysis:
+        return ""
+    parts = []
+    for name, ind in report.primary_analysis.indicators.items():
+        if name.startswith("_") or ind.weight == 0:
+            continue
+        label = _IND_SHORT.get(name, name)
+        parts.append(f"{label}{_IND_ARROW.get(ind.signal, '→')}")
+    return " ".join(parts)
 
 
 def _live_price_line(symbol: str) -> Optional[str]:
@@ -263,46 +287,26 @@ def format_signal_message(report: SignalReport) -> str:
     emoji = _SIGNAL_EMOJI.get(sig, "⬜")
     qe = _QUALITY_EMOJI.get(report.signal_quality, "")
     label = _signal_label(sig)
-    mtf_tag = "✅ MTF aligned" if report.mtf_agreement else "❌ MTF diverged"
-
-    lines: list[str] = []
+    mtf_tag = "MTF ✅" if report.mtf_agreement else "MTF ❌"
+    tf = report.primary_analysis.timeframe if report.primary_analysis else ""
 
     now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
     regime_str = _escape_md(report.market_regime.replace("_", " ").title())
     vol_str = _escape_md(str(report.volatility_regime))
-    lines += [
-        f"🔍 *{report.display_name} Signal Analysis*  ({report.primary_analysis.timeframe if report.primary_analysis else ''})",
-        f"🕐 {now}",
-        "",
-        f"💰 Close: *{_fmt_price(report.current_price)}*",
+
+    lines: list[str] = [
+        f"{emoji} *{label} · {report.display_name} · {tf}*  {qe}{report.signal_quality}",
+        f"🕐 {now} · 💰 {_fmt_price(report.current_price)}",
     ]
     live = _live_price_line(report.symbol)
     if live:
         lines.append(live)
     lines += [
-        f"📊 Regime: {regime_str} | Vol: {vol_str}",
-        "",
-        "━━━━━━━━━━━━━━━━━━━━",
-        f"{emoji} *{label}*  {qe}{report.signal_quality}  |  Score: {report.final_score:+.1f}  |  {report.final_confidence:.0f}% conf",
-        f"_{_escape_md(report.action_recommendation)}_",
-        f"{mtf_tag}",
+        f"Score *{report.final_score:+.1f}* · Conf *{report.final_confidence:.0f}%* · {mtf_tag}",
+        f"📊 Regime: {regime_str} · Vol: {vol_str}",
     ]
-
-    mtf = _mtf_line(report)
-    if mtf:
-        lines += ["", f"📈 *MTF*: {mtf}"]
-
-    if report.primary_analysis:
-        lines += ["", "━━━━━━━━━━━━━━━━━━━━", "📋 *Indicators*"]
-        for name, ind in report.primary_analysis.indicators.items():
-            if name.startswith("_") or ind.weight == 0:
-                continue
-            lines.append(_indicator_line(name, ind))
-
-    if report.warnings:
-        lines.append("")
-        for w in report.warnings:
-            lines.append(f"⚠️ {_escape_md(w)}")
+    if report.action_recommendation:
+        lines.append(f"_{_escape_md(report.action_recommendation)}_")
 
     if report.trade_setup and sig not in (SignalStrength.NEUTRAL,):
         ts = report.trade_setup
@@ -311,26 +315,42 @@ def format_signal_message(report: SignalReport) -> str:
         reward_usd = ts.reward_pips * ts.pip_value
         lines += [
             "",
-            "━━━━━━━━━━━━━━━━━━━━",
-            f"🚀 *TRADE SETUP*",
+            "🚀 *TRADE SETUP*",
             f"{dir_emoji} Direction: *{ts.direction}*",
-            f"▶ Entry:  `{_fmt_price(ts.entry_price)}`",
-            f"🛑 SL:     `{_fmt_price(ts.stop_loss)}`  (${risk_usd:.2f} / {ts.risk_pips:.0f} pips)",
-            f"🎯 TP1:    `{_fmt_price(ts.take_profit_1)}`  (${reward_usd:.2f} / {ts.reward_pips:.0f} pips)",
+            f"▶ Entry `{_fmt_price(ts.entry_price)}`",
+            f"🛑 SL   `{_fmt_price(ts.stop_loss)}`  (${risk_usd:.2f} / {ts.risk_pips:.0f}p)",
+            f"🎯 TP1  `{_fmt_price(ts.take_profit_1)}`  (${reward_usd:.2f} / {ts.reward_pips:.0f}p)",
         ]
         if ts.take_profit_2:
-            lines.append(f"🎯 TP2:    `{_fmt_price(ts.take_profit_2)}`")
-        lines.append(f"⚖️  RR:     1:{ts.risk_reward:.1f}")
+            lines.append(f"🎯 TP2  `{_fmt_price(ts.take_profit_2)}`")
+
+        # Informational fixed-pip target: where a 100-pip TP would sit for
+        # this setup (display-only — not part of the backtested exit model).
+        info_dist = _INFO_TP_PIPS * ts.pip_value
+        if ts.direction == "BUY":
+            info_price = ts.entry_price + info_dist
+        else:
+            info_price = ts.entry_price - info_dist
+        risk_dist = abs(ts.entry_price - ts.stop_loss)
+        info_r = f" · ≈{info_dist / risk_dist:.1f}R" if risk_dist > 0 else ""
+        lines.append(
+            f"🎯 TP {_INFO_TP_PIPS}p: `{_fmt_price(info_price)}`  (${info_dist:.2f}{info_r})"
+        )
+        lines.append(f"⚖️ RR 1:{ts.risk_reward:.1f}")
 
         # Trailing-stop management plan (matches backtested exit model:
-        # TP1 is advisory, the trail is the exit). Pullback setups run
-        # tighter stops and arm later.
-        sig_cfg = PlatformConfig().signals
+        # TP1 is advisory, the trail is the exit). Params are stamped on
+        # the setup by the producing config so per-TF factory overrides
+        # (M15 arm 0.70, M5 dist 0.85) are what subscribers see; only a
+        # legacy unstamped setup falls back to platform defaults.
         is_pb = getattr(ts, "strategy", "GENERIC").endswith("_PB")
+        arm_r = getattr(ts, "trail_arm_r", None)
+        dist_r = getattr(ts, "trail_dist_r", None)
+        if arm_r is None or dist_r is None:
+            sig_cfg = PlatformConfig().signals
             arm_r = sig_cfg.pb_trail_arm_r if is_pb else sig_cfg.trail_arm_r
             dist_r = sig_cfg.pb_trail_dist_r if is_pb else sig_cfg.trail_dist_r
         if arm_r > 0:
-            risk_dist = abs(ts.entry_price - ts.stop_loss)
             if ts.direction == "BUY":
                 arm_price = ts.entry_price + arm_r * risk_dist
             else:
@@ -338,12 +358,24 @@ def format_signal_message(report: SignalReport) -> str:
             trail_dist = dist_r * risk_dist
             tag = " (pullback)" if is_pb else ""
             lines.append(
-                f"🔄 Trail{tag}:  once price hits `{_fmt_price(arm_price)}` "
+                f"🔄 Trail{tag}: arm `{_fmt_price(arm_price)}` "
                 f"(+{arm_r:.1f}R), trail SL ${trail_dist:.2f} "
                 f"behind best price — TP1 is a guide, let it run"
             )
 
-    lines += ["", "━━━━━━━━━━━━━━━━━━━━", _DISCLAIMER]
+    extra: list[str] = []
+    mtf = _mtf_line(report)
+    if mtf:
+        extra.append(f"📈 MTF: {mtf}")
+    ind_summary = _indicator_summary(report)
+    if ind_summary:
+        extra.append(f"📋 {ind_summary}")
+    for w in report.warnings:
+        extra.append(f"⚠️ {_escape_md(w)}")
+    if extra:
+        lines += [""] + extra
+
+    lines += ["", _DISCLAIMER]
 
     return "\n".join(lines)
 
@@ -475,12 +507,13 @@ async def run_signal(
 
 _QUALITY_RANK = {"A+": 4, "A": 3, "B": 2, "C": 1}
 _ALERT_QUALITY_CYCLE = ["C", "B", "A"]   # C+ → B+ → A-only
-_alert_min_quality = "C"
+_alert_min_quality = _APP.alert_min_quality
 
 # Fingerprint of the last alert per (symbol, timeframe) so a persisting
 # signal doesn't re-send an identical message every candle.
 _last_alert_fp: Dict[str, tuple] = {}
 _last_alert_ts: Dict[str, float] = {}
+_last_alert_setup: Dict[str, tuple] = {}   # key → (entry, risk) of last ALERTED setup
 _ALERT_REARM_SECONDS = 3600   # identical signal re-alerts after 1h
 
 
@@ -489,40 +522,54 @@ def _passes_quality(quality: str) -> bool:
 
 
 def _alert_fingerprint(report: SignalReport) -> tuple:
-    """Alert identity for dedup. Includes a risk-relative entry bucket:
-    a persisting signal whose entry drifts a few ticks between candles
-    keeps the same fingerprint, while a setup that moved by half its risk
-    distance or more (new pullback, new swing anchor) re-alerts instead
-    of being swallowed for the full re-arm hour."""
+    """Alert identity for dedup: signal, quality, setup direction.
+
+    Entry movement is checked separately in _should_alert against the
+    last ALERTED entry/risk (a stable reference). Deriving a price bucket
+    from the CURRENT SL distance here made the fingerprint churn on
+    normal ATR/anchor drift — near-duplicate alerts every candle."""
     setup = report.trade_setup
-    entry_bucket = None
-    if setup is not None:
-        sl_dist = abs(setup.entry_price - setup.stop_loss)
-        entry_bucket = round(setup.entry_price / max(sl_dist * 0.5, 1e-9))
     return (
         report.final_signal.name,
         report.signal_quality,
         setup.direction if setup else None,
-        entry_bucket,
     )
 
 
 def _should_alert(key: str, report: SignalReport) -> bool:
     """True when this signal is new/changed for the key, or re-armed.
 
+    A persisting identical signal dedups for the re-arm hour, but a setup
+    whose entry moved >= half the last-alerted risk distance (new
+    pullback/swing anchor) re-alerts immediately.
+
     Pure check — call _commit_alert only after the message actually went
     out, so a failed Telegram send doesn't consume the alert for an hour.
     """
     fp = _alert_fingerprint(report)
-    return not (
-        _last_alert_fp.get(key) == fp
-        and time.time() - _last_alert_ts.get(key, 0) < _ALERT_REARM_SECONDS
-    )
+    if _last_alert_fp.get(key) != fp:
+        return True
+    if time.time() - _last_alert_ts.get(key, 0) >= _ALERT_REARM_SECONDS:
+        return True
+    setup = report.trade_setup
+    last = _last_alert_setup.get(key)
+    if setup is not None and last is not None:
+        last_entry, last_risk = last
+        if abs(setup.entry_price - last_entry) >= 0.5 * max(last_risk, 1e-9):
+            return True
+    return False
 
 
 def _commit_alert(key: str, report: SignalReport) -> None:
     _last_alert_fp[key] = _alert_fingerprint(report)
     _last_alert_ts[key] = time.time()
+    setup = report.trade_setup
+    if setup is not None:
+        _last_alert_setup[key] = (
+            setup.entry_price, abs(setup.entry_price - setup.stop_loss)
+        )
+    else:
+        _last_alert_setup.pop(key, None)
 
 
 # ── Candle-boundary scheduler ────────────────────────────────────────
@@ -580,7 +627,8 @@ async def _scan_timeframe(
 
     actionable = [
         r for r in reports
-        if _passes_quality(r.signal_quality)
+        if r.final_signal != SignalStrength.NEUTRAL   # same gate as breach path
+        and _passes_quality(r.signal_quality)
         and _should_alert(f"{r.symbol}_{tf.value}", r)
     ]
     sent_ok: Set[int] = set()
@@ -609,8 +657,6 @@ _last_price_ts: Dict[str, float] = {}
 _scan_cooldowns: Dict[str, float] = {}
 
 _cfg = PlatformConfig()
-_WS_API_KEY = os.environ.get("TWELVE_DATA_API_KEY") or _cfg.twelve_data_api_key
-_WS_URL = f"wss://ws.twelvedata.com/v1/quotes/price?apikey={_WS_API_KEY}"
 _WS_COOLDOWN = _cfg.ws_cooldown_seconds
 
 _breach_tasks: Set[asyncio.Task] = set()
@@ -656,13 +702,13 @@ def _cleanup_cooldowns() -> None:
         del _scan_cooldowns[k]
 
 
-async def _handle_breach(bot, ws_symbol: str, pair_symbol: str, price: float, level_name: str) -> None:
+async def _handle_breach(bot, pair_symbol: str, price: float, level_name: str) -> None:
     now = time.time()
     if now - _scan_cooldowns.get(pair_symbol, 0) < _WS_COOLDOWN:
         return
 
     _scan_cooldowns[pair_symbol] = now
-    display = PAIR_INFO.get(pair_symbol, {}).get("display", ws_symbol)
+    display = PAIR_INFO.get(pair_symbol, {}).get("display", pair_symbol)
     label = level_name.replace("_", " ").title()
     logger.info(f"Level breach: {display} crossed {label} at {price} — rescanning M15")
 
@@ -684,9 +730,9 @@ async def _handle_breach(bot, ws_symbol: str, pair_symbol: str, price: float, le
         f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed "
         f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n"
     )
-    detail = header + format_signal_message(report)
     sent = False
     for chat_id in _signal_targets():
+        detail = header + format_signal_message(report)
         try:
             await bot.send_message(chat_id, detail, parse_mode=ParseMode.MARKDOWN)
             sent = True
@@ -696,69 +742,48 @@ async def _handle_breach(bot, ws_symbol: str, pair_symbol: str, price: float, le
         _commit_alert(f"{pair_symbol}_{Timeframe.M15.value}", report)
 
 
-async def _ws_heartbeat(ws) -> None:
-    """Twelve Data drops idle WS connections; heartbeat every 10s keeps it
-    alive through quiet periods (weekends, low-liquidity hours)."""
-    while True:
-        await asyncio.sleep(10)
-        await ws.send(json.dumps({"action": "heartbeat"}))
+def _breach_task_done(task: asyncio.Task) -> None:
+    """Fire-and-forget breach tasks must still surface their failures —
+    without this, an exception sits unretrieved until GC and never
+    reaches the log file."""
+    _breach_tasks.discard(task)
+    if not task.cancelled() and task.exception() is not None:
+        logger.error("Breach alert task failed", exc_info=task.exception())
 
 
 async def _ws_monitor(bot) -> None:
+    """Level-breach tick monitor. The provider owns the connection and
+    protocol (`iter_ticks`); this loop owns reconnect/backoff and breach
+    dispatch. Providers without streaming quotes simply disable it."""
+    dp = _get_shared_dp()
+    if not dp.supports_streaming:
+        logger.info(
+            "Data provider %r has no streaming quotes — "
+            "level-breach monitor disabled", dp.name)
+        return
+
     backoff = 1
     while _feed_active():
-        hb_task: Optional[asyncio.Task] = None
         try:
-            async with websockets.connect(_WS_URL) as ws:
-                subscribe = {
-                    "action": "subscribe",
-                    "params": {"symbols": ",".join(_WS_SYMBOLS)},
-                }
-                await ws.send(json.dumps(subscribe))
-                logger.info(f"WebSocket connected, subscribed to {len(_WS_SYMBOLS)} symbols")
+            def _on_connect() -> None:
+                nonlocal backoff
                 backoff = 1
-                hb_task = asyncio.create_task(_ws_heartbeat(ws))
 
-                async for raw in ws:
+            async for pair_symbol, price in dp.iter_ticks(
+                    SCAN_PAIRS, on_connect=_on_connect):
                 if not _feed_active():
                     break
 
-                    try:
-                        msg = json.loads(raw)
-                    except json.JSONDecodeError:
-                        logger.warning(f"WebSocket: malformed JSON, skipping")
-                        continue
-
-                    event = msg.get("event")
-                    if event == "subscribe-status":
-                        fails = msg.get("fails") or []
-                        if fails:
-                            logger.warning(f"WebSocket: failed to subscribe: {fails}")
-                        continue
-                    if event != "price":
-                        continue
-
-                    ws_symbol = msg.get("symbol")
-                    pair_symbol = _WS_TO_PAIR.get(ws_symbol)
-                    if not pair_symbol:
-                        continue
-
-                    try:
-                        price = float(msg["price"])
-                    except (KeyError, ValueError, TypeError) as e:
-                        logger.warning(f"WebSocket: bad price data for {ws_symbol}: {e}")
-                        continue
-
                 breached = _check_level_breach(pair_symbol, price)
                 _last_prices[pair_symbol] = price
                 _last_price_ts[pair_symbol] = time.time()
 
                 if breached:
                     task = asyncio.create_task(
-                            _handle_breach(bot, ws_symbol, pair_symbol, price, breached)
+                        _handle_breach(bot, pair_symbol, price, breached)
                     )
                     _breach_tasks.add(task)
-                        task.add_done_callback(_breach_tasks.discard)
+                    task.add_done_callback(_breach_task_done)
 
                 _cleanup_cooldowns()
 
@@ -768,9 +793,6 @@ async def _ws_monitor(bot) -> None:
             logger.exception(f"WebSocket error, reconnecting in {backoff}s")
             await asyncio.sleep(backoff)
             backoff = min(backoff * 2, 60)
-        finally:
-            if hb_task is not None:
-                hb_task.cancel()
 
     logger.info("WebSocket monitor stopped")
 
@@ -785,20 +807,26 @@ async def _candle_loop(bot, tf: Timeframe, interval_minutes: int) -> None:
             if not _feed_active():
                 break
             logger.info(f"{label} auto-scan: firing")
+            try:
                 await _scan_timeframe(
                     bot, tf, AUTOSCAN_CONFIRMATIONS[tf]
                 )
+            except Exception:
+                # One failed cycle must not kill the loop for good —
+                # log it and wait for the next candle boundary.
+                logger.exception(f"{label} auto-scan: cycle failed")
     except asyncio.CancelledError:
         pass
     finally:
         logger.info(f"{label} auto-scan loop stopped")
 
 
+_TF_INTERVAL_MINUTES = {
+    Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.M30: 30,
+    Timeframe.H1: 60, Timeframe.H4: 240, Timeframe.D1: 1440,
+}
 _AUTOSCAN_LOOPS = [
-    (Timeframe.M5,  5),
-    (Timeframe.M15, 15),
-    (Timeframe.H1,  60),
-    (Timeframe.H4,  240),
+    (tf, _TF_INTERVAL_MINUTES[tf]) for tf in _APP.scan_timeframes
 ]
 
 
@@ -814,7 +842,8 @@ def _start_scheduler(bot) -> None:
             _autoscan_tasks.append(
                 asyncio.create_task(_candle_loop(bot, tf, interval))
             )
-        logger.info("Candle-boundary scheduler started (M5/M15/H1/H4)")
+        logger.info("Candle-boundary scheduler started (%s)",
+                    "/".join(tf.name for tf, _ in _AUTOSCAN_LOOPS))
 
     if _ws_task is None or _ws_task.done():
         _ws_task = asyncio.create_task(_ws_monitor(bot))
@@ -928,9 +957,9 @@ def _status_text() -> str:
 
     if _last_prices:
         lines.append("\n*Live Prices (WebSocket):*")
-        for ws_sym, pair in _WS_TO_PAIR.items():
+        for pair in SCAN_PAIRS:
             price = _last_prices.get(pair)
-            display = PAIR_INFO.get(pair, {}).get("display", ws_sym)
+            display = PAIR_INFO.get(pair, {}).get("display", pair)
             age = time.time() - _last_price_ts.get(pair, 0)
             if price is None:
                 lines.append(f"  {display}: waiting...")
@@ -1258,10 +1287,46 @@ async def _post_shutdown(app: Application) -> None:
 
 # ── Entry point ───────────────────────────────────────────────────────
 
-def main() -> None:
-    if not TOKEN:
-        sys.exit("TELEGRAM_TOKEN not set")
+async def _on_error(update: object, context: ContextTypes.DEFAULT_TYPE) -> None:
+    """python-telegram-bot routes every unhandled handler exception here;
+    without a registered error handler PTB only half-logs them."""
+    logger.error(
+        "Unhandled error in Telegram handler (update=%s)",
+        getattr(update, "update_id", update),
+        exc_info=context.error,
+    )
 
+
+def _install_excepthooks() -> None:
+    """Route uncaught exceptions from the main thread and worker threads
+    into the rotating log file instead of bare stderr, so a crash right
+    before exit is still recorded."""
+    def hook(exc_type, exc, tb):
+        if issubclass(exc_type, KeyboardInterrupt):
+            sys.__excepthook__(exc_type, exc, tb)
+            return
+        logger.critical("Uncaught exception", exc_info=(exc_type, exc, tb))
+
+    def thread_hook(args):
+        if issubclass(args.exc_type, SystemExit):
+            return
+        name = args.thread.name if args.thread else "?"
+        logger.critical(
+            f"Uncaught exception in thread {name}",
+            exc_info=(args.exc_type, args.exc_value, args.exc_traceback))
+
+    sys.excepthook = hook
+    threading.excepthook = thread_hook
+
+
+def main() -> None:
+    if not TOKEN or not CHAT_ID:
+        raise SystemExit(
+            "telegram_bot: missing Telegram credentials. Set telegram.token "
+            "and telegram.chat_id in config.json, or the TELEGRAM_TOKEN and "
+            "TELEGRAM_CHAT_ID environment variables.")
+
+    _install_excepthooks()
     logger.info("Starting ForexBot2 Telegram bot…")
 
     app = (
@@ -1281,6 +1346,7 @@ def main() -> None:
     app.add_handler(CommandHandler("clearcache", cmd_clearcache))
     app.add_handler(CallbackQueryHandler(cb_scan, pattern="^scan_"))
     app.add_handler(CallbackQueryHandler(cb_menu, pattern="^menu_"))
+    app.add_error_handler(_on_error)
 
     logger.info("Bot running — press Ctrl+C to stop")
     app.run_polling(allowed_updates=Update.ALL_TYPES)

test_ws_dual.py

@@ -1,108 +0,0 @@
-#!/usr/bin/env python3
-"""Test whether Twelve Data allows 2 simultaneous WebSocket connections for XAU/USD.
-
-Uses the twelvedata SDK which handles the WS protocol correctly.
-Runs connection 1, then adds connection 2 after 10s, monitors for 30s total.
-"""
-
-import threading
-import time
-
-from twelvedata import TDClient
-
-API_KEY = "f563b22ffdfd4924955decbe1a8c8701"
-SYMBOL = "XAU/USD"
-DURATION = 30
-
-results = {
-    "WS-1": {"ticks": 0, "errors": [], "alive": False},
-    "WS-2": {"ticks": 0, "errors": [], "alive": False},
-}
-
-
-def make_handlers(label):
-    def on_event(event):
-        if event.get("event") == "price":
-            results[label]["ticks"] += 1
-            price = event.get("price", "?")
-            print(f"[{label}] tick #{results[label]['ticks']:3d}  price={price}")
-
-    def on_error(error):
-        results[label]["errors"].append(str(error))
-        print(f"[{label}] ERROR: {error}")
-
-    def on_close(status):
-        results[label]["alive"] = False
-        print(f"[{label}] CLOSED (status={status})")
-
-    def on_connect():
-        results[label]["alive"] = True
-        print(f"[{label}] CONNECTED — subscribed to {SYMBOL}")
-
-    return on_event, on_error, on_close, on_connect
-
-
-def run_ws(label):
-    td = TDClient(apikey=API_KEY)
-    on_event, on_error, on_close, on_connect = make_handlers(label)
-    ws = td.websocket(
-        symbols=SYMBOL,
-        on_event=on_event,
-        on_error=on_error,
-        on_close=on_close,
-        on_connect=on_connect,
-    )
-    ws.connect()
-    return ws
-
-
-def main():
-    print(f"Testing dual WebSocket for {SYMBOL} ({DURATION}s)\n")
-
-    print("Starting WS-1...")
-    ws1 = run_ws("WS-1")
-
-    time.sleep(10)
-    print(f"\nWS-1 ticks so far: {results['WS-1']['ticks']}")
-    print("Starting WS-2...\n")
-    ws2 = run_ws("WS-2")
-
-    remaining = DURATION - 10
-    time.sleep(remaining)
-
-    print("\nStopping connections...")
-    try:
-        ws1.disconnect()
-    except Exception:
-        pass
-    try:
-        ws2.disconnect()
-    except Exception:
-        pass
-
-    time.sleep(1)
-
-    print("\n" + "=" * 50)
-    print("RESULTS")
-    print("=" * 50)
-    for label, info in sorted(results.items()):
-        status = "alive" if info["alive"] else "dead"
-        errs = info["errors"] or ["none"]
-        print(f"  {label}: {status}  ticks={info['ticks']}  errors={errs[0]}")
-
-    ws1_ok = results["WS-1"]["ticks"] > 0 and not results["WS-1"]["errors"]
-    ws2_ok = results["WS-2"]["ticks"] > 0 and not results["WS-2"]["errors"]
-    ws1_died = results["WS-1"]["alive"] is False and results["WS-1"]["ticks"] > 0
-
-    if ws1_ok and ws2_ok:
-        print("\n✓ Both connections worked — dual WS is supported")
-    elif ws2_ok and ws1_died:
-        print("\n✗ WS-2 replaced WS-1 — only 1 connection allowed at a time")
-    elif ws1_ok and not ws2_ok:
-        print("\n✗ WS-2 was rejected — only 1 connection allowed")
-    else:
-        print("\n? Inconclusive — check errors above")
-
-
-if __name__ == "__main__":
-    main()