dashboard: skip DOM rebuild on unchanged poll, publish-time staleness, fix boundary countdowns

poll() now compares raw payload text and re-renders only what a changed
file can affect, preserving selection, disclosure, and scroll state. Hero
freshness reflects publish time (STALE warning past 15 min while market
open), and countdowns resolve their wrap interval from data-interval
instead of reverse-looking shared TF boundaries in snapshot.timeframes.

.gitignore

@@ -36,3 +36,4 @@ backtest/__pycache__/
 .env.*
 logs/
 backtest/research/commit_ladder_2026-08/
+backtest/research/llm_news_eval_2026-08/

CLAUDE.md

@@ -11,8 +11,9 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
 ## Commands
 
 ```bash
-python test_all.py                      # QA suite: 302 tests, 20 modules, no network (mocked)
+python test_all.py                      # QA suite: 374 tests, 23 modules, no network (mocked)
-python main.py --pair XAUUSD --timeframe H1 --detail   # CLI signal report (--json for full dump)
+python main.py --pair XAUUSD --timeframe H1 --detail   # CLI signal report (--json for full dump,
+                                        #  --panel to run the advisory LLM agent panel on-demand)
 python telegram_bot.py                  # the bot (refuses to boot without credentials)
 python oanda_daemon.py                  # auto-execution daemon on Oanda practice account
 python dashboard_publisher.py --once    # web dashboard: one scan+publish cycle to $DASHBOARD_OUT_DIR
@@ -24,22 +25,25 @@ python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-0
     --primary-tf 15min --confirm-tfs 1h,4h
 # key flags: --spread --quality A+,A,B,C --strategies --workers --max-hold-bars
 #            --warmup-bars --cooldown-bars --no-cache --export-csv out.csv --detail
+#            --fixed-rr-tp N (research-only: hard TP at N×risk, trail off; verdict
+#            2026-08-05: NOT shipped — backtest/research/fixedtp_2026-08-05.md)
 ```
 
-**`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. When patching weekend logic in tests, patch `providers.base._is_forex_weekend`.
+**`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. When patching weekend logic in tests, patch `providers.base._is_forex_weekend` — but know there are THREE independent weekend implementations (`providers/base.py`, `signal_aggregator.py`, and an inline copy in `backtest/gpu_backtest.py:_is_xau_blocked_by_calendar`); patching the provider one does not reach the other two.
 
 Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets).
 
 ## Configuration & credentials
 
-Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`, `DEEPSEEK_API_KEY` (news LLM; the `deepseek` config.json block holds model/enabled/timeout/daily cap).
+Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`, `DEEPSEEK_API_KEY` (news LLM; the `deepseek` config.json block holds model/enabled/timeout/daily cap). The `agent_panel` config.json block (all-defaults = feature dark) controls the advisory LLM agent panel: `enabled`, per-surface `telegram`/`dashboard` toggles, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`.
 
 ## Architecture
 
 ### End-to-end pipeline (every scan)
 
 ```
-providers/<active>.fetch_ohlcv()       per TF: throttle, fetch, drop forming candle, validate/clean, cache
+providers/<active>.fetch_ohlcv()       per TF: single-flight fetch, drop forming candle, validate/clean, cache
+                                       (only Twelve Data rate-throttles; Oanda has no client throttle)
   → engine.TechnicalEngine.analyze()   per TF: 9 scored indicators + ATR context → regime → composite score
   → signal_aggregator.analyze_pair()   live entry point (weekend/day filters, fetch, staleness)
       └─ analyze_from_dataframes()     replayable core: MTF agreement → news → bonuses/penalties →
@@ -53,12 +57,13 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
 
 - **`providers/`** — one class per provider subclassing `BaseDataProvider` (`providers/base.py`), which owns everything the pipeline depends on: TTL+boundary cache, forming-candle drop, freshness check, validate/clean, paginated `fetch_range` (feeds the backtester from any provider), optional tick streaming. Subclasses implement only `_fetch_raw` + symbol/interval maps. Registry + `make_data_provider(config, name=None)` in `providers/__init__.py`; config.json `data_providers.active` selects the live candle feed and the optional `data_providers.streaming` scalar selects a different provider for the tick stream. A test enforces that subclasses don't override shared base methods (`supports_streaming`/`iter_ticks` are the sanctioned streaming override points). Tiingo streams mid-price ticks (`wss://api.tiingo.com/fx`) — do not re-add spread display from its free FX feed (two-tier indicative quotes, removed 2026-07-23); the no-spread rule extends to the WS stream.
 - **`engine.py`** — `TechnicalEngine`: 9 scored indicators (ema_trend, rsi, macd, bollinger, adx, roc, keltner, divergence, candlestick) + ATR as context. Regime classification (trending_up/down, `trending` = mixed/direction-UNKNOWN, transitioning, ranging) drives per-indicator weight multipliers and regime-aware RSI reinterpretation. Compute order matters (ATR first; MACD always computed even when not a pillar — divergence needs the histogram). Private series ride on the result as `_underscore` attrs; scorers/formatters skip anything `_`-prefixed or weight-0.
-- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
+- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. The PB/CP generators run AFTER the gate chain and force quality to "B"; the CP path deliberately accepts grade-F composites and re-checks only session/day/big-candle (via warning prefixes) plus its own SELL/vol/RSI gates — it does NOT re-check exhaustion or the M5 min-score/MTF gates. **This is validated behavior, not a bug** (2026-08-04 audit: 226/586 trades in the shipped M5 12mo book are XAU_CP, +38.9R, 104 of them exhaustion-flagged and net-positive) — changing it is a rule-1 revalidation event. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
 - **`pair_signals/`** — per-pair config factories. **Always use the factory (`produce_signal` / `make_config`), never bare `PlatformConfig()`** — the factory applies the config.json overlay first, then the validated per-TF deltas (M5 hour-9 block, M15 trail arm 0.70, M15 SELL-hour block, etc.; evidence in the docstring). A test enforces factory ≡ inline-config equivalence. Confirmation map: M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1].
 - **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that.
+- **`agent_panel.py`** — ai-hedge-fund-style advisory panel: 4 DeepSeek persona calls (macro/technical/risk/contrarian, concurrent on a panel-private pool) + 1 synthesizer producing an **ADVISORY-ONLY** consensus (`AGREE/NEUTRAL/OPPOSE` + confidence + rationale) written onto `SignalReport.panel_*` fields. Attached **post-hoc** by consumers (`telegram_bot` alert path, `main --panel`, `dashboard_publisher`) — never runs inside `analyze_from_dataframes` (backtest parity, test-locked) and no gate reads its fields. Static prompts (prefix caching); spend bounded by an exact per-bar cache, a reuse TTL keyed (symbol, TF, direction) so persisting signals don't re-spend every bar, and a `max_panels_per_day` UTC budget (default 20 ≈ ≤100 requests) on top of the shared 500/day client cap. `<2` valid persona takes skips the synthesizer; any failure degrades to no panel, never a changed signal.
 - **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
-- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown, trail ratchets at bar close exactly once per bar. Accepted divergence: one position total.
+- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`).
-- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. Deployment checklist: `deploy/README-dashboard.md`.
+- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. News/calendar rows are clickable (detail dialog); headlines carry `id` = sha1(headline) in news.json, and exactly one on-demand endpoint exists: `static/news_summary.php` (installed by `install_static`), which serves a DeepSeek gold-impact summary per headline — whitelist-gated against news.json ids, cache-forever in `ai/`, own fail-closed daily cap (150/day, file-based, separate from the publisher's), key at `/etc/gold-dashboard/deepseek.key`, never reads repo config. Deployment checklist: `deploy/README-dashboard.md`.
 - **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
 
 ## Rules for making changes (read before editing signal logic)
@@ -73,6 +78,13 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
 8. Don't add config fields nothing reads; don't disable indicators in production paths (`disabled_indicators` is for experiments only).
 9. EUR/USD was removed 2026-07-16 after a no-edge verdict (cost-to-volatility; last commit with EUR support: `7d3fd95`). Do not re-add without new data. BTC/ETH work lives on unmerged branches only.
 
+## 2026-08-04 QA audit — owner decisions ledger
+
+- **Weekend calendar: FIXED (owner-approved 2026-08-04).** All three `_is_forex_weekend` copies are NY-anchored (Fri 17:00 → Sun 18:00 ET; during US DST that is Fri 21:00 UTC — measured on 5yr of Oanda bars). Kills the weekly false-"STALE DATA/feed lagging" hours. Book impact, fully accounted: M15 byte-identical; M5 12mo dropped exactly ONE trade — a phantom whose signal fired on the final summer-Friday bar (close 21:00 UTC) and "filled" at Sunday's reopen 49 h later (−1.08R, live-impossible). M5 12mo reference at HEAD: 555 trades, +98.64R PF≈1.50 @ $0.65 (was 556/+97.56/1.48). A Module 21 test sweeps all three copies hourly across both DST transitions.
+- **Spread in exit-trigger detection (rule 6): owner DECLINED 2026-08-04.** Backtest stop triggers stay on raw mid High/Low; the measured optimism (~4–6% of total R: −3.55R @ $0.30, −4.24R @ $0.65 on the M15 12mo book) is accepted and books stay comparable. Re-open only as a conscious re-pin campaign.
+- **`FB2_EMA_LO`: HARDENED 2026-08-04 (delegated).** Research override for the EMA200-proximity band, default = tuned 0.5; an unparseable value now degrades to the default with a warning instead of crashing every scan. Documented here — it is a knob that exists.
+- **H4 seeding divergence: parked.** Live H4 fetches only 380 bars vs the backtest's unbounded history — ~2% of sampled H4 bars flip an indicator signal (composite Δ up to 12.3). M5/M15/H1 clean at their real fetch depths. Fold into any future H4 campaign; H4 remains "unproven for auto-execution".
+
 ## Current state notes (2026-07-30)
 
 - Oanda v20 is the live feed (`data_providers.active: "oanda"`), but the pinned backtest books are still Twelve-Data-pinned and the backtest cost assumption is $0.30 while measured real spreads are ~$0.65–0.72 — re-pin books on Oanda bars before treating feed-switch comparisons as valid (see Oanda/TD divergence study in memory).

OANDA_MIGRATION_ROADMAP.md

@@ -1,85 +0,0 @@
-# Full Migration to Oanda v20 as Market-Data Source — Roadmap
-
-**Status: partially executed (updated 2026-08-03).** Phases 1–2 shipped 2026-07-28 via the `providers/` package refactor and the live feed switched to Oanda (`data_providers.active: "oanda"`). Phases 3–6 remain unbuilt: the level-breach stream monitor is simply disabled for Oanda, and the pinned backtest books are still Twelve-Data-pinned at the $0.30 cost assumption. Owner decision 2026-07-24 stands: Twelve Data is a **permanent fallback** (no removal phase).
-
-## Context
-
-The daemon already executes trades on an Oanda practice account, and since 2026-07-28 the signal feed (Telegram bot, CLI, backtester) also runs on Oanda. Oanda's API is free with no metering (120 req/s vs TD's 55 credits/min), serves real bid/ask spreads, and matches the actual trading calendar. The 2026-07-24 divergence study (`backtest/research/oanda_td_divergence.py`) proved the switch is **not plumbing-only**: steady-state bars differ by median $0.69 (11% of M15 ATR), bar direction agrees only 89.9%, TD prints ~4,000 phantom Saturday bars/yr + 21:00 UTC rollover bars that contaminate Monday indicator windows, and Oanda's real spread (~$0.72 liquid / $0.84 rollover) is ~2.4× the $0.30 the validated books assume. So the migration = plumbing (Phases 1–3, inert) + shadow evidence (4) + book re-validation with owner gates (5) + reversible cutover (6). CLAUDE.md rules 1 (no tuned-value change without re-validation) and 4 (live == backtest parity) govern throughout.
-
-**Master sequencing rule:** Phases 1–3 land inert behind `data_providers.active == "twelve_data"`. Nothing changes live behavior until the Phase 6 config flip, which stays reversible forever (TD is a permanent fallback).
-
-## Phase 1 — Provider plumbing (inert)
-
-Files: `oanda_provider.py`, `data_provider.py`, `config.py`, `config.json.example`.
-
-1. **Constructor self-resolution**: `OandaDataProvider.__init__(config, api_token=None, base_url=None)` — defaults resolve via existing `app_config.oanda_api_token()` / `oanda_base_url()`; explicit args keep `oanda_daemon.py:180` working. Fail lazily at first fetch (mirror ForexDataProvider's `_td is None` pattern) so credential-less construction doesn't break the hermetic test header.
-2. **Registry**: add `"oanda": OandaDataProvider` to `PROVIDER_REGISTRY` (`data_provider.py:392`); import at bottom of `data_provider.py` (one-direction edge, no cycle — `oanda_provider` must never import from `data_provider`). Add `PROVIDER_NAME` class attr to both providers. Cutover config needs an `"oanda": {}` block under `data_providers` (creds stay in the existing top-level `oanda` section / env).
-3. **Dedupe shared constants**: move `CANDLE_SECONDS` / `BARS_PER_DAY` to `config.py`, alias as class attrs in both providers — all existing call sites (`signal_aggregator.py:18,217,640,942`, `telegram_bot.py:1062`, `gpu_backtest.py:975,1706`, tests) keep working unchanged. Also move `_is_forex_weekend` to `config.py` eventually (currently duplicated + sync-tested).
-4. **`_fetch_window(symbol, tf, start, end)`** on both providers — the backtester's common interface. Contract: raw closed bars labeled in [start, end], oldest-first, capitalized OHLCV, UTC index; empty DataFrame (not exception) for empty windows; caller keeps windows ≤ ~4500 bars. TD: thin wrapper over existing `_fetch_twelve_data(start_date=, end_date=)` (keep its exception behavior byte-identical). Oanda: new `from`/`to` RFC3339 request (`price=MBA, dailyAlignment=0, alignmentTimezone=Etc/UTC`); refactor `_fetch_oanda_candles`'s parsing body (complete-filter, mid extraction, spread cache) into a shared `_parse_candles()`. The from+count quirk (Oanda's `count` includes the boundary bar; see `backtest/research/fetch_oanda_m15.py`) does not apply to from/to windows. Add a no-op-by-construction `_drop_forming_candle` to OandaDataProvider (`gpu_backtest.py:1025` calls it unconditionally). No throttle needed for Oanda.
-5. **Tests (extend Module 15)**: registry selection; constructor self-resolution + explicit-args-win; `_fetch_window` both providers (mock `_session.get` / `_fetch_twelve_data`); CANDLE_SECONDS/BARS_PER_DAY cross-provider sync; provider-contract hasattr test covering everything callers reach for (`fetch_ohlcv, get_pair_info, clear_cache, _is_data_fresh, _has_latest_closed_bar, _validate_and_clean, _drop_forming_candle, _fetch_window, CANDLE_SECONDS, BARS_PER_DAY, _cache, _cache_timestamps, PROVIDER_NAME`); port Module 11's two cache/boundary tests (`test_all.py:2278,2313`) to Oanda variants patching `_fetch_oanda_candles`. Tests that swap `set_app_config` must restore `_TEST_APP` in `finally`.
-
-## Phase 2 — Backtester provider-agnosticism
-
-File: `backtest/gpu_backtest.py` (`fetch_history`, lines 943–1036).
-
-1. Line 984: `provider._fetch_twelve_data(...)` → `provider._fetch_window(...)`. Keep the backward-stepping chunk loop, `empty_streak` bail, cursor stepping byte-identical (rule 6 — this loop produced every pinned cache).
-2. Lines 975/1706: `ForexDataProvider.BARS_PER_DAY` → `provider.BARS_PER_DAY`.
-3. **Provider-namespaced cache keys** in `_cache_path` (line 938): TD keeps legacy names (pinned parquets stay loadable, needed for A/B), Oanda gets `oanda_` prefix (`oanda_XAU_USD_15min_....parquet`) — no collisions. Comment the asymmetry.
-4. `clean=True/False` raw-path semantics untouched.
-5. **Verification before merge**: one M15 12-month TD replay pre/post commit — trade books must be identical. **Alignment audit**: fetch one month of Oanda H4 + D1 via `_fetch_window`, assert label sets match TD's cached labels (H4 at 00/04/08 UTC, D1 at 00:00 UTC) — a mismatch would silently shift every MTF confirmation.
-6. Tests: `fetch_history` dispatch with a stub provider (concat/dedupe/trim, clean=False skips cleaning); `_cache_path` namespacing regression.
-
-## Phase 3 — Telegram bot: TD WebSocket → Oanda pricing stream
-
-File: `telegram_bot.py`; new resolver in `app_config.py`.
-
-1. New `oanda_stream_url()` resolver: env `OANDA_STREAM_URL` → config.json `oanda.stream_url` (add to `_KNOWN_OANDA`) → derived `oanda_base_url().replace("api-", "stream-", 1)`.
-2. Pure parser `_parse_stream_line(line) -> Optional[(pair_symbol, mid_price)]`: type==PRICE → mid from closeoutBid/Ask (fallback bids[0]/asks[0]); HEARTBEAT/garbage → None. `_STREAM_TO_PAIR` from `OandaDataProvider.OANDA_INSTRUMENT` (replaces TD-display-keyed `_WS_TO_PAIR`).
-3. Reader thread (no new deps): `requests.get(.../v3/accounts/{id}/pricing/stream?instruments=..., stream=True, timeout=(10,30))`, iterate `iter_lines(chunk_size=1)` — **chunk_size=1 is load-bearing** (urllib3 buffering vs 5s server heartbeats); push lines to asyncio queue via `loop.call_soon_threadsafe`; interrupt a blocked read with `resp.close()` from the async side.
-4. Async monitor `_oanda_stream_monitor`: dedicated thread (NOT the 4-worker scan executor), `await asyncio.wait_for(queue.get(), timeout=15)` as staleness watchdog (3× server heartbeat) → reconnect with existing exponential-backoff pattern. PRICE lines feed the identical existing body: `_check_level_breach` → `_handle_breach` → M15 rescan, 5-min cooldown. No client heartbeat.
-5. **Provider-gated selection** in `_start_scheduler` (line 889): `active_provider == "oanda"` → stream monitor, else existing `_ws_monitor`. This is the rollback hinge and stays permanently (TD fallback). `_stop_scheduler` must stop the reader thread. Generalize `/status` labels (`_status_text` iterates `SCAN_PAIRS`, not `_WS_TO_PAIR`).
-6. Tests: `_parse_stream_line` variants (PRICE/closeout, PRICE/arrays, HEARTBEAT, malformed, unknown instrument); `_STREAM_TO_PAIR` round-trip; monitor selection under both `active` values; watchdog-reconnect (structure one connection lifecycle as a testable inner coroutine).
-
-## Phase 4 — Shadow / parity evidence (3–4 weeks, no hot-path code)
-
-The daemon already runs the full signal stack on Oanda bars. Formalize evidence before any cutover:
-
-1. **Code parity (exact)**: replay the daemon's shadow window through the backtester on Oanda data (Phase 2 enables this); diff against daemon-logged decisions per (tf, bar_close_ts) — signal, quality, direction, entry/SL/TP. Acceptance: 100% modulo documented daemon divergences (single-position rule, quality gate). Any mismatch is a bug, not feed noise.
-2. **Feed divergence (measured, not zero-gated)**: `backtest/research/shadow_compare.py` parses `logs/telegram.log` + daemon log into per-scan rows: signal agreement rate, direction flips, grade drift, clustering by hour/weekday. Expect high-single-digit % divergence concentrated near grade thresholds and Monday sessions (TD Saturday/rollover bars). Deliverable: written note attributing every divergence class.
-3. **Operational telemetry**: boundary grace-retry frequency (is Oanda's just-closed bar present at boundary+10s?), `get_spread()` time series (live input to the Phase 5 cost decision), stream disconnect counts, Monday-open indicator snapshots TD-vs-Oanda.
-4. **Owner note**: at cutover the data feed host family should match the execution host (practice vs live pricing can micro-differ).
-
-## Phase 5 — Re-validation campaign (rule 1, owner gates)
-
-Fetch Oanda history (M5/M15/H1/H4/D1, 2020-07 → present; Oanda XAU reaches ~2005). Run matrix per primary TF (M5/M15/H1, current factory configs): 12-mo train/val halves, 5-yr, OOS 2024-06→2025-03 — each at `--spread 0.30` (pure feed-effect A/B vs pinned TD books), `0.70` (honest Oanda cost), `0.85` (rollover stress). Caution: Jul–Nov 2025 is provider-sensitive (weekly offsets −$6..+$18.5) — lean on 5-yr + OOS for decisions. Acceptance unchanged: positive both halves, PF ≥ 1.3, trailing exits, next-open fills, OOS positive.
-
-**Owner decision gates (never silent):**
-1. **Canonical cost** — (a) `round_trip_cost=0.70` everywhere (moves the live TP1 spread gate $9→$21 — a strategy re-tune); (b) `default_spread=0.70` for backtest PnL but retune `min_tp_cost_multiple` 30→~13 to preserve the validated $9 gate; (c) keep $0.30 with documented optimism (only if $0.70 books pass anyway). Full acceptance matrix at chosen values.
-2. **Degradation handling** per failing TF: re-run tuning grids on Oanda data / drop TF from `scanning.timeframes` / keep Oanda execution-only (status quo).
-3. **Re-pin**: passing books become the new pinned CSVs + factory-docstring evidence; CLAUDE.md tuned-value annotations update in the cutover commit, not before.
-
-Files (per decisions): `pair_signals/xauusd.py`, `config.py` (`PAIR_INFO.default_spread`, `SignalConfig.round_trip_cost`/`min_tp_cost_multiple`), `test_all.py` tuned-constant assertions, CLAUDE.md.
-
-## Phase 6 — Cutover (reversible forever; no TD removal)
-
-1. config.json: `data_providers.active: "oanda"` + `"oanda": {}` block; restart bot. Everything flips through two seams: `make_data_provider` (bot shared dp, CLI, backtester) and the Phase 3 monitor selector. Any Phase 5 tuned-value changes land in this commit with their validation records.
-2. **Bake ≥1 week** (full weekly cycle incl. Monday open): grace-retry rate, stream stability, alert plausibility vs daemon, `/cache` freshness.
-3. **Rollback = flip `active` back** — TD code, key, caches, and WS monitor all stay maintained indefinitely (owner decision). Dual-provider posture: cross-provider sync tests (Phase 1) keep both honest; TD subscription/key must stay valid to remain a real fallback.
-4. Docs: CLAUDE.md (pipeline header, data_provider section, env-var table + `OANDA_STREAM_URL`, backtester cache note, comparability note: "pre-Oanda books are TD-feed books"), `docs/` guides if they name the data source.
-
-## Key risks
-
-1. **Cost-gate coupling**: naively setting `round_trip_cost=0.70` silently moves the live $9 TP1 gate to $21 — owner gate 5.1 exists precisely for this.
-2. **Cutover before re-pinning** — feed divergence is material (11% of ATR, 10% direction flips); sequencing rule prevents it.
-3. **Hermetic test header** — `_TEST_APP` pins twelve_data; provider-swapping tests must restore it or later modules fail mysteriously.
-4. **Stream liveness triad** — `chunk_size=1`, `resp.close()` interruption, 15s watchdog.
-5. **H4/D1 alignment** — audit before trusting any Oanda backtest.
-6. **Monday/rollover divergence class** — attribute in Phase 4 evidence; do not "fix".
-
-## Verification (per phase)
-
-- Phases 1–3: `python test_all.py` green (235 + new); live behavior provably unchanged (TD active).
-- Phase 2: pre/post TD book identity diff; Oanda H4/D1 alignment audit.
-- Phase 3: soak a second bot instance with `active=oanda` scratch config; observe stream reconnects in log.
-- Phase 5: full acceptance matrix, owner sign-offs recorded in factory docstrings.
-- Phase 6: 1-week bake checklist above; rollback drill (flip back, confirm TD scan cycle) once during bake.

agent_panel.py

@@ -0,0 +1,582 @@
+"""
+ForexBot2 — LLM agent panel (advisory consensus)
+=================================================
+ai-hedge-fund-style persona panel for XAU/USD signals: four DeepSeek
+personas (macro, technical, risk, contrarian) each review a finished
+SignalReport from their own angle, then a synthesizer chair weighs the
+takes and issues a consensus verdict (AGREE / NEUTRAL / OPPOSE with
+confidence and a short rationale).
+
+Two invariants, both load-bearing (CLAUDE.md rules 1 and 4):
+
+- ADVISORY / DISPLAY-ONLY. The verdict is written onto SignalReport
+  fields that no gate and no scoring path ever reads. Any failure —
+  LLM down, cap exhausted, garbage reply — degrades to "no panel",
+  never to a changed or blocked signal.
+- NEVER runs inside ``analyze_from_dataframes``. The backtester replays
+  that function; consumers (telegram_bot, main --panel,
+  dashboard_publisher) attach the panel POST-HOC via ``attach_panel``,
+  so replayed books can never see a panel field. Backtest parity holds
+  by construction.
+
+Cost model: every full panel spends 5 requests (4 personas + 1
+synthesizer) against the shared DeepSeek daily cap (llm_client). Two
+layers bound the spend on top of that cap:
+
+- ``max_panels_per_day`` (config, default 20) — a UTC-day budget of
+  panel *attempts*, mirroring the client's request cap.
+- A reuse TTL (default 3600 s) keyed on (symbol, timeframe, direction):
+  a persisting signal re-serves its cached verdict across bars instead
+  of re-spending. Without this, one sticky M5 dashboard signal would
+  burn 5 calls per 5-minute bar — 1,440/day against the 500 cap.
+
+Prompts are STATIC module constants (DeepSeek prefix-cache pricing —
+see news_engine.py); all variable content travels in the user payload.
+"""
+
+import datetime as _dt
+import logging
+import threading
+import time
+from collections import OrderedDict
+from concurrent.futures import ThreadPoolExecutor, wait as _fwait
+from dataclasses import dataclass, field
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
+
+from app_config import get_app_config
+from llm_client import DEFAULT_TIMEOUT, get_llm_client
+
+if TYPE_CHECKING:  # type hints only — keep the coupling one-way
+    from news_engine import FundamentalContext
+    from signal_aggregator import SignalReport
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_MIN_QUALITY = "B"
+DEFAULT_MAX_PANELS_PER_DAY = 20
+DEFAULT_REUSE_TTL_SECONDS = 3600.0
+
+# Quality ranks for the min_quality gate. Deliberately a local copy —
+# telegram_bot's ranking is private and dashboard_publisher must not
+# import telegram_bot (hard independence rule).
+_QUALITY_RANK = {"A+": 4, "A": 3, "B": 2, "C": 1}
+
+
+# ── DeepSeek prompts ─────────────────────────────────────────────────
+# STATIC constants on purpose: DeepSeek prices a repeated byte-identical
+# prompt prefix at the cache-hit rate (~50x cheaper), so nothing here may
+# be interpolated or timestamped. All variable content travels in the
+# user payload. Panel output is display-only — it never gates signals.
+
+_GOLD_DOCTRINE = """\
+Gold doctrine: hawkish Fed / rate hikes / rising real yields / strong
+dollar are gold-bearish; easing / rate cuts / safe-haven flows /
+geopolitical stress / weak dollar are gold-bullish.
+"""
+
+_TAKE_CONTRACT = """\
+Your review is advisory research on an already-generated signal; you are
+one voice on a panel, not the decision maker.
+Hard rules: use ONLY the data in the JSON you receive — do not invent
+numbers, events, or price levels. If the evidence your role needs is
+thin or missing, answer "neutral" with confidence below 30, unless your
+role's instructions say otherwise.
+Confidence scale (0-100): 80-100 strong one-sided evidence; 50-79 solid
+case with some doubt; 30-49 mixed evidence; below 30 little basis to
+judge.
+Write the note in plain everyday English for someone who has never
+traded: short simple sentences, no trading jargon, no acronyms, no
+abbreviations. Say "profit versus loss", not "RR"; "the 4-hour chart",
+not "H4"; describe what an indicator MEANS ("the price has climbed
+unusually fast"), never its name or number.
+Reply with ONLY JSON:
+{"stance": "support"|"oppose"|"neutral", "confidence": <0-100>,
+ "note": "<max 25 words>"}
+"stance" says whether YOUR analysis supports the proposed trade
+direction, opposes it, or is genuinely mixed; "confidence" is how sure
+you are of your stance; "note" is your single most important point.
+"""
+
+_MACRO_SYSTEM = """\
+You are the macro analyst on a gold (XAU/USD) trade-review panel. You
+receive JSON: a proposed trade brief ("brief": direction, timeframe,
+quality, price, regime) plus the current fundamental picture — recent
+"headlines", today's high-impact "events" (released: true means the
+number is out), a news "sentiment" score for gold, and an optional
+"news_summary" blurb.
+Judge ONE thing: does the macro backdrop support the proposed direction?
+Weigh released surprises over stale headlines; an imminent unreleased
+high-impact event is a reason for lower confidence, not a direction.
+Stance rules:
+- support: the released data and news flow clearly push gold in the
+  proposed direction.
+- oppose: they clearly push gold the other way.
+- neutral: the picture is mixed or stale — or there are no meaningful
+  headlines or events to judge (score that low confidence).
+""" + _GOLD_DOCTRINE + """\
+Example: brief direction BUY, headlines dominated by hawkish Fed
+repricing and a hot CPI print -> {"stance": "oppose", "confidence": 70,
+"note": "Inflation is running hot, so interest rates should stay high — that usually pushes gold down."}
+Example: brief direction BUY, headlines led by an escalating conflict
+and a soft jobs report -> {"stance": "support", "confidence": 75,
+"note": "Worries about war and a cooling economy usually send money into gold."}
+""" + _TAKE_CONTRACT
+
+_TECH_SYSTEM = """\
+You are the technical analyst on a gold (XAU/USD) trade-review panel.
+You receive JSON: a proposed trade brief ("brief") plus the technical
+evidence — per-"indicators" signal/confidence, multi-timeframe
+"mtf" signals and scores, "key_levels" versus the current price, and an
+"exhaustion" count of consecutive same-direction bars flagged by the
+engine.
+Judge ONE thing: is the technical picture internally coherent for the
+proposed direction? Look for indicators fighting each other, the signal
+fighting the regime (e.g. a BUY in trending_down), higher timeframes
+disagreeing, price pressed into a key level, or an exhausted move being
+chased.
+Stance rules:
+- support: the indicators, higher timeframes, and regime mostly agree
+  with the proposed direction and none of the conflicts above appear.
+- oppose: one or more of those conflicts is present and serious.
+- neutral: the evidence genuinely cuts both ways, or too much of it is
+  missing to judge (score that low confidence).
+Example: brief direction SELL in regime trending_up with H1 and H4 both
+BUY -> {"stance": "oppose", "confidence": 80,
+"note": "This bets on gold falling while the bigger picture still points up."}
+Example: brief direction BUY in regime trending_up with M15 and H1 both
+BUY and exhaustion 0 -> {"stance": "support", "confidence": 70,
+"note": "Every way of looking at the chart points up, so buying goes with the flow."}
+""" + _TAKE_CONTRACT
+
+_RISK_SYSTEM = """\
+You are the risk manager on a gold (XAU/USD) trade-review panel. You
+receive JSON: a proposed trade brief ("brief") plus the risk facts —
+the trade "setup" (entry, stop_loss, take_profit_1/2, risk_reward,
+risk_dollars), the "volatility" regime, engine "warnings", the calendar
+"event_risk" level, and the assumed "spread".
+Judge ONE thing: is this setup's risk acceptable as constructed? Check
+reward-to-risk adequacy, stop distance versus the volatility regime
+(a tight stop in extreme vol dies to noise; a huge stop in low vol is
+oversized), spread as a fraction of the stop, and event landmines about
+to hit.
+Stance rules:
+- support: the likely reward clearly justifies the risk, the stop fits
+  the current volatility, and no event landmine is imminent.
+- oppose: reward too thin for the risk, a stop mismatched to the
+  volatility, spread eating a big share of the stop, an imminent
+  high-impact event — or no setup at all (a missing setup means the
+  signal graded too low for one; oppose unless the rest is
+  exceptional).
+- neutral: acceptable but unremarkable — nothing dangerous, nothing
+  attractive.
+Example: RR 1.4 with volatility extreme and a high-impact event due ->
+{"stance": "oppose", "confidence": 65,
+"note": "The possible profit is small for such wild price swings, and big economic news is due soon."}
+Example: RR 2.4 with normal volatility and no events due ->
+{"stance": "support", "confidence": 70,
+"note": "The possible profit is more than double the possible loss, and conditions look calm."}
+""" + _TAKE_CONTRACT
+
+_CONTRARIAN_SYSTEM = """\
+You are the contrarian on a gold (XAU/USD) trade-review panel — the
+designated skeptic. You receive JSON: a proposed trade brief ("brief"),
+the engine's own "bullish_reasons" and "bearish_reasons", its
+"warnings", and an optional "news_summary".
+Your job: construct the strongest honest case AGAINST the proposed
+trade, then score how damaging that case is. Use "oppose" only when the
+counter-case is genuinely strong, "neutral" when it is real but modest,
+and "support" when even the skeptic finds nothing that bites.
+""" + _GOLD_DOCTRINE + """\
+Example: a BUY whose own engine lists three bearish reasons and a
+counter-trend warning -> {"stance": "oppose", "confidence": 60,
+"note": "The system's own list of negatives looks stronger than its reason to buy."}
+""" + _TAKE_CONTRACT
+
+_SYNTH_SYSTEM = """\
+You are the chair of a gold (XAU/USD) trade-review panel. You receive
+JSON: the proposed trade "direction" and the panel's "takes" — one per
+analyst ("persona", "stance", "confidence", "note").
+Weigh the takes into ONE consensus on whether to act on the signal:
+"AGREE" (the panel backs the trade), "OPPOSE" (the panel is against it),
+or "NEUTRAL" (genuinely split or low conviction). Weigh a high-
+confidence oppose from any single analyst seriously — one strong veto
+can outweigh two lukewarm supports. Do not average blindly.
+Base the verdict and rationale ONLY on the takes given — do not add
+outside analysis.
+Confidence scale (0-100): 80-100 the panel clearly leans one way; 50-79
+a lean with real dissent; below 50 split or low conviction.
+Write the rationale in plain everyday English for someone who has never
+traded: short simple sentences, no trading jargon, no acronyms, no
+abbreviations.
+Example rationale: "Three of four analysts are against this: the
+possible profit is too small and the price has already climbed
+unusually fast. Only the news backdrop supports buying."
+Reply with ONLY JSON:
+{"verdict": "AGREE"|"NEUTRAL"|"OPPOSE", "confidence": <0-100>,
+ "rationale": "<max 40 words>"}
+"""
+
+
+# ── Result types ─────────────────────────────────────────────────────
+
+@dataclass
+class PersonaTake:
+    persona: str        # "macro" | "technical" | "risk" | "contrarian"
+    stance: str         # "support" | "oppose" | "neutral"
+    confidence: float   # 0-100
+    note: str
+
+
+@dataclass
+class PanelVerdict:
+    verdict: str        # "AGREE" | "NEUTRAL" | "OPPOSE"
+    confidence: float   # 0-100
+    rationale: str
+    takes: List[PersonaTake] = field(default_factory=list)
+    created_ts: float = 0.0
+
+
+# ── Payload builders (pure functions; llm_client json-dumps sorted) ──
+
+def _direction(report: "SignalReport") -> Optional[str]:
+    """BUY/SELL the panel should review; None = nothing to review."""
+    if report.trade_setup is not None:
+        return report.trade_setup.direction
+    if report.final_signal.value > 0:
+        return "BUY"
+    if report.final_signal.value < 0:
+        return "SELL"
+    return None
+
+
+def _brief(report: "SignalReport") -> Dict[str, Any]:
+    analysis = report.primary_analysis
+    return {
+        "pair": report.display_name,
+        "timeframe": analysis.timeframe if analysis else "",
+        "direction": _direction(report),
+        "quality": report.signal_quality,
+        "score": round(report.final_score, 1),
+        "confidence": round(report.final_confidence),
+        "price": round(report.current_price, 2),
+        "market_regime": report.market_regime,
+        "volatility_regime": report.volatility_regime,
+        "mtf_agreement": report.mtf_agreement,
+    }
+
+
+def _macro_payload(report: "SignalReport",
+                   context: Optional["FundamentalContext"]) -> Dict[str, Any]:
+    headlines: List[str] = []
+    events: List[Dict[str, Any]] = []
+    if context is not None:
+        headlines = [h.headline for h in context.news_headlines[:10]]
+        events = [
+            {"event": e.event, "currency": e.currency,
+             "actual": e.actual, "forecast": e.forecast,
+             "released": not e.is_upcoming}
+            for e in context.events_today if e.is_high_impact
+        ][:8]
+    return {
+        "brief": _brief(report),
+        "headlines": headlines,
+        "events": events,
+        "sentiment": round(report.news_sentiment, 3),
+        "news_summary": report.news_summary,
+    }
+
+
+def _technical_payload(report: "SignalReport") -> Dict[str, Any]:
+    indicators: Dict[str, Dict[str, Any]] = {}
+    key_levels: Dict[str, float] = {}
+    analysis = report.primary_analysis
+    if analysis is not None:
+        # Same visibility filter as main.report_to_dict: private series
+        # ride as _-prefixed pseudo-indicators, weight-0 = context-only.
+        indicators = {
+            name: {"signal": ind.signal.name,
+                   "confidence": round(ind.confidence)}
+            for name, ind in analysis.indicators.items()
+            if not name.startswith("_") and ind.weight != 0
+        }
+        key_levels = {k: round(v, 2) for k, v in analysis.key_levels.items()}
+    return {
+        "brief": _brief(report),
+        "indicators": indicators,
+        "mtf": {tf: {"signal": sig.name,
+                     "score": round(report.mtf_scores.get(tf, 0.0), 1)}
+                for tf, sig in report.mtf_signals.items()},
+        "key_levels": key_levels,
+        "exhaustion": report.exhaustion_count,
+    }
+
+
+def _risk_payload(report: "SignalReport") -> Dict[str, Any]:
+    payload: Dict[str, Any] = {
+        "brief": _brief(report),
+        "volatility": report.volatility_regime,
+        "warnings": list(report.warnings),
+        "event_risk": report.event_risk,
+        "event_description": report.event_description,
+        "spread": round(report.spread_estimate, 2),
+    }
+    ts = report.trade_setup
+    if ts is not None:
+        payload["setup"] = {
+            "direction": ts.direction,
+            "entry": round(ts.entry_price, 2),
+            "stop_loss": round(ts.stop_loss, 2),
+            "take_profit_1": round(ts.take_profit_1, 2),
+            "take_profit_2": round(ts.take_profit_2, 2),
+            "risk_reward": round(ts.risk_reward, 2),
+            "risk_dollars": round(ts.risk_pips * ts.pip_value, 2),
+        }
+    return payload
+
+
+def _contrarian_payload(report: "SignalReport") -> Dict[str, Any]:
+    return {
+        "brief": _brief(report),
+        "bullish_reasons": report.bullish_reasons[:3],
+        "bearish_reasons": report.bearish_reasons[:3],
+        "warnings": list(report.warnings),
+        "news_summary": report.news_summary,
+    }
+
+
+# ── Orchestration ────────────────────────────────────────────────────
+
+# Panel-private pool: the 4 persona calls run concurrently WITHOUT
+# occupying telegram_bot's shared scan executor (which has 4 workers —
+# borrowing them would stall every other scan for the panel's duration).
+_pool = ThreadPoolExecutor(max_workers=4, thread_name_prefix="agent-panel")
+
+_VALID_STANCES = ("support", "oppose", "neutral")
+_VALID_VERDICTS = ("AGREE", "NEUTRAL", "OPPOSE")
+
+
+def _clamp_confidence(raw: Any) -> Optional[float]:
+    try:
+        return max(0.0, min(100.0, float(raw)))
+    except (TypeError, ValueError):
+        return None
+
+
+def _validate_take(persona: str, reply: Any) -> Optional[PersonaTake]:
+    if not isinstance(reply, dict):
+        return None
+    stance = reply.get("stance")
+    if stance not in _VALID_STANCES:
+        return None
+    confidence = _clamp_confidence(reply.get("confidence"))
+    if confidence is None:
+        return None
+    note = str(reply.get("note", "")).strip()[:160]
+    return PersonaTake(persona=persona, stance=stance,
+                       confidence=confidence, note=note)
+
+
+def _ask_persona(persona: str, system: str,
+                 payload: Dict[str, Any]) -> Optional[PersonaTake]:
+    # Client resolved at call time: tests install stubs via lc._client.
+    reply = get_llm_client().chat_json(system, payload)
+    return _validate_take(persona, reply)
+
+
+def run_panel(report: "SignalReport",
+              context: Optional["FundamentalContext"] = None,
+              ) -> Optional[PanelVerdict]:
+    """Run 4 personas + synthesizer; None = no verdict (fail-safe).
+
+    Requires ≥2 valid persona takes before spending the synthesizer
+    request; the synthesizer's reply is validated strictly and there is
+    deliberately NO programmatic fallback verdict — a panel that could
+    not really convene reports nothing rather than something invented.
+    """
+    direction = _direction(report)
+    if direction is None:
+        return None
+    jobs = (
+        ("macro", _MACRO_SYSTEM, _macro_payload(report, context)),
+        ("technical", _TECH_SYSTEM, _technical_payload(report)),
+        ("risk", _RISK_SYSTEM, _risk_payload(report)),
+        ("contrarian", _CONTRARIAN_SYSTEM, _contrarian_payload(report)),
+    )
+    futures = [_pool.submit(_ask_persona, name, system, payload)
+               for name, system, payload in jobs]
+    # The HTTP timeout inside the client bounds each call; the margin
+    # covers pool queueing. Stragglers past the wait are abandoned (their
+    # threads finish harmlessly) and simply don't vote.
+    client_timeout = getattr(get_llm_client(), "_timeout", DEFAULT_TIMEOUT)
+    done, _ = _fwait(futures, timeout=client_timeout + 8.0)
+    takes = [f.result() for f in done if not f.exception()]
+    takes = [t for t in takes if t is not None]
+    if len(takes) < 2:
+        logger.debug("agent panel: %d/4 persona takes — skipping synthesizer",
+                     len(takes))
+        return None
+    order = {"macro": 0, "technical": 1, "risk": 2, "contrarian": 3}
+    takes.sort(key=lambda t: order.get(t.persona, 9))
+    reply = get_llm_client().chat_json(_SYNTH_SYSTEM, {
+        "direction": direction,
+        "takes": [{"persona": t.persona, "stance": t.stance,
+                   "confidence": t.confidence, "note": t.note}
+                  for t in takes],
+    })
+    if not isinstance(reply, dict):
+        return None
+    verdict = reply.get("verdict")
+    confidence = _clamp_confidence(reply.get("confidence"))
+    if verdict not in _VALID_VERDICTS or confidence is None:
+        return None
+    rationale = str(reply.get("rationale", "")).strip()[:240]
+    return PanelVerdict(verdict=verdict, confidence=confidence,
+                        rationale=rationale, takes=takes,
+                        created_ts=time.time())
+
+
+# ── Cache, budget, attach ────────────────────────────────────────────
+
+_state_lock = threading.Lock()
+# Exact result per reviewed bar: repeated attaches on the same signal
+# (re-scan, multi-consumer) are free.
+_cache: "OrderedDict[Tuple[str, str, str, str], PanelVerdict]" = OrderedDict()
+_CACHE_MAX = 32
+# Reuse layer: a persisting signal (same symbol/tf/direction, newer bar)
+# re-serves its last verdict for reuse_ttl_seconds instead of re-spending
+# 5 requests per bar (see module docstring for the M5 dashboard math).
+_last_by_dir: Dict[Tuple[str, str, str], PanelVerdict] = {}
+# UTC-day budget of panel attempts (mirrors llm_client._under_cap).
+_panel_day: Optional[_dt.date] = None
+_panel_count: int = 0
+_warned_budget = False
+
+
+def reset_panel_state() -> None:
+    """Drop caches and counters (tests / config reloads)."""
+    global _panel_day, _panel_count, _warned_budget
+    with _state_lock:
+        _cache.clear()
+        _last_by_dir.clear()
+        _panel_day = None
+        _panel_count = 0
+        _warned_budget = False
+
+
+def panel_health() -> Dict[str, Any]:
+    """Read-only counters for status displays."""
+    settings = get_app_config().agent_panel_settings
+    cap = int(settings.get("max_panels_per_day", DEFAULT_MAX_PANELS_PER_DAY))
+    today = _dt.datetime.now(_dt.timezone.utc).date()
+    with _state_lock:
+        used = _panel_count if _panel_day == today else 0
+    return {"used_today": used, "cap": cap}
+
+
+def _under_panel_budget(cap: int) -> bool:
+    """Count one panel attempt against the UTC-day budget."""
+    global _panel_day, _panel_count, _warned_budget
+    today = _dt.datetime.now(_dt.timezone.utc).date()
+    with _state_lock:
+        if _panel_day != today:
+            _panel_day = today
+            _panel_count = 0
+            _warned_budget = False
+        if _panel_count >= cap:
+            if not _warned_budget:
+                _warned_budget = True
+                logger.warning(
+                    "agent panel: daily budget (%d) exhausted — no more "
+                    "panels until UTC midnight", cap)
+            return False
+        _panel_count += 1
+        return True
+
+
+def _write_fields(report: "SignalReport", v: PanelVerdict) -> None:
+    report.panel_verdict = v.verdict
+    report.panel_confidence = float(v.confidence)
+    report.panel_rationale = v.rationale
+    report.panel_takes = [
+        {"persona": t.persona, "stance": t.stance,
+         "confidence": t.confidence, "note": t.note} for t in v.takes]
+
+
+def attach_panel(report: "SignalReport",
+                 settings: Optional[Dict[str, Any]] = None,
+                 surface: str = "telegram",
+                 force: bool = False,
+                 context: Optional["FundamentalContext"] = None) -> bool:
+    """Best-effort panel attach; True iff panel fields were populated.
+
+    Advisory feature contract: NEVER raises — an exploding panel must
+    not cost an alert send or a dashboard publish. ``force`` (CLI
+    --panel) bypasses the enabled/surface/quality toggles but never the
+    daily budget or the shared LLM cap.
+    """
+    try:
+        if settings is None:
+            settings = get_app_config().agent_panel_settings
+        if not force:
+            if not settings.get("enabled", False):
+                return False
+            surface_on = {
+                "telegram": settings.get("telegram", True),
+                "dashboard": settings.get("dashboard", False),
+            }.get(surface, True)
+            if not surface_on:
+                return False
+            min_quality = settings.get("min_quality", DEFAULT_MIN_QUALITY)
+            if (_QUALITY_RANK.get(report.signal_quality, -1)
+                    < _QUALITY_RANK.get(min_quality, 2)):
+                return False
+        direction = _direction(report)
+        if direction is None:
+            return False
+
+        analysis = report.primary_analysis
+        tf = analysis.timeframe if analysis else ""
+        exact_key = (report.symbol, tf, report.timestamp, direction)
+        dir_key = (report.symbol, tf, direction)
+        ttl = float(settings.get("reuse_ttl_seconds",
+                                 DEFAULT_REUSE_TTL_SECONDS))
+        with _state_lock:
+            hit = _cache.get(exact_key)
+            if hit is None:
+                prior = _last_by_dir.get(dir_key)
+                if prior is not None and time.time() - prior.created_ts < ttl:
+                    hit = prior
+        if hit is not None:
+            _write_fields(report, hit)
+            return True
+
+        cap = int(settings.get("max_panels_per_day",
+                               DEFAULT_MAX_PANELS_PER_DAY))
+        if not _under_panel_budget(cap):
+            return False
+
+        if context is None:
+            # Best-effort; the news engine's class-level 600 s caches make
+            # this nearly free when warm in both bot and dashboard.
+            try:
+                from config import PlatformConfig
+                from news_engine import NewsEngine
+                context = NewsEngine(
+                    PlatformConfig().news).get_fundamental_context()
+            except Exception as e:
+                logger.debug("agent panel: news context unavailable: %s", e)
+                context = None
+
+        verdict = run_panel(report, context)
+        if verdict is None:
+            return False
+        _write_fields(report, verdict)
+        with _state_lock:
+            _cache[exact_key] = verdict
+            while len(_cache) > _CACHE_MAX:
+                _cache.popitem(last=False)
+            _last_by_dir[dir_key] = verdict
+        return True
+    except Exception:
+        logger.exception("agent panel: attach failed — signal unaffected")
+        return False

app_config.py

@@ -86,7 +86,7 @@ _VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
 _SCANNABLE_TFS = ("M5", "M15", "H1", "H4")
 
 _KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level",
-              "oanda", "deepseek"}
+              "oanda", "deepseek", "agent_panel"}
 _KNOWN_OANDA = {
     "account_id", "api_token", "base_url", "lot_size",
     "quality_gate", "max_hold_bars",
@@ -94,6 +94,10 @@ _KNOWN_OANDA = {
 _KNOWN_DEEPSEEK = {
     "api_key", "model", "enabled", "timeout_seconds", "daily_request_cap",
 }
+_KNOWN_AGENT_PANEL = {
+    "enabled", "telegram", "dashboard", "min_quality",
+    "max_panels_per_day", "reuse_ttl_seconds",
+}
 _KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
 _KNOWN_SCANNING = {"pairs", "timeframes"}
 _KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
@@ -118,6 +122,7 @@ class AppConfig:
     pair_overrides: Dict[str, Dict[str, Any]] = field(default_factory=dict)
     oanda_settings: Dict[str, Any] = field(default_factory=dict)
     deepseek_settings: Dict[str, Any] = field(default_factory=dict)
+    agent_panel_settings: Dict[str, Any] = field(default_factory=dict)
     log_level: str = "INFO"
 
 
@@ -276,6 +281,31 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
                 "config.json: deepseek.model must be a non-empty string")
     app.deepseek_settings = dict(deepseek)
 
+    panel = raw.get("agent_panel", {})
+    _check_keys(panel, _KNOWN_AGENT_PANEL, '"agent_panel"')
+    for flag in ("enabled", "telegram", "dashboard"):
+        if flag in panel and not isinstance(panel[flag], bool):
+            raise AppConfigError(
+                f"config.json: agent_panel.{flag} must be true or false")
+    if "min_quality" in panel and panel["min_quality"] not in _VALID_QUALITIES:
+        raise AppConfigError(
+            f'config.json: agent_panel.min_quality "{panel["min_quality"]}" '
+            f"not in {_VALID_QUALITIES}")
+    if "max_panels_per_day" in panel:
+        mpd = panel["max_panels_per_day"]
+        if isinstance(mpd, bool) or not isinstance(mpd, int) or mpd < 1:
+            raise AppConfigError(
+                "config.json: agent_panel.max_panels_per_day must be a "
+                "positive integer")
+    if "reuse_ttl_seconds" in panel:
+        ttl = panel["reuse_ttl_seconds"]
+        if isinstance(ttl, bool) or not isinstance(ttl, (int, float)) \
+                or ttl < 0:
+            raise AppConfigError(
+                "config.json: agent_panel.reuse_ttl_seconds must be a "
+                "non-negative number")
+    app.agent_panel_settings = dict(panel)
+
     level = raw.get("log_level", app.log_level)
     if level not in _VALID_LOG_LEVELS:
         raise AppConfigError(

backtest/gpu_backtest.py

@@ -40,10 +40,13 @@ from dataclasses import dataclass, field
 from datetime import datetime, timedelta, timezone
 from pathlib import Path
 from typing import Dict, List, Optional, Tuple
+from zoneinfo import ZoneInfo
 
 import numpy as np
 import pandas as pd
 
+_NY_TZ = ZoneInfo("America/New_York")
+
 # Ensure project root on sys.path so this script can be invoked from anywhere.
 PROJECT_ROOT = Path(__file__).resolve().parent.parent
 if str(PROJECT_ROOT) not in sys.path:
@@ -402,6 +405,11 @@ class GPUTechnicalEngine(TechnicalEngine):
         # cache key = timeframe.value (e.g. "15min", "1h", "4h")
         self.caches = caches
         self._current_tf: Optional[str] = None
+        # Cache-miss fallbacks recompute with the live engine's pandas
+        # math (different EWM seeding than the precomputed cache). That
+        # is a parity hazard, so it must never happen silently.
+        self.fallback_count = 0
+        self._fallback_warned = False
 
     def analyze(self, df: pd.DataFrame, symbol: str, timeframe: str) -> TechnicalAnalysis:
         # Latch the timeframe so per-indicator overrides know which cache to use.
@@ -419,9 +427,23 @@ class GPUTechnicalEngine(TechnicalEngine):
             return None, -1
         cache = self.caches.get(self._current_tf)
         if cache is None:
+            self._note_fallback(f"no cache for TF {self._current_tf!r}")
             return None, -1
         pos = cache.index_pos.get(df.index[-1])
-        return (cache, pos) if pos is not None else (None, -1)
+        if pos is None:
+            self._note_fallback(
+                f"{self._current_tf}: slice end {df.index[-1]} not in cache")
+            return None, -1
+        return cache, pos
+
+    def _note_fallback(self, why: str) -> None:
+        self.fallback_count += 1
+        if not self._fallback_warned:
+            self._fallback_warned = True
+            logger.warning(
+                "GPU cache miss → falling back to live-engine recompute "
+                "(%s). Further misses counted silently; check "
+                "fallback_count at end of run.", why)
 
     # ── EMA overrides ──────────────────────────────────────────────
 
@@ -826,7 +848,12 @@ class GPUTechnicalEngine(TechnicalEngine):
         c = self.config
         n = c.roc_period
 
-        if pos < n + 1:
+        # Mirror the live guard (engine._compute_roc: len < n + 5 → skip,
+        # i.e. emit only from pos >= n + 4). Unreachable in practice —
+        # analyze() floors at 30 bars and n+4 < 30 for shipped periods —
+        # but keep the engines textually aligned so a future period/floor
+        # change can't open a silent divergence window.
+        if pos < n + 4:
             return
 
         roc_val = float(cache.roc[pos])
@@ -876,6 +903,11 @@ class GPUTechnicalEngine(TechnicalEngine):
             return super()._compute_keltner(df, result)
 
         c = self.config
+        # Mirror the live guard (engine._compute_keltner: len < max(kc_period,
+        # kc_atr_period) + 5 → skip). Unreachable below the 30-bar analyze()
+        # floor with shipped periods; kept aligned for the same reason as ROC.
+        if pos < max(c.kc_period, c.kc_atr_period) + 4:
+            return
         latest = float(df["Close"].iloc[-1])
         kc_upper_val = float(cache.kc_upper[pos])
         kc_lower_val = float(cache.kc_lower[pos])
@@ -979,6 +1011,16 @@ def fetch_history(
     path = _cache_path(symbol, timeframe, start, end, provider.name)
     if use_cache and path.exists():
         logger.info(f"Loading cached {symbol} {timeframe.value}: {path.name}")
+        # Files written before the raw-cache change (2026-07-08, commit
+        # 6a96ed6) contain CLEANED bars; loading one with clean=False
+        # would silently miss the spike bars the raw path exists to model.
+        # The key format predates versioning, so detect by mtime.
+        _RAW_CACHE_EPOCH = 1783468800.0   # 2026-07-08 00:00 UTC
+        if path.stat().st_mtime < _RAW_CACHE_EPOCH:
+            logger.error(
+                f"{path.name} predates the raw-cache format (2026-07-08) "
+                f"and likely contains CLEANED bars — delete it and refetch "
+                f"before trusting trade resolution on this window")
         df = pd.read_parquet(path)
         if df.index.tz is None:
             df.index = df.index.tz_localize("UTC")
@@ -1130,6 +1172,15 @@ def _simulate_trailing(
         reason = "TIMEOUT"
         for j in range(n):
             if np.isnan(h[j]):
+                # Data ends here: close at the LAST VALID bar, not at
+                # c[n-1] (which may itself be NaN — a NaN exit price used
+                # to poison r_multiple and every downstream metric).
+                if j == 0:
+                    exit_idx = 0
+                    reason = "NO_DATA"
+                else:
+                    exit_idx = j - 1
+                    exit_price = float(c[exit_idx])
                 break
             hit = (l[j] <= cur_sl) if buy else (h[j] >= cur_sl)
             if hit:
@@ -1151,8 +1202,15 @@ def _simulate_trailing(
                        else peak + dist * risk)
                 cur_sl = max(cur_sl, lvl) if buy else min(cur_sl, lvl)
 
+        if reason == "NO_DATA":
+            t.exit_reason = reason
+            continue
         if exit_price is None:
             exit_price = float(c[exit_idx])
+        if not np.isfinite(exit_price):
+            # Last-resort guard: never emit a NaN exit/R into the book.
+            t.exit_reason = "NO_DATA"
+            continue
         t.exit_reason = reason
         t.bars_held = exit_idx + 1
         t.exit_price = float(exit_price)
@@ -1325,6 +1383,12 @@ def simulate_trades(
             last_idx = int(n_valid) - 1
             t.exit_price = float(closes_arr[i, last_idx])
             t.exit_time = exit_times[i][last_idx]
+            if not np.isfinite(t.exit_price):
+                # NaN close on the final window bar (corrupt raw data)
+                # must not become a NaN R.
+                t.exit_reason = "NO_DATA"
+                t.exit_price = None
+                continue
 
         # Realized R from actual fill prices, net of round-trip spread.
         risk = abs(t.entry_price - t.sl)
@@ -1377,6 +1441,12 @@ class BacktestConfig:
     # Round-trip transaction cost in price units, charged on every trade
     # (XAU/USD retail spread ≈ $0.30). 0 disables.
     spread: float = 0.30
+    # RESEARCH (2026-08): >0 replaces the exit model with a hard TP at
+    # this multiple of risk from entry (2.0 = fixed 1:2 RR); trailing is
+    # disabled for ALL trades, including per-trade XAU_PB overrides.
+    # Entries/stops/gates are untouched, so the signal book is identical
+    # to the baseline — only the exit policy changes. Never shipped.
+    fixed_rr_tp: float = 0.0
     # Replay worker processes for the bar loop. 0 = auto (cpu_count - 2,
     # capped at 12), 1 = sequential in-process. Signal evaluation per bar
     # is pure (past data only), so it fans out across forked workers; the
@@ -1389,18 +1459,22 @@ class BacktestConfig:
 def _is_xau_blocked_by_calendar(ts: pd.Timestamp, cfg: PlatformConfig) -> Optional[str]:
     """Apply XAU weekend + allowed-weekday filter matching live `analyze_pair`.
 
-    Forex market closed: Friday 22:00 UTC -> Sunday 22:00 UTC (Sydney open).
+    Market closed Friday 17:00 -> Sunday 18:00 NEW YORK time (= Fri
-    Must match `_is_forex_weekend` in signal_aggregator.py exactly.
+    21:00 UTC in US summer, 22:00 in winter — matches measured Oanda
+    feed behavior; 2026-08-04 audit I43). Must match `_is_forex_weekend`
+    in signal_aggregator.py exactly (rule 4; Module 21 agreement test).
+    Note the weekday gates below stay on UTC weekday — they mirror the
+    live gates, which evaluate UTC weekdays.
     """
     xau = cfg.xau_strategy
     if xau.weekend_filter:
-        wd = ts.weekday()
+        ny = ts.astimezone(_NY_TZ)
-        hour = ts.hour
+        wd = ny.weekday()
-        if wd == 4 and hour >= 22:
+        if wd == 4 and ny.hour >= 17:
             return "WEEKEND_FILTER"
         if wd == 5:
             return "WEEKEND_FILTER"
-        if wd == 6 and hour < 22:
+        if wd == 6 and ny.hour < 18:
             return "WEEKEND_FILTER"
     if xau.allowed_weekdays and ts.weekday() not in xau.allowed_weekdays:
         return "DAY_FILTER"
@@ -1617,6 +1691,43 @@ def _reanchor_entries_to_raw(
         )
 
 
+def _apply_cooldown(
+    candidates: List[Tuple[int, Trade]], cooldown_bars: int
+) -> List[Tuple[int, Trade]]:
+    """Ordered cooldown pass over (bar_pos, Trade) candidates: drop any
+    candidate within `cooldown_bars` bars of the previously ACCEPTED
+    entry — exactly what a sequential replay loop would have skipped.
+    Candidates must already be sorted by position (stable merge from the
+    worker spans)."""
+    accepted: List[Tuple[int, Trade]] = []
+    last_entry_pos = -10_000
+    for pos, trade in candidates:
+        if pos - last_entry_pos < cooldown_bars:
+            continue
+        accepted.append((pos, trade))
+        last_entry_pos = pos
+    return accepted
+
+
+def _apply_fixed_rr_tp(trades: List[Trade], mult: float) -> None:
+    """RESEARCH: replace each trade's TP with a hard target at
+    `mult` × |entry − sl| and zero the per-trade trail params so the
+    XAU_PB overrides cannot resurrect trailing. Risk is measured off the
+    final (re-anchored) entry/SL, so call this after
+    `_reanchor_entries_to_raw`. The caller must also pass
+    trail_arm_r=0 to `simulate_trades` — dispatch is on the run-level arg."""
+    for t in trades:
+        risk = abs(t.entry_price - t.sl)
+        t.tp1 = (t.entry_price + mult * risk if t.direction == "BUY"
+                 else t.entry_price - mult * risk)
+        t.tp2 = t.tp1              # static branch never reads tp2; CSV hygiene
+        t.trail_arm_r = 0.0
+        t.trail_dist_r = 0.0
+        t.rr_planned = mult
+        if t.pip_value:
+            t.reward_pips = mult * risk / t.pip_value
+
+
 def run_backtest(
     bt_cfg: BacktestConfig,
     platform_cfg: Optional[PlatformConfig] = None,
@@ -1744,18 +1855,10 @@ def run_backtest(
         candidates.extend(cands)
     candidates.sort(key=lambda pc: pc[0])
 
-    # ── Phase 2: entry cooldown — the only sequential rule. A candidate
+    # ── Phase 2: entry cooldown — the only sequential rule.
-    # inside the cooldown window of the previously ACCEPTED entry is
+    accepted = _apply_cooldown(candidates, bt_cfg.cooldown_bars)
-    # dropped, exactly as the sequential loop skipped those bars.
+    trades: List[Trade] = [t for _, t in accepted]
-    trades: List[Trade] = []
+    signal_ts_list: List[pd.Timestamp] = [primary_df.index[p] for p, _ in accepted]
-    signal_ts_list: List[pd.Timestamp] = []
-    last_entry_pos = -10_000
-    for pos, trade in candidates:
-        if pos - last_entry_pos < bt_cfg.cooldown_bars:
-            continue
-        trades.append(trade)
-        signal_ts_list.append(primary_df.index[pos])
-        last_entry_pos = pos
 
     logger.info(
         f"Replay complete: {len(trades)} entries in "
@@ -1775,10 +1878,17 @@ def run_backtest(
         use_cache=bt_cfg.use_cache, clean=False,
     )
     _reanchor_entries_to_raw(trades, signal_ts_list, sim_df)
+    if bt_cfg.fixed_rr_tp > 0:
+        _apply_fixed_rr_tp(trades, bt_cfg.fixed_rr_tp)
+        logger.info(
+            f"RESEARCH override: fixed-RR TP = {bt_cfg.fixed_rr_tp} x risk; "
+            f"trailing exit DISABLED for all {len(trades)} trades (incl. XAU_PB)"
+        )
+    run_arm = 0.0 if bt_cfg.fixed_rr_tp > 0 else platform_cfg.signals.trail_arm_r
     t_sim = time.perf_counter()
     simulate_trades(trades, sim_df, max_bars=bt_cfg.max_hold_bars,
                     spread=bt_cfg.spread,
-                    trail_arm_r=platform_cfg.signals.trail_arm_r,
+                    trail_arm_r=run_arm,
                     trail_dist_r=platform_cfg.signals.trail_dist_r)
     logger.info(
         f"Trade simulation took {time.perf_counter() - t_sim:.2f}s "
@@ -1828,6 +1938,14 @@ def _summarize(rs: List[float]) -> Dict:
 
 def compute_metrics(trades: List[Trade]) -> Metrics:
     resolved = [t for t in trades if t.exit_reason and t.exit_reason != "NO_DATA"]
+    # Last line of defense: one non-finite R (NaN exit on corrupt data)
+    # would silently NaN every aggregate below. Exclude loudly.
+    bad = [t for t in resolved if not np.isfinite(t.r_multiple)]
+    if bad:
+        logger.error(
+            "compute_metrics: excluding %d trade(s) with non-finite R "
+            "(corrupt exit data) — first at %s", len(bad), bad[0].entry_time)
+        resolved = [t for t in resolved if np.isfinite(t.r_multiple)]
     if not resolved:
         return Metrics(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, {}, {}, {})
 
@@ -2114,6 +2232,10 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
                    help="Override signals.trail_arm_r for this run (research knob)")
     p.add_argument("--trail-dist", type=float, default=None,
                    help="Override signals.trail_dist_r for this run (research knob)")
+    p.add_argument("--fixed-rr-tp", type=float, default=None,
+                   help="RESEARCH: exit at a hard TP of MULT x risk (e.g. 2.0 = "
+                        "fixed 1:2 RR) instead of the trailing stop; "
+                        "entries/stops/gates unchanged.")
     p.add_argument("--quality", default="A+,A,B,C", help="Comma-separated allowed grades")
     p.add_argument("--strategies", default="",
                    help="Comma-separated strategy tags to include (XAU,XAU_PB,XAU_CP,GENERIC). Empty = all.")
@@ -2237,6 +2359,16 @@ def main(argv: Optional[List[str]] = None) -> int:
         # make_config; the fallback must match (live == backtest).
         from app_config import apply_pair_overrides  # noqa: E402
         apply_pair_overrides(platform_cfg)
+    if args.fixed_rr_tp is not None:
+        if args.fixed_rr_tp <= 0:
+            raise SystemExit("--fixed-rr-tp must be > 0")
+        if (args.trail_arm is not None or args.trail_dist is not None or
+                any(s.split("=", 1)[0].strip().startswith(("signals.trail",
+                                                           "signals.pb_trail"))
+                    for s in (args.set or []))):
+            raise SystemExit(
+                "--fixed-rr-tp conflicts with --trail-arm/--trail-dist/"
+                "--set signals.*trail* (the flag disables trailing entirely)")
     if args.swing_sr_lookback > 0:
         platform_cfg.technical.swing_sr_lookback = args.swing_sr_lookback
     if args.trail_arm is not None:
@@ -2283,6 +2415,7 @@ def main(argv: Optional[List[str]] = None) -> int:
         strategies_only=[s.strip() for s in args.strategies.split(",") if s.strip()],
         spread=spread,
         workers=args.workers,
+        fixed_rr_tp=args.fixed_rr_tp or 0.0,
     )
 
     logger.info(

backtest/research/fixedtp_2026-08-05.md

@@ -0,0 +1,123 @@
+# Fixed-RR hard-TP vs shipped trailing exit — research campaign 2026-08-05
+
+**Verdict: DO NOT SHIP. The trailing exit stays.** The fixed 2R target beats the trail on
+total R in every 2024+ window but the edge is a trend-regime artifact: it inverts in the
+2021–2023 ranging years, doubles drawdown, cuts win rate 10–19 points, and doubles hold
+times (amplifying the accepted live one-position divergence). Details below.
+
+## Setup
+
+- Code: HEAD `fc5ce85` + the `--fixed-rr-tp` research flag (uncommitted at run time).
+  The flag replaces every trade's TP with `mult × |entry − sl|` AFTER re-anchoring and
+  disables trailing (run-level and per-trade PB params) — entries, stops, gates, and the
+  cooldown pass are untouched, so both arms replay **trade-identical books** (verified:
+  paired join on entry_time+direction matches 1:1 in every window). Only the exit policy
+  differs → per-trade paired comparison.
+- Provider: Oanda (cache-warm; keys `oanda_XAU_USD_{5min,15min,1h}_{20250603_20260804,
+  20240331_20250330,20210531_20260804}`). Charged cost `--spread 0.65` (real measured
+  round trip); $0.30 letter-cost books derived analytically as `r + 0.35/risk` and
+  verified exact against one real `--spread 0.30` run (max per-trade error 1.8e-15 —
+  spread never enters trigger detection, per the owner-declined 2026-08-04 decision).
+- Control sanity gate: fresh M5 12mo control reproduced the pinned HEAD reference
+  exactly — 555 trades, +98.64R, PF 1.483, maxDD −10.73R.
+- Commands (control = same without the flag):
+  `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --primary-tf {5min,15min}
+   --start 2025-08-04 --end 2026-08-04 --spread 0.65 [--fixed-rr-tp 2.0]
+   --export-csv backtest/fixedtp_*.csv`
+  OOS `--start 2024-06-01 --end 2025-03-30`; 5yr `--start 2021-08-01 --end 2026-08-04`.
+- Books pinned as `backtest/fixedtp_{m5,m15}_{12mo,oos,5yr}_{ctrl,rr20,...}.csv`
+  (per-trade indicator breakdown, agreement counts, regime, vol-regime, weekday,
+  MAE/MFE, bars-held — the full instructions_for_backtest record).
+
+## Headline (@ $0.65)
+
+| Window | Arm | n | R | PF | WR | maxDD | recov | medBars | exits |
+|---|---|---|---|---|---|---|---|---|---|
+| M5 12mo | trail | 555 | +98.64 | 1.483 | 52.4% | −10.7 | 9.2 | 28 | TRAIL 361 / SL 183 / TO 11 |
+| M5 12mo | 2R hard | 555 | +124.85 | 1.378 | 42.9% | −24.5 | 5.1 | 50 | SL 312 / TP1 210 / TO 33 |
+| M15 12mo | trail | 397 | +73.93 | 1.525 | 62.2% | −10.8 | 6.8 | 28 | TRAIL 255 / SL 135 / TO 7 |
+| M15 12mo | 2R hard | 397 | +95.95 | 1.431 | 44.3% | −11.2 | 8.6 | 39 | SL 215 / TP1 151 / TO 31 |
+| M5 OOS | trail | 86 | +9.45 | 1.332 | 53.5% | −8.2 | 1.2 | 70 | — |
+| M5 OOS | 2R hard | 86 | +18.83 | 1.407 | 45.3% | −9.7 | 1.9 | 124 | — |
+| M15 OOS | trail | 299 | +38.77 | 1.360 | 59.2% | −15.5 | 2.5 | 38 | — |
+| M15 OOS | 2R hard | 299 | +72.14 | 1.435 | 45.2% | −13.9 | 5.2 | 82 | — |
+
+Halves (12mo, split 2026-02-03): M5 rr20 +67.99/+56.86 (both positive; H2 PF 1.325 vs
+ctrl 1.503), M15 rr20 +73.74/+22.21 (both positive; ctrl H2 +7.23 — the known H2-flat).
+At $0.30 all deltas keep the same sign (e.g. M5 12mo +109.0 vs +135.2).
+
+So the **letter** of rule 1 passes for rr20 on both TFs. The 5yr decomposition is why it
+still must not ship.
+
+## Exit migration (paired, M5 12mo; same pattern everywhere)
+
+- SL→SL 183 trades: ΔR 0 (identical by construction — stops untouched).
+- TRAIL→TP1 207 trades: +248.9R → +424.5R (**+175.6R**) — winners the trail exits early.
+- TRAIL→SL 129 trades: +19.0R → −136.8R (**−155.7R**) — retraces the trail survives
+  become full stop-outs.
+- TRAIL→TIMEOUT 25: +7.4R. Net = +26.2R. The whole effect is these two big flows nearly
+  cancelling; which one wins is a regime property, not an exit-policy edge.
+- Trades with ctrl MFE ≥ 2R: 95; hard cap forfeits only +10.4R vs trail on M5 (the trail
+  already gives ~0.75R back from peak, so a 2R cap costs little *in a trend*).
+
+## Mini-grid (12mo @ $0.65) — the tell
+
+| mult | M5 R / PF / DD | M15 R / PF / DD |
+|---|---|---|
+| trail | +98.6 / 1.483 / −10.7 | +73.9 / 1.525 / −10.8 |
+| 1.0R | +52.5 / 1.205 / −12.8 | +67.1 / 1.410 / −7.3 |
+| 1.5R | +80.7 / 1.263 / −21.4 | +66.9 / 1.325 / −11.2 |
+| 2.0R | +124.9 / 1.378 / −24.5 | +96.0 / 1.431 / −11.2 |
+| 2.5R | +147.7 / 1.426 / −23.6 | +110.4 / 1.469 / −15.9 |
+| 3.0R | +155.8 / 1.432 / −21.6 | +135.8 / 1.562 / −17.5 |
+
+Monotone in the multiple on both TFs — "2.0" is not an optimum, it's a point on a curve
+that says *never take profit* (3R's M5 TIMEOUT bucket alone is +84.8R). That is the
+signature of a one-way trend sample, not of a superior exit rule. It also reconfirms the
+original finding that SHORT fixed targets (≤1.5R) are strictly bad (cf. RR-0.5 removal,
+`config.py:126-133`: +48R → +105R).
+
+## 5yr regime decomposition (2021-08-01 → 2026-08-04 @ $0.65) — the verdict
+
+M15 (real sample in every year): ctrl +127.97 PF 1.209 DD −35.8; rr20 +158.97 PF 1.170
+DD **−59.8**; rr30 +204.03 PF 1.203 DD **−98.3**.
+
+| Year | n | trail | 2R hard | 3R hard |
+|---|---|---|---|---|
+| 2021 | 88 | **+4.00** | −16.09 | −17.41 |
+| 2022 | 284 | −9.54 | −7.63 | −26.74 |
+| 2023 | 227 | **+7.79** | −1.71 | −5.81 |
+| 2024 | 345 | +40.44 | +64.98 | +89.14 |
+| 2025 | 405 | +49.33 | +69.39 | +104.42 |
+| 2026 | 229 | +35.95 | +50.04 | +60.43 |
+
+M5 5yr mirrors it (ctrl +128.4 PF 1.387 DD −12.8 vs rr20 +160.2 PF 1.306 DD −24.7;
+2022: −1.1 vs −5.2), though pre-2024 M5 samples are thin (12–60 trades/yr).
+
+The hard target's entire outperformance lives in the 2024–2026 gold trend. In ranging
+years it flips sign exactly where the trail stays positive (M15 2021, 2023), and the
+further the target, the worse the ranging-year bleed and the deeper the DD. The 12mo,
+halves, and OOS windows all sit inside the trend era, which is why the rule-1 letter
+passes — this is precisely the within-regime-split trap rule 1's OOS clause exists to
+catch, one level up.
+
+## Additional non-ship reasons
+
+- **Live divergence grows with hold time.** Median hold doubles (M5 28→50 bars; OOS
+  70→124). The daemon holds one position total; the backtester doesn't model that
+  blocking. Longer holds = more live signals skipped that the paper book banks.
+- **WR 52→43% (M5) / 62→44% (M15), max loss streak 6→11 (M15)** — materially harder to
+  operate and size.
+- Aggregate tables (session/day/regime/vol, streaks, quarterly equity, rolling-20 WR)
+  computed and consistent — no bucket reverses the conclusion; both arms' 12mo profit is
+  concentrated in Q1-2026 (+87.7 ctrl / +102.0 rr20).
+
+## Status
+
+- `--fixed-rr-tp` stays as a documented research-only flag (default off; conflicts with
+  `--trail-arm/--trail-dist/--set signals.*trail*` are hard errors). Module 22 tests
+  (5) pin the override math, TP fill + spread, SL-first tie, PB-trail-disable, and CLI
+  guards. Full suite: 346 green.
+- Re-open only if gold's regime turns and a *regime-conditional* exit (trail in range,
+  target in trend) is proposed with a pre-registered mechanism — a plain fixed ratio is
+  now a closed hypothesis at every multiple in {0.5, 1.0, 1.5, 2.0, 2.5, 3.0}.

backtest/research/llm_news_eval.py

@@ -0,0 +1,697 @@
+"""
+DeepSeek news-analysis validation harness — XAU/USD.
+====================================================
+Research-only (no production code changes). Answers: is the DeepSeek
+layer's news analysis *correct*?
+
+Part A (events): replays every high-impact USD release with a genuine
+surprise (2024-01 → 2026-07, from backtest/cache/calendar.parquet)
+through the REAL `NewsEngine._llm_event_direction` — real prompt, real
+payload, real validation — and scores its "gold bullish/bearish" calls
+against gold's actual move (Oanda M15 mids) at +15m/30m/1h/4h/24h.
+Baselines: mechanical surprise-sign doctrine, FF actualBetterWorse,
+always-bullish/bearish (2024-26 is a gold bull regime), coin flip.
+
+Part B (headlines): a curated golden set (llm_news_golden.csv) through
+the REAL `NewsEngine._llm_headline_scores` in production-sized batches,
+vs the keyword scorer it overrides. Includes repeat-batch (determinism)
+and shuffled-order (cross-contamination) probes.
+
+Implementation notes that keep the replay faithful to production:
+  * `_llm_event_direction`/`_llm_headline_scores` resolve the client via
+    `get_llm_client()`, so we seed the module singleton with a research
+    client (raised daily cap: 586 events > production cap 500).
+  * The event cache key omits the forecast, so a repeat `actual` would
+    silently re-serve a direction computed for a different forecast —
+    the cache is cleared before every call.
+  * A "neutral" reply and a hard failure both return None; they are
+    disambiguated by reading the event cache back after the call
+    (absent = failure, "" = neutral, "bullish:/bearish: …" = call).
+  * Raw request/response bytes are captured with a requests response
+    hook — the request-building code path is untouched.
+
+Usage (from repo root):
+  venv/bin/python backtest/research/llm_news_eval.py events --limit 5
+  venv/bin/python backtest/research/llm_news_eval.py events --determinism 20
+  venv/bin/python backtest/research/llm_news_eval.py headlines
+  venv/bin/python backtest/research/llm_news_eval.py report
+
+Artifacts land in backtest/research/llm_news_eval_2026-08/ (JSONL replay
+logs are resume-safe: completed events are never re-paid). `report` is a
+pure offline function of the artifacts.
+"""
+
+import argparse
+import json
+import math
+import random
+import sys
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT))
+
+import llm_client
+from app_config import deepseek_api_key, get_app_config
+from config import NewsConfig
+from llm_client import DEFAULT_MODEL, DeepSeekClient
+from news_engine import EconomicEvent, NewsEngine
+
+RESEARCH_DIR = Path(__file__).parent
+ART_DIR = RESEARCH_DIR / "llm_news_eval_2026-08"
+CALENDAR = REPO_ROOT / "backtest" / "cache" / "calendar.parquet"
+BARS = (REPO_ROOT / "backtest" / "cache" /
+        "oanda_XAU_USD_15min_20210531_20260804.parquet")
+GOLDEN = RESEARCH_DIR / "llm_news_golden.csv"
+EVENTS_JSONL = ART_DIR / "events_replay.jsonl"
+HEADLINES_JSONL = ART_DIR / "headlines_replay.jsonl"
+SCORED_CSV = ART_DIR / "events_scored.csv"
+
+HORIZONS = [("15m", 15), ("30m", 30), ("1h", 60), ("4h", 240),
+            ("24h", 1440)]
+NEUTRAL_BAND = 0.15  # |xau| below this = neutral prediction (Part B)
+BATCH_CAP = 25       # production batch cap (news_engine.py:783)
+EXPECTED_CORPUS = 586
+
+
+# ── shared plumbing ──────────────────────────────────────────────────
+
+class _Recorder:
+    """requests response hook capturing the exact wire traffic."""
+
+    def __init__(self):
+        self.last = None
+
+    def hook(self, resp, **kwargs):
+        body = resp.request.body
+        if isinstance(body, bytes):
+            body = body.decode("utf-8", "replace")
+        self.last = {
+            "request_body": body,
+            "http_status": resp.status_code,
+            "response_body": resp.text,
+            "elapsed_s": round(resp.elapsed.total_seconds(), 3),
+        }
+        return resp
+
+    def pop(self):
+        cap, self.last = self.last, None
+        return cap or {}
+
+
+def install_research_client(timeout: float):
+    """Build a research DeepSeekClient (raised cap) and seed the module
+    singleton so the production code paths use it. Returns (client,
+    recorder)."""
+    app = get_app_config()
+    key = deepseek_api_key(app)
+    ds = app.deepseek_settings
+    if not key:
+        sys.exit("No DeepSeek API key (env DEEPSEEK_API_KEY or "
+                 "config.json deepseek.api_key) — aborting before spend.")
+    if ds.get("enabled") is False:
+        sys.exit("deepseek.enabled is false in config.json — aborting.")
+    client = DeepSeekClient(
+        api_key=key,
+        model=str(ds.get("model", DEFAULT_MODEL)),
+        timeout=timeout,
+        daily_cap=10 ** 6,  # research: corpus 586 > production cap 500
+    )
+    rec = _Recorder()
+    client._session.hooks["response"].append(rec.hook)
+    with llm_client._client_lock:
+        llm_client._client = client
+    return client, rec
+
+
+def _s(x):
+    """Parquet str columns carry float NaN — EconomicEvent needs None."""
+    return None if pd.isna(x) else str(x)
+
+
+def load_corpus():
+    """High-impact USD releases with a genuine surprise, through the
+    production `EconomicEvent.surprise` gate. Asserts run before any
+    API spend."""
+    df = pd.read_parquet(CALENDAR)
+    df = df[(df["impactName"] == "high") & (df["currency"] == "USD")]
+    rows = []
+    for r in df.itertuples():
+        ev = EconomicEvent(
+            time=datetime.fromtimestamp(int(r.dateline), tz=timezone.utc),
+            currency="USD", impact="high", event=_s(r.name) or "",
+            forecast=_s(r.forecast), previous=_s(r.previous),
+            actual=_s(r.actual),
+        )
+        if ev.surprise not in ("positive", "negative"):
+            continue
+        rows.append({
+            "ffid": int(r.id), "dateline": int(r.dateline),
+            "utc_time": ev.time.isoformat(), "name": ev.event,
+            "actual": ev.actual, "forecast": ev.forecast,
+            "previous": ev.previous, "surprise": ev.surprise,
+            "better_worse": int(r.actualBetterWorse), "ev": ev,
+        })
+    ids = [d["ffid"] for d in rows]
+    if len(rows) != EXPECTED_CORPUS:
+        sys.exit(f"corpus size {len(rows)} != {EXPECTED_CORPUS} — "
+                 "calendar.parquet changed; re-verify before spending.")
+    if len(set(ids)) != len(ids):
+        sys.exit("FF ids not unique — resume keying would break.")
+    # Pin the actualBetterWorse mapping on two unambiguous NFP prints.
+    nfp = {d["utc_time"][:10]: d["better_worse"] for d in rows
+           if d["name"] == "Non-Farm Employment Change"}
+    if nfp.get("2024-01-05") != 1 or nfp.get("2024-05-03") != 2:
+        sys.exit("actualBetterWorse mapping pins failed (expected "
+                 "1=better-for-USD on 2024-01-05 beat, 2=worse on "
+                 "2024-05-03 miss).")
+    rows.sort(key=lambda d: (d["dateline"], d["ffid"]))
+    return rows
+
+
+def _load_jsonl(path: Path):
+    recs = []
+    if path.exists():
+        with open(path) as f:
+            for line in f:
+                line = line.strip()
+                if line:
+                    recs.append(json.loads(line))
+    return recs
+
+
+# ── Part A: event replay ─────────────────────────────────────────────
+
+def _extract_reason(cap):
+    try:
+        content = json.loads(cap["response_body"])[
+            "choices"][0]["message"]["content"]
+        return json.loads(content).get("reason")
+    except Exception:
+        return None
+
+
+def cmd_events(args):
+    ART_DIR.mkdir(exist_ok=True)
+    corpus = load_corpus()
+    if args.limit:
+        corpus = corpus[:args.limit]
+    engine = NewsEngine(NewsConfig())
+    existing = _load_jsonl(EVENTS_JSONL)
+    done1 = {r["ffid"] for r in existing
+             if r.get("run_pass") == 1 and r.get("llm_direction")}
+    done2 = {r["ffid"] for r in existing
+             if r.get("run_pass") == 2 and r.get("llm_direction")}
+    pending = [d for d in corpus if d["ffid"] not in done1]
+    print(f"corpus {len(corpus)} | already recorded {len(done1)} | "
+          f"pending {len(pending)} | determinism re-calls "
+          f"{args.determinism} | model calls are ~100 tokens each "
+          f"(deepseek-v4-flash, static-prefix cached) → cost ≈ cents")
+    if not pending and not args.determinism:
+        print("nothing to do")
+        return
+    _, rec = install_research_client(args.timeout)
+
+    out = open(EVENTS_JSONL, "a")
+
+    def call_one(d, run_pass):
+        # Cache key omits forecast → cleared per call so 196 colliding
+        # corpus rows can't be served a stale direction.
+        with NewsEngine._cache_lock:
+            NewsEngine._llm_event_cache.clear()
+        ev = d["ev"]
+        engine._llm_event_direction(ev)
+        key = (ev.event, ev.currency, str(ev.actual or ""))
+        with NewsEngine._cache_lock:
+            cached = NewsEngine._llm_event_cache.get(key)
+        cap = rec.pop()
+        if cached is None:
+            direction, text = None, None       # hard failure → retry
+        elif cached == "":
+            direction, text = "neutral", ""
+        else:
+            direction, text = cached.split(":", 1)[0], cached
+        recd = {k: d[k] for k in (
+            "ffid", "dateline", "utc_time", "name", "actual", "forecast",
+            "previous", "surprise", "better_worse")}
+        recd.update({
+            "llm_direction": direction, "llm_text": text,
+            "reason_raw": _extract_reason(cap), "run_pass": run_pass,
+            "ts": datetime.now(timezone.utc).isoformat(), **cap,
+        })
+        out.write(json.dumps(recd) + "\n")
+        out.flush()
+        return direction
+
+    remaining = pending
+    for npass in range(1, args.retry_passes + 1):
+        if not remaining:
+            break
+        failed = []
+        for i, d in enumerate(remaining):
+            if call_one(d, 1) is None:
+                failed.append(d)
+            if args.sleep:
+                time.sleep(args.sleep)
+            if (i + 1) % 25 == 0:
+                print(f"  pass {npass}: {i + 1}/{len(remaining)} "
+                      f"({len(failed)} failed)")
+        print(f"pass {npass} done: {len(remaining) - len(failed)} ok, "
+              f"{len(failed)} failed")
+        remaining = failed
+    if remaining:
+        print(f"WARNING: {len(remaining)} events still unrecorded after "
+              f"{args.retry_passes} passes — re-run `events` to resume.")
+
+    if args.determinism:
+        ok_ids = {x["ffid"] for x in remaining}
+        done_now = [d for d in corpus if d["ffid"] not in ok_ids]
+        done_now.sort(key=lambda d: (d["name"], d["dateline"]))
+        stride = max(1, len(done_now) // args.determinism)
+        sample = done_now[::stride][:args.determinism]
+        todo = [d for d in sample if d["ffid"] not in done2]
+        print(f"determinism probe: {len(todo)} re-calls "
+              f"({len(sample) - len(todo)} already recorded)")
+        for d in todo:
+            call_one(d, 2)
+            if args.sleep:
+                time.sleep(args.sleep)
+    out.close()
+    print("events replay complete →", EVENTS_JSONL)
+
+
+# ── Part B: golden headlines ─────────────────────────────────────────
+
+def _load_golden():
+    g = pd.read_csv(GOLDEN)
+    need = {"headline", "category", "expected_xau_sign", "rationale"}
+    if not need <= set(g.columns):
+        sys.exit(f"golden CSV missing columns {need - set(g.columns)}")
+    if not g["expected_xau_sign"].isin([-1, 0, 1]).all():
+        sys.exit("expected_xau_sign must be -1/0/+1")
+    if not g["headline"].is_unique:
+        sys.exit("duplicate headlines in golden CSV")
+    return g
+
+
+def cmd_headlines(args):
+    ART_DIR.mkdir(exist_ok=True)
+    g = _load_golden()
+    heads = list(g["headline"])
+    _, rec = install_research_client(args.timeout)
+    engine = NewsEngine(NewsConfig())
+    out = open(HEADLINES_JSONL, "w")  # tiny (~9 calls): fresh each run
+
+    def run_batch(texts, tag, batch_idx):
+        scores = None
+        for attempt in range(3):
+            scores = engine._llm_headline_scores(texts)
+            cap = rec.pop()
+            if scores is not None:
+                break
+            print(f"  {tag} batch {batch_idx} attempt {attempt + 1} "
+                  f"failed (http {cap.get('http_status')})")
+            time.sleep(2.0)
+        out.write(json.dumps({
+            "run_tag": tag, "batch": batch_idx, "headlines": texts,
+            "scores": scores,
+            "ts": datetime.now(timezone.utc).isoformat(), **cap,
+        }) + "\n")
+        out.flush()
+        return scores
+
+    batches = [heads[i:i + BATCH_CAP] for i in range(0, len(heads),
+                                                     BATCH_CAP)]
+    for bi, b in enumerate(batches):
+        ok = run_batch(b, "main", bi) is not None
+        print(f"main batch {bi} ({len(b)} headlines): "
+              f"{'ok' if ok else 'FAILED'}")
+        time.sleep(args.sleep)
+    run_batch(batches[0], "repeat", 0)
+    print("repeat probe done")
+    shuffled = heads[:]
+    random.Random(20260806).shuffle(shuffled)
+    for bi in range(0, len(shuffled), BATCH_CAP):
+        run_batch(shuffled[bi:bi + BATCH_CAP], "shuffled",
+                  bi // BATCH_CAP)
+    print("shuffled probe done →", HEADLINES_JSONL)
+    out.close()
+
+
+# ── report: pure offline function of the artifacts ───────────────────
+
+def _binom_p(k, n):
+    """Exact two-sided binomial p-value vs p=0.5 (no scipy)."""
+    if n == 0:
+        return float("nan")
+    dist = [math.comb(n, i) for i in range(n + 1)]
+    tot = float(2 ** n)
+    pk = dist[k]
+    return min(1.0, sum(c for c in dist if c <= pk) / tot)
+
+
+def _mcnemar_p(hit_a, hit_b):
+    """Exact McNemar via binomial on discordant pairs. Inputs: aligned
+    boolean lists."""
+    b = sum(1 for x, y in zip(hit_a, hit_b) if x and not y)
+    c = sum(1 for x, y in zip(hit_a, hit_b) if not x and y)
+    return b, c, _binom_p(min(b, c), b + c)
+
+
+def _latest_events(recs):
+    """Last pass-1 record per ffid (retries append; last wins)."""
+    out = {}
+    for r in recs:
+        if r.get("run_pass") == 1:
+            out[r["ffid"]] = r
+    return list(out.values())
+
+
+def _fmt_rate(hits, n, label=""):
+    if n == 0:
+        return f"{label:<24} n=0"
+    p = _binom_p(hits, n)
+    return (f"{label:<24} {hits:>4}/{n:<4} = {hits / n * 100:5.1f}%  "
+            f"(binom p={p:.3f})")
+
+
+def report_events():
+    recs = _latest_events(_load_jsonl(EVENTS_JSONL))
+    if not recs:
+        print("no event replay records — run `events` first")
+        return
+    bars = pd.read_parquet(BARS)
+    closes = bars["Close"].to_numpy()
+    close_ns = (bars.index + pd.Timedelta(minutes=15)).asi8
+
+    def anchor(t_ns, h_min):
+        i = np.searchsorted(close_ns, t_ns, side="right") - 1
+        if i < 0:
+            return None
+        stale_ns = max(45.0, h_min / 4.0) * 60e9
+        if t_ns - close_ns[i] > stale_ns:
+            return None
+        return closes[i]
+
+    rows, skip = [], {h: 0 for h, _ in HORIZONS}
+    base_skipped = 0
+    for r in recs:
+        t_ns = int(r["dateline"]) * 10 ** 9
+        base = anchor(t_ns, 0)
+        if base is None:
+            base_skipped += 1
+            continue
+        row = dict(r)
+        row.pop("request_body", None)
+        row.pop("response_body", None)
+        row["year"] = r["utc_time"][:4]
+        row["base"] = base
+        for hname, hmin in HORIZONS:
+            p = anchor(t_ns + hmin * 60 * 10 ** 9, hmin)
+            if p is None:
+                skip[hname] += 1
+                row[f"d_{hname}"] = np.nan
+            else:
+                row[f"d_{hname}"] = p - base
+        rows.append(row)
+    df = pd.DataFrame(rows)
+    df.to_csv(SCORED_CSV, index=False)
+
+    n_fail = int((df["llm_direction"].isna()).sum())
+    print("═" * 70)
+    print("PART A — event gold-direction vs actual XAU move")
+    print("═" * 70)
+    print(f"events replayed: {len(df)} (baseline-price skipped: "
+          f"{base_skipped}) | LLM hard failures: {n_fail}")
+    print("horizon skip counts (stale/closed):",
+          {k: v for k, v in skip.items() if v})
+    dist = df["llm_direction"].value_counts(dropna=False).to_dict()
+    print("LLM call distribution:", dist)
+
+    # Unconditional per-horizon noise scale (null): median |Δ| over all
+    # 2024+ bar times, position-shifted.
+    bars24 = bars[bars.index >= "2024-01-01"]
+    null_med = {}
+    for hname, hmin in HORIZONS:
+        k = hmin // 15
+        d = (bars24["Close"].shift(-k) - bars24["Close"]).abs().dropna()
+        null_med[hname] = float(d.median())
+    print("unconditional null median |Δ| ($):",
+          {k: round(v, 2) for k, v in null_med.items()})
+
+    # Strategy call columns: +1 gold up, −1 gold down, 0/NaN excluded.
+    df["c_llm"] = df["llm_direction"].map(
+        {"bullish": 1, "bearish": -1}).fillna(0)
+    df["c_doctrine"] = df["surprise"].map({"positive": -1, "negative": 1})
+    df["c_ffbw"] = df["better_worse"].map({1: -1, 2: 1}).fillna(0)
+    df["c_bull"] = 1
+    df["c_bear"] = -1
+
+    strategies = [("LLM", "c_llm"), ("doctrine(surprise)", "c_doctrine"),
+                  ("FF betterWorse", "c_ffbw"), ("always-bullish", "c_bull"),
+                  ("always-bearish", "c_bear")]
+
+    for hname, _ in HORIZONS:
+        dcol = f"d_{hname}"
+        sub = df[df[dcol].notna()]
+        print(f"\n── horizon {hname} (n scored bars={len(sub)}) ──")
+        zeros = int((sub[dcol] == 0).sum())
+        if zeros:
+            print(f"  Δ==0 count (scored as miss): {zeros}")
+        for label, col in strategies:
+            s = sub[sub[col] != 0]
+            hits = int((np.sign(s[dcol]) == s[col]).sum())
+            print("  " + _fmt_rate(hits, len(s), label))
+        # noise-filtered LLM hit rate
+        s = sub[(sub["c_llm"] != 0) &
+                (sub[dcol].abs() >= null_med[hname])]
+        hits = int((np.sign(s[dcol]) == s["c_llm"]).sum())
+        print("  " + _fmt_rate(hits, len(s),
+                               f"LLM |Δ|≥null-med"))
+        # event-window magnitude vs null
+        mag = sub[dcol].abs().median()
+        print(f"  event-window median |Δ| = ${mag:.2f} "
+              f"(null ${null_med[hname]:.2f})")
+        # paired McNemar: LLM vs each baseline on common directional set
+        for label, col in [("doctrine", "c_doctrine"),
+                           ("FF betterWorse", "c_ffbw"),
+                           ("always-bullish", "c_bull")]:
+            s = sub[(sub["c_llm"] != 0) & (sub[col] != 0)]
+            ha = list(np.sign(s[dcol]) == s["c_llm"])
+            hb = list(np.sign(s[dcol]) == s[col])
+            b, c, p = _mcnemar_p(ha, hb)
+            print(f"  McNemar LLM vs {label:<15} b={b:>3} c={c:>3} "
+                  f"p={p:.3f}")
+
+    # neutral-call flatness (15m)
+    neu = df[(df["llm_direction"] == "neutral") & df["d_15m"].notna()]
+    dire = df[(df["c_llm"] != 0) & df["d_15m"].notna()]
+    if len(neu):
+        print(f"\nneutral calls (n={len(neu)}): median |Δ_15m| "
+              f"${neu['d_15m'].abs().median():.2f} vs directional "
+              f"(n={len(dire)}) ${dire['d_15m'].abs().median():.2f} "
+              f"(null ${null_med['15m']:.2f})")
+
+    # breakdowns at 1h
+    dcol = "d_1h"
+    sub = df[df[dcol].notna() & (df["c_llm"] != 0)]
+    print("\n── LLM hit rate by event name (1h, top 10 by count) ──")
+    for name, grp in sorted(sub.groupby("name"), key=lambda kv: -len(kv[1]))[:10]:
+        hits = int((np.sign(grp[dcol]) == grp["c_llm"]).sum())
+        print("  " + _fmt_rate(hits, len(grp), name[:24]))
+    print("── LLM hit rate by year (1h) ──")
+    for yr, grp in sub.groupby("year"):
+        hits = int((np.sign(grp[dcol]) == grp["c_llm"]).sum())
+        print("  " + _fmt_rate(hits, len(grp), yr))
+
+    # cluster view: shared datelines
+    print("\n── same-timestamp clusters ──")
+    clus = df.groupby("dateline").filter(lambda g: len(g) > 1)
+    groups = list(clus.groupby("dateline"))
+    consistent = 0
+    net_hits, net_n = 0, 0
+    for _, g in groups:
+        signs = set(g.loc[g["c_llm"] != 0, "c_llm"])
+        if len(signs) <= 1:
+            consistent += 1
+        net = int(g["c_llm"].sum())
+        d = g["d_1h"].iloc[0]
+        if net != 0 and pd.notna(d) and d != 0:
+            net_n += 1
+            if np.sign(d) == np.sign(net):
+                net_hits += 1
+    print(f"clusters: {len(groups)} datelines "
+          f"({len(clus)} events, {len(clus) / max(len(df), 1) * 100:.0f}% of "
+          f"corpus) | internally consistent: {consistent}/{len(groups)}")
+    print("  " + _fmt_rate(net_hits, net_n, "cluster net-call (1h)"))
+
+    # determinism probe
+    p2 = {r["ffid"]: r for r in _load_jsonl(EVENTS_JSONL)
+          if r.get("run_pass") == 2 and r.get("llm_direction")}
+    if p2:
+        p1 = {r["ffid"]: r for r in recs}
+        agree = same_reason = n = 0
+        for fid, r2 in p2.items():
+            r1 = p1.get(fid)
+            if not r1 or not r1.get("llm_direction"):
+                continue
+            n += 1
+            if r1["llm_direction"] == r2["llm_direction"]:
+                agree += 1
+            if r1.get("reason_raw") == r2.get("reason_raw"):
+                same_reason += 1
+        print(f"\ndeterminism probe: direction agreement {agree}/{n}, "
+              f"identical reason text {same_reason}/{n}")
+
+    # stratified reason examples
+    print("\n── reason samples (1h hit/miss) ──")
+    sub = df[df[dcol].notna() & (df["c_llm"] != 0)].copy()
+    sub["hit"] = np.sign(sub[dcol]) == sub["c_llm"]
+    for verdict in (True, False):
+        picked, seen = [], set()
+        for _, r in sub[sub["hit"] == verdict].iterrows():
+            if r["name"] in seen:
+                continue
+            seen.add(r["name"])
+            picked.append(r)
+            if len(picked) == 10:
+                break
+        tag = "HIT " if verdict else "MISS"
+        for r in picked:
+            print(f"  [{tag}] {r['utc_time'][:16]} {r['name'][:30]:<30} "
+                  f"a={r['actual']} f={r['forecast']} → "
+                  f"{r['llm_direction']:<7} Δ1h={r[dcol]:+7.2f}  "
+                  f"“{(r['reason_raw'] or '')[:60]}”")
+    print(f"\nscored table → {SCORED_CSV}")
+
+
+def report_headlines():
+    recs = _load_jsonl(HEADLINES_JSONL)
+    if not recs:
+        print("\n(no headline replay records — run `headlines` first)")
+        return
+    g = _load_golden()
+    engine = NewsEngine(NewsConfig())
+
+    def collect(tag):
+        m = {}
+        for r in recs:
+            if r["run_tag"] == tag and r["scores"]:
+                for h, sc in zip(r["headlines"], r["scores"]):
+                    m[h] = tuple(sc)
+        return m
+
+    main, shuf = collect("main"), collect("shuffled")
+    print("\n" + "═" * 70)
+    print("PART B — golden headline set: LLM vs keyword baseline")
+    print("═" * 70)
+    missing = [h for h in g["headline"] if h not in main]
+    if missing:
+        print(f"WARNING: {len(missing)} headlines missing from main run")
+
+    def cls(x):
+        if x >= NEUTRAL_BAND:
+            return 1
+        if x <= -NEUTRAL_BAND:
+            return -1
+        return 0
+
+    rows = []
+    for _, r in g.iterrows():
+        h = r["headline"]
+        if h not in main:
+            continue
+        llm_xau = main[h][1]
+        kw = engine._analyze_headline_sentiment(h, for_currency="XAU")
+        rows.append({
+            "headline": h, "category": r["category"],
+            "exp": int(r["expected_xau_sign"]),
+            "llm_xau": llm_xau, "llm_cls": cls(llm_xau),
+            "kw": kw, "kw_cls": cls(kw),
+        })
+    t = pd.DataFrame(rows)
+    for who in ("llm", "kw"):
+        acc = (t[f"{who}_cls"] == t["exp"]).mean()
+        print(f"{who.upper():<4} 3-class accuracy (band ±{NEUTRAL_BAND}): "
+              f"{(t[f'{who}_cls'] == t['exp']).sum()}/{len(t)} "
+              f"= {acc * 100:.1f}%")
+    ha = list(t["llm_cls"] == t["exp"])
+    hb = list(t["kw_cls"] == t["exp"])
+    b, c, p = _mcnemar_p(ha, hb)
+    print(f"paired McNemar LLM vs keyword: b={b} c={c} p={p:.3f}")
+
+    print("\nper-category accuracy (LLM | keyword):")
+    for cat, grp in t.groupby("category"):
+        a1 = (grp["llm_cls"] == grp["exp"]).sum()
+        a2 = (grp["kw_cls"] == grp["exp"]).sum()
+        print(f"  {cat:<18} {a1:>2}/{len(grp):<2} | {a2:>2}/{len(grp)}")
+
+    print("\nLLM confusion (rows=expected, cols=predicted -1/0/+1):")
+    cm = pd.crosstab(t["exp"], t["llm_cls"]).reindex(
+        index=[-1, 0, 1], columns=[-1, 0, 1], fill_value=0)
+    print(cm.to_string())
+
+    print("\nLLM misclassified headlines:")
+    for _, r in t[t["llm_cls"] != t["exp"]].iterrows():
+        print(f"  exp {r['exp']:+d} got {r['llm_xau']:+.2f} "
+              f"[{r['category']}] {r['headline'][:70]}")
+
+    # repeat-batch determinism
+    m_b0 = [r for r in recs if r["run_tag"] == "main" and r["batch"] == 0]
+    r_b0 = [r for r in recs if r["run_tag"] == "repeat"]
+    if m_b0 and r_b0 and m_b0[0]["scores"] and r_b0[0]["scores"]:
+        a = np.array(m_b0[0]["scores"], dtype=float)
+        b_ = np.array(r_b0[0]["scores"], dtype=float)
+        exact = int((a == b_).all(axis=1).sum())
+        print(f"\nrepeat-batch: {exact}/{len(a)} score pairs identical, "
+              f"max |diff| = {np.abs(a - b_).max():.3f}")
+
+    # shuffled-order drift
+    if shuf:
+        both = [h for h in t["headline"] if h in shuf]
+        drift = [abs(main[h][1] - shuf[h][1]) for h in both]
+        if drift:
+            print(f"shuffled-order xau drift over {len(both)} headlines: "
+                  f"mean {np.mean(drift):.3f}, max {np.max(drift):.3f}")
+
+
+def cmd_report(args):
+    report_events()
+    report_headlines()
+
+
+# ── CLI ──────────────────────────────────────────────────────────────
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[1])
+    sub = ap.add_subparsers(dest="cmd", required=True)
+
+    e = sub.add_parser("events", help="replay release surprises")
+    e.add_argument("--limit", type=int, default=0,
+                   help="smoke-test on first N events")
+    e.add_argument("--determinism", type=int, default=0,
+                   help="re-call N stratified events a second time")
+    e.add_argument("--sleep", type=float, default=0.3)
+    e.add_argument("--retry-passes", type=int, default=3)
+    e.add_argument("--timeout", type=float, default=30.0)
+    e.set_defaults(fn=cmd_events)
+
+    h = sub.add_parser("headlines", help="golden-set probes")
+    h.add_argument("--sleep", type=float, default=0.3)
+    h.add_argument("--timeout", type=float, default=30.0)
+    h.set_defaults(fn=cmd_headlines)
+
+    r = sub.add_parser("report", help="offline scoring report")
+    r.set_defaults(fn=cmd_report)
+
+    args = ap.parse_args()
+    args.fn(args)
+
+
+if __name__ == "__main__":
+    main()

backtest/research/llm_news_eval_2026-08-06.md

@@ -0,0 +1,86 @@
+# DeepSeek news-analysis validation — 2026-08-06
+
+**Verdict: headline sentiment scoring is excellent; event-direction calls are
+directionally sound but ~11% of them misread the numeric comparison, and the
+failure concentrates on Unemployment Claims / close K-M-suffixed values.**
+
+Method (owner-directed, token-lean): independent expert re-analysis of a
+stratified sample of DeepSeek's recorded answers, not a full statistical
+backtest. Harness: `backtest/research/llm_news_eval.py` (replays releases
+through the REAL `_llm_event_direction` / `_llm_headline_scores` — real
+prompts, payloads, validation; raw wire traffic logged). 135 of the 586-event
+corpus were replayed before the full run was stopped by owner request
+(artifacts retained in `llm_news_eval_2026-08/`, resume-safe).
+
+## Part A — event gold-direction (28-event stratified sample, Jan–Jun 2024)
+
+Reviewer agreement with DeepSeek: **24/28 (86%)**. All 4 disagreements are
+DeepSeek errors, in exactly two failure modes:
+
+1. **Numeric comparison misread (3/28 ≈ 11%)** — the doctrine logic is right
+   but the premise is backwards:
+   - JOLTS 8.79M vs f 8.84M (a miss) → DS: "Job openings **above** forecast" → bearish (should be bullish)
+   - Claims 202K vs f 217K (fewer claims) → DS: "Claims **higher** than forecast" → bullish (should be bearish)
+   - Claims 202K vs f 209K — same backwards read, same wrong call
+   Both Unemployment Claims events in the sample were misread the same way —
+   looks systematic on that event type, possibly an anchoring prior that
+   "claims rose". All misreads involve close values with K/M suffixes.
+2. **Direction field contradicts its own reason (1/28)** — Empire State
+   −43.7 vs f −4.9: reason text says "gold may rise", `gold_direction` says
+   `bearish`.
+
+Where the numbers were read correctly, hawkish/dovish→gold mapping was
+consistently correct (CPI/PPI/NFP/ADP/GDP/PMI/wages/unemployment-rate all
+handled properly, including inverted metrics when read right).
+
+Price cross-check (1h horizon, cluster-confounded, indicative only):
+reviewer calls 22/28 with the subsequent move, DeepSeek 20/28.
+
+## Part B — headline sentiment (78-headline golden set)
+
+DeepSeek matched the reviewer's expected gold-direction label on
+**78/78 (100%)** — including all 8 negation traps ("signals no rate hike",
+"no rate cuts until…") and all 6 inverted-metric phrasings ("claims hit
+two-year high"). The keyword scorer it overrides got **24/78 (31%)**.
+Repeat-batch and shuffled-order probes recorded (headlines_replay.jsonl).
+
+## Implications
+
+- The headline-sentiment override (the main live LLM surface) is validated —
+  a large, measured upgrade over keywords.
+- The event-direction enrichment ("— gold bullish/bearish" appended to
+  surprise descriptions) is right ~85–90% of the time but can state a
+  factually backwards comparison in the displayed reason. It is display-only
+  (never gates signals), so exposure is a possibly-misleading Telegram/CLI
+  line, not trades.
+- **Hardening SHIPPED same day (owner-approved):** the payload now carries a
+  code-computed `vs_forecast` field ("above/below/in line with forecast",
+  from `EconomicEvent.surprise`) and the prompt tells the model to trust it.
+  First iteration fixed the arithmetic but exposed a masked doctrine error
+  (model read "claims below forecast" correctly, then called it a *weaker*
+  labor market — and regressed the unemployment-rate case the same way), so
+  the prompt's inverted-metrics line was expanded to an explicit rule plus a
+  worked claims example. Verification: **18/18** — all 4 original misreads
+  fixed, 11/11 no-regression cases unchanged, 3/3 fresh probes on the side
+  the prompt example does NOT cover (claims 245K vs f 220K → bullish,
+  unemployment 4.1% vs 3.9% → bullish, JOLTS 9.10M vs 8.80M → bearish).
+  Suite green at 346 tests, incl. a new payload assertion in Module 19.
+- Production observations found while building the harness (not fixed):
+  `_llm_event_cache` key omits forecast → a repeated `actual` re-serves a
+  direction computed for a different forecast; a "neutral" reply is
+  indistinguishable from a failure in the return value.
+
+## Reproduce
+
+```
+venv/bin/python backtest/research/llm_news_eval.py events --limit 5   # smoke
+venv/bin/python backtest/research/llm_news_eval.py events --determinism 20
+venv/bin/python backtest/research/llm_news_eval.py headlines
+venv/bin/python backtest/research/llm_news_eval.py report             # offline
+```
+
+Corpus: 586 high-impact USD surprises 2024-01→2026-07 (calendar.parquet,
+NFP-verified UTC); prices: Oanda M15 mids. Full-run cost ≈ cents. The
+`report` subcommand (hit rates vs always-bullish/doctrine/FF-betterWorse
+baselines, McNemar, cluster view) is built and untested at scale — run it
+after completing the replay if a statistical answer is ever wanted.

backtest/research/llm_news_golden.csv

@@ -0,0 +1,79 @@
+headline,category,expected_xau_sign,rationale
+"Powell says restrictive policy needed for longer as inflation proves sticky",hawkish_fed,-1,"higher-for-longer rates lift real yields, gold-bearish"
+"Fed minutes show officials leaning toward two more rate hikes this year",hawkish_fed,-1,"more hikes = stronger dollar/yields"
+"Hot CPI print fuels bets on additional Fed tightening",hawkish_fed,-1,"tightening expectations weigh on gold"
+"Fed's Waller: higher-for-longer rates warranted by resilient economy",hawkish_fed,-1,"hawkish guidance"
+"FOMC raises rates 50bp, flags further increases ahead",hawkish_fed,-1,"outsized hike plus hawkish guidance"
+"Treasury yields spike as Fed pushes back on rate cut hopes",hawkish_fed,-1,"cut hopes dashed, yields up"
+"Core inflation accelerates, cementing case for hawkish Fed stance",hawkish_fed,-1,"hawkish repricing"
+"Fed dot plot shifts higher; markets price out easing for the year",hawkish_fed,-1,"easing priced out"
+"Fed cuts rates 25bp and signals more easing to come",dovish_fed,1,"easing cycle is gold-bullish"
+"Powell hints at rate cuts as inflation cools toward target",dovish_fed,1,"cut guidance"
+"FOMC minutes reveal growing support for policy easing",dovish_fed,1,"dovish minutes"
+"Fed's Williams says rate cuts appropriate later this year",dovish_fed,1,"cut guidance"
+"Soft inflation data boosts odds of a dovish Fed pivot",dovish_fed,1,"pivot odds up"
+"Fed pauses hikes, markets price aggressive easing cycle",dovish_fed,1,"easing priced in"
+"Central banks turn dovish as global growth falters",dovish_fed,1,"global easing backdrop"
+"Fed to restart bond purchases to support flagging economy",dovish_fed,1,"QE is gold-bullish"
+"Dollar index hits 12-month high on robust US economic data",strong_usd,-1,"strong dollar pressures gold"
+"US GDP grows 4.1% annualized, blowing past expectations",strong_usd,-1,"strong economy, hawkish implication"
+"Blockbuster jobs report sends dollar surging across the board",strong_usd,-1,"strong jobs lift dollar and yields"
+"US retail sales smash forecasts, underscoring consumer strength",strong_usd,-1,"strong data"
+"Greenback rallies as US data outshines Europe and Asia",strong_usd,-1,"dollar strength"
+"ISM services index jumps to two-year high on strong demand",strong_usd,-1,"strong data"
+"Consumer confidence surges to post-pandemic high in the US",strong_usd,-1,"strong data"
+"Dollar tumbles to eight-month low after dismal manufacturing data",weak_usd,1,"weak dollar lifts gold"
+"US economy adds far fewer jobs than expected; dollar slides",weak_usd,1,"weak jobs, dovish implication"
+"Greenback weakens as US retail sales unexpectedly contract",weak_usd,1,"weak data"
+"US GDP growth revised down sharply, stoking slowdown fears",weak_usd,1,"slowdown is dovish"
+"Dollar slumps as trade deficit balloons to record",weak_usd,1,"dollar weakness"
+"Soft services PMI adds to evidence of cooling US economy",weak_usd,1,"cooling economy"
+"US consumer sentiment plunges to two-year low",weak_usd,1,"weak data"
+"Real yields climb to multi-year highs as TIPS sell off",real_yields,-1,"rising real yields raise gold's opportunity cost"
+"10-year Treasury yield surges past 5% on supply glut",real_yields,-1,"yield spike"
+"Rising real interest rates dim appeal of non-yielding assets",real_yields,-1,"explicit opportunity-cost story"
+"Treasury yields sink as investors flock to bonds",real_yields,1,"falling yields support gold"
+"Real yields turn negative as inflation outpaces rates",real_yields,1,"negative real yields are gold's best backdrop"
+"Bond rally drives 10-year yield to lowest level in a year",real_yields,1,"falling yields"
+"Geopolitical tensions escalate in Middle East, investors seek safety",safe_haven,1,"safe-haven bid"
+"Stocks crater worldwide as banking crisis fears spread",safe_haven,1,"crisis flight to safety"
+"Escalating conflict drives flight to safe-haven assets",safe_haven,1,"explicit haven flow"
+"Investors dump risk assets after surprise sovereign default",safe_haven,1,"default shock"
+"Military escalation on NATO border rattles global markets",safe_haven,1,"geopolitical stress"
+"Wall Street plunges 4% in worst session since pandemic",safe_haven,1,"risk-off"
+"Currency crisis in emerging markets sparks contagion fears",safe_haven,1,"contagion, haven bid"
+"Nuclear rhetoric intensifies, sending investors to havens",safe_haven,1,"extreme geopolitical stress"
+"Ceasefire agreement lifts global markets, haven demand fades",risk_on,-1,"haven unwind"
+"Stocks roar to record highs as trade deal signed",risk_on,-1,"risk-on drains haven demand"
+"Risk appetite returns as banking fears subside",risk_on,-1,"risk-on"
+"Global equities rally on strong earnings and easing tensions",risk_on,-1,"risk-on"
+"Gold hits record high as central banks keep buying bullion",gold_direct,1,"direct demand"
+"Gold ETFs post biggest weekly inflows in three years",gold_direct,1,"investment inflows"
+"China's central bank extends gold buying spree to 18th month",gold_direct,1,"official-sector demand"
+"Miners struggle to meet gold demand as supply deficit widens",gold_direct,1,"supply deficit"
+"Gold slides 2% as investors book profits after record run",gold_direct,-1,"profit-taking pressure"
+"Gold tumbles below key support as ETF outflows accelerate",gold_direct,-1,"outflows"
+"Bullion demand weakens as Indian jewellery buying stalls",gold_direct,-1,"physical demand soft"
+"Speculators slash bullish gold bets as dollar strengthens",gold_direct,-1,"positioning unwind"
+"Fed holds rates steady, signals no rate hike on the horizon",negation_traps,1,"dovish hold; keyword layer sees the phrase rate hike and flips it gold-bearish"
+"Powell says no rate cuts until inflation shows sustained cooling",negation_traps,-1,"hawkish hold; keyword layer sees rate cut and calls it gold-bullish"
+"Recession fears fade as US data streak beats estimates",negation_traps,-1,"fading fear is risk-on; keywords see recession plus beats and cancel out"
+"Inflation fails to accelerate, easing pressure on the Fed",negation_traps,1,"disinflation opens door to cuts"
+"Fed's Bowman: not yet confident inflation is beaten, cuts premature",negation_traps,-1,"hawkish pushback phrased around negations"
+"Central bank stops short of hiking, but warns tightening may resume",negation_traps,-1,"hawkish hold with tightening bias"
+"No safe-haven bid for gold despite equity wobble, analysts say",negation_traps,-1,"explicitly negated haven demand"
+"Jobs report not as weak as feared, dollar recovers",negation_traps,-1,"less-bad data is dollar-positive"
+"US manufacturing output plunges as new orders dry up",inverted_metrics,1,"weak economy is dovish/gold-bullish; keyword layer reads plunges as bearish"
+"US factory activity slumps to three-year low",inverted_metrics,1,"weak data; keyword reads slumps as bearish"
+"Consumer spending falls for third straight month",inverted_metrics,1,"weak data; keyword reads falls as bearish"
+"Private payrolls growth slides to weakest pace since 2020",inverted_metrics,1,"weak jobs; keyword reads slides as bearish"
+"Jobless claims hit two-year high as layoffs spread",inverted_metrics,1,"higher claims = weaker economy = gold-bullish"
+"US unemployment rate hits 4.5%, highest since 2021",inverted_metrics,1,"higher unemployment = dovish = gold-bullish"
+"Lakers clinch playoff berth with overtime win over Suns",distractors,0,"irrelevant"
+"New Marvel film smashes box office records on opening weekend",distractors,0,"irrelevant"
+"Apple unveils thinner MacBook with faster chip lineup",distractors,0,"irrelevant"
+"Scientists discover new deep-sea species off Pacific coast",distractors,0,"irrelevant"
+"Heavy rain forecast for the Midwest through the weekend",distractors,0,"irrelevant"
+"Streaming service raises subscription prices in Europe",distractors,0,"irrelevant"
+"Electric vehicle startup recalls 12,000 cars over software glitch",distractors,0,"irrelevant"
+"Local elections see record turnout in several US states",distractors,0,"irrelevant"

config.json.example

@@ -40,5 +40,13 @@
     "timeout_seconds": 12,
     "daily_request_cap": 500
   },
+  "agent_panel": {
+    "enabled": false,
+    "telegram": true,
+    "dashboard": false,
+    "min_quality": "B",
+    "max_panels_per_day": 20,
+    "reuse_ttl_seconds": 3600
+  },
   "log_level": "INFO"
 }

config.py

@@ -166,6 +166,9 @@ class NewsConfig:
     # Must stay below pre_event_buffer_minutes so an event can't cross
     # into the danger window while a stale cache says it's clear.
     context_cache_ttl_seconds: int = 600
+    # Maximum age of RSS headlines accepted for scoring. Older headlines
+    # are discarded before keyword analysis and LLM enrichment.
+    headline_max_age_seconds: int = 43200  # 12 hours
 
     rss_feeds: List[str] = field(default_factory=lambda: [
         "https://investinglive.com/feed/",

dashboard_publisher.py

@@ -37,6 +37,7 @@ Out dir precedence: --out → DASHBOARD_OUT_DIR env → /var/www/trade
 
 import argparse
 import json
+import hashlib
 import logging
 import math
 import numbers
@@ -48,6 +49,7 @@ from datetime import datetime, timezone
 from logging.handlers import TimedRotatingFileHandler
 from typing import Dict, List, Optional
 
+from agent_panel import attach_panel
 from config import PlatformConfig, Timeframe, XAUUSD
 from main import report_to_dict
 from news_engine import NewsEngine
@@ -76,6 +78,11 @@ SCAN_BUFFER_SECONDS = 10
 SNAPSHOT_FILE = "snapshot.json"
 NEWS_FILE = "news.json"
 PAGE_FILE = "dashboard.html"
+SUMMARY_PHP_FILE = "news_summary.php"
+# Repo files copied into the out dir at daemon start (git pull + restart
+# updates them). The PHP endpoint reads news.json in place, so it lives
+# next to the page.
+STATIC_FILES = [PAGE_FILE, SUMMARY_PHP_FILE]
 
 
 # ══════════════════════════════════════════════════════════════════
@@ -180,9 +187,15 @@ def event_to_dict(ev) -> dict:
 
 
 def headline_to_dict(h) -> dict:
+    # id uses the same sha1(headline) scheme as NewsEngine._llm_sent_cache;
+    # it is the join key the PHP summary endpoint whitelists against.
     return {
+        "id": hashlib.sha1(h.headline.encode("utf-8")).hexdigest(),
         "headline": h.headline,
         "source": h.source,
+        "link": h.link,
+        "description": h.description,
+        "publisher": h.publisher,
         "timestamp": h.timestamp.astimezone(timezone.utc).isoformat()
                      if h.timestamp else None,
         "currencies": list(h.currencies_mentioned),
@@ -231,8 +244,11 @@ def seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
     elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
     interval_seconds = interval_minutes * 60
     remaining = interval_seconds - elapsed_seconds + buffer
-    if remaining <= buffer:
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
-        remaining += interval_seconds
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
     return remaining
 
 
@@ -321,22 +337,34 @@ class DashboardPublisher:
         return self._news_engine
 
     def install_static(self) -> None:
-        """Copy the repo's dashboard.html into the out dir (atomic), so
+        """Copy the repo's static files (page + PHP summary endpoint)
-        git pull + restart also updates the page."""
+        into the out dir (atomic), so git pull + restart also updates
-        src = os.path.join(os.path.dirname(os.path.abspath(__file__)),
+        them. The ai/ cache dir is NOT created here — the daemon runs as
-                           "static", PAGE_FILE)
+        a different user than Apache; that dir is a one-time deploy step
-        if not os.path.exists(src):
+        (deploy/README-dashboard.md)."""
-            logger.warning(f"{src} missing — page not installed")
+        static_dir = os.path.join(
-            return
+            os.path.dirname(os.path.abspath(__file__)), "static")
-        dst = os.path.join(self.out_dir, PAGE_FILE)
+        for name in STATIC_FILES:
-        shutil.copyfile(src, dst + ".tmp")
+            src = os.path.join(static_dir, name)
-        os.replace(dst + ".tmp", dst)
+            if not os.path.exists(src):
+                logger.warning(f"{src} missing — not installed")
+                continue
+            dst = os.path.join(self.out_dir, name)
+            shutil.copyfile(src, dst + ".tmp")
+            os.replace(dst + ".tmp", dst)
 
     def scan_tf(self, tf: Timeframe) -> None:
         started = time.time()
         try:
             report = produce_signal(XAUUSD, tf, self._provider(),
                                     CONFIRMATIONS[tf])
+            # Advisory AI panel, post-hoc and display-only. Off unless
+            # config.json agent_panel.dashboard is true. The reuse-TTL
+            # cache + max_panels_per_day budget bound the spend against
+            # this process's own DeepSeek cap (a persisting M5 signal
+            # re-serves its cached verdict instead of re-running every
+            # bar); a fresh panel adds ≤~25 s before this TF publishes.
+            attach_panel(report, surface="dashboard")
             self.snapshots[tf.value] = {
                 "status": "ok",
                 "scanned_at": datetime.now(timezone.utc).isoformat(),
@@ -368,18 +396,29 @@ class DashboardPublisher:
         except Exception:
             logger.exception("news refresh failed — keeping last payload")
 
-    def publish_all(self) -> None:
+    def publish_snapshot(self) -> None:
-        now = datetime.now(timezone.utc)
+        publish(build_snapshot_payload(
-        publish(build_snapshot_payload(self.snapshots, now),
+            self.snapshots, datetime.now(timezone.utc)),
-                os.path.join(self.out_dir, SNAPSHOT_FILE))
+            os.path.join(self.out_dir, SNAPSHOT_FILE))
+
+    def publish_news(self) -> None:
         if self.news_payload is not None:
             publish(self.news_payload, os.path.join(self.out_dir, NEWS_FILE))
 
+    def publish_all(self) -> None:
+        self.publish_snapshot()
+        self.publish_news()
+
     def run_cycle(self, tfs: List[Timeframe]) -> None:
         for tf in tfs:  # sequential — see module docstring (cache dedup)
             self.scan_tf(tf)
+        # Snapshot lands BEFORE the news refresh: when the news TTL has
+        # expired, refresh_news does FF + 4 RSS fetches (and possibly
+        # DeepSeek calls) — 20-30 s that must not keep fresh signals off
+        # the page.
+        self.publish_snapshot()
         self.refresh_news()
-        self.publish_all()
+        self.publish_news()
 
     def run_forever(self) -> None:
         self.run_cycle(DASHBOARD_TFS)  # populate the page at boot

deploy/README-dashboard.md

@@ -2,10 +2,12 @@
 
 The dashboard is a **static publisher**: `dashboard_publisher.py` runs as its
 own daemon, scans XAU/USD at candle closes, and atomically writes
-`dashboard.html` + `snapshot.json` + `news.json` into a folder Apache already
+`dashboard.html` + `news_summary.php` + `snapshot.json` + `news.json` into a
-serves. Apache does all web serving — no new port, no Python web server, and
+folder Apache already serves. Apache does all web serving — no new port, no
-the telegram bot process is completely unaffected by anything that happens
+Python web server, and the telegram bot process is completely unaffected by
-here (see the module docstring for the independence rules).
+anything that happens here (see the module docstring for the independence
+rules). The one dynamic piece is `news_summary.php` (§6): an on-demand
+DeepSeek summary for a clicked headline, whitelist-gated against news.json.
 
 All steps below run **on the live server** (`/home/Forex/ForexBot2`).
 
@@ -22,9 +24,10 @@ No new Python dependencies are required.
 ## 2. Make the web folder writable
 
 The dashboard publishes straight into the portal docroot `/var/www/trade/`
-(owner decision 2026-08-03) — it adds exactly three files there:
+(owner decision 2026-08-03) — it adds exactly four files there:
-`dashboard.html`, `snapshot.json`, `news.json`. The daemon user must be able
+`dashboard.html`, `news_summary.php`, `snapshot.json`, `news.json` (plus the
-to write them:
+apache-owned `ai/` cache dir from §6). The daemon user must be able to write
+them:
 
 ```bash
 # if the portal files can be owned by the daemon user:
@@ -72,8 +75,9 @@ every 3 h (240 backups), same scheme as the bot.
 
 ## 5. Optional Apache polish
 
-The page cache-busts its JSON fetches (`?t=<epoch>`), so no Apache config is
+The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache
-strictly needed. To be explicit anyway:
+answers unchanged files with 304), so no Apache config is strictly needed. To
+be explicit anyway:
 
 ```apache
 <Directory "/var/www/trade">
@@ -87,15 +91,67 @@ The page is deliberately public (owner decision 2026-08-03). To restrict it
 later, add standard Apache basic auth (`htpasswd` + `Require valid-user`) on
 that directory — no code changes needed.
 
+## 6. AI summary endpoint (PHP) — one-time setup
+
+Clicking a newswire headline opens a detail dialog that fetches an on-demand
+DeepSeek gold-impact summary from `news_summary.php` (installed next to the
+page by the daemon). Owner decision 2026-08-07: a real on-demand endpoint on
+the public page, accepted with these guards baked into the PHP — the id must
+match a headline in the published news.json (arbitrary text can never reach
+the LLM), each headline's summary is cached forever in `ai/` (≤1 API call per
+headline, ever), and a file-based daily cap fails **closed** on I/O errors.
+
+Requirements: `php` + the curl extension (`php -m | grep curl`).
+
+```bash
+# 1. SELinux: allow httpd outbound HTTPS — WITHOUT this every summary call
+#    dies as a curl timeout even with perfect perms (RHEL-family default).
+sudo setsebool -P httpd_can_network_connect 1
+
+# 2. API key OUTSIDE the docroot, readable by Apache only:
+sudo install -d -o root -g root -m 755 /etc/gold-dashboard
+sudo sh -c 'umask 137; echo "sk-..." > /etc/gold-dashboard/deepseek.key'
+sudo chown root:apache /etc/gold-dashboard/deepseek.key   # mode 640
+
+# 3. apache-owned cache dir (the daemon runs as Forex and can't create it):
+sudo install -d -o apache -g apache -m 755 /var/www/trade/ai
+sudo semanage fcontext -a -t httpd_sys_rw_content_t '/var/www/trade/ai(/.*)?'
+sudo restorecon -Rv /var/www/trade/ai
+```
+
+Knobs (Apache `SetEnv` in the vhost, both optional):
+`AI_SUMMARY_DAILY_CAP` (default 150), `DEEPSEEK_KEY_FILE` (default
+`/etc/gold-dashboard/deepseek.key`).
+
+Verify from the shell — expect 400 / 404 / 200 / 200-from-cache:
+
+```bash
+curl -s "http://<server>/news_summary.php?id=zzz"                 # {"error":"bad_id"}
+curl -s "http://<server>/news_summary.php?id=$(printf 'a%.0s' {1..40})"  # unknown_id
+ID=$(python3 -c "import json;print(json.load(open('/var/www/trade/news.json'))['headlines'][0]['id'])")
+curl -s "http://<server>/news_summary.php?id=$ID"   # real summary (2-12 s)
+curl -s "http://<server>/news_summary.php?id=$ID"   # instant, from ai/<id>.json
+```
+
 ## Operational notes
 
 - **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
-  re-copies `static/dashboard.html` into the web folder at startup.
+  re-copies `static/dashboard.html` and `static/news_summary.php` into the
+  web folder at startup.
 - **If the daemon dies**, systemd restarts it in 30 s; worst case the page
   shows stale data with its own "updated N min ago" evidence. Apache and the
   telegram bot are unaffected.
 - **DeepSeek**: the publisher process has its own daily request cap
   (`deepseek.daily_request_cap`, default 500) separate from the bot's —
   expect roughly up to ~430 extra calls/day worst case, V4-Flash pricing.
+- **Two independent DeepSeek meters** hit the same API key: the publisher's
+  in-process cap above, and the PHP endpoint's file cap (`ai/.ht_cap.json`,
+  default 150/day). They don't communicate — worst-case spend is their sum;
+  DeepSeek-side spend alerting is the true backstop. Kept separate on
+  purpose (independence rule: the PHP never imports/reads bot code or
+  config.json).
+- **Restart window**: for the first minutes after a daemon restart the new
+  page can meet a pre-upgrade news.json without headline ids — those rows
+  simply render non-clickable until the first news publish.
 - **Credentials**: uses the same env/.env/config.json resolution as the rest
   of the platform (Oanda candles feed; news needs no key; DeepSeek optional).

docs/architecture_diagram.py

@@ -0,0 +1,199 @@
+"""ForexBot2 architecture diagram → docs/architecture.png.
+
+Regenerate after architecture changes:  venv/bin/python docs/architecture_diagram.py
+(matplotlib only — no graphviz on the dev box)."""
+import os
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
+
+W, H = 100.0, 100.0
+fig, ax = plt.subplots(figsize=(16, 11.5))
+ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
+
+INK = "#263238"; MUT = "#607D8B"; ARR = "#455A64"; HAUL = "#7B8FA1"
+
+def box(x, y, w, h, title, lines, fc, ec, title_fs=8.2, body_fs=6.6, lw=1.3):
+    ax.add_patch(FancyBboxPatch((x, y), w, h,
+                 boxstyle="round,pad=0.35,rounding_size=0.9",
+                 facecolor=fc, edgecolor=ec, linewidth=lw, zorder=2))
+    cy = y + h - 1.9
+    ax.text(x + w / 2, cy, title, ha="center", va="center", fontsize=title_fs,
+            fontweight="bold", color=INK, zorder=3)
+    cy -= 2.3
+    for ln in lines:
+        ax.text(x + w / 2, cy, ln, ha="center", va="center", fontsize=body_fs,
+                color=INK, zorder=3)
+        cy -= 1.85
+
+def band(x, y, w, h, label, color):
+    ax.add_patch(FancyBboxPatch((x, y), w, h,
+                 boxstyle="round,pad=0.3,rounding_size=1.2",
+                 facecolor="none", edgecolor=color, linewidth=1.0,
+                 linestyle=(0, (4, 3)), zorder=1))
+    ax.text(x + 0.9, y + h - 1.4, label, ha="left", va="center",
+            fontsize=7.2, fontweight="bold", color=color, zorder=3)
+
+def arrow(p1, p2, color=ARR, lw=1.5, style="-|>", conn="arc3,rad=0.0",
+          label=None, lx=0, ly=0, ls="solid", fs=6.4, rot=0):
+    ax.add_patch(FancyArrowPatch(p1, p2, arrowstyle=style, mutation_scale=11,
+                 color=color, linewidth=lw, linestyle=ls,
+                 connectionstyle=conn, zorder=4))
+    if label:
+        mx, my = (p1[0] + p2[0]) / 2 + lx, (p1[1] + p2[1]) / 2 + ly
+        ax.text(mx, my, label, fontsize=fs, color=MUT, ha="center",
+                va="center", zorder=5, rotation=rot,
+                bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
+
+# ── title ────────────────────────────────────────────────────────────
+ax.text(2, 98.4, "ForexBot2 — XAU/USD Signal-Intelligence Platform",
+        fontsize=15, fontweight="bold", color=INK, ha="left")
+ax.text(2, 96.0, "one pipeline, four delivery surfaces · closed bars only · live == backtest parity",
+        fontsize=8.5, color=MUT, ha="left")
+
+EXT_FC, EXT_EC = "#ECEFF1", "#90A4AE"
+DAT_FC, DAT_EC = "#E3F2FD", "#5C9BD1"
+NWS_FC, NWS_EC = "#E0F2F1", "#3D9C8F"
+COR_FC, COR_EC = "#FFFDF5", "#C9A227"
+REP_FC, REP_EC = "#FFE9A8", "#B8860B"
+DEL_FC, DEL_EC = "#E8F5E9", "#5FAF63"
+BTS_FC, BTS_EC = "#EDE7F6", "#8E6BC6"
+
+# ── external services (top band) ─────────────────────────────────────
+band(1, 85.0, 98, 9.4, "EXTERNAL SERVICES", "#78909C")
+box(2.5, 86.2, 15.5, 6.4, "Oanda v20 API",
+    ["candles (live feed)", "practice orders"], EXT_FC, EXT_EC)
+box(19.5, 86.2, 15.5, 6.4, "Twelve Data · Tiingo",
+    ["registered fallbacks", "(Tiingo: tick stream)"], EXT_FC, EXT_EC)
+box(36.5, 86.2, 15.5, 6.4, "Forex Factory",
+    ["economic calendar", "MyFXBook RSS fallback"], EXT_FC, EXT_EC)
+box(53.5, 86.2, 12.5, 6.4, "News RSS ×4",
+    ["gold / macro", "headlines"], EXT_FC, EXT_EC)
+box(67.5, 86.2, 13.5, 6.4, "DeepSeek API",
+    ["V4-Flash LLM", "thinking: disabled"], EXT_FC, EXT_EC)
+box(82.5, 86.2, 15, 6.4, "Telegram API",
+    ["owner bot +", "subscriber channel"], EXT_FC, EXT_EC)
+
+# ── data & context layer ─────────────────────────────────────────────
+box(2.5, 69.8, 33, 11.0, "providers/  (BaseDataProvider)",
+    ["single-flight fetch · TTL+boundary cache",
+     "forming-candle DROP · validate/clean",
+     "paginated fetch_range · optional tick stream",
+     "active feed knob: data_providers.active = oanda"],
+    DAT_FC, DAT_EC)
+box(38.5, 69.8, 33, 11.0, "news_engine.py  +  llm_client.py",
+    ["calendar events + keyword sentiment (gold-inverse)",
+     "DeepSeek enrichment: batch headline scores,",
+     "event gold-direction, market blurb — display-only,",
+     "keyword fallback, daily request cap"],
+    NWS_FC, NWS_EC)
+box(74.5, 69.8, 23, 11.0, "app_config.py",
+    ["credentials & deployment:",
+     "env → config.json → refuse",
+     "(no hardcoded fallbacks)",
+     ".env loaded at import"],
+    "#FAFAFA", "#9E9E9E")
+
+# ── signal core container ────────────────────────────────────────────
+ax.add_patch(FancyBboxPatch((2.5, 40.5), 95, 26.5,
+             boxstyle="round,pad=0.35,rounding_size=1.2",
+             facecolor=COR_FC, edgecolor=COR_EC, linewidth=1.6, zorder=1))
+ax.text(50, 65.2, "SIGNAL CORE — runs identically in live scans and backtests (closed bars only)",
+        ha="center", fontsize=8.4, fontweight="bold", color="#8a6d1a", zorder=3)
+
+box(4.5, 51.8, 28, 12.0, "engine.py — TechnicalEngine",
+    ["9 scored indicators + ATR context:",
+     "EMA · RSI · MACD · BB · ADX · ROC ·",
+     "Keltner · divergence · candlestick",
+     "regime → adaptive weights → composite"],
+    "white", COR_EC)
+box(4.5, 42.0, 28, 8.4, "pair_signals/xauusd.py — factory",
+    ["ALWAYS used: config.json overlay +",
+     "validated per-TF deltas (hour blocks, trail…)"],
+    "white", COR_EC)
+box(35.5, 42.0, 60, 20.7, "signal_aggregator.py — analyze_pair → analyze_from_dataframes",
+    ["multi-TF agreement  (M5→[M15,H1] · M15→[M5,H1] · H1→[M15,H4] · H4→[H1,D1])",
+     "→ news context (warn-only by default) → diminishing-decay penalties",
+     "→ exhaustion hard block → quality grade A+…F",
+     "→ ordered post-grade gate chain (session · XAU SELL vol/hour · regime · M5 noise)",
+     "→ setup generators: XAU ATR (A+/A) · generic swing · XAU_PB · XAU_CP",
+     "→ entry · SL · TP1/TP2 · trailing plan"],
+    "white", COR_EC, body_fs=6.9)
+
+# ── SignalReport hub ─────────────────────────────────────────────────
+box(31, 33.2, 38, 5.2, "SignalReport",
+    ["signal (−2…+2) · grade · trade setup · reasons · news summary"],
+    REP_FC, REP_EC, title_fs=8.8)
+
+# ── delivery surfaces ────────────────────────────────────────────────
+band(1, 13.2, 98, 17.2, "DELIVERY SURFACES", "#4C8F50")
+box(2.5, 15.0, 12, 12.6, "main.py",
+    ["CLI report", "--detail", "--json"], DEL_FC, DEL_EC)
+box(16, 15.0, 24, 12.6, "oanda_daemon.py + executor",
+    ["practice auto-execution",
+     "book-parity rules: no broker TP,",
+     "5-bar cooldown, once-per-bar",
+     "trail, OPEN_ONLY fills,",
+     "boot reconcile · orders → Oanda"],
+    DEL_FC, DEL_EC, title_fs=7.8)
+box(42.5, 15.0, 31, 12.6, "dashboard_publisher.py  →  Apache",
+    ["independent daemon · scans at candle closes",
+     "atomic publish to /var/www/trade:",
+     "dashboard.html · snapshot.json · news.json",
+     "+ news_summary.php — on-demand AI",
+     "gold-impact summary (whitelist · cache · cap)"],
+    DEL_FC, DEL_EC)
+box(76, 15.0, 21.5, 12.6, "telegram_bot.py",
+    ["candle-boundary auto-scans", "quality threshold + dedup",
+     "alerts + LLM context line", "WS level-breach monitor"],
+    DEL_FC, DEL_EC)
+
+# ── backtest / QA ────────────────────────────────────────────────────
+box(2.5, 3.2, 71.8, 8.0, "backtest/gpu_backtest.py — bar-by-bar replay through the SAME analyze_from_dataframes",
+    ["realism (results-critical): next-open entries · raw-path resolution · spread charged · SL-first ties · close-only trail",
+     "CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
+    BTS_FC, BTS_EC)
+box(77.5, 3.2, 20, 8.0, "test_all.py",
+    ["359 tests · 22 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
+
+# ── arrows: short structural hops (solid dark) ───────────────────────
+arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2)          # oanda → providers
+arrow((27, 86.2), (24, 81.2), ls=(0, (3, 2)), lw=1.1)                  # fallbacks → providers
+arrow((44, 86.2), (48, 81.2))                                          # FF → news
+arrow((59.5, 86.2), (56, 81.2))                                        # RSS → news
+arrow((72, 86.2), (62, 81.2), label="LLM", lx=3.4, ly=0.6)             # deepseek → news
+arrow((19, 69.8), (19, 64.2), label="DataFrames (UTC, closed bars)", lx=0.2, ly=0.9)
+arrow((33.5, 57.5), (35.5, 55.5))                                      # engine → aggregator
+arrow((32.5, 46.0), (35.5, 48.0))                                      # factory → aggregator
+arrow((55, 69.8), (55, 63.1), label="warn-only context", lx=0, ly=1.0)
+arrow((86, 69.8), (86, 67.3), ls=(0, (3, 2)), lw=1.1)                  # config → core
+arrow((50, 40.5), (50, 38.8))                                          # core → report
+arrow((36, 34.5), (8.5, 27.95), conn="arc3,rad=0.12")                  # report → cli
+arrow((40, 33.2), (28, 27.95), conn="arc3,rad=0.06")                   # report → daemon
+arrow((60, 33.2), (58, 27.95))                                         # report → dashboard
+arrow((66, 34.5), (86.5, 27.95), conn="arc3,rad=-0.12")                # report → telegram
+
+# ── long-haul edges (thin dashed, routed through gutters) ────────────
+# telegram bot ↔ Telegram API — right-margin arc, clear of app_config
+arrow((95, 86.2), (95, 27.95), conn="arc3,rad=0.15", color=HAUL,
+      lw=1.2, ls=(0, (4, 2)))
+ax.text(99.2, 57, "alerts / commands", fontsize=6.4, color=MUT,
+        rotation=90, ha="center", va="center")
+# browser click → news_summary.php → DeepSeek
+# (oanda_daemon's order flow is noted inside its box — no long arc)
+arrow((66, 27.95), (73, 86.2), conn="arc3,rad=-0.18", color=HAUL,
+      lw=1.2, ls=(0, (4, 2)),
+      label="on-demand AI summary\n(browser click → PHP)", lx=2.5, ly=11.6)
+# providers.fetch_range → backtester — far-left margin channel
+arrow((1.5, 69.8), (1.5, 11.0), color=BTS_EC, lw=1.1, ls=(0, (3, 2)),
+      label="fetch_range\n(history)", lx=3.6, ly=-1.2)
+# backtester replays the signal core — right gutter between boxes
+arrow((74.0, 11.6), (74.9, 40.3), color=HAUL, lw=1.3)
+ax.text(81.8, 38.9, "replays the exact signal code", fontsize=6.4,
+        color=MUT, ha="center", va="center", zorder=5,
+        bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
+
+out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
+fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
+print(f"saved {out}")

llm_client.py

@@ -24,7 +24,8 @@ import datetime as _dt
 import json
 import logging
 import threading
-from typing import Any, Dict, Optional
+import time
+from typing import Any, Dict, Optional, Tuple
 
 import requests
 
@@ -66,6 +67,10 @@ class DeepSeekClient:
         self._count_day: Optional[_dt.date] = None
         self._count = 0
         self._warned: set = set()
+        # Health stamps for status displays: last successful call and
+        # last failure (ts, kind). Read-only for consumers.
+        self.last_ok: float = 0.0
+        self.last_error: Optional[Tuple[float, str]] = None
 
     @property
     def available(self) -> bool:
@@ -125,9 +130,11 @@ class DeepSeekClient:
             resp = self._session.post(
                 DEEPSEEK_URL, json=body, timeout=self._timeout)
         except requests.RequestException as e:
+            self.last_error = (time.time(), "network")
             self._warn_once("network", "DeepSeek call failed: %s", e)
             return None
         if resp.status_code != 200:
+            self.last_error = (time.time(), f"http_{resp.status_code}")
             self._warn_once(
                 f"http_{resp.status_code}",
                 "DeepSeek HTTP %s: %s", resp.status_code, resp.text[:200])
@@ -137,8 +144,11 @@ class DeepSeekClient:
         except ValueError:
             parsed = None
         if parsed is None:
+            self.last_error = (time.time(), "parse")
             self._warn_once("parse", "DeepSeek response was not the "
                             "expected JSON object — ignoring")
+        else:
+            self.last_ok = time.time()
         return parsed
 
     @staticmethod

llm_provider_switch_plan.txt

@@ -0,0 +1,252 @@
+================================================================================
+MULTI-PROVIDER LLM LAYER - PLAN
+(DeepSeek for news summary, Anthropic Claude for agent panel)
+Written 2026-08-08. Plan only - no code has been changed.
+================================================================================
+
+WHY
+---
+Today every LLM feature (news enrichment + agent panel) goes through one
+hardwired client: llm_client.py -> DeepSeekClient, a module singleton reached
+via get_llm_client(). This plan describes how to route different features to
+different providers - e.g. keep DeepSeek for cheap high-volume news scoring,
+put the agent panel on Anthropic Claude for better reasoning.
+
+The repo already solved this problem once for market data: the providers/
+package (BaseDataProvider owns shared machinery, subclasses implement only
+_fetch_raw, a registry + one config knob selects the provider). This plan
+mirrors that proven pattern at the LLM layer, scaled down to a single module.
+
+BLAST RADIUS (verified by grep)
+-------------------------------
+get_llm_client() call sites:
+  - news_engine.py ....... 4 sites
+  - agent_panel.py ....... 3 sites (+1 timeout getattr)
+  - telegram_bot.py ...... 1 site (/status health; reads private _count,
+                           _count_day, _daily_cap fields)
+  - test_all.py .......... 41 locations stub via  lc._client = _StubLLM(...)
+Out of scope: static/news_summary.php (own PHP DeepSeek call + key file,
+separate deployment).
+
+DESIGN INVARIANTS (carried over unchanged)
+------------------------------------------
+1. Fail-safe contract: every call returns Optional[Dict]; any failure -> None;
+   callers keep their non-LLM fallbacks; LLM output never gates signals.
+2. Single attempt, no retries (news cache cadence is the retry); per-provider
+   UTC daily cap; warn-once logging; health stamps (last_ok, last_error).
+3. Static system prompts stay byte-identical constants - provider-neutral,
+   and they map to prompt caching on BOTH providers (DeepSeek prefix cache;
+   Anthropic cache_control ephemeral).
+4. Test compatibility: all 41 existing lc._client stub sites keep working
+   with zero edits.
+
+================================================================================
+1. llm_client.py REFACTOR  (~195 -> ~430 lines, no new files)
+================================================================================
+
+1a. BaseLLMClient - shared machinery (extracted from today's DeepSeekClient)
+    - __init__(api_key, model, timeout, daily_cap, enabled)
+    - public: name ("deepseek"/"anthropic"), timeout property,
+      usage() -> (used_today, cap)   # so telegram_bot stops reading privates
+    - owns: available property, _warn_once, _under_cap() (UTC-day counter,
+      lock), health stamps, and the template method:
+
+        def chat_json(self, system_prompt, user_payload):
+            if not self.available or not self._under_cap():
+                return None
+            try:
+                return self._request(system_prompt, user_payload)  # hook
+            except <provider-agnostic error mapping>:
+                # stamps last_error, returns None
+
+    - subclasses implement ONLY _request(system_prompt, payload) -> dict|None.
+
+1b. DeepSeekClient(BaseLLMClient) - behavior-identical move
+    Current requests-based logic relocates into _request: same URL,
+    thinking:disabled, response_format:json_object, temperature 0,
+    max_tokens 2000, sort_keys=True payload, _parse_response.
+    Review criterion: Module 19 tests pass UNMODIFIED (they patch
+    client._session, which stays).
+
+1c. AnthropicClient(BaseLLMClient) - new, official SDK
+    - Official `anthropic` Python SDK (add to requirements.txt), NOT raw
+      requests. LAZY IMPORT inside __init__: package missing -> warn-once +
+      client unavailable, so DeepSeek-only deployments never install it.
+    - Constructor: anthropic.Anthropic(api_key=..., timeout=<configured>,
+      max_retries=0)   # house rule: single attempt
+    - _request:
+        resp = self._client.messages.create(
+            model=self._model,                  # default "claude-opus-5"
+            max_tokens=4000,                    # covers thinking + JSON reply
+            output_config={"effort": "low"},    # cheap structured takes
+            system=[{"type": "text", "text": system_prompt,
+                     "cache_control": {"type": "ephemeral"}}],
+            messages=[{"role": "user",
+                       "content": json.dumps(user_payload, sort_keys=True)}],
+        )
+        if resp.stop_reason == "refusal":       # Opus 5: HTTP 200, check first
+            return None
+        text = first text block in resp.content
+        return self._parse_json(text)   # tolerant: strip ```json fences,
+                                        # json.loads, dict check
+    - DELIBERATELY ABSENT (would 400 on Claude Opus 5): temperature (removed
+      on Opus 5 - do NOT copy from the DeepSeek client), budget_tokens,
+      prefills, response_format.
+    - Error mapping: APIConnectionError -> "network",
+      APIStatusError -> "http_<code>", parse fail -> "parse"
+      (same last_error kinds the /status display already understands).
+    - Defaults: model claude-opus-5 (config-overridable, e.g. claude-haiku-4-5
+      for ~5x cheaper), timeout 30s (Claude slower than V4-Flash), own
+      daily_request_cap 500.
+
+1d. Registry + per-consumer routing
+        _PROVIDERS = {"deepseek": DeepSeekClient, "anthropic": AnthropicClient}
+        LLM_CONSUMERS = ("news", "agent_panel")
+        _clients = {}     # keyed by PROVIDER name -> two consumers on the
+                          # same provider share one instance and one cap
+        _client = None    # kept: GLOBAL TEST OVERRIDE
+
+        def get_llm_client(consumer="news"):
+            if _client is not None:          # tests: lc._client = stub
+                return _client               #   -> shadows ALL consumers
+            name = get_app_config().llm_routing.get(consumer, "deepseek")
+            create-once per provider, return _clients[name]
+
+        def reset_llm_client():              # clears _client AND _clients
+
+    Keeping _client as a documented test-override global is what makes the
+    41 existing stub sites work untouched.
+
+================================================================================
+2. CONFIG  (app_config.py + config.json.example)
+================================================================================
+- New top-level "anthropic" block, mirroring "deepseek":
+    {api_key, model, enabled, timeout_seconds, daily_request_cap}
+  plus anthropic_api_key() resolver: ANTHROPIC_API_KEY env -> config.json -> "".
+- New top-level "llm" ROUTING block:
+    {"news": "deepseek", "agent_panel": "deepseek"}
+  Validation: keys within {news, agent_panel}; values within registered
+  provider names. Absent block -> {} -> everything on DeepSeek (today's
+  behavior; zero-migration default). AppConfig.llm_routing: Dict[str, str].
+
+- The target state is then PURE CONFIG - no code changes on switch day:
+
+    "llm":       { "news": "deepseek", "agent_panel": "anthropic" },
+    "anthropic": { "api_key": "", "model": "claude-opus-5", "enabled": true,
+                   "timeout_seconds": 30, "daily_request_cap": 500 }
+
+================================================================================
+3. CONSUMER EDITS  (mechanical, ~10 lines total)
+================================================================================
+- news_engine.py:  4 call sites -> get_llm_client("news")
+- agent_panel.py:  3 call sites -> get_llm_client("agent_panel");
+                   getattr(client, "_timeout", ...) -> public client.timeout
+- telegram_bot.py _news_health_lines: iterate the DISTINCT providers actually
+  routed (news + panel); per provider print "<name>: used/cap calls today"
+  via the new usage() accessor instead of private fields. DeepSeek keeps its
+  "headlines scored" extra; cap/error warnings work off base-class stamps.
+- NO changes to prompts, payload builders, caches, or the panel's own
+  max_panels_per_day budget - all provider-agnostic already.
+
+================================================================================
+4. TESTS  (new "Module 24: LLM provider routing", ~12 tests, ~200 lines)
+================================================================================
+Reuse _StubLLM; new _FakeAnthropic (fake messages.create returning a stub
+response object) injected as client._client.
+
+ 1. app_config: anthropic block parses/validates/defaults empty.
+ 2. app_config: llm routing block - valid map round-trips; unknown consumer
+    key, unknown provider name, non-dict -> AppConfigError; absent -> {}.
+ 3. anthropic_api_key resolves env -> config.json -> empty.
+ 4. Base contract on BOTH subclasses (parametrized): keyless/disabled -> no
+    call; daily cap blocks + UTC-roll reset; failure stamps last_error,
+    success stamps last_ok.
+ 5. AnthropicClient parse: clean JSON -> dict; ```json-fenced -> dict;
+    non-JSON -> None; stop_reason "refusal" -> None; no text block -> None.
+ 6. AnthropicClient request shape: NO temperature key, max_retries=0,
+    cache_control present on system block (assert on captured kwargs).
+ 7. AnthropicClient with `anthropic` package missing (simulated) ->
+    available False, chat_json None, no crash at import.
+ 8. Routing: no llm block -> both consumers get the SAME DeepSeek singleton
+    (shared cap); routed config -> news gets DeepSeekClient while
+    agent_panel gets AnthropicClient; same-provider consumers share one
+    instance.
+ 9. Global lc._client override shadows BOTH consumers (locks the 41 legacy
+    stub sites' semantics).
+10. reset_llm_client() clears both the override and the per-provider cache.
+11. End-to-end: routing agent_panel->anthropic, attach_panel reaches the
+    anthropic fake (5 calls) while a news call reaches the deepseek stub.
+12. telegram _news_health_lines renders per-provider lines from usage()
+    without touching private fields.
+
+GATE: Modules 19 and 23 must pass UNMODIFIED - the no-regression proof for
+the refactor.
+
+================================================================================
+5. DOCS
+================================================================================
+- CLAUDE.md: llm_client bullet -> provider-abstracted (BaseLLMClient;
+  deepseek + anthropic registered); config.json "llm" block routes consumers;
+  absent = all-DeepSeek. Add ANTHROPIC_API_KEY to env-var list. Test counts.
+- config.json.example: both new blocks.
+
+================================================================================
+COMMIT SEQUENCE  (each commit green)
+================================================================================
+1. llm_client: extract BaseLLMClient; DeepSeekClient unchanged behavior
+   (+ base-contract tests)  - pure refactor, Modules 19/23 green untouched.
+2. config: anthropic block + llm consumer-routing block (+ config tests).
+3. llm_client: AnthropicClient (official SDK, lazy import) + requirements.txt
+   (+ parse/shape tests).
+4. routing: per-consumer get_llm_client + health lines - consumer call-site
+   edits, telegram usage() lines, routing/e2e tests.
+5. docs: CLAUDE.md + example config.
+
+================================================================================
+VERIFICATION
+================================================================================
+- python test_all.py green at every commit (~374 -> ~386).
+- Live smoke A (no behavior change): default config, main.py --panel ->
+  still DeepSeek, /status unchanged.
+- Live smoke B (the goal): llm.agent_panel="anthropic" + real
+  ANTHROPIC_API_KEY, run --panel -> one real Claude panel (5 requests,
+  validates JSON contract + latency + refusal-free path) while a news
+  refresh still hits DeepSeek; /status shows two provider lines.
+- Backtest untouched by construction (no LLM in replay path); one 30-day
+  run as a formality.
+
+================================================================================
+EFFORT ESTIMATE
+================================================================================
+0 new files, 8 modified, ~520 lines, est. 3-5 hours.
+
+  llm_client.py        base class + AnthropicClient + routing   +235
+  test_all.py          Module 24                                +200
+  app_config.py        anthropic + llm blocks, resolver          +55
+  telegram_bot.py      per-provider health lines                 +15
+  news_engine.py       consumer tags on call sites                +4
+  agent_panel.py       consumer tags on call sites                +4
+  config.json.example  blocks                                    +14
+  requirements.txt     anthropic                                  +1
+  CLAUDE.md            docs                                      +10
+
+================================================================================
+KEY DECISIONS EMBEDDED IN THIS PLAN
+================================================================================
+- Official Anthropic SDK, not raw requests (current API guidance for Python;
+  lazy import keeps it optional for DeepSeek-only installs).
+- Default Claude model: claude-opus-5 ($5/$25 per MTok; a full 5-call panel
+  ~ 8k in / 500 out ~ $0.05). Config-overridable to claude-haiku-4-5.
+- Two Claude gotchas handled: temperature is REMOVED on Opus 5 (sending it
+  400s - must not be copied from the DeepSeek request body), and refusals
+  arrive as HTTP 200 with stop_reason "refusal" (checked before parsing,
+  mapped to the normal None/fallback path).
+- Caps are per-provider, not per-consumer: two consumers on one provider
+  share its cap (today's semantics); splitting across two providers doubles
+  total headroom naturally.
+- The panel's own max_panels_per_day budget is untouched - it sits above
+  whichever provider serves it.
+- Not in scope: static/news_summary.php (own PHP client/key), streaming,
+  tool use, per-consumer model overrides (add a "model" key per consumer
+  later if ever needed).
+================================================================================

main.py

@@ -20,6 +20,7 @@ if hasattr(sys.stdout, "reconfigure"):
 if hasattr(sys.stderr, "reconfigure"):
     sys.stderr.reconfigure(encoding="utf-8")
 
+from agent_panel import attach_panel
 from config import PlatformConfig, Timeframe, SignalStrength, XAUUSD
 from pair_signals import PAIR_PRODUCERS
 from signal_aggregator import SignalAggregator, SignalReport
@@ -163,6 +164,13 @@ def print_signal_report(report: SignalReport, detail: bool = False):
         print(f"  {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
     if report.news_summary:
         print(f"  {Colors.MAGENTA}🧠{Colors.RESET} {report.news_summary}")
+    if report.panel_verdict:
+        print(f"  {Colors.CYAN}🤖{Colors.RESET} AI Panel: "
+              f"{Colors.BOLD}{report.panel_verdict}{Colors.RESET} "
+              f"({report.panel_confidence:.0f}%) — {report.panel_rationale}")
+        for t in report.panel_takes:
+            print(f"     {Colors.GRAY}{t['persona']:<11} {t['stance']:<8} "
+                  f"{t['note']}{Colors.RESET}")
 
     # ── 6. DETAIL (only with --detail) ─────────────────────────────
     if detail and report.primary_analysis:
@@ -209,6 +217,10 @@ def report_to_dict(report: SignalReport) -> dict:
         "news_sentiment": round(report.news_sentiment, 3),
         "news_description": report.news_description,
         "news_summary": report.news_summary,
+        "panel_verdict": report.panel_verdict,
+        "panel_confidence": round(report.panel_confidence, 1),
+        "panel_rationale": report.panel_rationale,
+        "panel_takes": report.panel_takes,
         "bullish_reasons": report.bullish_reasons,
         "bearish_reasons": report.bearish_reasons,
         "warnings": report.warnings,
@@ -268,6 +280,9 @@ Examples:
                         help="Primary analysis timeframe (default: H1)")
     parser.add_argument("--detail", action="store_true",
                         help="Show full indicator breakdown")
+    parser.add_argument("--panel", action="store_true",
+                        help="Run the LLM agent panel on the result "
+                             "(advisory only; needs a DeepSeek key)")
     parser.add_argument("--json", action="store_true",
                         help="Output as JSON")
     parser.add_argument("--verbose", "-v", action="store_true",
@@ -323,6 +338,11 @@ def run_live(args, config: PlatformConfig):
     aggregator = SignalAggregator(config)
     report = aggregator.analyze_pair(symbol=config.symbol)
 
+    # Explicit user intent: force bypasses the enabled/surface/quality
+    # toggles, but the daily panel budget and LLM cap still apply.
+    if getattr(args, "panel", False):
+        attach_panel(report, surface="cli", force=True)
+
     if args.json:
         print(json.dumps(report_to_dict(report), indent=2))
         return

news_engine.py

@@ -9,6 +9,7 @@ A perfect technical setup can be destroyed by an unexpected NFP print.
 """
 
 import hashlib
+import html
 import logging
 import re
 import threading
@@ -57,12 +58,22 @@ Examples:
 
 _LLM_EVENT_SYSTEM = """\
 You are a forex/gold analyst. You receive one released economic event as
-JSON: {"event", "currency", "actual", "forecast", "previous"}.
+JSON: {"event", "currency", "actual", "forecast", "previous",
-Decide the likely effect of this RELEASE (actual vs forecast) on the
+"vs_forecast"}. "vs_forecast" states whether the actual print came in
-GOLD (XAU/USD) price. A stronger-than-forecast USD economy or a hawkish
+above or below the forecast; it is computed programmatically — trust it
-implication is usually gold-bearish; weaker data or a dovish implication
+over your own comparison of the numbers. If it is "unknown", compare
-is usually gold-bullish. Mind inverted metrics: higher unemployment or
+actual and forecast yourself.
-jobless claims = weaker economy.
+Decide the likely effect of this RELEASE on the GOLD (XAU/USD) price.
+A stronger-than-forecast USD economy or a hawkish implication is usually
+gold-bearish; weaker data or a dovish implication is usually
+gold-bullish.
+Inverted metrics — for unemployment rate and unemployment/jobless
+claims a LOWER number means a STRONGER labor market: below forecast =
+strong economy = usually gold-bearish; above forecast = weak economy =
+usually gold-bullish.
+Example: {"event": "Unemployment Claims", "actual": "202K", "forecast":
+"217K", "vs_forecast": "below forecast"} ->
+{"gold_direction": "bearish", "reason": "fewer claims show strong labor market"}
 Reply with ONLY JSON:
 {"gold_direction": "bullish"|"bearish"|"neutral", "reason": "<max 10 words>"}
 """
@@ -93,6 +104,44 @@ def _parse_release_value(raw: Optional[str]) -> Optional[float]:
     return float(match.group(1)) * multiplier
 
 
+_DESC_MAX_CHARS = 500
+_MAX_LINK_CHARS = 2048
+
+
+def _clean_html_text(raw: Optional[str],
+                     limit: int = _DESC_MAX_CHARS) -> Optional[str]:
+    """RSS <description> → plain text: unescape entities, strip tags,
+    collapse whitespace, truncate. Unescape and strip both run twice
+    because feeds routinely double-encode (&amp;lt;p&amp;gt; wrapping
+    real markup) — the second unescape can reveal fresh tags."""
+    if not raw:
+        return None
+    text = html.unescape(raw)
+    text = re.sub(r"<[^>]+>", " ", text)
+    text = html.unescape(text)
+    text = re.sub(r"<[^>]+>", " ", text)
+    text = re.sub(r"\s+", " ", text).strip()
+    if not text:
+        return None
+    if len(text) > limit:
+        text = text[:limit].rstrip() + "…"
+    return text
+
+
+def _safe_link(raw: Optional[str]) -> Optional[str]:
+    """Article URL for display: http(s) only (an RSS <link> is untrusted
+    and lands in an href), bounded length (Google News redirect URLs run
+    1-2 KB; anything past that is bloat, not a link)."""
+    if not raw:
+        return None
+    link = raw.strip()
+    if not link.lower().startswith(("http://", "https://")):
+        return None
+    if len(link) > _MAX_LINK_CHARS:
+        return None
+    return link
+
+
 @dataclass
 class EconomicEvent:
     """Single economic calendar event."""
@@ -145,6 +194,11 @@ class NewsSentiment:
     # Per-currency score: monetary-policy keywords flip sign for XAU
     # (hawkish is dollar-bullish but gold-bearish).
     sentiment_by_currency: Dict[str, float] = field(default_factory=dict)
+    # Display-only enrichment (dashboard detail dialog). Optional with
+    # defaults so existing constructor calls keep working.
+    link: Optional[str] = None         # article URL (Google News = redirect)
+    description: Optional[str] = None  # plain text, tags stripped, ≤500 chars
+    publisher: Optional[str] = None    # Google News <source> text, else None
 
 
 @dataclass
@@ -250,6 +304,14 @@ class NewsEngine:
     _LLM_SENT_CACHE_MAX = 400
     _LLM_EVENT_CACHE_MAX = 100
 
+    # Per-source fetch health for status displays (telegram /status).
+    # name → {"ok": ts|None, "err": ts|None, "detail": str, "extra": str}.
+    # All writers run under _cache_lock (fetches happen inside
+    # _cached_fetch); readers render without the lock — worst case a
+    # display line is one cycle stale. A source is "failing" when its
+    # last error is newer than its last success.
+    _source_health: Dict[str, dict] = {}
+
     def __init__(self, config: NewsConfig):
         self.config = config
         self._session = requests.Session()
@@ -453,6 +515,36 @@ class NewsEngine:
         "canada": "CAD", "new zealand": "NZD",
     }
 
+    @classmethod
+    def _record_source(cls, name: str, ok: bool, detail: str = "",
+                       extra: str = "") -> None:
+        """Track one dependency's fetch outcome for status displays.
+        Logs state TRANSITIONS so the log file carries the story without
+        a line every 10-min cycle: first failure WARNING (with detail),
+        repeat failures DEBUG, recovery INFO."""
+        h = cls._source_health.setdefault(
+            name, {"ok": None, "err": None, "detail": "", "extra": ""})
+        was_failing = h["err"] is not None and (
+            h["ok"] is None or h["err"] > h["ok"])
+        if ok:
+            h["ok"] = time.time()
+            h["extra"] = extra
+            if was_failing:
+                logger.info("Source recovered: %s", name)
+        else:
+            h["err"] = time.time()
+            h["detail"] = detail
+            if was_failing:
+                logger.debug("Source still failing: %s (%s)", name, detail)
+            else:
+                logger.warning("Source failing: %s (%s)", name, detail)
+
+    @staticmethod
+    def source_is_failing(entry: Optional[dict]) -> bool:
+        """True when the entry's last error is newer than its last ok."""
+        return bool(entry and entry["err"] is not None and
+                    (entry["ok"] is None or entry["err"] > entry["ok"]))
+
     def _cached_fetch(self, cache_attr: str, fetch_fn) -> list:
         """Return cached data for `cache_attr` or refetch when the TTL
         expired. Successful results live for context_cache_ttl_seconds;
@@ -485,14 +577,27 @@ class NewsEngine:
         Returns an empty list (with a log warning) if unavailable —
         we never generate fake events since that causes constant false alarms.
         """
-        for fetcher in (self._try_fetch_calendar_api, self._try_fetch_myfxbook_rss):
+        sources = (
+            (self._try_fetch_calendar_api, "calendar:forexfactory",
+             "ForexFactory"),
+            (self._try_fetch_myfxbook_rss, "calendar:myfxbook", "MyFXBook"),
+        )
+        for fetcher, src, label in sources:
             try:
                 events = fetcher()
                 if events:
+                    self._record_source(src, True)
+                    self._record_source(
+                        "calendar", True,
+                        extra=f"{len(events)} events via {label}")
                     return events
+                self._record_source(src, False, "no events returned")
             except Exception as e:
+                self._record_source(
+                    src, False, f"{type(e).__name__}: {e}")
                 logger.debug(f"Calendar fetch exception ({fetcher.__name__}): {e}")
 
+        self._record_source("calendar", False, "all sources failed")
         logger.info(
             "Economic calendar data unavailable. "
             "Event risk checks will be skipped for this cycle."
@@ -639,11 +744,16 @@ class NewsEngine:
         """Fetch and analyze news headlines for sentiment."""
         headlines = []
         seen_titles = set()
+        ok_feeds = 0
 
         for feed_url in self.config.rss_feeds:
+            host = feed_url.split("/")[2] if "//" in feed_url else feed_url
+            kept_before = len(headlines)
             try:
                 resp = self._session.get(feed_url, timeout=10)
                 if resp.status_code != 200:
+                    self._record_source(
+                        f"rss:{host}", False, f"HTTP {resp.status_code}")
                     continue
 
                 root = ET.fromstring(resp.content)
@@ -660,8 +770,8 @@ class NewsEngine:
                         pub_date_el.text if pub_date_el is not None else None
                     )
 
-                    # Only recent headlines (last 24h)
+                    # Only recent headlines (configurable: headline_max_age_seconds)
-                    if pub_date and (datetime.now(timezone.utc) - pub_date).total_seconds() > 86400:
+                    if pub_date and (datetime.now(timezone.utc) - pub_date).total_seconds() > self.config.headline_max_age_seconds:
                         continue
 
                     # Dedup within and across feeds so a syndicated story
@@ -676,6 +786,21 @@ class NewsEngine:
                     currencies = self._extract_currencies(title)
 
                     if currencies:
+                        link_el = item.find("link")
+                        desc_el = item.find("description")
+                        src_el = item.find("source")
+                        # itertext() survives CDATA and stray child
+                        # elements where .text stops at the first child.
+                        desc = _clean_html_text(
+                            "".join(desc_el.itertext())
+                            if desc_el is not None else None)
+                        # Google News descriptions are often just the
+                        # headline again inside an anchor — worthless.
+                        if desc and desc.strip() == title.strip():
+                            desc = None
+                        publisher = None
+                        if src_el is not None:
+                            publisher = (src_el.text or "").strip() or None
                         headlines.append(NewsSentiment(
                             headline=title,
                             source=feed_url,
@@ -687,11 +812,27 @@ class NewsEngine:
                                 cur: self._analyze_headline_sentiment(title, cur)
                                 for cur in currencies
                             },
+                            link=_safe_link(
+                                link_el.text if link_el is not None else None),
+                            description=desc,
+                            publisher=publisher,
                         ))
 
+                ok_feeds += 1
+                self._record_source(
+                    f"rss:{host}", True,
+                    extra=f"{len(headlines) - kept_before} kept")
             except Exception as e:
+                self._record_source(
+                    f"rss:{host}", False, f"{type(e).__name__}: {e}")
                 logger.debug(f"Failed to fetch RSS from {feed_url}: {e}")
 
+        self._record_source(
+            "headlines", ok_feeds > 0,
+            detail="all feeds failed" if ok_feeds == 0 else "",
+            extra=(f"{len(headlines)} headlines from "
+                   f"{ok_feeds}/{len(self.config.rss_feeds)} feeds"))
+
         try:
             self._apply_llm_sentiment(headlines)
         except Exception as e:
@@ -802,7 +943,13 @@ class NewsEngine:
     def _llm_event_direction(self, event: EconomicEvent) -> Optional[str]:
         """Gold direction for a released surprise, e.g. "bearish: strong
         jobs lift yields". None → caller keeps the numeric description.
-        Cached per release; failures are NOT cached (retry next scan)."""
+        Cached per release; failures are NOT cached (retry next scan).
+
+        The actual-vs-forecast comparison is precomputed and sent as
+        "vs_forecast": the 2026-08-06 validation found the model reads
+        close K/M-suffixed pairs backwards ~11% of the time (e.g. claims
+        202K vs 217K called "higher than forecast") when left to compare
+        the raw numbers itself."""
         key = (event.event, event.currency, str(event.actual or ""))
         with NewsEngine._cache_lock:
             if key in NewsEngine._llm_event_cache:
@@ -810,12 +957,18 @@ class NewsEngine:
         client = get_llm_client()
         if not client.available:
             return None
+        vs_forecast = {
+            "positive": "above forecast",
+            "negative": "below forecast",
+            "inline": "in line with forecast",
+        }.get(event.surprise or "", "unknown")
         result = client.chat_json(_LLM_EVENT_SYSTEM, {
             "event": event.event,
             "currency": event.currency,
             "actual": event.actual,
             "forecast": event.forecast,
             "previous": event.previous,
+            "vs_forecast": vs_forecast,
         })
         if not result:
             return None

oanda_daemon.py

@@ -90,8 +90,17 @@ STATE_FILE = os.path.join(
 def _oanda_setting_int(env_key: str, json_key: str, default: int) -> int:
     env = os.environ.get(env_key)
     if env:
-        return int(env)
+        try:
-    return int(get_app_config().oanda_settings.get(json_key, default))
+            return int(env)
+        except ValueError:
+            raise SystemExit(
+                f"{env_key}={env!r} is not an integer — refusing to start")
+    try:
+        return int(get_app_config().oanda_settings.get(json_key, default))
+    except (TypeError, ValueError):
+        raise SystemExit(
+            f"config.json oanda.{json_key} is not an integer — "
+            f"refusing to start")
 
 
 def _oanda_setting_str(env_key: str, json_key: str, default: str) -> str:
@@ -130,8 +139,11 @@ def _seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
     elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
     interval_seconds = interval_minutes * 60
     remaining = interval_seconds - elapsed_seconds + buffer
-    if remaining <= buffer:
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
-        remaining += interval_seconds
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
     return remaining
 
 # ── Trade state persistence ───────────────────────────────────────────
@@ -165,7 +177,13 @@ class TradingDaemon:
         base_url = oanda_base_url()
         lot_size = _oanda_setting_int("OANDA_LOT_SIZE", "lot_size", 1)
         self._quality_gate = _oanda_setting_str(
-            "OANDA_QUALITY_GATE", "quality_gate", "C")
+            "OANDA_QUALITY_GATE", "quality_gate", "C").strip()
+        if self._quality_gate not in QUALITY_RANK:
+            # Fail CLOSED: an unrecognized gate must never silently fall
+            # back to the most permissive rank (QUALITY_RANK.get default).
+            raise SystemExit(
+                f"OANDA_QUALITY_GATE/quality_gate {self._quality_gate!r} "
+                f"is not one of {sorted(QUALITY_RANK)} — refusing to start")
         self._max_hold_bars = _oanda_setting_int(
             "OANDA_MAX_HOLD_BARS", "max_hold_bars", 300)
         self._price_precision = PAIR_INFO[SYMBOL]["price_precision"]
@@ -275,60 +293,86 @@ class TradingDaemon:
     # ── Position discovery (restart recovery) ──────────────────────
 
     def _discover_existing_positions(self) -> None:
+        self._reconcile_positions("boot discovery")
+
+    def _reconcile_positions(self, reason: str) -> None:
+        """Re-sync tracked state with the broker's open trades. Safe to
+        call mid-run (lock-guarded); used at boot and whenever a fill or
+        placement error suggests state and broker may have diverged."""
         try:
             open_trades = self._executor.get_open_trades(INSTRUMENT)
         except OandaAPIError:
-            logger.exception("Failed to query open trades at boot")
+            logger.exception(
+                "Failed to query open trades (%s) — reconcile skipped",
+                reason)
             return
 
-        saved = self._state.get("trades", {})
         live_ids = {t["id"] for t in open_trades}
 
-        stale = [tid for tid in saved if tid not in live_ids]
+        with self._state_lock:
-        for tid in stale:
+            saved = self._state.get("trades", {})
-            logger.info(
-                "Stale trade %s in state file (closed on Oanda), removing", tid)
-            del saved[tid]
 
-        for trade in open_trades:
+            stale = [tid for tid in saved if tid not in live_ids]
-            tid = trade["id"]
+            for tid in stale:
-            if tid in saved:
                 logger.info(
-                    "Recovered trade %s from state: %s %s @ %.2f, SL %.2f",
+                    "Stale trade %s in state file (closed on Oanda), "
-                    tid, saved[tid].get("direction"), INSTRUMENT,
+                    "removing", tid)
-                    saved[tid].get("entry"), saved[tid].get("current_sl"))
+                del saved[tid]
-                continue
 
-            direction = "BUY" if float(trade.get("currentUnits", 0)) > 0 else "SELL"
+            for trade in open_trades:
-            entry = float(trade.get("price", 0))
+                tid = trade["id"]
-            sl_order = trade.get("stopLossOrder", {})
+                if tid in saved:
-            sl = float(sl_order.get("price", entry))
+                    logger.info(
-            risk = abs(entry - sl) if sl else 1.0
+                        "Recovered trade %s from state: %s %s @ %.2f, SL %.2f",
+                        tid, saved[tid].get("direction"), INSTRUMENT,
+                        saved[tid].get("entry"), saved[tid].get("current_sl"))
+                    continue
 
-            saved[tid] = {
+                direction = ("BUY" if float(trade.get("currentUnits", 0)) > 0
-                "tf": "H1",
+                             else "SELL")
-                "direction": direction,
+                entry = float(trade.get("price", 0))
-                "entry": entry,
+                sl_order = trade.get("stopLossOrder") or {}
-                "sl_initial": sl,
+                sl_price = sl_order.get("price")
-                "current_sl": sl,
+                if sl_price is None:
-                "risk": risk,
+                    # No broker-side stop at all: risk is undefined, so the
-                "peak": entry,
+                    # trailing model cannot manage this trade. Scream — a
-                "trail_armed": False,
+                    # naked position needs a human (or a policy) to set a
-                "trail_arm_r": 0.6,
+                    # stop; fabricating one here would invent trading
-                "trail_dist_r": 0.75,
+                    # behavior the books never validated.
-                "bars_held": 0,
+                    logger.critical(
-                "last_bar_ts": None,
+                        "Adopted trade %s (%s @ %.2f) has NO STOP LOSS on "
-                "signal_bar_ts": None,
+                        "the broker — it will NOT be trailed. Set a stop "
-            }
+                        "manually or close it.", tid, direction, entry)
-            logger.info(
+                    sl = entry
-                "Adopted unknown trade %s: %s @ %.2f, SL %.2f, risk %.2f "
+                    risk = 0.0
-                "(conservative defaults)",
+                else:
-                tid, direction, entry, sl, risk)
+                    sl = float(sl_price)
+                    risk = abs(entry - sl)
 
-        self._state["trades"] = saved
+                saved[tid] = {
-        _save_state(self._state)
+                    "tf": "H1",
+                    "direction": direction,
+                    "entry": entry,
+                    "sl_initial": sl,
+                    "current_sl": sl,
+                    "risk": risk,
+                    "peak": entry,
+                    "trail_armed": False,
+                    "trail_arm_r": 0.6,
+                    "trail_dist_r": 0.75,
+                    "bars_held": 0,
+                    "last_bar_ts": None,
+                    "signal_bar_ts": None,
+                }
+                logger.info(
+                    "Adopted unknown trade %s: %s @ %.2f, SL %.2f, risk %.2f "
+                    "(conservative defaults)",
+                    tid, direction, entry, sl, risk)
+
+            self._state["trades"] = saved
+            _save_state(self._state)
         logger.info(
-            "Position discovery: %d active trade(s)", len(saved))
+            "Position reconcile (%s): %d active trade(s)", reason, len(saved))
 
     # ── Candle-boundary loop ───────────────────────────────────────
 
@@ -521,9 +565,15 @@ class TradingDaemon:
                     stop_loss=ts.stop_loss,
                 )
             except OandaAPIError:
+                # The order is never retried (a timeout after a server-side
+                # fill would double-up on retry), but the server MAY have
+                # filled before the error reached us — reconcile so a live
+                # position can't stay untracked until the next restart.
                 logger.exception(
-                    "Failed to place %s order for %s %s",
+                    "Failed to place %s order for %s %s — reconciling "
+                    "broker positions",
                     ts.direction, SYMBOL, tf.value)
+                self._reconcile_positions("order placement error")
                 return
 
             fill_tx = result.get("orderFillTransaction")
@@ -535,7 +585,16 @@ class TradingDaemon:
             trade_opened = fill_tx.get("tradeOpened", {})
             trade_id = trade_opened.get("tradeID")
             if not trade_id:
-                logger.warning("No trade ID in fill response")
+                # A fill with no tradeOpened means the order interacted
+                # with an existing broker position (tradesClosed /
+                # tradeReduced on a netting account) — state and broker
+                # have diverged. Surface it and re-sync instead of
+                # silently forgetting a real fill.
+                logger.critical(
+                    "Order FILLED but opened no trade (netting close/"
+                    "reduce?) — fill: closed=%s reduced=%s. Reconciling.",
+                    fill_tx.get("tradesClosed"), fill_tx.get("tradeReduced"))
+                self._reconcile_positions("fill without tradeOpened")
                 return
 
             fill_price = float(fill_tx.get("price", ts.entry_price))
@@ -587,8 +646,12 @@ class TradingDaemon:
     # ── Trailing stop management ───────────────────────────────────
 
     def _manage_trailing(self, tf: Timeframe) -> None:
+        # Copy the per-trade dicts too: the loop mutates info (peak,
+        # bars_held, last_bar_ts) and must not touch live state outside
+        # the lock — another thread may be serializing it in _save_state.
         with self._state_lock:
-            trades = dict(self._state.get("trades", {}))
+            trades = {tid: dict(info)
+                      for tid, info in self._state.get("trades", {}).items()}
 
         for trade_id, info in trades.items():
             if info.get("tf") != tf.value:
@@ -634,13 +697,16 @@ class TradingDaemon:
                     self._executor.close_trade(trade_id)
                 except OandaAPIError:
                     # Keep tracking — the trade is still live on the
-                    # broker; we retry the close at the next bar.
+                    # broker (incl. a 200 whose close order was cancelled,
+                    # e.g. MARKET_HALTED — close_trade raises on no-fill);
+                    # we retry the close at the next bar.
                     logger.exception(
                         "Failed to close timed-out trade %s — will retry",
                         trade_id)
                     with self._state_lock:
-                        self._state["trades"][trade_id] = info
+                        if trade_id in self._state["trades"]:
-                        _save_state(self._state)
+                            self._state["trades"][trade_id] = info
+                            _save_state(self._state)
                     continue
                 with self._state_lock:
                     self._state["trades"].pop(trade_id, None)
@@ -651,6 +717,14 @@ class TradingDaemon:
             bar_high = float(last_bar["High"])
             bar_low = float(last_bar["Low"])
 
+            if info.get("risk", 0) <= 0:
+                # Adopted naked position (no broker SL → risk undefined):
+                # never manageable by the trailing model — keep it loud
+                # instead of silently skipping forever.
+                logger.warning(
+                    "Trade %s has risk<=0 (no stop loss) — trailing "
+                    "skipped; set a stop manually", trade_id)
+
             new_sl = self._compute_trail(info, bar_high, bar_low)
 
             if new_sl is not None:
@@ -667,8 +741,13 @@ class TradingDaemon:
                             trade_id)
 
             with self._state_lock:
-                self._state["trades"][trade_id] = info
+                # Write back ONLY if the trade is still tracked — another
-                _save_state(self._state)
+                # loop's _check_closed_trades may have popped it while we
+                # iterated (broker-side close). Re-inserting would create
+                # a ghost that blocks new entries.
+                if trade_id in self._state["trades"]:
+                    self._state["trades"][trade_id] = info
+                    _save_state(self._state)
 
     def _compute_trail(
         self, info: Dict, bar_high: float, bar_low: float

oanda_executor.py

@@ -69,6 +69,13 @@ class OandaExecutor:
                 "instrument": instrument,
                 "units": signed_units,
                 "timeInForce": "FOK",
+                # The book model is one position total; on a netting
+                # account an opposite-side order would silently CLOSE an
+                # existing (possibly untracked) position instead of opening
+                # the signal trade. OPEN_ONLY makes that case fail loudly
+                # (order cancelled) so the divergence surfaces instead of
+                # executing an unintended close.
+                "positionFill": "OPEN_ONLY",
                 "stopLossOnFill": {
                     "price": self._fmt_price(stop_loss),
                 },
@@ -156,7 +163,18 @@ class OandaExecutor:
                f"/trades/{trade_id}/close")
         resp = self._request_with_retry("PUT", url, json={"units": "ALL"})
         data = self._parse_json(resp)
-        close_tx = data.get("orderFillTransaction", {})
+        close_tx = data.get("orderFillTransaction")
+        if not close_tx:
+            # A 200 whose close order was created but NOT filled (e.g.
+            # cancelled with MARKET_HALTED during the settlement pause)
+            # means the position is STILL OPEN. Raising keeps the caller
+            # tracking it and retrying — silently returning here would
+            # orphan a live position.
+            reason = data.get("orderCancelTransaction", {}).get(
+                "reason", "no orderFillTransaction in response")
+            raise OandaAPIError(
+                f"Close of trade {trade_id} did not fill: {reason}",
+                resp.status_code, str(data)[:300])
         close_price = float(close_tx.get("price", 0))
         pl = float(close_tx.get("pl", 0))
         logger.info(

pair_signals/xauusd.py

@@ -197,6 +197,13 @@ def make_config(
         config.xau_strategy.blocked_hours_utc = [
             h for h in config.xau_strategy.blocked_hours_utc if h != 21
         ] + [9]
+        # Deliberately NOT mirrored into config.backtest.blocked_hours_utc:
+        # that copy is the generic non-XAU fallback carrying the JSON base
+        # only. Propagating a gold-validated per-TF delta there would leak
+        # XAU-calibrated hours to a future pair (pair-isolation rule).
+        # Every XAU consumer branches to the xau_strategy copy. Enforced
+        # by the Module 13 test "M5 factory appends hour 9 on top of a
+        # JSON base".
         # BUY-only chart patterns on M5 — see module docstring.
         config.xau_strategy.cp_enabled = True
         config.xau_strategy.cp_patterns = ["inv_hs", "double_bottom", "bull_flag"]

providers/base.py

@@ -24,6 +24,7 @@ import time
 from abc import ABC, abstractmethod
 from datetime import datetime, timedelta
 from typing import AsyncIterator, Callable, Dict, Optional, Sequence, Tuple
+from zoneinfo import ZoneInfo
 
 import pandas as pd
 
@@ -31,19 +32,36 @@ from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
 
 logger = logging.getLogger(__name__)
 
+# The gold/FX market week is defined in New York time (see
+# _is_forex_weekend); a fixed UTC hour is wrong for half the year.
+_NY_TZ = ZoneInfo("America/New_York")
+
 
 def _is_forex_weekend(now=None) -> bool:
-    """Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC."""
+    """Gold/FX market closed for the weekend.
+
+    The market week is anchored to NEW YORK time — Friday 17:00 ET →
+    Sunday 18:00 ET — not a fixed UTC hour. During US DST the feed
+    actually stops at 21:00 UTC (measured on 5 years of Oanda bars:
+    159/162 summer Fridays; winter close 22:00, winter reopen Sunday
+    23:00 UTC). The old hardcoded Fri 22:00→Sun 22:00 UTC rule produced
+    two false-"STALE DATA/feed lagging" hours every week (2026-08-04
+    audit I43, fixed per owner decision).
+
+    Kept as three deliberate copies (here, signal_aggregator.py,
+    gpu_backtest._is_xau_blocked_by_calendar) — a Module 21 test asserts
+    they agree hour-by-hour across DST transitions.
+    """
     if now is None:
         now = pd.Timestamp.now(tz="UTC")
-    wd = now.weekday()
+    ny = now.astimezone(_NY_TZ)
-    hour = now.hour
+    wd = ny.weekday()
-    if wd == 4 and hour >= 22:
+    if wd == 4:                      # Friday
-        return True
+        return ny.hour >= 17
-    if wd == 5:
+    if wd == 5:                      # Saturday
-        return True
-    if wd == 6 and hour < 22:
         return True
+    if wd == 6:                      # Sunday
+        return ny.hour < 18
     return False
 
 
@@ -86,9 +104,15 @@ class BaseDataProvider(ABC):
         self.config = config
         self._cache: Dict[str, pd.DataFrame] = {}
         self._cache_timestamps: Dict[str, float] = {}
+        # Requested fetch depth per key: a cached short frame must not
+        # satisfy a caller asking for a longer lookback.
+        self._cache_outputsize: Dict[str, int] = {}
         # The bot shares one provider across a thread pool; cache eviction
         # is check-then-act and needs locking.
         self._cache_lock = threading.Lock()
+        # Single-flight: one API fetch per (symbol, TF) at a time; other
+        # threads wait and then read the cache.
+        self._inflight: Dict[str, threading.Event] = {}
 
     # ── Abstract: the only provider-specific piece ─────────────────
 
@@ -119,68 +143,97 @@ class BaseDataProvider(ABC):
             lookback_days = self.config.data_lookback_days
 
         cache_key = f"{symbol}_{timeframe.value}"
-        now = time.time()
-
-        cache_ttl = self.CANDLE_SECONDS.get(timeframe, self.config.refresh_interval)
-        with self._cache_lock:
-            cached = self._cache.get(cache_key)
-            cached_at = self._cache_timestamps.get(cache_key)
-        if (
-            cached is not None
-            and cached_at is not None
-            and (now - cached_at) < cache_ttl
-            and not cached.empty
-        ):
-            if (self._is_data_fresh(cached, symbol, timeframe)
-                    and self._has_latest_closed_bar(cached, timeframe)):
-                return cached
-            logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
-            with self._cache_lock:
-                self._cache.pop(cache_key, None)
-                self._cache_timestamps.pop(cache_key, None)
-
         outputsize = max(
             300,
             min(int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20,
                 self.MAX_BARS_PER_REQUEST),
         )
-        df = self._fetch_raw(symbol, timeframe, outputsize)
-        df = self._drop_forming_candle(df, timeframe)
-        df = self._validate_and_clean(df, symbol, timeframe)
 
-        # The closed-bar guarantee must hold for FRESH responses too, not
+        while True:
-        # just cache hits: the scanner fires seconds after the boundary,
+            now = time.time()
-        # and a lagging feed can answer without the just-closed bar — the
+            cache_ttl = self.CANDLE_SECONDS.get(
-        # scan would silently analyze (and session-check) the previous
+                timeframe, self.config.refresh_interval)
-        # bar. Give the feed one short grace retry, then proceed loudly.
+            with self._cache_lock:
-        if not df.empty and not self._has_latest_closed_bar(df, timeframe):
+                cached = self._cache.get(cache_key)
-            time.sleep(self.FRESH_FETCH_RETRY_SECONDS)
+                cached_at = self._cache_timestamps.get(cache_key)
-            try:
+                cached_size = self._cache_outputsize.get(cache_key, 0)
-                retry = self._fetch_raw(symbol, timeframe, outputsize)
+            if (
-                retry = self._drop_forming_candle(retry, timeframe)
+                cached is not None
-                retry = self._validate_and_clean(retry, symbol, timeframe)
+                and cached_at is not None
-                if retry is not None and not retry.empty:
+                and (now - cached_at) < cache_ttl
-                    df = retry
+                and not cached.empty
-            except DataIntegrityError as e:
+                # A caller asking for MORE history than the cached fetch
+                # requested must not be served the short frame silently.
+                and cached_size >= outputsize
+            ):
+                if (self._is_data_fresh(cached, symbol, timeframe)
+                        and self._has_latest_closed_bar(cached, timeframe)):
+                    return cached
+                logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
+                with self._cache_lock:
+                    self._cache.pop(cache_key, None)
+                    self._cache_timestamps.pop(cache_key, None)
+                    self._cache_outputsize.pop(cache_key, None)
+
+            # Single-flight: concurrent scans for the same (symbol, TF)
+            # must not each hit the API (the four TF loops fire together
+            # at aligned boundaries and Oanda has no client throttle).
+            # First caller claims the key and fetches; the rest wait,
+            # then re-check the cache.
+            with self._cache_lock:
+                inflight = self._inflight.get(cache_key)
+                if inflight is None:
+                    self._inflight[cache_key] = threading.Event()
+                    break
+            inflight.wait(timeout=90)
+
+        try:
+            df = self._fetch_raw(symbol, timeframe, outputsize)
+            df = self._drop_forming_candle(df, timeframe)
+            df = self._validate_and_clean(df, symbol, timeframe)
+
+            # The closed-bar guarantee must hold for FRESH responses too,
+            # not just cache hits: the scanner fires seconds after the
+            # boundary, and a lagging feed can answer without the
+            # just-closed bar — the scan would silently analyze (and
+            # session-check) the previous bar. Give the feed one short
+            # grace retry, then proceed loudly.
+            if not df.empty and not self._has_latest_closed_bar(df, timeframe):
+                time.sleep(self.FRESH_FETCH_RETRY_SECONDS)
+                try:
+                    retry = self._fetch_raw(symbol, timeframe, outputsize)
+                    retry = self._drop_forming_candle(retry, timeframe)
+                    retry = self._validate_and_clean(retry, symbol, timeframe)
+                    if retry is not None and not retry.empty:
+                        df = retry
+                except DataIntegrityError as e:
+                    logger.warning(
+                        f"{symbol} {timeframe.value}: boundary retry refetch "
+                        f"failed ({e}) — keeping original data")
+                if not self._has_latest_closed_bar(df, timeframe):
+                    logger.warning(
+                        f"{symbol} {timeframe.value}: feed lagging — just-closed bar "
+                        f"still missing after retry; this scan may run one bar behind"
+                    )
+
+            if not self._is_data_fresh(df, symbol, timeframe):
                 logger.warning(
-                    f"{symbol} {timeframe.value}: boundary retry refetch "
+                    f"STALE DATA from API: {symbol} {timeframe.value} — "
-                    f"failed ({e}) — keeping original data")
+                    f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
-            if not self._has_latest_closed_bar(df, timeframe):
-                logger.warning(
-                    f"{symbol} {timeframe.value}: feed lagging — just-closed bar "
-                    f"still missing after retry; this scan may run one bar behind"
                 )
 
-        if not self._is_data_fresh(df, symbol, timeframe):
+            with self._cache_lock:
-            logger.warning(
+                self._cache[cache_key] = df
-                f"STALE DATA from API: {symbol} {timeframe.value} — "
+                self._cache_timestamps[cache_key] = now
-                f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
+                self._cache_outputsize[cache_key] = outputsize
-            )
+            return df
-
+        finally:
-        with self._cache_lock:
+            # Release waiters even when the fetch raised — they re-check
-            self._cache[cache_key] = df
+            # the cache and (on a miss) claim the key themselves.
-            self._cache_timestamps[cache_key] = now
+            with self._cache_lock:
-        return df
+                ev = self._inflight.pop(cache_key, None)
+            if ev is not None:
+                ev.set()
 
     def fetch_range(
         self,
@@ -199,6 +252,7 @@ class BaseDataProvider(ABC):
         chunks = []
         cursor_end = end
         empty_streak = 0
+        error_streak = 0
         while cursor_end > start:
             cursor_start = max(start, cursor_end - timedelta(days=days_per_chunk))
             try:
@@ -207,19 +261,27 @@ class BaseDataProvider(ABC):
                     start=cursor_start, end=cursor_end,
                 )
             except Exception as e:
-                # Providers raise for windows with no data (weekends,
+                # A raising chunk is a PROVIDER ERROR (auth, 429, 5xx,
-                # holidays, asset gaps). Skip the window, step back, retry
+                # malformed payload) — not "no data". Three in a row means
-                # — but bail after a few consecutive empties.
+                # the rest of the history is unreachable: raise instead of
+                # returning a silently truncated frame (the backtester
+                # would treat it as complete).
                 logger.warning(
                     f"Chunk {cursor_start.date()}–{cursor_end.date()} failed: {e}"
                 )
-                empty_streak += 1
+                error_streak += 1
-                if empty_streak >= 3:
+                if error_streak >= 3:
-                    break
+                    raise self.DATA_ERROR(
+                        f"{symbol} {timeframe.value}: {error_streak} "
+                        f"consecutive chunk failures walking back from "
+                        f"{cursor_end.date()} — refusing to return "
+                        f"truncated history (last error: {e})") from e
                 cursor_end = cursor_start - timedelta(seconds=1)
                 continue
 
             if chunk is None or chunk.empty:
+                # Genuinely empty windows (weekends/holidays/asset gaps)
+                # are normal; several in a row = start of history.
                 empty_streak += 1
                 if empty_streak >= 3:
                     break
@@ -227,9 +289,18 @@ class BaseDataProvider(ABC):
                 continue
 
             empty_streak = 0
+            error_streak = 0
             chunks.append(chunk)
             # Step back; dedup on concat.
-            cursor_end = chunk.index[0] - timedelta(seconds=1)
+            new_cursor = chunk.index[0] - timedelta(seconds=1)
+            if new_cursor >= cursor_end:
+                # A provider ignoring start/end would pin the cursor and
+                # loop forever — refuse to spin.
+                logger.warning(
+                    f"{symbol} {timeframe.value}: pagination made no "
+                    f"progress at {cursor_end} — stopping")
+                break
+            cursor_end = new_cursor
             if len(chunk) < 2:
                 break
 
@@ -243,7 +314,17 @@ class BaseDataProvider(ABC):
         df = self._drop_forming_candle(df, timeframe)
         start_ts = pd.Timestamp(start)
         end_ts = pd.Timestamp(end)
-        return df[(df.index >= start_ts) & (df.index <= end_ts)]
+        df = df[(df.index >= start_ts) & (df.index <= end_ts)]
+        # Coverage sanity: a head materially later than the requested
+        # start means history was truncated (empty-streak bail or provider
+        # depth limit). Loud, not fatal — some assets genuinely start late.
+        if not df.empty and (df.index[0] - start_ts) > pd.Timedelta(days=7):
+            logger.error(
+                f"{symbol} {timeframe.value}: fetched history starts "
+                f"{df.index[0]} but {start_ts} was requested — "
+                f"~{(df.index[0] - start_ts).days} days missing from the "
+                f"head; treat downstream results accordingly")
+        return df
 
     def is_data_fresh(
         self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
@@ -323,8 +404,19 @@ class BaseDataProvider(ABC):
             return df
         candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
         now = pd.Timestamp.now(tz="UTC")
-        if df.index[-1] + pd.Timedelta(seconds=candle_seconds) > now:
+        # Loop, not a single slice: a misbehaving feed can return MORE
+        # than one not-yet-closed bar, and a single iloc[:-1] would let
+        # the residual forming bar through cleaning (future-tolerance is
+        # +5 min) straight into the signal engine.
+        dropped = 0
+        while (not df.empty
+               and df.index[-1] + pd.Timedelta(seconds=candle_seconds) > now):
             df = df.iloc[:-1]
+            dropped += 1
+        if dropped > 1:
+            logger.warning(
+                "%s: feed returned %d forming/future bars — all dropped",
+                timeframe.value, dropped)
         return df
 
     def _is_data_fresh(self, df: pd.DataFrame, symbol: str, timeframe: Timeframe) -> bool:
@@ -372,6 +464,20 @@ class BaseDataProvider(ABC):
             logger.warning(f"{symbol}: Dropping {future_mask.sum()} future timestamps")
             df = df[~future_mask]
 
+        # Non-positive prices are corrupt, never real gold quotes. Reject
+        # the row BEFORE the consistency repair below — repairing High/Low
+        # from a corrupt 0.0 Open/Close would propagate the zero into
+        # previously-valid fields (2026-08-04 audit, I45).
+        nonpositive = (df[["Open", "High", "Low", "Close"]] <= 0).any(axis=1)
+        if nonpositive.any():
+            logger.warning(
+                f"{symbol}: Dropping {nonpositive.sum()} bars with "
+                f"non-positive prices (corrupt feed data)")
+            df = df[~nonpositive]
+            if df.empty:
+                raise self.DATA_ERROR(
+                    f"{symbol}: all bars had non-positive prices")
+
         invalid_ohlc = (
             (df["High"] < df["Low"]) |
             (df["High"] < df["Open"]) |
@@ -386,7 +492,10 @@ class BaseDataProvider(ABC):
                 df.loc[idx, "High"] = max(vals + [df.loc[idx, "High"]])
                 df.loc[idx, "Low"]  = min(vals + [df.loc[idx, "Low"]])
 
+        # inf returns (division by a zero/na Close) would NaN the std and
+        # silently disarm the whole 10σ pass — neutralize them first.
         returns = df["Close"].pct_change()
+        returns = returns.replace([float("inf"), float("-inf")], float("nan"))
         std_ret = returns.std()
         if std_ret > 0:
             dev = returns - returns.mean()

providers/oanda.py

@@ -8,6 +8,7 @@ Oanda's candle JSON, and exposes the real bid/ask spread from MBA candles.
 """
 
 import logging
+import time
 from datetime import datetime
 from typing import Dict, Optional
 
@@ -31,6 +32,7 @@ class OandaDataProvider(BaseDataProvider):
 
     name = "oanda"
     DATA_ERROR = OandaDataError
+    RETRY_DELAY_S = 2.0
 
     OANDA_GRANULARITY = {
         Timeframe.M1: "M1",
@@ -124,11 +126,35 @@ class OandaDataProvider(BaseDataProvider):
         else:
             params["count"] = min(count, self.MAX_BARS_PER_REQUEST)
 
-        try:
+        # One retry absorbs Oanda's sporadic spurious 401s (~15-20/day on
-            resp = self._session.get(url, params=params, timeout=30)
+        # the practice API — the same token succeeds on adjacent requests)
-        except requests.RequestException as e:
+        # plus transient 5xx/network faults. A real credential problem
-            raise OandaDataError(
+        # fails both attempts and raises exactly as before. Other 4xx
-                f"Oanda candle fetch failed for {instrument}: {e}") from e
+        # (bad request, 429) are never retried.
+        last_err: Optional[OandaDataError] = None
+        for attempt in range(2):
+            if attempt:
+                time.sleep(self.RETRY_DELAY_S)
+            try:
+                resp = self._session.get(url, params=params, timeout=30)
+            except requests.RequestException as e:
+                last_err = OandaDataError(
+                    f"Oanda candle fetch failed for {instrument}: {e}")
+                last_err.__cause__ = e
+                continue
+            if resp.status_code == 401 or resp.status_code >= 500:
+                last_err = OandaDataError(
+                    f"Oanda candle fetch failed for {instrument}: "
+                    f"HTTP {resp.status_code}: {resp.text[:300]}")
+                if attempt == 0:
+                    logger.warning(
+                        f"Oanda fetch for {instrument} got HTTP "
+                        f"{resp.status_code}; retrying in "
+                        f"{self.RETRY_DELAY_S}s")
+                continue
+            break
+        else:
+            raise last_err
 
         if resp.status_code != 200:
             raise OandaDataError(
@@ -149,23 +175,37 @@ class OandaDataProvider(BaseDataProvider):
         rows = []
         last_bid_close = None
         last_ask_close = None
+        skipped_malformed = 0
         for c in candles:
             if not c.get("complete", False):
                 continue
             mid = c.get("mid", {})
             bid = c.get("bid", {})
             ask = c.get("ask", {})
+            # A candle missing any OHLC field must be SKIPPED, never
+            # defaulted to 0.0: a zero price flows through cleaning as a
+            # "legit" bar and poisons ATR/Keltner/SL math — and a
+            # mid-frame zero Close makes pct_change() emit inf, which
+            # NaNs the std and disarms the 10σ outlier filter for the
+            # whole frame (2026-08-04 audit, I45).
+            if any(mid.get(k) is None for k in ("o", "h", "l", "c")):
+                skipped_malformed += 1
+                continue
             rows.append({
                 "Datetime": pd.Timestamp(c["time"]).tz_convert("UTC"),
-                "Open": float(mid.get("o", 0)),
+                "Open": float(mid["o"]),
-                "High": float(mid.get("h", 0)),
+                "High": float(mid["h"]),
-                "Low": float(mid.get("l", 0)),
+                "Low": float(mid["l"]),
-                "Close": float(mid.get("c", 0)),
+                "Close": float(mid["c"]),
                 "Volume": int(c.get("volume", 0)),
             })
             if bid and ask:
                 last_bid_close = float(bid.get("c", 0))
                 last_ask_close = float(ask.get("c", 0))
+        if skipped_malformed:
+            logger.warning(
+                "Oanda returned %d malformed candle(s) for %s (missing "
+                "OHLC fields) — skipped", skipped_malformed, instrument)
 
         if not rows:
             raise OandaDataError(

providers/twelve_data.py

@@ -210,6 +210,12 @@ class TwelveDataProvider(BaseDataProvider):
         The TD WS speaks display symbols ("XAU/USD"); ticks are mapped back
         to the canonical pair symbols passed in. The caller owns the
         reconnect/backoff loop around this generator.
+
+        `on_connect` fires on the FIRST RECEIVED frame, not after the
+        subscribe send (2026-08-04 audit H38b, mirroring the Tiingo
+        provider) — firing pre-receive let a rejected API key reset the
+        caller's reconnect backoff on every attempt, pinning it at the
+        minimum forever.
         """
         import websockets
 
@@ -222,11 +228,14 @@ class TwelveDataProvider(BaseDataProvider):
             }
             await ws.send(json.dumps(subscribe))
             logger.info(f"WebSocket connected, subscribed to {len(ws_to_pair)} symbols")
-            if on_connect is not None:
-                on_connect()
             hb_task = asyncio.create_task(self._ws_heartbeat(ws))
+            connected = False
             try:
                 async for raw in ws:
+                    if not connected:
+                        connected = True
+                        if on_connect is not None:
+                            on_connect()
                     parsed = self._parse_ws_price(raw)
                     if parsed is None:
                         continue

signal_aggregator.py

@@ -10,9 +10,12 @@ import os
 from dataclasses import dataclass, field
 from datetime import datetime, timezone
 from typing import Dict, List, Optional
+from zoneinfo import ZoneInfo
 
 import pandas as pd
 
+_NY_TZ = ZoneInfo("America/New_York")
+
 from chart_patterns import detect_chart_pattern
 from config import PlatformConfig, SignalStrength, Timeframe
 from providers import BaseDataProvider, make_data_provider
@@ -23,15 +26,19 @@ logger = logging.getLogger(__name__)
 
 
 def _is_forex_weekend(now_utc: datetime) -> bool:
-    """Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC (Sydney open)."""
+    """Gold/FX market closed for the weekend: Friday 17:00 → Sunday 18:00
-    wd = now_utc.weekday()
+    NEW YORK time (= Fri 21:00 UTC in US summer, 22:00 in winter —
-    hour = now_utc.hour
+    matches measured Oanda feed behavior; 2026-08-04 audit I43).
-    if wd == 4 and hour >= 22:  # Friday after 22:00
+    Deliberate copy of providers.base._is_forex_weekend — a Module 21
-        return True
+    test asserts the copies agree across DST transitions."""
-    if wd == 5:  # All Saturday
+    ny = now_utc.astimezone(_NY_TZ)
-        return True
+    wd = ny.weekday()
-    if wd == 6 and hour < 22:  # Sunday before 22:00
+    if wd == 4:                      # Friday
+        return ny.hour >= 17
+    if wd == 5:                      # Saturday
         return True
+    if wd == 6:                      # Sunday
+        return ny.hour < 18
     return False
 
 
@@ -81,6 +88,15 @@ class SignalReport:
     news_description: str = ""
     news_summary: str = ""  # LLM market-context blurb (display-only)
 
+    # LLM agent-panel consensus (advisory/DISPLAY-ONLY). Populated
+    # post-hoc by consumers via agent_panel.attach_panel — NEVER inside
+    # analyze_from_dataframes (backtest parity) and never read by any
+    # gate or scoring path.
+    panel_verdict: str = ""        # "AGREE" / "NEUTRAL" / "OPPOSE" / ""
+    panel_confidence: float = 0.0
+    panel_rationale: str = ""
+    panel_takes: List[Dict] = field(default_factory=list)
+
     final_signal: SignalStrength = SignalStrength.NEUTRAL
     final_score: float = 0.0
     final_confidence: float = 0.0
@@ -1253,7 +1269,17 @@ class SignalAggregator:
             ema200 = primary.key_levels.get("ema_trend")
             if ema200 and ema200 > 0 and report.current_price > 0:
                 ema_dist_pct = abs(report.current_price - ema200) / ema200 * 100.0
-                _lo = float(os.environ.get("FB2_EMA_LO", "0.5"))
+                # FB2_EMA_LO: research override for the EMA200-proximity
+                # penalty band; the tuned default is 0.5. An unparseable
+                # value must degrade to the default, never crash the scan
+                # (2026-08-04 audit; hardened per owner delegation).
+                try:
+                    _lo = float(os.environ.get("FB2_EMA_LO", "0.5"))
+                except (TypeError, ValueError):
+                    logger.warning(
+                        "FB2_EMA_LO=%r is not a number — using tuned "
+                        "default 0.5", os.environ.get("FB2_EMA_LO"))
+                    _lo = 0.5
                 if ema_dist_pct < _lo:
                     report.warnings.append(f"NEAR EMA200 ({ema_dist_pct:.2f}%) — chop risk")
                     penalties.append(("ema200_near", 5))

static/news_summary.php

@@ -0,0 +1,245 @@
+<?php
+/**
+ * On-demand gold-impact AI summary for one published headline.
+ *
+ * The ONLY dynamic piece of the otherwise-static dashboard. Serves
+ * GET ?id=<sha1-of-headline> and returns {"summary": ..., "generated_at": ...}.
+ *
+ * Abuse model (the page is public):
+ *   - WHITELIST: the id must match a headline in the local news.json —
+ *     arbitrary text can never reach the LLM through this endpoint.
+ *   - CACHE FOREVER: headlines are immutable, so each id costs at most
+ *     one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
+ *     the whitelist on purpose: summaries stay reachable after the
+ *     headline ages out of the 24 h feed window.
+ *   - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
+ *     errors (a perms mistake must not mean unmetered spend).
+ *
+ * DeepSeek doctrine (mirrors llm_client.py — deliberate local copy, the
+ * dashboard never imports bot code and this file never reads the repo's
+ * config.json):
+ *   - STATIC byte-identical system prompt (prefix-cache pricing).
+ *   - "thinking": disabled is MANDATORY for V4-Flash (else it burns the
+ *     timeout reasoning), temperature 0, json response_format.
+ *   - One attempt, no retries; every failure returns a JSON error the
+ *     page renders as a graceful fallback.
+ *
+ * Key file lives OUTSIDE the docroot: /etc/gold-dashboard/deepseek.key
+ * (root:apache 640). Override paths/caps via Apache SetEnv:
+ * DEEPSEEK_KEY_FILE, AI_SUMMARY_DAILY_CAP.
+ */
+
+const NEWS_FILE = __DIR__ . '/news.json';
+const CACHE_DIR = __DIR__ . '/ai';   // apache-owned; created at deploy time
+const API_URL   = 'https://api.deepseek.com/chat/completions';
+const MODEL     = 'deepseek-v4-flash';
+const TIMEOUT   = 12;
+const MAX_TOKENS = 200;
+const SUMMARY_MAX_CHARS = 400;
+
+// .ht_ prefix: Apache's default <Files ".ht*"> rule denies these even
+// though they sit in the public docroot.
+const CAP_FILE = CACHE_DIR . '/.ht_cap.json';
+
+const SYSTEM_PROMPT = <<<'PROMPT'
+You are a forex/gold news analyst. You receive one news item as JSON:
+{"headline", "description", "score", "xau_score", "keywords", "currencies"}.
+"xau_score" is the platform's automated gold-effect score for this item,
+-1.0 (gold down) to 1.0 (gold up); "score" is overall market sentiment.
+Both are automated and can be wrong — judge the news yourself.
+In 1-2 plain-English sentences (max 45 words) explain what this news
+means for the GOLD (XAU/USD) price and WHY it is bullish, bearish, or
+neutral. If your reading clearly contradicts xau_score, add "(the
+automated score disagrees)". Never write the words "xau_score" or
+"score" — a general reader sees this text.
+Gold doctrine: hawkish Fed / rate hikes / rising real yields / strong
+dollar are gold-bearish; easing / rate cuts / safe-haven flows /
+geopolitical stress / weak dollar are gold-bullish.
+Reply with ONLY JSON: {"summary": "..."}
+PROMPT;
+
+function respond(int $status, array $obj, string $cache = 'no-store'): void
+{
+    http_response_code($status);
+    header('Content-Type: application/json; charset=utf-8');
+    header('Cache-Control: ' . $cache);
+    echo json_encode($obj, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
+    exit;
+}
+
+function atomic_write(string $path, string $data): bool
+{
+    $tmp = $path . '.tmp.' . getmypid();
+    if (@file_put_contents($tmp, $data) === false) {
+        return false;
+    }
+    if (!@rename($tmp, $path)) {
+        @unlink($tmp);
+        return false;
+    }
+    return true;
+}
+
+// ── 1. id validation ────────────────────────────────────────────────
+$id = $_GET['id'] ?? '';
+if (!preg_match('/^[a-f0-9]{40}$/', $id)) {
+    respond(400, ['error' => 'bad_id']);
+}
+
+// ── 2. cache first (immutable headlines → summaries never change) ───
+$cachePath = CACHE_DIR . '/' . $id . '.json';
+$cached = @file_get_contents($cachePath);
+if ($cached !== false && $cached !== '') {
+    http_response_code(200);
+    header('Content-Type: application/json; charset=utf-8');
+    header('Cache-Control: max-age=3600');
+    echo $cached;
+    exit;
+}
+
+// ── 3. whitelist against the published feed ─────────────────────────
+$news = json_decode((string)@file_get_contents(NEWS_FILE), true);
+if (!is_array($news)) {
+    respond(503, ['error' => 'no_feed']);
+}
+$hit = null;
+foreach ($news['headlines'] ?? [] as $h) {
+    // Fallback sha1(): a publisher that predates the id field still
+    // serves a valid whitelist during the restart window.
+    $hid = $h['id'] ?? sha1($h['headline'] ?? '');
+    if ($hid === $id) {
+        $hit = $h;
+        break;
+    }
+}
+if ($hit === null) {
+    respond(404, ['error'   => 'unknown_id',
+                  'message' => 'Headline is no longer in the feed.']);
+}
+
+// ── 4. daily cap (fail closed) ──────────────────────────────────────
+$dailyCap = (int)(getenv('AI_SUMMARY_DAILY_CAP') ?: 150);
+$capFh = @fopen(CAP_FILE, 'c+');
+if ($capFh === false || !flock($capFh, LOCK_EX)) {
+    respond(503, ['error' => 'cap_io']);
+}
+$today = gmdate('Y-m-d');
+$cap = json_decode((string)stream_get_contents($capFh), true);
+if (!is_array($cap) || ($cap['day'] ?? '') !== $today) {
+    $cap = ['day' => $today, 'count' => 0];
+}
+if ($cap['count'] >= $dailyCap) {
+    flock($capFh, LOCK_UN);
+    fclose($capFh);
+    respond(429, ['error' => 'cap_exceeded']);
+}
+// Increment BEFORE the LLM call — attempts count against the cap
+// (same semantics as llm_client._under_cap).
+$cap['count']++;
+rewind($capFh);
+if (ftruncate($capFh, 0) === false
+        || fwrite($capFh, json_encode($cap)) === false) {
+    flock($capFh, LOCK_UN);
+    fclose($capFh);
+    respond(503, ['error' => 'cap_io']);
+}
+fflush($capFh);
+flock($capFh, LOCK_UN);
+fclose($capFh);
+
+// ── 5. API key ──────────────────────────────────────────────────────
+$keyFile = getenv('DEEPSEEK_KEY_FILE') ?: '/etc/gold-dashboard/deepseek.key';
+$apiKey = trim((string)@file_get_contents($keyFile));
+if ($apiKey === '') {
+    respond(503, ['error' => 'no_key']);
+}
+
+// ── 6. per-id stampede lock, then re-check the cache ────────────────
+$lockFh = @fopen(CACHE_DIR . '/.ht_lock_' . $id, 'c');
+if ($lockFh !== false) {
+    flock($lockFh, LOCK_EX);   // blocks while a concurrent winner generates
+    $cached = @file_get_contents($cachePath);
+    if ($cached !== false && $cached !== '') {
+        flock($lockFh, LOCK_UN);
+        fclose($lockFh);
+        http_response_code(200);
+        header('Content-Type: application/json; charset=utf-8');
+        header('Cache-Control: max-age=3600');
+        echo $cached;
+        exit;
+    }
+}
+
+// ── 7. DeepSeek call (exact chat_json body shape) ───────────────────
+$userPayload = json_encode([
+    'headline'    => $hit['headline'] ?? '',
+    'description' => $hit['description'] ?? null,
+    'score'       => $hit['score'] ?? null,
+    'xau_score'   => $hit['score_by_currency']['XAU'] ?? null,
+    'keywords'    => $hit['keywords'] ?? [],
+    'currencies'  => $hit['currencies'] ?? [],
+], JSON_UNESCAPED_UNICODE);
+
+$body = json_encode([
+    'model'    => MODEL,
+    'messages' => [
+        ['role' => 'system', 'content' => SYSTEM_PROMPT],
+        ['role' => 'user',   'content' => $userPayload],
+    ],
+    'response_format' => ['type' => 'json_object'],
+    'temperature'     => 0,
+    'thinking'        => ['type' => 'disabled'],
+    'max_tokens'      => MAX_TOKENS,
+]);
+
+$ch = curl_init(API_URL);
+curl_setopt_array($ch, [
+    CURLOPT_POST           => true,
+    CURLOPT_POSTFIELDS     => $body,
+    CURLOPT_HTTPHEADER     => ['Content-Type: application/json',
+                               'Authorization: Bearer ' . $apiKey],
+    CURLOPT_RETURNTRANSFER => true,
+    CURLOPT_TIMEOUT        => TIMEOUT,
+]);
+$raw = curl_exec($ch);
+$httpCode = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
+curl_close($ch);
+
+// ── 8. parse (any failure → 502, never cached) ──────────────────────
+$summary = null;
+if ($raw !== false && $httpCode === 200) {
+    $resp = json_decode($raw, true);
+    $content = $resp['choices'][0]['message']['content'] ?? '';
+    $parsed = json_decode((string)$content, true);
+    if (is_array($parsed) && is_string($parsed['summary'] ?? null)
+            && trim($parsed['summary']) !== '') {
+        $summary = trim($parsed['summary']);
+        if (mb_strlen($summary) > SUMMARY_MAX_CHARS) {
+            $summary = mb_substr($summary, 0, SUMMARY_MAX_CHARS) . '…';
+        }
+    }
+}
+if ($lockFh !== false && $summary === null) {
+    flock($lockFh, LOCK_UN);
+    fclose($lockFh);
+}
+if ($summary === null) {
+    respond(502, ['error' => 'llm_failed']);
+}
+
+// ── 9. cache + respond ──────────────────────────────────────────────
+$out = json_encode(['summary' => $summary, 'generated_at' => gmdate('c')],
+                   JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
+if (!atomic_write($cachePath, $out)) {
+    // Degrade to per-click generation rather than a broken feature —
+    // the daily cap still bounds spend.
+    error_log('news_summary.php: cache write failed for ' . $cachePath);
+}
+if ($lockFh !== false) {
+    flock($lockFh, LOCK_UN);
+    fclose($lockFh);
+}
+http_response_code(200);
+header('Content-Type: application/json; charset=utf-8');
+header('Cache-Control: max-age=3600');
+echo $out;

telegram_bot.py

@@ -71,6 +71,8 @@ from app_config import (
     telegram_channel_id,
 )
 from providers import BaseDataProvider, make_data_provider
+from agent_panel import attach_panel, panel_health
+from llm_client import get_llm_client
 from news_engine import FundamentalContext, NewsEngine
 from pair_signals import PAIR_PRODUCERS
 from signal_aggregator import SignalAggregator, SignalReport
@@ -103,6 +105,12 @@ logging.basicConfig(level=getattr(logging, _APP.log_level, logging.INFO),
                     force=True)
 logging.getLogger("httpx").setLevel(logging.WARNING)
 logging.getLogger("httpcore").setLevel(logging.WARNING)
+# Never let DEBUG runs write credentials into the 30-day rotating log:
+# the websockets client logger emits the connect URI (Twelve Data puts
+# the API key in the query string) and frame contents (Tiingo's auth
+# frame carries the key) — 2026-08-04 audit, finding K47.
+logging.getLogger("websockets").setLevel(logging.INFO)
+logging.getLogger("websockets.client").setLevel(logging.INFO)
 logger = logging.getLogger("telegram_bot")
 
 TOKEN = telegram_token()
@@ -400,6 +408,17 @@ def format_signal_message(report: SignalReport) -> str:
         extra.append(f"⚠️ {_escape_md(w)}")
     if report.news_summary:
         extra.append(f"📰 {_escape_md(report.news_summary)}")
+    if report.panel_verdict:
+        mark = {"AGREE": "✅", "OPPOSE": "⛔", "NEUTRAL": "➖"}.get(
+            report.panel_verdict, "")
+        extra.append(
+            f"🤖 AI Panel {mark} *{_escape_md(report.panel_verdict)}* "
+            f"({report.panel_confidence:.0f}%) — "
+            f"{_escape_md(report.panel_rationale)}")
+        if report.panel_takes:
+            votes = " · ".join(
+                f"{t['persona']}: {t['stance']}" for t in report.panel_takes)
+            extra.append(f"   _{_escape_md(votes)}_")
     if extra:
         lines += [""] + extra
 
@@ -600,6 +619,45 @@ def _commit_alert(key: str, report: SignalReport) -> None:
         _last_alert_setup.pop(key, None)
 
 
+# Per-key locks serializing the _should_alert → send → _commit_alert
+# window. The auto-scan loop and a concurrent breach handler share the
+# "XAU/USD_15min" dedup key; without the lock both pass _should_alert
+# while the other's send is still awaiting Telegram, and the same signal
+# goes out twice (2026-08-04 audit H36). Per-key rather than one global
+# lock so a slow send on one timeframe cannot head-of-line-block another
+# timeframe's alert. Keys are symbol × TF, so the dict stays tiny.
+_alert_locks: Dict[str, asyncio.Lock] = {}
+
+
+def _alert_lock(key: str) -> asyncio.Lock:
+    lock = _alert_locks.get(key)
+    if lock is None:
+        lock = _alert_locks[key] = asyncio.Lock()
+    return lock
+
+
+async def _send_alert(bot, chat_id: int, text: str) -> bool:
+    """Markdown alert send with a plain-text fallback on parse failures,
+    mirroring _safe_edit (2026-08-04 audit H35 — alert paths previously
+    dropped the message on a BadRequest with only a warning logged).
+    Returns True when a message reached the chat by either attempt."""
+    try:
+        await bot.send_message(chat_id, text, parse_mode=ParseMode.MARKDOWN)
+        return True
+    except BadRequest as exc:
+        logger.warning(f"Markdown alert failed for chat {chat_id} ({exc}); "
+                       f"retrying as plain text")
+        try:
+            await bot.send_message(chat_id, text)
+            return True
+        except Exception as exc2:
+            logger.warning(f"Failed to send alert to chat {chat_id}: {exc2}")
+            return False
+    except Exception as exc:
+        logger.warning(f"Failed to send alert to chat {chat_id}: {exc}")
+        return False
+
+
 # ── Candle-boundary scheduler ────────────────────────────────────────
 
 _autoscan_chats: Set[int] = set()
@@ -628,8 +686,11 @@ def _seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
     elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
     interval_seconds = interval_minutes * 60
     remaining = interval_seconds - elapsed_seconds + buffer
-    if remaining <= buffer:
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
-        remaining += interval_seconds
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
     return remaining
 
 
@@ -644,7 +705,13 @@ async def _scan_timeframe(
     reports: List[SignalReport] = []
     for i, result in enumerate(results):
         if isinstance(result, Exception):
-            logger.exception(f"Auto-scan failed for {SCAN_PAIRS[i]} {tf.value}: {result}")
+            # gather(return_exceptions=True) means there is no active
+            # exception here — logger.exception would log "NoneType: None"
+            # instead of the real stack; pass the instance explicitly.
+            logger.error(
+                f"Auto-scan failed for {SCAN_PAIRS[i]} {tf.value}: {result}",
+                exc_info=result,
+            )
         else:
             reports.append(result)
 
@@ -653,36 +720,58 @@ async def _scan_timeframe(
 
     _update_level_cache(reports)
 
-    actionable = [
+    candidates = [
         r for r in reports
         if r.final_signal != SignalStrength.NEUTRAL   # same gate as breach path
         and _passes_quality(r.signal_quality)
-        and _should_alert(f"{r.symbol}_{tf.value}", r)
     ]
-    sent_ok: Set[int] = set()
+    for report in candidates:
-    for chat_id in _signal_targets():
+        key = f"{report.symbol}_{tf.value}"
-        for i, report in enumerate(actionable):
+        # The lock spans check → send → commit so a concurrent breach
+        # handler on the same key cannot double-alert (H36).
+        async with _alert_lock(key):
+            if not _should_alert(key, report):
+                continue
+            # Panel AFTER the quality gate + dedup decision so no LLM
+            # spend on suppressed alerts. Runs on the scan executor to
+            # keep the event loop free; this key's alert lock is held
+            # for the panel's wall time (≤~25 s), which only delays a
+            # same-key duplicate that _should_alert would drop anyway.
+            await asyncio.get_running_loop().run_in_executor(
+                _executor, attach_panel, report)
             detail = format_signal_message(report)
-            try:
+            delivered = False
-                await bot.send_message(chat_id, detail, parse_mode=ParseMode.MARKDOWN)
+            for chat_id in _signal_targets():
-                sent_ok.add(i)
+                if await _send_alert(bot, chat_id, detail):
-            except Exception as exc:
+                    delivered = True
-                logger.warning(f"Failed to send signal for chat {chat_id}: {exc}")
+            # Commit only when the alert reached at least one target;
-    # Commit fingerprints only for alerts that reached at least one target;
+            # a fully failed send stays eligible for the next scan.
-    # a fully failed send stays eligible for the next scan.
+            if delivered:
-    for i, report in enumerate(actionable):
+                _commit_alert(key, report)
-        if i in sent_ok:
-            _commit_alert(f"{report.symbol}_{tf.value}", report)
 
 
 # ── WebSocket level-breach monitor ──────────────────────────────────
 
-_level_cache: Dict[str, Dict[str, float]] = {}
+# Level cache keyed symbol → timeframe → {level_name: price}, with a
-_level_cache_ts: Dict[str, float] = {}
+# matching per-TF write timestamp. Keyed per TF (2026-08-04 audit H34)
+# because every scan path writes here — the M5/M15/H1/H4 boundary loops,
+# the breach rescan, the manual Scan-All button — and a single flat dict
+# meant the last writer wiped the other timeframes' swing/EMA levels
+# within minutes. Breach checks scan the union of all fresh TF entries.
+_level_cache: Dict[str, Dict[str, Dict[str, float]]] = {}
+_level_cache_ts: Dict[str, Dict[str, float]] = {}
 _LEVEL_CACHE_TTL = 3600
 _last_prices: Dict[str, float] = {}
 _last_price_ts: Dict[str, float] = {}
 _scan_cooldowns: Dict[str, float] = {}
+# Breaches are edge-triggered (only the crossing tick fires), so one that
+# lands while the symbol is inside the rescan cooldown would otherwise be
+# dropped forever (H33b). Park it here; the tick loop fires it once the
+# cooldown expires. symbol → (price, level_name, recorded_at).
+_pending_breach: Dict[str, tuple] = {}
+# A failed breach rescan shrinks the just-set cooldown to this retry
+# window instead of silencing the symbol for the full cooldown (H33a).
+_BREACH_RETRY_SECONDS = 30
 
 _cfg = PlatformConfig()
 _WS_COOLDOWN = _cfg.ws_cooldown_seconds
@@ -691,6 +780,8 @@ _breach_tasks: Set[asyncio.Task] = set()
 
 
 def _update_level_cache(reports: List[SignalReport]) -> None:
+    """Each report updates only its own timeframe's slot — an M5 scan can
+    no longer overwrite the H1/H4 swing levels (H34)."""
     now = time.time()
     for r in reports:
         if r.primary_analysis and r.primary_analysis.key_levels:
@@ -699,26 +790,31 @@ def _update_level_cache(reports: List[SignalReport]) -> None:
                 if k in _LEVEL_KEYS
             }
             if levels:
-                _level_cache[r.symbol] = levels
+                tf = r.primary_analysis.timeframe
-                _level_cache_ts[r.symbol] = now
+                _level_cache.setdefault(r.symbol, {})[tf] = levels
+                _level_cache_ts.setdefault(r.symbol, {})[tf] = now
 
 
 def _check_level_breach(symbol: str, price: float) -> Optional[str]:
-    levels = _level_cache.get(symbol)
+    """Scan the union of every fresh timeframe's levels; returns
-    if not levels:
+    '<tf> <level_name>' (e.g. '15min swing_high_1') on a crossing so the
-        return None
+    alert label says which timeframe's level broke."""
-
+    per_tf = _level_cache.get(symbol)
-    cache_age = time.time() - _level_cache_ts.get(symbol, 0)
+    if not per_tf:
-    if cache_age > _LEVEL_CACHE_TTL:
         return None
 
     last = _last_prices.get(symbol)
     if last is None:
         return None
 
-    for name, level in levels.items():
+    now = time.time()
-        if (last < level <= price) or (last > level >= price):
+    ts_map = _level_cache_ts.get(symbol, {})
-            return name
+    for tf, levels in per_tf.items():
+        if now - ts_map.get(tf, 0) > _LEVEL_CACHE_TTL:
+            continue   # stale TF entry — skip, fresher TFs still count
+        for name, level in levels.items():
+            if (last < level <= price) or (last > level >= price):
+                return f"{tf} {name}"
 
     return None
 
@@ -733,9 +829,17 @@ def _cleanup_cooldowns() -> None:
 async def _handle_breach(bot, pair_symbol: str, price: float, level_name: str) -> None:
     now = time.time()
     if now - _scan_cooldowns.get(pair_symbol, 0) < _WS_COOLDOWN:
+        # Park the crossing instead of dropping it — price holding beyond
+        # the level never re-fires the edge trigger, so without this the
+        # breach would be lost forever (H33b). The tick loop fires it
+        # once the cooldown expires; a newer crossing overwrites it.
+        _pending_breach[pair_symbol] = (price, level_name, now)
         return
 
+    # Early-set on purpose: this is what stops concurrent breach tasks
+    # from stacking rescans. The failure path below rolls it back.
     _scan_cooldowns[pair_symbol] = now
+    _pending_breach.pop(pair_symbol, None)   # this crossing is being handled
     display = PAIR_INFO.get(pair_symbol, {}).get("display", pair_symbol)
     label = level_name.replace("_", " ").title()
     logger.info(f"Level breach: {display} crossed {label} at {price} — rescanning M15")
@@ -745,29 +849,46 @@ async def _handle_breach(bot, pair_symbol: str, price: float, level_name: str) -
         _update_level_cache([report])
     except Exception:
         logger.exception(f"Breach scan failed for {display}")
+        # Shrink the just-set cooldown to a short retry window so a
+        # transient fetch error doesn't silence the symbol for the full
+        # cooldown (H33a); the floor bounds the retry rate while the
+        # provider stays down. Re-park the breach so the tick loop
+        # retries it (setdefault: a newer crossing recorded meanwhile
+        # wins over this stale one).
+        retry = min(_BREACH_RETRY_SECONDS, _WS_COOLDOWN)
+        _scan_cooldowns[pair_symbol] = now - _WS_COOLDOWN + retry
+        _pending_breach.setdefault(pair_symbol, (price, level_name, now))
         return
 
     # Alert only when the rescan produced an actionable, changed signal —
     # same gates as the auto-scanner, so a breach can't spam the channel.
     if (report.final_signal == SignalStrength.NEUTRAL
-            or not _passes_quality(report.signal_quality)
+            or not _passes_quality(report.signal_quality)):
-            or not _should_alert(f"{pair_symbol}_{Timeframe.M15.value}", report)):
         return
 
-    header = (
+    key = f"{pair_symbol}_{Timeframe.M15.value}"
-        f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed "
+    # Lock spans check → send → commit; without it this handler and an
-        f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n"
+    # in-flight M15 auto-scan both pass _should_alert and double-alert
-    )
+    # the same signal (H36).
-    sent = False
+    async with _alert_lock(key):
-    for chat_id in _signal_targets():
+        if not _should_alert(key, report):
+            return
+
+        # Same post-gate panel attach as the auto-scan path (see
+        # _scan_timeframe for the spend/lock rationale).
+        await asyncio.get_running_loop().run_in_executor(
+            _executor, attach_panel, report)
+        header = (
+            f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed "
+            f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n"
+        )
         detail = header + format_signal_message(report)
-        try:
+        sent = False
-            await bot.send_message(chat_id, detail, parse_mode=ParseMode.MARKDOWN)
+        for chat_id in _signal_targets():
-            sent = True
+            if await _send_alert(bot, chat_id, detail):
-        except Exception as exc:
+                sent = True
-            logger.warning(f"Failed to send breach alert to chat {chat_id}: {exc}")
+        if sent:
-    if sent:
+            _commit_alert(key, report)
-        _commit_alert(f"{pair_symbol}_{Timeframe.M15.value}", report)
 
 
 def _breach_task_done(task: asyncio.Task) -> None:
@@ -799,13 +920,16 @@ async def _ws_monitor(bot) -> None:
 
     backoff = 1
     while _feed_active():
+        got_tick = False
         try:
-            def _on_connect() -> None:
+            async for pair_symbol, price in dp.iter_ticks(SCAN_PAIRS):
-                nonlocal backoff
+                if not got_tick:
-                backoff = 1
+                    # Reset backoff only once the connection actually
-
+                    # produced data — resetting on connect let a
-            async for pair_symbol, price in dp.iter_ticks(
+                    # rejected key or accept-then-close server pin the
-                    SCAN_PAIRS, on_connect=_on_connect):
+                    # backoff at minimum forever (H38a/H38b).
+                    got_tick = True
+                    backoff = 1
                 if not _feed_active():
                     break
 
@@ -814,11 +938,32 @@ async def _ws_monitor(bot) -> None:
                 _last_price_ts[pair_symbol] = time.time()
 
                 if breached:
+                    # A fresh crossing supersedes any parked one.
+                    _pending_breach.pop(pair_symbol, None)
                     task = asyncio.create_task(
                         _handle_breach(bot, pair_symbol, price, breached)
                     )
                     _breach_tasks.add(task)
                     task.add_done_callback(_breach_task_done)
+                elif pair_symbol in _pending_breach:
+                    pend = _pending_breach[pair_symbol]
+                    if time.time() - pend[2] > _LEVEL_CACHE_TTL:
+                        # Ancient parked breach (feed gap/weekend) — an
+                        # alert about an hours-old crossing would only
+                        # mislead; the level cache it came from is stale
+                        # anyway.
+                        _pending_breach.pop(pair_symbol)
+                    elif (time.time() - _scan_cooldowns.get(pair_symbol, 0)
+                            >= _WS_COOLDOWN):
+                        # Crossing parked during the cooldown: fire it now
+                        # that the window expired — the edge trigger alone
+                        # would have dropped it forever (H33b).
+                        _pending_breach.pop(pair_symbol)
+                        task = asyncio.create_task(
+                            _handle_breach(bot, pair_symbol, pend[0], pend[1])
+                        )
+                        _breach_tasks.add(task)
+                        task.add_done_callback(_breach_task_done)
 
                 _cleanup_cooldowns()
 
@@ -828,6 +973,16 @@ async def _ws_monitor(bot) -> None:
             logger.exception(f"WebSocket error, reconnecting in {backoff}s")
             await asyncio.sleep(backoff)
             backoff = min(backoff * 2, 60)
+        else:
+            # Clean server close: the async-for ended without raising.
+            # This must back off and log like the error path — silently
+            # reconnecting with zero delay hot-loops against a server
+            # that accepts and immediately closes (H38a).
+            if _feed_active():
+                logger.warning(
+                    f"WebSocket closed cleanly, reconnecting in {backoff}s")
+                await asyncio.sleep(backoff)
+                backoff = min(backoff * 2, 60)
 
     logger.info("WebSocket monitor stopped")
 
@@ -870,15 +1025,25 @@ _ws_task: Optional[asyncio.Task] = None
 
 def _start_scheduler(bot) -> None:
     global _ws_task
-    alive = [t for t in _autoscan_tasks if not t.done()]
+    # Per-TF respawn (2026-08-04 audit H37): drop finished loop tasks and
-    if not alive:
+    # recreate only the missing ones. The old all-or-nothing alive check
-        _autoscan_tasks.clear()
+    # left a single dead TF loop unmonitored forever while its siblings
-        for tf, interval in _AUTOSCAN_LOOPS:
+    # lived — and with the channel feed active, /autoscan never reaches
-            _autoscan_tasks.append(
+    # _stop_scheduler, so nothing short of a process restart recovered it.
-                asyncio.create_task(_candle_loop(bot, tf, interval))
+    _autoscan_tasks[:] = [t for t in _autoscan_tasks if not t.done()]
-            )
+    alive_names = {t.get_name() for t in _autoscan_tasks}
+    started = []
+    for tf, interval in _AUTOSCAN_LOOPS:
+        name = f"candle_loop_{tf.name}"
+        if name in alive_names:
+            continue
+        _autoscan_tasks.append(
+            asyncio.create_task(_candle_loop(bot, tf, interval), name=name)
+        )
+        started.append(tf.name)
+    if started:
         logger.info("Candle-boundary scheduler started (%s)",
-                    "/".join(tf.name for tf, _ in _AUTOSCAN_LOOPS))
+                    "/".join(started))
 
     if _ws_task is None or _ws_task.done():
         _ws_task = asyncio.create_task(_ws_monitor(bot))
@@ -973,6 +1138,91 @@ async def cmd_help(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
     await update.message.reply_text(_HELP_TEXT, parse_mode=ParseMode.MARKDOWN)
 
 
+def _fmt_age(ts: Optional[float]) -> str:
+    """Compact age like '45s', '12m', '3.2h' for status lines."""
+    if not ts:
+        return "?"
+    s = time.time() - ts
+    if s < 90:
+        return f"{s:.0f}s"
+    if s < 5400:
+        return f"{s / 60:.0f}m"
+    return f"{s / 3600:.1f}h"
+
+
+def _news_health_lines() -> List[str]:
+    """News / calendar / DeepSeek source health for /status.
+
+    Reads the class-level `NewsEngine._source_health` written by the
+    scan-path fetches plus the LLM client's health stamps — display
+    only, never triggers a fetch. ⚠️ marks a degraded dependency;
+    details live in the rotating log (transition-logged WARNINGs)."""
+    lines = ["\n*News & AI:*"]
+    # Snapshot: a scan thread may insert a source's FIRST entry while we
+    # iterate (dict-changed-size RuntimeError); entry mutation is safe.
+    for _ in range(3):
+        try:
+            sh = dict(NewsEngine._source_health)
+            break
+        except RuntimeError:
+            continue
+    else:
+        sh = {}
+    if not sh:
+        lines.append("  _no news cycle yet (fills after first scan)_")
+    else:
+        cal = sh.get("calendar")
+        ff = sh.get("calendar:forexfactory")
+        if cal:
+            if NewsEngine.source_is_failing(cal):
+                lines.append(f"  ⚠️ Calendar: unavailable — {cal['detail']}"
+                             f" ({_fmt_age(cal['err'])})")
+            elif NewsEngine.source_is_failing(ff):
+                lines.append(f"  ⚠️ Calendar: {cal['extra']} — ForexFactory"
+                             f" down: {ff['detail']} ({_fmt_age(ff['err'])})")
+            else:
+                lines.append(f"  ✅ Calendar: {cal['extra']}"
+                             f" ({_fmt_age(cal['ok'])} ago)")
+        heads = sh.get("headlines")
+        if heads:
+            failing = NewsEngine.source_is_failing(heads)
+            mark = "⚠️" if failing else "✅"
+            detail = f" — {heads['detail']}" if failing else ""
+            ts = heads["err"] if failing else heads["ok"]
+            lines.append(f"  {mark} Headlines: {heads['extra']}{detail}"
+                         f" ({_fmt_age(ts)} ago)")
+        for name, entry in sorted(sh.items()):
+            if name.startswith("rss:") and NewsEngine.source_is_failing(entry):
+                lines.append(f"  ⚠️ {name[4:]}: {entry['detail']}"
+                             f" ({_fmt_age(entry['err'])})")
+    client = get_llm_client()
+    if not client.available:
+        lines.append("  DeepSeek: off — keyword scoring only")
+    else:
+        # The client's counter only resets on the first call of a new
+        # UTC day — show 0 (not yesterday's total, or a phantom
+        # cap-exhausted warning) until that reset happens.
+        today = datetime.now(timezone.utc).date()
+        used = client._count if client._count_day == today else 0
+        cap = client._daily_cap
+        scored = len(NewsEngine._llm_sent_cache)
+        err = client.last_error
+        if used >= cap:
+            lines.append(f"  ⚠️ DeepSeek: daily cap {cap} exhausted —"
+                         " keyword fallback until UTC midnight")
+        elif err and err[0] > client.last_ok:
+            lines.append(f"  ⚠️ DeepSeek: last call failed ({err[1]},"
+                         f" {_fmt_age(err[0])}) — keyword fallback")
+        else:
+            lines.append(f"  ✅ DeepSeek: {used}/{cap} calls today,"
+                         f" {scored} headlines scored")
+        if get_app_config().agent_panel_settings.get("enabled", False):
+            ph = panel_health()
+            lines.append(f"  🤖 AI panel: {ph['used_today']}/{ph['cap']}"
+                         " panels today")
+    return lines
+
+
 def _status_text() -> str:
     lines = ["📊 *ForexBot2 Status*\n"]
 
@@ -1006,9 +1256,16 @@ def _status_text() -> str:
         lines.append("\n_No live prices yet (WS not streaming)_")
 
     cached_pairs = len(_level_cache)
-    cached_levels = sum(len(v) for v in _level_cache.values())
+    cached_levels = sum(
+        len(levels) for per_tf in _level_cache.values()
+        for levels in per_tf.values()
+    )
     if cached_levels:
-        oldest_age = time.time() - min(_level_cache_ts.values()) if _level_cache_ts else 0
+        all_ts = [
+            ts for per_tf in _level_cache_ts.values()
+            for ts in per_tf.values()
+        ]
+        oldest_age = time.time() - min(all_ts) if all_ts else 0
         lines.append(f"\n*Level Cache:* {cached_levels} levels across {cached_pairs} pairs (oldest {oldest_age:.0f}s)")
     else:
         lines.append("\n*Level Cache:* empty — breach alerts start after first candle scan")
@@ -1024,6 +1281,8 @@ def _status_text() -> str:
     if active_breach:
         lines.append(f"*Breach tasks:* {active_breach} running")
 
+    lines.extend(_news_health_lines())
+
     lines.append(f"\n*Alert quality:* {_alert_min_quality}+ "
                  f"| re-arm {_ALERT_REARM_SECONDS // 60}m")
     return "\n".join(lines)
@@ -1311,7 +1570,11 @@ async def _post_init(app: Application) -> None:
                 logger.error(f"Initial scan failed for {SCAN_PAIRS[i]}: {r}")
         if init_reports:
             _update_level_cache(init_reports)
-            logger.info(f"Level cache populated: {sum(len(v) for v in _level_cache.values())} levels")
+            n_levels = sum(
+                len(levels) for per_tf in _level_cache.values()
+                for levels in per_tf.values()
+            )
+            logger.info(f"Level cache populated: {n_levels} levels")
     except Exception:
         logger.exception("Initial scan failed — WS will wait for first candle scan")