news: DeepSeek validation, vs_forecast fix, /status source health

Validation (llm_news_eval.py harness + 78-headline golden CSV + verdict
doc): headline scoring 78/78 vs keyword baseline 24/78 — validated;
event gold-direction calls misread close K/M numeric comparisons ~11%
of the time (both claims cases in sample, JOLTS 8.79M vs 8.84M).

Fix: event payload now carries code-computed "vs_forecast" (from
EconomicEvent.surprise) and the prompt gains an explicit inverted-
metrics rule + worked claims example — iteration 1 (field alone) fixed
the arithmetic but exposed a masked inversion error. 18/18 on live
re-verification (4 misreads fixed, 11 regression cases clean, 3 fresh
both-side probes).

/status: new "News & AI" section — per-source health for the calendar
chain (which of FF/MyFXBook served), each RSS feed, and DeepSeek
(daily count, last error kind), written passively by the existing
fetch paths under _cache_lock. Failures WARN once on transition
(repeats DEBUG, recovery INFO) so the rotating log carries outage
stories without 10-min spam.

test_all.py: +5 Module 19 tests (351 total); also carries the Module 22
fixed-RR-TP tests from the prior research session (shared file).
Replay artifacts dir gitignored like commit_ladder.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017mbXs9HRawZhD1uZ3ze7zQ

CLAUDE.md

@@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
 ## Commands
 
 ```bash
-python test_all.py                      # QA suite: 359 tests, 22 modules, no network (mocked)
+python test_all.py                      # QA suite: 338 tests, 21 modules, no network (mocked)
 python main.py --pair XAUUSD --timeframe H1 --detail   # CLI signal report (--json for full dump)
 python telegram_bot.py                  # the bot (refuses to boot without credentials)
 python oanda_daemon.py                  # auto-execution daemon on Oanda practice account
@@ -61,7 +61,7 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
 - **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that.
 - **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
 - **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`).
-- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. News/calendar rows are clickable (detail dialog); headlines carry `id` = sha1(headline) in news.json, and exactly one on-demand endpoint exists: `static/news_summary.php` (installed by `install_static`), which serves a DeepSeek gold-impact summary per headline — whitelist-gated against news.json ids, cache-forever in `ai/`, own fail-closed daily cap (150/day, file-based, separate from the publisher's), key at `/etc/gold-dashboard/deepseek.key`, never reads repo config. Deployment checklist: `deploy/README-dashboard.md`.
+- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. Deployment checklist: `deploy/README-dashboard.md`.
 - **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
 
 ## Rules for making changes (read before editing signal logic)

dashboard_publisher.py

@@ -37,7 +37,6 @@ Out dir precedence: --out → DASHBOARD_OUT_DIR env → /var/www/trade
 
 import argparse
 import json
-import hashlib
 import logging
 import math
 import numbers
@@ -77,11 +76,6 @@ SCAN_BUFFER_SECONDS = 10
 SNAPSHOT_FILE = "snapshot.json"
 NEWS_FILE = "news.json"
 PAGE_FILE = "dashboard.html"
-SUMMARY_PHP_FILE = "news_summary.php"
-# Repo files copied into the out dir at daemon start (git pull + restart
-# updates them). The PHP endpoint reads news.json in place, so it lives
-# next to the page.
-STATIC_FILES = [PAGE_FILE, SUMMARY_PHP_FILE]
 
 
 # ══════════════════════════════════════════════════════════════════
@@ -186,15 +180,9 @@ def event_to_dict(ev) -> dict:
 
 
 def headline_to_dict(h) -> dict:
-    # id uses the same sha1(headline) scheme as NewsEngine._llm_sent_cache;
-    # it is the join key the PHP summary endpoint whitelists against.
     return {
-        "id": hashlib.sha1(h.headline.encode("utf-8")).hexdigest(),
         "headline": h.headline,
         "source": h.source,
-        "link": h.link,
-        "description": h.description,
-        "publisher": h.publisher,
         "timestamp": h.timestamp.astimezone(timezone.utc).isoformat()
                      if h.timestamp else None,
         "currencies": list(h.currencies_mentioned),
@@ -336,21 +324,16 @@ class DashboardPublisher:
         return self._news_engine
 
     def install_static(self) -> None:
-        """Copy the repo's static files (page + PHP summary endpoint)
+        """Copy the repo's dashboard.html into the out dir (atomic), so
-        into the out dir (atomic), so git pull + restart also updates
+        git pull + restart also updates the page."""
-        them. The ai/ cache dir is NOT created here — the daemon runs as
+        src = os.path.join(os.path.dirname(os.path.abspath(__file__)),
-        a different user than Apache; that dir is a one-time deploy step
+                           "static", PAGE_FILE)
-        (deploy/README-dashboard.md)."""
+        if not os.path.exists(src):
-        static_dir = os.path.join(
+            logger.warning(f"{src} missing — page not installed")
-            os.path.dirname(os.path.abspath(__file__)), "static")
+            return
-        for name in STATIC_FILES:
+        dst = os.path.join(self.out_dir, PAGE_FILE)
-            src = os.path.join(static_dir, name)
+        shutil.copyfile(src, dst + ".tmp")
-            if not os.path.exists(src):
+        os.replace(dst + ".tmp", dst)
-                logger.warning(f"{src} missing — not installed")
-                continue
-            dst = os.path.join(self.out_dir, name)
-            shutil.copyfile(src, dst + ".tmp")
-            os.replace(dst + ".tmp", dst)
 
     def scan_tf(self, tf: Timeframe) -> None:
         started = time.time()
@@ -388,29 +371,18 @@ class DashboardPublisher:
         except Exception:
             logger.exception("news refresh failed — keeping last payload")
 
-    def publish_snapshot(self) -> None:
+    def publish_all(self) -> None:
-        publish(build_snapshot_payload(
+        now = datetime.now(timezone.utc)
-            self.snapshots, datetime.now(timezone.utc)),
+        publish(build_snapshot_payload(self.snapshots, now),
-            os.path.join(self.out_dir, SNAPSHOT_FILE))
+                os.path.join(self.out_dir, SNAPSHOT_FILE))
-
-    def publish_news(self) -> None:
         if self.news_payload is not None:
             publish(self.news_payload, os.path.join(self.out_dir, NEWS_FILE))
 
-    def publish_all(self) -> None:
-        self.publish_snapshot()
-        self.publish_news()
-
     def run_cycle(self, tfs: List[Timeframe]) -> None:
         for tf in tfs:  # sequential — see module docstring (cache dedup)
             self.scan_tf(tf)
-        # Snapshot lands BEFORE the news refresh: when the news TTL has
-        # expired, refresh_news does FF + 4 RSS fetches (and possibly
-        # DeepSeek calls) — 20-30 s that must not keep fresh signals off
-        # the page.
-        self.publish_snapshot()
         self.refresh_news()
-        self.publish_news()
+        self.publish_all()
 
     def run_forever(self) -> None:
         self.run_cycle(DASHBOARD_TFS)  # populate the page at boot

deploy/README-dashboard.md

@@ -2,12 +2,10 @@
 
 The dashboard is a **static publisher**: `dashboard_publisher.py` runs as its
 own daemon, scans XAU/USD at candle closes, and atomically writes
-`dashboard.html` + `news_summary.php` + `snapshot.json` + `news.json` into a
+`dashboard.html` + `snapshot.json` + `news.json` into a folder Apache already
-folder Apache already serves. Apache does all web serving — no new port, no
+serves. Apache does all web serving — no new port, no Python web server, and
-Python web server, and the telegram bot process is completely unaffected by
+the telegram bot process is completely unaffected by anything that happens
-anything that happens here (see the module docstring for the independence
+here (see the module docstring for the independence rules).
-rules). The one dynamic piece is `news_summary.php` (§6): an on-demand
-DeepSeek summary for a clicked headline, whitelist-gated against news.json.
 
 All steps below run **on the live server** (`/home/Forex/ForexBot2`).
 
@@ -24,10 +22,9 @@ No new Python dependencies are required.
 ## 2. Make the web folder writable
 
 The dashboard publishes straight into the portal docroot `/var/www/trade/`
-(owner decision 2026-08-03) — it adds exactly four files there:
+(owner decision 2026-08-03) — it adds exactly three files there:
-`dashboard.html`, `news_summary.php`, `snapshot.json`, `news.json` (plus the
+`dashboard.html`, `snapshot.json`, `news.json`. The daemon user must be able
-apache-owned `ai/` cache dir from §6). The daemon user must be able to write
+to write them:
-them:
 
 ```bash
 # if the portal files can be owned by the daemon user:
@@ -75,9 +72,8 @@ every 3 h (240 backups), same scheme as the bot.
 
 ## 5. Optional Apache polish
 
-The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache
+The page cache-busts its JSON fetches (`?t=<epoch>`), so no Apache config is
-answers unchanged files with 304), so no Apache config is strictly needed. To
+strictly needed. To be explicit anyway:
-be explicit anyway:
 
 ```apache
 <Directory "/var/www/trade">
@@ -91,67 +87,15 @@ The page is deliberately public (owner decision 2026-08-03). To restrict it
 later, add standard Apache basic auth (`htpasswd` + `Require valid-user`) on
 that directory — no code changes needed.
 
-## 6. AI summary endpoint (PHP) — one-time setup
-
-Clicking a newswire headline opens a detail dialog that fetches an on-demand
-DeepSeek gold-impact summary from `news_summary.php` (installed next to the
-page by the daemon). Owner decision 2026-08-07: a real on-demand endpoint on
-the public page, accepted with these guards baked into the PHP — the id must
-match a headline in the published news.json (arbitrary text can never reach
-the LLM), each headline's summary is cached forever in `ai/` (≤1 API call per
-headline, ever), and a file-based daily cap fails **closed** on I/O errors.
-
-Requirements: `php` + the curl extension (`php -m | grep curl`).
-
-```bash
-# 1. SELinux: allow httpd outbound HTTPS — WITHOUT this every summary call
-#    dies as a curl timeout even with perfect perms (RHEL-family default).
-sudo setsebool -P httpd_can_network_connect 1
-
-# 2. API key OUTSIDE the docroot, readable by Apache only:
-sudo install -d -o root -g root -m 755 /etc/gold-dashboard
-sudo sh -c 'umask 137; echo "sk-..." > /etc/gold-dashboard/deepseek.key'
-sudo chown root:apache /etc/gold-dashboard/deepseek.key   # mode 640
-
-# 3. apache-owned cache dir (the daemon runs as Forex and can't create it):
-sudo install -d -o apache -g apache -m 755 /var/www/trade/ai
-sudo semanage fcontext -a -t httpd_sys_rw_content_t '/var/www/trade/ai(/.*)?'
-sudo restorecon -Rv /var/www/trade/ai
-```
-
-Knobs (Apache `SetEnv` in the vhost, both optional):
-`AI_SUMMARY_DAILY_CAP` (default 150), `DEEPSEEK_KEY_FILE` (default
-`/etc/gold-dashboard/deepseek.key`).
-
-Verify from the shell — expect 400 / 404 / 200 / 200-from-cache:
-
-```bash
-curl -s "http://<server>/news_summary.php?id=zzz"                 # {"error":"bad_id"}
-curl -s "http://<server>/news_summary.php?id=$(printf 'a%.0s' {1..40})"  # unknown_id
-ID=$(python3 -c "import json;print(json.load(open('/var/www/trade/news.json'))['headlines'][0]['id'])")
-curl -s "http://<server>/news_summary.php?id=$ID"   # real summary (2-12 s)
-curl -s "http://<server>/news_summary.php?id=$ID"   # instant, from ai/<id>.json
-```
-
 ## Operational notes
 
 - **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
-  re-copies `static/dashboard.html` and `static/news_summary.php` into the
+  re-copies `static/dashboard.html` into the web folder at startup.
-  web folder at startup.
 - **If the daemon dies**, systemd restarts it in 30 s; worst case the page
   shows stale data with its own "updated N min ago" evidence. Apache and the
   telegram bot are unaffected.
 - **DeepSeek**: the publisher process has its own daily request cap
   (`deepseek.daily_request_cap`, default 500) separate from the bot's —
   expect roughly up to ~430 extra calls/day worst case, V4-Flash pricing.
-- **Two independent DeepSeek meters** hit the same API key: the publisher's
-  in-process cap above, and the PHP endpoint's file cap (`ai/.ht_cap.json`,
-  default 150/day). They don't communicate — worst-case spend is their sum;
-  DeepSeek-side spend alerting is the true backstop. Kept separate on
-  purpose (independence rule: the PHP never imports/reads bot code or
-  config.json).
-- **Restart window**: for the first minutes after a daemon restart the new
-  page can meet a pre-upgrade news.json without headline ids — those rows
-  simply render non-clickable until the first news publish.
 - **Credentials**: uses the same env/.env/config.json resolution as the rest
   of the platform (Oanda candles feed; news needs no key; DeepSeek optional).

docs/architecture_diagram.py

@@ -1,199 +0,0 @@
-"""ForexBot2 architecture diagram → docs/architecture.png.
-
-Regenerate after architecture changes:  venv/bin/python docs/architecture_diagram.py
-(matplotlib only — no graphviz on the dev box)."""
-import os
-import matplotlib
-matplotlib.use("Agg")
-import matplotlib.pyplot as plt
-from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
-
-W, H = 100.0, 100.0
-fig, ax = plt.subplots(figsize=(16, 11.5))
-ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
-
-INK = "#263238"; MUT = "#607D8B"; ARR = "#455A64"; HAUL = "#7B8FA1"
-
-def box(x, y, w, h, title, lines, fc, ec, title_fs=8.2, body_fs=6.6, lw=1.3):
-    ax.add_patch(FancyBboxPatch((x, y), w, h,
-                 boxstyle="round,pad=0.35,rounding_size=0.9",
-                 facecolor=fc, edgecolor=ec, linewidth=lw, zorder=2))
-    cy = y + h - 1.9
-    ax.text(x + w / 2, cy, title, ha="center", va="center", fontsize=title_fs,
-            fontweight="bold", color=INK, zorder=3)
-    cy -= 2.3
-    for ln in lines:
-        ax.text(x + w / 2, cy, ln, ha="center", va="center", fontsize=body_fs,
-                color=INK, zorder=3)
-        cy -= 1.85
-
-def band(x, y, w, h, label, color):
-    ax.add_patch(FancyBboxPatch((x, y), w, h,
-                 boxstyle="round,pad=0.3,rounding_size=1.2",
-                 facecolor="none", edgecolor=color, linewidth=1.0,
-                 linestyle=(0, (4, 3)), zorder=1))
-    ax.text(x + 0.9, y + h - 1.4, label, ha="left", va="center",
-            fontsize=7.2, fontweight="bold", color=color, zorder=3)
-
-def arrow(p1, p2, color=ARR, lw=1.5, style="-|>", conn="arc3,rad=0.0",
-          label=None, lx=0, ly=0, ls="solid", fs=6.4, rot=0):
-    ax.add_patch(FancyArrowPatch(p1, p2, arrowstyle=style, mutation_scale=11,
-                 color=color, linewidth=lw, linestyle=ls,
-                 connectionstyle=conn, zorder=4))
-    if label:
-        mx, my = (p1[0] + p2[0]) / 2 + lx, (p1[1] + p2[1]) / 2 + ly
-        ax.text(mx, my, label, fontsize=fs, color=MUT, ha="center",
-                va="center", zorder=5, rotation=rot,
-                bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
-
-# ── title ────────────────────────────────────────────────────────────
-ax.text(2, 98.4, "ForexBot2 — XAU/USD Signal-Intelligence Platform",
-        fontsize=15, fontweight="bold", color=INK, ha="left")
-ax.text(2, 96.0, "one pipeline, four delivery surfaces · closed bars only · live == backtest parity",
-        fontsize=8.5, color=MUT, ha="left")
-
-EXT_FC, EXT_EC = "#ECEFF1", "#90A4AE"
-DAT_FC, DAT_EC = "#E3F2FD", "#5C9BD1"
-NWS_FC, NWS_EC = "#E0F2F1", "#3D9C8F"
-COR_FC, COR_EC = "#FFFDF5", "#C9A227"
-REP_FC, REP_EC = "#FFE9A8", "#B8860B"
-DEL_FC, DEL_EC = "#E8F5E9", "#5FAF63"
-BTS_FC, BTS_EC = "#EDE7F6", "#8E6BC6"
-
-# ── external services (top band) ─────────────────────────────────────
-band(1, 85.0, 98, 9.4, "EXTERNAL SERVICES", "#78909C")
-box(2.5, 86.2, 15.5, 6.4, "Oanda v20 API",
-    ["candles (live feed)", "practice orders"], EXT_FC, EXT_EC)
-box(19.5, 86.2, 15.5, 6.4, "Twelve Data · Tiingo",
-    ["registered fallbacks", "(Tiingo: tick stream)"], EXT_FC, EXT_EC)
-box(36.5, 86.2, 15.5, 6.4, "Forex Factory",
-    ["economic calendar", "MyFXBook RSS fallback"], EXT_FC, EXT_EC)
-box(53.5, 86.2, 12.5, 6.4, "News RSS ×4",
-    ["gold / macro", "headlines"], EXT_FC, EXT_EC)
-box(67.5, 86.2, 13.5, 6.4, "DeepSeek API",
-    ["V4-Flash LLM", "thinking: disabled"], EXT_FC, EXT_EC)
-box(82.5, 86.2, 15, 6.4, "Telegram API",
-    ["owner bot +", "subscriber channel"], EXT_FC, EXT_EC)
-
-# ── data & context layer ─────────────────────────────────────────────
-box(2.5, 69.8, 33, 11.0, "providers/  (BaseDataProvider)",
-    ["single-flight fetch · TTL+boundary cache",
-     "forming-candle DROP · validate/clean",
-     "paginated fetch_range · optional tick stream",
-     "active feed knob: data_providers.active = oanda"],
-    DAT_FC, DAT_EC)
-box(38.5, 69.8, 33, 11.0, "news_engine.py  +  llm_client.py",
-    ["calendar events + keyword sentiment (gold-inverse)",
-     "DeepSeek enrichment: batch headline scores,",
-     "event gold-direction, market blurb — display-only,",
-     "keyword fallback, daily request cap"],
-    NWS_FC, NWS_EC)
-box(74.5, 69.8, 23, 11.0, "app_config.py",
-    ["credentials & deployment:",
-     "env → config.json → refuse",
-     "(no hardcoded fallbacks)",
-     ".env loaded at import"],
-    "#FAFAFA", "#9E9E9E")
-
-# ── signal core container ────────────────────────────────────────────
-ax.add_patch(FancyBboxPatch((2.5, 40.5), 95, 26.5,
-             boxstyle="round,pad=0.35,rounding_size=1.2",
-             facecolor=COR_FC, edgecolor=COR_EC, linewidth=1.6, zorder=1))
-ax.text(50, 65.2, "SIGNAL CORE — runs identically in live scans and backtests (closed bars only)",
-        ha="center", fontsize=8.4, fontweight="bold", color="#8a6d1a", zorder=3)
-
-box(4.5, 51.8, 28, 12.0, "engine.py — TechnicalEngine",
-    ["9 scored indicators + ATR context:",
-     "EMA · RSI · MACD · BB · ADX · ROC ·",
-     "Keltner · divergence · candlestick",
-     "regime → adaptive weights → composite"],
-    "white", COR_EC)
-box(4.5, 42.0, 28, 8.4, "pair_signals/xauusd.py — factory",
-    ["ALWAYS used: config.json overlay +",
-     "validated per-TF deltas (hour blocks, trail…)"],
-    "white", COR_EC)
-box(35.5, 42.0, 60, 20.7, "signal_aggregator.py — analyze_pair → analyze_from_dataframes",
-    ["multi-TF agreement  (M5→[M15,H1] · M15→[M5,H1] · H1→[M15,H4] · H4→[H1,D1])",
-     "→ news context (warn-only by default) → diminishing-decay penalties",
-     "→ exhaustion hard block → quality grade A+…F",
-     "→ ordered post-grade gate chain (session · XAU SELL vol/hour · regime · M5 noise)",
-     "→ setup generators: XAU ATR (A+/A) · generic swing · XAU_PB · XAU_CP",
-     "→ entry · SL · TP1/TP2 · trailing plan"],
-    "white", COR_EC, body_fs=6.9)
-
-# ── SignalReport hub ─────────────────────────────────────────────────
-box(31, 33.2, 38, 5.2, "SignalReport",
-    ["signal (−2…+2) · grade · trade setup · reasons · news summary"],
-    REP_FC, REP_EC, title_fs=8.8)
-
-# ── delivery surfaces ────────────────────────────────────────────────
-band(1, 13.2, 98, 17.2, "DELIVERY SURFACES", "#4C8F50")
-box(2.5, 15.0, 12, 12.6, "main.py",
-    ["CLI report", "--detail", "--json"], DEL_FC, DEL_EC)
-box(16, 15.0, 24, 12.6, "oanda_daemon.py + executor",
-    ["practice auto-execution",
-     "book-parity rules: no broker TP,",
-     "5-bar cooldown, once-per-bar",
-     "trail, OPEN_ONLY fills,",
-     "boot reconcile · orders → Oanda"],
-    DEL_FC, DEL_EC, title_fs=7.8)
-box(42.5, 15.0, 31, 12.6, "dashboard_publisher.py  →  Apache",
-    ["independent daemon · scans at candle closes",
-     "atomic publish to /var/www/trade:",
-     "dashboard.html · snapshot.json · news.json",
-     "+ news_summary.php — on-demand AI",
-     "gold-impact summary (whitelist · cache · cap)"],
-    DEL_FC, DEL_EC)
-box(76, 15.0, 21.5, 12.6, "telegram_bot.py",
-    ["candle-boundary auto-scans", "quality threshold + dedup",
-     "alerts + LLM context line", "WS level-breach monitor"],
-    DEL_FC, DEL_EC)
-
-# ── backtest / QA ────────────────────────────────────────────────────
-box(2.5, 3.2, 71.8, 8.0, "backtest/gpu_backtest.py — bar-by-bar replay through the SAME analyze_from_dataframes",
-    ["realism (results-critical): next-open entries · raw-path resolution · spread charged · SL-first ties · close-only trail",
-     "CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
-    BTS_FC, BTS_EC)
-box(77.5, 3.2, 20, 8.0, "test_all.py",
-    ["359 tests · 22 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
-
-# ── arrows: short structural hops (solid dark) ───────────────────────
-arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2)          # oanda → providers
-arrow((27, 86.2), (24, 81.2), ls=(0, (3, 2)), lw=1.1)                  # fallbacks → providers
-arrow((44, 86.2), (48, 81.2))                                          # FF → news
-arrow((59.5, 86.2), (56, 81.2))                                        # RSS → news
-arrow((72, 86.2), (62, 81.2), label="LLM", lx=3.4, ly=0.6)             # deepseek → news
-arrow((19, 69.8), (19, 64.2), label="DataFrames (UTC, closed bars)", lx=0.2, ly=0.9)
-arrow((33.5, 57.5), (35.5, 55.5))                                      # engine → aggregator
-arrow((32.5, 46.0), (35.5, 48.0))                                      # factory → aggregator
-arrow((55, 69.8), (55, 63.1), label="warn-only context", lx=0, ly=1.0)
-arrow((86, 69.8), (86, 67.3), ls=(0, (3, 2)), lw=1.1)                  # config → core
-arrow((50, 40.5), (50, 38.8))                                          # core → report
-arrow((36, 34.5), (8.5, 27.95), conn="arc3,rad=0.12")                  # report → cli
-arrow((40, 33.2), (28, 27.95), conn="arc3,rad=0.06")                   # report → daemon
-arrow((60, 33.2), (58, 27.95))                                         # report → dashboard
-arrow((66, 34.5), (86.5, 27.95), conn="arc3,rad=-0.12")                # report → telegram
-
-# ── long-haul edges (thin dashed, routed through gutters) ────────────
-# telegram bot ↔ Telegram API — right-margin arc, clear of app_config
-arrow((95, 86.2), (95, 27.95), conn="arc3,rad=0.15", color=HAUL,
-      lw=1.2, ls=(0, (4, 2)))
-ax.text(99.2, 57, "alerts / commands", fontsize=6.4, color=MUT,
-        rotation=90, ha="center", va="center")
-# browser click → news_summary.php → DeepSeek
-# (oanda_daemon's order flow is noted inside its box — no long arc)
-arrow((66, 27.95), (73, 86.2), conn="arc3,rad=-0.18", color=HAUL,
-      lw=1.2, ls=(0, (4, 2)),
-      label="on-demand AI summary\n(browser click → PHP)", lx=2.5, ly=11.6)
-# providers.fetch_range → backtester — far-left margin channel
-arrow((1.5, 69.8), (1.5, 11.0), color=BTS_EC, lw=1.1, ls=(0, (3, 2)),
-      label="fetch_range\n(history)", lx=3.6, ly=-1.2)
-# backtester replays the signal core — right gutter between boxes
-arrow((74.0, 11.6), (74.9, 40.3), color=HAUL, lw=1.3)
-ax.text(81.8, 38.9, "replays the exact signal code", fontsize=6.4,
-        color=MUT, ha="center", va="center", zorder=5,
-        bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
-
-out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
-fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
-print(f"saved {out}")

news_engine.py

@@ -9,7 +9,6 @@ A perfect technical setup can be destroyed by an unexpected NFP print.
 """
 
 import hashlib
-import html
 import logging
 import re
 import threading
@@ -104,44 +103,6 @@ def _parse_release_value(raw: Optional[str]) -> Optional[float]:
     return float(match.group(1)) * multiplier
 
 
-_DESC_MAX_CHARS = 500
-_MAX_LINK_CHARS = 2048
-
-
-def _clean_html_text(raw: Optional[str],
-                     limit: int = _DESC_MAX_CHARS) -> Optional[str]:
-    """RSS <description> → plain text: unescape entities, strip tags,
-    collapse whitespace, truncate. Unescape and strip both run twice
-    because feeds routinely double-encode (&amp;lt;p&amp;gt; wrapping
-    real markup) — the second unescape can reveal fresh tags."""
-    if not raw:
-        return None
-    text = html.unescape(raw)
-    text = re.sub(r"<[^>]+>", " ", text)
-    text = html.unescape(text)
-    text = re.sub(r"<[^>]+>", " ", text)
-    text = re.sub(r"\s+", " ", text).strip()
-    if not text:
-        return None
-    if len(text) > limit:
-        text = text[:limit].rstrip() + "…"
-    return text
-
-
-def _safe_link(raw: Optional[str]) -> Optional[str]:
-    """Article URL for display: http(s) only (an RSS <link> is untrusted
-    and lands in an href), bounded length (Google News redirect URLs run
-    1-2 KB; anything past that is bloat, not a link)."""
-    if not raw:
-        return None
-    link = raw.strip()
-    if not link.lower().startswith(("http://", "https://")):
-        return None
-    if len(link) > _MAX_LINK_CHARS:
-        return None
-    return link
-
-
 @dataclass
 class EconomicEvent:
     """Single economic calendar event."""
@@ -194,11 +155,6 @@ class NewsSentiment:
     # Per-currency score: monetary-policy keywords flip sign for XAU
     # (hawkish is dollar-bullish but gold-bearish).
     sentiment_by_currency: Dict[str, float] = field(default_factory=dict)
-    # Display-only enrichment (dashboard detail dialog). Optional with
-    # defaults so existing constructor calls keep working.
-    link: Optional[str] = None         # article URL (Google News = redirect)
-    description: Optional[str] = None  # plain text, tags stripped, ≤500 chars
-    publisher: Optional[str] = None    # Google News <source> text, else None
 
 
 @dataclass
@@ -786,21 +742,6 @@ class NewsEngine:
                     currencies = self._extract_currencies(title)
 
                     if currencies:
-                        link_el = item.find("link")
-                        desc_el = item.find("description")
-                        src_el = item.find("source")
-                        # itertext() survives CDATA and stray child
-                        # elements where .text stops at the first child.
-                        desc = _clean_html_text(
-                            "".join(desc_el.itertext())
-                            if desc_el is not None else None)
-                        # Google News descriptions are often just the
-                        # headline again inside an anchor — worthless.
-                        if desc and desc.strip() == title.strip():
-                            desc = None
-                        publisher = None
-                        if src_el is not None:
-                            publisher = (src_el.text or "").strip() or None
                         headlines.append(NewsSentiment(
                             headline=title,
                             source=feed_url,
@@ -812,10 +753,6 @@ class NewsEngine:
                                 cur: self._analyze_headline_sentiment(title, cur)
                                 for cur in currencies
                             },
-                            link=_safe_link(
-                                link_el.text if link_el is not None else None),
-                            description=desc,
-                            publisher=publisher,
                         ))
 
                 ok_feeds += 1

providers/oanda.py

@@ -8,7 +8,6 @@ Oanda's candle JSON, and exposes the real bid/ask spread from MBA candles.
 """
 
 import logging
-import time
 from datetime import datetime
 from typing import Dict, Optional
 
@@ -32,7 +31,6 @@ class OandaDataProvider(BaseDataProvider):
 
     name = "oanda"
     DATA_ERROR = OandaDataError
-    RETRY_DELAY_S = 2.0
 
     OANDA_GRANULARITY = {
         Timeframe.M1: "M1",
@@ -126,35 +124,11 @@ class OandaDataProvider(BaseDataProvider):
         else:
             params["count"] = min(count, self.MAX_BARS_PER_REQUEST)
 
-        # One retry absorbs Oanda's sporadic spurious 401s (~15-20/day on
+        try:
-        # the practice API — the same token succeeds on adjacent requests)
+            resp = self._session.get(url, params=params, timeout=30)
-        # plus transient 5xx/network faults. A real credential problem
+        except requests.RequestException as e:
-        # fails both attempts and raises exactly as before. Other 4xx
+            raise OandaDataError(
-        # (bad request, 429) are never retried.
+                f"Oanda candle fetch failed for {instrument}: {e}") from e
-        last_err: Optional[OandaDataError] = None
-        for attempt in range(2):
-            if attempt:
-                time.sleep(self.RETRY_DELAY_S)
-            try:
-                resp = self._session.get(url, params=params, timeout=30)
-            except requests.RequestException as e:
-                last_err = OandaDataError(
-                    f"Oanda candle fetch failed for {instrument}: {e}")
-                last_err.__cause__ = e
-                continue
-            if resp.status_code == 401 or resp.status_code >= 500:
-                last_err = OandaDataError(
-                    f"Oanda candle fetch failed for {instrument}: "
-                    f"HTTP {resp.status_code}: {resp.text[:300]}")
-                if attempt == 0:
-                    logger.warning(
-                        f"Oanda fetch for {instrument} got HTTP "
-                        f"{resp.status_code}; retrying in "
-                        f"{self.RETRY_DELAY_S}s")
-                continue
-            break
-        else:
-            raise last_err
 
         if resp.status_code != 200:
             raise OandaDataError(

static/news_summary.php

@@ -1,245 +0,0 @@
-<?php
-/**
- * On-demand gold-impact AI summary for one published headline.
- *
- * The ONLY dynamic piece of the otherwise-static dashboard. Serves
- * GET ?id=<sha1-of-headline> and returns {"summary": ..., "generated_at": ...}.
- *
- * Abuse model (the page is public):
- *   - WHITELIST: the id must match a headline in the local news.json —
- *     arbitrary text can never reach the LLM through this endpoint.
- *   - CACHE FOREVER: headlines are immutable, so each id costs at most
- *     one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
- *     the whitelist on purpose: summaries stay reachable after the
- *     headline ages out of the 24 h feed window.
- *   - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
- *     errors (a perms mistake must not mean unmetered spend).
- *
- * DeepSeek doctrine (mirrors llm_client.py — deliberate local copy, the
- * dashboard never imports bot code and this file never reads the repo's
- * config.json):
- *   - STATIC byte-identical system prompt (prefix-cache pricing).
- *   - "thinking": disabled is MANDATORY for V4-Flash (else it burns the
- *     timeout reasoning), temperature 0, json response_format.
- *   - One attempt, no retries; every failure returns a JSON error the
- *     page renders as a graceful fallback.
- *
- * Key file lives OUTSIDE the docroot: /etc/gold-dashboard/deepseek.key
- * (root:apache 640). Override paths/caps via Apache SetEnv:
- * DEEPSEEK_KEY_FILE, AI_SUMMARY_DAILY_CAP.
- */
-
-const NEWS_FILE = __DIR__ . '/news.json';
-const CACHE_DIR = __DIR__ . '/ai';   // apache-owned; created at deploy time
-const API_URL   = 'https://api.deepseek.com/chat/completions';
-const MODEL     = 'deepseek-v4-flash';
-const TIMEOUT   = 12;
-const MAX_TOKENS = 200;
-const SUMMARY_MAX_CHARS = 400;
-
-// .ht_ prefix: Apache's default <Files ".ht*"> rule denies these even
-// though they sit in the public docroot.
-const CAP_FILE = CACHE_DIR . '/.ht_cap.json';
-
-const SYSTEM_PROMPT = <<<'PROMPT'
-You are a forex/gold news analyst. You receive one news item as JSON:
-{"headline", "description", "score", "xau_score", "keywords", "currencies"}.
-"xau_score" is the platform's automated gold-effect score for this item,
--1.0 (gold down) to 1.0 (gold up); "score" is overall market sentiment.
-Both are automated and can be wrong — judge the news yourself.
-In 1-2 plain-English sentences (max 45 words) explain what this news
-means for the GOLD (XAU/USD) price and WHY it is bullish, bearish, or
-neutral. If your reading clearly contradicts xau_score, add "(the
-automated score disagrees)". Never write the words "xau_score" or
-"score" — a general reader sees this text.
-Gold doctrine: hawkish Fed / rate hikes / rising real yields / strong
-dollar are gold-bearish; easing / rate cuts / safe-haven flows /
-geopolitical stress / weak dollar are gold-bullish.
-Reply with ONLY JSON: {"summary": "..."}
-PROMPT;
-
-function respond(int $status, array $obj, string $cache = 'no-store'): void
-{
-    http_response_code($status);
-    header('Content-Type: application/json; charset=utf-8');
-    header('Cache-Control: ' . $cache);
-    echo json_encode($obj, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
-    exit;
-}
-
-function atomic_write(string $path, string $data): bool
-{
-    $tmp = $path . '.tmp.' . getmypid();
-    if (@file_put_contents($tmp, $data) === false) {
-        return false;
-    }
-    if (!@rename($tmp, $path)) {
-        @unlink($tmp);
-        return false;
-    }
-    return true;
-}
-
-// ── 1. id validation ────────────────────────────────────────────────
-$id = $_GET['id'] ?? '';
-if (!preg_match('/^[a-f0-9]{40}$/', $id)) {
-    respond(400, ['error' => 'bad_id']);
-}
-
-// ── 2. cache first (immutable headlines → summaries never change) ───
-$cachePath = CACHE_DIR . '/' . $id . '.json';
-$cached = @file_get_contents($cachePath);
-if ($cached !== false && $cached !== '') {
-    http_response_code(200);
-    header('Content-Type: application/json; charset=utf-8');
-    header('Cache-Control: max-age=3600');
-    echo $cached;
-    exit;
-}
-
-// ── 3. whitelist against the published feed ─────────────────────────
-$news = json_decode((string)@file_get_contents(NEWS_FILE), true);
-if (!is_array($news)) {
-    respond(503, ['error' => 'no_feed']);
-}
-$hit = null;
-foreach ($news['headlines'] ?? [] as $h) {
-    // Fallback sha1(): a publisher that predates the id field still
-    // serves a valid whitelist during the restart window.
-    $hid = $h['id'] ?? sha1($h['headline'] ?? '');
-    if ($hid === $id) {
-        $hit = $h;
-        break;
-    }
-}
-if ($hit === null) {
-    respond(404, ['error'   => 'unknown_id',
-                  'message' => 'Headline is no longer in the feed.']);
-}
-
-// ── 4. daily cap (fail closed) ──────────────────────────────────────
-$dailyCap = (int)(getenv('AI_SUMMARY_DAILY_CAP') ?: 150);
-$capFh = @fopen(CAP_FILE, 'c+');
-if ($capFh === false || !flock($capFh, LOCK_EX)) {
-    respond(503, ['error' => 'cap_io']);
-}
-$today = gmdate('Y-m-d');
-$cap = json_decode((string)stream_get_contents($capFh), true);
-if (!is_array($cap) || ($cap['day'] ?? '') !== $today) {
-    $cap = ['day' => $today, 'count' => 0];
-}
-if ($cap['count'] >= $dailyCap) {
-    flock($capFh, LOCK_UN);
-    fclose($capFh);
-    respond(429, ['error' => 'cap_exceeded']);
-}
-// Increment BEFORE the LLM call — attempts count against the cap
-// (same semantics as llm_client._under_cap).
-$cap['count']++;
-rewind($capFh);
-if (ftruncate($capFh, 0) === false
-        || fwrite($capFh, json_encode($cap)) === false) {
-    flock($capFh, LOCK_UN);
-    fclose($capFh);
-    respond(503, ['error' => 'cap_io']);
-}
-fflush($capFh);
-flock($capFh, LOCK_UN);
-fclose($capFh);
-
-// ── 5. API key ──────────────────────────────────────────────────────
-$keyFile = getenv('DEEPSEEK_KEY_FILE') ?: '/etc/gold-dashboard/deepseek.key';
-$apiKey = trim((string)@file_get_contents($keyFile));
-if ($apiKey === '') {
-    respond(503, ['error' => 'no_key']);
-}
-
-// ── 6. per-id stampede lock, then re-check the cache ────────────────
-$lockFh = @fopen(CACHE_DIR . '/.ht_lock_' . $id, 'c');
-if ($lockFh !== false) {
-    flock($lockFh, LOCK_EX);   // blocks while a concurrent winner generates
-    $cached = @file_get_contents($cachePath);
-    if ($cached !== false && $cached !== '') {
-        flock($lockFh, LOCK_UN);
-        fclose($lockFh);
-        http_response_code(200);
-        header('Content-Type: application/json; charset=utf-8');
-        header('Cache-Control: max-age=3600');
-        echo $cached;
-        exit;
-    }
-}
-
-// ── 7. DeepSeek call (exact chat_json body shape) ───────────────────
-$userPayload = json_encode([
-    'headline'    => $hit['headline'] ?? '',
-    'description' => $hit['description'] ?? null,
-    'score'       => $hit['score'] ?? null,
-    'xau_score'   => $hit['score_by_currency']['XAU'] ?? null,
-    'keywords'    => $hit['keywords'] ?? [],
-    'currencies'  => $hit['currencies'] ?? [],
-], JSON_UNESCAPED_UNICODE);
-
-$body = json_encode([
-    'model'    => MODEL,
-    'messages' => [
-        ['role' => 'system', 'content' => SYSTEM_PROMPT],
-        ['role' => 'user',   'content' => $userPayload],
-    ],
-    'response_format' => ['type' => 'json_object'],
-    'temperature'     => 0,
-    'thinking'        => ['type' => 'disabled'],
-    'max_tokens'      => MAX_TOKENS,
-]);
-
-$ch = curl_init(API_URL);
-curl_setopt_array($ch, [
-    CURLOPT_POST           => true,
-    CURLOPT_POSTFIELDS     => $body,
-    CURLOPT_HTTPHEADER     => ['Content-Type: application/json',
-                               'Authorization: Bearer ' . $apiKey],
-    CURLOPT_RETURNTRANSFER => true,
-    CURLOPT_TIMEOUT        => TIMEOUT,
-]);
-$raw = curl_exec($ch);
-$httpCode = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
-curl_close($ch);
-
-// ── 8. parse (any failure → 502, never cached) ──────────────────────
-$summary = null;
-if ($raw !== false && $httpCode === 200) {
-    $resp = json_decode($raw, true);
-    $content = $resp['choices'][0]['message']['content'] ?? '';
-    $parsed = json_decode((string)$content, true);
-    if (is_array($parsed) && is_string($parsed['summary'] ?? null)
-            && trim($parsed['summary']) !== '') {
-        $summary = trim($parsed['summary']);
-        if (mb_strlen($summary) > SUMMARY_MAX_CHARS) {
-            $summary = mb_substr($summary, 0, SUMMARY_MAX_CHARS) . '…';
-        }
-    }
-}
-if ($lockFh !== false && $summary === null) {
-    flock($lockFh, LOCK_UN);
-    fclose($lockFh);
-}
-if ($summary === null) {
-    respond(502, ['error' => 'llm_failed']);
-}
-
-// ── 9. cache + respond ──────────────────────────────────────────────
-$out = json_encode(['summary' => $summary, 'generated_at' => gmdate('c')],
-                   JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
-if (!atomic_write($cachePath, $out)) {
-    // Degrade to per-click generation rather than a broken feature —
-    // the daily cap still bounds spend.
-    error_log('news_summary.php: cache write failed for ' . $cachePath);
-}
-if ($lockFh !== false) {
-    flock($lockFh, LOCK_UN);
-    fclose($lockFh);
-}
-http_response_code(200);
-header('Content-Type: application/json; charset=utf-8');
-header('Cache-Control: max-age=3600');
-echo $out;

test_all.py

@@ -3453,89 +3453,6 @@ def _():
     assert abs(spread - 0.30) < 0.01, f"Expected spread ~0.30, got {spread}"
 
 
-def _make_oanda_provider(responses):
-    """Bare OandaDataProvider whose session pops canned responses in order."""
-    provider = providers.oanda.OandaDataProvider.__new__(
-        providers.oanda.OandaDataProvider)
-    provider.config = PlatformConfig()
-    provider._base_url = "https://test"
-    provider._cache = {}
-    provider._cache_timestamps = {}
-    provider._cache_outputsize = {}
-    provider._cache_lock = threading.Lock()
-    provider._inflight = {}
-    provider._spread_cache = {}
-    queue = list(responses)
-    provider._session = type("S", (), {
-        "get": lambda self, *a, **k: queue.pop(0),
-        "headers": {},
-    })()
-    return provider
-
-
-def _oanda_resp(status_code, candles=None, text=""):
-    return type("MockResp", (), {
-        "status_code": status_code,
-        "text": text,
-        "json": lambda self: {"candles": candles or [], "instrument": "XAU_USD"},
-    })()
-
-
-_OANDA_OK_CANDLE = {
-    "complete": True,
-    "time": "2026-07-20T10:00:00.000000000Z",
-    "mid": {"o": "2400.00", "h": "2405.00", "l": "2398.00", "c": "2403.00"},
-    "bid": {"o": "2399.85", "h": "2404.85", "l": "2397.85", "c": "2402.85"},
-    "ask": {"o": "2400.15", "h": "2405.15", "l": "2398.15", "c": "2403.15"},
-    "volume": 1234,
-}
-
-
-@test("providers.oanda: transient 401 is retried once and recovers")
-def _():
-    provider = _make_oanda_provider([
-        _oanda_resp(401, text='{"errorMessage":"Insufficient authorization"}'),
-        _oanda_resp(200, candles=[_OANDA_OK_CANDLE]),
-    ])
-    orig_sleep = providers.oanda.time.sleep
-    slept = []
-    providers.oanda.time.sleep = lambda s: slept.append(s)
-    try:
-        df = provider._fetch_oanda_candles("XAU/USD", Timeframe.H1, 300)
-    finally:
-        providers.oanda.time.sleep = orig_sleep
-    assert len(df) == 1, f"Expected recovered fetch with 1 bar, got {len(df)}"
-    assert slept == [providers.oanda.OandaDataProvider.RETRY_DELAY_S], \
-        f"Expected one retry sleep, got {slept}"
-
-
-@test("providers.oanda: persistent 401 raises after retry; 4xx never retried")
-def _():
-    orig_sleep = providers.oanda.time.sleep
-    providers.oanda.time.sleep = lambda s: None
-    try:
-        provider = _make_oanda_provider([
-            _oanda_resp(401, text='{"errorMessage":"Insufficient authorization"}'),
-            _oanda_resp(401, text='{"errorMessage":"Insufficient authorization"}'),
-        ])
-        try:
-            provider._fetch_oanda_candles("XAU/USD", Timeframe.H1, 300)
-            assert False, "Persistent 401 should raise OandaDataError"
-        except providers.oanda.OandaDataError as e:
-            assert "401" in str(e)
-
-        # Non-auth 4xx must fail immediately — only one response queued,
-        # so a retry attempt would IndexError on the empty queue.
-        provider = _make_oanda_provider([_oanda_resp(400, text="bad request")])
-        try:
-            provider._fetch_oanda_candles("XAU/USD", Timeframe.H1, 300)
-            assert False, "HTTP 400 should raise OandaDataError"
-        except providers.oanda.OandaDataError as e:
-            assert "400" in str(e)
-    finally:
-        providers.oanda.time.sleep = orig_sleep
-
-
 @test("providers.oanda: instrument name mapping")
 def _():
     provider = providers.oanda.OandaDataProvider.__new__(
@@ -5295,97 +5212,6 @@ def _():
         NewsEngine._source_health = saved
 
 
-@test("NewsEngine: RSS parser captures link/description/publisher")
-def _():
-    cfg = NewsConfig()
-    cfg.rss_feeds = ["https://news.google.com/rss/search?q=gold"]
-    ne = NewsEngine(cfg)
-    saved = NewsEngine._source_health
-    NewsEngine._source_health = {}
-    # Google-News-shaped item: redirect <link>, HTML-escaped <description>,
-    # <source> carrying the real publisher.
-    rss = (b"<rss><channel><item>"
-           b"<title>Gold surges as Fed signals rate cuts - Reuters</title>"
-           b"<link>https://news.google.com/rss/articles/CBMiAbC?oc=5</link>"
-           b"<description>&lt;a href=\"https://x\"&gt;Gold &amp;amp; dollar "
-           b"diverge as the Fed turns dovish.&lt;/a&gt;</description>"
-           b"<source url=\"https://www.reuters.com\">Reuters</source>"
-           b"</item></channel></rss>")
-
-    class _Sess:
-        def get(self, url, timeout=None):
-            return type("R", (), {"status_code": 200, "content": rss})()
-
-    ne._session = _Sess()
-    try:
-        heads = ne._fetch_news_sentiment_raw()
-        assert len(heads) == 1, heads
-        h = heads[0]
-        assert h.link == "https://news.google.com/rss/articles/CBMiAbC?oc=5"
-        # tags stripped, entities (incl. double-encoded &amp;amp;) decoded
-        assert h.description == "Gold & dollar diverge as the Fed turns dovish."
-        assert h.publisher == "Reuters"
-    finally:
-        NewsEngine._source_health = saved
-
-
-@test("NewsEngine: enrichment fields default None / reject unsafe values")
-def _():
-    cfg = NewsConfig()
-    cfg.rss_feeds = ["https://feed.example/rss"]
-    ne = NewsEngine(cfg)
-    saved = NewsEngine._source_health
-    NewsEngine._source_health = {}
-    long_link = b"https://a.example/" + b"x" * 2100
-    rss = (b"<rss><channel>"
-           # bare item: no link/description/source at all
-           b"<item><title>Dollar steady before Fed minutes</title></item>"
-           # javascript: link + description that merely echoes the title
-           b"<item><title>Gold slips on strong dollar data</title>"
-           b"<link>javascript:alert(1)</link>"
-           b"<description>Gold slips on strong dollar data</description></item>"
-           # over-long link
-           b"<item><title>Fed officials split on rate cuts</title>"
-           b"<link>" + long_link + b"</link></item>"
-           b"</channel></rss>")
-
-    class _Sess:
-        def get(self, url, timeout=None):
-            return type("R", (), {"status_code": 200, "content": rss})()
-
-    ne._session = _Sess()
-    try:
-        heads = ne._fetch_news_sentiment_raw()
-        assert len(heads) == 3, heads
-        by_title = {h.headline: h for h in heads}
-        bare = by_title["Dollar steady before Fed minutes"]
-        assert bare.link is None and bare.description is None \
-            and bare.publisher is None
-        js = by_title["Gold slips on strong dollar data"]
-        assert js.link is None          # non-http(s) scheme rejected
-        assert js.description is None   # identical to title → dropped
-        assert by_title["Fed officials split on rate cuts"].link is None
-    finally:
-        NewsEngine._source_health = saved
-
-
-@test("NewsEngine: _clean_html_text strips/unescapes/truncates")
-def _():
-    from news_engine import _clean_html_text, _safe_link
-    assert _clean_html_text(None) is None
-    assert _clean_html_text("") is None
-    assert _clean_html_text("<p></p>") is None
-    assert _clean_html_text("<p>Fed <b>holds</b>\n rates</p>") == "Fed holds rates"
-    assert _clean_html_text("&amp;lt;b&amp;gt;x&amp;lt;/b&amp;gt; &amp; y") == "x & y"
-    long = _clean_html_text("a" * 600)
-    assert len(long) == 501 and long.endswith("…")
-    assert _safe_link("  https://a.example/x ") == "https://a.example/x"
-    assert _safe_link("ftp://a.example/x") is None
-    assert _safe_link("javascript:alert(1)") is None
-    assert _safe_link("https://" + "x" * 2100) is None
-    assert _safe_link(None) is None
-
-
 @test("/status renders news/calendar/AI health with ⚠️ degradations")
 def _():
     import telegram_bot as tb
@@ -5631,45 +5457,6 @@ def _():
     assert payload["headlines"][1]["score"] is None  # NaN → None
     assert payload["events"][0]["event"] == "ISM"
     assert payload["sentiment"] == {"XAU": 0.6, "USD": -0.1}
-    # Every headline carries the 40-hex id the PHP endpoint whitelists on.
-    import re as _re
-    assert all(_re.fullmatch(r"[0-9a-f]{40}", h["id"])
-               for h in payload["headlines"])
-
-
-@test("dashboard: headline_to_dict id matches LLM cache scheme + enrichment")
-def _():
-    import hashlib
-    h = NewsSentiment(
-        headline="Gold surges", source="http://feed/b",
-        timestamp=datetime(2026, 8, 7, 10, 0, tzinfo=timezone.utc),
-        currencies_mentioned=["XAU"], sentiment_score=0.6,
-        keywords=["surge"], sentiment_by_currency={"XAU": 0.6},
-        link="https://x.example/a", description="Body.", publisher="Reuters")
-    d = dpub.headline_to_dict(h)
-    # Same sha1(headline) scheme as NewsEngine._llm_sent_cache — the id is
-    # the join key news_summary.php validates against.
-    assert d["id"] == hashlib.sha1(b"Gold surges").hexdigest()
-    assert d["link"] == "https://x.example/a"
-    assert d["description"] == "Body."
-    assert d["publisher"] == "Reuters"
-    # Pre-enrichment NewsSentiment (defaults) still serializes cleanly.
-    bare = NewsSentiment(
-        headline="Old", source="s", timestamp=None,
-        currencies_mentioned=[], sentiment_score=0.0, keywords=[])
-    d2 = dpub.headline_to_dict(bare)
-    assert d2["link"] is None and d2["description"] is None \
-        and d2["publisher"] is None
-    json.dumps([d, d2], allow_nan=False)
-
-
-@test("dashboard: install_static installs page + PHP endpoint atomically")
-def _():
-    with tempfile.TemporaryDirectory() as tmp:
-        pub = dpub.DashboardPublisher(out_dir=tmp)
-        pub.install_static()
-        assert sorted(os.listdir(tmp)) == \
-            ["dashboard.html", "news_summary.php"]  # both installed, no .tmp
 
 
 @test("dashboard: publish is atomic and leaves no temp file")
@@ -5682,21 +5469,6 @@ def _():
         assert os.listdir(tmp) == ["snapshot.json"]  # no .tmp left
 
 
-@test("dashboard: run_cycle publishes snapshot BEFORE news refresh")
-def _():
-    # News I/O (FF + RSS + DeepSeek) can take 20-30 s — fresh signals
-    # must reach the page first (2026-08-06 sync-latency fix).
-    pub = dpub.DashboardPublisher(out_dir="/nonexistent")
-    order = []
-    pub.scan_tf = lambda tf: order.append(f"scan:{tf.value}")
-    pub.publish_snapshot = lambda: order.append("snapshot")
-    pub.refresh_news = lambda: order.append("news_refresh")
-    pub.publish_news = lambda: order.append("news_publish")
-    pub.run_cycle([Timeframe.M5])
-    assert order == ["scan:5min", "snapshot", "news_refresh",
-                     "news_publish"], order
-
-
 @test("dashboard: seconds_until_boundary bounds and constants")
 def _():
     for interval in (5, 15, 60, 240):