news: DeepSeek validation, vs_forecast fix, /status source health

Validation (llm_news_eval.py harness + 78-headline golden CSV + verdict
doc): headline scoring 78/78 vs keyword baseline 24/78 — validated;
event gold-direction calls misread close K/M numeric comparisons ~11%
of the time (both claims cases in sample, JOLTS 8.79M vs 8.84M).

Fix: event payload now carries code-computed "vs_forecast" (from
EconomicEvent.surprise) and the prompt gains an explicit inverted-
metrics rule + worked claims example — iteration 1 (field alone) fixed
the arithmetic but exposed a masked inversion error. 18/18 on live
re-verification (4 misreads fixed, 11 regression cases clean, 3 fresh
both-side probes).

/status: new "News & AI" section — per-source health for the calendar
chain (which of FF/MyFXBook served), each RSS feed, and DeepSeek
(daily count, last error kind), written passively by the existing
fetch paths under _cache_lock. Failures WARN once on transition
(repeats DEBUG, recovery INFO) so the rotating log carries outage
stories without 10-min spam.

test_all.py: +5 Module 19 tests (351 total); also carries the Module 22
fixed-RR-TP tests from the prior research session (shared file).
Replay artifacts dir gitignored like commit_ladder.

Claude-Session: https://claude.ai/code/session_017mbXs9HRawZhD1uZ3ze7zQ

backtest/research/llm_news_eval_2026-08-06.md

@@ -0,0 +1,86 @@
+# DeepSeek news-analysis validation — 2026-08-06
+
+**Verdict: headline sentiment scoring is excellent; event-direction calls are
+directionally sound but ~11% of them misread the numeric comparison, and the
+failure concentrates on Unemployment Claims / close K-M-suffixed values.**
+
+Method (owner-directed, token-lean): independent expert re-analysis of a
+stratified sample of DeepSeek's recorded answers, not a full statistical
+backtest. Harness: `backtest/research/llm_news_eval.py` (replays releases
+through the REAL `_llm_event_direction` / `_llm_headline_scores` — real
+prompts, payloads, validation; raw wire traffic logged). 135 of the 586-event
+corpus were replayed before the full run was stopped by owner request
+(artifacts retained in `llm_news_eval_2026-08/`, resume-safe).
+
+## Part A — event gold-direction (28-event stratified sample, Jan–Jun 2024)
+
+Reviewer agreement with DeepSeek: **24/28 (86%)**. All 4 disagreements are
+DeepSeek errors, in exactly two failure modes:
+
+1. **Numeric comparison misread (3/28 ≈ 11%)** — the doctrine logic is right
+   but the premise is backwards:
+   - JOLTS 8.79M vs f 8.84M (a miss) → DS: "Job openings **above** forecast" → bearish (should be bullish)
+   - Claims 202K vs f 217K (fewer claims) → DS: "Claims **higher** than forecast" → bullish (should be bearish)
+   - Claims 202K vs f 209K — same backwards read, same wrong call
+   Both Unemployment Claims events in the sample were misread the same way —
+   looks systematic on that event type, possibly an anchoring prior that
+   "claims rose". All misreads involve close values with K/M suffixes.
+2. **Direction field contradicts its own reason (1/28)** — Empire State
+   −43.7 vs f −4.9: reason text says "gold may rise", `gold_direction` says
+   `bearish`.
+
+Where the numbers were read correctly, hawkish/dovish→gold mapping was
+consistently correct (CPI/PPI/NFP/ADP/GDP/PMI/wages/unemployment-rate all
+handled properly, including inverted metrics when read right).
+
+Price cross-check (1h horizon, cluster-confounded, indicative only):
+reviewer calls 22/28 with the subsequent move, DeepSeek 20/28.
+
+## Part B — headline sentiment (78-headline golden set)
+
+DeepSeek matched the reviewer's expected gold-direction label on
+**78/78 (100%)** — including all 8 negation traps ("signals no rate hike",
+"no rate cuts until…") and all 6 inverted-metric phrasings ("claims hit
+two-year high"). The keyword scorer it overrides got **24/78 (31%)**.
+Repeat-batch and shuffled-order probes recorded (headlines_replay.jsonl).
+
+## Implications
+
+- The headline-sentiment override (the main live LLM surface) is validated —
+  a large, measured upgrade over keywords.
+- The event-direction enrichment ("— gold bullish/bearish" appended to
+  surprise descriptions) is right ~85–90% of the time but can state a
+  factually backwards comparison in the displayed reason. It is display-only
+  (never gates signals), so exposure is a possibly-misleading Telegram/CLI
+  line, not trades.
+- **Hardening SHIPPED same day (owner-approved):** the payload now carries a
+  code-computed `vs_forecast` field ("above/below/in line with forecast",
+  from `EconomicEvent.surprise`) and the prompt tells the model to trust it.
+  First iteration fixed the arithmetic but exposed a masked doctrine error
+  (model read "claims below forecast" correctly, then called it a *weaker*
+  labor market — and regressed the unemployment-rate case the same way), so
+  the prompt's inverted-metrics line was expanded to an explicit rule plus a
+  worked claims example. Verification: **18/18** — all 4 original misreads
+  fixed, 11/11 no-regression cases unchanged, 3/3 fresh probes on the side
+  the prompt example does NOT cover (claims 245K vs f 220K → bullish,
+  unemployment 4.1% vs 3.9% → bullish, JOLTS 9.10M vs 8.80M → bearish).
+  Suite green at 346 tests, incl. a new payload assertion in Module 19.
+- Production observations found while building the harness (not fixed):
+  `_llm_event_cache` key omits forecast → a repeated `actual` re-serves a
+  direction computed for a different forecast; a "neutral" reply is
+  indistinguishable from a failure in the return value.
+
+## Reproduce
+
+```
+venv/bin/python backtest/research/llm_news_eval.py events --limit 5   # smoke
+venv/bin/python backtest/research/llm_news_eval.py events --determinism 20
+venv/bin/python backtest/research/llm_news_eval.py headlines
+venv/bin/python backtest/research/llm_news_eval.py report             # offline
+```
+
+Corpus: 586 high-impact USD surprises 2024-01→2026-07 (calendar.parquet,
+NFP-verified UTC); prices: Oanda M15 mids. Full-run cost ≈ cents. The
+`report` subcommand (hit rates vs always-bullish/doctrine/FF-betterWorse
+baselines, McNemar, cluster view) is built and untested at scale — run it
+after completing the replay if a statistical answer is ever wanted.