news corpus for a local scorer + agent panel v2 + dashboard read-only (2026-09-17); books byte-identical

Agent panel v2: personas return p_win anchored to the book base rates with a code-computed decision context; verdict = equal-weight mean (verdict_source "mean"); only the macro persona runs live (agent_panel.personas ["macro"]); prompt version and exact payloads logged; offline replay harness scripts/panel_replay.py; base-rate table agent_panel_base_rates.json (scripts/build_panel_base_rates.py). Backtester `strategy` column now carries the generator's own stamp (label-only; five reference books re-pinned, all 62 other columns identical). News corpus (after evaluating the owner's ML-NEWS scikit model against DeepSeek: golden set 39/78 vs 78/78, out-of-sample agreement 64% against a 90% self-agreement ceiling; verdict: fallback only, not a replacement): - NewsSentiment.xau_score + archive column xau_score keep DeepSeek's gold score for every headline (930 of the first 1,546 DeepSeek rows had none); an existing newsdata.db gains the column in place on first open - headlines naming no tracked currency are scored and archived (currencies_json []) but never returned to the platform - _apply_llm_sentiment sends every new headline in its cycle, 25 per call; a failed call ends the cycle's sending Dashboard publisher never writes a database: main() calls app_config.set_db_writes_for_process(False), honoured by signal_log_enabled() and news_archive_enabled() (owner decision: the bot is the single writer). Test suite: the hermetic app config now points both writers at a suite-private temp dir — the RSS fixture tests were writing fixture headlines into the real logs/newsdata.db — with a Module 32 test pinning it. 529 tests.
2026-09-17 19:49:29 -04:00
parent 183df7832b
commit ad14ea4fed
21 changed files with 3803 additions and 259 deletions
@@ -21,7 +21,7 @@ python -m venv venv && venv/bin/pip install -r requirements.txt   # Python 3.12;
 cp config.json.example config.json                                 # then fill in the keys (see section 5)
 venv/bin/python test_all.py                                        # QA suite, no network; prints its own count
-                                                                   #   (506 tests / 33 modules as of 2026-09-15)
+                                                                   #   (529 tests / 34 modules as of 2026-09-17)
 venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail      # one signal report in the terminal
                                                                   #   --json full dump, --panel adds the advisory LLM panel,
                                                                   #   --timeframe M5 | M15 | H1 | H4
@@ -164,11 +164,20 @@ later generator shipped a setup.
  into the Apache folder.
 Two advisory add-ons are attached **after** the scan by the surface, never inside the signal
-code: the LLM agent panel (`agent_panel.py`, four DeepSeek personas, verdict AGREE / NEUTRAL /
+code: the LLM agent panel (`agent_panel.py`; since 2026-09-17 only the **macro** persona runs by
-OPPOSE) and the violent-move watch (`violent_watch.py`, a "the next hour may be wild" line). No
+default, `agent_panel.personas: ["macro"]`, one DeepSeek call per panel — the technical, risk and
-gate reads either of them.
+contrarian personas carried no information in two live evaluations and stay available for research;
 the persona returns a probability that the trade closes profitably, anchored to the setup class's
 base rate from the validated books, shown as the "AI estimate"; with several personas configured
 the estimate is their equal-weight mean, `verdict_source: "mean"`; the older `macro` stance and
 `chair` synthesizer paths remain for comparison) and the violent-move
 watch (`violent_watch.py`, a "the next hour may be wild" line). No gate reads either of them. The
 live wrapper `analyze_pair` hands the fetched frames to the add-ons through a private
 `report._frames` attribute so the panel's price context sees exactly the bars the engine saw; the
 replayable core never sets it.
-**11. Logging.** `signal_log.py` writes one row per live scan into `logs/signals.db` (signal,
+**11. Logging.** `signal_log.py` writes one row per live scan (from the bot and the CLI; the
 dashboard process never writes either database, section 3) into `logs/signals.db` (signal,
 grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the
 gate that removed a setup, a gate-config fingerprint and an engine fingerprint). The engine
 fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log
@@ -203,7 +212,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 |---|---|---|---|
 | `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`) | pair_signals, signal_aggregator, agent_panel, violent_watch, app_config | dashboard_publisher (formatting helpers) |
 | `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
-| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` |
+| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes `signals.db` or `newsdata.db`: `main()` calls `app_config.set_db_writes_for_process(False)`, a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running) | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` (test-enforced) |
 | `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`) | | copied by the publisher |
 | `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher |
 | `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | |
@@ -236,8 +245,8 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 |---|---|
 | `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; optional DeepSeek enrichment; display and warnings only |
 | `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback |
-| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook |
+| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call |
-| `agent_panel.py` | Advisory persona panel (macro / technical / risk / contrarian + optional chair). Attached post-hoc; the alert verdict is the macro persona by default. With `verdict_source: "chair"`, `_apply_oppose_guard` downgrades a chair OPPOSE to NEUTRAL unless the macro persona opposes, or technical opposes while macro is not supportive. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
+| `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
 | `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |
 ### Logging and evaluation
@@ -249,6 +258,9 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 | `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) |
 | `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) |
 | `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) |
 | `scripts/panel_replay.py` | Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (exact `panel_payloads_json` when present, else from the stored JSON blobs, interned news and the kept Oanda cache), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. `--dry-run`, `--max-calls`, `--limit`, `--out`. Read-only on the DB; 4 DeepSeek calls per event |
 | `scripts/build_panel_base_rates.py` | Builds `agent_panel_base_rates.json` (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin |
 | `agent_panel_base_rates.json` | The base-rate table the panel anchors its probabilities to (built 2026-09-17 from the relabelled W2/W3 books) |
 ### Backtesting and research
@@ -269,7 +281,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 | `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
 | `config.json.example` | The schema of `config.json` (which is gitignored) |
 | `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
-| `test_all.py` | The QA suite (33 modules) |
+| `test_all.py` | The QA suite (34 modules; Module 35 is the agent-panel v2 module) |
 ---
@@ -328,7 +340,7 @@ gates live in `config.py` and the pair factory, never in `config.json`. Blocks (
 | `scanning` | `pairs`, `timeframes` |
 | `pairs."XAU/USD".blocked_hours_utc` | The base list of blocked hours (default 0, 1, 2, 10, 11, 13, 16, 21) |
 | `deepseek` | `api_key`, `model`, `enabled`, `timeout_seconds`, `daily_request_cap` (env `DEEPSEEK_API_KEY`) |
-| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`macro` or `chair`). All defaults = feature off |
+| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`mean`, the default since 2026-09-17; `macro` or `chair` for comparison), `personas` (default `["macro"]`; any of macro / technical / risk / contrarian; `chair` needs two or more). All defaults = feature off |
 | `violent_watch` | `enabled`, `alert_line`, `headsup`, `max_headsup_per_day`, `headsup_min_level` (env `VIOLENT_WATCH_ENABLED`). Off by default |
 | `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
 | `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
@@ -347,7 +359,10 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
 ## 6. Testing
 - `venv/bin/python test_all.py` runs everything with mocked network. Expect
-  `RESULTS: 506 passed, 0 failed, 506 total` (2026-09-15).
+  `RESULTS: 529 passed, 0 failed, 529 total` (2026-09-17). The suite's hermetic app config points
  both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote
  fixture headlines into the real `logs/newsdata.db`, and a Module 32 test now pins the isolation. One Twelve Data interval-map test
  reaches the network and can time out on a slow connection; a rerun clears it.
 - Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
 - When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the
  other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar.
@@ -435,11 +450,18 @@ All of these run offline from the kept cache windows (NumPy CPU backend, 0 cache
 | Book | Window | Trades / R / PF | MD5 of the CSV |
 |---|---|---|---|
-| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `29120a1ee32979431d062f210d41a9d1` |
+| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `49f8e602fc6ae769713c94ef7b257810` |
-| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `d185f099482a07f29502cb5a371a9bbf` |
+| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `b4aa07e33e17e23774c486a6900d673b` |
-| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `6f8f24b49597ced28f50a19e08c3472f` |
+| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `9fd624fb387c36052122bd1830cf9c32` |
-| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `3b45d12bade823178b267c13bc698458` |
+| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `7b848c917dcb5c371ecbb9f8f7292278` |
-| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `e75adae13c839b15736f69c0114e6aa5` |
+| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `186e677fbbd53cca735c94094c23ce05` |
 Re-pinned 2026-09-17 for a **label-only** change: the exported `strategy` column now carries the
 generator's own stamp (`XAU_ATR` / `XAU_PB` / `XAU_CP` / `GENERIC`) instead of a tag inferred
 from the first reason's text, which had labelled every gold ATR setup `GENERIC`. All 62 other
 columns of all five books are byte-identical to the previous pins (`29120a1e…`, `d185f099…`,
 `6f8f24b4…`, `3b45d12b…`, `e75adae1…`); trades, R and PF are unchanged. `--strategies` accepts the
 live names (`XAU` still means `XAU_ATR`).
 Other current-HEAD figures at $0.65: M5 W3 (3-year) 870 / +153.6R / PF 1.50; M15 W3 1273 /
 +186.6R / PF 1.39; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54;
@@ -495,6 +517,15 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
 - `pyarrow` (Parquet) is installed in the dev venv only; live code never touches Parquet.
 - The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped
  every H1 and H4 scan once (2026-08-02).
 - The agent panel's price context reads the frames `analyze_pair` stashes on `report._frames`.
  A report built by hand (scripts, tests) has no frames, so pass them to `_price_context` /
  `run_panel` explicitly or the context carries geometry and base rate only. `scripts/panel_replay.py`
  rebuilds them from the kept Oanda cache; a database copied from live is never migrated (read-only),
  so the replay tolerates missing v2 columns and rebuilds payloads from the stored blobs.
 - The AI estimate is a calibrated base rate, not a filter: on the 37 resolved setups of 2026-09 the
  v2 personas sat within ±2 points of the class win rate (Brier 0.262 vs 0.263 for the base rate
  alone, 0.299 for the old macro verdict) and separated nothing. Do not gate on it; do not iterate
  prompts on that sample again (`backtest/research/panel_v2_2026-09-17.md`).
 ---
@@ -504,6 +535,8 @@ One line per validated change or verdict, newest first. Full evidence in the nam
 | Date | Decision | Where the evidence is |
 |---|---|---|
 | 2026-09-17 | Local news scorer evaluated (the owner's scikit TF-IDF model in `~/Development/ML-NEWS`, trained on 606 DeepSeek gold labels): golden set 39/78 vs DeepSeek 78/78 vs keywords 24/78; out-of-sample agreement with DeepSeek 64% (DeepSeek agrees with itself 90%); cleaning / negation marking / label weighting move nothing; backfilling the 961 missing gold labels lifts macro-headline agreement 38% → 52% but only reshuffles golden-set errors. Verdict: a fallback, not a replacement. SHIPPED to grow a complete corpus: `xau_score` archived for every headline, no-currency headlines scored and archived (never shown), every new headline sent in its cycle (25 per call). Dashboard process made database-read-only (`set_db_writes_for_process`); the bot is the single writer. Books unaffected (news never enters signals) | `~/Development/ML-NEWS/eval/RESULTS_2026-09-17.md` (evaluation scripts live outside the repo by owner instruction) |
 | 2026-09-17 | Live review of two weeks of `signals.db`: live == backtest bar for bar (26/29 trades matched, R within 0.14); the raw advisory −7.15R is cluster re-entries the validated books never take (cooldown view −1.6R, backtester +1.1R on the window, an ordinary fortnight); composite score inversely related to the 24 h move on all timeframes (two-week bear reading, test in-book with `--record-bars`); big-candle gate re-tested and REJECTED again (removes +32R / +60R of net-positive trades). Agent panel v2 SHIPPED: probabilities anchored to book base rates, code-computed decision context, no news doctrine, mean aggregator (`verdict_source: "mean"`), prompt version logged, offline replay harness; owner decision the same day: only the macro persona runs live (`personas: ["macro"]`, 1 call per panel; technical / risk / contrarian carried no information in either evaluation, kept for research); backtester `strategy` tag fixed (label-only, books re-pinned). Delivery findings: dashboard daemon on stale code since Sep 10, bot outage Sep 9 | `backtest/research/signal_log_review_2026-09-17.md`, `backtest/research/panel_v2_2026-09-17.md` |
 | 2026-09-15 | Backtest folder cleanup: `diagnose_filters.py`, `shadow_scan.py`, and the two one-time news scripts removed (last commit with them `931b265`); every pinned CSV, the research output folders, the BTC/ETH cache snapshots, the Tiingo caches and 13 uncited Oanda windows moved to `~/Development/ForexBot2_trash_2026-09-15/` (346 MB) pending the owner's delete; all five reference MD5s reproduced afterwards. Suite 506 / 33 | commit `b0b71e1` |
 | 2026-09-15 | AI market-outlook feature removed (module, hourly Telegram brief, `--outlook`, `outlooks` writer, config block). Legacy config block tolerated with a warning. Books byte-identical | commit `931b265` |
 | 2026-09-13 | Bug audit, 11 delivery/measurement fixes, books byte-identical: NEUTRAL-signal C/B setups now alert (about 17% of the H1 12-month book's R, +3.7R of +22.2R, was never sent); a missing confirmation frame fails the scan; pandas `utcnow` replaced and `pandas<4`, `numpy<3` pinned; advisory outcomes keep locked-profit floors (`GAP_LOCKED`); scan-outcome labeler bounded to 72 h; settlement pause is a scheduled closure; `fetch_range` retries pages and audits gaps; cache-fallback count logged and must be 0; partial `regime_weights_override` raises; the bot scans each bar once and attaches its log handler in `main()` only; the dashboard catches up missed timeframes. Verified sound and unchanged: composite maths, scan-outcome labels, outcome exits, Oanda alignment, no look-ahead | plan file `~/.claude/plans/you-are-python-software-joyful-teacup.md`, commit `2836024` |
@@ -0,0 +1,398 @@
 {
 "schema": "agent-panel-base-rates-v1",
 "generated_at": "2026-09-17T15:26:15+00:00",
 "key": "timeframe|strategy|grade|direction ('*' = any)",
 "min_n_for_exact_class": 30,
 "books": {
  "5min": {
   "file": "m5_w2.csv",
   "md5": "49f8e602fc6ae769713c94ef7b257810",
   "trades": 781,
   "first_entry": "2024-09-06 18:00:00+00:00",
   "last_entry": "2026-09-04 15:15:00+00:00"
  },
  "15min": {
   "file": "m15_w2.csv",
   "md5": "b4aa07e33e17e23774c486a6900d673b",
   "trades": 931,
   "first_entry": "2024-09-05 06:15:00+00:00",
   "last_entry": "2026-09-04 12:45:00+00:00"
  },
  "1h": {
   "file": "h1_w3.csv",
   "md5": "7b848c917dcb5c371ecbb9f8f7292278",
   "trades": 253,
   "first_entry": "2023-09-20 18:00:00+00:00",
   "last_entry": "2026-08-21 07:00:00+00:00"
  },
  "4h": {
   "file": "h4_w3.csv",
   "md5": "186e677fbbd53cca735c94094c23ce05",
   "trades": 114,
   "first_entry": "2023-10-02 08:00:00+00:00",
   "last_entry": "2026-09-01 12:00:00+00:00"
  }
 },
 "classes": {
  "15min|*|*|*": {
   "n": 931,
   "win_rate": 60.47,
   "mean_r": 0.157,
   "tp1_reach_rate": 52.31,
   "median_bars_held": 30,
   "book": "m15_w2.csv"
  },
  "15min|*|*|BUY": {
   "n": 635,
   "win_rate": 63.31,
   "mean_r": 0.2309,
   "tp1_reach_rate": 53.54,
   "median_bars_held": 33,
   "book": "m15_w2.csv"
  },
  "15min|*|*|SELL": {
   "n": 296,
   "win_rate": 54.39,
   "mean_r": -0.0016,
   "tp1_reach_rate": 49.66,
   "median_bars_held": 20,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|A+|BUY": {
   "n": 2,
   "win_rate": 0.0,
   "mean_r": -0.544,
   "tp1_reach_rate": 0.0,
   "median_bars_held": 32,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|A+|SELL": {
   "n": 1,
   "win_rate": 0.0,
   "mean_r": -1.0276,
   "tp1_reach_rate": 0.0,
   "median_bars_held": 34,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|A|BUY": {
   "n": 208,
   "win_rate": 67.31,
   "mean_r": 0.3158,
   "tp1_reach_rate": 59.13,
   "median_bars_held": 44,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|A|SELL": {
   "n": 84,
   "win_rate": 52.38,
   "mean_r": -0.0288,
   "tp1_reach_rate": 42.86,
   "median_bars_held": 43,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|B|BUY": {
   "n": 282,
   "win_rate": 62.41,
   "mean_r": 0.1869,
   "tp1_reach_rate": 46.1,
   "median_bars_held": 38,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|B|SELL": {
   "n": 120,
   "win_rate": 57.5,
   "mean_r": 0.0242,
   "tp1_reach_rate": 45.0,
   "median_bars_held": 27,
   "book": "m15_w2.csv"
  },
  "15min|GENERIC|C|BUY": {
   "n": 60,
   "win_rate": 65.0,
   "mean_r": 0.395,
   "tp1_reach_rate": 60.0,
   "median_bars_held": 24,
   "book": "m15_w2.csv"
  },
  "15min|XAU_ATR|A+|BUY": {
   "n": 3,
   "win_rate": 66.67,
   "mean_r": 0.2684,
   "tp1_reach_rate": 33.33,
   "median_bars_held": 13,
   "book": "m15_w2.csv"
  },
  "15min|XAU_ATR|A+|SELL": {
   "n": 13,
   "win_rate": 69.23,
   "mean_r": 0.4437,
   "tp1_reach_rate": 61.54,
   "median_bars_held": 5,
   "book": "m15_w2.csv"
  },
  "15min|XAU_ATR|A|BUY": {
   "n": 80,
   "win_rate": 56.25,
   "mean_r": 0.06,
   "tp1_reach_rate": 62.5,
   "median_bars_held": 10,
   "book": "m15_w2.csv"
  },
  "15min|XAU_ATR|A|SELL": {
   "n": 78,
   "win_rate": 50.0,
   "mean_r": -0.073,
   "tp1_reach_rate": 62.82,
   "median_bars_held": 6,
   "book": "m15_w2.csv"
  },
  "1h|*|*|*": {
   "n": 253,
   "win_rate": 64.43,
   "mean_r": 0.2768,
   "tp1_reach_rate": 74.31,
   "median_bars_held": 29,
   "book": "h1_w3.csv"
  },
  "1h|*|*|BUY": {
   "n": 253,
   "win_rate": 64.43,
   "mean_r": 0.2768,
   "tp1_reach_rate": 74.31,
   "median_bars_held": 29,
   "book": "h1_w3.csv"
  },
  "1h|GENERIC|A+|BUY": {
   "n": 1,
   "win_rate": 0.0,
   "mean_r": -0.1689,
   "tp1_reach_rate": 100.0,
   "median_bars_held": 45,
   "book": "h1_w3.csv"
  },
  "1h|GENERIC|A|BUY": {
   "n": 57,
   "win_rate": 66.67,
   "mean_r": 0.2243,
   "tp1_reach_rate": 75.44,
   "median_bars_held": 45,
   "book": "h1_w3.csv"
  },
  "1h|GENERIC|B|BUY": {
   "n": 112,
   "win_rate": 63.39,
   "mean_r": 0.2432,
   "tp1_reach_rate": 74.11,
   "median_bars_held": 38,
   "book": "h1_w3.csv"
  },
  "1h|GENERIC|C|BUY": {
   "n": 26,
   "win_rate": 73.08,
   "mean_r": 0.6826,
   "tp1_reach_rate": 80.77,
   "median_bars_held": 29,
   "book": "h1_w3.csv"
  },
  "1h|XAU_ATR|A+|BUY": {
   "n": 1,
   "win_rate": 0.0,
   "mean_r": -0.1181,
   "tp1_reach_rate": 0.0,
   "median_bars_held": 11,
   "book": "h1_w3.csv"
  },
  "1h|XAU_ATR|A|BUY": {
   "n": 56,
   "win_rate": 62.5,
   "mean_r": 0.224,
   "tp1_reach_rate": 71.43,
   "median_bars_held": 8,
   "book": "h1_w3.csv"
  },
  "4h|*|*|*": {
   "n": 114,
   "win_rate": 57.89,
   "mean_r": 0.1902,
   "tp1_reach_rate": 67.54,
   "median_bars_held": 12,
   "book": "h4_w3.csv"
  },
  "4h|*|*|BUY": {
   "n": 94,
   "win_rate": 61.7,
   "mean_r": 0.2409,
   "tp1_reach_rate": 70.21,
   "median_bars_held": 9,
   "book": "h4_w3.csv"
  },
  "4h|*|*|SELL": {
   "n": 20,
   "win_rate": 40.0,
   "mean_r": -0.0484,
   "tp1_reach_rate": 55.0,
   "median_bars_held": 20,
   "book": "h4_w3.csv"
  },
  "4h|GENERIC|A|BUY": {
   "n": 4,
   "win_rate": 75.0,
   "mean_r": 0.9295,
   "tp1_reach_rate": 75.0,
   "median_bars_held": 18,
   "book": "h4_w3.csv"
  },
  "4h|GENERIC|A|SELL": {
   "n": 1,
   "win_rate": 0.0,
   "mean_r": -0.2277,
   "tp1_reach_rate": 100.0,
   "median_bars_held": 29,
   "book": "h4_w3.csv"
  },
  "4h|GENERIC|B|BUY": {
   "n": 39,
   "win_rate": 79.49,
   "mean_r": 0.4706,
   "tp1_reach_rate": 82.05,
   "median_bars_held": 27,
   "book": "h4_w3.csv"
  },
  "4h|GENERIC|B|SELL": {
   "n": 12,
   "win_rate": 25.0,
   "mean_r": -0.4106,
   "tp1_reach_rate": 41.67,
   "median_bars_held": 40,
   "book": "h4_w3.csv"
  },
  "4h|GENERIC|C|BUY": {
   "n": 5,
   "win_rate": 40.0,
   "mean_r": -0.3577,
   "tp1_reach_rate": 60.0,
   "median_bars_held": 8,
   "book": "h4_w3.csv"
  },
  "4h|XAU_ATR|A+|BUY": {
   "n": 4,
   "win_rate": 50.0,
   "mean_r": 0.017,
   "tp1_reach_rate": 50.0,
   "median_bars_held": 4,
   "book": "h4_w3.csv"
  },
  "4h|XAU_ATR|A+|SELL": {
   "n": 2,
   "win_rate": 100.0,
   "mean_r": 0.6889,
   "tp1_reach_rate": 50.0,
   "median_bars_held": 4,
   "book": "h4_w3.csv"
  },
  "4h|XAU_ATR|A|BUY": {
   "n": 42,
   "win_rate": 47.62,
   "mean_r": 0.0547,
   "tp1_reach_rate": 61.9,
   "median_bars_held": 7,
   "book": "h4_w3.csv"
  },
  "4h|XAU_ATR|A|SELL": {
   "n": 5,
   "win_rate": 60.0,
   "mean_r": 0.5618,
   "tp1_reach_rate": 80.0,
   "median_bars_held": 14,
   "book": "h4_w3.csv"
  },
  "5min|*|*|*": {
   "n": 781,
   "win_rate": 52.88,
   "mean_r": 0.187,
   "tp1_reach_rate": 56.34,
   "median_bars_held": 34,
   "book": "m5_w2.csv"
  },
  "5min|*|*|BUY": {
   "n": 683,
   "win_rate": 53.15,
   "mean_r": 0.201,
   "tp1_reach_rate": 54.61,
   "median_bars_held": 33,
   "book": "m5_w2.csv"
  },
  "5min|*|*|SELL": {
   "n": 98,
   "win_rate": 51.02,
   "mean_r": 0.0894,
   "tp1_reach_rate": 68.37,
   "median_bars_held": 44,
   "book": "m5_w2.csv"
  },
  "5min|GENERIC|A|BUY": {
   "n": 139,
   "win_rate": 61.15,
   "mean_r": 0.2662,
   "tp1_reach_rate": 74.82,
   "median_bars_held": 77,
   "book": "m5_w2.csv"
  },
  "5min|GENERIC|B|BUY": {
   "n": 153,
   "win_rate": 50.98,
   "mean_r": 0.2315,
   "tp1_reach_rate": 72.55,
   "median_bars_held": 61,
   "book": "m5_w2.csv"
  },
  "5min|GENERIC|B|SELL": {
   "n": 98,
   "win_rate": 51.02,
   "mean_r": 0.0894,
   "tp1_reach_rate": 68.37,
   "median_bars_held": 44,
   "book": "m5_w2.csv"
  },
  "5min|GENERIC|C|BUY": {
   "n": 21,
   "win_rate": 47.62,
   "mean_r": -0.0144,
   "tp1_reach_rate": 57.14,
   "median_bars_held": 35,
   "book": "m5_w2.csv"
  },
  "5min|XAU_ATR|A+|BUY": {
   "n": 1,
   "win_rate": 0.0,
   "mean_r": -0.0329,
   "tp1_reach_rate": 0.0,
   "median_bars_held": 12,
   "book": "m5_w2.csv"
  },
  "5min|XAU_ATR|A|BUY": {
   "n": 31,
   "win_rate": 54.84,
   "mean_r": 0.288,
   "tp1_reach_rate": 67.74,
   "median_bars_held": 12,
   "book": "m5_w2.csv"
  },
  "5min|XAU_CP|A|BUY": {
   "n": 17,
   "win_rate": 41.18,
   "mean_r": 0.2498,
   "tp1_reach_rate": 29.41,
   "median_bars_held": 26,
   "book": "m5_w2.csv"
  },
  "5min|XAU_CP|B|BUY": {
   "n": 321,
   "win_rate": 51.71,
   "mean_r": 0.1621,
   "tp1_reach_rate": 37.38,
   "median_bars_held": 19,
   "book": "m5_w2.csv"
  }
 }
 }
@@ -794,6 +794,145 @@ def persona_scorecard(events: Sequence[SignalEvent], db_path: str) -> List[dict]
            for (p, s), v in sorted(agg.items())]
 # ── Probability scorecard (agent panel v2, 2026-09-17) ───────────────
 #
 # Scores the panel as a PREDICTOR of the trade closing profitably, per
 # prompt version: Brier score and log-loss per persona and for the mean,
 # calibration by probability band, discrimination (win rate / mean R by
 # band), the constant-voter rate, rank correlation of p with R, and the
 # counterfactual books. v1 rows (stance + confidence) are mapped onto the
 # same 0-100 probability scale with agent_panel.pseudo_probability so the
 # two eras are comparable; nothing here is a decision (owner rule: no
 # persona-weight tuning before ~300 graded outcomes).
 def _p_bands(pairs: Sequence[Tuple[float, float]]) -> Dict[str, dict]:
    from agent_panel import P_OPPOSE, P_SUPPORT
    out = {}
    for label, pick in (("<=40", lambda p: p <= P_OPPOSE),
                        ("40-60", lambda p: P_OPPOSE < p < P_SUPPORT),
                        (">=60", lambda p: p >= P_SUPPORT)):
        sel = [(p, r) for p, r in pairs if pick(p)]
        out[label] = ({"n": len(sel),
                       "mean_p": sum(p for p, _ in sel) / len(sel),
                       "win_rate": sum(1 for _, r in sel if r > 0) / len(sel),
                       "mean_r": sum(r for _, r in sel) / len(sel),
                       "sum_r": sum(r for _, r in sel)}
                      if sel else {"n": 0})
    return out
 def _score_pairs(pairs: Sequence[Tuple[float, float]]) -> Optional[dict]:
    from agent_panel import P_OPPOSE, P_SUPPORT, derive_stance
    if not pairs:
        return None
    ys = [1.0 if r > 0 else 0.0 for _, r in pairs]
    ps = [p / 100.0 for p, _ in pairs]
    eps = 1e-3
    brier = sum((p - y) ** 2 for p, y in zip(ps, ys)) / len(ps)
    ll = -sum(y * math.log(min(1 - eps, max(eps, p)))
              + (1 - y) * math.log(min(1 - eps, max(eps, 1 - p)))
              for p, y in zip(ps, ys)) / len(ps)
    stances = [derive_stance(p) for p, _ in pairs]
    modal = max(set(stances), key=stances.count)
    rho = None
    if len(pairs) >= 5:
        a = pd.Series([p for p, _ in pairs]).rank()
        b = pd.Series([r for _, r in pairs]).rank()
        c = a.corr(b)
        rho = None if c != c else float(c)
    return {"n": len(pairs), "brier": brier, "logloss": ll,
            "mean_p": sum(p for p, _ in pairs) / len(pairs),
            "win_rate": sum(ys) / len(ys),
            "modal": modal, "constant_rate": stances.count(modal) / len(stances),
            "rho_p_r": rho, "bands": _p_bands(pairs),
            "sum_all": sum(r for _, r in pairs),
            "sum_skip_oppose": sum(r for p, r in pairs if p > P_OPPOSE),
            "sum_support_only": sum(r for p, r in pairs if p >= P_SUPPORT)}
 def probability_scorecard(events: Sequence[SignalEvent], db_path: str
                          ) -> Dict[str, Dict[str, Optional[dict]]]:
    """{prompt_version: {"mean": stats, "<persona>": stats, "base_rate": stats}}
    over resolved events (one vote per event, first row with a panel)."""
    from agent_panel import pseudo_probability
    by_row = _row_index(events)
    con = open_readonly(db_path)
    have = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
    cols = ["id", "panel_takes_json"]
    for c in ("panel_probs_json", "panel_prompt_version", "panel_probability",
              "panel_context_json"):
        cols.append(c if c in have else f"NULL AS {c}")
    per_version: Dict[str, Dict[str, List[Tuple[float, float]]]] = {}
    seen: set = set()
    try:
        for rid, takes_json, probs_json, version, p_mean, ctx_json in con.execute(
                f"SELECT {', '.join(cols)} FROM signals"
                " WHERE panel_takes_json IS NOT NULL ORDER BY id"):
            ev = by_row.get(rid)
            if ev is None or ev.r_headline is None:
                continue
            ekey = ev.rows[0].id if ev.rows else rid
            if ekey in seen:
                continue
            seen.add(ekey)
            r = ev.r_headline
            version = version or "v1"
            bucket = per_version.setdefault(version, {})
            probs: Dict[str, float] = {}
            try:
                source = json.loads(probs_json) if probs_json else json.loads(takes_json)
            except (TypeError, ValueError):
                continue
            for t in source or []:
                p = t.get("p_win")
                if p is None and t.get("stance") in ("support", "oppose", "neutral"):
                    p = pseudo_probability(t["stance"], float(t.get("confidence") or 0.0))
                if p is not None:
                    probs[str(t.get("persona"))] = float(p)
                    bucket.setdefault(str(t.get("persona")), []).append((float(p), r))
            if p_mean is not None:
                bucket.setdefault("mean", []).append((float(p_mean), r))
            elif len(probs) >= 3:
                bucket.setdefault("mean", []).append(
                    (sum(probs.values()) / len(probs), r))
            try:
                base = (json.loads(ctx_json) if ctx_json else {}).get("base_rate") or {}
                if base.get("win_rate") is not None:
                    bucket.setdefault("base_rate", []).append((float(base["win_rate"]), r))
            except (TypeError, ValueError, AttributeError):
                pass
    finally:
        con.close()
    return {v: {k: _score_pairs(pairs) for k, pairs in sorted(b.items())}
            for v, b in sorted(per_version.items())}
 def print_probability_scorecard(card: Dict[str, Dict[str, Optional[dict]]]) -> None:
    print("\n── Probability scorecard (Brier: lower is better; 0.25 = coin flip) " + "─" * 6)
    if not card:
        print("  no resolved events with a panel yet")
        return
    for version, preds in card.items():
        print(f"  prompt version {version}")
        print(f"    {'predictor':12s} {'n':>3s} {'Brier':>6s} {'logloss':>7s} {'mean p':>6s} "
              f"{'win%':>5s} {'modal':>8s} {'const%':>6s} {'rho(p,R)':>8s} "
              f"{'R all':>7s} {'skip<=40':>8s} {'>=60only':>8s}")
        for name, st in preds.items():
            if not st:
                continue
            rho = f"{st['rho_p_r']:+8.2f}" if st["rho_p_r"] is not None else "       -"
            print(f"    {name:12s} {st['n']:3d} {st['brier']:6.3f} {st['logloss']:7.3f} "
                  f"{st['mean_p']:6.1f} {100 * st['win_rate']:5.0f} {st['modal']:>8s} "
                  f"{100 * st['constant_rate']:6.0f} {rho} {st['sum_all']:+7.2f} "
                  f"{st['sum_skip_oppose']:+8.2f} {st['sum_support_only']:+8.2f}")
        mean = preds.get("mean")
        if mean:
            cells = "  ".join(
                f"{k}: n={v['n']} p={v['mean_p']:.0f} win={100 * v['win_rate']:.0f}% R={v['mean_r']:+.2f}"
                if v["n"] else f"{k}: n=0" for k, v in mean["bands"].items())
            print(f"    calibration of the mean: {cells}")
 # ── News sentiment lead/lag ───────────────────────────────────────────
@@ -1309,6 +1448,7 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
        for p in persona_scorecard(events, args.db):
            print(f"  {p['persona']:11} {p['stance']:8} n={p['n']:3d} "
                  f"mean {p['mean_r']:+.3f}R  win {100 * p['win_rate']:.0f}%")
        print_probability_scorecard(probability_scorecard(events, args.db))
        if args.news:
            print_news(news_lead_lag(rows, streams),
                       news_extremes(rows, streams),
@@ -105,7 +105,7 @@ _KNOWN_AGENT_PANEL = {
    "enabled", "telegram", "dashboard", "min_quality",
    "max_panels_per_day", "reuse_ttl_seconds",
    "candles_enabled", "candle_bars_primary", "candle_bars_confirm",
-    "verdict_source",
+    "verdict_source", "personas",
 }
 _KNOWN_VIOLENT_WATCH = {"enabled", "alert_line", "headsup",
                        "max_headsup_per_day", "headsup_min_level"}
@@ -330,10 +330,27 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
                    f"config.json: agent_panel.{key} must be an integer "
                    "between 1 and 100")
    if "verdict_source" in panel and panel["verdict_source"] not in \
-            ("macro", "chair"):
+            ("mean", "macro", "chair"):
        raise AppConfigError(
-            'config.json: agent_panel.verdict_source must be "macro" or '
+            'config.json: agent_panel.verdict_source must be "mean", '
-            '"chair"')
+            '"macro" or "chair"')
    if "personas" in panel:
        names = panel["personas"]
        valid = ("macro", "technical", "risk", "contrarian")
        if (not isinstance(names, list) or not names
                or any(not isinstance(n, str) or n not in valid for n in names)
                or len(set(names)) != len(names)):
            raise AppConfigError(
                "config.json: agent_panel.personas must be a non-empty list "
                f"of distinct names from {valid}")
        if panel.get("verdict_source") == "chair" and len(names) < 2:
            raise AppConfigError(
                'config.json: agent_panel.verdict_source "chair" needs at '
                "least two personas")
        if panel.get("verdict_source") == "macro" and "macro" not in names:
            raise AppConfigError(
                'config.json: agent_panel.verdict_source "macro" needs the '
                "macro persona in agent_panel.personas")
    app.agent_panel_settings = dict(panel)
    if "market_outlook" in raw:
@@ -492,10 +509,34 @@ def violent_watch_enabled(app: Optional[AppConfig] = None) -> bool:
    return bool(app.violent_watch_settings.get("enabled", False))
 # ── Process-level database write switch ──────────────────────────────
 #: True in every process by default. The dashboard publisher turns it
 #: off at startup (owner decision 2026-09-17): it is not always running,
 #: so the Telegram bot must be the single writer of signals.db and
 #: newsdata.db — two writers were also two processes contending for the
 #: same WAL files. Every SQLite writer (signal_log, news_archive) reads
 #: the switch through signal_log_enabled() / news_archive_enabled(), so
 #: a switched-off process cannot write whatever config.json or the
 #: environment says. Read-only users of the databases are unaffected.
 _process_db_writes = True
 def set_db_writes_for_process(enabled: bool) -> None:
    global _process_db_writes
    _process_db_writes = bool(enabled)
 def db_writes_enabled() -> bool:
    return _process_db_writes
 # ── Signal-log resolution helpers (env → config.json → default) ──────
 def signal_log_enabled(app: Optional[AppConfig] = None) -> bool:
    """Default ON — signal history is the platform's audit trail."""
    if not _process_db_writes:
        return False
    env = os.environ.get("SIGNAL_LOG_ENABLED")
    if env:
        return env.strip().lower() in ("1", "true", "yes", "on")
@@ -535,6 +576,8 @@ def news_archive_enabled(app: Optional[AppConfig] = None) -> bool:
    scorer and cannot be backfilled once a fetch window has passed. It
    lives in its own DB so a signals.db rotation cannot take it with it
    (which is how three weeks of news were lost on 2026-09-04)."""
    if not _process_db_writes:
        return False
    env = os.environ.get("NEWS_ARCHIVE_ENABLED")
    if env:
        return env.strip().lower() in ("1", "true", "yes", "on")
@@ -1634,16 +1634,16 @@ def _replay_span(span: Tuple[int, int]):
        if report.signal_quality not in bt_cfg.quality_filter:
            continue
-        # Strategy tag (from reasons[0] which the aggregator pre-pends).
+        # Strategy tag: the generator's own stamp (XAU_ATR / XAU_PB /
-        tag = "GENERIC"
+        # XAU_CP / GENERIC), the same string the live log stores in
-        first_bull = report.bullish_reasons[0] if report.bullish_reasons else ""
+        # setup_json.strategy. Until 2026-09-17 this was inferred from the
-        first_bear = report.bearish_reasons[0] if report.bearish_reasons else ""
+        # text of the FIRST reason ("XAU STRATEGY" -> "XAU"), which
-        if "XAU PULLBACK" in first_bull or "XAU PULLBACK" in first_bear:
+        # labelled every gold ATR setup whose first reason was another
-            tag = "XAU_PB"
+        # indicator — and every SELL — as GENERIC (live review §6.3).
-        elif "XAU CHART PATTERN" in first_bull or "XAU CHART PATTERN" in first_bear:
+        # Label-only: entries, exits and R never depended on it, so the
-            tag = "XAU_CP"
+        # books are trade-identical; the CSV `strategy` column changed
-        elif "XAU STRATEGY" in first_bull:
+        # and the reference MD5s were re-pinned with that proof.
-            tag = "XAU"
+        tag = str(getattr(report.trade_setup, "strategy", "") or "GENERIC")
        if bt_cfg.strategies_only and tag not in bt_cfg.strategies_only:
            continue
@@ -2378,7 +2378,9 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
                        "entries/stops/gates unchanged.")
    p.add_argument("--quality", default="A+,A,B,C", help="Comma-separated allowed grades")
    p.add_argument("--strategies", default="",
-                   help="Comma-separated strategy tags to include (XAU,XAU_PB,XAU_CP,GENERIC). Empty = all.")
+                   help="Comma-separated strategy tags to include "
                        "(XAU_ATR,XAU_PB,XAU_CP,GENERIC; the legacy name XAU "
                        "means XAU_ATR). Empty = all.")
    p.add_argument("--cp", action="store_true",
                   help="Force-enable the chart-pattern entry (XAU_CP) regardless "
                        "of the factory default — for validating the candidate.")
@@ -2566,7 +2568,8 @@ def main(argv: Optional[List[str]] = None) -> int:
        max_hold_bars=args.max_hold_bars,
        use_cache=not args.no_cache,
        quality_filter=[q.strip() for q in args.quality.split(",") if q.strip()],
-        strategies_only=[s.strip() for s in args.strategies.split(",") if s.strip()],
+        strategies_only=[("XAU_ATR" if s.strip() == "XAU" else s.strip())
                         for s in args.strategies.split(",") if s.strip()],
        spread=spread,
        workers=args.workers,
        fixed_rr_tp=args.fixed_rr_tp or 0.0,
@@ -0,0 +1,181 @@
 # Agent panel v2 — probabilities anchored to base rates — 2026-09-17
 **Why.** The live review of 2026-09-04 → 09-17 (`signal_log_review_2026-09-17.md`, §5 and §7.1)
 found the panel's verdict was tilted the wrong way: the macro persona's stance was the verdict,
 its only input was headlines (a one-day-lagging echo of price, `news_sentiment_value_2026-09-04.md`),
 opposed setups reached their first target more often than agreed ones (54% vs 42%), three of four
 personas voted the same way on nearly every setup (technical 79% support, risk 79% oppose,
 contrarian 65% oppose), and the stated confidence was uncalibrated. The chair had already been
 shown anti-informative on 2026-09-03. Nothing versioned the prompts, so no change could be measured.
 **What shipped (code, all advisory-only; the panel still never runs inside
 `analyze_from_dataframes` and no gate reads it).**
 | Piece | Where |
 |---|---|
 | Every persona returns `p_win` (0-100, the chance the trade closes profitably under its trailing exit) plus one `reason_for` and one `reason_against`; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose) | `agent_panel.py` `_TAKE_CONTRACT`, `derive_stance`, `_validate_take` (v1 stance+confidence replies still accepted and mapped to a pseudo-probability) |
 | Verdict = equal-weight mean of the personas' `p_win`, quorum 3 of 4, no LLM aggregator (`verdict_source: "mean"`, the new default; `macro` and `chair` kept for comparison) | `mean_verdict`, `run_panel`, `app_config.py` |
 | A code-computed `context` block in every payload: stop / target in ATR and R, cost-to-risk, the class **base rate** from the validated books, prior move 4 h / 24 h in ATR, distance from the 24 h extreme and its age, `move_started_at`, largest bar of the last 3 in ATR, signal-bar body, event timing against the class's expected hold, penalty tags with the book's verdict on them | `_price_context`, `_base_rate`, `agent_panel_base_rates.json` (built by `scripts/build_panel_base_rates.py` from the MD5-verified books) |
 | Macro persona: no news doctrine; judges whether news is NEW relative to `move_started_at`; receives the ATR-normalised candles (as does the contrarian) | `_MACRO_SYSTEM`, `_panel_jobs` |
 | Technical persona: indicator agreement declared uninformative; judges higher-timeframe conflict and key levels. Risk: only setup-specific risks versus the class. Contrarian: the engine's own warnings are not evidence | `_TECH_SYSTEM`, `_RISK_SYSTEM`, `_CONTRARIAN_SYSTEM` |
 | `PANEL_PROMPT_VERSION` logged with every verdict and folded into the payload hash; per-persona probabilities and reasons, the exact per-persona payloads and the context logged (`panel_prompt_version`, `panel_probability`, `panel_probs_json`, `panel_payloads_json`, `panel_context_json`) | `signal_log.py` additive migration, `update_panel`, `log_panel_update` |
 | Live frames handed to the add-ons through `report._frames` in `analyze_pair` (never in the core) | `signal_aggregator.py` |
 | Plain-language "AI estimate" line on Telegram, CLI and dashboard; legacy sources render as before | `telegram_bot._panel_estimate_lines`, `main.py`, `static/dashboard.js` |
 | Scorecard: Brier, log-loss, calibration bands, discrimination, constant-voter rate, ρ(p, R), counterfactual books, split by prompt version | `analyze_signal_log.probability_scorecard` (printed by `--stored-candles`) |
 | Offline replay harness: rebuilds each persona's payload for every resolved logged setup (exact payloads when logged, else from the stored blobs, interned news and the kept Oanda cache, events evaluated as of the signal bar), runs the current prompts, scores them against the outcome and against the logged v1 takes and the base-rate predictor | `scripts/panel_replay.py` (`--dry-run`, `--max-calls`) |
 | Backtester export `strategy` tag fixed to the generator's own stamp (label-only; every gold ATR setup was exported as `GENERIC`) | `backtest/gpu_backtest.py`; books re-pinned, see below |
 Tests: `test_all.py` Module 35 (15 tests) plus the extended Module 23 parity test; suite
 `RESULTS: 521 passed, 0 failed, 521 total`.
 ## Reference books re-pinned (label-only)
 The five books were regenerated with the committed code (stashed) and with the fix, on the same
 cache. Dropping the `strategy` column, every book is byte-identical to its previous pin; trades, R
 and PF are unchanged. New labels: M15 W2 174 `XAU_ATR` / 757 `GENERIC`; M5 W2 32 / 411 / 338
 `XAU_CP`; M15 OOS 1 / 354; H1 W3 57 / 196; H4 W3 53 / 61.
 | Book | old MD5 | new MD5 |
 |---|---|---|
 | M5 W2 | `29120a1ee32979431d062f210d41a9d1` | `49f8e602fc6ae769713c94ef7b257810` |
 | M15 W2 | `d185f099482a07f29502cb5a371a9bbf` | `b4aa07e33e17e23774c486a6900d673b` |
 | M15 OOS | `6f8f24b49597ced28f50a19e08c3472f` | `9fd624fb387c36052122bd1830cf9c32` |
 | H1 W3 | `3b45d12bade823178b267c13bc698458` | `7b848c917dcb5c371ecbb9f8f7292278` |
 | H4 W3 | `e75adae13c839b15736f69c0114e6aa5` | `186e677fbbd53cca735c94094c23ce05` |
 `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all ten runs.
 ## Base-rate table
 `agent_panel_base_rates.json`: 45 classes (34 exact) from the relabelled M5 W2, M15 W2, H1 W3 and
 H4 W3 books; key `timeframe|strategy|grade|direction`, fallbacks `tf|*|*|direction` and `tf|*|*|*`;
 the exact class is used only with n ≥ 30. Examples: M15 all 931 trades win 60.5%, median hold 30
 bars; M15 SELL 296 / 54.4%; M15 `XAU_ATR` A SELL 78 / 50.0%, median hold 6 bars; M5 `XAU_CP` B
 BUY 321 / 51.7%; H1 all 253 / 64.4%.
 ## Pre-registered acceptance (from the approved plan)
 On the resolved logged setups (37 today), a new prompt version ships only if: (1) no persona's
 modal stance is above 65%; (2) Brier ≤ the logged v1 takes and ≤ the base-rate-only predictor;
 (3) the skip-OPPOSE counterfactual does not remove a net-positive bucket. Outcome = advisory R > 0
 at $0.65. Predictors compared on the same events: base-rate only (the class win rate for every
 trade), the logged v1 takes (stance × confidence → pseudo-probability; 27 events carry them), and
 the new prompts run live through DeepSeek on the rebuilt payloads (4 calls per event).
 ## Replay round 1 (draft prompts)
 | predictor | n | Brier | log-loss | mean p | modal stance | constant % | R skip ≤ 40 |
 |---|---|---|---|---|---|---|---|
 | base-rate only | 37 | 0.263 | 0.719 | 54.8 | neutral | 81 | −7.15 (nothing skipped) |
 | v1 logged, mean | 27 | 0.270 | 0.738 | 48.2 | neutral | 48 | −4.00 |
 | v1 logged, macro | 27 | 0.299 | 0.855 | 56.5 | support | 44 | −2.82 |
 | v1 logged, technical | 27 | 0.371 | 1.011 | 71.7 | support | 78 | −6.47 |
 | v1 logged, risk | 27 | 0.343 | 0.947 | 25.7 | oppose | 78 | +0.03 |
 | v2 draft, mean | 37 | 0.257 | 0.707 | 51.3 | neutral | 95 | −7.15 (nothing skipped) |
 Like-for-like on the 27 events with v1 takes: Brier v2 0.257, v1 mean 0.270, v1 macro 0.299,
 base rate 0.259, coin flip 0.250. Rank correlation of p with R: v2 −0.05, v1 mean +0.14, v1 macro
 +0.36, base rate +0.16 (n = 27; the ±0.35 confidence band swallows all of them).
 Reading: the draft is calibrated (it sits 3.5 points under the base rate with a spread of 1.7
 points) and no longer directional, but it carries no rank information: winners averaged p = 50.9,
 losers 51.7. The old macro verdict had some ordering (+0.36) and terrible calibration (82%
 confidence on the three losing FOMC shorts); the new one has calibration and no ordering. Criterion
 (2) passes, (3) is vacuous (no OPPOSE issued), (1) fails **in the opposite sense to the one it was
 written for**: the personas cluster on neutral around the base rate instead of voting a constant
 direction. That is hedging, not bias.
 **A flaw in the draft prompts, found from the per-event reasons.** The five lowest probabilities
 went to the Sep 10 and Sep 14 shorts that won (+1.31, +1.05, +0.70R), because the technical and
 contrarian prompts told the personas to lower p when "the move has already run far" or "a big
 candle just printed". That instruction encoded the live fortnight's pattern, which the two-year
 books contradict (stretched entries are the system's best trades, 2026-08-04; the big-candle gate
 was re-rejected in the review: those buckets are +32R and +60R). The final prompts remove that
 instruction, state the book's finding in the context legend and in both personas, and limit
 downward moves to higher-timeframe conflict, key levels, event timing inside the expected hold,
 news newer than the move, and a stop far wider than the class norm.
 ## Replay round 2 (final prompts, the text that ships)
 Same 37 events, same rebuilt payloads, the final prompt text (the technical and contrarian
 personas told that a long prior move or a big candle is not a reason by itself; downward moves
 limited to higher-timeframe conflict, key levels, event timing, news newer than the move, and a
 stop far wider than the class norm).
 | predictor | n | Brier | log-loss | mean p | modal stance | directional % | R skip ≤ 40 | R ≥ 60 only |
 |---|---|---|---|---|---|---|---|---|
 | base-rate only | 37 | 0.263 | 0.719 | 54.8 | neutral | 19 | −7.15 | −3.07 |
 | v1 logged, mean | 27 | 0.270 | 0.738 | 48.2 | neutral | 52 | −4.00 | −1.66 |
 | v1 logged, macro | 27 | 0.299 | 0.855 | 56.5 | support | 56 | −2.82 | −2.16 |
 | **v2 final, mean** | 37 | **0.262** | 0.717 | 53.8 | neutral | 8 | −7.15 | −1.98 |
 | v2 final, macro | 37 | 0.259 | 0.712 | 53.4 | neutral | 11 | | |
 | v2 final, technical | 37 | 0.270 | 0.734 | 54.2 | neutral | 16 | | |
 | v2 final, risk | 37 | 0.258 | 0.709 | 53.3 | neutral | 11 | | |
 | v2 final, contrarian | 37 | 0.262 | 0.718 | 54.1 | neutral | 11 | | |
 Like-for-like on the 27 events with v1 takes: Brier v2 final 0.260, v1 mean 0.270, v1 macro
 0.299, base rate 0.259. Rank correlation of p with R: v2 final +0.13 (round 1 −0.05), v1 mean
 +0.14, v1 macro +0.36. Calibration of the v2 mean: 34 events in the 40–60 band averaged p = 53
 and won 47%; the 3 events at 60–61 won 1. The final personas sit 1.0 point under the base rate
 (sd 1.8, no event moved 5 points or more); winners averaged p = 53.1, losers 54.3.
 Acceptance: (2) passes — Brier below the logged v1 takes and equal to the base-rate predictor
 (0.262 vs 0.263; 0.260 vs 0.259 on the common 27); (3) passes vacuously (no OPPOSE issued); (1)
 fails as written (92% neutral) and passes as re-specified below (directional share 8%). The
 correction removed the chase penalty: the winning Sep 10 / Sep 14 shorts moved from 45–47 to
 49–51, and the reasons against now name what the books do not already price (a release due in 20
 minutes, price sitting on a support level) instead of "the move already ran far".
 Cost: 2 × 148 DeepSeek calls (both rounds), inside the shared daily cap of 500.
 ## Decision and what it means for the alert
 The shipped verdict is a probability anchored to the validated base rate that moves only for
 facts the books do not already price. On this sample it is calibrated and uninformative: the line
 will usually read "about a 50% chance … (usual for this kind of setup: 55%)". That is the honest
 state of the evidence — the 65-feature sizing model was a clean null too — and it is strictly
 better than the previous line, which asserted 82% confidence in the wrong direction. The
 information the line adds is the base rate itself, the biggest concrete worry (event inside the
 hold, news newer than the move), and each persona's reason. It never gates; the user guide says
 so.
 Re-specified criterion (1) for future prompt versions: the share of *directional* stances
 (support + oppose) on one side must not exceed 65% — neutral clustering around a calibrated base
 rate is not the pathology the criterion targets.
 ## Persona set (owner decision, same day)
 The owner's rule: a persona that does not give correct guidance is removed, to save confusion,
 tokens and log space. Evidence per persona:
 | persona | v1 (27 outcomes) | v2 final (37 outcomes) | verdict |
 |---|---|---|---|
 | macro | Brier 0.299, rank correlation with R **+0.36**, 88% / 25% win split on 2026-09-03 | Brier 0.259, ρ −0.06 | the only voice that ever showed ordering, and the only one holding facts the engine lacks (news) → **kept** |
 | technical | support on 79%, Brier 0.371 (worst) | Brier 0.270 (worst), ρ −0.20 | re-reads the indicators the signal was built from → removed |
 | risk | oppose on 79%, Brier 0.343 | Brier 0.258, ρ −0.02 | its inputs (stop in ATR, cost, event timing) are now computed in the context block → removed |
 | contrarian | oppose on 65%, Brier 0.322 | Brier 0.262, ρ −0.05 | oppose by construction, no information → removed |
 Shipped as `agent_panel.personas` (default `["macro"]`; `app_config` validates the names, `chair`
 needs two or more, `macro` source needs the macro persona). One DeepSeek call per panel instead of
 four (three quarters of the panel's token spend), `panel_payloads_json` about 4 KB instead of about
 14 KB, `panel_takes_json` one entry, no persona votes line in the alert. The mean verdict with one
 persona is that persona's probability (quorum = min(3, personas)). The full panel remains one
 config line away for research replays (`scripts/panel_replay.py --personas macro,technical,risk,contrarian`).
 ## Next evaluation
 - Every live v2 verdict logs its exact payloads, so `scripts/panel_replay.py` can re-score any
  future prompt on identical evidence. At 100 resolved v2 outcomes (about five weeks): the
  `--stored-candles` probability scorecard by prompt version. At 300: fit a probability-band →
  observed-win-rate table on the first half by time, verify on the second, ship it as a numpy
  lookup; only then revisit persona weights (owner rule).
 - Do not iterate prompts on the current 37 events again: any further gain there is in-sample.
 - Deployment: the live box needs the new code (both the bot and the dashboard daemon, which has
  run `2ac7301` since Sep 10); no config change is required (`verdict_source` defaults to
  `mean`); the additive migration runs on first open.
 Commands: `venv/bin/python scripts/build_panel_base_rates.py --book 5min=<m5_w2.csv> --book
 15min=<m15_w2.csv> --book 1h=<h1_w3.csv> --book 4h=<h4_w3.csv> --out agent_panel_base_rates.json`;
 `venv/bin/python scripts/panel_replay.py --db logs/signals.db --dry-run`; `venv/bin/python
 scripts/panel_replay.py --db logs/signals.db --max-calls 160 --out replay.csv`;
 `venv/bin/python analyze_signal_log.py --db logs/signals.db --stored-candles`.
@@ -0,0 +1,483 @@
 # Live signal-log review — 2026-09-17
 **Source evidence:** `logs/signals.db` copied from the live server on 2026-09-17 (rows
 2026-09-04 00:59 → 2026-09-17 13:25 UTC; 5,487 rows, 5,481 `scan` + 6 `failed`, 0 `blocked`;
 telegram 3,458 / dashboard 2,029). Opened read-only. Cross-checked with
 `analyze_signal_log.py --parity | --scan-outcomes | --stored-candles` (all read-only), and with a
 live-vs-backtest parity replay of the same window through `backtest/gpu_backtest.py` at HEAD
 (`5521998`). No tuned value is proposed; this is a health check and a descriptive scorecard.
 ## Headline
 1. **Live equals backtest.** Over 2026-09-06 → 2026-09-17 the validated code produced 29 trades;
   26 of them appear in the live log on the same bar with the same direction, grade and entry,
   and the live advisory R agrees with the backtest R to within 0.14 (mean difference −0.006 over
   23 resolved matches). The 3 backtest-only trades are the Sep 9 bot outage (2) and one M15 bar
   scanned live on Sep 8 with the pre-RSI-change M15 engine (1). All 20 live-only setups are
   consecutive-bar re-entries that fall inside the backtester's 5-bar entry cooldown.
 2. **The raw advisory book reads −7.15R over 37 resolved setups, but that number counts cluster
   re-entries the validated book never takes.** The like-for-like views are flat: cooldown-equivalent
   live book −1.62R over 25 resolved (PF 0.84); backtest over the same window +1.09R over 29 trades
   (M15 −0.32R / 13, M5 +2.43R / 15, H1 −1.01R / 1, H4 0). Against the two-year books, a 12–15 trade
   fortnight summing between −0.7R and +2.4R sits at the 25th–49th percentile: an ordinary fortnight.
   The 12 resolved cooldown re-entries alone cost −5.54R.
 3. **The market was a one-way bear fortnight with an FOMC hike in it:** 4,477 → 4,367 (−2.5%),
   low 4,235 on Sep 16, daily ranges of 12–30 ATR, then +95$ on Sep 17. Every directional statistic
   below is drift-adjusted against the same-timeframe mean move; the raw 24 h drift was −4.2 ATR on M5.
 4. **Direction vs market.** At 1–4 h the composite carried no information this fortnight (Spearman
   rank correlation of score with the 4 h move ≈ 0 on every timeframe). At 24 h it was *inversely*
   related to the move on all three timeframes (M5 ρ = −0.11, M15 −0.20, H1 −0.33, bootstrap CIs
   exclude zero, n = 1,734 / 570 / 150 unique scans): high scores marked bounce tops in the
   downtrend and deep negative scores marked lows that bounced. This is the same pattern the
   2026-09-08 review saw in 16 of 18 episodes, now with n. It is a two-week bear-market reading,
   not a verdict on the two-year edge, and it is the one hypothesis worth a proper in-book test.
 5. **AI panel.** 34 unique setups carried a panel (27 resolved). Skipping the macro persona's OPPOSE
   would have avoided 7 resolved trades worth −2.52R (kept book −2.82R vs −5.35R), but the same
   opposed trades moved *in the trade's favour* at 24 h (drift-adjusted hit 4/5) and 6 further
   OPPOSE verdicts are on the Sep 17 long cluster the backtester currently marks +1.13R. Technical
   (79% support), risk (79% oppose) and contrarian (65% oppose) are near-constant voters. Verdict
   confidence is uncalibrated (62% / 50% / 60% correct across the three bands). No evidence either
   way at this n; keep logging.
 6. **Delivery findings** (no book impact): the dashboard daemon has run stale code (`2ac7301`) since
   Sep 10 13:28 while the bot moved to `2836024` on Sep 14, and it was down from Sep 7 02:20 to
   Sep 10 13:28; the bot was down Sep 9 00:05 → 19:36 UTC (two validated trades missed, +0.37R);
   the live pipeline has no entry cooldown, so consecutive-bar setups are logged, scored and (about
   one in three) re-alerted; the backtester's CSV labels the gold ATR setups `GENERIC` while live
   stamps them `XAU_ATR` (geometry and R identical).
 ## Method
 - Rows deduped to the first row per (timeframe, bar label); `bar_ts` NULL on 365 rows (354 from the
  pre-Sep-7 logging code) falls back to `indicators_json[tf].timestamp`; 11 rows with no indicators
  (price 0) dropped; `scan_kind in ('blocked','failed')` skipped. Unique bars: M5 2,289, M15 763,
  H1 191, H4 53.
 - Setups: 49 unique (M5 24, M15 24, H1 1, H4 0). Outcome = the DB's own advisory model
  (`advisory_r_multiple`, net R at $0.65, next-open entry, stop first, close-only trail): 37 RESOLVED
  (−7.15R, reproduced exactly), 11 PENDING (the Sep 17 long cluster), 1 UNRESOLVABLE.
 - Forward moves: `scan_outcomes` (`fwd_{1h,4h,24h}_atr`, + = up; 24 h only where `complete = 1`).
  "Move" = move in the signal's direction; "excess" = the same after subtracting the same-timeframe
  mean move over all scans (the drift); "adj hit" = share of excess > 0.
 - Three book views: (1) every unique setup; (2) cooldown-equivalent = after an accepted setup, skip
  the next 5 primary bars (the backtester's rule); (3) alerted = the bot's dedup replayed on the
  telegram rows (fingerprint (signal, direction) per timeframe; re-alert after 3,600 s, on a grade
  upgrade, or when the entry moved ≥ 0.5 × last risk; grade ≥ C; setup required; state reset at
  each restart = code-version change). View 3 is a reconstruction; `logs/telegram.log` on the live
  box is the ground truth.
 - Bootstrap null: the M15 and M5 two-year books regenerated offline from the kept cache with the
  section-9 harness (`--start 2024-09-05 --end 2026-09-05 --spread 0.65 --workers 12`), MD5s
  `d185f099482a07f29502cb5a371a9bbf` and `29120a1ee32979431d062f210d41a9d1` (both equal to the
  pinned values; `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0`); 20,000 fixed-n resamples
  of per-trade R plus every 13-calendar-day window of the book.
 - Parity replay: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --spread 0.65 --workers 12
  --primary-tf {15min,5min,1h,4h} --start 2026-09-06 --export-csv ... --detail` (no `--end`, so the
  end is the run time; no `--set`). New cache windows, cited here and therefore kept:
  `oanda_XAU_USD_{5min,15min,1h}_20260706_20260917.parquet`,
  `oanda_XAU_USD_4h_20260115_20260917.parquet`, `oanda_XAU_USD_1d_20221106_20260917.parquet`.
  `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all four runs. The fetch logged two
  interior gaps: M5/M15 2026-09-07 18:15–22:00 (Labor Day, Oanda published no bars — matches the 46
  stale-bar rows live logged) and H4 2026-04-02 → 04-05 (Good Friday). Matching key: (timeframe,
  backtest entry bar − 1 bar) == live bar label.
 - Nothing was written to `signals.db`; `scripts/backfill_signal_outcomes.py` and
  `scripts/label_scan_outcomes.py` were run with `--dry-run` only.
 ## 1. Integrity and coverage
 | Check | Result |
 |---|---|
 | Surface parity (1,953 shared bars) | 0 signal, grade or setup-presence divergences; 8 score differences, none changing a signal or grade: 6 on H1/H4 bars scored by the two code versions (≤ 0.2 pts on H1, up to 7 pts on H4 at Sep 15 00:00: consistent with different confirmation-frame seeding), 2 on the dashboard's Sep 10 13:28 restart catch-up |
 | Code versions live | telegram: c44813f → de36b10 (Sep 7 15:08) → 3afc62c → 28b591d (Sep 8) → 2ac7301 (Sep 9 21:45) → **2836024 (Sep 14 01:46)**. dashboard: c44813f (26 rows, Sep 7 01:00–02:20), then **2ac7301 from Sep 10 13:28 to now** |
 | Dashboard daemon | down Sep 7 02:20 → Sep 10 13:28 (3.5 days); since then on stale code. Still logs 11 "Primary timeframe stale" M5 rows every day at 21:xx UTC (the settlement pause; a scheduled closure in 2836024) |
 | Telegram bot | down Sep 9 00:05 → 19:36 UTC. Cost: M15 BUY B Sep 9 05:45 (+0.23R) and M5 BUY A Sep 9 06:00 (+0.14R) in the backtest, never scanned live |
 | Labor Day (Sep 7) | Oanda published no bars 18:25–22:00 UTC (confirmed by the fresh fetch); 46 stale-bar rows logged; no holiday awareness (known) |
 | Failed scans | 6 rows at 06:11–06:17 UTC on Sep 15 and Sep 16, both surfaces, primary frame missing: an Oanda blip longer than the retry ladder's 45 s budget. 11 pre-ladder "data unavailable" rows on Sep 4/7 |
 | Scan-outcome labelling | 2,467 unique bars complete, 418 labelled but < 72 h old, 408 unlabelled and < 25 h old, 3 unlabelled and older. Last label 13:20 today: the labeler is healthy |
 | Blocked scans | none logged (`log_blocked_scans` off); the Monday 00:xx UTC hours are therefore empty by design |
 | Engine fingerprints | M15 `dab4d03725d8` → `536f84340ea0` at Sep 9 21:45 (RSI × 0.5). M5, H1, H4 unchanged all window |
 | Panel | 35 rows, ≤ 9 per day against the cap of 20, all cache MISS, all `verdict_source = macro` |
 ## 2. Market context
 | UTC day | open | high | low | close | change $ | range / ATR |
 |---|---|---|---|---|---|---|
 | Sep 4 | 4,480 | 4,491 | 4,366 | 4,431 | −49 | 30 |
 | Sep 7 (Labor Day) | 4,424 | 4,430 | 4,381 | 4,423 | 0 | 13 |
 | Sep 8 | 4,423 | 4,443 | 4,345 | 4,350 | −73 | 20 |
 | Sep 9 (bot down) | 4,350 | 4,418 | 4,347 | 4,394 | +44 | 24 |
 | Sep 10 | 4,394 | 4,435 | 4,311 | 4,315 | −79 | 25 |
 | Sep 11 | 4,315 | 4,403 | 4,292 | 4,349 | +33 | 20 |
 | Sep 14 | 4,333 | 4,355 | 4,254 | 4,288 | −45 | 21 |
 | Sep 15 | 4,288 | 4,318 | 4,261 | 4,284 | −5 | 14 |
 | Sep 16 (FOMC hike 18:00) | 4,284 | 4,367 | 4,235 | 4,272 | −12 | 30 |
 | Sep 17 (to 13:20) | 4,272 | 4,381 | 4,267 | 4,367 | +95 | 23 |
 Drift baseline (mean forward move over **all** unique scans, ATR units, + = up):
 | tf | 1 h | 4 h | 24 h |
 |---|---|---|---|
 | M5 | −0.23 (n 1,992) | −0.63 (1,998) | −4.21 (1,734) |
 | M15 | −0.09 (661) | −0.36 (667) | −2.31 (570) |
 | H1 | −0.06 (161) | −0.22 (167) | −0.75 (150) |
 ## 3. Direction and score against the realised move (all unique scans)
 Move is measured in the signal's direction; excess is after removing the drift above.
 | tf | direction | n | 1 h move / excess | 4 h move / excess / hit | 24 h move / excess / adj hit |
 |---|---|---|---|---|---|
 | M5 | LONG | 18 | −1.10 / −0.87 | −1.27 / −0.64 / 39% | −6.83 / −2.62 / 47% |
 | M5 | SHORT | 66 | +0.35 / +0.12 | +0.14 / −0.49 / 53% | −1.13 / −5.34 / 31% |
 | M15 | LONG | 7 | −0.81 / −0.72 | −0.31 / +0.05 / 43% | −4.03 / −1.72 / 33% |
 | M15 | SHORT | 39 | +0.27 / +0.18 | +0.60 / +0.23 / 69% | −0.68 / −2.98 / 30% |
 | H1 | SHORT | 4 | −0.25 / −0.31 | −0.66 / −0.88 / 50% | −2.05 / −2.80 / 0% |
 Reading: M15 shorts were right for about four hours (69% hit, +0.23 ATR excess) and wrong by the
 next day (the bear-trend lows they sold bounced). Longs were wrong at every horizon. Nothing here
 beats drift at 24 h.
 Score buckets (direction = sign of score, includes NEUTRAL scans):
 | tf | bucket | n | 4 h move / excess | 24 h move / excess / adj hit |
 |---|---|---|---|---|
 | M5 | ≥ 35 | 22 | −1.83 / −1.19 | −13.06 / −8.85 / 18% |
 | M5 | 25..35 | 47 | −1.11 / −0.48 | −9.61 / −5.40 / 28% |
 | M5 | −35..−25 | 101 | −0.95 / −1.58 | +0.11 / −4.10 / 31% |
 | M5 | ≤ −35 | 41 | −2.06 / −2.69 | −0.15 / −4.36 / 25% |
 | M15 | 15..25 | 56 | −0.68 / −0.31 | −5.15 / −2.85 / 35% |
 | M15 | 25..35 | 17 | −0.13 / +0.23 | −3.20 / −0.90 / 44% |
 | M15 | −35..−25 | 37 | +0.65 / +0.28 | +0.42 / −1.88 / 38% |
 | M15 | ≤ −35 | 24 | +0.47 / +0.10 | −0.98 / −3.28 / 29% |
 Spearman rank correlation of the composite score with the forward move (+ would mean high score →
 price up), with bootstrap 95% CI:
 | tf | 1 h | 4 h | 24 h |
 |---|---|---|---|
 | M5 (n ≈ 1,990 / 1,734) | −0.05 [−0.09, −0.01] | −0.03 [−0.07, +0.02] | **−0.11 [−0.15, −0.06]** |
 | M15 (n ≈ 665 / 570) | 0.00 [−0.08, +0.08] | −0.02 [−0.09, +0.06] | **−0.20 [−0.27, −0.12]** |
 | H1 (n ≈ 165 / 150) | −0.09 [−0.25, +0.06] | −0.08 [−0.24, +0.07] | **−0.33 [−0.47, −0.17]** |
 Grades: M15 grade-A rows (n = 12) had a 24 h excess of −4.8 ATR and an adjusted hit of 17%; M5
 grade-B rows (n = 15) −1.9 ATR excess at 4 h, hit 20%. Gates: the only gate that removed A/B/C
 directional rows this fortnight was the spread gate on M5 (n = 10; 24 h excess +0.03 vs −5.33 for
 the shipped M5 setups): tiny and descriptive. The M15 engine change at Sep 9 21:45 leaves only 10
 directional rows before it; nothing can be said about it.
 ## 4. The trade book
 View 1, every unique setup (advisory R at $0.65; PENDING excluded from R):
 | cut | n | resolved | wins | sum R | mean R | PF |
 |---|---|---|---|---|---|---|
 | **all** | 49 | 37 | 17 (46%) | **−7.15** | −0.19 | 0.60 |
 | M5 | 24 | 14 | 6 | −1.08 | −0.08 | 0.83 |
 | M15 | 24 | 22 | 11 | −5.06 | −0.23 | 0.51 |
 | H1 | 1 | 1 | 0 | −1.01 | | |
 | BUY / SELL | 30 / 19 | 18 / 19 | 8 / 9 | −3.49 / −3.67 | −0.19 / −0.19 | 0.59 / 0.61 |
 | grade A / B / C | 27 / 19 / 2 | 20 / 15 / 2 | 9 / 7 / 1 | −4.87 / −2.75 / +0.47 | | 0.48 / 0.63 / 1.46 |
 | XAU_ATR / GENERIC / XAU_CP | 10 / 35 / 4 | 9 / 25 / 3 | 5 / 10 / 2 | −1.29 / −7.03 / +1.17 | | 0.69 / 0.45 / 2.10 |
 | regime transitioning / trending_down / trending_up | 12 / 15 / 16 | 11 / 15 / 6 | 4 / 8 / 2 | −6.02 / −0.70 / −2.06 | | 0.16 / 0.89 / 0.08 |
 | volatility extreme / normal | 31 / 9 | 22 / 8 | 10 / 4 | −4.32 / +0.94 | | 0.59 / 1.30 |
 | watch level none / watch / hot | 10 / 24 / 12 | 10 / 20 / 5 | 6 / 7 / 4 | +0.57 / −7.73 / +2.11 | | 1.18 / 0.38 / 10.5 |
 | hour 08 UTC | 7 | 7 | 0 | −7.14 | −1.02 | 0.00 |
 | hour 12 UTC | 13 | 6 | 5 | +4.15 | +0.69 | 19.7 |
 | day Sep 14 / Sep 15 / Sep 16 | 4 / 6 / 11 | 4 / 6 / 11 | 4 / 1 / 4 | +3.40 / −5.01 / −4.53 | | |
 Exits: 17 SL, 18 TRAIL, 2 GAP_LOCKED. Setup-level forward moves agree with the advisory model:
 M15 setups −0.60 ATR excess at 4 h (hit 35%), −3.33 at 24 h; M5 −1.49 at 4 h, −5.33 at 24 h.
 View 2, cooldown-equivalent (skip 5 primary bars after an accepted setup): 28 setups, 25 resolved,
 14 wins (56%), **−1.62R**, PF 0.84; M5 +0.11R (12), M15 −0.71R (12), H1 −1.01R; SELL +1.11R over 10
 (70% win), BUY −2.73R over 15.
 View 3, reconstructed Telegram alerts: 36 alerts in 9 trading days (about 4 a day), 29 resolved, 14
 wins (48%), **−4.88R**, PF 0.64; M15 −3.97R (16), M5 +0.11R (12), H1 −1.01R. The re-alert rules let
 through 2 of the 6 Sep 15 shorts and all 3 FOMC-evening shorts.
 Backtest over the same window (HEAD): **29 trades, +1.09R**. Matched-resolved 23: live +0.47R vs
 backtest +0.62R (the difference is the Sep 8 07:30 M15 short, GAP_LOCKED live at +1.13 vs +1.28
 on the raw path). Matched but live still pending 3: backtest +1.13R. Backtest-only 3: −0.66R.
 Live-only cooldown re-entries 20 (8 pending): the 12 resolved ones sum to **−5.54R**.
 Bootstrap against the two-year books (per-trade R resampled; P = share of draws at or below the
 observed sum):
 | view | tf | n | observed | book median | 5–95% | P(≤ obs) |
 |---|---|---|---|---|---|---|
 | view 1 (raw advisory) | M15 | 22 | −5.06R | +3.2 | −4.8..+12.0 | 4.6% |
 | view 1 | M5 | 14 | −1.08R | +2.4 | −4.2..+10.3 | 20% |
 | view 2 (cooldown) | M15 | 12 | −0.71R | +1.8 | −4.1..+8.3 | 25% |
 | view 2 | M5 | 12 | +0.11R | +2.0 | −4.0..+9.3 | 31% |
 | backtest over window | M15 | 13 | −0.32R | +1.9 | −4.2..+8.7 | 28% |
 | backtest over window | M5 | 15 | +2.43R | +2.6 | −4.3..+10.7 | 49% |
 Calendar check: 36% of the M15 book's 13-day windows and 35% of the M5 book's are negative; among
 M5 windows where price fell 3% or more the median is +0.2R with 46% negative, median BUY −1.2R and
 median SELL +1.5R. This fortnight behaved the way the book says a falling fortnight behaves. The
 view-1 M15 figure looks extreme only because its 22 "trades" are 12 independent entries plus 10
 correlated re-entries.
 Special buckets:
 - **M15 grade-A/A+ SELL** (the parked cycle-2 candidate): live 11 setups, 5 wins, −2.77R (PF 0.55),
  but 6 of the 11 are Sep 15 cluster re-entries; the cooldown-equivalent five are +1.13, +0.36, +1.05,
  +0.09, −1.02 = +1.61R. Two-year book: 176 trades, −3.4R, mean −0.02R. The live sample does not
  move the candidate either way; it stays parked.
 - **FOMC evening** (Sep 16 18:45, 19:00, 19:15 UTC): three M15 shorts, all −1R, 45–75 minutes after
  the 18:00 decision, all `watch_level = watch` (reading `extreme`), none `hot` because the hot rule
  needs a high-impact USD release within ±30 min. The macro persona AGREED on all three at 82%
  confidence ("the Fed raised rates, which pushes gold down"); gold rose 95$ the next day. The
  backtester took the 18:45 trade too (−1.02R), so it is validated behaviour; 19:00 and 19:15 are
  cooldown re-entries.
 - **Sep 15 07:45–08:45 M15 STRONG_SELL cluster**: six setups on consecutive bars, one +0.09R and
  five −1.02R (−5.01R). The backtester took the first (+0.09R) and skipped the rest.
 - **XAU_CP**: 4 setups (2 inverse head-and-shoulders, 2 double bottoms), 3 resolved, +1.17R; both
  in-window ones matched the backtester exactly.
 - **Exhaustion count at entry**: 0 → +2.57R (4 resolved), 1 → −4.85R (10), 3 → +0.26R (8),
  4 → −3.24R (11). Descriptive only; the 2026-09-06 campaign already closed exhaustion sizing.
 ## 5. The AI panel
 Unique setups with a panel 34 (27 resolved); macro persona is the verdict.
 | verdict | n | resolved | wins | sum R | PF | 4 h excess / adj hit | 24 h excess / adj hit |
 |---|---|---|---|---|---|---|---|
 | AGREE | 12 | 12 | 6 (50%) | −2.16 | 0.59 | −2.51 / 22% (n 9) | −7.13 / 22% (n 9) |
 | NEUTRAL | 9 | 8 | 5 (62%) | −0.67 | 0.79 | +1.48 / 75% (n 8) | −2.31 / 29% (n 7) |
 | OPPOSE | 13 | 7 | 2 (29%) | −2.52 | 0.39 | −0.72 / 20% (n 5) | +0.69 / 80% (n 5) |
 | skip OPPOSE (counterfactual) | 21 | 20 | 11 (55%) | −2.82 | 0.66 | | |
 | AGREE only | 12 | 12 | 6 (50%) | −2.16 | 0.59 | | |
 By realised R the OPPOSE verdict was mildly useful (it flagged the two worst-PF groups: opposed
 BUYs −1.61R over 4, opposed SELLs −0.91R over 3). By forward move the same opposed trades went the
 trade's way at 24 h (4 of 5 after drift). Six pending OPPOSE verdicts sit on the Sep 17 long
 cluster, which the backtester currently marks +0.90, +0.38, −0.15R. Nothing here is decidable at
 n = 7; it is consistent with the 2026-09-03 finding that only the macro persona carries information.
 Personas (stance share / mean R when they held it): macro support 35% (−0.18R) / neutral 26%
 (−0.08R) / oppose 38% (−0.36R); technical support **79%** / oppose 12%; risk oppose **79%** /
 support 3%; contrarian oppose **65%**. Three of four personas vote almost the same way on every
 setup, so they add little beyond their prior. "Take only setups the risk persona did not oppose"
 leaves 7 setups (+0.03R): a curiosity, not a rule. Number of opposing personas (1 / 2 / 3) gives
 PF 0.57 / 0.57 / 0.57. Verdict confidence bands < 40 / 40–59 / ≥ 60 were 62% / 50% / 60% correct
 (n 8 / 4 / 15): no calibration.
 ## 6. Findings, tagged
 **Observations (small sample, descriptive):**
 - The system is doing what was validated: bar-for-bar parity with the backtester, and the
  backtester itself made +1.1R on this window, a 30th-percentile fortnight for the book.
 - The live loss is concentrated in cluster re-entries (−5.5R), the Sep 15 08:xx short run and the
  FOMC-evening shorts, and in the `transitioning` regime (PF 0.16 over 11) and `watch` level
  (−7.7R over 20). Two weeks; no action.
 - The composite score was inversely related to the 24 h move on all three timeframes with CIs
  excluding zero, and uninformative at 1–4 h. Two weeks of a single bear regime cannot overturn the
  two-year book, but it is the strongest live pattern seen twice now (2026-09-08 and here).
 - Panel: OPPOSE mildly helpful by R, anti-informative by 24 h move; three personas near-constant.
 **Delivery-side fix candidates (no book impact):**
 1. Redeploy the dashboard daemon to `2836024` (or later) and install the systemd unit; it has run
   `2ac7301` since Sep 10 and was dead for 3.5 days before that. After redeploy, re-check the 8 score
   divergences between surfaces on H1/H4 bars.
 2. Decide whether the live pipeline should mirror the backtester's 5-bar entry cooldown for
   alerts and for the advisory scorecard. Today every consecutive-bar setup is logged as its own
   trade and about one in three is re-alerted, so the live book systematically over-counts
   correlated entries relative to every validated number. A cooldown view could be added to
   `analyze_signal_log.py` (the existing `one_position_view` is close), and an alert cooldown is
   the owner's call.
 3. Backtester CSV `strategy` column: the gold ATR setups appear as `GENERIC` while live stamps
   `XAU_ATR`; entries, stops and R are identical, so this is a labelling gap only. Cause: the
   backtester derives its tag from the text of the first bullish/bearish reason (`"XAU STRATEGY"`
   in the first bullish reason → `XAU`) instead of reading `trade_setup.strategy`, so SELL ATR
   setups and any ATR setup whose first reason is another indicator fall through to `GENERIC`.
   The same tag feeds the research-only `--strategies` filter. Book-neutral; a one-line fix plus
   a test when convenient.
 4. Bot uptime: the 19.5 h Sep 9 outage cost two validated trades; the daily 21:xx stale rows and
   the Labor Day stale run are known and harmless.
 **Needs a rule-1 backtest (nothing proposed to ship):**
 - Score-vs-24 h inversion: test in-book with the replay recorder (`--record --record-bars` writes
  forward 1 h / 4 h / 24 h moves for every analysed bar over the two-year window). If the rank
  correlation is also negative across the book, the gate chain's thresholds, not the weights, are
  the place to look; if it is positive, this fortnight was regime-specific.
 - M15 grade-A SELL block stays parked (cycle-2 candidate; live adds n = 5 cooldown-equivalent
  trades at +1.6R and n = 11 raw at −2.8R).
 ## Caveats
 Section 7 adds four scratchpad research runs of gate 6h on the two-year cache (no new cache
 windows; `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all four).
 Nine trading days, one regime (bear trend with an FOMC hike), 37 resolved advisory outcomes of
 which 12 are correlated re-entries, the advisory model is cleaned-mid evidence and not execution,
 the 24 h horizon is drift-dominated (drift-adjusted numbers are shown but a single trend cannot be
 fully removed), the M15 engine changed mid-sample (Sep 9 21:45), the dashboard and the bot ran
 different code from Sep 14, and the alerted view is a reconstruction. The `--stored-candles`
 replay closes only 26 of the 37 DB-resolved setups because the stored candle streams have the
 Labor Day and Sep 9 gaps; where both resolve they agree to the cent.
 ## Appendix: the 49 unique setups
 R at $0.65 (advisory model); ex24 = 24 h move in the trade direction minus the same-timeframe drift,
 ATR units; BT = backtester over the same window (M = matched, C = live-only inside the cooldown,
 – = before the parity window).
 | bar (UTC) | tf | signal | g | score | dir | strategy | regime | R | exit | ex24 | panel | BT |
 |---|---|---|---|---|---|---|---|---|---|---|---|---|
 | 09-04 05:35 | M5 | BUY | B | +8.7 | BUY | XAU_CP | ranging | −1.06 | SL | −7.4 | NEUTRAL | – |
 | 09-04 07:45 | M15 | STRONG_BUY | A | +37.3 | BUY | XAU_ATR | transitioning | −1.04 | SL | −5.1 | NEUTRAL | – |
 | 09-04 15:10 | M5 | BUY | B | +8.8 | BUY | XAU_CP | ranging | unres. | | +3.1 | OPPOSE | – |
 | 09-07 08:00 | M15 | BUY | C | +26.7 | BUY | GENERIC | ranging | −1.02 | SL | | | M (bt grade B, pre-RSI engine) |
 | 09-07 12:15 | M5 | BUY | B | +22.2 | BUY | XAU_CP | trending | +2.04 | TRAIL | | NEUTRAL | M |
 | 09-08 07:30 | M15 | SELL | A | −34.2 | SELL | GENERIC | trending_down | +1.13 | GAP_LOCKED | +3.0 | OPPOSE | M (bt +1.28) |
 | 09-08 07:45 | M15 | SELL | B | −31.7 | SELL | GENERIC | trending_down | +0.84 | GAP_LOCKED | +2.7 | | C |
 | 09-08 08:05 | M5 | SELL | B | −35.4 | SELL | GENERIC | trending_down | −1.03 | SL | +3.7 | OPPOSE | M |
 | 09-10 05:10 | M5 | BUY | B | +20.5 | BUY | XAU_CP | ranging | +0.18 | TRAIL | −18.3 | AGREE | M |
 | 09-10 05:40 | M5 | BUY | A | +33.4 | BUY | GENERIC | trending_up | −1.03 | SL | −15.7 | AGREE | M |
 | 09-10 05:55 | M5 | BUY | B | +29.8 | BUY | GENERIC | trending_up | −1.02 | SL | −15.2 | | C |
 | 09-10 09:15 | M15 | SELL | B | −29.1 | SELL | GENERIC | transitioning | +0.07 | TRAIL | +4.2 | NEUTRAL | M |
 | 09-10 12:05 | M5 | SELL | B | −30.3 | SELL | GENERIC | trending_down | +1.31 | TRAIL | +3.6 | AGREE | M |
 | 09-10 12:15 | M15 | SELL | A | −34.3 | SELL | GENERIC | trending_down | +0.36 | TRAIL | +0.3 | AGREE | M |
 | 09-10 12:30 | M15 | SELL | B | −32.6 | SELL | GENERIC | trending_down | −0.22 | TRAIL | −3.4 | AGREE | C |
 | 09-14 09:15 | M15 | STRONG_SELL | A | −42.7 | SELL | XAU_ATR | trending_down | +1.05 | TRAIL | −0.1 | AGREE | M |
 | 09-14 09:30 | M15 | STRONG_SELL | A | −42.6 | SELL | XAU_ATR | trending_down | +0.70 | TRAIL | −1.0 | | C |
 | 09-14 12:45 | M5 | SELL | B | −36.5 | SELL | GENERIC | trending_down | +0.16 | TRAIL | −6.1 | AGREE | M |
 | 09-14 15:25 | M5 | BUY | C | +25.1 | BUY | GENERIC | ranging | +1.49 | TRAIL | +4.3 | | M |
 | 09-15 07:45 | M15 | STRONG_SELL | A | −37.5 | SELL | GENERIC | trending_down | +0.09 | TRAIL | −8.7 | AGREE | M |
 | 09-15 08:00 | M15 | STRONG_SELL | A | −42.0 | SELL | GENERIC | trending_down | −1.02 | SL | −9.5 | OPPOSE | C |
 | 09-15 08:10 | M5 | SELL | B | −30.3 | SELL | GENERIC | trending_down | −1.02 | SL | −15.6 | AGREE | M |
 | 09-15 08:15 | M15 | STRONG_SELL | A | −36.7 | SELL | GENERIC | trending_down | −1.02 | SL | −9.6 | | C |
 | 09-15 08:30 | M15 | STRONG_SELL | A | −41.6 | SELL | GENERIC | trending_down | −1.02 | SL | −9.7 | | C |
 | 09-15 08:45 | M15 | STRONG_SELL | A | −45.1 | SELL | GENERIC | trending_down | −1.02 | SL | −10.5 | | C |
 | 09-16 03:00 | M5 | STRONG_BUY | A | +36.0 | BUY | GENERIC | trending_up | +0.14 | TRAIL | −4.2 | NEUTRAL | M |
 | 09-16 03:00 | M15 | BUY | A | +34.0 | BUY | XAU_ATR | transitioning | +0.14 | TRAIL | −3.5 | NEUTRAL | M |
 | 09-16 03:10 | M5 | STRONG_BUY | A | +38.6 | BUY | GENERIC | trending_up | −0.16 | TRAIL | −4.6 | | C |
 | 09-16 03:15 | M15 | BUY | A | +30.9 | BUY | XAU_ATR | transitioning | +0.43 | TRAIL | −2.8 | | C |
 | 09-16 06:00 | H1 | BUY | B | +28.0 | BUY | GENERIC | transitioning | −1.01 | SL | −0.7 | NEUTRAL | M |
 | 09-16 07:00 | M15 | BUY | B | +29.1 | BUY | GENERIC | trending_up | +0.05 | TRAIL | +0.5 | NEUTRAL | M |
 | 09-16 07:35 | M5 | STRONG_BUY | A | +37.3 | BUY | GENERIC | trending_up | −0.04 | TRAIL | +3.2 | OPPOSE | M |
 | 09-16 15:05 | M5 | BUY | B | +29.7 | BUY | GENERIC | transitioning | −1.04 | SL | | OPPOSE | M |
 | 09-16 18:45 | M15 | SELL | A | −34.6 | SELL | XAU_ATR | transitioning | −1.02 | SL | | AGREE | M |
 | 09-16 19:00 | M15 | STRONG_SELL | A | −35.4 | SELL | XAU_ATR | transitioning | −1.02 | SL | | AGREE | C |
 | 09-16 19:15 | M15 | SELL | B | −26.6 | SELL | GENERIC | transitioning | −1.01 | SL | | AGREE | C |
 | 09-17 07:15 | M15 | BUY | A | +31.5 | BUY | XAU_ATR | transitioning | −1.03 | SL | | OPPOSE | M |
 | 09-17 07:25 | M5 | STRONG_BUY | A | +35.9 | BUY | GENERIC | trending_up | pending | | | OPPOSE | M (bt +0.90) |
 | 09-17 07:30 | M5 | STRONG_BUY | A | +36.2 | BUY | GENERIC | trending_up | pending | | | | C |
 | 09-17 07:30 | M15 | BUY | B | +28.7 | BUY | GENERIC | transitioning | pending | | | | C |
 | 09-17 07:35 | M5 | STRONG_BUY | A | +35.5 | BUY | GENERIC | trending_up | pending | | | | C |
 | 09-17 12:00 | M5 | BUY | B | +34.3 | BUY | GENERIC | trending_up | pending | | | NEUTRAL | M (bt +0.38) |
 | 09-17 12:00 | M15 | STRONG_BUY | A | +35.9 | BUY | XAU_ATR | transitioning | +0.50 | TRAIL | | OPPOSE | M |
 | 09-17 12:05 | M5 | STRONG_BUY | A | +37.2 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
 | 09-17 12:10 | M5 | BUY | B | +34.4 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
 | 09-17 12:15 | M5 | STRONG_BUY | A | +38.9 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
 | 09-17 12:15 | M15 | STRONG_BUY | A+ | +43.7 | BUY | XAU_ATR | trending_up | pending | | | OPPOSE | C |
 | 09-17 12:20 | M5 | STRONG_BUY | A | +39.1 | BUY | GENERIC | trending_up | pending | | | | C |
 | 09-17 12:25 | M5 | STRONG_BUY | A | +40.0 | BUY | GENERIC | trending_up | pending | | | | M (bt −0.15) |
 Backtest-only trades: M15 BUY B 09-08 04:00 (−1.03R; live scored the bar +24.7 NEUTRAL D on the
 pre-RSI-change engine), M15 BUY B 09-09 05:45 (+0.23R) and M5 BUY A 09-09 06:00 (+0.14R), both
 during the bot outage.
 ## 7. The owner's four suspicions, checked
 The owner's objective for this review (2026-09-17): is anything functioning poorly in one of four
 specific ways. Each is checked on the 49 unique live setups (37 resolved) and compared with the
 two-year books where the books carry the same measure. "Reached TP1" means the trade's best
 excursion (MFE) reached the planned first target; the advisory model itself exits on the trail, so
 TP1 is advisory.
 ### 7.1 Does the AI panel OPPOSE trades that go on to hit their targets? — yes, as often as not
 | macro verdict | setups | reached TP1 | reached TP2 | resolved wins | mean MFE |
 |---|---|---|---|---|---|
 | AGREE | 12 | 5 (42%) | 4 | 6 / 12 | 0.87R |
 | NEUTRAL | 9 | 5 (56%) | 3 | 5 / 8 | 0.96R |
 | OPPOSE | 13 | **7 (54%)** | 5 | 2 / 7 | 0.64R |
 Opposed trades reached their first target more often than agreed ones. Concrete cases: OPPOSE at
 60% on the Sep 8 07:30 M15 short ("most news points to gold rising") that made +1.13R and hit TP2;
 OPPOSE at 62–65% on five of the Sep 17 longs that ran +0.7R to +1.3R (four still open); AGREE at 82%
 on all three FOMC-evening shorts ("the Fed raised rates, which pushes gold down"), all −1R, gold
 +95$ the next day. The macro persona's notes quote the news narrative of the moment, and the
 2026-09-04 study showed that narrative trails price by about a day. The realised-R view (opposed
 trades 2/7 wins) looks better for the panel only because the opposed winners are mostly the
 pending Sep 17 longs. Structure: technical supports 79% of setups, risk opposes 79%, contrarian
 65% — three near-constant voters. **Verdict: OPPOSE has no value as a veto and must never gate;
 it is presented as "news context" at best.** (observation; the panel is already advisory-only)
 ### 7.2 Wrong direction, or entering late after the move is exhausted? — late by design; this fortnight the continuation failed
 - 11 of 37 resolved setups never reached +0.3R (wrong from the first bar); 17 of 37 were stopped.
  Book norm: 20–22% never reach 0.3R; live 30%.
 - **30 of 37 entries came after price had already moved ≥ 2 ATR in the trade direction over the
  previous 4 h**: PF 0.47, −7.8R. The 7 entries after a smaller prior move: PF 1.21, +0.7R.
 - The engine's own exhaustion tags do **not** separate the losers: trades carrying
  `rsi_extended` / `bb_overextended` at entry (21) had PF 0.68 and −2.8R; the 16 without had PF
  0.53 and −4.4R. The 2026-08-04 stretch verdict (stretched entries are the best book trades) and
  the 2026-09-06 sizing campaign already closed exhaustion-based sizing; the live fortnight adds
  nothing that would reopen it.
 - Three recurring shapes: (a) M15 shorts fire after a 4–6 ATR down leg, are right for four hours
  (69% hit) and wrong by the next day (the low bounces); (b) longs in the downtrend fired at bounce
  tops — Sep 10 05:40 and 05:55 at the 24 h high, followed by a 20 ATR fall; (c) the FOMC shorts
  entered 45–75 min after the decision, 0.2–0.9 ATR above the day's low, after a 4–5 ATR drop: they
  sold the bottom. Every one of these is the momentum design working as validated; the two-year
  book earns its edge on exactly these entries and this fortnight they failed.
  (observation)
 ### 7.3 Do signals fire on big volatile candles? — yes, and that is where the whole loss sits
 | entry context (last 3 primary bars) | resolved | wins | reached TP1 | sum R | PF |
 |---|---|---|---|---|---|
 | a bar ≥ 2 ATR among the last 3 | 15 | 4 (27%) | 5 | **−7.18** | 0.25 |
 | the signal bar itself ≥ 2 ATR | 7 | 1 (14%) | 2 | −3.44 | 0.23 |
 | signal-bar body ≥ 1 ATR in the trade direction | 12 | 5 (42%) | 7 | −3.63 | 0.34 |
 | no bar ≥ 2 ATR among the last 3 | 22 | 13 (59%) | 13 | **+0.02** | 1.00 |
 The 15 setups fired within three bars of a ≥ 2 ATR candle carry the entire fortnight's loss; the
 other 22 broke even at the book's win rate. Largest cases: Sep 10 12:30 M15 short on a 4.2 ATR bar
 (−0.22R), the FOMC 18:45 / 19:00 shorts on 2.5–2.7 ATR bars (−1R each), Sep 14 09:15 short on a
 2.7 ATR bar (+1.05R). A gate for exactly this exists (`big_candle_atr_multiple`, gate 6h) and ships
 off because the 2026-07-30 test found the removed trades net-positive at every multiple on the long
 books. The live fortnight contradicts that book verdict, so the gate was re-run on the two-year
 M15 and M5 books (below).
 Re-test of gate 6h on the two-year books (`--big-candle 2.0 --big-candle-hours {1,24}`, same
 harness as the reference books, control = the MD5-verified W2 books; "removed" = control trades
 absent from the candidate, approximate because the cooldown displaces entries):
 | book | trades | R | PF | win | max DD | H1 / H2 | ΔR vs control | removed trades |
 |---|---|---|---|---|---|---|---|---|
 | M15 control | 931 | +146.1 | 1.43 | 60.5% | 16.1 | +67.7 / +78.4 | | |
 | M15 gate 2 ATR, 1 h lookback | 643 | +117.6 | 1.51 | 61.9% | 14.4 | +58.4 / +59.2 | **−28.5** | 347 trades, **+32.5R**, 58% win |
 | M15 gate 2 ATR, 24 h lookback | 3 | +0.5 | | | | | −145.6 | 929 trades, +147.0R |
 | M5 control | 781 | +146.1 | 1.52 | 52.9% | 10.1 | +41.0 / +105.1 | | |
 | M5 gate 2 ATR, 1 h lookback | 434 | +86.5 | 1.56 | 53.5% | 10.4 | +31.2 / +55.3 | **−59.6** | 368 trades, **+60.2R**, 52% win |
 | M5 gate 2 ATR, 24 h lookback | 0 | | | | | | −146.1 | all |
 Same verdict as 2026-07-30: over two years the trades entered within an hour of a ≥ 2 ATR candle
 are a net-positive bucket on both timeframes (+32R and +60R at the book's own win rates), and the
 gate buys a few points of profit factor by giving up a fifth to two fifths of the R. The live
 fortnight's −7.2R in that bucket is a bad draw from a positive population, not a malfunction.
 **Gate stays off; not a rule-1 candidate.** (needs-a-rule-1-backtest: done here, fails)
 ### 7.4 Are take-profit targets too far, with the market reversing first? — targets are normal in R; the stops are what is big
 Planned geometry by generator (live): M15 GENERIC TP1 = 1.0R with stops of 3.6–4.4 ATR ($21–62,
 one of $121 on the FOMC 19:15 short); M15 XAU_ATR TP1 = 0.69R with a 2 ATR stop ($16 target, $23
 stop); M5 GENERIC TP1 = 0.5R with stops of 4.7–7.3 ATR ($13–32 target, $26–65 stop); XAU_CP TP1 =
 0.88R. As a share of price the live GENERIC stops (medians 0.6–1.0%) sit above the book medians
 (0.58–0.73%) but inside the book's 90th percentiles (0.97–1.66%); only the $121 FOMC stop (2.85%)
 is outside. Capping the stop in ATR was tested and refuted in the 2026-09 campaign.
 | measure | live (37 resolved) | M15 book | M5 book |
 |---|---|---|---|
 | reached TP1 | 49% | 52% | 56% |
 | never reached 0.3R | 30% | 20% | 22% |
 | gave back (MFE ≥ 0.5R, closed ≤ 0) | 16% | 11% | 18% |
 | MFE 0.6–1.0R and closed ≤ 0 | 4 of 8 | | |
 So the targets were reached about as often as in the books; what was worse was the share of trades
 that never moved in favour at all. Trades that did reach 0.6–1.0R often ended flat or negative
 because the trail arms at 0.6R (M5) / 0.7R (M15) and gives back to the arm level. A fixed exit at TP1
 would have made this fortnight −3.55R instead of −7.15R, which is the same direction as the
 2026-08-05 fixed-target study (better in 2024+ trend windows, worse in 2021 and 2023, drawdown
 doubled), so nothing changes there. (observation; `--fixed-rr-tp` remains research-only)
@@ -46,7 +46,8 @@
    "candles_enabled": true,
    "candle_bars_primary": 20,
    "candle_bars_confirm": 10,
-    "verdict_source": "macro"
+    "verdict_source": "mean",
    "personas": ["macro"]
  },
  "violent_watch": {
    "enabled": false,
@@ -18,6 +18,10 @@ immune to anything on the web side):
  * The only shared-code dependency beyond the standard signal path is
    the additive `NewsEngine.get_week_events` accessor, which no
    production signal path calls.
  * Never writes a database. It is not always running, so the Telegram
    bot is the single writer of `signals.db` and `newsdata.db`; main()
    switches every SQLite writer off for this process
    (`app_config.set_db_writes_for_process`, owner decision 2026-09-17).
 Scheduling: every M15/H1/H4 boundary is also an M5 boundary (all
 intervals are multiples of 5, anchored to UTC midnight), so one
@@ -58,7 +62,8 @@ from typing import Dict, List, Optional
 import pandas as pd
 from agent_panel import attach_panel
-from app_config import get_app_config, violent_watch_enabled
+from app_config import (get_app_config, set_db_writes_for_process,
                        violent_watch_enabled)
 from config import PlatformConfig, Timeframe, XAUUSD
 from main import report_to_dict
 from news_engine import NewsEngine
@@ -553,6 +558,14 @@ def main(argv=None) -> None:
    args = ap.parse_args(argv)
    _setup_logging()
    # This process never writes a database. It is not always running, so
    # the Telegram bot is the single writer of signals.db and newsdata.db
    # (owner decision 2026-09-17). The switch is process-wide and beats
    # config.json / env; set here in main(), not at import, so importing
    # the module (the test suite does) leaves every other writer on.
    set_db_writes_for_process(False)
    logger.info("database writes disabled for this process "
                "(the Telegram bot is the only writer)")
    os.makedirs(args.out, exist_ok=True)
    pub = DashboardPublisher(args.out)
    pub.install_static()
@@ -70,6 +70,7 @@
 <tr><td><b>MTF aligned</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr>
 <tr><td><b>Warnings ⚠️</b></td><td>Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings.</td></tr>
 <tr><td><b>Violent-move watch 🌪</b></td><td>An occasional extra line saying the next hour looks unusually likely to produce a very large candle — because a high-impact US release is due (or just landed), because the clock is in a historically wild slot, or because price is stretched outside its bands. It is a heads-up about <i>volatility</i>, not a direction call, and it <b>never</b> changes the signal, the grade or the trade setup — the same alert would be sent without it. Do not close or tighten an open trade because of it: every such protective action was tested over three years and lost money. Your stop is the protection.</td></tr>
 <tr><td><b>AI estimate 🤖</b></td><td>An advisory line such as <i>"AI estimate: about a 45% chance this trade pays off (usual for this kind of setup: 60%). Biggest worry: interest-rate decision in about 2 hours. Still being tested."</i> One AI reviewer (the news reader) receives the recent headlines together with facts computed by code — how far the move has already run, whether a very large candle just printed, how the stop and target compare with the usual, how trades of this exact kind fared in the two-year test, and whether major scheduled news lands before the trade would normally finish — and gives a probability that the trade ends in profit plus one reason for and one against. (Three further reviewers, chart, risk and sceptic, were switched off on 2026-09-17: in two evaluations they added nothing beyond the base rate and mostly voted the same way on every setup.) It is <b>not</b> a recommendation and it <b>never</b> changes the signal, the grade or the setup. Until about 300 of these estimates have been graded against real outcomes, treat it as context, not as a filter: the earlier AGREE/OPPOSE version of this line opposed target-reaching trades as often as it agreed.</td></tr>
 <tr><td><b>Market context 📰</b></td><td>An AI-written note in plain English on what is moving the gold price right now, why it matters, and the next report to watch. It is built only from headlines published in the last 4 hours that lean one way (neutral items are dropped) plus today's high-impact releases. Informational only — it never blocks or grades a signal.</td></tr>
 <tr><td><b>SL / TP1 / RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer. TP1 is the nearest opposing level. <b>1R</b> = the entry-to-SL distance = the amount you risk.</td></tr>
 <tr><td><b>Trail plan 🔄</b></td><td>The real exit. Once price moves +0.6R in your favor, move the stop to trail 0.75R behind the best price and update it at each candle close. <b>TP1 is advisory — let winners run.</b></td></tr>
@@ -168,7 +168,23 @@ def print_signal_report(report: SignalReport, detail: bool = False):
        print(f"  {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
    if report.news_summary:
        print(f"  {Colors.MAGENTA}🧠{Colors.RESET} {report.news_summary}")
-    if report.panel_verdict:
+    if report.panel_verdict and report.panel_verdict_source == "mean":
        base = (report.panel_context or {}).get("base_rate") or {}
        usual = base.get("win_rate")
        usual_txt = (f" (usual for this kind of setup: {usual:.0f}%)"
                     if usual is not None else "")
        print(f"  {Colors.CYAN}🤖{Colors.RESET} AI estimate: about a "
              f"{Colors.BOLD}{report.panel_probability:.0f}%{Colors.RESET} "
              f"chance this trade pays off{usual_txt}. "
              f"{report.panel_rationale} Still being tested.")
        for t in report.panel_takes:
            p_win = t.get("p_win")
            p_txt = f"{p_win:3.0f}%" if p_win is not None else "  - "
            label = t['persona'] if len(report.panel_takes) > 1 else "reasons"
            print(f"     {Colors.GRAY}{label:<11} {p_txt}  "
                  f"for: {t.get('reason_for') or '-'}  |  against: "
                  f"{t.get('reason_against') or '-'}{Colors.RESET}")
    elif report.panel_verdict:
        panel_label = ("AI macro view"
                      if report.panel_verdict_source == "macro" else "AI Panel")
        print(f"  {Colors.CYAN}🤖{Colors.RESET} {panel_label}: "
@@ -230,6 +246,10 @@ def report_to_dict(report: SignalReport) -> dict:
        "panel_created_at": report.panel_created_at,
        "panel_cache_status": report.panel_cache_status,
        "panel_payload_hash": report.panel_payload_hash,
        "panel_verdict_source": report.panel_verdict_source,
        "panel_prompt_version": report.panel_prompt_version,
        "panel_probability": round(report.panel_probability, 1),
        "panel_context": report.panel_context,
        "watch_line": report.watch_line,
        "bullish_reasons": report.bullish_reasons,
        "bearish_reasons": report.bearish_reasons,
@@ -43,10 +43,19 @@ Contract
  headline seen 50 times is one row with ``last_seen`` bumped.
 * **stdlib only** (``sqlite3``), like ``signal_log``.
-Known corpus limitation: a headline mentioning no tracked currency is
+Two columns exist for training only (2026-09-17):
-never constructed by the news engine at all (``news_engine`` drops it
+
-before scoring), so DeepSeek never labels it either and it cannot appear
+* ``xau_score`` — DeepSeek's gold-specific score for the headline,
-here.
+  whatever currencies it names. ``score_by_currency_json["XAU"]``
  carries it only for gold-keyword headlines; 930 of the first 1,546
  DeepSeek rows had no gold label without this column. NULL for a
  keyword-scored row and for rows written before the column existed
  (an existing database is migrated in place, values are not
  recoverable).
 * ``currencies_json`` ``[]`` marks a headline naming no tracked currency.
  The news engine used to drop those before scoring; now it scores and
  archives them but never returns them to the platform. About 23% of a
  day's feed; DeepSeek gave 10 of 24 a gold direction.
 """
 from __future__ import annotations
@@ -85,7 +94,8 @@ CREATE TABLE IF NOT EXISTS headlines (
    score_source TEXT NOT NULL DEFAULT 'keyword',
    teacher_model TEXT,
    teacher_prompt_sha1 TEXT,
-    origin TEXT NOT NULL DEFAULT 'live'
+    origin TEXT NOT NULL DEFAULT 'live',
    xau_score REAL
 );
 CREATE INDEX IF NOT EXISTS idx_headlines_source ON headlines (score_source);
 CREATE INDEX IF NOT EXISTS idx_headlines_first_seen ON headlines (first_seen);
@@ -155,8 +165,22 @@ class NewsArchive:
            self._with_lock_retry(
                lambda: self._conn.execute("PRAGMA journal_mode=WAL"))
            self._with_lock_retry(lambda: self._conn.executescript(_SCHEMA))
            self._with_lock_retry(self._migrate)
            self._conn.commit()
    #: Columns added after the first databases were created, in order.
    #: CREATE TABLE IF NOT EXISTS leaves an existing table untouched, so
    #: each one is added in place on open; existing rows read NULL.
    _ADDED_COLUMNS = (("xau_score", "REAL"),)
    def _migrate(self) -> None:
        present = {row[1] for row in
                   self._conn.execute("PRAGMA table_info(headlines)")}
        for name, decl in self._ADDED_COLUMNS:
            if name not in present:
                self._conn.execute(
                    f"ALTER TABLE headlines ADD COLUMN {name} {decl}")
    @staticmethod
    def _with_lock_retry(fn, attempts: int = 8, delay: float = 0.05):
        """Run ``fn`` again on 'database is locked' (concurrent openers
@@ -202,8 +226,8 @@ class NewsArchive:
            " published_at, first_seen, last_seen, seen_count,"
            " score, score_by_currency_json, currencies_json,"
            " keywords_json, score_source, teacher_model,"
-            " teacher_prompt_sha1, origin"
+            " teacher_prompt_sha1, origin, xau_score"
-            ") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?)"
+            ") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?,?)"
            " ON CONFLICT(hash) DO UPDATE SET"
            # The backfill stamps historical timestamps, so clamp rather
            # than assign: first_seen only ever moves earlier, last_seen
@@ -236,6 +260,13 @@ class NewsArchive:
            "           ELSE headlines.teacher_prompt_sha1 END,"
            f"  score_source = CASE WHEN {wins} THEN excluded.score_source"
            "           ELSE headlines.score_source END,"
            # Same upgrade-only rule as score, and COALESCE on top: a
            # keyword re-sighting carries no gold score and loses on
            # rank, and even a same-rank write without one (a future
            # backfill shape) must never blank a stored value.
            f"  xau_score = CASE WHEN {wins}"
            "           THEN COALESCE(excluded.xau_score, headlines.xau_score)"
            "           ELSE headlines.xau_score END,"
            # Enrichment fills gaps but never blanks what is already there
            # (the legacy backfill shape carries no description or link).
            "  description = COALESCE(excluded.description, headlines.description),"
@@ -246,9 +277,9 @@ class NewsArchive:
        # The rank is bound once per CASE arm. Positional binding means a
        # miscount silently shifts every parameter, so assert it instead of
-        # trusting the arithmetic: 17 INSERT values + one rank per arm.
+        # trusting the arithmetic: 18 INSERT values + one rank per arm.
        n_rank = sql.count("? >= ")
-        n_values = 17
+        n_values = 18
        assert sql.count("?") == n_values + n_rank, (
            f"placeholder count drifted: {sql.count('?')} "
            f"!= {n_values} + {n_rank}")
@@ -264,6 +295,7 @@ class NewsArchive:
                row["currencies_json"], row["keywords_json"],
                row["score_source"], row["teacher_model"],
                row["teacher_prompt_sha1"], row.get("origin", "live"),
                row.get("xau_score"),
            ) + (rank,) * n_rank)
        def _write() -> None:
@@ -321,6 +353,7 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
        "teacher_prompt_sha1": (
            teacher_prompt_sha1 if source == "llm" else None),
        "origin": origin,
        "xau_score": _finite_score(getattr(headline, "xau_score", None)),
    }
@@ -226,6 +226,15 @@ class NewsSentiment:
    # artifact). Without it the two are indistinguishable in the archive
    # and a model trained on the mix inherits the keyword scorer's errors.
    score_source: str = "keyword"
    # DeepSeek's gold-specific score for THIS headline, whatever currencies
    # it names. sentiment_by_currency["XAU"] carries it only when the
    # headline matched a gold keyword; for a pure Fed / dollar / yields
    # headline it used to be discarded, which left 930 of the first 1,546
    # archived DeepSeek rows without a gold label (eval 2026-09-17).
    # Nothing displayed or averaged reads it — the archive stores it as
    # training data for a local scorer. None until DeepSeek has scored
    # the headline (the keyword scorer never sets it).
    xau_score: Optional[float] = None
@dataclass
@@ -371,6 +380,9 @@ class NewsEngine:
    _llm_summary_cache: Tuple[float, str] = (0.0, "")
    _LLM_SENT_CACHE_MAX = 400
    _LLM_EVENT_CACHE_MAX = 100
    #: Headlines per DeepSeek sentiment call — the reply must fit the
    #: client's max_tokens (~10 tokens per headline, 2000 max).
    _LLM_BATCH_CAP = 25
    # Per-source fetch health for status displays (telegram /status).
    # name → {"ok": ts|None, "err": ts|None, "detail": str, "extra": str}.
@@ -811,7 +823,8 @@ class NewsEngine:
    def _fetch_news_sentiment_raw(self) -> List[NewsSentiment]:
        """Fetch and analyze news headlines for sentiment."""
-        headlines = []
+        headlines = []      # names a tracked currency: what the platform uses
        unmatched = []      # names none: scored and archived, never shown
        seen_titles = set()
        ok_feeds = 0
        undated = 0
@@ -861,7 +874,6 @@ class NewsEngine:
                    sentiment = self._analyze_headline_sentiment(title)
                    currencies = self._extract_currencies(title)
                    if currencies:
                    link_el = item.find("link")
                    desc_el = item.find("description")
                    src_el = item.find("source")
@@ -877,7 +889,7 @@ class NewsEngine:
                    publisher = None
                    if src_el is not None:
                        publisher = (src_el.text or "").strip() or None
-                        headlines.append(NewsSentiment(
+                    entry = NewsSentiment(
                        headline=title,
                        source=feed_url,
                        timestamp=pub_date or datetime.now(timezone.utc),
@@ -892,7 +904,18 @@ class NewsEngine:
                            link_el.text if link_el is not None else None),
                        description=desc,
                        publisher=publisher,
-                        ))
+                    )
                    # A headline naming no tracked currency stays invisible
                    # to the platform (no per-currency mean, no display,
                    # no panel) but is scored and archived since
                    # 2026-09-17: it was 23% of a day's feed, and DeepSeek
                    # gave 10 of those 24 a gold direction — the Iran /
                    # Hormuz stories were the strongest gold headlines of
                    # the day. Training data only.
                    if currencies:
                        headlines.append(entry)
                    else:
                        unmatched.append(entry)
                ok_feeds += 1
                self._record_source(
@@ -905,14 +928,21 @@ class NewsEngine:
        if undated:
            logger.debug(f"RSS: dropped {undated} headlines without a publish time")
        if unmatched:
            logger.debug(f"RSS: {len(unmatched)} headlines name no tracked "
                         "currency (scored and archived, not shown)")
        self._record_source(
            "headlines", ok_feeds > 0,
            detail="all feeds failed" if ok_feeds == 0 else "",
            extra=(f"{len(headlines)} headlines from "
                   f"{ok_feeds}/{len(self.config.rss_feeds)} feeds"))
        # Score and archive EVERYTHING fetched; hand back only the
        # currency-matched headlines so display, per-currency means, the
        # panel and the summary see exactly what they saw before.
        scored = headlines + unmatched
        try:
-            self._apply_llm_sentiment(headlines)
+            self._apply_llm_sentiment(scored)
        except Exception as e:
            logger.debug(f"LLM sentiment failed, keyword scores stand: {e}")
@@ -927,7 +957,7 @@ class NewsEngine:
        # effect — it must never cost us the headlines we just fetched.
        try:
            news_archive.archive_headlines(
-                headlines,
+                scored,
                teacher_model=getattr(get_llm_client(), "_model", None),
                teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1)
        except Exception as e:
@@ -1003,24 +1033,31 @@ class NewsEngine:
        """Overwrite keyword sentiment on `headlines` with LLM scores.
        Runs inside _cached_fetch, i.e. UNDER _cache_lock — cache access
-        is direct and the (rare) HTTP call blocks sibling scans at most
+        is direct and the (rare) HTTP calls block sibling scans at most
        once per cache window, matching the existing shared-fetch design.
-        Only headlines missing from the cache are sent; a failed batch
+        Only headlines missing from the cache are sent, in batches of
-        changes nothing.
+        _LLM_BATCH_CAP; a failed batch changes nothing and ends the
        cycle's sending.
        """
        if not headlines or not get_llm_client().available:
            return
        cache = NewsEngine._llm_sent_cache
        keyed = [(hashlib.sha1(h.headline.encode()).hexdigest(), h)
                 for h in headlines]
-        # Cap the batch so the reply fits the client's max_tokens — the
+        # Every new headline goes out this cycle, one call per batch.
-        # overflow stays keyword-scored this cycle and, still uncached,
+        # Until 2026-09-17 only the first 25 went and the overflow stayed
-        # goes out with the next window's batch.
+        # keyword-scored (31 of the first 1,577 archived headlines never
-        new = [(k, h) for k, h in keyed if k not in cache][:25]
+        # got a DeepSeek label that way). A failed call ends the loop: a
-        if new:
+        # timeout or the daily cap would fail the next one too, and what
-            scores = self._llm_headline_scores([h.headline for _, h in new])
+        # is left stays keyword-scored and, still uncached, goes out with
-            if scores is not None:
+        # the next window.
-                for (k, _), sc in zip(new, scores):
+        new = [(k, h) for k, h in keyed if k not in cache]
        for start in range(0, len(new), self._LLM_BATCH_CAP):
            batch = new[start:start + self._LLM_BATCH_CAP]
            scores = self._llm_headline_scores([h.headline for _, h in batch])
            if scores is None:
                break
            for (k, _), sc in zip(batch, scores):
                cache[k] = sc
        while len(cache) > self._LLM_SENT_CACHE_MAX:
            cache.pop(next(iter(cache)))
@@ -1034,6 +1071,7 @@ class NewsEngine:
                cur: (xau if cur == "XAU" else overall)
                for cur in h.currencies_mentioned
            }
            h.xau_score = xau
            h.score_source = "llm"
    def _llm_event_direction(self, event: EconomicEvent) -> Optional[str]:
@@ -0,0 +1,121 @@
 #!/usr/bin/env python3
 """Build agent_panel_base_rates.json from the reference backtest books.
 The agent panel (agent_panel.py, prompt version v2) anchors every
 persona's probability to how trades of the same class fared in the
 validated books: (timeframe, strategy, grade, direction) with fallbacks
 to (timeframe, direction) and (timeframe). This script turns the exported
 63-column book CSVs into that table. Research-side (pandas only); the
 live module reads the JSON and never imports this.
 Usage (the section-9 harness books, regenerated from the kept cache):
    venv/bin/python scripts/build_panel_base_rates.py \\
        --book 5min=/path/m5_w2.csv --book 15min=/path/m15_w2.csv \\
        --book 1h=/path/h1_w3.csv --book 4h=/path/h4_w3.csv \\
        --out agent_panel_base_rates.json
 Statistics per class: n, win_rate (percent of trades with r_multiple >
 0), mean_r, tp1_reach_rate (percent whose best excursion reached the
 planned first target: mfe_r >= rr_planned), median_bars_held. Every
 class with n >= 1 is written; agent_panel._base_rate applies the
 minimum-n rule (30) and the fallback order at read time.
 Re-run whenever a reference book is re-pinned (CLAUDE.md section 9); the
 JSON records each book's file name, MD5 and trade count so the table's
 provenance is auditable.
 """
 from __future__ import annotations
 import argparse
 import hashlib
 import json
 import sys
 from datetime import datetime, timezone
 from typing import Dict, List
 import pandas as pd
 STATS_COLS = ("r_multiple", "mfe_r", "rr_planned", "bars_held",
              "strategy", "quality", "direction")
 def _md5(path: str) -> str:
    h = hashlib.md5()
    with open(path, "rb") as fh:
        for chunk in iter(lambda: fh.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()
 def _stats(df: pd.DataFrame, book: str) -> Dict[str, object]:
    r = df["r_multiple"].astype(float)
    reached = (df["mfe_r"].astype(float) >= df["rr_planned"].astype(float))
    return {
        "n": int(len(df)),
        "win_rate": round(float((r > 0).mean() * 100.0), 2),
        "mean_r": round(float(r.mean()), 4),
        "tp1_reach_rate": round(float(reached.mean() * 100.0), 2),
        "median_bars_held": int(round(float(df["bars_held"].median()))),
        "book": book,
    }
 def build(books: Dict[str, str]) -> Dict[str, object]:
    classes: Dict[str, Dict[str, object]] = {}
    meta: Dict[str, Dict[str, object]] = {}
    for tf, path in books.items():
        df = pd.read_csv(path)
        missing = [c for c in STATS_COLS if c not in df.columns]
        if missing:
            raise SystemExit(f"{path}: missing columns {missing}")
        book = path.rsplit("/", 1)[-1]
        meta[tf] = {"file": book, "md5": _md5(path), "trades": int(len(df)),
                    "first_entry": str(df["entry_time"].min()),
                    "last_entry": str(df["entry_time"].max())}
        if df.empty:
            continue
        classes[f"{tf}|*|*|*"] = _stats(df, book)
        for direction, g in df.groupby("direction"):
            classes[f"{tf}|*|*|{direction}"] = _stats(g, book)
        for (strategy, grade, direction), g in df.groupby(
                ["strategy", "quality", "direction"]):
            classes[f"{tf}|{strategy}|{grade}|{direction}"] = _stats(g, book)
    return {
        "schema": "agent-panel-base-rates-v1",
        "generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
        "key": "timeframe|strategy|grade|direction ('*' = any)",
        "min_n_for_exact_class": 30,
        "books": meta,
        "classes": dict(sorted(classes.items())),
    }
 def main(argv: List[str] | None = None) -> int:
    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    ap.add_argument("--book", action="append", default=[], metavar="TF=CSV",
                    help="timeframe (5min/15min/1h/4h) = exported book CSV")
    ap.add_argument("--out", default="agent_panel_base_rates.json")
    args = ap.parse_args(argv)
    books: Dict[str, str] = {}
    for item in args.book:
        if "=" not in item:
            ap.error(f"--book expects TF=CSV, got {item!r}")
        tf, path = item.split("=", 1)
        books[tf.strip()] = path.strip()
    if not books:
        ap.error("at least one --book is required")
    table = build(books)
    with open(args.out, "w", encoding="utf-8") as fh:
        json.dump(table, fh, indent=1, sort_keys=False)
        fh.write("\n")
    n_exact = sum(1 for k in table["classes"] if "|*|" not in k)
    print(f"wrote {args.out}: {len(table['classes'])} classes "
          f"({n_exact} exact) from {len(books)} book(s)")
    for tf, m in table["books"].items():
        print(f"  {tf:6s} {m['file']}  md5={m['md5']}  trades={m['trades']}")
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -0,0 +1,514 @@
 #!/usr/bin/env python3
 """Offline replay of the agent panel on logged setups, scored against the
 realised outcome — so a prompt or payload version is judged on the SAME
 trades as its predecessor before it is enabled live.
    venv/bin/python scripts/panel_replay.py --db logs/signals.db --dry-run
    venv/bin/python scripts/panel_replay.py --db logs/signals.db --max-calls 160 --out replay.csv
 What it does
 ------------
 1. Loads every resolved setup event (analyze_signal_log's dedup: one
   event per identical setup, first occurrence) whose advisory outcome is
   RESOLVED; the outcome is `advisory_r_multiple > 0` (net R at $0.65).
 2. Rebuilds what each persona would see. Rows logged by agent_panel v2
   carry the exact payloads (panel_payloads_json); older rows are
   reconstructed from the stored indicators_json / setup_json / mtf_json /
   penalty_tags_json / reasons_json / warnings_json, the interned
   headlines and events (signals_with_news), and candles from the kept
   Oanda cache windows so the price path is what the scan saw. Calendar
   events are evaluated AS OF the signal bar (released / minutes_until),
   not as of today.
 3. Runs the CURRENT prompts (agent_panel.PANEL_PROMPT_VERSION) through the
   real DeepSeek client on each payload, validates the replies with the
   live code path, aggregates with agent_panel.mean_verdict.
 4. Scores three predictors on the same events: the new run, the logged
   v1 takes (stance x confidence -> pseudo-probability, when present) and
   the base-rate-only predictor (the class win rate for every trade):
   Brier score, log-loss, calibration by probability band, discrimination
   (mean R and win rate by band), constant-voter rate per persona, and the
   counterfactual books (all / skip p <= 40 / only p >= 60).
 Spend: 4 calls per event against the shared DeepSeek daily cap (500);
 `--max-calls` is a hard stop. `--dry-run` builds and writes the payloads
 without a single call. Read-only on the database.
 Caveat printed with every summary: a few dozen outcomes from one regime
 can reject a pathological prompt (constant voting, narrative anchoring);
 they cannot prove accuracy. The live scorecard at 100 and 300 outcomes
 (analyze_signal_log.py --stored-candles) is the real test.
 """
 from __future__ import annotations
 import argparse
 import glob
 import json
 import math
 import os
 import sys
 from datetime import datetime, timedelta, timezone
 from types import SimpleNamespace
 from typing import Any, Dict, List, Optional, Sequence, Tuple
 import pandas as pd
 ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 if ROOT not in sys.path:
    sys.path.insert(0, ROOT)
 import agent_panel as ap                                   # noqa: E402
 from analyze_signal_log import (cluster_setup_rows, load_rows,   # noqa: E402
                                open_readonly)
 from config import SignalStrength, Timeframe               # noqa: E402
 from news_engine import (EconomicEvent, FundamentalContext,      # noqa: E402
                         NewsSentiment)
 from signal_aggregator import SignalReport, TradeSetup     # noqa: E402
 from signal_log import read_news                           # noqa: E402
 PERSONAS = ap.ALL_PERSONAS      # research default: score every persona
 CACHE_DIR = os.path.join(ROOT, "backtest", "cache")
 _TF = {tf.value: tf for tf in Timeframe}
 # ── Reconstruction ────────────────────────────────────────────────────
 class _AsOfEvent(EconomicEvent):
    """EconomicEvent whose `is_upcoming` / `minutes_until` are evaluated
    relative to the signal bar, not the wall clock."""
    def __init__(self, as_of: datetime, **kw):
        super().__init__(**kw)
        self._as_of = as_of
    @property
    def is_upcoming(self) -> bool:  # type: ignore[override]
        return self.time > self._as_of
    @property
    def minutes_until(self) -> float:  # type: ignore[override]
        return (self.time - self._as_of).total_seconds() / 60.0
 def _dt(s: Optional[str]) -> Optional[datetime]:
    if not s:
        return None
    try:
        d = datetime.fromisoformat(str(s).replace("Z", "+00:00"))
    except ValueError:
        return None
    return d if d.tzinfo else d.replace(tzinfo=timezone.utc)
 def _loads(s: Any, default: Any) -> Any:
    if s in (None, "", "null"):
        return default
    try:
        return json.loads(s)
    except (TypeError, ValueError):
        return default
 def _analysis(tf: str, blob: Dict[str, Any]) -> SimpleNamespace:
    inds = {}
    for name, d in (blob.get("indicators") or {}).items():
        try:
            sig = SignalStrength[d.get("signal", "NEUTRAL")]
        except KeyError:
            sig = SignalStrength.NEUTRAL
        inds[name] = SimpleNamespace(
            signal=sig, confidence=float(d.get("confidence") or 0.0),
            weight=float(d.get("weight") or 0.0), value=d.get("value"),
            description=d.get("description") or "")
    return SimpleNamespace(timeframe=tf, timestamp=blob.get("timestamp"),
                           key_levels=dict(blob.get("key_levels") or {}),
                           indicators=inds)
 def _context_from_row(events_json: Optional[str], headlines_json: Optional[str],
                      as_of: datetime) -> FundamentalContext:
    ctx = FundamentalContext()
    ev = _loads(events_json, {}) or {}
    for key in ("events_today", "events_upcoming_1h"):
        out = []
        for e in ev.get(key) or []:
            t = _dt(e.get("time"))
            if t is None:
                continue
            out.append(_AsOfEvent(as_of, time=t, currency=e.get("currency") or "",
                                  impact=e.get("impact") or "", event=e.get("event") or "",
                                  forecast=e.get("forecast"), previous=e.get("previous"),
                                  actual=e.get("actual")))
        setattr(ctx, key, out)
    ctx.news_sentiment = dict(ev.get("sentiment_by_currency") or {})
    ctx.calendar_available = bool(ev.get("calendar_available", False))
    for h in _loads(headlines_json, []) or []:
        t = _dt(h.get("timestamp")) or as_of
        ctx.news_headlines.append(NewsSentiment(
            headline=h.get("headline") or "", source=h.get("source") or "",
            timestamp=t, currencies_mentioned=list(h.get("currencies") or []),
            sentiment_score=float(h.get("score") or 0.0),
            keywords=list(h.get("keywords") or []),
            sentiment_by_currency={k: float(v) for k, v in
                                   (h.get("score_by_currency") or {}).items()
                                   if v is not None},
            score_source=h.get("score_source") or "keyword"))
    return ctx
 _frame_cache: Dict[str, pd.DataFrame] = {}
 _have_cols: set = set()
 def _col(name: str) -> str:
    """Column reference that tolerates a database logged before the
    column existed (a read-only copy is never migrated)."""
    return name if name in _have_cols else f"NULL AS {name}"
 def _cached_frame(tf: str) -> Optional[pd.DataFrame]:
    """Newest kept Oanda cache window for `tf` (raw bars, UTC index)."""
    if tf in _frame_cache:
        return _frame_cache[tf]
    files = sorted(glob.glob(os.path.join(CACHE_DIR, f"oanda_XAU_USD_{tf}_*.parquet")),
                   key=lambda p: p.rsplit("_", 1)[-1])
    df = None
    if files:
        try:
            df = pd.read_parquet(files[-1])
            if df.index.tz is None:
                df.index = df.index.tz_localize("UTC")
        except Exception:
            df = None
    _frame_cache[tf] = df
    return df
 def _frames_as_of(tfs: Sequence[str], bar_ts: datetime) -> Dict[Timeframe, pd.DataFrame]:
    out: Dict[Timeframe, pd.DataFrame] = {}
    for tf in tfs:
        df = _cached_frame(tf)
        tfe = _TF.get(tf)
        if df is None or tfe is None:
            continue
        sl = df[df.index <= pd.Timestamp(bar_ts)]
        if len(sl) >= 3:
            out[tfe] = sl
    return out
 def build_report(con, rowid: int) -> Tuple[Optional[SignalReport], Optional[FundamentalContext],
                                           Dict[Timeframe, pd.DataFrame], Optional[Dict[str, Any]]]:
    """(report, context, frames, stored_payloads) for one logged row."""
    row = con.execute(
        "SELECT symbol, timeframe, " + _col("bar_ts") + ", logged_at, signal, score,"
        " confidence, quality, price, market_regime, volatility_regime, mtf_json,"
        " setup_json, reasons_json, warnings_json, event_risk, event_description,"
        " news_sentiment, indicators_json, " + _col("penalty_tags_json") + ", "
        + _col("exhaustion_count") + ", " + _col("penalty_total") + ", "
        + _col("momentum_fading") + ", " + _col("panel_payloads_json")
        + " FROM signals WHERE id=?", (rowid,)).fetchone()
    if row is None:
        return None, None, {}, None
    (symbol, tf, bar_ts, logged_at, signal, score, conf, quality, price, regime, vol,
     mtf_json, setup_json, reasons_json, warnings_json, event_risk, event_desc, news_sent,
     ind_json, pen_json, exh, pen_total, mom, payloads_json) = row
    inds = _loads(ind_json, {}) or {}
    bar_label = bar_ts or (inds.get(tf) or {}).get("timestamp")
    as_of_bar = _dt(bar_label) or _dt(logged_at)
    if as_of_bar is None:
        return None, None, {}, None
    secs = ap._TF_SECONDS.get(tf, 900)
    as_of = as_of_bar + timedelta(seconds=secs)     # the bar closed
    report = SignalReport(symbol=symbol, display_name="XAU/USD",
                          timestamp=str(bar_label or logged_at))
    try:
        report.final_signal = SignalStrength[signal]
    except KeyError:
        report.final_signal = SignalStrength.NEUTRAL
    report.final_score = float(score or 0.0)
    report.final_confidence = float(conf or 0.0)
    report.signal_quality = quality or ""
    report.current_price = float(price or 0.0)
    report.market_regime = regime or ""
    report.volatility_regime = vol or ""
    mtf = _loads(mtf_json, {}) or {}
    report.mtf_agreement = bool(mtf.get("agreement", False))
    for k, v in (mtf.get("signals") or {}).items():
        try:
            report.mtf_signals[k] = SignalStrength[v]
        except KeyError:
            report.mtf_signals[k] = SignalStrength.NEUTRAL
    report.mtf_scores = {k: float(v or 0.0) for k, v in (mtf.get("scores") or {}).items()}
    reasons = _loads(reasons_json, {}) or {}
    report.bullish_reasons = list(reasons.get("bullish") or [])
    report.bearish_reasons = list(reasons.get("bearish") or [])
    report.warnings = list(_loads(warnings_json, []) or [])
    report.event_risk = event_risk or "clear"
    report.event_description = event_desc or ""
    report.news_sentiment = float(news_sent or 0.0)
    report.exhaustion_count = int(exh or 0)
    report.penalty_tags = {str(k): float(v or 0.0)
                           for k, v in (_loads(pen_json, {}) or {}).items()}
    report.penalty_total = float(pen_total or 0.0)
    report.momentum_fading = bool(mom)
    setup = _loads(setup_json, None)
    if setup:
        report.trade_setup = TradeSetup(
            pair=symbol, display_name="XAU/USD", direction=setup.get("direction"),
            entry_price=float(setup.get("entry") or 0.0),
            stop_loss=float(setup.get("stop_loss") or 0.0),
            take_profit_1=float(setup.get("take_profit_1") or 0.0),
            take_profit_2=float(setup.get("take_profit_2") or 0.0),
            risk_reward=float(setup.get("risk_reward") or 0.0),
            risk_pips=float(setup.get("risk_pips") or 0.0),
            reward_pips=float(setup.get("reward_pips") or 0.0),
            pip_value=0.10, strategy=setup.get("strategy") or "GENERIC",
            setup_pattern=setup.get("setup_pattern"),
            round_trip_cost=setup.get("round_trip_cost"),
            trail_arm_r=setup.get("trail_arm_r"), trail_dist_r=setup.get("trail_dist_r"))
    if tf in inds:
        report.primary_analysis = _analysis(tf, inds[tf])
    report.confirmation_analyses = [_analysis(k, v) for k, v in inds.items() if k != tf]
    events_json, headlines_json = read_news(con, rowid)
    context = _context_from_row(events_json, headlines_json, as_of)
    frames = _frames_as_of([tf] + [k for k in inds if k != tf], as_of_bar)
    return report, context, frames, _loads(payloads_json, None)
 def build_jobs(report: SignalReport, context: FundamentalContext,
               frames: Dict[Timeframe, pd.DataFrame], stored: Optional[Dict[str, Any]],
               settings: Dict[str, Any]) -> Tuple[Tuple[str, str, Dict[str, Any]], ...]:
    """Persona jobs on the CURRENT prompts: stored payloads when the row
    has them, else rebuilt through the live payload builders."""
    systems = {"macro": ap._MACRO_SYSTEM, "technical": ap._TECH_SYSTEM,
               "risk": ap._RISK_SYSTEM, "contrarian": ap._CONTRARIAN_SYSTEM}
    if stored and all(p in stored for p in PERSONAS):
        return tuple((p, systems[p], stored[p]) for p in PERSONAS)
    candles: Dict[str, Any] = {}
    analyses = [report.primary_analysis] + list(report.confirmation_analyses)
    counts = [int(settings.get("candle_bars_primary", ap.DEFAULT_CANDLE_BARS_PRIMARY))] + \
             [int(settings.get("candle_bars_confirm", ap.DEFAULT_CANDLE_BARS_CONFIRM))] * \
             len(report.confirmation_analyses)
    for analysis, n in zip(analyses, counts):
        if analysis is None:
            continue
        atr = ap._finite(getattr((analysis.indicators or {}).get("atr"), "value", None))
        df = frames.get(_TF.get(analysis.timeframe))
        if atr and atr > 0 and df is not None and len(df) >= 2:
            candles[analysis.timeframe] = ap._normalize_candles(df, atr, n)
    ctx = ap._price_context(report, frames, context)
    return ap._panel_jobs(report, context, candles or None, ctx, PERSONAS)
 # ── Scoring ───────────────────────────────────────────────────────────
 def _brier(ps: Sequence[float], ys: Sequence[int]) -> float:
    return sum((p / 100.0 - y) ** 2 for p, y in zip(ps, ys)) / len(ps)
 def _logloss(ps: Sequence[float], ys: Sequence[int]) -> float:
    eps = 1e-3
    tot = 0.0
    for p, y in zip(ps, ys):
        q = min(1 - eps, max(eps, p / 100.0))
        tot += -(y * math.log(q) + (1 - y) * math.log(1 - q))
    return tot / len(ps)
 def score_predictor(name: str, ps: Sequence[Optional[float]], rs: Sequence[float]) -> Dict[str, Any]:
    pairs = [(p, r) for p, r in zip(ps, rs) if p is not None]
    if not pairs:
        return {"predictor": name, "n": 0}
    P = [p for p, _ in pairs]; R = [r for _, r in pairs]; Y = [1 if r > 0 else 0 for r in R]
    bands = {}
    for label, lo, hi in (("<=40", -1, ap.P_OPPOSE), ("40-60", ap.P_OPPOSE, ap.P_SUPPORT),
                          (">=60", ap.P_SUPPORT, 101)):
        sel = [(p, r) for p, r in pairs if (lo < p <= hi if label == "<=40" else
                                             lo < p < hi if label == "40-60" else p >= lo)]
        if sel:
            bands[label] = {"n": len(sel), "mean_p": round(sum(p for p, _ in sel) / len(sel), 1),
                            "win_rate": round(100 * sum(1 for _, r in sel if r > 0) / len(sel), 1),
                            "mean_r": round(sum(r for _, r in sel) / len(sel), 3),
                            "sum_r": round(sum(r for _, r in sel), 2)}
        else:
            bands[label] = {"n": 0}
    stances = [ap.derive_stance(p) for p in P]
    modal = max(set(stances), key=stances.count)
    return {
        "predictor": name, "n": len(P),
        "brier": round(_brier(P, Y), 4), "logloss": round(_logloss(P, Y), 4),
        "mean_p": round(sum(P) / len(P), 1), "win_rate": round(100 * sum(Y) / len(Y), 1),
        "bands": bands, "modal_stance": modal,
        "constant_voter_rate": round(100 * stances.count(modal) / len(stances), 1),
        "sum_r_all": round(sum(R), 2),
        "sum_r_skip_oppose": round(sum(r for p, r in pairs if p > ap.P_OPPOSE), 2),
        "sum_r_support_only": round(sum(r for p, r in pairs if p >= ap.P_SUPPORT), 2),
    }
 def print_summary(results: List[Dict[str, Any]]) -> None:
    print("\n== Predictor scorecard (outcome = advisory R > 0 at $0.65) ==")
    print(f"{'predictor':22s} {'n':>3s} {'Brier':>6s} {'logloss':>7s} {'mean p':>6s} {'win%':>5s} "
          f"{'modal':>8s} {'const%':>6s} {'R all':>7s} {'R skip<=40':>10s} {'R >=60 only':>11s}")
    for r in results:
        if r.get("n", 0) == 0:
            print(f"{r['predictor']:22s}   0"); continue
        print(f"{r['predictor']:22s} {r['n']:3d} {r['brier']:6.3f} {r['logloss']:7.3f} {r['mean_p']:6.1f} "
              f"{r['win_rate']:5.1f} {r['modal_stance']:>8s} {r['constant_voter_rate']:6.1f} "
              f"{r['sum_r_all']:+7.2f} {r['sum_r_skip_oppose']:+10.2f} {r['sum_r_support_only']:+11.2f}")
    print("\n  calibration bands (n / mean p / observed win% / mean R):")
    for r in results:
        if r.get("n", 0) == 0:
            continue
        cells = "  ".join(f"{k}: {v['n']}/{v.get('mean_p', '-')}/{v.get('win_rate', '-')}/{v.get('mean_r', '-')}"
                          if v["n"] else f"{k}: 0" for k, v in r["bands"].items())
        print(f"  {r['predictor']:22s} {cells}")
    print("\nCaveat: a few dozen outcomes from one regime can reject a pathological prompt "
          "(constant voting, narrative anchoring); they cannot prove accuracy.")
 # ── Main ──────────────────────────────────────────────────────────────
 def main(argv: Optional[Sequence[str]] = None) -> int:
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("--db", default=os.path.join(ROOT, "logs", "signals.db"))
    parser.add_argument("--limit", type=int, default=0, help="max events (0 = all)")
    parser.add_argument("--max-calls", type=int, default=200,
                        help="hard stop on LLM requests (4 per event)")
    parser.add_argument("--dry-run", action="store_true",
                        help="build payloads only; no LLM call")
    parser.add_argument("--out", default=None, help="per-event CSV (and .payloads.json on --dry-run)")
    parser.add_argument("--include-pending", action="store_true",
                        help="also build payloads for unresolved setups (dry-run only)")
    parser.add_argument("--personas", default=",".join(ap.ALL_PERSONAS),
                        help="comma-separated personas to run (default: all four, so every "
                             "voice is scored; the live default is agent_panel.personas = macro)")
    args = parser.parse_args(argv)
    global PERSONAS
    PERSONAS = ap.normalize_personas([p for p in args.personas.split(",") if p.strip()])
    rows = load_rows(args.db)
    events = cluster_setup_rows(rows)
    con = open_readonly(args.db)
    global _have_cols
    _have_cols = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
    settings = {"candle_bars_primary": ap.DEFAULT_CANDLE_BARS_PRIMARY,
                "candle_bars_confirm": ap.DEFAULT_CANDLE_BARS_CONFIRM}
    picked = []
    for ev in events:
        res = next((r for r in ev.rows if r.advisory_status == "RESOLVED" and r.advisory_r is not None), None)
        if res is None and not (args.dry_run and args.include_pending):
            continue
        picked.append((ev, res))
    if args.limit:
        picked = picked[:args.limit]
    print(f"events: {len(events)} total, {len(picked)} selected "
          f"({'dry run' if args.dry_run else 'live calls, max ' + str(args.max_calls)}), "
          f"prompt version {ap.PANEL_PROMPT_VERSION}, personas {','.join(PERSONAS)}")
    client = None if args.dry_run else ap.get_llm_client()
    calls = 0
    per_event: List[Dict[str, Any]] = []
    payload_dump: Dict[str, Any] = {}
    for ev, res in picked:
        rep = ev.representative
        # Prefer a member row that already carries the v2 payloads.
        rowid = rep.id
        for r in ev.rows:
            has = (con.execute("SELECT panel_payloads_json IS NOT NULL FROM signals"
                               " WHERE id=?", (r.id,)).fetchone()
                   if "panel_payloads_json" in _have_cols else None)
            if has and has[0]:
                rowid = r.id
                break
        report, context, frames, stored = build_report(con, rowid)
        if report is None or report.trade_setup is None:
            continue
        jobs = build_jobs(report, context, frames, stored, settings)
        ctx = next((payload.get("context") for _, _, payload in jobs if payload.get("context")), {}) or {}
        base_p = (ctx.get("base_rate") or {}).get("win_rate")
        # logged v1 takes (any member row) -> pseudo-probabilities
        v1 = {}
        for r in ev.rows:
            takes = con.execute("SELECT panel_takes_json, " + _col("panel_prompt_version")
                                + " FROM signals WHERE id=?", (r.id,)).fetchone()
            if takes and takes[0]:
                for t in _loads(takes[0], []) or []:
                    p = t.get("p_win")
                    if p is None and t.get("stance") in ap._VALID_STANCES:
                        p = ap.pseudo_probability(t["stance"], float(t.get("confidence") or 0))
                    if p is not None:
                        v1[t.get("persona")] = float(p)
                v1["_version"] = takes[1] or "v1-logged"
                break
        entry: Dict[str, Any] = {
            "event_row": rep.id, "bar": report.timestamp, "tf": rep.timeframe,
            "direction": report.trade_setup.direction, "grade": rep.quality,
            "strategy": report.trade_setup.strategy,
            "R": res.advisory_r if res else None, "stored_payloads": bool(stored),
            "base_p": base_p, "prior_move_4h_atr": ctx.get("prior_move_4h_atr"),
            "move_from_start_atr": ctx.get("move_from_24h_start_atr"),
            "big_candle_last3": ctx.get("big_candle_last3"),
        }
        for p in PERSONAS:
            entry[f"v1_{p}"] = v1.get(p)
        v1_ps = [v1[p] for p in PERSONAS if p in v1]
        entry["v1_mean"] = round(sum(v1_ps) / len(v1_ps), 1) if len(v1_ps) >= min(ap.MEAN_QUORUM, len(v1_ps) or 1) and v1_ps else None
        if args.dry_run:
            payload_dump[str(rep.id)] = {name: payload for name, _, payload in jobs}
        else:
            takes = []
            for name, system, payload in jobs:
                if calls >= args.max_calls:
                    break
                calls += 1
                reply = client.chat_json(system, payload)
                take = ap._validate_take(name, reply)
                if take is not None:
                    takes.append(take)
                    entry[f"new_{name}"] = take.p_win
                    entry[f"new_{name}_for"] = take.reason_for
                    entry[f"new_{name}_against"] = take.reason_against
            v = ap.mean_verdict(takes, quorum=min(ap.MEAN_QUORUM, len(jobs)))
            entry["new_mean"] = v.probability if v else None
            entry["new_verdict"] = v.verdict if v else None
            print(f"  #{rep.id:<5d} {report.timestamp[:16]} {rep.timeframe:5s} "
                  f"{report.trade_setup.direction:4s} {rep.quality:2s} R={entry['R']:+.2f} "
                  f"base={base_p} new_mean={entry['new_mean']} "
                  f"({', '.join(f'{t.persona[:4]} {t.p_win:.0f}' for t in takes)})  "
                  f"v1_mean={entry['v1_mean']}")
        per_event.append(entry)
        if not args.dry_run and calls >= args.max_calls:
            print(f"  stopped at --max-calls {args.max_calls}")
            break
    con.close()
    if args.dry_run:
        print(f"built payloads for {len(per_event)} events")
        if args.out:
            with open(args.out.rsplit('.', 1)[0] + ".payloads.json", "w") as fh:
                json.dump(payload_dump, fh, indent=1, sort_keys=True)
            print(f"wrote {args.out.rsplit('.', 1)[0]}.payloads.json")
    scored = [e for e in per_event if e["R"] is not None]
    if scored:
        rs = [e["R"] for e in scored]
        results = [score_predictor("base-rate only",
                                   [e["base_p"] if e["base_p"] is not None else None for e in scored], rs),
                   score_predictor("v1 logged (mean)", [e["v1_mean"] for e in scored], rs)]
        for p in PERSONAS:
            results.append(score_predictor(f"v1 logged {p}", [e.get(f"v1_{p}") for e in scored], rs))
        if not args.dry_run:
            results.append(score_predictor(f"NEW {ap.PANEL_PROMPT_VERSION} (mean)",
                                           [e.get("new_mean") for e in scored], rs))
            for p in PERSONAS:
                results.append(score_predictor(f"NEW {p}", [e.get(f"new_{p}") for e in scored], rs))
        print_summary(results)
    if args.out and per_event:
        pd.DataFrame(per_event).to_csv(args.out, index=False)
        print(f"wrote {args.out} ({len(per_event)} events)")
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -115,7 +115,16 @@ class SignalReport:
    panel_created_at: str = ""
    panel_cache_status: str = ""
    panel_payload_hash: str = ""
-    panel_verdict_source: str = ""  # "macro" / "chair" / ""
+    panel_verdict_source: str = ""  # "mean" / "macro" / "chair" / ""
    # v2 panel (2026-09-17): the mean persona probability (0-100) that
    # the trade closes profitably, the prompt version that produced it,
    # the exact per-persona payloads and the shared code-computed
    # context — all advisory/DISPLAY-ONLY, logged for the offline replay
    # harness (scripts/panel_replay.py) and the calibration scorecard.
    panel_probability: float = 0.0
    panel_prompt_version: str = ""
    panel_payloads: Dict = field(default_factory=dict)
    panel_context: Dict = field(default_factory=dict)
    # Violent-move watch line (advisory/DISPLAY-ONLY). Stamped post-hoc
    # by surfaces via violent_watch.attach_watch — never populated
@@ -277,6 +286,12 @@ class SignalAggregator:
            dfs=dfs,
            use_news=True,
        )
        # Live-only hand-off for the post-hoc advisory add-ons (the agent
        # panel's price context reads the frames this scan analysed, so
        # it sees exactly the bars the engine saw). A private attribute
        # like _signal_log_rowid: not a dataclass field, never read by
        # the core, absent on backtest replays by construction.
        report._frames = dfs
        if stale_tfs:
            report.warnings.append(f"STALE DATA on {', '.join(stale_tfs)} — signal may be unreliable")
@@ -152,7 +152,12 @@ CREATE TABLE IF NOT EXISTS signals (
    panel_payload_hash TEXT,
    advisory_outcome_status TEXT,
    advisory_r_multiple REAL,
-    advisory_outcome_json TEXT
+    advisory_outcome_json TEXT,
    panel_prompt_version TEXT,
    panel_probability REAL,
    panel_probs_json TEXT,
    panel_payloads_json TEXT,
    panel_context_json TEXT
 );
 CREATE INDEX IF NOT EXISTS idx_signals_sym_tf_ts
    ON signals (symbol, timeframe, logged_at);
@@ -225,6 +230,16 @@ _ADDITIVE_SIGNAL_COLS = (
    ("watch_level", "TEXT"),
    ("watch_kinds", "TEXT"),
    ("watch_line", "TEXT"),
    # 2026-09-17 agent-panel v2 (see agent_panel.py): prompt era, the mean
    # persona probability, per-persona probabilities + reasons, the exact
    # per-persona payloads and the shared context — written post-hoc by
    # update_panel so scripts/panel_replay.py can re-score any prompt
    # version on the same evidence.
    ("panel_prompt_version", "TEXT"),
    ("panel_probability", "REAL"),
    ("panel_probs_json", "TEXT"),
    ("panel_payloads_json", "TEXT"),
    ("panel_context_json", "TEXT"),
 )
 # Read-side compatibility view `signals_with_news`: legacy rows keep
@@ -1025,6 +1040,11 @@ class SignalLogger:
        cache_status: Optional[str] = None,
        payload_hash: Optional[str] = None,
        verdict_source: Optional[str] = None,
        prompt_version: Optional[str] = None,
        probability: Optional[float] = None,
        probs_json: Optional[str] = None,
        payloads_json: Optional[str] = None,
        context_json: Optional[str] = None,
    ) -> None:
        """Fill panel provenance; seeds an outcome only for rows that
        log() did not already seed (rows written by an older process)."""
@@ -1051,11 +1071,18 @@ class SignalLogger:
                " panel_cache_status=COALESCE(?, panel_cache_status),"
                " panel_payload_hash=COALESCE(?, panel_payload_hash),"
                " panel_verdict_source=COALESCE(?, panel_verdict_source),"
                " panel_prompt_version=COALESCE(?, panel_prompt_version),"
                " panel_probability=COALESCE(?, panel_probability),"
                " panel_probs_json=COALESCE(?, panel_probs_json),"
                " panel_payloads_json=COALESCE(?, panel_payloads_json),"
                " panel_context_json=COALESCE(?, panel_context_json),"
                " advisory_outcome_status=COALESCE(advisory_outcome_status, ?),"
                " advisory_outcome_json=COALESCE(advisory_outcome_json, ?)"
                " WHERE id=?",
                (verdict, confidence, rationale, takes_json,
                 created_at, cache_status, payload_hash, verdict_source,
                 prompt_version, probability, probs_json, payloads_json,
                 context_json,
                 advisory_status, advisory_json, rowid))
            self._conn.commit()
@@ -1592,15 +1619,30 @@ def log_panel_update(report: Any) -> None:
            lg = _loggers.get(path)
        if lg is None:
            return
        takes = getattr(report, "panel_takes", None) or []
        probs = [{"persona": t.get("persona"), "p_win": _finite(t.get("p_win")),
                  "stance": t.get("stance"),
                  "confidence": _finite(t.get("confidence")),
                  "reason_for": t.get("reason_for") or "",
                  "reason_against": t.get("reason_against") or ""}
                 for t in takes if isinstance(t, dict)]
        payloads = getattr(report, "panel_payloads", None) or {}
        context = getattr(report, "panel_context", None) or {}
        probability = _finite(getattr(report, "panel_probability", None))
        lg.update_panel(
            int(rowid), verdict,
            _finite(getattr(report, "panel_confidence", None)),
            getattr(report, "panel_rationale", "") or "",
-            _dumps(getattr(report, "panel_takes", None) or []),
+            _dumps(takes),
            created_at=getattr(report, "panel_created_at", "") or None,
            cache_status=getattr(report, "panel_cache_status", "") or None,
            payload_hash=getattr(report, "panel_payload_hash", "") or None,
-            verdict_source=getattr(report, "panel_verdict_source", "") or None)
+            verdict_source=getattr(report, "panel_verdict_source", "") or None,
            prompt_version=getattr(report, "panel_prompt_version", "") or None,
            probability=probability if probability else None,
            probs_json=_dumps(probs) if probs else None,
            payloads_json=_dumps(payloads) if payloads else None,
            context_json=_dumps(context) if context else None)
    except Exception:
        logger.warning("signal_log: failed to persist panel verdict",
                       exc_info=True)
@@ -471,7 +471,23 @@ function detailHTML(tf) {
      ${esc(r.news_summary || "")}</div>`);
  }
-  if (r.panel_verdict) {
+  if (r.panel_verdict && r.panel_verdict_source === "mean") {
    // v2 panel: a probability anchored to the setup class's base rate,
    // one reason each way per persona. Advisory only; no verdict colour.
    const base = (r.panel_context && r.panel_context.base_rate) || {};
    const usual = base.win_rate != null ? ` (usual for this kind of setup: ${Math.round(base.win_rate)}%)` : "";
    const takes = (r.panel_takes || []).map(t =>
      `<tr><td class="ind-k">${esc(t.persona)}</td>
        <td class="num text-end">${t.p_win != null ? Math.round(t.p_win) + "%" : "—"}</td>
        <td class="small text-secondary">${esc(t.reason_for || "")}</td>
        <td class="small text-secondary">${esc(t.reason_against || "")}</td></tr>`).join("");
    out += sec("AI estimate (advisory, still being tested)", `<div class="text-secondary small mb-2">
      About a <strong>${Math.round(r.panel_probability || 0)}%</strong> chance this trade pays off${esc(usual)}.<br>
      ${esc(r.panel_rationale || "")}</div>
      ${takes ? `<div class="detail-table"><table class="table table-sm d-table">
        <thead><tr><th></th><th class="text-end">chance</th><th>for</th><th>against</th></tr></thead>
        <tbody>${takes}</tbody></table></div>` : ""}`);
  } else if (r.panel_verdict) {
    const vcls = r.panel_verdict === "AGREE" ? "text-success"
      : r.panel_verdict === "OPPOSE" ? "text-danger" : "text-secondary";
    const takes = (r.panel_takes || []).map(t =>
@@ -324,6 +324,36 @@ _DISCLAIMER = (
 )
 def _panel_estimate_lines(report: SignalReport) -> list[str]:
    """v2 panel (verdict_source="mean"): a plain-language probability, the
    class base rate it is anchored to, the biggest worry, and each
    persona's own estimate. Advisory only — no colour, no verdict word,
    never a recommendation (2026-09-17 review: the old AGREE/OPPOSE line
    opposed target-reaching trades as often as it agreed)."""
    base = (report.panel_context or {}).get("base_rate") or {}
    usual = base.get("win_rate")
    line = (f"🤖 AI estimate: about a *{report.panel_probability:.0f}%* "
            f"chance this trade pays off")
    if usual is not None:
        line += f" (usual for this kind of setup: {usual:.0f}%)"
    line += "."
    if report.panel_rationale:
        line += f" {_escape_md(report.panel_rationale)}"
    worst = min((t for t in report.panel_takes
                 if t.get("p_win") is not None and t.get("reason_against")),
                key=lambda t: t["p_win"], default=None)
    if worst is not None:
        line += f" Biggest worry: {_escape_md(worst['reason_against'])}"
    line += " Still being tested."
    out = [line]
    if len(report.panel_takes) > 1:      # one persona: the headline says it all
        votes = " · ".join(
            f"{t['persona']} {t['p_win']:.0f}%" if t.get("p_win") is not None
            else f"{t['persona']} -" for t in report.panel_takes)
        out.append(f"   _{_escape_md(votes)}_")
    return out
 def format_signal_message(report: SignalReport) -> str:
    sig = report.final_signal
    emoji = _SIGNAL_EMOJI.get(sig, "⬜")
@@ -423,7 +453,9 @@ def format_signal_message(report: SignalReport) -> str:
        extra.append(_escape_md(report.watch_line))
    if report.news_summary:
        extra.append(f"📰 {_escape_md(report.news_summary)}")
-    if report.panel_verdict:
+    if report.panel_verdict and report.panel_verdict_source == "mean":
        extra.extend(_panel_estimate_lines(report))
    elif report.panel_verdict:
        mark = {"AGREE": "✅", "OPPOSE": "⛔", "NEUTRAL": "➖"}.get(
            report.panel_verdict, "")
        panel_label = ("AI macro view"