news corpus for a local scorer + agent panel v2 + dashboard read-only (2026-09-17); books byte-identical

Agent panel v2: personas return p_win anchored to the book base rates with a code-computed decision context; verdict = equal-weight mean (verdict_source "mean"); only the macro persona runs live (agent_panel.personas ["macro"]); prompt version and exact payloads logged; offline replay harness scripts/panel_replay.py; base-rate table agent_panel_base_rates.json (scripts/build_panel_base_rates.py). Backtester `strategy` column now carries the generator's own stamp (label-only; five reference books re-pinned, all 62 other columns identical). News corpus (after evaluating the owner's ML-NEWS scikit model against DeepSeek: golden set 39/78 vs 78/78, out-of-sample agreement 64% against a 90% self-agreement ceiling; verdict: fallback only, not a replacement): - NewsSentiment.xau_score + archive column xau_score keep DeepSeek's gold score for every headline (930 of the first 1,546 DeepSeek rows had none); an existing newsdata.db gains the column in place on first open - headlines naming no tracked currency are scored and archived (currencies_json []) but never returned to the platform - _apply_llm_sentiment sends every new headline in its cycle, 25 per call; a failed call ends the cycle's sending Dashboard publisher never writes a database: main() calls app_config.set_db_writes_for_process(False), honoured by signal_log_enabled() and news_archive_enabled() (owner decision: the bot is the single writer). Test suite: the hermetic app config now points both writers at a suite-private temp dir — the RSS fixture tests were writing fixture headlines into the real logs/newsdata.db — with a Module 32 test pinning it. 529 tests.
2026-09-17 19:49:29 -04:00
parent 183df7832b
commit ad14ea4fed
21 changed files with 3803 additions and 259 deletions
@@ -21,7 +21,7 @@ python -m venv venv && venv/bin/pip install -r requirements.txt   # Python 3.12;
 cp config.json.example config.json                                 # then fill in the keys (see section 5)

 venv/bin/python test_all.py                                        # QA suite, no network; prints its own count
-                                                                   #   (506 tests / 33 modules as of 2026-09-15)
+                                                                   #   (529 tests / 34 modules as of 2026-09-17)
 venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail      # one signal report in the terminal
                                                                   #   --json full dump, --panel adds the advisory LLM panel,
                                                                   #   --timeframe M5 | M15 | H1 | H4
@@ -164,11 +164,20 @@ later generator shipped a setup.
  into the Apache folder.

 Two advisory add-ons are attached **after** the scan by the surface, never inside the signal
-code: the LLM agent panel (`agent_panel.py`, four DeepSeek personas, verdict AGREE / NEUTRAL /
-OPPOSE) and the violent-move watch (`violent_watch.py`, a "the next hour may be wild" line). No
-gate reads either of them.
+code: the LLM agent panel (`agent_panel.py`; since 2026-09-17 only the **macro** persona runs by
+default, `agent_panel.personas: ["macro"]`, one DeepSeek call per panel — the technical, risk and
+contrarian personas carried no information in two live evaluations and stay available for research;
+the persona returns a probability that the trade closes profitably, anchored to the setup class's
+base rate from the validated books, shown as the "AI estimate"; with several personas configured
+the estimate is their equal-weight mean, `verdict_source: "mean"`; the older `macro` stance and
+`chair` synthesizer paths remain for comparison) and the violent-move
+watch (`violent_watch.py`, a "the next hour may be wild" line). No gate reads either of them. The
+live wrapper `analyze_pair` hands the fetched frames to the add-ons through a private
+`report._frames` attribute so the panel's price context sees exactly the bars the engine saw; the
+replayable core never sets it.

-**11. Logging.** `signal_log.py` writes one row per live scan into `logs/signals.db` (signal,
+**11. Logging.** `signal_log.py` writes one row per live scan (from the bot and the CLI; the
+dashboard process never writes either database, section 3) into `logs/signals.db` (signal,
 grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the
 gate that removed a setup, a gate-config fingerprint and an engine fingerprint). The engine
 fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log
@@ -203,7 +212,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 |---|---|---|---|
 | `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`) | pair_signals, signal_aggregator, agent_panel, violent_watch, app_config | dashboard_publisher (formatting helpers) |
 | `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
-| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` |
+| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes `signals.db` or `newsdata.db`: `main()` calls `app_config.set_db_writes_for_process(False)`, a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running) | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` (test-enforced) |
 | `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`) | | copied by the publisher |
 | `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher |
 | `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | |
@@ -236,8 +245,8 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 |---|---|
 | `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; optional DeepSeek enrichment; display and warnings only |
 | `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback |
-| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook |
-| `agent_panel.py` | Advisory persona panel (macro / technical / risk / contrarian + optional chair). Attached post-hoc; the alert verdict is the macro persona by default. With `verdict_source: "chair"`, `_apply_oppose_guard` downgrades a chair OPPOSE to NEUTRAL unless the macro persona opposes, or technical opposes while macro is not supportive. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
+| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call |
+| `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
 | `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |

 ### Logging and evaluation
@@ -249,6 +258,9 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 | `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) |
 | `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) |
 | `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) |
+| `scripts/panel_replay.py` | Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (exact `panel_payloads_json` when present, else from the stored JSON blobs, interned news and the kept Oanda cache), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. `--dry-run`, `--max-calls`, `--limit`, `--out`. Read-only on the DB; 4 DeepSeek calls per event |
+| `scripts/build_panel_base_rates.py` | Builds `agent_panel_base_rates.json` (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin |
+| `agent_panel_base_rates.json` | The base-rate table the panel anchors its probabilities to (built 2026-09-17 from the relabelled W2/W3 books) |

 ### Backtesting and research

@@ -269,7 +281,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
 | `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
 | `config.json.example` | The schema of `config.json` (which is gitignored) |
 | `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
-| `test_all.py` | The QA suite (33 modules) |
+| `test_all.py` | The QA suite (34 modules; Module 35 is the agent-panel v2 module) |

 ---

@@ -328,7 +340,7 @@ gates live in `config.py` and the pair factory, never in `config.json`. Blocks (
 | `scanning` | `pairs`, `timeframes` |
 | `pairs."XAU/USD".blocked_hours_utc` | The base list of blocked hours (default 0, 1, 2, 10, 11, 13, 16, 21) |
 | `deepseek` | `api_key`, `model`, `enabled`, `timeout_seconds`, `daily_request_cap` (env `DEEPSEEK_API_KEY`) |
-| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`macro` or `chair`). All defaults = feature off |
+| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`mean`, the default since 2026-09-17; `macro` or `chair` for comparison), `personas` (default `["macro"]`; any of macro / technical / risk / contrarian; `chair` needs two or more). All defaults = feature off |
 | `violent_watch` | `enabled`, `alert_line`, `headsup`, `max_headsup_per_day`, `headsup_min_level` (env `VIOLENT_WATCH_ENABLED`). Off by default |
 | `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
 | `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
@@ -347,7 +359,10 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
 ## 6. Testing

 - `venv/bin/python test_all.py` runs everything with mocked network. Expect
-  `RESULTS: 506 passed, 0 failed, 506 total` (2026-09-15).
+  `RESULTS: 529 passed, 0 failed, 529 total` (2026-09-17). The suite's hermetic app config points
+  both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote
+  fixture headlines into the real `logs/newsdata.db`, and a Module 32 test now pins the isolation. One Twelve Data interval-map test
+  reaches the network and can time out on a slow connection; a rerun clears it.
 - Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
 - When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the
  other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar.
@@ -435,11 +450,18 @@ All of these run offline from the kept cache windows (NumPy CPU backend, 0 cache

 | Book | Window | Trades / R / PF | MD5 of the CSV |
 |---|---|---|---|
-| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `29120a1ee32979431d062f210d41a9d1` |
-| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `d185f099482a07f29502cb5a371a9bbf` |
-| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `6f8f24b49597ced28f50a19e08c3472f` |
-| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `3b45d12bade823178b267c13bc698458` |
-| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `e75adae13c839b15736f69c0114e6aa5` |
+| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `49f8e602fc6ae769713c94ef7b257810` |
+| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `b4aa07e33e17e23774c486a6900d673b` |
+| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `9fd624fb387c36052122bd1830cf9c32` |
+| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `7b848c917dcb5c371ecbb9f8f7292278` |
+| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `186e677fbbd53cca735c94094c23ce05` |
+
+Re-pinned 2026-09-17 for a **label-only** change: the exported `strategy` column now carries the
+generator's own stamp (`XAU_ATR` / `XAU_PB` / `XAU_CP` / `GENERIC`) instead of a tag inferred
+from the first reason's text, which had labelled every gold ATR setup `GENERIC`. All 62 other
+columns of all five books are byte-identical to the previous pins (`29120a1e…`, `d185f099…`,
+`6f8f24b4…`, `3b45d12b…`, `e75adae1…`); trades, R and PF are unchanged. `--strategies` accepts the
+live names (`XAU` still means `XAU_ATR`).

 Other current-HEAD figures at $0.65: M5 W3 (3-year) 870 / +153.6R / PF 1.50; M15 W3 1273 /
 +186.6R / PF 1.39; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54;
@@ -495,6 +517,15 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
 - `pyarrow` (Parquet) is installed in the dev venv only; live code never touches Parquet.
 - The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped
  every H1 and H4 scan once (2026-08-02).
+- The agent panel's price context reads the frames `analyze_pair` stashes on `report._frames`.
+  A report built by hand (scripts, tests) has no frames, so pass them to `_price_context` /
+  `run_panel` explicitly or the context carries geometry and base rate only. `scripts/panel_replay.py`
+  rebuilds them from the kept Oanda cache; a database copied from live is never migrated (read-only),
+  so the replay tolerates missing v2 columns and rebuilds payloads from the stored blobs.
+- The AI estimate is a calibrated base rate, not a filter: on the 37 resolved setups of 2026-09 the
+  v2 personas sat within ±2 points of the class win rate (Brier 0.262 vs 0.263 for the base rate
+  alone, 0.299 for the old macro verdict) and separated nothing. Do not gate on it; do not iterate
+  prompts on that sample again (`backtest/research/panel_v2_2026-09-17.md`).

 ---

@@ -504,6 +535,8 @@ One line per validated change or verdict, newest first. Full evidence in the nam

 | Date | Decision | Where the evidence is |
 |---|---|---|
+| 2026-09-17 | Local news scorer evaluated (the owner's scikit TF-IDF model in `~/Development/ML-NEWS`, trained on 606 DeepSeek gold labels): golden set 39/78 vs DeepSeek 78/78 vs keywords 24/78; out-of-sample agreement with DeepSeek 64% (DeepSeek agrees with itself 90%); cleaning / negation marking / label weighting move nothing; backfilling the 961 missing gold labels lifts macro-headline agreement 38% → 52% but only reshuffles golden-set errors. Verdict: a fallback, not a replacement. SHIPPED to grow a complete corpus: `xau_score` archived for every headline, no-currency headlines scored and archived (never shown), every new headline sent in its cycle (25 per call). Dashboard process made database-read-only (`set_db_writes_for_process`); the bot is the single writer. Books unaffected (news never enters signals) | `~/Development/ML-NEWS/eval/RESULTS_2026-09-17.md` (evaluation scripts live outside the repo by owner instruction) |
+| 2026-09-17 | Live review of two weeks of `signals.db`: live == backtest bar for bar (26/29 trades matched, R within 0.14); the raw advisory −7.15R is cluster re-entries the validated books never take (cooldown view −1.6R, backtester +1.1R on the window, an ordinary fortnight); composite score inversely related to the 24 h move on all timeframes (two-week bear reading, test in-book with `--record-bars`); big-candle gate re-tested and REJECTED again (removes +32R / +60R of net-positive trades). Agent panel v2 SHIPPED: probabilities anchored to book base rates, code-computed decision context, no news doctrine, mean aggregator (`verdict_source: "mean"`), prompt version logged, offline replay harness; owner decision the same day: only the macro persona runs live (`personas: ["macro"]`, 1 call per panel; technical / risk / contrarian carried no information in either evaluation, kept for research); backtester `strategy` tag fixed (label-only, books re-pinned). Delivery findings: dashboard daemon on stale code since Sep 10, bot outage Sep 9 | `backtest/research/signal_log_review_2026-09-17.md`, `backtest/research/panel_v2_2026-09-17.md` |
 | 2026-09-15 | Backtest folder cleanup: `diagnose_filters.py`, `shadow_scan.py`, and the two one-time news scripts removed (last commit with them `931b265`); every pinned CSV, the research output folders, the BTC/ETH cache snapshots, the Tiingo caches and 13 uncited Oanda windows moved to `~/Development/ForexBot2_trash_2026-09-15/` (346 MB) pending the owner's delete; all five reference MD5s reproduced afterwards. Suite 506 / 33 | commit `b0b71e1` |
 | 2026-09-15 | AI market-outlook feature removed (module, hourly Telegram brief, `--outlook`, `outlooks` writer, config block). Legacy config block tolerated with a warning. Books byte-identical | commit `931b265` |
 | 2026-09-13 | Bug audit, 11 delivery/measurement fixes, books byte-identical: NEUTRAL-signal C/B setups now alert (about 17% of the H1 12-month book's R, +3.7R of +22.2R, was never sent); a missing confirmation frame fails the scan; pandas `utcnow` replaced and `pandas<4`, `numpy<3` pinned; advisory outcomes keep locked-profit floors (`GAP_LOCKED`); scan-outcome labeler bounded to 72 h; settlement pause is a scheduled closure; `fetch_range` retries pages and audits gaps; cache-fallback count logged and must be 0; partial `regime_weights_override` raises; the bot scans each bar once and attaches its log handler in `main()` only; the dashboard catches up missed timeframes. Verified sound and unchanged: composite maths, scan-outcome labels, outcome exits, Oanda alignment, no look-ahead | plan file `~/.claude/plans/you-are-python-software-joyful-teacup.md`, commit `2836024` |