harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

news corpus for a local scorer + agent panel v2 + dashboard read-only (2026-09-17); books byte-identical

Browse Source 
Agent panel v2: personas return p_win anchored to the book base rates with a
code-computed decision context; verdict = equal-weight mean (verdict_source
"mean"); only the macro persona runs live (agent_panel.personas ["macro"]);
prompt version and exact payloads logged; offline replay harness
scripts/panel_replay.py; base-rate table agent_panel_base_rates.json
(scripts/build_panel_base_rates.py). Backtester `strategy` column now carries
the generator's own stamp (label-only; five reference books re-pinned, all
62 other columns identical).

News corpus (after evaluating the owner's ML-NEWS scikit model against
DeepSeek: golden set 39/78 vs 78/78, out-of-sample agreement 64% against a
90% self-agreement ceiling; verdict: fallback only, not a replacement):
- NewsSentiment.xau_score + archive column xau_score keep DeepSeek's gold
  score for every headline (930 of the first 1,546 DeepSeek rows had none);
  an existing newsdata.db gains the column in place on first open
- headlines naming no tracked currency are scored and archived
  (currencies_json []) but never returned to the platform
- _apply_llm_sentiment sends every new headline in its cycle, 25 per call;
  a failed call ends the cycle's sending

Dashboard publisher never writes a database: main() calls
app_config.set_db_writes_for_process(False), honoured by signal_log_enabled()
and news_archive_enabled() (owner decision: the bot is the single writer).

Test suite: the hermetic app config now points both writers at a
suite-private temp dir — the RSS fixture tests were writing fixture
headlines into the real logs/newsdata.db — with a Module 32 test pinning it.
529 tests.
This commit is contained in:
Harwinder Sekhon
2026-09-17 19:49:29 -04:00
parent 183df7832b
commit ad14ea4fed
21 changed files with 3803 additions and 259 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+49 -16
View File
@@ -21,7 +21,7 @@ python -m venv venv && venv/bin/pip install -r requirements.txt # Python 3.12;
cp config.json.example config.json # then fill in the keys (see section 5)
venv/bin/python test_all.py # QA suite, no network; prints its own count
# (506 tests / 33 modules as of 2026-09-15)
# (529 tests / 34 modules as of 2026-09-17)
venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail # one signal report in the terminal
# --json full dump, --panel adds the advisory LLM panel,
# --timeframe M5 | M15 | H1 | H4
@@ -164,11 +164,20 @@ later generator shipped a setup.
into the Apache folder.
Two advisory add-ons are attached **after** the scan by the surface, never inside the signal
code: the LLM agent panel (`agent_panel.py`, four DeepSeek personas, verdict AGREE / NEUTRAL /
OPPOSE) and the violent-move watch (`violent_watch.py`, a "the next hour may be wild" line). No
gate reads either of them.
code: the LLM agent panel (`agent_panel.py`; since 2026-09-17 only the **macro** persona runs by
default, `agent_panel.personas: ["macro"]`, one DeepSeek call per panel — the technical, risk and
contrarian personas carried no information in two live evaluations and stay available for research;
the persona returns a probability that the trade closes profitably, anchored to the setup class's
base rate from the validated books, shown as the "AI estimate"; with several personas configured
the estimate is their equal-weight mean, `verdict_source: "mean"`; the older `macro` stance and
`chair` synthesizer paths remain for comparison) and the violent-move
watch (`violent_watch.py`, a "the next hour may be wild" line). No gate reads either of them. The
live wrapper `analyze_pair` hands the fetched frames to the add-ons through a private
`report._frames` attribute so the panel's price context sees exactly the bars the engine saw; the
replayable core never sets it.
**11. Logging.** `signal_log.py` writes one row per live scan into `logs/signals.db` (signal,
**11. Logging.** `signal_log.py` writes one row per live scan (from the bot and the CLI; the
dashboard process never writes either database, section 3) into `logs/signals.db` (signal,
grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the
gate that removed a setup, a gate-config fingerprint and an engine fingerprint). The engine
fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log
@@ -203,7 +212,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
|---|---|---|---|
| `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`) | pair_signals, signal_aggregator, agent_panel, violent_watch, app_config | dashboard_publisher (formatting helpers) |
| `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` |
| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes `signals.db` or `newsdata.db`: `main()` calls `app_config.set_db_writes_for_process(False)`, a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running) | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` (test-enforced) |
| `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`) | | copied by the publisher |
| `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher |
| `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | |
@@ -236,8 +245,8 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
|---|---|
| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; optional DeepSeek enrichment; display and warnings only |
| `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback |
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook |
| `agent_panel.py` | Advisory persona panel (macro / technical / risk / contrarian + optional chair). Attached post-hoc; the alert verdict is the macro persona by default. With `verdict_source: "chair"`, `_apply_oppose_guard` downgrades a chair OPPOSE to NEUTRAL unless the macro persona opposes, or technical opposes while macro is not supportive. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call |
| `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
| `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |
### Logging and evaluation
@@ -249,6 +258,9 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) |
| `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) |
| `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) |
| `scripts/panel_replay.py` | Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (exact `panel_payloads_json` when present, else from the stored JSON blobs, interned news and the kept Oanda cache), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. `--dry-run`, `--max-calls`, `--limit`, `--out`. Read-only on the DB; 4 DeepSeek calls per event |
| `scripts/build_panel_base_rates.py` | Builds `agent_panel_base_rates.json` (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin |
| `agent_panel_base_rates.json` | The base-rate table the panel anchors its probabilities to (built 2026-09-17 from the relabelled W2/W3 books) |
### Backtesting and research
@@ -269,7 +281,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
| `config.json.example` | The schema of `config.json` (which is gitignored) |
| `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
| `test_all.py` | The QA suite (33 modules) |
| `test_all.py` | The QA suite (34 modules; Module 35 is the agent-panel v2 module) |
---
@@ -328,7 +340,7 @@ gates live in `config.py` and the pair factory, never in `config.json`. Blocks (
| `scanning` | `pairs`, `timeframes` |
| `pairs."XAU/USD".blocked_hours_utc` | The base list of blocked hours (default 0, 1, 2, 10, 11, 13, 16, 21) |
| `deepseek` | `api_key`, `model`, `enabled`, `timeout_seconds`, `daily_request_cap` (env `DEEPSEEK_API_KEY`) |
| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`macro` or `chair`). All defaults = feature off |
| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`mean`, the default since 2026-09-17; `macro` or `chair` for comparison), `personas` (default `["macro"]`; any of macro / technical / risk / contrarian; `chair` needs two or more). All defaults = feature off |
| `violent_watch` | `enabled`, `alert_line`, `headsup`, `max_headsup_per_day`, `headsup_min_level` (env `VIOLENT_WATCH_ENABLED`). Off by default |
| `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
| `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
@@ -347,7 +359,10 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
## 6. Testing
- `venv/bin/python test_all.py` runs everything with mocked network. Expect
`RESULTS: 506 passed, 0 failed, 506 total` (2026-09-15).
`RESULTS: 529 passed, 0 failed, 529 total` (2026-09-17). The suite's hermetic app config points
both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote
fixture headlines into the real `logs/newsdata.db`, and a Module 32 test now pins the isolation. One Twelve Data interval-map test
reaches the network and can time out on a slow connection; a rerun clears it.
- Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
- When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the
other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar.
@@ -435,11 +450,18 @@ All of these run offline from the kept cache windows (NumPy CPU backend, 0 cache
| Book | Window | Trades / R / PF | MD5 of the CSV |
|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `29120a1ee32979431d062f210d41a9d1` |
| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `d185f099482a07f29502cb5a371a9bbf` |
| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `6f8f24b49597ced28f50a19e08c3472f` |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `3b45d12bade823178b267c13bc698458` |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `e75adae13c839b15736f69c0114e6aa5` |
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `49f8e602fc6ae769713c94ef7b257810` |
| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `b4aa07e33e17e23774c486a6900d673b` |
| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `9fd624fb387c36052122bd1830cf9c32` |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `7b848c917dcb5c371ecbb9f8f7292278` |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `186e677fbbd53cca735c94094c23ce05` |
Re-pinned 2026-09-17 for a **label-only** change: the exported `strategy` column now carries the
generator's own stamp (`XAU_ATR` / `XAU_PB` / `XAU_CP` / `GENERIC`) instead of a tag inferred
from the first reason's text, which had labelled every gold ATR setup `GENERIC`. All 62 other
columns of all five books are byte-identical to the previous pins (`29120a1e…`, `d185f099…`,
`6f8f24b4…`, `3b45d12b…`, `e75adae1…`); trades, R and PF are unchanged. `--strategies` accepts the
live names (`XAU` still means `XAU_ATR`).
Other current-HEAD figures at $0.65: M5 W3 (3-year) 870 / +153.6R / PF 1.50; M15 W3 1273 /
+186.6R / PF 1.39; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54;
@@ -495,6 +517,15 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
- `pyarrow` (Parquet) is installed in the dev venv only; live code never touches Parquet.
- The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped
every H1 and H4 scan once (2026-08-02).
- The agent panel's price context reads the frames `analyze_pair` stashes on `report._frames`.
A report built by hand (scripts, tests) has no frames, so pass them to `_price_context` /
`run_panel` explicitly or the context carries geometry and base rate only. `scripts/panel_replay.py`
rebuilds them from the kept Oanda cache; a database copied from live is never migrated (read-only),
so the replay tolerates missing v2 columns and rebuilds payloads from the stored blobs.
- The AI estimate is a calibrated base rate, not a filter: on the 37 resolved setups of 2026-09 the
v2 personas sat within ±2 points of the class win rate (Brier 0.262 vs 0.263 for the base rate
alone, 0.299 for the old macro verdict) and separated nothing. Do not gate on it; do not iterate
prompts on that sample again (`backtest/research/panel_v2_2026-09-17.md`).
---
@@ -504,6 +535,8 @@ One line per validated change or verdict, newest first. Full evidence in the nam
| Date | Decision | Where the evidence is |
|---|---|---|
| 2026-09-17 | Local news scorer evaluated (the owner's scikit TF-IDF model in `~/Development/ML-NEWS`, trained on 606 DeepSeek gold labels): golden set 39/78 vs DeepSeek 78/78 vs keywords 24/78; out-of-sample agreement with DeepSeek 64% (DeepSeek agrees with itself 90%); cleaning / negation marking / label weighting move nothing; backfilling the 961 missing gold labels lifts macro-headline agreement 38% → 52% but only reshuffles golden-set errors. Verdict: a fallback, not a replacement. SHIPPED to grow a complete corpus: `xau_score` archived for every headline, no-currency headlines scored and archived (never shown), every new headline sent in its cycle (25 per call). Dashboard process made database-read-only (`set_db_writes_for_process`); the bot is the single writer. Books unaffected (news never enters signals) | `~/Development/ML-NEWS/eval/RESULTS_2026-09-17.md` (evaluation scripts live outside the repo by owner instruction) |
| 2026-09-17 | Live review of two weeks of `signals.db`: live == backtest bar for bar (26/29 trades matched, R within 0.14); the raw advisory −7.15R is cluster re-entries the validated books never take (cooldown view −1.6R, backtester +1.1R on the window, an ordinary fortnight); composite score inversely related to the 24 h move on all timeframes (two-week bear reading, test in-book with `--record-bars`); big-candle gate re-tested and REJECTED again (removes +32R / +60R of net-positive trades). Agent panel v2 SHIPPED: probabilities anchored to book base rates, code-computed decision context, no news doctrine, mean aggregator (`verdict_source: "mean"`), prompt version logged, offline replay harness; owner decision the same day: only the macro persona runs live (`personas: ["macro"]`, 1 call per panel; technical / risk / contrarian carried no information in either evaluation, kept for research); backtester `strategy` tag fixed (label-only, books re-pinned). Delivery findings: dashboard daemon on stale code since Sep 10, bot outage Sep 9 | `backtest/research/signal_log_review_2026-09-17.md`, `backtest/research/panel_v2_2026-09-17.md` |
| 2026-09-15 | Backtest folder cleanup: `diagnose_filters.py`, `shadow_scan.py`, and the two one-time news scripts removed (last commit with them `931b265`); every pinned CSV, the research output folders, the BTC/ETH cache snapshots, the Tiingo caches and 13 uncited Oanda windows moved to `~/Development/ForexBot2_trash_2026-09-15/` (346 MB) pending the owner's delete; all five reference MD5s reproduced afterwards. Suite 506 / 33 | commit `b0b71e1` |
| 2026-09-15 | AI market-outlook feature removed (module, hourly Telegram brief, `--outlook`, `outlooks` writer, config block). Legacy config block tolerated with a warning. Books byte-identical | commit `931b265` |
| 2026-09-13 | Bug audit, 11 delivery/measurement fixes, books byte-identical: NEUTRAL-signal C/B setups now alert (about 17% of the H1 12-month book's R, +3.7R of +22.2R, was never sent); a missing confirmation frame fails the scan; pandas `utcnow` replaced and `pandas<4`, `numpy<3` pinned; advisory outcomes keep locked-profit floors (`GAP_LOCKED`); scan-outcome labeler bounded to 72 h; settlement pause is a scheduled closure; `fetch_range` retries pages and audits gaps; cache-fallback count logged and must be 0; partial `regime_weights_override` raises; the bot scans each bar once and attaches its log handler in `main()` only; the dashboard catches up missed timeframes. Verified sound and unchanged: composite maths, scan-outcome labels, outcome exits, Oanda alignment, no look-ahead | plan file `~/.claude/plans/you-are-python-software-joyful-teacup.md`, commit `2836024` |
agent_panel.py
+636 -145
View File
File diff suppressed because it is too large Load Diff
agent_panel_base_rates.json
+398
View File
@@ -0,0 +1,398 @@
{
"schema": "agent-panel-base-rates-v1",
"generated_at": "2026-09-17T15:26:15+00:00",
"key": "timeframe|strategy|grade|direction ('*' = any)",
"min_n_for_exact_class": 30,
"books": {
"5min": {
"file": "m5_w2.csv",
"md5": "49f8e602fc6ae769713c94ef7b257810",
"trades": 781,
"first_entry": "2024-09-06 18:00:00+00:00",
"last_entry": "2026-09-04 15:15:00+00:00"
},
"15min": {
"file": "m15_w2.csv",
"md5": "b4aa07e33e17e23774c486a6900d673b",
"trades": 931,
"first_entry": "2024-09-05 06:15:00+00:00",
"last_entry": "2026-09-04 12:45:00+00:00"
},
"1h": {
"file": "h1_w3.csv",
"md5": "7b848c917dcb5c371ecbb9f8f7292278",
"trades": 253,
"first_entry": "2023-09-20 18:00:00+00:00",
"last_entry": "2026-08-21 07:00:00+00:00"
},
"4h": {
"file": "h4_w3.csv",
"md5": "186e677fbbd53cca735c94094c23ce05",
"trades": 114,
"first_entry": "2023-10-02 08:00:00+00:00",
"last_entry": "2026-09-01 12:00:00+00:00"
}
},
"classes": {
"15min|*|*|*": {
"n": 931,
"win_rate": 60.47,
"mean_r": 0.157,
"tp1_reach_rate": 52.31,
"median_bars_held": 30,
"book": "m15_w2.csv"
},
"15min|*|*|BUY": {
"n": 635,
"win_rate": 63.31,
"mean_r": 0.2309,
"tp1_reach_rate": 53.54,
"median_bars_held": 33,
"book": "m15_w2.csv"
},
"15min|*|*|SELL": {
"n": 296,
"win_rate": 54.39,
"mean_r": -0.0016,
"tp1_reach_rate": 49.66,
"median_bars_held": 20,
"book": "m15_w2.csv"
},
"15min|GENERIC|A+|BUY": {
"n": 2,
"win_rate": 0.0,
"mean_r": -0.544,
"tp1_reach_rate": 0.0,
"median_bars_held": 32,
"book": "m15_w2.csv"
},
"15min|GENERIC|A+|SELL": {
"n": 1,
"win_rate": 0.0,
"mean_r": -1.0276,
"tp1_reach_rate": 0.0,
"median_bars_held": 34,
"book": "m15_w2.csv"
},
"15min|GENERIC|A|BUY": {
"n": 208,
"win_rate": 67.31,
"mean_r": 0.3158,
"tp1_reach_rate": 59.13,
"median_bars_held": 44,
"book": "m15_w2.csv"
},
"15min|GENERIC|A|SELL": {
"n": 84,
"win_rate": 52.38,
"mean_r": -0.0288,
"tp1_reach_rate": 42.86,
"median_bars_held": 43,
"book": "m15_w2.csv"
},
"15min|GENERIC|B|BUY": {
"n": 282,
"win_rate": 62.41,
"mean_r": 0.1869,
"tp1_reach_rate": 46.1,
"median_bars_held": 38,
"book": "m15_w2.csv"
},
"15min|GENERIC|B|SELL": {
"n": 120,
"win_rate": 57.5,
"mean_r": 0.0242,
"tp1_reach_rate": 45.0,
"median_bars_held": 27,
"book": "m15_w2.csv"
},
"15min|GENERIC|C|BUY": {
"n": 60,
"win_rate": 65.0,
"mean_r": 0.395,
"tp1_reach_rate": 60.0,
"median_bars_held": 24,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A+|BUY": {
"n": 3,
"win_rate": 66.67,
"mean_r": 0.2684,
"tp1_reach_rate": 33.33,
"median_bars_held": 13,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A+|SELL": {
"n": 13,
"win_rate": 69.23,
"mean_r": 0.4437,
"tp1_reach_rate": 61.54,
"median_bars_held": 5,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A|BUY": {
"n": 80,
"win_rate": 56.25,
"mean_r": 0.06,
"tp1_reach_rate": 62.5,
"median_bars_held": 10,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A|SELL": {
"n": 78,
"win_rate": 50.0,
"mean_r": -0.073,
"tp1_reach_rate": 62.82,
"median_bars_held": 6,
"book": "m15_w2.csv"
},
"1h|*|*|*": {
"n": 253,
"win_rate": 64.43,
"mean_r": 0.2768,
"tp1_reach_rate": 74.31,
"median_bars_held": 29,
"book": "h1_w3.csv"
},
"1h|*|*|BUY": {
"n": 253,
"win_rate": 64.43,
"mean_r": 0.2768,
"tp1_reach_rate": 74.31,
"median_bars_held": 29,
"book": "h1_w3.csv"
},
"1h|GENERIC|A+|BUY": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.1689,
"tp1_reach_rate": 100.0,
"median_bars_held": 45,
"book": "h1_w3.csv"
},
"1h|GENERIC|A|BUY": {
"n": 57,
"win_rate": 66.67,
"mean_r": 0.2243,
"tp1_reach_rate": 75.44,
"median_bars_held": 45,
"book": "h1_w3.csv"
},
"1h|GENERIC|B|BUY": {
"n": 112,
"win_rate": 63.39,
"mean_r": 0.2432,
"tp1_reach_rate": 74.11,
"median_bars_held": 38,
"book": "h1_w3.csv"
},
"1h|GENERIC|C|BUY": {
"n": 26,
"win_rate": 73.08,
"mean_r": 0.6826,
"tp1_reach_rate": 80.77,
"median_bars_held": 29,
"book": "h1_w3.csv"
},
"1h|XAU_ATR|A+|BUY": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.1181,
"tp1_reach_rate": 0.0,
"median_bars_held": 11,
"book": "h1_w3.csv"
},
"1h|XAU_ATR|A|BUY": {
"n": 56,
"win_rate": 62.5,
"mean_r": 0.224,
"tp1_reach_rate": 71.43,
"median_bars_held": 8,
"book": "h1_w3.csv"
},
"4h|*|*|*": {
"n": 114,
"win_rate": 57.89,
"mean_r": 0.1902,
"tp1_reach_rate": 67.54,
"median_bars_held": 12,
"book": "h4_w3.csv"
},
"4h|*|*|BUY": {
"n": 94,
"win_rate": 61.7,
"mean_r": 0.2409,
"tp1_reach_rate": 70.21,
"median_bars_held": 9,
"book": "h4_w3.csv"
},
"4h|*|*|SELL": {
"n": 20,
"win_rate": 40.0,
"mean_r": -0.0484,
"tp1_reach_rate": 55.0,
"median_bars_held": 20,
"book": "h4_w3.csv"
},
"4h|GENERIC|A|BUY": {
"n": 4,
"win_rate": 75.0,
"mean_r": 0.9295,
"tp1_reach_rate": 75.0,
"median_bars_held": 18,
"book": "h4_w3.csv"
},
"4h|GENERIC|A|SELL": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.2277,
"tp1_reach_rate": 100.0,
"median_bars_held": 29,
"book": "h4_w3.csv"
},
"4h|GENERIC|B|BUY": {
"n": 39,
"win_rate": 79.49,
"mean_r": 0.4706,
"tp1_reach_rate": 82.05,
"median_bars_held": 27,
"book": "h4_w3.csv"
},
"4h|GENERIC|B|SELL": {
"n": 12,
"win_rate": 25.0,
"mean_r": -0.4106,
"tp1_reach_rate": 41.67,
"median_bars_held": 40,
"book": "h4_w3.csv"
},
"4h|GENERIC|C|BUY": {
"n": 5,
"win_rate": 40.0,
"mean_r": -0.3577,
"tp1_reach_rate": 60.0,
"median_bars_held": 8,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A+|BUY": {
"n": 4,
"win_rate": 50.0,
"mean_r": 0.017,
"tp1_reach_rate": 50.0,
"median_bars_held": 4,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A+|SELL": {
"n": 2,
"win_rate": 100.0,
"mean_r": 0.6889,
"tp1_reach_rate": 50.0,
"median_bars_held": 4,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A|BUY": {
"n": 42,
"win_rate": 47.62,
"mean_r": 0.0547,
"tp1_reach_rate": 61.9,
"median_bars_held": 7,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A|SELL": {
"n": 5,
"win_rate": 60.0,
"mean_r": 0.5618,
"tp1_reach_rate": 80.0,
"median_bars_held": 14,
"book": "h4_w3.csv"
},
"5min|*|*|*": {
"n": 781,
"win_rate": 52.88,
"mean_r": 0.187,
"tp1_reach_rate": 56.34,
"median_bars_held": 34,
"book": "m5_w2.csv"
},
"5min|*|*|BUY": {
"n": 683,
"win_rate": 53.15,
"mean_r": 0.201,
"tp1_reach_rate": 54.61,
"median_bars_held": 33,
"book": "m5_w2.csv"
},
"5min|*|*|SELL": {
"n": 98,
"win_rate": 51.02,
"mean_r": 0.0894,
"tp1_reach_rate": 68.37,
"median_bars_held": 44,
"book": "m5_w2.csv"
},
"5min|GENERIC|A|BUY": {
"n": 139,
"win_rate": 61.15,
"mean_r": 0.2662,
"tp1_reach_rate": 74.82,
"median_bars_held": 77,
"book": "m5_w2.csv"
},
"5min|GENERIC|B|BUY": {
"n": 153,
"win_rate": 50.98,
"mean_r": 0.2315,
"tp1_reach_rate": 72.55,
"median_bars_held": 61,
"book": "m5_w2.csv"
},
"5min|GENERIC|B|SELL": {
"n": 98,
"win_rate": 51.02,
"mean_r": 0.0894,
"tp1_reach_rate": 68.37,
"median_bars_held": 44,
"book": "m5_w2.csv"
},
"5min|GENERIC|C|BUY": {
"n": 21,
"win_rate": 47.62,
"mean_r": -0.0144,
"tp1_reach_rate": 57.14,
"median_bars_held": 35,
"book": "m5_w2.csv"
},
"5min|XAU_ATR|A+|BUY": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.0329,
"tp1_reach_rate": 0.0,
"median_bars_held": 12,
"book": "m5_w2.csv"
},
"5min|XAU_ATR|A|BUY": {
"n": 31,
"win_rate": 54.84,
"mean_r": 0.288,
"tp1_reach_rate": 67.74,
"median_bars_held": 12,
"book": "m5_w2.csv"
},
"5min|XAU_CP|A|BUY": {
"n": 17,
"win_rate": 41.18,
"mean_r": 0.2498,
"tp1_reach_rate": 29.41,
"median_bars_held": 26,
"book": "m5_w2.csv"
},
"5min|XAU_CP|B|BUY": {
"n": 321,
"win_rate": 51.71,
"mean_r": 0.1621,
"tp1_reach_rate": 37.38,
"median_bars_held": 19,
"book": "m5_w2.csv"
}
}
}
analyze_signal_log.py
+140
View File
@@ -794,6 +794,145 @@ def persona_scorecard(events: Sequence[SignalEvent], db_path: str) -> List[dict]
for (p, s), v in sorted(agg.items())]
# ── Probability scorecard (agent panel v2, 2026-09-17) ───────────────
#
# Scores the panel as a PREDICTOR of the trade closing profitably, per
# prompt version: Brier score and log-loss per persona and for the mean,
# calibration by probability band, discrimination (win rate / mean R by
# band), the constant-voter rate, rank correlation of p with R, and the
# counterfactual books. v1 rows (stance + confidence) are mapped onto the
# same 0-100 probability scale with agent_panel.pseudo_probability so the
# two eras are comparable; nothing here is a decision (owner rule: no
# persona-weight tuning before ~300 graded outcomes).
def _p_bands(pairs: Sequence[Tuple[float, float]]) -> Dict[str, dict]:
from agent_panel import P_OPPOSE, P_SUPPORT
out = {}
for label, pick in (("<=40", lambda p: p <= P_OPPOSE),
("40-60", lambda p: P_OPPOSE < p < P_SUPPORT),
(">=60", lambda p: p >= P_SUPPORT)):
sel = [(p, r) for p, r in pairs if pick(p)]
out[label] = ({"n": len(sel),
"mean_p": sum(p for p, _ in sel) / len(sel),
"win_rate": sum(1 for _, r in sel if r > 0) / len(sel),
"mean_r": sum(r for _, r in sel) / len(sel),
"sum_r": sum(r for _, r in sel)}
if sel else {"n": 0})
return out
def _score_pairs(pairs: Sequence[Tuple[float, float]]) -> Optional[dict]:
from agent_panel import P_OPPOSE, P_SUPPORT, derive_stance
if not pairs:
return None
ys = [1.0 if r > 0 else 0.0 for _, r in pairs]
ps = [p / 100.0 for p, _ in pairs]
eps = 1e-3
brier = sum((p - y) ** 2 for p, y in zip(ps, ys)) / len(ps)
ll = -sum(y * math.log(min(1 - eps, max(eps, p)))
+ (1 - y) * math.log(min(1 - eps, max(eps, 1 - p)))
for p, y in zip(ps, ys)) / len(ps)
stances = [derive_stance(p) for p, _ in pairs]
modal = max(set(stances), key=stances.count)
rho = None
if len(pairs) >= 5:
a = pd.Series([p for p, _ in pairs]).rank()
b = pd.Series([r for _, r in pairs]).rank()
c = a.corr(b)
rho = None if c != c else float(c)
return {"n": len(pairs), "brier": brier, "logloss": ll,
"mean_p": sum(p for p, _ in pairs) / len(pairs),
"win_rate": sum(ys) / len(ys),
"modal": modal, "constant_rate": stances.count(modal) / len(stances),
"rho_p_r": rho, "bands": _p_bands(pairs),
"sum_all": sum(r for _, r in pairs),
"sum_skip_oppose": sum(r for p, r in pairs if p > P_OPPOSE),
"sum_support_only": sum(r for p, r in pairs if p >= P_SUPPORT)}
def probability_scorecard(events: Sequence[SignalEvent], db_path: str
) -> Dict[str, Dict[str, Optional[dict]]]:
"""{prompt_version: {"mean": stats, "<persona>": stats, "base_rate": stats}}
over resolved events (one vote per event, first row with a panel)."""
from agent_panel import pseudo_probability
by_row = _row_index(events)
con = open_readonly(db_path)
have = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
cols = ["id", "panel_takes_json"]
for c in ("panel_probs_json", "panel_prompt_version", "panel_probability",
"panel_context_json"):
cols.append(c if c in have else f"NULL AS {c}")
per_version: Dict[str, Dict[str, List[Tuple[float, float]]]] = {}
seen: set = set()
try:
for rid, takes_json, probs_json, version, p_mean, ctx_json in con.execute(
f"SELECT {', '.join(cols)} FROM signals"
" WHERE panel_takes_json IS NOT NULL ORDER BY id"):
ev = by_row.get(rid)
if ev is None or ev.r_headline is None:
continue
ekey = ev.rows[0].id if ev.rows else rid
if ekey in seen:
continue
seen.add(ekey)
r = ev.r_headline
version = version or "v1"
bucket = per_version.setdefault(version, {})
probs: Dict[str, float] = {}
try:
source = json.loads(probs_json) if probs_json else json.loads(takes_json)
except (TypeError, ValueError):
continue
for t in source or []:
p = t.get("p_win")
if p is None and t.get("stance") in ("support", "oppose", "neutral"):
p = pseudo_probability(t["stance"], float(t.get("confidence") or 0.0))
if p is not None:
probs[str(t.get("persona"))] = float(p)
bucket.setdefault(str(t.get("persona")), []).append((float(p), r))
if p_mean is not None:
bucket.setdefault("mean", []).append((float(p_mean), r))
elif len(probs) >= 3:
bucket.setdefault("mean", []).append(
(sum(probs.values()) / len(probs), r))
try:
base = (json.loads(ctx_json) if ctx_json else {}).get("base_rate") or {}
if base.get("win_rate") is not None:
bucket.setdefault("base_rate", []).append((float(base["win_rate"]), r))
except (TypeError, ValueError, AttributeError):
pass
finally:
con.close()
return {v: {k: _score_pairs(pairs) for k, pairs in sorted(b.items())}
for v, b in sorted(per_version.items())}
def print_probability_scorecard(card: Dict[str, Dict[str, Optional[dict]]]) -> None:
print("\n── Probability scorecard (Brier: lower is better; 0.25 = coin flip) " + "─" * 6)
if not card:
print(" no resolved events with a panel yet")
return
for version, preds in card.items():
print(f" prompt version {version}")
print(f" {'predictor':12s} {'n':>3s} {'Brier':>6s} {'logloss':>7s} {'mean p':>6s} "
f"{'win%':>5s} {'modal':>8s} {'const%':>6s} {'rho(p,R)':>8s} "
f"{'R all':>7s} {'skip<=40':>8s} {'>=60only':>8s}")
for name, st in preds.items():
if not st:
continue
rho = f"{st['rho_p_r']:+8.2f}" if st["rho_p_r"] is not None else " -"
print(f" {name:12s} {st['n']:3d} {st['brier']:6.3f} {st['logloss']:7.3f} "
f"{st['mean_p']:6.1f} {100 * st['win_rate']:5.0f} {st['modal']:>8s} "
f"{100 * st['constant_rate']:6.0f} {rho} {st['sum_all']:+7.2f} "
f"{st['sum_skip_oppose']:+8.2f} {st['sum_support_only']:+8.2f}")
mean = preds.get("mean")
if mean:
cells = " ".join(
f"{k}: n={v['n']} p={v['mean_p']:.0f} win={100 * v['win_rate']:.0f}% R={v['mean_r']:+.2f}"
if v["n"] else f"{k}: n=0" for k, v in mean["bands"].items())
print(f" calibration of the mean: {cells}")
# ── News sentiment lead/lag ───────────────────────────────────────────
@@ -1309,6 +1448,7 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
for p in persona_scorecard(events, args.db):
print(f" {p['persona']:11} {p['stance']:8} n={p['n']:3d} "
f"mean {p['mean_r']:+.3f}R win {100 * p['win_rate']:.0f}%")
print_probability_scorecard(probability_scorecard(events, args.db))
if args.news:
print_news(news_lead_lag(rows, streams),
news_extremes(rows, streams),
app_config.py
+47 -4
View File
@@ -105,7 +105,7 @@ _KNOWN_AGENT_PANEL = {
"enabled", "telegram", "dashboard", "min_quality",
"max_panels_per_day", "reuse_ttl_seconds",
"candles_enabled", "candle_bars_primary", "candle_bars_confirm",
"verdict_source",
"verdict_source", "personas",
}
_KNOWN_VIOLENT_WATCH = {"enabled", "alert_line", "headsup",
"max_headsup_per_day", "headsup_min_level"}
@@ -330,10 +330,27 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
f"config.json: agent_panel.{key} must be an integer "
"between 1 and 100")
if "verdict_source" in panel and panel["verdict_source"] not in \
("macro", "chair"):
("mean", "macro", "chair"):
raise AppConfigError(
'config.json: agent_panel.verdict_source must be "macro" or '
'"chair"')
'config.json: agent_panel.verdict_source must be "mean", '
'"macro" or "chair"')
if "personas" in panel:
names = panel["personas"]
valid = ("macro", "technical", "risk", "contrarian")
if (not isinstance(names, list) or not names
or any(not isinstance(n, str) or n not in valid for n in names)
or len(set(names)) != len(names)):
raise AppConfigError(
"config.json: agent_panel.personas must be a non-empty list "
f"of distinct names from {valid}")
if panel.get("verdict_source") == "chair" and len(names) < 2:
raise AppConfigError(
'config.json: agent_panel.verdict_source "chair" needs at '
"least two personas")
if panel.get("verdict_source") == "macro" and "macro" not in names:
raise AppConfigError(
'config.json: agent_panel.verdict_source "macro" needs the '
"macro persona in agent_panel.personas")
app.agent_panel_settings = dict(panel)
if "market_outlook" in raw:
@@ -492,10 +509,34 @@ def violent_watch_enabled(app: Optional[AppConfig] = None) -> bool:
return bool(app.violent_watch_settings.get("enabled", False))
# ── Process-level database write switch ──────────────────────────────
#: True in every process by default. The dashboard publisher turns it
#: off at startup (owner decision 2026-09-17): it is not always running,
#: so the Telegram bot must be the single writer of signals.db and
#: newsdata.db — two writers were also two processes contending for the
#: same WAL files. Every SQLite writer (signal_log, news_archive) reads
#: the switch through signal_log_enabled() / news_archive_enabled(), so
#: a switched-off process cannot write whatever config.json or the
#: environment says. Read-only users of the databases are unaffected.
_process_db_writes = True
def set_db_writes_for_process(enabled: bool) -> None:
global _process_db_writes
_process_db_writes = bool(enabled)
def db_writes_enabled() -> bool:
return _process_db_writes
# ── Signal-log resolution helpers (env → config.json → default) ──────
def signal_log_enabled(app: Optional[AppConfig] = None) -> bool:
"""Default ON — signal history is the platform's audit trail."""
if not _process_db_writes:
return False
env = os.environ.get("SIGNAL_LOG_ENABLED")
if env:
return env.strip().lower() in ("1", "true", "yes", "on")
@@ -535,6 +576,8 @@ def news_archive_enabled(app: Optional[AppConfig] = None) -> bool:
scorer and cannot be backfilled once a fetch window has passed. It
lives in its own DB so a signals.db rotation cannot take it with it
(which is how three weeks of news were lost on 2026-09-04)."""
if not _process_db_writes:
return False
env = os.environ.get("NEWS_ARCHIVE_ENABLED")
if env:
return env.strip().lower() in ("1", "true", "yes", "on")
backtest/gpu_backtest.py
+15 -12
View File
@@ -1634,16 +1634,16 @@ def _replay_span(span: Tuple[int, int]):
if report.signal_quality not in bt_cfg.quality_filter:
continue
# Strategy tag (from reasons[0] which the aggregator pre-pends).
tag = "GENERIC"
first_bull = report.bullish_reasons[0] if report.bullish_reasons else ""
first_bear = report.bearish_reasons[0] if report.bearish_reasons else ""
if "XAU PULLBACK" in first_bull or "XAU PULLBACK" in first_bear:
tag = "XAU_PB"
elif "XAU CHART PATTERN" in first_bull or "XAU CHART PATTERN" in first_bear:
tag = "XAU_CP"
elif "XAU STRATEGY" in first_bull:
tag = "XAU"
# Strategy tag: the generator's own stamp (XAU_ATR / XAU_PB /
# XAU_CP / GENERIC), the same string the live log stores in
# setup_json.strategy. Until 2026-09-17 this was inferred from the
# text of the FIRST reason ("XAU STRATEGY" -> "XAU"), which
# labelled every gold ATR setup whose first reason was another
# indicator — and every SELL — as GENERIC (live review §6.3).
# Label-only: entries, exits and R never depended on it, so the
# books are trade-identical; the CSV `strategy` column changed
# and the reference MD5s were re-pinned with that proof.
tag = str(getattr(report.trade_setup, "strategy", "") or "GENERIC")
if bt_cfg.strategies_only and tag not in bt_cfg.strategies_only:
continue
@@ -2378,7 +2378,9 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
"entries/stops/gates unchanged.")
p.add_argument("--quality", default="A+,A,B,C", help="Comma-separated allowed grades")
p.add_argument("--strategies", default="",
help="Comma-separated strategy tags to include (XAU,XAU_PB,XAU_CP,GENERIC). Empty = all.")
help="Comma-separated strategy tags to include "
"(XAU_ATR,XAU_PB,XAU_CP,GENERIC; the legacy name XAU "
"means XAU_ATR). Empty = all.")
p.add_argument("--cp", action="store_true",
help="Force-enable the chart-pattern entry (XAU_CP) regardless "
"of the factory default — for validating the candidate.")
@@ -2566,7 +2568,8 @@ def main(argv: Optional[List[str]] = None) -> int:
max_hold_bars=args.max_hold_bars,
use_cache=not args.no_cache,
quality_filter=[q.strip() for q in args.quality.split(",") if q.strip()],
strategies_only=[s.strip() for s in args.strategies.split(",") if s.strip()],
strategies_only=[("XAU_ATR" if s.strip() == "XAU" else s.strip())
for s in args.strategies.split(",") if s.strip()],
spread=spread,
workers=args.workers,
fixed_rr_tp=args.fixed_rr_tp or 0.0,
backtest/research/panel_v2_2026-09-17.md
+181
View File
@@ -0,0 +1,181 @@
# Agent panel v2 — probabilities anchored to base rates — 2026-09-17
**Why.** The live review of 2026-09-04 → 09-17 (`signal_log_review_2026-09-17.md`, §5 and §7.1)
found the panel's verdict was tilted the wrong way: the macro persona's stance was the verdict,
its only input was headlines (a one-day-lagging echo of price, `news_sentiment_value_2026-09-04.md`),
opposed setups reached their first target more often than agreed ones (54% vs 42%), three of four
personas voted the same way on nearly every setup (technical 79% support, risk 79% oppose,
contrarian 65% oppose), and the stated confidence was uncalibrated. The chair had already been
shown anti-informative on 2026-09-03. Nothing versioned the prompts, so no change could be measured.
**What shipped (code, all advisory-only; the panel still never runs inside
`analyze_from_dataframes` and no gate reads it).**
| Piece | Where |
|---|---|
| Every persona returns `p_win` (0-100, the chance the trade closes profitably under its trailing exit) plus one `reason_for` and one `reason_against`; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose) | `agent_panel.py` `_TAKE_CONTRACT`, `derive_stance`, `_validate_take` (v1 stance+confidence replies still accepted and mapped to a pseudo-probability) |
| Verdict = equal-weight mean of the personas' `p_win`, quorum 3 of 4, no LLM aggregator (`verdict_source: "mean"`, the new default; `macro` and `chair` kept for comparison) | `mean_verdict`, `run_panel`, `app_config.py` |
| A code-computed `context` block in every payload: stop / target in ATR and R, cost-to-risk, the class **base rate** from the validated books, prior move 4 h / 24 h in ATR, distance from the 24 h extreme and its age, `move_started_at`, largest bar of the last 3 in ATR, signal-bar body, event timing against the class's expected hold, penalty tags with the book's verdict on them | `_price_context`, `_base_rate`, `agent_panel_base_rates.json` (built by `scripts/build_panel_base_rates.py` from the MD5-verified books) |
| Macro persona: no news doctrine; judges whether news is NEW relative to `move_started_at`; receives the ATR-normalised candles (as does the contrarian) | `_MACRO_SYSTEM`, `_panel_jobs` |
| Technical persona: indicator agreement declared uninformative; judges higher-timeframe conflict and key levels. Risk: only setup-specific risks versus the class. Contrarian: the engine's own warnings are not evidence | `_TECH_SYSTEM`, `_RISK_SYSTEM`, `_CONTRARIAN_SYSTEM` |
| `PANEL_PROMPT_VERSION` logged with every verdict and folded into the payload hash; per-persona probabilities and reasons, the exact per-persona payloads and the context logged (`panel_prompt_version`, `panel_probability`, `panel_probs_json`, `panel_payloads_json`, `panel_context_json`) | `signal_log.py` additive migration, `update_panel`, `log_panel_update` |
| Live frames handed to the add-ons through `report._frames` in `analyze_pair` (never in the core) | `signal_aggregator.py` |
| Plain-language "AI estimate" line on Telegram, CLI and dashboard; legacy sources render as before | `telegram_bot._panel_estimate_lines`, `main.py`, `static/dashboard.js` |
| Scorecard: Brier, log-loss, calibration bands, discrimination, constant-voter rate, ρ(p, R), counterfactual books, split by prompt version | `analyze_signal_log.probability_scorecard` (printed by `--stored-candles`) |
| Offline replay harness: rebuilds each persona's payload for every resolved logged setup (exact payloads when logged, else from the stored blobs, interned news and the kept Oanda cache, events evaluated as of the signal bar), runs the current prompts, scores them against the outcome and against the logged v1 takes and the base-rate predictor | `scripts/panel_replay.py` (`--dry-run`, `--max-calls`) |
| Backtester export `strategy` tag fixed to the generator's own stamp (label-only; every gold ATR setup was exported as `GENERIC`) | `backtest/gpu_backtest.py`; books re-pinned, see below |
Tests: `test_all.py` Module 35 (15 tests) plus the extended Module 23 parity test; suite
`RESULTS: 521 passed, 0 failed, 521 total`.
## Reference books re-pinned (label-only)
The five books were regenerated with the committed code (stashed) and with the fix, on the same
cache. Dropping the `strategy` column, every book is byte-identical to its previous pin; trades, R
and PF are unchanged. New labels: M15 W2 174 `XAU_ATR` / 757 `GENERIC`; M5 W2 32 / 411 / 338
`XAU_CP`; M15 OOS 1 / 354; H1 W3 57 / 196; H4 W3 53 / 61.
| Book | old MD5 | new MD5 |
|---|---|---|
| M5 W2 | `29120a1ee32979431d062f210d41a9d1` | `49f8e602fc6ae769713c94ef7b257810` |
| M15 W2 | `d185f099482a07f29502cb5a371a9bbf` | `b4aa07e33e17e23774c486a6900d673b` |
| M15 OOS | `6f8f24b49597ced28f50a19e08c3472f` | `9fd624fb387c36052122bd1830cf9c32` |
| H1 W3 | `3b45d12bade823178b267c13bc698458` | `7b848c917dcb5c371ecbb9f8f7292278` |
| H4 W3 | `e75adae13c839b15736f69c0114e6aa5` | `186e677fbbd53cca735c94094c23ce05` |
`Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all ten runs.
## Base-rate table
`agent_panel_base_rates.json`: 45 classes (34 exact) from the relabelled M5 W2, M15 W2, H1 W3 and
H4 W3 books; key `timeframe|strategy|grade|direction`, fallbacks `tf|*|*|direction` and `tf|*|*|*`;
the exact class is used only with n ≥ 30. Examples: M15 all 931 trades win 60.5%, median hold 30
bars; M15 SELL 296 / 54.4%; M15 `XAU_ATR` A SELL 78 / 50.0%, median hold 6 bars; M5 `XAU_CP` B
BUY 321 / 51.7%; H1 all 253 / 64.4%.
## Pre-registered acceptance (from the approved plan)
On the resolved logged setups (37 today), a new prompt version ships only if: (1) no persona's
modal stance is above 65%; (2) Brier ≤ the logged v1 takes and ≤ the base-rate-only predictor;
(3) the skip-OPPOSE counterfactual does not remove a net-positive bucket. Outcome = advisory R > 0
at $0.65. Predictors compared on the same events: base-rate only (the class win rate for every
trade), the logged v1 takes (stance × confidence → pseudo-probability; 27 events carry them), and
the new prompts run live through DeepSeek on the rebuilt payloads (4 calls per event).
## Replay round 1 (draft prompts)
| predictor | n | Brier | log-loss | mean p | modal stance | constant % | R skip ≤ 40 |
|---|---|---|---|---|---|---|---|
| base-rate only | 37 | 0.263 | 0.719 | 54.8 | neutral | 81 | −7.15 (nothing skipped) |
| v1 logged, mean | 27 | 0.270 | 0.738 | 48.2 | neutral | 48 | −4.00 |
| v1 logged, macro | 27 | 0.299 | 0.855 | 56.5 | support | 44 | −2.82 |
| v1 logged, technical | 27 | 0.371 | 1.011 | 71.7 | support | 78 | −6.47 |
| v1 logged, risk | 27 | 0.343 | 0.947 | 25.7 | oppose | 78 | +0.03 |
| v2 draft, mean | 37 | 0.257 | 0.707 | 51.3 | neutral | 95 | −7.15 (nothing skipped) |
Like-for-like on the 27 events with v1 takes: Brier v2 0.257, v1 mean 0.270, v1 macro 0.299,
base rate 0.259, coin flip 0.250. Rank correlation of p with R: v2 −0.05, v1 mean +0.14, v1 macro
+0.36, base rate +0.16 (n = 27; the ±0.35 confidence band swallows all of them).
Reading: the draft is calibrated (it sits 3.5 points under the base rate with a spread of 1.7
points) and no longer directional, but it carries no rank information: winners averaged p = 50.9,
losers 51.7. The old macro verdict had some ordering (+0.36) and terrible calibration (82%
confidence on the three losing FOMC shorts); the new one has calibration and no ordering. Criterion
(2) passes, (3) is vacuous (no OPPOSE issued), (1) fails **in the opposite sense to the one it was
written for**: the personas cluster on neutral around the base rate instead of voting a constant
direction. That is hedging, not bias.
**A flaw in the draft prompts, found from the per-event reasons.** The five lowest probabilities
went to the Sep 10 and Sep 14 shorts that won (+1.31, +1.05, +0.70R), because the technical and
contrarian prompts told the personas to lower p when "the move has already run far" or "a big
candle just printed". That instruction encoded the live fortnight's pattern, which the two-year
books contradict (stretched entries are the system's best trades, 2026-08-04; the big-candle gate
was re-rejected in the review: those buckets are +32R and +60R). The final prompts remove that
instruction, state the book's finding in the context legend and in both personas, and limit
downward moves to higher-timeframe conflict, key levels, event timing inside the expected hold,
news newer than the move, and a stop far wider than the class norm.
## Replay round 2 (final prompts, the text that ships)
Same 37 events, same rebuilt payloads, the final prompt text (the technical and contrarian
personas told that a long prior move or a big candle is not a reason by itself; downward moves
limited to higher-timeframe conflict, key levels, event timing, news newer than the move, and a
stop far wider than the class norm).
| predictor | n | Brier | log-loss | mean p | modal stance | directional % | R skip ≤ 40 | R ≥ 60 only |
|---|---|---|---|---|---|---|---|---|
| base-rate only | 37 | 0.263 | 0.719 | 54.8 | neutral | 19 | −7.15 | −3.07 |
| v1 logged, mean | 27 | 0.270 | 0.738 | 48.2 | neutral | 52 | −4.00 | −1.66 |
| v1 logged, macro | 27 | 0.299 | 0.855 | 56.5 | support | 56 | −2.82 | −2.16 |
| **v2 final, mean** | 37 | **0.262** | 0.717 | 53.8 | neutral | 8 | −7.15 | −1.98 |
| v2 final, macro | 37 | 0.259 | 0.712 | 53.4 | neutral | 11 | | |
| v2 final, technical | 37 | 0.270 | 0.734 | 54.2 | neutral | 16 | | |
| v2 final, risk | 37 | 0.258 | 0.709 | 53.3 | neutral | 11 | | |
| v2 final, contrarian | 37 | 0.262 | 0.718 | 54.1 | neutral | 11 | | |
Like-for-like on the 27 events with v1 takes: Brier v2 final 0.260, v1 mean 0.270, v1 macro
0.299, base rate 0.259. Rank correlation of p with R: v2 final +0.13 (round 1 −0.05), v1 mean
+0.14, v1 macro +0.36. Calibration of the v2 mean: 34 events in the 40–60 band averaged p = 53
and won 47%; the 3 events at 60–61 won 1. The final personas sit 1.0 point under the base rate
(sd 1.8, no event moved 5 points or more); winners averaged p = 53.1, losers 54.3.
Acceptance: (2) passes — Brier below the logged v1 takes and equal to the base-rate predictor
(0.262 vs 0.263; 0.260 vs 0.259 on the common 27); (3) passes vacuously (no OPPOSE issued); (1)
fails as written (92% neutral) and passes as re-specified below (directional share 8%). The
correction removed the chase penalty: the winning Sep 10 / Sep 14 shorts moved from 45–47 to
49–51, and the reasons against now name what the books do not already price (a release due in 20
minutes, price sitting on a support level) instead of "the move already ran far".
Cost: 2 × 148 DeepSeek calls (both rounds), inside the shared daily cap of 500.
## Decision and what it means for the alert
The shipped verdict is a probability anchored to the validated base rate that moves only for
facts the books do not already price. On this sample it is calibrated and uninformative: the line
will usually read "about a 50% chance … (usual for this kind of setup: 55%)". That is the honest
state of the evidence — the 65-feature sizing model was a clean null too — and it is strictly
better than the previous line, which asserted 82% confidence in the wrong direction. The
information the line adds is the base rate itself, the biggest concrete worry (event inside the
hold, news newer than the move), and each persona's reason. It never gates; the user guide says
so.
Re-specified criterion (1) for future prompt versions: the share of *directional* stances
(support + oppose) on one side must not exceed 65% — neutral clustering around a calibrated base
rate is not the pathology the criterion targets.
## Persona set (owner decision, same day)
The owner's rule: a persona that does not give correct guidance is removed, to save confusion,
tokens and log space. Evidence per persona:
| persona | v1 (27 outcomes) | v2 final (37 outcomes) | verdict |
|---|---|---|---|
| macro | Brier 0.299, rank correlation with R **+0.36**, 88% / 25% win split on 2026-09-03 | Brier 0.259, ρ −0.06 | the only voice that ever showed ordering, and the only one holding facts the engine lacks (news) → **kept** |
| technical | support on 79%, Brier 0.371 (worst) | Brier 0.270 (worst), ρ −0.20 | re-reads the indicators the signal was built from → removed |
| risk | oppose on 79%, Brier 0.343 | Brier 0.258, ρ −0.02 | its inputs (stop in ATR, cost, event timing) are now computed in the context block → removed |
| contrarian | oppose on 65%, Brier 0.322 | Brier 0.262, ρ −0.05 | oppose by construction, no information → removed |
Shipped as `agent_panel.personas` (default `["macro"]`; `app_config` validates the names, `chair`
needs two or more, `macro` source needs the macro persona). One DeepSeek call per panel instead of
four (three quarters of the panel's token spend), `panel_payloads_json` about 4 KB instead of about
14 KB, `panel_takes_json` one entry, no persona votes line in the alert. The mean verdict with one
persona is that persona's probability (quorum = min(3, personas)). The full panel remains one
config line away for research replays (`scripts/panel_replay.py --personas macro,technical,risk,contrarian`).
## Next evaluation
- Every live v2 verdict logs its exact payloads, so `scripts/panel_replay.py` can re-score any
future prompt on identical evidence. At 100 resolved v2 outcomes (about five weeks): the
`--stored-candles` probability scorecard by prompt version. At 300: fit a probability-band →
observed-win-rate table on the first half by time, verify on the second, ship it as a numpy
lookup; only then revisit persona weights (owner rule).
- Do not iterate prompts on the current 37 events again: any further gain there is in-sample.
- Deployment: the live box needs the new code (both the bot and the dashboard daemon, which has
run `2ac7301` since Sep 10); no config change is required (`verdict_source` defaults to
`mean`); the additive migration runs on first open.
Commands: `venv/bin/python scripts/build_panel_base_rates.py --book 5min=<m5_w2.csv> --book
15min=<m15_w2.csv> --book 1h=<h1_w3.csv> --book 4h=<h4_w3.csv> --out agent_panel_base_rates.json`;
`venv/bin/python scripts/panel_replay.py --db logs/signals.db --dry-run`; `venv/bin/python
scripts/panel_replay.py --db logs/signals.db --max-calls 160 --out replay.csv`;
`venv/bin/python analyze_signal_log.py --db logs/signals.db --stored-candles`.
backtest/research/signal_log_review_2026-09-17.md
+483
View File
@@ -0,0 +1,483 @@
# Live signal-log review — 2026-09-17
**Source evidence:** `logs/signals.db` copied from the live server on 2026-09-17 (rows
2026-09-04 00:59 → 2026-09-17 13:25 UTC; 5,487 rows, 5,481 `scan` + 6 `failed`, 0 `blocked`;
telegram 3,458 / dashboard 2,029). Opened read-only. Cross-checked with
`analyze_signal_log.py --parity | --scan-outcomes | --stored-candles` (all read-only), and with a
live-vs-backtest parity replay of the same window through `backtest/gpu_backtest.py` at HEAD
(`5521998`). No tuned value is proposed; this is a health check and a descriptive scorecard.
## Headline
1. **Live equals backtest.** Over 2026-09-06 → 2026-09-17 the validated code produced 29 trades;
26 of them appear in the live log on the same bar with the same direction, grade and entry,
and the live advisory R agrees with the backtest R to within 0.14 (mean difference −0.006 over
23 resolved matches). The 3 backtest-only trades are the Sep 9 bot outage (2) and one M15 bar
scanned live on Sep 8 with the pre-RSI-change M15 engine (1). All 20 live-only setups are
consecutive-bar re-entries that fall inside the backtester's 5-bar entry cooldown.
2. **The raw advisory book reads −7.15R over 37 resolved setups, but that number counts cluster
re-entries the validated book never takes.** The like-for-like views are flat: cooldown-equivalent
live book −1.62R over 25 resolved (PF 0.84); backtest over the same window +1.09R over 29 trades
(M15 −0.32R / 13, M5 +2.43R / 15, H1 −1.01R / 1, H4 0). Against the two-year books, a 12–15 trade
fortnight summing between −0.7R and +2.4R sits at the 25th–49th percentile: an ordinary fortnight.
The 12 resolved cooldown re-entries alone cost −5.54R.
3. **The market was a one-way bear fortnight with an FOMC hike in it:** 4,477 → 4,367 (−2.5%),
low 4,235 on Sep 16, daily ranges of 12–30 ATR, then +95$ on Sep 17. Every directional statistic
below is drift-adjusted against the same-timeframe mean move; the raw 24 h drift was −4.2 ATR on M5.
4. **Direction vs market.** At 1–4 h the composite carried no information this fortnight (Spearman
rank correlation of score with the 4 h move ≈ 0 on every timeframe). At 24 h it was *inversely*
related to the move on all three timeframes (M5 ρ = −0.11, M15 −0.20, H1 −0.33, bootstrap CIs
exclude zero, n = 1,734 / 570 / 150 unique scans): high scores marked bounce tops in the
downtrend and deep negative scores marked lows that bounced. This is the same pattern the
2026-09-08 review saw in 16 of 18 episodes, now with n. It is a two-week bear-market reading,
not a verdict on the two-year edge, and it is the one hypothesis worth a proper in-book test.
5. **AI panel.** 34 unique setups carried a panel (27 resolved). Skipping the macro persona's OPPOSE
would have avoided 7 resolved trades worth −2.52R (kept book −2.82R vs −5.35R), but the same
opposed trades moved *in the trade's favour* at 24 h (drift-adjusted hit 4/5) and 6 further
OPPOSE verdicts are on the Sep 17 long cluster the backtester currently marks +1.13R. Technical
(79% support), risk (79% oppose) and contrarian (65% oppose) are near-constant voters. Verdict
confidence is uncalibrated (62% / 50% / 60% correct across the three bands). No evidence either
way at this n; keep logging.
6. **Delivery findings** (no book impact): the dashboard daemon has run stale code (`2ac7301`) since
Sep 10 13:28 while the bot moved to `2836024` on Sep 14, and it was down from Sep 7 02:20 to
Sep 10 13:28; the bot was down Sep 9 00:05 → 19:36 UTC (two validated trades missed, +0.37R);
the live pipeline has no entry cooldown, so consecutive-bar setups are logged, scored and (about
one in three) re-alerted; the backtester's CSV labels the gold ATR setups `GENERIC` while live
stamps them `XAU_ATR` (geometry and R identical).
## Method
- Rows deduped to the first row per (timeframe, bar label); `bar_ts` NULL on 365 rows (354 from the
pre-Sep-7 logging code) falls back to `indicators_json[tf].timestamp`; 11 rows with no indicators
(price 0) dropped; `scan_kind in ('blocked','failed')` skipped. Unique bars: M5 2,289, M15 763,
H1 191, H4 53.
- Setups: 49 unique (M5 24, M15 24, H1 1, H4 0). Outcome = the DB's own advisory model
(`advisory_r_multiple`, net R at $0.65, next-open entry, stop first, close-only trail): 37 RESOLVED
(−7.15R, reproduced exactly), 11 PENDING (the Sep 17 long cluster), 1 UNRESOLVABLE.
- Forward moves: `scan_outcomes` (`fwd_{1h,4h,24h}_atr`, + = up; 24 h only where `complete = 1`).
"Move" = move in the signal's direction; "excess" = the same after subtracting the same-timeframe
mean move over all scans (the drift); "adj hit" = share of excess > 0.
- Three book views: (1) every unique setup; (2) cooldown-equivalent = after an accepted setup, skip
the next 5 primary bars (the backtester's rule); (3) alerted = the bot's dedup replayed on the
telegram rows (fingerprint (signal, direction) per timeframe; re-alert after 3,600 s, on a grade
upgrade, or when the entry moved ≥ 0.5 × last risk; grade ≥ C; setup required; state reset at
each restart = code-version change). View 3 is a reconstruction; `logs/telegram.log` on the live
box is the ground truth.
- Bootstrap null: the M15 and M5 two-year books regenerated offline from the kept cache with the
section-9 harness (`--start 2024-09-05 --end 2026-09-05 --spread 0.65 --workers 12`), MD5s
`d185f099482a07f29502cb5a371a9bbf` and `29120a1ee32979431d062f210d41a9d1` (both equal to the
pinned values; `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0`); 20,000 fixed-n resamples
of per-trade R plus every 13-calendar-day window of the book.
- Parity replay: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --spread 0.65 --workers 12
--primary-tf {15min,5min,1h,4h} --start 2026-09-06 --export-csv ... --detail` (no `--end`, so the
end is the run time; no `--set`). New cache windows, cited here and therefore kept:
`oanda_XAU_USD_{5min,15min,1h}_20260706_20260917.parquet`,
`oanda_XAU_USD_4h_20260115_20260917.parquet`, `oanda_XAU_USD_1d_20221106_20260917.parquet`.
`Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all four runs. The fetch logged two
interior gaps: M5/M15 2026-09-07 18:15–22:00 (Labor Day, Oanda published no bars — matches the 46
stale-bar rows live logged) and H4 2026-04-02 → 04-05 (Good Friday). Matching key: (timeframe,
backtest entry bar − 1 bar) == live bar label.
- Nothing was written to `signals.db`; `scripts/backfill_signal_outcomes.py` and
`scripts/label_scan_outcomes.py` were run with `--dry-run` only.
## 1. Integrity and coverage
| Check | Result |
|---|---|
| Surface parity (1,953 shared bars) | 0 signal, grade or setup-presence divergences; 8 score differences, none changing a signal or grade: 6 on H1/H4 bars scored by the two code versions (≤ 0.2 pts on H1, up to 7 pts on H4 at Sep 15 00:00: consistent with different confirmation-frame seeding), 2 on the dashboard's Sep 10 13:28 restart catch-up |
| Code versions live | telegram: c44813f → de36b10 (Sep 7 15:08) → 3afc62c → 28b591d (Sep 8) → 2ac7301 (Sep 9 21:45) → **2836024 (Sep 14 01:46)**. dashboard: c44813f (26 rows, Sep 7 01:00–02:20), then **2ac7301 from Sep 10 13:28 to now** |
| Dashboard daemon | down Sep 7 02:20 → Sep 10 13:28 (3.5 days); since then on stale code. Still logs 11 "Primary timeframe stale" M5 rows every day at 21:xx UTC (the settlement pause; a scheduled closure in 2836024) |
| Telegram bot | down Sep 9 00:05 → 19:36 UTC. Cost: M15 BUY B Sep 9 05:45 (+0.23R) and M5 BUY A Sep 9 06:00 (+0.14R) in the backtest, never scanned live |
| Labor Day (Sep 7) | Oanda published no bars 18:25–22:00 UTC (confirmed by the fresh fetch); 46 stale-bar rows logged; no holiday awareness (known) |
| Failed scans | 6 rows at 06:11–06:17 UTC on Sep 15 and Sep 16, both surfaces, primary frame missing: an Oanda blip longer than the retry ladder's 45 s budget. 11 pre-ladder "data unavailable" rows on Sep 4/7 |
| Scan-outcome labelling | 2,467 unique bars complete, 418 labelled but < 72 h old, 408 unlabelled and < 25 h old, 3 unlabelled and older. Last label 13:20 today: the labeler is healthy |
| Blocked scans | none logged (`log_blocked_scans` off); the Monday 00:xx UTC hours are therefore empty by design |
| Engine fingerprints | M15 `dab4d03725d8` → `536f84340ea0` at Sep 9 21:45 (RSI × 0.5). M5, H1, H4 unchanged all window |
| Panel | 35 rows, ≤ 9 per day against the cap of 20, all cache MISS, all `verdict_source = macro` |
## 2. Market context
| UTC day | open | high | low | close | change $ | range / ATR |
|---|---|---|---|---|---|---|
| Sep 4 | 4,480 | 4,491 | 4,366 | 4,431 | −49 | 30 |
| Sep 7 (Labor Day) | 4,424 | 4,430 | 4,381 | 4,423 | 0 | 13 |
| Sep 8 | 4,423 | 4,443 | 4,345 | 4,350 | −73 | 20 |
| Sep 9 (bot down) | 4,350 | 4,418 | 4,347 | 4,394 | +44 | 24 |
| Sep 10 | 4,394 | 4,435 | 4,311 | 4,315 | −79 | 25 |
| Sep 11 | 4,315 | 4,403 | 4,292 | 4,349 | +33 | 20 |
| Sep 14 | 4,333 | 4,355 | 4,254 | 4,288 | −45 | 21 |
| Sep 15 | 4,288 | 4,318 | 4,261 | 4,284 | −5 | 14 |
| Sep 16 (FOMC hike 18:00) | 4,284 | 4,367 | 4,235 | 4,272 | −12 | 30 |
| Sep 17 (to 13:20) | 4,272 | 4,381 | 4,267 | 4,367 | +95 | 23 |
Drift baseline (mean forward move over **all** unique scans, ATR units, + = up):
| tf | 1 h | 4 h | 24 h |
|---|---|---|---|
| M5 | −0.23 (n 1,992) | −0.63 (1,998) | −4.21 (1,734) |
| M15 | −0.09 (661) | −0.36 (667) | −2.31 (570) |
| H1 | −0.06 (161) | −0.22 (167) | −0.75 (150) |
## 3. Direction and score against the realised move (all unique scans)
Move is measured in the signal's direction; excess is after removing the drift above.
| tf | direction | n | 1 h move / excess | 4 h move / excess / hit | 24 h move / excess / adj hit |
|---|---|---|---|---|---|
| M5 | LONG | 18 | −1.10 / −0.87 | −1.27 / −0.64 / 39% | −6.83 / −2.62 / 47% |
| M5 | SHORT | 66 | +0.35 / +0.12 | +0.14 / −0.49 / 53% | −1.13 / −5.34 / 31% |
| M15 | LONG | 7 | −0.81 / −0.72 | −0.31 / +0.05 / 43% | −4.03 / −1.72 / 33% |
| M15 | SHORT | 39 | +0.27 / +0.18 | +0.60 / +0.23 / 69% | −0.68 / −2.98 / 30% |
| H1 | SHORT | 4 | −0.25 / −0.31 | −0.66 / −0.88 / 50% | −2.05 / −2.80 / 0% |
Reading: M15 shorts were right for about four hours (69% hit, +0.23 ATR excess) and wrong by the
next day (the bear-trend lows they sold bounced). Longs were wrong at every horizon. Nothing here
beats drift at 24 h.
Score buckets (direction = sign of score, includes NEUTRAL scans):
| tf | bucket | n | 4 h move / excess | 24 h move / excess / adj hit |
|---|---|---|---|---|
| M5 | ≥ 35 | 22 | −1.83 / −1.19 | −13.06 / −8.85 / 18% |
| M5 | 25..35 | 47 | −1.11 / −0.48 | −9.61 / −5.40 / 28% |
| M5 | −35..−25 | 101 | −0.95 / −1.58 | +0.11 / −4.10 / 31% |
| M5 | ≤ −35 | 41 | −2.06 / −2.69 | −0.15 / −4.36 / 25% |
| M15 | 15..25 | 56 | −0.68 / −0.31 | −5.15 / −2.85 / 35% |
| M15 | 25..35 | 17 | −0.13 / +0.23 | −3.20 / −0.90 / 44% |
| M15 | −35..−25 | 37 | +0.65 / +0.28 | +0.42 / −1.88 / 38% |
| M15 | ≤ −35 | 24 | +0.47 / +0.10 | −0.98 / −3.28 / 29% |
Spearman rank correlation of the composite score with the forward move (+ would mean high score →
price up), with bootstrap 95% CI:
| tf | 1 h | 4 h | 24 h |
|---|---|---|---|
| M5 (n ≈ 1,990 / 1,734) | −0.05 [−0.09, −0.01] | −0.03 [−0.07, +0.02] | **−0.11 [−0.15, −0.06]** |
| M15 (n ≈ 665 / 570) | 0.00 [−0.08, +0.08] | −0.02 [−0.09, +0.06] | **−0.20 [−0.27, −0.12]** |
| H1 (n ≈ 165 / 150) | −0.09 [−0.25, +0.06] | −0.08 [−0.24, +0.07] | **−0.33 [−0.47, −0.17]** |
Grades: M15 grade-A rows (n = 12) had a 24 h excess of −4.8 ATR and an adjusted hit of 17%; M5
grade-B rows (n = 15) −1.9 ATR excess at 4 h, hit 20%. Gates: the only gate that removed A/B/C
directional rows this fortnight was the spread gate on M5 (n = 10; 24 h excess +0.03 vs −5.33 for
the shipped M5 setups): tiny and descriptive. The M15 engine change at Sep 9 21:45 leaves only 10
directional rows before it; nothing can be said about it.
## 4. The trade book
View 1, every unique setup (advisory R at $0.65; PENDING excluded from R):
| cut | n | resolved | wins | sum R | mean R | PF |
|---|---|---|---|---|---|---|
| **all** | 49 | 37 | 17 (46%) | **−7.15** | −0.19 | 0.60 |
| M5 | 24 | 14 | 6 | −1.08 | −0.08 | 0.83 |
| M15 | 24 | 22 | 11 | −5.06 | −0.23 | 0.51 |
| H1 | 1 | 1 | 0 | −1.01 | | |
| BUY / SELL | 30 / 19 | 18 / 19 | 8 / 9 | −3.49 / −3.67 | −0.19 / −0.19 | 0.59 / 0.61 |
| grade A / B / C | 27 / 19 / 2 | 20 / 15 / 2 | 9 / 7 / 1 | −4.87 / −2.75 / +0.47 | | 0.48 / 0.63 / 1.46 |
| XAU_ATR / GENERIC / XAU_CP | 10 / 35 / 4 | 9 / 25 / 3 | 5 / 10 / 2 | −1.29 / −7.03 / +1.17 | | 0.69 / 0.45 / 2.10 |
| regime transitioning / trending_down / trending_up | 12 / 15 / 16 | 11 / 15 / 6 | 4 / 8 / 2 | −6.02 / −0.70 / −2.06 | | 0.16 / 0.89 / 0.08 |
| volatility extreme / normal | 31 / 9 | 22 / 8 | 10 / 4 | −4.32 / +0.94 | | 0.59 / 1.30 |
| watch level none / watch / hot | 10 / 24 / 12 | 10 / 20 / 5 | 6 / 7 / 4 | +0.57 / −7.73 / +2.11 | | 1.18 / 0.38 / 10.5 |
| hour 08 UTC | 7 | 7 | 0 | −7.14 | −1.02 | 0.00 |
| hour 12 UTC | 13 | 6 | 5 | +4.15 | +0.69 | 19.7 |
| day Sep 14 / Sep 15 / Sep 16 | 4 / 6 / 11 | 4 / 6 / 11 | 4 / 1 / 4 | +3.40 / −5.01 / −4.53 | | |
Exits: 17 SL, 18 TRAIL, 2 GAP_LOCKED. Setup-level forward moves agree with the advisory model:
M15 setups −0.60 ATR excess at 4 h (hit 35%), −3.33 at 24 h; M5 −1.49 at 4 h, −5.33 at 24 h.
View 2, cooldown-equivalent (skip 5 primary bars after an accepted setup): 28 setups, 25 resolved,
14 wins (56%), **−1.62R**, PF 0.84; M5 +0.11R (12), M15 −0.71R (12), H1 −1.01R; SELL +1.11R over 10
(70% win), BUY −2.73R over 15.
View 3, reconstructed Telegram alerts: 36 alerts in 9 trading days (about 4 a day), 29 resolved, 14
wins (48%), **−4.88R**, PF 0.64; M15 −3.97R (16), M5 +0.11R (12), H1 −1.01R. The re-alert rules let
through 2 of the 6 Sep 15 shorts and all 3 FOMC-evening shorts.
Backtest over the same window (HEAD): **29 trades, +1.09R**. Matched-resolved 23: live +0.47R vs
backtest +0.62R (the difference is the Sep 8 07:30 M15 short, GAP_LOCKED live at +1.13 vs +1.28
on the raw path). Matched but live still pending 3: backtest +1.13R. Backtest-only 3: −0.66R.
Live-only cooldown re-entries 20 (8 pending): the 12 resolved ones sum to **−5.54R**.
Bootstrap against the two-year books (per-trade R resampled; P = share of draws at or below the
observed sum):
| view | tf | n | observed | book median | 5–95% | P(≤ obs) |
|---|---|---|---|---|---|---|
| view 1 (raw advisory) | M15 | 22 | −5.06R | +3.2 | −4.8..+12.0 | 4.6% |
| view 1 | M5 | 14 | −1.08R | +2.4 | −4.2..+10.3 | 20% |
| view 2 (cooldown) | M15 | 12 | −0.71R | +1.8 | −4.1..+8.3 | 25% |
| view 2 | M5 | 12 | +0.11R | +2.0 | −4.0..+9.3 | 31% |
| backtest over window | M15 | 13 | −0.32R | +1.9 | −4.2..+8.7 | 28% |
| backtest over window | M5 | 15 | +2.43R | +2.6 | −4.3..+10.7 | 49% |
Calendar check: 36% of the M15 book's 13-day windows and 35% of the M5 book's are negative; among
M5 windows where price fell 3% or more the median is +0.2R with 46% negative, median BUY −1.2R and
median SELL +1.5R. This fortnight behaved the way the book says a falling fortnight behaves. The
view-1 M15 figure looks extreme only because its 22 "trades" are 12 independent entries plus 10
correlated re-entries.
Special buckets:
- **M15 grade-A/A+ SELL** (the parked cycle-2 candidate): live 11 setups, 5 wins, −2.77R (PF 0.55),
but 6 of the 11 are Sep 15 cluster re-entries; the cooldown-equivalent five are +1.13, +0.36, +1.05,
+0.09, −1.02 = +1.61R. Two-year book: 176 trades, −3.4R, mean −0.02R. The live sample does not
move the candidate either way; it stays parked.
- **FOMC evening** (Sep 16 18:45, 19:00, 19:15 UTC): three M15 shorts, all −1R, 45–75 minutes after
the 18:00 decision, all `watch_level = watch` (reading `extreme`), none `hot` because the hot rule
needs a high-impact USD release within ±30 min. The macro persona AGREED on all three at 82%
confidence ("the Fed raised rates, which pushes gold down"); gold rose 95$ the next day. The
backtester took the 18:45 trade too (−1.02R), so it is validated behaviour; 19:00 and 19:15 are
cooldown re-entries.
- **Sep 15 07:45–08:45 M15 STRONG_SELL cluster**: six setups on consecutive bars, one +0.09R and
five −1.02R (−5.01R). The backtester took the first (+0.09R) and skipped the rest.
- **XAU_CP**: 4 setups (2 inverse head-and-shoulders, 2 double bottoms), 3 resolved, +1.17R; both
in-window ones matched the backtester exactly.
- **Exhaustion count at entry**: 0 → +2.57R (4 resolved), 1 → −4.85R (10), 3 → +0.26R (8),
4 → −3.24R (11). Descriptive only; the 2026-09-06 campaign already closed exhaustion sizing.
## 5. The AI panel
Unique setups with a panel 34 (27 resolved); macro persona is the verdict.
| verdict | n | resolved | wins | sum R | PF | 4 h excess / adj hit | 24 h excess / adj hit |
|---|---|---|---|---|---|---|---|
| AGREE | 12 | 12 | 6 (50%) | −2.16 | 0.59 | −2.51 / 22% (n 9) | −7.13 / 22% (n 9) |
| NEUTRAL | 9 | 8 | 5 (62%) | −0.67 | 0.79 | +1.48 / 75% (n 8) | −2.31 / 29% (n 7) |
| OPPOSE | 13 | 7 | 2 (29%) | −2.52 | 0.39 | −0.72 / 20% (n 5) | +0.69 / 80% (n 5) |
| skip OPPOSE (counterfactual) | 21 | 20 | 11 (55%) | −2.82 | 0.66 | | |
| AGREE only | 12 | 12 | 6 (50%) | −2.16 | 0.59 | | |
By realised R the OPPOSE verdict was mildly useful (it flagged the two worst-PF groups: opposed
BUYs −1.61R over 4, opposed SELLs −0.91R over 3). By forward move the same opposed trades went the
trade's way at 24 h (4 of 5 after drift). Six pending OPPOSE verdicts sit on the Sep 17 long
cluster, which the backtester currently marks +0.90, +0.38, −0.15R. Nothing here is decidable at
n = 7; it is consistent with the 2026-09-03 finding that only the macro persona carries information.
Personas (stance share / mean R when they held it): macro support 35% (−0.18R) / neutral 26%
(−0.08R) / oppose 38% (−0.36R); technical support **79%** / oppose 12%; risk oppose **79%** /
support 3%; contrarian oppose **65%**. Three of four personas vote almost the same way on every
setup, so they add little beyond their prior. "Take only setups the risk persona did not oppose"
leaves 7 setups (+0.03R): a curiosity, not a rule. Number of opposing personas (1 / 2 / 3) gives
PF 0.57 / 0.57 / 0.57. Verdict confidence bands < 40 / 40–59 / ≥ 60 were 62% / 50% / 60% correct
(n 8 / 4 / 15): no calibration.
## 6. Findings, tagged
**Observations (small sample, descriptive):**
- The system is doing what was validated: bar-for-bar parity with the backtester, and the
backtester itself made +1.1R on this window, a 30th-percentile fortnight for the book.
- The live loss is concentrated in cluster re-entries (−5.5R), the Sep 15 08:xx short run and the
FOMC-evening shorts, and in the `transitioning` regime (PF 0.16 over 11) and `watch` level
(−7.7R over 20). Two weeks; no action.
- The composite score was inversely related to the 24 h move on all three timeframes with CIs
excluding zero, and uninformative at 1–4 h. Two weeks of a single bear regime cannot overturn the
two-year book, but it is the strongest live pattern seen twice now (2026-09-08 and here).
- Panel: OPPOSE mildly helpful by R, anti-informative by 24 h move; three personas near-constant.
**Delivery-side fix candidates (no book impact):**
1. Redeploy the dashboard daemon to `2836024` (or later) and install the systemd unit; it has run
`2ac7301` since Sep 10 and was dead for 3.5 days before that. After redeploy, re-check the 8 score
divergences between surfaces on H1/H4 bars.
2. Decide whether the live pipeline should mirror the backtester's 5-bar entry cooldown for
alerts and for the advisory scorecard. Today every consecutive-bar setup is logged as its own
trade and about one in three is re-alerted, so the live book systematically over-counts
correlated entries relative to every validated number. A cooldown view could be added to
`analyze_signal_log.py` (the existing `one_position_view` is close), and an alert cooldown is
the owner's call.
3. Backtester CSV `strategy` column: the gold ATR setups appear as `GENERIC` while live stamps
`XAU_ATR`; entries, stops and R are identical, so this is a labelling gap only. Cause: the
backtester derives its tag from the text of the first bullish/bearish reason (`"XAU STRATEGY"`
in the first bullish reason → `XAU`) instead of reading `trade_setup.strategy`, so SELL ATR
setups and any ATR setup whose first reason is another indicator fall through to `GENERIC`.
The same tag feeds the research-only `--strategies` filter. Book-neutral; a one-line fix plus
a test when convenient.
4. Bot uptime: the 19.5 h Sep 9 outage cost two validated trades; the daily 21:xx stale rows and
the Labor Day stale run are known and harmless.
**Needs a rule-1 backtest (nothing proposed to ship):**
- Score-vs-24 h inversion: test in-book with the replay recorder (`--record --record-bars` writes
forward 1 h / 4 h / 24 h moves for every analysed bar over the two-year window). If the rank
correlation is also negative across the book, the gate chain's thresholds, not the weights, are
the place to look; if it is positive, this fortnight was regime-specific.
- M15 grade-A SELL block stays parked (cycle-2 candidate; live adds n = 5 cooldown-equivalent
trades at +1.6R and n = 11 raw at −2.8R).
## Caveats
Section 7 adds four scratchpad research runs of gate 6h on the two-year cache (no new cache
windows; `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all four).
Nine trading days, one regime (bear trend with an FOMC hike), 37 resolved advisory outcomes of
which 12 are correlated re-entries, the advisory model is cleaned-mid evidence and not execution,
the 24 h horizon is drift-dominated (drift-adjusted numbers are shown but a single trend cannot be
fully removed), the M15 engine changed mid-sample (Sep 9 21:45), the dashboard and the bot ran
different code from Sep 14, and the alerted view is a reconstruction. The `--stored-candles`
replay closes only 26 of the 37 DB-resolved setups because the stored candle streams have the
Labor Day and Sep 9 gaps; where both resolve they agree to the cent.
## Appendix: the 49 unique setups
R at $0.65 (advisory model); ex24 = 24 h move in the trade direction minus the same-timeframe drift,
ATR units; BT = backtester over the same window (M = matched, C = live-only inside the cooldown,
– = before the parity window).
| bar (UTC) | tf | signal | g | score | dir | strategy | regime | R | exit | ex24 | panel | BT |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 09-04 05:35 | M5 | BUY | B | +8.7 | BUY | XAU_CP | ranging | −1.06 | SL | −7.4 | NEUTRAL | – |
| 09-04 07:45 | M15 | STRONG_BUY | A | +37.3 | BUY | XAU_ATR | transitioning | −1.04 | SL | −5.1 | NEUTRAL | – |
| 09-04 15:10 | M5 | BUY | B | +8.8 | BUY | XAU_CP | ranging | unres. | | +3.1 | OPPOSE | – |
| 09-07 08:00 | M15 | BUY | C | +26.7 | BUY | GENERIC | ranging | −1.02 | SL | | | M (bt grade B, pre-RSI engine) |
| 09-07 12:15 | M5 | BUY | B | +22.2 | BUY | XAU_CP | trending | +2.04 | TRAIL | | NEUTRAL | M |
| 09-08 07:30 | M15 | SELL | A | −34.2 | SELL | GENERIC | trending_down | +1.13 | GAP_LOCKED | +3.0 | OPPOSE | M (bt +1.28) |
| 09-08 07:45 | M15 | SELL | B | −31.7 | SELL | GENERIC | trending_down | +0.84 | GAP_LOCKED | +2.7 | | C |
| 09-08 08:05 | M5 | SELL | B | −35.4 | SELL | GENERIC | trending_down | −1.03 | SL | +3.7 | OPPOSE | M |
| 09-10 05:10 | M5 | BUY | B | +20.5 | BUY | XAU_CP | ranging | +0.18 | TRAIL | −18.3 | AGREE | M |
| 09-10 05:40 | M5 | BUY | A | +33.4 | BUY | GENERIC | trending_up | −1.03 | SL | −15.7 | AGREE | M |
| 09-10 05:55 | M5 | BUY | B | +29.8 | BUY | GENERIC | trending_up | −1.02 | SL | −15.2 | | C |
| 09-10 09:15 | M15 | SELL | B | −29.1 | SELL | GENERIC | transitioning | +0.07 | TRAIL | +4.2 | NEUTRAL | M |
| 09-10 12:05 | M5 | SELL | B | −30.3 | SELL | GENERIC | trending_down | +1.31 | TRAIL | +3.6 | AGREE | M |
| 09-10 12:15 | M15 | SELL | A | −34.3 | SELL | GENERIC | trending_down | +0.36 | TRAIL | +0.3 | AGREE | M |
| 09-10 12:30 | M15 | SELL | B | −32.6 | SELL | GENERIC | trending_down | −0.22 | TRAIL | −3.4 | AGREE | C |
| 09-14 09:15 | M15 | STRONG_SELL | A | −42.7 | SELL | XAU_ATR | trending_down | +1.05 | TRAIL | −0.1 | AGREE | M |
| 09-14 09:30 | M15 | STRONG_SELL | A | −42.6 | SELL | XAU_ATR | trending_down | +0.70 | TRAIL | −1.0 | | C |
| 09-14 12:45 | M5 | SELL | B | −36.5 | SELL | GENERIC | trending_down | +0.16 | TRAIL | −6.1 | AGREE | M |
| 09-14 15:25 | M5 | BUY | C | +25.1 | BUY | GENERIC | ranging | +1.49 | TRAIL | +4.3 | | M |
| 09-15 07:45 | M15 | STRONG_SELL | A | −37.5 | SELL | GENERIC | trending_down | +0.09 | TRAIL | −8.7 | AGREE | M |
| 09-15 08:00 | M15 | STRONG_SELL | A | −42.0 | SELL | GENERIC | trending_down | −1.02 | SL | −9.5 | OPPOSE | C |
| 09-15 08:10 | M5 | SELL | B | −30.3 | SELL | GENERIC | trending_down | −1.02 | SL | −15.6 | AGREE | M |
| 09-15 08:15 | M15 | STRONG_SELL | A | −36.7 | SELL | GENERIC | trending_down | −1.02 | SL | −9.6 | | C |
| 09-15 08:30 | M15 | STRONG_SELL | A | −41.6 | SELL | GENERIC | trending_down | −1.02 | SL | −9.7 | | C |
| 09-15 08:45 | M15 | STRONG_SELL | A | −45.1 | SELL | GENERIC | trending_down | −1.02 | SL | −10.5 | | C |
| 09-16 03:00 | M5 | STRONG_BUY | A | +36.0 | BUY | GENERIC | trending_up | +0.14 | TRAIL | −4.2 | NEUTRAL | M |
| 09-16 03:00 | M15 | BUY | A | +34.0 | BUY | XAU_ATR | transitioning | +0.14 | TRAIL | −3.5 | NEUTRAL | M |
| 09-16 03:10 | M5 | STRONG_BUY | A | +38.6 | BUY | GENERIC | trending_up | −0.16 | TRAIL | −4.6 | | C |
| 09-16 03:15 | M15 | BUY | A | +30.9 | BUY | XAU_ATR | transitioning | +0.43 | TRAIL | −2.8 | | C |
| 09-16 06:00 | H1 | BUY | B | +28.0 | BUY | GENERIC | transitioning | −1.01 | SL | −0.7 | NEUTRAL | M |
| 09-16 07:00 | M15 | BUY | B | +29.1 | BUY | GENERIC | trending_up | +0.05 | TRAIL | +0.5 | NEUTRAL | M |
| 09-16 07:35 | M5 | STRONG_BUY | A | +37.3 | BUY | GENERIC | trending_up | −0.04 | TRAIL | +3.2 | OPPOSE | M |
| 09-16 15:05 | M5 | BUY | B | +29.7 | BUY | GENERIC | transitioning | −1.04 | SL | | OPPOSE | M |
| 09-16 18:45 | M15 | SELL | A | −34.6 | SELL | XAU_ATR | transitioning | −1.02 | SL | | AGREE | M |
| 09-16 19:00 | M15 | STRONG_SELL | A | −35.4 | SELL | XAU_ATR | transitioning | −1.02 | SL | | AGREE | C |
| 09-16 19:15 | M15 | SELL | B | −26.6 | SELL | GENERIC | transitioning | −1.01 | SL | | AGREE | C |
| 09-17 07:15 | M15 | BUY | A | +31.5 | BUY | XAU_ATR | transitioning | −1.03 | SL | | OPPOSE | M |
| 09-17 07:25 | M5 | STRONG_BUY | A | +35.9 | BUY | GENERIC | trending_up | pending | | | OPPOSE | M (bt +0.90) |
| 09-17 07:30 | M5 | STRONG_BUY | A | +36.2 | BUY | GENERIC | trending_up | pending | | | | C |
| 09-17 07:30 | M15 | BUY | B | +28.7 | BUY | GENERIC | transitioning | pending | | | | C |
| 09-17 07:35 | M5 | STRONG_BUY | A | +35.5 | BUY | GENERIC | trending_up | pending | | | | C |
| 09-17 12:00 | M5 | BUY | B | +34.3 | BUY | GENERIC | trending_up | pending | | | NEUTRAL | M (bt +0.38) |
| 09-17 12:00 | M15 | STRONG_BUY | A | +35.9 | BUY | XAU_ATR | transitioning | +0.50 | TRAIL | | OPPOSE | M |
| 09-17 12:05 | M5 | STRONG_BUY | A | +37.2 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
| 09-17 12:10 | M5 | BUY | B | +34.4 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
| 09-17 12:15 | M5 | STRONG_BUY | A | +38.9 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
| 09-17 12:15 | M15 | STRONG_BUY | A+ | +43.7 | BUY | XAU_ATR | trending_up | pending | | | OPPOSE | C |
| 09-17 12:20 | M5 | STRONG_BUY | A | +39.1 | BUY | GENERIC | trending_up | pending | | | | C |
| 09-17 12:25 | M5 | STRONG_BUY | A | +40.0 | BUY | GENERIC | trending_up | pending | | | | M (bt −0.15) |
Backtest-only trades: M15 BUY B 09-08 04:00 (−1.03R; live scored the bar +24.7 NEUTRAL D on the
pre-RSI-change engine), M15 BUY B 09-09 05:45 (+0.23R) and M5 BUY A 09-09 06:00 (+0.14R), both
during the bot outage.
## 7. The owner's four suspicions, checked
The owner's objective for this review (2026-09-17): is anything functioning poorly in one of four
specific ways. Each is checked on the 49 unique live setups (37 resolved) and compared with the
two-year books where the books carry the same measure. "Reached TP1" means the trade's best
excursion (MFE) reached the planned first target; the advisory model itself exits on the trail, so
TP1 is advisory.
### 7.1 Does the AI panel OPPOSE trades that go on to hit their targets? — yes, as often as not
| macro verdict | setups | reached TP1 | reached TP2 | resolved wins | mean MFE |
|---|---|---|---|---|---|
| AGREE | 12 | 5 (42%) | 4 | 6 / 12 | 0.87R |
| NEUTRAL | 9 | 5 (56%) | 3 | 5 / 8 | 0.96R |
| OPPOSE | 13 | **7 (54%)** | 5 | 2 / 7 | 0.64R |
Opposed trades reached their first target more often than agreed ones. Concrete cases: OPPOSE at
60% on the Sep 8 07:30 M15 short ("most news points to gold rising") that made +1.13R and hit TP2;
OPPOSE at 62–65% on five of the Sep 17 longs that ran +0.7R to +1.3R (four still open); AGREE at 82%
on all three FOMC-evening shorts ("the Fed raised rates, which pushes gold down"), all −1R, gold
+95$ the next day. The macro persona's notes quote the news narrative of the moment, and the
2026-09-04 study showed that narrative trails price by about a day. The realised-R view (opposed
trades 2/7 wins) looks better for the panel only because the opposed winners are mostly the
pending Sep 17 longs. Structure: technical supports 79% of setups, risk opposes 79%, contrarian
65% — three near-constant voters. **Verdict: OPPOSE has no value as a veto and must never gate;
it is presented as "news context" at best.** (observation; the panel is already advisory-only)
### 7.2 Wrong direction, or entering late after the move is exhausted? — late by design; this fortnight the continuation failed
- 11 of 37 resolved setups never reached +0.3R (wrong from the first bar); 17 of 37 were stopped.
Book norm: 20–22% never reach 0.3R; live 30%.
- **30 of 37 entries came after price had already moved ≥ 2 ATR in the trade direction over the
previous 4 h**: PF 0.47, −7.8R. The 7 entries after a smaller prior move: PF 1.21, +0.7R.
- The engine's own exhaustion tags do **not** separate the losers: trades carrying
`rsi_extended` / `bb_overextended` at entry (21) had PF 0.68 and −2.8R; the 16 without had PF
0.53 and −4.4R. The 2026-08-04 stretch verdict (stretched entries are the best book trades) and
the 2026-09-06 sizing campaign already closed exhaustion-based sizing; the live fortnight adds
nothing that would reopen it.
- Three recurring shapes: (a) M15 shorts fire after a 4–6 ATR down leg, are right for four hours
(69% hit) and wrong by the next day (the low bounces); (b) longs in the downtrend fired at bounce
tops — Sep 10 05:40 and 05:55 at the 24 h high, followed by a 20 ATR fall; (c) the FOMC shorts
entered 45–75 min after the decision, 0.2–0.9 ATR above the day's low, after a 4–5 ATR drop: they
sold the bottom. Every one of these is the momentum design working as validated; the two-year
book earns its edge on exactly these entries and this fortnight they failed.
(observation)
### 7.3 Do signals fire on big volatile candles? — yes, and that is where the whole loss sits
| entry context (last 3 primary bars) | resolved | wins | reached TP1 | sum R | PF |
|---|---|---|---|---|---|
| a bar ≥ 2 ATR among the last 3 | 15 | 4 (27%) | 5 | **−7.18** | 0.25 |
| the signal bar itself ≥ 2 ATR | 7 | 1 (14%) | 2 | −3.44 | 0.23 |
| signal-bar body ≥ 1 ATR in the trade direction | 12 | 5 (42%) | 7 | −3.63 | 0.34 |
| no bar ≥ 2 ATR among the last 3 | 22 | 13 (59%) | 13 | **+0.02** | 1.00 |
The 15 setups fired within three bars of a ≥ 2 ATR candle carry the entire fortnight's loss; the
other 22 broke even at the book's win rate. Largest cases: Sep 10 12:30 M15 short on a 4.2 ATR bar
(−0.22R), the FOMC 18:45 / 19:00 shorts on 2.5–2.7 ATR bars (−1R each), Sep 14 09:15 short on a
2.7 ATR bar (+1.05R). A gate for exactly this exists (`big_candle_atr_multiple`, gate 6h) and ships
off because the 2026-07-30 test found the removed trades net-positive at every multiple on the long
books. The live fortnight contradicts that book verdict, so the gate was re-run on the two-year
M15 and M5 books (below).
Re-test of gate 6h on the two-year books (`--big-candle 2.0 --big-candle-hours {1,24}`, same
harness as the reference books, control = the MD5-verified W2 books; "removed" = control trades
absent from the candidate, approximate because the cooldown displaces entries):
| book | trades | R | PF | win | max DD | H1 / H2 | ΔR vs control | removed trades |
|---|---|---|---|---|---|---|---|---|
| M15 control | 931 | +146.1 | 1.43 | 60.5% | 16.1 | +67.7 / +78.4 | | |
| M15 gate 2 ATR, 1 h lookback | 643 | +117.6 | 1.51 | 61.9% | 14.4 | +58.4 / +59.2 | **−28.5** | 347 trades, **+32.5R**, 58% win |
| M15 gate 2 ATR, 24 h lookback | 3 | +0.5 | | | | | −145.6 | 929 trades, +147.0R |
| M5 control | 781 | +146.1 | 1.52 | 52.9% | 10.1 | +41.0 / +105.1 | | |
| M5 gate 2 ATR, 1 h lookback | 434 | +86.5 | 1.56 | 53.5% | 10.4 | +31.2 / +55.3 | **−59.6** | 368 trades, **+60.2R**, 52% win |
| M5 gate 2 ATR, 24 h lookback | 0 | | | | | | −146.1 | all |
Same verdict as 2026-07-30: over two years the trades entered within an hour of a ≥ 2 ATR candle
are a net-positive bucket on both timeframes (+32R and +60R at the book's own win rates), and the
gate buys a few points of profit factor by giving up a fifth to two fifths of the R. The live
fortnight's −7.2R in that bucket is a bad draw from a positive population, not a malfunction.
**Gate stays off; not a rule-1 candidate.** (needs-a-rule-1-backtest: done here, fails)
### 7.4 Are take-profit targets too far, with the market reversing first? — targets are normal in R; the stops are what is big
Planned geometry by generator (live): M15 GENERIC TP1 = 1.0R with stops of 3.6–4.4 ATR ($21–62,
one of $121 on the FOMC 19:15 short); M15 XAU_ATR TP1 = 0.69R with a 2 ATR stop ($16 target, $23
stop); M5 GENERIC TP1 = 0.5R with stops of 4.7–7.3 ATR ($13–32 target, $26–65 stop); XAU_CP TP1 =
0.88R. As a share of price the live GENERIC stops (medians 0.6–1.0%) sit above the book medians
(0.58–0.73%) but inside the book's 90th percentiles (0.97–1.66%); only the $121 FOMC stop (2.85%)
is outside. Capping the stop in ATR was tested and refuted in the 2026-09 campaign.
| measure | live (37 resolved) | M15 book | M5 book |
|---|---|---|---|
| reached TP1 | 49% | 52% | 56% |
| never reached 0.3R | 30% | 20% | 22% |
| gave back (MFE ≥ 0.5R, closed ≤ 0) | 16% | 11% | 18% |
| MFE 0.6–1.0R and closed ≤ 0 | 4 of 8 | | |
So the targets were reached about as often as in the books; what was worse was the share of trades
that never moved in favour at all. Trades that did reach 0.6–1.0R often ended flat or negative
because the trail arms at 0.6R (M5) / 0.7R (M15) and gives back to the arm level. A fixed exit at TP1
would have made this fortnight −3.55R instead of −7.15R, which is the same direction as the
2026-08-05 fixed-target study (better in 2024+ trend windows, worse in 2021 and 2023, drawdown
doubled), so nothing changes there. (observation; `--fixed-rr-tp` remains research-only)
config.json.example
+2 -1
View File
@@ -46,7 +46,8 @@
"candles_enabled": true,
"candle_bars_primary": 20,
"candle_bars_confirm": 10,
"verdict_source": "macro"
"verdict_source": "mean",
"personas": ["macro"]
},
"violent_watch": {
"enabled": false,
dashboard_publisher.py
+14 -1
View File
@@ -18,6 +18,10 @@ immune to anything on the web side):
* The only shared-code dependency beyond the standard signal path is
the additive `NewsEngine.get_week_events` accessor, which no
production signal path calls.
* Never writes a database. It is not always running, so the Telegram
bot is the single writer of `signals.db` and `newsdata.db`; main()
switches every SQLite writer off for this process
(`app_config.set_db_writes_for_process`, owner decision 2026-09-17).
Scheduling: every M15/H1/H4 boundary is also an M5 boundary (all
intervals are multiples of 5, anchored to UTC midnight), so one
@@ -58,7 +62,8 @@ from typing import Dict, List, Optional
import pandas as pd
from agent_panel import attach_panel
from app_config import get_app_config, violent_watch_enabled
from app_config import (get_app_config, set_db_writes_for_process,
violent_watch_enabled)
from config import PlatformConfig, Timeframe, XAUUSD
from main import report_to_dict
from news_engine import NewsEngine
@@ -553,6 +558,14 @@ def main(argv=None) -> None:
args = ap.parse_args(argv)
_setup_logging()
# This process never writes a database. It is not always running, so
# the Telegram bot is the single writer of signals.db and newsdata.db
# (owner decision 2026-09-17). The switch is process-wide and beats
# config.json / env; set here in main(), not at import, so importing
# the module (the test suite does) leaves every other writer on.
set_db_writes_for_process(False)
logger.info("database writes disabled for this process "
"(the Telegram bot is the only writer)")
os.makedirs(args.out, exist_ok=True)
pub = DashboardPublisher(args.out)
pub.install_static()
docs/user_guide.html
+1
View File
@@ -70,6 +70,7 @@
<tr><td><b>MTF aligned</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr>
<tr><td><b>Warnings ⚠️</b></td><td>Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings.</td></tr>
<tr><td><b>Violent-move watch 🌪</b></td><td>An occasional extra line saying the next hour looks unusually likely to produce a very large candle — because a high-impact US release is due (or just landed), because the clock is in a historically wild slot, or because price is stretched outside its bands. It is a heads-up about <i>volatility</i>, not a direction call, and it <b>never</b> changes the signal, the grade or the trade setup — the same alert would be sent without it. Do not close or tighten an open trade because of it: every such protective action was tested over three years and lost money. Your stop is the protection.</td></tr>
<tr><td><b>AI estimate 🤖</b></td><td>An advisory line such as <i>"AI estimate: about a 45% chance this trade pays off (usual for this kind of setup: 60%). Biggest worry: interest-rate decision in about 2 hours. Still being tested."</i> One AI reviewer (the news reader) receives the recent headlines together with facts computed by code — how far the move has already run, whether a very large candle just printed, how the stop and target compare with the usual, how trades of this exact kind fared in the two-year test, and whether major scheduled news lands before the trade would normally finish — and gives a probability that the trade ends in profit plus one reason for and one against. (Three further reviewers, chart, risk and sceptic, were switched off on 2026-09-17: in two evaluations they added nothing beyond the base rate and mostly voted the same way on every setup.) It is <b>not</b> a recommendation and it <b>never</b> changes the signal, the grade or the setup. Until about 300 of these estimates have been graded against real outcomes, treat it as context, not as a filter: the earlier AGREE/OPPOSE version of this line opposed target-reaching trades as often as it agreed.</td></tr>
<tr><td><b>Market context 📰</b></td><td>An AI-written note in plain English on what is moving the gold price right now, why it matters, and the next report to watch. It is built only from headlines published in the last 4 hours that lean one way (neutral items are dropped) plus today's high-impact releases. Informational only — it never blocks or grades a signal.</td></tr>
<tr><td><b>SL / TP1 / RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer. TP1 is the nearest opposing level. <b>1R</b> = the entry-to-SL distance = the amount you risk.</td></tr>
<tr><td><b>Trail plan 🔄</b></td><td>The real exit. Once price moves +0.6R in your favor, move the stop to trail 0.75R behind the best price and update it at each candle close. <b>TP1 is advisory — let winners run.</b></td></tr>
main.py
+21 -1
View File
@@ -168,7 +168,23 @@ def print_signal_report(report: SignalReport, detail: bool = False):
print(f" {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
if report.news_summary:
print(f" {Colors.MAGENTA}🧠{Colors.RESET} {report.news_summary}")
if report.panel_verdict:
if report.panel_verdict and report.panel_verdict_source == "mean":
base = (report.panel_context or {}).get("base_rate") or {}
usual = base.get("win_rate")
usual_txt = (f" (usual for this kind of setup: {usual:.0f}%)"
if usual is not None else "")
print(f" {Colors.CYAN}🤖{Colors.RESET} AI estimate: about a "
f"{Colors.BOLD}{report.panel_probability:.0f}%{Colors.RESET} "
f"chance this trade pays off{usual_txt}. "
f"{report.panel_rationale} Still being tested.")
for t in report.panel_takes:
p_win = t.get("p_win")
p_txt = f"{p_win:3.0f}%" if p_win is not None else " - "
label = t['persona'] if len(report.panel_takes) > 1 else "reasons"
print(f" {Colors.GRAY}{label:<11} {p_txt} "
f"for: {t.get('reason_for') or '-'} | against: "
f"{t.get('reason_against') or '-'}{Colors.RESET}")
elif report.panel_verdict:
panel_label = ("AI macro view"
if report.panel_verdict_source == "macro" else "AI Panel")
print(f" {Colors.CYAN}🤖{Colors.RESET} {panel_label}: "
@@ -230,6 +246,10 @@ def report_to_dict(report: SignalReport) -> dict:
"panel_created_at": report.panel_created_at,
"panel_cache_status": report.panel_cache_status,
"panel_payload_hash": report.panel_payload_hash,
"panel_verdict_source": report.panel_verdict_source,
"panel_prompt_version": report.panel_prompt_version,
"panel_probability": round(report.panel_probability, 1),
"panel_context": report.panel_context,
"watch_line": report.watch_line,
"bullish_reasons": report.bullish_reasons,
"bearish_reasons": report.bearish_reasons,
news_archive.py
+42 -9
View File
@@ -43,10 +43,19 @@ Contract
headline seen 50 times is one row with ``last_seen`` bumped.
* **stdlib only** (``sqlite3``), like ``signal_log``.
Known corpus limitation: a headline mentioning no tracked currency is
never constructed by the news engine at all (``news_engine`` drops it
before scoring), so DeepSeek never labels it either and it cannot appear
here.
Two columns exist for training only (2026-09-17):
* ``xau_score`` — DeepSeek's gold-specific score for the headline,
whatever currencies it names. ``score_by_currency_json["XAU"]``
carries it only for gold-keyword headlines; 930 of the first 1,546
DeepSeek rows had no gold label without this column. NULL for a
keyword-scored row and for rows written before the column existed
(an existing database is migrated in place, values are not
recoverable).
* ``currencies_json`` ``[]`` marks a headline naming no tracked currency.
The news engine used to drop those before scoring; now it scores and
archives them but never returns them to the platform. About 23% of a
day's feed; DeepSeek gave 10 of 24 a gold direction.
"""
from __future__ import annotations
@@ -85,7 +94,8 @@ CREATE TABLE IF NOT EXISTS headlines (
score_source TEXT NOT NULL DEFAULT 'keyword',
teacher_model TEXT,
teacher_prompt_sha1 TEXT,
origin TEXT NOT NULL DEFAULT 'live'
origin TEXT NOT NULL DEFAULT 'live',
xau_score REAL
);
CREATE INDEX IF NOT EXISTS idx_headlines_source ON headlines (score_source);
CREATE INDEX IF NOT EXISTS idx_headlines_first_seen ON headlines (first_seen);
@@ -155,8 +165,22 @@ class NewsArchive:
self._with_lock_retry(
lambda: self._conn.execute("PRAGMA journal_mode=WAL"))
self._with_lock_retry(lambda: self._conn.executescript(_SCHEMA))
self._with_lock_retry(self._migrate)
self._conn.commit()
#: Columns added after the first databases were created, in order.
#: CREATE TABLE IF NOT EXISTS leaves an existing table untouched, so
#: each one is added in place on open; existing rows read NULL.
_ADDED_COLUMNS = (("xau_score", "REAL"),)
def _migrate(self) -> None:
present = {row[1] for row in
self._conn.execute("PRAGMA table_info(headlines)")}
for name, decl in self._ADDED_COLUMNS:
if name not in present:
self._conn.execute(
f"ALTER TABLE headlines ADD COLUMN {name} {decl}")
@staticmethod
def _with_lock_retry(fn, attempts: int = 8, delay: float = 0.05):
"""Run ``fn`` again on 'database is locked' (concurrent openers
@@ -202,8 +226,8 @@ class NewsArchive:
" published_at, first_seen, last_seen, seen_count,"
" score, score_by_currency_json, currencies_json,"
" keywords_json, score_source, teacher_model,"
" teacher_prompt_sha1, origin"
") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?)"
" teacher_prompt_sha1, origin, xau_score"
") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?,?)"
" ON CONFLICT(hash) DO UPDATE SET"
# The backfill stamps historical timestamps, so clamp rather
# than assign: first_seen only ever moves earlier, last_seen
@@ -236,6 +260,13 @@ class NewsArchive:
" ELSE headlines.teacher_prompt_sha1 END,"
f" score_source = CASE WHEN {wins} THEN excluded.score_source"
" ELSE headlines.score_source END,"
# Same upgrade-only rule as score, and COALESCE on top: a
# keyword re-sighting carries no gold score and loses on
# rank, and even a same-rank write without one (a future
# backfill shape) must never blank a stored value.
f" xau_score = CASE WHEN {wins}"
" THEN COALESCE(excluded.xau_score, headlines.xau_score)"
" ELSE headlines.xau_score END,"
# Enrichment fills gaps but never blanks what is already there
# (the legacy backfill shape carries no description or link).
" description = COALESCE(excluded.description, headlines.description),"
@@ -246,9 +277,9 @@ class NewsArchive:
# The rank is bound once per CASE arm. Positional binding means a
# miscount silently shifts every parameter, so assert it instead of
# trusting the arithmetic: 17 INSERT values + one rank per arm.
# trusting the arithmetic: 18 INSERT values + one rank per arm.
n_rank = sql.count("? >= ")
n_values = 17
n_values = 18
assert sql.count("?") == n_values + n_rank, (
f"placeholder count drifted: {sql.count('?')} "
f"!= {n_values} + {n_rank}")
@@ -264,6 +295,7 @@ class NewsArchive:
row["currencies_json"], row["keywords_json"],
row["score_source"], row["teacher_model"],
row["teacher_prompt_sha1"], row.get("origin", "live"),
row.get("xau_score"),
) + (rank,) * n_rank)
def _write() -> None:
@@ -321,6 +353,7 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
"teacher_prompt_sha1": (
teacher_prompt_sha1 if source == "llm" else None),
"origin": origin,
"xau_score": _finite_score(getattr(headline, "xau_score", None)),
}
news_engine.py
+55 -17
View File
@@ -226,6 +226,15 @@ class NewsSentiment:
# artifact). Without it the two are indistinguishable in the archive
# and a model trained on the mix inherits the keyword scorer's errors.
score_source: str = "keyword"
# DeepSeek's gold-specific score for THIS headline, whatever currencies
# it names. sentiment_by_currency["XAU"] carries it only when the
# headline matched a gold keyword; for a pure Fed / dollar / yields
# headline it used to be discarded, which left 930 of the first 1,546
# archived DeepSeek rows without a gold label (eval 2026-09-17).
# Nothing displayed or averaged reads it — the archive stores it as
# training data for a local scorer. None until DeepSeek has scored
# the headline (the keyword scorer never sets it).
xau_score: Optional[float] = None
@dataclass
@@ -371,6 +380,9 @@ class NewsEngine:
_llm_summary_cache: Tuple[float, str] = (0.0, "")
_LLM_SENT_CACHE_MAX = 400
_LLM_EVENT_CACHE_MAX = 100
#: Headlines per DeepSeek sentiment call — the reply must fit the
#: client's max_tokens (~10 tokens per headline, 2000 max).
_LLM_BATCH_CAP = 25
# Per-source fetch health for status displays (telegram /status).
# name → {"ok": ts|None, "err": ts|None, "detail": str, "extra": str}.
@@ -811,7 +823,8 @@ class NewsEngine:
def _fetch_news_sentiment_raw(self) -> List[NewsSentiment]:
"""Fetch and analyze news headlines for sentiment."""
headlines = []
headlines = [] # names a tracked currency: what the platform uses
unmatched = [] # names none: scored and archived, never shown
seen_titles = set()
ok_feeds = 0
undated = 0
@@ -861,7 +874,6 @@ class NewsEngine:
sentiment = self._analyze_headline_sentiment(title)
currencies = self._extract_currencies(title)
if currencies:
link_el = item.find("link")
desc_el = item.find("description")
src_el = item.find("source")
@@ -877,7 +889,7 @@ class NewsEngine:
publisher = None
if src_el is not None:
publisher = (src_el.text or "").strip() or None
headlines.append(NewsSentiment(
entry = NewsSentiment(
headline=title,
source=feed_url,
timestamp=pub_date or datetime.now(timezone.utc),
@@ -892,7 +904,18 @@ class NewsEngine:
link_el.text if link_el is not None else None),
description=desc,
publisher=publisher,
))
)
# A headline naming no tracked currency stays invisible
# to the platform (no per-currency mean, no display,
# no panel) but is scored and archived since
# 2026-09-17: it was 23% of a day's feed, and DeepSeek
# gave 10 of those 24 a gold direction — the Iran /
# Hormuz stories were the strongest gold headlines of
# the day. Training data only.
if currencies:
headlines.append(entry)
else:
unmatched.append(entry)
ok_feeds += 1
self._record_source(
@@ -905,14 +928,21 @@ class NewsEngine:
if undated:
logger.debug(f"RSS: dropped {undated} headlines without a publish time")
if unmatched:
logger.debug(f"RSS: {len(unmatched)} headlines name no tracked "
"currency (scored and archived, not shown)")
self._record_source(
"headlines", ok_feeds > 0,
detail="all feeds failed" if ok_feeds == 0 else "",
extra=(f"{len(headlines)} headlines from "
f"{ok_feeds}/{len(self.config.rss_feeds)} feeds"))
# Score and archive EVERYTHING fetched; hand back only the
# currency-matched headlines so display, per-currency means, the
# panel and the summary see exactly what they saw before.
scored = headlines + unmatched
try:
self._apply_llm_sentiment(headlines)
self._apply_llm_sentiment(scored)
except Exception as e:
logger.debug(f"LLM sentiment failed, keyword scores stand: {e}")
@@ -927,7 +957,7 @@ class NewsEngine:
# effect — it must never cost us the headlines we just fetched.
try:
news_archive.archive_headlines(
headlines,
scored,
teacher_model=getattr(get_llm_client(), "_model", None),
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1)
except Exception as e:
@@ -1003,24 +1033,31 @@ class NewsEngine:
"""Overwrite keyword sentiment on `headlines` with LLM scores.
Runs inside _cached_fetch, i.e. UNDER _cache_lock — cache access
is direct and the (rare) HTTP call blocks sibling scans at most
is direct and the (rare) HTTP calls block sibling scans at most
once per cache window, matching the existing shared-fetch design.
Only headlines missing from the cache are sent; a failed batch
changes nothing.
Only headlines missing from the cache are sent, in batches of
_LLM_BATCH_CAP; a failed batch changes nothing and ends the
cycle's sending.
"""
if not headlines or not get_llm_client().available:
return
cache = NewsEngine._llm_sent_cache
keyed = [(hashlib.sha1(h.headline.encode()).hexdigest(), h)
for h in headlines]
# Cap the batch so the reply fits the client's max_tokens — the
# overflow stays keyword-scored this cycle and, still uncached,
# goes out with the next window's batch.
new = [(k, h) for k, h in keyed if k not in cache][:25]
if new:
scores = self._llm_headline_scores([h.headline for _, h in new])
if scores is not None:
for (k, _), sc in zip(new, scores):
# Every new headline goes out this cycle, one call per batch.
# Until 2026-09-17 only the first 25 went and the overflow stayed
# keyword-scored (31 of the first 1,577 archived headlines never
# got a DeepSeek label that way). A failed call ends the loop: a
# timeout or the daily cap would fail the next one too, and what
# is left stays keyword-scored and, still uncached, goes out with
# the next window.
new = [(k, h) for k, h in keyed if k not in cache]
for start in range(0, len(new), self._LLM_BATCH_CAP):
batch = new[start:start + self._LLM_BATCH_CAP]
scores = self._llm_headline_scores([h.headline for _, h in batch])
if scores is None:
break
for (k, _), sc in zip(batch, scores):
cache[k] = sc
while len(cache) > self._LLM_SENT_CACHE_MAX:
cache.pop(next(iter(cache)))
@@ -1034,6 +1071,7 @@ class NewsEngine:
cur: (xau if cur == "XAU" else overall)
for cur in h.currencies_mentioned
}
h.xau_score = xau
h.score_source = "llm"
def _llm_event_direction(self, event: EconomicEvent) -> Optional[str]:
scripts/build_panel_base_rates.py
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env python3
"""Build agent_panel_base_rates.json from the reference backtest books.
The agent panel (agent_panel.py, prompt version v2) anchors every
persona's probability to how trades of the same class fared in the
validated books: (timeframe, strategy, grade, direction) with fallbacks
to (timeframe, direction) and (timeframe). This script turns the exported
63-column book CSVs into that table. Research-side (pandas only); the
live module reads the JSON and never imports this.
Usage (the section-9 harness books, regenerated from the kept cache):
venv/bin/python scripts/build_panel_base_rates.py \\
--book 5min=/path/m5_w2.csv --book 15min=/path/m15_w2.csv \\
--book 1h=/path/h1_w3.csv --book 4h=/path/h4_w3.csv \\
--out agent_panel_base_rates.json
Statistics per class: n, win_rate (percent of trades with r_multiple >
0), mean_r, tp1_reach_rate (percent whose best excursion reached the
planned first target: mfe_r >= rr_planned), median_bars_held. Every
class with n >= 1 is written; agent_panel._base_rate applies the
minimum-n rule (30) and the fallback order at read time.
Re-run whenever a reference book is re-pinned (CLAUDE.md section 9); the
JSON records each book's file name, MD5 and trade count so the table's
provenance is auditable.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import sys
from datetime import datetime, timezone
from typing import Dict, List
import pandas as pd
STATS_COLS = ("r_multiple", "mfe_r", "rr_planned", "bars_held",
"strategy", "quality", "direction")
def _md5(path: str) -> str:
h = hashlib.md5()
with open(path, "rb") as fh:
for chunk in iter(lambda: fh.read(1 << 20), b""):
h.update(chunk)
return h.hexdigest()
def _stats(df: pd.DataFrame, book: str) -> Dict[str, object]:
r = df["r_multiple"].astype(float)
reached = (df["mfe_r"].astype(float) >= df["rr_planned"].astype(float))
return {
"n": int(len(df)),
"win_rate": round(float((r > 0).mean() * 100.0), 2),
"mean_r": round(float(r.mean()), 4),
"tp1_reach_rate": round(float(reached.mean() * 100.0), 2),
"median_bars_held": int(round(float(df["bars_held"].median()))),
"book": book,
}
def build(books: Dict[str, str]) -> Dict[str, object]:
classes: Dict[str, Dict[str, object]] = {}
meta: Dict[str, Dict[str, object]] = {}
for tf, path in books.items():
df = pd.read_csv(path)
missing = [c for c in STATS_COLS if c not in df.columns]
if missing:
raise SystemExit(f"{path}: missing columns {missing}")
book = path.rsplit("/", 1)[-1]
meta[tf] = {"file": book, "md5": _md5(path), "trades": int(len(df)),
"first_entry": str(df["entry_time"].min()),
"last_entry": str(df["entry_time"].max())}
if df.empty:
continue
classes[f"{tf}|*|*|*"] = _stats(df, book)
for direction, g in df.groupby("direction"):
classes[f"{tf}|*|*|{direction}"] = _stats(g, book)
for (strategy, grade, direction), g in df.groupby(
["strategy", "quality", "direction"]):
classes[f"{tf}|{strategy}|{grade}|{direction}"] = _stats(g, book)
return {
"schema": "agent-panel-base-rates-v1",
"generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
"key": "timeframe|strategy|grade|direction ('*' = any)",
"min_n_for_exact_class": 30,
"books": meta,
"classes": dict(sorted(classes.items())),
}
def main(argv: List[str] | None = None) -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--book", action="append", default=[], metavar="TF=CSV",
help="timeframe (5min/15min/1h/4h) = exported book CSV")
ap.add_argument("--out", default="agent_panel_base_rates.json")
args = ap.parse_args(argv)
books: Dict[str, str] = {}
for item in args.book:
if "=" not in item:
ap.error(f"--book expects TF=CSV, got {item!r}")
tf, path = item.split("=", 1)
books[tf.strip()] = path.strip()
if not books:
ap.error("at least one --book is required")
table = build(books)
with open(args.out, "w", encoding="utf-8") as fh:
json.dump(table, fh, indent=1, sort_keys=False)
fh.write("\n")
n_exact = sum(1 for k in table["classes"] if "|*|" not in k)
print(f"wrote {args.out}: {len(table['classes'])} classes "
f"({n_exact} exact) from {len(books)} book(s)")
for tf, m in table["books"].items():
print(f" {tf:6s} {m['file']} md5={m['md5']} trades={m['trades']}")
return 0
if __name__ == "__main__":
sys.exit(main())
scripts/panel_replay.py
+514
View File
@@ -0,0 +1,514 @@
#!/usr/bin/env python3
"""Offline replay of the agent panel on logged setups, scored against the
realised outcome — so a prompt or payload version is judged on the SAME
trades as its predecessor before it is enabled live.
venv/bin/python scripts/panel_replay.py --db logs/signals.db --dry-run
venv/bin/python scripts/panel_replay.py --db logs/signals.db --max-calls 160 --out replay.csv
What it does
------------
1. Loads every resolved setup event (analyze_signal_log's dedup: one
event per identical setup, first occurrence) whose advisory outcome is
RESOLVED; the outcome is `advisory_r_multiple > 0` (net R at $0.65).
2. Rebuilds what each persona would see. Rows logged by agent_panel v2
carry the exact payloads (panel_payloads_json); older rows are
reconstructed from the stored indicators_json / setup_json / mtf_json /
penalty_tags_json / reasons_json / warnings_json, the interned
headlines and events (signals_with_news), and candles from the kept
Oanda cache windows so the price path is what the scan saw. Calendar
events are evaluated AS OF the signal bar (released / minutes_until),
not as of today.
3. Runs the CURRENT prompts (agent_panel.PANEL_PROMPT_VERSION) through the
real DeepSeek client on each payload, validates the replies with the
live code path, aggregates with agent_panel.mean_verdict.
4. Scores three predictors on the same events: the new run, the logged
v1 takes (stance x confidence -> pseudo-probability, when present) and
the base-rate-only predictor (the class win rate for every trade):
Brier score, log-loss, calibration by probability band, discrimination
(mean R and win rate by band), constant-voter rate per persona, and the
counterfactual books (all / skip p <= 40 / only p >= 60).
Spend: 4 calls per event against the shared DeepSeek daily cap (500);
`--max-calls` is a hard stop. `--dry-run` builds and writes the payloads
without a single call. Read-only on the database.
Caveat printed with every summary: a few dozen outcomes from one regime
can reject a pathological prompt (constant voting, narrative anchoring);
they cannot prove accuracy. The live scorecard at 100 and 300 outcomes
(analyze_signal_log.py --stored-candles) is the real test.
"""
from __future__ import annotations
import argparse
import glob
import json
import math
import os
import sys
from datetime import datetime, timedelta, timezone
from types import SimpleNamespace
from typing import Any, Dict, List, Optional, Sequence, Tuple
import pandas as pd
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if ROOT not in sys.path:
sys.path.insert(0, ROOT)
import agent_panel as ap # noqa: E402
from analyze_signal_log import (cluster_setup_rows, load_rows, # noqa: E402
open_readonly)
from config import SignalStrength, Timeframe # noqa: E402
from news_engine import (EconomicEvent, FundamentalContext, # noqa: E402
NewsSentiment)
from signal_aggregator import SignalReport, TradeSetup # noqa: E402
from signal_log import read_news # noqa: E402
PERSONAS = ap.ALL_PERSONAS # research default: score every persona
CACHE_DIR = os.path.join(ROOT, "backtest", "cache")
_TF = {tf.value: tf for tf in Timeframe}
# ── Reconstruction ────────────────────────────────────────────────────
class _AsOfEvent(EconomicEvent):
"""EconomicEvent whose `is_upcoming` / `minutes_until` are evaluated
relative to the signal bar, not the wall clock."""
def __init__(self, as_of: datetime, **kw):
super().__init__(**kw)
self._as_of = as_of
@property
def is_upcoming(self) -> bool: # type: ignore[override]
return self.time > self._as_of
@property
def minutes_until(self) -> float: # type: ignore[override]
return (self.time - self._as_of).total_seconds() / 60.0
def _dt(s: Optional[str]) -> Optional[datetime]:
if not s:
return None
try:
d = datetime.fromisoformat(str(s).replace("Z", "+00:00"))
except ValueError:
return None
return d if d.tzinfo else d.replace(tzinfo=timezone.utc)
def _loads(s: Any, default: Any) -> Any:
if s in (None, "", "null"):
return default
try:
return json.loads(s)
except (TypeError, ValueError):
return default
def _analysis(tf: str, blob: Dict[str, Any]) -> SimpleNamespace:
inds = {}
for name, d in (blob.get("indicators") or {}).items():
try:
sig = SignalStrength[d.get("signal", "NEUTRAL")]
except KeyError:
sig = SignalStrength.NEUTRAL
inds[name] = SimpleNamespace(
signal=sig, confidence=float(d.get("confidence") or 0.0),
weight=float(d.get("weight") or 0.0), value=d.get("value"),
description=d.get("description") or "")
return SimpleNamespace(timeframe=tf, timestamp=blob.get("timestamp"),
key_levels=dict(blob.get("key_levels") or {}),
indicators=inds)
def _context_from_row(events_json: Optional[str], headlines_json: Optional[str],
as_of: datetime) -> FundamentalContext:
ctx = FundamentalContext()
ev = _loads(events_json, {}) or {}
for key in ("events_today", "events_upcoming_1h"):
out = []
for e in ev.get(key) or []:
t = _dt(e.get("time"))
if t is None:
continue
out.append(_AsOfEvent(as_of, time=t, currency=e.get("currency") or "",
impact=e.get("impact") or "", event=e.get("event") or "",
forecast=e.get("forecast"), previous=e.get("previous"),
actual=e.get("actual")))
setattr(ctx, key, out)
ctx.news_sentiment = dict(ev.get("sentiment_by_currency") or {})
ctx.calendar_available = bool(ev.get("calendar_available", False))
for h in _loads(headlines_json, []) or []:
t = _dt(h.get("timestamp")) or as_of
ctx.news_headlines.append(NewsSentiment(
headline=h.get("headline") or "", source=h.get("source") or "",
timestamp=t, currencies_mentioned=list(h.get("currencies") or []),
sentiment_score=float(h.get("score") or 0.0),
keywords=list(h.get("keywords") or []),
sentiment_by_currency={k: float(v) for k, v in
(h.get("score_by_currency") or {}).items()
if v is not None},
score_source=h.get("score_source") or "keyword"))
return ctx
_frame_cache: Dict[str, pd.DataFrame] = {}
_have_cols: set = set()
def _col(name: str) -> str:
"""Column reference that tolerates a database logged before the
column existed (a read-only copy is never migrated)."""
return name if name in _have_cols else f"NULL AS {name}"
def _cached_frame(tf: str) -> Optional[pd.DataFrame]:
"""Newest kept Oanda cache window for `tf` (raw bars, UTC index)."""
if tf in _frame_cache:
return _frame_cache[tf]
files = sorted(glob.glob(os.path.join(CACHE_DIR, f"oanda_XAU_USD_{tf}_*.parquet")),
key=lambda p: p.rsplit("_", 1)[-1])
df = None
if files:
try:
df = pd.read_parquet(files[-1])
if df.index.tz is None:
df.index = df.index.tz_localize("UTC")
except Exception:
df = None
_frame_cache[tf] = df
return df
def _frames_as_of(tfs: Sequence[str], bar_ts: datetime) -> Dict[Timeframe, pd.DataFrame]:
out: Dict[Timeframe, pd.DataFrame] = {}
for tf in tfs:
df = _cached_frame(tf)
tfe = _TF.get(tf)
if df is None or tfe is None:
continue
sl = df[df.index <= pd.Timestamp(bar_ts)]
if len(sl) >= 3:
out[tfe] = sl
return out
def build_report(con, rowid: int) -> Tuple[Optional[SignalReport], Optional[FundamentalContext],
Dict[Timeframe, pd.DataFrame], Optional[Dict[str, Any]]]:
"""(report, context, frames, stored_payloads) for one logged row."""
row = con.execute(
"SELECT symbol, timeframe, " + _col("bar_ts") + ", logged_at, signal, score,"
" confidence, quality, price, market_regime, volatility_regime, mtf_json,"
" setup_json, reasons_json, warnings_json, event_risk, event_description,"
" news_sentiment, indicators_json, " + _col("penalty_tags_json") + ", "
+ _col("exhaustion_count") + ", " + _col("penalty_total") + ", "
+ _col("momentum_fading") + ", " + _col("panel_payloads_json")
+ " FROM signals WHERE id=?", (rowid,)).fetchone()
if row is None:
return None, None, {}, None
(symbol, tf, bar_ts, logged_at, signal, score, conf, quality, price, regime, vol,
mtf_json, setup_json, reasons_json, warnings_json, event_risk, event_desc, news_sent,
ind_json, pen_json, exh, pen_total, mom, payloads_json) = row
inds = _loads(ind_json, {}) or {}
bar_label = bar_ts or (inds.get(tf) or {}).get("timestamp")
as_of_bar = _dt(bar_label) or _dt(logged_at)
if as_of_bar is None:
return None, None, {}, None
secs = ap._TF_SECONDS.get(tf, 900)
as_of = as_of_bar + timedelta(seconds=secs) # the bar closed
report = SignalReport(symbol=symbol, display_name="XAU/USD",
timestamp=str(bar_label or logged_at))
try:
report.final_signal = SignalStrength[signal]
except KeyError:
report.final_signal = SignalStrength.NEUTRAL
report.final_score = float(score or 0.0)
report.final_confidence = float(conf or 0.0)
report.signal_quality = quality or ""
report.current_price = float(price or 0.0)
report.market_regime = regime or ""
report.volatility_regime = vol or ""
mtf = _loads(mtf_json, {}) or {}
report.mtf_agreement = bool(mtf.get("agreement", False))
for k, v in (mtf.get("signals") or {}).items():
try:
report.mtf_signals[k] = SignalStrength[v]
except KeyError:
report.mtf_signals[k] = SignalStrength.NEUTRAL
report.mtf_scores = {k: float(v or 0.0) for k, v in (mtf.get("scores") or {}).items()}
reasons = _loads(reasons_json, {}) or {}
report.bullish_reasons = list(reasons.get("bullish") or [])
report.bearish_reasons = list(reasons.get("bearish") or [])
report.warnings = list(_loads(warnings_json, []) or [])
report.event_risk = event_risk or "clear"
report.event_description = event_desc or ""
report.news_sentiment = float(news_sent or 0.0)
report.exhaustion_count = int(exh or 0)
report.penalty_tags = {str(k): float(v or 0.0)
for k, v in (_loads(pen_json, {}) or {}).items()}
report.penalty_total = float(pen_total or 0.0)
report.momentum_fading = bool(mom)
setup = _loads(setup_json, None)
if setup:
report.trade_setup = TradeSetup(
pair=symbol, display_name="XAU/USD", direction=setup.get("direction"),
entry_price=float(setup.get("entry") or 0.0),
stop_loss=float(setup.get("stop_loss") or 0.0),
take_profit_1=float(setup.get("take_profit_1") or 0.0),
take_profit_2=float(setup.get("take_profit_2") or 0.0),
risk_reward=float(setup.get("risk_reward") or 0.0),
risk_pips=float(setup.get("risk_pips") or 0.0),
reward_pips=float(setup.get("reward_pips") or 0.0),
pip_value=0.10, strategy=setup.get("strategy") or "GENERIC",
setup_pattern=setup.get("setup_pattern"),
round_trip_cost=setup.get("round_trip_cost"),
trail_arm_r=setup.get("trail_arm_r"), trail_dist_r=setup.get("trail_dist_r"))
if tf in inds:
report.primary_analysis = _analysis(tf, inds[tf])
report.confirmation_analyses = [_analysis(k, v) for k, v in inds.items() if k != tf]
events_json, headlines_json = read_news(con, rowid)
context = _context_from_row(events_json, headlines_json, as_of)
frames = _frames_as_of([tf] + [k for k in inds if k != tf], as_of_bar)
return report, context, frames, _loads(payloads_json, None)
def build_jobs(report: SignalReport, context: FundamentalContext,
frames: Dict[Timeframe, pd.DataFrame], stored: Optional[Dict[str, Any]],
settings: Dict[str, Any]) -> Tuple[Tuple[str, str, Dict[str, Any]], ...]:
"""Persona jobs on the CURRENT prompts: stored payloads when the row
has them, else rebuilt through the live payload builders."""
systems = {"macro": ap._MACRO_SYSTEM, "technical": ap._TECH_SYSTEM,
"risk": ap._RISK_SYSTEM, "contrarian": ap._CONTRARIAN_SYSTEM}
if stored and all(p in stored for p in PERSONAS):
return tuple((p, systems[p], stored[p]) for p in PERSONAS)
candles: Dict[str, Any] = {}
analyses = [report.primary_analysis] + list(report.confirmation_analyses)
counts = [int(settings.get("candle_bars_primary", ap.DEFAULT_CANDLE_BARS_PRIMARY))] + \
[int(settings.get("candle_bars_confirm", ap.DEFAULT_CANDLE_BARS_CONFIRM))] * \
len(report.confirmation_analyses)
for analysis, n in zip(analyses, counts):
if analysis is None:
continue
atr = ap._finite(getattr((analysis.indicators or {}).get("atr"), "value", None))
df = frames.get(_TF.get(analysis.timeframe))
if atr and atr > 0 and df is not None and len(df) >= 2:
candles[analysis.timeframe] = ap._normalize_candles(df, atr, n)
ctx = ap._price_context(report, frames, context)
return ap._panel_jobs(report, context, candles or None, ctx, PERSONAS)
# ── Scoring ───────────────────────────────────────────────────────────
def _brier(ps: Sequence[float], ys: Sequence[int]) -> float:
return sum((p / 100.0 - y) ** 2 for p, y in zip(ps, ys)) / len(ps)
def _logloss(ps: Sequence[float], ys: Sequence[int]) -> float:
eps = 1e-3
tot = 0.0
for p, y in zip(ps, ys):
q = min(1 - eps, max(eps, p / 100.0))
tot += -(y * math.log(q) + (1 - y) * math.log(1 - q))
return tot / len(ps)
def score_predictor(name: str, ps: Sequence[Optional[float]], rs: Sequence[float]) -> Dict[str, Any]:
pairs = [(p, r) for p, r in zip(ps, rs) if p is not None]
if not pairs:
return {"predictor": name, "n": 0}
P = [p for p, _ in pairs]; R = [r for _, r in pairs]; Y = [1 if r > 0 else 0 for r in R]
bands = {}
for label, lo, hi in (("<=40", -1, ap.P_OPPOSE), ("40-60", ap.P_OPPOSE, ap.P_SUPPORT),
(">=60", ap.P_SUPPORT, 101)):
sel = [(p, r) for p, r in pairs if (lo < p <= hi if label == "<=40" else
lo < p < hi if label == "40-60" else p >= lo)]
if sel:
bands[label] = {"n": len(sel), "mean_p": round(sum(p for p, _ in sel) / len(sel), 1),
"win_rate": round(100 * sum(1 for _, r in sel if r > 0) / len(sel), 1),
"mean_r": round(sum(r for _, r in sel) / len(sel), 3),
"sum_r": round(sum(r for _, r in sel), 2)}
else:
bands[label] = {"n": 0}
stances = [ap.derive_stance(p) for p in P]
modal = max(set(stances), key=stances.count)
return {
"predictor": name, "n": len(P),
"brier": round(_brier(P, Y), 4), "logloss": round(_logloss(P, Y), 4),
"mean_p": round(sum(P) / len(P), 1), "win_rate": round(100 * sum(Y) / len(Y), 1),
"bands": bands, "modal_stance": modal,
"constant_voter_rate": round(100 * stances.count(modal) / len(stances), 1),
"sum_r_all": round(sum(R), 2),
"sum_r_skip_oppose": round(sum(r for p, r in pairs if p > ap.P_OPPOSE), 2),
"sum_r_support_only": round(sum(r for p, r in pairs if p >= ap.P_SUPPORT), 2),
}
def print_summary(results: List[Dict[str, Any]]) -> None:
print("\n== Predictor scorecard (outcome = advisory R > 0 at $0.65) ==")
print(f"{'predictor':22s} {'n':>3s} {'Brier':>6s} {'logloss':>7s} {'mean p':>6s} {'win%':>5s} "
f"{'modal':>8s} {'const%':>6s} {'R all':>7s} {'R skip<=40':>10s} {'R >=60 only':>11s}")
for r in results:
if r.get("n", 0) == 0:
print(f"{r['predictor']:22s} 0"); continue
print(f"{r['predictor']:22s} {r['n']:3d} {r['brier']:6.3f} {r['logloss']:7.3f} {r['mean_p']:6.1f} "
f"{r['win_rate']:5.1f} {r['modal_stance']:>8s} {r['constant_voter_rate']:6.1f} "
f"{r['sum_r_all']:+7.2f} {r['sum_r_skip_oppose']:+10.2f} {r['sum_r_support_only']:+11.2f}")
print("\n calibration bands (n / mean p / observed win% / mean R):")
for r in results:
if r.get("n", 0) == 0:
continue
cells = " ".join(f"{k}: {v['n']}/{v.get('mean_p', '-')}/{v.get('win_rate', '-')}/{v.get('mean_r', '-')}"
if v["n"] else f"{k}: 0" for k, v in r["bands"].items())
print(f" {r['predictor']:22s} {cells}")
print("\nCaveat: a few dozen outcomes from one regime can reject a pathological prompt "
"(constant voting, narrative anchoring); they cannot prove accuracy.")
# ── Main ──────────────────────────────────────────────────────────────
def main(argv: Optional[Sequence[str]] = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--db", default=os.path.join(ROOT, "logs", "signals.db"))
parser.add_argument("--limit", type=int, default=0, help="max events (0 = all)")
parser.add_argument("--max-calls", type=int, default=200,
help="hard stop on LLM requests (4 per event)")
parser.add_argument("--dry-run", action="store_true",
help="build payloads only; no LLM call")
parser.add_argument("--out", default=None, help="per-event CSV (and .payloads.json on --dry-run)")
parser.add_argument("--include-pending", action="store_true",
help="also build payloads for unresolved setups (dry-run only)")
parser.add_argument("--personas", default=",".join(ap.ALL_PERSONAS),
help="comma-separated personas to run (default: all four, so every "
"voice is scored; the live default is agent_panel.personas = macro)")
args = parser.parse_args(argv)
global PERSONAS
PERSONAS = ap.normalize_personas([p for p in args.personas.split(",") if p.strip()])
rows = load_rows(args.db)
events = cluster_setup_rows(rows)
con = open_readonly(args.db)
global _have_cols
_have_cols = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
settings = {"candle_bars_primary": ap.DEFAULT_CANDLE_BARS_PRIMARY,
"candle_bars_confirm": ap.DEFAULT_CANDLE_BARS_CONFIRM}
picked = []
for ev in events:
res = next((r for r in ev.rows if r.advisory_status == "RESOLVED" and r.advisory_r is not None), None)
if res is None and not (args.dry_run and args.include_pending):
continue
picked.append((ev, res))
if args.limit:
picked = picked[:args.limit]
print(f"events: {len(events)} total, {len(picked)} selected "
f"({'dry run' if args.dry_run else 'live calls, max ' + str(args.max_calls)}), "
f"prompt version {ap.PANEL_PROMPT_VERSION}, personas {','.join(PERSONAS)}")
client = None if args.dry_run else ap.get_llm_client()
calls = 0
per_event: List[Dict[str, Any]] = []
payload_dump: Dict[str, Any] = {}
for ev, res in picked:
rep = ev.representative
# Prefer a member row that already carries the v2 payloads.
rowid = rep.id
for r in ev.rows:
has = (con.execute("SELECT panel_payloads_json IS NOT NULL FROM signals"
" WHERE id=?", (r.id,)).fetchone()
if "panel_payloads_json" in _have_cols else None)
if has and has[0]:
rowid = r.id
break
report, context, frames, stored = build_report(con, rowid)
if report is None or report.trade_setup is None:
continue
jobs = build_jobs(report, context, frames, stored, settings)
ctx = next((payload.get("context") for _, _, payload in jobs if payload.get("context")), {}) or {}
base_p = (ctx.get("base_rate") or {}).get("win_rate")
# logged v1 takes (any member row) -> pseudo-probabilities
v1 = {}
for r in ev.rows:
takes = con.execute("SELECT panel_takes_json, " + _col("panel_prompt_version")
+ " FROM signals WHERE id=?", (r.id,)).fetchone()
if takes and takes[0]:
for t in _loads(takes[0], []) or []:
p = t.get("p_win")
if p is None and t.get("stance") in ap._VALID_STANCES:
p = ap.pseudo_probability(t["stance"], float(t.get("confidence") or 0))
if p is not None:
v1[t.get("persona")] = float(p)
v1["_version"] = takes[1] or "v1-logged"
break
entry: Dict[str, Any] = {
"event_row": rep.id, "bar": report.timestamp, "tf": rep.timeframe,
"direction": report.trade_setup.direction, "grade": rep.quality,
"strategy": report.trade_setup.strategy,
"R": res.advisory_r if res else None, "stored_payloads": bool(stored),
"base_p": base_p, "prior_move_4h_atr": ctx.get("prior_move_4h_atr"),
"move_from_start_atr": ctx.get("move_from_24h_start_atr"),
"big_candle_last3": ctx.get("big_candle_last3"),
}
for p in PERSONAS:
entry[f"v1_{p}"] = v1.get(p)
v1_ps = [v1[p] for p in PERSONAS if p in v1]
entry["v1_mean"] = round(sum(v1_ps) / len(v1_ps), 1) if len(v1_ps) >= min(ap.MEAN_QUORUM, len(v1_ps) or 1) and v1_ps else None
if args.dry_run:
payload_dump[str(rep.id)] = {name: payload for name, _, payload in jobs}
else:
takes = []
for name, system, payload in jobs:
if calls >= args.max_calls:
break
calls += 1
reply = client.chat_json(system, payload)
take = ap._validate_take(name, reply)
if take is not None:
takes.append(take)
entry[f"new_{name}"] = take.p_win
entry[f"new_{name}_for"] = take.reason_for
entry[f"new_{name}_against"] = take.reason_against
v = ap.mean_verdict(takes, quorum=min(ap.MEAN_QUORUM, len(jobs)))
entry["new_mean"] = v.probability if v else None
entry["new_verdict"] = v.verdict if v else None
print(f" #{rep.id:<5d} {report.timestamp[:16]} {rep.timeframe:5s} "
f"{report.trade_setup.direction:4s} {rep.quality:2s} R={entry['R']:+.2f} "
f"base={base_p} new_mean={entry['new_mean']} "
f"({', '.join(f'{t.persona[:4]} {t.p_win:.0f}' for t in takes)}) "
f"v1_mean={entry['v1_mean']}")
per_event.append(entry)
if not args.dry_run and calls >= args.max_calls:
print(f" stopped at --max-calls {args.max_calls}")
break
con.close()
if args.dry_run:
print(f"built payloads for {len(per_event)} events")
if args.out:
with open(args.out.rsplit('.', 1)[0] + ".payloads.json", "w") as fh:
json.dump(payload_dump, fh, indent=1, sort_keys=True)
print(f"wrote {args.out.rsplit('.', 1)[0]}.payloads.json")
scored = [e for e in per_event if e["R"] is not None]
if scored:
rs = [e["R"] for e in scored]
results = [score_predictor("base-rate only",
[e["base_p"] if e["base_p"] is not None else None for e in scored], rs),
score_predictor("v1 logged (mean)", [e["v1_mean"] for e in scored], rs)]
for p in PERSONAS:
results.append(score_predictor(f"v1 logged {p}", [e.get(f"v1_{p}") for e in scored], rs))
if not args.dry_run:
results.append(score_predictor(f"NEW {ap.PANEL_PROMPT_VERSION} (mean)",
[e.get("new_mean") for e in scored], rs))
for p in PERSONAS:
results.append(score_predictor(f"NEW {p}", [e.get(f"new_{p}") for e in scored], rs))
print_summary(results)
if args.out and per_event:
pd.DataFrame(per_event).to_csv(args.out, index=False)
print(f"wrote {args.out} ({len(per_event)} events)")
return 0
if __name__ == "__main__":
sys.exit(main())
signal_aggregator.py
+16 -1
View File
@@ -115,7 +115,16 @@ class SignalReport:
panel_created_at: str = ""
panel_cache_status: str = ""
panel_payload_hash: str = ""
panel_verdict_source: str = "" # "macro" / "chair" / ""
panel_verdict_source: str = "" # "mean" / "macro" / "chair" / ""
# v2 panel (2026-09-17): the mean persona probability (0-100) that
# the trade closes profitably, the prompt version that produced it,
# the exact per-persona payloads and the shared code-computed
# context — all advisory/DISPLAY-ONLY, logged for the offline replay
# harness (scripts/panel_replay.py) and the calibration scorecard.
panel_probability: float = 0.0
panel_prompt_version: str = ""
panel_payloads: Dict = field(default_factory=dict)
panel_context: Dict = field(default_factory=dict)
# Violent-move watch line (advisory/DISPLAY-ONLY). Stamped post-hoc
# by surfaces via violent_watch.attach_watch — never populated
@@ -277,6 +286,12 @@ class SignalAggregator:
dfs=dfs,
use_news=True,
)
# Live-only hand-off for the post-hoc advisory add-ons (the agent
# panel's price context reads the frames this scan analysed, so
# it sees exactly the bars the engine saw). A private attribute
# like _signal_log_rowid: not a dataclass field, never read by
# the core, absent on backtest replays by construction.
report._frames = dfs
if stale_tfs:
report.warnings.append(f"STALE DATA on {', '.join(stale_tfs)} — signal may be unreliable")
signal_log.py
+45 -3
View File
@@ -152,7 +152,12 @@ CREATE TABLE IF NOT EXISTS signals (
panel_payload_hash TEXT,
advisory_outcome_status TEXT,
advisory_r_multiple REAL,
advisory_outcome_json TEXT
advisory_outcome_json TEXT,
panel_prompt_version TEXT,
panel_probability REAL,
panel_probs_json TEXT,
panel_payloads_json TEXT,
panel_context_json TEXT
);
CREATE INDEX IF NOT EXISTS idx_signals_sym_tf_ts
ON signals (symbol, timeframe, logged_at);
@@ -225,6 +230,16 @@ _ADDITIVE_SIGNAL_COLS = (
("watch_level", "TEXT"),
("watch_kinds", "TEXT"),
("watch_line", "TEXT"),
# 2026-09-17 agent-panel v2 (see agent_panel.py): prompt era, the mean
# persona probability, per-persona probabilities + reasons, the exact
# per-persona payloads and the shared context — written post-hoc by
# update_panel so scripts/panel_replay.py can re-score any prompt
# version on the same evidence.
("panel_prompt_version", "TEXT"),
("panel_probability", "REAL"),
("panel_probs_json", "TEXT"),
("panel_payloads_json", "TEXT"),
("panel_context_json", "TEXT"),
)
# Read-side compatibility view `signals_with_news`: legacy rows keep
@@ -1025,6 +1040,11 @@ class SignalLogger:
cache_status: Optional[str] = None,
payload_hash: Optional[str] = None,
verdict_source: Optional[str] = None,
prompt_version: Optional[str] = None,
probability: Optional[float] = None,
probs_json: Optional[str] = None,
payloads_json: Optional[str] = None,
context_json: Optional[str] = None,
) -> None:
"""Fill panel provenance; seeds an outcome only for rows that
log() did not already seed (rows written by an older process)."""
@@ -1051,11 +1071,18 @@ class SignalLogger:
" panel_cache_status=COALESCE(?, panel_cache_status),"
" panel_payload_hash=COALESCE(?, panel_payload_hash),"
" panel_verdict_source=COALESCE(?, panel_verdict_source),"
" panel_prompt_version=COALESCE(?, panel_prompt_version),"
" panel_probability=COALESCE(?, panel_probability),"
" panel_probs_json=COALESCE(?, panel_probs_json),"
" panel_payloads_json=COALESCE(?, panel_payloads_json),"
" panel_context_json=COALESCE(?, panel_context_json),"
" advisory_outcome_status=COALESCE(advisory_outcome_status, ?),"
" advisory_outcome_json=COALESCE(advisory_outcome_json, ?)"
" WHERE id=?",
(verdict, confidence, rationale, takes_json,
created_at, cache_status, payload_hash, verdict_source,
prompt_version, probability, probs_json, payloads_json,
context_json,
advisory_status, advisory_json, rowid))
self._conn.commit()
@@ -1592,15 +1619,30 @@ def log_panel_update(report: Any) -> None:
lg = _loggers.get(path)
if lg is None:
return
takes = getattr(report, "panel_takes", None) or []
probs = [{"persona": t.get("persona"), "p_win": _finite(t.get("p_win")),
"stance": t.get("stance"),
"confidence": _finite(t.get("confidence")),
"reason_for": t.get("reason_for") or "",
"reason_against": t.get("reason_against") or ""}
for t in takes if isinstance(t, dict)]
payloads = getattr(report, "panel_payloads", None) or {}
context = getattr(report, "panel_context", None) or {}
probability = _finite(getattr(report, "panel_probability", None))
lg.update_panel(
int(rowid), verdict,
_finite(getattr(report, "panel_confidence", None)),
getattr(report, "panel_rationale", "") or "",
_dumps(getattr(report, "panel_takes", None) or []),
_dumps(takes),
created_at=getattr(report, "panel_created_at", "") or None,
cache_status=getattr(report, "panel_cache_status", "") or None,
payload_hash=getattr(report, "panel_payload_hash", "") or None,
verdict_source=getattr(report, "panel_verdict_source", "") or None)
verdict_source=getattr(report, "panel_verdict_source", "") or None,
prompt_version=getattr(report, "panel_prompt_version", "") or None,
probability=probability if probability else None,
probs_json=_dumps(probs) if probs else None,
payloads_json=_dumps(payloads) if payloads else None,
context_json=_dumps(context) if context else None)
except Exception:
logger.warning("signal_log: failed to persist panel verdict",
exc_info=True)
static/dashboard.js
+17 -1
View File
@@ -471,7 +471,23 @@ function detailHTML(tf) {
${esc(r.news_summary || "")}</div>`);
}
if (r.panel_verdict) {
if (r.panel_verdict && r.panel_verdict_source === "mean") {
// v2 panel: a probability anchored to the setup class's base rate,
// one reason each way per persona. Advisory only; no verdict colour.
const base = (r.panel_context && r.panel_context.base_rate) || {};
const usual = base.win_rate != null ? ` (usual for this kind of setup: ${Math.round(base.win_rate)}%)` : "";
const takes = (r.panel_takes || []).map(t =>
`<tr><td class="ind-k">${esc(t.persona)}</td>
<td class="num text-end">${t.p_win != null ? Math.round(t.p_win) + "%" : "—"}</td>
<td class="small text-secondary">${esc(t.reason_for || "")}</td>
<td class="small text-secondary">${esc(t.reason_against || "")}</td></tr>`).join("");
out += sec("AI estimate (advisory, still being tested)", `<div class="text-secondary small mb-2">
About a <strong>${Math.round(r.panel_probability || 0)}%</strong> chance this trade pays off${esc(usual)}.<br>
${esc(r.panel_rationale || "")}</div>
${takes ? `<div class="detail-table"><table class="table table-sm d-table">
<thead><tr><th></th><th class="text-end">chance</th><th>for</th><th>against</th></tr></thead>
<tbody>${takes}</tbody></table></div>` : ""}`);
} else if (r.panel_verdict) {
const vcls = r.panel_verdict === "AGREE" ? "text-success"
: r.panel_verdict === "OPPOSE" ? "text-danger" : "text-secondary";
const takes = (r.panel_takes || []).map(t =>
telegram_bot.py
+33 -1
View File
@@ -324,6 +324,36 @@ _DISCLAIMER = (
)
def _panel_estimate_lines(report: SignalReport) -> list[str]:
"""v2 panel (verdict_source="mean"): a plain-language probability, the
class base rate it is anchored to, the biggest worry, and each
persona's own estimate. Advisory only — no colour, no verdict word,
never a recommendation (2026-09-17 review: the old AGREE/OPPOSE line
opposed target-reaching trades as often as it agreed)."""
base = (report.panel_context or {}).get("base_rate") or {}
usual = base.get("win_rate")
line = (f"🤖 AI estimate: about a *{report.panel_probability:.0f}%* "
f"chance this trade pays off")
if usual is not None:
line += f" (usual for this kind of setup: {usual:.0f}%)"
line += "."
if report.panel_rationale:
line += f" {_escape_md(report.panel_rationale)}"
worst = min((t for t in report.panel_takes
if t.get("p_win") is not None and t.get("reason_against")),
key=lambda t: t["p_win"], default=None)
if worst is not None:
line += f" Biggest worry: {_escape_md(worst['reason_against'])}"
line += " Still being tested."
out = [line]
if len(report.panel_takes) > 1: # one persona: the headline says it all
votes = " · ".join(
f"{t['persona']} {t['p_win']:.0f}%" if t.get("p_win") is not None
else f"{t['persona']} -" for t in report.panel_takes)
out.append(f" _{_escape_md(votes)}_")
return out
def format_signal_message(report: SignalReport) -> str:
sig = report.final_signal
emoji = _SIGNAL_EMOJI.get(sig, "⬜")
@@ -423,7 +453,9 @@ def format_signal_message(report: SignalReport) -> str:
extra.append(_escape_md(report.watch_line))
if report.news_summary:
extra.append(f"📰 {_escape_md(report.news_summary)}")
if report.panel_verdict:
if report.panel_verdict and report.panel_verdict_source == "mean":
extra.extend(_panel_estimate_lines(report))
elif report.panel_verdict:
mark = {"AGREE": "✅", "OPPOSE": "⛔", "NEUTRAL": "➖"}.get(
report.panel_verdict, "")
panel_label = ("AI macro view"
test_all.py
+935 -9
View File
File diff suppressed because it is too large Load Diff