docs: rewrite CLAUDE.md as a plain-language developer guide

Same facts, new shape: quick start; the scan pipeline from trigger to
delivery to logging (A to Z) with the real thresholds and gate order;
a file-by-file guide with the import graph; data providers; config;
testing; the nine change rules; the validated per-timeframe overrides;
reference books with MD5s and the kept cache windows; a traps list;
and the decision ledger condensed to one row per dated decision with a
pointer to its research report.

Every claim was checked against the code by a reviewer pass; the
mechanism invariants the old file carried (MACD always computed, the
_-prefixed convention, PB/CP force grade B, penalties hit confidence,
gate 6b label-hour vs close-hour, the news-archive hook position, the
ordered cooldown pass, engine_fingerprint split rule, oppose guard,
holiday blindness) are kept. The previous 58 KB version is in git
history at b0b71e1.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

.gitignore

@@ -0,0 +1,31 @@
+# Secrets
+.env
+.env.local
+config.json
+
+# Editor / tooling
+.claude/
+.vscode/
+.idea/
+
+# Python
+__pycache__/
+*.pyc
+*.egg-info/
+.venv/
+venv/
+.coverage
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Logs
+*.log
+
+# Backtesting data (keep scripts, ignore generated files)
+backtest/cache/
+backtest/*.csv
+backtest/__pycache__/
+.env.*
+logs/

CLAUDE.md

@@ -0,0 +1,536 @@
+# ForexBot2 — developer guide
+
+ForexBot2 is a signal-intelligence system for one market: **gold against the US dollar (XAU/USD)**.
+It does not place trades. Every few minutes it downloads price candles, scores them with nine
+technical indicators, checks that several timeframes agree, grades the result (A+ down to F), and
+turns a good grade into a written trade plan (entry, stop loss, two targets, trailing rule). The
+plan is delivered to a Telegram bot, a command-line report, and a public web dashboard. A
+backtester replays history through the *same* signal code, so a backtest result is what the live
+scanner would have said on that day.
+
+This file is the map of the system. Read section 2 first if you are new. Section 7 lists the rules
+you must follow before changing anything that affects signals. Section 10 lists the traps. Section
+11 is the history of every validated change. The research reports behind those decisions live in `backtest/research/*.md`.
+
+---
+
+## 1. Quick start
+
+```bash
+python -m venv venv && venv/bin/pip install -r requirements.txt   # Python 3.12; a venv/ already exists on the dev box
+cp config.json.example config.json                                 # then fill in the keys (see section 5)
+
+venv/bin/python test_all.py                                        # QA suite, no network; prints its own count
+                                                                   #   (506 tests / 33 modules as of 2026-09-15)
+venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail      # one signal report in the terminal
+                                                                   #   --json full dump, --panel adds the advisory LLM panel,
+                                                                   #   --timeframe M5 | M15 | H1 | H4
+venv/bin/python telegram_bot.py                                    # the bot (refuses to start without credentials)
+venv/bin/python dashboard_publisher.py --once                      # one scan + publish to $DASHBOARD_OUT_DIR
+                                                                   #   (without --once it runs as a candle-close daemon)
+
+# Backtests (timeframe names are 5min / 15min / 1h / 4h, not M5/M15)
+venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --days 365
+venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 \
+    --primary-tf 15min --spread 0.65 --workers 12 --export-csv book.csv
+```
+
+Backtest flags worth knowing: `--spread` (round-trip cost in dollars; the default is 0.30, the
+realistic Oanda figure is 0.65 and must be passed explicitly), `--quality A+,A,B,C`,
+`--strategies`, `--workers`, `--max-hold-bars`, `--warmup-bars`, `--cooldown-bars`, `--no-cache`,
+`--export-csv out.csv`, `--detail`, `--set PATH=JSON` (research override of any config value; an
+unknown path is a hard exit), `--export-features` (appends the 65 decision-time columns; never use
+it on a parity run because it changes the file), `--fixed-rr-tp N` (research only, not shipped),
+`--record DIR [--record-bars]` (research recorder, see section 9).
+
+`test_all.py` is **not pytest**. It uses its own `@test("name")` decorator that runs each test the
+moment the file is imported. Run the whole file; it prints per-module results and exits non-zero
+on any failure. There is no way to run one test on its own. Tests must stay green, and every new
+gate or setting needs a test.
+
+---
+
+## 2. How a scan works, from A to Z
+
+A "scan" is one analysis of one timeframe. The bot runs one scan per timeframe at every candle
+close; the CLI and the dashboard run the same scan on demand. Everything below happens inside one
+scan, in this order.
+
+**1. Trigger.** `telegram_bot.py` wakes 10 seconds after each candle boundary (M5 every 5 min,
+M15 every 15 min, H1 hourly, H4 every 4 hours). `dashboard_publisher.py` has its own copy of the
+same schedule. `main.py` runs once when you call it.
+
+**2. Build the configuration.** `pair_signals/xauusd.py` (`make_config`) creates a
+`PlatformConfig` from `config.py`, applies the deployment overlay from `config.json`
+(`app_config.py`), then applies the validated per-timeframe overrides (section 8). Never build a
+bare `PlatformConfig()` for a real scan; always go through the factory. The confirmation
+timeframes come from the factory map: M5 → M15 + H1, M15 → M5 + H1, H1 → M15 + H4, H4 → H1 + D1.
+
+**3. Fetch candles.** `signal_aggregator.analyze_pair` (the live entry point) first checks the
+calendar: on the forex weekend or a blocked weekday it returns a "blocked" scan and does nothing
+else. Otherwise it asks the active data provider (`providers/`, Oanda today) for the primary
+timeframe and each confirmation timeframe. The provider base class drops the still-forming candle,
+validates and cleans the bars, caches them until the next candle close, and checks freshness. If
+any required frame cannot be fetched after the retry ladder, the scan is logged as "failed" and no
+analysis runs.
+
+**4. Score each timeframe.** `engine.py` (`TechnicalEngine.analyze`) computes ATR first, then nine
+scored indicators: EMA trend (9/21/50/200 ribbon), RSI, MACD, Bollinger bands, ADX, rate of change,
+Keltner channel, divergence, and candlestick patterns. Each indicator votes on a five-step scale
+(STRONG SELL, SELL, NEUTRAL, BUY, STRONG BUY) with a confidence. The engine classifies the market
+regime from ADX and the EMA ribbon: `trending_up`, `trending_down`, `trending` (strong trend,
+mixed direction), `transitioning` (ADX rising fast), or `ranging`. Each regime has its own table of
+per-indicator weight **multipliers** (`regime_weight_table` in `engine.py` is the single source of
+truth); the effective weight is the indicator's base weight × the regime multiplier. The composite
+score is the confidence-weighted average of the votes (vote value × effective weight × confidence),
+scaled by 50 and clipped to −100…+100. Divergence and candlestick only vote when a pattern exists;
+a missing vote is "no opinion", not zero. Three engine conventions matter when editing: MACD is
+always computed even when it is not a scored pillar, because divergence needs its histogram; RSI is
+reinterpreted per regime after classification (`rsi_mode = "regime_aware"`; treating the mixed
+`trending` regime as an uptrend was a −6.8R bias); and private helper series ride on the result as
+`_underscore` attributes, which every scorer and formatter skips along with weight-0 indicators, so
+a new indicator must follow that convention.
+
+**5. Multi-timeframe agreement.** `analyze_from_dataframes` compares the primary timeframe's
+direction with the two confirmation timeframes. Agreement needs 2 of 3. Disagreement costs a
+penalty and blocks the top grades.
+
+**6. News context (warning only).** `news_engine.py` reads the Forex Factory calendar (MyFXBook
+RSS as fallback) and four RSS feeds, scores headlines for gold, and can ask DeepSeek to improve
+the scores. This produces warnings and a text summary. **By design it never changes a signal.**
+The backtester runs with news off, so the live decision must be purely technical too.
+
+**7. Penalties, then the signal and the grade.** When the timeframes agree, confidence gains 10
+points and the score is blended 60/40 with the mean of all timeframe scores. Then a list of
+penalties (opposing timeframes, stretched price, fading momentum, and so on) is subtracted from the
+**confidence** with diminishing weight: each further penalty counts half, and the total is capped
+at 30 points. Penalties never reduce the score itself. If two or more "exhaustion" penalties fire
+together with fading momentum, the trade is hard-blocked (confidence set to 0). Then:
+
+| Result | Rule |
+|---|---|
+| STRONG BUY / STRONG SELL | score ≥ 35 (≤ −35) and confidence ≥ 55 |
+| BUY / SELL | score ≥ 25 (≤ −25) and confidence ≥ 50 |
+| NEUTRAL | anything else |
+| Grade A+ | abs score ≥ 40, confidence ≥ 65, timeframes agree |
+| Grade A | abs score ≥ 33, confidence ≥ 55, timeframes agree |
+| Grade B | abs score ≥ 28, confidence ≥ 50 |
+| Grade C | abs score ≥ 25, confidence ≥ 45 |
+| Grade D | abs score ≥ 15 |
+| Grade F | below that |
+
+**8. The gate chain.** A series of checks can lower the grade or remove the setup. They run in this
+order (`signal_aggregator.py`, steps 6b to 6i): session and day filter (the bar-label hour and weekday
+against the blocked hours and allowed weekdays; the live wrapper and the backtester apply the same
+list to the bar-close time as well, see section 8); transitioning-regime boost (a B/C/D can become A or B when a
+trend is emerging); exhausted-trend cap (in a trending regime with two or more exhaustion tags, a C or D is pinned
+to D and loses its setup); SELL quality gate
+(shorts need a grade in `sell_min_quality`; H1's list is empty, so every H1 short is blocked);
+multi-timeframe opposition gate; SELL volatility gate (shorts are allowed only when volatility is high or extreme; low- or
+normal-volatility shorts are blocked);
+regime blocks (M15 blocks the mixed `trending` regime; M5 blocks shorts in `transitioning`);
+SELL-hour gate (M15 blocks shorts at 15:00 and 17:00 UTC); big-candle gate (shipped off); and on
+M5 six extra noise gates (minimum score, required agreement, confirmation strength, SELL grade
+cap, chop-transition volatility, SELL RSI floor). Every gate that removes a setup writes a warning
+that names itself.
+
+**9. Trade setup.** Grades A+ and A get the gold ATR setup (`_generate_xau_atr_trade_setup`): stop
+2 × ATR; target 1 at 2 × ATR for A+ and 1.5 × ATR for A (1 × ATR for an A carrying two or more
+exhaustion tags); target 2 at 1.5 × the target-1 distance. Other tradeable grades get the generic
+swing-based setup, which floors the stop at $8 (80 pips). Both are checked against the spread gate:
+target 1 must be at least 30 × the round-trip cost ($9 at the $0.30 cost setting). Two extra entry paths run after the
+gates: the M5 pullback (`XAU_PB`, shipped off) and the chart-pattern entry (`XAU_CP`,
+`chart_patterns.py`: M5 only, buy-side patterns inverse head and shoulders, double bottom, bull
+flag). Both late paths run after the gate chain and force the grade to B, so an F-composite
+chart-pattern trade shows as a B in books and alerts. The chart-pattern path deliberately accepts
+weak composites and re-checks the session, day and big-candle rules, the multi-timeframe-opposition
+rule and the ATR floor, plus its own SELL, volatility and RSI checks; it does not re-check
+exhaustion or the M5 minimum-score and confirmation gates. That is validated behaviour, not a bug
+(section 11, 2026-08-04). A shipped setup always carries a BUY or SELL direction, even
+when the raw signal was NEUTRAL at grade C (confidence 45–50): the direction is the sign of the
+score. Every setup is stamped with its strategy name and its trailing parameters. Finally, a stale
+"costs would eat the edge" veto from an earlier generator is moved to `cleaned_warnings` when a
+later generator shipped a setup.
+
+**10. Delivery.** The `SignalReport` goes to the surface that asked for it:
+
+- *Telegram* (`telegram_bot.py`): an alert is sent only when the signal is not NEUTRAL, the grade
+  reaches `alert_min_quality`, **and** a setup exists. A fingerprint (signal + direction) stops
+  repeats; a grade upgrade re-alerts. The fingerprint is committed only after a successful send,
+  so a failed send stays eligible next scan. Every send is logged with the signal-log row id. The
+  level-breach monitor reuses the bar's cached M15 report (`_cached_or_scan`) instead of rescanning.
+- *CLI* (`main.py`): prints the report; `--json` dumps everything.
+- *Dashboard* (`dashboard_publisher.py`): writes `snapshot.json`, `news.json` and the static page
+  into the Apache folder.
+
+Two advisory add-ons are attached **after** the scan by the surface, never inside the signal
+code: the LLM agent panel (`agent_panel.py`, four DeepSeek personas, verdict AGREE / NEUTRAL /
+OPPOSE) and the violent-move watch (`violent_watch.py`, a "the next hour may be wild" line). No
+gate reads either of them.
+
+**11. Logging.** `signal_log.py` writes one row per live scan into `logs/signals.db` (signal,
+grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the
+gate that removed a setup, a gate-config fingerprint and an engine fingerprint). The engine
+fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log
+comparisons across such a deploy must be split at the deploy time (the 2026-09-09 M15 RSI change is
+one). `news_archive.py` writes every headline and
+its score into `logs/newsdata.db`. Both are fail-safe: a logging error can never break a scan.
+
+**12. Measurement, later.** Two models read the stored candles and score what happened next:
+advisory outcomes (`advisory_outcome.py`: replays each logged setup with next-open entry, stop
+first, close-only trailing; a cleaned-mid evidence model, not execution P&L) and scan outcomes
+(model-free forward moves 1 h / 4 h / 24 h after every scan). `analyze_signal_log.py` turns the
+database into reports.
+
+**The backtester is the same pipeline on history.** `backtest/gpu_backtest.py` loads candles from
+`backtest/cache/*.parquet` (fetched once through the same provider code), precomputes every
+indicator series over the whole window, and calls `analyze_from_dataframes` bar by bar with news
+off. Trades are resolved on the raw price path with next-open entries, spread charged per trade,
+stop-first ties, gap fills at the open, and the trailing stop moved at bar close only. Time filters
+use bar-close time in both paths, so live and backtest agree. The replay fans out over worker
+processes; the 5-bar entry cooldown, the only cross-bar state, is applied in an ordered pass
+afterwards, which is why parallel and sequential books are identical.
+
+---
+
+## 3. File-by-file guide
+
+Arrows show who imports whom (generated from the code on 2026-09-15).
+
+### Entry points and surfaces
+
+| File | Role | Uses | Used by |
+|---|---|---|---|
+| `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`) | pair_signals, signal_aggregator, agent_panel, violent_watch, app_config | dashboard_publisher (formatting helpers) |
+| `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
+| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` |
+| `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`) | | copied by the publisher |
+| `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher |
+| `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | |
+
+### Signal core (the part the backtester replays)
+
+| File | Role | Uses | Used by |
+|---|---|---|---|
+| `config.py` | Every tunable value: `TechnicalConfig`, `SignalConfig`, `NewsConfig`, `XauStrategyConfig`, `FilterConfig`, wrapped in `PlatformConfig`; `Timeframe`, `SignalStrength`, `PAIR_INFO` | nothing | everything |
+| `pair_signals/xauusd.py` | The config factory: overlay + validated per-timeframe overrides + confirmation map. Its docstring is the evidence log for every override | config, app_config, engine | main, telegram_bot, dashboard_publisher, backtester |
+| `pair_signals/__init__.py` | Registry of pair producers (`PAIR_PRODUCERS`) and `produce_signal` | config, signal_aggregator, providers, xauusd | the surfaces, backtester |
+| `engine.py` | Indicators, regime, composite score (`TechnicalEngine`) | config | signal_aggregator, backtester, replay_record, pair factory |
+| `signal_aggregator.py` | Multi-timeframe merge, penalties, grade, gate chain, setup generators, reasons; `analyze_pair` (live) and `analyze_from_dataframes` (replayable core) | engine, chart_patterns, news_engine, providers, signal_log, config | main, telegram_bot, backtester, pair_signals, analyze_signal_log (agent_panel only for type hints) |
+| `chart_patterns.py` | The XAU_CP pattern detector | | signal_aggregator |
+| `position_sizing.py` | Decision-time feature builder (65 columns) and the reserved stake multiplier (always 1.0). Bounded lookbacks keep live == backtest | config | backtester (`--export-features`) |
+
+### Data providers
+
+| File | Role |
+|---|---|
+| `providers/base.py` | `BaseDataProvider`: cache with candle-duration TTL, forming-candle drop, validation and cleaning, freshness check, weekend and settlement-pause calendar, live fetch depth, paginated `fetch_range` for the backtester, optional tick stream |
+| `providers/oanda.py` | Oanda v20 candles (the live feed since 2026-07-28); retry ladder 2 / 5 / 10 s under a 45 s budget |
+| `providers/twelve_data.py` | Twelve Data candles + WebSocket quotes; registered fallback, but signal-incompatible with Oanda (see section 4) |
+| `providers/tiingo.py` | Tiingo mid-price candles (research only) and the tick stream used by the breach monitor |
+| `providers/__init__.py` | `PROVIDER_REGISTRY` and `make_data_provider(config, name=None)` |
+
+### News and LLM
+
+| File | Role |
+|---|---|
+| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; optional DeepSeek enrichment; display and warnings only |
+| `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback |
+| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook |
+| `agent_panel.py` | Advisory persona panel (macro / technical / risk / contrarian + optional chair). Attached post-hoc; the alert verdict is the macro persona by default. With `verdict_source: "chair"`, `_apply_oppose_guard` downgrades a chair OPPOSE to NEUTRAL unless the macro persona opposes, or technical opposes while macro is not supportive. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
+| `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |
+
+### Logging and evaluation
+
+| File | Role |
+|---|---|
+| `signal_log.py` | SQLite logger for every live scan (`signals`, `news_snapshots`, `scan_outcomes`; view `signals_with_news`); opportunistic scan-outcome labelling; advisory outcome seeding |
+| `advisory_outcome.py` | The deterministic outcome model for logged setups |
+| `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) |
+| `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) |
+| `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) |
+
+### Backtesting and research
+
+| File | Role |
+|---|---|
+| `backtest/gpu_backtest.py` | The backtester (CuPy on a GPU, NumPy otherwise; check the `Backend:` log line) |
+| `backtest/replay_record.py` | Opt-in `--record` recorder: why every accepted trade was taken, plus calendar events. Book-neutral by contract |
+| `backtest/replay_scorecard.py` | Read-only reader of recorded runs (`DIR --check` verifies its invariants) |
+| `scripts/ff_calendar_scraper.py` | Builds `backtest/cache/calendar.parquet` (Forex Factory events) for the recorder |
+| `backtest/instructions_for_backtest.txt` | What every backtest report must record. Follow it |
+| `backtest/research/*.md` | The decision ledgers (one per campaign). Rule-1 evidence |
+| `backtest/cache/` (gitignored) | Candle caches keyed by provider, symbol, timeframe and padded window; only the Oanda windows listed in section 9 are kept |
+
+### Configuration and deployment
+
+| File | Role |
+|---|---|
+| `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
+| `config.json.example` | The schema of `config.json` (which is gitignored) |
+| `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
+| `test_all.py` | The QA suite (33 modules) |
+
+---
+
+## 4. Data providers
+
+All candle data goes through `providers/base.py`. Whatever the source, the pipeline sees the same
+thing: a UTC, oldest-first DataFrame with capitalised OHLCV columns, closed bars only. The base
+class owns the behaviour the signals depend on; a provider only implements `_fetch_raw` plus its
+symbol and interval maps. A test enforces that subclasses do not override the shared methods
+(`supports_streaming` and `iter_ticks` are the sanctioned streaming hooks).
+
+**Which provider does what today**
+
+| Provider | Live candles | Tick stream | Notes |
+|---|---|---|---|
+| Oanda v20 (`data_providers.active: "oanda"`) | yes, since 2026-07-28 | no | Needs `oanda.api_token` and `oanda.base_url`; no account id. No client-side throttle. A legacy `oanda.account_id` key is tolerated and ignored |
+| Tiingo (`data_providers.streaming: "tiingo"`) | research only | yes (`wss://api.tiingo.com/fx`, mid-price) | Never show a spread from its free quote feed (two-tier indicative quotes) |
+| Twelve Data | registered fallback | yes | About 19% of M5 bars disagree in direction with Oanda and highs sit about $1 too high. On an Oanda outage, stop scanning rather than switch feeds |
+
+**Behaviour that matters for signals**
+
+- The forming candle is always dropped. Live signals would repaint otherwise.
+- Cleaning is a bad-print filter, not a volatility filter: a bar more than 10σ from the previous
+  close is removed only when the next close snaps back at least 80% of the way. Real news bars
+  stay, and the newest bar is always kept. (`OUTLIER_SIGMA`, `OUTLIER_REVERT_FRACTION`.)
+- Every live fetch asks for at least 1,000 bars (`MIN_LIVE_BARS`): M5 and M15 5,000, H1 1,460,
+  H4 and D1 1,000. The 200-bar EMA needs that much history to match the backtester.
+- Weekend: New-York anchored, Friday 17:00 to Sunday 18:00 New York time. There are three
+  independent copies of this rule on the signal path (`providers/base.py`,
+  `signal_aggregator.py`, and `gpu_backtest._is_xau_blocked_by_calendar`) plus one in
+  `advisory_outcome.py`. Patch all of them, or none; a Module 21 test sweeps all four hour by hour
+  across both daylight-saving transitions.
+- The daily settlement pause (Monday to Thursday 17:00 New York, 21:00 UTC in summer) counts as a
+  scheduled closure since 2026-09-13. A "stale data" warning at any other weekday hour is a real
+  feed stall.
+- The backtester fetches history through `fetch_range` (paginated, retries a failed page, logs an
+  error for interior gaps longer than 3 bars) and stores it under `backtest/cache/`.
+
+---
+
+## 5. Configuration and credentials
+
+Rule for every credential and deployment setting: **environment variable → `config.json` → refuse
+to run.** There are no hard-coded fallbacks. A gitignored `.env` at the repo root is loaded at
+import time and behaves like real environment variables.
+
+`config.json` holds secrets and deployment settings only. Signal logic, weights, thresholds and
+gates live in `config.py` and the pair factory, never in `config.json`. Blocks (see
+`config.json.example` for the exact shape):
+
+| Block | What it controls |
+|---|---|
+| `telegram` | `token`, `chat_id`, `channel_id`, `alert_min_quality` (env `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`) |
+| `data_providers` | `active` (live candle feed), `streaming` (tick feed), per-provider API keys (env `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`) |
+| `oanda` | `api_token`, `base_url` (env `OANDA_API_TOKEN`, `OANDA_BASE_URL`) |
+| `scanning` | `pairs`, `timeframes` |
+| `pairs."XAU/USD".blocked_hours_utc` | The base list of blocked hours (default 0, 1, 2, 10, 11, 13, 16, 21) |
+| `deepseek` | `api_key`, `model`, `enabled`, `timeout_seconds`, `daily_request_cap` (env `DEEPSEEK_API_KEY`) |
+| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`macro` or `chair`). All defaults = feature off |
+| `violent_watch` | `enabled`, `alert_line`, `headsup`, `max_headsup_per_day`, `headsup_min_level` (env `VIOLENT_WATCH_ENABLED`). Off by default |
+| `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
+| `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
+| `log_level` | Logging level |
+
+A legacy top-level `market_outlook` block is tolerated and ignored with one warning (the feature
+was removed 2026-09-15). `FB2_EMA_LO` is a research-only environment knob for the EMA200 proximity
+band (default 0.5; an unparseable value falls back to the default with a warning).
+
+Files the system writes: `logs/signals.db`, `logs/newsdata.db`, `logs/telegram.log` (one file per
+day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `immutable=1`
+(that mode cannot see rows still in the write-ahead log).
+
+---
+
+## 6. Testing
+
+- `venv/bin/python test_all.py` runs everything with mocked network. Expect
+  `RESULTS: 506 passed, 0 failed, 506 total` (2026-09-15).
+- Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
+- When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the
+  other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar.
+  `_candles_json` stores the last 14 bars, so a longer fixture silently loses its head.
+- A test walks the code with `ast` and forbids `sklearn`, `scipy`, `lightgbm` and `torch` in live
+  modules and in `requirements.txt`. Research happens in the dev `venv/`; anything shipped is
+  numpy-only.
+- Behaviour-neutral changes are proven by regenerating the reference books (section 9) and
+  showing the MD5s unchanged.
+
+---
+
+## 7. Rules for changing signal logic
+
+1. **Never change a tuned value without re-validation.** Acceptance: positive total R in both
+   halves of a window of at least 12 months, profit factor ≥ 1.3, at realistic cost ($0.65 round
+   trip headline; also report $0.30 for comparability with older books), trailing exits, next-open
+   fills. Marginal edges must also pass the 2024-06-01 → 2025-03-30 out-of-sample window. Backtest
+   windows go back at most 3 years (owner preference, 2026-09-05). `round_trip_cost` stays at 0.30
+   because it also drives the live target floor (0.65 would move target 1's floor from about $9 to
+   $19.50); changing it is its own rule-1 event.
+2. **Per-timeframe overrides live in the pair factory**, never in the global defaults. The same
+   knob has had opposite effects on different timeframes several times. Record the evidence in the
+   factory docstring.
+3. **Pair isolation.** Gold-specific gates are keyed on the pair's display name so a future pair
+   cannot inherit gold-calibrated logic by accident.
+4. **Live equals backtest.** A new live gate goes into the replay path in the same commit. All time
+   filters use bar-close time in both paths.
+5. **ROC stays weighted** even though it correlates with MACD, Keltner and EMA. Removing it
+   backtested 13R worse; the thresholds were calibrated with it.
+6. **The backtester's realism settings are results-critical** (raw-path resolution, spread
+   charging, next-open entry, stop-first ties, close-only trailing). Touching them breaks
+   comparability with every recorded book.
+7. **The forming candle stays dropped.**
+8. Do not add config fields nothing reads. Do not disable indicators in production paths
+   (`disabled_indicators` is for experiments only; a 0.0 regime multiplier in a per-timeframe
+   `regime_weights_override` is equivalent for the composite and is the sanctioned vehicle for an
+   indicator-off candidate).
+9. EUR/USD was removed on 2026-07-16 after a no-edge verdict (last commit with it `7d3fd95`).
+   BTC/ETH work exists only on unmerged branches. Do not re-add either without new data.
+
+Process rules that go with them: every backtest report follows
+`backtest/instructions_for_backtest.txt`; every candidate is compared with a control run on the
+same code and cache, never with an old book directly; the `Backend:` line and the "GPU cache
+fallbacks: 0" line are part of the evidence; keep `docs/` in sync when behaviour changes.
+
+---
+
+## 8. Validated configuration (what is actually running)
+
+Defaults in `config.py` are the tuned gold configuration. The factory adds these per-timeframe
+overrides (each one has its validation record in the `pair_signals/xauusd.py` docstring):
+
+| Timeframe | Confirmations | Overrides on top of the defaults |
+|---|---|---|
+| M5 | M15, H1 | hour 9 UTC blocked and hour 21 unblocked; chart-pattern entries on (buy-side only: inverse head and shoulders, double bottom, bull flag); pullback entries off; partial EMA alignment scores neutral; trailing stop arms at 0.60R and trails 0.85R; shorts blocked in the `transitioning` regime |
+| M15 | M5, H1 | target floor 1R; the mixed `trending` regime blocked; shorts blocked at 15:00 and 17:00 UTC; trailing stop arms at 0.70R and trails 0.75R; RSI weight halved in every regime |
+| H1 | M15, H4 | every short blocked (`sell_min_quality` empty) |
+| H4 | H1, D1 | none. H4 remains "unproven" (few trades) |
+
+Shared defaults worth knowing: blocked hours 0, 1, 2, 10, 11, 13, 16, 21 UTC; trading days Monday
+to Friday plus Sunday evening; shorts need grade A+, A or B and high-or-extreme volatility;
+trailing stop arms at 0.6R and trails 0.75R; target floor 0.5R; round-trip cost $0.30 with
+target 1 at least 30 × cost; the generic and chart-pattern setups floor the stop at $8; the gold
+ATR setup uses a 1 × ATR target before the grade scale (2× at A+, 1.5× at A) and a 2 × ATR stop; the H4 regime filter is off; news never gates.
+
+One semantic to know: `blocked_hours_utc` is applied to the bar **label** hour inside the core
+and to the bar **close** hour in the live wrapper and the backtester. So H1 can only trade bars
+closing 04–09, 15 and 18–20 UTC and H4 only bars closing 08:00 or 12:00. Both paths agree, and
+every book was validated that way. Do not "fix" one side alone.
+
+---
+
+## 9. Reference books and how to reproduce them
+
+Pinned trade books are no longer kept in the tree (owner decision 2026-09-15; the old ones sit in
+`~/Development/ForexBot2_trash_2026-09-15/` until the owner deletes that folder). The gate is the MD5 of
+the exported 63-column CSV (`md5sum` on the whole file). Regenerate a book from the cache when a
+trade-level diff is needed; each run takes seconds to a few minutes.
+
+Harness for every reference book: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD
+--spread 0.65 --workers 12 --primary-tf <tf> --start <start> --end <end> --export-csv <out>`,
+factory config, no `--set`, `--quality`, `--strategies`, `--confirm-tfs` or `--export-features`.
+All of these run offline from the kept cache windows (NumPy CPU backend, 0 cache fallbacks).
+
+| Book | Window | Trades / R / PF | MD5 of the CSV |
+|---|---|---|---|
+| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `29120a1ee32979431d062f210d41a9d1` |
+| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `d185f099482a07f29502cb5a371a9bbf` |
+| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `6f8f24b49597ced28f50a19e08c3472f` |
+| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `3b45d12bade823178b267c13bc698458` |
+| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `e75adae13c839b15736f69c0114e6aa5` |
+
+Other current-HEAD figures at $0.65: M5 W3 (3-year) 870 / +153.6R / PF 1.50; M15 W3 1273 /
++186.6R / PF 1.39; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54;
+H1 12-month (2025-08-04 → 2026-08-04) 91 / +22.2R / PF 1.79. All five MD5s above were
+reproduced on 2026-09-15 after the cache prune.
+
+Kept cache windows (`backtest/cache/oanda_XAU_USD_<tf>_<paddedStart>_<end>.parquet`; the padded
+start is the window start minus 62 days for M5/M15/H1, 234 for H4 and 1,400 for D1, so the D1
+and H4 files of one run have different start dates than its M5 file): 3-year `20230705_20260905`,
+2-year `20240705_20260905`, OOS `20240331_20250330`, M5 12-month `20250613_20260814`, M15/H1
+12-month `20250603_20260804`, 5-year `20210531_20260804`, 7.6-year `20181031_20260807`, the
+2026-08-14 parity window `20250814_20260814`, their D1/H4 pads, and `calendar.parquet`.
+
+The research recorder (`--record DIR [--record-bars]`) writes Parquet files describing every
+accepted trade (indicator votes on all three timeframes, penalties, setup, outcome, calendar
+events). It never changes the book; pair every recorded run with a control run and show equal
+MD5s. Scorecards from it are in-book statistics, never decisions.
+
+---
+
+## 10. Things that bite
+
+- Never build a bare `PlatformConfig()` for a real scan; use the factory. A test enforces factory
+  equivalence with the inline construction the backtester uses.
+- The backtester names timeframes `5min`, `15min`, `1h`, `4h`. `M5` is a hard exit.
+- `--spread` defaults to 0.30. Pass 0.65 for every headline result.
+- `--export-features` changes the CSV; never use it on a parity run.
+- The signal log's `spread` column is the median bar range, not a broker spread. `actionable`
+  means grade ≥ C and a setup exists; the `action` text is written before the gate chain and can
+  disagree with the final grade. `scan_kind` is `blocked` (weekend or day filter), `failed` (a
+  required frame was missing) or normal; readers skip the first two. Several surfaces log the
+  same bar, so readers dedup by the first row per (`timeframe`, `bar_ts`). Same-setup re-logs
+  within 60 minutes are `DUPLICATE` rows; sum advisory R over `RESOLVED` rows only.
+- Headlines and events in the signal log are content-addressed into `news_snapshots`; read
+  through the `signals_with_news` view or `signal_log.read_news`.
+- A non-empty `cleaned_warnings_json` is not a chart-pattern marker (a stale spread-gate veto lands
+  there too since 2026-09-08); use `setup_json.strategy`.
+- The `--stored-candles` scorecard in `analyze_signal_log.py` still advances outcomes without the
+  locked-profit floor step that the logger and the backfill script use (owner decision pending), so
+  the two measurement paths can disagree on gap rows.
+- There is no holiday awareness anywhere. On Labor Day 2026 Oanda published no bars for hours, the
+  bot rescanned the stale bar 43 times, and the violent-move watch called the 12–14 UTC slot hot.
+  Interior gaps in history are logged, never fatal, for the same reason.
+- Existing databases still contain an empty-going-forward `outlooks` table from the removed
+  market-outlook feature. Nothing reads it.
+- The news score is a trailing echo of the last day's price move (r ≈ 0.67 with the past day,
+  ≈ 0.03 with the next). Both news gating flags stay off.
+- The agent panel never runs inside `analyze_from_dataframes`, and its cached verdict is never
+  reused when the evidence changed; the daily cap is the real spend bound.
+- Live == backtest also means the backtester never writes to `signals.db` and never builds a
+  `NewsEngine`.
+- Twelve Data candles are not a drop-in replacement for Oanda (section 4).
+- `pyarrow` (Parquet) is installed in the dev venv only; live code never touches Parquet.
+- The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped
+  every H1 and H4 scan once (2026-08-02).
+
+---
+
+## 11. Decision ledger
+
+One line per validated change or verdict, newest first. Full evidence in the named report.
+
+| Date | Decision | Where the evidence is |
+|---|---|---|
+| 2026-09-15 | Backtest folder cleanup: `diagnose_filters.py`, `shadow_scan.py`, and the two one-time news scripts removed (last commit with them `931b265`); every pinned CSV, the research output folders, the BTC/ETH cache snapshots, the Tiingo caches and 13 uncited Oanda windows moved to `~/Development/ForexBot2_trash_2026-09-15/` (346 MB) pending the owner's delete; all five reference MD5s reproduced afterwards. Suite 506 / 33 | commit `b0b71e1` |
+| 2026-09-15 | AI market-outlook feature removed (module, hourly Telegram brief, `--outlook`, `outlooks` writer, config block). Legacy config block tolerated with a warning. Books byte-identical | commit `931b265` |
+| 2026-09-13 | Bug audit, 11 delivery/measurement fixes, books byte-identical: NEUTRAL-signal C/B setups now alert (about 17% of the H1 12-month book's R, +3.7R of +22.2R, was never sent); a missing confirmation frame fails the scan; pandas `utcnow` replaced and `pandas<4`, `numpy<3` pinned; advisory outcomes keep locked-profit floors (`GAP_LOCKED`); scan-outcome labeler bounded to 72 h; settlement pause is a scheduled closure; `fetch_range` retries pages and audits gaps; cache-fallback count logged and must be 0; partial `regime_weights_override` raises; the bot scans each bar once and attaches its log handler in `main()` only; the dashboard catches up missed timeframes. Verified sound and unchanged: composite maths, scan-outcome labels, outcome exits, Oanda alignment, no look-ahead | plan file `~/.claude/plans/you-are-python-software-joyful-teacup.md`, commit `2836024` |
+| 2026-09-10 | Improvement cycle 1: 3-year baseline re-established; cost ladder shows no red flag (PF ≥ 1.3 everywhere at $0.80); the intraday edge is a fat right tail (top 5% of trades carry 94–96% of M5/M15 R). H1 divergence-to-zero REJECTED. M15 RSI × 0.5 sits on a stable plateau. SHIPPED: live fetches floor at 1,000 bars and the backtester pads history to match (books trade-identical). Parked cycle-2 candidate: block M15 grade-A SELLs (230 trades, −14.4R, negative in every year and OOS) | `backtest/research/cycle_2026-09-10.md` |
+| 2026-09-09 | Weight test on the recorded replays: M15 RSI multiplier × 0.5 in every regime PASSED (2-year +117.6R → +146.1R, OOS +40.6 → +60.7R) and SHIPPED as an M15-only override; all M5 MACD candidates FAILED; M15 EMA × 2 and divergence-off FAILED | `backtest/research/weight_test_2026-09-09.md` |
+| 2026-09-09 | Replay recorder shipped (`--record`, `--record-bars`), books byte-identical. First look is descriptive only: Bollinger opposes 94% of M5 entries; divergence-opposed M5 trades did best; bar-level baseline drifts +1.9 ATR/24 h on M5, so adjust for drift before judging BUY votes | `backtest/research/replay_recording_2026-09-09.md` |
+| 2026-09-08 | Delivery fixes: stale spread-gate vetoes moved to `cleaned_warnings` (books identical; only the warning text on 20–33% of A-grade alerts changed); alerts require a setup; every send logged with its row id; `telegram.log` rotates daily × 30 | commit history, 2026-09-08 |
+| 2026-09-08 | News archive shipped (`logs/newsdata.db`). Do not train a local headline scorer before about 4,000 headlines over 3+ weeks, time-split, headline text only, graded against the golden set (`git show 1cebebf^:backtest/research/llm_news_golden.csv`) | `news_archive.py` docstring, `backtest/research/llm_news_eval_2026-08-06.md` |
+| 2026-09-07 | Signal-side outlier filter fixed (it deleted about 36 real M5 news bars a year in the backtester and a different 24 live). Book impact at $0.65, 12 months: M5 +112.2R → +108.8R, M15 +73.9R → +74.6R, H1 unchanged. Oanda retry ladder added; a scan that loses its primary frame is `failed` | CLAUDE.md history, commit 2026-09-07 |
+| 2026-09-06 | Stake-sizing campaign FAILED its pre-registered rule on M5 (clean null) and M15 (lift inside the shuffle null). Nothing changing numbers shipped; `--export-features` and the penalty tags remain. M5 stretch measures are positive in both halves (do not propose exhaustion sizing on M5); bursts cluster 12–15 UTC and 01 UTC and do not hurt entries; every open-trade burst protection lost money | `backtest/research/sizing_campaign_2026-09.md` |
+| 2026-09-06 | Violent-move watch shipped dark (display only). Hot = high-impact USD release within ±30 min, or a hot clock slot plus a price extreme; flags 8.6% of bars at 3–5× odds. Context readings never raise the level | `violent_watch.py` docstring, sizing campaign addenda |
+| 2026-09-06 | Stale-code cleanup: research tooling, Dukascopy study, diagrams and plans deleted (history before `953c5ad` has them); `main.py` M5 confirmations fixed to M15 + H1; dead functions removed | commits of 2026-09-06 |
+| 2026-09-04 | News sentiment verdict: the score trails price, it does not lead it. Headline window 4 h, neutral headlines dropped, beginner-language summary shipped; gating flags stay off | `backtest/research/news_sentiment_value_2026-09-04.md` |
+| 2026-09-03 | Signal-log campaign shipped two gates: M5 shorts blocked in `transitioning` (+7.9R/12 mo) and every H1 short blocked (5-year +16.7R, OOS +11.5R). Rejected: M15 SELL-B block, Bollinger-off. Parked: divergence-off. Panel verdict switched to the macro persona (owner decision); the chair's OPPOSE was anti-informative (opposed trades made +30R) | `backtest/research/signal_log_campaign_2026-09.md`, `signal_log_eval_2026-08-18.md` |
+| 2026-09-03 | Auto-execution (Oanda daemon and executor) removed; the owner does not auto-trade (last commit with it `31002cb`) | commit `31002cb` |
+| 2026-08-15 | Agent-panel technical persona gets ATR-normalised candles and raw indicator values | `backtest/research/ai_signal_enrichment_2026-08-15.md` |
+| 2026-08-14 | Books re-pinned on Oanda bars; Twelve Data books superseded. Feed divergence and Oanda-Tiingo signal parity measured | `repin_verification_2026-08-14.md`, `feed_divergence_2026-08-14.md`, `signal_parity_2026-08-14.md` |
+| 2026-08-06 | DeepSeek headline scoring validated (78/78 vs 24/78 for keywords); event misreads fixed with the `vs_forecast` field | `backtest/research/llm_news_eval_2026-08-06.md` |
+| 2026-08-05 | Fixed-RR hard target vs trailing exit: NOT shipped (passes the letter of rule 1 on 2024+ windows but is a trend artefact; M15 negative in 2021 and 2023, drawdown doubles). `--fixed-rr-tp` stays as a research flag | `backtest/research/fixedtp_2026-08-05.md` |
+| 2026-08-04 | QA audit. Weekend calendar made New-York anchored (removed one live-impossible phantom trade). Spread inside exit-trigger detection: owner DECLINED (4–6% optimism accepted, books stay comparable). `FB2_EMA_LO` hardened. M5 pullback entries disabled (PF 1.26 → 1.44 at $0.65). Partial EMA alignment set neutral on M5. XAU_CP gate bypass documented as validated behaviour (226 of 586 M5 trades, +38.9R) | memory files `qa-audit-2026-08-04`, `m5-tuning-2026-08-04`, `m5-round2-2026-08-04` |
+| 2026-07-30 | M5 relaxation shipped (score 25, confidence 5, hour 21 unblocked); big-candle gate rejected (ships off) | memory `relaxation-campaign-2026-07-30` |
+| 2026-07-28 | Oanda v20 becomes the live feed | commit `d2d4fb4` |
+| 2026-07-24 | Real gold spread measured at about $0.65 round trip (9M Dukascopy ticks, 12 months); $0.30 kept only for cross-book comparability and the live target floor | memory `dukascopy-spread-study` |
+| 2026-07-21 | M15 trailing arm 0.70 and M5 trailing distance 0.85 adopted | memory `m15-trail-grid-2026-07-21` |
+| 2026-07-16 | M15 SELL-hour block (15 and 17 UTC) adopted; EUR/USD removed (no edge) | memory `xau-m15-loss-analysis`, `eur-no-edge-verdict` |
+| 2026-07-11 | M15: mixed `trending` regime blocked, target floor 1R; M5: hour 9 blocked | pair factory docstring |
+| 2026-07-08 | Backtester realism fixes (spread charging, gap fills, next-open entry, stop-first ties) | memory `backtester-realism-fixes` |
+
+Older memory notes live in `~/.claude/projects/-home-harwinder-Development-ForexBot2/memory/`.
+The previous, longer version of this file is in git history before 2026-09-15.

advisory_outcome.py

@@ -0,0 +1,387 @@
+"""Deterministic advisory trade outcomes from stored closed candles.
+
+This module measures the hypothetical setup reviewed by the LLM panel. It
+never changes a signal, panel verdict, order, or daemon state. The model uses
+the same next-open reanchoring and close-only trailing rules as the backtester,
+but its evidence is the cleaned mid-candle snapshots stored in signals.db.
+"""
+
+from __future__ import annotations
+
+import logging
+import math
+from datetime import datetime, timedelta
+from typing import Any, Dict, Iterable, List, Optional, Tuple
+from zoneinfo import ZoneInfo
+
+logger = logging.getLogger(__name__)
+
+MODEL = "stored-closed-mid-next-open-trail-v1"
+MAX_HOLD_BARS = 300
+REFERENCE_COST = 0.30
+REALISTIC_COST = 0.65
+
+_TF_ALIASES = {
+    "M1": "1min", "1MIN": "1min", "1min": "1min",
+    "M5": "5min", "5MIN": "5min", "5min": "5min",
+    "M15": "15min", "15MIN": "15min", "15min": "15min",
+    "M30": "30min", "30MIN": "30min", "30min": "30min",
+    "H1": "1h", "1H": "1h", "1h": "1h",
+    "H4": "4h", "4H": "4h", "4h": "4h",
+    "D1": "1day", "1D": "1day", "1d": "1day",
+    "1DAY": "1day", "1day": "1day",
+}
+
+_TF_SECONDS = {
+    "1min": 60,
+    "5min": 5 * 60,
+    "15min": 15 * 60,
+    "30min": 30 * 60,
+    "1h": 60 * 60,
+    "4h": 4 * 60 * 60,
+    "1day": 24 * 60 * 60,
+}
+
+_NY_TZ = ZoneInfo("America/New_York")
+
+
+def _is_forex_weekend(now: datetime) -> bool:
+    """NY-anchored XAU weekend closure; test-locked to the signal copies."""
+    ny = now.astimezone(_NY_TZ)
+    weekday = ny.weekday()
+    if weekday == 4:
+        return ny.hour >= 17
+    if weekday == 5:
+        return True
+    if weekday == 6:
+        return ny.hour < 18
+    return False
+
+
+def normalize_timeframe(value: Any) -> str:
+    text = str(getattr(value, "value", value) or "")
+    return _TF_ALIASES.get(text, _TF_ALIASES.get(text.upper(), text))
+
+
+def timeframe_value(mapping: Any, timeframe: Any) -> Any:
+    """Read a timeframe-keyed snapshot regardless of enum/storage alias."""
+    if not isinstance(mapping, dict):
+        return None
+    wanted = normalize_timeframe(timeframe)
+    for key, value in mapping.items():
+        if normalize_timeframe(key) == wanted:
+            return value
+    return None
+
+
+def _finite(value: Any) -> Optional[float]:
+    try:
+        number = float(value)
+    except (TypeError, ValueError):
+        return None
+    return number if math.isfinite(number) else None
+
+
+def _timestamp(value: Any) -> Optional[datetime]:
+    try:
+        return datetime.fromisoformat(str(value).replace(" ", "T"))
+    except (TypeError, ValueError):
+        return None
+
+
+def initial_outcome(
+    setup: Optional[Dict[str, Any]],
+    indicators: Optional[Dict[str, Any]],
+    timeframe: Any,
+) -> Tuple[str, Dict[str, Any]]:
+    """Freeze the setup and deterministic evaluation policy."""
+    if not setup:
+        return "NOT_APPLICABLE", {
+            "model": MODEL,
+            "status": "NOT_APPLICABLE",
+            "reason": "panel row has no trade setup",
+        }
+
+    tf = normalize_timeframe(timeframe)
+    analysis = timeframe_value(indicators, tf)
+    signal_bar = analysis.get("timestamp") if isinstance(analysis, dict) else None
+    direction = str(setup.get("direction", ""))
+    entry = _finite(setup.get("entry"))
+    stop = _finite(setup.get("stop_loss"))
+    tp1 = _finite(setup.get("take_profit_1"))
+    arm = _finite(setup.get("trail_arm_r"))
+    distance = _finite(setup.get("trail_dist_r"))
+    risk = abs(entry - stop) if entry is not None and stop is not None else 0.0
+    if (direction not in ("BUY", "SELL") or not signal_bar
+            or _timestamp(signal_bar) is None or risk <= 0 or tp1 is None
+            or arm is None or distance is None or arm < 0 or distance < 0):
+        return "UNRESOLVABLE", {
+            "model": MODEL,
+            "status": "UNRESOLVABLE",
+            "reason": "invalid or incomplete setup/outcome seed",
+        }
+
+    state = {
+        "model": MODEL,
+        "status": "PENDING",
+        "timeframe": tf,
+        "signal_bar_at": str(signal_bar),
+        "setup": {
+            "direction": direction,
+            "reported_entry": entry,
+            "stop_distance": risk,
+            "tp1_distance": abs(tp1 - entry),
+            "strategy": setup.get("strategy"),
+            "setup_pattern": setup.get("setup_pattern"),
+        },
+        "policy": {
+            "entry": "next_primary_bar_open_reanchored",
+            "max_hold_bars": MAX_HOLD_BARS,
+            "trail_arm_r": arm,
+            "trail_dist_r": distance,
+            "tp1_is_advisory": arm > 0,
+            "reference_cost": REFERENCE_COST,
+            "realistic_cost": REALISTIC_COST,
+        },
+        "progress": {
+            "last_processed_bar": str(signal_bar),
+            "entry_at": None,
+            "entry_price": None,
+            "initial_stop": None,
+            "current_stop": None,
+            "peak": None,
+            "bars_held": 0,
+            "mfe_r": 0.0,
+            "mae_r": 0.0,
+        },
+        "result": None,
+    }
+    return "PENDING", state
+
+
+def _valid_bars(bars: Iterable[Dict[str, Any]]) -> List[Dict[str, Any]]:
+    valid = []
+    for bar in bars:
+        stamp = _timestamp(bar.get("timestamp")) if isinstance(bar, dict) else None
+        values = [_finite(bar.get(key)) for key in ("open", "high", "low", "close")]
+        if stamp is None or any(value is None for value in values):
+            continue
+        valid.append({
+            "timestamp": str(bar["timestamp"]),
+            "_timestamp": stamp,
+            "open": values[0], "high": values[1],
+            "low": values[2], "close": values[3],
+        })
+    valid.sort(key=lambda item: item["_timestamp"])
+    return valid
+
+
+def _has_continuity(
+    bars: List[Dict[str, Any]], last: datetime, timeframe: str
+) -> bool:
+    """Reject missing tradable bars while allowing scheduled XAU closures."""
+    seconds = _TF_SECONDS.get(normalize_timeframe(timeframe))
+    if seconds is None:
+        return False
+    step = timedelta(seconds=seconds)
+    previous = last
+    seen = {last}
+    for bar in bars:
+        stamp = bar["_timestamp"]
+        if stamp <= previous or stamp in seen:
+            continue
+        expected = previous + step
+        while expected < stamp:
+            ny = expected.astimezone(_NY_TZ)
+            daily_maintenance = ny.weekday() < 4 and ny.hour == 17
+            if not (_is_forex_weekend(expected) or daily_maintenance):
+                return False
+            expected += step
+        if expected != stamp:
+            return False
+        previous = stamp
+        seen.add(stamp)
+    return True
+
+
+def _finish(
+    state: Dict[str, Any], reason: str, exit_at: str, exit_price: float
+) -> None:
+    setup = state["setup"]
+    progress = state["progress"]
+    entry = float(progress["entry_price"])
+    risk = float(setup["stop_distance"])
+    pnl = ((exit_price - entry) if setup["direction"] == "BUY"
+           else (entry - exit_price))
+    gross_r = pnl / risk
+    state["status"] = "RESOLVED"
+    state["result"] = {
+        "exit_at": exit_at,
+        "exit_price": exit_price,
+        "exit_reason": reason,
+        "bars_held": progress["bars_held"],
+        "gross_r": gross_r,
+        "net_r_030": (pnl - REFERENCE_COST) / risk,
+        "net_r_065": (pnl - REALISTIC_COST) / risk,
+        "mfe_r": progress["mfe_r"],
+        "mae_r": progress["mae_r"],
+    }
+
+
+def advance_outcome(
+    state: Dict[str, Any], bars: Iterable[Dict[str, Any]]
+) -> Dict[str, Any]:
+    """Advance one pending outcome with overlapping closed-bar snapshots."""
+    if state.get("status") != "PENDING":
+        return state
+    try:
+        progress = state["progress"]
+        setup = state["setup"]
+        policy = state["policy"]
+        last = _timestamp(progress["last_processed_bar"])
+        signal_bar = _timestamp(state["signal_bar_at"])
+        if last is None or signal_bar is None:
+            raise ValueError("invalid progress timestamp")
+
+        buy = setup["direction"] == "BUY"
+        risk = float(setup["stop_distance"])
+        trail_arm = float(policy["trail_arm_r"])
+        trail_distance = float(policy["trail_dist_r"])
+
+        valid_bars = _valid_bars(bars)
+        has_future = any(bar["_timestamp"] > last for bar in valid_bars)
+        has_overlap = any(bar["_timestamp"] == last for bar in valid_bars)
+        if has_future and not has_overlap:
+            state["status"] = "UNRESOLVABLE"
+            state["result"] = None
+            state["reason"] = "stored candle snapshots lost continuity"
+            return state
+        if has_future and not _has_continuity(valid_bars, last,
+                                              state["timeframe"]):
+            state["status"] = "UNRESOLVABLE"
+            state["result"] = None
+            state["reason"] = "stored candle snapshots contain an internal gap"
+            return state
+
+        for bar in valid_bars:
+            stamp = bar["_timestamp"]
+            if stamp <= signal_bar or stamp <= last:
+                continue
+
+            if progress["entry_price"] is None:
+                entry = bar["open"]
+                initial_stop = entry - risk if buy else entry + risk
+                progress.update({
+                    "entry_at": bar["timestamp"],
+                    "entry_price": entry,
+                    "initial_stop": initial_stop,
+                    "current_stop": initial_stop,
+                    "peak": entry,
+                })
+
+            entry = float(progress["entry_price"])
+            current_stop = float(progress["current_stop"])
+            initial_stop = float(progress["initial_stop"])
+            progress["bars_held"] += 1
+            progress["last_processed_bar"] = bar["timestamp"]
+            last = stamp
+
+            if buy:
+                mfe = (bar["high"] - entry) / risk
+                mae = (entry - bar["low"]) / risk
+                stop_hit = bar["low"] <= current_stop
+            else:
+                mfe = (entry - bar["low"]) / risk
+                mae = (bar["high"] - entry) / risk
+                stop_hit = bar["high"] >= current_stop
+            progress["mfe_r"] = max(float(progress["mfe_r"]), max(0.0, mfe))
+            progress["mae_r"] = max(float(progress["mae_r"]), max(0.0, mae))
+
+            if stop_hit:
+                exit_price = current_stop
+                if (buy and bar["open"] < current_stop) or \
+                        (not buy and bar["open"] > current_stop):
+                    exit_price = bar["open"]
+                reason = "SL" if current_stop == initial_stop else "TRAIL"
+                _finish(state, reason, bar["timestamp"], exit_price)
+                return state
+
+            if trail_arm <= 0:
+                target = entry + setup["tp1_distance"] if buy \
+                    else entry - setup["tp1_distance"]
+                target_hit = bar["high"] >= target if buy else bar["low"] <= target
+                if target_hit:
+                    exit_price = target
+                    if (buy and bar["open"] > target) or \
+                            (not buy and bar["open"] < target):
+                        exit_price = bar["open"]
+                    _finish(state, "TP1", bar["timestamp"], exit_price)
+                    return state
+            else:
+                favorable = bar["high"] if buy else bar["low"]
+                peak = max(float(progress["peak"]), favorable) if buy \
+                    else min(float(progress["peak"]), favorable)
+                progress["peak"] = peak
+                excursion = ((peak - entry) if buy else (entry - peak)) / risk
+                if excursion >= trail_arm:
+                    level = peak - trail_distance * risk if buy \
+                        else peak + trail_distance * risk
+                    progress["current_stop"] = max(current_stop, level) if buy \
+                        else min(current_stop, level)
+
+            if progress["bars_held"] >= int(policy["max_hold_bars"]):
+                _finish(state, "TIMEOUT", bar["timestamp"], bar["close"])
+                return state
+        return state
+    except Exception as exc:
+        # Still fail-safe (never raises into a scan), but no longer
+        # SILENT: a real defect here used to be indistinguishable from a
+        # legitimate weekend/downtime gap, discoverable only by grepping
+        # advisory_outcome_json. Callers persist the state either way.
+        logger.exception(
+            "advisory outcome evaluator failed for the %s outcome seeded "
+            "at %s — recorded UNRESOLVABLE",
+            state.get("timeframe"), state.get("signal_bar_at"))
+        state["status"] = "UNRESOLVABLE"
+        state["result"] = None
+        state["reason"] = f"outcome evaluator error: {type(exc).__name__}"
+        return state
+
+
+def resolve_locked_floor(state: Dict[str, Any]) -> Dict[str, Any]:
+    """If an UNRESOLVABLE gap outcome had already trailed its stop past the
+    entry, finish it at that stop as a FLOOR (the true outcome is at least
+    this good) instead of discarding it. Returns the state (mutated)."""
+    if state.get("status") != "UNRESOLVABLE":
+        return state
+    reason = str(state.get("reason", ""))
+    # Both gap reasons ("lost continuity" / "contain an internal gap")
+    # share this prefix; an invalid seed or evaluator error never floors.
+    if not reason.startswith("stored candle snapshots"):
+        return state
+    progress = state.get("progress")
+    setup = state.get("setup")
+    if not isinstance(progress, dict) or not isinstance(setup, dict):
+        return state
+    entry = _finite(progress.get("entry_price"))
+    stop = _finite(progress.get("current_stop"))
+    risk = _finite(setup.get("stop_distance"))
+    exit_at = progress.get("last_processed_bar")
+    if (entry is None or stop is None or risk is None or risk <= 0
+            or not exit_at):
+        return state
+    buy = setup.get("direction") == "BUY"
+    locked = stop >= entry if buy else stop <= entry
+    if not locked:
+        return state
+    _finish(state, "GAP_LOCKED", str(exit_at), stop)
+    state["result"]["floor"] = True
+    state["reason"] = (
+        f"{reason} — resolved at the locked trailing stop (floor)")
+    return state
+
+
+def canonical_r(state: Dict[str, Any]) -> Optional[float]:
+    result = state.get("result")
+    value = result.get("net_r_065") if isinstance(result, dict) else None
+    return _finite(value)

app_config.py

@@ -0,0 +1,579 @@
+"""
+app_config.py — deployment-settings overlay (config.json)
+=========================================================
+
+Loads the gitignored config.json at the repo root ONCE per process.
+Deployment surface ONLY: Telegram credentials, data-provider selection
+and API keys, scanned pairs/timeframes, per-pair base blocked hours,
+alert-quality boot default, log level. Signal logic, weights,
+thresholds and gates stay in config.py / pair_signals.
+
+Precedence for every setting: env var → config.json → code default.
+A repo-root .env file (gitignored) is loaded at import via setdefault,
+so it behaves exactly like real environment variables except that a
+real shell export always wins. Secrets have NO code default — with
+neither env nor config.json the bot refuses to start
+(telegram_bot.main) and data fetches fail with a named error
+(data_provider).
+
+Parity contract (test-enforced): with no config.json, or one holding
+the config.json.example defaults, every produced PlatformConfig is
+byte-identical to the pure-code defaults. apply_pair_overrides() runs
+in the pair factory BEFORE the validated per-timeframe deltas, so the
+bot, CLI and backtester all see the same overlay (live == backtest).
+"""
+
+import json
+import logging
+import os
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional
+
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+logger = logging.getLogger(__name__)
+
+CONFIG_PATH = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), "config.json")
+ENV_PATH = os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), ".env")
+
+
+def load_dotenv(path: str = ENV_PATH) -> None:
+    """Minimal .env loader (no python-dotenv dependency).
+
+    KEY=VALUE lines land in os.environ via setdefault — a variable
+    already exported in the shell always wins, keeping the
+    env → config.json precedence intact (.env is just a convenient
+    place to put env vars). Comments, blank lines, ``export `` prefixes
+    and single/double quotes are handled; anything else is skipped.
+    """
+    try:
+        with open(path, encoding="utf-8") as fh:
+            lines = fh.read().splitlines()
+    except OSError:
+        return
+    for line in lines:
+        line = line.strip()
+        if not line or line.startswith("#") or "=" not in line:
+            continue
+        if line.startswith("export "):
+            line = line[len("export "):]
+        key, _, value = line.partition("=")
+        key, value = key.strip(), value.strip()
+        if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
+            value = value[1:-1]
+        elif value[:1] not in "\"'":
+            # Unquoted value: whitespace-then-# starts an inline comment
+            # (KEY=abc  # prod). Quote the value to keep a literal " #".
+            for sep in (" #", "\t#"):
+                cut = value.find(sep)
+                if cut != -1:
+                    value = value[:cut].rstrip()
+        if key:
+            os.environ.setdefault(key, value)
+
+
+# Module import == process boot, before any resolver can run.
+load_dotenv()
+
+_VALID_QUALITIES = ("A+", "A", "B", "C")
+_VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
+# Timeframes the bot's candle-boundary scheduler supports (must have
+# TF_CONFIRMATIONS entries in telegram_bot.py).
+_SCANNABLE_TFS = ("M5", "M15", "H1", "H4")
+
+_KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level",
+               "oanda", "deepseek", "agent_panel",
+               # "market_outlook" is TOLERATED but ignored: the AI market
+               # outlook was removed 2026-09-15 (last commit 2836024). Live
+               # config.json files still carry the block and must keep
+               # loading; load_app_config logs one warning when it is there.
+               "market_outlook",
+               "violent_watch", "signal_log", "news_archive"}
+_KNOWN_OANDA = {
+    # "account_id" is TOLERATED but ignored: it belonged to the auto-
+    # execution daemon removed 2026-09-03 (last commit 31002cb). The candle
+    # provider needs only api_token + base_url; older config.json files
+    # still carry the key and must keep loading.
+    "account_id", "api_token", "base_url",
+}
+_KNOWN_DEEPSEEK = {
+    "api_key", "model", "enabled", "timeout_seconds", "daily_request_cap",
+}
+_KNOWN_AGENT_PANEL = {
+    "enabled", "telegram", "dashboard", "min_quality",
+    "max_panels_per_day", "reuse_ttl_seconds",
+    "candles_enabled", "candle_bars_primary", "candle_bars_confirm",
+    "verdict_source",
+}
+_KNOWN_VIOLENT_WATCH = {"enabled", "alert_line", "headsup",
+                        "max_headsup_per_day", "headsup_min_level"}
+_KNOWN_SIGNAL_LOG = {"enabled", "path", "log_blocked_scans"}
+_KNOWN_NEWS_ARCHIVE = {"enabled", "path"}
+_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
+_KNOWN_SCANNING = {"pairs", "timeframes"}
+_KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
+
+# Violent-move watch (display-only heads-up) — dark by default.
+_VIOLENT_WATCH_DEFAULTS = {
+    "enabled": False,
+    "alert_line": True,
+    "headsup": True,
+    "max_headsup_per_day": 12,
+    "headsup_min_level": "hot",
+}
+_VALID_WATCH_LEVELS = ("watch", "hot")
+
+
+class AppConfigError(Exception):
+    """config.json is present but invalid — fail fast at boot."""
+
+
+@dataclass
+class AppConfig:
+    telegram_token: str = ""
+    telegram_chat_id: str = ""
+    telegram_channel_id: str = ""          # "" = channel feed disabled
+    alert_min_quality: str = "C"
+    active_provider: str = "twelve_data"
+    streaming_provider: Optional[str] = None   # None = active provider
+    provider_settings: Dict[str, Dict[str, Any]] = field(default_factory=dict)
+    scan_pairs: List[str] = field(default_factory=lambda: [XAUUSD])
+    scan_timeframes: List[Timeframe] = field(default_factory=lambda: [
+        Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4])
+    pair_overrides: Dict[str, Dict[str, Any]] = field(default_factory=dict)
+    oanda_settings: Dict[str, Any] = field(default_factory=dict)
+    deepseek_settings: Dict[str, Any] = field(default_factory=dict)
+    agent_panel_settings: Dict[str, Any] = field(default_factory=dict)
+    violent_watch_settings: Dict[str, Any] = field(
+        default_factory=lambda: dict(_VIOLENT_WATCH_DEFAULTS))
+    signal_log_settings: Dict[str, Any] = field(default_factory=dict)
+    news_archive_settings: Dict[str, Any] = field(default_factory=dict)
+    log_level: str = "INFO"
+
+
+def _check_keys(section: Dict[str, Any], known: set, where: str) -> None:
+    if not isinstance(section, dict):
+        raise AppConfigError(f"config.json: {where} must be a JSON object")
+    unknown = sorted(set(section) - known)
+    if unknown:
+        raise AppConfigError(
+            f"config.json: unknown key(s) {unknown} in {where}; "
+            f"known keys: {sorted(known)}")
+
+
+def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
+    """Parse + validate config.json. Missing file → pure defaults."""
+    app = AppConfig()
+    if not os.path.exists(path):
+        logger.warning(
+            "config.json not found at %s — running on code defaults "
+            "(secrets must come from environment variables)", path)
+        return app
+    try:
+        with open(path, encoding="utf-8") as fh:
+            raw = json.load(fh)
+    except json.JSONDecodeError as e:
+        raise AppConfigError(f"config.json is not valid JSON: {e}") from e
+    if not isinstance(raw, dict):
+        raise AppConfigError("config.json top level must be a JSON object")
+    _check_keys(raw, _KNOWN_TOP, "top level")
+
+    tg = raw.get("telegram", {})
+    _check_keys(tg, _KNOWN_TELEGRAM, '"telegram"')
+    app.telegram_token = str(tg.get("token", app.telegram_token))
+    app.telegram_chat_id = str(tg.get("chat_id", app.telegram_chat_id))
+    app.telegram_channel_id = str(
+        tg.get("channel_id", app.telegram_channel_id))
+    quality = tg.get("alert_min_quality", app.alert_min_quality)
+    if quality not in _VALID_QUALITIES:
+        raise AppConfigError(
+            f'config.json: telegram.alert_min_quality "{quality}" '
+            f"not in {_VALID_QUALITIES}")
+    app.alert_min_quality = quality
+
+    dp = raw.get("data_providers")
+    if dp is not None:
+        if not isinstance(dp, dict):
+            raise AppConfigError('config.json: "data_providers" must be an object')
+        app.active_provider = str(dp.get("active", app.active_provider))
+        app.provider_settings = {}
+        for name, block in dp.items():
+            # "active" and "streaming" are reserved scalars, not blocks.
+            if name in ("active", "streaming"):
+                continue
+            if not isinstance(block, dict):
+                raise AppConfigError(
+                    f'config.json: data_providers.{name} must be an object')
+            app.provider_settings[name] = dict(block)
+        if app.active_provider not in app.provider_settings:
+            raise AppConfigError(
+                f'config.json: data_providers.active "{app.active_provider}" '
+                "has no matching provider block")
+        streaming = dp.get("streaming")
+        if streaming is not None:
+            if not isinstance(streaming, str):
+                raise AppConfigError(
+                    'config.json: data_providers.streaming must be a string '
+                    "(a provider name)")
+            if streaming not in app.provider_settings:
+                raise AppConfigError(
+                    f'config.json: data_providers.streaming "{streaming}" '
+                    "has no matching provider block")
+            app.streaming_provider = streaming
+
+    scanning = raw.get("scanning", {})
+    _check_keys(scanning, _KNOWN_SCANNING, '"scanning"')
+    if "pairs" in scanning:
+        pairs = scanning["pairs"]
+        if not isinstance(pairs, list):
+            raise AppConfigError('config.json: scanning.pairs must be a list')
+        bad = [p for p in pairs if p not in PAIR_INFO]
+        if bad:
+            raise AppConfigError(
+                f"config.json: unknown pair(s) {bad} in scanning.pairs; "
+                f"known: {sorted(PAIR_INFO)}")
+        app.scan_pairs = list(pairs)
+    if "timeframes" in scanning:
+        names = scanning["timeframes"]
+        if not isinstance(names, list):
+            raise AppConfigError(
+                'config.json: scanning.timeframes must be a list')
+        bad = [n for n in names if n not in _SCANNABLE_TFS]
+        if bad:
+            raise AppConfigError(
+                f"config.json: unsupported timeframe(s) {bad} in "
+                f"scanning.timeframes; supported: {list(_SCANNABLE_TFS)}")
+        app.scan_timeframes = [Timeframe[n] for n in names]
+
+    pair_section = raw.get("pairs", {})
+    if not isinstance(pair_section, dict):
+        raise AppConfigError('config.json: "pairs" must be a JSON object')
+    for symbol, overrides in pair_section.items():
+        if symbol not in PAIR_INFO:
+            raise AppConfigError(
+                f'config.json: unknown pair "{symbol}" in pairs; '
+                f"known: {sorted(PAIR_INFO)}")
+        _check_keys(overrides, _KNOWN_PAIR_KEYS, f'pairs."{symbol}"')
+        if "blocked_hours_utc" in overrides:
+            hours = overrides["blocked_hours_utc"]
+            valid = isinstance(hours, list) and all(
+                isinstance(h, int) and not isinstance(h, bool)
+                and 0 <= h <= 23 for h in hours)
+            if not valid:
+                raise AppConfigError(
+                    f'config.json: pairs."{symbol}".blocked_hours_utc must '
+                    "be a list of integers in 0-23")
+    app.pair_overrides = {s: dict(o) for s, o in pair_section.items()}
+
+    oanda = raw.get("oanda", {})
+    _check_keys(oanda, _KNOWN_OANDA, '"oanda"')
+    app.oanda_settings = dict(oanda)
+
+    deepseek = raw.get("deepseek", {})
+    _check_keys(deepseek, _KNOWN_DEEPSEEK, '"deepseek"')
+    if "enabled" in deepseek and not isinstance(deepseek["enabled"], bool):
+        raise AppConfigError(
+            "config.json: deepseek.enabled must be true or false")
+    if "timeout_seconds" in deepseek:
+        ts = deepseek["timeout_seconds"]
+        if isinstance(ts, bool) or not isinstance(ts, (int, float)) or ts <= 0:
+            raise AppConfigError(
+                "config.json: deepseek.timeout_seconds must be a positive "
+                "number")
+    if "daily_request_cap" in deepseek:
+        cap = deepseek["daily_request_cap"]
+        if isinstance(cap, bool) or not isinstance(cap, int) or cap < 1:
+            raise AppConfigError(
+                "config.json: deepseek.daily_request_cap must be a positive "
+                "integer")
+    if "model" in deepseek:
+        if not isinstance(deepseek["model"], str) or not deepseek["model"]:
+            raise AppConfigError(
+                "config.json: deepseek.model must be a non-empty string")
+    app.deepseek_settings = dict(deepseek)
+
+    panel = raw.get("agent_panel", {})
+    _check_keys(panel, _KNOWN_AGENT_PANEL, '"agent_panel"')
+    for flag in ("enabled", "telegram", "dashboard"):
+        if flag in panel and not isinstance(panel[flag], bool):
+            raise AppConfigError(
+                f"config.json: agent_panel.{flag} must be true or false")
+    if "min_quality" in panel and panel["min_quality"] not in _VALID_QUALITIES:
+        raise AppConfigError(
+            f'config.json: agent_panel.min_quality "{panel["min_quality"]}" '
+            f"not in {_VALID_QUALITIES}")
+    if "max_panels_per_day" in panel:
+        mpd = panel["max_panels_per_day"]
+        if isinstance(mpd, bool) or not isinstance(mpd, int) or mpd < 1:
+            raise AppConfigError(
+                "config.json: agent_panel.max_panels_per_day must be a "
+                "positive integer")
+    if "reuse_ttl_seconds" in panel:
+        ttl = panel["reuse_ttl_seconds"]
+        if isinstance(ttl, bool) or not isinstance(ttl, (int, float)) \
+                or ttl < 0:
+            raise AppConfigError(
+                "config.json: agent_panel.reuse_ttl_seconds must be a "
+                "non-negative number")
+    if "candles_enabled" in panel \
+            and not isinstance(panel["candles_enabled"], bool):
+        raise AppConfigError(
+            "config.json: agent_panel.candles_enabled must be true or false")
+    for key in ("candle_bars_primary", "candle_bars_confirm"):
+        if key in panel:
+            n = panel[key]
+            if isinstance(n, bool) or not isinstance(n, int) \
+                    or not 1 <= n <= 100:
+                raise AppConfigError(
+                    f"config.json: agent_panel.{key} must be an integer "
+                    "between 1 and 100")
+    if "verdict_source" in panel and panel["verdict_source"] not in \
+            ("macro", "chair"):
+        raise AppConfigError(
+            'config.json: agent_panel.verdict_source must be "macro" or '
+            '"chair"')
+    app.agent_panel_settings = dict(panel)
+
+    if "market_outlook" in raw:
+        # The AI market-outlook feature was removed 2026-09-15; older
+        # config.json files still carry its block and must keep loading.
+        logger.warning(
+            "config.json: market_outlook block is ignored (feature removed "
+            "2026-09-15) — delete it")
+
+    watch = raw.get("violent_watch", {})
+    _check_keys(watch, _KNOWN_VIOLENT_WATCH, '"violent_watch"')
+    for flag in ("enabled", "alert_line", "headsup"):
+        if flag in watch and not isinstance(watch[flag], bool):
+            raise AppConfigError(
+                f"config.json: violent_watch.{flag} must be true or false")
+    if "max_headsup_per_day" in watch:
+        cap = watch["max_headsup_per_day"]
+        if isinstance(cap, bool) or not isinstance(cap, int) or cap < 0:
+            raise AppConfigError(
+                "config.json: violent_watch.max_headsup_per_day must be a "
+                "non-negative integer")
+    if ("headsup_min_level" in watch
+            and watch["headsup_min_level"] not in _VALID_WATCH_LEVELS):
+        raise AppConfigError(
+            f'config.json: violent_watch.headsup_min_level '
+            f'"{watch["headsup_min_level"]}" not in {_VALID_WATCH_LEVELS}')
+    app.violent_watch_settings.update(watch)
+
+    slog = raw.get("signal_log", {})
+    _check_keys(slog, _KNOWN_SIGNAL_LOG, '"signal_log"')
+    if "enabled" in slog and not isinstance(slog["enabled"], bool):
+        raise AppConfigError(
+            "config.json: signal_log.enabled must be true or false")
+    if "path" in slog and (not isinstance(slog["path"], str)
+                           or not slog["path"]):
+        raise AppConfigError(
+            "config.json: signal_log.path must be a non-empty string")
+    if ("log_blocked_scans" in slog
+            and not isinstance(slog["log_blocked_scans"], bool)):
+        raise AppConfigError(
+            "config.json: signal_log.log_blocked_scans must be true or false")
+    app.signal_log_settings = dict(slog)
+
+    narch = raw.get("news_archive", {})
+    _check_keys(narch, _KNOWN_NEWS_ARCHIVE, '"news_archive"')
+    if "enabled" in narch and not isinstance(narch["enabled"], bool):
+        raise AppConfigError(
+            "config.json: news_archive.enabled must be true or false")
+    if "path" in narch and (not isinstance(narch["path"], str)
+                            or not narch["path"]):
+        raise AppConfigError(
+            "config.json: news_archive.path must be a non-empty string")
+    app.news_archive_settings = dict(narch)
+
+    level = raw.get("log_level", app.log_level)
+    if level not in _VALID_LOG_LEVELS:
+        raise AppConfigError(
+            f'config.json: log_level "{level}" not in {_VALID_LOG_LEVELS}')
+    app.log_level = level
+    return app
+
+
+# ── Process-wide singleton ────────────────────────────────────────────
+
+_cached: Optional[AppConfig] = None
+
+
+def get_app_config() -> AppConfig:
+    global _cached
+    if _cached is None:
+        _cached = load_app_config()
+    return _cached
+
+
+def set_app_config(app: Optional[AppConfig]) -> None:
+    """Override the process-wide AppConfig (tests/experiments).
+    Pass None to re-enable lazy loading from CONFIG_PATH."""
+    global _cached
+    _cached = app
+
+
+# ── Resolution helpers: env var → config.json → default ───────────────
+
+def telegram_token(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("TELEGRAM_TOKEN") or app.telegram_token
+
+
+def telegram_chat_id(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("TELEGRAM_CHAT_ID") or app.telegram_chat_id
+
+
+def telegram_channel_id(app: Optional[AppConfig] = None) -> str:
+    """Env set-but-empty ("") means "disable the feed" and beats JSON,
+    so this checks None-ness, not truthiness."""
+    app = app or get_app_config()
+    env = os.environ.get("TELEGRAM_CHANNEL_ID")
+    return env if env is not None else app.telegram_channel_id
+
+
+# Per-provider env-var overrides for API keys (env → config.json → "").
+_PROVIDER_ENV_KEYS = {
+    "twelve_data": "TWELVE_DATA_API_KEY",
+    "tiingo": "TIINGO_API_KEY",
+}
+
+
+def provider_api_key(
+    app: Optional[AppConfig] = None, provider: Optional[str] = None
+) -> str:
+    """API key for `provider` (default: the ACTIVE provider). The env
+    override is per-provider (TWELVE_DATA_API_KEY applies only to
+    twelve_data, TIINGO_API_KEY only to tiingo)."""
+    app = app or get_app_config()
+    name = provider or app.active_provider
+    env_key = _PROVIDER_ENV_KEYS.get(name)
+    if env_key:
+        env = os.environ.get(env_key)
+        if env:
+            return env
+    block = app.provider_settings.get(name, {})
+    return str(block.get("api_key", ""))
+
+
+# ── Oanda resolution helpers (env → config.json → default) ───────────
+
+def oanda_api_token(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("OANDA_API_TOKEN") or str(
+        app.oanda_settings.get("api_token", ""))
+
+
+def oanda_base_url(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("OANDA_BASE_URL") or str(
+        app.oanda_settings.get("base_url", "https://api-fxpractice.oanda.com"))
+
+
+# ── DeepSeek resolution helper (env → config.json → "") ──────────────
+
+def deepseek_api_key(app: Optional[AppConfig] = None) -> str:
+    app = app or get_app_config()
+    return os.environ.get("DEEPSEEK_API_KEY") or str(
+        app.deepseek_settings.get("api_key", ""))
+
+
+# ── Violent-move watch resolution helper ─────────────────────────────
+
+def violent_watch_enabled(app: Optional[AppConfig] = None) -> bool:
+    """Whether the display-only violent-move watch is on (env wins)."""
+    env = os.environ.get("VIOLENT_WATCH_ENABLED")
+    if env:
+        return env.strip().lower() in ("1", "true", "yes", "on")
+    app = app or get_app_config()
+    return bool(app.violent_watch_settings.get("enabled", False))
+
+
+# ── Signal-log resolution helpers (env → config.json → default) ──────
+
+def signal_log_enabled(app: Optional[AppConfig] = None) -> bool:
+    """Default ON — signal history is the platform's audit trail."""
+    env = os.environ.get("SIGNAL_LOG_ENABLED")
+    if env:
+        return env.strip().lower() in ("1", "true", "yes", "on")
+    app = app or get_app_config()
+    return bool(app.signal_log_settings.get("enabled", True))
+
+
+def signal_log_blocked_scans(app: Optional[AppConfig] = None) -> bool:
+    """Whether weekend/day-filter early returns are logged (as minimal
+    rows without news/candles). Default ON — they are liveness evidence;
+    turn off to keep the history table to real scans only."""
+    env = os.environ.get("SIGNAL_LOG_BLOCKED_SCANS")
+    if env:
+        return env.strip().lower() in ("1", "true", "yes", "on")
+    app = app or get_app_config()
+    return bool(app.signal_log_settings.get("log_blocked_scans", True))
+
+
+def signal_log_path(app: Optional[AppConfig] = None) -> str:
+    """Absolute path to the SQLite signal-history DB. A relative config
+    value resolves against the repo root so CWD never matters."""
+    env = os.environ.get("SIGNAL_LOG_PATH")
+    if env:
+        path = env
+    else:
+        app = app or get_app_config()
+        path = str(app.signal_log_settings.get("path", "logs/signals.db"))
+    if not os.path.isabs(path):
+        path = os.path.join(os.path.dirname(os.path.abspath(__file__)), path)
+    return path
+
+
+# ── News-archive resolution helpers (env → config.json → default) ────
+
+def news_archive_enabled(app: Optional[AppConfig] = None) -> bool:
+    """Default ON — the headline corpus is training data for a local
+    scorer and cannot be backfilled once a fetch window has passed. It
+    lives in its own DB so a signals.db rotation cannot take it with it
+    (which is how three weeks of news were lost on 2026-09-04)."""
+    env = os.environ.get("NEWS_ARCHIVE_ENABLED")
+    if env:
+        return env.strip().lower() in ("1", "true", "yes", "on")
+    app = app or get_app_config()
+    return bool(app.news_archive_settings.get("enabled", True))
+
+
+def news_archive_path(app: Optional[AppConfig] = None) -> str:
+    """Absolute path to the SQLite headline corpus. A relative config
+    value resolves against the repo root so CWD never matters."""
+    env = os.environ.get("NEWS_ARCHIVE_PATH")
+    if env:
+        path = env
+    else:
+        app = app or get_app_config()
+        path = str(app.news_archive_settings.get("path", "logs/newsdata.db"))
+    if not os.path.isabs(path):
+        path = os.path.join(os.path.dirname(os.path.abspath(__file__)), path)
+    return path
+
+
+# ── PlatformConfig overlay ────────────────────────────────────────────
+
+def apply_pair_overrides(
+    cfg: PlatformConfig, app: Optional[AppConfig] = None
+) -> PlatformConfig:
+    """Stamp deployment settings from config.json onto a PlatformConfig.
+
+    Called by the pair factories BEFORE their validated per-timeframe
+    deltas, so e.g. the M5 hour-9 append lands on top of the JSON base.
+    The pair's base blocked_hours_utc is set on BOTH mirrored copies
+    (xau_strategy + backtest) so they cannot drift.
+    """
+    app = app or get_app_config()
+    overrides = app.pair_overrides.get(cfg.symbol, {})
+    if "blocked_hours_utc" in overrides:
+        hours = list(overrides["blocked_hours_utc"])
+        cfg.xau_strategy.blocked_hours_utc = hours
+        cfg.backtest.blocked_hours_utc = list(hours)
+    cfg.scan_pairs = list(app.scan_pairs)
+    cfg.scan_timeframes = list(app.scan_timeframes)
+    return cfg

backtest/instructions_for_backtest.txt

@@ -0,0 +1,40 @@
+Backtest reporting checklist — ForexBot2 (XAU/USD)
+==================================================
+Every backtest report must record:
+- The exact command line (window, --primary-tf, --spread, and any --set /
+  --quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
+- Trades, total R, win rate, profit factor and max drawdown for the whole
+  window AND for both halves, at both cost conventions ($0.65 measured-Oanda
+  headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
+- Per trade (the exported CSV already carries these): entry/exit time and
+  price, exit reason, R multiple, MFE/MAE, entry reasons, and every
+  indicator's signal / value / agreement at decision time.
+- Breakdowns by session/hour, weekday, regime and volatility regime, and,
+  for any candidate change, the removed/added/shifted trade diff against the
+  control book (the 5-bar cooldown displaces entries, so judge run-level
+  numbers, never the added-trade bucket alone).
+- The md5 of the exported book whenever the run is a parity or regression check.
+Replays go through the real signal code (analyze_from_dataframes) — never a
+re-implementation.
+
+--- 2026-09-06: decision-time feature export ---
+- `--export-features` appends stake_multiplier, r_sized and the 65 `f_*` decision-time
+  features (position_sizing.FEATURE_ORDER) AFTER the 63 pinned columns; default export is
+  byte-identical to the pinned books.
+- Any sizing report must show the FLAT book and the SIZED book side by side and the
+  label-shuffle null; the 2026-09-06 campaign verdict (nothing ships,
+  backtest/research/sizing_campaign_2026-09.md) is the reference.
+
+--- 2026-09-09: replay recording (--record DIR [--record-bars]) ---
+- `--record DIR` writes manifest.json + trades/indicators/events parquet for every ACCEPTED trade:
+  every indicator's signal / value / confidence / BASE weight / regime multiplier / EFFECTIVE
+  weight / contribution on the primary AND both confirmation timeframes, penalties, the setup, the
+  outcome, and the Forex Factory calendar events around the trade (backtest/cache/calendar.parquet,
+  refreshed with scripts/ff_calendar_scraper.py). `--record-bars` adds one row per ANALYZED primary
+  bar with the same votes plus forward moves (1h/4h/24h in ATR units).
+- Recording is read-only w.r.t. the signal path: any recorded run must be paired with a control
+  run (same CLI without --record) and the report must show the two books' MD5s equal
+  (manifest.book_csv_md5 is the md5 of the 63-column book).
+- Scorecards and "what-if" tables from these files (backtest/replay_scorecard.py) are IN-BOOK
+  descriptive statistics: they cannot add trades a change would create and ignore the gates.
+  Any weight / indicator change they suggest is a hypothesis for a full rule-1 rerun, never a decision.

backtest/replay_scorecard.py

@@ -0,0 +1,894 @@
+#!/usr/bin/env python3
+"""
+backtest/replay_scorecard.py — read a replay record, score the indicators
+=========================================================================
+
+READ-ONLY analysis of one (or several) replay-record folders written by
+``backtest/replay_record.py``: which indicator votes actually earned
+their weight in the book, and what would have changed if one of them had
+been removed or re-weighted.
+
+Six lenses:
+
+1. INDICATOR SCORECARD — for every timeframe role (primary / c1 / c2) and
+   every indicator: mean R when that indicator AGREED with the trade,
+   when it OPPOSED it, when it was NEUTRAL, and when it did not vote at
+   all ("absent" — divergence and candlestick only fire on a pattern, so
+   a missing row is "no opinion", never a zero vote).
+
+2. CONTRIBUTION CORRELATION — rank correlation between how much an
+   indicator pushed the composite in the trade's own direction and the R
+   the trade made. Positive = pushing harder went with better trades.
+
+3. SCORECARD BY REGIME — the same agree/oppose/neutral split, primary
+   timeframe only, cut by the market regime at decision time. Weights
+   are already regime-dependent; this says whether that is paying.
+
+4. CALENDAR BUCKETS — trades entered just before / just after a
+   high-impact USD release, trades that were held through one, quiet
+   trades, and US holidays.
+
+5. WHAT-IF (--what-if) — recompute each trade's primary composite with
+   one indicator dropped or re-weighted, and count the trades that would
+   have lost their direction or changed strength band. IN-BOOK ONLY: it
+   can never show the trades a change would ADD, and it ignores every
+   gate. Any real change needs a full backtest rerun (CLAUDE.md rule 1).
+
+6. BAR-LEVEL (--bars, needs bars.parquet) — the same what-if over EVERY
+   analyzed bar, not just the accepted trades, scored against the plain
+   forward move in ATRs; plus a per-indicator table asking whether that
+   indicator's vote, on its own, pointed the right way.
+
+``--check`` re-derives the record's own arithmetic (weights, weighted
+scores, composites, regime multipliers, bar/trade counts) and exits
+non-zero if any invariant is broken.
+
+Usage:
+    python backtest/replay_scorecard.py backtest/records/m5_12mo
+    python backtest/replay_scorecard.py RUN_A RUN_B --min-n 30
+    python backtest/replay_scorecard.py RUN --what-if
+    python backtest/replay_scorecard.py RUN --bars
+    python backtest/replay_scorecard.py RUN --check
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+from pathlib import Path
+from typing import Dict, List, Optional, Sequence, Tuple
+
+import numpy as np
+import pandas as pd
+
+# Ensure project root on sys.path so this script can be invoked from
+# anywhere (same idiom as backtest/gpu_backtest.py).
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+# Contract defaults. backtest.replay_record owns the real values; they are
+# copied in by _load_record_module() at CLI time so that the compute
+# functions below stay unit-testable without that module present.
+INDICATORS: Tuple[str, ...] = (
+    "ema_trend", "rsi", "macd", "bollinger", "adx",
+    "roc", "keltner", "divergence", "candlestick",
+)
+ROLES: Tuple[str, ...] = ("primary", "c1", "c2")
+COMPOSITE_THRESHOLDS: Dict[str, float] = {"strong": 40.0, "weak": 15.0}
+
+# Monkeypatch hook: set this to a callable
+# ``(eff_weights, wscores) -> (score, signal_int)`` to run the compute
+# functions without backtest.replay_record on the path.
+_composite_from_rows = None
+
+BUCKETS = ("agree", "oppose", "neutral", "absent")
+
+
+# ── record module (lazy — compute functions must not need it) ─────────
+
+
+def _load_record_module():
+    """Import backtest.replay_record and adopt its constants."""
+    try:
+        from backtest import replay_record  # noqa: E402 (lazy by design)
+    except Exception as exc:                 # pragma: no cover - CLI path
+        raise ImportError(
+            "backtest/replay_record.py is not importable "
+            f"({exc}). It is the loader for replay-record folders; "
+            "this script cannot read a run without it."
+        ) from exc
+    global INDICATORS, ROLES, COMPOSITE_THRESHOLDS, _composite_from_rows
+    INDICATORS = tuple(getattr(replay_record, "INDICATORS", INDICATORS))
+    ROLES = tuple(getattr(replay_record, "ROLES", ROLES))
+    COMPOSITE_THRESHOLDS = dict(
+        getattr(replay_record, "COMPOSITE_THRESHOLDS", COMPOSITE_THRESHOLDS))
+    if _composite_from_rows is None:
+        _composite_from_rows = replay_record.composite_from_rows
+    return replay_record
+
+
+def _composite(eff_weights, wscores) -> Tuple[float, int]:
+    """composite_from_rows, via the hook or a lazy import."""
+    global _composite_from_rows
+    if _composite_from_rows is None:
+        _composite_from_rows = _load_record_module().composite_from_rows
+    return _composite_from_rows(eff_weights, wscores)
+
+
+def _weak() -> float:
+    return float(COMPOSITE_THRESHOLDS.get("weak", 15.0))
+
+
+def _strong() -> float:
+    return float(COMPOSITE_THRESHOLDS.get("strong", 40.0))
+
+
+# ── small formatting helpers ─────────────────────────────────────────
+
+
+def _fmt_r(value) -> str:
+    return "     -" if value is None or pd.isna(value) else f"{value:+6.3f}"
+
+
+def _fmt_total(value) -> str:
+    return "      -" if value is None or pd.isna(value) else f"{value:+7.1f}"
+
+
+def _fmt_pct(value) -> str:
+    return "   -" if value is None or pd.isna(value) else f"{100 * value:3.0f}%"
+
+
+def _fmt_num(value, nd: int = 3) -> str:
+    return "     -" if value is None or pd.isna(value) else f"{value:+.{nd}f}"
+
+
+def _dir_sign(direction) -> float:
+    return -1.0 if str(direction).upper().startswith("S") else 1.0
+
+
+def _stats(r: pd.Series) -> Dict[str, float]:
+    """n / mean R / win rate / total R for one bucket of trades."""
+    r = pd.to_numeric(pd.Series(r), errors="coerce").dropna()
+    if r.empty:
+        return {"n": 0, "mean_r": float("nan"),
+                "win_rate": float("nan"), "total_r": 0.0}
+    return {"n": int(len(r)), "mean_r": float(r.mean()),
+            "win_rate": float((r > 0).mean()), "total_r": float(r.sum())}
+
+
+def _spearman(a: pd.Series, b: pd.Series) -> float:
+    """Rank correlation without scipy (average-tie ranks + corrcoef)."""
+    frame = pd.DataFrame({"a": pd.to_numeric(a, errors="coerce"),
+                          "b": pd.to_numeric(b, errors="coerce")}).dropna()
+    if len(frame) < 3:
+        return float("nan")
+    ra, rb = frame["a"].rank(), frame["b"].rank()
+    if ra.nunique() < 2 or rb.nunique() < 2:
+        return float("nan")
+    return float(np.corrcoef(ra.to_numpy(), rb.to_numpy())[0, 1])
+
+
+def _indicator_names(indicators: pd.DataFrame) -> List[str]:
+    """Contract order first, then anything extra the record carried."""
+    seen = set(indicators["indicator"].dropna().unique()) \
+        if "indicator" in indicators.columns else set()
+    names = [i for i in INDICATORS]
+    names += sorted(n for n in seen if n not in set(names))
+    return names
+
+
+def _roles_present(indicators: pd.DataFrame) -> List[str]:
+    if "tf_role" not in indicators.columns:
+        return []
+    seen = set(indicators["tf_role"].dropna().unique())
+    roles = [r for r in ROLES if r in seen]
+    roles += sorted(r for r in seen if r not in set(ROLES))
+    return roles
+
+
+# ── 1. indicator scorecard ───────────────────────────────────────────
+
+
+def indicator_scorecard(trades: pd.DataFrame, indicators: pd.DataFrame,
+                        min_n: int = 20) -> pd.DataFrame:
+    """Mean R per (timeframe role, indicator, agree/oppose/neutral/absent).
+
+    The trade universe for a role is every trade that has at least one
+    indicator row for that role, so "absent" means "this indicator did
+    not vote on that trade", not "the role was missing".
+    """
+    out: List[dict] = []
+    hidden = 0
+    if trades.empty or indicators.empty:
+        frame = pd.DataFrame(out, columns=["tf_role", "indicator", "bucket",
+                                           "n", "mean_r", "win_rate",
+                                           "total_r"])
+        frame.attrs["n_hidden"] = 0
+        frame.attrs["min_n"] = min_n
+        return frame
+    r_by_trade = trades.set_index("trade_id")["r_multiple"]
+    for role in _roles_present(indicators):
+        role_rows = indicators[indicators["tf_role"] == role]
+        universe = pd.Index(role_rows["trade_id"].unique())
+        universe = universe.intersection(r_by_trade.index)
+        if len(universe) == 0:
+            continue
+        for name in _indicator_names(indicators):
+            sub = role_rows[role_rows["indicator"] == name]
+            agrees = dict(zip(sub["trade_id"], sub["agrees"]))
+            groups: Dict[str, List] = {b: [] for b in BUCKETS}
+            for tid in universe:
+                bucket = agrees.get(tid)
+                key = "absent" if bucket is None or pd.isna(bucket) \
+                    else str(bucket)
+                groups.setdefault(key, []).append(tid)
+            for bucket, tids in groups.items():
+                stat = _stats(r_by_trade.loc[tids]) if tids else _stats(
+                    pd.Series(dtype=float))
+                if stat["n"] < min_n:
+                    hidden += 1 if stat["n"] else 0
+                    continue
+                out.append({"tf_role": role, "indicator": name,
+                            "bucket": bucket, **stat})
+    frame = pd.DataFrame(out, columns=["tf_role", "indicator", "bucket",
+                                       "n", "mean_r", "win_rate", "total_r"])
+    frame.attrs["n_hidden"] = hidden
+    frame.attrs["min_n"] = min_n
+    return frame
+
+
+def contribution_corr(trades: pd.DataFrame,
+                      indicators: pd.DataFrame) -> pd.DataFrame:
+    """Rank correlation of (push in the trade's direction) vs R made."""
+    out: List[dict] = []
+    if trades.empty or indicators.empty:
+        return pd.DataFrame(out, columns=["tf_role", "indicator", "n",
+                                          "spearman"])
+    keep = trades[["trade_id", "direction", "r_multiple"]].copy()
+    keep["dir_sign"] = keep["direction"].map(_dir_sign)
+    merged = indicators.merge(keep, on="trade_id", how="inner")
+    merged["contribution"] = pd.to_numeric(
+        merged["wscore"], errors="coerce") * merged["dir_sign"]
+    for role in _roles_present(indicators):
+        for name in _indicator_names(indicators):
+            sub = merged[(merged["tf_role"] == role)
+                         & (merged["indicator"] == name)]
+            if sub.empty:
+                continue
+            out.append({"tf_role": role, "indicator": name, "n": int(len(sub)),
+                        "spearman": _spearman(sub["contribution"],
+                                              sub["r_multiple"])})
+    return pd.DataFrame(out, columns=["tf_role", "indicator", "n", "spearman"])
+
+
+def print_indicator_scorecard(frame: pd.DataFrame,
+                              corr: Optional[pd.DataFrame] = None) -> None:
+    print(f"\n{'─' * 74}\n  INDICATOR SCORECARD — mean R by what the "
+          f"indicator said about the trade\n{'─' * 74}")
+    print("  Read: 'agree' = the indicator pointed the same way as the trade "
+          "and\n  'absent' = it never voted (divergence/candlestick only fire "
+          "on a pattern).")
+    if frame.empty:
+        print("  (nothing to show)")
+        return
+    min_n = frame.attrs.get("min_n", 0)
+    for role in frame["tf_role"].drop_duplicates():
+        print(f"\n  {role.upper()} timeframe")
+        print(f"    {'indicator':12} {'bucket':8} {'n':>5}  {'mean R':>6}  "
+              f"{'win':>4}  {'total R':>7}")
+        for _, row in frame[frame["tf_role"] == role].iterrows():
+            print(f"    {row['indicator']:12} {row['bucket']:8} "
+                  f"{int(row['n']):5d}  {_fmt_r(row['mean_r'])}  "
+                  f"{_fmt_pct(row['win_rate'])}  "
+                  f"{_fmt_total(row['total_r'])}")
+    hidden = frame.attrs.get("n_hidden", 0)
+    if hidden:
+        print(f"\n  ({hidden} buckets hidden — fewer than {min_n} trades)")
+    if corr is not None and not corr.empty:
+        print("\n  Rank correlation, how hard it pushed vs what the trade "
+              "made\n  (positive = pushing harder went with better trades; "
+              "0 = no relation):")
+        for role in corr["tf_role"].drop_duplicates():
+            part = corr[corr["tf_role"] == role]
+            cells = "  ".join(
+                f"{r['indicator']}:{_fmt_num(r['spearman'], 2)}"
+                for _, r in part.iterrows())
+            print(f"    {role:8} {cells}")
+
+
+# ── 2. scorecard by regime ───────────────────────────────────────────
+
+
+def scorecard_by_regime(trades: pd.DataFrame, indicators: pd.DataFrame,
+                        min_n: int = 20) -> pd.DataFrame:
+    """Primary-timeframe agree/oppose/neutral split, cut by market regime."""
+    out: List[dict] = []
+    cols = ["regime", "indicator", "bucket", "n", "mean_r", "win_rate",
+            "total_r"]
+    if trades.empty or indicators.empty or "regime" not in trades.columns:
+        return pd.DataFrame(out, columns=cols)
+    prim = indicators[indicators["tf_role"] == "primary"]
+    if prim.empty:
+        return pd.DataFrame(out, columns=cols)
+    regime_of = trades.set_index("trade_id")["regime"]
+    r_by_trade = trades.set_index("trade_id")["r_multiple"]
+    for regime in sorted(str(x) for x in regime_of.dropna().unique()):
+        tids = pd.Index(regime_of[regime_of == regime].index)
+        sub = prim[prim["trade_id"].isin(tids)]
+        if sub.empty:
+            continue
+        for name in _indicator_names(indicators):
+            rows = sub[sub["indicator"] == name]
+            for bucket in ("agree", "oppose", "neutral"):
+                sel = rows[rows["agrees"] == bucket]["trade_id"]
+                stat = _stats(r_by_trade.reindex(sel))
+                if stat["n"] < min_n:
+                    continue
+                out.append({"regime": regime, "indicator": name,
+                            "bucket": bucket, **stat})
+    return pd.DataFrame(out, columns=cols)
+
+
+def print_scorecard_by_regime(frame: pd.DataFrame, min_n: int = 20) -> None:
+    print(f"\n{'─' * 74}\n  BY MARKET REGIME — primary timeframe only"
+          f"\n{'─' * 74}")
+    print("  Read: the same agree/oppose/neutral split, but only inside one "
+          "kind of\n  market — weights already change by regime, this says "
+          "whether that pays.")
+    if frame.empty:
+        print(f"  (no bucket reached {min_n} trades)")
+        return
+    for regime in frame["regime"].drop_duplicates():
+        print(f"\n  regime: {regime}")
+        for _, row in frame[frame["regime"] == regime].iterrows():
+            print(f"    {row['indicator']:12} {row['bucket']:8} "
+                  f"{int(row['n']):5d}  {_fmt_r(row['mean_r'])}  "
+                  f"{_fmt_pct(row['win_rate'])}  "
+                  f"{_fmt_total(row['total_r'])}")
+
+
+# ── 3. calendar buckets ──────────────────────────────────────────────
+
+
+def calendar_buckets(trades: pd.DataFrame) -> pd.DataFrame:
+    """Trades grouped by their position around high-impact USD releases."""
+    cols = ["bucket", "n", "mean_r", "win_rate", "total_r"]
+    if trades.empty:
+        return pd.DataFrame([], columns=cols)
+
+    def col(name, default=np.nan) -> pd.Series:
+        if name in trades.columns:
+            return trades[name]
+        return pd.Series([default] * len(trades), index=trades.index)
+
+    nxt = pd.to_numeric(col("ev_next_hi_usd_min"), errors="coerce")
+    prev = pd.to_numeric(col("ev_prev_hi_usd_min"), errors="coerce")
+    during = pd.to_numeric(col("ev_hi_usd_during_hold"), errors="coerce")
+    holiday = col("us_holiday", False).fillna(False).astype(bool)
+    r = trades["r_multiple"]
+    masks = [
+        ("all trades", pd.Series(True, index=trades.index)),
+        ("entered <=30 min BEFORE a USD high-impact release", nxt <= 30),
+        ("entered <=30 min AFTER one", prev <= 30),
+        ("held through >=1 release", during >= 1),
+        ("quiet (no release within 24 h either side, none during hold)",
+         nxt.isna() & prev.isna() & (during.fillna(0) == 0)),
+        ("US holiday", holiday),
+    ]
+    out = [{"bucket": label, **_stats(r[mask.fillna(False)])}
+           for label, mask in masks]
+    return pd.DataFrame(out, columns=cols)
+
+
+def print_calendar_buckets(frame: pd.DataFrame) -> None:
+    print(f"\n{'─' * 74}\n  CALENDAR — trades around high-impact USD news"
+          f"\n{'─' * 74}")
+    print("  Read: one trade can land in more than one row (a trade entered "
+          "before a\n  release is usually also holding through it), so the "
+          "rows do not add up.")
+    if frame.empty:
+        print("  (nothing to show)")
+        return
+    for _, row in frame.iterrows():
+        print(f"    {row['bucket']:58} n={int(row['n']):5d}  "
+              f"{_fmt_r(row['mean_r'])}  {_fmt_pct(row['win_rate'])}  "
+              f"{_fmt_total(row['total_r'])}")
+
+
+# ── 4. what-if (in-book only) ────────────────────────────────────────
+
+WHATIF_HEADER = (
+    "IN-BOOK ONLY — this cannot add trades the change would create and "
+    "ignores the gates;\n  any real change needs a full backtest rerun "
+    "(rule 1)."
+)
+
+
+def _primary_rows_by_trade(indicators: pd.DataFrame) -> Dict[object, Tuple]:
+    """trade_id -> (indicator names, eff_weights, wscores) for the primary TF."""
+    cache: Dict[object, Tuple] = {}
+    if indicators.empty:
+        return cache
+    prim = indicators[indicators["tf_role"] == "primary"]
+    for tid, group in prim.groupby("trade_id"):
+        cache[tid] = (group["indicator"].to_numpy(),
+                      pd.to_numeric(group["eff_weight"],
+                                    errors="coerce").to_numpy(float),
+                      pd.to_numeric(group["wscore"],
+                                    errors="coerce").to_numpy(float))
+    return cache
+
+
+def what_if(trades: pd.DataFrame, indicators: pd.DataFrame,
+            drop: Optional[str] = None,
+            scale: Optional[Tuple[str, float]] = None,
+            _rows: Optional[Dict[object, Tuple]] = None) -> pd.DataFrame:
+    """Recompute each trade's primary composite with one indicator
+    removed (``drop="bollinger"``) or re-weighted
+    (``scale=("bollinger", 0.5)``)."""
+    cols = ["trade_id", "direction", "r_multiple", "composite_recorded",
+            "composite_new", "loses_direction", "band_change"]
+    if trades.empty:
+        return pd.DataFrame([], columns=cols)
+    rows = _primary_rows_by_trade(indicators) if _rows is None else _rows
+    weak, strong = _weak(), _strong()
+    out: List[dict] = []
+    for _, trade in trades.iterrows():
+        tid = trade["trade_id"]
+        names, eff, ws = rows.get(
+            tid, (np.array([]), np.array([]), np.array([])))
+        eff, ws = eff.copy(), ws.copy()
+        if drop is not None and len(names):
+            keep = names != drop
+            eff, ws = eff[keep], ws[keep]
+        elif scale is not None and len(names):
+            target, factor = scale
+            hit = names == target
+            eff = np.where(hit, eff * float(factor), eff)
+            ws = np.where(hit, ws * float(factor), ws)
+        new_score, _ = _composite(eff, ws)
+        recorded = trade.get("primary_composite", np.nan)
+        recorded = float(recorded) if pd.notna(recorded) else float("nan")
+        sign = _dir_sign(trade.get("direction"))
+        loses = bool(abs(new_score) < weak
+                     or np.sign(new_score) != np.sign(sign))
+        band = bool((abs(recorded) >= strong) != (abs(new_score) >= strong)) \
+            if not np.isnan(recorded) else False
+        out.append({"trade_id": tid, "direction": trade.get("direction"),
+                    "r_multiple": trade.get("r_multiple"),
+                    "composite_recorded": recorded,
+                    "composite_new": float(new_score),
+                    "loses_direction": loses, "band_change": band})
+    return pd.DataFrame(out, columns=cols)
+
+
+def what_if_summary(frame: pd.DataFrame) -> Dict[str, float]:
+    """How many trades the change would unsettle, and the R they carried."""
+    if frame.empty:
+        return {"n": 0, "n_loses_direction": 0, "r_of_lost": 0.0,
+                "n_band_change": 0, "r_of_band_change": 0.0}
+    lost = frame[frame["loses_direction"].astype(bool)]
+    band = frame[frame["band_change"].astype(bool)]
+    return {"n": int(len(frame)),
+            "n_loses_direction": int(len(lost)),
+            "r_of_lost": float(pd.to_numeric(
+                lost["r_multiple"], errors="coerce").sum()),
+            "n_band_change": int(len(band)),
+            "r_of_band_change": float(pd.to_numeric(
+                band["r_multiple"], errors="coerce").sum())}
+
+
+def what_if_grid(trades: pd.DataFrame,
+                 indicators: pd.DataFrame) -> pd.DataFrame:
+    """drop / x0.5 / x1.5 for every indicator, one row each."""
+    rows = _primary_rows_by_trade(indicators)
+    out: List[dict] = []
+    for name in _indicator_names(indicators):
+        for label, kwargs in (("drop", {"drop": name}),
+                              ("x0.5", {"scale": (name, 0.5)}),
+                              ("x1.5", {"scale": (name, 1.5)})):
+            summary = what_if_summary(
+                what_if(trades, indicators, _rows=rows, **kwargs))
+            out.append({"indicator": name, "change": label, **summary})
+    return pd.DataFrame(out, columns=["indicator", "change", "n",
+                                      "n_loses_direction", "r_of_lost",
+                                      "n_band_change", "r_of_band_change"])
+
+
+def print_what_if(frame: pd.DataFrame) -> None:
+    print(f"\n{'─' * 74}\n  WHAT-IF — drop or re-weight one indicator"
+          f"\n{'─' * 74}")
+    print(f"  {WHATIF_HEADER}")
+    print("  Read: 'loses direction' = the trade's own score would no longer "
+          "clear the\n  entry band (or flips sign); 'band change' = it would "
+          "cross the strong/weak line.")
+    if frame.empty:
+        print("  (nothing to show)")
+        return
+    print(f"\n    {'indicator':12} {'change':6} {'lost':>5} {'R lost':>8}  "
+          f"{'band':>5} {'R band':>8}")
+    for _, row in frame.iterrows():
+        print(f"    {row['indicator']:12} {row['change']:6} "
+              f"{int(row['n_loses_direction']):5d} "
+              f"{_fmt_total(row['r_of_lost'])}  "
+              f"{int(row['n_band_change']):5d} "
+              f"{_fmt_total(row['r_of_band_change'])}")
+
+
+# ── 5. bar-level lenses ──────────────────────────────────────────────
+
+
+def _bar_composites(bars: pd.DataFrame, drop: Optional[str] = None,
+                    scale: Optional[Tuple[str, float]] = None) -> np.ndarray:
+    """Primary composite for every bar, recomputed from the stored parts."""
+    names = [n for n in INDICATORS
+             if f"primary_{n}_eff_weight" in bars.columns
+             and f"primary_{n}_wscore" in bars.columns]
+    if not names:
+        return np.zeros(len(bars))
+    # copy=True: a to_numpy() view can be read-only under copy-on-write.
+    eff = bars[[f"primary_{n}_eff_weight" for n in names]].to_numpy(
+        dtype=float, copy=True)
+    ws = bars[[f"primary_{n}_wscore" for n in names]].to_numpy(
+        dtype=float, copy=True)
+    if drop is not None and drop in names:
+        idx = names.index(drop)
+        eff[:, idx] = np.nan
+        ws[:, idx] = np.nan
+    elif scale is not None and scale[0] in names:
+        idx = names.index(scale[0])
+        eff[:, idx] = eff[:, idx] * float(scale[1])
+        ws[:, idx] = ws[:, idx] * float(scale[1])
+    total = np.nansum(eff, axis=1)
+    numer = np.nansum(ws, axis=1)
+    score = np.divide(numer, total, out=np.zeros_like(numer),
+                      where=total > 0) * 50.0
+    return np.clip(score, -100.0, 100.0)
+
+
+def _signed_mean(bars: pd.DataFrame, mask: np.ndarray, sign: np.ndarray,
+                 column: str) -> float:
+    if column not in bars.columns or not mask.any():
+        return float("nan")
+    values = pd.to_numeric(bars[column], errors="coerce").to_numpy(float)
+    picked = values[mask] * sign[mask]
+    picked = picked[np.isfinite(picked)]
+    return float(picked.mean()) if picked.size else float("nan")
+
+
+def bars_what_if(bars: pd.DataFrame, drop: Optional[str] = None,
+                 scale: Optional[Tuple[str, float]] = None) -> Dict[str, dict]:
+    """Every analyzed bar, not just the trades: which bars would newly
+    become directional, and which would fall back to neutral."""
+    label = f"drop {drop}" if drop else (
+        f"{scale[0]} x{scale[1]}" if scale else "unchanged")
+    if bars is None or bars.empty:
+        return {"label": label, "n_bars": 0,
+                "new_cross": {"n": 0}, "drop_out": {"n": 0}}
+    recorded = pd.to_numeric(bars.get("primary_composite"),
+                             errors="coerce").to_numpy(float)
+    new = _bar_composites(bars, drop=drop, scale=scale)
+    weak = _weak()
+    rec_dir = np.abs(np.nan_to_num(recorded)) >= weak
+    new_dir = np.abs(new) >= weak
+    cross = (~rec_dir) & new_dir
+    fell = rec_dir & (~new_dir)
+    sign_new = np.sign(new)
+    sign_rec = np.sign(np.nan_to_num(recorded))
+    out = {"label": label, "n_bars": int(len(bars))}
+    for key, mask, sign in (("new_cross", cross, sign_new),
+                            ("drop_out", fell, sign_rec)):
+        out[key] = {
+            "n": int(mask.sum()),
+            "fwd_4h_atr": _signed_mean(bars, mask, sign, "fwd_4h_atr"),
+            "fwd_24h_atr": _signed_mean(bars, mask, sign, "fwd_24h_atr"),
+        }
+    return out
+
+
+def bars_indicator_edge(bars: pd.DataFrame) -> pd.DataFrame:
+    """Does this indicator's vote, on its own, point the right way?"""
+    cols = ["indicator", "bucket", "n", "fwd_4h_signed", "fwd_24h_signed"]
+    if bars is None or bars.empty:
+        return pd.DataFrame([], columns=cols)
+    out: List[dict] = []
+    for name in INDICATORS:
+        column = f"primary_{name}_signal"
+        if column not in bars.columns:
+            continue
+        signal = pd.to_numeric(bars[column], errors="coerce").to_numpy(float)
+        buckets = (("buy-side", signal > 0, np.ones(len(bars))),
+                   ("sell-side", signal < 0, -np.ones(len(bars))),
+                   ("neutral", signal == 0, np.ones(len(bars))),
+                   ("absent", np.isnan(signal), np.ones(len(bars))))
+        for bucket, mask, sign in buckets:
+            out.append({
+                "indicator": name, "bucket": bucket, "n": int(mask.sum()),
+                "fwd_4h_signed": _signed_mean(bars, mask, sign, "fwd_4h_atr"),
+                "fwd_24h_signed": _signed_mean(bars, mask, sign,
+                                               "fwd_24h_atr")})
+    return pd.DataFrame(out, columns=cols)
+
+
+def print_bars_indicator_edge(frame: pd.DataFrame) -> None:
+    print(f"\n{'─' * 74}\n  EVERY BAR — where did price go after each vote?"
+          f"\n{'─' * 74}")
+    print("  Read: the number is the average move over the next 4 h / 24 h in "
+          "ATRs,\n  flipped so that positive always means the vote was right. "
+          "'neutral' and\n  'absent' rows are the plain move (no direction "
+          "claimed).")
+    if frame.empty:
+        print("  (no bars table in this run)")
+        return
+    print(f"\n    {'indicator':12} {'vote':10} {'n':>7}  {'4 h':>7}  "
+          f"{'24 h':>7}")
+    for _, row in frame.iterrows():
+        print(f"    {row['indicator']:12} {row['bucket']:10} "
+              f"{int(row['n']):7d}  {_fmt_num(row['fwd_4h_signed'])}  "
+              f"{_fmt_num(row['fwd_24h_signed'])}")
+
+
+def print_bars_what_if(results: Sequence[dict]) -> None:
+    print(f"\n{'─' * 74}\n  EVERY BAR — what-if: bars gained and bars lost"
+          f"\n{'─' * 74}")
+    print("  Read: 'new' bars would start calling a direction they do not "
+          "call today,\n  'lost' bars would stop. The R-free forward move "
+          "says whether that is good.")
+    print(f"\n    {'change':16} {'new':>6} {'4 h':>7} {'24 h':>7}   "
+          f"{'lost':>6} {'4 h':>7} {'24 h':>7}")
+    for res in results:
+        new, fell = res["new_cross"], res["drop_out"]
+        print(f"    {res['label']:16} {new['n']:6d} "
+              f"{_fmt_num(new.get('fwd_4h_atr'))} "
+              f"{_fmt_num(new.get('fwd_24h_atr'))}   "
+              f"{fell['n']:6d} {_fmt_num(fell.get('fwd_4h_atr'))} "
+              f"{_fmt_num(fell.get('fwd_24h_atr'))}")
+
+
+# ── 6. invariant checks ──────────────────────────────────────────────
+
+CHECK_NAMES = ("n_trades", "primary_tf", "confirm_roles", "eff_weight",
+               "wscore", "composite", "regime_mult", "bars")
+
+
+def _ts_key(value) -> str:
+    try:
+        stamp = pd.Timestamp(value)
+        if stamp.tzinfo is None:
+            stamp = stamp.tz_localize("UTC")
+        return stamp.tz_convert("UTC").isoformat()
+    except Exception:
+        return str(value)
+
+
+def check_run(run: dict) -> List[str]:
+    """Re-derive the record's own arithmetic. Empty list = everything held."""
+    fails: List[str] = []
+    manifest = run.get("manifest") or {}
+    trades = run.get("trades")
+    indicators = run.get("indicators")
+    bars = run.get("bars")
+    trades = pd.DataFrame() if trades is None else trades
+    indicators = pd.DataFrame() if indicators is None else indicators
+
+    want = manifest.get("n_trades")
+    if want is not None and int(want) != len(trades):
+        fails.append(f"n_trades: manifest says {want}, trades.parquet has "
+                     f"{len(trades)}")
+
+    if "primary_tf" in trades.columns:
+        blank = trades["primary_tf"].isna() | (
+            trades["primary_tf"].astype(str).str.strip() == "")
+        if blank.any():
+            fails.append(f"primary_tf: {int(blank.sum())} trades have no "
+                         "primary timeframe")
+    elif len(trades):
+        fails.append("primary_tf: column missing from trades")
+
+    for role in ("c1", "c2"):
+        column = f"{role}_tf"
+        if column not in trades.columns or not len(trades):
+            continue
+        present = trades[column].notna() & (
+            trades[column].astype(str).str.strip() != "")
+        share = present.mean()
+        if share <= 0.99:
+            fails.append(f"confirm_roles: {role} present on "
+                         f"{int(present.sum())}/{len(trades)} trades "
+                         f"({100 * share:.1f}%)")
+
+    if not indicators.empty:
+        base = pd.to_numeric(indicators["base_weight"], errors="coerce")
+        mult = pd.to_numeric(indicators["regime_mult"], errors="coerce")
+        eff = pd.to_numeric(indicators["eff_weight"], errors="coerce")
+        bad = (base * mult - eff).abs() > 1e-9
+        if bad.any():
+            fails.append(f"eff_weight: {int(bad.sum())} rows where "
+                         "base_weight x regime_mult != eff_weight")
+        signal = pd.to_numeric(indicators["signal"], errors="coerce")
+        conf = pd.to_numeric(indicators["confidence"], errors="coerce")
+        ws = pd.to_numeric(indicators["wscore"], errors="coerce")
+        bad = (signal * eff * conf / 100.0 - ws).abs() > 1e-9
+        if bad.any():
+            fails.append(f"wscore: {int(bad.sum())} rows where "
+                         "signal x eff_weight x confidence/100 != wscore")
+
+        offenders: List[str] = []
+        for (tid, role), group in indicators.groupby(["trade_id", "tf_role"]):
+            column = f"{role}_composite"
+            if column not in trades.columns:
+                continue
+            match = trades[trades["trade_id"] == tid]
+            if match.empty:
+                continue
+            want_score = match.iloc[0][column]
+            if pd.isna(want_score):
+                continue
+            got, _ = _composite(group["eff_weight"].tolist(),
+                                group["wscore"].tolist())
+            if abs(float(got) - float(want_score)) > 1e-6:
+                offenders.append(f"trade {tid} {role}: rows give {got:.6f}, "
+                                 f"record says {float(want_score):.6f}")
+        if offenders:
+            fails.append(f"composite: {len(offenders)} trade/role composites "
+                         f"disagree with their rows — " +
+                         "; ".join(offenders[:5]))
+
+        table = manifest.get("regime_weight_table") or {}
+        if table and "regime" in indicators.columns:
+            bad_pairs: List[str] = []
+            pairs = indicators[["regime", "indicator",
+                                "regime_mult"]].drop_duplicates()
+            for _, row in pairs.iterrows():
+                want_mult = float(table.get(str(row["regime"]), {}).get(
+                    str(row["indicator"]), 1.0))
+                if abs(float(row["regime_mult"]) - want_mult) > 1e-9:
+                    bad_pairs.append(f"{row['regime']}/{row['indicator']}: "
+                                     f"{row['regime_mult']} != {want_mult}")
+            if bad_pairs:
+                fails.append("regime_mult: " + "; ".join(bad_pairs[:5]))
+
+    if bars is not None and not bars.empty:
+        n_candidates = manifest.get("n_candidates")
+        n_trades = manifest.get("n_trades", len(trades))
+        if "has_setup" in bars.columns and n_candidates is not None:
+            setups = int(pd.to_numeric(bars["has_setup"],
+                                       errors="coerce").fillna(0).sum())
+            if not (setups >= int(n_candidates) >= int(n_trades)):
+                fails.append(f"bars: has_setup={setups} >= n_candidates="
+                             f"{n_candidates} >= n_trades={n_trades} does not "
+                             "hold")
+        n_analyzed = (manifest.get("bars") or {}).get("n_analyzed")
+        if n_analyzed is not None and int(n_analyzed) != len(bars):
+            fails.append(f"bars: manifest n_analyzed={n_analyzed}, "
+                         f"bars.parquet has {len(bars)} rows")
+        if "bar_ts" in bars.columns and "bar_ts" in trades.columns:
+            known = {_ts_key(v) for v in bars["bar_ts"]}
+            missing = [t for t in trades["bar_ts"]
+                       if _ts_key(t) not in known]
+            if missing:
+                fails.append(f"bars: {len(missing)} trade bar_ts values are "
+                             f"not in bars.parquet (e.g. {missing[0]})")
+    return fails
+
+
+def print_check(fails: Sequence[str]) -> None:
+    print(f"\n{'─' * 74}\n  CHECK — does the record agree with itself?"
+          f"\n{'─' * 74}")
+    for name in CHECK_NAMES:
+        hits = [f for f in fails if f.startswith(name + ":")]
+        if hits:
+            for message in hits:
+                print(f"  FAIL  {message}")
+        else:
+            print(f"  PASS  {name}")
+    extra = [f for f in fails
+             if not any(f.startswith(n + ":") for n in CHECK_NAMES)]
+    for message in extra:
+        print(f"  FAIL  {message}")
+    print("\n  " + ("all invariants held" if not fails
+                    else f"{len(fails)} invariant(s) broken"))
+
+
+# ── run header ───────────────────────────────────────────────────────
+
+
+def print_run_header(manifest: dict, path: str) -> None:
+    window = manifest.get("window")
+    if isinstance(window, dict):
+        window = f"{window.get('start')} -> {window.get('end')}"
+    confirm = manifest.get("confirmation_tfs")
+    if isinstance(confirm, (list, tuple)):
+        confirm = ",".join(str(c) for c in confirm)
+    print(f"\n{'═' * 74}")
+    print(f"  RUN {path}")
+    print(f"{'═' * 74}")
+    print(f"  primary {manifest.get('primary_tf', '?')}  "
+          f"confirm {confirm or '?'}  window {window or '?'}")
+    print(f"  trades {manifest.get('n_trades', '?')}  "
+          f"candidates {manifest.get('n_candidates', '?')}  "
+          f"analyzed bars "
+          f"{(manifest.get('bars') or {}).get('n_analyzed', '?')}  "
+          f"spread {manifest.get('spread', '?')}")
+    if manifest.get("book_csv_md5"):
+        print(f"  book md5 {manifest['book_csv_md5']}")
+
+
+# ── main ─────────────────────────────────────────────────────────────
+
+
+def main(argv: Optional[Sequence[str]] = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[1])
+    parser.add_argument("dirs", nargs="+", metavar="DIR",
+                        help="replay-record folder(s) written by "
+                             "backtest/replay_record.py")
+    parser.add_argument("--what-if", action="store_true",
+                        help="drop / re-weight each indicator (in-book only)")
+    parser.add_argument("--bars", action="store_true",
+                        help="bar-level lenses (needs bars.parquet)")
+    parser.add_argument("--check", action="store_true",
+                        help="verify the record's own arithmetic")
+    parser.add_argument("--min-n", type=int, default=20,
+                        help="hide buckets with fewer trades than this")
+    args = parser.parse_args(argv)
+
+    try:
+        record = _load_record_module()
+    except ImportError as exc:
+        print(str(exc))
+        return 2
+
+    failed = False
+    for path in args.dirs:
+        if not os.path.isdir(path):
+            print(f"not a directory: {path}")
+            failed = True
+            continue
+        try:
+            run = record.load_run(path)
+        except Exception as exc:
+            print(f"could not read {path}: {exc}")
+            failed = True
+            continue
+        manifest = run.get("manifest") or {}
+        trades = run.get("trades")
+        indicators = run.get("indicators")
+        bars = run.get("bars")
+        print_run_header(manifest, path)
+        if trades is None or trades.empty:
+            print("  (no trades in this run)")
+            continue
+
+        print_indicator_scorecard(
+            indicator_scorecard(trades, indicators, min_n=args.min_n),
+            contribution_corr(trades, indicators))
+        print_scorecard_by_regime(
+            scorecard_by_regime(trades, indicators, min_n=args.min_n),
+            min_n=args.min_n)
+        print_calendar_buckets(calendar_buckets(trades))
+
+        if args.what_if:
+            print_what_if(what_if_grid(trades, indicators))
+
+        if args.bars:
+            if bars is None or bars.empty:
+                print("\n  --bars: this run has no bars.parquet")
+            else:
+                print_bars_indicator_edge(bars_indicator_edge(bars))
+                print_bars_what_if(
+                    [bars_what_if(bars, drop=name)
+                     for name in _indicator_names(indicators)])
+                print(f"\n  {WHATIF_HEADER}")
+
+        if args.check:
+            fails = check_run(run)
+            print_check(fails)
+            failed = failed or bool(fails)
+
+    return 1 if failed else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

backtest/research/ai_signal_enrichment_2026-08-15.md

@@ -0,0 +1,116 @@
+# AI signal enrichment — analysis + build record (2026-08-15)
+
+Owner questions:
+1. If the AI panel also received indicator values and (limited) candle data, would the system be more accurate?
+2. If AI reviewed every candle alongside the local score system, would prediction accuracy justify the token burn — or is there a middle ground?
+
+## What the panel already received (pre-change baseline)
+
+The four personas (`agent_panel.py`) already got a *distilled* picture:
+per-indicator signal+confidence, MTF signals/scores, key levels,
+exhaustion count, full trade-setup math, warnings, headlines/events/
+sentiment. Missing: raw indicator **values** and any **candle** data.
+
+Cost guardrails already in place: 5 requests/panel, `max_panels_per_day`
+20 (≤100 requests/day), 3600 s reuse-TTL, per-bar exact cache, shared
+500 req/day DeepSeek cap, static prompts on the ~50× prefix cache.
+
+## Evidence used
+
+- **LLM is strong at text, weak at raw numerics** (measured in-repo,
+  `llm_news_eval_2026-08-06.md`): headline sentiment 100% (78/78) vs
+  keyword scorer 31%; but ~11% of event-direction calls misread close
+  numeric comparisons until code pre-computed the comparison
+  (`vs_forecast`) and the model only mapped it to doctrine.
+- **The book any filter would act on** (M5 canonical,
+  `oanda_m5_12mo_065_nopb_20260814.csv`): 577 trades, +106.82R,
+  PF 1.51, win rate 52.9%; A-grade 59.5%, B-grade 51.4% (+67.8R of the
+  total), C-grade 43.8% (−0.23R, 16 trades). Average edge ≈ 0.185R/trade.
+
+## Answer 1 — enrich the existing panel? YES (cheap, safe)
+
+- A compact candle is ~15–25 tokens; 20 primary-TF + 10 confirm-TF bars
+  ≈ 500–1,500 tokens per persona call → cents/day at the 20-panel cap.
+  **Tokens are not the constraint; the request cap is.**
+- Measured after the build (dry-test wire log): enriched technical
+  payload with 20 candles ≈ **~380 tokens**; replies ≈ 25–35 tokens
+  under the new 15-word note cap.
+- Caveats: (a) the panel is advisory-only — richer payloads improve the
+  human's context, not the book's P&L; (b) raw prices are the model's
+  weakest input, so candles are sent **ATR-normalized** (code does the
+  arithmetic, LLM reads the shape) — the same pattern that fixed the
+  event-direction misreads.
+
+## Answer 2 — AI on every candle? NO
+
+- **Feasibility**: M5+M15+H1+H4 ≈ 414 bars/day → one call/bar nearly
+  exhausts the shared 500/day cap (news needs it too); a panel per bar
+  ≈ 2,070/day = 4× over. Most of that spend reviews HOLD bars.
+- **Parity (rule 4)**: an AI gate cannot be replayed by the backtester
+  (news off; no recorded verdicts) — every pinned book becomes
+  non-comparable, and model-version drift silently mutates the strategy.
+- **Edge**: the engine already compresses 380 bars into 9 tuned
+  indicators + regime + MTF validated over 5 years; an LLM's generic
+  prior on raw candles won't beat fitted thresholds, and the margin it
+  would filter is ~0.185R/trade. B-grade holds +67.8R at 51.4% win —
+  a blunt veto is net-negative.
+
+## The middle ground (adopted)
+
+1. **Enrich payloads** (built today, see below) — near-zero cost, no
+   rule-1 exposure.
+2. **Log panel verdicts to signals.db** (built today) — creates the
+   "does OPPOSE predict losers?" dataset with zero behavior change.
+3. **After 4–8 weeks**: join logged verdicts to outcomes; simulate an
+   OPPOSE-veto against the pinned book offline. Only then consider any
+   gating, as a full rule-1 revalidation.
+
+## Build record (2026-08-15, owner-approved)
+
+- `agent_panel.py`
+  - Technical persona payload now carries indicator **values** (units
+    legend added to the prompt) and **ATR-normalized candles** per TF:
+    `{"ref_close", "atr", "bars": [[o,h,l,c]…]}` oldest-first, each price
+    as `(price − ref_close) / atr`, 1 decimal. Window: 20 bars primary /
+    10 per confirmation TF.
+  - New config keys (validated in `app_config.py`, in
+    `config.json.example`): `candles_enabled` (default true),
+    `candle_bars_primary` (20), `candle_bars_confirm` (10).
+  - Panel-private provider singleton (`_dp`, patchable in tests);
+    TTL cache keeps fetches near-free; any failure → candles silently
+    omitted, panel unaffected.
+  - Prompt hardening: training data declared stale and OFF-LIMITS;
+    "if a fact is not in the JSON, treat it as unknown"; thin evidence
+    → neutral < 30 (unchanged); note cap 25→**15 words**, rationale
+    40→**30 words** (output-token savings).
+  - `attach_panel` now calls `signal_log.log_panel_update(report)` after
+    every successful attach (fresh, exact-cache, and reuse-TTL paths).
+- `signal_log.py`
+  - `log()` returns the rowid; `log_signal` stashes it on the report as
+    `_signal_log_rowid`; new `log_panel_update()` fills the 4 panel
+    columns on that row post-hoc. No rowid (backtest replay, logging
+    disabled, other process) → silent no-op. Fail-safe contract kept.
+- Tests: 381 → **388 passed** (7 new: candle normalization math, payload
+  values/candles, context windows + ATR gating + disable flag, attach
+  injection (technical persona only), prompt-contract assertions,
+  panel row update, rowid hand-off + no-op paths). Existing attach tests
+  stub `ap._dp` to stay off the network.
+- **Live dry test** (`panel_payload_drytest_2026-08-15.py` — script removed 2026-09-06, raw replies kept in the `.jsonl` — fabricated
+  numbers only, 13 real DeepSeek requests total): **7/7 probes pass** —
+  coherent-bullish → support@70; bearish-vs-BUY → oppose@85; genuinely
+  mixed → neutral@30; empty payload → neutral@20 (contract); no invented
+  numbers in any note; all notes ≤ 15 words; synthesizer AGREE with a
+  21-word rationale. Raw wire log:
+  `backtest/research/panel_payload_drytest_2026-08-15.jsonl`.
+  - Notable: an early P3 variant with H1 SELL against a BUY brief got
+    oppose@60 — *correct* per the prompt (higher-TF disagreement is a
+    listed conflict), so the probe was tightened to a genuinely mixed
+    scenario. Documented so nobody "fixes" the prompt for it later.
+
+## What this deliberately did NOT change
+
+- No gate reads panel fields; the panel remains advisory-only.
+- No backtest-path change (`analyze_from_dataframes` untouched; parity
+  test still locks panel fields empty in replay).
+- No verdict-based filtering — that decision waits for the signals.db
+  dataset (4–8 weeks) and an offline veto replay against the pinned book.

backtest/research/cycle_2026-09-10.md

@@ -0,0 +1,287 @@
+# Improvement cycle 1 — 2026-09-10 (HEAD baseline, robustness challenge, seeding parity)
+
+## Purpose
+
+First cycle under the owner's standing brief (evidence-driven incremental improvement of
+the existing XAU/USD system). No new strategy idea is tested. The cycle (1) re-establishes
+the baseline at HEAD and proves the harness reproduces the pinned books, (2) challenges the
+current edge — cost sensitivity, concentration, and the parameter neighbourhood of the M15
+RSI × 0.5 change shipped 2026-09-09, (3) reruns the one parked candidate with prior rule-1
+evidence (H1 divergence off) through the production vehicle it now has, and (4) measures
+and fixes a live-vs-backtest indicator-seeding gap (H4 fetches 380 bars, D1 300, while the
+backtester seeds from its whole cached history). Everything under "Pre-registered" was
+written before the corresponding run.
+
+## Harness
+
+`venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --spread 0.65 --workers 12
+--primary-tf {5min|15min|1h|4h}`, pair-factory config (no `--quality/--strategies/
+--confirm-tfs`). Windows: **W3** 2023-09-05→2026-09-05 (halves at 2025-03-05; last-12-months
+slice from 2025-09-05 reported descriptively), **W2** 2024-09-05→2026-09-05 (halves at
+2025-09-05), **OOS** 2024-06-01→2025-03-30. Code: git `2ac7301` (branch `sql-integration`).
+Backend: NumPy CPU (CuPy imports but has no CUDA toolkit on this box). Bars: cached
+`backtest/cache/oanda_XAU_USD_{5min,15min,1h,4h}_20230705_20260905.parquet`,
+`_1d_20221110_20260905`, `_{5min,15min,1h}_20240705_20260905`, `_{…}_20240331_20250330`.
+Outputs: `backtest/research/cycle_2026-09-10/` (gitignored). Every candidate is compared
+with a CONTROL produced on this harness in this session. `--set
+technical.regime_weights_override=<JSON>` replaces the WHOLE table, so each candidate
+passes the complete table with one indicator's multipliers changed.
+
+## Pre-registered rules (written 2026-09-10 before any run)
+
+### Step 0 — reproducibility gate
+The HEAD factory run must reproduce the ledger's MD5s: M5 W2
+`29120a1ee32979431d062f210d41a9d1`, M15 W2 `d185f099482a07f29502cb5a371a9bbf`, M15 OOS
+`6f8f24b49597ced28f50a19e08c3472f`. A mismatch stops the cycle until explained.
+
+### Step 1 — diagnostics (descriptive; flags fixed in advance)
+- Cost ladder $0.65 / 0.80 / 1.00 / 1.30 on W3 per TF. Red flag for a TF: at $0.80 PF < 1.3
+  or either W3 half ≤ 0.
+- Concentration/stability: share of total R in the top 5% of trades, % of calendar months
+  positive, longest losing streak, max DD and R per calendar year. No action threshold.
+- Bucket screen over the recorded W3 trades (direction, quality, regime, strategy,
+  entry bar-close hour, weekday, `us_holiday`, `ev_entry_within_30m_hi_usd`,
+  `ev_hi_usd_during_hold`, year/quarter; plus M15/H4 SELL-in-transitioning and holiday
+  trades). A bucket becomes a cycle-2 candidate ONLY if n ≥ 30, R < 0 in each of the three
+  12-month slices AND in OOS, and a mechanism can be stated. No gate is built this cycle.
+
+### C1 — H1 divergence multiplier → 0 in every regime (H1-only)
+Vehicle: `--set technical.regime_weights_override=<default table with divergence 0.0 in
+all five regimes>`, primary 1h, W3 + OOS, control = the Step-0 HEAD run.
+PASS only if ALL: (a) ΔR ≥ +5R on W3; (b) ΔR > 0 in both W3 halves; (c) PF ≥ 1.3 and not
+more than 0.05 below control; (d) max DD ≤ control + 2R; (e) trade count within ±25%;
+(f) OOS ΔR ≥ 0. Last-12-months slice reported, not a clause. No rescue, no regime-family
+fine-tuning after seeing the result. A pass is a shipping PROPOSAL for the owner.
+
+### C2 — M15 RSI multiplier neighbourhood (stability of the live change)
+Ladder k ∈ {0, 0.25, 0.5 (HEAD), 0.75, 1.0 (pre-2026-09-09), 1.5}, full table with
+rsi × k, on W2, OOS and W3. Reading, with G(k) = R(k) − R(1.0) on W2: **stable plateau** if
+G(0.25) ≥ 0.5·G(0.5) and G(0.75) ≥ 0.5·G(0.5) and the OOS deltas at 0.25 and 0.75 are ≥ 0;
+**spike** if either neighbour retains < 50% of G(0.5). Outcome is a report to the owner;
+no different k is proposed whatever the result.
+
+### Step 3 — seeding-depth parity
+3a: on the deepest cached H4/D1 frames (H1 as control), sampled bars over the last two
+years, compare `TechnicalEngine.analyze()` on the trailing N bars with the full-history
+result; `MIN_SEED_BARS` = smallest N with zero indicator-signal flips on both H4 and D1.
+3b: live per-TF fetch floor raised to that depth (no book changes; the backtester is the
+reference). 3c: backtester slow-TF pad raised to the same depth, shipped only if the M5
+and M15 W3 books stay byte-identical and the H1/H4 differences are confined to the
+window-start seeding zone; the H1/H4 re-pin is the owner's decision.
+
+## Results
+
+(filled in as runs complete)
+
+### Step 0 — HEAD baseline (W3, $0.65, factory config; recorded with `--record`)
+
+| TF | trades | total R | WR | PF | max DD | half 1 (→2025-03-05) | half 2 | last 12 mo | SELL bucket | book MD5 |
+|---|---|---|---|---|---|---|---|---|---|---|
+| M5 | 870 | +153.64 | 53.3% | 1.50 | 10.1 | 136 / +16.73 / PF 1.36 | 734 / +136.91 / 1.52 | 593 / +119.65 / 1.57 | 100 / +8.95 / 1.25 | `c2722f9830528658b53854f6bae61b8c` |
+| M15 | 1273 | +186.61 | 59.4% | 1.39 | 19.3 | 556 / +75.87 / 1.37 | 717 / +110.74 / 1.41 | 474 / +81.04 / 1.47 | 404 / −15.18 / 0.91 | `b2cfa9e2e6cd3997f4d5ca9b49cab90e` |
+| H1 | 253 | +70.02 | 64.4% | 1.98 | 4.4 | 115 / +26.26 / 1.79 | 138 / +43.76 / 2.15 | 89 / +24.62 / 1.96 | 0 (all shorts blocked) | `b99111132d9ecb80c2998ed5aee2f3ea` |
+| H4 | 114 | +21.68 | 57.9% | 1.59 | 5.4 | 52 / +4.91 / 1.26 | 62 / +16.77 / 1.94 | 46 / +12.34 / 1.79 | 20 / −0.97 / 0.89 | `4508e7a3402e008ba1d93805094cd981` |
+
+These equal the numbers CLAUDE.md records for HEAD (M5 870/+153.6/1.50, M15 1273/+186.61/1.39,
+H1 253/+70.0/1.98, H4 114/+21.7/1.59). Engine / gate fingerprints from the manifests:
+M5 `f28b1a8c409d`/`bc0991492340`, M15 `536f84340ea0`/`b2920991cdb8` (new since the RSI
+override), H1 `6efb6d365f01`/`0b67d738dc15`, H4 `ed96f258b10e`/`0c3b51f063f9`.
+`replay_scorecard.py --check` on the four recorded runs: see `step0.out`.
+Per calendar year (R / PF): M5 2023 +2.4/1.47, 2024 **+1.0/1.02** (102 trades — a flat
+year), 2025 +40.5/1.34, 2026 +109.7/1.77; M15 +13.2/1.40, +52.9/1.36, +77.3/1.43,
++43.2/1.37; H1 +5.5/2.60, +14.7/1.58, +33.7/2.11, +16.1/2.35; H4 +1.2/1.28, +4.6/1.39,
++7.7/1.74, +8.2/1.78.
+
+### Step 3a — seeding-depth measurement (`seed_depth.out`)
+
+Deepest Oanda caches (H4 12,419 bars, D1 2,617, H1 45,914; 2018→2026-08-06), bars sampled
+every 12th (H4, 267 bars) / 2nd (D1, 313) / 48th (H1, 247) over the last two years;
+`TechnicalEngine.analyze()` on the trailing N bars vs on the full history; identical
+results with raw frames and with frames cleaned by `validate_and_clean` (the frame-wide
+σ of the outlier filter did not add a single flip).
+
+| TF | N = live depth today | bars with ≥1 indicator flip | which | max composite Δ | N = 600 … 5000 |
+|---|---|---|---|---|---|
+| H4 | 380 | 2 / 267 | ema_trend | 4.66 | 0 flips at every depth |
+| D1 | 300 | 1 / 313 | ema_trend | 9.47 | 0 flips at every depth |
+| H1 | 1460 (control) | 0 / 247 | — | 0.00 | 0 |
+
+Only the 200-bar EMA trend reading is affected, as the `adjust=False` seeding predicts;
+composite-signal and regime never flipped in the sample. Pre-registered pick = smallest
+zero-flip depth = 600; shipped floor = **1000** (seed weight e^-10, a ×40 margin on a
+sample of a few hundred bars; costs nothing — one request either way). The 2026-08-04
+audit's "~2% of H4 bars" was measured on a different sample; this sample says 0.75%
+(H4) and 0.3% (D1) of bars, with composite errors up to 9.5 points on the ones that flip.
+
+### Step 1 — concentration, stability, bucket screen (W3 recorded books, `diag_w3_head.txt`)
+
+| TF | top 5% of trades carry | months positive | longest losing streak |
+|---|---|---|---|
+| M5 | 94% of total R (44 trades, +144R of +154R) | 62% (23/37) | 8 |
+| M15 | 96% (64 trades, +179R of +187R) | 68% (25/37) | 9 |
+| H1 | 48% (13 trades) | 71% (25/35) | 4 |
+| H4 | 80% (6 trades) | 50% (15/30) | 6 |
+
+The intraday edge is a fat right tail: trailing exits let a few dozen runners pay for
+everything else (M5 2024 was flat over 102 trades). H1 is the only timeframe whose edge is
+spread across its book.
+
+Bucket screen (pre-registered criterion: n ≥ 30 and R < 0 in each 12-month slice y1/y2/y3
+AND in the OOS slice of the same book). Exactly one bucket qualifies (besides the trivial
+`exit_reason = SL`):
+- **M15 SELL grade A**: 230 trades, −14.4R, PF 0.86, y1 −7.4 (67) / y2 −0.4 (80) /
+  y3 −6.5 (83) / OOS −6.9 (62). Context: M15 SELL overall 404 / −15.2R / PF 0.91 (SELL
+  max DD 36R on its own curve); SELL|B 160 / −5.5R (y3 +4.6 — not qualifying);
+  SELL|A+ 14 / +4.7R; SELL|trending_down 301 / −10.9R (y2 +0.9 — not qualifying).
+  This is the same bucket the 2026-09-03 campaign tested as `sell_min_quality=["B"]`
+  (allow only B-grade M15 shorts) and REJECTED: 12mo +0.7R, 5-yr −8.9R with max DD 35.8 →
+  41.4R, OOS +8.3R — A-grade M15 shorts paid in 2021–2023. Under the owner's ≤3-year
+  window they lose in every slice. Recorded as the ONE cycle-2 candidate from this screen
+  (M15 `sell_min_quality` excluding "A" via the factory), to be run against a control with
+  the 3-yr DD clause; the 5-yr verdict is the fact on the other side of the scale. Not
+  built this cycle (rule: no gate from the screen).
+- Not qualifying / no evidence: holiday-day trades are POSITIVE on M15 (27 / +10.6R) and
+  small everywhere; entries within 30 min after a high-impact USD release are positive
+  (M15 114 / +9.3R); M15 SELL-in-transitioning (the gate-6c bucket) 75 / −2.2R with y2
+  and y3 positive; M15 `vol_regime = high` 291 / +4.1R / PF 1.03 (y3 +12.6); hour 7 UTC
+  −4.0R (OOS +2.4). M5/H1/H4 produced no qualifying bucket.
+
+### Step 0 — reproducibility gate: PASS
+
+| book | MD5 at HEAD | ledger MD5 | result |
+|---|---|---|---|
+| M5 W2 | `29120a1ee32979431d062f210d41a9d1` | same | equal |
+| M15 W2 | `d185f099482a07f29502cb5a371a9bbf` | same | equal |
+| M15 OOS | `6f8f24b49597ced28f50a19e08c3472f` | same | equal |
+| H1 OOS (new control, no prior MD5) | `c02eebc9066916d7637bbf5796993f87` | — | recorded |
+
+`replay_scorecard.py --check`: all eight invariants hold on every recorded W3 run.
+
+### Step 3b — live floor shipped (commit `9942736`) and verified on the live feed
+
+`BaseDataProvider.MIN_LIVE_BARS = 1000`, applied in a new pure helper
+`_live_outputsize(timeframe, lookback_days)`; requested depths now M5 5000, M15 5000,
+H1 1460 (unchanged), **H4 1000 (was 380), D1 1000 (was 300)**. The backtester never calls
+`fetch_ohlcv`, so no book changes. Tests: new Module 16 depth test, Module 21 cache test
+updated from 300 to 1000; suite **500 passed**.
+Live verification 2026-09-10 (real Oanda fetch, same latest closed bar, engine on the
+frame cleaned exactly as live does):
+- H4 (bar 2026-09-10 04:00 UTC): 380 / 1000 / 5000 bars → identical readings
+  (composite +15.03, ranging) — no flip on today's bar.
+- D1 (bar 2026-09-09): **300 bars → ema_trend SELL, composite −9.53; 1000 and 5000 bars →
+  ema_trend NEUTRAL, composite −8.87.** The old depth was mis-reading the daily trend
+  today; the new floor agrees with the converged value.
+
+### Step 1 — cost ladder (W3, same trade lists; `--spread` only re-prices; mechanics check: the H1 $1.00 total predicted from the $0.65 book (−0.35 $/trade ÷ risk) equals the rerun to the cent)
+
+| TF | $0.65 R / PF / DD | $0.80 R / PF / DD (halves) | $1.00 R / PF / DD | $1.30 R / PF / DD | red flag ($0.80: PF < 1.3 or a half ≤ 0)? |
+|---|---|---|---|---|---|
+| M5 | +153.6 / 1.50 / 10.1 | +145.6 / 1.46 / 11.0 (+14.9 / +130.7) | +134.9 / 1.42 / 12.2 | +118.8 / 1.36 / 14.1 | no |
+| M15 | +186.6 / 1.39 / 19.3 | +176.4 / 1.37 / 19.9 (+70.0 / +106.4) | +162.8 / 1.34 / 20.8 | +142.4 / 1.29 / 22.6 | no (PF dips under 1.3 only at $1.30) |
+| H1 | +70.0 / 1.98 / 4.4 | +68.9 / 1.96 / 4.5 (+25.6 / +43.2) | +67.3 / 1.93 / 4.5 | +65.0 / 1.89 / 4.7 | no |
+| H4 | +21.7 / 1.59 / 5.4 | +21.3 / 1.57 / 5.5 (+4.7 / +16.6) | +20.8 / 1.56 / 5.7 | +20.1 / 1.53 / 6.0 | no |
+
+Doubling the assumed round-trip cost ($0.65 → $1.30) costs M5 23% and M15 24% of total R
+and H1/H4 about 7%; the H1 book is the least cost-sensitive because its stops are the
+widest in dollars. The M5 first half is thin at every price (+8.7R at $1.30 over 136
+trades), which is the 2024 flat year again, not a cost effect.
+
+### C1 — H1 divergence multiplier → 0 in every regime: **FAIL (a, b, c) — closed**
+
+| book | trades | total R | ΔR | half 1 | half 2 | PF | max DD | last 12 mo | OOS |
+|---|---|---|---|---|---|---|---|---|---|
+| H1 control (HEAD, W3) | 253 | +70.02 | | +26.26 | +43.76 | 1.98 | 4.4 | +24.62 (89) | 78 / +16.68 / PF 1.74 |
+| H1 divergence 0 (W3) | 280 | +71.01 | **+0.99** | +28.70 (+2.4) | +42.30 (**−1.5**) | **1.89** | 4.8 | +23.09 (102) | 88 / +21.91 / PF 1.92 (+5.2) |
+
+Clauses: (a) ΔR +1.0 < +5 FAIL; (b) second half negative FAIL; (c) PF 1.98 → 1.89, drop
+0.09 > 0.05 FAIL; (d) DD +0.4 pass; (e) trades +10.7% pass; (f) OOS +5.2 pass. Calendar
+years: 2024 +14.7 → +19.1, 2025 +33.7 → +34.1, 2026 +16.1 → +12.4 — the gain sits in
+2024, the most recent year gives it back. The OOS candidate book (88 / +21.91 / PF 1.92)
+is exactly the 2026-09-03 campaign's `b12_h1_nosell_nodiv_oos` under
+`disabled_indicators` — so a 0.0 regime multiplier is mechanically the same as disabling
+the indicator for the composite (engine skips only base weight 0, but an effective weight
+of 0 adds nothing to score, weight sum or confidence); the only residual difference is
+display (reasons text still lists the vote). The parked candidate's earlier evidence
+(5-yr +9.6R over no-shorts) was carried by 2021–2023; on the owner's 3-year window it is
+noise with a worse profit factor. Not shipped; do not re-propose without new years.
+
+### C2 — M15 RSI multiplier neighbourhood: **STABLE PLATEAU** (the 2026-09-09 ship stands)
+
+Full regime table with the RSI multiplier × k (k = 1.0 is the pre-2026-09-09 table,
+k = 0.5 is HEAD; every other point is a `--set` run on the same harness).
+
+| k | W2 trades / R / PF / DD (halves) | G(k) = R − R(1.0) on W2 | OOS trades / R / PF / DD | ΔOOS | W3 trades / R / PF / DD (halves) |
+|---|---|---|---|---|---|
+| 0 (RSI off) | 1091 / +145.46 / 1.35 / 18.5 (+74.1 / +71.3) | +27.8 | 418 / +51.89 / 1.34 / 19.5 | +11.3 | 1490 / +188.14 / 1.33 / 22.7 (+69.3 / +118.8) |
+| 0.25 | 1008 / +139.93 / 1.37 / 20.1 (+66.1 / +73.9) | +22.3 (78% of G(0.5)) | 390 / +59.55 / 1.44 / 17.3 | +19.0 | 1374 / +187.58 / 1.36 / 21.4 (+77.7 / +109.9) |
+| **0.5 (HEAD)** | 931 / +146.14 / 1.43 / 16.1 (+66.1 / +80.0) | **+28.5** | 355 / +60.67 / 1.49 / 17.1 | +20.1 | 1273 / +186.61 / 1.39 / 19.3 (+75.9 / +110.7) |
+| 0.75 | 854 / +137.83 / 1.45 / 14.9 (+47.6 / +90.3) | +20.2 (71%) | 326 / +55.56 / 1.49 / 15.6 | +15.0 | 1176 / +178.81 / 1.42 / 17.2 (+66.9 / +111.9) |
+| 1.0 (old) | 793 / +117.62 / 1.41 / 14.8 (+38.2 / +79.4) | 0 | 300 / +40.59 / 1.38 / 15.5 | 0 | 1092 / +151.90 / 1.38 / 15.5 (+55.7 / +96.2) |
+| 1.5 | 676 / +110.17 / 1.46 / 14.2 (+39.4 / +70.7) | −7.5 | 266 / +37.27 / 1.39 / 18.8 | −3.3 | 949 / +136.20 / 1.39 / 18.8 (+42.6 / +93.6) |
+
+Reading (rule fixed in advance): G(0.25) = 78% and G(0.75) = 71% of G(0.5), both ≥ 50%;
+OOS deltas +19.0 and +15.0, both ≥ 0 → **stable plateau, not a spike**. The response is
+monotone in k on every window (R falls as RSI's weight rises; trade count falls ~90 per
+0.25 step), PF peaks in the 0.5–0.75 band, and drawdown grows as k → 0 (RSI off has the
+same R as HEAD with PF 1.35 and DD 22.7 on W3). The shipped 0.5 sits inside a broad
+region, on its PF/DD-favourable side. Under this rule no other k is proposed.
+
+### Step 3c — backtester history pad targets the same 1000-bar seed (`_history_pad_days`)
+
+Change: the per-TF warm-up pad now targets `BaseDataProvider.MIN_LIVE_BARS` (1000 bars,
+converted to calendar days with 7/5) instead of 210 bars, so a window's first bars are
+seeded like the rest of the window and like live. Intraday pads are unchanged (62 days
+already hold ≥ 1488 bars), so M5/M15/H1 keep their cache windows; H4 now pads 234 days
+(was 62 → 372 bars) and D1 1400 days (was 299 → ~210 bars for a 200-bar EMA). New caches
+`oanda_XAU_USD_4h_20230114_20260905.parquet` (5,826 bars) and
+`_1d_20191105_20260905.parquet` (2,124 bars); the first H4 chunk (2023-01-14, a Saturday)
+is legitimately empty. The raw-path fetch for trade resolution now reuses the primary
+timeframe's own padded start (one cache file per TF, as before). Test: Module 21
+"history pad seeds slow TFs like live"; suite **501 passed**.
+
+W3 books before → after, trade-by-trade (`bookdiff.py` with the recorded runs):
+
+| TF | trades / R before | trades / R after | trades added / removed / R changed | what actually differs |
+|---|---|---|---|---|
+| H1 | 253 / +70.02 | 253 / +70.02 | 0 / 0 / 0 | ONE confirmation reading: 2023-09-20 18:00 (15 days into the window), H4 ema_trend STRONG_BUY → BUY (value 1.0 → 0.5, confidence 90 → 65); that trade's blended `score` moves 0.21; nothing else in 5,761 recorded indicator rows |
+| H4 | 114 / +21.68 | 114 / +21.68 | 0 / 0 / 0 | no signal flips; 20 D1 rows (ADX/RSI/MACD Wilder recursions) differ at the 1e-6…1e-9 level, all Oct–Dec 2023; exported floats differ by ≤ 1.7e-10 (score ≤ 1.4e-9, R ≤ 1e-15) |
+
+Both differences sit in the window-start seeding zone, exactly where the deeper seed was
+meant to act, and neither moves a trade. The canonical H1 book therefore needs NO re-pin
+(trade content identical); only the CSV bytes differ (H1 `3b45d12bade823178b267c13bc698458`
+vs `b99111132d9ecb80c2998ed5aee2f3ea`, H4 `e75adae13c839b15736f69c0114e6aa5` vs
+`4508e7a3402e008ba1d93805094cd981`), which the parity recipe must state.
+
+M5 and M15 on the patched harness: **byte-identical** to Step 0 (M5
+`c2722f9830528658b53854f6bae61b8c`, M15 `b2cfa9e2e6cd3997f4d5ca9b49cab90e`) — the pad
+change did not leak into the intraday books. Shipped (commit below); no re-pin needed.
+
+## Verdicts
+
+| item | decision | one-line reason |
+|---|---|---|
+| Step 0 reproducibility | PASS | three ledger MD5s reproduced at HEAD; scorecard invariants hold |
+| Cost ladder | no red flag | every TF keeps PF ≥ 1.3 with positive halves at $0.80; edge survives $1.00 on all four |
+| Concentration | fact recorded | M5/M15: top 5% of trades carry 94–96% of R; H1 48% |
+| Bucket screen | one cycle-2 candidate | M15 grade-A SELLs (230 / −14.4R, negative in y1, y2, y3 and OOS); 5-yr history says the opposite — run it against a control with the DD clause before believing it |
+| C1 H1 divergence → 0 | **REJECT** | ΔR +1.0 (< +5), second half −1.5R, PF 1.98 → 1.89; OOS-only gain |
+| C2 M15 RSI ladder | stable plateau | neighbours keep 78% / 71% of the shipped gain, OOS positive at both; ship stands |
+| 3a/3b live seeding floor | **ACCEPT, shipped** (`9942736`) | measured flips at 380/300 bars, none at ≥ 600; live D1 flipped on 2026-09-10 at the old depth; books untouched |
+| 3c backtester pad | **ACCEPT, shipped** | no trade moved in any book; M5/M15 byte-identical; H1/H4 differ only in window-start numerics |
+
+## What NOT to re-propose (added to the ledger's closed list)
+- H1 divergence weight 0 (any vehicle) without new years of data.
+- M15 RSI multiplier other than 0.5 on the strength of a single-point rerun; the ladder is on record.
+- Cost-driven gates: the books are not cost-fragile at $0.80–$1.00.
+
+## Shipped
+
+- `9942736` providers: `MIN_LIVE_BARS = 1000` live floor + `_live_outputsize` (Module 16
+  test; Module 21 expectation 300 → 1000). Suite 500.
+- backtester: `_history_pad_days` (62 / 234 / 1400 days for intraday / H4 / D1), raw-path
+  fetch reuses the primary's start (Module 21 test). Suite 501. Live H1/H4 scans and the
+  books now share the same seeding depth; pinned H1/H4 CSVs are trade-identical but no
+  longer byte-reproducible (float noise in exported indicator columns — same status the
+  M5/M15 pins have had since 2026-09-07).
+- No signal-core change, no docs change (nothing user-visible moved), both news gating
+  flags untouched.

backtest/research/feed_divergence_2026-08-14.md

@@ -0,0 +1,81 @@
+# Feed Divergence Study — Oanda vs Tiingo vs Twelve Data (Phase 1)
+
+**Date:** 2026-08-14
+**Test plan:** `docs/data_source_change_test_plan.md` (removed 2026-09-06, in git history; the pending Phase 4 shadow scan is noted in CLAUDE.md)
+**Tool:** `scripts/feed_divergence_study.py` (script removed 2026-09-06, in git history) (fetch_range over 365d, parquet-cached)
+**Window:** 2025-08-14 → 2026-08-14 (12 months); Oanda-vs-TD truncated at TD cache head (2026-07-23)
+
+## Headline
+
+Three mid-price feeds for the *same* XAU/USD instrument are **not** interchangeable:
+
+| Pair | mean \|ΔClose\| | p95 \|ΔClose\| | direction agree (M5/M15/H1) | bias | extra bars |
+|---|---|---|---|---|---|
+| Oanda vs **Tiingo** | ~$0.13 | ~$0.29 | 97.7% / 98.3% / 98.5% | ~0 | Tiingo +3–5% (weekend) |
+| Oanda vs **Twelve Data** | ~$3.0 | ~$11.9 | 81.0% / 88.7% / 92.7% | −$1.06 (TD high) | — |
+
+- **Oanda ↔ Tiingo are near-identical** on shared bars (~1–2% of ATR on close), but Tiingo emits ~22 hours of Sunday bars/week that Oanda omits.
+- **Oanda ↔ Twelve Data diverge massively** — ~$3 mean close gap (≈30% of M5 ATR, ~30% of M15 ATR), a systematic ~$1 bias (TD quoted ~$1 high), and 19% of M5 bars even disagree on bar *direction*. This is the documented "Oanda/TD divergence" that motivated re-pinning the books on Oanda bars.
+
+## Oanda vs Tiingo — detailed
+
+Both feeds: same bar grid, same head/tail timestamps. Tiingo simply has more bars (Sunday bars).
+
+### Coverage (12 months)
+
+| TF | Oanda bars | Tiingo bars | shared | Oanda-only | Tiingo-only |
+|---|---|---|---|---|---|
+| M5 | 70,901 | 73,241 | 70,807 (96.6%) | 94 (0.13%) | 2,434 (3.32%) |
+| M15 | 23,640 | 24,922 | 23,615 (94.7%) | 25 (0.11%) | 1,307 (5.24%) |
+| H1 | 5,912 | 6,246 | 5,909 (94.6%) | 3 (0.05%) | 337 (5.40%) |
+| H4 | 1,597 | 1,657 | 1,597 (96.4%) | 0 | 60 (3.62%) |
+
+- **Tiingo-only bars cluster exclusively on Sundays** (e.g. M15: 88/Sunday; H1: 22/Sunday; H4: 5/Sunday). Tiingo's FX endpoint resamples continuously and does **not** apply the FX weekend calendar — it emits ~22h of flat/stale Sunday bars that Oanda omits.
+- Oanda-only bars cluster on holidays (2025-11-27 Thanksgiving, 2025-12-10, 2026-06-25) — a handful of bars, negligible.
+
+### Per-bar OHLC diffs (Oanda − Tiingo, $)
+
+| TF | Close mean | Close p95 | Close max | \|ΔClose\| / ATR14 (mean / p95) |
+|---|---|---|---|---|
+| M5 | $0.123 | $0.285 | $26.10 | 2.1% / 5.0% |
+| M15 | $0.128 | $0.290 | $26.10 | 1.3% / 2.9% |
+| H1 | $0.145 | $0.335 | $26.10 | 0.7% / 1.6% |
+| H4 | $0.168 | $0.340 | $17.53 | 0.4% / 0.8% |
+
+- Bias is ~zero on every TF (no systematic one-feed-high).
+- Tail outliers (p99/max up to $26–88) occur during fast moves — the two feeds' bar extremes disagree on the exact tick, as expected.
+- **Direction agreement is 97.7–98.4%**, return correlation 0.995–0.9997.
+- Divergence is concentrated at the Friday-close / Sunday-reopen boundary (UTC hours 20–22; mean |ΔClose| ~2–4× the 08–19h baseline).
+
+### Phase 1 pass/fail vs plan thresholds
+
+| Criterion | Plan threshold | Result | Verdict |
+|---|---|---|---|
+| timestamp overlap | ≥ 99.5% | 94.6–96.6% | **FAIL** (Tiingo weekend bars) |
+| Close p95 \|Δ\| | ≤ 0.01% of price | ~0.0067% | PASS |
+| no systematic bias | mean Δ ≈ 0 | ~0 | PASS |
+
+The overlap "failure" is a *coverage* difference, not a data-quality one: the shared bars agree tightly, but Tiingo's extra Sunday bars mean the feeds have a different bar *count*. The signal pipeline's own `_is_forex_weekend` filter (NY-anchored) gates those Sunday bars, so live impact depends on whether any code trusts the feed's raw bar set directly.
+
+## Oanda vs Twelve Data — detailed (M5/M15/H1, cached TD parquets)
+
+| TF | common | mean \|ΔClose\| | median \|ΔClose\| | p95 | max | bias (TD−Oanda) | dir agree |
+|---|---|---|---|---|---|---|---|
+| M5 | 66,246 | $2.999 | $1.118 | $11.934 | $101.85 | +1.05 | 80.96% |
+| M15 | 22,090 | $2.951 | $1.051 | $11.848 | $101.85 | +1.08 | 88.71% |
+| H1 | 5,526 | $2.902 | $0.985 | $11.908 | $70.94 | +1.06 | 92.67% |
+
+- The ~$3 mean / ~$1 median / ~$12 p95 gap is **~23× larger** than the Oanda–Tiingo gap, and the ~$1 bias is systematic (Twelve Data quotes gold ~$1 high vs Oanda's true mid).
+- M5 direction agreement of **81%** means one bar in five disagrees on direction — M5 signals on TD bars are effectively a *different instrument* than on Oanda bars.
+- This matches the prior Oanda/TD M15 study of 2026-07-24 (`oanda_td_divergence.py`, removed 2026-09-06 as superseded by this report; mean |ΔClose| ~$3.3) and confirms the standing rule: **pinned Twelve-Data books are not comparable to Oanda live signals without a re-pin.**
+
+## Conclusions for the data-source switch
+
+1. **Oanda → Tiingo (or back) is signal-safe in magnitude** (price diffs ~1–2% of ATR, 98% direction agreement), *provided* the weekend-bar coverage difference is handled — Tiingo's Sunday bars must be gated by the aggregator's NY-anchored weekend filter, not by raw bar count.
+2. **Oanda → Twelve Data is a recalibration, not a formality.** Phase 2/3 must quantify the actual signal/book delta; the $3/$1 bias alone guarantees a large fraction of marginal bars flip.
+3. Re-pinning the books on Oanda bars (Phase 3) remains mandatory before any TD-pinned baseline is treated as a live expectation.
+
+## Next steps
+
+- Phase 2: signal-level replay parity (`analyze_from_dataframes` on both feeds) to turn price-diff stats into flip rates.
+- Phase 3: backtest re-pin on Oanda bars at $0.30 and $0.65.

backtest/research/fixedtp_2026-08-05.md

@@ -0,0 +1,123 @@
+# Fixed-RR hard-TP vs shipped trailing exit — research campaign 2026-08-05
+
+**Verdict: DO NOT SHIP. The trailing exit stays.** The fixed 2R target beats the trail on
+total R in every 2024+ window but the edge is a trend-regime artifact: it inverts in the
+2021–2023 ranging years, doubles drawdown, cuts win rate 10–19 points, and doubles hold
+times (amplifying the accepted live one-position divergence). Details below.
+
+## Setup
+
+- Code: HEAD `fc5ce85` + the `--fixed-rr-tp` research flag (uncommitted at run time).
+  The flag replaces every trade's TP with `mult × |entry − sl|` AFTER re-anchoring and
+  disables trailing (run-level and per-trade PB params) — entries, stops, gates, and the
+  cooldown pass are untouched, so both arms replay **trade-identical books** (verified:
+  paired join on entry_time+direction matches 1:1 in every window). Only the exit policy
+  differs → per-trade paired comparison.
+- Provider: Oanda (cache-warm; keys `oanda_XAU_USD_{5min,15min,1h}_{20250603_20260804,
+  20240331_20250330,20210531_20260804}`). Charged cost `--spread 0.65` (real measured
+  round trip); $0.30 letter-cost books derived analytically as `r + 0.35/risk` and
+  verified exact against one real `--spread 0.30` run (max per-trade error 1.8e-15 —
+  spread never enters trigger detection, per the owner-declined 2026-08-04 decision).
+- Control sanity gate: fresh M5 12mo control reproduced the pinned HEAD reference
+  exactly — 555 trades, +98.64R, PF 1.483, maxDD −10.73R.
+- Commands (control = same without the flag):
+  `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --primary-tf {5min,15min}
+   --start 2025-08-04 --end 2026-08-04 --spread 0.65 [--fixed-rr-tp 2.0]
+   --export-csv backtest/fixedtp_*.csv`
+  OOS `--start 2024-06-01 --end 2025-03-30`; 5yr `--start 2021-08-01 --end 2026-08-04`.
+- Books pinned as `backtest/fixedtp_{m5,m15}_{12mo,oos,5yr}_{ctrl,rr20,...}.csv`
+  (per-trade indicator breakdown, agreement counts, regime, vol-regime, weekday,
+  MAE/MFE, bars-held — the full instructions_for_backtest record).
+
+## Headline (@ $0.65)
+
+| Window | Arm | n | R | PF | WR | maxDD | recov | medBars | exits |
+|---|---|---|---|---|---|---|---|---|---|
+| M5 12mo | trail | 555 | +98.64 | 1.483 | 52.4% | −10.7 | 9.2 | 28 | TRAIL 361 / SL 183 / TO 11 |
+| M5 12mo | 2R hard | 555 | +124.85 | 1.378 | 42.9% | −24.5 | 5.1 | 50 | SL 312 / TP1 210 / TO 33 |
+| M15 12mo | trail | 397 | +73.93 | 1.525 | 62.2% | −10.8 | 6.8 | 28 | TRAIL 255 / SL 135 / TO 7 |
+| M15 12mo | 2R hard | 397 | +95.95 | 1.431 | 44.3% | −11.2 | 8.6 | 39 | SL 215 / TP1 151 / TO 31 |
+| M5 OOS | trail | 86 | +9.45 | 1.332 | 53.5% | −8.2 | 1.2 | 70 | — |
+| M5 OOS | 2R hard | 86 | +18.83 | 1.407 | 45.3% | −9.7 | 1.9 | 124 | — |
+| M15 OOS | trail | 299 | +38.77 | 1.360 | 59.2% | −15.5 | 2.5 | 38 | — |
+| M15 OOS | 2R hard | 299 | +72.14 | 1.435 | 45.2% | −13.9 | 5.2 | 82 | — |
+
+Halves (12mo, split 2026-02-03): M5 rr20 +67.99/+56.86 (both positive; H2 PF 1.325 vs
+ctrl 1.503), M15 rr20 +73.74/+22.21 (both positive; ctrl H2 +7.23 — the known H2-flat).
+At $0.30 all deltas keep the same sign (e.g. M5 12mo +109.0 vs +135.2).
+
+So the **letter** of rule 1 passes for rr20 on both TFs. The 5yr decomposition is why it
+still must not ship.
+
+## Exit migration (paired, M5 12mo; same pattern everywhere)
+
+- SL→SL 183 trades: ΔR 0 (identical by construction — stops untouched).
+- TRAIL→TP1 207 trades: +248.9R → +424.5R (**+175.6R**) — winners the trail exits early.
+- TRAIL→SL 129 trades: +19.0R → −136.8R (**−155.7R**) — retraces the trail survives
+  become full stop-outs.
+- TRAIL→TIMEOUT 25: +7.4R. Net = +26.2R. The whole effect is these two big flows nearly
+  cancelling; which one wins is a regime property, not an exit-policy edge.
+- Trades with ctrl MFE ≥ 2R: 95; hard cap forfeits only +10.4R vs trail on M5 (the trail
+  already gives ~0.75R back from peak, so a 2R cap costs little *in a trend*).
+
+## Mini-grid (12mo @ $0.65) — the tell
+
+| mult | M5 R / PF / DD | M15 R / PF / DD |
+|---|---|---|
+| trail | +98.6 / 1.483 / −10.7 | +73.9 / 1.525 / −10.8 |
+| 1.0R | +52.5 / 1.205 / −12.8 | +67.1 / 1.410 / −7.3 |
+| 1.5R | +80.7 / 1.263 / −21.4 | +66.9 / 1.325 / −11.2 |
+| 2.0R | +124.9 / 1.378 / −24.5 | +96.0 / 1.431 / −11.2 |
+| 2.5R | +147.7 / 1.426 / −23.6 | +110.4 / 1.469 / −15.9 |
+| 3.0R | +155.8 / 1.432 / −21.6 | +135.8 / 1.562 / −17.5 |
+
+Monotone in the multiple on both TFs — "2.0" is not an optimum, it's a point on a curve
+that says *never take profit* (3R's M5 TIMEOUT bucket alone is +84.8R). That is the
+signature of a one-way trend sample, not of a superior exit rule. It also reconfirms the
+original finding that SHORT fixed targets (≤1.5R) are strictly bad (cf. RR-0.5 removal,
+`config.py:126-133`: +48R → +105R).
+
+## 5yr regime decomposition (2021-08-01 → 2026-08-04 @ $0.65) — the verdict
+
+M15 (real sample in every year): ctrl +127.97 PF 1.209 DD −35.8; rr20 +158.97 PF 1.170
+DD **−59.8**; rr30 +204.03 PF 1.203 DD **−98.3**.
+
+| Year | n | trail | 2R hard | 3R hard |
+|---|---|---|---|---|
+| 2021 | 88 | **+4.00** | −16.09 | −17.41 |
+| 2022 | 284 | −9.54 | −7.63 | −26.74 |
+| 2023 | 227 | **+7.79** | −1.71 | −5.81 |
+| 2024 | 345 | +40.44 | +64.98 | +89.14 |
+| 2025 | 405 | +49.33 | +69.39 | +104.42 |
+| 2026 | 229 | +35.95 | +50.04 | +60.43 |
+
+M5 5yr mirrors it (ctrl +128.4 PF 1.387 DD −12.8 vs rr20 +160.2 PF 1.306 DD −24.7;
+2022: −1.1 vs −5.2), though pre-2024 M5 samples are thin (12–60 trades/yr).
+
+The hard target's entire outperformance lives in the 2024–2026 gold trend. In ranging
+years it flips sign exactly where the trail stays positive (M15 2021, 2023), and the
+further the target, the worse the ranging-year bleed and the deeper the DD. The 12mo,
+halves, and OOS windows all sit inside the trend era, which is why the rule-1 letter
+passes — this is precisely the within-regime-split trap rule 1's OOS clause exists to
+catch, one level up.
+
+## Additional non-ship reasons
+
+- **Live divergence grows with hold time.** Median hold doubles (M5 28→50 bars; OOS
+  70→124). The daemon holds one position total; the backtester doesn't model that
+  blocking. Longer holds = more live signals skipped that the paper book banks.
+- **WR 52→43% (M5) / 62→44% (M15), max loss streak 6→11 (M15)** — materially harder to
+  operate and size.
+- Aggregate tables (session/day/regime/vol, streaks, quarterly equity, rolling-20 WR)
+  computed and consistent — no bucket reverses the conclusion; both arms' 12mo profit is
+  concentrated in Q1-2026 (+87.7 ctrl / +102.0 rr20).
+
+## Status
+
+- `--fixed-rr-tp` stays as a documented research-only flag (default off; conflicts with
+  `--trail-arm/--trail-dist/--set signals.*trail*` are hard errors). Module 22 tests
+  (5) pin the override math, TP fill + spread, SL-first tie, PB-trail-disable, and CLI
+  guards. Full suite: 346 green.
+- Re-open only if gold's regime turns and a *regime-conditional* exit (trail in range,
+  target in trend) is proposed with a pre-registered mechanism — a plain fixed ratio is
+  now a closed hypothesis at every multiple in {0.5, 1.0, 1.5, 2.0, 2.5, 3.0}.

backtest/research/llm_news_eval_2026-08-06.md

@@ -0,0 +1,86 @@
+# DeepSeek news-analysis validation — 2026-08-06
+
+**Verdict: headline sentiment scoring is excellent; event-direction calls are
+directionally sound but ~11% of them misread the numeric comparison, and the
+failure concentrates on Unemployment Claims / close K-M-suffixed values.**
+
+Method (owner-directed, token-lean): independent expert re-analysis of a
+stratified sample of DeepSeek's recorded answers, not a full statistical
+backtest. Harness: `backtest/research/llm_news_eval.py` (replays releases
+through the REAL `_llm_event_direction` / `_llm_headline_scores` — real
+prompts, payloads, validation; raw wire traffic logged). 135 of the 586-event
+corpus were replayed before the full run was stopped by owner request
+(artifacts retained in `llm_news_eval_2026-08/`, resume-safe).
+
+## Part A — event gold-direction (28-event stratified sample, Jan–Jun 2024)
+
+Reviewer agreement with DeepSeek: **24/28 (86%)**. All 4 disagreements are
+DeepSeek errors, in exactly two failure modes:
+
+1. **Numeric comparison misread (3/28 ≈ 11%)** — the doctrine logic is right
+   but the premise is backwards:
+   - JOLTS 8.79M vs f 8.84M (a miss) → DS: "Job openings **above** forecast" → bearish (should be bullish)
+   - Claims 202K vs f 217K (fewer claims) → DS: "Claims **higher** than forecast" → bullish (should be bearish)
+   - Claims 202K vs f 209K — same backwards read, same wrong call
+   Both Unemployment Claims events in the sample were misread the same way —
+   looks systematic on that event type, possibly an anchoring prior that
+   "claims rose". All misreads involve close values with K/M suffixes.
+2. **Direction field contradicts its own reason (1/28)** — Empire State
+   −43.7 vs f −4.9: reason text says "gold may rise", `gold_direction` says
+   `bearish`.
+
+Where the numbers were read correctly, hawkish/dovish→gold mapping was
+consistently correct (CPI/PPI/NFP/ADP/GDP/PMI/wages/unemployment-rate all
+handled properly, including inverted metrics when read right).
+
+Price cross-check (1h horizon, cluster-confounded, indicative only):
+reviewer calls 22/28 with the subsequent move, DeepSeek 20/28.
+
+## Part B — headline sentiment (78-headline golden set)
+
+DeepSeek matched the reviewer's expected gold-direction label on
+**78/78 (100%)** — including all 8 negation traps ("signals no rate hike",
+"no rate cuts until…") and all 6 inverted-metric phrasings ("claims hit
+two-year high"). The keyword scorer it overrides got **24/78 (31%)**.
+Repeat-batch and shuffled-order probes recorded (headlines_replay.jsonl).
+
+## Implications
+
+- The headline-sentiment override (the main live LLM surface) is validated —
+  a large, measured upgrade over keywords.
+- The event-direction enrichment ("— gold bullish/bearish" appended to
+  surprise descriptions) is right ~85–90% of the time but can state a
+  factually backwards comparison in the displayed reason. It is display-only
+  (never gates signals), so exposure is a possibly-misleading Telegram/CLI
+  line, not trades.
+- **Hardening SHIPPED same day (owner-approved):** the payload now carries a
+  code-computed `vs_forecast` field ("above/below/in line with forecast",
+  from `EconomicEvent.surprise`) and the prompt tells the model to trust it.
+  First iteration fixed the arithmetic but exposed a masked doctrine error
+  (model read "claims below forecast" correctly, then called it a *weaker*
+  labor market — and regressed the unemployment-rate case the same way), so
+  the prompt's inverted-metrics line was expanded to an explicit rule plus a
+  worked claims example. Verification: **18/18** — all 4 original misreads
+  fixed, 11/11 no-regression cases unchanged, 3/3 fresh probes on the side
+  the prompt example does NOT cover (claims 245K vs f 220K → bullish,
+  unemployment 4.1% vs 3.9% → bullish, JOLTS 9.10M vs 8.80M → bearish).
+  Suite green at 346 tests, incl. a new payload assertion in Module 19.
+- Production observations found while building the harness (not fixed):
+  `_llm_event_cache` key omits forecast → a repeated `actual` re-serves a
+  direction computed for a different forecast; a "neutral" reply is
+  indistinguishable from a failure in the return value.
+
+## Reproduce
+
+```
+venv/bin/python backtest/research/llm_news_eval.py events --limit 5   # smoke
+venv/bin/python backtest/research/llm_news_eval.py events --determinism 20
+venv/bin/python backtest/research/llm_news_eval.py headlines
+venv/bin/python backtest/research/llm_news_eval.py report             # offline
+```
+
+Corpus: 586 high-impact USD surprises 2024-01→2026-07 (calendar.parquet,
+NFP-verified UTC); prices: Oanda M15 mids. Full-run cost ≈ cents. The
+`report` subcommand (hit rates vs always-bullish/doctrine/FF-betterWorse
+baselines, McNemar, cluster view) is built and untested at scale — run it
+after completing the replay if a statistical answer is ever wanted.

backtest/research/news_sentiment_value_2026-09-04.md

@@ -0,0 +1,129 @@
+# News sentiment vs. price — does the news score carry information? — 2026-09-04
+
+**Source evidence:** `logs/signals.db` 2026-08-13 → 2026-09-03 (14,683 rows, 13,947 with
+a price; candle snapshots from 08-18, full-shape headline blobs from 08-19), read with
+`mode=ro`. A continuous H1/M15/M5 candle stream was rebuilt from the stored 14-bar
+`candles_json` snapshots (H1: 293 bars, 08-18 05:00 → 09-03 22:00). Headlines were
+resolved through `news_snapshots` (4,054 distinct headlines, 2,024 gold-tagged).
+Outcomes are the repo's own advisory model (`advisory_r_multiple` on the 144 `RESOLVED`
+rows, R at $0.65). One-year context from the Oanda H1 parquet
+(`backtest/cache/oanda_XAU_USD_1h_20250814_20260814.parquet`). No backtest: the
+backtester runs with news off, and no headline history exists beyond these 3 weeks, so a
+replay cannot test news. The same check reruns with
+`python analyze_signal_log.py --stored-candles --news`.
+
+Question asked by the owner: the log reads bullish most of the time — what did the
+indicators and the news say when gold fell, and is news scoring a waste?
+
+## Findings
+
+**The numeric news score is a rear-view mirror.** `news_sentiment` (= mean gold-headline
+score − mean dollar-headline score over the last 12 h) tracks what gold did over the
+*previous* day and says nothing stable about the next one.
+
+| hourly mean sentiment vs. gold return (n≈293 h) | Pearson |
+|---|---|
+| past 24 h | **+0.67** |
+| past 8 h / 4 h / 1 h | +0.32 / +0.20 / +0.07 |
+| next 1 h / 4 h / 8 h / 24 h | +0.05 / +0.10 / +0.21 / +0.14 |
+| next 8 h, first half (08-18→25) / second half (08-26→09-03) | **−0.37 / +0.12** |
+| next 24 h, first half / second half | −0.12 / −0.09 |
+| daily mean vs same-day / previous-day / next-day move | +0.39 / +0.37 / **+0.03** |
+| *change* in sentiment vs next 4 h / 8 h / 24 h | −0.09 / −0.03 / −0.01 |
+
+Partial correlation with the next 8 h after removing the past-24 h return is +0.26 pooled
+but flips sign between halves. Alternative aggregates (gold-specific score only, dropping
+price-report headlines, only headlines < 2 h old) show the same shape: past-24 h
+correlation 0.59–0.82, no stable forward correlation. A naive "hold in the direction of
+sentiment" rule made +$272 over 279 hours vs +$120 buy-and-hold, but trading the *change*
+in sentiment lost −$200: the gain is trend persistence (sentiment ≈ trailing return), not
+news.
+
+**Why the log reads bullish.** 62% of rows have sentiment ≥ +0.1, 21% ≤ −0.1. Component
+means: gold headlines +0.158, dollar headlines +0.048. The window was a 16% August rally,
+so the average headline was bullish because price had been rising. Of the gold-tagged
+headlines, 42% merely restate the chart (price-up reports 489, mean +0.37; price-down 366,
+mean −0.18) and 18% are long-horizon forecast pieces ("$5,400 target", "$50,000/oz"; 60%
+positive, 21% of all positive score mass). The per-headline LLM labels are correct — every
+regex-flagged "mis-score" was a correctly scored "yields fall → gold up" headline — which
+matches the 78/78 golden set of 2026-08-06. The averaging, the 12 h window and the
+headline universe are what destroy the information.
+
+**What news and indicators said on the way down** (top $4,697 on 08-25 00:00 UTC, low
+$4,283 on 09-02 03:00; an 8.8% drop in a week):
+
+| moment | price | news sentiment | engine (M15) |
+|---|---|---|---|
+| top, 08-25 00:00 | 4,697 | +0.30, rising to **+0.44** two hours after the top (window high) | ranging, composite +5.9; 9 BUY setups that day: −7.4R, 11% win |
+| 08-26/27 | −$60 | still +0.15…+0.37; first bearish hour 08-26 20:00, **44 h after the top**, price already −$102 | trending_down 134/138 rows, composite −4.6/−7.8; SELL setups 0/3 on 08-27 |
+| 08-28 Warsh speech 13:00–16:00 | −$122 in 3 h, −$140 day | +0.24…+0.30 overnight, **0.00 at 13:00**, −0.17 only at 16:00; summary all morning: "a hawkish tone could pressure gold, dovish hints may lift it" | composite −6.2; SELL setups 4/4, +3.3R |
+| 09-01 | −$128 day | −0.04…−0.20, "gold under pressure" all day (correct, describing the move) | trending_down 275 rows, composite −12.2, 51 SELL rows; SELL setups 7/7, +3.5R |
+| low, 09-02 03:00 | 4,283 | −0.16 at the low, −0.19 six hours later; first bullish hour 09-02 19:00, **16 h after the low**, price already +$108 | SELL setups at the bottom 0/2, −2.05R |
+
+Sentiment peaked at the top and troughed at the bottom. The engine also lagged (it is
+trend-following) but flipped regime and composite within hours; its SELL setups won on the
+trend days and lost at the turns, the same as its BUYs.
+
+**Outcomes** (144 RESOLVED, +37.8R): BUY 121 / +43.2R / 70% win; SELL 23 / −5.4R / 48%.
+
+| direction × sentiment bucket (≥+0.15 bull, ≤−0.05 bear) | n | R | mean | win |
+|---|---|---|---|---|
+| BUY, bullish news | 81 | +32.0 | +0.40 | 73% |
+| BUY, neutral | 34 | +10.6 | +0.31 | 68% |
+| BUY, bearish news | 6 | +0.65 | +0.11 | 50% |
+| SELL, bearish news | 14 | +0.13 | +0.01 | 71% |
+| SELL, neutral | 5 | −1.46 | −0.29 | 20% |
+| SELL, bullish news | 4 | −4.08 | −1.02 | 0% |
+
+The splits are the trend confound again ("BUY with bullish news" = "BUY in an uptrend").
+Cleanest counter-example: 08-25, the most bullish news day of the drop period (+0.25),
+9 BUYs at the top lost −7.4R. corr(sentiment, R): BUY +0.10, SELL −0.45 (n=23).
+
+**Macro persona** (reads the same headlines plus high-impact events): macro=support 51
+BUYs → +32.6R, 90% win; macro=neutral 16 → −3.2R, 38%. Within bullish-sentiment BUYs:
+support 42 → +25.2R (88%) vs neutral 7 → −2.9R (29%). On 08-25 all 7 panel rows were
+macro=neutral ("awaiting PCE, mixed") while the numeric aggregate said +0.25 bullish.
+Confounded with regime (64 of 84 supports were trending_up) and n≈16 on the informative
+side. Not enough to promote or demote; the standing rule (re-score at ~300 outcomes)
+applies.
+
+**Event risk** cannot be evaluated here: only 4 event windows in the sample
+(08-19 17:00, 08-26 12:00, 08-28 13:00, 09-01 13:00), none produced a setup. The standing
+evidence remains the 2026-07-11 audit (event gates would cost +4.6…+7.2R/yr).
+
+**One-year context:** daily moves ≤ −$100 happened 17 times in the year to 2026-08-14;
+15 of 47 rolling 3-week windows contained a ≥ 8% drawdown. This sample is one ordinary
+rally-then-drop regime.
+
+**Simulated 4 h window + neutral drop** (the change shipped with this note, on 11,136
+full-shape scans): never leaves a scan empty (median 19 directional headlines, 8
+gold-specific; worst hour 02:00 UTC has < 3 gold headlines 28% of the time), the aggregate
+correlates 0.78 with the old one and becomes bolder (share |score| ≥ 0.3 rises 17% → 41%),
+with the same lack of forward information (next-24 h corr +0.20; next-8 h halves
+−0.22 / −0.10).
+
+## Method
+
+- Hourly join convention: a scan in hour H (boundary + 10 s) sees the bar closed at H,
+  i.e. bar H's open; forward returns run from bar H's open, past returns end there.
+- Correlations are Pearson on hourly means (no scipy); halves split at 2026-08-26 00:00.
+- Headline classes by regex on the title (price-up / price-down / forecast-style); the
+  gold-specific score is `score_by_currency.XAU` where present, else `score`.
+- Outcome buckets use the representative row's `news_sentiment`; macro stance from
+  `panel_takes_json`.
+- Everything above reproduces from the DB alone; nothing depends on live feeds.
+
+## Decisions
+
+- `sentiment_affects_signals` and `event_risk_caps_signals` stay `False`. No gate is
+  built in either direction (follow or fade): 3 weeks, one regime.
+- The headline labelling is not the problem; the pool and the averaging are. Shipped the
+  same day: `headline_max_age_seconds` 12 h → 4 h with undated items dropped, and
+  `get_fundamental_context` drops headlines whose every relevant score is inside ±0.1
+  (`_is_neutral_headline`). The market-summary prompt now writes for someone who has
+  never traded, sends headlines newest-first with the platform's own gold direction, and
+  caps at 400 chars. **`news_sentiment` rows from the deploy date onward are a mean over
+  directional ≤ 4 h headlines — log comparisons must split at that date.**
+- Keep the macro persona running and logging; re-check with `--news` and the persona
+  scorecard at ~300 outcomes.
+- Refuted hypothesis, do not re-propose without new data: "news sentiment leads price".

backtest/research/repin_verification_2026-08-14.md

@@ -0,0 +1,40 @@
+# Backtest Re-pin on Oanda bars — verification (Phase 3)
+
+**Date:** 2026-08-14 (verification; original books were generated 2026-08-04, current-HEAD M5 canonical book exported 2026-08-14)
+**Scope:** confirm the pinned Oanda books exist, satisfy rule 1, and quantify the Oanda↔TD delta that motivated them.
+
+## Re-pin state
+
+The re-pin is **already done** — full Oanda-bar books exist in `backtest/oanda_*.csv` for M5/M15/H1/H4, at both cost assumptions, with in-sample (12mo), out-of-sample (2024-06 → 2025-03), and 5-year variants. The shipped configs are `m5` (pullback OFF — `_nopb`) and `m15`.
+
+### M5 canonical-reference reconciliation (2026-08-14)
+
+The dated current-HEAD export `backtest/oanda_m5_12mo_065_nopb_20260814.csv` is the canonical M5 12-month reference going forward: **577 trades, +106.82R, PF 1.51**, with both 12-month halves positive (+50.80R / +56.02R; PF 1.46 / 1.55). It covers 2025-08-14 → 2026-08-14, runs the shipped M5 factory (XAU_PB off), charges $0.65 round trip, and uses next-open/raw-path/close-only-trail realism unchanged.
+
+Earlier M5 snapshots (`oanda_m5_12mo_065_nopb.csv`: 586 / +98.11R / 1.45; `m5_12mo_shipped.csv`: 588 / +96.60R / 1.44; and the 555 / +98.64R ledger entry) remain historical evidence from different HEADs around the weekend-calendar and related QA work. They must not be cited as the current-HEAD reference.
+
+### Rule-1 summary at realistic cost ($0.65 round trip)
+
+| TF | 12mo in-sample | 12mo H1 / H2 | OOS (2024-06→2025-03) |
+|---|---|---|---|
+| M15 | +73.93R PF 1.53 | +66.71 / +7.23 (PF 1.10) | +38.77R PF 1.36 |
+| M5 (shipped, nopb) | +106.82R PF 1.51 (current HEAD) | +50.80 / +56.02 (PF 1.46 / 1.55) | +9.52R PF 1.34 |
+| H1 | +23.97R PF 1.55 | — | +5.15R PF 1.13 |
+| H4 | +16.14R PF 2.28 | — | +1.31R PF 1.14 |
+
+- **M5 (shipped)** passes rule 1 cleanly: both current-HEAD 12mo halves positive with PF ≥ 1.3, and OOS positive with PF 1.34.
+- **M15** passes in-sample and OOS (PF 1.53 / 1.36) but its second half is soft (PF 1.10) — positive R, below the 1.3 bar. Pre-existing condition, not a re-pin artifact.
+- **H1/H4** pass in-sample but OOS is marginal (PF 1.13–1.14) — consistent with the standing "H4 unproven for auto-execution" note.
+
+## Oanda ↔ Twelve Data trade-level delta
+
+Not worth a trade-by-trade book diff: Phase 1 already showed the two feeds disagree on **bar direction** 19% of the time on M5 (11% on M15), with a ~$3 mean / ~$1 systematic close gap. A TD-pinned book is a different instrument's book — this is the documented re-pin motivator, and the Oanda books above are the replacement baseline. A formal trade-level reconciliation would show near-total mismatch and add no decision value.
+
+## Deliverable status
+
+- [x] Re-pinned books exist on Oanda bars (12mo + OOS + 5yr, $0.30 + $0.65).
+- [x] Rule 1 verified for the shipped TFs (M5 clean; M15 in-sample+OOS pass, H2 soft; H1/H4 OOS marginal).
+- [x] Cost model: $0.65 matches measured Oanda spread (~$0.65–0.72); $0.30 retained for cross-book comparability.
+- [x] Oanda↔TD delta quantified (Phase 1) — no further book diff needed.
+
+Remaining Phase-3 follow-up: none. Owner decision 2026-08-14 retained dual reporting: $0.65 is the realistic headline while $0.30 remains for TD-era cross-book comparability and the live TP spread gate.

backtest/research/replay_recording_2026-09-09.md

@@ -0,0 +1,148 @@
+# Replay recording campaign — 2026-09-09 (2-year Oanda window, M5 + M15, $0.65)
+
+## Purpose and owner decisions
+
+The owner asked for a 2-year replay of the M5 and M15 timeframes that writes down, for
+every trade setup the system generates, the indicator weights and values the decision was
+made with and the economic-calendar events around it, in a form that a later study can use
+to decide whether indicator weights should change, or indicators be dropped or replaced.
+Owner decisions (2026-09-09): also record one compact row per ANALYZED bar (not only the
+bars that became trades), and restore the Forex Factory calendar scraper to `scripts/`.
+
+Nothing in the signal core changed. The recorder is opt-in (`--record DIR`, `--record-bars`)
+and read-only with respect to the signal path; the trade book with recording on is
+byte-identical to the book with it off (MD5 below).
+
+## Harness
+
+`backtest/gpu_backtest.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 --spread 0.65
+--workers 12 --primary-tf {5min|15min}`, pair-factory config (no --set/--quality/--strategies/
+--confirm-tfs; confirmation TFs from the factory map: M5→15min,1h and M15→5min,1h),
+`Backend: NumPy CPU fallback` (CuPy cannot find CUDA on this box). Code at git `28b591d`
+plus the recorder (this campaign). Bars fetched once from Oanda into
+`backtest/cache/oanda_XAU_USD_{5min,15min,1h}_20240705_20260905.parquet` (62-day warm-up pad)
+and read from the cache by every later run. Each timeframe was run twice: a CONTROL run
+(`--export-csv book_control.csv`) and a RECORDED run (`--export-csv book.csv --record DIR
+--record-bars`). Outputs in `backtest/research/replay_2026-09/{m5,m15}/` (gitignored).
+
+## Books (identical for control and recorded — that is the point)
+
+| book | trades | total R | WR | PF | max DD |
+|---|---|---|---|---|---|
+| M5 whole window | 781 | +146.07 | 52.9% | 1.52 | 10.1 |
+|   first half | 189 | +28.01 | 49.7% | 1.41 | 8.4 |
+|   second half | 592 | +118.06 | 53.9% | 1.56 | 10.1 |
+| M15 whole window | 793 | +117.62 | 60.4% | 1.41 | 14.8 |
+|   first half | 383 | +38.22 | 58.5% | 1.26 | 14.8 |
+|   second half | 410 | +79.40 | 62.2% | 1.55 | 9.9 |
+
+Halves split the entry-time range in two; M5 trades cluster in the second year.
+
+### Behaviour-neutrality check
+
+| TF | control MD5 | recorded MD5 | manifest `book_csv_md5` |
+|---|---|---|---|
+| M5 | 29120a1ee32979431d062f210d41a9d1 | 29120a1ee32979431d062f210d41a9d1 | same |
+| M15 | e62d3ded56f35572f52a96f604a99cf4 | e62d3ded56f35572f52a96f604a99cf4 | same |
+
+Replay time M5 114 s / M15 35 s on 12 workers (control runs 110 s / 35 s — recording costs
+seconds). `backtest/replay_scorecard.py DIR --check` passes all eight self-consistency
+invariants on both runs: trade counts match the manifest; every trade has its primary block
+and both confirmation blocks; base weight × regime multiplier = effective weight and
+contribution = vote × effective weight × confidence/100 on every one of the 17,869 (M5) /
+18,070 (M15) indicator rows; the composite score is reproduced from the stored rows on every
+trade and timeframe; every regime multiplier equals the table in the manifest; the bar table
+has exactly one row per analyzed bar and contains every trade's bar.
+
+### Cross-check against the owner's 3-year baseline books (same window, pre-2026-09-07 code)
+
+| TF | 2-yr run | baseline `sizing_2026-09/base_*_3yr_065.csv` cut to the window | shared entries |
+|---|---|---|---|
+| M5 | 781 / +146.07R | 769 / +153.58R | 757 (24 only new, 12 only baseline) |
+| M15 | 793 / +117.62R | 788 / +115.78R | 774 (19 only new, 14 only baseline) |
+
+The baseline files are dated 2026-09-05 and predate the 2026-09-07 outlier-filter change
+(which moved the 3-yr M5 book 857→870 trades, +161.3→+153.6R), so the differences are the
+expected ones; byte identity was never expected here. The behaviour-neutrality proof is the
+MD5 pair above, on identical code.
+
+## What was recorded (storage decision: Parquet, one folder per run)
+
+Parquet keeps exact typed numbers and UTC timestamps, compresses well (M5: 20 MB for 86k
+bars × 172 columns), and pandas reads a file in one line; the repo already uses it for the
+bar cache. SQLite would need JSON blobs or many tables with string timestamps; CSV loses
+types and is several times larger. DuckDB can query the parquet files directly if SQL is
+ever wanted.
+
+| file | grain | M5 rows | M15 rows |
+|---|---|---|---|
+| `manifest.json` | CLI, window, spread, TFs, engine + gate fingerprints, git SHA, backend, regime weight table in force, observed base weights, composite thresholds, bar counts + skip reasons, cache file names, calendar fingerprint, table row counts, book MD5 | 1 | 1 |
+| `trades.parquet` | one row per ACCEPTED trade: the 63 book columns + per-timeframe regime / composite / confidence / ATR / vote sums, penalty tags, setup fields, all warnings and reasons, fingerprints, calendar scalars | 781 | 793 |
+| `indicators.parquet` | one row per trade × timeframe × indicator that voted: vote, value, confidence, base weight, regime multiplier, effective weight, contribution, agree/oppose/neutral, regime | 17,869 | 18,070 |
+| `events.parquet` | one row per trade × calendar event from 4 h before entry to exit (or 4 h after entry), plus all-day rows on the entry's New-York date, with phase before / during_hold / after_exit / all_day | 7,803 | 12,932 |
+| `bars.parquet` | one row per ANALYZED primary bar (after weekend/session filters, before the setup gates): final score/signal/quality, has_setup, per-timeframe regime/composite and per-indicator vote/value/confidence/effective weight/contribution as float32, plus forward moves 1 h / 4 h / 24 h and MFE/MAE in ATRs on the raw bars | 86,192 | 29,917 |
+
+Candidates before the 5-bar cooldown: M5 1,279 → 781 accepted; M15 1,535 → 793. An
+indicator that did not fire (divergence, candlestick) has NO row / NaN — never a zero vote.
+Reader: `backtest.replay_record.load_run(dir)`. First analysis pass:
+`backtest/replay_scorecard.py DIR --what-if --bars` (full output saved as `scorecard.txt`
+in each run folder).
+
+## First look (descriptive statistics only — NOT evidence for a change)
+
+Everything below is in-book: it cannot see trades a change would have created, and it
+ignores the MTF/penalty/gate chain that sits between the composite and the entry. The 2-year
+window is a strong gold up-trend, so the "no vote" baseline itself drifts up about +0.37 ATR
+per 4 h and +1.9 ATR per 24 h on M5 (+0.15 / +0.9 on M15); any BUY-vote number must be read
+against that baseline, not against zero.
+
+Trade-level (primary timeframe):
+- Bollinger opposes the entry on 731 of 781 M5 trades (94%): a mean-reversion reading
+  inside a trend-following entry — the "constant bias" seen in the 2026-08-04 audit — and
+  those trades carry +152R. RSI is neutral on 73% of M5 entries.
+- M5 trades where divergence OPPOSED the entry did best (158 trades, +0.37R mean, 63% wins)
+  and trades where it was absent did worst (546, +0.13R); the same "inverted" pattern as the
+  2026-08-04 in-book audit, which replayed WORSE when "fixed" — the reminder that this table
+  is a hypothesis generator.
+- What-if (M5): removing ema_trend would drop 206 accepted trades carrying +36.8R below the
+  entry band; removing MACD 211 / +21.2R; ADX 127 / +22.6R. On M15 hardly any trade loses
+  direction from a single-indicator removal (the entries sit further above the band); the
+  changes show up as strong/weak band moves instead.
+- Calendar: M5 trades held through at least one high-impact USD release did better
+  (69 trades, +0.42R mean vs +0.19 overall; M15 224 trades, +0.23 vs +0.15) — the same
+  direction as the 2026-09-06 addendum. Entries within 30 min after a release are slightly
+  negative on both TFs (40 and 35 trades — too few to act on).
+
+Bar-level (all analyzed bars, drift NOT yet removed):
+- Every indicator's SELL-side votes were followed, on average, by a move AGAINST the vote
+  on both timeframes (e.g. M5 ema_trend −0.27 ATR / 4 h, divergence −0.63); this is the
+  up-drift plus the known "SELL negative everywhere" finding. The next analysis step is to
+  subtract the per-bar baseline (or split by regime) before scoring votes.
+- Above-baseline BUY callers on M5 at 4 h: ADX +0.50, keltner +0.50, RSI +0.45,
+  candlestick +0.46, ROC +0.46, ema_trend (+0.67 vs baseline +0.32 among its own neutral
+  bars); below baseline: divergence +0.30, Bollinger +0.36.
+- Bars what-if (M5): dropping divergence would newly qualify 2,268 bars whose later move
+  averaged +0.26 ATR / 4 h and drop 3,611 bars averaging −0.26 — the one single-indicator
+  removal where both sides point the same way. Dropping ema_trend or ADX points the other
+  way (the bars they add did worse, the bars they remove did better).
+
+## Pre-registered rule for anything that comes out of these files
+
+A candidate change (weight, regime multiplier, dropped or replaced indicator) is a
+hypothesis until it passes CLAUDE.md rule 1 on a full rerun through
+`analyze_from_dataframes`: positive total R in BOTH halves of a ≥12-month window, PF ≥ 1.3
+at $0.65, trailing exits, next-open fills, plus the 2024-06→2025-03 out-of-sample window for
+marginal edges, compared with a CONTROL produced on the same harness. Nothing in this
+campaign changes any number.
+
+## Suggested next analysis (not done here)
+
+1. Drift-adjust the bar table (subtract the neutral/absent baseline per regime and hour) and
+   re-score each indicator's standalone hit rate by timeframe role.
+2. For the 2–3 most suspect indicators (divergence, Bollinger, RSI on M5), run the real
+   rule-1 rerun with `--set technical.regime_weights_override=...` or `disabled_indicators`
+   and compare against the control books in this folder.
+3. Re-run this recording after any config change so the folders stay comparable (the
+   manifest carries the engine fingerprint and the regime table for exactly that purpose).
+
+Tests: Module 33 "backtest replay recording" (12 tests); suite green.

backtest/research/signal_log_campaign_2026-09.md

@@ -0,0 +1,182 @@
+# Signal-log audit → prediction-quality campaign — 2026-09-03
+
+**Source evidence:** `logs/signals.db` 2026-08-13 → 2026-09-03 (14,683 rows, candle
+snapshots from 08-18), replayed with the repo's own advisory outcome model
+(`analyze_signal_log.py --stored-candles`; the replay reproduces
+`advisory_outcome.py` to 5e-17 on the 49 panel-seeded rows), cross-checked
+against the pinned Oanda books (`backtest/oanda_*_065*.csv`).
+
+## Live-log findings that motivated the campaign
+
+| View (3 weeks, $0.65 rt) | n | WR | R | PF |
+|---|---|---|---|---|
+| every unique setup event | 144 | 67% | +37.8 | 1.93 |
+| opportunities (first per tf+direction, 60 min) | 66 | 67% | +18.3 | 1.92 |
+| one-position-per-TF (5-bar cooldown) | 32 | 62% | +6.6 | 1.53 |
+| direction BUY / SELL | 121 / 23 | 70% / 48% | +43.2 / −5.4 | 2.53 / 0.56 |
+
+- Two rally days (08-19, 08-21) contribute essentially all of the R.
+- SELL is negative in every view; in the books SELL is the weakest bucket on
+  every TF (H1 SELL −16.7R/5yr PF 0.77, −11.5R OOS PF 0.30; M15 SELL −11.7R OOS;
+  M5 SELL in `transitioning` −7.9R/12mo PF 0.21, −5.2R/5yr PF 0.49).
+- Bollinger is anti-predictive on all four TFs (forward-return hit 0.26–0.50 at
+  12 bars) and "opposes" 93% of taken book trades; divergence is inverted on
+  M15/H1 live and trades where it opposed outperform in the books.
+- Refuted live hypotheses (books disagree): ADX<20 gate, exhaustion gate,
+  NEAR-EMA200 gate, EXTREME-volatility gate, capping SL/ATR.
+
+## Method (rule 1)
+
+Harness: `backtest/gpu_backtest.py --pair XAUUSD --spread 0.65` on the CPU
+backend (CuPy present but the GPU was not reachable; the M5 12mo control
+reproduces the pinned `oanda_m5_12mo_065_nopb_20260814.csv` trade-for-trade
+except the two trades on the end date, which `--end` excludes). Every candidate
+is compared with a CONTROL produced on the same harness, never with the pinned
+book directly. Windows (cache start = start − 62 days):
+
+| window | `--start` | `--end` |
+|---|---|---|
+| 12mo (M15/H1) | 2025-08-04 | 2026-08-04 |
+| 12mo (M5 canonical) | 2025-08-14 | 2026-08-14 |
+| 5yr | 2021-08-01 | 2026-08-04 |
+| OOS | 2024-06-01 | 2025-03-30 |
+
+Acceptance: positive total R in both 12mo halves, PF ≥ 1.3, OOS not worse
+than control, and a candidate that removes a net-positive bucket for a PF-only
+gain is rejected (big-candle precedent). Runner:
+`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py` (scratchpad
+tooling, not retained — both shipped gates are reproduced by a plain pair-factory
+run at HEAD; see the parity recipe in `sizing_campaign_2026-09.md`).
+
+Candidates (all `--set`, no code):
+
+- **B1 SELL gating** — H1 `xau_strategy.sell_min_quality=[]` (block all H1
+  shorts); M5 `xau_strategy.sell_blocked_regimes=["transitioning"]`; M15
+  `xau_strategy.sell_min_quality=["B"]`.
+- **B2 divergence off** — `technical.disabled_indicators=["divergence"]` on
+  M15 and H1 (M5 divergence-down was rejected 2026-08-04; not re-run).
+- **B3 Bollinger weight 0 in trending regimes** — full
+  `technical.regime_weights_override` with bollinger 0.0 in
+  trending_up/trending_down/trending, ranging/transitioning unchanged; M5, M15, H1.
+
+## Results
+
+Every control reproduced its pinned book: M15 12mo (`ctl_m15_12mo`) is exact
+at 397/+73.93R/PF 1.53; all three H1 controls are exact; the M5 12mo control
+(`base_m5_12mo`) is exact except for the two trades that fall on the end
+date, which `--end` excludes on this harness (see Method above).
+
+| book | n | R | PF | DD | H1 | H2 | sellN | sellR | ΔR vs control | ΔPF |
+|---|---|---|---|---|---|---|---|---|---|---|
+| b1_h1_nosell_12mo | 91 | +22.2 | 1.79 | 4.1 | +20.6 | +1.6 | 0 | +0.0 | -1.8 | +0.25 |
+| b1_h1_nosell_5yr | 335 | +49.1 | 1.46 | 13.2 | -5.0 | +54.1 | 0 | +0.0 | +16.7 | +0.28 |
+| b1_h1_nosell_oos | 78 | +16.7 | 1.74 | 3.6 | +7.3 | +9.4 | 0 | +0.0 | +11.5 | +0.61 |
+| b1_m15_sellB_12mo | 341 | +74.6 | 1.62 | 10.8 | +68.4 | +6.2 | 82 | +9.1 | +0.7 | +0.09 |
+| b1_m15_sellB_5yr | 1378 | +119.1 | 1.22 | 41.4 | -21.3 | +140.3 | 335 | +3.7 | -8.9 | +0.02 |
+| b1_m15_sellB_oos | 272 | +47.1 | 1.51 | 13.8 | +12.3 | +34.8 | 59 | -3.4 | +8.3 | +0.15 |
+| b1_m5_selltrans_12mo | 557 | +112.2 | 1.56 | 9.9 | +55.9 | +56.4 | 75 | +14.8 | +7.9 | +0.06 |
+| b1_m5_selltrans_5yr | 889 | +133.7 | 1.41 | 12.8 | -8.3 | +142.0 | 88 | +9.0 | +5.3 | +0.03 |
+| b1_m5_selltrans_oos | 86 | +9.4 | 1.33 | 8.2 | -1.4 | +10.9 | 4 | +1.5 | +0.0 | +0.00 |
+| b2_h1_nodiv_12mo | 149 | +28.6 | 1.62 | 6.5 | +23.6 | +5.0 | 49 | +5.5 | +4.6 | +0.07 |
+| b2_h1_nodiv_5yr | 567 | +40.5 | 1.20 | 14.1 | -5.2 | +45.6 | 203 | -18.2 | +8.1 | +0.02 |
+| b2_h1_nodiv_oos | 119 | +7.6 | 1.17 | 7.7 | -1.9 | +9.5 | 31 | -14.3 | +2.4 | +0.04 |
+| b2_m15_nodiv_12mo | 439 | +81.3 | 1.51 | 13.1 | +72.3 | +8.9 | 147 | +7.0 | +7.4 | -0.02 |
+| b2_m15_nodiv_5yr | 1771 | +139.9 | 1.20 | 47.4 | -16.2 | +156.1 | 578 | +11.7 | +11.9 | -0.01 |
+| b2_m15_nodiv_oos | 341 | +52.3 | 1.43 | 17.1 | +15.6 | +36.7 | 93 | -10.0 | +13.5 | +0.07 |
+| b3_h1_bb0_12mo | 154 | +31.2 | 1.64 | 9.4 | +22.8 | +8.4 | 52 | +6.7 | +7.3 | +0.09 |
+| b3_h1_bb0_5yr | 583 | +37.3 | 1.18 | 18.3 | -7.4 | +44.7 | 206 | -20.4 | +4.9 | +0.00 |
+| b3_h1_bb0_oos | 122 | +5.6 | 1.12 | 8.7 | -2.0 | +7.6 | 32 | -13.8 | +0.4 | -0.01 |
+| b3_m15_bb0_12mo | 439 | +49.9 | 1.29 | 19.4 | +61.6 | -11.8 | 151 | +0.3 | -24.0 | -0.23 |
+| b3_m15_bb0_5yr | 1790 | +126.6 | 1.18 | 42.3 | -1.8 | +128.5 | 592 | +17.4 | -1.3 | -0.03 |
+| b3_m15_bb0_oos | 345 | +55.5 | 1.47 | 12.4 | +18.2 | +37.3 | 94 | -8.4 | +16.7 | +0.11 |
+| b3_m5_bb0_12mo | 621 | +84.0 | 1.35 | 16.6 | +49.3 | +34.7 | 104 | +9.7 | -20.4 | -0.14 |
+| b3_m5_bb0_5yr | 987 | +113.0 | 1.31 | 15.3 | -7.7 | +120.6 | 134 | +12.0 | -15.5 | -0.08 |
+| b3_m5_bb0_oos | 93 | +10.6 | 1.35 | 5.6 | -2.9 | +13.5 | 9 | +3.7 | +1.2 | +0.01 |
+| b12_h1_nosell_nodiv_12mo | 100 | +23.1 | 1.75 | 4.6 | +21.8 | +1.3 | 0 | +0.0 | -0.8 | +0.20 |
+| b12_h1_nosell_nodiv_5yr | 364 | +58.7 | 1.52 | 12.9 | -4.1 | +62.8 | 0 | +0.0 | +26.3 | +0.34 |
+| b12_h1_nosell_nodiv_oos | 88 | +21.9 | 1.92 | 4.8 | +9.4 | +12.5 | 0 | +0.0 | +16.8 | +0.79 |
+| base_m5_12mo | 575 | +104.3 | 1.50 | 10.7 | +50.8 | +53.5 | 93 | +6.9 | — | — |
+| ctl_h1_12mo | 138 | +24.0 | 1.55 | 8.0 | +19.4 | +4.6 | 47 | +1.8 | — | — |
+| ctl_h1_5yr | 515 | +32.4 | 1.18 | 15.2 | -5.9 | +38.3 | 180 | -16.7 | — | — |
+| ctl_h1_oos | 104 | +5.1 | 1.13 | 7.5 | -0.9 | +6.1 | 26 | -11.5 | — | — |
+| ctl_m15_12mo | 397 | +73.9 | 1.53 | 10.8 | +66.7 | +7.2 | 138 | +8.5 | — | — |
+| ctl_m15_5yr | 1578 | +128.0 | 1.21 | 35.8 | -3.4 | +131.4 | 535 | +12.6 | — | — |
+| ctl_m15_oos | 299 | +38.8 | 1.36 | 15.5 | +5.9 | +32.9 | 86 | -11.7 | — | — |
+| ctl_m5_5yr | 911 | +128.4 | 1.39 | 12.8 | -8.3 | +136.7 | 110 | +3.8 | — | — |
+| ctl_m5_oos | 86 | +9.4 | 1.33 | 8.2 | -1.4 | +10.9 | 4 | +1.5 | — | — |
+
+Raw table (n / R / PF / WR% / DD / H1 / H2 / PF-half-1 / PF-half-2 /
+sellN / sellR / ΔR / ΔPF / Δn vs control; control and `base_m5_12mo`
+rows carry no delta):
+
+```
+book                           n        R    PF   WR%    DD      H1      H2   PF1   PF2 sellN   sellR |      ΔR    ΔPF    Δn
+b1_h1_nosell_12mo             91    +22.2  1.79  62.6   4.1   +20.6    +1.6  2.10  1.18     0    +0.0 |    -1.8  +0.25  -47
+b1_h1_nosell_5yr             335    +49.1  1.46  58.2  13.2    -5.0   +54.1  0.87  1.78     0    +0.0 |   +16.7  +0.28 -180
+b1_h1_nosell_oos              78    +16.7  1.74  62.8   3.6    +7.3    +9.4  1.67  1.81     0    +0.0 |   +11.5  +0.61  -26
+b1_m15_sellB_12mo            341    +74.6  1.62  62.8  10.8   +68.4    +6.2  2.15  1.10    82    +9.1 |    +0.7  +0.09  -56
+b1_m15_sellB_5yr            1378   +119.1  1.22  57.0  41.4   -21.3  +140.3  0.91  1.46   335    +3.7 |    -8.9  +0.02 -200
+b1_m15_sellB_oos             272    +47.1  1.51  60.7  13.8   +12.3   +34.8  1.24  1.85    59    -3.4 |    +8.3  +0.15  -27
+b1_m5_selltrans_12mo         557   +112.2  1.56  53.7   9.9   +55.9   +56.4  1.54  1.58    75   +14.8 |    +7.9  +0.06  -18
+b1_m5_selltrans_5yr          889   +133.7  1.41  51.5  12.8    -8.3  +142.0  0.84  1.52    88    +9.0 |    +5.3  +0.03  -22
+b1_m5_selltrans_oos           86     +9.4  1.33  53.5   8.2    -1.4   +10.9  0.91  1.95     4    +1.5 |    +0.0  +0.00   +0
+b2_h1_nodiv_12mo             149    +28.6  1.62  62.4   6.5   +23.6    +5.0  1.93  1.24    49    +5.5 |    +4.6  +0.07  +11
+b2_h1_nodiv_5yr              567    +40.5  1.20  56.3  14.1    -5.2   +45.6  0.93  1.37   203   -18.2 |    +8.1  +0.02  +52
+b2_h1_nodiv_oos              119     +7.6  1.17  56.3   7.7    -1.9    +9.5  0.91  1.44    31   -14.3 |    +2.4  +0.04  +15
+b2_m15_nodiv_12mo            439    +81.3  1.51  60.8  13.1   +72.3    +8.9  1.90  1.11   147    +7.0 |    +7.4  -0.02  +42
+b2_m15_nodiv_5yr            1771   +139.9  1.20  55.8  47.4   -16.2  +156.1  0.94  1.39   578   +11.7 |   +11.9  -0.01 +193
+b2_m15_nodiv_oos             341    +52.3  1.43  59.2  17.1   +15.6   +36.7  1.23  1.68    93   -10.0 |   +13.5  +0.07  +42
+b3_h1_bb0_12mo               154    +31.2  1.64  60.4   9.4   +22.8    +8.4  1.89  1.36    52    +6.7 |    +7.3  +0.09  +16
+b3_h1_bb0_5yr                583    +37.3  1.18  54.5  18.3    -7.4   +44.7  0.90  1.35   206   -20.4 |    +4.9  +0.00  +68
+b3_h1_bb0_oos                122     +5.6  1.12  55.7   8.7    -2.0    +7.6  0.91  1.33    32   -13.8 |    +0.4  -0.01  +18
+b3_m15_bb0_12mo              439    +49.9  1.29  59.0  19.4   +61.6   -11.8  1.77  0.87   151    +0.3 |   -24.0  -0.23  +42
+b3_m15_bb0_5yr              1790   +126.6  1.18  56.1  42.3    -1.8  +128.5  0.99  1.31   592   +17.4 |    -1.3  -0.03 +212
+b3_m15_bb0_oos               345    +55.5  1.47  61.2  12.4   +18.2   +37.3  1.29  1.66    94    -8.4 |   +16.7  +0.11  +46
+b3_m5_bb0_12mo               621    +84.0  1.35  51.4  16.6   +49.3   +34.7  1.43  1.28   104    +9.7 |   -20.4  -0.14  +46
+b3_m5_bb0_5yr                987   +113.0  1.31  50.7  15.3    -7.7  +120.6  0.85  1.38   134   +12.0 |   -15.5  -0.08  +76
+b3_m5_bb0_oos                 93    +10.6  1.35  54.8   5.6    -2.9   +13.5  0.85  2.17     9    +3.7 |    +1.2  +0.01   +7
+b12_h1_nosell_nodiv_12mo     100    +23.1  1.75  63.0   4.6   +21.8    +1.3  2.00  1.15     0    +0.0 |    -0.8  +0.20  -38
+b12_h1_nosell_nodiv_5yr      364    +58.7  1.52  59.6  12.9    -4.1   +62.8  0.90  1.87     0    +0.0 |   +26.3  +0.34 -151
+b12_h1_nosell_nodiv_oos       88    +21.9  1.92  63.6   4.8    +9.4   +12.5  1.93  1.92     0    +0.0 |   +16.8  +0.79  -16
+base_m5_12mo                 575   +104.3  1.50  52.9  10.7   +50.8   +53.5  1.46  1.53    93    +6.9
+ctl_h1_12mo                  138    +24.0  1.55  60.9   8.0   +19.4    +4.6  1.84  1.22    47    +1.8
+ctl_h1_5yr                   515    +32.4  1.18  55.7  15.2    -5.9   +38.3  0.91  1.34   180   -16.7
+ctl_h1_oos                   104     +5.1  1.13  56.7   7.5    -0.9    +6.1  0.95  1.32    26   -11.5
+ctl_m15_12mo                 397    +73.9  1.53  62.2  10.8   +66.7    +7.2  1.95  1.10   138    +8.5
+ctl_m15_5yr                 1578   +128.0  1.21  56.5  35.8    -3.4  +131.4  0.99  1.37   535   +12.6
+ctl_m15_oos                  299    +38.8  1.36  59.2  15.5    +5.9   +32.9  1.10  1.66    86   -11.7
+ctl_m5_5yr                   911   +128.4  1.39  51.3  12.8    -8.3  +136.7  0.84  1.48   110    +3.8
+ctl_m5_oos                    86     +9.4  1.33  53.5   8.2    -1.4   +10.9  0.91  1.95     4    +1.5
+```
+
+## Decisions
+
+- **SHIPPED** — M5 `xau_strategy.sell_blocked_regimes=["transitioning"]`
+  (b1_m5_selltrans): 12mo +7.9R, 5yr +5.3R, OOS unchanged (no such trades
+  in that window); positive in both 12mo halves on every window. H1
+  `xau_strategy.sell_min_quality=[]` (b1_h1_nosell, all H1 shorts blocked):
+  5yr +16.7R and OOS +11.5R more than pay for the 12mo cost of -1.8R; H1
+  shorts were -16.7R/5yr and -11.5R/OOS in the control. Both land in
+  `pair_signals/xauusd.py`.
+- **REJECTED** — M15 `sell_min_quality=["B"]` (b1_m15_sellB): 5yr -8.9R
+  with max DD widening 35.8 -> 41.4R for a marginal +0.7R on 12mo; fails
+  the drawdown clause. All three B3 Bollinger-weight-0-in-trending-regimes
+  runs (b3_h1_bb0, b3_m15_bb0, b3_m5_bb0): negative on M5 (-20.4R/12mo,
+  -15.5R/5yr) and M15 (-24.0R/12mo, PF 1.29 -> still below the 12mo H2
+  going negative at -11.8R); H1's B3 5yr gain (+4.9R) doesn't clear the
+  bar once M5/M15 fail outright — Bollinger's constant-oppose vote stays
+  a load-bearing quality bar, consistent with the 2026-08-04 audit.
+- **DECLINED** — B2-M15 (`technical.disabled_indicators=["divergence"]`
+  on M15, b2_m15_nodiv): R improves on every window (+7.4/+11.9/+13.5R)
+  but 5yr max DD widens 35.8 -> 47.4R while PF stays flat (1.21 -> 1.20) —
+  more risk for no risk-adjusted gain; owner call.
+- **PARKED** — B2-H1 (`technical.disabled_indicators=["divergence"]` on
+  H1, b2_h1_nodiv, standalone: +4.6/+8.1/+2.4R, PF 1.62/1.20/1.17). The
+  combined run with the H1 no-short override (b12_h1_nosell_nodiv) passes
+  rule 1 and beats the no-shorts override alone on both 5yr and OOS: 12mo
+  100/+23.1R PF 1.75 (halves +21.8/+1.3), 5yr 364/+58.7R PF 1.52, OOS
+  88/+21.9R PF 1.92. Not shipped anyway: it would route through
+  `technical.disabled_indicators`, which CLAUDE.md rule 8 reserves for
+  experiments, not production paths. Owner decision; recommended vehicle
+  is an H1-only `regime_weights_override` with divergence weight 0 plus a
+  fresh validation run under that mechanism.

backtest/research/signal_log_eval_2026-08-18.md

@@ -0,0 +1,113 @@
+# Signal-log live performance evaluation — 2026-08-18
+
+**Tool:** `analyze_signal_log.py` (new, repo root). Read-only on
+`logs/signals.db` (sqlite `immutable=1`); replays every logged trade
+setup through the backtester's own `simulate_trades` on the RAW Oanda
+price path (next-open fill, SL/TP distances preserved, SL-first ties,
+gap-at-open fills, close-only trail ratchet with the per-setup stamped
+trail params). Reproduce:
+
+```bash
+python analyze_signal_log.py --csv backtest/signal_log_eval_20260818.csv
+```
+
+## Window & sample
+
+- Log window: **2026-08-13 14:31 → 2026-08-18 14:13 UTC (5 days)** —
+  the entire signal_log history to date.
+- 2,654 rows (telegram 2,180 / dashboard 474); M5 1,308, M15 1,189,
+  H1 119, H4 38.
+- 39 rows carry a `trade_setup` → **17 unique signal events** after
+  dedup (cross-source double-logs + persisting-signal re-fires collapse
+  on (tf, direction, entry, SL, strategy) with a 60-min gap).
+- **No real executions exist** (`oanda_state.json` empty, no
+  `oanda_daemon` rows) — every outcome below is a hypothetical replayed
+  fill, not a broker fill.
+- Cost: dual convention — **$0.65 rt headline**, $0.30 comparative
+  (derived analytically, R is linear in cost).
+
+## Trade replay results
+
+| First logged | TF | Signal | Grade | Strategy | Fill | Exit | Bars | R@0.65 |
+|---|---|---|---|---|---|---|---|---|
+| 08-13 15:30 | 5min | SELL | B | GENERIC | 4365.39 | TRAIL | 19 | −0.20 |
+| 08-13 18:00 | 15min | SELL | B | GENERIC | 4356.05 | TRAIL | 44 | +0.91 |
+| 08-14 07:45 | 5min | BUY | B | XAU_CP | 4339.72 | TRAIL | 60 | +2.56 |
+| 08-14 12:15 | 15min | BUY | B | GENERIC | 4371.91 | TRAIL | 27 | +0.06 |
+| 08-14 12:25 | 5min | STRONG_BUY | A | GENERIC | 4379.12 | TRAIL | 277 | +0.20 |
+| 08-14 12:30 | 5min | STRONG_BUY | A | GENERIC | 4379.87 | TRAIL | 276 | +0.16 |
+| 08-14 12:30 | 15min | BUY | B | GENERIC | 4379.87 | TRAIL | 63 | +0.18 |
+| 08-17 08:00 | 1h | BUY | C | GENERIC | 4405.19 | TRAIL | 18 | −0.13 |
+| 08-17 14:15 | 5min | STRONG_BUY | A | GENERIC | 4412.35 | TRAIL | 46 | −0.10 |
+| 08-17 14:20 | 5min | STRONG_BUY | A | GENERIC | 4418.55 | TRAIL | 125 | −0.23 |
+| 08-17 14:25 | 5min | STRONG_BUY | A | GENERIC | 4424.10 | SL | 151 | −1.02 |
+| 08-17 14:30 | 15min | BUY | A | GENERIC | 4418.23 | SL | 17 | −1.03 |
+| 08-17 14:45 | 15min | STRONG_BUY | A | GENERIC | 4417.49 | SL | 16 | −1.03 |
+| 08-17 15:00 | 15min | STRONG_BUY | A | GENERIC | 4417.48 | SL | 15 | −1.03 |
+| 08-17 15:00 | 5min | NEUTRAL | C | GENERIC | 4417.48 | TRAIL | 117 | −0.16 |
+| 08-17 15:15 | 15min | STRONG_BUY | A | GENERIC | 4421.30 | SL | 14 | −1.04 |
+| 08-17 15:30 | 15min | BUY | B | GENERIC | 4425.70 | SL | 85 | −1.02 |
+
+**Aggregate: 17 closed, 6 wins (35%), total −2.91R @0.65 / −2.67R
+@0.30, PF 0.58.** By grade: A n=9 → −5.12R; B n=6 → +2.50R; C n=2 →
+−0.30R.
+
+Reading: the book is two regimes. Everything through the 08-14 rally
+made money (+4.7R across 8 events, incl. the XAU_CP +2.56R); the 08-17
+afternoon BUY/STRONG_BUY cluster chased the spike to ~4425 and got
+stopped as price faded — 9 events, −7.6R combined. One bad afternoon
+dominates the 5-day sample. Notable: a NEUTRAL-graded row carried a
+setup (post-gate CP/PB-style path) and lost small.
+
+## Advisory panel scorecard
+
+Only **7 verdicts** logged so far (2 NEUTRAL, 5 OPPOSE, 0 AGREE) — the
+panel's min-quality gate + daily budget keep it dark on most scans.
+
+| Row | When | Signal | Verdict | Outcome basis | Impact |
+|---|---|---|---|---|---|
+| #1358 | 08-17 07:40 M5 | BUY B | NEUTRAL @55% | +4h move | −$8.19 in dir (pass was right) |
+| #1361 | 08-17 07:45 M5 | BUY A | NEUTRAL @55% | +4h move | −$12.43 in dir (pass was right) |
+| #1550 | 08-17 14:15 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 0.10R |
+| #1553 | 08-17 14:25 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 1.02R |
+| #1555 | 08-17 14:30 M15 | BUY A | OPPOSE @75% | replay | saved 1.03R |
+| #1561 | 08-17 14:45 M15 | STRONG_BUY A | OPPOSE @75% | replay | saved 1.03R |
+| #1588 | 08-17 15:30 M15 | BUY B | OPPOSE @75% | replay | saved 1.02R |
+
+**Every OPPOSE was correct: ≈ +4.2R would have been lost taking those
+five trades.** The panel's rationale ("price climbed too fast, profit
+too small") correctly identified the 08-17 exhaustion spike that the
+composite scored A. Both NEUTRAL passes also preceded adverse moves.
+Caveat: n=7, one market episode, zero AGREE verdicts — the panel has
+only proven itself as a veto so far, never as a confirmation.
+
+## Directional hit-rate (non-setup signals)
+
+27 deduped non-setup BUY/SELL rows, forward close-to-close on M5:
+
+| Horizon | Hit rate | Mean move in direction |
+|---|---|---|
+| +1h | 48% | −$0.72 |
+| +4h | 44% | −$2.30 |
+| +24h | 7% | −$27.45 |
+
+The 24h figure is dominated by the 08-17 buy-cluster reversal; not
+independently meaningful at n=27.
+
+## Method notes / caveats
+
+- **Small sample**: 5 days, 17 events, one trending episode + one
+  reversal. Directional read only; do not tune anything on this.
+- Entry model = next bar open after the scan boundary (live scans run
+  at boundary+10s), SL/TP distances preserved from the logged setup —
+  identical to the backtester replay loop.
+- Cross-source dedup verified: telegram+dashboard double-logs and
+  5-min re-fires of a persisting signal collapse to one event each.
+- The stray repo-root `signals.db` (828 rows, Aug 13–14, pre-panel
+  schema) is an early duplicate subset of `logs/signals.db` — excluded.
+- Tests: Module 25 in `test_all.py` was only added on 2026-09-03 (the
+  2026-08-18 claim of 7 tests was never true at HEAD). It now covers
+  boundary flooring, clustering (shared `signal_log.setup_key`), trade
+  building, the stored-candle replay books, surface parity, WAL-visible
+  loading (`mode=ro`; `immutable=1` missed the newest rows) and hit-rate
+  signing.

backtest/research/signal_parity_2026-08-14.md

@@ -0,0 +1,59 @@
+# Signal-Level Replay Parity — Oanda vs Tiingo (Phase 2)
+
+**Date:** 2026-08-14
+**Tool:** `scripts/signal_parity_study.py` (script removed 2026-09-06, in git history) (replays real `analyze_from_dataframes`, news off)
+**Window:** 2025-08-14 → 2026-08-14 (12 months); M15 stride 8, M5 stride 24 → ~2,900 sampled decisions each
+**Preceding:** Phase 1 price-diff study (`feed_divergence_2026-08-14.md`)
+
+## Headline
+
+On the ~94–96% of bars the two feeds share, **Oanda and Tiingo produce near-identical signals**:
+
+| Metric | M15 | M5 |
+|---|---|---|
+| same direction | 98.32% | 98.16% |
+| same grade | 94.13% | 95.23% |
+| same setup presence | 98.83% | 99.49% |
+| same direction + grade + setup | 93.41% | 94.21% |
+| opposite-sign flips (BUY↔SELL) | **0** | **0** |
+| direction flips on actionable bars | 6.6% | 10.9% |
+| grade changed on actionable bars | 14.2% | 7.8% |
+| mean \|Δ final_score\| | 1.76 | 1.78 |
+| mean \|Δ primary_score\| | 1.73 | 1.77 |
+| same strategy tag | 100% | 100% |
+
+Every direction flip is **actionable↔NEUTRAL**, never a sign reversal: the feeds never disagree on buy-vs-sell, only on *whether* a marginal bar crosses the action threshold. Flipping bars cluster at the Friday-close/Sunday-reopen boundary (UTC hours 22–23) and in the Asian hours (03–04).
+
+## Setup levels diverge even when the signal agrees
+
+On bars where **both** feeds emit a setup with the same tag:
+
+| | M15 (n=119) | M5 (n=39) |
+|---|---|---|
+| entry mean \|Δ\| | $0.12 | $0.10 |
+| SL mean \|Δ\| | $1.83 | $2.79 |
+| TP1 mean \|Δ\| | $2.47 | $1.52 |
+| TP2 mean \|Δ\| | $3.74 | $2.46 |
+| TP1 p95 \|Δ\| | $17.4 | $2.3 |
+
+Entry price is the current close (≈ identical across feeds), but **SL/TP are recomputed from ATR and recent swing extremes, which differ more** — mean SL drift ~$2–3 (p95 up to $10–26 on M15), i.e. 20–40% of ATR. So even an identical signal places stops/targets at meaningfully different prices on a different feed.
+
+## Phase 2 pass/fail vs plan thresholds
+
+| Criterion | Plan threshold | M15 / M5 | Verdict |
+|---|---|---|---|
+| identical direction+grade | ≥ 97% | 93.4% / 94.2% | **marginal** (below 97%, above 93%) |
+| opposite-sign flips | (none specified) | 0 / 0 | PASS |
+| flips clustered at marginal composites | no session/regime clustering | yes (boundary hours) | PASS |
+
+The "identical direction+grade" metric lands slightly under the proposed 97% bar — but the shortfall is grade *letters* (A+ vs A vs B) changing on marginal composites, not direction. If the acceptance criterion is "no opposite-sign disagreement and ~98% direction identity", this passes comfortably.
+
+## Conclusion
+
+Oanda ↔ Tiingo is **signal-equivalent for direction** (98% identical, zero sign flips) over 12 months. The residual deltas are:
+
+1. **~6% of decisions differ in grade letter** on marginal composites (A+↔A↔B near thresholds).
+2. **SL/TP levels drift ~$2–4 mean** (up to ~$20 p95 on M15) even for identical signals — a real P&L difference at the trade level.
+3. **Tiingo's extra Sunday bars** (Phase 1) are gated by the aggregator's NY-anchored weekend filter, so they do not inject phantom signals, but they do slightly shift indicator history at Monday opens.
+
+Net: switching Oanda→Tiingo changes a handful of marginal decisions and moves stop/target levels, but does not change *what the system is doing* (direction, strategy, setup presence). Oanda↔Twelve Data was not replayed — Phase 1 already showed 81% M5 bar-direction disagreement and a ~$3/$1 systematic gap, so a TD replay would trivially fail (that is the documented re-pin motivator).

backtest/research/sizing_campaign_2026-09.md

@@ -0,0 +1,348 @@
+# Stake-sizing campaign — 2026-09-05 (3-year Oanda window, $0.65)
+
+## Purpose and owner decisions
+Owner asked (2026-09-05): predict direction and choose order size better; protect against
+trend exhaustion at signal time; map hours/days where institutions move gold violently;
+backtests ≤ 3 years. Decisions taken in planning:
+- Order size = a per-signal RELATIVE stake ("Stake: 0.7× normal"), not account-based units.
+- Exhaustion protection = RISK LESS, DON'T SKIP (every hard exhaustion block tested since
+  2026-07 removed net-positive trades — the winners are the stretched runs).
+- Model only M5 and M15; H1/H4 measured descriptively.
+- scikit-learn / lightgbm allowed in ./venv for research only; the shipped artefact is
+  numpy-only JSON. Neither library enters requirements.txt.
+- No tuned threshold, gate or exit changes (rule 1). Sizing is a post-decision overlay computed
+  by `position_sizing.build_features` / `stake_multiplier` INSIDE `analyze_from_dataframes`, so
+  live and backtest use the same call on the same objects (rule 4).
+
+## Harness
+`backtest/gpu_backtest.py --pair XAUUSD --start 2023-09-05 --end 2026-09-05 --spread 0.65
+--workers 12`, pair-factory config (no --set/--quality/--strategies/--confirm-tfs), NumPy CPU
+backend (CuPy cannot locate CUDA_PATH on this box; the 2026-09-03 pinned books were also CPU).
+Cache: `oanda_XAU_USD_{5min,15min,1h,4h}_20230705_20260905.parquet`, `…_1d_20221110_20260905`.
+Feature books add `--export-features` (65 `f_*` columns + `stake_multiplier` + `r_sized`; the
+default export stays the pinned 63-column shape). Outputs in `backtest/research/sizing_2026-09/`
+(gitignored).
+
+## Baseline (control books, unmodified backtester, 2026-09-05 23:36–23:41 UTC)
+| TF | trades | total R | WR | PF | max DD |
+|---|---|---|---|---|---|
+| M5 | 857 | +161.32 | 53.7% | 1.53 | 9.85 |
+| M15 | 1086 | +148.69 | 59.1% | 1.37 | 15.47 |
+| H1 | 255 | +70.52 | 64.3% | 1.98 | 4.43 |
+| H4 | 113 | +22.68 | 58.4% | 1.63 | 5.39 |
+
+### Behaviour-neutrality checks (2026-09-05 23:57 UTC)
+- 12-month M5 canonical window (2025-08-14 → 2026-08-14, $0.65) re-run on the modified tree:
+  `book_diff` vs `backtest/oanda_m5_12mo_065_nopb_20260903.csv` = ΔR +0.0, Δtrades +0, and the
+  two CSVs have the SAME MD5 (`a28fed4becd056466c26d6a947e1a0e5`). Record-keeping fields,
+  `SizingConfig`, and the `--export-features` hook change nothing when the flag is off.
+- Feature books: the 63 shared columns are value-identical to the control books on every TF
+  (`assert_frame_equal`). A column-ORDER quirk was found and fixed the same day: pandas orders
+  export columns by first appearance, so when the first trade of a book lacks an indicator
+  (the 3-year M5 book's first trade had no candlestick pattern) that indicator's columns were
+  appended after the extras. `trades_to_dataframe` now forces the extras to the tail; test
+  extended with that exact case (435 tests green).
+
+## Pre-registered acceptance rule (written 2026-09-05, BEFORE any model fit)
+A per-timeframe stake-multiplier model may ship only if, evaluated on OUT-OF-SAMPLE quarters
+only (all calendar quarters after a 12-month burn-in, each predicted by a model trained
+exclusively on trades whose entry time is ≥ 300 primary bars before that quarter's start,
+expanding window, standardisation fit on train only), ALL of the following hold:
+1. Sized total R > flat total R in BOTH halves of the OOS period (split at the median OOS
+   entry time).
+2. Sized total R > flat total R over the full OOS period.
+3. Sized max drawdown (0-prepended cumulative sized-R curve, as `compute_metrics`) ≤ flat max
+   drawdown × 1.10.
+4. Sized OOS book has PF ≥ 1.3 and positive total R in both halves (the rule-1 floor).
+5. Real OOS lift > the 95th percentile of the lift distribution from 50 label-shuffle refits
+   (R permuted within training windows only; test labels and walk-forward geometry unchanged).
+6. Mean multiplier over OOS trades within 1.00 ± 0.05; every multiplier in [0.5, 1.5].
+Plus the mechanical check: the `--sizing` full-window replay reproduces the flat book's
+entry/exit/`r_multiple` columns exactly (sizing changes no entries).
+Model family and size map (fixed): primary = ridge regression on winsorised R (clip to
+[−1.5, +4]); secondary = L2 logistic on R>0 (ships only if the primary fails and it passes);
+simplicity baseline = one-feature model on `f_is_buy` (the shipped model must beat its OOS
+lift, else the simpler one ships if it passes); upper bound = lightgbm on all `f_*` (never
+ships; if its OOS lift is inside the shuffle null, stop); ablation = exhaustion-block-only model.
+Map: `m = 0.5 + 1.0 × clip((z − q10)/(q90 − q10), 0, 1)` with q10/q90 from TRAIN predictions,
+times a normaliser making the mean train multiplier 1.0, clipped to [0.5, 1.5], rounded to 0.05.
+`SHIPPED_FEATURES` (≤ 12 per TF, ≥ 2 from the exhaustion block) are chosen from the Stage-B
+sign-stable screen on the 3-year feature books and recorded below BEFORE the first fit; any
+later change restarts the shuffle control. If any clause fails on a timeframe, that timeframe
+ships nothing. No widening the clip, changing the split, dropping the purge, re-picking
+features or adding quarters to rescue a result.
+
+## Pre-registered feature lists
+Derived MECHANICALLY (script in the session log, saved as `sizing_2026-09/preregistered_features.json`):
+sign-stable `f_*` features from the Stage-B screen (top-minus-bottom quintile mean-R gap has the
+same sign in both halves, both halves ≥ 10 trades per bucket), zero-variance columns removed,
+ranked by min(|gap_h1|, |gap_h2|), top 12; the ≥ 2-from-the-exhaustion-block clause was already
+satisfied on both TFs. Written 2026-09-06 00:05 UTC, before any Stage-D fit.
+- **M5 (29 sign-stable candidates):** f_sl_atr, f_confidence, f_rsi, f_mtf_c1, f_swing_dist_atr,
+  f_ret_48_atr, f_tag_no_pullback, f_rsi_dir, f_first_friday, f_mtf_agreement,
+  f_exhaustion_count, f_ret_24_atr. Exhaustion-block members: f_swing_dist_atr,
+  f_tag_no_pullback, f_exhaustion_count (all three run POSITIVE in both halves).
+- **M15 (37 sign-stable candidates):** f_rsi, f_bb_pos, f_day_of_month, f_tag_ema200_near,
+  f_atr_pct, f_is_buy, f_confidence, f_close_pos_in_bar, f_tag_weak_pullback, f_tag_exhaustion,
+  f_penalty_total, f_ret_6_atr. Exhaustion-block members: f_tag_exhaustion, f_penalty_total
+  (both NEGATIVE in both halves; f_tag_weak_pullback and f_tag_ema200_near also negative).
+Exhaustion-only ablation list (fixed by the plan): f_swing_dist_atr, f_ema21_dist_atr,
+f_ema200_dist_atr, f_exhaustion_count, f_penalty_total, f_tag_bb_overextended,
+f_tag_rsi_extended, f_tag_no_pullback, f_tag_exhaustion.
+
+## Stage B — book breakdown (3-year feature books)
+Tool: `backtest/research/book_report.py` (outputs in `sizing_2026-09/report_{5min,15min,1h,4h}/`).
+Halves = median TRADE split (M5 cut 2025-12-12 because trade density doubled in the last year;
+M15 cut 2025-04-25; H1 2025-04-09).
+
+**M5 (857 trades, +161.3R, PF 1.53, DD 9.85; halves +51.0R PF 1.33 / +110.3R PF 1.74; max 11
+wins / 8 losses in a row; longest flat stretch 88 trades).** Mix: GENERIC 469, XAU_CP 388, BUY
+765 / SELL 92 (SELLs +11.8R, positive both halves). Grade: A +0.26R/trade (227) > B +0.17 (608)
+> C −0.01 (21). Session (decision hour): NY 17–22 best (+0.31R/trade, PF 1.88), overlap 13–16
+weakest (+0.08, PF 1.22) — both stable. Weekday: Wed and Fri best and stable; Mon/Tue flip sign.
+Feature screen: 29 sign-stable `f_*` features. **Every stretch measure is POSITIVE**: wider
+stop in ATR (+0.22/+0.23 gap), higher RSI (+0.20/+0.19), farther from the last swing
+(+0.18/+0.27), 48-bar run-up (+0.18/+0.20), the `no_pullback` tag (+0.17/+0.28), more exhaustion
+tags (+0.12/+0.24). Only ATR-relative-to-500-bar-mean is negative (−0.11/−0.12: entries during
+a volatility spike do worse). "Risk less when stretched" has NO support on M5 — a model that
+honours the data will size stretched M5 trades UP, not down.
+
+**M15 (1086 trades, +148.7R, PF 1.37, DD 15.47; halves +65.8R / +82.9R; max 12 wins / 8 losses;
+flat stretch 131 trades).** All GENERIC; BUY 739 (+158.4R, PF 1.62) vs SELL 347 (−9.7R, PF 0.93;
+−19.2R in half 1, +9.4R in half 2). Grade NOT monotone: C +0.45R/trade (82, PF 2.41) > A +0.14
+(481) > B +0.08 (516) — stable in both halves (matches the 2026-08-04 quant review). Session:
+Asian and NY ≈ +0.18–0.20R/trade, London and overlap ≈ +0.09–0.10 (all stable). Weekday:
+Wednesday flat/negative in both halves (229 trades, −0.6R); Thu/Fri flip sign between halves.
+Feature screen: 37 sign-stable features. Here the exhaustion block runs NEGATIVE: `exhaustion`
+tag (−0.19/−0.10), `weak_pullback` tag (−0.24/−0.11), `ema200_near` tag (−0.17/−0.32), penalty
+total (−0.16/−0.10), ATR percentile (−0.20/−0.15); while raw RSI (+0.31/+0.24) and Bollinger
+position (+0.36/+0.22) are positive. So on M15 "stretched by the aggregator's own penalty
+logic" is bad while "strong momentum reading" is good — the sizing model gets both.
+
+**H1 (255 trades, +70.5R, PF 1.98, DD 4.43; halves +28.7R PF 1.79 / +41.9R PF 2.17)** and
+**H4 (113 trades, +22.7R, PF 1.63, DD 5.39)** are descriptive only. H1 is all-BUY by
+construction (shorts blocked 2026-09-03); Mon/Thu strongest, Friday the only losing weekday
+(−1.4R/50, flips between halves); 03:00 UTC decision hour is the best single hour (+20.0R/40,
+stable) and 04:00–05:00 negative.
+
+**Labelling quirk (pre-existing, not fixed):** 33 M5 trades carry the backtester `strategy`
+tag GENERIC while their setup strategy is XAU_ATR (`f_strat_atr = 1`); the CSV tag and the
+setup's own string disagree for the ATR path. Labels only — R is unaffected.
+
+**Stop-trigger check: PASSED** (both TFs have many sign-stable features) → proceed to Stage D.
+
+## Stage C — institutional burst / time map
+Tool: `backtest/research/burst_map.py` on the 3-year raw bars (`burst_5min/`, `burst_15min/`
+under `sizing_2026-09/`). Burst = bar range ≥ K × prior Wilder ATR(14). Base rate at K=3:
+0.86 % of M5 bars (1840), 1.19 % of M15 bars (846).
+
+**Where the violent bars cluster (bar-CLOSE hour, UTC, K=3 burst % of bars):**
+- 12:00–13:00 (US data window): M5 2.4 % / 2.3 %, M15 3.7 % / 5.0 % — 3–4× the base rate;
+  Friday 12:00–13:00 is the densest weekday cell on M5 (3.4 % / 3.1 %). Largest mean 1-hour
+  dollar moves are at close-hours 13–15 (M5: $7.8 / $9.4 / $8.5 vs ≈$5 elsewhere).
+- 01:00 (Asia open): M5 2.9 %, M15 3.3 %. Also 05:00 (Tokyo/London hand-over) ≈1.2 % on M5.
+- 22:00–23:00 (post-settlement thin book): ≈1.3–1.4 % on M5; Sunday reopen 22–23 h is the most
+  burst-prone slot of all (2.4–3.0 % of bars) but the calendar already blocks Sunday entries
+  before 18:00 NY.
+- Quietest hours: 03:00–04:00 and 16:00–17:00 (0.14–0.3 %).
+
+**Post-burst drift (in the burst's own direction, ATR units):** ≈ 0 at every horizon on both
+TFs (M5 6/12/24/48-bar means −0.01/−0.01/+0.09/+0.05; M15 −0.08/+0.14/+0.18/+0.17; % positive
+46–51 %). London-session bursts on M5 show a mild short-horizon reversal (42 % positive at 6
+bars). No continuation or reversal edge worth a rule.
+
+**Do bursts hurt the system's entries? No.** Trades DECIDED within 12 bars after a K=3 burst:
+M5 149 trades, mean +0.199R, PF 1.62 vs +0.186R / PF 1.52 for the rest; M15 242 trades,
++0.160R / PF 1.47 vs +0.130R / PF 1.34. At K=4/5 the after-burst group is better still
+(M5 +0.276R PF 1.98; M15 +0.259R PF 1.80). This re-confirms the 2026-07-30 big-candle-gate
+rejection with 3 years of data: expansion bars are part of the entry signal, not a danger.
+(Group (b) "burst while open" is survivorship-shaped — winners live longer — and is not evidence.)
+
+**The 40 largest 1-hour moves** are almost all one episode (2026-01-28 → 2026-02-06, incl. a
+−$394 hour at 15:45 UTC on 2026-01-29, and 2026-03-23), not recurring clock slots; only 3 of 40
+fall in the 12:30 data slot and 1 in the Wednesday 18:00 central-bank slot (slot labels only —
+no economic calendar file exists in the repo). Violent moves are episodic regime events; the
+recurring, predictable clustering is the 12:00–15:00 UTC US-data window and the 01:00 Asia open.
+
+**Use in this campaign:** `f_bars_since_burst` and the hour/session features are already in the
+feature set, so Stage D can weigh them as a stake input; no entry gate is proposed (rule 1).
+
+## Stage D — walk-forward results (2026-09-06 00:20 UTC)
+Driver: `backtest/research/sizing_model.py` (numpy closed-form ridge + Newton logistic; lightgbm
+4.7 + scikit-learn 1.9 installed in ./venv for the upper bound only; NOT in requirements.txt).
+Outputs: `sizing_2026-09/sm_{m5,m15}_{ridge,logistic}/` (summary.txt, oos_trades.csv,
+quarters.csv, coefs_by_quarter.csv, shuffle_lifts.csv). Validation of the driver: planted
+synthetic effect recovered (lift +116.7R, shuffle rank 100 %, all clauses PASS); pure-noise
+book correctly FAILS (rank 32 %); ridge matches `lstsq` to 2e-16; logistic gradient < 3e-15.
+
+**M5 (5 OOS quarters 2025Q3→2026Q3, 619 trades — 2024Q4–2025Q2 skipped by the pre-registered
+min-train rule because only 96/119/142 trades precede them; 2026Q1 alone is +92R of the +125R
+flat OOS):**
+| model | flat R | sized R | lift | PF | DD | mean m | shuffle p95 | rank | verdict |
+|---|---|---|---|---|---|---|---|---|---|
+| ridge (primary) | +125.41 | +121.98 | −3.43 | 1.63 | 10.69 | 0.891 | +13.69 | 48 % | FAIL (1,2,5,6) |
+| logistic | +125.41 | +128.40 | +2.99 | 1.66 | 8.70 | 0.900 | +14.39 | 62 % | FAIL |
+| isbuy baseline | +125.41 | +123.22 | −2.20 | 1.55 | 11.92 | 1.000 | +7.05 | 4 % | FAIL |
+| exhaustion-only | +125.41 | +118.03 | −7.38 | 1.59 | 10.88 | 0.918 | +14.36 | 28 % | FAIL |
+| lightgbm (all 65) | +125.41 | +116.05 | −9.36 | 1.51 | 11.64 | 1.007 | +7.60 | 10 % | FAIL |
+Ridge weights (standardised, 5 quarters): `f_tag_no_pullback` +0.018 sign-stable 5/5 — the
+"no pullback" flag INCREASES the stake; `f_swing_dist_atr` and `f_exhaustion_count` ≈ 0 and
+sign-unstable. Exhaustion-only sizing LOSES 7.4R. **A clean null: nothing in the 65 features
+predicts M5 trade outcomes beyond noise, linear or non-linear.**
+
+**M15 (8 OOS quarters 2024Q4→2026Q3, 762 trades; flat +115.93R PF 1.42 DD 13.73):**
+| model | sized R | lift | PF | DD | mean m | shuffle p95 | rank | clauses |
+|---|---|---|---|---|---|---|---|---|
+| ridge (primary) | +142.04 | +26.11 | 1.53 | 14.46 | 0.980 | +28.50 | 88 % | 1,2,3,4,6 PASS · **5 FAIL** |
+| logistic | +144.78 | +28.84 | 1.53 | 13.97 | 1.002 | +31.89 | 92 % | 1,2,3,4,6 PASS · **5 FAIL** |
+| isbuy baseline | +139.38 | +23.45 | 1.50 | 16.44 | 1.007 | +24.47 | 58 % | FAIL (3, 5) |
+| exhaustion-only | +123.83 | +7.90 | 1.45 | 14.85 | 1.010 | +19.77 | 70 % | FAIL |
+| lightgbm (all 65) | +123.87 | +7.94 | 1.44 | 14.74 | 0.996 | +17.66 | 60 % | FAIL |
+Ridge weights sign-stable in 8/8 quarters: `f_tag_exhaustion` −0.022, `f_tag_ema200_near`
+−0.033 (both SHRINK the stake — the "risk less when stretched" direction), `f_rsi` +0.028,
+`f_bb_pos` +0.030, `f_is_buy` +0.029, `f_close_pos_in_bar` +0.022, `f_ret_6_atr` +0.022;
+λ = 1000 every quarter (heavy shrinkage — the score is weak and the map is essentially a rank
+transform). Sized R beat flat in 6 of 8 quarters and in both halves, BUT the label-shuffle
+null has p95 = +28.5R: a random re-weighting of these 12 pre-screened features produces the
+same lift 12 % of the time, and the one-feature BUY/SELL baseline already delivers +23.5R of
+the +26.1R. **The model's information beyond "buys are better than sells" (≈ +2.7R over 762
+trades) is indistinguishable from noise.**
+
+## Verdict (pre-registered rule applied, no rescue attempted)
+**FAIL on both timeframes → no stake multiplier ships.** No artifact, no `sizing.enabled`, no
+alert line, no `--sizing` backtester flag. What remains in the tree is behaviour-neutral
+research plumbing: `position_sizing.build_features` (65 decision-time features; the multiplier
+path returns 1.0 and no artifact exists), `SignalReport.penalty_tags/penalty_total/
+momentum_fading` (record-keeping), `TradeSetup.stake_multiplier` (None; reserved), the
+`--export-features` flag, `book_report.py`, `burst_map.py`, `sizing_model.py`, and Module 29
+tests. `SizingConfig` was REMOVED again before merge (rule 8: nothing reads it).
+
+**What the owner should take from this (plain language):**
+1. On the 5-minute chart there is nothing to size on. Stretched trades are the good ones; the
+   system's exhaustion flags carry no usable information about the outcome; a flexible model
+   given all 65 facts does worse than flat sizing out of sample.
+2. On the 15-minute chart the only robust sizing signal is BUY vs SELL, which the gates already
+   act on. The exhaustion flag does point the "risk less" way in every one of 8 unseen
+   quarters, but the money it would make (≈ +3R per year) is inside noise at 1,086 trades.
+3. "Predict direction better" with a learned second opinion is closed at this sample size:
+   the strongest non-linear model (all features, gradient boosting) fails on both charts.
+
+**Open avenues (not rescues — each needs a NEW pre-registration):**
+- Re-run the M15 exhaustion-tag sizing when the book has ≈ 2× the trades (≈ 2028) or on a
+  sealed forward sample; the sign is stable, the magnitude is not proven.
+- The shuffle null is wide because the size map rank-stretches a weak score to 0.5–1.5×; a
+  narrower map (e.g. 0.75–1.25×) would shrink both lift and null — pre-register before testing.
+- Session and weekday structure (M5: NY best / overlap weakest; M15: Wednesday flat) is stable
+  across halves but was rejected as a gate repeatedly; as a *size* input it was inside the
+  12-feature model and did not survive the null.
+
+## Addendum 2026-09-06 — violent bars: signs before them, and protecting OPEN trades
+Owner follow-up: "we might have an order running for hours before a violent move; we want
+protection." Tools: `backtest/research/burst_precursors.py` (signs before a burst) and
+`backtest/research/open_trade_bursts.py` (exact replay of every book trade on raw bars with the
+backtester's exit model — reproduces `r_multiple` on 100 % of trades — plus protection variants).
+Outputs: `sizing_2026-09/burst_precursors_{m5,m15}.txt`, `open_trade_bursts_{m5,m15}.txt`.
+
+**Signs before a burst (both TFs, both halves stable):** the clock (close-hours 01, 12, 13 UTC
+hold ~40 % of bursts in 13 % of bars); tick activity in the last 3 bars vs 50-bar mean (top
+quintile ≈ 2× rate); the last bars already wider than normal (top quintile ≈ 2.2×; the
+tightest quintile ≈ 0.3–0.5× — NOT calm-before-storm at bar scale); a move already under way
+over the past hour (≈ 1.6×); a burst minutes ago (≈ 1.4×). Out of sample (fit half 1, score
+half 2) a logistic model reaches AUC 0.75–0.77 for the next bar; the top 5 % of moments carry a
+3.4–5.5 % burst chance vs 0.8–1.1 % base, and 24 % vs 9 % (M5) for "within the next hour".
+Burst direction continues the prior hour's drift 61 % of the time. Signs say "ripe", not "now".
+
+**Damage to open trades (base exits):** M5 — 164 trades (19 %) meet an opposing burst while
+open: mean open R +0.42 just before → −0.06 just after → final +0.01; 43 % are stopped on that
+bar; 54 % end as losers. M15 — 246 trades (23 %): +0.27 → −0.20 → final −0.12; 39 % stopped on
+the bar. Trades already in profit before the burst still finish positive on average (M5 +0.53,
+M15 +0.27). Bursts WITH the trade (n 176 / 246) finish at +1.11 / +0.83R — they are where the
+winners come from.
+
+**Protection variants — ALL negative on both TFs, both halves:**
+| variant | M5 ΔR (h1/h2) | M15 ΔR (h1/h2) |
+|---|---|---|
+| flatten before US-data hours (12–13 UTC) | −31.9 (−13.6/−18.3) | −103.0 (−67.4/−35.6) |
+| flatten before Asian open (00–01 UTC) | −38.7 (−22.0/−16.7) | −73.0 (−42.0/−30.9) |
+| protective 0.5R stop through the data hours | −29.6 (−12.3/−17.3) | −82.3 (−55.5/−26.8) |
+| protective 0.5R stop through the Asian open | −19.2 (−9.5/−9.7) | −35.1 (−17.3/−17.8) |
+| exit next open after an opposing burst | −19.1 (−6.5/−12.5) | −6.9 (−11.9/+5.0) |
+| 0.5R protective stop after an opposing burst | −9.0 (−3.0/−6.0) | −8.3 (−15.5/+7.3) |
+The existing stop/trail already contains the opposing-burst damage (≈ 0.4–0.5R per hit); every
+pre-emptive rule removes far more from the with-trade bursts than it saves. **Verdict: no
+open-trade protection rule ships; do not re-propose flatten/tighten-before-window or
+react-to-burst exits without a new mechanism.** Display-only "violent-move conditions" note on
+alerts is the only harmless use of the precursor signs.
+
+## Addendum 2026-09-06 — calendar events, indicator states and time slots before violent bars
+Tools: `backtest/research/calendar_impact.py` (needs `backtest/cache/calendar.parquet`, scraped
+2023-09-01→2026-09-10 with `ff_calendar_scraper.py`: 15,124 events, 1,055 high-impact USD in
+window) and `backtest/research/burst_indicator_map.py` (states from the backtester's own
+`precompute_indicators`, taken at t−1). Outputs `sizing_2026-09/calendar_impact_{m5,m15}.txt`,
+`burst_indicator_map_{m5,m15}.txt`. Burst = range ≥ 3× prior ATR14; lifts quoted are stable in
+BOTH halves on both TFs unless noted.
+
+**Calendar.** The M5 bar containing a high-impact USD release averages 4.4× normal range; 54 %
+of such releases produce a burst within the hour (base 9 %). By event: Non-Farm Payrolls (with
+Unemployment Rate / Avg Hourly Earnings) 97 %, 9.3× bar, $11.9 mean 1-h move; CPI 91 %, 8.3×,
+$9.0; FOMC decision/statement 87.5 %, 7.1×, $18.2 (press conference a further $20 hour); FOMC
+projections 92 %; Retail Sales 64 %; PPI 54 %; Core PCE 50 %; Unemployment Claims 44 %; GDP
+45–58 %; ISM PMIs 26 %; Powell speaks 33 %; Trump speaks 21 %; UoM sentiment ≈ 5 %. 20 % (M5)
+/ 31 % (M15) of ALL bursts fall within ±30 min of a high-impact USD release (3.7–4.2 % of time);
+in the 30 min AFTER one, next-bar burst odds are 8–17 % (7–16× base); 15 min before, 2–3 %.
+Open trades that held through a high-impact release did BETTER (M5 +0.41R vs +0.16R; M15 +0.27
+vs +0.07), even when the first 15-min move went against them (M5 +0.09, M15 +0.22). Flattening
+5 min before a release: −5.4R (M5) / −59.3R (M15); 0.5R protective stop through the hour:
+−2.4R / −43.1R. **Events are the biggest single cause of violent bars and still not a reason
+to touch an open trade.**
+
+**Indicator states before a burst (next-bar lift, M5 / M15).** Price OUTSIDE the Bollinger
+bands: below 2.9× / 1.8–2.9×, above 2.1× / 2.2×, middle 0.2–0.8 band 0.7×. RSI < 30: 2.6–2.9× /
+1.8–2.2×; RSI ≥ 70: 1.9–2.1× / 1.6–1.9×; RSI 40–60: 0.7× (calmest). Close outside the Keltner
+channel: 1.4–2.2×; inside 0.76×. ADX RISING > 3: 1.4–1.5× / 1.2×; ADX FALLING < −3: 0.5–0.6×
+(safest state); the ADX level itself carries nothing. MACD histogram negative and still falling
+1.3× / 1.2–1.4×; histogram fading (either sign) 0.8×. Fully bearish 4-EMA stack 1.3–1.4× /
+1.3×; "mostly bull" 0.8× on M5. Regime: transitioning 1.4–1.6× / 1.5–1.6×, ranging 0.88×.
+Volatility regime: extreme 1.5–1.6× / 1.4–1.9×, low 0.6–0.8×. ROC beyond ±0.3 %: 1.2–1.9×
+(>1 %: 1.3–10×, n small). Bollinger-width squeeze: 0.59× on M5 (LESS likely next bar — narrow
+bands are not a warning); TTM squeeze (bands inside Keltner) 1.6× on M15 only. EMA200 distance,
+day of month, first-Friday-as-a-day: nothing stable (first Friday matters only through its
+12:30 slot).
+
+**Time slots (M5 next-bar burst %, base 0.86 %):** Fri 01:xx 3.4 %, Fri 12:xx 3.4 %, Tue/Wed
+01:xx 3.2 %, Thu 12:xx 3.1 %, Fri 13:xx 3.1 %, Sun 22:xx reopen 3.0 %, Thu 01:xx 2.7 %, Wed
+12:xx 2.6 %, Sun 23:xx 2.4 %, Tue 00:xx 2.3 % (all 2.5–3.9× base). Quietest: 03–04 and 16–17.
+
+**Direction of the burst (share going UP):** fully-bull stack 62 % (M5) / 50 % (M15); fully-bear
+stack 26 % / 29 %; RSI < 30 → 22 % / 30 % up (i.e. 70–78 % DOWN); RSI ≥ 70 → 64 % / 52 % up;
+MACD neg & falling → 30 % / 34 % up; trending_down regime → 26 % / 26 % up; trending_up → 60 % /
+46 % up. Bearish states foretell a DOWN burst reliably on both TFs; bullish states foretell an
+UP burst on M5 only.
+
+**Reading:** violent bars arrive when price is already at an extreme with momentum accelerating
+(outside the bands, RSI extreme, ADX rising, MACD still falling, transitioning regime, extreme
+volatility) and/or a release is minutes away — "the storm is already brewing", not "calm before
+the storm". Calm mid-band bars with fading momentum and falling ADX are the safest. None of it
+changes trade handling (protection tests above); it is display material for a warning line.
+
+## Addendum 2026-09-06 — violent-move WATCH rule (display-only feature, `violent_watch.py`)
+Owner asked for a warning. Delivery: a line on signal alerts + a standalone heads-up to the owner
+(once per hour per trigger set, daily cap), dark by default (`violent_watch` config block).
+The first draft rule ("hot" = calendar | ≥3 indicator readings | clock & ≥2 readings) was
+evaluated bar-by-bar on 3 years (`backtest/research/watch_rule_eval.py`, scratch variant
+sweep): it flagged 27 % of bars hot at only ≈2× odds and would have sent ≈13 heads-ups/day on
+M5 — rejected. Sweep of 11 variants (M5 / M15, both halves):
+| hot definition | bars hot | next-bar lift h1/h2 | next-hour rate | hot hours/day |
+|---|---|---|---|---|
+| calendar only (hi-impact USD ±30 min) | 3.4 % | 6.7×/4.4× · 9.1×/6.4× | 31 % · 28 % | 2.6 |
+| clock slot only | 18.8 % | 3.1×/2.0× · 3.6×/2.6× | 16 % · 13 % | 5.3 |
+| ≥2 price-extreme readings | 14.6 % | 2.3×/2.2× · 1.8×/2.0× | 11 % · 6 % | 10.7 |
+| **calendar ∨ (clock ∧ ≥1 price-extreme reading)** | **8.6 %** | **4.9×/3.1× · 5.9×/4.2×** | **21 % · 17 %** | **4.1 · 3.5** |
+| draft (cal ∨ readings≥3 ∨ clock∧readings≥2) | 26.9 % | 2.4×/2.0× · 2.4×/2.1× | 13 % · 8 % | 13.0 |
+**Shipped rule:** price-extreme readings = outside the Bollinger bands, RSI < 30 or ≥ 70, close
+outside the Keltner channel (the neutral readings — ADX rising, extreme volatility, transitioning
+— are dropped from the level logic; they are common and only ≈1.5×). `hot` = high-impact USD
+release within 30 min or 30 min after, OR a hot clock slot with ≥ 1 price-extreme reading; `watch`
+= clock slot alone or ≥ 2 price-extreme readings (≈2× odds). Direction sentence from the bear/bull
+polarity of the readings plus MACD acceleration and trend regime ("DOWN" on both TFs, "UP" on M5
+only). Heads-up only at `hot`. Expected ≈4 hot hours/day → ≤4 heads-ups/day after per-hour dedup.

backtest/research/weight_test_2026-09-09.md

@@ -0,0 +1,187 @@
+# Weight-change test campaign — 2026-09-09 (does the recorded evidence improve predictions?)
+
+## Purpose
+
+First use of the replay-recording files (`replay_recording_2026-09-09.md`): take candidate
+changes to the regime weight table suggested by the per-bar evidence, rerun the FULL backtest
+with each, and compare against the same-harness control books. This is the "before versus
+after" that measures improvement. Everything below the rule was written before any candidate
+was run.
+
+## Harness (identical to the recording campaign)
+
+`backtest/gpu_backtest.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 --spread 0.65
+--workers N --primary-tf {5min|15min}` + `--set technical.regime_weights_override=<full table>`
+(the field replaces the whole table, so every candidate passes the complete table with one
+cell family changed). Controls: `backtest/research/replay_2026-09/{m5,m15}/book_control.csv`
+(same code, no override). OOS window: `--start 2024-06-01 --end 2025-03-30` (cached
+`oanda_XAU_USD_*_20240331_20250330.parquet`), control produced on the same harness.
+Backend NumPy CPU. Code: git `af06685`.
+
+## Pre-registered candidate-selection rule (written 2026-09-09 before the screen ran)
+
+1. A candidate changes ONE indicator's regime multipliers only (no code, no gate, no
+   threshold change), on ONE timeframe. Allowed forms: multiplier → 0 in all regimes,
+   × 0.5 in all regimes, × 2 in all regimes, or the same applied to a single regime family
+   (trending / ranging / transitioning) when the screen shows the effect is regime-specific.
+2. The screen (scratch script over `bars.parquet`, both TFs): for each candidate, recompute
+   every analyzed bar's PRIMARY composite from the recorded effective weights and
+   contributions, find the bars that would NEWLY clear ±15 and the bars that would DROP OUT,
+   and score the candidate by the net drift-adjusted signed 4-hour forward move
+   (gained bars minus lost bars; drift = the mean unsigned 4-hour move of all analyzed bars
+   in the same regime). Top 3 per timeframe by that score are run; nothing else.
+3. Earlier verdicts on the same knob are recorded next to the candidate but do not veto it
+   (this harness — 2-year Oanda window, post-2026-09-07 outlier rule — is new).
+
+## Pre-registered acceptance rule (vs the same-harness control)
+
+A candidate PASSES only if ALL of the following hold on the 2-year window at $0.65:
+- a. ΔR ≥ +5R overall versus the control (smaller differences are noise on ~800 trades);
+- b. ΔR > 0 in BOTH fixed halves: 2024-09-05→2025-09-05 and 2025-09-05→2026-09-05;
+- c. PF ≥ 1.3 and not more than 0.05 below the control's PF;
+- d. max drawdown not more than 2R worse than the control's;
+- e. trade count within ±25% of the control's (guards against "fewer trades" artefacts);
+- f. OOS window 2024-06-01→2025-03-30: ΔR ≥ 0 versus the OOS control.
+Failing candidates are recorded and closed. No rescue: no multiplier fine-tuning after
+seeing a result, no second window shopping. A passing candidate is a SHIPPING PROPOSAL for
+the owner (rule 2: it would land as a per-timeframe `regime_weights_override` in
+`pair_signals/xauusd.py` with tests), not an automatic change.
+
+## Screen results (scratch script over `bars.parquet`, both TFs; composite reproduced from the stored columns within 3.4e-6 on every bar)
+
+Drift per regime (mean 4 h move of all analyzed bars, ATR units, M5): trending_up +0.55,
+trending_down +0.16, trending +0.35, transitioning +0.64, ranging +0.38 — a strong up-drift
+that every BUY vote inherits, hence the per-regime adjustment.
+
+Standalone votes after drift removal (mean signed 4 h move; positive = the vote pointed the
+right way), M5 highlights:
+
+| indicator | regime family | buy votes | sell votes | reading |
+|---|---|---|---|---|
+| macd | trending | −0.079 (n 8,717) | −0.154 (n 8,891) | wrong way on BOTH sides; also at 24 h (−0.22 / −0.56) |
+| macd | transitioning | +0.273 (1,738) | +0.288 (1,885) | best caller in that regime |
+| rsi | transitioning | −1.089 (357) | −0.146 (570) | wrong way, small n |
+| bollinger | transitioning | −0.261 (2,668) | −0.337 (2,675) | wrong way |
+| bollinger | trending | +0.084 | +0.091 | mildly right |
+| ema_trend | ranging / transitioning | +0.33 / +0.42 | ≈0 | buy side right, sell side nothing |
+| adx | trending | +0.048 | +0.041 | never fires outside trending |
+
+Candidate screen (net drift-adjusted 4 h move of gained minus lost bars; the 24 h column must
+agree in sign for a candidate to be run). No candidate flips a bar's direction — one
+indicator's weight only moves bars in and out of the neutral band.
+
+M5 top of the board: `macd:zero_trending` +1235 (24 h +4181, agrees; 3,406 gained / 7,339 lost,
+touches 23% of directional bars), `macd:double_ranging` +1186 (24 h −21, rejected),
+`divergence:zero_all` +1026 (24 h −633, rejected), `bollinger:zero_ranging` +940 (24 h −728,
+rejected), `bollinger:zero_all` +800 (24 h −1654, rejected), `macd:zero_all` +705 (24 h +1644,
+agrees; touches 50% of directional bars), `macd:half_trending` +689 (24 h +334, agrees).
+Bottom: `ema_trend:zero_all` −389, `macd:double_trending` −540, `macd:zero_transitioning` −607.
+
+M15 top of the board: `ema_trend:double_all` +329 (24 h +1209, agrees), `macd:zero_all` +293
+(24 h −79, rejected), `divergence:zero_all` +264 (24 h −480, rejected),
+`divergence:zero_trending` +246 (24 h +390, agrees), `rsi:half_all` +221 (24 h +99, agrees).
+Bottom: `macd:double_trending` −337, `macd:double_all` −323.
+
+Candidates run (rule 2 above, 24 h-sign filter applied):
+- M5-1 `macd:zero_trending` — MACD multiplier 2.0 → 0 in trending_up/trending_down/trending.
+- M5-2 `macd:zero_all` — MACD multiplier → 0 in every regime.
+- M5-3 `macd:half_trending` — MACD multiplier 2.0 → 1.0 in the trending family.
+- M15-1 `ema_trend:double_all` — ema_trend multiplier × 2 in every regime.
+- M15-2 `divergence:zero_trending` — divergence multiplier 1.0 → 0 in the trending family.
+- M15-3 `rsi:half_all` — RSI multiplier × 0.5 in every regime.
+Prior verdicts on the same knobs: none for MACD or ema_trend; "divergence-down" on M5 was
+rejected 2026-08-04 and "divergence-off" on M15 was declined 2026-09 (R up, 5-yr DD 35.8 →
+47.4R); neither is the same change as M15-2 (trending family only).
+
+Half of each board's top entries fail the 24 h check, some violently (`bollinger:zero_all` on
+M5: +800 at 4 h, −1654 at 24 h) — a 4 h-only screen is a coin flip. Scores are sums, so
+volume can carry them: `macd:zero_all` scores by removing 16,170 directional bars whose mean
+was −0.07, while its gained bars average −0.09.
+
+## Rerun results (2-year window 2024-09-05→2026-09-05, $0.65; halves fixed at 2025-09-05; OOS 2024-06-01→2025-03-30)
+
+| candidate | trades | total R | ΔR | H1 ΔR | H2 ΔR | PF | max DD | OOS ΔR | verdict |
+|---|---|---|---|---|---|---|---|---|---|
+| M5 control | 781 | +146.07 | | | | 1.52 | 10.1 | (86 / +9.47, PF 1.33) | |
+| M5-1 macd → 0 in trending | 808 | +105.35 | −40.7 | +1.9 | −42.6 | 1.35 | 17.1 | +0.2 | FAIL a,b,c,d |
+| M5-2 macd → 0 everywhere | 699 | +99.79 | −46.3 | +0.5 | −46.8 | 1.38 | 17.2 | −1.8 | FAIL a,b,c,d,f |
+| M5-3 macd × 0.5 in trending | 804 | +136.12 | −10.0 | +7.4 | −17.4 | 1.47 | 15.2 | +6.8 | FAIL a,b,c,d |
+| M15 control | 793 | +117.62 | | | | 1.41 | 14.8 | (300 / +40.59, PF 1.38) | |
+| M15-1 ema_trend × 2 everywhere | 1229 | +148.20 | +30.6 | +10.0 | +20.5 | 1.32 | 26.4 | +0.7 | FAIL c,d,e (+55% trades) |
+| M15-2 divergence → 0 in trending | 861 | +127.69 | +10.1 | −3.0 | +13.1 | 1.41 | 18.4 | +3.6 | FAIL b,d |
+| **M15-3 rsi × 0.5 everywhere** | **931** | **+146.14** | **+28.5** | **+27.9** | **+0.6** | **1.43** | **16.1** | **+20.1 (355 / +60.67, PF 1.49)** | **PASS a–f** |
+
+The M5 screen was wrong in every case: the three MACD candidates that scored best on the
+per-bar screen (and were the only ones whose 24 h score agreed) lose 10–46R in the full
+replay, all of it in the second year. The per-bar composite screen sees threshold crossings;
+it does not see that MACD's "wrong-way" votes in trending regimes are doing useful work
+downstream (the exhaustion penalties, the M5 min-score gate and the setup generators all sit
+between the composite and a trade). Lesson recorded: the screen ranks hypotheses, the replay
+judges them, and on M5 the two disagreed 3/3.
+
+### M15-3 trade diff (control → candidate, 2-year)
+
+- 721 shared entries: control +124.7R → candidate +119.4R (56 shared trades changed R because
+  the score moved their grade and therefore the setup distances).
+- 210 added trades: +26.8R, mean +0.127R, WR 59.5% — 157 BUY (+32.5R) and 53 SELL (−5.8R);
+  156 are B-grade (+19.1R), 201 sit in the 25–40 score band; by regime ranging 63 (+13.1R),
+  trending_up 88 (+12.1R), transitioning 32 (+2.9R), trending_down 27 (−1.3R).
+- 72 removed trades: −7.1R, WR 50%.
+- By year: 2024 (Sep–Dec) +19.4 → +26.7R, 2025 +51.9 → +77.4R, 2026 (Jan–Sep) +46.3 → +42.1R.
+  The 2-year gain is concentrated in 2025; the fixed second half passes rule b by only +0.6R.
+- OOS (2024-06→2025-03): 300 → 355 trades, +40.6 → +60.7R, PF 1.38 → 1.49, WR 59.3 → 60.8%.
+
+Mechanism in plain terms: RSI is neutral on most M15 entries and, when it votes, it votes
+against the trend (a mean-reversion reading). Halving its weight lets more trend-following
+entries clear the score band, mostly BUYs, in a data set that is a gold bull market
+throughout. The OOS window improves in quality (PF, win rate), not only in count, which is
+the strongest single fact in the candidate's favour; the year-by-year unevenness is the
+strongest fact against.
+
+### Robustness check on the 3-year window (owner's maximum; 2023-09-05→2026-09-05, cached bars)
+
+| book | trades | total R | WR | PF | max DD |
+|---|---|---|---|---|---|
+| M15 control 3-yr | 1092 | +151.90 | 59.4% | 1.38 | 15.5 |
+| M15-3 rsi × 0.5, 3-yr | 1273 | +186.61 | 59.4% | 1.39 | 19.3 |
+| extra year only (2023-09-05→2024-09-05, never seen by the screen): control | 299 | +33.23 | 56.9% | 1.29 | 14.9 |
+| extra year only: candidate | 342 | +39.43 | 56.4% | 1.29 | 19.3 |
+
+Calendar years, control → candidate: 2023 (Sep–Dec) +10.1 → +13.2R; 2024 +42.6 → +52.9R;
+2025 +51.9 → +77.4R; 2026 (Jan–Sep) +47.3 → +43.2R. Three of four years improve; the most
+recent one does not, and the 3-year max drawdown is 3.8R deeper (all of it in the 2023-24 year).
+
+## Verdict
+
+- **The process works end to end**: recorded evidence → screen → pre-registered rule → full
+  replay against same-harness controls → one pass, five fails, each with the reason recorded.
+- **M5**: all three MACD candidates FAIL (−10 to −46R). Closed. The per-bar screen is not to
+  be trusted on M5 without a replay; it ranked them first.
+- **M15-1 (ema_trend × 2)** FAILS on PF, drawdown and trade count; **M15-2 (divergence → 0 in
+  trending)** FAILS the first half and the drawdown. Both closed.
+- **M15-3 (RSI multiplier × 0.5 in every regime) PASSES the pre-registered rule** (a–f) and the
+  3-year robustness check is positive (+34.7R, PF 1.38 → 1.39, extra year +6.2R). It is a
+  SHIPPING PROPOSAL for the owner, with these facts on the other side of the scale: the 2-year
+  second half passes by only +0.6R, 2026 is −4R, the added trades are mostly B-grade BUYs in a
+  bull-market data set (added SELLs lose), and the 3-year max drawdown deepens 15.5 → 19.3R.
+  If shipped it lands as an M15-only `technical.regime_weights_override` in
+  `pair_signals/xauusd.py` (rule 2) with the full table (RSI 1.0 → 0.5 in the trending family,
+  1.5 → 0.75 in ranging, 1.0 → 0.5 in transitioning), a factory-equivalence test update and a
+  docs note; the M15 canonical book would then be re-pinned from `candidates/m15_rsi:half_all_2yr.csv`
+  (931 / +146.14R / PF 1.43 at $0.65). Not shipped in this campaign.
+
+Books and logs: `backtest/research/replay_2026-09/candidates/` and `.../oos/` (gitignored).
+
+## Shipped (owner decision 2026-09-09, same day)
+
+M15 override 5 in `pair_signals/xauusd.py`: `technical.regime_weights_override` = the shipped
+table with the RSI multiplier halved in every regime (0.5 trending family / 0.75 ranging /
+0.5 transitioning), derived from the default table and pinned literally by the Module 9
+factory-equivalence test; one more Module 33 test locks that M5/H1/H4 carry no override.
+Parity proof — the factory run with NO `--set` flag reproduces the validated candidate books
+byte-for-byte: 2-year MD5 `d185f099482a07f29502cb5a371a9bbf` (931 / +146.14R / PF 1.43) and
+OOS MD5 `6f8f24b49597ced28f50a19e08c3472f` (355 / +60.67R / PF 1.49). Canonical M15 book
+re-pinned to `backtest/oanda_m15_2yr_065_20260909.csv` (copy of the 2-year factory book).
+Docs: `docs/user_guide.html` and `docs/beginner_guide.html` carry a one-line note (PDF builds
+remain behind, as before). Suite 499 tests. Live M15 scans get a new `engine_fingerprint`
+from the next deploy; signal-log comparisons across it must split at the deploy time.

chart_patterns.py

@@ -0,0 +1,289 @@
+"""
+Classical chart-pattern detection for the XAU_CP entry archetype.
+================================================================
+
+Third entry archetype (after the composite-breakout generators and the
+XAU_PB pullback): trade the completion of a proven multi-bar price
+pattern. Pattern set is deliberately tight — the classical patterns
+with the strongest documented statistics (Bulkowski, *Encyclopedia of
+Chart Patterns*) that are unambiguous from OHLC at bar close:
+
+    double_bottom  bullish reversal   two matching swing lows, break of
+                                      the interim peak (neckline)
+    double_top     bearish reversal   mirrored
+    inv_hs         bullish reversal   inverse head & shoulders, break of
+                                      the neckline (the LOWER of the two
+                                      peaks flanking the head — the earlier
+                                      horizontal trigger, no sloped fits;
+                                      validated as-is, see detector note)
+    hs_top         bearish reversal   head & shoulders top, mirrored
+    bull_flag      bullish continuation  strong pole, shallow ordered
+                                      retrace, close at a new high
+    bear_flag      bearish continuation  mirrored
+
+Triangles / wedges / cup-and-handle are excluded on purpose: weaker
+historical stats and apex-timing ambiguity — more parameters for less
+edge.
+
+Design contract (mirrors the rest of the signal stack):
+- Detectors see ONLY the visible closed-bar slice the aggregator was
+  given; the last row is the just-closed signal bar. No forming candle,
+  no future bars → identical behavior live and in the bar-replay
+  backtester, and signals never repaint.
+- All geometric thresholds are in ATR units so one parameter set is
+  scale-free across timeframes. ATR comes from the engine (Wilder 14).
+- Trigger discipline: a pattern fires only on the bar that COMPLETES it
+  — the first close beyond the neckline/boundary since the pattern
+  formed. One breakout → at most one signal.
+- Swing points use the same local-extremum definition as the engine's
+  structure S/R (window w bars each side), so a "swing" here is the
+  same object a chartist sees on the engine's key levels.
+
+Every detector returns None or a dict:
+    {"pattern": str, "direction": "BUY"|"SELL",
+     "anchor": float,     # SL reference (pattern invalidation price)
+     "trigger": float,    # neckline / boundary that was broken
+     "height": float}     # measured-move distance in price units
+
+Defaults below are the textbook definitions, not tuned values. The
+validation record for which patterns/timeframes are enabled lives in
+pair_signals/xauusd.py (per repo rules, per-TF adoption happens in the
+factory, never in global defaults).
+"""
+
+from typing import Dict, List, Optional
+
+import numpy as np
+import pandas as pd
+
+__all__ = ["detect_chart_pattern", "PATTERN_DETECTORS"]
+
+
+def _swing_indices(values: np.ndarray, w: int, find_high: bool) -> List[int]:
+    """Indices of local extrema with `w` bars on each side.
+
+    A bar i is a swing high if values[i] is the strict maximum of
+    values[i-w : i+w+1] (ties reject — a flat top is not a clean swing).
+    The last `w` bars can never qualify, which is exactly right: a swing
+    isn't confirmed until `w` bars close after it.
+    """
+    n = len(values)
+    out: List[int] = []
+    for i in range(w, n - w):
+        window = values[i - w : i + w + 1]
+        if np.isnan(window).any():
+            continue
+        center = values[i]
+        rest = np.delete(window, w)
+        if find_high:
+            if center > rest.max():
+                out.append(i)
+        else:
+            if center < rest.min():
+                out.append(i)
+    return out
+
+
+def _first_close_beyond(closes: np.ndarray, start: int, level: float,
+                        above: bool) -> bool:
+    """True iff the LAST close is the first one beyond `level` since
+    `start` (exclusive of the last bar)."""
+    last = closes[-1]
+    prior = closes[start:-1]
+    if above:
+        return last > level and (len(prior) == 0 or prior.max() <= level)
+    return last < level and (len(prior) == 0 or prior.min() >= level)
+
+
+# ── Double bottom / top ──────────────────────────────────────────────
+
+def _detect_double(h, l, c, atr, cfg, bullish: bool) -> Optional[Dict]:
+    w = cfg.cp_swing_window
+    ext = l if bullish else h
+    idx = _swing_indices(ext, w, find_high=not bullish)
+    if len(idx) < 2:
+        return None
+    i1, i2 = idx[-2], idx[-1]
+    n = len(c)
+    if i2 - i1 < cfg.cp_min_sep_bars:
+        return None
+    if (n - 1) - i2 > cfg.cp_recency_bars:
+        return None
+    e1, e2 = ext[i1], ext[i2]
+    if abs(e1 - e2) > cfg.cp_tolerance_atr * atr:
+        return None
+
+    if bullish:
+        neckline = float(h[i1 : i2 + 1].max())
+        floor = min(e1, e2)
+        height = neckline - floor
+    else:
+        neckline = float(l[i1 : i2 + 1].min())
+        ceil_ = max(e1, e2)
+        height = ceil_ - neckline
+
+    if not (cfg.cp_min_height_atr * atr <= height <= cfg.cp_max_height_atr * atr):
+        return None
+
+    # Pattern must not have been invalidated after the second extreme:
+    # a lower low (bullish) / higher high (bearish) breaks the pattern.
+    if bullish:
+        if i2 + 1 < n and l[i2 + 1 :].min() < min(e1, e2) - 0.1 * atr:
+            return None
+        if not _first_close_beyond(c, i2, neckline, above=True):
+            return None
+        return {"pattern": "double_bottom", "direction": "BUY",
+                "anchor": float(min(e1, e2)), "trigger": neckline,
+                "height": float(height)}
+    else:
+        if i2 + 1 < n and h[i2 + 1 :].max() > max(e1, e2) + 0.1 * atr:
+            return None
+        if not _first_close_beyond(c, i2, neckline, above=False):
+            return None
+        return {"pattern": "double_top", "direction": "SELL",
+                "anchor": float(max(e1, e2)), "trigger": neckline,
+                "height": float(height)}
+
+
+# ── Head & shoulders (top and inverse) ───────────────────────────────
+
+def _detect_hs(h, l, c, atr, cfg, bullish: bool) -> Optional[Dict]:
+    """bullish=True → inverse H&S (three swing LOWS, head deepest)."""
+    w = cfg.cp_swing_window
+    ext = l if bullish else h
+    idx = _swing_indices(ext, w, find_high=not bullish)
+    if len(idx) < 3:
+        return None
+    i1, i2, i3 = idx[-3], idx[-2], idx[-1]
+    n = len(c)
+    if (n - 1) - i3 > cfg.cp_recency_bars:
+        return None
+    s1, head, s2 = ext[i1], ext[i2], ext[i3]
+
+    if bullish:
+        # Head strictly the deepest low; shoulders roughly level.
+        if not (head < s1 - cfg.cp_hs_head_min_atr * atr
+                and head < s2 - cfg.cp_hs_head_min_atr * atr):
+            return None
+        if abs(s1 - s2) > cfg.cp_hs_shoulder_tol_atr * atr:
+            return None
+        # Neckline: the LOWER of the two interim peaks — the EARLIER of
+        # the two horizontal triggers (max() would demand clearing both
+        # peaks). The whole validated CP book (M5 BUY-only) ran on this
+        # trigger — do not flip to max() without full re-validation.
+        p1 = float(h[i1 : i2 + 1].max())
+        p2 = float(h[i2 : i3 + 1].max())
+        neckline = min(p1, p2)
+        height = neckline - head
+    else:
+        if not (head > s1 + cfg.cp_hs_head_min_atr * atr
+                and head > s2 + cfg.cp_hs_head_min_atr * atr):
+            return None
+        if abs(s1 - s2) > cfg.cp_hs_shoulder_tol_atr * atr:
+            return None
+        t1 = float(l[i1 : i2 + 1].min())
+        t2 = float(l[i2 : i3 + 1].min())
+        neckline = max(t1, t2)
+        height = head - neckline
+
+    if not (cfg.cp_min_height_atr * atr <= height <= cfg.cp_max_height_atr * atr):
+        return None
+
+    if bullish:
+        # Invalidated if price undercut the head after the right shoulder.
+        if i3 + 1 < n and l[i3 + 1 :].min() < head - 0.1 * atr:
+            return None
+        if not _first_close_beyond(c, i3, neckline, above=True):
+            return None
+        return {"pattern": "inv_hs", "direction": "BUY",
+                "anchor": float(s2), "trigger": neckline,
+                "height": float(height)}
+    else:
+        if i3 + 1 < n and h[i3 + 1 :].max() > head + 0.1 * atr:
+            return None
+        if not _first_close_beyond(c, i3, neckline, above=False):
+            return None
+        return {"pattern": "hs_top", "direction": "SELL",
+                "anchor": float(s2), "trigger": neckline,
+                "height": float(height)}
+
+
+# ── Flags (bull and bear) ────────────────────────────────────────────
+
+def _detect_flag(h, l, c, atr, cfg, bullish: bool) -> Optional[Dict]:
+    n = len(c)
+    lo_r, hi_r = cfg.cp_flag_retrace
+    if bullish:
+        # Pole top = highest high BEFORE the trigger bar.
+        ip = int(np.argmax(h[:-1]))
+        flag_len = (n - 1) - ip
+        if not (cfg.cp_flag_min_bars <= flag_len <= cfg.cp_flag_max_bars):
+            return None
+        pole_start = max(0, ip - cfg.cp_flag_pole_bars + 1)
+        pole_low = float(l[pole_start : ip + 1].min())
+        pole = float(h[ip]) - pole_low
+        if pole < cfg.cp_flag_pole_atr * atr:
+            return None
+        flag_low = float(l[ip + 1 : n].min())
+        retrace = (float(h[ip]) - flag_low) / pole
+        if not (lo_r <= retrace <= hi_r):
+            return None
+        if not _first_close_beyond(c, ip, float(h[ip]), above=True):
+            return None
+        return {"pattern": "bull_flag", "direction": "BUY",
+                "anchor": flag_low, "trigger": float(h[ip]),
+                "height": float(pole)}
+    else:
+        ip = int(np.argmin(l[:-1]))
+        flag_len = (n - 1) - ip
+        if not (cfg.cp_flag_min_bars <= flag_len <= cfg.cp_flag_max_bars):
+            return None
+        pole_start = max(0, ip - cfg.cp_flag_pole_bars + 1)
+        pole_high = float(h[pole_start : ip + 1].max())
+        pole = pole_high - float(l[ip])
+        if pole < cfg.cp_flag_pole_atr * atr:
+            return None
+        flag_high = float(h[ip + 1 : n].max())
+        retrace = (flag_high - float(l[ip])) / pole
+        if not (lo_r <= retrace <= hi_r):
+            return None
+        if not _first_close_beyond(c, ip, float(l[ip]), above=False):
+            return None
+        return {"pattern": "bear_flag", "direction": "SELL",
+                "anchor": flag_high, "trigger": float(l[ip]),
+                "height": float(pole)}
+
+
+# Priority order: most structured / rarest first — if two patterns
+# complete on the same bar, the more specific one names the trade.
+PATTERN_DETECTORS = {
+    "inv_hs":        lambda h, l, c, atr, cfg: _detect_hs(h, l, c, atr, cfg, True),
+    "hs_top":        lambda h, l, c, atr, cfg: _detect_hs(h, l, c, atr, cfg, False),
+    "double_bottom": lambda h, l, c, atr, cfg: _detect_double(h, l, c, atr, cfg, True),
+    "double_top":    lambda h, l, c, atr, cfg: _detect_double(h, l, c, atr, cfg, False),
+    "bull_flag":     lambda h, l, c, atr, cfg: _detect_flag(h, l, c, atr, cfg, True),
+    "bear_flag":     lambda h, l, c, atr, cfg: _detect_flag(h, l, c, atr, cfg, False),
+}
+
+
+def detect_chart_pattern(df: pd.DataFrame, atr: float, cfg) -> Optional[Dict]:
+    """Run enabled detectors over the last cp_lookback(+1) bars.
+
+    `df` is the visible history at decision time (last row = signal
+    bar). Returns the first match in PATTERN_DETECTORS priority order,
+    or None.
+    """
+    if atr <= 0 or len(df) < cfg.cp_lookback + 1:
+        return None
+    tail = df.iloc[-(cfg.cp_lookback + 1):]
+    h = tail["High"].to_numpy(dtype=np.float64)
+    l = tail["Low"].to_numpy(dtype=np.float64)
+    c = tail["Close"].to_numpy(dtype=np.float64)
+    enabled = set(cfg.cp_patterns)
+    for name, det in PATTERN_DETECTORS.items():
+        if name not in enabled:
+            continue
+        hit = det(h, l, c, atr, cfg)
+        if hit is not None:
+            return hit
+    return None

config.json.example

@@ -0,0 +1,68 @@
+{
+  "telegram": {
+    "token": "123456789:REPLACE_WITH_YOUR_BOT_TOKEN",
+    "chat_id": "REPLACE_WITH_OWNER_CHAT_ID",
+    "channel_id": "-100REPLACE_WITH_CHANNEL_ID_OR_EMPTY_STRING",
+    "alert_min_quality": "C"
+  },
+  "data_providers": {
+    "active": "oanda",
+    "streaming": "tiingo",
+    "twelve_data": {
+      "api_key": "REPLACE_WITH_YOUR_TWELVE_DATA_API_KEY"
+    },
+    "oanda": {},
+    "tiingo": {
+      "api_key": "REPLACE_WITH_YOUR_TIINGO_API_KEY_OR_EMPTY"
+    }
+  },
+  "scanning": {
+    "pairs": ["XAU/USD"],
+    "timeframes": ["M5", "M15", "H1", "H4"]
+  },
+  "pairs": {
+    "XAU/USD": {
+      "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21]
+    }
+  },
+  "oanda": {
+    "api_token": "REPLACE_WITH_API_TOKEN",
+    "base_url": "https://api-fxpractice.oanda.com"
+  },
+  "deepseek": {
+    "api_key": "REPLACE_WITH_YOUR_DEEPSEEK_API_KEY_OR_EMPTY",
+    "model": "deepseek-v4-flash",
+    "enabled": true,
+    "timeout_seconds": 12,
+    "daily_request_cap": 500
+  },
+  "agent_panel": {
+    "enabled": false,
+    "telegram": true,
+    "dashboard": false,
+    "min_quality": "B",
+    "max_panels_per_day": 20,
+    "reuse_ttl_seconds": 3600,
+    "candles_enabled": true,
+    "candle_bars_primary": 20,
+    "candle_bars_confirm": 10,
+    "verdict_source": "macro"
+  },
+  "violent_watch": {
+    "enabled": false,
+    "alert_line": true,
+    "headsup": true,
+    "max_headsup_per_day": 12,
+    "headsup_min_level": "hot"
+  },
+  "signal_log": {
+    "enabled": true,
+    "path": "logs/signals.db",
+    "log_blocked_scans": true
+  },
+  "news_archive": {
+    "enabled": true,
+    "path": "logs/newsdata.db"
+  },
+  "log_level": "INFO"
+}

config.py

@@ -1,8 +1,6 @@
 """
-Forex Signal Intelligence Platform - Configuration
-===================================================
-Central configuration for all signal analysis parameters.
-Tuned by market regime and validated against historical performance.
+ForexBot2 — Configuration (XAU/USD focused)
+==========================================
 """
 
 from dataclasses import dataclass, field
@@ -29,168 +27,327 @@ class SignalStrength(Enum):
     STRONG_SELL = -2
 
 
-class MarketSession(Enum):
-    SYDNEY = "sydney"
-    TOKYO = "tokyo"
-    LONDON = "london"
-    NEW_YORK = "new_york"
-    OVERLAP_LONDON_NY = "london_ny_overlap"
+# ── Supported pairs ──────────────────────────────────────────────
+XAUUSD = "XAU/USD"
 
-
-# ── Major Forex Pairs ──────────────────────────────────────────────
-MAJOR_PAIRS = [
-    "EURUSD=X", "GBPUSD=X", "USDJPY=X", "USDCHF=X",
-    "AUDUSD=X", "USDCAD=X", "NZDUSD=X",
-]
-
-CROSS_PAIRS = [
-    "EURGBP=X", "EURJPY=X", "GBPJPY=X", "AUDJPY=X",
-    "EURCHF=X", "CADJPY=X",
-]
-
-# Yahoo Finance symbol mapping for display
-SYMBOL_DISPLAY = {
-    "EURUSD=X": "EUR/USD", "GBPUSD=X": "GBP/USD",
-    "USDJPY=X": "USD/JPY", "USDCHF=X": "USD/CHF",
-    "AUDUSD=X": "AUD/USD", "USDCAD=X": "USD/CAD",
-    "NZDUSD=X": "NZD/USD", "EURGBP=X": "EUR/GBP",
-    "EURJPY=X": "EUR/JPY", "GBPJPY=X": "GBP/JPY",
-    "AUDJPY=X": "AUD/JPY", "EURCHF=X": "EUR/CHF",
-    "CADJPY=X": "CAD/JPY",
+# price_precision: decimals for SL/TP rounding and display.
+# default_spread: $0.30 round-trip cost retained for TD-era cross-book
+# comparability and used when the backtester receives no --spread override.
+# Oanda's measured round trip is $0.65-$0.72; research reports headline it
+# separately with --spread 0.65.
+PAIR_INFO = {
+    XAUUSD: {"display": "XAU/USD", "pip_value": 0.10,
+             "price_precision": 2, "default_spread": 0.30},
 }
 
 
 @dataclass
 class TechnicalConfig:
-    """Technical indicator parameters - carefully tuned for Forex."""
-    # EMA periods (fast/medium/slow for trend detection)
+    """Technical indicator parameters."""
     ema_fast: int = 9
     ema_medium: int = 21
     ema_slow: int = 50
     ema_trend: int = 200
+    # 4-EMA mode only: score partial ribbon alignment (2-3 of 4 checks)
+    # as NEUTRAL instead of a weak BUY/SELL. Research knob (2026-08-04
+    # indicator audit: partial-alignment trades averaged -0.008R vs
+    # +0.230R fully aligned on the M5 12mo Oanda book); default False =
+    # legacy behavior until a replay campaign validates flipping it.
+    ema_partial_neutral: bool = False
 
-    # RSI
     rsi_period: int = 14
     rsi_overbought: float = 70.0
     rsi_oversold: float = 30.0
     rsi_neutral_upper: float = 60.0
     rsi_neutral_lower: float = 40.0
 
-    # MACD (12, 26, 9 is standard)
     macd_fast: int = 12
     macd_slow: int = 26
     macd_signal: int = 9
 
-    # Bollinger Bands
     bb_period: int = 20
     bb_std: float = 2.0
 
-    # ATR for volatility measurement
     atr_period: int = 14
 
-    # Stochastic
-    stoch_k: int = 14
-    stoch_d: int = 3
-    stoch_overbought: float = 80.0
-    stoch_oversold: float = 20.0
-
-    # ADX for trend strength
     adx_period: int = 14
     adx_strong_trend: float = 25.0
     adx_very_strong: float = 50.0
 
-    # Ichimoku
-    ichimoku_tenkan: int = 9
-    ichimoku_kijun: int = 26
-    ichimoku_senkou_b: int = 52
+    divergence_lookback: int = 40
+    swing_sr_lookback: int = 50
 
-    # Volume profile
-    volume_ma_period: int = 20
+    disabled_indicators: List[str] = field(default_factory=list)
 
-    # Pivot points lookback
-    pivot_lookback: int = 1
+    ema_mode: str = "4ema"          # "4ema" (9/21/50/200) or "2ema" (50/200)
+    rsi_mode: str = "regime_aware"  # "classic" or "regime_aware"
+    macd_as_pillar: bool = True     # MACD histogram is a scored signal
 
-    # Divergence detection — how many bars back to scan for swing points
-    divergence_lookback: int = 14
+    # ROC (Rate of Change)
+    roc_period: int = 10
+    roc_threshold: float = 1.0     # % threshold for strong signal
+
+    # Keltner Channels
+    kc_period: int = 20
+    kc_atr_period: int = 10
+    kc_multiplier: float = 1.5
+
+    # ADX slope (trend acceleration detection)
+    adx_slope_lookback: int = 5
+    adx_slope_emerging_threshold: float = 3.0
+
+    regime_weights_override: Dict[str, Dict[str, float]] = field(default_factory=dict)
 
 
 @dataclass
 class SignalConfig:
-    """Signal scoring and filtering parameters."""
-    # Minimum confidence to emit a signal (0-100)
-    min_confidence: float = 55.0
+    """Signal scoring & filtering."""
+    mtf_agreement_threshold: int = 2  # of 3 timeframes
 
-    # Weight allocation for composite score
-    weights: Dict[str, float] = field(default_factory=lambda: {
-        "trend": 0.25,          # EMA alignment, ADX
-        "momentum": 0.20,       # RSI, Stochastic, MACD
-        "volatility": 0.15,     # BB position, ATR regime
-        "price_action": 0.15,   # Candlestick patterns, S/R
-        "multi_timeframe": 0.15,# Higher TF confirmation
-        "sentiment": 0.10,      # News/calendar events
-    })
+    min_rr_ratio: float = 0.5
+    # TP tightening (May 2026) was tuned on the pre-spread backtester.
+    # Re-tested 2026-07-08 with $0.30 spread + gap fills: 0.8 nearly
+    # halves total R (+36.1R vs +66.8R, worse in both train/val halves)
+    # — smaller wins make the fixed spread a larger share of every TP.
+    tp_tightening_factor: float = 1.0
 
-    # Multi-timeframe confirmation requirements
-    mtf_agreement_threshold: int = 2  # out of 3 timeframes must agree
+    # ── Trade geometry ────────────────────────────────────────────
+    # Deliberately $0.30 for TD-era cross-book comparability. Oanda's measured
+    # round trip is $0.65-$0.72, reported separately in re-pin campaigns.
+    # This is also a LIVE gate input: increasing it to $0.65 raises the TP1
+    # floor from about $9 to $19.50, so it requires rule-1 revalidation.
+    round_trip_cost: float = 0.30
+    # Spread gate: reject setups whose TP1 distance is below
+    # round_trip_cost × this multiple (caps cost at ~3% of target).
+    # Validated 2026-07: small-target trades lose in both data halves
+    # on M5 while TP≥$10 trades earn ~+0.08R/trade.
+    min_tp_cost_multiple: float = 30.0
+    # Fixed SL floor in pips (80 pips = $8 on XAU). 0 disables — SL then
+    # floors at ATR × v2_sl_atr_multiplier only.
+    min_sl_pips: float = 80.0
+    # TP1 floor as a fraction of SL distance (0.5 = historical RR 0.5).
+    tp_floor_rr: float = 0.5
 
-    # Signal cooldown (prevent over-signalling same direction)
-    cooldown_bars: int = 5
-
-    # Risk management defaults
-    default_risk_pct: float = 1.0  # risk 1% per trade
-    default_rr_ratio: float = 2.0  # 1:2 risk-reward minimum
+    # ── Trailing-stop exit (2026-07) ──────────────────────────────
+    # Once favorable excursion reaches trail_arm_r × risk, the stop
+    # trails trail_dist_r × risk behind the best price (updated at bar
+    # close). TP1 becomes advisory — the trail is the exit. Validated
+    # on 12mo M5: +105R vs +48R with the fixed RR-0.5 target, balanced
+    # train/val, positive every quarter, PF 2.27. The RR-0.5 target was
+    # the binding constraint — winners kept running after TP.
+    # Set trail_arm_r = 0 to disable and exit at TP1 as before.
+    trail_arm_r: float = 0.6
+    trail_dist_r: float = 0.75
+    # Pullback (XAU_PB) trades run tighter stops (~$8 vs ~$25), so the
+    # global 0.6R arm fires on noise. Arm later; whole 0.8-arm row of
+    # the grid is +35-38R vs +32R at global params (balanced halves).
+    pb_trail_arm_r: float = 0.8
+    pb_trail_dist_r: float = 0.75
 
 
 @dataclass
 class NewsConfig:
-    """Economic calendar and sentiment configuration."""
-    # High-impact event currencies
-    tracked_currencies: List[str] = field(default_factory=lambda: [
-        "USD", "EUR", "GBP", "JPY", "CHF", "AUD", "CAD", "NZD"
-    ])
-
-    # Event impact levels to monitor
-    min_impact: str = "medium"  # "low", "medium", "high"
-
-    # Pre-event buffer (minutes before high-impact event = no new signals)
+    """Economic calendar & sentiment."""
+    tracked_currencies: List[str] = field(default_factory=lambda: ["USD", "XAU"])
     pre_event_buffer_minutes: int = 30
-
-    # Post-event volatility window (minutes after event = cautious signals)
     post_event_buffer_minutes: int = 15
+    # Whether event risk gates the SIGNAL (danger: confidence hard-capped
+    # at 30; caution: 15-point penalty). Warnings are always emitted
+    # regardless. Default False = XAU-tuned: replaying the live gates
+    # against the validated 12mo XAU M5 book (547 trades, +139.9R)
+    # showed they suppress 25-35 trades worth +4.6..+7.2R/yr — trades
+    # entered near USD releases OUTPERFORM (+0.475R vs +0.256R avg).
+    # Audit 2026-07-11 (calendar-replay study; tooling since removed).
+    event_risk_caps_signals: bool = False
+    # Whether headline sentiment moves the SIGNAL (aligned: +5 confidence;
+    # conflicting: 10-point penalty). Warnings are always emitted
+    # regardless. Default False = news is informational only; decisions
+    # are pure technical, matching the backtester (use_news=False).
+    sentiment_affects_signals: bool = False
+    # Wider stay-out window after a high-impact release that surprised
+    # vs forecast (inline releases use post_event_buffer_minutes).
+    post_event_surprise_buffer_minutes: int = 45
+    # How long fetched calendar/headline data is reused across scans.
+    # Must stay below pre_event_buffer_minutes so an event can't cross
+    # into the danger window while a stale cache says it's clear.
+    context_cache_ttl_seconds: int = 600
+    # Maximum age of RSS headlines accepted for scoring. Older headlines
+    # (and headlines with no parsable publish time) are discarded before
+    # keyword analysis and LLM enrichment. 12 h → 4 h on 2026-09-04: the
+    # 3-week signal log showed the 12 h mean was a trailing echo of the
+    # previous day's price move (see backtest/research/
+    # news_sentiment_value_2026-09-04.md). Display/warning-only either way.
+    headline_max_age_seconds: int = 14400  # 4 hours
 
-    # News sentiment sources
     rss_feeds: List[str] = field(default_factory=lambda: [
-        "https://www.forexlive.com/feed/",
+        "https://investinglive.com/feed/",
         "https://www.fxstreet.com/rss",
+        # Google News query targeted at gold drivers — also carries Kitco,
+        # Reuters etc. whose direct feeds are unavailable.
+        "https://news.google.com/rss/search?q=gold+price+OR+fed+OR+inflation&hl=en-US&gl=US&ceid=US:en",
+        # CNBC "Economy" section
+        "https://search.cnbc.com/rs/search/combinedcms/view.xml?partnerId=wrss01&id=20910258",
     ])
 
 
+@dataclass
+class XauStrategyConfig:
+    """XAU/USD strategy config."""
+    enabled: bool = True
+    tp_atr_multiplier: float = 1.0
+    sl_atr_multiplier: float = 2.0
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [0, 1, 2, 10, 11, 13, 16, 21])
+    weekend_filter: bool = True
+    min_atr_threshold: float = 1.0
+    v2_sl_atr_multiplier: float = 2.0
+    allowed_weekdays: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 6])  # Mon-Fri + Sun (Sydney open)
+    sell_min_quality: List[str] = field(default_factory=lambda: ["A+", "A", "B"])
+    # Volatility regimes in which SELL setups are allowed (low/normal-vol
+    # shorts lose consistently against gold's macro uptrend).
+    sell_vol_regimes: List[str] = field(default_factory=lambda: ["high", "extreme"])
+    # Market regimes in which no XAU setup may fire (any direction).
+    # Empty by default; per-timeframe factories opt in — e.g. the mixed
+    # "trending" regime (strong ADX, EMAs disagree) on M15.
+    blocked_regimes: List[str] = field(default_factory=list)
+    # Market regimes in which XAU SELL setups may not fire. Empty by
+    # default; per-timeframe factories opt in.
+    sell_blocked_regimes: List[str] = field(default_factory=list)
+    # Bar-CLOSE (decision-time) UTC hours in which XAU SELL setups may
+    # not fire. Empty by default; per-timeframe factories opt in.
+    # BUYs are unaffected (unlike blocked_hours_utc, which is both-way).
+    sell_blocked_hours_utc: List[int] = field(default_factory=list)
+
+    # ── M5-specific filters ──────────────────────────────────────
+    # M5 signals are noisy; require stronger confirmation before
+    # generating trade setups. Without these, M5 is breakeven
+    # (+0.01R/trade) with 21% drawdown.
+    # Relaxed 2026-07-30 (30→25, 10→5) in the relaxation campaign:
+    # the combined config (with the M5 hour-21 unblock) beat control
+    # in ALL 6 calendar years of the 5yr Oanda replay (+230.9R vs
+    # +168.0R, PF 1.26 vs 1.21, max DD 21.3 vs 27.0), 12mo +107.4R
+    # vs +84.6R PF 1.36 both halves positive, OOS 2024-06→2025-03
+    # +26.1R vs +17.2R, and at $0.70 spread +86.9R vs +66.4R.
+    # Score floors below 25 are vacuous (grade C requires |score|≥25).
+    m5_require_mtf: bool = True
+    m5_min_abs_score: float = 25.0
+    m5_min_confirmation_score: float = 5.0
+    # A-grade M5 SELLs enter at the extreme of temporary dips and
+    # lose (-0.017R/trade on 163 trades) — maximum conviction at
+    # the worst entry. Cap SELL quality so A-grade can't trigger
+    # the ATR-based setup generator; B-grade swing setups survive
+    # because they catch moderate drops, not extremes.
+    m5_sell_max_quality: List[str] = field(default_factory=lambda: ["B", "C"])
+    # 'high' vol regime on M5 is chop transition — elevated noise
+    # without directional expansion. 61% WR / negative total across
+    # 12mo while low/normal/extreme are all positive (validated 2026-07).
+    m5_blocked_vol_regimes: List[str] = field(default_factory=lambda: ["high"])
+    # Block M5 SELLs when RSI is already < this floor: the dip is
+    # extended and mean-reverts before the target (negative in both
+    # halves of 6mo and 12mo backtests).
+    m5_sell_min_rsi: float = 25.0
+
+    # ── M5 pullback entry (XAU_PB) ──
+    # Second entry archetype: enter on the retrace to EMA21/KC-mid in
+    # an established trend instead of at Keltner-breakout extension.
+    # Validated 2026-07: +38.3R/254 trades on 12mo, positive both
+    # halves and all four full quarters, core trades undisturbed.
+    m5_pullback_enabled: bool = True
+    # Composite score is weaker mid-pullback BY DESIGN (price sits
+    # below the momentum bands) — separate, lower floor than Gate 1.
+    m5_pullback_min_score: float = 15.0
+    m5_pullback_adx_min: float = 20.0
+    # RSI must be in the pullback zone, NOT the extended zone the
+    # breakout entries fire in (those run 65-80).
+    m5_pullback_rsi_buy: tuple = (35.0, 62.0)
+    m5_pullback_rsi_sell: tuple = (38.0, 65.0)
+    # Retrace depth from the pre-pullback swing extreme, in ATRs.
+    # Shallower = still extended; deeper = trend likely broken.
+    # M5 gold trend legs measure 3-4 ATR swing-to-trough (median 3.9),
+    # so the ceiling sits well above that of higher TFs.
+    m5_pullback_retrace_atr: tuple = (0.8, 6.0)
+    # Price must have touched EMA21/KC-mid within this many bars.
+    m5_pullback_touch_bars: int = 3
+
+    # ── Chart-pattern entry (XAU_CP) ──────────────────────────────
+    # Third entry archetype: completion of a classical chart pattern
+    # (see chart_patterns.py for the pattern set and design contract).
+    # Defaults are textbook geometry, NOT tuned values. Disabled by
+    # default — per-TF adoption happens in pair_signals/xauusd.py only
+    # after the pattern book passes the standard validation rule
+    # (positive R both halves of ≥12mo, PF ≥ 1.3, $0.30 spread,
+    # trailing exits, next-open fills).
+    cp_enabled: bool = False
+    cp_patterns: List[str] = field(default_factory=lambda: [
+        "inv_hs", "hs_top", "double_bottom", "double_top",
+        "bull_flag", "bear_flag",
+    ])
+    cp_lookback: int = 72          # bars scanned for pattern structure
+    cp_swing_window: int = 3       # bars each side to confirm a swing
+    cp_recency_bars: int = 20      # last extreme must be this fresh
+    cp_min_sep_bars: int = 8       # min bars between double extremes
+    cp_tolerance_atr: float = 0.5  # double top/bottom level match tol
+    cp_min_height_atr: float = 1.5   # pattern height floor (noise gate)
+    cp_max_height_atr: float = 8.0   # pattern height ceiling
+    cp_hs_head_min_atr: float = 0.5  # head must exceed shoulders by this
+    cp_hs_shoulder_tol_atr: float = 1.0  # shoulder level match tol
+    cp_flag_pole_atr: float = 3.0    # min pole size
+    cp_flag_pole_bars: int = 12      # pole measured over ≤ this many bars
+    cp_flag_min_bars: int = 3        # flag consolidation length bounds
+    cp_flag_max_bars: int = 12
+    cp_flag_retrace: tuple = (0.1, 0.5)  # flag depth as fraction of pole
+
+    # ── Big-candle entry gate ──────────────────────────────────────
+    # When the recent window on the scan's own primary TF contains an
+    # abnormally large candle (bar range >= K x the PRIOR bar's ATR —
+    # institutional burst; price may reverse violently), no XAU entry
+    # of ANY kind may fire (core, XAU_PB, XAU_CP). 0.0 = disabled.
+    # Per-TF factories opt in only after the standard validation rule.
+    big_candle_atr_multiple: float = 0.0
+    big_candle_lookback_hours: float = 24.0
+
+
+@dataclass
+class FilterConfig:
+    """Regime and session filters for live signals."""
+    regime_filter: bool = False
+    regime_filter_timeframe: Timeframe = Timeframe.H4
+    regime_filter_min_adx: float = 25.0
+
+    session_filter: bool = True
+    session_start_hour: int = 3
+    session_end_hour: int = 23
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [0, 1, 2, 10, 11, 13, 16, 21])
+
+    allowed_weekdays: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 6])  # Mon-Fri + Sun (Sydney open)
+
+
 @dataclass
 class PlatformConfig:
-    """Overall platform configuration."""
+    """Top-level configuration."""
     technical: TechnicalConfig = field(default_factory=TechnicalConfig)
     signals: SignalConfig = field(default_factory=SignalConfig)
     news: NewsConfig = field(default_factory=NewsConfig)
+    backtest: FilterConfig = field(default_factory=FilterConfig)
+    xau_strategy: XauStrategyConfig = field(default_factory=XauStrategyConfig)
 
-    # Default pairs to monitor
-    watch_pairs: List[str] = field(default_factory=lambda: MAJOR_PAIRS)
+    symbol: str = XAUUSD
+    display_name: str = "XAU/USD"
 
-    # Analysis timeframes (primary + confirmation)
     primary_timeframe: Timeframe = Timeframe.H1
     confirmation_timeframes: List[Timeframe] = field(
         default_factory=lambda: [Timeframe.M15, Timeframe.H4]
     )
 
-    # Data source
-    data_lookback_days: int = 60  # enough for 200 EMA on H1
+    data_lookback_days: int = 60
 
-    # Twelve Data API key (overridden by TWELVE_DATA_API_KEY env var)
-    twelve_data_api_key: str = "f563b22ffdfd4924955decbe1a8c8701"
+    refresh_interval: int = 300
 
-    # Dashboard refresh interval (seconds)
-    refresh_interval: int = 300  # 5 minutes
+    scan_pairs: List[str] = field(
+        default_factory=lambda: [XAUUSD]
+    )
+    scan_timeframes: List[Timeframe] = field(
+        default_factory=lambda: [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4]
+    )
+    ws_cooldown_seconds: int = 300
 
-    # Logging
     log_level: str = "INFO"
-    log_signals_to_file: bool = True
-    signal_log_path: str = "signals.log"

dashboard_publisher.py

@@ -0,0 +1,567 @@
+#!/usr/bin/env python3
+"""
+XAU/USD static dashboard publisher.
+
+Standalone daemon that scans XAU/USD at candle closes and atomically
+writes `snapshot.json` + `news.json` (plus copies of the static
+`dashboard.html` / `.css` / `.js` and `news_summary.php`) into an
+Apache-served directory. Apache does
+all web serving — this process never opens a port, so a failure here
+can only ever mean a stale page.
+
+Independence rules (owner requirement — the telegram bot must be
+immune to anything on the web side):
+  * Runs as its own process; never imported by, and never imports,
+    `telegram_bot` (telegram_bot resolves credentials at import). The
+    confirmation map and boundary math are therefore duplicated here
+    as local constants.
+  * The only shared-code dependency beyond the standard signal path is
+    the additive `NewsEngine.get_week_events` accessor, which no
+    production signal path calls.
+
+Scheduling: every M15/H1/H4 boundary is also an M5 boundary (all
+intervals are multiples of 5, anchored to UTC midnight), so one
+sequential loop sleeps to the next M5 boundary (+10 s buffer) and scans
+whichever timeframes just closed. Due-classification uses the RAW wake
+time — flooring 12:00:10 to the 5-minute grid yields the 12:00 boundary
+that just closed. Do NOT subtract the buffer first: 12:00:10 − 15 s
+floors to 11:55 and silently skips every H1/H4 scan (v1 bug, 2026-08-02).
+A cycle can overrun its 5-minute slot (or the process can be down), so
+each wake also catches up the timeframes whose latest closed boundary is
+newer than their last scanned one (`catch_up_timeframes`) — without it the
+page served an hours-old H1/H4 report behind a fresh countdown
+(2026-09-13 audit).
+
+Scans run sequentially M5→M15→H1→H4 through one shared data provider so
+later timeframes hit the warm boundary cache — that ordering IS the
+duplicate-fetch dedup (providers have no in-flight dedup; Oanda has no
+throttle). Keep it sequential.
+
+Run:  python dashboard_publisher.py [--once] [--out DIR]
+Out dir precedence: --out → DASHBOARD_OUT_DIR env → /var/www/trade
+"""
+
+import argparse
+import json
+import hashlib
+import logging
+import math
+import numbers
+import os
+import shutil
+import sys
+import time
+from datetime import datetime, timedelta, timezone
+from logging.handlers import TimedRotatingFileHandler
+from typing import Dict, List, Optional
+
+import pandas as pd
+
+from agent_panel import attach_panel
+from app_config import get_app_config, violent_watch_enabled
+from config import PlatformConfig, Timeframe, XAUUSD
+from main import report_to_dict
+from news_engine import NewsEngine
+from pair_signals import produce_signal
+from providers import make_data_provider
+from violent_watch import attach_watch
+import providers.base as _providers_base
+
+logger = logging.getLogger("dashboard")
+
+DASHBOARD_TFS = [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4]
+
+# Local copy of the bot's confirmation map (see independence rules above).
+CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+TF_INTERVAL_MINUTES = {
+    Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.H1: 60, Timeframe.H4: 240,
+}
+
+DEFAULT_OUT_DIR = "/var/www/trade"
+SCAN_BUFFER_SECONDS = 10
+SNAPSHOT_FILE = "snapshot.json"
+NEWS_FILE = "news.json"
+PAGE_FILE = "dashboard.html"
+SUMMARY_PHP_FILE = "news_summary.php"
+# Repo files copied into the out dir at daemon start (git pull + restart
+# updates them). The PHP endpoint reads news.json in place, so it lives
+# next to the page.
+STATIC_FILES = [PAGE_FILE, "dashboard.css", "dashboard.js", SUMMARY_PHP_FILE]
+
+
+# ══════════════════════════════════════════════════════════════════
+#  Pure layer (no network, fully covered by test_all.py)
+# ══════════════════════════════════════════════════════════════════
+
+def _safe_num(v):
+    """Coerce to a JSON-safe float; NaN/±inf become None."""
+    try:
+        f = float(v)
+    except (TypeError, ValueError):
+        return v
+    if math.isnan(f) or math.isinf(f):
+        return None
+    return f
+
+
+def _sanitize(obj):
+    """Recursively make a payload safe for json.dumps(allow_nan=False):
+    NaN/inf → None, numpy scalars → native ints/floats."""
+    if isinstance(obj, bool):
+        return obj
+    if isinstance(obj, dict):
+        return {k: _sanitize(v) for k, v in obj.items()}
+    if isinstance(obj, (list, tuple)):
+        return [_sanitize(v) for v in obj]
+    if isinstance(obj, numbers.Integral):
+        return int(obj)
+    if isinstance(obj, numbers.Real):
+        return _safe_num(obj)
+    return obj
+
+
+def analysis_to_dict(analysis) -> Optional[dict]:
+    """Serialize a TechnicalAnalysis. Iterates analysis.indicators only —
+    never vars()/asdict(), which would drag dynamic pandas attrs
+    (_rsi_series, _macd_histogram, …) into the payload. ATR (weight 0)
+    is surfaced separately as context, not in the directional grid."""
+    if analysis is None:
+        return None
+    indicators = {}
+    atr_context = None
+    for name, ind in analysis.indicators.items():
+        if name.startswith("_"):
+            continue
+        entry = {
+            "signal": ind.signal.name,
+            "value": _safe_num(ind.value),
+            "confidence": _safe_num(ind.confidence),
+            "weight": _safe_num(ind.weight),
+            "description": ind.description,
+        }
+        if name == "atr":
+            atr_context = entry
+        elif ind.weight != 0:
+            indicators[name] = entry
+    return {
+        "timeframe": analysis.timeframe,
+        "timestamp": str(analysis.timestamp),
+        "composite_signal": analysis.composite_signal.name,
+        "composite_score": _safe_num(analysis.composite_score),
+        "confidence": _safe_num(analysis.confidence),
+        "market_regime": analysis.market_regime,
+        "volatility_regime": analysis.volatility_regime,
+        "pattern_signals": list(analysis.pattern_signals),
+        "key_levels": {k: _safe_num(v) for k, v in analysis.key_levels.items()},
+        "indicators": indicators,
+        "atr_context": atr_context,
+    }
+
+
+def report_to_webapp_dict(report) -> dict:
+    """main.report_to_dict + the per-TF detail the dashboard needs."""
+    d = report_to_dict(report)
+    d["primary_signal"] = report.primary_signal.name
+    d["primary_score"] = _safe_num(report.primary_score)
+    d["mtf_scores"] = {k: _safe_num(v) for k, v in report.mtf_scores.items()}
+    d["exhaustion_count"] = report.exhaustion_count
+    d["primary_analysis"] = analysis_to_dict(report.primary_analysis)
+    d["confirmation_analyses"] = [
+        analysis_to_dict(a) for a in report.confirmation_analyses
+    ]
+    return _sanitize(d)
+
+
+def event_to_dict(ev) -> dict:
+    """EconomicEvent → dict. surprise is a property, so asdict() would
+    skip it — added explicitly.
+    actual/forecast/previous stay strings as published ('228K', '0.3%')."""
+    return {
+        "time": ev.time.astimezone(timezone.utc).isoformat(),
+        "currency": ev.currency,
+        "impact": ev.impact,
+        "event": ev.event,
+        "forecast": ev.forecast,
+        "previous": ev.previous,
+        "actual": ev.actual,
+        "surprise": ev.surprise,
+    }
+
+
+def headline_to_dict(h) -> dict:
+    # id uses the same sha1(headline) scheme as NewsEngine._llm_sent_cache;
+    # it is the join key the PHP summary endpoint whitelists against.
+    return {
+        "id": hashlib.sha1(h.headline.encode("utf-8")).hexdigest(),
+        "headline": h.headline,
+        "source": h.source,
+        "link": h.link,
+        "description": h.description,
+        "publisher": h.publisher,
+        "timestamp": h.timestamp.astimezone(timezone.utc).isoformat()
+                     if h.timestamp else None,
+        "currencies": list(h.currencies_mentioned),
+        "score": _safe_num(h.sentiment_score),
+        "keywords": list(h.keywords),
+        "score_by_currency": {
+            k: _safe_num(v) for k, v in h.sentiment_by_currency.items()
+        },
+    }
+
+
+def build_news_payload(context, week_events, now: datetime) -> dict:
+    """news.json: full calendar week + scored headlines + LLM summary."""
+    headlines = sorted(
+        context.news_headlines,
+        key=lambda h: h.timestamp or now, reverse=True,
+    )
+    return _sanitize({
+        "generated_at": now.isoformat(),
+        "llm_summary": context.llm_summary,
+        "sentiment": dict(context.news_sentiment),
+        "high_impact_warning": context.high_impact_warning,
+        "warning_message": context.warning_message,
+        "calendar_available": context.calendar_available,
+        "events": [event_to_dict(e) for e in week_events],
+        "headlines": [headline_to_dict(h) for h in headlines],
+    })
+
+
+def due_timeframes(wake_dt: datetime) -> List[Timeframe]:
+    """Timeframes whose candle just closed at the boundary this wake
+    belongs to. Takes the RAW wake time (boundary + buffer): flooring to
+    the 5-minute grid yields the boundary that just passed. Never
+    subtract the buffer first — see module docstring (v1 bug).
+
+    A timeframe whose just-closed bar would carry a label inside gold's
+    daily settlement pause is skipped: no such bar exists, so the scan
+    would only re-read the previous one (2026-09-13 audit)."""
+    minutes = wake_dt.hour * 60 + wake_dt.minute
+    floored = minutes - (minutes % 5)
+    boundary = wake_dt.replace(
+        hour=floored // 60, minute=floored % 60, second=0, microsecond=0)
+    return [
+        tf for tf in DASHBOARD_TFS
+        if floored % TF_INTERVAL_MINUTES[tf] == 0
+        and not _providers_base._is_daily_maintenance(
+            boundary - timedelta(minutes=TF_INTERVAL_MINUTES[tf]))
+    ]
+
+
+def catch_up_timeframes(wake_dt: datetime,
+                        last_boundary: Dict[str, float]) -> List[Timeframe]:
+    """Timeframes whose most recent closed boundary (<= wake) is newer than
+    the boundary of their last successful scan — i.e. boundaries that were
+    missed while a cycle overran or the process was down. A TF never scanned
+    is due. Excludes bars whose label sits in the daily settlement pause
+    (same rule as due_timeframes).
+
+    `due_timeframes` only ever names the timeframes whose boundary coincides
+    with THIS wake, so an overrunning cycle (scan + advisory panel + news can
+    take minutes) or a restart left the page serving the previous H1/H4
+    report for up to 1 h / 4 h while advertising a fresh next_close_epoch
+    (2026-09-13 audit)."""
+    minutes = wake_dt.hour * 60 + wake_dt.minute
+    out = []
+    for tf in DASHBOARD_TFS:
+        interval = TF_INTERVAL_MINUTES[tf]
+        floored = minutes - (minutes % interval)
+        last_closed = wake_dt.replace(
+            hour=floored // 60, minute=floored % 60, second=0, microsecond=0)
+        # No bar carries a label inside a scheduled closure (settlement
+        # pause or weekend), so nothing can have been "missed" there — and a
+        # weekend-blocked scan never advances the ledger, so without the
+        # weekend half of this test every weekend wake would re-scan
+        # M15/H1/H4 for nothing (~1,800 blocked rows per weekend).
+        # (Whole-span rule, as in providers.base._has_latest_closed_bar: an
+        # H4 bar labelled Sunday 20:00 UTC starts in the weekend but runs
+        # through the reopen, so it exists.)
+        label = last_closed - timedelta(minutes=interval)
+        if (_providers_base._is_scheduled_closure(label)
+                and _providers_base._is_scheduled_closure(
+                    last_closed - timedelta(seconds=1))):
+            continue
+        scanned = last_boundary.get(tf.value)
+        if scanned is None or scanned < last_closed.timestamp():
+            out.append(tf)
+    return out
+
+
+def seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
+    # Same math as telegram_bot.py — kept local so the dashboard never
+    # imports bot modules (independence rules above).
+    now = datetime.now(timezone.utc)
+    total_minutes = now.hour * 60 + now.minute
+    elapsed_minutes = total_minutes % interval_minutes
+    elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
+    interval_seconds = interval_minutes * 60
+    remaining = interval_seconds - elapsed_seconds + buffer
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
+    return remaining
+
+
+def _empty_tf_entry() -> dict:
+    return {"status": "warming", "scanned_at": None, "scan_seconds": None,
+            "error": None, "error_at": None, "report": None}
+
+
+def build_snapshot_payload(snapshots: Dict[str, dict], now: datetime) -> dict:
+    """snapshot.json: one entry per dashboard TF. Timeframes not yet
+    scanned appear as status="warming" so the page can render a
+    placeholder instead of a hole. server_time_epoch lets the client
+    compute its clock offset once so countdowns don't drift."""
+    timeframes = {}
+    for tf in DASHBOARD_TFS:
+        interval = TF_INTERVAL_MINUTES[tf]
+        minutes = now.hour * 60 + now.minute
+        secs_into = (minutes % interval) * 60 + now.second + now.microsecond / 1e6
+        next_close = now.timestamp() + (interval * 60 - secs_into)
+        entry = _empty_tf_entry()
+        entry.update(snapshots.get(tf.value) or {})
+        entry["next_close_epoch"] = next_close
+        entry["next_scan_epoch"] = next_close + SCAN_BUFFER_SECONDS
+        timeframes[tf.value] = entry
+    return _sanitize({
+        "symbol": XAUUSD,
+        "server_time_epoch": now.timestamp(),
+        "market_closed": _providers_base._is_forex_weekend(now),
+        "timeframes": timeframes,
+    })
+
+
+def publish(obj: dict, path: str) -> None:
+    """Atomic JSON write: temp file + os.replace, so Apache never serves
+    a torn file. allow_nan=False — a NaN slipping through would render
+    the page blank (JSON.parse rejects bare NaN)."""
+    data = json.dumps(obj, allow_nan=False, separators=(",", ":"))
+    tmp = path + ".tmp"
+    with open(tmp, "w", encoding="utf-8") as fh:
+        fh.write(data)
+    os.replace(tmp, path)
+
+
+# ══════════════════════════════════════════════════════════════════
+#  Runtime layer (constructed only from main(); never at import)
+# ══════════════════════════════════════════════════════════════════
+
+def _setup_logging(level: str = "INFO") -> None:
+    log_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "logs")
+    os.makedirs(log_dir, exist_ok=True)
+    log_file = os.path.join(log_dir, "dashboard.log")
+
+    fmt = logging.Formatter(
+        "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S")
+
+    file_handler = TimedRotatingFileHandler(
+        log_file, when="h", interval=3, backupCount=240, utc=True)
+    file_handler.setFormatter(fmt)
+
+    console_handler = logging.StreamHandler(sys.stdout)
+    console_handler.setFormatter(fmt)
+
+    logging.basicConfig(
+        level=getattr(logging, level.upper(), logging.INFO),
+        handlers=[file_handler, console_handler],
+        force=True)
+
+
+class DashboardPublisher:
+    def __init__(self, out_dir: str):
+        self.out_dir = out_dir
+        self.snapshots: Dict[str, dict] = {}
+        self._reports = {}
+        # tf.value → epoch seconds of the boundary the last successful scan
+        # belonged to; feeds catch_up_timeframes (2026-09-13 audit).
+        self._last_boundary: Dict[str, float] = {}
+        self.news_payload: Optional[dict] = None
+        self._dp = None
+        self._news_engine: Optional[NewsEngine] = None
+
+    def _provider(self):
+        if self._dp is None:
+            self._dp = make_data_provider(PlatformConfig())
+        return self._dp
+
+    def _news(self) -> NewsEngine:
+        if self._news_engine is None:
+            self._news_engine = NewsEngine(PlatformConfig().news)
+        return self._news_engine
+
+    def install_static(self) -> None:
+        """Copy the repo's static files (page + PHP summary endpoint)
+        into the out dir (atomic), so git pull + restart also updates
+        them. The ai/ cache dir is NOT created here — the daemon runs as
+        a different user than Apache; that dir is a one-time deploy step
+        (deploy/README-dashboard.md)."""
+        static_dir = os.path.join(
+            os.path.dirname(os.path.abspath(__file__)), "static")
+        for name in STATIC_FILES:
+            src = os.path.join(static_dir, name)
+            if not os.path.exists(src):
+                logger.warning(f"{src} missing — not installed")
+                continue
+            dst = os.path.join(self.out_dir, name)
+            shutil.copyfile(src, dst + ".tmp")
+            os.replace(dst + ".tmp", dst)
+
+    def scan_tf(self, tf: Timeframe,
+                boundary_epoch: Optional[float] = None) -> None:
+        """Scan one timeframe and store its snapshot entry.
+
+        `boundary_epoch` is the fallback boundary recorded when the report
+        carries no analysis (weekend/blocked or failed scan) or its bar
+        label will not parse; normally the boundary comes from the report
+        itself (bar label + interval)."""
+        started = time.time()
+        try:
+            report = produce_signal(XAUUSD, tf, self._provider(),
+                                    CONFIRMATIONS[tf])
+            # Display-only violent-move watch (dark by default). Never
+            # raises and never touches the signal — only watch_line.
+            if violent_watch_enabled():
+                attach_watch(report, tf,
+                             get_app_config().violent_watch_settings)
+            # Which boundary this report belongs to: the bar LABEL it closed
+            # on plus one interval. Recorded so a boundary missed by an
+            # overrunning cycle or a restart is caught up on the next wake
+            # instead of sitting stale (2026-09-13 audit).
+            boundary = boundary_epoch
+            if report.primary_analysis is not None:
+                try:
+                    boundary = (pd.Timestamp(report.timestamp).timestamp()
+                                + TF_INTERVAL_MINUTES[tf] * 60)
+                except Exception:
+                    boundary = boundary_epoch
+            if boundary is not None:
+                self._last_boundary[tf.value] = boundary
+            self._reports[tf.value] = report
+            self.snapshots[tf.value] = {
+                "status": "ok",
+                "scanned_at": datetime.now(timezone.utc).isoformat(),
+                "scan_seconds": round(time.time() - started, 2),
+                "error": None,
+                "error_at": None,
+                "report": report_to_webapp_dict(report),
+            }
+            logger.info(f"{tf.value}: {report.final_signal.name} "
+                        f"q={report.signal_quality or '-'} "
+                        f"({time.time() - started:.1f}s)")
+        except Exception:
+            # Keep the last good report; the page shows it with an
+            # error badge instead of going blank.
+            logger.exception(f"{tf.value} scan failed")
+            self._reports.pop(tf.value, None)
+            entry = self.snapshots.get(tf.value) or _empty_tf_entry()
+            entry["status"] = "error"
+            # The public snapshot carries a stable message; exception
+            # detail remains in dashboard.log for the operator.
+            entry["error"] = "Scan unavailable"
+            entry["error_at"] = datetime.now(timezone.utc).isoformat()
+            self.snapshots[tf.value] = entry
+
+    def attach_panels(self, tfs: List[Timeframe]) -> bool:
+        """Add advisory panels after publishing the technical snapshot.
+
+        A panel is display-only and can take tens of seconds on a cold LLM
+        request. Keeping its report object locally lets the page receive the
+        candle-close signal first, then a second atomic snapshot only when a
+        panel produced a result.
+        """
+        updated = False
+        for tf in tfs:
+            report = self._reports.get(tf.value)
+            if report is None:
+                continue
+            if not attach_panel(report, surface="dashboard"):
+                continue
+            entry = self.snapshots.get(tf.value)
+            if entry is None or entry.get("status") != "ok":
+                continue
+            entry["report"] = report_to_webapp_dict(report)
+            updated = True
+        return updated
+
+    def refresh_news(self) -> None:
+        try:
+            ne = self._news()
+            context = ne.get_fundamental_context()
+            week = ne.get_week_events()
+            self.news_payload = build_news_payload(
+                context, week, datetime.now(timezone.utc))
+        except Exception:
+            logger.exception("news refresh failed — keeping last payload")
+
+    def publish_snapshot(self) -> None:
+        publish(build_snapshot_payload(
+            self.snapshots, datetime.now(timezone.utc)),
+            os.path.join(self.out_dir, SNAPSHOT_FILE))
+
+    def publish_news(self) -> None:
+        if self.news_payload is not None:
+            publish(self.news_payload, os.path.join(self.out_dir, NEWS_FILE))
+
+    def run_cycle(self, tfs: List[Timeframe]) -> None:
+        for tf in tfs:  # sequential — see module docstring (cache dedup)
+            self.scan_tf(tf)
+        # Technical signals land before any optional slow work. News and the
+        # advisory panel are display-only and may take 20-30 seconds cold.
+        self.publish_snapshot()
+        if self.attach_panels(tfs):
+            self.publish_snapshot()
+        self.refresh_news()
+        self.publish_news()
+
+    def run_forever(self) -> None:
+        self.run_cycle(DASHBOARD_TFS)  # populate the page at boot
+        while True:
+            delay = seconds_until_boundary(5, buffer=SCAN_BUFFER_SECONDS)
+            logger.info(f"next M5 boundary in {delay:.0f}s")
+            time.sleep(delay)
+            try:
+                now = datetime.now(timezone.utc)
+                due = due_timeframes(now)
+                logger.info(f"boundary wake — scanning {[t.value for t in due]}")
+                missed = catch_up_timeframes(now, self._last_boundary)
+                extra = [tf for tf in missed if tf not in due]
+                if extra:
+                    logger.info(f"catch-up — scanning "
+                                f"{[t.value for t in extra]} (missed boundary)")
+                self.run_cycle(due + extra)
+            except Exception:
+                logger.exception("cycle failed")
+
+
+def main(argv=None) -> None:
+    ap = argparse.ArgumentParser(description="XAU/USD static dashboard publisher")
+    ap.add_argument("--once", action="store_true",
+                    help="one scan+publish cycle, then exit")
+    ap.add_argument("--out",
+                    default=os.environ.get("DASHBOARD_OUT_DIR", DEFAULT_OUT_DIR),
+                    help="output directory served by Apache "
+                         f"(default: $DASHBOARD_OUT_DIR or {DEFAULT_OUT_DIR})")
+    args = ap.parse_args(argv)
+
+    _setup_logging()
+    os.makedirs(args.out, exist_ok=True)
+    pub = DashboardPublisher(args.out)
+    pub.install_static()
+    if args.once:
+        pub.run_cycle(DASHBOARD_TFS)
+        print(f"Published to {args.out}")
+        return
+    pub.run_forever()
+
+
+if __name__ == "__main__":
+    main()

data_provider.py

@@ -1,547 +0,0 @@
-"""
-Forex Signal Intelligence Platform - Data Provider
-====================================================
-Retrieves and validates market data from Twelve Data API.
-Falls back to synthetic GBM data if the API is unavailable.
-"""
-
-import hashlib
-import logging
-import os
-import time
-from collections import deque
-from datetime import datetime, timedelta, timezone
-from typing import Dict, List, Optional
-
-import numpy as np
-import pandas as pd
-import requests
-
-from config import PlatformConfig, Timeframe, SYMBOL_DISPLAY
-
-logger = logging.getLogger(__name__)
-
-
-class DataIntegrityError(Exception):
-    """Raised when data fails validation checks."""
-    pass
-
-
-class ForexDataProvider:
-    """
-    Forex data provider backed by Twelve Data API.
-
-    Twelve Data has native 4h candles, eliminating the H1→H4 resampling
-    hack needed for Yahoo Finance. Free tier: 800 credits/day, 8 req/min.
-    """
-
-    TD_BASE_URL = "https://api.twelvedata.com"
-
-    TD_INTERVALS = {
-        Timeframe.M1:  "1min",
-        Timeframe.M5:  "5min",
-        Timeframe.M15: "15min",
-        Timeframe.M30: "30min",
-        Timeframe.H1:  "1h",
-        Timeframe.H4:  "4h",
-        Timeframe.D1:  "1day",
-        Timeframe.W1:  "1week",
-    }
-
-    # Approximate bars per day for each timeframe (used to size outputsize)
-    BARS_PER_DAY = {
-        Timeframe.M1:  1440,
-        Timeframe.M5:  288,
-        Timeframe.M15: 96,
-        Timeframe.M30: 48,
-        Timeframe.H1:  24,
-        Timeframe.H4:  6,
-        Timeframe.D1:  1,
-        Timeframe.W1:  0.143,
-    }
-
-    # Free plan: 8 API credits per minute (each symbol = 1 credit)
-    RATE_LIMIT = 8
-    RATE_WINDOW = 60.0
-
-    def __init__(self, config: PlatformConfig):
-        self.config = config
-        self._api_key = (
-            os.environ.get("TWELVE_DATA_API_KEY")
-            or config.twelve_data_api_key
-        )
-        self._cache: Dict[str, pd.DataFrame] = {}
-        self._cache_timestamps: Dict[str, float] = {}
-        self._request_times: deque = deque()  # timestamps of recent API credits used
-        self._session = requests.Session()
-        self._session.headers.update({
-            "User-Agent": "Mozilla/5.0 (compatible; ForexIntel/1.0)"
-        })
-
-    @staticmethod
-    def _to_td_symbol(symbol: str) -> str:
-        """Convert internal symbol (EURUSD=X) to Twelve Data format (EUR/USD)."""
-        clean = symbol.replace("=X", "")
-        return f"{clean[:3]}/{clean[3:]}"
-
-    def fetch_ohlcv(
-        self,
-        symbol: str,
-        timeframe: Timeframe,
-        lookback_days: Optional[int] = None,
-    ) -> pd.DataFrame:
-        """
-        Fetch OHLCV data for a Forex pair with full validation.
-
-        Returns a DataFrame with columns: Open, High, Low, Close, Volume
-        Index is a timezone-aware DatetimeIndex in UTC.
-        """
-        if lookback_days is None:
-            lookback_days = self.config.data_lookback_days
-
-        cache_key = f"{symbol}_{timeframe.value}"
-        now = time.time()
-
-        if (
-            cache_key in self._cache_timestamps
-            and (now - self._cache_timestamps[cache_key]) < self.config.refresh_interval
-            and cache_key in self._cache
-        ):
-            cached = self._cache.get(cache_key)
-            if cached is not None and not cached.empty:
-                return cached
-
-        df = self._fetch_twelve_data(symbol, timeframe, lookback_days)
-        df = self._validate_and_clean(df, symbol, timeframe)
-
-        self._cache[cache_key] = df
-        self._cache_timestamps[cache_key] = now
-        return df
-
-    def prefetch_batch(
-        self,
-        symbols: List[str],
-        timeframes: List[Timeframe],
-        lookback_days: Optional[int] = None,
-    ) -> None:
-        """
-        Batch-fetch all symbols for each timeframe in a single API call per TF.
-
-        Twelve Data accepts comma-separated symbols, so 7 pairs × 3 timeframes
-        = 3 requests instead of 21 — well within the free-tier 8 req/min limit.
-        Results are stored in the local cache; subsequent fetch_ohlcv calls
-        for the same symbol/TF will be served from cache.
-        """
-        if lookback_days is None:
-            lookback_days = self.config.data_lookback_days
-
-        for tf in timeframes:
-            try:
-                self._batch_fetch_timeframe(symbols, tf, lookback_days)
-                time.sleep(1.0)  # brief pause between TF requests
-            except Exception as e:
-                logger.warning(f"Batch prefetch failed for {tf.value}: {e}")
-
-    def fetch_multiple(
-        self,
-        symbols: List[str],
-        timeframe: Timeframe,
-        lookback_days: Optional[int] = None,
-    ) -> Dict[str, pd.DataFrame]:
-        """Fetch data for multiple symbols, returning dict of DataFrames."""
-        results = {}
-        for symbol in symbols:
-            try:
-                df = self.fetch_ohlcv(symbol, timeframe, lookback_days)
-                results[symbol] = df
-            except Exception as e:
-                logger.warning(f"Failed to fetch {symbol}: {e}")
-        return results
-
-    def get_latest_price(self, symbol: str) -> Optional[float]:
-        """Return the most recent close price for a symbol."""
-        try:
-            df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
-            if df is not None and not df.empty:
-                return float(df["Close"].iloc[-1])
-        except Exception:
-            pass
-        return None
-
-    def get_spread_estimate(self, symbol: str) -> float:
-        """Estimate typical spread from recent M5 high-low ranges."""
-        try:
-            df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
-            if df is not None and not df.empty:
-                return float((df["High"] - df["Low"]).median())
-        except Exception:
-            pass
-
-        fallback_spreads = {
-            "EURUSD=X": 0.00010, "GBPUSD=X": 0.00015,
-            "USDJPY=X": 0.010,   "USDCHF=X": 0.00015,
-            "AUDUSD=X": 0.00015, "USDCAD=X": 0.00018,
-            "NZDUSD=X": 0.00020,
-        }
-        return fallback_spreads.get(symbol, 0.00020)
-
-    def get_pair_info(self, symbol: str) -> Dict:
-        """Return display name and pip value for a pair."""
-        display = SYMBOL_DISPLAY.get(symbol, symbol)
-        is_jpy = "JPY" in symbol
-        return {
-            "symbol": symbol,
-            "display": display,
-            "pip_value": 0.01 if is_jpy else 0.0001,
-            "is_jpy_pair": is_jpy,
-        }
-
-    # ── Private Methods ────────────────────────────────────────────
-
-    def _throttle(self, credits: int = 1) -> None:
-        """
-        Block until `credits` API credits are available.
-
-        Twelve Data free tier: 8 credits/minute. Each symbol in any request
-        (single or batch) consumes 1 credit. This method enforces the limit
-        regardless of whether requests are batched or individual.
-        """
-        while True:
-            now = time.time()
-            # Evict credits older than the rate window
-            while self._request_times and now - self._request_times[0] >= self.RATE_WINDOW:
-                self._request_times.popleft()
-
-            available = self.RATE_LIMIT - len(self._request_times)
-            if available >= credits:
-                for _ in range(credits):
-                    self._request_times.append(time.time())
-                return
-
-            # Wait until the oldest credit expires
-            wait = (self._request_times[0] + self.RATE_WINDOW) - now + 0.2
-            logger.info(
-                f"Rate limit: {len(self._request_times)}/{self.RATE_LIMIT} credits used "
-                f"— waiting {wait:.1f}s for next window..."
-            )
-            time.sleep(wait)
-
-    def _batch_fetch_timeframe(
-        self, symbols: List[str], timeframe: Timeframe, lookback_days: int
-    ) -> None:
-        """
-        Single API call fetching all symbols for one timeframe.
-        Twelve Data returns a dict keyed by symbol when multiple are requested.
-        Parsed DataFrames are validated and stored in the cache.
-        """
-        if not self._api_key:
-            return
-
-        td_symbols = [self._to_td_symbol(s) for s in symbols]
-        interval = self.TD_INTERVALS[timeframe]
-        outputsize = min(
-            int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20, 5000
-        )
-        outputsize = max(outputsize, 300)
-
-        # Each symbol in a batch request costs 1 credit
-        self._throttle(credits=len(symbols))
-
-        resp = self._session.get(
-            f"{self.TD_BASE_URL}/time_series",
-            params={
-                "symbol": ",".join(td_symbols),
-                "interval": interval,
-                "outputsize": outputsize,
-                "apikey": self._api_key,
-                "timezone": "UTC",
-                "format": "json",
-            },
-            timeout=30,
-        )
-        resp.raise_for_status()
-        data = resp.json()
-
-        # If the whole batch request returned a top-level error, raise it
-        if isinstance(data, dict) and data.get("status") == "error":
-            raise DataIntegrityError(
-                f"Batch API error ({timeframe.value}): {data.get('message', 'unknown')}"
-            )
-
-        # Batch response: {"EUR/USD": {...}, "GBP/USD": {...}, ...}
-        now = time.time()
-        for symbol, td_sym in zip(symbols, td_symbols):
-            try:
-                sym_data = data.get(td_sym, {})
-                if sym_data.get("status") == "error":
-                    logger.warning(
-                        f"Batch fetch error for {td_sym}: {sym_data.get('message')}"
-                    )
-                    continue
-
-                values = sym_data.get("values")
-                if not values:
-                    logger.warning(f"No values in batch response for {td_sym}")
-                    continue
-
-                records = [
-                    {
-                        "Datetime": pd.Timestamp(v["datetime"], tz="UTC"),
-                        "Open":   float(v["open"]),
-                        "High":   float(v["high"]),
-                        "Low":    float(v["low"]),
-                        "Close":  float(v["close"]),
-                        "Volume": float(v.get("volume") or 0),
-                    }
-                    for v in reversed(values)
-                ]
-                df = pd.DataFrame(records).set_index("Datetime")
-                df.index.name = "Datetime"
-                df = self._validate_and_clean(df, symbol, timeframe)
-
-                cache_key = f"{symbol}_{timeframe.value}"
-                self._cache[cache_key] = df
-                self._cache_timestamps[cache_key] = now
-                logger.debug(f"Batch cached {len(df)} bars for {td_sym} @ {interval}")
-
-            except Exception as e:
-                logger.warning(f"Failed to parse batch data for {td_sym}: {e}")
-
-    def _fetch_twelve_data(
-        self, symbol: str, timeframe: Timeframe, lookback_days: int
-    ) -> pd.DataFrame:
-        """
-        Fetch OHLCV from Twelve Data REST API.
-        Falls back to synthetic GBM data if API is unavailable or key is missing.
-        """
-        if not self._api_key:
-            logger.warning("No Twelve Data API key — using synthetic data. "
-                           "Set TWELVE_DATA_API_KEY env var or config.twelve_data_api_key.")
-            return self._generate_synthetic_data(symbol, timeframe, lookback_days)
-
-        td_symbol = self._to_td_symbol(symbol)
-        interval = self.TD_INTERVALS[timeframe]
-        outputsize = min(
-            int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20,
-            5000,
-        )
-        outputsize = max(outputsize, 300)
-
-        self._throttle(credits=1)
-
-        try:
-            resp = self._session.get(
-                f"{self.TD_BASE_URL}/time_series",
-                params={
-                    "symbol": td_symbol,
-                    "interval": interval,
-                    "outputsize": outputsize,
-                    "apikey": self._api_key,
-                    "timezone": "UTC",
-                    "format": "json",
-                },
-                timeout=15,
-            )
-            resp.raise_for_status()
-            data = resp.json()
-
-            if data.get("status") == "error":
-                raise DataIntegrityError(
-                    f"Twelve Data API error for {td_symbol}: {data.get('message', 'unknown')}"
-                )
-
-            values = data.get("values")
-            if not values:
-                raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
-
-            # Values arrive newest-first; reverse to chronological order
-            records = [
-                {
-                    "Datetime": pd.Timestamp(v["datetime"], tz="UTC"),
-                    "Open":   float(v["open"]),
-                    "High":   float(v["high"]),
-                    "Low":    float(v["low"]),
-                    "Close":  float(v["close"]),
-                    "Volume": float(v.get("volume") or 0),
-                }
-                for v in reversed(values)
-            ]
-
-            df = pd.DataFrame(records).set_index("Datetime")
-            df.index.name = "Datetime"
-            logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
-            return df
-
-        except DataIntegrityError:
-            raise
-        except Exception as e:
-            logger.info(f"Twelve Data fetch failed for {symbol} ({e}), using synthetic data.")
-            return self._generate_synthetic_data(symbol, timeframe, lookback_days)
-
-    def _generate_synthetic_data(
-        self, symbol: str, timeframe: Timeframe, lookback_days: int
-    ) -> pd.DataFrame:
-        """
-        Generate realistic synthetic Forex OHLCV using GBM with mean-reversion.
-        Used as a fallback when live data is unavailable.
-        """
-        pair_params = {
-            "EURUSD=X": {"price": 1.1350, "vol": 0.07, "drift": -0.001},
-            "GBPUSD=X": {"price": 1.3200, "vol": 0.08, "drift":  0.002},
-            "USDJPY=X": {"price": 144.50, "vol": 0.09, "drift":  0.003},
-            "USDCHF=X": {"price": 0.8850, "vol": 0.07, "drift": -0.001},
-            "AUDUSD=X": {"price": 0.6550, "vol": 0.10, "drift": -0.002},
-            "USDCAD=X": {"price": 1.3650, "vol": 0.07, "drift":  0.001},
-            "NZDUSD=X": {"price": 0.6100, "vol": 0.10, "drift": -0.001},
-            "EURGBP=X": {"price": 0.8590, "vol": 0.06, "drift":  0.000},
-            "EURJPY=X": {"price": 163.90, "vol": 0.09, "drift":  0.002},
-            "GBPJPY=X": {"price": 190.70, "vol": 0.11, "drift":  0.003},
-            "AUDJPY=X": {"price": 94.60,  "vol": 0.11, "drift":  0.001},
-            "EURCHF=X": {"price": 0.9380, "vol": 0.05, "drift":  0.000},
-            "CADJPY=X": {"price": 105.80, "vol": 0.09, "drift":  0.001},
-        }
-
-        params = pair_params.get(symbol, {"price": 1.0000, "vol": 0.08, "drift": 0.000})
-        base_price = params["price"]
-        annual_vol = params["vol"]
-        annual_drift = params["drift"]
-
-        tf_minutes = {
-            Timeframe.M1: 1, Timeframe.M5: 5, Timeframe.M15: 15,
-            Timeframe.M30: 30, Timeframe.H1: 60, Timeframe.H4: 240,
-            Timeframe.D1: 1440, Timeframe.W1: 10080,
-        }
-        bar_minutes = tf_minutes.get(timeframe, 60)
-        bars_per_year = (365.25 * 24 * 60) / bar_minutes
-        dt = 1.0 / bars_per_year
-
-        n_bars = min(int(lookback_days * 24 * 60 / bar_minutes), 2000)
-        n_bars = max(n_bars, 300)
-
-        seed_base = int(hashlib.md5(symbol.encode()).hexdigest()[:8], 16)
-        time_component = int(time.time()) // 3600
-        np.random.seed((seed_base + time_component) % (2**31))
-
-        bar_vol = annual_vol * np.sqrt(dt)
-        bar_drift = annual_drift * dt
-
-        regime_len = np.random.randint(20, 80)
-        trend_strength = np.random.normal(0, 0.3)
-
-        price = base_price
-        prices = [price]
-
-        for i in range(1, n_bars):
-            if i % regime_len == 0:
-                regime_len = np.random.randint(20, 80)
-                trend_strength = np.random.normal(0, 0.3)
-
-            mr_force = -0.001 * (price - base_price) / base_price
-            ret = bar_drift + trend_strength * dt + mr_force + bar_vol * np.random.randn()
-
-            if np.random.random() < 0.02:
-                ret *= np.random.uniform(2.0, 4.0)
-
-            price *= (1 + ret)
-            prices.append(price)
-
-        prices = np.array(prices)
-        opens = np.roll(prices, 1)
-        opens[0] = prices[0]
-
-        typical_range = bar_vol * base_price * 1.5
-        highs = np.maximum(prices, opens) + np.abs(np.random.normal(0, typical_range * 0.6, n_bars))
-        lows  = np.minimum(prices, opens) - np.abs(np.random.normal(0, typical_range * 0.6, n_bars))
-        highs = np.maximum(highs, np.maximum(opens, prices))
-        lows  = np.minimum(lows,  np.minimum(opens, prices))
-
-        volume = np.random.lognormal(mean=10, sigma=0.5, size=n_bars).astype(int)
-
-        freq_map = {
-            Timeframe.M1: "1min",  Timeframe.M5: "5min",
-            Timeframe.M15: "15min", Timeframe.M30: "30min",
-            Timeframe.H1: "1h",    Timeframe.H4: "4h",
-            Timeframe.D1: "D",     Timeframe.W1: "W",
-        }
-        freq = freq_map.get(timeframe, "1h")
-        end_time = pd.Timestamp.now(tz="UTC").floor(f"{bar_minutes}min")
-        index = pd.date_range(end=end_time, periods=n_bars, freq=freq, tz="UTC")
-
-        df = pd.DataFrame({
-            "Open": opens, "High": highs, "Low": lows,
-            "Close": prices, "Volume": volume,
-        }, index=index)
-        df.index.name = "Datetime"
-
-        logger.info(
-            f"Generated {n_bars} synthetic {timeframe.value} bars for "
-            f"{SYMBOL_DISPLAY.get(symbol, symbol)} @ {prices[-1]:.5f}"
-        )
-        return df
-
-    def _validate_and_clean(
-        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
-    ) -> pd.DataFrame:
-        """
-        Validate and clean OHLCV data.
-
-        Steps: drop all-NaN rows → ffill small gaps → remove future timestamps
-        → fix OHLC inconsistencies → remove >10-sigma outliers → sort & deduplicate.
-        """
-        if df is None or df.empty:
-            raise DataIntegrityError(f"Empty dataset for {symbol}")
-
-        original_len = len(df)
-
-        df = df.dropna(subset=["Open", "High", "Low", "Close"], how="all")
-        df = df.ffill(limit=3)
-        df = df.dropna(subset=["Close"])
-
-        now = pd.Timestamp.utcnow()
-        future_mask = df.index > now + pd.Timedelta(minutes=5)
-        if future_mask.any():
-            logger.warning(f"{symbol}: Dropping {future_mask.sum()} future timestamps")
-            df = df[~future_mask]
-
-        invalid_ohlc = (
-            (df["High"] < df["Low"]) |
-            (df["High"] < df["Open"]) |
-            (df["High"] < df["Close"]) |
-            (df["Low"] > df["Open"]) |
-            (df["Low"] > df["Close"])
-        )
-        if invalid_ohlc.any():
-            logger.warning(f"{symbol}: Fixing {invalid_ohlc.sum()} OHLC-inconsistent bars")
-            for idx in df[invalid_ohlc].index:
-                vals = [df.loc[idx, "Open"], df.loc[idx, "Close"]]
-                df.loc[idx, "High"] = max(vals + [df.loc[idx, "High"]])
-                df.loc[idx, "Low"]  = min(vals + [df.loc[idx, "Low"]])
-
-        returns = df["Close"].pct_change()
-        std_ret = returns.std()
-        if std_ret > 0:
-            outlier_mask = (returns - returns.mean()).abs() > 10 * std_ret
-            if outlier_mask.any():
-                logger.warning(f"{symbol}: Removing {outlier_mask.sum()} extreme outlier bars")
-                df = df[~outlier_mask]
-
-        min_required = max(
-            self.config.technical.ema_trend,
-            self.config.technical.ichimoku_senkou_b,
-            200,
-        ) + 10
-
-        if len(df) < min_required:
-            logger.warning(
-                f"{symbol} {timeframe.value}: Only {len(df)} bars "
-                f"(need {min_required}). Some indicators will be partial."
-            )
-
-        df = df.sort_index()
-        df = df[~df.index.duplicated(keep="last")]
-
-        cleaned = original_len - len(df)
-        if cleaned > 0:
-            logger.info(f"{symbol}: Cleaned {cleaned}/{original_len} bars")
-
-        return df

deploy/README-dashboard.md

@@ -0,0 +1,161 @@
+# Deploying the web dashboard (live LAMP server)
+
+The dashboard is a **static publisher**: `dashboard_publisher.py` runs as its
+own daemon, scans XAU/USD at candle closes, and atomically writes
+`dashboard.html` + `news_summary.php` + `snapshot.json` + `news.json` into a
+folder Apache already serves. Apache does all web serving — no new port, no
+Python web server, and the telegram bot process is completely unaffected by
+anything that happens here (see the module docstring for the independence
+rules). The one dynamic piece is `news_summary.php` (§6): an on-demand
+DeepSeek summary for a clicked headline, whitelist-gated against news.json.
+
+All steps below run **on the live server** (`/home/Forex/ForexBot2`).
+
+## 1. Get the code
+
+```bash
+cd /home/Forex/ForexBot2
+git pull
+venv/bin/python test_all.py        # optional sanity: suite must be green
+```
+
+No new Python dependencies are required.
+
+## 2. Make the web folder writable
+
+The dashboard publishes straight into the portal docroot `/var/www/trade/`
+(owner decision 2026-08-03) — it adds exactly six files there:
+`dashboard.html`, `dashboard.css`, `dashboard.js`, `news_summary.php`,
+`snapshot.json`, `news.json` (plus the
+apache-owned `ai/` cache dir from §6). The daemon user must be able to write
+them:
+
+```bash
+# if the portal files can be owned by the daemon user:
+sudo chown Forex: /var/www/trade
+# otherwise grant write without changing ownership:
+sudo setfacl -m u:Forex:rwx /var/www/trade
+```
+
+(A different path is one edit away: the unit's `Environment=DASHBOARD_OUT_DIR=`
+line, or `--out` for one-off runs.)
+
+SELinux (RHEL/Alma/CentOS enforcing): directories under `/var/www` normally
+already carry `httpd_sys_content_t`, and new files inherit the parent label, so
+no action is usually needed. If Apache 403s despite correct permissions:
+
+```bash
+sudo restorecon -R /var/www/trade
+```
+
+## 3. Smoke-test once
+
+```bash
+venv/bin/python dashboard_publisher.py --once
+ls /var/www/trade    # dashboard.html/.css/.js news_summary.php snapshot.json news.json added next to the portal
+```
+
+Then open `http://<server>/dashboard.html` (path relative to however the
+vhost maps `/var/www/trade`) — all four timeframe cards, the calendar, and
+the newswire should populate.
+
+## 4. Install the daemon
+
+```bash
+sudo cp deploy/gold-dashboard.service /etc/systemd/system/
+# edit User= if the repo owner is not "Forex"
+sudo systemctl daemon-reload
+sudo systemctl enable --now gold-dashboard
+systemctl status gold-dashboard
+tail -f logs/dashboard.log
+```
+
+The daemon scans all four timeframes at boot, then wakes at every M5
+boundary (+10 s) and rescans whichever timeframes just closed. Logs rotate
+every 3 h (240 backups); the bot's own `telegram.log` rotates daily (30
+backups) since 2026-09-08.
+
+## 5. Optional Apache polish
+
+The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache
+answers unchanged files with 304), so no Apache config is strictly needed. To
+be explicit anyway:
+
+```apache
+<Directory "/var/www/trade">
+    <FilesMatch "\.json$">
+        Header set Cache-Control "no-cache"
+    </FilesMatch>
+</Directory>
+```
+
+The page is deliberately public (owner decision 2026-08-03). To restrict it
+later, add standard Apache basic auth (`htpasswd` + `Require valid-user`) on
+that directory — no code changes needed.
+
+## 6. AI summary endpoint (PHP) — one-time setup
+
+Clicking a newswire headline opens a detail dialog that fetches an on-demand
+DeepSeek gold-impact summary from `news_summary.php` (installed next to the
+page by the daemon). Owner decision 2026-08-07: a real on-demand endpoint on
+the public page, accepted with these guards baked into the PHP — the id must
+match a headline in the published news.json (arbitrary text can never reach
+the LLM), each headline's summary is cached forever in `ai/` (≤1 API call per
+headline, ever), and a file-based daily cap fails **closed** on I/O errors.
+Since 2026-09-04 the engine publishes only headlines up to 4 h old whose
+score leans one way (neutral items are dropped), so a headline leaves the
+whitelist sooner: a dialog left open past that point returns `unknown_id`
+("This headline has left the live feed"); summaries already cached in `ai/`
+stay reachable.
+
+Requirements: `php` + the curl extension (`php -m | grep curl`).
+
+```bash
+# 1. SELinux: allow httpd outbound HTTPS — WITHOUT this every summary call
+#    dies as a curl timeout even with perfect perms (RHEL-family default).
+sudo setsebool -P httpd_can_network_connect 1
+
+# 2. API key OUTSIDE the docroot, readable by Apache only:
+sudo install -d -o root -g root -m 755 /etc/gold-dashboard
+sudo sh -c 'umask 137; echo "sk-..." > /etc/gold-dashboard/deepseek.key'
+sudo chown root:apache /etc/gold-dashboard/deepseek.key   # mode 640
+
+# 3. apache-owned cache dir (the daemon runs as Forex and can't create it):
+sudo install -d -o apache -g apache -m 755 /var/www/trade/ai
+sudo semanage fcontext -a -t httpd_sys_rw_content_t '/var/www/trade/ai(/.*)?'
+sudo restorecon -Rv /var/www/trade/ai
+```
+
+Knobs (Apache `SetEnv` in the vhost, both optional):
+`AI_SUMMARY_DAILY_CAP` (default 150), `DEEPSEEK_KEY_FILE` (default
+`/etc/gold-dashboard/deepseek.key`).
+
+Verify from the shell — expect 400 / 404 / 200 / 200-from-cache:
+
+```bash
+curl -s "http://<server>/news_summary.php?id=zzz"                 # {"error":"bad_id"}
+curl -s "http://<server>/news_summary.php?id=$(printf 'a%.0s' {1..40})"  # unknown_id
+ID=$(python3 -c "import json;print(json.load(open('/var/www/trade/news.json'))['headlines'][0]['id'])")
+curl -s "http://<server>/news_summary.php?id=$ID"   # real summary (2-12 s)
+curl -s "http://<server>/news_summary.php?id=$ID"   # instant, from ai/<id>.json
+```
+
+## Operational notes
+
+- **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
+  re-copies `static/dashboard.html`, `dashboard.css`, `dashboard.js` and
+  `news_summary.php` into the web folder at startup.
+- **If the daemon dies**, systemd restarts it in 30 s; worst case the page
+  shows stale data with its own "updated N min ago" evidence. Apache and the
+  telegram bot are unaffected.
+- **DeepSeek**: the publisher process has its own daily request cap
+  (`deepseek.daily_request_cap`, default 500) separate from the bot's —
+  expect roughly up to ~430 extra calls/day worst case, V4-Flash pricing.
+- **Two independent DeepSeek meters** hit the same API key: the publisher's
+  in-process cap above, and the PHP endpoint's file cap (`ai/.ht_cap.json`,
+  default 150/day). They don't communicate — worst-case spend is their sum;
+  DeepSeek-side spend alerting is the true backstop. Kept separate on
+  purpose (independence rule: the PHP never imports/reads bot code or
+  config.json).
+- **Credentials**: uses the same env/.env/config.json resolution as the rest
+  of the platform (Oanda candles feed; news needs no key; DeepSeek optional).

deploy/gold-dashboard.service

@@ -0,0 +1,23 @@
+# systemd unit for the XAU/USD static dashboard publisher.
+# Install (as root, on the live server):
+#   cp deploy/gold-dashboard.service /etc/systemd/system/
+#   systemctl daemon-reload && systemctl enable --now gold-dashboard
+# See deploy/README-dashboard.md for the full checklist.
+
+[Unit]
+Description=XAU/USD dashboard publisher (writes static JSON/HTML for Apache)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+# Adjust to the account that owns /home/Forex/ForexBot2:
+User=Forex
+WorkingDirectory=/home/Forex/ForexBot2
+Environment=DASHBOARD_OUT_DIR=/var/www/trade
+ExecStart=/home/Forex/ForexBot2/venv/bin/python dashboard_publisher.py
+Restart=always
+RestartSec=30
+
+[Install]
+WantedBy=multi-user.target

docs/beginner_guide.html

@@ -0,0 +1,170 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>ForexBot2 — Beginner's Guide</title>
+<style>
+  @page {
+    size: A4;
+    margin: 22mm 18mm;
+    @bottom-center { content: "ForexBot2 Beginner's Guide — page " counter(page) " of " counter(pages); font-size: 8pt; color: #888; }
+  }
+  * { box-sizing: border-box; }
+  body { font-family: "DejaVu Sans", Helvetica, Arial, sans-serif; font-size: 10.5pt; line-height: 1.65; color: #1c2733; margin: 0; }
+  h1 { font-size: 21pt; margin: 0 0 2mm; color: #0d1b2a; }
+  .subtitle { color: #5a6b7c; font-size: 10.5pt; margin-bottom: 8mm; }
+  h2 { font-size: 13.5pt; color: #0d1b2a; border-bottom: 2px solid #d4a017; padding-bottom: 1.2mm; margin: 8mm 0 3mm; page-break-after: avoid; }
+  h3 { font-size: 11pt; margin: 5mm 0 1.5mm; color: #23374d; page-break-after: avoid; }
+  p { margin: 0 0 2.8mm; }
+  table { border-collapse: collapse; width: 100%; margin: 3mm 0 4mm; font-size: 9.3pt; page-break-inside: avoid; }
+  th { background: #0d1b2a; color: #fff; text-align: left; padding: 1.8mm 2.5mm; font-weight: 600; }
+  td { border-bottom: 1px solid #dde4ea; padding: 1.8mm 2.5mm; vertical-align: top; }
+  tr:nth-child(even) td { background: #f4f7fa; }
+  table.breakable { page-break-inside: auto; }
+  table.breakable tr { page-break-inside: avoid; }
+  .signal-box { background: #f4f7fa; border: 1px solid #c9d4de; border-left: 4px solid #d4a017; border-radius: 2mm; padding: 3.5mm 4.5mm; font-family: "DejaVu Sans Mono", monospace; font-size: 8.6pt; line-height: 1.55; white-space: pre-wrap; margin: 3mm 0 4mm; page-break-inside: avoid; }
+  .callout { background: #fdf6e3; border: 1px solid #e8d49a; border-left: 4px solid #d4a017; border-radius: 2mm; padding: 3mm 4.5mm; margin: 3mm 0 4mm; page-break-inside: avoid; }
+  .callout.red { background: #fdf0ee; border-color: #eab4ac; border-left-color: #c0392b; }
+  .callout.green { background: #eef7ee; border-color: #b3d9b3; border-left-color: #2e7d32; }
+  .callout b:first-child { display: block; margin-bottom: 1mm; }
+  .example { background: #eef2f6; border-radius: 2mm; padding: 3mm 4.5mm; margin: 2mm 0 4mm; page-break-inside: avoid; }
+  .example b:first-child { color: #23374d; }
+  ul, ol { margin: 0 0 3mm; padding-left: 6mm; }
+  li { margin-bottom: 1.6mm; }
+  code { font-family: "DejaVu Sans Mono", monospace; font-size: 8.8pt; background: #eef2f6; padding: 0.3mm 1.2mm; border-radius: 1mm; }
+  .small { font-size: 8.5pt; color: #5a6b7c; }
+  .disclaimer { border: 1.5px solid #c0392b; border-radius: 2mm; padding: 3.5mm 4.5mm; margin-top: 6mm; font-size: 9pt; page-break-inside: avoid; }
+  .disclaimer b { color: #c0392b; }
+</style>
+</head>
+<body>
+
+<h1>ForexBot2 — Beginner's Guide</h1>
+<div class="subtitle">Everything explained in plain words. No experience needed. &nbsp;·&nbsp; July 2026</div>
+
+<h2>1 · What is this bot?</h2>
+<p>ForexBot2 is a computer program that watches the <b>price of gold</b> all day and all night. It studies the price charts using nine different mathematical checks. When the checks agree that the price is likely to move in one direction, it sends you a message on <b>Telegram</b> telling you:</p>
+<ul>
+<li>whether to <b>buy</b> or <b>sell</b>,</li>
+<li>at what price to get in,</li>
+<li>at what price to get out if the trade goes wrong (so your loss stays small),</li>
+<li>and how to lock in profit if the trade goes well.</li>
+</ul>
+<p>Most days the bot stays quiet for hours. <b>That is normal.</b> Quiet means "nothing worth trading right now." Never trade just because you are bored.</p>
+
+<h2>2 · Ten words you need to know</h2>
+<p>Every trading word used in this guide (and in the bot's messages) is explained here. Come back to this table whenever you see a word you don't remember.</p>
+<table class="breakable">
+<tr><th style="width:26%">Word</th><th>Plain meaning</th></tr>
+<tr><td><b>BUY (going "long")</b></td><td>You make money if the gold price goes <b>up</b>.</td></tr>
+<tr><td><b>SELL (going "short")</b></td><td>You make money if the gold price goes <b>down</b>. Yes, that's possible — your broker lets you sell gold you don't own and buy it back cheaper later.</td></tr>
+<tr><td><b>Entry</b></td><td>The price at which you open the trade.</td></tr>
+<tr><td><b>Stop loss (SL)</b></td><td>A safety brake. You tell your broker: "if the price reaches this level, close my trade automatically." It puts a hard limit on how much one trade can lose. <b>Every trade must have one.</b></td></tr>
+<tr><td><b>Take profit (TP)</b></td><td>The opposite of a stop loss: a level where the trade closes automatically <b>with a profit</b>. In this system it is only a rough guide — see section 6.</td></tr>
+<tr><td><b>Risk, or "R"</b></td><td>The amount of money you have chosen to lose if the trade fails. If you decide to risk $20, then <b>1R = $20</b>. A trade that wins "+2R" made you $40. This is how traders measure results.</td></tr>
+<tr><td><b>Trailing stop ("trail")</b></td><td>A stop loss that <b>follows the price</b> as it moves in your favor, but never moves backwards. Explained with numbers in section 6.</td></tr>
+<tr><td><b>Timeframe (TF)</b></td><td>The size of each candle on the chart. <b>M5</b> = each candle is 5 minutes, <b>M15</b> = 15 minutes, <b>H1</b> = 1 hour, <b>H4</b> = 4 hours. Bigger timeframe = slower, calmer trading.</td></tr>
+<tr><td><b>XAU/USD</b></td><td>Just the official symbol for "gold priced in US dollars."</td></tr>
+<tr><td><b>Spread</b></td><td>The tiny difference between the buying price and the selling price. It's your broker's fee, paid on every trade.</td></tr>
+</table>
+
+<h2>3 · Reading a bot message</h2>
+<p>Here is a typical signal, with plain-English notes below it:</p>
+<div class="signal-box">🔍 XAU/USD Signal Analysis  (H1)          ← gold, on the 1-hour chart
+🟢 BUY   Grade: A  |  64% confidence      ← what to do + how good the setup looks
+✅ MTF aligned                            ← bigger and smaller charts agree
+📋 Indicators …                           ← the bot's 9 checks (you can skip this part)
+━━━━━━━━━━━━━━━━━━━━
+🚀 TRADE SETUP                            ← ↓ THIS is the part you act on ↓
+📈 Direction: BUY
+▶ Entry:  4,132.50                        ← open the trade near this price
+🛑 SL:     4,088.20                        ← your safety brake (stop loss)
+🎯 TP1:    4,176.80                        ← rough profit target (a guide, not a rule)
+🔄 Trail:  once price hits 4,159.08,
+   trail SL $33.23 behind best price      ← how to lock in profit (see section 6)</div>
+<ul>
+<li><b>Grade (A+, A, B, C):</b> the bot's own mark for how clean the setup is. A+ is best. It never sends anything worse than C.</li>
+<li><b>Confidence and the indicator list:</b> extra detail for curious people. You don't need to understand it to trade the signal.</li>
+<li><b>⚠️ Warning lines:</b> things the bot noticed, like high volatility. They are "be aware," not "don't trade."</li>
+<li><b>🌪 Violent-move watch:</b> now and then the bot adds a line saying the next hour could produce very big candles — usually because an important American economic number is about to come out, or the market is in an hour that is historically wild, or the price has stretched a long way from normal. It is only a "hold on tight" note. It does not change the signal, and it is not a reason to close or move your stop loss early — the bot was tested doing exactly that and it lost money. Your stop loss is already your protection; leave it where it is.</li>
+<li><b>📰 News line:</b> a short plain-English note on what is moving the gold price right now and why, written from the last few hours of headlines. It is background information, not a reason to trade.</li>
+<li><b>The trade setup at the bottom is the only part you must act on.</b></li>
+</ul>
+
+<h2>4 · How much money to put in (the most important section)</h2>
+<p>You never put your whole account into one trade. You decide how much you are willing to <b>lose</b>, and work backwards from there. The rule:</p>
+<div class="callout"><b>Risk 1% of your account per trade. Never more.</b> If you have $2,000, that means one trade may lose at most $20. Losing trades are a normal part of the system — 1% means a bad week stings but never hurts you badly.</div>
+<div class="example"><b>Worked example.</b><br>
+Your account: <b>$2,000</b>. Your risk: 1% = <b>$20</b>. So for this trade, 1R = $20.<br>
+The signal says: Entry 4,132.50, Stop loss 4,088.20. The gap between them is <b>$44.30 per ounce</b> of gold.<br>
+How much gold to buy: $20 ÷ $44.30 = <b>0.45 ounces</b>.<br>
+Now, whatever happens: if the price falls to the stop loss, you lose about $20 — and no more. If the trade wins 2R, you make about $40.</div>
+<p class="small">Most broker apps have a built-in calculator: enter the stop-loss price and the amount you want to risk ($20), and it works out the position size for you. Use it — it avoids mistakes. (On many platforms gold is traded in "lots," where 0.01 lot = 1 ounce.)</p>
+
+<h2>5 · What to do when a signal arrives — step by step</h2>
+<ol>
+<li><b>Check the timeframe</b> in the first line. Only act on the one timeframe you have chosen to follow (see section 7). Ignore the rest.</li>
+<li><b>Work out your position size</b> using the 1% rule above. Do this before anything else.</li>
+<li><b>Open the trade</b> in your broker app, in the direction shown (BUY or SELL), at the current market price. Try to act soon after the message; if the price has already run far past the entry price, skip the trade — another signal will come.</li>
+<li><b>Set the stop loss immediately</b>, at the exact SL price in the message. Not "in your head" — in the app. Gold can move fast, and the stop loss is what protects you while you sleep.</li>
+<li><b>Then manage the trade with the trailing stop</b>, explained next.</li>
+</ol>
+
+<h2>6 · The trailing stop — how winners get paid</h2>
+<p>The take-profit price (TP1) in the message is only a guide. The system makes its money by <b>letting winning trades run</b> and following them with a moving safety brake. Every signal message tells you exactly when to start and how far behind to follow. Here is the idea with numbers:</p>
+<div class="example"><b>Example.</b> You bought gold at <b>4,132</b>, stop loss at <b>4,088</b> (risking $44 per ounce).<br>
+The message says: <i>"once price hits 4,159, trail SL $33 behind best price."</i><br><br>
+<b>Step 1 — wait.</b> Until gold reaches 4,159, do nothing. If it falls to 4,088 first, the stop loss closes the trade. That loss is normal and already budgeted.<br>
+<b>Step 2 — the price reaches 4,159.</b> Now move your stop loss up to 4,159 − 33 = <b>4,126</b>. From here on, the worst case is roughly break-even.<br>
+<b>Step 3 — keep following.</b> Each time the price makes a new high, move the stop up so it stays $33 below that high. Price climbs to 4,180? Stop goes to 4,147 — profit is now locked in. Price climbs to 4,210? Stop goes to 4,177.<br>
+<b>Step 4 — the price turns.</b> Eventually gold falls back $33 from its best point and hits your stop. The trade closes by itself, and you keep everything above your entry. <b>You never decide when to take profit — the trail decides.</b></div>
+<ul>
+<li>Only move the stop <b>up</b> (for a BUY). Never move it back down. For a SELL, everything is mirrored: the stop only moves down.</li>
+<li>Move it when a candle finishes, not every second. Checking once per candle (every hour, on the 1-hour timeframe) is enough.</li>
+<li>Many broker apps can do this automatically — look for a "trailing stop" order and set the distance from the message.</li>
+</ul>
+
+<h2>7 · Which timeframe should you follow?</h2>
+<p>The bot sends signals from several chart speeds. As a beginner, <b>follow only the 1-hour (H1) signals</b> and ignore the others. Here is why:</p>
+<table>
+<tr><th></th><th>M5 (5-minute)</th><th>M15 (15-minute)</th><th>H1 (1-hour) ★ start here</th></tr>
+<tr><td>Signals per week</td><td>about 10</td><td>about 10</td><td>about 3</td></tr>
+<tr><td>A trade usually lasts</td><td>a few hours</td><td>about half a day</td><td>half a day to a few days</td></tr>
+<tr><td>Time you have to react</td><td>1–2 minutes</td><td>up to 15 minutes</td><td>up to an hour</td></tr>
+<tr><td>Suits</td><td>full-time / automated</td><td>experienced, quick</td><td>anyone with a job and a life</td></tr>
+</table>
+<ul>
+<li><b>H1 is forgiving.</b> You have up to an hour to act on a message, and small delays barely matter. On the 5-minute signals, being two minutes late already costs real money — they are not for manual beginners.</li>
+<li><b>H1 trades are held longer</b> — often overnight, sometimes over days. That is normal. Your stop loss is working for you while you sleep.</li>
+<li><b>H1 is BUY-only.</b> All H1 SELL signals are blocked (since 2026-09-03) because they were a losing side on this timeframe — expect to see BUY signals here, never SELL.</li>
+<li><b>M5 also holds back some SELL signals</b> — specifically while the market is in a choppy "transitioning" phase, since those shorts tested poorly.</li>
+<li><b>M15 got a small tune-up on 2026-09-09</b> — one of its nine indicators (RSI) now counts for half as much, which means a few more M15 signals per week. Nothing changes in how you act on a signal.</li>
+<li><b>Never take the same signal on two timeframes.</b> They are usually the same gold move seen at different zoom levels. Taking both doubles your risk, not your profit.</li>
+</ul>
+
+<h2>8 · What to expect — read this before your first trade</h2>
+<ul>
+<li><b>You will lose about 4 trades out of every 10.</b> That is how the system is designed. Losses are small (1R each) and wins are bigger on average. The profit comes from many trades added together, not from any single one.</li>
+<li><b>Losing streaks happen.</b> Five or six losses in a row will happen at some point every year. It does not mean the system is broken. This is exactly why each loss is capped at 1% of your account.</li>
+<li><b>Expect a rough patch.</b> At some point in a normal year, your account will likely be around 10% below its best point before recovering. If a 10% dip would make you panic or bet bigger to "win it back," trade smaller (0.5% risk) or don't trade.</li>
+<li><b>Judge the system over months, not days.</b> On the 1-hour timeframe there are only ~3 trades a week. One bad week means nothing; look at results per quarter.</li>
+<li><b>Real results will be a little worse than tests.</b> The bot's numbers come from careful historical testing, but tests can't include your reaction speed or your broker's exact prices. Expect a bit less, not a bit more.</li>
+</ul>
+
+<h2>9 · The five rules</h2>
+<div class="callout red">
+<b>Break these and no signal system can save you.</b>
+<ol style="margin-bottom:0">
+<li><b>Risk 1% per trade, always.</b> Never increase the size to win back a loss.</li>
+<li><b>Every trade gets a stop loss in the app, at entry time.</b> No exceptions, no "mental" stops.</li>
+<li><b>One timeframe, one open trade at a time.</b></li>
+<li><b>Missed the entry? Let it go.</b> Chasing a price that already moved turns a good trade into a gamble.</li>
+<li><b>Only trade money you can afford to lose completely.</b> Never rent, savings you need, or borrowed money.</li>
+</ol>
+</div>
+<div class="callout green"><b>A good first month:</b> follow the H1 signals with the smallest position size your broker allows (or a demo account), write down every trade, and practice the trailing stop until it feels routine. Only then trade at 1% risk.</div>
+
+<div class="disclaimer"><b>⚠️ Risk warning.</b> This bot and guide are for education and research only. They are <b>not financial advice</b>. Trading gold and currencies involves a high risk of losing money, and most beginners lose. Past results — including tested results — do not guarantee future profits. Never trade money you cannot afford to lose. If you are unsure, speak to a licensed financial adviser.</div>
+
+</body>
+</html>

docs/user_guide.html

@@ -0,0 +1,143 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>ForexBot2 — User Guide</title>
+<style>
+  @page {
+    size: A4;
+    margin: 22mm 18mm;
+    @bottom-center { content: "ForexBot2 User Guide — page " counter(page) " of " counter(pages); font-size: 8pt; color: #888; }
+  }
+  * { box-sizing: border-box; }
+  body { font-family: "DejaVu Sans", Helvetica, Arial, sans-serif; font-size: 10pt; line-height: 1.55; color: #1c2733; margin: 0; }
+  h1 { font-size: 21pt; margin: 0 0 2mm; color: #0d1b2a; }
+  .subtitle { color: #5a6b7c; font-size: 10.5pt; margin-bottom: 8mm; }
+  h2 { font-size: 13.5pt; color: #0d1b2a; border-bottom: 2px solid #d4a017; padding-bottom: 1.2mm; margin: 8mm 0 3mm; page-break-after: avoid; }
+  h3 { font-size: 11pt; margin: 5mm 0 1.5mm; color: #23374d; page-break-after: avoid; }
+  p { margin: 0 0 2.5mm; }
+  table { border-collapse: collapse; width: 100%; margin: 3mm 0 4mm; font-size: 9pt; page-break-inside: avoid; }
+  th { background: #0d1b2a; color: #fff; text-align: left; padding: 1.8mm 2.5mm; font-weight: 600; }
+  td { border-bottom: 1px solid #dde4ea; padding: 1.6mm 2.5mm; }
+  tr:nth-child(even) td { background: #f4f7fa; }
+  .signal-box { background: #f4f7fa; border: 1px solid #c9d4de; border-left: 4px solid #d4a017; border-radius: 2mm; padding: 3.5mm 4.5mm; font-family: "DejaVu Sans Mono", monospace; font-size: 8.4pt; line-height: 1.5; white-space: pre-wrap; margin: 3mm 0 4mm; page-break-inside: avoid; }
+  .callout { background: #fdf6e3; border: 1px solid #e8d49a; border-left: 4px solid #d4a017; border-radius: 2mm; padding: 3mm 4.5mm; margin: 3mm 0 4mm; page-break-inside: avoid; }
+  .callout.red { background: #fdf0ee; border-color: #eab4ac; border-left-color: #c0392b; }
+  .callout b:first-child { display: block; margin-bottom: 1mm; }
+  ul, ol { margin: 0 0 3mm; padding-left: 6mm; }
+  li { margin-bottom: 1.4mm; }
+  code { font-family: "DejaVu Sans Mono", monospace; font-size: 8.6pt; background: #eef2f6; padding: 0.3mm 1.2mm; border-radius: 1mm; }
+  .small { font-size: 8.5pt; color: #5a6b7c; }
+  .disclaimer { border: 1.5px solid #c0392b; border-radius: 2mm; padding: 3.5mm 4.5mm; margin-top: 6mm; font-size: 9pt; page-break-inside: avoid; }
+  .disclaimer b { color: #c0392b; }
+</style>
+</head>
+<body>
+
+<h1>ForexBot2 — User Guide</h1>
+<div class="subtitle">How to read, act on, and survive the signals &nbsp;·&nbsp; July 2026</div>
+
+<h2>1 · What the bot does</h2>
+<p>ForexBot2 analyzes <b>gold (XAU/USD)</b> around the clock on several chart timeframes at once. On every candle close it scores nine technical indicators (trend, momentum, volatility, divergences, candlestick patterns), adapts the weighting to the current market regime (trending / ranging / transitioning), checks that higher timeframes agree, adjusts for news and economic-calendar events, and — only when everything lines up — sends a <b>signal with a complete trade plan</b> to Telegram: direction, entry, stop loss, target, and a trailing-stop management plan.</p>
+<p>Most of the time it sends nothing. Silence means no edge was found; that is the bot working correctly, not a malfunction.</p>
+
+<h2>2 · Anatomy of a signal</h2>
+<div class="signal-box">🔍 XAU/USD Signal Analysis  (M15)
+🕐 2026-07-10 14:00 UTC
+💰 Close: 4,132.50
+📊 Regime: Trending Up | Vol: normal
+━━━━━━━━━━━━━━━━━━━━
+🟢 BUY   🅰 A  |  Score: +38.2  |  64% conf
+✅ MTF aligned
+📈 MTF: M5 ✅  H1 ✅
+📋 Indicators … (9 rows)
+⚠️ EXTREME volatility — wider stops needed
+━━━━━━━━━━━━━━━━━━━━
+🚀 TRADE SETUP
+📈 Direction: BUY
+▶ Entry:  4,132.50
+🛑 SL:     4,103.10   ($29.40 / 294 pips)
+🎯 TP1:    4,161.90   ($29.40 / 294 pips)
+⚖️ RR:     1:1.0
+🔄 Trail: once price hits 4,150.14 (+0.6R),
+   trail SL $22.05 behind best price —
+   TP1 is a guide, let it run</div>
+
+<table>
+<tr><th style="width:28%">Field</th><th>What it means</th></tr>
+<tr><td><b>Signal + grade</b></td><td>Direction (BUY / SELL / STRONG_*) plus a quality grade <b>A+ → F</b>. A trade setup is only attached at grade C or better. Higher grades = more indicator agreement and cleaner regime.</td></tr>
+<tr><td><b>Score / confidence</b></td><td>The regime-weighted composite of all nine indicators. Bigger absolute score = stronger consensus.</td></tr>
+<tr><td><b>MTF aligned</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr>
+<tr><td><b>Warnings ⚠️</b></td><td>Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings.</td></tr>
+<tr><td><b>Violent-move watch 🌪</b></td><td>An occasional extra line saying the next hour looks unusually likely to produce a very large candle — because a high-impact US release is due (or just landed), because the clock is in a historically wild slot, or because price is stretched outside its bands. It is a heads-up about <i>volatility</i>, not a direction call, and it <b>never</b> changes the signal, the grade or the trade setup — the same alert would be sent without it. Do not close or tighten an open trade because of it: every such protective action was tested over three years and lost money. Your stop is the protection.</td></tr>
+<tr><td><b>Market context 📰</b></td><td>An AI-written note in plain English on what is moving the gold price right now, why it matters, and the next report to watch. It is built only from headlines published in the last 4 hours that lean one way (neutral items are dropped) plus today's high-impact releases. Informational only — it never blocks or grades a signal.</td></tr>
+<tr><td><b>SL / TP1 / RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer. TP1 is the nearest opposing level. <b>1R</b> = the entry-to-SL distance = the amount you risk.</td></tr>
+<tr><td><b>Trail plan 🔄</b></td><td>The real exit. Once price moves +0.6R in your favor, move the stop to trail 0.75R behind the best price and update it at each candle close. <b>TP1 is advisory — let winners run.</b></td></tr>
+</table>
+
+<h2>3 · Acting on a signal — step by step</h2>
+<ol>
+<li><b>Size the position first.</b> Decide your risk per trade — <b>0.5–1% of account, never more</b>. Position size = (account × risk%) ÷ stop distance in $. The stop distance varies per signal, so the size changes every trade; the dollar risk must not.</li>
+<li><b>Enter promptly at market.</b> All backtested results assume entry at the open of the candle after the signal. On M5, minutes matter (see §4). If you are very late — price already moved a large part of the way to the arm level — skip the trade. There is always another signal.</li>
+<li><b>Place the stop loss immediately</b> at the given SL. No mental stops — gold gaps and spikes.</li>
+<li><b>Manage with the trail, not TP1.</b> When price reaches the arm level printed in the message, start trailing the stop the stated $ distance behind the best price, tightening only at candle close. This is what produced the backtested results; taking fixed TP1 profits gives most of the edge away.</li>
+<li><b>One signal, one trade.</b> The M5, M15 and H1 books confirm off each other and are usually long or short gold at the same time. Taking the same move on multiple timeframes multiplies risk, not diversification. Pick your timeframe (§4) and ignore the others.</li>
+</ol>
+
+<h2>4 · Choosing your timeframe</h2>
+<p>Twelve-month Oanda-candle backtest (Aug 2025 – Aug 2026, $0.65 round-trip cost, entry at next candle open):</p>
+<table>
+<tr><th></th><th>M5</th><th>M15</th><th>H1</th></tr>
+<tr><td>Trades / week</td><td>~11.3</td><td>~7.6</td><td>~2.7</td></tr>
+<tr><td>Total result (12 mo)</td><td>+106.8R</td><td>+73.9R</td><td>+24.0R</td></tr>
+<tr><td>Win rate / profit factor</td><td>52.9% / 1.51</td><td>62.2% / 1.53</td><td>60.9% / 1.55</td></tr>
+<tr><td>Typical (median) hold</td><td>2.4 h</td><td>7 h</td><td>15.5 h</td></tr>
+<tr><td>90% of trades close within</td><td>10.5 h</td><td>29.8 h</td><td>3.2 days</td></tr>
+<tr><td>Typical stop distance</td><td>~$21</td><td>~$32</td><td>~$52</td></tr>
+<tr><td>Worst drawdown</td><td>10.7R</td><td>10.8R</td><td>8.0R</td></tr>
+</table>
+<ul>
+<li><b>M5 — intraday, execution-critical.</b> Highest total edge, but stops are tight: a couple of dollars of slippage or a late entry eats a meaningful share of every trade. Suitable only if you can act within a minute or two of the alert (or automate). Trades open and close within a session; you sleep flat. SELL signals are held back while the market regime reads "transitioning" — a validated filter, not a bug if you notice fewer shorts there.</li>
+<li><b>M15 — the middle path.</b> Same frequency as M5, wider stops, up to 15 minutes to act. About half your trades will still be open overnight. Since 2026-09-09 the M15 scoring gives its RSI reading half the weight it used to (RSI mostly voted against the trend on this timeframe), so expect roughly 9 signals a week instead of 7–8; in the two-year test (Sep 2024 – Sep 2026) that change lifted M15 from +117.6R to +146.1R with the same win rate and a slightly better profit factor, and it also improved the separate out-of-sample period. The M15 column above predates it.</li>
+<li><b>H1 — swing trading around a normal life.</b> Best quality per trade and very forgiving of slow manual execution (you have up to an hour). The costs: positions held for days, through news and overnight sessions, and a <b>lumpy</b> equity curve — one quarter of the test year was negative and most of the profit came in a single quarter. Judge it on quarters, not weeks. H1 no longer issues SELL signals at all (blocked since 2026-09-03) — it is BUY-only on this timeframe.</li>
+</ul>
+<div class="callout"><b>If you trade manually from the Telegram alerts, H1 (or M15) is the honest choice.</b> M5's backtested edge assumes near-instant fills that a human copying signals rarely achieves.</div>
+
+<h2>5 · What to expect (and what not to)</h2>
+<ul>
+<li><b>You will lose roughly 38–47% of trades.</b> A 53–62% win rate means losing streaks of 5+ are routine over a year. The system's edge is the average across hundreds of trades, not any single one.</li>
+<li><b>Stops hit fast, winners take time.</b> Losing trades die in roughly half the time winners take. An early stop-out is the system doing its job.</li>
+<li><b>Drawdowns of ~10R are normal.</b> At 1% risk per trade, expect to be ~10% down from a peak at some point in a normal year. This is why risking more than 1% per trade is how accounts die.</li>
+<li><b>Backtest ≠ live.</b> Results above charge real spreads and realistic fills, but cannot model your personal latency, broker slippage, or requotes. Assume live results will be somewhat worse — especially on M5.</li>
+<li><b>Quiet periods are normal.</b> Filters block bad hours, bad regimes and misaligned timeframes. Days can pass without a signal.</li>
+</ul>
+
+<h2>6 · Rules that keep you alive</h2>
+<div class="callout red">
+<b>The five rules</b>
+<ol style="margin-bottom:0">
+<li>Risk a fixed 0.5–1% per trade. Never "make it back" by sizing up after losses.</li>
+<li>Hard stop loss in the market on every trade, placed at entry time.</li>
+<li>One timeframe, one trade at a time. No stacking, no pyramiding onto bot signals.</li>
+<li>Missed the entry? Skip it. Chasing converts a positive-edge trade into a coin flip.</li>
+<li>Evaluate performance monthly at the earliest, quarterly for H1. Never after a bad day.</li>
+</ol>
+</div>
+
+<h2>7 · Bot commands (owner)</h2>
+<table>
+<tr><th style="width:30%">Command</th><th>What it does</th></tr>
+<tr><td><code>/start</code></td><td>Show the button menu (manual scans per timeframe, Scan All TFs).</td></tr>
+<tr><td><code>/scan</code></td><td>Run an immediate analysis of the primary timeframe.</td></tr>
+<tr><td><code>/autoscan</code></td><td>Toggle automatic scanning (M5 / M15 / H1 / H4 at each candle close).</td></tr>
+<tr><td><code>/status</code></td><td>Uptime, auto-scan state, WebSocket price-monitor state.</td></tr>
+<tr><td><code>/news</code></td><td>Net XAU/USD news score from the last 4 hours of headlines (neutral headlines are removed before averaging; informational only — never gates signals).</td></tr>
+<tr><td><code>/cache</code> · <code>/clearcache</code></td><td>Inspect / reset the market-data cache.</td></tr>
+<tr><td><code>/help</code></td><td>Command reference.</td></tr>
+</table>
+<p class="small">The bot also streams live prices over WebSocket and fires an immediate scan when price breaches a key level (swing S/R, Bollinger, Keltner, EMA-200). Commands respond only to the configured owner chat; subscribers receive the signal feed via the channel.</p>
+
+<div class="disclaimer"><b>⚠️ Risk disclaimer.</b> This bot and guide are for educational and research purposes only and are <b>not financial advice</b>. Trading forex, gold and CFDs on margin carries a high risk of loss; past (and backtested) performance does not guarantee future results. Do your own analysis and never risk money you cannot afford to lose.</div>
+
+</body>
+</html>

llm_client.py

@@ -0,0 +1,200 @@
+"""
+ForexBot2 — DeepSeek LLM client
+================================
+Thin requests-based client for DeepSeek's OpenAI-compatible chat API,
+used by `news_engine` for display/warning-only enrichment (headline
+sentiment, event surprise direction, market summary). Signals are NEVER
+gated on LLM output — every caller has a non-LLM fallback and treats a
+`None` return as "use the fallback".
+
+Design constraints:
+- One attempt per call, no retries: news runs inside scan cycles; the
+  news engine's 600 s context cache is the natural retry cadence.
+- A UTC-day request cap bounds spend even if a bug causes call storms.
+- Cost: DeepSeek caches identical prompt PREFIXES server-side (cache-hit
+  input is ~50x cheaper), so callers pass a STATIC system prompt (no
+  timestamps or interpolation) and put all variable content in the user
+  payload.
+
+This is a module-level singleton (`get_llm_client`) because the daily
+cap counter must survive the per-scan reconstruction of `NewsEngine`.
+"""
+
+import datetime as _dt
+import json
+import logging
+import threading
+import time
+from typing import Any, Dict, Optional, Tuple
+
+import requests
+
+from app_config import deepseek_api_key, get_app_config
+
+logger = logging.getLogger(__name__)
+
+DEEPSEEK_URL = "https://api.deepseek.com/chat/completions"
+DEFAULT_MODEL = "deepseek-v4-flash"
+# 8s timed out on a measured cold-prefix-cache headline batch (2026-08-03);
+# warm-cache calls come back well under that. 12s buys cold-start headroom
+# while still bounding worst-case scan latency (≤3 calls per cache window).
+DEFAULT_TIMEOUT = 12.0
+DEFAULT_DAILY_CAP = 500
+
+
+class DeepSeekClient:
+    """JSON-in/JSON-out chat wrapper. All failures return None."""
+
+    def __init__(
+        self,
+        api_key: str,
+        model: str = DEFAULT_MODEL,
+        timeout: float = DEFAULT_TIMEOUT,
+        daily_cap: int = DEFAULT_DAILY_CAP,
+        enabled: bool = True,
+    ):
+        self._api_key = api_key or ""
+        self._model = model
+        self._timeout = timeout
+        self._daily_cap = daily_cap
+        self._enabled = enabled
+        self._session = requests.Session()
+        self._session.headers.update({
+            "Content-Type": "application/json",
+            "Authorization": f"Bearer {self._api_key}",
+        })
+        self._lock = threading.Lock()
+        self._count_day: Optional[_dt.date] = None
+        self._count = 0
+        self._warned: set = set()
+        # Health stamps for status displays: last successful call and
+        # last failure (ts, kind). Read-only for consumers.
+        self.last_ok: float = 0.0
+        self.last_error: Optional[Tuple[float, str]] = None
+
+    @property
+    def available(self) -> bool:
+        return self._enabled and bool(self._api_key)
+
+    def _warn_once(self, kind: str, msg: str, *args: Any) -> None:
+        if kind in self._warned:
+            logger.debug(msg, *args)
+        else:
+            self._warned.add(kind)
+            logger.warning(msg, *args)
+
+    def _under_cap(self) -> bool:
+        """Count a request against the UTC-day cap; False = don't call."""
+        today = _dt.datetime.now(_dt.timezone.utc).date()
+        with self._lock:
+            if self._count_day != today:
+                self._count_day = today
+                self._count = 0
+                # Clear ALL warn-once kinds, not just "cap": the network/
+                # http_*/parse flags used to be set once and never reset,
+                # so in a daemon that runs for weeks the second and every
+                # later outage logged at DEBUG only.
+                self._warned.clear()
+            if self._count >= self._daily_cap:
+                self._warn_once(
+                    "cap", "DeepSeek daily request cap (%d) exhausted — "
+                    "falling back to keyword analysis until UTC midnight",
+                    self._daily_cap)
+                return False
+            self._count += 1
+            return True
+
+    def chat_json(self, system_prompt: str,
+                  user_payload: Dict[str, Any]) -> Optional[Dict[str, Any]]:
+        """One chat call; returns the model's JSON object or None."""
+        if not self.available or not self._under_cap():
+            return None
+        body = {
+            "model": self._model,
+            "messages": [
+                {"role": "system", "content": system_prompt},
+                {"role": "user",
+                 "content": json.dumps(user_payload, sort_keys=True)},
+            ],
+            "response_format": {"type": "json_object"},
+            "temperature": 0,
+            # V4-Flash is a hybrid thinking model; without this flag it
+            # burns hundreds of reasoning tokens per call — measured
+            # 2026-08-03: a 25-headline batch blew straight through the
+            # timeout AND could exhaust max_tokens before the answer,
+            # returning empty content. The doctrine + few-shot examples
+            # in the system prompts carry the quality instead.
+            "thinking": {"type": "disabled"},
+            # Must fit the largest reply: a full sentiment batch is
+            # ~10 tokens per headline; 1000 truncated mid-JSON on a real
+            # RSS cycle (2026-08-03).
+            "max_tokens": 2000,
+        }
+        try:
+            resp = self._session.post(
+                DEEPSEEK_URL, json=body, timeout=self._timeout)
+        except requests.RequestException as e:
+            self.last_error = (time.time(), "network")
+            self._warn_once("network", "DeepSeek call failed: %s", e)
+            return None
+        if resp.status_code != 200:
+            self.last_error = (time.time(), f"http_{resp.status_code}")
+            self._warn_once(
+                f"http_{resp.status_code}",
+                "DeepSeek HTTP %s: %s", resp.status_code, resp.text[:200])
+            return None
+        try:
+            parsed = self._parse_response(resp.json())
+        except ValueError:
+            parsed = None
+        if parsed is None:
+            self.last_error = (time.time(), "parse")
+            self._warn_once("parse", "DeepSeek response was not the "
+                            "expected JSON object — ignoring")
+        else:
+            self.last_ok = time.time()
+            # A recovered call re-arms the failure warnings so the NEXT
+            # outage is visible at WARNING again. "cap" keeps its own
+            # UTC-day reset in _under_cap. (Taken under the lock; safe
+            # here because chat_json holds no lock — _under_cap has
+            # already returned.)
+            if self._warned - {"cap"}:
+                with self._lock:
+                    self._warned = {k for k in self._warned if k == "cap"}
+        return parsed
+
+    @staticmethod
+    def _parse_response(body: Any) -> Optional[Dict[str, Any]]:
+        """Extract the assistant's JSON object from a chat completion."""
+        try:
+            content = body["choices"][0]["message"]["content"]
+        except (KeyError, IndexError, TypeError):
+            return None
+        try:
+            obj = json.loads(content)
+        except (TypeError, ValueError):
+            return None
+        return obj if isinstance(obj, dict) else None
+
+
+# ── Process-wide singleton ────────────────────────────────────────────
+
+_client: Optional[DeepSeekClient] = None
+_client_lock = threading.Lock()
+
+
+def get_llm_client() -> DeepSeekClient:
+    global _client
+    with _client_lock:
+        if _client is None:
+            settings = get_app_config().deepseek_settings
+            _client = DeepSeekClient(
+                api_key=deepseek_api_key(),
+                model=str(settings.get("model", DEFAULT_MODEL)),
+                timeout=float(settings.get("timeout_seconds",
+                                           DEFAULT_TIMEOUT)),
+                daily_cap=int(settings.get("daily_request_cap",
+                                           DEFAULT_DAILY_CAP)),
+                enabled=settings.get("enabled", True) is not False,
+            )
+        return _client

main.py

@@ -1,509 +1,379 @@
-#!/usr/bin/env python3
-"""
-Forex Signal Intelligence Platform
-====================================
-Main entry point. Runs full analysis and outputs formatted signal reports.
-
-Usage:
-    python main.py                          # Analyze all major pairs
-    python main.py --pairs EURUSD GBPUSD    # Specific pairs
-    python main.py --timeframe H4           # Different primary TF
-    python main.py --json                   # JSON output
-    python main.py --watch                  # Continuous monitoring mode
-"""
-
-import argparse
-import json
-import logging
-import sys
-import time
-from datetime import datetime, timezone
-from typing import List
-
-# Force UTF-8 output on Windows so box-drawing / Unicode chars render correctly
-if hasattr(sys.stdout, "reconfigure"):
-    sys.stdout.reconfigure(encoding="utf-8")
-if hasattr(sys.stderr, "reconfigure"):
-    sys.stderr.reconfigure(encoding="utf-8")
-
-from config import (
-    PlatformConfig, Timeframe, SignalStrength,
-    MAJOR_PAIRS, CROSS_PAIRS,
-)
-from signal_aggregator import SignalAggregator, SignalReport
-
-
-# ══════════════════════════════════════════════════════════════════
-#  TERMINAL FORMATTING
-# ══════════════════════════════════════════════════════════════════
-
-class Colors:
-    """ANSI color codes for terminal output."""
-    RESET = "\033[0m"
-    BOLD = "\033[1m"
-    DIM = "\033[2m"
-    UNDERLINE = "\033[4m"
-
-    RED = "\033[91m"
-    GREEN = "\033[92m"
-    YELLOW = "\033[93m"
-    BLUE = "\033[94m"
-    MAGENTA = "\033[95m"
-    CYAN = "\033[96m"
-    WHITE = "\033[97m"
-    GRAY = "\033[90m"
-
-    BG_RED = "\033[41m"
-    BG_GREEN = "\033[42m"
-    BG_YELLOW = "\033[43m"
-    BG_BLUE = "\033[44m"
-    BG_MAGENTA = "\033[45m"
-    BG_CYAN = "\033[46m"
-
-
-def signal_color(signal: SignalStrength) -> str:
-    """Get color for a signal strength."""
-    return {
-        SignalStrength.STRONG_BUY: Colors.GREEN + Colors.BOLD,
-        SignalStrength.BUY: Colors.GREEN,
-        SignalStrength.NEUTRAL: Colors.GRAY,
-        SignalStrength.SELL: Colors.RED,
-        SignalStrength.STRONG_SELL: Colors.RED + Colors.BOLD,
-    }.get(signal, Colors.RESET)
-
-
-def quality_color(quality: str) -> str:
-    """Get color for signal quality grade."""
-    return {
-        "A+": Colors.GREEN + Colors.BOLD,
-        "A": Colors.GREEN,
-        "B": Colors.CYAN,
-        "C": Colors.YELLOW,
-        "D": Colors.GRAY,
-        "F": Colors.DIM,
-    }.get(quality, Colors.RESET)
-
-
-def score_bar(score: float, width: int = 30) -> str:
-    """Create a visual score bar from -100 to +100."""
-    normalized = (score + 100) / 200  # 0 to 1
-    filled = int(normalized * width)
-    center = width // 2
-
-    bar = ""
-    for i in range(width):
-        if i == center:
-            bar += "│"
-        elif (score >= 0 and center < i <= filled) or \
-             (score < 0 and filled <= i < center):
-            bar += "█"
-        else:
-            bar += "░"
-
-    if score >= 0:
-        color = Colors.GREEN if score > 20 else Colors.GRAY
-    else:
-        color = Colors.RED if score < -20 else Colors.GRAY
-
-    return f"{color}{bar}{Colors.RESET}"
-
-
-def print_header():
-    """Print the platform header."""
-    now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
-    width = 80
-
-    print(f"\n{Colors.CYAN}{'═' * width}{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ╔═══════════════════════════════════════════════════════╗{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ║   FOREX SIGNAL INTELLIGENCE PLATFORM                 ║{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ║   Multi-Layer Signal Analysis Engine                  ║{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ╚═══════════════════════════════════════════════════════╝{Colors.RESET}")
-    print(f"{Colors.GRAY}  Scan Time: {now}{Colors.RESET}")
-    print(f"{Colors.CYAN}{'═' * width}{Colors.RESET}\n")
-
-
-def print_summary_table(reports: List[SignalReport]):
-    """Print a summary table of all pairs."""
-    print(f"{Colors.BOLD}{Colors.WHITE}  ┌─────────────┬───────────┬──────────────────────────────────┬────────┬────────────┐{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.WHITE}  │ Pair        │ Signal    │ Score                            │ Grade  │ Confidence │{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.WHITE}  ├─────────────┼───────────┼──────────────────────────────────┼────────┼────────────┤{Colors.RESET}")
-
-    for r in reports:
-        sig = r.final_signal.name.replace("_", " ")
-        sc = signal_color(r.final_signal)
-        qc = quality_color(r.signal_quality)
-        bar = score_bar(r.final_score, 30)
-
-        pair = f"{r.display_name:<11}"
-        signal_str = f"{sc}{sig:<9}{Colors.RESET}"
-        grade_str = f"{qc}{r.signal_quality:<6}{Colors.RESET}"
-        conf_str = f"{r.final_confidence:5.1f}%"
-
-        # Add warning indicator
-        warn = ""
-        if r.event_risk == "danger":
-            warn = f" {Colors.RED}⚠{Colors.RESET}"
-        elif r.event_risk == "caution":
-            warn = f" {Colors.YELLOW}△{Colors.RESET}"
-
-        print(f"  │ {pair} │ {signal_str} │ {bar} │ {grade_str} │ {conf_str:>8}{warn}  │")
-
-    print(f"{Colors.BOLD}{Colors.WHITE}  └─────────────┴───────────┴──────────────────────────────────┴────────┴────────────┘{Colors.RESET}")
-
-    # Legend
-    print(f"\n{Colors.GRAY}  Legend: "
-          f"{Colors.GREEN}██ Bullish{Colors.GRAY} | "
-          f"{Colors.RED}██ Bearish{Colors.GRAY} | "
-          f"│ Center (0) | "
-          f"{Colors.RED}⚠ Event Risk{Colors.GRAY} | "
-          f"{Colors.YELLOW}△ Caution{Colors.RESET}")
-
-
-def print_detailed_report(report: SignalReport):
-    """Print detailed analysis for a single pair."""
-    sc = signal_color(report.final_signal)
-    qc = quality_color(report.signal_quality)
-    sig_name = report.final_signal.name.replace("_", " ")
-
-    print(f"\n{Colors.CYAN}{'─' * 80}{Colors.RESET}")
-    print(f"  {Colors.BOLD}{Colors.WHITE}{report.display_name}{Colors.RESET}"
-          f"  │  Price: {Colors.BOLD}{report.current_price:.5f}{Colors.RESET}"
-          f"  │  Regime: {Colors.CYAN}{report.market_regime}{Colors.RESET}"
-          f"  │  Vol: {report.volatility_regime}")
-
-    print(f"\n  {Colors.BOLD}SIGNAL:{Colors.RESET} {sc}{sig_name}{Colors.RESET}"
-          f"  │  Score: {sc}{report.final_score:+.1f}{Colors.RESET}"
-          f"  │  Grade: {qc}{report.signal_quality}{Colors.RESET}"
-          f"  │  Confidence: {report.final_confidence:.0f}%")
-
-    print(f"  {Colors.BOLD}Action:{Colors.RESET} {report.action_recommendation}")
-
-    # Multi-timeframe breakdown
-    if report.mtf_signals:
-        print(f"\n  {Colors.BOLD}Multi-Timeframe:{Colors.RESET}", end="")
-        mtf_agree = f" {Colors.GREEN}✓ ALIGNED{Colors.RESET}" if report.mtf_agreement else f" {Colors.YELLOW}✗ DIVERGENT{Colors.RESET}"
-        print(mtf_agree)
-        for tf, sig in report.mtf_signals.items():
-            tf_sc = signal_color(sig)
-            tf_score = report.mtf_scores.get(tf, 0)
-            print(f"    {tf:>5}: {tf_sc}{sig.name.replace('_', ' '):<12}{Colors.RESET} ({tf_score:+.1f})")
-
-    # Indicators detail
-    if report.primary_analysis:
-        print(f"\n  {Colors.BOLD}Indicators ({report.primary_analysis.timeframe}):{Colors.RESET}")
-        for name, ind in report.primary_analysis.indicators.items():
-            if name.startswith("_") or ind.weight == 0:
-                continue
-            isc = signal_color(ind.signal)
-            ind_name = f"{ind.name:<18}"
-            print(f"    {ind_name} {isc}{ind.signal.name.replace('_', ' '):<12}{Colors.RESET}"
-                  f" ({ind.confidence:.0f}%) │ {Colors.GRAY}{ind.description}{Colors.RESET}")
-
-    # Bullish reasons
-    if report.bullish_reasons:
-        print(f"\n  {Colors.GREEN}▲ Bullish Factors:{Colors.RESET}")
-        for reason in report.bullish_reasons[:5]:
-            print(f"    {Colors.GREEN}+{Colors.RESET} {reason}")
-
-    # Bearish reasons
-    if report.bearish_reasons:
-        print(f"\n  {Colors.RED}▼ Bearish Factors:{Colors.RESET}")
-        for reason in report.bearish_reasons[:5]:
-            print(f"    {Colors.RED}-{Colors.RESET} {reason}")
-
-    # Warnings
-    if report.warnings:
-        print(f"\n  {Colors.YELLOW}⚠ Warnings:{Colors.RESET}")
-        for warning in report.warnings:
-            print(f"    {Colors.YELLOW}!{Colors.RESET} {warning}")
-
-    # News context
-    if report.news_description and report.news_description != "Neutral news sentiment":
-        print(f"\n  {Colors.MAGENTA}📰 News:{Colors.RESET} {report.news_description}")
-    if report.event_description and report.event_risk != "clear":
-        print(f"  {Colors.YELLOW}📅 Event:{Colors.RESET} {report.event_description}")
-
-    # Trade setup
-    if report.trade_setup:
-        ts = report.trade_setup
-        dir_color = Colors.GREEN if ts.direction == "BUY" else Colors.RED
-        print(f"\n  {Colors.BOLD}Trade Setup:{Colors.RESET}")
-        print(f"    Direction:     {dir_color}{ts.direction}{Colors.RESET}")
-        print(f"    Entry:         {ts.entry_price:.5f}")
-        print(f"    Stop Loss:     {Colors.RED}{ts.stop_loss:.5f}{Colors.RESET} ({ts.risk_pips:.1f} pips)")
-        print(f"    Take Profit 1: {Colors.GREEN}{ts.take_profit_1:.5f}{Colors.RESET} ({ts.reward_pips:.1f} pips)")
-        print(f"    Take Profit 2: {Colors.GREEN}{ts.take_profit_2:.5f}{Colors.RESET}")
-        print(f"    Risk:Reward:   1:{ts.risk_reward:.1f}")
-
-    # Key levels
-    if report.primary_analysis and report.primary_analysis.key_levels:
-        levels = report.primary_analysis.key_levels
-        print(f"\n  {Colors.BOLD}Key Levels:{Colors.RESET}")
-        level_names = ["r3", "r2", "r1", "pivot", "s1", "s2", "s3"]
-        for lvl in level_names:
-            if lvl in levels:
-                val = levels[lvl]
-                if val > report.current_price:
-                    color = Colors.RED  # resistance
-                else:
-                    color = Colors.GREEN  # support
-                label = lvl.upper()
-                print(f"    {label:>6}: {color}{val:.5f}{Colors.RESET}")
-
-
-def print_event_calendar(reports: List[SignalReport]):
-    """Print upcoming economic events affecting watched pairs."""
-    events_shown = set()
-    has_events = False
-
-    for report in reports:
-        if report.event_risk != "clear" and report.event_description not in events_shown:
-            if not has_events:
-                print(f"\n{Colors.YELLOW}{Colors.BOLD}  📅 ECONOMIC CALENDAR ALERTS{Colors.RESET}")
-                has_events = True
-            events_shown.add(report.event_description)
-            risk_color = Colors.RED if report.event_risk == "danger" else Colors.YELLOW
-            print(f"    {risk_color}● {report.event_description}{Colors.RESET} → affects {report.display_name}")
-
-    if not has_events:
-        print(f"\n{Colors.GRAY}  📅 No high-impact events in the near term{Colors.RESET}")
-
-
-def report_to_dict(report: SignalReport) -> dict:
-    """Convert a SignalReport to a JSON-serializable dict."""
-    d = {
-        "symbol": report.symbol,
-        "display_name": report.display_name,
-        "timestamp": report.timestamp,
-        "current_price": report.current_price,
-        "signal": report.final_signal.name,
-        "score": round(report.final_score, 2),
-        "confidence": round(report.final_confidence, 2),
-        "quality": report.signal_quality,
-        "action": report.action_recommendation,
-        "market_regime": report.market_regime,
-        "volatility_regime": report.volatility_regime,
-        "mtf_agreement": report.mtf_agreement,
-        "mtf_signals": {k: v.name for k, v in report.mtf_signals.items()},
-        "event_risk": report.event_risk,
-        "event_description": report.event_description,
-        "news_sentiment": round(report.news_sentiment, 3),
-        "bullish_reasons": report.bullish_reasons,
-        "bearish_reasons": report.bearish_reasons,
-        "warnings": report.warnings,
-    }
-    if report.trade_setup:
-        ts = report.trade_setup
-        d["trade_setup"] = {
-            "direction": ts.direction,
-            "entry": ts.entry_price,
-            "stop_loss": ts.stop_loss,
-            "take_profit_1": ts.take_profit_1,
-            "take_profit_2": ts.take_profit_2,
-            "risk_reward": ts.risk_reward,
-            "risk_pips": round(ts.risk_pips, 1),
-            "reward_pips": round(ts.reward_pips, 1),
-        }
-    if report.primary_analysis:
-        d["indicators"] = {}
-        for name, ind in report.primary_analysis.indicators.items():
-            if not name.startswith("_"):
-                d["indicators"][name] = {
-                    "signal": ind.signal.name,
-                    "confidence": round(ind.confidence, 1),
-                    "description": ind.description,
-                }
-        d["key_levels"] = {
-            k: round(v, 5) for k, v in report.primary_analysis.key_levels.items()
-        }
-    return d
-
-
-# ══════════════════════════════════════════════════════════════════
-#  MAIN
-# ══════════════════════════════════════════════════════════════════
-
-def parse_args():
-    parser = argparse.ArgumentParser(
-        description="Forex Signal Intelligence Platform",
-        formatter_class=argparse.RawDescriptionHelpFormatter,
-        epilog="""
-Examples:
-  python main.py                        Analyze all major pairs (H1)
-  python main.py --pairs EURUSD GBPUSD  Analyze specific pairs
-  python main.py --timeframe H4         Use 4-hour as primary TF
-  python main.py --all                  Include cross pairs
-  python main.py --json                 Output as JSON
-  python main.py --detail               Show detailed analysis per pair
-  python main.py --watch                Continuous monitoring (5min refresh)
-        """,
-    )
-    parser.add_argument(
-        "--pairs", nargs="+", default=None,
-        help="Specific pairs to analyze (e.g., EURUSD GBPUSD)",
-    )
-    parser.add_argument(
-        "--timeframe", "-tf", default="H1",
-        choices=["M5", "M15", "M30", "H1", "H4", "D1"],
-        help="Primary analysis timeframe (default: H1)",
-    )
-    parser.add_argument(
-        "--all", action="store_true",
-        help="Include cross pairs in analysis",
-    )
-    parser.add_argument(
-        "--json", action="store_true",
-        help="Output results as JSON",
-    )
-    parser.add_argument(
-        "--detail", action="store_true",
-        help="Show detailed analysis for each pair",
-    )
-    parser.add_argument(
-        "--top", type=int, default=None,
-        help="Show only top N signals by strength",
-    )
-    parser.add_argument(
-        "--watch", action="store_true",
-        help="Continuous monitoring mode (refreshes every 5 min)",
-    )
-    parser.add_argument(
-        "--min-grade", default="F",
-        choices=["A+", "A", "B", "C", "D", "F"],
-        help="Minimum signal quality to display",
-    )
-    parser.add_argument(
-        "--verbose", "-v", action="store_true",
-        help="Enable debug logging",
-    )
-    return parser.parse_args()
-
-
-def symbol_from_name(name: str) -> str:
-    """Convert pair name like 'EURUSD' to Yahoo symbol 'EURUSD=X'."""
-    name = name.upper().replace("/", "").replace("_", "")
-    if not name.endswith("=X"):
-        name += "=X"
-    return name
-
-
-def run_analysis(args) -> List[SignalReport]:
-    """Run a single analysis cycle."""
-    config = PlatformConfig()
-
-    # Configure timeframe
-    tf_map = {
-        "M5": Timeframe.M5, "M15": Timeframe.M15, "M30": Timeframe.M30,
-        "H1": Timeframe.H1, "H4": Timeframe.H4, "D1": Timeframe.D1,
-    }
-    config.primary_timeframe = tf_map[args.timeframe]
-
-    # Configure confirmation timeframes based on primary
-    tf_confirmations = {
-        "M5": [Timeframe.M1, Timeframe.M15],
-        "M15": [Timeframe.M5, Timeframe.H1],
-        "M30": [Timeframe.M15, Timeframe.H1],
-        "H1": [Timeframe.M15, Timeframe.H4],
-        "H4": [Timeframe.H1, Timeframe.D1],
-        "D1": [Timeframe.H4, Timeframe.W1],
-    }
-    config.confirmation_timeframes = tf_confirmations.get(
-        args.timeframe, [Timeframe.M15, Timeframe.H4]
-    )
-
-    # Configure pairs
-    if args.pairs:
-        config.watch_pairs = [symbol_from_name(p) for p in args.pairs]
-    elif args.all:
-        config.watch_pairs = MAJOR_PAIRS + CROSS_PAIRS
-    else:
-        config.watch_pairs = MAJOR_PAIRS
-
-    # Run analysis
-    aggregator = SignalAggregator(config)
-    reports = aggregator.analyze_all_pairs()
-
-    # Filter by grade
-    grade_order = ["A+", "A", "B", "C", "D", "F"]
-    min_idx = grade_order.index(args.min_grade) if args.min_grade in grade_order else len(grade_order) - 1
-    reports = [r for r in reports if grade_order.index(r.signal_quality) <= min_idx]
-
-    # Top N
-    if args.top:
-        reports = reports[:args.top]
-
-    return reports
-
-
-def main():
-    args = parse_args()
-
-    # Setup logging
-    log_level = logging.DEBUG if args.verbose else logging.WARNING
-    logging.basicConfig(
-        level=log_level,
-        format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
-        datefmt="%H:%M:%S",
-    )
-
-    if args.watch:
-        # Continuous monitoring mode
-        print(f"{Colors.BOLD}Starting continuous monitoring mode (Ctrl+C to stop)...{Colors.RESET}")
-        while True:
-            try:
-                reports = run_analysis(args)
-                if args.json:
-                    output = [report_to_dict(r) for r in reports]
-                    print(json.dumps(output, indent=2))
-                else:
-                    print("\033[2J\033[H")  # clear screen
-                    print_header()
-                    print_summary_table(reports)
-                    print_event_calendar(reports)
-                    if args.detail:
-                        for r in reports:
-                            print_detailed_report(r)
-                    print(f"\n{Colors.GRAY}  Next refresh in 5 minutes...{Colors.RESET}")
-
-                time.sleep(300)
-            except KeyboardInterrupt:
-                print(f"\n{Colors.YELLOW}Monitoring stopped.{Colors.RESET}")
-                break
-    else:
-        # Single run
-        print(f"{Colors.GRAY}  Fetching market data and running analysis...{Colors.RESET}\n")
-        reports = run_analysis(args)
-
-        if args.json:
-            output = [report_to_dict(r) for r in reports]
-            print(json.dumps(output, indent=2))
-        else:
-            print_header()
-            if reports:
-                print_summary_table(reports)
-                print_event_calendar(reports)
-
-                if args.detail:
-                    for r in reports:
-                        print_detailed_report(r)
-                else:
-                    # Show detail for top signals only
-                    actionable = [r for r in reports if r.signal_quality in ("A+", "A", "B")]
-                    if actionable:
-                        print(f"\n{Colors.BOLD}{Colors.WHITE}  ═══ ACTIONABLE SIGNALS (Grade B or above) ═══{Colors.RESET}")
-                        for r in actionable:
-                            print_detailed_report(r)
-                    else:
-                        print(f"\n{Colors.GRAY}  No high-grade signals detected. "
-                              f"Use --detail for full breakdown or --min-grade C to see more.{Colors.RESET}")
-            else:
-                print(f"{Colors.YELLOW}  No data available. Check your internet connection.{Colors.RESET}")
-
-            print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
-            print(f"{Colors.GRAY}  Disclaimer: This tool is for educational/research purposes only.{Colors.RESET}")
-            print(f"{Colors.GRAY}  Past performance ≠ future results. Always use proper risk management.{Colors.RESET}")
-            print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
-
-
-if __name__ == "__main__":
-    main()
+#!/usr/bin/env python3
+"""
+ForexBot2 — XAU/USD Signal Intelligence
+========================================
+Usage:
+    python main.py                              # Live signal on H1
+    python main.py --timeframe H4 --detail      # Detailed H4 signal
+    python main.py --json                       # JSON output
+"""
+
+import argparse
+import json
+import logging
+import sys
+from datetime import datetime, timezone
+
+# Force UTF-8 output on Windows so box-drawing chars render
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8")
+if hasattr(sys.stderr, "reconfigure"):
+    sys.stderr.reconfigure(encoding="utf-8")
+
+from agent_panel import attach_panel
+from app_config import get_app_config, violent_watch_enabled
+from config import PlatformConfig, Timeframe, SignalStrength, XAUUSD
+from pair_signals import PAIR_PRODUCERS
+from signal_aggregator import SignalAggregator, SignalReport
+from violent_watch import attach_watch
+
+
+# ══════════════════════════════════════════════════════════════════
+#  TERMINAL FORMATTING
+# ══════════════════════════════════════════════════════════════════
+
+class Colors:
+    RESET = "\033[0m"
+    BOLD = "\033[1m"
+    DIM = "\033[2m"
+    UNDERLINE = "\033[4m"
+    RED = "\033[91m"
+    GREEN = "\033[92m"
+    YELLOW = "\033[93m"
+    BLUE = "\033[94m"
+    MAGENTA = "\033[95m"
+    CYAN = "\033[96m"
+    WHITE = "\033[97m"
+    GRAY = "\033[90m"
+
+
+def signal_color(signal: SignalStrength) -> str:
+    return {
+        SignalStrength.STRONG_BUY:  Colors.GREEN + Colors.BOLD,
+        SignalStrength.BUY:         Colors.GREEN,
+        SignalStrength.NEUTRAL:     Colors.GRAY,
+        SignalStrength.SELL:        Colors.RED,
+        SignalStrength.STRONG_SELL: Colors.RED + Colors.BOLD,
+    }.get(signal, Colors.RESET)
+
+
+def quality_color(q: str) -> str:
+    return {
+        "A+": Colors.GREEN + Colors.BOLD,
+        "A":  Colors.GREEN,
+        "B":  Colors.CYAN,
+        "C":  Colors.YELLOW,
+        "D":  Colors.GRAY,
+        "F":  Colors.DIM,
+    }.get(q, Colors.RESET)
+
+
+def print_header(title: str = "XAU/USD SIGNAL INTELLIGENCE"):
+    now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
+    print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
+    print(f"{Colors.BOLD}{Colors.CYAN}  {title}{Colors.RESET}")
+    print(f"{Colors.GRAY}  Time: {now}{Colors.RESET}")
+    print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
+
+
+# ── Live signal renderer ───────────────────────────────────────────
+
+_INDICATOR_PREFIXES = (
+    "EMA Trend", "RSI", "Bollinger Bands", "Divergence",
+    "Candlestick", "ADX", "ATR", "News", "Regime",
+)
+
+
+def _compact_reason(reason: str) -> str:
+    """Shorten a verbose reason string for the compact view."""
+    s = reason
+    if s.startswith("STRONG: "):
+        s = s[8:]
+    # Strip the leading "<Indicator>: " prefix — it's always an indicator name
+    for prefix in _INDICATOR_PREFIXES:
+        if s.startswith(prefix + ": "):
+            s = s[len(prefix) + 2:]
+            break
+    # Drop trailing detail after first ' | '
+    if " | " in s:
+        s = s.split(" | ", 1)[0]
+    return s.strip()
+
+
+def _fmt_price(price: float) -> str:
+    """Gold-scale price, 2 dp."""
+    return f"{price:.2f}"
+
+
+def print_signal_report(report: SignalReport, detail: bool = False):
+    sc = signal_color(report.final_signal)
+    qc = quality_color(report.signal_quality)
+    sig_name = report.final_signal.name.replace("_", " ")
+
+    # ── 1. DECISION ────────────────────────────────────────────────
+    mtf_tag = (f"{Colors.GREEN}MTF✓{Colors.RESET}" if report.mtf_agreement
+               else f"{Colors.YELLOW}MTF✗{Colors.RESET}")
+    print(f"  {sc}▶ {sig_name}{Colors.RESET}  "
+          f"{qc}{report.signal_quality}{Colors.RESET} · "
+          f"{sc}{report.final_score:+.1f}{Colors.RESET} · "
+          f"{report.final_confidence:.0f}% · {mtf_tag}")
+    print(f"  {Colors.GRAY}{report.action_recommendation}{Colors.RESET}")
+
+    # ── 2. TRADE SETUP (if signal is actionable) ───────────────────
+    if report.trade_setup:
+        ts = report.trade_setup
+        dc = Colors.GREEN if ts.direction == "BUY" else Colors.RED
+        print(f"\n  {Colors.BOLD}Setup{Colors.RESET}  "
+              f"{dc}{ts.direction}{Colors.RESET} @ {Colors.BOLD}{_fmt_price(ts.entry_price)}{Colors.RESET}  "
+              f"→ SL {Colors.RED}{_fmt_price(ts.stop_loss)}{Colors.RESET} "
+              f"({ts.risk_pips:.1f}p)  "
+              f"→ TP {Colors.GREEN}{_fmt_price(ts.take_profit_1)}{Colors.RESET} "
+              f"({ts.reward_pips:.1f}p)  "
+              f"RR 1:{ts.risk_reward:.1f}")
+
+    # ── 3. CONTEXT (price · regime · vol · MTF breakdown) ──────────
+    print(f"\n  {Colors.WHITE}{report.display_name}{Colors.RESET} "
+          f"{Colors.BOLD}{_fmt_price(report.current_price)}{Colors.RESET} · "
+          f"{Colors.CYAN}{report.market_regime}{Colors.RESET} · "
+          f"vol {report.volatility_regime}")
+
+    if report.mtf_signals:
+        parts = []
+        for tf, sig in report.mtf_signals.items():
+            tf_sc = signal_color(sig)
+            sc_val = report.mtf_scores.get(tf, 0)
+            parts.append(f"{tf} {tf_sc}{sig.name.replace('_',' ')}{Colors.RESET} {sc_val:+.1f}")
+        print(f"  {Colors.GRAY}MTF{Colors.RESET}    " + "  ·  ".join(parts))
+
+    # ── 4. WHY — top 3 each side, compact ──────────────────────────
+    bulls = [_compact_reason(r) for r in report.bullish_reasons[:3]]
+    bears = [_compact_reason(r) for r in report.bearish_reasons[:3]]
+    if bulls or bears:
+        print()
+        rows = max(len(bulls), len(bears))
+        for i in range(rows):
+            left  = f"{Colors.GREEN}+{Colors.RESET} {bulls[i]}" if i < len(bulls) else ""
+            right = f"{Colors.RED}-{Colors.RESET} {bears[i]}" if i < len(bears) else ""
+            print(f"  {left:<60}  {right}")
+
+    # ── 5. WARNINGS / NEWS / EVENTS ────────────────────────────────
+    if report.warnings:
+        for w in report.warnings:
+            print(f"  {Colors.YELLOW}⚠{Colors.RESET}  {w}")
+    if report.watch_line:
+        print(f"  {Colors.YELLOW}🌪{Colors.RESET} {report.watch_line}")
+    if report.event_description and report.event_risk != "clear":
+        print(f"  {Colors.YELLOW}📅{Colors.RESET} {report.event_description}")
+    if report.news_description and report.news_description != "Neutral news sentiment":
+        print(f"  {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
+    if report.news_summary:
+        print(f"  {Colors.MAGENTA}🧠{Colors.RESET} {report.news_summary}")
+    if report.panel_verdict:
+        panel_label = ("AI macro view"
+                      if report.panel_verdict_source == "macro" else "AI Panel")
+        print(f"  {Colors.CYAN}🤖{Colors.RESET} {panel_label}: "
+              f"{Colors.BOLD}{report.panel_verdict}{Colors.RESET} "
+              f"({report.panel_confidence:.0f}%) — {report.panel_rationale}")
+        for t in report.panel_takes:
+            print(f"     {Colors.GRAY}{t['persona']:<11} {t['stance']:<8} "
+                  f"{t['note']}{Colors.RESET}")
+
+    # ── 6. DETAIL (only with --detail) ─────────────────────────────
+    if detail and report.primary_analysis:
+        print(f"\n  {Colors.BOLD}Indicators ({report.primary_analysis.timeframe}){Colors.RESET}")
+        for name, ind in report.primary_analysis.indicators.items():
+            if name.startswith("_") or ind.weight == 0:
+                continue
+            isc = signal_color(ind.signal)
+            print(f"    {ind.name:<16} {isc}{ind.signal.name.replace('_',' '):<11}{Colors.RESET} "
+                  f"{ind.confidence:>3.0f}% {Colors.GRAY}│ {ind.description}{Colors.RESET}")
+
+        levels = report.primary_analysis.key_levels
+        if levels:
+            swing_keys = sorted(
+                [k for k in levels if k.startswith("swing_")],
+                key=lambda k: levels[k], reverse=True,
+            )
+            if swing_keys:
+                print(f"\n  {Colors.BOLD}Structure Levels{Colors.RESET}")
+                for k in swing_keys:
+                    val = levels[k]
+                    color = Colors.RED if val > report.current_price else Colors.GREEN
+                    label = k.replace("_", " ").title()
+                    print(f"    {label:>20}  {color}{val:.3f}{Colors.RESET}")
+
+
+def report_to_dict(report: SignalReport) -> dict:
+    d = {
+        "symbol": report.symbol,
+        "display_name": report.display_name,
+        "timestamp": report.timestamp,
+        "current_price": report.current_price,
+        "signal": report.final_signal.name,
+        "score": round(report.final_score, 2),
+        "confidence": round(report.final_confidence, 2),
+        "quality": report.signal_quality,
+        "action": report.action_recommendation,
+        "market_regime": report.market_regime,
+        "volatility_regime": report.volatility_regime,
+        "mtf_agreement": report.mtf_agreement,
+        "mtf_signals": {k: v.name for k, v in report.mtf_signals.items()},
+        "event_risk": report.event_risk,
+        "event_description": report.event_description,
+        "news_sentiment": round(report.news_sentiment, 3),
+        "news_description": report.news_description,
+        "news_summary": report.news_summary,
+        "panel_verdict": report.panel_verdict,
+        "panel_confidence": round(report.panel_confidence, 1),
+        "panel_rationale": report.panel_rationale,
+        "panel_takes": report.panel_takes,
+        "panel_created_at": report.panel_created_at,
+        "panel_cache_status": report.panel_cache_status,
+        "panel_payload_hash": report.panel_payload_hash,
+        "watch_line": report.watch_line,
+        "bullish_reasons": report.bullish_reasons,
+        "bearish_reasons": report.bearish_reasons,
+        "warnings": report.warnings,
+    }
+    if report.trade_setup:
+        ts = report.trade_setup
+        d["trade_setup"] = {
+            "direction": ts.direction,
+            "entry": ts.entry_price,
+            "stop_loss": ts.stop_loss,
+            "take_profit_1": ts.take_profit_1,
+            "take_profit_2": ts.take_profit_2,
+            "risk_reward": ts.risk_reward,
+            "risk_pips": round(ts.risk_pips, 1),
+            "reward_pips": round(ts.reward_pips, 1),
+            "strategy": ts.strategy,
+            "setup_pattern": ts.setup_pattern,
+            "round_trip_cost": ts.round_trip_cost,
+            "trail_arm_r": ts.trail_arm_r,
+            "trail_dist_r": ts.trail_dist_r,
+        }
+    if report.primary_analysis:
+        d["indicators"] = {
+            name: {
+                "signal": ind.signal.name,
+                "confidence": round(ind.confidence, 1),
+                "description": ind.description,
+            }
+            for name, ind in report.primary_analysis.indicators.items()
+            if not name.startswith("_") and ind.weight != 0
+        }
+        d["key_levels"] = {
+            k: round(v, 5) for k, v in report.primary_analysis.key_levels.items()
+        }
+    return d
+
+
+
+# ══════════════════════════════════════════════════════════════════
+#  CLI
+# ══════════════════════════════════════════════════════════════════
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="ForexBot2 — XAU/USD Signal Intelligence",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  python main.py                          Live signal (H1, summary)
+  python main.py --detail                 Live signal with full breakdown
+  python main.py --timeframe H4 --detail  H4 timeframe
+  python main.py --json                   JSON output
+        """,
+    )
+    parser.add_argument("--pair", "-p", default="XAUUSD",
+                        help="Pair to analyze: XAUUSD (default: XAUUSD)")
+    parser.add_argument("--timeframe", "-tf", default="H1",
+                        choices=list(_TF_BY_NAME),
+                        help="Primary analysis timeframe (default: H1)")
+    parser.add_argument("--detail", action="store_true",
+                        help="Show full indicator breakdown")
+    parser.add_argument("--panel", action="store_true",
+                        help="Run the LLM agent panel on the result "
+                             "(advisory only; needs a DeepSeek key)")
+    parser.add_argument("--json", action="store_true",
+                        help="Output as JSON")
+    parser.add_argument("--verbose", "-v", action="store_true",
+                        help="Debug logging")
+
+    return parser.parse_args()
+
+
+# The four validated primary timeframes (the same set the bot and the
+# dashboard scan). M30/D1 were never validated as primaries.
+_TF_BY_NAME = {
+    "M5": Timeframe.M5, "M15": Timeframe.M15,
+    "H1": Timeframe.H1, "H4": Timeframe.H4,
+}
+
+
+def _resolve_pair(s: str) -> str:
+    aliases = {
+        "XAUUSD": XAUUSD, "XAU/USD": XAUUSD, "GOLD": XAUUSD,
+    }
+    key = s.upper().strip()
+    if key in aliases:
+        return aliases[key]
+    raise SystemExit(f"Unknown pair '{s}'. Try one of: {', '.join(sorted(aliases))}")
+
+
+def build_config(args) -> PlatformConfig:
+    """Per-pair tuned config from the pair_signals factory.
+
+    The factory owns the validated confirmation map (M5->[M15,H1],
+    M15->[M5,H1], H1->[M15,H4], H4->[H1,D1]); passing None here keeps
+    the CLI on the same map as the bot and the dashboard. A CLI-local
+    table once paired M5 with the unvalidated [M1, M15].
+    """
+    pair = _resolve_pair(args.pair)
+    return PAIR_PRODUCERS[pair](_TF_BY_NAME[args.timeframe], None)
+
+
+def run_live(args, config: PlatformConfig):
+    print(f"{Colors.GRAY}  Fetching market data and running analysis...{Colors.RESET}\n")
+    aggregator = SignalAggregator(config)
+    report = aggregator.analyze_pair(symbol=config.symbol, log_source="cli")
+
+    # Explicit user intent: force bypasses the enabled/surface/quality
+    # toggles, but the daily panel budget and LLM cap still apply.
+    if getattr(args, "panel", False):
+        attach_panel(report, surface="cli", force=True)
+
+    # Display-only violent-move watch (dark by default; never touches
+    # the signal, only report.watch_line).
+    if violent_watch_enabled():
+        attach_watch(report, config.primary_timeframe,
+                     get_app_config().violent_watch_settings)
+
+    if args.json:
+        payload = report_to_dict(report)
+        print(json.dumps(payload, indent=2))
+        return
+
+    print_header(f"{config.display_name} SIGNAL INTELLIGENCE")
+    print_signal_report(report, detail=args.detail)
+    print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
+    print(f"{Colors.GRAY}  Disclaimer: For educational/research purposes only.{Colors.RESET}")
+    print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
+
+
+def main():
+    args = parse_args()
+
+    log_level = logging.DEBUG if args.verbose else logging.WARNING
+    logging.basicConfig(
+        level=log_level,
+        format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+        datefmt="%H:%M:%S",
+    )
+
+    config = build_config(args)
+    run_live(args, config)
+
+
+if __name__ == "__main__":
+    main()

news_archive.py

@@ -0,0 +1,361 @@
+"""
+ForexBot2 — News archive (durable headline corpus)
+==================================================
+
+Every headline the news engine fetches is written here, exactly as it
+arrived, together with the score that was attached to it and **which
+scorer produced that score**. One row per distinct headline, kept
+forever.
+
+Why this exists
+---------------
+Headlines used to survive only as a side effect of signal logging, inside
+``logs/signals.db``. That database was rotated/rebuilt around 2026-09-04
+and three weeks of news — 4,054 distinct headlines, the corpus the
+``news_sentiment_value_2026-09-04.md`` study was written on — was lost.
+``logs/`` is gitignored, so git could not bring it back and nobody
+noticed until 2026-09-08.
+
+This module writes to its own database (``logs/newsdata.db`` by default)
+so signal logging can be wiped, rotated or rebuilt without touching the
+news corpus.
+
+What the corpus is for
+----------------------
+Training a small local scorer that reproduces DeepSeek's headline
+labelling without calling it. On the 2026-08-06 golden set DeepSeek
+scored 78/78 and the built-in keyword scorer it overrides scored 24/78
+(31%) — and the keyword score is what stands whenever DeepSeek is
+unavailable (no key, disabled, daily cap, timeout, HTTP/parse failure,
+or the >25-new-headlines-per-cycle batch overflow). ``score_source``
+records which one produced each row; training filters on ``'llm'``.
+
+Contract
+--------
+* **Never raises.** ``archive_headlines`` swallows every exception. A
+  logging failure must not break a scan — the same contract
+  ``signal_log.log_signal`` holds.
+* **Upgrade-only scores.** A row already marked ``llm`` is never
+  overwritten by a later ``keyword`` score. A headline first seen under
+  the batch-overflow cap is keyword-scored, then gets its DeepSeek score
+  on a later fetch window; the better label wins and is never demoted.
+* **Dedup by headline text.** ``hash`` = sha256 of the headline. A
+  headline seen 50 times is one row with ``last_seen`` bumped.
+* **stdlib only** (``sqlite3``), like ``signal_log``.
+
+Known corpus limitation: a headline mentioning no tracked currency is
+never constructed by the news engine at all (``news_engine`` drops it
+before scoring), so DeepSeek never labels it either and it cannot appear
+here.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+import os
+import sqlite3
+import threading
+import time
+from datetime import datetime, timezone
+from typing import Any, Dict, Iterable, Optional
+
+import app_config
+
+logger = logging.getLogger(__name__)
+
+
+_SCHEMA = """
+CREATE TABLE IF NOT EXISTS headlines (
+    hash TEXT PRIMARY KEY,
+    headline TEXT NOT NULL,
+    description TEXT,
+    link TEXT,
+    source TEXT,
+    publisher TEXT,
+    published_at TEXT,
+    first_seen TEXT NOT NULL,
+    last_seen TEXT NOT NULL,
+    seen_count INTEGER NOT NULL DEFAULT 1,
+    score REAL,
+    score_by_currency_json TEXT,
+    currencies_json TEXT,
+    keywords_json TEXT,
+    score_source TEXT NOT NULL DEFAULT 'keyword',
+    teacher_model TEXT,
+    teacher_prompt_sha1 TEXT,
+    origin TEXT NOT NULL DEFAULT 'live'
+);
+CREATE INDEX IF NOT EXISTS idx_headlines_source ON headlines (score_source);
+CREATE INDEX IF NOT EXISTS idx_headlines_first_seen ON headlines (first_seen);
+"""
+
+#: Rank of a score's provenance. A write only lands when its rank is >=
+#: the rank already stored, so a keyword score can never overwrite an LLM
+#: one (the batch-overflow case) but an LLM score always upgrades.
+_SOURCE_RANK = {"unknown": 0, "keyword": 1, "model": 2, "llm": 3}
+
+
+def _utc_now_iso() -> str:
+    return datetime.now(timezone.utc).isoformat()
+
+
+def headline_hash(headline: str) -> str:
+    """Primary key for a headline: sha256 of its exact text."""
+    return hashlib.sha256(headline.encode("utf-8")).hexdigest()
+
+
+def _json_or_none(value: Any) -> Optional[str]:
+    if value is None:
+        return None
+    try:
+        return json.dumps(value, default=str)
+    except Exception:
+        return None
+
+
+def _iso_or_none(value: Any) -> Optional[str]:
+    if value is None:
+        return None
+    if isinstance(value, datetime):
+        return value.isoformat()
+    return str(value)
+
+
+def _finite_score(value: Any) -> Optional[float]:
+    try:
+        out = float(value)
+    except (TypeError, ValueError):
+        return None
+    if out != out or out in (float("inf"), float("-inf")):
+        return None
+    return out
+
+
+class NewsArchive:
+    """Thread-safe SQLite writer for the headline corpus. One instance
+    per database path; safe to share across the bot's executor threads
+    and the dashboard daemon's pool."""
+
+    def __init__(self, path: str):
+        parent = os.path.dirname(path)
+        if parent:
+            os.makedirs(parent, exist_ok=True)
+        self._conn = sqlite3.connect(path, check_same_thread=False)
+        # busy_timeout BEFORE the journal-mode switch: the bot, the
+        # dashboard publisher and the tests open this file at once and
+        # the WAL switch needs a brief exclusive lock.
+        self._conn.execute("PRAGMA busy_timeout=5000")
+        self._lock = threading.Lock()
+        with self._lock:
+            # busy_timeout does not cover the WAL switch, which needs a
+            # brief exclusive lock: three processes creating this file at
+            # once will race it. Same retry signal_log uses.
+            self._with_lock_retry(
+                lambda: self._conn.execute("PRAGMA journal_mode=WAL"))
+            self._with_lock_retry(lambda: self._conn.executescript(_SCHEMA))
+            self._conn.commit()
+
+    @staticmethod
+    def _with_lock_retry(fn, attempts: int = 8, delay: float = 0.05):
+        """Run ``fn`` again on 'database is locked' (concurrent openers
+        racing the same schema step); other errors propagate at once."""
+        for attempt in range(attempts):
+            try:
+                return fn()
+            except sqlite3.OperationalError as exc:
+                if "locked" not in str(exc).lower() or attempt == attempts - 1:
+                    raise
+                time.sleep(delay * (attempt + 1))
+
+    def close(self) -> None:
+        with self._lock:
+            try:
+                self._conn.close()
+            except Exception:
+                pass
+
+    #: Rank of the score already stored, computed in SQL so the compare
+    #: happens inside the same atomic statement as the write.
+    _STORED_RANK = ("CASE headlines.score_source WHEN 'llm' THEN 3"
+                    " WHEN 'model' THEN 2 WHEN 'keyword' THEN 1 ELSE 0 END")
+
+    def store(self, rows: Iterable[Dict[str, Any]]) -> int:
+        """Insert-or-upgrade a batch of prepared rows. Returns the number
+        of rows written. Raises on a database failure — the module-level
+        wrapper is what guarantees a scan cannot break.
+
+        One atomic UPSERT per row. A read-then-write pair would lose an
+        increment whenever two processes (bot, dashboard publisher, tests)
+        first see the same headline at the same moment: both would find it
+        absent, one INSERT would win and the other's count would vanish.
+        """
+        rows = list(rows)
+        if not rows:
+            return 0
+        now = _utc_now_iso()
+        wins = f"? >= {self._STORED_RANK}"   # incoming rank vs stored rank
+        sql = (
+            "INSERT INTO headlines ("
+            " hash, headline, description, link, source, publisher,"
+            " published_at, first_seen, last_seen, seen_count,"
+            " score, score_by_currency_json, currencies_json,"
+            " keywords_json, score_source, teacher_model,"
+            " teacher_prompt_sha1, origin"
+            ") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?)"
+            " ON CONFLICT(hash) DO UPDATE SET"
+            # The backfill stamps historical timestamps, so clamp rather
+            # than assign: first_seen only ever moves earlier, last_seen
+            # only ever moves later.
+            "  first_seen = MIN(headlines.first_seen, excluded.first_seen),"
+            "  last_seen = MAX(headlines.last_seen, excluded.last_seen),"
+            "  seen_count = headlines.seen_count + 1,"
+            # Upgrade-only: a weaker scorer never overwrites a stronger
+            # one's label, but re-seeing a headline always counts.
+            f"  score = CASE WHEN {wins} THEN excluded.score"
+            "           ELSE headlines.score END,"
+            f"  score_by_currency_json = CASE WHEN {wins}"
+            "           THEN excluded.score_by_currency_json"
+            "           ELSE headlines.score_by_currency_json END,"
+            f"  currencies_json = CASE WHEN {wins} THEN excluded.currencies_json"
+            "           ELSE headlines.currencies_json END,"
+            f"  keywords_json = CASE WHEN {wins} THEN excluded.keywords_json"
+            "           ELSE headlines.keywords_json END,"
+            # COALESCE, not a bare overwrite: the backfill re-ingests
+            # llm-scored headlines with NO teacher identity (neither source
+            # DB recorded one), and rank >= rank is true, so a plain
+            # assignment blanks what the live hook already stored.
+            f"  teacher_model = CASE WHEN {wins}"
+            "           THEN COALESCE(excluded.teacher_model,"
+            "                         headlines.teacher_model)"
+            "           ELSE headlines.teacher_model END,"
+            f"  teacher_prompt_sha1 = CASE WHEN {wins}"
+            "           THEN COALESCE(excluded.teacher_prompt_sha1,"
+            "                         headlines.teacher_prompt_sha1)"
+            "           ELSE headlines.teacher_prompt_sha1 END,"
+            f"  score_source = CASE WHEN {wins} THEN excluded.score_source"
+            "           ELSE headlines.score_source END,"
+            # Enrichment fills gaps but never blanks what is already there
+            # (the legacy backfill shape carries no description or link).
+            "  description = COALESCE(excluded.description, headlines.description),"
+            "  link = COALESCE(excluded.link, headlines.link),"
+            "  publisher = COALESCE(excluded.publisher, headlines.publisher),"
+            "  published_at = COALESCE(excluded.published_at,"
+            "                          headlines.published_at)")
+
+        # The rank is bound once per CASE arm. Positional binding means a
+        # miscount silently shifts every parameter, so assert it instead of
+        # trusting the arithmetic: 17 INSERT values + one rank per arm.
+        n_rank = sql.count("? >= ")
+        n_values = 17
+        assert sql.count("?") == n_values + n_rank, (
+            f"placeholder count drifted: {sql.count('?')} "
+            f"!= {n_values} + {n_rank}")
+
+        params = []
+        for row in rows:
+            rank = _SOURCE_RANK.get(row["score_source"], 0)
+            params.append((
+                row["hash"], row["headline"], row["description"], row["link"],
+                row["source"], row["publisher"], row["published_at"],
+                row.get("first_seen") or now, row.get("last_seen") or now,
+                row["score"], row["score_by_currency_json"],
+                row["currencies_json"], row["keywords_json"],
+                row["score_source"], row["teacher_model"],
+                row["teacher_prompt_sha1"], row.get("origin", "live"),
+            ) + (rank,) * n_rank)
+
+        def _write() -> None:
+            # BEGIN IMMEDIATE takes the write lock up front, so contention
+            # fails here — before any row is applied — and the rollback
+            # makes the batch atomic if it fails later. Without this a
+            # partial executemany would be re-applied by the retry and
+            # double-count seen_count on the rows that already landed.
+            self._conn.execute("BEGIN IMMEDIATE")
+            try:
+                self._conn.executemany(sql, params)
+                self._conn.commit()
+            except Exception:
+                try:
+                    self._conn.rollback()
+                except Exception:
+                    pass
+                raise
+
+        with self._lock:
+            self._with_lock_retry(_write)
+        return len(rows)
+
+
+def build_row(headline: Any, *, teacher_model: Optional[str] = None,
+              teacher_prompt_sha1: Optional[str] = None,
+              origin: str = "live",
+              first_seen: Optional[str] = None,
+              last_seen: Optional[str] = None) -> Optional[Dict[str, Any]]:
+    """Flatten one ``NewsSentiment`` (or an equivalent object) into a row.
+
+    Returns None when the object carries no usable headline text."""
+    text = getattr(headline, "headline", None)
+    if not isinstance(text, str) or not text.strip():
+        return None
+    source = getattr(headline, "score_source", None) or "keyword"
+    return {
+        "hash": headline_hash(text),
+        "headline": text,
+        "description": getattr(headline, "description", None),
+        "link": getattr(headline, "link", None),
+        "source": getattr(headline, "source", None),
+        "publisher": getattr(headline, "publisher", None),
+        "published_at": _iso_or_none(getattr(headline, "timestamp", None)),
+        "first_seen": first_seen,
+        "last_seen": last_seen,
+        "score": _finite_score(getattr(headline, "sentiment_score", None)),
+        "score_by_currency_json": _json_or_none(
+            getattr(headline, "sentiment_by_currency", None)),
+        "currencies_json": _json_or_none(
+            getattr(headline, "currencies_mentioned", None)),
+        "keywords_json": _json_or_none(getattr(headline, "keywords", None)),
+        "score_source": source,
+        "teacher_model": teacher_model if source == "llm" else None,
+        "teacher_prompt_sha1": (
+            teacher_prompt_sha1 if source == "llm" else None),
+        "origin": origin,
+    }
+
+
+#: path -> NewsArchive
+_archives: Dict[str, NewsArchive] = {}
+_archives_lock = threading.Lock()
+
+
+def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
+                      teacher_prompt_sha1: Optional[str] = None,
+                      origin: str = "live") -> None:
+    """Persist a batch of scored headlines if the news_archive config
+    block enables it. NEVER raises — a logging failure must not break a
+    scan."""
+    try:
+        if not headlines:
+            return
+        if not app_config.news_archive_enabled():
+            return
+        rows = []
+        for headline in headlines:
+            row = build_row(headline, teacher_model=teacher_model,
+                            teacher_prompt_sha1=teacher_prompt_sha1,
+                            origin=origin)
+            if row is not None:
+                rows.append(row)
+        if not rows:
+            return
+        path = app_config.news_archive_path()
+        with _archives_lock:
+            archive = _archives.get(path)
+            if archive is None:
+                archive = NewsArchive(path)
+                _archives[path] = archive
+        archive.store(rows)
+    except Exception:
+        logger.warning("news_archive: failed to persist headlines",
+                       exc_info=True)

pair_signals/__init__.py

@@ -0,0 +1,47 @@
+"""
+Per-pair signal producers.
+
+Each pair module exposes:
+    make_config(primary_tf, confirmation_tfs) -> PlatformConfig
+
+To add a new pair: create pair_signals/<pair>.py with make_config(),
+then register it in this file.
+"""
+
+from typing import Optional, List, Dict, Callable
+
+from config import Timeframe
+from signal_aggregator import SignalAggregator, SignalReport
+from providers import BaseDataProvider
+
+PAIR_PRODUCERS: Dict[str, Callable] = {}
+
+
+def register_pair(symbol: str, config_factory: Callable):
+    PAIR_PRODUCERS[symbol] = config_factory
+
+
+def produce_signal(
+    symbol: str,
+    primary_tf: Timeframe,
+    data_provider: BaseDataProvider,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+    log_source: str = "dashboard",
+) -> SignalReport:
+    config_factory = PAIR_PRODUCERS.get(symbol)
+    if config_factory is None:
+        raise ValueError(
+            f"No signal producer registered for {symbol}. "
+            f"Available: {list(PAIR_PRODUCERS.keys())}"
+        )
+
+    config = config_factory(primary_tf, confirmation_tfs)
+    aggregator = SignalAggregator(config)
+    aggregator.data_provider = data_provider
+    return aggregator.analyze_pair(symbol=symbol, log_source=log_source)
+
+
+from pair_signals import xauusd as _xau_mod
+from config import XAUUSD
+
+register_pair(XAUUSD, _xau_mod.make_config)

pair_signals/xauusd.py

@@ -0,0 +1,314 @@
+"""XAU/USD signal producer config.
+
+PlatformConfig defaults ARE the tuned XAU/USD configuration, so this
+factory sets the timeframes plus documented per-timeframe overrides.
+It must stay equivalent to the inline PlatformConfig(symbol=XAUUSD,
+...) construction (plus those overrides) used by the backtester and
+Telegram bot (enforced by a test in test_all.py).
+
+M15 override (validated 2026-07-11, window 2025-03-30 -> 2026-05-31,
+$0.30 spread, trailing exits): tp_floor_rr 1.0 scores +116.6R/460
+trades PF 1.83 vs +105.4R/424 PF 1.82 at the 0.5 default — better in
+both train/val halves and on the 2024-06 -> 2025-03 out-of-sample
+window (+30.3R vs +10.6R; the 1R floor lifts targets past the min-TP
+spread gate, roughly doubling trades there). The same knob HURTS M5
+(pinned 12mo book: +131.3R/699 vs +139.9R/547, PF 1.55 vs 1.78), so it
+is per-timeframe, exactly as PENDING_STEPS' decision rule anticipated.
+
+M15 override 2 (validated 2026-07-11, 12mo window 2025-07-11 ->
+2026-07-11 + OOS 2024-06-01 -> 2025-03-30, $0.30 spread): block the
+mixed "trending" regime (ADX strong but EMAs unaligned — momentum
+without directional structure). Those trades lost -6.8R/24 on 12mo,
+negative in BOTH halves; blocking scores +65.7R/499 PF 1.38 DD 10.4R
+vs +59.0R/521 PF 1.32 DD 12.6R baseline, better in both halves. OOS
+is neutral (-1.4R = the 4 such trades that existed there; no power to
+confirm or refute). M5 never trades this regime (its gates already
+exclude it), so the block is M15-only. A second candidate — blocking
+transitioning-regime SELLs (-4.0R/47 on 12mo) — was REJECTED: those
+sells made +5.7R/32 in the 2024 OOS window (bull-market artifact).
+
+M15 override 3 (validated 2026-07-16, 5-year replay 2021-07-17 ->
+2026-07-16 under the audit-fixed backtester, $0.30 spread, trailing
+exits): block SELL entries whose bar CLOSES at 15:00 or 17:00 UTC
+(sell_blocked_hours_utc=[15, 17], gate 6g3 — BUYs unaffected; those
+hours flank the already-blocked hour 16). The pocket lost -33.5R/165
+trades, negative in 6/6 calendar years and in BOTH halves of the last
+12 months (H1 -6.8R, H2 -4.2R); each hour independently is 5/6 years
+negative with both halves negative (hours 14/18 fail that screen and
+stay open). Cross-TF the M5 book CONFIRMS (same pocket -14.1R/59,
+5/6 years negative) rather than contradicts — unlike the rejected
+trending_down block. Full replay with the gate: +176.7R/1627 PF 1.31
+DD 29.1R vs +144.9R/1781 PF 1.22 DD 34.4R control; better in ALL SIX
+years; last-12mo halves both improve (H1 +36.3 vs +29.5, H2 +46.9 vs
++43.2; 12mo PF 1.55 vs 1.43); OOS 2024-06-01 -> 2025-03-30 improves
+(+45.4R PF 1.47 DD 17.8 vs +41.9R PF 1.39 DD 21.7). SELL side goes
+-1.2R -> +30.6R over 5 years. BUYs at those hours made +34.3R, hence
+SELL-only rather than a blocked_hours_utc entry. Mechanism: NY-session
+shorts against gold's macro uptrend (15:00 = post-US-data reversal
+window, 17:00 = London close unwind).
+
+M5 override 2 (2026-07-16, chart patterns): enable BUY-only CP on M5
+(inv_hs, double_bottom, bull_flag — drop all bearish patterns). SELL CP
+is consistently negative (OOS -3.21R, 5yr +3.76R PF 1.07) while BUY CP
+is positive across windows: 12mo +41.8R PF 1.60 (train +9.66 PF 1.30,
+val +32.17 PF 1.86), 5yr +65.2R PF 1.48, OOS +1.20R (aggregate
+positive, val half -1.89R — marginal but sign-correct in aggregate).
+Gold's structural uptrend makes bullish patterns fundamentally more
+reliable. M15/H1 fail on all windows; M5-only enablement.
+
+M5 override 3 (validated 2026-07-21, 5-year replay 2021-07-21 ->
+2026-07-21, $0.30 spread, next-open fills): trailing distance widened
+to 0.85R (trail_dist_r 0.85, arm unchanged 0.60). Same 30-cell grid
+methodology as the M15 campaign, opposite finding — on M5 the arm
+point barely matters; the winners need a looser leash. Plateau arm
+0.5-0.7 x dist 0.85-1.0 (+227..+234R vs +218.0R baseline). Chosen
+0.60/0.85 (OOS-preferred end of the plateau): +231.2R/2028 PF 1.30 WR
+57.1% vs +218.0R PF 1.29 WR 58.3%; both last-12mo halves better
+(+56.8/+115.1 vs +54.0/+104.6); 5yr max DD 19.2R vs 19.1R (noise);
+OOS 2024-06-01 -> 2025-03-30 +14.5R DD 15.1 vs +11.1R DD 15.4.
+dist 1.0 was REJECTED at the OOS gate (+10.9R < baseline +11.1R)
+despite the best in-sample total (+233.3R) — the OOS ordering picks
+the conservative plateau edge. XAU_PB trades are unaffected (they
+carry their own validated pb_trail 0.8/0.75 per-trade overrides);
+XAU_CP trades share the global params and were validated inside this
+book. M15 arms at 0.70 (its own campaign, same day); the divergence
+is the per-TF pattern yet again.
+
+M15 override 4 (validated 2026-07-21, 5-year replay 2021-07-21 ->
+2026-07-21, $0.30 spread, next-open fills): trailing-stop arm raised to
+0.70R (trail_arm_r 0.70, dist unchanged 0.75). Full grid arm 0.3-0.9 x
+dist 0.5-1.0 (30 cells): every earlier-arm cell (0.3/0.45) is WORSE —
+the "arm earlier to save fading trades" hypothesis is refuted; the
+plateau is arm 0.70-0.80 x dist 0.70-0.85 (all >= +190R). Chosen
+plateau center 0.70/0.75: +210.7R/1629 PF 1.35 WR 57.8% vs baseline
+0.60/0.75 +175.7R PF 1.30 WR 53.9%; both last-12mo halves improve
+(+49.7/+50.0 vs +36.4/+46.0); per-year better 4/6 (2022 -0.9R, 2023
+-1.4R, both within tolerance); OOS 2024-06-01 -> 2025-03-30 +57.4R
+PF 1.57 DD 17.0R vs +45.4R PF 1.47 DD 17.8R — better on every OOS
+metric. One clause waived BY OWNER DECISION 2026-07-21: 5yr max DD is
+33.9R vs 29.1R baseline (+4.8R deeper worst episode, 2022-23 chop),
+accepted in exchange for +20% total R. M5 keeps the global 0.6 arm
+(same per-TF divergence pattern as tp_floor_rr and the hour-9 block).
+
+M15 override 5 — RSI regime multiplier x 0.5 in every regime
+(validated 2026-09-09, 2-year Oanda window 2024-09-05 -> 2026-09-05 at
+$0.65, pre-registered rule in backtest/research/
+weight_test_2026-09-09.md): 793/+117.62R PF 1.41 DD 14.8R ->
+931/+146.14R PF 1.43 DD 16.1R, fixed halves +27.9/+0.6R vs control;
+OOS 2024-06-01 -> 2025-03-30 300/+40.59R PF 1.38 -> 355/+60.67R
+PF 1.49; 3-year 1092/+151.90R PF 1.38 -> 1273/+186.61R PF 1.39 (max DD
+15.5 -> 19.3R, all in the 2023-24 year; 2026 -4R is the one calendar
+year that does not improve). Mechanism: RSI is neutral on ~73% of M15
+entries and votes against the trend when it fires; halving it lets more
+trend-following entries clear the score band (210 added trades +26.8R,
+mostly B-grade BUYs; 72 removed -7.1R). Candidate came from the
+recorded replay screen (replay_recording_2026-09-09.md); the same
+screen's M5 picks (MACD-down) all FAILED the replay, and ema_trend x 2
+/ divergence -> 0 on M15 failed too. Canonical M15 book re-pinned to
+backtest/oanda_m15_2yr_065_20260909.csv.
+
+M5 override (validated 2026-07-11, full-history replay 2020-03-17 ->
+2026-07-11 + 12mo book, $0.30 spread): block hour 9 UTC (post-London-
+open whipsaw). Hour-9 entries lost -28.9R/143 over 6 years, negative
+in 5/7 calendar years including winning years (2022 -9.2R). Blocking:
++207.1R vs +178.3R baseline on the 6-year replay, max DD 28.7R vs
+35.5R, improves 5/7 years (2023 goes -13.0R -> breakeven); the tuned
+12mo book is unchanged (+139.0R vs +139.3R). M15-and-up keep hour 9:
+it is POSITIVE there (M15 6yr +1.7R, M15 OOS +8.6R), so the block is
+M5-only, same per-timeframe pattern as the M15 overrides.
+
+M5 override 2 — relaxation campaign (validated 2026-07-30, Oanda
+bars, $0.30 spread, trailing exits): UNBLOCK hour 21 UTC on M5 only,
+combined with the config.py M5 gate relaxations (m5_min_abs_score
+30->25, m5_min_confirmation_score 10->5). The combined config beat
+the pre-relaxation control in ALL 6 calendar years of the 5yr replay
+2021-07-30 -> 2026-07-30 (+230.9R/2319 PF 1.26 DD 21.3 vs
++168.0R/2048 PF 1.21 DD 27.0), on 12mo (+107.4R PF 1.36, halves
++68.6/+38.8 vs +84.6R PF 1.33), on OOS 2024-06-01 -> 2025-03-30
+(+26.1R vs +17.2R), and at $0.70 spread (+86.9R vs +66.4R). Hour-21
+trades alone: +28.0R/22, positive every year — overturning the old
+block per the full-evidence rule. Hour 21 stays BLOCKED on M15+
+(12mo M15 unblock tested -0.3R). Declined-but-open from the same
+campaign: an M5 hour-22 block (+4.8R on 12mo — the burst-heavy
+"institutional" hour — but -3.2R on 5yr and -1.1R on OOS: regime-
+specific); M15 hour-16 unblock (+0.8R/+57 trades 12mo but -18.3R
+DD +34% on 5yr); every SELL-gate/vol/TP-size relaxation retested
+negative (see memory notes for the full grid).
+
+M5 override 4 — XAU_PB disabled (validated 2026-08-04, Oanda bars,
+$0.65 REAL spread — first campaign validated at measured cost on the
+live feed, per the 2026-08-04 re-pin baseline): m5_pullback_enabled
+False on M5. Loss forensics on the Oanda books identified XAU_PB as
+the largest wrong-direction loss source (12mo: -43.5R of the -156.6R
+wrong-direction bucket; 5yr: -274.5R). Disabling: 12mo +96.6R/588
+PF 1.44 max DD 13.9R halves +51.2/+45.4 vs +82.3R/788 PF 1.26 DD
+23.9R shipped (which FAILS the PF>=1.3 clause at real cost); OOS
+2024-06-01 -> 2025-03-30 +9.5R PF 1.34; 5yr +123.9R/956 PF 1.35 vs
+PF 1.16 with PB. PB's 5yr net contribution (+23.6R) was almost
+entirely the 2022 chop year (+36.1R vs -2.2R without) — a regime
+earner that has been a drag on trend-regime bars since; its tight
+stops are also the most cost-punished at $0.65. PB code and the
+pb_trail_* params STAY for future regime-gated experiments (same
+pattern as cp_enabled). Rejected in the same campaign (4 candidates
+total, N logged): hour-19 block (+3.7R 12mo but -3.3R OOS, -1.5R
+5yr — OOS gate, the classic trap); hours-14/15 block (-6.9R 12mo
+once PB is removed — their raw negativity WAS the PB losses);
+trail_dist_r 0.75 (-2.6R 12mo; the 0.88R average give-back is the
+validated price of riding trends, exits confirmed NOT the loss
+driver: exit-management accounts for 1-2% of lost R vs 50%
+wrong-direction / 49% entry-timing).
+
+M5 override 6 — SELL block in transitioning regime (validated
+2026-09-03, signal-log campaign, `backtest/research/
+signal_log_campaign_2026-09.md`, $0.65 round trip, same CPU harness
+as its control): sell_blocked_regimes=["transitioning"]. Live
+evidence: M5 SELLs in transitioning were -7.9R/12mo PF 0.21 and
+-5.2R/5yr PF 0.49 in the pinned books; the live signal log showed the
+same pocket -7.9R/PF 0.56 over 3 weeks. Blocking: 12mo (2025-08-14 ->
+2026-08-14) 557 trades +112.21R PF 1.56 (halves +55.9/+56.4, PF
+1.54/1.58) vs control 575/+104.34R PF 1.50; 5yr (2021-08-01 ->
+2026-08-04) 889/+133.7R PF 1.41 vs 911/+128.4R PF 1.39; OOS
+2024-06-01 -> 2025-03-30 unchanged (86/+9.45R PF 1.33 — no such
+trades in that window). Removed trades: 18 in 12mo (-7.9R), 22 in
+5yr (-5.3R).
+
+H1 override — all SELL entries blocked (validated 2026-09-03, same
+campaign): sell_min_quality=[] (gate 6e in signal_aggregator.py
+downgrades any actionable-grade SELL whose quality is not in this
+list to D, so an empty list downgrades every H1 short). H1 shorts
+were a structural loser: -16.7R over the 5yr control window and
+-11.5R OOS. Blocking: 12mo (2025-08-04 -> 2026-08-04) 91/+22.2R
+PF 1.79 (halves +20.6/+1.6) vs control 138/+24.0R PF 1.55 (12mo cost
+-1.8R); 5yr 335/+49.1R PF 1.46 vs 515/+32.4R PF 1.18; OOS 78/+16.7R
+PF 1.74 vs 104/+5.1R PF 1.13. New reference books:
+backtest/oanda_h1_{12mo,5yr,oos}_065_20260903.csv,
+backtest/oanda_m5_12mo_065_nopb_20260903.csv,
+backtest/oanda_m5_5yr_065_nopb_20260903.csv.
+
+Rejected/parked in the same campaign (see the memo for the full
+table): M15 sell_min_quality=["B"] (5yr -8.9R, max DD 35.8 -> 41.4R
+— REJECTED); Bollinger weight 0 in trending regimes on M5, M15, H1
+(M5 -20.4R/12mo, -15.5R/5yr; M15 -24.0R/12mo — REJECTED across the
+board); divergence disabled on M15 (R improves +7.4/+11.9/+13.5R
+across windows but 5yr max DD 35.8 -> 47.4R with PF flat — DECLINED,
+owner call); divergence disabled on H1 (+4.6/+8.1/+2.4R, PF
+1.62/1.20/1.17) — PARKED: it passes rule 1 in combination with the
+H1 no-short override (b12_h1_nosell_nodiv: 12mo 100/+23.1R PF 1.75,
+halves +21.8/+1.3; 5yr 364/+58.7R PF 1.52; OOS 88/+21.9R PF 1.92 —
+better than the no-shorts override alone on both 5yr and OOS), but
+shipping it would route through technical.disabled_indicators, which
+CLAUDE.md rule 8 reserves for experiments; owner decision, recommended
+vehicle is an H1-only regime_weights_override with divergence weight
+0 plus a fresh validation run.
+
+M5 override 5 — EMA partial-alignment squash (validated 2026-08-04,
+Oanda bars, $0.65 real spread): ema_partial_neutral True on M5 —
+partial 4-EMA ribbon alignment (2-3 of 4 checks) scores NEUTRAL
+instead of a weak BUY/SELL. Motivated by the indicator audit: partial-
+alignment trades averaged -0.008R/84 vs +0.230R/341 fully aligned.
+Replay (knob mirrored in the backtester's precomputed EMA scorer in
+the same commit — live==backtest parity): 12mo +97.6R/556 PF 1.48
+DD 10.7R halves +47.8/+49.8 vs control +96.6R/588 PF 1.44 DD 13.9R;
+5yr +127.4R/912 PF 1.38 DD 12.8R vs +123.9R/956 PF 1.35 DD 13.2R,
+2022 improves -2.2 -> -1.1; OOS 2024-06-01 -> 2025-03-30 +9.4R
+PF 1.33 vs +9.5R PF 1.34 — a one-trade tie, documented as such (the
+OOS window holds up; it neither prefers nor punishes). Net effect:
+32 fewer marginal trades, every risk-adjusted measure better.
+Rejected in the same round (trial-logged): confirmation-TF swap
+M5->[M15,H4] (12mo +104.0R PF 1.51 DD 10.3 but OOS +7.8R PF 1.27
+vs control +9.5R PF 1.34 and 2022 3x worse — the H4 confirm keeps
+endorsing stale direction in chop; OOS-ordering rejection, same
+precedent as trail dist 1.0); confirm [H1,H4] (-31.8R 12mo);
+triple-confirm [M15,H1,H4] (-4.9R); divergence weight -1/3
+(-3.8R 12mo); candlestick weight +50% (-3.0R 12mo). The weight
+vector keeps proving calibrated as a system: divergence/candlestick/
+bollinger all look mis-weighted in-book and all replay worse when
+"fixed" (bollinger weight-0: +80.9R PF 1.29 DD 23.8 — its constant
+oppose vote is a load-bearing quality bar, like ROC).
+"""
+
+from typing import Dict, Optional, List
+from config import PlatformConfig, TechnicalConfig, Timeframe
+from app_config import apply_pair_overrides
+
+_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+
+def _m15_regime_weights() -> Dict[str, Dict[str, float]]:
+    """Shipped regime weight table with the RSI multiplier halved in
+    every regime — the M15-only override validated 2026-09-09 (see the
+    module docstring, M15 override 5). Derived from the default table so
+    the intent ("RSI x 0.5, everything else as shipped") stays explicit;
+    the Module 9 equivalence test pins the resulting numbers."""
+    from engine import regime_weight_table  # local import: keep the factory import-light
+    table = regime_weight_table(TechnicalConfig())   # a fresh TechnicalConfig has no override -> the shipped table
+    return {regime: {name: (mult * 0.5 if name == "rsi" else mult)
+                     for name, mult in weights.items()}
+            for regime, weights in table.items()}
+
+
+def make_config(
+    primary_tf: Timeframe,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+) -> PlatformConfig:
+    config = PlatformConfig()
+    # Deployment overlay (config.json) BEFORE the validated per-TF
+    # deltas below, so e.g. the M5 hour-9 append lands on the JSON base.
+    apply_pair_overrides(config)
+    config.primary_timeframe = primary_tf
+    config.confirmation_timeframes = (
+        confirmation_tfs or _CONFIRMATIONS.get(primary_tf, [Timeframe.M15, Timeframe.H4])
+    )
+    if primary_tf == Timeframe.M5:
+        # Block hour 9 UTC on M5 only; UNBLOCK hour 21 on M5 only
+        # (positive every year of the 5yr replay) — see docstring.
+        config.xau_strategy.blocked_hours_utc = [
+            h for h in config.xau_strategy.blocked_hours_utc if h != 21
+        ] + [9]
+        # Deliberately NOT mirrored into config.backtest.blocked_hours_utc:
+        # that copy is the generic non-XAU fallback carrying the JSON base
+        # only. Propagating a gold-validated per-TF delta there would leak
+        # XAU-calibrated hours to a future pair (pair-isolation rule).
+        # Every XAU consumer branches to the xau_strategy copy. Enforced
+        # by the Module 13 test "M5 factory appends hour 9 on top of a
+        # JSON base".
+        # BUY-only chart patterns on M5 — see module docstring.
+        config.xau_strategy.cp_enabled = True
+        config.xau_strategy.cp_patterns = ["inv_hs", "double_bottom", "bull_flag"]
+        # XAU_PB pullback entries OFF — largest wrong-direction loss
+        # source at real cost on Oanda bars — see module docstring.
+        config.xau_strategy.m5_pullback_enabled = False
+        # Partial EMA-ribbon alignment scores NEUTRAL on M5 — its
+        # trades carried no edge — see module docstring.
+        config.technical.ema_partial_neutral = True
+        # Wider trailing distance on M5 only — see module docstring.
+        config.signals.trail_arm_r = 0.60
+        config.signals.trail_dist_r = 0.85
+        # Block SELL entries while the regime is "transitioning" on M5
+        # only — see module docstring (M5 override 6).
+        config.xau_strategy.sell_blocked_regimes = ["transitioning"]
+    if primary_tf == Timeframe.M15:
+        # 1R minimum target on M15 only — see module docstring.
+        config.signals.tp_floor_rr = 1.0
+        # Block the mixed "trending" regime on M15 only — see docstring.
+        config.xau_strategy.blocked_regimes = ["trending"]
+        # Block SELL entries at 15:00/17:00 UTC on M15 only — see docstring.
+        config.xau_strategy.sell_blocked_hours_utc = [15, 17]
+        # Later trailing-stop arm on M15 only — see module docstring.
+        config.signals.trail_arm_r = 0.70
+        config.signals.trail_dist_r = 0.75
+        # RSI regime multiplier × 0.5 in every regime on M15 only — see
+        # module docstring (M15 override 5, validated 2026-09-09).
+        config.technical.regime_weights_override = _m15_regime_weights()
+    if primary_tf == Timeframe.H1:
+        # Block every H1 SELL — the SELL quality gate (6e in
+        # signal_aggregator.py) downgrades any actionable-grade short
+        # to D when its grade is not in this list; an empty list means
+        # no grade qualifies. H1-only — see module docstring.
+        config.xau_strategy.sell_min_quality = []
+    return config

position_sizing.py

@@ -0,0 +1,611 @@
+"""
+ForexBot2 — Position sizing (stake-multiplier overlay)
+======================================================
+
+Single source of truth for **decision-time features** and the resulting
+**stake multiplier**. Everything here is designed to be called from
+inside the signal path so that live and backtest see byte-identical
+inputs (rule 4: live == backtest parity).
+
+Design contract
+---------------
+* **Bounded lookbacks.** No feature may read further back than the live
+  fetch depth of its timeframe (5000 bars on M5/M15, 1460 on H1, 380 on
+  H4). The primary frame is hard-capped at ``MAX_LOOKBACK_BARS`` rows,
+  confirmation frames at 60 rows, and the burst scan at 200 rows, so a
+  feature computed on a full backtest frame equals the same feature
+  computed on the shorter live tail.
+* **Closed bars only.** The frames handed in are exactly the frames
+  ``analyze_from_dataframes`` received: UTC tz-aware, oldest-first,
+  capitalized OHLCV columns, forming candle already dropped. The last
+  row is the decision bar; nothing peeks past it.
+* **Never raises into a scan.** ``build_features`` and
+  ``stake_multiplier`` swallow every exception (logged at DEBUG) and
+  fall back to neutral defaults. A sizing failure can never break a
+  signal.
+* **Advisory today.** ``stake_multiplier`` returns ``1.0`` in every case
+  at this stage — artifact evaluation lands in a later step. The guard
+  order is already implemented so the wiring is testable now.
+
+Dependencies are deliberately limited to numpy/pandas plus ``config``
+(the ``Timeframe`` enum): this module ships live and must not drag the
+provider/news stack into any process that only wants features.
+
+VERDICT 2026-09-06 (backtest/research/sizing_campaign_2026-09.md): the walk-forward
+stake-multiplier model FAILED the pre-registered acceptance rule on both M5 and
+M15 (M5 a clean null; M15 lift inside the label-shuffle null and explained by
+BUY-vs-SELL). NO artifact ships and nothing reads a multiplier live. This module
+is research plumbing: `build_features` feeds the backtester's --export-features
+flag; `stake_multiplier` returns 1.0 unless a validated artifact is ever added
+under a NEW pre-registration.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import math
+from pathlib import Path
+from typing import Dict, Optional, Sequence, Tuple
+
+import numpy as np
+import pandas as pd
+
+from config import SignalStrength, Timeframe
+
+logger = logging.getLogger(__name__)
+
+
+#: Bump whenever FEATURE_ORDER or any feature's definition changes.
+#: A trained artifact carrying a different value is ignored.
+FEATURE_VERSION = 1
+
+#: Hard cap on any primary-timeframe window used in this module.
+MAX_LOOKBACK_BARS = 500
+
+#: Bars of each confirmation timeframe used by the cross-TF features.
+CONFIRM_LOOKBACK_BARS = 60
+
+#: Window for the "bars since the last volatility burst" feature. Also
+#: the sentinel returned when no burst is present in the window.
+BURST_LOOKBACK_BARS = 200
+
+#: Burst threshold: bar range >= this multiple of the PRIOR bar's ATR.
+BURST_ATR_MULTIPLE = 3.0
+
+#: Candle duration in seconds. Deliberate local copy of
+#: ``providers.base.BaseDataProvider.CANDLE_SECONDS`` — importing the
+#: providers package here would pull requests/twelvedata into every
+#: feature-only consumer. A test asserts the copies agree.
+CANDLE_SECONDS: Dict[Timeframe, int] = {
+    Timeframe.M5: 300,
+    Timeframe.M15: 900,
+    Timeframe.H1: 3600,
+    Timeframe.H4: 14400,
+    Timeframe.D1: 86400,
+}
+
+#: Grade -> ordinal used by ``f_quality_rank`` (D/F both fall to 0).
+_QUALITY_RANK = {"A+": 4.0, "A": 3.0, "B": 2.0, "C": 1.0}
+
+_BULLISH = (SignalStrength.BUY, SignalStrength.STRONG_BUY)
+_BEARISH = (SignalStrength.SELL, SignalStrength.STRONG_SELL)
+
+
+FEATURE_ORDER: Tuple[str, ...] = (
+    # ── Time ──────────────────────────────────────────────────────
+    "f_hour_utc",
+    "f_hour_sin",
+    "f_hour_cos",
+    "f_weekday",
+    "f_sess_asian",
+    "f_sess_london",
+    "f_sess_overlap",
+    "f_sess_ny",
+    "f_minutes_to_hour",
+    "f_day_of_month",
+    "f_first_friday",
+    # ── Volatility / setup geometry ───────────────────────────────
+    "f_atr",
+    "f_atr_pct",
+    "f_atr_rel_500",
+    "f_range_pctile_500",
+    "f_bar_range_atr",
+    "f_close_pos_in_bar",
+    "f_risk_price",
+    "f_sl_atr",
+    "f_tp1_atr",
+    "f_rr_planned",
+    # ── Exhaustion block ──────────────────────────────────────────
+    "f_swing_dist_atr",
+    "f_ema21_dist_atr",
+    "f_ema200_dist_atr",
+    "f_ema200_dist_pct",
+    "f_exhaustion_count",
+    "f_tag_exhaustion",
+    "f_tag_no_pullback",
+    "f_tag_weak_pullback",
+    "f_tag_rsi_extended",
+    "f_tag_bb_overextended",
+    "f_tag_ema200_near",
+    "f_tag_ema200_far",
+    "f_penalty_total",
+    "f_momentum_fading",
+    # ── Signal quality ────────────────────────────────────────────
+    "f_score_dir",
+    "f_confidence",
+    "f_quality_rank",
+    "f_ind_agree_frac",
+    "f_ind_oppose_frac",
+    "f_is_buy",
+    "f_strat_atr",
+    "f_strat_pb",
+    "f_strat_cp",
+    "f_strat_generic",
+    # ── Multi-timeframe ───────────────────────────────────────────
+    "f_mtf_primary",
+    "f_mtf_c1",
+    "f_mtf_c2",
+    "f_mtf_agreement",
+    # ── Primary-TF indicators ─────────────────────────────────────
+    "f_rsi",
+    "f_rsi_dir",
+    "f_bb_pos",
+    "f_bb_pos_dir",
+    "f_adx",
+    "f_kc_ext_atr",
+    "f_roc_dir",
+    "f_macd_hist_dir",
+    # ── Price path ────────────────────────────────────────────────
+    "f_ret_6_atr",
+    "f_ret_12_atr",
+    "f_ret_24_atr",
+    "f_ret_48_atr",
+    "f_consec_closes_dir",
+    "f_bars_since_burst",
+    # ── Cross-timeframe extension ─────────────────────────────────
+    "f_h1_ext_atr",
+    "f_c_other_ext_atr",
+)
+
+#: Value each feature falls back to when it cannot be computed. Only
+#: ``f_bars_since_burst`` has a non-zero sentinel (its "none in window"
+#: cap); everything else degrades to a neutral 0.0.
+_DEFAULTS: Dict[str, float] = {name: 0.0 for name in FEATURE_ORDER}
+_DEFAULTS["f_bars_since_burst"] = float(BURST_LOOKBACK_BARS)
+
+#: Penalty tags recorded by SignalAggregator._compute_final_signal that
+#: this module exposes as one-hot features.
+_TAG_FEATURES = (
+    ("f_tag_exhaustion", "exhaustion"),
+    ("f_tag_no_pullback", "no_pullback"),
+    ("f_tag_weak_pullback", "weak_pullback"),
+    ("f_tag_rsi_extended", "rsi_extended"),
+    ("f_tag_bb_overextended", "bb_overextended"),
+    ("f_tag_ema200_near", "ema200_near"),
+    ("f_tag_ema200_far", "ema200_far"),
+)
+
+
+# ══════════════════════════════════════════════════════════════════
+#  Small helpers
+# ══════════════════════════════════════════════════════════════════
+
+def _finite(value, default: float = 0.0) -> float:
+    """Coerce to a finite Python float, falling back to ``default``."""
+    try:
+        out = float(value)
+    except (TypeError, ValueError):
+        return float(default)
+    if not math.isfinite(out):
+        return float(default)
+    return out
+
+
+def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
+    """Wilder ATR, identical convention to ``engine._compute_atr``.
+
+    True Range, then an EWM with ``alpha = 1/period``,
+    ``min_periods = period`` and ``adjust=False``.
+    """
+    high, low, close = df["High"], df["Low"], df["Close"]
+    tr1 = high - low
+    tr2 = (high - close.shift()).abs()
+    tr3 = (low - close.shift()).abs()
+    tr = pd.concat([tr1, tr2, tr3], axis=1).max(axis=1)
+    return tr.ewm(alpha=1.0 / period, min_periods=period, adjust=False).mean()
+
+
+def decision_time(primary_df: pd.DataFrame, tf: Timeframe) -> pd.Timestamp:
+    """The clock every gate uses: last CLOSED bar label + its duration."""
+    last = pd.Timestamp(primary_df.index[-1])
+    return last + pd.Timedelta(seconds=CANDLE_SECONDS.get(tf, 3600))
+
+
+def session_label(hour_utc: int) -> str:
+    """Trading-session bucket for a UTC hour."""
+    h = int(hour_utc)
+    if 3 <= h <= 6:
+        return "asian"
+    if 7 <= h <= 12:
+        return "london"
+    if 13 <= h <= 16:
+        return "overlap"
+    if 17 <= h <= 22:
+        return "ny"
+    return "off"
+
+
+# ══════════════════════════════════════════════════════════════════
+#  Artifact plumbing
+# ══════════════════════════════════════════════════════════════════
+
+#: path -> (mtime, artifact-or-None)
+_ARTIFACT_CACHE: Dict[str, Tuple[float, Optional[dict]]] = {}
+
+
+def artifact_path_for(primary_tf: Timeframe, base_dir) -> Path:
+    """Where the trained artifact for one primary timeframe lives."""
+    return Path(base_dir) / f"xauusd_{primary_tf.value}.json"
+
+
+def load_artifact(path) -> Optional[dict]:
+    """Load a sizing artifact (JSON object), cached on (path, mtime).
+
+    Returns ``None`` when the file is missing, unreadable, not JSON, or
+    not a JSON object. Never raises.
+    """
+    try:
+        p = Path(path)
+        mtime = p.stat().st_mtime
+    except Exception:
+        return None
+
+    key = str(p)
+    cached = _ARTIFACT_CACHE.get(key)
+    if cached is not None and cached[0] == mtime:
+        return cached[1]
+
+    data: Optional[dict]
+    try:
+        with p.open("r", encoding="utf-8") as fh:
+            loaded = json.load(fh)
+        data = loaded if isinstance(loaded, dict) else None
+    except Exception:
+        logger.debug("position_sizing: artifact %s is unreadable", key,
+                     exc_info=True)
+        data = None
+
+    _ARTIFACT_CACHE[key] = (mtime, data)
+    return data
+
+
+# ══════════════════════════════════════════════════════════════════
+#  Feature construction
+# ══════════════════════════════════════════════════════════════════
+
+def _direction(setup, report) -> float:
+    """+1.0 for a long, -1.0 for a short — from the TRADE, not the score."""
+    direction = getattr(setup, "direction", None)
+    if direction is None:
+        return 1.0 if _finite(getattr(report, "final_score", 0.0)) >= 0 else -1.0
+    return 1.0 if str(direction).upper() == "BUY" else -1.0
+
+
+def _ext_vs_sma(df: pd.DataFrame, d: float) -> float:
+    """Direction-signed distance from the 20-bar SMA in ATR units.
+
+    Computed on the LAST ``CONFIRM_LOOKBACK_BARS`` rows only, so the
+    value is identical on a full history and on its live-depth tail.
+    """
+    if df is None or len(df) < 21:
+        return 0.0
+    tail = df.iloc[-CONFIRM_LOOKBACK_BARS:]
+    sma = tail["Close"].rolling(20).mean().iloc[-1]
+    atr = wilder_atr(tail, 14).iloc[-1]
+    atr_f = _finite(atr, 0.0)
+    if atr_f <= 0:
+        return 0.0
+    return d * (_finite(tail["Close"].iloc[-1]) - _finite(sma)) / atr_f
+
+
+def _fill_time(out: Dict[str, float], dt: pd.Timestamp) -> None:
+    hour = int(dt.hour)
+    out["f_hour_utc"] = float(hour)
+    out["f_hour_sin"] = float(math.sin(2.0 * math.pi * hour / 24.0))
+    out["f_hour_cos"] = float(math.cos(2.0 * math.pi * hour / 24.0))
+    weekday = int(dt.weekday())
+    out["f_weekday"] = float(weekday)
+    label = session_label(hour)
+    out["f_sess_asian"] = 1.0 if label == "asian" else 0.0
+    out["f_sess_london"] = 1.0 if label == "london" else 0.0
+    out["f_sess_overlap"] = 1.0 if label == "overlap" else 0.0
+    out["f_sess_ny"] = 1.0 if label == "ny" else 0.0
+    out["f_minutes_to_hour"] = float((60 - int(dt.minute)) % 60)
+    out["f_day_of_month"] = float(int(dt.day))
+    out["f_first_friday"] = 1.0 if (weekday == 4 and int(dt.day) <= 7) else 0.0
+
+
+def _fill_geometry(out: Dict[str, float], P: pd.DataFrame, setup,
+                   atr: float, norm, d: float) -> None:
+    out["f_atr"] = max(0.0, _finite(atr))
+    ranges = (P["High"] - P["Low"]).astype(float)
+    last_range = _finite(ranges.iloc[-1])
+    mean_range = _finite(ranges.mean())
+    out["f_atr_rel_500"] = (_finite(atr) / mean_range) if mean_range > 0 else 0.0
+    n = int(len(ranges))
+    out["f_range_pctile_500"] = (
+        float((ranges.to_numpy() < last_range).sum()) / n if n else 0.0)
+    out["f_bar_range_atr"] = norm(last_range)
+
+    high = _finite(P["High"].iloc[-1])
+    low = _finite(P["Low"].iloc[-1])
+    close = _finite(P["Close"].iloc[-1])
+    span = high - low
+    if span > 0:
+        out["f_close_pos_in_bar"] = (
+            (close - low) / span if d > 0 else (high - close) / span)
+    else:
+        out["f_close_pos_in_bar"] = 0.5
+
+    entry = _finite(getattr(setup, "entry_price", None))
+    stop = _finite(getattr(setup, "stop_loss", None))
+    tp1 = _finite(getattr(setup, "take_profit_1", None))
+    risk = abs(entry - stop)
+    out["f_risk_price"] = risk
+    out["f_sl_atr"] = norm(risk)
+    out["f_tp1_atr"] = norm(abs(tp1 - entry))
+    out["f_rr_planned"] = _finite(getattr(setup, "risk_reward", None))
+
+
+def _fill_exhaustion(out: Dict[str, float], report, KL: Dict[str, float],
+                     px: float, d: float, norm) -> None:
+    if d > 0:
+        swings = [KL[k] for k in ("swing_low_1", "swing_low_2", "swing_low_3")
+                  if KL.get(k) is not None]
+        out["f_swing_dist_atr"] = norm(px - min(swings)) if swings else 0.0
+    else:
+        swings = [KL[k] for k in ("swing_high_1", "swing_high_2", "swing_high_3")
+                  if KL.get(k) is not None]
+        out["f_swing_dist_atr"] = norm(max(swings) - px) if swings else 0.0
+
+    ema21 = KL.get("ema_medium")
+    out["f_ema21_dist_atr"] = (
+        norm(d * (px - _finite(ema21))) if ema21 is not None else 0.0)
+
+    ema200 = KL.get("ema_trend")
+    ema200_f = _finite(ema200, 0.0) if ema200 is not None else 0.0
+    if ema200 is not None and ema200_f != 0.0:
+        out["f_ema200_dist_atr"] = norm(d * (px - ema200_f))
+        out["f_ema200_dist_pct"] = d * (px - ema200_f) / ema200_f * 100.0
+    else:
+        out["f_ema200_dist_atr"] = 0.0
+        out["f_ema200_dist_pct"] = 0.0
+
+    out["f_exhaustion_count"] = _finite(getattr(report, "exhaustion_count", 0))
+    tags = getattr(report, "penalty_tags", None) or {}
+    for feature, tag in _TAG_FEATURES:
+        out[feature] = 1.0 if tag in tags else 0.0
+    out["f_penalty_total"] = _finite(getattr(report, "penalty_total", 0.0))
+    out["f_momentum_fading"] = 1.0 if getattr(report, "momentum_fading", False) else 0.0
+
+
+def _fill_quality(out: Dict[str, float], report, setup, IND: dict,
+                  d: float) -> None:
+    out["f_score_dir"] = d * _finite(getattr(report, "final_score", 0.0))
+    out["f_confidence"] = _finite(getattr(report, "final_confidence", 0.0))
+    out["f_quality_rank"] = _QUALITY_RANK.get(
+        getattr(report, "signal_quality", ""), 0.0)
+
+    agree = oppose = total = 0
+    for name, ind in (IND or {}).items():
+        if str(name).startswith("_"):
+            continue
+        if _finite(getattr(ind, "weight", 0.0)) == 0.0:
+            continue
+        total += 1
+        signal = getattr(ind, "signal", None)
+        if d > 0:
+            if signal in _BULLISH:
+                agree += 1
+            elif signal in _BEARISH:
+                oppose += 1
+        else:
+            if signal in _BEARISH:
+                agree += 1
+            elif signal in _BULLISH:
+                oppose += 1
+    out["f_ind_agree_frac"] = (agree / total) if total else 0.0
+    out["f_ind_oppose_frac"] = (oppose / total) if total else 0.0
+
+    out["f_is_buy"] = 1.0 if d > 0 else 0.0
+    strategy = str(getattr(setup, "strategy", "") or "")
+    # The ATR path stamps "XAU_ATR"; "XAU" is accepted for legacy setups.
+    out["f_strat_atr"] = 1.0 if strategy in ("XAU_ATR", "XAU") else 0.0
+    out["f_strat_pb"] = 1.0 if strategy == "XAU_PB" else 0.0
+    out["f_strat_cp"] = 1.0 if strategy == "XAU_CP" else 0.0
+    out["f_strat_generic"] = 1.0 if strategy == "GENERIC" else 0.0
+
+
+def _fill_mtf(out: Dict[str, float], report, primary_tf: Timeframe,
+              confirmation_tfs: Sequence[Timeframe], d: float) -> None:
+    scores = getattr(report, "mtf_scores", None) or {}
+    out["f_mtf_primary"] = d * _finite(scores.get(primary_tf.value, 0.0))
+    confirms = list(confirmation_tfs or [])
+    for idx, key in ((0, "f_mtf_c1"), (1, "f_mtf_c2")):
+        if idx < len(confirms):
+            out[key] = d * _finite(scores.get(confirms[idx].value, 0.0))
+        else:
+            out[key] = 0.0
+    out["f_mtf_agreement"] = 1.0 if getattr(report, "mtf_agreement", False) else 0.0
+
+
+def _fill_indicators(out: Dict[str, float], IND: dict, d: float, norm) -> None:
+    IND = IND or {}
+
+    def _val(name, default):
+        ind = IND.get(name)
+        return _finite(getattr(ind, "value", None), default) if ind else default
+
+    rsi = _val("rsi", 50.0)
+    out["f_rsi"] = rsi
+    out["f_rsi_dir"] = d * (rsi - 50.0)
+    bb_pos = _val("bollinger", 0.5)          # bollinger.value = bb_position
+    out["f_bb_pos"] = bb_pos
+    out["f_bb_pos_dir"] = d * (bb_pos - 0.5)
+    out["f_adx"] = _val("adx", 0.0)
+    # keltner.value = close - kc_mid (signed extension from the mid line)
+    out["f_kc_ext_atr"] = norm(d * _val("keltner", 0.0))
+    out["f_roc_dir"] = d * _val("roc", 0.0)
+    # macd.value = histogram
+    out["f_macd_hist_dir"] = d * _val("macd", 0.0)
+
+
+def _fill_path(out: Dict[str, float], P: pd.DataFrame, d: float, norm) -> None:
+    close = P["Close"].astype(float).to_numpy()
+    n = len(close)
+    for lag, key in ((6, "f_ret_6_atr"), (12, "f_ret_12_atr"),
+                     (24, "f_ret_24_atr"), (48, "f_ret_48_atr")):
+        out[key] = norm(d * (close[-1] - close[-1 - lag])) if n >= lag + 1 else 0.0
+
+    run = 0.0
+    if n >= 2:
+        diffs = np.diff(close)
+        last = diffs[-1]
+        if last > 0:
+            sign = 1.0
+        elif last < 0:
+            sign = -1.0
+        else:
+            sign = 0.0
+        if sign != 0.0:
+            count = 0
+            for value in diffs[::-1]:
+                if (value > 0 and sign > 0) or (value < 0 and sign < 0):
+                    count += 1
+                else:
+                    break
+            run = sign * count
+    out["f_consec_closes_dir"] = max(-20.0, min(20.0, d * run))
+
+    out["f_bars_since_burst"] = float(BURST_LOOKBACK_BARS)
+    prev_atr = wilder_atr(P, 14).shift(1).to_numpy(dtype=float)
+    ranges = (P["High"] - P["Low"]).astype(float).to_numpy()
+    start = max(0, n - BURST_LOOKBACK_BARS)
+    for i in range(n - 1, start - 1, -1):
+        threshold = prev_atr[i]
+        if not math.isfinite(threshold) or threshold <= 0:
+            continue
+        if ranges[i] >= BURST_ATR_MULTIPLE * threshold:
+            out["f_bars_since_burst"] = float((n - 1) - i)
+            break
+
+
+def _fill_cross_tf(out: Dict[str, float], dfs: Dict[Timeframe, pd.DataFrame],
+                   confirmation_tfs: Sequence[Timeframe], d: float) -> None:
+    out["f_h1_ext_atr"] = _ext_vs_sma(dfs.get(Timeframe.H1), d)
+    out["f_c_other_ext_atr"] = 0.0
+    for tf in (confirmation_tfs or []):
+        if tf is Timeframe.H1:
+            continue
+        df = dfs.get(tf)
+        if df is None or len(df) < 21:
+            continue
+        out["f_c_other_ext_atr"] = _ext_vs_sma(df, d)
+        break
+
+
+def build_features(
+    report,
+    setup,
+    dfs: Dict[Timeframe, pd.DataFrame],
+    primary_tf: Timeframe,
+    confirmation_tfs: Sequence[Timeframe],
+) -> Dict[str, float]:
+    """Decision-time feature vector for one signal.
+
+    Returns a dict whose keys are exactly ``FEATURE_ORDER`` (in order),
+    every value a finite Python float. Degenerate inputs fall back to
+    the neutral defaults instead of raising — this runs inside a scan.
+    """
+    out: Dict[str, float] = dict(_DEFAULTS)
+
+    try:
+        P = dfs.get(primary_tf)
+        if P is None or len(P) == 0:
+            return {k: out[k] for k in FEATURE_ORDER}
+        P = P.iloc[-MAX_LOOKBACK_BARS:]
+
+        d = _direction(setup, report)
+        primary = getattr(report, "primary_analysis", None)
+        KL = dict(getattr(primary, "key_levels", None) or {})
+        IND = getattr(primary, "indicators", None) or {}
+
+        atr_raw = KL.get("atr")
+        atr = _finite(atr_raw, 0.0)
+        atr_ok = atr > 0
+        atr_div = atr if atr_ok else 1e-9
+
+        def norm(value: float) -> float:
+            """ATR-normalise, or 0.0 when ATR is missing/non-positive."""
+            return (_finite(value) / atr_div) if atr_ok else 0.0
+
+        px = _finite(P["Close"].iloc[-1])
+        out["f_atr_pct"] = _finite(KL.get("atr_pct"), 0.0)
+
+        for step, args in (
+            (_fill_time, (out, decision_time(P, primary_tf))),
+            (_fill_geometry, (out, P, setup, atr, norm, d)),
+            (_fill_exhaustion, (out, report, KL, px, d, norm)),
+            (_fill_quality, (out, report, setup, IND, d)),
+            (_fill_mtf, (out, report, primary_tf, confirmation_tfs, d)),
+            (_fill_indicators, (out, IND, d, norm)),
+            (_fill_path, (out, P, d, norm)),
+            (_fill_cross_tf, (out, dfs, confirmation_tfs, d)),
+        ):
+            try:
+                step(*args)
+            except Exception:
+                logger.debug("position_sizing: %s failed", step.__name__,
+                             exc_info=True)
+    except Exception:
+        logger.debug("position_sizing.build_features failed", exc_info=True)
+
+    return {k: _finite(out.get(k), _DEFAULTS[k]) for k in FEATURE_ORDER}
+
+
+# ══════════════════════════════════════════════════════════════════
+#  Stake multiplier
+# ══════════════════════════════════════════════════════════════════
+
+def stake_multiplier(
+    report,
+    setup,
+    dfs: Dict[Timeframe, pd.DataFrame],
+    primary_tf: Timeframe,
+    confirmation_tfs: Sequence[Timeframe],
+    artifact_dir=None,
+) -> float:
+    """Stake multiplier for a taken setup. Always ``1.0`` at this stage.
+
+    The guard order is already final — no artifact directory, a missing
+    artifact file, an artifact built for a different ``FEATURE_VERSION``,
+    or ANY exception all yield the neutral ``1.0``. Artifact evaluation
+    lands in a later step; until then this is wiring, not a decision.
+    """
+    try:
+        if not artifact_dir:
+            return 1.0
+        artifact = load_artifact(artifact_path_for(primary_tf, artifact_dir))
+        if artifact is None:
+            return 1.0
+        if artifact.get("feature_version") != FEATURE_VERSION:
+            logger.debug(
+                "position_sizing: artifact feature_version %r != %d — neutral",
+                artifact.get("feature_version"), FEATURE_VERSION)
+            return 1.0
+        # Step 1: evaluation not implemented yet.
+        return 1.0
+    except Exception:
+        logger.debug("position_sizing.stake_multiplier failed", exc_info=True)
+        return 1.0

providers/__init__.py

@@ -0,0 +1,57 @@
+"""
+ForexBot2 — Data-provider package
+==================================
+One class per provider, all subclassing `BaseDataProvider` (which owns
+caching, validation, boundary guarantees, and range pagination — the
+behavior the signal pipeline depends on).
+
+Adding a provider = one module implementing `_fetch_raw` + one registry
+entry + one config.json `data_providers` block. The live candle feed is
+selected by config.json `data_providers.active`; the optional
+`data_providers.streaming` scalar selects a different provider for the
+tick stream (level-breach monitor). Registration alone changes nothing.
+"""
+
+from typing import Dict, Optional
+
+from app_config import AppConfigError, get_app_config
+from config import PlatformConfig
+
+from providers.base import BaseDataProvider, DataIntegrityError, _is_forex_weekend
+from providers.twelve_data import TwelveDataProvider
+from providers.oanda import OandaDataError, OandaDataProvider
+from providers.tiingo import TiingoDataError, TiingoDataProvider
+
+#: config.json data_providers.active selects from here.
+PROVIDER_REGISTRY: Dict[str, type] = {
+    "twelve_data": TwelveDataProvider,
+    "oanda": OandaDataProvider,
+    "tiingo": TiingoDataProvider,
+}
+
+
+def make_data_provider(config: PlatformConfig,
+                       name: Optional[str] = None) -> BaseDataProvider:
+    """Build a provider by name (default: config.json data_providers.active)."""
+    if name is None:
+        name = get_app_config().active_provider
+    cls = PROVIDER_REGISTRY.get(name)
+    if cls is None:
+        raise AppConfigError(
+            f'Unknown data provider "{name}"; '
+            f"registered: {sorted(PROVIDER_REGISTRY)}")
+    return cls(config)
+
+
+__all__ = [
+    "BaseDataProvider",
+    "DataIntegrityError",
+    "OandaDataError",
+    "OandaDataProvider",
+    "TiingoDataError",
+    "TiingoDataProvider",
+    "TwelveDataProvider",
+    "PROVIDER_REGISTRY",
+    "make_data_provider",
+    "_is_forex_weekend",
+]

providers/base.py

@@ -0,0 +1,670 @@
+"""
+ForexBot2 — Provider base class
+================================
+`BaseDataProvider` owns everything that must behave identically no matter
+where the bars come from: in-memory caching with candle-duration TTL, the
+closed-bar boundary guarantee, freshness checks, validation/cleaning, the
+forming-candle drop, and paginated date-range fetching for the backtester.
+
+A concrete provider implements only:
+    _fetch_raw(symbol, timeframe, outputsize, start, end) -> pd.DataFrame
+returning a normalized frame (capitalized OHLCV columns, UTC tz-aware
+DatetimeIndex named "Datetime", oldest-first), plus optionally
+`get_spread` and the streaming-quote capability (`supports_streaming` +
+`iter_ticks`).
+
+The signal pipeline (engine + aggregator + backtester) consumes ONLY the
+public contract defined here, so swapping providers never touches signal
+code.
+"""
+
+import logging
+import threading
+import time
+from abc import ABC, abstractmethod
+from datetime import datetime, timedelta
+from typing import AsyncIterator, Callable, Dict, Optional, Sequence, Tuple
+from zoneinfo import ZoneInfo
+
+import pandas as pd
+
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+logger = logging.getLogger(__name__)
+
+# The gold/FX market week is defined in New York time (see
+# _is_forex_weekend); a fixed UTC hour is wrong for half the year.
+_NY_TZ = ZoneInfo("America/New_York")
+
+
+def _is_forex_weekend(now=None) -> bool:
+    """Gold/FX market closed for the weekend.
+
+    The market week is anchored to NEW YORK time — Friday 17:00 ET →
+    Sunday 18:00 ET — not a fixed UTC hour. During US DST the feed
+    actually stops at 21:00 UTC (measured on 5 years of Oanda bars:
+    159/162 summer Fridays; winter close 22:00, winter reopen Sunday
+    23:00 UTC). The old hardcoded Fri 22:00→Sun 22:00 UTC rule produced
+    two false-"STALE DATA/feed lagging" hours every week (2026-08-04
+    audit I43, fixed per owner decision).
+
+    Kept as three deliberate copies (here, signal_aggregator.py,
+    gpu_backtest._is_xau_blocked_by_calendar) — a Module 21 test asserts
+    they agree hour-by-hour across DST transitions.
+    """
+    if now is None:
+        now = pd.Timestamp.now(tz="UTC")
+    ny = now.astimezone(_NY_TZ)
+    wd = ny.weekday()
+    if wd == 4:                      # Friday
+        return ny.hour >= 17
+    if wd == 5:                      # Saturday
+        return True
+    if wd == 6:                      # Sunday
+        return ny.hour < 18
+    return False
+
+
+def _is_daily_maintenance(now=None) -> bool:
+    """Gold's daily settlement pause: Monday-Thursday 17:00-17:59 New York
+    (21:00 UTC in US summer, 22:00 in winter). No bar carries a label
+    inside this hour. Friday 17:00 is the weekend (see _is_forex_weekend).
+    Same rule as advisory_outcome._has_continuity's daily_maintenance."""
+    if now is None:
+        now = pd.Timestamp.now(tz="UTC")
+    ny = now.astimezone(_NY_TZ)
+    return ny.weekday() < 4 and ny.hour == 17
+
+
+def _is_scheduled_closure(now=None) -> bool:
+    """The feed is legitimately silent: weekend or daily settlement pause."""
+    return _is_forex_weekend(now) or _is_daily_maintenance(now)
+
+
+def _scheduled_closed_seconds(start, end) -> float:
+    """Seconds of the [start, end) span that fall inside a scheduled
+    closure (weekend or the daily settlement pause).
+
+    Bar age is measured from a bar LABEL, so at 21:30 UTC in summer the
+    20:55 M5 bar looks 35 minutes old even though the feed could not have
+    printed anything since 21:00. Subtracting the closed span keeps the
+    3-candle freshness limit meaningful instead of failing it — and
+    logging a false "feed lagging"/"STALE DATA" — every settlement hour
+    (2026-09-13 audit: 61 suppressed scans on three nights).
+    """
+    hour = pd.Timedelta(hours=1)
+    closed = 0.0
+    t = start.floor("h")
+    steps = 0
+    while t < end and steps < 24 * 10:
+        if _is_scheduled_closure(t):
+            closed += max(
+                0.0, (min(t + hour, end) - max(t, start)).total_seconds())
+        t += hour
+        steps += 1
+    return closed
+
+
+class DataIntegrityError(Exception):
+    """Raised when data fails validation checks."""
+    pass
+
+
+class BaseDataProvider(ABC):
+    """Shared OHLCV provider behavior; subclasses supply the raw fetch."""
+
+    #: Registry name; overridden per provider.
+    name: str = "base"
+
+    #: Exception type this provider raises for fetch/validation failures.
+    #: Provider-specific errors subclass DataIntegrityError so consumers
+    #: can catch one type regardless of the active provider.
+    DATA_ERROR = DataIntegrityError
+
+    BARS_PER_DAY = {
+        Timeframe.M1: 1440, Timeframe.M5: 288, Timeframe.M15: 96,
+        Timeframe.M30: 48, Timeframe.H1: 24, Timeframe.H4: 6,
+        Timeframe.D1: 1, Timeframe.W1: 0.143,
+    }
+
+    CANDLE_SECONDS = {
+        Timeframe.M1: 60, Timeframe.M5: 300, Timeframe.M15: 900,
+        Timeframe.M30: 1800, Timeframe.H1: 3600, Timeframe.H4: 14400,
+        Timeframe.D1: 86400, Timeframe.W1: 604800,
+    }
+
+    #: Hard per-request bar cap shared by Twelve Data and Oanda.
+    MAX_BARS_PER_REQUEST = 5000
+
+    #: Minimum bars per LIVE fetch on every timeframe (2026-09-10,
+    #: indicator-seeding parity with the backtester). The engine's EMAs
+    #: use ewm(adjust=False), so the first bar of the fetched frame keeps
+    #: weight (1-2/201)^n in the 200-bar trend EMA. With the 60-day
+    #: lookback, H4 fetched 380 bars (seed weight 2.3%) and D1 300 (5%)
+    #: while the backtester seeds from its whole cached history. Measured
+    #: on the deepest Oanda caches (267 H4 / 313 D1 bars sampled over two
+    #: years, raw and cleaned frames alike): ema_trend flipped on 2/267 H4
+    #: and 1/313 D1 bars at those depths (composite off by up to 9.5
+    #: points) and on none from 600 bars up; 1000 leaves a seed weight of
+    #: e^-10. M5/M15 (5000) and H1 (1460) already exceed it and are
+    #: unchanged. One request either way (cap 5000).
+    MIN_LIVE_BARS = 1000
+
+    # Grace wait before the one retry when a fresh response at a candle
+    # boundary is missing the just-closed bar (feed latency).
+    FRESH_FETCH_RETRY_SECONDS = 3.0
+
+    # Signal-side bad-print filter (see _validate_and_clean): a bar is a
+    # candidate when its close-to-close return sits more than
+    # OUTLIER_SIGMA standard deviations from the frame mean, and is
+    # dropped only when the NEXT close snaps back at least
+    # OUTLIER_REVERT_FRACTION of the way to the pre-spike close.
+    OUTLIER_SIGMA = 10.0
+    OUTLIER_REVERT_FRACTION = 0.8
+
+    def __init__(self, config: PlatformConfig):
+        self.config = config
+        self._cache: Dict[str, pd.DataFrame] = {}
+        self._cache_timestamps: Dict[str, float] = {}
+        # Requested fetch depth per key: a cached short frame must not
+        # satisfy a caller asking for a longer lookback.
+        self._cache_outputsize: Dict[str, int] = {}
+        # The bot shares one provider across a thread pool; cache eviction
+        # is check-then-act and needs locking.
+        self._cache_lock = threading.Lock()
+        # Single-flight: one API fetch per (symbol, TF) at a time; other
+        # threads wait and then read the cache.
+        self._inflight: Dict[str, threading.Event] = {}
+
+    # ── Abstract: the only provider-specific piece ─────────────────
+
+    @abstractmethod
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        """Fetch up to `outputsize` bars, optionally bounded by start/end
+        (tz-aware UTC datetimes). Must return a normalized frame:
+        capitalized OHLCV columns, UTC DatetimeIndex named "Datetime",
+        oldest-first. Raise self.DATA_ERROR on failure."""
+
+    # ── Public API ─────────────────────────────────────────────────
+
+    def _live_outputsize(self, timeframe: Timeframe, lookback_days: int) -> int:
+        """Bars requested for one live fetch: the lookback in bars, floored
+        at MIN_LIVE_BARS (indicator seeding) and capped at the request cap."""
+        wanted = int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20
+        return max(self.MIN_LIVE_BARS, min(wanted, self.MAX_BARS_PER_REQUEST))
+
+    def fetch_ohlcv(
+        self,
+        symbol: str = XAUUSD,
+        timeframe: Timeframe = Timeframe.H1,
+        lookback_days: Optional[int] = None,
+    ) -> pd.DataFrame:
+        """Fetch OHLCV with in-memory caching for live mode."""
+        if lookback_days is None:
+            lookback_days = self.config.data_lookback_days
+
+        cache_key = f"{symbol}_{timeframe.value}"
+        outputsize = self._live_outputsize(timeframe, lookback_days)
+
+        while True:
+            now = time.time()
+            cache_ttl = self.CANDLE_SECONDS.get(
+                timeframe, self.config.refresh_interval)
+            with self._cache_lock:
+                cached = self._cache.get(cache_key)
+                cached_at = self._cache_timestamps.get(cache_key)
+                cached_size = self._cache_outputsize.get(cache_key, 0)
+            if (
+                cached is not None
+                and cached_at is not None
+                and (now - cached_at) < cache_ttl
+                and not cached.empty
+                # A caller asking for MORE history than the cached fetch
+                # requested must not be served the short frame silently.
+                and cached_size >= outputsize
+            ):
+                if (self._is_data_fresh(cached, symbol, timeframe)
+                        and self._has_latest_closed_bar(cached, timeframe)):
+                    return cached
+                logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
+                with self._cache_lock:
+                    self._cache.pop(cache_key, None)
+                    self._cache_timestamps.pop(cache_key, None)
+                    self._cache_outputsize.pop(cache_key, None)
+
+            # Single-flight: concurrent scans for the same (symbol, TF)
+            # must not each hit the API (the four TF loops fire together
+            # at aligned boundaries and Oanda has no client throttle).
+            # First caller claims the key and fetches; the rest wait,
+            # then re-check the cache.
+            with self._cache_lock:
+                inflight = self._inflight.get(cache_key)
+                if inflight is None:
+                    self._inflight[cache_key] = threading.Event()
+                    break
+            inflight.wait(timeout=90)
+
+        try:
+            df = self._fetch_raw(symbol, timeframe, outputsize)
+            df = self._drop_forming_candle(df, timeframe)
+            df = self._validate_and_clean(df, symbol, timeframe)
+
+            # The closed-bar guarantee must hold for FRESH responses too,
+            # not just cache hits: the scanner fires seconds after the
+            # boundary, and a lagging feed can answer without the
+            # just-closed bar — the scan would silently analyze (and
+            # session-check) the previous bar. Give the feed one short
+            # grace retry, then proceed loudly.
+            if not df.empty and not self._has_latest_closed_bar(df, timeframe):
+                time.sleep(self.FRESH_FETCH_RETRY_SECONDS)
+                try:
+                    retry = self._fetch_raw(symbol, timeframe, outputsize)
+                    retry = self._drop_forming_candle(retry, timeframe)
+                    retry = self._validate_and_clean(retry, symbol, timeframe)
+                    if retry is not None and not retry.empty:
+                        df = retry
+                except DataIntegrityError as e:
+                    logger.warning(
+                        f"{symbol} {timeframe.value}: boundary retry refetch "
+                        f"failed ({e}) — keeping original data")
+                if not self._has_latest_closed_bar(df, timeframe):
+                    logger.warning(
+                        f"{symbol} {timeframe.value}: feed lagging — just-closed bar "
+                        f"still missing after retry; this scan may run one bar behind"
+                    )
+
+            if not self._is_data_fresh(df, symbol, timeframe):
+                logger.warning(
+                    f"STALE DATA from API: {symbol} {timeframe.value} — "
+                    f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
+                )
+
+            with self._cache_lock:
+                self._cache[cache_key] = df
+                self._cache_timestamps[cache_key] = now
+                self._cache_outputsize[cache_key] = outputsize
+            return df
+        finally:
+            # Release waiters even when the fetch raised — they re-check
+            # the cache and (on a miss) claim the key themselves.
+            with self._cache_lock:
+                ev = self._inflight.pop(cache_key, None)
+            if ev is not None:
+                ev.set()
+
+    def fetch_range(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        start: datetime,
+        end: datetime,
+    ) -> pd.DataFrame:
+        """Fetch the RAW closed-bar series between start/end, paginating
+        under the per-request bar cap. Returns uncleaned bars (the
+        backtester resolves trades on the raw price path); callers wanting
+        the signal-side series run `validate_and_clean` on the result."""
+        bars_per_day = self.BARS_PER_DAY[timeframe]
+        days_per_chunk = max(1, int(4500 / bars_per_day))
+
+        chunks = []
+        cursor_end = end
+        empty_streak = 0
+        window_errors = 0
+        while cursor_end > start:
+            cursor_start = max(start, cursor_end - timedelta(days=days_per_chunk))
+            try:
+                chunk = self._fetch_raw(
+                    symbol, timeframe, self.MAX_BARS_PER_REQUEST,
+                    start=cursor_start, end=cursor_end,
+                )
+            except Exception as e:
+                # A raising chunk is a PROVIDER ERROR (auth, 429, 5xx,
+                # malformed payload) — not "no data". RETRY THE SAME
+                # WINDOW: until 2026-09-13 a failure advanced the cursor
+                # past the window, so one or two NON-consecutive page
+                # failures (Oanda's sporadic 401/5xx, even after the
+                # provider's own retry ladder) silently dropped up to
+                # days_per_chunk of bars — 15 days of M5 — out of the
+                # middle of the frame, and only the HEAD was
+                # coverage-checked below, so the backtester cached the hole
+                # as complete history forever. Three failures on the SAME
+                # window means it is unreachable: raise instead of
+                # returning a frame with a hole in it.
+                logger.warning(
+                    f"Chunk {cursor_start.date()}–{cursor_end.date()} failed: {e}"
+                )
+                window_errors += 1
+                if window_errors >= 3:
+                    raise self.DATA_ERROR(
+                        f"{symbol} {timeframe.value}: {window_errors} "
+                        f"consecutive failures fetching window "
+                        f"{cursor_start.date()}–{cursor_end.date()} — "
+                        f"refusing to return truncated history "
+                        f"(last error: {e})") from e
+                continue        # re-request the SAME window
+
+            window_errors = 0
+            if chunk is None or chunk.empty:
+                # Genuinely empty windows (weekends/holidays/asset gaps)
+                # are normal; several in a row = start of history.
+                empty_streak += 1
+                if empty_streak >= 3:
+                    break
+                cursor_end = cursor_start - timedelta(seconds=1)
+                continue
+
+            empty_streak = 0
+            chunks.append(chunk)
+            # Step back; dedup on concat.
+            new_cursor = chunk.index[0] - timedelta(seconds=1)
+            if new_cursor >= cursor_end:
+                # A provider ignoring start/end would pin the cursor and
+                # loop forever — refuse to spin.
+                logger.warning(
+                    f"{symbol} {timeframe.value}: pagination made no "
+                    f"progress at {cursor_end} — stopping")
+                break
+            cursor_end = new_cursor
+            if len(chunk) < 2:
+                break
+
+        if not chunks:
+            raise RuntimeError(f"No data fetched for {symbol} {timeframe.value}")
+
+        df = pd.concat(chunks).sort_index()
+        df = df[~df.index.duplicated(keep="last")]
+        # Fetches ending "now" include the still-forming candle — drop it
+        # so callers only ever see closed bars.
+        df = self._drop_forming_candle(df, timeframe)
+        start_ts = pd.Timestamp(start)
+        end_ts = pd.Timestamp(end)
+        df = df[(df.index >= start_ts) & (df.index <= end_ts)]
+        # Coverage sanity: a head materially later than the requested
+        # start means history was truncated (empty-streak bail or provider
+        # depth limit). Loud, not fatal — some assets genuinely start late.
+        if not df.empty and (df.index[0] - start_ts) > pd.Timedelta(days=7):
+            logger.error(
+                f"{symbol} {timeframe.value}: fetched history starts "
+                f"{df.index[0]} but {start_ts} was requested — "
+                f"~{(df.index[0] - start_ts).days} days missing from the "
+                f"head; treat downstream results accordingly")
+        # INTERIOR-gap audit (2026-09-13): the head check above is blind to
+        # a hole in the MIDDLE of the frame, which is exactly what a
+        # dropped page used to leave behind. Missing labels inside a
+        # scheduled closure (weekend, daily settlement pause) are normal,
+        # so only unscheduled runs are reported — and a real market holiday
+        # (Christmas, Good Friday, July 4) looks identical to a lost page,
+        # so this is LOUD BUT NOT FATAL and the frame is returned unchanged.
+        if len(df) > 400_000:
+            logger.info(
+                f"{symbol} {timeframe.value}: interior-gap audit skipped "
+                f"({len(df)} rows)")
+        elif len(df) > 1:
+            step = pd.Timedelta(seconds=self.CANDLE_SECONDS[timeframe])
+            labels = df.index
+            # Missing labels between each consecutive pair; >3 is the floor
+            # worth a look (weekends/settlement hours leave 1-2 labels that
+            # the NY-anchored closure rule does not cover).
+            spans = ((labels[1:] - labels[:-1]) / step).astype("int64") - 1
+            gaps = []
+            for i in (spans > 3).nonzero()[0]:
+                missing = sum(
+                    1 for k in range(1, int(spans[i]) + 1)
+                    if not _is_scheduled_closure(labels[i] + k * step))
+                if missing > 3:
+                    gaps.append((labels[i], labels[i + 1], missing))
+            if gaps:
+                logger.error(
+                    f"{symbol} {timeframe.value}: {len(gaps)} interior "
+                    f"gap(s) of >3 unscheduled bars in fetched history — "
+                    f"first: {gaps[0]}; a page fetch may have been dropped "
+                    f"or the market was closed (holiday); verify before "
+                    f"trusting this window")
+        return df
+
+    def is_data_fresh(
+        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
+    ) -> bool:
+        """Public freshness check (last bar within 3 candle durations)."""
+        return self._is_data_fresh(df, symbol, timeframe)
+
+    def validate_and_clean(
+        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
+    ) -> pd.DataFrame:
+        """Public entry to the shared cleaning pipeline."""
+        return self._validate_and_clean(df, symbol, timeframe)
+
+    def clear_cache(self) -> int:
+        """Wipe all cached data. Returns number of entries cleared."""
+        with self._cache_lock:
+            count = len(self._cache)
+            self._cache.clear()
+            self._cache_timestamps.clear()
+        logger.info(f"Cache cleared: {count} entries wiped")
+        return count
+
+    def get_pair_info(self, symbol: str = XAUUSD) -> Dict:
+        info = PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])
+        return {"symbol": symbol, **info}
+
+    def get_spread(self, symbol: str = XAUUSD) -> float:
+        """Latest observed round-trip spread; providers without live
+        bid/ask fall back to the pair's assumed default."""
+        return PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])["default_spread"]
+
+    # ── Streaming-quote capability (optional) ──────────────────────
+
+    #: True when the provider can stream live ticks via `iter_ticks`.
+    supports_streaming: bool = False
+
+    def iter_ticks(
+        self,
+        symbols: Sequence[str],
+        on_connect: Optional[Callable[[], None]] = None,
+    ) -> AsyncIterator[Tuple[str, float]]:
+        """Async generator yielding (canonical_symbol, price) ticks for one
+        connection's lifetime; the caller owns reconnect/backoff. Providers
+        that support it set `supports_streaming = True` and override."""
+        raise NotImplementedError(
+            f"{self.name} provider does not support streaming quotes")
+
+    # ── Shared internals ───────────────────────────────────────────
+
+    def _has_latest_closed_bar(self, df: pd.DataFrame, timeframe: Timeframe,
+                               now=None) -> bool:
+        """Cached data must include the most recent fully closed bar.
+
+        The TTL alone is wall-clock based: a snapshot fetched mid-candle is
+        still within TTL at the next candle boundary but is missing the bar
+        that just closed, so every boundary scan would analyze — and run the
+        bar-close session check against — the previous bar.
+        """
+        if df is None or df.empty:
+            return False
+        if now is None:
+            now = pd.Timestamp.now(tz="UTC")
+        if _is_forex_weekend(now):
+            return True
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        dur = pd.Timedelta(seconds=candle_seconds)
+        expected_label = now.floor(dur) - dur
+        # No bar ever carries a label inside a scheduled closure, so the
+        # newest bar that MUST be present is the last label before it.
+        # Without this walk-back every scan from 21:05 to 22:00 UTC in
+        # summer evicted the cache, refetched twice and logged a false
+        # "feed lagging" (2026-09-13 audit).
+        # A label counts as "inside" only when the closure covers the bar's
+        # whole span: a D1 bar labelled Sunday 00:00 UTC starts in the
+        # weekend but runs through the Sunday reopen, so it exists and
+        # must be present (68 such bars in the 3-year Oanda D1 cache).
+        steps = 0
+        last_second = dur - pd.Timedelta(seconds=1)
+        while (_is_scheduled_closure(expected_label)
+               and _is_scheduled_closure(expected_label + last_second)
+               and steps < 2000):
+            expected_label -= dur
+            steps += 1
+        return df.index[-1] >= expected_label
+
+    def _drop_forming_candle(self, df: pd.DataFrame, timeframe: Timeframe) -> pd.DataFrame:
+        """Drop the still-forming last candle so analysis only sees closed bars.
+
+        Some feeds include the current partial candle; the scanner fires
+        seconds after the boundary, so without this guard every indicator
+        reads a seconds-old stub bar and signals repaint. Feeds that only
+        return complete candles pass through unchanged.
+        """
+        if df is None or df.empty:
+            return df
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        now = pd.Timestamp.now(tz="UTC")
+        # Loop, not a single slice: a misbehaving feed can return MORE
+        # than one not-yet-closed bar, and a single iloc[:-1] would let
+        # the residual forming bar through cleaning (future-tolerance is
+        # +5 min) straight into the signal engine.
+        dropped = 0
+        while (not df.empty
+               and df.index[-1] + pd.Timedelta(seconds=candle_seconds) > now):
+            df = df.iloc[:-1]
+            dropped += 1
+        if dropped > 1:
+            logger.warning(
+                "%s: feed returned %d forming/future bars — all dropped",
+                timeframe.value, dropped)
+        return df
+
+    def _is_data_fresh(self, df: pd.DataFrame, symbol: str, timeframe: Timeframe,
+                       now=None) -> bool:
+        """Check if the last candle is within acceptable age for this timeframe."""
+        if df is None or df.empty:
+            return False
+
+        if now is None:
+            now = pd.Timestamp.now(tz="UTC")
+
+        if _is_forex_weekend(now):
+            return True
+
+        last_bar = df.index[-1]
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        max_age_seconds = candle_seconds * 3
+        # Scheduled closures (weekend, daily settlement pause) are dead
+        # time in which no bar can print — they must not count as age.
+        age_seconds = ((now - last_bar).total_seconds()
+                       - _scheduled_closed_seconds(last_bar, now))
+
+        if age_seconds > max_age_seconds:
+            logger.warning(
+                f"{symbol} {timeframe.value}: last bar {last_bar.strftime('%Y-%m-%d %H:%M')} "
+                f"is {age_seconds / 60:.0f}m old (limit: {max_age_seconds / 60:.0f}m)"
+            )
+            return False
+
+        return True
+
+    def _validate_and_clean(
+        self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
+    ) -> pd.DataFrame:
+        if df is None or df.empty:
+            raise self.DATA_ERROR(f"Empty dataset for {symbol}")
+
+        original_len = len(df)
+
+        df = df.dropna(subset=["Open", "High", "Low", "Close"], how="all")
+        df = df.ffill(limit=3)
+        df = df.dropna(subset=["Close"])
+
+        df = df.sort_index()
+        df = df[~df.index.duplicated(keep="last")]
+
+        now = pd.Timestamp.now(tz="UTC")
+        future_mask = df.index > now + pd.Timedelta(minutes=5)
+        if future_mask.any():
+            logger.warning(f"{symbol}: Dropping {future_mask.sum()} future timestamps")
+            df = df[~future_mask]
+
+        # Non-positive prices are corrupt, never real gold quotes. Reject
+        # the row BEFORE the consistency repair below — repairing High/Low
+        # from a corrupt 0.0 Open/Close would propagate the zero into
+        # previously-valid fields (2026-08-04 audit, I45).
+        nonpositive = (df[["Open", "High", "Low", "Close"]] <= 0).any(axis=1)
+        if nonpositive.any():
+            logger.warning(
+                f"{symbol}: Dropping {nonpositive.sum()} bars with "
+                f"non-positive prices (corrupt feed data)")
+            df = df[~nonpositive]
+            if df.empty:
+                raise self.DATA_ERROR(
+                    f"{symbol}: all bars had non-positive prices")
+
+        invalid_ohlc = (
+            (df["High"] < df["Low"]) |
+            (df["High"] < df["Open"]) |
+            (df["High"] < df["Close"]) |
+            (df["Low"] > df["Open"]) |
+            (df["Low"] > df["Close"])
+        )
+        if invalid_ohlc.any():
+            logger.warning(f"{symbol}: Fixing {invalid_ohlc.sum()} OHLC-inconsistent bars")
+            for idx in df[invalid_ohlc].index:
+                vals = [df.loc[idx, "Open"], df.loc[idx, "Close"]]
+                df.loc[idx, "High"] = max(vals + [df.loc[idx, "High"]])
+                df.loc[idx, "Low"]  = min(vals + [df.loc[idx, "Low"]])
+
+        # inf returns (division by a zero/na Close) would NaN the std and
+        # silently disarm the whole 10σ pass — neutralize them first.
+        returns = df["Close"].pct_change()
+        returns = returns.replace([float("inf"), float("-inf")], float("nan"))
+        std_ret = returns.std()
+        if std_ret > 0:
+            # A 10σ close-to-close move is NOT bad data by default. On
+            # three years of Oanda XAU bars every such bar was a real
+            # news/reopen move (NFP, CPI, Sunday gap — 2026-09-07 audit:
+            # 155 flagged M5 bars, 7 snapped back), and dropping them hid
+            # the most violent bar of the day from ATR/exhaustion math —
+            # while the backtester (whole-frame σ) dropped a DIFFERENT
+            # set than live (fetch-window σ). A bad print has the one
+            # signature a real move lacks: the NEXT close returns to the
+            # pre-spike level. Drop a candidate only on that observed
+            # snap-back; the newest bar (no successor yet) is kept as
+            # real, and the snap-back bar itself is never a candidate
+            # (its residual against the pre-spike close is small).
+            dev = returns - returns.mean()
+            candidate = dev.abs() > self.OUTLIER_SIGMA * std_ret
+            close = df["Close"]
+            spike = close - close.shift(1)
+            residual = close.shift(-1) - close.shift(1)
+            snapped_back = residual.notna() & (
+                residual.abs()
+                <= (1.0 - self.OUTLIER_REVERT_FRACTION) * spike.abs()
+            )
+            outlier_mask = candidate & snapped_back
+            if outlier_mask.any():
+                logger.warning(
+                    f"{symbol}: Removing {int(outlier_mask.sum())} bad-print "
+                    f"bar(s) (>{self.OUTLIER_SIGMA:.0f}σ move that snapped back)")
+                df = df[~outlier_mask]
+
+        min_required = max(
+            self.config.technical.ema_trend,
+            200,
+        ) + 10
+
+        if len(df) < min_required:
+            logger.warning(
+                f"{symbol} {timeframe.value}: Only {len(df)} bars "
+                f"(need {min_required}). Some indicators will be partial."
+            )
+
+        cleaned = original_len - len(df)
+        if cleaned > 0:
+            logger.info(f"{symbol}: Cleaned {cleaned}/{original_len} bars")
+
+        return df

providers/oanda.py

@@ -0,0 +1,257 @@
+"""
+ForexBot2 — Oanda v20 data provider
+====================================
+OHLCV candles from the Oanda v20 REST API. All shared behavior (cache,
+validation, boundary guarantees, range pagination) lives in
+`BaseDataProvider`; this module only maps symbols/granularities, parses
+Oanda's candle JSON, and exposes the real bid/ask spread from MBA candles.
+"""
+
+import logging
+import time
+from datetime import datetime
+from typing import Dict, Optional
+
+import pandas as pd
+import requests
+
+from app_config import oanda_api_token, oanda_base_url
+from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
+
+from providers.base import BaseDataProvider, DataIntegrityError
+
+logger = logging.getLogger(__name__)
+
+
+class OandaDataError(DataIntegrityError):
+    """Raised when an Oanda data fetch fails or returns invalid data."""
+
+
+class OandaDataProvider(BaseDataProvider):
+    """Oanda v20 REST candle provider."""
+
+    name = "oanda"
+    DATA_ERROR = OandaDataError
+    # Transient-fault retry ladder (see _fetch_oanda_candles): sleeps
+    # between attempts, plus a wall-clock cap on the whole sequence.
+    RETRY_DELAYS_S = (2.0, 5.0, 10.0)
+    RETRY_BUDGET_S = 45.0
+    RETRY_DELAY_S = RETRY_DELAYS_S[0]  # first delay; kept for callers/tests
+
+    OANDA_GRANULARITY = {
+        Timeframe.M1: "M1",
+        Timeframe.M5: "M5",
+        Timeframe.M15: "M15",
+        Timeframe.M30: "M30",
+        Timeframe.H1: "H1",
+        Timeframe.H4: "H4",
+        Timeframe.D1: "D",
+        Timeframe.W1: "W",
+    }
+
+    OANDA_INSTRUMENT = {
+        "XAU/USD": "XAU_USD",
+    }
+
+    def __init__(
+        self,
+        config: PlatformConfig,
+        api_token: Optional[str] = None,
+        base_url: Optional[str] = None,
+    ):
+        super().__init__(config)
+        # Credentials default to the config.json `oanda` block / env vars
+        # so the provider registry can build this class with config only.
+        if api_token is None:
+            api_token = oanda_api_token()
+        if base_url is None:
+            base_url = oanda_base_url()
+        self._base_url = base_url.rstrip("/")
+        self._session = requests.Session()
+        self._session.headers.update({
+            "Authorization": f"Bearer {api_token}",
+            "Accept-Datetime-Format": "RFC3339",
+            "Content-Type": "application/json",
+        })
+        self._spread_cache: Dict[str, float] = {}
+
+    # ── Public extras ──────────────────────────────────────────────
+
+    def get_spread(self, symbol: str = XAUUSD) -> float:
+        return self._spread_cache.get(
+            symbol, PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])["default_spread"])
+
+    # ── Raw fetch ──────────────────────────────────────────────────
+
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        if start is not None or end is not None:
+            return self._fetch_oanda_candles(
+                symbol, timeframe, outputsize, start, end)
+        return self._fetch_oanda_candles(symbol, timeframe, outputsize)
+
+    def _to_oanda_instrument(self, symbol: str) -> str:
+        try:
+            return self.OANDA_INSTRUMENT[symbol]
+        except KeyError:
+            raise OandaDataError(
+                f"Unsupported symbol {symbol!r} for Oanda") from None
+
+    def _fetch_oanda_candles(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        count: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        instrument = self._to_oanda_instrument(symbol)
+        granularity = self.OANDA_GRANULARITY.get(timeframe)
+        if granularity is None:
+            raise OandaDataError(
+                f"Unsupported timeframe {timeframe} for Oanda")
+
+        url = f"{self._base_url}/v3/instruments/{instrument}/candles"
+        params = {
+            "granularity": granularity,
+            "price": "MBA",
+            "dailyAlignment": 0,
+            "alignmentTimezone": "Etc/UTC",
+        }
+        # v20 rejects `count` combined with both `from` and `to`; a date
+        # window replaces it (used by fetch_range pagination).
+        if start is not None and end is not None:
+            params["from"] = pd.Timestamp(start).isoformat()
+            params["to"] = pd.Timestamp(end).isoformat()
+        else:
+            params["count"] = min(count, self.MAX_BARS_PER_REQUEST)
+
+        # Retry ladder for transient faults: Oanda's sporadic spurious
+        # 401s (~15-20/day on the practice API — the same token succeeds
+        # on adjacent requests), 5xx and network errors. Other 4xx (bad
+        # request, 429) are never retried; a real credential problem
+        # fails every attempt and raises as before. The 2026-09-07
+        # signal-log audit found 11 boundary scans in two days that lost
+        # their primary frame at xx:00:22-:26 UTC: a single 2 s retry
+        # could not outlast a 10-15 s provider blip at the top of the
+        # hour, and a scan without its primary frame logs a price-0
+        # NEUTRAL row. RETRY_BUDGET_S bounds the whole sequence so a scan
+        # (three fetches) still finishes well inside an M5 bar.
+        last_err: Optional[OandaDataError] = None
+        resp = None
+        attempts = 1 + len(self.RETRY_DELAYS_S)
+        started = time.monotonic()
+        for attempt in range(attempts):
+            if attempt:
+                delay = self.RETRY_DELAYS_S[attempt - 1]
+                if time.monotonic() - started + delay > self.RETRY_BUDGET_S:
+                    logger.warning(
+                        f"Oanda fetch for {instrument}: retry budget "
+                        f"({self.RETRY_BUDGET_S:.0f}s) exhausted after "
+                        f"{attempt} attempt(s) — giving up")
+                    break
+                time.sleep(delay)
+            try:
+                resp = self._session.get(url, params=params, timeout=30)
+            except requests.RequestException as e:
+                last_err = OandaDataError(
+                    f"Oanda candle fetch failed for {instrument}: {e}")
+                last_err.__cause__ = e
+                resp = None
+                logger.warning(
+                    f"Oanda fetch for {instrument} attempt "
+                    f"{attempt + 1}/{attempts} failed: {e}")
+                continue
+            if resp.status_code == 401 or resp.status_code >= 500:
+                last_err = OandaDataError(
+                    f"Oanda candle fetch failed for {instrument}: "
+                    f"HTTP {resp.status_code}: {resp.text[:300]}")
+                logger.warning(
+                    f"Oanda fetch for {instrument} attempt "
+                    f"{attempt + 1}/{attempts} got HTTP {resp.status_code}")
+                resp = None
+                continue
+            break
+        if resp is None:
+            raise last_err
+
+        if resp.status_code != 200:
+            raise OandaDataError(
+                f"Oanda candle fetch failed for {instrument}: "
+                f"HTTP {resp.status_code}: {resp.text[:300]}")
+
+        try:
+            data = resp.json()
+        except ValueError as e:
+            raise OandaDataError(
+                f"Oanda candle response for {instrument} is not valid "
+                f"JSON: {e}") from e
+        candles = data.get("candles", [])
+        if not candles:
+            raise OandaDataError(
+                f"Empty candle response from Oanda for {instrument}")
+
+        rows = []
+        last_bid_close = None
+        last_ask_close = None
+        skipped_malformed = 0
+        for c in candles:
+            if not c.get("complete", False):
+                continue
+            mid = c.get("mid", {})
+            bid = c.get("bid", {})
+            ask = c.get("ask", {})
+            # A candle missing any OHLC field must be SKIPPED, never
+            # defaulted to 0.0: a zero price flows through cleaning as a
+            # "legit" bar and poisons ATR/Keltner/SL math — and a
+            # mid-frame zero Close makes pct_change() emit inf, which
+            # NaNs the std and disarms the 10σ outlier filter for the
+            # whole frame (2026-08-04 audit, I45).
+            if any(mid.get(k) is None for k in ("o", "h", "l", "c")):
+                skipped_malformed += 1
+                continue
+            rows.append({
+                "Datetime": pd.Timestamp(c["time"]).tz_convert("UTC"),
+                "Open": float(mid["o"]),
+                "High": float(mid["h"]),
+                "Low": float(mid["l"]),
+                "Close": float(mid["c"]),
+                "Volume": int(c.get("volume", 0)),
+            })
+            if bid and ask:
+                last_bid_close = float(bid.get("c", 0))
+                last_ask_close = float(ask.get("c", 0))
+        if skipped_malformed:
+            logger.warning(
+                "Oanda returned %d malformed candle(s) for %s (missing "
+                "OHLC fields) — skipped", skipped_malformed, instrument)
+
+        if not rows:
+            raise OandaDataError(
+                f"No complete candles from Oanda for {instrument}")
+
+        if last_bid_close and last_ask_close:
+            self._spread_cache[symbol] = round(
+                last_ask_close - last_bid_close, 4)
+
+        df = pd.DataFrame(rows)
+        df = df.set_index("Datetime")
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        df.index.name = "Datetime"
+
+        for col in ("Open", "High", "Low", "Close"):
+            df[col] = pd.to_numeric(df[col], errors="coerce")
+        df["Volume"] = pd.to_numeric(df["Volume"], errors="coerce").fillna(0).astype(float)
+
+        logger.debug(
+            "Fetched %d bars for %s @ %s from Oanda",
+            len(df), instrument, granularity)
+        return df

providers/tiingo.py

@@ -0,0 +1,311 @@
+"""
+ForexBot2 — Tiingo data provider
+=================================
+OHLCV candles from Tiingo's FX endpoint
+(`/tiingo/fx/{ticker}/prices`, resampled mid-price bars, no volume).
+Tiingo is streaming-only in production: its REST candles remain available
+for comparison research, but it is not a designated candle fallback.
+All shared behavior (cache, validation, boundary guarantees, range
+pagination) lives in `BaseDataProvider`; this module only maps
+symbols/resample frequencies, sizes the request window, and parses
+Tiingo's JSON into the normalized frame.
+
+Every failure mode raises `TiingoDataError` (a `DataIntegrityError`)
+with a specific message — the scan loops log it and carry on; nothing
+fails silently. NOTE: candles and mid-price ticks only. Tiingo's free
+`/tiingo/fx/top` quote endpoint publishes indicative two-tier spreads
+and must not be used for spread display (removed 2026-07-23) —
+`get_spread` deliberately stays on the base-class default, and the
+WebSocket stream (`iter_ticks`) exposes the mid price only, never
+bid/ask or a derived spread.
+"""
+
+import json
+import logging
+from datetime import datetime
+from typing import AsyncIterator, Callable, Optional, Sequence, Tuple
+
+import pandas as pd
+import requests
+
+from app_config import provider_api_key
+from config import PlatformConfig, Timeframe
+
+from providers.base import BaseDataProvider, DataIntegrityError
+
+logger = logging.getLogger(__name__)
+
+
+class TiingoDataError(DataIntegrityError):
+    """Raised when a Tiingo data fetch fails or returns invalid data."""
+
+
+class TiingoDataProvider(BaseDataProvider):
+    """Tiingo FX REST candle provider."""
+
+    name = "tiingo"
+    DATA_ERROR = TiingoDataError
+
+    BASE_URL = "https://api.tiingo.com"
+
+    TIINGO_RESAMPLE = {
+        Timeframe.M1: "1min",
+        Timeframe.M5: "5min",
+        Timeframe.M15: "15min",
+        Timeframe.M30: "30min",
+        Timeframe.H1: "1hour",
+        Timeframe.H4: "4hour",
+        Timeframe.D1: "1day",
+        # W1 intentionally unmapped: the FX endpoint has no weekly
+        # resample — requesting it raises loudly instead of guessing.
+    }
+
+    TIINGO_TICKER = {
+        "XAU/USD": "xauusd",
+    }
+
+    # Window padding over the theoretical bar span: FX is closed ~2/7 of
+    # the week (Fri 17:00 → Sun 18:00 New York; see providers/base.py), so
+    # a calendar window must be wider than `bars × candle_seconds` to yield
+    # the requested count.
+    WINDOW_PAD_FACTOR = 1.6
+
+    def __init__(self, config: PlatformConfig, api_key: Optional[str] = None):
+        super().__init__(config)
+        # Credentials default to env TIINGO_API_KEY → config.json
+        # data_providers.tiingo.api_key, so the provider registry can
+        # build this class with config only.
+        if api_key is None:
+            api_key = provider_api_key(provider="tiingo")
+        self._api_key = api_key or None
+        if self._api_key is None:
+            logger.warning(
+                "Tiingo provider built WITHOUT an API key — every fetch "
+                "will fail with a named error. Set the TIINGO_API_KEY env "
+                "var or data_providers.tiingo.api_key in config.json.")
+        self._session = requests.Session()
+        self._session.headers.update({
+            "Content-Type": "application/json",
+            # Header auth keeps the key out of URLs (and out of logs).
+            "Authorization": f"Token {api_key}" if api_key else "",
+        })
+
+    def _to_tiingo_ticker(self, symbol: str) -> str:
+        try:
+            return self.TIINGO_TICKER[symbol]
+        except KeyError:
+            raise TiingoDataError(
+                f"Unsupported symbol {symbol!r} for Tiingo") from None
+
+    # ── Raw fetch ──────────────────────────────────────────────────
+
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        if self._api_key is None:
+            raise TiingoDataError(
+                "No Tiingo API key. Set the TIINGO_API_KEY env var or "
+                "data_providers.tiingo.api_key in config.json.")
+
+        resample = self.TIINGO_RESAMPLE.get(timeframe)
+        if resample is None:
+            raise TiingoDataError(
+                f"Unsupported timeframe {timeframe} for Tiingo "
+                f"(no FX resample frequency)")
+
+        ticker = self._to_tiingo_ticker(symbol)
+
+        # Tiingo takes date windows, not bar counts: derive the start
+        # from the requested count for live fetches.
+        if start is None:
+            span = self.CANDLE_SECONDS[timeframe] * (outputsize + 10)
+            start_ts = (pd.Timestamp.now(tz="UTC")
+                        - pd.Timedelta(seconds=span * self.WINDOW_PAD_FACTOR))
+        else:
+            start_ts = pd.Timestamp(start)
+
+        params = {
+            "resampleFreq": resample,
+            "startDate": start_ts.strftime("%Y-%m-%dT%H:%M:%S"),
+        }
+        if end is not None:
+            params["endDate"] = pd.Timestamp(end).strftime("%Y-%m-%dT%H:%M:%S")
+
+        url = f"{self.BASE_URL}/tiingo/fx/{ticker}/prices"
+        try:
+            resp = self._session.get(url, params=params, timeout=30)
+        except requests.RequestException as e:
+            raise TiingoDataError(
+                f"Tiingo candle fetch failed for {ticker}: {e}") from e
+
+        if resp.status_code == 429:
+            raise TiingoDataError(
+                f"Tiingo rate limit hit for {ticker} (HTTP 429) — "
+                f"plan quota exhausted; scan will retry next boundary")
+        if resp.status_code in (401, 403):
+            raise TiingoDataError(
+                f"Tiingo auth failed for {ticker} (HTTP {resp.status_code}) "
+                f"— check TIINGO_API_KEY / config.json api_key")
+        if resp.status_code != 200:
+            raise TiingoDataError(
+                f"Tiingo candle fetch failed for {ticker}: "
+                f"HTTP {resp.status_code}: {resp.text[:300]}")
+
+        try:
+            data = resp.json()
+        except ValueError as e:
+            raise TiingoDataError(
+                f"Tiingo candle response for {ticker} is not valid "
+                f"JSON: {e}") from e
+
+        if not isinstance(data, list) or not data:
+            raise TiingoDataError(
+                f"Empty candle response from Tiingo for {ticker} "
+                f"(window {params['startDate']} → "
+                f"{params.get('endDate', 'now')}; check ticker and plan "
+                f"history limits)")
+
+        rows = []
+        for item in data:
+            if "date" not in item:
+                raise TiingoDataError(
+                    f"Malformed Tiingo candle for {ticker}: missing "
+                    f"'date' in {str(item)[:120]}")
+            rows.append({
+                "Datetime": pd.Timestamp(item["date"]),
+                "Open": item.get("open"),
+                "High": item.get("high"),
+                "Low": item.get("low"),
+                "Close": item.get("close"),
+                # FX endpoint has no volume; 0.0 keeps the engine contract.
+                "Volume": 0.0,
+            })
+
+        df = pd.DataFrame(rows)
+        df = df.set_index("Datetime")
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        else:
+            df.index = df.index.tz_convert("UTC")
+        df.index.name = "Datetime"
+
+        for col in ("Open", "High", "Low", "Close"):
+            df[col] = pd.to_numeric(df[col], errors="coerce")
+        if df["Close"].isna().all():
+            raise TiingoDataError(
+                f"Tiingo response for {ticker} has no numeric closes — "
+                f"feed format changed?")
+
+        logger.debug(
+            "Fetched %d bars for %s @ %s from Tiingo",
+            len(df), ticker, resample)
+        return df
+
+    # ── Streaming quotes ───────────────────────────────────────────
+
+    supports_streaming = True
+
+    WS_URL = "wss://api.tiingo.com/fx"
+
+    @staticmethod
+    def _parse_ws_message(raw: str) -> Optional[Tuple[str, float]]:
+        """Parse one WS message; return (tiingo_ticker, mid_price) for
+        quote updates, None for heartbeats ("H"), subscription acks
+        ("I"), malformed JSON, or quotes without a usable price.
+
+        Server error messages ("E", e.g. bad token) RAISE TiingoDataError:
+        Tiingo authenticates inside the socket via the subscribe message,
+        so an auth failure arrives as a data message followed by a clean
+        close — swallowing it would make the caller's reconnect loop
+        hammer the endpoint with no backoff (backoff only grows on
+        exceptions).
+        """
+        try:
+            msg = json.loads(raw)
+        except json.JSONDecodeError:
+            logger.warning("Tiingo WS: malformed JSON, skipping")
+            return None
+
+        mtype = msg.get("messageType")
+        if mtype == "H":
+            return None
+        if mtype == "I":
+            logger.info("Tiingo WS: subscription acknowledged")
+            return None
+        if mtype == "E":
+            raise TiingoDataError(
+                f"Tiingo WS server error: {str(msg.get('response'))[:200]}")
+        if mtype != "A":
+            return None
+
+        data = msg.get("data")
+        if not isinstance(data, list) or len(data) < 8 or data[0] != "Q":
+            return None
+        # Live-verified field order (2026-08-03): ["Q", ticker, timestamp,
+        # bidSize, bid, mid, askSize, ask] — ask is index 7, NOT 6 as the
+        # docs' table suggests (index 6 is askSize).
+        ticker = data[1]
+        bid, mid, ask = data[4], data[5], data[7]
+        if mid is None and bid is not None and ask is not None:
+            mid = (bid + ask) / 2
+        try:
+            return ticker, float(mid)
+        except (TypeError, ValueError):
+            logger.warning("Tiingo WS: quote for %s has no usable price",
+                           ticker)
+            return None
+
+    async def iter_ticks(
+        self,
+        symbols: Sequence[str],
+        on_connect: Optional[Callable[[], None]] = None,
+    ) -> AsyncIterator[Tuple[str, float]]:
+        """One connection's lifetime of (canonical_symbol, mid) ticks.
+
+        The caller owns the reconnect/backoff loop around this generator.
+        Divergences from the Twelve Data reference implementation, both
+        deliberate: `on_connect` fires on the FIRST RECEIVED message (the
+        "I" ack in practice) rather than after the subscribe send, so a
+        rejected token cannot reset the caller's backoff; and there is no
+        client heartbeat task — Tiingo sends "H" frames itself and the
+        websockets library's built-in ping keepalive detects dead sockets.
+        """
+        import websockets
+
+        if self._api_key is None:
+            raise TiingoDataError(
+                "No Tiingo API key. Set the TIINGO_API_KEY env var or "
+                "data_providers.tiingo.api_key in config.json.")
+
+        ws_to_pair = {self._to_tiingo_ticker(s): s for s in symbols}
+        async with websockets.connect(self.WS_URL) as ws:
+            await ws.send(json.dumps({
+                "eventName": "subscribe",
+                "authorization": self._api_key,
+                "eventData": {
+                    "thresholdLevel": 5,
+                    "tickers": sorted(ws_to_pair),
+                },
+            }))
+            logger.info(
+                "Tiingo WS connected, subscribing to %d tickers",
+                len(ws_to_pair))
+            connected = False
+            async for raw in ws:
+                parsed = self._parse_ws_message(raw)
+                if not connected:
+                    connected = True
+                    if on_connect is not None:
+                        on_connect()
+                if parsed is None:
+                    continue
+                pair_symbol = ws_to_pair.get(parsed[0])
+                if pair_symbol is None:
+                    continue
+                yield pair_symbol, parsed[1]

providers/twelve_data.py

@@ -0,0 +1,247 @@
+"""
+ForexBot2 — Twelve Data provider
+=================================
+OHLCV via the official `twelvedata` SDK plus streaming quotes over the
+Twelve Data WebSocket. All shared behavior (cache, validation, boundary
+guarantees, range pagination) lives in `BaseDataProvider`.
+"""
+
+import asyncio
+import json
+import logging
+import threading
+import time
+from collections import deque
+from datetime import datetime
+from typing import AsyncIterator, Callable, Optional, Sequence, Tuple
+
+import pandas as pd
+from twelvedata import TDClient
+
+from app_config import provider_api_key
+from config import PAIR_INFO, PlatformConfig, Timeframe
+
+from providers.base import BaseDataProvider, DataIntegrityError
+
+logger = logging.getLogger(__name__)
+
+
+class TwelveDataProvider(BaseDataProvider):
+    """Twelve Data SDK client with credit throttling and WS quotes."""
+
+    name = "twelve_data"
+
+    TD_INTERVALS = {
+        Timeframe.M1:  "1min",
+        Timeframe.M5:  "5min",
+        Timeframe.M15: "15min",
+        Timeframe.M30: "30min",
+        Timeframe.H1:  "1h",
+        Timeframe.H4:  "4h",
+        Timeframe.D1:  "1day",
+        Timeframe.W1:  "1week",
+    }
+
+    RATE_LIMIT = 55
+    RATE_WINDOW = 60.0
+
+    WS_URL = "wss://ws.twelvedata.com/v1/quotes/price?apikey={api_key}"
+
+    def __init__(self, config: PlatformConfig):
+        super().__init__(config)
+        self._api_key = provider_api_key(provider="twelve_data") or None
+        self._throttle_lock = threading.Lock()
+        self._request_times: deque = deque()
+        self._td: Optional[TDClient] = (
+            TDClient(apikey=self._api_key) if self._api_key else None
+        )
+
+    @staticmethod
+    def _to_td_symbol(symbol: str) -> str:
+        if "/" not in symbol:
+            raise DataIntegrityError(
+                f"Unsupported symbol {symbol!r} for Twelve Data")
+        return symbol
+
+    # ── Raw fetch ──────────────────────────────────────────────────
+
+    def _fetch_raw(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start: Optional[datetime] = None,
+        end: Optional[datetime] = None,
+    ) -> pd.DataFrame:
+        return self._fetch_twelve_data(
+            symbol, timeframe, outputsize,
+            start_date=start.strftime("%Y-%m-%d %H:%M:%S") if start else None,
+            end_date=end.strftime("%Y-%m-%d %H:%M:%S") if end else None,
+        )
+
+    def _throttle(self, credits: int = 1) -> None:
+        """Pro tier: 55 credits / minute. Block until budget allows."""
+        while True:
+            with self._throttle_lock:
+                now = time.time()
+                while self._request_times and now - self._request_times[0] >= self.RATE_WINDOW:
+                    self._request_times.popleft()
+
+                available = self.RATE_LIMIT - len(self._request_times)
+                if available >= credits:
+                    for _ in range(credits):
+                        self._request_times.append(time.time())
+                    return
+
+                used = len(self._request_times)
+                wait = (self._request_times[0] + self.RATE_WINDOW) - now + 0.2
+            logger.info(
+                f"Rate limit: {used}/{self.RATE_LIMIT} credits used "
+                f"— waiting {wait:.1f}s..."
+            )
+            time.sleep(wait)
+
+    def _fetch_twelve_data(
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start_date: Optional[str] = None,
+        end_date: Optional[str] = None,
+    ) -> pd.DataFrame:
+        if self._td is None:
+            raise DataIntegrityError(
+                "No data-provider API key. Set the TWELVE_DATA_API_KEY env "
+                "var or data_providers.twelve_data.api_key in config.json."
+            )
+
+        td_symbol = self._to_td_symbol(symbol)
+        interval = self.TD_INTERVALS[timeframe]
+
+        self._throttle(credits=1)
+
+        kwargs = {
+            "symbol": td_symbol,
+            "interval": interval,
+            "outputsize": min(outputsize, self.MAX_BARS_PER_REQUEST),
+            "timezone": "UTC",
+        }
+        if start_date:
+            kwargs["start_date"] = start_date
+        if end_date:
+            kwargs["end_date"] = end_date
+
+        try:
+            df = self._td.time_series(**kwargs).as_pandas()
+        except Exception as e:
+            raise DataIntegrityError(
+                f"Twelve Data fetch failed for {td_symbol}: {e}"
+            ) from e
+
+        if df is None or df.empty:
+            raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
+
+        # SDK returns lowercase columns; engine expects capitalised.
+        df = df.rename(columns={
+            "open": "Open", "high": "High", "low": "Low",
+            "close": "Close", "volume": "Volume",
+        })
+        for col in ("Open", "High", "Low", "Close", "Volume"):
+            if col in df.columns:
+                df[col] = pd.to_numeric(df[col], errors="coerce")
+        if "Volume" not in df.columns:
+            df["Volume"] = 0.0
+
+        # SDK returns newest-first; engine + indicators want oldest-first.
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        df.index.name = "Datetime"
+
+        logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
+        return df
+
+    # ── Streaming quotes ───────────────────────────────────────────
+
+    supports_streaming = True
+
+    @staticmethod
+    def _parse_ws_price(raw: str) -> Optional[Tuple[str, float]]:
+        """Parse one WS message; return (ws_symbol, price) for price
+        events, None for everything else (logged where noteworthy)."""
+        try:
+            msg = json.loads(raw)
+        except json.JSONDecodeError:
+            logger.warning("WebSocket: malformed JSON, skipping")
+            return None
+
+        event = msg.get("event")
+        if event == "subscribe-status":
+            fails = msg.get("fails") or []
+            if fails:
+                logger.warning(f"WebSocket: failed to subscribe: {fails}")
+            return None
+        if event != "price":
+            return None
+
+        ws_symbol = msg.get("symbol")
+        try:
+            price = float(msg["price"])
+        except (KeyError, ValueError, TypeError) as e:
+            logger.warning(f"WebSocket: bad price data for {ws_symbol}: {e}")
+            return None
+        return ws_symbol, price
+
+    @staticmethod
+    async def _ws_heartbeat(ws) -> None:
+        """Twelve Data drops idle WS connections; heartbeat every 10s keeps
+        it alive through quiet periods (weekends, low-liquidity hours)."""
+        while True:
+            await asyncio.sleep(10)
+            await ws.send(json.dumps({"action": "heartbeat"}))
+
+    async def iter_ticks(
+        self,
+        symbols: Sequence[str],
+        on_connect: Optional[Callable[[], None]] = None,
+    ) -> AsyncIterator[Tuple[str, float]]:
+        """One connection's lifetime of (canonical_symbol, price) ticks.
+
+        The TD WS speaks display symbols ("XAU/USD"); ticks are mapped back
+        to the canonical pair symbols passed in. The caller owns the
+        reconnect/backoff loop around this generator.
+
+        `on_connect` fires on the FIRST RECEIVED frame, not after the
+        subscribe send (2026-08-04 audit H38b, mirroring the Tiingo
+        provider) — firing pre-receive let a rejected API key reset the
+        caller's reconnect backoff on every attempt, pinning it at the
+        minimum forever.
+        """
+        import websockets
+
+        ws_to_pair = {PAIR_INFO[s]["display"]: s for s in symbols}
+        url = self.WS_URL.format(api_key=self._api_key or "")
+        async with websockets.connect(url) as ws:
+            subscribe = {
+                "action": "subscribe",
+                "params": {"symbols": ",".join(ws_to_pair)},
+            }
+            await ws.send(json.dumps(subscribe))
+            logger.info(f"WebSocket connected, subscribed to {len(ws_to_pair)} symbols")
+            hb_task = asyncio.create_task(self._ws_heartbeat(ws))
+            connected = False
+            try:
+                async for raw in ws:
+                    if not connected:
+                        connected = True
+                        if on_connect is not None:
+                            on_connect()
+                    parsed = self._parse_ws_price(raw)
+                    if parsed is None:
+                        continue
+                    pair_symbol = ws_to_pair.get(parsed[0])
+                    if pair_symbol is None:
+                        continue
+                    yield pair_symbol, parsed[1]
+            finally:
+                hb_task.cancel()

requirements.txt

@@ -0,0 +1,6 @@
+numpy>=1.24,<3
+pandas>=2.0,<4
+requests>=2.28
+twelvedata>=1.2.0
+python-telegram-bot>=20.0
+websockets>=12.0

script.txt

@@ -1 +0,0 @@
-pip install pandas numpy requests python-dateutil -q

scripts/backfill_signal_outcomes.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+"""
+backfill_signal_outcomes.py — seed and advance advisory outcomes for every
+logged trade setup (not only panel rows), one per advisory event.
+
+Pass A (offline, no network): assign setup_key / advisory_event_id to all
+legacy setup rows, mark same-setup re-logs within 60 min as DUPLICATE, and
+seed a PENDING outcome on each canonical row that has none. Existing
+(panel-seeded) outcomes are never overwritten.
+
+Pass B: advance PENDING rows from the closed-candle streams already stored
+in the DB (union across every snapshot of the timeframe — no provider).
+
+Pass C (--reseed-unresolvable): bounded re-seed of continuity-broken rows,
+then Pass B again.
+
+Safe beside the live writers: short per-row transactions with a busy
+timeout. Read-only preview with --dry-run.
+
+    python scripts/backfill_signal_outcomes.py --dry-run
+    python scripts/backfill_signal_outcomes.py
+    python scripts/backfill_signal_outcomes.py --reseed-unresolvable
+"""
+import argparse
+import os
+import sqlite3
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import app_config  # noqa: E402
+from signal_log import SignalLogger  # noqa: E402
+
+
+def _status_counts(con: sqlite3.Connection) -> dict:
+    return dict(con.execute(
+        "SELECT COALESCE(advisory_outcome_status, 'NULL'), COUNT(*)"
+        " FROM signals WHERE setup_json IS NOT NULL AND setup_json != 'null'"
+        " GROUP BY 1").fetchall())
+
+
+def main(argv=None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[1])
+    ap.add_argument("--db", default=None, help="defaults to the signal_log.path setting in config.json")
+    ap.add_argument("--dry-run", action="store_true")
+    ap.add_argument("--reseed-unresolvable", action="store_true")
+    ap.add_argument("--symbol", default="XAU/USD")
+    args = ap.parse_args(argv)
+    path = args.db or app_config.signal_log_path()
+    if not os.path.exists(path):
+        print(f"no database at {path}")
+        return 1
+    if args.dry_run:
+        con = sqlite3.connect(f"file:{os.path.abspath(path)}?mode=ro", uri=True)
+        print("setup rows by outcome status:", _status_counts(con))
+        have = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
+        if "setup_key" in have:
+            n_keyless = con.execute(
+                "SELECT COUNT(*) FROM signals WHERE setup_json IS NOT NULL"
+                " AND setup_json != 'null' AND setup_key IS NULL").fetchone()[0]
+            print(f"setup rows without setup_key: {n_keyless}")
+        else:
+            print("setup_key column not present yet (schema migrates on"
+                  " the first non-dry run)")
+        con.close()
+        return 0
+    lg = SignalLogger(path)
+    try:
+        print("before:", _status_counts(lg._conn))
+        print("pass A (seed/dedup):", lg.backfill_seeds())
+        total = 0
+        for _ in range(50):
+            n = lg.resolve_from_store(args.symbol)
+            total += n
+            if n == 0:
+                break
+        print(f"pass B (advance from stored bars): {total} rows changed")
+        if args.reseed_unresolvable:
+            reseeded = lg.reseed_unresolvable()
+            print(f"pass C (re-seed): {len(reseeded)} rows re-seeded")
+            for _ in range(50):
+                if lg.resolve_from_store(args.symbol) == 0:
+                    break
+        print("after:", _status_counts(lg._conn))
+    finally:
+        lg.close()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/ff_calendar_scraper.py

@@ -0,0 +1,144 @@
+"""Forex Factory calendar scraper -> backtest/cache/calendar.parquet.
+
+Fetches weekly calendar pages and parses the event JSON embedded in
+`window.calendarComponentStates` (the `days:` array). Event `dateline`
+is a unix epoch, so timestamps carry no timezone ambiguity — verified
+2026-07-11 against known release times (NFP 12:30/13:30 UTC across DST,
+ECB 12:15/13:15) and against EUR/USD M15 volatility: the bar containing
+a high-impact event averages 3.5x normal range, and shifting event
+times by ±1h destroys that alignment.
+
+Usage:
+    python scripts/ff_calendar_scraper.py --start 2024-01-01 --end 2026-07-13
+    python scripts/ff_calendar_scraper.py            # refresh: last stored week -> today
+
+Merges into the existing parquet on event (id, dateline), so refreshing
+is idempotent and also picks up revised `actual` values for recent weeks.
+Be polite: ~1.6s between requests (a full year is ~52 requests).
+"""
+import argparse
+import json
+import time
+from datetime import date, timedelta
+from pathlib import Path
+from typing import List, Optional
+
+import pandas as pd
+import requests
+
+CACHE_DIR = Path(__file__).resolve().parents[1] / "backtest" / "cache"
+PARQUET = CACHE_DIR / "calendar.parquet"
+
+UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
+      "(KHTML, like Gecko) Chrome/126.0 Safari/537.36")
+
+FIELDS = ["id", "dateline", "name", "country", "currency", "impactName",
+          "timeLabel", "timeMasked", "actual", "forecast", "previous",
+          "revision", "actualBetterWorse"]
+
+
+def week_url(d: date) -> str:
+    return (f"https://www.forexfactory.com/calendar?week="
+            f"{d.strftime('%b').lower()}{d.day}.{d.year}")
+
+
+def extract_days(html: str) -> Optional[list]:
+    """Balanced-bracket extraction of the `days:` JSON array."""
+    i = html.find("days:")
+    if i < 0:
+        return None
+    j = html.find("[", i)
+    depth, in_str, esc = 0, False, False
+    for k in range(j, len(html)):
+        ch = html[k]
+        if in_str:
+            if esc:
+                esc = False
+            elif ch == "\\":
+                esc = True
+            elif ch == '"':
+                in_str = False
+        else:
+            if ch == '"':
+                in_str = True
+            elif ch == "[":
+                depth += 1
+            elif ch == "]":
+                depth -= 1
+                if depth == 0:
+                    return json.loads(html[j:k + 1])
+    return None
+
+
+def fetch_week(sess: requests.Session, d: date, retries: int = 4) -> Optional[List[dict]]:
+    for attempt in range(retries):
+        try:
+            r = sess.get(week_url(d), timeout=30)
+            if r.status_code == 200:
+                days = extract_days(r.text)
+                if days is not None:
+                    return [{k: e.get(k) for k in FIELDS}
+                            for day in days for e in day.get("events", [])]
+        except requests.RequestException:
+            pass
+        time.sleep(5 * (attempt + 1))
+    return None
+
+
+def scrape(start: date, end: date) -> pd.DataFrame:
+    sess = requests.Session()
+    sess.headers["User-Agent"] = UA
+    rows: List[dict] = []
+    failed: List[date] = []
+    d = start
+    while d <= end:
+        week_rows = fetch_week(sess, d)
+        if week_rows is None:
+            failed.append(d)
+            print(f"FAILED week {d}")
+        else:
+            rows.extend(week_rows)
+        d += timedelta(days=7)
+        time.sleep(1.6)
+    if failed:
+        print(f"WARNING: {len(failed)} weeks failed: {failed}")
+    return pd.DataFrame(rows)
+
+
+def merge_into_store(new: pd.DataFrame, path: Path = PARQUET) -> pd.DataFrame:
+    if path.exists():
+        old = pd.read_parquet(path)
+        combined = pd.concat([old, new])
+    else:
+        combined = new
+    # keep="last": a refresh overwrites stale rows (actuals fill in
+    # after release, revisions land weeks later).
+    combined = (combined.drop_duplicates(subset=["id", "dateline"], keep="last")
+                .sort_values("dateline").reset_index(drop=True))
+    combined.to_parquet(path, index=False)
+    return combined
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--start", type=date.fromisoformat, default=None)
+    ap.add_argument("--end", type=date.fromisoformat, default=date.today())
+    args = ap.parse_args()
+
+    start = args.start
+    if start is None:
+        if PARQUET.exists():
+            last = pd.read_parquet(PARQUET).dateline.max()
+            start = pd.Timestamp(last, unit="s").date() - timedelta(days=7)
+        else:
+            start = date(2024, 1, 1)
+    print(f"Scraping {start} -> {args.end}")
+    new = scrape(start, args.end)
+    combined = merge_into_store(new)
+    print(f"Store: {len(combined)} events "
+          f"({pd.Timestamp(combined.dateline.min(), unit='s')} -> "
+          f"{pd.Timestamp(combined.dateline.max(), unit='s')}) -> {PARQUET}")
+
+
+if __name__ == "__main__":
+    main()

scripts/label_scan_outcomes.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""
+label_scan_outcomes.py — batch driver for the scan_outcomes table.
+
+Labels model-free forward returns (1/4/24 h, $ and ATR-normalized, plus
+MFE/MAE, a 3xATR burst flag and a completeness flag) for every logged scan
+whose 24 h window has closed, from the closed-candle streams already
+stored in the DB. No provider, no network.
+
+The live loggers do the same work opportunistically at the end of every
+scan (time-boxed to ~2 s); this script is the catch-up pass for a backlog
+— a freshly imported database, or rows whose bars only arrived later.
+
+Idempotent: a row is rewritten only when its numbers change, so a settled
+database costs one read pass. Safe beside the live writers (short per-row
+transactions with a busy timeout). Read-only preview with --dry-run.
+
+    python scripts/label_scan_outcomes.py --dry-run
+    python scripts/label_scan_outcomes.py
+    python scripts/label_scan_outcomes.py --db logs/signals.db --budget 5000
+"""
+import argparse
+import os
+import sqlite3
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import app_config  # noqa: E402
+from signal_log import SCAN_LABEL_MIN_AGE_H, SignalLogger  # noqa: E402
+
+CHUNK = 2000
+
+
+def _counts(con: sqlite3.Connection, symbol: str,
+            labelled: bool = True) -> dict:
+    """Candidate / labelled / complete / pending counts (read-only).
+
+    The age filter uses logged_at (always an ISO string) as a proxy for
+    the bar close — the labeller itself parses the real bar label."""
+    scans = ("SELECT COUNT(*) FROM signals s WHERE s.symbol = ?"
+             " AND s.indicators_json IS NOT NULL AND s.price > 0")
+    due = (scans + " AND s.logged_at <"
+           f" datetime('now', '-{int(SCAN_LABEL_MIN_AGE_H)} hours')")
+    out = {
+        "scan rows with indicators": con.execute(scans, (symbol,)).fetchone()[0],
+        "of those, window elapsed": con.execute(due, (symbol,)).fetchone()[0],
+    }
+    if labelled:
+        out["labelled"] = con.execute(
+            "SELECT COUNT(*) FROM scan_outcomes").fetchone()[0]
+        out["complete"] = con.execute(
+            "SELECT COUNT(*) FROM scan_outcomes WHERE complete = 1"
+        ).fetchone()[0]
+        out["pending (labelled, incomplete)"] = con.execute(
+            "SELECT COUNT(*) FROM scan_outcomes WHERE complete = 0"
+        ).fetchone()[0]
+    return out
+
+
+def main(argv=None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[1])
+    ap.add_argument("--db", default=None, help="defaults to the signal_log.path setting in config.json")
+    ap.add_argument("--dry-run", action="store_true",
+                    help="read-only candidate counts, writes nothing")
+    ap.add_argument("--budget", type=int, default=100000,
+                    help="maximum rows to write in this run")
+    ap.add_argument("--symbol", default="XAU/USD")
+    args = ap.parse_args(argv)
+    path = args.db or app_config.signal_log_path()
+    if not os.path.exists(path):
+        print(f"no database at {path}")
+        return 1
+
+    if args.dry_run:
+        con = sqlite3.connect(f"file:{os.path.abspath(path)}?mode=ro", uri=True)
+        try:
+            have = {r[0] for r in con.execute(
+                "SELECT name FROM sqlite_master WHERE type='table'")}
+            if "scan_outcomes" not in have:
+                print("scan_outcomes table not present yet (created on the"
+                      " first non-dry run / next logged scan)")
+            for key, value in _counts(con, args.symbol,
+                                      labelled="scan_outcomes" in have).items():
+                print(f"  {key}: {value}")
+        finally:
+            con.close()
+        return 0
+
+    lg = SignalLogger(path)
+    try:
+        print("before:", _counts(lg._conn, args.symbol))
+        written = 0
+        previous_complete = -1
+        while written < args.budget:
+            n = lg.label_scan_outcomes(
+                budget=min(CHUNK, args.budget - written), time_budget=120.0)
+            written += n
+            complete = lg._conn.execute(
+                "SELECT COUNT(*) FROM scan_outcomes WHERE complete = 1"
+            ).fetchone()[0]
+            # Stop on a pass that wrote nothing, or one that could not
+            # complete a single further row (only re-labelling stragglers).
+            if n == 0 or complete == previous_complete:
+                break
+            previous_complete = complete
+        print(f"labelled (rows written): {written}")
+        print("after:", _counts(lg._conn, args.symbol))
+    finally:
+        lg.close()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

static/dashboard.css

@@ -0,0 +1,351 @@
+/* Gold Desk — XAU/USD dashboard styles
+ *
+ * Builds on Bootstrap 5.3 color modes (data-bs-theme). Only app-specific
+ * tokens and components live here; layout, typography, tables, badges,
+ * cards, alerts, modals, etc. come from Bootstrap.
+ */
+
+:root {
+  --xau-font-sans: "Inter", system-ui, -apple-system, "Segoe UI", sans-serif;
+  --xau-font-mono: "JetBrains Mono", ui-monospace, "SF Mono", Consolas, Menlo, monospace;
+  --xau-gold: #f5b301;
+}
+
+/* Forced dark theme (owner decision 2026-08-16): dark background, pure
+ * white text (no gray), vivid red/green for data accents. Loaded after
+ * Bootstrap so these overrides win. */
+[data-bs-theme="dark"] {
+  --bs-body-bg: #0a0e14;
+  --bs-secondary-bg: #111823;
+  --bs-tertiary-bg: #0e141d;
+  --bs-body-color: #ffffff;
+  --bs-emphasis-color: #ffffff;
+  --bs-heading-color: #ffffff;
+  --bs-secondary-color: rgba(255, 255, 255, 0.92);
+  --bs-tertiary-color: rgba(255, 255, 255, 0.8);
+  --bs-border-color: rgba(255, 255, 255, 0.14);
+  --bs-card-bg: #0e141d;
+  --bs-card-border-color: rgba(255, 255, 255, 0.12);
+  --bs-modal-bg: #0e141d;
+  --bs-success: #22c55e;
+  --bs-success-rgb: 34, 197, 94;
+  --bs-success-text-emphasis: #4ade80;
+  --bs-success-bg-subtle: rgba(34, 197, 94, 0.16);
+  --bs-success-border-subtle: rgba(34, 197, 94, 0.45);
+  --bs-danger: #ef4444;
+  --bs-danger-rgb: 239, 68, 68;
+  --bs-danger-text-emphasis: #f87171;
+  --bs-danger-bg-subtle: rgba(239, 68, 68, 0.16);
+  --bs-danger-border-subtle: rgba(239, 68, 68, 0.45);
+}
+
+html, body {
+  font-family: var(--xau-font-sans);
+  letter-spacing: -0.005em;
+  -webkit-font-smoothing: antialiased;
+  background-color: var(--bs-body-bg);
+  color: var(--bs-body-color);
+}
+
+.num, .xau-price, .xau-symbol {
+  font-family: var(--xau-font-mono);
+  font-variant-numeric: tabular-nums;
+}
+
+.xau-symbol {
+  font-size: 0.95rem;
+  font-weight: 700;
+  letter-spacing: 0.08em;
+  color: var(--xau-gold);
+}
+
+.xau-price {
+  line-height: 1;
+  color: #ffffff;
+}
+
+.live-dot {
+  display: inline-block;
+  width: 8px;
+  height: 8px;
+  border-radius: 50%;
+  background: var(--bs-success);
+  margin-right: 5px;
+  animation: pulse 2.4s ease-in-out infinite;
+}
+
+.live-dot.closed {
+  background: var(--bs-warning);
+  animation: none;
+}
+
+@keyframes pulse {
+  50% { opacity: 0.35; }
+}
+
+/* signal-coloured accents */
+.sig-buy   { --xau-signal: var(--bs-success); }
+.sig-sell  { --xau-signal: var(--bs-danger); }
+.sig-neutral { --xau-signal: var(--bs-secondary); }
+
+.border-signal {
+  border-left: 3px solid var(--xau-signal, var(--bs-secondary)) !important;
+}
+
+.text-signal {
+  color: var(--xau-signal, var(--bs-secondary)) !important;
+}
+
+.tf-name {
+  font-family: var(--xau-font-mono);
+  font-weight: 700;
+  font-size: 1.1rem;
+  color: var(--bs-primary);
+  letter-spacing: 0.05em;
+}
+
+.tf-selector {
+  display: grid;
+  grid-template-columns: repeat(4, minmax(0, 1fr));
+  gap: 0.5rem;
+  margin-bottom: 0.75rem;
+}
+
+.tf-select {
+  min-width: 0;
+  padding: 0.7rem 0.8rem;
+  border: 1px solid var(--bs-border-color);
+  border-radius: var(--bs-border-radius);
+  background: var(--bs-tertiary-bg);
+  color: var(--bs-body-color);
+  text-align: left;
+  transition: border-color 0.15s ease, background-color 0.15s ease, box-shadow 0.15s ease;
+}
+
+.tf-select:hover {
+  border-color: rgba(var(--bs-primary-rgb), 0.65);
+}
+
+.tf-select[aria-pressed="true"] {
+  border-color: var(--xau-gold);
+  background: rgba(245, 179, 1, 0.09);
+  box-shadow: inset 0 0 0 1px rgba(245, 179, 1, 0.16);
+}
+
+.tf-select .tf-name {
+  font-size: 1rem;
+}
+
+.workspace-card {
+  overflow: hidden;
+}
+
+.workspace-signal {
+  font-family: var(--xau-font-mono);
+  font-size: clamp(1.55rem, 3vw, 2.35rem);
+  font-weight: 700;
+  letter-spacing: 0.03em;
+  line-height: 1;
+}
+
+.workspace-meta {
+  min-width: 0;
+}
+
+.workspace-reasons > div + div {
+  margin-top: 0.35rem;
+}
+
+/* score bar: zero-centered diverging with midpoint tick */
+.scorebar {
+  display: flex;
+  align-items: center;
+  gap: 0.75rem;
+}
+
+.scorebar .gauge-container {
+  position: relative;
+  flex: 1;
+  min-width: 60px;
+  height: 10px;
+  border-radius: 5px;
+  background: var(--bs-secondary-bg);
+  overflow: hidden;
+}
+
+.scorebar .gauge-container::after {
+  content: "";
+  position: absolute;
+  left: 50%;
+  top: 0;
+  bottom: 0;
+  width: 2px;
+  background: var(--bs-secondary);
+  transform: translateX(-1px);
+}
+
+.scorebar .fill {
+  position: absolute;
+  top: 0;
+  bottom: 0;
+  border-radius: 4px;
+  width: var(--score-width);
+  transition: width 0.25s ease;
+}
+
+.scorebar .fill.pos {
+  left: 50%;
+  background: var(--bs-success);
+}
+
+.scorebar .fill.neg {
+  right: 50%;
+  background: var(--bs-danger);
+}
+
+.scorebar .val {
+  width: 58px;
+  text-align: right;
+  font-size: 0.95rem;
+  font-weight: 500;
+}
+
+.scorebar .lbl {
+  font-size: 0.8rem;
+  color: var(--bs-secondary);
+  width: 44px;
+}
+
+/* rail chips in hero */
+.rail-chip {
+  border: 1px solid var(--bs-border-color);
+  border-radius: var(--bs-border-radius);
+  background: var(--bs-tertiary-bg);
+  color: var(--bs-body-color);
+  font-family: var(--xau-font-mono);
+  font-size: 0.9rem;
+  line-height: 1.3;
+  padding: 0.3rem 0.5rem;
+}
+
+.rail-chip:hover, .rail-chip[aria-current="true"] {
+  border-color: var(--xau-gold);
+}
+
+.rail-chip:focus-visible, .tf-select:focus-visible, .wire-item:focus-visible,
+.cal-event:focus-visible {
+  outline: 2px solid var(--xau-gold);
+  outline-offset: 2px;
+}
+
+/* detail dialog sections */
+.d-stat .k {
+  display: block;
+  font-size: 0.7rem;
+  letter-spacing: 0.08em;
+  text-transform: uppercase;
+  color: var(--bs-secondary);
+  font-weight: 600;
+}
+
+.d-stat .v {
+  font-size: 1.15rem;
+  font-weight: 600;
+}
+
+.d-table {
+  font-size: 0.9rem;
+}
+
+.detail-table {
+  overflow-x: auto;
+}
+
+.d-table .ind-k {
+  font-family: var(--xau-font-mono);
+  font-weight: 600;
+  white-space: nowrap;
+}
+
+.wbar {
+  display: block;
+  height: 4px;
+  border-radius: 2px;
+  background: var(--bs-secondary-bg);
+  margin-top: 3px;
+}
+
+.wbar > i {
+  display: block;
+  height: 100%;
+  border-radius: 2px;
+  background: var(--bs-primary);
+}
+
+/* news */
+.wire-item {
+  display: flex;
+  width: 100%;
+  border: 0;
+  border-radius: var(--bs-border-radius);
+  background: transparent;
+  color: inherit;
+  cursor: pointer;
+  font: inherit;
+  text-align: start;
+}
+
+.wire-item:hover {
+  background: rgba(var(--bs-primary-rgb), 0.08);
+}
+
+.cal-event {
+  display: block;
+  width: 100%;
+  padding: 0;
+  border: 0;
+  background: transparent;
+  color: inherit;
+  text-align: inherit;
+}
+
+.cal-event:hover {
+  color: var(--bs-primary);
+  text-decoration: underline;
+}
+
+.wire-score {
+  font-family: var(--xau-font-mono);
+  font-size: 0.85rem;
+  font-weight: 700;
+  padding: 2px 7px;
+  border-radius: 5px;
+  white-space: nowrap;
+}
+
+/* calendar highlight for next upcoming event */
+.cal-next-up td:first-child {
+  box-shadow: inset 4px 0 0 var(--bs-primary);
+  padding-left: 0.75rem !important;
+}
+
+/* reduced motion */
+@media (prefers-reduced-motion: reduce) {
+  .live-dot { animation: none; }
+  .scorebar .fill { transition: none; }
+  * { transition: none !important; }
+}
+
+@media (max-width: 575.98px) {
+  .tf-selector {
+    grid-template-columns: repeat(2, minmax(0, 1fr));
+  }
+
+  .workspace-card .card-body {
+    padding: 1rem;
+  }
+
+  .rail-chip {
+    flex: 1 1 calc(50% - 0.5rem);
+  }
+}

static/dashboard.html

@@ -0,0 +1,149 @@
+<!DOCTYPE html>
+<html lang="en" data-bs-theme="dark">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>Gold Desk — XAU/USD</title>
+<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3E%3Ccircle cx='8' cy='8' r='7' fill='%23f5b301'/%3E%3C/svg%3E">
+<link rel="preconnect" href="https://fonts.googleapis.com">
+<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
+<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css" crossorigin="anonymous">
+<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.3/font/bootstrap-icons.css" crossorigin="anonymous">
+<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;700&display=swap">
+<link rel="stylesheet" href="dashboard.css">
+</head>
+<body>
+
+<div class="container-fluid container-xl py-4">
+
+  <header class="card mb-3" id="hero">
+    <div class="card-body d-flex align-items-center flex-wrap gap-3">
+      <div>
+        <h1 class="xau-symbol mb-1"><i class="bi bi-coin me-1"></i>XAU/USD</h1>
+        <div class="display-5 fw-semibold xau-price" id="hero-price">—</div>
+      </div>
+      <div class="d-flex flex-wrap gap-2" id="hero-rail" aria-label="Select timeframe"></div>
+      <div class="ms-auto text-end">
+        <div id="hero-status" role="status"><span class="live-dot"></span>…</div>
+        <div class="small text-secondary"><i class="bi bi-hourglass-split me-1"></i>M5 close in <span class="num" id="hero-cd">--:--</span></div>
+        <div class="small text-secondary" id="hero-updated">updated —</div>
+      </div>
+    </div>
+  </header>
+
+  <div class="alert alert-warning d-none" id="banner-closed" role="alert">
+    <i class="bi bi-moon-fill me-2"></i>Market closed (weekend) — showing Friday's last state. Calendar and news stay live.
+  </div>
+  <div class="alert alert-danger d-none" id="banner-fail" role="alert"></div>
+  <div class="visually-hidden" id="dashboard-live" aria-live="polite"></div>
+
+  <main>
+    <section class="card mb-4 d-none" id="context" aria-label="Market context">
+      <div class="card-body d-flex flex-wrap gap-3 align-items-start">
+        <div class="flex-grow-1" id="ctx-blurb"><strong>Market context</strong></div>
+        <div class="d-flex flex-wrap gap-2" id="ctx-chips"></div>
+      </div>
+    </section>
+
+    <section class="mb-4" aria-labelledby="workspace-heading">
+      <h2 class="text-uppercase text-secondary small fw-bold mb-3 d-flex align-items-center gap-2" id="workspace-heading">
+        <i class="bi bi-crosshair text-primary"></i>
+        Signal workspace
+        <span class="fw-normal text-body-secondary" style="text-transform:none">— select a timeframe</span>
+      </h2>
+      <div id="tf-grid"></div>
+    </section>
+
+    <section aria-labelledby="fundamentals-heading">
+      <h2 class="text-uppercase text-secondary small fw-bold mb-3 mt-4 d-flex align-items-center gap-2" id="fundamentals-heading">
+        <i class="bi bi-globe2 text-primary"></i>Fundamentals
+      </h2>
+      <div class="row g-3">
+        <div class="col-lg-8">
+          <div class="card h-100">
+            <div class="card-body">
+              <div class="d-flex align-items-center gap-3 mb-3 flex-wrap">
+                <h3 class="h5 card-title mb-0"><i class="bi bi-calendar-event me-2 text-primary"></i>Economic calendar</h3>
+                <span class="text-secondary small" id="calendar-note">times local · USD events</span>
+                <div class="form-check form-switch ms-auto">
+                  <input class="form-check-input" type="checkbox" id="hi-only">
+                  <label class="form-check-label small" for="hi-only">high impact only</label>
+                </div>
+              </div>
+              <div class="table-responsive">
+                <table class="table table-sm">
+                  <thead>
+                    <tr>
+                      <th>Time</th>
+                      <th>Cur</th>
+                      <th>Impact</th>
+                      <th>Event</th>
+                      <th class="text-end">Fcst</th>
+                      <th class="text-end">Prev</th>
+                      <th class="text-end">Actual</th>
+                    </tr>
+                  </thead>
+                  <tbody id="cal-body"><tr><td colspan="7" class="text-secondary text-center py-3">Loading calendar…</td></tr></tbody>
+                </table>
+              </div>
+            </div>
+          </div>
+        </div>
+        <div class="col-lg-4">
+          <div class="card h-100">
+            <div class="card-body">
+              <div class="d-flex align-items-center gap-3 mb-3 flex-wrap">
+                <h3 class="h5 card-title mb-0"><i class="bi bi-newspaper me-2 text-primary"></i>Newswire</h3>
+                <span class="text-secondary small" id="wire-note"></span>
+              </div>
+              <div id="wire-total" class="d-none"></div>
+              <div id="wire"></div>
+            </div>
+          </div>
+        </div>
+      </div>
+    </section>
+  </main>
+
+  <footer class="mt-4 pt-3 border-top text-secondary small">
+    <i class="bi bi-info-circle me-1"></i>
+    Signals are generated on closed candles only and refresh at candle boundaries
+    (M5/M15/H1/H4). Not financial advice — educational and research use only.
+  </footer>
+
+</div>
+
+<div class="modal fade" id="tf-detail" tabindex="-1" aria-labelledby="detail-title" aria-hidden="true">
+  <div class="modal-dialog modal-xl modal-dialog-scrollable">
+    <div class="modal-content">
+      <div class="modal-header">
+        <div>
+          <h5 class="modal-title mb-0" id="detail-title">—</h5>
+          <div class="small text-secondary">
+            <span class="num" id="detail-price">—</span>
+            <span id="detail-sub"></span>
+          </div>
+        </div>
+        <button id="detail-close" type="button" class="btn-close" aria-label="Close detail view"></button>
+      </div>
+      <div class="modal-body" id="detail-body"></div>
+    </div>
+  </div>
+</div>
+
+<div class="modal fade" id="news-detail" tabindex="-1" aria-labelledby="nd-title" aria-hidden="true">
+  <div class="modal-dialog modal-lg modal-dialog-scrollable">
+    <div class="modal-content">
+      <div class="modal-header">
+        <h5 class="modal-title" id="nd-title">News detail</h5>
+        <button id="nd-close" type="button" class="btn-close" aria-label="Close news detail"></button>
+      </div>
+      <div class="modal-body" id="nd-body"></div>
+    </div>
+  </div>
+</div>
+
+<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js" crossorigin="anonymous"></script>
+<script src="dashboard.js"></script>
+</body>
+</html>

static/dashboard.js

@@ -0,0 +1,968 @@
+"use strict";
+/* Gold Desk — XAU/USD dashboard frontend
+ *
+ * Reads snapshot.json + news.json published by dashboard_publisher.py and
+ * renders a responsive, keyboard-accessible dashboard. Uses Bootstrap 5 for
+ * layout/components/modals and a small custom CSS file for app-specific
+ * visuals (score bar, signal colours, mono fonts).
+ *
+ * This file is intentionally self-contained: it mirrors a few constants from
+ * the Python signal pipeline so the HTML does not depend on telegram_bot.
+ */
+
+const IND_ORDER = ["ema_trend", "rsi", "macd", "bollinger", "adx", "roc", "keltner", "divergence", "candlestick"];
+const IND_SHORT = {
+  ema_trend: "EMA", rsi: "RSI", macd: "MACD", bollinger: "BB",
+  adx: "ADX", roc: "ROC", keltner: "KC", divergence: "Div", candlestick: "Cndl"
+};
+const ARROW = { STRONG_BUY: "⇈", BUY: "↑", NEUTRAL: "→", SELL: "↓", STRONG_SELL: "⇊" };
+const TF_LABEL = { "5min": "M5", "15min": "M15", "1h": "H1", "4h": "H4" };
+const TF_ORDER = ["5min", "15min", "1h", "4h"];
+const TF_SECONDS = { "5min": 300, "15min": 900, "1h": 3600, "4h": 14400 };
+const FALLBACK_POLL_MS = 60000;
+const RETRY_MS = 15000;
+const HIGH_IMPACT_PREF = "goldDeskHighImpactOnly";
+const TIMEFRAME_PREF = "goldDeskSelectedTimeframe";
+
+const LEVEL_LABEL = {
+  ema_fast: "EMA fast", ema_medium: "EMA medium", ema_slow: "EMA slow", ema_trend: "EMA trend",
+  bb_upper: "Bollinger upper", bb_mid: "Bollinger mid", bb_lower: "Bollinger lower",
+  kc_upper: "Keltner upper", kc_mid: "Keltner mid", kc_lower: "Keltner lower",
+  swing_high_1: "Swing high 1", swing_high_2: "Swing high 2", swing_high_3: "Swing high 3",
+  swing_low_1: "Swing low 1", swing_low_2: "Swing low 2", swing_low_3: "Swing low 3"
+};
+
+let snapshot = null, news = null;
+let clockOffset = 0;
+let lastPollOk = null;
+let lastSnapText = null, lastNewsText = null;
+let refetchTimer = null;
+let fallbackTimer = null;
+let pollInFlight = false;
+let pollQueued = false;
+let selectedTf = "5min";
+
+let detailTf = null;
+let ndState = null;
+const aiMemo = new Map();
+const aiErr = new Map();
+const aiPending = new Set();
+
+let tfModal = null;
+let newsModal = null;
+let tfModalOpen = false;
+let newsModalOpen = false;
+
+const $ = id => document.getElementById(id);
+
+/* ── helpers ──────────────────────────────────────────────────── */
+const esc = s => String(s ?? "").replace(/[&<>"']/g,
+  c => ({"&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;", "'": "&#39;"}[c]));
+
+const fmt$ = v => v == null ? "—" : Number(v).toLocaleString("en-US",
+  { minimumFractionDigits: 2, maximumFractionDigits: 2 });
+
+const sigClass = s => s?.includes("BUY") ? "buy" : s?.includes("SELL") ? "sell" : "neutral";
+const sigLabel = s => (s || "NEUTRAL").replace("_", " ");
+const nowServer = () => Date.now() / 1000 + clockOffset;
+
+function localGet(key) {
+  try { return window.localStorage.getItem(key); } catch { return null; }
+}
+
+function localSet(key, value) {
+  try { window.localStorage.setItem(key, value); } catch { /* storage is optional */ }
+}
+
+function announce(message) {
+  const live = $("dashboard-live");
+  if (live) live.textContent = message;
+}
+
+function fmtCountdown(secs) {
+  if (secs == null || !isFinite(secs)) return "--:--";
+  secs = Math.max(0, Math.floor(secs));
+  const h = Math.floor(secs / 3600), m = Math.floor((secs % 3600) / 60), s = secs % 60;
+  const mm = String(m).padStart(2, "0"), ss = String(s).padStart(2, "0");
+  return h > 0 ? `${h}:${mm}:${ss}` : `${mm}:${ss}`;
+}
+
+function fmtAge(iso) {
+  if (!iso) return "";
+  const secs = nowServer() - Date.parse(iso) / 1000;
+  if (secs < 90) return `${Math.max(0, Math.round(secs))}s ago`;
+  if (secs < 5400) return `${Math.round(secs / 60)}m ago`;
+  if (secs < 172800) return `${Math.round(secs / 3600)}h ago`;
+  return `${Math.round(secs / 86400)}d ago`;
+}
+
+const signed = (v, dp = 2) => v == null ? "—" : (v >= 0 ? "+" : "") + Number(v).toFixed(dp);
+const badgeCls = s => s === "buy" ? "bg-success-subtle text-success-emphasis border-success-subtle"
+                 : s === "sell" ? "bg-danger-subtle text-danger-emphasis border-danger-subtle"
+                 : "bg-secondary-subtle text-secondary-emphasis border-secondary-subtle";
+
+/* ── theme ────────────────────────────────────────────────────── */
+// Dark theme is forced site-wide (owner decision 2026-08-16): dark
+// background, pure-white text, red/green data accents.
+document.documentElement.setAttribute("data-bs-theme", "dark");
+
+/* ── hero ─────────────────────────────────────────────────────── */
+function renderHero() {
+  if (!snapshot) return;
+  let price = null;
+  for (const tf of TF_ORDER) {
+    const p = snapshot.timeframes[tf]?.report?.current_price;
+    if (p) { price = p; break; }
+  }
+  $("hero-price").textContent = price ? fmt$(price) : "—";
+
+  $("hero-rail").innerHTML = TF_ORDER.map(tf => {
+    const r = snapshot.timeframes[tf]?.report;
+    const sig = r?.signal || null;
+    const cls = sigClass(sig);
+    const arrow = sig ? (ARROW[sig] || "→") : "·";
+    const score = r?.score != null ? r.score.toFixed(0) : "–";
+    const textColor = cls === "buy" ? "text-success" : cls === "sell" ? "text-danger" : "text-secondary";
+    return `<button type="button" class="rail-chip" data-select-tf="${tf}"
+      aria-current="${tf === selectedTf ? "true" : "false"}"
+      title="${esc(sigLabel(sig))} (score ${esc(score)}) — select ${TF_LABEL[tf]} workspace">
+      <span class="tf-name">${TF_LABEL[tf]}</span>
+      <span class="${textColor} fw-bold">${arrow}</span>
+      <span>${esc(score)}</span>
+    </button>`;
+  }).join("");
+
+  const closed = snapshot.market_closed;
+  $("hero-status").innerHTML =
+    `<span class="live-dot${closed ? " closed" : ""}"></span>${closed ? "CLOSED" : "OPEN"}`;
+  $("banner-closed").classList.toggle("d-none", !closed);
+}
+
+/* ── market context strip ─────────────────────────────────────── */
+function renderContext() {
+  if (!news) return;
+  $("context").classList.remove("d-none");
+  $("ctx-blurb").innerHTML = news.llm_summary
+    ? `<strong>Market context:</strong> ${esc(news.llm_summary)}`
+    : `<strong>Market context:</strong> <span class="text-secondary">no summary available</span>`;
+
+  let chips = "";
+  for (const cur of ["XAU", "USD"]) {
+    const v = news.sentiment?.[cur];
+    if (v == null) continue;
+    const cls = v > 0.05 ? "text-bg-success" : v < -0.05 ? "text-bg-danger" : "text-bg-secondary";
+    chips += `<span class="badge ${cls}" title="Mean headline sentiment for ${cur} (−1…+1)">
+      ${cur} news <span class="num">${v >= 0 ? "+" : ""}${v.toFixed(2)}</span></span>`;
+  }
+  let risk = "clear", riskDesc = "No significant events";
+  if (snapshot) {
+    for (const tf of TF_ORDER) {
+      const r = snapshot.timeframes[tf]?.report;
+      if (r?.event_risk && r.event_risk !== "clear") { risk = r.event_risk; riskDesc = r.event_description; break; }
+      if (r?.event_description) riskDesc = r.event_description;
+    }
+  }
+  if (news.high_impact_warning) { risk = "danger"; riskDesc = news.warning_message || riskDesc; }
+  const riskCls = risk === "clear" ? "text-bg-success"
+                : risk === "danger" ? "text-bg-danger"
+                : "text-bg-warning";
+  chips += `<span class="badge ${riskCls}" title="${esc(riskDesc)}">event risk: ${risk.toUpperCase()}</span>`;
+  $("ctx-chips").innerHTML = chips;
+}
+
+/* ── timeframe cards ──────────────────────────────────────────── */
+function scoreBar(label, score) {
+  const v = Math.max(-100, Math.min(100, score ?? 0));
+  const width = (Math.abs(v) / 2).toFixed(1);
+  return `<div class="scorebar">
+    <span class="lbl">${label}</span>
+    <div class="gauge-container" aria-hidden="true">
+      <i class="fill ${v >= 0 ? "pos" : "neg"}" style="--score-width:${width}%"></i>
+    </div>
+    <span class="val num ${v > 0 ? "text-success" : v < 0 ? "text-danger" : ""}">${v >= 0 ? "+" : ""}${v.toFixed(1)}</span>
+  </div>`;
+}
+
+function setupCard(ts) {
+  if (!ts) return "";
+  const dir = ts.direction === "BUY" ? "buy" : "sell";
+  const dirIcon = ts.direction === "BUY" ? "bi-arrow-up-circle-fill" : "bi-arrow-down-circle-fill";
+  const trail = (ts.trail_arm_r != null && ts.trail_dist_r != null)
+    ? `arm ${ts.trail_arm_r}R · dist ${ts.trail_dist_r}R` : "platform default";
+  return `<div class="card border-signal sig-${dir} bg-body-tertiary">
+    <div class="card-body py-2 px-3">
+      <div class="d-flex align-items-baseline gap-2 mb-2">
+        <span class="fw-bold text-signal sig-${dir}"><i class="bi ${dirIcon} me-1"></i>${ts.direction}</span>
+        <span class="small text-secondary font-monospace">${esc(ts.strategy || "")}</span>
+        <span class="small text-secondary ms-auto font-monospace">R:R ${(+ts.risk_reward).toFixed(1)}</span>
+      </div>
+      <div class="table-responsive"><table class="table table-sm table-borderless mb-0">
+        <tbody><tr><td class="text-secondary"><i class="bi bi-box-arrow-in-right me-1"></i>Entry</td><td class="num">${fmt$(ts.entry)}</td>
+            <td class="text-secondary ps-4"><i class="bi bi-flag-fill me-1 text-success"></i>TP1</td><td class="num text-success">${fmt$(ts.take_profit_1)}</td></tr>
+        <tr><td class="text-secondary"><i class="bi bi-shield-x me-1 text-danger"></i>Stop</td><td class="num text-danger">${fmt$(ts.stop_loss)}</td>
+            <td class="text-secondary ps-4"><i class="bi bi-flag-fill me-1 text-success"></i>TP2</td><td class="num text-success">${fmt$(ts.take_profit_2)}</td></tr>
+        <tr><td class="text-secondary"><i class="bi bi-signpost-split me-1"></i>Trail</td><td class="num" colspan="3">${esc(trail)}</td></tr></tbody>
+      </table></div>
+    </div>
+  </div>`;
+}
+
+function workspacePanel(tf) {
+  const entry = snapshot?.timeframes?.[tf] || {};
+  const r = entry.report;
+  if (!r) {
+    return `<article class="card workspace-card border-signal sig-neutral" id="workspace-panel" aria-labelledby="workspace-panel-heading">
+      <div class="card-body py-4 text-center text-secondary">
+        <h3 class="tf-name mb-2" id="workspace-panel-heading">${TF_LABEL[tf]}</h3>
+        ${entry.status === "error"
+          ? `<span class="badge text-bg-danger">SCAN UNAVAILABLE</span><div class="small mt-2">Showing again when the next scan succeeds.</div>`
+          : "Warming up — no completed scan for this timeframe yet."}
+      </div>
+    </article>`;
+  }
+
+  const cls = sigClass(r.signal);
+  const textCls = cls === "buy" ? "text-success" : cls === "sell" ? "text-danger" : "text-secondary";
+  const qHi = r.quality === "A+" || r.quality === "A";
+  const pa = r.primary_analysis;
+  const meta = [r.market_regime, r.volatility_regime && r.volatility_regime + " volatility",
+    r.mtf_agreement ? "MTF aligned" : "MTF mixed"].filter(Boolean).join(" · ");
+  const facts = `<div class="row row-cols-2 row-cols-md-4 g-2 mt-1">
+    ${tile("Confidence", r.confidence != null ? Math.round(r.confidence) + "%" : "—")}
+    ${tile("Quality", esc(r.quality || "—"), qHi ? "text-primary" : "")}
+    ${tile("Event risk", esc((r.event_risk || "—").toUpperCase()), r.event_risk === "clear" ? "text-success" : r.event_risk ? "text-warning" : "")}
+    ${tile("MTF", r.mtf_agreement ? "aligned" : "mixed", r.mtf_agreement ? "text-success" : "")}
+  </div>`;
+  const setup = r.trade_setup
+    ? setupCard(r.trade_setup)
+    : `<div class="text-secondary small">No setup published. The signal has not cleared its setup gate.</div>`;
+  return `<article class="card workspace-card border-signal sig-${cls}" id="workspace-panel" aria-labelledby="workspace-panel-heading">
+    <div class="card-body p-3 p-lg-4">
+      <div class="row g-4">
+        <div class="col-lg-7 workspace-meta">
+          <div class="d-flex justify-content-between align-items-start gap-3 flex-wrap mb-3">
+            <div>
+              <div class="d-flex align-items-center gap-2 flex-wrap mb-2">
+                <h3 class="tf-name mb-0" id="workspace-panel-heading">${TF_LABEL[tf]} signal</h3>
+                <span class="badge ${badgeCls(cls)}">${esc(sigLabel(r.signal))}</span>
+                ${r.quality ? `<span class="badge ${qHi ? "text-bg-primary" : "text-bg-secondary"}">quality ${esc(r.quality)}</span>` : ""}
+                ${entry.status === "error" ? `<span class="badge text-bg-danger">STALE</span>` : ""}
+              </div>
+              <div class="workspace-signal ${textCls}">${esc(r.action || sigLabel(r.signal))}</div>
+            </div>
+            <button type="button" class="btn btn-sm btn-outline-primary" data-open-detail="${tf}">
+              Full analysis <i class="bi bi-arrow-up-right ms-1"></i>
+            </button>
+          </div>
+          ${scoreBar("score", pa ? pa.composite_score : r.score)}
+          <div class="small text-secondary mt-2">${esc(meta)}</div>
+          ${facts}
+          <div class="workspace-reasons mt-3 pt-3 border-top">${listBlock(r) || "<div class=\"small text-secondary\">No supporting reasons recorded on this scan.</div>"}</div>
+          <div class="small text-secondary mt-3">${entry.scanned_at ? "scanned " + fmtAge(entry.scanned_at) : ""} · closes in <span class="num" data-close="${entry.next_close_epoch || ""}" data-interval="${TF_SECONDS[tf]}">--:--</span></div>
+        </div>
+        <div class="col-lg-5">
+          <div class="small text-uppercase text-secondary fw-bold mb-2">Trade plan</div>
+          ${setup}
+        </div>
+      </div>
+    </div>
+  </article>`;
+}
+
+function renderCards() {
+  if (!snapshot) return;
+  if (!TF_LABEL[selectedTf]) selectedTf = "5min";
+  const selector = TF_ORDER.map(tf => {
+    const entry = snapshot.timeframes[tf] || {};
+    const r = entry.report;
+    const cls = r ? sigClass(r.signal) : "neutral";
+    const arrow = r ? (ARROW[r.signal] || "→") : "·";
+    const score = r?.score != null ? signed(r.score, 0) : "—";
+    const signal = r ? sigLabel(r.signal) : entry.status === "error" ? "UNAVAILABLE" : "WARMING";
+    return `<button type="button" class="tf-select" data-select-tf="${tf}" aria-pressed="${tf === selectedTf}">
+      <span class="d-flex align-items-center justify-content-between gap-2">
+        <span class="tf-name">${TF_LABEL[tf]}</span>
+        <span class="${cls === "buy" ? "text-success" : cls === "sell" ? "text-danger" : "text-secondary"} fw-bold">${arrow}</span>
+      </span>
+      <span class="small ${cls === "buy" ? "text-success" : cls === "sell" ? "text-danger" : "text-secondary"}">${esc(signal)}</span>
+      <span class="small num float-end">${esc(score)}</span>
+    </button>`;
+  }).join("");
+  $("tf-grid").innerHTML = `<div class="tf-selector">${selector}</div>${workspacePanel(selectedTf)}`;
+  tick();
+}
+
+/* ── timeframe detail dialog ──────────────────────────────────── */
+const tile = (k, v, cls = "") =>
+  `<div class="col"><div class="card d-stat bg-body-tertiary h-100"><div class="card-body p-2">
+    <span class="k">${esc(k)}</span>
+    <span class="v ${cls}">${v}</span>
+  </div></div></div>`;
+
+const SEC_ICON = {
+  "At a glance": "bi-speedometer2",
+  "Trade setup": "bi-bullseye",
+  "Why this signal": "bi-lightbulb",
+  "Multi-timeframe confirmation": "bi-layers",
+  "News context": "bi-newspaper",
+  "AI panel (advisory)": "bi-robot",
+  "Scan detail": "bi-clock-history",
+  "Score": "bi-graph-up",
+  "From the source": "bi-quote",
+  "AI gold-impact summary": "bi-stars",
+  "Timing": "bi-alarm",
+  "Numbers": "bi-calculator"
+};
+const sec = (title, html) => {
+  if (!html) return "";
+  const icon = SEC_ICON[title];
+  return `<section class="mb-4"><h6 class="text-uppercase text-secondary small fw-bold mb-2 d-flex align-items-center gap-2">${icon ? `<i class="bi ${icon} text-primary"></i>` : ""}<span>${esc(title)}</span><span class="flex-fill border-bottom"></span></h6>${html}</section>`;
+};
+
+function indTable(src) {
+  const rows = IND_ORDER.map(name => {
+    const i = src?.[name];
+    const k = `<td class="ind-k">${IND_SHORT[name]}</td>`;
+    if (!i) return `<tr>${k}<td class="text-secondary">— absent</td>
+      <td class="num text-end">—</td><td class="num text-end">—</td><td class="num text-end">—</td>
+      <td class="text-secondary small">no signal this scan</td></tr>`;
+    const w = i.weight == null ? null : +i.weight;
+    return `<tr>${k}
+      <td class="fw-bold ${sigClass(i.signal) === "buy" ? "text-success" : sigClass(i.signal) === "sell" ? "text-danger" : "text-secondary"}">${esc(sigLabel(i.signal))}</td>
+      <td class="num text-end">${i.value != null ? (+i.value).toFixed(2) : "—"}</td>
+      <td class="num text-end">${i.confidence != null ? Math.round(i.confidence) : "—"}</td>
+      <td class="num text-end">${w != null ? w.toFixed(1) : "—"}${w != null
+        ? `<i class="wbar"><i style="width:${Math.min(100, w / 2 * 100)}%"></i></i>` : ""}</td>
+      <td class="text-secondary small">${esc(i.description || "")}</td></tr>`;
+  }).join("");
+  return `<div class="detail-table"><table class="table table-sm d-table"><thead><tr>
+    <th>Indicator</th><th>Signal</th><th class="text-end">Value</th><th class="text-end">Conf</th>
+    <th class="text-end">Weight</th><th>Reading</th></tr></thead><tbody>${rows}</tbody></table></div>`;
+}
+
+function ladder(levels, price) {
+  if (!levels) return "";
+  const atr = levels.atr;
+  const rows = Object.keys(LEVEL_LABEL)
+    .filter(k => levels[k] != null)
+    .map(k => ({ label: LEVEL_LABEL[k], v: +levels[k] }));
+  if (!rows.length) return "";
+  if (price != null) rows.push({ here: true, v: +price });
+  rows.sort((a, b) => b.v - a.v);
+  const body = rows.map(row => {
+    if (row.here) return `<tr class="table-primary fw-bold"><td><i class="bi bi-geo-alt-fill me-1"></i>current price</td>
+      <td class="num text-end">${fmt$(row.v)}</td><td class="num text-end">—</td><td class="small text-secondary">spot</td></tr>`;
+    const d = price != null ? row.v - price : null;
+    const cls = d == null ? "" : d > 0 ? "text-danger" : "text-success";
+    const icon = d == null ? "" : d > 0 ? "bi-arrow-up-short" : "bi-arrow-down-short";
+    return `<tr>
+      <td class="text-secondary">${esc(row.label)}</td>
+      <td class="num text-end ${cls}">${fmt$(row.v)}</td>
+      <td class="num text-end ${cls}">${d == null ? "" : signed(d) + (atr ? ` · ${signed(d / atr, 2)} ATR` : "")}</td>
+      <td class="small ${cls}">${d == null ? "" : `<i class="bi ${icon}"></i>${d > 0 ? "resistance" : "support"}`}</td></tr>`;
+  }).join("");
+  return `<div class="detail-table"><table class="table table-sm ladder"><thead><tr><th>Level</th><th class="text-end">Price</th>
+    <th class="text-end">vs. spot</th><th>Role</th></tr></thead><tbody>${body}</tbody></table></div>`;
+}
+
+function analysisBlock(a, price) {
+  if (!a) return "";
+  const pats = a.pattern_signals?.length
+    ? `<div class="d-flex flex-wrap gap-1 mb-2">${a.pattern_signals.map(p =>
+        `<span class="badge text-bg-primary bg-opacity-10">${esc(p)}</span>`).join("")}</div>` : "";
+  const atr = a.atr_context
+    ? `<div class="small text-secondary mb-2">${esc(a.atr_context.description ||
+        "ATR " + (+a.atr_context.value).toFixed(2))}</div>` : "";
+  const lv = ladder(a.key_levels, price);
+  return indTable(a.indicators) + atr + pats +
+    (lv ? `<div class="small text-secondary fw-semibold mt-2 mb-1">Key levels</div>${lv}` : "");
+}
+
+function listBlock(r) {
+  const group = (title, arr, cls, icon) => arr?.length
+    ? `<div class="small text-secondary fw-semibold mt-2 mb-1">${title} (${arr.length})</div>
+       <div class="small">${arr.map(t => `<div class="${cls}"><i class="bi ${icon} me-1"></i>${esc(t)}</div>`).join("")}</div>` : "";
+  // Violent-move watch: display-only advisory line, stamped post-hoc by
+  // the publisher. Empty unless the feature is enabled.
+  const watch = r.watch_line
+    ? `<div class="small text-warning fw-semibold mt-2">${esc(r.watch_line)}</div>`
+    : "";
+  return watch
+       + group("Warnings", r.warnings, "text-warning", "bi-exclamation-triangle-fill")
+       + group("Bullish", r.bullish_reasons, "text-success", "bi-arrow-up-circle-fill")
+       + group("Bearish", r.bearish_reasons, "text-danger", "bi-arrow-down-circle-fill");
+}
+
+function detailHTML(tf) {
+  const entry = snapshot?.timeframes?.[tf] || {};
+  const r = entry.report;
+  if (!r) return `<div class="text-center text-secondary py-4">${entry.status === "error"
+    ? `Scan error — ${esc(entry.error || "unknown")}`
+    : "Warming up — no completed scan for this timeframe yet."}</div>`;
+
+  const pa = r.primary_analysis;
+  const price = r.current_price;
+  const cls = sigClass(r.signal);
+  const textCls = cls === "buy" ? "text-success" : cls === "sell" ? "text-danger" : "text-secondary";
+  const borderCls = cls === "buy" ? "border-start border-success border-4"
+                  : cls === "sell" ? "border-start border-danger border-4"
+                  : "border-start border-secondary border-4";
+
+  let out = `<div class="card ${borderCls} mb-3"><div class="card-body">
+    <div class="fw-bold fs-5 ${textCls}">${esc(r.action || sigLabel(r.signal))}</div>
+    ${r.news_description || r.event_description
+      ? `<div class="small text-secondary">${esc([r.event_description, r.news_description].filter(Boolean).join(" · "))}</div>`
+      : ""}
+  </div></div>`;
+
+  const ns = r.news_sentiment;
+  out += sec("At a glance", `<div class="row row-cols-2 row-cols-md-3 row-cols-lg-4 g-2">
+    ${tile("Signal", esc(sigLabel(r.signal)), textCls)}
+    ${tile("Score", signed(r.score, 1), textCls)}
+    ${tile("Confidence", r.confidence != null ? Math.round(r.confidence) + "%" : "—")}
+    ${tile("Quality", esc(r.quality || "—"))}
+    ${tile("Regime", esc(r.market_regime || "—"))}
+    ${tile("Volatility", esc(r.volatility_regime || "—"))}
+    ${tile("Event risk", esc((r.event_risk || "—").toUpperCase()),
+        r.event_risk === "clear" ? "text-success" : r.event_risk ? "text-warning" : "")}
+    ${tile("News sentiment", ns != null ? signed(ns, 2) : "—",
+        ns > 0.05 ? "text-success" : ns < -0.05 ? "text-danger" : "")}
+    ${tile("MTF", r.mtf_agreement ? "aligned" : "mixed", r.mtf_agreement ? "text-success" : "")}
+    ${tile("Primary score", signed(r.primary_score, 1), sigClass(r.primary_signal) === "buy" ? "text-success" : sigClass(r.primary_signal) === "sell" ? "text-danger" : "")}
+    ${tile("Exhaustion", esc(r.exhaustion_count ?? "—"), r.exhaustion_count ? "text-warning" : "")}
+  </div>`);
+
+  out += sec("Trade setup", r.trade_setup
+    ? setupCard(r.trade_setup)
+    : `<div class="text-secondary small">No setup published for this timeframe — the engine only emits entry/stop/target once a signal clears its quality gate.</div>`);
+
+  out += sec("Why this signal", listBlock(r) || `<div class="text-secondary small">No reasons recorded on this scan.</div>`);
+
+  out += sec(`${TF_LABEL[tf]} indicators (primary)`,
+    analysisBlock(pa, price) || indTable(r.indicators) +
+      (r.key_levels ? `<div class="small text-secondary fw-semibold mt-2 mb-1">Key levels</div>${ladder(r.key_levels, price)}` : ""));
+
+  const mtf = Object.entries(r.mtf_signals || {});
+  if (mtf.length) {
+    const chips = mtf.map(([k, sig]) => {
+      const s = r.mtf_scores?.[k];
+      const sc = sigClass(sig);
+      return `<span class="badge ${badgeCls(sc)}">${esc(TF_LABEL[k] || k)}
+        <span class="${sc === "buy" ? "text-success" : sc === "sell" ? "text-danger" : "text-secondary"}">${esc(sigLabel(sig))}</span>
+        <span class="num">${s != null ? signed(s, 1) : ""}</span></span>`;
+    }).join("");
+    const confs = (r.confirmation_analyses || []).filter(Boolean).map(a => {
+      const lbl = TF_LABEL[a.timeframe] || a.timeframe;
+      const sc = sigClass(a.composite_signal);
+      return `<details data-dkey="conf-${esc(a.timeframe)}">
+        <summary><strong>${esc(lbl)}</strong>
+          <span class="${sc === "buy" ? "text-success" : sc === "sell" ? "text-danger" : "text-secondary"} fw-bold">${esc(sigLabel(a.composite_signal))}</span>
+          <span class="text-secondary small">${signed(a.composite_score, 1)} · conf ${a.confidence != null ? Math.round(a.confidence) : "—"} · ${esc(a.market_regime || "")}</span></summary>
+        ${analysisBlock(a, price)}</details>`;
+    }).join("");
+    out += sec("Multi-timeframe confirmation",
+      `<div class="d-flex flex-wrap gap-2 align-items-center mb-2">${chips}
+        <span class="fw-bold ${r.mtf_agreement ? "text-success" : "text-secondary"}">${r.mtf_agreement ? "✓ aligned" : "✗ mixed"}</span></div>${confs}`);
+  }
+
+  if (r.news_summary || r.news_description) {
+    out += sec("News context", `<div class="text-secondary small">
+      ${r.news_description ? `<strong>${esc(r.news_description)}</strong>${ns != null ? ` (${signed(ns, 2)})` : ""}<br>` : ""}
+      ${esc(r.news_summary || "")}</div>`);
+  }
+
+  if (r.panel_verdict) {
+    const vcls = r.panel_verdict === "AGREE" ? "text-success"
+      : r.panel_verdict === "OPPOSE" ? "text-danger" : "text-secondary";
+    const takes = (r.panel_takes || []).map(t =>
+      `<tr><td class="ind-k">${esc(t.persona)}</td>
+        <td>${esc(t.stance)}</td>
+        <td class="num text-end">${t.confidence != null ? Math.round(t.confidence) : "—"}</td>
+        <td class="small text-secondary">${esc(t.note || "")}</td></tr>`).join("");
+    out += sec("AI panel (advisory)", `<div class="text-secondary small mb-2">
+      <strong class="${vcls}">${esc(r.panel_verdict)}</strong> ·
+      conf ${Math.round(r.panel_confidence || 0)}%<br>
+      ${esc(r.panel_rationale || "")}</div>
+      ${takes ? `<div class="detail-table"><table class="table table-sm d-table"><tbody>${takes}</tbody></table></div>` : ""}`);
+  }
+
+  out += sec("Scan detail", `<div class="detail-table"><table class="table table-sm d-table"><tbody>
+    <tr><td class="ind-k">Instrument</td><td colspan="5">${esc(r.display_name || r.symbol || "—")}</td></tr>
+    <tr><td class="ind-k">Candle</td><td colspan="5" class="num">${esc(r.timestamp || "—")}</td></tr>
+    <tr><td class="ind-k">Scanned</td><td colspan="5">${esc(entry.scanned_at || "—")}
+      (${fmtAge(entry.scanned_at)}${entry.scan_seconds ? `, took ${entry.scan_seconds}s` : ""})</td></tr>
+    <tr><td class="ind-k">Status</td><td colspan="5">${esc(entry.status || "—")}${entry.error
+      ? ` — <span class="text-danger">${esc(entry.error)}</span>
+          ${entry.error_at ? `(${fmtAge(entry.error_at)})` : ""}` : ""}</td></tr>
+   </tbody></table></div>`);
+  return out;
+}
+
+function renderDetail() {
+  if (!detailTf || !tfModalOpen) return;
+  const entry = snapshot?.timeframes?.[detailTf] || {};
+  const r = entry.report;
+  const openKeys = new Set([...document.querySelectorAll("#detail-body details[open][data-dkey]")]
+    .map(d => d.dataset.dkey));
+  const scroll = $("detail-body").scrollTop;
+
+  $("detail-title").textContent = TF_LABEL[detailTf] || detailTf;
+  $("detail-price").textContent = r?.current_price ? fmt$(r.current_price) : "—";
+  $("detail-sub").innerHTML = `${snapshot?.symbol ? esc(snapshot.symbol) + " · " : ""}closes in
+    <span class="num" data-close="${entry.next_close_epoch || ""}" data-interval="${TF_SECONDS[detailTf]}">--:--</span>`;
+  $("detail-body").innerHTML = detailHTML(detailTf);
+
+  document.querySelectorAll("#detail-body details[data-dkey]").forEach(d => {
+    if (openKeys.has(d.dataset.dkey)) d.open = true;
+  });
+  $("detail-body").scrollTop = scroll;
+  tick();
+}
+
+function openDetail(tf) {
+  if (!TF_LABEL[tf]) return;
+  detailTf = tf;
+  tfModal.show();
+  renderDetail();
+  $("detail-body").scrollTop = 0;
+  $("detail-close").focus();
+}
+
+$("detail-close")?.addEventListener("click", () => tfModal.hide());
+
+/* ── calendar ─────────────────────────────────────────────────── */
+function renderCalendar() {
+  const body = $("cal-body");
+  if (!news) return;
+  const zone = Intl.DateTimeFormat().resolvedOptions().timeZone || "local time";
+  $("calendar-note").textContent = `times local (${zone}) · USD events${news.generated_at ? " · updated " + fmtAge(news.generated_at) : ""}`;
+  const hiOnly = $("hi-only").checked;
+  let events = (news.events || []);
+  if (hiOnly) events = events.filter(e => e.impact === "high");
+  if (!events.length) {
+    body.innerHTML = `<tr><td colspan="7" class="text-secondary text-center py-3">${
+      news.calendar_available ? "No matching events this week." : "Calendar unavailable — source could not be reached."}</td></tr>`;
+    return;
+  }
+  const now = nowServer();
+  let nextUpMarked = false, lastDay = "", rows = "";
+  for (const e of events) {
+    const t = Date.parse(e.time) / 1000;
+    const d = new Date(t * 1000);
+    const day = d.toLocaleDateString(undefined, { weekday: "short", month: "short", day: "numeric" });
+    if (day !== lastDay) {
+      rows += `<tr class="table-group-divider"><td colspan="7" class="text-primary small fw-bold text-uppercase">${esc(day)}</td></tr>`;
+      lastDay = day;
+    }
+    const past = t < now;
+    const nextUp = !past && !nextUpMarked && (nextUpMarked = true);
+    const time = d.toLocaleTimeString(undefined, { hour: "2-digit", minute: "2-digit" });
+    const surp = e.surprise
+      ? `<span class="${e.surprise === "positive" ? "text-success" : e.surprise === "negative" ? "text-danger" : "text-secondary"}" title="actual ${e.surprise === "positive" ? "above" : e.surprise === "negative" ? "below" : "matches"} forecast (numeric only — not gold direction)">${e.surprise === "positive" ? "▲" : e.surprise === "negative" ? "▼" : "="}</span>` : "";
+    rows += `<tr class="${past ? "text-secondary" : ""}${nextUp ? " cal-next-up" : ""}">
+      <td class="num text-nowrap">${esc(time)}</td>
+      <td class="text-secondary">${esc(e.currency)}</td>
+      <td><span class="badge ${e.impact === "high" ? "text-bg-danger" : e.impact === "medium" ? "text-bg-warning" : "text-bg-secondary"}">${e.impact === "medium" ? "Med" : e.impact.charAt(0).toUpperCase() + e.impact.slice(1)}</span></td>
+      <td><button type="button" class="cal-event" data-ev-key="${esc(evKey(e))}" aria-haspopup="dialog">${esc(e.event)}</button></td>
+      <td class="num text-end">${esc(e.forecast ?? "—")}</td>
+      <td class="num text-end">${esc(e.previous ?? "—")}</td>
+      <td class="num text-end">${esc(e.actual ?? "—")} ${surp}</td></tr>`;
+  }
+  body.innerHTML = rows;
+}
+
+/* ── newswire ─────────────────────────────────────────────────── */
+function renderWireTotal() {
+  const box = $("wire-total");
+  const scored = (news?.headlines || []).filter(h => typeof h.score === "number");
+  if (!scored.length) { box.classList.add("d-none"); return; }
+  box.classList.remove("d-none");
+
+  const sum = scored.reduce((a, h) => a + h.score, 0);
+  const mean = sum / scored.length;
+  const up = scored.filter(h => h.score > 0.05).length;
+  const dn = scored.filter(h => h.score < -0.05).length;
+  const cls = mean > 0.05 ? "text-success" : mean < -0.05 ? "text-danger" : "text-secondary";
+  const dir = mean > 0.05 ? "BULLISH" : mean < -0.05 ? "BEARISH" : "NEUTRAL";
+  const wrapCls = mean > 0.05 ? "bg-success-subtle border-success-subtle"
+                : mean < -0.05 ? "bg-danger-subtle border-danger-subtle"
+                : "bg-body-tertiary";
+
+  box.className = `card mb-3 ${wrapCls}`;
+  box.innerHTML = `<div class="card-body p-2">
+    <div class="d-flex align-items-baseline gap-2 flex-wrap">
+      <span class="small text-secondary text-uppercase fw-bold">Total news score</span>
+      <span class="display-6 fw-bold ${cls}">${signed(mean, 2)}</span>
+      <span class="fw-bold ${cls}">${dir}</span>
+      <span class="ms-auto text-end small text-secondary">${scored.length} headlines · sum ${signed(sum, 1)}<br>
+        ${up} bullish · ${dn} bearish</span>
+    </div>
+  </div>`;
+  box.title = `Mean sentiment of all ${scored.length} scored headlines (−1…+1). Sum ${signed(sum, 1)}. Only headlines from the last 4 hours that lean one way are published (neutral ones are removed before scoring).`;
+}
+
+function renderWire() {
+  if (!news) return;
+  const items = (news.headlines || []).slice(0, 40);
+  $("wire-note").textContent = news.generated_at ? `updated ${fmtAge(news.generated_at)}` : "";
+  renderWireTotal();
+  if (!items.length) {
+    $("wire").innerHTML = `<div class="text-secondary small py-2">No gold or dollar headlines in the last 4 hours.</div>`;
+    return;
+  }
+  const item = h => {
+    const s = h.score;
+    const cls = s == null || Math.abs(s) < 0.05 ? "zero" : s > 0 ? "pos" : "neg";
+    const scoreCls = cls === "pos" ? "bg-success-subtle text-success"
+                   : cls === "neg" ? "bg-danger-subtle text-danger"
+                   : "bg-body-tertiary text-secondary";
+    const scoreTxt = s == null ? "·" : (s > 0 ? "+" : "") + s.toFixed(1);
+    let host = "";
+    try { host = new URL(h.source).hostname.replace(/^www\./, ""); } catch {}
+    const content = `<span class="wire-score ${scoreCls}" title="Headline sentiment ${esc(String(s ?? "n/a"))} (−1…+1)${h.currencies?.length ? " · " + esc(h.currencies.join(", ")) : ""}">${scoreTxt}</span>
+       <span class="small text-secondary flex-fill">${esc(h.headline)}</span>
+       <span class="small text-secondary text-nowrap">${esc(h.publisher || host)}${h.timestamp ? " · " + fmtAge(h.timestamp) : ""}</span>`;
+    if (!h.id) return `<div class="wire-item gap-2 align-items-baseline py-1 px-1">${content}</div>`;
+    return `<button type="button" class="wire-item gap-2 align-items-baseline py-1 px-1" data-news-id="${h.id}" aria-haspopup="dialog">
+      ${content}
+    </button>`;
+  };
+  const head = items.slice(0, 12).map(item).join("");
+  const rest = items.slice(12);
+  const moreOpen = $("wire")?.querySelector("details.more")?.open ?? false;
+  $("wire").innerHTML = head + (rest.length
+    ? `<details class="more mt-1"${moreOpen ? " open" : ""}><summary class="small text-secondary">${rest.length} more headlines…</summary>
+       <div class="mt-1">${rest.map(item).join("")}</div></details>` : "");
+}
+
+/* ── news / calendar detail dialog ────────────────────────────── */
+const evKey = e => `${e.time}|${e.currency}|${e.event}`;
+const findHeadline = id => (news?.headlines || []).find(h => h.id === id);
+const findEvent = key => (news?.events || []).find(e => evKey(e) === key);
+
+const AI_ERR_MSG = {
+  unknown_id: "This headline has left the live feed — no summary available.",
+  cap_exceeded: "Daily AI budget reached — try again tomorrow.",
+  no_key: "AI summaries are not configured on this server."
+};
+
+async function fetchAiSummary(id) {
+  aiPending.add(id);
+  try {
+    const res = await fetch("news_summary.php?id=" + encodeURIComponent(id));
+    const data = await res.json().catch(() => null);
+    if (res.ok && data?.summary) aiMemo.set(id, data);
+    else aiErr.set(id, data?.error || `http_${res.status}`);
+  } catch {
+    aiErr.set(id, "network");
+  } finally {
+    aiPending.delete(id);
+    if (ndState?.kind === "wire" && ndState.id === id) renderNewsDlg();
+  }
+}
+
+function scoreTiles(h) {
+  const cls = v => v > 0.05 ? "text-success" : v < -0.05 ? "text-danger" : "text-secondary";
+  let tiles = tile("Overall sentiment", signed(h.score, 2), cls(h.score));
+  const xau = h.score_by_currency?.XAU;
+  if (xau != null) tiles += tile("Gold (XAU) effect", signed(xau, 2), cls(xau));
+  for (const [cur, v] of Object.entries(h.score_by_currency || {})) {
+    if (cur !== "XAU") tiles += tile(cur, signed(v, 2), cls(v));
+  }
+  let out = `<div class="row row-cols-2 row-cols-md-3 g-2">${tiles}</div>`;
+  if (h.keywords?.length) {
+    out += `<div class="small text-secondary mt-2">Matched keywords: ${esc(h.keywords.join(", "))}</div>`;
+  }
+  out += `<div class="small text-secondary">Scores range −1 (bearish) to +1 (bullish); the gold score flips monetary-policy keywords (hawkish = dollar-bullish, gold-bearish).</div>`;
+  return out;
+}
+
+function aiSection(id) {
+  if (aiMemo.has(id)) {
+    const m = aiMemo.get(id);
+    return `<div class="text-secondary small">${esc(m.summary)}</div>` + (m.generated_at
+      ? `<div class="small text-secondary">generated ${fmtAge(m.generated_at)}</div>` : "");
+  }
+  if (aiPending.has(id)) return `<div class="text-secondary small">Generating gold-impact summary…</div>`;
+  if (aiErr.has(id)) {
+    return `<div class="text-secondary small">${esc(AI_ERR_MSG[aiErr.get(id)] || "AI summary unavailable right now.")}</div>`;
+  }
+  return `<div class="text-secondary small">Generating gold-impact summary…</div>`;
+}
+
+function wireDlgHTML() {
+  const fresh = findHeadline(ndState.id);
+  const h = fresh || ndState.snap;
+  if (!h) return "";
+  let host = "";
+  try { host = new URL(h.source).hostname.replace(/^www\./, ""); } catch {}
+  const when = h.timestamp
+    ? `${new Date(h.timestamp).toLocaleString(undefined, { dateStyle: "medium", timeStyle: "short" })} · ${fmtAge(h.timestamp)}` : "";
+  let out = `<div class="fs-5 fw-semibold mb-1">${esc(h.headline)}</div>
+    <div class="small text-secondary mb-3">${esc(h.publisher || host)}${when ? " · " + when : ""}</div>`;
+  if (!fresh) out += `<div class="small text-secondary mb-2">No longer in the live feed — showing saved copy.</div>`;
+  out += sec("Score", scoreTiles(h));
+  if (h.description) out += sec("From the source", `<div class="text-secondary small">${esc(h.description)}</div>`);
+  out += sec("AI gold-impact summary", aiSection(ndState.id));
+  if (h.link && /^https?:\/\//.test(h.link)) {
+    out += `<a class="btn btn-outline-primary" href="${esc(h.link)}" target="_blank" rel="noopener noreferrer">Read full article ↗</a>`;
+  }
+  return out;
+}
+
+function calDlgHTML() {
+  const fresh = findEvent(ndState.key);
+  const e = fresh || ndState.snap;
+  if (!e) return "";
+  const t = Date.parse(e.time) / 1000;
+  const when = new Date(e.time).toLocaleString(undefined, { dateStyle: "full", timeStyle: "short" });
+  const impact = `<span class="badge ${e.impact === "high" ? "text-bg-danger" : e.impact === "medium" ? "text-bg-warning" : "text-bg-secondary"}">${e.impact === "medium" ? "Med" : e.impact.charAt(0).toUpperCase() + e.impact.slice(1)}</span>`;
+  let out = `<div class="fs-5 fw-semibold mb-1">${esc(e.event)}</div>
+    <div class="small text-secondary mb-3">${esc(e.currency)} · ${when}</div>`;
+  const timing = t > nowServer()
+    ? `releases in <span class="num" data-close="${t}">--:--</span>`
+    : `released ${fmtAge(e.time)}`;
+  out += sec("Timing", `<div class="text-secondary small">${impact} impact · ${timing}</div>`);
+  const surpTxt = { positive: "actual above forecast", negative: "actual below forecast", inline: "actual matches forecast" }[e.surprise];
+  out += sec("Numbers", `<div class="row row-cols-3 g-2">${
+      tile("Forecast", esc(e.forecast ?? "—")) +
+      tile("Previous", esc(e.previous ?? "—")) +
+      tile("Actual", esc(e.actual ?? "—"), e.surprise === "positive" ? "text-success" : e.surprise === "negative" ? "text-danger" : "")
+    }</div>` + (surpTxt
+      ? `<div class="small text-secondary mt-2">${surpTxt} (numeric only — not gold direction)</div>` : ""));
+  return out;
+}
+
+function renderNewsDlg() {
+  if (!ndState || !newsModalOpen) return;
+  const scroll = $("nd-body").scrollTop;
+  $("nd-title").textContent = ndState.kind === "wire" ? "Newswire" : "Economic calendar";
+  $("nd-body").innerHTML = ndState.kind === "wire" ? wireDlgHTML() : calDlgHTML();
+  $("nd-body").scrollTop = scroll;
+  tick();
+}
+
+function openNewsDetail(id) {
+  const h = findHeadline(id);
+  if (!h) return;
+  ndState = { kind: "wire", id, snap: h };
+  aiErr.delete(id);
+  newsModal.show();
+  if (!aiMemo.has(id) && !aiPending.has(id)) fetchAiSummary(id);
+  renderNewsDlg();
+  $("nd-body").scrollTop = 0;
+  $("nd-close").focus();
+}
+
+function openCalDetail(key) {
+  const e = findEvent(key);
+  if (!e) return;
+  ndState = { kind: "cal", key, snap: e };
+  newsModal.show();
+  renderNewsDlg();
+  $("nd-body").scrollTop = 0;
+  $("nd-close").focus();
+}
+
+$("nd-close")?.addEventListener("click", () => newsModal.hide());
+
+/* ── polling ──────────────────────────────────────────────────── */
+async function fetchJSON(name) {
+  const res = await fetch(name, { cache: "no-cache" });
+  if (!res.ok) throw new Error(`${name}: HTTP ${res.status}`);
+  const dateHdr = res.headers.get("Date");
+  const text = await res.text();
+  return { text, serverNow: dateHdr ? Date.parse(dateHdr) / 1000 : null };
+}
+
+function tfFingerprint(entry) {
+  if (!entry) return "";
+  return JSON.stringify({
+    status: entry.status,
+    scanned_at: entry.scanned_at,
+    error: entry.error,
+    error_at: entry.error_at,
+    report: entry.report
+  });
+}
+
+function scheduleRefresh(delay) {
+  clearTimeout(refetchTimer);
+  refetchTimer = setTimeout(() => {
+    refetchTimer = null;
+    poll();
+  }, Math.max(0, delay));
+}
+
+function scheduleFallbackRefresh() {
+  clearTimeout(fallbackTimer);
+  fallbackTimer = setTimeout(() => {
+    fallbackTimer = null;
+    poll();
+    scheduleFallbackRefresh();
+  }, FALLBACK_POLL_MS);
+}
+
+function scheduleBoundaryRefresh() {
+  if (!snapshot) return;
+  const nextScan = Math.min(...TF_ORDER.map(tf =>
+    snapshot.timeframes[tf]?.next_scan_epoch || Infinity));
+  if (!isFinite(nextScan)) return;
+  const delay = (nextScan - nowServer() + 2) * 1000;
+  scheduleRefresh(delay > 0 ? delay : RETRY_MS);
+}
+
+async function poll() {
+  if (pollInFlight) {
+    pollQueued = true;
+    return;
+  }
+  pollInFlight = true;
+  try {
+    const [snap, nw] = await Promise.all([
+      fetchJSON("snapshot.json"),
+      fetchJSON("news.json").catch(() => null)
+    ]);
+    const priorSnapshot = snapshot;
+    let snapChanged = false;
+    let selectedChanged = false;
+    let cardsChanged = false;
+    let detailChanged = false;
+    if (snap.text !== lastSnapText) {
+      const incoming = JSON.parse(snap.text);
+      if (!priorSnapshot || incoming.server_time_epoch >= priorSnapshot.server_time_epoch) {
+        snapChanged = true;
+        selectedChanged = tfFingerprint(priorSnapshot?.timeframes?.[selectedTf]) !==
+          tfFingerprint(incoming.timeframes?.[selectedTf]);
+        // renderCards() redraws the selector for EVERY timeframe, so it
+        // must react to any of them changing — not just the selected one.
+        // Gating it on selectedChanged left the other buttons showing a
+        // stale arrow/score (up to 4 h on H4) while renderHero's rail,
+        // which redraws unconditionally, already showed the new value.
+        cardsChanged = TF_ORDER.some(tf =>
+          tfFingerprint(priorSnapshot?.timeframes?.[tf]) !==
+          tfFingerprint(incoming.timeframes?.[tf]));
+        detailChanged = detailTf != null && tfFingerprint(priorSnapshot?.timeframes?.[detailTf]) !==
+          tfFingerprint(incoming.timeframes?.[detailTf]);
+        snapshot = incoming;
+        lastSnapText = snap.text;
+      }
+    }
+    const newsChanged = nw != null && nw.text !== lastNewsText;
+    if (newsChanged) { news = JSON.parse(nw.text); lastNewsText = nw.text; }
+    clockOffset = (snap.serverNow ?? snapshot.server_time_epoch) - Date.now() / 1000;
+    lastPollOk = new Date();
+    $("banner-fail").classList.add("d-none");
+
+    if (snapChanged) {
+      renderHero();
+      if (cardsChanged) renderCards();
+      if (detailChanged) renderDetail();
+      if (selectedChanged) {
+        const signal = snapshot.timeframes[selectedTf]?.report?.signal;
+        if (signal) announce(`${TF_LABEL[selectedTf]} signal updated: ${sigLabel(signal)}.`);
+      }
+    }
+    if (newsChanged) {
+      renderCalendar();
+      renderWire();
+      renderNewsDlg();
+    }
+    if (snapChanged || newsChanged) renderContext();
+    tick();
+    scheduleBoundaryRefresh();
+  } catch (err) {
+    const last = lastPollOk ? ` Last update ${lastPollOk.toLocaleTimeString()}.` : "";
+    const b = $("banner-fail");
+    b.textContent = `Dashboard data is temporarily unavailable.${last} Retrying…`;
+    b.classList.remove("d-none");
+    scheduleRefresh(RETRY_MS);
+  } finally {
+    pollInFlight = false;
+    if (pollQueued) {
+      pollQueued = false;
+      poll();
+    }
+  }
+}
+
+function tick() {
+  if (!snapshot) return;
+  const now = nowServer();
+  document.querySelectorAll("[data-close]").forEach(el => {
+    const t = parseFloat(el.dataset.close);
+    if (!t) return;
+    let secs = t - now;
+    if (secs < 0) {
+      const interval = parseFloat(el.dataset.interval);
+      secs = interval > 0 ? ((secs % interval) + interval) % interval : 0;
+    }
+    el.textContent = fmtCountdown(secs);
+  });
+  const m5 = snapshot.timeframes["5min"]?.next_close_epoch;
+  if (m5) {
+    let secs = m5 - now;
+    if (secs < 0) secs = ((secs % TF_SECONDS["5min"]) + TF_SECONDS["5min"]) % TF_SECONDS["5min"];
+    $("hero-cd").textContent = fmtCountdown(secs);
+  }
+  const pubEpoch = snapshot.server_time_epoch;
+  if (pubEpoch) {
+    const age = now - pubEpoch;
+    const el = $("hero-updated");
+    const stale = !snapshot.market_closed && age > 900;
+    el.textContent = (stale ? "STALE — published " : "published ") +
+      fmtAge(new Date(pubEpoch * 1000).toISOString());
+    el.classList.toggle("text-warning", stale);
+    el.classList.toggle("fw-semibold", stale);
+  }
+}
+
+/* ── boot ─────────────────────────────────────────────────────── */
+document.addEventListener("DOMContentLoaded", () => {
+  tfModal = new bootstrap.Modal($("tf-detail"));
+  newsModal = new bootstrap.Modal($("news-detail"));
+  const storedTf = localGet(TIMEFRAME_PREF);
+  if (TF_LABEL[storedTf]) selectedTf = storedTf;
+  $("hi-only").checked = localGet(HIGH_IMPACT_PREF) === "true";
+
+  $("tf-detail")?.addEventListener("shown.bs.modal", () => { tfModalOpen = true; renderDetail(); });
+  $("tf-detail")?.addEventListener("hidden.bs.modal", () => { tfModalOpen = false; detailTf = null; });
+  $("news-detail")?.addEventListener("shown.bs.modal", () => { newsModalOpen = true; renderNewsDlg(); });
+  $("news-detail")?.addEventListener("hidden.bs.modal", () => { newsModalOpen = false; ndState = null; });
+
+  const selectTf = tf => {
+    if (!TF_LABEL[tf]) return;
+    const changed = selectedTf !== tf;
+    selectedTf = tf;
+    localSet(TIMEFRAME_PREF, tf);
+    renderHero();
+    renderCards();
+    if (changed) announce(`${TF_LABEL[tf]} workspace selected.`);
+  };
+  $("hero-rail")?.addEventListener("click", e => {
+    const button = e.target.closest("[data-select-tf]");
+    if (button) selectTf(button.dataset.selectTf);
+  });
+  $("tf-grid")?.addEventListener("click", e => {
+    const detail = e.target.closest("[data-open-detail]");
+    if (detail) return openDetail(detail.dataset.openDetail);
+    const button = e.target.closest("[data-select-tf]");
+    if (button) selectTf(button.dataset.selectTf);
+  });
+  $("wire")?.addEventListener("click", e => {
+    const button = e.target.closest("[data-news-id]");
+    if (button) openNewsDetail(button.dataset.newsId);
+  });
+  $("cal-body")?.addEventListener("click", e => {
+    const button = e.target.closest("[data-ev-key]");
+    if (button) openCalDetail(button.dataset.evKey);
+  });
+  $("hi-only")?.addEventListener("change", () => {
+    localSet(HIGH_IMPACT_PREF, String($("hi-only").checked));
+    renderCalendar();
+  });
+
+  poll();
+  scheduleFallbackRefresh();
+  setInterval(tick, 1000);
+});

static/news_summary.php

@@ -0,0 +1,266 @@
+<?php
+/**
+ * On-demand gold-impact AI summary for one published headline.
+ *
+ * The ONLY dynamic piece of the otherwise-static dashboard. Serves
+ * GET ?id=<sha1-of-headline> and returns {"summary": ..., "generated_at": ...}.
+ *
+ * Abuse model (the page is public):
+ *   - WHITELIST: the id must match a headline in the local news.json —
+ *     arbitrary text can never reach the LLM through this endpoint.
+ *   - CACHE FOREVER: headlines are immutable, so each id costs at most
+ *     one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
+ *     the whitelist on purpose: summaries stay reachable after the
+ *     headline ages out of the 4 h feed window.
+ *   - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
+ *     errors (a perms mistake must not mean unmetered spend).
+ *
+ * DeepSeek doctrine (mirrors llm_client.py — deliberate local copy, the
+ * dashboard never imports bot code and this file never reads the repo's
+ * config.json):
+ *   - STATIC byte-identical system prompt (prefix-cache pricing).
+ *   - "thinking": disabled is MANDATORY for V4-Flash (else it burns the
+ *     timeout reasoning), temperature 0, json response_format.
+ *   - One attempt, no retries; every failure returns a JSON error the
+ *     page renders as a graceful fallback.
+ *
+ * Key file lives OUTSIDE the docroot: /etc/gold-dashboard/deepseek.key
+ * (root:apache 640). Override paths/caps via Apache SetEnv:
+ * DEEPSEEK_KEY_FILE, AI_SUMMARY_DAILY_CAP.
+ */
+
+const NEWS_FILE = __DIR__ . '/news.json';
+const CACHE_DIR = __DIR__ . '/ai';   // apache-owned; created at deploy time
+const API_URL   = 'https://api.deepseek.com/chat/completions';
+const MODEL     = 'deepseek-v4-flash';
+const TIMEOUT   = 12;
+const MAX_TOKENS = 200;
+const SUMMARY_MAX_CHARS = 400;
+
+// .ht_ prefix: Apache's default <Files ".ht*"> rule denies these even
+// though they sit in the public docroot.
+const CAP_FILE = CACHE_DIR . '/.ht_cap.json';
+
+const SYSTEM_PROMPT = <<<'PROMPT'
+You are a forex/gold news analyst. You receive one news item as JSON:
+{"headline", "description", "score", "xau_score", "keywords", "currencies"}.
+"xau_score" is the platform's automated gold-effect score for this item,
+-1.0 (gold down) to 1.0 (gold up); "score" is overall market sentiment.
+Both are automated and can be wrong — judge the news yourself.
+In 1-2 plain-English sentences (max 45 words) explain what this news
+means for the GOLD (XAU/USD) price and WHY it is bullish, bearish, or
+neutral. If your reading clearly contradicts xau_score, add "(the
+automated score disagrees)". Never write the words "xau_score" or
+"score" — a general reader sees this text. Write for someone who has
+never traded: short simple sentences, no acronyms or abbreviations
+(say "the US central bank", not "the Fed"; "interest rates", not
+"yields").
+Gold doctrine: hawkish Fed / rate hikes / rising real yields / strong
+dollar are gold-bearish; easing / rate cuts / safe-haven flows /
+geopolitical stress / weak dollar are gold-bullish.
+Reply with ONLY JSON: {"summary": "..."}
+PROMPT;
+
+function respond(int $status, array $obj, string $cache = 'no-store'): void
+{
+    http_response_code($status);
+    header('Content-Type: application/json; charset=utf-8');
+    header('Cache-Control: ' . $cache);
+    echo json_encode($obj, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
+    exit;
+}
+
+function atomic_write(string $path, string $data): bool
+{
+    $tmp = $path . '.tmp.' . getmypid();
+    $written = @file_put_contents($tmp, $data);
+    // A SHORT write (disk full / quota) returns a byte count, NOT false.
+    // Renaming that into place would publish a truncated JSON body which
+    // the cache-first branch above then serves forever.
+    if ($written === false || $written !== strlen($data)) {
+        @unlink($tmp);
+        return false;
+    }
+    if (!@rename($tmp, $path)) {
+        @unlink($tmp);
+        return false;
+    }
+    return true;
+}
+
+// ── 1. id validation ────────────────────────────────────────────────
+$id = $_GET['id'] ?? '';
+if (!preg_match('/^[a-f0-9]{40}$/', $id)) {
+    respond(400, ['error' => 'bad_id']);
+}
+
+// ── 2. cache first (immutable headlines → summaries never change) ───
+$cachePath = CACHE_DIR . '/' . $id . '.json';
+$cached = @file_get_contents($cachePath);
+if ($cached !== false && $cached !== '') {
+    http_response_code(200);
+    header('Content-Type: application/json; charset=utf-8');
+    header('Cache-Control: max-age=3600');
+    echo $cached;
+    exit;
+}
+
+// ── 3. whitelist against the published feed ─────────────────────────
+$news = json_decode((string)@file_get_contents(NEWS_FILE), true);
+if (!is_array($news)) {
+    respond(503, ['error' => 'no_feed']);
+}
+$hit = null;
+foreach ($news['headlines'] ?? [] as $h) {
+    $hid = $h['id'] ?? '';
+    if ($hid === $id) {
+        $hit = $h;
+        break;
+    }
+}
+if ($hit === null) {
+    respond(404, ['error'   => 'unknown_id',
+                  'message' => 'Headline is no longer in the feed.']);
+}
+
+// ── 4. API key ──────────────────────────────────────────────────────
+$keyFile = getenv('DEEPSEEK_KEY_FILE') ?: '/etc/gold-dashboard/deepseek.key';
+$apiKey = trim((string)@file_get_contents($keyFile));
+if ($apiKey === '') {
+    respond(503, ['error' => 'no_key']);
+}
+
+// ── 5. per-id stampede lock, then re-check the cache ────────────────
+$lockFh = @fopen(CACHE_DIR . '/.ht_lock_' . $id, 'c');
+if ($lockFh !== false) {
+    flock($lockFh, LOCK_EX);   // blocks while a concurrent winner generates
+    $cached = @file_get_contents($cachePath);
+    if ($cached !== false && $cached !== '') {
+        flock($lockFh, LOCK_UN);
+        fclose($lockFh);
+        http_response_code(200);
+        header('Content-Type: application/json; charset=utf-8');
+        header('Cache-Control: max-age=3600');
+        echo $cached;
+        exit;
+    }
+}
+$releaseLock = function () use (&$lockFh) {
+    if ($lockFh !== false) {
+        @flock($lockFh, LOCK_UN);
+        @fclose($lockFh);
+        $lockFh = false;
+    }
+};
+
+// ── 6. daily cap (fail closed) ──────────────────────────────────────
+// Counted HERE, not before the lock: a burst of clicks on one headline
+// makes exactly one DeepSeek call, and the losers are served from the
+// re-checked cache above — charging them against the budget (and
+// charging a request that then found no key) could exhaust the whole
+// 150/day allowance with zero LLM calls made.
+$dailyCap = (int)(getenv('AI_SUMMARY_DAILY_CAP') ?: 150);
+$capFh = @fopen(CAP_FILE, 'c+');
+if ($capFh === false || !flock($capFh, LOCK_EX)) {
+    $releaseLock();
+    respond(503, ['error' => 'cap_io']);
+}
+$today = gmdate('Y-m-d');
+$cap = json_decode((string)stream_get_contents($capFh), true);
+if (!is_array($cap) || ($cap['day'] ?? '') !== $today) {
+    $cap = ['day' => $today, 'count' => 0];
+}
+if ($cap['count'] >= $dailyCap) {
+    flock($capFh, LOCK_UN);
+    fclose($capFh);
+    $releaseLock();
+    respond(429, ['error' => 'cap_exceeded']);
+}
+// Increment BEFORE the LLM call — attempts count against the cap
+// (same semantics as llm_client._under_cap).
+$cap['count']++;
+rewind($capFh);
+if (ftruncate($capFh, 0) === false
+        || fwrite($capFh, json_encode($cap)) === false) {
+    flock($capFh, LOCK_UN);
+    fclose($capFh);
+    $releaseLock();
+    respond(503, ['error' => 'cap_io']);
+}
+fflush($capFh);
+flock($capFh, LOCK_UN);
+fclose($capFh);
+
+// ── 7. DeepSeek call (exact chat_json body shape) ───────────────────
+$userPayload = json_encode([
+    'headline'    => $hit['headline'] ?? '',
+    'description' => $hit['description'] ?? null,
+    'score'       => $hit['score'] ?? null,
+    'xau_score'   => $hit['score_by_currency']['XAU'] ?? null,
+    'keywords'    => $hit['keywords'] ?? [],
+    'currencies'  => $hit['currencies'] ?? [],
+], JSON_UNESCAPED_UNICODE);
+
+$body = json_encode([
+    'model'    => MODEL,
+    'messages' => [
+        ['role' => 'system', 'content' => SYSTEM_PROMPT],
+        ['role' => 'user',   'content' => $userPayload],
+    ],
+    'response_format' => ['type' => 'json_object'],
+    'temperature'     => 0,
+    'thinking'        => ['type' => 'disabled'],
+    'max_tokens'      => MAX_TOKENS,
+]);
+
+$ch = curl_init(API_URL);
+curl_setopt_array($ch, [
+    CURLOPT_POST           => true,
+    CURLOPT_POSTFIELDS     => $body,
+    CURLOPT_HTTPHEADER     => ['Content-Type: application/json',
+                               'Authorization: Bearer ' . $apiKey],
+    CURLOPT_RETURNTRANSFER => true,
+    CURLOPT_TIMEOUT        => TIMEOUT,
+]);
+$raw = curl_exec($ch);
+$httpCode = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
+curl_close($ch);
+
+// ── 8. parse (any failure → 502, never cached) ──────────────────────
+$summary = null;
+if ($raw !== false && $httpCode === 200) {
+    $resp = json_decode($raw, true);
+    $content = $resp['choices'][0]['message']['content'] ?? '';
+    $parsed = json_decode((string)$content, true);
+    if (is_array($parsed) && is_string($parsed['summary'] ?? null)
+            && trim($parsed['summary']) !== '') {
+        $summary = trim($parsed['summary']);
+        if (mb_strlen($summary) > SUMMARY_MAX_CHARS) {
+            $summary = mb_substr($summary, 0, SUMMARY_MAX_CHARS) . '…';
+        }
+    }
+}
+if ($lockFh !== false && $summary === null) {
+    flock($lockFh, LOCK_UN);
+    fclose($lockFh);
+}
+if ($summary === null) {
+    respond(502, ['error' => 'llm_failed']);
+}
+
+// ── 9. cache + respond ──────────────────────────────────────────────
+$out = json_encode(['summary' => $summary, 'generated_at' => gmdate('c')],
+                   JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
+if (!atomic_write($cachePath, $out)) {
+    // Degrade to per-click generation rather than a broken feature —
+    // the daily cap still bounds spend.
+    error_log('news_summary.php: cache write failed for ' . $cachePath);
+}
+if ($lockFh !== false) {
+    flock($lockFh, LOCK_UN);
+    fclose($lockFh);
+}
+http_response_code(200);
+header('Content-Type: application/json; charset=utf-8');
+header('Cache-Control: max-age=3600');
+echo $out;

violent_watch.py

@@ -0,0 +1,426 @@
+"""
+ForexBot2 — violent-move watch (DISPLAY-ONLY heads-up)
+=======================================================
+Flags the conditions under which a "violent bar" (range >= 3x the prior
+ATR14) is unusually likely, so a human reading an alert knows the next
+hour may be wild. It is a WARNING LINE and a standalone heads-up
+message — nothing here ever changes a signal, grade, gate, trade setup
+or backtest result.
+
+Hard independence rule (mirrors agent_panel.py):
+this module is NEVER imported by `signal_aggregator`, `engine`, or
+anything under `backtest/`. Surfaces (telegram_bot, main, the dashboard
+publisher) attach the result POST-HOC via `attach_watch`, exactly the
+way panels are attached.
+
+Evidence — 3-year study, backtest/research/sizing_campaign_2026-09.md
+(addenda):
+  * A violent bar is a bar whose range >= 3x the prior ATR14.
+  * Hot clock slots run about 3-4x the normal rate: bars closing
+    12:00-14:59 UTC on weekdays, the 01:xx UTC hour on any weekday, and
+    the Sunday reopen (22:xx-23:xx UTC).
+  * The 30 minutes after a high-impact US release run about 10x normal.
+    Share of releases followed by a violent bar within the hour: NFP
+    97%, CPI 91%, FOMC 88%, retail sales 64%, PPI 54%, PCE 50%,
+    jobless claims 44%.
+  * Indicator readings taken BEFORE the bar each lift the odds 1.3-3x:
+    price outside the Bollinger bands, RSI < 30 or >= 70, close outside
+    the Keltner channel, a rising ADX (slope > 3), volatility regime
+    "extreme", market regime "transitioning", and MACD histogram
+    negative and accelerating down (the engine emits STRONG_SELL for
+    exactly that case).
+  * Only the three PRICE-EXTREME readings (Bollinger, RSI, Keltner)
+    earn a level. A bar-by-bar sweep of the 3-year window showed the
+    first draft — which let any two readings raise the level — flagged
+    27% of all bars "hot" at merely ~2x odds (~13 heads-ups/day). The
+    shipped rule (calendar, or a clock slot plus one price extreme)
+    flags 8.6% of bars: next-bar odds 3-5x normal, ~20% chance within
+    the hour, about 4 hot hours a day. The remaining readings (ADX
+    slope, volatility regime, regime transition, MACD acceleration) are
+    CONTEXT: they are printed once a level exists, and never create one.
+  * Direction: bearish readings preceded a DOWN burst 70-78% of the
+    time on BOTH M5 and M15. Bullish readings preceded an UP burst only
+    ~60% of the time and only on M5 — so an "up" call is made on M5
+    alone.
+  * Open trades are NOT to be touched. Every protective action tested
+    (early close, tightened stop, partial exit) lost money; the stop
+    already IS the protection. The heads-up says so explicitly.
+"""
+
+import logging
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta, timezone
+from typing import Any, Dict, List, Optional, Tuple
+
+from config import SignalStrength, Timeframe
+
+logger = logging.getLogger(__name__)
+
+# Calendar window either side of a high-impact US release (minutes).
+RELEASE_WINDOW_MINUTES = 30
+# Hot clock slots (UTC), from the 3-year bar study.
+US_DATA_HOURS = (12, 13, 14)
+ASIAN_OPEN_HOUR = 1
+SUNDAY_REOPEN_HOURS = (22, 23)
+# ADX slope above this counts as "trend strength rising".
+ADX_SLOPE_HOT = 3.0
+
+_LEVEL_NONE = "none"
+_LEVEL_WATCH = "watch"
+_LEVEL_HOT = "hot"
+
+
+@dataclass
+class ViolentWatch:
+    """One scan's violent-move assessment. Advisory/display-only."""
+    level: str = _LEVEL_NONE          # "none" | "watch" | "hot"
+    triggers: List[str] = field(default_factory=list)   # plain language
+    kinds: List[str] = field(default_factory=list)      # calendar/clock/extreme
+    readings: int = 0                 # PRICE-EXTREME readings (0..3)
+    likely_direction: str = ""        # "down" | "up" | ""
+
+    def line(self) -> str:
+        """One-line alert text; "" when nothing fired."""
+        if self.level == _LEVEL_NONE or not self.triggers:
+            return ""
+        if self.level == _LEVEL_HOT:
+            head = "🌪 Violent-move watch (HOT): "
+            odds = " — odds of a violent bar about 3–5× normal"
+        else:
+            head = "🌪 Violent-move watch (watch): "
+            odds = " — about 2× normal"
+        tail = ""
+        if self.likely_direction == "down":
+            tail = " — if one comes it has usually gone DOWN"
+        elif self.likely_direction == "up":
+            tail = " — if one comes it has usually gone UP"
+        return head + " · ".join(self.triggers) + odds + tail
+
+
+# ── Calendar ─────────────────────────────────────────────────────────
+
+def _is_high_impact_usd(event: Any) -> bool:
+    """USD + high impact, tolerating either the `impact` string or the
+    `is_high_impact` property (EconomicEvent exposes both)."""
+    try:
+        if str(getattr(event, "currency", "")).upper() != "USD":
+            return False
+        impact = getattr(event, "impact", None)
+        if isinstance(impact, str):
+            return impact.strip().lower() == "high"
+        return bool(getattr(event, "is_high_impact", False))
+    except Exception:
+        return False
+
+
+def _calendar_trigger(events: Optional[List[Any]],
+                      now: datetime) -> Optional[str]:
+    """Soonest high-impact USD release inside +/-30 min of `now`."""
+    if not events:
+        return None
+    window = timedelta(minutes=RELEASE_WINDOW_MINUTES)
+    best: Optional[Tuple[float, str]] = None
+    for ev in events or []:
+        if not _is_high_impact_usd(ev):
+            continue
+        when = getattr(ev, "time", None)
+        if not isinstance(when, datetime) or when.tzinfo is None:
+            continue
+        try:
+            delta = when - now
+        except Exception:
+            continue
+        name = str(getattr(ev, "event", "") or "US release").strip()
+        if timedelta(0) < delta <= window:
+            minutes = max(1, int(round(delta.total_seconds() / 60)))
+            text = f"US high-impact release in {minutes} min ({name})"
+            rank = abs(delta.total_seconds())
+        elif -window <= delta < timedelta(0):
+            minutes = max(1, int(round(-delta.total_seconds() / 60)))
+            text = f"US release just out ({name}, {minutes} min ago)"
+            rank = abs(delta.total_seconds())
+        else:
+            continue
+        if best is None or rank < best[0]:
+            best = (rank, text)
+    return None if best is None else best[1]
+
+
+# ── Clock ────────────────────────────────────────────────────────────
+
+def _clock_trigger(now: datetime) -> Optional[str]:
+    """Hot clock slot for `now` (UTC), or None."""
+    try:
+        moment = now.astimezone(timezone.utc) if now.tzinfo else now
+    except Exception:
+        return None
+    weekday, hour = moment.weekday(), moment.hour   # Mon=0 .. Sun=6
+    if weekday <= 4 and hour in US_DATA_HOURS:
+        return "US data hours (12:00–15:00 UTC)"
+    if weekday <= 4 and hour == ASIAN_OPEN_HOUR:
+        return "Asian open hour (01:00 UTC)"
+    if weekday == 6 and hour in SUNDAY_REOPEN_HOURS:
+        return "Sunday reopen"
+    return None
+
+
+# ── Indicator readings ───────────────────────────────────────────────
+
+def _indicator_value(analysis: Any, name: str) -> Optional[float]:
+    try:
+        ind = (analysis.indicators or {}).get(name)
+        if ind is None:
+            return None
+        value = float(ind.value)
+    except Exception:
+        return None
+    if value != value:      # NaN
+        return None
+    return value
+
+
+def _indicator_signal(analysis: Any, name: str) -> Optional[SignalStrength]:
+    try:
+        ind = (analysis.indicators or {}).get(name)
+        return None if ind is None else ind.signal
+    except Exception:
+        return None
+
+
+def _keltner_side(analysis: Any) -> Optional[str]:
+    """"above" / "below" / None — is the close outside the channel?
+
+    The engine's keltner indicator carries `value = close - kc_mid` and
+    stashes kc_upper/kc_lower/kc_mid in `key_levels` (engine.py
+    _compute_keltner). Comparing the value against the band half-widths
+    is therefore exact AND survives the squeeze branches, which
+    overwrite the indicator's signal/description but still publish the
+    same levels. The description text is only a fallback for analyses
+    that were built without key_levels.
+    """
+    value = _indicator_value(analysis, "keltner")
+    levels = getattr(analysis, "key_levels", None) or {}
+    try:
+        upper = levels.get("kc_upper")
+        lower = levels.get("kc_lower")
+        mid = levels.get("kc_mid")
+        if value is not None and None not in (upper, lower, mid):
+            if value > float(upper) - float(mid):
+                return "above"
+            if value < float(lower) - float(mid):
+                return "below"
+            return None
+    except Exception:
+        pass
+    try:
+        desc = str((analysis.indicators or {})["keltner"].description).lower()
+    except Exception:
+        return None
+    if "above keltner upper" in desc:
+        return "above"
+    if "below keltner lower" in desc:
+        return "below"
+    return None
+
+
+def _collect_readings(analysis: Any) -> Tuple[List[str], List[str], int, int]:
+    """(price_extreme_phrases, context_phrases, bearish, bullish).
+
+    Only the PRICE-EXTREME group earns a level (see the module
+    docstring); the context group is printed alongside once a level
+    exists but can never create one. Both groups feed direction.
+    """
+    extreme: List[str] = []
+    context: List[str] = []
+    bear = bull = 0
+    if analysis is None:
+        return extreme, context, bear, bull
+
+    # ── Price extremes (level-bearing) ──
+    bb = _indicator_value(analysis, "bollinger")
+    if bb is not None:
+        if bb < 0:
+            extreme.append("price below the Bollinger bands")
+            bear += 1
+        elif bb > 1:
+            extreme.append("price above the Bollinger bands")
+            bull += 1
+
+    rsi = _indicator_value(analysis, "rsi")
+    if rsi is not None:
+        if rsi < 30:
+            extreme.append(f"RSI {rsi:.0f}, oversold extreme")
+            bear += 1
+        elif rsi >= 70:
+            extreme.append(f"RSI {rsi:.0f}, overbought extreme")
+            bull += 1
+
+    kc_side = _keltner_side(analysis)
+    if kc_side is not None:
+        extreme.append("price outside the Keltner channel")
+        if kc_side == "above":
+            bull += 1
+        else:
+            bear += 1
+
+    # ── Context (never raises the level) ──
+    slope = getattr(analysis, "_adx_slope", None)
+    try:
+        if slope is not None and float(slope) > ADX_SLOPE_HOT:
+            context.append("trend strength rising")
+    except (TypeError, ValueError):
+        pass
+
+    if str(getattr(analysis, "volatility_regime", "")) == "extreme":
+        context.append("volatility extreme")
+
+    regime = str(getattr(analysis, "market_regime", ""))
+    if regime == "transitioning":
+        context.append("regime transitioning")
+
+    macd = _indicator_signal(analysis, "macd")
+    if macd == SignalStrength.STRONG_SELL:
+        context.append("downside momentum accelerating")
+        bear += 1
+    elif macd == SignalStrength.STRONG_BUY:
+        context.append("upside momentum accelerating")
+        bull += 1
+
+    # Regime contributes direction only — it has no phrase of its own
+    # beyond "regime transitioning" above.
+    if regime == "trending_down":
+        bear += 1
+    elif regime == "trending_up":
+        bull += 1
+
+    return extreme, context, bear, bull
+
+
+# ── Assessment ───────────────────────────────────────────────────────
+
+def assess(report: Any,
+           now: datetime,
+           events: Optional[List[Any]] = None,
+           primary_tf: Optional[Timeframe] = None) -> ViolentWatch:
+    """Score violent-move conditions. Every input may be missing or
+    malformed — the answer then degrades to "no trigger". Never raises."""
+    triggers: List[str] = []
+    kinds: List[str] = []
+
+    calendar = _calendar_trigger(events, now)
+    if calendar:
+        triggers.append(calendar)
+        kinds.append("calendar")
+
+    clock = _clock_trigger(now)
+    if clock:
+        triggers.append(clock)
+        kinds.append("clock")
+
+    analysis = getattr(report, "primary_analysis", None)
+    extreme_phrases, context_phrases, bear, bull = _collect_readings(analysis)
+    triggers.extend(extreme_phrases)
+    readings = len(extreme_phrases)
+    if readings >= 1:
+        kinds.append("extreme")
+
+    # Level rule (bar-by-bar validated, see module docstring): only
+    # price extremes and the clock/calendar create a level.
+    if calendar or (clock and readings >= 1):
+        level = _LEVEL_HOT
+    elif clock or readings >= 2:
+        level = _LEVEL_WATCH
+    else:
+        level = _LEVEL_NONE
+
+    # Context readings are colour, printed only once a level exists.
+    if level != _LEVEL_NONE:
+        triggers.extend(context_phrases)
+
+    direction = ""
+    if bear >= 2 and bear > bull:
+        direction = "down"
+    elif bull >= 2 and bull > bear:
+        # Bullish readings only earned a call on M5 in the study.
+        tf_value = getattr(primary_tf, "value", primary_tf)
+        if tf_value == Timeframe.M5.value:
+            direction = "up"
+
+    return ViolentWatch(level=level, triggers=triggers, kinds=kinds,
+                        readings=readings, likely_direction=direction)
+
+
+def _log_watch(report: Any, watch: "ViolentWatch") -> None:
+    """Persist the assessment onto the scan's signals.db row (fail-safe).
+
+    The row is inserted inside analyze_pair — BEFORE a surface attaches
+    the watch — so these columns arrive by post-hoc update, exactly like
+    the panel verdict. Level and kinds are written even when the display
+    line is off: evaluating the watch needs the assessment, not the text.
+    """
+    try:
+        from signal_log import log_watch_update
+        log_watch_update(report, watch)
+    except Exception:
+        logger.debug("violent watch: logging failed", exc_info=True)
+
+
+def attach_watch(report: Any,
+                 primary_tf: Optional[Timeframe],
+                 settings: Dict[str, Any],
+                 now: Optional[datetime] = None) -> ViolentWatch:
+    """Stamp `report.watch_line` with the display line (advisory only).
+
+    Contract: NEVER raises. A broken watch must not cost an alert send
+    or a dashboard publish — a failure logs and returns level "none".
+    """
+    try:
+        settings = settings or {}
+        if not settings.get("enabled", False):
+            return ViolentWatch()
+        moment = now or datetime.now(timezone.utc)
+        events: Optional[List[Any]] = None
+        try:
+            # Lazy import keeps this module import-light and free of any
+            # signal-path dependency. The calendar rides NewsEngine's
+            # class-level cache, so a warm cache costs no network I/O.
+            from config import PlatformConfig
+            from news_engine import NewsEngine
+            events = NewsEngine(PlatformConfig().news).get_week_events(["USD"])
+        except Exception as exc:
+            logger.debug("violent watch: calendar unavailable (%s)", exc)
+        watch = assess(report, moment, events, primary_tf)
+        if settings.get("alert_line", True):
+            report.watch_line = watch.line()
+        _log_watch(report, watch)
+        return watch
+    except Exception:
+        logger.exception("violent watch: assessment failed")
+        return ViolentWatch()
+
+
+def should_send_headsup(state: Dict[str, Any],
+                        now: datetime,
+                        kinds: List[str],
+                        max_per_day: int) -> Tuple[bool, str]:
+    """Rate-limit the standalone heads-up message.
+
+    `state` is a mutable dict owned by the caller:
+        {"day": date, "count": int, "sent": {(hour_iso, kinds): True}}
+    One message per UTC hour per trigger combination, capped per UTC day.
+    """
+    day = now.date()
+    if state.get("day") != day:
+        state["day"] = day
+        state["count"] = 0
+        state["sent"] = {}
+    sent = state.setdefault("sent", {})
+    if int(state.get("count", 0)) >= int(max_per_day):
+        return False, "daily cap"
+    hour_bucket = now.replace(minute=0, second=0, microsecond=0).isoformat()
+    key = (hour_bucket, tuple(sorted(kinds)))
+    if key in sent:
+        return False, "already sent this hour for these triggers"
+    sent[key] = True
+    state["count"] = int(state.get("count", 0)) + 1
+    return True, "ok"