Daily-trend veto on M15 + lot-size / already-open lines (campaign 2026-09-30), and the 2026-10-01 whole-codebase bug review

Two bodies of work that touch the same files, so one commit.

Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule,
written before the runs):
- SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto
  "both"). A finished setup against a clear daily trend (closed D1 bars, Sunday
  stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier)
  is removed as the last step of the core. One definition
  (signal_aggregator.daily_trend_series) for live, the bar replay and the twin's
  host side; D1 is a required frame of the M15 scan; mirrored in
  cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs
  1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs
  2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R.
- FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1
  (−9.7R), buy-only and sell-only on M15, the wide-stop cap
  (xau_strategy.wide_stop_cap_atr stays 0.0, a research knob).
- New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins
  unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15
  pins byte for byte. Twin and reference script MD5-equal on every window;
  recorder and feature-export parity re-proved; base-rate table rebuilt.
- DISPLAY (books byte-identical): exposure.py — the lot size for
  trade_size.risk_per_trade (new config.json block, off by default, private to
  the owner) and the count of alerted same-direction plans still open
  (signals.alerted_at, read-only open_alerted_setups); the daily trend in words
  on Telegram, the CLI and the dashboard; signals.daily_trend logged.
- providers/oanda: a date window with no bars is an empty page, not an error
  (a D1 pad starting on a Saturday could not be fetched).
- Reports only, nothing shipped: seven context screens (no state nominated) and
  eight outside data sources (none adds a forecast to gold's own daily trend).
- Independent verification (report section 5.13): every figure reproduced on
  both scripts, scoring identical to the previous commit on all four
  timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not
  a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022;
  the neighbour clause is thin (a simple 100-day average misses the R line by
  1.2R); "never adds a trade" is an observation, not a guarantee.

Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects
elsewhere, plus 4 found by an independent review of the fixes; all fixed, each
with a test; no trade book changed:
- telegram_bot: one _deliver_alert for the candle scan and the level break. The
  row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and
  un-stamped when no chat received it or the task is cancelled mid-stamp, so
  two alerts at one candle close count each other and no SQLite write runs on
  the event loop. The 5-bar cooldown counts bars of the scan's own frame
  (== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after
  the settlement pause) and is restored from signals.db at start-up. The
  lot-size line goes to the owner's own chat only (a group the owner ran
  /autoscan in used to receive it). /status gains a news-model line and escapes
  failure details.
- signal_log: mark_alerted returns the stamp it wrote; unmark_alerted;
  last_alerted_bars (read-only).
- advisory_outcome: candle continuity is judged bar by bar inside the walk; a
  hole after a trade's exit no longer voids it. The stored-candle replay
  resolves 108 of 111 logged setups instead of 99, all equal to the live
  resolver's R.
- providers/oanda: an empty date window is accepted only when it is 5 days or
  shorter and no malformed candle was skipped; a longer one is retried, then
  refused, instead of being cached as history with a hole.
- news_engine: blank RSS titles skipped; one unreadable headline no longer
  costs the local model its whole batch (checked on the real bundle); model
  health recorded; no DeepSeek audit call in a process that does not archive;
  event-direction cache keyed per release; peek_week_events never fetches.
- llm_client: chat_json never raises and never logs the API key (requests
  quotes the Authorization value in InvalidHeader; error bodies are redacted).
- signal_aggregator: the scan's news block is guarded; the replay path
  (use_news=False) assigns exactly the values it did before.
- agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input
  (event timing over the cached calendar week, headlines newest first).
  scripts/panel_replay.py replays the logged payload per persona;
  scripts/label_scan_outcomes.py counts with an ISO cutoff.
- test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34);
  tests that write Parquet print a notice and return without pyarrow.

Verification after the last code change: suite 632 passed, 0 failed (also with
both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the
five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37
green; static Python 3.9 / SQLite 3.34 check clean on every touched file.

Evidence: backtest/research/daily_trend_campaign_2026-09-30.md,
context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6,
8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
2026-10-01 10:28:28 -04:00
parent 0c56b4e155
commit 1793d9167a
31 changed files with 6021 additions and 286 deletions
+206 -37
View File
@@ -23,7 +23,7 @@ cp config.json.example config.json # then fill i
venv/bin/pip install cupy-cuda12x "nvidia-cuda-runtime-cu12==12.9.*" # optional, dev box: backtester GPU backend venv/bin/pip install cupy-cuda12x "nvidia-cuda-runtime-cu12==12.9.*" # optional, dev box: backtester GPU backend
# (CuPy from pip needs that header wheel to compile kernels) # (CuPy from pip needs that header wheel to compile kernels)
venv/bin/python test_all.py # QA suite, no network; prints its own count venv/bin/python test_all.py # QA suite, no network; prints its own count
# (581 tests / 38 modules as of 2026-09-27) # (632 tests / 40 modules as of 2026-10-01)
venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail # one signal report in the terminal venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail # one signal report in the terminal
# --json full dump, --panel adds the advisory LLM panel, # --json full dump, --panel adds the advisory LLM panel,
# --timeframe M5 | M15 | H1 | H4 # --timeframe M5 | M15 | H1 | H4
@@ -48,7 +48,10 @@ unknown path is a hard exit), `--export-features` (appends the 65 decision-time
it on a parity run because it changes the file), `--fixed-rr-tp N` (research only, not shipped), it on a parity run because it changes the file), `--fixed-rr-tp N` (research only, not shipped),
`--set signals.grading_mode=min|product` (research only, 2026-09-27: grade on one conviction number `--set signals.grading_mode=min|product` (research only, 2026-09-27: grade on one conviction number
instead of the score + confidence floors; both FAILED rule 1 on M5 and M15, default `pair` is the instead of the score + confidence floors; both FAILED rule 1 on M5 and M15, default `pair` is the
shipped rule and byte-identical), `--record DIR [--record-bars]` (research recorder, see section 9), `--backend auto|gpu|cpu` (`auto` shipped rule and byte-identical), `--set xau_strategy.daily_trend_veto=off|both|buy|sell` (2026-09-30:
the M15 factory ships `both`; pass `off` to reproduce a pre-veto M15 book; `buy` / `sell` and every
mode on M5 / H1 FAILED their rule), `--set xau_strategy.wide_stop_cap_atr=K` (research only: FAILED
on 2026-09-30, ships 0 = off), `--record DIR [--record-bars]` (research recorder, see section 9), `--backend auto|gpu|cpu` (`auto`
uses the CuPy GPU backend when CUDA works and falls back to NumPy with the reason in the `Backend:` uses the CuPy GPU backend when CUDA works and falls back to NumPy with the reason in the `Backend:`
line; `gpu` refuses to fall back; `cpu` forces the NumPy twins for a parity run; the env var line; `gpu` refuses to fall back; `cpu` forces the NumPy twins for a parity run; the env var
`FB2_BACKEND` sets the default). Both backends produce byte-identical books (section 9). `FB2_BACKEND` sets the default). Both backends produce byte-identical books (section 9).
@@ -75,6 +78,8 @@ same schedule. `main.py` runs once when you call it.
(`app_config.py`), then applies the validated per-timeframe overrides (section 8). Never build a (`app_config.py`), then applies the validated per-timeframe overrides (section 8). Never build a
bare `PlatformConfig()` for a real scan; always go through the factory. The confirmation bare `PlatformConfig()` for a real scan; always go through the factory. The confirmation
timeframes come from the factory map: M5 → M15 + H1, M15 → M5 + H1, H1 → M15 + H4, H4 → H1 + D1. timeframes come from the factory map: M5 → M15 + H1, M15 → M5 + H1, H1 → M15 + H4, H4 → H1 + D1.
Since 2026-09-30 an M15 scan also needs the daily (D1) frame, not as a confirmation but as the
input of the daily-trend veto (step 8); it is a required frame like the others.
**3. Fetch candles.** `signal_aggregator.analyze_pair` (the live entry point) first checks the **3. Fetch candles.** `signal_aggregator.analyze_pair` (the live entry point) first checks the
calendar: on the forex weekend or a blocked weekday it returns a "blocked" scan and does nothing calendar: on the forex weekend or a blocked weekday it returns a "blocked" scan and does nothing
@@ -146,6 +151,16 @@ M5 six extra noise gates (minimum score, required agreement, confirmation streng
cap, chop-transition volatility, SELL RSI floor). Every gate that removes a setup writes a warning cap, chop-transition volatility, SELL RSI floor). Every gate that removes a setup writes a warning
that names itself. that names itself.
One more gate runs at the very end, after the setup generators and the late entry paths of step 9
(`_apply_daily_trend_veto`, step 12 in the code; M15 only since 2026-09-30): the **daily-trend
veto**. The daily trend is read from closed D1 bars with the provider's Sunday stub bars dropped
(`signal_aggregator.daily_trend_series`): +1 when the last daily close is above its 50-day EMA and
above the close 60 kept days earlier, −1 when below both, else 0. A finished setup that goes
against a non-zero state is removed (grade D, no setup, warning `DAILY TREND FILTER: BUY against a
falling daily trend — no trade`). It runs after every archetype on purpose, so a chart-pattern
setup is vetoed too. Where the veto is off (M5, H1, H4) the live wrapper still stamps
`report.daily_trend` for the alert text and the signal log; nothing reads it back there.
**9. Trade setup.** Grades A+ and A get the gold ATR setup (`_generate_xau_atr_trade_setup`): stop **9. Trade setup.** Grades A+ and A get the gold ATR setup (`_generate_xau_atr_trade_setup`): stop
2 × ATR; target 1 at 2 × ATR for A+ and 1.5 × ATR for A (1 × ATR for an A carrying two or more 2 × ATR; target 1 at 2 × ATR for A+ and 1.5 × ATR for A (1 × ATR for an A carrying two or more
exhaustion tags); target 2 at 1.5 × the target-1 distance. Other tradeable grades get the generic swing-based setup, exhaustion tags); target 2 at 1.5 × the target-1 distance. Other tradeable grades get the generic swing-based setup,
@@ -175,7 +190,12 @@ later generator shipped a setup.
reaches `alert_min_quality`, **and** a setup exists. Since 2026-09-26 no second alert goes out on a reaches `alert_min_quality`, **and** a setup exists. Since 2026-09-26 no second alert goes out on a
timeframe within 5 bars of the last alerted setup's bar (`_ALERT_COOLDOWN_BARS`, the backtester's timeframe within 5 bars of the last alerted setup's bar (`_ALERT_COOLDOWN_BARS`, the backtester's
entry cooldown; consecutive-bar re-entries were −25.1R of the −37.8R live book of 2026-09-04 → entry cooldown; consecutive-bar re-entries were −25.1R of the −37.8R live book of 2026-09-04 →
09-25). After that, a fingerprint (signal + direction) stops repeats; a grade upgrade re-alerts. 09-25). Since 2026-10-01 the five bars are COUNTED in the scan's own primary frame
(`_primary_bars_since`, the backtester's position difference; clock time is only the fallback
for a report without frames) and the last alerted bar per timeframe is read back from
`signals.db` at start-up (`signal_log.last_alerted_bars` → `_restore_alert_cooldowns`), so
neither the settlement pause, the weekend nor a restart lets live alert where the books do not
enter. After that, a fingerprint (signal + direction) stops repeats; a grade upgrade re-alerts.
Since 2026-09-27 the header leads with the grade (`Grade A · BUY · XAU/USD · M15`) and the raw Since 2026-09-27 the header leads with the grade (`Grade A · BUY · XAU/USD · M15`) and the raw
score / confidence line is replaced by words from `main.plain_words` (`Direction: strong buy · score / confidence line is replaced by words from `main.plain_words` (`Direction: strong buy ·
Reliability: good · timeframes agree`; the cut-offs are the grade table's own, so the words never Reliability: good · timeframes agree`; the cut-offs are the grade table's own, so the words never
@@ -187,12 +207,29 @@ later generator shipped a setup.
(`_book_rate_line`, from `agent_panel_base_rates.json`). The fingerprint is committed only after a successful send, (`_book_rate_line`, from `agent_panel_base_rates.json`). The fingerprint is committed only after a successful send,
so a failed send stays eligible next scan. Every send is logged with the signal-log row id. The so a failed send stays eligible next scan. Every send is logged with the signal-log row id. The
level-breach monitor reuses the bar's cached M15 report (`_cached_or_scan`) instead of rescanning. level-breach monitor reuses the bar's cached M15 report (`_cached_or_scan`) instead of rescanning.
- *CLI* (`main.py`): prints the report; `--json` dumps everything. Since 2026-09-30 three more display lines: the daily trend in words under the regime line
(`🧭 Daily trend: rising — this buy goes with it`, `main.daily_trend_words`), and inside the setup
block the lot size for the owner's risk limit (`📏`, off until `trade_size.risk_per_trade` is set)
and the count of alerted same-direction plans still open (`🧱`). The size line is private:
`format_signal_message(report, channel=True)` builds the copy without it unless
`trade_size.telegram_channel` is true, and since 2026-10-01 `_broadcast` sends the owner copy to
the owner's own chat only (`_is_owner_chat`): the channel and any other target (a group the owner
used `/autoscan` or a scan button in; the owner lock is per user) get the copy without it.
Both alert paths (candle scan, level break) end in one `_deliver_alert`: the panel, then on the
one-worker `alert-db` pool the exposure lines are refreshed and the row is stamped `alerted_at`
(`signal_log.mark_alerted`), then the send. Stamping BEFORE the send is what lets two alerts at
the same candle close count each other; an alert that reaches no chat clears its own stamp
(`unmark_alerted`) and commits nothing. No database write runs on the event loop.
- *CLI* (`main.py`): prints the report; `--json` dumps everything (including `daily_trend`,
`daily_trend_words`, `size_line`, `open_line`).
- *Dashboard* (`dashboard_publisher.py`): writes `snapshot.json`, `news.json` and the static page - *Dashboard* (`dashboard_publisher.py`): writes `snapshot.json`, `news.json` and the static page
into the Apache folder. into the Apache folder. Each card shows the daily-trend words; a plan shows the already-open
line, and the size line only when `trade_size.dashboard` is true (the page is public).
Two advisory add-ons are attached **after** the scan by the surface, never inside the signal Three display add-ons are attached **after** the scan by the surface, never inside the signal
code: the LLM agent panel (`agent_panel.py`; since 2026-09-17 only the **macro** persona runs by code. The first is `exposure.attach_exposure` (2026-09-30): the lot-size line and the
already-open line, a leaf module that imports only `app_config` and `signal_log`. The other two are
advisory: the LLM agent panel (`agent_panel.py`; since 2026-09-17 only the **macro** persona runs by
default, `agent_panel.personas: ["macro"]`, one DeepSeek call per panel — the technical, risk and default, `agent_panel.personas: ["macro"]`, one DeepSeek call per panel — the technical, risk and
contrarian personas carried no information in two live evaluations and stay available for research; contrarian personas carried no information in two live evaluations and stay available for research;
the persona returns a probability that the trade closes profitably, anchored to the setup class's the persona returns a probability that the trade closes profitably, anchored to the setup class's
@@ -207,22 +244,30 @@ replayable core never sets it.
**11. Logging.** `signal_log.py` writes one row per live scan (from the bot and the CLI; the **11. Logging.** `signal_log.py` writes one row per live scan (from the bot and the CLI; the
dashboard process never writes either database, section 3) into `logs/signals.db` (signal, dashboard process never writes either database, section 3) into `logs/signals.db` (signal,
grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the
gate that removed a setup, a gate-config fingerprint and an engine fingerprint). The engine gate that removed a setup, a gate-config fingerprint and an engine fingerprint; since 2026-09-30
also `daily_trend` on every real scan and `alerted_at` on rows whose alert reached a chat). The engine
fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log
comparisons across such a deploy must be split at the deploy time (the 2026-09-09 M15 RSI change is comparisons across such a deploy must be split at the deploy time (the 2026-09-09 M15 RSI change is
one). `news_archive.py` writes every headline and one). The gate fingerprint changes when a gate ships: the M15 daily-trend veto moved M15 rows from
`f9b2f801de54` to `8a8075688d67` (engine fingerprint unchanged), so M15 comparisons split there too.
`news_archive.py` writes every headline and
its score into `logs/newsdata.db`. Both are fail-safe: a logging error can never break a scan. its score into `logs/newsdata.db`. Both are fail-safe: a logging error can never break a scan.
**12. Measurement, later.** Two models read the stored candles and score what happened next: **12. Measurement, later.** Two models read the stored candles and score what happened next:
advisory outcomes (`advisory_outcome.py`: replays each logged setup with next-open entry, stop advisory outcomes (`advisory_outcome.py`: replays each logged setup with next-open entry, stop
first, close-only trailing; a cleaned-mid evidence model, not execution P&L) and scan outcomes first, close-only trailing; a cleaned-mid evidence model, not execution P&L) and scan outcomes
(model-free forward moves 1 h / 4 h / 24 h after every scan). `analyze_signal_log.py` turns the (model-free forward moves 1 h / 4 h / 24 h after every scan). `analyze_signal_log.py` turns the
database into reports. database into reports. Since 2026-10-01 the outcome model checks candle continuity bar by bar as
it walks: a hole in the stored candles AFTER a trade's exit no longer voids the trade, and a hole
reached while it is open ends the walk with the progress kept (so the locked-profit floor is the
same whether the stream arrives in one pass or many).
**The backtester is the same pipeline on history.** `backtest/gpu_backtest.py` loads candles from **The backtester is the same pipeline on history.** `backtest/gpu_backtest.py` loads candles from
`backtest/cache/*.parquet` (fetched once through the same provider code), precomputes every `backtest/cache/*.parquet` (fetched once through the same provider code), precomputes every
indicator series over the whole window, and calls `analyze_from_dataframes` bar by bar with news indicator series over the whole window, and calls `analyze_from_dataframes` bar by bar with news
off. Trades are resolved on the raw price path with next-open entries, spread charged per trade, off. When the daily-trend veto is on it also loads the D1 frame (1,400-day pad) and slices it as-of
like every other frame, so the veto sees exactly the closed daily bars a live scan would. Trades
are resolved on the raw price path with next-open entries, spread charged per trade,
stop-first ties, gap fills at the open, and the trailing stop moved at bar close only. Time filters stop-first ties, gap fills at the open, and the trailing stop moved at bar close only. Time filters
use bar-close time in both paths, so live and backtest agree. The replay fans out over worker use bar-close time in both paths, so live and backtest agree. The replay fans out over worker
processes; the 5-bar entry cooldown, the only cross-bar state, is applied in an ordered pass processes; the 5-bar entry cooldown, the only cross-bar state, is applied in an ordered pass
@@ -238,9 +283,10 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| File | Role | Uses | Used by | | File | Role | Uses | Used by |
|---|---|---|---| |---|---|---|---|
| `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`) | pair_signals, signal_aggregator, agent_panel, violent_watch, app_config | dashboard_publisher (formatting helpers) | | `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`); the display helpers every surface shares (`plain_words`, `daily_trend_words`) | pair_signals, signal_aggregator, agent_panel, violent_watch, exposure, app_config | dashboard_publisher and telegram_bot (formatting helpers) |
| `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) | | `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, exposure, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes `signals.db` or `newsdata.db`: `main()` calls `app_config.set_db_writes_for_process(False)`, a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running) | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` (test-enforced) | | `exposure.py` | Display-only lot-size line and already-open line (`attach_exposure`, 2026-09-30). `lot_size` floors to the lot step in Decimal so the risk limit is never exceeded; `size_shared` answers whether config.json shares the size with a public audience. A leaf: never raises, never read by the core | app_config, signal_log | main, telegram_bot, dashboard_publisher |
| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes `signals.db` or `newsdata.db`: `main()` calls `app_config.set_db_writes_for_process(False)`, a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running). It does READ `signals.db` (`mode=ro`) for the already-open line since 2026-09-30 | pair_signals, providers, main, agent_panel, violent_watch, exposure, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` (test-enforced) |
| `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`). Bootstrap 5 + jQuery + Chart.js + Font Awesome straight from a CDN, no build step; M5/M15/H1/H4 each get a card in one column grid, with a diverging Chart.js bar chart for direction and one per card for the indicator votes | | copied by the publisher | | `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`). Bootstrap 5 + jQuery + Chart.js + Font Awesome straight from a CDN, no build step; M5/M15/H1/H4 each get a card in one column grid, with a diverging Chart.js bar chart for direction and one per card for the indicator votes | | copied by the publisher |
| `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher | | `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher |
| `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | | | `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | |
@@ -253,7 +299,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| `pair_signals/xauusd.py` | The config factory: overlay + validated per-timeframe overrides + confirmation map. Its docstring is the evidence log for every override | config, app_config, engine | main, telegram_bot, dashboard_publisher, backtester | | `pair_signals/xauusd.py` | The config factory: overlay + validated per-timeframe overrides + confirmation map. Its docstring is the evidence log for every override | config, app_config, engine | main, telegram_bot, dashboard_publisher, backtester |
| `pair_signals/__init__.py` | Registry of pair producers (`PAIR_PRODUCERS`) and `produce_signal` | config, signal_aggregator, providers, xauusd | the surfaces, backtester | | `pair_signals/__init__.py` | Registry of pair producers (`PAIR_PRODUCERS`) and `produce_signal` | config, signal_aggregator, providers, xauusd | the surfaces, backtester |
| `engine.py` | Indicators, regime, composite score (`TechnicalEngine`) | config | signal_aggregator, backtester, replay_record, pair factory | | `engine.py` | Indicators, regime, composite score (`TechnicalEngine`) | config | signal_aggregator, backtester, replay_record, pair factory |
| `signal_aggregator.py` | Multi-timeframe merge, penalties, grade, gate chain, setup generators, reasons; `analyze_pair` (live) and `analyze_from_dataframes` (replayable core) | engine, chart_patterns, news_engine, providers, signal_log, config | main, telegram_bot, backtester, pair_signals, analyze_signal_log (agent_panel only for type hints) | | `signal_aggregator.py` | Multi-timeframe merge, penalties, grade, gate chain, setup generators, reasons, the daily-trend veto and its one definition (`daily_trend_series` / `daily_trend` / `daily_trend_needed`, shared by live, the replay and the twin's host side); `analyze_pair` (live) and `analyze_from_dataframes` (replayable core) | engine, chart_patterns, news_engine, providers, signal_log, config | main, telegram_bot, backtester, pair_signals, analyze_signal_log (agent_panel only for type hints) |
| `chart_patterns.py` | The XAU_CP pattern detector | | signal_aggregator | | `chart_patterns.py` | The XAU_CP pattern detector | | signal_aggregator |
| `position_sizing.py` | Decision-time feature builder (65 columns) and the reserved stake multiplier (always 1.0). Bounded lookbacks keep live == backtest | config | backtester (`--export-features`) | | `position_sizing.py` | Decision-time feature builder (65 columns) and the reserved stake multiplier (always 1.0). Bounded lookbacks keep live == backtest | config | backtester (`--export-features`) |
@@ -272,9 +318,9 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| File | Role | | File | Role |
|---|---| |---|---|
| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; headline scores upgraded by the local XAU model (`news_model.py`, mode `primary` live since 2026-09-27) or by DeepSeek (modes `off` / `fallback`, `_enrich_headline_scores`); DeepSeek keeps event direction and the summary; display and warnings only | | `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; headline scores upgraded by the local XAU model (`news_model.py`, mode `primary` live since 2026-09-27) or by DeepSeek (modes `off` / `fallback`, `_enrich_headline_scores`); DeepSeek keeps event direction and the summary; display and warnings only |
| `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback | | `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback. `chat_json` never raises and never logs the API key (2026-10-01: a key with a line break made `requests` quote the `Authorization` value in its error text); an unserialisable payload is refused before the cap is counted |
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call. **Frozen in `news_model.mode` `primary` (live since 2026-09-27)**: the local model's output is not training data, so nothing is archived except the optional DeepSeek audit copies (`deepseek_audit_per_day`, shipped 0); `off` / `fallback` archive as before, a `fallback` model row carrying `teacher_model` `xau_transformer@<trained_at>` | | `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call. **Frozen in `news_model.mode` `primary` (live since 2026-09-27)**: the local model's output is not training data, so nothing is archived except the optional DeepSeek audit copies (`deepseek_audit_per_day`, shipped 0); `off` / `fallback` archive as before, a `fallback` model row carrying `teacher_model` `xau_transformer@<trained_at>` |
| `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half | | `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. v2.2 (2026-10-01) changed the persona's INPUT, not the prompt text: the event timing reads the cached calendar week `attach_panel` hands to `_price_context` (`NewsEngine.peek_week_events`, which never fetches; a hold past UTC midnight sees tomorrow's release), and the ten headlines are the newest ten, not the first ten in feed order. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
| `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc | | `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |
| `news_model.py` | The local headline scorer (2026-09-27): vendored CPU runtime of the owner's ML-NEWS XAU transformer (DeBERTa-v3-small, two heads: moves gold? / up or down?, ONNX; `onnxruntime` + `tokenizers` imported lazily inside `load_model`, so the bot starts without them) plus a load-once, never-raising singleton (`get_news_model`, `teacher_tag`, `warm_up_in_background`). Gold head only: `news_engine._apply_model_sentiment` writes `xau_score` and XAU's per-currency entry, never USD. Bundle (280 MB) lives outside the tree at `news_model.path`. Spec and deviations: `docs/news_model_integration_spec_2026-09-27.md`; ops: `deploy/README-news-model.md` | | `news_model.py` | The local headline scorer (2026-09-27): vendored CPU runtime of the owner's ML-NEWS XAU transformer (DeBERTa-v3-small, two heads: moves gold? / up or down?, ONNX; `onnxruntime` + `tokenizers` imported lazily inside `load_model`, so the bot starts without them) plus a load-once, never-raising singleton (`get_news_model`, `teacher_tag`, `warm_up_in_background`). Gold head only: `news_engine._apply_model_sentiment` writes `xau_score` and XAU's per-currency entry, never USD. Bundle (280 MB) lives outside the tree at `news_model.path`. Spec and deviations: `docs/news_model_integration_spec_2026-09-27.md`; ops: `deploy/README-news-model.md` |
@@ -282,14 +328,14 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| File | Role | | File | Role |
|---|---| |---|---|
| `signal_log.py` | SQLite logger for every live scan (`signals`, `news_snapshots`, `scan_outcomes`; view `signals_with_news`); opportunistic scan-outcome labelling; advisory outcome seeding | | `signal_log.py` | SQLite logger for every live scan (`signals`, `news_snapshots`, `scan_outcomes`; view `signals_with_news`); opportunistic scan-outcome labelling; advisory outcome seeding; `mark_alerted` (stamps `alerted_at` just before the alert is sent and returns the stamp it wrote), `unmark_alerted` (clears that stamp when no chat received the alert), the read-only `open_alerted_setups` (alerted plans whose outcome is still `PENDING`, one per event, stale rows dropped) and the read-only `last_alerted_bars` (newest alerted bar per timeframe, for the bot's cooldown after a restart) |
| `advisory_outcome.py` | The deterministic outcome model for logged setups | | `advisory_outcome.py` | The deterministic outcome model for logged setups. Continuity of the stored candles is judged one step at a time inside the walk (`_next_bar_follows`); `_has_continuity` stays all-or-nothing for the scan-outcome labeller |
| `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) | | `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) |
| `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) | | `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) |
| `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) | | `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) |
| `scripts/panel_replay.py` | Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (exact `panel_payloads_json` when present, else from the stored JSON blobs, interned news and the kept Oanda cache), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. `--dry-run`, `--max-calls`, `--limit`, `--out`. Read-only on the DB; 4 DeepSeek calls per event | | `scripts/panel_replay.py` | Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (the exact logged one from `panel_payloads_json` per persona when the row has it, macro alone on live rows since 2026-09-17; else from the stored JSON blobs, interned news and the kept Oanda cache; the CSV column `stored_payloads` names the personas that replayed a logged payload), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. `--dry-run`, `--max-calls`, `--limit`, `--out`. Read-only on the DB; 4 DeepSeek calls per event |
| `scripts/build_panel_base_rates.py` | Builds `agent_panel_base_rates.json` (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin | | `scripts/build_panel_base_rates.py` | Builds `agent_panel_base_rates.json` (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin |
| `agent_panel_base_rates.json` | The base-rate table the panel anchors its probabilities to (built 2026-09-17 from the relabelled W2/W3 books) | | `agent_panel_base_rates.json` | The base-rate table the panel anchors its probabilities to (rebuilt 2026-10-01 from the reference books of section 9, the M15 one with the daily-trend veto; `generated_at` and each book's MD5 are in the file) |
### Backtesting and research ### Backtesting and research
@@ -313,7 +359,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys | | `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
| `config.json.example` | The schema of `config.json` (which is gitignored) | | `config.json.example` | The schema of `config.json` (which is gitignored) |
| `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon | | `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
| `test_all.py` | The QA suite (38 modules; Module 36 is the GPU backend parity module, Module 37 the CUDA decision-stage parity module, Module 38 the local news model) | | `test_all.py` | The QA suite (40 modules; Module 36 is the GPU backend parity module, Module 37 the CUDA decision-stage parity module, Module 38 the local news model, Module 39 the daily-trend veto and the wide-stop cap, Module 40 the trade-size / already-open / daily-trend display lines) |
--- ---
@@ -350,7 +396,20 @@ symbol and interval maps. A test enforces that subclasses do not override the sh
scheduled closure since 2026-09-13. A "stale data" warning at any other weekday hour is a real scheduled closure since 2026-09-13. A "stale data" warning at any other weekday hour is a real
feed stall. feed stall.
- The backtester fetches history through `fetch_range` (paginated, retries a failed page, logs an - The backtester fetches history through `fetch_range` (paginated, retries a failed page, logs an
error for interior gaps longer than 3 bars) and stores it under `backtest/cache/`. error for interior gaps longer than 3 bars) and stores it under `backtest/cache/`. A DATE window
with no bars in it (a closed day at the head of a padded window) is an empty page, not a failure:
until 2026-09-30 the Oanda provider raised on it and `fetch_range` aborted after three tries, so
a D1 pad starting on a Saturday could not be fetched at all. Only a closure-sized window gets
that reading (`EMPTY_WINDOW_MAX`, 5 days: a weekend joined to a holiday, and no malformed candle
skipped); an empty answer for a longer window (a normal 15-day M5 page) raises, so `fetch_range`
retries it and refuses the history after three tries instead of caching a hole (2026-10-01).
A `count` request (the live path) with nothing in it still raises.
- Daily (D1) candles are UTC days (`dailyAlignment=0`). Oanda also emits a 1–2 hour stub bar
labelled Sunday for the reopening (about 600 of them since 2015); it is a real bar for the H4
confirmation engine, and the daily-trend definition drops Sunday labels before it computes
anything. Since 2026-09-30 D1 is a required frame of every M15 scan (live and backtest) and is
fetched for display on M5 and H1 (cached until the next daily close; a failure there leaves the
line empty and never fails the scan).
--- ---
@@ -377,6 +436,7 @@ gates live in `config.py` and the pair factory, never in `config.json`. Blocks (
| `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) | | `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
| `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) | | `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
| `news_model` | `path` (bundle directory, required when mode ≠ off), `threads` (default 4), `mode` (`off` default = DeepSeek/keyword path; `fallback` = model only where DeepSeek failed; `primary` = model scores every headline, no DeepSeek headline calls), `deepseek_audit_per_day` (default 0; primary only, archive-only DeepSeek labels). Env `NEWS_MODEL_MODE` overrides `mode` per process (the dashboard unit). Live: `primary` on both processes | | `news_model` | `path` (bundle directory, required when mode ≠ off), `threads` (default 4), `mode` (`off` default = DeepSeek/keyword path; `fallback` = model only where DeepSeek failed; `primary` = model scores every headline, no DeepSeek headline calls), `deepseek_audit_per_day` (default 0; primary only, archive-only DeepSeek labels). Env `NEWS_MODEL_MODE` overrides `mode` per process (the dashboard unit). Live: `primary` on both processes |
| `trade_size` | Lot-size guidance on the alert (display only, 2026-09-30): `risk_per_trade` (US dollars; 0 or absent = the size line is off), `contract_size` (ounces per lot, default 100), `lot_step` (default 0.01), `telegram_channel` and `dashboard` (default false: the size shows the owner's risk, so every Telegram chat other than the owner's own (`telegram.chat_id`) — the channel, a group — and the public page get it only when these are true). No env override |
| `log_level` | Logging level | | `log_level` | Logging level |
A legacy top-level `market_outlook` block is tolerated and ignored with one warning (the feature A legacy top-level `market_outlook` block is tolerated and ignored with one warning (the feature
@@ -392,7 +452,7 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
## 6. Testing ## 6. Testing
- `venv/bin/python test_all.py` runs everything with mocked network. Expect - `venv/bin/python test_all.py` runs everything with mocked network. Expect
`RESULTS: 581 passed, 0 failed, 581 total` (2026-09-27). Module 38's last test loads the `RESULTS: 632 passed, 0 failed, 632 total` (2026-10-01). Module 38's last test loads the
real news-model bundle when `NEWS_MODEL_DIR` (or the configured `news_model.path`) exists and real news-model bundle when `NEWS_MODEL_DIR` (or the configured `news_model.path`) exists and
prints SKIP otherwise. On a box with a visible CUDA device prints SKIP otherwise. On a box with a visible CUDA device
Module 36 fails if the GPU backend falls back (the reason is in the message); with Module 36 fails if the GPU backend falls back (the reason is in the message); with
@@ -403,6 +463,12 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote
fixture headlines into the real `logs/newsdata.db`, and a Module 32 test now pins the isolation. One Twelve Data interval-map test fixture headlines into the real `logs/newsdata.db`, and a Module 32 test now pins the isolation. One Twelve Data interval-map test
reaches the network and can time out on a slow connection; a rerun clears it. reaches the network and can time out on a slow connection; a rerun clears it.
- The suite is meant to run on the live box too (Python 3.9, SQLite 3.34, no `pyarrow`, no CUDA):
the four tests that must write a Parquet fixture print a notice and return when no Parquet
engine is installed, and a test that needs an older database builds it forwards
(`_legacy_signals_db39`: the base schema plus every additive column but the one under test),
never with `ALTER TABLE … DROP COLUMN` (SQLite 3.35). Verified 2026-10-01 with both Parquet
engines blocked and `FB2_BACKEND=cpu`: 0 failures.
- Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps. - Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
- When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the - When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the
other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar. other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar.
@@ -423,7 +489,13 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
fills. Marginal edges must also pass the 2024-06-01 → 2025-03-30 out-of-sample window. Backtest fills. Marginal edges must also pass the 2024-06-01 → 2025-03-30 out-of-sample window. Backtest
windows go back at most 3 years (owner preference, 2026-09-05). `round_trip_cost` stays at 0.30 windows go back at most 3 years (owner preference, 2026-09-05). `round_trip_cost` stays at 0.30
because it also drives the live target floor (0.65 would move target 1's floor from about $9 to because it also drives the live target floor (0.65 would move target 1's floor from about $9 to
$19.50); changing it is its own rule-1 event. $19.50); changing it is its own rule-1 event. A change whose purpose is to cut the losing runs
rather than add R may instead be judged by a **risk-first rule** (owner decision 2026-09-30),
written before the runs: 3-year R at least 97% of the control, profit factor up by 0.10 or more,
max drawdown not deeper, the 2019 → 2026 window better on R, on drawdown and in 2021 + 2022, an
unseen holdout, both costs, and at least 3 of 4 neighbouring definitions holding. Under it the
halves of a purely rising window are reported, not clauses. The owner chooses the rule per
campaign; the default stays the acceptance above.
2. **Per-timeframe overrides live in the pair factory**, never in the global defaults. The same 2. **Per-timeframe overrides live in the pair factory**, never in the global defaults. The same
knob has had opposite effects on different timeframes several times. Record the evidence in the knob has had opposite effects on different timeframes several times. Record the evidence in the
factory docstring. factory docstring.
@@ -466,7 +538,7 @@ overrides (each one has its validation record in the `pair_signals/xauusd.py` do
| Timeframe | Confirmations | Overrides on top of the defaults | | Timeframe | Confirmations | Overrides on top of the defaults |
|---|---|---| |---|---|---|
| M5 | M15, H1 | hour 9 UTC blocked and hour 21 unblocked; chart-pattern entries on (buy-side only: inverse head and shoulders, double bottom, bull flag); pullback entries off; partial EMA alignment scores neutral; trailing stop arms at 0.60R and trails 0.85R; shorts blocked in the `transitioning` regime | | M5 | M15, H1 | hour 9 UTC blocked and hour 21 unblocked; chart-pattern entries on (buy-side only: inverse head and shoulders, double bottom, bull flag); pullback entries off; partial EMA alignment scores neutral; trailing stop arms at 0.60R and trails 0.85R; shorts blocked in the `transitioning` regime |
| M15 | M5, H1 | target floor 1R; the mixed `trending` regime blocked; shorts blocked at 15:00 and 17:00 UTC; trailing stop arms at 0.70R and trails 0.75R; RSI weight halved in every regime; grade-A shorts blocked (`sell_min_quality` `A+, B`) | | M15 | M5, H1 (+ D1 for the veto) | target floor 1R; the mixed `trending` regime blocked; shorts blocked at 15:00 and 17:00 UTC; trailing stop arms at 0.70R and trails 0.75R; RSI weight halved in every regime; grade-A shorts blocked (`sell_min_quality` `A+, B`); daily-trend veto in both directions (`daily_trend_veto` `both`, since 2026-09-30) |
| H1 | M15, H4 | every short blocked (`sell_min_quality` empty) | | H1 | M15, H4 | every short blocked (`sell_min_quality` empty) |
| H4 | H1, D1 | none. H4 remains "unproven" (few trades) | | H4 | H1, D1 | none. H4 remains "unproven" (few trades) |
@@ -475,7 +547,13 @@ to Friday plus Sunday evening; shorts need grade A+, A or B and high-or-extreme
trailing stop arms at 0.6R and trails 0.75R; gated target floor 0.5R and printed target floor 1R; trailing stop arms at 0.6R and trails 0.75R; gated target floor 0.5R and printed target floor 1R;
round-trip cost $0.30 with round-trip cost $0.30 with
target 1 at least 30 × cost; the generic and chart-pattern setups floor the stop at $8; the gold target 1 at least 30 × cost; the generic and chart-pattern setups floor the stop at $8; the gold
ATR setup uses a 1 × ATR target before the grade scale (2× at A+, 1.5× at A) and a 2 × ATR stop; the H4 regime filter is off; news never gates. ATR setup uses a 1 × ATR target before the grade scale (2× at A+, 1.5× at A) and a 2 × ATR stop; the H4 regime filter is off; news never gates;
the daily-trend veto is off (M15 alone turns it on) and the wide-stop cap is off everywhere
(`wide_stop_cap_atr` 0.0, a research knob that failed its rule).
The daily-trend veto failed the same rule on M5 (it deepens the 3-year losing run from 10.1R to
15.2R and loses the 2019 → 2026 window) and on H1 (−9.7R of +70.0R). Do not turn it on there
without a new campaign; the daily trend is printed on those alerts as context only.
One semantic to know: `blocked_hours_utc` is applied to the bar **label** hour inside the core One semantic to know: `blocked_hours_utc` is applied to the bar **label** hour inside the core
and to the bar **close** hour in the live wrapper and the backtester. So H1 can only trade bars and to the bar **close** hour in the live wrapper and the backtester. So H1 can only trade bars
@@ -501,11 +579,19 @@ CuPy GPU backend and `--backend cpu` both reproduce every MD5 below; earlier rep
| Book | Window | Trades / R / PF | MD5 of the CSV | | Book | Window | Trades / R / PF | MD5 of the CSV |
|---|---|---|---| |---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `e599c8a8652f95b7fcdf5a7ef2b564f5` | | M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `e599c8a8652f95b7fcdf5a7ef2b564f5` |
| M15 W2 | 2024-09-05 → 2026-09-05 | 813 / +154.15 / 1.53 | `5b65ecf6fad9a8da412749066fd50d36` | | M15 W2 | 2024-09-05 → 2026-09-05 | 597 / +155.89 / 1.78 | `dfc647ee02afd179dd9e031f344ed380` |
| M15 OOS | 2024-06-01 → 2025-03-30 | 313 / +68.50 / 1.67 | `09b8a8e7f5833436c20aef60ca0a7ff7` | | M15 OOS | 2024-06-01 → 2025-03-30 | 258 / +63.78 / 1.80 | `e87b6de12f90f3b5a5d41a953ee9ff7a` |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `a44fd19b4067f46e5bd98d50cc3eff92` | | H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `a44fd19b4067f46e5bd98d50cc3eff92` |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `f3587bae3c5dc3eda468ea3588b17dcb` | | H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `f3587bae3c5dc3eda468ea3588b17dcb` |
Re-pinned 2026-09-30. M15 only: the daily-trend veto (M15 override 7). W2 216 trades removed,
+1.7R, max drawdown 15.6R → 11.4R; OOS 55 removed, −4.7R (the accepted cost of the risk-first rule
in a rising window), drawdown 12.4R → 9.1R; previous pins `5b65ecf6…` (813 / +154.15 / 1.53) and
`09b8a8e7…` (313 / +68.50 / 1.67). M15 W3 with the veto 824 / +205.12 / 1.73, max drawdown 12.4R,
`ad3b0dbda0bf77b659caea1c53be6a50` (was `70195fb0…`, 1105 / +195.79 / 1.49, drawdown 20.9R). The
M5, H1 and H4 pins are unchanged. An M15 book needs the D1 frame of its window (1,400-day pad) in
the cache; `--set xau_strategy.daily_trend_veto=off` reproduces the previous M15 pins byte for byte.
Re-pinned 2026-09-26. M15: the grade-A SELL block (M15 override 6, 162 trades removed, +8.0R); Re-pinned 2026-09-26. M15: the grade-A SELL block (M15 override 6, 162 trades removed, +8.0R);
previous pins `b4aa07e3…` (931 / +146.14 / 1.43) and `9fd624fb…` (355 / +60.67 / 1.49); M15 W3 previous pins `b4aa07e3…` (931 / +146.14 / 1.43) and `9fd624fb…` (355 / +60.67 / 1.49); M15 W3
(3-year) with the block 1105 / +195.79 / 1.49, `70195fb0e6865b3a46ee8a9aa8844e51`. All four (3-year) with the block 1105 / +195.79 / 1.49, `70195fb0e6865b3a46ee8a9aa8844e51`. All four
@@ -520,18 +606,28 @@ columns of all five books are byte-identical to the previous pins (`29120a1e…`
`6f8f24b4…`, `3b45d12b…`, `e75adae1…`); trades, R and PF are unchanged. `--strategies` accepts the `6f8f24b4…`, `3b45d12b…`, `e75adae1…`); trades, R and PF are unchanged. `--strategies` accepts the
live names (`XAU` still means `XAU_ATR`). live names (`XAU` still means `XAU_ATR`).
Other current-HEAD figures at $0.65: M5 W3 (3-year) 870 / +153.6R / PF 1.50; M15 W3 1273 / Other current figures at $0.65 (2026-10-01): M5 W3 (3-year) 870 / +153.64R / PF 1.50 / max
+186.6R / PF 1.39; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54; drawdown 10.1R (`d20ec8a3…`); M15 W3 824 / +205.12R / PF 1.73 / 12.4R; M5 12-month (2025-08-14 →
H1 12-month (2025-08-04 → 2026-08-04) 91 / +22.2R / PF 1.79. All five MD5s above were 2026-08-14) 560 / +108.81R / PF 1.54; M15 12-month (2025-08-04 → 2026-08-04) 266 / +92.3R /
reproduced on 2026-09-15 after the cache prune and on 2026-09-18 on both backends PF 2.12 / 8.3R; H1 12-month (2025-08-04 → 2026-08-04) 91 / +22.2R / PF 1.79. The long robustness
(`backtest/research/gpu_backend_2026-09-18.md`). window 2019-01-01 → 2026-08-07 (allowed as supporting evidence, owner decision 2026-09-30): M15
1473 / +223.33R / PF 1.42 / max drawdown 41.5R (`aa5253aa…`; by year +14.0, +45.1, −27.3, +3.2,
+1.8, +53.5, +92.5, +40.6), M5 1052 / +128.10R / PF 1.34 / 18.4R (`3f0c091c…`; −1.0, −2.8, −1.7,
−2.2, −2.8, +1.0, +40.5, +97.0: the M5 rules take few trades and earn nothing before 2025). The
unseen holdout 2026-09-05 → 2026-10-01: M15 18 / −4.62R, M5 33 / −2.45R, H1 3 / −3.04R. The pins of
the table above were reproduced on 2026-09-15 after the cache prune, on 2026-09-18 on both
backends (`backtest/research/gpu_backend_2026-09-18.md`) and on 2026-10-01 with both scripts.
The CUDA research twin (`backtest/cudaBasedBacktesting.py`, section 3) reproduced all five MD5s The CUDA research twin (`backtest/cudaBasedBacktesting.py`, section 3) reproduced all five MD5s
above on 2026-09-18 in 0.8–2.4 s per book, plus the four 6-month acceptance books (2026-03-18 → above on 2026-09-18 in 0.8–2.4 s per book, plus the four 6-month acceptance books (2026-03-18 →
2026-09-18, M5 and M15 at $0.65 and $0.30: `86111c36…`, `e70d0252…`, `b8c98083…`, `e66b5fbd…`, 2026-09-18, M5 and M15 at $0.65 and $0.30: `86111c36…`, `e70d0252…`, `b8c98083…`, `e66b5fbd…`,
each equal to the reference script's), the M15 OOS `--export-features` CSV (`6239d197…`) and each equal to the reference script's), the M15 OOS `--export-features` CSV (`6239d197…`) and
`--record --record-bars` tables, and ten research-flag variants `--record --record-bars` tables, and ten research-flag variants
(`backtest/research/cuda_backtester_2026-09-18.md`). (`backtest/research/cuda_backtester_2026-09-18.md`). Parity was re-proved for the 2026-09-30 change:
the five pins above with both scripts, six `--set` books on W3 (veto both / buy / sell and the stop
cap on M15, veto and cap on M5), the M15 OOS `--export-features` CSV with the veto on
(`732f396bb3ee6036b62750a8c755194e`) and its `--record --record-bars` tables
(`backtest/research/daily_trend_campaign_2026-09-30.md`).
Kept cache windows (`backtest/cache/oanda_XAU_USD_<tf>_<paddedStart>_<end>.parquet`; the padded Kept cache windows (`backtest/cache/oanda_XAU_USD_<tf>_<paddedStart>_<end>.parquet`; the padded
start is the window start minus 62 days for M5/M15/H1, 234 for H4 and 1,400 for D1, so the D1 start is the window start minus 62 days for M5/M15/H1, 234 for H4 and 1,400 for D1, so the D1
@@ -539,7 +635,12 @@ and H4 files of one run have different start dates than its M5 file): 3-year `20
2-year `20240705_20260905`, OOS `20240331_20250330`, M5 12-month `20250613_20260814`, M15/H1 2-year `20240705_20260905`, OOS `20240331_20250330`, M5 12-month `20250613_20260814`, M15/H1
12-month `20250603_20260804`, 5-year `20210531_20260804`, 7.6-year `20181031_20260807`, the 12-month `20250603_20260804`, 5-year `20210531_20260804`, 7.6-year `20181031_20260807`, the
2026-08-14 parity window `20250814_20260814`, the 6-month CUDA acceptance window 2026-08-14 parity window `20250814_20260814`, the 6-month CUDA acceptance window
`20260115_20260918` (M5/M15/H1), their D1/H4 pads, and `calendar.parquet`. `20260115_20260918` (M5/M15/H1), their D1/H4 pads, and `calendar.parquet`. Added 2026-09-30 for
the daily-trend veto (an M15 run needs the D1 frame of its window): `1d_20201105_20260905` (W2),
`1d_20200801_20250330` (OOS), `1d_20150303_20260807` (the 7.6-year window), `1d_20211004_20260804`
and `4h_20241213_20260804` (the M15 / H1 12-month window), and the holdout window
`20260705_20261001` (M5 / M15 / H1) with `1d_20221105_20261001` and `4h_20260114_20261001`. The
M15 3-year run reads `1d_20191105_20260905`, which H4 W3 already kept.
The research recorder (`--record DIR [--record-bars]`) writes Parquet files describing every The research recorder (`--record DIR [--record-bars]`) writes Parquet files describing every
accepted trade (indicator votes on all three timeframes, penalties, setup, outcome, calendar accepted trade (indicator votes on all three timeframes, penalties, setup, outcome, calendar
@@ -567,7 +668,9 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
there too since 2026-09-08); use `setup_json.strategy`. there too since 2026-09-08); use `setup_json.strategy`.
- The `--stored-candles` scorecard in `analyze_signal_log.py` still advances outcomes without the - The `--stored-candles` scorecard in `analyze_signal_log.py` still advances outcomes without the
locked-profit floor step that the logger and the backfill script use (owner decision pending), so locked-profit floor step that the logger and the backfill script use (owner decision pending), so
the two measurement paths can disagree on gap rows. the two measurement paths can disagree on gap rows. Those are now only the trades still OPEN at
a hole (3 of 111 setups on the 2026-10-01 database copy; until then the replay also threw away
every trade that had exited before a later hole, 12 of 111).
- There is no holiday awareness anywhere. On Labor Day 2026 Oanda published no bars for hours, the - There is no holiday awareness anywhere. On Labor Day 2026 Oanda published no bars for hours, the
bot rescanned the stale bar 43 times, and the violent-move watch called the 12–14 UTC slot hot. bot rescanned the stale bar 43 times, and the violent-move watch called the 12–14 UTC slot hot.
Interior gaps in history are logged, never fatal, for the same reason. Interior gaps in history are logged, never fatal, for the same reason.
@@ -584,6 +687,29 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
≈ 0.03 with the next). Both news gating flags stay off. ≈ 0.03 with the next). Both news gating flags stay off.
- The level-breach rescan and the candle loop both skip the settlement-pause window - The level-breach rescan and the candle loop both skip the settlement-pause window
(`_closed_bar_in_settlement`); a rescan there only re-reads the last pre-pause bar. (`_closed_bar_in_settlement`); a rescan there only re-reads the last pre-pause bar.
- The alert cooldown counts bars of `report._frames` (set by `analyze_pair`). A report built by
hand has no frames and falls back to clock time, which releases the first bars after the
settlement pause or the weekend too early; that is the fallback, not the rule.
- An alert's row is stamped `alerted_at` BEFORE the send and the stamp is cleared when no chat
received it, also when the alert's task is cancelled while the stamp is being written (a
shutdown at that moment: the worker job cannot be stopped, so its future's done-callback
undoes it). A cancellation in the MIDDLE of the send keeps the stamp, because a chat may
already hold the message. Two residues, both accepted: if a send fails completely after
another alert already counted the row, the other alert's "already open" line is one too
high; and a process KILLED between the stamp and the send leaves a stamp on an alert nobody
received (it also seeds the cooldown at the next start, which errs on the quiet side).
- `news_engine._record_source("news_model", …)` is the only sign that the local headline model
failed a batch (WARNING once, `/status` line `News model: not scoring`); in `primary` mode
nothing else scores those headlines for the cycle. The line describes the latest cycle that
needed the model: a cycle with nothing for it to score (fallback mode with DeepSeek healthy)
drops an older failure instead of leaving it on `/status`. `/status` failure details are
exception text and are Markdown-escaped; an unpaired underscore made Telegram reject the reply.
- The agent panel's scorecards split at `PANEL_PROMPT_VERSION` v2.2 (2026-10-01): same prompt
text, different input (calendar-week event timing, headlines newest first). `_price_context`
never fetches the calendar itself; the offline replay passes an as-of context and must not see
today's events.
- The Oanda empty-window rule cannot catch an anomalous empty answer for a window of 5 days or
less (a short head page; every 3-day M1 page, a timeframe nothing fetches).
- The agent panel never runs inside `analyze_from_dataframes`, and its cached verdict is never - The agent panel never runs inside `analyze_from_dataframes`, and its cached verdict is never
reused when the evidence changed; the daily cap is the real spend bound. reused when the evidence changed; the daily cap is the real spend bound.
- Live == backtest also means the backtester never writes to `signals.db` and never builds a - Live == backtest also means the backtester never writes to `signals.db` and never builds a
@@ -624,6 +750,45 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
and never feed the printed target back into a gate. and never feed the printed target back into a gate.
- "Stop hunting" was tested on the live log (2026-09-26): 71% of live stop-outs ran a full stop - "Stop hunting" was tested on the live log (2026-09-26): 71% of live stop-outs ran a full stop
distance or more past the stop, the same as the books' 70–72%. The losers were real moves. distance or more past the stop, the same as the books' 70–72%. The losers were real moves.
- The daily-trend veto has ONE definition (`signal_aggregator.daily_trend_series`; EMA span 50 and
lookback 60 are module constants, validated literals). Live, the bar replay and the twin's host
side (`trend_flags`) all call it; the kernel only receives a flag per bar. Changing the
literals, the Sunday filter or the as-of rule (a daily bar labelled D counts from D + 1 day ≤ the
primary bar's close) is a rule-1 event and a parity event. The live frame holds 1,000 daily bars
and a backtest pad 1,400 days; the EMA seed has decayed below float precision in both, so the
states agree.
- With the veto on, D1 is a REQUIRED frame of the M15 scan: a failed D1 fetch fails the scan
(`DATA UNAVAILABLE: 1d — scan failed`), it does not fall back to "no veto". After a deploy watch
that rate. Inside the core a missing D1 slice only adds the notice `Daily trend filter requested
but 1d data unavailable` and ships the setup; the replay reaches that state only before the
first daily bar of its pad.
- The veto is the LAST step of the core (after the chart-pattern path and the stale-veto cleanup),
so `gate_reason` lists `DAILY TREND FILTER` ahead of the gate chain. The rule itself only
removes a plan. In a book a removed entry also moves the 5-bar cooldown, which could free a
later entry: that is possible in principle and happened 0 times in every book of the campaign
(an observation, not a guarantee; count the added trades in any new comparison). The wide-stop cap never
changes the trade set either (it is applied after the min-RR and spread gates); a smaller stop
can lower the printed 1R target, never raise it.
- The control for any M15 candidate is now the factory config WITH the veto. Comparing against a
pre-2026-09-30 M15 number needs `--set xau_strategy.daily_trend_veto=off` (reproduces
`5b65ecf6…` / `09b8a8e7…` / `70195fb0…`).
- The veto does not find big losers. The trades it removes are break-even after costs (−0.03R to
−0.05R each against +0.15R to +0.25R for the kept ones); the gain is fewer flat trades and
shorter losing runs. The SELL half is carried by the rising 3-year window and the BUY half by the
falling years of 2019 → 2026; `buy`-only and `sell`-only both fail. 2021 is not improved and 2023
is worse: in a sideways year the daily trend flips late.
- The lot-size line reveals the owner's risk per trade. It is stamped per surface
(`exposure.attach_exposure`) and the channel copy of an alert is built separately
(`format_signal_message(report, channel=True)`); never print `report.size_line` on a public
surface without checking `exposure.size_shared`. The dashboard snapshot is public.
- "Already open" is the advisory outcome model's view of ALERTED plans (`alerted_at` set, outcome
`PENDING`), not a broker position. Rows alerted before the 2026-09-30 deploy carry no
`alerted_at`, so the line starts empty and fills as alerts go out; a `PENDING` row older than the
model's own time-out (300 bars, with a weekend allowance) is not shown. Stacked entries were not
worse trades in the books, so the line is information, never a reason the code skips an alert.
- `signals.daily_trend` is the core's value on M15 and the live wrapper's display stamp on M5, H1
and H4 (`_stamp_daily_trend_for_display`, one extra cached D1 fetch on M5 / H1); it is NULL on
blocked and failed scans. The stamp sits in `analyze_pair`, never in `analyze_from_dataframes`.
- The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped - The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped
every H1 and H4 scan once (2026-08-02). every H1 and H4 scan once (2026-08-02).
- The agent panel's price context reads the frames `analyze_pair` stashes on `report._frames`. - The agent panel's price context reads the frames `analyze_pair` stashes on `report._frames`.
@@ -644,6 +809,10 @@ One line per validated change or verdict, newest first. Full evidence in the nam
| Date | Decision | Where the evidence is | | Date | Decision | Where the evidence is |
|---|---|---| |---|---|---|
| 2026-10-01 | Bug review of the whole codebase (owner request), then the fixes. Signal path clean: indicators, grading, gates, the daily-trend veto, the as-of logic and both backtesters checked, and no live module needs a newer Python or SQLite than the server has. 17 defects elsewhere, all FIXED, each with a test; books byte-identical (the five pins reproduced on both scripts, 0 cache fallbacks). **Alert path**: the "already open" line was stamped at scan time, so two alerts at one candle close never counted each other (the M5 and M15 books enter together 42 times in three years); the row is now stamped just before the send on a one-worker `alert-db` pool and un-stamped when no chat received it (`_deliver_alert`, one path for the candle scan and the level break; no SQLite write on the event loop any more). The 5-bar cooldown counted clock time and forgot its state at every restart: it now counts the scan's own bars like `_apply_cooldown` (the old rule released the 22:00 UTC bar after the settlement pause: 6 M15 and 3 M5 entries the books never take in three years) and is restored from `signals.db` at start-up. The lot-size line went to every target that was not the channel (a group the owner ran `/autoscan` in): now the owner's own chat only. **Measurement**: the outcome model rejected a whole candle stream for a hole anywhere in it; continuity is now judged bar by bar in the walk, so the stored-candle replay resolves 108 of 111 logged setups instead of 99, all 108 equal to the R the live resolver stored. **History fetch**: an empty answer is accepted only for a window of 5 days or less with no malformed candle (a longer one is retried, then refused). **News**: one unreadable headline no longer costs the local model its whole batch (verified on the real bundle), a failing batch is a WARNING and a `/status` line, no DeepSeek audit call in a process that does not archive, the event-direction cache is per release. **LLM client**: `chat_json` never raises and never logs the key; the scan's news block is guarded (M15 OOS pair re-proved, Module 37 green). **Panel** (`PANEL_PROMPT_VERSION` v2.2, a new scorecard era): event timing over the cached calendar week, headlines newest first; the replay script uses the logged macro payloads it used to ignore (39 of 110 events). **Tests**: no `DROP COLUMN` (SQLite 3.35), Parquet-writing tests skip without `pyarrow`, the suite passes with both Parquet engines blocked and no GPU. Also fixed on the way: `/status` failure details are Markdown-escaped. **Independent review of these fixes** (a second agent, 20,000-case differential fuzz of the outcome model among its checks): four further LOW defects, fixed the same day — an alert cancelled while its row was being stamped kept the stamp; the panel's calendar read could trigger a 40 s refetch (now `peek_week_events`); a stale model-failure line could stay on `/status`; an HTTP error body was logged without redaction. **Independent verification of the 2026-09-30 direction change**: every figure reproduced, no signal changed by this pass, caveats recorded in section 5.13 of the campaign report. Suite 632 / 40 modules | plan file `~/.claude/plans/you-are-a-quant-velvety-wave.md`; sections 2, 4, 6 and 10 of this file |
| 2026-10-01 | New data sources, report only (owner request). Eight outside series tested against gold's own daily trend: the dollar (EUR/USD), silver, the US 10-year note, the real yield and the broad dollar index (FRED), the gold fund's share count, CFTC managed-money positioning, and Oanda's own client positioning (the position-book endpoint DOES answer on the practice API, with history to 2017; the plan had expected it gone). None shows a forward move of gold beyond gold's own trend (largest t-values 1.2 and 1.8); on the books only one flag, M5 trades against the broad dollar index (−11.2R over 435, losing in both eras), which is the 2025–2026 rally seen from another angle and does nothing on M15: WATCH, a hypothesis. Nothing ships, no feed added, all data stayed in the scratchpad | `backtest/research/new_inputs_2026-10.md` |
| 2026-10-01 | Context screens after the veto (owner request): weekly trend, four-hour trend, daily-trend strength, daily volatility state, place in the 20-day range, tick-volume surge, time to a high-impact US release; rule written first (a state needs 8% of both books and a loss in all four slices to earn a replay). NO state nominated on M15 or M5, no replay, nothing ships. Description: M15 has no losing pocket left in these views; M5 earns only where everything agrees (with the weekly and four-hour trend +0.16R to +0.24R per trade, against them about 0) and near the 20-day extreme in the trade's direction, so a "do not chase" filter would cut the part that pays | `backtest/research/context_screens_2026-10.md` |
| 2026-09-30 | Direction and setup-size campaign (owner request; RISK-FIRST rule approved by the owner, 2019 → 2026 allowed as supporting evidence). **Daily-trend veto SHIPPED on M15** (override 7, `daily_trend_veto` `both`): a finished setup against a clear daily trend (close vs 50-day EMA and vs 60 days earlier, closed D1 bars, Sunday stubs dropped) is removed. PASSED every clause: W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R; both costs; 3 of 4 neighbouring definitions. Accepted cost: W2 first half +61.8R vs +70.1R, old OOS window −4.7R. FAILED: `buy`-only and `sell`-only on M15, every mode on M5 (W3 drawdown 10.1R → 15.2R, long window worse) and on H1 (−9.7R). **Wide-stop cap FAILED** rule C (K = 6 +11.8R on W3 but K = 5 −18.6R on the long window: a peak, not a plateau); knob kept at 0.0. M15 books re-pinned (W2 `dfc647ee…`, OOS `e87b6de1…`), M5 / H1 / H4 pins unchanged, twin MD5-equal throughout, recorder parity re-proved. **Display lines SHIPPED** (books byte-identical): lot size for the owner's risk limit (`trade_size` in config.json, private by default), alerted same-direction plans still open (`signals.alerted_at`, read-only `open_alerted_setups`), the daily trend in words on every surface and in every logged row. Measured on the way: taking every M5 + M15 alert, two entries in three already have a same-direction plan open and up to 13 are open at once; those stacked entries are not worse trades. FIXED: the Oanda provider raised on an empty date window, so `fetch_range` aborted on a pad starting on a closed day. Stops and exits re-screened (25 variants): at a plateau, tighter is far worse. Suite 607 / 40 modules | `backtest/research/daily_trend_campaign_2026-09-30.md`; factory docstring M15 override 7 |
| 2026-09-27 | Local news scorer, round two: the owner's ML-NEWS model is now a DeBERTa-v3-small two-head transformer (gold-relevant? then bullish/bearish), distilled from 17,245 DeepSeek gold labels (Sep 8–22 archive + a 2019–2026 Reuters/CNBC/MarketWatch crawl labelled by DeepSeek, 560 calls). Golden set 72/78 (DeepSeek 78, keywords 24, the 09-17 TF-IDF 39), agreement with DeepSeek on unseen live days ~72% (ceiling 90%), 19 ms/headline on one CPU thread as int8-embedding ONNX; passed the four promotion criteria fixed on 09-22. OWNER DECISIONS: it is to score headlines here in mode `primary` with NO DeepSeek for headline scoring (DeepSeek stays for event direction, summaries, the dashboard endpoint); model files live OUTSIDE this tree at a configured path (280 MB, never in git); shadow mode declined (the archive already allows offline replay). IMPLEMENTED the same day (branch `sql-integration`): `news_model.py` (vendored runtime + never-raising singleton, lazy `onnxruntime` / `tokenizers`), `news_model` config section + `NEWS_MODEL_MODE` env, `news_engine._enrich_headline_scores` / `_apply_model_sentiment` (gold-only rules A–F) / `_audit_with_deepseek`, archive provenance for `model` rows, `deploy/README-news-model.md`, Module 38 (15 tests, one against the real bundle: bullish / neutral labels, 7 ms per headline on the dev box). Two further owner decisions during implementation: **`primary` mode stops writing `newsdata.db`** (only the DeepSeek audit sample, shipped 0, is archived) and **both processes load the model**. Books unaffected (news never enters signals); no signal file touched, no parity event. ML-NEWS `news_transformer.py` received the same two `load_model` edits (commit `9496cd9` there) | `docs/news_model_integration_spec_2026-09-27.md`; `~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md` | | 2026-09-27 | Local news scorer, round two: the owner's ML-NEWS model is now a DeBERTa-v3-small two-head transformer (gold-relevant? then bullish/bearish), distilled from 17,245 DeepSeek gold labels (Sep 8–22 archive + a 2019–2026 Reuters/CNBC/MarketWatch crawl labelled by DeepSeek, 560 calls). Golden set 72/78 (DeepSeek 78, keywords 24, the 09-17 TF-IDF 39), agreement with DeepSeek on unseen live days ~72% (ceiling 90%), 19 ms/headline on one CPU thread as int8-embedding ONNX; passed the four promotion criteria fixed on 09-22. OWNER DECISIONS: it is to score headlines here in mode `primary` with NO DeepSeek for headline scoring (DeepSeek stays for event direction, summaries, the dashboard endpoint); model files live OUTSIDE this tree at a configured path (280 MB, never in git); shadow mode declined (the archive already allows offline replay). IMPLEMENTED the same day (branch `sql-integration`): `news_model.py` (vendored runtime + never-raising singleton, lazy `onnxruntime` / `tokenizers`), `news_model` config section + `NEWS_MODEL_MODE` env, `news_engine._enrich_headline_scores` / `_apply_model_sentiment` (gold-only rules A–F) / `_audit_with_deepseek`, archive provenance for `model` rows, `deploy/README-news-model.md`, Module 38 (15 tests, one against the real bundle: bullish / neutral labels, 7 ms per headline on the dev box). Two further owner decisions during implementation: **`primary` mode stops writing `newsdata.db`** (only the DeepSeek audit sample, shipped 0, is archived) and **both processes load the model**. Books unaffected (news never enters signals); no signal file touched, no parity event. ML-NEWS `news_transformer.py` received the same two `load_model` edits (commit `9496cd9` there) | `docs/news_model_integration_spec_2026-09-27.md`; `~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md` |
| 2026-09-27 | Single "conviction" number vs the two-number grade (owner question: one number that is easier to understand and to decide on). Pre-registered; switch `signals.grading_mode` (`pair` default = the shipped rule, byte-identical on both scripts, all five pins reproduced; `min` = min(\|score\|, 0.6 × conf); `product` = \|score\| × conf / 100; thresholds from the control's corner points, no tuning; mirrored in K2, twin MD5-equal to the reference on both new paths). FAILED on M5 and M15: K-MIN moves 43–54 trades for −1.2R / −2.9R (a, b, e / a, b, d); K-PROD −29R on M15 with PF 1.29 and W3 DD 28.9R, neutral on M5. H1 / H4 K-PROD +11.9R / +17.0R on W3 by taking 40–100 extra low-score trades (H1 PF 1.98 → 1.75, DD doubles): a hypothesis, not evidence. Nothing ships; the knob stays research-only. Code review (high) before the runs: 6 findings fixed, 1 recorded. SHIPPED the display half of the question (owner request the same day): alerts lead with the grade and print score / confidence as words (`main.plain_words`, same line on Telegram, `/status`, CLI and the dashboard card); books untouched. Suite 566 | `backtest/research/conviction_campaign_2026-09-27.md` | | 2026-09-27 | Single "conviction" number vs the two-number grade (owner question: one number that is easier to understand and to decide on). Pre-registered; switch `signals.grading_mode` (`pair` default = the shipped rule, byte-identical on both scripts, all five pins reproduced; `min` = min(\|score\|, 0.6 × conf); `product` = \|score\| × conf / 100; thresholds from the control's corner points, no tuning; mirrored in K2, twin MD5-equal to the reference on both new paths). FAILED on M5 and M15: K-MIN moves 43–54 trades for −1.2R / −2.9R (a, b, e / a, b, d); K-PROD −29R on M15 with PF 1.29 and W3 DD 28.9R, neutral on M5. H1 / H4 K-PROD +11.9R / +17.0R on W3 by taking 40–100 extra low-score trades (H1 PF 1.98 → 1.75, DD doubles): a hypothesis, not evidence. Nothing ships; the knob stays research-only. Code review (high) before the runs: 6 findings fixed, 1 recorded. SHIPPED the display half of the question (owner request the same day): alerts lead with the grade and print score / confidence as words (`main.plain_words`, same line on Telegram, `/status`, CLI and the dashboard card); books untouched. Suite 566 | `backtest/research/conviction_campaign_2026-09-27.md` |
| 2026-09-26 | Quant review + live log 2026-09-04 → 09-25 (100 setups replayed on the scans' stored candles): stop-hunt claim REFUTED (71% of stop-outs ran ≥ 1R past the stop, same as the books); −25.1R of the −37.8R raw live book were consecutive-bar re-entries the backtester's cooldown never takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week windows. SHIPPED: 5-bar alert cooldown; alert wording (trail first as the tested exit, TP1 a guide with its R, planned RR, validated-book line per class); M15 grade-A SELL block (`sell_min_quality` `A+, B`, PASSED a–f: W2 +154.15R/813 PF 1.53 vs +146.14/931 PF 1.43, OOS +68.5 vs +60.67, W3 DD +8%); 1R printed-target floor in every generator after the gates (`tp_print_floor_rr`, owner rule "TP1 at least 1:1"; label-only re-pin of all four timeframes). FAILED: M15 `min_rr_ratio` 1.0 (trade-identical, −2.0R); a pre-gate target floor (−24.6R). OWNER'S CALL: M5 `tp_floor_rr` 1.0 adds +48.9R on W2 with both halves up but max DD 10.1 → 15.9R (ladder 0.6–1.0 non-monotone); not shipped. Telegram / dashboard AI block rewritten for a new trader (owner rule): one plain headline against the usual odds, the chance "in 100", one reason for and one against, no verdict word or confidence figure; persona prompts ban trading jargon (`PANEL_PROMPT_VERSION` v2.1). Two delivery bugs from the DB checks fixed: the macro-source panel verdict never stored its probability (all v2 rows NULL); the level-breach rescan re-logged the 20:45 M15 bar three times every evening after the settlement pause (guard added to `_handle_breach`). Indicator grid and volume / daily-direction screens (Part C) not run | `backtest/research/rr_and_sell_campaign_2026-09-26.md` | | 2026-09-26 | Quant review + live log 2026-09-04 → 09-25 (100 setups replayed on the scans' stored candles): stop-hunt claim REFUTED (71% of stop-outs ran ≥ 1R past the stop, same as the books); −25.1R of the −37.8R raw live book were consecutive-bar re-entries the backtester's cooldown never takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week windows. SHIPPED: 5-bar alert cooldown; alert wording (trail first as the tested exit, TP1 a guide with its R, planned RR, validated-book line per class); M15 grade-A SELL block (`sell_min_quality` `A+, B`, PASSED a–f: W2 +154.15R/813 PF 1.53 vs +146.14/931 PF 1.43, OOS +68.5 vs +60.67, W3 DD +8%); 1R printed-target floor in every generator after the gates (`tp_print_floor_rr`, owner rule "TP1 at least 1:1"; label-only re-pin of all four timeframes). FAILED: M15 `min_rr_ratio` 1.0 (trade-identical, −2.0R); a pre-gate target floor (−24.6R). OWNER'S CALL: M5 `tp_floor_rr` 1.0 adds +48.9R on W2 with both halves up but max DD 10.1 → 15.9R (ladder 0.6–1.0 non-monotone); not shipped. Telegram / dashboard AI block rewritten for a new trader (owner rule): one plain headline against the usual odds, the chance "in 100", one reason for and one against, no verdict word or confidence figure; persona prompts ban trading jargon (`PANEL_PROMPT_VERSION` v2.1). Two delivery bugs from the DB checks fixed: the macro-source panel verdict never stored its probability (all v2 rows NULL); the level-breach rescan re-logged the 20:45 M15 bar three times every evening after the settlement pause (guard added to `_handle_breach`). Indicator grid and volume / daily-direction screens (Part C) not run | `backtest/research/rr_and_sell_campaign_2026-09-26.md` |
+43 -11
View File
@@ -176,10 +176,31 @@ def _valid_bars(bars: Iterable[Dict[str, Any]]) -> List[Dict[str, Any]]:
return valid return valid
def _next_bar_follows(
previous: datetime, stamp: datetime, step: timedelta
) -> bool:
"""True when ``stamp`` is the next tradable bar after ``previous``: every
bar slot between the two lies inside a scheduled XAU closure (the weekend
or the daily 17:00 New York maintenance hour) and ``stamp`` is on the bar
grid. ``step`` must be positive."""
expected = previous + step
while expected < stamp:
ny = expected.astimezone(_NY_TZ)
daily_maintenance = ny.weekday() < 4 and ny.hour == 17
if not (_is_forex_weekend(expected) or daily_maintenance):
return False
expected += step
return expected == stamp
def _has_continuity( def _has_continuity(
bars: List[Dict[str, Any]], last: datetime, timeframe: str bars: List[Dict[str, Any]], last: datetime, timeframe: str
) -> bool: ) -> bool:
"""Reject missing tradable bars while allowing scheduled XAU closures.""" """Reject missing tradable bars while allowing scheduled XAU closures.
All-or-nothing over the whole of ``bars`` (the scan-outcome labeller
calls it one bar at a time). advance_outcome does not use it: it checks
each step as it walks, so a hole AFTER a trade's exit cannot void it."""
seconds = _TF_SECONDS.get(normalize_timeframe(timeframe)) seconds = _TF_SECONDS.get(normalize_timeframe(timeframe))
if seconds is None: if seconds is None:
return False return False
@@ -190,14 +211,7 @@ def _has_continuity(
stamp = bar["_timestamp"] stamp = bar["_timestamp"]
if stamp <= previous or stamp in seen: if stamp <= previous or stamp in seen:
continue continue
expected = previous + step if not _next_bar_follows(previous, stamp, step):
while expected < stamp:
ny = expected.astimezone(_NY_TZ)
daily_maintenance = ny.weekday() < 4 and ny.hour == 17
if not (_is_forex_weekend(expected) or daily_maintenance):
return False
expected += step
if expected != stamp:
return False return False
previous = stamp previous = stamp
seen.add(stamp) seen.add(stamp)
@@ -256,18 +270,36 @@ def advance_outcome(
state["result"] = None state["result"] = None
state["reason"] = "stored candle snapshots lost continuity" state["reason"] = "stored candle snapshots lost continuity"
return state return state
if has_future and not _has_continuity(valid_bars, last, if not has_future:
state["timeframe"]): return state
seconds = _TF_SECONDS.get(normalize_timeframe(state["timeframe"]))
if seconds is None:
state["status"] = "UNRESOLVABLE" state["status"] = "UNRESOLVABLE"
state["result"] = None state["result"] = None
state["reason"] = "stored candle snapshots contain an internal gap" state["reason"] = "stored candle snapshots contain an internal gap"
return state return state
step = timedelta(seconds=seconds)
for bar in valid_bars: for bar in valid_bars:
stamp = bar["_timestamp"] stamp = bar["_timestamp"]
if stamp <= signal_bar or stamp <= last: if stamp <= signal_bar or stamp <= last:
continue continue
# Continuity is judged ONE STEP AT A TIME, only as far as the
# trade is still open. Until 2026-10-01 the whole stream was
# checked before the walk, so a trade that had exited days
# before a later hole (a holiday, an outage) was thrown away:
# 9 of 111 setups in the September log, 7 of them with an exit
# the live resolver had recorded. A hole reached while the
# trade is open still ends the walk; the progress made up to
# the bar before it stays, so resolve_locked_floor floors from
# the latest stop.
if not _next_bar_follows(last, stamp, step):
state["status"] = "UNRESOLVABLE"
state["result"] = None
state["reason"] = "stored candle snapshots contain an internal gap"
return state
if progress["entry_price"] is None: if progress["entry_price"] is None:
entry = bar["open"] entry = bar["open"]
initial_stop = entry - risk if buy else entry + risk initial_stop = entry - risk if buy else entry + risk
+60 -10
View File
@@ -44,6 +44,10 @@ stop/target in ATR, the setup class's base rate from the two-year book,
event timing against the expected hold) so the model judges facts event timing against the expected hold) so the model judges facts
instead of narrative; ``PANEL_PROMPT_VERSION`` is logged with every instead of narrative; ``PANEL_PROMPT_VERSION`` is logged with every
verdict and is part of the payload hash so prompt eras never mix. verdict and is part of the payload hash so prompt eras never mix.
Since v2.2 (2026-10-01) the event timing reads the cached calendar week
(a hold that runs past UTC midnight sees tomorrow's release) and the
headline list is cut newest first; both change what the persona reads,
hence the new era.
Two invariants, both load-bearing (CLAUDE.md rules 1 and 4): Two invariants, both load-bearing (CLAUDE.md rules 1 and 4):
@@ -126,7 +130,10 @@ VERDICT_SOURCES = ("mean", "macro", "chair")
#: Logged with every verdict and folded into the payload hash. Bump on #: Logged with every verdict and folded into the payload hash. Bump on
#: ANY prompt or context-schema change so the scorecard can split eras #: ANY prompt or context-schema change so the scorecard can split eras
#: (analyze_signal_log splits by panel_prompt_version). #: (analyze_signal_log splits by panel_prompt_version).
PANEL_PROMPT_VERSION = "v2.1-2026-09-26" # v2.1: beginner wording rules in _TAKE_CONTRACT # v2.1: beginner wording rules in _TAKE_CONTRACT. v2.2: no prompt text
# changed, the persona's INPUT did (event timing from the cached calendar
# week, headlines newest first), so the scorecard starts a new era.
PANEL_PROMPT_VERSION = "v2.2-2026-10-01"
#: Symmetric probability bands (percent). A persona's stance and the #: Symmetric probability bands (percent). A persona's stance and the
#: mean verdict are DERIVED from p_win with the same thresholds, so #: mean verdict are DERIVED from p_win with the same thresholds, so
@@ -606,13 +613,31 @@ def _headline_gold_score(headline: Any) -> Optional[float]:
return max(-1.0, min(1.0, score)) if math.isfinite(score) else None return max(-1.0, min(1.0, score)) if math.isfinite(score) else None
def _published_key(headline: Any) -> float:
"""Sort key for "newest first": the publish time as epoch seconds,
-inf when there is none. Never raises."""
stamp = getattr(headline, "timestamp", None)
try:
if isinstance(stamp, str):
stamp = _dt.datetime.fromisoformat(stamp.replace(" ", "T"))
value = float(stamp.timestamp())
return value if math.isfinite(value) else float("-inf")
except Exception:
return float("-inf")
def _directional_news_payload( def _directional_news_payload(
context: Optional["FundamentalContext"], limit: int = 10 context: Optional["FundamentalContext"], limit: int = 10
) -> List[Dict[str, Any]]: ) -> List[Dict[str, Any]]:
if context is None: if context is None:
return [] return []
out: List[Dict[str, Any]] = [] out: List[Dict[str, Any]] = []
for headline in context.news_headlines: # Newest first, THEN the cut: the context lists headlines feed by
# feed, so the first `limit` were whatever the first feed carried and
# a newer story from a later feed never reached the persona.
ordered = sorted(context.news_headlines or [], key=_published_key,
reverse=True)
for headline in ordered:
score = _headline_gold_score(headline) score = _headline_gold_score(headline)
if score is None or abs(score) < 0.1: if score is None or abs(score) < 0.1:
continue continue
@@ -906,9 +931,14 @@ def _primary_frame(report: "SignalReport", frames: Any) -> Any:
def _price_context(report: "SignalReport", frames: Any = None, def _price_context(report: "SignalReport", frames: Any = None,
context: Optional["FundamentalContext"] = None context: Optional["FundamentalContext"] = None,
week_events: Optional[List[Any]] = None
) -> Dict[str, Any]: ) -> Dict[str, Any]:
"""The shared "context" block (see module docstring). {} on failure.""" """The shared "context" block (see module docstring). {} on failure.
``week_events`` is the cached calendar week (attach_panel hands it
in); it is never fetched here, because the offline replay calls this
function with an as-of context and must not see today's calendar."""
out: Dict[str, Any] = {} out: Dict[str, Any] = {}
try: try:
direction = _direction(report) direction = _direction(report)
@@ -992,10 +1022,16 @@ def _price_context(report: "SignalReport", frames: Any = None,
hold_bars = (out.get("base_rate") or {}).get("median_bars_held") hold_bars = (out.get("base_rate") or {}).get("median_bars_held")
if hold_bars and secs: if hold_bars and secs:
out["expected_hold_minutes"] = int(hold_bars * secs // 60) out["expected_hold_minutes"] = int(hold_bars * secs // 60)
if context is not None: # The context alone knows today's events (UTC date) and the next
# hour: an H1 or H4 hold running past midnight into tomorrow's
# release read as event-free. The calendar week covers it.
pool = list(week_events or [])
if not pool and context is not None:
pool = list(getattr(context, "events_today", None) or []) + \
list(getattr(context, "events_upcoming_1h", None) or [])
if pool:
minutes = [] minutes = []
for ev in list(getattr(context, "events_today", None) or []) + \ for ev in pool:
list(getattr(context, "events_upcoming_1h", None) or []):
try: try:
if ev.is_high_impact and ev.is_upcoming: if ev.is_high_impact and ev.is_upcoming:
minutes.append(float(ev.minutes_until)) minutes.append(float(ev.minutes_until))
@@ -1466,21 +1502,35 @@ def attach_panel(report: "SignalReport",
if direction is None: if direction is None:
return False return False
week_events: Optional[List[Any]] = None
if context is None: if context is None:
# Best-effort; the news engine's class-level 600 s caches make # Best-effort; the news engine's class-level 600 s caches make
# this nearly free when warm in both bot and dashboard. # this nearly free when warm in both bot and dashboard.
try: try:
from config import PlatformConfig from config import PlatformConfig
from news_engine import NewsEngine from news_engine import NewsEngine
context = NewsEngine( engine = NewsEngine(PlatformConfig().news)
PlatformConfig().news).get_fundamental_context() context = engine.get_fundamental_context()
except Exception as e: except Exception as e:
logger.debug("agent panel: news context unavailable: %s", e) logger.debug("agent panel: news context unavailable: %s", e)
context = None context = None
else:
# The calendar week for the event-timing check, read from
# the class-level cache the context was just built from.
# peek, not get: get_week_events refreshes an expired
# cache, and with the calendar sources down that is a 40 s
# wait under the news lock while this alert's lock is held.
# Only on this path — a caller that passes its own context
# (the replay) passes its own time.
try:
week_events = engine.peek_week_events()
except Exception as e:
logger.debug("agent panel: calendar week unavailable: %s", e)
personas = normalize_personas(settings.get("personas", DEFAULT_PERSONAS)) personas = normalize_personas(settings.get("personas", DEFAULT_PERSONAS))
candles = _candle_context(report, settings) candles = _candle_context(report, settings)
ctx = _price_context(report, getattr(report, "_frames", None), context) ctx = _price_context(report, getattr(report, "_frames", None), context,
week_events)
jobs = _panel_jobs(report, context, candles, ctx, personas) jobs = _panel_jobs(report, context, candles, ctx, personas)
payload_hash = _panel_payload_hash(jobs) payload_hash = _panel_payload_hash(jobs)
reuse_hash = _panel_payload_hash(jobs, ignore_bar_timestamp=True) reuse_hash = _panel_payload_hash(jobs, ignore_bar_timestamp=True)
+52 -60
View File
@@ -1,6 +1,6 @@
{ {
"schema": "agent-panel-base-rates-v1", "schema": "agent-panel-base-rates-v1",
"generated_at": "2026-09-27T00:45:18+00:00", "generated_at": "2026-10-01T05:49:09+00:00",
"key": "timeframe|strategy|grade|direction ('*' = any)", "key": "timeframe|strategy|grade|direction ('*' = any)",
"min_n_for_exact_class": 30, "min_n_for_exact_class": 30,
"books": { "books": {
@@ -13,8 +13,8 @@
}, },
"15min": { "15min": {
"file": "m15_w2.csv", "file": "m15_w2.csv",
"md5": "5b65ecf6fad9a8da412749066fd50d36", "md5": "dfc647ee02afd179dd9e031f344ed380",
"trades": 813, "trades": 597,
"first_entry": "2024-09-05 06:15:00+00:00", "first_entry": "2024-09-05 06:15:00+00:00",
"last_entry": "2026-09-04 08:00:00+00:00" "last_entry": "2026-09-04 08:00:00+00:00"
}, },
@@ -35,27 +35,27 @@
}, },
"classes": { "classes": {
"15min|*|*|*": { "15min|*|*|*": {
"n": 813, "n": 597,
"win_rate": 62.24, "win_rate": 64.32,
"mean_r": 0.1896, "mean_r": 0.2611,
"tp1_reach_rate": 50.43, "tp1_reach_rate": 53.6,
"median_bars_held": 32, "median_bars_held": 34,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|*|*|BUY": { "15min|*|*|BUY": {
"n": 635, "n": 540,
"win_rate": 63.31, "win_rate": 64.44,
"mean_r": 0.2309, "mean_r": 0.2725,
"tp1_reach_rate": 51.65, "tp1_reach_rate": 53.52,
"median_bars_held": 33, "median_bars_held": 35,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|*|*|SELL": { "15min|*|*|SELL": {
"n": 178, "n": 57,
"win_rate": 58.43, "win_rate": 63.16,
"mean_r": 0.0424, "mean_r": 0.1536,
"tp1_reach_rate": 46.07, "tp1_reach_rate": 54.39,
"median_bars_held": 28, "median_bars_held": 27,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|GENERIC|A+|BUY": { "15min|GENERIC|A+|BUY": {
@@ -66,68 +66,60 @@
"median_bars_held": 32, "median_bars_held": 32,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|GENERIC|A+|SELL": {
"n": 1,
"win_rate": 0.0,
"mean_r": -1.0276,
"tp1_reach_rate": 0.0,
"median_bars_held": 34,
"book": "m15_w2.csv"
},
"15min|GENERIC|A|BUY": { "15min|GENERIC|A|BUY": {
"n": 208, "n": 191,
"win_rate": 67.31, "win_rate": 67.02,
"mean_r": 0.3158, "mean_r": 0.3152,
"tp1_reach_rate": 59.13, "tp1_reach_rate": 59.16,
"median_bars_held": 44, "median_bars_held": 45,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|GENERIC|B|BUY": { "15min|GENERIC|B|BUY": {
"n": 282, "n": 238,
"win_rate": 62.41, "win_rate": 62.18,
"mean_r": 0.1869, "mean_r": 0.2096,
"tp1_reach_rate": 46.1, "tp1_reach_rate": 47.06,
"median_bars_held": 38, "median_bars_held": 38,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|GENERIC|B|SELL": { "15min|GENERIC|B|SELL": {
"n": 161, "n": 52,
"win_rate": 58.39, "win_rate": 63.46,
"mean_r": 0.0273, "mean_r": 0.1569,
"tp1_reach_rate": 45.34, "tp1_reach_rate": 53.85,
"median_bars_held": 32, "median_bars_held": 28,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|GENERIC|C|BUY": { "15min|GENERIC|C|BUY": {
"n": 60, "n": 49,
"win_rate": 65.0, "win_rate": 71.43,
"mean_r": 0.395, "mean_r": 0.5518,
"tp1_reach_rate": 60.0, "tp1_reach_rate": 67.35,
"median_bars_held": 24, "median_bars_held": 27,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|XAU_ATR|A+|BUY": { "15min|XAU_ATR|A+|BUY": {
"n": 3, "n": 2,
"win_rate": 66.67, "win_rate": 50.0,
"mean_r": 0.2684, "mean_r": -0.4203,
"tp1_reach_rate": 33.33, "tp1_reach_rate": 0.0,
"median_bars_held": 13, "median_bars_held": 12,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|XAU_ATR|A+|SELL": { "15min|XAU_ATR|A+|SELL": {
"n": 16, "n": 5,
"win_rate": 62.5, "win_rate": 60.0,
"mean_r": 0.2612, "mean_r": 0.1192,
"tp1_reach_rate": 56.25, "tp1_reach_rate": 60.0,
"median_bars_held": 5, "median_bars_held": 6,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"15min|XAU_ATR|A|BUY": { "15min|XAU_ATR|A|BUY": {
"n": 80, "n": 58,
"win_rate": 56.25, "win_rate": 62.07,
"mean_r": 0.06, "mean_r": 0.2058,
"tp1_reach_rate": 47.5, "tp1_reach_rate": 53.45,
"median_bars_held": 10, "median_bars_held": 11,
"book": "m15_w2.csv" "book": "m15_w2.csv"
}, },
"1h|*|*|*": { "1h|*|*|*": {
+53 -1
View File
@@ -25,6 +25,7 @@ bot, CLI and backtester all see the same overlay (live == backtest).
import json import json
import logging import logging
import math
import os import os
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
@@ -90,7 +91,8 @@ _KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level",
# config.json files still carry the block and must keep # config.json files still carry the block and must keep
# loading; load_app_config logs one warning when it is there. # loading; load_app_config logs one warning when it is there.
"market_outlook", "market_outlook",
"violent_watch", "signal_log", "news_archive", "news_model"} "violent_watch", "signal_log", "news_archive", "news_model",
"trade_size"}
_KNOWN_OANDA = { _KNOWN_OANDA = {
# "account_id" is TOLERATED but ignored: it belonged to the auto- # "account_id" is TOLERATED but ignored: it belonged to the auto-
# execution daemon removed 2026-09-03 (last commit 31002cb). The candle # execution daemon removed 2026-09-03 (last commit 31002cb). The candle
@@ -113,6 +115,19 @@ _KNOWN_SIGNAL_LOG = {"enabled", "path", "log_blocked_scans"}
_KNOWN_NEWS_ARCHIVE = {"enabled", "path"} _KNOWN_NEWS_ARCHIVE = {"enabled", "path"}
_KNOWN_NEWS_MODEL = {"path", "threads", "mode", "deepseek_audit_per_day"} _KNOWN_NEWS_MODEL = {"path", "threads", "mode", "deepseek_audit_per_day"}
_NEWS_MODEL_MODES = ("off", "fallback", "primary") _NEWS_MODEL_MODES = ("off", "fallback", "primary")
# Lot-size guidance on the alert (exposure.py, display only). Off until
# the owner sets risk_per_trade. The size reveals the owner's risk per
# trade, so it is printed in the owner's own chat (telegram.chat_id) and
# the CLI only; telegram_channel switches it on for every other Telegram
# chat (the subscriber channel, a group), dashboard for the public page.
_KNOWN_TRADE_SIZE = {"risk_per_trade", "contract_size", "lot_step",
"telegram_channel", "dashboard"}
_TRADE_SIZE_DEFAULTS = {
"contract_size": 100.0, # ounces per 1.00 lot of XAU/USD
"lot_step": 0.01, # smallest size increment the broker accepts
"telegram_channel": False,
"dashboard": False,
}
_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"} _KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
_KNOWN_SCANNING = {"pairs", "timeframes"} _KNOWN_SCANNING = {"pairs", "timeframes"}
_KNOWN_PAIR_KEYS = {"blocked_hours_utc"} _KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
@@ -153,6 +168,7 @@ class AppConfig:
signal_log_settings: Dict[str, Any] = field(default_factory=dict) signal_log_settings: Dict[str, Any] = field(default_factory=dict)
news_archive_settings: Dict[str, Any] = field(default_factory=dict) news_archive_settings: Dict[str, Any] = field(default_factory=dict)
news_model_settings: Dict[str, Any] = field(default_factory=dict) news_model_settings: Dict[str, Any] = field(default_factory=dict)
trade_size_settings: Dict[str, Any] = field(default_factory=dict)
log_level: str = "INFO" log_level: str = "INFO"
@@ -432,6 +448,27 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
"is not off") "is not off")
app.news_model_settings = dict(nm) app.news_model_settings = dict(nm)
size = raw.get("trade_size", {})
_check_keys(size, _KNOWN_TRADE_SIZE, '"trade_size"')
for key in ("risk_per_trade", "contract_size", "lot_step"):
if key in size:
v = size[key]
if (isinstance(v, bool) or not isinstance(v, (int, float))
or not math.isfinite(v)):
raise AppConfigError(
f"config.json: trade_size.{key} must be a number")
# risk_per_trade 0 = the size line is off (the default)
if v < 0 or (v == 0 and key != "risk_per_trade"):
raise AppConfigError(
f"config.json: trade_size.{key} must be "
+ ("zero (off) or a positive number"
if key == "risk_per_trade" else "a positive number"))
for flag in ("telegram_channel", "dashboard"):
if flag in size and not isinstance(size[flag], bool):
raise AppConfigError(
f"config.json: trade_size.{flag} must be true or false")
app.trade_size_settings = dict(size)
level = raw.get("log_level", app.log_level) level = raw.get("log_level", app.log_level)
if level not in _VALID_LOG_LEVELS: if level not in _VALID_LOG_LEVELS:
raise AppConfigError( raise AppConfigError(
@@ -685,6 +722,21 @@ def news_archive_path(app: Optional[AppConfig] = None) -> str:
return path return path
# ── Lot-size guidance (config.json only; display) ────────────────────
def trade_size_settings(app: Optional[AppConfig] = None) -> Optional[Dict[str, Any]]:
"""The lot-size guidance settings with their defaults filled in, or
None when the owner has not set `trade_size.risk_per_trade` (the size
line is then off on every surface)."""
app = app or get_app_config()
block = app.trade_size_settings or {}
if not block.get("risk_per_trade"):
return None
out = dict(_TRADE_SIZE_DEFAULTS)
out.update(block)
return out
# ── PlatformConfig overlay ──────────────────────────────────────────── # ── PlatformConfig overlay ────────────────────────────────────────────
def apply_pair_overrides( def apply_pair_overrides(
+54 -5
View File
@@ -45,7 +45,9 @@ if str(PROJECT_ROOT) not in sys.path:
from config import PAIR_INFO, PlatformConfig, Timeframe # noqa: E402 from config import PAIR_INFO, PlatformConfig, Timeframe # noqa: E402
from providers import make_data_provider # noqa: E402 from providers import make_data_provider # noqa: E402
from providers.base import BaseDataProvider # noqa: E402 from providers.base import BaseDataProvider # noqa: E402
from signal_aggregator import SignalAggregator # noqa: E402 (fingerprints + the two static cleanups only) from signal_aggregator import ( # noqa: E402 (fingerprints, the two static cleanups, the daily-trend definition)
SignalAggregator, daily_trend_needed, daily_trend_series,
)
import position_sizing # noqa: E402 import position_sizing # noqa: E402
from backtest import gpu_kernels as _gk # noqa: E402 from backtest import gpu_kernels as _gk # noqa: E402
from backtest import cuda_signal_kernels as ck # noqa: E402 from backtest import cuda_signal_kernels as ck # noqa: E402
@@ -318,6 +320,12 @@ def format_warning(code: int, arg: float, ctx: Dict) -> str:
if n == "rf_blocked": if n == "rf_blocked":
return (f"REGIME FILTER: {ctx['rf_tf']} ADX={arg:.1f} < " return (f"REGIME FILTER: {ctx['rf_tf']} ADX={arg:.1f} < "
f"{ctx['regime_filter_min_adx']:.0f} — market choppy, skipping signal") f"{ctx['regime_filter_min_adx']:.0f} — market choppy, skipping signal")
if n == "trend_veto":
is_buy = arg > 0
return (f"DAILY TREND FILTER: {'BUY' if is_buy else 'SELL'} against a "
f"{'falling' if is_buy else 'rising'} daily trend — no trade")
if n == "trend_unavailable":
return "Daily trend filter requested but 1d data unavailable"
raise ValueError(f"unknown warning code {code}") raise ValueError(f"unknown warning code {code}")
@@ -379,6 +387,34 @@ def host_flags(primary_index: pd.DatetimeIndex, platform_cfg: PlatformConfig,
} }
def trend_flags(primary_index: pd.DatetimeIndex, d1: Optional[pd.DataFrame],
bt_cfg: BacktestConfig) -> np.ndarray:
"""Per primary bar: the daily trend the live code reads on the D1 slice
_replay_span hands it (+1 / 0 / -1), or ck.DAILY_TREND_UNAVAILABLE when
that slice holds no usable bar.
The definition is signal_aggregator.daily_trend_series, evaluated once on
the whole frame (every step is causal, so the value at a bar equals the
value on any prefix ending there). The as-of rule is the replay's: a D1
bar is visible once label + 1 day <= the primary bar's close, compared in
int64 nanoseconds; the Sunday stubs daily_trend_series drops are skipped
by searching its own (kept-bar) index."""
n = len(primary_index)
out = np.full(n, ck.DAILY_TREND_UNAVAILABLE, dtype=np.int8)
if d1 is None or len(d1) == 0:
return out
series = daily_trend_series(d1)
if len(series) == 0:
return out
primary_ns = primary_index.as_unit("ns").asi8
cutoff = primary_ns + (BAR_SECONDS[bt_cfg.primary_tf] - BAR_SECONDS[Timeframe.D1]) * 1_000_000_000
labels_ns = series.index.as_unit("ns").asi8
k = np.searchsorted(labels_ns, cutoff, side="right")
seen = k > 0
out[seen] = series.to_numpy()[k[seen] - 1]
return out
def replay_bounds(primary_df: pd.DataFrame, bt_cfg: BacktestConfig) -> Tuple[int, int]: def replay_bounds(primary_df: pd.DataFrame, bt_cfg: BacktestConfig) -> Tuple[int, int]:
bt_start_ts = pd.Timestamp(bt_cfg.start) bt_start_ts = pd.Timestamp(bt_cfg.start)
bt_end_ts = pd.Timestamp(bt_cfg.end) bt_end_ts = pd.Timestamp(bt_cfg.end)
@@ -421,6 +457,8 @@ def refuse_unsupported(platform_cfg: PlatformConfig, bt_cfg: BacktestConfig) ->
problems.append(f"technical.ema_mode={t.ema_mode!r}") problems.append(f"technical.ema_mode={t.ema_mode!r}")
if platform_cfg.signals.grading_mode not in ck.GRADING_MODE_CODES: if platform_cfg.signals.grading_mode not in ck.GRADING_MODE_CODES:
problems.append(f"signals.grading_mode={platform_cfg.signals.grading_mode!r}") problems.append(f"signals.grading_mode={platform_cfg.signals.grading_mode!r}")
if x.daily_trend_veto not in ck.DAILY_TREND_CODES:
problems.append(f"xau_strategy.daily_trend_veto={x.daily_trend_veto!r}")
if problems: if problems:
raise SystemExit("cudaBasedBacktesting.py cannot run this configuration: " raise SystemExit("cudaBasedBacktesting.py cannot run this configuration: "
+ "; ".join(problems) + ". Use backtest/gpu_backtest.py.") + "; ".join(problems) + ". Use backtest/gpu_backtest.py.")
@@ -456,6 +494,11 @@ def analyze_frames(bt_cfg: BacktestConfig, platform_cfg: PlatformConfig,
rf_cut = cuts[rtf] if rtf != bt_cfg.primary_tf else cp.arange(1, primary_dev["n"] + 1, dtype=cp.int64) rf_cut = cuts[rtf] if rtf != bt_cfg.primary_tf else cp.arange(1, primary_dev["n"] + 1, dtype=cp.int64)
rf = (dev[rtf]["adx"], rf_cut) rf = (dev[rtf]["adx"], rf_cut)
flags = host_flags(primary_df.index, platform_cfg, bt_cfg) flags = host_flags(primary_df.index, platform_cfg, bt_cfg)
# The daily trend per primary bar (read by K2 only when the veto is on).
flags["daily_trend"] = (
trend_flags(primary_df.index, dfs_full.get(Timeframe.D1), bt_cfg)
if daily_trend_needed(platform_cfg)
else np.zeros(len(primary_df), dtype=np.int8))
first_eligible_pos, last_pos = replay_bounds(primary_df, bt_cfg) first_eligible_pos, last_pos = replay_bounds(primary_df, bt_cfg)
logger.info(f"Replaying {bt_cfg.pair} {bt_cfg.primary_tf.value}: " logger.info(f"Replaying {bt_cfg.pair} {bt_cfg.primary_tf.value}: "
f"bars {first_eligible_pos}–{last_pos} of {len(primary_df)}") f"bars {first_eligible_pos}–{last_pos} of {len(primary_df)}")
@@ -508,6 +551,9 @@ def run_backtest_cuda(bt_cfg: BacktestConfig, platform_cfg: PlatformConfig) -> T
flt = platform_cfg.backtest flt = platform_cfg.backtest
if flt.regime_filter and flt.regime_filter_timeframe not in timeframes: if flt.regime_filter and flt.regime_filter_timeframe not in timeframes:
timeframes.append(flt.regime_filter_timeframe) timeframes.append(flt.regime_filter_timeframe)
# the D1 frame of the daily-trend veto, appended exactly like the reference script
if daily_trend_needed(platform_cfg) and Timeframe.D1 not in timeframes:
timeframes.append(Timeframe.D1)
dfs_full: Dict[Timeframe, pd.DataFrame] = {} dfs_full: Dict[Timeframe, pd.DataFrame] = {}
cache_names: Dict[str, str] = {} cache_names: Dict[str, str] = {}
tf_starts = {} tf_starts = {}
@@ -615,27 +661,30 @@ def _warning_ctx(ctx, pos: int, r2i: Dict, r2f: Dict) -> Dict:
_RF_CODES = {ck.WARN_CODES["rf_unavailable"], ck.WARN_CODES["rf_no_adx"], ck.WARN_CODES["rf_blocked"]} _RF_CODES = {ck.WARN_CODES["rf_unavailable"], ck.WARN_CODES["rf_no_adx"], ck.WARN_CODES["rf_blocked"]}
_TREND_CODES = {ck.WARN_CODES["trend_veto"], ck.WARN_CODES["trend_unavailable"]}
def build_warnings(ctx, pos: int, r2i: Dict, r2f: Dict, codes, args) -> Tuple[List[str], List[str]]: def build_warnings(ctx, pos: int, r2i: Dict, r2f: Dict, codes, args) -> Tuple[List[str], List[str]]:
"""(warnings, cleaned_warnings) of one analysed bar, after the XAU_CP """(warnings, cleaned_warnings) of one analysed bar, after the XAU_CP
cleanup (when a pattern setup was accepted) and the stale spread-gate cleanup (when a pattern setup was accepted) and the stale spread-gate
step (when a setup existed at step 11), with the regime-filter lines step (when a setup existed at step 11), with the daily-trend line of
appended last like the replay does.""" step 12 and then the regime-filter lines appended last, in the order
the live code and the replay write them."""
wctx = _warning_ctx(ctx, pos, r2i, r2f) wctx = _warning_ctx(ctx, pos, r2i, r2f)
n = r2i["warn_n"] n = r2i["warn_n"]
main: List[str] = [] main: List[str] = []
trend: List[str] = []
rf: List[str] = [] rf: List[str] = []
for q in range(n): for q in range(n):
code = int(codes[q]) code = int(codes[q])
text = format_warning(code, float(args[q]), wctx) text = format_warning(code, float(args[q]), wctx)
(rf if code in _RF_CODES else main).append(text) (rf if code in _RF_CODES else trend if code in _TREND_CODES else main).append(text)
ns = types.SimpleNamespace(warnings=main, cleaned_warnings=[], ns = types.SimpleNamespace(warnings=main, cleaned_warnings=[],
trade_setup=object() if r2i["strategy_pre_rf"] != 0 else None) trade_setup=object() if r2i["strategy_pre_rf"] != 0 else None)
if r2i["cp_accepted"]: if r2i["cp_accepted"]:
SignalAggregator._clean_accepted_cp_warnings(ns) SignalAggregator._clean_accepted_cp_warnings(ns)
SignalAggregator._drop_stale_spread_gate_vetoes(ns) SignalAggregator._drop_stale_spread_gate_vetoes(ns)
return list(ns.warnings) + rf, list(ns.cleaned_warnings) return list(ns.warnings) + trend + rf, list(ns.cleaned_warnings)
def build_reasons(ctx, r2i: Dict, r2f: Dict, inds: Dict[str, Dict]) -> Tuple[List[str], List[str]]: def build_reasons(ctx, r2i: Dict, r2f: Dict, inds: Dict[str, Dict]) -> Tuple[List[str], List[str]]:
+63 -2
View File
@@ -12,9 +12,13 @@ the repo and stays the reference:
thread analyses the prefix [0..j]. thread analyses the prefix [0..j].
K2 fb2_aggregate signal_aggregator.analyze_from_dataframes end to end K2 fb2_aggregate signal_aggregator.analyze_from_dataframes end to end
(MTF merge, penalties, grade, gates 6b-6i, the three (MTF merge, penalties, grade, gates 6b-6i, the three
setup generators, XAU_PB, XAU_CP, the stale-veto step) setup generators with the wide-stop cap, XAU_PB,
XAU_CP, the stale-veto step, the daily-trend veto)
plus the per-bar rules of gpu_backtest._replay_span. plus the per-bar rules of gpu_backtest._replay_span.
One thread per primary bar. One thread per primary bar. The daily trend itself
arrives as a host flag (cudaBasedBacktesting.
trend_flags): its pandas definition lives in
signal_aggregator.daily_trend_series only.
K3 fb2_features position_sizing.build_features (65 features), one K3 fb2_features position_sizing.build_features (65 features), one
thread per accepted trade (--export-features). thread per accepted trade (--export-features).
K4 fb2_cooldown gpu_backtest._apply_cooldown (single thread). K4 fb2_cooldown gpu_backtest._apply_cooldown (single thread).
@@ -155,6 +159,8 @@ WARN_NAMES = (
"rf_unavailable", # 33 regime filter: TF missing "rf_unavailable", # 33 regime filter: TF missing
"rf_no_adx", # 34 regime filter: ADX None "rf_no_adx", # 34 regime filter: ADX None
"rf_blocked", # 35 arg adx "rf_blocked", # 35 arg adx
"trend_veto", # 36 daily-trend veto: arg setup direction (+1 / -1)
"trend_unavailable", # 37 daily-trend veto on, no usable D1 bar
) )
WARN_CODES = {n: i for i, n in enumerate(WARN_NAMES)} WARN_CODES = {n: i for i, n in enumerate(WARN_NAMES)}
@@ -206,6 +212,13 @@ SKIP_NAMES = {1: "WEEKEND_FILTER", 2: "DAY_FILTER", 3: "SESSION_WINDOW"}
# 1 = min conviction, 2 = product conviction). Pinned by name, never by the # 1 = min conviction, 2 = product conviction). Pinned by name, never by the
# position in signal_aggregator.GRADING_MODES; pack_params checks the two agree. # position in signal_aggregator.GRADING_MODES; pack_params checks the two agree.
GRADING_MODE_CODES = {"pair": 0, "min": 1, "product": 2} GRADING_MODE_CODES = {"pair": 0, "min": 1, "product": 2}
# xau_strategy.daily_trend_veto (signal_aggregator.DAILY_TREND_MODES), pinned
# by name like the grading modes; pack_params checks the two agree. The
# per-bar trend itself is computed on the host (cudaBasedBacktesting.
# trend_flags, from signal_aggregator.daily_trend_series): +1 / 0 / -1, and
# DAILY_TREND_UNAVAILABLE where the D1 slice holds no usable bar.
DAILY_TREND_CODES = {"off": 0, "both": 1, "buy": 2, "sell": 3}
DAILY_TREND_UNAVAILABLE = 2
# ── Config packing ──────────────────────────────────────────────────── # ── Config packing ────────────────────────────────────────────────────
PARAMS_F = ( PARAMS_F = (
@@ -228,6 +241,7 @@ PARAMS_F = (
# all 0.0 under "pair"): aplus, a, b, c, strong, buy, boost_a, boost_b # all 0.0 under "pair"): aplus, a, b, c, strong, buy, boost_a, boost_b
"conv_aplus", "conv_a", "conv_b", "conv_c", "conv_aplus", "conv_a", "conv_b", "conv_c",
"conv_strong", "conv_buy", "conv_boost_a", "conv_boost_b", "conv_strong", "conv_buy", "conv_boost_a", "conv_boost_b",
"wide_stop_cap_atr", # 0.0 = off (GENERIC and XAU_CP stop cap, after the gates)
) )
PARAMS_I = ( PARAMS_I = (
"ema_trend_period", "ema_mode_2ema", "ema_partial_neutral", "rsi_regime_aware", "ema_trend_period", "ema_mode_2ema", "ema_partial_neutral", "rsi_regime_aware",
@@ -245,6 +259,7 @@ PARAMS_I = (
"quality_filter_mask", "strategies_mask", "primary_is_m5", "primary_bar_secs", "quality_filter_mask", "strategies_mask", "primary_is_m5", "primary_bar_secs",
"regime_filter", "n_confirm", "regime_filter", "n_confirm",
"grading_mode", # 0 pair (shipped) / 1 min / 2 product "grading_mode", # 0 pair (shipped) / 1 min / 2 product
"daily_trend_veto", # 0 off / 1 both / 2 buy / 3 sell (DAILY_TREND_CODES)
) )
# Fields of the four config dataclasses the kernels do NOT read, with the # Fields of the four config dataclasses the kernels do NOT read, with the
@@ -377,6 +392,12 @@ def pack_params(platform_cfg, bt_cfg) -> Tuple[np.ndarray, np.ndarray, np.ndarra
for name, val in zip(("conv_aplus", "conv_a", "conv_b", "conv_c", for name, val in zip(("conv_aplus", "conv_a", "conv_b", "conv_c",
"conv_strong", "conv_buy", "conv_boost_a", "conv_boost_b"), conv_t): "conv_strong", "conv_buy", "conv_boost_a", "conv_boost_b"), conv_t):
f[name] = val f[name] = val
f["wide_stop_cap_atr"] = x.wide_stop_cap_atr
# daily-trend veto: the live module owns the mode list (same local import)
from signal_aggregator import DAILY_TREND_MODES, daily_trend_mode_of
assert set(DAILY_TREND_MODES) == set(DAILY_TREND_CODES), (
f"signal_aggregator.DAILY_TREND_MODES {DAILY_TREND_MODES} != the kernel's {sorted(DAILY_TREND_CODES)}")
daily_trend_veto = DAILY_TREND_CODES[daily_trend_mode_of(x)]
from providers.base import BaseDataProvider # CANDLE_SECONDS (the aggregator's clock) from providers.base import BaseDataProvider # CANDLE_SECONDS (the aggregator's clock)
bar_secs = BaseDataProvider.CANDLE_SECONDS[platform_cfg.primary_timeframe] bar_secs = BaseDataProvider.CANDLE_SECONDS[platform_cfg.primary_timeframe]
strategies = list(bt_cfg.strategies_only or []) strategies = list(bt_cfg.strategies_only or [])
@@ -431,6 +452,7 @@ def pack_params(platform_cfg, bt_cfg) -> Tuple[np.ndarray, np.ndarray, np.ndarra
"regime_filter": 1 if b.regime_filter else 0, "regime_filter": 1 if b.regime_filter else 0,
"n_confirm": len(platform_cfg.confirmation_timeframes), "n_confirm": len(platform_cfg.confirmation_timeframes),
"grading_mode": GRADING_MODE_CODES[grading_mode], "grading_mode": GRADING_MODE_CODES[grading_mode],
"daily_trend_veto": daily_trend_veto,
} }
pf = np.array([float(f[name]) for name in PARAMS_F], dtype=np.float64) pf = np.array([float(f[name]) for name in PARAMS_F], dtype=np.float64)
pi = np.array([int(i[name]) for name in PARAMS_I], dtype=np.int32) pi = np.array([int(i[name]) for name in PARAMS_I], dtype=np.int32)
@@ -480,6 +502,9 @@ def _defines() -> str:
lines.append(f"#define Q_{name.replace('+', 'PLUS')} {code}") lines.append(f"#define Q_{name.replace('+', 'PLUS')} {code}")
for name, code in STRATEGY_CODES.items(): for name, code in STRATEGY_CODES.items():
lines.append(f"#define STRAT_{name or 'NONE'} {code}") lines.append(f"#define STRAT_{name or 'NONE'} {code}")
for name, code in DAILY_TREND_CODES.items():
lines.append(f"#define TREND_{name} {code}")
lines.append(f"#define FB2_TREND_UNAVAILABLE {DAILY_TREND_UNAVAILABLE}")
return "\n".join(lines) + "\n" return "\n".join(lines) + "\n"
@@ -1512,6 +1537,7 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
const double* __restrict__ rf_adx, const long long* __restrict__ rf_cut, const double* __restrict__ rf_adx, const long long* __restrict__ rf_cut,
const signed char* __restrict__ skip_reason, const signed char* __restrict__ label_hour, const signed char* __restrict__ skip_reason, const signed char* __restrict__ label_hour,
const signed char* __restrict__ label_wd, const signed char* __restrict__ close_hour, const signed char* __restrict__ label_wd, const signed char* __restrict__ close_hour,
const signed char* __restrict__ daily_trend_p,
const double* __restrict__ PF, const int* __restrict__ PI, const double* __restrict__ PF, const int* __restrict__ PI,
const int first_pos, const int n_bars, const int first_pos, const int n_bars,
double* __restrict__ OF2, int* __restrict__ OI2, double* __restrict__ OF2, int* __restrict__ OI2,
@@ -1933,6 +1959,16 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
if (!(rr < min_rr - 1e-9)) { if (!(rr < min_rr - 1e-9)) {
if (tp1_distance < min_tp) { WARN(W_spread_gate_generic, tp1_distance); } if (tp1_distance < min_tp) { WARN(W_spread_gate_generic, tp1_distance); }
else { else {
// wide-stop cap AFTER the gates (signal_aggregator._wide_stop_cap, 0 = off)
if (PF[PF_wide_stop_cap_atr] > 0.0 && xau_enabled && is_xau) {
const double stop_floor = PYMAX(atr * sl_atr_floor, min_sl_price);
const double cap_dist = PYMAX(PF[PF_wide_stop_cap_atr] * atr, stop_floor);
if (sl_distance > cap_dist) {
sl_distance = cap_dist;
sl_price = is_buy ? (entry - cap_dist) : (entry + cap_dist);
rr = tp1_distance / sl_distance;
}
}
{ // printed-target floor AFTER the gates (signal_aggregator 2026-09-26) { // printed-target floor AFTER the gates (signal_aggregator 2026-09-26)
const double floor_dist = sl_distance * PF[PF_tp_print_floor_rr]; const double floor_dist = sl_distance * PF[PF_tp_print_floor_rr];
if (tp1_distance < floor_dist) { if (tp1_distance < floor_dist) {
@@ -2079,6 +2115,15 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
} }
double rr = (sl_dist > 0.0) ? height / sl_dist : 0.0; double rr = (sl_dist > 0.0) ? height / sl_dist : 0.0;
if (!(rr < min_rr - 1e-9) && !(height < min_tp)) { if (!(rr < min_rr - 1e-9) && !(height < min_tp)) {
// wide-stop cap AFTER the gates (signal_aggregator._wide_stop_cap, 0 = off)
if (PF[PF_wide_stop_cap_atr] > 0.0 && xau_enabled && is_xau) {
const double cap_dist = PYMAX(PF[PF_wide_stop_cap_atr] * atr, min_sl_price);
if (sl_dist > cap_dist) {
sl_dist = cap_dist;
sl_ = is_buy ? (entry - cap_dist) : (entry + cap_dist);
rr = height / sl_dist;
}
}
{ // printed-target floor AFTER the gates (signal_aggregator 2026-09-26) { // printed-target floor AFTER the gates (signal_aggregator 2026-09-26)
double tp1_dist = height; double tp1_dist = height;
const double floor_dist = sl_dist * PF[PF_tp_print_floor_rr]; const double floor_dist = sl_dist * PF[PF_tp_print_floor_rr];
@@ -2103,7 +2148,22 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
} }
// ── replay: the optional higher-TF regime filter (after analyze_from_dataframes) ── // ── replay: the optional higher-TF regime filter (after analyze_from_dataframes) ──
// strategy_pre_rf = the setup at step 11 (the stale spread-gate cleanup on
// the host), i.e. BEFORE the daily-trend veto of step 12 and the filter.
K2I(strategy_pre_rf) = strategy; K2I(strategy_pre_rf) = strategy;
// ── 12. daily-trend veto (signal_aggregator._apply_daily_trend_veto) ──
if (PI[PI_daily_trend_veto] != 0 && xau_enabled && is_xau) {
const int td = daily_trend_p[pos];
if (td == FB2_TREND_UNAVAILABLE) { WARN(W_trend_unavailable, 0.0); }
else if (strategy != STRAT_NONE && td != 0) {
const int mode = PI[PI_daily_trend_veto];
const bool against = (direction > 0 && td < 0 && (mode == TREND_both || mode == TREND_buy))
|| (direction < 0 && td > 0 && (mode == TREND_both || mode == TREND_sell));
if (against) { WARN(W_trend_veto, (double)direction); quality = Q_D; strategy = STRAT_NONE; }
}
}
if (PI[PI_regime_filter]) { if (PI[PI_regime_filter]) {
const long long cut = rf_cut[pos]; const long long cut = rf_cut[pos];
if (cut == 0) { K2I(rf_state) = 1; WARN(W_rf_unavailable, 0.0); } if (cut == 0) { K2I(rf_state) = 1; WARN(W_rf_unavailable, 0.0); }
@@ -2780,6 +2840,7 @@ def aggregate(primary_dev: Dict, primary_an: TFAnalysis,
rf_adx, rf_cut, rf_adx, rf_cut,
_dev(flags["skip_reason"], np.int8), _dev(flags["label_hour"], np.int8), _dev(flags["skip_reason"], np.int8), _dev(flags["label_hour"], np.int8),
_dev(flags["label_wd"], np.int8), _dev(flags["close_hour"], np.int8), _dev(flags["label_wd"], np.int8), _dev(flags["close_hour"], np.int8),
_dev(flags["daily_trend"], np.int8),
_dev(pf, np.float64), _dev(pi, np.int32), _dev(pf, np.float64), _dev(pi, np.int32),
np.int32(first_pos), np.int32(n_bars), OF2, OI2, warn_code, warn_arg)) np.int32(first_pos), np.int32(n_bars), OF2, OI2, warn_code, warn_arg))
cp.cuda.Device().synchronize() cp.cuda.Device().synchronize()
+8 -1
View File
@@ -63,7 +63,9 @@ from config import ( # noqa: E402
) )
from providers import BaseDataProvider, make_data_provider # noqa: E402 from providers import BaseDataProvider, make_data_provider # noqa: E402
from engine import IndicatorSignal, TechnicalAnalysis, TechnicalEngine # noqa: E402 from engine import IndicatorSignal, TechnicalAnalysis, TechnicalEngine # noqa: E402
from signal_aggregator import SignalAggregator, SignalReport, TradeSetup # noqa: E402 from signal_aggregator import ( # noqa: E402
SignalAggregator, SignalReport, TradeSetup, daily_trend_needed,
)
import position_sizing # noqa: E402 import position_sizing # noqa: E402
from backtest import replay_record # noqa: E402 from backtest import replay_record # noqa: E402
from backtest import gpu_kernels as _gk # noqa: E402 from backtest import gpu_kernels as _gk # noqa: E402
@@ -2084,6 +2086,11 @@ def run_backtest(
flt = platform_cfg.backtest flt = platform_cfg.backtest
if flt.regime_filter and flt.regime_filter_timeframe not in timeframes: if flt.regime_filter and flt.regime_filter_timeframe not in timeframes:
timeframes.append(flt.regime_filter_timeframe) timeframes.append(flt.regime_filter_timeframe)
# Same for the daily-trend veto: analyze_pair fetches D1 as a required
# frame when it is on, and the core reads it from the slices (the
# as-of slicing in _replay_span hands it closed daily bars only).
if daily_trend_needed(platform_cfg) and Timeframe.D1 not in timeframes:
timeframes.append(Timeframe.D1)
dfs_full: Dict[Timeframe, pd.DataFrame] = {} dfs_full: Dict[Timeframe, pd.DataFrame] = {}
# Which parquet each TF came from (recorded in --record's manifest so # Which parquet each TF came from (recorded in --record's manifest so
# a study can prove two runs read the same bars). # a study can prove two runs read the same bars).
+23
View File
@@ -65,3 +65,26 @@ re-implementation.
and it refuses configurations beyond its compile-time limits (more than 4 confirmation and it refuses configurations beyond its compile-time limits (more than 4 confirmation
timeframes, divergence lookback >= 64, swing lookback >= 256, cp_lookback >= 128) with a timeframes, divergence lookback >= 64, swing lookback >= 256, cp_lookback >= 128) with a
message that points back to gpu_backtest.py. message that points back to gpu_backtest.py.
--- 2026-09-30: daily-trend veto (M15) and the D1 frame ---
- The M15 factory config carries xau_strategy.daily_trend_veto = "both" (M15 override 7). Both
backtesters then load the D1 frame as an EXTRA frame next to the primary and confirmation
frames (the "Confirm TFs:" line does not list it), cached under its own pad:
backtest/cache/oanda_XAU_USD_1d_<start minus 1400 days>_<end>.parquet. A new M15 window therefore
needs that file (one network fetch the first time; "GPU cache fallbacks: 0" is unaffected).
- The veto reads CLOSED daily bars only (a bar labelled D counts from D + 1 day <= the primary
bar's close), drops the provider's Sunday stub bars, and removes a finished setup that goes
against a non-zero state. Definition and constants: signal_aggregator.daily_trend_series.
- Research switches, both default-neutral elsewhere: --set xau_strategy.daily_trend_veto=
off|both|buy|sell and --set xau_strategy.wide_stop_cap_atr=K (K = 0 is off; the cap FAILED its
rule on 2026-09-30 and ships nowhere). A control for an M15 candidate is the factory config,
i.e. WITH the veto; to reproduce a pre-2026-09-30 M15 book pass --set
xau_strategy.daily_trend_veto=off.
- A candidate that is a FINAL veto (it only removes finished setups) may be screened on a research
harness (K1 + K2 of the twin, a mask on the final candidates before the cooldown, the shared
resolver) once that harness is shown to reproduce a shipped book byte for byte; the formal
numbers still come from the real code path with the paired gpu_backtest.py MD5.
- The 2026-09-30 report (backtest/research/daily_trend_campaign_2026-09-30.md) is the template for
a risk-first rule: rules, windows and neighbours written before the runs; the long window
(2019-01-01 -> 2026-08-07) and an unseen holdout as clauses; the rising-market cost reported,
not hidden.
@@ -0,0 +1,235 @@
# Context screens after the daily-trend veto — declared before looking, 2026-10-01
Follow-up to `daily_trend_campaign_2026-09-30.md` (owner decision 2026-09-30: "more context
screens"). The daily chart was information the engine had never seen, and it paid on M15. This
file asks the same question of seven more pieces of context. Sections 1 to 3 were written before
any number in section 4 existed.
## 1. Books and slices
The books as shipped on 2026-09-30, `--spread 0.65`, CUDA twin (byte-identical to
`gpu_backtest.py` on the paired runs of the campaign report):
| Book | Window | Trades / R | MD5 |
|---|---|---|---|
| M15 LONG (veto on) | 2019-01-01 → 2026-08-07 | 1473 / +223.33 | `aa5253aa…` |
| M15 W3 (veto on) | 2023-09-05 → 2026-09-05 | 824 / +205.12 | `ad3b0dbd…` |
| M5 LONG | 2019-01-01 → 2026-08-07 | 1052 / +128.10 | `3f0c091c…` |
| M5 W3 | 2023-09-05 → 2026-09-05 | 870 / +153.64 | `d20ec8a3…` |
Four slices per timeframe: LONG era A (entries 2019–2023), LONG era B (2024 → 2026-08-07), W3
first half (before 2025-03-05), W3 second half. W3 overlaps era B, so the four slices are not
independent; that is known and accepted for a screen.
## 2. The seven features (closed bars only, as of the trade's decision time)
Decision time = the trade's entry time (the entry is the open of the bar after the signal bar, so
it equals the signal bar's close). A bar counts only when it had closed by then.
Daily bars: provider D1 candles (UTC days), Sunday stub bars dropped, usable from label + 1 day.
1. **Weekly trend.** Weekly bars = the kept daily bars grouped into weeks ending Friday; a week is
usable once its Friday bar is. State +1 when the weekly close is above its 20-week EMA
(`ewm(span=20, adjust=False)`) AND above the close 12 weeks earlier; −1 when below both; else 0.
Trade state: `with` / `against` / `unclear`.
2. **Four-hour trend.** Provider H4 candles (UTC-aligned), usable from label + 4 h. State +1 when
the close is above its 50-bar EMA AND above the close 30 bars earlier; −1 when below both; else
0. Trade state: `with` / `against` / `unclear`.
3. **Daily-trend strength.** `(Close − Close 60 kept days earlier) / ATR14`, ATR14 = the simple
14-day mean of the true range, taken in the trade's direction (multiplied by −1 for a SELL).
Trade state by fixed cut-offs: `counter` (< 0), `weak` (0 to 5), `medium` (5 to 12),
`strong` (> 12 daily ATRs).
4. **Daily volatility state.** ATR14 / Close, ranked within its own trailing 250 kept days
(percentile, the current day included). Trade state: `quiet` (< 33%), `normal`, `wild` (> 67%).
5. **Place in the 20-day range.** `(Close − lowest Low of 20 kept days) / (highest High − lowest
Low)`, mirrored for a SELL (1 − value). Trade state: `near the far edge` (< 1/3: buying near
the 20-day low or selling near the high), `middle`, `near the chasing edge` (> 2/3: buying near
the 20-day high or selling near the low).
6. **Tick-volume surge.** Volume of the signal bar divided by the median volume of the same clock
slot (same hour and minute, UTC) over that slot's previous 20 bars. Trade state: `thin`
(< 0.8), `usual` (0.8 to 1.5), `surge` (> 1.5).
7. **Time to a high-impact US release** (`backtest/cache/calendar.parquet`, currency USD, impact
high, time not masked; the file starts 2023-09-01, so W3 only). Minutes from the decision time
to the nearest such release. Trade state: `0–30 min before`, `0–30 min after`,
`30–120 min before`, `30–120 min after`, `clear` (more than 2 hours either side).
Rows whose feature cannot be computed (not enough history) are counted as `n/a` and never flagged.
## 3. Rule
A screen is an in-book statistic: it cannot see the trades a change would add and it ignores the
5-bar cooldown. It can only nominate a hypothesis.
- A state is **nominated** only if, on one timeframe, it holds at least 8% of the trades of BOTH
books (LONG and W3) and its total R is negative in ALL FOUR slices.
- For each nominated state the mechanism is written down before its replay.
- At most **three** replays in total, on the final-veto research harness of the campaign (K1 + K2
of the twin, a mask on the final candidates before the cooldown, the shared resolver), judged by
the same risk-first rule T against the shipped book: W3 R ≥ 97% of control, W3 PF ≥ control +
0.10, W3 max drawdown ≤ control, LONG R ≥ control, LONG max drawdown ≤ control, 2021 + 2022
better, HOLDOUT R ≥ control − 0.5R, the same at $0.30, at least 20 trades differ. (No
neighbour-definition clause here: a pass only earns a full Stage-1/Stage-2 build, where the
neighbours are tested.)
- No nomination, no replay. No second cut-off, no other bins, no "best of seven" story afterwards.
With 7 features, about 25 states and 2 timeframes, a state that is negative in four overlapping
slices by luck alone is not rare; that is why a nomination is only a ticket to a replay.
## 4. Results
Run 2026-10-01 on the four books of section 1 (read from the campaign's exported CSVs; the feature
frames are the kept Oanda cache windows: daily `1d_20150303_20260807` / `1d_20191105_20260905`,
four-hour `4h_20181031_20260807` / `4h_20230114_20260905`, the books' own primary frames for
volume, `calendar.parquet` for releases). Every trade's decision time equals its entry time in
all four books (no entry sat behind a market closure). Cells are total R (trades); "mean" is R per
trade.
### 4.1 M15 (daily-trend veto on): LONG 1473 trades / +223.33R, W3 824 trades / +205.12R
**M15 — 1. weekly trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 78 (5%) | -15.0 / -0.193 | 25 (3%) | +3.9 / +0.157 | -18.3 (58) | +3.2 (20) | +3.8 (7) | +0.1 (18) | |
| unclear | 176 (12%) | +28.1 / +0.160 | 101 (12%) | +37.8 / +0.375 | +1.9 (102) | +26.2 (74) | +25.4 (61) | +12.5 (40) | |
| with | 1219 (83%) | +210.3 / +0.173 | 698 (85%) | +163.4 / +0.234 | +53.1 (568) | +157.2 (651) | +47.6 (316) | +115.8 (382) | |
**M15 — 2. four-hour trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 260 (18%) | +23.3 / +0.090 | 134 (16%) | +14.8 / +0.111 | +12.4 (137) | +10.9 (123) | +3.4 (59) | +11.4 (75) | |
| unclear | 167 (11%) | +16.8 / +0.100 | 84 (10%) | +16.7 / +0.199 | +0.3 (90) | +16.5 (77) | +1.8 (41) | +14.9 (43) | |
| with | 1046 (71%) | +183.3 / +0.175 | 606 (74%) | +173.6 / +0.286 | +24.1 (501) | +159.2 (545) | +71.5 (284) | +102.1 (322) | |
**M15 — 3. daily-trend strength**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| counter | 145 (10%) | -4.8 / -0.033 | 72 (9%) | +16.9 / +0.234 | -13.1 (92) | +8.3 (53) | +15.2 (39) | +1.7 (33) | |
| medium | 586 (40%) | +106.6 / +0.182 | 393 (48%) | +94.6 / +0.241 | +16.7 (199) | +89.9 (387) | +9.5 (162) | +85.1 (231) | |
| strong | 17 (1%) | +4.6 / +0.268 | 10 (1%) | +4.8 / +0.483 | -0.3 (7) | +4.8 (10) | -1.1 (1) | +5.9 (9) | |
| weak | 725 (49%) | +117.0 / +0.161 | 349 (42%) | +88.8 / +0.254 | +33.5 (430) | +83.5 (295) | +53.1 (182) | +35.7 (167) | |
**M15 — 4. daily volatility state**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| normal | 445 (30%) | +54.8 / +0.123 | 299 (36%) | +77.6 / +0.259 | -11.8 (194) | +66.6 (251) | +6.7 (118) | +70.8 (181) | |
| quiet | 373 (25%) | +64.1 / +0.172 | 152 (18%) | +51.7 / +0.340 | +24.2 (235) | +39.9 (138) | +38.9 (109) | +12.8 (43) | |
| wild | 655 (44%) | +104.4 / +0.159 | 373 (45%) | +75.9 / +0.203 | +24.3 (299) | +80.1 (356) | +31.1 (157) | +44.8 (216) | |
**M15 — 5. place in the 20-day range**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| middle | 420 (29%) | +54.9 / +0.131 | 230 (28%) | +32.3 / +0.140 | +25.3 (217) | +29.7 (203) | +30.9 (107) | +1.4 (123) | |
| near the chasing edge | 835 (57%) | +143.5 / +0.172 | 483 (59%) | +147.1 / +0.305 | +8.8 (400) | +134.7 (435) | +40.6 (227) | +106.5 (256) | |
| near the far edge | 218 (15%) | +24.8 / +0.114 | 111 (13%) | +25.7 / +0.231 | +2.6 (111) | +22.2 (107) | +5.3 (50) | +20.4 (61) | |
**M15 — 6. tick-volume surge**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| surge | 556 (38%) | +83.1 / +0.149 | 226 (27%) | +45.7 / +0.202 | +45.8 (355) | +37.3 (201) | +22.5 (110) | +23.1 (116) | |
| thin | 220 (15%) | +15.6 / +0.071 | 125 (15%) | +28.5 / +0.228 | -10.4 (99) | +26.0 (121) | +11.5 (48) | +17.0 (77) | |
| usual | 697 (47%) | +124.7 / +0.179 | 473 (57%) | +130.9 / +0.277 | +1.4 (274) | +123.2 (423) | +42.7 (226) | +88.3 (247) | |
**M15 — 7. time to a high-impact US release (W3 only)**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| 0–30 min after | | | 42 (5%) | +8.9 / +0.212 | | | +4.7 (27) | +4.2 (15) | |
| 0–30 min before | | | 26 (3%) | +8.7 / +0.336 | | | +7.8 (18) | +0.9 (8) | |
| 30–120 min after | | | 75 (9%) | +28.4 / +0.378 | | | +11.7 (51) | +16.6 (24) | |
| 30–120 min before | | | 22 (3%) | +16.1 / +0.732 | | | +12.9 (18) | +3.2 (4) | |
| clear | | | 659 (80%) | +143.0 / +0.217 | | | +39.6 (270) | +103.5 (389) | |
### 4.2 M5: LONG 1052 trades / +128.10R, W3 870 trades / +153.64R
**M5 — 1. weekly trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 258 (25%) | +11.0 / +0.043 | 202 (23%) | +6.6 / +0.033 | +9.4 (69) | +1.6 (189) | +3.8 (10) | +2.8 (192) | |
| unclear | 84 (8%) | +6.3 / +0.075 | 88 (10%) | +24.6 / +0.279 | -7.5 (30) | +13.8 (54) | +8.2 (12) | +16.4 (76) | |
| with | 710 (67%) | +110.8 / +0.156 | 580 (67%) | +122.5 / +0.211 | -12.3 (159) | +123.1 (551) | +4.7 (114) | +117.8 (466) | |
**M5 — 2. four-hour trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 332 (32%) | +3.0 / +0.009 | 255 (29%) | +11.2 / +0.044 | -5.5 (98) | +8.4 (234) | -0.9 (35) | +12.2 (220) | |
| unclear | 126 (12%) | +13.7 / +0.109 | 105 (12%) | +17.8 / +0.170 | +0.6 (28) | +13.1 (98) | +0.8 (13) | +17.0 (92) | |
| with | 594 (56%) | +111.4 / +0.188 | 510 (59%) | +124.6 / +0.244 | -5.5 (132) | +116.9 (462) | +16.9 (88) | +107.7 (422) | |
**M5 — 3. daily-trend strength**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| counter | 296 (28%) | +0.7 / +0.003 | 241 (28%) | -2.6 / -0.011 | +8.8 (87) | -8.0 (209) | +3.7 (18) | -6.3 (223) | |
| medium | 367 (35%) | +77.3 / +0.211 | 318 (37%) | +78.4 / +0.247 | -1.2 (49) | +78.4 (318) | +6.9 (53) | +71.6 (265) | |
| strong | 14 (1%) | +16.6 / +1.186 | 13 (1%) | +15.6 / +1.199 | +1.0 (1) | +15.6 (13) | +0.0 (0) | +15.6 (13) | |
| weak | 375 (36%) | +33.5 / +0.089 | 298 (34%) | +62.2 / +0.209 | -19.0 (121) | +52.5 (254) | +6.2 (65) | +56.1 (233) | |
**M5 — 4. daily volatility state**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| normal | 302 (29%) | +33.2 / +0.110 | 305 (35%) | +45.6 / +0.150 | -0.8 (63) | +34.0 (239) | -3.0 (42) | +48.6 (263) | |
| quiet | 114 (11%) | +18.8 / +0.165 | 72 (8%) | +17.9 / +0.249 | +3.0 (45) | +15.8 (69) | +10.9 (37) | +7.0 (35) | |
| wild | 636 (60%) | +76.1 / +0.120 | 493 (57%) | +90.1 / +0.183 | -12.6 (150) | +88.8 (486) | +8.8 (57) | +81.3 (436) | |
**M5 — 5. place in the 20-day range**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| middle | 323 (31%) | +11.8 / +0.037 | 271 (31%) | +18.8 / +0.069 | -6.8 (76) | +18.6 (247) | -1.8 (34) | +20.6 (237) | |
| near the chasing edge | 439 (42%) | +101.5 / +0.231 | 380 (44%) | +115.0 / +0.303 | -2.6 (103) | +104.1 (336) | +17.2 (77) | +97.8 (303) | |
| near the far edge | 290 (28%) | +14.8 / +0.051 | 219 (25%) | +19.9 / +0.091 | -1.0 (79) | +15.9 (211) | +1.4 (25) | +18.5 (194) | |
**M5 — 6. tick-volume surge**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| surge | 479 (46%) | +46.2 / +0.097 | 334 (38%) | +65.3 / +0.195 | -5.9 (176) | +52.1 (303) | +13.2 (64) | +52.1 (270) | |
| thin | 152 (14%) | +9.3 / +0.061 | 134 (15%) | +18.1 / +0.135 | -8.0 (22) | +17.3 (130) | -0.7 (7) | +18.8 (127) | |
| usual | 421 (40%) | +72.6 / +0.172 | 402 (46%) | +70.3 / +0.175 | +3.4 (60) | +69.2 (361) | +4.2 (65) | +66.1 (337) | |
**M5 — 7. time to a high-impact US release (W3 only)**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| 0–30 min after | | | 46 (5%) | -3.8 / -0.082 | | | -0.6 (15) | -3.2 (31) | |
| 0–30 min before | | | 18 (2%) | +1.8 / +0.098 | | | +0.7 (5) | +1.1 (13) | |
| 30–120 min after | | | 87 (10%) | +32.8 / +0.377 | | | +8.3 (31) | +24.4 (56) | |
| 30–120 min before | | | 18 (2%) | +3.4 / +0.187 | | | +2.7 (7) | +0.7 (11) | |
| clear | | | 701 (81%) | +119.5 / +0.171 | | | +5.6 (78) | +114.0 (623) | |
### 4.3 Verdict
**Nominated states: none.** No state on either timeframe holds 8% of both books and loses in all
four slices. By the rule of section 3 there is no replay and nothing ships.
What the tables do say (in-book description, not a decision):
- **M15, after the daily-trend veto, has no losing pocket left in these seven views.** The only
clearly negative cell is trades against the weekly trend in 2019–2023 (−18.3R over 58 trades),
5% of the book and positive since 2024. Trades with the four-hour trend earn +0.18R to +0.29R
each against +0.09R to +0.11R against it: weaker, still positive.
- **M5 earns its R where everything already agrees.** With the weekly trend +0.16R to +0.21R per
trade, against it +0.03R to +0.04R; with the four-hour trend +0.19R to +0.24R, against it
+0.01R to +0.04R; when the 60-day move runs against the trade, 0.00R (about 28% of the book).
Those against-the-tide M5 trades are flat, not losing, which is the same picture the formal
daily-trend test gave on M5: removing them removes no loss, and it made the worst losing run
deeper. They stay.
- **M5's profit sits near the 20-day extreme in the trade's direction** (+0.23R to +0.30R per
trade against +0.04R to +0.09R elsewhere). This repeats the 2026-08-04 finding that the best
trades are the stretched runs: a "do not chase" filter would cut the part of the book that pays.
- **Volume and news timing separate nothing reliably.** A volume surge is the best state in
2019–2023 on M15 and the middle state since; trades 30 to 120 minutes after a major US release
look good on both timeframes (+0.38R each) on 75 and 87 trades, and trades in the half hour
after a release are slightly negative on M5 (−3.8R over 46 trades, 5% of the book, below the
8% bar). Neither is a rule.
So the daily chart was the one piece of missing context that paid, on M15. Weekly and four-hour
context add nothing on top of it there, and nothing on M5 that survives the rule.
@@ -0,0 +1,639 @@
# Daily-trend veto and wide-stop cap — pre-registered campaign, 2026-09-30
Owner request (2026-09-30): improve the direction call and the trade setup size; anything may
change if the evidence supports it. This file holds the rules first, then the results. Sections 1
to 4 were written before any formal run. Section 2 records what was already looked at while
planning, so the reader knows which numbers are a confirmation and which are a first look.
## 1. What is tested
**Candidate T — daily-trend veto.** The engine sees at most the H1 chart (M5 confirms on M15 and
H1, M15 on M5 and H1). None of the 65 decision-time features of the earlier campaigns reads the
daily chart. The candidate drops a finished trade setup that goes against a clear daily trend.
Definition (fixed; `signal_aggregator.daily_trend_series`):
- daily candles from the provider (UTC-aligned, label = 00:00 UTC of the day), closed bars only:
a bar labelled D is used once D + 1 day ≤ the close of the primary bar;
- bars labelled Sunday are dropped first (Oanda's 1 to 2 hour reopening stub);
- `ema = Close.ewm(span=50, adjust=False).mean()`, `change = Close / Close.shift(60) − 1`;
- state +1 when `Close > ema` and `change > 0`; −1 when `Close < ema` and `change < 0`; else 0;
- after every setup archetype has run (gold ATR, swing, pullback, chart pattern) a setup against a
non-zero state is removed: grade D, no setup, warning `DAILY TREND FILTER: …`.
Switch `xau_strategy.daily_trend_veto`: `off` (default, books byte-identical), `both`, `buy`
(only buys against a falling daily trend are removed), `sell`.
**Candidate C — wide-stop cap.** `xau_strategy.wide_stop_cap_atr = K` caps the stop distance of
the swing (GENERIC) and chart-pattern setups at `max(K × ATR, the existing floor)`, applied after
the min-RR and spread gates so the set of trades cannot change. K = 6, with 5, 7 and 8 declared
as neighbours. M15 only (the exit-only screen was negative on M5).
## 2. What was already seen (planning phase, 2026-09-30, read-only, in memory)
Prototype: K1 and K2 of the CUDA twin, then a host-side mask on the final candidates before the
5-bar cooldown, then the shared resolver. The control rows of the prototype reproduce the pinned
books (M15 W3 1105 / +195.79R, M5 W3 870 / +153.64R, H1 W3 253 / +70.02R).
| Book ($0.65) | Control | Veto, both directions |
|---|---|---|
| M15 W3 2023-09-05 → 2026-09-05 | 1105 / +195.8R / PF 1.49 / DD 20.9 | 824 / +205.1R / PF 1.73 / DD 12.4 |
| M15 LONG 2019-01-01 → 2026-08-07 | 2091 / +192.5R / PF 1.24 / DD 64.8 | 1473 / +223.3R / PF 1.42 / DD 41.5 |
| M15 rising window 2024-06-01 → 2025-03-30 | 313 / +68.5R / PF 1.67 | 258 / +63.8R / PF 1.80 |
| M5 W3 | 870 / +153.6R / PF 1.50 / DD 10.1 | 668 / +159.1R / PF 1.70 / DD 15.2 |
| M5 W3, buy side only | | 741 / +156.5R / PF 1.60 / DD 9.6 |
| M5 LONG | 1052 / +128.1R / PF 1.34 / DD 18.4 | 790 / +127.5R / PF 1.45 / DD 25.2 |
| H1 W3 | 253 / +70.0R / PF 1.98 / DD 4.4 | 231 / +60.3R / PF 1.90 / DD 4.4 |
Also seen: neighbouring definitions on M15 (60-day change alone +188.3R W3 / +228.1R LONG;
90-day +196.1 / +228.8; 120-day +199.4 / +233.2; 100-day average +201.2 / +227.3); the share of
M15 W3 trades reaching +1R before −1R (58.6% → 61.4%); the live September 2026 setups with the
veto applied after the fact (cooldown view −11.6R → −6.0R to −8.8R); an exit-only re-simulation
of 25 stop and exit variants (cap 6 × ATR on M15 W3 +14.0R, LONG +11.8R; cap 5 negative on LONG);
two variants dropped at this stage and NOT tested formally: lifting the SELL gates when the daily
trend is down (worse on LONG on both timeframes), and removing the existing M15 short blocks
under the veto (costs 4 to 22R).
So the windows W3, W2, the rising window and LONG are a CONFIRMATION through the real code path.
First looks are: the HOLDOUT window through the backtester, the $0.30 numbers on LONG, and every
number produced by the shipped code rather than the prototype.
## 3. Harness
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf <tf> --start <s>
--end <e> --spread <0.65|0.30> [--set xau_strategy.daily_trend_veto=<mode>]
[--set xau_strategy.wide_stop_cap_atr=<K>] --export-csv <out>`; factory configuration otherwise.
Twin numbers count only with the paired `backtest/gpu_backtest.py` MD5 (same flags plus
`--workers 12`). Neighbouring trend definitions are not shipped code: they run on the research
harness (the prototype), after the harness book for the shipped definition is shown identical to
the twin's.
Windows: **W3** 2023-09-05 → 2026-09-05 (halves at 2025-03-05); **W2** 2024-09-05 → 2026-09-05
(halves at 2025-09-05); **RISING** 2024-06-01 → 2025-03-30 (the old out-of-sample window; a
rising market throughout); **LONG** 2019-01-01 → 2026-08-07; **HOLDOUT** 2026-09-05 → 2026-10-01
(fetched for this campaign, never seen by any tuning or by the prototype).
## 4. Acceptance rules (written before the runs)
**Rule T (risk-first; owner decision 2026-09-30).** Candidate T on a timeframe PASSES only if,
against the same-harness control:
- (a) W3 total R ≥ 97% of the control's at $0.65;
- (b) W3 profit factor ≥ control + 0.10;
- (c) W3 max drawdown ≤ the control's;
- (d) LONG: total R ≥ control, max drawdown ≤ control, and 2021 + 2022 combined R better;
- (e) HOLDOUT total R ≥ control − 0.5R;
- (f) clauses (a) to (c) also hold at $0.30;
- (g) at least 3 of the 4 neighbouring definitions (60-day change alone, 90-day, 120-day,
100-day average alone) satisfy (a) and (c): a plateau, not a spike;
- (h) at least 20 trades differ on W3.
Reported, NOT clauses (the owner accepted this cost on 2026-09-30): the W2 halves and the RISING
window, where the veto is expected to give up some R. The 2026-09-26 template (every half and the
old out-of-sample window must not lose) would reject this candidate by construction; that is the
difference between the two rules and it is deliberate.
Decision per timeframe: `both` if it passes; else `buy` if that passes; else no veto (the daily
trend is then shown on the alert as information only). H1 is run for the record.
**Rule C (standard).** K = 6 on M15 against the post-T control PASSES only if: W3 ΔR ≥ +5R; both
W3 halves ≥ control − 1R; PF ≥ control − 0.02; W3 max drawdown ≤ control + 10%; LONG total R ≥
control; the same sign of ΔR at $0.30; at least 20 trades differ; and the neighbours K = 5 and
K = 7 are not negative on W3 or on LONG.
No rescue for either rule: no second definition, no other K, no window shopping after the result.
## 5. Results
All runs 2026-09-30 on the dev box, branch `sql-integration`, working tree = HEAD `0c56b4e` plus
this campaign's code. Every twin log reads
`Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` and
`GPU cache fallbacks: 0 (no per-bar recompute path in cudaBasedBacktesting)`; every reference log
reads the same `Backend:` line and `GPU cache fallbacks: 0`. 86 formal twin books (5 windows × 2
costs × the variants below, plus the fixed-1R-target books). R is per trade in units of the
initial risk; "DD" is the largest peak-to-trough fall of the running total, in R.
### 5.1 Proofs before any formal number
- **Baseline.** Before any code change the five reference MD5s of CLAUDE.md §9 were reproduced on
both scripts (ten books).
- **Default-off neutrality.** With the new code and both knobs at their defaults the same ten
books are byte-identical to the pins (`e599c8a8…` M5 W2, `5b65ecf6…` M15 W2, `09b8a8e7…` M15
OOS, `a44fd19b…` H1 W3, `f3587bae…` H4 W3).
- **Twin = reference on every new path** (`--spread 0.65`, W3 2023-09-05 → 2026-09-05, equal MD5
from `backtest/gpu_backtest.py --workers 12` and `backtest/cudaBasedBacktesting.py`):
| Flags | Trades / R / DD | MD5 (both scripts) |
|---|---|---|
| M15 `--set xau_strategy.daily_trend_veto=both` | 824 / +205.12 / 12.39 | `ad3b0dbda0bf77b659caea1c53be6a50` |
| M15 `…=buy` | 999 / +188.21 / 20.87 | `6e0f3b742d9690f78c97319d1b1c8a43` |
| M15 `…=sell` | 930 / +212.70 / 13.72 | `31aa5762964d846f15d111cd4a718adc` |
| M15 `--set xau_strategy.wide_stop_cap_atr=6` | 1105 / +209.78 / 19.32 | `4ec0e4450d5dab143a4ed37e80545626` |
| M5 `--set xau_strategy.daily_trend_veto=both` | 668 / +159.07 / 15.20 | `39cc0f198d38b532720dce4f98d99452` |
| M5 `--set xau_strategy.wide_stop_cap_atr=6` | 870 / +141.57 / 13.95 | `4da0997256b272ffcd11a90bf9658eef` |
The M15 `both` book equals the planning prototype trade for trade (824 / +205.1R / PF 1.73 /
DD 12.4), so the shipped definition is the one that was screened.
- **Suite.** 595 passed / 0 failed after Stage 1 (Module 39 new, Module 37 extended).
### 5.2 Candidate T — every window, both costs
Command per row: `venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf
<15min|5min> --start <s> --end <e> --spread <0.65|0.30> [--set xau_strategy.daily_trend_veto=<mode>]
--export-csv <out>`. "control" = no `--set`.
**M15**
| Window | Cost | Variant | Trades | Total R | Win rate | PF | Max DD | First half | Second half | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|
| W3 | $0.65 | control | 1105 | +195.79 | 61.1% | 1.49 | 20.9 | +76.87 | +118.92 | 70195fb0 |
| W3 | $0.65 | both | 824 | +205.12 | 63.1% | 1.73 | 12.4 | +76.72 | +128.40 | ad3b0dbd |
| W3 | $0.65 | buy | 999 | +188.21 | 61.3% | 1.52 | 20.9 | +60.40 | +127.81 | 6e0f3b74 |
| W3 | $0.65 | sell | 930 | +212.70 | 62.7% | 1.66 | 13.7 | +93.19 | +119.52 | 31aa5762 |
| W3 | $0.30 | control | 1105 | +216.53 | 61.4% | 1.55 | 19.6 | +88.86 | +127.67 | ce074a1b |
| W3 | $0.30 | both | 824 | +221.18 | 63.6% | 1.81 | 11.4 | +86.41 | +134.77 | 96a87454 |
| W3 | $0.30 | buy | 999 | +207.23 | 61.7% | 1.59 | 19.6 | +71.83 | +135.41 | 2ada14e4 |
| W3 | $0.30 | sell | 930 | +230.48 | 63.1% | 1.73 | 12.8 | +103.45 | +127.03 | 49665c41 |
| W2 | $0.65 | control | 813 | +154.15 | 62.2% | 1.53 | 15.6 | +70.10 | +84.04 | 5b65ecf6 |
| W2 | $0.65 | both | 597 | +155.89 | 64.3% | 1.78 | 11.4 | +61.83 | +94.06 | dfc647ee |
| W2 | $0.65 | buy | 718 | +154.67 | 63.0% | 1.62 | 14.4 | +65.03 | +89.65 | f23bfda4 |
| W2 | $0.65 | sell | 692 | +155.36 | 63.3% | 1.65 | 13.7 | +66.91 | +88.46 | 1a071ac7 |
| W2 | $0.30 | control | 813 | +167.53 | 62.6% | 1.59 | 14.5 | +78.61 | +88.91 | 15486607 |
| W2 | $0.30 | both | 597 | +166.22 | 64.8% | 1.85 | 11.0 | +68.82 | +97.39 | d2f678bf |
| W2 | $0.30 | buy | 718 | +166.64 | 63.4% | 1.68 | 14.0 | +73.12 | +93.52 | f3abc1e8 |
| W2 | $0.30 | sell | 692 | +167.10 | 63.7% | 1.71 | 12.8 | +74.31 | +92.79 | d7d038fd |
| RISING | $0.65 | control | 313 | +68.50 | 63.6% | 1.67 | 12.4 | +18.87 | +49.63 | 09b8a8e7 |
| RISING | $0.65 | both | 258 | +63.78 | 64.3% | 1.80 | 9.1 | +23.97 | +39.81 | e87b6de1 |
| RISING | $0.65 | buy | 303 | +60.15 | 63.0% | 1.60 | 12.4 | +18.87 | +41.28 | f001a0a6 |
| RISING | $0.65 | sell | 268 | +72.13 | 64.9% | 1.89 | 9.1 | +23.97 | +48.16 | cbe4cb24 |
| RISING | $0.30 | control | 313 | +76.21 | 64.5% | 1.77 | 11.1 | +22.92 | +53.29 | 5fc34cef |
| RISING | $0.30 | both | 258 | +70.24 | 65.5% | 1.91 | 7.9 | +27.31 | +42.92 | 5de32f51 |
| RISING | $0.30 | buy | 303 | +67.59 | 64.0% | 1.70 | 11.1 | +22.92 | +44.67 | 69b41026 |
| RISING | $0.30 | sell | 268 | +78.86 | 66.0% | 2.00 | 7.9 | +27.31 | +51.54 | 3631466e |
| LONG | $0.65 | control | 2091 | +192.53 | 57.8% | 1.24 | 64.8 | | | c2228398 |
| LONG | $0.65 | both | 1473 | +223.33 | 59.7% | 1.42 | 41.5 | | | aa5253aa |
| LONG | $0.65 | buy | 1762 | +214.62 | 58.9% | 1.33 | 34.5 | | | 952d64d8 |
| LONG | $0.65 | sell | 1802 | +201.23 | 58.3% | 1.30 | 76.4 | | | d3965ab8 |
| LONG | $0.30 | control | 2091 | +240.00 | 58.8% | 1.31 | 49.6 | | | 0c58b300 |
| LONG | $0.30 | both | 1473 | +257.25 | 60.8% | 1.50 | 37.2 | | | 1835b341 |
| LONG | $0.30 | buy | 1762 | +254.33 | 59.9% | 1.40 | 29.5 | | | a2b519bd |
| LONG | $0.30 | sell | 1802 | +242.92 | 59.4% | 1.37 | 62.3 | | | f6edb469 |
| HOLDOUT | $0.65 | control | 24 | −6.63 | 50.0% | 0.42 | 7.0 | | | c42470a0 |
| HOLDOUT | $0.65 | both | 18 | −4.62 | 50.0% | 0.50 | 4.8 | | | fa50bcd0 |
| HOLDOUT | $0.65 | buy | 22 | −6.81 | 45.5% | 0.41 | 7.0 | | | 548a4bf4 |
| HOLDOUT | $0.65 | sell | 20 | −4.44 | 55.0% | 0.52 | 4.8 | | | 1459ef5f |
| HOLDOUT | $0.30 | control | 24 | −6.29 | 50.0% | 0.44 | 6.7 | | | d7fbee9d |
| HOLDOUT | $0.30 | both | 18 | −4.35 | 50.0% | 0.52 | 4.5 | | | 2179c08c |
| HOLDOUT | $0.30 | buy | 22 | −6.49 | 45.5% | 0.43 | 6.7 | | | 0b90faeb |
| HOLDOUT | $0.30 | sell | 20 | −4.15 | 55.0% | 0.54 | 4.5 | | | 70932b7e |
**M5**
| Window | Cost | Variant | Trades | Total R | Win rate | PF | Max DD | First half | Second half | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|
| W3 | $0.65 | control | 870 | +153.64 | 53.3% | 1.50 | 10.1 | +16.73 | +136.91 | d20ec8a3 |
| W3 | $0.65 | both | 668 | +159.07 | 53.9% | 1.70 | 15.2 | +14.01 | +145.06 | 39cc0f19 |
| W3 | $0.65 | buy | 741 | +156.53 | 53.4% | 1.60 | 9.6 | +15.68 | +140.85 | 631f25ac |
| W3 | $0.30 | control | 870 | +172.40 | 54.3% | 1.57 | 8.9 | +21.08 | +151.32 | 5acafcba |
| W3 | $0.30 | both | 668 | +174.43 | 54.8% | 1.79 | 14.3 | +18.09 | +156.34 | 34f2e282 |
| W3 | $0.30 | buy | 741 | +172.72 | 54.3% | 1.69 | 8.9 | +19.87 | +152.86 | c8bdc9e2 |
| W2 | $0.65 | control | 781 | +146.07 | 52.9% | 1.52 | 10.1 | +26.42 | +119.65 | e599c8a8 |
| W2 | $0.65 | both | 585 | +153.82 | 53.7% | 1.78 | 15.2 | +33.16 | +120.66 | b9fa356c |
| W2 | $0.65 | buy | 656 | +151.09 | 53.0% | 1.66 | 9.6 | +27.42 | +123.66 | 30e243b0 |
| W2 | $0.30 | control | 781 | +162.09 | 53.9% | 1.60 | 8.9 | +31.71 | +130.37 | 7ab48db9 |
| W2 | $0.30 | both | 585 | +166.59 | 54.7% | 1.87 | 14.3 | +38.05 | +128.54 | 3427b933 |
| W2 | $0.30 | buy | 656 | +164.66 | 54.0% | 1.74 | 8.9 | +32.58 | +132.08 | 7dd9e2a2 |
| RISING | $0.65 | control | 86 | +9.47 | 53.5% | 1.33 | 8.2 | +0.01 | +9.46 | 65c46e62 |
| RISING | $0.65 | both | 81 | +9.07 | 54.3% | 1.34 | 7.5 | +0.67 | +8.40 | e20fdef2 |
| RISING | $0.65 | buy | 85 | +10.55 | 54.1% | 1.39 | 7.2 | +0.01 | +10.54 | de24b2e8 |
| RISING | $0.30 | control | 86 | +12.42 | 54.7% | 1.46 | 6.7 | +1.65 | +10.77 | 5c154360 |
| RISING | $0.30 | both | 81 | +11.91 | 55.6% | 1.47 | 6.1 | +2.28 | +9.63 | 3b1a8f9e |
| RISING | $0.30 | buy | 85 | +13.46 | 55.3% | 1.52 | 5.7 | +1.65 | +11.81 | 21ffab88 |
| LONG | $0.65 | control | 1052 | +128.10 | 50.5% | 1.34 | 18.4 | | | 3f0c091c |
| LONG | $0.65 | both | 790 | +127.47 | 50.4% | 1.45 | 25.2 | | | 9872c068 |
| LONG | $0.65 | buy | 871 | +123.37 | 49.9% | 1.39 | 27.1 | | | 613dcf0e |
| LONG | $0.30 | control | 1052 | +153.02 | 51.7% | 1.41 | 13.0 | | | 6708d8cc |
| LONG | $0.30 | both | 790 | +147.13 | 51.5% | 1.54 | 19.6 | | | 201d968f |
| LONG | $0.30 | buy | 871 | +144.02 | 51.1% | 1.47 | 22.3 | | | d9cba459 |
| HOLDOUT | $0.65 | control | 33 | −2.45 | 51.5% | 0.82 | 6.1 | | | 351f7149 |
| HOLDOUT | $0.65 | both | 26 | −2.04 | 50.0% | 0.81 | 5.4 | | | b9c8cb82 |
| HOLDOUT | $0.65 | buy | 33 | −2.45 | 51.5% | 0.82 | 6.1 | | | 351f7149 |
| HOLDOUT | $0.30 | control | 33 | −1.85 | 51.5% | 0.86 | 5.6 | | | 5a6aea9c |
| HOLDOUT | $0.30 | both | 26 | −1.56 | 50.0% | 0.85 | 5.0 | | | ddd29068 |
| HOLDOUT | $0.30 | buy | 33 | −1.85 | 51.5% | 0.86 | 5.6 | | | 5a6aea9c |
**H1, for the record** ($0.65): W3 control 253 / +70.02R / PF 1.98 / DD 4.4 (`a44fd19b`), `both`
231 / +60.28R / PF 1.90 / DD 4.4 (`cdac563b`); HOLDOUT 3 / −3.04R in both (no trade removed).
Fails (a) (60.28 < 67.92) and (b). H4 was not run: it confirms on the daily chart already.
### 5.3 Per calendar year ($0.65), total R / trades
M15, LONG 2019-01-01 → 2026-08-07:
| Variant | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
|---|---|---|---|---|---|---|---|---|
| control | +9.0 / 117 | +48.8 / 283 | −25.6 / 210 | −24.6 / 267 | +11.7 / 211 | +53.3 / 343 | +81.2 / 425 | +38.7 / 235 |
| both | +14.0 / 88 | +45.1 / 207 | −27.3 / 121 | +3.2 / 149 | +1.8 / 163 | +53.5 / 275 | +92.5 / 341 | +40.6 / 129 |
| buy | +7.6 / 111 | +44.2 / 253 | −21.4 / 135 | +7.4 / 166 | +1.6 / 189 | +46.4 / 335 | +83.1 / 416 | +45.9 / 157 |
| sell | +15.4 / 94 | +49.7 / 237 | −31.5 / 196 | −28.8 / 250 | +11.9 / 185 | +60.4 / 283 | +90.6 / 350 | +33.4 / 207 |
M5, LONG:
| Variant | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
|---|---|---|---|---|---|---|---|---|
| control | −1.0 / 20 | −2.8 / 95 | −1.7 / 32 | −2.2 / 50 | −2.8 / 61 | +1.0 / 102 | +40.5 / 332 | +97.0 / 360 |
| both | +0.2 / 17 | −6.9 / 76 | −3.4 / 14 | −4.7 / 28 | −3.5 / 54 | +1.5 / 96 | +43.2 / 297 | +101.0 / 208 |
| buy | +0.2 / 17 | −6.7 / 83 | −3.2 / 16 | −6.0 / 30 | −4.6 / 56 | +2.1 / 101 | +40.5 / 329 | +101.0 / 239 |
M15, W3 per calendar year (total R / trades / max DD inside the year):
| Variant | 2023 (from 09-05) | 2024 | 2025 | 2026 (to 09-05) |
|---|---|---|---|---|
| control | +10.8 / 66 / 9.0 | +53.3 / 343 / 16.3 | +81.2 / 425 / 15.6 | +50.5 / 271 / 10.3 |
| both | +9.3 / 47 / 6.9 | +53.5 / 275 / 11.0 | +92.5 / 341 / 11.4 | +49.9 / 161 / 8.3 |
M15, R per month from 2025-09 (W3):
| Variant | 2025-09 | 10 | 11 | 12 | 2026-01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 (to 09-05) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| control | +14.7 | −1.4 | +3.7 | +19.4 | +29.1 | −2.2 | +9.5 | +4.4 | +0.0 | +1.7 | −4.6 | +12.4 | +0.2 |
| both | +19.4 | +0.6 | +8.4 | +18.7 | +32.4 | −2.4 | +1.5 | +7.2 | −2.4 | +5.1 | +2.3 | +5.9 | +0.2 |
Losing runs on M15: longest run of losing trades 9 → 8 (W3 and LONG); worst calendar month
−10.1R → −6.1R (W3, 2024-01) and −11.7R → −10.0R (LONG); losing months 11 → 8 of 37 on W3,
39 of 92 → 37 of 91 on LONG.
### 5.4 Rule T, clause by clause
**M15 `both` — PASS**
| Clause | Result | Numbers (candidate vs control) |
|---|---|---|
| (a) W3 R ≥ 97% of control | PASS | +205.12 vs +195.79 (97% = +189.92) |
| (b) W3 PF ≥ control + 0.10 | PASS | 1.734 vs 1.488 |
| (c) W3 max DD ≤ control | PASS | 12.39 vs 20.87 |
| (d) LONG R ≥ control | PASS | +223.33 vs +192.53 |
| (d) LONG max DD ≤ control | PASS | 41.54 vs 64.77 |
| (d) 2021 + 2022 better | PASS | −24.12 vs −50.21 |
| (e) HOLDOUT R ≥ control − 0.5 | PASS | −4.62 (18 trades) vs −6.63 (24) |
| (f) (a) at $0.30 | PASS | +221.18 vs +216.53 (97% = +210.04) |
| (f) (b) at $0.30 | PASS | 1.809 vs 1.551 |
| (f) (c) at $0.30 | PASS | 11.44 vs 19.56 |
| (g) ≥ 3 of 4 neighbours pass (a) and (c) | PASS | 3 of 4 (table 5.5) |
| (h) ≥ 20 trades differ on W3 | PASS | 281 removed, 0 added |
**M15 `buy` — FAIL**: (a) +188.21 < +189.92; (b) 1.523 < 1.588; the same two at $0.30.
**M15 `sell` — FAIL**: (d) LONG max DD 76.39 > 64.77 and 2021 + 2022 −60.29 < −50.21. It has the
best W3 total (+212.70R) and that is exactly the trap the long window was there to catch: in the
falling years the buys against the daily trend are the losers, and `sell` keeps them.
**M5 `both` — FAIL**: (c) W3 DD 15.20 > 10.11 (also at $0.30: 14.30 > 8.88); (d) LONG R
+127.47 < +128.10, LONG DD 25.25 > 18.39, 2021 + 2022 −8.02 < −3.91; (g) 0 of 4.
**M5 `buy` — FAIL**: (d) LONG R +123.37 < +128.10, LONG DD 27.14 > 18.39, 2021 + 2022 −9.13 <
−3.91; (g) 1 of 4.
**H1 `both` — FAIL** (a), (b).
### 5.5 Clause (g): neighbouring definitions (research harness)
The harness (K1 + K2 of the twin with the veto OFF, a host-side mask on the final candidates
before the cooldown, the shared resolver) reproduces the twin's `--set` books byte for byte for
the shipped definition, so it is the same thing as the shipped path for a final veto:
| Book ($0.65) | Harness CSV MD5 | Twin CSV MD5 |
|---|---|---|
| M15 W3 control | `70195fb0e6865b3a46ee8a9aa8844e51` | same |
| M15 W3 `both` | `ad3b0dbda0bf77b659caea1c53be6a50` | same |
| M15 LONG control | `c222839878f791e980b2dbc99fb6dfbd` | same |
| M15 LONG `both` | `aa5253aab206fd21fa6a8d5ba25b75b5` | same |
| M5 W3 control / `both` / `buy` | `d20ec8a3…` / `39cc0f19…` / `631f25ac…` | same |
| M5 LONG control / `both` / `buy` | `3f0c091c…` / `9872c068…` / `613dcf0e…` | same |
M15, veto in both directions, each neighbour replacing the definition of "clear daily trend":
| Definition | W3 trades | W3 R | W3 PF | W3 DD | (a) | (c) | LONG trades | LONG R | LONG PF | LONG DD |
|---|---|---|---|---|---|---|---|---|---|---|
| control | 1105 | +195.79 | 1.49 | 20.87 | (97% = +189.92) | | 2091 | +192.53 | 1.24 | 64.77 |
| **shipped: 50-day average AND 60-day change** | 824 | +205.12 | 1.73 | 12.39 | PASS | PASS | 1473 | +223.33 | 1.42 | 41.54 |
| 60-day change alone | 752 | +188.25 | 1.73 | 11.89 | FAIL | PASS | 1328 | +228.11 | 1.49 | 27.30 |
| 90-day change alone | 782 | +196.14 | 1.72 | 12.46 | PASS | PASS | 1412 | +228.75 | 1.46 | 37.36 |
| 120-day change alone | 781 | +199.36 | 1.75 | 12.26 | PASS | PASS | 1397 | +233.19 | 1.47 | 34.79 |
| 100-day average alone | 796 | +201.23 | 1.74 | 12.26 | PASS | PASS | 1375 | +227.30 | 1.47 | 32.56 |
3 of 4 pass. Every neighbour beats the control on PF and on drawdown in both windows; the one
miss is 1.7R under the 97% line. It is a plateau. (Each neighbour has a lower LONG drawdown than
the shipped definition; switching to one now would be the rescue the rule forbids.)
M5: `both` 0 of 4 (every neighbour keeps the 15.2R drawdown), `buy` 1 of 4 (90-day, 120-day and
100-day average all fall under the 97% line). No plateau on M5.
### 5.6 What the veto removes (M15, $0.65)
Zero trades are added in any window (W3 and LONG, every variant, both timeframes): the setups
inside a vetoed setup's 5-bar cooldown point the same way under the same daily state and are
vetoed with it, so the cooldown displaces nothing.
| Slice | W3 trades | W3 R | W3 mean R | LONG trades | LONG R | LONG mean R |
|---|---|---|---|---|---|---|
| all removed | 281 | −9.32 | −0.033 | 618 | −30.80 | −0.050 |
| BUY against a falling daily trend | 106 | +7.59 | +0.072 | 329 | −22.10 | −0.067 |
| SELL against a rising daily trend | 175 | −16.91 | −0.097 | 289 | −8.70 | −0.030 |
| removed, grade B | 210 | −11.53 | −0.055 | 408 | −25.69 | −0.063 |
| removed, grade A | 46 | +1.28 | +0.028 | 164 | −1.12 | −0.007 |
| removed, grade A+ | 14 | +4.26 | +0.304 | 26 | +1.81 | +0.069 |
| removed, grade C | 11 | −3.34 | −0.304 | 20 | −5.80 | −0.290 |
| kept (the new book) | 824 | +205.12 | +0.249 | 1473 | +223.33 | +0.152 |
| kept BUY | 763 | +194.20 | +0.255 | 1302 | +222.50 | +0.171 |
| kept SELL | 61 | +10.91 | +0.179 | 171 | +0.83 | +0.005 |
Reading: the removed trades are close to break-even after costs (−0.03 to −0.05R each) while the
kept trades earn +0.15 to +0.25R each. The veto does not find big losers; it drops about a quarter
of the trades, the ones with no edge, and with them the long flat-to-down stretches. The two
halves of the veto are carried by different windows: the SELL half by the rising 3-year window,
the BUY half by the falling years of the long window. Only the symmetric rule holds on both.
Share of trades that reach +1R before −1R (same entries, fixed 1R target, `--fixed-rr-tp 1.0`,
$0.65): M15 W3 58.6% → 61.4% (1105 → 824 trades), M15 LONG 55.9% → 58.2%; M5 W3 56.8% → 58.2%,
M5 LONG 55.3% → 56.1%.
Checklist breakdowns, M15 W3 → with the veto (trades / R), $0.65:
| Slice | Control | Veto | Change in R |
|---|---|---|---|
| regime ranging | 140 / +46.75 | 106 / +49.23 | +2.48 |
| regime transitioning | 264 / +47.05 | 199 / +49.47 | +2.42 |
| regime trending_down | 165 / −4.39 | 43 / +9.71 | +14.10 |
| regime trending_up | 536 / +106.38 | 476 / +96.70 | −9.68 |
| volatility extreme | 508 / +103.91 | 361 / +95.86 | −8.05 |
| volatility high | 228 / +4.00 | 135 / +18.32 | +14.32 |
| volatility normal | 202 / +58.96 | 177 / +57.01 | −1.96 |
| volatility low | 167 / +28.92 | 151 / +33.94 | +5.01 |
| BUY | 869 / +201.79 | 763 / +194.20 | −7.59 |
| SELL | 236 / −6.00 | 61 / +10.91 | +16.91 |
| setup GENERIC | 1004 / +184.73 | 757 / +192.15 | +7.42 |
| setup XAU_ATR | 101 / +11.07 | 67 / +12.97 | +1.90 |
| Monday | 188 / +36.54 | 145 / +52.46 | +15.92 |
| Tuesday | 240 / +44.89 | 186 / +41.82 | −3.07 |
| Wednesday | 234 / +12.08 | 165 / +13.36 | +1.28 |
| Thursday | 216 / +63.84 | 157 / +70.36 | +6.51 |
| Friday | 214 / +33.51 | 162 / +21.04 | −12.47 |
| Sunday evening | 13 / +4.94 | 9 / +6.08 | +1.14 |
By entry hour (UTC), change in R on W3 / on LONG: 03 +0.3 / +1.5; 04 −2.1 / −0.1; 05 −0.2 / +2.6;
06 +1.9 / −1.5; 07 +2.9 / +0.9; 08 −5.3 / −0.3; 09 −6.6 / −4.0; 12 −2.9 / +2.7; 14 +11.8 / +12.9;
15 −4.0 / +1.8; 17 −0.5 / −2.4; 18 +11.4 / +11.3; 19 +0.9 / +2.7; 20 +0.8 / +2.9; 22 +0.8 / −0.1.
The weekday and hour changes are not stable in sign between the two windows except hours 14 and 18
(positive in both) and hour 9 (negative in both); none of it is a reason to act.
### 5.7 Verdict T
- **M15: `both` PASSES every clause and SHIPS** (`pair_signals/xauusd.py`, M15 override 7).
- **M5: no veto.** `both` deepens the worst losing run (10.1R → 15.2R) and loses the long window;
`buy` passes the 3-year window and fails the long one, with no plateau. The M5 book earns
almost all of its R in 2025–2026 and the daily trend adds nothing reliable to it.
- **H1 / H4: no veto.** H1 loses 10R of 70R; H4 already confirms on the daily chart.
- On the timeframes without the veto the daily trend is shown on the alert as information only.
What the owner gives up, as agreed: in the purely rising stretch the veto costs R. W2 first half
(2024-09 → 2025-09) +61.83R vs +70.10R; the RISING window +63.78R vs +68.50R; W2 at $0.30
+166.22R vs +167.53R. What it buys: the worst losing run on the 3-year window falls from 20.9R to
12.4R, on the 7.6-year window from 64.8R to 41.5R, 2022 turns from −24.6R to +3.2R, and the three
weeks after the test window (HOLDOUT, never seen by any tuning) lose 4.6R instead of 6.6R. 2021
does not improve (−27.3R vs −25.6R) and 2023 is worse (+1.8R vs +11.7R): in a year that chops
sideways the daily trend flips late and the veto cannot help.
### 5.8 Candidate C — wide-stop cap on M15 (post-T control)
Control = the factory configuration after the M15 flip (daily-trend veto on). Command per row:
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 15min --start <s>
--end <e> --spread <0.65|0.30> [--set xau_strategy.wide_stop_cap_atr=<K>] --export-csv <out>`.
Paired proof on this control: W3, K = 6, `gpu_backtest.py --workers 12` and the twin both give
`f94f1429fb4c9172427ea34c2f4d92a1` (824 / +216.87R / DD 11.37).
| Window | Cost | K | Trades | Total R | Win rate | PF | Max DD | First half | Second half | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|
| W3 | $0.65 | off | 824 | +205.12 | 63.1% | 1.73 | 12.4 | +76.72 | +128.40 | ad3b0dbd |
| W3 | $0.65 | 5 | 824 | +211.26 | 61.4% | 1.74 | 15.0 | +81.77 | +129.48 | 055d72a4 |
| W3 | $0.65 | **6** | 824 | +216.87 | 62.9% | 1.78 | 11.4 | +80.67 | +136.20 | f94f1429 |
| W3 | $0.65 | 7 | 824 | +213.15 | 63.1% | 1.76 | 11.8 | +79.08 | +134.07 | 5ea4ab4b |
| W3 | $0.65 | 8 | 824 | +208.01 | 62.9% | 1.74 | 12.3 | +77.18 | +130.84 | 372ac298 |
| W3 | $0.30 | off | 824 | +221.18 | 63.6% | 1.81 | 11.4 | +86.41 | +134.77 | 96a87454 |
| W3 | $0.30 | 5 | 824 | +228.46 | 62.1% | 1.82 | 14.0 | +92.27 | +136.18 | 7453b5f7 |
| W3 | $0.30 | **6** | 824 | +233.37 | 63.6% | 1.86 | 11.0 | +90.67 | +142.70 | bf7e4157 |
| W3 | $0.30 | 7 | 824 | +229.37 | 63.7% | 1.84 | 11.0 | +88.88 | +140.48 | e44cb5b5 |
| W3 | $0.30 | 8 | 824 | +224.12 | 63.5% | 1.82 | 11.3 | +86.91 | +137.22 | c214455b |
| W2 | $0.65 | off | 597 | +155.89 | 64.3% | 1.78 | 11.4 | +61.83 | +94.06 | dfc647ee |
| W2 | $0.65 | 5 | 597 | +156.59 | 63.3% | 1.79 | 11.4 | +62.63 | +93.96 | 5e5c0207 |
| W2 | $0.65 | **6** | 597 | +160.79 | 64.0% | 1.81 | 11.4 | +61.31 | +99.48 | 3614c98a |
| W2 | $0.65 | 7 | 597 | +158.70 | 64.3% | 1.79 | 11.4 | +60.19 | +98.51 | afa7eb01 |
| W2 | $0.65 | 8 | 597 | +156.58 | 64.2% | 1.78 | 11.4 | +60.54 | +96.04 | ba29cc41 |
| RISING | $0.65 | off | 258 | +63.78 | 64.3% | 1.80 | 9.1 | +23.97 | +39.81 | e87b6de1 |
| RISING | $0.65 | 5 | 258 | +57.68 | 61.6% | 1.71 | 8.0 | +22.85 | +34.83 | 295731a3 |
| RISING | $0.65 | **6** | 258 | +58.03 | 64.0% | 1.72 | 8.6 | +23.84 | +34.19 | 8dba71d7 |
| RISING | $0.65 | 7 | 258 | +62.18 | 64.3% | 1.78 | 8.8 | +26.44 | +35.74 | c461380f |
| RISING | $0.65 | 8 | 258 | +62.12 | 63.6% | 1.77 | 9.1 | +24.06 | +38.07 | fce21577 |
| LONG | $0.65 | off | 1473 | +223.33 | 59.7% | 1.42 | 41.5 | | | aa5253aa |
| LONG | $0.65 | 5 | 1473 | +204.76 | 58.4% | 1.38 | 53.1 | | | b365f28f |
| LONG | $0.65 | **6** | 1473 | +233.56 | 59.7% | 1.44 | 47.2 | | | 4aa12918 |
| LONG | $0.65 | 7 | 1473 | +228.83 | 59.8% | 1.43 | 46.5 | | | 6e91e0dd |
| LONG | $0.65 | 8 | 1473 | +223.79 | 59.7% | 1.42 | 43.2 | | | 36267a43 |
| LONG | $0.30 | off | 1473 | +257.25 | 60.8% | 1.50 | 37.2 | | | 1835b341 |
| LONG | $0.30 | 5 | 1473 | +241.64 | 59.8% | 1.46 | 44.9 | | | 6a346020 |
| LONG | $0.30 | **6** | 1473 | +268.63 | 60.8% | 1.52 | 41.6 | | | 526ffa4a |
| LONG | $0.30 | 7 | 1473 | +263.16 | 60.9% | 1.51 | 42.1 | | | 3f42a60e |
| LONG | $0.30 | 8 | 1473 | +257.84 | 60.8% | 1.50 | 38.9 | | | ffbba3e4 |
| HOLDOUT | $0.65 | off, 6, 7, 8 | 18 | −4.62 | 50.0% | 0.50 | 4.8 | | | fa50bcd0 |
| HOLDOUT | $0.65 | 5 | 18 | −4.62 | 50.0% | 0.50 | 4.8 | | | 3ff32a0d |
(W2 and RISING at $0.30 and the HOLDOUT at $0.30 are in the same direction as the $0.65 rows:
W2 +171.35 vs +166.22 for K = 6; RISING +64.66 vs +70.24; HOLDOUT unchanged.)
LONG per calendar year ($0.65), total R (trades are identical: 88 / 207 / 121 / 149 / 163 / 275 /
341 / 129):
| K | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
|---|---|---|---|---|---|---|---|---|
| off | +14.0 | +45.1 | −27.3 | +3.2 | +1.8 | +53.5 | +92.5 | +40.6 |
| 5 | +19.0 | +27.9 | −32.4 | −1.0 | +2.5 | +63.1 | +89.2 | +36.6 |
| 6 | +19.5 | +46.7 | −31.8 | +1.8 | +4.7 | +59.8 | +88.1 | +44.8 |
| 7 | +16.5 | +45.4 | −32.2 | +4.3 | +1.8 | +59.2 | +91.1 | +42.7 |
| 8 | +14.5 | +43.9 | −29.0 | +3.4 | +2.9 | +54.4 | +92.3 | +41.4 |
Trades the cap touches (the trade set is identical by construction; only stops, and through them
results, move). All of them are swing (GENERIC) setups; the gold ATR setup is never capped:
| Window | K | Stops changed | R of those trades: control → capped |
|---|---|---|---|
| W3 | 5 / 6 / 7 / 8 | 281 / 168 / 89 / 44 | +66.4 → +72.5 / +49.2 → +60.9 / +25.6 → +33.6 / +12.2 → +15.1 |
| LONG | 5 / 6 / 7 / 8 | 597 / 350 / 170 / 83 | +97.5 → +78.9 / +56.7 → +66.9 / +37.0 → +42.5 / +12.5 → +13.0 |
| RISING | 5 / 6 / 7 / 8 | 97 / 58 / 25 / 12 | +28.1 → +22.0 / +21.0 → +15.3 / +13.1 → +11.5 / +5.7 → +4.0 |
| HOLDOUT | 5 / 6 / 7 / 8 | 3 / 0 / 0 / 0 | −3.03 → −3.04 |
**Rule C, K = 6**
| Clause | Result | Numbers |
|---|---|---|
| W3 change in R ≥ +5R | PASS | +11.75 (+216.87 vs +205.12) |
| W3 first half ≥ control − 1R | PASS | +80.67 vs +76.72 |
| W3 second half ≥ control − 1R | PASS | +136.20 vs +128.40 |
| W3 PF ≥ control − 0.02 | PASS | 1.777 vs 1.734 |
| W3 max DD ≤ control + 10% | PASS | 11.37 vs 12.39 |
| LONG total R ≥ control | PASS | +233.56 vs +223.33 (but max DD 47.19 vs 41.54) |
| same sign at $0.30 | PASS | +12.19 (+233.37 vs +221.18) |
| ≥ 20 trades differ | PASS | 168 stops and results changed |
| neighbour K = 5 not negative on W3 | PASS | +6.14 |
| **neighbour K = 5 not negative on LONG** | **FAIL** | **−18.57** |
| neighbour K = 7 not negative on W3 | PASS | +8.04 |
| neighbour K = 7 not negative on LONG | PASS | +5.50 |
**Verdict C: FAIL. Nothing ships; `wide_stop_cap_atr` stays 0.0 (research knob only).** K = 6 is a
narrow peak, not a plateau: one step tighter loses 18.6R over the long window (2020 alone −17R),
one step wider keeps half the gain, two steps wider keeps none. The cap also costs R in the rising
window at every K (−1.6R to −6.1R), deepens the long-window drawdown at every K (41.5R → 43.2R to
53.1R) and changes nothing in the unseen three weeks. This agrees with the planning-phase screen
of 25 stop and exit variants: the stop geometry is at a plateau, and tightening it is the one
direction that reliably loses. No second K is tried (the rule's no-rescue clause).
### 5.9 What shipped, and the re-pinned books
Shipped: `pair_signals/xauusd.py` M15 override 7, `config.xau_strategy.daily_trend_veto = "both"`
on M15 only. Not shipped: any veto on M5 / H1 / H4; the wide-stop cap (knob kept at 0.0).
Reference books after the flip, factory configuration, `--spread 0.65`, regenerated with BOTH
scripts (`gpu_backtest.py --workers 12` and `cudaBasedBacktesting.py`), `Backend: CuPy GPU
(NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, `GPU cache fallbacks: 0`:
| Book | Window | Trades / R / PF / DD | MD5 (reference = twin) | |
|---|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 / 10.1 | `e599c8a8652f95b7fcdf5a7ef2b564f5` | unchanged |
| M15 W2 | 2024-09-05 → 2026-09-05 | 597 / +155.89 / 1.78 / 11.4 | `dfc647ee02afd179dd9e031f344ed380` | re-pinned (was `5b65ecf6…`, 813 / +154.15 / 1.53) |
| M15 OOS | 2024-06-01 → 2025-03-30 | 258 / +63.78 / 1.80 / 9.1 | `e87b6de12f90f3b5a5d41a953ee9ff7a` | re-pinned (was `09b8a8e7…`, 313 / +68.50 / 1.67) |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.02 / 1.98 / 4.4 | `a44fd19b4067f46e5bd98d50cc3eff92` | unchanged |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `f3587bae3c5dc3eda468ea3588b17dcb` | unchanged |
| M15 W3 | 2023-09-05 → 2026-09-05 | 824 / +205.12 / 1.73 / 12.4 | `ad3b0dbda0bf77b659caea1c53be6a50` | was `70195fb0…`, 1105 / +195.79 / 1.49 |
The factory books are byte-identical to the `--set xau_strategy.daily_trend_veto=both` books of
section 5.2, as they must be.
### 5.10 A fetch bug found on the way (fixed, not a signal change)
The first HOLDOUT and RISING runs with the daily frame failed with `OandaDataError: XAU/USD 1d:
3 consecutive failures fetching window 2020-08-01–2020-08-01 … (Empty candle response)`. Since
2026-09-13 `fetch_range` retries a failing page and aborts after three tries, while the Oanda
provider raised on an EMPTY date window. A padded window that starts on a closed day (the daily
pad is 1,400 days = exactly 200 weeks, so a window starting on a Saturday pads to a Saturday)
begins with a page nothing traded in, and the whole fetch aborted. Fix in `providers/oanda.py`:
a DATE-WINDOW request with no candles returns an empty frame (a `count` request, the live path,
still raises); two Module 15 tests. Books are unaffected (every pinned window was already cached;
the ten MD5s reproduce). New cache files fetched for this campaign: the daily frames
`oanda_XAU_USD_1d_20201105_20260905`, `_20150303_20260807`, `_20200801_20250330`,
`_20221105_20261001`, and the HOLDOUT window `5min` / `15min` / `1h` `_20260705_20261001` and
`4h_20260114_20261001`. The two interior gaps the HOLDOUT fetch logged are market closures
(Labor Day 2026-09-07, Good Friday 2026-04-02), not dropped pages.
### 5.11 Setup size: what the evidence supports, and the display lines that shipped
"Trade setup size" was tested two ways.
**The stop itself.** Candidate C (section 5.8) fails its rule, and the planning-phase screen of
25 stop and exit variants found nothing better than the shipped geometry (a flat 2 × ATR stop on
M15 turns +196R into +3R). The stops stay as they are.
**How much is at risk at once.** If every alert is taken, the plans pile up. Shipped books, W3,
$0.65 (`M15` = veto on):
| Books taken together | Trades | Same-direction plans already open at an entry: mean | median | max | Entries with ≥ 1 open | with ≥ 3 open | Open at once while anything is open: mean | max | Share of the time something is open |
|---|---|---|---|---|---|---|---|---|---|
| M15 alone | 824 | 1.17 | 1 | 6 | 62% | 15% | 1.70 | 7 | 41% |
| M5 alone | 870 | 0.69 | 0 | 6 | 40% | 7% | 1.36 | 7 | 23% |
| M5 + M15 | 1694 | 1.70 | 1 | 12 | 66% | 26% | 1.97 | 13 | 51% |
| M5 + M15 + H1 | 1947 | 2.75 | 2 | 15 | 78% | 43% | 2.60 | 16 | 65% |
The stacked entries are not worse trades (M5 + M15, by the number of same-direction plans already
open at the entry: 0 → +0.154R per trade over 577; 1 → +0.218R over 410; 2 → +0.241R over 271;
3–4 → +0.282R over 267; 5 or more → +0.236R over 169). So there is no evidence for skipping them,
and the 2026-09-06 sizing campaign already showed that varying the stake by signal features does
not pay. What the numbers do say is that "1% per alert" is in practice several percent on one move
of gold. That is a sizing fact for the reader, so it ships as information, not as a gate:
- **Size line** (`exposure.py`, off until `config.json` `trade_size.risk_per_trade` is set): the
number of lots that keeps the loss at the printed stop inside the owner's risk limit, rounded
DOWN to the lot step. Private by default (owner chats and the CLI); the channel and the public
dashboard get it only with `trade_size.telegram_channel` / `trade_size.dashboard`.
- **Already-open line**: how many ALERTED plans in the same direction are still open in the
advisory outcome model (`signal_log.open_alerted_setups`, a read-only query; the bot stamps
`alerted_at` on a row once its alert reached a chat).
- **Daily-trend line** on every surface (`main.daily_trend_words`): on M15 the veto has already
removed a plan against a clear daily trend; on M5 / H1 / H4, where the veto failed its rule, the
line is context ("Daily trend: falling — this buy goes against it"). The live wrapper stamps the
trend on every logged scan row (`signals.daily_trend`), so the live record can answer the M5
question again after a few months.
None of the three is read by a gate, the backtester or the twin.
### 5.12 Final verification (2026-10-01)
- `venv/bin/python test_all.py`: **607 passed, 0 failed** (40 modules: Module 39 daily-trend veto
and wide-stop cap, Module 40 the display lines, Module 37 extended, two Module 15 tests for the
empty-window fetch).
- The five reference books regenerated with both scripts after the last code change: ten MD5s
equal the table in section 5.9.
- Recorder and feature export with the veto on, M15 2024-06-01 → 2025-03-30, `--export-features
--record DIR --record-bars` on both scripts: feature CSV `732f396bb3ee6036b62750a8c755194e` on
both; `trades` 258 × 137, `indicators` 5900 × 15, `events` 5098 × 14 and `bars` 12315 × 172
identical; both manifests carry book MD5 `e87b6de12f90f3b5a5d41a953ee9ff7a` and differ only in
`cli`, `created_at`, `replay_seconds`, `workers`; `replay_scorecard.py DIR --check`: all
invariants held.
- `agent_panel_base_rates.json` rebuilt from the re-pinned books (M15 timeframe row 597 trades,
64.3% winners, +0.261R; was 813 / 62.2% / +0.190R).
### 5.13 Independent verification (2026-10-01)
An agent that had not done the work re-derived the change list from `git diff HEAD` and
re-ran the books itself (24 on the twin, 9 on the reference script, all offline, 0 cache
fallbacks). Every figure of sections 5.2 to 5.9 it was given matched to the cent, on both
scripts where it ran both: the M15 books with the veto on and off (W3, W2, the 2024-06-01 →
2025-03-30 window, LONG with its per-year R, the holdout), the unchanged M5 / H1 / H4 pins, the
failed M5, H1, `buy` and `sell` variants, and the removed-trade tables (281 removed on W3, 0
added, all 63 columns of the 824 kept trades identical). Its own implementation of the rule
from closed daily bars predicts the removed trades exactly (281 / 281, 618 / 618, 55 / 55); the
same rule fed the still-forming day mis-predicts 52, 129 and 14 of them, so there is no
look-ahead. The scoring configuration is identical to HEAD on all four timeframes; the only
decision-side difference is the M15 veto.
It also raised four points this report did not make clearly enough. They change no verdict;
they are recorded so the size of the claim is not overstated.
1. **Clause (g) is thinner than "3 of 4" reads.** The "100-day average alone" neighbour of
table 5.5 is an EXPONENTIAL average, the same kind as the shipped 50-day one (harness key
`e100`). With a SIMPLE 100-day average the M15 W3 book is 781 / +188.68R / PF 1.70 /
drawdown 12.27R: it beats the control on profit factor and drawdown like the others, but
sits 1.2R under the 97% line (+189.92R), so with that reading the count would be 2 of 4.
The four definitions were fixed before the runs in the harness's own form and the clause
passes as written; the honest summary is "five alternative definitions, all better than the
control on profit factor and drawdown, two of them 1.2R and 1.7R short on total R".
2. **"The veto never adds a trade" is an observation, not a guarantee.** The rule itself only
removes a plan. In a book, removing an entry also moves the 5-bar cooldown, which could
free a later entry; that happened 0 times in every control / candidate pair compared (13).
3. **It is a risk filter, not a better direction call.** Nothing in how direction is scored
changed, and a vetoed bar still shows its buy or sell reading. The removed plans were close
to break-even (−0.03R each on W3, −0.05R on LONG; 55% winners against 63% for the kept
ones), so total R barely moves while profit factor and the losing runs improve.
4. **The help is concentrated.** Effect of the rule by year on LONG (shipped minus off):
2019 +4.9R, 2020 −3.7R, 2021 −1.8R, 2022 **+27.8R**, 2023 **−10.0R**, 2024 +0.1R, 2025
+11.4R, 2026 +1.9R; total +30.8R, of which 2022 (buys against a falling daily trend) is
nine tenths. On W3 the whole gain in R comes from sells removed in 2025, and the drawdown
cut from 20.9R to 12.4R is one episode: 13 sells between 2023-12-01 and 2024-02-23
(−8.5R). In a year where the daily trend flips often (2023) the rule costs. The multi-year
windows had been looked at with the prototype before rule T was written (section 2), and
the only period nobody had seen is four weeks long and differs by six trades.
Cross-checked here from the formal books (a filter on the control book reproduces the shipped
book and the exponential neighbour exactly, and gives the simple-average figure above).
## 6. Summary of decisions
| Item | Verdict |
|---|---|
| Daily-trend veto, M15, both directions | **PASSED rule T on every clause, SHIPPED** (factory M15 override 7) |
| Daily-trend veto, M15, buy-only / sell-only | FAILED (W3 R and PF / LONG drawdown and 2021–22) |
| Daily-trend veto, M5 (both, buy-only) | FAILED (W3 drawdown; LONG R, drawdown, 2021–22; no plateau) |
| Daily-trend veto, H1 | FAILED (W3 −9.7R); H4 not run |
| Wide-stop cap K = 6 on M15 | FAILED rule C (neighbour K = 5 loses 18.6R on LONG); knob kept at 0.0, research-only |
| Lot-size, already-open and daily-trend lines | SHIPPED, display only (books byte-identical) |
| Oanda empty date window | FIXED (fetch_range no longer aborts on a closed-day page) |
| Seven further context screens | no state nominated, nothing shipped (`context_screens_2026-10.md`) |
| Eight outside data sources (dollar, silver, bonds, real yield, fund flows, futures and Oanda client positioning) | none adds a forecast to gold's own daily trend; broad dollar index on M5 = WATCH; nothing shipped, no feed added (`new_inputs_2026-10.md`) |
+304
View File
@@ -0,0 +1,304 @@
# New data sources for the direction call — a report-only study, 2026-10-01
Owner decision 2026-09-30: after the daily-trend work, study new data sources and report; nothing
ships from this file. The engine reads gold's own price and nothing else. The question here is
whether anything OUTSIDE gold's own chart tells the direction better than gold's chart already
does. Sections 1 to 3 were written before any data was fetched.
## 1. Sources
| # | Source | What it is | Where from | When a value is usable |
|---|---|---|---|---|
| 1 | Dollar | EUR/USD daily candles, read upside down (a falling EUR/USD is a rising dollar) | Oanda v20 candles, the live feed's own token | daily close + 0 (same rule as gold's daily bars) |
| 2 | Silver | XAG/USD daily candles | Oanda v20 | daily close |
| 3 | US 10-year note | USB10Y_USD daily candles (price up = yields down), if Oanda offers it | Oanda v20 | daily close |
| 4 | Real yield | US 10-year inflation-indexed yield, `DFII10` | FRED (public CSV) | the day's value from two days later (it is published the next US afternoon) |
| 5 | Broad dollar index | `DTWEXBGS` | FRED | same two-day rule |
| 6 | Gold fund holdings | SPDR Gold Shares, shares outstanding (each share is a fixed slice of the vault, so creations and redemptions are the fund's gold flow) | the fund manager's public history file (`navhist-us-en-gld.xlsx`); the older tonnes archive now serves a PDF | the day's value from two days later |
| 7 | Futures positioning | CFTC disaggregated report, gold (code 088691), managed-money net long | CFTC public reporting site | a Tuesday report from the following Monday |
| 8 | Oanda client positioning | position book for XAU_USD: the share of Oanda clients' open gold positions that are long (by count) | Oanda v20 `/v3/instruments/XAU_USD/positionBook`, one snapshot per day at 00:00 UTC | at once (the snapshot is taken at the daily close) |
Everything fetched stays in the session scratchpad. Nothing is added to the repository's cache.
Amendment before any analysis (2026-10-01, after the access probes and nothing else): the plan
expected source 8 to be gone. It is not: on the practice API the position book answers for every
date probed back to 2017, so it joins the study with the definition below. The probe printed ten
long-share readings (between 60% and 75% long, higher at higher gold prices) and no forward
returns; that is everything that was seen before the definition was fixed. Source 6 uses shares
outstanding instead of tonnes because that is the file that can still be downloaded.
## 2. One state per source
The same shape as the shipped daily trend, so the comparison is like for like. For a daily series
`x` (Sunday stub bars dropped for the Oanda candles): `up` when `x` is above its 50-day EMA AND
above its value 60 days earlier, `down` when below both, else `unclear`. Read for gold:
| Source | Gold-friendly state | Gold-hostile state |
|---|---|---|
| Dollar (EUR/USD) | EUR/USD up (dollar falling) | EUR/USD down |
| Silver | up | down |
| 10-year note price | up (yields falling) | down |
| Real yield | down | up |
| Broad dollar index | down | up |
| Fund holdings | shares outstanding above the level 20 trading days earlier | below |
| Futures positioning | managed-money net long above its level 4 reports earlier | below |
| Oanda client positioning | long share `down` (the crowd is cutting longs or adding shorts) | long share `up` (the crowd is adding longs) |
The positioning state is read the contrarian way on purpose and before looking: small traders
tend to buy dips and sell rallies, so a growing long share is expected to go with a falling price.
Test A is two-sided, so if the truth is the other way round it will show as a real spread with the
opposite sign and is reported as such; Test B is judged on the declared reading only.
## 3. Two tests and the verdict rule
**Test A — forward move of gold.** Every closed gold daily bar from 2019-01-01 to 2026-08-07.
Forward move = gold's close 5 days later minus today's close, in units of gold's own 14-day ATR
(and the same for 20 days). For each source: mean forward move on gold-friendly days minus mean
on gold-hostile days, (i) over all days and (ii) inside each of gold's own three daily-trend
states (rising / falling / unclear), in two eras (2019–2023, 2024–2026). The spread is called real
only if it has the same sign in both eras and its t-value, computed on every fifth day so the
5-day windows do not overlap, is at least 2 in each era.
**Test B — the shipped books.** M15 LONG (daily-trend veto on, 1473 trades) and M5 LONG (1052
trades), $0.65. Each trade is labelled `friendly` / `hostile` / `unclear` from the source state at
its decision time, mirrored for a SELL. A source is interesting on a timeframe only if its
`hostile` trades are at least 8% of the book and lose (total R < 0) in BOTH eras.
**Verdict per source.**
- `GO` (worth a pre-registered campaign and the feed work it needs) only if Test A calls the
spread real INSIDE gold's own trend states (that is the "beyond gold's own chart" part) AND
Test B finds the hostile trades losing in both eras on at least one timeframe.
- `WATCH` if exactly one of the two holds.
- `NO-GO` otherwise, or when the source cannot be fetched reliably.
A `GO` is a ticket to a campaign, never a shipping decision: a live feed for a second instrument
needs its own provider path, cache, staleness rule and live == backtest proof.
Two points of method were fixed while the analysis script was written, before it produced any
output. (1) The broad dollar index is published weekly (the H.10 release, on Monday, for the week
before), not daily, so the "two-day rule" of the table would have looked ahead: each week's final
state is used from the Tuesday after that week. (2) "Inside gold's own trend states" is computed
as one stratified number per era: the forward move minus the mean forward move of that day's gold
trend state (rising / unclear / falling), friendly days against hostile days, t-value on every
fifth day. That is the column the verdict reads; the unstratified spread is printed beside it.
## 4. Results
Run 2026-10-01. Fetched into the session scratchpad: Oanda daily candles for EUR_USD, XAG_USD,
USB10Y_USD and XAU_USD (2015-01-01 → 2026-09-30, about 3,650 bars each, about 600 Sunday stubs
dropped); the Oanda XAU_USD position book, one snapshot per day at 00:00 UTC from 2018-06-01
(2,609 days asked, 97.7% answered at exactly 00:00, the rest from the last snapshot up to six
hours earlier, 45 days with none, all before mid-2022); FRED `DFII10` and `DTWEXBGS`; the gold
fund's history file (5,540 days since 2004); 1,059 weekly CFTC reports since 2006.
### 4.1 What was fetched
| Source | Days with a state, 2019-01-01 → 2026-08-07 | friendly / unclear / hostile | Same side as gold's own daily trend (days where both are clear) |
|---|---|---|---|
| 1 dollar (EUR/USD) | 1973 | 37% / 18% / 44% | 67% of 1363 |
| 2 silver | 1973 | 49% / 21% / 31% | 88% of 1387 |
| 3 10-year note price | 1973 | 39% / 16% / 46% | 73% of 1398 |
| 4 real yield (DFII10) | 1973 | 42% / 20% / 39% | 73% of 1353 |
| 5 broad dollar index | 1973 | 39% / 19% / 42% | 62% of 1362 |
| 6 gold fund shares | 1973 | 51% / 0% / 49% | 77% of 1639 |
| 7 futures positioning | 1973 | 51% / 0% / 49% | 59% of 1644 |
| 8 Oanda client positioning | 1973 | 37% / 15% / 48% | 36% of 1414 |
Gold itself, same days: rising 58%, unclear 17%, falling 25%; mean 5-day forward move +0.27 ATR (rising +0.32, unclear +0.30, falling +0.14).
### 4.2 Test A — gold's forward move, friendly days minus hostile days (in daily ATRs)
| Source | Era | Days friendly / hostile | 5-day spread, all days (t) | 5-day spread inside gold's trend states (t) | 20-day spread inside gold's trend states (t) | Real? |
|---|---|---|---|---|---|---|
| 1 dollar (EUR/USD) | 2019–2023 | 443 / 623 | -0.04 (-0.3) | -0.05 (-0.4) | +0.35 (+0.7) | |
| 1 dollar (EUR/USD) | 2024–2026 | 296 / 249 | +0.21 (+0.7) | +0.20 (+0.7) | +0.80 (+0.5) | |
| 1 dollar (EUR/USD) | verdict | | | same sign in both eras: no; |t| ≥ 2 in both: no | | not real |
| 2 silver | 2019–2023 | 548 / 449 | -0.21 (-0.9) | -0.25 (-1.0) | -1.00 (-1.2) | |
| 2 silver | 2024–2026 | 411 / 160 | -0.10 (-0.7) | -0.20 (-0.7) | -0.64 (-0.2) | |
| 2 silver | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 3 10-year note price | 2019–2023 | 515 / 603 | +0.08 (+0.3) | +0.04 (+0.1) | +0.46 (+0.5) | |
| 3 10-year note price | 2024–2026 | 253 / 295 | +0.23 (+0.4) | +0.17 (+0.4) | +1.02 (-0.2) | |
| 3 10-year note price | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 4 real yield (DFII10) | 2019–2023 | 583 / 468 | +0.04 (+0.4) | +0.01 (+0.2) | -0.20 (-0.5) | |
| 4 real yield (DFII10) | 2024–2026 | 242 / 294 | +0.20 (+0.7) | +0.14 (+0.6) | +1.22 (+0.5) | |
| 4 real yield (DFII10) | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 5 broad dollar index | 2019–2023 | 497 / 536 | +0.03 (-0.1) | +0.02 (-0.1) | -0.12 (-0.5) | |
| 5 broad dollar index | 2024–2026 | 269 / 293 | +0.02 (-0.2) | +0.04 (-0.0) | +0.71 (+0.1) | |
| 5 broad dollar index | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 6 gold fund shares | 2019–2023 | 609 / 681 | +0.20 (+1.4) | +0.17 (+1.2) | +0.37 (+1.0) | |
| 6 gold fund shares | 2024–2026 | 393 / 281 | +0.40 (+1.7) | +0.36 (+1.8) | +1.09 (-0.1) | |
| 6 gold fund shares | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 7 futures positioning | 2019–2023 | 640 / 656 | +0.06 (+0.0) | +0.04 (-0.1) | +1.03 (+0.6) | |
| 7 futures positioning | 2024–2026 | 369 / 308 | -0.01 (+0.3) | +0.11 (+0.8) | +1.23 (+1.7) | |
| 7 futures positioning | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 8 Oanda client positioning | 2019–2023 | 636 / 428 | +0.20 (+1.2) | +0.19 (+1.1) | +0.53 (+0.1) | |
| 8 Oanda client positioning | 2024–2026 | 97 / 516 | -0.01 (-0.6) | +0.33 (+0.3) | n/a (too few friendly days) | |
| 8 Oanda client positioning | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
### 4.3 Test B — the shipped books, by the source's state at the trade's decision time
**M15 LONG (daily-trend veto on): 1473 trades, +223.3R** (era A = 2019–2023: 728 trades, +36.8R; era B = 2024–2026: 745 trades, +186.6R)
| Source | Trade state | Trades (share) | Total R / mean | Era A R (trades) | Era B R (trades) | Hostile ≥ 8% and losing in both eras |
|---|---|---|---|---|---|---|
| 1 dollar (EUR/USD) | friendly | 725 (49%) | +127.3 / +0.176 | +32.0 (345) | +95.4 (380) | |
| 1 dollar (EUR/USD) | unclear | 252 (17%) | +13.0 / +0.052 | -15.3 (127) | +28.3 (125) | |
| 1 dollar (EUR/USD) | hostile | 496 (34%) | +83.0 / +0.167 | +20.1 (256) | +62.9 (240) | no |
| 2 silver | friendly | 937 (64%) | +154.9 / +0.165 | +26.8 (406) | +128.0 (531) | |
| 2 silver | unclear | 304 (21%) | +1.3 / +0.004 | -18.2 (196) | +19.5 (108) | |
| 2 silver | hostile | 232 (16%) | +67.1 / +0.289 | +28.1 (126) | +39.0 (106) | no |
| 3 10-year note price | friendly | 753 (51%) | +107.0 / +0.142 | +34.6 (397) | +72.4 (356) | |
| 3 10-year note price | unclear | 254 (17%) | +39.1 / +0.154 | +11.3 (121) | +27.8 (133) | |
| 3 10-year note price | hostile | 466 (32%) | +77.2 / +0.166 | -9.2 (210) | +86.4 (256) | no |
| 4 real yield (DFII10) | friendly | 723 (49%) | +97.7 / +0.135 | +45.5 (405) | +52.2 (318) | |
| 4 real yield (DFII10) | unclear | 313 (21%) | +30.8 / +0.098 | -12.5 (142) | +43.2 (171) | |
| 4 real yield (DFII10) | hostile | 437 (30%) | +94.9 / +0.217 | +3.8 (181) | +91.1 (256) | no |
| 5 broad dollar index | friendly | 647 (44%) | +122.5 / +0.189 | +37.5 (315) | +85.0 (332) | |
| 5 broad dollar index | unclear | 277 (19%) | +17.2 / +0.062 | -4.7 (168) | +21.8 (109) | |
| 5 broad dollar index | hostile | 549 (37%) | +83.7 / +0.152 | +3.9 (245) | +79.8 (304) | no |
| 6 gold fund shares | friendly | 1031 (70%) | +183.6 / +0.178 | +52.7 (519) | +130.9 (512) | |
| 6 gold fund shares | unclear | 7 (0%) | -4.7 / -0.668 | -2.9 (4) | -1.8 (3) | |
| 6 gold fund shares | hostile | 435 (30%) | +44.4 / +0.102 | -13.1 (205) | +57.5 (230) | no |
| 7 futures positioning | friendly | 789 (54%) | +115.5 / +0.146 | +22.3 (422) | +93.3 (367) | |
| 7 futures positioning | unclear | 0 (0%) | +0.0 / | +0.0 (0) | +0.0 (0) | |
| 7 futures positioning | hostile | 684 (46%) | +107.8 / +0.158 | +14.5 (306) | +93.3 (378) | no |
| 8 Oanda client positioning | friendly | 394 (27%) | +41.7 / +0.106 | +15.9 (326) | +25.9 (68) | |
| 8 Oanda client positioning | unclear | 198 (13%) | +26.9 / +0.136 | +13.7 (122) | +13.2 (76) | |
| 8 Oanda client positioning | hostile | 881 (60%) | +154.7 / +0.176 | +7.2 (280) | +147.5 (601) | no |
**M5 LONG: 1052 trades, +128.1R** (era A = 2019–2023: 258 trades, -10.4R; era B = 2024–2026: 794 trades, +138.5R)
| Source | Trade state | Trades (share) | Total R / mean | Era A R (trades) | Era B R (trades) | Hostile ≥ 8% and losing in both eras |
|---|---|---|---|---|---|---|
| 1 dollar (EUR/USD) | friendly | 439 (42%) | +85.3 / +0.194 | -11.5 (103) | +96.8 (336) | |
| 1 dollar (EUR/USD) | unclear | 211 (20%) | +28.6 / +0.136 | -0.9 (34) | +29.5 (177) | |
| 1 dollar (EUR/USD) | hostile | 402 (38%) | +14.2 / +0.035 | +2.0 (121) | +12.2 (281) | no |
| 2 silver | friendly | 556 (53%) | +89.3 / +0.161 | -7.6 (113) | +96.8 (443) | |
| 2 silver | unclear | 143 (14%) | +7.3 / +0.051 | -9.9 (45) | +17.2 (98) | |
| 2 silver | hostile | 353 (34%) | +31.5 / +0.089 | +7.1 (100) | +24.4 (253) | no |
| 3 10-year note price | friendly | 432 (41%) | +51.6 / +0.120 | -0.6 (119) | +52.2 (313) | |
| 3 10-year note price | unclear | 169 (16%) | -9.5 / -0.056 | -13.7 (49) | +4.2 (120) | |
| 3 10-year note price | hostile | 451 (43%) | +86.0 / +0.191 | +3.9 (90) | +82.1 (361) | no |
| 4 real yield (DFII10) | friendly | 377 (36%) | +4.2 / +0.011 | -15.0 (115) | +19.2 (262) | |
| 4 real yield (DFII10) | unclear | 186 (18%) | +32.3 / +0.174 | -7.0 (45) | +39.3 (141) | |
| 4 real yield (DFII10) | hostile | 489 (46%) | +91.6 / +0.187 | +11.6 (98) | +80.0 (391) | no |
| 5 broad dollar index | friendly | 447 (42%) | +131.2 / +0.294 | -0.7 (91) | +131.9 (356) | |
| 5 broad dollar index | unclear | 170 (16%) | +8.1 / +0.047 | -6.0 (40) | +14.1 (130) | |
| 5 broad dollar index | hostile | 435 (41%) | -11.2 / -0.026 | -3.7 (127) | -7.5 (308) | YES |
| 6 gold fund shares | friendly | 600 (57%) | +86.9 / +0.145 | -15.8 (142) | +102.7 (458) | |
| 6 gold fund shares | unclear | 3 (0%) | -1.8 / -0.608 | +0.0 (0) | -1.8 (3) | |
| 6 gold fund shares | hostile | 449 (43%) | +43.0 / +0.096 | +5.4 (116) | +37.6 (333) | no |
| 7 futures positioning | friendly | 493 (47%) | +41.1 / +0.083 | -17.0 (120) | +58.1 (373) | |
| 7 futures positioning | unclear | 0 (0%) | +0.0 / | +0.0 (0) | +0.0 (0) | |
| 7 futures positioning | hostile | 559 (53%) | +87.0 / +0.156 | +6.6 (138) | +80.4 (421) | no |
| 8 Oanda client positioning | friendly | 313 (30%) | +17.9 / +0.057 | -1.9 (99) | +19.8 (214) | |
| 8 Oanda client positioning | unclear | 92 (9%) | +2.3 / +0.025 | +0.5 (40) | +1.9 (52) | |
| 8 Oanda client positioning | hostile | 647 (62%) | +107.9 / +0.167 | -8.9 (119) | +116.9 (528) | no |
### 4.4 Verdict per source
| Source | Test A: real spread beyond gold's own trend | Test B: hostile trades ≥ 8% and losing in both eras (M15 / M5) | Verdict |
|---|---|---|---|
| 1 dollar (EUR/USD) | no | no / no | NO-GO |
| 2 silver | no | no / no | NO-GO |
| 3 10-year note price | no | no / no | NO-GO |
| 4 real yield (DFII10) | no | no / no | NO-GO |
| 5 broad dollar index | no | no / yes | WATCH |
| 6 gold fund shares | no | no / no | NO-GO |
| 7 futures positioning | no | no / no | NO-GO |
| 8 Oanda client positioning | no | no / no | NO-GO |
### 4.5 The one flag, looked at more closely (after the fact: description, not a test)
**M5 LONG: gold's own daily trend (rows) x 5 broad dollar index (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +118.8 (352) | +1.9 (105) | -0.1 (215) |
| unclear | +6.8 (33) | -2.1 (12) | +2.3 (73) |
| against | +5.6 (62) | +8.3 (53) | -13.3 (147) |
**M5 LONG: gold's own daily trend (rows) x 1 dollar (EUR/USD) (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +76.8 (372) | +23.0 (109) | +20.7 (191) |
| unclear | +4.5 (37) | -5.5 (19) | +7.9 (62) |
| against | +4.0 (30) | +11.1 (83) | -14.4 (149) |
**M5 LONG: broad-dollar hostile trades by year** — 2019: +1.4 (14); 2020: -3.4 (45); 2021: -0.6 (21); 2022: -1.1 (36); 2023: -0.1 (11); 2024: +3.5 (68); 2025: -8.5 (107); 2026: -2.4 (133)
**M5 LONG: broad-dollar friendly trades by year** — 2019: -1.5 (3); 2020: +0.0 (45); 2021: -0.0 (9); 2022: +0.0 (7); 2023: +0.7 (27); 2024: -0.1 (19); 2025: +40.3 (160); 2026: +91.7 (177)
W3 friendly: 397 trades (46%), +145.1R, halves +8.2 / +136.9
W3 unclear: 150 trades (17%), +15.6R, halves -3.0 / +18.7
W3 hostile: 323 trades (37%), -7.1R, halves +11.6 / -18.7
**M15 LONG (veto on): gold's own daily trend (rows) x 5 broad dollar index (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +93.2 (561) | +48.0 (196) | +68.5 (416) |
| unclear | +29.3 (86) | -30.8 (81) | +15.2 (133) |
| against | +0.0 (0) | +0.0 (0) | +0.0 (0) |
**M15 LONG (veto on): gold's own daily trend (rows) x 1 dollar (EUR/USD) (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +99.4 (635) | +41.8 (176) | +68.5 (362) |
| unclear | +27.9 (90) | -28.8 (76) | +14.5 (134) |
| against | +0.0 (0) | +0.0 (0) | +0.0 (0) |
**M15 LONG (veto on): broad-dollar hostile trades by year** — 2019: +12.5 (57); 2020: +11.5 (74); 2021: -6.7 (44); 2022: -9.2 (43); 2023: -4.1 (27); 2024: +65.1 (179); 2025: +9.9 (95); 2026: +4.8 (30)
**M15 LONG (veto on): broad-dollar friendly trades by year** — 2019: +4.7 (16); 2020: +25.0 (114); 2021: -8.9 (46); 2022: +8.2 (51); 2023: +8.5 (88); 2024: -4.4 (56); 2025: +52.1 (192); 2026: +37.3 (84)
W3 friendly: 373 trades (45%), +103.5R, halves +4.0 / +99.4
W3 unclear: 130 trades (16%), +28.2R, halves +6.2 / +22.1
W3 hostile: 321 trades (39%), +73.4R, halves +66.5 / +6.9
### 4.6 Reading
- **Nothing outside gold's own chart passes Test A.** For all eight sources the 5-day forward move
of gold on "friendly" days is not reliably different from "hostile" days once gold's own daily
trend is accounted for (largest t-values 1.2 and 1.8, the gold fund's share count; everything
else is under 1.2 in at least one era). The familiar stories (a falling dollar, falling real
yields, rising silver, fund inflows) describe gold's past move well: they sit on the same side
as gold's own daily trend on 59% to 88% of clear days. They do not add a forecast to it.
- **Oanda client positioning is available but not useful in this form.** The plan expected the
endpoint to be gone; it answers, with history back to 2017. Read the contrarian way (a growing
long share is hostile), the spread has the expected sign and no weight (t 1.1 and 0.3), and in
2024–2026 the crowd was adding longs on 516 of 613 clear days while gold rose. The trades taken
on those "hostile" days are the best part of both books. Only the daily state was tested; the
intraday snapshots (one per hour) were not.
- **The one flag is the broad dollar index on M5** (`WATCH`): the 435 M5 trades taken against the
dollar's trend lost 11.2R in total and lost in both eras, while the 447 taken with it made
+131.2R. The closer look shows what that is. All of the M5 profit sits in one cell: trades with
gold's own daily trend AND with the dollar's trend (+118.8R over 352 trades); the same trades
against the dollar's trend made nothing (−0.1R over 215). But every R of the friendly side was
earned in 2025 and 2026 (+40.3R and +91.7R), that is inside the one long rally from 2,600 to the
March 2026 peak (in 2025 the broad dollar fell 7.5% while gold rose 65%); before 2025 the
dollar's state separates nothing on M5 because the M5 book earned nothing at all. In the 3-year
window the hostile trades are positive in the first half (+11.6R) and negative only in the
second (−18.7R). On M15 the same state separates nothing: the hostile
trades made +65R in 2024 alone, a year when gold and the dollar rose together. The timelier
dollar measure (EUR/USD) shows the same lean on M5 without losing in both eras. So this is one
market phase seen from another angle, not a second source of information; it stays a
hypothesis, and it is the same picture as the context screens: M5 pays only when everything
agrees and has paid only since 2025.
### 4.7 Verdict
| Source | Verdict | Why |
|---|---|---|
| Dollar via EUR/USD | NO-GO | no forward spread; hostile trades are not losers in both eras |
| Silver | NO-GO | 88% the same information as gold's own trend; the spread has the wrong sign |
| US 10-year note | NO-GO | no forward spread |
| Real yield | NO-GO | no forward spread; on M5 the "hostile" trades are the better ones |
| Broad dollar index | WATCH | M5 trades against it lose in both eras, but the forward test finds nothing and the effect is the 2025–2026 phase; weekly data, four to ten days old when used |
| Gold fund share count | NO-GO | the nearest miss in the forward test (t 1.2 and 1.8), nothing on the books |
| Futures positioning | NO-GO | no forward spread, no difference on the books |
| Oanda client positioning | NO-GO in this daily form | available, but the contrarian reading failed in the rising years |
**Nothing ships and no feed is added.** The direction call was improved by the one input that
passed a formal test, gold's own daily chart on M15. If the M5 question is taken up again, the
honest test is forward: `signals.daily_trend` is now logged on every scan, so a few months of live
M5 rows can be split by the daily trend, and by the dollar's trend after the fact, without touching
the signal code.
+7
View File
@@ -71,5 +71,12 @@
"mode": "off", "mode": "off",
"deepseek_audit_per_day": 0 "deepseek_audit_per_day": 0
}, },
"trade_size": {
"risk_per_trade": 0,
"contract_size": 100,
"lot_step": 0.01,
"telegram_channel": false,
"dashboard": false
},
"log_level": "INFO" "log_level": "INFO"
} }
+23
View File
@@ -325,6 +325,29 @@ class XauStrategyConfig:
big_candle_atr_multiple: float = 0.0 big_candle_atr_multiple: float = 0.0
big_candle_lookback_hours: float = 24.0 big_candle_lookback_hours: float = 24.0
# ── Daily-trend veto (2026-09-30) ──────────────────────────────
# The engine sees at most the H1 chart, and on M15 trades against a
# CLEAR daily trend earn nothing after costs. The daily trend is +1 /
# -1 when the last closed D1 close sits above / below its 50-bar EMA
# AND above / below the close 60 daily bars earlier, else 0
# (signal_aggregator.daily_trend_series; Sunday stub bars dropped).
# "both" removes any finished setup that goes against a non-zero
# daily trend, "buy" only BUYs against a falling one, "sell" only
# SELLs against a rising one, "off" nothing. Off by default; the M15
# factory sets "both" (override 7). It FAILED its rule on M5 and H1
# and as "buy" / "sell" on M15
# (backtest/research/daily_trend_campaign_2026-09-30.md).
daily_trend_veto: str = "off"
# ── Wide-stop cap (2026-09-30, research knob) ──────────────────
# Caps the stop distance of the swing (GENERIC) and chart-pattern
# setups at max(K x ATR, the setup's own stop floor). Applied AFTER
# the min-RR and spread gates, like the printed-target floor, so
# the set of trades cannot change. 0.0 = off, and it ships off
# everywhere: K = 6 FAILED its rule on M15 (a narrow peak; K = 5
# loses 18.6R over 2019-2026). Same campaign report.
wide_stop_cap_atr: float = 0.0
@dataclass @dataclass
class FilterConfig: class FilterConfig:
+6
View File
@@ -65,6 +65,7 @@ from agent_panel import attach_panel
from app_config import (get_app_config, set_db_writes_for_process, from app_config import (get_app_config, set_db_writes_for_process,
violent_watch_enabled) violent_watch_enabled)
from config import PlatformConfig, Timeframe, XAUUSD from config import PlatformConfig, Timeframe, XAUUSD
from exposure import attach_exposure
from main import report_to_dict from main import report_to_dict
from news_engine import NewsEngine from news_engine import NewsEngine
from pair_signals import produce_signal from pair_signals import produce_signal
@@ -437,6 +438,11 @@ class DashboardPublisher:
if violent_watch_enabled(): if violent_watch_enabled():
attach_watch(report, tf, attach_watch(report, tf,
get_app_config().violent_watch_settings) get_app_config().violent_watch_settings)
# Display-only "already open" line (a read-only query on the
# bot's signals.db — this process still never writes it) and,
# only when config.json sets trade_size.dashboard, the lot
# size: the page is public and the size shows the owner's risk.
attach_exposure(report, surface="dashboard")
# Which boundary this report belongs to: the bar LABEL it closed # Which boundary this report belongs to: the bar LABEL it closed
# on plus one interval. Recorded so a boundary missed by an # on plus one interval. Recorded so a boundary missed by an
# overrunning cycle or a restart is caught up on the next wake # overrunning cycle or a restart is caught up on the next wake
+11
View File
@@ -77,6 +77,17 @@ boundary (+10 s) and rescans whichever timeframes just closed. Logs rotate
every 3 h (240 backups); the bot's own `telegram.log` rotates daily (30 every 3 h (240 backups); the bot's own `telegram.log` rotates daily (30
backups) since 2026-09-08. backups) since 2026-09-08.
Since 2026-09-30 each card also shows the daily trend, and a trade plan can
carry an "Already open" line. That line is one **read-only** query on the
bot's `logs/signals.db` (`mode=ro`; this process still never writes either
database), so the daemon user needs read access to that file and its `-wal`
/ `-shm` companions. If it cannot read them the line is simply absent. The
lot-size line is NOT published here unless `config.json` sets
`"trade_size": {"dashboard": true}`: the page is public and the size shows
the owner's risk per trade. The M15 scan now needs the daily (D1) candles as
a required frame, so a D1 fetch failure fails that card's scan like any other
missing frame.
## 5. Optional Apache polish ## 5. Optional Apache polish
The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache
+11 -3
View File
@@ -40,7 +40,7 @@
<body> <body>
<h1>ForexBot2 — Beginner's Guide</h1> <h1>ForexBot2 — Beginner's Guide</h1>
<div class="subtitle">Everything explained in plain words. No experience needed. &nbsp;·&nbsp; July 2026</div> <div class="subtitle">Everything explained in plain words. No experience needed. &nbsp;·&nbsp; updated October 2026</div>
<h2>1 · What is this bot?</h2> <h2>1 · What is this bot?</h2>
<p>ForexBot2 is a computer program that watches the <b>price of gold</b> all day and all night. It studies the price charts using nine different mathematical checks. When the checks agree that the price is likely to move in one direction, it sends you a message on <b>Telegram</b> telling you:</p> <p>ForexBot2 is a computer program that watches the <b>price of gold</b> all day and all night. It studies the price charts using nine different mathematical checks. When the checks agree that the price is likely to move in one direction, it sends you a message on <b>Telegram</b> telling you:</p>
@@ -74,12 +74,17 @@
🟢 Grade A · BUY · XAU/USD · H1 🥇 ← how good the setup is + what to do 🟢 Grade A · BUY · XAU/USD · H1 🥇 ← how good the setup is + what to do
📐 Direction: strong buy · Reliability: 📐 Direction: strong buy · Reliability:
good · timeframes agree ← the bot's two numbers, in words good · timeframes agree ← the bot's two numbers, in words
🧭 Daily trend: rising — this buy
goes with it ← which way gold has been heading for weeks
📋 Indicators … ← the bot's 9 checks (you can skip this part) 📋 Indicators … ← the bot's 9 checks (you can skip this part)
━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━
🚀 TRADE SETUP ← ↓ THIS is the part you act on ↓ 🚀 TRADE SETUP ← ↓ THIS is the part you act on ↓
📈 Direction: BUY 📈 Direction: BUY
▶ Entry: 4,132.50 ← open the trade near this price ▶ Entry: 4,132.50 ← open the trade near this price
🛑 SL: 4,088.20 ← your safety brake (stop loss) 🛑 SL: 4,088.20 ← your safety brake (stop loss)
🧱 Already open: 1 buy plan alerted
earlier (H1) — taking this one adds
to the same position ← you may already be in this move
🔄 Exit rule (tested): once price reaches 🔄 Exit rule (tested): once price reaches
4,159.08 (+0.6R), trail the SL $33.23 4,159.08 (+0.6R), trail the SL $33.23
behind the best price, moved at behind the best price, moved at
@@ -91,6 +96,8 @@
<ul> <ul>
<li><b>Grade (A+, A, B, C):</b> the bot's own mark for how clean the setup is. A+ is best. It never sends anything worse than C.</li> <li><b>Grade (A+, A, B, C):</b> the bot's own mark for how clean the setup is. A+ is best. It never sends anything worse than C.</li>
<li><b>Direction and reliability:</b> plain words for the bot's two internal numbers. Direction says how strongly the checks lean one way (very strong, strong, clear, weak). Reliability says how much the bot trusts that reading (high, good, fair, low). "Timeframes agree" means the bigger and smaller charts point the same way.</li> <li><b>Direction and reliability:</b> plain words for the bot's two internal numbers. Direction says how strongly the checks lean one way (very strong, strong, clear, weak). Reliability says how much the bot trusts that reading (high, good, fair, low). "Timeframes agree" means the bigger and smaller charts point the same way.</li>
<li><b>🧭 Daily trend:</b> which way the gold price has been heading over the last weeks, read from the daily chart: rising, falling, or no clear direction. A trade that goes the same way as the daily trend is swimming with the tide. On the 15-minute signals the bot no longer sends a trade that goes against a clear daily trend at all, because years of tests showed those trades earned nothing. On the other signals the line is simply something to know.</li>
<li><b>🧱 Already open:</b> appears when the bot has already sent buy (or sell) plans that are still running. A new signal in the same direction is usually the same gold move seen again. The new signal is not a worse one, but if you took the earlier one, taking this one too means you are risking more on a single move. As a beginner, skip it, or split your usual risk between the two.</li>
<li><b>The indicator list:</b> extra detail for curious people. You don't need to understand it to trade the signal.</li> <li><b>The indicator list:</b> extra detail for curious people. You don't need to understand it to trade the signal.</li>
<li><b>⚠️ Warning lines:</b> things the bot noticed, like high volatility. They are "be aware," not "don't trade."</li> <li><b>⚠️ Warning lines:</b> things the bot noticed, like high volatility. They are "be aware," not "don't trade."</li>
<li><b>🌪 Violent-move watch:</b> now and then the bot adds a line saying the next hour could produce very big candles — usually because an important American economic number is about to come out, or the market is in an hour that is historically wild, or the price has stretched a long way from normal. It is only a "hold on tight" note. It does not change the signal, and it is not a reason to close or move your stop loss early — the bot was tested doing exactly that and it lost money. Your stop loss is already your protection; leave it where it is.</li> <li><b>🌪 Violent-move watch:</b> now and then the bot adds a line saying the next hour could produce very big candles — usually because an important American economic number is about to come out, or the market is in an hour that is historically wild, or the price has stretched a long way from normal. It is only a "hold on tight" note. It does not change the signal, and it is not a reason to close or move your stop loss early — the bot was tested doing exactly that and it lost money. Your stop loss is already your protection; leave it where it is.</li>
@@ -106,7 +113,7 @@ Your account: <b>$2,000</b>. Your risk: 1% = <b>$20</b>. So for this trade, 1R =
The signal says: Entry 4,132.50, Stop loss 4,088.20. The gap between them is <b>$44.30 per ounce</b> of gold.<br> The signal says: Entry 4,132.50, Stop loss 4,088.20. The gap between them is <b>$44.30 per ounce</b> of gold.<br>
How much gold to buy: $20 ÷ $44.30 = <b>0.45 ounces</b>.<br> How much gold to buy: $20 ÷ $44.30 = <b>0.45 ounces</b>.<br>
Now, whatever happens: if the price falls to the stop loss, you lose about $20 — and no more. If the trade wins 2R, you make about $40.</div> Now, whatever happens: if the price falls to the stop loss, you lose about $20 — and no more. If the trade wins 2R, you make about $40.</div>
<p class="small">Most broker apps have a built-in calculator: enter the stop-loss price and the amount you want to risk ($20), and it works out the position size for you. Use it — it avoids mistakes. (On many platforms gold is traded in "lots," where 0.01 lot = 1 ounce.)</p> <p class="small">Most broker apps have a built-in calculator: enter the stop-loss price and the amount you want to risk ($20), and it works out the position size for you. Use it — it avoids mistakes. (On many platforms gold is traded in "lots," where 0.01 lot = 1 ounce.) The bot's owner can also switch on a <b>📏 Size</b> line that does this sum for their own risk limit; it is shown only in the owner's private chat, so if you do not see it, use the calculator.</p>
<h2>5 · What to do when a signal arrives — step by step</h2> <h2>5 · What to do when a signal arrives — step by step</h2>
<ol> <ol>
@@ -135,7 +142,7 @@ The message says: <i>"once price hits 4,159, trail SL $33 behind best price."</i
<p>The bot sends signals from several chart speeds. As a beginner, <b>follow only the 1-hour (H1) signals</b> and ignore the others. Here is why:</p> <p>The bot sends signals from several chart speeds. As a beginner, <b>follow only the 1-hour (H1) signals</b> and ignore the others. Here is why:</p>
<table> <table>
<tr><th></th><th>M5 (5-minute)</th><th>M15 (15-minute)</th><th>H1 (1-hour) ★ start here</th></tr> <tr><th></th><th>M5 (5-minute)</th><th>M15 (15-minute)</th><th>H1 (1-hour) ★ start here</th></tr>
<tr><td>Signals per week</td><td>about 10</td><td>about 10</td><td>about 3</td></tr> <tr><td>Signals per week</td><td>about 10</td><td>about 5</td><td>about 2</td></tr>
<tr><td>A trade usually lasts</td><td>a few hours</td><td>about half a day</td><td>half a day to a few days</td></tr> <tr><td>A trade usually lasts</td><td>a few hours</td><td>about half a day</td><td>half a day to a few days</td></tr>
<tr><td>Time you have to react</td><td>1–2 minutes</td><td>up to 15 minutes</td><td>up to an hour</td></tr> <tr><td>Time you have to react</td><td>1–2 minutes</td><td>up to 15 minutes</td><td>up to an hour</td></tr>
<tr><td>Suits</td><td>full-time / automated</td><td>experienced, quick</td><td>anyone with a job and a life</td></tr> <tr><td>Suits</td><td>full-time / automated</td><td>experienced, quick</td><td>anyone with a job and a life</td></tr>
@@ -146,6 +153,7 @@ The message says: <i>"once price hits 4,159, trail SL $33 behind best price."</i
<li><b>H1 is BUY-only.</b> All H1 SELL signals are blocked (since 2026-09-03) because they were a losing side on this timeframe — expect to see BUY signals here, never SELL.</li> <li><b>H1 is BUY-only.</b> All H1 SELL signals are blocked (since 2026-09-03) because they were a losing side on this timeframe — expect to see BUY signals here, never SELL.</li>
<li><b>M5 also holds back some SELL signals</b> — specifically while the market is in a choppy "transitioning" phase, since those shorts tested poorly.</li> <li><b>M5 also holds back some SELL signals</b> — specifically while the market is in a choppy "transitioning" phase, since those shorts tested poorly.</li>
<li><b>M15 got a small tune-up on 2026-09-09</b> — one of its nine indicators (RSI) now counts for half as much, which means a few more M15 signals per week. Nothing changes in how you act on a signal.</li> <li><b>M15 got a small tune-up on 2026-09-09</b> — one of its nine indicators (RSI) now counts for half as much, which means a few more M15 signals per week. Nothing changes in how you act on a signal.</li>
<li><b>M15 now respects the daily trend (since 2026-09-30).</b> A 15-minute trade that goes against a clear daily trend is no longer sent. That removes about one signal in four, the ones that made no money in the tests, and makes the bad stretches shorter. Nothing changes in how you act on the signals you do receive.</li>
<li><b>Never take the same signal on two timeframes.</b> They are usually the same gold move seen at different zoom levels. Taking both doubles your risk, not your profit.</li> <li><b>Never take the same signal on two timeframes.</b> They are usually the same gold move seen at different zoom levels. Taking both doubles your risk, not your profit.</li>
</ul> </ul>
+40 -33
View File
@@ -35,42 +35,48 @@
<body> <body>
<h1>ForexBot2 — User Guide</h1> <h1>ForexBot2 — User Guide</h1>
<div class="subtitle">How to read, act on, and survive the signals &nbsp;·&nbsp; July 2026</div> <div class="subtitle">How to read, act on, and survive the signals &nbsp;·&nbsp; updated October 2026</div>
<h2>1 · What the bot does</h2> <h2>1 · What the bot does</h2>
<p>ForexBot2 analyzes <b>gold (XAU/USD)</b> around the clock on several chart timeframes at once. On every candle close it scores nine technical indicators (trend, momentum, volatility, divergences, candlestick patterns), adapts the weighting to the current market regime (trending / ranging / transitioning), checks that higher timeframes agree, adjusts for news and economic-calendar events, and — only when everything lines up — sends a <b>signal with a complete trade plan</b> to Telegram: direction, entry, stop loss, target, and a trailing-stop management plan.</p> <p>ForexBot2 analyzes <b>gold (XAU/USD)</b> around the clock on several chart timeframes at once. On every candle close it scores nine technical indicators (trend, momentum, volatility, divergences, candlestick patterns), adapts the weighting to the current market regime (trending / ranging / transitioning), checks that higher timeframes agree, adjusts for news and economic-calendar events, and — only when everything lines up — sends a <b>signal with a complete trade plan</b> to Telegram: direction, entry, stop loss, target, and a trailing-stop management plan.</p>
<p>Most of the time it sends nothing. Silence means no edge was found; that is the bot working correctly, not a malfunction.</p> <p>Most of the time it sends nothing. Silence means no edge was found; that is the bot working correctly, not a malfunction.</p>
<h2>2 · Anatomy of a signal</h2> <h2>2 · Anatomy of a signal</h2>
<div class="signal-box">🔍 XAU/USD Signal Analysis (M15) <div class="signal-box">🟢 Grade A · STRONG BUY · XAU/USD · 15min 🥇
🕐 2026-07-10 14:00 UTC 🕐 2026-09-04 08:00 UTC · 💰 4483.41
💰 Close: 4,132.50 📐 Direction: strong buy · Reliability: high · timeframes agree
📊 Regime: Trending Up | Vol: normal 📊 Regime: Transitioning · Vol: high
━━━━━━━━━━━━━━━━━━━━ 🧭 Daily trend: rising — this buy goes with it
🟢 BUY 🅰 A | Score: +38.2 | 64% conf STRONG BUY — High-quality setup, consider entry
✅ MTF aligned
📈 MTF: M5 ✅ H1 ✅
📋 Indicators … (9 rows)
⚠️ EXTREME volatility — wider stops needed
━━━━━━━━━━━━━━━━━━━━
🚀 TRADE SETUP 🚀 TRADE SETUP
📈 Direction: BUY 📈 Direction: BUY
▶ Entry: 4,132.50 ▶ Entry 4483.41
🛑 SL: 4,103.10 ($29.40 / 294 pips = 1R) 🛑 SL 4468.92 ($14.49 / 145p = 1R)
📏 Size: 0.06 lots risks $86.94 at the stop
(your limit $100.00 per trade) ← owner's chats only
🧱 Already open: 2 buy plans alerted earlier (2 on M15)
— taking this one adds to the same position
🔄 Exit rule (tested): once price reaches 🔄 Exit rule (tested): once price reaches
4,150.14 (+0.6R), trail the SL $22.05 4493.55 (+0.7R), trail the SL $10.87
behind the best price, moving it at behind the best price, moving it at
candle close only. Otherwise the SL is the exit. candle close only. Otherwise the SL is the exit.
🎯 TP1 guide 4,161.90 ($29.40 / 294 pips = 1.0R) 🎯 TP1 guide 4497.90 ($14.49 / 145p = 1.0R)
🎯 TP2 guide 4,176.60 (1.5R) 🎯 TP2 guide 4505.15 (1.5R)
⚖️ Planned TP1 RR 1:1.0 (guide only — the trail is the exit) ⚖️ Planned TP1 RR 1:1.0 (guide only — the trail is the exit)
📚 Book (this setup class): 312 trades, 58% winners, avg +0.19R per trade</div> 📚 Book (this setup class): 58 trades, 62% winners, avg +0.21R per trade
📈 Other timeframes: 15min: STRONG BUY +46.5 | 5min: BUY +22.8 | 1h: NEUTRAL +14.2
📋 EMA⇈ RSI→ BB↓ ADX→ MACD⇈ ROC→ KC⇈</div>
<table> <table>
<tr><th style="width:28%">Field</th><th>What it means</th></tr> <tr><th style="width:28%">Field</th><th>What it means</th></tr>
<tr><td><b>Signal + grade</b></td><td>Direction (BUY / SELL / STRONG_*) plus a quality grade <b>A+ → F</b>. A trade setup is only attached at grade C or better. Higher grades = more indicator agreement and cleaner regime.</td></tr> <tr><td><b>Signal + grade</b></td><td>Direction (BUY / SELL / STRONG_*) plus a quality grade <b>A+ → F</b>. A trade setup is only attached at grade C or better. Higher grades = more indicator agreement and cleaner regime.</td></tr>
<tr><td><b>Direction / reliability line</b></td><td>Since 2026-09-27 the alert prints the two raw numbers in words. <i>Direction</i> is the regime-weighted composite of all nine indicators: <b>very strong</b> (|score| ≥ 40), <b>strong</b> (≥ 33), <b>clear</b> (≥ 25), <b>weak</b> (≥ 15) or <b>no clear direction</b>, with buy or sell from its sign. <i>Reliability</i> is the confidence after the penalties: <b>high</b> (≥ 65), <b>good</b> (≥ 55), <b>fair</b> (≥ 45) or <b>low</b>. The cut-offs are the grade table's own, so the words never disagree with the grade. The numbers themselves remain in <code>--json</code>, <code>--detail</code> and the dashboard meters.</td></tr> <tr><td><b>Direction / reliability line</b></td><td>Since 2026-09-27 the alert prints the two raw numbers in words. <i>Direction</i> is the regime-weighted composite of all nine indicators: <b>very strong</b> (|score| ≥ 40), <b>strong</b> (≥ 33), <b>clear</b> (≥ 25), <b>weak</b> (≥ 15) or <b>no clear direction</b>, with buy or sell from its sign. <i>Reliability</i> is the confidence after the penalties: <b>high</b> (≥ 65), <b>good</b> (≥ 55), <b>fair</b> (≥ 45) or <b>low</b>. The cut-offs are the grade table's own, so the words never disagree with the grade. The numbers themselves remain in <code>--json</code>, <code>--detail</code> and the dashboard meters.</td></tr>
<tr><td><b>MTF aligned</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr> <tr><td><b>Timeframes agree</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr>
<tr><td><b>Daily trend 🧭</b></td><td>The direction of the daily chart, from closed daily candles only: <b>rising</b> when the last daily close is above its 50-day average <i>and</i> above the close 60 trading days earlier, <b>falling</b> when it is below both, otherwise <b>no clear direction</b>. The line also says whether the plan goes with it or against it. <b>On M15 a plan against a clear daily trend is not sent at all</b> (since 2026-09-30): in the three-year test those plans earned nothing after costs, and dropping them cut the worst losing run from 20.9R to 12.4R while the total rose from +195.8R to +205.1R; over 2019–2026 the worst run fell from 64.8R to 41.5R. On M5, H1 and H4 the same rule was tested and did not pass, so there the line is context only — a plan "against" the daily trend is still a valid signal on those timeframes.</td></tr>
<tr><td><b>Size 📏</b></td><td>How many lots keep the loss at the printed stop inside the risk limit set in <code>config.json</code> (<code>trade_size.risk_per_trade</code>, in US dollars; 1 lot = 100 ounces unless <code>contract_size</code> says otherwise, and the lot step is 0.01 unless <code>lot_step</code> says otherwise). The size is rounded <b>down</b> to the broker's lot step, so the money at risk is at or under the limit, never over. When even the smallest lot would break the limit the line says so and gives the amount; skip that trade or accept the larger risk knowingly. The line appears only when the limit is set, and only in the owner's own chat with the bot and on the command line: the channel, any group the bot posts in and the public dashboard do not show it unless <code>trade_size.telegram_channel</code> / <code>trade_size.dashboard</code> are switched on.</td></tr>
<tr><td><b>Already open 🧱</b></td><td>How many plans in the <i>same direction</i> the bot alerted earlier are still open in its own outcome tracking (stop not hit, trail not closed), and on which timeframes. Taking every alert stacks positions on one move of gold: in the three-year M5 + M15 test two entries in three already had a same-direction plan open and one in four had three or more. Those stacked entries were not worse trades, so the line is not a reason to skip; it is a reminder that the total at risk is the sum of every open plan, not the risk of this one alert. The count comes from the bot's model of the plans it sent, not from your broker account. Alerts sent at the same candle close are counted too: when the 5-minute and the 15-minute alert go out together, the second one already shows the first.</td></tr>
<tr><td><b>Warnings ⚠️</b></td><td>Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings.</td></tr> <tr><td><b>Warnings ⚠️</b></td><td>Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings.</td></tr>
<tr><td><b>Violent-move watch 🌪</b></td><td>An occasional extra line saying the next hour looks unusually likely to produce a very large candle — because a high-impact US release is due (or just landed), because the clock is in a historically wild slot, or because price is stretched outside its bands. It is a heads-up about <i>volatility</i>, not a direction call, and it <b>never</b> changes the signal, the grade or the trade setup — the same alert would be sent without it. Do not close or tighten an open trade because of it: every such protective action was tested over three years and lost money. Your stop is the protection.</td></tr> <tr><td><b>Violent-move watch 🌪</b></td><td>An occasional extra line saying the next hour looks unusually likely to produce a very large candle — because a high-impact US release is due (or just landed), because the clock is in a historically wild slot, or because price is stretched outside its bands. It is a heads-up about <i>volatility</i>, not a direction call, and it <b>never</b> changes the signal, the grade or the trade setup — the same alert would be sent without it. Do not close or tighten an open trade because of it: every such protective action was tested over three years and lost money. Your stop is the protection.</td></tr>
<tr><td><b>AI check 🤖</b></td><td>An advisory block written for a new trader, such as <i>"AI check: the odds look worse than usual. New data just came out against this buy. Chance it pays off: about 45 in 100 (usual for this kind of setup: 60 in 100). For: Before the news, the price was climbing. Against: Fresh data just came out against gold. This check is an experiment. Do not trade on it alone."</i> The first sentence compares the estimate with the usual odds for this kind of setup (8 points either way counts as a difference); there is no verdict word and no confidence percentage. One AI reviewer (the news reader) receives the recent headlines together with facts computed by code — how far the move has already run, whether a very large candle just printed, how the stop and target compare with the usual, how trades of this exact kind fared in the two-year test, and whether major scheduled news lands before the trade would normally finish — and gives a probability that the trade ends in profit plus one reason for and one against. (Three further reviewers, chart, risk and sceptic, were switched off on 2026-09-17: in two evaluations they added nothing beyond the base rate and mostly voted the same way on every setup.) It is <b>not</b> a recommendation and it <b>never</b> changes the signal, the grade or the setup. Until about 300 of these estimates have been graded against real outcomes, treat it as context, not as a filter: the earlier AGREE/OPPOSE version of this line opposed target-reaching trades as often as it agreed.</td></tr> <tr><td><b>AI check 🤖</b></td><td>An advisory block written for a new trader, such as <i>"AI check: the odds look worse than usual. New data just came out against this buy. Chance it pays off: about 45 in 100 (usual for this kind of setup: 60 in 100). For: Before the news, the price was climbing. Against: Fresh data just came out against gold. This check is an experiment. Do not trade on it alone."</i> The first sentence compares the estimate with the usual odds for this kind of setup (8 points either way counts as a difference); there is no verdict word and no confidence percentage. One AI reviewer (the news reader) receives the recent headlines together with facts computed by code — how far the move has already run, whether a very large candle just printed, how the stop and target compare with the usual, how trades of this exact kind fared in the two-year test, and whether major scheduled news lands before the trade would normally finish — and gives a probability that the trade ends in profit plus one reason for and one against. (Three further reviewers, chart, risk and sceptic, were switched off on 2026-09-17: in two evaluations they added nothing beyond the base rate and mostly voted the same way on every setup.) It is <b>not</b> a recommendation and it <b>never</b> changes the signal, the grade or the setup. Until about 300 of these estimates have been graded against real outcomes, treat it as context, not as a filter: the earlier AGREE/OPPOSE version of this line opposed target-reaching trades as often as it agreed.</td></tr>
@@ -78,33 +84,34 @@
<tr><td><b>SL / TP1 guide / planned RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer (grade A+/A setups use a 2 × ATR stop). <b>1R</b> = the entry-to-SL distance = the amount you risk. TP1 is a <i>guide</i> (the nearest opposing level or a multiple of ATR, never printed closer than 1R since 2026-09-26); the "planned TP1 RR" is TP1 ÷ 1R and is always at least 1:1. It is <b>not</b> the expected payoff: the tested exit is the trail, and the validated books earn their profit from trailed runners (M5 average win about +1.0R against an average loss of −0.76R; M15 +0.87R against −0.93R).</td></tr> <tr><td><b>SL / TP1 guide / planned RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer (grade A+/A setups use a 2 × ATR stop). <b>1R</b> = the entry-to-SL distance = the amount you risk. TP1 is a <i>guide</i> (the nearest opposing level or a multiple of ATR, never printed closer than 1R since 2026-09-26); the "planned TP1 RR" is TP1 ÷ 1R and is always at least 1:1. It is <b>not</b> the expected payoff: the tested exit is the trail, and the validated books earn their profit from trailed runners (M5 average win about +1.0R against an average loss of −0.76R; M15 +0.87R against −0.93R).</td></tr>
<tr><td><b>Exit rule 🔄</b></td><td>The real, backtested exit. Once price moves the stated R in your favor (0.6R on M5, 0.7R on M15), move the stop to trail the stated $ distance behind the best price and update it at each candle close only. Until then the original SL is the exit. <b>TP1 is advisory — let winners run.</b></td></tr> <tr><td><b>Exit rule 🔄</b></td><td>The real, backtested exit. Once price moves the stated R in your favor (0.6R on M5, 0.7R on M15), move the stop to trail the stated $ distance behind the best price and update it at each candle close only. Until then the original SL is the exit. <b>TP1 is advisory — let winners run.</b></td></tr>
<tr><td><b>Book line 📚</b></td><td>What this class of setup (timeframe, setup type, grade, direction) actually did in the validated two-year backtests: number of trades, share of winners and average R per trade. When the exact class has fewer than 30 trades the line falls back to all setups of that direction on the timeframe, then to the whole timeframe.</td></tr> <tr><td><b>Book line 📚</b></td><td>What this class of setup (timeframe, setup type, grade, direction) actually did in the validated two-year backtests: number of trades, share of winners and average R per trade. When the exact class has fewer than 30 trades the line falls back to all setups of that direction on the timeframe, then to the whole timeframe.</td></tr>
<tr><td><b>One alert per move</b></td><td>After an alert, the bot sends no new alert on that timeframe for the next 5 candles (25 min on M5, 75 min on M15), the same entry cooldown the backtester applies. Consecutive-bar re-entries into the same move were the largest live-only loss in September 2026.</td></tr> <tr><td><b>One alert per move</b></td><td>After an alert, the bot sends no new alert on that timeframe for the next 5 candles (25 min on M5, 75 min on M15), the same entry cooldown the backtester applies. Consecutive-bar re-entries into the same move were the largest live-only loss in September 2026. The five candles are candles that actually traded: the daily one-hour pause and the weekend do not use them up, so the first candle after the pause is still held if fewer than five have printed since the alert. The bot also remembers its last alert across a restart.</td></tr>
</table> </table>
<h2>3 · Acting on a signal — step by step</h2> <h2>3 · Acting on a signal — step by step</h2>
<ol> <ol>
<li><b>Size the position first.</b> Decide your risk per trade — <b>0.5–1% of account, never more</b>. Position size = (account × risk%) ÷ stop distance in $. The stop distance varies per signal, so the size changes every trade; the dollar risk must not.</li> <li><b>Size the position first.</b> Decide your risk per trade — <b>0.5–1% of account, never more</b>. Position size = (account × risk%) ÷ stop distance in $. The stop distance varies per signal, so the size changes every trade; the dollar risk must not. With <code>trade_size.risk_per_trade</code> set in <code>config.json</code> the alert does this sum for you (the 📏 line). If the 🧱 line shows plans already open in the same direction, the new trade adds to them: keep the <i>total</i> at risk across all open plans inside what you can afford on one move.</li>
<li><b>Enter promptly at market.</b> All backtested results assume entry at the open of the candle after the signal. On M5, minutes matter (see §4). If you are very late — price already moved a large part of the way to the arm level — skip the trade. There is always another signal.</li> <li><b>Enter promptly at market.</b> All backtested results assume entry at the open of the candle after the signal. On M5, minutes matter (see §4). If you are very late — price already moved a large part of the way to the arm level — skip the trade. There is always another signal.</li>
<li><b>Place the stop loss immediately</b> at the given SL. No mental stops — gold gaps and spikes.</li> <li><b>Place the stop loss immediately</b> at the given SL. No mental stops — gold gaps and spikes.</li>
<li><b>Manage with the trail, not TP1.</b> When price reaches the arm level printed in the message, start trailing the stop the stated $ distance behind the best price, tightening only at candle close. This is what produced the backtested results; taking fixed TP1 profits gives most of the edge away.</li> <li><b>Manage with the trail, not TP1.</b> When price reaches the arm level printed in the message, start trailing the stop the stated $ distance behind the best price, tightening only at candle close. This is what produced the backtested results; taking fixed TP1 profits gives most of the edge away.</li>
<li><b>One signal, one trade.</b> The M5, M15 and H1 books confirm off each other and are usually long or short gold at the same time. Taking the same move on multiple timeframes multiplies risk, not diversification. Pick your timeframe (§4) and ignore the others.</li> <li><b>One signal, one trade.</b> The M5, M15 and H1 books confirm off each other and are usually long or short gold at the same time. Taking the same move on multiple timeframes multiplies risk, not diversification. Pick your timeframe (§4) and ignore the others. The 🧱 "already open" line tells you when an alert would add to plans that are still running.</li>
</ol> </ol>
<h2>4 · Choosing your timeframe</h2> <h2>4 · Choosing your timeframe</h2>
<p>Twelve-month Oanda-candle backtest (Aug 2025 – Aug 2026, $0.65 round-trip cost, entry at next candle open):</p> <p>Twelve-month Oanda-candle backtest with the rules as they run today (Aug 2025 – Aug 2026, $0.65 round-trip cost, entry at next candle open; regenerated 2026-10-01):</p>
<table> <table>
<tr><th></th><th>M5</th><th>M15</th><th>H1</th></tr> <tr><th></th><th>M5</th><th>M15</th><th>H1</th></tr>
<tr><td>Trades / week</td><td>~11.3</td><td>~7.6</td><td>~2.7</td></tr> <tr><td>Trades / week</td><td>~10.7</td><td>~5.1</td><td>~1.7</td></tr>
<tr><td>Total result (12 mo)</td><td>+106.8R</td><td>+73.9R</td><td>+24.0R</td></tr> <tr><td>Total result (12 mo)</td><td>+108.8R</td><td>+92.3R</td><td>+22.2R</td></tr>
<tr><td>Win rate / profit factor</td><td>52.9% / 1.51</td><td>62.2% / 1.53</td><td>60.9% / 1.55</td></tr> <tr><td>Win rate / profit factor</td><td>53.6% / 1.54</td><td>68.0% / 2.12</td><td>62.6% / 1.79</td></tr>
<tr><td>Typical (median) hold</td><td>2.4 h</td><td>7 h</td><td>15.5 h</td></tr> <tr><td>Typical (median) hold</td><td>2.5 h</td><td>7.8 h</td><td>21 h</td></tr>
<tr><td>90% of trades close within</td><td>10.5 h</td><td>29.8 h</td><td>3.2 days</td></tr> <tr><td>90% of trades close within</td><td>11.2 h</td><td>29.4 h</td><td>3.2 days</td></tr>
<tr><td>Typical stop distance</td><td>~$21</td><td>~$32</td><td>~$52</td></tr> <tr><td>Typical stop distance</td><td>~$21</td><td>~$32</td><td>~$48</td></tr>
<tr><td>Worst drawdown</td><td>10.7R</td><td>10.8R</td><td>8.0R</td></tr> <tr><td>Worst drawdown</td><td>10.1R</td><td>8.3R</td><td>4.1R</td></tr>
</table> </table>
<p class="small">This was a strongly rising year for gold until March 2026 and a falling, choppy one after. Four fifths of the M5 total (+89.6R of +108.8R) came in January–March 2026; M15 earned in every quarter but only +2.3R since July. Over 2019–2026 the M15 rules made +223R with one losing year (2021, −27R) and two years close to flat (2022 and 2023); the M5 rules took few trades and made nothing before 2025. Judge the next twelve months by the long record, not by this table.</p>
<ul> <ul>
<li><b>M5 — intraday, execution-critical.</b> Highest total edge, but stops are tight: a couple of dollars of slippage or a late entry eats a meaningful share of every trade. Suitable only if you can act within a minute or two of the alert (or automate). Trades open and close within a session; you sleep flat. SELL signals are held back while the market regime reads "transitioning" — a validated filter, not a bug if you notice fewer shorts there.</li> <li><b>M5 — intraday, execution-critical.</b> Highest total edge, but stops are tight: a couple of dollars of slippage or a late entry eats a meaningful share of every trade. Suitable only if you can act within a minute or two of the alert (or automate). Trades open and close within a session; you sleep flat. SELL signals are held back while the market regime reads "transitioning" — a validated filter, not a bug if you notice fewer shorts there.</li>
<li><b>M15 — the middle path.</b> Same frequency as M5, wider stops, up to 15 minutes to act. About half your trades will still be open overnight. Since 2026-09-09 the M15 scoring gives its RSI reading half the weight it used to (RSI mostly voted against the trend on this timeframe), so expect roughly 9 signals a week instead of 7–8; in the two-year test (Sep 2024 – Sep 2026) that change lifted M15 from +117.6R to +146.1R with the same win rate and a slightly better profit factor, and it also improved the separate out-of-sample period. The M15 column above predates it.</li> <li><b>M15 — the middle path.</b> Same frequency as M5, wider stops, up to 15 minutes to act. About half your trades will still be open overnight. Since 2026-09-09 the M15 scoring gives its RSI reading half the weight it used to (RSI mostly voted against the trend on this timeframe), so expect roughly 9 signals a week instead of 7–8; in the two-year test (Sep 2024 – Sep 2026) that change lifted M15 from +117.6R to +146.1R with the same win rate and a slightly better profit factor, and it also improved the separate out-of-sample period. Since 2026-09-26 grade-A shorts are blocked on M15, and since 2026-09-30 any M15 plan against a clear daily trend is dropped (the 🧭 line, §2). Together these leave about 5 signals a week. The daily-trend rule is a trade of profit for safety: it removes about a quarter of the trades, the ones that earned nothing after costs, and shortens the losing runs; in a stretch where gold only rises it gives up a little (Sep 2024 – Sep 2025: +61.8R instead of +70.1R).</li>
<li><b>H1 — swing trading around a normal life.</b> Best quality per trade and very forgiving of slow manual execution (you have up to an hour). The costs: positions held for days, through news and overnight sessions, and a <b>lumpy</b> equity curve — one quarter of the test year was negative and most of the profit came in a single quarter. Judge it on quarters, not weeks. H1 no longer issues SELL signals at all (blocked since 2026-09-03) — it is BUY-only on this timeframe.</li> <li><b>H1 — swing trading around a normal life.</b> Best quality per trade and very forgiving of slow manual execution (you have up to an hour). The costs: positions held for days, through news and overnight sessions, and a <b>lumpy</b> equity curve — one quarter of the test year was negative and most of the profit came in a single quarter. Judge it on quarters, not weeks. H1 no longer issues SELL signals at all (blocked since 2026-09-03) — it is BUY-only on this timeframe.</li>
</ul> </ul>
<div class="callout"><b>If you trade manually from the Telegram alerts, H1 (or M15) is the honest choice.</b> M5's backtested edge assumes near-instant fills that a human copying signals rarely achieves.</div> <div class="callout"><b>If you trade manually from the Telegram alerts, H1 (or M15) is the honest choice.</b> M5's backtested edge assumes near-instant fills that a human copying signals rarely achieves.</div>
@@ -113,7 +120,7 @@
<ul> <ul>
<li><b>You will lose roughly 38–47% of trades.</b> A 53–62% win rate means losing streaks of 5+ are routine over a year. The system's edge is the average across hundreds of trades, not any single one.</li> <li><b>You will lose roughly 38–47% of trades.</b> A 53–62% win rate means losing streaks of 5+ are routine over a year. The system's edge is the average across hundreds of trades, not any single one.</li>
<li><b>Stops hit fast, winners take time.</b> Losing trades die in roughly half the time winners take. An early stop-out is the system doing its job.</li> <li><b>Stops hit fast, winners take time.</b> Losing trades die in roughly half the time winners take. An early stop-out is the system doing its job.</li>
<li><b>Drawdowns of ~10R are normal.</b> At 1% risk per trade, expect to be ~10% down from a peak at some point in a normal year. This is why risking more than 1% per trade is how accounts die.</li> <li><b>Drawdowns of ~10R are normal, and worse has happened.</b> At 1% risk per trade, expect to be ~10% down from a peak at some point in a normal year. Over 2019–2026 the M15 rules as they run today had one losing run of 41.5R that took from late 2020 to early 2022 to bottom out, while gold drifted sideways and down. This is why risking more than 1% per trade is how accounts die.</li>
<li><b>Backtest ≠ live.</b> Results above charge real spreads and realistic fills, but cannot model your personal latency, broker slippage, or requotes. Assume live results will be somewhat worse — especially on M5.</li> <li><b>Backtest ≠ live.</b> Results above charge real spreads and realistic fills, but cannot model your personal latency, broker slippage, or requotes. Assume live results will be somewhat worse — especially on M5.</li>
<li><b>Quiet periods are normal.</b> Filters block bad hours, bad regimes and misaligned timeframes. Days can pass without a signal.</li> <li><b>Quiet periods are normal.</b> Filters block bad hours, bad regimes and misaligned timeframes. Days can pass without a signal.</li>
</ul> </ul>
@@ -136,7 +143,7 @@
<tr><td><code>/start</code></td><td>Show the button menu (manual scans per timeframe, Scan All TFs).</td></tr> <tr><td><code>/start</code></td><td>Show the button menu (manual scans per timeframe, Scan All TFs).</td></tr>
<tr><td><code>/scan</code></td><td>Run an immediate analysis of the primary timeframe.</td></tr> <tr><td><code>/scan</code></td><td>Run an immediate analysis of the primary timeframe.</td></tr>
<tr><td><code>/autoscan</code></td><td>Toggle automatic scanning (M5 / M15 / H1 / H4 at each candle close).</td></tr> <tr><td><code>/autoscan</code></td><td>Toggle automatic scanning (M5 / M15 / H1 / H4 at each candle close).</td></tr>
<tr><td><code>/status</code></td><td>Uptime, auto-scan state, WebSocket price-monitor state.</td></tr> <tr><td><code>/status</code></td><td>Uptime, auto-scan state, WebSocket price-monitor state, and the health of the news sources: the calendar, the headline feeds, the local headline model (a ⚠️ line "News model: not scoring" means this cycle's headlines carry the simple keyword scores) and DeepSeek.</td></tr>
<tr><td><code>/news</code></td><td>Net XAU/USD news score from the last 4 hours of headlines (neutral headlines are removed before averaging; informational only — never gates signals).</td></tr> <tr><td><code>/news</code></td><td>Net XAU/USD news score from the last 4 hours of headlines (neutral headlines are removed before averaging; informational only — never gates signals).</td></tr>
<tr><td><code>/cache</code> · <code>/clearcache</code></td><td>Inspect / reset the market-data cache.</td></tr> <tr><td><code>/cache</code> · <code>/clearcache</code></td><td>Inspect / reset the market-data cache.</td></tr>
<tr><td><code>/help</code></td><td>Command reference.</td></tr> <tr><td><code>/help</code></td><td>Command reference.</td></tr>
+160
View File
@@ -0,0 +1,160 @@
"""
ForexBot2 — trade size and open-exposure lines (display only)
==============================================================
Two lines a surface prints next to a trade setup. Nothing here is read by
the signal core, a gate or the backtester:
* size line — how many lots keep the loss at the stop inside the risk the
owner set in config.json (`trade_size.risk_per_trade`). Off until that
is set. The size reveals the owner's risk per trade, so it goes to the
owner's own chat (telegram.chat_id) and the CLI only, unless
`trade_size.telegram_channel` (every other Telegram chat: the channel,
a group) / `trade_size.dashboard` switch it on for the public surfaces.
* open line — how many ALERTED plans in the same direction are still open
in the advisory outcome model (logs/signals.db, read-only). Taking every
alert stacks positions: on the 3-year M5 + M15 books two entries in
three already have a same-direction plan open, one in four has three or
more, and up to 13 are open at once
(backtest/research/daily_trend_campaign_2026-09-30.md, section 5.11).
Those stacked entries did not pay worse; the point of the line is that
the full risk on each alert is several times that risk on one move.
Attached post-hoc by the surface (Telegram bot, CLI, dashboard), the same
pattern as violent_watch.attach_watch. Never raises: a display line must
not break a scan or an alert.
"""
import logging
from decimal import ROUND_FLOOR, Decimal, InvalidOperation
from typing import Any, Dict, List, Optional, Tuple
import app_config
import signal_log
logger = logging.getLogger(__name__)
# Surfaces a size line may be attached for, and the config flag that
# shares it with a public audience (None = the owner's own surface).
SURFACES = {"telegram": None, "cli": None, "dashboard": "dashboard"}
_TF_LABEL = {"5min": "M5", "15min": "M15", "1h": "H1", "4h": "H4"}
def lot_size(stop_distance: float, risk: float, contract_size: float = 100.0,
lot_step: float = 0.01) -> Tuple[float, float]:
"""(lots, money at risk with those lots): the largest multiple of
`lot_step` whose loss at the stop stays within `risk`.
One lot is `contract_size` ounces, so a $1 move is worth
`contract_size` dollars per lot. Decimal arithmetic with a floor, so
the size never rounds UP past the risk limit and 0.07 never prints as
0.0699. (0.0, 0.0) when any input is unusable."""
try:
d, r = Decimal(str(stop_distance)), Decimal(str(risk))
c, s = Decimal(str(contract_size)), Decimal(str(lot_step))
except (InvalidOperation, ValueError):
return 0.0, 0.0
if not (d.is_finite() and r.is_finite() and c.is_finite() and s.is_finite()):
return 0.0, 0.0
if d <= 0 or r <= 0 or c <= 0 or s <= 0:
return 0.0, 0.0
steps = (r / (d * c * s)).to_integral_value(rounding=ROUND_FLOOR)
lots = steps * s
return float(lots), float(lots * d * c)
def _decimals(step: float) -> int:
text = format(Decimal(str(step)).normalize(), "f")
return len(text.split(".")[1]) if "." in text else 0
def size_line(setup: Any, settings: Optional[Dict[str, Any]]) -> str:
""""Size: 0.05 lots risks $99.00 at the stop (your limit $100.00 per
trade)", or "" when the size is not configured or the setup has no
usable stop.
The stop distance is taken from the prices as they are printed (two
decimals, subtracted exactly), so the line agrees with the entry and
stop above it and a $20.00 stop is never read as $20.000000000000004."""
if setup is None or not settings:
return ""
try:
risk = float(settings.get("risk_per_trade") or 0.0)
contract = float(settings.get("contract_size", 100.0))
step = float(settings.get("lot_step", 0.01))
distance = abs(Decimal(f"{float(setup.entry_price):.2f}")
- Decimal(f"{float(setup.stop_loss):.2f}"))
except (TypeError, ValueError, InvalidOperation):
return ""
if not risk > 0 or not distance.is_finite() or not distance > 0:
return ""
lots, at_risk = lot_size(distance, risk, contract, step)
places = _decimals(step)
if lots <= 0:
smallest = float(distance) * contract * step
if not smallest > 0:
return ""
return (f"Size: the stop is too wide for your ${risk:,.2f} limit — the smallest "
f"size ({step:.{places}f} lots) would risk ${smallest:,.2f}")
return (f"Size: {lots:.{places}f} lots risks ${at_risk:,.2f} at the stop "
f"(your limit ${risk:,.2f} per trade)")
def open_line(rows: Optional[List[Dict[str, Any]]], direction: str) -> str:
""""Already open: 2 buy plans alerted earlier (M5, M15) — taking this
one adds to the same position", or "" when no alerted plan in
`direction` is still open."""
same = [r for r in (rows or []) if r.get("direction") == direction]
if not same:
return ""
counts: Dict[str, int] = {}
for r in same:
label = _TF_LABEL.get(str(r.get("timeframe")), str(r.get("timeframe")))
counts[label] = counts.get(label, 0) + 1
where = ", ".join(f"{n} on {tf}" if n > 1 else tf for tf, n in counts.items())
n = len(same)
return (f"Already open: {n} {direction.lower()} plan{'s' if n != 1 else ''} alerted "
f"earlier ({where}) — taking this one adds to the same position")
def size_shared(flag: str) -> bool:
"""True when config.json shares the size line with a public audience
(`flag` = "telegram_channel" or "dashboard"). Never raises."""
try:
settings = app_config.trade_size_settings()
return bool(settings and settings.get(flag))
except Exception:
return False
def attach_exposure(report: Any, surface: str = "telegram") -> Any:
"""Stamp report.size_line and report.open_line for `surface`
("telegram" / "cli": the owner's own view; "dashboard": the size only
when trade_size.dashboard is true). Display only; never raises."""
report.size_line = ""
report.open_line = ""
setup = getattr(report, "trade_setup", None)
if setup is None:
return report
try:
settings = app_config.trade_size_settings()
flag = SURFACES.get(surface, "unknown-surface")
if settings and (flag is None or settings.get(flag)):
report.size_line = size_line(setup, settings)
except Exception:
logger.debug("exposure: size line unavailable", exc_info=True)
try:
# The same identity the logger stores for this scan's row, so the
# plan being alerted now never counts itself (a re-alert of an
# unchanged plan is the same position, not a second one).
analysis = getattr(report, "primary_analysis", None)
key = signal_log.setup_key(
{"direction": setup.direction, "entry": setup.entry_price,
"stop_loss": setup.stop_loss,
"strategy": getattr(setup, "strategy", None)},
analysis.timeframe if analysis is not None else "")
rows = signal_log.open_alerted_setups(report.symbol, exclude_key=key)
report.open_line = open_line(rows, setup.direction)
except Exception:
logger.debug("exposure: open line unavailable", exc_info=True)
return report
+55 -8
View File
@@ -53,7 +53,10 @@ class DeepSeekClient:
daily_cap: int = DEFAULT_DAILY_CAP, daily_cap: int = DEFAULT_DAILY_CAP,
enabled: bool = True, enabled: bool = True,
): ):
self._api_key = api_key or "" # Whitespace around a key is never part of it, and a trailing line
# break (a key pasted from a file) is not a legal header value:
# every call would fail before it left the process.
self._api_key = (api_key or "").strip()
self._model = model self._model = model
self._timeout = timeout self._timeout = timeout
self._daily_cap = daily_cap self._daily_cap = daily_cap
@@ -104,17 +107,36 @@ class DeepSeekClient:
self._count += 1 self._count += 1
return True return True
def _redact(self, text: Any) -> str:
"""Exception text that is safe to log: never contains the key."""
out = str(text)
return out.replace(self._api_key, "<api key>") if self._api_key else out
def chat_json(self, system_prompt: str, def chat_json(self, system_prompt: str,
user_payload: Dict[str, Any]) -> Optional[Dict[str, Any]]: user_payload: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""One chat call; returns the model's JSON object or None.""" """One chat call; returns the model's JSON object or None.
if not self.available or not self._under_cap():
Never raises: every caller reads None as "use the fallback", and
the news enrichment runs inside a live scan."""
if not self.available:
return None
try:
content = json.dumps(user_payload, sort_keys=True)
except (TypeError, ValueError) as e:
# Before the cap is counted: a payload that cannot be sent
# must not spend one of the day's requests.
self.last_error = (time.time(), "payload")
self._warn_once(
"payload", "DeepSeek payload is not JSON-serialisable "
"(%s) — call skipped", type(e).__name__)
return None
if not self._under_cap():
return None return None
body = { body = {
"model": self._model, "model": self._model,
"messages": [ "messages": [
{"role": "system", "content": system_prompt}, {"role": "system", "content": system_prompt},
{"role": "user", {"role": "user", "content": content},
"content": json.dumps(user_payload, sort_keys=True)},
], ],
"response_format": {"type": "json_object"}, "response_format": {"type": "json_object"},
"temperature": 0, "temperature": 0,
@@ -133,19 +155,44 @@ class DeepSeekClient:
try: try:
resp = self._session.post( resp = self._session.post(
DEEPSEEK_URL, json=body, timeout=self._timeout) DEEPSEEK_URL, json=body, timeout=self._timeout)
except requests.exceptions.InvalidHeader:
# requests quotes the offending header VALUE in this error, and
# the only header built from configuration is Authorization:
# the text would put the API key in the log. Fixed wording.
self.last_error = (time.time(), "header")
self._warn_once(
"header", "DeepSeek call not sent: the API key holds a "
"character that is not allowed in an HTTP header (a line "
"break?) — check deepseek.api_key / DEEPSEEK_API_KEY")
return None
except requests.RequestException as e: except requests.RequestException as e:
self.last_error = (time.time(), "network") self.last_error = (time.time(), "network")
self._warn_once("network", "DeepSeek call failed: %s", e) self._warn_once("network", "DeepSeek call failed: %s",
self._redact(e))
return None
except Exception as e:
# Anything else raised inside the HTTP stack (a header that
# cannot be encoded, a transport bug) must not reach a scan
# either. Type name only: the text may quote the request.
self.last_error = (time.time(), "client")
self._warn_once("client", "DeepSeek call failed inside the "
"HTTP client: %s", type(e).__name__)
return None return None
if resp.status_code != 200: if resp.status_code != 200:
self.last_error = (time.time(), f"http_{resp.status_code}") self.last_error = (time.time(), f"http_{resp.status_code}")
self._warn_once( self._warn_once(
f"http_{resp.status_code}", f"http_{resp.status_code}",
"DeepSeek HTTP %s: %s", resp.status_code, resp.text[:200]) "DeepSeek HTTP %s: %s", resp.status_code,
# Redact BEFORE cutting: a gateway error page can echo the
# request headers, and a key cut at the 200-character
# mark would be left partly visible.
self._redact(resp.text)[:200])
return None return None
try: try:
parsed = self._parse_response(resp.json()) parsed = self._parse_response(resp.json())
except ValueError: except Exception:
# ValueError for a body that is not JSON; anything else (a
# pathological nesting depth) is just as unusable.
parsed = None parsed = None
if parsed is None: if parsed is None:
self.last_error = (time.time(), "parse") self.last_error = (time.time(), "parse")
+36
View File
@@ -23,6 +23,7 @@ if hasattr(sys.stderr, "reconfigure"):
from agent_panel import attach_panel from agent_panel import attach_panel
from app_config import get_app_config, violent_watch_enabled from app_config import get_app_config, violent_watch_enabled
from config import PlatformConfig, Timeframe, SignalStrength, XAUUSD from config import PlatformConfig, Timeframe, SignalStrength, XAUUSD
from exposure import attach_exposure
from pair_signals import PAIR_PRODUCERS from pair_signals import PAIR_PRODUCERS
from signal_aggregator import SignalAggregator, SignalReport from signal_aggregator import SignalAggregator, SignalReport
from violent_watch import attach_watch from violent_watch import attach_watch
@@ -143,6 +144,25 @@ def plain_words(report: SignalReport) -> str:
f"Reliability: {reliability_words(report.final_confidence)} · {agree}") f"Reliability: {reliability_words(report.final_confidence)} · {agree}")
def daily_trend_words(report: SignalReport) -> str:
"""The daily trend in words, and whether the trade plan goes with it
("Daily trend: rising — this buy goes with it"). "" when the scan
carries no daily trend. Display only (2026-09-30): on M15 the signal
core already drops a plan against a clear daily trend; on the other
timeframes the line is information for the reader."""
trend = getattr(report, "daily_trend", None)
if trend is None:
return ""
if trend == 0:
return "Daily trend: no clear direction"
word = "rising" if trend > 0 else "falling"
setup = report.trade_setup
if setup is None:
return f"Daily trend: {word}"
side = "with" if (setup.direction == "BUY") == (trend > 0) else "against"
return f"Daily trend: {word} — this {setup.direction.lower()} goes {side} it"
def print_signal_report(report: SignalReport, detail: bool = False): def print_signal_report(report: SignalReport, detail: bool = False):
sc = signal_color(report.final_signal) sc = signal_color(report.final_signal)
qc = quality_color(report.signal_quality) qc = quality_color(report.signal_quality)
@@ -156,6 +176,9 @@ def print_signal_report(report: SignalReport, detail: bool = False):
f"{sc}{report.final_score:+.1f}{Colors.RESET} · " f"{sc}{report.final_score:+.1f}{Colors.RESET} · "
f"{report.final_confidence:.0f}% · {mtf_tag}") f"{report.final_confidence:.0f}% · {mtf_tag}")
print(f" {plain_words(report)}") print(f" {plain_words(report)}")
trend_words = daily_trend_words(report)
if trend_words:
print(f" {trend_words}")
print(f" {Colors.GRAY}{report.action_recommendation}{Colors.RESET}") print(f" {Colors.GRAY}{report.action_recommendation}{Colors.RESET}")
# ── 2. TRADE SETUP (if signal is actionable) ─────────────────── # ── 2. TRADE SETUP (if signal is actionable) ───────────────────
@@ -182,6 +205,11 @@ def print_signal_report(report: SignalReport, detail: bool = False):
f"{_fmt_price(arm_price)} (+{arm_r:.1f}R), trail the SL " f"{_fmt_price(arm_price)} (+{arm_r:.1f}R), trail the SL "
f"${dist_r * risk_dist:.2f} behind the best price at candle close; " f"${dist_r * risk_dist:.2f} behind the best price at candle close; "
f"TP1 is a guide only.{Colors.RESET}") f"TP1 is a guide only.{Colors.RESET}")
# Lot size for the owner's risk limit and the alerted plans still
# open in this direction (exposure.attach_exposure; display only).
for line in (report.size_line, report.open_line):
if line:
print(f" {Colors.BOLD}{line}{Colors.RESET}")
# ── 3. CONTEXT (price · regime · vol · MTF breakdown) ────────── # ── 3. CONTEXT (price · regime · vol · MTF breakdown) ──────────
print(f"\n {Colors.WHITE}{report.display_name}{Colors.RESET} " print(f"\n {Colors.WHITE}{report.display_name}{Colors.RESET} "
@@ -304,6 +332,10 @@ def report_to_dict(report: SignalReport) -> dict:
"panel_probability": round(report.panel_probability, 1), "panel_probability": round(report.panel_probability, 1),
"panel_context": report.panel_context, "panel_context": report.panel_context,
"watch_line": report.watch_line, "watch_line": report.watch_line,
"daily_trend": report.daily_trend,
"daily_trend_words": daily_trend_words(report),
"size_line": report.size_line,
"open_line": report.open_line,
"bullish_reasons": report.bullish_reasons, "bullish_reasons": report.bullish_reasons,
"bearish_reasons": report.bearish_reasons, "bearish_reasons": report.bearish_reasons,
"warnings": report.warnings, "warnings": report.warnings,
@@ -422,6 +454,10 @@ def run_live(args, config: PlatformConfig):
attach_watch(report, config.primary_timeframe, attach_watch(report, config.primary_timeframe,
get_app_config().violent_watch_settings) get_app_config().violent_watch_settings)
# Display-only lot size (off until config.json sets trade_size) and
# the alerted plans still open in this direction.
attach_exposure(report, surface="cli")
if args.json: if args.json:
payload = report_to_dict(report) payload = report_to_dict(report)
print(json.dumps(payload, indent=2)) print(json.dumps(payload, indent=2))
+74 -7
View File
@@ -384,7 +384,7 @@ class NewsEngine:
# writer runs inside _cached_fetch); the other two take the lock # writer runs inside _cached_fetch); the other two take the lock
# around cache access but keep HTTP calls outside it. # around cache access but keep HTTP calls outside it.
_llm_sent_cache: Dict[str, Tuple[float, float]] = {} # sha1 → (s, xau) _llm_sent_cache: Dict[str, Tuple[float, float]] = {} # sha1 → (s, xau)
_llm_event_cache: Dict[Tuple[str, str, str], str] = {} _llm_event_cache: Dict[Tuple[str, str, str, str, str], str] = {}
_llm_summary_cache: Tuple[float, str] = (0.0, "") _llm_summary_cache: Tuple[float, str] = (0.0, "")
_LLM_SENT_CACHE_MAX = 400 _LLM_SENT_CACHE_MAX = 400
_LLM_EVENT_CACHE_MAX = 100 _LLM_EVENT_CACHE_MAX = 100
@@ -609,6 +609,23 @@ class NewsEngine:
key=lambda e: e.time, key=lambda e: e.time,
) )
def peek_week_events(
self, currencies: Optional[List[str]] = None
) -> List[EconomicEvent]:
"""get_week_events WITHOUT the refresh: whatever calendar the
class-level cache holds right now, however old, filtered and sorted
the same way. Never fetches and never waits on the cache lock, for
a caller that must not add a network wait: the agent panel runs
while an alert is waiting to go out, and a refetch with both
calendar sources down takes about 40 seconds."""
if currencies is None:
currencies = self.config.tracked_currencies
_stamp, events = NewsEngine._calendar_cache
return sorted(
(e for e in events if e.currency in currencies),
key=lambda e: e.time,
)
# ── Private Methods ──────────────────────────────────────────── # ── Private Methods ────────────────────────────────────────────
# Maps country names in MyFXBook titles to currency codes # Maps country names in MyFXBook titles to currency codes
@@ -871,6 +888,11 @@ class NewsEngine:
continue continue
title = title_el.text or "" title = title_el.text or ""
# A blank title carries nothing to score, show or
# archive, and one such item made the local model
# reject the whole batch (2026-10-01).
if not title.strip():
continue
pub_date = self._parse_rss_date( pub_date = self._parse_rss_date(
pub_date_el.text if pub_date_el is not None else None pub_date_el.text if pub_date_el is not None else None
) )
@@ -1173,11 +1195,27 @@ class NewsEngine:
is re-sent to DeepSeek (fallback mode retries, audit sample); is re-sent to DeepSeek (fallback mode retries, audit sample);
F. only score_source == "keyword" rows change; an llm row stands. F. only score_source == "keyword" rows change; an llm row stands.
""" """
bundle = news_model.get_news_model() if not headlines:
if bundle is None or not headlines:
return 0 return 0
todo = [h for h in headlines if h.score_source == "keyword"] bundle = news_model.get_news_model()
if bundle is None:
self._record_source("news_model", False,
"not loaded in this process")
return 0
# score_batch cleans every text first and raises on one that is
# empty afterwards ("&nbsp;" is), which used to cost the whole
# batch its scores; such a row keeps its keyword score instead.
todo = [h for h in headlines if h.score_source == "keyword"
and self._model_can_score(h.headline)]
if not todo: if not todo:
# Nothing for the model this cycle (fallback mode with DeepSeek
# healthy, or no readable headline). A failure an earlier batch
# recorded says nothing about now and must not sit on /status
# as if it were current; the next batch records its own outcome
# and warns again if it fails.
if self.source_is_failing(
NewsEngine._source_health.get("news_model")):
NewsEngine._source_health.pop("news_model", None)
return 0 return 0
try: try:
results = news_model.score_batch(bundle, [h.headline for h in todo]) results = news_model.score_batch(bundle, [h.headline for h in todo])
@@ -1185,8 +1223,13 @@ class NewsEngine:
raise ValueError( raise ValueError(
f"{len(results)} results for {len(todo)} headlines") f"{len(results)} results for {len(todo)} headlines")
except Exception as e: except Exception as e:
logger.debug(f"news model scoring failed, keyword scores stand: {e}") # Keyword scores stand for the cycle. Loud once (WARNING on the
# first failure, INFO on recovery) and shown by /status: in
# primary mode nothing else scores these headlines.
self._record_source("news_model", False,
f"{type(e).__name__}: {e}")
return 0 return 0
self._record_source("news_model", True, extra=f"{len(todo)} scored")
for h, r in zip(todo, results): for h, r in zip(todo, results):
raw = float(r["score"]) # archive value raw = float(r["score"]) # archive value
shown = raw if r["relevant"] else 0.0 # display value shown = raw if r["relevant"] else 0.0 # display value
@@ -1198,6 +1241,15 @@ class NewsEngine:
h.score_source = "model" h.score_source = "model"
return len(todo) return len(todo)
@staticmethod
def _model_can_score(text: str) -> bool:
"""True when the local model's own cleaner accepts the text."""
try:
news_model.clean_headline(text)
return True
except Exception:
return False
def _audit_with_deepseek( def _audit_with_deepseek(
self, headlines: List[NewsSentiment] self, headlines: List[NewsSentiment]
) -> List[NewsSentiment]: ) -> List[NewsSentiment]:
@@ -1207,10 +1259,15 @@ class NewsEngine:
numbers. One batch per cycle at most; the pairs go into numbers. One batch per cycle at most; the pairs go into
_llm_sent_cache so a headline is never audited twice. This is the _llm_sent_cache so a headline is never audited twice. This is the
only place DeepSeek is called for headlines in primary mode; with only place DeepSeek is called for headlines in primary mode; with
the cap at 0 (the shipped value) it returns [] without a call.""" the cap at 0 (the shipped value) it returns [] without a call.
The copies exist only to be archived, so a process that does not
write the archive (the dashboard daemon, NEWS_ARCHIVE_ENABLED=0)
makes no call either."""
cap = app_config.news_model_audit_per_day() cap = app_config.news_model_audit_per_day()
if cap <= 0 or not headlines or not get_llm_client().available: if cap <= 0 or not headlines or not get_llm_client().available:
return [] return []
if not app_config.news_archive_enabled():
return []
today = datetime.now(timezone.utc).date().isoformat() today = datetime.now(timezone.utc).date().isoformat()
if NewsEngine._audit_day != today: if NewsEngine._audit_day != today:
NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0 NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0
@@ -1253,7 +1310,17 @@ class NewsEngine:
close K/M-suffixed pairs backwards ~11% of the time (e.g. claims close K/M-suffixed pairs backwards ~11% of the time (e.g. claims
202K vs 217K called "higher than forecast") when left to compare 202K vs 217K called "higher than forecast") when left to compare
the raw numbers itself.""" the raw numbers itself."""
key = (event.event, event.currency, str(event.actual or "")) # Forecast and release time are part of the key: the direction is
# a reading of actual AGAINST forecast, and next month's release
# can print the same actual against a different forecast. The time
# is keyed in UTC so one instant is one key whatever offset the
# feed printed it with.
try:
when = event.time.astimezone(timezone.utc).isoformat()
except Exception:
when = str(event.time)
key = (event.event, event.currency, str(event.actual or ""),
str(event.forecast or ""), when)
with NewsEngine._cache_lock: with NewsEngine._cache_lock:
if key in NewsEngine._llm_event_cache: if key in NewsEngine._llm_event_cache:
return NewsEngine._llm_event_cache[key] or None return NewsEngine._llm_event_cache[key] or None
+45
View File
@@ -130,6 +130,45 @@ on M15 (trade-identical, -2.0R W2, -2.8R W3: the swing-anchored
GENERIC stop is not better than the 2 x ATR stop on the same grade-A GENERIC stop is not better than the 2 x ATR stop on the same grade-A
signals); the combination with this block (-0.7R vs the block alone). signals); the combination with this block (-0.7R vs the block alone).
M15 override 7 — daily-trend veto (validated 2026-09-30,
backtest/research/daily_trend_campaign_2026-09-30.md, pre-registered
RISK-FIRST rule approved by the owner that day, CUDA twin runs paired
with gpu_backtest.py MD5s, $0.65): daily_trend_veto="both". The engine
sees at most the H1 chart. The daily trend is +1 when the last CLOSED
daily bar closes above its 50-day EMA AND above the close 60 daily
bars earlier, -1 when below both, else 0 (Oanda's Sunday stub bars are
dropped; signal_aggregator.daily_trend_series). After every setup
archetype has run, a setup AGAINST a non-zero state is dropped (grade
D, no setup, warning "DAILY TREND FILTER: ..."). Candidate vs control:
3-year W3 2023-09-05 -> 2026-09-05 824/+205.12R PF 1.73 DD 12.4R vs
1105/+195.79R PF 1.49 DD 20.9R (halves +76.7/+128.4 vs +76.9/+118.9;
at $0.30 +221.18R PF 1.81 DD 11.4R vs +216.53R PF 1.55 DD 19.6R);
7.6-year LONG 2019-01-01 -> 2026-08-07 1473/+223.33R PF 1.42 DD 41.5R
vs 2091/+192.53R PF 1.24 DD 64.8R (2022 +3.2R vs -24.6R; 2021 NOT
improved, -27.3R vs -25.6R; 2023 worse, +1.8R vs +11.7R); the unseen
HOLDOUT 2026-09-05 -> 2026-10-01 18/-4.62R vs 24/-6.63R; 3 of 4
neighbouring definitions pass (90-day change, 120-day change, 100-day
average; the 60-day change alone misses the 97% line by 1.7R). Trade
diff W3: 281 removed (-9.3R: BUY 106/+7.6R, SELL 175/-16.9R), 0 added;
the removed trades are break-even after costs (-0.03R each) against
+0.25R for the kept ones, so the veto drops the no-edge quarter of the
book rather than a pocket of big losers. ACCEPTED COST (owner decision
2026-09-30): R given up while gold only rises — 2-year W2 597/+155.89R
PF 1.78 DD 11.4R vs 813/+154.15R PF 1.53 DD 15.6R with the first half
+61.8R vs +70.1R, and the old OOS window 2024-06-01 -> 2025-03-30
258/+63.78R PF 1.80 DD 9.1R vs 313/+68.50R PF 1.67 DD 12.4R. The
2026-09-26 rule template (no half and no OOS window may lose) would
have rejected it; the risk-first rule (3-year R >= 97%, PF +0.10, DD
not deeper, long window better on R, DD and 2021+2022, holdout, both
costs, plateau) is the owner's deliberate choice. M15 ONLY: the same
veto FAILED on M5 ("both": W3 DD 10.1 -> 15.2R and the long window
worse on R and DD; "buy": passes W3, loses the long window, no
plateau) and on H1 (W3 +60.3R vs +70.0R). "buy"-only and "sell"-only
FAILED on M15 as well ("sell" has the best W3 total, +212.7R, and a
LONG DD of 76.4R vs 64.8R: in falling years the buys against the daily
trend are the losers). The live wrapper and both backtesters load D1
as an extra REQUIRED frame while the veto is on.
M5 override (validated 2026-07-11, full-history replay 2020-03-17 -> M5 override (validated 2026-07-11, full-history replay 2020-03-17 ->
2026-07-11 + 12mo book, $0.30 spread): block hour 9 UTC (post-London- 2026-07-11 + 12mo book, $0.30 spread): block hour 9 UTC (post-London-
open whipsaw). Hour-9 entries lost -28.9R/143 over 6 years, negative open whipsaw). Hour-9 entries lost -28.9R/143 over 6 years, negative
@@ -332,6 +371,12 @@ def make_config(
# actionable-grade short whose grade is not listed) — see module # actionable-grade short whose grade is not listed) — see module
# docstring (M15 override 6, validated 2026-09-26). # docstring (M15 override 6, validated 2026-09-26).
config.xau_strategy.sell_min_quality = ["A+", "B"] config.xau_strategy.sell_min_quality = ["A+", "B"]
# Daily-trend veto on M15 only: a finished setup against a clear
# daily trend (price vs its 50-day average AND the 60-day change)
# is dropped. The live wrapper and both backtesters load the D1
# frame for it — see module docstring (M15 override 7, validated
# 2026-09-30). M5, H1 and H4 FAILED the same rule and stay "off".
config.xau_strategy.daily_trend_veto = "both"
if primary_tf == Timeframe.H1: if primary_tf == Timeframe.H1:
# Block every H1 SELL — the SELL quality gate (6e in # Block every H1 SELL — the SELL quality gate (6e in
# signal_aggregator.py) downgrades any actionable-grade short # signal_aggregator.py) downgrades any actionable-grade short
+44
View File
@@ -37,6 +37,10 @@ class OandaDataProvider(BaseDataProvider):
RETRY_DELAYS_S = (2.0, 5.0, 10.0) RETRY_DELAYS_S = (2.0, 5.0, 10.0)
RETRY_BUDGET_S = 45.0 RETRY_BUDGET_S = 45.0
RETRY_DELAY_S = RETRY_DELAYS_S[0] # first delay; kept for callers/tests RETRY_DELAY_S = RETRY_DELAYS_S[0] # first delay; kept for callers/tests
# The longest date window that may come back empty and still mean
# "nothing traded here": a weekend joined to a holiday. A longer window
# with no bars is an anomaly, not a closure (see _fetch_oanda_candles).
EMPTY_WINDOW_MAX = pd.Timedelta(days=5)
OANDA_GRANULARITY = { OANDA_GRANULARITY = {
Timeframe.M1: "M1", Timeframe.M1: "M1",
@@ -96,6 +100,26 @@ class OandaDataProvider(BaseDataProvider):
symbol, timeframe, outputsize, start, end) symbol, timeframe, outputsize, start, end)
return self._fetch_oanda_candles(symbol, timeframe, outputsize) return self._fetch_oanda_candles(symbol, timeframe, outputsize)
@staticmethod
def _empty_frame() -> pd.DataFrame:
"""A normalized frame with no bars (a date window nothing traded in)."""
df = pd.DataFrame(
{col: pd.Series(dtype="float64")
for col in ("Open", "High", "Low", "Close", "Volume")},
index=pd.DatetimeIndex([], tz="UTC", name="Datetime"))
return df
@classmethod
def _closure_sized(cls, start: Optional[datetime],
end: Optional[datetime]) -> bool:
"""True for a date window short enough to be one scheduled closure."""
if start is None or end is None:
return False
try:
return pd.Timestamp(end) - pd.Timestamp(start) <= cls.EMPTY_WINDOW_MAX
except (TypeError, ValueError):
return False
def _to_oanda_instrument(self, symbol: str) -> str: def _to_oanda_instrument(self, symbol: str) -> str:
try: try:
return self.OANDA_INSTRUMENT[symbol] return self.OANDA_INSTRUMENT[symbol]
@@ -192,8 +216,25 @@ class OandaDataProvider(BaseDataProvider):
raise OandaDataError( raise OandaDataError(
f"Oanda candle response for {instrument} is not valid " f"Oanda candle response for {instrument} is not valid "
f"JSON: {e}") from e f"JSON: {e}") from e
# A DATE WINDOW with no bars in it (a weekend, a holiday, the closed
# days before a window's first session) is not a failure: fetch_range
# walks history in windows and counts an empty one as "nothing traded
# here". Raising for it made every backtest window whose padded start
# falls on a closed day unfetchable once fetch_range began retrying a
# failing window and aborting after three tries (2026-09-13); found
# 2026-09-30 on the D1 pad of a window starting on a Saturday. A
# `count` request (the live path) with nothing in it stays an error.
# Only a CLOSURE-SIZED window gets that reading (2026-10-01 review):
# an empty answer for a longer one (a 15-day M5 page, a 46-day M15
# page) cannot be a closure, and accepting it let fetch_range step
# over the window and cache history with a hole in it. Raising makes
# fetch_range retry the same window and refuse after three tries.
# (An M1 page is 3 days, so it would still pass; M1 is not fetched.)
closure_sized = self._closure_sized(start, end)
candles = data.get("candles", []) candles = data.get("candles", [])
if not candles: if not candles:
if closure_sized:
return self._empty_frame()
raise OandaDataError( raise OandaDataError(
f"Empty candle response from Oanda for {instrument}") f"Empty candle response from Oanda for {instrument}")
@@ -233,6 +274,9 @@ class OandaDataProvider(BaseDataProvider):
"OHLC fields) — skipped", skipped_malformed, instrument) "OHLC fields) — skipped", skipped_malformed, instrument)
if not rows: if not rows:
# Malformed candles mean the window DID trade: never "no bars".
if closure_sized and not skipped_malformed:
return self._empty_frame()
raise OandaDataError( raise OandaDataError(
f"No complete candles from Oanda for {instrument}") f"No complete candles from Oanda for {instrument}")
+16 -5
View File
@@ -23,6 +23,8 @@ import argparse
import os import os
import sqlite3 import sqlite3
import sys import sys
from datetime import datetime, timedelta, timezone
from typing import Optional
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
@@ -33,18 +35,27 @@ CHUNK = 2000
def _counts(con: sqlite3.Connection, symbol: str, def _counts(con: sqlite3.Connection, symbol: str,
labelled: bool = True) -> dict: labelled: bool = True, now: Optional[datetime] = None) -> dict:
"""Candidate / labelled / complete / pending counts (read-only). """Candidate / labelled / complete / pending counts (read-only).
The age filter uses logged_at (always an ISO string) as a proxy for The age filter uses logged_at (always an ISO string) as a proxy for
the bar close — the labeller itself parses the real bar label.""" the bar close — the labeller itself parses the real bar label. The
cutoff is built here in logged_at's own format (UTC, 'T' separator),
the labeller's convention: SQLite's datetime() prints a space, which
sorts before the 'T', so every row logged on the cutoff's calendar
day was left out of the count."""
moment = now or datetime.now(timezone.utc)
if moment.tzinfo is None:
moment = moment.replace(tzinfo=timezone.utc)
cutoff = moment.astimezone(timezone.utc) - timedelta(
hours=SCAN_LABEL_MIN_AGE_H)
scans = ("SELECT COUNT(*) FROM signals s WHERE s.symbol = ?" scans = ("SELECT COUNT(*) FROM signals s WHERE s.symbol = ?"
" AND s.indicators_json IS NOT NULL AND s.price > 0") " AND s.indicators_json IS NOT NULL AND s.price > 0")
due = (scans + " AND s.logged_at <" due = scans + " AND s.logged_at < ?"
f" datetime('now', '-{int(SCAN_LABEL_MIN_AGE_H)} hours')")
out = { out = {
"scan rows with indicators": con.execute(scans, (symbol,)).fetchone()[0], "scan rows with indicators": con.execute(scans, (symbol,)).fetchone()[0],
"of those, window elapsed": con.execute(due, (symbol,)).fetchone()[0], "of those, window elapsed": con.execute(
due, (symbol, cutoff.isoformat())).fetchone()[0],
} }
if labelled: if labelled:
out["labelled"] = con.execute( out["labelled"] = con.execute(
+20 -6
View File
@@ -275,15 +275,26 @@ def build_report(con, rowid: int) -> Tuple[Optional[SignalReport], Optional[Fund
return report, context, frames, _loads(payloads_json, None) return report, context, frames, _loads(payloads_json, None)
def stored_personas(stored: Any) -> List[str]:
"""The requested personas whose exact payload the row logged."""
if not isinstance(stored, dict):
return []
return [p for p in PERSONAS if isinstance(stored.get(p), dict)]
def build_jobs(report: SignalReport, context: FundamentalContext, def build_jobs(report: SignalReport, context: FundamentalContext,
frames: Dict[Timeframe, pd.DataFrame], stored: Optional[Dict[str, Any]], frames: Dict[Timeframe, pd.DataFrame], stored: Optional[Dict[str, Any]],
settings: Dict[str, Any]) -> Tuple[Tuple[str, str, Dict[str, Any]], ...]: settings: Dict[str, Any]) -> Tuple[Tuple[str, str, Dict[str, Any]], ...]:
"""Persona jobs on the CURRENT prompts: stored payloads when the row """Persona jobs on the CURRENT prompts. A persona whose exact payload
has them, else rebuilt through the live payload builders.""" the row logged replays that payload; every other persona is rebuilt
through the live payload builders. Per persona, not all-or-nothing:
the live panel stores only the personas it ran (macro alone since
2026-09-17), so "all four or none" never used a stored payload."""
systems = {"macro": ap._MACRO_SYSTEM, "technical": ap._TECH_SYSTEM, systems = {"macro": ap._MACRO_SYSTEM, "technical": ap._TECH_SYSTEM,
"risk": ap._RISK_SYSTEM, "contrarian": ap._CONTRARIAN_SYSTEM} "risk": ap._RISK_SYSTEM, "contrarian": ap._CONTRARIAN_SYSTEM}
if stored and all(p in stored for p in PERSONAS): kept = {p: stored[p] for p in stored_personas(stored)}
return tuple((p, systems[p], stored[p]) for p in PERSONAS) if len(kept) == len(PERSONAS):
return tuple((p, systems[p], kept[p]) for p in PERSONAS)
candles: Dict[str, Any] = {} candles: Dict[str, Any] = {}
analyses = [report.primary_analysis] + list(report.confirmation_analyses) analyses = [report.primary_analysis] + list(report.confirmation_analyses)
counts = [int(settings.get("candle_bars_primary", ap.DEFAULT_CANDLE_BARS_PRIMARY))] + \ counts = [int(settings.get("candle_bars_primary", ap.DEFAULT_CANDLE_BARS_PRIMARY))] + \
@@ -297,7 +308,8 @@ def build_jobs(report: SignalReport, context: FundamentalContext,
if atr and atr > 0 and df is not None and len(df) >= 2: if atr and atr > 0 and df is not None and len(df) >= 2:
candles[analysis.timeframe] = ap._normalize_candles(df, atr, n) candles[analysis.timeframe] = ap._normalize_candles(df, atr, n)
ctx = ap._price_context(report, frames, context) ctx = ap._price_context(report, frames, context)
return ap._panel_jobs(report, context, candles or None, ctx, PERSONAS) rebuilt = ap._panel_jobs(report, context, candles or None, ctx, PERSONAS)
return tuple((p, system, kept.get(p, payload)) for p, system, payload in rebuilt)
# ── Scoring ─────────────────────────────────────────────────────────── # ── Scoring ───────────────────────────────────────────────────────────
@@ -445,7 +457,9 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
"event_row": rep.id, "bar": report.timestamp, "tf": rep.timeframe, "event_row": rep.id, "bar": report.timestamp, "tf": rep.timeframe,
"direction": report.trade_setup.direction, "grade": rep.quality, "direction": report.trade_setup.direction, "grade": rep.quality,
"strategy": report.trade_setup.strategy, "strategy": report.trade_setup.strategy,
"R": res.advisory_r if res else None, "stored_payloads": bool(stored), # which personas replayed the exact logged payload ("" = all rebuilt)
"R": res.advisory_r if res else None,
"stored_payloads": ",".join(stored_personas(stored)),
"base_p": base_p, "prior_move_4h_atr": ctx.get("prior_move_4h_atr"), "base_p": base_p, "prior_move_4h_atr": ctx.get("prior_move_4h_atr"),
"move_from_start_atr": ctx.get("move_from_24h_start_atr"), "move_from_start_atr": ctx.get("move_from_24h_start_atr"),
"big_candle_last3": ctx.get("big_candle_last3"), "big_candle_last3": ctx.get("big_candle_last3"),
+205 -11
View File
@@ -14,6 +14,7 @@ from datetime import datetime, timezone
from typing import Dict, List, Optional from typing import Dict, List, Optional
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
import numpy as np
import pandas as pd import pandas as pd
_NY_TZ = ZoneInfo("America/New_York") _NY_TZ = ZoneInfo("America/New_York")
@@ -155,6 +156,65 @@ def _transitioning_boost(abs_score: float, confidence: float, mtf: bool, mode: s
return None return None
# ── Daily trend (campaign 2026-09-30) ────────────────────────────────
# One definition for the live scan, the bar replay and the CUDA twin's host
# side (backtest/cudaBasedBacktesting.trend_flags calls daily_trend_series on
# the whole D1 frame and indexes it). Every step is causal (the Sunday
# filter, the EMA recursion, the shift), so the value on a prefix slice is
# bit-identical to the value at the same bar of the full series.
DAILY_TREND_MODES = ("off", "both", "buy", "sell")
DAILY_TREND_EMA_SPAN = 50 # validated literals (do not edit): the
DAILY_TREND_LOOKBACK = 60 # campaign tested exactly this definition
_DAILY_TREND_PREFIX = "DAILY TREND FILTER:"
def daily_trend_mode_of(xau_cfg) -> str:
"""The validated `xau_strategy.daily_trend_veto`; an unknown value is a hard error."""
mode = xau_cfg.daily_trend_veto
if mode not in DAILY_TREND_MODES:
raise ValueError(
f"xau_strategy.daily_trend_veto={mode!r}; expected one of {DAILY_TREND_MODES}"
)
return mode
def daily_trend_needed(config: PlatformConfig) -> bool:
"""True when the scan needs the D1 frame: the veto is on for a gold config."""
return (daily_trend_mode_of(config.xau_strategy) != "off"
and config.xau_strategy.enabled
and config.display_name == "XAU/USD")
def daily_trend_series(d1: Optional[pd.DataFrame]) -> pd.Series:
"""Daily trend per CLOSED D1 bar: +1 when the close is above its 50-bar
EMA and above the close 60 bars earlier, -1 when below both, else 0.
`d1` must hold closed bars only (the provider drops the forming candle;
the replay slices as-of). Bars labelled Sunday are dropped first: the
provider's UTC-aligned daily candles include a 1-2 hour stub for the
Sunday reopening, which is not a trading day. The first 60 kept bars
read 0 (no 60-bar change yet)."""
if d1 is None or len(d1) == 0:
return pd.Series([], dtype="int8")
kept = d1[d1.index.weekday != 6]
close = kept["Close"].astype("float64")
ema = close.ewm(span=DAILY_TREND_EMA_SPAN, adjust=False).mean()
change = close / close.shift(DAILY_TREND_LOOKBACK) - 1.0
up = (close > ema) & (change > 0)
down = (close < ema) & (change < 0)
state = np.where(up, 1, np.where(down, -1, 0)).astype("int8")
return pd.Series(state, index=kept.index)
def daily_trend(d1: Optional[pd.DataFrame]) -> Optional[int]:
"""The daily trend at the last closed D1 bar of `d1` (+1 / 0 / -1), or
None when the frame holds no usable bar."""
series = daily_trend_series(d1)
if len(series) == 0:
return None
return int(series.iloc[-1])
def _is_forex_weekend(now_utc: datetime) -> bool: def _is_forex_weekend(now_utc: datetime) -> bool:
"""Gold/FX market closed for the weekend: Friday 17:00 → Sunday 18:00 """Gold/FX market closed for the weekend: Friday 17:00 → Sunday 18:00
NEW YORK time (= Fri 21:00 UTC in US summer, 22:00 in winter — NEW YORK time (= Fri 21:00 UTC in US summer, 22:00 in winter —
@@ -284,6 +344,17 @@ class SignalReport:
penalty_tags: Dict[str, float] = field(default_factory=dict) penalty_tags: Dict[str, float] = field(default_factory=dict)
penalty_total: float = 0.0 penalty_total: float = 0.0
momentum_fading: bool = False momentum_fading: bool = False
# Daily trend at the last closed D1 bar (+1 up / 0 unclear / -1 down);
# None = not computed. Set by the core when the daily-trend veto is on
# (_apply_daily_trend_veto); where the veto is off the LIVE wrapper
# stamps it for the alert text and the signal log only
# (_stamp_daily_trend_for_display) — no gate reads it there.
daily_trend: Optional[int] = None
# Lot-size and open-exposure lines (advisory/DISPLAY-ONLY). Stamped
# post-hoc by surfaces via exposure.attach_exposure — never populated
# inside analyze_from_dataframes and never read by any gate.
size_line: str = ""
open_line: str = ""
class SignalAggregator: class SignalAggregator:
@@ -292,6 +363,11 @@ class SignalAggregator:
def __init__(self, config: PlatformConfig): def __init__(self, config: PlatformConfig):
self.config = config self.config = config
grading_mode_of(config.signals) # fail at start, not at the first bar grading_mode_of(config.signals) # fail at start, not at the first bar
daily_trend_mode_of(config.xau_strategy)
if not config.xau_strategy.wide_stop_cap_atr >= 0:
raise ValueError(
f"xau_strategy.wide_stop_cap_atr={config.xau_strategy.wide_stop_cap_atr!r}; "
f"expected a number >= 0 (0 = off)")
self.data_provider = make_data_provider(config) self.data_provider = make_data_provider(config)
self.tech_engine = TechnicalEngine(config.technical) self.tech_engine = TechnicalEngine(config.technical)
self._news_engine: Optional[NewsEngine] = None # lazy self._news_engine: Optional[NewsEngine] = None # lazy
@@ -362,6 +438,12 @@ class SignalAggregator:
bt = self.config.backtest bt = self.config.backtest
if bt.regime_filter and bt.regime_filter_timeframe not in timeframes: if bt.regime_filter and bt.regime_filter_timeframe not in timeframes:
timeframes.append(bt.regime_filter_timeframe) timeframes.append(bt.regime_filter_timeframe)
# The daily-trend veto reads the D1 frame inside the core, so it is
# a REQUIRED frame when the veto is on (a failed fetch fails the
# scan below, as for any other frame). The backtesters append it
# under the same guard (rule 4).
if daily_trend_needed(self.config) and Timeframe.D1 not in timeframes:
timeframes.append(Timeframe.D1)
dfs: Dict[Timeframe, pd.DataFrame] = {} dfs: Dict[Timeframe, pd.DataFrame] = {}
stale_tfs = [] stale_tfs = []
@@ -465,9 +547,34 @@ class SignalAggregator:
f"{symbol}: live session filter errored — filter SKIPPED this scan" f"{symbol}: live session filter errored — filter SKIPPED this scan"
) )
self._stamp_daily_trend_for_display(report, dfs)
self._log_signal(report, log_source, dfs) self._log_signal(report, log_source, dfs)
return report return report
def _stamp_daily_trend_for_display(
self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
) -> None:
"""Daily trend for the alert text and the signal log on the
timeframes where the daily-trend veto is OFF (the core stamps it
where the veto is on). Display and record-keeping only: the frame
is never added to `dfs`, nothing reads the value back, and a failed
fetch leaves it None instead of failing the scan. The scan's own D1
frame is used when it has one (H4 confirms on D1); otherwise the
provider's D1 frame, cached until the next daily close and
refreshed by the M15 scan, which needs it as a required frame.
Live wrapper only — the backtesters never come through here."""
if report.daily_trend is not None or report.display_name != "XAU/USD":
return
if report.primary_analysis is None:
return
try:
d1 = dfs.get(Timeframe.D1)
if d1 is None:
d1 = self.data_provider.fetch_ohlcv(report.symbol, Timeframe.D1)
report.daily_trend = daily_trend(d1)
except Exception:
logger.debug("daily trend (display) unavailable", exc_info=True)
def _log_signal( def _log_signal(
self, report: SignalReport, source: Optional[str], self, report: SignalReport, source: Optional[str],
dataframes: Optional[Dict[Timeframe, pd.DataFrame]] = None, dataframes: Optional[Dict[Timeframe, pd.DataFrame]] = None,
@@ -541,6 +648,13 @@ class SignalAggregator:
bt.session_end_hour], bt.session_end_hour],
"regime_filter": bool(bt.regime_filter), "regime_filter": bool(bt.regime_filter),
} }
# 2026-09-30 knobs enter the hash only when they are in force (the
# grading_mode precedent in engine_config_fingerprint): a config
# that leaves them off keeps the fingerprint its live rows carry.
if xs.daily_trend_veto != "off":
payload["daily_trend_veto"] = xs.daily_trend_veto
if xs.wide_stop_cap_atr > 0:
payload["wide_stop_cap_atr"] = xs.wide_stop_cap_atr
return hashlib.sha1( return hashlib.sha1(
json.dumps(payload, sort_keys=True, default=str).encode("utf-8") json.dumps(payload, sort_keys=True, default=str).encode("utf-8")
).hexdigest()[:12] ).hexdigest()[:12]
@@ -1032,6 +1146,14 @@ class SignalAggregator:
if height < min_tp: if height < min_tp:
return return
# Wide-stop cap AFTER the gates (xau_strategy.wide_stop_cap_atr,
# 0 = off), same contract as the swing generator's.
cap_dist = self._wide_stop_cap(report, atr, min_sl_price)
if cap_dist is not None and sl_dist > cap_dist:
sl_dist = cap_dist
sl = entry - cap_dist if is_buy else entry + cap_dist
rr = height / sl_dist
# Printed-target floor AFTER the gates (2026-09-26, see # Printed-target floor AFTER the gates (2026-09-26, see
# config.SignalConfig.tp_print_floor_rr): the measured move stays # config.SignalConfig.tp_print_floor_rr): the measured move stays
# the gated target; the printed guide is at least the floor. # the gated target; the printed guide is at least the floor.
@@ -1215,23 +1337,26 @@ class SignalAggregator:
report.mtf_agreement = self._check_mtf_agreement(report) report.mtf_agreement = self._check_mtf_agreement(report)
# ── 4. Fundamental context (optional) ────────────────────── # ── 4. Fundamental context (optional) ──────────────────────
# News is display and warning only (it never gates), so a failure
# anywhere in it must not cost the scan: the report then carries
# no news at all, the values the replay path (use_news=False)
# always has. All five fields are set together or not at all.
news_fields = ("clear", "", 0.0, "", "")
if use_news: if use_news:
try:
news = self._get_news_engine() news = self._get_news_engine()
if context is None: if context is None:
context = news.get_fundamental_context() context = news.get_fundamental_context()
event_risk, event_desc = news.get_event_risk_level(symbol, context) event_risk, event_desc = news.get_event_risk_level(symbol, context)
report.event_risk = event_risk
report.event_description = event_desc
sentiment, sent_desc = news.get_pair_sentiment(symbol, context) sentiment, sent_desc = news.get_pair_sentiment(symbol, context)
report.news_sentiment = sentiment news_fields = (event_risk, event_desc, sentiment, sent_desc,
report.news_description = sent_desc context.llm_summary)
report.news_summary = context.llm_summary except Exception:
else: logger.warning(
report.event_risk = "clear" f"News context failed for {symbol} — the scan continues "
report.event_description = "" f"without news", exc_info=True)
report.news_sentiment = 0.0 (report.event_risk, report.event_description, report.news_sentiment,
report.news_description = "" report.news_description, report.news_summary) = news_fields
report.news_summary = ""
# ── 5. Spread estimate from primary data (avoids extra API call) ─ # ── 5. Spread estimate from primary data (avoids extra API call) ─
try: try:
@@ -1471,6 +1596,12 @@ class SignalAggregator:
# cost-floor veto no longer describes it. # cost-floor veto no longer describes it.
self._drop_stale_spread_gate_vetoes(report) self._drop_stale_spread_gate_vetoes(report)
# ── 12. Daily-trend veto (opt-in via per-TF factories) ───────
# Last on purpose: a FINAL veto on whatever archetype shipped the
# setup (core, XAU_PB or XAU_CP), which is exactly what the
# campaign replayed. Mirrored in cuda_signal_kernels.py K2.
self._apply_daily_trend_veto(report, dfs)
return report return report
# ── Internals ────────────────────────────────────────────────── # ── Internals ──────────────────────────────────────────────────
@@ -1543,6 +1674,57 @@ class SignalAggregator:
report.warnings[:] = kept report.warnings[:] = kept
report.cleaned_warnings.extend(stale) report.cleaned_warnings.extend(stale)
def _apply_daily_trend_veto(
self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
) -> None:
"""Step 12: remove a finished setup that goes against a clear daily
trend (xau_strategy.daily_trend_veto, off by default).
Reads the D1 frame from `dfs` (closed bars only: the live wrapper
fetches it as a required frame, the replay slices it as-of), stamps
report.daily_trend, and follows the gate-chain convention when it
fires: grade D, no setup, a warning that names the gate. The final
signal and the action text are left as they are."""
xcfg = self.config.xau_strategy
mode = daily_trend_mode_of(xcfg)
if mode == "off" or not xcfg.enabled or report.display_name != "XAU/USD":
return
state = daily_trend(dfs.get(Timeframe.D1))
report.daily_trend = state
if state is None:
# Mixed case like the regime filter's twin: never a gate prefix.
report.warnings.append(
"Daily trend filter requested but 1d data unavailable")
return
setup = report.trade_setup
if setup is None or state == 0:
return
is_buy = setup.direction == "BUY"
against = ((is_buy and state < 0 and mode in ("both", "buy"))
or (not is_buy and state > 0 and mode in ("both", "sell")))
if not against:
return
report.warnings.append(
f"{_DAILY_TREND_PREFIX} {setup.direction} against a "
f"{'falling' if is_buy else 'rising'} daily trend — no trade"
)
report.signal_quality = "D"
report.trade_setup = None
def _wide_stop_cap(self, report: SignalReport, atr: float,
stop_floor: float) -> Optional[float]:
"""The largest stop distance xau_strategy.wide_stop_cap_atr allows
for a swing or chart-pattern setup: max(K x ATR, the setup's own
stop floor); None when the cap is off (K = 0) or the pair is not
gold. Applied by the generators AFTER their min-RR and spread
gates, so the gates keep judging the uncapped geometry and the
set of trades cannot change (the printed-target floor precedent)."""
xcfg = self.config.xau_strategy
k = xcfg.wide_stop_cap_atr
if not k > 0 or not xcfg.enabled or report.display_name != "XAU/USD":
return None
return max(k * atr, stop_floor)
def _printed_target_floor(self) -> float: def _printed_target_floor(self) -> float:
"""The floor every generator applies to the PRINTED target 1, as a """The floor every generator applies to the PRINTED target 1, as a
fraction of the stop: max(tp_floor_rr, tp_print_floor_rr). Applied fraction of the stop: max(tp_floor_rr, tp_print_floor_rr). Applied
@@ -2024,6 +2206,18 @@ class SignalAggregator:
) )
return None return None
# Wide-stop cap AFTER the gates (xau_strategy.wide_stop_cap_atr,
# 0 = off): the gates above judged the swing-anchored stop, so the
# same trades ship; only the stop (the risk unit) is pulled in,
# never inside the generator's own floor. Mirrored in
# cuda_signal_kernels.py K2.
cap_dist = self._wide_stop_cap(
report, atr, max(atr * sl_atr_floor, min_sl_price))
if cap_dist is not None and sl_distance > cap_dist:
sl_distance = cap_dist
sl = entry - cap_dist if is_buy else entry + cap_dist
rr = tp1_distance / sl_distance
# Printed-target floor AFTER the gates (2026-09-26, see # Printed-target floor AFTER the gates (2026-09-26, see
# config.SignalConfig.tp_print_floor_rr): the gates above judged # config.SignalConfig.tp_print_floor_rr): the gates above judged
# the swing/ATR target, so the trade is unchanged; only the guide # the swing/ATR target, so the trade is unchanged; only the guide
+220 -5
View File
@@ -84,6 +84,23 @@ or news_engine) so this module stays at the bottom of the import graph.
time-boxed; batch driver: scripts/label_scan_outcomes.py). Readers time-boxed; batch driver: scripts/label_scan_outcomes.py). Readers
must dedup by the FIRST row per (timeframe, bar_ts) — several surfaces must dedup by the FIRST row per (timeframe, bar_ts) — several surfaces
log the same bar. log the same bar.
2026-09-30 daily-trend veto:
* Row column ``daily_trend`` (+1 / 0 / -1 from closed D1 bars: computed
by the core where the veto is on, stamped by the live wrapper for
display where it is off; NULL on blocked / failed scans and when the
daily frame could not be read) and the gate prefix ``DAILY TREND
FILTER`` in ``gate_reason``.
* Row column ``alerted_at`` (UTC time of the Telegram alert for this row;
``mark_alerted``) and the read-only helper ``open_alerted_setups`` —
alerted plans whose advisory outcome is still PENDING — behind the
"already open" line on the alert (exposure.py). Since 2026-10-01 the
stamp is written just BEFORE the send and cleared again
(``unmark_alerted``) when no chat received the alert, so two alerts
sent at the same candle close count each other; ``last_alerted_bars``
reads the newest alerted bar per timeframe so the bot's entry cooldown
survives a restart.
""" """
import hashlib import hashlib
@@ -100,6 +117,7 @@ from typing import Any, Dict, Iterable, List, Optional, Tuple
import app_config import app_config
from advisory_outcome import ( from advisory_outcome import (
MAX_HOLD_BARS,
advance_outcome, advance_outcome,
canonical_r, canonical_r,
initial_outcome, initial_outcome,
@@ -240,6 +258,14 @@ _ADDITIVE_SIGNAL_COLS = (
("panel_probs_json", "TEXT"), ("panel_probs_json", "TEXT"),
("panel_payloads_json", "TEXT"), ("panel_payloads_json", "TEXT"),
("panel_context_json", "TEXT"), ("panel_context_json", "TEXT"),
# 2026-09-30 daily-trend veto (see module docstring).
("daily_trend", "INTEGER"),
# 2026-09-30: UTC time of the Telegram alert for THIS row (mark_alerted:
# written by the bot just before the send, cleared by unmark_alerted
# when no chat received it). NULL on every row that was scanned but not
# alerted — the "which plans did the reader actually receive" key the
# open-exposure line reads (open_alerted_setups).
("alerted_at", "TEXT"),
) )
# Read-side compatibility view `signals_with_news`: legacy rows keep # Read-side compatibility view `signals_with_news`: legacy rows keep
@@ -262,6 +288,11 @@ _GATE_PREFIXES = (
"Primary timeframe stale", "Primary timeframe stale",
"Primary timeframe data unavailable", "Primary timeframe data unavailable",
"DATA UNAVAILABLE:", "DATA UNAVAILABLE:",
# The daily-trend veto removes a setup that every earlier gate had let
# through, so on such a row it IS the reason: listed ahead of the chain
# because the vetoed setup's stale warnings (a cleaned SPREAD GATE, the
# chart-pattern path's cleaned vetoes) are still in the pool.
"DAILY TREND FILTER",
"REGIME FILTER", "REGIME FILTER",
"EXHAUSTION BLOCK", "EXHAUSTION BLOCK",
"EXHAUSTED TREND FILTER", "EXHAUSTED TREND FILTER",
@@ -313,6 +344,7 @@ _SIGNAL_INSERT_COLS = (
"config_fingerprint", "config_fingerprint",
"bar_ts", "engine_fingerprint", "code_version", "penalty_tags_json", "bar_ts", "engine_fingerprint", "code_version", "penalty_tags_json",
"penalty_total", "momentum_fading", "exhaustion_count", "penalty_total", "momentum_fading", "exhaustion_count",
"daily_trend",
) )
_INSERT = ( _INSERT = (
"INSERT INTO signals (" + ", ".join(_SIGNAL_INSERT_COLS) + ") VALUES (" "INSERT INTO signals (" + ", ".join(_SIGNAL_INSERT_COLS) + ") VALUES ("
@@ -330,6 +362,12 @@ def _finite(x: Any) -> Optional[float]:
return f if math.isfinite(f) else None return f if math.isfinite(f) else None
def _daily_trend_value(x: Any) -> Optional[int]:
"""The report's daily trend as -1 / 0 / +1, or None when it carries none."""
f = _finite(x)
return None if f is None else int(f)
def _iso(ts: Any) -> Optional[str]: def _iso(ts: Any) -> Optional[str]:
if ts is None: if ts is None:
return None return None
@@ -815,6 +853,7 @@ def _report_row(
"momentum_fading": int(bool(getattr(report, "momentum_fading", False))), "momentum_fading": int(bool(getattr(report, "momentum_fading", False))),
"exhaustion_count": int( "exhaustion_count": int(
_finite(getattr(report, "exhaustion_count", 0)) or 0), _finite(getattr(report, "exhaustion_count", 0)) or 0),
"daily_trend": _daily_trend_value(getattr(report, "daily_trend", None)),
} }
@@ -903,6 +942,11 @@ class SignalLogger:
self._conn.execute( self._conn.execute(
"CREATE INDEX IF NOT EXISTS idx_signals_tf_bar" "CREATE INDEX IF NOT EXISTS idx_signals_tf_bar"
" ON signals (timeframe, bar_ts)") " ON signals (timeframe, bar_ts)")
# Partial index: only alerted rows (a few per day), so the
# open-exposure read on every scan never walks the table.
self._conn.execute(
"CREATE INDEX IF NOT EXISTS idx_signals_alerted"
" ON signals (symbol, alerted_at) WHERE alerted_at IS NOT NULL")
self._conn.commit() self._conn.commit()
def log( def log(
@@ -1106,6 +1150,31 @@ class SignalLogger:
(str(level or ""), kinds_text, str(line or ""), int(rowid))) (str(level or ""), kinds_text, str(line or ""), int(rowid)))
self._conn.commit() self._conn.commit()
def mark_alerted(self, rowid: int,
when_iso: Optional[str] = None) -> Optional[str]:
"""Stamp the time of the alert for this row. The first stamp wins:
a re-alert of the same row keeps the original time. Returns the
stamp when THIS call wrote it and None when the row already had
one (or does not exist), so a caller that must undo its own stamp
can never clear an earlier alert's."""
when_iso = when_iso or datetime.now(timezone.utc).isoformat()
with self._lock:
cur = self._conn.execute(
"UPDATE signals SET alerted_at=?"
" WHERE id=? AND alerted_at IS NULL", (when_iso, int(rowid)))
self._conn.commit()
return when_iso if cur.rowcount == 1 else None
def unmark_alerted(self, rowid: int, when_iso: str) -> bool:
"""Clear the stamp mark_alerted returned (the alert reached no
chat). Only that exact stamp is cleared; True when it was."""
with self._lock:
cur = self._conn.execute(
"UPDATE signals SET alerted_at=NULL"
" WHERE id=? AND alerted_at=?", (int(rowid), str(when_iso)))
self._conn.commit()
return cur.rowcount == 1
def resolve_pending( def resolve_pending(
self, symbol: str, candles_json: Optional[str], limit: int = 100 self, symbol: str, candles_json: Optional[str], limit: int = 100
) -> int: ) -> int:
@@ -1488,11 +1557,12 @@ class SignalLogger:
old = json.loads(state_json) if state_json else {} old = json.loads(state_json) if state_json else {}
if not _is_gap_reason(old.get("reason")): if not _is_gap_reason(old.get("reason")):
continue continue
# The locked trailing stop lives ONLY in this progress # The locked trailing stop lives in this progress block. A
# block, and a re-seed rebuilds the walk from scratch over # re-seed would rebuild the walk from scratch only to stop
# a stream whose hole is still there (the continuity check # at the same hole (advance_outcome checks continuity bar
# is all-or-nothing), so the floor would be lost for good. # by bar and keeps the progress up to the hole) and be
# Finish such a row here; re-seed only what has nothing. # floored at the same stop, so finish such a row here;
# re-seed only what has nothing locked.
floored = resolve_locked_floor(json.loads(_dumps(old))) floored = resolve_locked_floor(json.loads(_dumps(old)))
if floored.get("status") == "RESOLVED": if floored.get("status") == "RESOLVED":
self._conn.execute( self._conn.execute(
@@ -1673,6 +1743,151 @@ def log_watch_update(report: Any, watch: Any) -> None:
exc_info=True) exc_info=True)
def _logged_row(report: Any) -> Optional[Tuple["SignalLogger", int]]:
"""(logger, rowid) of the row log_signal wrote for this scan; None for
a report that was never logged or a process with logging off."""
rowid = getattr(report, "_signal_log_rowid", None)
if rowid is None or not app_config.signal_log_enabled():
return None
path = app_config.signal_log_path()
with _loggers_lock:
lg = _loggers.get(path)
return None if lg is None else (lg, int(rowid))
def mark_alerted(report: Any) -> Optional[str]:
"""Stamp `alerted_at` on the row logged for this scan. The bot calls
this just BEFORE it sends the alert, so an alert sent a moment later
on another timeframe already counts this plan, and undoes it with
unmark_alerted when no chat received the message. Same hand-off as
log_panel_update (the private rowid from log_signal). Returns the
stamp this call wrote; None when nothing was written (never logged,
logging off, or the row already carried a stamp). NEVER raises."""
try:
hit = _logged_row(report)
return None if hit is None else hit[0].mark_alerted(hit[1])
except Exception:
logger.warning("signal_log: failed to mark the alerted row",
exc_info=True)
return None
def unmark_alerted(report: Any, stamp: Optional[str]) -> bool:
"""Clear the stamp mark_alerted(report) returned: the alert reached no
chat, so the row must not count as a plan the reader holds. A no-op
without a stamp; only that exact stamp is cleared. NEVER raises."""
if not stamp:
return False
try:
hit = _logged_row(report)
return False if hit is None else hit[0].unmark_alerted(hit[1], stamp)
except Exception:
logger.warning("signal_log: failed to clear the alerted stamp",
exc_info=True)
return False
# A PENDING outcome older than this is not shown as "open": the advisory
# model times a trade out after MAX_HOLD_BARS bars, so an older PENDING row
# is one the resolver never reached (bot outage, candle gap), not a trade
# still running. Calendar allowance: 7/5 for weekends plus three days.
def _open_setup_max_age_seconds(timeframe: Any) -> float:
bar = _TF_SECONDS.get(normalize_timeframe(timeframe), 3600)
return MAX_HOLD_BARS * bar * 1.4 + 3 * 86400
def open_alerted_setups(symbol: str, exclude_key: Optional[str] = None,
now: Optional[datetime] = None,
path: Optional[str] = None) -> List[Dict[str, Any]]:
"""ALERTED trade plans whose advisory outcome is still open, oldest
first: ``[{id, event_id, timeframe, direction, entry, stop_loss,
alerted_at, setup_key}]``, one entry per advisory event.
Read-only (a ``mode=ro`` connection of its own, so the dashboard
process, which never writes this database, can call it). "Open" means
the row's advisory event is still PENDING in the cleaned-mid outcome
model: next-open entry, stop first, close-only trailing. That is the
model's view of a plan the reader MAY have taken, not a broker
position. ``exclude_key`` drops the plan being alerted now from its
own count. NEVER raises: any failure (no database yet, an older schema
without ``alerted_at``) returns an empty list."""
try:
db_path = path or app_config.signal_log_path()
if not os.path.exists(db_path):
return []
now = now or datetime.now(timezone.utc)
con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=2.0)
try:
rows = con.execute(
"SELECT a.id, COALESCE(a.advisory_event_id, a.id), a.timeframe,"
" a.setup_json, a.alerted_at, a.setup_key"
" FROM signals a JOIN signals e"
" ON e.id = COALESCE(a.advisory_event_id, a.id)"
" WHERE a.symbol=? AND a.alerted_at IS NOT NULL"
" AND e.advisory_outcome_status='PENDING'"
" ORDER BY a.id", (symbol,)).fetchall()
finally:
con.close()
out: List[Dict[str, Any]] = []
seen = set()
for rowid, event_id, timeframe, setup_json, alerted_at, key in rows:
if event_id in seen or (exclude_key and key == exclude_key):
continue
try:
age = (now - datetime.fromisoformat(alerted_at)).total_seconds()
except (TypeError, ValueError):
continue
if age > _open_setup_max_age_seconds(timeframe):
continue
setup = json.loads(setup_json) if setup_json else {}
direction = setup.get("direction")
if direction not in ("BUY", "SELL"):
continue
seen.add(event_id)
out.append({
"id": int(rowid), "event_id": int(event_id),
"timeframe": normalize_timeframe(timeframe),
"direction": direction,
"entry": _finite(setup.get("entry")),
"stop_loss": _finite(setup.get("stop_loss")),
"alerted_at": alerted_at, "setup_key": key,
})
return out
except Exception:
logger.debug("signal_log: open alerted setups unavailable",
exc_info=True)
return []
def last_alerted_bars(path: Optional[str] = None
) -> Dict[Tuple[str, str], str]:
"""``{(symbol, timeframe): bar label}`` of the newest ALERTED row per
symbol and timeframe (newest = highest id). The bot seeds its 5-bar
entry cooldown from this at start-up, so a restart does not forget the
last alert. Read-only (``mode=ro``); NEVER raises: no database, an
older schema or a locked file returns ``{}``."""
try:
db_path = path or app_config.signal_log_path()
if not os.path.exists(db_path):
return {}
con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=2.0)
try:
rows = con.execute(
"SELECT s.symbol, s.timeframe, s.bar_ts FROM signals s"
" JOIN (SELECT MAX(id) AS id FROM signals"
" WHERE alerted_at IS NOT NULL"
" GROUP BY symbol, timeframe) m ON m.id = s.id"
).fetchall()
finally:
con.close()
return {(str(symbol), normalize_timeframe(timeframe)): str(bar)
for symbol, timeframe, bar in rows if bar}
except Exception:
logger.debug("signal_log: last alerted bars unavailable",
exc_info=True)
return {}
def reset_loggers_for_tests() -> None: def reset_loggers_for_tests() -> None:
"""Close and drop all cached loggers (test isolation).""" """Close and drop all cached loggers (test isolation)."""
with _loggers_lock: with _loggers_lock:
+15 -4
View File
@@ -549,6 +549,7 @@ function cardShell(tf) {
<span class="grade d-none" data-f="grade"></span> <span class="grade d-none" data-f="grade"></span>
</div> </div>
<p class="tf-plain d-none" data-f="plain"></p> <p class="tf-plain d-none" data-f="plain"></p>
<p class="tf-plain d-none" data-f="trend"></p>
<p class="tf-action" data-f="action"></p> <p class="tf-action" data-f="action"></p>
<div class="tf-readouts"> <div class="tf-readouts">
<div> <div>
@@ -589,7 +590,7 @@ function buildCards() {
TF_ORDER.forEach(tf => indChart(tf)); TF_ORDER.forEach(tf => indChart(tf));
} }
function planHTML(ts) { function planHTML(ts, r) {
if (!ts) { if (!ts) {
return `<div class="plan plan-empty"> return `<div class="plan plan-empty">
<i class="fa-regular fa-circle-xmark me-1" aria-hidden="true"></i> <i class="fa-regular fa-circle-xmark me-1" aria-hidden="true"></i>
@@ -601,6 +602,11 @@ function planHTML(ts) {
const trail = (ts.trail_arm_r != null && ts.trail_dist_r != null) const trail = (ts.trail_arm_r != null && ts.trail_dist_r != null)
? `Exit rule (tested): once the trade is ${ts.trail_arm_r}R in profit, the stop trails ${ts.trail_dist_r}R behind the best price, moved at candle close. Until then the stop is the exit; targets are guides.` ? `Exit rule (tested): once the trade is ${ts.trail_arm_r}R in profit, the stop trails ${ts.trail_dist_r}R behind the best price, moved at candle close. Until then the stop is the exit; targets are guides.`
: "Exit rule: the trailing stop uses the platform default; targets are guides."; : "Exit rule: the trailing stop uses the platform default; targets are guides.";
// Lot size (present only when the owner shares it on this page) and the
// alerted plans still open in this direction. Both lines are written by
// exposure.py; the page prints them as they come, no arithmetic here.
const notes = [r && r.size_line, r && r.open_line].filter(Boolean)
.map(t => `<div class="plan-trail">${esc(t)}</div>`).join("");
return `<div class="plan ${buy ? "sig-buy" : "sig-sell"}"> return `<div class="plan ${buy ? "sig-buy" : "sig-sell"}">
<div class="plan-head"> <div class="plan-head">
<i class="fa-solid ${buy ? "fa-arrow-trend-up" : "fa-arrow-trend-down"}" aria-hidden="true"></i> <i class="fa-solid ${buy ? "fa-arrow-trend-up" : "fa-arrow-trend-down"}" aria-hidden="true"></i>
@@ -618,7 +624,7 @@ function planHTML(ts) {
<div><dt><i class="fa-solid fa-flag-checkered" aria-hidden="true"></i>Target 2</dt> <div><dt><i class="fa-solid fa-flag-checkered" aria-hidden="true"></i>Target 2</dt>
<dd class="up">${fmt$(ts.take_profit_2)}</dd></div> <dd class="up">${fmt$(ts.take_profit_2)}</dd></div>
</dl> </dl>
<div class="plan-trail">${esc(trail)}</div> <div class="plan-trail">${esc(trail)}</div>${notes}
</div>`; </div>`;
} }
@@ -673,6 +679,9 @@ function renderCard(tf) {
: actionTail(r.action)); : actionTail(r.action));
// Plain words for the two numbers below (from main.plain_words, same on every surface). // Plain words for the two numbers below (from main.plain_words, same on every surface).
f("plain").toggleClass("d-none", !r.plain_words).text(r.plain_words || ""); f("plain").toggleClass("d-none", !r.plain_words).text(r.plain_words || "");
// The daily trend in words (main.daily_trend_words). On M15 a plan against
// a clear daily trend is removed by the signal core; elsewhere this is context.
f("trend").toggleClass("d-none", !r.daily_trend_words).text(r.daily_trend_words || "");
const score = r.score == null ? 0 : r.score; const score = r.score == null ? 0 : r.score;
f("score").text(signed(r.score, 1)) f("score").text(signed(r.score, 1))
@@ -692,7 +701,7 @@ function renderCard(tf) {
r.mtf_agreement ? "chip-up" : "") + r.mtf_agreement ? "chip-up" : "") +
(r.exhaustion_count ? chip("fa-battery-quarter", `${r.exhaustion_count} exhaustion signs`, "chip-warn") : "")); (r.exhaustion_count ? chip("fa-battery-quarter", `${r.exhaustion_count} exhaustion signs`, "chip-warn") : ""));
f("plan").html(planHTML(r.trade_setup)); f("plan").html(planHTML(r.trade_setup, r));
f("flags").html(flagsHTML(r, 2)); f("flags").html(flagsHTML(r, 2));
updateIndChart(tf); updateIndChart(tf);
} }
@@ -838,11 +847,13 @@ function detailHTML(tf) {
${tile("Event risk", esc(r.event_risk || "—"), r.event_risk === "clear" ? "up" : r.event_risk ? "text-warning" : "")} ${tile("Event risk", esc(r.event_risk || "—"), r.event_risk === "clear" ? "up" : r.event_risk ? "text-warning" : "")}
${tile("News tone", ns != null ? signed(ns, 2) : "—", ns > 0.05 ? "up" : ns < -0.05 ? "down" : "")} ${tile("News tone", ns != null ? signed(ns, 2) : "—", ns > 0.05 ? "up" : ns < -0.05 ? "down" : "")}
${tile("Timeframes", r.mtf_agreement ? "agree" : "mixed", r.mtf_agreement ? "up" : "")} ${tile("Timeframes", r.mtf_agreement ? "agree" : "mixed", r.mtf_agreement ? "up" : "")}
${tile("Daily trend", r.daily_trend == null ? "—" : r.daily_trend > 0 ? "rising" : r.daily_trend < 0 ? "falling" : "no clear direction",
r.daily_trend > 0 ? "up" : r.daily_trend < 0 ? "down" : "")}
${tile("This timeframe alone", signed(r.primary_score, 1), toneClass(r.primary_signal))} ${tile("This timeframe alone", signed(r.primary_score, 1), toneClass(r.primary_signal))}
${tile("Exhaustion signs", esc(r.exhaustion_count == null ? "—" : r.exhaustion_count), r.exhaustion_count ? "text-warning" : "")} ${tile("Exhaustion signs", esc(r.exhaustion_count == null ? "—" : r.exhaustion_count), r.exhaustion_count ? "text-warning" : "")}
</div>`); </div>`);
out += sec("Trade plan", planHTML(r.trade_setup)); out += sec("Trade plan", planHTML(r.trade_setup, r));
out += sec("Why this signal", reasonsBlock(r) || `<div class="flat">Nothing was recorded on this scan.</div>`); out += sec("Why this signal", reasonsBlock(r) || `<div class="flat">Nothing was recorded on this scan.</div>`);
out += sec(`${TF_LABEL[tf]} indicators`, out += sec(`${TF_LABEL[tf]} indicators`,
analysisBlock(pa, price) || indTable(r.indicators) + analysisBlock(pa, price) || indTable(r.indicators) +
+275 -43
View File
@@ -74,11 +74,13 @@ from app_config import (
from providers import BaseDataProvider, make_data_provider from providers import BaseDataProvider, make_data_provider
from providers.base import _is_daily_maintenance from providers.base import _is_daily_maintenance
from agent_panel import attach_panel, panel_health from agent_panel import attach_panel, panel_health
from exposure import attach_exposure, size_shared
from llm_client import get_llm_client from llm_client import get_llm_client
from news_engine import FundamentalContext, NewsEngine from news_engine import FundamentalContext, NewsEngine
from pair_signals import PAIR_PRODUCERS from pair_signals import PAIR_PRODUCERS
from signal_aggregator import SignalAggregator, SignalReport from signal_aggregator import SignalAggregator, SignalReport
from signal_log import gate_reason from signal_log import (gate_reason, last_alerted_bars, mark_alerted,
unmark_alerted)
from violent_watch import attach_watch, should_send_headsup from violent_watch import attach_watch, should_send_headsup
if TYPE_CHECKING: # type hints only — ViolentWatch is never needed at runtime here if TYPE_CHECKING: # type hints only — ViolentWatch is never needed at runtime here
@@ -199,6 +201,16 @@ TF_CONFIRMATIONS = {
} }
_executor = ThreadPoolExecutor(max_workers=4) _executor = ThreadPoolExecutor(max_workers=4)
# ONE worker for an alert's database steps (refresh the "already open" line,
# stamp the row as alerted, undo the stamp). A single worker is the mutual
# exclusion: one alert's read and write run back to back, in arrival order,
# so of two alerts sent at the same candle close the second counts the
# first. It also keeps SQLite off the event loop (the logger's lock can be
# held for seconds by the outcome labeller) and never queues behind the
# scan pool above. Not an asyncio.Lock: on Python 3.9 one created at
# import binds to a loop that is not the running one.
_alert_db_executor = ThreadPoolExecutor(
max_workers=1, thread_name_prefix="alert-db")
# ── Markdown escaping ───────────────────────────────────────────────── # ── Markdown escaping ─────────────────────────────────────────────────
@@ -409,7 +421,23 @@ def _plain_words_line(report: SignalReport) -> str:
f"Reliability: *{reliability_words(report.final_confidence)}* · {agree}") f"Reliability: *{reliability_words(report.final_confidence)}* · {agree}")
def format_signal_message(report: SignalReport) -> str: def _daily_trend_line(report: SignalReport) -> str:
""""🧭 Daily trend: rising — this buy goes with it" (main.daily_trend_words,
the same words on every surface), or "" when the scan carries no daily
trend. On M15 a plan against a clear daily trend never reaches an
alert (the core removes it); on the other timeframes the line is
information only."""
from main import daily_trend_words # display helper, no side effects
words = daily_trend_words(report)
return f"🧭 {_escape_md(words)}" if words else ""
def format_signal_message(report: SignalReport, channel: bool = False) -> str:
"""The alert / scan text. `channel=True` builds the copy for every
audience but the owner's own chat (the subscriber channel, a group
the bot was switched on in): identical, except that the lot-size line
(which reveals the owner's risk per trade) is left out unless
config.json sets trade_size.telegram_channel."""
sig = report.final_signal sig = report.final_signal
emoji = _SIGNAL_EMOJI.get(sig, "⬜") emoji = _SIGNAL_EMOJI.get(sig, "⬜")
qe = _QUALITY_EMOJI.get(report.signal_quality, "") qe = _QUALITY_EMOJI.get(report.signal_quality, "")
@@ -431,6 +459,9 @@ def format_signal_message(report: SignalReport) -> str:
_plain_words_line(report), _plain_words_line(report),
f"📊 Regime: {regime_str} · Vol: {vol_str}", f"📊 Regime: {regime_str} · Vol: {vol_str}",
] ]
trend_line = _daily_trend_line(report)
if trend_line:
lines.append(trend_line)
if report.action_recommendation: if report.action_recommendation:
# Pattern identifiers use underscores, which legacy Telegram Markdown # Pattern identifiers use underscores, which legacy Telegram Markdown
# can misparse even when escaped inside this italicized action line. # can misparse even when escaped inside this italicized action line.
@@ -457,6 +488,15 @@ def format_signal_message(report: SignalReport) -> str:
f"▶ Entry `{_fmt_price(ts.entry_price)}`", f"▶ Entry `{_fmt_price(ts.entry_price)}`",
f"🛑 SL `{_fmt_price(ts.stop_loss)}` (${risk_usd:.2f} / {ts.risk_pips:.0f}p = 1R)", f"🛑 SL `{_fmt_price(ts.stop_loss)}` (${risk_usd:.2f} / {ts.risk_pips:.0f}p = 1R)",
] ]
# Lot size for the owner's risk limit and the alerted plans still
# open in this direction (exposure.attach_exposure stamps both
# post-hoc; empty unless configured / unless something is open).
# The size is private: the channel copy carries it only when
# config.json says so.
if report.size_line and (not channel or size_shared("telegram_channel")):
lines.append(f"📏 {_escape_md(report.size_line)}")
if report.open_line:
lines.append(f"🧱 {_escape_md(report.open_line)}")
# Trailing-stop management plan (matches backtested exit model: # Trailing-stop management plan (matches backtested exit model:
# TP1 is advisory, the trail is the exit). Params are stamped on # TP1 is advisory, the trail is the exit). Params are stamped on
@@ -611,7 +651,13 @@ def _run_signal_sync(
config = PAIR_PRODUCERS[symbol](timeframe, confirmation_tfs) config = PAIR_PRODUCERS[symbol](timeframe, confirmation_tfs)
aggregator = SignalAggregator(config) aggregator = SignalAggregator(config)
aggregator.data_provider = _get_shared_dp() aggregator.data_provider = _get_shared_dp()
return aggregator.analyze_pair(symbol=symbol, log_source="telegram") report = aggregator.analyze_pair(symbol=symbol, log_source="telegram")
# Display-only lot-size and "already open" lines (never raises; one
# read-only query on signals.db). Stamped here, on the scan executor,
# for the manual scans. An ALERT refreshes them just before it is sent
# (_deliver_alert): every timeframe scans at the same candle close, so
# the line stamped here cannot know about an alert going out beside it.
return attach_exposure(report, surface="telegram")
async def run_signal( async def run_signal(
@@ -761,11 +807,10 @@ def _key_bar_seconds(key: str) -> Optional[int]:
return None return None
def _report_bar_ts(report: SignalReport) -> Optional[pd.Timestamp]: def _bar_label_ts(label: Any) -> Optional[pd.Timestamp]:
"""The report's primary bar label as a UTC timestamp; None when it """A bar label as a UTC timestamp; None when it cannot be parsed."""
cannot be parsed (the cooldown then does not apply)."""
try: try:
ts = pd.Timestamp(report.timestamp) ts = pd.Timestamp(label)
except Exception: except Exception:
return None return None
if pd.isna(ts): if pd.isna(ts):
@@ -773,9 +818,45 @@ def _report_bar_ts(report: SignalReport) -> Optional[pd.Timestamp]:
return ts.tz_localize("UTC") if ts.tzinfo is None else ts.tz_convert("UTC") return ts.tz_localize("UTC") if ts.tzinfo is None else ts.tz_convert("UTC")
def _report_bar_ts(report: SignalReport) -> Optional[pd.Timestamp]:
"""The report's primary bar label as a UTC timestamp; None when it
cannot be parsed (the cooldown then does not apply)."""
return _bar_label_ts(report.timestamp)
def _primary_bars_since(key: str, report: SignalReport, last: pd.Timestamp,
now_bar: pd.Timestamp) -> Optional[int]:
"""How many primary bars lie after `last`, up to and including the
report's own bar, in the frame the scan analysed (report._frames):
the backtester's position difference `pos - last_entry_pos`. None when
the report carries no frame for the key's timeframe or its bar is not
in it; the caller then falls back to clock time."""
try:
tf = Timeframe(key.rsplit("_", 1)[-1])
df = (getattr(report, "_frames", None) or {}).get(tf)
if df is None or len(df) == 0:
return None
idx = df.index
if getattr(idx, "tz", None) is None:
idx = idx.tz_localize("UTC")
if now_bar not in idx:
return None
return int(((idx > last) & (idx <= now_bar)).sum())
except Exception:
return None
def _inside_alert_cooldown(key: str, report: SignalReport) -> bool: def _inside_alert_cooldown(key: str, report: SignalReport) -> bool:
"""True while this key sits within _ALERT_COOLDOWN_BARS primary bars """True while this key sits within _ALERT_COOLDOWN_BARS primary bars
of its last ALERTED setup's bar (the backtester's entry cooldown).""" of its last ALERTED setup's bar (the backtester's entry cooldown).
Bars are COUNTED in the scan's own primary frame, like the
backtester's bar positions. Clock time is not the same thing: across
the daily settlement pause or the weekend an hour or two days pass
while no bar prints, and a clock rule released the 22:00 UTC bar that
no book ever enters (six M15 and three M5 entries in three years,
2026-10-01 review). Clock time remains the fallback for a report
without frames."""
last = _last_alert_bar.get(key) last = _last_alert_bar.get(key)
if last is None or report.trade_setup is None: if last is None or report.trade_setup is None:
return False return False
@@ -783,8 +864,30 @@ def _inside_alert_cooldown(key: str, report: SignalReport) -> bool:
now_bar = _report_bar_ts(report) now_bar = _report_bar_ts(report)
if secs is None or now_bar is None: if secs is None or now_bar is None:
return False return False
elapsed = (now_bar - last).total_seconds() if now_bar < last:
return 0 <= elapsed < _ALERT_COOLDOWN_BARS * secs return False
bars = _primary_bars_since(key, report, last, now_bar)
if bars is not None:
return bars < _ALERT_COOLDOWN_BARS
return (now_bar - last).total_seconds() < _ALERT_COOLDOWN_BARS * secs
def _restore_alert_cooldowns(bars: Dict[Any, str]) -> int:
"""Seed the cooldown memory from signal_log.last_alerted_bars() after a
start-up: {(symbol, timeframe): bar label} of the newest alerted row.
Without it every restart forgot the last alert and the next bar could
alert again inside the 5 bars. Only the cooldown is restored; the
dedup fingerprints start empty, as before. A key already in memory is
left alone. Returns how many keys were seeded."""
restored = 0
for (symbol, timeframe), label in (bars or {}).items():
bar = _bar_label_ts(label)
key = f"{symbol}_{timeframe}"
if bar is None or key in _last_alert_bar:
continue
_last_alert_bar[key] = bar
restored += 1
return restored
@@ -844,6 +947,10 @@ def _should_alert(key: str, report: SignalReport) -> bool:
def _commit_alert(key: str, report: SignalReport) -> None: def _commit_alert(key: str, report: SignalReport) -> None:
# In-memory dedup state only. The signal-log stamp of the alert is
# written (and, for an undelivered alert, undone) by _deliver_alert on
# the alert-db worker: a database write does not belong on the event
# loop this function runs on.
_last_alert_fp[key] = _alert_fingerprint(report) _last_alert_fp[key] = _alert_fingerprint(report)
_last_alert_ts[key] = time.time() _last_alert_ts[key] = time.time()
_last_alert_quality[key] = report.signal_quality _last_alert_quality[key] = report.signal_quality
@@ -898,17 +1005,123 @@ async def _send_alert(bot, chat_id: int, text: str) -> bool:
return False return False
async def _broadcast(bot, text: str) -> int: def _is_owner_chat(chat_id: Any) -> bool:
"""True for the owner's own chat with the bot, the one audience of the
private lot-size line. The owner lock is per USER, so /autoscan or a
scan button used inside a group makes that group a target: it is not
the owner's chat and gets the copy without the size."""
return OWNER_ID is not None and chat_id == OWNER_ID
async def _broadcast(bot, text: str, channel_text: Optional[str] = None) -> int:
"""Send `text` to every signal target; returns how many chats got it. """Send `text` to every signal target; returns how many chats got it.
Shared by the scan and breach paths so the delivery rule cannot Shared by the scan and breach paths so the delivery rule cannot
drift between them.""" drift between them. `channel_text`, when given, is what every target
except the owner's own chat receives instead (the copy without the
owner's private lot-size line)."""
delivered = 0 delivered = 0
for chat_id in _signal_targets(): for chat_id in _signal_targets():
if await _send_alert(bot, chat_id, text): body = (text if channel_text is None or _is_owner_chat(chat_id)
else channel_text)
if await _send_alert(bot, chat_id, body):
delivered += 1 delivered += 1
return delivered return delivered
def _stamp_exposure_and_mark(report: SignalReport) -> Optional[str]:
"""On the alert-db worker: refresh the lot-size and "already open"
lines, then stamp the report's signal-log row as alerted. Read first,
write second, nothing in between: the next alert on the worker sees
this stamp. Returns the stamp to undo if no chat receives the alert
(None when nothing was written). Never raises."""
attach_exposure(report, surface="telegram")
return mark_alerted(report)
def _undo_cancelled_stamp(report: SignalReport, done: Any) -> None:
"""Done-callback of the stamp job when the alert's task was cancelled
while that job was queued or running (a shutdown, or auto-scan switched
off, at the wrong moment). Cancelling a task cannot stop a job the
worker has already started, so its stamp lands although nothing will be
sent: clear it. Runs on the alert-db worker right after the job, also
when the pool has already been told to shut down. Never raises."""
try:
if done.cancelled():
return # never ran: nothing was written
stamp = done.result()
if stamp:
unmark_alerted(report, stamp)
except Exception:
logger.warning("alert: could not clear the stamp of a cancelled "
"alert", exc_info=True)
async def _undo_alert_mark(report: SignalReport, stamp: Optional[str]) -> None:
"""The alert reached no chat: clear the stamp this attempt wrote (and
only that one), so the row does not count as a plan the reader holds.
Never raises."""
if not stamp:
return
try:
await asyncio.get_running_loop().run_in_executor(
_alert_db_executor, unmark_alerted, report, stamp)
except Exception:
logger.warning("alert: could not clear the alerted stamp of an "
"undelivered alert", exc_info=True)
async def _deliver_alert(bot, key: str, report: SignalReport, tf_label: str,
header: str = "", kind: str = "alert") -> int:
"""Send one alert: panel → exposure lines + alerted stamp → both
copies → commit. The ONE delivery path of the candle scan and the
level break; the caller holds _alert_lock(key) and has passed
_should_alert. Returns how many chats received it.
The row is stamped BEFORE the send so that an alert going out at the
same candle close on another timeframe already counts this plan in its
"already open" line (2026-10-01: stamped after the send, two alerts at
one boundary never counted each other). A send that reaches no chat
undoes the stamp and commits nothing, so the setup stays eligible for
the next scan; so does a cancellation before the send has started.
Residue: if this send fails completely after another alert has already
counted it, that alert's line is one too high."""
loop = asyncio.get_running_loop()
# Panel AFTER the quality gate + dedup decision so no LLM spend on
# suppressed alerts. Runs on the scan executor to keep the event loop
# free; this key's alert lock is held for the panel's wall time
# (≤~25 s), which only delays a same-key duplicate that _should_alert
# would drop anyway.
await loop.run_in_executor(_executor, attach_panel, report)
# submit(), not run_in_executor(): the job's own future is kept so that
# a cancellation HERE (nothing has been sent yet) can still undo the
# stamp the job writes.
job = _alert_db_executor.submit(_stamp_exposure_and_mark, report)
try:
stamp = await asyncio.wrap_future(job, loop=loop)
except asyncio.CancelledError:
job.add_done_callback(
functools.partial(_undo_cancelled_stamp, report))
raise
delivered = 0
try:
detail = header + format_signal_message(report)
delivered = await _broadcast(
bot, detail, header + format_signal_message(report, channel=True))
except Exception:
# (A cancellation is not an Exception: on shutdown mid-send a chat
# may already hold the message, so the stamp stays.)
await _undo_alert_mark(report, stamp)
raise
if not delivered:
await _undo_alert_mark(report, stamp)
return 0
# Commit only when the alert reached at least one target; a fully
# failed send stays eligible for the next scan.
_commit_alert(key, report)
logger.info(_alert_sent_line(report, tf_label, delivered, kind=kind))
return delivered
# ── Candle-boundary scheduler ──────────────────────────────────────── # ── Candle-boundary scheduler ────────────────────────────────────────
_autoscan_chats: Set[int] = set() _autoscan_chats: Set[int] = set()
@@ -1062,20 +1275,7 @@ async def _scan_timeframe(
async with _alert_lock(key): async with _alert_lock(key):
if not _should_alert(key, report): if not _should_alert(key, report):
continue continue
# Panel AFTER the quality gate + dedup decision so no LLM await _deliver_alert(bot, key, report, tf.value)
# spend on suppressed alerts. Runs on the scan executor to
# keep the event loop free; this key's alert lock is held
# for the panel's wall time (≤~25 s), which only delays a
# same-key duplicate that _should_alert would drop anyway.
await asyncio.get_running_loop().run_in_executor(
_executor, attach_panel, report)
detail = format_signal_message(report)
delivered = await _broadcast(bot, detail)
# Commit only when the alert reached at least one target;
# a fully failed send stays eligible for the next scan.
if delivered:
_commit_alert(key, report)
logger.info(_alert_sent_line(report, tf.value, delivered))
# ── WebSocket level-breach monitor ────────────────────────────────── # ── WebSocket level-breach monitor ──────────────────────────────────
@@ -1222,20 +1422,16 @@ async def _handle_breach(bot, pair_symbol: str, price: float, level_name: str) -
if not _should_alert(key, report): if not _should_alert(key, report):
return return
# Same post-gate panel attach as the auto-scan path (see # The same delivery as the auto-scan path (panel, the refreshed
# _scan_timeframe for the spend/lock rationale). # "already open" line, stamp, send, commit), under the level-break
await asyncio.get_running_loop().run_in_executor( # header. The cached M15 report this path reuses can be up to a
_executor, attach_panel, report) # bar old; _deliver_alert refreshes its exposure lines.
header = ( header = (
f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed " f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed "
f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n" f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n"
) )
detail = header + format_signal_message(report) await _deliver_alert(bot, key, report, Timeframe.M15.value,
sent = await _broadcast(bot, detail) header=header, kind="breach alert")
if sent:
_commit_alert(key, report)
logger.info(_alert_sent_line(report, Timeframe.M15.value, sent,
kind="breach alert"))
def _breach_task_done(task: asyncio.Task) -> None: def _breach_task_done(task: asyncio.Task) -> None:
@@ -1517,7 +1713,9 @@ def _news_health_lines() -> List[str]:
Reads the class-level `NewsEngine._source_health` written by the Reads the class-level `NewsEngine._source_health` written by the
scan-path fetches plus the LLM client's health stamps — display scan-path fetches plus the LLM client's health stamps — display
only, never triggers a fetch. ⚠️ marks a degraded dependency; only, never triggers a fetch. ⚠️ marks a degraded dependency;
details live in the rotating log (transition-logged WARNINGs).""" details live in the rotating log (transition-logged WARNINGs).
Failure details are exception text, so they are Markdown-escaped: an
unpaired underscore in one would make Telegram reject the reply."""
lines = ["\n*News & AI:*"] lines = ["\n*News & AI:*"]
# Snapshot: a scan thread may insert a source's FIRST entry while we # Snapshot: a scan thread may insert a source's FIRST entry while we
# iterate (dict-changed-size RuntimeError); entry mutation is safe. # iterate (dict-changed-size RuntimeError); entry mutation is safe.
@@ -1536,11 +1734,13 @@ def _news_health_lines() -> List[str]:
ff = sh.get("calendar:forexfactory") ff = sh.get("calendar:forexfactory")
if cal: if cal:
if NewsEngine.source_is_failing(cal): if NewsEngine.source_is_failing(cal):
lines.append(f" ⚠️ Calendar: unavailable — {cal['detail']}" lines.append(" ⚠️ Calendar: unavailable — "
f"{_escape_md(str(cal['detail']))}"
f" ({_fmt_age(cal['err'])})") f" ({_fmt_age(cal['err'])})")
elif NewsEngine.source_is_failing(ff): elif NewsEngine.source_is_failing(ff):
lines.append(f" ⚠️ Calendar: {cal['extra']} — ForexFactory" lines.append(f" ⚠️ Calendar: {cal['extra']} — ForexFactory"
f" down: {ff['detail']} ({_fmt_age(ff['err'])})") f" down: {_escape_md(str(ff['detail']))}"
f" ({_fmt_age(ff['err'])})")
else: else:
lines.append(f" ✅ Calendar: {cal['extra']}" lines.append(f" ✅ Calendar: {cal['extra']}"
f" ({_fmt_age(cal['ok'])} ago)") f" ({_fmt_age(cal['ok'])} ago)")
@@ -1548,13 +1748,27 @@ def _news_health_lines() -> List[str]:
if heads: if heads:
failing = NewsEngine.source_is_failing(heads) failing = NewsEngine.source_is_failing(heads)
mark = "⚠️" if failing else "✅" mark = "⚠️" if failing else "✅"
detail = f" — {heads['detail']}" if failing else "" detail = (f" — {_escape_md(str(heads['detail']))}"
if failing else "")
ts = heads["err"] if failing else heads["ok"] ts = heads["err"] if failing else heads["ok"]
lines.append(f" {mark} Headlines: {heads['extra']}{detail}" lines.append(f" {mark} Headlines: {heads['extra']}{detail}"
f" ({_fmt_age(ts)} ago)") f" ({_fmt_age(ts)} ago)")
# The local headline scorer (news_model modes fallback / primary).
# Present only in a process that ran it; a failing batch leaves the
# cycle's headlines on keyword scores, which nothing else showed.
model = sh.get("news_model")
if model:
if NewsEngine.source_is_failing(model):
lines.append(" ⚠️ News model: not scoring — "
f"{_escape_md(str(model['detail']))}"
f" ({_fmt_age(model['err'])})")
else:
lines.append(f" ✅ News model: {model['extra']}"
f" ({_fmt_age(model['ok'])} ago)")
for name, entry in sorted(sh.items()): for name, entry in sorted(sh.items()):
if name.startswith("rss:") and NewsEngine.source_is_failing(entry): if name.startswith("rss:") and NewsEngine.source_is_failing(entry):
lines.append(f" ⚠️ {name[4:]}: {entry['detail']}" lines.append(f" ⚠️ {_escape_md(name[4:])}: "
f"{_escape_md(str(entry['detail']))}"
f" ({_fmt_age(entry['err'])})") f" ({_fmt_age(entry['err'])})")
client = get_llm_client() client = get_llm_client()
if not client.available: if not client.available:
@@ -1786,7 +2000,12 @@ async def cb_scan(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
try: try:
report = await run_signal(btn["symbol"], btn["tf"]) report = await run_signal(btn["symbol"], btn["tf"])
text = format_signal_message(report) # The reply lands in the chat the button was pressed in. Only the
# owner's own chat gets the private lot-size line; the owner
# pressing it inside a group must not show the group their risk.
chat = update.effective_chat
text = format_signal_message(
report, channel=not _is_owner_chat(chat.id if chat else None))
except Exception as exc: except Exception as exc:
logger.exception("Signal scan failed") logger.exception("Signal scan failed")
text = f"❌ Scan failed: {_escape_md(str(exc))}" text = f"❌ Scan failed: {_escape_md(str(exc))}"
@@ -1914,6 +2133,18 @@ async def _post_init(app: Application) -> None:
if CHAT_ID: if CHAT_ID:
_autoscan_chats.add(int(CHAT_ID)) _autoscan_chats.add(int(CHAT_ID))
# The 5-bar entry cooldown lives in memory; seed it from the signal log
# so a restart (a deploy, the weekly timer) cannot re-alert inside it.
# Before the feed check: /autoscan can start the loops later.
try:
bars = await asyncio.get_running_loop().run_in_executor(
_alert_db_executor, last_alerted_bars)
restored = _restore_alert_cooldowns(bars)
logger.info(f"Alert cooldown restored from the signal log: "
f"{restored} timeframe(s)")
except Exception:
logger.exception("Alert cooldown restore failed — starting without it")
if not _feed_active(): if not _feed_active():
logger.info("Neither TELEGRAM_CHAT_ID nor TELEGRAM_CHANNEL_ID set — " logger.info("Neither TELEGRAM_CHAT_ID nor TELEGRAM_CHANNEL_ID set — "
"auto-scan disabled on boot. Use /autoscan in chat to enable.") "auto-scan disabled on boot. Use /autoscan in chat to enable.")
@@ -1980,6 +2211,7 @@ async def _post_shutdown(app: Application) -> None:
logger.info("Shutting down ForexBot2...") logger.info("Shutting down ForexBot2...")
_stop_scheduler() _stop_scheduler()
_executor.shutdown(wait=False) _executor.shutdown(wait=False)
_alert_db_executor.shutdown(wait=False)
logger.info("Shutdown complete") logger.info("Shutdown complete")
+2998 -14
View File
File diff suppressed because it is too large Load Diff