207 Commits

Author SHA1 Message Date
Harwinder Sekhon 2cc5915b00 Daily-trend veto on M15 + lot-size / already-open lines (campaign 2026-09-30), and the 2026-10-01 whole-codebase bug review
Two bodies of work that touch the same files, so one commit.

Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule,
written before the runs):
- SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto
  "both"). A finished setup against a clear daily trend (closed D1 bars, Sunday
  stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier)
  is removed as the last step of the core. One definition
  (signal_aggregator.daily_trend_series) for live, the bar replay and the twin's
  host side; D1 is a required frame of the M15 scan; mirrored in
  cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs
  1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs
  2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R.
- FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1
  (−9.7R), buy-only and sell-only on M15, the wide-stop cap
  (xau_strategy.wide_stop_cap_atr stays 0.0, a research knob).
- New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins
  unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15
  pins byte for byte. Twin and reference script MD5-equal on every window;
  recorder and feature-export parity re-proved; base-rate table rebuilt.
- DISPLAY (books byte-identical): exposure.py — the lot size for
  trade_size.risk_per_trade (new config.json block, off by default, private to
  the owner) and the count of alerted same-direction plans still open
  (signals.alerted_at, read-only open_alerted_setups); the daily trend in words
  on Telegram, the CLI and the dashboard; signals.daily_trend logged.
- providers/oanda: a date window with no bars is an empty page, not an error
  (a D1 pad starting on a Saturday could not be fetched).
- Reports only, nothing shipped: seven context screens (no state nominated) and
  eight outside data sources (none adds a forecast to gold's own daily trend).
- Independent verification (report section 5.13): every figure reproduced on
  both scripts, scoring identical to the previous commit on all four
  timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not
  a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022;
  the neighbour clause is thin (a simple 100-day average misses the R line by
  1.2R); "never adds a trade" is an observation, not a guarantee.

Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects
elsewhere, plus 4 found by an independent review of the fixes; all fixed, each
with a test; no trade book changed:
- telegram_bot: one _deliver_alert for the candle scan and the level break. The
  row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and
  un-stamped when no chat received it or the task is cancelled mid-stamp, so
  two alerts at one candle close count each other and no SQLite write runs on
  the event loop. The 5-bar cooldown counts bars of the scan's own frame
  (== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after
  the settlement pause) and is restored from signals.db at start-up. The
  lot-size line goes to the owner's own chat only (a group the owner ran
  /autoscan in used to receive it). /status gains a news-model line and escapes
  failure details.
- signal_log: mark_alerted returns the stamp it wrote; unmark_alerted;
  last_alerted_bars (read-only).
- advisory_outcome: candle continuity is judged bar by bar inside the walk; a
  hole after a trade's exit no longer voids it. The stored-candle replay
  resolves 108 of 111 logged setups instead of 99, all equal to the live
  resolver's R.
- providers/oanda: an empty date window is accepted only when it is 5 days or
  shorter and no malformed candle was skipped; a longer one is retried, then
  refused, instead of being cached as history with a hole.
- news_engine: blank RSS titles skipped; one unreadable headline no longer
  costs the local model its whole batch (checked on the real bundle); model
  health recorded; no DeepSeek audit call in a process that does not archive;
  event-direction cache keyed per release; peek_week_events never fetches.
- llm_client: chat_json never raises and never logs the API key (requests
  quotes the Authorization value in InvalidHeader; error bodies are redacted).
- signal_aggregator: the scan's news block is guarded; the replay path
  (use_news=False) assigns exactly the values it did before.
- agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input
  (event timing over the cached calendar week, headlines newest first).
  scripts/panel_replay.py replays the logged payload per persona;
  scripts/label_scan_outcomes.py counts with an ISO cutoff.
- test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34);
  tests that write Parquet print a notice and return without pyarrow.

Verification after the last code change: suite 632 passed, 0 failed (also with
both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the
five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37
green; static Python 3.9 / SQLite 3.34 check clean on every touched file.

Evidence: backtest/research/daily_trend_campaign_2026-09-30.md,
context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6,
8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7).
2026-10-01 10:28:28 -04:00
Harwinder Sekhon 42025da0b0 Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72%
agreement with DeepSeek on unseen days) replaces DeepSeek for headline
scoring. DeepSeek keeps event direction, the summary and the dashboard
endpoint.

- news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers,
  so the bot starts without them) + load-once, never-raising singleton.
- news_engine: _enrich_headline_scores mode switch (off/fallback/primary),
  _apply_model_sentiment (gold head only: xau_score + XAU entry, never
  USD; not-relevant shows 0.0; never touches _llm_sent_cache),
  _audit_with_deepseek (archive-only labels, deepseek_audit_per_day).
- Owner decision: primary mode writes nothing to logs/newsdata.db except
  audit copies — the model's output is not training data.
- app_config: news_model section (path/threads/mode/deepseek_audit_per_day)
  + NEWS_MODEL_MODE env override; validated at boot.
- news_archive: model rows carry teacher_model xau_transformer@<trained_at>.
- deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a
  commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers.
- test_all.py Module 38 (15 tests, one against the real bundle). Suite 581.
- CLAUDE.md, spec status note, handoff marked implemented.

Books unaffected: news never enters signals; no signal file touched.
2026-09-27 21:02:42 -04:00
Harwinder Sekhon d81cdf845b Conviction campaign 2026-09-27 (FAILED, research knob kept) + plain-words alert wording
Owner question: can score and confidence be combined into one number that is
easier to understand and for the system to decide on?

Research (pre-registered, nothing ships to the signal path):
- signals.grading_mode: "pair" (default, the shipped two-floor rule, byte-
  identical: all five reference MD5s reproduce on both scripts), "min"
  (min(|score|, 0.6*conf)), "product" (|score|*conf/100); thresholds fixed from
  the control's corner points, no tuning. Helpers _final_signal_and_grade /
  _transitioning_boost / conviction / CONVICTION_THRESHOLDS in signal_aggregator;
  mirrored in K2 (GRADING_MODE_CODES, PF_conv_* packed from the live module);
  twin refuses other values; M15 OOS MD5 equal on both scripts for both modes.
- Verdict: FAILED on M5 and M15. K-MIN moves 43-54 trades for -1.2R / -2.9R
  (clauses a,b,e / a,b,d); K-PROD -29R on M15 (PF 1.29, W3 DD 28.9R), neutral
  on M5. H1/H4 product +11.9R / +17.0R on W3 from extra low-score trades
  (H1 PF 1.98 -> 1.75, DD doubles): hypothesis only.
- Code review (high) before the runs: 6 findings fixed, 1 recorded
  (engine_config_fingerprint omits the switch under "pair", so live rows keep
  their fingerprint).

Delivery (books untouched): alerts lead with the grade and print the two raw
numbers in words (main.plain_words: "Direction: strong buy · Reliability: good
· timeframes agree", cut-offs = the grade table's own) on Telegram, /status,
the CLI and the dashboard card (plain_words in the snapshot); "MTF" label
replaced by "Other timeframes"; guides updated. Suite 566 passed.

Evidence: backtest/research/conviction_campaign_2026-09-27.md
2026-09-27 14:54:36 -04:00
Harwinder Sekhon 4e0a083955 Quant review 2026-09-26: alert cooldown, plain-language alerts, M15 grade-A SELL block, 1R printed targets, panel/breach fixes
Live signals.db review (2026-09-04 → 09-25, 100 unique setups replayed on the
scans' stored candles): the stop-hunt claim is refuted (71% of stop-outs ran
≥ 1R past the stop, the same as the books' 70–72%); −25.1R of the −37.8R raw
live book were consecutive-bar re-entries the backtester's 5-bar cooldown never
takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week
windows.

Delivery (book-neutral):
- telegram_bot: 5-bar alert cooldown per timeframe (_ALERT_COOLDOWN_BARS =
  backtest.cooldown_bars); setup block prints the trail first as the tested
  exit, TP1/TP2 as guides with their R, "Planned TP1 RR", and one validated-book
  line per setup class (agent_panel_base_rates.json); the informational 100-pip
  TP line is removed (owner request).
- AI block rewritten for a new trader: "AI check: the odds look better than /
  worse than / about usual", the chance "in 100" against the usual odds, one
  reason for and one against, no verdict word or confidence figure; rendered
  whenever the verdict carries a probability. agent_panel: persona prompts ban
  trading jargon (PANEL_PROMPT_VERSION v2.1-2026-09-26); the macro-source
  verdict now carries its persona's p_win (every live v2 row had
  panel_probability NULL). Dashboard and guides mirror the wording.
- telegram_bot: level-breach rescans skip the settlement-pause window (the
  20:45 M15 bar was re-logged at 22:01/22:06/22:11 every evening).

Signal (pre-registered rule-1 campaign, CUDA twin paired with gpu_backtest.py):
- M15 grade-A SELLs blocked (sell_min_quality ["A+","B"], M15 override 6):
  W2 813/+154.15R PF 1.53 vs 931/+146.14R PF 1.43, OOS +68.5 vs +60.67,
  W3 max DD 20.9 vs 19.3R. PASSED a–f.
- Printed target 1 floored at 1R in every generator (signals.tp_print_floor_rr),
  applied AFTER the min-RR and spread gates so no trade changes; mirrored in
  cuda_signal_kernels.py K2. Label-only re-pin of all four timeframes (only
  tp1/rr_planned columns move). A floor applied before the gate cost −24.6R and
  was rejected.
- FAILED: M15 min_rr_ratio 1.0 (trade-identical, −2.0R). Owner's call: M5
  tp_floor_rr 1.0 (+48.9R W2, max DD 10.1 → 15.9R), not shipped.

New pins: M5 W2 e599c8a8…, M15 W2 5b65ecf6…, M15 OOS 09b8a8e7…, H1 W3
a44fd19b…, H4 W3 f3587bae…; twin and reference MD5s equal on every window.
Base-rate table rebuilt from the new pins. Suite 558 passed.

Evidence: backtest/research/rr_and_sell_campaign_2026-09-26.md (plus the
2026-09-18 loss-anatomy and wrong-direction reports it builds on).
2026-09-26 21:06:05 -04:00
Harwinder Sekhon 266f204d35 dashboard: mark neutral votes and no-pattern rows on the indicator chart
A NEUTRAL vote (value 0) and an absent divergence/candlestick entry both
drew no bar, so RSI, Div and Cndl looked missing on the live page. The
engine computes them on every scan; divergence and candlestick only exist
when a pattern is found (by design, "no opinion"). The chart now draws a
grey dot on the zero line for a neutral vote and a faint "no pattern" tag
for an absent pattern indicator; rows hover as a whole. Display-only,
signal code and books untouched. Suite 554/554.
2026-09-22 14:29:21 -04:00
Harwinder Sekhon 3005e57367 dashboard: rebuild the page on Bootstrap 5 + jQuery + Chart.js + Font Awesome
One card per timeframe (M5/M15/H1/H4) in a Bootstrap column grid, above a
direction board that charts every timeframe's score as a diverging bar.
Each card carries its own indicator-vote chart, and the newswire gets a
headline-tone bar. Bootstrap Icons and the hand-rolled CSS gauges are gone;
dashboard.css is now the token set plus the app-specific pieces only.

Chart marks use a diverging pair validated on this surface (#0d9f7a /
#e8503a: lightness band, chroma floor, colour-vision separation, contrast).
Lighter steps of the same two hues carry the colour where it has to work as
text, and direction never travels as colour alone -- an arrow icon and the
word ride with it.

Behaviour is unchanged: same snapshot.json / news.json contract, the same
coordinated refresh loop (boundary poll, 60 s fallback, 15 s retry,
in-flight guard, stale-snapshot guard), per-timeframe change detection,
modal state preservation, HTML escaping, the news_summary.php contract and
the warming / scan-failed / weekend states. Nothing on the signal path is
touched, so no book moves.

Verified headlessly against a fixture replayed through the real signal code
(wide / laptop / tablet / mobile, both dialogs, every degraded state): no
console errors, html-validate clean.

Suite 553 -- the old static-UI test splits into one that pins the CDN stack
and one that pins a card and a chart per timeframe against
dashboard_publisher.DASHBOARD_TFS.
2026-09-21 19:25:52 -04:00
Harwinder Sekhon 32ad73c59a backtest: full-GPU research twin (cudaBasedBacktesting.py + cuda_signal_kernels.py); books byte-identical to gpu_backtest.py (2026-09-18)
Owner decisions: research only (live scans stay on the CPU code), gpu_backtest.py
stays the base reference, every research flag supported, the one exemption from
instructions item 6 recorded in backtest/instructions_for_backtest.txt.

- backtest/cuda_signal_kernels.py: the CUDA decision stage. K1 = engine.py + the
  GPUTechnicalEngine overrides (one thread per bar), K2 = analyze_from_dataframes
  end to end incl. gates 6b-6i, the four setup generators and the regime filter
  (one thread per primary bar), K3 = position_sizing.build_features, K4 cooldown,
  K5 prefix median, K6 recorder bar rows. Config packer with generated #defines;
  leaf module; --fmad=false; Python evaluation order kept (ternary min/max,
  pairwise sums, Neumaier sum(), exact round(x, 2)).
- backtest/cudaBasedBacktesting.py: same CLI, cache files and writers as the
  reference; the host turns kernel codes into the exact strings; no CPU path.
- test_all.py Module 37 (16 tests): every stage pinned against the live code;
  suite 552 / 552 on the GPU, without CUDA and with FB2_BACKEND=cpu.
- Proof: five reference MD5s, four 6-month acceptance books (M5/M15, $0.65/$0.30),
  M15 OOS --export-features and --record --record-bars tables, ten research-flag
  variants, and the owner's five-year M5 and M15 books, all byte-identical to
  gpu_backtest.py. Wall time 0.8-6.6 s per book against 13-353 s.
- CLAUDE.md sections 1, 3, 6, 7, 9, 10, 11 and the instructions addendum: any
  signal change is now a parity event for the twin.
2026-09-18 16:11:47 -04:00
Harwinder Sekhon 02589cbb88 backtest: real CuPy GPU backend (kernels are bit-identical CPU twins); books byte-identical on both backends (2026-09-18)
CuPy from pip had never compiled a kernel on the dev box (no CUDA_PATH for the
splayed CUDA-12 wheel layout); the probe swallowed the error and every book to
date was built on the NumPy path under a "GPU backtester" name.

- backtest/gpu_kernels.py (new): lazily probed, never-silent BACKEND state
  (CUDA_PATH pointed at the nvidia-cuda-runtime-cu12 header wheel; the probe
  compiles a kernel and keeps the failure reason), --backend auto|gpu|cpu and
  FB2_BACKEND, and three CUDA kernels compiled with --fmad=false: ewm_batch
  (the _gpu_ewm_alpha recurrence, one thread per series), roll_mean_var
  (pandas 3.0.2 roll_mean/roll_var replica, Kahan + Welford + unstable
  recompute), resolve_trades (trailing and static exits, one thread per trade).
- gpu_backtest.py: precompute_indicators and simulate_trades dispatch to the
  GPU twins or to the untouched NumPy paths; caches are host NumPy and device
  memory is released before the replay workers fork; the Backend: line names
  the device or the fallback reason.
- test_all.py Module 36 (7 tests): a visible CUDA device must never fall back
  silently; kernels vs their CPU references on synthetic edge cases; backend
  selection; import guards (cupy banned from the live roots). Suite 536.
- CLAUDE.md, instructions_for_backtest.txt, research note with the evidence.

Verification: all five reference MD5s reproduced on the GPU and with
--backend cpu (ten runs, GPU cache fallbacks 0); precompute GPU == CPU on all
42 cached windows (2.34 M bars); suite green with GPU, with
CUDA_VISIBLE_DEVICES="" and with FB2_BACKEND=cpu; --workers 1 and --record
pairs equal. The per-bar replay stays the live Python signal code on CPU
workers by design; wall time is roughly unchanged (precompute ~3x faster).
2026-09-18 10:39:55 -04:00
Harwinder Sekhon ad14ea4fed news corpus for a local scorer + agent panel v2 + dashboard read-only (2026-09-17); books byte-identical
Agent panel v2: personas return p_win anchored to the book base rates with a
code-computed decision context; verdict = equal-weight mean (verdict_source
"mean"); only the macro persona runs live (agent_panel.personas ["macro"]);
prompt version and exact payloads logged; offline replay harness
scripts/panel_replay.py; base-rate table agent_panel_base_rates.json
(scripts/build_panel_base_rates.py). Backtester `strategy` column now carries
the generator's own stamp (label-only; five reference books re-pinned, all
62 other columns identical).

News corpus (after evaluating the owner's ML-NEWS scikit model against
DeepSeek: golden set 39/78 vs 78/78, out-of-sample agreement 64% against a
90% self-agreement ceiling; verdict: fallback only, not a replacement):
- NewsSentiment.xau_score + archive column xau_score keep DeepSeek's gold
  score for every headline (930 of the first 1,546 DeepSeek rows had none);
  an existing newsdata.db gains the column in place on first open
- headlines naming no tracked currency are scored and archived
  (currencies_json []) but never returned to the platform
- _apply_llm_sentiment sends every new headline in its cycle, 25 per call;
  a failed call ends the cycle's sending

Dashboard publisher never writes a database: main() calls
app_config.set_db_writes_for_process(False), honoured by signal_log_enabled()
and news_archive_enabled() (owner decision: the bot is the single writer).

Test suite: the hermetic app config now points both writers at a
suite-private temp dir — the RSS fixture tests were writing fixture
headlines into the real logs/newsdata.db — with a Module 32 test pinning it.
529 tests.
2026-09-17 19:49:29 -04:00
Harwinder Sekhon 183df7832b docs: rewrite CLAUDE.md as a plain-language developer guide
Same facts, new shape: quick start; the scan pipeline from trigger to
delivery to logging (A to Z) with the real thresholds and gate order;
a file-by-file guide with the import graph; data providers; config;
testing; the nine change rules; the validated per-timeframe overrides;
reference books with MD5s and the kept cache windows; a traps list;
and the decision ledger condensed to one row per dated decision with a
pointer to its research report.

Every claim was checked against the code by a reviewer pass; the
mechanism invariants the old file carried (MACD always computed, the
_-prefixed convention, PB/CP force grade B, penalties hit confidence,
gate 6b label-hour vs close-hour, the news-archive hook position, the
ordered cooldown pass, engine_fingerprint split rule, oppose guard,
holiday blindness) are kept. The previous 58 KB version is in git
history at b0b71e1.
2026-09-15 20:40:05 -04:00
Harwinder Sekhon 9adaee65d2 backtest cleanup: drop four research/one-time scripts; books and stale cache moved out (owner decisions 2026-09-15)
Discussion-first cleanup of backtest/ and scripts/. Nothing in the signal
core or the backtester changed.

Removed (last commit containing them: 931b265):
- backtest/diagnose_filters.py + its two Module 18 tests
- scripts/shadow_scan.py (Phase 4 shadow scan, never run)
- scripts/backfill_news_archive.py, scripts/migrate_signal_log_news.py
  (one-time scripts; the interning test keeps its live-path half, the
  prune test goes)

Kept: the replay-recording stack (replay_record.py, replay_scorecard.py,
ff_calendar_scraper.py, cache/calendar.parquet, Module 33).

Local gitignored data was moved to ~/Development/ForexBot2_trash_2026-09-15
(346 MB) for the owner to delete: all 35 pinned backtest/*.csv books, the
three research output folders, the BTC/ETH cache snapshots, the Tiingo
parquets and 13 uncited Oanda windows. The three research-folder ignore
rules are dropped with the folders.

Verification: all five gate books regenerate offline from the pruned cache
with the recorded MD5s (M5 W2 29120a1e, M15 W2 d185f099, M15 OOS 6f8f24b4,
H1 W3 3b45d12b, H4 W3 e75adae1; NumPy CPU, 0 cache fallbacks). Suite
506 passed, 0 failed, 33 modules.
2026-09-15 20:15:06 -04:00
Harwinder Sekhon c916ecf213 remove the AI market-outlook feature (delivery-side only; books byte-identical)
Owner decision 2026-09-15. Gone: market_outlook.py, the post-H1-scan Telegram
outlook DM, main.py --outlook, the outlooks table DDL/migration/writer in
signal_log.py, the market_outlook config block + MARKET_OUTLOOK_ENABLED, and
test Module 26. Last commit containing the feature: 2836024.

- app_config: "market_outlook" stays in _KNOWN_TOP as a TOLERATED, ignored
  legacy key (oanda.account_id precedent) with one warning, so a live
  config.json that still carries the block keeps booting.
- signal_log: existing DBs keep their orphan outlooks table and rows (no
  DROP TABLE); fresh DBs never create it.
- tests: Module 26's 4-thread concurrent legacy-DB migration test (the
  suite's only one) moved to Module 31; new test for the tolerated block.
  519 -> 509 tests, 34 -> 33 modules, 0 failed.
- docs: CLAUDE.md usage/config/file-map lines and a dated state note.

Verification: W2 books (2024-09-05 -> 2026-09-05, $0.65, NumPy CPU, 0 cache
fallbacks) byte-identical before/after — M5 29120a1ee32979431d062f210d41a9d1
(781 / +146.07R / PF 1.52), M15 d185f099482a07f29502cb5a371a9bbf
(931 / +146.14R / PF 1.43).
2026-09-15 19:58:47 -04:00
Harwinder Sekhon 7936c2fe6e audit 2026-09-13: eleven delivery/measurement fixes; books byte-identical
Signal path
- A shipped setup always names a direction: grade C needs confidence
  >= 45 (gate 6c grants B at >= 45) while BUY/SELL needs >= 50, so C/B
  reports reached the alert gate as NEUTRAL with a full setup. The
  backtester traded them (the replay never reads final_signal) while
  Telegram rejected NEUTRAL first — 5/91 trades, +3.7R of +22.2R, in
  the H1 12-month book. _attach_core_setup stamps the setup's direction
  and repairs the pre-gate action text; no-setup reports stay NEUTRAL.
- analyze_pair fails the scan when ANY required frame is missing
  ("DATA UNAVAILABLE:", scan_kind='failed'); a lost confirmation frame
  used to run with an unearned -15 MTF penalty.

Outcomes and labels
- advisory_outcome.resolve_locked_floor: a gap the database cannot fill
  resolves at the locked trailing stop (GAP_LOCKED, result.floor) when
  the stop had crossed the entry; the rebuild fires on both gap reasons
  (rows 915/920: +1.14R / +0.84R floors; resolved sum -2.46R -> -0.48R).
- label_scan_outcomes: SCAN_LABEL_MAX_AGE_H = 72 h, stale rows never
  anchor the shared bar stream, labeled_at bumped on re-examination,
  _stream_since warns at its row cap (the Labor-Day pile would have
  blinded scan_outcomes around 2026-09-14).

Providers and scheduling
- The daily settlement pause (Mon-Thu 17:00 New York) is a scheduled
  closure for _has_latest_closed_bar (whole-span label walk-back) and
  _is_data_fresh (closed seconds are not age); bot and dashboard skip
  scans whose just-closed label sits in the pause (61 false stale rows
  in three nights).
- fetch_range retries a failed page in place (three failures on one
  window raise) and logs an ERROR for interior gaps of >3 unscheduled
  bars.
- pd.Timestamp.utcnow() replaced in providers/base.py and engine.py;
  requirements pin pandas<4, numpy<3.
- telegram_bot: boundary loops share one scan per bar with the breach
  handler (_remember_scan keys on the analysed bar; breach defers 20 s
  after an M15 boundary); the rotating file handler attaches in main()
  only (the test suite used to write into logs/telegram.log).
- dashboard_publisher: catch_up_timeframes re-scans boundaries missed
  while a cycle overran or the process was down.

Backtester and engine guards
- _replay_span returns the per-span GPU cache-miss count; run_backtest
  sums, logs it (must be 0) and writes cache_fallbacks to the manifest.
- regime_weight_table raises on a partial regime_weights_override.

Verification: 519 tests; M5/M15 2-yr, M15 OOS, H1/H4 3-yr books
byte-identical before and after; live signals.db untouched (checked on
a copy). CLAUDE.md carries the dated entry and the blocked-hours union
semantics that every book was validated under.

Claude-Session: https://claude.ai/code/session_0139xz1J6YccndA2be6yFnEo
2026-09-13 21:42:37 -04:00
Harwinder Sekhon be72f3f350 research: improvement cycle 1 ledger (baseline, cost ladder, C1 reject, C2 plateau, seeding parity)
backtest/research/cycle_2026-09-10.md: pre-registered rules, HEAD baseline
on the 3-year window (three pinned MD5s reproduced), cost ladder and
concentration facts, bucket screen (one cycle-2 candidate: M15 grade-A
SELLs), C1 H1 divergence->0 REJECTED, C2 M15 RSI-multiplier ladder =
stable plateau, seeding-depth measurement and the two shipped parity
fixes. CLAUDE.md: live fetch depth, backtester pad, 501 tests, the
2026-09-10 state entry; corrected the docs note (no PDF builds exist)
and recorded pyarrow as a dev-venv-only dependency.

Claude-Session: https://claude.ai/code/session_01NQDjW8rXYrJR4JDWMWKUij
2026-09-10 08:29:14 -04:00
Harwinder Sekhon c742c65ef6 backtest: seed slow-TF history to the live 1000-bar floor (_history_pad_days)
The per-TF warm-up pad before --start targeted 210 bars, so a 3-year H4
book started with 372 H4 bars and ~210 D1 bars of seeding while the
live scanner now fetches at least MIN_LIVE_BARS (1000) on every
timeframe. _history_pad_days keeps the 62-day intraday pad (already
>= 1488 bars, so M5/M15/H1 cache windows are untouched) and pads H4 by
234 and D1 by 1400 calendar days; the raw-path fetch for trade
resolution reuses the primary timeframe's own start (one cache file
per TF, as before).

Verified on the 3-year books at $0.65: M5 and M15 byte-identical
(c2722f98…, b2cfa9e2…); H1 and H4 trade-for-trade identical (253 /
+70.02R, 114 / +21.68R) — the only differences are one H4
confirmation reading 15 days into the H1 window and 1e-6…1e-9 numerics
in D1 Wilder recursions during Oct–Dec 2023, i.e. the window-start
seeding zone. Evidence: backtest/research/cycle_2026-09-10.md.

Tests: Module 21 "history pad seeds slow TFs like live". Suite 501 passed.

Claude-Session: https://claude.ai/code/session_01NQDjW8rXYrJR4JDWMWKUij
2026-09-10 08:29:04 -04:00
Harwinder Sekhon 1bc6a27445 providers: floor every live fetch at 1000 bars (H4/D1 indicator-seeding parity)
The engine's EMAs use ewm(adjust=False), so the first bar of a fetched
frame keeps weight (1-2/201)^n in the 200-bar trend EMA. With the 60-day
lookback the live H4 frame had 380 bars (seed weight 2.3%) and D1 300
(5%), while the backtester seeds from its whole cached history. Measured
on the deepest Oanda caches (267 H4 / 313 D1 bars sampled over two years,
raw and cleaned frames): ema_trend flipped on 2/267 H4 and 1/313 D1 bars
at those depths (composite off by up to 9.5 points) and on none from 600
bars up. MIN_LIVE_BARS = 1000 leaves a seed weight of e^-10; M5/M15
(5000) and H1 (1460) already exceed it and are unchanged. One request
either way. The backtester never calls fetch_ohlcv, so no book changes.

Tests: new Module 16 depth test; the Module 21 cache test's 300-bar
expectation becomes 1000. Suite 500 passed.

Claude-Session: https://claude.ai/code/session_01NQDjW8rXYrJR4JDWMWKUij
2026-09-10 07:55:20 -04:00
Harwinder Sekhon be4db8b2a7 xauusd M15: halve the RSI regime multiplier in every regime (M15 override 5, rule-1 validated)
Owner-approved 2026-09-09 after the pre-registered weight test
(backtest/research/weight_test_2026-09-09.md). M15-only
technical.regime_weights_override = the shipped table with rsi x 0.5
(0.5 trending family / 0.75 ranging / 0.5 transitioning), derived in the
factory and pinned literally by the Module 9 equivalence test; a Module 33
test locks that M5/H1/H4 carry no override.

Evidence, 2-year Oanda window 2024-09-05..2026-09-05 at $0.65: 793/+117.62R
PF 1.41 -> 931/+146.14R PF 1.43 (fixed halves +27.9/+0.6R vs control); OOS
2024-06-01..2025-03-30 300/+40.59R PF 1.38 -> 355/+60.67R PF 1.49; 3-year
+151.90R -> +186.61R (max DD 15.5 -> 19.3R, 2026 -4R). Parity: the factory
run with no --set flag reproduces the validated candidate books byte-for-
byte (2-yr MD5 d185f099..., OOS 6f8f24b4...). M15 canonical book re-pinned
to backtest/oanda_m15_2yr_065_20260909.csv. Docs carry a one-line note.

Claude-Session: https://claude.ai/code/session_01Nc9Ninzyi18QkWezrxeBvB
2026-09-09 17:36:41 -04:00
Harwinder Sekhon 9d7f7ff4ec research: weight-change test on the recorded replay files (pre-registered; M15 RSI x0.5 passes, not shipped)
First use of the --record evidence: a drift-adjusted per-bar screen ranked
regime-multiplier candidates, the top 3 per timeframe went through the full
2-year replay against same-harness controls plus the 2024-06..2025-03 OOS
window under a rule written before any run. M5: all three MACD candidates
fail (-10..-46R). M15: ema_trend x2 and divergence->0 (trending) fail;
RSI x0.5 in every regime passes every check (2-yr +117.6 -> +146.1R PF 1.43,
OOS +40.6 -> +60.7R PF 1.49, 3-yr +151.9 -> +186.6R) with the caveats recorded
(2026 -4R, 3-yr max DD 15.5 -> 19.3R, added trades mostly B-grade BUYs).
Nothing shipped; the passing candidate is an owner decision.

Claude-Session: https://claude.ai/code/session_01Nc9Ninzyi18QkWezrxeBvB
2026-09-09 17:26:21 -04:00
Harwinder Sekhon daa2369ccb backtest: --record/--record-bars Parquet recorder + 2-year M5/M15 replay with indicator weights and calendar
Adds an opt-in recorder to the backtester so a replay keeps the evidence each
trade setup was decided on: every indicator's vote, value, confidence, base
weight, regime multiplier, effective weight and contribution on the primary and
both confirmation timeframes, penalties, the setup and outcome, and the Forex
Factory calendar events around the trade (4 h before entry to exit, all-day rows
on the New-York date). --record-bars adds one float32 row per analyzed bar with
the same votes plus 1 h/4 h/24 h forward moves in ATRs. Output is Parquet in a
folder per run with a manifest carrying the engine/gate fingerprints, the regime
weight table in force and the book MD5.

Nothing in the signal core changes: the trade book is byte-identical with
recording on or off (control-vs-recorded MD5 equal on both 2-year runs, M5
781 / +146.07R / PF 1.52, M15 793 / +117.62R / PF 1.41 at $0.65). Module 33
adds 12 tests (498 total): snapshot reproduces the engine's composite, absence
is no vote, calendar join edges/phases/holiday, write/load round-trip,
_replay_span parity with recording on, CLI flags, import and source guards.

Also: backtest/replay_scorecard.py (first-look scorecards, calendar buckets,
in-book and per-bar what-ifs, --check invariants), scripts/ff_calendar_scraper.py
restored from git history (the only way to regenerate the gitignored calendar
file), report in backtest/research/replay_recording_2026-09-09.md.

Claude-Session: https://claude.ai/code/session_01Nc9Ninzyi18QkWezrxeBvB
2026-09-09 16:41:49 -04:00
Harwinder Sekhon d05348d1e4 news archive: fix three bugs found reviewing 301bf90
1. A backfill pass silently wiped teacher identity from live rows.
   The provenance compare is `>=`, and backfill rows are llm-scored but
   carry no teacher_model/teacher_prompt_sha1 (neither source DB recorded
   one), so a bare assignment blanked what the live hook had stored.
   Re-running the backfill — e.g. after importing the live server's
   signals.db, which is the next planned step — would have erased it from
   every live row. COALESCE now keeps a stored identity when the incoming
   one is NULL, while a genuinely newer teacher still replaces it.
   Timestamps clamp for the same reason: backfill stamps historical
   values, so first_seen only moves earlier and last_seen only later.

2. A retried batch could be applied twice. executemany does not roll
   itself back, so a batch that half-landed and then hit a lock was
   re-applied whole by _with_lock_retry, double-counting seen_count on
   the rows that already landed. Writes now run in an explicit BEGIN
   IMMEDIATE with a rollback, which also takes the write lock up front so
   contention fails before any row is applied.

3. The archive hook could break a news fetch. archive_headlines swallows
   its own failures, but its ARGUMENTS are evaluated first, so a raising
   get_llm_client() escaped that guard. Guarded at the call site too:
   archiving is a side effect and must never cost us the headlines we
   just fetched.

Both 1 and 2 were verified to reproduce against 301bf90 before fixing
(teacher_model -> None; seen_count 3 instead of 2 on the rows that
landed pre-failure) and are now regression tests.

Also fixed two weak tests of my own: the disabled-by-config test never
pointed NEWS_ARCHIVE_PATH at its temp dir, so its assertion passed
regardless of behaviour and the write went to the real archive; and the
neutral-headline test asserted the headline was archived without showing
it is dropped downstream, which is the entire reason the hook sits where
it does. Positional binding of the provenance rank is now asserted rather
than trusted, since a miscount would shift every parameter silently.

483 -> 486 tests.

Claude-Session: https://claude.ai/code/session_01Bzv4ETVapFNauFDQht7QtA
2026-09-08 13:04:05 -04:00
Harwinder Sekhon e5d0260814 news archive: keep every headline + its DeepSeek score in its own DB
Headlines only ever survived as a side effect of signal logging, and
logs/signals.db was rotated around 2026-09-04 — destroying the 3-week,
4,054-headline corpus news_sentiment_value_2026-09-04.md was written on.
logs/ is gitignored, so git could not recover it and nobody noticed for
four days. There is no way to backfill a fetch window once it passes.

news_archive.py writes to its OWN database (logs/newsdata.db) so a
signal-log rotation cannot take the news with it. One row per distinct
headline (sha256 key) with the text, description, link, publisher,
publish time, the score, and — new — which scorer produced that score.

Provenance is the load-bearing part. DeepSeek scores 78/78 on the
2026-08-06 golden set; the keyword scorer it overrides scores 24/78
(31%), and that is what silently stands on no key, disabled, daily cap,
timeout, HTTP/parse failure, or the >25-new-headlines-per-cycle batch
overflow. Until now both landed in the same column, indistinguishable,
so a model trained on the mix would inherit the keyword scorer's errors.
NewsSentiment.score_source now records it, mirrored into signals.db via
signal_log._headline_to_dict, and teacher_model/teacher_prompt_sha1 keep
labels made under a changed prompt from being pooled with older ones.

Writes are one atomic UPSERT per row, upgrade-only: an llm score always
replaces a keyword one (the batch-overflow tail gets labelled on the
next window in a long-running process — verified 13/13), a keyword score
never overwrites an llm one, and re-seeing a headline always counts. A
read-then-write pair lost increments when the bot, the dashboard
publisher and the tests first saw a headline together, and the WAL
switch itself needed _with_lock_retry; both are regression-tested with
three concurrent writers.

The hook sits at the end of _fetch_news_sentiment_raw, BEFORE
_is_neutral_headline runs in get_fundamental_context. Archiving
downstream of that ±0.1 filter is why signals.db holds almost no neutral
headlines and would have taught a trained scorer that "neutral" barely
exists.

scripts/backfill_news_archive.py seeds it from both DB shapes on disk,
read-only and idempotent, recomputing provenance by comparing each
stored score against the keyword scorer (neither source recorded it):
1,016 distinct headlines, 846 DeepSeek-labelled, growing ~1,400/week.

Nothing in the signal core changed. Both news gating flags stay False,
no gate reads any of this, and the backtester never builds a NewsEngine
so replays never archive (test-locked). 468 -> 483 tests, including a
new guard that AST-walks live code for sklearn/scipy/lightgbm imports —
that check did not exist before.

Do not train yet: ~800 labels gives a holdout too small to grade
(SE +/-0.032). Target is ~4,000 deduplicated headlines over >=3 weeks.

Claude-Session: https://claude.ai/code/session_01Bzv4ETVapFNauFDQht7QtA
2026-09-08 12:35:57 -04:00
Harwinder Sekhon 2e332545a9 alerts: require a trade setup, log every send, drop stale spread-gate vetoes, daily log rotation
Delivery-side fixes from the 2026-09-08 three-day signal-log + telegram-log
review. Nothing in the signal core changes: the 3-year H1 and M15 books at
$0.65 were regenerated before/after and are identical on every trade column
(H1 253 / +70.02R / PF 1.98, M15 1092 / +151.90R / PF 1.38); only the
exported `warnings` text differs, on 50/253 H1 and 358/1092 M15 trades.

- signal_aggregator: step 7 moves into `_attach_core_setup`; a final pass
  after the PB/CP archetypes (`_drop_stale_spread_gate_vetoes`) moves a core
  generator's "SPREAD GATE: ... costs would eat the edge" veto to
  `cleaned_warnings` whenever a later generator shipped a setup. The
  2026-09-08 07:45 UTC A-grade alert carried that line beside a $26.49 TP1;
  a fifth of H1 and a third of M15 A-grade alerts had been doing the same.
  The prefix is now the `_SPREAD_GATE_PREFIX` constant shared by both
  emitters and both cleanups.
- telegram_bot: alerts require a trade setup (`_is_alert_candidate`, shared
  by the scan and breach paths; no-setup candidates log
  `alert skipped: ... (gate reason)`). The 07:50 UTC B-grade SELL with no
  setup had gone out under the same warning.
- telegram_bot: every delivered alert / breach alert / violent-move heads-up
  logs an `... sent:` INFO line with the signal-log row id (only failures
  were logged before, so the review had to reconstruct sends from panel
  rows). Send fan-out is one `_broadcast` helper for both paths.
- telegram_bot: `telegram.log` rotates daily x 30 instead of 3-hourly x 240
  (old `_HH` files on the live box need one manual delete).
- signal_log / CLAUDE.md: `cleaned_warnings_json` is no longer a CP-only
  marker; docs say so.
- test_all: 468 tests (+2): stale-veto cleanup incl. the PB-shaped case and
  the both-vetoed case, and the alert-candidate rule with its skip line; the
  rotation test asserts MIDNIGHT / interval 86400 / 30 backups.

Claude-Session: https://claude.ai/code/session_018xY4NPPrx6NJoG6QWHzwpd
2026-09-08 11:13:52 -04:00
Harwinder Sekhon d2e679cd20 providers: keep real news bars (snap-back bad-print rule), Oanda retry ladder, scan_kind=failed
2026-09-07 signal-log inspection follow-up — three fixes that do not depend on sample size.

- providers/base.py: the signal-side 10σ outlier filter is now a bad-print filter. A >10σ
  close-to-close bar is dropped only when the NEXT close snaps back ≥80% of the way
  (OUTLIER_SIGMA / OUTLIER_REVERT_FRACTION); persisting moves and the newest bar are kept.
  The unconditional rule deleted ~36 real M5 news bars/yr in the backtester (whole-frame σ)
  and a different ~24/yr live (fetch-window σ) — e.g. the 2026-09-04 12:30 UTC NFP bar
  vanished from M5+M15 and the 12:35 scan was a byte-identical repeat of 12:30. 3-yr Oanda
  drops are now M5 2 / M15 1 / H1 0.
- providers/oanda.py: transient 401/5xx/network faults retry along RETRY_DELAYS_S=(2,5,10)s
  under RETRY_BUDGET_S=45s. 11 boundary scans in two days had lost their primary frame at
  xx:00:22–:26 UTC; one 2 s retry could not outlast the top-of-hour blip.
- signal_aggregator/_log_signal: a scan whose primary fetch failed logs scan_kind='failed'
  (price 0, no signal); analyze_signal_log.parity_report skips it like 'blocked'.
- test_all.py: 466 tests (+6): retry ladder walks all delays, 5xx blip recovers on the third
  attempt, retry budget stops the ladder, persisting/newest/partially-retraced 10σ bars kept,
  failed-scan row kind, parity skip.
- CLAUDE.md: cleaning rule, retry ladder, failed scan kind, dated current-state note.

Book impact at $0.65 (rule 1 still passes everywhere, both halves positive):
  12mo  M5 557/+112.21/PF1.56 → 560/+108.81/1.54   M15 397/+73.93/1.53 → 398/+74.61/1.53   H1 identical
  3yr   M5 857/+161.3/1.53 → 870/+153.6/1.50   M15 1086/+148.7/1.37 → 1092/+151.9/1.38
        H1 255/+70.5/1.98 → 253/+70.0/1.98   H4 113/+22.7/1.63 → 114/+21.7/1.59
Only trades on the previously-deleted event bars move; the pinned M5/M15 CSVs are no longer
byte-reproducible and should be regenerated at the next re-pin.

Claude-Session: https://claude.ai/code/session_01NDnBxJ2y8GRUM6ZNFJs1hZ
2026-09-07 11:07:29 -04:00
Harwinder Sekhon 35f1eb17b8 chore: drop unreachable branches outside the signal core (no behaviour change)
- providers: _to_oanda_instrument / _to_tiingo_ticker / _to_td_symbol no longer
  guess a wire symbol for pairs that are not registered — the registries are
  the only source, so an unknown symbol raises the provider's DataIntegrityError
  instead of silently fetching something else. Every scanned symbol is validated
  against PAIR_INFO at import, so nothing reachable changes.
- main / telegram_bot _fmt_price: the sub-$10 five-decimal branch could never
  fire for gold.
- news_engine._parse_pair: BASE/QUOTE only; the Yahoo "=X" fallback was
  unreachable (its test goes with it).
- dashboard_publisher.event_to_dict: drop minutes_until (frozen at publish
  time, misleading) and is_high_impact — dashboard.js never read either.
- static/dashboard.html: unused id="cal-table".

Tried and reverted: trimming the LogRow fields nothing in analyze_signal_log
reads — the tests read them to verify the loader round-trips those DB columns,
so they stay.

Suite: 460 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:47:45 -04:00
Harwinder Sekhon c2f98938a3 signal_aggregator + engine: drop write-only counters and unreachable branches (no behaviour change)
- SignalAggregator.pb_funnel / cp_funnel and the 19 _reject() ticks inside
  _try_m5_pullback / _try_chart_pattern: class-level rejection counters nothing
  in the repo reads. Every gate keeps its early return; only the counter bump
  before it is gone.
- _TP_QUALITY_SCALE: the "B"/"C" entries and the 0.75 default could never be
  selected — the XAU ATR generator is only called for A+/A. The lookup is now
  a direct index so a violated invariant fails loudly instead of scaling silently.
- engine._classify_regime no longer takes the unused DataFrame argument, and the
  structure S/R branch tests the 5-minute frame only (M1 is never a primary).

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:46:33 -04:00
Harwinder Sekhon c004e3f30b chore: keep only what the system needs — drop research tooling, study scripts, diagram, plan
Owner directive 2026-09-06: Dukascopy and similar research material is irrelevant to
the running system; keep only code and files with a role in system functioning.

Deleted (all in git history before this commit):
- backtest/research/ tooling and data: book_report, book_diff, burst_map,
  burst_precursors, burst_indicator_map, open_trade_bursts, calendar_impact,
  watch_rule_eval, sizing_model, commit_ladder, llm_news_eval + llm_news_golden.csv,
  ff_calendar_scraper, duka_spread + duka_spread_by_hour.csv (the Dukascopy spread
  study), feed_divergence_oanda_vs_tiingo.csv, the panel dry-test JSONL, __init__.py
- scripts/feed_divergence_study.py, scripts/signal_parity_study.py (completed studies;
  their reports keep the results)
- docs/architecture.png + architecture_diagram.py (developer illustration)
- docs/data_source_change_test_plan.md (Phase 4 shadow scan stays noted in CLAUDE.md)

Kept: the campaign .md reports in backtest/research/ as the decision ledger CLAUDE.md
cites (rule 1 evidence); scripts/shadow_scan.py (pending operational step);
backtest/diagnose_filters.py (test-locked backtester diagnostic).

Local, untracked material removed on this box: .superpowers/, walkforward/, baseline/,
the commit-ladder and LLM-eval result folders, the Dukascopy boundary CSV; the
sizing_2026-09 folder is pruned to the four 3-year baseline books CLAUDE.md names.

.gitignore no longer lists the removed research outputs; the backtest instructions
and CLAUDE.md no longer point at deleted tools.

Suite: 461 passed, 0 failed (nothing imported the removed files).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:35:41 -04:00
Harwinder Sekhon de3f8a6ab8 CLAUDE.md: sync after the 2026-09-06 stale-code cleanup
- test count 461 / 31 modules
- main.py --timeframe accepts M5/M15/H1/H4 and takes confirmation TFs from the
  pair factory's map
- point the dual-cost note at the Dukascopy study that produced the $0.65
  figure (duka_spread.py is evidence, not dead code)
- current-state note listing what the cleanup removed

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:01:07 -04:00
Harwinder Sekhon c00ed4471b chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
  was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
  config.json migration that merged the same day; its plan sibling was already
  deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
  Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
  the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
  divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
  and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
  its .jsonl evidence stays

Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.

Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
  backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
  to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
  (the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
  actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
  4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
  analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
  says 4 requests by default; tiingo comment cites the NY-anchored weekend;
  Module 21 title no longer says "execution lifecycle"

Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:01:07 -04:00
Harwinder Sekhon 2502bcc396 config: remove the never-read PlatformConfig.pip_value (rule 8)
Every live reader of a pip value goes through PAIR_INFO[...]["pip_value"]
(signal_aggregator, gpu_backtest, diagnose_filters) or TradeSetup.pip_value;
no code read the PlatformConfig field. The three PlatformConfig(...)
constructions that passed it (gpu_backtest x2, diagnose_filters) and the
inline construction in the factory-equivalence test drop the kwarg. Neither
config fingerprint hashes the field, so logged fingerprints are unchanged.
The removed-field test now also asserts pip_value stays gone.

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:57:40 -04:00
Harwinder Sekhon 04bdc1737f signal_aggregator + chart_patterns: drop dead parameters and shadowed literals (no behaviour change)
- analyze_pair(context=...): no caller ever supplied it (main, telegram_bot,
  pair_signals, tests), so the value was always None; the parameter and the
  pass-through to analyze_from_dataframes go (analyze_from_dataframes keeps
  its own context parameter).
- The three specific "SELL FILTER: quality|volatility|regime" stale-prefix
  entries were shadowed by the general "SELL FILTER: " entry in the same
  startswith() tuple; every emitter carries the general prefix.
- chart_patterns: the detectors never read the Open series, so the `o`
  parameter is dropped from the three detectors, the PATTERN_DETECTORS
  lambdas and the dispatcher.
- Corrected the pb_funnel/cp_funnel comment: nothing reads those counters
  (the "read by backtest tooling" claim was stale); they stay as cheap
  diagnostics.

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:55:26 -04:00
Harwinder Sekhon 3cf3c60b89 engine + gpu_backtest: drop the write-only _kc_squeeze stamps (no behaviour change)
result._kc_squeeze / result._kc_squeeze_released were assigned by both the live
TechnicalEngine and the backtester's GPUTechnicalEngine and read by nothing:
the only four occurrences in the repo were the assignments. The squeeze state
still feeds the Keltner signal text where it is computed. Removed from both
engines in one commit (rule 4).

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:53:43 -04:00
Harwinder Sekhon 5885021dda main: build_config uses the pair factory's confirmation map (fixes an M5 divergence)
main.py kept a private confirmation table and passed it into the pair factory,
which honours any non-None list. For M5 that table said [M1, M15] while the
factory, telegram_bot.TF_CONFIRMATIONS and dashboard_publisher.CONFIRMATIONS
all use the validated [M15, H1] - so `main.py --timeframe M5` scanned with an
unvalidated pairing (and an extra M1 fetch) that no other surface used.

- build_config now returns PAIR_PRODUCERS[pair](tf, None): the factory map is
  the single source of truth. The unreachable bare-PlatformConfig fallback is
  gone here and in telegram_bot._run_signal_sync (every symbol that reaches
  either is registered).
- --timeframe offers the four validated primaries (M5, M15, H1, H4). M30 and
  D1 were never validated and silently fell through to the generic [M15, H4]
  map; they are removed from the CLI.
- The "all valid timeframes" test now asserts each CLI timeframe's
  confirmation list equals pair_signals.xauusd._CONFIRMATIONS - the check
  that would have caught this.

CLI behaviour change: M5 confirmations [M1, M15] -> [M15, H1]; M30/D1 rejected.
Bot and dashboard scans are unchanged.

Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:51:18 -04:00
Harwinder Sekhon 1b40f2166e chore: remove test-only helpers and the duplicate confirmation map (no behaviour change)
- telegram_bot._format_scan_summary: multi-pair "auto-scan summary" leftover;
  production sends per-report format_signal_message. Its only caller was a test.
- telegram_bot.scan_keyboard: thin wrapper only a test called; main_menu_keyboard
  already builds from _scan_button_rows directly.
- telegram_bot.AUTOSCAN_CONFIRMATIONS was byte-identical to TF_CONFIRMATIONS;
  the candle loop now reads the one map (app_config comment updated).
- SignalLogger.news_for: one-line wrapper over the module-level read_news that
  every real reader uses; the test now calls read_news like its neighbours.
- pair_signals.get_supported_pairs: PAIR_PRODUCERS is the registry; the test
  lists it directly.

Suite: 461 passed, 0 failed (two helper-only tests removed).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:50:21 -04:00
Harwinder Sekhon 985ec83ef3 chore: remove dead functions and write-only state (no behaviour change)
Zero callers anywhere (tests included), verified by grep/AST:
- dashboard_publisher.DashboardPublisher.publish_all (run_cycle inlines both publishes)
- signal_log.SignalLogger.advance_with_bars (the backfill script uses
  backfill_seeds / resolve_from_store / reseed_unresolvable) and the never-read
  SignalLogger.path property plus its write-only self._path
- llm_client.reset_llm_client (tests reset the singleton by assignment)
- market_outlook._last_run_ts, residue of the old "now - last < 3600" dedup;
  _last_run_hour (the UTC hour bucket) is the live key
- news_engine FundamentalContext.event_impact_on_pairs: written once, read by
  nothing; its only feeder NewsEngine._get_affected_pairs and that helper's
  unit test go with it
- position_sizing.LIVE_DEPTH_BARS (referenced only by its own docstring, which
  now states the depths inline), the unused Iterable import and the unused
  atr_ok parameter of _fill_geometry
- analyze_signal_log: unused `field` import and the unused `rows` parameter of
  persona_scorecard / news_outcome_buckets (both read the DB directly)

The two backfill/label scripts' --db help text now names the config.json
setting it actually falls back to.

Suite: 463 passed, 0 failed (one test removed with its subject).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:48:24 -04:00
Harwinder Sekhon c94f78e0b0 chore: drop unused imports and a dead constant (no behaviour change)
- backtest/diagnose_filters: unused BacktestConfig import
- pair_signals/__init__: unused PlatformConfig import
- scripts/shadow_scan: unused pandas import
- research burst_precursors / calendar_impact / duka_spread: unused sys / COST / struct
- test_all: MagicMock, StringIO, Colors and two function-local imports that
  nothing in the test bodies used
- telegram_bot: ViolentWatch is only a type hint -> imported under TYPE_CHECKING

compare_oanda_tiingo.py and oanda_td_divergence.py are left alone here because
the hygiene commit deletes them.

Suite: 464 passed, 0 failed (unchanged).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:46:15 -04:00
Harwinder Sekhon d757278b35 config: drop the dead oanda.account_id (auto-execution leftover), keep it tolerated
The candle provider needs only oanda.api_token + base_url; account_id belonged to the
auto-execution daemon removed 2026-09-03 (31002cb). Removed from config.json.example and
the unused oanda_account_id() accessor deleted; the key stays in _KNOWN_OANDA so existing
config.json files keep loading. CLAUDE.md env-var list updated. Test locks all three.

Suite: 464 passed.

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:40:44 -04:00
Harwinder Sekhon 05e71ce1e2 CLAUDE.md: remove stale facts, record the 3-year backtest preference
- test count no longer hardcoded (463 / 31 modules as of 2026-09-06)
- main.py --outlook, backtest --set / --export-features listed
- delivery surfaces include the web dashboard; SignalReport consumers include the publisher
- scikit-learn / lightgbm are research-only venv installs, never requirements
- rule 1: owner preference (2026-09-05) that backtests go back at most 3 years, with the
  3-year baseline books and where they live
- "Current state notes" header un-dated; the 2026-08-14 canonical-book line marked superseded
  by the 2026-09-03 book (M15 reference unchanged since 2026-08-04)
- docs: PDF builds noted as behind the HTML guides

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:23:46 -04:00
Harwinder Sekhon fc55da17db signal_log: weight-research data points, scan_outcomes forward labels, watch provenance
New additive columns on every logged scan: bar_ts (primary closed-bar label, indexed with
timeframe), engine_fingerprint (hash of the scoring config incl. the regime weight table in
force - SignalAggregator.engine_config_fingerprint, twin of the gate fingerprint), code_version
(short git SHA), penalty_tags_json / penalty_total / momentum_fading / exhaustion_count, and
post-hoc watch_level / watch_kinds / watch_line written by violent_watch.attach_watch via
signal_log.update_watch. Each indicators_json entry now also carries eff_weight (base weight x
regime multiplier, the weight the composite actually used) and wscore.

engine: IndicatorSignal.effective_weight / weighted_score are stamped inside _compute_composite
(record-keeping only); regime_weight_table(config) is the single source of truth for the
multiplier table (override honoured). Composite output byte-identical (120-report hash proof).

New table scan_outcomes: model-free forward moves per scan (fwd_1h/4h/24h in $ and ATRs,
mfe/mae over 24h, burst_1h, bars_used, complete) measured on the stored candle stream with the
advisory model's continuity rule; labelled opportunistically at the end of log() (time-boxed,
throttled, idempotent) - no daemon; scripts/label_scan_outcomes.py drives the backlog
(--dry-run is read-only). analyze_signal_log --scan-outcomes: forward move + hit rate by
TF x signal/grade with a NEUTRAL baseline and burst rate by watch level. Readers dedup by the
first row per (timeframe, bar_ts).

Module 31 (10 tests). Suite: 463 passed. Backtester untouched (test-locked).

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:22:59 -04:00
Harwinder Sekhon e79b533a60 violent-move watch: display-only warning line + owner heads-up (dark by default)
New violent_watch.py assesses each scan from the cached Forex Factory week events, the
UTC clock and the report's own indicator states, and stamps SignalReport.watch_line
post-hoc (telegram alerts, CLI, dashboard). A standalone heads-up goes to the owner on
the fastest scanned timeframe when the level is "hot", once per UTC hour per trigger
set, capped per day. Config block `violent_watch` (enabled False, alert_line, headsup,
max_headsup_per_day, headsup_min_level); env VIOLENT_WATCH_ENABLED.

Rule chosen from a 3-year bar-by-bar sweep (backtest/research/watch_rule_eval.py,
recorded in sizing_campaign_2026-09.md): hot = high-impact USD release within 30 min
(before or after) OR a hot clock slot (weekday 12-14 UTC, 01 UTC, Sunday 22-23 UTC)
with at least one price-extreme reading (outside Bollinger bands, RSI <30/>=70, outside
Keltner); ~8.6% of bars, next-bar odds 3-5x normal, ~4 hot hours/day. watch = clock
slot alone or two price-extreme readings (~2x). Context readings (ADX rising, extreme
volatility, transitioning, MACD acceleration) are phrases only - letting them raise the
level flagged 27% of bars at ~2x and would have sent ~13 heads-ups/day (rejected).
Direction sentence: DOWN when bearish readings dominate (70-78% on both TFs), UP on M5
only. Never a gate or exit: every open-trade protection tested lost money.

Test-locked out of analyze_from_dataframes / engine / backtester. Module 30 (18 tests).
Suite: 453 passed. Guide PDFs not rebuilt.

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 21:19:24 -04:00
Harwinder Sekhon 6bf58fd9d4 research: 3-year stake-sizing campaign + violent-move studies (nothing changes signals)
Walk-forward "second opinion" stake multiplier (meta-labelling on 65 decision-time
features, 3yr Oanda books @ $0.65) FAILED the pre-registered rule on M5 (clean null)
and M15 (lift inside the label-shuffle null, explained by BUY-vs-SELL). Nothing that
changes numbers ships; the 12-month M5 book re-run is byte-identical to the pinned one.

Behaviour-neutral plumbing:
- position_sizing.build_features (65 features; multiplier path returns 1.0, no artifact)
- SignalReport.penalty_tags / penalty_total / momentum_fading (record-keeping)
- TradeSetup.stake_multiplier = None (reserved)
- gpu_backtest --export-features (default export stays the pinned 63 columns; extras
  forced to the tail even when the first trade lacks an indicator)

Research tools (backtest/research): book_report, burst_map, burst_precursors,
open_trade_bursts, calendar_impact, burst_indicator_map, sizing_model, watch_rule_eval.
Findings recorded in backtest/research/sizing_campaign_2026-09.md: M5 stretch measures
are positive in both halves; every open-trade protection against violent bars loses on
both TFs; high-impact USD releases cause 20-31% of violent bars but trades held through
them do better; violent bars come when price is already at an extreme, not from calm.

Module 29 tests (13). Suite: 435 passed.

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 21:19:24 -04:00
Harwinder Sekhon bed8b549ac audit: fix 9 defects from the 2026-09-04 hidden-bug sweep
None of these touch the signal path. The 12-month M15 book
(2024-03-31..2025-03-30, $0.65, cached Oanda bars) is byte-identical
before and after (md5 c59d3e7e685c3c880b614640adfea74b, 375 trades),
and signal/indicator logging to logs/signals.db is unchanged — verified
end to end: per-TF indicators for all 3 TFs, 14 candles each, MTF and
reasons all still persisted.

agent_panel: the >=2-take quorum guards the CHAIR's synthesizer spend,
  but ran BEFORE the macro branch — so the shipped default
  (verdict_source="macro", which never calls the synthesizer) silently
  dropped a good macro verdict whenever the other three personas were
  slow. Quorum moved below the macro path.
analyze_signal_log: _forward_return clamped to the last available bar
  when the horizon had not elapsed, reporting a near-zero "forward move"
  for every row logged within `hours` of the end of the data — on every
  run, blended into the hit-rate/mean-move aggregate. Now returns None.
analyze_signal_log: persona_scorecard iterated raw rows through an index
  that maps EVERY member row of an event, so a re-logged scan appended
  the same event R twice. One vote per event, matching
  news_outcome_buckets. Measured inflation on the live DB: 1.03x.
signal_log: dedup required a non-negative logged_at gap, but logged_at is
  stamped before the row wins the write lock (busy_timeout 5 s), so two
  processes logging one setup at a boundary can invert timestamp order
  against id order and seed a second event. Now abs(gap); same in
  backfill_seeds. Zero occurrences in the current 366 setup rows.
advisory_outcome: the catch-all turned real evaluator faults into the
  same UNRESOLVABLE used for legitimate weekend gaps, with nothing in the
  log. Still fail-safe, now logged.
llm_client: only the "cap" warn-once flag was ever reset, so in a daemon
  the second and every later network/http/parse outage logged at DEBUG
  only. Cleared on UTC-day roll and re-armed by a recovered call.
dashboard.js: renderCards() redraws the selector for every timeframe but
  was gated on the SELECTED one changing, leaving the other buttons stale
  (up to 4 h on H4) while renderHero's rail already showed the new value.
news_summary.php: atomic_write treated a short write (disk full) as
  success, publishing a truncated body the cache-first branch then served
  forever; and the daily cap was charged before the no-key check and
  before the stampede lock re-checked the cache, so a click burst could
  exhaust the 150/day budget with zero LLM calls.
migrate_signal_log_news: logged_at is ISO with 'T' while SQLite's
  datetime() uses a space, so string-comparing them put the whole cutoff
  calendar day on the wrong side and it never pruned. NOTE: this now
  deletes one day more than before — it only runs when invoked by hand.

Tests 414 -> 422 (new Module 28, one per fix). The existing
"<2 valid takes" test was retargeted at the chair — the case it actually
protects — rather than deleted, with new coverage for macro-alone.

Not fixed, recorded for the owner:
- Gate 6c's boost condition is textually identical to the C-grade
  definition, so every C in a "transitioning" regime is promoted to B
  before gate 6e reads it, making sell_min_quality's exclusion of "C" a
  no-op on M15/H4 (H1 and M5 are immune by other means). The backtester
  runs the same code, so every pinned book already includes this —
  changing it is a rule-1 revalidation event.
- Safe-haven/geopolitical headlines score exactly 0.00 in the keyword
  fallback and are now dropped entirely by the ±0.1 neutral filter,
  contradicting the platform's own gold doctrine. Bites only when
  DeepSeek is capped/disabled/failing.

Claude-Session: https://claude.ai/code/session_01Ah5EB1goi5inQmSd37Ne6a
2026-09-04 20:19:08 -04:00
Harwinder Sekhon a3d7594f0c news: beginner-language summary, 4 h headline window, drop neutral headlines; --news log report
Verdict from the 3-week signal log (backtest/research/news_sentiment_value_2026-09-04.md):
news_sentiment is a trailing-24h echo of price (r +0.67 with the past day, +0.03 with
the next; most bullish 2 h after the 08-25 top, bearish 44 h later; most bearish at the
09-02 low, bullish 16 h later). Per-headline LLM labels are correct; the 12 h pool and
the averaging are what destroy the information. Both gating flags stay False.

- news_engine: _LLM_SUMMARY_SYSTEM rewritten for someone who has never traded (no
  acronyms, two worked examples); summary payload newest-first with the platform's own
  gold direction; reply cap 300 -> 400 chars.
- config: headline_max_age_seconds 43200 -> 14400; RSS items without a parsable
  pubDate are dropped (they used to be stamped "now" and never aged out).
- news_engine.get_fundamental_context drops headlines whose every relevant score is
  inside +-0.1 (_is_neutral_headline) before the per-currency mean; news_sentiment
  from this date is a mean over directional <=4 h headlines (log comparisons split here).
- static: dashboard copy for the 4 h pool; news_summary.php prompt gets the same style rule.
- analyze_signal_log: --news (under --stored-candles) prints sentiment lead/lag with
  split halves, sentiment at the window high/low with flip lags, and outcome buckets by
  direction x news x macro stance; LogRow gains news_sentiment/event_risk; _row_index shared.
- tests: 406 -> 414 (age window, undated drop, neutral filter, summary prompt/payload/cap,
  --news helpers, load_rows round-trip); RSS fixtures now carry a pubDate.
- docs: user/beginner guides, CLAUDE.md, deploy README.

Claude-Session: https://claude.ai/code/session_0169BgFtWjp1SEAa5392Q9jH
2026-09-04 15:14:58 -04:00
Harwinder Sekhon 593ddfe81f agent_panel: alerts use the macro persona's verdict; all 4 takes still logged
agent_panel.verdict_source defaults to "macro": the macro take's stance is the
verdict, the synthesizer call is skipped (4 LLM calls per panel instead of 5);
"chair" restores the four-voice consensus + OPPOSE guard. Verdict source is
cached per mode and logged (panel_verdict_source). 406 tests.

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 21:16:45 -04:00
Harwinder Sekhon 354778683e Remove Oanda auto-execution daemon/executor (not auto-trading)
Deletes oanda_daemon.py, oanda_executor.py, their 28 tests and the daemon-only
config keys (oanda.lot_size/quality_gate/max_hold_bars). The Oanda data
provider and credentials stay — it is the live candle feed. 402 tests.

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 21:00:21 -04:00
Harwinder Sekhon 9a8a2c1aa1 xauusd factory: block M5 shorts in transitioning regime, block all H1 shorts (rule-1 validated)
M5 sell_blocked_regimes=[transitioning]: 12mo +7.9R (PF 1.50->1.56, both halves up),
5yr +5.3R, OOS unchanged. H1 sell_min_quality=[]: 5yr +16.7R (PF 1.18->1.46),
OOS +11.5R (PF 1.13->1.74), 12mo -1.8R with both halves positive.
Campaign table, rejected and parked candidates: backtest/research/signal_log_campaign_2026-09.md

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 20:47:33 -04:00
Harwinder Sekhon b1e880be4f signal_log: outcomes for every setup, event dedup, interned news; fix over-scan, dead outlook, panel OPPOSE bias
- seed an advisory outcome on every setup row; same-setup re-logs within 60 min
  are DUPLICATE rows (advisory_event_id); resolver rebuilds bar streams across gaps
- headlines/events interned into news_snapshots (DB 667 MB -> 262 MB); readers
  use signals_with_news / read_news and mode=ro (immutable=1 missed WAL rows)
- post-gate truth columns: gate_reason, actionable, cleaned_warnings, scan_kind,
  config_fingerprint; SELL-hour gates fail closed
- telegram: breach rescan reuses this bar's report; grade dropped from alert
  fingerprint (upgrade-only re-alert)
- market_outlook: hour-bucket dedup (old rule skipped alternate hours), skip heartbeats
- agent_panel: macro-anchored OPPOSE guard (opposed trades were +30R over 3 weeks)
- analyze_signal_log --stored-candles/--parity; backfill + news migration scripts
- tests: Module 25 (analyze_signal_log) + 27 (observability); 429 total

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 20:47:33 -04:00
root 83ffd290ca agent_panel: harden evidence and track outcomes 2026-08-23 12:57:05 -04:00
root a87ed7ba63 Fix chart pattern alert Markdown 2026-08-20 21:03:30 -04:00
Harwinder Sekhon 036ddde06d dashboard: support Python 3.9 import 2026-08-18 18:08:38 -04:00
Harwinder Sekhon 410bf5ebc4 signal_log: persist market snapshots 2026-08-18 15:24:37 -04:00
Harwinder Sekhon 71eb703355 dashboard: redesign signal workspace and refresh flow 2026-08-18 14:54:47 -04:00
Harwinder Sekhon 1f632eaa6a market_outlook: add hourly advisory brief 2026-08-18 13:52:18 -04:00
Harwinder Sekhon bd62028b2f dashboard: color ladder levels red/green on cells (Bootstrap table overrides row colors), add direction icons 2026-08-16 21:02:24 -04:00
Harwinder Sekhon d4137d862a dashboard: fix gauge charts (install gauge.js, render after polls), force dark theme with white text and red/green accents, add icons 2026-08-16 20:55:25 -04:00
Harwinder Sekhon c248b39cc5 Replace Google Charts with Chart.js horizontal bars and set dark theme default 2026-08-16 20:28:54 -04:00
Harwinder Sekhon 96c926803d agent_panel: ATR-normalized candle + indicator-value enrichment, hardened prompts, verdict logging 2026-08-15 15:24:28 -04:00
Harwinder Sekhon 8b7b63853c Update gauge colors: green for positive scores, red for negative scores 2026-08-15 11:38:25 -04:00
Harwinder Sekhon 5694a1ed0c Replace score bars with Google Gauge visualization 2026-08-15 11:34:19 -04:00
Harwinder Sekhon a979dcba4b signal_log: persist agent panel verdict + per-persona takes to SQLite
Add panel_verdict, panel_confidence, panel_rationale, panel_takes_json
columns to the signals table (additive, idempotent ALTER TABLE migration
for existing DBs). _report_row now reads the panel fields attach_panel
writes onto the SignalReport, so LLM panel verdicts stop evaporating on
process exit. Also documents the Oanda re-pin (round_trip_cost stays
$0.30 for cross-book comparability and the live TP gate; $0.65 is the
measured headline) and adds the feed-divergence / parity / shadow-scan
study tooling referenced by CLAUDE.md.
2026-08-14 20:41:54 -04:00
Harwinder Sekhon be5144bc67 dashboard: redesign on Bootstrap 5, split page into external css/js
Page now uses Bootstrap 5.3 color modes (data-bs-theme), Bootstrap Icons,
and Bootstrap modals. Custom code shrinks to dashboard.css (app tokens,
score bar, signal accents) and dashboard.js (rendering/polling logic,
ported verbatim from the inline script). dashboard_publisher copies the
new css/js into the docroot; all 381 tests still green.
2026-08-13 23:30:30 -04:00
Harwinder Sekhon e609b433b0 signal_log: persist every live scan (signal, grade, price, news) to SQLite
New signal_log.py logs each SignalReport to logs/signals.db via a
fail-safe hook in SignalAggregator.analyze_pair — the live-only entry
point used by the bot, daemon, CLI and dashboard. Backtester calls
analyze_from_dataframes directly, so live==backtest parity is preserved
(test-locked). Rows carry signal/grade/price/spread, MTF + trade-setup +
reasons/warnings JSON, and news-at-scan-time (calendar events + scored
headlines JSON via NewsEngine class-level caches).

Config: signal_log block (enabled, path; env SIGNAL_LOG_ENABLED/PATH),
source tagged per consumer (telegram/oanda_daemon/cli/dashboard).
381 tests pass (24 modules).
2026-08-13 10:24:49 -04:00
Harwinder Sekhon 887aaed6f2 dashboard: skip DOM rebuild on unchanged poll, publish-time staleness, fix boundary countdowns
poll() now compares raw payload text and re-renders only what a changed
file can affect, preserving selection, disclosure, and scroll state. Hero
freshness reflects publish time (STALE warning past 15 min while market
open), and countdowns resolve their wrap interval from data-interval
instead of reverse-looking shared TF boundaries in snapshot.timeframes.
2026-08-09 18:15:31 -04:00
Harwinder Sekhon 7e8ed754ff news: reduce headline max age from 24h to 12h, make configurable
- Added headline_max_age_seconds to NewsConfig (default 43200 = 12h)
- Replaced hardcoded 86400 in _fetch_news_sentiment_raw with config ref
- All downstream consumers (keyword scoring, LLM scoring, LLM summary,
  agent panel macro/contrarian) auto-filter through the same path
2026-08-09 16:49:48 -04:00
Harwinder Sekhon 9769a0c84d agent_panel: port ai-hedge-fund prompt rubric — stance rules, anchored confidence, no-invention
Prompt-only change (static constants; plain-language output contract
byte-identical). Ported from virattt/ai-hedge-fund's persona rubric:

- _TAKE_CONTRACT hard rules: reason only from the JSON given, never
  invent numbers/events/levels; thin or missing evidence -> neutral
  with confidence below 30 (deferring to role-specific rules).
- Anchored confidence bands in the contract and synthesizer, replacing
  the unanchored "how sure you are".
- Per-stance support/oppose/neutral criteria for macro, technical, and
  risk (risk's missing-setup->oppose folded into its oppose bullet).
- One plain-language support example beside each oppose example for
  macro/technical/risk, fixing the all-oppose few-shot bias.
  Contrarian's designed skeptic tilt untouched.

News-engine prompts untouched (pinned by 78/78 + 18/18 validations).
374 tests green; real-DeepSeek smoke: macro correctly answered
neutral/20 on empty news context, confidences spread 20-60 across the
bands, all notes <=25 words, no jargon.

Claude-Session: https://claude.ai/code/session_01TyZx9iRJwfbPTPnCBqcwTR
2026-08-08 10:48:10 -04:00
Harwinder Sekhon 964239e67d docs: multi-provider LLM layer plan (DeepSeek news / Anthropic panel)
Reference plan only - no code changes. Describes the BaseLLMClient
refactor, AnthropicClient on the official SDK, per-consumer routing via
a config.json "llm" block, tests, commit sequence, and effort estimate
for the day we route the agent panel to Claude while news stays on
DeepSeek.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 23:02:08 -04:00
Harwinder Sekhon 5e3758bd53 agent_panel: plain-language output — simple, short, no jargon/acronyms
Owner request: panel notes and rationale must read like plain everyday
English for a non-trader. Style rule added to the shared take contract
and synthesizer prompt, all few-shot examples rewritten in the target
voice (examples carry the style), and the Telegram votes line now spells
out full stances ("macro: support") instead of "macro:sup".

Before: "BUY against Bollinger SELL and price above upper band; 4h
neutral, mixed MTF"  →  After: "The price is above its usual range, and
the bigger picture is not clearly rising, so buying now is risky."

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:50:10 -04:00
Harwinder Sekhon c146fffb5b docs: agent panel module + config block
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:35:15 -04:00
Harwinder Sekhon 38c81251e8 cli+dashboard: --panel flag, panel dict keys, TF-dialog panel section
main.py: --panel runs the agent panel on demand (force bypasses
enabled/surface/quality toggles; daily budget + LLM cap still apply),
prints the verdict + persona takes, and report_to_dict carries the four
panel keys — which auto-propagates them into the dashboard's
snapshot.json via report_to_webapp_dict. dashboard_publisher attaches
in scan_tf gated by agent_panel.dashboard (default off); dashboard.html
renders an "AI panel (advisory)" section in the TF detail dialog.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:34:30 -04:00
Harwinder Sekhon a7c2bfc7da telegram: attach AI panel to alert-quality sends (advisory block)
Auto-scan and level-breach alert paths attach the panel AFTER the
quality gate and dedup decision (no LLM spend on suppressed alerts),
via run_in_executor so the event loop stays free. Alert message gains a
two-line 🤖 AI Panel block (verdict + confidence + rationale, persona
votes); /status shows the daily panel budget when the feature is on.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:32:57 -04:00
Harwinder Sekhon 7be2231eb3 agent_panel: persona/synthesizer advisory consensus module
ai-hedge-fund-style panel: 4 DeepSeek personas (macro, technical, risk,
contrarian) review a finished SignalReport concurrently on a
panel-private thread pool, then a synthesizer chair issues
AGREE/NEUTRAL/OPPOSE + confidence + rationale. Advisory/display-only by
construction: consumers attach post-hoc via attach_panel; the fields
live on SignalReport with empty defaults and analyze_from_dataframes
never touches them (backtest parity, locked by test).

Spend control on top of the shared 500/day client cap: max_panels_per_day
UTC budget (default 20), exact per-bar result cache, and a reuse TTL
keyed (symbol, tf, direction) so a persisting signal re-serves its
verdict across bars instead of burning 5 requests per bar. <2 valid
takes skips the synthesizer; any failure degrades to no panel.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:31:50 -04:00
Harwinder Sekhon 0b604f1ad0 config: agent_panel block (enabled/surfaces/min_quality/daily budget)
New optional config.json block for the upcoming LLM agent-panel feature:
enabled, telegram, dashboard (per-surface toggles), min_quality,
max_panels_per_day, reuse_ttl_seconds. Absent block → {} → feature dark.
Validation mirrors the deepseek block. Module 23 opened with parse +
reject tests.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:26:43 -04:00
Harwinder Sekhon 250078c987 docs: architecture diagram (PNG + matplotlib generator)
End-to-end map: external services → providers/news layer → signal core
(engine + factory + aggregator) → SignalReport → four delivery surfaces,
plus the backtester replaying the same core. Regenerate with
venv/bin/python docs/architecture_diagram.py after architecture changes.

Claude-Session: https://claude.ai/code/session_01Aq1jZ6jBsBdEC2xT3u46SA
2026-08-07 09:21:52 -04:00
Harwinder Sekhon d1d002a4f7 dashboard: clickable news — detail dialogs + on-demand AI summary endpoint
- news_engine: RSS parser captures link/description/publisher (three new
  optional NewsSentiment fields); _clean_html_text/_safe_link helpers
- news.json: headlines carry id = sha1(headline) + enrichment fields
- static/news_summary.php (NEW): on-demand DeepSeek gold-impact summary,
  whitelist-gated against news.json ids, cache-forever in ai/, fail-closed
  file-based daily cap (150/day), key at /etc/gold-dashboard/deepseek.key;
  installed to the docroot by install_static
- dashboard.html: newswire + calendar detail dialogs (score tiles, source
  description, article link, AI summary with loading/error states);
  delegated on stable containers, dialog keyed by id across polls
- tests: +6 (357 total); docs: README-dashboard §6 one-time setup
  (key file, apache-owned ai/, SELinux httpd_can_network_connect), CLAUDE.md

Claude-Session: https://claude.ai/code/session_01Aq1jZ6jBsBdEC2xT3u46SA
2026-08-07 09:08:49 -04:00
root edefd90aaa oanda: retry once on transient 401/5xx/network faults in candle fetch
The practice API rejects ~15-20 requests/day with a spurious 401
"Insufficient authorization" while the same token succeeds on adjacent
requests; each one silently cost a full scan cycle on that timeframe.
One retry after RETRY_DELAY_S (2 s) absorbs these plus transient
5xx/network faults. Real credential failures fail both attempts and
raise as before; other 4xx (bad request, 429) are never retried.

Tests: transient-401 recovery, persistent-401 raise, 400 not retried.

Claude-Session: https://claude.ai/code/session_01Tujtmmhxj9mxKjPFSoVjmP
2026-08-06 23:06:29 -04:00
Harwinder Sekhon b21c75a1f8 dashboard: Gold Desk v2 UI — light/dark theme, TF detail dialog, wire score
Owner-supplied page design replacing the dark-only v1: Bootstrap 5
colour tokens with a luma-inverted dark mode (toggle + system-pref +
localStorage), compact clickable TF cards opening a full-detail modal
(all-nine indicator table with value/conf/weight, price ladder of key
levels vs spot, per-confirmation-TF drilldowns, provenance), newswire
aggregate sentiment box, Inter/JetBrains Mono webfonts.

Re-applied the c70f58a sync fixes the source copy predated: 5 s
conditional-GET polling and the +2 s boundary refetch sniper. Data
logic otherwise unchanged; all fields verified against
report_to_webapp_dict output.

Claude-Session: https://claude.ai/code/session_017mbXs9HRawZhD1uZ3ze7zQ
2026-08-06 22:44:48 -04:00
Harwinder Sekhon a2f951f5c9 dashboard: cut page sync latency to ~scan-time + ≤5 s
Three display-side fixes, no signal-logic changes:
- run_cycle publishes snapshot.json BEFORE the news refresh — news I/O
  (FF + 4 RSS + possible DeepSeek calls) took 20-30 s on TTL-expired
  cycles and kept fresh signals off the page (publish_all split into
  publish_snapshot/publish_news, kept for boot)
- page polls every 5 s (was 20 s) using conditional GETs: dropped the
  ?t= cache-buster + no-store so Apache answers unchanged files with
  bodyless 304s
- boundary refetch sniper re-armed: fires 2 s after next_scan_epoch
  regardless of distance (the old delay<POLL_MS guard would have been
  dead code at 5 s polling)

New Module 20 test locks the snapshot-before-news ordering. 352 tests.

Claude-Session: https://claude.ai/code/session_017mbXs9HRawZhD1uZ3ze7zQ
2026-08-06 22:35:13 -04:00
Harwinder Sekhon 858c12280a news: DeepSeek validation, vs_forecast fix, /status source health
Validation (llm_news_eval.py harness + 78-headline golden CSV + verdict
doc): headline scoring 78/78 vs keyword baseline 24/78 — validated;
event gold-direction calls misread close K/M numeric comparisons ~11%
of the time (both claims cases in sample, JOLTS 8.79M vs 8.84M).

Fix: event payload now carries code-computed "vs_forecast" (from
EconomicEvent.surprise) and the prompt gains an explicit inverted-
metrics rule + worked claims example — iteration 1 (field alone) fixed
the arithmetic but exposed a masked inversion error. 18/18 on live
re-verification (4 misreads fixed, 11 regression cases clean, 3 fresh
both-side probes).

/status: new "News & AI" section — per-source health for the calendar
chain (which of FF/MyFXBook served), each RSS feed, and DeepSeek
(daily count, last error kind), written passively by the existing
fetch paths under _cache_lock. Failures WARN once on transition
(repeats DEBUG, recovery INFO) so the rotating log carries outage
stories without 10-min spam.

test_all.py: +5 Module 19 tests (351 total); also carries the Module 22
fixed-RR-TP tests from the prior research session (shared file).
Replay artifacts dir gitignored like commit_ladder.

Claude-Session: https://claude.ai/code/session_017mbXs9HRawZhD1uZ3ze7zQ
2026-08-06 22:21:02 -04:00
Harwinder Sekhon 525a1cd1c1 research: --fixed-rr-tp backtest flag + 2026-08-05 verdict (NOT shipped)
Hard TP at N×risk beats the trail on all 2024+ windows but the grid is
monotone to 3R (trend artifact) and M15 rr20 goes negative in 2021+2023
with doubled DD — trailing exits stay. Flag is research-only; its
Module 22 tests land with the next commit (shared test_all.py).

Claude-Session: https://claude.ai/code/session_017mbXs9HRawZhD1uZ3ze7zQ
2026-08-06 22:20:43 -04:00
Harwinder Sekhon b693f757f8 Remove OANDA_MIGRATION_ROADMAP.md (reference-only doc, superseded)
Deletion was already staged in the working tree by the owner; the
roadmap was approved 2026-07-24 as reference only and its plan file
lives outside the repo. Content remains in git history.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 20:26:54 -04:00
Harwinder Sekhon 7dbcbc6c82 audit: NY-anchored weekend calendar (all 3 copies) + FB2_EMA_LO crash-proofing
Owner decisions from the 2026-08-04 audit report:
- spread-in-exit-triggers: DECLINED (books stay comparable; ledger in
  CLAUDE.md records the measured ~4-6% optimism).
- weekend clock: FIXED. The market week is Fri 17:00 -> Sun 18:00 New
  York time, not Fri/Sun 22:00 UTC — during US DST the Oanda feed stops
  at 21:00 UTC (measured: 159/162 summer weekends). All three weekend
  implementations (providers/base, signal_aggregator, gpu_backtest
  calendar filter) now share the NY-anchored rule; a new test sweeps
  them hourly across both DST transitions and asserts agreement. No NTP
  needed — the host clock was fine, the constant was wrong.
  Ends the weekly false-"STALE DATA / feed lagging" hours (summer Fri
  21-22 UTC, winter Sun 22-23 UTC).
- FB2_EMA_LO (delegated): kept as a research knob but crash-proofed —
  an unparseable value degrades to the tuned default 0.5 with a warning
  instead of killing every scan. Documented in CLAUDE.md.

Book gates: M15 12mo pinned book byte-identical (+78.7111R, 397
trades). M5 12mo drops exactly ONE trade (556 -> 555, +97.56 ->
+98.64R): a phantom whose signal fired on the final summer-Friday bar
(decision 21:00 UTC = NY 17:00 market close) and was "filled" at
Sunday's reopen 49 hours later, -1.08R — an entry live trading can
never take. The backtest no longer contains it; live == backtest
parity improved. M15 unaffected because hour 21 is in its blocked set.

Suite: 341/341 green.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 20:21:34 -04:00
Harwinder Sekhon 5b3f47f3e1 audit: scheduler cleanup, WS-log secret hygiene, simulator test coverage, CLAUDE.md sync (Batch G)
2026-08-04 Fable 5 QA audit, final code batch.

- Boundary math: removed the provably-dead `remaining <= buffer` guard
  from all three scheduler copies (518,400-instant frozen-clock sweep:
  never fired) and documented the exact-boundary-instant semantics.
- websockets/websockets.client loggers pinned to INFO so a DEBUG run
  can never write the TD key (connect URI) or Tiingo auth frame into
  the 30-day rotating log. (.env also chmod 600, untracked.)
- Cooldown pass factored into _apply_cooldown (pure, testable) — the
  only sequential replay rule finally has direct tests.
- New simulator behavior tests (previously ZERO coverage of the trade
  resolver): same-bar SL+TP tie → SL-first, gap-through-SL fills at
  the open, TIMEOUT pricing + round-trip spread charging, trailing
  ratchet arms at bar close and fills next-bar-only, cooldown ordering.
- CLAUDE.md synced to verified reality: 338 tests/21 modules; three
  weekend-implementation copies caveat; throttle claim corrected
  (single-flight, TD-only throttle); daemon trail/cooldown parity
  caveats (once-per-OBSERVED-bar, wall-clock cooldown) + OPEN_ONLY and
  reconcile notes; CP-path gate semantics documented as validated
  behavior; new "known open items" section for the owner-decision
  findings (weekend DST constant, spread-in-trigger rule-6 item, H4
  seeding divergence, FB2_EMA_LO).

Gates: suite 338/338 green; pinned M15 book and M5 HEAD baseline both
reproduced identically after the cooldown refactor.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 20:08:37 -04:00
Harwinder Sekhon 99eb268cf3 audit: alert delivery, provider robustness, backtester NaN/cache guards (Batches C+E+F)
2026-08-04 Fable 5 QA audit. Three batches sharing one test file, all
verified by runnable repros before fixing, all gated below.

Batch C — telegram alert delivery (verified by h4 harness):
- Failed breach rescan shrinks the cooldown to 30 s instead of
  silencing the symbol for 5 min; breach during cooldown is PARKED and
  fired on the first tick after expiry (was dropped forever).
- Level cache keyed symbol→TF (M5 scans no longer wipe H1/H4 swing
  levels within minutes); breach labels TF-qualified.
- Markdown alert sends get a plain-text fallback (mirroring _safe_edit);
  fingerprint commits only when a message actually landed.
- Per-key asyncio.Lock closes the dedup TOCTOU between auto-scan and
  breach sends (forced-interleave repro produced duplicate alerts).
- _start_scheduler respawns dead per-TF loops (was all-or-nothing).
- Clean WS close now backs off (was a zero-delay reconnect hot loop);
  backoff resets only after a connection that received ticks; TD
  iter_ticks fires on_connect on the first RECEIVED frame (invalid key
  can no longer pin the backoff at minimum).
- Scan-failure logs carry the real traceback (was "NoneType: None").

Batch E — providers (verified by h5 harness):
- fetch_range: 3 consecutive chunk ERRORS raise instead of returning
  silently truncated history (verified 731-day silent gap repro; also
  covers 429 storms); pagination no-progress guard; head-truncation
  ERROR log.
- Single-flight per (symbol,TF): 4 concurrent scans = 1 API call (was
  4); cache records requested depth so a longer-lookback call refetches.
- Multi-bar forming/future tail fully dropped (was iloc[:-1] once).
- Oanda: malformed candles skipped, never 0.0-defaulted; cleaning drops
  non-positive-price rows BEFORE OHLC repair; inf returns neutralized so
  one corrupt bar can no longer disarm the whole 10-sigma filter.

Batch F — backtester internals (verified by h6 harness):
- NaN bar in a trailing window exits at the last valid bar (was NaN R
  poisoning every metric); static-path TIMEOUT NaN guard;
  compute_metrics excludes non-finite R loudly. Zero real-data exposure
  (0 NaN rows in all 10 Oanda parquets) — defensive.
- fetch_history ERROR-logs pre-2026-07-08 cache files (cleaned-bars era)
  when loaded for raw-path resolution.

Gates: suite 333/333 green; pinned M15 12mo book reproduced identically
(397 trades, +78.7111R) after E and after F; M5 HEAD-config baseline
reproduced identically (556 trades, +97.5604R) after F.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 20:02:57 -04:00
Harwinder Sekhon 18e4fd0403 audit: gate-chain verdicts — lock shipped M5 entry-path state, document hour-delta isolation
2026-08-04 audit, Batch D. Verification outcomes (no behavior changes —
comment + tests only):

- CP gate-bypass (quality forced to B after the gate chain, exhaustion/
  M5-score gates not re-checked) is VALIDATED behavior, not a bug: the
  pinned shipped-config M5 12mo book carries 226/586 XAU_CP trades
  (+38.9R), 104 with EXHAUST warnings (+15.3R), 175 below the M5
  min-score gate (+28.0R). Removing the "holes" removes validated
  profit. New test pins the shipped state (PB off, CP BUY-only on M5,
  ema_partial_neutral M5-only) so any flip is a conscious revalidated
  decision.
- E18 (xau_strategy vs backtest blocked-hours "drift") REFUTED as a
  defect: the divergence is required by pair isolation — the backtest
  copy is the generic non-XAU fallback and must not inherit
  gold-validated deltas. Documented in the factory + a test asserting
  the delta stays XAU-side.
- 6c-upgrade re-arming an exhaustion-D refuted (exhaustion zeroes
  confidence below 6c's floor; regimes mutually exclusive). "WEEKEND
  FILTER" CP-suppression branch is defense-in-depth for direct
  analyze_from_dataframes callers, kept.

Suite: 316 green.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 19:52:07 -04:00
Harwinder Sekhon ebb83749ef audit: GPU-engine cache-miss observability + dead-guard alignment + full parity test
2026-08-04 audit, Batch B. Verified first (h1 harness): the two engines
are byte-identical on every reachable input — 370-prefix flat-bar deck,
warmup band, 322 real-data M15 positions incl. all 7 zero-range bars,
zero diffs. The claimed ADX NaN desync / warmup-guard divergences are
unreachable below the shared 30-bar analyze() floor. Changes here are
therefore observability + hygiene, proven behavior-neutral by exact
reproduction of the pinned M15 12mo book (397 trades, +78.7111R,
byte-identical trade rows).

- _resolve_pos: cache-miss fallback to live-engine recompute now warns
  on first miss and counts every miss (was fully silent — a run could
  switch math with no trace).
- ROC/Keltner warmup guards textually aligned with the live engine so a
  future period/floor change can't open a silent divergence window.
- New regression: full-sweep parity test comparing ALL indicator
  signals/confidences, composite, and regimes on every prefix incl. a
  flat zero-DM/zero-TR run (the old parity test skipped
  ADX/MACD/ROC/Keltner/RSI entirely); plus a loud-fallback test.

Suite: 314 green. Book gate: oanda_m15_12mo_030 reproduced identically.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 19:49:08 -04:00
Harwinder Sekhon 8acc1e18f8 audit: harden Oanda execution lifecycle (close-fill, netting, races, fail-closed config)
2026-08-04 Fable 5 QA audit, Batch A. All fixes are execution-robustness
only — no signal code touched (git diff scope: daemon/executor/tests).
Findings verified by runnable repros before fixing (h3 harness):

- close_trade: a 200 whose close order was created but NOT filled (e.g.
  MARKET_HALTED during the settlement pause) now raises instead of
  returning; the daemon's timeout path keeps tracking and retries. The
  old path popped state and orphaned a live position forever
  (reconciliation was boot-only).
- place_market_order: positionFill=OPEN_ONLY — on a netting account an
  opposite-side order can no longer silently close an untracked
  position; the divergence surfaces as an order cancel.
- _execute_trade: a fill without tradeOpened (netting close/reduce) and
  any placement OandaAPIError (server may have filled before a network
  timeout) now trigger _reconcile_positions instead of silent return.
- _discover_existing_positions → lock-guarded _reconcile_positions,
  callable mid-run; adoption of a position with NO broker stop now
  screams (CRITICAL + per-bar warning) instead of computing sl=entry,
  risk=0 and silently never trailing.
- _manage_trailing: snapshot deep-copies per-trade dicts (no mutation of
  live state outside the lock); write-back skips trades popped by
  _check_closed_trades mid-iteration (no ghost resurrection).
- Fail-closed config: unknown OANDA_QUALITY_GATE refuses to start
  (previously a typo silently became the most permissive gate "C");
  non-integer OANDA_LOT_SIZE/MAX_HOLD_BARS exit with a clear message.

Suite: 302 → 312 tests (Module 21), all green.

Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
2026-08-04 19:46:37 -04:00
Harwinder Sekhon 28a1c375bd M5 at real cost: disable XAU_PB, squash partial EMA alignment
Both validated at the measured $0.65 spread on Oanda bars (12mo halves +
PF, OOS 2024-06->2025-03, 5yr; full evidence in pair_signals/xauusd.py
M5 overrides 4-5):

- m5_pullback_enabled False on M5: loss forensics identified XAU_PB as
  the largest wrong-direction loss source; 12mo PF 1.26 (failing the
  acceptance bar) -> 1.44, max DD 23.9R -> 13.9R. PB code stays for
  future regime-gated experiments.
- New TechnicalConfig.ema_partial_neutral (default False), True on M5
  only: partial 4-EMA ribbon alignment (2-3 of 4 checks) scores NEUTRAL
  instead of a weak BUY/SELL (-0.008R/84 vs +0.230R/341 fully aligned).
  Squash mirrored in the backtester's precomputed EMA scorer in this
  same commit (live==backtest parity). 12mo +97.6R/556 PF 1.48 DD
  10.7R halves +47.8/+49.8; 5yr +127.4R PF 1.38; OOS one-trade tie.
- Rejected and logged (memory + docstring): hour-19 block (OOS fail),
  hours-14/15, trail dist 0.75, confirm-TF swaps incl M15+H4 (OOS
  ordering), divergence/candlestick weight changes, bollinger removal.
- Tests 296 -> 302: partial-alignment fixture test, factory-equivalence
  mirrors, per-TF isolation test, BC-gate fixture force-enables PB.

Claude-Session: https://claude.ai/code/session_01H1eKWZsAVapJ1C4winYLqX
2026-08-04 16:54:09 -04:00
Harwinder Sekhon 775330ff16 /news: normalize rounded-zero score so "-0.00" renders as "+0.00"
A balanced feed (XAU and USD means nearly cancelling) yields a tiny
negative net that "+.2f" rendered as "-0.00" — seen live 2026-08-03.
Regression test uses that day's actual per-currency means (300 tests).

Claude-Session: https://claude.ai/code/session_017975c1TufZ3xRYAq8iWTCS
2026-08-03 18:06:44 -04:00
Harwinder Sekhon 1f54452007 Telegram /news command: net XAU/USD news sentiment score on demand
Owner-locked /news replies with the existing get_pair_sentiment() net
score (XAU mean - USD mean, -1..+1) and its label; a caveat line marks
the no-headlines case so a feed outage can't read as neutral. Fetch runs
in the shared executor with a placeholder-then-edit reply. Display-only:
news still never gates signals. Registered in handler block, command
menu, help text, docstring, and user guide; 3 new builder tests (299).

Claude-Session: https://claude.ai/code/session_017975c1TufZ3xRYAq8iWTCS
2026-08-03 18:01:42 -04:00
Harwinder Sekhon be5dd4a25b Dashboard publishes directly into /var/www/trade (no subfolder)
Adds exactly three files next to the portal: dashboard.html,
snapshot.json, news.json. README covers chown vs setfacl for write access.

Claude-Session: https://claude.ai/code/session_01Wc9ZoqKLYBT2QJcpRWKoGm
2026-08-03 17:17:37 -04:00
Harwinder Sekhon 0445a20a76 Dashboard out-dir default → /var/www/trade/gold-dashboard (live portal docroot)
Claude-Session: https://claude.ai/code/session_01Wc9ZoqKLYBT2QJcpRWKoGm
2026-08-03 17:14:08 -04:00
Harwinder Sekhon 4e654b5edb Web dashboard v2: static publisher + Apache, with calendar & news panels
- dashboard_publisher.py: standalone candle-close daemon that atomically
  writes snapshot.json/news.json/dashboard.html to an Apache-served dir
  (DASHBOARD_OUT_DIR). Fully independent of telegram_bot/oanda_daemon —
  local confirmation map + boundary math, no imports either way, no new
  deps, no open port. Due-TF classification from the RAW wake time.
- static/dashboard.html: self-contained dark page — hero rail, four TF
  cards (score bar, 3x3 indicator grid, MTF, setup, reasons), DeepSeek
  market context, week economic calendar, scored newswire. Clock offset
  from the HTTP Date header (publish-time epoch is stale between closes).
- news_engine.get_week_events(): additive read-only accessor over the
  cached FF week; no signal path calls it.
- deploy/: systemd unit + live-server (LAMP/SELinux) checklist.
- test_all.py: Module 20 (9 tests) + get_week_events test; 296 green.

Claude-Session: https://claude.ai/code/session_01Wc9ZoqKLYBT2QJcpRWKoGm
2026-08-03 17:10:29 -04:00
Harwinder Sekhon 9e1c321f05 Add commit-ladder campaign driver; ignore its 26MB results folder
Added: backtest/research/commit_ladder.py (recreates historical configs
via --set on the fixed harness; 2026-08-03 campaign confirmed HEAD is
the best config on all timeframes). Results stay local like all pinned
trade books.

Claude-Session: https://claude.ai/code/session_01Nsd8HcrV67GbCZXxZEn1Ma
2026-08-03 13:29:18 -04:00
Harwinder Sekhon 6fff06ff90 DeepSeek V4-Flash news enrichment: LLM sentiment, surprise direction, summary
Three display/warning-only upgrades to the news engine, all behind the
existing keyword path as fallback — signals are never gated on LLM
output (event_risk_caps_signals / sentiment_affects_signals stay False
per the 2026-07 audit; backtester use_news=False untouched):

- Headline sentiment: one batched call per 600s cache window (capped at
  25 new headlines, sha1-cached so repeats are never re-sent) overrides
  the keyword scores per currency with the gold-inverse doctrine encoded
  in the prompt.
- Release surprises: get_event_risk_level's description now says which
  way the print pulls GOLD ("— gold bearish: strong jobs lift yields"),
  fixing the documented numeric-only gap; risk level and windows
  unchanged.
- Market summary: 1-2 sentence macro blurb on SignalReport.news_summary,
  rendered in Telegram alerts (📰) and the CLI (🧠).

New llm_client.py: requests-only OpenAI-compatible client (no SDK), one
attempt per call, UTC-day request cap, singleton so the cap survives
per-scan NewsEngine reconstruction. Config: deepseek block in
config.json + DEEPSEEK_API_KEY env; absent key = fully inert.

Live-measured API behavior baked in: V4-Flash is a hybrid thinking model
— requests send thinking:disabled (reasoning otherwise blew the timeout
and could exhaust max_tokens before the answer); prompts are static
module constants so DeepSeek's prefix cache prices repeat calls at the
cache-hit rate; scores use compact [s, xau] pairs; timeout 12s.

Tests 273 → 286 (new Module 19 + deepseek config tests).

Claude-Session: https://claude.ai/code/session_01Nsd8HcrV67GbCZXxZEn1Ma
2026-08-03 13:25:02 -04:00
Harwinder Sekhon f7d8f1113c Tiingo WebSocket streaming: re-enable the level-breach monitor
The monitor has been dead since the Oanda feed switch (Oanda's provider
does not stream). New config.json scalar data_providers.streaming selects
a tick-stream provider independent of the candle feed; telegram_bot
resolves it via _get_streaming_dp() (separate instance, candle cache
stays on the active provider) and falls back to the active provider when
unset — behavior is byte-identical to before when the key is absent.

TiingoDataProvider grows the two sanctioned streaming overrides:
supports_streaming=True and iter_ticks() on wss://api.tiingo.com/fx
(mid price only — the no-spread rule extends to the stream). Two
deliberate divergences from the TD reference implementation, both
because Tiingo authenticates inside the socket: server "E" messages
raise TiingoDataError (a swallowed auth error would end the generator
cleanly and reconnect-hammer with zero backoff), and on_connect fires on
the first received message rather than after the subscribe send. Field
order of quote frames was live-verified: ask is data[7], not data[6] as
the docs table suggests (data[6] is askSize).

make_data_provider(config, name=None) now takes an explicit provider
name; keyless streaming providers disable the monitor with one warning
instead of a traceback-per-minute loop.

Tests 266 → 273 (WS parser, ticker round-trip, streaming config scalar,
explicit-name dispatch, streaming-dp resolution, monitor auto-disable).

Claude-Session: https://claude.ai/code/session_01Nsd8HcrV67GbCZXxZEn1Ma
2026-08-03 13:11:36 -04:00
Harwinder Sekhon bcda6e0ebd Cleanup: remove dead code, stale docs, and all non-main branches
- Delete back-compat shims data_provider.py / oanda_provider.py;
  test_all.py now imports providers.* directly (ForexDataProvider name
  retired; the two shim-only tests removed, 268 -> 266 tests)
- Delete backtest/research/calendar_store.py (no importer, no entry
  point); config.py audit comment reworded
- Delete executed plan doc (2026-07-21 centralized-config) and the two
  unbuildable guide PDFs; HTML guides remain the source of truth
- OANDA_MIGRATION_ROADMAP.md status header corrected: phases 1-2
  shipped 2026-07-28, feed is live on Oanda; phases 3-6 still unbuilt
- Branches deleted locally + on both remotes: oanda_demo (merged),
  web-dashboard, eurusd-modular, reversal-exit-alert (owner-approved
  permanent deletion); July stash dropped

Claude-Session: https://claude.ai/code/session_019zR58p8ACW1oJ5QXVUZ5es
2026-08-03 11:08:48 -04:00
Harwinder Sekhon 042b079861 Big-candle gate (default-OFF) + M5 relaxation campaign + --set tooling
Gate 6h (big-candle entry gate): blocks all XAU entries (core, XAU_PB,
XAU_CP) when the primary TF's last 24h contains a bar with range >= K x
prior-bar ATR. Ships default-OFF: validation rejected every threshold
(K 3-6 on M5/M15 12mo) — removed trades were net-POSITIVE at every cell;
M5 quiet-window trades are the only losing bucket. Config knobs
big_candle_atr_multiple/lookback_hours, --big-candle backtest flag,
Module 17 tests.

M5 relaxation SHIPPED (validated 2026-07-30, Oanda bars, $0.30 spread):
m5_min_abs_score 30->25, m5_min_confirmation_score 10->5, hour-21 UTC
unblocked M5-only. Beats control in ALL 6 calendar years of the 5yr
replay (+230.9R/2319 PF 1.26 DD 21.3 vs +168.0R/2048 PF 1.21 DD 27.0),
12mo +107.4R PF 1.36 vs +84.6R PF 1.33, OOS +26.1R vs +17.2R, $0.70
spread +86.9R vs +66.4R. M15: every relaxation retested negative —
config unchanged. Declined-but-open: M5 hour-22 block, M15 hour-16
unblock (both fail 5yr/OOS). Evidence in the factory docstring.

Tooling: generic --set PATH=JSON research override in gpu_backtest
(typo-proof, logged); diagnose_filters filter_tags refreshed for gates
6d-6i with a staleness-guard test (Module 18); research scripts
hour_analysis/burst_profile/book_diff. Suite: 268 tests.

Claude-Session: https://claude.ai/code/session_014jqHpmNYmwYMowF6bPZyeD
2026-07-30 11:30:20 -04:00
Harwinder Sekhon dc7beb2877 Rebuild CLAUDE.md: lean architecture reference
Replaces the deleted exhaustive knob-by-knob reference with a compact
guide: commands, config precedence, pipeline architecture, rules for
changes, and current-state notes (Oanda feed, TD-pinned books).

Claude-Session: https://claude.ai/code/session_014jqHpmNYmwYMowF6bPZyeD
2026-07-30 11:30:07 -04:00
Harwinder Sekhon 637c39e201 Switch active data source to Oanda v20 (practice feed)
config.json.example: data_providers.active = "oanda" (credentials stay
in env/.env or the top-level oanda block; TD and Tiingo remain
registered fallbacks). Backtest parquet cache is now keyed per provider
— Oanda and TD bars diverge (2026-07-24 study), so a feed switch must
never serve another feed's cached bars; twelve_data keeps the legacy
un-prefixed filename so pinned-book caches stay valid. 254 tests green;
live smoke-tested against the practice API (66 contract checks: all
scanned TFs, boundary/freshness guarantees, real spread $0.71,
fetch_range pagination, typed error paths, CLI + backtest end-to-end).

Claude-Session: https://claude.ai/code/session_01Mc7XKYyPySVXiUpcN82QEK
2026-07-28 20:12:59 -04:00
Harwinder Sekhon 083a4e8d63 Provider abstraction: providers/ package with BaseDataProvider template
One class per provider (TwelveData, Oanda, Tiingo), all subclassing
BaseDataProvider which owns cache, validation/cleaning, freshness +
boundary guarantees, fetch_range pagination, and streaming stubs —
subclasses implement only _fetch_raw. config.json data_providers.active
selects the live feed; data_provider.py / oanda_provider.py stay as
back-compat shims. Module 16 provider-abstraction tests (253 total green).

Claude-Session: https://claude.ai/code/session_01Mc7XKYyPySVXiUpcN82QEK
2026-07-28 20:09:00 -04:00
Harwinder Sekhon f760e9070a Research: Dukascopy real-spread study + Oanda/TD bar-divergence study
Scripts + summary CSVs for the 2026-07-24 studies (real XAU spread ~$0.65
tradable hours; Oanda vs TD bars diverge ~11% of ATR with 10% direction
flips). Large raw data files (parquet, per-boundary samples) gitignored.

Claude-Session: https://claude.ai/code/session_01Mc7XKYyPySVXiUpcN82QEK
2026-07-28 20:08:53 -04:00
Harwinder Sekhon 9a384935bf Add Oanda migration roadmap (reference, not scheduled)
Approved 2026-07-24: 6-phase plan for fully switching the signal feed
from Twelve Data to Oanda v20 (inert plumbing -> backtester agnosticism
-> pricing-stream monitor -> shadow parity -> book re-validation with
owner cost gates -> reversible cutover). TD remains a permanent
fallback. Grounded in the 2026-07-24 divergence study (10% bar-direction
flips, 11%-of-ATR noise, real Oanda spread ~$0.72 vs assumed $0.30).

Claude-Session: https://claude.ai/code/session_01YNvxXtgz62F7kTKZZsAAAE
2026-07-24 15:12:37 -04:00
Harwinder Sekhon d669227ddc Oanda demo daemon: auto-execution on practice account (XAU/USD)
New independent trading daemon (oanda_daemon.py) that places orders on
an Oanda v20 practice account instead of sending Telegram alerts.
Oanda-only market data (oanda_provider.py, mid candles + real bid/ask
spread) through the unchanged signal pipeline; order execution and
trade management via oanda_executor.py (raw requests, no new deps).

Book parity with the backtester: no broker-side TP (exits = SL/TRAIL/
TIMEOUT only), bar-close-only trail ratchet with each closed bar
processed exactly once, 5-bar per-TF entry cooldown, structural SL
from the setup. Accepted divergence: one position total (skipped
signals logged for book reconciliation).

Safety/ops: fail-fast boot on rejected credentials (transient outages
continue), market orders never retried (double-fill risk), entry lock
against aligned-boundary races, restart recovery + adoption via
oanda_state.json, hourly heartbeat, asyncio exception handler, every
failure path logged. Prices formatted at instrument precision.

Config: env OANDA_* -> config.json "oanda" section (validated in
app_config). Tests: Module 15 (30 tests), suite 240 green.

Claude-Session: https://claude.ai/code/session_01VUnCqjWBuvKRSjLwHeB9BJ
2026-07-24 14:10:45 -04:00
Harwinder Sekhon c3329d6cd5 Telegram alert redesign: verdict-first layout + informational 100-pip TP
format_signal_message restyled (display-only, zero signal-logic impact):
verdict header first (signal - pair - TF - quality), trade setup moved
above the detail, per-indicator breakdown condensed to one arrow line
(_indicator_summary), heavy dividers dropped. New TP 100p line in every
setup block: entry +/- 100 pips ($10.00 on XAU) with its R-multiple vs
the stop (_INFO_TP_PIPS, formatter-only - not stamped on TradeSetup,
not in CLI/JSON/backtester). Trailing-plan logic unchanged (stamped
per-TF params, pullback fallback). Removed dead _indicator_line and
_SIGNAL_TAG. Tests 210 -> 212 (100p TP math BUY/SELL/no-setup, one-line
indicator summary).

Claude-Session: https://claude.ai/code/session_01BoZKi67Ct6QTeiFJmybXSw
2026-07-24 10:58:22 -04:00
Harwinder Sekhon 7a810c4a4c Remove Tiingo spread monitor (indicative two-tier feed, unvalidatable)
Tiingo's free /tiingo/fx/top publishes only a two-tier indicative
spread for gold ($0.10 liquid hours / $0.70 rollover+news; verified on
273 production samples + live probes) — mid ± a posted increment, not
executable depth, so it can never validate the $0.30 backtest cost
assumption. Removed one day after shipping: tiingo_provider.py, the
sampler/quote plumbing in telegram_bot.py, the channel_spread_detail
config knob and tiingo_api_key resolver, and Module 14's Tiingo tests
(221 -> 210). CLAUDE.md documents the verdict: do not re-add a spread
display from Tiingo's free FX feed.

Claude-Session: https://claude.ai/code/session_01BoZKi67Ct6QTeiFJmybXSw
2026-07-24 10:56:55 -04:00
Harwinder Sekhon 3a95a37a0c Accuracy audit fixes: alerts carry validated per-TF trail plans + 13 more
Five-agent audit of the signal path; all fixes are delivery/measurement
side. Regression-proven behavior-neutral on validated signals: the 5yr
M15 replay reproduces the pinned book exactly (+210.65R PF 1.35
WR 57.8% MaxDD 33.90R). Tests 215 -> 221.

- TradeSetup carries trail_arm_r/trail_dist_r stamped from the producing
  config; Telegram trail plan reads them (was showing default 0.6/0.75
  instead of the validated M15 arm 0.70 / M5 dist 0.85)
- Alert dedup: fingerprint = (signal, quality, direction); entry movement
  compared against last-ALERTED entry/risk (old SL-dist bucket churned on
  ATR drift -> near-duplicate alerts every candle)
- Fresh fetches enforce the closed-bar boundary check with one 3s grace
  retry, then warn (feed lag no longer silently analyzes the prior bar)
- Backtest max DD includes the zero baseline (opening losing streaks
  were undercounted; pinned M15 DD unchanged)
- Replay loads the regime-filter TF when the H4-ADX gate is on (was a
  silent no-op for M5/M15 primaries)
- clear_cache under _cache_lock; dead ROC zero-cross branches removed
  from both engines (provably unreachable); RSS dates via
  parsedate_to_datetime (named zones no longer stamped UTC); event risk
  grades from the soonest event; .env inline comments stripped;
  auto-scan skips NEUTRAL like the breach path; report_to_dict honors
  the weight-0 skip rule and emits strategy/trail params; PB funnel
  counts spread_gate/accepted; session-filter fail-open paths log loudly
- Docs corrected to match validated code (H&S neckline = lower peak,
  exhaustion anchors farthest tracked swing) with re-validation warnings

Claude-Session: https://claude.ai/code/session_01R5soRUeRBi8kHMB99csrJy
2026-07-23 10:31:13 -04:00
Harwinder Sekhon f814485615 Tiingo bid/ask spread monitor + logging hardening
- tiingo_provider.py: /tiingo/fx/top quotes (10s TTL cache, graceful
  None on any failure); spread log writer. Signals never consume
  bid/ask; warn-only by design (no historical bid/ask to validate a
  gate against — live == backtest rule).
- app_config: repo-root .env loader (setdefault; shell env still wins),
  tiingo_api_key resolver (TIINGO_API_KEY/TIINGO → config.json),
  telegram.channel_spread_detail flag (default false).
- telegram_bot: 5-min spread sampler → logs/spread_log.csv (weekend-
  aware), alert-time quote refresh, bid/ask + wide-spread warning in
  signal messages, /status spread line. Raw numbers to owner DM always;
  channel only while channel_spread_detail=true (Tiingo free tier is
  internal-use-only; flip before public launch). Free-tier limit math:
  worst case ~41 req/h vs 50/h cap, one pair.
- Logging: 3h rotation x 240 backups (30-day retention), full-date
  stamps, force=True root handlers; candle loops + sampler survive
  failed cycles; breach-task exceptions logged via done-callback;
  PTB error handler; sys/threading excepthooks in main().
- test_all.py: Module 14 (tiingo, dotenv, spread routing, loop
  resilience, rotation) — 215 tests green.

Claude-Session: https://claude.ai/code/session_01HVRmy6KspMs96o5PTPYFAB
2026-07-22 17:18:14 -04:00
Harwinder Sekhon 65a1debc4e gitignore: cover M5 trail-grid research CSVs
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 19:39:08 -04:00
Harwinder Sekhon 1499bb7cf2 M5 trail dist 0.85 (validated grid, arm unchanged)
Same-day sibling of the M15 campaign, opposite finding: on M5 the arm
point barely matters — winners need a looser leash. 5yr grid plateau
arm 0.5-0.7 x dist 0.85-1.0; shipped the OOS-preferred edge 0.60/0.85:
+231.2R PF 1.30 vs +218.0R PF 1.29 baseline, both halves better, DD
flat (19.2 vs 19.1), OOS +14.5R vs +11.1R. dist 1.0 rejected at the
OOS gate despite best in-sample total. XAU_PB pb_trail unaffected.
Factory replay reproduces the grid cell; 201/201 tests.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 19:13:01 -04:00
Harwinder Sekhon 902122dbfe M15 trail arm 0.70 (validated grid) + backtest/live parity hardening
Parity: replay now applies the live _apply_regime_filter_live method
under the live guard (function-identity, default-off); unregistered-pair
fallback applies the config.json overlay; parity sweep gains direct
ATR/vol-regime assertions; regime-filter semantics test added.

Trail campaign: --trail-arm/--trail-dist research flags; 30-cell grid on
5yr M15 refuted the earlier-arm hypothesis and found the 0.70-0.80 arm
plateau. Shipped M15-only trail_arm_r=0.70 (factory override):
+210.7R PF 1.35 WR 57.8% vs +175.7R PF 1.30 WR 53.9% baseline; both
halves and 2024-06->2025-03 OOS better (+57.4 vs +45.4); 5yr max-DD
clause (33.9R vs 29.1R) explicitly waived by owner. M5 unchanged.
Factory replay reproduces the grid cell trade-for-trade; 201/201 tests.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 16:40:33 -04:00
Harwinder Sekhon f0f9dcb169 Remove dead code: legacy config fields, unused helpers/imports, stray files
- Delete test_ws_dual.py (standalone probe, unused) and a stale tracked log
- config.py: drop MarketSession enum, is_jpy_pair, candle_buffer_seconds,
  and the never-read legacy SignalConfig/XauStrategyConfig fields
- data_provider.py: drop uncalled get_latest_price/get_spread_estimate
- Remove unused imports (engine, telegram_bot, main, news_engine)
- test_all.py: drop/trim the corresponding defaults assertions (199 tests)
- CLAUDE.md updated to match

Zero behavior change: no production code read any removed field
(verified reference-by-reference); suite green at 199/199.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 14:06:55 -04:00
Harwinder Sekhon c8a2c83820 Merge branch 'centralized-config': config.json deployment overlay
Centralized deployment settings (Telegram credentials, data-provider
keys/selection, scan pairs/timeframes, per-pair base blocked hours,
alert quality, log level) into a gitignored config.json with env-var
precedence. Hardcoded credentials removed from source. Zero signal-
logic change — parity pinned by tests (200/200).

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:44:34 -04:00
Harwinder Sekhon 2d3e524f1c Final review fixes: scrub tracked credentials, trim scannable TFs, absolute parity pin, scanning type-checks
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:42:00 -04:00
Harwinder Sekhon f017f5aef5 gitignore: cover .env.* swap files and the whole logs/ directory
Rotated logs (telegram.log.YYYY-MM-DD) don't match *.log and a vim
.env.swp was nearly committed by a bulk add.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:30:21 -04:00
Harwinder Sekhon a56a622846 Docs: correct PlatformConfig overlay scope (credentials never pass through the factory)
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:30:15 -04:00
Harwinder Sekhon b9633389fa Docs: config.json deployment overlay (CLAUDE.md repo map, env table, test count)
Documents the app_config.py / config.json overlay shipped in Tasks 1-4:
repo-map rows for app_config.py and config.json.example, env-var table
precedence (env -> config.json -> none, hardcoded fallbacks removed
2026-07-21), PlatformConfig/pair_signals overlay call order, and the
real 199-tests/13-modules suite count. Also fixes a stale SCAN_PAIRS
comment in telegram_bot.py that still named PlatformConfig.scan_pairs
as the deployment switch instead of config.json's scanning.pairs.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:25:21 -04:00
Harwinder Sekhon 6e2fdbd684 telegram_bot: credentials/quality/scan-loops from config.json, startup guard
- Credentials, alert quality, scan pairs/timeframes, log level and
  candle-loop intervals now resolve via app_config (env -> config.json
  -> code default) instead of hardcoded fallbacks.
- _get_shared_dp() uses make_data_provider() for provider-registry
  dispatch.
- main() refuses to start without a token/chat_id, raising SystemExit
  with an actionable message instead of a bare sys.exit string.
- test_all.py Module 13: two new tests covering credential/quality/
  loop wiring via a reload, and the startup guard.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:18:05 -04:00
Harwinder Sekhon 80a2b89769 Data provider: key from config.json/env, provider registry, drop hardcoded key
Resolves the Twelve Data API key through app_config.provider_api_key()
(env → config.json → no default) instead of a hardcoded field on
PlatformConfig. Adds data_provider.PROVIDER_REGISTRY + make_data_provider()
so signal_aggregator and the backtester build the active provider through
one seam; telegram_bot.py gets the same one-line WS-key fix now (per Task 4's
already-planned edit for that line) so the suite doesn't crash on import
between this commit and Task 4's full wiring pass.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:10:58 -04:00
Harwinder Sekhon 1610507d57 Wire config.json overlay into the pair factory (before per-TF deltas)
Implements Task 2: apply_pair_overrides in xauusd.make_config before
the validated per-TF deltas, so e.g. M5 hour-9 appends on top of the
JSON base. Updated Module 9 equivalence test and added three parity pins.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:59:12 -04:00
Harwinder Sekhon 1dab14d298 app_config: reject non-object config.json sections with AppConfigError
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:54:46 -04:00
Harwinder Sekhon cbc01565a2 Add app_config.py: config.json loader, validation, env-precedence resolvers
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:47:48 -04:00
Harwinder Sekhon 53b65e1bbb Plan: centralized config.json implementation (5 TDD tasks)
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:44:24 -04:00
Harwinder Sekhon 1a141abc7c Add config.json.example template; gitignore config.json
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:32:28 -04:00
Harwinder Sekhon 096b69686d Spec: centralized config.json design (deployment settings overlay)
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:25:08 -04:00
Harwinder Sekhon c21c102536 News informational-only: gate sentiment bonus/penalty off by default
Add NewsConfig.sentiment_affects_signals (default False), gating the +5
aligned-sentiment confidence bonus and the 10-pt news_conflict penalty.
With event_risk_caps_signals already False, live decisions are now pure
technical, matching the backtester (use_news=False). Warnings and the
sentiment display are unchanged. Adds a gating regression test.

Claude-Session: https://claude.ai/code/session_01FMaaPdrrrQuh1VvGQH1icN
2026-07-17 16:03:31 -04:00
harry c87bb14830 Merge pull request 'Remove EUR/USD support — XAU/USD-only codebase' (#1) from xau-only-chart-patterns into main
Reviewed-on: http://192.168.55.11:3000/harry/ForexBot2/pulls/1
2026-07-16 20:09:21 +00:00
Harwinder Sekhon a3279e54fa Enable BUY-only chart patterns on M5 — bearish patterns dropped (OOS negative)
M5 factory: cp_enabled=True, patterns limited to inv_hs, double_bottom,
bull_flag. SELL CP lost -3.21R on OOS; BUY CP: +41.8R PF 1.60 on 12mo,
+65.2R PF 1.48 on 5yr. Adds +22R to the M5 book (708 vs 513 trades).
M15/H1 remain disabled — all failed OOS validation.

Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 16:08:13 -04:00
Harwinder Sekhon 1bce289132 Docs: CLAUDE.md — document XAU_CP archetype (pipeline step 11, generator D, cp_* knobs)
Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 14:31:45 -04:00
Harwinder Sekhon f3d3e56aa8 Add chart-pattern entry archetype (XAU_CP) — disabled by default
Third entry archetype: completion of classical chart patterns (double
bottom/top, H&S top / inverse H&S, bull/bear flag) — the tight canon
with the strongest documented stats. New chart_patterns.py detector
module (pure functions, ATR-unit geometry, closed-bar slices only,
first-close-beyond trigger discipline so a breakout fires exactly
once and never repaints).

Pipeline step 10 in analyze_from_dataframes: fills only empty setup
slots, allows weak-composite F grades (normal at reversal completion)
but inherits session/day blocks, mirrors MTF-opposition + SELL
vol/hour + M5 protections. Geometry: SL = pattern anchor -/+ 0.5 ATR
(min $8), TP1 = measured move, $9 spread gate, trailing exit does the
real work. Strategy tag XAU_CP.

cp_enabled defaults to False everywhere — per-TF adoption happens in
pair_signals/xauusd.py only after the standard validation rule passes.
Backtester: --cp / --cp-patterns flags to test the candidate, XAU_CP
tag recognition.

Suite: 183/183 (21 new tests: detectors, trigger discipline,
invalidation, gates, pipeline, default-off safety).

Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 14:23:17 -04:00
Harwinder Sekhon 14448c10e8 Remove EUR/USD support — XAU/USD-only codebase
EUR/USD never had a validated edge (2026-07-11/12 verdict: no config
passed validation on M5/M15/H1 at 1-pip cost; cost-to-volatility root
cause). Removed: EurUsdStrategyConfig, pair_signals/eurusd.py, EUR
gates 6h + weekend/day filters + strategy marker + trade geometry in
signal_aggregator, EUR pair registry/aliases in main.py and
gpu_backtest.py, EUR/USD news pair mappings, test_all.py Module 10.
Generic multi-currency news keyword infra kept (GBP/JPY/EUR keyword
table is not pair code). Full EUR history remains at 7d3fd95.

Suite: 162/162 green (was 187; 25 EUR-specific tests removed).

Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 14:06:27 -04:00
Harwinder Sekhon 665ffdda48 Fix on Sell order lossing by blocking time 2026-07-16 12:17:56 -04:00
Harwinder Sekhon fe7d4c5d4a Audit fixes: cache boundary staleness, 6e session bypass, GPU engine parity
Fixes all high+medium findings from the 2026-07-16 full-codebase audit.

High:
- data_provider: cache now rejects snapshots missing the latest closed
  bar (_has_latest_closed_bar, weekend-exempt). The wall-clock TTL served
  mid-candle snapshots at the next boundary, so live M15/H1 scans ran one
  full bar behind and the bar-close session check evaluated the wrong hour.
- signal_aggregator: gate 6e (SELL quality) only acts on actionable
  grades. It was re-grading session-blocked F reports to D, which
  re-opened the M5 pullback path (entry check: quality != "F") on
  session-blocked bars.
- gpu_backtest: port the live EMA-ribbon compression/expansion logic
  (+10 conf on expansion) into the GPU engine. Its absence diverged
  composite score/conf on 7-9% of bars and flipped signals on ~0.2%.

Medium:
- gpu_backtest: Bollinger precompute uses the exact pandas rolling calls
  (cumsum E[x2]-E[x]2 lost precision at gold price levels; bb_position
  error up to 7.6e-3 crossed signal thresholds, worst during squeezes).
- gpu_backtest: --confirm-tfs defaults to the pair factory's live
  confirmation map instead of a hardcoded 1h,4h.
- gpu_backtest: entries re-anchor to the first RAW bar after the signal
  bar, so >10-sigma spike bars dropped by cleaning are no longer skipped
  at entry (favorable bias).
- gpu_backtest: per-TF warmup pad — D1 confirms get >=210 bars at window
  start (62-day pad gave ~61 bars: no EMA200, unlike live).
- data_provider: thread-safety locks for cache + throttle (shared
  provider across the bot's 4-thread pool); outlier filter keeps the
  snap-back bar after a spike instead of punching a two-bar hole.
- telegram_bot: alert fingerprint gains a risk-relative entry bucket
  (materially-moved setups re-alert) and is committed only after a
  successful send (failed sends no longer consume the alert for 1h).
- engine: disabled_indicators=["macd"] now removes the MACD pillar from
  scoring (histogram still computed for divergence).

Verification: test_all.py 187/187 (new Module 11: 8 audit-fix
regressions incl. a live-vs-GPU parity sweep with ribbon coverage);
real-data parity sweep over 813 slices of 14mo XAU M15: zero divergence
(pre-fix: 7-9% of bars diverged); both original bug repros no longer
reproduce; cached end-to-end backtest run passes with the live confirm
map. Books shift slightly vs pre-fix runs — comparability note added to
CLAUDE.md; pinned tuned numbers need re-validation under the fixed
backtester.
2026-07-16 10:24:39 -04:00
Harwinder Sekhon 4d54fddf80 Docs: delete DOCUMENTATION.md — both halves superseded
Unlike USAGE.md (a USD/JPY-era fossil), this doc was structurally sound:
right pair, real architecture, correct composite formula, and 8 of its 9
regime-weight cells match engine.py. It is removed because both of its
sections now have live successors, and because the numbers it does state
are wrong in ways that mislead:

- Section 1 (trader-facing) → docs/user_guide.html + beginner_guide.html,
  which are newer and cover the trailing-stop exit model this doc never
  mentions.
- Section 2 (developer reference) → CLAUDE.md.

Stale/incorrect content it carried: the entire quality-grade table was
wrong (claimed A+ 45/70, A 30/60, B 20/50, C 12/40, D 8+; the code grades
40/65, 33/55, 28/50, 25/45, 15), Divergence's trending weight was listed
2.0 vs 1.0 in code, and it predates the 2026-07 work entirely — zero
mentions of XAU_PB or trailing exits (the actual default exit model), and
near-zero coverage of the EUR/USD port.

CLAUDE.md's header now points at docs/ as the human-facing docs instead
of disclaiming this file.
2026-07-16 09:34:04 -04:00
Harwinder Sekhon aaf1d37b14 Docs: delete stale USAGE.md and superseded pullback plan
USAGE.md documented a different product: USD/JPY (unsupported — the
code is XAU/USD + EUR/USD), 16 main.py flags that don't exist
(--backtest, --optimize, --auto-session-filter, …; main.py has five),
and backtest.py/metrics.py, which aren't in the repo. Two claims were
actively misleading: it called the session filter "default off" (it is
True by default, 03:00-23:00 not 07-16) and gave regime-min-adx as 30
(real default 25). CLAUDE.md now covers the CLI, bot and backtester
correctly, so this is a delete rather than a rewrite.

PULLBACK_ENTRY_PLAN.txt was a pre-implementation draft whose spec no
longer matches what shipped — it specifies a 0.5-2.5 ATR retrace window
and a detector in engine.py, while XAU_PB ships 0.8-6.0 (the drafted
window rejected 81% of valid pullbacks) with the detector in
signal_aggregator.py so live and backtest share one code path. Its
validated outcome (+38.3R / 254 trades / 12mo M5) and every shipped
gate are recorded in CLAUDE.md and the config docstrings.

Also drops the now-dangling references to both files from CLAUDE.md
and the config.py XAU_PB comment.
2026-07-16 09:30:50 -04:00
Harwinder Sekhon 18b5ef27d2 Docs: rewrite CLAUDE.md as a complete codebase reference
Replaces the summary-level doc with a full spec derived from a read of
every core module, so contributors (and low-context models) don't have
to infer behavior or rediscover tuned values:

- repo map with line counts; flags DOCUMENTATION.md/USAGE.md as stale
- every config knob with its real default and the backtest evidence
  behind it; marks the legacy/unused fields (SignalConfig.weights,
  min_confidence, cooldown_bars, XAU h4/h1 gates) so nothing new is
  built on dead knobs
- exact indicator logic for all 9 scored indicators + ATR context,
  regime classification rules, regime-aware RSI zone tables, and the
  regime-multiplier table copied from _compute_composite (the old
  table did not match the code)
- aggregator pipeline in source order: MTF agreement, the 12 penalties
  and the diminishing cap, exhaustion block, signal/grade thresholds,
  and gates 6b-6i with exact numbers
- all three trade-setup generators with SL/TP geometry, spread gates,
  and the trailing-exit model
- bot/CLI/news/data-provider/backtester mechanics incl. realism model
  (raw-path resolution, next-open entry, SL-first ties, parallel replay)
- "Rules for making changes": validation acceptance rule, pair
  isolation, live==backtest parity, ROC's load-bearing weight

Also corrects details the old doc got wrong: test count is 179 (not
~176), EUR ships off by default, backtest/*.csv are gitignored local
artifacts rather than test-enforced pins.
2026-07-16 09:26:06 -04:00
Harwinder Sekhon 26f0681ad1 Docs: user guide + beginner guide PDFs
Two printable guides in docs/ (HTML source + weasyprint-rendered A4
PDFs): a trader-facing user guide (signal anatomy, trail management,
M5/M15/H1 comparison from the 12mo books, expectations, commands) and
a plain-language beginner edition (glossary, worked sizing example,
step-by-step trailing-stop walkthrough, H1-first guidance).
2026-07-12 13:42:21 -04:00
Harwinder Sekhon dde41fbeb8 Audit fixes: mixed-regime bias, live==backtest session parity, parallel replay
Fix 1 — mixed-"trending" regime (ADX strong, EMAs unaligned) was
hard-coded bullish in 5 sites: RSI reinterpretation used the uptrend
playbook (RSI 38-55 -> STRONG_BUY conf 75), BUY-only confidence
bonuses, BUY-only extension penalties, no counter-trend penalty.
Now: extremes-only RSI (as transitioning), no directional bonuses,
symmetric extension penalties. 6yr validation: H1 +67.2->+61.9R
(mixed skew 16B/11S -> 4B/7S), H4 +31.6->+27.5R (the 4 bias BUYs),
M15 +152.6->+154.7R, M5 +207.1->+206.1R with maxDD 28.7->24.3R.
The ~8R aggregate give-back is bull-market subsidy from the bug.

Fix 2 — live session filter now applies the EXACT replay-loop
predicate (session window + blocked hours on the signal bar's CLOSE
time, from the bar timestamp not wall clock). Replaces a wall-clock
filter gated on a "XAU STRATEGY" marker that no code writes, which
made live diverge from every validated book.

Fix 3 — delete dead _check_xau_oversold_bounce and
_generate_xau_trade_setup (unreachable; the latter hardcoded BUY and
skipped the spread/RR gates).

Backtester: parallel bar replay via forked workers (--workers,
default auto = cores-2 capped 12). Per-bar evaluation is pure; the
entry cooldown is applied in an ordered phase-2 pass. Proven
byte-identical to sequential on the same window; ~300 -> 1030-1680
bars/s (6yr M5: ~26min -> 5-8min). --workers 1 = sequential path.

Tests: 179 passed (new mixed-regime RSI regression test).
2026-07-11 22:57:37 -04:00
Harwinder Sekhon 6f69b9dd90 XAU: two validated per-TF gates — M15 mixed-regime block, M5 hour-9 block
M15 mixed-"trending" regime block (ADX strong, EMAs unaligned =
momentum without direction): -6.8R/24 trades on 12mo, negative both
halves; blocking scores +65.7R/499 PF 1.38 DD 10.4R vs +59.0R/521
PF 1.32 DD 12.6R, better both halves, OOS 2024-06->2025-03 neutral
(-1.4R = the 4 such trades there). M5 never trades this regime.
New XauStrategyConfig.blocked_regimes / sell_blocked_regimes knobs
(default empty) + aggregator gate 6g2; M15 factory opts in.

M5 hour-9 UTC block (post-London-open whipsaw): -28.9R/143 over the
6-year book, negative 5/7 calendar years. Full-history replay with
block: +207.1R vs +178.3R, max DD 28.7R vs 35.5R, 2023 -13.0R ->
breakeven; tuned 12mo book unchanged (139.0 vs 139.3). Hour 9 is
positive on M15 (+1.7R 6yr, +8.6R OOS) so the block is factory
M5-only.

Rejected after validation (documented in xauusd.py + session memory):
transitioning-SELL block (bull-market artifact, +5.7R in 2024 OOS),
M15 trending_down block (M5 cross-TF contradiction), losing-streak
circuit breakers, 30d efficiency-ratio trend gate, and all entry-level
indicator filters (winners == losers at entry on every feature).

Full-history context (Twelve Data 5min starts 2020-03-16): M5 6.3yr
+207R PF ~1.3 post-fix with losing year 2021; M15 +152.6R PF 1.19
DD 49.7R — edge is regime-dependent, chop years are structural.

Tests: 178 passed (new gate test + factory-equivalence updates).
2026-07-11 21:38:42 -04:00
Harwinder Sekhon a2bf80054c Telegram: subscriber channel feed, owner lock, full risk disclaimer
- TELEGRAM_CHANNEL_ID mirrors auto-scan signal + breach alerts to a
  subscriber channel (Stars-subscription monetization; Telegram manages
  membership). Feed keeps running even when the owner's DM auto-scan
  is toggled off.
- _owner_only decorator locks all commands/buttons to TELEGRAM_CHAT_ID;
  others get a refusal pointing at the channel.
- Every signal-bearing message (detailed signal, scan summary, all-TF
  summary, breach alerts) now ends with a not-financial-advice /
  educational-use disclaimer via a single _DISCLAIMER constant.
2026-07-11 18:37:50 -04:00
Harwinder Sekhon 60fb526fab EUR: document closure of calendar + H1-hybrid avenues
Surprise-conditioned entries lose 2-5p at every horizon both years;
avoidance filter marginal for EUR and profit-removing for XAU; the
H1-direction + M15-pullback prototype failed its pre-registered kill
criteria (2024-25 OOS negative). EUR stays informational-only.
2026-07-11 10:26:32 -04:00
Harwinder Sekhon 18abff85a3 XAU M15: adopt 1R minimum targets (per-timeframe, validated)
Finishes PENDING_STEPS Step 1 under the current trailing-exit model.
M15 window 2025-03-30->2026-05-31 ($0.30 spread): +116.6R/460 PF 1.83
vs +105.4R/424 PF 1.82 baseline, better in both halves; 2024-06->
2025-03 OOS +30.3R vs +10.6R control. Same knob on the pinned M5 book
is WORSE (+131.3R/699 vs +139.9R/547) so the override is M15-only in
the xauusd factory, per the pre-registered decision rule. M5 path
unchanged (factory override gated on primary_tf == M15).
2026-07-11 10:22:33 -04:00
Harwinder Sekhon 183f148bb4 Live news gate: stop capping signal confidence near events (XAU default)
Replaying the live danger/caution gates against the validated 12mo XAU
M5 book showed they suppress 25-35 trades worth +4.6..+7.2R/yr — trades
entered near USD releases outperform the book average (+0.475R vs
+0.256R). New NewsConfig.event_risk_caps_signals (default False =
XAU-tuned) keeps EVENT RISK warnings in alerts but no longer guts
confidence; EUR factory re-enables the caps (evidence is gold-specific).
Pinned XAU M5 regression: 547 trades, +139.94R, trade-identical.
Tests: 177 passed.
2026-07-11 10:02:27 -04:00
Harwinder Sekhon 528a308d36 Add calendar research infra: FF scraper + verified event store
Scrapes Forex Factory weekly pages (epoch datelines, actual/forecast/
impact/betterWorse) into backtest/cache/calendar.parquet (gitignored;
rebuild with the scraper). Timestamps verified: NFP/CPI/ECB times exact
across DST, and high-impact event bars average 3.5x ATR with the spike
vanishing at +/-1h shifts. calendar_store provides precise_events and
event_window_mask for screens and counterfactuals.
2026-07-11 09:47:11 -04:00
Harwinder Sekhon 9c72e90ce2 EUR M15/H1 investigation: document no-validated-edge verdict
Five independent tests at 1-pip cost (M15 wide-open both windows,
transitioning-slice OOS, mean-reversion trade sims, forward-return
screens on M15+M5, H1 both windows) — nothing clears cost in both
2024-25 and 2025-26. Root cause: spread is ~10% of M15 ATR on EUR.
EUR stays out of scan_pairs; open avenues documented in docstring.
2026-07-11 09:02:24 -04:00
Harwinder Sekhon 8b086f4f49 EUR M5: commit best-found config after realism re-tune — no validated edge
12mo grid campaign (2025-07-08 → 2026-07-08, 1-pip round-trip cost,
train/val split at 2026-01-08). Donor config scored -6.7R/34 trades.
Findings: transitioning regime -27R/88 wide-open; A-grade entries lose
(-26R) while B-grade wins; Fridays -17R; hours 8/12/13/18/19 UTC bleed;
score 30-35 band holds the train-half losses; trailing (0.45R arm/0.5R
dist) beats static TP1 and gold's 0.6/0.75.

Best-found config: -1.06R / 53 trades / PF 0.93 / 62.3% WR — breakeven,
FAILS acceptance (positive both halves, PF>=1.3). Per plan, EUR ships
functional but stays out of default scan_pairs. Grid results and
methodology documented in the module docstring; full numbers in
backtest/cache/eurv2_*.csv.
2026-07-10 22:14:49 -04:00
Harwinder Sekhon f02f4b4d37 Update CLAUDE.md for multi-pair architecture 2026-07-10 21:16:34 -04:00
Harwinder Sekhon 74829541d6 Live path: config-driven multi-pair scanning (Telegram bot + CLI)
- telegram_bot: SCAN_PAIRS/_WS_TO_PAIR/SCAN_BUTTONS all derive from
  PlatformConfig.scan_pairs (the single pair on/off switch); buttons and
  keyboards are generated per pair; _run_signal_sync builds per-pair
  configs via pair_signals factories (was: XAU defaults for any symbol);
  scan-all works per pair; PB badge matches any *_PB tag; daily-rotating
  file logs in logs/telegram.log (30-day retention)
- main.py: --pair argument (XAUUSD | EURUSD + aliases) routed through
  pair_signals factories; price display is precision-aware (2dp gold,
  5dp FX)

scan_pairs default stays [XAUUSD] until the EUR M5 re-tune validates.
2026-07-10 21:13:42 -04:00
Harwinder Sekhon 6546b89d6b Add EUR/USD test suite + map EUR news events to EUR/USD
Module 10 (21 tests): EUR config/registry/factory, EUR filter pipeline
(regime/quality/score/confidence/RSI-extended/weekend), EUR trade
geometry (swing SL buffers, pip-denominated spread gate, per-pair gate
contrast), and cross-pair isolation regressions (XAU report bit-equal
with eur_strategy flipped; XAU gates never fire on EUR reports).

news_engine: _get_affected_pairs now maps USD events to EUR/USD too and
adds an EUR entry (calendar warnings reach EUR scans).

175/175 pass.
2026-07-10 21:06:55 -04:00
Harwinder Sekhon 9ed07d4b65 Backtester: EUR/USD support with per-pair configs and default spread
- EURUSD aliases in PAIR_ALIASES
- main() builds PlatformConfig via pair_signals.PAIR_PRODUCERS factories
  (per-pair regime weights/gates/sessions); unregistered pairs keep the
  inline construction. XAU factory is asdict-identical to inline
  (enforced by test) so XAU runs are unchanged.
- --spread default now resolves from PAIR_INFO default_spread
  (XAU $0.30, EUR 0.0001) instead of a hardcoded gold value
- EUR strategy tag: renamed EUR_PB→EUR, detected in both bullish and
  bearish reasons[0] (SELL setups mark the bearish list)

Smoke-validated: 30-day EUR M5 run produces EUR-tagged trades with
1-pip round-trip cost charged.
2026-07-10 21:02:44 -04:00
Harwinder Sekhon 0da5ed03af Add EUR/USD signal gates, trade geometry, and weekend/session filters
Re-grafted from feature/eurusd-support onto the evolved aggregator:
- 6h EUR gate chain (regime, quality, exhaustion, EMA200, session,
  score/confidence, volatility, indicator agreement, RSI-extended),
  active only for EUR/USD with eur_strategy.enabled
- _generate_trade_setup: EUR branch reads EurUsdStrategyConfig only
  (min_sl_pips, min_rr, mature-move + quality-scaled TP, sl_atr_floor,
  swing-buffer SL) with a pip-denominated spread gate; adds a
  pip_value<=0 guard; non-EUR branch is arithmetically unchanged
  (sl_swing_buffer=1.0 extraction only)
- EUR weekend/weekday pre-filter in analyze_pair mirroring the XAU block
- "EUR STRATEGY" reasons[0] marker so the backtester tags EUR trades
2026-07-10 20:57:54 -04:00
Harwinder Sekhon db50bf02d7 Gate all XAU-only aggregator logic by pair
The SELL quality gate (6e), SELL volatility gate (6g), M5 noise gates
(6h), ATR trade-setup generator, and M5 pullback archetype are all
gold-tuned but previously keyed only on xau_strategy.enabled or the
timeframe, so they would have silently applied gold constants to any
other pair. Each now also requires display_name == "XAU/USD".

No behavior change for XAU (conditions are tautologically true for the
only registered pair today) — verified by test suite and pinned-window
backtest CSV diff.
2026-07-10 20:53:23 -04:00
Harwinder Sekhon 1bc48915bd Add pair_signals per-pair producer registry
Ported from feature/eurusd-support: PAIR_PRODUCERS registry with
make_config() factories per pair. xauusd.make_config is a pure
PlatformConfig-defaults wrapper (equality with the inline construction
is enforced by test); eurusd.make_config carries the donor branch's
regime weights and gates as a pre-realism baseline.

Adds 4 registry tests. 154/154 pass.
2026-07-10 20:52:06 -04:00
Harwinder Sekhon 9d3729ea73 Add EUR/USD pair constants and EurUsdStrategyConfig
- EURUSD in PAIR_INFO; new per-pair price_precision and default_spread
  fields on both pairs (backtester consumes default_spread when --spread
  is omitted)
- EurUsdStrategyConfig ported from feature/eurusd-support plus EUR-local
  cost knobs (round_trip_cost=1 pip, min_tp_cost_multiple=10,
  min_sl_pips=8) so SignalConfig's gold-dollar values never apply to EUR;
  donor-tuned values are a pre-realism baseline pending re-tune
- PlatformConfig.eur_strategy field; track EUR in news currencies
2026-07-10 20:50:08 -04:00
Harwinder Sekhon f3b90d3862 Fix 3 engine bugs: swing-point NaN false positives, ROC div-by-zero, ADX epsilon
Ported from feature/eurusd-support (e6e09c5), pair-neutral fixes:
- _find_swing_points: all(<empty generator>) is vacuously True, so a value
  surrounded by NaNs was reported as a swing point; now requires at least
  one valid neighbor on each side
- ROC: guard against zero close at the lookback positions (inf scores)
- ADX: epsilon denominator instead of NaN-replace to avoid NaN cascade

Adds 8 regression tests (Module 9). 150/150 pass.
2026-07-10 20:48:56 -04:00
Harwinder Sekhon ea306df484 Overhaul news engine: fix false signals, add surprise detection + caching
Improvements from a live-verified audit of the news/calendar pipeline:

- Currency tagging: keyword matching now uses letter-boundary regexes
  and drops ambiguous "us "/"eu "/"uk " tokens — "versus" no longer
  tags headlines as USD (confirmed live false positive polluting the
  XAU/USD sentiment average).

- Fetch caching: calendar events and headlines are cached class-level
  (10 min TTL, 60s retry after empty/failed fetch, lock-guarded for
  concurrent scans). Stops per-scan refetching that was tripping the
  Forex Factory rate limit; TTL stays under the 30-min pre-event
  buffer so the danger window can't go stale.

- Surprise detection: EconomicEvent.surprise is computed from parsed
  actual vs forecast (handles %, K/M/B/T suffixes, negatives). A
  surprise print widens the post-release caution window to 45 min;
  inline releases keep the original 15 min.

- Gold-aware sentiment: headlines are scored per currency, with
  monetary-policy keywords (hawkish/rate hike/tightening vs dovish/
  rate cut/easing/stimulus) inverted for XAU — "hawkish Fed" headlines
  scored +1.0 for gold before, -1.0 now (verified on live feed).

- New sentiment sources: Google News gold-focused query and CNBC
  Economy RSS (verified live; 40 → 74 unique headlines). Added
  cross-feed title dedup so syndicated stories count once.

- events_today now buckets by UTC date (feed timestamps carry US-
  Eastern offsets); --json output includes news_description.

- Removed dead NewsConfig.min_impact and unused HIGH_IMPACT_EVENTS.

Tests: 142 passed (7 new covering boundary matching, surprise
classification, post-event windows, cache reuse, gold inversion).
2026-07-10 09:43:11 -04:00
harwinder 1038a53bac Add M5 pullback entry archetype (XAU_PB): +140R combined on 12mo
Second entry pattern alongside the Keltner-breakout: enter on the
retrace to EMA21/KC-mid in an established trend (EMA >=3/4 aligned,
ADX>=20, retrace 0.8-6.0 ATR, resumption bar closed back in trend
direction, RSI in the pullback zone not the extended zone). Detector
lives in the aggregator so live and GPU-backtest paths share one
implementation. Own gates (composite floor 15, confirmation-TF
agreement, MTF opposition, vol regimes) with a diagnostic funnel.

PB stops anchor to the pullback swing (~$8 vs ~$25 breakout), so PB
trades get their own trail arm (0.8R vs 0.6R global) via per-trade
trail overrides in the backtester and strategy-aware Telegram alerts.

12mo M5 validation: PB alone +38.3R/254 trades, positive both halves
and all four full quarters; core breakout trades undisturbed
(+101.9R vs +105.0 baseline). Combined book: 549 trades (~10.6/wk),
+140.1R, PF 1.78, DD 11.3R vs 311 trades/+105.0R before.
2026-07-09 14:21:59 -04:00
harwinder 305f1d1857 Add pullback entry pattern implementation plan
Design doc for a second M5 entry archetype: enter on retrace to
EMA21/KC-mid in an established trend instead of at Keltner-breakout
extension. Includes detector spec, geometry, validation protocol
(both-halves + quarterly + displacement checks), and the +105R
baseline to beat. Session/frequency knobs (blocked hours, cooldown)
were tested 2026-07-09 and confirmed already optimal — this is the
remaining evidence-backed entry-side direction.
2026-07-09 13:51:00 -04:00
harwinder 0393087168 Add M5 gates 5-6 and trailing-stop exits (+48R → +105R on 12mo)
Gates 5-6 in the M5 filter: block 'high' vol regime entries (chop
transition — 61% WR, negative total while low/normal/extreme all
positive) and block SELLs with RSI<25 (dip already extended, mean-
reverts before target). 365d: 419→311 trades, +41→+48.4R, DD 11.2→8.0R.

Replace the fixed RR-0.5 take-profit with a trailing stop: arm at
+0.6R favorable excursion, trail 0.75R behind best price, stop levels
update at bar close (conservative). The RR-0.5 target was the binding
constraint — winners kept running after TP. Whole trail parameter grid
(arm 0.4-1.2 × dist 0.5-1.25) lands +90-103R, a plateau not a spike;
also +91.7R on the pre-gate 419-trade population. TP1 stays advisory;
Telegram alerts include the arm price and trail distance. max_hold
200→300 bars (trailing exits need room; +10R).

12mo M5, same 311 trades: +105.0R total, PF 2.27, DD 7.9R, 61% WR,
train/val +54.6/+50.4, every quarter positive.
2026-07-09 13:34:57 -04:00
harwinder 8687f98b39 Fix M5 signal quality: ATR ordering bug, noise gates, blocked-hour leak (+41R)
ATR was computed after EMAs so ribbon compression detection was dead code
on all timeframes — moved ATR first. Extended M5 swing S/R lookback from
50 to 120 bars. Added four M5-specific filters: min score, MTF agreement,
confirmation TF conviction, and SELL quality cap (A-grade M5 shorts enter
at dip extremes and lose). Fixed backtester blocked-hour check to use bar
close time matching the live bot. Added trade geometry config (spread gate,
SL floor, TP floor RR) and primary_tf mismatch guard in backtester.

Backtest: 1133 trades/+11.5R → 419 trades/+41.0R, PF 1.03→1.37, DD 21→11R.
Train/val split positive in both halves (+19.8R/+21.3R).
2026-07-09 12:48:36 -04:00
harwinder 2291fc3936 Disable TP tightening: tuned pre-spread, halves total R under real fills
A/B on M15 2025-03-30→2026-05-31 with $0.30 spread + gap-through fills
(511 identical entries, only exits differ):
  factor 0.8 / min_rr 0.4: +36.1R (avg +0.071R, SELL +1.2R)
  factor 1.0 / min_rr 0.5: +66.8R (avg +0.131R, SELL +9.6R)
Worse in both train and validation halves and both directions. The 20%
tighter targets buy +1.8pt win rate but shrink every win while the fixed
spread stays constant. The knob remains for re-tuning with costs included.
2026-07-08 21:35:59 -04:00
harwinder 3a85c6e23b Add WebSocket dual-connection test script
Probes whether Twelve Data allows two simultaneous WS connections
for one symbol (SDK-based, 30s run, prints per-connection verdict).
2026-07-08 21:32:46 -04:00
harwinder 09ee5208d4 Telegram bot: WS heartbeat, breach alerts, alert dedup, interactive menu
WebSocket:
- Send heartbeat every 10s — Twelve Data drops idle connections, so the
  monitor previously died on weekends/quiet hours and churned reconnects
- Log failed symbol subscriptions from subscribe-status events
- Stamp fresh live tick price + age into every alert message

Alerts:
- Re-enable level-breach alerts: WS breach triggers M15 rescan and sends
  the report when actionable (was scanning and discarding the result)
- Dedup: identical (signal, quality, direction) per pair/TF is sent once,
  re-armed after 1h — no more repeat alerts every candle
- Configurable minimum alert quality (C+/B+/A), cycled from the menu
- Trade setup shows dollar risk/reward alongside pips

Interactive menu (/start, /scan):
- Timeframe scans, Scan All, auto-scan ON/OFF toggle, alert-quality
  cycle, status, and help as inline buttons
- Markdown-safe edits: 'message not modified' swallowed, parse failures
  fall back to plain text; exception text escaped
2026-07-08 21:32:46 -04:00
harwinder 1f076c0b54 Fix live/backtest parity, SELL-side losses, and indicator repairs (+42% total R)
Live pipeline:
- Drop still-forming candle before analysis (signals no longer repaint)
- Direction-aware MTF agreement + opposition gate (no trade against 2-of-3 TF consensus)
- SELL volatility gate: block shorts in low/normal vol regimes (config sell_vol_regimes)
- Divergence: 40-bar lookback, price/indicator swings must align within ±2 bars
  (fired on 0.5% of trades before, 29% after), trending weight 2.0→1.0
- RSI regime_aware by default; transitioning regime only signals at extremes
- ADX ±DM tie no longer nudges bearish; TP2 clamped beyond TP1

Backtester realism:
- Charge $0.30 round-trip spread (--spread), gap-through SL/TP fills at bar open
- Resolve trades on RAW price path (cache stores raw bars, cleaning runs on load)
- Weekend/session filters use bar-close decision time to match live wall clock

Validated per-step on M15 2025-03-30→2026-05-31 under identical pricing:
+46.9R → +66.8R, win rate 69.9% → 73.0%, SELL side -2.3R → +9.6R.
Tested-and-reverted: ROC weight removal (-13R), BB fade suppression (-11.6R),
candlestick de-weight (-2R) — composite thresholds are co-tuned with weight mass.
2026-07-08 21:32:46 -04:00
harwinder baeee4997a Fix trade setup generation broken by TP tightening factor
The TP floor calculations used raw min_rr (0.4) but after 20% tightening
the effective RR dropped to ~0.32, failing the min_rr check and returning
None for all trade setups. Pre-compensate the floor with min_rr/tf so
post-tightening RR still meets the minimum.
2026-05-24 18:22:57 -04:00
harwinder bb40bceb4c Tighten TP targets by 20% for faster hits (configurable tp_tightening_factor=0.8)
Applied uniformly across all 3 trade setup functions. Lowered min_rr_ratio
0.5→0.4 proportionally to preserve trade acceptance rate. SL unchanged.
2026-05-22 10:44:52 -04:00
Harwinder c46bb85012 Enable M5 auto-scan alerts and remove live dashboard
M5 already broadcast through the shared _scan_timeframe path but was
documented as data-only (docstring, CLAUDE.md, DOCUMENTATION.md, plus a
phantom `if tf == M5: return` step that never existed in code). Align all
docs/comments so M5 alerting at C+ grade is the intended, documented behavior.

Remove the live dashboard entirely per request: the 30s refresh loop,
per-scan refresh, boot send, and the dashboard-only stats layer
(_signal_stats / _stats_updated / _update_signal_stats and its call sites,
_format_stats_dashboard and its helper constants). Drop now-unused ZoneInfo
import and dashboard mentions from help/autoscan/boot messages and the test
suite. The WebSocket level-breach monitor and level cache are untouched.

Tests: 136 passed, 0 failed.
2026-05-20 12:45:27 -04:00
harwinder cc53448c34 Fix weekend filter: use precise forex market hours (Fri 22:00–Sun 22:00 UTC)
The old `weekday() >= 5` check blocked all of Sunday, but the forex
market reopens Sunday 22:00 UTC (Sydney open). Now uses proper time-based
check so the bot fetches data and generates signals on Sunday evening.
2026-05-17 19:23:21 -04:00
harwinder 15a7074574 Add backtest scripts to version control
Track gpu_backtest.py, diagnose_filters.py, and instructions.
Data files (CSVs, parquet cache) remain gitignored.
2026-05-15 19:49:30 -04:00
harwinder 35e9b3750d Reduce A-grade TP when exhausted: RR 0.75→0.50 on 2+ exhaustion tags
A-grade trades entered exhausted moves with TP too far (0.75R), causing
57.7% WR and +0.25R over 30 days. Store exhaustion_count on SignalReport
and scale TP to B-level (1.0×) when A-grade has 2+ exhaustion tags.
Backtest: A-grade 73.1% WR / +3.25R, B/C unchanged.
2026-05-15 19:46:31 -04:00
harwinder 68cc5b28a5 Align FilterConfig with XAU strategy: blocked hours [0,1,2], session 3-23 UTC, Mon-Fri 2026-05-15 15:36:03 -04:00
harwinder 5c0adfab27 Add M5 to Telegram scan buttons and Scan All TFs 2026-05-15 15:26:00 -04:00
harwinder b7bfd9c830 Reduce blocked hours, enable M5 alerts, show NY time on dashboard
- Blocked hours reduced to 0-2 UTC only (daily rollover)
- M5 signals now broadcast to Telegram like M15/H1/H4
- Live dashboard shows New York time (auto EDT/EST via zoneinfo)
2026-05-15 15:23:48 -04:00
harwinder 04328eaeab Refocus on XAU/USD: add momentum indicators, tune TP, exhaustion block
Strip EUR/USD, GBP/USD, USD/JPY pairs and strategies from entire codebase.
Add 3 new scored indicators (ROC, Keltner/TTM Squeeze, ADX slope) and
enable MACD-as-pillar + divergence detection for early trend entry.
Introduce "transitioning" regime (ADX < 25, slope > 3.0) to catch trend
births before ADX confirms. Gold-specific composite weights per regime.

Tune TP multipliers: quality-tiered scaling (A+/A get 1.5-2.0x ATR via
XAU ATR method, B/C use swing-level targeting). Backtest: +24R total,
65% WR, PF 1.12 over 365 days.

Add momentum-confirmed exhaustion block: hard-reject trades when 2+
static exhaustion signals fire AND ROC/MACD confirm fading momentum.
Prevents late trend entries without killing strong trend trades.

Telegram: simplify buttons to timeframe-only labels, add "Scan All TFs"
multi-timeframe summary, add /scan command, fix BB/KC level breach keys,
fix price formatting (.2f for gold), fix deprecated asyncio call, fix
"1 pairs" grammar and 60s/30s dashboard refresh mismatch.
2026-05-15 10:51:51 -04:00
harwinder bb3c475f22 Add web dashboard (Redis + FastAPI + WebSocket), scope XAU penalties
Signal daemon (signal_web.py): candle-boundary scanner mirroring
telegram_bot.py, publishes serialized SignalReport JSON to Redis.
Web API (web/app.py): thin FastAPI reader behind Apache, serves
/api/signals REST + /ws WebSocket from Redis pub/sub.
Frontend (web/static/index.html): 3-pair grid, color-coded signals,
indicator breakdown per timeframe, trade setup display.

Also: XAU-specific penalty scoping in signal_aggregator (softer
confidence penalties for gold, original strict values for forex),
pair-aware session filter routing XAU to its own blocked hours.
2026-05-14 23:15:37 -04:00
harwinder 31ecdf4292 Fix news engine (keywords, calendar fallback, RSS URL), remove Telegram guards
News engine: expand sentiment keywords (18%→60% headline coverage), add
MyFXBook RSS as fallback calendar source when Forex Factory returns 429,
add retry/backoff on calendar API rate limits, update stale ForexLive URL
to investinglive.com, add date formats without seconds for MyFXBook RSS.

Telegram bot: remove flood control, rate limiter, and dashboard failure
tracking — all sends go directly through bot.send_message/edit_message_text.
2026-05-14 13:42:32 -04:00
harwinder 39c91c1aa4 Fix duplicate-index crash in data validation, add Telegram flood control
- Move index dedup before OHLC consistency fix in _validate_and_clean
  to prevent crash when df.loc[dup_idx] returns Series instead of scalar
- Add global flood backoff: detect RetryAfter, pause all sends/edits
  until ban expires instead of retrying and escalating the ban
- Stop dashboard edit-fail → new-message spam cascade: track consecutive
  failures, only resend after 3 failed edits
- Increase dashboard refresh from 5s to 30s, min send interval to 1.5s
- Add comprehensive QA test suite (140 tests across all 7 modules)
2026-05-14 13:04:28 -04:00
harwinder cd69980f1f Harden Telegram bot, add EUR/GBP strategies, parallelize scans
Telegram reliability: rate-limited sends (1 msg/s), dashboard lock,
WebSocket JSON/price error handling, markdown escaping, level cache
TTL, cooldown cleanup, breach task tracking, graceful shutdown.

Telegram speed: parallelize 3-pair scans with asyncio.gather(),
increase thread pool 2→4, parallelize boot-up initial scan.

Strategy: add EUR/USD and GBP/USD pullback-buy strategies with
per-pair configs, optimize XAU TP to 0.9×ATR, block hours 8/9
for XAU and hour 17 for GBP.

Backtest: add --max-hours flag to evaluate.py for time-exit support.
2026-05-13 21:07:49 -04:00
Harwinder 67a9169eb3 Fix dashboard TF key mismatch, increase refresh to 5s
Timeframe keys stored as enum values (15min/1h/4h) but dashboard
looked for M15/1h/4h — GBP/USD showed no price or signal text.
Now uses Timeframe enum directly with display labels. Reduced
dashboard refresh from 60s to 5s with throttle guard.
2026-05-13 18:51:02 -04:00
Harwinder f99fcc60b9 Add live dashboard with 60s auto-refresh, disable breakout alerts
Replace per-scan Telegram messages with a single dashboard message
that edits in-place every 60 seconds. Each pair shows all TFs
horizontally with short signal labels (Buy/Sell/Neutral). WebSocket
live prices feed into the dashboard. Level-breach alerts disabled
(WS still updates cache silently). Removed /stats command.
2026-05-13 18:34:44 -04:00
harwinder 4d3f05e2ba Fix live session filter overriding XAU strategy signals
The general session filter ran after analyze_from_dataframes() and
wiped out XAU oversold bounce signals at most hours. Now skips the
session filter when the XAU strategy has already fired — the strategy
has its own backtested hour filter.
2026-05-13 17:31:46 -04:00
harwinder 2deea1ed7f Add XAU/USD oversold-bounce strategy (79.7% WR, PF 2.06)
BUY-only strategy for gold: enters when M15 RSI drops below oversold
threshold while price remains above EMA200 on all three timeframes
(M15, H1, H4). Uses ATR-based TP/SL (0.7x/2.0x) sized from M15 ATR.
Isolated to XAU/USD only — other pairs completely unaffected.
2026-05-13 17:25:49 -04:00
harwinder 89030c96b6 Backtest tuning, data freshness guard, Telegram improvements
- Drop USD/JPY from scan pairs (zero signals across all timeframes)
- Disable divergence indicator (zero impact in 14-config lab test)
- Parallelize evaluate.py with ProcessPoolExecutor (16 cores)
- Add --pair CLI filter to evaluate.py for single-pair backtests
- Add data freshness guard: stale cache auto-invalidated, primary TF
  stale data suppresses signals to prevent trading on old data
- Add /cache command: shows fetch age and last bar per pair/TF
- Add /clearcache command: wipes in-memory cache for fresh API fetch
- Remove BB from WebSocket breach alerts (keep structural levels only)
- Show live prices only for WS-monitored pairs (EUR/USD, XAU/USD)
- Replace arrow emojis with text tags in indicator lines
2026-05-12 22:56:55 -04:00
harwinder 59a18b8bc6 Add project documentation and MT5 integration plan
DOCUMENTATION.md: Two-section guide covering the full project —
plain-English trader guide (indicators, scoring, trade setups) and
developer reference (architecture, data flow, every threshold/weight).

mt5.md: Architecture plan for automated order execution via MT5 bridge
(REST API on Windows, async client on Linux, risk-based lot sizing).
2026-05-11 19:50:21 -04:00
harwinder 49260abaa2 Fix trade setup generation, M5 data-only mode, WS symbol limits, command menu
- Trade setup TP now scales to SL distance ensuring min RR is met when
  no viable swing target exists; skip swing targets with poor RR
- Add float tolerance to RR gate to prevent edge-case rejection
- M5 auto-scan runs for level cache only, no Telegram broadcast
- WebSocket limited to EUR/USD + XAU/USD (Grow plan constraint)
- Register bot command menu via set_my_commands on startup
2026-05-11 18:58:40 -04:00
harwinder 4ce09c2f79 Suppress Telegram messages when no actionable signals found
Auto-scan still runs and updates the level cache every cycle,
but only sends to Telegram when quality A+/A/B/C signals exist.
2026-05-11 18:03:49 -04:00
harwinder 02eedcc875 QA fixes: H4 timing bug, /status + /help commands, WS cold-start
- Fix H4 boundary calculation using total minutes (hour*60+min) not just minute
- Add /help command listing all bot commands
- Add /status command showing WS connection, live prices, level cache, cooldowns
- Run initial M15 scan on boot to populate level cache before WS starts
- Update /start message to mention new commands
2026-05-11 17:16:07 -04:00
harwinder 8ead05432e Tune engine from backtesting: wider SL, capped TP, counter-trend filter
Backtest-driven fixes (0% win → 33% win, PF 0.00 → 1.05 over 7 days):
- SL: swing buffer 0.5→1.0 ATR, min distance 1.0→1.5 ATR (more room to breathe)
- TP: cap swing targets at ATR×3.0, fallback ATR×2.0 (was targeting unreachable levels)
- Mature-move TP: when score>30 (A/B grade), tighten TP to ATR×1.5 (move already extended)
- Counter-trend penalty: -15 confidence when signal fights regime direction
- EMA200 penalties halved: -20→-10 (was killing EUR/USD signals)
- Score thresholds lowered: BUY/SELL ±15→±12, STRONG ±40→±35
- Quality grades relaxed, C-grade now gets trade setups and Telegram alerts
- min_rr_ratio: 1.5→1.2
- Gitignore backtest/ directory
2026-05-11 17:00:12 -04:00
harwinder 90b7e0c54e Buy signal included 2026-05-11 16:37:45 -04:00
harwinder 058ef57d8f Pro tier upgrade: quant rearchitect + WebSocket level-breach monitor
- Restore XAU/USD (Pro plan), drop USD/CAD, raise rate limit to 55/min
- Reduce signal engine to 4 uncorrelated pillars (EMA, RSI, BB, Candlestick)
  with regime-adaptive weights (trending boosts EMA, ranging boosts RSI+BB)
- Add structure-based S/R (swing highs/lows) for trade setup targeting
- Fix confidence death spiral: all adjustments now additive, not multiplicative
- Dynamic R:R trade setup: SL behind swing level, TP at next structure level,
  reject if R:R < 1.5
- Fix news engine for all 4 pairs (was hardcoded to USDJPY)
- Add WebSocket monitor: streams live ticks, triggers M15 scan on level breach
  with 5-min cooldown per pair — catches breakouts between candle boundaries
- Add correlation filter (EUR/USD+GBP/USD, XAU/USD+USD/JPY exposure warnings)
2026-05-11 16:33:23 -04:00
harwinder cc66e65f2d Replace XAU/USD with USD/CAD — TwelveData has no gold data
XAU/USD is not available in TwelveData's symbol list.
Swapped to USD/CAD across config, scan pairs, Telegram buttons,
and added CAD to tracked news currencies.
2026-05-11 15:50:48 -04:00
harwinder 873cf5487e Add 4-pair candle-boundary auto-scanner (free tier safe)
Expand from 2 to 4 pairs (EUR/USD, USD/JPY, GBP/USD, XAU/USD) with
automatic scanning at M15 and H1 candle close boundaries. Scheduler
staggers H1 by 60s to stay within 8 credits/min; daily budget is
480 of 800 allowed. Cache TTL is now candle-duration-aware so cross-TF
confirmation data is reused without extra API calls. Spread estimate
uses primary-TF data instead of fetching M5 separately.
2026-05-11 15:30:11 -04:00
Harwinder a708609f8b Add EUR/USD support and 4-button Telegram scanner
Add EURUSD pair with dynamic pair info lookup. Telegram bot now shows
4 scan buttons: EUR/USD 15M, EUR/USD 1H, USD/JPY 15M, USD/JPY 1H.
Each indicator displays directional arrows in the message output.
Price formatting adapts per pair (5dp for EUR, 3dp for JPY).
2026-05-10 23:49:36 -04:00
Harwinder 7ccd8c4b04 Strip backtesting, rebalance indicator weights toward leading signals
Remove backtest.py, backtest_gpu.py, metrics.py, local CSV data,
Dukascopy loader, and all backtest CLI modes from main.py. Rebalance
engine weights: leading indicators (divergence, RSI, stochastic, BB)
now carry 66% of signal weight vs 34% for lagging (EMA, MACD). Add
ws_twelvedata.py for live tick streaming. Drop pyarrow and tqdm deps.
2026-05-10 23:40:15 -04:00
Harwinder 2e09237671 Add USAGE_LINUX.md — bot and CLI run instructions for Linux server 2026-05-07 23:12:55 -04:00
Harwinder a12084537d Add async Telegram bot — live scan button with trade setup
Introduces telegram_bot.py: python-telegram-bot v20+ async bot that
presents a persistent Scan Signal inline button. Each press fetches a
live USD/JPY signal via SignalAggregator and formats it with emoji,
MTF breakdown, top bull/bear reasons, and warnings. STRONG_BUY /
STRONG_SELL signals additionally render the full trade setup (entry,
SL, TP1, TP2, RR). Adds python-telegram-bot>=20.0 to requirements.txt.
2026-05-07 23:05:01 -04:00
Harwinder a55aca21b1 Round-2 weight optimisation — reduce EMA trend, boost stochastic
GPU sweep (36 combos, RTX 4070 SUPER) on H4 2022-2026 relative to the
round-1 baseline revealed EMA trend was over-weighted at 3.6. Halving
it and nudging stochastic up yields cleaner, higher-quality signals.

Weight changes (backtest_gpu.py _BASE_W + engine.py):
  ema_trend:  3.6 -> 1.8  (over-weighted; halving tightens signal filter)
  stochastic: 1.6 -> 2.4  (x1.5; improved WR to 40.4% in sweep)
  macd:       3.3          (unchanged — dominant signal, kept as anchor)
  adx/ichimoku/rsi/bollinger: unchanged

Combined result vs round-1 (H4 local data 2022-2026):
  Trades:      160 -> 73   (-87 — filters out marginal setups)
  Win Rate:   38.1% -> 43.8%  (+5.7pp)
  Profit Factor: 1.19 -> 1.52
  Return:    +21.30% -> +23.86%  (+2.57pp)
  Max DD:     -8.52% -> -9.32%
  Sharpe:      0.47  ->  0.71   (+51%)
  Sortino:     1.91  ->  3.06   (+60%)

Fewer trades at much higher quality. The lower EMA weight means only
strong, multi-indicator confluences clear the grade threshold — the
73 remaining trades have 43.8% WR vs 38.1% and expectancy nearly 3x.
2026-05-07 22:24:48 -04:00
Harwinder e57f4a1ecd Enable GPU scoring via CuPy — RTX 4070 SUPER confirmed 6.3x speedup
Install nvidia pip packages for CUDA 12 runtime DLLs (no full toolkit needed):
  nvidia-cuda-nvrtc-cu12, nvidia-cuda-runtime-cu12,
  nvidia-curand-cu12, nvidia-cublas-cu12

Add _register_nvidia_dlls() in backtest_gpu.py that auto-discovers and
registers nvidia package DLL directories (site-packages/nvidia/*/bin/)
via os.add_dll_directory() before CuPy is imported. Suppresses the
spurious "CUDA path could not be detected" warning from cupy._environment.

Backend line now reads: CuPy 14.0.1 (RTX GPU)
Benchmark: scoring pass on 100k bars — NumPy 1.1ms vs CuPy 0.18ms = 6.3x.
Results are identical to CPU path (deterministic).
2026-05-07 22:22:21 -04:00
Harwinder 17cb63e86e GPU backtester, optimised weights, local CSV data pipeline
## What changed

### backtest_gpu.py (new — ~1,300 lines)
Standalone GPU-accelerated backtester completely separate from backtest.py.
Computes all indicator series in one vectorised O(N) pass over the full price
history, then replays positions with a sequential state machine.

Key design:
- VectorIndicators: pandas ewm/rolling (C-optimised) for all 7 indicators
- CuPy GPU backend with automatic NumPy fallback (no hard dependency)
- Two-phase API: prepare_arrays() once, score_and_run() many times
- --optimize flag: 36-combo weight sweep (1 baseline + 7 ablations + 28
  multiplier runs) — calls prepare_arrays() once, replays all combos in seconds
- Full CLI: --tf, --local, --start/--end, --regime-filter, --regime-tf,
  --session-filter, --optimize, --journal, --journal-csv

O(N) vs O(N^2): backtest.py recomputes all indicators per bar (O(N^2));
backtest_gpu.py computes each indicator series once and reads bar T in O(1).
On 100,000 H1 bars: ~4 seconds vs >10 minutes.

### data_provider.py
Added load_local_csv() method — reads MetaTrader tab-delimited CSV exports
from market_data/ directory. Format: Time|Open|High|Low|Close|Volume|Spread
(spread column dropped on load, timestamps localised to UTC). Applies the same
_validate_and_clean() pipeline as the Twelve Data path. LOCAL_CSV_MAP maps
each Timeframe enum to its filename.

market_data/ files added (MetaTrader exports, UTC):
  USDJPY_M15.csv  100,000 bars  2022-05-04 to 2026-05-08
  USDJPY_H1.csv   100,000 bars  2010-04-23 to 2026-05-08
  USDJPY_H4.csv    25,855 bars  2010-04-22 to 2026-05-08
  USDJPY_D1.csv     5,020 bars  2010-04-22 to 2026-05-08

16 years of H1/H4/D1 vs ~6 weeks of H1 on Twelve Data free plan. Use --local
for all backtests; Twelve Data reserved for live signal feed.

### engine.py — optimised indicator weights
36-combo sweep on H4 2022-2026 found MACD and EMA trend are the dominant
signals; RSI and Bollinger are net drag on this trending pair.

Old → new weights:
  ema_trend:  1.2 → 3.6  (×3 — strongest directional signal)
  macd:       1.1 → 3.3  (×3 — ablating it is the single worst move: -42%)
  stochastic: 0.8 → 1.6  (×2 — clean timing, 40% WR at this level)
  adx:        1.0 → 0.5  (÷2 — partially redundant with EMA stack)
  ichimoku:   1.1 → 0.55 (÷2 — lags too much at H4, hurts in ranging)
  rsi:        0.3 → 0.0  (ablated — net drag across all test windows)
  bollinger:  0.9 → 0.0  (ablated — counter-trend noise on trending pair)

RSI and Bollinger are still computed and displayed in live output for
reference; weight=0 means they contribute nothing to the composite score.

Combined result vs baseline (H4 2022-2026, local data):
  Return:   +7.12% → +21.30%  (+14.2pp)
  Max DD:   -9.05% → -8.52%   (tighter — gains did not come at expense of risk)
  Sharpe:    0.23  →  0.47    (2x improvement)
  Sortino:   1.38  →  1.91
  Win Rate: 36.3%  → 38.1%

### backtest.py
Added 16-thread CPU parallelism for signal pre-computation via
ThreadPoolExecutor. _precompute_all_signals() dispatches each bar to a worker;
pandas/NumPy C extensions release the GIL enabling true parallel execution.
Results are collected and replayed in the main loop, replacing the per-bar
recompute. Also added --regime-tf CLI flag so regime filter timeframe can be
specified independently of the primary TF.

### signal_aggregator.py
EMA200 distance gate: confidence *= 0.55 when price is within 0.5% of EMA200
(chop zone) or beyond 1.5% (overextended). Sweet spot is 0.5-1.5% from EMA200
where trending momentum is cleanest. Strongest single filter found: lifted
cumulative return +13.88% to +35.50% in ablation window.

### ENGINE_LOGIC.md (new)
Full reference document covering: architecture, data pipeline, all indicator
logic, composite scoring formula, optimised weights with rationale, regime
classification, EMA200 gate, MTF alignment, grade system, backtesting results,
filter catalogue, and configuration reference.
2026-05-07 22:12:43 -04:00
Harwinder 5a392a578d Add EMA200 distance gate — strongest single signal-quality filter
Across 2024 trend / 2023 chop / 2026 hostile windows, distance from the
H1 EMA200 was empirically the most discriminative variable. Sweet spot
0.5-1.5% from EMA200; closer = false-breakout chop, farther = chase.

Outside the band, confidence is multiplied by 0.55 so the trade falls
below the B-grade threshold and is rejected.

Cumulative return across the 3 windows: +13.88% baseline -> +35.50%.
Win rate improved in every regime; 2023 chop turned positive (-2.56% to
+14.94%).

Also extends the trade journal with per-timeframe MTF scores, RSI at
entry, and EMA200 distance for future filter mining; surfaces ema_trend
in key_levels so the gate can read it.
2026-05-07 19:28:19 -04:00
Harwinder be42736223 Document session filter and auto-toggle flags in USAGE 2026-05-07 18:44:41 -04:00
Harwinder 2e5d66db1b Add session filter + auto-toggle, extend trade journal diagnostics
Session filter restricts trades to a UTC hour window (default 07-16,
London + NY) — opt-in via --session-filter, with --session-start /
--session-end overrides. Wired into both the backtester (cheap hour
check next to the regime gate) and live mode (downgrades signal to
NEUTRAL/F when current UTC hour is outside the window).

Auto-toggle (--auto-session-filter) tracks rolling mean pnl_pct over
the last N closed trades; when it falls below threshold, the session
filter activates, releasing once expectancy recovers. Default lookback
10 trades, threshold -0.3%.

Trade dataclass extended with diagnostic fields captured at entry:
entry_hour, entry_dow, ADX, ATR%, market_regime, volatility_regime,
mtf_agreement. These let --journal-csv exports surface loss-pattern
clusters when a window underperforms.

Validated on 2024 trend / 2023 chop / 2026 recent windows. Auto-toggle
preserves ~50% of trend edge while still cutting ~25% of recent-regime
losses — pure regime filter is still best on cumulative, so both
session flags ship default-off.
2026-05-07 18:38:49 -04:00
Harwinder fb1eb5b73f Swap raw requests for Twelve Data Python SDK
data_provider now uses TDClient.time_series().as_pandas() instead of
hand-rolled REST calls. Throttle, validation, and parquet caching are
unchanged. Adds twelvedata>=1.2.0 to requirements.txt.
2026-05-07 18:38:38 -04:00
Harwinder 28a00b7da0 Compact hierarchical live signal output
Restructure print_signal_report so decision (signal · grade · score ·
confidence · MTF) sits at the top, followed by trade setup, context line,
MTF breakdown, top supporting/opposing reasons in two columns, warnings,
and an optional detail block under --detail. Adds _compact_reason helper
that strips known indicator name prefixes via _INDICATOR_PREFIXES so
reason lines stay short.
2026-05-07 17:53:50 -04:00
Harwinder 3e078cfe7d Wire regime filter into live mode
--regime-filter previously affected only backtests. Live mode now also
checks higher-TF ADX after analysis: signals are downgraded to NEUTRAL
with a warning when ADX < threshold, and confirming when above.

This brings live behavior into parity with what the backtester measures.
2026-05-07 17:40:39 -04:00
Harwinder ce76684a62 Add indicator ablation framework + apply ablation-driven cleanup
Framework:
- TechnicalConfig.disabled_indicators skips named indicators in engine.analyze
- Optimizer supports disable_indicator override for parallel ablation sweeps
- main.py --ablate runs leave-one-out across all 10 directional indicators

Cleanup applied (validated across 2024 trend / 2023 chop / 2026 recent):
- Drop pivot by default (cumulative +9.5% to disable)
- Drop candlestick by default (+3.4% cumulative)
- Regime-gate ichimoku: 1.3x in trending, 0.2x in ranging (was a chop villain)
- Trim RSI weight 1.0 -> 0.3 (cumulative noise across regimes)

Result: +10.60% on 2024 trend (+33.24% -> +43.84%, Sharpe 2.17 -> 2.77),
+15.71% on 2023 chop (-18.27% -> -2.56%). Recent regime still adverse.
2026-05-07 17:33:07 -04:00
Harwinder 045f4248f3 Refocus on USD/JPY and add walk-forward backtester with regime filter
- Strip multi-pair plumbing; drop synthetic-data fallback
- Add BacktestConfig + walk-forward replay through the live signal pipeline
- Add metrics module (PF, Sharpe, Sortino, max DD, trade journal)
- Add parallel grid optimizer with disk parquet cache
- Add H4 ADX regime gate (--regime-filter, --regime-min-adx) — cuts chop
  losses ~84% in 2023 H1 while preserving trend edge
2026-05-07 17:15:41 -04:00
Harwinder c04f451c0c Remove redundant script.txt and tracked __pycache__
script.txt was superseded by requirements.txt. Python bytecode should
not be version-controlled — added .gitignore to keep cache and venv
out of the repo going forward.
2026-05-06 22:25:43 -04:00
Harwinder 0a374cfea7 Add requirements.txt and usage guide 2026-05-04 10:51:03 -04:00
53 changed files with 22757 additions and 1491 deletions
+370 -54
View File
@@ -20,8 +20,10 @@ you must follow before changing anything that affects signals. Section 10 lists
python -m venv venv && venv/bin/pip install -r requirements.txt # Python 3.12; a venv/ already exists on the dev box
cp config.json.example config.json # then fill in the keys (see section 5)
venv/bin/pip install cupy-cuda12x "nvidia-cuda-runtime-cu12==12.9.*" # optional, dev box: backtester GPU backend
# (CuPy from pip needs that header wheel to compile kernels)
venv/bin/python test_all.py # QA suite, no network; prints its own count
# (506 tests / 33 modules as of 2026-09-15)
# (632 tests / 40 modules as of 2026-10-01)
venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail # one signal report in the terminal
# --json full dump, --panel adds the advisory LLM panel,
# --timeframe M5 | M15 | H1 | H4
@@ -33,6 +35,9 @@ venv/bin/python dashboard_publisher.py --once # one scan +
venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --days 365
venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 \
--primary-tf 15min --spread 0.65 --workers 12 --export-csv book.csv
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 \
--primary-tf 15min --spread 0.65 --export-csv book.csv # research twin: same flags, every stage on
# the GPU, seconds per book, same MD5 (section 9)
```
Backtest flags worth knowing: `--spread` (round-trip cost in dollars; the default is 0.30, the
@@ -41,7 +46,15 @@ realistic Oanda figure is 0.65 and must be passed explicitly), `--quality A+,A,B
`--export-csv out.csv`, `--detail`, `--set PATH=JSON` (research override of any config value; an
unknown path is a hard exit), `--export-features` (appends the 65 decision-time columns; never use
it on a parity run because it changes the file), `--fixed-rr-tp N` (research only, not shipped),
`--record DIR [--record-bars]` (research recorder, see section 9).
`--set signals.grading_mode=min|product` (research only, 2026-09-27: grade on one conviction number
instead of the score + confidence floors; both FAILED rule 1 on M5 and M15, default `pair` is the
shipped rule and byte-identical), `--set xau_strategy.daily_trend_veto=off|both|buy|sell` (2026-09-30:
the M15 factory ships `both`; pass `off` to reproduce a pre-veto M15 book; `buy` / `sell` and every
mode on M5 / H1 FAILED their rule), `--set xau_strategy.wide_stop_cap_atr=K` (research only: FAILED
on 2026-09-30, ships 0 = off), `--record DIR [--record-bars]` (research recorder, see section 9), `--backend auto|gpu|cpu` (`auto`
uses the CuPy GPU backend when CUDA works and falls back to NumPy with the reason in the `Backend:`
line; `gpu` refuses to fall back; `cpu` forces the NumPy twins for a parity run; the env var
`FB2_BACKEND` sets the default). Both backends produce byte-identical books (section 9).
`test_all.py` is **not pytest**. It uses its own `@test("name")` decorator that runs each test the
moment the file is imported. Run the whole file; it prints per-module results and exits non-zero
@@ -65,6 +78,8 @@ same schedule. `main.py` runs once when you call it.
(`app_config.py`), then applies the validated per-timeframe overrides (section 8). Never build a
bare `PlatformConfig()` for a real scan; always go through the factory. The confirmation
timeframes come from the factory map: M5 → M15 + H1, M15 → M5 + H1, H1 → M15 + H4, H4 → H1 + D1.
Since 2026-09-30 an M15 scan also needs the daily (D1) frame, not as a confirmation but as the
input of the daily-trend veto (step 8); it is a required frame like the others.
**3. Fetch candles.** `signal_aggregator.analyze_pair` (the live entry point) first checks the
calendar: on the forex weekend or a blocked weekday it returns a "blocked" scan and does nothing
@@ -96,8 +111,9 @@ direction with the two confirmation timeframes. Agreement needs 2 of 3. Disagree
penalty and blocks the top grades.
**6. News context (warning only).** `news_engine.py` reads the Forex Factory calendar (MyFXBook
RSS as fallback) and four RSS feeds, scores headlines for gold, and can ask DeepSeek to improve
the scores. This produces warnings and a text summary. **By design it never changes a signal.**
RSS as fallback) and four RSS feeds and scores headlines for gold: the keyword scorer first, then
the local XAU transformer (`news_model.py`, mode `primary`) or DeepSeek (modes `off` / `fallback`)
overwrites those scores. This produces warnings and a text summary. **By design it never changes a signal.**
The backtester runs with news off, so the live decision must be purely technical too.
**7. Penalties, then the signal and the grade.** When the timeframes agree, confidence gains 10
@@ -125,7 +141,8 @@ against the blocked hours and allowed weekdays; the live wrapper and the backtes
list to the bar-close time as well, see section 8); transitioning-regime boost (a B/C/D can become A or B when a
trend is emerging); exhausted-trend cap (in a trending regime with two or more exhaustion tags, a C or D is pinned
to D and loses its setup); SELL quality gate
(shorts need a grade in `sell_min_quality`; H1's list is empty, so every H1 short is blocked);
(shorts need a grade in `sell_min_quality`; H1's list is empty, so every H1 short is blocked; M15's
list is `A+, B`, so every M15 grade-A short is blocked since 2026-09-26);
multi-timeframe opposition gate; SELL volatility gate (shorts are allowed only when volatility is high or extreme; low- or
normal-volatility shorts are blocked);
regime blocks (M15 blocks the mixed `trending` regime; M5 blocks shorts in `transitioning`);
@@ -134,10 +151,25 @@ M5 six extra noise gates (minimum score, required agreement, confirmation streng
cap, chop-transition volatility, SELL RSI floor). Every gate that removes a setup writes a warning
that names itself.
One more gate runs at the very end, after the setup generators and the late entry paths of step 9
(`_apply_daily_trend_veto`, step 12 in the code; M15 only since 2026-09-30): the **daily-trend
veto**. The daily trend is read from closed D1 bars with the provider's Sunday stub bars dropped
(`signal_aggregator.daily_trend_series`): +1 when the last daily close is above its 50-day EMA and
above the close 60 kept days earlier, −1 when below both, else 0. A finished setup that goes
against a non-zero state is removed (grade D, no setup, warning `DAILY TREND FILTER: BUY against a
falling daily trend — no trade`). It runs after every archetype on purpose, so a chart-pattern
setup is vetoed too. Where the veto is off (M5, H1, H4) the live wrapper still stamps
`report.daily_trend` for the alert text and the signal log; nothing reads it back there.
**9. Trade setup.** Grades A+ and A get the gold ATR setup (`_generate_xau_atr_trade_setup`): stop
2 × ATR; target 1 at 2 × ATR for A+ and 1.5 × ATR for A (1 × ATR for an A carrying two or more
exhaustion tags); target 2 at 1.5 × the target-1 distance. Other tradeable grades get the generic
swing-based setup, which floors the stop at $8 (80 pips). Both are checked against the spread gate:
exhaustion tags); target 2 at 1.5 × the target-1 distance. Other tradeable grades get the generic swing-based setup,
which floors the stop at $8 (80 pips). Since 2026-09-26 **every generator prints target 1 at no less
than 1R** (`signals.tp_print_floor_rr`, effective floor `max(tp_floor_rr, tp_print_floor_rr)`),
applied **after** the min-RR and spread gates have judged the generator's own target, so which
generator serves a signal, the stop and the books are unchanged; only the exported `tp1` /
`rr_planned` columns move (a floor applied before the gate cost −24.6R on the M15 two-year book,
see the 2026-09-26 report). The planned ratio is a guide: the tested exit is the trailing stop. Both are checked against the spread gate:
target 1 must be at least 30 × the round-trip cost ($9 at the $0.30 cost setting). Two extra entry paths run after the
gates: the M5 pullback (`XAU_PB`, shipped off) and the chart-pattern entry (`XAU_CP`,
`chart_patterns.py`: M5 only, buy-side patterns inverse head and shoulders, double bottom, bull
@@ -155,37 +187,87 @@ later generator shipped a setup.
**10. Delivery.** The `SignalReport` goes to the surface that asked for it:
- *Telegram* (`telegram_bot.py`): an alert is sent only when the signal is not NEUTRAL, the grade
reaches `alert_min_quality`, **and** a setup exists. A fingerprint (signal + direction) stops
repeats; a grade upgrade re-alerts. The fingerprint is committed only after a successful send,
reaches `alert_min_quality`, **and** a setup exists. Since 2026-09-26 no second alert goes out on a
timeframe within 5 bars of the last alerted setup's bar (`_ALERT_COOLDOWN_BARS`, the backtester's
entry cooldown; consecutive-bar re-entries were −25.1R of the −37.8R live book of 2026-09-04 →
09-25). Since 2026-10-01 the five bars are COUNTED in the scan's own primary frame
(`_primary_bars_since`, the backtester's position difference; clock time is only the fallback
for a report without frames) and the last alerted bar per timeframe is read back from
`signals.db` at start-up (`signal_log.last_alerted_bars` → `_restore_alert_cooldowns`), so
neither the settlement pause, the weekend nor a restart lets live alert where the books do not
enter. After that, a fingerprint (signal + direction) stops repeats; a grade upgrade re-alerts.
Since 2026-09-27 the header leads with the grade (`Grade A · BUY · XAU/USD · M15`) and the raw
score / confidence line is replaced by words from `main.plain_words` (`Direction: strong buy ·
Reliability: good · timeframes agree`; the cut-offs are the grade table's own, so the words never
disagree with the grade; the numbers stay in `--json`, `--detail`, the dashboard meters and the
signal log). The `/status` summary and the dashboard card carry the same words (`plain_words` in
the snapshot).
The setup block prints the trailing rule first as the tested exit, TP1/TP2 as guides with their R,
the "Planned TP1 RR", and one line of validated-book statistics for the setup class
(`_book_rate_line`, from `agent_panel_base_rates.json`). The fingerprint is committed only after a successful send,
so a failed send stays eligible next scan. Every send is logged with the signal-log row id. The
level-breach monitor reuses the bar's cached M15 report (`_cached_or_scan`) instead of rescanning.
- *CLI* (`main.py`): prints the report; `--json` dumps everything.
Since 2026-09-30 three more display lines: the daily trend in words under the regime line
(`🧭 Daily trend: rising — this buy goes with it`, `main.daily_trend_words`), and inside the setup
block the lot size for the owner's risk limit (`📏`, off until `trade_size.risk_per_trade` is set)
and the count of alerted same-direction plans still open (`🧱`). The size line is private:
`format_signal_message(report, channel=True)` builds the copy without it unless
`trade_size.telegram_channel` is true, and since 2026-10-01 `_broadcast` sends the owner copy to
the owner's own chat only (`_is_owner_chat`): the channel and any other target (a group the owner
used `/autoscan` or a scan button in; the owner lock is per user) get the copy without it.
Both alert paths (candle scan, level break) end in one `_deliver_alert`: the panel, then on the
one-worker `alert-db` pool the exposure lines are refreshed and the row is stamped `alerted_at`
(`signal_log.mark_alerted`), then the send. Stamping BEFORE the send is what lets two alerts at
the same candle close count each other; an alert that reaches no chat clears its own stamp
(`unmark_alerted`) and commits nothing. No database write runs on the event loop.
- *CLI* (`main.py`): prints the report; `--json` dumps everything (including `daily_trend`,
`daily_trend_words`, `size_line`, `open_line`).
- *Dashboard* (`dashboard_publisher.py`): writes `snapshot.json`, `news.json` and the static page
into the Apache folder.
into the Apache folder. Each card shows the daily-trend words; a plan shows the already-open
line, and the size line only when `trade_size.dashboard` is true (the page is public).
Two advisory add-ons are attached **after** the scan by the surface, never inside the signal
code: the LLM agent panel (`agent_panel.py`, four DeepSeek personas, verdict AGREE / NEUTRAL /
OPPOSE) and the violent-move watch (`violent_watch.py`, a "the next hour may be wild" line). No
gate reads either of them.
Three display add-ons are attached **after** the scan by the surface, never inside the signal
code. The first is `exposure.attach_exposure` (2026-09-30): the lot-size line and the
already-open line, a leaf module that imports only `app_config` and `signal_log`. The other two are
advisory: the LLM agent panel (`agent_panel.py`; since 2026-09-17 only the **macro** persona runs by
default, `agent_panel.personas: ["macro"]`, one DeepSeek call per panel — the technical, risk and
contrarian personas carried no information in two live evaluations and stay available for research;
the persona returns a probability that the trade closes profitably, anchored to the setup class's
base rate from the validated books, shown as the "AI estimate"; with several personas configured
the estimate is their equal-weight mean, `verdict_source: "mean"`; the older `macro` stance and
`chair` synthesizer paths remain for comparison) and the violent-move
watch (`violent_watch.py`, a "the next hour may be wild" line). No gate reads either of them. The
live wrapper `analyze_pair` hands the fetched frames to the add-ons through a private
`report._frames` attribute so the panel's price context sees exactly the bars the engine saw; the
replayable core never sets it.
**11. Logging.** `signal_log.py` writes one row per live scan into `logs/signals.db` (signal,
**11. Logging.** `signal_log.py` writes one row per live scan (from the bot and the CLI; the
dashboard process never writes either database, section 3) into `logs/signals.db` (signal,
grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the
gate that removed a setup, a gate-config fingerprint and an engine fingerprint). The engine
gate that removed a setup, a gate-config fingerprint and an engine fingerprint; since 2026-09-30
also `daily_trend` on every real scan and `alerted_at` on rows whose alert reached a chat). The engine
fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log
comparisons across such a deploy must be split at the deploy time (the 2026-09-09 M15 RSI change is
one). `news_archive.py` writes every headline and
one). The gate fingerprint changes when a gate ships: the M15 daily-trend veto moved M15 rows from
`f9b2f801de54` to `8a8075688d67` (engine fingerprint unchanged), so M15 comparisons split there too.
`news_archive.py` writes every headline and
its score into `logs/newsdata.db`. Both are fail-safe: a logging error can never break a scan.
**12. Measurement, later.** Two models read the stored candles and score what happened next:
advisory outcomes (`advisory_outcome.py`: replays each logged setup with next-open entry, stop
first, close-only trailing; a cleaned-mid evidence model, not execution P&L) and scan outcomes
(model-free forward moves 1 h / 4 h / 24 h after every scan). `analyze_signal_log.py` turns the
database into reports.
database into reports. Since 2026-10-01 the outcome model checks candle continuity bar by bar as
it walks: a hole in the stored candles AFTER a trade's exit no longer voids the trade, and a hole
reached while it is open ends the walk with the progress kept (so the locked-profit floor is the
same whether the stream arrives in one pass or many).
**The backtester is the same pipeline on history.** `backtest/gpu_backtest.py` loads candles from
`backtest/cache/*.parquet` (fetched once through the same provider code), precomputes every
indicator series over the whole window, and calls `analyze_from_dataframes` bar by bar with news
off. Trades are resolved on the raw price path with next-open entries, spread charged per trade,
off. When the daily-trend veto is on it also loads the D1 frame (1,400-day pad) and slices it as-of
like every other frame, so the veto sees exactly the closed daily bars a live scan would. Trades
are resolved on the raw price path with next-open entries, spread charged per trade,
stop-first ties, gap fills at the open, and the trailing stop moved at bar close only. Time filters
use bar-close time in both paths, so live and backtest agree. The replay fans out over worker
processes; the 5-bar entry cooldown, the only cross-bar state, is applied in an ordered pass
@@ -201,10 +283,11 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| File | Role | Uses | Used by |
|---|---|---|---|
| `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`) | pair_signals, signal_aggregator, agent_panel, violent_watch, app_config | dashboard_publisher (formatting helpers) |
| `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap | pair_signals, providers, main, agent_panel, violent_watch, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` |
| `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`) | | copied by the publisher |
| `main.py` | Command-line signal report (`--detail`, `--json`, `--panel`); the display helpers every surface shares (`plain_words`, `daily_trend_words`) | pair_signals, signal_aggregator, agent_panel, violent_watch, exposure, app_config | dashboard_publisher and telegram_bot (formatting helpers) |
| `telegram_bot.py` | Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, `/scan /status /news /autoscan /cache /clearcache /help /start` | pair_signals, signal_aggregator, providers, agent_panel, violent_watch, exposure, news_engine, llm_client, signal_log, app_config | nobody (a top-level process) |
| `exposure.py` | Display-only lot-size line and already-open line (`attach_exposure`, 2026-09-30). `lot_size` floors to the lot step in Decimal so the risk limit is never exceeded; `size_shared` answers whether config.json shares the size with a public audience. A leaf: never raises, never read by the core | app_config, signal_log | main, telegram_bot, dashboard_publisher |
| `dashboard_publisher.py` | Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes `signals.db` or `newsdata.db`: `main()` calls `app_config.set_db_writes_for_process(False)`, a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running). It does READ `signals.db` (`mode=ro`) for the already-open line since 2026-09-30 | pair_signals, providers, main, agent_panel, violent_watch, exposure, news_engine, app_config | nobody. Hard rule: never imports and is never imported by `telegram_bot` (test-enforced) |
| `static/dashboard.html`, `.css`, `.js` | The public dashboard page (reads `snapshot.json` / `news.json`). Bootstrap 5 + jQuery + Chart.js + Font Awesome straight from a CDN, no build step; M5/M15/H1/H4 each get a card in one column grid, with a diverging Chart.js bar chart for direction and one per card for the indicator votes | | copied by the publisher |
| `static/news_summary.php` | The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against `news.json`, own fail-closed daily cap, key read from `/etc/gold-dashboard/deepseek.key` (never repo config) | | installed by the publisher |
| `docs/user_guide.html`, `docs/beginner_guide.html` | Human guides. Keep them in sync with signal behaviour. There are no PDF builds | | |
@@ -216,7 +299,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| `pair_signals/xauusd.py` | The config factory: overlay + validated per-timeframe overrides + confirmation map. Its docstring is the evidence log for every override | config, app_config, engine | main, telegram_bot, dashboard_publisher, backtester |
| `pair_signals/__init__.py` | Registry of pair producers (`PAIR_PRODUCERS`) and `produce_signal` | config, signal_aggregator, providers, xauusd | the surfaces, backtester |
| `engine.py` | Indicators, regime, composite score (`TechnicalEngine`) | config | signal_aggregator, backtester, replay_record, pair factory |
| `signal_aggregator.py` | Multi-timeframe merge, penalties, grade, gate chain, setup generators, reasons; `analyze_pair` (live) and `analyze_from_dataframes` (replayable core) | engine, chart_patterns, news_engine, providers, signal_log, config | main, telegram_bot, backtester, pair_signals, analyze_signal_log (agent_panel only for type hints) |
| `signal_aggregator.py` | Multi-timeframe merge, penalties, grade, gate chain, setup generators, reasons, the daily-trend veto and its one definition (`daily_trend_series` / `daily_trend` / `daily_trend_needed`, shared by live, the replay and the twin's host side); `analyze_pair` (live) and `analyze_from_dataframes` (replayable core) | engine, chart_patterns, news_engine, providers, signal_log, config | main, telegram_bot, backtester, pair_signals, analyze_signal_log (agent_panel only for type hints) |
| `chart_patterns.py` | The XAU_CP pattern detector | | signal_aggregator |
| `position_sizing.py` | Decision-time feature builder (65 columns) and the reserved stake multiplier (always 1.0). Bounded lookbacks keep live == backtest | config | backtester (`--export-features`) |
@@ -234,27 +317,34 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| File | Role |
|---|---|
| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; optional DeepSeek enrichment; display and warnings only |
| `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback |
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook |
| `agent_panel.py` | Advisory persona panel (macro / technical / risk / contrarian + optional chair). Attached post-hoc; the alert verdict is the macro persona by default. With `verdict_source: "chair"`, `_apply_oppose_guard` downgrades a chair OPPOSE to NEUTRAL unless the macro persona opposes, or technical opposes while macro is not supportive. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; headline scores upgraded by the local XAU model (`news_model.py`, mode `primary` live since 2026-09-27) or by DeepSeek (modes `off` / `fallback`, `_enrich_headline_scores`); DeepSeek keeps event direction and the summary; display and warnings only |
| `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback. `chat_json` never raises and never logs the API key (2026-10-01: a key with a line break made `requests` quote the `Authorization` value in its error text); an unserialisable payload is refused before the cap is counted |
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call. **Frozen in `news_model.mode` `primary` (live since 2026-09-27)**: the local model's output is not training data, so nothing is archived except the optional DeepSeek audit copies (`deepseek_audit_per_day`, shipped 0); `off` / `fallback` archive as before, a `fallback` model row carrying `teacher_model` `xau_transformer@<trained_at>` |
| `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. v2.2 (2026-10-01) changed the persona's INPUT, not the prompt text: the event timing reads the cached calendar week `attach_panel` hands to `_price_context` (`NewsEngine.peek_week_events`, which never fetches; a hold past UTC midnight sees tomorrow's release), and the ten headlines are the newest ten, not the first ten in feed order. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
| `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |
| `news_model.py` | The local headline scorer (2026-09-27): vendored CPU runtime of the owner's ML-NEWS XAU transformer (DeBERTa-v3-small, two heads: moves gold? / up or down?, ONNX; `onnxruntime` + `tokenizers` imported lazily inside `load_model`, so the bot starts without them) plus a load-once, never-raising singleton (`get_news_model`, `teacher_tag`, `warm_up_in_background`). Gold head only: `news_engine._apply_model_sentiment` writes `xau_score` and XAU's per-currency entry, never USD. Bundle (280 MB) lives outside the tree at `news_model.path`. Spec and deviations: `docs/news_model_integration_spec_2026-09-27.md`; ops: `deploy/README-news-model.md` |
### Logging and evaluation
| File | Role |
|---|---|
| `signal_log.py` | SQLite logger for every live scan (`signals`, `news_snapshots`, `scan_outcomes`; view `signals_with_news`); opportunistic scan-outcome labelling; advisory outcome seeding |
| `advisory_outcome.py` | The deterministic outcome model for logged setups |
| `signal_log.py` | SQLite logger for every live scan (`signals`, `news_snapshots`, `scan_outcomes`; view `signals_with_news`); opportunistic scan-outcome labelling; advisory outcome seeding; `mark_alerted` (stamps `alerted_at` just before the alert is sent and returns the stamp it wrote), `unmark_alerted` (clears that stamp when no chat received the alert), the read-only `open_alerted_setups` (alerted plans whose outcome is still `PENDING`, one per event, stale rows dropped) and the read-only `last_alerted_bars` (newest alerted bar per timeframe, for the bot's cooldown after a restart) |
| `advisory_outcome.py` | The deterministic outcome model for logged setups. Continuity of the stored candles is judged one step at a time inside the walk (`_next_bar_follows`); `_has_continuity` stays all-or-nothing for the scan-outcome labeller |
| `analyze_signal_log.py` | Read-only reports on `signals.db`: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (`--stored-candles`, `--news`, `--parity`, `--scan-outcomes`) |
| `scripts/backfill_signal_outcomes.py` | Batch seeding / advancing of advisory outcomes on an imported database (`--dry-run`, `--reseed-unresolvable`) |
| `scripts/label_scan_outcomes.py` | Batch labelling of scan outcomes for a backlog (`--dry-run`) |
| `scripts/panel_replay.py` | Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (the exact logged one from `panel_payloads_json` per persona when the row has it, macro alone on live rows since 2026-09-17; else from the stored JSON blobs, interned news and the kept Oanda cache; the CSV column `stored_payloads` names the personas that replayed a logged payload), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. `--dry-run`, `--max-calls`, `--limit`, `--out`. Read-only on the DB; 4 DeepSeek calls per event |
| `scripts/build_panel_base_rates.py` | Builds `agent_panel_base_rates.json` (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin |
| `agent_panel_base_rates.json` | The base-rate table the panel anchors its probabilities to (rebuilt 2026-10-01 from the reference books of section 9, the M15 one with the daily-trend veto; `generated_at` and each book's MD5 are in the file) |
### Backtesting and research
| File | Role |
|---|---|
| `backtest/gpu_backtest.py` | The backtester (CuPy on a GPU, NumPy otherwise; check the `Backend:` log line) |
| `backtest/gpu_backtest.py` | The backtester. The indicator precompute over the full history and the resolution of every trade run on the GPU when CUDA works (`--backend`, `Backend:` log line names the device or the fallback reason); the per-bar replay through `analyze_from_dataframes` is the live Python signal code in forked CPU workers by design and is 80–95% of wall time |
| `backtest/gpu_kernels.py` | CuPy backend state (lazy probe, never silent, sets `CUDA_PATH` to the pip header wheel) and the three CUDA kernels: EWM batch, pandas rolling mean/var replica, trade resolver. Each is a bit-for-bit twin of the NumPy/pandas code kept in `gpu_backtest.py` (`--fmad=false`, one thread per series or trade); the twins are pinned by Module 36 and by the reference MD5s |
| `backtest/cudaBasedBacktesting.py` | Research-only full-GPU twin of the backtester (2026-09-18): same CLI, same cache files, same 63-column book byte for byte, but the per-bar decision runs in CUDA kernels instead of the live Python code, so a book takes seconds (M5 two-year 2.4 s against 115 s). Reuses the loader, the precompute, the resolver, the metrics and the writers from `gpu_backtest.py`; the host only turns the kernels' numeric codes into the exact strings. No CPU path: `--backend cpu` is refused. Its numbers count as evidence only while the MD5 parity with `gpu_backtest.py` holds (section 7) |
| `backtest/cuda_signal_kernels.py` | The CUDA decision stage: K1 (`engine.py` plus the `GPUTechnicalEngine` overrides, one thread per bar), K2 (`analyze_from_dataframes` end to end with the gates, the setup generators, XAU_PB, XAU_CP, the regime filter and the replay's per-bar rules, one thread per primary bar), K3 (`position_sizing.build_features`), K4 (cooldown), K5 (prefix median), K6 (recorder bar rows); the config packer and the name lists the `#define`s are generated from. A leaf module that never imports the live code; pinned by Module 37 |
| `backtest/replay_record.py` | Opt-in `--record` recorder: why every accepted trade was taken, plus calendar events. Book-neutral by contract |
| `backtest/replay_scorecard.py` | Read-only reader of recorded runs (`DIR --check` verifies its invariants) |
| `scripts/ff_calendar_scraper.py` | Builds `backtest/cache/calendar.parquet` (Forex Factory events) for the recorder |
@@ -269,7 +359,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
| `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
| `config.json.example` | The schema of `config.json` (which is gitignored) |
| `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
| `test_all.py` | The QA suite (33 modules) |
| `test_all.py` | The QA suite (40 modules; Module 36 is the GPU backend parity module, Module 37 the CUDA decision-stage parity module, Module 38 the local news model, Module 39 the daily-trend veto and the wide-stop cap, Module 40 the trade-size / already-open / daily-trend display lines) |
---
@@ -306,7 +396,20 @@ symbol and interval maps. A test enforces that subclasses do not override the sh
scheduled closure since 2026-09-13. A "stale data" warning at any other weekday hour is a real
feed stall.
- The backtester fetches history through `fetch_range` (paginated, retries a failed page, logs an
error for interior gaps longer than 3 bars) and stores it under `backtest/cache/`.
error for interior gaps longer than 3 bars) and stores it under `backtest/cache/`. A DATE window
with no bars in it (a closed day at the head of a padded window) is an empty page, not a failure:
until 2026-09-30 the Oanda provider raised on it and `fetch_range` aborted after three tries, so
a D1 pad starting on a Saturday could not be fetched at all. Only a closure-sized window gets
that reading (`EMPTY_WINDOW_MAX`, 5 days: a weekend joined to a holiday, and no malformed candle
skipped); an empty answer for a longer window (a normal 15-day M5 page) raises, so `fetch_range`
retries it and refuses the history after three tries instead of caching a hole (2026-10-01).
A `count` request (the live path) with nothing in it still raises.
- Daily (D1) candles are UTC days (`dailyAlignment=0`). Oanda also emits a 1–2 hour stub bar
labelled Sunday for the reopening (about 600 of them since 2015); it is a real bar for the H4
confirmation engine, and the daily-trend definition drops Sunday labels before it computes
anything. Since 2026-09-30 D1 is a required frame of every M15 scan (live and backtest) and is
fetched for display on M5 and H1 (cached until the next daily close; a failure there leaves the
line empty and never fails the scan).
---
@@ -328,10 +431,12 @@ gates live in `config.py` and the pair factory, never in `config.json`. Blocks (
| `scanning` | `pairs`, `timeframes` |
| `pairs."XAU/USD".blocked_hours_utc` | The base list of blocked hours (default 0, 1, 2, 10, 11, 13, 16, 21) |
| `deepseek` | `api_key`, `model`, `enabled`, `timeout_seconds`, `daily_request_cap` (env `DEEPSEEK_API_KEY`) |
| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`macro` or `chair`). All defaults = feature off |
| `agent_panel` | `enabled`, `telegram`, `dashboard`, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, candle payload knobs, `verdict_source` (`mean`, the default since 2026-09-17; `macro` or `chair` for comparison), `personas` (default `["macro"]`; any of macro / technical / risk / contrarian; `chair` needs two or more). All defaults = feature off |
| `violent_watch` | `enabled`, `alert_line`, `headsup`, `max_headsup_per_day`, `headsup_min_level` (env `VIOLENT_WATCH_ENABLED`). Off by default |
| `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
| `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
| `news_model` | `path` (bundle directory, required when mode ≠ off), `threads` (default 4), `mode` (`off` default = DeepSeek/keyword path; `fallback` = model only where DeepSeek failed; `primary` = model scores every headline, no DeepSeek headline calls), `deepseek_audit_per_day` (default 0; primary only, archive-only DeepSeek labels). Env `NEWS_MODEL_MODE` overrides `mode` per process (the dashboard unit). Live: `primary` on both processes |
| `trade_size` | Lot-size guidance on the alert (display only, 2026-09-30): `risk_per_trade` (US dollars; 0 or absent = the size line is off), `contract_size` (ounces per lot, default 100), `lot_step` (default 0.01), `telegram_channel` and `dashboard` (default false: the size shows the owner's risk, so every Telegram chat other than the owner's own (`telegram.chat_id`) — the channel, a group — and the public page get it only when these are true). No env override |
| `log_level` | Logging level |
A legacy top-level `market_outlook` block is tolerated and ignored with one warning (the feature
@@ -347,7 +452,23 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
## 6. Testing
- `venv/bin/python test_all.py` runs everything with mocked network. Expect
`RESULTS: 506 passed, 0 failed, 506 total` (2026-09-15).
`RESULTS: 632 passed, 0 failed, 632 total` (2026-10-01). Module 38's last test loads the
real news-model bundle when `NEWS_MODEL_DIR` (or the configured `news_model.path`) exists and
prints SKIP otherwise. On a box with a visible CUDA device
Module 36 fails if the GPU backend falls back (the reason is in the message); with
`FB2_BACKEND=cpu` or no CUDA the GPU-path tests print a notice and the NumPy twins are tested.
Module 37 (the CUDA decision stage) adds about 85 s on the GPU because its references replay
two synthetic fixtures through the live signal code bar by bar; without CUDA each of its device
tests prints a notice and only the arithmetic rule, packing, flag and source-guard tests run. The suite's hermetic app config points
both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote
fixture headlines into the real `logs/newsdata.db`, and a Module 32 test now pins the isolation. One Twelve Data interval-map test
reaches the network and can time out on a slow connection; a rerun clears it.
- The suite is meant to run on the live box too (Python 3.9, SQLite 3.34, no `pyarrow`, no CUDA):
the four tests that must write a Parquet fixture print a notice and return when no Parquet
engine is installed, and a test that needs an older database builds it forwards
(`_legacy_signals_db39`: the base schema plus every additive column but the one under test),
never with `ALTER TABLE … DROP COLUMN` (SQLite 3.35). Verified 2026-10-01 with both Parquet
engines blocked and `FB2_BACKEND=cpu`: 0 failures.
- Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
- When a test needs the weekend rule, patch `providers.base._is_forex_weekend` and remember the
other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar.
@@ -368,7 +489,13 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
fills. Marginal edges must also pass the 2024-06-01 → 2025-03-30 out-of-sample window. Backtest
windows go back at most 3 years (owner preference, 2026-09-05). `round_trip_cost` stays at 0.30
because it also drives the live target floor (0.65 would move target 1's floor from about $9 to
$19.50); changing it is its own rule-1 event.
$19.50); changing it is its own rule-1 event. A change whose purpose is to cut the losing runs
rather than add R may instead be judged by a **risk-first rule** (owner decision 2026-09-30),
written before the runs: 3-year R at least 97% of the control, profit factor up by 0.10 or more,
max drawdown not deeper, the 2019 → 2026 window better on R, on drawdown and in 2021 + 2022, an
unseen holdout, both costs, and at least 3 of 4 neighbouring definitions holding. Under it the
halves of a purely rising window are reported, not clauses. The owner chooses the rule per
campaign; the default stays the acceptance above.
2. **Per-timeframe overrides live in the pair factory**, never in the global defaults. The same
knob has had opposite effects on different timeframes several times. Record the evidence in the
factory docstring.
@@ -391,8 +518,15 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
Process rules that go with them: every backtest report follows
`backtest/instructions_for_backtest.txt`; every candidate is compared with a control run on the
same code and cache, never with an old book directly; the `Backend:` line and the "GPU cache
fallbacks: 0" line are part of the evidence; keep `docs/` in sync when behaviour changes.
same code and cache, never with an old book directly; the `Backend:` line (since 2026-09-18 it
names the CUDA device or the fallback reason) and the "GPU cache fallbacks: 0" line are part of
the evidence; keep `docs/` in sync when behaviour changes. Any change to `engine.py`,
`signal_aggregator.py`, `chart_patterns.py`, `position_sizing.py`, the recorder's row layout or
the `GPUTechnicalEngine` overrides is also a **parity event for the CUDA research twin**
(`backtest/cudaBasedBacktesting.py`): mirror it in `backtest/cuda_signal_kernels.py` in the same
commit, keep Module 37 green, and regenerate at least the M15 OOS book with both scripts and show
equal MD5s before a number from the twin is used as evidence. Until then its books are not
evidence.
---
@@ -404,15 +538,22 @@ overrides (each one has its validation record in the `pair_signals/xauusd.py` do
| Timeframe | Confirmations | Overrides on top of the defaults |
|---|---|---|
| M5 | M15, H1 | hour 9 UTC blocked and hour 21 unblocked; chart-pattern entries on (buy-side only: inverse head and shoulders, double bottom, bull flag); pullback entries off; partial EMA alignment scores neutral; trailing stop arms at 0.60R and trails 0.85R; shorts blocked in the `transitioning` regime |
| M15 | M5, H1 | target floor 1R; the mixed `trending` regime blocked; shorts blocked at 15:00 and 17:00 UTC; trailing stop arms at 0.70R and trails 0.75R; RSI weight halved in every regime |
| M15 | M5, H1 (+ D1 for the veto) | target floor 1R; the mixed `trending` regime blocked; shorts blocked at 15:00 and 17:00 UTC; trailing stop arms at 0.70R and trails 0.75R; RSI weight halved in every regime; grade-A shorts blocked (`sell_min_quality` `A+, B`); daily-trend veto in both directions (`daily_trend_veto` `both`, since 2026-09-30) |
| H1 | M15, H4 | every short blocked (`sell_min_quality` empty) |
| H4 | H1, D1 | none. H4 remains "unproven" (few trades) |
Shared defaults worth knowing: blocked hours 0, 1, 2, 10, 11, 13, 16, 21 UTC; trading days Monday
to Friday plus Sunday evening; shorts need grade A+, A or B and high-or-extreme volatility;
trailing stop arms at 0.6R and trails 0.75R; target floor 0.5R; round-trip cost $0.30 with
trailing stop arms at 0.6R and trails 0.75R; gated target floor 0.5R and printed target floor 1R;
round-trip cost $0.30 with
target 1 at least 30 × cost; the generic and chart-pattern setups floor the stop at $8; the gold
ATR setup uses a 1 × ATR target before the grade scale (2× at A+, 1.5× at A) and a 2 × ATR stop; the H4 regime filter is off; news never gates.
ATR setup uses a 1 × ATR target before the grade scale (2× at A+, 1.5× at A) and a 2 × ATR stop; the H4 regime filter is off; news never gates;
the daily-trend veto is off (M15 alone turns it on) and the wide-stop cap is off everywhere
(`wide_stop_cap_atr` 0.0, a research knob that failed its rule).
The daily-trend veto failed the same rule on M5 (it deepens the 3-year losing run from 10.1R to
15.2R and loses the 2019 → 2026 window) and on H1 (−9.7R of +70.0R). Do not turn it on there
without a new campaign; the daily trend is printed on those alerts as context only.
One semantic to know: `blocked_hours_utc` is applied to the bar **label** hour inside the core
and to the bar **close** hour in the live wrapper and the backtester. So H1 can only trade bars
@@ -431,27 +572,75 @@ trade-level diff is needed; each run takes seconds to a few minutes.
Harness for every reference book: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD
--spread 0.65 --workers 12 --primary-tf <tf> --start <start> --end <end> --export-csv <out>`,
factory config, no `--set`, `--quality`, `--strategies`, `--confirm-tfs` or `--export-features`.
All of these run offline from the kept cache windows (NumPy CPU backend, 0 cache fallbacks).
All of these run offline from the kept cache windows (0 cache fallbacks). Since 2026-09-18 the
CuPy GPU backend and `--backend cpu` both reproduce every MD5 below; earlier reports say
`NumPy CPU fallback` because CuPy could not compile kernels on the dev box until then.
| Book | Window | Trades / R / PF | MD5 of the CSV |
|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `29120a1ee32979431d062f210d41a9d1` |
| M15 W2 | 2024-09-05 → 2026-09-05 | 931 / +146.14 / 1.43 | `d185f099482a07f29502cb5a371a9bbf` |
| M15 OOS | 2024-06-01 → 2025-03-30 | 355 / +60.67 / 1.49 | `6f8f24b49597ced28f50a19e08c3472f` |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `3b45d12bade823178b267c13bc698458` |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `e75adae13c839b15736f69c0114e6aa5` |
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 | `e599c8a8652f95b7fcdf5a7ef2b564f5` |
| M15 W2 | 2024-09-05 → 2026-09-05 | 597 / +155.89 / 1.78 | `dfc647ee02afd179dd9e031f344ed380` |
| M15 OOS | 2024-06-01 → 2025-03-30 | 258 / +63.78 / 1.80 | `e87b6de12f90f3b5a5d41a953ee9ff7a` |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.0 / 1.98 | `a44fd19b4067f46e5bd98d50cc3eff92` |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `f3587bae3c5dc3eda468ea3588b17dcb` |
Other current-HEAD figures at $0.65: M5 W3 (3-year) 870 / +153.6R / PF 1.50; M15 W3 1273 /
+186.6R / PF 1.39; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54;
H1 12-month (2025-08-04 → 2026-08-04) 91 / +22.2R / PF 1.79. All five MD5s above were
reproduced on 2026-09-15 after the cache prune.
Re-pinned 2026-09-30. M15 only: the daily-trend veto (M15 override 7). W2 216 trades removed,
+1.7R, max drawdown 15.6R → 11.4R; OOS 55 removed, −4.7R (the accepted cost of the risk-first rule
in a rising window), drawdown 12.4R → 9.1R; previous pins `5b65ecf6…` (813 / +154.15 / 1.53) and
`09b8a8e7…` (313 / +68.50 / 1.67). M15 W3 with the veto 824 / +205.12 / 1.73, max drawdown 12.4R,
`ad3b0dbda0bf77b659caea1c53be6a50` (was `70195fb0…`, 1105 / +195.79 / 1.49, drawdown 20.9R). The
M5, H1 and H4 pins are unchanged. An M15 book needs the D1 frame of its window (1,400-day pad) in
the cache; `--set xau_strategy.daily_trend_veto=off` reproduces the previous M15 pins byte for byte.
Re-pinned 2026-09-26. M15: the grade-A SELL block (M15 override 6, 162 trades removed, +8.0R);
previous pins `b4aa07e3…` (931 / +146.14 / 1.43) and `9fd624fb…` (355 / +60.67 / 1.49); M15 W3
(3-year) with the block 1105 / +195.79 / 1.49, `70195fb0e6865b3a46ee8a9aa8844e51`. All four
timeframes: the 1R printed-target floor, **label-only** (M5 W2: 544 rows change `tp1` / `rr_planned`,
H1 W3 252, H4 W3 108, M15 W2 80; trades and R identical to `49f8e602…`, `7b848c91…`, `186e677f…`
and to the M15 block-only book `497520ef…`; every `rr_planned` is now ≥ 1.0).
Re-pinned 2026-09-17 for a **label-only** change: the exported `strategy` column now carries the
generator's own stamp (`XAU_ATR` / `XAU_PB` / `XAU_CP` / `GENERIC`) instead of a tag inferred
from the first reason's text, which had labelled every gold ATR setup `GENERIC`. All 62 other
columns of all five books are byte-identical to the previous pins (`29120a1e…`, `d185f099…`,
`6f8f24b4…`, `3b45d12b…`, `e75adae1…`); trades, R and PF are unchanged. `--strategies` accepts the
live names (`XAU` still means `XAU_ATR`).
Other current figures at $0.65 (2026-10-01): M5 W3 (3-year) 870 / +153.64R / PF 1.50 / max
drawdown 10.1R (`d20ec8a3…`); M15 W3 824 / +205.12R / PF 1.73 / 12.4R; M5 12-month (2025-08-14 →
2026-08-14) 560 / +108.81R / PF 1.54; M15 12-month (2025-08-04 → 2026-08-04) 266 / +92.3R /
PF 2.12 / 8.3R; H1 12-month (2025-08-04 → 2026-08-04) 91 / +22.2R / PF 1.79. The long robustness
window 2019-01-01 → 2026-08-07 (allowed as supporting evidence, owner decision 2026-09-30): M15
1473 / +223.33R / PF 1.42 / max drawdown 41.5R (`aa5253aa…`; by year +14.0, +45.1, −27.3, +3.2,
+1.8, +53.5, +92.5, +40.6), M5 1052 / +128.10R / PF 1.34 / 18.4R (`3f0c091c…`; −1.0, −2.8, −1.7,
−2.2, −2.8, +1.0, +40.5, +97.0: the M5 rules take few trades and earn nothing before 2025). The
unseen holdout 2026-09-05 → 2026-10-01: M15 18 / −4.62R, M5 33 / −2.45R, H1 3 / −3.04R. The pins of
the table above were reproduced on 2026-09-15 after the cache prune, on 2026-09-18 on both
backends (`backtest/research/gpu_backend_2026-09-18.md`) and on 2026-10-01 with both scripts.
The CUDA research twin (`backtest/cudaBasedBacktesting.py`, section 3) reproduced all five MD5s
above on 2026-09-18 in 0.8–2.4 s per book, plus the four 6-month acceptance books (2026-03-18 →
2026-09-18, M5 and M15 at $0.65 and $0.30: `86111c36…`, `e70d0252…`, `b8c98083…`, `e66b5fbd…`,
each equal to the reference script's), the M15 OOS `--export-features` CSV (`6239d197…`) and
`--record --record-bars` tables, and ten research-flag variants
(`backtest/research/cuda_backtester_2026-09-18.md`). Parity was re-proved for the 2026-09-30 change:
the five pins above with both scripts, six `--set` books on W3 (veto both / buy / sell and the stop
cap on M15, veto and cap on M5), the M15 OOS `--export-features` CSV with the veto on
(`732f396bb3ee6036b62750a8c755194e`) and its `--record --record-bars` tables
(`backtest/research/daily_trend_campaign_2026-09-30.md`).
Kept cache windows (`backtest/cache/oanda_XAU_USD_<tf>_<paddedStart>_<end>.parquet`; the padded
start is the window start minus 62 days for M5/M15/H1, 234 for H4 and 1,400 for D1, so the D1
and H4 files of one run have different start dates than its M5 file): 3-year `20230705_20260905`,
2-year `20240705_20260905`, OOS `20240331_20250330`, M5 12-month `20250613_20260814`, M15/H1
12-month `20250603_20260804`, 5-year `20210531_20260804`, 7.6-year `20181031_20260807`, the
2026-08-14 parity window `20250814_20260814`, their D1/H4 pads, and `calendar.parquet`.
2026-08-14 parity window `20250814_20260814`, the 6-month CUDA acceptance window
`20260115_20260918` (M5/M15/H1), their D1/H4 pads, and `calendar.parquet`. Added 2026-09-30 for
the daily-trend veto (an M15 run needs the D1 frame of its window): `1d_20201105_20260905` (W2),
`1d_20200801_20250330` (OOS), `1d_20150303_20260807` (the 7.6-year window), `1d_20211004_20260804`
and `4h_20241213_20260804` (the M15 / H1 12-month window), and the holdout window
`20260705_20261001` (M5 / M15 / H1) with `1d_20221105_20261001` and `4h_20260114_20261001`. The
M15 3-year run reads `1d_20191105_20260905`, which H4 W3 already kept.
The research recorder (`--record DIR [--record-bars]`) writes Parquet files describing every
accepted trade (indicator votes on all three timeframes, penalties, setup, outcome, calendar
@@ -479,22 +668,138 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
there too since 2026-09-08); use `setup_json.strategy`.
- The `--stored-candles` scorecard in `analyze_signal_log.py` still advances outcomes without the
locked-profit floor step that the logger and the backfill script use (owner decision pending), so
the two measurement paths can disagree on gap rows.
the two measurement paths can disagree on gap rows. Those are now only the trades still OPEN at
a hole (3 of 111 setups on the 2026-10-01 database copy; until then the replay also threw away
every trade that had exited before a later hole, 12 of 111).
- There is no holiday awareness anywhere. On Labor Day 2026 Oanda published no bars for hours, the
bot rescanned the stale bar 43 times, and the violent-move watch called the 12–14 UTC slot hot.
Interior gaps in history are logged, never fatal, for the same reason.
- Existing databases still contain an empty-going-forward `outlooks` table from the removed
market-outlook feature. Nothing reads it.
- The local news model has a gold head only. `_apply_model_sentiment` may write `xau_score`, XAU's
per-currency entry and, for a headline naming no other currency, the overall score; USD and every
other currency keep the keyword value (the pair sentiment is XAU − USD). It never touches
`_llm_sent_cache`, and a not-relevant headline displays 0.0 while `xau_score` keeps the raw
number. In `primary` mode nothing is archived but the DeepSeek audit copies, and the dashboard
process never archives in any mode (its DB writes are off), so a retrain needs
`deepseek_audit_per_day > 0`, a spell in `fallback`, or a labelling run in ML-NEWS.
- The news score is a trailing echo of the last day's price move (r ≈ 0.67 with the past day,
≈ 0.03 with the next). Both news gating flags stay off.
- The level-breach rescan and the candle loop both skip the settlement-pause window
(`_closed_bar_in_settlement`); a rescan there only re-reads the last pre-pause bar.
- The alert cooldown counts bars of `report._frames` (set by `analyze_pair`). A report built by
hand has no frames and falls back to clock time, which releases the first bars after the
settlement pause or the weekend too early; that is the fallback, not the rule.
- An alert's row is stamped `alerted_at` BEFORE the send and the stamp is cleared when no chat
received it, also when the alert's task is cancelled while the stamp is being written (a
shutdown at that moment: the worker job cannot be stopped, so its future's done-callback
undoes it). A cancellation in the MIDDLE of the send keeps the stamp, because a chat may
already hold the message. Two residues, both accepted: if a send fails completely after
another alert already counted the row, the other alert's "already open" line is one too
high; and a process KILLED between the stamp and the send leaves a stamp on an alert nobody
received (it also seeds the cooldown at the next start, which errs on the quiet side).
- `news_engine._record_source("news_model", …)` is the only sign that the local headline model
failed a batch (WARNING once, `/status` line `News model: not scoring`); in `primary` mode
nothing else scores those headlines for the cycle. The line describes the latest cycle that
needed the model: a cycle with nothing for it to score (fallback mode with DeepSeek healthy)
drops an older failure instead of leaving it on `/status`. `/status` failure details are
exception text and are Markdown-escaped; an unpaired underscore made Telegram reject the reply.
- The agent panel's scorecards split at `PANEL_PROMPT_VERSION` v2.2 (2026-10-01): same prompt
text, different input (calendar-week event timing, headlines newest first). `_price_context`
never fetches the calendar itself; the offline replay passes an as-of context and must not see
today's events.
- The Oanda empty-window rule cannot catch an anomalous empty answer for a window of 5 days or
less (a short head page; every 3-day M1 page, a timeframe nothing fetches).
- The agent panel never runs inside `analyze_from_dataframes`, and its cached verdict is never
reused when the evidence changed; the daily cap is the real spend bound.
- Live == backtest also means the backtester never writes to `signals.db` and never builds a
`NewsEngine`.
- Twelve Data candles are not a drop-in replacement for Oanda (section 4).
- `pyarrow` (Parquet) is installed in the dev venv only; live code never touches Parquet.
- CuPy from pip (`cupy-cuda12x`) cannot compile a kernel without the CUDA headers; the
`nvidia-cuda-runtime-cu12` wheel provides them and `backtest/gpu_kernels.py` points `CUDA_PATH`
at it. Until 2026-09-18 that failure was swallowed: every book was built on NumPy while the file
was called a GPU backtester. Now the fallback reason is in the `Backend:` line and Module 36
fails on a box that has a visible GPU but no working backend. `cupy` is banned from the live
roots by the same AST guard as scikit-learn.
- The CUDA kernels are bit-for-bit replicas of the CPU code: one thread per series (EWM, rolling)
or per trade, compiled with `--fmad=false`. Do not "optimise" them with a parallel scan or a
two-pass variance: the last bits change and a threshold can flip somewhere in 500k scans. A
single sequential series is not faster on the GPU than pandas' C loop; the win is the batch
across series and trades, and the replay itself stays on the CPU (same code as live).
- The replay workers are forked after the CUDA context exists and never touch the device (every
cache array is host NumPy, asserted before the fork). Python 3.12 may warn about forking a
multi-threaded process under `-X dev`; that is expected.
- `backtest/cudaBasedBacktesting.py` is a CUDA re-implementation of the decision stage, exempt
from "replays go through the real signal code" only while its book MD5 equals
`gpu_backtest.py`'s. A signal change that is not mirrored in `cuda_signal_kernels.py` makes the
twin drift silently: no test compares it with the live code except Module 37 and the paired
MD5 runs. It has no CPU fallback, refuses `--backend cpu`, and every description, warning and
reason string is produced on the host from the same f-string templates as the live code; the
kernels emit codes and numbers only.
- The kernels keep Python's evaluation order on purpose: `min`/`max` as argument-order ternaries
(never `fmin`/`fmax`), numpy's pairwise summation for `mean`/`nanmean`, pandas' online rolling
and EWM recurrences, Python 3.12's Neumaier-compensated `sum()` for the multi-timeframe mean,
`round(x, 2)` through an error-free product, `atr_percentile = count / 50 * 100` (not
`count * 2`: 15/50·100 = 30.000000000000004 flips "low" to "normal"). A cleaner formulation
flips a threshold somewhere in 500k bars. The as-of alignment compares int64 nanoseconds; a
mixed-unit index once returned slice length 0 for every confirmation frame.
- A printed-target change is not free: the spread gate judges the target, so a bigger target can
switch a signal from the GENERIC swing stop to the XAU_ATR 2 × ATR stop. The 1R printed floor
(`tp_print_floor_rr`) sits after the gates in all three generators for that reason; keep it there,
and never feed the printed target back into a gate.
- "Stop hunting" was tested on the live log (2026-09-26): 71% of live stop-outs ran a full stop
distance or more past the stop, the same as the books' 70–72%. The losers were real moves.
- The daily-trend veto has ONE definition (`signal_aggregator.daily_trend_series`; EMA span 50 and
lookback 60 are module constants, validated literals). Live, the bar replay and the twin's host
side (`trend_flags`) all call it; the kernel only receives a flag per bar. Changing the
literals, the Sunday filter or the as-of rule (a daily bar labelled D counts from D + 1 day ≤ the
primary bar's close) is a rule-1 event and a parity event. The live frame holds 1,000 daily bars
and a backtest pad 1,400 days; the EMA seed has decayed below float precision in both, so the
states agree.
- With the veto on, D1 is a REQUIRED frame of the M15 scan: a failed D1 fetch fails the scan
(`DATA UNAVAILABLE: 1d — scan failed`), it does not fall back to "no veto". After a deploy watch
that rate. Inside the core a missing D1 slice only adds the notice `Daily trend filter requested
but 1d data unavailable` and ships the setup; the replay reaches that state only before the
first daily bar of its pad.
- The veto is the LAST step of the core (after the chart-pattern path and the stale-veto cleanup),
so `gate_reason` lists `DAILY TREND FILTER` ahead of the gate chain. The rule itself only
removes a plan. In a book a removed entry also moves the 5-bar cooldown, which could free a
later entry: that is possible in principle and happened 0 times in every book of the campaign
(an observation, not a guarantee; count the added trades in any new comparison). The wide-stop cap never
changes the trade set either (it is applied after the min-RR and spread gates); a smaller stop
can lower the printed 1R target, never raise it.
- The control for any M15 candidate is now the factory config WITH the veto. Comparing against a
pre-2026-09-30 M15 number needs `--set xau_strategy.daily_trend_veto=off` (reproduces
`5b65ecf6…` / `09b8a8e7…` / `70195fb0…`).
- The veto does not find big losers. The trades it removes are break-even after costs (−0.03R to
−0.05R each against +0.15R to +0.25R for the kept ones); the gain is fewer flat trades and
shorter losing runs. The SELL half is carried by the rising 3-year window and the BUY half by the
falling years of 2019 → 2026; `buy`-only and `sell`-only both fail. 2021 is not improved and 2023
is worse: in a sideways year the daily trend flips late.
- The lot-size line reveals the owner's risk per trade. It is stamped per surface
(`exposure.attach_exposure`) and the channel copy of an alert is built separately
(`format_signal_message(report, channel=True)`); never print `report.size_line` on a public
surface without checking `exposure.size_shared`. The dashboard snapshot is public.
- "Already open" is the advisory outcome model's view of ALERTED plans (`alerted_at` set, outcome
`PENDING`), not a broker position. Rows alerted before the 2026-09-30 deploy carry no
`alerted_at`, so the line starts empty and fills as alerts go out; a `PENDING` row older than the
model's own time-out (300 bars, with a weekend allowance) is not shown. Stacked entries were not
worse trades in the books, so the line is information, never a reason the code skips an alert.
- `signals.daily_trend` is the core's value on M15 and the live wrapper's display stamp on M5, H1
and H4 (`_stamp_daily_trend_for_display`, one extra cached D1 fetch on M5 / H1); it is NULL on
blocked and failed scans. The stamp sits in `analyze_pair`, never in `analyze_from_dataframes`.
- The dashboard's due-timeframe check uses the raw wake time; flooring `wake − buffer` skipped
every H1 and H4 scan once (2026-08-02).
- The agent panel's price context reads the frames `analyze_pair` stashes on `report._frames`.
A report built by hand (scripts, tests) has no frames, so pass them to `_price_context` /
`run_panel` explicitly or the context carries geometry and base rate only. `scripts/panel_replay.py`
rebuilds them from the kept Oanda cache; a database copied from live is never migrated (read-only),
so the replay tolerates missing v2 columns and rebuilds payloads from the stored blobs.
- The AI estimate is a calibrated base rate, not a filter: on the 37 resolved setups of 2026-09 the
v2 personas sat within ±2 points of the class win rate (Brier 0.262 vs 0.263 for the base rate
alone, 0.299 for the old macro verdict) and separated nothing. Do not gate on it; do not iterate
prompts on that sample again (`backtest/research/panel_v2_2026-09-17.md`).
---
@@ -504,6 +809,17 @@ One line per validated change or verdict, newest first. Full evidence in the nam
| Date | Decision | Where the evidence is |
|---|---|---|
| 2026-10-01 | Bug review of the whole codebase (owner request), then the fixes. Signal path clean: indicators, grading, gates, the daily-trend veto, the as-of logic and both backtesters checked, and no live module needs a newer Python or SQLite than the server has. 17 defects elsewhere, all FIXED, each with a test; books byte-identical (the five pins reproduced on both scripts, 0 cache fallbacks). **Alert path**: the "already open" line was stamped at scan time, so two alerts at one candle close never counted each other (the M5 and M15 books enter together 42 times in three years); the row is now stamped just before the send on a one-worker `alert-db` pool and un-stamped when no chat received it (`_deliver_alert`, one path for the candle scan and the level break; no SQLite write on the event loop any more). The 5-bar cooldown counted clock time and forgot its state at every restart: it now counts the scan's own bars like `_apply_cooldown` (the old rule released the 22:00 UTC bar after the settlement pause: 6 M15 and 3 M5 entries the books never take in three years) and is restored from `signals.db` at start-up. The lot-size line went to every target that was not the channel (a group the owner ran `/autoscan` in): now the owner's own chat only. **Measurement**: the outcome model rejected a whole candle stream for a hole anywhere in it; continuity is now judged bar by bar in the walk, so the stored-candle replay resolves 108 of 111 logged setups instead of 99, all 108 equal to the R the live resolver stored. **History fetch**: an empty answer is accepted only for a window of 5 days or less with no malformed candle (a longer one is retried, then refused). **News**: one unreadable headline no longer costs the local model its whole batch (verified on the real bundle), a failing batch is a WARNING and a `/status` line, no DeepSeek audit call in a process that does not archive, the event-direction cache is per release. **LLM client**: `chat_json` never raises and never logs the key; the scan's news block is guarded (M15 OOS pair re-proved, Module 37 green). **Panel** (`PANEL_PROMPT_VERSION` v2.2, a new scorecard era): event timing over the cached calendar week, headlines newest first; the replay script uses the logged macro payloads it used to ignore (39 of 110 events). **Tests**: no `DROP COLUMN` (SQLite 3.35), Parquet-writing tests skip without `pyarrow`, the suite passes with both Parquet engines blocked and no GPU. Also fixed on the way: `/status` failure details are Markdown-escaped. **Independent review of these fixes** (a second agent, 20,000-case differential fuzz of the outcome model among its checks): four further LOW defects, fixed the same day — an alert cancelled while its row was being stamped kept the stamp; the panel's calendar read could trigger a 40 s refetch (now `peek_week_events`); a stale model-failure line could stay on `/status`; an HTTP error body was logged without redaction. **Independent verification of the 2026-09-30 direction change**: every figure reproduced, no signal changed by this pass, caveats recorded in section 5.13 of the campaign report. Suite 632 / 40 modules | plan file `~/.claude/plans/you-are-a-quant-velvety-wave.md`; sections 2, 4, 6 and 10 of this file |
| 2026-10-01 | New data sources, report only (owner request). Eight outside series tested against gold's own daily trend: the dollar (EUR/USD), silver, the US 10-year note, the real yield and the broad dollar index (FRED), the gold fund's share count, CFTC managed-money positioning, and Oanda's own client positioning (the position-book endpoint DOES answer on the practice API, with history to 2017; the plan had expected it gone). None shows a forward move of gold beyond gold's own trend (largest t-values 1.2 and 1.8); on the books only one flag, M5 trades against the broad dollar index (−11.2R over 435, losing in both eras), which is the 2025–2026 rally seen from another angle and does nothing on M15: WATCH, a hypothesis. Nothing ships, no feed added, all data stayed in the scratchpad | `backtest/research/new_inputs_2026-10.md` |
| 2026-10-01 | Context screens after the veto (owner request): weekly trend, four-hour trend, daily-trend strength, daily volatility state, place in the 20-day range, tick-volume surge, time to a high-impact US release; rule written first (a state needs 8% of both books and a loss in all four slices to earn a replay). NO state nominated on M15 or M5, no replay, nothing ships. Description: M15 has no losing pocket left in these views; M5 earns only where everything agrees (with the weekly and four-hour trend +0.16R to +0.24R per trade, against them about 0) and near the 20-day extreme in the trade's direction, so a "do not chase" filter would cut the part that pays | `backtest/research/context_screens_2026-10.md` |
| 2026-09-30 | Direction and setup-size campaign (owner request; RISK-FIRST rule approved by the owner, 2019 → 2026 allowed as supporting evidence). **Daily-trend veto SHIPPED on M15** (override 7, `daily_trend_veto` `both`): a finished setup against a clear daily trend (close vs 50-day EMA and vs 60 days earlier, closed D1 bars, Sunday stubs dropped) is removed. PASSED every clause: W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R; both costs; 3 of 4 neighbouring definitions. Accepted cost: W2 first half +61.8R vs +70.1R, old OOS window −4.7R. FAILED: `buy`-only and `sell`-only on M15, every mode on M5 (W3 drawdown 10.1R → 15.2R, long window worse) and on H1 (−9.7R). **Wide-stop cap FAILED** rule C (K = 6 +11.8R on W3 but K = 5 −18.6R on the long window: a peak, not a plateau); knob kept at 0.0. M15 books re-pinned (W2 `dfc647ee…`, OOS `e87b6de1…`), M5 / H1 / H4 pins unchanged, twin MD5-equal throughout, recorder parity re-proved. **Display lines SHIPPED** (books byte-identical): lot size for the owner's risk limit (`trade_size` in config.json, private by default), alerted same-direction plans still open (`signals.alerted_at`, read-only `open_alerted_setups`), the daily trend in words on every surface and in every logged row. Measured on the way: taking every M5 + M15 alert, two entries in three already have a same-direction plan open and up to 13 are open at once; those stacked entries are not worse trades. FIXED: the Oanda provider raised on an empty date window, so `fetch_range` aborted on a pad starting on a closed day. Stops and exits re-screened (25 variants): at a plateau, tighter is far worse. Suite 607 / 40 modules | `backtest/research/daily_trend_campaign_2026-09-30.md`; factory docstring M15 override 7 |
| 2026-09-27 | Local news scorer, round two: the owner's ML-NEWS model is now a DeBERTa-v3-small two-head transformer (gold-relevant? then bullish/bearish), distilled from 17,245 DeepSeek gold labels (Sep 8–22 archive + a 2019–2026 Reuters/CNBC/MarketWatch crawl labelled by DeepSeek, 560 calls). Golden set 72/78 (DeepSeek 78, keywords 24, the 09-17 TF-IDF 39), agreement with DeepSeek on unseen live days ~72% (ceiling 90%), 19 ms/headline on one CPU thread as int8-embedding ONNX; passed the four promotion criteria fixed on 09-22. OWNER DECISIONS: it is to score headlines here in mode `primary` with NO DeepSeek for headline scoring (DeepSeek stays for event direction, summaries, the dashboard endpoint); model files live OUTSIDE this tree at a configured path (280 MB, never in git); shadow mode declined (the archive already allows offline replay). IMPLEMENTED the same day (branch `sql-integration`): `news_model.py` (vendored runtime + never-raising singleton, lazy `onnxruntime` / `tokenizers`), `news_model` config section + `NEWS_MODEL_MODE` env, `news_engine._enrich_headline_scores` / `_apply_model_sentiment` (gold-only rules A–F) / `_audit_with_deepseek`, archive provenance for `model` rows, `deploy/README-news-model.md`, Module 38 (15 tests, one against the real bundle: bullish / neutral labels, 7 ms per headline on the dev box). Two further owner decisions during implementation: **`primary` mode stops writing `newsdata.db`** (only the DeepSeek audit sample, shipped 0, is archived) and **both processes load the model**. Books unaffected (news never enters signals); no signal file touched, no parity event. ML-NEWS `news_transformer.py` received the same two `load_model` edits (commit `9496cd9` there) | `docs/news_model_integration_spec_2026-09-27.md`; `~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md` |
| 2026-09-27 | Single "conviction" number vs the two-number grade (owner question: one number that is easier to understand and to decide on). Pre-registered; switch `signals.grading_mode` (`pair` default = the shipped rule, byte-identical on both scripts, all five pins reproduced; `min` = min(\|score\|, 0.6 × conf); `product` = \|score\| × conf / 100; thresholds from the control's corner points, no tuning; mirrored in K2, twin MD5-equal to the reference on both new paths). FAILED on M5 and M15: K-MIN moves 43–54 trades for −1.2R / −2.9R (a, b, e / a, b, d); K-PROD −29R on M15 with PF 1.29 and W3 DD 28.9R, neutral on M5. H1 / H4 K-PROD +11.9R / +17.0R on W3 by taking 40–100 extra low-score trades (H1 PF 1.98 → 1.75, DD doubles): a hypothesis, not evidence. Nothing ships; the knob stays research-only. Code review (high) before the runs: 6 findings fixed, 1 recorded. SHIPPED the display half of the question (owner request the same day): alerts lead with the grade and print score / confidence as words (`main.plain_words`, same line on Telegram, `/status`, CLI and the dashboard card); books untouched. Suite 566 | `backtest/research/conviction_campaign_2026-09-27.md` |
| 2026-09-26 | Quant review + live log 2026-09-04 → 09-25 (100 setups replayed on the scans' stored candles): stop-hunt claim REFUTED (71% of stop-outs ran ≥ 1R past the stop, same as the books); −25.1R of the −37.8R raw live book were consecutive-bar re-entries the backtester's cooldown never takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week windows. SHIPPED: 5-bar alert cooldown; alert wording (trail first as the tested exit, TP1 a guide with its R, planned RR, validated-book line per class); M15 grade-A SELL block (`sell_min_quality` `A+, B`, PASSED a–f: W2 +154.15R/813 PF 1.53 vs +146.14/931 PF 1.43, OOS +68.5 vs +60.67, W3 DD +8%); 1R printed-target floor in every generator after the gates (`tp_print_floor_rr`, owner rule "TP1 at least 1:1"; label-only re-pin of all four timeframes). FAILED: M15 `min_rr_ratio` 1.0 (trade-identical, −2.0R); a pre-gate target floor (−24.6R). OWNER'S CALL: M5 `tp_floor_rr` 1.0 adds +48.9R on W2 with both halves up but max DD 10.1 → 15.9R (ladder 0.6–1.0 non-monotone); not shipped. Telegram / dashboard AI block rewritten for a new trader (owner rule): one plain headline against the usual odds, the chance "in 100", one reason for and one against, no verdict word or confidence figure; persona prompts ban trading jargon (`PANEL_PROMPT_VERSION` v2.1). Two delivery bugs from the DB checks fixed: the macro-source panel verdict never stored its probability (all v2 rows NULL); the level-breach rescan re-logged the 20:45 M15 bar three times every evening after the settlement pause (guard added to `_handle_breach`). Indicator grid and volume / daily-direction screens (Part C) not run | `backtest/research/rr_and_sell_campaign_2026-09-26.md` |
| 2026-09-18 | Full-GPU research twin SHIPPED: `backtest/cudaBasedBacktesting.py` + `backtest/cuda_signal_kernels.py` (owner decisions: research only, `gpu_backtest.py` stays the base reference, every research flag supported, the one exemption from instructions item 6 recorded there). Proof: the five reference MD5s, four 6-month acceptance books (2026-03-18 → 2026-09-18, M5 and M15, $0.65 and $0.30), the M15 OOS `--export-features` CSV and `--record --record-bars` tables, and ten research-flag variants, all byte-identical to the reference script; Module 37 (16 tests) pins every stage against the live code. Books take 0.8–2.4 s instead of 13–115 s. Rule added: any signal change is a parity event for the twin | `backtest/research/cuda_backtester_2026-09-18.md` |
| 2026-09-18 | GPU backend made real: CuPy had never compiled a kernel on the dev box (`CUDA_PATH` missing for the pip layout), so every book to date was NumPy. SHIPPED: `backtest/gpu_kernels.py` (lazy, loud backend probe; `nvidia-cuda-runtime-cu12` header wheel; CUDA kernels for the EWM batch, the pandas rolling mean/var replica and the trade resolver, each a bit-for-bit twin of the CPU code), `--backend auto/gpu/cpu` + `FB2_BACKEND`, Module 36. All five reference MD5s reproduced on the GPU and with `--backend cpu`; precompute GPU == CPU on all 42 cached windows (2.34 M bars). Wall time roughly unchanged: the replay is CPU by design | `backtest/research/gpu_backend_2026-09-18.md` |
| 2026-09-17 | Local news scorer evaluated (the owner's scikit TF-IDF model in `~/Development/ML-NEWS`, trained on 606 DeepSeek gold labels): golden set 39/78 vs DeepSeek 78/78 vs keywords 24/78; out-of-sample agreement with DeepSeek 64% (DeepSeek agrees with itself 90%); cleaning / negation marking / label weighting move nothing; backfilling the 961 missing gold labels lifts macro-headline agreement 38% → 52% but only reshuffles golden-set errors. Verdict: a fallback, not a replacement. SHIPPED to grow a complete corpus: `xau_score` archived for every headline, no-currency headlines scored and archived (never shown), every new headline sent in its cycle (25 per call). Dashboard process made database-read-only (`set_db_writes_for_process`); the bot is the single writer. Books unaffected (news never enters signals) | `~/Development/ML-NEWS/eval/RESULTS_2026-09-17.md` (evaluation scripts live outside the repo by owner instruction) |
| 2026-09-17 | Live review of two weeks of `signals.db`: live == backtest bar for bar (26/29 trades matched, R within 0.14); the raw advisory −7.15R is cluster re-entries the validated books never take (cooldown view −1.6R, backtester +1.1R on the window, an ordinary fortnight); composite score inversely related to the 24 h move on all timeframes (two-week bear reading, test in-book with `--record-bars`); big-candle gate re-tested and REJECTED again (removes +32R / +60R of net-positive trades). Agent panel v2 SHIPPED: probabilities anchored to book base rates, code-computed decision context, no news doctrine, mean aggregator (`verdict_source: "mean"`), prompt version logged, offline replay harness; owner decision the same day: only the macro persona runs live (`personas: ["macro"]`, 1 call per panel; technical / risk / contrarian carried no information in either evaluation, kept for research); backtester `strategy` tag fixed (label-only, books re-pinned). Delivery findings: dashboard daemon on stale code since Sep 10, bot outage Sep 9 | `backtest/research/signal_log_review_2026-09-17.md`, `backtest/research/panel_v2_2026-09-17.md` |
| 2026-09-15 | Backtest folder cleanup: `diagnose_filters.py`, `shadow_scan.py`, and the two one-time news scripts removed (last commit with them `931b265`); every pinned CSV, the research output folders, the BTC/ETH cache snapshots, the Tiingo caches and 13 uncited Oanda windows moved to `~/Development/ForexBot2_trash_2026-09-15/` (346 MB) pending the owner's delete; all five reference MD5s reproduced afterwards. Suite 506 / 33 | commit `b0b71e1` |
| 2026-09-15 | AI market-outlook feature removed (module, hourly Telegram brief, `--outlook`, `outlooks` writer, config block). Legacy config block tolerated with a warning. Books byte-identical | commit `931b265` |
| 2026-09-13 | Bug audit, 11 delivery/measurement fixes, books byte-identical: NEUTRAL-signal C/B setups now alert (about 17% of the H1 12-month book's R, +3.7R of +22.2R, was never sent); a missing confirmation frame fails the scan; pandas `utcnow` replaced and `pandas<4`, `numpy<3` pinned; advisory outcomes keep locked-profit floors (`GAP_LOCKED`); scan-outcome labeler bounded to 72 h; settlement pause is a scheduled closure; `fetch_range` retries pages and audits gaps; cache-fallback count logged and must be 0; partial `regime_weights_override` raises; the bot scans each bar once and attaches its log handler in `main()` only; the dashboard catches up missed timeframes. Verified sound and unchanged: composite maths, scan-outcome labels, outcome exits, Oanda alignment, no look-ahead | plan file `~/.claude/plans/you-are-python-software-joyful-teacup.md`, commit `2836024` |
+81
View File
@@ -0,0 +1,81 @@
# Handoff: the local news model and ForexBot2 (2026-09-27)
**Status: IMPLEMENTED later the same day** (`news_model.py`, `news_model` config section, Module 38,
`deploy/README-news-model.md`). Two decisions were added during implementation: `primary` mode writes
nothing to `logs/newsdata.db` except the DeepSeek audit sample, and both processes load the model. The
authoritative record is the status note at the top of `docs/news_model_integration_spec_2026-09-27.md`
and the CLAUDE.md ledger row; this file is kept as the historical handoff and can be deleted.
Read this before touching headline scoring. It records what exists, what the owner decided, and what
was left to build at the time of writing.
## What exists
The owner's news model lives in a separate project, `~/Development/ML-NEWS`
(git `ssh://git@192.168.0.11:222/harry/News-ML-Decoder.git`). On 2026-09-27 it became a
DeBERTa-v3-small two-head transformer, exported to ONNX for CPU:
- Question 1: does this headline move the gold price? Question 2: if so, up or down?
- Trained to copy DeepSeek's gold score on 17,245 headlines: the Sep 8–22 `newsdata.db` archive plus a
2019–2026 Reuters / CNBC / MarketWatch crawl that DeepSeek labelled (560 calls).
- Golden set 72/78. DeepSeek 78, the keyword scorer 24, the 2026-09-17 TF-IDF model 39.
- Agreement with DeepSeek on unseen live-feed days about 72% (DeepSeek agrees with itself about 90%).
- 19 ms per headline on one CPU thread, 9 ms on four. Needs only `onnxruntime`, `tokenizers`, `numpy`.
- Bundle: `ML-NEWS/models/xau_transformer/{model.onnx (271 MB), tokenizer.json, meta.json}`.
- Full results: `~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md`.
It passed the four promotion criteria fixed in advance on 2026-09-22 (golden ≥ 55/78, distractors ≥ 6/8,
unseen-days agreement ≥ 62.8%, latency ≤ 20 ms). This supersedes the 2026-09-17 verdict "a fallback,
not a replacement" in the CLAUDE.md ledger.
## What the owner decided (2026-09-27)
1. **Use it as the primary headline scorer, with no DeepSeek calls for headline scoring.** DeepSeek
stays for everything else: calendar-event direction (`_llm_event_direction`), the news summary, the
dashboard's on-demand summary endpoint.
2. **Model files live outside the ForexBot2 tree**, for example `/home/Forex/models/xau_transformer`,
copied by hand, path set in `config.json`. Never in git. ForexBot2 keeps deploying by `git pull`.
3. **No shadow mode.** The archive already stores every headline with its score, so offline replay
covers the comparison.
4. Accepted consequence: with DeepSeek off for headlines, the archive stops receiving fresh DeepSeek
labels, which is what ML-NEWS retrains on. An optional config knob (`deepseek_audit_per_day`,
default 0) can label a few headlines a day for the archive only.
## What is left to build
Everything. The specification is `docs/news_model_integration_spec_2026-09-27.md` in this repo
(identical to `ML-NEWS/docs/FOREXBOT2_INTEGRATION_SPEC.md`). In one screen:
| File | Change |
|---|---|
| `requirements.txt` | add `onnxruntime`, `tokenizers` (no "torch"/"transformers" anywhere in the file; a test greps for them) |
| `news_model.py` (new) | vendored copy of ML-NEWS `news_transformer.py` + a load-once singleton that never raises; onnxruntime imported lazily |
| `app_config.py` | optional `news_model` section: `path`, `threads`, `mode` (`off`/`fallback`/`primary`), `deepseek_audit_per_day`; `NEWS_MODEL_MODE` env override |
| `news_engine.py` | `_apply_model_sentiment` + a mode switch at the call site (~line 945); model score goes into `xau_score` and the XAU entry only, never into USD |
| `news_archive.py` | store `teacher_model` for `model` rows (the `model` rank already exists) |
| `test_all.py` | 11 tests listed in the spec |
| `deploy/README-news-model.md` (new) | ops for both the bot and the dashboard daemon |
| `config.json` (live) | the new section |
Rules that must survive any implementation:
- The model has a gold head only. **Never write its score into a non-XAU currency**:
`get_pair_sentiment` computes `XAU_mean − USD_mean` and the signal aggregator acts on it.
- A headline the model calls not-relevant shows 0.0, whatever the raw score; the raw score still goes
to `xau_score` for training data.
- Never write model results into `_llm_sent_cache`; that cache decides what is re-sent to DeepSeek.
- If the model fails to load, `primary` falls back to the DeepSeek path for that cycle, not to keywords.
- Both the bot and `dashboard_publisher.py` build a `NewsEngine`; each process that loads the model
holds about 0.6 GB. Decide per process with the env override.
- News never enters signals. CLAUDE.md §7 applies unchanged; no book is affected by any of this.
## How to verify when it is built
`venv/bin/python test_all.py` green; one news cycle in `primary` mode archives only `score_source='model'`
rows with `xau_score` set and zero DeepSeek headline calls; renaming the model directory away still
starts both processes and yields `llm` rows; `mode="off"` reproduces today's rows exactly.
## Re-verify before editing
Every file:line number in the spec was checked on 2026-09-27 against `6af1237` and drifts with each
commit. Check them first.
+43 -11
View File
@@ -176,10 +176,31 @@ def _valid_bars(bars: Iterable[Dict[str, Any]]) -> List[Dict[str, Any]]:
return valid
def _next_bar_follows(
previous: datetime, stamp: datetime, step: timedelta
) -> bool:
"""True when ``stamp`` is the next tradable bar after ``previous``: every
bar slot between the two lies inside a scheduled XAU closure (the weekend
or the daily 17:00 New York maintenance hour) and ``stamp`` is on the bar
grid. ``step`` must be positive."""
expected = previous + step
while expected < stamp:
ny = expected.astimezone(_NY_TZ)
daily_maintenance = ny.weekday() < 4 and ny.hour == 17
if not (_is_forex_weekend(expected) or daily_maintenance):
return False
expected += step
return expected == stamp
def _has_continuity(
bars: List[Dict[str, Any]], last: datetime, timeframe: str
) -> bool:
"""Reject missing tradable bars while allowing scheduled XAU closures."""
"""Reject missing tradable bars while allowing scheduled XAU closures.
All-or-nothing over the whole of ``bars`` (the scan-outcome labeller
calls it one bar at a time). advance_outcome does not use it: it checks
each step as it walks, so a hole AFTER a trade's exit cannot void it."""
seconds = _TF_SECONDS.get(normalize_timeframe(timeframe))
if seconds is None:
return False
@@ -190,14 +211,7 @@ def _has_continuity(
stamp = bar["_timestamp"]
if stamp <= previous or stamp in seen:
continue
expected = previous + step
while expected < stamp:
ny = expected.astimezone(_NY_TZ)
daily_maintenance = ny.weekday() < 4 and ny.hour == 17
if not (_is_forex_weekend(expected) or daily_maintenance):
return False
expected += step
if expected != stamp:
if not _next_bar_follows(previous, stamp, step):
return False
previous = stamp
seen.add(stamp)
@@ -256,18 +270,36 @@ def advance_outcome(
state["result"] = None
state["reason"] = "stored candle snapshots lost continuity"
return state
if has_future and not _has_continuity(valid_bars, last,
state["timeframe"]):
if not has_future:
return state
seconds = _TF_SECONDS.get(normalize_timeframe(state["timeframe"]))
if seconds is None:
state["status"] = "UNRESOLVABLE"
state["result"] = None
state["reason"] = "stored candle snapshots contain an internal gap"
return state
step = timedelta(seconds=seconds)
for bar in valid_bars:
stamp = bar["_timestamp"]
if stamp <= signal_bar or stamp <= last:
continue
# Continuity is judged ONE STEP AT A TIME, only as far as the
# trade is still open. Until 2026-10-01 the whole stream was
# checked before the walk, so a trade that had exited days
# before a later hole (a holiday, an outage) was thrown away:
# 9 of 111 setups in the September log, 7 of them with an exit
# the live resolver had recorded. A hole reached while the
# trade is open still ends the walk; the progress made up to
# the bar before it stays, so resolve_locked_floor floors from
# the latest stop.
if not _next_bar_follows(last, stamp, step):
state["status"] = "UNRESOLVABLE"
state["result"] = None
state["reason"] = "stored candle snapshots contain an internal gap"
return state
if progress["entry_price"] is None:
entry = bar["open"]
initial_stop = entry - risk if buy else entry + risk
+712 -156
View File
File diff suppressed because it is too large Load Diff
+374
View File
@@ -0,0 +1,374 @@
{
"schema": "agent-panel-base-rates-v1",
"generated_at": "2026-10-01T05:49:09+00:00",
"key": "timeframe|strategy|grade|direction ('*' = any)",
"min_n_for_exact_class": 30,
"books": {
"5min": {
"file": "m5_w2.csv",
"md5": "e599c8a8652f95b7fcdf5a7ef2b564f5",
"trades": 781,
"first_entry": "2024-09-06 18:00:00+00:00",
"last_entry": "2026-09-04 15:15:00+00:00"
},
"15min": {
"file": "m15_w2.csv",
"md5": "dfc647ee02afd179dd9e031f344ed380",
"trades": 597,
"first_entry": "2024-09-05 06:15:00+00:00",
"last_entry": "2026-09-04 08:00:00+00:00"
},
"1h": {
"file": "h1_w3.csv",
"md5": "a44fd19b4067f46e5bd98d50cc3eff92",
"trades": 253,
"first_entry": "2023-09-20 18:00:00+00:00",
"last_entry": "2026-08-21 07:00:00+00:00"
},
"4h": {
"file": "h4_w3.csv",
"md5": "f3587bae3c5dc3eda468ea3588b17dcb",
"trades": 114,
"first_entry": "2023-10-02 08:00:00+00:00",
"last_entry": "2026-09-01 12:00:00+00:00"
}
},
"classes": {
"15min|*|*|*": {
"n": 597,
"win_rate": 64.32,
"mean_r": 0.2611,
"tp1_reach_rate": 53.6,
"median_bars_held": 34,
"book": "m15_w2.csv"
},
"15min|*|*|BUY": {
"n": 540,
"win_rate": 64.44,
"mean_r": 0.2725,
"tp1_reach_rate": 53.52,
"median_bars_held": 35,
"book": "m15_w2.csv"
},
"15min|*|*|SELL": {
"n": 57,
"win_rate": 63.16,
"mean_r": 0.1536,
"tp1_reach_rate": 54.39,
"median_bars_held": 27,
"book": "m15_w2.csv"
},
"15min|GENERIC|A+|BUY": {
"n": 2,
"win_rate": 0.0,
"mean_r": -0.544,
"tp1_reach_rate": 0.0,
"median_bars_held": 32,
"book": "m15_w2.csv"
},
"15min|GENERIC|A|BUY": {
"n": 191,
"win_rate": 67.02,
"mean_r": 0.3152,
"tp1_reach_rate": 59.16,
"median_bars_held": 45,
"book": "m15_w2.csv"
},
"15min|GENERIC|B|BUY": {
"n": 238,
"win_rate": 62.18,
"mean_r": 0.2096,
"tp1_reach_rate": 47.06,
"median_bars_held": 38,
"book": "m15_w2.csv"
},
"15min|GENERIC|B|SELL": {
"n": 52,
"win_rate": 63.46,
"mean_r": 0.1569,
"tp1_reach_rate": 53.85,
"median_bars_held": 28,
"book": "m15_w2.csv"
},
"15min|GENERIC|C|BUY": {
"n": 49,
"win_rate": 71.43,
"mean_r": 0.5518,
"tp1_reach_rate": 67.35,
"median_bars_held": 27,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A+|BUY": {
"n": 2,
"win_rate": 50.0,
"mean_r": -0.4203,
"tp1_reach_rate": 0.0,
"median_bars_held": 12,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A+|SELL": {
"n": 5,
"win_rate": 60.0,
"mean_r": 0.1192,
"tp1_reach_rate": 60.0,
"median_bars_held": 6,
"book": "m15_w2.csv"
},
"15min|XAU_ATR|A|BUY": {
"n": 58,
"win_rate": 62.07,
"mean_r": 0.2058,
"tp1_reach_rate": 53.45,
"median_bars_held": 11,
"book": "m15_w2.csv"
},
"1h|*|*|*": {
"n": 253,
"win_rate": 64.43,
"mean_r": 0.2768,
"tp1_reach_rate": 54.15,
"median_bars_held": 29,
"book": "h1_w3.csv"
},
"1h|*|*|BUY": {
"n": 253,
"win_rate": 64.43,
"mean_r": 0.2768,
"tp1_reach_rate": 54.15,
"median_bars_held": 29,
"book": "h1_w3.csv"
},
"1h|GENERIC|A+|BUY": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.1689,
"tp1_reach_rate": 0.0,
"median_bars_held": 45,
"book": "h1_w3.csv"
},
"1h|GENERIC|A|BUY": {
"n": 57,
"win_rate": 66.67,
"mean_r": 0.2243,
"tp1_reach_rate": 52.63,
"median_bars_held": 45,
"book": "h1_w3.csv"
},
"1h|GENERIC|B|BUY": {
"n": 112,
"win_rate": 63.39,
"mean_r": 0.2432,
"tp1_reach_rate": 51.79,
"median_bars_held": 38,
"book": "h1_w3.csv"
},
"1h|GENERIC|C|BUY": {
"n": 26,
"win_rate": 73.08,
"mean_r": 0.6826,
"tp1_reach_rate": 73.08,
"median_bars_held": 29,
"book": "h1_w3.csv"
},
"1h|XAU_ATR|A+|BUY": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.1181,
"tp1_reach_rate": 0.0,
"median_bars_held": 11,
"book": "h1_w3.csv"
},
"1h|XAU_ATR|A|BUY": {
"n": 56,
"win_rate": 62.5,
"mean_r": 0.224,
"tp1_reach_rate": 53.57,
"median_bars_held": 8,
"book": "h1_w3.csv"
},
"4h|*|*|*": {
"n": 114,
"win_rate": 57.89,
"mean_r": 0.1902,
"tp1_reach_rate": 48.25,
"median_bars_held": 12,
"book": "h4_w3.csv"
},
"4h|*|*|BUY": {
"n": 94,
"win_rate": 61.7,
"mean_r": 0.2409,
"tp1_reach_rate": 52.13,
"median_bars_held": 9,
"book": "h4_w3.csv"
},
"4h|*|*|SELL": {
"n": 20,
"win_rate": 40.0,
"mean_r": -0.0484,
"tp1_reach_rate": 30.0,
"median_bars_held": 20,
"book": "h4_w3.csv"
},
"4h|GENERIC|A|BUY": {
"n": 4,
"win_rate": 75.0,
"mean_r": 0.9295,
"tp1_reach_rate": 75.0,
"median_bars_held": 18,
"book": "h4_w3.csv"
},
"4h|GENERIC|A|SELL": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.2277,
"tp1_reach_rate": 0.0,
"median_bars_held": 29,
"book": "h4_w3.csv"
},
"4h|GENERIC|B|BUY": {
"n": 39,
"win_rate": 79.49,
"mean_r": 0.4706,
"tp1_reach_rate": 64.1,
"median_bars_held": 27,
"book": "h4_w3.csv"
},
"4h|GENERIC|B|SELL": {
"n": 12,
"win_rate": 25.0,
"mean_r": -0.4106,
"tp1_reach_rate": 16.67,
"median_bars_held": 40,
"book": "h4_w3.csv"
},
"4h|GENERIC|C|BUY": {
"n": 5,
"win_rate": 40.0,
"mean_r": -0.3577,
"tp1_reach_rate": 20.0,
"median_bars_held": 8,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A+|BUY": {
"n": 4,
"win_rate": 50.0,
"mean_r": 0.017,
"tp1_reach_rate": 50.0,
"median_bars_held": 4,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A+|SELL": {
"n": 2,
"win_rate": 100.0,
"mean_r": 0.6889,
"tp1_reach_rate": 50.0,
"median_bars_held": 4,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A|BUY": {
"n": 42,
"win_rate": 47.62,
"mean_r": 0.0547,
"tp1_reach_rate": 42.86,
"median_bars_held": 7,
"book": "h4_w3.csv"
},
"4h|XAU_ATR|A|SELL": {
"n": 5,
"win_rate": 60.0,
"mean_r": 0.5618,
"tp1_reach_rate": 60.0,
"median_bars_held": 14,
"book": "h4_w3.csv"
},
"5min|*|*|*": {
"n": 781,
"win_rate": 52.88,
"mean_r": 0.187,
"tp1_reach_rate": 41.23,
"median_bars_held": 34,
"book": "m5_w2.csv"
},
"5min|*|*|BUY": {
"n": 683,
"win_rate": 53.15,
"mean_r": 0.201,
"tp1_reach_rate": 41.0,
"median_bars_held": 33,
"book": "m5_w2.csv"
},
"5min|*|*|SELL": {
"n": 98,
"win_rate": 51.02,
"mean_r": 0.0894,
"tp1_reach_rate": 42.86,
"median_bars_held": 44,
"book": "m5_w2.csv"
},
"5min|GENERIC|A|BUY": {
"n": 139,
"win_rate": 61.15,
"mean_r": 0.2662,
"tp1_reach_rate": 53.96,
"median_bars_held": 77,
"book": "m5_w2.csv"
},
"5min|GENERIC|B|BUY": {
"n": 153,
"win_rate": 50.98,
"mean_r": 0.2315,
"tp1_reach_rate": 42.48,
"median_bars_held": 61,
"book": "m5_w2.csv"
},
"5min|GENERIC|B|SELL": {
"n": 98,
"win_rate": 51.02,
"mean_r": 0.0894,
"tp1_reach_rate": 42.86,
"median_bars_held": 44,
"book": "m5_w2.csv"
},
"5min|GENERIC|C|BUY": {
"n": 21,
"win_rate": 47.62,
"mean_r": -0.0144,
"tp1_reach_rate": 42.86,
"median_bars_held": 35,
"book": "m5_w2.csv"
},
"5min|XAU_ATR|A+|BUY": {
"n": 1,
"win_rate": 0.0,
"mean_r": -0.0329,
"tp1_reach_rate": 0.0,
"median_bars_held": 12,
"book": "m5_w2.csv"
},
"5min|XAU_ATR|A|BUY": {
"n": 31,
"win_rate": 54.84,
"mean_r": 0.288,
"tp1_reach_rate": 51.61,
"median_bars_held": 12,
"book": "m5_w2.csv"
},
"5min|XAU_CP|A|BUY": {
"n": 17,
"win_rate": 41.18,
"mean_r": 0.2498,
"tp1_reach_rate": 29.41,
"median_bars_held": 26,
"book": "m5_w2.csv"
},
"5min|XAU_CP|B|BUY": {
"n": 321,
"win_rate": 51.71,
"mean_r": 0.1621,
"tp1_reach_rate": 34.27,
"median_bars_held": 19,
"book": "m5_w2.csv"
}
}
}
+140
View File
@@ -794,6 +794,145 @@ def persona_scorecard(events: Sequence[SignalEvent], db_path: str) -> List[dict]
for (p, s), v in sorted(agg.items())]
# ── Probability scorecard (agent panel v2, 2026-09-17) ───────────────
#
# Scores the panel as a PREDICTOR of the trade closing profitably, per
# prompt version: Brier score and log-loss per persona and for the mean,
# calibration by probability band, discrimination (win rate / mean R by
# band), the constant-voter rate, rank correlation of p with R, and the
# counterfactual books. v1 rows (stance + confidence) are mapped onto the
# same 0-100 probability scale with agent_panel.pseudo_probability so the
# two eras are comparable; nothing here is a decision (owner rule: no
# persona-weight tuning before ~300 graded outcomes).
def _p_bands(pairs: Sequence[Tuple[float, float]]) -> Dict[str, dict]:
from agent_panel import P_OPPOSE, P_SUPPORT
out = {}
for label, pick in (("<=40", lambda p: p <= P_OPPOSE),
("40-60", lambda p: P_OPPOSE < p < P_SUPPORT),
(">=60", lambda p: p >= P_SUPPORT)):
sel = [(p, r) for p, r in pairs if pick(p)]
out[label] = ({"n": len(sel),
"mean_p": sum(p for p, _ in sel) / len(sel),
"win_rate": sum(1 for _, r in sel if r > 0) / len(sel),
"mean_r": sum(r for _, r in sel) / len(sel),
"sum_r": sum(r for _, r in sel)}
if sel else {"n": 0})
return out
def _score_pairs(pairs: Sequence[Tuple[float, float]]) -> Optional[dict]:
from agent_panel import P_OPPOSE, P_SUPPORT, derive_stance
if not pairs:
return None
ys = [1.0 if r > 0 else 0.0 for _, r in pairs]
ps = [p / 100.0 for p, _ in pairs]
eps = 1e-3
brier = sum((p - y) ** 2 for p, y in zip(ps, ys)) / len(ps)
ll = -sum(y * math.log(min(1 - eps, max(eps, p)))
+ (1 - y) * math.log(min(1 - eps, max(eps, 1 - p)))
for p, y in zip(ps, ys)) / len(ps)
stances = [derive_stance(p) for p, _ in pairs]
modal = max(set(stances), key=stances.count)
rho = None
if len(pairs) >= 5:
a = pd.Series([p for p, _ in pairs]).rank()
b = pd.Series([r for _, r in pairs]).rank()
c = a.corr(b)
rho = None if c != c else float(c)
return {"n": len(pairs), "brier": brier, "logloss": ll,
"mean_p": sum(p for p, _ in pairs) / len(pairs),
"win_rate": sum(ys) / len(ys),
"modal": modal, "constant_rate": stances.count(modal) / len(stances),
"rho_p_r": rho, "bands": _p_bands(pairs),
"sum_all": sum(r for _, r in pairs),
"sum_skip_oppose": sum(r for p, r in pairs if p > P_OPPOSE),
"sum_support_only": sum(r for p, r in pairs if p >= P_SUPPORT)}
def probability_scorecard(events: Sequence[SignalEvent], db_path: str
) -> Dict[str, Dict[str, Optional[dict]]]:
"""{prompt_version: {"mean": stats, "<persona>": stats, "base_rate": stats}}
over resolved events (one vote per event, first row with a panel)."""
from agent_panel import pseudo_probability
by_row = _row_index(events)
con = open_readonly(db_path)
have = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
cols = ["id", "panel_takes_json"]
for c in ("panel_probs_json", "panel_prompt_version", "panel_probability",
"panel_context_json"):
cols.append(c if c in have else f"NULL AS {c}")
per_version: Dict[str, Dict[str, List[Tuple[float, float]]]] = {}
seen: set = set()
try:
for rid, takes_json, probs_json, version, p_mean, ctx_json in con.execute(
f"SELECT {', '.join(cols)} FROM signals"
" WHERE panel_takes_json IS NOT NULL ORDER BY id"):
ev = by_row.get(rid)
if ev is None or ev.r_headline is None:
continue
ekey = ev.rows[0].id if ev.rows else rid
if ekey in seen:
continue
seen.add(ekey)
r = ev.r_headline
version = version or "v1"
bucket = per_version.setdefault(version, {})
probs: Dict[str, float] = {}
try:
source = json.loads(probs_json) if probs_json else json.loads(takes_json)
except (TypeError, ValueError):
continue
for t in source or []:
p = t.get("p_win")
if p is None and t.get("stance") in ("support", "oppose", "neutral"):
p = pseudo_probability(t["stance"], float(t.get("confidence") or 0.0))
if p is not None:
probs[str(t.get("persona"))] = float(p)
bucket.setdefault(str(t.get("persona")), []).append((float(p), r))
if p_mean is not None:
bucket.setdefault("mean", []).append((float(p_mean), r))
elif len(probs) >= 3:
bucket.setdefault("mean", []).append(
(sum(probs.values()) / len(probs), r))
try:
base = (json.loads(ctx_json) if ctx_json else {}).get("base_rate") or {}
if base.get("win_rate") is not None:
bucket.setdefault("base_rate", []).append((float(base["win_rate"]), r))
except (TypeError, ValueError, AttributeError):
pass
finally:
con.close()
return {v: {k: _score_pairs(pairs) for k, pairs in sorted(b.items())}
for v, b in sorted(per_version.items())}
def print_probability_scorecard(card: Dict[str, Dict[str, Optional[dict]]]) -> None:
print("\n── Probability scorecard (Brier: lower is better; 0.25 = coin flip) " + "─" * 6)
if not card:
print(" no resolved events with a panel yet")
return
for version, preds in card.items():
print(f" prompt version {version}")
print(f" {'predictor':12s} {'n':>3s} {'Brier':>6s} {'logloss':>7s} {'mean p':>6s} "
f"{'win%':>5s} {'modal':>8s} {'const%':>6s} {'rho(p,R)':>8s} "
f"{'R all':>7s} {'skip<=40':>8s} {'>=60only':>8s}")
for name, st in preds.items():
if not st:
continue
rho = f"{st['rho_p_r']:+8.2f}" if st["rho_p_r"] is not None else " -"
print(f" {name:12s} {st['n']:3d} {st['brier']:6.3f} {st['logloss']:7.3f} "
f"{st['mean_p']:6.1f} {100 * st['win_rate']:5.0f} {st['modal']:>8s} "
f"{100 * st['constant_rate']:6.0f} {rho} {st['sum_all']:+7.2f} "
f"{st['sum_skip_oppose']:+8.2f} {st['sum_support_only']:+8.2f}")
mean = preds.get("mean")
if mean:
cells = " ".join(
f"{k}: n={v['n']} p={v['mean_p']:.0f} win={100 * v['win_rate']:.0f}% R={v['mean_r']:+.2f}"
if v["n"] else f"{k}: n=0" for k, v in mean["bands"].items())
print(f" calibration of the mean: {cells}")
# ── News sentiment lead/lag ───────────────────────────────────────────
@@ -1309,6 +1448,7 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
for p in persona_scorecard(events, args.db):
print(f" {p['persona']:11} {p['stance']:8} n={p['n']:3d} "
f"mean {p['mean_r']:+.3f}R win {100 * p['win_rate']:.0f}%")
print_probability_scorecard(probability_scorecard(events, args.db))
if args.news:
print_news(news_lead_lag(rows, streams),
news_extremes(rows, streams),
+186 -5
View File
@@ -25,6 +25,7 @@ bot, CLI and backtester all see the same overlay (live == backtest).
import json
import logging
import math
import os
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
@@ -90,7 +91,8 @@ _KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level",
# config.json files still carry the block and must keep
# loading; load_app_config logs one warning when it is there.
"market_outlook",
"violent_watch", "signal_log", "news_archive"}
"violent_watch", "signal_log", "news_archive", "news_model",
"trade_size"}
_KNOWN_OANDA = {
# "account_id" is TOLERATED but ignored: it belonged to the auto-
# execution daemon removed 2026-09-03 (last commit 31002cb). The candle
@@ -105,12 +107,27 @@ _KNOWN_AGENT_PANEL = {
"enabled", "telegram", "dashboard", "min_quality",
"max_panels_per_day", "reuse_ttl_seconds",
"candles_enabled", "candle_bars_primary", "candle_bars_confirm",
"verdict_source",
"verdict_source", "personas",
}
_KNOWN_VIOLENT_WATCH = {"enabled", "alert_line", "headsup",
"max_headsup_per_day", "headsup_min_level"}
_KNOWN_SIGNAL_LOG = {"enabled", "path", "log_blocked_scans"}
_KNOWN_NEWS_ARCHIVE = {"enabled", "path"}
_KNOWN_NEWS_MODEL = {"path", "threads", "mode", "deepseek_audit_per_day"}
_NEWS_MODEL_MODES = ("off", "fallback", "primary")
# Lot-size guidance on the alert (exposure.py, display only). Off until
# the owner sets risk_per_trade. The size reveals the owner's risk per
# trade, so it is printed in the owner's own chat (telegram.chat_id) and
# the CLI only; telegram_channel switches it on for every other Telegram
# chat (the subscriber channel, a group), dashboard for the public page.
_KNOWN_TRADE_SIZE = {"risk_per_trade", "contract_size", "lot_step",
"telegram_channel", "dashboard"}
_TRADE_SIZE_DEFAULTS = {
"contract_size": 100.0, # ounces per 1.00 lot of XAU/USD
"lot_step": 0.01, # smallest size increment the broker accepts
"telegram_channel": False,
"dashboard": False,
}
_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
_KNOWN_SCANNING = {"pairs", "timeframes"}
_KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
@@ -150,6 +167,8 @@ class AppConfig:
default_factory=lambda: dict(_VIOLENT_WATCH_DEFAULTS))
signal_log_settings: Dict[str, Any] = field(default_factory=dict)
news_archive_settings: Dict[str, Any] = field(default_factory=dict)
news_model_settings: Dict[str, Any] = field(default_factory=dict)
trade_size_settings: Dict[str, Any] = field(default_factory=dict)
log_level: str = "INFO"
@@ -330,10 +349,27 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
f"config.json: agent_panel.{key} must be an integer "
"between 1 and 100")
if "verdict_source" in panel and panel["verdict_source"] not in \
("macro", "chair"):
("mean", "macro", "chair"):
raise AppConfigError(
'config.json: agent_panel.verdict_source must be "macro" or '
'"chair"')
'config.json: agent_panel.verdict_source must be "mean", '
'"macro" or "chair"')
if "personas" in panel:
names = panel["personas"]
valid = ("macro", "technical", "risk", "contrarian")
if (not isinstance(names, list) or not names
or any(not isinstance(n, str) or n not in valid for n in names)
or len(set(names)) != len(names)):
raise AppConfigError(
"config.json: agent_panel.personas must be a non-empty list "
f"of distinct names from {valid}")
if panel.get("verdict_source") == "chair" and len(names) < 2:
raise AppConfigError(
'config.json: agent_panel.verdict_source "chair" needs at '
"least two personas")
if panel.get("verdict_source") == "macro" and "macro" not in names:
raise AppConfigError(
'config.json: agent_panel.verdict_source "macro" needs the '
"macro persona in agent_panel.personas")
app.agent_panel_settings = dict(panel)
if "market_outlook" in raw:
@@ -388,6 +424,51 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
"config.json: news_archive.path must be a non-empty string")
app.news_archive_settings = dict(narch)
nm = raw.get("news_model", {})
_check_keys(nm, _KNOWN_NEWS_MODEL, '"news_model"')
if "mode" in nm and nm["mode"] not in _NEWS_MODEL_MODES:
raise AppConfigError(
'config.json: news_model.mode must be "off", "fallback" or "primary"')
if "threads" in nm and (not isinstance(nm["threads"], int)
or isinstance(nm["threads"], bool)
or nm["threads"] < 1):
raise AppConfigError(
"config.json: news_model.threads must be a positive integer")
if "deepseek_audit_per_day" in nm and (
not isinstance(nm["deepseek_audit_per_day"], int)
or isinstance(nm["deepseek_audit_per_day"], bool)
or nm["deepseek_audit_per_day"] < 0):
raise AppConfigError(
"config.json: news_model.deepseek_audit_per_day must be a "
"non-negative integer")
if nm.get("mode", "off") != "off" and (
not isinstance(nm.get("path"), str) or not nm.get("path")):
raise AppConfigError(
"config.json: news_model.path is required when news_model.mode "
"is not off")
app.news_model_settings = dict(nm)
size = raw.get("trade_size", {})
_check_keys(size, _KNOWN_TRADE_SIZE, '"trade_size"')
for key in ("risk_per_trade", "contract_size", "lot_step"):
if key in size:
v = size[key]
if (isinstance(v, bool) or not isinstance(v, (int, float))
or not math.isfinite(v)):
raise AppConfigError(
f"config.json: trade_size.{key} must be a number")
# risk_per_trade 0 = the size line is off (the default)
if v < 0 or (v == 0 and key != "risk_per_trade"):
raise AppConfigError(
f"config.json: trade_size.{key} must be "
+ ("zero (off) or a positive number"
if key == "risk_per_trade" else "a positive number"))
for flag in ("telegram_channel", "dashboard"):
if flag in size and not isinstance(size[flag], bool):
raise AppConfigError(
f"config.json: trade_size.{flag} must be true or false")
app.trade_size_settings = dict(size)
level = raw.get("log_level", app.log_level)
if level not in _VALID_LOG_LEVELS:
raise AppConfigError(
@@ -492,10 +573,34 @@ def violent_watch_enabled(app: Optional[AppConfig] = None) -> bool:
return bool(app.violent_watch_settings.get("enabled", False))
# ── Process-level database write switch ──────────────────────────────
#: True in every process by default. The dashboard publisher turns it
#: off at startup (owner decision 2026-09-17): it is not always running,
#: so the Telegram bot must be the single writer of signals.db and
#: newsdata.db — two writers were also two processes contending for the
#: same WAL files. Every SQLite writer (signal_log, news_archive) reads
#: the switch through signal_log_enabled() / news_archive_enabled(), so
#: a switched-off process cannot write whatever config.json or the
#: environment says. Read-only users of the databases are unaffected.
_process_db_writes = True
def set_db_writes_for_process(enabled: bool) -> None:
global _process_db_writes
_process_db_writes = bool(enabled)
def db_writes_enabled() -> bool:
return _process_db_writes
# ── Signal-log resolution helpers (env → config.json → default) ──────
def signal_log_enabled(app: Optional[AppConfig] = None) -> bool:
"""Default ON — signal history is the platform's audit trail."""
if not _process_db_writes:
return False
env = os.environ.get("SIGNAL_LOG_ENABLED")
if env:
return env.strip().lower() in ("1", "true", "yes", "on")
@@ -528,6 +633,65 @@ def signal_log_path(app: Optional[AppConfig] = None) -> str:
return path
# ── Local news model (env → config.json → default) ───────────────────
#
# The owner's ML-NEWS XAU transformer scores headlines locally
# (news_model.py). "off" is today's DeepSeek/keyword path; "fallback" runs
# the model only on rows DeepSeek left keyword-scored; "primary" scores
# every headline with the model and never calls DeepSeek for headlines
# (except the optional audit sample). NEWS_MODEL_MODE lets one process
# (the dashboard daemon's systemd unit) pick its own mode: each process
# that loads the model holds ~0.6 GB.
def news_model_mode(app: Optional[AppConfig] = None) -> str:
env = os.environ.get("NEWS_MODEL_MODE")
if env:
mode = env.strip().lower()
if mode in _NEWS_MODEL_MODES:
return mode
logger.warning(
'NEWS_MODEL_MODE=%r is not one of %s; using config.json',
env, _NEWS_MODEL_MODES)
app = app or get_app_config()
mode = app.news_model_settings.get("mode", "off")
return mode if mode in _NEWS_MODEL_MODES else "off"
def news_model_path(app: Optional[AppConfig] = None) -> Optional[str]:
"""Absolute path to the bundle directory (model.onnx, tokenizer.json,
meta.json), or None when not configured. A relative value resolves
against the repo root like news_archive_path."""
app = app or get_app_config()
path = app.news_model_settings.get("path")
if not isinstance(path, str) or not path:
return None
if not os.path.isabs(path):
path = os.path.join(os.path.dirname(os.path.abspath(__file__)), path)
return path
def news_model_threads(app: Optional[AppConfig] = None) -> int:
"""ONNX intra-op threads (measured 2026-09-27: 1 → 19 ms per headline,
4 → 9 ms, 16 → 13 ms)."""
app = app or get_app_config()
try:
return max(1, int(app.news_model_settings.get("threads", 4)))
except (TypeError, ValueError):
return 4
def news_model_audit_per_day(app: Optional[AppConfig] = None) -> int:
"""primary mode only: headlines per UTC day also sent to DeepSeek so
the archive keeps receiving fresh training labels. Archive-only, never
displayed. 0 = no DeepSeek headline calls at all (owner decision
2026-09-27)."""
app = app or get_app_config()
try:
return max(0, int(app.news_model_settings.get("deepseek_audit_per_day", 0)))
except (TypeError, ValueError):
return 0
# ── News-archive resolution helpers (env → config.json → default) ────
def news_archive_enabled(app: Optional[AppConfig] = None) -> bool:
@@ -535,6 +699,8 @@ def news_archive_enabled(app: Optional[AppConfig] = None) -> bool:
scorer and cannot be backfilled once a fetch window has passed. It
lives in its own DB so a signals.db rotation cannot take it with it
(which is how three weeks of news were lost on 2026-09-04)."""
if not _process_db_writes:
return False
env = os.environ.get("NEWS_ARCHIVE_ENABLED")
if env:
return env.strip().lower() in ("1", "true", "yes", "on")
@@ -556,6 +722,21 @@ def news_archive_path(app: Optional[AppConfig] = None) -> str:
return path
# ── Lot-size guidance (config.json only; display) ────────────────────
def trade_size_settings(app: Optional[AppConfig] = None) -> Optional[Dict[str, Any]]:
"""The lot-size guidance settings with their defaults filled in, or
None when the owner has not set `trade_size.risk_per_trade` (the size
line is then off on every surface)."""
app = app or get_app_config()
block = app.trade_size_settings or {}
if not block.get("risk_per_trade"):
return None
out = dict(_TRADE_SIZE_DEFAULTS)
out.update(block)
return out
# ── PlatformConfig overlay ────────────────────────────────────────────
def apply_pair_overrides(
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+330 -73
View File
@@ -19,7 +19,9 @@ Key properties:
- Trade simulation is full candle walk on the primary timeframe using
High/Low; if both SL and TP1 hit in the same bar, SL is assumed first
(conservative).
- Ships with CuPy fallback to NumPy so the script runs anywhere.
- Ships with CuPy fallback to NumPy so the script runs anywhere. The CUDA
kernels (backtest/gpu_kernels.py) are bit-for-bit replicas of the NumPy
path, so both backends produce the same book; --backend cpu forces NumPy.
Usage:
python backtest/gpu_backtest.py --pair XAUUSD --days 365
@@ -61,39 +63,27 @@ from config import ( # noqa: E402
)
from providers import BaseDataProvider, make_data_provider # noqa: E402
from engine import IndicatorSignal, TechnicalAnalysis, TechnicalEngine # noqa: E402
from signal_aggregator import SignalAggregator, SignalReport, TradeSetup # noqa: E402
from signal_aggregator import ( # noqa: E402
SignalAggregator, SignalReport, TradeSetup, daily_trend_needed,
)
import position_sizing # noqa: E402
from backtest import replay_record # noqa: E402
from backtest import gpu_kernels as _gk # noqa: E402
# ─────────────────────────────────────────────────────────────────────
# GPU shim — CuPy if available AND CUDA runtime works, else NumPy.
# An importable CuPy can still fail at first kernel launch if CUDA libs
# are missing (containers, headless boxes), so verify with a tiny op.
# GPU backend — backtest/gpu_kernels.py owns the CuPy state (probed
# lazily and never silently: a fallback carries its reason into the
# "Backend:" line) and the CUDA kernels. Every kernel is a bit-for-bit
# replica of the NumPy/pandas code kept in this file as the CPU path, so
# `--backend cpu` and the GPU produce the same book (proof: the five
# reference MD5s, CLAUDE.md section 9). The forked replay workers never
# touch the device: every precomputed series is moved to host first.
# ─────────────────────────────────────────────────────────────────────
def _detect_gpu():
try:
import cupy as cp_mod # type: ignore
# Force runtime init: a kernel launch will surface missing libs.
_ = cp_mod.asarray(np.zeros(2, dtype=np.float64))
_ = cp_mod.cumsum(_).get()
return cp_mod, True
except Exception:
return np, False
BACKEND = _gk.BACKEND
cp, _GPU_AVAILABLE = _detect_gpu()
def _to_gpu(arr: np.ndarray):
"""Move a numpy array to GPU (or no-op if CuPy unavailable)."""
return cp.asarray(arr) if _GPU_AVAILABLE else arr
def _to_cpu(arr) -> np.ndarray:
"""Pull array back to host."""
if _GPU_AVAILABLE and hasattr(arr, "get"):
return arr.get()
return np.asarray(arr)
def _gpu_on() -> bool:
return BACKEND.available
logger = logging.getLogger("gpu_backtest")
@@ -215,18 +205,29 @@ class IndicatorCache:
def precompute_indicators(
df: pd.DataFrame, tech_cfg, gpu_log: bool = False
) -> IndicatorCache:
"""Compute every causal indicator series once on the full df."""
"""Compute every causal indicator series once on the full df.
Dispatches to the CUDA kernels when the backend is available and to
the NumPy/pandas code otherwise. The two are bit-identical (Module 36
and the 42-window check in backtest/research/gpu_backend_2026-09-18.md).
"""
if gpu_log:
logger.info(
f"Pre-computing indicators on {len(df)} bars "
f"({'CuPy GPU' if _gpu_on() else 'NumPy CPU'} backend)"
)
if _gpu_on():
return _precompute_indicators_gpu(df, tech_cfg)
return _precompute_indicators_cpu(df, tech_cfg)
def _precompute_indicators_cpu(df: pd.DataFrame, tech_cfg) -> IndicatorCache:
"""The NumPy/pandas path — the reference the CUDA twin replicates."""
close = df["Close"].to_numpy(dtype=np.float64)
high = df["High"].to_numpy(dtype=np.float64)
low = df["Low"].to_numpy(dtype=np.float64)
n = len(close)
if gpu_log:
logger.info(
f"Pre-computing indicators on {n} bars "
f"({'GPU' if _GPU_AVAILABLE else 'CPU'} backend)"
)
# ── EMAs (use ewm span semantics, identical to pandas .ewm(span=, adjust=False).mean()) ─
ema_fast = _gpu_ewm_span(close, tech_cfg.ema_fast)
ema_medium = _gpu_ewm_span(close, tech_cfg.ema_medium)
@@ -388,6 +389,187 @@ def precompute_indicators(
)
def _assert_host_cache(cache: IndicatorCache) -> None:
"""Every array the forked workers will read must be plain NumPy."""
for name, val in vars(cache).items():
if name in ("df", "index_pos", "rsi_series", "macd_hist_series") or val is None:
continue
if type(val) is not np.ndarray:
raise TypeError(f"IndicatorCache.{name} is {type(val).__name__}, not np.ndarray")
def _precompute_indicators_gpu(df: pd.DataFrame, tech_cfg) -> IndicatorCache:
"""CUDA twin of `_precompute_indicators_cpu`.
Same expressions in the same order (single-op CuPy ufuncs cannot fuse
into an FMA and double add/mul/div/sqrt are correctly rounded on both
sides); the recurrences and the pandas rolling statistics run in the
kernels of backtest/gpu_kernels.py, one thread per series. Everything
comes back to the host once at the end: the replay workers read NumPy.
"""
cp = BACKEND.cp
nan = float("nan")
close_h = df["Close"].to_numpy(dtype=np.float64)
n = len(close_h)
close = cp.asarray(close_h)
high = cp.asarray(df["High"].to_numpy(dtype=np.float64))
low = cp.asarray(df["Low"].to_numpy(dtype=np.float64))
# ── RSI inputs (delta[0] is NaN; NaN > 0 is False so gain[0] = 0.0) ──
delta = cp.empty(n, dtype=cp.float64)
delta[0] = nan
delta[1:] = close[1:] - close[:-1]
gain = cp.where(delta > 0, delta, 0.0)
loss = cp.where(delta < 0, -delta, 0.0)
# ── True range (fmax ignores NaN, like the CPU path) ──
prev_close = cp.empty(n, dtype=cp.float64)
prev_close[0] = nan
prev_close[1:] = close[:-1]
tr1 = high - low
tr2 = cp.abs(high - prev_close)
tr3 = cp.abs(low - prev_close)
tr2[0] = nan
tr3[0] = nan
tr = cp.fmax(cp.fmax(tr1, tr2), tr3)
# ── Directional movement ──
plus_dm_raw = cp.empty(n, dtype=cp.float64)
minus_dm_raw = cp.empty(n, dtype=cp.float64)
plus_dm_raw[0] = nan
minus_dm_raw[0] = nan
plus_dm_raw[1:] = high[1:] - high[:-1]
minus_dm_raw[1:] = -(low[1:] - low[:-1])
cond_plus = (plus_dm_raw > minus_dm_raw) & (plus_dm_raw > 0)
plus_dm = cp.where(cond_plus, plus_dm_raw, 0.0)
cond_minus = (minus_dm_raw > plus_dm_raw) & (minus_dm_raw > 0)
minus_dm = cp.where(cond_minus, minus_dm_raw, 0.0)
# ── Stage 1: every recurrence that depends only on price, one launch ──
def span_alpha(span: int) -> float:
return 2.0 / (span + 1.0) # identical to _gpu_ewm_span
rsi_p = tech_cfg.rsi_period
atr_p = tech_cfg.atr_period
adx_p = tech_cfg.adx_period
kc_period = getattr(tech_cfg, 'kc_period', 20)
kc_atr_period = getattr(tech_cfg, 'kc_atr_period', 10)
kc_multiplier = getattr(tech_cfg, 'kc_multiplier', 1.5)
have_trend = n >= tech_cfg.ema_trend
jobs = [
(close, span_alpha(tech_cfg.ema_fast), 0),
(close, span_alpha(tech_cfg.ema_medium), 0),
(close, span_alpha(tech_cfg.ema_slow), 0),
(close, span_alpha(tech_cfg.macd_fast), 0),
(close, span_alpha(tech_cfg.macd_slow), 0),
(close, span_alpha(kc_period), 0),
(gain, 1.0 / rsi_p, rsi_p),
(loss, 1.0 / rsi_p, rsi_p),
(tr, 1.0 / atr_p, atr_p),
(tr, 1.0 / adx_p, adx_p),
(plus_dm, 1.0 / adx_p, adx_p),
(minus_dm, 1.0 / adx_p, adx_p),
(tr, 1.0 / kc_atr_period, kc_atr_period),
]
if have_trend:
jobs.append((close, span_alpha(tech_cfg.ema_trend), 0))
out = _gk.ewm_batch_dev([j[0] for j in jobs], [j[1] for j in jobs],
[j[2] for j in jobs])
(ema_fast, ema_medium, ema_slow, ema_macd_fast, ema_macd_slow, kc_mid,
avg_gain, avg_loss, atr, atr_adx, plus_dm_smooth, minus_dm_smooth,
kc_atr) = out[:13]
ema_trend = out[13] if have_trend else None
# ── RSI ──
avg_loss_safe = cp.where(avg_loss == 0, 1e-10, avg_loss)
rs = avg_gain / avg_loss_safe
rsi = 100.0 - (100.0 / (1.0 + rs))
# ── MACD line; DI / DX ──
macd_line = ema_macd_fast - ema_macd_slow
atr_adx_safe = cp.where(atr_adx == 0, nan, atr_adx)
plus_di = 100.0 * plus_dm_smooth / atr_adx_safe
minus_di = 100.0 * minus_dm_smooth / atr_adx_safe
di_sum = plus_di + minus_di
di_sum_safe = cp.where(di_sum == 0, nan, di_sum)
dx = 100.0 * cp.abs(plus_di - minus_di) / di_sum_safe
# ── Stage 2: recurrences on stage-1 outputs, one launch ──
signal_line, adx = _gk.ewm_batch_dev(
[macd_line, dx], [span_alpha(tech_cfg.macd_signal), 1.0 / adx_p], [0, adx_p])
macd_hist = macd_line - signal_line
# ── Bollinger: the pandas rolling replica (mean + var, then zsqrt) ──
(bb_mid,), (bb_var,) = _gk.roll_mean_var_dev([close], tech_cfg.bb_period, ddof=0)
bb_std = _gk.roll_std_from_var_dev(bb_var)
bb_upper = bb_mid + tech_cfg.bb_std * bb_std
bb_lower = bb_mid - tech_cfg.bb_std * bb_std
bb_range = bb_upper - bb_lower
bb_range_safe = cp.where(bb_range == 0, nan, bb_range)
bb_position = (close - bb_lower) / bb_range_safe
bb_position = cp.where(cp.isnan(bb_position), 0.5, bb_position)
bb_mid_safe = cp.where(bb_mid == 0, nan, bb_mid)
bb_band_width = (bb_upper - bb_lower) / bb_mid_safe
# 20-bar mean of the band width: the CPU path uses the literal 20.
(bb_avg_width,), _unused = _gk.roll_mean_var_dev([bb_band_width], 20, ddof=0)
# ── ADX slope ──
adx_slope_lookback = getattr(tech_cfg, 'adx_slope_lookback', 5)
adx_slope = cp.full(n, nan, dtype=cp.float64)
if n > adx_slope_lookback:
adx_slope[adx_slope_lookback:] = adx[adx_slope_lookback:] - adx[:-adx_slope_lookback]
# ── ROC ──
roc_period = getattr(tech_cfg, 'roc_period', 10)
roc = cp.full(n, nan, dtype=cp.float64)
if n > roc_period:
prev_close = close[:-roc_period]
prev_safe = cp.where(prev_close == 0, nan, prev_close)
roc[roc_period:] = ((close[roc_period:] - prev_close) / prev_safe) * 100.0
# ── Keltner ──
kc_upper = kc_mid + kc_multiplier * kc_atr
kc_lower = kc_mid - kc_multiplier * kc_atr
kc_squeeze = ((bb_lower > kc_lower) & (bb_upper < kc_upper)).astype(cp.float64)
# ── One transfer to the host; the workers read NumPy only ──
def h(a):
return a.get()
rsi_h = h(rsi)
macd_hist_h = h(macd_hist)
cache = IndicatorCache(
df=df,
index_pos={ts: i for i, ts in enumerate(df.index)},
ema_fast=h(ema_fast),
ema_medium=h(ema_medium),
ema_slow=h(ema_slow),
ema_trend=h(ema_trend) if ema_trend is not None else None,
rsi=rsi_h,
rsi_series=pd.Series(rsi_h, index=df.index, name="rsi"),
macd_hist=macd_hist_h,
macd_hist_series=pd.Series(macd_hist_h, index=df.index, name="macd_hist"),
bb_mid=h(bb_mid),
bb_upper=h(bb_upper),
bb_lower=h(bb_lower),
bb_position=h(bb_position),
bb_band_width=h(bb_band_width),
bb_avg_width=h(bb_avg_width),
atr=h(atr),
adx=h(adx),
plus_di=h(plus_di),
minus_di=h(minus_di),
adx_slope=h(adx_slope),
roc=h(roc),
kc_mid=h(kc_mid),
kc_upper=h(kc_upper),
kc_lower=h(kc_lower),
kc_squeeze=h(kc_squeeze),
)
_assert_host_cache(cache)
return cache
# ═════════════════════════════════════════════════════════════════════
# GPU-AWARE TECHNICAL ENGINE (subclasses live engine, lookup-only)
# ═════════════════════════════════════════════════════════════════════
@@ -1330,6 +1512,15 @@ def simulate_trades(
closes_arr[i, :n] = sim_close[entry_pos:end_pos]
exit_times.append(list(sim_index[entry_pos:end_pos]))
if _gpu_on():
# One CUDA thread per trade, both exit modes (bit-identical twin of
# the two CPU paths below; Module 36 and the reference MD5s).
_resolve_trades_gpu(
trades, opens_arr, highs_arr, lows_arr, closes_arr,
valid_lens, exit_times, spread, trail_arm_r, trail_dist_r,
)
return
if trail_arm_r > 0:
_simulate_trailing(
trades, opens_arr, highs_arr, lows_arr, closes_arr,
@@ -1337,42 +1528,33 @@ def simulate_trades(
)
return
# GPU vectorized: for each trade, scan its window for first SL/TP hit.
# CPU static path, vectorized: for each trade, scan its window for the
# first SL/TP hit.
sls = np.array([t.sl for t in trades], dtype=np.float64)[:, None]
tps = np.array([t.tp1 for t in trades], dtype=np.float64)[:, None]
is_buy = np.array([t.direction == "BUY" for t in trades], dtype=bool)[:, None]
h_g = _to_gpu(highs_arr)
l_g = _to_gpu(lows_arr)
c_g = _to_gpu(closes_arr)
sls_g = _to_gpu(sls)
tps_g = _to_gpu(tps)
is_buy_g = _to_gpu(is_buy)
# Hit masks. For BUY: SL when low<=sl, TP when high>=tp.
# For SELL: SL when high>=sl, TP when low<=tp.
sl_hit_buy = l_g <= sls_g
tp_hit_buy = h_g >= tps_g
sl_hit_sell = h_g >= sls_g
tp_hit_sell = l_g <= tps_g
sl_hit_buy = lows_arr <= sls
tp_hit_buy = highs_arr >= tps
sl_hit_sell = highs_arr >= sls
tp_hit_sell = lows_arr <= tps
sl_hit = cp.where(is_buy_g, sl_hit_buy, sl_hit_sell)
tp_hit = cp.where(is_buy_g, tp_hit_buy, tp_hit_sell)
sl_hit = np.where(is_buy, sl_hit_buy, sl_hit_sell)
tp_hit = np.where(is_buy, tp_hit_buy, tp_hit_sell)
# Mask out NaN bars (no data past valid_lens).
valid_mask = ~cp.isnan(h_g)
valid_mask = ~np.isnan(highs_arr)
sl_hit = sl_hit & valid_mask
tp_hit = tp_hit & valid_mask
# First-hit indices (or max_bars if never).
idx_template = cp.arange(max_bars)[None, :]
sl_idx = cp.where(sl_hit, idx_template, max_bars)
tp_idx = cp.where(tp_hit, idx_template, max_bars)
first_sl = cp.min(sl_idx, axis=1)
first_tp = cp.min(tp_idx, axis=1)
sl_first_arr = _to_cpu(first_sl)
tp_first_arr = _to_cpu(first_tp)
idx_template = np.arange(max_bars)[None, :]
sl_idx = np.where(sl_hit, idx_template, max_bars)
tp_idx = np.where(tp_hit, idx_template, max_bars)
sl_first_arr = np.min(sl_idx, axis=1)
tp_first_arr = np.min(tp_idx, axis=1)
# Resolve each trade and compute realized MFE/MAE *up to exit bar*
# (a trade closed at bar 5 cannot have meaningful excursions at bar 12).
@@ -1447,6 +1629,60 @@ def simulate_trades(
t.mae_r = float(max(0.0, mae))
def _resolve_trades_gpu(
trades: List[Trade],
opens_arr: np.ndarray,
highs_arr: np.ndarray,
lows_arr: np.ndarray,
closes_arr: np.ndarray,
valid_lens: np.ndarray,
exit_times: List[List[pd.Timestamp]],
spread: float,
trail_arm_r: float,
trail_dist_r: float,
) -> None:
"""CUDA twin of `_simulate_trailing` (trail_arm_r > 0) and of the static
SL/TP1 branch of `simulate_trades`: the kernel walks every trade's window
in its own thread and returns exit index/price/reason, bars held, R and
the excursions; this host loop only writes them onto the Trade objects,
with the same 'not set' cases as the CPU code."""
trailing = trail_arm_r > 0
entries = np.array([t.entry_price for t in trades], dtype=np.float64)
sls = np.array([t.sl for t in trades], dtype=np.float64)
tps = np.array([t.tp1 for t in trades], dtype=np.float64)
is_buy = np.array([t.direction == "BUY" for t in trades], dtype=np.int8)
arms = np.array([t.trail_arm_r if t.trail_arm_r > 0 else trail_arm_r
for t in trades], dtype=np.float64)
dists = np.array([t.trail_dist_r if t.trail_dist_r > 0 else trail_dist_r
for t in trades], dtype=np.float64)
res = _gk.resolve_trades(
opens_arr, highs_arr, lows_arr, closes_arr, valid_lens,
entries, sls, tps, is_buy, arms, dists, spread, trailing)
for i, t in enumerate(trades):
code = int(res["reason"][i])
if code == _gk.REASON_NO_DATA:
t.exit_reason = "NO_DATA"
continue
bars = int(res["bars_held"][i])
exit_idx = int(res["exit_idx"][i])
if code == _gk.REASON_NO_DATA_AFTER_TIMEOUT:
# Static path: the TIMEOUT bookkeeping ran before the NaN close
# was seen, so bars_held and exit_time are kept and the price
# is None (never a NaN R in the book).
t.exit_reason = "NO_DATA"
t.bars_held = bars
t.exit_price = None
t.exit_time = exit_times[i][exit_idx]
continue
t.exit_reason = _gk.REASON_NAMES[code]
t.bars_held = bars
t.exit_price = float(res["exit_price"][i])
t.exit_time = exit_times[i][exit_idx]
t.r_multiple = float(res["r"][i])
t.mfe_r = float(res["mfe"][i])
t.mae_r = float(res["mae"][i])
# ═════════════════════════════════════════════════════════════════════
# BACKTEST RUNNER
# ═════════════════════════════════════════════════════════════════════
@@ -1634,16 +1870,16 @@ def _replay_span(span: Tuple[int, int]):
if report.signal_quality not in bt_cfg.quality_filter:
continue
# Strategy tag (from reasons[0] which the aggregator pre-pends).
tag = "GENERIC"
first_bull = report.bullish_reasons[0] if report.bullish_reasons else ""
first_bear = report.bearish_reasons[0] if report.bearish_reasons else ""
if "XAU PULLBACK" in first_bull or "XAU PULLBACK" in first_bear:
tag = "XAU_PB"
elif "XAU CHART PATTERN" in first_bull or "XAU CHART PATTERN" in first_bear:
tag = "XAU_CP"
elif "XAU STRATEGY" in first_bull:
tag = "XAU"
# Strategy tag: the generator's own stamp (XAU_ATR / XAU_PB /
# XAU_CP / GENERIC), the same string the live log stores in
# setup_json.strategy. Until 2026-09-17 this was inferred from the
# text of the FIRST reason ("XAU STRATEGY" -> "XAU"), which
# labelled every gold ATR setup whose first reason was another
# indicator — and every SELL — as GENERIC (live review §6.3).
# Label-only: entries, exits and R never depended on it, so the
# books are trade-identical; the CSV `strategy` column changed
# and the reference MD5s were re-pinned with that proof.
tag = str(getattr(report.trade_setup, "strategy", "") or "GENERIC")
if bt_cfg.strategies_only and tag not in bt_cfg.strategies_only:
continue
@@ -1850,6 +2086,11 @@ def run_backtest(
flt = platform_cfg.backtest
if flt.regime_filter and flt.regime_filter_timeframe not in timeframes:
timeframes.append(flt.regime_filter_timeframe)
# Same for the daily-trend veto: analyze_pair fetches D1 as a required
# frame when it is on, and the core reads it from the slices (the
# as-of slicing in _replay_span hands it closed daily bars only).
if daily_trend_needed(platform_cfg) and Timeframe.D1 not in timeframes:
timeframes.append(Timeframe.D1)
dfs_full: Dict[Timeframe, pd.DataFrame] = {}
# Which parquet each TF came from (recorded in --record's manifest so
# a study can prove two runs read the same bars).
@@ -1878,7 +2119,12 @@ def run_backtest(
tf.value: precompute_indicators(dfs_full[tf], platform_cfg.technical, gpu_log=True)
for tf in timeframes
}
logger.info(f"Indicator pre-compute took {time.perf_counter() - t0:.2f}s")
logger.info(
f"Indicator pre-compute took {time.perf_counter() - t0:.2f}s "
f"({'CuPy GPU' if _gpu_on() else 'NumPy CPU'})")
# The replay workers are forked below and must never touch the device:
# every cache array is already host NumPy; hand pooled memory back too.
BACKEND.release_device_memory()
# ── Inject GPU engine into a fresh aggregator ──
aggregator = SignalAggregator(platform_cfg)
@@ -1998,7 +2244,7 @@ def run_backtest(
trail_dist_r=platform_cfg.signals.trail_dist_r)
logger.info(
f"Trade simulation took {time.perf_counter() - t_sim:.2f}s "
f"(GPU={'yes' if _GPU_AVAILABLE else 'no'})"
f"({'CuPy GPU' if _gpu_on() else 'NumPy CPU'})"
)
if bt_cfg.record_dir:
@@ -2009,7 +2255,7 @@ def run_backtest(
n_primary_bars=len(primary_df), n_bars_in_window=n_bars,
skipped=skipped_reasons, n_candidates=len(candidates),
workers=workers,
backend="CuPy GPU" if _GPU_AVAILABLE else "NumPy CPU fallback",
backend=BACKEND.describe(),
cache_files=cache_names, replay_seconds=replay_seconds,
cache_fallbacks=cache_fallbacks),
sim_df=sim_df, bar_chunks=bar_chunks or None)
@@ -2378,7 +2624,9 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
"entries/stops/gates unchanged.")
p.add_argument("--quality", default="A+,A,B,C", help="Comma-separated allowed grades")
p.add_argument("--strategies", default="",
help="Comma-separated strategy tags to include (XAU,XAU_PB,XAU_CP,GENERIC). Empty = all.")
help="Comma-separated strategy tags to include "
"(XAU_ATR,XAU_PB,XAU_CP,GENERIC; the legacy name XAU "
"means XAU_ATR). Empty = all.")
p.add_argument("--cp", action="store_true",
help="Force-enable the chart-pattern entry (XAU_CP) regardless "
"of the factory default — for validating the candidate.")
@@ -2419,6 +2667,12 @@ def _parse_args(argv: Optional[List[str]] = None) -> argparse.Namespace:
help="Print full per-trade reasoning (indicators, signals, agreement, why entered)")
p.add_argument("--workers", type=int, default=0,
help="Replay worker processes (0=auto: cpu-2 capped at 12, 1=sequential)")
p.add_argument("--backend", choices=("auto", "gpu", "cpu"), default=None,
help="Compute backend for the indicator pre-compute and the "
"trade resolution: auto = CuPy when CUDA works, else "
"NumPy (the reason is logged); gpu = fail loudly without "
"CUDA; cpu = force the NumPy twins (parity runs). The "
"env var FB2_BACKEND sets the default.")
p.add_argument("--reasoning-samples", type=int, default=3,
help="If --detail not set, print this many sample trades' full reasoning")
p.add_argument("--log-level", default="INFO")
@@ -2484,6 +2738,8 @@ def main(argv: Optional[List[str]] = None) -> int:
level=args.log_level.upper(),
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
if args.backend:
BACKEND.set_mode(args.backend)
pair = _resolve_pair(args.pair)
primary_tf = _resolve_tf(args.primary_tf)
@@ -2566,7 +2822,8 @@ def main(argv: Optional[List[str]] = None) -> int:
max_hold_bars=args.max_hold_bars,
use_cache=not args.no_cache,
quality_filter=[q.strip() for q in args.quality.split(",") if q.strip()],
strategies_only=[s.strip() for s in args.strategies.split(",") if s.strip()],
strategies_only=[("XAU_ATR" if s.strip() == "XAU" else s.strip())
for s in args.strategies.split(",") if s.strip()],
spread=spread,
workers=args.workers,
fixed_rr_tp=args.fixed_rr_tp or 0.0,
@@ -2589,7 +2846,7 @@ def main(argv: Optional[List[str]] = None) -> int:
)
logger.info(
f"Backend: {'CuPy GPU' if _GPU_AVAILABLE else 'NumPy CPU fallback'} | "
f"Backend: {BACKEND.describe()} | "
f"Pair: {PAIR_INFO[pair]['display']} | Window: {start.date()} → {end.date()} | "
f"Confirm TFs: {','.join(tf.value for tf in confirm_tfs)} | Spread: {spread}"
)
+699
View File
@@ -0,0 +1,699 @@
"""
backtest/gpu_kernels.py — the CUDA side of the backtester.
Owns two things:
1. ``BACKEND`` — the one CuPy/CUDA state object the backtester consults.
It is probed lazily (never at import: test_all.py, analyze_signal_log.py
and replay_record import the backtester without wanting a CUDA context)
and it never fails silently: when CUDA is unusable the reason is kept in
``BACKEND.reason`` and printed in the ``Backend:`` log line. Before
2026-09-18 the probe swallowed "Failed to auto-detect CUDA root
directory" and every book was built on the NumPy path while the file
was called a GPU backtester.
CuPy from pip (cupy-cuda12x) needs the CUDA headers to JIT kernels. They
come from the ``nvidia-cuda-runtime-cu12`` wheel, whose splayed layout
has no single root, so CuPy cannot find them on its own: this module
points ``CUDA_PATH`` at that wheel when the variable is unset.
2. Three CUDA kernels, each a bit-for-bit replica of the CPU code it
replaces (proven on every cached window, see
backtest/research/gpu_backend_2026-09-18.md):
- ``ewm_batch`` — pandas ``ewm(alpha, min_periods, adjust=False)``
as implemented by ``gpu_backtest._gpu_ewm_alpha``;
one CUDA thread per series (the recurrence is
order-dependent, so a parallel scan would change
the last bits — that is not acceptable here).
- ``roll_mean_var`` — pandas 3.0.2 ``rolling(window).mean()`` and
``.var(ddof)`` (Kahan-compensated add/remove and
Welford with the numerically-unstable recompute),
one thread per series.
- ``resolve_trades``— ``gpu_backtest._simulate_trailing`` and the static
SL/TP1 branch of ``simulate_trades``, one thread per
trade.
Every kernel is compiled with ``--fmad=false`` so ``a*b + c`` rounds
twice exactly like the NumPy/Python code, and every kernel object lives
in a module global (a forked worker must never drop the last reference
and unload a CUDA module it did not create). The backtester's forked
replay workers never touch the device: precompute results are moved to
host NumPy arrays before the fork.
Import as ``from backtest import gpu_kernels`` (package form — test_all.py
puts backtest/ on sys.path and imports the backtester under two names; the
backend state must live in one module object).
"""
from __future__ import annotations
import contextlib
import importlib.util
import logging
import os
import sys
from typing import Dict, List, Optional, Sequence
import numpy as np
logger = logging.getLogger("gpu_backtest.kernels")
ENV_BACKEND = "FB2_BACKEND" # auto | gpu | cpu
KERNEL_OPTIONS = ("--fmad=false",) # no FMA contraction: two roundings, like the CPU
_MODES = ("auto", "gpu", "cpu")
# Exit-reason codes written by resolve_trades (see the host mapping in
# gpu_backtest._resolve_trades_gpu).
REASON_NO_DATA = 0
REASON_SL = 1
REASON_TRAIL = 2
REASON_TP1 = 3
REASON_TIMEOUT = 4
REASON_NO_DATA_AFTER_TIMEOUT = 5 # static path: bars_held/exit_time set, price None
REASON_NAMES = {
REASON_NO_DATA: "NO_DATA", REASON_SL: "SL", REASON_TRAIL: "TRAIL",
REASON_TP1: "TP1", REASON_TIMEOUT: "TIMEOUT",
REASON_NO_DATA_AFTER_TIMEOUT: "NO_DATA",
}
# ═════════════════════════════════════════════════════════════════════
# CUDA root discovery for pip-installed CuPy
# ═════════════════════════════════════════════════════════════════════
def _cuda_runtime_wheel_dir() -> Optional[str]:
"""Directory of the nvidia-cuda-runtime-cu12 wheel (holds include/)."""
try:
spec = importlib.util.find_spec("nvidia.cuda_runtime")
except (ImportError, ValueError):
spec = None
candidates: List[str] = []
if spec is not None and spec.submodule_search_locations:
candidates.extend(spec.submodule_search_locations)
try:
import site
for sp in list(site.getsitepackages()) + [site.getusersitepackages()]:
candidates.append(os.path.join(sp, "nvidia", "cuda_runtime"))
except Exception:
pass
for d in candidates:
if os.path.isfile(os.path.join(d, "include", "cuda_runtime.h")):
return d
return None
def ensure_cuda_path() -> str:
"""Point CUDA_PATH at the runtime-header wheel when it is unset.
Returns a short note for the log ("" when nothing was changed). Runs at
import (cheap: no CuPy import, no CUDA context) so that any later kernel
compile — including one triggered by a test that imports cupy itself —
finds the headers.
"""
current = os.environ.get("CUDA_PATH", "")
if current and os.path.isdir(current):
return ""
wheel = _cuda_runtime_wheel_dir()
if wheel is None:
return ""
os.environ["CUDA_PATH"] = wheel
# CuPy caches a failed lookup; if it was imported before this ran, drop
# that cache so the new CUDA_PATH is seen (private, hence guarded).
env_mod = sys.modules.get("cupy._environment")
if env_mod is not None and getattr(env_mod, "_cuda_path", "") is None:
env_mod._cuda_path = ""
return f"CUDA_PATH set to the nvidia-cuda-runtime wheel: {wheel}"
_CUDA_PATH_NOTE = ensure_cuda_path()
# ═════════════════════════════════════════════════════════════════════
# Backend state
# ═════════════════════════════════════════════════════════════════════
_PROBE_SRC = r'''
extern "C" __global__
void fb2_probe(const double* __restrict__ x, double* __restrict__ y, const int n) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n) y[i] = x[i] * 2.0 + 1.0;
}
'''
class Backend:
"""CuPy availability, selection mode and the device description."""
def __init__(self) -> None:
mode = os.environ.get(ENV_BACKEND, "auto").strip().lower() or "auto"
if mode not in _MODES:
logger.warning("%s=%r is not one of %s; using auto", ENV_BACKEND, mode, _MODES)
mode = "auto"
self.mode = mode
self._probed = False
self._ok = False
self.reason = ""
self.device_name = ""
self.cupy_version = ""
self.cp = None # the cupy module once probed OK
# ── selection ────────────────────────────────────────────────────
def set_mode(self, mode: str) -> None:
mode = (mode or "auto").strip().lower()
if mode not in _MODES:
raise ValueError(f"backend mode must be one of {_MODES}, got {mode!r}")
self.mode = mode
if mode == "gpu" and not self.available:
raise RuntimeError(
f"--backend gpu requested but CUDA is unavailable: {self.reason}")
def force_cpu(self) -> None:
self.mode = "cpu"
@contextlib.contextmanager
def cpu_only(self):
"""Temporarily route everything to the NumPy twins (tests)."""
old = self.mode
self.mode = "cpu"
try:
yield
finally:
self.mode = old
@property
def available(self) -> bool:
if self.mode == "cpu":
return False
self._probe()
return self._ok
def describe(self) -> str:
"""The text after ``Backend:`` in the run log and the --record manifest.
Prefixes are load-bearing: "CuPy GPU" / "NumPy CPU fallback" are what
every report and the Module 33 manifest fixture use."""
if self.available:
return f"CuPy GPU ({self.device_name}, cupy {self.cupy_version})"
if self.mode == "cpu":
return f"NumPy CPU fallback (forced: --backend cpu / {ENV_BACKEND}=cpu)"
return f"NumPy CPU fallback ({self.reason})"
# ── probe ────────────────────────────────────────────────────────
def _probe(self) -> None:
if self._probed:
return
self._probed = True
try:
import cupy as cp # type: ignore
a = cp.asarray(np.zeros(4, dtype=np.float64))
_ = cp.cumsum(a).get()
# The compile probe is the step that used to fail: an importable
# CuPy still cannot JIT without the CUDA headers.
k = cp.RawKernel(_PROBE_SRC, "fb2_probe", options=KERNEL_OPTIONS)
out = cp.empty(4, dtype=np.float64)
k((1,), (4,), (a, out, np.int32(4)))
cp.cuda.Device().synchronize()
if not np.array_equal(out.get(), np.ones(4)):
raise RuntimeError("probe kernel returned wrong values")
props = cp.cuda.runtime.getDeviceProperties(cp.cuda.Device().id)
name = props.get("name", b"")
if isinstance(name, bytes):
name = name.decode("utf-8", "replace")
self.cp = cp
self._ok = True
self.device_name = str(name)
self.cupy_version = str(cp.__version__)
logger.info("CUDA backend ready: %s, cupy %s%s", self.device_name,
self.cupy_version,
f" ({_CUDA_PATH_NOTE})" if _CUDA_PATH_NOTE else "")
except Exception as e: # noqa: BLE001 — any failure means CPU, loudly
first = str(e).strip().splitlines()[0] if str(e).strip() else ""
self.reason = f"{type(e).__name__}: {first}"[:240]
self._ok = False
logger.warning("CUDA backend unavailable, NumPy CPU fallback: %s", self.reason)
def release_device_memory(self) -> None:
"""Synchronise and return pooled device/pinned memory before a fork."""
if not self._ok or self.cp is None:
return
cp = self.cp
cp.cuda.Device().synchronize()
cp.get_default_memory_pool().free_all_blocks()
cp.get_default_pinned_memory_pool().free_all_blocks()
BACKEND = Backend()
def _require_cp():
if not BACKEND.available:
raise RuntimeError(f"CUDA backend unavailable: {BACKEND.reason or BACKEND.mode}")
return BACKEND.cp
# ═════════════════════════════════════════════════════════════════════
# Kernel sources
# ═════════════════════════════════════════════════════════════════════
# Replica of gpu_backtest._gpu_ewm_alpha: y[i] = alpha*x[i] + (1-alpha)*y[i-1],
# seeded at the first non-NaN x; a NaN input carries y forward without
# advancing the non-NaN counter; output is NaN until min_periods non-NaN
# inputs were seen. Rounding intrinsics pin the two-multiply-one-add order.
_EWM_SRC = r'''
extern "C" __global__
void fb2_ewm_batch(const double* __restrict__ x, double* __restrict__ y,
const long long* __restrict__ offs, const long long* __restrict__ lens,
const double* __restrict__ alphas, const int* __restrict__ minp,
const int n_series) {
int s = blockIdx.x * blockDim.x + threadIdx.x;
if (s >= n_series) return;
const double alpha = alphas[s];
const double one_minus = __dsub_rn(1.0, alpha);
const int mp = minp[s];
const double* xs = x + offs[s];
double* ys = y + offs[s];
const long long n = lens[s];
double last_y = nan("");
int seen = 0;
for (long long i = 0; i < n; ++i) {
const double xi = xs[i];
if (isnan(xi)) {
ys[i] = (!isnan(last_y) && seen >= mp) ? last_y : nan("");
continue;
}
if (isnan(last_y)) last_y = xi;
else last_y = __dadd_rn(__dmul_rn(alpha, xi), __dmul_rn(one_minus, last_y));
seen += 1;
ys[i] = (seen >= mp) ? last_y : nan("");
}
}
'''
# Replica of pandas 3.0.2 pandas/_libs/window/aggregations.pyx roll_mean and
# roll_var for a fixed window, closed='right', min_periods=window (the
# defaults of Series.rolling(window)). Kahan-compensated running sum with
# the sign count and consecutive-same-value guard (GH#42064), Welford's
# online variance with Kahan compensation and the InvCondTol recompute.
_ROLL_SRC = r'''
#define FB2_EPS 2.220446049250313e-16
#define FB2_INVCONDTOL (FB2_EPS * 1e3)
__device__ __forceinline__ void fb2_add_mean(double val, long long* nobs, double* sum_x,
long long* neg_ct, double* comp, long long* ncsv, double* prev_value) {
if (val == val) {
*nobs += 1;
double y = val - *comp;
double t = *sum_x + y;
*comp = t - *sum_x - y;
*sum_x = t;
if (signbit(val)) *neg_ct += 1;
if (val == *prev_value) *ncsv += 1; else *ncsv = 1;
*prev_value = val;
}
}
__device__ __forceinline__ void fb2_remove_mean(double val, long long* nobs, double* sum_x,
long long* neg_ct, double* comp) {
if (val == val) {
*nobs -= 1;
double y = -val - *comp;
double t = *sum_x + y;
*comp = t - *sum_x - y;
*sum_x = t;
if (signbit(val)) *neg_ct -= 1;
}
}
__device__ __forceinline__ void fb2_add_var(double val, double* nobs, double* mean_x,
double* ssqdm_x, double* comp, bool* unstable) {
if (val != val) return;
double prev_m2 = *ssqdm_x;
*nobs += 1;
double prev_mean = *mean_x - *comp;
double y = val - *comp;
double t = y - *mean_x;
*comp = t + *mean_x - y;
double delta = t;
if (*nobs != 0) *mean_x = *mean_x + delta / *nobs; else *mean_x = 0;
*ssqdm_x = *ssqdm_x + (val - prev_mean) * (val - *mean_x);
if (prev_m2 * FB2_INVCONDTOL > *ssqdm_x) *unstable = true;
}
__device__ __forceinline__ void fb2_remove_var(double val, double* nobs, double* mean_x,
double* ssqdm_x, double* comp, bool* unstable) {
if (val == val) {
double prev_m2 = *ssqdm_x;
*nobs -= 1;
if (*nobs != 0) {
double prev_mean = *mean_x - *comp;
double y = val - *comp;
double t = y - *mean_x;
*comp = t + *mean_x - y;
double delta = t;
*mean_x = *mean_x - delta / *nobs;
*ssqdm_x = *ssqdm_x - (val - prev_mean) * (val - *mean_x);
if (prev_m2 * FB2_INVCONDTOL > *ssqdm_x) *unstable = true;
} else {
*mean_x = 0; *ssqdm_x = 0; *unstable = false;
}
}
}
extern "C" __global__
void fb2_roll_mean_var(const double* __restrict__ x, double* __restrict__ out_mean,
double* __restrict__ out_var,
const long long* __restrict__ offs, const long long* __restrict__ lens,
const int n_series, const int window, const int ddof) {
int s = blockIdx.x * blockDim.x + threadIdx.x;
if (s >= n_series) return;
const double* xs = x + offs[s];
double* om = out_mean + offs[s];
double* ov = out_var + offs[s];
const long long n = lens[s];
if (n <= 0) return;
const long long minp = window; // min_periods defaults to the window
// roll_mean state
double comp_add = 0, comp_rem = 0, sum_x = 0, prev_value = xs[0];
long long nobs = 0, neg_ct = 0, ncsv = 0;
// roll_var state (nobs is a double in pandas' roll_var)
double vnobs = 0, mean_x = 0, ssqdm_x = 0, vcomp_add = 0, vcomp_rem = 0;
bool unstable = false;
for (long long i = 0; i < n; ++i) {
long long st = i - window + 1; if (st < 0) st = 0; // start[i]
long long en = i + 1; // end[i]
long long pst = (i - 1) - window + 1; if (pst < 0) pst = 0; // start[i-1]
long long pen = i; // end[i-1]
// ---- mean
if (i == 0) {
comp_add = comp_rem = sum_x = 0; nobs = neg_ct = 0; prev_value = xs[st]; ncsv = 0;
for (long long j = st; j < en; ++j)
fb2_add_mean(xs[j], &nobs, &sum_x, &neg_ct, &comp_add, &ncsv, &prev_value);
} else {
for (long long j = pst; j < st; ++j)
fb2_remove_mean(xs[j], &nobs, &sum_x, &neg_ct, &comp_rem);
for (long long j = pen; j < en; ++j)
fb2_add_mean(xs[j], &nobs, &sum_x, &neg_ct, &comp_add, &ncsv, &prev_value);
}
double rm;
if (nobs >= minp && nobs > 0) {
rm = sum_x / (double)nobs;
if (ncsv >= nobs) rm = prev_value;
else if (neg_ct == 0 && rm < 0) rm = 0;
else if (neg_ct == nobs && rm > 0) rm = 0;
} else {
rm = nan("");
}
om[i] = rm;
// ---- var
bool recompute = (i == 0);
if (!recompute) {
for (long long j = pst; j < st; ++j)
fb2_remove_var(xs[j], &vnobs, &mean_x, &ssqdm_x, &vcomp_rem, &unstable);
for (long long j = pen; j < en; ++j)
fb2_add_var(xs[j], &vnobs, &mean_x, &ssqdm_x, &vcomp_add, &unstable);
}
if (recompute || unstable) {
mean_x = ssqdm_x = vnobs = vcomp_add = vcomp_rem = 0;
for (long long j = st; j < en; ++j)
fb2_add_var(xs[j], &vnobs, &mean_x, &ssqdm_x, &vcomp_add, &unstable);
unstable = false;
}
ov[i] = (vnobs >= (double)minp && vnobs > (double)ddof)
? ssqdm_x / (vnobs - (double)ddof) : nan("");
}
}
'''
# Replica of gpu_backtest._simulate_trailing (trailing != 0) and of the
# static SL/TP1 branch of gpu_backtest.simulate_trades (trailing == 0).
# Python max/min are written out as (b > a) ? b : a so NaN handling matches.
_TRADES_SRC = r'''
#define R_NO_DATA 0
#define R_SL 1
#define R_TRAIL 2
#define R_TP1 3
#define R_TIMEOUT 4
#define R_NO_DATA_AFTER_TIMEOUT 5
__device__ __forceinline__ void fb2_excursions(const double* h, const double* l, long long bars,
bool buy, double entry, double risk, double* mfe_out, double* mae_out) {
// np.nanmax(h[:bars]) / np.nanmin(l[:bars]); all-NaN gives NaN.
double hmax = nan(""), lmin = nan("");
for (long long j = 0; j < bars; ++j) {
double hv = h[j], lv = l[j];
if (!isnan(hv) && (isnan(hmax) || hv > hmax)) hmax = hv;
if (!isnan(lv) && (isnan(lmin) || lv < lmin)) lmin = lv;
}
double mfe, mae;
if (buy) { mfe = (hmax - entry) / risk; mae = (entry - lmin) / risk; }
else { mfe = (entry - lmin) / risk; mae = (hmax - entry) / risk; }
*mfe_out = (mfe > 0.0) ? mfe : 0.0; // Python max(0.0, x): NaN -> 0.0
*mae_out = (mae > 0.0) ? mae : 0.0;
}
extern "C" __global__
void fb2_resolve_trades(const double* __restrict__ opens, const double* __restrict__ highs,
const double* __restrict__ lows, const double* __restrict__ closes,
const long long* __restrict__ valid_lens,
const double* __restrict__ entries, const double* __restrict__ sls,
const double* __restrict__ tps, const signed char* __restrict__ is_buy,
const double* __restrict__ arms, const double* __restrict__ dists,
const double spread, const int trailing, const int max_bars,
const int n_trades,
long long* __restrict__ exit_idx_out, double* __restrict__ exit_price_out,
int* __restrict__ reason_out, long long* __restrict__ bars_held_out,
double* __restrict__ r_out, double* __restrict__ mfe_out,
double* __restrict__ mae_out) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= n_trades) return;
const long long base = (long long)i * max_bars;
const double* o = opens + base;
const double* h = highs + base;
const double* l = lows + base;
const double* c = closes + base;
const long long n = valid_lens[i];
const bool buy = is_buy[i] != 0;
const double entry = entries[i];
const double sl = sls[i];
const double tp = tps[i];
// defaults (Trade dataclass defaults / "not set")
exit_idx_out[i] = -1; exit_price_out[i] = nan(""); reason_out[i] = R_NO_DATA;
bars_held_out[i] = 0; r_out[i] = 0.0; mfe_out[i] = 0.0; mae_out[i] = 0.0;
if (n == 0) return;
if (trailing) {
const double risk = fabs(entry - sl);
if (risk <= 0) return; // NO_DATA
const double arm = arms[i], dist = dists[i];
double cur_sl = sl, peak = entry;
bool have_price = false; double exit_price = nan("");
long long exit_idx = n - 1; int reason = R_TIMEOUT;
for (long long j = 0; j < n; ++j) {
if (isnan(h[j])) {
if (j == 0) { exit_idx = 0; reason = R_NO_DATA; }
else { exit_idx = j - 1; exit_price = c[exit_idx]; have_price = true; }
break;
}
const bool hit = buy ? (l[j] <= cur_sl) : (h[j] >= cur_sl);
if (hit) {
exit_price = cur_sl; have_price = true;
if ((buy && o[j] < cur_sl) || (!buy && o[j] > cur_sl)) exit_price = o[j];
reason = (cur_sl == sl) ? R_SL : R_TRAIL;
exit_idx = j;
break;
}
const double fav = buy ? h[j] : l[j];
peak = buy ? ((fav > peak) ? fav : peak) : ((fav < peak) ? fav : peak);
const double exc = (buy ? (peak - entry) : (entry - peak)) / risk;
if (exc >= arm) {
const double lvl = buy ? (peak - dist * risk) : (peak + dist * risk);
cur_sl = buy ? ((lvl > cur_sl) ? lvl : cur_sl) : ((lvl < cur_sl) ? lvl : cur_sl);
}
}
if (reason == R_NO_DATA) return;
if (!have_price) exit_price = c[exit_idx];
if (!isfinite(exit_price)) return; // NO_DATA, nothing else set
const long long bars = exit_idx + 1;
reason_out[i] = reason; bars_held_out[i] = bars; exit_idx_out[i] = exit_idx;
exit_price_out[i] = exit_price;
const double pnl = buy ? (exit_price - entry) : (entry - exit_price);
r_out[i] = (pnl - spread) / risk;
fb2_excursions(h, l, bars, buy, entry, risk, &mfe_out[i], &mae_out[i]);
return;
}
// ---- static SL / TP1: first-hit scan over the whole window, NaN highs masked
long long sl_idx = max_bars, tp_idx = max_bars;
for (long long j = 0; j < max_bars; ++j) {
if (isnan(h[j])) continue;
const bool sl_hit = buy ? (l[j] <= sl) : (h[j] >= sl);
const bool tp_hit = buy ? (h[j] >= tp) : (l[j] <= tp);
if (sl_hit && j < sl_idx) sl_idx = j;
if (tp_hit && j < tp_idx) tp_idx = j;
}
long long exit_idx; double exit_price; int reason; long long bars;
if (sl_idx < n && sl_idx <= tp_idx) {
reason = R_SL; bars = sl_idx + 1; exit_price = sl; exit_idx = sl_idx;
const double bar_open = o[sl_idx];
if (!isnan(bar_open)) {
if ((buy && bar_open < sl) || (!buy && bar_open > sl)) exit_price = bar_open;
}
} else if (tp_idx < n) {
reason = R_TP1; bars = tp_idx + 1; exit_price = tp; exit_idx = tp_idx;
const double bar_open = o[tp_idx];
if (!isnan(bar_open)) {
if ((buy && bar_open > tp) || (!buy && bar_open < tp)) exit_price = bar_open;
}
} else {
reason = R_TIMEOUT; bars = n; exit_idx = n - 1; exit_price = c[exit_idx];
if (!isfinite(exit_price)) {
reason_out[i] = R_NO_DATA_AFTER_TIMEOUT; bars_held_out[i] = bars;
exit_idx_out[i] = exit_idx;
return; // exit_price None, R untouched
}
}
reason_out[i] = reason; bars_held_out[i] = bars; exit_idx_out[i] = exit_idx;
exit_price_out[i] = exit_price;
const double risk = fabs(entry - sl);
if (risk > 0) {
const double pnl = buy ? (exit_price - entry) : (entry - exit_price);
r_out[i] = (pnl - spread) / risk;
}
if (bars > 0 && risk > 0)
fb2_excursions(h, l, bars, buy, entry, risk, &mfe_out[i], &mae_out[i]);
}
'''
_KERNELS: Dict[str, object] = {} # module-level: never collected in a child
def _kernel(name: str):
cp = _require_cp()
k = _KERNELS.get(name)
if k is None:
src = {"ewm": _EWM_SRC, "roll": _ROLL_SRC, "trades": _TRADES_SRC}[name]
fn = {"ewm": "fb2_ewm_batch", "roll": "fb2_roll_mean_var",
"trades": "fb2_resolve_trades"}[name]
k = cp.RawKernel(src, fn, options=KERNEL_OPTIONS)
_KERNELS[name] = k
return k
def _ragged(cp, series: Sequence) -> tuple:
"""Concatenate 1-D float64 series on the device; return (flat, offs, lens)."""
arrs = [cp.asarray(s, dtype=cp.float64).ravel() for s in series]
lens = np.array([int(a.shape[0]) for a in arrs], dtype=np.int64)
offs = np.zeros(len(arrs), dtype=np.int64)
if len(arrs) > 1:
offs[1:] = np.cumsum(lens)[:-1]
flat = cp.concatenate(arrs) if arrs else cp.empty(0, dtype=cp.float64)
return flat, cp.asarray(offs), cp.asarray(lens), offs, lens
def _launch_1d(n: int, threads: int = 128) -> tuple:
return ((max(1, (n + threads - 1) // threads),), (threads,))
# ═════════════════════════════════════════════════════════════════════
# Public wrappers (device in/out; callers .get() once)
# ═════════════════════════════════════════════════════════════════════
def ewm_batch_dev(series: Sequence, alphas: Sequence[float],
min_periods: Sequence[int]) -> List:
"""EWM of every series in one launch. Inputs may be NumPy or CuPy;
returns CuPy views (one per series)."""
cp = _require_cp()
if len(series) != len(alphas) or len(series) != len(min_periods):
raise ValueError("series, alphas and min_periods must have equal length")
if not series:
return []
flat, d_offs, d_lens, offs, lens = _ragged(cp, series)
out = cp.empty_like(flat)
k = _kernel("ewm")
grid, block = _launch_1d(len(series))
k(grid, block, (flat, out, d_offs, d_lens,
cp.asarray(np.asarray(alphas, dtype=np.float64)),
cp.asarray(np.asarray(min_periods, dtype=np.int32)),
np.int32(len(series))))
return [out[o:o + n] for o, n in zip(offs, lens)]
def ewm_batch(series: Sequence[np.ndarray], alphas: Sequence[float],
min_periods: Sequence[int]) -> List[np.ndarray]:
"""Host convenience: NumPy in, NumPy out."""
return [y.get() for y in ewm_batch_dev(series, alphas, min_periods)]
def roll_mean_var_dev(series: Sequence, window: int, ddof: int = 0) -> tuple:
"""pandas rolling(window).mean() and .var(ddof) of every series in one
launch; returns (means, vars) as lists of CuPy views."""
cp = _require_cp()
if not series:
return [], []
flat, d_offs, d_lens, offs, lens = _ragged(cp, series)
out_mean = cp.empty_like(flat)
out_var = cp.empty_like(flat)
k = _kernel("roll")
grid, block = _launch_1d(len(series))
k(grid, block, (flat, out_mean, out_var, d_offs, d_lens,
np.int32(len(series)), np.int32(window), np.int32(ddof)))
means = [out_mean[o:o + n] for o, n in zip(offs, lens)]
vars_ = [out_var[o:o + n] for o, n in zip(offs, lens)]
return means, vars_
def roll_std_from_var_dev(var):
"""pandas zsqrt: sqrt(var) with a negative variance clipped to 0."""
cp = _require_cp()
return cp.where(var < 0, 0.0, cp.sqrt(var))
def roll_mean_var(series: Sequence[np.ndarray], window: int, ddof: int = 0) -> tuple:
"""Host convenience: (means, vars, stds) as NumPy arrays."""
means, vars_ = roll_mean_var_dev(series, window, ddof)
stds = [roll_std_from_var_dev(v) for v in vars_]
return ([m.get() for m in means], [v.get() for v in vars_],
[s.get() for s in stds])
def resolve_trades(opens: np.ndarray, highs: np.ndarray, lows: np.ndarray,
closes: np.ndarray, valid_lens: np.ndarray,
entries: np.ndarray, sls: np.ndarray, tps: np.ndarray,
is_buy: np.ndarray, arms: np.ndarray, dists: np.ndarray,
spread: float, trailing: bool) -> Dict[str, np.ndarray]:
"""Resolve every trade's exit on the device; one thread per trade.
``opens/highs/lows/closes`` are (n_trades, max_bars) float64 windows
padded with NaN, ``valid_lens`` the real length of each. Returns host
arrays: exit_idx, exit_price, reason (REASON_* codes), bars_held, r,
mfe, mae.
"""
cp = _require_cp()
n_trades, max_bars = opens.shape
if n_trades == 0:
empty = np.zeros(0)
return {k: empty for k in ("exit_idx", "exit_price", "reason",
"bars_held", "r", "mfe", "mae")}
d = lambda a, dt: cp.asarray(np.ascontiguousarray(a, dtype=dt)) # noqa: E731
d_o, d_h, d_l, d_c = (d(opens, np.float64), d(highs, np.float64),
d(lows, np.float64), d(closes, np.float64))
o_exit_idx = cp.empty(n_trades, dtype=np.int64)
o_exit_price = cp.empty(n_trades, dtype=np.float64)
o_reason = cp.empty(n_trades, dtype=np.int32)
o_bars = cp.empty(n_trades, dtype=np.int64)
o_r = cp.empty(n_trades, dtype=np.float64)
o_mfe = cp.empty(n_trades, dtype=np.float64)
o_mae = cp.empty(n_trades, dtype=np.float64)
k = _kernel("trades")
grid, block = _launch_1d(n_trades)
k(grid, block, (d_o, d_h, d_l, d_c, d(valid_lens, np.int64),
d(entries, np.float64), d(sls, np.float64), d(tps, np.float64),
d(is_buy, np.int8), d(arms, np.float64), d(dists, np.float64),
np.float64(spread), np.int32(1 if trailing else 0),
np.int32(max_bars), np.int32(n_trades),
o_exit_idx, o_exit_price, o_reason, o_bars, o_r, o_mfe, o_mae))
cp.cuda.Device().synchronize()
return dict(exit_idx=o_exit_idx.get(), exit_price=o_exit_price.get(),
reason=o_reason.get(), bars_held=o_bars.get(), r=o_r.get(),
mfe=o_mfe.get(), mae=o_mae.get())
+50
View File
@@ -38,3 +38,53 @@ re-implementation.
- Scorecards and "what-if" tables from these files (backtest/replay_scorecard.py) are IN-BOOK
descriptive statistics: they cannot add trades a change would create and ignore the gates.
Any weight / indicator change they suggest is a hypothesis for a full rule-1 rerun, never a decision.
--- 2026-09-18: GPU backend (backtest/gpu_kernels.py) ---
- The "Backend:" line now names the CUDA device ("CuPy GPU (<device>, cupy <version>)") or the
reason for the NumPy fallback; quote it verbatim. `--backend cpu` forces the NumPy twins, `gpu`
refuses to fall back. Books are byte-identical on both backends (reference MD5s, CLAUDE.md §9).
- Any change to a kernel or to its CPU twin is a parity event: re-run Module 36, regenerate the
five reference books on BOTH backends and show the ten MD5s unchanged. Never replace a kernel
with a differently ordered algorithm (parallel scan, two-pass variance) even if it is faster.
--- 2026-09-18: CUDA research twin (backtest/cudaBasedBacktesting.py) ---
- Exemption from "replays go through the real signal code" above, granted by the owner for this
ONE research script: its per-bar decision is a CUDA re-implementation of engine.py,
signal_aggregator.py and chart_patterns.py (backtest/cuda_signal_kernels.py). gpu_backtest.py
stays the base reference and live scans never use the twin.
- Its numbers count as evidence only while parity holds. After ANY change to engine.py,
signal_aggregator.py, chart_patterns.py, position_sizing.py, the recorder's row layout or the
GPUTechnicalEngine overrides: mirror the change in cuda_signal_kernels.py in the same commit,
keep test_all.py Module 37 green, and regenerate at least the M15 OOS book (2024-06-01 ->
2025-03-30, $0.65) with BOTH scripts and show equal MD5s before quoting a CUDA-script result.
- A report built from the twin quotes both scripts' "Backend:" lines (the twin's necessarily
starts with "CuPy GPU (") and the paired MD5 of at least one book from the reference script on
the same code and cache. Whole-window and half-window figures, costs and breakdowns are then
read from the twin's book exactly as from the reference's; the CSVs are identical.
- The twin has no CPU path (--backend cpu / FB2_BACKEND=cpu are refused), --workers is ignored,
and it refuses configurations beyond its compile-time limits (more than 4 confirmation
timeframes, divergence lookback >= 64, swing lookback >= 256, cp_lookback >= 128) with a
message that points back to gpu_backtest.py.
--- 2026-09-30: daily-trend veto (M15) and the D1 frame ---
- The M15 factory config carries xau_strategy.daily_trend_veto = "both" (M15 override 7). Both
backtesters then load the D1 frame as an EXTRA frame next to the primary and confirmation
frames (the "Confirm TFs:" line does not list it), cached under its own pad:
backtest/cache/oanda_XAU_USD_1d_<start minus 1400 days>_<end>.parquet. A new M15 window therefore
needs that file (one network fetch the first time; "GPU cache fallbacks: 0" is unaffected).
- The veto reads CLOSED daily bars only (a bar labelled D counts from D + 1 day <= the primary
bar's close), drops the provider's Sunday stub bars, and removes a finished setup that goes
against a non-zero state. Definition and constants: signal_aggregator.daily_trend_series.
- Research switches, both default-neutral elsewhere: --set xau_strategy.daily_trend_veto=
off|both|buy|sell and --set xau_strategy.wide_stop_cap_atr=K (K = 0 is off; the cap FAILED its
rule on 2026-09-30 and ships nowhere). A control for an M15 candidate is the factory config,
i.e. WITH the veto; to reproduce a pre-2026-09-30 M15 book pass --set
xau_strategy.daily_trend_veto=off.
- A candidate that is a FINAL veto (it only removes finished setups) may be screened on a research
harness (K1 + K2 of the twin, a mask on the final candidates before the cooldown, the shared
resolver) once that harness is shown to reproduce a shipped book byte for byte; the formal
numbers still come from the real code path with the paired gpu_backtest.py MD5.
- The 2026-09-30 report (backtest/research/daily_trend_campaign_2026-09-30.md) is the template for
a risk-first rule: rules, windows and neighbours written before the runs; the long window
(2019-01-01 -> 2026-08-07) and an unseen holdout as clauses; the rising-market cost reported,
not hidden.
@@ -0,0 +1,235 @@
# Context screens after the daily-trend veto — declared before looking, 2026-10-01
Follow-up to `daily_trend_campaign_2026-09-30.md` (owner decision 2026-09-30: "more context
screens"). The daily chart was information the engine had never seen, and it paid on M15. This
file asks the same question of seven more pieces of context. Sections 1 to 3 were written before
any number in section 4 existed.
## 1. Books and slices
The books as shipped on 2026-09-30, `--spread 0.65`, CUDA twin (byte-identical to
`gpu_backtest.py` on the paired runs of the campaign report):
| Book | Window | Trades / R | MD5 |
|---|---|---|---|
| M15 LONG (veto on) | 2019-01-01 → 2026-08-07 | 1473 / +223.33 | `aa5253aa…` |
| M15 W3 (veto on) | 2023-09-05 → 2026-09-05 | 824 / +205.12 | `ad3b0dbd…` |
| M5 LONG | 2019-01-01 → 2026-08-07 | 1052 / +128.10 | `3f0c091c…` |
| M5 W3 | 2023-09-05 → 2026-09-05 | 870 / +153.64 | `d20ec8a3…` |
Four slices per timeframe: LONG era A (entries 2019–2023), LONG era B (2024 → 2026-08-07), W3
first half (before 2025-03-05), W3 second half. W3 overlaps era B, so the four slices are not
independent; that is known and accepted for a screen.
## 2. The seven features (closed bars only, as of the trade's decision time)
Decision time = the trade's entry time (the entry is the open of the bar after the signal bar, so
it equals the signal bar's close). A bar counts only when it had closed by then.
Daily bars: provider D1 candles (UTC days), Sunday stub bars dropped, usable from label + 1 day.
1. **Weekly trend.** Weekly bars = the kept daily bars grouped into weeks ending Friday; a week is
usable once its Friday bar is. State +1 when the weekly close is above its 20-week EMA
(`ewm(span=20, adjust=False)`) AND above the close 12 weeks earlier; −1 when below both; else 0.
Trade state: `with` / `against` / `unclear`.
2. **Four-hour trend.** Provider H4 candles (UTC-aligned), usable from label + 4 h. State +1 when
the close is above its 50-bar EMA AND above the close 30 bars earlier; −1 when below both; else
0. Trade state: `with` / `against` / `unclear`.
3. **Daily-trend strength.** `(Close − Close 60 kept days earlier) / ATR14`, ATR14 = the simple
14-day mean of the true range, taken in the trade's direction (multiplied by −1 for a SELL).
Trade state by fixed cut-offs: `counter` (< 0), `weak` (0 to 5), `medium` (5 to 12),
`strong` (> 12 daily ATRs).
4. **Daily volatility state.** ATR14 / Close, ranked within its own trailing 250 kept days
(percentile, the current day included). Trade state: `quiet` (< 33%), `normal`, `wild` (> 67%).
5. **Place in the 20-day range.** `(Close − lowest Low of 20 kept days) / (highest High − lowest
Low)`, mirrored for a SELL (1 − value). Trade state: `near the far edge` (< 1/3: buying near
the 20-day low or selling near the high), `middle`, `near the chasing edge` (> 2/3: buying near
the 20-day high or selling near the low).
6. **Tick-volume surge.** Volume of the signal bar divided by the median volume of the same clock
slot (same hour and minute, UTC) over that slot's previous 20 bars. Trade state: `thin`
(< 0.8), `usual` (0.8 to 1.5), `surge` (> 1.5).
7. **Time to a high-impact US release** (`backtest/cache/calendar.parquet`, currency USD, impact
high, time not masked; the file starts 2023-09-01, so W3 only). Minutes from the decision time
to the nearest such release. Trade state: `0–30 min before`, `0–30 min after`,
`30–120 min before`, `30–120 min after`, `clear` (more than 2 hours either side).
Rows whose feature cannot be computed (not enough history) are counted as `n/a` and never flagged.
## 3. Rule
A screen is an in-book statistic: it cannot see the trades a change would add and it ignores the
5-bar cooldown. It can only nominate a hypothesis.
- A state is **nominated** only if, on one timeframe, it holds at least 8% of the trades of BOTH
books (LONG and W3) and its total R is negative in ALL FOUR slices.
- For each nominated state the mechanism is written down before its replay.
- At most **three** replays in total, on the final-veto research harness of the campaign (K1 + K2
of the twin, a mask on the final candidates before the cooldown, the shared resolver), judged by
the same risk-first rule T against the shipped book: W3 R ≥ 97% of control, W3 PF ≥ control +
0.10, W3 max drawdown ≤ control, LONG R ≥ control, LONG max drawdown ≤ control, 2021 + 2022
better, HOLDOUT R ≥ control − 0.5R, the same at $0.30, at least 20 trades differ. (No
neighbour-definition clause here: a pass only earns a full Stage-1/Stage-2 build, where the
neighbours are tested.)
- No nomination, no replay. No second cut-off, no other bins, no "best of seven" story afterwards.
With 7 features, about 25 states and 2 timeframes, a state that is negative in four overlapping
slices by luck alone is not rare; that is why a nomination is only a ticket to a replay.
## 4. Results
Run 2026-10-01 on the four books of section 1 (read from the campaign's exported CSVs; the feature
frames are the kept Oanda cache windows: daily `1d_20150303_20260807` / `1d_20191105_20260905`,
four-hour `4h_20181031_20260807` / `4h_20230114_20260905`, the books' own primary frames for
volume, `calendar.parquet` for releases). Every trade's decision time equals its entry time in
all four books (no entry sat behind a market closure). Cells are total R (trades); "mean" is R per
trade.
### 4.1 M15 (daily-trend veto on): LONG 1473 trades / +223.33R, W3 824 trades / +205.12R
**M15 — 1. weekly trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 78 (5%) | -15.0 / -0.193 | 25 (3%) | +3.9 / +0.157 | -18.3 (58) | +3.2 (20) | +3.8 (7) | +0.1 (18) | |
| unclear | 176 (12%) | +28.1 / +0.160 | 101 (12%) | +37.8 / +0.375 | +1.9 (102) | +26.2 (74) | +25.4 (61) | +12.5 (40) | |
| with | 1219 (83%) | +210.3 / +0.173 | 698 (85%) | +163.4 / +0.234 | +53.1 (568) | +157.2 (651) | +47.6 (316) | +115.8 (382) | |
**M15 — 2. four-hour trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 260 (18%) | +23.3 / +0.090 | 134 (16%) | +14.8 / +0.111 | +12.4 (137) | +10.9 (123) | +3.4 (59) | +11.4 (75) | |
| unclear | 167 (11%) | +16.8 / +0.100 | 84 (10%) | +16.7 / +0.199 | +0.3 (90) | +16.5 (77) | +1.8 (41) | +14.9 (43) | |
| with | 1046 (71%) | +183.3 / +0.175 | 606 (74%) | +173.6 / +0.286 | +24.1 (501) | +159.2 (545) | +71.5 (284) | +102.1 (322) | |
**M15 — 3. daily-trend strength**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| counter | 145 (10%) | -4.8 / -0.033 | 72 (9%) | +16.9 / +0.234 | -13.1 (92) | +8.3 (53) | +15.2 (39) | +1.7 (33) | |
| medium | 586 (40%) | +106.6 / +0.182 | 393 (48%) | +94.6 / +0.241 | +16.7 (199) | +89.9 (387) | +9.5 (162) | +85.1 (231) | |
| strong | 17 (1%) | +4.6 / +0.268 | 10 (1%) | +4.8 / +0.483 | -0.3 (7) | +4.8 (10) | -1.1 (1) | +5.9 (9) | |
| weak | 725 (49%) | +117.0 / +0.161 | 349 (42%) | +88.8 / +0.254 | +33.5 (430) | +83.5 (295) | +53.1 (182) | +35.7 (167) | |
**M15 — 4. daily volatility state**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| normal | 445 (30%) | +54.8 / +0.123 | 299 (36%) | +77.6 / +0.259 | -11.8 (194) | +66.6 (251) | +6.7 (118) | +70.8 (181) | |
| quiet | 373 (25%) | +64.1 / +0.172 | 152 (18%) | +51.7 / +0.340 | +24.2 (235) | +39.9 (138) | +38.9 (109) | +12.8 (43) | |
| wild | 655 (44%) | +104.4 / +0.159 | 373 (45%) | +75.9 / +0.203 | +24.3 (299) | +80.1 (356) | +31.1 (157) | +44.8 (216) | |
**M15 — 5. place in the 20-day range**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| middle | 420 (29%) | +54.9 / +0.131 | 230 (28%) | +32.3 / +0.140 | +25.3 (217) | +29.7 (203) | +30.9 (107) | +1.4 (123) | |
| near the chasing edge | 835 (57%) | +143.5 / +0.172 | 483 (59%) | +147.1 / +0.305 | +8.8 (400) | +134.7 (435) | +40.6 (227) | +106.5 (256) | |
| near the far edge | 218 (15%) | +24.8 / +0.114 | 111 (13%) | +25.7 / +0.231 | +2.6 (111) | +22.2 (107) | +5.3 (50) | +20.4 (61) | |
**M15 — 6. tick-volume surge**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| surge | 556 (38%) | +83.1 / +0.149 | 226 (27%) | +45.7 / +0.202 | +45.8 (355) | +37.3 (201) | +22.5 (110) | +23.1 (116) | |
| thin | 220 (15%) | +15.6 / +0.071 | 125 (15%) | +28.5 / +0.228 | -10.4 (99) | +26.0 (121) | +11.5 (48) | +17.0 (77) | |
| usual | 697 (47%) | +124.7 / +0.179 | 473 (57%) | +130.9 / +0.277 | +1.4 (274) | +123.2 (423) | +42.7 (226) | +88.3 (247) | |
**M15 — 7. time to a high-impact US release (W3 only)**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| 0–30 min after | | | 42 (5%) | +8.9 / +0.212 | | | +4.7 (27) | +4.2 (15) | |
| 0–30 min before | | | 26 (3%) | +8.7 / +0.336 | | | +7.8 (18) | +0.9 (8) | |
| 30–120 min after | | | 75 (9%) | +28.4 / +0.378 | | | +11.7 (51) | +16.6 (24) | |
| 30–120 min before | | | 22 (3%) | +16.1 / +0.732 | | | +12.9 (18) | +3.2 (4) | |
| clear | | | 659 (80%) | +143.0 / +0.217 | | | +39.6 (270) | +103.5 (389) | |
### 4.2 M5: LONG 1052 trades / +128.10R, W3 870 trades / +153.64R
**M5 — 1. weekly trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 258 (25%) | +11.0 / +0.043 | 202 (23%) | +6.6 / +0.033 | +9.4 (69) | +1.6 (189) | +3.8 (10) | +2.8 (192) | |
| unclear | 84 (8%) | +6.3 / +0.075 | 88 (10%) | +24.6 / +0.279 | -7.5 (30) | +13.8 (54) | +8.2 (12) | +16.4 (76) | |
| with | 710 (67%) | +110.8 / +0.156 | 580 (67%) | +122.5 / +0.211 | -12.3 (159) | +123.1 (551) | +4.7 (114) | +117.8 (466) | |
**M5 — 2. four-hour trend**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| against | 332 (32%) | +3.0 / +0.009 | 255 (29%) | +11.2 / +0.044 | -5.5 (98) | +8.4 (234) | -0.9 (35) | +12.2 (220) | |
| unclear | 126 (12%) | +13.7 / +0.109 | 105 (12%) | +17.8 / +0.170 | +0.6 (28) | +13.1 (98) | +0.8 (13) | +17.0 (92) | |
| with | 594 (56%) | +111.4 / +0.188 | 510 (59%) | +124.6 / +0.244 | -5.5 (132) | +116.9 (462) | +16.9 (88) | +107.7 (422) | |
**M5 — 3. daily-trend strength**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| counter | 296 (28%) | +0.7 / +0.003 | 241 (28%) | -2.6 / -0.011 | +8.8 (87) | -8.0 (209) | +3.7 (18) | -6.3 (223) | |
| medium | 367 (35%) | +77.3 / +0.211 | 318 (37%) | +78.4 / +0.247 | -1.2 (49) | +78.4 (318) | +6.9 (53) | +71.6 (265) | |
| strong | 14 (1%) | +16.6 / +1.186 | 13 (1%) | +15.6 / +1.199 | +1.0 (1) | +15.6 (13) | +0.0 (0) | +15.6 (13) | |
| weak | 375 (36%) | +33.5 / +0.089 | 298 (34%) | +62.2 / +0.209 | -19.0 (121) | +52.5 (254) | +6.2 (65) | +56.1 (233) | |
**M5 — 4. daily volatility state**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| normal | 302 (29%) | +33.2 / +0.110 | 305 (35%) | +45.6 / +0.150 | -0.8 (63) | +34.0 (239) | -3.0 (42) | +48.6 (263) | |
| quiet | 114 (11%) | +18.8 / +0.165 | 72 (8%) | +17.9 / +0.249 | +3.0 (45) | +15.8 (69) | +10.9 (37) | +7.0 (35) | |
| wild | 636 (60%) | +76.1 / +0.120 | 493 (57%) | +90.1 / +0.183 | -12.6 (150) | +88.8 (486) | +8.8 (57) | +81.3 (436) | |
**M5 — 5. place in the 20-day range**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| middle | 323 (31%) | +11.8 / +0.037 | 271 (31%) | +18.8 / +0.069 | -6.8 (76) | +18.6 (247) | -1.8 (34) | +20.6 (237) | |
| near the chasing edge | 439 (42%) | +101.5 / +0.231 | 380 (44%) | +115.0 / +0.303 | -2.6 (103) | +104.1 (336) | +17.2 (77) | +97.8 (303) | |
| near the far edge | 290 (28%) | +14.8 / +0.051 | 219 (25%) | +19.9 / +0.091 | -1.0 (79) | +15.9 (211) | +1.4 (25) | +18.5 (194) | |
**M5 — 6. tick-volume surge**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| surge | 479 (46%) | +46.2 / +0.097 | 334 (38%) | +65.3 / +0.195 | -5.9 (176) | +52.1 (303) | +13.2 (64) | +52.1 (270) | |
| thin | 152 (14%) | +9.3 / +0.061 | 134 (15%) | +18.1 / +0.135 | -8.0 (22) | +17.3 (130) | -0.7 (7) | +18.8 (127) | |
| usual | 421 (40%) | +72.6 / +0.172 | 402 (46%) | +70.3 / +0.175 | +3.4 (60) | +69.2 (361) | +4.2 (65) | +66.1 (337) | |
**M5 — 7. time to a high-impact US release (W3 only)**
| State | LONG trades (share) | LONG R / mean | W3 trades (share) | W3 R / mean | LONG era A (2019–2023) | LONG era B (2024–2026) | W3 first half | W3 second half | Nominated |
|---|---|---|---|---|---|---|---|---|---|
| 0–30 min after | | | 46 (5%) | -3.8 / -0.082 | | | -0.6 (15) | -3.2 (31) | |
| 0–30 min before | | | 18 (2%) | +1.8 / +0.098 | | | +0.7 (5) | +1.1 (13) | |
| 30–120 min after | | | 87 (10%) | +32.8 / +0.377 | | | +8.3 (31) | +24.4 (56) | |
| 30–120 min before | | | 18 (2%) | +3.4 / +0.187 | | | +2.7 (7) | +0.7 (11) | |
| clear | | | 701 (81%) | +119.5 / +0.171 | | | +5.6 (78) | +114.0 (623) | |
### 4.3 Verdict
**Nominated states: none.** No state on either timeframe holds 8% of both books and loses in all
four slices. By the rule of section 3 there is no replay and nothing ships.
What the tables do say (in-book description, not a decision):
- **M15, after the daily-trend veto, has no losing pocket left in these seven views.** The only
clearly negative cell is trades against the weekly trend in 2019–2023 (−18.3R over 58 trades),
5% of the book and positive since 2024. Trades with the four-hour trend earn +0.18R to +0.29R
each against +0.09R to +0.11R against it: weaker, still positive.
- **M5 earns its R where everything already agrees.** With the weekly trend +0.16R to +0.21R per
trade, against it +0.03R to +0.04R; with the four-hour trend +0.19R to +0.24R, against it
+0.01R to +0.04R; when the 60-day move runs against the trade, 0.00R (about 28% of the book).
Those against-the-tide M5 trades are flat, not losing, which is the same picture the formal
daily-trend test gave on M5: removing them removes no loss, and it made the worst losing run
deeper. They stay.
- **M5's profit sits near the 20-day extreme in the trade's direction** (+0.23R to +0.30R per
trade against +0.04R to +0.09R elsewhere). This repeats the 2026-08-04 finding that the best
trades are the stretched runs: a "do not chase" filter would cut the part of the book that pays.
- **Volume and news timing separate nothing reliably.** A volume surge is the best state in
2019–2023 on M15 and the middle state since; trades 30 to 120 minutes after a major US release
look good on both timeframes (+0.38R each) on 75 and 87 trades, and trades in the half hour
after a release are slightly negative on M5 (−3.8R over 46 trades, 5% of the book, below the
8% bar). Neither is a rule.
So the daily chart was the one piece of missing context that paid, on M15. Weekly and four-hour
context add nothing on top of it there, and nothing on M5 that survives the rule.
@@ -0,0 +1,250 @@
# Single "conviction" number vs the two-number grade — pre-registered campaign, 2026-09-27
Owner question (2026-09-27): "Can there be a combination of both [score and confidence] that is
easier to understand and also for the system to make decisions?" Chosen route (owner): research a
single decision number. This file was written BEFORE any candidate run. Rules first, results
appended below them. Nothing ships from this campaign; a pass is a hypothesis for a per-timeframe
override in a separate commit (CLAUDE.md rule 2).
## What is tested
Today (`signal_aggregator._compute_final_signal`, gate 6c) a grade needs BOTH a score floor and a
confidence floor: A+ |score| ≥ 40 & conf ≥ 65 & mtf; A 33 & 55 & mtf; B 28 & 50; C 25 & 45;
D |score| ≥ 15; STRONG 35 & 55; BUY/SELL 25 & 50; 6c boost A 30 & 50 & mtf, B 25 & 45.
Candidates replace the pair with one number. Thresholds are fixed analytically from the control's
corner points — no grid, no tuning (a data-mined threshold would need its own holdout):
- **K-MIN**: `conviction = min(|score|, 0.6 × confidence)`. Grades A+ ≥ 40 (mtf), A ≥ 33 (mtf),
B ≥ 28, C ≥ 25, D |score| ≥ 15 unchanged, else F; STRONG ≥ 35, BUY/SELL ≥ 25; 6c boost A ≥ 30
(mtf), B ≥ 25. (The control's corner ratios score/conf are 0.615, 0.60, 0.56, 0.556, so 0.6 is the
natural single scale.) Against the control: A+ needs conf ≥ 66.7 (was 65), B ≥ 46.7 (50),
C ≥ 41.7 (45), STRONG ≥ 58.3 (55), BUY/SELL ≥ 41.7 (50); score floors unchanged.
- **K-PROD**: `conviction = |score| × confidence / 100`. A+ ≥ 26 (40×65), A ≥ 18.15 (33×55),
B ≥ 14 (28×50), C ≥ 11.25 (25×45), D |score| ≥ 15, else F; STRONG ≥ 19.25 (35×55),
BUY/SELL ≥ 12.5 (25×50); 6c A ≥ 15 (30×50), B ≥ 11.25 (25×45). A genuine trade-off: high
confidence can lift a mid score and vice versa. Interaction on record: a product B/C with
|score| < 25 is demoted to D on M5 by the live gate 1 (`m5_min_abs_score` 25); the gate stays,
the count of demotions is reported (Step E). Property on record (code review, before the runs):
the D rung stays `|score| ≥ 15` in both candidates, and under the product form B/C can sit below
it (B needs |score| ≥ 14.74 at the 95 confidence cap, C ≥ 11.85), so an M15/H1/H4 product book can
hold a trade with |score| < 15. Kept as pre-registered (a score floor would make it a pair rule
again); the count of |score| < 15 and < 25 trades per candidate book is reported.
- Geometric mean √(|score|·conf) is a monotone transform of the product (same partition at
thresholds √T) — not a separate candidate.
- Direction stays sign(score); mtf agreement stays required for A+/A; the exhaustion hard block
(conf = 0) gives conviction 0 in both forms, so NEUTRAL and at most D, as today.
Switch: `signals.grading_mode` (`"pair"` default = today's literal rule, `"min"`, `"product"`),
read in `_compute_final_signal` and gate 6c (`signal_aggregator.py` helpers `_signal_pair`,
`_quality_pair`, `_transitioning_boost`, `conviction`, `CONVICTION_THRESHOLDS`), mirrored in K2
(`backtest/cuda_signal_kernels.py`, `PI_grading_mode`, `PF_conv_*` packed from the live module),
refused by the twin for other values. `pair` is byte-identical to HEAD `1454ea1` (Step A proves it).
Note: `engine_config_fingerprint` hashes `SignalConfig`, so the live fingerprint changes at the
deploy of this field even in `pair` mode (label-only; split signal-log comparisons there).
## Harness
Twin command template (research twin, CLAUDE.md §3/§9; seconds per book):
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --spread <0.65|0.30>
--primary-tf <5min|15min|1h|4h> --start <s> --end <e> [--set signals.grading_mode=<min|product>]
--export-csv <out>`. Reference script (`backtest/gpu_backtest.py`, same flags, `--workers 12`)
for the neutrality proof and every paired MD5. Factory configuration; `--set` is the only override.
Evidence order (all must hold before any candidate number is read):
1. **Neutrality**: with the new code and no `--set`, the five pins reproduce on BOTH scripts
(M5 W2 `e599c8a8652f95b7fcdf5a7ef2b564f5`, M15 W2 `5b65ecf6fad9a8da412749066fd50d36`,
M15 OOS `09b8a8e7f5833436c20aef60ca0a7ff7`, H1 W3 `a44fd19b4067f46e5bd98d50cc3eff92`,
H4 W3 `f3587bae3c5dc3eda468ea3588b17dcb`), plus the twin with an explicit
`--set signals.grading_mode=pair` on M15 OOS.
2. **Twin parity on the new paths**: each candidate's M15 OOS $0.65 book is MD5-equal on both scripts.
3. A passing candidate is re-run once on `gpu_backtest.py` (W2, $0.65) and the two MD5s must be equal.
Windows: W2 2024-09-05 → 2026-09-05 (halves split at 2025-09-05); OOS 2024-06-01 → 2025-03-30
(halves at 2024-10-30); W3 2023-09-05 → 2026-09-05 (halves at 2025-03-05; max-DD clause); 5-year
2021-08-01 → 2026-08-04 (per-year clause; cache `20210531_20260804`). Costs: $0.65 headline, $0.30
comparability. Books are written outside the tree (§9 keeps no CSVs in-tree).
Code review gate (owner instruction): the diff is reviewed with the `code-review` skill before Step A;
outcome recorded here. Outcome (2026-09-27, `code-review` high, before Step A): 7 findings, 6 fixed in the same working
tree — kernel mode codes pinned by name (`GRADING_MODE_CODES`) and checked against the live list;
no zero-threshold fallback for a non-pair mode; the twin's refusal reads the kernel's map; `conviction()`
refuses modes it has no formula for; one `_final_signal_and_grade` dispatch point; `engine_config_fingerprint`
omits the switch under `pair` so live rows keep their fingerprint; CLAUDE.md test counts updated. One
recorded, not changed: the D rung under the product form (above). Suite green after the fixes.
## Candidates
| # | TF | `--set` | Hypothesis |
|---|---|---|---|
| K-MIN | M5, M15 (H1/H4 for the record) | `signals.grading_mode=min` | one scale on the score axis; nearly the control, loosens B/C/BUY confidence, tightens A+ |
| K-PROD | M5, M15 (H1/H4 for the record) | `signals.grading_mode=product` | genuine trade-off between the two axes; confidence can lift a mid score |
## Acceptance rule (CLAUDE.md rule 1, written before the runs)
A candidate PASSES only if, at $0.65, on W2: (a) total R ≥ control + 5R; (b) both halves ≥ the
control's halves − 1R (no half may fund the other); (c) PF ≥ 1.3 and ≥ control − 0.02; (d) OOS
total R ≥ control − 1R; (e) W3 max drawdown ≤ control + 10%; (f) the same sign of ΔR at $0.30.
A candidate that changes fewer than 20 trades on W2 is "no evidence", not a pass.
Judge run-level numbers, never the added/removed bucket alone (the 5-bar cooldown displaces
entries). Trade diff keyed on (entry time, direction).
Campaign clause (g): on the 5-year window at $0.65, no calendar year in which the control is
positive goes below −5R under the candidate (trend-artefact guard, `fixedtp_2026-08-05.md`
precedent); a W2 gain that lives in a single year is read as an artefact and noted.
The rule is judged independently on M5 W2 and M15 W2: the mode is read per config, so a pass ships
as a per-timeframe factory override (rule 2) and a timeframe where it fails keeps `pair`. H1 and H4
are run on W3 for the record (ΔR at $0.65, trades changed); they are not ship candidates here — a
positive H1/H4 result is a hypothesis for its own pre-registration. Nothing ships if no clause set
passes; the field stays a research-only knob (precedent `--fixed-rr-tp`).
## Run list
Step 0 suite green (Module 37 on the GPU) and the review gate. Step A neutrality (11 runs).
Step B twin parity on the new paths (4 runs). Step C books on the twin: {M5 W2, M5 OOS, M5 W3,
M15 W2, M15 OOS, M15 W3, H1 W3, H4 W3} × {$0.65, $0.30} × {control, min, product}. Step D per-year
(5-year, M5 and M15, $0.65, control/min/product). Step E K-PROD M5 W2 $0.65 with
`--record DIR --record-bars` (MD5 equal to the Step C book) to count `m5_score` demotions.
Step F reference-script rerun for any passer.
## Results
<!-- appended after the runs; nothing above this line changes -->
Reference script `Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, `GPU cache fallbacks: 0`;
twin `Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, `GPU cache fallbacks: 0 (no per-bar
recompute path in cudaBasedBacktesting)`. Every run from the kept cache windows.
### Step A — neutrality (new code, `grading_mode` at its default, no `--set`)
| Book | Pin | `gpu_backtest.py` | `cudaBasedBacktesting.py` |
|---|---|---|---|
| M5 W2 $0.65 | `e599c8a8652f95b7fcdf5a7ef2b564f5` | equal | equal |
| M15 W2 $0.65 | `5b65ecf6fad9a8da412749066fd50d36` | equal | equal |
| M15 OOS $0.65 | `09b8a8e7f5833436c20aef60ca0a7ff7` | equal | equal; also equal with an explicit `--set signals.grading_mode=pair` |
| H1 W3 $0.65 | `a44fd19b4067f46e5bd98d50cc3eff92` | equal | equal |
| H4 W3 $0.65 | `f3587bae3c5dc3eda468ea3588b17dcb` | equal | equal |
The refactor (helpers + switch) is byte-neutral on both scripts: the five pins stand.
### Step B — twin parity on the new code paths (M15 OOS $0.65)
| Candidate | `gpu_backtest.py` | `cudaBasedBacktesting.py` |
|---|---|---|
| K-MIN | `22726d8500383ce02e0a3cbf0412f6d8` | `22726d8500383ce02e0a3cbf0412f6d8` (equal) |
| K-PROD | `d1ae11651d5d7c8066b7e8addc0c7f36` | `d1ae11651d5d7c8066b7e8addc0c7f36` (equal) |
The twin's books below therefore count as evidence (instructions item 6, CUDA-twin addendum).
### Books (all on the twin; the control rows reproduce the reference script's MD5s)
Halves: W2 at 2025-09-05, OOS at 2024-10-30, W3 at 2025-03-05. Trades / R from the 63-column book.
| TF | Window | Cost | Candidate | Trades | Total R | H1 | H2 | WR | PF | Max DD | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| M5 | W2 | $0.65 | control | 781 | +146.07 | +26.42 | +119.65 | 52.9% | 1.52 | 10.1 | e599c8a8 |
| M5 | W2 | $0.65 | K-MIN | 812 | +144.86 | +24.73 | +120.13 | 52.8% | 1.50 | 11.5 | 2ab2b573 |
| M5 | W2 | $0.65 | K-PROD | 805 | +147.33 | +29.27 | +118.06 | 52.5% | 1.52 | 11.6 | a37dbb71 |
| M5 | W2 | $0.30 | control / K-MIN / K-PROD | 781 / 812 / 805 | +162.09 / +161.17 / +163.96 | +31.71 / +30.15 / +34.70 | +130.37 / +131.02 / +129.26 | | 1.60 / 1.57 / 1.59 | 8.9 / 10.6 / 10.9 | 7ab48db9 / 64f26c30 / ed14d7ec |
| M5 | OOS | $0.65 | control | 86 | +9.47 | +0.01 | +9.46 | 53.5% | 1.33 | 8.2 | 65c46e62 |
| M5 | OOS | $0.65 | K-MIN | 94 | +14.32 | +2.37 | +11.94 | 56.4% | 1.51 | 6.4 | bec7c87f |
| M5 | OOS | $0.65 | K-PROD | 91 | +13.03 | +0.10 | +12.93 | 54.9% | 1.45 | 4.5 | 217d0134 |
| M5 | OOS | $0.30 | control / K-MIN / K-PROD | 86 / 94 / 91 | +12.42 / +17.34 / +16.05 | | | | 1.46 / 1.66 / 1.58 | 6.7 / 4.8 / 4.1 | |
| M5 | W3 | $0.65 | control | 870 | +153.64 | +16.73 | +136.91 | 53.3% | 1.50 | 10.1 | d20ec8a3 |
| M5 | W3 | $0.65 | K-MIN | 905 | +153.84 | +20.64 | +133.20 | 53.3% | 1.48 | 11.5 | 5ad0c381 |
| M5 | W3 | $0.65 | K-PROD | 896 | +154.60 | +20.12 | +134.48 | 52.9% | 1.49 | 11.6 | 466bda90 |
| M5 | W3 | $0.30 | control / K-MIN / K-PROD | 870 / 905 / 896 | +172.40 / +172.92 / +173.99 | | | | 1.57 / 1.55 / 1.56 | 8.9 / 10.6 / 10.9 | |
| M15 | W2 | $0.65 | control | 813 | +154.15 | +70.10 | +84.04 | 62.2% | 1.53 | 15.6 | 5b65ecf6 |
| M15 | W2 | $0.65 | K-MIN | 827 | +151.25 | +69.05 | +82.21 | 61.9% | 1.51 | 16.0 | 5c7918dc |
| M15 | W2 | $0.65 | K-PROD | 1106 | +125.17 | +62.26 | +62.91 | 58.2% | 1.29 | 28.9 | ecfb62e5 |
| M15 | W2 | $0.30 | control / K-MIN / K-PROD | 813 / 827 / 1106 | +167.53 / +164.82 / +144.91 | +78.61 / +77.62 / +74.30 | +88.91 / +87.20 / +70.61 | | 1.59 / 1.57 / 1.34 | 14.5 / 14.9 / 26.8 | 15486607 / 5ab311cf / 582019a7 |
| M15 | OOS | $0.65 | control | 313 | +68.50 | +18.87 | +49.63 | 63.6% | 1.67 | 12.4 | 09b8a8e7 |
| M15 | OOS | $0.65 | K-MIN | 314 | +65.64 | +19.20 | +46.44 | 63.4% | 1.64 | 12.4 | 22726d85 |
| M15 | OOS | $0.65 | K-PROD | 408 | +53.98 | +26.67 | +27.32 | 59.1% | 1.36 | 17.3 | d1ae1165 |
| M15 | OOS | $0.30 | control / K-MIN / K-PROD | 313 / 314 / 408 | +76.21 / +73.36 / +64.35 | | | | 1.77 / 1.74 / 1.44 | 11.1 / 11.1 / 16.4 | |
| M15 | W3 | $0.65 | control | 1105 | +195.79 | +76.87 | +118.92 | 61.1% | 1.49 | 20.9 | 70195fb0 |
| M15 | W3 | $0.65 | K-MIN | 1126 | +191.25 | +71.90 | +119.35 | 60.7% | 1.47 | 21.3 | 23edd8bc |
| M15 | W3 | $0.65 | K-PROD | 1478 | +186.24 | +84.70 | +101.54 | 58.3% | 1.33 | 28.9 | 0246cbb4 |
| M15 | W3 | $0.30 | control / K-MIN / K-PROD | 1105 / 1126 / 1478 | +216.53 / +212.32 / +215.54 | | | | 1.55 / 1.53 / 1.39 | 19.6 / 20.0 / 26.8 | |
| H1 | W3 | $0.65 | control | 253 | +70.02 | +26.26 | +43.76 | 64.4% | 1.98 | 4.4 | a44fd19b |
| H1 | W3 | $0.65 | K-MIN | 268 | +74.81 | +28.75 | +46.06 | 64.9% | 1.99 | 5.7 | 24674555 |
| H1 | W3 | $0.65 | K-PROD | 351 | +81.88 | +30.68 | +51.19 | 60.1% | 1.75 | 8.5 | 2e8df9cc |
| H1 | W3 | $0.30 | control / K-MIN / K-PROD | 253 / 268 / 351 | +72.74 / +77.64 / +85.91 | | | | 2.03 / 2.04 / 1.80 | 4.3 / 5.6 / 8.3 | |
| H4 | W3 | $0.65 | control | 114 | +21.68 | +4.91 | +16.77 | 57.9% | 1.59 | 5.4 | f3587bae |
| H4 | W3 | $0.65 | K-MIN | 117 | +22.57 | +4.91 | +17.66 | 58.1% | 1.59 | 5.4 | 3ed9a44e |
| H4 | W3 | $0.65 | K-PROD | 151 | +38.69 | +13.97 | +24.72 | 58.9% | 1.78 | 6.3 | b696b617 |
| H4 | W3 | $0.30 | control / K-MIN / K-PROD | 114 / 117 / 151 | +22.52 / +23.43 / +39.88 | | | | 1.61 / 1.62 / 1.81 | 5.0 / 5.0 / 6.0 | |
### Trade diffs at $0.65 (key = entry time + direction; W2 for M5/M15, W3 for H1/H4)
- M5 K-MIN: 37 added (+1.74R), 6 removed (+4.53R), 775 common (+143.12R vs +141.54R in the control:
the cooldown shifts a few entries). Grade mix B 572 → 606, A 187 → 186, C 21 → 19. Changed 43.
- M5 K-PROD: 74 added (+16.51R), 50 removed (+10.12R), 731 common (+130.83R vs +135.95R). Grade
mix A 187 → 263, A+ 1 → 3, C 21 → 8; XAU_ATR setups 32 → 57 (more A grades reach the ATR
generator). Changed 124.
- M15 K-MIN: 34 added (−0.34R), 20 removed (+3.42R), 793 common (+151.59R vs +150.73R). A+ 22 → 10
(the A+ confidence floor rises to 66.7), B 443 → 465. Changed 54.
- M15 K-PROD: 544 added (+2.66R), 251 removed (+16.64R), 562 common (+122.51R vs +137.50R). The
book grows 813 → 1106 (C 60 → 196, A+ 22 → 79); 295 candidate trades carry |score| < 25 and earn
+5.11R between them. Changed 795.
- H1 (for the record): K-MIN +20 / −5, 248 common, ΔR +4.79; K-PROD +126 / −28, 225 common,
ΔR +11.86 with PF 1.98 → 1.75 and max DD 4.4 → 8.5R; 63 trades with |score| < 25 (+7.38R).
- H4 (for the record): K-MIN +3 / −0, ΔR +0.89 (3 trades changed: no evidence); K-PROD +40 / −3,
ΔR +17.01 with PF 1.59 → 1.78 and DD 5.4 → 6.3R, 28 trades with |score| < 25 (+15.41R).
- No candidate book on any timeframe holds a trade with |score| < 15 except the M5 chart-pattern
entries the control also has (127 XAU_CP trades, forced grade B by design, identical in all three
M5 books). The D-rung property recorded above did not materialise.
- Step E (M5 W2 $0.65 K-PROD with `--record --record-bars`, book MD5 `a37dbb71…` = the Step C
book): 82 analysed bars carry the gate-1 warning `M5 FILTER: |score| … < 25 minimum` as their
first warning, every one of them ending at grade D. That is the whole M5 gate-1 interaction.
### Per year, 5-year window 2021-08-01 → 2026-08-04 at $0.65 (twin)
| TF | Year | control n / R | K-MIN n / R | K-PROD n / R |
|---|---|---|---|---|
| M5 | 2021 | 12 / −1.99 | 14 / −0.28 | 13 / −0.55 |
| M5 | 2022 | 50 / −2.18 | 54 / −0.43 | 52 / +1.29 |
| M5 | 2023 | 61 / −2.75 | 62 / −2.10 | 61 / −3.57 |
| M5 | 2024 | 102 / +0.97 | 108 / +4.05 | 106 / +4.19 |
| M5 | 2025 | 332 / +40.54 | 346 / +38.87 | 342 / +41.63 |
| M5 | 2026 | 348 / +92.49 | 361 / +87.15 | 358 / +85.73 |
| M5 | total | 905 / +127.08 (PF 1.39, DD 13.5) | 945 / +127.25 (1.37, 12.4) | 932 / +128.72 (1.38, 12.2) |
| M15 | 2021 | 78 / −10.88 | 84 / −11.20 | 90 / −6.29 |
| M15 | 2022 | 267 / −24.63 | 273 / −26.34 | 340 / −47.01 |
| M15 | 2023 | 211 / +11.74 | 216 / +11.10 | 261 / +8.63 |
| M15 | 2024 | 343 / +53.35 | 345 / +49.12 | 448 / +62.45 |
| M15 | 2025 | 425 / +81.16 | 438 / +85.91 | 576 / +83.40 |
| M15 | 2026 | 224 / +38.10 | 227 / +32.98 | 317 / +4.32 |
| M15 | total | 1548 / +148.84 (PF 1.25, DD 48.3) | 1583 / +141.58 (1.23, 48.7) | 2032 / +105.50 (1.13, 77.4) |
### Verdicts against the pre-registered rule (W2 at $0.65 unless stated)
| Candidate | TF | (a) ΔR ≥ +5 | (b) halves ≥ ctrl − 1 | (c) PF ≥ 1.3 & ≥ ctrl − 0.02 | (d) OOS ≥ ctrl − 1 | (e) W3 DD ≤ ctrl + 10% | (f) same sign at $0.30 | (g) 5-yr years | changed | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|
| K-MIN | M5 | ✗ −1.21 | ✗ H1 24.73 < 25.42 | ✓ 1.50 | ✓ +14.32 vs 9.47 | ✗ 11.5 > 11.1 | ✓ −0.92 | ✓ | 43 | **FAIL (a, b, e)** |
| K-PROD | M5 | ✗ +1.26 | ✗ H2 118.06 < 118.65 | ✓ 1.52 | ✓ +13.03 | ✗ 11.6 > 11.1 | ✓ +1.87 | ✓ | 124 | **FAIL (a, b, e)** |
| K-MIN | M15 | ✗ −2.90 | ✗ 69.05 < 69.10, 82.21 < 83.04 | ✓ 1.51 | ✗ 65.64 < 67.50 | ✓ 21.3 ≤ 23.0 | ✓ −2.71 | ✓ | 54 | **FAIL (a, b, d)** |
| K-PROD | M15 | ✗ −28.98 | ✗ | ✗ 1.29 | ✗ 53.98 | ✗ 28.9 > 23.0 | ✓ −22.62 | ✓ (2026 +4.32, still positive) | 795 | **FAIL (a–e)** |
Reading. K-MIN is the control with a slightly looser confidence floor on B/C/BUY and a tighter one
on A+: it moves 43–54 trades on W2 and changes nothing of substance (−1.2R on M5, −2.9R on M15, PF
within 0.02, OOS mixed). It is a re-labelling of the same rule with a small loss, not a better
decision number. K-PROD is a different rule: on M15 it lets confidence substitute for score, grows
the book by a third and gives back −29R with PF 1.29 and a W3 drawdown of 28.9R (5-year 77.4R, PF
1.13); on M5 it is neutral. On H1 and H4 the product rule adds R (+11.9R and +17.0R on W3) by
taking 40–100 extra low-score trades; H1's PF drops 1.98 → 1.75 and its drawdown doubles, H4 stays
"unproven" (114 → 151 trades). Both are hypotheses for their own pre-registration, not evidence.
## Not shipped (2026-09-27)
No candidate passes on M5 or M15. The two-number rule stays as validated; `signals.grading_mode`
remains a research-only knob (default `pair`, byte-identical books). Follow-ups:
- The "easier to understand" half of the owner's question is a display task: the grade already
combines score, confidence and timeframe agreement, and the surfaces can lead with it and
word the two raw numbers in plain language (no signal change, books untouched).
- H1 / H4 K-PROD (+11.9R / +17.0R on W3, more trades, lower H1 PF, higher DD) may be pre-registered
on its own if the owner wants it; H4 has too few trades for rule 1 and H1's edge already lives
in a small book (253 trades in 3 years).
- Do not re-run these two formulas on M5 or M15 with other thresholds: a tuned threshold would
need its own holdout and the corner-point versions here bracket the plausible range.
@@ -0,0 +1,215 @@
# Full-GPU research twin of the backtester — 2026-09-18
Behaviour-neutral by construction and by proof. `backtest/cudaBasedBacktesting.py` runs every stage
of a backtest on the GPU, including the per-bar decision that `gpu_backtest.py` still runs through
the live Python signal code. Nothing in the signal logic, the weights, the gates or the exit rules
moved; the proof is that both scripts export byte-identical books (same CSV MD5) on every window
and every research flag tried, listed in section 3.
## 1. Owner decisions
- Research and backtesting only. Live scans stay on the CPU signal code; the Telegram bot, the CLI
and the dashboard never import the new files (Module 37 sweeps the live roots).
- `gpu_backtest.py` stays in the tree as the **base reference**. The twin must reproduce its
exported book byte for byte for the same arguments; the MD5 of the CSV is the acceptance gate.
- `--record`, `--record-bars`, `--export-features` and `--fixed-rr-tp` all work on the GPU path.
- Acceptance: 6-month M5 and M15 books, window 2026-03-18 → 2026-09-18, both scripts, identical
CSV MD5 at $0.65 (headline) and $0.30 (comparability). That window was not cached; the first
control run fetched M5/M15/H1 from Oanda (padded 62 days, `oanda_XAU_USD_<tf>_20260115_20260918`)
and both scripts read the same files afterwards.
- `backtest/instructions_for_backtest.txt` item 6 ("replays go through the real signal code, never
a re-implementation") is waived for this ONE research script; the dated addendum in that file
records the exemption and the new obligation: any signal change is a parity event, re-prove the
MD5 with both scripts before a number from the twin counts as evidence.
## 2. What shipped
| File | Role |
|---|---|
| `backtest/cudaBasedBacktesting.py` (1,028 lines) | Same CLI as the reference (it reuses `_parse_args`), same cache files, same report and CSV writers. Reuses by import everything that is not the replay: the loader and its cleaning, `precompute_indicators` (GPU), the trade resolver (GPU), `_reanchor_entries_to_raw`, `_apply_fixed_rr_tp`, `compute_metrics`, `print_report`, `trades_to_dataframe`, `replay_record.write_run`. Replaces `_replay_span` + the worker pool + `_apply_cooldown` with kernel launches and host builders that turn the kernels' numeric codes into the exact strings (indicator descriptions, warnings, reasons) with the same f-string templates as `engine.py` / `signal_aggregator.py`, then call the real `SignalAggregator._clean_accepted_cp_warnings` and `_drop_stale_spread_gate_vetoes`. No CPU path: `--backend cpu`, `FB2_BACKEND=cpu` or a failed probe exit with a message that points to `gpu_backtest.py`; configurations beyond the compile-time limits (more than 4 confirmation frames, divergence lookback ≥ 64, swing lookback ≥ 256, `cp_lookback` ≥ 128, a big-candle window over 500 bars) are refused the same way. |
| `backtest/cuda_signal_kernels.py` (2,914 lines, most of it CUDA C in strings) | K1 `fb2_analyze_tf` (one thread per bar of one timeframe; the nine `GPUTechnicalEngine` overrides plus the live engine code they do not cover: divergence, structure S/R, candlestick, regime, regime-aware RSI, composite, the `len < 30` empty analysis, `ema_mode="2ema"`). K2 `fb2_aggregate` (one thread per primary bar; `analyze_from_dataframes` end to end: the multi-timeframe merge, bonuses, penalties in the post-sort decay order with the 30 cap, exhaustion block, signal, grade, gates 6b–6i, the XAU_ATR / GENERIC / XAU_PB / XAU_CP generators, the NEUTRAL-at-grade-C direction, `_apply_regime_filter_live`, plus the replay's quality / strategy filters, next-open entry and distance-preserving re-anchor). K3 `fb2_features` (the 65 `position_sizing.build_features` columns per accepted trade). K4 cooldown (single thread, the only cross-bar rule). K5 prefix median (the snapshot's `spread_estimate`). K6 recorder bar rows (the 160-column float32 row). The config packer (`PARAMS_F` / `PARAMS_I` name lists → generated `#define`s, so a name cannot drift), the classification table of every config field the kernels do not read, and thin test launches of the device helpers. Compiled with `--fmad=false`; imports only numpy, `gpu_kernels`, `config`, `chart_patterns` (pattern priority order) and `position_sizing` constants. |
| `test_all.py` Module 37 (16 tests) | Section 4. |
| `backtest/instructions_for_backtest.txt` | The 2026-09-18 CUDA-twin addendum. |
What stays on the host, on purpose: loading and cleaning the parquet cache; the calendar and
session flags (pandas time-zone arithmetic over the whole index: NY weekend, UTC weekday, blocked
close hours, and the three clocks the gates use: bar label for gate 6b, bar close for gate 6g3 and
XAU_CP, decision time for the replay skip); the text; the writers.
Rules that keep the twins bit-identical (they cost most of the debugging time and are now pinned
by tests): doubles everywhere; Python `min()` / `max()` as argument-order ternaries, never
`fmin` / `fmax`; every sum in Python's iteration order; numpy's pairwise summation (8 accumulators,
128-blocks) for `np.mean` / `np.nanmean`; Python 3.12's Neumaier-compensated builtin `sum()` for
the multi-timeframe score mean; pandas' online rolling mean and EWM recurrences copied from
`gpu_kernels.py` for the restarted tails the features use; `round(x, 2)` reproduced exactly with an
error-free product (the host additionally re-checks Python's `round()` on the device doubles of
every accepted trade); `atr_percentile = count / 50 * 100` (not `count * 2`: 15/50·100 =
30.000000000000004 flips "low" to "normal"); the as-of alignment compares int64 nanoseconds.
## 3. Evidence
Environment: NVIDIA GeForce RTX 4070 SUPER, CuPy 14.0.1 (`cupy-cuda12x` with the
`nvidia-cuda-runtime-cu12` header wheel), NumPy 2.2.6, pandas 3.0.2, Python 3.12, 16 CPUs. Code:
this working tree on branch `sql-integration` on top of `bac11d8`. `Backend:` lines verbatim:
reference `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, twin
`CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`; both log `GPU cache fallbacks: 0` (the
twin's line adds "no per-bar recompute path in cudaBasedBacktesting").
### 3.1 Acceptance: 6 months, both scripts, both costs
Harness: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --primary-tf <tf> --start
2026-03-18 --end 2026-09-18 --spread <sp> --workers 12 --export-csv <out>` for the control and
`venv/bin/python backtest/cudaBasedBacktesting.py` with the same arguments (no `--workers`) for the
twin; factory config, no overrides.
| Primary | Cost | Control MD5 | Twin MD5 | Equal | Control wall | Twin wall |
|---|---|---|---|---|---|---|
| M5 | $0.65 | `86111c36c1a07e543a65f0cd09d56ebf` | `86111c36c1a07e543a65f0cd09d56ebf` | ✓ | 51.8 s (replay 43.8 s) | 1.27 s |
| M5 | $0.30 | `e70d025205844ecedc2b9e9c08f38f77` | `e70d025205844ecedc2b9e9c08f38f77` | ✓ | 47.1 s (replay 45.9 s) | 1.65 s |
| M15 | $0.65 | `b8c980833f1a32bd59930bd037c75b89` | `b8c980833f1a32bd59930bd037c75b89` | ✓ | 20.4 s (replay 18.7 s) | 1.24 s |
| M15 | $0.30 | `e66b5fbdaace1d3fda4a76a0e2d47d69` | `e66b5fbdaace1d3fda4a76a0e2d47d69` | ✓ | 14.8 s (replay 13.5 s) | 1.50 s |
Because the CSVs are identical the figures below are the same from either script (`NO_DATA` rows
excluded; halves split at 2026-06-18). They are reported because the instructions require them,
not as a validation: nothing changed, and the window is 6 months (rule 1 needs 12).
| Primary | Cost | Segment | Trades | Total R | Win rate | PF | Max DD (R) |
|---|---|---|---|---|---|---|---|
| M5 | $0.65 | whole | 283 | +41.67 | 56.5% | 1.42 | 10.1 |
| M5 | $0.65 | first half | 146 | +29.58 | 58.9% | 1.64 | 6.6 |
| M5 | $0.65 | second half | 137 | +12.09 | 54.0% | 1.23 | 8.3 |
| M5 | $0.30 | whole | 283 | +46.38 | 57.2% | 1.48 | 8.4 |
| M5 | $0.30 | first half | 146 | +32.07 | 60.3% | 1.71 | 6.4 |
| M5 | $0.30 | second half | 137 | +14.30 | 54.0% | 1.28 | 7.4 |
| M15 | $0.65 | whole | 238 | +16.35 | 60.1% | 1.19 | 14.9 |
| M15 | $0.65 | first half | 120 | +14.09 | 63.3% | 1.37 | 7.6 |
| M15 | $0.65 | second half | 118 | +2.26 | 56.8% | 1.05 | 14.4 |
| M15 | $0.30 | whole | 238 | +19.20 | 60.9% | 1.23 | 13.8 |
| M15 | $0.30 | first half | 120 | +15.37 | 65.0% | 1.41 | 7.1 |
| M15 | $0.30 | second half | 118 | +3.82 | 56.8% | 1.08 | 13.8 |
By generator at $0.65: M5 GENERIC 163 trades +42.4R, XAU_CP 104 −2.7R, XAU_ATR 16 +2.0R (BUY 244
+25.7R, SELL 39 +16.0R); M15 GENERIC 149 +26.3R, XAU_ATR 89 −10.0R (BUY 144 +9.7R, SELL 94
+6.7R). The M15 second half is the same weak stretch the 2026-09-17 live review saw; no decision
follows from a 6-month window.
### 3.2 The five reference books (kept cache, `--spread 0.65`, factory config)
| Book | Window | Twin MD5 | Pinned (CLAUDE.md §9) | Twin wall |
|---|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | `49f8e602fc6ae769713c94ef7b257810` | ✓ | 2.4 s (reference 114.8 s) |
| M15 W2 | 2024-09-05 → 2026-09-05 | `b4aa07e33e17e23774c486a6900d673b` | ✓ | 2.3 s (37.2 s) |
| M15 OOS | 2024-06-01 → 2025-03-30 | `9fd624fb387c36052122bd1830cf9c32` | ✓ | 1.5 s (15.6 s) |
| H1 W3 | 2023-09-05 → 2026-09-05 | `7b848c917dcb5c371ecbb9f8f7292278` | ✓ | 1.3 s (13.0 s) |
| H4 W3 | 2023-09-05 → 2026-09-05 | `186e677fbbd53cca735c94094c23ce05` | ✓ | 0.8 s (21.2 s) |
Reference wall times are the GPU-backend rows of `gpu_backend_2026-09-18.md`. On the twin the
decision stage itself is milliseconds (M5 W2: K1 218,088 bars of three timeframes in 0.020 s, K2
141,819 primary bars in 0.002 s); what remains is loading and cleaning the parquet files, the
indicator precompute (1.3 s on the two-year M5 window), the host formatting of the accepted trades
and the CSV write.
### 3.3 Research flags and knobs (M15 OOS window unless stated; both scripts, `--spread 0.65`)
| Variant | Rows | Twin MD5 (prefix) | Equal |
|---|---|---|---|
| `--export-features` (130 columns) | 355 | `6239d197` | ✓ |
| `--record DIR --record-bars` | 355 | book `9fd624fb`; `trades` (355 × 138), `indicators` (8,120 × 15), `events` (6,902 × 14), `bars` (12,315 × 172) parquet tables equal via `DataFrame.equals`; manifests differ only in `cli`, `created_at`, `replay_seconds`, `workers` | ✓ |
| `--fixed-rr-tp 2.0` | 355 | `95a7d2ab` | ✓ |
| `--quality A+,A` | 249 | `1883c346` | ✓ |
| `--strategies XAU` | 3 | `c07a4025` | ✓ |
| `--cp` (all six chart patterns on M15) | 396 | `7ff51617` | ✓ |
| `--big-candle 3.0` | 104 | `76f76f3b` | ✓ |
| `--set backtest.regime_filter=true` (H4 ADX gate, a fourth frame) | 179 | `0a65af6b` | ✓ |
| `--set technical.ema_mode=2ema` | 368 | `faaa7e86` | ✓ |
| M5, `--set xau_strategy.m5_pullback_enabled=true --quality A+,A,B,C,D` | 279 | `1811117a` | ✓ |
| `--trail-arm 0.5 --trail-dist 0.9 --cooldown-bars 3 --max-hold-bars 150` | 414 | `9665d753` | ✓ |
| H1, `--swing-sr-lookback 80 --warmup-bars 300` | 75 | `f8ef2b9f` | ✓ |
One variant could not run on either script: H1 with `--confirm-tfs 15min,4h,1d` on the OOS
window needs a D1 file padded 1,400 days (2020-08-01) that is not cached, and Oanda returned an
empty page for that start; both scripts fail identically in `fetch_range`. Not a parity issue.
### 3.4 Test suite, three ways (final tree)
| Run | Result | Wall | Module 37 note |
|---|---|---|---|
| default (GPU) | `RESULTS: 552 passed, 0 failed, 552 total` | 116 s | every device test exercised |
| `CUDA_VISIBLE_DEVICES=""` | `RESULTS: 552 passed, 0 failed, 552 total` | 30 s | `CUDA unavailable here: NumPy CPU fallback (CUDARuntimeError: cudaErrorNoDevice: no CUDA-capable device is detected)` printed; device tests print a notice |
| `FB2_BACKEND=cpu` | `RESULTS: 552 passed, 0 failed, 552 total` | 31 s | `NumPy CPU fallback (forced: --backend cpu / FB2_BACKEND=cpu)` printed; the twin's refusal of a cpu-forced backend is exercised for real |
536 tests before this change, 16 added by Module 37.
## 4. Module 37 (what each test pins)
1. `fb2_round2` / `round1` against Python `round(x, 2)` / `round(x, 1)` on 400k doubles including
every `.xx5` half in ±300 and signed zero; without CUDA the rule is checked in exact arithmetic.
2. Parameter packing round-trips every decision leaf of the four factories (M5 blocked hours with
9 and without 21, M15 SELL hours 15/17 and the RSI × 0.5 table, H1 empty SELL quality, M15
`tp_floor_rr` 1.0, the six-regime × nine-indicator table, `FB2_EMA_LO` read once with the same
fallback); every field of the four config dataclasses is either packed or classified host-only,
so a new field fails the suite until it is placed; the `#define`s are generated from the lists.
3. K1 against `GPUTechnicalEngine.analyze` on every bar of a 600-bar fixture (doji, hammer,
shooting star, engulfing, marubozu, a flat stretch, plateaus) for 5min/15min/1h/4h under their
factories, partial-EMA off, `2ema`, classic RSI, disabled indicators and a 150-bar frame: valid
flag, composite, regime, volatility, every indicator's value / signal / confidence / description
/ effective weight / weighted score, key levels, pattern signals.
4. The arithmetic twins: numpy pairwise sums and `Series.mean` for n = 1…600, `nanmean` with NaNs,
EWM and rolling means restarted on tails, Wilder ATR on tails, `_ext_vs_sma`, the prefix median
with ties, CPython float floor division; and the rolling helper text is the prefix of
`gpu_kernels._ROLL_SRC`.
5. `align_cut` against the replay's `searchsorted` rule on gapped, late-starting M15/H1/H4/D1
frames for M5 and H1 primaries (`cut == 0` included).
6. `host_flags` against `_is_xau_blocked_by_calendar` plus the session block bar by bar across
both 2026 DST shifts, an H4 primary (label hour ≠ close hour), the day filter, and each filter
switched off.
7. K2 against `analyze_from_dataframes` on every analysed bar: the M5 factory over a 21-day
synthetic fixture (4,046 bars: all six grades, GENERIC and XAU_CP setups, both directions, the
mixed `trending` regime with both score signs, 23 distinct warnings); the M5 research knobs
(all six chart patterns, big-candle 3.0, the H4 regime filter, `FB2_EMA_LO=0.9`, grades to D;
every second bar); the M15 factory (1,182 bars, 62 candidates, XAU_ATR and GENERIC, the SELL,
exhaustion-block and spread gates); the H1 factory (shorts blocked) and the H4 factory with a
D1 confirmation shorter than 30 bars on a 30-day fixture. Compared per bar: score, confidence,
signal, grade, agreement, exhaustion count, momentum fading, penalty total and tags, the full
warning and cleaned-warning lists, both reason lists, price, spread estimate, regime,
volatility, the setup (strategy, direction, entry, stop, targets, rounded RR, pattern) and the
replay's candidate fields.
8. The device chart-pattern detector against `detect_chart_pattern` with all six patterns on the
fixture and on its mirror image (bear patterns), zero ATR included; the device big-candle
detector against `_detect_big_candle` (index and ratio) with injected abnormal bars.
9. K4 against `_apply_cooldown` (cooldown 0 / 1 / 5 / 17, empty, a dense run).
10. `Trade` rows, the 65 features, the recorder snapshots and the per-bar float32 matrix against a
real `_replay_span` over the whole M5 fixture with the pullback path re-enabled (XAU_PB, XAU_CP
and GENERIC trades; `dataclasses.asdict` field by field, exact floats, tz-aware entry times, the
CSV frame, dtypes and bytes with the research columns).
11. End to end: `cudaBasedBacktesting.main` against `gpu_backtest.main` on the fixture written to a
temporary cache directory (M15, two weeks): plain book MD5 equal; `--fixed-rr-tp 2.0
--export-features --record DIR --record-bars` MD5 equal and the four parquet tables equal.
12. Refusals: `--backend cpu`, a cpu-forced backend and a failed probe exit before the cache is
touched; each compile-time limit is refused with a message naming the knob and pointing to
`gpu_backtest.py`, the largest supported values pass; the parser is the reference script's;
`require_gpu_or_exit("gpu")` and the `Backend:` text on a working box.
13. Source guards: the script never calls `analyze_from_dataframes` or `_replay_span`, never
imports cupy or a live writer; the kernel module stays a leaf (no `engine`,
`signal_aggregator`, `gpu_backtest`, `replay_record`, live writer or ML import at module level),
pins `options=KERNEL_OPTIONS` (`--fmad=false`) and contains no `fmin(` / `fmax(`; the shared
code tables equal the recorder's; the live roots import neither new module.
On the GPU the module takes about 85 s, almost all of it the CPU references (about 5 ms per bar
through the live signal code). Without CUDA the device tests print a notice and the module takes
about a second.
## 5. Honest notes
- The twin is a second implementation of the decision stage. The MD5 gate proves equality on the
windows and flags tried, not in general; a future signal change that is not mirrored in the
kernels will make the twin drift silently. That is why the instructions addendum and CLAUDE.md
section 7 make every signal change a parity event, and why the reference script stays.
- Speed comes from the batch, not from the per-bar work: the decision kernels run one thread per
bar over the whole window at once. The remaining wall time (about a second per book) is parquet
loading, cleaning, the precompute and the host formatting of the accepted trades.
- `--workers` is accepted and ignored (logged); the twin never forks.
- Nothing in `engine.py`, `signal_aggregator.py`, `chart_patterns.py`, `position_sizing.py`,
`replay_record.py`, `gpu_backtest.py` or `gpu_kernels.py` changed. `git diff --stat` touches
`test_all.py`, `CLAUDE.md` and `backtest/instructions_for_backtest.txt` and adds the two new
modules and this note.
@@ -0,0 +1,639 @@
# Daily-trend veto and wide-stop cap — pre-registered campaign, 2026-09-30
Owner request (2026-09-30): improve the direction call and the trade setup size; anything may
change if the evidence supports it. This file holds the rules first, then the results. Sections 1
to 4 were written before any formal run. Section 2 records what was already looked at while
planning, so the reader knows which numbers are a confirmation and which are a first look.
## 1. What is tested
**Candidate T — daily-trend veto.** The engine sees at most the H1 chart (M5 confirms on M15 and
H1, M15 on M5 and H1). None of the 65 decision-time features of the earlier campaigns reads the
daily chart. The candidate drops a finished trade setup that goes against a clear daily trend.
Definition (fixed; `signal_aggregator.daily_trend_series`):
- daily candles from the provider (UTC-aligned, label = 00:00 UTC of the day), closed bars only:
a bar labelled D is used once D + 1 day ≤ the close of the primary bar;
- bars labelled Sunday are dropped first (Oanda's 1 to 2 hour reopening stub);
- `ema = Close.ewm(span=50, adjust=False).mean()`, `change = Close / Close.shift(60) − 1`;
- state +1 when `Close > ema` and `change > 0`; −1 when `Close < ema` and `change < 0`; else 0;
- after every setup archetype has run (gold ATR, swing, pullback, chart pattern) a setup against a
non-zero state is removed: grade D, no setup, warning `DAILY TREND FILTER: …`.
Switch `xau_strategy.daily_trend_veto`: `off` (default, books byte-identical), `both`, `buy`
(only buys against a falling daily trend are removed), `sell`.
**Candidate C — wide-stop cap.** `xau_strategy.wide_stop_cap_atr = K` caps the stop distance of
the swing (GENERIC) and chart-pattern setups at `max(K × ATR, the existing floor)`, applied after
the min-RR and spread gates so the set of trades cannot change. K = 6, with 5, 7 and 8 declared
as neighbours. M15 only (the exit-only screen was negative on M5).
## 2. What was already seen (planning phase, 2026-09-30, read-only, in memory)
Prototype: K1 and K2 of the CUDA twin, then a host-side mask on the final candidates before the
5-bar cooldown, then the shared resolver. The control rows of the prototype reproduce the pinned
books (M15 W3 1105 / +195.79R, M5 W3 870 / +153.64R, H1 W3 253 / +70.02R).
| Book ($0.65) | Control | Veto, both directions |
|---|---|---|
| M15 W3 2023-09-05 → 2026-09-05 | 1105 / +195.8R / PF 1.49 / DD 20.9 | 824 / +205.1R / PF 1.73 / DD 12.4 |
| M15 LONG 2019-01-01 → 2026-08-07 | 2091 / +192.5R / PF 1.24 / DD 64.8 | 1473 / +223.3R / PF 1.42 / DD 41.5 |
| M15 rising window 2024-06-01 → 2025-03-30 | 313 / +68.5R / PF 1.67 | 258 / +63.8R / PF 1.80 |
| M5 W3 | 870 / +153.6R / PF 1.50 / DD 10.1 | 668 / +159.1R / PF 1.70 / DD 15.2 |
| M5 W3, buy side only | | 741 / +156.5R / PF 1.60 / DD 9.6 |
| M5 LONG | 1052 / +128.1R / PF 1.34 / DD 18.4 | 790 / +127.5R / PF 1.45 / DD 25.2 |
| H1 W3 | 253 / +70.0R / PF 1.98 / DD 4.4 | 231 / +60.3R / PF 1.90 / DD 4.4 |
Also seen: neighbouring definitions on M15 (60-day change alone +188.3R W3 / +228.1R LONG;
90-day +196.1 / +228.8; 120-day +199.4 / +233.2; 100-day average +201.2 / +227.3); the share of
M15 W3 trades reaching +1R before −1R (58.6% → 61.4%); the live September 2026 setups with the
veto applied after the fact (cooldown view −11.6R → −6.0R to −8.8R); an exit-only re-simulation
of 25 stop and exit variants (cap 6 × ATR on M15 W3 +14.0R, LONG +11.8R; cap 5 negative on LONG);
two variants dropped at this stage and NOT tested formally: lifting the SELL gates when the daily
trend is down (worse on LONG on both timeframes), and removing the existing M15 short blocks
under the veto (costs 4 to 22R).
So the windows W3, W2, the rising window and LONG are a CONFIRMATION through the real code path.
First looks are: the HOLDOUT window through the backtester, the $0.30 numbers on LONG, and every
number produced by the shipped code rather than the prototype.
## 3. Harness
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf <tf> --start <s>
--end <e> --spread <0.65|0.30> [--set xau_strategy.daily_trend_veto=<mode>]
[--set xau_strategy.wide_stop_cap_atr=<K>] --export-csv <out>`; factory configuration otherwise.
Twin numbers count only with the paired `backtest/gpu_backtest.py` MD5 (same flags plus
`--workers 12`). Neighbouring trend definitions are not shipped code: they run on the research
harness (the prototype), after the harness book for the shipped definition is shown identical to
the twin's.
Windows: **W3** 2023-09-05 → 2026-09-05 (halves at 2025-03-05); **W2** 2024-09-05 → 2026-09-05
(halves at 2025-09-05); **RISING** 2024-06-01 → 2025-03-30 (the old out-of-sample window; a
rising market throughout); **LONG** 2019-01-01 → 2026-08-07; **HOLDOUT** 2026-09-05 → 2026-10-01
(fetched for this campaign, never seen by any tuning or by the prototype).
## 4. Acceptance rules (written before the runs)
**Rule T (risk-first; owner decision 2026-09-30).** Candidate T on a timeframe PASSES only if,
against the same-harness control:
- (a) W3 total R ≥ 97% of the control's at $0.65;
- (b) W3 profit factor ≥ control + 0.10;
- (c) W3 max drawdown ≤ the control's;
- (d) LONG: total R ≥ control, max drawdown ≤ control, and 2021 + 2022 combined R better;
- (e) HOLDOUT total R ≥ control − 0.5R;
- (f) clauses (a) to (c) also hold at $0.30;
- (g) at least 3 of the 4 neighbouring definitions (60-day change alone, 90-day, 120-day,
100-day average alone) satisfy (a) and (c): a plateau, not a spike;
- (h) at least 20 trades differ on W3.
Reported, NOT clauses (the owner accepted this cost on 2026-09-30): the W2 halves and the RISING
window, where the veto is expected to give up some R. The 2026-09-26 template (every half and the
old out-of-sample window must not lose) would reject this candidate by construction; that is the
difference between the two rules and it is deliberate.
Decision per timeframe: `both` if it passes; else `buy` if that passes; else no veto (the daily
trend is then shown on the alert as information only). H1 is run for the record.
**Rule C (standard).** K = 6 on M15 against the post-T control PASSES only if: W3 ΔR ≥ +5R; both
W3 halves ≥ control − 1R; PF ≥ control − 0.02; W3 max drawdown ≤ control + 10%; LONG total R ≥
control; the same sign of ΔR at $0.30; at least 20 trades differ; and the neighbours K = 5 and
K = 7 are not negative on W3 or on LONG.
No rescue for either rule: no second definition, no other K, no window shopping after the result.
## 5. Results
All runs 2026-09-30 on the dev box, branch `sql-integration`, working tree = HEAD `0c56b4e` plus
this campaign's code. Every twin log reads
`Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` and
`GPU cache fallbacks: 0 (no per-bar recompute path in cudaBasedBacktesting)`; every reference log
reads the same `Backend:` line and `GPU cache fallbacks: 0`. 86 formal twin books (5 windows × 2
costs × the variants below, plus the fixed-1R-target books). R is per trade in units of the
initial risk; "DD" is the largest peak-to-trough fall of the running total, in R.
### 5.1 Proofs before any formal number
- **Baseline.** Before any code change the five reference MD5s of CLAUDE.md §9 were reproduced on
both scripts (ten books).
- **Default-off neutrality.** With the new code and both knobs at their defaults the same ten
books are byte-identical to the pins (`e599c8a8…` M5 W2, `5b65ecf6…` M15 W2, `09b8a8e7…` M15
OOS, `a44fd19b…` H1 W3, `f3587bae…` H4 W3).
- **Twin = reference on every new path** (`--spread 0.65`, W3 2023-09-05 → 2026-09-05, equal MD5
from `backtest/gpu_backtest.py --workers 12` and `backtest/cudaBasedBacktesting.py`):
| Flags | Trades / R / DD | MD5 (both scripts) |
|---|---|---|
| M15 `--set xau_strategy.daily_trend_veto=both` | 824 / +205.12 / 12.39 | `ad3b0dbda0bf77b659caea1c53be6a50` |
| M15 `…=buy` | 999 / +188.21 / 20.87 | `6e0f3b742d9690f78c97319d1b1c8a43` |
| M15 `…=sell` | 930 / +212.70 / 13.72 | `31aa5762964d846f15d111cd4a718adc` |
| M15 `--set xau_strategy.wide_stop_cap_atr=6` | 1105 / +209.78 / 19.32 | `4ec0e4450d5dab143a4ed37e80545626` |
| M5 `--set xau_strategy.daily_trend_veto=both` | 668 / +159.07 / 15.20 | `39cc0f198d38b532720dce4f98d99452` |
| M5 `--set xau_strategy.wide_stop_cap_atr=6` | 870 / +141.57 / 13.95 | `4da0997256b272ffcd11a90bf9658eef` |
The M15 `both` book equals the planning prototype trade for trade (824 / +205.1R / PF 1.73 /
DD 12.4), so the shipped definition is the one that was screened.
- **Suite.** 595 passed / 0 failed after Stage 1 (Module 39 new, Module 37 extended).
### 5.2 Candidate T — every window, both costs
Command per row: `venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf
<15min|5min> --start <s> --end <e> --spread <0.65|0.30> [--set xau_strategy.daily_trend_veto=<mode>]
--export-csv <out>`. "control" = no `--set`.
**M15**
| Window | Cost | Variant | Trades | Total R | Win rate | PF | Max DD | First half | Second half | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|
| W3 | $0.65 | control | 1105 | +195.79 | 61.1% | 1.49 | 20.9 | +76.87 | +118.92 | 70195fb0 |
| W3 | $0.65 | both | 824 | +205.12 | 63.1% | 1.73 | 12.4 | +76.72 | +128.40 | ad3b0dbd |
| W3 | $0.65 | buy | 999 | +188.21 | 61.3% | 1.52 | 20.9 | +60.40 | +127.81 | 6e0f3b74 |
| W3 | $0.65 | sell | 930 | +212.70 | 62.7% | 1.66 | 13.7 | +93.19 | +119.52 | 31aa5762 |
| W3 | $0.30 | control | 1105 | +216.53 | 61.4% | 1.55 | 19.6 | +88.86 | +127.67 | ce074a1b |
| W3 | $0.30 | both | 824 | +221.18 | 63.6% | 1.81 | 11.4 | +86.41 | +134.77 | 96a87454 |
| W3 | $0.30 | buy | 999 | +207.23 | 61.7% | 1.59 | 19.6 | +71.83 | +135.41 | 2ada14e4 |
| W3 | $0.30 | sell | 930 | +230.48 | 63.1% | 1.73 | 12.8 | +103.45 | +127.03 | 49665c41 |
| W2 | $0.65 | control | 813 | +154.15 | 62.2% | 1.53 | 15.6 | +70.10 | +84.04 | 5b65ecf6 |
| W2 | $0.65 | both | 597 | +155.89 | 64.3% | 1.78 | 11.4 | +61.83 | +94.06 | dfc647ee |
| W2 | $0.65 | buy | 718 | +154.67 | 63.0% | 1.62 | 14.4 | +65.03 | +89.65 | f23bfda4 |
| W2 | $0.65 | sell | 692 | +155.36 | 63.3% | 1.65 | 13.7 | +66.91 | +88.46 | 1a071ac7 |
| W2 | $0.30 | control | 813 | +167.53 | 62.6% | 1.59 | 14.5 | +78.61 | +88.91 | 15486607 |
| W2 | $0.30 | both | 597 | +166.22 | 64.8% | 1.85 | 11.0 | +68.82 | +97.39 | d2f678bf |
| W2 | $0.30 | buy | 718 | +166.64 | 63.4% | 1.68 | 14.0 | +73.12 | +93.52 | f3abc1e8 |
| W2 | $0.30 | sell | 692 | +167.10 | 63.7% | 1.71 | 12.8 | +74.31 | +92.79 | d7d038fd |
| RISING | $0.65 | control | 313 | +68.50 | 63.6% | 1.67 | 12.4 | +18.87 | +49.63 | 09b8a8e7 |
| RISING | $0.65 | both | 258 | +63.78 | 64.3% | 1.80 | 9.1 | +23.97 | +39.81 | e87b6de1 |
| RISING | $0.65 | buy | 303 | +60.15 | 63.0% | 1.60 | 12.4 | +18.87 | +41.28 | f001a0a6 |
| RISING | $0.65 | sell | 268 | +72.13 | 64.9% | 1.89 | 9.1 | +23.97 | +48.16 | cbe4cb24 |
| RISING | $0.30 | control | 313 | +76.21 | 64.5% | 1.77 | 11.1 | +22.92 | +53.29 | 5fc34cef |
| RISING | $0.30 | both | 258 | +70.24 | 65.5% | 1.91 | 7.9 | +27.31 | +42.92 | 5de32f51 |
| RISING | $0.30 | buy | 303 | +67.59 | 64.0% | 1.70 | 11.1 | +22.92 | +44.67 | 69b41026 |
| RISING | $0.30 | sell | 268 | +78.86 | 66.0% | 2.00 | 7.9 | +27.31 | +51.54 | 3631466e |
| LONG | $0.65 | control | 2091 | +192.53 | 57.8% | 1.24 | 64.8 | | | c2228398 |
| LONG | $0.65 | both | 1473 | +223.33 | 59.7% | 1.42 | 41.5 | | | aa5253aa |
| LONG | $0.65 | buy | 1762 | +214.62 | 58.9% | 1.33 | 34.5 | | | 952d64d8 |
| LONG | $0.65 | sell | 1802 | +201.23 | 58.3% | 1.30 | 76.4 | | | d3965ab8 |
| LONG | $0.30 | control | 2091 | +240.00 | 58.8% | 1.31 | 49.6 | | | 0c58b300 |
| LONG | $0.30 | both | 1473 | +257.25 | 60.8% | 1.50 | 37.2 | | | 1835b341 |
| LONG | $0.30 | buy | 1762 | +254.33 | 59.9% | 1.40 | 29.5 | | | a2b519bd |
| LONG | $0.30 | sell | 1802 | +242.92 | 59.4% | 1.37 | 62.3 | | | f6edb469 |
| HOLDOUT | $0.65 | control | 24 | −6.63 | 50.0% | 0.42 | 7.0 | | | c42470a0 |
| HOLDOUT | $0.65 | both | 18 | −4.62 | 50.0% | 0.50 | 4.8 | | | fa50bcd0 |
| HOLDOUT | $0.65 | buy | 22 | −6.81 | 45.5% | 0.41 | 7.0 | | | 548a4bf4 |
| HOLDOUT | $0.65 | sell | 20 | −4.44 | 55.0% | 0.52 | 4.8 | | | 1459ef5f |
| HOLDOUT | $0.30 | control | 24 | −6.29 | 50.0% | 0.44 | 6.7 | | | d7fbee9d |
| HOLDOUT | $0.30 | both | 18 | −4.35 | 50.0% | 0.52 | 4.5 | | | 2179c08c |
| HOLDOUT | $0.30 | buy | 22 | −6.49 | 45.5% | 0.43 | 6.7 | | | 0b90faeb |
| HOLDOUT | $0.30 | sell | 20 | −4.15 | 55.0% | 0.54 | 4.5 | | | 70932b7e |
**M5**
| Window | Cost | Variant | Trades | Total R | Win rate | PF | Max DD | First half | Second half | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|
| W3 | $0.65 | control | 870 | +153.64 | 53.3% | 1.50 | 10.1 | +16.73 | +136.91 | d20ec8a3 |
| W3 | $0.65 | both | 668 | +159.07 | 53.9% | 1.70 | 15.2 | +14.01 | +145.06 | 39cc0f19 |
| W3 | $0.65 | buy | 741 | +156.53 | 53.4% | 1.60 | 9.6 | +15.68 | +140.85 | 631f25ac |
| W3 | $0.30 | control | 870 | +172.40 | 54.3% | 1.57 | 8.9 | +21.08 | +151.32 | 5acafcba |
| W3 | $0.30 | both | 668 | +174.43 | 54.8% | 1.79 | 14.3 | +18.09 | +156.34 | 34f2e282 |
| W3 | $0.30 | buy | 741 | +172.72 | 54.3% | 1.69 | 8.9 | +19.87 | +152.86 | c8bdc9e2 |
| W2 | $0.65 | control | 781 | +146.07 | 52.9% | 1.52 | 10.1 | +26.42 | +119.65 | e599c8a8 |
| W2 | $0.65 | both | 585 | +153.82 | 53.7% | 1.78 | 15.2 | +33.16 | +120.66 | b9fa356c |
| W2 | $0.65 | buy | 656 | +151.09 | 53.0% | 1.66 | 9.6 | +27.42 | +123.66 | 30e243b0 |
| W2 | $0.30 | control | 781 | +162.09 | 53.9% | 1.60 | 8.9 | +31.71 | +130.37 | 7ab48db9 |
| W2 | $0.30 | both | 585 | +166.59 | 54.7% | 1.87 | 14.3 | +38.05 | +128.54 | 3427b933 |
| W2 | $0.30 | buy | 656 | +164.66 | 54.0% | 1.74 | 8.9 | +32.58 | +132.08 | 7dd9e2a2 |
| RISING | $0.65 | control | 86 | +9.47 | 53.5% | 1.33 | 8.2 | +0.01 | +9.46 | 65c46e62 |
| RISING | $0.65 | both | 81 | +9.07 | 54.3% | 1.34 | 7.5 | +0.67 | +8.40 | e20fdef2 |
| RISING | $0.65 | buy | 85 | +10.55 | 54.1% | 1.39 | 7.2 | +0.01 | +10.54 | de24b2e8 |
| RISING | $0.30 | control | 86 | +12.42 | 54.7% | 1.46 | 6.7 | +1.65 | +10.77 | 5c154360 |
| RISING | $0.30 | both | 81 | +11.91 | 55.6% | 1.47 | 6.1 | +2.28 | +9.63 | 3b1a8f9e |
| RISING | $0.30 | buy | 85 | +13.46 | 55.3% | 1.52 | 5.7 | +1.65 | +11.81 | 21ffab88 |
| LONG | $0.65 | control | 1052 | +128.10 | 50.5% | 1.34 | 18.4 | | | 3f0c091c |
| LONG | $0.65 | both | 790 | +127.47 | 50.4% | 1.45 | 25.2 | | | 9872c068 |
| LONG | $0.65 | buy | 871 | +123.37 | 49.9% | 1.39 | 27.1 | | | 613dcf0e |
| LONG | $0.30 | control | 1052 | +153.02 | 51.7% | 1.41 | 13.0 | | | 6708d8cc |
| LONG | $0.30 | both | 790 | +147.13 | 51.5% | 1.54 | 19.6 | | | 201d968f |
| LONG | $0.30 | buy | 871 | +144.02 | 51.1% | 1.47 | 22.3 | | | d9cba459 |
| HOLDOUT | $0.65 | control | 33 | −2.45 | 51.5% | 0.82 | 6.1 | | | 351f7149 |
| HOLDOUT | $0.65 | both | 26 | −2.04 | 50.0% | 0.81 | 5.4 | | | b9c8cb82 |
| HOLDOUT | $0.65 | buy | 33 | −2.45 | 51.5% | 0.82 | 6.1 | | | 351f7149 |
| HOLDOUT | $0.30 | control | 33 | −1.85 | 51.5% | 0.86 | 5.6 | | | 5a6aea9c |
| HOLDOUT | $0.30 | both | 26 | −1.56 | 50.0% | 0.85 | 5.0 | | | ddd29068 |
| HOLDOUT | $0.30 | buy | 33 | −1.85 | 51.5% | 0.86 | 5.6 | | | 5a6aea9c |
**H1, for the record** ($0.65): W3 control 253 / +70.02R / PF 1.98 / DD 4.4 (`a44fd19b`), `both`
231 / +60.28R / PF 1.90 / DD 4.4 (`cdac563b`); HOLDOUT 3 / −3.04R in both (no trade removed).
Fails (a) (60.28 < 67.92) and (b). H4 was not run: it confirms on the daily chart already.
### 5.3 Per calendar year ($0.65), total R / trades
M15, LONG 2019-01-01 → 2026-08-07:
| Variant | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
|---|---|---|---|---|---|---|---|---|
| control | +9.0 / 117 | +48.8 / 283 | −25.6 / 210 | −24.6 / 267 | +11.7 / 211 | +53.3 / 343 | +81.2 / 425 | +38.7 / 235 |
| both | +14.0 / 88 | +45.1 / 207 | −27.3 / 121 | +3.2 / 149 | +1.8 / 163 | +53.5 / 275 | +92.5 / 341 | +40.6 / 129 |
| buy | +7.6 / 111 | +44.2 / 253 | −21.4 / 135 | +7.4 / 166 | +1.6 / 189 | +46.4 / 335 | +83.1 / 416 | +45.9 / 157 |
| sell | +15.4 / 94 | +49.7 / 237 | −31.5 / 196 | −28.8 / 250 | +11.9 / 185 | +60.4 / 283 | +90.6 / 350 | +33.4 / 207 |
M5, LONG:
| Variant | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
|---|---|---|---|---|---|---|---|---|
| control | −1.0 / 20 | −2.8 / 95 | −1.7 / 32 | −2.2 / 50 | −2.8 / 61 | +1.0 / 102 | +40.5 / 332 | +97.0 / 360 |
| both | +0.2 / 17 | −6.9 / 76 | −3.4 / 14 | −4.7 / 28 | −3.5 / 54 | +1.5 / 96 | +43.2 / 297 | +101.0 / 208 |
| buy | +0.2 / 17 | −6.7 / 83 | −3.2 / 16 | −6.0 / 30 | −4.6 / 56 | +2.1 / 101 | +40.5 / 329 | +101.0 / 239 |
M15, W3 per calendar year (total R / trades / max DD inside the year):
| Variant | 2023 (from 09-05) | 2024 | 2025 | 2026 (to 09-05) |
|---|---|---|---|---|
| control | +10.8 / 66 / 9.0 | +53.3 / 343 / 16.3 | +81.2 / 425 / 15.6 | +50.5 / 271 / 10.3 |
| both | +9.3 / 47 / 6.9 | +53.5 / 275 / 11.0 | +92.5 / 341 / 11.4 | +49.9 / 161 / 8.3 |
M15, R per month from 2025-09 (W3):
| Variant | 2025-09 | 10 | 11 | 12 | 2026-01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 (to 09-05) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| control | +14.7 | −1.4 | +3.7 | +19.4 | +29.1 | −2.2 | +9.5 | +4.4 | +0.0 | +1.7 | −4.6 | +12.4 | +0.2 |
| both | +19.4 | +0.6 | +8.4 | +18.7 | +32.4 | −2.4 | +1.5 | +7.2 | −2.4 | +5.1 | +2.3 | +5.9 | +0.2 |
Losing runs on M15: longest run of losing trades 9 → 8 (W3 and LONG); worst calendar month
−10.1R → −6.1R (W3, 2024-01) and −11.7R → −10.0R (LONG); losing months 11 → 8 of 37 on W3,
39 of 92 → 37 of 91 on LONG.
### 5.4 Rule T, clause by clause
**M15 `both` — PASS**
| Clause | Result | Numbers (candidate vs control) |
|---|---|---|
| (a) W3 R ≥ 97% of control | PASS | +205.12 vs +195.79 (97% = +189.92) |
| (b) W3 PF ≥ control + 0.10 | PASS | 1.734 vs 1.488 |
| (c) W3 max DD ≤ control | PASS | 12.39 vs 20.87 |
| (d) LONG R ≥ control | PASS | +223.33 vs +192.53 |
| (d) LONG max DD ≤ control | PASS | 41.54 vs 64.77 |
| (d) 2021 + 2022 better | PASS | −24.12 vs −50.21 |
| (e) HOLDOUT R ≥ control − 0.5 | PASS | −4.62 (18 trades) vs −6.63 (24) |
| (f) (a) at $0.30 | PASS | +221.18 vs +216.53 (97% = +210.04) |
| (f) (b) at $0.30 | PASS | 1.809 vs 1.551 |
| (f) (c) at $0.30 | PASS | 11.44 vs 19.56 |
| (g) ≥ 3 of 4 neighbours pass (a) and (c) | PASS | 3 of 4 (table 5.5) |
| (h) ≥ 20 trades differ on W3 | PASS | 281 removed, 0 added |
**M15 `buy` — FAIL**: (a) +188.21 < +189.92; (b) 1.523 < 1.588; the same two at $0.30.
**M15 `sell` — FAIL**: (d) LONG max DD 76.39 > 64.77 and 2021 + 2022 −60.29 < −50.21. It has the
best W3 total (+212.70R) and that is exactly the trap the long window was there to catch: in the
falling years the buys against the daily trend are the losers, and `sell` keeps them.
**M5 `both` — FAIL**: (c) W3 DD 15.20 > 10.11 (also at $0.30: 14.30 > 8.88); (d) LONG R
+127.47 < +128.10, LONG DD 25.25 > 18.39, 2021 + 2022 −8.02 < −3.91; (g) 0 of 4.
**M5 `buy` — FAIL**: (d) LONG R +123.37 < +128.10, LONG DD 27.14 > 18.39, 2021 + 2022 −9.13 <
−3.91; (g) 1 of 4.
**H1 `both` — FAIL** (a), (b).
### 5.5 Clause (g): neighbouring definitions (research harness)
The harness (K1 + K2 of the twin with the veto OFF, a host-side mask on the final candidates
before the cooldown, the shared resolver) reproduces the twin's `--set` books byte for byte for
the shipped definition, so it is the same thing as the shipped path for a final veto:
| Book ($0.65) | Harness CSV MD5 | Twin CSV MD5 |
|---|---|---|
| M15 W3 control | `70195fb0e6865b3a46ee8a9aa8844e51` | same |
| M15 W3 `both` | `ad3b0dbda0bf77b659caea1c53be6a50` | same |
| M15 LONG control | `c222839878f791e980b2dbc99fb6dfbd` | same |
| M15 LONG `both` | `aa5253aab206fd21fa6a8d5ba25b75b5` | same |
| M5 W3 control / `both` / `buy` | `d20ec8a3…` / `39cc0f19…` / `631f25ac…` | same |
| M5 LONG control / `both` / `buy` | `3f0c091c…` / `9872c068…` / `613dcf0e…` | same |
M15, veto in both directions, each neighbour replacing the definition of "clear daily trend":
| Definition | W3 trades | W3 R | W3 PF | W3 DD | (a) | (c) | LONG trades | LONG R | LONG PF | LONG DD |
|---|---|---|---|---|---|---|---|---|---|---|
| control | 1105 | +195.79 | 1.49 | 20.87 | (97% = +189.92) | | 2091 | +192.53 | 1.24 | 64.77 |
| **shipped: 50-day average AND 60-day change** | 824 | +205.12 | 1.73 | 12.39 | PASS | PASS | 1473 | +223.33 | 1.42 | 41.54 |
| 60-day change alone | 752 | +188.25 | 1.73 | 11.89 | FAIL | PASS | 1328 | +228.11 | 1.49 | 27.30 |
| 90-day change alone | 782 | +196.14 | 1.72 | 12.46 | PASS | PASS | 1412 | +228.75 | 1.46 | 37.36 |
| 120-day change alone | 781 | +199.36 | 1.75 | 12.26 | PASS | PASS | 1397 | +233.19 | 1.47 | 34.79 |
| 100-day average alone | 796 | +201.23 | 1.74 | 12.26 | PASS | PASS | 1375 | +227.30 | 1.47 | 32.56 |
3 of 4 pass. Every neighbour beats the control on PF and on drawdown in both windows; the one
miss is 1.7R under the 97% line. It is a plateau. (Each neighbour has a lower LONG drawdown than
the shipped definition; switching to one now would be the rescue the rule forbids.)
M5: `both` 0 of 4 (every neighbour keeps the 15.2R drawdown), `buy` 1 of 4 (90-day, 120-day and
100-day average all fall under the 97% line). No plateau on M5.
### 5.6 What the veto removes (M15, $0.65)
Zero trades are added in any window (W3 and LONG, every variant, both timeframes): the setups
inside a vetoed setup's 5-bar cooldown point the same way under the same daily state and are
vetoed with it, so the cooldown displaces nothing.
| Slice | W3 trades | W3 R | W3 mean R | LONG trades | LONG R | LONG mean R |
|---|---|---|---|---|---|---|
| all removed | 281 | −9.32 | −0.033 | 618 | −30.80 | −0.050 |
| BUY against a falling daily trend | 106 | +7.59 | +0.072 | 329 | −22.10 | −0.067 |
| SELL against a rising daily trend | 175 | −16.91 | −0.097 | 289 | −8.70 | −0.030 |
| removed, grade B | 210 | −11.53 | −0.055 | 408 | −25.69 | −0.063 |
| removed, grade A | 46 | +1.28 | +0.028 | 164 | −1.12 | −0.007 |
| removed, grade A+ | 14 | +4.26 | +0.304 | 26 | +1.81 | +0.069 |
| removed, grade C | 11 | −3.34 | −0.304 | 20 | −5.80 | −0.290 |
| kept (the new book) | 824 | +205.12 | +0.249 | 1473 | +223.33 | +0.152 |
| kept BUY | 763 | +194.20 | +0.255 | 1302 | +222.50 | +0.171 |
| kept SELL | 61 | +10.91 | +0.179 | 171 | +0.83 | +0.005 |
Reading: the removed trades are close to break-even after costs (−0.03 to −0.05R each) while the
kept trades earn +0.15 to +0.25R each. The veto does not find big losers; it drops about a quarter
of the trades, the ones with no edge, and with them the long flat-to-down stretches. The two
halves of the veto are carried by different windows: the SELL half by the rising 3-year window,
the BUY half by the falling years of the long window. Only the symmetric rule holds on both.
Share of trades that reach +1R before −1R (same entries, fixed 1R target, `--fixed-rr-tp 1.0`,
$0.65): M15 W3 58.6% → 61.4% (1105 → 824 trades), M15 LONG 55.9% → 58.2%; M5 W3 56.8% → 58.2%,
M5 LONG 55.3% → 56.1%.
Checklist breakdowns, M15 W3 → with the veto (trades / R), $0.65:
| Slice | Control | Veto | Change in R |
|---|---|---|---|
| regime ranging | 140 / +46.75 | 106 / +49.23 | +2.48 |
| regime transitioning | 264 / +47.05 | 199 / +49.47 | +2.42 |
| regime trending_down | 165 / −4.39 | 43 / +9.71 | +14.10 |
| regime trending_up | 536 / +106.38 | 476 / +96.70 | −9.68 |
| volatility extreme | 508 / +103.91 | 361 / +95.86 | −8.05 |
| volatility high | 228 / +4.00 | 135 / +18.32 | +14.32 |
| volatility normal | 202 / +58.96 | 177 / +57.01 | −1.96 |
| volatility low | 167 / +28.92 | 151 / +33.94 | +5.01 |
| BUY | 869 / +201.79 | 763 / +194.20 | −7.59 |
| SELL | 236 / −6.00 | 61 / +10.91 | +16.91 |
| setup GENERIC | 1004 / +184.73 | 757 / +192.15 | +7.42 |
| setup XAU_ATR | 101 / +11.07 | 67 / +12.97 | +1.90 |
| Monday | 188 / +36.54 | 145 / +52.46 | +15.92 |
| Tuesday | 240 / +44.89 | 186 / +41.82 | −3.07 |
| Wednesday | 234 / +12.08 | 165 / +13.36 | +1.28 |
| Thursday | 216 / +63.84 | 157 / +70.36 | +6.51 |
| Friday | 214 / +33.51 | 162 / +21.04 | −12.47 |
| Sunday evening | 13 / +4.94 | 9 / +6.08 | +1.14 |
By entry hour (UTC), change in R on W3 / on LONG: 03 +0.3 / +1.5; 04 −2.1 / −0.1; 05 −0.2 / +2.6;
06 +1.9 / −1.5; 07 +2.9 / +0.9; 08 −5.3 / −0.3; 09 −6.6 / −4.0; 12 −2.9 / +2.7; 14 +11.8 / +12.9;
15 −4.0 / +1.8; 17 −0.5 / −2.4; 18 +11.4 / +11.3; 19 +0.9 / +2.7; 20 +0.8 / +2.9; 22 +0.8 / −0.1.
The weekday and hour changes are not stable in sign between the two windows except hours 14 and 18
(positive in both) and hour 9 (negative in both); none of it is a reason to act.
### 5.7 Verdict T
- **M15: `both` PASSES every clause and SHIPS** (`pair_signals/xauusd.py`, M15 override 7).
- **M5: no veto.** `both` deepens the worst losing run (10.1R → 15.2R) and loses the long window;
`buy` passes the 3-year window and fails the long one, with no plateau. The M5 book earns
almost all of its R in 2025–2026 and the daily trend adds nothing reliable to it.
- **H1 / H4: no veto.** H1 loses 10R of 70R; H4 already confirms on the daily chart.
- On the timeframes without the veto the daily trend is shown on the alert as information only.
What the owner gives up, as agreed: in the purely rising stretch the veto costs R. W2 first half
(2024-09 → 2025-09) +61.83R vs +70.10R; the RISING window +63.78R vs +68.50R; W2 at $0.30
+166.22R vs +167.53R. What it buys: the worst losing run on the 3-year window falls from 20.9R to
12.4R, on the 7.6-year window from 64.8R to 41.5R, 2022 turns from −24.6R to +3.2R, and the three
weeks after the test window (HOLDOUT, never seen by any tuning) lose 4.6R instead of 6.6R. 2021
does not improve (−27.3R vs −25.6R) and 2023 is worse (+1.8R vs +11.7R): in a year that chops
sideways the daily trend flips late and the veto cannot help.
### 5.8 Candidate C — wide-stop cap on M15 (post-T control)
Control = the factory configuration after the M15 flip (daily-trend veto on). Command per row:
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 15min --start <s>
--end <e> --spread <0.65|0.30> [--set xau_strategy.wide_stop_cap_atr=<K>] --export-csv <out>`.
Paired proof on this control: W3, K = 6, `gpu_backtest.py --workers 12` and the twin both give
`f94f1429fb4c9172427ea34c2f4d92a1` (824 / +216.87R / DD 11.37).
| Window | Cost | K | Trades | Total R | Win rate | PF | Max DD | First half | Second half | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|
| W3 | $0.65 | off | 824 | +205.12 | 63.1% | 1.73 | 12.4 | +76.72 | +128.40 | ad3b0dbd |
| W3 | $0.65 | 5 | 824 | +211.26 | 61.4% | 1.74 | 15.0 | +81.77 | +129.48 | 055d72a4 |
| W3 | $0.65 | **6** | 824 | +216.87 | 62.9% | 1.78 | 11.4 | +80.67 | +136.20 | f94f1429 |
| W3 | $0.65 | 7 | 824 | +213.15 | 63.1% | 1.76 | 11.8 | +79.08 | +134.07 | 5ea4ab4b |
| W3 | $0.65 | 8 | 824 | +208.01 | 62.9% | 1.74 | 12.3 | +77.18 | +130.84 | 372ac298 |
| W3 | $0.30 | off | 824 | +221.18 | 63.6% | 1.81 | 11.4 | +86.41 | +134.77 | 96a87454 |
| W3 | $0.30 | 5 | 824 | +228.46 | 62.1% | 1.82 | 14.0 | +92.27 | +136.18 | 7453b5f7 |
| W3 | $0.30 | **6** | 824 | +233.37 | 63.6% | 1.86 | 11.0 | +90.67 | +142.70 | bf7e4157 |
| W3 | $0.30 | 7 | 824 | +229.37 | 63.7% | 1.84 | 11.0 | +88.88 | +140.48 | e44cb5b5 |
| W3 | $0.30 | 8 | 824 | +224.12 | 63.5% | 1.82 | 11.3 | +86.91 | +137.22 | c214455b |
| W2 | $0.65 | off | 597 | +155.89 | 64.3% | 1.78 | 11.4 | +61.83 | +94.06 | dfc647ee |
| W2 | $0.65 | 5 | 597 | +156.59 | 63.3% | 1.79 | 11.4 | +62.63 | +93.96 | 5e5c0207 |
| W2 | $0.65 | **6** | 597 | +160.79 | 64.0% | 1.81 | 11.4 | +61.31 | +99.48 | 3614c98a |
| W2 | $0.65 | 7 | 597 | +158.70 | 64.3% | 1.79 | 11.4 | +60.19 | +98.51 | afa7eb01 |
| W2 | $0.65 | 8 | 597 | +156.58 | 64.2% | 1.78 | 11.4 | +60.54 | +96.04 | ba29cc41 |
| RISING | $0.65 | off | 258 | +63.78 | 64.3% | 1.80 | 9.1 | +23.97 | +39.81 | e87b6de1 |
| RISING | $0.65 | 5 | 258 | +57.68 | 61.6% | 1.71 | 8.0 | +22.85 | +34.83 | 295731a3 |
| RISING | $0.65 | **6** | 258 | +58.03 | 64.0% | 1.72 | 8.6 | +23.84 | +34.19 | 8dba71d7 |
| RISING | $0.65 | 7 | 258 | +62.18 | 64.3% | 1.78 | 8.8 | +26.44 | +35.74 | c461380f |
| RISING | $0.65 | 8 | 258 | +62.12 | 63.6% | 1.77 | 9.1 | +24.06 | +38.07 | fce21577 |
| LONG | $0.65 | off | 1473 | +223.33 | 59.7% | 1.42 | 41.5 | | | aa5253aa |
| LONG | $0.65 | 5 | 1473 | +204.76 | 58.4% | 1.38 | 53.1 | | | b365f28f |
| LONG | $0.65 | **6** | 1473 | +233.56 | 59.7% | 1.44 | 47.2 | | | 4aa12918 |
| LONG | $0.65 | 7 | 1473 | +228.83 | 59.8% | 1.43 | 46.5 | | | 6e91e0dd |
| LONG | $0.65 | 8 | 1473 | +223.79 | 59.7% | 1.42 | 43.2 | | | 36267a43 |
| LONG | $0.30 | off | 1473 | +257.25 | 60.8% | 1.50 | 37.2 | | | 1835b341 |
| LONG | $0.30 | 5 | 1473 | +241.64 | 59.8% | 1.46 | 44.9 | | | 6a346020 |
| LONG | $0.30 | **6** | 1473 | +268.63 | 60.8% | 1.52 | 41.6 | | | 526ffa4a |
| LONG | $0.30 | 7 | 1473 | +263.16 | 60.9% | 1.51 | 42.1 | | | 3f42a60e |
| LONG | $0.30 | 8 | 1473 | +257.84 | 60.8% | 1.50 | 38.9 | | | ffbba3e4 |
| HOLDOUT | $0.65 | off, 6, 7, 8 | 18 | −4.62 | 50.0% | 0.50 | 4.8 | | | fa50bcd0 |
| HOLDOUT | $0.65 | 5 | 18 | −4.62 | 50.0% | 0.50 | 4.8 | | | 3ff32a0d |
(W2 and RISING at $0.30 and the HOLDOUT at $0.30 are in the same direction as the $0.65 rows:
W2 +171.35 vs +166.22 for K = 6; RISING +64.66 vs +70.24; HOLDOUT unchanged.)
LONG per calendar year ($0.65), total R (trades are identical: 88 / 207 / 121 / 149 / 163 / 275 /
341 / 129):
| K | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 |
|---|---|---|---|---|---|---|---|---|
| off | +14.0 | +45.1 | −27.3 | +3.2 | +1.8 | +53.5 | +92.5 | +40.6 |
| 5 | +19.0 | +27.9 | −32.4 | −1.0 | +2.5 | +63.1 | +89.2 | +36.6 |
| 6 | +19.5 | +46.7 | −31.8 | +1.8 | +4.7 | +59.8 | +88.1 | +44.8 |
| 7 | +16.5 | +45.4 | −32.2 | +4.3 | +1.8 | +59.2 | +91.1 | +42.7 |
| 8 | +14.5 | +43.9 | −29.0 | +3.4 | +2.9 | +54.4 | +92.3 | +41.4 |
Trades the cap touches (the trade set is identical by construction; only stops, and through them
results, move). All of them are swing (GENERIC) setups; the gold ATR setup is never capped:
| Window | K | Stops changed | R of those trades: control → capped |
|---|---|---|---|
| W3 | 5 / 6 / 7 / 8 | 281 / 168 / 89 / 44 | +66.4 → +72.5 / +49.2 → +60.9 / +25.6 → +33.6 / +12.2 → +15.1 |
| LONG | 5 / 6 / 7 / 8 | 597 / 350 / 170 / 83 | +97.5 → +78.9 / +56.7 → +66.9 / +37.0 → +42.5 / +12.5 → +13.0 |
| RISING | 5 / 6 / 7 / 8 | 97 / 58 / 25 / 12 | +28.1 → +22.0 / +21.0 → +15.3 / +13.1 → +11.5 / +5.7 → +4.0 |
| HOLDOUT | 5 / 6 / 7 / 8 | 3 / 0 / 0 / 0 | −3.03 → −3.04 |
**Rule C, K = 6**
| Clause | Result | Numbers |
|---|---|---|
| W3 change in R ≥ +5R | PASS | +11.75 (+216.87 vs +205.12) |
| W3 first half ≥ control − 1R | PASS | +80.67 vs +76.72 |
| W3 second half ≥ control − 1R | PASS | +136.20 vs +128.40 |
| W3 PF ≥ control − 0.02 | PASS | 1.777 vs 1.734 |
| W3 max DD ≤ control + 10% | PASS | 11.37 vs 12.39 |
| LONG total R ≥ control | PASS | +233.56 vs +223.33 (but max DD 47.19 vs 41.54) |
| same sign at $0.30 | PASS | +12.19 (+233.37 vs +221.18) |
| ≥ 20 trades differ | PASS | 168 stops and results changed |
| neighbour K = 5 not negative on W3 | PASS | +6.14 |
| **neighbour K = 5 not negative on LONG** | **FAIL** | **−18.57** |
| neighbour K = 7 not negative on W3 | PASS | +8.04 |
| neighbour K = 7 not negative on LONG | PASS | +5.50 |
**Verdict C: FAIL. Nothing ships; `wide_stop_cap_atr` stays 0.0 (research knob only).** K = 6 is a
narrow peak, not a plateau: one step tighter loses 18.6R over the long window (2020 alone −17R),
one step wider keeps half the gain, two steps wider keeps none. The cap also costs R in the rising
window at every K (−1.6R to −6.1R), deepens the long-window drawdown at every K (41.5R → 43.2R to
53.1R) and changes nothing in the unseen three weeks. This agrees with the planning-phase screen
of 25 stop and exit variants: the stop geometry is at a plateau, and tightening it is the one
direction that reliably loses. No second K is tried (the rule's no-rescue clause).
### 5.9 What shipped, and the re-pinned books
Shipped: `pair_signals/xauusd.py` M15 override 7, `config.xau_strategy.daily_trend_veto = "both"`
on M15 only. Not shipped: any veto on M5 / H1 / H4; the wide-stop cap (knob kept at 0.0).
Reference books after the flip, factory configuration, `--spread 0.65`, regenerated with BOTH
scripts (`gpu_backtest.py --workers 12` and `cudaBasedBacktesting.py`), `Backend: CuPy GPU
(NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, `GPU cache fallbacks: 0`:
| Book | Window | Trades / R / PF / DD | MD5 (reference = twin) | |
|---|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | 781 / +146.07 / 1.52 / 10.1 | `e599c8a8652f95b7fcdf5a7ef2b564f5` | unchanged |
| M15 W2 | 2024-09-05 → 2026-09-05 | 597 / +155.89 / 1.78 / 11.4 | `dfc647ee02afd179dd9e031f344ed380` | re-pinned (was `5b65ecf6…`, 813 / +154.15 / 1.53) |
| M15 OOS | 2024-06-01 → 2025-03-30 | 258 / +63.78 / 1.80 / 9.1 | `e87b6de12f90f3b5a5d41a953ee9ff7a` | re-pinned (was `09b8a8e7…`, 313 / +68.50 / 1.67) |
| H1 W3 | 2023-09-05 → 2026-09-05 | 253 / +70.02 / 1.98 / 4.4 | `a44fd19b4067f46e5bd98d50cc3eff92` | unchanged |
| H4 W3 | 2023-09-05 → 2026-09-05 | 114 / +21.7 / 1.59 | `f3587bae3c5dc3eda468ea3588b17dcb` | unchanged |
| M15 W3 | 2023-09-05 → 2026-09-05 | 824 / +205.12 / 1.73 / 12.4 | `ad3b0dbda0bf77b659caea1c53be6a50` | was `70195fb0…`, 1105 / +195.79 / 1.49 |
The factory books are byte-identical to the `--set xau_strategy.daily_trend_veto=both` books of
section 5.2, as they must be.
### 5.10 A fetch bug found on the way (fixed, not a signal change)
The first HOLDOUT and RISING runs with the daily frame failed with `OandaDataError: XAU/USD 1d:
3 consecutive failures fetching window 2020-08-01–2020-08-01 … (Empty candle response)`. Since
2026-09-13 `fetch_range` retries a failing page and aborts after three tries, while the Oanda
provider raised on an EMPTY date window. A padded window that starts on a closed day (the daily
pad is 1,400 days = exactly 200 weeks, so a window starting on a Saturday pads to a Saturday)
begins with a page nothing traded in, and the whole fetch aborted. Fix in `providers/oanda.py`:
a DATE-WINDOW request with no candles returns an empty frame (a `count` request, the live path,
still raises); two Module 15 tests. Books are unaffected (every pinned window was already cached;
the ten MD5s reproduce). New cache files fetched for this campaign: the daily frames
`oanda_XAU_USD_1d_20201105_20260905`, `_20150303_20260807`, `_20200801_20250330`,
`_20221105_20261001`, and the HOLDOUT window `5min` / `15min` / `1h` `_20260705_20261001` and
`4h_20260114_20261001`. The two interior gaps the HOLDOUT fetch logged are market closures
(Labor Day 2026-09-07, Good Friday 2026-04-02), not dropped pages.
### 5.11 Setup size: what the evidence supports, and the display lines that shipped
"Trade setup size" was tested two ways.
**The stop itself.** Candidate C (section 5.8) fails its rule, and the planning-phase screen of
25 stop and exit variants found nothing better than the shipped geometry (a flat 2 × ATR stop on
M15 turns +196R into +3R). The stops stay as they are.
**How much is at risk at once.** If every alert is taken, the plans pile up. Shipped books, W3,
$0.65 (`M15` = veto on):
| Books taken together | Trades | Same-direction plans already open at an entry: mean | median | max | Entries with ≥ 1 open | with ≥ 3 open | Open at once while anything is open: mean | max | Share of the time something is open |
|---|---|---|---|---|---|---|---|---|---|
| M15 alone | 824 | 1.17 | 1 | 6 | 62% | 15% | 1.70 | 7 | 41% |
| M5 alone | 870 | 0.69 | 0 | 6 | 40% | 7% | 1.36 | 7 | 23% |
| M5 + M15 | 1694 | 1.70 | 1 | 12 | 66% | 26% | 1.97 | 13 | 51% |
| M5 + M15 + H1 | 1947 | 2.75 | 2 | 15 | 78% | 43% | 2.60 | 16 | 65% |
The stacked entries are not worse trades (M5 + M15, by the number of same-direction plans already
open at the entry: 0 → +0.154R per trade over 577; 1 → +0.218R over 410; 2 → +0.241R over 271;
3–4 → +0.282R over 267; 5 or more → +0.236R over 169). So there is no evidence for skipping them,
and the 2026-09-06 sizing campaign already showed that varying the stake by signal features does
not pay. What the numbers do say is that "1% per alert" is in practice several percent on one move
of gold. That is a sizing fact for the reader, so it ships as information, not as a gate:
- **Size line** (`exposure.py`, off until `config.json` `trade_size.risk_per_trade` is set): the
number of lots that keeps the loss at the printed stop inside the owner's risk limit, rounded
DOWN to the lot step. Private by default (owner chats and the CLI); the channel and the public
dashboard get it only with `trade_size.telegram_channel` / `trade_size.dashboard`.
- **Already-open line**: how many ALERTED plans in the same direction are still open in the
advisory outcome model (`signal_log.open_alerted_setups`, a read-only query; the bot stamps
`alerted_at` on a row once its alert reached a chat).
- **Daily-trend line** on every surface (`main.daily_trend_words`): on M15 the veto has already
removed a plan against a clear daily trend; on M5 / H1 / H4, where the veto failed its rule, the
line is context ("Daily trend: falling — this buy goes against it"). The live wrapper stamps the
trend on every logged scan row (`signals.daily_trend`), so the live record can answer the M5
question again after a few months.
None of the three is read by a gate, the backtester or the twin.
### 5.12 Final verification (2026-10-01)
- `venv/bin/python test_all.py`: **607 passed, 0 failed** (40 modules: Module 39 daily-trend veto
and wide-stop cap, Module 40 the display lines, Module 37 extended, two Module 15 tests for the
empty-window fetch).
- The five reference books regenerated with both scripts after the last code change: ten MD5s
equal the table in section 5.9.
- Recorder and feature export with the veto on, M15 2024-06-01 → 2025-03-30, `--export-features
--record DIR --record-bars` on both scripts: feature CSV `732f396bb3ee6036b62750a8c755194e` on
both; `trades` 258 × 137, `indicators` 5900 × 15, `events` 5098 × 14 and `bars` 12315 × 172
identical; both manifests carry book MD5 `e87b6de12f90f3b5a5d41a953ee9ff7a` and differ only in
`cli`, `created_at`, `replay_seconds`, `workers`; `replay_scorecard.py DIR --check`: all
invariants held.
- `agent_panel_base_rates.json` rebuilt from the re-pinned books (M15 timeframe row 597 trades,
64.3% winners, +0.261R; was 813 / 62.2% / +0.190R).
### 5.13 Independent verification (2026-10-01)
An agent that had not done the work re-derived the change list from `git diff HEAD` and
re-ran the books itself (24 on the twin, 9 on the reference script, all offline, 0 cache
fallbacks). Every figure of sections 5.2 to 5.9 it was given matched to the cent, on both
scripts where it ran both: the M15 books with the veto on and off (W3, W2, the 2024-06-01 →
2025-03-30 window, LONG with its per-year R, the holdout), the unchanged M5 / H1 / H4 pins, the
failed M5, H1, `buy` and `sell` variants, and the removed-trade tables (281 removed on W3, 0
added, all 63 columns of the 824 kept trades identical). Its own implementation of the rule
from closed daily bars predicts the removed trades exactly (281 / 281, 618 / 618, 55 / 55); the
same rule fed the still-forming day mis-predicts 52, 129 and 14 of them, so there is no
look-ahead. The scoring configuration is identical to HEAD on all four timeframes; the only
decision-side difference is the M15 veto.
It also raised four points this report did not make clearly enough. They change no verdict;
they are recorded so the size of the claim is not overstated.
1. **Clause (g) is thinner than "3 of 4" reads.** The "100-day average alone" neighbour of
table 5.5 is an EXPONENTIAL average, the same kind as the shipped 50-day one (harness key
`e100`). With a SIMPLE 100-day average the M15 W3 book is 781 / +188.68R / PF 1.70 /
drawdown 12.27R: it beats the control on profit factor and drawdown like the others, but
sits 1.2R under the 97% line (+189.92R), so with that reading the count would be 2 of 4.
The four definitions were fixed before the runs in the harness's own form and the clause
passes as written; the honest summary is "five alternative definitions, all better than the
control on profit factor and drawdown, two of them 1.2R and 1.7R short on total R".
2. **"The veto never adds a trade" is an observation, not a guarantee.** The rule itself only
removes a plan. In a book, removing an entry also moves the 5-bar cooldown, which could
free a later entry; that happened 0 times in every control / candidate pair compared (13).
3. **It is a risk filter, not a better direction call.** Nothing in how direction is scored
changed, and a vetoed bar still shows its buy or sell reading. The removed plans were close
to break-even (−0.03R each on W3, −0.05R on LONG; 55% winners against 63% for the kept
ones), so total R barely moves while profit factor and the losing runs improve.
4. **The help is concentrated.** Effect of the rule by year on LONG (shipped minus off):
2019 +4.9R, 2020 −3.7R, 2021 −1.8R, 2022 **+27.8R**, 2023 **−10.0R**, 2024 +0.1R, 2025
+11.4R, 2026 +1.9R; total +30.8R, of which 2022 (buys against a falling daily trend) is
nine tenths. On W3 the whole gain in R comes from sells removed in 2025, and the drawdown
cut from 20.9R to 12.4R is one episode: 13 sells between 2023-12-01 and 2024-02-23
(−8.5R). In a year where the daily trend flips often (2023) the rule costs. The multi-year
windows had been looked at with the prototype before rule T was written (section 2), and
the only period nobody had seen is four weeks long and differs by six trades.
Cross-checked here from the formal books (a filter on the control book reproduces the shipped
book and the exponential neighbour exactly, and gives the simple-average figure above).
## 6. Summary of decisions
| Item | Verdict |
|---|---|
| Daily-trend veto, M15, both directions | **PASSED rule T on every clause, SHIPPED** (factory M15 override 7) |
| Daily-trend veto, M15, buy-only / sell-only | FAILED (W3 R and PF / LONG drawdown and 2021–22) |
| Daily-trend veto, M5 (both, buy-only) | FAILED (W3 drawdown; LONG R, drawdown, 2021–22; no plateau) |
| Daily-trend veto, H1 | FAILED (W3 −9.7R); H4 not run |
| Wide-stop cap K = 6 on M15 | FAILED rule C (neighbour K = 5 loses 18.6R on LONG); knob kept at 0.0, research-only |
| Lot-size, already-open and daily-trend lines | SHIPPED, display only (books byte-identical) |
| Oanda empty date window | FIXED (fetch_range no longer aborts on a closed-day page) |
| Seven further context screens | no state nominated, nothing shipped (`context_screens_2026-10.md`) |
| Eight outside data sources (dollar, silver, bonds, real yield, fund flows, futures and Oanda client positioning) | none adds a forecast to gold's own daily trend; broad dollar index on M5 = WATCH; nothing shipped, no feed added (`new_inputs_2026-10.md`) |
+137
View File
@@ -0,0 +1,137 @@
# GPU backend made real — 2026-09-18
Behaviour-neutral change. No signal logic, weight, gate or exit rule moved; the proof is the ten
MD5s below (five reference books on the GPU, five with `--backend cpu`), all equal to the pins in
CLAUDE.md section 9. The books therefore carry the same trades, R, win rate, profit factor and
drawdown, in both halves and at both cost conventions, as the pinned books do.
## 1. What was wrong
`backtest/gpu_backtest.py` probed CuPy at import with a `cumsum` and swallowed any exception.
On the dev box the first kernel compile raised
```
RuntimeError: Failed to auto-detect CUDA root directory. Please specify `CUDA_PATH` environment
variable if you are using CUDA versions not yet supported by CuPy.
```
CuPy 14.0.1 from pip (`cupy-cuda12x`) was installed with only the `nvidia-cuda-nvrtc-cu12`
wheel; the CUDA-12 pip layout is splayed (no `/usr/local/cuda`, no `nvcc`), so CuPy's
`_get_cuda_path()` returned `None` and NVRTC could not find the headers. Every run since the
Oanda re-pin logged `Backend: NumPy CPU fallback`; every research report in this folder records
it. Independently of that, almost nothing in the file was written for the GPU: the EWM
precompute was a pure-Python loop, Bollinger was pandas `rolling`, and the trailing-stop
resolver (the exit mode of every reference book) was a per-trade per-bar Python loop. The only
CuPy code was a first-hit reduction in the static SL/TP branch, which the reference books never
execute.
## 2. What shipped
- `nvidia-cuda-runtime-cu12==12.9.79` installed into the dev venv (headers only, not
`requirements.txt`; same status as `cupy` and `pyarrow`).
- `backtest/gpu_kernels.py`: the backend state (`BACKEND`, probed lazily so importing the
backtester or `analyze_signal_log.py` creates no CUDA context; the probe compiles a kernel, and
a failure keeps its reason), `CUDA_PATH` pointed at the header wheel when unset, and three CUDA
kernels compiled with `--fmad=false`:
- `ewm_batch`: one thread per series, the exact recurrence of `_gpu_ewm_alpha` (seed at the
first non-NaN value, NaN carries the state without advancing the `min_periods` counter).
- `roll_mean_var`: pandas 3.0.2 `roll_mean` (Kahan add/remove, sign count, consecutive-same-
value guard) and `roll_var` (Welford + Kahan, `InvCondTol` unstable recompute); std as
pandas' `zsqrt`.
- `resolve_trades`: one thread per trade; trailing mode mirrors `_simulate_trailing` line by
line, static mode mirrors the SL-first/TP1/TIMEOUT branch, including the "NaN close after a
TIMEOUT" bookkeeping; R and MFE/MAE on the device.
- `backtest/gpu_backtest.py`: `precompute_indicators` dispatches to `_precompute_indicators_gpu`
(CuPy elementwise ops in the original order, two EWM launches and one rolling launch per
timeframe, one transfer to host at the end, host-only cache asserted before the fork) or to the
untouched NumPy path; `simulate_trades` dispatches to `_resolve_trades_gpu` or to the untouched
CPU paths; pooled device memory is released before the workers are forked; `--backend
auto|gpu|cpu` (env `FB2_BACKEND`); the `Backend:` line names the device or the fallback reason.
- `test_all.py` Module 36 (7 tests): a visible CUDA device must never fall back silently; the
three kernels against their CPU references on synthetic edge cases (NaN head, interior NaN,
flat window, cancellation spike, gaps, NaN bars, zero risk, per-trade trail overrides);
`--backend` / `FB2_BACKEND` / `cpu_only()`; import guards (`cupy` is now banned from the live
roots by the Module 32 AST guard as well).
Why the kernels are sequential per series: EWM and rolling statistics are order-dependent
floating point. A parallel scan or a two-pass variance changes the last bits, and the file's own
history shows what that does (a cumsum Bollinger shortcut moved `bb_position` by up to 8e-3 and
crossed signal thresholds). The GPU's gain here is the batch across series and across trades,
not per-series speed: on a single very long series the sequential kernel is slower than pandas'
C loop.
## 3. Evidence
Environment: NVIDIA GeForce RTX 4070 SUPER, driver 615.71.09 (CUDA 13.4 UMD), CuPy 14.0.1
(`cupy-cuda12x`), `nvidia-cuda-nvrtc-cu12` 12.9.86, `nvidia-cuda-runtime-cu12` 12.9.79, NumPy
2.2.6, pandas 3.0.2, Python 3.12, 16 CPUs. Code: this working tree on branch `sql-integration`
on top of `1f4741b`.
### 3.1 Precompute equality on every cached window
GPU vs CPU `precompute_indicators`, every `IndicatorCache` field compared with
`np.array_equal(..., equal_nan=True)`:
```
precompute GPU == CPU on 42/42 windows, 2342009 bars total; gpu 13.71s (incl. cold JIT) vs cpu 40.18s
```
### 3.2 Test suite, three ways
| Run | Result | Module 36 note |
|---|---|---|
| default (GPU) | `RESULTS: 536 passed, 0 failed, 536 total` | kernels exercised |
| `CUDA_VISIBLE_DEVICES=""` | `RESULTS: 536 passed, 0 failed, 536 total` | `NumPy CPU fallback (CUDARuntimeError: cudaErrorNoDevice: ...)` printed |
| `FB2_BACKEND=cpu` | `RESULTS: 536 passed, 0 failed, 536 total` | `NumPy CPU fallback (forced: ...)` printed |
### 3.3 Reference books on both backends
Harness: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --spread 0.65 --workers 12
--primary-tf <tf> --start <s> --end <e> --export-csv <out>` (factory config; no `--set`,
`--quality`, `--strategies`, `--confirm-tfs` or `--export-features`), plus `--backend cpu` for
the CPU column. `GPU cache fallbacks: 0` on every run.
| Book | Window | Backend | Precompute | Replay | Trade sim | Wall | MD5 | Pinned |
|---|---|---|---|---|---|---|---|---|
| M5 W2 | 2024-09-05 → 2026-09-05 | CuPy GPU | 1.31 s | 112.0 s | 0.31 s | 114.8 s | `49f8e602fc6ae769713c94ef7b257810` | ✓ |
| M5 W2 | | NumPy CPU (`--backend cpu`) | 3.74 s | 112.6 s | 0.22 s | 117.0 s | `49f8e602fc6ae769713c94ef7b257810` | ✓ |
| M15 W2 | 2024-09-05 → 2026-09-05 | CuPy GPU | 1.29 s | 34.6 s | 0.33 s | 37.2 s | `b4aa07e33e17e23774c486a6900d673b` | ✓ |
| M15 W2 | | NumPy CPU | 3.69 s | 34.6 s | 0.28 s | 39.0 s | `b4aa07e33e17e23774c486a6900d673b` | ✓ |
| M15 OOS | 2024-06-01 → 2025-03-30 | CuPy GPU | 0.63 s | 14.0 s | 0.05 s | 15.6 s | `9fd624fb387c36052122bd1830cf9c32` | ✓ |
| M15 OOS | | NumPy CPU | 1.70 s | 13.9 s | 0.09 s | 16.1 s | `9fd624fb387c36052122bd1830cf9c32` | ✓ |
| H1 W3 | 2023-09-05 → 2026-09-05 | CuPy GPU | 0.61 s | 11.3 s | 0.07 s | 13.0 s | `7b848c917dcb5c371ecbb9f8f7292278` | ✓ |
| H1 W3 | | NumPy CPU | 1.68 s | 11.1 s | 0.07 s | 13.3 s | `7b848c917dcb5c371ecbb9f8f7292278` | ✓ |
| H4 W3 | 2023-09-05 → 2026-09-05 | CuPy GPU | 0.20 s | 20.0 s (1 worker) | 0.13 s | 21.2 s | `186e677fbbd53cca735c94094c23ce05` | ✓ |
| H4 W3 | | NumPy CPU | 0.46 s | 19.9 s (1 worker) | 0.02 s | 20.7 s | `186e677fbbd53cca735c94094c23ce05` | ✓ |
`Backend:` lines verbatim: `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` and
`NumPy CPU fallback (forced: --backend cpu / FB2_BACKEND=cpu)`. The GPU precompute figures are
warm (the kernel cache under `~/.cupy/kernel_cache` was already populated; a cold first run adds
roughly a second of JIT once per machine). The trade-simulation figures on the GPU include the
host↔device transfers and the launch of a kernel that runs a few hundred threads; on these trade
counts it is not faster than the CPU loop, which is expected. The before/after picture is exactly
as planned: the precompute is about 3× faster, everything else is unchanged, and the replay
through the live signal code (the same bars/s on both rows) is where the wall time lives.
### 3.4 Other checks
- Single worker on the GPU (`--workers 1`, H1 W3): MD5 `7b848c917dcb5c371ecbb9f8f7292278`, equal
to the 12-worker book (the CUDA context exists in the parent before the fork; the workers never
touch it).
- `--record` pair on the GPU (H4 W3): control `186e677f…` == recorded `186e677f…` ==
`manifest.book_csv_md5`; the manifest records `backend = CuPy GPU (NVIDIA GeForce RTX 4070 SUPER,
cupy 14.0.1)` and `cache_fallbacks = 0`.
- `CUDA_VISIBLE_DEVICES="" ... --backend gpu` exits with
`RuntimeError: --backend gpu requested but CUDA is unavailable: CUDARuntimeError: cudaErrorNoDevice ...`;
the same run in `auto` mode logs the warning and
`Backend: NumPy CPU fallback (CUDARuntimeError: cudaErrorNoDevice: no CUDA-capable device is detected)`.
- `analyze_signal_log.py --stored-candles` (which reuses `simulate_trades`) on the imported
`logs/signals.db`: the report is line-for-line identical on the GPU and with `FB2_BACKEND=cpu`.
- `nvidia-smi --query-compute-apps` during the M5 precompute listed the backtester process on the
device (184 MiB).
- Scratchpad experiments that preceded the implementation: the EWM kernel bit-identical to
`_gpu_ewm_alpha` on 9 series over the 154k-bar M5 Close (95 ms incl. JIT vs 1.29 s Python);
the rolling kernel bit-identical to pandas on all 84 series of the 42 cached windows.
- `git diff --stat` touches `backtest/gpu_backtest.py`, `test_all.py`, `CLAUDE.md`,
`backtest/instructions_for_backtest.txt` and adds `backtest/gpu_kernels.py` and this note;
`engine.py`, `signal_aggregator.py` and `requirements.txt` are untouched.
@@ -0,0 +1,165 @@
# Why the losing trades lost — two-year M5 and M15 books, 2026-09-18
Owner question: for every losing trade in the two-year books, was it the wrong direction, a late
entry, a target the market reversed before reaching, or a bad hour or day? Descriptive, in-book
statistics; nothing here is a decision (rule 1 needs a control-versus-candidate run).
## 1. Data
The validated two-year windows, 2024-09-05 → 2026-09-05, regenerated on the CUDA twin with the
decision-time features attached (`--export-features`), $0.65 round trip, factory configuration:
```
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 5min --start 2024-09-05 --end 2026-09-05 --spread 0.65 --export-features --export-csv w2_5min_feat.csv
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 15min --start 2024-09-05 --end 2026-09-05 --spread 0.65 --export-features --export-csv w2_15min_feat.csv
```
`Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, 0 cache fallbacks, 781 and
931 trades, the pinned books (`49f8e602…`, `b4aa07e3…`) with the research columns appended. Every
stretch feature is signed in the trade's direction (positive = price had already moved the
trade's way), so buys and sells read alike. "R" is risk units: profit or loss divided by the
distance from entry to the initial stop. A loser is any trade with negative R.
| Book | Trades | Losers | Loss rate | Loss R | Win R | Net R | Full-stop losers | Trailing-stop losers |
|---|---|---|---|---|---|---|---|---|
| M5 | 781 | 368 | 47.1% | −278.9 | +424.9 | +146.1 | 247 (−257.8R) | 115 (−19.1R) |
| M15 | 931 | 368 | 39.5% | −341.0 | +487.1 | +146.1 | 324 (−336.4R) | 39 (−2.6R) |
Median loss −1.02R / −1.03R: a loser is almost always a full stop. The trailing exits' losses are
small by construction.
## 2. Price path of the losers (what the market did after entry)
| Path after entry | M5 losers | Share | M5 loss R | M15 losers | Share | M15 loss R |
|---|---|---|---|---|---|---|
| Never went our way (best excursion under 0.25R) | 153 | 42% | −157.9 (57%) | 162 | 44% | −166.9 (49%) |
| Shallow move, then reversal (0.25R to 0.75R) | 165 | 45% | −115.7 (41%) | 185 | 50% | −171.6 (50%) |
| Gave back a real advance (0.75R or more) | 50 | 14% | −5.3 (2%) | 21 | 6% | −2.5 (1%) |
- **Wrong from the start** is the largest single cause by R: over half the M5 loss and half the
M15 loss comes from trades whose best moment was under a quarter of the risk. Median hold 12
bars (M5) and 18 bars (M15): they were stopped fast.
- **Reversals after a genuine advance cost almost nothing** (1–2% of loss R): the trailing stop
arms at 0.6R (M5) / 0.7R (M15) and limits the give-back. On M5, 81 losers (22%) did reach the
full target-1 distance (median target-1 is only 0.51R on M5) and still closed red; they cost
−23.8R. On M15 that is 15 losers, −7.1R.
- Losers that reached at least half the target-1 distance: M5 157 (43%, −78.8R), M15 115 (31%,
−80.8R). So roughly a third to two fifths of losers "were winning at some point".
### Would a nearer hard target have helped? (all trades; exit at X when the best excursion reached X)
| Hard target | M5 losers rescued | M5 winners capped | M5 net | M15 losers rescued | M15 winners capped | M15 net |
|---|---|---|---|---|---|---|
| 0.50R | 142 (+118.0R) | 267 (−259.2R) | +45.4 | 102 (+118.5R) | 329 (−266.0R) | +58.6 |
| 0.75R | 50 (+42.8R) | 203 (−201.4R) | +72.1 | 21 (+18.2R) | 244 (−194.6R) | +97.5 |
| 1.00R | 3 (+4.8R) | 166 (−154.5R) | +97.3 | 2 (+2.4R) | 180 (−142.1R) | +128.5 |
| 1.50R | 0 | 94 (−91.5R) | +117.2 | 0 | 97 (−74.7R) | +143.5 |
| 2.00R | 0 | 53 (−56.2R) | +132.6 | 0 | 58 (−37.5R) | +152.2 |
| actual (trailing) | | | **+146.1** | | | **+146.1** |
A nearer target rescues losers but caps far more winners; the book's R lives in the right tail
(top 5% of trades = 94–96% of R, cycle 1). Only a 2R hard target on M15 edges the trailing exit
(+6R), which is the 2026-08-05 fixed-RR finding: it passes on 2024+ windows and is negative in
2021 and 2023 with double the drawdown. Not re-proposed.
## 3. Late entry: does chasing lose?
No. Loss rate falls as the entry gets "later"; the pullback-style entries are the worst bucket.
| Move over the previous 12 bars in the trade's direction | M5 trades | Loss rate | Avg R | M15 trades | Loss rate | Avg R |
|---|---|---|---|---|---|---|
| Against the last 12 bars (pullback entry) | 5 | 80% | −0.39 | 14 | 64% | −0.13 |
| 0 to 1 ATR | 60 | 50% | +0.18 | 43 | 44% | −0.06 |
| 1 to 2 ATR | 157 | 50% | +0.14 | 148 | 38% | +0.23 |
| 2 to 3 ATR | 180 | 42% | +0.28 | 288 | 41% | +0.19 |
| 3 ATR or more (chased) | 379 | 47% | +0.17 | 438 | 38% | +0.14 |
| Hourly stretch (distance above the 20-bar hourly mean) | M5 loss rate | M5 net R | M15 loss rate | M15 net R |
|---|---|---|---|---|
| Below the mean | 49% | +19.4 | 43% | +12.7 |
| 0 to 1 ATR | 50% | +21.9 | 45% | +25.0 |
| 1 to 2 ATR | 47% | +42.6 | 41% | +52.4 |
| 2 ATR or more (stretched) | 45% | +62.2 | 35% | +56.1 |
Same picture for the 48-bar move (M5: 4 ATR or more = 44% loss rate and +128.5R of the +146.1R),
the distance from the 21-bar average, consecutive closes and the exhaustion tags (two or more
tags: 45% / 40% loss rate, the best bucket on M5). This is the 2026-08-04 age/stretch verdict
again: the best trades are the stretched runs. "Late entry" is not a cause of losses here.
## 4. Wrong direction: what predicted the wrong side?
| Condition at entry | M5 trades | Loss rate | Net R | M15 trades | Loss rate | Net R |
|---|---|---|---|---|---|---|
| Timeframes agreed | 579 | 46% | +128.4 | 897 | 39% | +135.3 |
| Timeframes did not agree | 202 | 51% | +17.7 | 34 | 44% | +10.9 |
| With the trend | 475 | 45% | +93.5 | 596 | 40% | +68.1 |
| No trend, ranging | 169 | 49% | +25.7 | 120 | 37% | +47.0 |
| No trend, transitioning | 76 | 51% | +24.0 | 213 | 41% | +31.7 |
| No trend, mixed trending | 61 | 54% | +2.9 | (blocked on M15) | | |
| Confidence penalty 20 points or more | 94 | 55% | −3.3 | 11 | 64% | −0.4 |
| Buy | 683 | 47% | +137.3 | 635 | 37% | +146.6 |
| Sell | 98 | 49% | +8.8 | 296 | 46% | −0.5 |
- Disagreement between timeframes and a mixed regime raise the loss rate by about 5 points on M5
and remove most of the average R; the heavily penalised entries (20+ points) are net negative
on both books but are few.
- Indicator opposition does not predict the wrong side: the M5 bucket with **no** opposing
indicator loses most often (58%, 40 trades). Direction was not "obviously wrong" at entry.
- M15 shorts: 296 trades, 46% losers, −0.5R net. Every M15 R came from buys. This is the parked
cycle-2 candidate (`cycle_2026-09-10.md`); unchanged status.
## 5. When we lost
Entry hour is the open of the bar after the signal bar (Coordinated Universal Time). Cells under
30 trades are noise.
| M5 hour | Trades | Loss rate | Net R | | M15 hour | Trades | Loss rate | Net R |
|---|---|---|---|---|---|---|---|---|
| 08:00 | 62 | 55% | −0.7 | | 07:00 | 68 | 47% | −4.1 |
| 14:00 | 117 | 53% | +4.7 | | 09:00 | 75 | 47% | +3.8 |
| 17:00 | 35 | 49% | −4.0 | | 18:00 | 45 | 47% | −2.1 |
| 22:00 | 31 | 55% | +4.0 | | 04:00 | 28 | 43% | +1.9 |
| 20:00 | 42 | 52% | +30.1 | | 14:00 | 154 | 40% | +16.9 |
| 06:00 | 60 | 40% | +26.5 | | 12:00 | 68 | 31% | +33.8 |
The weakest hours are 08:00 and 17:00 on M5 (14:00 is the busiest hour and carries the most
loss R, −44.9R, at a 53% loss rate but still nets +4.7R) and 07:00, 09:00 and 18:00 on M15. No
hour is both large and clearly negative; 20:00 on M5 loses 52% of the time and is the second
best hour by R (+30.1R), which is the whole shape of this edge.
| Weekday | M5 trades | Loss rate | Net R | M15 trades | Loss rate | Net R |
|---|---|---|---|---|---|---|
| Monday | 153 | 45% | +17.6 | 169 | 39% | +28.2 |
| Tuesday | 159 | 48% | +21.3 | 212 | 33% | +50.4 |
| Wednesday | 176 | 45% | +46.6 | 189 | **51%** | **−4.0** |
| Thursday | 135 | 51% | +9.2 | 179 | 40% | +30.6 |
| Friday | 151 | 48% | +47.7 | 169 | 36% | +34.2 |
Wednesday is the one clear day effect: on M15 it loses more than half the time and is the only
negative weekday (−86.2R of loss R, the worst of any day); the worst M15 weekday-hour cells are
all Wednesday (14:00: 26 trades, 16 losers, −7.2R; 18:00: 13, 8, −4.7R; 09:00: 20, 14, −3.3R;
08:00: 16, 9, −3.0R). On M5 Wednesday is fine (+46.6R) and Thursday 20:00 is the worst cell (11
trades, 9 losers, −6.5R).
Calendar-day clusters are not the story: the ten worst days hold 11% (M5) and 9% (M15) of all
loss R, no single day cost more than −4.2R, and only 27 of 333 (M5) and 15 of 449 (M15) entry
days had three or more losers. Losers entered within two hours of the previous loser (re-entries
into the same move): M5 46 (−39.5R, 14% of loss R), M15 22 (−21.7R, 6%). By month, July is
negative on M15 in both years (2025-07 −7.5R at 57% losers, 2026-07 −8.6R at 58%) and M5 lost in
2024-Q4 and 2025-07/08.
## 6. Summary for the owner
| Question | Answer (both books unless stated) |
|---|---|
| Wrong direction? | Yes for about 43% of losers by count and about half of the loss R: they never moved a quarter of the risk our way and were stopped within 12–18 bars. Nothing at entry flagged them: indicator opposition does not predict it, timeframe disagreement and mixed regimes add only ~5 points of loss rate. |
| Late entry? | No. Loss rate falls as the prior move grows; the few pullback-style entries lose 64–80%. Stretched entries carry most of the R. (Confirms the 2026-08-04 verdict.) |
| Target too far, market reversed first? | Rarely the cause of a loss: reversals after a 0.75R advance cost 1–2% of loss R because the trailing stop limits them. About a third of losers were up at least half the target-1 distance before losing; a nearer hard target would rescue them but cap far more winners (net +45R to +133R against +146R). |
| Bad hours? | M5: 08:00 and 17:00 lose most often; 14:00 carries the most loss R but nets positive. M15: 07:00, 09:00 and 18:00. No large hour is clearly negative. |
| Bad days? | Wednesday on M15 (51% losers, −4.0R net, all four worst weekday-hour cells). Not a single-day-cluster problem on either book. |
None of this is a change proposal. A Wednesday or hour block on M15, or a shorter M15 SELL list,
would each be a rule-1 campaign with a control run, the OOS window and the refuted-hypothesis
list in mind (the 2026-07 M15 loss analysis rejected several hour variants). The most useful
follow-up, if any, is the parked M15 grade-A SELL block, which this table supports without
adding to it.
@@ -0,0 +1,149 @@
# M15 five-year book on the CUDA research twin — 2026-09-18
Owner request: a five-year M15 backtest with `backtest/cudaBasedBacktesting.py`. This window goes
past the three-year cap the owner set on 2026-09-05 (CLAUDE.md rule 1); it was run as an explicit
owner-requested exception. It is a read-out of the shipped configuration on a longer window, not
a candidate change: nothing was tuned, nothing ships from it.
## 1. Runs
Window 2021-08-01 → 2026-08-04 (the kept 5-year cache, padded files
`oanda_XAU_USD_{5min,15min,1h}_20210531_20260804.parquet`; 0 cache fallbacks; 122,489 M15 bars,
118,389 replayed; skipped 43,245 `SESSION_WINDOW`, 247 `WEEKEND_FILTER`). Factory config, no
overrides.
```
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 15min --start 2021-08-01 --end 2026-08-04 --spread 0.65 --export-csv cuda_m15_5yr_0.65.csv
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 15min --start 2021-08-01 --end 2026-08-04 --spread 0.30 --export-csv cuda_m15_5yr_0.30.csv
venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --primary-tf 15min --start 2021-08-01 --end 2026-08-04 --spread 0.65 --workers 12 --export-csv ctrl_m15_5yr_0.65.csv # paired reference
```
| Run | Backend line (verbatim) | Wall | Book MD5 |
|---|---|---|---|
| twin $0.65 | `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` | 6.6 s (K1 520,523 bars 0.046 s, K2 118,389 bars 0.001 s, trade sim 0.80 s) | `d5e432bdb3dc9923e00594125b4867dd` |
| twin $0.30 | `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` | 5.3 s | `4ac5c0307123557c0740e57f5b91ff72` |
| reference $0.65 | `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, `GPU cache fallbacks: 0` | 93.7 s (replay 89.2 s, 12 workers) | `d5e432bdb3dc9923e00594125b4867dd` — identical to the twin's (`cmp` equal) |
The paired reference run is the parity evidence the 2026-09-18 addendum in
`backtest/instructions_for_backtest.txt` requires before a number from the twin is quoted.
## 2. Results
1,822 trades at both costs (1,087 `TRAIL`, 666 `SL`, 69 `TIMEOUT`, no `NO_DATA`). Halves split at
2024-02-02.
| Cost | Segment | Trades | Total R | Win rate | PF | Max DD (R) |
|---|---|---|---|---|---|---|
| $0.65 | whole 2021-08-01 → 2026-08-04 | 1822 | +157.43 | 56.8% | 1.22 | 43.6 |
| $0.65 | first half 2021-08-01 → 2024-02-01 | 704 | −9.61 | 52.4% | 0.97 | 43.6 |
| $0.65 | second half 2024-02-02 → 2026-08-04 | 1118 | +167.04 | 59.5% | 1.40 | 17.1 |
| $0.30 | whole | 1822 | +196.77 | 57.7% | 1.28 | 36.3 |
| $0.30 | first half | 704 | +9.38 | 54.0% | 1.03 | 36.3 |
| $0.30 | second half | 1118 | +187.39 | 60.1% | 1.46 | 14.9 |
Read against rule 1 for the record: the five-year window does **not** pass it (the first half is
negative at $0.65 and the whole-window PF is 1.22). The shipped M15 configuration was validated on
2024-09-05 → 2026-09-05 (+146.1R, PF 1.43) and the 2024-06 → 2025-03 out-of-sample window
(+60.7R, PF 1.49); the extra 2021–2023 years are where the R is missing, which is the same picture
the Oanda re-pin of 2026-08-04 gave ("all timeframes earn only 2024+") and the reason for the
three-year cap.
### Per year
| Year | Trades | $0.65 R | WR | PF | Max DD | $0.30 R | PF |
|---|---|---|---|---|---|---|---|
| 2021 (Aug–Dec) | 97 | +1.17 | 47.4% | 1.03 | 15.5 | +3.85 | 1.10 |
| 2022 | 318 | −15.37 | 51.9% | 0.89 | 39.1 | −6.73 | 0.95 |
| 2023 | 264 | +10.91 | 55.7% | 1.10 | 15.7 | +17.86 | 1.17 |
| 2024 | 397 | +52.86 | 58.7% | 1.36 | 17.1 | +62.53 | 1.44 |
| 2025 | 480 | +77.35 | 60.0% | 1.43 | 14.8 | +85.69 | 1.48 |
| 2026 (Jan–Aug 4) | 266 | +30.51 | 58.3% | 1.30 | 13.2 | +33.57 | 1.34 |
The 43.6R drawdown is entirely 2021-Q4 → 2022-Q3 (quarterly R: −13.5, +13.5, −7.6, −22.6); from
2024 on the worst quarter is −3.4R (2024-Q2) and the worst since is −8.3R for the five weeks of
2026-Q3 in the window.
### Breakdowns ($0.65)
| Grade | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| A+ | 36 | +5.48 | 63.9% | 1.41 | 3.3 |
| A | 970 | +92.30 | 56.5% | 1.24 | 21.3 |
| B | 728 | +41.16 | 56.5% | 1.14 | 31.3 |
| C | 88 | +18.49 | 59.1% | 1.55 | 4.8 |
| Generator | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| GENERIC | 1658 | +153.28 | 57.0% | 1.24 | 43.6 |
| XAU_ATR | 164 | +4.15 | 54.3% | 1.06 | 17.0 |
XAU_ATR produces no trade before 2024 (5 in 2024, 48 in 2025, 111 in 2026). The gold ATR setup
is gated by dollar-denominated floors (`min_atr_threshold`, the $8 stop floor, the 30 × $0.30
target floor); at 2021–2023 gold prices and volatility they blocked the generator and the GENERIC
swing setup took every A-grade signal. The pre-2024 book therefore runs under an effectively
different setup mix. Observation only; no proposal.
| Direction | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| BUY | 1212 | +147.91 | 58.3% | 1.32 | 48.3 |
| SELL | 610 | +9.52 | 53.8% | 1.04 | 36.2 |
Direction by year (trades / R): BUY 2021 58 / −9.8, 2022 215 / −28.9, 2023 166 / +10.9, 2024
273 / +56.8, 2025 344 / +95.2, 2026 156 / +23.7; SELL 2021 39 / +10.9, 2022 103 / +13.6, 2023
98 / 0.0, 2024 124 / −3.9, 2025 136 / −17.9, 2026 110 / +6.8. Shorts carried the book in the
2021–2022 bear phase and lost in the 2024–2025 rally; the parked cycle-2 candidate (block M15
grade-A SELLs, `cycle_2026-09-10.md`) is about the recent years only and this table does not
change its status.
| Regime | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| ranging | 168 | +45.30 | 61.3% | 1.73 | 7.7 |
| transitioning | 400 | +51.25 | 58.2% | 1.33 | 7.0 |
| trending_up | 788 | +55.43 | 56.9% | 1.18 | 37.3 |
| trending_down | 466 | +5.44 | 53.6% | 1.03 | 26.0 |
(The mixed `trending` regime is blocked on M15, so it has no row.)
| Volatility | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| low | 193 | +34.75 | 60.1% | 1.48 | 11.2 |
| normal | 251 | +41.65 | 61.0% | 1.50 | 20.6 |
| high | 393 | +2.77 | 53.7% | 1.02 | 19.2 |
| extreme | 985 | +78.27 | 56.2% | 1.20 | 32.0 |
| Entry hour UTC | Trades | Total R | WR | PF | | Entry hour UTC | Trades | Total R | WR | PF |
|---|---|---|---|---|---|---|---|---|---|---|
| 03 | 98 | +8.18 | 60.2% | 1.24 | | 14 | 381 | +24.60 | 55.4% | 1.16 |
| 04 | 38 | +4.62 | 55.3% | 1.29 | | 15 | 146 | +17.60 | 57.5% | 1.30 |
| 05 | 77 | +16.50 | 61.0% | 1.58 | | 17 | 79 | +27.77 | 68.4% | 2.20 |
| 06 | 117 | +27.74 | 66.7% | 1.85 | | 18 | 112 | −6.65 | 50.9% | 0.87 |
| 07 | 116 | −9.29 | 50.0% | 0.83 | | 19 | 80 | +10.95 | 62.5% | 1.40 |
| 08 | 158 | +1.62 | 53.2% | 1.03 | | 20 | 55 | +11.68 | 61.8% | 1.66 |
| 09 | 145 | −7.31 | 49.7% | 0.89 | | 22 | 40 | +12.48 | 67.5% | 2.01 |
| 12 | 180 | +16.94 | 54.4% | 1.22 | | | | | | |
(Entry hour is the open of the bar after the signal bar; the blocked close hours 0, 1, 2, 10, 11,
13, 16, 21 leave the gaps.)
| Weekday | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| Mon | 309 | +25.97 | 57.3% | 1.21 | 15.8 |
| Tue | 394 | +40.58 | 60.2% | 1.29 | 17.4 |
| Wed | 386 | −18.07 | 50.5% | 0.90 | 25.4 |
| Thu | 356 | +50.99 | 56.5% | 1.37 | 22.3 |
| Fri | 358 | +47.13 | 58.4% | 1.35 | 19.3 |
| Sun | 19 | +10.82 | 78.9% | 3.64 | 2.1 |
Tail: the top 5% of trades (91) carry +249.8R of the +157.4R total; median R +0.113; largest win
+9.07R, largest loss −2.04R; mean hold 70.5 bars. Same fat-right-tail shape as the 3-year cycle-1
reading (94–96% of R in the top 5% there).
## 3. What to do with this
Nothing. The numbers describe the shipped configuration on a window the owner does not use for
decisions; the per-hour, per-weekday and per-direction tables are in-book statistics on a
five-year mix of setup gates and must not be turned into filters (rule 1 needs a control-vs-
candidate run on the validated windows, and the 2026-08-04 age/stretch and 2026-09-06 sizing
verdicts already refute the obvious ones). If the owner wants the pre-2024 weakness explained,
the honest next step is a `--record --record-bars` run on this window with the twin (seconds) and
a scorecard on the dollar-floor gates by year, not a parameter change.
+156
View File
@@ -0,0 +1,156 @@
# M5 five-year book on the CUDA research twin — 2026-09-18
Owner request, the companion of `m15_5yr_cuda_2026-09-18.md`: a five-year M5 backtest with
`backtest/cudaBasedBacktesting.py`. Same owner-requested exception to the three-year cap
(CLAUDE.md rule 1), same status: a read-out of the shipped configuration on a longer window,
nothing tuned, nothing ships.
## 1. Runs
Window 2021-08-01 → 2026-08-04 (kept 5-year cache, padded files
`oanda_XAU_USD_{5min,15min,1h}_20210531_20260804.parquet`; 0 cache fallbacks; 367,394 M5 bars,
355,096 replayed; skipped 139,064 `SESSION_WINDOW`, 247 `WEEKEND_FILTER`). Factory config (M5:
hour 9 blocked and 21 open, chart-pattern entries on, pullbacks off), no overrides.
```
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 5min --start 2021-08-01 --end 2026-08-04 --spread 0.65 --export-csv cuda_m5_5yr_0.65.csv
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf 5min --start 2021-08-01 --end 2026-08-04 --spread 0.30 --export-csv cuda_m5_5yr_0.30.csv
venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --primary-tf 5min --start 2021-08-01 --end 2026-08-04 --spread 0.65 --workers 12 --export-csv ctrl_m5_5yr_0.65.csv # paired reference
```
| Run | Backend line (verbatim) | Wall | Book MD5 |
|---|---|---|---|
| twin $0.65 | `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` | 5.2 s (K1 520,523 bars 0.045 s, K2 355,096 bars 0.004 s, trade sim 0.38 s) | `0429885713ca34653a39754c013db501` |
| twin $0.30 | `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)` | 5.6 s | `87bfdba14090459f715e1ee1f751a6cb` |
| reference $0.65 | `CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, `GPU cache fallbacks: 0` | 353.2 s (replay 348.7 s, 12 workers, 1,018 bars/s) | `0429885713ca34653a39754c013db501` — identical to the twin's (`cmp` equal) |
## 2. Results
905 trades at both costs (561 `TRAIL`, 290 `SL`, 54 `TIMEOUT`, no `NO_DATA`). Halves split at
2024-02-02.
| Cost | Segment | Trades | Total R | Win rate | PF | Max DD (R) |
|---|---|---|---|---|---|---|
| $0.65 | whole 2021-08-01 → 2026-08-04 | 905 | +127.08 | 51.4% | 1.39 | 13.5 |
| $0.65 | first half 2021-08-01 → 2024-02-01 | 129 | −8.98 | 43.4% | 0.82 | 12.4 |
| $0.65 | second half 2024-02-02 → 2026-08-04 | 776 | +136.06 | 52.7% | 1.49 | 9.6 |
| $0.30 | whole | 905 | +147.83 | 52.5% | 1.46 | 10.6 |
| $0.30 | first half | 129 | −5.20 | 45.0% | 0.89 | 10.6 |
| $0.30 | second half | 776 | +153.03 | 53.7% | 1.57 | 8.9 |
Against rule 1 for the record: the five-year window fails it (first half negative), on a first
half of only 129 trades. The shipped M5 configuration was validated on 2024-09-05 → 2026-09-05
(+146.1R, PF 1.52, 781 trades) and the 12-month window (+108.8R, PF 1.54).
### Per year
| Year | Trades | $0.65 R | WR | PF | Max DD | $0.30 R | PF |
|---|---|---|---|---|---|---|---|
| 2021 (Aug–Dec) | 12 | −1.99 | 41.7% | 0.59 | 2.5 | −1.60 | 0.65 |
| 2022 | 50 | −2.18 | 44.0% | 0.90 | 11.1 | −0.71 | 0.97 |
| 2023 | 61 | −2.75 | 42.6% | 0.87 | 10.3 | −1.00 | 0.95 |
| 2024 | 102 | +0.97 | 49.0% | 1.02 | 8.2 | +4.30 | 1.11 |
| 2025 | 332 | +40.54 | 50.6% | 1.34 | 9.6 | +48.63 | 1.42 |
| 2026 (Jan–Aug 4) | 348 | +92.49 | 55.7% | 1.77 | 9.2 | +98.21 | 1.83 |
Two things the M15 window did not show as sharply:
1. **M5 barely trades before 2025.** 225 trades in the first 3.4 years against 680 in the last
1.6. Trade generation on M5 is governed by dollar-denominated floors: target 1 must clear
30 × $0.30 = $9 (`SPREAD GATE`), the generic and chart-pattern stops are floored at $8, and a
chart pattern needs a $9 height. At 2021–2023 prices and volatility an M5 ATR of $1.5–3 puts a
1.5 × ATR target far below the floor, so the spread gate vetoed almost every setup and only
high-volatility bursts survived. The pre-2025 M5 book is therefore a burst-only sample, not the
shipped strategy under normal conditions.
2. **One quarter carries the window.** 2026-Q1 made +90.2R of the +127.1R total (71%); 2025 Q1–Q2
another +32.3R. Quarterly R outside those three quarters ranges −6.5 … +5.8. This is the
fat-right-tail shape of every M5 reading (cycle 1: the top 5% of trades = 94–96% of R; here the
top 5%, 45 trades, carry +145.8R of +127.1R) compounded by a single very favourable quarter.
### Breakdowns ($0.65)
| Grade | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| A+ | 1 | −0.03 | 0.0% | 0.00 | 0.0 |
| A | 247 | +53.72 | 55.1% | 1.69 | 7.2 |
| B | 637 | +73.66 | 50.2% | 1.30 | 16.4 |
| C | 20 | −0.26 | 45.0% | 0.97 | 3.2 |
| Generator | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| GENERIC | 447 | +70.36 | 52.8% | 1.46 | 8.1 |
| XAU_CP | 427 | +48.52 | 49.9% | 1.29 | 14.4 |
| XAU_ATR | 31 | +8.20 | 51.6% | 1.89 | 3.8 |
Generator by year (trades / R): GENERIC 2021 4 / +0.4, 2022 23 / +4.4, 2023 32 / −2.9, 2024
37 / +6.6, 2025 170 / +18.1, 2026 181 / +43.8; XAU_CP 2021 8 / −2.4, 2022 27 / −6.5, 2023
29 / +0.2, 2024 65 / −5.7, 2025 159 / +21.6, 2026 139 / +41.3; XAU_ATR appears only in 2025 (3)
and 2026 (28), the same dollar-floor effect as on M15. The chart-pattern path is negative through
2024 (−14.4R over 129 trades) and positive since (+62.9R over 298); its validation
(2026-08-04, +38.9R of the 12-month book) covered the recent regime only.
| Direction | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| BUY | 808 | +119.68 | 51.4% | 1.41 | 15.2 |
| SELL | 97 | +7.40 | 51.5% | 1.21 | 11.4 |
Shorts are nearly absent before 2025 (9 trades in 2021–2024) under the M5 SELL gates; 2025 32 /
−2.8R, 2026 56 / +12.0R.
| Regime | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| trending_up | 481 | +75.44 | 53.4% | 1.44 | 11.9 |
| ranging | 174 | +24.78 | 50.0% | 1.36 | 8.3 |
| trending_down | 94 | +6.61 | 51.1% | 1.19 | 12.4 |
| transitioning | 84 | +19.59 | 46.4% | 1.69 | 6.9 |
| trending (mixed) | 72 | +0.67 | 47.2% | 1.02 | 10.5 |
| Volatility | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| low | 250 | +62.92 | 51.6% | 1.70 | 12.4 |
| normal | 169 | +0.69 | 47.9% | 1.01 | 12.5 |
| extreme | 486 | +63.47 | 52.5% | 1.38 | 13.3 |
(`high` has no row: the M5 chop-transition gate blocks it.)
| Entry hour UTC | Trades | Total R | WR | PF | | Entry hour UTC | Trades | Total R | WR | PF |
|---|---|---|---|---|---|---|---|---|---|---|
| 03 | 52 | +8.49 | 50.0% | 1.41 | | 15 | 121 | +12.81 | 53.7% | 1.30 |
| 04 | 31 | +3.22 | 61.3% | 1.27 | | 17 | 47 | −2.61 | 48.9% | 0.86 |
| 05 | 50 | +7.15 | 52.0% | 1.32 | | 18 | 51 | +13.17 | 58.8% | 1.76 |
| 06 | 56 | +24.31 | 58.9% | 2.47 | | 19 | 51 | +2.96 | 49.0% | 1.18 |
| 07 | 52 | +2.50 | 53.8% | 1.14 | | 20 | 43 | +31.45 | 46.5% | 2.80 |
| 08 | 63 | −4.88 | 41.3% | 0.81 | | 21 | 12 | +16.02 | 75.0% | 6.16 |
| 12 | 69 | +7.01 | 53.6% | 1.28 | | 22 | 37 | +4.14 | 45.9% | 1.32 |
| 14 | 170 | +1.37 | 47.6% | 1.02 | | | | | | |
(Entry hour is the open of the bar after the signal bar; M5 blocks close hours 0, 1, 2, 9, 10,
11, 13, 16 and leaves 21 open.)
| Weekday | Trades | Total R | WR | PF | Max DD |
|---|---|---|---|---|---|
| Mon | 172 | +20.80 | 55.8% | 1.36 | 12.3 |
| Tue | 180 | +24.22 | 52.8% | 1.36 | 13.2 |
| Wed | 200 | +30.84 | 48.5% | 1.43 | 11.4 |
| Thu | 167 | +5.27 | 48.5% | 1.08 | 13.3 |
| Fri | 177 | +41.41 | 50.3% | 1.62 | 9.2 |
| Sun | 9 | +4.55 | 77.8% | 3.22 | 1.0 |
Tail: median R +0.031; largest win +5.53R, largest loss −1.84R; mean hold 76.6 bars.
## 3. What to do with this
Nothing ships. Two honest readings for the owner:
- The five-year M5 figure (+127R) is not "five years of edge"; it is the validated 2024-09 →
2026-09 book plus three and a half years in which the dollar floors kept M5 almost out of the
market and the few trades that got through lost a little. The three-year cap already keeps
decisions inside the period where the gates behave as designed.
- The concentration in 2026-Q1 (71% of the window's R) is a fact about the shipped M5 edge, not
about the window length. It is consistent with every earlier M5 reading and is the reason the
advisory outcomes over a quiet fortnight can look nothing like the book.
If the owner wants the floor effect measured rather than inferred, a `--record --record-bars`
run of this window on the twin (seconds) with a by-year count of `SPREAD GATE` and `LOW ATR
FILTER` warnings per analysed bar would settle it. Changing the floors is a rule-1 event in its
own right (`round_trip_cost` drives the live target floor, CLAUDE.md rule 1) and is not proposed.
+304
View File
@@ -0,0 +1,304 @@
# New data sources for the direction call — a report-only study, 2026-10-01
Owner decision 2026-09-30: after the daily-trend work, study new data sources and report; nothing
ships from this file. The engine reads gold's own price and nothing else. The question here is
whether anything OUTSIDE gold's own chart tells the direction better than gold's chart already
does. Sections 1 to 3 were written before any data was fetched.
## 1. Sources
| # | Source | What it is | Where from | When a value is usable |
|---|---|---|---|---|
| 1 | Dollar | EUR/USD daily candles, read upside down (a falling EUR/USD is a rising dollar) | Oanda v20 candles, the live feed's own token | daily close + 0 (same rule as gold's daily bars) |
| 2 | Silver | XAG/USD daily candles | Oanda v20 | daily close |
| 3 | US 10-year note | USB10Y_USD daily candles (price up = yields down), if Oanda offers it | Oanda v20 | daily close |
| 4 | Real yield | US 10-year inflation-indexed yield, `DFII10` | FRED (public CSV) | the day's value from two days later (it is published the next US afternoon) |
| 5 | Broad dollar index | `DTWEXBGS` | FRED | same two-day rule |
| 6 | Gold fund holdings | SPDR Gold Shares, shares outstanding (each share is a fixed slice of the vault, so creations and redemptions are the fund's gold flow) | the fund manager's public history file (`navhist-us-en-gld.xlsx`); the older tonnes archive now serves a PDF | the day's value from two days later |
| 7 | Futures positioning | CFTC disaggregated report, gold (code 088691), managed-money net long | CFTC public reporting site | a Tuesday report from the following Monday |
| 8 | Oanda client positioning | position book for XAU_USD: the share of Oanda clients' open gold positions that are long (by count) | Oanda v20 `/v3/instruments/XAU_USD/positionBook`, one snapshot per day at 00:00 UTC | at once (the snapshot is taken at the daily close) |
Everything fetched stays in the session scratchpad. Nothing is added to the repository's cache.
Amendment before any analysis (2026-10-01, after the access probes and nothing else): the plan
expected source 8 to be gone. It is not: on the practice API the position book answers for every
date probed back to 2017, so it joins the study with the definition below. The probe printed ten
long-share readings (between 60% and 75% long, higher at higher gold prices) and no forward
returns; that is everything that was seen before the definition was fixed. Source 6 uses shares
outstanding instead of tonnes because that is the file that can still be downloaded.
## 2. One state per source
The same shape as the shipped daily trend, so the comparison is like for like. For a daily series
`x` (Sunday stub bars dropped for the Oanda candles): `up` when `x` is above its 50-day EMA AND
above its value 60 days earlier, `down` when below both, else `unclear`. Read for gold:
| Source | Gold-friendly state | Gold-hostile state |
|---|---|---|
| Dollar (EUR/USD) | EUR/USD up (dollar falling) | EUR/USD down |
| Silver | up | down |
| 10-year note price | up (yields falling) | down |
| Real yield | down | up |
| Broad dollar index | down | up |
| Fund holdings | shares outstanding above the level 20 trading days earlier | below |
| Futures positioning | managed-money net long above its level 4 reports earlier | below |
| Oanda client positioning | long share `down` (the crowd is cutting longs or adding shorts) | long share `up` (the crowd is adding longs) |
The positioning state is read the contrarian way on purpose and before looking: small traders
tend to buy dips and sell rallies, so a growing long share is expected to go with a falling price.
Test A is two-sided, so if the truth is the other way round it will show as a real spread with the
opposite sign and is reported as such; Test B is judged on the declared reading only.
## 3. Two tests and the verdict rule
**Test A — forward move of gold.** Every closed gold daily bar from 2019-01-01 to 2026-08-07.
Forward move = gold's close 5 days later minus today's close, in units of gold's own 14-day ATR
(and the same for 20 days). For each source: mean forward move on gold-friendly days minus mean
on gold-hostile days, (i) over all days and (ii) inside each of gold's own three daily-trend
states (rising / falling / unclear), in two eras (2019–2023, 2024–2026). The spread is called real
only if it has the same sign in both eras and its t-value, computed on every fifth day so the
5-day windows do not overlap, is at least 2 in each era.
**Test B — the shipped books.** M15 LONG (daily-trend veto on, 1473 trades) and M5 LONG (1052
trades), $0.65. Each trade is labelled `friendly` / `hostile` / `unclear` from the source state at
its decision time, mirrored for a SELL. A source is interesting on a timeframe only if its
`hostile` trades are at least 8% of the book and lose (total R < 0) in BOTH eras.
**Verdict per source.**
- `GO` (worth a pre-registered campaign and the feed work it needs) only if Test A calls the
spread real INSIDE gold's own trend states (that is the "beyond gold's own chart" part) AND
Test B finds the hostile trades losing in both eras on at least one timeframe.
- `WATCH` if exactly one of the two holds.
- `NO-GO` otherwise, or when the source cannot be fetched reliably.
A `GO` is a ticket to a campaign, never a shipping decision: a live feed for a second instrument
needs its own provider path, cache, staleness rule and live == backtest proof.
Two points of method were fixed while the analysis script was written, before it produced any
output. (1) The broad dollar index is published weekly (the H.10 release, on Monday, for the week
before), not daily, so the "two-day rule" of the table would have looked ahead: each week's final
state is used from the Tuesday after that week. (2) "Inside gold's own trend states" is computed
as one stratified number per era: the forward move minus the mean forward move of that day's gold
trend state (rising / unclear / falling), friendly days against hostile days, t-value on every
fifth day. That is the column the verdict reads; the unstratified spread is printed beside it.
## 4. Results
Run 2026-10-01. Fetched into the session scratchpad: Oanda daily candles for EUR_USD, XAG_USD,
USB10Y_USD and XAU_USD (2015-01-01 → 2026-09-30, about 3,650 bars each, about 600 Sunday stubs
dropped); the Oanda XAU_USD position book, one snapshot per day at 00:00 UTC from 2018-06-01
(2,609 days asked, 97.7% answered at exactly 00:00, the rest from the last snapshot up to six
hours earlier, 45 days with none, all before mid-2022); FRED `DFII10` and `DTWEXBGS`; the gold
fund's history file (5,540 days since 2004); 1,059 weekly CFTC reports since 2006.
### 4.1 What was fetched
| Source | Days with a state, 2019-01-01 → 2026-08-07 | friendly / unclear / hostile | Same side as gold's own daily trend (days where both are clear) |
|---|---|---|---|
| 1 dollar (EUR/USD) | 1973 | 37% / 18% / 44% | 67% of 1363 |
| 2 silver | 1973 | 49% / 21% / 31% | 88% of 1387 |
| 3 10-year note price | 1973 | 39% / 16% / 46% | 73% of 1398 |
| 4 real yield (DFII10) | 1973 | 42% / 20% / 39% | 73% of 1353 |
| 5 broad dollar index | 1973 | 39% / 19% / 42% | 62% of 1362 |
| 6 gold fund shares | 1973 | 51% / 0% / 49% | 77% of 1639 |
| 7 futures positioning | 1973 | 51% / 0% / 49% | 59% of 1644 |
| 8 Oanda client positioning | 1973 | 37% / 15% / 48% | 36% of 1414 |
Gold itself, same days: rising 58%, unclear 17%, falling 25%; mean 5-day forward move +0.27 ATR (rising +0.32, unclear +0.30, falling +0.14).
### 4.2 Test A — gold's forward move, friendly days minus hostile days (in daily ATRs)
| Source | Era | Days friendly / hostile | 5-day spread, all days (t) | 5-day spread inside gold's trend states (t) | 20-day spread inside gold's trend states (t) | Real? |
|---|---|---|---|---|---|---|
| 1 dollar (EUR/USD) | 2019–2023 | 443 / 623 | -0.04 (-0.3) | -0.05 (-0.4) | +0.35 (+0.7) | |
| 1 dollar (EUR/USD) | 2024–2026 | 296 / 249 | +0.21 (+0.7) | +0.20 (+0.7) | +0.80 (+0.5) | |
| 1 dollar (EUR/USD) | verdict | | | same sign in both eras: no; |t| ≥ 2 in both: no | | not real |
| 2 silver | 2019–2023 | 548 / 449 | -0.21 (-0.9) | -0.25 (-1.0) | -1.00 (-1.2) | |
| 2 silver | 2024–2026 | 411 / 160 | -0.10 (-0.7) | -0.20 (-0.7) | -0.64 (-0.2) | |
| 2 silver | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 3 10-year note price | 2019–2023 | 515 / 603 | +0.08 (+0.3) | +0.04 (+0.1) | +0.46 (+0.5) | |
| 3 10-year note price | 2024–2026 | 253 / 295 | +0.23 (+0.4) | +0.17 (+0.4) | +1.02 (-0.2) | |
| 3 10-year note price | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 4 real yield (DFII10) | 2019–2023 | 583 / 468 | +0.04 (+0.4) | +0.01 (+0.2) | -0.20 (-0.5) | |
| 4 real yield (DFII10) | 2024–2026 | 242 / 294 | +0.20 (+0.7) | +0.14 (+0.6) | +1.22 (+0.5) | |
| 4 real yield (DFII10) | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 5 broad dollar index | 2019–2023 | 497 / 536 | +0.03 (-0.1) | +0.02 (-0.1) | -0.12 (-0.5) | |
| 5 broad dollar index | 2024–2026 | 269 / 293 | +0.02 (-0.2) | +0.04 (-0.0) | +0.71 (+0.1) | |
| 5 broad dollar index | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 6 gold fund shares | 2019–2023 | 609 / 681 | +0.20 (+1.4) | +0.17 (+1.2) | +0.37 (+1.0) | |
| 6 gold fund shares | 2024–2026 | 393 / 281 | +0.40 (+1.7) | +0.36 (+1.8) | +1.09 (-0.1) | |
| 6 gold fund shares | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 7 futures positioning | 2019–2023 | 640 / 656 | +0.06 (+0.0) | +0.04 (-0.1) | +1.03 (+0.6) | |
| 7 futures positioning | 2024–2026 | 369 / 308 | -0.01 (+0.3) | +0.11 (+0.8) | +1.23 (+1.7) | |
| 7 futures positioning | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
| 8 Oanda client positioning | 2019–2023 | 636 / 428 | +0.20 (+1.2) | +0.19 (+1.1) | +0.53 (+0.1) | |
| 8 Oanda client positioning | 2024–2026 | 97 / 516 | -0.01 (-0.6) | +0.33 (+0.3) | n/a (too few friendly days) | |
| 8 Oanda client positioning | verdict | | | same sign in both eras: yes; |t| ≥ 2 in both: no | | not real |
### 4.3 Test B — the shipped books, by the source's state at the trade's decision time
**M15 LONG (daily-trend veto on): 1473 trades, +223.3R** (era A = 2019–2023: 728 trades, +36.8R; era B = 2024–2026: 745 trades, +186.6R)
| Source | Trade state | Trades (share) | Total R / mean | Era A R (trades) | Era B R (trades) | Hostile ≥ 8% and losing in both eras |
|---|---|---|---|---|---|---|
| 1 dollar (EUR/USD) | friendly | 725 (49%) | +127.3 / +0.176 | +32.0 (345) | +95.4 (380) | |
| 1 dollar (EUR/USD) | unclear | 252 (17%) | +13.0 / +0.052 | -15.3 (127) | +28.3 (125) | |
| 1 dollar (EUR/USD) | hostile | 496 (34%) | +83.0 / +0.167 | +20.1 (256) | +62.9 (240) | no |
| 2 silver | friendly | 937 (64%) | +154.9 / +0.165 | +26.8 (406) | +128.0 (531) | |
| 2 silver | unclear | 304 (21%) | +1.3 / +0.004 | -18.2 (196) | +19.5 (108) | |
| 2 silver | hostile | 232 (16%) | +67.1 / +0.289 | +28.1 (126) | +39.0 (106) | no |
| 3 10-year note price | friendly | 753 (51%) | +107.0 / +0.142 | +34.6 (397) | +72.4 (356) | |
| 3 10-year note price | unclear | 254 (17%) | +39.1 / +0.154 | +11.3 (121) | +27.8 (133) | |
| 3 10-year note price | hostile | 466 (32%) | +77.2 / +0.166 | -9.2 (210) | +86.4 (256) | no |
| 4 real yield (DFII10) | friendly | 723 (49%) | +97.7 / +0.135 | +45.5 (405) | +52.2 (318) | |
| 4 real yield (DFII10) | unclear | 313 (21%) | +30.8 / +0.098 | -12.5 (142) | +43.2 (171) | |
| 4 real yield (DFII10) | hostile | 437 (30%) | +94.9 / +0.217 | +3.8 (181) | +91.1 (256) | no |
| 5 broad dollar index | friendly | 647 (44%) | +122.5 / +0.189 | +37.5 (315) | +85.0 (332) | |
| 5 broad dollar index | unclear | 277 (19%) | +17.2 / +0.062 | -4.7 (168) | +21.8 (109) | |
| 5 broad dollar index | hostile | 549 (37%) | +83.7 / +0.152 | +3.9 (245) | +79.8 (304) | no |
| 6 gold fund shares | friendly | 1031 (70%) | +183.6 / +0.178 | +52.7 (519) | +130.9 (512) | |
| 6 gold fund shares | unclear | 7 (0%) | -4.7 / -0.668 | -2.9 (4) | -1.8 (3) | |
| 6 gold fund shares | hostile | 435 (30%) | +44.4 / +0.102 | -13.1 (205) | +57.5 (230) | no |
| 7 futures positioning | friendly | 789 (54%) | +115.5 / +0.146 | +22.3 (422) | +93.3 (367) | |
| 7 futures positioning | unclear | 0 (0%) | +0.0 / | +0.0 (0) | +0.0 (0) | |
| 7 futures positioning | hostile | 684 (46%) | +107.8 / +0.158 | +14.5 (306) | +93.3 (378) | no |
| 8 Oanda client positioning | friendly | 394 (27%) | +41.7 / +0.106 | +15.9 (326) | +25.9 (68) | |
| 8 Oanda client positioning | unclear | 198 (13%) | +26.9 / +0.136 | +13.7 (122) | +13.2 (76) | |
| 8 Oanda client positioning | hostile | 881 (60%) | +154.7 / +0.176 | +7.2 (280) | +147.5 (601) | no |
**M5 LONG: 1052 trades, +128.1R** (era A = 2019–2023: 258 trades, -10.4R; era B = 2024–2026: 794 trades, +138.5R)
| Source | Trade state | Trades (share) | Total R / mean | Era A R (trades) | Era B R (trades) | Hostile ≥ 8% and losing in both eras |
|---|---|---|---|---|---|---|
| 1 dollar (EUR/USD) | friendly | 439 (42%) | +85.3 / +0.194 | -11.5 (103) | +96.8 (336) | |
| 1 dollar (EUR/USD) | unclear | 211 (20%) | +28.6 / +0.136 | -0.9 (34) | +29.5 (177) | |
| 1 dollar (EUR/USD) | hostile | 402 (38%) | +14.2 / +0.035 | +2.0 (121) | +12.2 (281) | no |
| 2 silver | friendly | 556 (53%) | +89.3 / +0.161 | -7.6 (113) | +96.8 (443) | |
| 2 silver | unclear | 143 (14%) | +7.3 / +0.051 | -9.9 (45) | +17.2 (98) | |
| 2 silver | hostile | 353 (34%) | +31.5 / +0.089 | +7.1 (100) | +24.4 (253) | no |
| 3 10-year note price | friendly | 432 (41%) | +51.6 / +0.120 | -0.6 (119) | +52.2 (313) | |
| 3 10-year note price | unclear | 169 (16%) | -9.5 / -0.056 | -13.7 (49) | +4.2 (120) | |
| 3 10-year note price | hostile | 451 (43%) | +86.0 / +0.191 | +3.9 (90) | +82.1 (361) | no |
| 4 real yield (DFII10) | friendly | 377 (36%) | +4.2 / +0.011 | -15.0 (115) | +19.2 (262) | |
| 4 real yield (DFII10) | unclear | 186 (18%) | +32.3 / +0.174 | -7.0 (45) | +39.3 (141) | |
| 4 real yield (DFII10) | hostile | 489 (46%) | +91.6 / +0.187 | +11.6 (98) | +80.0 (391) | no |
| 5 broad dollar index | friendly | 447 (42%) | +131.2 / +0.294 | -0.7 (91) | +131.9 (356) | |
| 5 broad dollar index | unclear | 170 (16%) | +8.1 / +0.047 | -6.0 (40) | +14.1 (130) | |
| 5 broad dollar index | hostile | 435 (41%) | -11.2 / -0.026 | -3.7 (127) | -7.5 (308) | YES |
| 6 gold fund shares | friendly | 600 (57%) | +86.9 / +0.145 | -15.8 (142) | +102.7 (458) | |
| 6 gold fund shares | unclear | 3 (0%) | -1.8 / -0.608 | +0.0 (0) | -1.8 (3) | |
| 6 gold fund shares | hostile | 449 (43%) | +43.0 / +0.096 | +5.4 (116) | +37.6 (333) | no |
| 7 futures positioning | friendly | 493 (47%) | +41.1 / +0.083 | -17.0 (120) | +58.1 (373) | |
| 7 futures positioning | unclear | 0 (0%) | +0.0 / | +0.0 (0) | +0.0 (0) | |
| 7 futures positioning | hostile | 559 (53%) | +87.0 / +0.156 | +6.6 (138) | +80.4 (421) | no |
| 8 Oanda client positioning | friendly | 313 (30%) | +17.9 / +0.057 | -1.9 (99) | +19.8 (214) | |
| 8 Oanda client positioning | unclear | 92 (9%) | +2.3 / +0.025 | +0.5 (40) | +1.9 (52) | |
| 8 Oanda client positioning | hostile | 647 (62%) | +107.9 / +0.167 | -8.9 (119) | +116.9 (528) | no |
### 4.4 Verdict per source
| Source | Test A: real spread beyond gold's own trend | Test B: hostile trades ≥ 8% and losing in both eras (M15 / M5) | Verdict |
|---|---|---|---|
| 1 dollar (EUR/USD) | no | no / no | NO-GO |
| 2 silver | no | no / no | NO-GO |
| 3 10-year note price | no | no / no | NO-GO |
| 4 real yield (DFII10) | no | no / no | NO-GO |
| 5 broad dollar index | no | no / yes | WATCH |
| 6 gold fund shares | no | no / no | NO-GO |
| 7 futures positioning | no | no / no | NO-GO |
| 8 Oanda client positioning | no | no / no | NO-GO |
### 4.5 The one flag, looked at more closely (after the fact: description, not a test)
**M5 LONG: gold's own daily trend (rows) x 5 broad dollar index (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +118.8 (352) | +1.9 (105) | -0.1 (215) |
| unclear | +6.8 (33) | -2.1 (12) | +2.3 (73) |
| against | +5.6 (62) | +8.3 (53) | -13.3 (147) |
**M5 LONG: gold's own daily trend (rows) x 1 dollar (EUR/USD) (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +76.8 (372) | +23.0 (109) | +20.7 (191) |
| unclear | +4.5 (37) | -5.5 (19) | +7.9 (62) |
| against | +4.0 (30) | +11.1 (83) | -14.4 (149) |
**M5 LONG: broad-dollar hostile trades by year** — 2019: +1.4 (14); 2020: -3.4 (45); 2021: -0.6 (21); 2022: -1.1 (36); 2023: -0.1 (11); 2024: +3.5 (68); 2025: -8.5 (107); 2026: -2.4 (133)
**M5 LONG: broad-dollar friendly trades by year** — 2019: -1.5 (3); 2020: +0.0 (45); 2021: -0.0 (9); 2022: +0.0 (7); 2023: +0.7 (27); 2024: -0.1 (19); 2025: +40.3 (160); 2026: +91.7 (177)
W3 friendly: 397 trades (46%), +145.1R, halves +8.2 / +136.9
W3 unclear: 150 trades (17%), +15.6R, halves -3.0 / +18.7
W3 hostile: 323 trades (37%), -7.1R, halves +11.6 / -18.7
**M15 LONG (veto on): gold's own daily trend (rows) x 5 broad dollar index (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +93.2 (561) | +48.0 (196) | +68.5 (416) |
| unclear | +29.3 (86) | -30.8 (81) | +15.2 (133) |
| against | +0.0 (0) | +0.0 (0) | +0.0 (0) |
**M15 LONG (veto on): gold's own daily trend (rows) x 1 dollar (EUR/USD) (columns)** — total R (trades)
| Gold's own daily trend vs the trade | dollar friendly | dollar unclear | dollar hostile |
|---|---|---|---|
| with | +99.4 (635) | +41.8 (176) | +68.5 (362) |
| unclear | +27.9 (90) | -28.8 (76) | +14.5 (134) |
| against | +0.0 (0) | +0.0 (0) | +0.0 (0) |
**M15 LONG (veto on): broad-dollar hostile trades by year** — 2019: +12.5 (57); 2020: +11.5 (74); 2021: -6.7 (44); 2022: -9.2 (43); 2023: -4.1 (27); 2024: +65.1 (179); 2025: +9.9 (95); 2026: +4.8 (30)
**M15 LONG (veto on): broad-dollar friendly trades by year** — 2019: +4.7 (16); 2020: +25.0 (114); 2021: -8.9 (46); 2022: +8.2 (51); 2023: +8.5 (88); 2024: -4.4 (56); 2025: +52.1 (192); 2026: +37.3 (84)
W3 friendly: 373 trades (45%), +103.5R, halves +4.0 / +99.4
W3 unclear: 130 trades (16%), +28.2R, halves +6.2 / +22.1
W3 hostile: 321 trades (39%), +73.4R, halves +66.5 / +6.9
### 4.6 Reading
- **Nothing outside gold's own chart passes Test A.** For all eight sources the 5-day forward move
of gold on "friendly" days is not reliably different from "hostile" days once gold's own daily
trend is accounted for (largest t-values 1.2 and 1.8, the gold fund's share count; everything
else is under 1.2 in at least one era). The familiar stories (a falling dollar, falling real
yields, rising silver, fund inflows) describe gold's past move well: they sit on the same side
as gold's own daily trend on 59% to 88% of clear days. They do not add a forecast to it.
- **Oanda client positioning is available but not useful in this form.** The plan expected the
endpoint to be gone; it answers, with history back to 2017. Read the contrarian way (a growing
long share is hostile), the spread has the expected sign and no weight (t 1.1 and 0.3), and in
2024–2026 the crowd was adding longs on 516 of 613 clear days while gold rose. The trades taken
on those "hostile" days are the best part of both books. Only the daily state was tested; the
intraday snapshots (one per hour) were not.
- **The one flag is the broad dollar index on M5** (`WATCH`): the 435 M5 trades taken against the
dollar's trend lost 11.2R in total and lost in both eras, while the 447 taken with it made
+131.2R. The closer look shows what that is. All of the M5 profit sits in one cell: trades with
gold's own daily trend AND with the dollar's trend (+118.8R over 352 trades); the same trades
against the dollar's trend made nothing (−0.1R over 215). But every R of the friendly side was
earned in 2025 and 2026 (+40.3R and +91.7R), that is inside the one long rally from 2,600 to the
March 2026 peak (in 2025 the broad dollar fell 7.5% while gold rose 65%); before 2025 the
dollar's state separates nothing on M5 because the M5 book earned nothing at all. In the 3-year
window the hostile trades are positive in the first half (+11.6R) and negative only in the
second (−18.7R). On M15 the same state separates nothing: the hostile
trades made +65R in 2024 alone, a year when gold and the dollar rose together. The timelier
dollar measure (EUR/USD) shows the same lean on M5 without losing in both eras. So this is one
market phase seen from another angle, not a second source of information; it stays a
hypothesis, and it is the same picture as the context screens: M5 pays only when everything
agrees and has paid only since 2025.
### 4.7 Verdict
| Source | Verdict | Why |
|---|---|---|
| Dollar via EUR/USD | NO-GO | no forward spread; hostile trades are not losers in both eras |
| Silver | NO-GO | 88% the same information as gold's own trend; the spread has the wrong sign |
| US 10-year note | NO-GO | no forward spread |
| Real yield | NO-GO | no forward spread; on M5 the "hostile" trades are the better ones |
| Broad dollar index | WATCH | M5 trades against it lose in both eras, but the forward test finds nothing and the effect is the 2025–2026 phase; weekly data, four to ten days old when used |
| Gold fund share count | NO-GO | the nearest miss in the forward test (t 1.2 and 1.8), nothing on the books |
| Futures positioning | NO-GO | no forward spread, no difference on the books |
| Oanda client positioning | NO-GO in this daily form | available, but the contrarian reading failed in the rising years |
**Nothing ships and no feed is added.** The direction call was improved by the one input that
passed a formal test, gold's own daily chart on M15. If the M5 question is taken up again, the
honest test is forward: `signals.daily_trend` is now logged on every scan, so a few months of live
M5 rows can be split by the daily trend, and by the dollar's trend after the fact, without touching
the signal code.
+181
View File
@@ -0,0 +1,181 @@
# Agent panel v2 — probabilities anchored to base rates — 2026-09-17
**Why.** The live review of 2026-09-04 → 09-17 (`signal_log_review_2026-09-17.md`, §5 and §7.1)
found the panel's verdict was tilted the wrong way: the macro persona's stance was the verdict,
its only input was headlines (a one-day-lagging echo of price, `news_sentiment_value_2026-09-04.md`),
opposed setups reached their first target more often than agreed ones (54% vs 42%), three of four
personas voted the same way on nearly every setup (technical 79% support, risk 79% oppose,
contrarian 65% oppose), and the stated confidence was uncalibrated. The chair had already been
shown anti-informative on 2026-09-03. Nothing versioned the prompts, so no change could be measured.
**What shipped (code, all advisory-only; the panel still never runs inside
`analyze_from_dataframes` and no gate reads it).**
| Piece | Where |
|---|---|
| Every persona returns `p_win` (0-100, the chance the trade closes profitably under its trailing exit) plus one `reason_for` and one `reason_against`; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose) | `agent_panel.py` `_TAKE_CONTRACT`, `derive_stance`, `_validate_take` (v1 stance+confidence replies still accepted and mapped to a pseudo-probability) |
| Verdict = equal-weight mean of the personas' `p_win`, quorum 3 of 4, no LLM aggregator (`verdict_source: "mean"`, the new default; `macro` and `chair` kept for comparison) | `mean_verdict`, `run_panel`, `app_config.py` |
| A code-computed `context` block in every payload: stop / target in ATR and R, cost-to-risk, the class **base rate** from the validated books, prior move 4 h / 24 h in ATR, distance from the 24 h extreme and its age, `move_started_at`, largest bar of the last 3 in ATR, signal-bar body, event timing against the class's expected hold, penalty tags with the book's verdict on them | `_price_context`, `_base_rate`, `agent_panel_base_rates.json` (built by `scripts/build_panel_base_rates.py` from the MD5-verified books) |
| Macro persona: no news doctrine; judges whether news is NEW relative to `move_started_at`; receives the ATR-normalised candles (as does the contrarian) | `_MACRO_SYSTEM`, `_panel_jobs` |
| Technical persona: indicator agreement declared uninformative; judges higher-timeframe conflict and key levels. Risk: only setup-specific risks versus the class. Contrarian: the engine's own warnings are not evidence | `_TECH_SYSTEM`, `_RISK_SYSTEM`, `_CONTRARIAN_SYSTEM` |
| `PANEL_PROMPT_VERSION` logged with every verdict and folded into the payload hash; per-persona probabilities and reasons, the exact per-persona payloads and the context logged (`panel_prompt_version`, `panel_probability`, `panel_probs_json`, `panel_payloads_json`, `panel_context_json`) | `signal_log.py` additive migration, `update_panel`, `log_panel_update` |
| Live frames handed to the add-ons through `report._frames` in `analyze_pair` (never in the core) | `signal_aggregator.py` |
| Plain-language "AI estimate" line on Telegram, CLI and dashboard; legacy sources render as before | `telegram_bot._panel_estimate_lines`, `main.py`, `static/dashboard.js` |
| Scorecard: Brier, log-loss, calibration bands, discrimination, constant-voter rate, ρ(p, R), counterfactual books, split by prompt version | `analyze_signal_log.probability_scorecard` (printed by `--stored-candles`) |
| Offline replay harness: rebuilds each persona's payload for every resolved logged setup (exact payloads when logged, else from the stored blobs, interned news and the kept Oanda cache, events evaluated as of the signal bar), runs the current prompts, scores them against the outcome and against the logged v1 takes and the base-rate predictor | `scripts/panel_replay.py` (`--dry-run`, `--max-calls`) |
| Backtester export `strategy` tag fixed to the generator's own stamp (label-only; every gold ATR setup was exported as `GENERIC`) | `backtest/gpu_backtest.py`; books re-pinned, see below |
Tests: `test_all.py` Module 35 (15 tests) plus the extended Module 23 parity test; suite
`RESULTS: 521 passed, 0 failed, 521 total`.
## Reference books re-pinned (label-only)
The five books were regenerated with the committed code (stashed) and with the fix, on the same
cache. Dropping the `strategy` column, every book is byte-identical to its previous pin; trades, R
and PF are unchanged. New labels: M15 W2 174 `XAU_ATR` / 757 `GENERIC`; M5 W2 32 / 411 / 338
`XAU_CP`; M15 OOS 1 / 354; H1 W3 57 / 196; H4 W3 53 / 61.
| Book | old MD5 | new MD5 |
|---|---|---|
| M5 W2 | `29120a1ee32979431d062f210d41a9d1` | `49f8e602fc6ae769713c94ef7b257810` |
| M15 W2 | `d185f099482a07f29502cb5a371a9bbf` | `b4aa07e33e17e23774c486a6900d673b` |
| M15 OOS | `6f8f24b49597ced28f50a19e08c3472f` | `9fd624fb387c36052122bd1830cf9c32` |
| H1 W3 | `3b45d12bade823178b267c13bc698458` | `7b848c917dcb5c371ecbb9f8f7292278` |
| H4 W3 | `e75adae13c839b15736f69c0114e6aa5` | `186e677fbbd53cca735c94094c23ce05` |
`Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all ten runs.
## Base-rate table
`agent_panel_base_rates.json`: 45 classes (34 exact) from the relabelled M5 W2, M15 W2, H1 W3 and
H4 W3 books; key `timeframe|strategy|grade|direction`, fallbacks `tf|*|*|direction` and `tf|*|*|*`;
the exact class is used only with n ≥ 30. Examples: M15 all 931 trades win 60.5%, median hold 30
bars; M15 SELL 296 / 54.4%; M15 `XAU_ATR` A SELL 78 / 50.0%, median hold 6 bars; M5 `XAU_CP` B
BUY 321 / 51.7%; H1 all 253 / 64.4%.
## Pre-registered acceptance (from the approved plan)
On the resolved logged setups (37 today), a new prompt version ships only if: (1) no persona's
modal stance is above 65%; (2) Brier ≤ the logged v1 takes and ≤ the base-rate-only predictor;
(3) the skip-OPPOSE counterfactual does not remove a net-positive bucket. Outcome = advisory R > 0
at $0.65. Predictors compared on the same events: base-rate only (the class win rate for every
trade), the logged v1 takes (stance × confidence → pseudo-probability; 27 events carry them), and
the new prompts run live through DeepSeek on the rebuilt payloads (4 calls per event).
## Replay round 1 (draft prompts)
| predictor | n | Brier | log-loss | mean p | modal stance | constant % | R skip ≤ 40 |
|---|---|---|---|---|---|---|---|
| base-rate only | 37 | 0.263 | 0.719 | 54.8 | neutral | 81 | −7.15 (nothing skipped) |
| v1 logged, mean | 27 | 0.270 | 0.738 | 48.2 | neutral | 48 | −4.00 |
| v1 logged, macro | 27 | 0.299 | 0.855 | 56.5 | support | 44 | −2.82 |
| v1 logged, technical | 27 | 0.371 | 1.011 | 71.7 | support | 78 | −6.47 |
| v1 logged, risk | 27 | 0.343 | 0.947 | 25.7 | oppose | 78 | +0.03 |
| v2 draft, mean | 37 | 0.257 | 0.707 | 51.3 | neutral | 95 | −7.15 (nothing skipped) |
Like-for-like on the 27 events with v1 takes: Brier v2 0.257, v1 mean 0.270, v1 macro 0.299,
base rate 0.259, coin flip 0.250. Rank correlation of p with R: v2 −0.05, v1 mean +0.14, v1 macro
+0.36, base rate +0.16 (n = 27; the ±0.35 confidence band swallows all of them).
Reading: the draft is calibrated (it sits 3.5 points under the base rate with a spread of 1.7
points) and no longer directional, but it carries no rank information: winners averaged p = 50.9,
losers 51.7. The old macro verdict had some ordering (+0.36) and terrible calibration (82%
confidence on the three losing FOMC shorts); the new one has calibration and no ordering. Criterion
(2) passes, (3) is vacuous (no OPPOSE issued), (1) fails **in the opposite sense to the one it was
written for**: the personas cluster on neutral around the base rate instead of voting a constant
direction. That is hedging, not bias.
**A flaw in the draft prompts, found from the per-event reasons.** The five lowest probabilities
went to the Sep 10 and Sep 14 shorts that won (+1.31, +1.05, +0.70R), because the technical and
contrarian prompts told the personas to lower p when "the move has already run far" or "a big
candle just printed". That instruction encoded the live fortnight's pattern, which the two-year
books contradict (stretched entries are the system's best trades, 2026-08-04; the big-candle gate
was re-rejected in the review: those buckets are +32R and +60R). The final prompts remove that
instruction, state the book's finding in the context legend and in both personas, and limit
downward moves to higher-timeframe conflict, key levels, event timing inside the expected hold,
news newer than the move, and a stop far wider than the class norm.
## Replay round 2 (final prompts, the text that ships)
Same 37 events, same rebuilt payloads, the final prompt text (the technical and contrarian
personas told that a long prior move or a big candle is not a reason by itself; downward moves
limited to higher-timeframe conflict, key levels, event timing, news newer than the move, and a
stop far wider than the class norm).
| predictor | n | Brier | log-loss | mean p | modal stance | directional % | R skip ≤ 40 | R ≥ 60 only |
|---|---|---|---|---|---|---|---|---|
| base-rate only | 37 | 0.263 | 0.719 | 54.8 | neutral | 19 | −7.15 | −3.07 |
| v1 logged, mean | 27 | 0.270 | 0.738 | 48.2 | neutral | 52 | −4.00 | −1.66 |
| v1 logged, macro | 27 | 0.299 | 0.855 | 56.5 | support | 56 | −2.82 | −2.16 |
| **v2 final, mean** | 37 | **0.262** | 0.717 | 53.8 | neutral | 8 | −7.15 | −1.98 |
| v2 final, macro | 37 | 0.259 | 0.712 | 53.4 | neutral | 11 | | |
| v2 final, technical | 37 | 0.270 | 0.734 | 54.2 | neutral | 16 | | |
| v2 final, risk | 37 | 0.258 | 0.709 | 53.3 | neutral | 11 | | |
| v2 final, contrarian | 37 | 0.262 | 0.718 | 54.1 | neutral | 11 | | |
Like-for-like on the 27 events with v1 takes: Brier v2 final 0.260, v1 mean 0.270, v1 macro
0.299, base rate 0.259. Rank correlation of p with R: v2 final +0.13 (round 1 −0.05), v1 mean
+0.14, v1 macro +0.36. Calibration of the v2 mean: 34 events in the 40–60 band averaged p = 53
and won 47%; the 3 events at 60–61 won 1. The final personas sit 1.0 point under the base rate
(sd 1.8, no event moved 5 points or more); winners averaged p = 53.1, losers 54.3.
Acceptance: (2) passes — Brier below the logged v1 takes and equal to the base-rate predictor
(0.262 vs 0.263; 0.260 vs 0.259 on the common 27); (3) passes vacuously (no OPPOSE issued); (1)
fails as written (92% neutral) and passes as re-specified below (directional share 8%). The
correction removed the chase penalty: the winning Sep 10 / Sep 14 shorts moved from 45–47 to
49–51, and the reasons against now name what the books do not already price (a release due in 20
minutes, price sitting on a support level) instead of "the move already ran far".
Cost: 2 × 148 DeepSeek calls (both rounds), inside the shared daily cap of 500.
## Decision and what it means for the alert
The shipped verdict is a probability anchored to the validated base rate that moves only for
facts the books do not already price. On this sample it is calibrated and uninformative: the line
will usually read "about a 50% chance … (usual for this kind of setup: 55%)". That is the honest
state of the evidence — the 65-feature sizing model was a clean null too — and it is strictly
better than the previous line, which asserted 82% confidence in the wrong direction. The
information the line adds is the base rate itself, the biggest concrete worry (event inside the
hold, news newer than the move), and each persona's reason. It never gates; the user guide says
so.
Re-specified criterion (1) for future prompt versions: the share of *directional* stances
(support + oppose) on one side must not exceed 65% — neutral clustering around a calibrated base
rate is not the pathology the criterion targets.
## Persona set (owner decision, same day)
The owner's rule: a persona that does not give correct guidance is removed, to save confusion,
tokens and log space. Evidence per persona:
| persona | v1 (27 outcomes) | v2 final (37 outcomes) | verdict |
|---|---|---|---|
| macro | Brier 0.299, rank correlation with R **+0.36**, 88% / 25% win split on 2026-09-03 | Brier 0.259, ρ −0.06 | the only voice that ever showed ordering, and the only one holding facts the engine lacks (news) → **kept** |
| technical | support on 79%, Brier 0.371 (worst) | Brier 0.270 (worst), ρ −0.20 | re-reads the indicators the signal was built from → removed |
| risk | oppose on 79%, Brier 0.343 | Brier 0.258, ρ −0.02 | its inputs (stop in ATR, cost, event timing) are now computed in the context block → removed |
| contrarian | oppose on 65%, Brier 0.322 | Brier 0.262, ρ −0.05 | oppose by construction, no information → removed |
Shipped as `agent_panel.personas` (default `["macro"]`; `app_config` validates the names, `chair`
needs two or more, `macro` source needs the macro persona). One DeepSeek call per panel instead of
four (three quarters of the panel's token spend), `panel_payloads_json` about 4 KB instead of about
14 KB, `panel_takes_json` one entry, no persona votes line in the alert. The mean verdict with one
persona is that persona's probability (quorum = min(3, personas)). The full panel remains one
config line away for research replays (`scripts/panel_replay.py --personas macro,technical,risk,contrarian`).
## Next evaluation
- Every live v2 verdict logs its exact payloads, so `scripts/panel_replay.py` can re-score any
future prompt on identical evidence. At 100 resolved v2 outcomes (about five weeks): the
`--stored-candles` probability scorecard by prompt version. At 300: fit a probability-band →
observed-win-rate table on the first half by time, verify on the second, ship it as a numpy
lookup; only then revisit persona weights (owner rule).
- Do not iterate prompts on the current 37 events again: any further gain there is in-sample.
- Deployment: the live box needs the new code (both the bot and the dashboard daemon, which has
run `2ac7301` since Sep 10); no config change is required (`verdict_source` defaults to
`mean`); the additive migration runs on first open.
Commands: `venv/bin/python scripts/build_panel_base_rates.py --book 5min=<m5_w2.csv> --book
15min=<m15_w2.csv> --book 1h=<h1_w3.csv> --book 4h=<h4_w3.csv> --out agent_panel_base_rates.json`;
`venv/bin/python scripts/panel_replay.py --db logs/signals.db --dry-run`; `venv/bin/python
scripts/panel_replay.py --db logs/signals.db --max-calls 160 --out replay.csv`;
`venv/bin/python analyze_signal_log.py --db logs/signals.db --stored-candles`.
@@ -0,0 +1,183 @@
# Risk-reward floor and M15 grade-A SELL block — pre-registered campaign, 2026-09-26
Owner questions (2026-09-26): plans sometimes show a target smaller than the stop (RR 0.75 / 0.5);
and "we still lose money". The live review of `logs/signals.db` (2026-09-04 → 2026-09-25, 100
unique setups replayed on the scans' own stored candles) found: no stop-hunt signature (52 stop-outs,
71% ran ≥ 1R past the stop, the same as the books' 70–72%); −25.1R of the −37.8R raw live book were
consecutive-bar re-entries the backtester's 5-bar cooldown never takes; the cooldown view
(−12.7R / 57) sits at the 3rd–5th percentile of three-week windows of the two-year books.
This file was written BEFORE any candidate run. Rules first, results appended below them.
## Harness
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf <tf> --start <s>
--end <e> --spread <0.65|0.30> [--set ...] --export-csv <out>` (CUDA research twin; its parity
with `gpu_backtest.py` was re-proven today: the M15 and M5 two-year controls reproduce the pinned
MD5s `b4aa07e33e17e23774c486a6900d673b` and `49f8e602fc6ae769713c94ef7b257810`). Factory
configuration; `--set` is the only override. A passing candidate is re-run once on
`gpu_backtest.py` before it ships and the two MD5s must be equal.
Windows: W2 2024-09-05 → 2026-09-05 (halves split at 2025-09-05); OOS 2024-06-01 → 2025-03-30
(halves at the midpoint 2024-10-30); W3 2023-09-05 → 2026-09-05 (the owner's 3-year maximum,
halves at 2025-03-05; used for the max-DD clause). Costs $0.65 (headline) and $0.30.
## Candidates
| # | TF | `--set` | Hypothesis |
|---|---|---|---|
| B1 | M15 | `signals.min_rr_ratio=1.0` | Vetoes the grade-A XAU_ATR plan (RR 0.75 / 0.5); the signal falls through to the GENERIC swing setup whose M15 floor is already 1R. Every M15 plan then shows RR ≥ 1. Two-year book: XAU_ATR 174 / +5.7R / 24% wrong-direction vs GENERIC 757 / +140.5R / 16% |
| B2 | M5 | `signals.tp_floor_rr=1.0` | Retest of the 1R target floor on M5 (failed 2026-07-11 on Twelve Data bars at $0.30). Expected to fail again: it lifts low-ATR targets past the $9 spread gate |
| B3 | M15 | `xau_strategy.sell_min_quality=["A+","B"]` | Parked cycle-2 candidate: M15 grade-A SELLs 230 / −14.4R, negative in y1, y2, y3 and OOS (cycle_2026-09-10.md). Counter-fact on record: those shorts paid in 2021–22 |
| B4 | M15 | B1 + B3 | The shipped combination if both pass alone |
## Acceptance rule (CLAUDE.md rule 1, written before the runs)
A candidate PASSES only if, at $0.65, on W2: (a) total R ≥ control + 5R; (b) both halves ≥ the
control's halves − 1R (no half may fund the other); (c) PF ≥ 1.3 and ≥ control − 0.02; (d) OOS
total R ≥ control − 1R; (e) W3 max drawdown ≤ control + 10%; (f) the same sign of ΔR at $0.30.
A candidate that changes fewer than 20 trades on W2 is "no evidence", not a pass. B4 must pass
the rule on its own numbers, not by inheritance.
Judge run-level numbers, never the added/removed bucket alone (the 5-bar cooldown displaces
entries). Trade diff keyed on (entry time, direction).
## Results (all runs on the CUDA twin, `Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, 0 cache fallbacks; controls reproduce the pinned MD5s)
| TF | Window | Cost | Candidate | Trades | Total R | H1 | H2 | WR | PF | Max DD | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| M15 | W2 | $0.65 | control | 931 | +146.14 | +66.14 | +80.00 | 60.5% | 1.43 | 16.1 | b4aa07e3 |
| M15 | W2 | $0.65 | B1 | 931 | +144.16 | +65.09 | +79.07 | 60.3% | 1.42 | 16.1 | 80428e98 |
| M15 | W2 | $0.65 | **B3** | 813 | **+154.15** | +70.10 | +84.04 | 62.2% | 1.53 | 15.6 | 497520ef |
| M15 | W2 | $0.65 | B4 | 813 | +153.41 | +70.73 | +82.68 | 62.1% | 1.53 | 15.6 | 177b716d |
| M15 | W2 | $0.30 | control | 931 | +161.44 | +75.63 | +85.81 | 61.0% | 1.48 | 14.9 | 6bdea010 |
| M15 | W2 | $0.30 | B1 | 931 | +158.17 | +74.41 | +83.75 | 60.7% | 1.47 | 14.9 | 21dcfc9e |
| M15 | W2 | $0.30 | B3 | 813 | +167.53 | +78.61 | +88.91 | 62.6% | 1.59 | 14.5 | aff050f7 |
| M15 | W2 | $0.30 | B4 | 813 | +166.06 | +79.12 | +86.95 | 62.5% | 1.58 | 14.8 | 03a412b8 |
| M15 | OOS | $0.65 | control | 355 | +60.67 | +9.14 | +51.52 | 60.8% | 1.49 | 17.1 | 9fd624fb |
| M15 | OOS | $0.65 | B1 | 355 | +60.67 | +9.14 | +51.52 | 60.8% | 1.49 | 17.1 | 9fd624fb (identical) |
| M15 | OOS | $0.65 | B3 = B4 | 313 | +68.50 | +18.87 | +49.63 | 63.6% | 1.67 | 12.4 | 09b8a8e7 |
| M15 | OOS | $0.30 | control | 355 | +69.18 | +13.48 | +55.70 | 61.7% | 1.57 | 14.9 | 30a82817 |
| M15 | OOS | $0.30 | B3 = B4 | 313 | +76.21 | +22.92 | +53.29 | 64.5% | 1.77 | 11.1 | 5fc34cef |
| M15 | W3 | $0.65 | control | 1273 | +186.61 | +75.87 | +110.74 | 59.4% | 1.39 | 19.3 | 5194e5d4 |
| M15 | W3 | $0.65 | B1 | 1273 | +183.78 | +75.83 | +107.95 | 59.2% | 1.39 | 19.3 | 2d5cfc31 |
| M15 | W3 | $0.65 | B3 | 1105 | +195.79 | +76.87 | +118.92 | 61.1% | 1.49 | 20.9 | 8bed7199 |
| M15 | W3 | $0.65 | B4 | 1105 | +194.19 | +76.00 | +118.18 | 61.0% | 1.48 | 20.9 | 05201726 |
| M15 | W3 | $0.30 | control / B1 / B3 / B4 | 1273 / 1273 / 1105 / 1105 | +210.42 / +206.25 / +216.53 / +214.19 | | | | 1.45 / 1.44 / 1.55 / 1.54 | 17.7 / 17.7 / 19.6 / 19.6 | |
| M5 | W2 | $0.65 | control | 781 | +146.07 | +26.42 | +119.65 | 52.9% | 1.52 | 10.1 | 49f8e602 |
| M5 | W2 | $0.65 | B2 | 1105 | +194.94 | +54.11 | +140.83 | 53.6% | 1.50 | 15.9 | 351f9eee |
| M5 | W2 | $0.30 | control / B2 | 781 / 1105 | +162.09 / +220.32 | +31.71 / +65.22 | +130.37 / +155.10 | | 1.60 / 1.58 | 8.9 / 13.7 | |
| M5 | OOS | $0.65 | control | 86 | +9.47 | +0.01 | +9.46 | 53.5% | 1.33 | 8.2 | da497d74 |
| M5 | OOS | $0.65 | B2 | 235 | +13.17 | −6.01 | +19.18 | 52.3% | 1.17 | 12.3 | 3cece33b |
| M5 | OOS | $0.30 | control / B2 | 86 / 235 | +12.42 / +20.66 | +1.65 / −2.04 | +10.77 / +22.69 | | 1.46 / 1.28 | 6.7 / 9.5 | |
| M5 | W3 | $0.65 | control | 870 | +153.64 | +16.73 | +136.91 | 53.3% | 1.50 | 10.1 | 408de695 |
| M5 | W3 | $0.65 | B2 | 1333 | +223.87 | +47.49 | +176.38 | 53.4% | 1.48 | 15.9 | f89176d6 |
| M5 | W3 | $0.30 | control / B2 | 870 / 1333 | +172.40 / +256.20 | | | | 1.57 / 1.56 | 8.9 / 13.7 | |
Trade diffs on W2 at $0.65 (key = entry time + direction):
- B1: 0 added, 0 removed, 931 common. The 158 grade-A XAU_ATR setups become GENERIC (strategy
mix 915 / 16 vs 757 / 174); same bars, same direction, a swing-anchored stop instead of 2 × ATR.
- B3: 162 removed (−8.11R, all grade-A SELLs), 44 added (−0.11R, cooldown displacement), 769 common.
Direction mix 635 BUY / 178 SELL vs 635 / 296.
- B2: 392 added (+63.63R), 68 removed (+17.23R), 713 common.
### Verdicts against the pre-registered rule
| # | (a) ΔR ≥ +5 W2 | (b) halves | (c) PF | (d) OOS | (e) W3 DD ≤ +10% | (f) $0.30 sign | Verdict |
|---|---|---|---|---|---|---|---|
| B1 | −2.0 ✗ | −1.05 / −0.93 ✓ | 1.42 ✓ | 0 ✓ | 19.3 ✓ | −3.3 ✓ | **FAIL (a)**. Trade-identical; the GENERIC stop is not better than the 2 × ATR stop on the same grade-A signals (W3 −2.8R). Not shipped |
| B2 | +48.9 ✓ | +27.7 / +21.2 ✓ | 1.50 ✓ (−0.02) | +3.7 ✓ (but OOS PF 1.17, OOS H1 −6.0) | 15.9 vs 10.1 = +57% ✗ | +58.2 ✓ | **FAIL (e)**. Owner's call, see the ladder below |
| B3 | +8.0 ✓ | +4.0 / +4.0 ✓ | 1.53 ✓ | +7.8 ✓ | 20.9 vs 19.3 = +8% ✓ | +6.1 ✓ | **PASS a–f — SHIPPED** |
| B4 | +7.3 ✓ | ✓ | 1.53 ✓ | +7.8 ✓ | +8% ✓ | +4.6 ✓ | Passes, but 0.7R below B3 alone on W2 and 1.6R on W3; B1's part adds nothing. Not shipped |
B3 per year on W3 ($0.65, control → B3): 2023 (Sep–Dec) +13.2 → +10.8; 2024 +52.9 → +53.3;
2025 +77.3 → +81.2; 2026 (to Sep 5) +43.2 → +50.5. Removed grade-A SELLs by year: 2023 15 / +1.4R,
2024 76 / −1.3R, 2025 79 / −8.6R, 2026 60 / −5.8R. The max-DD episode is the same one in both books
(2023-12-01 → 2024-02-23), 19.3R → 20.9R. M15 SELLs remaining under B3 on W3: A+ 19 / +2.6R,
B 217 / −8.6R. The 5-year fact on the other side of the scale stands (shorts paid in 2021–22).
### M5 target-floor ladder (informational, run after B2's verdict; NOT pre-registered, NOT shipped)
`--set signals.tp_floor_rr=<x>` on M5, $0.65:
| Floor | W2 trades / R / PF / DD | OOS trades / R / PF / DD | W3 trades / R / PF / DD |
|---|---|---|---|
| 0.5 (shipped) | 781 / +146.1 / 1.52 / 10.1 | 86 / +9.5 / 1.33 / 8.2 | 870 / +153.6 / 1.50 / 10.1 |
| 0.6 | 868 / +159.1 / 1.51 / 10.2 | 105 / +1.5 / 1.04 / 12.5 | 979 / +164.5 / 1.46 / 17.1 |
| 0.75 | 975 / +192.4 / 1.57 / 13.4 | 147 / +13.5 / 1.31 / 9.4 | 1133 / +211.6 / 1.54 / 13.4 |
| 0.9 | 1053 / +209.3 / 1.58 / 13.5 | 194 / +22.8 / 1.41 / 7.5 | 1252 / +238.7 / 1.55 / 13.5 |
| 1.0 (B2) | 1105 / +194.9 / 1.50 / 15.9 | 235 / +13.2 / 1.17 / 12.3 | 1333 / +223.9 / 1.48 / 15.9 |
Reading: the 2026-07-11 verdict ("a 1R floor hurts M5") was made on Twelve Data bars at $0.30 with
the pre-relaxation gates and the pullback entries on; on Oanda bars at $0.65 with today's gates the
floor lifts small-ATR targets past the $9 spread gate and those trades earn (B2 added trades on W3:
554 / +90.8R, 53% winners, median stop $12 vs $20, 962 of 1,105 plans at exactly RR 1.0; positive
in every year; by volatility regime extreme +44.6R, low +32.0R, normal +14.2R). The cost is a deeper
drawdown in the same May–July 2026 episode (10.1R → 13.4–15.9R) and a non-monotone ladder (0.9 beats
1.0 on every window, 0.6 is worse than 0.5 out of sample), which is why nothing from it ships now.
If the owner wants M5 plans at RR ≥ 1 (the original request), the honest candidate is a fresh
pre-registered run at ONE value chosen before looking (0.75 or 1.0), with the DD clause waived
explicitly as on 2026-07-21, plus the cost ladder ($0.80, $1.00) because the added trades carry
$12 stops. Until then M5 plans keep 0.5R guides and the alert says so.
## Shipped (2026-09-26)
1. **A1 — alert cooldown** (`telegram_bot.py`): no new alert on a timeframe within 5 bars of the
last alerted setup's bar, `_ALERT_COOLDOWN_BARS = 5` = `backtest.cooldown_bars`. Book-neutral.
2. **A2 — alert wording** (`telegram_bot.py`, `main.py`, `docs/user_guide.html`): the trail is
printed first as the tested exit; TP1/TP2 are guides with their R; "Planned TP1 RR"; one
validated-book line per setup class from `agent_panel_base_rates.json`. Book-neutral.
3. **B3 — M15 grade-A SELL block** (`pair_signals/xauusd.py`, M15 override 6). Re-pins the M15
books (below).
4. **XAU_ATR target floor** (`signal_aggregator._generate_xau_atr_trade_setup`, mirrored in
`cuda_signal_kernels.py` K2): the gold ATR generator now honours `tp_floor_rr` like the GENERIC
one, compared as multipliers so the non-binding case is bit-identical. Binds only on M15 (floor
1R): grade-A plans print TP1 at 2 × ATR = 1R instead of 1.5 × ATR = 0.75R. TP1 is advisory in the
backtester, so the only book effect is through the $9 spread gate (parity MD5s below).
## Addendum (same day): printed-target floor extended to every generator
Owner rule: "make sure TP1 is at least 1:1". Shipped as `signals.tp_print_floor_rr = 1.0`
(effective floor `max(tp_floor_rr, tp_print_floor_rr)`), applied after the min-RR and spread gates in
the gold ATR, GENERIC and chart-pattern generators (Python and CUDA K2). Label-only by construction
and by measurement (twin books vs the previous pins, same code otherwise):
| Book | Rows | R before → after | Columns that differ | Rows with a new `tp1` | min `rr_planned` before → after |
|---|---|---|---|---|---|
| M5 W2 | 781 | +146.07 → +146.07 | `tp1`, `rr_planned` | 544 (GENERIC 408, XAU_CP 105, XAU_ATR 31) | 0.5 → 1.0 |
| M15 W2 | 813 | +154.15 → +154.15 | none | 0 | 1.0 → 1.0 |
| H1 W3 | 253 | +70.02 → +70.02 | `tp1`, `rr_planned` | 252 | 0.5 → 1.0 |
| H4 W3 | 114 | +21.68 → +21.68 | `tp1`, `rr_planned` | 108 | 0.5 → 1.0 |
New pins: M5 W2 `e599c8a8652f95b7fcdf5a7ef2b564f5`, H1 W3 `a44fd19b4067f46e5bd98d50cc3eff92`,
H4 W3 `f3587bae3c5dc3eda468ea3588b17dcb`; M15 W2 `5b65ecf6…` and OOS `09b8a8e7…` unchanged. Reference
script MD5s: see the parity lines appended below. The gated floor (`tp_floor_rr`, 0.5 on M5 / 1.0 on
M15) is untouched: raising it is the B2 question above, a book change, and remains the owner's call.
## Addendum 2 (same day): consistency checks on `signals.db` — two delivery bugs fixed
Read-only checks over the 8,453 rows (2026-09-04 → 09-25): scan latency (median 10–15 s after the
candle close on every timeframe; 2–3% of rows read the previous bar, all in 18–22 UTC around the
Labor Day gap and the settlement pause), `actionable` / setup consistency (clean), gate reasons for
actionable grades without a setup (all `SPREAD GATE`, small-ATR M5 bars), outcome states (97 resolved,
36 duplicates, 1 unresolvable for a candle gap), scan-outcome labelling (7,054 of 7,634 complete),
8 `failed` scans (Oanda "primary timeframe data unavailable", notably 06:11–06:17 UTC on both Sep 15
and Sep 16), dashboard-vs-bot disagreement on the same bar (4 bars of 3,300).
Two bugs, both fixed and tested, books untouched:
1. **Panel probability never stored under `verdict_source: "macro"`** (the live setting): all 33 v2
rows had `panel_probability` NULL although `panel_probs_json` carried the macro persona's `p_win`.
Cause: the macro-source `PanelVerdict` carried no `probability`, `_write_fields` wrote 0.0 and the
logger's `probability if probability else None` dropped it. Fix: the macro verdict's probability is
the macro take's `p_win` (`agent_panel.run_panel`). Without it the planned re-evaluation at 100 /
300 outcomes would have had no live probabilities to score.
2. **Breach rescans re-log the last pre-pause M15 bar.** Every evening the 20:45 UTC bar was logged
again at 22:01, 22:06 and 22:11 (Sep 17–24) by the tick-driven level-breach rescan: the first
post-pause bar closes at 22:15, so the frame still ends at 20:45. The candle loop already skips
that window (`_closed_bar_in_settlement`); `_handle_breach` now applies the same guard. Cosmetic
for alerts (dedup + cooldown), but it polluted the log's per-bar dedup and the parity reports.
Not bugs, recorded: 380 `blocked` weekend rows on Sep 20 carry no `bar_ts` by design; the 43× repeat of
the Sep 7 18:25 M5 bar is the known holiday gap (no holiday calendar); the H4 loop produced no setup
in three weeks (its two tradeable close hours).
@@ -0,0 +1,483 @@
# Live signal-log review — 2026-09-17
**Source evidence:** `logs/signals.db` copied from the live server on 2026-09-17 (rows
2026-09-04 00:59 → 2026-09-17 13:25 UTC; 5,487 rows, 5,481 `scan` + 6 `failed`, 0 `blocked`;
telegram 3,458 / dashboard 2,029). Opened read-only. Cross-checked with
`analyze_signal_log.py --parity | --scan-outcomes | --stored-candles` (all read-only), and with a
live-vs-backtest parity replay of the same window through `backtest/gpu_backtest.py` at HEAD
(`5521998`). No tuned value is proposed; this is a health check and a descriptive scorecard.
## Headline
1. **Live equals backtest.** Over 2026-09-06 → 2026-09-17 the validated code produced 29 trades;
26 of them appear in the live log on the same bar with the same direction, grade and entry,
and the live advisory R agrees with the backtest R to within 0.14 (mean difference −0.006 over
23 resolved matches). The 3 backtest-only trades are the Sep 9 bot outage (2) and one M15 bar
scanned live on Sep 8 with the pre-RSI-change M15 engine (1). All 20 live-only setups are
consecutive-bar re-entries that fall inside the backtester's 5-bar entry cooldown.
2. **The raw advisory book reads −7.15R over 37 resolved setups, but that number counts cluster
re-entries the validated book never takes.** The like-for-like views are flat: cooldown-equivalent
live book −1.62R over 25 resolved (PF 0.84); backtest over the same window +1.09R over 29 trades
(M15 −0.32R / 13, M5 +2.43R / 15, H1 −1.01R / 1, H4 0). Against the two-year books, a 12–15 trade
fortnight summing between −0.7R and +2.4R sits at the 25th–49th percentile: an ordinary fortnight.
The 12 resolved cooldown re-entries alone cost −5.54R.
3. **The market was a one-way bear fortnight with an FOMC hike in it:** 4,477 → 4,367 (−2.5%),
low 4,235 on Sep 16, daily ranges of 12–30 ATR, then +95$ on Sep 17. Every directional statistic
below is drift-adjusted against the same-timeframe mean move; the raw 24 h drift was −4.2 ATR on M5.
4. **Direction vs market.** At 1–4 h the composite carried no information this fortnight (Spearman
rank correlation of score with the 4 h move ≈ 0 on every timeframe). At 24 h it was *inversely*
related to the move on all three timeframes (M5 ρ = −0.11, M15 −0.20, H1 −0.33, bootstrap CIs
exclude zero, n = 1,734 / 570 / 150 unique scans): high scores marked bounce tops in the
downtrend and deep negative scores marked lows that bounced. This is the same pattern the
2026-09-08 review saw in 16 of 18 episodes, now with n. It is a two-week bear-market reading,
not a verdict on the two-year edge, and it is the one hypothesis worth a proper in-book test.
5. **AI panel.** 34 unique setups carried a panel (27 resolved). Skipping the macro persona's OPPOSE
would have avoided 7 resolved trades worth −2.52R (kept book −2.82R vs −5.35R), but the same
opposed trades moved *in the trade's favour* at 24 h (drift-adjusted hit 4/5) and 6 further
OPPOSE verdicts are on the Sep 17 long cluster the backtester currently marks +1.13R. Technical
(79% support), risk (79% oppose) and contrarian (65% oppose) are near-constant voters. Verdict
confidence is uncalibrated (62% / 50% / 60% correct across the three bands). No evidence either
way at this n; keep logging.
6. **Delivery findings** (no book impact): the dashboard daemon has run stale code (`2ac7301`) since
Sep 10 13:28 while the bot moved to `2836024` on Sep 14, and it was down from Sep 7 02:20 to
Sep 10 13:28; the bot was down Sep 9 00:05 → 19:36 UTC (two validated trades missed, +0.37R);
the live pipeline has no entry cooldown, so consecutive-bar setups are logged, scored and (about
one in three) re-alerted; the backtester's CSV labels the gold ATR setups `GENERIC` while live
stamps them `XAU_ATR` (geometry and R identical).
## Method
- Rows deduped to the first row per (timeframe, bar label); `bar_ts` NULL on 365 rows (354 from the
pre-Sep-7 logging code) falls back to `indicators_json[tf].timestamp`; 11 rows with no indicators
(price 0) dropped; `scan_kind in ('blocked','failed')` skipped. Unique bars: M5 2,289, M15 763,
H1 191, H4 53.
- Setups: 49 unique (M5 24, M15 24, H1 1, H4 0). Outcome = the DB's own advisory model
(`advisory_r_multiple`, net R at $0.65, next-open entry, stop first, close-only trail): 37 RESOLVED
(−7.15R, reproduced exactly), 11 PENDING (the Sep 17 long cluster), 1 UNRESOLVABLE.
- Forward moves: `scan_outcomes` (`fwd_{1h,4h,24h}_atr`, + = up; 24 h only where `complete = 1`).
"Move" = move in the signal's direction; "excess" = the same after subtracting the same-timeframe
mean move over all scans (the drift); "adj hit" = share of excess > 0.
- Three book views: (1) every unique setup; (2) cooldown-equivalent = after an accepted setup, skip
the next 5 primary bars (the backtester's rule); (3) alerted = the bot's dedup replayed on the
telegram rows (fingerprint (signal, direction) per timeframe; re-alert after 3,600 s, on a grade
upgrade, or when the entry moved ≥ 0.5 × last risk; grade ≥ C; setup required; state reset at
each restart = code-version change). View 3 is a reconstruction; `logs/telegram.log` on the live
box is the ground truth.
- Bootstrap null: the M15 and M5 two-year books regenerated offline from the kept cache with the
section-9 harness (`--start 2024-09-05 --end 2026-09-05 --spread 0.65 --workers 12`), MD5s
`d185f099482a07f29502cb5a371a9bbf` and `29120a1ee32979431d062f210d41a9d1` (both equal to the
pinned values; `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0`); 20,000 fixed-n resamples
of per-trade R plus every 13-calendar-day window of the book.
- Parity replay: `venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --spread 0.65 --workers 12
--primary-tf {15min,5min,1h,4h} --start 2026-09-06 --export-csv ... --detail` (no `--end`, so the
end is the run time; no `--set`). New cache windows, cited here and therefore kept:
`oanda_XAU_USD_{5min,15min,1h}_20260706_20260917.parquet`,
`oanda_XAU_USD_4h_20260115_20260917.parquet`, `oanda_XAU_USD_1d_20221106_20260917.parquet`.
`Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all four runs. The fetch logged two
interior gaps: M5/M15 2026-09-07 18:15–22:00 (Labor Day, Oanda published no bars — matches the 46
stale-bar rows live logged) and H4 2026-04-02 → 04-05 (Good Friday). Matching key: (timeframe,
backtest entry bar − 1 bar) == live bar label.
- Nothing was written to `signals.db`; `scripts/backfill_signal_outcomes.py` and
`scripts/label_scan_outcomes.py` were run with `--dry-run` only.
## 1. Integrity and coverage
| Check | Result |
|---|---|
| Surface parity (1,953 shared bars) | 0 signal, grade or setup-presence divergences; 8 score differences, none changing a signal or grade: 6 on H1/H4 bars scored by the two code versions (≤ 0.2 pts on H1, up to 7 pts on H4 at Sep 15 00:00: consistent with different confirmation-frame seeding), 2 on the dashboard's Sep 10 13:28 restart catch-up |
| Code versions live | telegram: c44813f → de36b10 (Sep 7 15:08) → 3afc62c → 28b591d (Sep 8) → 2ac7301 (Sep 9 21:45) → **2836024 (Sep 14 01:46)**. dashboard: c44813f (26 rows, Sep 7 01:00–02:20), then **2ac7301 from Sep 10 13:28 to now** |
| Dashboard daemon | down Sep 7 02:20 → Sep 10 13:28 (3.5 days); since then on stale code. Still logs 11 "Primary timeframe stale" M5 rows every day at 21:xx UTC (the settlement pause; a scheduled closure in 2836024) |
| Telegram bot | down Sep 9 00:05 → 19:36 UTC. Cost: M15 BUY B Sep 9 05:45 (+0.23R) and M5 BUY A Sep 9 06:00 (+0.14R) in the backtest, never scanned live |
| Labor Day (Sep 7) | Oanda published no bars 18:25–22:00 UTC (confirmed by the fresh fetch); 46 stale-bar rows logged; no holiday awareness (known) |
| Failed scans | 6 rows at 06:11–06:17 UTC on Sep 15 and Sep 16, both surfaces, primary frame missing: an Oanda blip longer than the retry ladder's 45 s budget. 11 pre-ladder "data unavailable" rows on Sep 4/7 |
| Scan-outcome labelling | 2,467 unique bars complete, 418 labelled but < 72 h old, 408 unlabelled and < 25 h old, 3 unlabelled and older. Last label 13:20 today: the labeler is healthy |
| Blocked scans | none logged (`log_blocked_scans` off); the Monday 00:xx UTC hours are therefore empty by design |
| Engine fingerprints | M15 `dab4d03725d8` → `536f84340ea0` at Sep 9 21:45 (RSI × 0.5). M5, H1, H4 unchanged all window |
| Panel | 35 rows, ≤ 9 per day against the cap of 20, all cache MISS, all `verdict_source = macro` |
## 2. Market context
| UTC day | open | high | low | close | change $ | range / ATR |
|---|---|---|---|---|---|---|
| Sep 4 | 4,480 | 4,491 | 4,366 | 4,431 | −49 | 30 |
| Sep 7 (Labor Day) | 4,424 | 4,430 | 4,381 | 4,423 | 0 | 13 |
| Sep 8 | 4,423 | 4,443 | 4,345 | 4,350 | −73 | 20 |
| Sep 9 (bot down) | 4,350 | 4,418 | 4,347 | 4,394 | +44 | 24 |
| Sep 10 | 4,394 | 4,435 | 4,311 | 4,315 | −79 | 25 |
| Sep 11 | 4,315 | 4,403 | 4,292 | 4,349 | +33 | 20 |
| Sep 14 | 4,333 | 4,355 | 4,254 | 4,288 | −45 | 21 |
| Sep 15 | 4,288 | 4,318 | 4,261 | 4,284 | −5 | 14 |
| Sep 16 (FOMC hike 18:00) | 4,284 | 4,367 | 4,235 | 4,272 | −12 | 30 |
| Sep 17 (to 13:20) | 4,272 | 4,381 | 4,267 | 4,367 | +95 | 23 |
Drift baseline (mean forward move over **all** unique scans, ATR units, + = up):
| tf | 1 h | 4 h | 24 h |
|---|---|---|---|
| M5 | −0.23 (n 1,992) | −0.63 (1,998) | −4.21 (1,734) |
| M15 | −0.09 (661) | −0.36 (667) | −2.31 (570) |
| H1 | −0.06 (161) | −0.22 (167) | −0.75 (150) |
## 3. Direction and score against the realised move (all unique scans)
Move is measured in the signal's direction; excess is after removing the drift above.
| tf | direction | n | 1 h move / excess | 4 h move / excess / hit | 24 h move / excess / adj hit |
|---|---|---|---|---|---|
| M5 | LONG | 18 | −1.10 / −0.87 | −1.27 / −0.64 / 39% | −6.83 / −2.62 / 47% |
| M5 | SHORT | 66 | +0.35 / +0.12 | +0.14 / −0.49 / 53% | −1.13 / −5.34 / 31% |
| M15 | LONG | 7 | −0.81 / −0.72 | −0.31 / +0.05 / 43% | −4.03 / −1.72 / 33% |
| M15 | SHORT | 39 | +0.27 / +0.18 | +0.60 / +0.23 / 69% | −0.68 / −2.98 / 30% |
| H1 | SHORT | 4 | −0.25 / −0.31 | −0.66 / −0.88 / 50% | −2.05 / −2.80 / 0% |
Reading: M15 shorts were right for about four hours (69% hit, +0.23 ATR excess) and wrong by the
next day (the bear-trend lows they sold bounced). Longs were wrong at every horizon. Nothing here
beats drift at 24 h.
Score buckets (direction = sign of score, includes NEUTRAL scans):
| tf | bucket | n | 4 h move / excess | 24 h move / excess / adj hit |
|---|---|---|---|---|
| M5 | ≥ 35 | 22 | −1.83 / −1.19 | −13.06 / −8.85 / 18% |
| M5 | 25..35 | 47 | −1.11 / −0.48 | −9.61 / −5.40 / 28% |
| M5 | −35..−25 | 101 | −0.95 / −1.58 | +0.11 / −4.10 / 31% |
| M5 | ≤ −35 | 41 | −2.06 / −2.69 | −0.15 / −4.36 / 25% |
| M15 | 15..25 | 56 | −0.68 / −0.31 | −5.15 / −2.85 / 35% |
| M15 | 25..35 | 17 | −0.13 / +0.23 | −3.20 / −0.90 / 44% |
| M15 | −35..−25 | 37 | +0.65 / +0.28 | +0.42 / −1.88 / 38% |
| M15 | ≤ −35 | 24 | +0.47 / +0.10 | −0.98 / −3.28 / 29% |
Spearman rank correlation of the composite score with the forward move (+ would mean high score →
price up), with bootstrap 95% CI:
| tf | 1 h | 4 h | 24 h |
|---|---|---|---|
| M5 (n ≈ 1,990 / 1,734) | −0.05 [−0.09, −0.01] | −0.03 [−0.07, +0.02] | **−0.11 [−0.15, −0.06]** |
| M15 (n ≈ 665 / 570) | 0.00 [−0.08, +0.08] | −0.02 [−0.09, +0.06] | **−0.20 [−0.27, −0.12]** |
| H1 (n ≈ 165 / 150) | −0.09 [−0.25, +0.06] | −0.08 [−0.24, +0.07] | **−0.33 [−0.47, −0.17]** |
Grades: M15 grade-A rows (n = 12) had a 24 h excess of −4.8 ATR and an adjusted hit of 17%; M5
grade-B rows (n = 15) −1.9 ATR excess at 4 h, hit 20%. Gates: the only gate that removed A/B/C
directional rows this fortnight was the spread gate on M5 (n = 10; 24 h excess +0.03 vs −5.33 for
the shipped M5 setups): tiny and descriptive. The M15 engine change at Sep 9 21:45 leaves only 10
directional rows before it; nothing can be said about it.
## 4. The trade book
View 1, every unique setup (advisory R at $0.65; PENDING excluded from R):
| cut | n | resolved | wins | sum R | mean R | PF |
|---|---|---|---|---|---|---|
| **all** | 49 | 37 | 17 (46%) | **−7.15** | −0.19 | 0.60 |
| M5 | 24 | 14 | 6 | −1.08 | −0.08 | 0.83 |
| M15 | 24 | 22 | 11 | −5.06 | −0.23 | 0.51 |
| H1 | 1 | 1 | 0 | −1.01 | | |
| BUY / SELL | 30 / 19 | 18 / 19 | 8 / 9 | −3.49 / −3.67 | −0.19 / −0.19 | 0.59 / 0.61 |
| grade A / B / C | 27 / 19 / 2 | 20 / 15 / 2 | 9 / 7 / 1 | −4.87 / −2.75 / +0.47 | | 0.48 / 0.63 / 1.46 |
| XAU_ATR / GENERIC / XAU_CP | 10 / 35 / 4 | 9 / 25 / 3 | 5 / 10 / 2 | −1.29 / −7.03 / +1.17 | | 0.69 / 0.45 / 2.10 |
| regime transitioning / trending_down / trending_up | 12 / 15 / 16 | 11 / 15 / 6 | 4 / 8 / 2 | −6.02 / −0.70 / −2.06 | | 0.16 / 0.89 / 0.08 |
| volatility extreme / normal | 31 / 9 | 22 / 8 | 10 / 4 | −4.32 / +0.94 | | 0.59 / 1.30 |
| watch level none / watch / hot | 10 / 24 / 12 | 10 / 20 / 5 | 6 / 7 / 4 | +0.57 / −7.73 / +2.11 | | 1.18 / 0.38 / 10.5 |
| hour 08 UTC | 7 | 7 | 0 | −7.14 | −1.02 | 0.00 |
| hour 12 UTC | 13 | 6 | 5 | +4.15 | +0.69 | 19.7 |
| day Sep 14 / Sep 15 / Sep 16 | 4 / 6 / 11 | 4 / 6 / 11 | 4 / 1 / 4 | +3.40 / −5.01 / −4.53 | | |
Exits: 17 SL, 18 TRAIL, 2 GAP_LOCKED. Setup-level forward moves agree with the advisory model:
M15 setups −0.60 ATR excess at 4 h (hit 35%), −3.33 at 24 h; M5 −1.49 at 4 h, −5.33 at 24 h.
View 2, cooldown-equivalent (skip 5 primary bars after an accepted setup): 28 setups, 25 resolved,
14 wins (56%), **−1.62R**, PF 0.84; M5 +0.11R (12), M15 −0.71R (12), H1 −1.01R; SELL +1.11R over 10
(70% win), BUY −2.73R over 15.
View 3, reconstructed Telegram alerts: 36 alerts in 9 trading days (about 4 a day), 29 resolved, 14
wins (48%), **−4.88R**, PF 0.64; M15 −3.97R (16), M5 +0.11R (12), H1 −1.01R. The re-alert rules let
through 2 of the 6 Sep 15 shorts and all 3 FOMC-evening shorts.
Backtest over the same window (HEAD): **29 trades, +1.09R**. Matched-resolved 23: live +0.47R vs
backtest +0.62R (the difference is the Sep 8 07:30 M15 short, GAP_LOCKED live at +1.13 vs +1.28
on the raw path). Matched but live still pending 3: backtest +1.13R. Backtest-only 3: −0.66R.
Live-only cooldown re-entries 20 (8 pending): the 12 resolved ones sum to **−5.54R**.
Bootstrap against the two-year books (per-trade R resampled; P = share of draws at or below the
observed sum):
| view | tf | n | observed | book median | 5–95% | P(≤ obs) |
|---|---|---|---|---|---|---|
| view 1 (raw advisory) | M15 | 22 | −5.06R | +3.2 | −4.8..+12.0 | 4.6% |
| view 1 | M5 | 14 | −1.08R | +2.4 | −4.2..+10.3 | 20% |
| view 2 (cooldown) | M15 | 12 | −0.71R | +1.8 | −4.1..+8.3 | 25% |
| view 2 | M5 | 12 | +0.11R | +2.0 | −4.0..+9.3 | 31% |
| backtest over window | M15 | 13 | −0.32R | +1.9 | −4.2..+8.7 | 28% |
| backtest over window | M5 | 15 | +2.43R | +2.6 | −4.3..+10.7 | 49% |
Calendar check: 36% of the M15 book's 13-day windows and 35% of the M5 book's are negative; among
M5 windows where price fell 3% or more the median is +0.2R with 46% negative, median BUY −1.2R and
median SELL +1.5R. This fortnight behaved the way the book says a falling fortnight behaves. The
view-1 M15 figure looks extreme only because its 22 "trades" are 12 independent entries plus 10
correlated re-entries.
Special buckets:
- **M15 grade-A/A+ SELL** (the parked cycle-2 candidate): live 11 setups, 5 wins, −2.77R (PF 0.55),
but 6 of the 11 are Sep 15 cluster re-entries; the cooldown-equivalent five are +1.13, +0.36, +1.05,
+0.09, −1.02 = +1.61R. Two-year book: 176 trades, −3.4R, mean −0.02R. The live sample does not
move the candidate either way; it stays parked.
- **FOMC evening** (Sep 16 18:45, 19:00, 19:15 UTC): three M15 shorts, all −1R, 45–75 minutes after
the 18:00 decision, all `watch_level = watch` (reading `extreme`), none `hot` because the hot rule
needs a high-impact USD release within ±30 min. The macro persona AGREED on all three at 82%
confidence ("the Fed raised rates, which pushes gold down"); gold rose 95$ the next day. The
backtester took the 18:45 trade too (−1.02R), so it is validated behaviour; 19:00 and 19:15 are
cooldown re-entries.
- **Sep 15 07:45–08:45 M15 STRONG_SELL cluster**: six setups on consecutive bars, one +0.09R and
five −1.02R (−5.01R). The backtester took the first (+0.09R) and skipped the rest.
- **XAU_CP**: 4 setups (2 inverse head-and-shoulders, 2 double bottoms), 3 resolved, +1.17R; both
in-window ones matched the backtester exactly.
- **Exhaustion count at entry**: 0 → +2.57R (4 resolved), 1 → −4.85R (10), 3 → +0.26R (8),
4 → −3.24R (11). Descriptive only; the 2026-09-06 campaign already closed exhaustion sizing.
## 5. The AI panel
Unique setups with a panel 34 (27 resolved); macro persona is the verdict.
| verdict | n | resolved | wins | sum R | PF | 4 h excess / adj hit | 24 h excess / adj hit |
|---|---|---|---|---|---|---|---|
| AGREE | 12 | 12 | 6 (50%) | −2.16 | 0.59 | −2.51 / 22% (n 9) | −7.13 / 22% (n 9) |
| NEUTRAL | 9 | 8 | 5 (62%) | −0.67 | 0.79 | +1.48 / 75% (n 8) | −2.31 / 29% (n 7) |
| OPPOSE | 13 | 7 | 2 (29%) | −2.52 | 0.39 | −0.72 / 20% (n 5) | +0.69 / 80% (n 5) |
| skip OPPOSE (counterfactual) | 21 | 20 | 11 (55%) | −2.82 | 0.66 | | |
| AGREE only | 12 | 12 | 6 (50%) | −2.16 | 0.59 | | |
By realised R the OPPOSE verdict was mildly useful (it flagged the two worst-PF groups: opposed
BUYs −1.61R over 4, opposed SELLs −0.91R over 3). By forward move the same opposed trades went the
trade's way at 24 h (4 of 5 after drift). Six pending OPPOSE verdicts sit on the Sep 17 long
cluster, which the backtester currently marks +0.90, +0.38, −0.15R. Nothing here is decidable at
n = 7; it is consistent with the 2026-09-03 finding that only the macro persona carries information.
Personas (stance share / mean R when they held it): macro support 35% (−0.18R) / neutral 26%
(−0.08R) / oppose 38% (−0.36R); technical support **79%** / oppose 12%; risk oppose **79%** /
support 3%; contrarian oppose **65%**. Three of four personas vote almost the same way on every
setup, so they add little beyond their prior. "Take only setups the risk persona did not oppose"
leaves 7 setups (+0.03R): a curiosity, not a rule. Number of opposing personas (1 / 2 / 3) gives
PF 0.57 / 0.57 / 0.57. Verdict confidence bands < 40 / 40–59 / ≥ 60 were 62% / 50% / 60% correct
(n 8 / 4 / 15): no calibration.
## 6. Findings, tagged
**Observations (small sample, descriptive):**
- The system is doing what was validated: bar-for-bar parity with the backtester, and the
backtester itself made +1.1R on this window, a 30th-percentile fortnight for the book.
- The live loss is concentrated in cluster re-entries (−5.5R), the Sep 15 08:xx short run and the
FOMC-evening shorts, and in the `transitioning` regime (PF 0.16 over 11) and `watch` level
(−7.7R over 20). Two weeks; no action.
- The composite score was inversely related to the 24 h move on all three timeframes with CIs
excluding zero, and uninformative at 1–4 h. Two weeks of a single bear regime cannot overturn the
two-year book, but it is the strongest live pattern seen twice now (2026-09-08 and here).
- Panel: OPPOSE mildly helpful by R, anti-informative by 24 h move; three personas near-constant.
**Delivery-side fix candidates (no book impact):**
1. Redeploy the dashboard daemon to `2836024` (or later) and install the systemd unit; it has run
`2ac7301` since Sep 10 and was dead for 3.5 days before that. After redeploy, re-check the 8 score
divergences between surfaces on H1/H4 bars.
2. Decide whether the live pipeline should mirror the backtester's 5-bar entry cooldown for
alerts and for the advisory scorecard. Today every consecutive-bar setup is logged as its own
trade and about one in three is re-alerted, so the live book systematically over-counts
correlated entries relative to every validated number. A cooldown view could be added to
`analyze_signal_log.py` (the existing `one_position_view` is close), and an alert cooldown is
the owner's call.
3. Backtester CSV `strategy` column: the gold ATR setups appear as `GENERIC` while live stamps
`XAU_ATR`; entries, stops and R are identical, so this is a labelling gap only. Cause: the
backtester derives its tag from the text of the first bullish/bearish reason (`"XAU STRATEGY"`
in the first bullish reason → `XAU`) instead of reading `trade_setup.strategy`, so SELL ATR
setups and any ATR setup whose first reason is another indicator fall through to `GENERIC`.
The same tag feeds the research-only `--strategies` filter. Book-neutral; a one-line fix plus
a test when convenient.
4. Bot uptime: the 19.5 h Sep 9 outage cost two validated trades; the daily 21:xx stale rows and
the Labor Day stale run are known and harmless.
**Needs a rule-1 backtest (nothing proposed to ship):**
- Score-vs-24 h inversion: test in-book with the replay recorder (`--record --record-bars` writes
forward 1 h / 4 h / 24 h moves for every analysed bar over the two-year window). If the rank
correlation is also negative across the book, the gate chain's thresholds, not the weights, are
the place to look; if it is positive, this fortnight was regime-specific.
- M15 grade-A SELL block stays parked (cycle-2 candidate; live adds n = 5 cooldown-equivalent
trades at +1.6R and n = 11 raw at −2.8R).
## Caveats
Section 7 adds four scratchpad research runs of gate 6h on the two-year cache (no new cache
windows; `Backend: NumPy CPU fallback`, `GPU cache fallbacks: 0` on all four).
Nine trading days, one regime (bear trend with an FOMC hike), 37 resolved advisory outcomes of
which 12 are correlated re-entries, the advisory model is cleaned-mid evidence and not execution,
the 24 h horizon is drift-dominated (drift-adjusted numbers are shown but a single trend cannot be
fully removed), the M15 engine changed mid-sample (Sep 9 21:45), the dashboard and the bot ran
different code from Sep 14, and the alerted view is a reconstruction. The `--stored-candles`
replay closes only 26 of the 37 DB-resolved setups because the stored candle streams have the
Labor Day and Sep 9 gaps; where both resolve they agree to the cent.
## Appendix: the 49 unique setups
R at $0.65 (advisory model); ex24 = 24 h move in the trade direction minus the same-timeframe drift,
ATR units; BT = backtester over the same window (M = matched, C = live-only inside the cooldown,
– = before the parity window).
| bar (UTC) | tf | signal | g | score | dir | strategy | regime | R | exit | ex24 | panel | BT |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 09-04 05:35 | M5 | BUY | B | +8.7 | BUY | XAU_CP | ranging | −1.06 | SL | −7.4 | NEUTRAL | – |
| 09-04 07:45 | M15 | STRONG_BUY | A | +37.3 | BUY | XAU_ATR | transitioning | −1.04 | SL | −5.1 | NEUTRAL | – |
| 09-04 15:10 | M5 | BUY | B | +8.8 | BUY | XAU_CP | ranging | unres. | | +3.1 | OPPOSE | – |
| 09-07 08:00 | M15 | BUY | C | +26.7 | BUY | GENERIC | ranging | −1.02 | SL | | | M (bt grade B, pre-RSI engine) |
| 09-07 12:15 | M5 | BUY | B | +22.2 | BUY | XAU_CP | trending | +2.04 | TRAIL | | NEUTRAL | M |
| 09-08 07:30 | M15 | SELL | A | −34.2 | SELL | GENERIC | trending_down | +1.13 | GAP_LOCKED | +3.0 | OPPOSE | M (bt +1.28) |
| 09-08 07:45 | M15 | SELL | B | −31.7 | SELL | GENERIC | trending_down | +0.84 | GAP_LOCKED | +2.7 | | C |
| 09-08 08:05 | M5 | SELL | B | −35.4 | SELL | GENERIC | trending_down | −1.03 | SL | +3.7 | OPPOSE | M |
| 09-10 05:10 | M5 | BUY | B | +20.5 | BUY | XAU_CP | ranging | +0.18 | TRAIL | −18.3 | AGREE | M |
| 09-10 05:40 | M5 | BUY | A | +33.4 | BUY | GENERIC | trending_up | −1.03 | SL | −15.7 | AGREE | M |
| 09-10 05:55 | M5 | BUY | B | +29.8 | BUY | GENERIC | trending_up | −1.02 | SL | −15.2 | | C |
| 09-10 09:15 | M15 | SELL | B | −29.1 | SELL | GENERIC | transitioning | +0.07 | TRAIL | +4.2 | NEUTRAL | M |
| 09-10 12:05 | M5 | SELL | B | −30.3 | SELL | GENERIC | trending_down | +1.31 | TRAIL | +3.6 | AGREE | M |
| 09-10 12:15 | M15 | SELL | A | −34.3 | SELL | GENERIC | trending_down | +0.36 | TRAIL | +0.3 | AGREE | M |
| 09-10 12:30 | M15 | SELL | B | −32.6 | SELL | GENERIC | trending_down | −0.22 | TRAIL | −3.4 | AGREE | C |
| 09-14 09:15 | M15 | STRONG_SELL | A | −42.7 | SELL | XAU_ATR | trending_down | +1.05 | TRAIL | −0.1 | AGREE | M |
| 09-14 09:30 | M15 | STRONG_SELL | A | −42.6 | SELL | XAU_ATR | trending_down | +0.70 | TRAIL | −1.0 | | C |
| 09-14 12:45 | M5 | SELL | B | −36.5 | SELL | GENERIC | trending_down | +0.16 | TRAIL | −6.1 | AGREE | M |
| 09-14 15:25 | M5 | BUY | C | +25.1 | BUY | GENERIC | ranging | +1.49 | TRAIL | +4.3 | | M |
| 09-15 07:45 | M15 | STRONG_SELL | A | −37.5 | SELL | GENERIC | trending_down | +0.09 | TRAIL | −8.7 | AGREE | M |
| 09-15 08:00 | M15 | STRONG_SELL | A | −42.0 | SELL | GENERIC | trending_down | −1.02 | SL | −9.5 | OPPOSE | C |
| 09-15 08:10 | M5 | SELL | B | −30.3 | SELL | GENERIC | trending_down | −1.02 | SL | −15.6 | AGREE | M |
| 09-15 08:15 | M15 | STRONG_SELL | A | −36.7 | SELL | GENERIC | trending_down | −1.02 | SL | −9.6 | | C |
| 09-15 08:30 | M15 | STRONG_SELL | A | −41.6 | SELL | GENERIC | trending_down | −1.02 | SL | −9.7 | | C |
| 09-15 08:45 | M15 | STRONG_SELL | A | −45.1 | SELL | GENERIC | trending_down | −1.02 | SL | −10.5 | | C |
| 09-16 03:00 | M5 | STRONG_BUY | A | +36.0 | BUY | GENERIC | trending_up | +0.14 | TRAIL | −4.2 | NEUTRAL | M |
| 09-16 03:00 | M15 | BUY | A | +34.0 | BUY | XAU_ATR | transitioning | +0.14 | TRAIL | −3.5 | NEUTRAL | M |
| 09-16 03:10 | M5 | STRONG_BUY | A | +38.6 | BUY | GENERIC | trending_up | −0.16 | TRAIL | −4.6 | | C |
| 09-16 03:15 | M15 | BUY | A | +30.9 | BUY | XAU_ATR | transitioning | +0.43 | TRAIL | −2.8 | | C |
| 09-16 06:00 | H1 | BUY | B | +28.0 | BUY | GENERIC | transitioning | −1.01 | SL | −0.7 | NEUTRAL | M |
| 09-16 07:00 | M15 | BUY | B | +29.1 | BUY | GENERIC | trending_up | +0.05 | TRAIL | +0.5 | NEUTRAL | M |
| 09-16 07:35 | M5 | STRONG_BUY | A | +37.3 | BUY | GENERIC | trending_up | −0.04 | TRAIL | +3.2 | OPPOSE | M |
| 09-16 15:05 | M5 | BUY | B | +29.7 | BUY | GENERIC | transitioning | −1.04 | SL | | OPPOSE | M |
| 09-16 18:45 | M15 | SELL | A | −34.6 | SELL | XAU_ATR | transitioning | −1.02 | SL | | AGREE | M |
| 09-16 19:00 | M15 | STRONG_SELL | A | −35.4 | SELL | XAU_ATR | transitioning | −1.02 | SL | | AGREE | C |
| 09-16 19:15 | M15 | SELL | B | −26.6 | SELL | GENERIC | transitioning | −1.01 | SL | | AGREE | C |
| 09-17 07:15 | M15 | BUY | A | +31.5 | BUY | XAU_ATR | transitioning | −1.03 | SL | | OPPOSE | M |
| 09-17 07:25 | M5 | STRONG_BUY | A | +35.9 | BUY | GENERIC | trending_up | pending | | | OPPOSE | M (bt +0.90) |
| 09-17 07:30 | M5 | STRONG_BUY | A | +36.2 | BUY | GENERIC | trending_up | pending | | | | C |
| 09-17 07:30 | M15 | BUY | B | +28.7 | BUY | GENERIC | transitioning | pending | | | | C |
| 09-17 07:35 | M5 | STRONG_BUY | A | +35.5 | BUY | GENERIC | trending_up | pending | | | | C |
| 09-17 12:00 | M5 | BUY | B | +34.3 | BUY | GENERIC | trending_up | pending | | | NEUTRAL | M (bt +0.38) |
| 09-17 12:00 | M15 | STRONG_BUY | A | +35.9 | BUY | XAU_ATR | transitioning | +0.50 | TRAIL | | OPPOSE | M |
| 09-17 12:05 | M5 | STRONG_BUY | A | +37.2 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
| 09-17 12:10 | M5 | BUY | B | +34.4 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
| 09-17 12:15 | M5 | STRONG_BUY | A | +38.9 | BUY | GENERIC | trending_up | pending | | | OPPOSE | C |
| 09-17 12:15 | M15 | STRONG_BUY | A+ | +43.7 | BUY | XAU_ATR | trending_up | pending | | | OPPOSE | C |
| 09-17 12:20 | M5 | STRONG_BUY | A | +39.1 | BUY | GENERIC | trending_up | pending | | | | C |
| 09-17 12:25 | M5 | STRONG_BUY | A | +40.0 | BUY | GENERIC | trending_up | pending | | | | M (bt −0.15) |
Backtest-only trades: M15 BUY B 09-08 04:00 (−1.03R; live scored the bar +24.7 NEUTRAL D on the
pre-RSI-change engine), M15 BUY B 09-09 05:45 (+0.23R) and M5 BUY A 09-09 06:00 (+0.14R), both
during the bot outage.
## 7. The owner's four suspicions, checked
The owner's objective for this review (2026-09-17): is anything functioning poorly in one of four
specific ways. Each is checked on the 49 unique live setups (37 resolved) and compared with the
two-year books where the books carry the same measure. "Reached TP1" means the trade's best
excursion (MFE) reached the planned first target; the advisory model itself exits on the trail, so
TP1 is advisory.
### 7.1 Does the AI panel OPPOSE trades that go on to hit their targets? — yes, as often as not
| macro verdict | setups | reached TP1 | reached TP2 | resolved wins | mean MFE |
|---|---|---|---|---|---|
| AGREE | 12 | 5 (42%) | 4 | 6 / 12 | 0.87R |
| NEUTRAL | 9 | 5 (56%) | 3 | 5 / 8 | 0.96R |
| OPPOSE | 13 | **7 (54%)** | 5 | 2 / 7 | 0.64R |
Opposed trades reached their first target more often than agreed ones. Concrete cases: OPPOSE at
60% on the Sep 8 07:30 M15 short ("most news points to gold rising") that made +1.13R and hit TP2;
OPPOSE at 62–65% on five of the Sep 17 longs that ran +0.7R to +1.3R (four still open); AGREE at 82%
on all three FOMC-evening shorts ("the Fed raised rates, which pushes gold down"), all −1R, gold
+95$ the next day. The macro persona's notes quote the news narrative of the moment, and the
2026-09-04 study showed that narrative trails price by about a day. The realised-R view (opposed
trades 2/7 wins) looks better for the panel only because the opposed winners are mostly the
pending Sep 17 longs. Structure: technical supports 79% of setups, risk opposes 79%, contrarian
65% — three near-constant voters. **Verdict: OPPOSE has no value as a veto and must never gate;
it is presented as "news context" at best.** (observation; the panel is already advisory-only)
### 7.2 Wrong direction, or entering late after the move is exhausted? — late by design; this fortnight the continuation failed
- 11 of 37 resolved setups never reached +0.3R (wrong from the first bar); 17 of 37 were stopped.
Book norm: 20–22% never reach 0.3R; live 30%.
- **30 of 37 entries came after price had already moved ≥ 2 ATR in the trade direction over the
previous 4 h**: PF 0.47, −7.8R. The 7 entries after a smaller prior move: PF 1.21, +0.7R.
- The engine's own exhaustion tags do **not** separate the losers: trades carrying
`rsi_extended` / `bb_overextended` at entry (21) had PF 0.68 and −2.8R; the 16 without had PF
0.53 and −4.4R. The 2026-08-04 stretch verdict (stretched entries are the best book trades) and
the 2026-09-06 sizing campaign already closed exhaustion-based sizing; the live fortnight adds
nothing that would reopen it.
- Three recurring shapes: (a) M15 shorts fire after a 4–6 ATR down leg, are right for four hours
(69% hit) and wrong by the next day (the low bounces); (b) longs in the downtrend fired at bounce
tops — Sep 10 05:40 and 05:55 at the 24 h high, followed by a 20 ATR fall; (c) the FOMC shorts
entered 45–75 min after the decision, 0.2–0.9 ATR above the day's low, after a 4–5 ATR drop: they
sold the bottom. Every one of these is the momentum design working as validated; the two-year
book earns its edge on exactly these entries and this fortnight they failed.
(observation)
### 7.3 Do signals fire on big volatile candles? — yes, and that is where the whole loss sits
| entry context (last 3 primary bars) | resolved | wins | reached TP1 | sum R | PF |
|---|---|---|---|---|---|
| a bar ≥ 2 ATR among the last 3 | 15 | 4 (27%) | 5 | **−7.18** | 0.25 |
| the signal bar itself ≥ 2 ATR | 7 | 1 (14%) | 2 | −3.44 | 0.23 |
| signal-bar body ≥ 1 ATR in the trade direction | 12 | 5 (42%) | 7 | −3.63 | 0.34 |
| no bar ≥ 2 ATR among the last 3 | 22 | 13 (59%) | 13 | **+0.02** | 1.00 |
The 15 setups fired within three bars of a ≥ 2 ATR candle carry the entire fortnight's loss; the
other 22 broke even at the book's win rate. Largest cases: Sep 10 12:30 M15 short on a 4.2 ATR bar
(−0.22R), the FOMC 18:45 / 19:00 shorts on 2.5–2.7 ATR bars (−1R each), Sep 14 09:15 short on a
2.7 ATR bar (+1.05R). A gate for exactly this exists (`big_candle_atr_multiple`, gate 6h) and ships
off because the 2026-07-30 test found the removed trades net-positive at every multiple on the long
books. The live fortnight contradicts that book verdict, so the gate was re-run on the two-year
M15 and M5 books (below).
Re-test of gate 6h on the two-year books (`--big-candle 2.0 --big-candle-hours {1,24}`, same
harness as the reference books, control = the MD5-verified W2 books; "removed" = control trades
absent from the candidate, approximate because the cooldown displaces entries):
| book | trades | R | PF | win | max DD | H1 / H2 | ΔR vs control | removed trades |
|---|---|---|---|---|---|---|---|---|
| M15 control | 931 | +146.1 | 1.43 | 60.5% | 16.1 | +67.7 / +78.4 | | |
| M15 gate 2 ATR, 1 h lookback | 643 | +117.6 | 1.51 | 61.9% | 14.4 | +58.4 / +59.2 | **−28.5** | 347 trades, **+32.5R**, 58% win |
| M15 gate 2 ATR, 24 h lookback | 3 | +0.5 | | | | | −145.6 | 929 trades, +147.0R |
| M5 control | 781 | +146.1 | 1.52 | 52.9% | 10.1 | +41.0 / +105.1 | | |
| M5 gate 2 ATR, 1 h lookback | 434 | +86.5 | 1.56 | 53.5% | 10.4 | +31.2 / +55.3 | **−59.6** | 368 trades, **+60.2R**, 52% win |
| M5 gate 2 ATR, 24 h lookback | 0 | | | | | | −146.1 | all |
Same verdict as 2026-07-30: over two years the trades entered within an hour of a ≥ 2 ATR candle
are a net-positive bucket on both timeframes (+32R and +60R at the book's own win rates), and the
gate buys a few points of profit factor by giving up a fifth to two fifths of the R. The live
fortnight's −7.2R in that bucket is a bad draw from a positive population, not a malfunction.
**Gate stays off; not a rule-1 candidate.** (needs-a-rule-1-backtest: done here, fails)
### 7.4 Are take-profit targets too far, with the market reversing first? — targets are normal in R; the stops are what is big
Planned geometry by generator (live): M15 GENERIC TP1 = 1.0R with stops of 3.6–4.4 ATR ($21–62,
one of $121 on the FOMC 19:15 short); M15 XAU_ATR TP1 = 0.69R with a 2 ATR stop ($16 target, $23
stop); M5 GENERIC TP1 = 0.5R with stops of 4.7–7.3 ATR ($13–32 target, $26–65 stop); XAU_CP TP1 =
0.88R. As a share of price the live GENERIC stops (medians 0.6–1.0%) sit above the book medians
(0.58–0.73%) but inside the book's 90th percentiles (0.97–1.66%); only the $121 FOMC stop (2.85%)
is outside. Capping the stop in ATR was tested and refuted in the 2026-09 campaign.
| measure | live (37 resolved) | M15 book | M5 book |
|---|---|---|---|
| reached TP1 | 49% | 52% | 56% |
| never reached 0.3R | 30% | 20% | 22% |
| gave back (MFE ≥ 0.5R, closed ≤ 0) | 16% | 11% | 18% |
| MFE 0.6–1.0R and closed ≤ 0 | 4 of 8 | | |
So the targets were reached about as often as in the books; what was worse was the share of trades
that never moved in favour at all. Trades that did reach 0.6–1.0R often ended flat or negative
because the trail arms at 0.6R (M5) / 0.7R (M15) and gives back to the arm level. A fixed exit at TP1
would have made this fortnight −3.55R instead of −7.15R, which is the same direction as the
2026-08-05 fixed-target study (better in 2024+ trend windows, worse in 2021 and 2023, drawdown
doubled), so nothing changes there. (observation; `--fixed-rr-tp` remains research-only)
@@ -0,0 +1,614 @@
# Wrong-direction losers: source, indicator values and timing — two-year M5 and M15 books, 2026-09-18
Follow-up to `loss_anatomy_2026-09-18.md`. Owner questions: (1) do the wrong-direction losers come
from chart patterns or from indicator-driven setups, (2) what did each indicator read on those
trades compared with the trades that got the direction right, (3) were there hours or days when
the direction was wrong more often. Same two-year books (2024-09-05 → 2026-09-05, $0.65),
regenerated on the CUDA twin with `--record --export-features`; the recorder's `indicators`
table holds every indicator's vote, value, confidence, effective weight and contribution on the
primary and both confirmation timeframes for every accepted trade. In-book statistics; no decision.
## 5min: 781 trades — 153 wrong-direction losers (20%), 215 other losers, 413 winners
Wrong direction = a losing trade whose best excursion never reached a quarter of the initial risk.
### 1. Where the wrong-direction trades came from (5min)
| Setup type | Trades | Wrong-direction | Rate | Share of all wrong-direction | Other losers | Winners | Net R |
|---|---|---|---|---|---|---|---|
| GENERIC | 411 | 73 | 18% | 48% | 115 | 223 | +80.9 |
| XAU_ATR | 32 | 6 | 19% | 4% | 9 | 17 | +8.9 |
| XAU_CP | 338 | 74 | 22% | 48% | 91 | 173 | +56.3 |
Chart patterns (5min):
| Pattern | Trades | Wrong-direction | Rate | Winners | Net R |
|---|---|---|---|---|---|
| bull_flag | 166 | 35 | 21% | 87 | +30.2 |
| double_bottom | 86 | 15 | 17% | 49 | +21.5 |
| inv_hs | 86 | 24 | 28% | 37 | +4.6 |
By grade and by regime (5min):
| Grade | Trades | Wrong-direction rate | | Regime | Trades | Wrong-direction rate |
|---|---|---|---|---|---|---|
| A | 187 | 18% | ranging | 169 | 22% |
| A+ | 1 | 0% | transitioning | 76 | 20% |
| B | 572 | 20% | trending | 61 | 21% |
| C | 21 | 29% | trending_down | 95 | 18% |
| | | | trending_up | 380 | 19% |
By direction: BUY: 683 trades, 20% wrong-direction; SELL: 98 trades, 17% wrong-direction.
### 2. The signal itself at entry (5min): wrong-direction losers vs winners
Values signed in the trade's direction where a sign applies (positive = in the trade's favour). The separation column is the probability that a random winner shows a higher value than a random wrong-direction loser: 0.50 means the measure does not separate them at all.
| Measure | Wrong-direction median | Winner median | Other-loser median | Separation |
|---|---|---|---|---|
| Composite score of the primary timeframe (in trade direction) | +31.75 | +33.07 | +32.77 | 0.53 |
| Final score after the multi-timeframe blend (in trade direction) | +29.05 | +30.08 | +29.63 | 0.53 |
| Final confidence | +53.97 | +54.66 | +53.65 | 0.54 |
| Composite score of confirmation 1 (in trade direction) | +20.56 | +20.60 | +18.22 | 0.54 |
| Composite score of confirmation 2 (in trade direction) | +16.99 | +16.50 | +17.34 | 0.51 |
| Indicators agreeing (count) | +4.00 | +5.00 | +4.00 | 0.54 |
| Indicators opposing (count) | +1.00 | +1.00 | +1.00 | 0.53 |
| Confidence penalty (points) | +16.94 | +16.62 | +16.62 | 0.50 |
| Exhaustion tags | +2.00 | +3.00 | +2.00 | 0.55 |
| Primary ATR as % of price | +0.12 | +0.12 | +0.13 | 0.51 |
| Prior 12-bar move in trade direction (ATR) | +2.91 | +2.92 | +2.90 | 0.53 |
| Hourly stretch (ATR) | +1.56 | +1.51 | +1.43 | 0.51 |
| Planned target 1 (R) | +0.73 | +0.50 | +0.53 | 0.46 |
### 3a. Each indicator on the primary timeframe (5min)
Agree = the indicator voted in the trade's direction; oppose = against. Value is signed in the trade's direction (RSI minus 50, Bollinger position minus 0.5, MACD and Keltner in ATR units, EMA alignment, rate of change in percent; ADX is unsigned strength). Contribution = the indicator's weighted score in the trade's favour.
| Indicator | Present (wrong / win) | Agree % (wrong / win) | Oppose % (wrong / win) | Median value (wrong / win) | Median confidence (wrong / win) | Mean contribution (wrong / win) | Separation of value |
|---|---|---|---|---|---|---|---|
| EMA trend (alignment) | 100% / 100% | 78% / 82% | 0% / 0% | +1.00 / +1.00 | 90 / 90 | +2.38 / +2.57 | 0.52 |
| RSI | 100% / 100% | 6% / 8% | 20% / 18% | +18.33 / +20.61 | 30 / 30 | -0.14 / -0.11 | 0.56 |
| Bollinger position | 100% / 100% | 0% / 0% | 93% / 95% | +0.47 / +0.47 | 45 / 45 | -0.60 / -0.59 | 0.52 |
| ADX (trend strength) | 100% / 100% | 63% / 69% | 3% / 1% | +29.88 / +30.40 | 70 / 70 | +0.17 / +0.19 | 0.51 |
| MACD histogram | 100% / 100% | 84% / 86% | 0% / 0% | +0.19 / +0.18 | 67 / 66 | +3.22 / +3.35 | 0.50 |
| Rate of change | 100% / 100% | 76% / 77% | 1% / 1% | +0.29 / +0.32 | 52 / 52 | +0.83 / +0.83 | 0.55 |
| Keltner extension | 100% / 100% | 73% / 80% | 0% / 0% | +2.08 / +2.21 | 65 / 65 | +1.29 / +1.44 | 0.55 |
| Divergence | 24% / 34% | 36% / 28% | 64% / 70% | +1.00 / +1.00 | 70 / 70 | -0.10 / -0.39 | 0.54 |
| Candlestick | 35% / 30% | 93% / 90% | 0% / 0% | +1.00 / +1.00 | 50 / 50 | +0.40 / +0.39 | 0.48 |
### 3b. Each indicator on the first confirmation timeframe
Agree = the indicator voted in the trade's direction; oppose = against. Value is signed in the trade's direction (RSI minus 50, Bollinger position minus 0.5, MACD and Keltner in ATR units, EMA alignment, rate of change in percent; ADX is unsigned strength). Contribution = the indicator's weighted score in the trade's favour.
| Indicator | Present (wrong / win) | Agree % (wrong / win) | Oppose % (wrong / win) | Median value (wrong / win) | Median confidence (wrong / win) | Mean contribution (wrong / win) | Separation of value |
|---|---|---|---|---|---|---|---|
| EMA trend (alignment) | 100% / 100% | 57% / 62% | 1% / 2% | +1.00 / +1.00 | 90 / 90 | +1.56 / +1.60 | 0.52 |
| RSI | 100% / 100% | 11% / 6% | 22% / 33% | +16.43 / +17.10 | 30 / 30 | -0.13 / -0.28 | 0.53 |
| Bollinger position | 100% / 100% | 1% / 0% | 71% / 79% | +0.43 / +0.46 | 45 / 45 | -0.46 / -0.58 | 0.55 |
| ADX (trend strength) | 100% / 100% | 51% / 47% | 10% / 6% | +27.18 / +25.70 | 55 / 55 | +0.12 / +0.11 | 0.46 |
| MACD histogram | 100% / 100% | 61% / 70% | 3% / 2% | +0.16 / +0.19 | 61 / 65 | +2.29 / +2.68 | 0.55 |
| Rate of change | 100% / 100% | 63% / 65% | 7% / 4% | +0.38 / +0.45 | 53 / 53 | +0.64 / +0.71 | 0.55 |
| Keltner extension | 100% / 100% | 60% / 67% | 0% / 1% | +1.78 / +1.92 | 65 / 65 | +1.17 / +1.33 | 0.53 |
| Divergence | 38% / 36% | 24% / 40% | 71% / 60% | +1.00 / +1.00 | 70 / 70 | -0.67 / -0.09 | 0.51 |
| Candlestick | 34% / 35% | 65% / 71% | 10% / 6% | +1.00 / +1.00 | 50 / 50 | +0.25 / +0.29 | 0.53 |
### 3c. Each indicator on the second confirmation timeframe
Agree = the indicator voted in the trade's direction; oppose = against. Value is signed in the trade's direction (RSI minus 50, Bollinger position minus 0.5, MACD and Keltner in ATR units, EMA alignment, rate of change in percent; ADX is unsigned strength). Contribution = the indicator's weighted score in the trade's favour.
| Indicator | Present (wrong / win) | Agree % (wrong / win) | Oppose % (wrong / win) | Median value (wrong / win) | Median confidence (wrong / win) | Mean contribution (wrong / win) | Separation of value |
|---|---|---|---|---|---|---|---|
| EMA trend (alignment) | 100% / 100% | 41% / 45% | 5% / 6% | +0.50 / +0.50 | 50 / 90 | +0.94 / +1.04 | 0.51 |
| RSI | 100% / 100% | 7% / 6% | 23% / 23% | +11.38 / +11.44 | 30 / 30 | -0.17 / -0.17 | 0.51 |
| Bollinger position | 100% / 100% | 7% / 5% | 58% / 62% | +0.37 / +0.37 | 45 / 45 | -0.39 / -0.42 | 0.53 |
| ADX (trend strength) | 100% / 100% | 42% / 42% | 16% / 15% | +26.89 / +26.78 | 55 / 55 | +0.07 / +0.07 | 0.48 |
| MACD histogram | 100% / 100% | 58% / 62% | 9% / 7% | +0.16 / +0.14 | 63 / 61 | +1.74 / +1.98 | 0.50 |
| Rate of change | 100% / 100% | 60% / 67% | 10% / 10% | +0.59 / +0.66 | 55 / 56 | +0.69 / +0.80 | 0.52 |
| Keltner extension | 100% / 100% | 48% / 44% | 3% / 3% | +1.26 / +1.31 | 65 / 60 | +0.86 / +0.68 | 0.51 |
| Divergence | 31% / 29% | 34% / 44% | 66% / 54% | +1.00 / +1.00 | 70 / 70 | -0.49 / -0.06 | 0.49 |
| Candlestick | 27% / 30% | 73% / 70% | 10% / 10% | +1.00 / +1.00 | 50 / 50 | +0.29 / +0.27 | 0.48 |
### 4. When the direction was wrong (5min)
Base rate: 20% of all trades were wrong-direction losers. Cells under 20 trades are noise.
| Entry hour (Coordinated Universal Time) | Trades | Wrong-direction | Rate | | Weekday | Trades | Wrong-direction | Rate |
|---|---|---|---|---|---|---|---|---|
| 03:00 | 50 | 8 | 16% | Mon | 153 | 27 | 18% |
| 04:00 | 30 | 7 | 23% | Tue | 159 | 35 | 22% |
| 05:00 | 52 | 17 | 33% | Wed | 176 | 33 | 19% |
| 06:00 | 60 | 9 | 15% | Thu | 135 | 28 | 21% |
| 07:00 | 51 | 8 | 16% | Fri | 151 | 28 | 19% |
| 08:00 | 62 | 15 | 24% | Sun | 7 | 2 | 29% |
| 12:00 | 63 | 11 | 17% | | | | |
| 14:00 | 117 | 22 | 19% | | | | |
| 15:00 | 90 | 15 | 17% | | | | |
| 17:00 | 35 | 8 | 23% | | | | |
| 18:00 | 41 | 8 | 20% | | | | |
| 19:00 | 45 | 7 | 16% | | | | |
| 20:00 | 42 | 9 | 21% | | | | |
| 21:00 | 12 | 3 | 25% | | | | |
| 22:00 | 31 | 6 | 19% | | | | |
Weekday-hour cells with at least 10 trades and the highest wrong-direction rate (5min):
| Day | Hour | Trades | Wrong-direction | Rate |
|---|---|---|---|---|
| Thu | 20:00 | 11 | 6 | 55% |
| Tue | 05:00 | 16 | 8 | 50% |
| Wed | 18:00 | 11 | 4 | 36% |
| Wed | 08:00 | 17 | 6 | 35% |
| Wed | 06:00 | 14 | 4 | 29% |
| Thu | 12:00 | 14 | 4 | 29% |
Months with the highest wrong-direction rate: 2024-12 60% (3 of 5); 2025-08 33% (5 of 15); 2026-09 31% (4 of 13); 2025-09 27% (9 of 33).
### Appendix 5min: every wrong-direction loser with its primary-timeframe indicator values
Columns: entry time, setup (pattern), grade, direction, regime, final score in trade direction / confidence, then per indicator `signed value (vote)` where the vote is −2 strong sell … +2 strong buy; blank = the indicator did not vote. Values as defined in section 3. R = realised result.
| Entry | Setup | Grade | Dir | Regime | Score/Conf | EMA | RSI | Bollinger | ADX | MACD | ROC | Keltner | Div | Candle | R |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2024-09-13 17:05 | XAU_CP (bull_flag) | B | BUY | ranging | +17/57 | +1.00 (+2) | +15.26 (-1) | +0.41 (-1) | +22.50 (+0) | +0.05 (+1) | +0.08 (+1) | +1.66 (+1) | | | -1.08 |
| 2024-09-18 18:35 | GENERIC | A | BUY | trending_up | +34/60 | +1.00 (+2) | +31.29 (+0) | +0.51 (-1) | +31.03 (+1) | +0.64 (+2) | +1.11 (+1) | +3.99 (+1) | | +1.00 (+1) | -1.02 |
| 2024-10-22 17:45 | XAU_CP (bull_flag) | A | BUY | trending_up | +35/58 | +1.00 (+2) | +18.50 (+0) | +0.35 (-1) | +27.02 (+1) | +0.13 (+2) | +0.12 (+1) | +2.01 (+1) | | | -1.08 |
| 2024-11-06 04:50 | XAU_CP (inv_hs) | B | BUY | ranging | +24/40 | +0.00 (+0) | +8.85 (+0) | +0.46 (-1) | +17.57 (+0) | +0.28 (+2) | +0.31 (+1) | +1.46 (+1) | +1.00 (+1) | | -1.08 |
| 2024-11-08 14:10 | XAU_CP (bull_flag) | A | BUY | transitioning | +32/60 | +1.00 (+2) | +19.99 (+0) | +0.59 (-1) | +21.16 (+0) | +0.33 (+2) | +0.43 (+1) | +2.73 (+1) | | | -1.08 |
| 2024-12-02 14:15 | XAU_CP (bull_flag) | B | BUY | ranging | +14/36 | +1.00 (+2) | +17.74 (-1) | +0.53 (-1) | +18.08 (+0) | +0.15 (+2) | +0.14 (+0) | +1.97 (+1) | | | -1.08 |
| 2024-12-09 15:15 | XAU_CP (bull_flag) | B | BUY | trending_up | +28/47 | +1.00 (+2) | +31.38 (+0) | +0.42 (-1) | +43.81 (+1) | +0.21 (+0) | +0.38 (+1) | +2.69 (+1) | | | -1.08 |
| 2024-12-26 15:50 | XAU_CP (bull_flag) | B | BUY | trending_up | +35/55 | +1.00 (+2) | +29.64 (+0) | +0.42 (-1) | +35.94 (+1) | +0.41 (+2) | +0.30 (+1) | +3.30 (+1) | | | -1.08 |
| 2025-03-25 14:05 | XAU_CP (bull_flag) | B | BUY | trending_up | +43/41 | +1.00 (+2) | +28.08 (+0) | +0.58 (-1) | +26.66 (+1) | +0.40 (+2) | +0.31 (+1) | +3.17 (+1) | | | -1.07 |
| 2025-04-09 19:45 | XAU_ATR | A | BUY | transitioning | +31/65 | +1.00 (+2) | +17.01 (+0) | +0.41 (-1) | +24.65 (+0) | +0.37 (+2) | +1.01 (+1) | +2.11 (+1) | +1.00 (+1) | +2.00 (+1) | -1.04 |
| 2025-04-10 05:05 | XAU_CP (double_bottom) | B | BUY | trending_up | +18/60 | +1.00 (+2) | +14.04 (+1) | +0.59 (-1) | +41.41 (+1) | -0.15 (+0) | +0.14 (+1) | +1.47 (+0) | +1.00 (-1) | | -1.05 |
| 2025-04-16 22:10 | GENERIC | A | BUY | trending_up | +37/60 | +1.00 (+2) | +27.80 (+0) | +0.79 (-1) | +26.61 (+1) | +0.41 (+2) | +0.50 (+1) | +4.20 (+1) | | +1.00 (+1) | -1.03 |
| 2025-04-22 05:00 | GENERIC | C | BUY | trending_up | +25/51 | +1.00 (+2) | +12.19 (+1) | -0.06 (+0) | +41.38 (+1) | -0.38 (+0) | +0.02 (+1) | +0.73 (+0) | | | -1.07 |
| 2025-04-23 06:55 | GENERIC | B | SELL | trending_down | +32/54 | +1.00 (-2) | +22.24 (+0) | +0.51 (+1) | +41.78 (-1) | +0.13 (-1) | +0.35 (-1) | +3.09 (-1) | | -2.00 (-1) | -1.02 |
| 2025-04-24 22:30 | GENERIC | A | BUY | trending_up | +37/59 | +1.00 (+2) | +33.05 (+0) | +0.70 (-1) | +46.34 (+1) | +0.39 (+2) | +0.53 (+1) | +4.25 (+1) | | +1.00 (+1) | -1.02 |
| 2025-04-27 22:15 | GENERIC | A | BUY | trending_up | +36/56 | +1.00 (+2) | +28.40 (+0) | +0.55 (-1) | +54.48 (+1) | +0.33 (+2) | +0.53 (+1) | +3.82 (+1) | | +1.00 (+1) | -1.02 |
| 2025-04-28 15:20 | XAU_CP (bull_flag) | B | BUY | trending_up | +25/48 | +1.00 (+2) | +25.67 (+0) | +0.32 (-1) | +39.47 (+1) | +0.24 (+0) | +0.68 (+1) | +3.01 (+1) | | | -1.05 |
| 2025-04-28 18:50 | GENERIC | B | BUY | trending_up | +31/58 | +1.00 (+2) | +21.76 (+0) | +0.43 (-1) | +34.69 (+1) | +0.20 (+2) | +0.41 (+1) | +2.28 (+1) | +1.00 (-1) | +1.00 (+1) | -1.04 |
| 2025-05-06 20:55 | GENERIC | A | BUY | trending_up | +41/60 | +1.00 (+2) | +26.50 (+0) | +0.57 (-1) | +31.62 (+1) | +0.33 (+2) | +0.34 (+1) | +3.85 (+1) | | +1.00 (+1) | -1.04 |
| 2025-05-13 05:15 | XAU_CP (bull_flag) | B | BUY | trending | +25/55 | +0.50 (+0) | +33.63 (-1) | +0.57 (-1) | +38.02 (+1) | +0.54 (+2) | +0.74 (+1) | +4.77 (+1) | | +1.00 (+1) | -1.05 |
| 2025-05-16 12:10 | GENERIC | B | SELL | trending_down | +30/54 | +1.00 (-2) | +20.02 (+0) | +0.35 (+1) | +49.72 (-1) | +0.13 (-2) | +0.51 (-1) | +2.45 (-1) | -1.00 (+1) | | -1.02 |
| 2025-05-21 05:55 | GENERIC | A | BUY | transitioning | +32/69 | +1.00 (+2) | +23.49 (-1) | +0.69 (-1) | +23.44 (+0) | +0.39 (+2) | +0.48 (+1) | +3.10 (+1) | | | -1.03 |
| 2025-05-21 19:30 | XAU_CP (bull_flag) | B | BUY | transitioning | +30/67 | +1.00 (+2) | +24.56 (-1) | +0.50 (-1) | +20.86 (+0) | +0.31 (+2) | +0.28 (+1) | +2.56 (+1) | | | -1.08 |
| 2025-05-28 08:15 | XAU_CP (bull_flag) | A | BUY | trending_up | +38/56 | +1.00 (+2) | +22.27 (+0) | +0.46 (-1) | +47.44 (+1) | +0.26 (+2) | +0.39 (+1) | +2.66 (+1) | | | -1.08 |
| 2025-06-02 12:25 | XAU_CP (bull_flag) | A | BUY | trending_up | +38/55 | +1.00 (+2) | +18.99 (+0) | +0.47 (-1) | +33.76 (+1) | +0.21 (+2) | +0.20 (+1) | +2.23 (+1) | | | -1.06 |
| 2025-06-03 14:10 | GENERIC | B | SELL | trending_down | +35/54 | +1.00 (-2) | +20.43 (+0) | +0.48 (+1) | +32.72 (-1) | +0.47 (-2) | +0.64 (+0) | +2.54 (-1) | | | -1.03 |
| 2025-06-06 04:50 | XAU_CP (bull_flag) | B | BUY | trending_up | +40/39 | +1.00 (+2) | +20.30 (+0) | +0.31 (-1) | +27.03 (+1) | +0.07 (+2) | +0.12 (+1) | +1.83 (+1) | | | -1.08 |
| 2025-06-13 06:40 | XAU_CP (inv_hs) | B | BUY | ranging | +13/61 | +0.50 (+0) | +6.92 (+0) | +0.44 (-1) | +18.37 (+0) | +0.14 (+2) | +0.13 (+1) | +0.85 (+0) | | | -1.07 |
| 2025-07-14 07:30 | XAU_CP (bull_flag) | A | BUY | trending_up | +36/55 | +1.00 (+2) | +23.25 (+0) | +0.48 (-1) | +27.82 (+1) | +0.34 (+2) | +0.39 (+1) | +3.23 (+1) | | +1.00 (+1) | -1.07 |
| 2025-07-14 14:55 | GENERIC | B | SELL | trending_down | +34/52 | +1.00 (-2) | +17.02 (+0) | +0.45 (+1) | +54.01 (-1) | +0.24 (-2) | +0.30 (+0) | +2.27 (-1) | | | -1.03 |
| 2025-07-17 12:45 | GENERIC | B | SELL | trending_down | +30/59 | +1.00 (-2) | +24.27 (+0) | +0.75 (+1) | +31.17 (-1) | +0.29 (-2) | +0.35 (-1) | +3.08 (-1) | -1.00 (+1) | -1.00 (-1) | -1.03 |
| 2025-08-05 15:50 | XAU_CP (bull_flag) | B | BUY | trending_up | +20/53 | +1.00 (+2) | +17.80 (+0) | +0.35 (-1) | +42.84 (+1) | -0.15 (+0) | +0.11 (+1) | +1.97 (+1) | +1.00 (-1) | +1.00 (+1) | -1.08 |
| 2025-08-07 07:35 | GENERIC | B | BUY | trending_up | +33/51 | +1.00 (+2) | +32.53 (+0) | +0.48 (-1) | +43.71 (+1) | +0.45 (+2) | +0.27 (+0) | +3.98 (+1) | | | -1.03 |
| 2025-08-07 20:30 | XAU_CP (bull_flag) | B | BUY | trending_up | +25/50 | +1.00 (+2) | +27.04 (+0) | +0.29 (+0) | +30.42 (+1) | +0.22 (+0) | +0.12 (+1) | +2.27 (+1) | | | -1.08 |
| 2025-08-22 14:35 | GENERIC | A | BUY | trending_up | +37/60 | +1.00 (+2) | +36.01 (+0) | +0.51 (-1) | +50.07 (+1) | +0.75 (+2) | +1.18 (+1) | +5.04 (+1) | +1.00 (+1) | +1.00 (+1) | -0.01 |
| 2025-08-28 12:10 | XAU_CP (bull_flag) | B | BUY | trending_up | +38/55 | +1.00 (+2) | +21.43 (+0) | +0.42 (-1) | +35.24 (+1) | +0.31 (+2) | +0.26 (+1) | +2.80 (+1) | | | -1.08 |
| 2025-09-03 18:10 | GENERIC | B | BUY | trending_up | +31/55 | +1.00 (+2) | +24.49 (+0) | +0.48 (-1) | +39.40 (+1) | +0.04 (+1) | +0.15 (+1) | +2.59 (+1) | | +1.00 (+1) | -1.04 |
| 2025-09-04 05:00 | XAU_CP (inv_hs) | B | BUY | trending | +9/0 | -0.50 (+0) | +1.00 (+0) | +0.49 (-1) | +40.71 (-1) | +0.33 (+2) | -0.01 (-1) | +0.52 (+0) | | | -1.08 |
| 2025-09-09 05:40 | XAU_CP (bull_flag) | A | BUY | trending_up | +37/60 | +1.00 (+2) | +14.70 (+1) | +0.38 (-1) | +30.75 (+1) | +0.05 (+2) | +0.08 (+1) | +1.87 (+1) | | | -1.08 |
| 2025-09-12 14:05 | XAU_CP (inv_hs) | B | BUY | ranging | +9/25 | +1.00 (+2) | +13.89 (-1) | +0.31 (-1) | +15.85 (+0) | +0.19 (+2) | +0.09 (+1) | +1.22 (+0) | | +1.00 (+1) | -1.08 |
| 2025-09-17 18:10 | GENERIC | B | BUY | trending_up | +32/59 | +1.00 (+2) | +33.30 (+0) | +0.97 (-1) | +25.59 (-1) | +0.26 (+2) | +0.63 (+1) | +2.93 (+0) | | | -1.02 |
| 2025-09-22 18:25 | XAU_CP (bull_flag) | B | BUY | trending_up | +23/48 | +1.00 (+2) | +23.92 (+0) | +0.32 (-1) | +45.99 (+1) | -0.07 (+0) | +0.12 (+0) | +2.29 (+1) | | +1.00 (+1) | -1.08 |
| 2025-09-25 12:25 | XAU_CP (inv_hs) | B | BUY | ranging | +15/21 | +1.00 (+2) | +5.24 (+0) | +0.20 (+0) | +16.32 (+0) | +0.16 (+2) | +0.08 (+0) | +0.67 (+0) | | | -1.08 |
| 2025-09-30 14:55 | XAU_CP (bull_flag) | B | BUY | trending | +21/49 | +0.50 (+0) | +22.91 (-1) | +0.37 (-1) | +32.40 (+1) | +0.25 (+2) | +0.68 (+1) | +2.82 (+1) | | +1.00 (+1) | -1.04 |
| 2025-09-30 17:25 | XAU_CP (inv_hs) | B | BUY | ranging | -7/24 | +1.00 (+2) | +12.12 (-1) | +0.34 (-1) | +18.10 (+0) | -0.01 (+0) | +0.09 (+0) | +1.04 (+0) | | +0.00 (+0) | -1.08 |
| 2025-10-01 08:20 | GENERIC | A | BUY | trending_up | +39/58 | +1.00 (+2) | +29.73 (+0) | +0.55 (-1) | +31.76 (+1) | +0.57 (+2) | +0.57 (+1) | +4.37 (+1) | | +1.00 (+1) | -1.02 |
| 2025-10-06 05:10 | GENERIC | A | BUY | trending_up | +40/61 | +1.00 (+2) | +25.52 (+0) | +0.55 (-1) | +57.26 (+1) | +0.15 (+2) | +0.39 (+1) | +3.07 (+1) | | +1.00 (+1) | -1.03 |
| 2025-10-09 05:35 | XAU_CP (bull_flag) | B | BUY | ranging | +3/34 | +1.00 (+2) | +19.07 (-1) | +0.34 (-1) | +22.72 (+0) | -0.07 (+0) | +0.08 (+1) | +1.65 (+1) | | | -1.07 |
| 2025-10-14 05:20 | GENERIC | B | BUY | trending_up | +38/54 | +1.00 (+2) | +26.11 (+0) | +0.58 (-1) | +25.38 (+1) | +0.15 (+2) | +0.34 (+0) | +2.61 (+1) | | | -1.02 |
| 2025-10-15 08:10 | GENERIC | A | BUY | trending_up | +40/59 | +1.00 (+2) | +32.15 (+0) | +0.55 (-1) | +27.48 (+1) | +0.23 (+2) | +0.34 (+1) | +3.33 (+1) | | +1.00 (+1) | -1.02 |
| 2025-10-17 04:05 | GENERIC | B | BUY | transitioning | +29/72 | +1.00 (+2) | +20.77 (-1) | +0.49 (-1) | +23.22 (+0) | +0.26 (+2) | +0.34 (+1) | +2.45 (+1) | | | -1.02 |
| 2025-10-17 05:00 | XAU_CP (double_bottom) | B | BUY | ranging | -4/48 | +1.00 (+2) | +15.91 (-1) | +0.52 (-1) | +18.94 (+0) | +0.01 (+1) | +0.05 (+1) | +2.03 (+1) | +1.00 (-1) | | -1.03 |
| 2025-10-21 03:40 | XAU_CP (inv_hs) | B | BUY | ranging | +22/33 | +0.00 (+0) | +2.55 (+0) | +0.26 (+0) | +21.77 (+0) | +0.25 (+2) | +0.28 (+1) | +0.53 (+0) | +1.00 (+1) | | -1.05 |
| 2025-10-23 14:10 | GENERIC | B | BUY | trending_up | +35/53 | +1.00 (+2) | +15.86 (+0) | +0.39 (-1) | +27.39 (+1) | +0.10 (+2) | +0.31 (+1) | +1.83 (+1) | | | -1.01 |
| 2025-10-23 18:15 | XAU_CP (double_bottom) | B | BUY | ranging | +18/34 | +0.50 (+0) | +4.92 (+0) | +0.42 (-1) | +23.85 (+0) | +0.15 (+2) | +0.33 (+1) | +0.80 (+0) | +1.00 (+1) | +1.00 (+1) | -1.04 |
| 2025-10-24 14:15 | XAU_CP (bull_flag) | B | BUY | trending | +15/30 | +0.50 (+0) | +24.15 (-1) | +0.45 (-1) | +43.08 (+1) | +0.04 (+1) | +0.41 (+1) | +2.69 (+1) | | | -1.18 |
| 2025-10-30 20:05 | XAU_CP (bull_flag) | B | BUY | trending_up | +21/26 | +1.00 (+2) | +15.88 (+0) | +0.29 (+0) | +25.73 (+1) | -0.05 (+0) | +0.02 (+1) | +1.40 (+0) | | +1.00 (+1) | -1.07 |
| 2025-10-31 12:50 | GENERIC | A | BUY | trending_up | +36/60 | +1.00 (+2) | +22.69 (+0) | +0.55 (-1) | +27.66 (+1) | +0.47 (+2) | +0.54 (+1) | +2.98 (+1) | | | -1.03 |
| 2025-11-03 15:15 | GENERIC | B | BUY | trending_up | +32/52 | +1.00 (+2) | +16.98 (+0) | +0.43 (-1) | +27.07 (+1) | +0.20 (+2) | +0.26 (+1) | +1.47 (+0) | | | -1.04 |
| 2025-11-05 08:35 | GENERIC | C | BUY | trending_up | +27/53 | +1.00 (+2) | +14.30 (+1) | +0.32 (-1) | +33.40 (+1) | +0.25 (+0) | +0.29 (+1) | +1.58 (+1) | | | -1.03 |
| 2025-11-11 03:20 | GENERIC | A | BUY | trending_up | +40/59 | +1.00 (+2) | +16.16 (+0) | +0.62 (-1) | +34.85 (+1) | +0.15 (+2) | +0.27 (+1) | +2.21 (+1) | | | -1.03 |
| 2025-11-11 04:00 | GENERIC | C | BUY | trending_up | +26/54 | +1.00 (+2) | +13.25 (+1) | +0.30 (-1) | +40.00 (+1) | +0.05 (+0) | +0.14 (+0) | +1.57 (+1) | | +1.00 (+1) | -1.03 |
| 2025-11-12 18:30 | XAU_CP (double_bottom) | B | BUY | trending_up | +19/49 | +1.00 (+2) | +15.06 (+0) | +0.49 (-1) | +45.47 (+1) | -0.16 (+0) | +0.06 (+0) | +1.61 (+1) | +1.00 (-1) | | -1.03 |
| 2025-11-14 03:25 | XAU_CP (bull_flag) | B | BUY | trending | +17/29 | +0.50 (+0) | +21.20 (-1) | +0.37 (-1) | +39.69 (+1) | +0.09 (+2) | +0.18 (+0) | +2.10 (+1) | | | -1.07 |
| 2025-11-19 15:00 | GENERIC | C | BUY | ranging | +28/66 | +1.00 (+2) | +15.01 (-1) | +0.77 (-1) | +14.50 (+0) | +0.20 (+2) | +0.40 (+1) | +2.17 (+2) | | +1.00 (+1) | -1.03 |
| 2025-11-24 21:45 | XAU_CP (bull_flag) | B | BUY | trending_up | +23/52 | +1.00 (+2) | +29.57 (+0) | +0.53 (-1) | +51.36 (+1) | -0.05 (+0) | +0.22 (+1) | +3.03 (+1) | +1.00 (-1) | +1.00 (+1) | -1.06 |
| 2025-12-03 14:40 | GENERIC | B | BUY | trending_up | +28/51 | +1.00 (+2) | +11.96 (+1) | +0.31 (-1) | +37.04 (+1) | +0.03 (+2) | +0.18 (+0) | +1.35 (+0) | | | -1.03 |
| 2025-12-08 03:20 | XAU_CP (bull_flag) | B | BUY | ranging | +16/37 | +0.50 (+0) | +15.10 (-1) | +0.46 (-1) | +20.01 (+0) | +0.24 (+2) | +0.22 (+1) | +1.64 (+1) | | +1.00 (+1) | -1.08 |
| 2025-12-12 14:55 | XAU_CP (double_bottom) | B | BUY | ranging | -2/40 | +1.00 (+2) | +14.50 (-1) | +0.67 (-1) | +16.54 (+0) | +0.04 (+1) | +0.14 (+1) | +1.54 (+0) | | | -1.03 |
| 2025-12-12 20:10 | XAU_CP (inv_hs) | B | BUY | ranging | +5/42 | +0.00 (+0) | +10.85 (-1) | +0.58 (-1) | +10.27 (+0) | +0.12 (+2) | +0.13 (+1) | +1.29 (+0) | | +1.00 (+1) | -1.06 |
| 2025-12-19 14:40 | XAU_CP (double_bottom) | B | BUY | ranging | -1/37 | +1.00 (+2) | +10.55 (-1) | +0.52 (-1) | +14.44 (+0) | +0.06 (+1) | +0.10 (+1) | +1.30 (+0) | | | -1.05 |
| 2025-12-22 14:20 | XAU_CP (bull_flag) | B | BUY | trending_up | +35/50 | +1.00 (+2) | +16.35 (+0) | +0.36 (-1) | +27.17 (+1) | +0.11 (+2) | +0.20 (+0) | +1.70 (+1) | | | -1.07 |
| 2025-12-24 15:45 | GENERIC | B | SELL | trending_down | +32/54 | +1.00 (-2) | +18.47 (+0) | +0.52 (+1) | +44.48 (-1) | +0.18 (-2) | +0.27 (+0) | +2.24 (-1) | -1.00 (-1) | | -1.02 |
| 2026-01-05 19:00 | XAU_CP (double_bottom) | B | BUY | ranging | +3/36 | +1.00 (+2) | +7.10 (+0) | +0.48 (-1) | +9.67 (+0) | +0.04 (+1) | +0.07 (+0) | +0.85 (+0) | | | -1.05 |
| 2026-01-08 21:00 | GENERIC | B | BUY | trending_up | +30/51 | +1.00 (+2) | +22.66 (+0) | +0.36 (-1) | +37.50 (+1) | +0.46 (+0) | +0.43 (+1) | +2.73 (+1) | +1.00 (+1) | | -1.03 |
| 2026-01-13 14:50 | GENERIC | B | BUY | trending_up | +29/55 | +1.00 (+2) | +23.19 (+0) | +0.61 (-1) | +39.89 (+1) | +0.12 (+2) | +0.48 (+1) | +2.96 (+1) | +1.00 (-1) | | -1.02 |
| 2026-01-15 03:30 | GENERIC | B | SELL | trending_down | +36/54 | +1.00 (-2) | +18.63 (+0) | +0.51 (+1) | +35.24 (-1) | +0.28 (-2) | +0.45 (+0) | +2.10 (-1) | | | -1.02 |
| 2026-01-21 06:30 | GENERIC | B | BUY | trending_up | +30/51 | +1.00 (+2) | +22.05 (+0) | +0.60 (-1) | +36.19 (+1) | -0.03 (+0) | +0.20 (+1) | +2.14 (+1) | | | -1.02 |
| 2026-01-26 06:55 | GENERIC | B | BUY | transitioning | +29/59 | +1.00 (+2) | +17.05 (+0) | +0.48 (-1) | +23.57 (+0) | +0.42 (+2) | +0.39 (+0) | +2.24 (+1) | | | -1.02 |
| 2026-01-27 04:05 | GENERIC | B | BUY | ranging | +29/59 | +1.00 (+2) | +8.36 (+0) | +0.26 (+0) | +7.19 (+0) | +0.01 (+2) | +0.21 (+1) | +0.80 (+2) | | | -1.05 |
| 2026-01-28 08:35 | GENERIC | B | BUY | trending_up | +40/54 | +1.00 (+2) | +30.26 (+0) | +0.49 (-1) | +40.92 (+1) | +0.10 (+2) | +0.45 (+1) | +2.45 (+1) | | | -1.02 |
| 2026-01-29 21:30 | XAU_CP (bull_flag) | B | BUY | trending | +4/22 | +0.50 (+0) | +20.47 (-1) | +0.35 (-1) | +28.64 (+1) | +0.34 (+0) | +1.19 (+0) | +2.27 (+1) | | +1.00 (+1) | -1.02 |
| 2026-02-02 06:50 | GENERIC | B | SELL | trending_down | +36/52 | +1.00 (-2) | +19.15 (+0) | +0.35 (+1) | +57.06 (-1) | +0.24 (+0) | +2.41 (-2) | +2.15 (-1) | | | -1.00 |
| 2026-02-03 07:40 | XAU_ATR | A | BUY | trending_up | +39/59 | +1.00 (+2) | +27.24 (+0) | +0.52 (-1) | +42.97 (+1) | +0.24 (+2) | +1.17 (+1) | +3.55 (+1) | | +1.00 (+1) | -1.02 |
| 2026-02-04 12:35 | XAU_CP (inv_hs) | B | BUY | trending | +29/40 | +0.50 (+0) | +11.30 (+0) | +0.59 (-1) | +26.88 (+1) | +0.30 (+2) | +0.60 (+1) | +1.67 (+1) | | +1.00 (+1) | -1.02 |
| 2026-02-10 04:50 | XAU_CP (double_bottom) | B | BUY | ranging | +7/38 | -0.50 (+0) | +4.07 (+0) | +0.59 (-1) | +22.19 (+0) | +0.10 (+2) | +0.06 (+1) | +0.62 (+0) | | | -1.03 |
| 2026-02-10 14:35 | XAU_CP (bull_flag) | B | BUY | transitioning | +38/40 | +1.00 (+2) | +18.48 (+0) | +0.47 (-1) | +24.35 (+0) | +0.35 (+2) | +0.44 (+1) | +2.33 (+1) | | | -1.05 |
| 2026-02-12 19:50 | XAU_CP (inv_hs) | B | BUY | ranging | +17/33 | +0.00 (+0) | +4.15 (+0) | +0.42 (-1) | +21.09 (+0) | +0.30 (+2) | +0.37 (+1) | +0.82 (+0) | +1.00 (+1) | +1.00 (+1) | -1.02 |
| 2026-02-13 04:20 | XAU_CP (double_bottom) | B | BUY | trending | +4/25 | +0.50 (+0) | +15.70 (+0) | +0.65 (-1) | +28.32 (+1) | -0.01 (+0) | +0.29 (+1) | +1.65 (+0) | | | -1.03 |
| 2026-02-13 19:25 | GENERIC | B | BUY | trending_up | +28/59 | +1.00 (+2) | +16.04 (+0) | +0.47 (-1) | +35.73 (+1) | +0.13 (+2) | +0.30 (+0) | +1.88 (+1) | +1.00 (-1) | | -1.12 |
| 2026-02-16 15:10 | GENERIC | B | SELL | trending_down | +30/56 | +1.00 (-2) | +19.21 (+0) | +0.56 (+1) | +38.66 (-1) | +0.26 (-2) | +0.35 (+0) | +2.08 (-1) | -1.00 (+1) | | -1.04 |
| 2026-02-20 14:55 | GENERIC | B | BUY | transitioning | +27/58 | +1.00 (+2) | +24.61 (-1) | +0.47 (-1) | +21.69 (+0) | +0.23 (+2) | +0.37 (+1) | +2.21 (+1) | | | -1.02 |
| 2026-02-24 05:55 | XAU_CP (double_bottom) | B | BUY | trending | +23/38 | +0.00 (+0) | +9.09 (+0) | +0.75 (-1) | +30.00 (+1) | +0.25 (+2) | +0.19 (+1) | +1.27 (+0) | | | -1.03 |
| 2026-02-25 17:10 | GENERIC | B | BUY | transitioning | +28/58 | +1.00 (+2) | +17.72 (+0) | +0.40 (-1) | +18.15 (+0) | +0.18 (+2) | +0.34 (+0) | +1.94 (+1) | | | -1.02 |
| 2026-02-26 20:15 | GENERIC | A | BUY | transitioning | +31/59 | +1.00 (+2) | +13.98 (+0) | +0.49 (-1) | +18.72 (+0) | +0.16 (+2) | +0.21 (+1) | +1.79 (+1) | | | -1.03 |
| 2026-03-02 07:35 | GENERIC | B | BUY | trending_up | +29/51 | +1.00 (+2) | +23.33 (+0) | +0.34 (-1) | +44.16 (+1) | +0.30 (+0) | +0.62 (+1) | +2.49 (+1) | | | -1.02 |
| 2026-03-02 08:35 | GENERIC | C | BUY | trending_up | +28/45 | +1.00 (+2) | +14.65 (+1) | +0.25 (+0) | +42.91 (+1) | -0.13 (+0) | +0.21 (+1) | +1.34 (+0) | | | -1.03 |
| 2026-03-03 15:20 | GENERIC | B | SELL | trending_down | +35/50 | +1.00 (-2) | +24.35 (+0) | +0.39 (+1) | +47.11 (-1) | +0.25 (+0) | +2.36 (-2) | +2.28 (-1) | | | -1.00 |
| 2026-03-06 15:25 | XAU_ATR | A | BUY | transitioning | +33/64 | +1.00 (+2) | +24.51 (-1) | +0.52 (-1) | +23.88 (+0) | +0.41 (+2) | +1.15 (+1) | +2.89 (+1) | | +1.00 (+1) | -1.02 |
| 2026-03-10 06:00 | GENERIC | A | BUY | trending_up | +37/61 | +1.00 (+2) | +16.30 (+0) | +0.54 (-1) | +29.88 (+1) | +0.22 (+2) | +0.20 (+1) | +1.87 (+1) | | | -1.04 |
| 2026-03-12 20:35 | GENERIC | B | SELL | trending_down | +32/54 | +1.00 (-2) | +23.40 (+0) | +0.62 (+1) | +59.86 (-1) | +0.14 (-2) | +0.40 (+0) | +2.23 (-1) | -1.00 (+1) | +0.00 (+0) | -1.02 |
| 2026-03-13 08:05 | GENERIC | B | SELL | trending_down | +30/54 | +1.00 (-2) | +22.31 (+0) | +0.75 (+1) | +43.85 (-1) | +0.09 (-1) | +0.43 (-1) | +2.83 (-1) | | -1.00 (-1) | -1.02 |
| 2026-03-17 05:15 | GENERIC | A | BUY | trending_up | +36/56 | +1.00 (+2) | +16.67 (+0) | +0.45 (-1) | +33.11 (+1) | +0.05 (+2) | +0.22 (+1) | +1.73 (+1) | | | -1.03 |
| 2026-03-25 07:20 | XAU_CP (inv_hs) | B | BUY | trending | +22/30 | +0.00 (+0) | +7.12 (+0) | +0.37 (-1) | +34.57 (-1) | +0.29 (+2) | +0.25 (+1) | +1.23 (+0) | | +1.00 (+1) | -1.03 |
| 2026-03-27 06:15 | GENERIC | A | BUY | trending_up | +37/58 | +1.00 (+2) | +18.33 (+0) | +0.45 (-1) | +41.33 (+1) | +0.10 (+2) | +0.61 (+1) | +2.31 (+1) | | | -1.02 |
| 2026-03-27 19:30 | XAU_CP (inv_hs) | B | BUY | ranging | +13/55 | +1.00 (+2) | +10.69 (-1) | +0.30 (-1) | +23.51 (+0) | +0.13 (+2) | +0.10 (+0) | +1.21 (+0) | | +1.00 (+1) | -1.03 |
| 2026-03-31 05:05 | XAU_CP (inv_hs) | B | BUY | ranging | +12/30 | +1.00 (+2) | +5.79 (+0) | +0.49 (-1) | +18.60 (+0) | +0.16 (+2) | +0.26 (+1) | +0.78 (+0) | | +2.00 (+1) | -1.04 |
| 2026-03-31 05:40 | GENERIC | C | BUY | ranging | +26/78 | +1.00 (+2) | +10.36 (-1) | +0.62 (-1) | +17.59 (+0) | +0.21 (+2) | +0.35 (+1) | +1.58 (+2) | +1.00 (+1) | | -1.03 |
| 2026-04-02 20:05 | GENERIC | B | BUY | trending_up | +32/59 | +1.00 (+2) | +9.57 (+1) | +0.28 (+0) | +25.77 (+1) | +0.05 (+2) | +0.02 (+1) | +1.15 (+0) | | | -1.84 |
| 2026-04-13 17:40 | XAU_ATR | A | BUY | trending_up | +35/63 | +1.00 (+2) | +15.45 (+0) | +0.34 (-1) | +31.06 (+1) | +0.13 (+2) | +0.18 (+1) | +1.61 (+1) | | | -1.04 |
| 2026-04-20 22:20 | GENERIC | A | BUY | trending_up | +36/61 | +1.00 (+2) | +27.62 (+0) | +0.57 (-1) | +26.15 (+1) | +0.47 (+2) | +0.33 (+1) | +3.27 (+1) | | +1.00 (+1) | -1.03 |
| 2026-04-21 19:25 | XAU_CP (inv_hs) | B | BUY | ranging | +18/35 | -0.50 (+0) | +5.37 (+0) | +0.39 (-1) | +19.15 (+0) | +0.31 (+2) | +0.26 (+1) | +0.91 (+0) | +1.00 (+1) | +2.00 (+1) | -1.04 |
| 2026-05-03 22:45 | XAU_CP (inv_hs) | B | BUY | ranging | +14/37 | +0.50 (+0) | +11.54 (-1) | +0.47 (-1) | +16.73 (+0) | +0.31 (+2) | +0.25 (+0) | +1.67 (+1) | | | -1.04 |
| 2026-05-06 07:00 | XAU_CP (bull_flag) | B | BUY | trending_up | +26/51 | +1.00 (+2) | +16.40 (+0) | +0.38 (-1) | +42.22 (+1) | +0.07 (+2) | +0.13 (+0) | +1.78 (+1) | +1.00 (-1) | +1.00 (+1) | -1.05 |
| 2026-05-06 14:45 | GENERIC | B | BUY | trending_up | +32/58 | +1.00 (+2) | +14.48 (+1) | +0.31 (-1) | +25.32 (+1) | +0.15 (+2) | +0.20 (+1) | +1.40 (+0) | | | -1.03 |
| 2026-05-12 12:55 | XAU_CP (double_bottom) | B | BUY | ranging | +10/29 | +0.50 (+0) | +9.80 (+0) | +0.41 (-1) | +14.24 (+0) | +0.10 (+2) | +0.11 (+1) | +1.23 (+0) | | | -1.03 |
| 2026-05-18 08:55 | GENERIC | A | BUY | transitioning | +31/65 | +0.50 (+0) | +11.76 (+0) | +0.47 (-1) | +17.94 (+0) | +0.09 (+2) | +0.21 (+1) | +1.36 (+2) | | | -1.03 |
| 2026-05-19 03:40 | XAU_CP (inv_hs) | B | BUY | trending | +22/31 | +0.00 (+0) | -1.77 (+0) | +0.10 (+0) | +26.13 (-1) | +0.12 (+2) | +0.05 (+1) | +0.15 (+0) | | +1.00 (+1) | -1.06 |
| 2026-05-19 08:00 | XAU_CP (double_bottom) | B | BUY | ranging | -6/38 | +0.50 (+0) | +12.94 (-1) | +0.58 (-1) | +15.75 (+0) | +0.10 (+2) | +0.10 (+1) | +1.51 (+0) | +1.00 (-1) | | -1.03 |
| 2026-05-26 12:40 | GENERIC | B | SELL | trending_down | +36/53 | +1.00 (-2) | +17.48 (+0) | +0.35 (+1) | +34.84 (-1) | +0.14 (-2) | +0.23 (-1) | +1.95 (-1) | | -1.00 (-1) | -1.03 |
| 2026-06-01 05:20 | XAU_CP (double_bottom) | B | BUY | ranging | +16/38 | -0.50 (+0) | +7.13 (+0) | +0.64 (-1) | +23.13 (+0) | +0.31 (+2) | +0.13 (+1) | +1.25 (+0) | +1.00 (+1) | | -1.04 |
| 2026-06-02 07:00 | GENERIC | B | BUY | trending_up | +29/57 | +1.00 (+2) | +22.04 (+0) | +0.38 (-1) | +55.11 (+1) | +0.05 (+2) | +0.33 (+1) | +2.17 (+1) | +1.00 (-1) | +1.00 (+1) | -1.03 |
| 2026-06-02 22:10 | XAU_CP (inv_hs) | B | BUY | ranging | -6/34 | +0.00 (+0) | +10.43 (-1) | +0.52 (-1) | +12.77 (+0) | +0.11 (+2) | +0.04 (+1) | +1.34 (+0) | +1.00 (-1) | | -1.07 |
| 2026-06-11 17:45 | XAU_ATR | A | BUY | trending_up | +40/63 | +1.00 (+2) | +34.65 (+0) | +0.67 (-1) | +26.06 (+1) | +0.75 (+2) | +2.13 (+2) | +5.33 (+1) | | | -1.03 |
| 2026-06-18 17:55 | XAU_CP (inv_hs) | B | BUY | ranging | +9/30 | +0.00 (+0) | +5.05 (+0) | +0.49 (-1) | +17.20 (+0) | +0.32 (+2) | +0.20 (+0) | +0.94 (+0) | | | -1.05 |
| 2026-06-25 08:40 | XAU_CP (bull_flag) | B | BUY | transitioning | +20/59 | +0.50 (+0) | +15.26 (+0) | +0.49 (-1) | +18.75 (+0) | +0.33 (+2) | +0.60 (+0) | +1.85 (+1) | | +1.00 (+1) | -1.05 |
| 2026-06-26 15:50 | GENERIC | B | BUY | trending_up | +28/57 | +1.00 (+2) | +18.75 (+0) | +0.51 (-1) | +31.65 (+1) | +0.02 (+1) | +0.35 (+1) | +1.79 (+1) | | +0.00 (+0) | -1.03 |
| 2026-06-30 08:05 | XAU_CP (bull_flag) | B | BUY | trending_up | +22/51 | +1.00 (+2) | +20.40 (+0) | +0.41 (-1) | +48.76 (+1) | -0.17 (+0) | +0.25 (+1) | +2.06 (+1) | | | -1.05 |
| 2026-07-01 14:35 | XAU_CP (bull_flag) | B | BUY | trending_up | +24/43 | +1.00 (+2) | +28.07 (+0) | +0.33 (-1) | +61.44 (+1) | +0.21 (+0) | +0.31 (+0) | +3.08 (+1) | | +1.00 (+1) | -1.02 |
| 2026-07-06 07:45 | XAU_CP (inv_hs) | B | BUY | ranging | +16/40 | +0.00 (+0) | +13.57 (-1) | +0.69 (-1) | +23.57 (+0) | +0.30 (+2) | +0.29 (+1) | +1.89 (+1) | | +1.00 (+1) | -1.03 |
| 2026-07-13 18:55 | XAU_CP (double_bottom) | B | BUY | trending | +25/34 | +0.00 (+0) | +7.33 (+0) | +0.46 (-1) | +30.23 (+1) | +0.41 (+2) | +0.33 (+1) | +1.22 (+0) | +1.00 (+1) | | -1.03 |
| 2026-07-22 03:15 | GENERIC | B | BUY | trending_up | +29/54 | +1.00 (+2) | +25.11 (+0) | +0.52 (-1) | +61.73 (+1) | +0.04 (+2) | +0.30 (+0) | +2.48 (+1) | +1.00 (-1) | +0.00 (+0) | -1.03 |
| 2026-07-24 15:40 | GENERIC | B | BUY | trending_up | +28/57 | +1.00 (+2) | +17.48 (+0) | +0.46 (-1) | +36.38 (+1) | +0.20 (+2) | +0.37 (+1) | +1.78 (+1) | +1.00 (-1) | | -1.03 |
| 2026-07-29 06:35 | GENERIC | B | BUY | trending_up | +33/55 | +1.00 (+2) | +18.79 (+0) | +0.51 (-1) | +43.50 (+1) | +0.25 (+2) | +0.11 (+0) | +2.41 (+1) | | +1.00 (+1) | -1.03 |
| 2026-07-30 05:50 | GENERIC | B | SELL | trending_down | +31/55 | +1.00 (-2) | +23.06 (+0) | +0.53 (+1) | +49.98 (-1) | +0.14 (-2) | +0.34 (+0) | +2.33 (-1) | | | -1.03 |
| 2026-07-30 14:05 | XAU_ATR | A | BUY | trending_up | +40/56 | +1.00 (+2) | +19.24 (+0) | +0.50 (-1) | +39.62 (+1) | +0.39 (+2) | +0.76 (+0) | +2.18 (+1) | | | -1.03 |
| 2026-07-30 20:05 | GENERIC | A | BUY | transitioning | +31/64 | +1.00 (+2) | +15.51 (+0) | +0.46 (-1) | +22.45 (+0) | +0.21 (+2) | +0.31 (+1) | +2.10 (+1) | +1.00 (+1) | | -1.03 |
| 2026-08-04 08:05 | XAU_CP (inv_hs) | B | BUY | ranging | +17/39 | +1.00 (+2) | +14.09 (-1) | +0.60 (-1) | +11.51 (+0) | +0.13 (+2) | +0.22 (+1) | +1.70 (+1) | | | -1.06 |
| 2026-08-04 17:05 | GENERIC | A | BUY | trending_up | +42/61 | +1.00 (+2) | +21.70 (+0) | +0.73 (-1) | +25.53 (+1) | +0.26 (+2) | +0.42 (+1) | +3.09 (+1) | | +1.00 (+1) | -1.03 |
| 2026-08-05 06:40 | XAU_CP (bull_flag) | B | BUY | trending_up | +35/50 | +1.00 (+2) | +24.58 (+0) | +0.34 (-1) | +52.31 (+1) | +0.13 (+2) | +0.26 (+0) | +2.69 (+1) | | | -1.04 |
| 2026-08-07 12:40 | GENERIC | B | BUY | trending_up | +29/61 | +1.00 (+2) | +29.17 (+0) | +0.75 (-1) | +28.67 (+1) | +0.34 (+2) | +0.95 (+1) | +4.06 (+1) | +1.00 (-1) | | -0.28 |
| 2026-08-07 15:15 | XAU_CP (inv_hs) | B | BUY | ranging | +10/26 | +1.00 (+2) | +8.65 (+0) | +0.27 (+0) | +11.18 (+0) | +0.00 (+1) | +0.09 (+1) | +1.01 (+0) | | | -1.03 |
| 2026-08-12 03:10 | GENERIC | B | BUY | trending_up | +33/55 | +1.00 (+2) | +24.59 (+0) | +0.42 (-1) | +41.86 (+1) | +0.14 (+2) | +0.27 (+1) | +2.51 (+1) | +1.00 (-1) | +1.00 (+1) | -1.03 |
| 2026-08-12 14:20 | XAU_CP (inv_hs) | B | BUY | ranging | +8/32 | +1.00 (+2) | +11.94 (-1) | +0.38 (-1) | +11.54 (+0) | +0.12 (+2) | +0.40 (+1) | +1.33 (+0) | | | -1.02 |
| 2026-08-19 20:55 | GENERIC | A | BUY | trending_up | +35/56 | +1.00 (+2) | +23.02 (+0) | +0.47 (-1) | +37.89 (+1) | +0.12 (+2) | +0.32 (+1) | +2.48 (+1) | | +1.00 (+1) | -1.03 |
| 2026-08-20 12:35 | GENERIC | B | SELL | trending_down | +33/51 | +1.00 (-2) | +22.78 (+0) | +0.43 (+1) | +34.84 (-1) | +0.25 (-2) | +0.48 (+0) | +2.76 (-1) | | | -1.02 |
| 2026-08-26 15:20 | GENERIC | B | SELL | trending_down | +40/55 | +1.00 (-2) | +20.64 (+0) | +0.49 (+1) | +26.53 (-1) | +0.37 (-2) | +0.56 (+0) | +2.13 (-1) | | | -1.02 |
| 2026-08-31 08:05 | XAU_CP (bull_flag) | B | BUY | trending | +20/37 | +0.50 (+0) | +13.73 (+0) | +0.48 (-1) | +30.30 (+1) | +0.03 (+1) | +0.28 (+1) | +1.38 (+2) | +1.00 (-1) | | -1.06 |
| 2026-08-31 14:25 | GENERIC | B | SELL | trending_down | +30/55 | +1.00 (-2) | +18.49 (+0) | +0.41 (+1) | +36.73 (-1) | +0.13 (-2) | +0.44 (-1) | +1.96 (-1) | -1.00 (+1) | | -1.02 |
| 2026-09-02 08:10 | XAU_CP (inv_hs) | B | BUY | ranging | -14/35 | +0.50 (+0) | +12.23 (-1) | +0.39 (-1) | +19.76 (+0) | -0.05 (+0) | +0.04 (+1) | +1.13 (+0) | +1.00 (-1) | | -1.05 |
| 2026-09-03 08:30 | GENERIC | B | BUY | transitioning | +25/54 | +1.00 (+2) | +10.52 (+0) | +0.42 (-1) | +19.46 (+0) | +0.22 (+2) | +0.24 (+1) | +1.22 (+0) | | +1.00 (+1) | -1.04 |
| 2026-09-04 05:40 | XAU_CP (inv_hs) | B | BUY | ranging | +9/37 | +0.50 (+0) | +7.39 (+0) | +0.51 (-1) | +13.54 (+0) | +0.10 (+2) | +0.07 (+1) | +1.12 (+0) | | | -1.06 |
| 2026-09-04 15:15 | XAU_CP (double_bottom) | B | BUY | ranging | +9/23 | +0.00 (+0) | +9.02 (+0) | +0.48 (-1) | +14.40 (+0) | +0.20 (+2) | +0.28 (+1) | +1.34 (+0) | | | -0.39 |
## 15min: 931 trades — 162 wrong-direction losers (17%), 206 other losers, 563 winners
Wrong direction = a losing trade whose best excursion never reached a quarter of the initial risk.
### 1. Where the wrong-direction trades came from (15min)
| Setup type | Trades | Wrong-direction | Rate | Share of all wrong-direction | Other losers | Winners | Net R |
|---|---|---|---|---|---|---|---|
| GENERIC | 757 | 120 | 16% | 74% | 169 | 468 | +140.5 |
| XAU_ATR | 174 | 42 | 24% | 26% | 37 | 95 | +5.7 |
By grade and by regime (15min):
| Grade | Trades | Wrong-direction rate | | Regime | Trades | Wrong-direction rate |
|---|---|---|---|---|---|---|
| A | 450 | 18% | ranging | 120 | 15% |
| A+ | 19 | 32% | transitioning | 213 | 15% |
| B | 402 | 15% | trending_down | 219 | 21% |
| C | 60 | 20% | trending_up | 379 | 18% |
By direction: BUY: 635 trades, 16% wrong-direction; SELL: 296 trades, 20% wrong-direction.
### 2. The signal itself at entry (15min): wrong-direction losers vs winners
Values signed in the trade's direction where a sign applies (positive = in the trade's favour). The separation column is the probability that a random winner shows a higher value than a random wrong-direction loser: 0.50 means the measure does not separate them at all.
| Measure | Wrong-direction median | Winner median | Other-loser median | Separation |
|---|---|---|---|---|
| Composite score of the primary timeframe (in trade direction) | +36.20 | +35.85 | +36.41 | 0.48 |
| Final score after the multi-timeframe blend (in trade direction) | +33.38 | +32.89 | +33.27 | 0.47 |
| Final confidence | +60.01 | +60.41 | +60.10 | 0.52 |
| Composite score of confirmation 1 (in trade direction) | +28.13 | +28.12 | +26.84 | 0.50 |
| Composite score of confirmation 2 (in trade direction) | +19.83 | +20.82 | +19.74 | 0.51 |
| Indicators agreeing (count) | +5.00 | +5.00 | +5.00 | 0.48 |
| Indicators opposing (count) | +1.00 | +1.00 | +1.00 | 0.54 |
| Confidence penalty (points) | +14.41 | +14.00 | +14.00 | 0.50 |
| Exhaustion tags | +3.00 | +2.00 | +2.00 | 0.49 |
| Primary ATR as % of price | +0.19 | +0.18 | +0.16 | 0.46 |
| Prior 12-bar move in trade direction (ATR) | +2.69 | +2.92 | +2.95 | 0.54 |
| Hourly stretch (ATR) | +1.64 | +1.75 | +1.53 | 0.53 |
| Planned target 1 (R) | +1.00 | +1.00 | +1.00 | 0.54 |
### 3a. Each indicator on the primary timeframe (15min)
Agree = the indicator voted in the trade's direction; oppose = against. Value is signed in the trade's direction (RSI minus 50, Bollinger position minus 0.5, MACD and Keltner in ATR units, EMA alignment, rate of change in percent; ADX is unsigned strength). Contribution = the indicator's weighted score in the trade's favour.
| Indicator | Present (wrong / win) | Agree % (wrong / win) | Oppose % (wrong / win) | Median value (wrong / win) | Median confidence (wrong / win) | Mean contribution (wrong / win) | Separation of value |
|---|---|---|---|---|---|---|---|
| EMA trend (alignment) | 100% / 100% | 95% / 95% | 0% / 1% | +1.00 / +1.00 | 90 / 90 | +2.49 / +2.62 | 0.55 |
| RSI | 100% / 100% | 12% / 9% | 12% / 20% | +18.78 / +20.21 | 30 / 30 | +0.00 / -0.05 | 0.52 |
| Bollinger position | 100% / 100% | 0% / 0% | 94% / 94% | +0.52 / +0.51 | 65 / 65 | -0.59 / -0.63 | 0.51 |
| ADX (trend strength) | 100% / 100% | 68% / 63% | 2% / 1% | +30.09 / +27.91 | 70 / 70 | +0.18 / +0.18 | 0.48 |
| MACD histogram | 100% / 100% | 95% / 92% | 0% / 0% | +0.19 / +0.22 | 70 / 70 | +3.96 / +3.86 | 0.53 |
| Rate of change | 100% / 100% | 83% / 87% | 0% / 0% | +0.47 / +0.47 | 54 / 54 | +1.07 / +1.12 | 0.52 |
| Keltner extension | 100% / 100% | 93% / 93% | 0% / 0% | +2.22 / +2.28 | 65 / 65 | +2.15 / +2.18 | 0.52 |
| Divergence | 31% / 30% | 41% / 34% | 55% / 64% | +1.00 / +1.00 | 70 / 70 | +0.12 / -0.06 | 0.53 |
| Candlestick | 27% / 32% | 81% / 80% | 2% / 4% | +1.00 / +1.00 | 50 / 50 | +0.33 / +0.33 | 0.52 |
### 3b. Each indicator on the first confirmation timeframe
Agree = the indicator voted in the trade's direction; oppose = against. Value is signed in the trade's direction (RSI minus 50, Bollinger position minus 0.5, MACD and Keltner in ATR units, EMA alignment, rate of change in percent; ADX is unsigned strength). Contribution = the indicator's weighted score in the trade's favour.
| Indicator | Present (wrong / win) | Agree % (wrong / win) | Oppose % (wrong / win) | Median value (wrong / win) | Median confidence (wrong / win) | Mean contribution (wrong / win) | Separation of value |
|---|---|---|---|---|---|---|---|
| EMA trend (alignment) | 100% / 100% | 100% / 100% | 0% / 0% | +1.00 / +1.00 | 90 / 90 | +2.97 / +2.99 | 0.51 |
| RSI | 100% / 100% | 15% / 14% | 14% / 13% | +19.07 / +19.39 | 30 / 30 | -0.01 / -0.01 | 0.51 |
| Bollinger position | 100% / 100% | 0% / 0% | 85% / 87% | +0.44 / +0.44 | 45 / 45 | -0.48 / -0.48 | 0.49 |
| ADX (trend strength) | 100% / 100% | 75% / 76% | 1% / 0% | +31.67 / +31.41 | 70 / 70 | +0.21 / +0.21 | 0.50 |
| MACD histogram | 100% / 100% | 59% / 60% | 0% / 1% | +0.22 / +0.19 | 61 / 60 | +2.47 / +2.41 | 0.47 |
| Rate of change | 100% / 100% | 56% / 61% | 1% / 2% | +0.28 / +0.26 | 51 / 51 | +0.68 / +0.69 | 0.47 |
| Keltner extension | 100% / 100% | 79% / 81% | 0% / 0% | +2.06 / +2.08 | 65 / 65 | +1.36 / +1.38 | 0.51 |
| Divergence | 27% / 31% | 19% / 16% | 79% / 83% | +1.00 / +1.00 | 70 / 70 | -0.76 / -0.82 | 0.57 |
| Candlestick | 30% / 28% | 39% / 52% | 29% / 10% | +0.00 / +0.00 | 50 / 50 | +0.04 / +0.18 | 0.56 |
### 3c. Each indicator on the second confirmation timeframe
Agree = the indicator voted in the trade's direction; oppose = against. Value is signed in the trade's direction (RSI minus 50, Bollinger position minus 0.5, MACD and Keltner in ATR units, EMA alignment, rate of change in percent; ADX is unsigned strength). Contribution = the indicator's weighted score in the trade's favour.
| Indicator | Present (wrong / win) | Agree % (wrong / win) | Oppose % (wrong / win) | Median value (wrong / win) | Median confidence (wrong / win) | Mean contribution (wrong / win) | Separation of value |
|---|---|---|---|---|---|---|---|
| EMA trend (alignment) | 100% / 100% | 66% / 76% | 12% / 10% | +0.50 / +1.00 | 65 / 90 | +1.28 / +1.49 | 0.57 |
| RSI | 100% / 100% | 12% / 10% | 23% / 26% | +12.73 / +13.85 | 30 / 30 | -0.12 / -0.10 | 0.54 |
| Bollinger position | 100% / 100% | 0% / 1% | 71% / 72% | +0.42 / +0.43 | 45 / 45 | -0.50 / -0.53 | 0.52 |
| ADX (trend strength) | 100% / 100% | 48% / 49% | 9% / 7% | +26.30 / +26.57 | 55 / 55 | +0.11 / +0.12 | 0.47 |
| MACD histogram | 100% / 100% | 75% / 69% | 2% / 2% | +0.15 / +0.16 | 63 / 63 | +2.69 / +2.49 | 0.50 |
| Rate of change | 100% / 100% | 76% / 73% | 2% / 5% | +0.56 / +0.57 | 55 / 55 | +0.87 / +0.84 | 0.49 |
| Keltner extension | 100% / 100% | 48% / 53% | 0% / 0% | +1.45 / +1.60 | 60 / 65 | +0.87 / +0.99 | 0.54 |
| Divergence | 27% / 30% | 48% / 40% | 52% / 57% | +1.00 / +1.00 | 70 / 70 | +0.06 / -0.18 | 0.57 |
| Candlestick | 31% / 31% | 78% / 75% | 8% / 6% | +1.00 / +1.00 | 50 / 50 | +0.33 / +0.32 | 0.52 |
### 4. When the direction was wrong (15min)
Base rate: 17% of all trades were wrong-direction losers. Cells under 20 trades are noise.
| Entry hour (Coordinated Universal Time) | Trades | Wrong-direction | Rate | | Weekday | Trades | Wrong-direction | Rate |
|---|---|---|---|---|---|---|---|---|
| 03:00 | 74 | 8 | 11% | Mon | 169 | 29 | 17% |
| 04:00 | 28 | 5 | 18% | Tue | 212 | 33 | 16% |
| 05:00 | 61 | 12 | 20% | Wed | 189 | 42 | 22% |
| 06:00 | 79 | 16 | 20% | Thu | 179 | 27 | 15% |
| 07:00 | 68 | 13 | 19% | Fri | 169 | 29 | 17% |
| 08:00 | 77 | 13 | 17% | Sun | 13 | 2 | 15% |
| 09:00 | 75 | 12 | 16% | | | | |
| 12:00 | 68 | 11 | 16% | | | | |
| 14:00 | 154 | 25 | 16% | | | | |
| 15:00 | 65 | 9 | 14% | | | | |
| 17:00 | 34 | 5 | 15% | | | | |
| 18:00 | 45 | 12 | 27% | | | | |
| 19:00 | 38 | 6 | 16% | | | | |
| 20:00 | 34 | 8 | 24% | | | | |
| 22:00 | 31 | 7 | 23% | | | | |
Weekday-hour cells with at least 10 trades and the highest wrong-direction rate (15min):
| Day | Hour | Trades | Wrong-direction | Rate |
|---|---|---|---|---|
| Wed | 18:00 | 13 | 5 | 38% |
| Tue | 05:00 | 19 | 7 | 37% |
| Tue | 06:00 | 13 | 4 | 31% |
| Wed | 09:00 | 20 | 6 | 30% |
| Wed | 19:00 | 10 | 3 | 30% |
| Fri | 15:00 | 18 | 5 | 28% |
Months with the highest wrong-direction rate: 2026-07 29% (11 of 38); 2026-02 28% (8 of 29); 2025-07 24% (11 of 46); 2025-04 24% (10 of 42).
### Appendix 15min: every wrong-direction loser with its primary-timeframe indicator values
Columns: entry time, setup (pattern), grade, direction, regime, final score in trade direction / confidence, then per indicator `signed value (vote)` where the vote is −2 strong sell … +2 strong buy; blank = the indicator did not vote. Values as defined in section 3. R = realised result.
| Entry | Setup | Grade | Dir | Regime | Score/Conf | EMA | RSI | Bollinger | ADX | MACD | ROC | Keltner | Div | Candle | R |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2024-09-11 07:00 | GENERIC | A | BUY | trending_up | +44/65 | +1.00 (+2) | +26.75 (+0) | +0.59 (-1) | +32.23 (+1) | +0.27 (+2) | +0.29 (+1) | +2.40 (+1) | | | -1.06 |
| 2024-09-19 09:30 | GENERIC | A | BUY | trending_up | +37/60 | +1.00 (+2) | +29.79 (+0) | +0.46 (-1) | +39.62 (+1) | +0.26 (+2) | +0.58 (+1) | +3.55 (+1) | +1.00 (-1) | | -1.03 |
| 2024-09-24 05:00 | GENERIC | A | BUY | trending_up | +40/64 | +1.00 (+2) | +25.13 (+0) | +0.51 (-1) | +26.94 (+1) | +0.43 (+2) | +0.40 (+1) | +3.08 (+1) | | | -1.05 |
| 2024-09-25 14:30 | GENERIC | B | BUY | transitioning | +30/66 | +1.00 (+2) | +16.25 (+0) | +0.50 (-1) | +17.01 (+0) | +0.23 (+2) | +0.32 (+1) | +1.58 (+1) | | | -1.07 |
| 2024-09-30 19:15 | GENERIC | B | SELL | trending_down | +32/57 | +1.00 (-2) | +11.98 (-1) | +0.38 (+1) | +27.91 (-1) | +0.09 (-2) | +0.35 (-1) | +1.44 (+0) | -1.00 (+1) | | -1.06 |
| 2024-10-01 17:15 | GENERIC | A | BUY | trending_up | +39/56 | +1.00 (+2) | +16.01 (+0) | +0.40 (-1) | +31.70 (+1) | +0.06 (+2) | +0.31 (+1) | +1.69 (+1) | | | -1.03 |
| 2024-10-02 14:15 | GENERIC | C | BUY | ranging | +35/48 | +0.50 (+1) | +12.27 (-1) | +0.54 (-1) | +21.45 (+0) | +0.29 (+2) | +0.35 (+1) | +1.60 (+2) | +1.00 (+1) | | -1.06 |
| 2024-10-14 06:30 | GENERIC | A | BUY | trending_up | +41/59 | +1.00 (+2) | +24.92 (+0) | +0.50 (-1) | +30.45 (+1) | +0.27 (+2) | +0.38 (+1) | +3.19 (+1) | | | -1.05 |
| 2024-10-23 18:15 | GENERIC | B | SELL | trending_down | +30/53 | +1.00 (-2) | +18.28 (+0) | +0.23 (+0) | +48.61 (-1) | -0.08 (+0) | +0.18 (-1) | +1.91 (-1) | | | -1.05 |
| 2024-10-29 05:15 | GENERIC | A | BUY | trending_up | +35/59 | +1.00 (+2) | +16.86 (+0) | +0.34 (-1) | +36.62 (+1) | +0.04 (+1) | +0.11 (+1) | +2.05 (+1) | | +1.00 (+1) | -1.06 |
| 2024-11-05 15:00 | GENERIC | B | BUY | transitioning | +27/49 | +0.50 (+1) | +12.55 (+0) | +0.44 (-1) | +15.51 (+0) | +0.16 (+2) | +0.24 (+1) | +1.33 (+0) | | +1.00 (+1) | -1.05 |
| 2024-12-03 14:15 | GENERIC | B | BUY | transitioning | +28/57 | +1.00 (+2) | +17.30 (+0) | +0.58 (-1) | +18.00 (+0) | +0.29 (+2) | +0.30 (+0) | +2.12 (+1) | | +0.00 (+0) | -1.05 |
| 2024-12-18 20:45 | GENERIC | A | SELL | trending_down | +36/64 | +1.00 (-2) | +34.41 (+0) | +0.60 (+1) | +46.37 (-1) | +0.70 (-2) | +1.81 (-1) | +5.26 (-1) | -1.00 (+1) | | -0.51 |
| 2024-12-23 06:45 | GENERIC | C | BUY | ranging | +26/69 | +1.00 (+2) | +21.04 (-1) | +0.73 (-1) | +13.63 (+0) | +0.20 (+2) | +0.28 (+1) | +2.33 (+1) | +1.00 (+1) | +1.00 (+1) | -1.06 |
| 2025-01-06 07:45 | GENERIC | A | SELL | trending_down | +40/57 | +1.00 (-2) | +28.81 (+0) | +0.53 (+1) | +38.88 (-1) | +0.26 (-2) | +0.35 (-1) | +3.23 (-1) | | -1.00 (-1) | -1.03 |
| 2025-01-16 17:15 | GENERIC | A | BUY | trending_up | +33/57 | +1.00 (+2) | +19.76 (+0) | +0.32 (-1) | +28.49 (+1) | +0.06 (+2) | +0.18 (+1) | +1.71 (+1) | +1.00 (-1) | | -1.05 |
| 2025-01-22 09:30 | GENERIC | C | BUY | ranging | +25/61 | +1.00 (+2) | +20.25 (-1) | +0.68 (-1) | +19.66 (+0) | +0.21 (+2) | +0.54 (+1) | +2.89 (+1) | | +1.00 (+1) | -1.04 |
| 2025-01-24 15:30 | GENERIC | A | BUY | trending_up | +41/60 | +1.00 (+2) | +14.43 (+1) | +0.49 (-1) | +29.77 (+1) | +0.05 (+1) | +0.23 (+1) | +1.55 (+2) | | | -1.36 |
| 2025-01-29 19:30 | GENERIC | A | SELL | trending_down | +38/60 | +1.00 (-2) | +20.97 (+0) | +0.57 (+1) | +35.32 (-1) | +0.13 (-2) | +0.21 (-1) | +2.05 (-1) | | | -1.05 |
| 2025-02-14 18:15 | GENERIC | B | SELL | trending_down | +29/52 | +1.00 (-2) | +28.07 (+0) | +0.37 (+1) | +58.15 (-1) | +0.27 (+0) | +0.72 (-1) | +3.07 (-1) | | | -1.01 |
| 2025-02-19 18:15 | GENERIC | A | SELL | trending_down | +33/57 | +0.50 (-1) | +17.52 (+0) | +0.46 (+1) | +40.68 (-1) | +0.23 (-2) | +0.51 (-1) | +2.04 (-1) | | -1.00 (-1) | -1.05 |
| 2025-02-20 08:30 | GENERIC | B | BUY | trending_up | +31/59 | +1.00 (+2) | +20.59 (+0) | +0.46 (-1) | +36.49 (+1) | +0.19 (+2) | +0.21 (+0) | +2.18 (+1) | +1.00 (-1) | | -1.04 |
| 2025-02-20 14:15 | GENERIC | A | SELL | trending_down | +34/60 | +0.50 (-1) | +29.24 (+0) | +0.75 (+1) | +33.74 (-1) | +0.53 (-2) | +0.82 (-1) | +3.90 (-1) | -1.00 (-1) | | -1.03 |
| 2025-02-21 08:45 | GENERIC | A | SELL | trending_down | +39/60 | +1.00 (-2) | +15.23 (+0) | +0.52 (+1) | +28.31 (-1) | +0.14 (-2) | +0.24 (+0) | +1.62 (-2) | | | -1.04 |
| 2025-02-26 14:15 | GENERIC | A+ | SELL | transitioning | +46/69 | +1.00 (-2) | +24.52 (+1) | +0.95 (+1) | +15.41 (+0) | +0.31 (-2) | +0.64 (-1) | +3.45 (-2) | | | -1.03 |
| 2025-03-10 18:00 | GENERIC | A | SELL | trending_down | +41/62 | +1.00 (-2) | +20.95 (+0) | +0.59 (+1) | +30.43 (-1) | +0.36 (-2) | +0.76 (-1) | +2.44 (-1) | | +0.00 (+0) | -1.02 |
| 2025-03-13 03:15 | GENERIC | B | BUY | trending_up | +33/62 | +1.00 (+2) | +18.58 (+0) | +0.54 (-1) | +38.42 (+1) | +0.12 (+2) | +0.22 (+1) | +1.96 (+1) | +1.00 (-1) | +0.00 (+0) | -1.06 |
| 2025-03-14 04:30 | GENERIC | A | BUY | trending_up | +36/64 | +1.00 (+2) | +10.74 (+1) | +0.38 (-1) | +28.65 (+1) | +0.03 (+2) | +0.11 (+0) | +1.10 (+2) | +1.00 (+0) | | -1.07 |
| 2025-03-21 14:15 | GENERIC | A | SELL | transitioning | +32/66 | +1.00 (-2) | +31.05 (+1) | +0.69 (+1) | +24.00 (+0) | +0.60 (-2) | +1.02 (-1) | +4.30 (-1) | -1.00 (-1) | | -0.48 |
| 2025-03-24 18:15 | GENERIC | A | SELL | trending_down | +43/59 | +1.00 (-2) | +22.03 (+0) | +0.41 (+1) | +36.90 (-1) | +0.15 (-2) | +0.37 (-1) | +2.30 (-1) | | | -1.04 |
| 2025-03-26 07:45 | GENERIC | A | BUY | trending_up | +38/61 | +1.00 (+2) | +20.83 (+0) | +0.43 (-1) | +25.98 (+1) | +0.46 (+2) | +0.46 (+1) | +2.52 (+1) | | +2.00 (+1) | -1.05 |
| 2025-03-28 07:00 | GENERIC | A | BUY | trending_up | +42/60 | +1.00 (+2) | +27.74 (+0) | +0.58 (-1) | +58.16 (+1) | +0.13 (+2) | +0.35 (+1) | +2.86 (+1) | | +1.00 (+1) | -1.04 |
| 2025-03-31 06:15 | GENERIC | A | BUY | trending_up | +45/58 | +1.00 (+2) | +31.08 (+0) | +0.55 (-1) | +38.63 (+1) | +0.15 (+2) | +0.56 (+1) | +3.07 (+1) | | | -1.03 |
| 2025-04-02 08:45 | GENERIC | A | BUY | trending_up | +36/59 | +1.00 (+2) | +16.39 (+0) | +0.60 (-1) | +28.22 (+1) | +0.35 (+2) | +0.56 (+1) | +2.31 (+1) | | | -1.03 |
| 2025-04-04 08:30 | GENERIC | A | SELL | transitioning | +35/70 | +1.00 (-2) | +20.71 (+1) | +0.63 (+1) | +23.21 (+0) | +0.26 (-2) | +0.78 (-1) | +2.31 (-1) | | | -1.02 |
| 2025-04-08 06:45 | XAU_ATR | A | BUY | transitioning | +33/61 | +0.50 (+1) | +17.12 (+0) | +0.62 (-1) | +19.34 (+0) | +0.17 (+2) | +0.50 (+0) | +2.00 (+2) | | | -1.05 |
| 2025-04-11 15:30 | GENERIC | C | BUY | trending_up | +26/54 | +1.00 (+2) | +10.11 (+1) | +0.38 (-1) | +25.55 (+1) | +0.04 (+2) | +0.17 (+0) | +1.13 (+0) | +1.00 (-1) | | -1.02 |
| 2025-04-15 05:00 | GENERIC | A | BUY | trending_up | +40/60 | +1.00 (+2) | +21.62 (+0) | +0.41 (-1) | +38.85 (+1) | +0.18 (+2) | +0.33 (+1) | +2.21 (+1) | | | -1.05 |
| 2025-04-16 22:15 | GENERIC | B | BUY | trending_up | +29/60 | +1.00 (+2) | +26.69 (+0) | +0.69 (-1) | +47.37 (+1) | +0.06 (+1) | +0.48 (+1) | +3.24 (+1) | +1.00 (-1) | | -1.03 |
| 2025-04-22 06:00 | GENERIC | B | BUY | trending_up | +33/53 | +1.00 (+2) | +22.70 (+0) | +0.31 (-1) | +33.83 (+1) | +0.00 (+1) | +0.16 (+0) | +2.49 (+1) | | | -1.01 |
| 2025-04-23 14:15 | XAU_ATR | A | SELL | trending_down | +41/64 | +1.00 (-2) | +23.94 (+0) | +0.63 (+1) | +29.93 (-1) | +0.41 (-2) | +1.94 (-1) | +3.07 (-1) | | | -1.02 |
| 2025-04-27 22:15 | GENERIC | A | BUY | trending_up | +34/60 | +0.50 (+1) | +26.61 (+0) | +0.59 (-1) | +32.73 (+1) | +0.54 (+2) | +1.07 (+1) | +3.67 (+1) | | +0.00 (+0) | -1.02 |
| 2025-04-28 19:00 | GENERIC | B | BUY | trending_up | +31/57 | +1.00 (+2) | +25.89 (+0) | +0.36 (-1) | +33.20 (+1) | +0.07 (+2) | +0.44 (+1) | +2.60 (+1) | +1.00 (-1) | +1.00 (+1) | -1.02 |
| 2025-05-06 20:00 | GENERIC | A | BUY | trending_up | +34/56 | +1.00 (+2) | +22.74 (+0) | +0.39 (-1) | +33.89 (+1) | +0.01 (+1) | +0.24 (+1) | +2.47 (+1) | | | -1.01 |
| 2025-05-13 07:00 | GENERIC | B | BUY | trending_up | +32/53 | +0.50 (+1) | +15.52 (+0) | +0.38 (-1) | +29.53 (+1) | +0.23 (+2) | +0.73 (+1) | +2.14 (+1) | | | -1.03 |
| 2025-05-15 06:00 | GENERIC | A | SELL | trending_down | +42/57 | +1.00 (-2) | +30.34 (+0) | +0.40 (+1) | +46.37 (-1) | +0.28 (-2) | +0.79 (-1) | +3.35 (-1) | | | -1.01 |
| 2025-05-15 22:15 | GENERIC | B | BUY | trending_up | +32/60 | +1.00 (+2) | +28.60 (+0) | +0.58 (-1) | +52.87 (+1) | +0.04 (+1) | +0.61 (+1) | +3.44 (+1) | | | -1.02 |
| 2025-05-21 06:00 | GENERIC | B | BUY | trending_up | +31/57 | +1.00 (+2) | +18.38 (+0) | +0.55 (-1) | +33.47 (+1) | +0.07 (+1) | +0.26 (+0) | +2.05 (+1) | +1.00 (+0) | | -1.03 |
| 2025-05-21 14:15 | GENERIC | B | SELL | ranging | +30/59 | -0.00 (+0) | +8.56 (+0) | +0.41 (+1) | +19.48 (+0) | +0.13 (-2) | +0.27 (-1) | +0.95 (-2) | | | -1.04 |
| 2025-05-23 15:00 | GENERIC | B | BUY | trending_up | +34/54 | +1.00 (+2) | +17.55 (+0) | +0.36 (-1) | +32.24 (+1) | +0.02 (+1) | +0.35 (+1) | +1.67 (+1) | | | -1.02 |
| 2025-05-28 08:15 | GENERIC | A | BUY | transitioning | +33/59 | +0.50 (+1) | +18.88 (+0) | +0.55 (-1) | +18.80 (+0) | +0.40 (+2) | +0.43 (+1) | +2.69 (+1) | | | -1.02 |
| 2025-06-03 14:15 | GENERIC | A | SELL | trending_down | +42/62 | +0.50 (-1) | +22.91 (+0) | +0.83 (+1) | +30.59 (-1) | +0.21 (-2) | +0.57 (-1) | +2.56 (-2) | | | -1.02 |
| 2025-06-04 09:00 | GENERIC | B | BUY | ranging | +30/58 | +1.00 (+2) | +7.95 (+0) | +0.48 (-1) | +19.78 (+0) | +0.12 (+2) | +0.21 (+0) | +0.99 (+2) | | | -1.04 |
| 2025-06-05 12:15 | GENERIC | B | BUY | trending_up | +31/53 | +1.00 (+2) | +14.88 (+1) | +0.23 (+0) | +37.61 (+1) | +0.11 (+0) | +0.31 (+1) | +1.55 (+1) | +1.00 (+1) | +1.00 (+1) | -1.02 |
| 2025-06-11 12:45 | GENERIC | B | BUY | trending_up | +30/61 | +1.00 (+2) | +22.31 (+0) | +0.60 (-1) | +28.25 (+1) | +0.21 (+2) | +0.60 (+1) | +2.69 (+1) | +1.00 (-1) | | -1.02 |
| 2025-06-19 06:30 | XAU_ATR | A+ | SELL | trending_down | +44/66 | +1.00 (-2) | +29.47 (+0) | +0.77 (+1) | +25.08 (-1) | +0.43 (-2) | +0.69 (-1) | +3.67 (-1) | | | -1.06 |
| 2025-06-19 17:30 | GENERIC | B | BUY | ranging | +28/71 | +0.00 (+0) | +8.99 (+0) | +0.50 (-1) | +13.98 (+0) | +0.19 (+2) | +0.27 (+1) | +1.00 (+2) | +1.00 (+1) | | -1.05 |
| 2025-06-20 03:15 | GENERIC | A | SELL | trending_down | +44/65 | +1.00 (-2) | +23.87 (+0) | +0.64 (+1) | +25.94 (-1) | +0.32 (-2) | +0.58 (-1) | +2.77 (-1) | | | -1.31 |
| 2025-07-02 14:15 | GENERIC | B | SELL | ranging | +28/65 | -0.00 (+0) | +11.39 (+1) | +0.60 (+1) | +18.62 (+0) | +0.20 (-2) | +0.23 (-1) | +1.39 (-2) | -1.00 (-1) | | -1.05 |
| 2025-07-02 22:30 | GENERIC | A | BUY | trending_up | +42/60 | +1.00 (+2) | +27.17 (+0) | +0.56 (-1) | +33.53 (+1) | +0.17 (+2) | +0.29 (+1) | +3.19 (+1) | | | -1.05 |
| 2025-07-03 07:00 | GENERIC | A | BUY | transitioning | +33/68 | +1.00 (+2) | +18.01 (+0) | +0.53 (-1) | +22.58 (+0) | +0.36 (+2) | +0.37 (+1) | +2.31 (+1) | | +2.00 (+1) | -1.07 |
| 2025-07-09 09:00 | GENERIC | A | SELL | trending_down | +42/61 | +1.00 (-2) | +14.97 (-1) | +0.46 (+1) | +42.70 (-1) | +0.05 (-2) | +0.07 (-1) | +1.32 (-2) | -1.00 (+1) | | -1.04 |
| 2025-07-13 22:45 | GENERIC | B | BUY | transitioning | +25/63 | +1.00 (+2) | +18.21 (+0) | +0.57 (-1) | +23.10 (+0) | +0.28 (+2) | +0.37 (+0) | +2.40 (+1) | | -1.00 (-1) | -1.03 |
| 2025-07-14 14:45 | GENERIC | A | SELL | trending_down | +34/56 | +0.50 (-1) | +22.15 (+0) | +0.51 (+1) | +30.45 (-1) | +0.34 (-2) | +0.60 (-1) | +3.02 (-1) | | -1.00 (-1) | -1.03 |
| 2025-07-16 15:45 | GENERIC | B | BUY | trending_up | +28/54 | +0.50 (+1) | +23.53 (+0) | +0.59 (-1) | +26.48 (+1) | +0.40 (+2) | +0.78 (+0) | +2.46 (+1) | | | -1.02 |
| 2025-07-17 12:45 | GENERIC | B | SELL | trending_down | +31/62 | +1.00 (-2) | +21.53 (+0) | +0.92 (+1) | +32.72 (-1) | +0.13 (-1) | +0.43 (-1) | +3.02 (-2) | -1.00 (+1) | -1.00 (-1) | -1.03 |
| 2025-07-28 04:00 | GENERIC | B | BUY | ranging | +31/66 | +0.00 (+0) | +13.35 (-1) | +0.72 (-1) | +15.96 (+0) | +0.25 (+2) | +0.24 (+1) | +1.83 (+2) | | +1.00 (+1) | -1.04 |
| 2025-07-30 09:00 | GENERIC | B | BUY | transitioning | +44/46 | +0.50 (+1) | +9.33 (+0) | +0.50 (-1) | +20.43 (+0) | +0.16 (+2) | +0.26 (+1) | +1.12 (+2) | | +1.00 (+1) | -1.07 |
| 2025-07-31 09:15 | GENERIC | B | BUY | trending_up | +31/58 | +0.50 (+1) | +24.94 (+0) | +0.48 (-1) | +53.19 (+1) | +0.18 (+2) | +0.36 (+1) | +2.05 (+1) | | | -1.03 |
| 2025-08-01 08:00 | GENERIC | B | SELL | ranging | +35/51 | +1.00 (-2) | +12.21 (+1) | +0.59 (+1) | +16.74 (+0) | +0.19 (-2) | +0.19 (-1) | +1.30 (-2) | -1.00 (-1) | | -1.06 |
| 2025-08-12 12:30 | GENERIC | B | SELL | transitioning | +26/56 | +1.00 (-2) | +12.67 (+0) | +0.52 (+1) | +22.61 (+0) | +0.11 (-2) | +0.18 (-1) | +1.49 (+0) | | | -1.04 |
| 2025-08-25 15:00 | GENERIC | B | BUY | transitioning | +28/56 | +1.00 (+2) | +13.30 (+0) | +0.47 (-1) | +15.91 (+0) | +0.28 (+2) | +0.16 (+0) | +1.81 (+1) | | +0.00 (+0) | -1.05 |
| 2025-09-02 04:45 | GENERIC | C | BUY | trending_up | +26/51 | +1.00 (+2) | +14.00 (+1) | +0.23 (+0) | +45.04 (+1) | -0.05 (+0) | +0.11 (+1) | +1.35 (+0) | | | -1.05 |
| 2025-09-03 18:15 | GENERIC | A | BUY | trending_up | +42/57 | +1.00 (+2) | +27.13 (+0) | +0.45 (-1) | +40.58 (+1) | +0.11 (+2) | +0.40 (+1) | +2.74 (+1) | | +1.00 (+1) | -1.02 |
| 2025-09-04 03:15 | GENERIC | A | SELL | trending_down | +37/61 | +0.50 (-1) | +31.86 (+0) | +0.75 (+1) | +41.07 (-1) | +0.51 (-2) | +1.20 (-1) | +4.49 (-1) | | | -1.01 |
| 2025-09-09 05:45 | GENERIC | B | BUY | trending_up | +30/59 | +1.00 (+2) | +20.85 (+0) | +0.38 (-1) | +31.37 (+1) | +0.10 (+2) | +0.24 (+1) | +1.90 (+1) | +1.00 (-1) | +1.00 (+1) | -1.04 |
| 2025-09-12 05:15 | GENERIC | B | BUY | trending_up | +29/52 | +1.00 (+2) | +27.41 (+0) | +0.30 (-1) | +42.45 (+1) | +0.15 (+0) | +0.23 (+1) | +2.42 (+1) | | | -1.03 |
| 2025-09-12 14:30 | GENERIC | B | BUY | ranging | +31/62 | +1.00 (+2) | +9.10 (+0) | +0.35 (-1) | +18.60 (+0) | +0.17 (+2) | +0.16 (+1) | +0.97 (+2) | +1.00 (+1) | | -1.03 |
| 2025-09-17 19:00 | XAU_ATR | A | SELL | trending_down | +40/62 | +0.50 (-1) | +27.64 (+0) | +0.90 (+1) | +35.59 (-1) | +0.36 (-2) | +1.02 (-1) | +2.84 (-2) | -1.00 (-1) | | -1.03 |
| 2025-09-23 09:15 | GENERIC | A | BUY | trending_up | +35/61 | +1.00 (+2) | +30.56 (+0) | +0.74 (-1) | +26.20 (+1) | +0.50 (+2) | +0.84 (+1) | +4.62 (+1) | +1.00 (-1) | +1.00 (+1) | -1.02 |
| 2025-09-24 19:00 | GENERIC | B | SELL | trending_down | +32/55 | +0.50 (-1) | +33.01 (+0) | +0.41 (+1) | +60.16 (-1) | +0.20 (-2) | +0.48 (-1) | +2.97 (-1) | | | -1.01 |
| 2025-09-25 08:15 | GENERIC | A | BUY | transitioning | +33/59 | +0.50 (+1) | +19.02 (+0) | +0.63 (-1) | +21.43 (+0) | +0.35 (+2) | +0.53 (+1) | +2.49 (+1) | | | -1.03 |
| 2025-09-30 05:00 | GENERIC | B | BUY | trending_up | +28/52 | +1.00 (+2) | +31.25 (+0) | +0.40 (-1) | +46.47 (+1) | +0.26 (+0) | +0.55 (+1) | +2.97 (+1) | | | -1.02 |
| 2025-10-01 17:30 | GENERIC | C | BUY | trending_up | +26/46 | +0.50 (+1) | -1.86 (+2) | +0.04 (+0) | +30.25 (-1) | +0.04 (+2) | -0.06 (+0) | -0.09 (+0) | +1.00 (+1) | | -1.05 |
| 2025-10-14 05:00 | GENERIC | B | BUY | trending_up | +28/51 | +1.00 (+2) | +24.63 (+0) | +0.36 (-1) | +46.45 (+1) | +0.12 (+0) | +0.62 (+1) | +2.48 (+1) | | +1.00 (+1) | -1.01 |
| 2025-10-15 08:15 | GENERIC | A | BUY | trending_up | +36/65 | +1.00 (+2) | +23.55 (+0) | +0.56 (-1) | +33.49 (+1) | +0.17 (+2) | +0.58 (+1) | +2.59 (+1) | +1.00 (-1) | | -1.02 |
| 2025-10-17 04:15 | XAU_ATR | A | BUY | trending_up | +33/65 | +1.00 (+2) | +13.10 (+1) | +0.27 (+0) | +26.90 (+1) | +0.08 (+2) | +1.87 (+1) | +1.54 (+1) | +1.00 (-1) | | -1.02 |
| 2025-10-17 14:15 | XAU_ATR | A | SELL | trending_down | +44/62 | +0.50 (-1) | +26.73 (+0) | +0.82 (+1) | +57.50 (-1) | +0.32 (-2) | +2.09 (-2) | +3.86 (-1) | | -1.00 (-1) | -1.01 |
| 2025-10-20 20:00 | GENERIC | B | BUY | trending_up | +31/60 | +1.00 (+2) | +31.72 (+0) | +0.63 (-1) | +39.31 (+1) | +0.02 (+1) | +0.82 (+1) | +3.12 (+1) | +1.00 (-1) | | -1.01 |
| 2025-11-03 08:00 | GENERIC | B | BUY | ranging | +31/68 | +1.00 (+2) | +18.98 (-1) | +0.55 (-1) | +19.00 (+0) | +0.12 (+2) | +0.33 (+0) | +1.64 (+2) | | | -1.02 |
| 2025-11-11 03:15 | GENERIC | A | BUY | trending_up | +36/59 | +1.00 (+2) | +29.76 (+0) | +0.41 (-1) | +42.80 (+1) | +0.07 (+2) | +0.34 (+1) | +2.39 (+1) | +1.00 (-1) | +1.00 (+1) | -1.02 |
| 2025-11-14 03:30 | GENERIC | B | BUY | transitioning | +28/61 | +0.50 (+1) | +17.19 (+0) | +0.49 (-1) | +22.45 (+0) | +0.41 (+2) | +0.64 (+1) | +2.15 (+1) | | +1.00 (+1) | -1.02 |
| 2025-11-20 08:30 | XAU_ATR | A+ | SELL | trending_down | +49/73 | +1.00 (-2) | +17.81 (+0) | +0.61 (+1) | +25.46 (-1) | +0.18 (-2) | +0.51 (-1) | +1.83 (-2) | | | -1.03 |
| 2025-12-02 09:45 | GENERIC | B | SELL | trending_down | +30/59 | +0.50 (-1) | +23.37 (+0) | +0.66 (+1) | +35.83 (-1) | +0.25 (-2) | +0.55 (-1) | +2.63 (-1) | | -1.00 (-1) | -1.02 |
| 2025-12-08 06:00 | GENERIC | C | BUY | ranging | +27/64 | +0.00 (+0) | +13.04 (-1) | +0.58 (-1) | +17.25 (+0) | +0.19 (+2) | +0.17 (+1) | +1.53 (+2) | | | -1.04 |
| 2025-12-30 12:45 | GENERIC | B | BUY | trending_up | +32/56 | +0.50 (+1) | +18.68 (+0) | +0.45 (-1) | +28.35 (+1) | +0.20 (+2) | +0.54 (+1) | +2.12 (+1) | | +1.00 (+1) | -1.02 |
| 2025-12-31 06:15 | XAU_ATR | A | SELL | trending_down | +40/59 | +1.00 (-2) | +17.08 (+0) | +0.54 (+1) | +29.28 (-1) | +0.38 (-2) | +1.21 (+0) | +2.39 (-1) | -1.00 (-1) | | -1.02 |
| 2025-12-31 09:00 | XAU_ATR | A | SELL | trending_down | +37/63 | +1.00 (-2) | +11.79 (-1) | +0.28 (+0) | +27.94 (-1) | +0.01 (-1) | +0.00 (-1) | +1.54 (-1) | | | -1.03 |
| 2026-01-02 18:15 | XAU_ATR | A | SELL | trending_down | +40/59 | +1.00 (-2) | +24.54 (+0) | +0.36 (+1) | +49.77 (-1) | +0.13 (-2) | +0.73 (-1) | +2.49 (-1) | -1.00 (-1) | | -2.02 |
| 2026-01-21 06:30 | XAU_ATR | A | BUY | trending_up | +42/59 | +1.00 (+2) | +35.63 (+0) | +0.46 (-1) | +60.02 (+1) | +0.05 (+2) | +0.80 (+1) | +2.99 (+1) | | | -1.03 |
| 2026-01-21 09:30 | GENERIC | C | BUY | trending_up | +26/55 | +1.00 (+2) | +13.96 (+1) | +0.32 (-1) | +36.84 (+1) | -0.14 (+0) | +0.58 (+1) | +1.55 (+1) | | | -1.01 |
| 2026-01-21 20:15 | GENERIC | B | SELL | trending_down | +29/53 | +0.50 (-1) | +13.16 (-1) | +0.41 (+1) | +33.57 (-1) | +0.24 (-2) | +0.47 (+0) | +1.71 (-1) | | | -1.01 |
| 2026-01-22 07:30 | XAU_ATR | A | BUY | trending_up | +42/63 | +1.00 (+2) | +17.98 (+0) | +0.48 (-1) | +25.34 (+1) | +0.47 (+2) | +0.83 (+1) | +2.47 (+1) | | | -1.03 |
| 2026-01-27 07:45 | GENERIC | C | BUY | ranging | +28/74 | +1.00 (+2) | +13.95 (-1) | +0.47 (-1) | +14.32 (+0) | +0.11 (+2) | +0.58 (+1) | +1.25 (+2) | | | -1.01 |
| 2026-02-02 06:45 | XAU_ATR | A+ | SELL | trending_down | +50/69 | +1.00 (-2) | +27.60 (+0) | +0.59 (+1) | +41.91 (-1) | +0.25 (-2) | +4.50 (-2) | +2.71 (-1) | | | -1.00 |
| 2026-02-12 05:30 | GENERIC | C | BUY | trending_up | +30/46 | +0.50 (+1) | +4.16 (+2) | +0.37 (-1) | +30.27 (+1) | +0.16 (+2) | +0.11 (+0) | +0.57 (+0) | +1.00 (+1) | | -1.03 |
| 2026-02-13 19:15 | GENERIC | B | BUY | transitioning | +26/60 | +0.50 (+1) | +15.67 (+0) | +0.43 (-1) | +17.03 (+0) | +0.09 (+2) | +0.45 (+1) | +1.65 (+1) | | | -1.01 |
| 2026-02-19 07:15 | GENERIC | A | BUY | trending_up | +35/58 | +1.00 (+2) | +31.23 (+0) | +0.49 (-1) | +31.11 (+1) | +0.54 (+2) | +0.74 (+0) | +3.54 (+1) | | | -1.01 |
| 2026-02-20 09:30 | XAU_ATR | A | BUY | transitioning | +31/64 | +1.00 (+2) | +15.12 (+0) | +0.40 (-1) | +23.97 (+0) | +0.19 (+2) | +0.27 (+1) | +1.77 (+1) | | | -1.03 |
| 2026-02-24 14:15 | XAU_ATR | A | SELL | trending_down | +38/61 | +0.50 (-1) | +30.33 (+0) | +0.73 (+1) | +36.23 (-1) | +0.42 (-2) | +1.41 (-1) | +3.53 (-1) | | | -1.02 |
| 2026-02-24 20:45 | GENERIC | C | BUY | ranging | +26/77 | +0.50 (+1) | +11.52 (-1) | +0.52 (-1) | +19.92 (+0) | +0.15 (+2) | +0.15 (+1) | +1.40 (+2) | | | -1.03 |
| 2026-02-26 20:15 | XAU_ATR | A | BUY | trending_up | +34/58 | +1.00 (+2) | +13.52 (+1) | +0.43 (-1) | +26.99 (-1) | +0.18 (+2) | +0.43 (+1) | +1.44 (+0) | | +1.00 (+1) | -1.03 |
| 2026-03-03 06:15 | XAU_ATR | A | SELL | transitioning | +31/71 | -0.00 (+0) | +23.82 (+1) | +0.84 (+1) | +20.07 (+0) | +0.45 (-2) | +1.48 (-1) | +3.29 (-1) | -1.00 (-1) | | -1.02 |
| 2026-03-04 18:15 | GENERIC | B | SELL | transitioning | +25/54 | +1.00 (-2) | +14.30 (+0) | +0.41 (+1) | +18.17 (+0) | +0.11 (-2) | +0.37 (-1) | +1.38 (+0) | | | -1.01 |
| 2026-03-06 15:30 | GENERIC | B | BUY | trending_up | +32/60 | +0.50 (+1) | +15.62 (+0) | +0.68 (-1) | +28.45 (+1) | +0.37 (+2) | +1.55 (+1) | +2.38 (+1) | +1.00 (+1) | | -1.01 |
| 2026-03-06 18:45 | GENERIC | B | BUY | trending_up | +29/58 | +0.50 (+1) | +15.43 (+0) | +0.34 (-1) | +28.83 (+1) | +0.07 (+2) | +0.41 (+1) | +1.70 (+1) | +1.00 (+1) | +1.00 (+1) | -1.01 |
| 2026-03-11 20:00 | XAU_ATR | A | BUY | trending_up | +38/70 | +0.50 (+1) | +5.36 (+1) | +0.55 (-1) | +36.85 (-1) | +0.19 (+2) | +0.22 (+1) | +0.80 (+2) | +1.00 (+1) | | -1.04 |
| 2026-03-17 04:45 | GENERIC | A | BUY | trending_up | +33/57 | +0.50 (+1) | +17.67 (+0) | +0.44 (-1) | +29.28 (+1) | +0.17 (+2) | +0.36 (+1) | +2.05 (+1) | +1.00 (+1) | | -1.02 |
| 2026-03-23 07:30 | XAU_ATR | A | SELL | trending_down | +34/57 | +1.00 (-2) | +28.60 (+0) | +0.55 (+1) | +55.46 (-1) | +0.42 (-2) | +5.31 (+0) | +3.62 (-1) | -1.00 (+1) | +0.00 (+0) | -1.01 |
| 2026-03-23 14:30 | GENERIC | B | BUY | trending_up | +36/51 | +0.50 (+1) | +12.86 (+1) | +0.32 (-1) | +33.75 (+1) | +0.20 (+2) | +2.45 (+2) | +1.76 (+1) | +1.00 (-1) | | -1.00 |
| 2026-03-26 08:45 | XAU_ATR | A | SELL | trending_down | +37/57 | +1.00 (-2) | +20.38 (+0) | +0.35 (+1) | +33.75 (-1) | +0.14 (-2) | +0.02 (+0) | +2.42 (-1) | | | -1.02 |
| 2026-03-27 05:45 | GENERIC | B | BUY | transitioning | +28/61 | +0.50 (+1) | +21.60 (-1) | +0.48 (-1) | +24.67 (+0) | +0.17 (+2) | +0.81 (+1) | +2.43 (+1) | | | -1.01 |
| 2026-03-30 12:30 | GENERIC | B | BUY | trending_up | +30/64 | +1.00 (+2) | +18.40 (+0) | +0.54 (-1) | +38.35 (+1) | +0.13 (+2) | +0.73 (+1) | +1.81 (+1) | +1.00 (-1) | +1.00 (+1) | -1.01 |
| 2026-04-01 15:45 | XAU_ATR | A+ | BUY | trending_up | +51/66 | +1.00 (+2) | +20.06 (+0) | +0.64 (-1) | +25.46 (+1) | +0.19 (+2) | +1.14 (+1) | +2.23 (+2) | | | -1.02 |
| 2026-04-06 08:45 | GENERIC | B | BUY | transitioning | +26/59 | +0.50 (+1) | +15.79 (+0) | +0.48 (-1) | +21.95 (+0) | +0.25 (+2) | +0.69 (+0) | +1.95 (+1) | | | -1.01 |
| 2026-04-07 06:30 | XAU_ATR | A | SELL | trending_down | +33/66 | +1.00 (-2) | +4.02 (-2) | +0.08 (+0) | +27.89 (-1) | +0.02 (-2) | +0.19 (-1) | +0.45 (+0) | | -1.00 (-1) | -1.03 |
| 2026-04-20 22:30 | XAU_ATR | A | BUY | transitioning | +31/64 | +1.00 (+2) | +16.74 (+0) | +0.54 (-1) | +18.60 (+0) | +0.20 (+2) | +0.31 (+1) | +1.90 (+1) | | +0.00 (+0) | -1.05 |
| 2026-04-21 20:00 | GENERIC | B | SELL | trending_down | +33/52 | +1.00 (-2) | +18.16 (+0) | +0.44 (+1) | +40.09 (-1) | +0.08 (-2) | +0.44 (+0) | +2.27 (-1) | -1.00 (+1) | | -0.31 |
| 2026-04-24 14:30 | GENERIC | B | BUY | trending_up | +30/53 | +0.50 (+1) | +14.74 (+1) | +0.42 (-1) | +26.53 (+1) | +0.20 (+2) | +0.40 (+0) | +1.70 (+1) | | | -1.01 |
| 2026-04-27 03:45 | GENERIC | B | BUY | trending_up | +31/58 | +0.50 (+1) | +20.30 (+0) | +0.51 (-1) | +32.62 (+1) | +0.51 (+2) | +0.76 (+1) | +2.62 (+1) | | +1.00 (+1) | -1.03 |
| 2026-05-04 18:30 | XAU_ATR | A | SELL | trending_down | +34/55 | +1.00 (-2) | +17.75 (+0) | +0.27 (+0) | +35.48 (-1) | +0.01 (-2) | +0.22 (+0) | +1.57 (-1) | | | -1.03 |
| 2026-05-07 22:45 | XAU_ATR | A | SELL | trending_down | +39/62 | +0.50 (-1) | +13.87 (-1) | +0.51 (+1) | +38.20 (-1) | +0.10 (-2) | +0.48 (-1) | +1.54 (-2) | | -1.00 (-1) | -1.03 |
| 2026-05-11 06:00 | XAU_ATR | A | SELL | trending_down | +37/65 | +1.00 (-2) | +25.44 (+0) | +0.64 (+1) | +38.34 (-1) | +0.17 (-2) | +0.79 (-1) | +2.47 (-1) | -1.00 (+1) | | -1.03 |
| 2026-05-13 06:15 | GENERIC | C | BUY | ranging | +31/47 | +0.00 (+0) | +7.24 (+0) | +0.52 (-1) | +17.80 (+0) | +0.11 (+2) | +0.14 (+1) | +1.04 (+2) | | | -1.02 |
| 2026-05-20 03:15 | XAU_ATR | A | SELL | trending_down | +41/64 | +1.00 (-2) | +15.51 (+0) | +0.49 (+1) | +30.96 (-1) | +0.20 (-2) | +1.05 (-1) | +1.78 (-1) | | | -1.03 |
| 2026-05-21 18:00 | GENERIC | B | BUY | transitioning | +27/63 | +0.50 (+1) | +23.63 (-1) | +0.60 (-1) | +22.98 (+0) | +0.49 (+2) | +1.06 (+1) | +2.60 (+1) | | | -1.01 |
| 2026-05-22 14:15 | GENERIC | B | SELL | ranging | +32/70 | +1.00 (-2) | +15.71 (+1) | +0.70 (+1) | +15.74 (+0) | +0.16 (-2) | +0.53 (-1) | +2.12 (-2) | | | -1.29 |
| 2026-06-02 07:15 | GENERIC | A | BUY | trending_up | +36/55 | +1.00 (+2) | +26.76 (+0) | +0.41 (-1) | +30.49 (+1) | +0.33 (+2) | +0.66 (+0) | +3.07 (+1) | | | -1.01 |
| 2026-06-08 05:45 | XAU_ATR | A | SELL | trending_down | +35/57 | +1.00 (-2) | +16.14 (+0) | +0.63 (+1) | +36.04 (-1) | +0.09 (-1) | +0.40 (-1) | +2.05 (-1) | | -1.00 (-1) | -1.03 |
| 2026-06-10 12:15 | XAU_ATR | A | SELL | trending_down | +34/63 | +1.00 (-2) | +23.68 (+0) | +0.62 (+1) | +37.31 (-1) | +0.15 (-2) | +0.84 (-1) | +2.36 (-1) | -1.00 (+1) | | -1.02 |
| 2026-06-12 09:15 | XAU_ATR | A | BUY | transitioning | +36/70 | +1.00 (+2) | +18.55 (+0) | +0.52 (-1) | +23.45 (+0) | +0.41 (+2) | +1.14 (+1) | +2.25 (+1) | +1.00 (+1) | | -1.03 |
| 2026-06-15 14:30 | XAU_ATR | A | BUY | trending_up | +39/64 | +1.00 (+2) | +20.56 (+0) | +0.51 (-1) | +37.03 (+1) | +0.14 (+2) | +0.65 (+1) | +1.96 (+1) | | | -1.04 |
| 2026-06-19 05:00 | XAU_ATR | A+ | SELL | trending_down | +49/66 | +1.00 (-2) | +37.80 (+0) | +0.59 (+1) | +49.95 (-1) | +0.41 (-2) | +1.29 (-1) | +4.55 (-1) | | | -1.04 |
| 2026-06-26 15:45 | XAU_ATR | A | BUY | trending_up | +33/60 | +1.00 (+2) | +24.05 (+0) | +0.47 (-1) | +41.85 (+1) | +0.19 (+2) | +1.10 (+1) | +2.37 (+1) | +1.00 (-1) | | -1.03 |
| 2026-06-30 14:30 | GENERIC | B | BUY | trending_up | +32/64 | +0.50 (+1) | +20.61 (+0) | +0.70 (-1) | +28.39 (+1) | +0.13 (+2) | +0.50 (+1) | +2.30 (+2) | +1.00 (-1) | | -1.01 |
| 2026-07-06 14:45 | GENERIC | B | SELL | transitioning | +28/67 | +0.50 (-1) | +15.58 (+0) | +0.52 (+1) | +21.20 (+0) | +0.11 (-2) | +0.62 (-1) | +1.61 (-1) | | | -1.02 |
| 2026-07-07 05:15 | GENERIC | B | SELL | trending_down | +28/55 | +0.50 (-1) | +21.44 (+0) | +0.45 (+1) | +40.22 (-1) | +0.14 (-2) | +0.42 (-1) | +2.05 (-1) | -1.00 (+1) | | -1.02 |
| 2026-07-07 12:15 | GENERIC | A | BUY | trending_up | +35/58 | +0.50 (+1) | +24.63 (+0) | +0.79 (-1) | +27.96 (+1) | +0.50 (+2) | +0.97 (+1) | +3.81 (+1) | | | -1.01 |
| 2026-07-14 12:45 | XAU_ATR | A | BUY | transitioning | +33/65 | +0.50 (+1) | +34.78 (-1) | +1.05 (-1) | +20.21 (+0) | +0.42 (+2) | +1.86 (+1) | +5.42 (+1) | | | -1.03 |
| 2026-07-20 03:45 | XAU_ATR | A | BUY | transitioning | +32/61 | +0.50 (+1) | +16.30 (+0) | +0.49 (-1) | +20.82 (+0) | +0.34 (+2) | +0.42 (+1) | +1.92 (+1) | | | -1.04 |
| 2026-07-22 12:30 | GENERIC | B | BUY | ranging | +31/69 | +1.00 (+2) | +15.32 (-1) | +0.74 (-1) | +14.12 (+0) | +0.20 (+2) | +0.43 (+1) | +1.92 (+2) | | | -1.03 |
| 2026-07-27 07:30 | GENERIC | B | BUY | trending_up | +33/60 | +1.00 (+2) | +10.74 (+1) | +0.55 (-1) | +29.55 (+1) | +0.09 (+2) | +0.31 (+1) | +1.35 (+0) | | | -1.03 |
| 2026-07-27 14:30 | XAU_ATR | A | SELL | transitioning | +32/59 | +0.50 (-1) | +15.83 (+0) | +0.56 (+1) | +24.69 (+0) | +0.27 (-2) | +0.63 (-1) | +2.04 (-1) | | -1.00 (-1) | -1.04 |
| 2026-07-29 14:15 | GENERIC | A | SELL | trending_down | +39/61 | +1.00 (-2) | +24.24 (+0) | +0.62 (+1) | +30.53 (-1) | +0.35 (-2) | +0.78 (-1) | +2.89 (-1) | | | -1.01 |
| 2026-07-29 18:15 | XAU_ATR | A | BUY | trending_up | +36/62 | +0.50 (+1) | +30.49 (+0) | +0.76 (-1) | +37.37 (+1) | +0.59 (+2) | +1.76 (+1) | +4.27 (+1) | | | -1.03 |
| 2026-07-30 14:45 | XAU_ATR | A | BUY | trending_up | +42/59 | +1.00 (+2) | +18.68 (+0) | +0.57 (-1) | +33.18 (+1) | +0.20 (+2) | +0.78 (+1) | +2.07 (+1) | | +1.00 (+1) | -1.02 |
| 2026-08-04 17:15 | GENERIC | A | BUY | trending_up | +42/61 | +1.00 (+2) | +18.27 (+0) | +0.53 (-1) | +31.75 (+1) | +0.10 (+2) | +0.50 (+1) | +1.96 (+1) | | | -1.02 |
| 2026-08-06 14:45 | XAU_ATR | A | BUY | ranging | +37/67 | +1.00 (+2) | +5.66 (+0) | +0.28 (+0) | +13.07 (+0) | +0.12 (+2) | +0.40 (+1) | +0.85 (+2) | | +1.00 (+1) | -1.03 |
| 2026-08-20 12:30 | GENERIC | B | SELL | transitioning | +26/61 | +0.50 (-1) | +23.58 (+1) | +0.62 (+1) | +22.33 (+0) | +0.37 (-2) | +0.51 (+0) | +2.59 (-1) | | | -1.01 |
| 2026-08-27 14:15 | XAU_ATR | A | SELL | trending_down | +37/61 | +1.00 (-2) | +16.18 (+0) | +0.44 (+1) | +32.76 (-1) | +0.11 (-2) | +0.52 (-1) | +1.43 (-2) | -1.00 (+1) | | -1.03 |
| 2026-08-28 07:00 | XAU_ATR | A | BUY | transitioning | +36/61 | +0.00 (+0) | +16.94 (+0) | +0.93 (-1) | +12.29 (+0) | +0.37 (+2) | +0.61 (+1) | +3.10 (+2) | +1.00 (+1) | +1.00 (+1) | -1.04 |
| 2026-09-04 08:00 | XAU_ATR | A | BUY | transitioning | +39/76 | +1.00 (+2) | +10.83 (+0) | +0.57 (-1) | +14.15 (+0) | +0.23 (+2) | +0.35 (+0) | +1.43 (+2) | | | -1.04 |
## Summary for the owner
| Question | Five-minute book | Fifteen-minute book |
|---|---|---|
| Where do the wrong-direction trades come from? | Half from chart patterns (74 of 153) and half from indicator setups (73 generic, 6 gold volatility-sized). Chart patterns are wrong a little more often (22% of their trades against 18%); the inverse head-and-shoulders is the weak one (28% wrong, only +4.6R on 86 trades), the double bottom the strong one (17%, +21.5R). | No chart patterns on this timeframe. The gold volatility-sized setup (the grade-A setup) is wrong 24% of the time against 16% for the generic swing setup, and nets only +5.7R on 174 trades; three quarters of the wrong-direction count is generic simply because generic is four fifths of the book. |
| What did the indicators read on them? | Practically the same as on the winners. Every separation value is between 0.46 and 0.57 (0.50 = no separation): composite score +31.8 against +33.1, confidence 54 against 55, five of nine indicators agreeing in both groups. The only mild differences: the Keltner vote agreed 73% against 80%, RSI sat at +18 against +21 above the midline, a divergence was present on 24% against 34%. | Same picture, if anything slightly inverted: wrong-direction trades had a marginally higher composite (+36.2 against +35.9) and more exhaustion tags (3 against 2). The only differences above noise sit on the confirmation frames: the first confirmation's candlestick opposed 29% of the wrong-direction trades against 10% of winners (present on 30%), and the second confirmation's EMA alignment agreed 66% against 76%. |
| Hours or days? | 05:00 (33% wrong on 52 trades against a 20% base) and 08:00 (24%); the cells Tuesday 05:00 (8 of 16) and Thursday 20:00 (6 of 11). December 2024 and August 2025 were the worst months. | 18:00 (27% on 45 trades against a 17% base) and 22:00 (23%); Wednesday is the only weekday above the base (22% on 189 trades), with Wednesday 18:00 (5 of 13) and 09:00 (6 of 20) the worst cells. July was the worst month in both years (29% and 24%). |
Reading: the wrong-direction losers were not visibly wrong at entry. The indicator set, the
composite, the confidence, the confirmation frames and the stretch measures all read the same as
on the winners, which means the direction errors are the noise floor of this signal set, not a
systematic misread that another threshold would remove. The two structural readings, the inverse
head-and-shoulders on M5 and the gold volatility-sized setup on M15, and the Wednesday / July
weakness on M15, are candidates for a rule-1 campaign at most, not changes. The per-trade
appendix above lists every wrong-direction loser with its nine primary-timeframe indicator values.
+15 -1
View File
@@ -46,7 +46,8 @@
"candles_enabled": true,
"candle_bars_primary": 20,
"candle_bars_confirm": 10,
"verdict_source": "macro"
"verdict_source": "mean",
"personas": ["macro"]
},
"violent_watch": {
"enabled": false,
@@ -64,5 +65,18 @@
"enabled": true,
"path": "logs/newsdata.db"
},
"news_model": {
"path": "/home/Forex/models/xau_transformer",
"threads": 4,
"mode": "off",
"deepseek_audit_per_day": 0
},
"trade_size": {
"risk_per_trade": 0,
"contract_size": 100,
"lot_step": 0.01,
"telegram_channel": false,
"dashboard": false
},
"log_level": "INFO"
}
+43
View File
@@ -103,6 +103,16 @@ class TechnicalConfig:
class SignalConfig:
"""Signal scoring & filtering."""
mtf_agreement_threshold: int = 2 # of 3 timeframes
# RESEARCH-ONLY grading rule switch (campaign 2026-09-27,
# backtest/research/conviction_campaign_2026-09-27.md). "pair" is the
# shipped two-number rule (a score floor AND a confidence floor, the
# literal table in signal_aggregator._quality_pair); "min" and "product"
# grade on ONE conviction number (min(|score|, 0.6 x confidence) or
# |score| x confidence / 100, thresholds in
# signal_aggregator.CONVICTION_THRESHOLDS). Read by
# SignalAggregator._compute_final_signal and gate 6c; packed for the CUDA
# twin. Any value other than "pair" is a rule-1 event.
grading_mode: str = "pair"
min_rr_ratio: float = 0.5
# TP tightening (May 2026) was tuned on the pre-spread backtester.
@@ -126,7 +136,17 @@ class SignalConfig:
# floors at ATR × v2_sl_atr_multiplier only.
min_sl_pips: float = 80.0
# TP1 floor as a fraction of SL distance (0.5 = historical RR 0.5).
# Part of the setup geometry the gates judge (min-RR, spread gate).
tp_floor_rr: float = 0.5
# Printed-target floor (2026-09-26, owner rule "TP1 at least 1:1"):
# every generator raises the PRINTED TP1 to at least this fraction
# of the stop AFTER the min-RR and spread gates have judged the
# generator's own target, so which generator serves a signal (and
# the stop) never changes and the books stay trade-identical; only
# the exported tp1 / rr_planned columns move. TP1 is advisory in the
# tested exit (the trail), so this is a guide, not a book change.
# The effective floor is max(tp_floor_rr, tp_print_floor_rr).
tp_print_floor_rr: float = 1.0
# ── Trailing-stop exit (2026-07) ──────────────────────────────
# Once favorable excursion reaches trail_arm_r × risk, the stop
@@ -305,6 +325,29 @@ class XauStrategyConfig:
big_candle_atr_multiple: float = 0.0
big_candle_lookback_hours: float = 24.0
# ── Daily-trend veto (2026-09-30) ──────────────────────────────
# The engine sees at most the H1 chart, and on M15 trades against a
# CLEAR daily trend earn nothing after costs. The daily trend is +1 /
# -1 when the last closed D1 close sits above / below its 50-bar EMA
# AND above / below the close 60 daily bars earlier, else 0
# (signal_aggregator.daily_trend_series; Sunday stub bars dropped).
# "both" removes any finished setup that goes against a non-zero
# daily trend, "buy" only BUYs against a falling one, "sell" only
# SELLs against a rising one, "off" nothing. Off by default; the M15
# factory sets "both" (override 7). It FAILED its rule on M5 and H1
# and as "buy" / "sell" on M15
# (backtest/research/daily_trend_campaign_2026-09-30.md).
daily_trend_veto: str = "off"
# ── Wide-stop cap (2026-09-30, research knob) ──────────────────
# Caps the stop distance of the swing (GENERIC) and chart-pattern
# setups at max(K x ATR, the setup's own stop floor). Applied AFTER
# the min-RR and spread gates, like the printed-target floor, so
# the set of trades cannot change. 0.0 = off, and it ships off
# everywhere: K = 6 FAILED its rule on M15 (a narrow peak; K = 5
# loses 18.6R over 2019-2026). Same campaign report.
wide_stop_cap_atr: float = 0.0
@dataclass
class FilterConfig:
+20 -1
View File
@@ -18,6 +18,10 @@ immune to anything on the web side):
* The only shared-code dependency beyond the standard signal path is
the additive `NewsEngine.get_week_events` accessor, which no
production signal path calls.
* Never writes a database. It is not always running, so the Telegram
bot is the single writer of `signals.db` and `newsdata.db`; main()
switches every SQLite writer off for this process
(`app_config.set_db_writes_for_process`, owner decision 2026-09-17).
Scheduling: every M15/H1/H4 boundary is also an M5 boundary (all
intervals are multiples of 5, anchored to UTC midnight), so one
@@ -58,8 +62,10 @@ from typing import Dict, List, Optional
import pandas as pd
from agent_panel import attach_panel
from app_config import get_app_config, violent_watch_enabled
from app_config import (get_app_config, set_db_writes_for_process,
violent_watch_enabled)
from config import PlatformConfig, Timeframe, XAUUSD
from exposure import attach_exposure
from main import report_to_dict
from news_engine import NewsEngine
from pair_signals import produce_signal
@@ -432,6 +438,11 @@ class DashboardPublisher:
if violent_watch_enabled():
attach_watch(report, tf,
get_app_config().violent_watch_settings)
# Display-only "already open" line (a read-only query on the
# bot's signals.db — this process still never writes it) and,
# only when config.json sets trade_size.dashboard, the lot
# size: the page is public and the size shows the owner's risk.
attach_exposure(report, surface="dashboard")
# Which boundary this report belongs to: the bar LABEL it closed
# on plus one interval. Recorded so a boundary missed by an
# overrunning cycle or a restart is caught up on the next wake
@@ -553,6 +564,14 @@ def main(argv=None) -> None:
args = ap.parse_args(argv)
_setup_logging()
# This process never writes a database. It is not always running, so
# the Telegram bot is the single writer of signals.db and newsdata.db
# (owner decision 2026-09-17). The switch is process-wide and beats
# config.json / env; set here in main(), not at import, so importing
# the module (the test suite does) leaves every other writer on.
set_db_writes_for_process(False)
logger.info("database writes disabled for this process "
"(the Telegram bot is the only writer)")
os.makedirs(args.out, exist_ok=True)
pub = DashboardPublisher(args.out)
pub.install_static()
+14 -1
View File
@@ -19,7 +19,9 @@ git pull
venv/bin/python test_all.py # optional sanity: suite must be green
```
No new Python dependencies are required.
Two optional Python packages (`onnxruntime`, `tokenizers`) arrive with the local news model;
`venv/bin/pip install -r requirements.txt` after the pull. See `deploy/README-news-model.md`,
which also says when to keep this daemon on DeepSeek (`NEWS_MODEL_MODE=off` in the unit).
## 2. Make the web folder writable
@@ -75,6 +77,17 @@ boundary (+10 s) and rescans whichever timeframes just closed. Logs rotate
every 3 h (240 backups); the bot's own `telegram.log` rotates daily (30
backups) since 2026-09-08.
Since 2026-09-30 each card also shows the daily trend, and a trade plan can
carry an "Already open" line. That line is one **read-only** query on the
bot's `logs/signals.db` (`mode=ro`; this process still never writes either
database), so the daemon user needs read access to that file and its `-wal`
/ `-shm` companions. If it cannot read them the line is simply absent. The
lot-size line is NOT published here unless `config.json` sets
`"trade_size": {"dashboard": true}`: the page is public and the size shows
the owner's risk per trade. The M15 scan now needs the daily (D1) candles as
a required frame, so a D1 fetch failure fails that card's scan like any other
missing frame.
## 5. Optional Apache polish
The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache
+185
View File
@@ -0,0 +1,185 @@
# Deploying the local news model (live server)
Since 2026-09-27 ForexBot2 can score news headlines with the owner's ML-NEWS gold transformer
instead of DeepSeek. The model runs on the CPU inside the bot process (and inside the dashboard
daemon), needs two extra Python packages and a 280 MB model folder that lives **outside** the git
tree. DeepSeek keeps its other jobs: the direction of a calendar surprise, the news summary and the
dashboard's on-demand summary endpoint.
What the model is, in one paragraph: a DeBERTa-v3-small transformer with two heads (does this
headline move gold? if so, up or down?) exported to ONNX, trained to copy DeepSeek's gold score on
17,245 headlines. On the 78-headline golden set it scores 72 (DeepSeek 78, the old keyword scorer
24); on unseen live days it agrees with DeepSeek about 72% of the time (DeepSeek agrees with itself
about 90%). It scores one headline in about 10–20 ms on one CPU thread. It is **gold only**: every
other currency's headline sentiment stays with the keyword scorer in `primary` mode.
Owner decisions this deployment implements:
- Mode `primary`: the model scores every headline; **no DeepSeek calls for headline scoring**.
- **The news archive stops growing.** In `primary` mode the model's own output is not archived (it is
not training data). Only the optional DeepSeek audit sample (`deepseek_audit_per_day`, shipped 0)
writes rows. `logs/newsdata.db` stays as it is; nothing is migrated or deleted.
- Both processes (bot and dashboard daemon) load the model so Telegram and the dashboard show the same
headline scores. Each ONNX session holds about 0.6 GB of RAM.
## 0. Check memory
```bash
free -m
```
Each process that loads the model needs about 0.7 GB of headroom; two processes about 1.5 GB. If the
box cannot spare that, keep the model on the bot only: uncomment the `Environment=NEWS_MODEL_MODE=off`
line in `deploy/gold-dashboard.service` (step 5). The dashboard then keeps scoring its headlines with
DeepSeek and its scores may differ from the bot's. The dashboard process never writes the archive in
any mode, so this choice does not affect training data.
## 1. Code and dependencies
```bash
cd /home/Forex/ForexBot2
git pull
venv/bin/pip install -r requirements.txt # adds onnxruntime and tokenizers
```
`requirements.txt` must never contain the training stack (no torch, no transformers); the test suite
checks that. The live box runs Python 3.9, where pip resolves onnxruntime 1.19.2 and tokenizers
0.22.2; the bundle of 2026-09-27 scores identically there and on the dev box (1.30.0 / 0.23.2).
## 2. Copy the model files (outside git)
On the machine that holds ML-NEWS (`~/Development/ML-NEWS/models/xau_transformer/`), copy exactly
three files. Skip the `torch/` and `torch_final/` subfolders; they are training checkpoints.
```bash
ssh Forex@LIVE 'mkdir -p /home/Forex/models/xau_transformer'
rsync -av --progress \
~/Development/ML-NEWS/models/xau_transformer/{model.onnx,tokenizer.json,meta.json} \
Forex@LIVE:/home/Forex/models/xau_transformer/
```
Verify on the live box:
```bash
sha256sum /home/Forex/models/xau_transformer/model.onnx
# bundle of 2026-09-27: 9e7a64d3ae1df335059ce829cafcf6e4544b85a4f72cc88923c2e42c48e975a0
sha256sum /home/Forex/models/xau_transformer/tokenizer.json
# bundle of 2026-09-27: 045f93f80eb9ad18e559647563fb827dea59d93258fdb8b922de82bc53666d16
python3 -c "import json; print(json.load(open('/home/Forex/models/xau_transformer/meta.json'))['trained_at'])"
# 2026-09-27T23:15:30.403442+00:00
```
The files should be owned by (or readable by) the account the bot runs as. Read-only is enough.
## 3. Configure
Add one block to `/home/Forex/ForexBot2/config.json` (any position; `config.json.example` shows it).
Leave the `deepseek` block as it is.
```json
"news_model": {
"path": "/home/Forex/models/xau_transformer",
"threads": 2,
"mode": "primary",
"deepseek_audit_per_day": 0
}
```
(`threads` is clamped to the core count; the live box has 2 cores, hence 2. Measured there: load
0.8 s, 16 ms per headline, 633 MB resident for the session.)
| Key | Meaning |
|---|---|
| `path` | folder with `model.onnx`, `tokenizer.json`, `meta.json`; required when `mode` is not `off` |
| `threads` | ONNX threads per process. Measured: 1 thread 19 ms per headline, 4 threads 9 ms, 16 threads 13 ms. 4 is right for a 4–8 core box |
| `mode` | `off` = today's DeepSeek/keyword path, nothing loads. `fallback` = DeepSeek first, the model only for headlines DeepSeek failed on. `primary` = the model scores everything, DeepSeek never sees a headline (except the audit sample) |
| `deepseek_audit_per_day` | `primary` only: headlines per UTC day also sent to DeepSeek and archived as `llm` rows (one call of up to 25 per cycle at most). They are training labels for the next ML-NEWS retrain; the bot never displays them. 0 = no DeepSeek headline calls at all |
The environment variable `NEWS_MODEL_MODE=off|fallback|primary` overrides `mode` for one process. It
exists so the dashboard daemon's systemd unit can pick its own mode.
A config error (unknown key, bad mode, missing path) stops the bot at boot with a clear message, like
every other `config.json` mistake.
## 4. Smoke test without starting the bot
```bash
cd /home/Forex/ForexBot2
venv/bin/python -c "
import news_model as m
b = m.get_news_model()
print(m.score_news(b, 'Gold jumps as dollar slides'))
print(m.score_news(b, 'Red Lobster endless shrimp losses'))"
```
Expect a log line `news model loaded from /home/Forex/models/xau_transformer (trained_at 2026-09-27…,
threshold 0.2, 4 threads)`, then `label: bullish` for the first headline and `label: neutral` for the
second. If instead you see `news model unavailable, DeepSeek/keyword path stays: …`, the message names
the cause (missing file, missing package, bad JSON). The bot would still start in that state and
would score with DeepSeek for the whole process lifetime; fix the cause and restart.
Then run the suite. With the model folder configured, the integration test loads the real bundle:
```bash
venv/bin/python test_all.py # expect RESULTS: … passed, 0 failed
```
## 5. Restart both processes
Bot: restart it the way it is run on this box (nohup / screen / its unit). Dashboard:
```bash
sudo systemctl restart gold-dashboard
```
If the dashboard must not load the model (step 0), first uncomment the
`Environment=NEWS_MODEL_MODE=off` line in `deploy/gold-dashboard.service`, then
`sudo cp deploy/gold-dashboard.service /etc/systemd/system/ && sudo systemctl daemon-reload` before
the restart.
## 6. What to check afterwards
- **Logs** (`logs/telegram.log`, `logs/dashboard.log`): exactly one `news model loaded` line per
process, no `news model unavailable` line. If the level is DEBUG you also see
`news archive: primary mode, 0 DeepSeek audit rows stored, N model/keyword rows not archived` every
news cycle.
- **`top`**: each process about 0.6 GB more resident memory than before; CPU near 0% between scans
(thread spinning is disabled in the runtime).
- **DeepSeek spend**: the daily request count no longer moves with the headline count; only the event
direction, the summary and the panel use it.
- **Archive frozen** (the intended state with audit 0):
```bash
sqlite3 -readonly logs/newsdata.db "SELECT MAX(first_seen), COUNT(*) FROM headlines;"
```
The newest `first_seen` stops advancing after the restart. With `deepseek_audit_per_day` > 0 you
see at most that many new rows a day, all `score_source = 'llm'`.
- **Telegram / dashboard**: headline scores still appear; the wording and layout are unchanged, only
the numbers now come from the model. A headline the model calls irrelevant to gold shows as neutral.
## 7. Rollback
Set `"mode": "fallback"` (DeepSeek first, the model only fills gaps and the archive grows again) or
`"mode": "off"` (exactly the pre-2026-09-27 behaviour) in `config.json`, or export
`NEWS_MODEL_MODE=off` for one process, and restart. No data migration in either direction.
## 8. Retraining cadence
When ML-NEWS ships a new bundle, copy the three files over the old ones (step 2), check the new
`trained_at`, and restart both processes. Rows written in `fallback` mode carry
`teacher_model = xau_transformer@<trained_at>`, so the archive shows which bundle scored them.
With `deepseek_audit_per_day` at 0 the archive receives no new DeepSeek labels, so the next retrain
needs one of: a positive audit value for a few weeks, a spell in `fallback` mode, or a separate
labelling run in ML-NEWS on freshly crawled headlines (`eval/eval_archive.py --scorer transformer` on a
copy of the live archive gives the model-versus-DeepSeek agreement on whatever labelled rows exist).
## Known limits
- The model copies DeepSeek's opinion; it has not learned from price moves. DeepSeek itself is only
about 90% consistent with itself, and the model agrees with it about 72% of the time.
- Gold only. In `primary` mode every other currency's headline sentiment is the keyword scorer (24 of
78 on the golden set), which is what those currencies got whenever DeepSeek was unavailable before.
- News never enters signals (CLAUDE.md section 7). No trade, grade or backtest book changes with any of
this; the model only changes the headline lines people read.
+4
View File
@@ -15,6 +15,10 @@ Type=simple
User=Forex
WorkingDirectory=/home/Forex/ForexBot2
Environment=DASHBOARD_OUT_DIR=/var/www/trade
# The local news model (deploy/README-news-model.md) loads in this process
# too (~0.6 GB RSS) so the dashboard shows the bot's headline scores.
# Uncomment to keep this daemon on DeepSeek when memory is short:
#Environment=NEWS_MODEL_MODE=off
ExecStart=/home/Forex/ForexBot2/venv/bin/python dashboard_publisher.py
Restart=always
RestartSec=30
+24 -9
View File
@@ -40,7 +40,7 @@
<body>
<h1>ForexBot2 — Beginner's Guide</h1>
<div class="subtitle">Everything explained in plain words. No experience needed. &nbsp;·&nbsp; July 2026</div>
<div class="subtitle">Everything explained in plain words. No experience needed. &nbsp;·&nbsp; updated October 2026</div>
<h2>1 · What is this bot?</h2>
<p>ForexBot2 is a computer program that watches the <b>price of gold</b> all day and all night. It studies the price charts using nine different mathematical checks. When the checks agree that the price is likely to move in one direction, it sends you a message on <b>Telegram</b> telling you:</p>
@@ -71,20 +71,34 @@
<h2>3 · Reading a bot message</h2>
<p>Here is a typical signal, with plain-English notes below it:</p>
<div class="signal-box">🔍 XAU/USD Signal Analysis (H1) ← gold, on the 1-hour chart
🟢 BUY Grade: A | 64% confidence ← what to do + how good the setup looks
✅ MTF aligned ← bigger and smaller charts agree
🟢 Grade A · BUY · XAU/USD · H1 🥇 ← how good the setup is + what to do
📐 Direction: strong buy · Reliability:
good · timeframes agree ← the bot's two numbers, in words
🧭 Daily trend: rising — this buy
goes with it ← which way gold has been heading for weeks
📋 Indicators … ← the bot's 9 checks (you can skip this part)
━━━━━━━━━━━━━━━━━━━━
🚀 TRADE SETUP ← ↓ THIS is the part you act on ↓
📈 Direction: BUY
▶ Entry: 4,132.50 ← open the trade near this price
🛑 SL: 4,088.20 ← your safety brake (stop loss)
🎯 TP1: 4,176.80 ← rough profit target (a guide, not a rule)
🔄 Trail: once price hits 4,159.08,
trail SL $33.23 behind best price ← how to lock in profit (see section 6)</div>
🧱 Already open: 1 buy plan alerted
earlier (H1) — taking this one adds
to the same position ← you may already be in this move
🔄 Exit rule (tested): once price reaches
4,159.08 (+0.6R), trail the SL $33.23
behind the best price, moved at
candle close only. ← how to lock in profit (see section 6)
🎯 TP1 guide 4,176.80 (1.0R) ← rough profit target (a guide, not a rule)
⚖️ Planned TP1 RR 1:1.0 (guide only)
📚 Book (this setup class): 312 trades,
58% winners, avg +0.19R per trade ← what this kind of signal did in the backtests</div>
<ul>
<li><b>Grade (A+, A, B, C):</b> the bot's own mark for how clean the setup is. A+ is best. It never sends anything worse than C.</li>
<li><b>Confidence and the indicator list:</b> extra detail for curious people. You don't need to understand it to trade the signal.</li>
<li><b>Direction and reliability:</b> plain words for the bot's two internal numbers. Direction says how strongly the checks lean one way (very strong, strong, clear, weak). Reliability says how much the bot trusts that reading (high, good, fair, low). "Timeframes agree" means the bigger and smaller charts point the same way.</li>
<li><b>🧭 Daily trend:</b> which way the gold price has been heading over the last weeks, read from the daily chart: rising, falling, or no clear direction. A trade that goes the same way as the daily trend is swimming with the tide. On the 15-minute signals the bot no longer sends a trade that goes against a clear daily trend at all, because years of tests showed those trades earned nothing. On the other signals the line is simply something to know.</li>
<li><b>🧱 Already open:</b> appears when the bot has already sent buy (or sell) plans that are still running. A new signal in the same direction is usually the same gold move seen again. The new signal is not a worse one, but if you took the earlier one, taking this one too means you are risking more on a single move. As a beginner, skip it, or split your usual risk between the two.</li>
<li><b>The indicator list:</b> extra detail for curious people. You don't need to understand it to trade the signal.</li>
<li><b>⚠️ Warning lines:</b> things the bot noticed, like high volatility. They are "be aware," not "don't trade."</li>
<li><b>🌪 Violent-move watch:</b> now and then the bot adds a line saying the next hour could produce very big candles — usually because an important American economic number is about to come out, or the market is in an hour that is historically wild, or the price has stretched a long way from normal. It is only a "hold on tight" note. It does not change the signal, and it is not a reason to close or move your stop loss early — the bot was tested doing exactly that and it lost money. Your stop loss is already your protection; leave it where it is.</li>
<li><b>📰 News line:</b> a short plain-English note on what is moving the gold price right now and why, written from the last few hours of headlines. It is background information, not a reason to trade.</li>
@@ -99,7 +113,7 @@ Your account: <b>$2,000</b>. Your risk: 1% = <b>$20</b>. So for this trade, 1R =
The signal says: Entry 4,132.50, Stop loss 4,088.20. The gap between them is <b>$44.30 per ounce</b> of gold.<br>
How much gold to buy: $20 ÷ $44.30 = <b>0.45 ounces</b>.<br>
Now, whatever happens: if the price falls to the stop loss, you lose about $20 — and no more. If the trade wins 2R, you make about $40.</div>
<p class="small">Most broker apps have a built-in calculator: enter the stop-loss price and the amount you want to risk ($20), and it works out the position size for you. Use it — it avoids mistakes. (On many platforms gold is traded in "lots," where 0.01 lot = 1 ounce.)</p>
<p class="small">Most broker apps have a built-in calculator: enter the stop-loss price and the amount you want to risk ($20), and it works out the position size for you. Use it — it avoids mistakes. (On many platforms gold is traded in "lots," where 0.01 lot = 1 ounce.) The bot's owner can also switch on a <b>📏 Size</b> line that does this sum for their own risk limit; it is shown only in the owner's private chat, so if you do not see it, use the calculator.</p>
<h2>5 · What to do when a signal arrives — step by step</h2>
<ol>
@@ -128,7 +142,7 @@ The message says: <i>"once price hits 4,159, trail SL $33 behind best price."</i
<p>The bot sends signals from several chart speeds. As a beginner, <b>follow only the 1-hour (H1) signals</b> and ignore the others. Here is why:</p>
<table>
<tr><th></th><th>M5 (5-minute)</th><th>M15 (15-minute)</th><th>H1 (1-hour) ★ start here</th></tr>
<tr><td>Signals per week</td><td>about 10</td><td>about 10</td><td>about 3</td></tr>
<tr><td>Signals per week</td><td>about 10</td><td>about 5</td><td>about 2</td></tr>
<tr><td>A trade usually lasts</td><td>a few hours</td><td>about half a day</td><td>half a day to a few days</td></tr>
<tr><td>Time you have to react</td><td>1–2 minutes</td><td>up to 15 minutes</td><td>up to an hour</td></tr>
<tr><td>Suits</td><td>full-time / automated</td><td>experienced, quick</td><td>anyone with a job and a life</td></tr>
@@ -139,6 +153,7 @@ The message says: <i>"once price hits 4,159, trail SL $33 behind best price."</i
<li><b>H1 is BUY-only.</b> All H1 SELL signals are blocked (since 2026-09-03) because they were a losing side on this timeframe — expect to see BUY signals here, never SELL.</li>
<li><b>M5 also holds back some SELL signals</b> — specifically while the market is in a choppy "transitioning" phase, since those shorts tested poorly.</li>
<li><b>M15 got a small tune-up on 2026-09-09</b> — one of its nine indicators (RSI) now counts for half as much, which means a few more M15 signals per week. Nothing changes in how you act on a signal.</li>
<li><b>M15 now respects the daily trend (since 2026-09-30).</b> A 15-minute trade that goes against a clear daily trend is no longer sent. That removes about one signal in four, the ones that made no money in the tests, and makes the bad stretches shorter. Nothing changes in how you act on the signals you do receive.</li>
<li><b>Never take the same signal on two timeframes.</b> They are usually the same gold move seen at different zoom levels. Taking both doubles your risk, not your profit.</li>
</ul>
@@ -0,0 +1,535 @@
# Specification: run the ML-NEWS XAU transformer inside ForexBot2
Status: **IMPLEMENTED 2026-09-27** (ForexBot2 branch `sql-integration`, on top of `6af1237`; suite
Module 38). The text below is the specification as written before the implementation; file:line
numbers refer to `6af1237` and have drifted. Where the implementation differs, this list wins:
- **`primary` mode does not archive the cycle's rows.** Owner decision during implementation: once the
local model scores, `logs/newsdata.db` stops growing, because the model's own output is not training
data. Only the DeepSeek audit copies (`deepseek_audit_per_day`, shipped 0) are written. `off` and
`fallback` archive as before (a `fallback` model row carries `model_teacher`). §5.4 and §9 read
accordingly.
- `_audit_with_deepseek` **returns** the `llm` copies; there is no `_pending_audit_rows` attribute.
The mode switch of §5.2 lives in `NewsEngine._enrich_headline_scores(scored, mode)`, which returns
those copies, so the call site and the tests share one function.
- `config.json.example` ships `"mode": "off"`; the live server sets `primary`
(`deploy/README-news-model.md`).
- §8 step 0 was wrong: the dashboard daemon never writes the archive in any mode
(`set_db_writes_for_process(False)`), so its DeepSeek rows are not "free audit labels". Both
processes load the model (owner decision); `NEWS_MODEL_MODE=off` stays as the memory fallback.
- `warm_up_in_background()` is implemented (module-level once flag; a `NewsEngine` is built per scan).
- `news_model.get_news_model()` also treats a missing `path` as a load failure (one warning).
- `requirements.txt` ranges: `onnxruntime>=1.19,<2`, `tokenizers>=0.22,<1`. The live box runs
Python 3.9, where only 1.19.2 / 0.22.2 install; verified on 2026-09-28 to score the bundle
identically to the dev box's 1.30.0 / 0.23.2 (golden pair, four probe headlines).
- ML-NEWS `news_transformer.py` received the §3.4 edits in the same session; the vendored copy differs
only in `load_model` having no default directory.
Repos:
- **ML-NEWS** `~/Development/ML-NEWS` (git `ssh://git@192.168.0.11:222/harry/News-ML-Decoder.git`).
Produces the model. Nothing in ForexBot2 imports from it.
- **ForexBot2** `~/Development/ForexBot2`, live at `/home/Forex/ForexBot2`, deployed by `git pull`.
Consumes the model.
---
## 1. Context and goal
### 1.1 What the model is
`ML-NEWS/models/xau_transformer/` holds a fine-tuned `microsoft/deberta-v3-small` with two heads,
exported to ONNX:
| File | Size | Purpose |
|---|---|---|
| `model.onnx` | 271 MB | encoder + heads; int8 embedding table, fp32 compute |
| `tokenizer.json` | 8 MB | HF `tokenizers` fast tokenizer |
| `meta.json` | 4 KB | thresholds, `trained_at`, `relevance_threshold`, `export_checks`, training stats |
Inputs: one headline, ≤ 64 tokens. Outputs per headline (from `news_transformer.score_batch`):
```python
{"headline": str, "p_relevant": float, "p_bullish": float,
"relevant": bool, # p_relevant >= meta["relevance_threshold"] (0.2 for the shipped bundle)
"direction": "bullish"|"bearish",
"label": "bullish"|"bearish"|"neutral", # neutral when not relevant
"score": float} # p_relevant * (2*p_bullish - 1), in [-1, 1]
```
`score` is on the same scale as DeepSeek's `xau` score that ForexBot2 already stores in
`xau_score`. The model was trained to copy DeepSeek (`deepseek-v4-flash`, prompt sha1 `d32e04c6…`, the
same prompt as `news_engine._LLM_SENTIMENT_PROMPT_SHA1`) on 17,245 headlines. Quality on
2026-09-27: golden set 72/78 (DeepSeek 78, keyword scorer 24), agreement with DeepSeek on unseen live
days ~72%, CPU latency 19 ms/headline on 1 thread, 9 ms on 4. Full numbers: `ML-NEWS/eval/RESULTS_2026-09-27.md`.
**The model scores gold only.** It has no head for USD, EUR or anything else. This drives most of
the rules in §5.
### 1.2 How ForexBot2 scores headlines today
`news_engine.py`, inside `NewsEngine._fetch_news_sentiment_raw` (called via
`_cached_fetch("_headlines_cache", …)` at :822, which holds `NewsEngine._cache_lock`, :629-647):
1. RSS items become `NewsSentiment` objects (:891-910) with the **keyword scorer** filling
`sentiment_score` and `sentiment_by_currency[cur]` for each currency in `currencies_mentioned`,
`score_source="keyword"`, `xau_score=None`.
2. `scored = headlines + unmatched` (:944); `self._apply_llm_sentiment(scored)` (:945, in try/except).
`_apply_llm_sentiment` (:1032-1065) sends headlines not in `NewsEngine._llm_sent_cache` to DeepSeek
in batches of `_LLM_BATCH_CAP = 25` via `self._llm_headline_scores(texts)` (:1009) and, on success,
overwrites `sentiment_score`, `sentiment_by_currency` (XAU gets the gold score, every other currency
the overall score), sets `xau_score`, `score_source="llm"`. A failed batch stops sending; those
headlines stay keyword-scored and **stay out of the cache**, so they are re-sent next window.
3. `news_archive.archive_headlines(scored, teacher_model=…, teacher_prompt_sha1=…)` (:958-964).
The archive UPSERT (news_archive.py:203-260) keeps the higher-ranked source:
`_SOURCE_RANK = {"unknown": 0, "keyword": 1, "model": 2, "llm": 3}` (:107). `"model"` is already
reserved for "a trained local artifact" (news_engine.py:223-227). `build_row` (:323-354) stores
`teacher_model`/`teacher_prompt_sha1` only when `source == "llm"`.
4. Downstream consumers of the scores: `get_fundamental_context` (drops |score| <
`NEUTRAL_HEADLINE_THRESHOLD = 0.1`, :256), `get_pair_sentiment` (:553-557, computes
`XAU_mean − USD_mean`), `signal_aggregator.py:1677/1723` (acts on that when
`sentiment_affects_signals`), `agent_panel.py`, the dashboard.
Two processes run this code: the bot (`venv/bin/python telegram_bot.py`, CLAUDE.md:30) and the
dashboard daemon (`dashboard_publisher.py:401-403` builds its own `NewsEngine`, `:503` calls
`get_fundamental_context()`; unit `deploy/gold-dashboard.service`). They share `config.json` and the
archive DB.
### 1.3 Decisions taken by the owner (2026-09-27)
- **Mode `primary`, no DeepSeek for headline scoring.** The model scores every headline. DeepSeek stays
for everything else it does (`_llm_event_direction` for calendar surprises, the news summary, the
dashboard's on-demand `news_summary.php`).
- **Model files live outside the ForexBot2 tree**, path in `config.json`.
- Accepted consequence: the archive stops receiving fresh DeepSeek labels, so the ML-NEWS weekly
retrain and the "replay and compare" check lose their data source. Mitigation kept optional:
`deepseek_audit_per_day` (§4) labels a few headlines a day for the archive only. Default 0.
### 1.4 Non-goals
- Removing the DeepSeek client or its non-headline uses.
- Scoring non-gold currencies with the model.
- Changing ML-NEWS training. (The one ML-NEWS change is §3.4, a runtime option.)
- Training from price moves.
---
## 2. Target layout on the live server
```
/home/Forex/ForexBot2/ # git pull as today
news_model.py # NEW, vendored runtime (§3)
requirements.txt # + onnxruntime, tokenizers (§3.1)
app_config.py, news_engine.py, news_archive.py, test_all.py # edited (§4, §5, §6, §7)
config.json # + "news_model" section (§4)
deploy/README-news-model.md # NEW ops checklist (§8)
deploy/gold-dashboard.service # maybe + Environment=NEWS_MODEL_MODE=off (§8)
/home/Forex/models/xau_transformer/ # NEW, outside git, copied by hand
model.onnx tokenizer.json meta.json
```
Memory: each process that loads the model holds ~0.5–0.7 GB RSS for the ONNX session. Two processes
→ ~1.5 GB. `free -m` before deciding whether the dashboard daemon also runs it (§8 step 0).
---
## 3. `news_model.py` (new, ForexBot2)
### 3.1 Dependencies
Add to `requirements.txt`:
```
onnxruntime>=1.30,<2
tokenizers>=0.23,<1
```
Pin to the versions that verified the bundle in ML-NEWS `.venv` (check `pip show` there before
writing). `tokenizer.json` compatibility across older `tokenizers` releases is not guaranteed.
**Do not write the words "torch" or "transformers" anywhere in requirements.txt**, not even in a
comment: `test_all.py:11437-11462` greps the file for them and fails.
### 3.2 Source
Copy `ML-NEWS/news_transformer.py` → `ForexBot2/news_model.py`. Keep verbatim: `clean_headline`,
`_sigmoid`, `load_model`, `score_batch`, `score_news`, constants `MAX_LENGTH = 64`,
`RELEVANT_MIN = 0.25`, `RELEVANT_MAX_IRRELEVANT = 0.10`. Drop the CLI `main` or keep it, either is fine.
Add a docstring line: `Vendored from ML-NEWS news_transformer.py at commit <sha>; bundle trained_at
<meta.json trained_at>.`
**Imports.** `onnxruntime` and `tokenizers` must stay imported *inside* `load_model` (as in
news_transformer.py:58-59). Module top imports only `numpy`, `json`, `logging`, `os`, `pathlib`,
`threading`. Reason: `news_engine.py` will `import news_model` at its top; if onnxruntime is missing on
a box, the bot must still start.
### 3.3 Singleton
```python
_lock = threading.Lock()
_bundle: Optional[dict] = None
_failed: bool = False # tried and failed; don't retry, don't re-log
def get_news_model() -> Optional[dict]:
"""The loaded bundle, or None. Loads once per process. Never raises."""
global _bundle, _failed
if _bundle is not None or _failed:
return _bundle
with _lock:
if _bundle is not None or _failed:
return _bundle
try:
import app_config
path = app_config.news_model_path()
threads = app_config.news_model_threads()
b = load_model(path, threads=threads)
logger.info("news model loaded from %s (trained_at %s, threshold %s, %d threads)",
path, b["meta"].get("trained_at"), b["meta"].get("relevance_threshold"), threads)
_bundle = b
except Exception as e: # ImportError, FileNotFoundError, bad JSON, ORT errors
logger.warning("news model unavailable, DeepSeek/keyword path stays: %s: %s",
type(e).__name__, e)
_failed = True
return _bundle
def reset_for_tests() -> None: # clears _bundle/_failed; used by test_all.py
```
Optional: `warm_up_in_background()` that calls `get_news_model()` in a daemon thread at
`NewsEngine.__init__`, so the first news cycle isn't blocked by the ~3–5 s session creation under
`_cache_lock`. Nice to have, not required.
### 3.4 Two changes inside `load_model` (also apply to ML-NEWS `news_transformer.py` so the copies stay identical)
```python
options = ort.SessionOptions()
if threads:
options.intra_op_num_threads = max(1, min(int(threads), os.cpu_count() or 1))
options.inter_op_num_threads = 1
# ORT intra-op threads spin-wait after each run by default; in a mostly idle
# bot process that is 4 cores at 100% between cycles.
options.add_session_config_entry("session.intra_op.allow_spinning", "0")
```
### 3.5 Contract `score_batch(bundle, headlines: list[str]) -> list[dict]`
Unchanged from ML-NEWS. Empty or whitespace-only headline raises `ValueError` from `clean_headline`;
callers pass already-non-empty `NewsSentiment.headline`, but §5 still wraps per-batch in try/except.
Batch size: `_apply_model_sentiment` sends all of a cycle's headlines in one call (typically 30–150);
`score_batch` pads to the longest in the batch, ≤ 64 tokens, so one call of 150 is ~1 s on 4 threads.
---
## 4. Configuration
### 4.1 `config.json` section (add to `config.json.example` too)
```json
"news_model": {
"path": "/home/Forex/models/xau_transformer",
"threads": 4,
"mode": "primary",
"deepseek_audit_per_day": 0
}
```
| Key | Type | Default if absent | Meaning |
|---|---|---|---|
| `path` | str | required when mode ≠ off | directory with the three bundle files |
| `threads` | int ≥ 1 | 4 | ORT intra-op threads (measured: 1→19 ms, 4→9 ms, 16→13 ms) |
| `mode` | `"off"`/`"fallback"`/`"primary"` | `"off"` | see §5.2 |
| `deepseek_audit_per_day` | int ≥ 0 | 0 | primary mode only: headlines/day to also send to DeepSeek, archive-only |
Whole section absent → `mode="off"` → today's behaviour exactly.
Environment override: `NEWS_MODEL_MODE=off|fallback|primary` wins over config.json. Purpose: the
dashboard daemon's systemd unit can turn the model off for that process only. Pattern to copy:
`NEWS_ARCHIVE_ENABLED` in `app_config.py:574-583`.
### 4.2 `app_config.py` edits (follow the `news_archive` section, :397-406, accessors :574/:588)
1. `_KNOWN_TOP` (:86-92): add `"news_model"`. **Required**; `_check_keys(raw, _KNOWN_TOP, "top level")`
at :181 makes boot fail on an unknown top-level key.
2. `_KNOWN_NEWS_MODEL = {"path", "threads", "mode", "deepseek_audit_per_day"}` next to
`_KNOWN_NEWS_ARCHIVE` (:113).
3. `AppConfig` dataclass (~:147): `news_model_settings: Dict[str, Any] = field(default_factory=dict)`.
4. In `load_app_config`, after the `news_archive` block:
```python
nm = raw.get("news_model", {})
_check_keys(nm, _KNOWN_NEWS_MODEL, '"news_model"')
if "mode" in nm and nm["mode"] not in ("off", "fallback", "primary"):
raise AppConfigError('config.json: news_model.mode must be "off", "fallback" or "primary"')
if "threads" in nm and (not isinstance(nm["threads"], int) or isinstance(nm["threads"], bool) or nm["threads"] < 1):
raise AppConfigError("config.json: news_model.threads must be a positive integer")
if "deepseek_audit_per_day" in nm and (not isinstance(nm["deepseek_audit_per_day"], int) or nm["deepseek_audit_per_day"] < 0):
raise AppConfigError("config.json: news_model.deepseek_audit_per_day must be a non-negative integer")
if nm.get("mode", "off") != "off" and not isinstance(nm.get("path"), str):
raise AppConfigError("config.json: news_model.path is required when mode is not off")
app.news_model_settings = dict(nm)
```
5. Accessors:
```python
def news_model_mode() -> str: # env NEWS_MODEL_MODE overrides; default "off"
def news_model_path() -> Optional[str]
def news_model_threads() -> int # default 4
def news_model_audit_per_day() -> int # default 0
```
`NewsConfig` in `config.py:169` is **not** touched: tests construct it bare as `NewsConfig()`, and it
holds feed/threshold settings, not paths. `news_engine.py` reads the new settings through `app_config`
the same way it reaches DeepSeek through `get_llm_client()`.
---
## 5. `news_engine.py` changes
### 5.1 New method `_apply_model_sentiment`
Place next to `_apply_llm_sentiment` (:1032). Runs under `_cache_lock` like it.
```python
def _apply_model_sentiment(self, headlines: List[NewsSentiment]) -> int:
"""Score with the local XAU transformer every headline that is still
keyword-scored. Returns the number scored. Never raises.
The model has a gold head only. It writes xau_score always, XAU's
per-currency score when XAU is mentioned, and the overall score only
when the headline names no other currency. USD/EUR/... entries keep
their keyword score: get_pair_sentiment subtracts USD from XAU and
must not see the same number on both sides.
"""
bundle = news_model.get_news_model()
if bundle is None or not headlines:
return 0
todo = [h for h in headlines if h.score_source == "keyword"]
if not todo:
return 0
try:
results = news_model.score_batch(bundle, [h.headline for h in todo])
except Exception as e:
logger.debug(f"news model scoring failed, keyword scores stand: {e}")
return 0
for h, r in zip(todo, results):
raw = float(r["score"]) # archive value, unmodified
shown = raw if r["relevant"] else 0.0 # display value, see rule D
h.xau_score = raw
if "XAU" in h.currencies_mentioned:
h.sentiment_by_currency["XAU"] = shown
if set(h.currencies_mentioned) <= {"XAU"}: # [] or ["XAU"]
h.sentiment_score = shown
h.score_source = "model"
return len(todo)
```
Rules, with the reason each exists:
- **A. Never write the model score into a non-XAU currency.** `get_pair_sentiment` (:553-557) forms
`XAU_mean − USD_mean`; `signal_aggregator.py:1677/1723` acts on it. The DeepSeek path puts its
*overall* score into USD; the model has no overall score, so USD keeps the keyword number.
- **B. `sentiment_score` from the model only when the headline names no other currency.** The archive's
`score` column is `sentiment_score`, and the ML-NEWS replay eval reads it; for a gold-only headline it
should be the model's number, for "Fed hikes, dollar jumps" it stays keyword so the row is honest
about what the model did and didn't score. `score_source="model"` therefore means "XAU score is the
model's" — amend the dataclass comment at :223-227 to say so.
- **C. `xau_score` is always the raw signed score.** That column exists for training data
(:229-237); keep it unfiltered.
- **D. Display fields get 0.0 when `relevant` is False.** A headline the model calls not-relevant can
still carry |score| up to ~0.2 from a confident direction head; `NEUTRAL_HEADLINE_THRESHOLD = 0.1`
(:256) would let 0.15 through as "leaning bearish". Zeroing the shown value makes "neutral" mean
neutral. (0.1 happens to equal the model's irrelevant band `RELEVANT_MAX_IRRELEVANT`, so the two
notions agree.)
- **E. Do not touch `NewsEngine._llm_sent_cache`.** That cache decides what is re-sent to DeepSeek. In
`fallback` mode a headline that failed DeepSeek must stay uncached so it is retried next window and
its archive row upgrades `model` → `llm` through the rank rule (news_archive.py:203-260).
- **F. Only `score_source == "keyword"` rows are touched.** Never overwrite an `llm` row in memory.
### 5.2 Call site (:944-947)
Replace
```python
scored = headlines + unmatched
try:
self._apply_llm_sentiment(scored)
except Exception as e:
logger.debug(f"LLM sentiment failed, keyword scores stand: {e}")
```
with
```python
scored = headlines + unmatched
mode = app_config.news_model_mode()
try:
if mode == "primary":
n = self._apply_model_sentiment(scored)
if n == 0 and news_model.get_news_model() is None:
# model unavailable this process: degrade to DeepSeek, not to keywords
self._apply_llm_sentiment(scored)
else:
self._audit_with_deepseek(scored) # no-op when audit_per_day == 0
elif mode == "fallback":
self._apply_llm_sentiment(scored)
self._apply_model_sentiment(scored) # only what DeepSeek left keyword-scored
else:
self._apply_llm_sentiment(scored)
except Exception as e:
logger.debug(f"headline sentiment enrichment failed, keyword scores stand: {e}")
```
`import news_model` and `import app_config` at the module top of news_engine.py (news_model's top
imports only numpy, §3.2).
Mode semantics:
| mode | DeepSeek for headlines | model | when a step fails |
|---|---|---|---|
| `off` | as today | never | keyword scores stand (today) |
| `fallback` | first, as today | on rows DeepSeek left keyword-scored | keyword scores stand |
| `primary` | never (except audit) | on all rows | model missing → DeepSeek path for the cycle; model raises mid-batch → keyword |
### 5.3 Audit sample (primary mode, optional feature, default off)
```python
_audit_day: str = "" # class attrs, like the caches
_audit_sent_today: int = 0
def _audit_with_deepseek(self, headlines):
cap = app_config.news_model_audit_per_day()
if cap <= 0 or not get_llm_client().available:
return
today = datetime.now(timezone.utc).date().isoformat()
if NewsEngine._audit_day != today:
NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0
room = cap - NewsEngine._audit_sent_today
if room <= 0:
return
cand = [h for h in headlines if h.score_source == "model"
and hashlib.sha1(h.headline.encode()).hexdigest() not in NewsEngine._llm_sent_cache]
batch = cand[:min(room, self._LLM_BATCH_CAP)]
if not batch:
return
scores = self._llm_headline_scores([h.headline for h in batch])
if scores is None:
return
NewsEngine._audit_sent_today += len(batch)
copies = []
for h, (overall, xau) in zip(batch, scores):
NewsEngine._llm_sent_cache[hashlib.sha1(h.headline.encode()).hexdigest()] = (overall, xau)
c = copy.copy(h) # the displayed object keeps the model score
c.sentiment_score = overall
c.sentiment_by_currency = {cur: (xau if cur == "XAU" else overall) for cur in h.currencies_mentioned}
c.xau_score = xau
c.score_source = "llm"
copies.append(c)
self._pending_audit_rows = copies # archived by the call site right after `scored`
```
The call site then archives `scored` first and `self._pending_audit_rows` second (same
`teacher_model`/`teacher_prompt_sha1` as today's DeepSeek rows); the rank rule upgrades those rows
to `llm`, the in-memory objects the bot displays keep the model's numbers. One batch per cycle at
most, ≤ `cap` per UTC day. This is the only place DeepSeek is called for headlines in primary mode.
### 5.4 Archive call (:958-964)
Pass, in addition to today's kwargs:
```python
model_teacher=news_model.teacher_tag() # "xau_transformer@<meta.trained_at>" or None
```
where `teacher_tag()` reads the loaded bundle's meta (None if not loaded). See §6.
### 5.5 Dataclass comment
`NewsSentiment.score_source` comment (:223-227): add `"model" = the local XAU transformer scored
xau_score (and XAU's per-currency score); other currencies remain keyword-scored.`
---
## 6. `news_archive.py` changes
No schema change. `_SOURCE_RANK` (:107) and `_STORED_RANK` (:203-206) already rank `model` = 2.
- `archive_headlines(headlines, *, teacher_model=None, teacher_prompt_sha1=None, origin="live",
model_teacher=None)` — new kwarg, passed through to `build_row`.
- `build_row` (:323-354):
```python
"teacher_model": (teacher_model if source == "llm"
else model_teacher if source == "model" else None),
"teacher_prompt_sha1": (teacher_prompt_sha1 if source in ("llm", "model") else None),
```
For a model row `teacher_prompt_sha1` is the DeepSeek prompt the model was distilled from (the same
constant the bot already passes), which is the truthful provenance.
- Effect: a retrain is visible in the data (`teacher_model` changes), and the UPSERT's
`COALESCE(excluded.teacher_model, headlines.teacher_model)` (:253-260) only applies when the new
rank ≥ stored rank, so an `llm` row's teacher is never replaced by a model tag.
---
## 7. Tests (`test_all.py`)
The suite is a plain script: `@test(name)` (:51-64) runs each function at definition time and
tallies; the file ends with `sys.exit(1 if FAIL > 0 else 0)`. Run: `venv/bin/python test_all.py`.
Stubbing patterns in use: instance-level `ne._llm_headline_scores = lambda texts: …` with
`lc._client = _StubLLM({...})` and `finally` cleanup (:5196-5215); module-level swap
`saved = _ne_mod.get_llm_client; _ne_mod.get_llm_client = stub; … finally restore` (:11391-11424).
Add, using the same patterns and a fake bundle (`news_model.score_batch` swapped on the module):
1. **model scores keyword rows only.** Two `NewsSentiment`s, one `score_source="llm"`; after
`_apply_model_sentiment`, the llm one is byte-identical, the keyword one has `score_source="model"`,
`xau_score` set, `sentiment_by_currency["XAU"]` = score, `["USD"]` unchanged (rule A).
2. **overall score rule (B).** `currencies_mentioned=["XAU"]` → `sentiment_score` = model score;
`["USD","XAU"]` → `sentiment_score` unchanged; `[]` → model score.
3. **relevance zeroing (D).** stub returns `relevant=False, score=-0.18` → `sentiment_by_currency["XAU"]
== 0.0`, `xau_score == -0.18`.
4. **cache untouched (E).** `_llm_sent_cache` empty before and after.
5. **failure isolation.** stub raises → all rows still `keyword`, method returns 0, no exception.
`news_model.get_news_model()` returning None → returns 0.
6. **primary degrades to DeepSeek.** `get_news_model` → None, `_llm_headline_scores` stubbed → rows
end up `llm`.
7. **audit sample.** `deepseek_audit_per_day=3`, 10 model rows → exactly 3 sent, displayed objects
still `model`, `_pending_audit_rows` has 3 `llm` copies; second call same day sends 0.
8. **config.** unknown key in `news_model` → `AppConfigError`; `mode="banana"` → error; section
absent → `news_model_mode() == "off"`; `NEWS_MODEL_MODE=off` env beats `"mode": "primary"`.
9. **archive.** `build_row` of a `model` headline with `model_teacher="xau_transformer@T"` stores it;
an `llm` row stores the DeepSeek teacher; existing rank tests still pass.
10. **import hygiene.** update the docstring of the test at :11437-11462 (it says numpy-only); add
an assertion that `requirements.txt` lists `onnxruntime` and `tokenizers`; keep its "no torch /
transformers" checks.
11. **integration, skipped when the bundle is absent** (`os.environ.get("NEWS_MODEL_DIR")` or the
configured path missing → print SKIP): load, `score_news(b, "Fed cuts rates, signals more
easing")["label"] == "bullish"`, `score_news(b, "Red Lobster endless shrimp losses")["label"] ==
"neutral"`, latency of 20 calls < 1 s.
`test_all.py:3675-3690` loads `config.json.example` for pair output only; adding the section there is
safe.
---
## 8. Ops checklist → `deploy/README-news-model.md` (new)
Covers **both** processes: the bot (`telegram_bot.py`) and the dashboard daemon
(`deploy/gold-dashboard.service`).
0. `free -m`. Each model-loading process needs ~0.7 GB headroom. If only the bot should score with
the model, add `Environment=NEWS_MODEL_MODE=off` to `deploy/gold-dashboard.service` (the dashboard
then keeps using DeepSeek for its headlines; those `llm` rows land in the shared archive and act
as free audit labels).
1. `cd /home/Forex/ForexBot2 && git pull && venv/bin/pip install -r requirements.txt`.
2. `mkdir -p /home/Forex/models/xau_transformer` and copy `model.onnx`, `tokenizer.json`, `meta.json`
from the ML-NEWS machine (rsync, ~280 MB). Verify `sha256sum model.onnx` matches the source.
Owner: the account the bot runs as; read-only is enough.
3. Add the `news_model` section to `config.json` (§4.1) with `"mode": "primary"`. Leave the
`deepseek` section unchanged.
4. Smoke test without starting the bot:
```
venv/bin/python -c "import news_model as m; b=m.get_news_model(); print(m.score_news(b,'Gold jumps as dollar slides'))"
```
Expect `label: bullish`, and the "news model loaded" log line.
5. `venv/bin/python test_all.py` — green (integration test runs because the dir exists).
6. Restart the bot; restart the dashboard unit. Check:
- log: one "news model loaded" line per process, no "news model unavailable";
- `top`: RSS ~+0.6 GB per process, idle CPU ~0% between scans (spinning disabled);
- archive: new rows have `score_source='model'`, `xau_score` set, `teacher_model`
`xau_transformer@…`; DeepSeek daily counter for headlines stays at 0 (audit off).
7. Rollback: set `"mode": "fallback"` or `"off"` (or the env var), restart. No data migration needed.
8. Retrain cadence: when ML-NEWS ships a new bundle, copy the three files over the old ones and
restart both processes. `meta.json.trained_at` changes, so the archive shows which model scored which
rows. If `deepseek_audit_per_day` is set, ML-NEWS's `eval/eval_archive.py --scorer transformer` on a
copy of the live archive gives the model-vs-DeepSeek agreement on those rows.
---
## 9. Acceptance criteria (end-to-end)
- With `mode="primary"` and DeepSeek enabled, one live news cycle archives only `model` rows (plus
≤ `deepseek_audit_per_day` `llm` rows), and the cycle's added latency is < 1 s for ~100 headlines.
- With the model directory renamed away, both processes start, log one warning each, and the cycle's
rows are `llm` (DeepSeek path), not `keyword`.
- With `onnxruntime` uninstalled in a scratch venv, `python -c "import news_engine"` succeeds.
- A headline naming USD and XAU ends with `sentiment_by_currency["USD"]` equal to the keyword value
and `["XAU"]` equal to the model value.
- `mode="off"` reproduces today's archive rows exactly (`llm`/`keyword`, same teacher fields).
- `venv/bin/python test_all.py` exits 0.
## 10. Known limits to state in the README
- The model copies DeepSeek's opinion (72% agreement on unseen live days, 72/78 golden); it does not
learn from price. DeepSeek itself is ~90% self-consistent.
- Gold only. Every other currency's sentiment remains the ~31%-accurate keyword scorer in primary
mode, which is what those currencies get today whenever DeepSeek is unavailable.
- With audit off, no new DeepSeek labels accrue; the ML-NEWS retrain loop needs either
`deepseek_audit_per_day > 0`, the dashboard left on DeepSeek, or a separate labelling run.
+49 -36
View File
@@ -35,70 +35,83 @@
<body>
<h1>ForexBot2 — User Guide</h1>
<div class="subtitle">How to read, act on, and survive the signals &nbsp;·&nbsp; July 2026</div>
<div class="subtitle">How to read, act on, and survive the signals &nbsp;·&nbsp; updated October 2026</div>
<h2>1 · What the bot does</h2>
<p>ForexBot2 analyzes <b>gold (XAU/USD)</b> around the clock on several chart timeframes at once. On every candle close it scores nine technical indicators (trend, momentum, volatility, divergences, candlestick patterns), adapts the weighting to the current market regime (trending / ranging / transitioning), checks that higher timeframes agree, adjusts for news and economic-calendar events, and — only when everything lines up — sends a <b>signal with a complete trade plan</b> to Telegram: direction, entry, stop loss, target, and a trailing-stop management plan.</p>
<p>Most of the time it sends nothing. Silence means no edge was found; that is the bot working correctly, not a malfunction.</p>
<h2>2 · Anatomy of a signal</h2>
<div class="signal-box">🔍 XAU/USD Signal Analysis (M15)
🕐 2026-07-10 14:00 UTC
💰 Close: 4,132.50
📊 Regime: Trending Up | Vol: normal
━━━━━━━━━━━━━━━━━━━━
🟢 BUY 🅰 A | Score: +38.2 | 64% conf
✅ MTF aligned
📈 MTF: M5 ✅ H1 ✅
📋 Indicators … (9 rows)
⚠️ EXTREME volatility — wider stops needed
━━━━━━━━━━━━━━━━━━━━
<div class="signal-box">🟢 Grade A · STRONG BUY · XAU/USD · 15min 🥇
🕐 2026-09-04 08:00 UTC · 💰 4483.41
📐 Direction: strong buy · Reliability: high · timeframes agree
📊 Regime: Transitioning · Vol: high
🧭 Daily trend: rising — this buy goes with it
STRONG BUY — High-quality setup, consider entry
🚀 TRADE SETUP
📈 Direction: BUY
▶ Entry: 4,132.50
🛑 SL: 4,103.10 ($29.40 / 294 pips)
🎯 TP1: 4,161.90 ($29.40 / 294 pips)
⚖️ RR: 1:1.0
🔄 Trail: once price hits 4,150.14 (+0.6R),
trail SL $22.05 behind best price —
TP1 is a guide, let it run</div>
▶ Entry 4483.41
🛑 SL 4468.92 ($14.49 / 145p = 1R)
📏 Size: 0.06 lots risks $86.94 at the stop
(your limit $100.00 per trade) ← owner's chats only
🧱 Already open: 2 buy plans alerted earlier (2 on M15)
— taking this one adds to the same position
🔄 Exit rule (tested): once price reaches
4493.55 (+0.7R), trail the SL $10.87
behind the best price, moving it at
candle close only. Otherwise the SL is the exit.
🎯 TP1 guide 4497.90 ($14.49 / 145p = 1.0R)
🎯 TP2 guide 4505.15 (1.5R)
⚖️ Planned TP1 RR 1:1.0 (guide only — the trail is the exit)
📚 Book (this setup class): 58 trades, 62% winners, avg +0.21R per trade
📈 Other timeframes: 15min: STRONG BUY +46.5 | 5min: BUY +22.8 | 1h: NEUTRAL +14.2
📋 EMA⇈ RSI→ BB↓ ADX→ MACD⇈ ROC→ KC⇈</div>
<table>
<tr><th style="width:28%">Field</th><th>What it means</th></tr>
<tr><td><b>Signal + grade</b></td><td>Direction (BUY / SELL / STRONG_*) plus a quality grade <b>A+ → F</b>. A trade setup is only attached at grade C or better. Higher grades = more indicator agreement and cleaner regime.</td></tr>
<tr><td><b>Score / confidence</b></td><td>The regime-weighted composite of all nine indicators. Bigger absolute score = stronger consensus.</td></tr>
<tr><td><b>MTF aligned</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr>
<tr><td><b>Direction / reliability line</b></td><td>Since 2026-09-27 the alert prints the two raw numbers in words. <i>Direction</i> is the regime-weighted composite of all nine indicators: <b>very strong</b> (|score| ≥ 40), <b>strong</b> (≥ 33), <b>clear</b> (≥ 25), <b>weak</b> (≥ 15) or <b>no clear direction</b>, with buy or sell from its sign. <i>Reliability</i> is the confidence after the penalties: <b>high</b> (≥ 65), <b>good</b> (≥ 55), <b>fair</b> (≥ 45) or <b>low</b>. The cut-offs are the grade table's own, so the words never disagree with the grade. The numbers themselves remain in <code>--json</code>, <code>--detail</code> and the dashboard meters.</td></tr>
<tr><td><b>Timeframes agree</b></td><td>The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5.</td></tr>
<tr><td><b>Daily trend 🧭</b></td><td>The direction of the daily chart, from closed daily candles only: <b>rising</b> when the last daily close is above its 50-day average <i>and</i> above the close 60 trading days earlier, <b>falling</b> when it is below both, otherwise <b>no clear direction</b>. The line also says whether the plan goes with it or against it. <b>On M15 a plan against a clear daily trend is not sent at all</b> (since 2026-09-30): in the three-year test those plans earned nothing after costs, and dropping them cut the worst losing run from 20.9R to 12.4R while the total rose from +195.8R to +205.1R; over 2019–2026 the worst run fell from 64.8R to 41.5R. On M5, H1 and H4 the same rule was tested and did not pass, so there the line is context only — a plan "against" the daily trend is still a valid signal on those timeframes.</td></tr>
<tr><td><b>Size 📏</b></td><td>How many lots keep the loss at the printed stop inside the risk limit set in <code>config.json</code> (<code>trade_size.risk_per_trade</code>, in US dollars; 1 lot = 100 ounces unless <code>contract_size</code> says otherwise, and the lot step is 0.01 unless <code>lot_step</code> says otherwise). The size is rounded <b>down</b> to the broker's lot step, so the money at risk is at or under the limit, never over. When even the smallest lot would break the limit the line says so and gives the amount; skip that trade or accept the larger risk knowingly. The line appears only when the limit is set, and only in the owner's own chat with the bot and on the command line: the channel, any group the bot posts in and the public dashboard do not show it unless <code>trade_size.telegram_channel</code> / <code>trade_size.dashboard</code> are switched on.</td></tr>
<tr><td><b>Already open 🧱</b></td><td>How many plans in the <i>same direction</i> the bot alerted earlier are still open in its own outcome tracking (stop not hit, trail not closed), and on which timeframes. Taking every alert stacks positions on one move of gold: in the three-year M5 + M15 test two entries in three already had a same-direction plan open and one in four had three or more. Those stacked entries were not worse trades, so the line is not a reason to skip; it is a reminder that the total at risk is the sum of every open plan, not the risk of this one alert. The count comes from the bot's model of the plans it sent, not from your broker account. Alerts sent at the same candle close are counted too: when the 5-minute and the 15-minute alert go out together, the second one already shows the first.</td></tr>
<tr><td><b>Warnings ⚠️</b></td><td>Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings.</td></tr>
<tr><td><b>Violent-move watch 🌪</b></td><td>An occasional extra line saying the next hour looks unusually likely to produce a very large candle — because a high-impact US release is due (or just landed), because the clock is in a historically wild slot, or because price is stretched outside its bands. It is a heads-up about <i>volatility</i>, not a direction call, and it <b>never</b> changes the signal, the grade or the trade setup — the same alert would be sent without it. Do not close or tighten an open trade because of it: every such protective action was tested over three years and lost money. Your stop is the protection.</td></tr>
<tr><td><b>AI check 🤖</b></td><td>An advisory block written for a new trader, such as <i>"AI check: the odds look worse than usual. New data just came out against this buy. Chance it pays off: about 45 in 100 (usual for this kind of setup: 60 in 100). For: Before the news, the price was climbing. Against: Fresh data just came out against gold. This check is an experiment. Do not trade on it alone."</i> The first sentence compares the estimate with the usual odds for this kind of setup (8 points either way counts as a difference); there is no verdict word and no confidence percentage. One AI reviewer (the news reader) receives the recent headlines together with facts computed by code — how far the move has already run, whether a very large candle just printed, how the stop and target compare with the usual, how trades of this exact kind fared in the two-year test, and whether major scheduled news lands before the trade would normally finish — and gives a probability that the trade ends in profit plus one reason for and one against. (Three further reviewers, chart, risk and sceptic, were switched off on 2026-09-17: in two evaluations they added nothing beyond the base rate and mostly voted the same way on every setup.) It is <b>not</b> a recommendation and it <b>never</b> changes the signal, the grade or the setup. Until about 300 of these estimates have been graded against real outcomes, treat it as context, not as a filter: the earlier AGREE/OPPOSE version of this line opposed target-reaching trades as often as it agreed.</td></tr>
<tr><td><b>Market context 📰</b></td><td>An AI-written note in plain English on what is moving the gold price right now, why it matters, and the next report to watch. It is built only from headlines published in the last 4 hours that lean one way (neutral items are dropped) plus today's high-impact releases. Informational only — it never blocks or grades a signal.</td></tr>
<tr><td><b>SL / TP1 / RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer. TP1 is the nearest opposing level. <b>1R</b> = the entry-to-SL distance = the amount you risk.</td></tr>
<tr><td><b>Trail plan 🔄</b></td><td>The real exit. Once price moves +0.6R in your favor, move the stop to trail 0.75R behind the best price and update it at each candle close. <b>TP1 is advisory — let winners run.</b></td></tr>
<tr><td><b>SL / TP1 guide / planned RR</b></td><td>Stop loss sits behind the nearest swing level with an ATR buffer (grade A+/A setups use a 2 × ATR stop). <b>1R</b> = the entry-to-SL distance = the amount you risk. TP1 is a <i>guide</i> (the nearest opposing level or a multiple of ATR, never printed closer than 1R since 2026-09-26); the "planned TP1 RR" is TP1 ÷ 1R and is always at least 1:1. It is <b>not</b> the expected payoff: the tested exit is the trail, and the validated books earn their profit from trailed runners (M5 average win about +1.0R against an average loss of −0.76R; M15 +0.87R against −0.93R).</td></tr>
<tr><td><b>Exit rule 🔄</b></td><td>The real, backtested exit. Once price moves the stated R in your favor (0.6R on M5, 0.7R on M15), move the stop to trail the stated $ distance behind the best price and update it at each candle close only. Until then the original SL is the exit. <b>TP1 is advisory — let winners run.</b></td></tr>
<tr><td><b>Book line 📚</b></td><td>What this class of setup (timeframe, setup type, grade, direction) actually did in the validated two-year backtests: number of trades, share of winners and average R per trade. When the exact class has fewer than 30 trades the line falls back to all setups of that direction on the timeframe, then to the whole timeframe.</td></tr>
<tr><td><b>One alert per move</b></td><td>After an alert, the bot sends no new alert on that timeframe for the next 5 candles (25 min on M5, 75 min on M15), the same entry cooldown the backtester applies. Consecutive-bar re-entries into the same move were the largest live-only loss in September 2026. The five candles are candles that actually traded: the daily one-hour pause and the weekend do not use them up, so the first candle after the pause is still held if fewer than five have printed since the alert. The bot also remembers its last alert across a restart.</td></tr>
</table>
<h2>3 · Acting on a signal — step by step</h2>
<ol>
<li><b>Size the position first.</b> Decide your risk per trade — <b>0.5–1% of account, never more</b>. Position size = (account × risk%) ÷ stop distance in $. The stop distance varies per signal, so the size changes every trade; the dollar risk must not.</li>
<li><b>Size the position first.</b> Decide your risk per trade — <b>0.5–1% of account, never more</b>. Position size = (account × risk%) ÷ stop distance in $. The stop distance varies per signal, so the size changes every trade; the dollar risk must not. With <code>trade_size.risk_per_trade</code> set in <code>config.json</code> the alert does this sum for you (the 📏 line). If the 🧱 line shows plans already open in the same direction, the new trade adds to them: keep the <i>total</i> at risk across all open plans inside what you can afford on one move.</li>
<li><b>Enter promptly at market.</b> All backtested results assume entry at the open of the candle after the signal. On M5, minutes matter (see §4). If you are very late — price already moved a large part of the way to the arm level — skip the trade. There is always another signal.</li>
<li><b>Place the stop loss immediately</b> at the given SL. No mental stops — gold gaps and spikes.</li>
<li><b>Manage with the trail, not TP1.</b> When price reaches the arm level printed in the message, start trailing the stop the stated $ distance behind the best price, tightening only at candle close. This is what produced the backtested results; taking fixed TP1 profits gives most of the edge away.</li>
<li><b>One signal, one trade.</b> The M5, M15 and H1 books confirm off each other and are usually long or short gold at the same time. Taking the same move on multiple timeframes multiplies risk, not diversification. Pick your timeframe (§4) and ignore the others.</li>
<li><b>One signal, one trade.</b> The M5, M15 and H1 books confirm off each other and are usually long or short gold at the same time. Taking the same move on multiple timeframes multiplies risk, not diversification. Pick your timeframe (§4) and ignore the others. The 🧱 "already open" line tells you when an alert would add to plans that are still running.</li>
</ol>
<h2>4 · Choosing your timeframe</h2>
<p>Twelve-month Oanda-candle backtest (Aug 2025 – Aug 2026, $0.65 round-trip cost, entry at next candle open):</p>
<p>Twelve-month Oanda-candle backtest with the rules as they run today (Aug 2025 – Aug 2026, $0.65 round-trip cost, entry at next candle open; regenerated 2026-10-01):</p>
<table>
<tr><th></th><th>M5</th><th>M15</th><th>H1</th></tr>
<tr><td>Trades / week</td><td>~11.3</td><td>~7.6</td><td>~2.7</td></tr>
<tr><td>Total result (12 mo)</td><td>+106.8R</td><td>+73.9R</td><td>+24.0R</td></tr>
<tr><td>Win rate / profit factor</td><td>52.9% / 1.51</td><td>62.2% / 1.53</td><td>60.9% / 1.55</td></tr>
<tr><td>Typical (median) hold</td><td>2.4 h</td><td>7 h</td><td>15.5 h</td></tr>
<tr><td>90% of trades close within</td><td>10.5 h</td><td>29.8 h</td><td>3.2 days</td></tr>
<tr><td>Typical stop distance</td><td>~$21</td><td>~$32</td><td>~$52</td></tr>
<tr><td>Worst drawdown</td><td>10.7R</td><td>10.8R</td><td>8.0R</td></tr>
<tr><td>Trades / week</td><td>~10.7</td><td>~5.1</td><td>~1.7</td></tr>
<tr><td>Total result (12 mo)</td><td>+108.8R</td><td>+92.3R</td><td>+22.2R</td></tr>
<tr><td>Win rate / profit factor</td><td>53.6% / 1.54</td><td>68.0% / 2.12</td><td>62.6% / 1.79</td></tr>
<tr><td>Typical (median) hold</td><td>2.5 h</td><td>7.8 h</td><td>21 h</td></tr>
<tr><td>90% of trades close within</td><td>11.2 h</td><td>29.4 h</td><td>3.2 days</td></tr>
<tr><td>Typical stop distance</td><td>~$21</td><td>~$32</td><td>~$48</td></tr>
<tr><td>Worst drawdown</td><td>10.1R</td><td>8.3R</td><td>4.1R</td></tr>
</table>
<p class="small">This was a strongly rising year for gold until March 2026 and a falling, choppy one after. Four fifths of the M5 total (+89.6R of +108.8R) came in January–March 2026; M15 earned in every quarter but only +2.3R since July. Over 2019–2026 the M15 rules made +223R with one losing year (2021, −27R) and two years close to flat (2022 and 2023); the M5 rules took few trades and made nothing before 2025. Judge the next twelve months by the long record, not by this table.</p>
<ul>
<li><b>M5 — intraday, execution-critical.</b> Highest total edge, but stops are tight: a couple of dollars of slippage or a late entry eats a meaningful share of every trade. Suitable only if you can act within a minute or two of the alert (or automate). Trades open and close within a session; you sleep flat. SELL signals are held back while the market regime reads "transitioning" — a validated filter, not a bug if you notice fewer shorts there.</li>
<li><b>M15 — the middle path.</b> Same frequency as M5, wider stops, up to 15 minutes to act. About half your trades will still be open overnight. Since 2026-09-09 the M15 scoring gives its RSI reading half the weight it used to (RSI mostly voted against the trend on this timeframe), so expect roughly 9 signals a week instead of 7–8; in the two-year test (Sep 2024 – Sep 2026) that change lifted M15 from +117.6R to +146.1R with the same win rate and a slightly better profit factor, and it also improved the separate out-of-sample period. The M15 column above predates it.</li>
<li><b>M15 — the middle path.</b> Same frequency as M5, wider stops, up to 15 minutes to act. About half your trades will still be open overnight. Since 2026-09-09 the M15 scoring gives its RSI reading half the weight it used to (RSI mostly voted against the trend on this timeframe), so expect roughly 9 signals a week instead of 7–8; in the two-year test (Sep 2024 – Sep 2026) that change lifted M15 from +117.6R to +146.1R with the same win rate and a slightly better profit factor, and it also improved the separate out-of-sample period. Since 2026-09-26 grade-A shorts are blocked on M15, and since 2026-09-30 any M15 plan against a clear daily trend is dropped (the 🧭 line, §2). Together these leave about 5 signals a week. The daily-trend rule is a trade of profit for safety: it removes about a quarter of the trades, the ones that earned nothing after costs, and shortens the losing runs; in a stretch where gold only rises it gives up a little (Sep 2024 – Sep 2025: +61.8R instead of +70.1R).</li>
<li><b>H1 — swing trading around a normal life.</b> Best quality per trade and very forgiving of slow manual execution (you have up to an hour). The costs: positions held for days, through news and overnight sessions, and a <b>lumpy</b> equity curve — one quarter of the test year was negative and most of the profit came in a single quarter. Judge it on quarters, not weeks. H1 no longer issues SELL signals at all (blocked since 2026-09-03) — it is BUY-only on this timeframe.</li>
</ul>
<div class="callout"><b>If you trade manually from the Telegram alerts, H1 (or M15) is the honest choice.</b> M5's backtested edge assumes near-instant fills that a human copying signals rarely achieves.</div>
@@ -107,7 +120,7 @@
<ul>
<li><b>You will lose roughly 38–47% of trades.</b> A 53–62% win rate means losing streaks of 5+ are routine over a year. The system's edge is the average across hundreds of trades, not any single one.</li>
<li><b>Stops hit fast, winners take time.</b> Losing trades die in roughly half the time winners take. An early stop-out is the system doing its job.</li>
<li><b>Drawdowns of ~10R are normal.</b> At 1% risk per trade, expect to be ~10% down from a peak at some point in a normal year. This is why risking more than 1% per trade is how accounts die.</li>
<li><b>Drawdowns of ~10R are normal, and worse has happened.</b> At 1% risk per trade, expect to be ~10% down from a peak at some point in a normal year. Over 2019–2026 the M15 rules as they run today had one losing run of 41.5R that took from late 2020 to early 2022 to bottom out, while gold drifted sideways and down. This is why risking more than 1% per trade is how accounts die.</li>
<li><b>Backtest ≠ live.</b> Results above charge real spreads and realistic fills, but cannot model your personal latency, broker slippage, or requotes. Assume live results will be somewhat worse — especially on M5.</li>
<li><b>Quiet periods are normal.</b> Filters block bad hours, bad regimes and misaligned timeframes. Days can pass without a signal.</li>
</ul>
@@ -130,7 +143,7 @@
<tr><td><code>/start</code></td><td>Show the button menu (manual scans per timeframe, Scan All TFs).</td></tr>
<tr><td><code>/scan</code></td><td>Run an immediate analysis of the primary timeframe.</td></tr>
<tr><td><code>/autoscan</code></td><td>Toggle automatic scanning (M5 / M15 / H1 / H4 at each candle close).</td></tr>
<tr><td><code>/status</code></td><td>Uptime, auto-scan state, WebSocket price-monitor state.</td></tr>
<tr><td><code>/status</code></td><td>Uptime, auto-scan state, WebSocket price-monitor state, and the health of the news sources: the calendar, the headline feeds, the local headline model (a ⚠️ line "News model: not scoring" means this cycle's headlines carry the simple keyword scores) and DeepSeek.</td></tr>
<tr><td><code>/news</code></td><td>Net XAU/USD news score from the last 4 hours of headlines (neutral headlines are removed before averaging; informational only — never gates signals).</td></tr>
<tr><td><code>/cache</code> · <code>/clearcache</code></td><td>Inspect / reset the market-data cache.</td></tr>
<tr><td><code>/help</code></td><td>Command reference.</td></tr>
+160
View File
@@ -0,0 +1,160 @@
"""
ForexBot2 — trade size and open-exposure lines (display only)
==============================================================
Two lines a surface prints next to a trade setup. Nothing here is read by
the signal core, a gate or the backtester:
* size line — how many lots keep the loss at the stop inside the risk the
owner set in config.json (`trade_size.risk_per_trade`). Off until that
is set. The size reveals the owner's risk per trade, so it goes to the
owner's own chat (telegram.chat_id) and the CLI only, unless
`trade_size.telegram_channel` (every other Telegram chat: the channel,
a group) / `trade_size.dashboard` switch it on for the public surfaces.
* open line — how many ALERTED plans in the same direction are still open
in the advisory outcome model (logs/signals.db, read-only). Taking every
alert stacks positions: on the 3-year M5 + M15 books two entries in
three already have a same-direction plan open, one in four has three or
more, and up to 13 are open at once
(backtest/research/daily_trend_campaign_2026-09-30.md, section 5.11).
Those stacked entries did not pay worse; the point of the line is that
the full risk on each alert is several times that risk on one move.
Attached post-hoc by the surface (Telegram bot, CLI, dashboard), the same
pattern as violent_watch.attach_watch. Never raises: a display line must
not break a scan or an alert.
"""
import logging
from decimal import ROUND_FLOOR, Decimal, InvalidOperation
from typing import Any, Dict, List, Optional, Tuple
import app_config
import signal_log
logger = logging.getLogger(__name__)
# Surfaces a size line may be attached for, and the config flag that
# shares it with a public audience (None = the owner's own surface).
SURFACES = {"telegram": None, "cli": None, "dashboard": "dashboard"}
_TF_LABEL = {"5min": "M5", "15min": "M15", "1h": "H1", "4h": "H4"}
def lot_size(stop_distance: float, risk: float, contract_size: float = 100.0,
lot_step: float = 0.01) -> Tuple[float, float]:
"""(lots, money at risk with those lots): the largest multiple of
`lot_step` whose loss at the stop stays within `risk`.
One lot is `contract_size` ounces, so a $1 move is worth
`contract_size` dollars per lot. Decimal arithmetic with a floor, so
the size never rounds UP past the risk limit and 0.07 never prints as
0.0699. (0.0, 0.0) when any input is unusable."""
try:
d, r = Decimal(str(stop_distance)), Decimal(str(risk))
c, s = Decimal(str(contract_size)), Decimal(str(lot_step))
except (InvalidOperation, ValueError):
return 0.0, 0.0
if not (d.is_finite() and r.is_finite() and c.is_finite() and s.is_finite()):
return 0.0, 0.0
if d <= 0 or r <= 0 or c <= 0 or s <= 0:
return 0.0, 0.0
steps = (r / (d * c * s)).to_integral_value(rounding=ROUND_FLOOR)
lots = steps * s
return float(lots), float(lots * d * c)
def _decimals(step: float) -> int:
text = format(Decimal(str(step)).normalize(), "f")
return len(text.split(".")[1]) if "." in text else 0
def size_line(setup: Any, settings: Optional[Dict[str, Any]]) -> str:
""""Size: 0.05 lots risks $99.00 at the stop (your limit $100.00 per
trade)", or "" when the size is not configured or the setup has no
usable stop.
The stop distance is taken from the prices as they are printed (two
decimals, subtracted exactly), so the line agrees with the entry and
stop above it and a $20.00 stop is never read as $20.000000000000004."""
if setup is None or not settings:
return ""
try:
risk = float(settings.get("risk_per_trade") or 0.0)
contract = float(settings.get("contract_size", 100.0))
step = float(settings.get("lot_step", 0.01))
distance = abs(Decimal(f"{float(setup.entry_price):.2f}")
- Decimal(f"{float(setup.stop_loss):.2f}"))
except (TypeError, ValueError, InvalidOperation):
return ""
if not risk > 0 or not distance.is_finite() or not distance > 0:
return ""
lots, at_risk = lot_size(distance, risk, contract, step)
places = _decimals(step)
if lots <= 0:
smallest = float(distance) * contract * step
if not smallest > 0:
return ""
return (f"Size: the stop is too wide for your ${risk:,.2f} limit — the smallest "
f"size ({step:.{places}f} lots) would risk ${smallest:,.2f}")
return (f"Size: {lots:.{places}f} lots risks ${at_risk:,.2f} at the stop "
f"(your limit ${risk:,.2f} per trade)")
def open_line(rows: Optional[List[Dict[str, Any]]], direction: str) -> str:
""""Already open: 2 buy plans alerted earlier (M5, M15) — taking this
one adds to the same position", or "" when no alerted plan in
`direction` is still open."""
same = [r for r in (rows or []) if r.get("direction") == direction]
if not same:
return ""
counts: Dict[str, int] = {}
for r in same:
label = _TF_LABEL.get(str(r.get("timeframe")), str(r.get("timeframe")))
counts[label] = counts.get(label, 0) + 1
where = ", ".join(f"{n} on {tf}" if n > 1 else tf for tf, n in counts.items())
n = len(same)
return (f"Already open: {n} {direction.lower()} plan{'s' if n != 1 else ''} alerted "
f"earlier ({where}) — taking this one adds to the same position")
def size_shared(flag: str) -> bool:
"""True when config.json shares the size line with a public audience
(`flag` = "telegram_channel" or "dashboard"). Never raises."""
try:
settings = app_config.trade_size_settings()
return bool(settings and settings.get(flag))
except Exception:
return False
def attach_exposure(report: Any, surface: str = "telegram") -> Any:
"""Stamp report.size_line and report.open_line for `surface`
("telegram" / "cli": the owner's own view; "dashboard": the size only
when trade_size.dashboard is true). Display only; never raises."""
report.size_line = ""
report.open_line = ""
setup = getattr(report, "trade_setup", None)
if setup is None:
return report
try:
settings = app_config.trade_size_settings()
flag = SURFACES.get(surface, "unknown-surface")
if settings and (flag is None or settings.get(flag)):
report.size_line = size_line(setup, settings)
except Exception:
logger.debug("exposure: size line unavailable", exc_info=True)
try:
# The same identity the logger stores for this scan's row, so the
# plan being alerted now never counts itself (a re-alert of an
# unchanged plan is the same position, not a second one).
analysis = getattr(report, "primary_analysis", None)
key = signal_log.setup_key(
{"direction": setup.direction, "entry": setup.entry_price,
"stop_loss": setup.stop_loss,
"strategy": getattr(setup, "strategy", None)},
analysis.timeframe if analysis is not None else "")
rows = signal_log.open_alerted_setups(report.symbol, exclude_key=key)
report.open_line = open_line(rows, setup.direction)
except Exception:
logger.debug("exposure: open line unavailable", exc_info=True)
return report
+55 -8
View File
@@ -53,7 +53,10 @@ class DeepSeekClient:
daily_cap: int = DEFAULT_DAILY_CAP,
enabled: bool = True,
):
self._api_key = api_key or ""
# Whitespace around a key is never part of it, and a trailing line
# break (a key pasted from a file) is not a legal header value:
# every call would fail before it left the process.
self._api_key = (api_key or "").strip()
self._model = model
self._timeout = timeout
self._daily_cap = daily_cap
@@ -104,17 +107,36 @@ class DeepSeekClient:
self._count += 1
return True
def _redact(self, text: Any) -> str:
"""Exception text that is safe to log: never contains the key."""
out = str(text)
return out.replace(self._api_key, "<api key>") if self._api_key else out
def chat_json(self, system_prompt: str,
user_payload: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""One chat call; returns the model's JSON object or None."""
if not self.available or not self._under_cap():
"""One chat call; returns the model's JSON object or None.
Never raises: every caller reads None as "use the fallback", and
the news enrichment runs inside a live scan."""
if not self.available:
return None
try:
content = json.dumps(user_payload, sort_keys=True)
except (TypeError, ValueError) as e:
# Before the cap is counted: a payload that cannot be sent
# must not spend one of the day's requests.
self.last_error = (time.time(), "payload")
self._warn_once(
"payload", "DeepSeek payload is not JSON-serialisable "
"(%s) — call skipped", type(e).__name__)
return None
if not self._under_cap():
return None
body = {
"model": self._model,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user",
"content": json.dumps(user_payload, sort_keys=True)},
{"role": "user", "content": content},
],
"response_format": {"type": "json_object"},
"temperature": 0,
@@ -133,19 +155,44 @@ class DeepSeekClient:
try:
resp = self._session.post(
DEEPSEEK_URL, json=body, timeout=self._timeout)
except requests.exceptions.InvalidHeader:
# requests quotes the offending header VALUE in this error, and
# the only header built from configuration is Authorization:
# the text would put the API key in the log. Fixed wording.
self.last_error = (time.time(), "header")
self._warn_once(
"header", "DeepSeek call not sent: the API key holds a "
"character that is not allowed in an HTTP header (a line "
"break?) — check deepseek.api_key / DEEPSEEK_API_KEY")
return None
except requests.RequestException as e:
self.last_error = (time.time(), "network")
self._warn_once("network", "DeepSeek call failed: %s", e)
self._warn_once("network", "DeepSeek call failed: %s",
self._redact(e))
return None
except Exception as e:
# Anything else raised inside the HTTP stack (a header that
# cannot be encoded, a transport bug) must not reach a scan
# either. Type name only: the text may quote the request.
self.last_error = (time.time(), "client")
self._warn_once("client", "DeepSeek call failed inside the "
"HTTP client: %s", type(e).__name__)
return None
if resp.status_code != 200:
self.last_error = (time.time(), f"http_{resp.status_code}")
self._warn_once(
f"http_{resp.status_code}",
"DeepSeek HTTP %s: %s", resp.status_code, resp.text[:200])
"DeepSeek HTTP %s: %s", resp.status_code,
# Redact BEFORE cutting: a gateway error page can echo the
# request headers, and a key cut at the 200-character
# mark would be left partly visible.
self._redact(resp.text)[:200])
return None
try:
parsed = self._parse_response(resp.json())
except ValueError:
except Exception:
# ValueError for a body that is not JSON; anything else (a
# pathological nesting depth) is just as unusable.
parsed = None
if parsed is None:
self.last_error = (time.time(), "parse")
+113 -4
View File
@@ -23,6 +23,7 @@ if hasattr(sys.stderr, "reconfigure"):
from agent_panel import attach_panel
from app_config import get_app_config, violent_watch_enabled
from config import PlatformConfig, Timeframe, SignalStrength, XAUUSD
from exposure import attach_exposure
from pair_signals import PAIR_PRODUCERS
from signal_aggregator import SignalAggregator, SignalReport
from violent_watch import attach_watch
@@ -105,6 +106,63 @@ def _fmt_price(price: float) -> str:
return f"{price:.2f}"
# ── Plain words for the two raw numbers (owner request 2026-09-27) ────
# The words sit on the grade table's own floors (CLAUDE.md §2: |score|
# 40 / 33 / 28 / 25 / 15, confidence 65 / 55 / 50 / 45), so they can never
# disagree with the printed grade. Display only: no gate reads them, and the
# numbers stay in --json, --detail and the dashboard meters.
def direction_words(score: float) -> str:
strength_of = abs(score)
side = "buy" if score > 0 else "sell" if score < 0 else ""
if strength_of >= 40:
strength = "very strong"
elif strength_of >= 33:
strength = "strong"
elif strength_of >= 25:
strength = "clear"
elif strength_of >= 15:
strength = "weak"
else:
return "no clear direction"
return f"{strength} {side}".strip()
def reliability_words(confidence: float) -> str:
if confidence >= 65:
return "high"
if confidence >= 55:
return "good"
if confidence >= 45:
return "fair"
return "low"
def plain_words(report: SignalReport) -> str:
"""One line a new trader can read instead of "Score +38.2 · Conf 62%"."""
agree = "timeframes agree" if report.mtf_agreement else "timeframes disagree"
return (f"Direction: {direction_words(report.final_score)} · "
f"Reliability: {reliability_words(report.final_confidence)} · {agree}")
def daily_trend_words(report: SignalReport) -> str:
"""The daily trend in words, and whether the trade plan goes with it
("Daily trend: rising — this buy goes with it"). "" when the scan
carries no daily trend. Display only (2026-09-30): on M15 the signal
core already drops a plan against a clear daily trend; on the other
timeframes the line is information for the reader."""
trend = getattr(report, "daily_trend", None)
if trend is None:
return ""
if trend == 0:
return "Daily trend: no clear direction"
word = "rising" if trend > 0 else "falling"
setup = report.trade_setup
if setup is None:
return f"Daily trend: {word}"
side = "with" if (setup.direction == "BUY") == (trend > 0) else "against"
return f"Daily trend: {word} — this {setup.direction.lower()} goes {side} it"
def print_signal_report(report: SignalReport, detail: bool = False):
sc = signal_color(report.final_signal)
qc = quality_color(report.signal_quality)
@@ -117,6 +175,10 @@ def print_signal_report(report: SignalReport, detail: bool = False):
f"{qc}{report.signal_quality}{Colors.RESET} · "
f"{sc}{report.final_score:+.1f}{Colors.RESET} · "
f"{report.final_confidence:.0f}% · {mtf_tag}")
print(f" {plain_words(report)}")
trend_words = daily_trend_words(report)
if trend_words:
print(f" {trend_words}")
print(f" {Colors.GRAY}{report.action_recommendation}{Colors.RESET}")
# ── 2. TRADE SETUP (if signal is actionable) ───────────────────
@@ -126,10 +188,28 @@ def print_signal_report(report: SignalReport, detail: bool = False):
print(f"\n {Colors.BOLD}Setup{Colors.RESET} "
f"{dc}{ts.direction}{Colors.RESET} @ {Colors.BOLD}{_fmt_price(ts.entry_price)}{Colors.RESET} "
f"→ SL {Colors.RED}{_fmt_price(ts.stop_loss)}{Colors.RESET} "
f"({ts.risk_pips:.1f}p) "
f"→ TP {Colors.GREEN}{_fmt_price(ts.take_profit_1)}{Colors.RESET} "
f"({ts.risk_pips:.1f}p = 1R) "
f"→ TP1 guide {Colors.GREEN}{_fmt_price(ts.take_profit_1)}{Colors.RESET} "
f"({ts.reward_pips:.1f}p) "
f"RR 1:{ts.risk_reward:.1f}")
f"planned TP1 RR 1:{ts.risk_reward:.1f}")
# The trail is the tested exit (the backtester never exits at
# TP1); print it with the setup so the CLI says the same thing
# as the Telegram alert (2026-09-26).
risk_dist = abs(ts.entry_price - ts.stop_loss)
arm_r = getattr(ts, "trail_arm_r", 0.0) or 0.0
dist_r = getattr(ts, "trail_dist_r", 0.0) or 0.0
if arm_r > 0 and risk_dist > 0:
arm_price = (ts.entry_price + arm_r * risk_dist if ts.direction == "BUY"
else ts.entry_price - arm_r * risk_dist)
print(f" {Colors.GRAY}Exit rule (tested): once price reaches "
f"{_fmt_price(arm_price)} (+{arm_r:.1f}R), trail the SL "
f"${dist_r * risk_dist:.2f} behind the best price at candle close; "
f"TP1 is a guide only.{Colors.RESET}")
# Lot size for the owner's risk limit and the alerted plans still
# open in this direction (exposure.attach_exposure; display only).
for line in (report.size_line, report.open_line):
if line:
print(f" {Colors.BOLD}{line}{Colors.RESET}")
# ── 3. CONTEXT (price · regime · vol · MTF breakdown) ──────────
print(f"\n {Colors.WHITE}{report.display_name}{Colors.RESET} "
@@ -168,7 +248,23 @@ def print_signal_report(report: SignalReport, detail: bool = False):
print(f" {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
if report.news_summary:
print(f" {Colors.MAGENTA}🧠{Colors.RESET} {report.news_summary}")
if report.panel_verdict:
if report.panel_verdict and report.panel_verdict_source == "mean":
base = (report.panel_context or {}).get("base_rate") or {}
usual = base.get("win_rate")
usual_txt = (f" (usual for this kind of setup: {usual:.0f}%)"
if usual is not None else "")
print(f" {Colors.CYAN}🤖{Colors.RESET} AI estimate: about a "
f"{Colors.BOLD}{report.panel_probability:.0f}%{Colors.RESET} "
f"chance this trade pays off{usual_txt}. "
f"{report.panel_rationale} Still being tested.")
for t in report.panel_takes:
p_win = t.get("p_win")
p_txt = f"{p_win:3.0f}%" if p_win is not None else " - "
label = t['persona'] if len(report.panel_takes) > 1 else "reasons"
print(f" {Colors.GRAY}{label:<11} {p_txt} "
f"for: {t.get('reason_for') or '-'} | against: "
f"{t.get('reason_against') or '-'}{Colors.RESET}")
elif report.panel_verdict:
panel_label = ("AI macro view"
if report.panel_verdict_source == "macro" else "AI Panel")
print(f" {Colors.CYAN}🤖{Colors.RESET} {panel_label}: "
@@ -213,6 +309,7 @@ def report_to_dict(report: SignalReport) -> dict:
"score": round(report.final_score, 2),
"confidence": round(report.final_confidence, 2),
"quality": report.signal_quality,
"plain_words": plain_words(report),
"action": report.action_recommendation,
"market_regime": report.market_regime,
"volatility_regime": report.volatility_regime,
@@ -230,7 +327,15 @@ def report_to_dict(report: SignalReport) -> dict:
"panel_created_at": report.panel_created_at,
"panel_cache_status": report.panel_cache_status,
"panel_payload_hash": report.panel_payload_hash,
"panel_verdict_source": report.panel_verdict_source,
"panel_prompt_version": report.panel_prompt_version,
"panel_probability": round(report.panel_probability, 1),
"panel_context": report.panel_context,
"watch_line": report.watch_line,
"daily_trend": report.daily_trend,
"daily_trend_words": daily_trend_words(report),
"size_line": report.size_line,
"open_line": report.open_line,
"bullish_reasons": report.bullish_reasons,
"bearish_reasons": report.bearish_reasons,
"warnings": report.warnings,
@@ -349,6 +454,10 @@ def run_live(args, config: PlatformConfig):
attach_watch(report, config.primary_timeframe,
get_app_config().violent_watch_settings)
# Display-only lot size (off until config.json sets trade_size) and
# the alerted plans still open in this direction.
attach_exposure(report, surface="cli")
if args.json:
payload = report_to_dict(report)
print(json.dumps(payload, indent=2))
+57 -14
View File
@@ -43,10 +43,19 @@ Contract
headline seen 50 times is one row with ``last_seen`` bumped.
* **stdlib only** (``sqlite3``), like ``signal_log``.
Known corpus limitation: a headline mentioning no tracked currency is
never constructed by the news engine at all (``news_engine`` drops it
before scoring), so DeepSeek never labels it either and it cannot appear
here.
Two columns exist for training only (2026-09-17):
* ``xau_score`` — DeepSeek's gold-specific score for the headline,
whatever currencies it names. ``score_by_currency_json["XAU"]``
carries it only for gold-keyword headlines; 930 of the first 1,546
DeepSeek rows had no gold label without this column. NULL for a
keyword-scored row and for rows written before the column existed
(an existing database is migrated in place, values are not
recoverable).
* ``currencies_json`` ``[]`` marks a headline naming no tracked currency.
The news engine used to drop those before scoring; now it scores and
archives them but never returns them to the platform. About 23% of a
day's feed; DeepSeek gave 10 of 24 a gold direction.
"""
from __future__ import annotations
@@ -85,7 +94,8 @@ CREATE TABLE IF NOT EXISTS headlines (
score_source TEXT NOT NULL DEFAULT 'keyword',
teacher_model TEXT,
teacher_prompt_sha1 TEXT,
origin TEXT NOT NULL DEFAULT 'live'
origin TEXT NOT NULL DEFAULT 'live',
xau_score REAL
);
CREATE INDEX IF NOT EXISTS idx_headlines_source ON headlines (score_source);
CREATE INDEX IF NOT EXISTS idx_headlines_first_seen ON headlines (first_seen);
@@ -155,8 +165,22 @@ class NewsArchive:
self._with_lock_retry(
lambda: self._conn.execute("PRAGMA journal_mode=WAL"))
self._with_lock_retry(lambda: self._conn.executescript(_SCHEMA))
self._with_lock_retry(self._migrate)
self._conn.commit()
#: Columns added after the first databases were created, in order.
#: CREATE TABLE IF NOT EXISTS leaves an existing table untouched, so
#: each one is added in place on open; existing rows read NULL.
_ADDED_COLUMNS = (("xau_score", "REAL"),)
def _migrate(self) -> None:
present = {row[1] for row in
self._conn.execute("PRAGMA table_info(headlines)")}
for name, decl in self._ADDED_COLUMNS:
if name not in present:
self._conn.execute(
f"ALTER TABLE headlines ADD COLUMN {name} {decl}")
@staticmethod
def _with_lock_retry(fn, attempts: int = 8, delay: float = 0.05):
"""Run ``fn`` again on 'database is locked' (concurrent openers
@@ -202,8 +226,8 @@ class NewsArchive:
" published_at, first_seen, last_seen, seen_count,"
" score, score_by_currency_json, currencies_json,"
" keywords_json, score_source, teacher_model,"
" teacher_prompt_sha1, origin"
") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?)"
" teacher_prompt_sha1, origin, xau_score"
") VALUES (?,?,?,?,?,?,?,?,?,1,?,?,?,?,?,?,?,?,?)"
" ON CONFLICT(hash) DO UPDATE SET"
# The backfill stamps historical timestamps, so clamp rather
# than assign: first_seen only ever moves earlier, last_seen
@@ -236,6 +260,13 @@ class NewsArchive:
" ELSE headlines.teacher_prompt_sha1 END,"
f" score_source = CASE WHEN {wins} THEN excluded.score_source"
" ELSE headlines.score_source END,"
# Same upgrade-only rule as score, and COALESCE on top: a
# keyword re-sighting carries no gold score and loses on
# rank, and even a same-rank write without one (a future
# backfill shape) must never blank a stored value.
f" xau_score = CASE WHEN {wins}"
" THEN COALESCE(excluded.xau_score, headlines.xau_score)"
" ELSE headlines.xau_score END,"
# Enrichment fills gaps but never blanks what is already there
# (the legacy backfill shape carries no description or link).
" description = COALESCE(excluded.description, headlines.description),"
@@ -246,9 +277,9 @@ class NewsArchive:
# The rank is bound once per CASE arm. Positional binding means a
# miscount silently shifts every parameter, so assert it instead of
# trusting the arithmetic: 17 INSERT values + one rank per arm.
# trusting the arithmetic: 18 INSERT values + one rank per arm.
n_rank = sql.count("? >= ")
n_values = 17
n_values = 18
assert sql.count("?") == n_values + n_rank, (
f"placeholder count drifted: {sql.count('?')} "
f"!= {n_values} + {n_rank}")
@@ -264,6 +295,7 @@ class NewsArchive:
row["currencies_json"], row["keywords_json"],
row["score_source"], row["teacher_model"],
row["teacher_prompt_sha1"], row.get("origin", "live"),
row.get("xau_score"),
) + (rank,) * n_rank)
def _write() -> None:
@@ -292,9 +324,17 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
teacher_prompt_sha1: Optional[str] = None,
origin: str = "live",
first_seen: Optional[str] = None,
last_seen: Optional[str] = None) -> Optional[Dict[str, Any]]:
last_seen: Optional[str] = None,
model_teacher: Optional[str] = None) -> Optional[Dict[str, Any]]:
"""Flatten one ``NewsSentiment`` (or an equivalent object) into a row.
Provenance: an ``llm`` row stores ``teacher_model`` (the DeepSeek
model name); a ``model`` row stores ``model_teacher`` (the local
scorer's tag, ``xau_transformer@<trained_at>``, news_model.teacher_tag).
Both carry ``teacher_prompt_sha1``: for a model row it is the DeepSeek
prompt the model was distilled from, which is the truthful lineage.
Keyword rows carry neither.
Returns None when the object carries no usable headline text."""
text = getattr(headline, "headline", None)
if not isinstance(text, str) or not text.strip():
@@ -317,10 +357,12 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
getattr(headline, "currencies_mentioned", None)),
"keywords_json": _json_or_none(getattr(headline, "keywords", None)),
"score_source": source,
"teacher_model": teacher_model if source == "llm" else None,
"teacher_model": (teacher_model if source == "llm"
else model_teacher if source == "model" else None),
"teacher_prompt_sha1": (
teacher_prompt_sha1 if source == "llm" else None),
teacher_prompt_sha1 if source in ("llm", "model") else None),
"origin": origin,
"xau_score": _finite_score(getattr(headline, "xau_score", None)),
}
@@ -331,7 +373,8 @@ _archives_lock = threading.Lock()
def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
teacher_prompt_sha1: Optional[str] = None,
origin: str = "live") -> None:
origin: str = "live",
model_teacher: Optional[str] = None) -> None:
"""Persist a batch of scored headlines if the news_archive config
block enables it. NEVER raises — a logging failure must not break a
scan."""
@@ -344,7 +387,7 @@ def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
for headline in headlines:
row = build_row(headline, teacher_model=teacher_model,
teacher_prompt_sha1=teacher_prompt_sha1,
origin=origin)
origin=origin, model_teacher=model_teacher)
if row is not None:
rows.append(row)
if not rows:
+331 -57
View File
@@ -8,6 +8,7 @@ Key principle: High-impact news events override technical signals.
A perfect technical setup can be destroyed by an unexpected NFP print.
"""
import copy
import hashlib
import html
import logging
@@ -23,7 +24,9 @@ from typing import Dict, List, Optional, Tuple
import requests
import app_config
import news_archive
import news_model
from config import NewsConfig
from llm_client import get_llm_client
@@ -222,10 +225,24 @@ class NewsSentiment:
publisher: Optional[str] = None # Google News <source> text, else None
# Which scorer produced sentiment_score / sentiment_by_currency:
# "keyword" (the built-in scorer, ~31% on the 2026-08-06 golden set),
# "llm" (DeepSeek, 78/78 on the same set) or "model" (a trained local
# artifact). Without it the two are indistinguishable in the archive
# and a model trained on the mix inherits the keyword scorer's errors.
# "llm" (DeepSeek, 78/78 on the same set) or "model" (the local XAU
# transformer, news_model.py, 72/78; since 2026-09-27). "model" means
# the model scored xau_score and XAU's per-currency entry; every
# other currency's entry, and sentiment_score when the headline names
# another currency, remain keyword-scored — the model has a gold head
# only. Without the tag the scorers are indistinguishable in the
# archive and a model trained on the mix inherits the keyword errors.
score_source: str = "keyword"
# The gold-specific score for THIS headline (DeepSeek's, or the local
# model's raw signed score), whatever currencies it names.
# sentiment_by_currency["XAU"] carries it only when the headline
# matched a gold keyword; for a pure Fed / dollar / yields headline it
# used to be discarded, which left 930 of the first 1,546 archived
# DeepSeek rows without a gold label (eval 2026-09-17). Nothing
# displayed or averaged reads it — the archive stores it as training
# data. None until DeepSeek or the model has scored the headline (the
# keyword scorer never sets it).
xau_score: Optional[float] = None
@dataclass
@@ -367,10 +384,19 @@ class NewsEngine:
# writer runs inside _cached_fetch); the other two take the lock
# around cache access but keep HTTP calls outside it.
_llm_sent_cache: Dict[str, Tuple[float, float]] = {} # sha1 → (s, xau)
_llm_event_cache: Dict[Tuple[str, str, str], str] = {}
_llm_event_cache: Dict[Tuple[str, str, str, str, str], str] = {}
_llm_summary_cache: Tuple[float, str] = (0.0, "")
_LLM_SENT_CACHE_MAX = 400
_LLM_EVENT_CACHE_MAX = 100
#: Headlines per DeepSeek sentiment call — the reply must fit the
#: client's max_tokens (~10 tokens per headline, 2000 max).
_LLM_BATCH_CAP = 25
# primary-mode DeepSeek audit sample (news_model.deepseek_audit_per_day):
# how many model-scored headlines went to DeepSeek today, archive-only.
# Written under _cache_lock like _llm_sent_cache.
_audit_day: str = ""
_audit_sent_today: int = 0
# Per-source fetch health for status displays (telegram /status).
# name → {"ok": ts|None, "err": ts|None, "detail": str, "extra": str}.
@@ -386,6 +412,14 @@ class NewsEngine:
self._session.headers.update({
"User-Agent": "Mozilla/5.0 (compatible; ForexIntel/1.0)"
})
# Create the local scorer's ONNX session off the scan thread so
# the first news cycle does not pay 3-5 s under _cache_lock.
# Idempotent per process (a NewsEngine is built for every scan).
try:
if app_config.news_model_mode() != "off":
news_model.warm_up_in_background()
except Exception as e: # never let the warm-up cost a scan
logger.debug(f"news model warm-up skipped: {e}")
def get_fundamental_context(
self, currencies: Optional[List[str]] = None
@@ -575,6 +609,23 @@ class NewsEngine:
key=lambda e: e.time,
)
def peek_week_events(
self, currencies: Optional[List[str]] = None
) -> List[EconomicEvent]:
"""get_week_events WITHOUT the refresh: whatever calendar the
class-level cache holds right now, however old, filtered and sorted
the same way. Never fetches and never waits on the cache lock, for
a caller that must not add a network wait: the agent panel runs
while an alert is waiting to go out, and a refetch with both
calendar sources down takes about 40 seconds."""
if currencies is None:
currencies = self.config.tracked_currencies
_stamp, events = NewsEngine._calendar_cache
return sorted(
(e for e in events if e.currency in currencies),
key=lambda e: e.time,
)
# ── Private Methods ────────────────────────────────────────────
# Maps country names in MyFXBook titles to currency codes
@@ -811,7 +862,8 @@ class NewsEngine:
def _fetch_news_sentiment_raw(self) -> List[NewsSentiment]:
"""Fetch and analyze news headlines for sentiment."""
headlines = []
headlines = [] # names a tracked currency: what the platform uses
unmatched = [] # names none: scored and archived, never shown
seen_titles = set()
ok_feeds = 0
undated = 0
@@ -836,6 +888,11 @@ class NewsEngine:
continue
title = title_el.text or ""
# A blank title carries nothing to score, show or
# archive, and one such item made the local model
# reject the whole batch (2026-10-01).
if not title.strip():
continue
pub_date = self._parse_rss_date(
pub_date_el.text if pub_date_el is not None else None
)
@@ -861,38 +918,48 @@ class NewsEngine:
sentiment = self._analyze_headline_sentiment(title)
currencies = self._extract_currencies(title)
link_el = item.find("link")
desc_el = item.find("description")
src_el = item.find("source")
# itertext() survives CDATA and stray child
# elements where .text stops at the first child.
desc = _clean_html_text(
"".join(desc_el.itertext())
if desc_el is not None else None)
# Google News descriptions are often just the
# headline again inside an anchor — worthless.
if desc and desc.strip() == title.strip():
desc = None
publisher = None
if src_el is not None:
publisher = (src_el.text or "").strip() or None
entry = NewsSentiment(
headline=title,
source=feed_url,
timestamp=pub_date or datetime.now(timezone.utc),
currencies_mentioned=currencies,
sentiment_score=sentiment,
keywords=self._extract_keywords(title),
sentiment_by_currency={
cur: self._analyze_headline_sentiment(title, cur)
for cur in currencies
},
link=_safe_link(
link_el.text if link_el is not None else None),
description=desc,
publisher=publisher,
)
# A headline naming no tracked currency stays invisible
# to the platform (no per-currency mean, no display,
# no panel) but is scored and archived since
# 2026-09-17: it was 23% of a day's feed, and DeepSeek
# gave 10 of those 24 a gold direction — the Iran /
# Hormuz stories were the strongest gold headlines of
# the day. Training data only.
if currencies:
link_el = item.find("link")
desc_el = item.find("description")
src_el = item.find("source")
# itertext() survives CDATA and stray child
# elements where .text stops at the first child.
desc = _clean_html_text(
"".join(desc_el.itertext())
if desc_el is not None else None)
# Google News descriptions are often just the
# headline again inside an anchor — worthless.
if desc and desc.strip() == title.strip():
desc = None
publisher = None
if src_el is not None:
publisher = (src_el.text or "").strip() or None
headlines.append(NewsSentiment(
headline=title,
source=feed_url,
timestamp=pub_date or datetime.now(timezone.utc),
currencies_mentioned=currencies,
sentiment_score=sentiment,
keywords=self._extract_keywords(title),
sentiment_by_currency={
cur: self._analyze_headline_sentiment(title, cur)
for cur in currencies
},
link=_safe_link(
link_el.text if link_el is not None else None),
description=desc,
publisher=publisher,
))
headlines.append(entry)
else:
unmatched.append(entry)
ok_feeds += 1
self._record_source(
@@ -905,16 +972,26 @@ class NewsEngine:
if undated:
logger.debug(f"RSS: dropped {undated} headlines without a publish time")
if unmatched:
logger.debug(f"RSS: {len(unmatched)} headlines name no tracked "
"currency (scored and archived, not shown)")
self._record_source(
"headlines", ok_feeds > 0,
detail="all feeds failed" if ok_feeds == 0 else "",
extra=(f"{len(headlines)} headlines from "
f"{ok_feeds}/{len(self.config.rss_feeds)} feeds"))
# Score and archive EVERYTHING fetched; hand back only the
# currency-matched headlines so display, per-currency means, the
# panel and the summary see exactly what they saw before.
scored = headlines + unmatched
mode = app_config.news_model_mode()
audit_rows: List[NewsSentiment] = []
try:
self._apply_llm_sentiment(headlines)
audit_rows = self._enrich_headline_scores(scored, mode)
except Exception as e:
logger.debug(f"LLM sentiment failed, keyword scores stand: {e}")
logger.debug(f"headline sentiment enrichment failed, keyword "
f"scores stand: {e}")
# Archive BEFORE any downstream filtering. get_fundamental_context
# drops every headline inside ±NEUTRAL_HEADLINE_THRESHOLD, so a
@@ -925,16 +1002,65 @@ class NewsEngine:
# arguments are evaluated BEFORE that function's own try/except, so
# a raising get_llm_client() would escape it. Archiving is a side
# effect — it must never cost us the headlines we just fetched.
#
# In "primary" mode the local model is the scorer and its output
# is not training data, so the cycle's rows are NOT archived
# (owner decision 2026-09-27: stop writing newsdata.db once the
# model scores). Only the DeepSeek audit copies, the one source of
# fresh labels, are stored; with deepseek_audit_per_day 0 the
# archive is frozen. "off" and "fallback" archive as before.
try:
news_archive.archive_headlines(
headlines,
teacher_model=getattr(get_llm_client(), "_model", None),
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1)
teacher = getattr(get_llm_client(), "_model", None)
if mode == "primary":
if audit_rows:
news_archive.archive_headlines(
audit_rows, teacher_model=teacher,
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1)
logger.debug(
f"news archive: primary mode, {len(audit_rows)} DeepSeek "
f"audit rows stored, {len(scored)} model/keyword rows "
f"not archived")
else:
news_archive.archive_headlines(
scored, teacher_model=teacher,
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1,
model_teacher=news_model.teacher_tag())
except Exception as e:
logger.debug(f"news archive skipped, headlines unaffected: {e}")
return headlines
def _enrich_headline_scores(
self, scored: List[NewsSentiment], mode: str
) -> List[NewsSentiment]:
"""Upgrade keyword scores on `scored` per the news_model mode.
Returns the DeepSeek audit copies to archive (primary mode only,
usually empty). Runs under _cache_lock like the scorers it calls.
| mode | DeepSeek for headlines | local model |
|----------|------------------------|----------------------------|
| off | every new headline | never |
| fallback | first, as before | rows DeepSeek left keyword |
| primary | audit sample only | every row |
primary with the model unavailable in this process (missing
files, onnxruntime not installed) degrades to the DeepSeek path
for the cycle, never straight to keywords; a model failure mid-
batch leaves keyword scores standing (see _apply_model_sentiment).
"""
if mode == "primary":
n = self._apply_model_sentiment(scored)
if n == 0 and news_model.get_news_model() is None:
self._apply_llm_sentiment(scored)
return []
return self._audit_with_deepseek(scored)
if mode == "fallback":
self._apply_llm_sentiment(scored)
self._apply_model_sentiment(scored)
return []
self._apply_llm_sentiment(scored)
return []
def _analyze_headline_sentiment(
self, headline: str, for_currency: Optional[str] = None
) -> float:
@@ -1003,27 +1129,34 @@ class NewsEngine:
"""Overwrite keyword sentiment on `headlines` with LLM scores.
Runs inside _cached_fetch, i.e. UNDER _cache_lock — cache access
is direct and the (rare) HTTP call blocks sibling scans at most
is direct and the (rare) HTTP calls block sibling scans at most
once per cache window, matching the existing shared-fetch design.
Only headlines missing from the cache are sent; a failed batch
changes nothing.
Only headlines missing from the cache are sent, in batches of
_LLM_BATCH_CAP; a failed batch changes nothing and ends the
cycle's sending.
"""
if not headlines or not get_llm_client().available:
return
cache = NewsEngine._llm_sent_cache
keyed = [(hashlib.sha1(h.headline.encode()).hexdigest(), h)
for h in headlines]
# Cap the batch so the reply fits the client's max_tokens — the
# overflow stays keyword-scored this cycle and, still uncached,
# goes out with the next window's batch.
new = [(k, h) for k, h in keyed if k not in cache][:25]
if new:
scores = self._llm_headline_scores([h.headline for _, h in new])
if scores is not None:
for (k, _), sc in zip(new, scores):
cache[k] = sc
while len(cache) > self._LLM_SENT_CACHE_MAX:
cache.pop(next(iter(cache)))
# Every new headline goes out this cycle, one call per batch.
# Until 2026-09-17 only the first 25 went and the overflow stayed
# keyword-scored (31 of the first 1,577 archived headlines never
# got a DeepSeek label that way). A failed call ends the loop: a
# timeout or the daily cap would fail the next one too, and what
# is left stays keyword-scored and, still uncached, goes out with
# the next window.
new = [(k, h) for k, h in keyed if k not in cache]
for start in range(0, len(new), self._LLM_BATCH_CAP):
batch = new[start:start + self._LLM_BATCH_CAP]
scores = self._llm_headline_scores([h.headline for _, h in batch])
if scores is None:
break
for (k, _), sc in zip(batch, scores):
cache[k] = sc
while len(cache) > self._LLM_SENT_CACHE_MAX:
cache.pop(next(iter(cache)))
for k, h in keyed:
sc = cache.get(k)
if sc is None:
@@ -1034,8 +1167,139 @@ class NewsEngine:
cur: (xau if cur == "XAU" else overall)
for cur in h.currencies_mentioned
}
h.xau_score = xau
h.score_source = "llm"
# ── Local XAU transformer (news_model.py, since 2026-09-27) ──────
def _apply_model_sentiment(self, headlines: List[NewsSentiment]) -> int:
"""Score with the local XAU transformer every headline that is
still keyword-scored. Returns the number scored. Never raises.
The model has a gold head only. It writes xau_score always, XAU's
per-currency score when XAU is mentioned, and the overall score
only when the headline names no other currency. USD/EUR/...
entries keep their keyword score: get_pair_sentiment subtracts
USD from XAU and must not see the same number on both sides.
Rules (spec 2026-09-27):
A. never write the model score into a non-XAU currency;
B. sentiment_score from the model only for [] or ["XAU"] — the
archive's `score` column must be honest about what the model
did and did not score;
C. xau_score is the raw signed score (training data, unfiltered);
D. displayed values are 0.0 when the model says not relevant — a
confident direction head can still give |score| ~0.2, which
NEUTRAL_HEADLINE_THRESHOLD (0.1) would show as "leaning";
E. NewsEngine._llm_sent_cache is never touched — it decides what
is re-sent to DeepSeek (fallback mode retries, audit sample);
F. only score_source == "keyword" rows change; an llm row stands.
"""
if not headlines:
return 0
bundle = news_model.get_news_model()
if bundle is None:
self._record_source("news_model", False,
"not loaded in this process")
return 0
# score_batch cleans every text first and raises on one that is
# empty afterwards ("&nbsp;" is), which used to cost the whole
# batch its scores; such a row keeps its keyword score instead.
todo = [h for h in headlines if h.score_source == "keyword"
and self._model_can_score(h.headline)]
if not todo:
# Nothing for the model this cycle (fallback mode with DeepSeek
# healthy, or no readable headline). A failure an earlier batch
# recorded says nothing about now and must not sit on /status
# as if it were current; the next batch records its own outcome
# and warns again if it fails.
if self.source_is_failing(
NewsEngine._source_health.get("news_model")):
NewsEngine._source_health.pop("news_model", None)
return 0
try:
results = news_model.score_batch(bundle, [h.headline for h in todo])
if len(results) != len(todo):
raise ValueError(
f"{len(results)} results for {len(todo)} headlines")
except Exception as e:
# Keyword scores stand for the cycle. Loud once (WARNING on the
# first failure, INFO on recovery) and shown by /status: in
# primary mode nothing else scores these headlines.
self._record_source("news_model", False,
f"{type(e).__name__}: {e}")
return 0
self._record_source("news_model", True, extra=f"{len(todo)} scored")
for h, r in zip(todo, results):
raw = float(r["score"]) # archive value
shown = raw if r["relevant"] else 0.0 # display value
h.xau_score = raw
if "XAU" in h.currencies_mentioned:
h.sentiment_by_currency["XAU"] = shown
if set(h.currencies_mentioned) <= {"XAU"}: # [] or ["XAU"]
h.sentiment_score = shown
h.score_source = "model"
return len(todo)
@staticmethod
def _model_can_score(text: str) -> bool:
"""True when the local model's own cleaner accepts the text."""
try:
news_model.clean_headline(text)
return True
except Exception:
return False
def _audit_with_deepseek(
self, headlines: List[NewsSentiment]
) -> List[NewsSentiment]:
"""primary mode: send up to news_model.deepseek_audit_per_day
model-scored headlines per UTC day to DeepSeek and return `llm`
COPIES for the archive. The displayed objects keep the model's
numbers. One batch per cycle at most; the pairs go into
_llm_sent_cache so a headline is never audited twice. This is the
only place DeepSeek is called for headlines in primary mode; with
the cap at 0 (the shipped value) it returns [] without a call.
The copies exist only to be archived, so a process that does not
write the archive (the dashboard daemon, NEWS_ARCHIVE_ENABLED=0)
makes no call either."""
cap = app_config.news_model_audit_per_day()
if cap <= 0 or not headlines or not get_llm_client().available:
return []
if not app_config.news_archive_enabled():
return []
today = datetime.now(timezone.utc).date().isoformat()
if NewsEngine._audit_day != today:
NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0
room = cap - NewsEngine._audit_sent_today
if room <= 0:
return []
cache = NewsEngine._llm_sent_cache
cand = [h for h in headlines if h.score_source == "model"
and hashlib.sha1(h.headline.encode()).hexdigest() not in cache]
batch = cand[:min(room, self._LLM_BATCH_CAP)]
if not batch:
return []
scores = self._llm_headline_scores([h.headline for h in batch])
if scores is None:
return []
NewsEngine._audit_sent_today += len(batch)
copies = []
for h, (overall, xau) in zip(batch, scores):
cache[hashlib.sha1(h.headline.encode()).hexdigest()] = (overall, xau)
c = copy.copy(h)
c.sentiment_score = overall
c.sentiment_by_currency = {
cur: (xau if cur == "XAU" else overall)
for cur in h.currencies_mentioned
}
c.xau_score = xau
c.score_source = "llm"
copies.append(c)
while len(cache) > self._LLM_SENT_CACHE_MAX:
cache.pop(next(iter(cache)))
return copies
def _llm_event_direction(self, event: EconomicEvent) -> Optional[str]:
"""Gold direction for a released surprise, e.g. "bearish: strong
jobs lift yields". None → caller keeps the numeric description.
@@ -1046,7 +1310,17 @@ class NewsEngine:
close K/M-suffixed pairs backwards ~11% of the time (e.g. claims
202K vs 217K called "higher than forecast") when left to compare
the raw numbers itself."""
key = (event.event, event.currency, str(event.actual or ""))
# Forecast and release time are part of the key: the direction is
# a reading of actual AGAINST forecast, and next month's release
# can print the same actual against a different forecast. The time
# is keyed in UTC so one instant is one key whatever offset the
# feed printed it with.
try:
when = event.time.astimezone(timezone.utc).isoformat()
except Exception:
when = str(event.time)
key = (event.event, event.currency, str(event.actual or ""),
str(event.forecast or ""), when)
with NewsEngine._cache_lock:
if key in NewsEngine._llm_event_cache:
return NewsEngine._llm_event_cache[key] or None
+203
View File
@@ -0,0 +1,203 @@
"""Local XAU headline scorer: the owner's ML-NEWS transformer on CPU.
Vendored from ML-NEWS news_transformer.py at commit afa26dc; bundle
trained_at 2026-09-27T23:15:30.403442+00:00. Keep the scoring functions
identical to that file (the one difference: ``load_model`` has no default
directory here, the path always comes from config.json); ML-NEWS trains
and exports, ForexBot2 only scores.
A fine-tuned DeBERTa-v3-small encoder with two heads, exported to ONNX
with an int8 embedding table. Scoring needs only onnxruntime, tokenizers
and numpy. For each headline:
- ``p_relevant``: probability the news moves the gold price (trained on
DeepSeek |xau| >= 0.25 vs <= 0.10)
- ``p_bullish``: probability that, if it does, gold goes up
- ``relevant``: ``p_relevant >= relevance_threshold`` (from meta.json)
- ``label``: ``bullish`` / ``bearish``, or ``neutral`` when not relevant
- ``score``: ``p_relevant * (2 * p_bullish - 1)`` in [-1, 1], the same
scale as DeepSeek's gold score the archive stores in ``xau_score``
The model has a GOLD head only. news_engine writes its score into
``xau_score`` and the XAU per-currency entry, never into USD or any other
currency (get_pair_sentiment subtracts USD from XAU).
Import contract: ``onnxruntime`` and ``tokenizers`` are imported INSIDE
``load_model`` so ``import news_model`` (and therefore ``import
news_engine``) succeeds on a box without them; ``get_news_model`` then
returns None once, with one warning, and the DeepSeek path stays.
Configuration comes from app_config (``news_model`` section, env
``NEWS_MODEL_MODE``): ``news_model_path()``, ``news_model_threads()``.
"""
from __future__ import annotations
import html
import json
import logging
import os
import threading
from pathlib import Path
from typing import Any, Iterable, Optional
import numpy as np
logger = logging.getLogger(__name__)
MAX_LENGTH = 64
#: DeepSeek |xau| at or above this is "moves gold"; at or below
#: RELEVANT_MAX_IRRELEVANT it is "does not"; the band between carries no
#: relevance label in training.
RELEVANT_MIN = 0.25
RELEVANT_MAX_IRRELEVANT = 0.10
def clean_headline(headline: str) -> str:
"""The exact text normalisation used in training and scoring."""
text = " ".join(html.unescape(str(headline or "")).split())
if not text:
raise ValueError("headline is empty")
return text
def _sigmoid(x: np.ndarray) -> np.ndarray:
return 1.0 / (1.0 + np.exp(-x))
def load_model(model_dir: str | Path,
threads: int | None = None) -> dict[str, Any]:
"""Load the ONNX model, tokenizer and metadata. Only load trusted files."""
import onnxruntime as ort
from tokenizers import Tokenizer
model_dir = Path(model_dir)
meta_path = model_dir / "meta.json"
if not meta_path.is_file():
raise FileNotFoundError(f"model not found: {meta_path}")
meta = json.loads(meta_path.read_text(encoding="utf-8"))
tokenizer = Tokenizer.from_file(str(model_dir / "tokenizer.json"))
tokenizer.enable_truncation(max_length=int(meta.get("max_length", MAX_LENGTH)))
tokenizer.enable_padding(pad_id=int(meta["pad_token_id"]), pad_token=meta["pad_token"])
options = ort.SessionOptions()
if threads:
options.intra_op_num_threads = max(1, min(int(threads), os.cpu_count() or 1))
options.inter_op_num_threads = 1
# ORT intra-op threads spin-wait after each run by default; in a mostly
# idle bot process that is 4 cores at 100% between cycles.
options.add_session_config_entry("session.intra_op.allow_spinning", "0")
session = ort.InferenceSession(str(model_dir / meta.get("onnx_file", "model.onnx")),
options, providers=["CPUExecutionProvider"])
return {"meta": meta, "tokenizer": tokenizer, "session": session}
def score_batch(bundle: dict[str, Any], headlines: Iterable[str]) -> list[dict[str, Any]]:
texts = [clean_headline(h) for h in headlines]
if not texts:
return []
encodings = bundle["tokenizer"].encode_batch(texts)
ids = np.array([e.ids for e in encodings], dtype=np.int64)
mask = np.array([e.attention_mask for e in encodings], dtype=np.int64)
rel_logit, dir_logit = bundle["session"].run(
["relevance_logit", "direction_logit"],
{"input_ids": ids, "attention_mask": mask})
p_rel = _sigmoid(rel_logit.astype(np.float64))
p_bull = _sigmoid(dir_logit.astype(np.float64))
threshold = float(bundle["meta"].get("relevance_threshold", 0.5))
out = []
for text, pr, pb in zip(texts, p_rel, p_bull):
relevant = bool(pr >= threshold)
direction = "bullish" if pb >= 0.5 else "bearish"
label = direction if relevant else "neutral"
confidence = pr * max(pb, 1 - pb) if relevant else 1 - pr
out.append({
"target": "XAUUSD",
"text": text,
"relevant": relevant,
"p_relevant": round(float(pr), 4),
"direction": direction,
"p_bullish": round(float(pb), 4),
"label": label,
"score": round(float(pr * (2 * pb - 1)), 4),
"confidence": round(float(confidence), 4),
})
return out
def score_news(bundle: dict[str, Any], headline: str) -> dict[str, Any]:
return score_batch(bundle, [headline])[0]
# ── Process-wide singleton (ForexBot2 addition, not in ML-NEWS) ──────────
#
# One ONNX session per process (~0.6 GB RSS, 3–5 s to create). A failed
# load is remembered: the bot logs one warning and keeps the DeepSeek /
# keyword path for the rest of its life instead of retrying every cycle.
_lock = threading.Lock()
_bundle: Optional[dict] = None
_failed: bool = False # tried and failed; don't retry, don't re-log
_warm_started: bool = False
def get_news_model() -> Optional[dict]:
"""The loaded bundle, or None. Loads once per process. Never raises."""
global _bundle, _failed
if _bundle is not None or _failed:
return _bundle
with _lock:
if _bundle is not None or _failed:
return _bundle
try:
import app_config
path = app_config.news_model_path()
if not path:
raise FileNotFoundError("news_model.path is not configured")
threads = app_config.news_model_threads()
b = load_model(path, threads=threads)
logger.info(
"news model loaded from %s (trained_at %s, threshold %s, "
"%d threads)", path, b["meta"].get("trained_at"),
b["meta"].get("relevance_threshold"), threads)
_bundle = b
except Exception as e: # ImportError, FileNotFoundError, bad JSON, ORT
logger.warning(
"news model unavailable, DeepSeek/keyword path stays: %s: %s",
type(e).__name__, e)
_failed = True
return _bundle
def teacher_tag() -> Optional[str]:
"""Provenance tag for archive rows scored by the loaded model, e.g.
``xau_transformer@2026-09-27T23:15:30…``. Reads the already-loaded
bundle only; never triggers a load. None when nothing is loaded."""
b = _bundle
if b is None:
return None
return f"xau_transformer@{b['meta'].get('trained_at', 'unknown')}"
def warm_up_in_background() -> None:
"""Create the ONNX session on a daemon thread, once per process, so
the first news cycle does not pay the 3–5 s load under
NewsEngine._cache_lock. Idempotent; a later get_news_model() call
simply waits on the lock if the thread is still loading."""
global _warm_started
if _warm_started or _bundle is not None or _failed:
return
with _lock:
if _warm_started:
return
_warm_started = True
threading.Thread(target=get_news_model, name="news-model-warmup",
daemon=True).start()
def reset_for_tests() -> None:
"""Forget any loaded or failed state (test_all.py only)."""
global _bundle, _failed, _warm_started
with _lock:
_bundle = None
_failed = False
_warm_started = False
+72
View File
@@ -107,6 +107,68 @@ screen's M5 picks (MACD-down) all FAILED the replay, and ema_trend x 2
/ divergence -> 0 on M15 failed too. Canonical M15 book re-pinned to
backtest/oanda_m15_2yr_065_20260909.csv.
M15 override 6 — grade-A SELLs blocked (validated 2026-09-26,
backtest/research/rr_and_sell_campaign_2026-09-26.md, pre-registered
rule, CUDA twin runs paired with gpu_backtest.py MD5s, $0.65):
sell_min_quality=["A+", "B"] on M15, so gate 6e downgrades every
grade-A short to D (A+ and B shorts still trade). The bucket was the
one cycle-2 candidate from the 2026-09-10 screen: M15 grade-A SELLs
230 / -14.4R on the 3-year window, negative in every 12-month slice
and in the OOS window. Candidate vs control: 2-year W2 2024-09-05 ->
2026-09-05 813/+154.15R PF 1.53 DD 15.6R (halves +70.1/+84.0) vs
931/+146.14R PF 1.43 DD 16.1R (+66.1/+80.0); OOS 2024-06-01 ->
2025-03-30 313/+68.50R PF 1.67 DD 12.4R vs 355/+60.67R PF 1.49 DD
17.1R; 3-year W3 2023-09-05 -> 2026-09-05 1105/+195.79R PF 1.49 DD
20.9R vs 1273/+186.61R PF 1.39 DD 19.3R (+8%, inside the 10% clause),
better in 2024, 2025 and 2026, -2.4R in the 4 months of 2023; at $0.30
+167.53R vs +161.44R (W2). Trade diff on W2: 162 removed (-8.1R, all
grade-A SELLs), 44 added (-0.1R, cooldown displacement). Counter-fact
on record: M15 A-grade shorts paid in the 2021-22 bear phase
(m15_5yr_cuda_2026-09-18.md), outside the owner's 3-year window.
Tested and NOT shipped in the same campaign: signals.min_rr_ratio 1.0
on M15 (trade-identical, -2.0R W2, -2.8R W3: the swing-anchored
GENERIC stop is not better than the 2 x ATR stop on the same grade-A
signals); the combination with this block (-0.7R vs the block alone).
M15 override 7 — daily-trend veto (validated 2026-09-30,
backtest/research/daily_trend_campaign_2026-09-30.md, pre-registered
RISK-FIRST rule approved by the owner that day, CUDA twin runs paired
with gpu_backtest.py MD5s, $0.65): daily_trend_veto="both". The engine
sees at most the H1 chart. The daily trend is +1 when the last CLOSED
daily bar closes above its 50-day EMA AND above the close 60 daily
bars earlier, -1 when below both, else 0 (Oanda's Sunday stub bars are
dropped; signal_aggregator.daily_trend_series). After every setup
archetype has run, a setup AGAINST a non-zero state is dropped (grade
D, no setup, warning "DAILY TREND FILTER: ..."). Candidate vs control:
3-year W3 2023-09-05 -> 2026-09-05 824/+205.12R PF 1.73 DD 12.4R vs
1105/+195.79R PF 1.49 DD 20.9R (halves +76.7/+128.4 vs +76.9/+118.9;
at $0.30 +221.18R PF 1.81 DD 11.4R vs +216.53R PF 1.55 DD 19.6R);
7.6-year LONG 2019-01-01 -> 2026-08-07 1473/+223.33R PF 1.42 DD 41.5R
vs 2091/+192.53R PF 1.24 DD 64.8R (2022 +3.2R vs -24.6R; 2021 NOT
improved, -27.3R vs -25.6R; 2023 worse, +1.8R vs +11.7R); the unseen
HOLDOUT 2026-09-05 -> 2026-10-01 18/-4.62R vs 24/-6.63R; 3 of 4
neighbouring definitions pass (90-day change, 120-day change, 100-day
average; the 60-day change alone misses the 97% line by 1.7R). Trade
diff W3: 281 removed (-9.3R: BUY 106/+7.6R, SELL 175/-16.9R), 0 added;
the removed trades are break-even after costs (-0.03R each) against
+0.25R for the kept ones, so the veto drops the no-edge quarter of the
book rather than a pocket of big losers. ACCEPTED COST (owner decision
2026-09-30): R given up while gold only rises — 2-year W2 597/+155.89R
PF 1.78 DD 11.4R vs 813/+154.15R PF 1.53 DD 15.6R with the first half
+61.8R vs +70.1R, and the old OOS window 2024-06-01 -> 2025-03-30
258/+63.78R PF 1.80 DD 9.1R vs 313/+68.50R PF 1.67 DD 12.4R. The
2026-09-26 rule template (no half and no OOS window may lose) would
have rejected it; the risk-first rule (3-year R >= 97%, PF +0.10, DD
not deeper, long window better on R, DD and 2021+2022, holdout, both
costs, plateau) is the owner's deliberate choice. M15 ONLY: the same
veto FAILED on M5 ("both": W3 DD 10.1 -> 15.2R and the long window
worse on R and DD; "buy": passes W3, loses the long window, no
plateau) and on H1 (W3 +60.3R vs +70.0R). "buy"-only and "sell"-only
FAILED on M15 as well ("sell" has the best W3 total, +212.7R, and a
LONG DD of 76.4R vs 64.8R: in falling years the buys against the daily
trend are the losers). The live wrapper and both backtesters load D1
as an extra REQUIRED frame while the veto is on.
M5 override (validated 2026-07-11, full-history replay 2020-03-17 ->
2026-07-11 + 12mo book, $0.30 spread): block hour 9 UTC (post-London-
open whipsaw). Hour-9 entries lost -28.9R/143 over 6 years, negative
@@ -305,6 +367,16 @@ def make_config(
# RSI regime multiplier × 0.5 in every regime on M15 only — see
# module docstring (M15 override 5, validated 2026-09-09).
config.technical.regime_weights_override = _m15_regime_weights()
# Grade-A SELLs blocked on M15 only (gate 6e downgrades an
# actionable-grade short whose grade is not listed) — see module
# docstring (M15 override 6, validated 2026-09-26).
config.xau_strategy.sell_min_quality = ["A+", "B"]
# Daily-trend veto on M15 only: a finished setup against a clear
# daily trend (price vs its 50-day average AND the 60-day change)
# is dropped. The live wrapper and both backtesters load the D1
# frame for it — see module docstring (M15 override 7, validated
# 2026-09-30). M5, H1 and H4 FAILED the same rule and stay "off".
config.xau_strategy.daily_trend_veto = "both"
if primary_tf == Timeframe.H1:
# Block every H1 SELL — the SELL quality gate (6e in
# signal_aggregator.py) downgrades any actionable-grade short
+44
View File
@@ -37,6 +37,10 @@ class OandaDataProvider(BaseDataProvider):
RETRY_DELAYS_S = (2.0, 5.0, 10.0)
RETRY_BUDGET_S = 45.0
RETRY_DELAY_S = RETRY_DELAYS_S[0] # first delay; kept for callers/tests
# The longest date window that may come back empty and still mean
# "nothing traded here": a weekend joined to a holiday. A longer window
# with no bars is an anomaly, not a closure (see _fetch_oanda_candles).
EMPTY_WINDOW_MAX = pd.Timedelta(days=5)
OANDA_GRANULARITY = {
Timeframe.M1: "M1",
@@ -96,6 +100,26 @@ class OandaDataProvider(BaseDataProvider):
symbol, timeframe, outputsize, start, end)
return self._fetch_oanda_candles(symbol, timeframe, outputsize)
@staticmethod
def _empty_frame() -> pd.DataFrame:
"""A normalized frame with no bars (a date window nothing traded in)."""
df = pd.DataFrame(
{col: pd.Series(dtype="float64")
for col in ("Open", "High", "Low", "Close", "Volume")},
index=pd.DatetimeIndex([], tz="UTC", name="Datetime"))
return df
@classmethod
def _closure_sized(cls, start: Optional[datetime],
end: Optional[datetime]) -> bool:
"""True for a date window short enough to be one scheduled closure."""
if start is None or end is None:
return False
try:
return pd.Timestamp(end) - pd.Timestamp(start) <= cls.EMPTY_WINDOW_MAX
except (TypeError, ValueError):
return False
def _to_oanda_instrument(self, symbol: str) -> str:
try:
return self.OANDA_INSTRUMENT[symbol]
@@ -192,8 +216,25 @@ class OandaDataProvider(BaseDataProvider):
raise OandaDataError(
f"Oanda candle response for {instrument} is not valid "
f"JSON: {e}") from e
# A DATE WINDOW with no bars in it (a weekend, a holiday, the closed
# days before a window's first session) is not a failure: fetch_range
# walks history in windows and counts an empty one as "nothing traded
# here". Raising for it made every backtest window whose padded start
# falls on a closed day unfetchable once fetch_range began retrying a
# failing window and aborting after three tries (2026-09-13); found
# 2026-09-30 on the D1 pad of a window starting on a Saturday. A
# `count` request (the live path) with nothing in it stays an error.
# Only a CLOSURE-SIZED window gets that reading (2026-10-01 review):
# an empty answer for a longer one (a 15-day M5 page, a 46-day M15
# page) cannot be a closure, and accepting it let fetch_range step
# over the window and cache history with a hole in it. Raising makes
# fetch_range retry the same window and refuse after three tries.
# (An M1 page is 3 days, so it would still pass; M1 is not fetched.)
closure_sized = self._closure_sized(start, end)
candles = data.get("candles", [])
if not candles:
if closure_sized:
return self._empty_frame()
raise OandaDataError(
f"Empty candle response from Oanda for {instrument}")
@@ -233,6 +274,9 @@ class OandaDataProvider(BaseDataProvider):
"OHLC fields) — skipped", skipped_malformed, instrument)
if not rows:
# Malformed candles mean the window DID trade: never "no bars".
if closure_sized and not skipped_malformed:
return self._empty_frame()
raise OandaDataError(
f"No complete candles from Oanda for {instrument}")
+4
View File
@@ -4,3 +4,7 @@ requests>=2.28
twelvedata>=1.2.0
python-telegram-bot>=20.0
websockets>=12.0
# news_model.py CPU runtime. Dev box (Python 3.12): 1.30.0 / 0.23.2; live box
# (Python 3.9): 1.19.2 / 0.22.2 — identical scores on the 2026-09-27 bundle.
onnxruntime>=1.19,<2
tokenizers>=0.22,<1
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env python3
"""Build agent_panel_base_rates.json from the reference backtest books.
The agent panel (agent_panel.py, prompt version v2) anchors every
persona's probability to how trades of the same class fared in the
validated books: (timeframe, strategy, grade, direction) with fallbacks
to (timeframe, direction) and (timeframe). This script turns the exported
63-column book CSVs into that table. Research-side (pandas only); the
live module reads the JSON and never imports this.
Usage (the section-9 harness books, regenerated from the kept cache):
venv/bin/python scripts/build_panel_base_rates.py \\
--book 5min=/path/m5_w2.csv --book 15min=/path/m15_w2.csv \\
--book 1h=/path/h1_w3.csv --book 4h=/path/h4_w3.csv \\
--out agent_panel_base_rates.json
Statistics per class: n, win_rate (percent of trades with r_multiple >
0), mean_r, tp1_reach_rate (percent whose best excursion reached the
planned first target: mfe_r >= rr_planned), median_bars_held. Every
class with n >= 1 is written; agent_panel._base_rate applies the
minimum-n rule (30) and the fallback order at read time.
Re-run whenever a reference book is re-pinned (CLAUDE.md section 9); the
JSON records each book's file name, MD5 and trade count so the table's
provenance is auditable.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import sys
from datetime import datetime, timezone
from typing import Dict, List
import pandas as pd
STATS_COLS = ("r_multiple", "mfe_r", "rr_planned", "bars_held",
"strategy", "quality", "direction")
def _md5(path: str) -> str:
h = hashlib.md5()
with open(path, "rb") as fh:
for chunk in iter(lambda: fh.read(1 << 20), b""):
h.update(chunk)
return h.hexdigest()
def _stats(df: pd.DataFrame, book: str) -> Dict[str, object]:
r = df["r_multiple"].astype(float)
reached = (df["mfe_r"].astype(float) >= df["rr_planned"].astype(float))
return {
"n": int(len(df)),
"win_rate": round(float((r > 0).mean() * 100.0), 2),
"mean_r": round(float(r.mean()), 4),
"tp1_reach_rate": round(float(reached.mean() * 100.0), 2),
"median_bars_held": int(round(float(df["bars_held"].median()))),
"book": book,
}
def build(books: Dict[str, str]) -> Dict[str, object]:
classes: Dict[str, Dict[str, object]] = {}
meta: Dict[str, Dict[str, object]] = {}
for tf, path in books.items():
df = pd.read_csv(path)
missing = [c for c in STATS_COLS if c not in df.columns]
if missing:
raise SystemExit(f"{path}: missing columns {missing}")
book = path.rsplit("/", 1)[-1]
meta[tf] = {"file": book, "md5": _md5(path), "trades": int(len(df)),
"first_entry": str(df["entry_time"].min()),
"last_entry": str(df["entry_time"].max())}
if df.empty:
continue
classes[f"{tf}|*|*|*"] = _stats(df, book)
for direction, g in df.groupby("direction"):
classes[f"{tf}|*|*|{direction}"] = _stats(g, book)
for (strategy, grade, direction), g in df.groupby(
["strategy", "quality", "direction"]):
classes[f"{tf}|{strategy}|{grade}|{direction}"] = _stats(g, book)
return {
"schema": "agent-panel-base-rates-v1",
"generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
"key": "timeframe|strategy|grade|direction ('*' = any)",
"min_n_for_exact_class": 30,
"books": meta,
"classes": dict(sorted(classes.items())),
}
def main(argv: List[str] | None = None) -> int:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--book", action="append", default=[], metavar="TF=CSV",
help="timeframe (5min/15min/1h/4h) = exported book CSV")
ap.add_argument("--out", default="agent_panel_base_rates.json")
args = ap.parse_args(argv)
books: Dict[str, str] = {}
for item in args.book:
if "=" not in item:
ap.error(f"--book expects TF=CSV, got {item!r}")
tf, path = item.split("=", 1)
books[tf.strip()] = path.strip()
if not books:
ap.error("at least one --book is required")
table = build(books)
with open(args.out, "w", encoding="utf-8") as fh:
json.dump(table, fh, indent=1, sort_keys=False)
fh.write("\n")
n_exact = sum(1 for k in table["classes"] if "|*|" not in k)
print(f"wrote {args.out}: {len(table['classes'])} classes "
f"({n_exact} exact) from {len(books)} book(s)")
for tf, m in table["books"].items():
print(f" {tf:6s} {m['file']} md5={m['md5']} trades={m['trades']}")
return 0
if __name__ == "__main__":
sys.exit(main())
+16 -5
View File
@@ -23,6 +23,8 @@ import argparse
import os
import sqlite3
import sys
from datetime import datetime, timedelta, timezone
from typing import Optional
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
@@ -33,18 +35,27 @@ CHUNK = 2000
def _counts(con: sqlite3.Connection, symbol: str,
labelled: bool = True) -> dict:
labelled: bool = True, now: Optional[datetime] = None) -> dict:
"""Candidate / labelled / complete / pending counts (read-only).
The age filter uses logged_at (always an ISO string) as a proxy for
the bar close — the labeller itself parses the real bar label."""
the bar close — the labeller itself parses the real bar label. The
cutoff is built here in logged_at's own format (UTC, 'T' separator),
the labeller's convention: SQLite's datetime() prints a space, which
sorts before the 'T', so every row logged on the cutoff's calendar
day was left out of the count."""
moment = now or datetime.now(timezone.utc)
if moment.tzinfo is None:
moment = moment.replace(tzinfo=timezone.utc)
cutoff = moment.astimezone(timezone.utc) - timedelta(
hours=SCAN_LABEL_MIN_AGE_H)
scans = ("SELECT COUNT(*) FROM signals s WHERE s.symbol = ?"
" AND s.indicators_json IS NOT NULL AND s.price > 0")
due = (scans + " AND s.logged_at <"
f" datetime('now', '-{int(SCAN_LABEL_MIN_AGE_H)} hours')")
due = scans + " AND s.logged_at < ?"
out = {
"scan rows with indicators": con.execute(scans, (symbol,)).fetchone()[0],
"of those, window elapsed": con.execute(due, (symbol,)).fetchone()[0],
"of those, window elapsed": con.execute(
due, (symbol, cutoff.isoformat())).fetchone()[0],
}
if labelled:
out["labelled"] = con.execute(
+528
View File
@@ -0,0 +1,528 @@
#!/usr/bin/env python3
"""Offline replay of the agent panel on logged setups, scored against the
realised outcome — so a prompt or payload version is judged on the SAME
trades as its predecessor before it is enabled live.
venv/bin/python scripts/panel_replay.py --db logs/signals.db --dry-run
venv/bin/python scripts/panel_replay.py --db logs/signals.db --max-calls 160 --out replay.csv
What it does
------------
1. Loads every resolved setup event (analyze_signal_log's dedup: one
event per identical setup, first occurrence) whose advisory outcome is
RESOLVED; the outcome is `advisory_r_multiple > 0` (net R at $0.65).
2. Rebuilds what each persona would see. Rows logged by agent_panel v2
carry the exact payloads (panel_payloads_json); older rows are
reconstructed from the stored indicators_json / setup_json / mtf_json /
penalty_tags_json / reasons_json / warnings_json, the interned
headlines and events (signals_with_news), and candles from the kept
Oanda cache windows so the price path is what the scan saw. Calendar
events are evaluated AS OF the signal bar (released / minutes_until),
not as of today.
3. Runs the CURRENT prompts (agent_panel.PANEL_PROMPT_VERSION) through the
real DeepSeek client on each payload, validates the replies with the
live code path, aggregates with agent_panel.mean_verdict.
4. Scores three predictors on the same events: the new run, the logged
v1 takes (stance x confidence -> pseudo-probability, when present) and
the base-rate-only predictor (the class win rate for every trade):
Brier score, log-loss, calibration by probability band, discrimination
(mean R and win rate by band), constant-voter rate per persona, and the
counterfactual books (all / skip p <= 40 / only p >= 60).
Spend: 4 calls per event against the shared DeepSeek daily cap (500);
`--max-calls` is a hard stop. `--dry-run` builds and writes the payloads
without a single call. Read-only on the database.
Caveat printed with every summary: a few dozen outcomes from one regime
can reject a pathological prompt (constant voting, narrative anchoring);
they cannot prove accuracy. The live scorecard at 100 and 300 outcomes
(analyze_signal_log.py --stored-candles) is the real test.
"""
from __future__ import annotations
import argparse
import glob
import json
import math
import os
import sys
from datetime import datetime, timedelta, timezone
from types import SimpleNamespace
from typing import Any, Dict, List, Optional, Sequence, Tuple
import pandas as pd
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if ROOT not in sys.path:
sys.path.insert(0, ROOT)
import agent_panel as ap # noqa: E402
from analyze_signal_log import (cluster_setup_rows, load_rows, # noqa: E402
open_readonly)
from config import SignalStrength, Timeframe # noqa: E402
from news_engine import (EconomicEvent, FundamentalContext, # noqa: E402
NewsSentiment)
from signal_aggregator import SignalReport, TradeSetup # noqa: E402
from signal_log import read_news # noqa: E402
PERSONAS = ap.ALL_PERSONAS # research default: score every persona
CACHE_DIR = os.path.join(ROOT, "backtest", "cache")
_TF = {tf.value: tf for tf in Timeframe}
# ── Reconstruction ────────────────────────────────────────────────────
class _AsOfEvent(EconomicEvent):
"""EconomicEvent whose `is_upcoming` / `minutes_until` are evaluated
relative to the signal bar, not the wall clock."""
def __init__(self, as_of: datetime, **kw):
super().__init__(**kw)
self._as_of = as_of
@property
def is_upcoming(self) -> bool: # type: ignore[override]
return self.time > self._as_of
@property
def minutes_until(self) -> float: # type: ignore[override]
return (self.time - self._as_of).total_seconds() / 60.0
def _dt(s: Optional[str]) -> Optional[datetime]:
if not s:
return None
try:
d = datetime.fromisoformat(str(s).replace("Z", "+00:00"))
except ValueError:
return None
return d if d.tzinfo else d.replace(tzinfo=timezone.utc)
def _loads(s: Any, default: Any) -> Any:
if s in (None, "", "null"):
return default
try:
return json.loads(s)
except (TypeError, ValueError):
return default
def _analysis(tf: str, blob: Dict[str, Any]) -> SimpleNamespace:
inds = {}
for name, d in (blob.get("indicators") or {}).items():
try:
sig = SignalStrength[d.get("signal", "NEUTRAL")]
except KeyError:
sig = SignalStrength.NEUTRAL
inds[name] = SimpleNamespace(
signal=sig, confidence=float(d.get("confidence") or 0.0),
weight=float(d.get("weight") or 0.0), value=d.get("value"),
description=d.get("description") or "")
return SimpleNamespace(timeframe=tf, timestamp=blob.get("timestamp"),
key_levels=dict(blob.get("key_levels") or {}),
indicators=inds)
def _context_from_row(events_json: Optional[str], headlines_json: Optional[str],
as_of: datetime) -> FundamentalContext:
ctx = FundamentalContext()
ev = _loads(events_json, {}) or {}
for key in ("events_today", "events_upcoming_1h"):
out = []
for e in ev.get(key) or []:
t = _dt(e.get("time"))
if t is None:
continue
out.append(_AsOfEvent(as_of, time=t, currency=e.get("currency") or "",
impact=e.get("impact") or "", event=e.get("event") or "",
forecast=e.get("forecast"), previous=e.get("previous"),
actual=e.get("actual")))
setattr(ctx, key, out)
ctx.news_sentiment = dict(ev.get("sentiment_by_currency") or {})
ctx.calendar_available = bool(ev.get("calendar_available", False))
for h in _loads(headlines_json, []) or []:
t = _dt(h.get("timestamp")) or as_of
ctx.news_headlines.append(NewsSentiment(
headline=h.get("headline") or "", source=h.get("source") or "",
timestamp=t, currencies_mentioned=list(h.get("currencies") or []),
sentiment_score=float(h.get("score") or 0.0),
keywords=list(h.get("keywords") or []),
sentiment_by_currency={k: float(v) for k, v in
(h.get("score_by_currency") or {}).items()
if v is not None},
score_source=h.get("score_source") or "keyword"))
return ctx
_frame_cache: Dict[str, pd.DataFrame] = {}
_have_cols: set = set()
def _col(name: str) -> str:
"""Column reference that tolerates a database logged before the
column existed (a read-only copy is never migrated)."""
return name if name in _have_cols else f"NULL AS {name}"
def _cached_frame(tf: str) -> Optional[pd.DataFrame]:
"""Newest kept Oanda cache window for `tf` (raw bars, UTC index)."""
if tf in _frame_cache:
return _frame_cache[tf]
files = sorted(glob.glob(os.path.join(CACHE_DIR, f"oanda_XAU_USD_{tf}_*.parquet")),
key=lambda p: p.rsplit("_", 1)[-1])
df = None
if files:
try:
df = pd.read_parquet(files[-1])
if df.index.tz is None:
df.index = df.index.tz_localize("UTC")
except Exception:
df = None
_frame_cache[tf] = df
return df
def _frames_as_of(tfs: Sequence[str], bar_ts: datetime) -> Dict[Timeframe, pd.DataFrame]:
out: Dict[Timeframe, pd.DataFrame] = {}
for tf in tfs:
df = _cached_frame(tf)
tfe = _TF.get(tf)
if df is None or tfe is None:
continue
sl = df[df.index <= pd.Timestamp(bar_ts)]
if len(sl) >= 3:
out[tfe] = sl
return out
def build_report(con, rowid: int) -> Tuple[Optional[SignalReport], Optional[FundamentalContext],
Dict[Timeframe, pd.DataFrame], Optional[Dict[str, Any]]]:
"""(report, context, frames, stored_payloads) for one logged row."""
row = con.execute(
"SELECT symbol, timeframe, " + _col("bar_ts") + ", logged_at, signal, score,"
" confidence, quality, price, market_regime, volatility_regime, mtf_json,"
" setup_json, reasons_json, warnings_json, event_risk, event_description,"
" news_sentiment, indicators_json, " + _col("penalty_tags_json") + ", "
+ _col("exhaustion_count") + ", " + _col("penalty_total") + ", "
+ _col("momentum_fading") + ", " + _col("panel_payloads_json")
+ " FROM signals WHERE id=?", (rowid,)).fetchone()
if row is None:
return None, None, {}, None
(symbol, tf, bar_ts, logged_at, signal, score, conf, quality, price, regime, vol,
mtf_json, setup_json, reasons_json, warnings_json, event_risk, event_desc, news_sent,
ind_json, pen_json, exh, pen_total, mom, payloads_json) = row
inds = _loads(ind_json, {}) or {}
bar_label = bar_ts or (inds.get(tf) or {}).get("timestamp")
as_of_bar = _dt(bar_label) or _dt(logged_at)
if as_of_bar is None:
return None, None, {}, None
secs = ap._TF_SECONDS.get(tf, 900)
as_of = as_of_bar + timedelta(seconds=secs) # the bar closed
report = SignalReport(symbol=symbol, display_name="XAU/USD",
timestamp=str(bar_label or logged_at))
try:
report.final_signal = SignalStrength[signal]
except KeyError:
report.final_signal = SignalStrength.NEUTRAL
report.final_score = float(score or 0.0)
report.final_confidence = float(conf or 0.0)
report.signal_quality = quality or ""
report.current_price = float(price or 0.0)
report.market_regime = regime or ""
report.volatility_regime = vol or ""
mtf = _loads(mtf_json, {}) or {}
report.mtf_agreement = bool(mtf.get("agreement", False))
for k, v in (mtf.get("signals") or {}).items():
try:
report.mtf_signals[k] = SignalStrength[v]
except KeyError:
report.mtf_signals[k] = SignalStrength.NEUTRAL
report.mtf_scores = {k: float(v or 0.0) for k, v in (mtf.get("scores") or {}).items()}
reasons = _loads(reasons_json, {}) or {}
report.bullish_reasons = list(reasons.get("bullish") or [])
report.bearish_reasons = list(reasons.get("bearish") or [])
report.warnings = list(_loads(warnings_json, []) or [])
report.event_risk = event_risk or "clear"
report.event_description = event_desc or ""
report.news_sentiment = float(news_sent or 0.0)
report.exhaustion_count = int(exh or 0)
report.penalty_tags = {str(k): float(v or 0.0)
for k, v in (_loads(pen_json, {}) or {}).items()}
report.penalty_total = float(pen_total or 0.0)
report.momentum_fading = bool(mom)
setup = _loads(setup_json, None)
if setup:
report.trade_setup = TradeSetup(
pair=symbol, display_name="XAU/USD", direction=setup.get("direction"),
entry_price=float(setup.get("entry") or 0.0),
stop_loss=float(setup.get("stop_loss") or 0.0),
take_profit_1=float(setup.get("take_profit_1") or 0.0),
take_profit_2=float(setup.get("take_profit_2") or 0.0),
risk_reward=float(setup.get("risk_reward") or 0.0),
risk_pips=float(setup.get("risk_pips") or 0.0),
reward_pips=float(setup.get("reward_pips") or 0.0),
pip_value=0.10, strategy=setup.get("strategy") or "GENERIC",
setup_pattern=setup.get("setup_pattern"),
round_trip_cost=setup.get("round_trip_cost"),
trail_arm_r=setup.get("trail_arm_r"), trail_dist_r=setup.get("trail_dist_r"))
if tf in inds:
report.primary_analysis = _analysis(tf, inds[tf])
report.confirmation_analyses = [_analysis(k, v) for k, v in inds.items() if k != tf]
events_json, headlines_json = read_news(con, rowid)
context = _context_from_row(events_json, headlines_json, as_of)
frames = _frames_as_of([tf] + [k for k in inds if k != tf], as_of_bar)
return report, context, frames, _loads(payloads_json, None)
def stored_personas(stored: Any) -> List[str]:
"""The requested personas whose exact payload the row logged."""
if not isinstance(stored, dict):
return []
return [p for p in PERSONAS if isinstance(stored.get(p), dict)]
def build_jobs(report: SignalReport, context: FundamentalContext,
frames: Dict[Timeframe, pd.DataFrame], stored: Optional[Dict[str, Any]],
settings: Dict[str, Any]) -> Tuple[Tuple[str, str, Dict[str, Any]], ...]:
"""Persona jobs on the CURRENT prompts. A persona whose exact payload
the row logged replays that payload; every other persona is rebuilt
through the live payload builders. Per persona, not all-or-nothing:
the live panel stores only the personas it ran (macro alone since
2026-09-17), so "all four or none" never used a stored payload."""
systems = {"macro": ap._MACRO_SYSTEM, "technical": ap._TECH_SYSTEM,
"risk": ap._RISK_SYSTEM, "contrarian": ap._CONTRARIAN_SYSTEM}
kept = {p: stored[p] for p in stored_personas(stored)}
if len(kept) == len(PERSONAS):
return tuple((p, systems[p], kept[p]) for p in PERSONAS)
candles: Dict[str, Any] = {}
analyses = [report.primary_analysis] + list(report.confirmation_analyses)
counts = [int(settings.get("candle_bars_primary", ap.DEFAULT_CANDLE_BARS_PRIMARY))] + \
[int(settings.get("candle_bars_confirm", ap.DEFAULT_CANDLE_BARS_CONFIRM))] * \
len(report.confirmation_analyses)
for analysis, n in zip(analyses, counts):
if analysis is None:
continue
atr = ap._finite(getattr((analysis.indicators or {}).get("atr"), "value", None))
df = frames.get(_TF.get(analysis.timeframe))
if atr and atr > 0 and df is not None and len(df) >= 2:
candles[analysis.timeframe] = ap._normalize_candles(df, atr, n)
ctx = ap._price_context(report, frames, context)
rebuilt = ap._panel_jobs(report, context, candles or None, ctx, PERSONAS)
return tuple((p, system, kept.get(p, payload)) for p, system, payload in rebuilt)
# ── Scoring ───────────────────────────────────────────────────────────
def _brier(ps: Sequence[float], ys: Sequence[int]) -> float:
return sum((p / 100.0 - y) ** 2 for p, y in zip(ps, ys)) / len(ps)
def _logloss(ps: Sequence[float], ys: Sequence[int]) -> float:
eps = 1e-3
tot = 0.0
for p, y in zip(ps, ys):
q = min(1 - eps, max(eps, p / 100.0))
tot += -(y * math.log(q) + (1 - y) * math.log(1 - q))
return tot / len(ps)
def score_predictor(name: str, ps: Sequence[Optional[float]], rs: Sequence[float]) -> Dict[str, Any]:
pairs = [(p, r) for p, r in zip(ps, rs) if p is not None]
if not pairs:
return {"predictor": name, "n": 0}
P = [p for p, _ in pairs]; R = [r for _, r in pairs]; Y = [1 if r > 0 else 0 for r in R]
bands = {}
for label, lo, hi in (("<=40", -1, ap.P_OPPOSE), ("40-60", ap.P_OPPOSE, ap.P_SUPPORT),
(">=60", ap.P_SUPPORT, 101)):
sel = [(p, r) for p, r in pairs if (lo < p <= hi if label == "<=40" else
lo < p < hi if label == "40-60" else p >= lo)]
if sel:
bands[label] = {"n": len(sel), "mean_p": round(sum(p for p, _ in sel) / len(sel), 1),
"win_rate": round(100 * sum(1 for _, r in sel if r > 0) / len(sel), 1),
"mean_r": round(sum(r for _, r in sel) / len(sel), 3),
"sum_r": round(sum(r for _, r in sel), 2)}
else:
bands[label] = {"n": 0}
stances = [ap.derive_stance(p) for p in P]
modal = max(set(stances), key=stances.count)
return {
"predictor": name, "n": len(P),
"brier": round(_brier(P, Y), 4), "logloss": round(_logloss(P, Y), 4),
"mean_p": round(sum(P) / len(P), 1), "win_rate": round(100 * sum(Y) / len(Y), 1),
"bands": bands, "modal_stance": modal,
"constant_voter_rate": round(100 * stances.count(modal) / len(stances), 1),
"sum_r_all": round(sum(R), 2),
"sum_r_skip_oppose": round(sum(r for p, r in pairs if p > ap.P_OPPOSE), 2),
"sum_r_support_only": round(sum(r for p, r in pairs if p >= ap.P_SUPPORT), 2),
}
def print_summary(results: List[Dict[str, Any]]) -> None:
print("\n== Predictor scorecard (outcome = advisory R > 0 at $0.65) ==")
print(f"{'predictor':22s} {'n':>3s} {'Brier':>6s} {'logloss':>7s} {'mean p':>6s} {'win%':>5s} "
f"{'modal':>8s} {'const%':>6s} {'R all':>7s} {'R skip<=40':>10s} {'R >=60 only':>11s}")
for r in results:
if r.get("n", 0) == 0:
print(f"{r['predictor']:22s} 0"); continue
print(f"{r['predictor']:22s} {r['n']:3d} {r['brier']:6.3f} {r['logloss']:7.3f} {r['mean_p']:6.1f} "
f"{r['win_rate']:5.1f} {r['modal_stance']:>8s} {r['constant_voter_rate']:6.1f} "
f"{r['sum_r_all']:+7.2f} {r['sum_r_skip_oppose']:+10.2f} {r['sum_r_support_only']:+11.2f}")
print("\n calibration bands (n / mean p / observed win% / mean R):")
for r in results:
if r.get("n", 0) == 0:
continue
cells = " ".join(f"{k}: {v['n']}/{v.get('mean_p', '-')}/{v.get('win_rate', '-')}/{v.get('mean_r', '-')}"
if v["n"] else f"{k}: 0" for k, v in r["bands"].items())
print(f" {r['predictor']:22s} {cells}")
print("\nCaveat: a few dozen outcomes from one regime can reject a pathological prompt "
"(constant voting, narrative anchoring); they cannot prove accuracy.")
# ── Main ──────────────────────────────────────────────────────────────
def main(argv: Optional[Sequence[str]] = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("--db", default=os.path.join(ROOT, "logs", "signals.db"))
parser.add_argument("--limit", type=int, default=0, help="max events (0 = all)")
parser.add_argument("--max-calls", type=int, default=200,
help="hard stop on LLM requests (4 per event)")
parser.add_argument("--dry-run", action="store_true",
help="build payloads only; no LLM call")
parser.add_argument("--out", default=None, help="per-event CSV (and .payloads.json on --dry-run)")
parser.add_argument("--include-pending", action="store_true",
help="also build payloads for unresolved setups (dry-run only)")
parser.add_argument("--personas", default=",".join(ap.ALL_PERSONAS),
help="comma-separated personas to run (default: all four, so every "
"voice is scored; the live default is agent_panel.personas = macro)")
args = parser.parse_args(argv)
global PERSONAS
PERSONAS = ap.normalize_personas([p for p in args.personas.split(",") if p.strip()])
rows = load_rows(args.db)
events = cluster_setup_rows(rows)
con = open_readonly(args.db)
global _have_cols
_have_cols = {r[1] for r in con.execute("PRAGMA table_info(signals)")}
settings = {"candle_bars_primary": ap.DEFAULT_CANDLE_BARS_PRIMARY,
"candle_bars_confirm": ap.DEFAULT_CANDLE_BARS_CONFIRM}
picked = []
for ev in events:
res = next((r for r in ev.rows if r.advisory_status == "RESOLVED" and r.advisory_r is not None), None)
if res is None and not (args.dry_run and args.include_pending):
continue
picked.append((ev, res))
if args.limit:
picked = picked[:args.limit]
print(f"events: {len(events)} total, {len(picked)} selected "
f"({'dry run' if args.dry_run else 'live calls, max ' + str(args.max_calls)}), "
f"prompt version {ap.PANEL_PROMPT_VERSION}, personas {','.join(PERSONAS)}")
client = None if args.dry_run else ap.get_llm_client()
calls = 0
per_event: List[Dict[str, Any]] = []
payload_dump: Dict[str, Any] = {}
for ev, res in picked:
rep = ev.representative
# Prefer a member row that already carries the v2 payloads.
rowid = rep.id
for r in ev.rows:
has = (con.execute("SELECT panel_payloads_json IS NOT NULL FROM signals"
" WHERE id=?", (r.id,)).fetchone()
if "panel_payloads_json" in _have_cols else None)
if has and has[0]:
rowid = r.id
break
report, context, frames, stored = build_report(con, rowid)
if report is None or report.trade_setup is None:
continue
jobs = build_jobs(report, context, frames, stored, settings)
ctx = next((payload.get("context") for _, _, payload in jobs if payload.get("context")), {}) or {}
base_p = (ctx.get("base_rate") or {}).get("win_rate")
# logged v1 takes (any member row) -> pseudo-probabilities
v1 = {}
for r in ev.rows:
takes = con.execute("SELECT panel_takes_json, " + _col("panel_prompt_version")
+ " FROM signals WHERE id=?", (r.id,)).fetchone()
if takes and takes[0]:
for t in _loads(takes[0], []) or []:
p = t.get("p_win")
if p is None and t.get("stance") in ap._VALID_STANCES:
p = ap.pseudo_probability(t["stance"], float(t.get("confidence") or 0))
if p is not None:
v1[t.get("persona")] = float(p)
v1["_version"] = takes[1] or "v1-logged"
break
entry: Dict[str, Any] = {
"event_row": rep.id, "bar": report.timestamp, "tf": rep.timeframe,
"direction": report.trade_setup.direction, "grade": rep.quality,
"strategy": report.trade_setup.strategy,
# which personas replayed the exact logged payload ("" = all rebuilt)
"R": res.advisory_r if res else None,
"stored_payloads": ",".join(stored_personas(stored)),
"base_p": base_p, "prior_move_4h_atr": ctx.get("prior_move_4h_atr"),
"move_from_start_atr": ctx.get("move_from_24h_start_atr"),
"big_candle_last3": ctx.get("big_candle_last3"),
}
for p in PERSONAS:
entry[f"v1_{p}"] = v1.get(p)
v1_ps = [v1[p] for p in PERSONAS if p in v1]
entry["v1_mean"] = round(sum(v1_ps) / len(v1_ps), 1) if len(v1_ps) >= min(ap.MEAN_QUORUM, len(v1_ps) or 1) and v1_ps else None
if args.dry_run:
payload_dump[str(rep.id)] = {name: payload for name, _, payload in jobs}
else:
takes = []
for name, system, payload in jobs:
if calls >= args.max_calls:
break
calls += 1
reply = client.chat_json(system, payload)
take = ap._validate_take(name, reply)
if take is not None:
takes.append(take)
entry[f"new_{name}"] = take.p_win
entry[f"new_{name}_for"] = take.reason_for
entry[f"new_{name}_against"] = take.reason_against
v = ap.mean_verdict(takes, quorum=min(ap.MEAN_QUORUM, len(jobs)))
entry["new_mean"] = v.probability if v else None
entry["new_verdict"] = v.verdict if v else None
print(f" #{rep.id:<5d} {report.timestamp[:16]} {rep.timeframe:5s} "
f"{report.trade_setup.direction:4s} {rep.quality:2s} R={entry['R']:+.2f} "
f"base={base_p} new_mean={entry['new_mean']} "
f"({', '.join(f'{t.persona[:4]} {t.p_win:.0f}' for t in takes)}) "
f"v1_mean={entry['v1_mean']}")
per_event.append(entry)
if not args.dry_run and calls >= args.max_calls:
print(f" stopped at --max-calls {args.max_calls}")
break
con.close()
if args.dry_run:
print(f"built payloads for {len(per_event)} events")
if args.out:
with open(args.out.rsplit('.', 1)[0] + ".payloads.json", "w") as fh:
json.dump(payload_dump, fh, indent=1, sort_keys=True)
print(f"wrote {args.out.rsplit('.', 1)[0]}.payloads.json")
scored = [e for e in per_event if e["R"] is not None]
if scored:
rs = [e["R"] for e in scored]
results = [score_predictor("base-rate only",
[e["base_p"] if e["base_p"] is not None else None for e in scored], rs),
score_predictor("v1 logged (mean)", [e["v1_mean"] for e in scored], rs)]
for p in PERSONAS:
results.append(score_predictor(f"v1 logged {p}", [e.get(f"v1_{p}") for e in scored], rs))
if not args.dry_run:
results.append(score_predictor(f"NEW {ap.PANEL_PROMPT_VERSION} (mean)",
[e.get("new_mean") for e in scored], rs))
for p in PERSONAS:
results.append(score_predictor(f"NEW {p}", [e.get(f"new_{p}") for e in scored], rs))
print_summary(results)
if args.out and per_event:
pd.DataFrame(per_event).to_csv(args.out, index=False)
print(f"wrote {args.out} ({len(per_event)} events)")
return 0
if __name__ == "__main__":
sys.exit(main())
+432 -48
View File
@@ -14,6 +14,7 @@ from datetime import datetime, timezone
from typing import Dict, List, Optional
from zoneinfo import ZoneInfo
import numpy as np
import pandas as pd
_NY_TZ = ZoneInfo("America/New_York")
@@ -33,6 +34,187 @@ import signal_log
logger = logging.getLogger(__name__)
# ── Grading rule (score + confidence -> final signal and grade) ─────────
# "pair" is the shipped rule; "min" / "product" are the 2026-09-27 research
# candidates (backtest/research/conviction_campaign_2026-09-27.md). The CUDA
# twin (backtest/cuda_signal_kernels.py, K2) mirrors these helpers and packs
# CONVICTION_THRESHOLDS from here, so this module is the single source.
GRADING_MODES = ("pair", "min", "product")
# (aplus, a, b, c, strong, buy, boost_a, boost_b) on the conviction scale.
# "min": the control's corners on the |score| axis (0.6 = the control's
# score / confidence ratio). "product": the control's corner products
# 40x65, 33x55, 28x50, 25x45, 35x55, 25x50, 30x50, 25x45, each / 100.
CONVICTION_THRESHOLDS = {
"min": (40.0, 33.0, 28.0, 25.0, 35.0, 25.0, 30.0, 25.0),
"product": (26.0, 18.15, 14.0, 11.25, 19.25, 12.5, 15.0, 11.25),
}
def grading_mode_of(signals_cfg) -> str:
"""The validated `signals.grading_mode`; an unknown value is a hard error."""
mode = signals_cfg.grading_mode
if mode not in GRADING_MODES:
raise ValueError(
f"signals.grading_mode={mode!r}; expected one of {GRADING_MODES}"
)
return mode
def conviction(abs_score: float, confidence: float, mode: str) -> float:
"""The single decision number of the research modes. Evaluation order is
load-bearing for the CUDA mirror: min(a, b) with b = 0.6 * confidence in
that argument order (PYMIN), product as (abs_score * confidence) / 100."""
if mode == "min":
return min(abs_score, 0.6 * confidence)
if mode == "product":
return abs_score * confidence / 100.0
raise ValueError(f"conviction() has no formula for grading_mode={mode!r}")
def _signal_pair(score: float, confidence: float) -> SignalStrength:
"""The shipped final-signal rule (validated literals; do not edit)."""
if score >= 35 and confidence >= 55:
return SignalStrength.STRONG_BUY
elif score >= 25 and confidence >= 50:
return SignalStrength.BUY
elif score <= -35 and confidence >= 55:
return SignalStrength.STRONG_SELL
elif score <= -25 and confidence >= 50:
return SignalStrength.SELL
return SignalStrength.NEUTRAL
def _quality_pair(abs_score: float, confidence: float, mtf: bool) -> str:
"""The shipped grade table (validated literals; do not edit)."""
if abs_score >= 40 and confidence >= 65 and mtf:
return "A+"
elif abs_score >= 33 and confidence >= 55 and mtf:
return "A"
elif abs_score >= 28 and confidence >= 50:
return "B"
elif abs_score >= 25 and confidence >= 45:
return "C"
elif abs_score >= 15:
return "D"
return "F"
def _signal_conv(score: float, conv: float, mode: str) -> SignalStrength:
t_strong, t_buy = CONVICTION_THRESHOLDS[mode][4:6]
if score > 0 and conv >= t_strong:
return SignalStrength.STRONG_BUY
elif score > 0 and conv >= t_buy:
return SignalStrength.BUY
elif score < 0 and conv >= t_strong:
return SignalStrength.STRONG_SELL
elif score < 0 and conv >= t_buy:
return SignalStrength.SELL
return SignalStrength.NEUTRAL
def _quality_conv(abs_score: float, conv: float, mtf: bool, mode: str) -> str:
t_aplus, t_a, t_b, t_c = CONVICTION_THRESHOLDS[mode][0:4]
if conv >= t_aplus and mtf:
return "A+"
elif conv >= t_a and mtf:
return "A"
elif conv >= t_b:
return "B"
elif conv >= t_c:
return "C"
elif abs_score >= 15:
return "D"
return "F"
def _final_signal_and_grade(score: float, confidence: float, mtf: bool, mode: str):
"""(final_signal, signal_quality) for the post-penalty score / confidence.
"pair" is the shipped rule; the research modes grade on one conviction
number. The single dispatch point the CUDA K2 twin mirrors."""
if mode == "pair":
return _signal_pair(score, confidence), _quality_pair(abs(score), confidence, mtf)
conv = conviction(abs(score), confidence, mode)
return _signal_conv(score, conv, mode), _quality_conv(abs(score), conv, mtf, mode)
def _transitioning_boost(abs_score: float, confidence: float, mtf: bool, mode: str) -> Optional[str]:
"""Gate 6c's grade lift in a transitioning regime: "A", "B" or None.
The "pair" branch is the shipped rule (validated literals; do not edit)."""
if mode == "pair":
if abs_score >= 30 and confidence >= 50 and mtf:
return "A"
elif abs_score >= 25 and confidence >= 45:
return "B"
return None
conv = conviction(abs_score, confidence, mode)
t_boost_a, t_boost_b = CONVICTION_THRESHOLDS[mode][6:8]
if conv >= t_boost_a and mtf:
return "A"
elif conv >= t_boost_b:
return "B"
return None
# ── Daily trend (campaign 2026-09-30) ────────────────────────────────
# One definition for the live scan, the bar replay and the CUDA twin's host
# side (backtest/cudaBasedBacktesting.trend_flags calls daily_trend_series on
# the whole D1 frame and indexes it). Every step is causal (the Sunday
# filter, the EMA recursion, the shift), so the value on a prefix slice is
# bit-identical to the value at the same bar of the full series.
DAILY_TREND_MODES = ("off", "both", "buy", "sell")
DAILY_TREND_EMA_SPAN = 50 # validated literals (do not edit): the
DAILY_TREND_LOOKBACK = 60 # campaign tested exactly this definition
_DAILY_TREND_PREFIX = "DAILY TREND FILTER:"
def daily_trend_mode_of(xau_cfg) -> str:
"""The validated `xau_strategy.daily_trend_veto`; an unknown value is a hard error."""
mode = xau_cfg.daily_trend_veto
if mode not in DAILY_TREND_MODES:
raise ValueError(
f"xau_strategy.daily_trend_veto={mode!r}; expected one of {DAILY_TREND_MODES}"
)
return mode
def daily_trend_needed(config: PlatformConfig) -> bool:
"""True when the scan needs the D1 frame: the veto is on for a gold config."""
return (daily_trend_mode_of(config.xau_strategy) != "off"
and config.xau_strategy.enabled
and config.display_name == "XAU/USD")
def daily_trend_series(d1: Optional[pd.DataFrame]) -> pd.Series:
"""Daily trend per CLOSED D1 bar: +1 when the close is above its 50-bar
EMA and above the close 60 bars earlier, -1 when below both, else 0.
`d1` must hold closed bars only (the provider drops the forming candle;
the replay slices as-of). Bars labelled Sunday are dropped first: the
provider's UTC-aligned daily candles include a 1-2 hour stub for the
Sunday reopening, which is not a trading day. The first 60 kept bars
read 0 (no 60-bar change yet)."""
if d1 is None or len(d1) == 0:
return pd.Series([], dtype="int8")
kept = d1[d1.index.weekday != 6]
close = kept["Close"].astype("float64")
ema = close.ewm(span=DAILY_TREND_EMA_SPAN, adjust=False).mean()
change = close / close.shift(DAILY_TREND_LOOKBACK) - 1.0
up = (close > ema) & (change > 0)
down = (close < ema) & (change < 0)
state = np.where(up, 1, np.where(down, -1, 0)).astype("int8")
return pd.Series(state, index=kept.index)
def daily_trend(d1: Optional[pd.DataFrame]) -> Optional[int]:
"""The daily trend at the last closed D1 bar of `d1` (+1 / 0 / -1), or
None when the frame holds no usable bar."""
series = daily_trend_series(d1)
if len(series) == 0:
return None
return int(series.iloc[-1])
def _is_forex_weekend(now_utc: datetime) -> bool:
"""Gold/FX market closed for the weekend: Friday 17:00 → Sunday 18:00
NEW YORK time (= Fri 21:00 UTC in US summer, 22:00 in winter —
@@ -115,7 +297,16 @@ class SignalReport:
panel_created_at: str = ""
panel_cache_status: str = ""
panel_payload_hash: str = ""
panel_verdict_source: str = "" # "macro" / "chair" / ""
panel_verdict_source: str = "" # "mean" / "macro" / "chair" / ""
# v2 panel (2026-09-17): the mean persona probability (0-100) that
# the trade closes profitably, the prompt version that produced it,
# the exact per-persona payloads and the shared code-computed
# context — all advisory/DISPLAY-ONLY, logged for the offline replay
# harness (scripts/panel_replay.py) and the calibration scorecard.
panel_probability: float = 0.0
panel_prompt_version: str = ""
panel_payloads: Dict = field(default_factory=dict)
panel_context: Dict = field(default_factory=dict)
# Violent-move watch line (advisory/DISPLAY-ONLY). Stamped post-hoc
# by surfaces via violent_watch.attach_watch — never populated
@@ -153,6 +344,17 @@ class SignalReport:
penalty_tags: Dict[str, float] = field(default_factory=dict)
penalty_total: float = 0.0
momentum_fading: bool = False
# Daily trend at the last closed D1 bar (+1 up / 0 unclear / -1 down);
# None = not computed. Set by the core when the daily-trend veto is on
# (_apply_daily_trend_veto); where the veto is off the LIVE wrapper
# stamps it for the alert text and the signal log only
# (_stamp_daily_trend_for_display) — no gate reads it there.
daily_trend: Optional[int] = None
# Lot-size and open-exposure lines (advisory/DISPLAY-ONLY). Stamped
# post-hoc by surfaces via exposure.attach_exposure — never populated
# inside analyze_from_dataframes and never read by any gate.
size_line: str = ""
open_line: str = ""
class SignalAggregator:
@@ -160,6 +362,12 @@ class SignalAggregator:
def __init__(self, config: PlatformConfig):
self.config = config
grading_mode_of(config.signals) # fail at start, not at the first bar
daily_trend_mode_of(config.xau_strategy)
if not config.xau_strategy.wide_stop_cap_atr >= 0:
raise ValueError(
f"xau_strategy.wide_stop_cap_atr={config.xau_strategy.wide_stop_cap_atr!r}; "
f"expected a number >= 0 (0 = off)")
self.data_provider = make_data_provider(config)
self.tech_engine = TechnicalEngine(config.technical)
self._news_engine: Optional[NewsEngine] = None # lazy
@@ -230,6 +438,12 @@ class SignalAggregator:
bt = self.config.backtest
if bt.regime_filter and bt.regime_filter_timeframe not in timeframes:
timeframes.append(bt.regime_filter_timeframe)
# The daily-trend veto reads the D1 frame inside the core, so it is
# a REQUIRED frame when the veto is on (a failed fetch fails the
# scan below, as for any other frame). The backtesters append it
# under the same guard (rule 4).
if daily_trend_needed(self.config) and Timeframe.D1 not in timeframes:
timeframes.append(Timeframe.D1)
dfs: Dict[Timeframe, pd.DataFrame] = {}
stale_tfs = []
@@ -277,6 +491,12 @@ class SignalAggregator:
dfs=dfs,
use_news=True,
)
# Live-only hand-off for the post-hoc advisory add-ons (the agent
# panel's price context reads the frames this scan analysed, so
# it sees exactly the bars the engine saw). A private attribute
# like _signal_log_rowid: not a dataclass field, never read by
# the core, absent on backtest replays by construction.
report._frames = dfs
if stale_tfs:
report.warnings.append(f"STALE DATA on {', '.join(stale_tfs)} — signal may be unreliable")
@@ -327,9 +547,34 @@ class SignalAggregator:
f"{symbol}: live session filter errored — filter SKIPPED this scan"
)
self._stamp_daily_trend_for_display(report, dfs)
self._log_signal(report, log_source, dfs)
return report
def _stamp_daily_trend_for_display(
self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
) -> None:
"""Daily trend for the alert text and the signal log on the
timeframes where the daily-trend veto is OFF (the core stamps it
where the veto is on). Display and record-keeping only: the frame
is never added to `dfs`, nothing reads the value back, and a failed
fetch leaves it None instead of failing the scan. The scan's own D1
frame is used when it has one (H4 confirms on D1); otherwise the
provider's D1 frame, cached until the next daily close and
refreshed by the M15 scan, which needs it as a required frame.
Live wrapper only — the backtesters never come through here."""
if report.daily_trend is not None or report.display_name != "XAU/USD":
return
if report.primary_analysis is None:
return
try:
d1 = dfs.get(Timeframe.D1)
if d1 is None:
d1 = self.data_provider.fetch_ohlcv(report.symbol, Timeframe.D1)
report.daily_trend = daily_trend(d1)
except Exception:
logger.debug("daily trend (display) unavailable", exc_info=True)
def _log_signal(
self, report: SignalReport, source: Optional[str],
dataframes: Optional[Dict[Timeframe, pd.DataFrame]] = None,
@@ -403,6 +648,13 @@ class SignalAggregator:
bt.session_end_hour],
"regime_filter": bool(bt.regime_filter),
}
# 2026-09-30 knobs enter the hash only when they are in force (the
# grading_mode precedent in engine_config_fingerprint): a config
# that leaves them off keeps the fingerprint its live rows carry.
if xs.daily_trend_veto != "off":
payload["daily_trend_veto"] = xs.daily_trend_veto
if xs.wide_stop_cap_atr > 0:
payload["wide_stop_cap_atr"] = xs.wide_stop_cap_atr
return hashlib.sha1(
json.dumps(payload, sort_keys=True, default=str).encode("utf-8")
).hexdigest()[:12]
@@ -416,9 +668,15 @@ class SignalAggregator:
multipliers actually in force (override included) and the
timeframe set. Logged on every row so indicator-weight research
can tell which rows share a scoring configuration."""
signals = asdict(self.config.signals)
# The research grading switch enters the hash only when it is in
# force: under the shipped "pair" rule the scoring is byte-identical
# to the pre-2026-09-27 code, so live rows keep the same fingerprint.
if signals.get("grading_mode") == "pair":
signals.pop("grading_mode")
payload = {
"technical": asdict(self.config.technical),
"signals": asdict(self.config.signals),
"signals": signals,
"regime_weights": regime_weight_table(self.config.technical),
"primary": self.config.primary_timeframe.value,
"confirm": [tf.value for tf in self.config.confirmation_timeframes],
@@ -888,6 +1146,30 @@ class SignalAggregator:
if height < min_tp:
return
# Wide-stop cap AFTER the gates (xau_strategy.wide_stop_cap_atr,
# 0 = off), same contract as the swing generator's.
cap_dist = self._wide_stop_cap(report, atr, min_sl_price)
if cap_dist is not None and sl_dist > cap_dist:
sl_dist = cap_dist
sl = entry - cap_dist if is_buy else entry + cap_dist
rr = height / sl_dist
# Printed-target floor AFTER the gates (2026-09-26, see
# config.SignalConfig.tp_print_floor_rr): the measured move stays
# the gated target; the printed guide is at least the floor.
# Mirrored in cuda_signal_kernels.py K2.
tp1_dist = height
floor_dist = sl_dist * self._printed_target_floor()
if tp1_dist < floor_dist:
tp1_dist = floor_dist
if is_buy:
tp1 = entry + floor_dist
tp2 = entry + 1.5 * floor_dist
else:
tp1 = entry - floor_dist
tp2 = entry - 1.5 * floor_dist
rr = tp1_dist / sl_dist
report.trade_setup = TradeSetup(
pair=report.symbol,
display_name=report.display_name,
@@ -898,7 +1180,7 @@ class SignalAggregator:
take_profit_2=tp2,
risk_reward=round(rr, 2),
risk_pips=sl_dist / pip_value,
reward_pips=height / pip_value,
reward_pips=tp1_dist / pip_value,
pip_value=pip_value,
strategy="XAU_CP",
setup_pattern=pat["pattern"],
@@ -1055,23 +1337,26 @@ class SignalAggregator:
report.mtf_agreement = self._check_mtf_agreement(report)
# ── 4. Fundamental context (optional) ──────────────────────
# News is display and warning only (it never gates), so a failure
# anywhere in it must not cost the scan: the report then carries
# no news at all, the values the replay path (use_news=False)
# always has. All five fields are set together or not at all.
news_fields = ("clear", "", 0.0, "", "")
if use_news:
news = self._get_news_engine()
if context is None:
context = news.get_fundamental_context()
event_risk, event_desc = news.get_event_risk_level(symbol, context)
report.event_risk = event_risk
report.event_description = event_desc
sentiment, sent_desc = news.get_pair_sentiment(symbol, context)
report.news_sentiment = sentiment
report.news_description = sent_desc
report.news_summary = context.llm_summary
else:
report.event_risk = "clear"
report.event_description = ""
report.news_sentiment = 0.0
report.news_description = ""
report.news_summary = ""
try:
news = self._get_news_engine()
if context is None:
context = news.get_fundamental_context()
event_risk, event_desc = news.get_event_risk_level(symbol, context)
sentiment, sent_desc = news.get_pair_sentiment(symbol, context)
news_fields = (event_risk, event_desc, sentiment, sent_desc,
context.llm_summary)
except Exception:
logger.warning(
f"News context failed for {symbol} — the scan continues "
f"without news", exc_info=True)
(report.event_risk, report.event_description, report.news_sentiment,
report.news_description, report.news_summary) = news_fields
# ── 5. Spread estimate from primary data (avoids extra API call) ─
try:
@@ -1116,10 +1401,14 @@ class SignalAggregator:
# ── 6c. Transitioning regime quality boost ──────────────────
if report.market_regime == "transitioning" and report.signal_quality in ("B", "C", "D"):
abs_score = abs(report.final_score)
if abs_score >= 30 and report.final_confidence >= 50 and report.mtf_agreement:
boost = _transitioning_boost(
abs_score, report.final_confidence, report.mtf_agreement,
grading_mode_of(self.config.signals),
)
if boost == "A":
report.signal_quality = "A"
report.action_recommendation = f"{report.final_signal.name.replace('_', ' ')} — Trend emerging, early entry"
elif abs_score >= 25 and report.final_confidence >= 45:
elif boost == "B":
report.signal_quality = "B"
report.action_recommendation = f"{report.final_signal.name.replace('_', ' ')} — Trend forming, watch closely"
@@ -1307,6 +1596,12 @@ class SignalAggregator:
# cost-floor veto no longer describes it.
self._drop_stale_spread_gate_vetoes(report)
# ── 12. Daily-trend veto (opt-in via per-TF factories) ───────
# Last on purpose: a FINAL veto on whatever archetype shipped the
# setup (core, XAU_PB or XAU_CP), which is exactly what the
# campaign replayed. Mirrored in cuda_signal_kernels.py K2.
self._apply_daily_trend_veto(report, dfs)
return report
# ── Internals ──────────────────────────────────────────────────
@@ -1379,6 +1674,64 @@ class SignalAggregator:
report.warnings[:] = kept
report.cleaned_warnings.extend(stale)
def _apply_daily_trend_veto(
self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
) -> None:
"""Step 12: remove a finished setup that goes against a clear daily
trend (xau_strategy.daily_trend_veto, off by default).
Reads the D1 frame from `dfs` (closed bars only: the live wrapper
fetches it as a required frame, the replay slices it as-of), stamps
report.daily_trend, and follows the gate-chain convention when it
fires: grade D, no setup, a warning that names the gate. The final
signal and the action text are left as they are."""
xcfg = self.config.xau_strategy
mode = daily_trend_mode_of(xcfg)
if mode == "off" or not xcfg.enabled or report.display_name != "XAU/USD":
return
state = daily_trend(dfs.get(Timeframe.D1))
report.daily_trend = state
if state is None:
# Mixed case like the regime filter's twin: never a gate prefix.
report.warnings.append(
"Daily trend filter requested but 1d data unavailable")
return
setup = report.trade_setup
if setup is None or state == 0:
return
is_buy = setup.direction == "BUY"
against = ((is_buy and state < 0 and mode in ("both", "buy"))
or (not is_buy and state > 0 and mode in ("both", "sell")))
if not against:
return
report.warnings.append(
f"{_DAILY_TREND_PREFIX} {setup.direction} against a "
f"{'falling' if is_buy else 'rising'} daily trend — no trade"
)
report.signal_quality = "D"
report.trade_setup = None
def _wide_stop_cap(self, report: SignalReport, atr: float,
stop_floor: float) -> Optional[float]:
"""The largest stop distance xau_strategy.wide_stop_cap_atr allows
for a swing or chart-pattern setup: max(K x ATR, the setup's own
stop floor); None when the cap is off (K = 0) or the pair is not
gold. Applied by the generators AFTER their min-RR and spread
gates, so the gates keep judging the uncapped geometry and the
set of trades cannot change (the printed-target floor precedent)."""
xcfg = self.config.xau_strategy
k = xcfg.wide_stop_cap_atr
if not k > 0 or not xcfg.enabled or report.display_name != "XAU/USD":
return None
return max(k * atr, stop_floor)
def _printed_target_floor(self) -> float:
"""The floor every generator applies to the PRINTED target 1, as a
fraction of the stop: max(tp_floor_rr, tp_print_floor_rr). Applied
after the gates (see config.SignalConfig.tp_print_floor_rr)."""
sig = self.config.signals
return max(sig.tp_floor_rr, getattr(sig, "tp_print_floor_rr", 0.0))
def _generate_xau_atr_trade_setup(
self, report: SignalReport, atr: float
) -> Optional[TradeSetup]:
@@ -1431,6 +1784,31 @@ class SignalAggregator:
)
return None
# Printed-target floor (2026-09-26): the guide target is at least
# max(tp_floor_rr, tp_print_floor_rr) x the stop (1R by default).
# With a 2 x ATR stop the grade-A target of 1.5 x ATR printed
# "RR 1:0.8" (1 x ATR with two exhaustion tags, "1:0.5"); on M15
# the factory floor is 1R, on M5 (0.5) nothing binds. Applied
# AFTER the gates on purpose: the min-RR and spread checks above
# keep judging the generator's own 1.5 x ATR target, so which
# generator serves the signal (and therefore the stop) is
# unchanged and the books are trade-identical (a floor applied
# before the spread gate moved 94 grade-A M15 setups onto the
# 2 x ATR stop and cost -24.6R on W2: rr_and_sell_campaign_
# 2026-09-26.md). TP1 is advisory in the backtester (the trail is
# the exit), so only the exported tp1/tp2/rr_planned columns
# move. Mirrored in cuda_signal_kernels.py K2.
floor_dist = sl_distance * self._printed_target_floor()
if tp_distance < floor_dist:
tp_distance = floor_dist
if is_buy:
tp1 = price + floor_dist
tp2 = price + floor_dist * 1.5
else:
tp1 = price - floor_dist
tp2 = price - floor_dist * 1.5
rr = tp_distance / sl_distance
return TradeSetup(
pair=report.symbol,
display_name=report.display_name,
@@ -1680,36 +2058,14 @@ class SignalAggregator:
)
confidence = 0.0
# Final signal
if score >= 35 and confidence >= 55:
report.final_signal = SignalStrength.STRONG_BUY
elif score >= 25 and confidence >= 50:
report.final_signal = SignalStrength.BUY
elif score <= -35 and confidence >= 55:
report.final_signal = SignalStrength.STRONG_SELL
elif score <= -25 and confidence >= 50:
report.final_signal = SignalStrength.SELL
else:
report.final_signal = SignalStrength.NEUTRAL
# Final signal (grading_mode: "pair" = the shipped rule; the research
# modes grade on one conviction number — helpers at module top)
report.final_signal, report.signal_quality = _final_signal_and_grade(
score, confidence, report.mtf_agreement, grading_mode_of(self.config.signals)
)
report.final_score = score
report.final_confidence = confidence
# Quality grade
abs_score = abs(score)
if abs_score >= 40 and confidence >= 65 and report.mtf_agreement:
report.signal_quality = "A+"
elif abs_score >= 33 and confidence >= 55 and report.mtf_agreement:
report.signal_quality = "A"
elif abs_score >= 28 and confidence >= 50:
report.signal_quality = "B"
elif abs_score >= 25 and confidence >= 45:
report.signal_quality = "C"
elif abs_score >= 15:
report.signal_quality = "D"
else:
report.signal_quality = "F"
if not report.action_recommendation:
sig_name = report.final_signal.name.replace("_", " ")
if report.signal_quality in ("A+", "A"):
@@ -1850,6 +2206,34 @@ class SignalAggregator:
)
return None
# Wide-stop cap AFTER the gates (xau_strategy.wide_stop_cap_atr,
# 0 = off): the gates above judged the swing-anchored stop, so the
# same trades ship; only the stop (the risk unit) is pulled in,
# never inside the generator's own floor. Mirrored in
# cuda_signal_kernels.py K2.
cap_dist = self._wide_stop_cap(
report, atr, max(atr * sl_atr_floor, min_sl_price))
if cap_dist is not None and sl_distance > cap_dist:
sl_distance = cap_dist
sl = entry - cap_dist if is_buy else entry + cap_dist
rr = tp1_distance / sl_distance
# Printed-target floor AFTER the gates (2026-09-26, see
# config.SignalConfig.tp_print_floor_rr): the gates above judged
# the swing/ATR target, so the trade is unchanged; only the guide
# moves. TP2 keeps its "at least 1 ATR beyond TP1" rule.
# Mirrored in cuda_signal_kernels.py K2.
floor_dist = sl_distance * self._printed_target_floor()
if tp1_distance < floor_dist:
tp1_distance = floor_dist
if is_buy:
tp1 = entry + floor_dist
tp2 = max(tp2, tp1 + atr)
else:
tp1 = entry - floor_dist
tp2 = min(tp2, tp1 - atr)
rr = tp1_distance / sl_distance
risk_pips = sl_distance / pip_value
reward_pips = tp1_distance / pip_value
+265 -8
View File
@@ -84,6 +84,23 @@ or news_engine) so this module stays at the bottom of the import graph.
time-boxed; batch driver: scripts/label_scan_outcomes.py). Readers
must dedup by the FIRST row per (timeframe, bar_ts) — several surfaces
log the same bar.
2026-09-30 daily-trend veto:
* Row column ``daily_trend`` (+1 / 0 / -1 from closed D1 bars: computed
by the core where the veto is on, stamped by the live wrapper for
display where it is off; NULL on blocked / failed scans and when the
daily frame could not be read) and the gate prefix ``DAILY TREND
FILTER`` in ``gate_reason``.
* Row column ``alerted_at`` (UTC time of the Telegram alert for this row;
``mark_alerted``) and the read-only helper ``open_alerted_setups`` —
alerted plans whose advisory outcome is still PENDING — behind the
"already open" line on the alert (exposure.py). Since 2026-10-01 the
stamp is written just BEFORE the send and cleared again
(``unmark_alerted``) when no chat received the alert, so two alerts
sent at the same candle close count each other; ``last_alerted_bars``
reads the newest alerted bar per timeframe so the bot's entry cooldown
survives a restart.
"""
import hashlib
@@ -100,6 +117,7 @@ from typing import Any, Dict, Iterable, List, Optional, Tuple
import app_config
from advisory_outcome import (
MAX_HOLD_BARS,
advance_outcome,
canonical_r,
initial_outcome,
@@ -152,7 +170,12 @@ CREATE TABLE IF NOT EXISTS signals (
panel_payload_hash TEXT,
advisory_outcome_status TEXT,
advisory_r_multiple REAL,
advisory_outcome_json TEXT
advisory_outcome_json TEXT,
panel_prompt_version TEXT,
panel_probability REAL,
panel_probs_json TEXT,
panel_payloads_json TEXT,
panel_context_json TEXT
);
CREATE INDEX IF NOT EXISTS idx_signals_sym_tf_ts
ON signals (symbol, timeframe, logged_at);
@@ -225,6 +248,24 @@ _ADDITIVE_SIGNAL_COLS = (
("watch_level", "TEXT"),
("watch_kinds", "TEXT"),
("watch_line", "TEXT"),
# 2026-09-17 agent-panel v2 (see agent_panel.py): prompt era, the mean
# persona probability, per-persona probabilities + reasons, the exact
# per-persona payloads and the shared context — written post-hoc by
# update_panel so scripts/panel_replay.py can re-score any prompt
# version on the same evidence.
("panel_prompt_version", "TEXT"),
("panel_probability", "REAL"),
("panel_probs_json", "TEXT"),
("panel_payloads_json", "TEXT"),
("panel_context_json", "TEXT"),
# 2026-09-30 daily-trend veto (see module docstring).
("daily_trend", "INTEGER"),
# 2026-09-30: UTC time of the Telegram alert for THIS row (mark_alerted:
# written by the bot just before the send, cleared by unmark_alerted
# when no chat received it). NULL on every row that was scanned but not
# alerted — the "which plans did the reader actually receive" key the
# open-exposure line reads (open_alerted_setups).
("alerted_at", "TEXT"),
)
# Read-side compatibility view `signals_with_news`: legacy rows keep
@@ -247,6 +288,11 @@ _GATE_PREFIXES = (
"Primary timeframe stale",
"Primary timeframe data unavailable",
"DATA UNAVAILABLE:",
# The daily-trend veto removes a setup that every earlier gate had let
# through, so on such a row it IS the reason: listed ahead of the chain
# because the vetoed setup's stale warnings (a cleaned SPREAD GATE, the
# chart-pattern path's cleaned vetoes) are still in the pool.
"DAILY TREND FILTER",
"REGIME FILTER",
"EXHAUSTION BLOCK",
"EXHAUSTED TREND FILTER",
@@ -298,6 +344,7 @@ _SIGNAL_INSERT_COLS = (
"config_fingerprint",
"bar_ts", "engine_fingerprint", "code_version", "penalty_tags_json",
"penalty_total", "momentum_fading", "exhaustion_count",
"daily_trend",
)
_INSERT = (
"INSERT INTO signals (" + ", ".join(_SIGNAL_INSERT_COLS) + ") VALUES ("
@@ -315,6 +362,12 @@ def _finite(x: Any) -> Optional[float]:
return f if math.isfinite(f) else None
def _daily_trend_value(x: Any) -> Optional[int]:
"""The report's daily trend as -1 / 0 / +1, or None when it carries none."""
f = _finite(x)
return None if f is None else int(f)
def _iso(ts: Any) -> Optional[str]:
if ts is None:
return None
@@ -800,6 +853,7 @@ def _report_row(
"momentum_fading": int(bool(getattr(report, "momentum_fading", False))),
"exhaustion_count": int(
_finite(getattr(report, "exhaustion_count", 0)) or 0),
"daily_trend": _daily_trend_value(getattr(report, "daily_trend", None)),
}
@@ -888,6 +942,11 @@ class SignalLogger:
self._conn.execute(
"CREATE INDEX IF NOT EXISTS idx_signals_tf_bar"
" ON signals (timeframe, bar_ts)")
# Partial index: only alerted rows (a few per day), so the
# open-exposure read on every scan never walks the table.
self._conn.execute(
"CREATE INDEX IF NOT EXISTS idx_signals_alerted"
" ON signals (symbol, alerted_at) WHERE alerted_at IS NOT NULL")
self._conn.commit()
def log(
@@ -1025,6 +1084,11 @@ class SignalLogger:
cache_status: Optional[str] = None,
payload_hash: Optional[str] = None,
verdict_source: Optional[str] = None,
prompt_version: Optional[str] = None,
probability: Optional[float] = None,
probs_json: Optional[str] = None,
payloads_json: Optional[str] = None,
context_json: Optional[str] = None,
) -> None:
"""Fill panel provenance; seeds an outcome only for rows that
log() did not already seed (rows written by an older process)."""
@@ -1051,11 +1115,18 @@ class SignalLogger:
" panel_cache_status=COALESCE(?, panel_cache_status),"
" panel_payload_hash=COALESCE(?, panel_payload_hash),"
" panel_verdict_source=COALESCE(?, panel_verdict_source),"
" panel_prompt_version=COALESCE(?, panel_prompt_version),"
" panel_probability=COALESCE(?, panel_probability),"
" panel_probs_json=COALESCE(?, panel_probs_json),"
" panel_payloads_json=COALESCE(?, panel_payloads_json),"
" panel_context_json=COALESCE(?, panel_context_json),"
" advisory_outcome_status=COALESCE(advisory_outcome_status, ?),"
" advisory_outcome_json=COALESCE(advisory_outcome_json, ?)"
" WHERE id=?",
(verdict, confidence, rationale, takes_json,
created_at, cache_status, payload_hash, verdict_source,
prompt_version, probability, probs_json, payloads_json,
context_json,
advisory_status, advisory_json, rowid))
self._conn.commit()
@@ -1079,6 +1150,31 @@ class SignalLogger:
(str(level or ""), kinds_text, str(line or ""), int(rowid)))
self._conn.commit()
def mark_alerted(self, rowid: int,
when_iso: Optional[str] = None) -> Optional[str]:
"""Stamp the time of the alert for this row. The first stamp wins:
a re-alert of the same row keeps the original time. Returns the
stamp when THIS call wrote it and None when the row already had
one (or does not exist), so a caller that must undo its own stamp
can never clear an earlier alert's."""
when_iso = when_iso or datetime.now(timezone.utc).isoformat()
with self._lock:
cur = self._conn.execute(
"UPDATE signals SET alerted_at=?"
" WHERE id=? AND alerted_at IS NULL", (when_iso, int(rowid)))
self._conn.commit()
return when_iso if cur.rowcount == 1 else None
def unmark_alerted(self, rowid: int, when_iso: str) -> bool:
"""Clear the stamp mark_alerted returned (the alert reached no
chat). Only that exact stamp is cleared; True when it was."""
with self._lock:
cur = self._conn.execute(
"UPDATE signals SET alerted_at=NULL"
" WHERE id=? AND alerted_at=?", (int(rowid), str(when_iso)))
self._conn.commit()
return cur.rowcount == 1
def resolve_pending(
self, symbol: str, candles_json: Optional[str], limit: int = 100
) -> int:
@@ -1461,11 +1557,12 @@ class SignalLogger:
old = json.loads(state_json) if state_json else {}
if not _is_gap_reason(old.get("reason")):
continue
# The locked trailing stop lives ONLY in this progress
# block, and a re-seed rebuilds the walk from scratch over
# a stream whose hole is still there (the continuity check
# is all-or-nothing), so the floor would be lost for good.
# Finish such a row here; re-seed only what has nothing.
# The locked trailing stop lives in this progress block. A
# re-seed would rebuild the walk from scratch only to stop
# at the same hole (advance_outcome checks continuity bar
# by bar and keeps the progress up to the hole) and be
# floored at the same stop, so finish such a row here;
# re-seed only what has nothing locked.
floored = resolve_locked_floor(json.loads(_dumps(old)))
if floored.get("status") == "RESOLVED":
self._conn.execute(
@@ -1592,15 +1689,30 @@ def log_panel_update(report: Any) -> None:
lg = _loggers.get(path)
if lg is None:
return
takes = getattr(report, "panel_takes", None) or []
probs = [{"persona": t.get("persona"), "p_win": _finite(t.get("p_win")),
"stance": t.get("stance"),
"confidence": _finite(t.get("confidence")),
"reason_for": t.get("reason_for") or "",
"reason_against": t.get("reason_against") or ""}
for t in takes if isinstance(t, dict)]
payloads = getattr(report, "panel_payloads", None) or {}
context = getattr(report, "panel_context", None) or {}
probability = _finite(getattr(report, "panel_probability", None))
lg.update_panel(
int(rowid), verdict,
_finite(getattr(report, "panel_confidence", None)),
getattr(report, "panel_rationale", "") or "",
_dumps(getattr(report, "panel_takes", None) or []),
_dumps(takes),
created_at=getattr(report, "panel_created_at", "") or None,
cache_status=getattr(report, "panel_cache_status", "") or None,
payload_hash=getattr(report, "panel_payload_hash", "") or None,
verdict_source=getattr(report, "panel_verdict_source", "") or None)
verdict_source=getattr(report, "panel_verdict_source", "") or None,
prompt_version=getattr(report, "panel_prompt_version", "") or None,
probability=probability if probability else None,
probs_json=_dumps(probs) if probs else None,
payloads_json=_dumps(payloads) if payloads else None,
context_json=_dumps(context) if context else None)
except Exception:
logger.warning("signal_log: failed to persist panel verdict",
exc_info=True)
@@ -1631,6 +1743,151 @@ def log_watch_update(report: Any, watch: Any) -> None:
exc_info=True)
def _logged_row(report: Any) -> Optional[Tuple["SignalLogger", int]]:
"""(logger, rowid) of the row log_signal wrote for this scan; None for
a report that was never logged or a process with logging off."""
rowid = getattr(report, "_signal_log_rowid", None)
if rowid is None or not app_config.signal_log_enabled():
return None
path = app_config.signal_log_path()
with _loggers_lock:
lg = _loggers.get(path)
return None if lg is None else (lg, int(rowid))
def mark_alerted(report: Any) -> Optional[str]:
"""Stamp `alerted_at` on the row logged for this scan. The bot calls
this just BEFORE it sends the alert, so an alert sent a moment later
on another timeframe already counts this plan, and undoes it with
unmark_alerted when no chat received the message. Same hand-off as
log_panel_update (the private rowid from log_signal). Returns the
stamp this call wrote; None when nothing was written (never logged,
logging off, or the row already carried a stamp). NEVER raises."""
try:
hit = _logged_row(report)
return None if hit is None else hit[0].mark_alerted(hit[1])
except Exception:
logger.warning("signal_log: failed to mark the alerted row",
exc_info=True)
return None
def unmark_alerted(report: Any, stamp: Optional[str]) -> bool:
"""Clear the stamp mark_alerted(report) returned: the alert reached no
chat, so the row must not count as a plan the reader holds. A no-op
without a stamp; only that exact stamp is cleared. NEVER raises."""
if not stamp:
return False
try:
hit = _logged_row(report)
return False if hit is None else hit[0].unmark_alerted(hit[1], stamp)
except Exception:
logger.warning("signal_log: failed to clear the alerted stamp",
exc_info=True)
return False
# A PENDING outcome older than this is not shown as "open": the advisory
# model times a trade out after MAX_HOLD_BARS bars, so an older PENDING row
# is one the resolver never reached (bot outage, candle gap), not a trade
# still running. Calendar allowance: 7/5 for weekends plus three days.
def _open_setup_max_age_seconds(timeframe: Any) -> float:
bar = _TF_SECONDS.get(normalize_timeframe(timeframe), 3600)
return MAX_HOLD_BARS * bar * 1.4 + 3 * 86400
def open_alerted_setups(symbol: str, exclude_key: Optional[str] = None,
now: Optional[datetime] = None,
path: Optional[str] = None) -> List[Dict[str, Any]]:
"""ALERTED trade plans whose advisory outcome is still open, oldest
first: ``[{id, event_id, timeframe, direction, entry, stop_loss,
alerted_at, setup_key}]``, one entry per advisory event.
Read-only (a ``mode=ro`` connection of its own, so the dashboard
process, which never writes this database, can call it). "Open" means
the row's advisory event is still PENDING in the cleaned-mid outcome
model: next-open entry, stop first, close-only trailing. That is the
model's view of a plan the reader MAY have taken, not a broker
position. ``exclude_key`` drops the plan being alerted now from its
own count. NEVER raises: any failure (no database yet, an older schema
without ``alerted_at``) returns an empty list."""
try:
db_path = path or app_config.signal_log_path()
if not os.path.exists(db_path):
return []
now = now or datetime.now(timezone.utc)
con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=2.0)
try:
rows = con.execute(
"SELECT a.id, COALESCE(a.advisory_event_id, a.id), a.timeframe,"
" a.setup_json, a.alerted_at, a.setup_key"
" FROM signals a JOIN signals e"
" ON e.id = COALESCE(a.advisory_event_id, a.id)"
" WHERE a.symbol=? AND a.alerted_at IS NOT NULL"
" AND e.advisory_outcome_status='PENDING'"
" ORDER BY a.id", (symbol,)).fetchall()
finally:
con.close()
out: List[Dict[str, Any]] = []
seen = set()
for rowid, event_id, timeframe, setup_json, alerted_at, key in rows:
if event_id in seen or (exclude_key and key == exclude_key):
continue
try:
age = (now - datetime.fromisoformat(alerted_at)).total_seconds()
except (TypeError, ValueError):
continue
if age > _open_setup_max_age_seconds(timeframe):
continue
setup = json.loads(setup_json) if setup_json else {}
direction = setup.get("direction")
if direction not in ("BUY", "SELL"):
continue
seen.add(event_id)
out.append({
"id": int(rowid), "event_id": int(event_id),
"timeframe": normalize_timeframe(timeframe),
"direction": direction,
"entry": _finite(setup.get("entry")),
"stop_loss": _finite(setup.get("stop_loss")),
"alerted_at": alerted_at, "setup_key": key,
})
return out
except Exception:
logger.debug("signal_log: open alerted setups unavailable",
exc_info=True)
return []
def last_alerted_bars(path: Optional[str] = None
) -> Dict[Tuple[str, str], str]:
"""``{(symbol, timeframe): bar label}`` of the newest ALERTED row per
symbol and timeframe (newest = highest id). The bot seeds its 5-bar
entry cooldown from this at start-up, so a restart does not forget the
last alert. Read-only (``mode=ro``); NEVER raises: no database, an
older schema or a locked file returns ``{}``."""
try:
db_path = path or app_config.signal_log_path()
if not os.path.exists(db_path):
return {}
con = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True, timeout=2.0)
try:
rows = con.execute(
"SELECT s.symbol, s.timeframe, s.bar_ts FROM signals s"
" JOIN (SELECT MAX(id) AS id FROM signals"
" WHERE alerted_at IS NOT NULL"
" GROUP BY symbol, timeframe) m ON m.id = s.id"
).fetchall()
finally:
con.close()
return {(str(symbol), normalize_timeframe(timeframe)): str(bar)
for symbol, timeframe, bar in rows if bar}
except Exception:
logger.debug("signal_log: last alerted bars unavailable",
exc_info=True)
return {}
def reset_loggers_for_tests() -> None:
"""Close and drop all cached loggers (test isolation)."""
with _loggers_lock:
+611 -249
View File
@@ -1,351 +1,713 @@
/* Gold Desk — XAU/USD dashboard styles
*
* Builds on Bootstrap 5.3 color modes (data-bs-theme). Only app-specific
* tokens and components live here; layout, typography, tables, badges,
* cards, alerts, modals, etc. come from Bootstrap.
* Bootstrap 5.3 supplies layout, cards, tables, badges, alerts, modals and
* the form switch; this file supplies the desk: the token set, the sticky
* ticker bar, the direction board, the four timeframe instruments and the
* chart surfaces Chart.js draws onto.
*
* Colour notes
* Dark only (owner decision 2026-08-16): dark ground, white text, red and
* green reserved for direction.
* --up / --down are the CHART mark colours: a diverging pair validated on
* this surface (lightness band, chroma floor, colour-vision separation,
* contrast). --up-text / --down-text are the lighter steps of the same two
* hues, used wherever the colour has to work as text. Direction is never
* colour alone — an arrow icon and the word BUY or SELL ride with it.
*/
:root {
--xau-font-sans: "Inter", system-ui, -apple-system, "Segoe UI", sans-serif;
--xau-font-mono: "JetBrains Mono", ui-monospace, "SF Mono", Consolas, Menlo, monospace;
--xau-gold: #f5b301;
--sans: "IBM Plex Sans", system-ui, -apple-system, "Segoe UI", sans-serif;
--mono: "IBM Plex Mono", ui-monospace, "SF Mono", Consolas, Menlo, monospace;
--desk-bg: #0c0f13;
--panel: #12161c;
--panel-2: #171d25;
--rule: rgba(255, 255, 255, 0.10);
--rule-soft: rgba(255, 255, 255, 0.06);
--gold: #f5b301;
--gold-wash: rgba(245, 179, 1, 0.12);
--up: #0d9f7a;
--down: #e8503a;
--flat: #6b7a8d;
--up-text: #2fd6a3;
--down-text: #ff7d66;
--up-wash: rgba(13, 159, 122, 0.16);
--down-wash: rgba(232, 80, 58, 0.16);
--grid: rgba(255, 255, 255, 0.07);
--zero: rgba(255, 255, 255, 0.30);
--axis: rgba(255, 255, 255, 0.64);
}
/* Forced dark theme (owner decision 2026-08-16): dark background, pure
* white text (no gray), vivid red/green for data accents. Loaded after
* Bootstrap so these overrides win. */
[data-bs-theme="dark"] {
--bs-body-bg: #0a0e14;
--bs-secondary-bg: #111823;
--bs-tertiary-bg: #0e141d;
--bs-body-bg: var(--desk-bg);
--bs-body-color: #ffffff;
--bs-emphasis-color: #ffffff;
--bs-heading-color: #ffffff;
--bs-secondary-color: rgba(255, 255, 255, 0.92);
--bs-tertiary-color: rgba(255, 255, 255, 0.8);
--bs-border-color: rgba(255, 255, 255, 0.14);
--bs-card-bg: #0e141d;
--bs-card-border-color: rgba(255, 255, 255, 0.12);
--bs-modal-bg: #0e141d;
--bs-success: #22c55e;
--bs-success-rgb: 34, 197, 94;
--bs-success-text-emphasis: #4ade80;
--bs-success-bg-subtle: rgba(34, 197, 94, 0.16);
--bs-success-border-subtle: rgba(34, 197, 94, 0.45);
--bs-danger: #ef4444;
--bs-danger-rgb: 239, 68, 68;
--bs-danger-text-emphasis: #f87171;
--bs-danger-bg-subtle: rgba(239, 68, 68, 0.16);
--bs-danger-border-subtle: rgba(239, 68, 68, 0.45);
--bs-tertiary-color: rgba(255, 255, 255, 0.78);
--bs-secondary-bg: var(--panel-2);
--bs-tertiary-bg: var(--panel-2);
--bs-border-color: var(--rule);
--bs-card-bg: var(--panel);
--bs-card-border-color: var(--rule);
--bs-modal-bg: var(--panel);
--bs-border-radius: 0.5rem;
--bs-primary: var(--gold);
--bs-primary-rgb: 245, 179, 1;
--bs-success: #0d9f7a;
--bs-success-rgb: 13, 159, 122;
--bs-success-text-emphasis: var(--up-text);
--bs-success-bg-subtle: var(--up-wash);
--bs-success-border-subtle: rgba(13, 159, 122, 0.45);
--bs-danger: #e8503a;
--bs-danger-rgb: 232, 80, 58;
--bs-danger-text-emphasis: var(--down-text);
--bs-danger-bg-subtle: var(--down-wash);
--bs-danger-border-subtle: rgba(232, 80, 58, 0.45);
--bs-warning: #e9a23b;
--bs-warning-rgb: 233, 162, 59;
--bs-link-color: var(--gold);
--bs-link-hover-color: #ffca3f;
}
html, body {
font-family: var(--xau-font-sans);
letter-spacing: -0.005em;
background: var(--desk-bg);
color: #fff;
font-family: var(--sans);
font-size: 0.9375rem;
-webkit-font-smoothing: antialiased;
background-color: var(--bs-body-bg);
color: var(--bs-body-color);
}
.num, .xau-price, .xau-symbol {
font-family: var(--xau-font-mono);
h1, h2, h3, h4, h5 { font-weight: 600; }
.num {
font-family: var(--mono);
font-variant-numeric: tabular-nums;
font-feature-settings: "zero" 1;
}
.xau-symbol {
font-size: 0.95rem;
font-weight: 700;
letter-spacing: 0.08em;
color: var(--xau-gold);
.up { color: var(--up-text); }
.down { color: var(--down-text); }
.flat { color: rgba(255, 255, 255, 0.72); }
.sig-buy { --edge: var(--up); --edge-text: var(--up-text); --edge-wash: var(--up-wash); }
.sig-sell { --edge: var(--down); --edge-text: var(--down-text); --edge-wash: var(--down-wash); }
.sig-neutral { --edge: var(--flat); --edge-text: rgba(255, 255, 255, 0.8); --edge-wash: rgba(255, 255, 255, 0.05); }
.desk-shell {
width: 100%;
max-width: 1560px;
margin: 0 auto;
padding: 0 1rem;
}
.xau-price {
/* ── ticker bar ────────────────────────────────────────────────── */
.desk-bar {
background: rgba(12, 15, 19, 0.94);
backdrop-filter: blur(8px);
border-bottom: 1px solid var(--rule);
padding: 0.55rem 0;
}
.desk-mark { color: var(--gold); font-size: 1rem; }
.desk-symbol {
font-family: var(--mono);
font-weight: 600;
letter-spacing: 0.06em;
color: var(--gold);
font-size: 0.9rem;
}
.desk-price {
font-size: 1.6rem;
font-weight: 600;
line-height: 1;
color: #ffffff;
}
.desk-rail { display: flex; gap: 0.4rem; flex-wrap: wrap; }
.rail-chip {
display: inline-flex;
align-items: baseline;
gap: 0.35rem;
border: 1px solid var(--rule);
border-left: 2px solid var(--edge, var(--flat));
border-radius: 0.3rem;
background: var(--panel);
color: #fff;
font-family: var(--mono);
font-size: 0.78rem;
padding: 0.2rem 0.45rem;
}
.rail-chip:hover { background: var(--panel-2); }
.rail-chip .rail-tf { font-weight: 600; }
.rail-chip .rail-score { color: var(--edge-text); }
.desk-status {
display: flex;
align-items: center;
gap: 1rem;
font-size: 0.8rem;
color: var(--bs-secondary-color);
}
.desk-meta { white-space: nowrap; }
.desk-meta i { opacity: 0.55; margin-right: 0.25rem; }
.desk-meta.is-stale { color: var(--bs-warning); font-weight: 600; }
.live-dot {
display: inline-block;
width: 8px;
height: 8px;
width: 7px;
height: 7px;
border-radius: 50%;
background: var(--bs-success);
margin-right: 5px;
background: var(--up-text);
margin-right: 0.4rem;
animation: pulse 2.4s ease-in-out infinite;
}
.live-dot.closed {
background: var(--bs-warning);
animation: none;
.live-dot.closed { background: var(--bs-warning); animation: none; }
@keyframes pulse { 50% { opacity: 0.3; } }
/* ── direction board ───────────────────────────────────────────── */
.board {
background: var(--panel);
border: 1px solid var(--rule);
border-radius: 0.6rem;
overflow: hidden;
}
@keyframes pulse {
50% { opacity: 0.35; }
.board-read {
display: flex;
flex-direction: column;
padding: 1.5rem 1.6rem;
border-right: 1px solid var(--rule);
}
/* signal-coloured accents */
.sig-buy { --xau-signal: var(--bs-success); }
.sig-sell { --xau-signal: var(--bs-danger); }
.sig-neutral { --xau-signal: var(--bs-secondary); }
.border-signal {
border-left: 3px solid var(--xau-signal, var(--bs-secondary)) !important;
.board-headline {
font-size: clamp(1.5rem, 2.4vw, 2rem);
font-weight: 600;
line-height: 1.15;
margin-bottom: 0.5rem;
max-width: 22ch;
}
.text-signal {
color: var(--xau-signal, var(--bs-secondary)) !important;
.board-sub {
color: var(--bs-secondary-color);
max-width: 46ch;
margin-bottom: 1.1rem;
}
.board-stats {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 0.75rem;
margin-top: auto;
border-top: 1px solid var(--rule);
padding-top: 0.9rem;
}
.board-stats .k {
display: block;
font-size: 0.72rem;
color: rgba(255, 255, 255, 0.7);
margin-bottom: 0.15rem;
}
.board-stats .v { font-size: 1.05rem; font-weight: 600; }
.board-stats .v small { font-weight: 400; font-size: 0.78rem; color: rgba(255, 255, 255, 0.7); }
.board-plot { padding: 1.5rem 1.6rem; }
.plot-title { font-size: 0.95rem; margin-bottom: 0.75rem; }
.plot-note {
font-size: 0.78rem;
color: rgba(255, 255, 255, 0.72);
margin: 0.6rem 0 0;
max-width: 62ch;
}
.chart-wrap { position: relative; }
.chart-direction { height: 190px; }
.chart-ind { height: 158px; margin-bottom: 0.9rem; }
.chart-tone { height: 58px; }
/* ── news voice inside the board ───────────────────────────────── */
.board-news {
border-left: 2px solid var(--gold);
padding-left: 0.85rem;
margin-bottom: 1.2rem;
}
.context-text {
margin: 0 0 0.5rem;
color: var(--bs-secondary-color);
font-size: 0.85rem;
max-width: 60ch;
}
.context-chips { display: flex; flex-wrap: wrap; gap: 0.4rem; }
.chip {
display: inline-flex;
align-items: center;
gap: 0.3rem;
font-size: 0.75rem;
padding: 0.2rem 0.5rem;
border-radius: 0.3rem;
border: 1px solid var(--rule);
background: var(--panel-2);
color: var(--bs-secondary-color);
white-space: nowrap;
}
.chip i { opacity: 0.7; }
.chip .num { font-weight: 600; }
.chip-up { border-color: rgba(13, 159, 122, 0.45); background: var(--up-wash); color: var(--up-text); }
.chip-down { border-color: rgba(232, 80, 58, 0.45); background: var(--down-wash); color: var(--down-text); }
.chip-warn { border-color: rgba(233, 162, 59, 0.5); background: rgba(233, 162, 59, 0.14); color: #ffc46b; }
.chip-gold { border-color: rgba(245, 179, 1, 0.5); background: var(--gold-wash); color: var(--gold); }
/* ── section headings ──────────────────────────────────────────── */
.sec-head {
display: flex;
align-items: baseline;
flex-wrap: wrap;
gap: 0.6rem;
margin-bottom: 0.85rem;
}
.sec-title { font-size: 1.05rem; margin: 0; }
.sec-note { font-size: 0.8rem; color: rgba(255, 255, 255, 0.72); }
/* ── timeframe instruments ─────────────────────────────────────── */
.tf-card {
height: 100%;
overflow: hidden;
background: var(--panel);
scroll-margin-top: 5.5rem;
}
.tf-card.is-target { outline: 2px solid var(--gold); outline-offset: 1px; }
.tf-card .card-body { padding: 0.95rem 1.05rem 0.6rem; }
.tf-edge { height: 3px; background: var(--edge, var(--flat)); }
.tf-head { display: flex; align-items: baseline; gap: 0.45rem; margin-bottom: 0.55rem; }
.tf-name {
font-family: var(--xau-font-mono);
font-weight: 700;
font-family: var(--mono);
font-size: 1.1rem;
color: var(--bs-primary);
letter-spacing: 0.05em;
font-weight: 600;
letter-spacing: 0.04em;
margin: 0;
color: var(--gold);
}
.tf-selector {
display: grid;
grid-template-columns: repeat(4, minmax(0, 1fr));
gap: 0.5rem;
margin-bottom: 0.75rem;
}
.tf-long { font-size: 0.75rem; color: rgba(255, 255, 255, 0.7); }
.tf-select {
min-width: 0;
padding: 0.7rem 0.8rem;
border: 1px solid var(--bs-border-color);
border-radius: var(--bs-border-radius);
background: var(--bs-tertiary-bg);
color: var(--bs-body-color);
text-align: left;
transition: border-color 0.15s ease, background-color 0.15s ease, box-shadow 0.15s ease;
}
.tf-clock { font-size: 0.75rem; color: rgba(255, 255, 255, 0.78); white-space: nowrap; }
.tf-clock i { opacity: 0.55; margin-right: 0.2rem; }
.tf-select:hover {
border-color: rgba(var(--bs-primary-rgb), 0.65);
}
.tf-select[aria-pressed="true"] {
border-color: var(--xau-gold);
background: rgba(245, 179, 1, 0.09);
box-shadow: inset 0 0 0 1px rgba(245, 179, 1, 0.16);
}
.tf-select .tf-name {
font-size: 1rem;
}
.workspace-card {
overflow: hidden;
}
.workspace-signal {
font-family: var(--xau-font-mono);
font-size: clamp(1.55rem, 3vw, 2.35rem);
font-weight: 700;
letter-spacing: 0.03em;
line-height: 1;
}
.workspace-meta {
min-width: 0;
}
.workspace-reasons > div + div {
margin-top: 0.35rem;
}
/* score bar: zero-centered diverging with midpoint tick */
.scorebar {
.tf-signal {
display: flex;
align-items: center;
gap: 0.75rem;
gap: 0.5rem;
font-size: 1.35rem;
font-weight: 600;
line-height: 1.1;
color: var(--edge-text, #fff);
}
.scorebar .gauge-container {
.tf-signal .fa-solid { font-size: 1.05rem; }
.grade {
font-family: var(--mono);
font-size: 0.72rem;
font-weight: 600;
padding: 0.1rem 0.4rem;
border-radius: 0.25rem;
border: 1px solid var(--rule);
color: rgba(255, 255, 255, 0.85);
align-self: center;
}
.grade.grade-high { border-color: rgba(245, 179, 1, 0.55); background: var(--gold-wash); color: var(--gold); }
.tf-plain {
font-size: 0.88rem;
margin: 0.35rem 0 0;
}
.tf-action {
font-size: 0.82rem;
color: var(--bs-secondary-color);
margin: 0.35rem 0 0.8rem;
min-height: 1.1rem;
}
.tf-readouts {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 0.75rem;
margin-bottom: 0.8rem;
}
.readout-k { display: block; font-size: 0.72rem; color: rgba(255, 255, 255, 0.7); }
.readout-v { font-size: 1.15rem; font-weight: 600; }
.meter {
position: relative;
flex: 1;
min-width: 60px;
height: 10px;
border-radius: 5px;
background: var(--bs-secondary-bg);
display: block;
height: 5px;
margin-top: 0.35rem;
border-radius: 3px;
background: rgba(255, 255, 255, 0.08);
overflow: hidden;
}
.scorebar .gauge-container::after {
.meter::after {
content: "";
position: absolute;
left: 50%;
top: 0;
bottom: 0;
width: 2px;
background: var(--bs-secondary);
transform: translateX(-1px);
width: 1px;
background: var(--zero);
}
.scorebar .fill {
.meter i {
position: absolute;
top: 0;
bottom: 0;
border-radius: 4px;
width: var(--score-width);
border-radius: 3px;
transition: width 0.25s ease;
}
.scorebar .fill.pos {
left: 50%;
background: var(--bs-success);
}
.meter i.pos { left: 50%; background: var(--up); }
.meter i.neg { right: 50%; background: var(--down); }
.scorebar .fill.neg {
right: 50%;
background: var(--bs-danger);
}
.meter-linear::after { display: none; }
.meter-linear i { left: 0; background: rgba(255, 255, 255, 0.55); }
.scorebar .val {
width: 58px;
text-align: right;
font-size: 0.95rem;
font-weight: 500;
}
.tf-chips { display: flex; flex-wrap: wrap; gap: 0.3rem; margin-bottom: 0.9rem; }
.tf-chips .chip { font-size: 0.72rem; padding: 0.15rem 0.4rem; }
.scorebar .lbl {
font-size: 0.8rem;
color: var(--bs-secondary);
width: 44px;
}
/* rail chips in hero */
.rail-chip {
border: 1px solid var(--bs-border-color);
border-radius: var(--bs-border-radius);
background: var(--bs-tertiary-bg);
color: var(--bs-body-color);
font-family: var(--xau-font-mono);
font-size: 0.9rem;
line-height: 1.3;
padding: 0.3rem 0.5rem;
}
.rail-chip:hover, .rail-chip[aria-current="true"] {
border-color: var(--xau-gold);
}
.rail-chip:focus-visible, .tf-select:focus-visible, .wire-item:focus-visible,
.cal-event:focus-visible {
outline: 2px solid var(--xau-gold);
outline-offset: 2px;
}
/* detail dialog sections */
.d-stat .k {
display: block;
font-size: 0.7rem;
letter-spacing: 0.08em;
text-transform: uppercase;
color: var(--bs-secondary);
.tf-sub {
font-size: 0.78rem;
font-weight: 600;
color: rgba(255, 255, 255, 0.82);
margin-bottom: 0.4rem;
}
.d-stat .v {
font-size: 1.15rem;
font-weight: 600;
/* trade plan */
.plan {
border: 1px solid var(--rule);
border-left: 2px solid var(--edge, var(--flat));
border-radius: 0.4rem;
background: var(--panel-2);
padding: 0.6rem 0.7rem;
}
.d-table {
font-size: 0.9rem;
}
.detail-table {
overflow-x: auto;
}
.d-table .ind-k {
font-family: var(--xau-font-mono);
font-weight: 600;
white-space: nowrap;
}
.wbar {
display: block;
height: 4px;
border-radius: 2px;
background: var(--bs-secondary-bg);
margin-top: 3px;
}
.wbar > i {
display: block;
height: 100%;
border-radius: 2px;
background: var(--bs-primary);
}
/* news */
.wire-item {
.plan-head {
display: flex;
width: 100%;
border: 0;
border-radius: var(--bs-border-radius);
background: transparent;
color: inherit;
cursor: pointer;
font: inherit;
text-align: start;
align-items: baseline;
gap: 0.4rem;
font-size: 0.82rem;
font-weight: 600;
color: var(--edge-text);
margin-bottom: 0.5rem;
}
.wire-item:hover {
background: rgba(var(--bs-primary-rgb), 0.08);
.plan-strategy {
font-family: var(--mono);
font-size: 0.7rem;
font-weight: 400;
color: rgba(255, 255, 255, 0.7);
}
.plan-rr { margin-left: auto; font-family: var(--mono); font-size: 0.75rem; color: rgba(255, 255, 255, 0.85); }
/* Two columns in a card, four across the wider dialog — never three,
which would strand Target 2 on a row of its own. */
.plan-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 0.35rem 1.25rem;
margin: 0;
}
@media (min-width: 768px) {
.d-sec .plan-grid { grid-template-columns: repeat(4, minmax(0, 1fr)); }
}
.plan-grid > div { display: flex; align-items: baseline; justify-content: space-between; gap: 0.5rem; }
.plan-grid dt { font-size: 0.75rem; font-weight: 400; color: rgba(255, 255, 255, 0.75); }
.plan-grid dt i { width: 0.9rem; opacity: 0.65; margin-right: 0.3rem; }
.plan-grid dd { margin: 0; font-family: var(--mono); font-size: 0.85rem; font-weight: 500; }
.plan-trail {
margin-top: 0.5rem;
padding-top: 0.45rem;
border-top: 1px solid var(--rule-soft);
font-size: 0.74rem;
color: rgba(255, 255, 255, 0.75);
}
.plan-empty {
border-left-color: var(--rule);
color: rgba(255, 255, 255, 0.75);
font-size: 0.8rem;
}
.tf-flags { margin-top: 0.7rem; display: grid; gap: 0.25rem; font-size: 0.76rem; }
.tf-flags i { width: 0.9rem; margin-right: 0.25rem; }
.tf-flags .flag-warn { color: #ffc46b; }
.tf-flags .flag-watch { color: var(--gold); font-weight: 500; }
.tf-card .card-footer {
display: flex;
align-items: center;
gap: 0.6rem;
background: transparent;
border-top: 1px solid var(--rule-soft);
padding: 0.5rem 1.05rem;
}
.tf-scanned { margin-left: auto; font-size: 0.72rem; color: rgba(255, 255, 255, 0.68); }
.btn-desk {
border: 1px solid var(--rule);
color: #fff;
background: var(--panel-2);
font-size: 0.78rem;
}
.btn-desk:hover { border-color: var(--gold); color: var(--gold); background: var(--panel-2); }
.btn-desk i { margin-right: 0.3rem; opacity: 0.8; }
.tf-empty {
padding: 1.6rem 0.4rem 2rem;
text-align: center;
color: rgba(255, 255, 255, 0.78);
font-size: 0.85rem;
}
.tf-empty i { display: block; font-size: 1.4rem; opacity: 0.5; margin-bottom: 0.5rem; }
/* ── calendar + wire ───────────────────────────────────────────── */
.card-head {
display: flex;
align-items: baseline;
flex-wrap: wrap;
gap: 0.6rem;
margin-bottom: 0.8rem;
}
.card-title { font-size: 0.95rem; margin: 0; }
.card-title i { color: var(--gold); margin-right: 0.45rem; }
.card-note { font-size: 0.75rem; color: rgba(255, 255, 255, 0.7); }
.form-check-label { font-size: 0.78rem; }
.form-switch .form-check-input:checked { background-color: var(--gold); border-color: var(--gold); }
.form-check-input:focus { box-shadow: none; border-color: var(--gold); }
/* The calendar fills whatever height the row settles on, then scrolls. */
.card-body:has(.cal-scroll) { display: flex; flex-direction: column; }
.cal-scroll { flex: 1 1 auto; max-height: 30rem; overflow-y: auto; }
.cal-table { font-size: 0.82rem; margin-bottom: 0; }
.cal-table thead th {
position: sticky;
top: 0;
z-index: 1;
background: var(--panel);
font-size: 0.72rem;
font-weight: 500;
color: rgba(255, 255, 255, 0.7);
border-bottom: 1px solid var(--rule);
}
.cal-table td { border-color: var(--rule-soft); }
.cal-table .empty { text-align: center; padding: 1.2rem 0; color: rgba(255, 255, 255, 0.7); }
.cal-day td { padding-top: 0.7rem; font-size: 0.75rem; font-weight: 600; color: var(--gold); }
.cal-past { color: rgba(255, 255, 255, 0.55); }
.cal-next td { background: rgba(245, 179, 1, 0.06); }
.cal-next td:first-child { box-shadow: inset 2px 0 0 var(--gold); }
.cal-event {
display: block;
width: 100%;
padding: 0;
border: 0;
background: transparent;
background: none;
color: inherit;
text-align: inherit;
text-align: left;
}
.cal-event:hover {
color: var(--bs-primary);
text-decoration: underline;
.cal-event:hover { color: var(--gold); text-decoration: underline; }
.impact {
display: inline-block;
font-size: 0.68rem;
padding: 0.1rem 0.35rem;
border-radius: 0.25rem;
border: 1px solid var(--rule);
}
.impact-high { border-color: rgba(232, 80, 58, 0.5); background: var(--down-wash); color: var(--down-text); }
.impact-medium { border-color: rgba(233, 162, 59, 0.5); background: rgba(233, 162, 59, 0.14); color: #ffc46b; }
.impact-low { color: rgba(255, 255, 255, 0.7); }
.tone { margin-bottom: 0.8rem; }
.tone-legend {
display: flex;
flex-wrap: wrap;
gap: 0.75rem;
font-size: 0.74rem;
color: rgba(255, 255, 255, 0.8);
}
.tone-legend span { display: inline-flex; align-items: center; gap: 0.3rem; }
.tone-key {
width: 9px;
height: 9px;
border-radius: 2px;
display: inline-block;
}
.wire-item {
display: flex;
gap: 0.5rem;
align-items: baseline;
width: 100%;
padding: 0.3rem 0.35rem;
border: 0;
border-radius: 0.3rem;
background: none;
color: inherit;
font: inherit;
text-align: left;
}
button.wire-item:hover { background: rgba(255, 255, 255, 0.05); }
.wire-score {
font-family: var(--xau-font-mono);
font-size: 0.85rem;
font-weight: 700;
padding: 2px 7px;
border-radius: 5px;
font-family: var(--mono);
font-size: 0.74rem;
font-weight: 600;
padding: 0.05rem 0.35rem;
border-radius: 0.25rem;
white-space: nowrap;
min-width: 2.7rem;
text-align: center;
}
/* calendar highlight for next upcoming event */
.cal-next-up td:first-child {
box-shadow: inset 4px 0 0 var(--bs-primary);
padding-left: 0.75rem !important;
.wire-score.pos { background: var(--up-wash); color: var(--up-text); }
.wire-score.neg { background: var(--down-wash); color: var(--down-text); }
.wire-score.zero { background: rgba(255, 255, 255, 0.06); color: rgba(255, 255, 255, 0.75); }
.wire-text { flex: 1 1 auto; font-size: 0.82rem; }
.wire-src { font-size: 0.72rem; color: rgba(255, 255, 255, 0.65); white-space: nowrap; }
.wire-more { font-size: 0.78rem; color: rgba(255, 255, 255, 0.78); cursor: pointer; padding: 0.35rem; }
/* ── modals ────────────────────────────────────────────────────── */
.modal-sub { font-size: 0.8rem; color: rgba(255, 255, 255, 0.78); }
.modal-sub .num { margin-right: 0.4rem; }
.d-sec { margin-bottom: 1.4rem; }
.d-sec > h6 {
display: flex;
align-items: center;
gap: 0.5rem;
font-size: 0.82rem;
font-weight: 600;
margin-bottom: 0.6rem;
}
.d-sec > h6 i { color: var(--gold); }
.d-rule { flex: 1 1 auto; height: 1px; background: var(--rule); }
.d-tiles { display: grid; grid-template-columns: repeat(auto-fit, minmax(7.5rem, 1fr)); gap: 0.5rem; }
.d-tile {
border: 1px solid var(--rule);
border-radius: 0.35rem;
background: var(--panel-2);
padding: 0.45rem 0.55rem;
}
.d-tile .k { display: block; font-size: 0.7rem; color: rgba(255, 255, 255, 0.7); }
.d-tile .v { font-size: 1.02rem; font-weight: 600; }
.d-table { font-size: 0.82rem; }
.d-table .ind-k { font-family: var(--mono); font-weight: 600; white-space: nowrap; }
.d-table td, .d-table th { border-color: var(--rule-soft); }
.detail-scroll { overflow-x: auto; }
.wbar { display: block; height: 3px; border-radius: 2px; background: rgba(255, 255, 255, 0.1); margin-top: 3px; }
.wbar > i { display: block; height: 100%; border-radius: 2px; background: var(--gold); }
.d-sec details {
border: 1px solid var(--rule);
border-radius: 0.4rem;
padding: 0.5rem 0.65rem;
margin-bottom: 0.5rem;
}
.d-sec summary { cursor: pointer; font-size: 0.85rem; }
.d-sec summary > * { margin-right: 0.4rem; }
.reasons { display: grid; gap: 0.25rem; font-size: 0.82rem; }
.reasons i { width: 1rem; margin-right: 0.3rem; }
.reason-group { font-size: 0.76rem; color: rgba(255, 255, 255, 0.72); margin: 0.6rem 0 0.25rem; }
.desk-foot {
margin-top: 2rem;
padding-top: 1rem;
border-top: 1px solid var(--rule);
font-size: 0.78rem;
color: rgba(255, 255, 255, 0.7);
}
.desk-foot i { color: rgba(255, 255, 255, 0.5); margin-right: 0.35rem; }
/* ── focus, motion, small screens ──────────────────────────────── */
.rail-chip:focus-visible,
.wire-item:focus-visible,
.cal-event:focus-visible,
.btn-desk:focus-visible,
.form-check-input:focus-visible,
summary:focus-visible {
outline: 2px solid var(--gold);
outline-offset: 2px;
border-radius: 0.25rem;
}
/* reduced motion */
@media (prefers-reduced-motion: reduce) {
.live-dot { animation: none; }
.scorebar .fill { transition: none; }
* { transition: none !important; }
*, *::before, *::after { transition: none !important; }
}
@media (max-width: 991.98px) {
.board-read { border-right: 0; border-bottom: 1px solid var(--rule); }
.board-read, .board-plot { padding: 1.15rem 1.15rem; }
}
@media (max-width: 575.98px) {
.tf-selector {
grid-template-columns: repeat(2, minmax(0, 1fr));
}
.workspace-card .card-body {
padding: 1rem;
}
.rail-chip {
flex: 1 1 calc(50% - 0.5rem);
}
.desk-price { font-size: 1.35rem; }
.desk-status { gap: 0.6rem; font-size: 0.75rem; }
.board-stats { grid-template-columns: 1fr; gap: 0.5rem; }
.desk-shell { padding: 0 0.75rem; }
}
+116 -90
View File
@@ -8,142 +8,168 @@
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.3/font/bootstrap-icons.css" crossorigin="anonymous">
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;700&display=swap">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.7.2/css/all.min.css" crossorigin="anonymous" referrerpolicy="no-referrer">
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;500;600&family=IBM+Plex+Sans:wght@400;500;600;700&display=swap">
<link rel="stylesheet" href="dashboard.css">
</head>
<body>
<div class="container-fluid container-xl py-4">
<header class="card mb-3" id="hero">
<div class="card-body d-flex align-items-center flex-wrap gap-3">
<div>
<h1 class="xau-symbol mb-1"><i class="bi bi-coin me-1"></i>XAU/USD</h1>
<div class="display-5 fw-semibold xau-price" id="hero-price">—</div>
</div>
<div class="d-flex flex-wrap gap-2" id="hero-rail" aria-label="Select timeframe"></div>
<div class="ms-auto text-end">
<div id="hero-status" role="status"><span class="live-dot"></span>…</div>
<div class="small text-secondary"><i class="bi bi-hourglass-split me-1"></i>M5 close in <span class="num" id="hero-cd">--:--</span></div>
<div class="small text-secondary" id="hero-updated">updated —</div>
</div>
<header class="desk-bar sticky-top">
<div class="desk-shell d-flex align-items-center flex-wrap gap-3">
<div class="d-flex align-items-baseline gap-2">
<i class="fa-solid fa-coins desk-mark" aria-hidden="true"></i>
<span class="desk-symbol">XAU/USD</span>
<span class="desk-price num" id="bar-price">—</span>
</div>
<nav class="desk-rail" id="bar-rail" aria-label="Jump to a timeframe"></nav>
<div class="desk-status ms-auto">
<span id="bar-market"><span class="live-dot"></span>Connecting</span>
<span class="desk-meta"><i class="fa-solid fa-hourglass-half" aria-hidden="true"></i>
5-minute close <span class="num" id="bar-cd">--:--</span></span>
<span class="desk-meta" id="bar-updated">waiting for data</span>
</div>
</header>
<div class="alert alert-warning d-none" id="banner-closed" role="alert">
<i class="bi bi-moon-fill me-2"></i>Market closed (weekend) — showing Friday's last state. Calendar and news stay live.
</div>
<div class="alert alert-danger d-none" id="banner-fail" role="alert"></div>
</header>
<main class="desk-shell pb-5">
<div class="alert alert-warning d-none mt-3" id="banner-closed" role="alert">
<i class="fa-solid fa-moon me-2" aria-hidden="true"></i>
Gold is closed for the weekend. The cards below hold Friday's last reading; news and the calendar keep updating.
</div>
<div class="alert alert-danger d-none mt-3" id="banner-fail" role="alert"></div>
<div class="visually-hidden" id="dashboard-live" aria-live="polite"></div>
<main>
<section class="card mb-4 d-none" id="context" aria-label="Market context">
<div class="card-body d-flex flex-wrap gap-3 align-items-start">
<div class="flex-grow-1" id="ctx-blurb"><strong>Market context</strong></div>
<div class="d-flex flex-wrap gap-2" id="ctx-chips"></div>
<section class="board mt-3" aria-labelledby="board-heading">
<div class="row g-0">
<div class="col-lg-4 board-read">
<h1 class="board-headline" id="board-heading">Reading the market</h1>
<p class="board-sub" id="board-sub">Waiting for the first scan to finish.</p>
<div class="board-news d-none" id="context">
<p class="context-text" id="ctx-blurb"></p>
<div class="context-chips" id="ctx-chips"></div>
</div>
<div class="board-stats" id="board-stats"></div>
</div>
</section>
<div class="col-lg-8 board-plot">
<h2 class="plot-title">Direction by timeframe</h2>
<div class="chart-wrap chart-direction">
<canvas id="chart-direction" role="img"
aria-label="Direction score of each timeframe, from minus 100 selling to plus 100 buying"></canvas>
</div>
<p class="plot-note">Bars grow out of the centre line. Right is buying pressure, left is selling
pressure, and the full scale runs to 100 either way.</p>
</div>
</div>
</section>
<section class="mb-4" aria-labelledby="workspace-heading">
<h2 class="text-uppercase text-secondary small fw-bold mb-3 d-flex align-items-center gap-2" id="workspace-heading">
<i class="bi bi-crosshair text-primary"></i>
Signal workspace
<span class="fw-normal text-body-secondary" style="text-transform:none">— select a timeframe</span>
</h2>
<div id="tf-grid"></div>
</section>
<section class="mt-4" aria-labelledby="tf-heading">
<div class="sec-head">
<h2 class="sec-title" id="tf-heading">Four reads on one market</h2>
<span class="sec-note">Every card is an independent scan. They are allowed to disagree.</span>
</div>
<div class="row g-3 row-cols-1 row-cols-md-2 row-cols-xxl-4" id="tf-grid"></div>
</section>
<section aria-labelledby="fundamentals-heading">
<h2 class="text-uppercase text-secondary small fw-bold mb-3 mt-4 d-flex align-items-center gap-2" id="fundamentals-heading">
<i class="bi bi-globe2 text-primary"></i>Fundamentals
</h2>
<div class="row g-3">
<div class="col-lg-8">
<div class="card h-100">
<div class="card-body">
<div class="d-flex align-items-center gap-3 mb-3 flex-wrap">
<h3 class="h5 card-title mb-0"><i class="bi bi-calendar-event me-2 text-primary"></i>Economic calendar</h3>
<span class="text-secondary small" id="calendar-note">times local · USD events</span>
<div class="form-check form-switch ms-auto">
<input class="form-check-input" type="checkbox" id="hi-only">
<label class="form-check-label small" for="hi-only">high impact only</label>
</div>
</div>
<div class="table-responsive">
<table class="table table-sm">
<thead>
<tr>
<th>Time</th>
<th>Cur</th>
<th>Impact</th>
<th>Event</th>
<th class="text-end">Fcst</th>
<th class="text-end">Prev</th>
<th class="text-end">Actual</th>
</tr>
</thead>
<tbody id="cal-body"><tr><td colspan="7" class="text-secondary text-center py-3">Loading calendar…</td></tr></tbody>
</table>
<section class="mt-4" aria-labelledby="fund-heading">
<div class="sec-head">
<h2 class="sec-title" id="fund-heading">What else is moving gold</h2>
<span class="sec-note">Context only. None of it changes a signal.</span>
</div>
<div class="row g-3">
<div class="col-xl-7">
<div class="card h-100">
<div class="card-body">
<div class="card-head">
<h3 class="card-title"><i class="fa-solid fa-calendar-day" aria-hidden="true"></i>Economic calendar</h3>
<span class="card-note" id="calendar-note">loading</span>
<div class="form-check form-switch ms-auto mb-0">
<input class="form-check-input" type="checkbox" id="hi-only">
<label class="form-check-label" for="hi-only">High impact only</label>
</div>
</div>
</div>
</div>
<div class="col-lg-4">
<div class="card h-100">
<div class="card-body">
<div class="d-flex align-items-center gap-3 mb-3 flex-wrap">
<h3 class="h5 card-title mb-0"><i class="bi bi-newspaper me-2 text-primary"></i>Newswire</h3>
<span class="text-secondary small" id="wire-note"></span>
</div>
<div id="wire-total" class="d-none"></div>
<div id="wire"></div>
<div class="table-responsive cal-scroll">
<table class="table table-sm align-middle cal-table">
<thead>
<tr>
<th scope="col">Time</th>
<th scope="col">Cur</th>
<th scope="col">Impact</th>
<th scope="col">Event</th>
<th scope="col" class="text-end">Forecast</th>
<th scope="col" class="text-end">Previous</th>
<th scope="col" class="text-end">Actual</th>
</tr>
</thead>
<tbody id="cal-body">
<tr><td colspan="7" class="empty">Loading the calendar</td></tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</section>
</main>
<div class="col-xl-5">
<div class="card h-100">
<div class="card-body">
<div class="card-head">
<h3 class="card-title"><i class="fa-solid fa-newspaper" aria-hidden="true"></i>Newswire</h3>
<span class="card-note ms-auto" id="wire-note"></span>
</div>
<div id="wire-tone" class="tone d-none">
<div class="chart-wrap chart-tone">
<canvas id="chart-tone" role="img" aria-label="Share of headlines leaning bullish, neutral or bearish for gold"></canvas>
</div>
<div class="tone-legend" id="tone-legend"></div>
</div>
<div id="wire"></div>
</div>
</div>
</div>
</div>
</section>
<footer class="mt-4 pt-3 border-top text-secondary small">
<i class="bi bi-info-circle me-1"></i>
Signals are generated on closed candles only and refresh at candle boundaries
(M5/M15/H1/H4). Not financial advice — educational and research use only.
<footer class="desk-foot">
<i class="fa-solid fa-circle-info" aria-hidden="true"></i>
Every signal is built from closed candles and refreshes when the next candle closes.
Educational and research use, not financial advice.
</footer>
</div>
</main>
<div class="modal fade" id="tf-detail" tabindex="-1" aria-labelledby="detail-title" aria-hidden="true">
<div class="modal fade" id="tf-detail" tabindex="-1" aria-labelledby="detail-title">
<div class="modal-dialog modal-xl modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<div>
<h5 class="modal-title mb-0" id="detail-title">—</h5>
<div class="small text-secondary">
<h5 class="modal-title" id="detail-title">Timeframe</h5>
<div class="modal-sub">
<span class="num" id="detail-price">—</span>
<span id="detail-sub"></span>
</div>
</div>
<button id="detail-close" type="button" class="btn-close" aria-label="Close detail view"></button>
<button id="detail-close" type="button" class="btn-close" aria-label="Close the full analysis"></button>
</div>
<div class="modal-body" id="detail-body"></div>
</div>
</div>
</div>
<div class="modal fade" id="news-detail" tabindex="-1" aria-labelledby="nd-title" aria-hidden="true">
<div class="modal fade" id="news-detail" tabindex="-1" aria-labelledby="nd-title">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title" id="nd-title">News detail</h5>
<button id="nd-close" type="button" class="btn-close" aria-label="Close news detail"></button>
<button id="nd-close" type="button" class="btn-close" aria-label="Close the news detail"></button>
</div>
<div class="modal-body" id="nd-body"></div>
</div>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/jquery@3.7.1/dist/jquery.min.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.7/dist/chart.umd.min.js" crossorigin="anonymous"></script>
<script src="dashboard.js"></script>
</body>
</html>
+947 -539
View File
File diff suppressed because it is too large Load Diff
+472 -66
View File
@@ -74,11 +74,13 @@ from app_config import (
from providers import BaseDataProvider, make_data_provider
from providers.base import _is_daily_maintenance
from agent_panel import attach_panel, panel_health
from exposure import attach_exposure, size_shared
from llm_client import get_llm_client
from news_engine import FundamentalContext, NewsEngine
from pair_signals import PAIR_PRODUCERS
from signal_aggregator import SignalAggregator, SignalReport
from signal_log import gate_reason
from signal_log import (gate_reason, last_alerted_bars, mark_alerted,
unmark_alerted)
from violent_watch import attach_watch, should_send_headsup
if TYPE_CHECKING: # type hints only — ViolentWatch is never needed at runtime here
@@ -199,6 +201,16 @@ TF_CONFIRMATIONS = {
}
_executor = ThreadPoolExecutor(max_workers=4)
# ONE worker for an alert's database steps (refresh the "already open" line,
# stamp the row as alerted, undo the stamp). A single worker is the mutual
# exclusion: one alert's read and write run back to back, in arrival order,
# so of two alerts sent at the same candle close the second counts the
# first. It also keeps SQLite off the event loop (the logger's lock can be
# held for seconds by the outcome labeller) and never queues behind the
# scan pool above. Not an asyncio.Lock: on Python 3.9 one created at
# import binds to a loop that is not the running one.
_alert_db_executor = ThreadPoolExecutor(
max_workers=1, thread_name_prefix="alert-db")
# ── Markdown escaping ─────────────────────────────────────────────────
@@ -275,7 +287,6 @@ def _mtf_line(report: SignalReport) -> str:
# Informational fixed-pip target shown on every trade setup (display-only,
# not part of the backtested exit model).
_INFO_TP_PIPS = 100
_IND_SHORT = {
"ema_trend": "EMA", "rsi": "RSI", "macd": "MACD", "bollinger": "BB",
@@ -324,12 +335,113 @@ _DISCLAIMER = (
)
def format_signal_message(report: SignalReport) -> str:
def _panel_odds_word(p: float, usual: Optional[float]) -> str:
"""Plain-language headline: the estimate against the usual odds for
this kind of setup (8 points either way), or against 60 / 40 when no
base rate is known. Never a verdict word."""
if usual is not None:
d = p - usual
if d >= 8:
return "the odds look better than usual"
if d <= -8:
return "the odds look worse than usual"
return "the odds look about usual"
if p >= 60:
return "the odds look good"
if p <= 40:
return "the odds look poor"
return "the odds look about even"
def _panel_lines(report: SignalReport) -> list[str]:
"""The AI review for a new trader (owner rule 2026-09-26: simple and
clear). One headline in everyday words, the chance "in 100" against
the usual odds for this kind of setup, one reason for and one
against, and a plain warning. Used whenever the verdict carries a
probability (the "mean" source, and "macro" since 2026-09-26); no
verdict word, no percentages that are not probabilities, no persona
names unless several personas ran (research configurations)."""
p = float(report.panel_probability or 0.0)
base = (report.panel_context or {}).get("base_rate") or {}
usual = base.get("win_rate")
head = f"🤖 AI check: {_panel_odds_word(p, usual)}."
if report.panel_rationale:
head += f" {_escape_md(report.panel_rationale)}"
chance = f" Chance it pays off: about *{p:.0f} in 100*"
if usual is not None:
chance += f" (usual for this kind of setup: {usual:.0f} in 100)"
out = [head, chance + "."]
takes = [t for t in report.panel_takes if t.get("p_win") is not None]
lead = max(takes, key=lambda t: abs(float(t["p_win"]) - 50.0), default=None)
if lead is not None:
if lead.get("reason_for"):
out.append(f" For: {_escape_md(lead['reason_for'])}")
if lead.get("reason_against"):
out.append(f" Against: {_escape_md(lead['reason_against'])}")
if len(takes) > 1: # one persona: the headline says it all
votes = " · ".join(f"{t['persona']} {float(t['p_win']):.0f}%" for t in takes)
out.append(f" _{_escape_md(votes)}_")
out.append(" This check is an experiment. Do not trade on it alone.")
return out
def _book_rate_line(report: SignalReport) -> Optional[str]:
"""One line of validated-book statistics for the setup's class
(timeframe × strategy × grade × direction, falling back to wider
classes exactly as the agent panel does). None when no table or no
primary analysis is available; never raises."""
ts = report.trade_setup
if ts is None or report.primary_analysis is None:
return None
try:
from agent_panel import _base_rate
rate = _base_rate(report.primary_analysis.timeframe,
getattr(ts, "strategy", None) or "GENERIC",
report.signal_quality, ts.direction)
except Exception:
logger.debug("book rate line unavailable", exc_info=True)
return None
if not rate or "win_rate" not in rate or "mean_r" not in rate:
return None
scope = {"class": "this setup class",
"timeframe_direction": f"{ts.direction} setups on this timeframe",
"timeframe": "this timeframe"}.get(rate.get("level"), "this class")
return (f"📚 Book ({scope}): {rate['n']} trades, "
f"{rate['win_rate']:.0f}% winners, avg {rate['mean_r']:+.2f}R per trade")
def _plain_words_line(report: SignalReport) -> str:
"""Grade first, then the two raw numbers in words (owner request
2026-09-27): "Direction: strong buy · Reliability: good · timeframes
agree" replaces "Score +38.2 · Conf 62% · MTF ✅". The words come from
main.plain_words so every surface says the same thing."""
from main import direction_words, reliability_words # display helper, no side effects
agree = "timeframes agree" if report.mtf_agreement else "timeframes disagree"
return (f"📐 Direction: *{direction_words(report.final_score)}* · "
f"Reliability: *{reliability_words(report.final_confidence)}* · {agree}")
def _daily_trend_line(report: SignalReport) -> str:
""""🧭 Daily trend: rising — this buy goes with it" (main.daily_trend_words,
the same words on every surface), or "" when the scan carries no daily
trend. On M15 a plan against a clear daily trend never reaches an
alert (the core removes it); on the other timeframes the line is
information only."""
from main import daily_trend_words # display helper, no side effects
words = daily_trend_words(report)
return f"🧭 {_escape_md(words)}" if words else ""
def format_signal_message(report: SignalReport, channel: bool = False) -> str:
"""The alert / scan text. `channel=True` builds the copy for every
audience but the owner's own chat (the subscriber channel, a group
the bot was switched on in): identical, except that the lot-size line
(which reveals the owner's risk per trade) is left out unless
config.json sets trade_size.telegram_channel."""
sig = report.final_signal
emoji = _SIGNAL_EMOJI.get(sig, "⬜")
qe = _QUALITY_EMOJI.get(report.signal_quality, "")
label = _signal_label(sig)
mtf_tag = "MTF ✅" if report.mtf_agreement else "MTF ❌"
tf = report.primary_analysis.timeframe if report.primary_analysis else ""
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
@@ -337,16 +449,19 @@ def format_signal_message(report: SignalReport) -> str:
vol_str = _escape_md(str(report.volatility_regime))
lines: list[str] = [
f"{emoji} *{label} · {report.display_name} · {tf}* {qe}{report.signal_quality}",
f"{emoji} *Grade {report.signal_quality} · {label} · {report.display_name} · {tf}* {qe}",
f"🕐 {now} · 💰 {_fmt_price(report.current_price)}",
]
live = _live_price_line(report.symbol)
if live:
lines.append(live)
lines += [
f"Score *{report.final_score:+.1f}* · Conf *{report.final_confidence:.0f}%* · {mtf_tag}",
_plain_words_line(report),
f"📊 Regime: {regime_str} · Vol: {vol_str}",
]
trend_line = _daily_trend_line(report)
if trend_line:
lines.append(trend_line)
if report.action_recommendation:
# Pattern identifiers use underscores, which legacy Telegram Markdown
# can misparse even when escaped inside this italicized action line.
@@ -358,30 +473,30 @@ def format_signal_message(report: SignalReport) -> str:
dir_emoji = "📈" if ts.direction == "BUY" else "📉"
risk_usd = ts.risk_pips * ts.pip_value
reward_usd = ts.reward_pips * ts.pip_value
# Order and wording (2026-09-26): the trail is the tested exit
# (the backtester never exits at TP1 — `_simulate_trailing` reads
# only the stop and the trailing rule), so it is printed first
# and TP1/TP2 are labelled guides with their R. The RR line is
# the PLANNED TP1 ratio, not the expected payoff: the validated
# books earn their R from the trailed runners (M5 average win
# +1.03R vs loss −0.76R; M15 +0.87R vs −0.93R).
risk_dist = abs(ts.entry_price - ts.stop_loss)
lines += [
"",
"🚀 *TRADE SETUP*",
f"{dir_emoji} Direction: *{ts.direction}*",
f"▶ Entry `{_fmt_price(ts.entry_price)}`",
f"🛑 SL `{_fmt_price(ts.stop_loss)}` (${risk_usd:.2f} / {ts.risk_pips:.0f}p)",
f"🎯 TP1 `{_fmt_price(ts.take_profit_1)}` (${reward_usd:.2f} / {ts.reward_pips:.0f}p)",
f"🛑 SL `{_fmt_price(ts.stop_loss)}` (${risk_usd:.2f} / {ts.risk_pips:.0f}p = 1R)",
]
if ts.take_profit_2:
lines.append(f"🎯 TP2 `{_fmt_price(ts.take_profit_2)}`")
# Informational fixed-pip target: where a 100-pip TP would sit for
# this setup (display-only — not part of the backtested exit model).
info_dist = _INFO_TP_PIPS * ts.pip_value
if ts.direction == "BUY":
info_price = ts.entry_price + info_dist
else:
info_price = ts.entry_price - info_dist
risk_dist = abs(ts.entry_price - ts.stop_loss)
info_r = f" · ≈{info_dist / risk_dist:.1f}R" if risk_dist > 0 else ""
lines.append(
f"🎯 TP {_INFO_TP_PIPS}p: `{_fmt_price(info_price)}` (${info_dist:.2f}{info_r})"
)
lines.append(f"⚖️ RR 1:{ts.risk_reward:.1f}")
# Lot size for the owner's risk limit and the alerted plans still
# open in this direction (exposure.attach_exposure stamps both
# post-hoc; empty unless configured / unless something is open).
# The size is private: the channel copy carries it only when
# config.json says so.
if report.size_line and (not channel or size_shared("telegram_channel")):
lines.append(f"📏 {_escape_md(report.size_line)}")
if report.open_line:
lines.append(f"🧱 {_escape_md(report.open_line)}")
# Trailing-stop management plan (matches backtested exit model:
# TP1 is advisory, the trail is the exit). Params are stamped on
@@ -403,15 +518,37 @@ def format_signal_message(report: SignalReport) -> str:
trail_dist = dist_r * risk_dist
tag = " (pullback)" if is_pb else ""
lines.append(
f"🔄 Trail{tag}: arm `{_fmt_price(arm_price)}` "
f"(+{arm_r:.1f}R), trail SL ${trail_dist:.2f} "
f"behind best price — TP1 is a guide, let it run"
f"🔄 Exit rule{tag} (tested): once price reaches "
f"`{_fmt_price(arm_price)}` (+{arm_r:.1f}R), trail the SL "
f"${trail_dist:.2f} behind the best price, moving it at "
f"candle close only. Otherwise the SL is the exit."
)
tp1_r = (abs(ts.take_profit_1 - ts.entry_price) / risk_dist
if risk_dist > 0 else 0.0)
lines.append(
f"🎯 TP1 guide `{_fmt_price(ts.take_profit_1)}` "
f"(${reward_usd:.2f} / {ts.reward_pips:.0f}p = {tp1_r:.1f}R)"
)
if ts.take_profit_2:
tp2_r = (abs(ts.take_profit_2 - ts.entry_price) / risk_dist
if risk_dist > 0 else 0.0)
lines.append(
f"🎯 TP2 guide `{_fmt_price(ts.take_profit_2)}` ({tp2_r:.1f}R)")
lines.append(f"⚖️ Planned TP1 RR 1:{ts.risk_reward:.1f} (guide only — the trail is the exit)")
# What this setup class actually paid in the validated books
# (agent_panel_base_rates.json, the table the AI estimate is
# anchored to): the honest expectation for a discretionary reader.
rate_line = _book_rate_line(report)
if rate_line:
lines.append(rate_line)
extra: list[str] = []
mtf = _mtf_line(report)
if mtf:
extra.append(f"📈 MTF: {mtf}")
extra.append(f"📈 Other timeframes: {mtf}")
ind_summary = _indicator_summary(report)
if ind_summary:
extra.append(f"📋 {ind_summary}")
@@ -423,7 +560,9 @@ def format_signal_message(report: SignalReport) -> str:
extra.append(_escape_md(report.watch_line))
if report.news_summary:
extra.append(f"📰 {_escape_md(report.news_summary)}")
if report.panel_verdict:
if report.panel_verdict and (report.panel_probability or 0) > 0:
extra.extend(_panel_lines(report))
elif report.panel_verdict:
mark = {"AGREE": "✅", "OPPOSE": "⛔", "NEUTRAL": "➖"}.get(
report.panel_verdict, "")
panel_label = ("AI macro view"
@@ -512,7 +651,13 @@ def _run_signal_sync(
config = PAIR_PRODUCERS[symbol](timeframe, confirmation_tfs)
aggregator = SignalAggregator(config)
aggregator.data_provider = _get_shared_dp()
return aggregator.analyze_pair(symbol=symbol, log_source="telegram")
report = aggregator.analyze_pair(symbol=symbol, log_source="telegram")
# Display-only lot-size and "already open" lines (never raises; one
# read-only query on signals.db). Stamped here, on the scan executor,
# for the manual scans. An ALERT refreshes them just before it is sent
# (_deliver_alert): every timeframe scans at the same candle close, so
# the line stamped here cannot know about an alert going out beside it.
return attach_exposure(report, surface="telegram")
async def run_signal(
@@ -638,6 +783,112 @@ def _alert_sent_line(report: SignalReport, tf_label: str, delivered_to: int,
_last_alert_quality: Dict[str, str] = {} # key → grade of last ALERTED signal
_last_alert_bar: Dict[str, pd.Timestamp] = {} # key → bar label of last ALERTED setup
# Entry cooldown, in primary bars, after an ALERTED setup (2026-09-26):
# the backtester's `_apply_cooldown` (cooldown_bars=5) never enters within
# 5 bars of an accepted entry, so every validated book counts one trade per
# move. Live had no such rule: the entry-moved re-alert below re-sent the
# same idea on consecutive bars while price ran (2026-09-15 08:00–08:45,
# 2026-09-18 06:00–06:30, 2026-09-22 18:00–19:40 …); over 2026-09-04 →
# 2026-09-25 those re-entries were 42 of 99 unique setups and −25.1R of
# the −37.8R raw advisory book (backtest/research/rr_and_sell_campaign_
# 2026-09-26.md). Must equal backtest.cooldown_bars in gpu_backtest.py.
_ALERT_COOLDOWN_BARS = 5
def _key_bar_seconds(key: str) -> Optional[int]:
"""Candle length for an alert key ("XAU/USD_15min" → 900). None when
the key carries no known timeframe (test keys), which disables the
bar cooldown for that key."""
tf_value = key.rsplit("_", 1)[-1]
for tf, secs in BaseDataProvider.CANDLE_SECONDS.items():
if tf.value == tf_value:
return secs
return None
def _bar_label_ts(label: Any) -> Optional[pd.Timestamp]:
"""A bar label as a UTC timestamp; None when it cannot be parsed."""
try:
ts = pd.Timestamp(label)
except Exception:
return None
if pd.isna(ts):
return None
return ts.tz_localize("UTC") if ts.tzinfo is None else ts.tz_convert("UTC")
def _report_bar_ts(report: SignalReport) -> Optional[pd.Timestamp]:
"""The report's primary bar label as a UTC timestamp; None when it
cannot be parsed (the cooldown then does not apply)."""
return _bar_label_ts(report.timestamp)
def _primary_bars_since(key: str, report: SignalReport, last: pd.Timestamp,
now_bar: pd.Timestamp) -> Optional[int]:
"""How many primary bars lie after `last`, up to and including the
report's own bar, in the frame the scan analysed (report._frames):
the backtester's position difference `pos - last_entry_pos`. None when
the report carries no frame for the key's timeframe or its bar is not
in it; the caller then falls back to clock time."""
try:
tf = Timeframe(key.rsplit("_", 1)[-1])
df = (getattr(report, "_frames", None) or {}).get(tf)
if df is None or len(df) == 0:
return None
idx = df.index
if getattr(idx, "tz", None) is None:
idx = idx.tz_localize("UTC")
if now_bar not in idx:
return None
return int(((idx > last) & (idx <= now_bar)).sum())
except Exception:
return None
def _inside_alert_cooldown(key: str, report: SignalReport) -> bool:
"""True while this key sits within _ALERT_COOLDOWN_BARS primary bars
of its last ALERTED setup's bar (the backtester's entry cooldown).
Bars are COUNTED in the scan's own primary frame, like the
backtester's bar positions. Clock time is not the same thing: across
the daily settlement pause or the weekend an hour or two days pass
while no bar prints, and a clock rule released the 22:00 UTC bar that
no book ever enters (six M15 and three M5 entries in three years,
2026-10-01 review). Clock time remains the fallback for a report
without frames."""
last = _last_alert_bar.get(key)
if last is None or report.trade_setup is None:
return False
secs = _key_bar_seconds(key)
now_bar = _report_bar_ts(report)
if secs is None or now_bar is None:
return False
if now_bar < last:
return False
bars = _primary_bars_since(key, report, last, now_bar)
if bars is not None:
return bars < _ALERT_COOLDOWN_BARS
return (now_bar - last).total_seconds() < _ALERT_COOLDOWN_BARS * secs
def _restore_alert_cooldowns(bars: Dict[Any, str]) -> int:
"""Seed the cooldown memory from signal_log.last_alerted_bars() after a
start-up: {(symbol, timeframe): bar label} of the newest alerted row.
Without it every restart forgot the last alert and the next bar could
alert again inside the 5 bars. Only the cooldown is restored; the
dedup fingerprints start empty, as before. A key already in memory is
left alone. Returns how many keys were seeded."""
restored = 0
for (symbol, timeframe), label in (bars or {}).items():
bar = _bar_label_ts(label)
key = f"{symbol}_{timeframe}"
if bar is None or key in _last_alert_bar:
continue
_last_alert_bar[key] = bar
restored += 1
return restored
def _alert_fingerprint(report: SignalReport) -> tuple:
@@ -667,6 +918,14 @@ def _should_alert(key: str, report: SignalReport) -> bool:
Pure check — call _commit_alert only after the message actually went
out, so a failed Telegram send doesn't consume the alert for an hour.
"""
if _inside_alert_cooldown(key, report):
# Same rule as the backtester: no second entry within 5 bars of
# the last one, whatever the grade or the entry drift did.
logger.info(
f"alert skipped: {key} {report.final_signal.name} grade "
f"{report.signal_quality} bar {report.timestamp} is inside the "
f"{_ALERT_COOLDOWN_BARS}-bar entry cooldown (backtester rule)")
return False
fp = _alert_fingerprint(report)
if _last_alert_fp.get(key) != fp:
return True
@@ -688,6 +947,10 @@ def _should_alert(key: str, report: SignalReport) -> bool:
def _commit_alert(key: str, report: SignalReport) -> None:
# In-memory dedup state only. The signal-log stamp of the alert is
# written (and, for an undelivered alert, undone) by _deliver_alert on
# the alert-db worker: a database write does not belong on the event
# loop this function runs on.
_last_alert_fp[key] = _alert_fingerprint(report)
_last_alert_ts[key] = time.time()
_last_alert_quality[key] = report.signal_quality
@@ -696,6 +959,9 @@ def _commit_alert(key: str, report: SignalReport) -> None:
_last_alert_setup[key] = (
setup.entry_price, abs(setup.entry_price - setup.stop_loss)
)
bar = _report_bar_ts(report)
if bar is not None:
_last_alert_bar[key] = bar
else:
_last_alert_setup.pop(key, None)
@@ -739,17 +1005,123 @@ async def _send_alert(bot, chat_id: int, text: str) -> bool:
return False
async def _broadcast(bot, text: str) -> int:
def _is_owner_chat(chat_id: Any) -> bool:
"""True for the owner's own chat with the bot, the one audience of the
private lot-size line. The owner lock is per USER, so /autoscan or a
scan button used inside a group makes that group a target: it is not
the owner's chat and gets the copy without the size."""
return OWNER_ID is not None and chat_id == OWNER_ID
async def _broadcast(bot, text: str, channel_text: Optional[str] = None) -> int:
"""Send `text` to every signal target; returns how many chats got it.
Shared by the scan and breach paths so the delivery rule cannot
drift between them."""
drift between them. `channel_text`, when given, is what every target
except the owner's own chat receives instead (the copy without the
owner's private lot-size line)."""
delivered = 0
for chat_id in _signal_targets():
if await _send_alert(bot, chat_id, text):
body = (text if channel_text is None or _is_owner_chat(chat_id)
else channel_text)
if await _send_alert(bot, chat_id, body):
delivered += 1
return delivered
def _stamp_exposure_and_mark(report: SignalReport) -> Optional[str]:
"""On the alert-db worker: refresh the lot-size and "already open"
lines, then stamp the report's signal-log row as alerted. Read first,
write second, nothing in between: the next alert on the worker sees
this stamp. Returns the stamp to undo if no chat receives the alert
(None when nothing was written). Never raises."""
attach_exposure(report, surface="telegram")
return mark_alerted(report)
def _undo_cancelled_stamp(report: SignalReport, done: Any) -> None:
"""Done-callback of the stamp job when the alert's task was cancelled
while that job was queued or running (a shutdown, or auto-scan switched
off, at the wrong moment). Cancelling a task cannot stop a job the
worker has already started, so its stamp lands although nothing will be
sent: clear it. Runs on the alert-db worker right after the job, also
when the pool has already been told to shut down. Never raises."""
try:
if done.cancelled():
return # never ran: nothing was written
stamp = done.result()
if stamp:
unmark_alerted(report, stamp)
except Exception:
logger.warning("alert: could not clear the stamp of a cancelled "
"alert", exc_info=True)
async def _undo_alert_mark(report: SignalReport, stamp: Optional[str]) -> None:
"""The alert reached no chat: clear the stamp this attempt wrote (and
only that one), so the row does not count as a plan the reader holds.
Never raises."""
if not stamp:
return
try:
await asyncio.get_running_loop().run_in_executor(
_alert_db_executor, unmark_alerted, report, stamp)
except Exception:
logger.warning("alert: could not clear the alerted stamp of an "
"undelivered alert", exc_info=True)
async def _deliver_alert(bot, key: str, report: SignalReport, tf_label: str,
header: str = "", kind: str = "alert") -> int:
"""Send one alert: panel → exposure lines + alerted stamp → both
copies → commit. The ONE delivery path of the candle scan and the
level break; the caller holds _alert_lock(key) and has passed
_should_alert. Returns how many chats received it.
The row is stamped BEFORE the send so that an alert going out at the
same candle close on another timeframe already counts this plan in its
"already open" line (2026-10-01: stamped after the send, two alerts at
one boundary never counted each other). A send that reaches no chat
undoes the stamp and commits nothing, so the setup stays eligible for
the next scan; so does a cancellation before the send has started.
Residue: if this send fails completely after another alert has already
counted it, that alert's line is one too high."""
loop = asyncio.get_running_loop()
# Panel AFTER the quality gate + dedup decision so no LLM spend on
# suppressed alerts. Runs on the scan executor to keep the event loop
# free; this key's alert lock is held for the panel's wall time
# (≤~25 s), which only delays a same-key duplicate that _should_alert
# would drop anyway.
await loop.run_in_executor(_executor, attach_panel, report)
# submit(), not run_in_executor(): the job's own future is kept so that
# a cancellation HERE (nothing has been sent yet) can still undo the
# stamp the job writes.
job = _alert_db_executor.submit(_stamp_exposure_and_mark, report)
try:
stamp = await asyncio.wrap_future(job, loop=loop)
except asyncio.CancelledError:
job.add_done_callback(
functools.partial(_undo_cancelled_stamp, report))
raise
delivered = 0
try:
detail = header + format_signal_message(report)
delivered = await _broadcast(
bot, detail, header + format_signal_message(report, channel=True))
except Exception:
# (A cancellation is not an Exception: on shutdown mid-send a chat
# may already hold the message, so the stamp stays.)
await _undo_alert_mark(report, stamp)
raise
if not delivered:
await _undo_alert_mark(report, stamp)
return 0
# Commit only when the alert reached at least one target; a fully
# failed send stays eligible for the next scan.
_commit_alert(key, report)
logger.info(_alert_sent_line(report, tf_label, delivered, kind=kind))
return delivered
# ── Candle-boundary scheduler ────────────────────────────────────────
_autoscan_chats: Set[int] = set()
@@ -903,20 +1275,7 @@ async def _scan_timeframe(
async with _alert_lock(key):
if not _should_alert(key, report):
continue
# Panel AFTER the quality gate + dedup decision so no LLM
# spend on suppressed alerts. Runs on the scan executor to
# keep the event loop free; this key's alert lock is held
# for the panel's wall time (≤~25 s), which only delays a
# same-key duplicate that _should_alert would drop anyway.
await asyncio.get_running_loop().run_in_executor(
_executor, attach_panel, report)
detail = format_signal_message(report)
delivered = await _broadcast(bot, detail)
# Commit only when the alert reached at least one target;
# a fully failed send stays eligible for the next scan.
if delivered:
_commit_alert(key, report)
logger.info(_alert_sent_line(report, tf.value, delivered))
await _deliver_alert(bot, key, report, tf.value)
# ── WebSocket level-breach monitor ──────────────────────────────────
@@ -999,6 +1358,17 @@ def _cleanup_cooldowns() -> None:
async def _handle_breach(bot, pair_symbol: str, price: float, level_name: str) -> None:
# No M15 bar exists to read while the bar that "just closed" falls in
# gold's daily settlement pause (21:00–22:00 UTC in summer) — the
# first post-pause bar closes 15 min after the pause ends. A rescan
# here only re-reads the last pre-pause bar: the 2026-09-26 review of
# signals.db found the 20:45 UTC bar logged again at 22:01, 22:06 and
# 22:11 every evening from these tick-driven rescans (the candle loop
# already skips the same window, `_closed_bar_in_settlement`).
if _closed_bar_in_settlement(15):
logger.info(f"Level breach: {pair_symbol} crossed {level_name} at {price} — "
f"skipped, the M15 bar to scan is inside the settlement pause")
return
now = time.time()
if now - _scan_cooldowns.get(pair_symbol, 0) < _WS_COOLDOWN:
# Park the crossing instead of dropping it — price holding beyond
@@ -1052,20 +1422,16 @@ async def _handle_breach(bot, pair_symbol: str, price: float, level_name: str) -
if not _should_alert(key, report):
return
# Same post-gate panel attach as the auto-scan path (see
# _scan_timeframe for the spend/lock rationale).
await asyncio.get_running_loop().run_in_executor(
_executor, attach_panel, report)
# The same delivery as the auto-scan path (panel, the refreshed
# "already open" line, stamp, send, commit), under the level-break
# header. The cached M15 report this path reuses can be up to a
# bar old; _deliver_alert refreshes its exposure lines.
header = (
f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed "
f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n"
)
detail = header + format_signal_message(report)
sent = await _broadcast(bot, detail)
if sent:
_commit_alert(key, report)
logger.info(_alert_sent_line(report, Timeframe.M15.value, sent,
kind="breach alert"))
await _deliver_alert(bot, key, report, Timeframe.M15.value,
header=header, kind="breach alert")
def _breach_task_done(task: asyncio.Task) -> None:
@@ -1347,7 +1713,9 @@ def _news_health_lines() -> List[str]:
Reads the class-level `NewsEngine._source_health` written by the
scan-path fetches plus the LLM client's health stamps — display
only, never triggers a fetch. ⚠️ marks a degraded dependency;
details live in the rotating log (transition-logged WARNINGs)."""
details live in the rotating log (transition-logged WARNINGs).
Failure details are exception text, so they are Markdown-escaped: an
unpaired underscore in one would make Telegram reject the reply."""
lines = ["\n*News & AI:*"]
# Snapshot: a scan thread may insert a source's FIRST entry while we
# iterate (dict-changed-size RuntimeError); entry mutation is safe.
@@ -1366,11 +1734,13 @@ def _news_health_lines() -> List[str]:
ff = sh.get("calendar:forexfactory")
if cal:
if NewsEngine.source_is_failing(cal):
lines.append(f" ⚠️ Calendar: unavailable — {cal['detail']}"
lines.append(" ⚠️ Calendar: unavailable — "
f"{_escape_md(str(cal['detail']))}"
f" ({_fmt_age(cal['err'])})")
elif NewsEngine.source_is_failing(ff):
lines.append(f" ⚠️ Calendar: {cal['extra']} — ForexFactory"
f" down: {ff['detail']} ({_fmt_age(ff['err'])})")
f" down: {_escape_md(str(ff['detail']))}"
f" ({_fmt_age(ff['err'])})")
else:
lines.append(f" ✅ Calendar: {cal['extra']}"
f" ({_fmt_age(cal['ok'])} ago)")
@@ -1378,13 +1748,27 @@ def _news_health_lines() -> List[str]:
if heads:
failing = NewsEngine.source_is_failing(heads)
mark = "⚠️" if failing else "✅"
detail = f" — {heads['detail']}" if failing else ""
detail = (f" — {_escape_md(str(heads['detail']))}"
if failing else "")
ts = heads["err"] if failing else heads["ok"]
lines.append(f" {mark} Headlines: {heads['extra']}{detail}"
f" ({_fmt_age(ts)} ago)")
# The local headline scorer (news_model modes fallback / primary).
# Present only in a process that ran it; a failing batch leaves the
# cycle's headlines on keyword scores, which nothing else showed.
model = sh.get("news_model")
if model:
if NewsEngine.source_is_failing(model):
lines.append(" ⚠️ News model: not scoring — "
f"{_escape_md(str(model['detail']))}"
f" ({_fmt_age(model['err'])})")
else:
lines.append(f" ✅ News model: {model['extra']}"
f" ({_fmt_age(model['ok'])} ago)")
for name, entry in sorted(sh.items()):
if name.startswith("rss:") and NewsEngine.source_is_failing(entry):
lines.append(f" ⚠️ {name[4:]}: {entry['detail']}"
lines.append(f" ⚠️ {_escape_md(name[4:])}: "
f"{_escape_md(str(entry['detail']))}"
f" ({_fmt_age(entry['err'])})")
client = get_llm_client()
if not client.available:
@@ -1616,7 +2000,12 @@ async def cb_scan(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
try:
report = await run_signal(btn["symbol"], btn["tf"])
text = format_signal_message(report)
# The reply lands in the chat the button was pressed in. Only the
# owner's own chat gets the private lot-size line; the owner
# pressing it inside a group must not show the group their risk.
chat = update.effective_chat
text = format_signal_message(
report, channel=not _is_owner_chat(chat.id if chat else None))
except Exception as exc:
logger.exception("Signal scan failed")
text = f"❌ Scan failed: {_escape_md(str(exc))}"
@@ -1692,6 +2081,7 @@ async def _cb_scan_all(query, symbol: str = XAUUSD) -> None:
"",
]
from main import direction_words, reliability_words # display helper (2026-09-27)
for r in reports:
emoji = _SIGNAL_EMOJI.get(r.final_signal, "⬜")
qe = _QUALITY_EMOJI.get(r.signal_quality, "")
@@ -1699,10 +2089,13 @@ async def _cb_scan_all(query, symbol: str = XAUUSD) -> None:
sig = _signal_label(r.final_signal)
regime = r.market_regime.replace("_", " ").title()
lines.append(
f"{emoji} *{tf_label}* {sig} {qe}{r.signal_quality} "
f"Score: {r.final_score:+.1f} {r.final_confidence:.0f}%"
f"{emoji} *{tf_label}* Grade {r.signal_quality} {qe} · {sig}"
)
lines.append(f" _{_escape_md(regime)}_ | {'✅ MTF' if r.mtf_agreement else '❌ MTF'}")
lines.append(
f" {direction_words(r.final_score)}, {reliability_words(r.final_confidence)} reliability, "
f"{'timeframes agree' if r.mtf_agreement else 'timeframes disagree'}"
)
lines.append(f" _{_escape_md(regime)}_")
if r.trade_setup and r.final_signal != SignalStrength.NEUTRAL:
ts = r.trade_setup
d = "📈" if ts.direction == "BUY" else "📉"
@@ -1740,6 +2133,18 @@ async def _post_init(app: Application) -> None:
if CHAT_ID:
_autoscan_chats.add(int(CHAT_ID))
# The 5-bar entry cooldown lives in memory; seed it from the signal log
# so a restart (a deploy, the weekly timer) cannot re-alert inside it.
# Before the feed check: /autoscan can start the loops later.
try:
bars = await asyncio.get_running_loop().run_in_executor(
_alert_db_executor, last_alerted_bars)
restored = _restore_alert_cooldowns(bars)
logger.info(f"Alert cooldown restored from the signal log: "
f"{restored} timeframe(s)")
except Exception:
logger.exception("Alert cooldown restore failed — starting without it")
if not _feed_active():
logger.info("Neither TELEGRAM_CHAT_ID nor TELEGRAM_CHANNEL_ID set — "
"auto-scan disabled on boot. Use /autoscan in chat to enable.")
@@ -1806,6 +2211,7 @@ async def _post_shutdown(app: Application) -> None:
logger.info("Shutting down ForexBot2...")
_stop_scheduler()
_executor.shutdown(wait=False)
_alert_db_executor.shutdown(wait=False)
logger.info("Shutdown complete")
+6489 -56
View File
File diff suppressed because it is too large Load Diff