Two bodies of work that touch the same files, so one commit.
Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule,
written before the runs):
- SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto
"both"). A finished setup against a clear daily trend (closed D1 bars, Sunday
stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier)
is removed as the last step of the core. One definition
(signal_aggregator.daily_trend_series) for live, the bar replay and the twin's
host side; D1 is a required frame of the M15 scan; mirrored in
cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs
1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs
2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R.
- FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1
(−9.7R), buy-only and sell-only on M15, the wide-stop cap
(xau_strategy.wide_stop_cap_atr stays 0.0, a research knob).
- New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins
unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15
pins byte for byte. Twin and reference script MD5-equal on every window;
recorder and feature-export parity re-proved; base-rate table rebuilt.
- DISPLAY (books byte-identical): exposure.py — the lot size for
trade_size.risk_per_trade (new config.json block, off by default, private to
the owner) and the count of alerted same-direction plans still open
(signals.alerted_at, read-only open_alerted_setups); the daily trend in words
on Telegram, the CLI and the dashboard; signals.daily_trend logged.
- providers/oanda: a date window with no bars is an empty page, not an error
(a D1 pad starting on a Saturday could not be fetched).
- Reports only, nothing shipped: seven context screens (no state nominated) and
eight outside data sources (none adds a forecast to gold's own daily trend).
- Independent verification (report section 5.13): every figure reproduced on
both scripts, scoring identical to the previous commit on all four
timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not
a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022;
the neighbour clause is thin (a simple 100-day average misses the R line by
1.2R); "never adds a trade" is an observation, not a guarantee.
Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects
elsewhere, plus 4 found by an independent review of the fixes; all fixed, each
with a test; no trade book changed:
- telegram_bot: one _deliver_alert for the candle scan and the level break. The
row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and
un-stamped when no chat received it or the task is cancelled mid-stamp, so
two alerts at one candle close count each other and no SQLite write runs on
the event loop. The 5-bar cooldown counts bars of the scan's own frame
(== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after
the settlement pause) and is restored from signals.db at start-up. The
lot-size line goes to the owner's own chat only (a group the owner ran
/autoscan in used to receive it). /status gains a news-model line and escapes
failure details.
- signal_log: mark_alerted returns the stamp it wrote; unmark_alerted;
last_alerted_bars (read-only).
- advisory_outcome: candle continuity is judged bar by bar inside the walk; a
hole after a trade's exit no longer voids it. The stored-candle replay
resolves 108 of 111 logged setups instead of 99, all equal to the live
resolver's R.
- providers/oanda: an empty date window is accepted only when it is 5 days or
shorter and no malformed candle was skipped; a longer one is retried, then
refused, instead of being cached as history with a hole.
- news_engine: blank RSS titles skipped; one unreadable headline no longer
costs the local model its whole batch (checked on the real bundle); model
health recorded; no DeepSeek audit call in a process that does not archive;
event-direction cache keyed per release; peek_week_events never fetches.
- llm_client: chat_json never raises and never logs the API key (requests
quotes the Authorization value in InvalidHeader; error bodies are redacted).
- signal_aggregator: the scan's news block is guarded; the replay path
(use_news=False) assigns exactly the values it did before.
- agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input
(event timing over the cached calendar week, headlines newest first).
scripts/panel_replay.py replays the logged payload per persona;
scripts/label_scan_outcomes.py counts with an ISO cutoff.
- test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34);
tests that write Parquet print a notice and return without pyarrow.
Verification after the last code change: suite 632 passed, 0 failed (also with
both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the
five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37
green; static Python 3.9 / SQLite 3.34 check clean on every touched file.
Evidence: backtest/research/daily_trend_campaign_2026-09-30.md,
context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6,
8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7).
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72%
agreement with DeepSeek on unseen days) replaces DeepSeek for headline
scoring. DeepSeek keeps event direction, the summary and the dashboard
endpoint.
- news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers,
so the bot starts without them) + load-once, never-raising singleton.
- news_engine: _enrich_headline_scores mode switch (off/fallback/primary),
_apply_model_sentiment (gold head only: xau_score + XAU entry, never
USD; not-relevant shows 0.0; never touches _llm_sent_cache),
_audit_with_deepseek (archive-only labels, deepseek_audit_per_day).
- Owner decision: primary mode writes nothing to logs/newsdata.db except
audit copies — the model's output is not training data.
- app_config: news_model section (path/threads/mode/deepseek_audit_per_day)
+ NEWS_MODEL_MODE env override; validated at boot.
- news_archive: model rows carry teacher_model xau_transformer@<trained_at>.
- deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a
commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers.
- test_all.py Module 38 (15 tests, one against the real bundle). Suite 581.
- CLAUDE.md, spec status note, handoff marked implemented.
Books unaffected: news never enters signals; no signal file touched.
Delivery-side fixes from the 2026-09-08 three-day signal-log + telegram-log
review. Nothing in the signal core changes: the 3-year H1 and M15 books at
$0.65 were regenerated before/after and are identical on every trade column
(H1 253 / +70.02R / PF 1.98, M15 1092 / +151.90R / PF 1.38); only the
exported `warnings` text differs, on 50/253 H1 and 358/1092 M15 trades.
- signal_aggregator: step 7 moves into `_attach_core_setup`; a final pass
after the PB/CP archetypes (`_drop_stale_spread_gate_vetoes`) moves a core
generator's "SPREAD GATE: ... costs would eat the edge" veto to
`cleaned_warnings` whenever a later generator shipped a setup. The
2026-09-08 07:45 UTC A-grade alert carried that line beside a $26.49 TP1;
a fifth of H1 and a third of M15 A-grade alerts had been doing the same.
The prefix is now the `_SPREAD_GATE_PREFIX` constant shared by both
emitters and both cleanups.
- telegram_bot: alerts require a trade setup (`_is_alert_candidate`, shared
by the scan and breach paths; no-setup candidates log
`alert skipped: ... (gate reason)`). The 07:50 UTC B-grade SELL with no
setup had gone out under the same warning.
- telegram_bot: every delivered alert / breach alert / violent-move heads-up
logs an `... sent:` INFO line with the signal-log row id (only failures
were logged before, so the review had to reconstruct sends from panel
rows). Send fan-out is one `_broadcast` helper for both paths.
- telegram_bot: `telegram.log` rotates daily x 30 instead of 3-hourly x 240
(old `_HH` files on the live box need one manual delete).
- signal_log / CLAUDE.md: `cleaned_warnings_json` is no longer a CP-only
marker; docs say so.
- test_all: 468 tests (+2): stale-veto cleanup incl. the PB-shaped case and
the both-vetoed case, and the alert-candidate rule with its skip line; the
rotation test asserts MIDNIGHT / interval 86400 / 30 backups.
Claude-Session: https://claude.ai/code/session_018xY4NPPrx6NJoG6QWHzwpd
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
Verdict from the 3-week signal log (backtest/research/news_sentiment_value_2026-09-04.md):
news_sentiment is a trailing-24h echo of price (r +0.67 with the past day, +0.03 with
the next; most bullish 2 h after the 08-25 top, bearish 44 h later; most bearish at the
09-02 low, bullish 16 h later). Per-headline LLM labels are correct; the 12 h pool and
the averaging are what destroy the information. Both gating flags stay False.
- news_engine: _LLM_SUMMARY_SYSTEM rewritten for someone who has never traded (no
acronyms, two worked examples); summary payload newest-first with the platform's own
gold direction; reply cap 300 -> 400 chars.
- config: headline_max_age_seconds 43200 -> 14400; RSS items without a parsable
pubDate are dropped (they used to be stamped "now" and never aged out).
- news_engine.get_fundamental_context drops headlines whose every relevant score is
inside +-0.1 (_is_neutral_headline) before the per-currency mean; news_sentiment
from this date is a mean over directional <=4 h headlines (log comparisons split here).
- static: dashboard copy for the 4 h pool; news_summary.php prompt gets the same style rule.
- analyze_signal_log: --news (under --stored-candles) prints sentiment lead/lag with
split halves, sentiment at the window high/low with flip lags, and outcome buckets by
direction x news x macro stance; LogRow gains news_sentiment/event_risk; _row_index shared.
- tests: 406 -> 414 (age window, undated drop, neutral filter, summary prompt/payload/cap,
--news helpers, load_rows round-trip); RSS fixtures now carry a pubDate.
- docs: user/beginner guides, CLAUDE.md, deploy README.
Claude-Session: https://claude.ai/code/session_0169BgFtWjp1SEAa5392Q9jH
- dashboard_publisher.py: standalone candle-close daemon that atomically
writes snapshot.json/news.json/dashboard.html to an Apache-served dir
(DASHBOARD_OUT_DIR). Fully independent of telegram_bot/oanda_daemon —
local confirmation map + boundary math, no imports either way, no new
deps, no open port. Due-TF classification from the RAW wake time.
- static/dashboard.html: self-contained dark page — hero rail, four TF
cards (score bar, 3x3 indicator grid, MTF, setup, reasons), DeepSeek
market context, week economic calendar, scored newswire. Clock offset
from the HTTP Date header (publish-time epoch is stale between closes).
- news_engine.get_week_events(): additive read-only accessor over the
cached FF week; no signal path calls it.
- deploy/: systemd unit + live-server (LAMP/SELinux) checklist.
- test_all.py: Module 20 (9 tests) + get_week_events test; 296 green.
Claude-Session: https://claude.ai/code/session_01Wc9ZoqKLYBT2QJcpRWKoGm
Harwinder Sekhon