1. A backfill pass silently wiped teacher identity from live rows.
The provenance compare is `>=`, and backfill rows are llm-scored but
carry no teacher_model/teacher_prompt_sha1 (neither source DB recorded
one), so a bare assignment blanked what the live hook had stored.
Re-running the backfill — e.g. after importing the live server's
signals.db, which is the next planned step — would have erased it from
every live row. COALESCE now keeps a stored identity when the incoming
one is NULL, while a genuinely newer teacher still replaces it.
Timestamps clamp for the same reason: backfill stamps historical
values, so first_seen only moves earlier and last_seen only later.
2. A retried batch could be applied twice. executemany does not roll
itself back, so a batch that half-landed and then hit a lock was
re-applied whole by _with_lock_retry, double-counting seen_count on
the rows that already landed. Writes now run in an explicit BEGIN
IMMEDIATE with a rollback, which also takes the write lock up front so
contention fails before any row is applied.
3. The archive hook could break a news fetch. archive_headlines swallows
its own failures, but its ARGUMENTS are evaluated first, so a raising
get_llm_client() escaped that guard. Guarded at the call site too:
archiving is a side effect and must never cost us the headlines we
just fetched.
Both 1 and 2 were verified to reproduce against 301bf90 before fixing
(teacher_model -> None; seen_count 3 instead of 2 on the rows that
landed pre-failure) and are now regression tests.
Also fixed two weak tests of my own: the disabled-by-config test never
pointed NEWS_ARCHIVE_PATH at its temp dir, so its assertion passed
regardless of behaviour and the write went to the real archive; and the
neutral-headline test asserted the headline was archived without showing
it is dropped downstream, which is the entire reason the hook sits where
it does. Positional binding of the provenance rank is now asserted rather
than trusted, since a miscount would shift every parameter silently.
483 -> 486 tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bzv4ETVapFNauFDQht7QtA
Headlines only ever survived as a side effect of signal logging, and
logs/signals.db was rotated around 2026-09-04 — destroying the 3-week,
4,054-headline corpus news_sentiment_value_2026-09-04.md was written on.
logs/ is gitignored, so git could not recover it and nobody noticed for
four days. There is no way to backfill a fetch window once it passes.
news_archive.py writes to its OWN database (logs/newsdata.db) so a
signal-log rotation cannot take the news with it. One row per distinct
headline (sha256 key) with the text, description, link, publisher,
publish time, the score, and — new — which scorer produced that score.
Provenance is the load-bearing part. DeepSeek scores 78/78 on the
2026-08-06 golden set; the keyword scorer it overrides scores 24/78
(31%), and that is what silently stands on no key, disabled, daily cap,
timeout, HTTP/parse failure, or the >25-new-headlines-per-cycle batch
overflow. Until now both landed in the same column, indistinguishable,
so a model trained on the mix would inherit the keyword scorer's errors.
NewsSentiment.score_source now records it, mirrored into signals.db via
signal_log._headline_to_dict, and teacher_model/teacher_prompt_sha1 keep
labels made under a changed prompt from being pooled with older ones.
Writes are one atomic UPSERT per row, upgrade-only: an llm score always
replaces a keyword one (the batch-overflow tail gets labelled on the
next window in a long-running process — verified 13/13), a keyword score
never overwrites an llm one, and re-seeing a headline always counts. A
read-then-write pair lost increments when the bot, the dashboard
publisher and the tests first saw a headline together, and the WAL
switch itself needed _with_lock_retry; both are regression-tested with
three concurrent writers.
The hook sits at the end of _fetch_news_sentiment_raw, BEFORE
_is_neutral_headline runs in get_fundamental_context. Archiving
downstream of that ±0.1 filter is why signals.db holds almost no neutral
headlines and would have taught a trained scorer that "neutral" barely
exists.
scripts/backfill_news_archive.py seeds it from both DB shapes on disk,
read-only and idempotent, recomputing provenance by comparing each
stored score against the keyword scorer (neither source recorded it):
1,016 distinct headlines, 846 DeepSeek-labelled, growing ~1,400/week.
Nothing in the signal core changed. Both news gating flags stay False,
no gate reads any of this, and the backtester never builds a NewsEngine
so replays never archive (test-locked). 468 -> 483 tests, including a
new guard that AST-walks live code for sklearn/scipy/lightgbm imports —
that check did not exist before.
Do not train yet: ~800 labels gives a holdout too small to grade
(SE +/-0.032). Target is ~4,000 deduplicated headlines over >=3 weeks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bzv4ETVapFNauFDQht7QtA
Delivery-side fixes from the 2026-09-08 three-day signal-log + telegram-log
review. Nothing in the signal core changes: the 3-year H1 and M15 books at
$0.65 were regenerated before/after and are identical on every trade column
(H1 253 / +70.02R / PF 1.98, M15 1092 / +151.90R / PF 1.38); only the
exported `warnings` text differs, on 50/253 H1 and 358/1092 M15 trades.
- signal_aggregator: step 7 moves into `_attach_core_setup`; a final pass
after the PB/CP archetypes (`_drop_stale_spread_gate_vetoes`) moves a core
generator's "SPREAD GATE: ... costs would eat the edge" veto to
`cleaned_warnings` whenever a later generator shipped a setup. The
2026-09-08 07:45 UTC A-grade alert carried that line beside a $26.49 TP1;
a fifth of H1 and a third of M15 A-grade alerts had been doing the same.
The prefix is now the `_SPREAD_GATE_PREFIX` constant shared by both
emitters and both cleanups.
- telegram_bot: alerts require a trade setup (`_is_alert_candidate`, shared
by the scan and breach paths; no-setup candidates log
`alert skipped: ... (gate reason)`). The 07:50 UTC B-grade SELL with no
setup had gone out under the same warning.
- telegram_bot: every delivered alert / breach alert / violent-move heads-up
logs an `... sent:` INFO line with the signal-log row id (only failures
were logged before, so the review had to reconstruct sends from panel
rows). Send fan-out is one `_broadcast` helper for both paths.
- telegram_bot: `telegram.log` rotates daily x 30 instead of 3-hourly x 240
(old `_HH` files on the live box need one manual delete).
- signal_log / CLAUDE.md: `cleaned_warnings_json` is no longer a CP-only
marker; docs say so.
- test_all: 468 tests (+2): stale-veto cleanup incl. the PB-shaped case and
the both-vetoed case, and the alert-candidate rule with its skip line; the
rotation test asserts MIDNIGHT / interval 86400 / 30 backups.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018xY4NPPrx6NJoG6QWHzwpd
2026-09-07 signal-log inspection follow-up — three fixes that do not depend on sample size.
- providers/base.py: the signal-side 10σ outlier filter is now a bad-print filter. A >10σ
close-to-close bar is dropped only when the NEXT close snaps back ≥80% of the way
(OUTLIER_SIGMA / OUTLIER_REVERT_FRACTION); persisting moves and the newest bar are kept.
The unconditional rule deleted ~36 real M5 news bars/yr in the backtester (whole-frame σ)
and a different ~24/yr live (fetch-window σ) — e.g. the 2026-09-04 12:30 UTC NFP bar
vanished from M5+M15 and the 12:35 scan was a byte-identical repeat of 12:30. 3-yr Oanda
drops are now M5 2 / M15 1 / H1 0.
- providers/oanda.py: transient 401/5xx/network faults retry along RETRY_DELAYS_S=(2,5,10)s
under RETRY_BUDGET_S=45s. 11 boundary scans in two days had lost their primary frame at
xx:00:22–:26 UTC; one 2 s retry could not outlast the top-of-hour blip.
- signal_aggregator/_log_signal: a scan whose primary fetch failed logs scan_kind='failed'
(price 0, no signal); analyze_signal_log.parity_report skips it like 'blocked'.
- test_all.py: 466 tests (+6): retry ladder walks all delays, 5xx blip recovers on the third
attempt, retry budget stops the ladder, persisting/newest/partially-retraced 10σ bars kept,
failed-scan row kind, parity skip.
- CLAUDE.md: cleaning rule, retry ladder, failed scan kind, dated current-state note.
Book impact at $0.65 (rule 1 still passes everywhere, both halves positive):
12mo M5 557/+112.21/PF1.56 → 560/+108.81/1.54 M15 397/+73.93/1.53 → 398/+74.61/1.53 H1 identical
3yr M5 857/+161.3/1.53 → 870/+153.6/1.50 M15 1086/+148.7/1.37 → 1092/+151.9/1.38
H1 255/+70.5/1.98 → 253/+70.0/1.98 H4 113/+22.7/1.63 → 114/+21.7/1.59
Only trades on the previously-deleted event bars move; the pinned M5/M15 CSVs are no longer
byte-reproducible and should be regenerated at the next re-pin.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDnBxJ2y8GRUM6ZNFJs1hZ
- providers: _to_oanda_instrument / _to_tiingo_ticker / _to_td_symbol no longer
guess a wire symbol for pairs that are not registered — the registries are
the only source, so an unknown symbol raises the provider's DataIntegrityError
instead of silently fetching something else. Every scanned symbol is validated
against PAIR_INFO at import, so nothing reachable changes.
- main / telegram_bot _fmt_price: the sub-$10 five-decimal branch could never
fire for gold.
- news_engine._parse_pair: BASE/QUOTE only; the Yahoo "=X" fallback was
unreachable (its test goes with it).
- dashboard_publisher.event_to_dict: drop minutes_until (frozen at publish
time, misleading) and is_high_impact — dashboard.js never read either.
- static/dashboard.html: unused id="cal-table".
Tried and reverted: trimming the LogRow fields nothing in analyze_signal_log
reads — the tests read them to verify the loader round-trips those DB columns,
so they stay.
Suite: 460 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
- SignalAggregator.pb_funnel / cp_funnel and the 19 _reject() ticks inside
_try_m5_pullback / _try_chart_pattern: class-level rejection counters nothing
in the repo reads. Every gate keeps its early return; only the counter bump
before it is gone.
- _TP_QUALITY_SCALE: the "B"/"C" entries and the 0.75 default could never be
selected — the XAU ATR generator is only called for A+/A. The lookup is now
a direct index so a violated invariant fails loudly instead of scaling silently.
- engine._classify_regime no longer takes the unused DataFrame argument, and the
structure S/R branch tests the 5-minute frame only (M1 is never a primary).
Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
Owner directive 2026-09-06: Dukascopy and similar research material is irrelevant to
the running system; keep only code and files with a role in system functioning.
Deleted (all in git history before this commit):
- backtest/research/ tooling and data: book_report, book_diff, burst_map,
burst_precursors, burst_indicator_map, open_trade_bursts, calendar_impact,
watch_rule_eval, sizing_model, commit_ladder, llm_news_eval + llm_news_golden.csv,
ff_calendar_scraper, duka_spread + duka_spread_by_hour.csv (the Dukascopy spread
study), feed_divergence_oanda_vs_tiingo.csv, the panel dry-test JSONL, __init__.py
- scripts/feed_divergence_study.py, scripts/signal_parity_study.py (completed studies;
their reports keep the results)
- docs/architecture.png + architecture_diagram.py (developer illustration)
- docs/data_source_change_test_plan.md (Phase 4 shadow scan stays noted in CLAUDE.md)
Kept: the campaign .md reports in backtest/research/ as the decision ledger CLAUDE.md
cites (rule 1 evidence); scripts/shadow_scan.py (pending operational step);
backtest/diagnose_filters.py (test-locked backtester diagnostic).
Local, untracked material removed on this box: .superpowers/, walkforward/, baseline/,
the commit-ladder and LLM-eval result folders, the Dukascopy boundary CSV; the
sizing_2026-09 folder is pruned to the four 3-year baseline books CLAUDE.md names.
.gitignore no longer lists the removed research outputs; the backtest instructions
and CLAUDE.md no longer point at deleted tools.
Suite: 461 passed, 0 failed (nothing imported the removed files).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
- test count 461 / 31 modules
- main.py --timeframe accepts M5/M15/H1/H4 and takes confirmation TFs from the
pair factory's map
- point the dual-cost note at the Dukascopy study that produced the $0.65
figure (duka_spread.py is evidence, not dead code)
- current-state note listing what the cleanup removed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
Every live reader of a pip value goes through PAIR_INFO[...]["pip_value"]
(signal_aggregator, gpu_backtest, diagnose_filters) or TradeSetup.pip_value;
no code read the PlatformConfig field. The three PlatformConfig(...)
constructions that passed it (gpu_backtest x2, diagnose_filters) and the
inline construction in the factory-equivalence test drop the kwarg. Neither
config fingerprint hashes the field, so logged fingerprints are unchanged.
The removed-field test now also asserts pip_value stays gone.
Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
- analyze_pair(context=...): no caller ever supplied it (main, telegram_bot,
pair_signals, tests), so the value was always None; the parameter and the
pass-through to analyze_from_dataframes go (analyze_from_dataframes keeps
its own context parameter).
- The three specific "SELL FILTER: quality|volatility|regime" stale-prefix
entries were shadowed by the general "SELL FILTER: " entry in the same
startswith() tuple; every emitter carries the general prefix.
- chart_patterns: the detectors never read the Open series, so the `o`
parameter is dropped from the three detectors, the PATTERN_DETECTORS
lambdas and the dispatcher.
- Corrected the pb_funnel/cp_funnel comment: nothing reads those counters
(the "read by backtest tooling" claim was stale); they stay as cheap
diagnostics.
Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
result._kc_squeeze / result._kc_squeeze_released were assigned by both the live
TechnicalEngine and the backtester's GPUTechnicalEngine and read by nothing:
the only four occurrences in the repo were the assignments. The squeeze state
still feeds the Keltner signal text where it is computed. Removed from both
engines in one commit (rule 4).
Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
main.py kept a private confirmation table and passed it into the pair factory,
which honours any non-None list. For M5 that table said [M1, M15] while the
factory, telegram_bot.TF_CONFIRMATIONS and dashboard_publisher.CONFIRMATIONS
all use the validated [M15, H1] - so `main.py --timeframe M5` scanned with an
unvalidated pairing (and an extra M1 fetch) that no other surface used.
- build_config now returns PAIR_PRODUCERS[pair](tf, None): the factory map is
the single source of truth. The unreachable bare-PlatformConfig fallback is
gone here and in telegram_bot._run_signal_sync (every symbol that reaches
either is registered).
- --timeframe offers the four validated primaries (M5, M15, H1, H4). M30 and
D1 were never validated and silently fell through to the generic [M15, H4]
map; they are removed from the CLI.
- The "all valid timeframes" test now asserts each CLI timeframe's
confirmation list equals pair_signals.xauusd._CONFIRMATIONS - the check
that would have caught this.
CLI behaviour change: M5 confirmations [M1, M15] -> [M15, H1]; M30/D1 rejected.
Bot and dashboard scans are unchanged.
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
- telegram_bot._format_scan_summary: multi-pair "auto-scan summary" leftover;
production sends per-report format_signal_message. Its only caller was a test.
- telegram_bot.scan_keyboard: thin wrapper only a test called; main_menu_keyboard
already builds from _scan_button_rows directly.
- telegram_bot.AUTOSCAN_CONFIRMATIONS was byte-identical to TF_CONFIRMATIONS;
the candle loop now reads the one map (app_config comment updated).
- SignalLogger.news_for: one-line wrapper over the module-level read_news that
every real reader uses; the test now calls read_news like its neighbours.
- pair_signals.get_supported_pairs: PAIR_PRODUCERS is the registry; the test
lists it directly.
Suite: 461 passed, 0 failed (two helper-only tests removed).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
Zero callers anywhere (tests included), verified by grep/AST:
- dashboard_publisher.DashboardPublisher.publish_all (run_cycle inlines both publishes)
- signal_log.SignalLogger.advance_with_bars (the backfill script uses
backfill_seeds / resolve_from_store / reseed_unresolvable) and the never-read
SignalLogger.path property plus its write-only self._path
- llm_client.reset_llm_client (tests reset the singleton by assignment)
- market_outlook._last_run_ts, residue of the old "now - last < 3600" dedup;
_last_run_hour (the UTC hour bucket) is the live key
- news_engine FundamentalContext.event_impact_on_pairs: written once, read by
nothing; its only feeder NewsEngine._get_affected_pairs and that helper's
unit test go with it
- position_sizing.LIVE_DEPTH_BARS (referenced only by its own docstring, which
now states the depths inline), the unused Iterable import and the unused
atr_ok parameter of _fill_geometry
- analyze_signal_log: unused `field` import and the unused `rows` parameter of
persona_scorecard / news_outcome_buckets (both read the DB directly)
The two backfill/label scripts' --db help text now names the config.json
setting it actually falls back to.
Suite: 463 passed, 0 failed (one test removed with its subject).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
- backtest/diagnose_filters: unused BacktestConfig import
- pair_signals/__init__: unused PlatformConfig import
- scripts/shadow_scan: unused pandas import
- research burst_precursors / calendar_impact / duka_spread: unused sys / COST / struct
- test_all: MagicMock, StringIO, Colors and two function-local imports that
nothing in the test bodies used
- telegram_bot: ViolentWatch is only a type hint -> imported under TYPE_CHECKING
compare_oanda_tiingo.py and oanda_td_divergence.py are left alone here because
the hygiene commit deletes them.
Suite: 464 passed, 0 failed (unchanged).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
The candle provider needs only oanda.api_token + base_url; account_id belonged to the
auto-execution daemon removed 2026-09-03 (31002cb). Removed from config.json.example and
the unused oanda_account_id() accessor deleted; the key stays in _KNOWN_OANDA so existing
config.json files keep loading. CLAUDE.md env-var list updated. Test locks all three.
Suite: 464 passed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
- test count no longer hardcoded (463 / 31 modules as of 2026-09-06)
- main.py --outlook, backtest --set / --export-features listed
- delivery surfaces include the web dashboard; SignalReport consumers include the publisher
- scikit-learn / lightgbm are research-only venv installs, never requirements
- rule 1: owner preference (2026-09-05) that backtests go back at most 3 years, with the
3-year baseline books and where they live
- "Current state notes" header un-dated; the 2026-08-14 canonical-book line marked superseded
by the 2026-09-03 book (M15 reference unchanged since 2026-08-04)
- docs: PDF builds noted as behind the HTML guides
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
New additive columns on every logged scan: bar_ts (primary closed-bar label, indexed with
timeframe), engine_fingerprint (hash of the scoring config incl. the regime weight table in
force - SignalAggregator.engine_config_fingerprint, twin of the gate fingerprint), code_version
(short git SHA), penalty_tags_json / penalty_total / momentum_fading / exhaustion_count, and
post-hoc watch_level / watch_kinds / watch_line written by violent_watch.attach_watch via
signal_log.update_watch. Each indicators_json entry now also carries eff_weight (base weight x
regime multiplier, the weight the composite actually used) and wscore.
engine: IndicatorSignal.effective_weight / weighted_score are stamped inside _compute_composite
(record-keeping only); regime_weight_table(config) is the single source of truth for the
multiplier table (override honoured). Composite output byte-identical (120-report hash proof).
New table scan_outcomes: model-free forward moves per scan (fwd_1h/4h/24h in $ and ATRs,
mfe/mae over 24h, burst_1h, bars_used, complete) measured on the stored candle stream with the
advisory model's continuity rule; labelled opportunistically at the end of log() (time-boxed,
throttled, idempotent) - no daemon; scripts/label_scan_outcomes.py drives the backlog
(--dry-run is read-only). analyze_signal_log --scan-outcomes: forward move + hit rate by
TF x signal/grade with a NEUTRAL baseline and burst rate by watch level. Readers dedup by the
first row per (timeframe, bar_ts).
Module 31 (10 tests). Suite: 463 passed. Backtester untouched (test-locked).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
New violent_watch.py assesses each scan from the cached Forex Factory week events, the
UTC clock and the report's own indicator states, and stamps SignalReport.watch_line
post-hoc (telegram alerts, CLI, dashboard). A standalone heads-up goes to the owner on
the fastest scanned timeframe when the level is "hot", once per UTC hour per trigger
set, capped per day. Config block `violent_watch` (enabled False, alert_line, headsup,
max_headsup_per_day, headsup_min_level); env VIOLENT_WATCH_ENABLED.
Rule chosen from a 3-year bar-by-bar sweep (backtest/research/watch_rule_eval.py,
recorded in sizing_campaign_2026-09.md): hot = high-impact USD release within 30 min
(before or after) OR a hot clock slot (weekday 12-14 UTC, 01 UTC, Sunday 22-23 UTC)
with at least one price-extreme reading (outside Bollinger bands, RSI <30/>=70, outside
Keltner); ~8.6% of bars, next-bar odds 3-5x normal, ~4 hot hours/day. watch = clock
slot alone or two price-extreme readings (~2x). Context readings (ADX rising, extreme
volatility, transitioning, MACD acceleration) are phrases only - letting them raise the
level flagged 27% of bars at ~2x and would have sent ~13 heads-ups/day (rejected).
Direction sentence: DOWN when bearish readings dominate (70-78% on both TFs), UP on M5
only. Never a gate or exit: every open-trade protection tested lost money.
Test-locked out of analyze_from_dataframes / engine / backtester. Module 30 (18 tests).
Suite: 453 passed. Guide PDFs not rebuilt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
Walk-forward "second opinion" stake multiplier (meta-labelling on 65 decision-time
features, 3yr Oanda books @ $0.65) FAILED the pre-registered rule on M5 (clean null)
and M15 (lift inside the label-shuffle null, explained by BUY-vs-SELL). Nothing that
changes numbers ships; the 12-month M5 book re-run is byte-identical to the pinned one.
Behaviour-neutral plumbing:
- position_sizing.build_features (65 features; multiplier path returns 1.0, no artifact)
- SignalReport.penalty_tags / penalty_total / momentum_fading (record-keeping)
- TradeSetup.stake_multiplier = None (reserved)
- gpu_backtest --export-features (default export stays the pinned 63 columns; extras
forced to the tail even when the first trade lacks an indicator)
Research tools (backtest/research): book_report, burst_map, burst_precursors,
open_trade_bursts, calendar_impact, burst_indicator_map, sizing_model, watch_rule_eval.
Findings recorded in backtest/research/sizing_campaign_2026-09.md: M5 stretch measures
are positive in both halves; every open-trade protection against violent bars loses on
both TFs; high-impact USD releases cause 20-31% of violent bars but trades held through
them do better; violent bars come when price is already at an extreme, not from calm.
Module 29 tests (13). Suite: 435 passed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
None of these touch the signal path. The 12-month M15 book
(2024-03-31..2025-03-30, $0.65, cached Oanda bars) is byte-identical
before and after (md5 c59d3e7e685c3c880b614640adfea74b, 375 trades),
and signal/indicator logging to logs/signals.db is unchanged — verified
end to end: per-TF indicators for all 3 TFs, 14 candles each, MTF and
reasons all still persisted.
agent_panel: the >=2-take quorum guards the CHAIR's synthesizer spend,
but ran BEFORE the macro branch — so the shipped default
(verdict_source="macro", which never calls the synthesizer) silently
dropped a good macro verdict whenever the other three personas were
slow. Quorum moved below the macro path.
analyze_signal_log: _forward_return clamped to the last available bar
when the horizon had not elapsed, reporting a near-zero "forward move"
for every row logged within `hours` of the end of the data — on every
run, blended into the hit-rate/mean-move aggregate. Now returns None.
analyze_signal_log: persona_scorecard iterated raw rows through an index
that maps EVERY member row of an event, so a re-logged scan appended
the same event R twice. One vote per event, matching
news_outcome_buckets. Measured inflation on the live DB: 1.03x.
signal_log: dedup required a non-negative logged_at gap, but logged_at is
stamped before the row wins the write lock (busy_timeout 5 s), so two
processes logging one setup at a boundary can invert timestamp order
against id order and seed a second event. Now abs(gap); same in
backfill_seeds. Zero occurrences in the current 366 setup rows.
advisory_outcome: the catch-all turned real evaluator faults into the
same UNRESOLVABLE used for legitimate weekend gaps, with nothing in the
log. Still fail-safe, now logged.
llm_client: only the "cap" warn-once flag was ever reset, so in a daemon
the second and every later network/http/parse outage logged at DEBUG
only. Cleared on UTC-day roll and re-armed by a recovered call.
dashboard.js: renderCards() redraws the selector for every timeframe but
was gated on the SELECTED one changing, leaving the other buttons stale
(up to 4 h on H4) while renderHero's rail already showed the new value.
news_summary.php: atomic_write treated a short write (disk full) as
success, publishing a truncated body the cache-first branch then served
forever; and the daily cap was charged before the no-key check and
before the stampede lock re-checked the cache, so a click burst could
exhaust the 150/day budget with zero LLM calls.
migrate_signal_log_news: logged_at is ISO with 'T' while SQLite's
datetime() uses a space, so string-comparing them put the whole cutoff
calendar day on the wrong side and it never pruned. NOTE: this now
deletes one day more than before — it only runs when invoked by hand.
Tests 414 -> 422 (new Module 28, one per fix). The existing
"<2 valid takes" test was retargeted at the chair — the case it actually
protects — rather than deleted, with new coverage for macro-alone.
Not fixed, recorded for the owner:
- Gate 6c's boost condition is textually identical to the C-grade
definition, so every C in a "transitioning" regime is promoted to B
before gate 6e reads it, making sell_min_quality's exclusion of "C" a
no-op on M15/H4 (H1 and M5 are immune by other means). The backtester
runs the same code, so every pinned book already includes this —
changing it is a rule-1 revalidation event.
- Safe-haven/geopolitical headlines score exactly 0.00 in the keyword
fallback and are now dropped entirely by the ±0.1 neutral filter,
contradicting the platform's own gold doctrine. Bites only when
DeepSeek is capped/disabled/failing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ah5EB1goi5inQmSd37Ne6a
Verdict from the 3-week signal log (backtest/research/news_sentiment_value_2026-09-04.md):
news_sentiment is a trailing-24h echo of price (r +0.67 with the past day, +0.03 with
the next; most bullish 2 h after the 08-25 top, bearish 44 h later; most bearish at the
09-02 low, bullish 16 h later). Per-headline LLM labels are correct; the 12 h pool and
the averaging are what destroy the information. Both gating flags stay False.
- news_engine: _LLM_SUMMARY_SYSTEM rewritten for someone who has never traded (no
acronyms, two worked examples); summary payload newest-first with the platform's own
gold direction; reply cap 300 -> 400 chars.
- config: headline_max_age_seconds 43200 -> 14400; RSS items without a parsable
pubDate are dropped (they used to be stamped "now" and never aged out).
- news_engine.get_fundamental_context drops headlines whose every relevant score is
inside +-0.1 (_is_neutral_headline) before the per-currency mean; news_sentiment
from this date is a mean over directional <=4 h headlines (log comparisons split here).
- static: dashboard copy for the 4 h pool; news_summary.php prompt gets the same style rule.
- analyze_signal_log: --news (under --stored-candles) prints sentiment lead/lag with
split halves, sentiment at the window high/low with flip lags, and outcome buckets by
direction x news x macro stance; LogRow gains news_sentiment/event_risk; _row_index shared.
- tests: 406 -> 414 (age window, undated drop, neutral filter, summary prompt/payload/cap,
--news helpers, load_rows round-trip); RSS fixtures now carry a pubDate.
- docs: user/beginner guides, CLAUDE.md, deploy README.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0169BgFtWjp1SEAa5392Q9jH
agent_panel.verdict_source defaults to "macro": the macro take's stance is the
verdict, the synthesizer call is skipped (4 LLM calls per panel instead of 5);
"chair" restores the four-voice consensus + OPPOSE guard. Verdict source is
cached per mode and logged (panel_verdict_source). 406 tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
Deletes oanda_daemon.py, oanda_executor.py, their 28 tests and the daemon-only
config keys (oanda.lot_size/quality_gate/max_hold_bars). The Oanda data
provider and credentials stay — it is the live candle feed. 402 tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
Add panel_verdict, panel_confidence, panel_rationale, panel_takes_json
columns to the signals table (additive, idempotent ALTER TABLE migration
for existing DBs). _report_row now reads the panel fields attach_panel
writes onto the SignalReport, so LLM panel verdicts stop evaporating on
process exit. Also documents the Oanda re-pin (round_trip_cost stays
$0.30 for cross-book comparability and the live TP gate; $0.65 is the
measured headline) and adds the feed-divergence / parity / shadow-scan
study tooling referenced by CLAUDE.md.
Page now uses Bootstrap 5.3 color modes (data-bs-theme), Bootstrap Icons,
and Bootstrap modals. Custom code shrinks to dashboard.css (app tokens,
score bar, signal accents) and dashboard.js (rendering/polling logic,
ported verbatim from the inline script). dashboard_publisher copies the
new css/js into the docroot; all 381 tests still green.
New signal_log.py logs each SignalReport to logs/signals.db via a
fail-safe hook in SignalAggregator.analyze_pair — the live-only entry
point used by the bot, daemon, CLI and dashboard. Backtester calls
analyze_from_dataframes directly, so live==backtest parity is preserved
(test-locked). Rows carry signal/grade/price/spread, MTF + trade-setup +
reasons/warnings JSON, and news-at-scan-time (calendar events + scored
headlines JSON via NewsEngine class-level caches).
Config: signal_log block (enabled, path; env SIGNAL_LOG_ENABLED/PATH),
source tagged per consumer (telegram/oanda_daemon/cli/dashboard).
381 tests pass (24 modules).
poll() now compares raw payload text and re-renders only what a changed
file can affect, preserving selection, disclosure, and scroll state. Hero
freshness reflects publish time (STALE warning past 15 min while market
open), and countdowns resolve their wrap interval from data-interval
instead of reverse-looking shared TF boundaries in snapshot.timeframes.
Prompt-only change (static constants; plain-language output contract
byte-identical). Ported from virattt/ai-hedge-fund's persona rubric:
- _TAKE_CONTRACT hard rules: reason only from the JSON given, never
invent numbers/events/levels; thin or missing evidence -> neutral
with confidence below 30 (deferring to role-specific rules).
- Anchored confidence bands in the contract and synthesizer, replacing
the unanchored "how sure you are".
- Per-stance support/oppose/neutral criteria for macro, technical, and
risk (risk's missing-setup->oppose folded into its oppose bullet).
- One plain-language support example beside each oppose example for
macro/technical/risk, fixing the all-oppose few-shot bias.
Contrarian's designed skeptic tilt untouched.
News-engine prompts untouched (pinned by 78/78 + 18/18 validations).
374 tests green; real-DeepSeek smoke: macro correctly answered
neutral/20 on empty news context, confidences spread 20-60 across the
bands, all notes <=25 words, no jargon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TyZx9iRJwfbPTPnCBqcwTR
Reference plan only - no code changes. Describes the BaseLLMClient
refactor, AnthropicClient on the official SDK, per-consumer routing via
a config.json "llm" block, tests, commit sequence, and effort estimate
for the day we route the agent panel to Claude while news stays on
DeepSeek.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
Owner request: panel notes and rationale must read like plain everyday
English for a non-trader. Style rule added to the shared take contract
and synthesizer prompt, all few-shot examples rewritten in the target
voice (examples carry the style), and the Telegram votes line now spells
out full stances ("macro: support") instead of "macro:sup".
Before: "BUY against Bollinger SELL and price above upper band; 4h
neutral, mixed MTF" → After: "The price is above its usual range, and
the bigger picture is not clearly rising, so buying now is risky."
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
main.py: --panel runs the agent panel on demand (force bypasses
enabled/surface/quality toggles; daily budget + LLM cap still apply),
prints the verdict + persona takes, and report_to_dict carries the four
panel keys — which auto-propagates them into the dashboard's
snapshot.json via report_to_webapp_dict. dashboard_publisher attaches
in scan_tf gated by agent_panel.dashboard (default off); dashboard.html
renders an "AI panel (advisory)" section in the TF detail dialog.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
Auto-scan and level-breach alert paths attach the panel AFTER the
quality gate and dedup decision (no LLM spend on suppressed alerts),
via run_in_executor so the event loop stays free. Alert message gains a
two-line 🤖 AI Panel block (verdict + confidence + rationale, persona
votes); /status shows the daily panel budget when the feature is on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
Harwinder Sekhon