Commit Graph

14 Commits

Author SHA1 Message Date
Harwinder Sekhon 3afc62c847 alerts: require a trade setup, log every send, drop stale spread-gate vetoes, daily log rotation
Delivery-side fixes from the 2026-09-08 three-day signal-log + telegram-log
review. Nothing in the signal core changes: the 3-year H1 and M15 books at
$0.65 were regenerated before/after and are identical on every trade column
(H1 253 / +70.02R / PF 1.98, M15 1092 / +151.90R / PF 1.38); only the
exported `warnings` text differs, on 50/253 H1 and 358/1092 M15 trades.

- signal_aggregator: step 7 moves into `_attach_core_setup`; a final pass
  after the PB/CP archetypes (`_drop_stale_spread_gate_vetoes`) moves a core
  generator's "SPREAD GATE: ... costs would eat the edge" veto to
  `cleaned_warnings` whenever a later generator shipped a setup. The
  2026-09-08 07:45 UTC A-grade alert carried that line beside a $26.49 TP1;
  a fifth of H1 and a third of M15 A-grade alerts had been doing the same.
  The prefix is now the `_SPREAD_GATE_PREFIX` constant shared by both
  emitters and both cleanups.
- telegram_bot: alerts require a trade setup (`_is_alert_candidate`, shared
  by the scan and breach paths; no-setup candidates log
  `alert skipped: ... (gate reason)`). The 07:50 UTC B-grade SELL with no
  setup had gone out under the same warning.
- telegram_bot: every delivered alert / breach alert / violent-move heads-up
  logs an `... sent:` INFO line with the signal-log row id (only failures
  were logged before, so the review had to reconstruct sends from panel
  rows). Send fan-out is one `_broadcast` helper for both paths.
- telegram_bot: `telegram.log` rotates daily x 30 instead of 3-hourly x 240
  (old `_HH` files on the live box need one manual delete).
- signal_log / CLAUDE.md: `cleaned_warnings_json` is no longer a CP-only
  marker; docs say so.
- test_all: 468 tests (+2): stale-veto cleanup incl. the PB-shaped case and
  the both-vetoed case, and the alert-candidate rule with its skip line; the
  rotation test asserts MIDNIGHT / interval 86400 / 30 backups.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018xY4NPPrx6NJoG6QWHzwpd
2026-09-08 11:13:52 -04:00
Harwinder Sekhon de36b104d9 providers: keep real news bars (snap-back bad-print rule), Oanda retry ladder, scan_kind=failed
2026-09-07 signal-log inspection follow-up — three fixes that do not depend on sample size.

- providers/base.py: the signal-side 10σ outlier filter is now a bad-print filter. A >10σ
  close-to-close bar is dropped only when the NEXT close snaps back ≥80% of the way
  (OUTLIER_SIGMA / OUTLIER_REVERT_FRACTION); persisting moves and the newest bar are kept.
  The unconditional rule deleted ~36 real M5 news bars/yr in the backtester (whole-frame σ)
  and a different ~24/yr live (fetch-window σ) — e.g. the 2026-09-04 12:30 UTC NFP bar
  vanished from M5+M15 and the 12:35 scan was a byte-identical repeat of 12:30. 3-yr Oanda
  drops are now M5 2 / M15 1 / H1 0.
- providers/oanda.py: transient 401/5xx/network faults retry along RETRY_DELAYS_S=(2,5,10)s
  under RETRY_BUDGET_S=45s. 11 boundary scans in two days had lost their primary frame at
  xx:00:22–:26 UTC; one 2 s retry could not outlast the top-of-hour blip.
- signal_aggregator/_log_signal: a scan whose primary fetch failed logs scan_kind='failed'
  (price 0, no signal); analyze_signal_log.parity_report skips it like 'blocked'.
- test_all.py: 466 tests (+6): retry ladder walks all delays, 5xx blip recovers on the third
  attempt, retry budget stops the ladder, persisting/newest/partially-retraced 10σ bars kept,
  failed-scan row kind, parity skip.
- CLAUDE.md: cleaning rule, retry ladder, failed scan kind, dated current-state note.

Book impact at $0.65 (rule 1 still passes everywhere, both halves positive):
  12mo  M5 557/+112.21/PF1.56 → 560/+108.81/1.54   M15 397/+73.93/1.53 → 398/+74.61/1.53   H1 identical
  3yr   M5 857/+161.3/1.53 → 870/+153.6/1.50   M15 1086/+148.7/1.37 → 1092/+151.9/1.38
        H1 255/+70.5/1.98 → 253/+70.0/1.98   H4 113/+22.7/1.63 → 114/+21.7/1.59
Only trades on the previously-deleted event bars move; the pinned M5/M15 CSVs are no longer
byte-reproducible and should be regenerated at the next re-pin.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NDnBxJ2y8GRUM6ZNFJs1hZ
2026-09-07 11:07:29 -04:00
Harwinder Sekhon fd1e136e24 chore: remove test-only helpers and the duplicate confirmation map (no behaviour change)
- telegram_bot._format_scan_summary: multi-pair "auto-scan summary" leftover;
  production sends per-report format_signal_message. Its only caller was a test.
- telegram_bot.scan_keyboard: thin wrapper only a test called; main_menu_keyboard
  already builds from _scan_button_rows directly.
- telegram_bot.AUTOSCAN_CONFIRMATIONS was byte-identical to TF_CONFIRMATIONS;
  the candle loop now reads the one map (app_config comment updated).
- SignalLogger.news_for: one-line wrapper over the module-level read_news that
  every real reader uses; the test now calls read_news like its neighbours.
- pair_signals.get_supported_pairs: PAIR_PRODUCERS is the registry; the test
  lists it directly.

Suite: 461 passed, 0 failed (two helper-only tests removed).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:50:21 -04:00
Harwinder Sekhon 69056958a7 chore: remove dead functions and write-only state (no behaviour change)
Zero callers anywhere (tests included), verified by grep/AST:
- dashboard_publisher.DashboardPublisher.publish_all (run_cycle inlines both publishes)
- signal_log.SignalLogger.advance_with_bars (the backfill script uses
  backfill_seeds / resolve_from_store / reseed_unresolvable) and the never-read
  SignalLogger.path property plus its write-only self._path
- llm_client.reset_llm_client (tests reset the singleton by assignment)
- market_outlook._last_run_ts, residue of the old "now - last < 3600" dedup;
  _last_run_hour (the UTC hour bucket) is the live key
- news_engine FundamentalContext.event_impact_on_pairs: written once, read by
  nothing; its only feeder NewsEngine._get_affected_pairs and that helper's
  unit test go with it
- position_sizing.LIVE_DEPTH_BARS (referenced only by its own docstring, which
  now states the depths inline), the unused Iterable import and the unused
  atr_ok parameter of _fill_geometry
- analyze_signal_log: unused `field` import and the unused `rows` parameter of
  persona_scorecard / news_outcome_buckets (both read the DB directly)

The two backfill/label scripts' --db help text now names the config.json
setting it actually falls back to.

Suite: 463 passed, 0 failed (one test removed with its subject).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:48:24 -04:00
Harwinder Sekhon e08133e2b5 signal_log: weight-research data points, scan_outcomes forward labels, watch provenance
New additive columns on every logged scan: bar_ts (primary closed-bar label, indexed with
timeframe), engine_fingerprint (hash of the scoring config incl. the regime weight table in
force - SignalAggregator.engine_config_fingerprint, twin of the gate fingerprint), code_version
(short git SHA), penalty_tags_json / penalty_total / momentum_fading / exhaustion_count, and
post-hoc watch_level / watch_kinds / watch_line written by violent_watch.attach_watch via
signal_log.update_watch. Each indicators_json entry now also carries eff_weight (base weight x
regime multiplier, the weight the composite actually used) and wscore.

engine: IndicatorSignal.effective_weight / weighted_score are stamped inside _compute_composite
(record-keeping only); regime_weight_table(config) is the single source of truth for the
multiplier table (override honoured). Composite output byte-identical (120-report hash proof).

New table scan_outcomes: model-free forward moves per scan (fwd_1h/4h/24h in $ and ATRs,
mfe/mae over 24h, burst_1h, bars_used, complete) measured on the stored candle stream with the
advisory model's continuity rule; labelled opportunistically at the end of log() (time-boxed,
throttled, idempotent) - no daemon; scripts/label_scan_outcomes.py drives the backlog
(--dry-run is read-only). analyze_signal_log --scan-outcomes: forward move + hit rate by
TF x signal/grade with a NEUTRAL baseline and burst rate by watch level. Readers dedup by the
first row per (timeframe, bar_ts).

Module 31 (10 tests). Suite: 463 passed. Backtester untouched (test-locked).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:22:59 -04:00
Harwinder Sekhon 59f69400a8 audit: fix 9 defects from the 2026-09-04 hidden-bug sweep
None of these touch the signal path. The 12-month M15 book
(2024-03-31..2025-03-30, $0.65, cached Oanda bars) is byte-identical
before and after (md5 c59d3e7e685c3c880b614640adfea74b, 375 trades),
and signal/indicator logging to logs/signals.db is unchanged — verified
end to end: per-TF indicators for all 3 TFs, 14 candles each, MTF and
reasons all still persisted.

agent_panel: the >=2-take quorum guards the CHAIR's synthesizer spend,
  but ran BEFORE the macro branch — so the shipped default
  (verdict_source="macro", which never calls the synthesizer) silently
  dropped a good macro verdict whenever the other three personas were
  slow. Quorum moved below the macro path.
analyze_signal_log: _forward_return clamped to the last available bar
  when the horizon had not elapsed, reporting a near-zero "forward move"
  for every row logged within `hours` of the end of the data — on every
  run, blended into the hit-rate/mean-move aggregate. Now returns None.
analyze_signal_log: persona_scorecard iterated raw rows through an index
  that maps EVERY member row of an event, so a re-logged scan appended
  the same event R twice. One vote per event, matching
  news_outcome_buckets. Measured inflation on the live DB: 1.03x.
signal_log: dedup required a non-negative logged_at gap, but logged_at is
  stamped before the row wins the write lock (busy_timeout 5 s), so two
  processes logging one setup at a boundary can invert timestamp order
  against id order and seed a second event. Now abs(gap); same in
  backfill_seeds. Zero occurrences in the current 366 setup rows.
advisory_outcome: the catch-all turned real evaluator faults into the
  same UNRESOLVABLE used for legitimate weekend gaps, with nothing in the
  log. Still fail-safe, now logged.
llm_client: only the "cap" warn-once flag was ever reset, so in a daemon
  the second and every later network/http/parse outage logged at DEBUG
  only. Cleared on UTC-day roll and re-armed by a recovered call.
dashboard.js: renderCards() redraws the selector for every timeframe but
  was gated on the SELECTED one changing, leaving the other buttons stale
  (up to 4 h on H4) while renderHero's rail already showed the new value.
news_summary.php: atomic_write treated a short write (disk full) as
  success, publishing a truncated body the cache-first branch then served
  forever; and the daily cap was charged before the no-key check and
  before the stampede lock re-checked the cache, so a click burst could
  exhaust the 150/day budget with zero LLM calls.
migrate_signal_log_news: logged_at is ISO with 'T' while SQLite's
  datetime() uses a space, so string-comparing them put the whole cutoff
  calendar day on the wrong side and it never pruned. NOTE: this now
  deletes one day more than before — it only runs when invoked by hand.

Tests 414 -> 422 (new Module 28, one per fix). The existing
"<2 valid takes" test was retargeted at the chair — the case it actually
protects — rather than deleted, with new coverage for macro-alone.

Not fixed, recorded for the owner:
- Gate 6c's boost condition is textually identical to the C-grade
  definition, so every C in a "transitioning" regime is promoted to B
  before gate 6e reads it, making sell_min_quality's exclusion of "C" a
  no-op on M15/H4 (H1 and M5 are immune by other means). The backtester
  runs the same code, so every pinned book already includes this —
  changing it is a rule-1 revalidation event.
- Safe-haven/geopolitical headlines score exactly 0.00 in the keyword
  fallback and are now dropped entirely by the ±0.1 neutral filter,
  contradicting the platform's own gold doctrine. Bites only when
  DeepSeek is capped/disabled/failing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ah5EB1goi5inQmSd37Ne6a
2026-09-04 20:19:08 -04:00
Harwinder Sekhon b8f903ad0e agent_panel: alerts use the macro persona's verdict; all 4 takes still logged
agent_panel.verdict_source defaults to "macro": the macro take's stance is the
verdict, the synthesizer call is skipped (4 LLM calls per panel instead of 5);
"chair" restores the four-voice consensus + OPPOSE guard. Verdict source is
cached per mode and logged (panel_verdict_source). 406 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 21:16:45 -04:00
Harwinder Sekhon ae6a6658f6 signal_log: outcomes for every setup, event dedup, interned news; fix over-scan, dead outlook, panel OPPOSE bias
- seed an advisory outcome on every setup row; same-setup re-logs within 60 min
  are DUPLICATE rows (advisory_event_id); resolver rebuilds bar streams across gaps
- headlines/events interned into news_snapshots (DB 667 MB -> 262 MB); readers
  use signals_with_news / read_news and mode=ro (immutable=1 missed WAL rows)
- post-gate truth columns: gate_reason, actionable, cleaned_warnings, scan_kind,
  config_fingerprint; SELL-hour gates fail closed
- telegram: breach rescan reuses this bar's report; grade dropped from alert
  fingerprint (upgrade-only re-alert)
- market_outlook: hour-bucket dedup (old rule skipped alternate hours), skip heartbeats
- agent_panel: macro-anchored OPPOSE guard (opposed trades were +30R over 3 weeks)
- analyze_signal_log --stored-candles/--parity; backfill + news migration scripts
- tests: Module 25 (analyze_signal_log) + 27 (observability); 429 total

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 20:47:33 -04:00
root 4dd0ca0b46 agent_panel: harden evidence and track outcomes 2026-08-23 12:57:05 -04:00
Harwinder Sekhon 0dbd94c03c signal_log: persist market snapshots 2026-08-18 15:24:37 -04:00
Harwinder Sekhon 2a2ccb13dd market_outlook: add hourly advisory brief 2026-08-18 13:52:18 -04:00
Harwinder Sekhon 7b60676a7a agent_panel: ATR-normalized candle + indicator-value enrichment, hardened prompts, verdict logging 2026-08-15 15:24:28 -04:00
Harwinder Sekhon 37ff5b383d signal_log: persist agent panel verdict + per-persona takes to SQLite
Add panel_verdict, panel_confidence, panel_rationale, panel_takes_json
columns to the signals table (additive, idempotent ALTER TABLE migration
for existing DBs). _report_row now reads the panel fields attach_panel
writes onto the SignalReport, so LLM panel verdicts stop evaporating on
process exit. Also documents the Oanda re-pin (round_trip_cost stays
$0.30 for cross-book comparability and the live TP gate; $0.65 is the
measured headline) and adds the feed-divergence / parity / shadow-scan
study tooling referenced by CLAUDE.md.
2026-08-14 20:41:54 -04:00
Harwinder Sekhon 4ffdd6bcb2 signal_log: persist every live scan (signal, grade, price, news) to SQLite
New signal_log.py logs each SignalReport to logs/signals.db via a
fail-safe hook in SignalAggregator.analyze_pair — the live-only entry
point used by the bot, daemon, CLI and dashboard. Backtester calls
analyze_from_dataframes directly, so live==backtest parity is preserved
(test-locked). Rows carry signal/grade/price/spread, MTF + trade-setup +
reasons/warnings JSON, and news-at-scan-time (calendar events + scored
headlines JSON via NewsEngine class-level caches).

Config: signal_log block (enabled, path; env SIGNAL_LOG_ENABLED/PATH),
source tagged per consumer (telegram/oanda_daemon/cli/dashboard).
381 tests pass (24 modules).
2026-08-13 10:24:49 -04:00