Commit Graph

193 Commits

Author SHA1 Message Date
Harwinder Sekhon 35f1eb17b8 chore: drop unreachable branches outside the signal core (no behaviour change)
- providers: _to_oanda_instrument / _to_tiingo_ticker / _to_td_symbol no longer
  guess a wire symbol for pairs that are not registered — the registries are
  the only source, so an unknown symbol raises the provider's DataIntegrityError
  instead of silently fetching something else. Every scanned symbol is validated
  against PAIR_INFO at import, so nothing reachable changes.
- main / telegram_bot _fmt_price: the sub-$10 five-decimal branch could never
  fire for gold.
- news_engine._parse_pair: BASE/QUOTE only; the Yahoo "=X" fallback was
  unreachable (its test goes with it).
- dashboard_publisher.event_to_dict: drop minutes_until (frozen at publish
  time, misleading) and is_high_impact — dashboard.js never read either.
- static/dashboard.html: unused id="cal-table".

Tried and reverted: trimming the LogRow fields nothing in analyze_signal_log
reads — the tests read them to verify the loader round-trips those DB columns,
so they stay.

Suite: 460 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:47:45 -04:00
Harwinder Sekhon c2f98938a3 signal_aggregator + engine: drop write-only counters and unreachable branches (no behaviour change)
- SignalAggregator.pb_funnel / cp_funnel and the 19 _reject() ticks inside
  _try_m5_pullback / _try_chart_pattern: class-level rejection counters nothing
  in the repo reads. Every gate keeps its early return; only the counter bump
  before it is gone.
- _TP_QUALITY_SCALE: the "B"/"C" entries and the 0.75 default could never be
  selected — the XAU ATR generator is only called for A+/A. The lookup is now
  a direct index so a violated invariant fails loudly instead of scaling silently.
- engine._classify_regime no longer takes the unused DataFrame argument, and the
  structure S/R branch tests the 5-minute frame only (M1 is never a primary).

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:46:33 -04:00
Harwinder Sekhon c004e3f30b chore: keep only what the system needs — drop research tooling, study scripts, diagram, plan
Owner directive 2026-09-06: Dukascopy and similar research material is irrelevant to
the running system; keep only code and files with a role in system functioning.

Deleted (all in git history before this commit):
- backtest/research/ tooling and data: book_report, book_diff, burst_map,
  burst_precursors, burst_indicator_map, open_trade_bursts, calendar_impact,
  watch_rule_eval, sizing_model, commit_ladder, llm_news_eval + llm_news_golden.csv,
  ff_calendar_scraper, duka_spread + duka_spread_by_hour.csv (the Dukascopy spread
  study), feed_divergence_oanda_vs_tiingo.csv, the panel dry-test JSONL, __init__.py
- scripts/feed_divergence_study.py, scripts/signal_parity_study.py (completed studies;
  their reports keep the results)
- docs/architecture.png + architecture_diagram.py (developer illustration)
- docs/data_source_change_test_plan.md (Phase 4 shadow scan stays noted in CLAUDE.md)

Kept: the campaign .md reports in backtest/research/ as the decision ledger CLAUDE.md
cites (rule 1 evidence); scripts/shadow_scan.py (pending operational step);
backtest/diagnose_filters.py (test-locked backtester diagnostic).

Local, untracked material removed on this box: .superpowers/, walkforward/, baseline/,
the commit-ladder and LLM-eval result folders, the Dukascopy boundary CSV; the
sizing_2026-09 folder is pruned to the four 3-year baseline books CLAUDE.md names.

.gitignore no longer lists the removed research outputs; the backtest instructions
and CLAUDE.md no longer point at deleted tools.

Suite: 461 passed, 0 failed (nothing imported the removed files).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:35:41 -04:00
Harwinder Sekhon de3f8a6ab8 CLAUDE.md: sync after the 2026-09-06 stale-code cleanup
- test count 461 / 31 modules
- main.py --timeframe accepts M5/M15/H1/H4 and takes confirmation TFs from the
  pair factory's map
- point the dual-cost note at the Dukascopy study that produced the $0.65
  figure (duka_spread.py is evidence, not dead code)
- current-state note listing what the cleanup removed

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:01:07 -04:00
Harwinder Sekhon c00ed4471b chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
  was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
  config.json migration that merged the same day; its plan sibling was already
  deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
  Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
  the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
  divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
  and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
  its .jsonl evidence stays

Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.

Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
  backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
  to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
  (the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
  actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
  4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
  analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
  says 4 requests by default; tiingo comment cites the NY-anchored weekend;
  Module 21 title no longer says "execution lifecycle"

Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:01:07 -04:00
Harwinder Sekhon 2502bcc396 config: remove the never-read PlatformConfig.pip_value (rule 8)
Every live reader of a pip value goes through PAIR_INFO[...]["pip_value"]
(signal_aggregator, gpu_backtest, diagnose_filters) or TradeSetup.pip_value;
no code read the PlatformConfig field. The three PlatformConfig(...)
constructions that passed it (gpu_backtest x2, diagnose_filters) and the
inline construction in the factory-equivalence test drop the kwarg. Neither
config fingerprint hashes the field, so logged fingerprints are unchanged.
The removed-field test now also asserts pip_value stays gone.

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:57:40 -04:00
Harwinder Sekhon 04bdc1737f signal_aggregator + chart_patterns: drop dead parameters and shadowed literals (no behaviour change)
- analyze_pair(context=...): no caller ever supplied it (main, telegram_bot,
  pair_signals, tests), so the value was always None; the parameter and the
  pass-through to analyze_from_dataframes go (analyze_from_dataframes keeps
  its own context parameter).
- The three specific "SELL FILTER: quality|volatility|regime" stale-prefix
  entries were shadowed by the general "SELL FILTER: " entry in the same
  startswith() tuple; every emitter carries the general prefix.
- chart_patterns: the detectors never read the Open series, so the `o`
  parameter is dropped from the three detectors, the PATTERN_DETECTORS
  lambdas and the dispatcher.
- Corrected the pb_funnel/cp_funnel comment: nothing reads those counters
  (the "read by backtest tooling" claim was stale); they stay as cheap
  diagnostics.

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:55:26 -04:00
Harwinder Sekhon 3cf3c60b89 engine + gpu_backtest: drop the write-only _kc_squeeze stamps (no behaviour change)
result._kc_squeeze / result._kc_squeeze_released were assigned by both the live
TechnicalEngine and the backtester's GPUTechnicalEngine and read by nothing:
the only four occurrences in the repo were the assignments. The squeeze state
still feeds the Keltner signal text where it is computed. Removed from both
engines in one commit (rule 4).

Parity: 12-month M5 and H1 books regenerated byte-identical
(md5 a28fed4becd056466c26d6a947e1a0e5 / d20405ef64f069770c258bb9bed1ac90).
Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:53:43 -04:00
Harwinder Sekhon 5885021dda main: build_config uses the pair factory's confirmation map (fixes an M5 divergence)
main.py kept a private confirmation table and passed it into the pair factory,
which honours any non-None list. For M5 that table said [M1, M15] while the
factory, telegram_bot.TF_CONFIRMATIONS and dashboard_publisher.CONFIRMATIONS
all use the validated [M15, H1] - so `main.py --timeframe M5` scanned with an
unvalidated pairing (and an extra M1 fetch) that no other surface used.

- build_config now returns PAIR_PRODUCERS[pair](tf, None): the factory map is
  the single source of truth. The unreachable bare-PlatformConfig fallback is
  gone here and in telegram_bot._run_signal_sync (every symbol that reaches
  either is registered).
- --timeframe offers the four validated primaries (M5, M15, H1, H4). M30 and
  D1 were never validated and silently fell through to the generic [M15, H4]
  map; they are removed from the CLI.
- The "all valid timeframes" test now asserts each CLI timeframe's
  confirmation list equals pair_signals.xauusd._CONFIRMATIONS - the check
  that would have caught this.

CLI behaviour change: M5 confirmations [M1, M15] -> [M15, H1]; M30/D1 rejected.
Bot and dashboard scans are unchanged.

Suite: 461 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:51:18 -04:00
Harwinder Sekhon 1b40f2166e chore: remove test-only helpers and the duplicate confirmation map (no behaviour change)
- telegram_bot._format_scan_summary: multi-pair "auto-scan summary" leftover;
  production sends per-report format_signal_message. Its only caller was a test.
- telegram_bot.scan_keyboard: thin wrapper only a test called; main_menu_keyboard
  already builds from _scan_button_rows directly.
- telegram_bot.AUTOSCAN_CONFIRMATIONS was byte-identical to TF_CONFIRMATIONS;
  the candle loop now reads the one map (app_config comment updated).
- SignalLogger.news_for: one-line wrapper over the module-level read_news that
  every real reader uses; the test now calls read_news like its neighbours.
- pair_signals.get_supported_pairs: PAIR_PRODUCERS is the registry; the test
  lists it directly.

Suite: 461 passed, 0 failed (two helper-only tests removed).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:50:21 -04:00
Harwinder Sekhon 985ec83ef3 chore: remove dead functions and write-only state (no behaviour change)
Zero callers anywhere (tests included), verified by grep/AST:
- dashboard_publisher.DashboardPublisher.publish_all (run_cycle inlines both publishes)
- signal_log.SignalLogger.advance_with_bars (the backfill script uses
  backfill_seeds / resolve_from_store / reseed_unresolvable) and the never-read
  SignalLogger.path property plus its write-only self._path
- llm_client.reset_llm_client (tests reset the singleton by assignment)
- market_outlook._last_run_ts, residue of the old "now - last < 3600" dedup;
  _last_run_hour (the UTC hour bucket) is the live key
- news_engine FundamentalContext.event_impact_on_pairs: written once, read by
  nothing; its only feeder NewsEngine._get_affected_pairs and that helper's
  unit test go with it
- position_sizing.LIVE_DEPTH_BARS (referenced only by its own docstring, which
  now states the depths inline), the unused Iterable import and the unused
  atr_ok parameter of _fill_geometry
- analyze_signal_log: unused `field` import and the unused `rows` parameter of
  persona_scorecard / news_outcome_buckets (both read the DB directly)

The two backfill/label scripts' --db help text now names the config.json
setting it actually falls back to.

Suite: 463 passed, 0 failed (one test removed with its subject).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:48:24 -04:00
Harwinder Sekhon c94f78e0b0 chore: drop unused imports and a dead constant (no behaviour change)
- backtest/diagnose_filters: unused BacktestConfig import
- pair_signals/__init__: unused PlatformConfig import
- scripts/shadow_scan: unused pandas import
- research burst_precursors / calendar_impact / duka_spread: unused sys / COST / struct
- test_all: MagicMock, StringIO, Colors and two function-local imports that
  nothing in the test bodies used
- telegram_bot: ViolentWatch is only a type hint -> imported under TYPE_CHECKING

compare_oanda_tiingo.py and oanda_td_divergence.py are left alone here because
the hygiene commit deletes them.

Suite: 464 passed, 0 failed (unchanged).

Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 16:46:15 -04:00
Harwinder Sekhon d757278b35 config: drop the dead oanda.account_id (auto-execution leftover), keep it tolerated
The candle provider needs only oanda.api_token + base_url; account_id belonged to the
auto-execution daemon removed 2026-09-03 (31002cb). Removed from config.json.example and
the unused oanda_account_id() accessor deleted; the key stays in _KNOWN_OANDA so existing
config.json files keep loading. CLAUDE.md env-var list updated. Test locks all three.

Suite: 464 passed.

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:40:44 -04:00
Harwinder Sekhon 05e71ce1e2 CLAUDE.md: remove stale facts, record the 3-year backtest preference
- test count no longer hardcoded (463 / 31 modules as of 2026-09-06)
- main.py --outlook, backtest --set / --export-features listed
- delivery surfaces include the web dashboard; SignalReport consumers include the publisher
- scikit-learn / lightgbm are research-only venv installs, never requirements
- rule 1: owner preference (2026-09-05) that backtests go back at most 3 years, with the
  3-year baseline books and where they live
- "Current state notes" header un-dated; the 2026-08-14 canonical-book line marked superseded
  by the 2026-09-03 book (M15 reference unchanged since 2026-08-04)
- docs: PDF builds noted as behind the HTML guides

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:23:46 -04:00
Harwinder Sekhon fc55da17db signal_log: weight-research data points, scan_outcomes forward labels, watch provenance
New additive columns on every logged scan: bar_ts (primary closed-bar label, indexed with
timeframe), engine_fingerprint (hash of the scoring config incl. the regime weight table in
force - SignalAggregator.engine_config_fingerprint, twin of the gate fingerprint), code_version
(short git SHA), penalty_tags_json / penalty_total / momentum_fading / exhaustion_count, and
post-hoc watch_level / watch_kinds / watch_line written by violent_watch.attach_watch via
signal_log.update_watch. Each indicators_json entry now also carries eff_weight (base weight x
regime multiplier, the weight the composite actually used) and wscore.

engine: IndicatorSignal.effective_weight / weighted_score are stamped inside _compute_composite
(record-keeping only); regime_weight_table(config) is the single source of truth for the
multiplier table (override honoured). Composite output byte-identical (120-report hash proof).

New table scan_outcomes: model-free forward moves per scan (fwd_1h/4h/24h in $ and ATRs,
mfe/mae over 24h, burst_1h, bars_used, complete) measured on the stored candle stream with the
advisory model's continuity rule; labelled opportunistically at the end of log() (time-boxed,
throttled, idempotent) - no daemon; scripts/label_scan_outcomes.py drives the backlog
(--dry-run is read-only). analyze_signal_log --scan-outcomes: forward move + hit rate by
TF x signal/grade with a NEUTRAL baseline and burst rate by watch level. Readers dedup by the
first row per (timeframe, bar_ts).

Module 31 (10 tests). Suite: 463 passed. Backtester untouched (test-locked).

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 22:22:59 -04:00
Harwinder Sekhon e79b533a60 violent-move watch: display-only warning line + owner heads-up (dark by default)
New violent_watch.py assesses each scan from the cached Forex Factory week events, the
UTC clock and the report's own indicator states, and stamps SignalReport.watch_line
post-hoc (telegram alerts, CLI, dashboard). A standalone heads-up goes to the owner on
the fastest scanned timeframe when the level is "hot", once per UTC hour per trigger
set, capped per day. Config block `violent_watch` (enabled False, alert_line, headsup,
max_headsup_per_day, headsup_min_level); env VIOLENT_WATCH_ENABLED.

Rule chosen from a 3-year bar-by-bar sweep (backtest/research/watch_rule_eval.py,
recorded in sizing_campaign_2026-09.md): hot = high-impact USD release within 30 min
(before or after) OR a hot clock slot (weekday 12-14 UTC, 01 UTC, Sunday 22-23 UTC)
with at least one price-extreme reading (outside Bollinger bands, RSI <30/>=70, outside
Keltner); ~8.6% of bars, next-bar odds 3-5x normal, ~4 hot hours/day. watch = clock
slot alone or two price-extreme readings (~2x). Context readings (ADX rising, extreme
volatility, transitioning, MACD acceleration) are phrases only - letting them raise the
level flagged 27% of bars at ~2x and would have sent ~13 heads-ups/day (rejected).
Direction sentence: DOWN when bearish readings dominate (70-78% on both TFs), UP on M5
only. Never a gate or exit: every open-trade protection tested lost money.

Test-locked out of analyze_from_dataframes / engine / backtester. Module 30 (18 tests).
Suite: 453 passed. Guide PDFs not rebuilt.

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 21:19:24 -04:00
Harwinder Sekhon 6bf58fd9d4 research: 3-year stake-sizing campaign + violent-move studies (nothing changes signals)
Walk-forward "second opinion" stake multiplier (meta-labelling on 65 decision-time
features, 3yr Oanda books @ $0.65) FAILED the pre-registered rule on M5 (clean null)
and M15 (lift inside the label-shuffle null, explained by BUY-vs-SELL). Nothing that
changes numbers ships; the 12-month M5 book re-run is byte-identical to the pinned one.

Behaviour-neutral plumbing:
- position_sizing.build_features (65 features; multiplier path returns 1.0, no artifact)
- SignalReport.penalty_tags / penalty_total / momentum_fading (record-keeping)
- TradeSetup.stake_multiplier = None (reserved)
- gpu_backtest --export-features (default export stays the pinned 63 columns; extras
  forced to the tail even when the first trade lacks an indicator)

Research tools (backtest/research): book_report, burst_map, burst_precursors,
open_trade_bursts, calendar_impact, burst_indicator_map, sizing_model, watch_rule_eval.
Findings recorded in backtest/research/sizing_campaign_2026-09.md: M5 stretch measures
are positive in both halves; every open-trade protection against violent bars loses on
both TFs; high-impact USD releases cause 20-31% of violent bars but trades held through
them do better; violent bars come when price is already at an extreme, not from calm.

Module 29 tests (13). Suite: 435 passed.

Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE
2026-09-05 21:19:24 -04:00
Harwinder Sekhon bed8b549ac audit: fix 9 defects from the 2026-09-04 hidden-bug sweep
None of these touch the signal path. The 12-month M15 book
(2024-03-31..2025-03-30, $0.65, cached Oanda bars) is byte-identical
before and after (md5 c59d3e7e685c3c880b614640adfea74b, 375 trades),
and signal/indicator logging to logs/signals.db is unchanged — verified
end to end: per-TF indicators for all 3 TFs, 14 candles each, MTF and
reasons all still persisted.

agent_panel: the >=2-take quorum guards the CHAIR's synthesizer spend,
  but ran BEFORE the macro branch — so the shipped default
  (verdict_source="macro", which never calls the synthesizer) silently
  dropped a good macro verdict whenever the other three personas were
  slow. Quorum moved below the macro path.
analyze_signal_log: _forward_return clamped to the last available bar
  when the horizon had not elapsed, reporting a near-zero "forward move"
  for every row logged within `hours` of the end of the data — on every
  run, blended into the hit-rate/mean-move aggregate. Now returns None.
analyze_signal_log: persona_scorecard iterated raw rows through an index
  that maps EVERY member row of an event, so a re-logged scan appended
  the same event R twice. One vote per event, matching
  news_outcome_buckets. Measured inflation on the live DB: 1.03x.
signal_log: dedup required a non-negative logged_at gap, but logged_at is
  stamped before the row wins the write lock (busy_timeout 5 s), so two
  processes logging one setup at a boundary can invert timestamp order
  against id order and seed a second event. Now abs(gap); same in
  backfill_seeds. Zero occurrences in the current 366 setup rows.
advisory_outcome: the catch-all turned real evaluator faults into the
  same UNRESOLVABLE used for legitimate weekend gaps, with nothing in the
  log. Still fail-safe, now logged.
llm_client: only the "cap" warn-once flag was ever reset, so in a daemon
  the second and every later network/http/parse outage logged at DEBUG
  only. Cleared on UTC-day roll and re-armed by a recovered call.
dashboard.js: renderCards() redraws the selector for every timeframe but
  was gated on the SELECTED one changing, leaving the other buttons stale
  (up to 4 h on H4) while renderHero's rail already showed the new value.
news_summary.php: atomic_write treated a short write (disk full) as
  success, publishing a truncated body the cache-first branch then served
  forever; and the daily cap was charged before the no-key check and
  before the stampede lock re-checked the cache, so a click burst could
  exhaust the 150/day budget with zero LLM calls.
migrate_signal_log_news: logged_at is ISO with 'T' while SQLite's
  datetime() uses a space, so string-comparing them put the whole cutoff
  calendar day on the wrong side and it never pruned. NOTE: this now
  deletes one day more than before — it only runs when invoked by hand.

Tests 414 -> 422 (new Module 28, one per fix). The existing
"<2 valid takes" test was retargeted at the chair — the case it actually
protects — rather than deleted, with new coverage for macro-alone.

Not fixed, recorded for the owner:
- Gate 6c's boost condition is textually identical to the C-grade
  definition, so every C in a "transitioning" regime is promoted to B
  before gate 6e reads it, making sell_min_quality's exclusion of "C" a
  no-op on M15/H4 (H1 and M5 are immune by other means). The backtester
  runs the same code, so every pinned book already includes this —
  changing it is a rule-1 revalidation event.
- Safe-haven/geopolitical headlines score exactly 0.00 in the keyword
  fallback and are now dropped entirely by the ±0.1 neutral filter,
  contradicting the platform's own gold doctrine. Bites only when
  DeepSeek is capped/disabled/failing.

Claude-Session: https://claude.ai/code/session_01Ah5EB1goi5inQmSd37Ne6a
2026-09-04 20:19:08 -04:00
Harwinder Sekhon a3d7594f0c news: beginner-language summary, 4 h headline window, drop neutral headlines; --news log report
Verdict from the 3-week signal log (backtest/research/news_sentiment_value_2026-09-04.md):
news_sentiment is a trailing-24h echo of price (r +0.67 with the past day, +0.03 with
the next; most bullish 2 h after the 08-25 top, bearish 44 h later; most bearish at the
09-02 low, bullish 16 h later). Per-headline LLM labels are correct; the 12 h pool and
the averaging are what destroy the information. Both gating flags stay False.

- news_engine: _LLM_SUMMARY_SYSTEM rewritten for someone who has never traded (no
  acronyms, two worked examples); summary payload newest-first with the platform's own
  gold direction; reply cap 300 -> 400 chars.
- config: headline_max_age_seconds 43200 -> 14400; RSS items without a parsable
  pubDate are dropped (they used to be stamped "now" and never aged out).
- news_engine.get_fundamental_context drops headlines whose every relevant score is
  inside +-0.1 (_is_neutral_headline) before the per-currency mean; news_sentiment
  from this date is a mean over directional <=4 h headlines (log comparisons split here).
- static: dashboard copy for the 4 h pool; news_summary.php prompt gets the same style rule.
- analyze_signal_log: --news (under --stored-candles) prints sentiment lead/lag with
  split halves, sentiment at the window high/low with flip lags, and outcome buckets by
  direction x news x macro stance; LogRow gains news_sentiment/event_risk; _row_index shared.
- tests: 406 -> 414 (age window, undated drop, neutral filter, summary prompt/payload/cap,
  --news helpers, load_rows round-trip); RSS fixtures now carry a pubDate.
- docs: user/beginner guides, CLAUDE.md, deploy README.

Claude-Session: https://claude.ai/code/session_0169BgFtWjp1SEAa5392Q9jH
2026-09-04 15:14:58 -04:00
Harwinder Sekhon 593ddfe81f agent_panel: alerts use the macro persona's verdict; all 4 takes still logged
agent_panel.verdict_source defaults to "macro": the macro take's stance is the
verdict, the synthesizer call is skipped (4 LLM calls per panel instead of 5);
"chair" restores the four-voice consensus + OPPOSE guard. Verdict source is
cached per mode and logged (panel_verdict_source). 406 tests.

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 21:16:45 -04:00
Harwinder Sekhon 354778683e Remove Oanda auto-execution daemon/executor (not auto-trading)
Deletes oanda_daemon.py, oanda_executor.py, their 28 tests and the daemon-only
config keys (oanda.lot_size/quality_gate/max_hold_bars). The Oanda data
provider and credentials stay — it is the live candle feed. 402 tests.

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 21:00:21 -04:00
Harwinder Sekhon 9a8a2c1aa1 xauusd factory: block M5 shorts in transitioning regime, block all H1 shorts (rule-1 validated)
M5 sell_blocked_regimes=[transitioning]: 12mo +7.9R (PF 1.50->1.56, both halves up),
5yr +5.3R, OOS unchanged. H1 sell_min_quality=[]: 5yr +16.7R (PF 1.18->1.46),
OOS +11.5R (PF 1.13->1.74), 12mo -1.8R with both halves positive.
Campaign table, rejected and parked candidates: backtest/research/signal_log_campaign_2026-09.md

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 20:47:33 -04:00
Harwinder Sekhon b1e880be4f signal_log: outcomes for every setup, event dedup, interned news; fix over-scan, dead outlook, panel OPPOSE bias
- seed an advisory outcome on every setup row; same-setup re-logs within 60 min
  are DUPLICATE rows (advisory_event_id); resolver rebuilds bar streams across gaps
- headlines/events interned into news_snapshots (DB 667 MB -> 262 MB); readers
  use signals_with_news / read_news and mode=ro (immutable=1 missed WAL rows)
- post-gate truth columns: gate_reason, actionable, cleaned_warnings, scan_kind,
  config_fingerprint; SELL-hour gates fail closed
- telegram: breach rescan reuses this bar's report; grade dropped from alert
  fingerprint (upgrade-only re-alert)
- market_outlook: hour-bucket dedup (old rule skipped alternate hours), skip heartbeats
- agent_panel: macro-anchored OPPOSE guard (opposed trades were +30R over 3 weeks)
- analyze_signal_log --stored-candles/--parity; backfill + news migration scripts
- tests: Module 25 (analyze_signal_log) + 27 (observability); 429 total

Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 20:47:33 -04:00
root 83ffd290ca agent_panel: harden evidence and track outcomes 2026-08-23 12:57:05 -04:00
root a87ed7ba63 Fix chart pattern alert Markdown 2026-08-20 21:03:30 -04:00
Harwinder Sekhon 036ddde06d dashboard: support Python 3.9 import 2026-08-18 18:08:38 -04:00
Harwinder Sekhon 410bf5ebc4 signal_log: persist market snapshots 2026-08-18 15:24:37 -04:00
Harwinder Sekhon 71eb703355 dashboard: redesign signal workspace and refresh flow 2026-08-18 14:54:47 -04:00
Harwinder Sekhon 1f632eaa6a market_outlook: add hourly advisory brief 2026-08-18 13:52:18 -04:00
Harwinder Sekhon bd62028b2f dashboard: color ladder levels red/green on cells (Bootstrap table overrides row colors), add direction icons 2026-08-16 21:02:24 -04:00
Harwinder Sekhon d4137d862a dashboard: fix gauge charts (install gauge.js, render after polls), force dark theme with white text and red/green accents, add icons 2026-08-16 20:55:25 -04:00
Harwinder Sekhon c248b39cc5 Replace Google Charts with Chart.js horizontal bars and set dark theme default 2026-08-16 20:28:54 -04:00
Harwinder Sekhon 96c926803d agent_panel: ATR-normalized candle + indicator-value enrichment, hardened prompts, verdict logging 2026-08-15 15:24:28 -04:00
Harwinder Sekhon 8b7b63853c Update gauge colors: green for positive scores, red for negative scores 2026-08-15 11:38:25 -04:00
Harwinder Sekhon 5694a1ed0c Replace score bars with Google Gauge visualization 2026-08-15 11:34:19 -04:00
Harwinder Sekhon a979dcba4b signal_log: persist agent panel verdict + per-persona takes to SQLite
Add panel_verdict, panel_confidence, panel_rationale, panel_takes_json
columns to the signals table (additive, idempotent ALTER TABLE migration
for existing DBs). _report_row now reads the panel fields attach_panel
writes onto the SignalReport, so LLM panel verdicts stop evaporating on
process exit. Also documents the Oanda re-pin (round_trip_cost stays
$0.30 for cross-book comparability and the live TP gate; $0.65 is the
measured headline) and adds the feed-divergence / parity / shadow-scan
study tooling referenced by CLAUDE.md.
2026-08-14 20:41:54 -04:00
Harwinder Sekhon be5144bc67 dashboard: redesign on Bootstrap 5, split page into external css/js
Page now uses Bootstrap 5.3 color modes (data-bs-theme), Bootstrap Icons,
and Bootstrap modals. Custom code shrinks to dashboard.css (app tokens,
score bar, signal accents) and dashboard.js (rendering/polling logic,
ported verbatim from the inline script). dashboard_publisher copies the
new css/js into the docroot; all 381 tests still green.
2026-08-13 23:30:30 -04:00
Harwinder Sekhon e609b433b0 signal_log: persist every live scan (signal, grade, price, news) to SQLite
New signal_log.py logs each SignalReport to logs/signals.db via a
fail-safe hook in SignalAggregator.analyze_pair — the live-only entry
point used by the bot, daemon, CLI and dashboard. Backtester calls
analyze_from_dataframes directly, so live==backtest parity is preserved
(test-locked). Rows carry signal/grade/price/spread, MTF + trade-setup +
reasons/warnings JSON, and news-at-scan-time (calendar events + scored
headlines JSON via NewsEngine class-level caches).

Config: signal_log block (enabled, path; env SIGNAL_LOG_ENABLED/PATH),
source tagged per consumer (telegram/oanda_daemon/cli/dashboard).
381 tests pass (24 modules).
2026-08-13 10:24:49 -04:00
Harwinder Sekhon 887aaed6f2 dashboard: skip DOM rebuild on unchanged poll, publish-time staleness, fix boundary countdowns
poll() now compares raw payload text and re-renders only what a changed
file can affect, preserving selection, disclosure, and scroll state. Hero
freshness reflects publish time (STALE warning past 15 min while market
open), and countdowns resolve their wrap interval from data-interval
instead of reverse-looking shared TF boundaries in snapshot.timeframes.
2026-08-09 18:15:31 -04:00
Harwinder Sekhon 7e8ed754ff news: reduce headline max age from 24h to 12h, make configurable
- Added headline_max_age_seconds to NewsConfig (default 43200 = 12h)
- Replaced hardcoded 86400 in _fetch_news_sentiment_raw with config ref
- All downstream consumers (keyword scoring, LLM scoring, LLM summary,
  agent panel macro/contrarian) auto-filter through the same path
2026-08-09 16:49:48 -04:00
Harwinder Sekhon 9769a0c84d agent_panel: port ai-hedge-fund prompt rubric — stance rules, anchored confidence, no-invention
Prompt-only change (static constants; plain-language output contract
byte-identical). Ported from virattt/ai-hedge-fund's persona rubric:

- _TAKE_CONTRACT hard rules: reason only from the JSON given, never
  invent numbers/events/levels; thin or missing evidence -> neutral
  with confidence below 30 (deferring to role-specific rules).
- Anchored confidence bands in the contract and synthesizer, replacing
  the unanchored "how sure you are".
- Per-stance support/oppose/neutral criteria for macro, technical, and
  risk (risk's missing-setup->oppose folded into its oppose bullet).
- One plain-language support example beside each oppose example for
  macro/technical/risk, fixing the all-oppose few-shot bias.
  Contrarian's designed skeptic tilt untouched.

News-engine prompts untouched (pinned by 78/78 + 18/18 validations).
374 tests green; real-DeepSeek smoke: macro correctly answered
neutral/20 on empty news context, confidences spread 20-60 across the
bands, all notes <=25 words, no jargon.

Claude-Session: https://claude.ai/code/session_01TyZx9iRJwfbPTPnCBqcwTR
2026-08-08 10:48:10 -04:00
Harwinder Sekhon 964239e67d docs: multi-provider LLM layer plan (DeepSeek news / Anthropic panel)
Reference plan only - no code changes. Describes the BaseLLMClient
refactor, AnthropicClient on the official SDK, per-consumer routing via
a config.json "llm" block, tests, commit sequence, and effort estimate
for the day we route the agent panel to Claude while news stays on
DeepSeek.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 23:02:08 -04:00
Harwinder Sekhon 5e3758bd53 agent_panel: plain-language output — simple, short, no jargon/acronyms
Owner request: panel notes and rationale must read like plain everyday
English for a non-trader. Style rule added to the shared take contract
and synthesizer prompt, all few-shot examples rewritten in the target
voice (examples carry the style), and the Telegram votes line now spells
out full stances ("macro: support") instead of "macro:sup".

Before: "BUY against Bollinger SELL and price above upper band; 4h
neutral, mixed MTF"  →  After: "The price is above its usual range, and
the bigger picture is not clearly rising, so buying now is risky."

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:50:10 -04:00
Harwinder Sekhon c146fffb5b docs: agent panel module + config block
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:35:15 -04:00
Harwinder Sekhon 38c81251e8 cli+dashboard: --panel flag, panel dict keys, TF-dialog panel section
main.py: --panel runs the agent panel on demand (force bypasses
enabled/surface/quality toggles; daily budget + LLM cap still apply),
prints the verdict + persona takes, and report_to_dict carries the four
panel keys — which auto-propagates them into the dashboard's
snapshot.json via report_to_webapp_dict. dashboard_publisher attaches
in scan_tf gated by agent_panel.dashboard (default off); dashboard.html
renders an "AI panel (advisory)" section in the TF detail dialog.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:34:30 -04:00
Harwinder Sekhon a7c2bfc7da telegram: attach AI panel to alert-quality sends (advisory block)
Auto-scan and level-breach alert paths attach the panel AFTER the
quality gate and dedup decision (no LLM spend on suppressed alerts),
via run_in_executor so the event loop stays free. Alert message gains a
two-line 🤖 AI Panel block (verdict + confidence + rationale, persona
votes); /status shows the daily panel budget when the feature is on.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:32:57 -04:00
Harwinder Sekhon 7be2231eb3 agent_panel: persona/synthesizer advisory consensus module
ai-hedge-fund-style panel: 4 DeepSeek personas (macro, technical, risk,
contrarian) review a finished SignalReport concurrently on a
panel-private thread pool, then a synthesizer chair issues
AGREE/NEUTRAL/OPPOSE + confidence + rationale. Advisory/display-only by
construction: consumers attach post-hoc via attach_panel; the fields
live on SignalReport with empty defaults and analyze_from_dataframes
never touches them (backtest parity, locked by test).

Spend control on top of the shared 500/day client cap: max_panels_per_day
UTC budget (default 20), exact per-bar result cache, and a reuse TTL
keyed (symbol, tf, direction) so a persisting signal re-serves its
verdict across bars instead of burning 5 requests per bar. <2 valid
takes skips the synthesizer; any failure degrades to no panel.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:31:50 -04:00
Harwinder Sekhon 0b604f1ad0 config: agent_panel block (enabled/surfaces/min_quality/daily budget)
New optional config.json block for the upcoming LLM agent-panel feature:
enabled, telegram, dashboard (per-surface toggles), min_quality,
max_panels_per_day, reuse_ttl_seconds. Absent block → {} → feature dark.
Validation mirrors the deepseek block. Module 23 opened with parse +
reject tests.

Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:26:43 -04:00
Harwinder Sekhon 250078c987 docs: architecture diagram (PNG + matplotlib generator)
End-to-end map: external services → providers/news layer → signal core
(engine + factory + aggregator) → SignalReport → four delivery surfaces,
plus the backtester replaying the same core. Regenerate with
venv/bin/python docs/architecture_diagram.py after architecture changes.

Claude-Session: https://claude.ai/code/session_01Aq1jZ6jBsBdEC2xT3u46SA
2026-08-07 09:21:52 -04:00
Harwinder Sekhon d1d002a4f7 dashboard: clickable news — detail dialogs + on-demand AI summary endpoint
- news_engine: RSS parser captures link/description/publisher (three new
  optional NewsSentiment fields); _clean_html_text/_safe_link helpers
- news.json: headlines carry id = sha1(headline) + enrichment fields
- static/news_summary.php (NEW): on-demand DeepSeek gold-impact summary,
  whitelist-gated against news.json ids, cache-forever in ai/, fail-closed
  file-based daily cap (150/day), key at /etc/gold-dashboard/deepseek.key;
  installed to the docroot by install_static
- dashboard.html: newswire + calendar detail dialogs (score tiles, source
  description, article link, AI summary with loading/error states);
  delegated on stable containers, dialog keyed by id across polls
- tests: +6 (357 total); docs: README-dashboard §6 one-time setup
  (key file, apache-owned ai/, SELinux httpd_can_network_connect), CLAUDE.md

Claude-Session: https://claude.ai/code/session_01Aq1jZ6jBsBdEC2xT3u46SA
2026-08-07 09:08:49 -04:00