Commit Graph

11 Commits

Author SHA1 Message Date
Harwinder Sekhon 1454ea1838 Quant review 2026-09-26: alert cooldown, plain-language alerts, M15 grade-A SELL block, 1R printed targets, panel/breach fixes
Live signals.db review (2026-09-04 → 09-25, 100 unique setups replayed on the
scans' stored candles): the stop-hunt claim is refuted (71% of stop-outs ran
≥ 1R past the stop, the same as the books' 70–72%); −25.1R of the −37.8R raw
live book were consecutive-bar re-entries the backtester's 5-bar cooldown never
takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week
windows.

Delivery (book-neutral):
- telegram_bot: 5-bar alert cooldown per timeframe (_ALERT_COOLDOWN_BARS =
  backtest.cooldown_bars); setup block prints the trail first as the tested
  exit, TP1/TP2 as guides with their R, "Planned TP1 RR", and one validated-book
  line per setup class (agent_panel_base_rates.json); the informational 100-pip
  TP line is removed (owner request).
- AI block rewritten for a new trader: "AI check: the odds look better than /
  worse than / about usual", the chance "in 100" against the usual odds, one
  reason for and one against, no verdict word or confidence figure; rendered
  whenever the verdict carries a probability. agent_panel: persona prompts ban
  trading jargon (PANEL_PROMPT_VERSION v2.1-2026-09-26); the macro-source
  verdict now carries its persona's p_win (every live v2 row had
  panel_probability NULL). Dashboard and guides mirror the wording.
- telegram_bot: level-breach rescans skip the settlement-pause window (the
  20:45 M15 bar was re-logged at 22:01/22:06/22:11 every evening).

Signal (pre-registered rule-1 campaign, CUDA twin paired with gpu_backtest.py):
- M15 grade-A SELLs blocked (sell_min_quality ["A+","B"], M15 override 6):
  W2 813/+154.15R PF 1.53 vs 931/+146.14R PF 1.43, OOS +68.5 vs +60.67,
  W3 max DD 20.9 vs 19.3R. PASSED a–f.
- Printed target 1 floored at 1R in every generator (signals.tp_print_floor_rr),
  applied AFTER the min-RR and spread gates so no trade changes; mirrored in
  cuda_signal_kernels.py K2. Label-only re-pin of all four timeframes (only
  tp1/rr_planned columns move). A floor applied before the gate cost −24.6R and
  was rejected.
- FAILED: M15 min_rr_ratio 1.0 (trade-identical, −2.0R). Owner's call: M5
  tp_floor_rr 1.0 (+48.9R W2, max DD 10.1 → 15.9R), not shipped.

New pins: M5 W2 e599c8a8…, M15 W2 5b65ecf6…, M15 OOS 09b8a8e7…, H1 W3
a44fd19b…, H4 W3 f3587bae…; twin and reference MD5s equal on every window.
Base-rate table rebuilt from the new pins. Suite 558 passed.

Evidence: backtest/research/rr_and_sell_campaign_2026-09-26.md (plus the
2026-09-18 loss-anatomy and wrong-direction reports it builds on).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-26 21:06:05 -04:00
Harwinder Sekhon 1f4741b557 news corpus for a local scorer + agent panel v2 + dashboard read-only (2026-09-17); books byte-identical
Agent panel v2: personas return p_win anchored to the book base rates with a
code-computed decision context; verdict = equal-weight mean (verdict_source
"mean"); only the macro persona runs live (agent_panel.personas ["macro"]);
prompt version and exact payloads logged; offline replay harness
scripts/panel_replay.py; base-rate table agent_panel_base_rates.json
(scripts/build_panel_base_rates.py). Backtester `strategy` column now carries
the generator's own stamp (label-only; five reference books re-pinned, all
62 other columns identical).

News corpus (after evaluating the owner's ML-NEWS scikit model against
DeepSeek: golden set 39/78 vs 78/78, out-of-sample agreement 64% against a
90% self-agreement ceiling; verdict: fallback only, not a replacement):
- NewsSentiment.xau_score + archive column xau_score keep DeepSeek's gold
  score for every headline (930 of the first 1,546 DeepSeek rows had none);
  an existing newsdata.db gains the column in place on first open
- headlines naming no tracked currency are scored and archived
  (currencies_json []) but never returned to the platform
- _apply_llm_sentiment sends every new headline in its cycle, 25 per call;
  a failed call ends the cycle's sending

Dashboard publisher never writes a database: main() calls
app_config.set_db_writes_for_process(False), honoured by signal_log_enabled()
and news_archive_enabled() (owner decision: the bot is the single writer).

Test suite: the hermetic app config now points both writers at a
suite-private temp dir — the RSS fixture tests were writing fixture
headlines into the real logs/newsdata.db — with a Module 32 test pinning it.
529 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-17 19:49:29 -04:00
Harwinder Sekhon 1f44d6bad6 chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
  was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
  config.json migration that merged the same day; its plan sibling was already
  deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
  Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
  the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
  divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
  and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
  its .jsonl evidence stays

Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.

Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
  backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
  to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
  (the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
  actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
  4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
  analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
  says 4 requests by default; tiingo comment cites the NY-anchored weekend;
  Module 21 title no longer says "execution lifecycle"

Suite: 461 passed, 0 failed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
2026-09-06 17:01:07 -04:00
Harwinder Sekhon 59f69400a8 audit: fix 9 defects from the 2026-09-04 hidden-bug sweep
None of these touch the signal path. The 12-month M15 book
(2024-03-31..2025-03-30, $0.65, cached Oanda bars) is byte-identical
before and after (md5 c59d3e7e685c3c880b614640adfea74b, 375 trades),
and signal/indicator logging to logs/signals.db is unchanged — verified
end to end: per-TF indicators for all 3 TFs, 14 candles each, MTF and
reasons all still persisted.

agent_panel: the >=2-take quorum guards the CHAIR's synthesizer spend,
  but ran BEFORE the macro branch — so the shipped default
  (verdict_source="macro", which never calls the synthesizer) silently
  dropped a good macro verdict whenever the other three personas were
  slow. Quorum moved below the macro path.
analyze_signal_log: _forward_return clamped to the last available bar
  when the horizon had not elapsed, reporting a near-zero "forward move"
  for every row logged within `hours` of the end of the data — on every
  run, blended into the hit-rate/mean-move aggregate. Now returns None.
analyze_signal_log: persona_scorecard iterated raw rows through an index
  that maps EVERY member row of an event, so a re-logged scan appended
  the same event R twice. One vote per event, matching
  news_outcome_buckets. Measured inflation on the live DB: 1.03x.
signal_log: dedup required a non-negative logged_at gap, but logged_at is
  stamped before the row wins the write lock (busy_timeout 5 s), so two
  processes logging one setup at a boundary can invert timestamp order
  against id order and seed a second event. Now abs(gap); same in
  backfill_seeds. Zero occurrences in the current 366 setup rows.
advisory_outcome: the catch-all turned real evaluator faults into the
  same UNRESOLVABLE used for legitimate weekend gaps, with nothing in the
  log. Still fail-safe, now logged.
llm_client: only the "cap" warn-once flag was ever reset, so in a daemon
  the second and every later network/http/parse outage logged at DEBUG
  only. Cleared on UTC-day roll and re-armed by a recovered call.
dashboard.js: renderCards() redraws the selector for every timeframe but
  was gated on the SELECTED one changing, leaving the other buttons stale
  (up to 4 h on H4) while renderHero's rail already showed the new value.
news_summary.php: atomic_write treated a short write (disk full) as
  success, publishing a truncated body the cache-first branch then served
  forever; and the daily cap was charged before the no-key check and
  before the stampede lock re-checked the cache, so a click burst could
  exhaust the 150/day budget with zero LLM calls.
migrate_signal_log_news: logged_at is ISO with 'T' while SQLite's
  datetime() uses a space, so string-comparing them put the whole cutoff
  calendar day on the wrong side and it never pruned. NOTE: this now
  deletes one day more than before — it only runs when invoked by hand.

Tests 414 -> 422 (new Module 28, one per fix). The existing
"<2 valid takes" test was retargeted at the chair — the case it actually
protects — rather than deleted, with new coverage for macro-alone.

Not fixed, recorded for the owner:
- Gate 6c's boost condition is textually identical to the C-grade
  definition, so every C in a "transitioning" regime is promoted to B
  before gate 6e reads it, making sell_min_quality's exclusion of "C" a
  no-op on M15/H4 (H1 and M5 are immune by other means). The backtester
  runs the same code, so every pinned book already includes this —
  changing it is a rule-1 revalidation event.
- Safe-haven/geopolitical headlines score exactly 0.00 in the keyword
  fallback and are now dropped entirely by the ±0.1 neutral filter,
  contradicting the platform's own gold doctrine. Bites only when
  DeepSeek is capped/disabled/failing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ah5EB1goi5inQmSd37Ne6a
2026-09-04 20:19:08 -04:00
Harwinder Sekhon b8f903ad0e agent_panel: alerts use the macro persona's verdict; all 4 takes still logged
agent_panel.verdict_source defaults to "macro": the macro take's stance is the
verdict, the synthesizer call is skipped (4 LLM calls per panel instead of 5);
"chair" restores the four-voice consensus + OPPOSE guard. Verdict source is
cached per mode and logged (panel_verdict_source). 406 tests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 21:16:45 -04:00
Harwinder Sekhon ae6a6658f6 signal_log: outcomes for every setup, event dedup, interned news; fix over-scan, dead outlook, panel OPPOSE bias
- seed an advisory outcome on every setup row; same-setup re-logs within 60 min
  are DUPLICATE rows (advisory_event_id); resolver rebuilds bar streams across gaps
- headlines/events interned into news_snapshots (DB 667 MB -> 262 MB); readers
  use signals_with_news / read_news and mode=ro (immutable=1 missed WAL rows)
- post-gate truth columns: gate_reason, actionable, cleaned_warnings, scan_kind,
  config_fingerprint; SELL-hour gates fail closed
- telegram: breach rescan reuses this bar's report; grade dropped from alert
  fingerprint (upgrade-only re-alert)
- market_outlook: hour-bucket dedup (old rule skipped alternate hours), skip heartbeats
- agent_panel: macro-anchored OPPOSE guard (opposed trades were +30R over 3 weeks)
- analyze_signal_log --stored-candles/--parity; backfill + news migration scripts
- tests: Module 25 (analyze_signal_log) + 27 (observability); 429 total

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
2026-09-03 20:47:33 -04:00
root 4dd0ca0b46 agent_panel: harden evidence and track outcomes 2026-08-23 12:57:05 -04:00
Harwinder Sekhon 7b60676a7a agent_panel: ATR-normalized candle + indicator-value enrichment, hardened prompts, verdict logging 2026-08-15 15:24:28 -04:00
Harwinder Sekhon 7e37aee0a6 agent_panel: port ai-hedge-fund prompt rubric — stance rules, anchored confidence, no-invention
Prompt-only change (static constants; plain-language output contract
byte-identical). Ported from virattt/ai-hedge-fund's persona rubric:

- _TAKE_CONTRACT hard rules: reason only from the JSON given, never
  invent numbers/events/levels; thin or missing evidence -> neutral
  with confidence below 30 (deferring to role-specific rules).
- Anchored confidence bands in the contract and synthesizer, replacing
  the unanchored "how sure you are".
- Per-stance support/oppose/neutral criteria for macro, technical, and
  risk (risk's missing-setup->oppose folded into its oppose bullet).
- One plain-language support example beside each oppose example for
  macro/technical/risk, fixing the all-oppose few-shot bias.
  Contrarian's designed skeptic tilt untouched.

News-engine prompts untouched (pinned by 78/78 + 18/18 validations).
374 tests green; real-DeepSeek smoke: macro correctly answered
neutral/20 on empty news context, confidences spread 20-60 across the
bands, all notes <=25 words, no jargon.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TyZx9iRJwfbPTPnCBqcwTR
2026-08-08 10:48:10 -04:00
Harwinder Sekhon 764cbc8a3d agent_panel: plain-language output — simple, short, no jargon/acronyms
Owner request: panel notes and rationale must read like plain everyday
English for a non-trader. Style rule added to the shared take contract
and synthesizer prompt, all few-shot examples rewritten in the target
voice (examples carry the style), and the Telegram votes line now spells
out full stances ("macro: support") instead of "macro:sup".

Before: "BUY against Bollinger SELL and price above upper band; 4h
neutral, mixed MTF"  →  After: "The price is above its usual range, and
the bigger picture is not clearly rising, so buying now is risky."

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:50:10 -04:00
Harwinder Sekhon 26c8449877 agent_panel: persona/synthesizer advisory consensus module
ai-hedge-fund-style panel: 4 DeepSeek personas (macro, technical, risk,
contrarian) review a finished SignalReport concurrently on a
panel-private thread pool, then a synthesizer chair issues
AGREE/NEUTRAL/OPPOSE + confidence + rationale. Advisory/display-only by
construction: consumers attach post-hoc via attach_panel; the fields
live on SignalReport with empty defaults and analyze_from_dataframes
never touches them (backtest parity, locked by test).

Spend control on top of the shared 500/day client cap: max_panels_per_day
UTC budget (default 20), exact per-bar result cache, and a reuse TTL
keyed (symbol, tf, direction) so a persisting signal re-serves its
verdict across bars instead of burning 5 requests per bar. <2 valid
takes skips the synthesizer; any failure degrades to no panel.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 22:31:50 -04:00