Commit Graph

201 Commits

Author SHA1 Message Date
Harwinder Sekhon 6e2fdbd684 telegram_bot: credentials/quality/scan-loops from config.json, startup guard
- Credentials, alert quality, scan pairs/timeframes, log level and
  candle-loop intervals now resolve via app_config (env -> config.json
  -> code default) instead of hardcoded fallbacks.
- _get_shared_dp() uses make_data_provider() for provider-registry
  dispatch.
- main() refuses to start without a token/chat_id, raising SystemExit
  with an actionable message instead of a bare sys.exit string.
- test_all.py Module 13: two new tests covering credential/quality/
  loop wiring via a reload, and the startup guard.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:18:05 -04:00
Harwinder Sekhon 80a2b89769 Data provider: key from config.json/env, provider registry, drop hardcoded key
Resolves the Twelve Data API key through app_config.provider_api_key()
(env → config.json → no default) instead of a hardcoded field on
PlatformConfig. Adds data_provider.PROVIDER_REGISTRY + make_data_provider()
so signal_aggregator and the backtester build the active provider through
one seam; telegram_bot.py gets the same one-line WS-key fix now (per Task 4's
already-planned edit for that line) so the suite doesn't crash on import
between this commit and Task 4's full wiring pass.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 13:10:58 -04:00
Harwinder Sekhon 1610507d57 Wire config.json overlay into the pair factory (before per-TF deltas)
Implements Task 2: apply_pair_overrides in xauusd.make_config before
the validated per-TF deltas, so e.g. M5 hour-9 appends on top of the
JSON base. Updated Module 9 equivalence test and added three parity pins.

Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:59:12 -04:00
Harwinder Sekhon 1dab14d298 app_config: reject non-object config.json sections with AppConfigError
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:54:46 -04:00
Harwinder Sekhon cbc01565a2 Add app_config.py: config.json loader, validation, env-precedence resolvers
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:47:48 -04:00
Harwinder Sekhon 53b65e1bbb Plan: centralized config.json implementation (5 TDD tasks)
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:44:24 -04:00
Harwinder Sekhon 1a141abc7c Add config.json.example template; gitignore config.json
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:32:28 -04:00
Harwinder Sekhon 096b69686d Spec: centralized config.json design (deployment settings overlay)
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
2026-07-21 12:25:08 -04:00
Harwinder Sekhon c21c102536 News informational-only: gate sentiment bonus/penalty off by default
Add NewsConfig.sentiment_affects_signals (default False), gating the +5
aligned-sentiment confidence bonus and the 10-pt news_conflict penalty.
With event_risk_caps_signals already False, live decisions are now pure
technical, matching the backtester (use_news=False). Warnings and the
sentiment display are unchanged. Adds a gating regression test.

Claude-Session: https://claude.ai/code/session_01FMaaPdrrrQuh1VvGQH1icN
2026-07-17 16:03:31 -04:00
harry c87bb14830 Merge pull request 'Remove EUR/USD support — XAU/USD-only codebase' (#1) from xau-only-chart-patterns into main
Reviewed-on: http://192.168.55.11:3000/harry/ForexBot2/pulls/1
2026-07-16 20:09:21 +00:00
Harwinder Sekhon a3279e54fa Enable BUY-only chart patterns on M5 — bearish patterns dropped (OOS negative)
M5 factory: cp_enabled=True, patterns limited to inv_hs, double_bottom,
bull_flag. SELL CP lost -3.21R on OOS; BUY CP: +41.8R PF 1.60 on 12mo,
+65.2R PF 1.48 on 5yr. Adds +22R to the M5 book (708 vs 513 trades).
M15/H1 remain disabled — all failed OOS validation.

Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 16:08:13 -04:00
Harwinder Sekhon 1bce289132 Docs: CLAUDE.md — document XAU_CP archetype (pipeline step 11, generator D, cp_* knobs)
Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 14:31:45 -04:00
Harwinder Sekhon f3d3e56aa8 Add chart-pattern entry archetype (XAU_CP) — disabled by default
Third entry archetype: completion of classical chart patterns (double
bottom/top, H&S top / inverse H&S, bull/bear flag) — the tight canon
with the strongest documented stats. New chart_patterns.py detector
module (pure functions, ATR-unit geometry, closed-bar slices only,
first-close-beyond trigger discipline so a breakout fires exactly
once and never repaints).

Pipeline step 10 in analyze_from_dataframes: fills only empty setup
slots, allows weak-composite F grades (normal at reversal completion)
but inherits session/day blocks, mirrors MTF-opposition + SELL
vol/hour + M5 protections. Geometry: SL = pattern anchor -/+ 0.5 ATR
(min $8), TP1 = measured move, $9 spread gate, trailing exit does the
real work. Strategy tag XAU_CP.

cp_enabled defaults to False everywhere — per-TF adoption happens in
pair_signals/xauusd.py only after the standard validation rule passes.
Backtester: --cp / --cp-patterns flags to test the candidate, XAU_CP
tag recognition.

Suite: 183/183 (21 new tests: detectors, trigger discipline,
invalidation, gates, pipeline, default-off safety).

Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 14:23:17 -04:00
Harwinder Sekhon 14448c10e8 Remove EUR/USD support — XAU/USD-only codebase
EUR/USD never had a validated edge (2026-07-11/12 verdict: no config
passed validation on M5/M15/H1 at 1-pip cost; cost-to-volatility root
cause). Removed: EurUsdStrategyConfig, pair_signals/eurusd.py, EUR
gates 6h + weekend/day filters + strategy marker + trade geometry in
signal_aggregator, EUR pair registry/aliases in main.py and
gpu_backtest.py, EUR/USD news pair mappings, test_all.py Module 10.
Generic multi-currency news keyword infra kept (GBP/JPY/EUR keyword
table is not pair code). Full EUR history remains at 7d3fd95.

Suite: 162/162 green (was 187; 25 EUR-specific tests removed).

Claude-Session: https://claude.ai/code/session_015kby7M9JwpYi8NQT2yoxAi
2026-07-16 14:06:27 -04:00
Harwinder Sekhon 665ffdda48 Fix on Sell order lossing by blocking time 2026-07-16 12:17:56 -04:00
Harwinder Sekhon fe7d4c5d4a Audit fixes: cache boundary staleness, 6e session bypass, GPU engine parity
Fixes all high+medium findings from the 2026-07-16 full-codebase audit.

High:
- data_provider: cache now rejects snapshots missing the latest closed
  bar (_has_latest_closed_bar, weekend-exempt). The wall-clock TTL served
  mid-candle snapshots at the next boundary, so live M15/H1 scans ran one
  full bar behind and the bar-close session check evaluated the wrong hour.
- signal_aggregator: gate 6e (SELL quality) only acts on actionable
  grades. It was re-grading session-blocked F reports to D, which
  re-opened the M5 pullback path (entry check: quality != "F") on
  session-blocked bars.
- gpu_backtest: port the live EMA-ribbon compression/expansion logic
  (+10 conf on expansion) into the GPU engine. Its absence diverged
  composite score/conf on 7-9% of bars and flipped signals on ~0.2%.

Medium:
- gpu_backtest: Bollinger precompute uses the exact pandas rolling calls
  (cumsum E[x2]-E[x]2 lost precision at gold price levels; bb_position
  error up to 7.6e-3 crossed signal thresholds, worst during squeezes).
- gpu_backtest: --confirm-tfs defaults to the pair factory's live
  confirmation map instead of a hardcoded 1h,4h.
- gpu_backtest: entries re-anchor to the first RAW bar after the signal
  bar, so >10-sigma spike bars dropped by cleaning are no longer skipped
  at entry (favorable bias).
- gpu_backtest: per-TF warmup pad — D1 confirms get >=210 bars at window
  start (62-day pad gave ~61 bars: no EMA200, unlike live).
- data_provider: thread-safety locks for cache + throttle (shared
  provider across the bot's 4-thread pool); outlier filter keeps the
  snap-back bar after a spike instead of punching a two-bar hole.
- telegram_bot: alert fingerprint gains a risk-relative entry bucket
  (materially-moved setups re-alert) and is committed only after a
  successful send (failed sends no longer consume the alert for 1h).
- engine: disabled_indicators=["macd"] now removes the MACD pillar from
  scoring (histogram still computed for divergence).

Verification: test_all.py 187/187 (new Module 11: 8 audit-fix
regressions incl. a live-vs-GPU parity sweep with ribbon coverage);
real-data parity sweep over 813 slices of 14mo XAU M15: zero divergence
(pre-fix: 7-9% of bars diverged); both original bug repros no longer
reproduce; cached end-to-end backtest run passes with the live confirm
map. Books shift slightly vs pre-fix runs — comparability note added to
CLAUDE.md; pinned tuned numbers need re-validation under the fixed
backtester.
2026-07-16 10:24:39 -04:00
Harwinder Sekhon 4d54fddf80 Docs: delete DOCUMENTATION.md — both halves superseded
Unlike USAGE.md (a USD/JPY-era fossil), this doc was structurally sound:
right pair, real architecture, correct composite formula, and 8 of its 9
regime-weight cells match engine.py. It is removed because both of its
sections now have live successors, and because the numbers it does state
are wrong in ways that mislead:

- Section 1 (trader-facing) → docs/user_guide.html + beginner_guide.html,
  which are newer and cover the trailing-stop exit model this doc never
  mentions.
- Section 2 (developer reference) → CLAUDE.md.

Stale/incorrect content it carried: the entire quality-grade table was
wrong (claimed A+ 45/70, A 30/60, B 20/50, C 12/40, D 8+; the code grades
40/65, 33/55, 28/50, 25/45, 15), Divergence's trending weight was listed
2.0 vs 1.0 in code, and it predates the 2026-07 work entirely — zero
mentions of XAU_PB or trailing exits (the actual default exit model), and
near-zero coverage of the EUR/USD port.

CLAUDE.md's header now points at docs/ as the human-facing docs instead
of disclaiming this file.
2026-07-16 09:34:04 -04:00
Harwinder Sekhon aaf1d37b14 Docs: delete stale USAGE.md and superseded pullback plan
USAGE.md documented a different product: USD/JPY (unsupported — the
code is XAU/USD + EUR/USD), 16 main.py flags that don't exist
(--backtest, --optimize, --auto-session-filter, …; main.py has five),
and backtest.py/metrics.py, which aren't in the repo. Two claims were
actively misleading: it called the session filter "default off" (it is
True by default, 03:00-23:00 not 07-16) and gave regime-min-adx as 30
(real default 25). CLAUDE.md now covers the CLI, bot and backtester
correctly, so this is a delete rather than a rewrite.

PULLBACK_ENTRY_PLAN.txt was a pre-implementation draft whose spec no
longer matches what shipped — it specifies a 0.5-2.5 ATR retrace window
and a detector in engine.py, while XAU_PB ships 0.8-6.0 (the drafted
window rejected 81% of valid pullbacks) with the detector in
signal_aggregator.py so live and backtest share one code path. Its
validated outcome (+38.3R / 254 trades / 12mo M5) and every shipped
gate are recorded in CLAUDE.md and the config docstrings.

Also drops the now-dangling references to both files from CLAUDE.md
and the config.py XAU_PB comment.
2026-07-16 09:30:50 -04:00
Harwinder Sekhon 18b5ef27d2 Docs: rewrite CLAUDE.md as a complete codebase reference
Replaces the summary-level doc with a full spec derived from a read of
every core module, so contributors (and low-context models) don't have
to infer behavior or rediscover tuned values:

- repo map with line counts; flags DOCUMENTATION.md/USAGE.md as stale
- every config knob with its real default and the backtest evidence
  behind it; marks the legacy/unused fields (SignalConfig.weights,
  min_confidence, cooldown_bars, XAU h4/h1 gates) so nothing new is
  built on dead knobs
- exact indicator logic for all 9 scored indicators + ATR context,
  regime classification rules, regime-aware RSI zone tables, and the
  regime-multiplier table copied from _compute_composite (the old
  table did not match the code)
- aggregator pipeline in source order: MTF agreement, the 12 penalties
  and the diminishing cap, exhaustion block, signal/grade thresholds,
  and gates 6b-6i with exact numbers
- all three trade-setup generators with SL/TP geometry, spread gates,
  and the trailing-exit model
- bot/CLI/news/data-provider/backtester mechanics incl. realism model
  (raw-path resolution, next-open entry, SL-first ties, parallel replay)
- "Rules for making changes": validation acceptance rule, pair
  isolation, live==backtest parity, ROC's load-bearing weight

Also corrects details the old doc got wrong: test count is 179 (not
~176), EUR ships off by default, backtest/*.csv are gitignored local
artifacts rather than test-enforced pins.
2026-07-16 09:26:06 -04:00
Harwinder Sekhon 26f0681ad1 Docs: user guide + beginner guide PDFs
Two printable guides in docs/ (HTML source + weasyprint-rendered A4
PDFs): a trader-facing user guide (signal anatomy, trail management,
M5/M15/H1 comparison from the 12mo books, expectations, commands) and
a plain-language beginner edition (glossary, worked sizing example,
step-by-step trailing-stop walkthrough, H1-first guidance).
2026-07-12 13:42:21 -04:00
Harwinder Sekhon dde41fbeb8 Audit fixes: mixed-regime bias, live==backtest session parity, parallel replay
Fix 1 — mixed-"trending" regime (ADX strong, EMAs unaligned) was
hard-coded bullish in 5 sites: RSI reinterpretation used the uptrend
playbook (RSI 38-55 -> STRONG_BUY conf 75), BUY-only confidence
bonuses, BUY-only extension penalties, no counter-trend penalty.
Now: extremes-only RSI (as transitioning), no directional bonuses,
symmetric extension penalties. 6yr validation: H1 +67.2->+61.9R
(mixed skew 16B/11S -> 4B/7S), H4 +31.6->+27.5R (the 4 bias BUYs),
M15 +152.6->+154.7R, M5 +207.1->+206.1R with maxDD 28.7->24.3R.
The ~8R aggregate give-back is bull-market subsidy from the bug.

Fix 2 — live session filter now applies the EXACT replay-loop
predicate (session window + blocked hours on the signal bar's CLOSE
time, from the bar timestamp not wall clock). Replaces a wall-clock
filter gated on a "XAU STRATEGY" marker that no code writes, which
made live diverge from every validated book.

Fix 3 — delete dead _check_xau_oversold_bounce and
_generate_xau_trade_setup (unreachable; the latter hardcoded BUY and
skipped the spread/RR gates).

Backtester: parallel bar replay via forked workers (--workers,
default auto = cores-2 capped 12). Per-bar evaluation is pure; the
entry cooldown is applied in an ordered phase-2 pass. Proven
byte-identical to sequential on the same window; ~300 -> 1030-1680
bars/s (6yr M5: ~26min -> 5-8min). --workers 1 = sequential path.

Tests: 179 passed (new mixed-regime RSI regression test).
2026-07-11 22:57:37 -04:00
Harwinder Sekhon 6f69b9dd90 XAU: two validated per-TF gates — M15 mixed-regime block, M5 hour-9 block
M15 mixed-"trending" regime block (ADX strong, EMAs unaligned =
momentum without direction): -6.8R/24 trades on 12mo, negative both
halves; blocking scores +65.7R/499 PF 1.38 DD 10.4R vs +59.0R/521
PF 1.32 DD 12.6R, better both halves, OOS 2024-06->2025-03 neutral
(-1.4R = the 4 such trades there). M5 never trades this regime.
New XauStrategyConfig.blocked_regimes / sell_blocked_regimes knobs
(default empty) + aggregator gate 6g2; M15 factory opts in.

M5 hour-9 UTC block (post-London-open whipsaw): -28.9R/143 over the
6-year book, negative 5/7 calendar years. Full-history replay with
block: +207.1R vs +178.3R, max DD 28.7R vs 35.5R, 2023 -13.0R ->
breakeven; tuned 12mo book unchanged (139.0 vs 139.3). Hour 9 is
positive on M15 (+1.7R 6yr, +8.6R OOS) so the block is factory
M5-only.

Rejected after validation (documented in xauusd.py + session memory):
transitioning-SELL block (bull-market artifact, +5.7R in 2024 OOS),
M15 trending_down block (M5 cross-TF contradiction), losing-streak
circuit breakers, 30d efficiency-ratio trend gate, and all entry-level
indicator filters (winners == losers at entry on every feature).

Full-history context (Twelve Data 5min starts 2020-03-16): M5 6.3yr
+207R PF ~1.3 post-fix with losing year 2021; M15 +152.6R PF 1.19
DD 49.7R — edge is regime-dependent, chop years are structural.

Tests: 178 passed (new gate test + factory-equivalence updates).
2026-07-11 21:38:42 -04:00
Harwinder Sekhon a2bf80054c Telegram: subscriber channel feed, owner lock, full risk disclaimer
- TELEGRAM_CHANNEL_ID mirrors auto-scan signal + breach alerts to a
  subscriber channel (Stars-subscription monetization; Telegram manages
  membership). Feed keeps running even when the owner's DM auto-scan
  is toggled off.
- _owner_only decorator locks all commands/buttons to TELEGRAM_CHAT_ID;
  others get a refusal pointing at the channel.
- Every signal-bearing message (detailed signal, scan summary, all-TF
  summary, breach alerts) now ends with a not-financial-advice /
  educational-use disclaimer via a single _DISCLAIMER constant.
2026-07-11 18:37:50 -04:00
Harwinder Sekhon 60fb526fab EUR: document closure of calendar + H1-hybrid avenues
Surprise-conditioned entries lose 2-5p at every horizon both years;
avoidance filter marginal for EUR and profit-removing for XAU; the
H1-direction + M15-pullback prototype failed its pre-registered kill
criteria (2024-25 OOS negative). EUR stays informational-only.
2026-07-11 10:26:32 -04:00
Harwinder Sekhon 18abff85a3 XAU M15: adopt 1R minimum targets (per-timeframe, validated)
Finishes PENDING_STEPS Step 1 under the current trailing-exit model.
M15 window 2025-03-30->2026-05-31 ($0.30 spread): +116.6R/460 PF 1.83
vs +105.4R/424 PF 1.82 baseline, better in both halves; 2024-06->
2025-03 OOS +30.3R vs +10.6R control. Same knob on the pinned M5 book
is WORSE (+131.3R/699 vs +139.9R/547) so the override is M15-only in
the xauusd factory, per the pre-registered decision rule. M5 path
unchanged (factory override gated on primary_tf == M15).
2026-07-11 10:22:33 -04:00
Harwinder Sekhon 183f148bb4 Live news gate: stop capping signal confidence near events (XAU default)
Replaying the live danger/caution gates against the validated 12mo XAU
M5 book showed they suppress 25-35 trades worth +4.6..+7.2R/yr — trades
entered near USD releases outperform the book average (+0.475R vs
+0.256R). New NewsConfig.event_risk_caps_signals (default False =
XAU-tuned) keeps EVENT RISK warnings in alerts but no longer guts
confidence; EUR factory re-enables the caps (evidence is gold-specific).
Pinned XAU M5 regression: 547 trades, +139.94R, trade-identical.
Tests: 177 passed.
2026-07-11 10:02:27 -04:00
Harwinder Sekhon 528a308d36 Add calendar research infra: FF scraper + verified event store
Scrapes Forex Factory weekly pages (epoch datelines, actual/forecast/
impact/betterWorse) into backtest/cache/calendar.parquet (gitignored;
rebuild with the scraper). Timestamps verified: NFP/CPI/ECB times exact
across DST, and high-impact event bars average 3.5x ATR with the spike
vanishing at +/-1h shifts. calendar_store provides precise_events and
event_window_mask for screens and counterfactuals.
2026-07-11 09:47:11 -04:00
Harwinder Sekhon 9c72e90ce2 EUR M15/H1 investigation: document no-validated-edge verdict
Five independent tests at 1-pip cost (M15 wide-open both windows,
transitioning-slice OOS, mean-reversion trade sims, forward-return
screens on M15+M5, H1 both windows) — nothing clears cost in both
2024-25 and 2025-26. Root cause: spread is ~10% of M15 ATR on EUR.
EUR stays out of scan_pairs; open avenues documented in docstring.
2026-07-11 09:02:24 -04:00
Harwinder Sekhon 8b086f4f49 EUR M5: commit best-found config after realism re-tune — no validated edge
12mo grid campaign (2025-07-08 → 2026-07-08, 1-pip round-trip cost,
train/val split at 2026-01-08). Donor config scored -6.7R/34 trades.
Findings: transitioning regime -27R/88 wide-open; A-grade entries lose
(-26R) while B-grade wins; Fridays -17R; hours 8/12/13/18/19 UTC bleed;
score 30-35 band holds the train-half losses; trailing (0.45R arm/0.5R
dist) beats static TP1 and gold's 0.6/0.75.

Best-found config: -1.06R / 53 trades / PF 0.93 / 62.3% WR — breakeven,
FAILS acceptance (positive both halves, PF>=1.3). Per plan, EUR ships
functional but stays out of default scan_pairs. Grid results and
methodology documented in the module docstring; full numbers in
backtest/cache/eurv2_*.csv.
2026-07-10 22:14:49 -04:00
Harwinder Sekhon f02f4b4d37 Update CLAUDE.md for multi-pair architecture 2026-07-10 21:16:34 -04:00
Harwinder Sekhon 74829541d6 Live path: config-driven multi-pair scanning (Telegram bot + CLI)
- telegram_bot: SCAN_PAIRS/_WS_TO_PAIR/SCAN_BUTTONS all derive from
  PlatformConfig.scan_pairs (the single pair on/off switch); buttons and
  keyboards are generated per pair; _run_signal_sync builds per-pair
  configs via pair_signals factories (was: XAU defaults for any symbol);
  scan-all works per pair; PB badge matches any *_PB tag; daily-rotating
  file logs in logs/telegram.log (30-day retention)
- main.py: --pair argument (XAUUSD | EURUSD + aliases) routed through
  pair_signals factories; price display is precision-aware (2dp gold,
  5dp FX)

scan_pairs default stays [XAUUSD] until the EUR M5 re-tune validates.
2026-07-10 21:13:42 -04:00
Harwinder Sekhon 6546b89d6b Add EUR/USD test suite + map EUR news events to EUR/USD
Module 10 (21 tests): EUR config/registry/factory, EUR filter pipeline
(regime/quality/score/confidence/RSI-extended/weekend), EUR trade
geometry (swing SL buffers, pip-denominated spread gate, per-pair gate
contrast), and cross-pair isolation regressions (XAU report bit-equal
with eur_strategy flipped; XAU gates never fire on EUR reports).

news_engine: _get_affected_pairs now maps USD events to EUR/USD too and
adds an EUR entry (calendar warnings reach EUR scans).

175/175 pass.
2026-07-10 21:06:55 -04:00
Harwinder Sekhon 9ed07d4b65 Backtester: EUR/USD support with per-pair configs and default spread
- EURUSD aliases in PAIR_ALIASES
- main() builds PlatformConfig via pair_signals.PAIR_PRODUCERS factories
  (per-pair regime weights/gates/sessions); unregistered pairs keep the
  inline construction. XAU factory is asdict-identical to inline
  (enforced by test) so XAU runs are unchanged.
- --spread default now resolves from PAIR_INFO default_spread
  (XAU $0.30, EUR 0.0001) instead of a hardcoded gold value
- EUR strategy tag: renamed EUR_PB→EUR, detected in both bullish and
  bearish reasons[0] (SELL setups mark the bearish list)

Smoke-validated: 30-day EUR M5 run produces EUR-tagged trades with
1-pip round-trip cost charged.
2026-07-10 21:02:44 -04:00
Harwinder Sekhon 0da5ed03af Add EUR/USD signal gates, trade geometry, and weekend/session filters
Re-grafted from feature/eurusd-support onto the evolved aggregator:
- 6h EUR gate chain (regime, quality, exhaustion, EMA200, session,
  score/confidence, volatility, indicator agreement, RSI-extended),
  active only for EUR/USD with eur_strategy.enabled
- _generate_trade_setup: EUR branch reads EurUsdStrategyConfig only
  (min_sl_pips, min_rr, mature-move + quality-scaled TP, sl_atr_floor,
  swing-buffer SL) with a pip-denominated spread gate; adds a
  pip_value<=0 guard; non-EUR branch is arithmetically unchanged
  (sl_swing_buffer=1.0 extraction only)
- EUR weekend/weekday pre-filter in analyze_pair mirroring the XAU block
- "EUR STRATEGY" reasons[0] marker so the backtester tags EUR trades
2026-07-10 20:57:54 -04:00
Harwinder Sekhon db50bf02d7 Gate all XAU-only aggregator logic by pair
The SELL quality gate (6e), SELL volatility gate (6g), M5 noise gates
(6h), ATR trade-setup generator, and M5 pullback archetype are all
gold-tuned but previously keyed only on xau_strategy.enabled or the
timeframe, so they would have silently applied gold constants to any
other pair. Each now also requires display_name == "XAU/USD".

No behavior change for XAU (conditions are tautologically true for the
only registered pair today) — verified by test suite and pinned-window
backtest CSV diff.
2026-07-10 20:53:23 -04:00
Harwinder Sekhon 1bc48915bd Add pair_signals per-pair producer registry
Ported from feature/eurusd-support: PAIR_PRODUCERS registry with
make_config() factories per pair. xauusd.make_config is a pure
PlatformConfig-defaults wrapper (equality with the inline construction
is enforced by test); eurusd.make_config carries the donor branch's
regime weights and gates as a pre-realism baseline.

Adds 4 registry tests. 154/154 pass.
2026-07-10 20:52:06 -04:00
Harwinder Sekhon 9d3729ea73 Add EUR/USD pair constants and EurUsdStrategyConfig
- EURUSD in PAIR_INFO; new per-pair price_precision and default_spread
  fields on both pairs (backtester consumes default_spread when --spread
  is omitted)
- EurUsdStrategyConfig ported from feature/eurusd-support plus EUR-local
  cost knobs (round_trip_cost=1 pip, min_tp_cost_multiple=10,
  min_sl_pips=8) so SignalConfig's gold-dollar values never apply to EUR;
  donor-tuned values are a pre-realism baseline pending re-tune
- PlatformConfig.eur_strategy field; track EUR in news currencies
2026-07-10 20:50:08 -04:00
Harwinder Sekhon f3b90d3862 Fix 3 engine bugs: swing-point NaN false positives, ROC div-by-zero, ADX epsilon
Ported from feature/eurusd-support (e6e09c5), pair-neutral fixes:
- _find_swing_points: all(<empty generator>) is vacuously True, so a value
  surrounded by NaNs was reported as a swing point; now requires at least
  one valid neighbor on each side
- ROC: guard against zero close at the lookback positions (inf scores)
- ADX: epsilon denominator instead of NaN-replace to avoid NaN cascade

Adds 8 regression tests (Module 9). 150/150 pass.
2026-07-10 20:48:56 -04:00
Harwinder Sekhon ea306df484 Overhaul news engine: fix false signals, add surprise detection + caching
Improvements from a live-verified audit of the news/calendar pipeline:

- Currency tagging: keyword matching now uses letter-boundary regexes
  and drops ambiguous "us "/"eu "/"uk " tokens — "versus" no longer
  tags headlines as USD (confirmed live false positive polluting the
  XAU/USD sentiment average).

- Fetch caching: calendar events and headlines are cached class-level
  (10 min TTL, 60s retry after empty/failed fetch, lock-guarded for
  concurrent scans). Stops per-scan refetching that was tripping the
  Forex Factory rate limit; TTL stays under the 30-min pre-event
  buffer so the danger window can't go stale.

- Surprise detection: EconomicEvent.surprise is computed from parsed
  actual vs forecast (handles %, K/M/B/T suffixes, negatives). A
  surprise print widens the post-release caution window to 45 min;
  inline releases keep the original 15 min.

- Gold-aware sentiment: headlines are scored per currency, with
  monetary-policy keywords (hawkish/rate hike/tightening vs dovish/
  rate cut/easing/stimulus) inverted for XAU — "hawkish Fed" headlines
  scored +1.0 for gold before, -1.0 now (verified on live feed).

- New sentiment sources: Google News gold-focused query and CNBC
  Economy RSS (verified live; 40 → 74 unique headlines). Added
  cross-feed title dedup so syndicated stories count once.

- events_today now buckets by UTC date (feed timestamps carry US-
  Eastern offsets); --json output includes news_description.

- Removed dead NewsConfig.min_impact and unused HIGH_IMPACT_EVENTS.

Tests: 142 passed (7 new covering boundary matching, surprise
classification, post-event windows, cache reuse, gold inversion).
2026-07-10 09:43:11 -04:00
harwinder 1038a53bac Add M5 pullback entry archetype (XAU_PB): +140R combined on 12mo
Second entry pattern alongside the Keltner-breakout: enter on the
retrace to EMA21/KC-mid in an established trend (EMA >=3/4 aligned,
ADX>=20, retrace 0.8-6.0 ATR, resumption bar closed back in trend
direction, RSI in the pullback zone not the extended zone). Detector
lives in the aggregator so live and GPU-backtest paths share one
implementation. Own gates (composite floor 15, confirmation-TF
agreement, MTF opposition, vol regimes) with a diagnostic funnel.

PB stops anchor to the pullback swing (~$8 vs ~$25 breakout), so PB
trades get their own trail arm (0.8R vs 0.6R global) via per-trade
trail overrides in the backtester and strategy-aware Telegram alerts.

12mo M5 validation: PB alone +38.3R/254 trades, positive both halves
and all four full quarters; core breakout trades undisturbed
(+101.9R vs +105.0 baseline). Combined book: 549 trades (~10.6/wk),
+140.1R, PF 1.78, DD 11.3R vs 311 trades/+105.0R before.
2026-07-09 14:21:59 -04:00
harwinder 305f1d1857 Add pullback entry pattern implementation plan
Design doc for a second M5 entry archetype: enter on retrace to
EMA21/KC-mid in an established trend instead of at Keltner-breakout
extension. Includes detector spec, geometry, validation protocol
(both-halves + quarterly + displacement checks), and the +105R
baseline to beat. Session/frequency knobs (blocked hours, cooldown)
were tested 2026-07-09 and confirmed already optimal — this is the
remaining evidence-backed entry-side direction.
2026-07-09 13:51:00 -04:00
harwinder 0393087168 Add M5 gates 5-6 and trailing-stop exits (+48R → +105R on 12mo)
Gates 5-6 in the M5 filter: block 'high' vol regime entries (chop
transition — 61% WR, negative total while low/normal/extreme all
positive) and block SELLs with RSI<25 (dip already extended, mean-
reverts before target). 365d: 419→311 trades, +41→+48.4R, DD 11.2→8.0R.

Replace the fixed RR-0.5 take-profit with a trailing stop: arm at
+0.6R favorable excursion, trail 0.75R behind best price, stop levels
update at bar close (conservative). The RR-0.5 target was the binding
constraint — winners kept running after TP. Whole trail parameter grid
(arm 0.4-1.2 × dist 0.5-1.25) lands +90-103R, a plateau not a spike;
also +91.7R on the pre-gate 419-trade population. TP1 stays advisory;
Telegram alerts include the arm price and trail distance. max_hold
200→300 bars (trailing exits need room; +10R).

12mo M5, same 311 trades: +105.0R total, PF 2.27, DD 7.9R, 61% WR,
train/val +54.6/+50.4, every quarter positive.
2026-07-09 13:34:57 -04:00
harwinder 8687f98b39 Fix M5 signal quality: ATR ordering bug, noise gates, blocked-hour leak (+41R)
ATR was computed after EMAs so ribbon compression detection was dead code
on all timeframes — moved ATR first. Extended M5 swing S/R lookback from
50 to 120 bars. Added four M5-specific filters: min score, MTF agreement,
confirmation TF conviction, and SELL quality cap (A-grade M5 shorts enter
at dip extremes and lose). Fixed backtester blocked-hour check to use bar
close time matching the live bot. Added trade geometry config (spread gate,
SL floor, TP floor RR) and primary_tf mismatch guard in backtester.

Backtest: 1133 trades/+11.5R → 419 trades/+41.0R, PF 1.03→1.37, DD 21→11R.
Train/val split positive in both halves (+19.8R/+21.3R).
2026-07-09 12:48:36 -04:00
harwinder 2291fc3936 Disable TP tightening: tuned pre-spread, halves total R under real fills
A/B on M15 2025-03-30→2026-05-31 with $0.30 spread + gap-through fills
(511 identical entries, only exits differ):
  factor 0.8 / min_rr 0.4: +36.1R (avg +0.071R, SELL +1.2R)
  factor 1.0 / min_rr 0.5: +66.8R (avg +0.131R, SELL +9.6R)
Worse in both train and validation halves and both directions. The 20%
tighter targets buy +1.8pt win rate but shrink every win while the fixed
spread stays constant. The knob remains for re-tuning with costs included.
2026-07-08 21:35:59 -04:00
harwinder 3a85c6e23b Add WebSocket dual-connection test script
Probes whether Twelve Data allows two simultaneous WS connections
for one symbol (SDK-based, 30s run, prints per-connection verdict).
2026-07-08 21:32:46 -04:00
harwinder 09ee5208d4 Telegram bot: WS heartbeat, breach alerts, alert dedup, interactive menu
WebSocket:
- Send heartbeat every 10s — Twelve Data drops idle connections, so the
  monitor previously died on weekends/quiet hours and churned reconnects
- Log failed symbol subscriptions from subscribe-status events
- Stamp fresh live tick price + age into every alert message

Alerts:
- Re-enable level-breach alerts: WS breach triggers M15 rescan and sends
  the report when actionable (was scanning and discarding the result)
- Dedup: identical (signal, quality, direction) per pair/TF is sent once,
  re-armed after 1h — no more repeat alerts every candle
- Configurable minimum alert quality (C+/B+/A), cycled from the menu
- Trade setup shows dollar risk/reward alongside pips

Interactive menu (/start, /scan):
- Timeframe scans, Scan All, auto-scan ON/OFF toggle, alert-quality
  cycle, status, and help as inline buttons
- Markdown-safe edits: 'message not modified' swallowed, parse failures
  fall back to plain text; exception text escaped
2026-07-08 21:32:46 -04:00
harwinder 1f076c0b54 Fix live/backtest parity, SELL-side losses, and indicator repairs (+42% total R)
Live pipeline:
- Drop still-forming candle before analysis (signals no longer repaint)
- Direction-aware MTF agreement + opposition gate (no trade against 2-of-3 TF consensus)
- SELL volatility gate: block shorts in low/normal vol regimes (config sell_vol_regimes)
- Divergence: 40-bar lookback, price/indicator swings must align within ±2 bars
  (fired on 0.5% of trades before, 29% after), trending weight 2.0→1.0
- RSI regime_aware by default; transitioning regime only signals at extremes
- ADX ±DM tie no longer nudges bearish; TP2 clamped beyond TP1

Backtester realism:
- Charge $0.30 round-trip spread (--spread), gap-through SL/TP fills at bar open
- Resolve trades on RAW price path (cache stores raw bars, cleaning runs on load)
- Weekend/session filters use bar-close decision time to match live wall clock

Validated per-step on M15 2025-03-30→2026-05-31 under identical pricing:
+46.9R → +66.8R, win rate 69.9% → 73.0%, SELL side -2.3R → +9.6R.
Tested-and-reverted: ROC weight removal (-13R), BB fade suppression (-11.6R),
candlestick de-weight (-2R) — composite thresholds are co-tuned with weight mass.
2026-07-08 21:32:46 -04:00
harwinder baeee4997a Fix trade setup generation broken by TP tightening factor
The TP floor calculations used raw min_rr (0.4) but after 20% tightening
the effective RR dropped to ~0.32, failing the min_rr check and returning
None for all trade setups. Pre-compensate the floor with min_rr/tf so
post-tightening RR still meets the minimum.
2026-05-24 18:22:57 -04:00
harwinder bb40bceb4c Tighten TP targets by 20% for faster hits (configurable tp_tightening_factor=0.8)
Applied uniformly across all 3 trade setup functions. Lowered min_rr_ratio
0.5→0.4 proportionally to preserve trade acceptance rate. SL unchanged.
2026-05-22 10:44:52 -04:00
Harwinder c46bb85012 Enable M5 auto-scan alerts and remove live dashboard
M5 already broadcast through the shared _scan_timeframe path but was
documented as data-only (docstring, CLAUDE.md, DOCUMENTATION.md, plus a
phantom `if tf == M5: return` step that never existed in code). Align all
docs/comments so M5 alerting at C+ grade is the intended, documented behavior.

Remove the live dashboard entirely per request: the 30s refresh loop,
per-scan refresh, boot send, and the dashboard-only stats layer
(_signal_stats / _stats_updated / _update_signal_stats and its call sites,
_format_stats_dashboard and its helper constants). Drop now-unused ZoneInfo
import and dashboard mentions from help/autoscan/boot messages and the test
suite. The WebSocket level-breach monitor and level cache are untouched.

Tests: 136 passed, 0 failed.
2026-05-20 12:45:27 -04:00