Accuracy audit fixes: alerts carry validated per-TF trail plans + 13 more

Five-agent audit of the signal path; all fixes are delivery/measurement
side. Regression-proven behavior-neutral on validated signals: the 5yr
M15 replay reproduces the pinned book exactly (+210.65R PF 1.35
WR 57.8% MaxDD 33.90R). Tests 215 -> 221.

- TradeSetup carries trail_arm_r/trail_dist_r stamped from the producing
  config; Telegram trail plan reads them (was showing default 0.6/0.75
  instead of the validated M15 arm 0.70 / M5 dist 0.85)
- Alert dedup: fingerprint = (signal, quality, direction); entry movement
  compared against last-ALERTED entry/risk (old SL-dist bucket churned on
  ATR drift -> near-duplicate alerts every candle)
- Fresh fetches enforce the closed-bar boundary check with one 3s grace
  retry, then warn (feed lag no longer silently analyzes the prior bar)
- Backtest max DD includes the zero baseline (opening losing streaks
  were undercounted; pinned M15 DD unchanged)
- Replay loads the regime-filter TF when the H4-ADX gate is on (was a
  silent no-op for M5/M15 primaries)
- clear_cache under _cache_lock; dead ROC zero-cross branches removed
  from both engines (provably unreachable); RSS dates via
  parsedate_to_datetime (named zones no longer stamped UTC); event risk
  grades from the soonest event; .env inline comments stripped;
  auto-scan skips NEUTRAL like the breach path; report_to_dict honors
  the weight-0 skip rule and emits strategy/trail params; PB funnel
  counts spread_gate/accepted; session-filter fail-open paths log loudly
- Docs corrected to match validated code (H&S neckline = lower peak,
  exhaustion anchors farthest tracked swing) with re-validation warnings

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R5soRUeRBi8kHMB99csrJy
This commit is contained in:
Harwinder Sekhon
2026-07-23 10:31:13 -04:00
parent 91b4a6bb5e
commit 89734c3a87
11 changed files with 329 additions and 68 deletions
+7 -7
View File
@@ -130,7 +130,7 @@ Results land in `TechnicalAnalysis.indicators: Dict[str, IndicatorSignal]` (fiel
| `macd` | 1.5 | Histogram = MACD line − signal line. Cross above 0 → BUY (conf 55+); accelerating (hist>prev>0) → STRONG_BUY; mirrored down; positive-but-fading / negative-but-fading → NEUTRAL (35); flat → NEUTRAL (30). Confidence scaled by |hist| vs its 20-bar mean | | `macd` | 1.5 | Histogram = MACD line − signal line. Cross above 0 → BUY (conf 55+); accelerating (hist>prev>0) → STRONG_BUY; mirrored down; positive-but-fading / negative-but-fading → NEUTRAL (35); flat → NEUTRAL (30). Confidence scaled by |hist| vs its 20-bar mean |
| `bollinger` | 1.5 | Position = (close−lower)/(upper−lower). >1.0 SELL (65), <0.0 BUY (65), >0.8 SELL (45), <0.2 BUY (45), else NEUTRAL (30). Squeeze = width < 0.75 × 20-bar avg width → +10 conf. Emits `bb_upper/lower/mid` | | `bollinger` | 1.5 | Position = (close−lower)/(upper−lower). >1.0 SELL (65), <0.0 BUY (65), >0.8 SELL (45), <0.2 BUY (45), else NEUTRAL (30). Squeeze = width < 0.75 × 20-bar avg width → +10 conf. Emits `bb_upper/lower/mid` |
| `adx` | 0.5 | Wilder ADX/DI (ties zero BOTH DMs — fixes a systematic bearish nudge). ADX ≥ 25 → BUY/SELL by DI dominance, else NEUTRAL. Conf 40/65/80 by ADX band. ADX slope (5-bar diff): <25 with slope > 3.0 → "TREND EMERGING" +15 conf; also strengthening/weakening adjustments. Sets `_adx_slope` | | `adx` | 0.5 | Wilder ADX/DI (ties zero BOTH DMs — fixes a systematic bearish nudge). ADX ≥ 25 → BUY/SELL by DI dominance, else NEUTRAL. Conf 40/65/80 by ADX band. ADX slope (5-bar diff): <25 with slope > 3.0 → "TREND EMERGING" +15 conf; also strengthening/weakening adjustments. Sets `_adx_slope` |
| `roc` | 1.5 | 10-period %change. Positive & rising → BUY (STRONG if >2× threshold=2%); negative & falling → SELL; zero-cross → BUY/SELL (65); else NEUTRAL. **Directionally redundant with MACD/Keltner/EMA (0.8–0.94 corr) but its weight is load-bearing: zeroing it backtested 13R WORSE — composite thresholds are calibrated with it. Do not remove without retuning thresholds** | | `roc` | 1.5 | 10-period %change. Positive & rising → BUY (STRONG if >2× threshold=2%); negative & falling → SELL (zero-crosses land in these branches by definition — a separate cross branch was provably dead and removed 2026-07-23); else NEUTRAL. **Directionally redundant with MACD/Keltner/EMA (0.8–0.94 corr) but its weight is load-bearing: zeroing it backtested 13R WORSE — composite thresholds are calibrated with it. Do not remove without retuning thresholds** |
| `keltner` | 1.5 | EMA20 mid ± 1.5×ATR(10). TTM squeeze = BB fully inside KC. Squeeze released (was→not) + MACD hist sign → STRONG_BUY/STRONG_SELL (80); squeeze active → NEUTRAL (60); close above/below channel → BUY/SELL continuation (65); inside → NEUTRAL (30). Emits `kc_upper/lower/mid` | | `keltner` | 1.5 | EMA20 mid ± 1.5×ATR(10). TTM squeeze = BB fully inside KC. Squeeze released (was→not) + MACD hist sign → STRONG_BUY/STRONG_SELL (80); squeeze active → NEUTRAL (60); close above/below channel → BUY/SELL continuation (65); inside → NEUTRAL (30). Emits `kc_upper/lower/mid` |
| `divergence` | 1.5 | RSI + MACD-hist vs price over 40 bars. Swing points (window 3) must align within ±2 bars to pair up; last two pairs: price LL + indicator HL = bullish, price HH + indicator LH = bearish. Any found → BUY/SELL/NEUTRAL-if-both, conf 70, value = count | | `divergence` | 1.5 | RSI + MACD-hist vs price over 40 bars. Swing points (window 3) must align within ±2 bars to pair up; last two pairs: price LL + indicator HL = bullish, price HH + indicator LH = bearish. Any found → BUY/SELL/NEUTRAL-if-both, conf 70, value = count |
| `candlestick` | 1.0 | Last bar: Doji (body<10% range), Hammer/Hanging Man (lower wick>2×body), Shooting Star/Inverted Hammer, Engulfing (vs prev bar), Marubozu (body>85%). Majority vote → signal, conf 50; +20 if pattern sits within 0.5 ATR of a swing S/R level | | `candlestick` | 1.0 | Last bar: Doji (body<10% range), Hammer/Hanging Man (lower wick>2×body), Shooting Star/Inverted Hammer, Engulfing (vs prev bar), Marubozu (body>85%). Majority vote → signal, conf 50; +20 if pattern sits within 0.5 ATR of a swing S/R level |
@@ -185,7 +185,7 @@ Two entry points:
6. **`_compute_final_signal`** (bonuses → event cap → penalties → thresholds → grade): 6. **`_compute_final_signal`** (bonuses → event cap → penalties → thresholds → grade):
- Bonuses: MTF agreement → +10 conf and `score = 0.6×primary + 0.4×mean(all TF scores)`; news sentiment aligned (|sent|>0.2) → +5 (gated by `news.sentiment_affects_signals`, False by default); in a *directional* trend, RSI in pullback zone → +5, BB position favorable → +5 (mixed `trending` regime gets NO directional bonus). - Bonuses: MTF agreement → +10 conf and `score = 0.6×primary + 0.4×mean(all TF scores)`; news sentiment aligned (|sent|>0.2) → +5 (gated by `news.sentiment_affects_signals`, False by default); in a *directional* trend, RSI in pullback zone → +5, BB position favorable → +5 (mixed `trending` regime gets NO directional bonus).
- Event risk `danger`: warning always; confidence hard-capped at 30 ONLY if `news.event_risk_caps_signals` (False by default — XAU-tuned). - Event risk `danger`: warning always; confidence hard-capped at 30 ONLY if `news.event_risk_caps_signals` (False by default — XAU-tuned).
- **Penalty system (diminishing, replaces old additive stacking):** collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: `mtf_disagreement` 15; `news_conflict` 10 (gated by `sentiment_affects_signals`, off by default — warning still emitted); `event_caution` 15 (config-gated); `counter_trend` 15; `extreme_vol` 10; `ema200_near` 5 (price within 0.5% of EMA200 — env `FB2_EMA_LO` overrides); `ema200_far` 5 (>2.5%); `exhaustion` 8 (price >3.5 ATR beyond nearest swing); `no_pullback` 8 (>2.5 ATR from EMA21 in trend) / `weak_pullback` 5 (>2.0); `rsi_extended` 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed `trending`); `bb_overextended` 8 (BB pos >0.85 BUY / <0.15 SELL). - **Penalty system (diminishing, replaces old additive stacking):** collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: `mtf_disagreement` 15; `news_conflict` 10 (gated by `sentiment_affects_signals`, off by default — warning still emitted); `event_caution` 15 (config-gated); `counter_trend` 15; `extreme_vol` 10; `ema200_near` 5 (price within 0.5% of EMA200 — env `FB2_EMA_LO` overrides); `ema200_far` 5 (>2.5%); `exhaustion` 8 (price >3.5 ATR beyond the FARTHEST of the 3 tracked swing levels — fires earlier than a nearest-swing rule would; calibrated as-is in every validated book); `no_pullback` 8 (>2.5 ATR from EMA21 in trend) / `weak_pullback` 5 (>2.0); `rsi_extended` 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed `trending`); `bb_overextended` 8 (BB pos >0.85 BUY / <0.15 SELL).
- **Exhaustion hard block**: ≥2 exhaustion-tagged penalties ({exhaustion, no_pullback, rsi_extended, bb_overextended} → `report.exhaustion_count`) AND momentum fading (ROC sign against score, or MACD signal against score) → confidence = 0. - **Exhaustion hard block**: ≥2 exhaustion-tagged penalties ({exhaustion, no_pullback, rsi_extended, bb_overextended} → `report.exhaustion_count`) AND momentum fading (ROC sign against score, or MACD signal against score) → confidence = 0.
- Confidence clamped [0, 95]. **Final signal**: score ≥35 & conf ≥55 STRONG_BUY; ≥25 & ≥50 BUY; mirrored sells; else NEUTRAL. - Confidence clamped [0, 95]. **Final signal**: score ≥35 & conf ≥55 STRONG_BUY; ≥25 & ≥50 BUY; mirrored sells; else NEUTRAL.
- **Quality grade**: |score|≥40 & conf≥65 & MTF → **A+**; ≥33 & ≥55 & MTF → **A**; ≥28 & ≥50 → **B**; ≥25 & ≥45 → **C**; ≥15 → **D**; else **F**. - **Quality grade**: |score|≥40 & conf≥65 & MTF → **A+**; ≥33 & ≥55 & MTF → **A**; ≥28 & ≥50 → **B**; ≥25 & ≥45 → **C**; ≥15 → **D**; else **F**.
@@ -218,7 +218,7 @@ Two entry points:
**D. `_try_chart_pattern` (XAU_CP)**: detection in `chart_patterns.py` — 6 classical patterns (double_bottom/double_top, inv_hs/hs_top, bull_flag/bear_flag), swing-point based (strict local extrema, window `cp_swing_window`=3), all thresholds in ATR units, scanned over the last `cp_lookback`=72 closed bars. Trigger = FIRST close beyond the neckline/boundary since the pattern formed (one breakout → one signal, no repaint); reversal patterns are invalidated by a new extreme beyond the pattern. No composite-score gate BY DESIGN. Gates: MTF non-opposition (6f mirror); SELL patterns respect `sell_vol_regimes`, `sell_blocked_hours_utc` (bar-close hour), M5 RSI floor; M5 blocks `m5_blocked_vol_regimes` both ways; ATR ≥ $1. Geometry: SL = pattern anchor ∓ 0.5 ATR (min $8); TP1 = measured move (pattern height), TP2 = 1.5×height; RR ≥ 0.5; spread gate $9. Sets `strategy="XAU_CP"` (global trailing params), grade min B, marker `"XAU CHART PATTERN: …"` at reasons[0]. Funnel in `SignalAggregator.cp_funnel`. **D. `_try_chart_pattern` (XAU_CP)**: detection in `chart_patterns.py` — 6 classical patterns (double_bottom/double_top, inv_hs/hs_top, bull_flag/bear_flag), swing-point based (strict local extrema, window `cp_swing_window`=3), all thresholds in ATR units, scanned over the last `cp_lookback`=72 closed bars. Trigger = FIRST close beyond the neckline/boundary since the pattern formed (one breakout → one signal, no repaint); reversal patterns are invalidated by a new extreme beyond the pattern. No composite-score gate BY DESIGN. Gates: MTF non-opposition (6f mirror); SELL patterns respect `sell_vol_regimes`, `sell_blocked_hours_utc` (bar-close hour), M5 RSI floor; M5 blocks `m5_blocked_vol_regimes` both ways; ATR ≥ $1. Geometry: SL = pattern anchor ∓ 0.5 ATR (min $8); TP1 = measured move (pattern height), TP2 = 1.5×height; RR ≥ 0.5; spread gate $9. Sets `strategy="XAU_CP"` (global trailing params), grade min B, marker `"XAU CHART PATTERN: …"` at reasons[0]. Funnel in `SignalAggregator.cp_funnel`.
`TradeSetup` fields: pair, direction, entry, SL, TP1, TP2, RR, risk/reward pips, pip_value, `strategy` ("GENERIC", "XAU_PB", or "XAU_CP" — selects trailing params; XAU_CP uses the global pair). `TradeSetup` fields: pair, direction, entry, SL, TP1, TP2, RR, risk/reward pips, pip_value, `strategy` ("GENERIC", "XAU_PB", or "XAU_CP" — selects trailing params; XAU_CP uses the global pair), `trail_arm_r`/`trail_dist_r` (stamped from the PRODUCING config at generation so per-TF factory overrides reach formatters; `None` = legacy unstamped → display falls back to platform defaults).
--- ---
@@ -235,7 +235,7 @@ Two entry points:
- Twelve Data SDK (`TDClient.time_series(...).as_pandas()`), symbols normalized to `"XAU/USD"` form; renames lowercase columns → capitalized; sorts oldest-first; localizes UTC. - Twelve Data SDK (`TDClient.time_series(...).as_pandas()`), symbols normalized to `"XAU/USD"` form; renames lowercase columns → capitalized; sorts oldest-first; localizes UTC.
- **Rate limit**: 55 credits/min sliding window (`_throttle`), blocks with sleep when exhausted. - **Rate limit**: 55 credits/min sliding window (`_throttle`), blocks with sleep when exhausted.
- **In-memory cache**: key `{symbol}_{tf}`, TTL = candle duration (M5 300 s … H4 14400 s); a cached frame is evicted and refetched if it fails the freshness check **or the boundary check** (`_has_latest_closed_bar`: the frame must contain the most recent fully closed bar, weekend-exempt — a wall-clock TTL alone would serve mid-candle snapshots at the next boundary and scans would run one bar behind). Cache and throttle are lock-protected (one provider is shared across the bot's thread pool). - **In-memory cache**: key `{symbol}_{tf}`, TTL = candle duration (M5 300 s … H4 14400 s); a cached frame is evicted and refetched if it fails the freshness check **or the boundary check** (`_has_latest_closed_bar`: the frame must contain the most recent fully closed bar, weekend-exempt — a wall-clock TTL alone would serve mid-candle snapshots at the next boundary and scans would run one bar behind). Fresh fetches enforce the same boundary guarantee: a response missing the just-closed bar (feed latency at the boundary) gets one grace retry after `FRESH_FETCH_RETRY_SECONDS` (3 s), then proceeds with a loud warning. Cache and throttle are lock-protected (one provider is shared across the bot's thread pool).
- **`_drop_forming_candle`**: Twelve Data includes the current partial candle; the scanner fires seconds after boundary, so the last bar is dropped if its close time is in the future. Without this every indicator would read a seconds-old stub and signals would repaint. - **`_drop_forming_candle`**: Twelve Data includes the current partial candle; the scanner fires seconds after boundary, so the last bar is dropped if its close time is in the future. Without this every indicator would read a seconds-old stub and signals would repaint.
- **Freshness** (`_is_data_fresh`): last bar older than 3× candle duration → stale (warns; live path suppresses signal if the PRIMARY TF is stale). Weekends exempt (Fri 22:00→Sun 22:00 UTC). - **Freshness** (`_is_data_fresh`): last bar older than 3× candle duration → stale (warns; live path suppresses signal if the PRIMARY TF is stale). Weekends exempt (Fri 22:00→Sun 22:00 UTC).
- **`_validate_and_clean`**: drop all-NaN rows → ffill (limit 3) → drop NaN closes → sort + dedupe (keep last) → drop future timestamps (>now+5 min) → repair OHLC inconsistencies (High=max, Low=min of O/C/existing) → remove >10σ return outliers (spike bar only — the opposite-direction snap-back bar right after it is kept, else every spike punched a two-bar hole) → warn if < ~210 bars. - **`_validate_and_clean`**: drop all-NaN rows → ffill (limit 3) → drop NaN closes → sort + dedupe (keep last) → drop future timestamps (>now+5 min) → repair OHLC inconsistencies (High=max, Low=min of O/C/existing) → remove >10σ return outliers (spike bar only — the opposite-direction snap-back bar right after it is kept, else every spike punched a two-bar hole) → warn if < ~210 bars.
@@ -270,11 +270,11 @@ Run: `python telegram_bot.py` (credentials resolve env `TELEGRAM_TOKEN`/`TELEGRA
- **Owner lock**: every command/button is wrapped in `_owner_only` — only `TELEGRAM_CHAT_ID` gets responses; others get a refusal. The bot is the owner's control panel; the product is the channel. - **Owner lock**: every command/button is wrapped in `_owner_only` — only `TELEGRAM_CHAT_ID` gets responses; others get a refusal. The bot is the owner's control panel; the product is the channel.
- **Subscriber channel** (`TELEGRAM_CHANNEL_ID`, `-100…`; `""` disables): auto-scan signal alerts + breach alerts also post there (bot needs channel-admin). Monetization = Telegram Stars subscription invite links; Telegram handles membership, the bot has zero payment code. The channel feed keeps running even when the owner toggles DM auto-scan off (`_feed_active()` = DM chats OR channel). - **Subscriber channel** (`TELEGRAM_CHANNEL_ID`, `-100…`; `""` disables): auto-scan signal alerts + breach alerts also post there (bot needs channel-admin). Monetization = Telegram Stars subscription invite links; Telegram handles membership, the bot has zero payment code. The channel feed keeps running even when the owner toggles DM auto-scan off (`_feed_active()` = DM chats OR channel).
- **Auto-scan**: boot-starts if `TELEGRAM_CHAT_ID` set (initial M15 scan populates the level cache first). Candle-boundary loops for M5/M15/H1/H4, firing at boundary + 10 s. Each scan runs all `scan_pairs` concurrently, updates the level cache, then broadcasts. - **Auto-scan**: boot-starts if `TELEGRAM_CHAT_ID` set (initial M15 scan populates the level cache first). Candle-boundary loops for M5/M15/H1/H4, firing at boundary + 10 s. Each scan runs all `scan_pairs` concurrently, updates the level cache, then broadcasts.
- **Alert gating**: quality threshold (`_alert_min_quality`, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction, entry bucket) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle, but a setup that moved ≥ half its risk distance (new pullback/swing anchor) re-alerts. The fingerprint is committed only **after** a message reaches ≥1 target (`_commit_alert`), so a failed Telegram send doesn't consume the alert. - **Alert gating**: quality threshold (`_alert_min_quality`, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle, but a setup whose entry moved ≥ half the last-ALERTED risk distance (new pullback/swing anchor; compared against the stored last-alert entry/risk, a stable reference immune to per-candle ATR drift) re-alerts. The fingerprint is committed only **after** a message reaches ≥1 target (`_commit_alert`), so a failed Telegram send doesn't consume the alert.
- **WebSocket monitor**: subscribes every `scan_pairs` symbol to TD quotes WS; 10 s heartbeat; exponential reconnect (≤60 s). On tick crossing any cached key level (`swing_high/low_1..3`, `ema_trend`, `bb_upper/lower`, `kc_upper/lower`; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown. - **WebSocket monitor**: subscribes every `scan_pairs` symbol to TD quotes WS; 10 s heartbeat; exponential reconnect (≤60 s). On tick crossing any cached key level (`swing_high/low_1..3`, `ema_trend`, `bb_upper/lower`, `kc_upper/lower`; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown.
- **Commands**: `/start` (menu), `/scan` (buttons: per pair — M5/M15/H1/H4 + Scan All TFs), `/autoscan` (toggle DM copy), `/status` (loops, WS, live prices, level cache, cooldowns), `/cache` (age per pair/TF), `/clearcache`, `/help`. - **Commands**: `/start` (menu), `/scan` (buttons: per pair — M5/M15/H1/H4 + Scan All TFs), `/autoscan` (toggle DM copy), `/status` (loops, WS, live prices, level cache, cooldowns), `/cache` (age per pair/TF), `/clearcache`, `/help`.
- **Tiingo spread monitor** (optional; on only when a Tiingo key resolves — see Environment variables): a 5-min sampler task fetches top-of-book bid/ask per scanned pair (`tiingo_provider.py`), logs to `logs/spread_log.csv` (skipping the weekend-closed window), and caches the latest quote; alert sends refresh the quote first, and signal messages then show live bid/ask + a warning when spread ≥ 2× the pair's assumed round-trip cost (`WIDE_SPREAD_MULTIPLE`). **Warn-only BY DESIGN** — no historical bid/ask exists, so a suppressing gate could never be validated (live == backtest rule); signals never consume bid/ask. `/status` shows the live spread per pair. **License (free tier is internal-use-only):** raw bid/ask/spread numbers go to owner DMs always; the channel gets them ONLY while config.json `telegram.channel_spread_detail=true` (the channel is internal/testing as of 2026-07-22 — **flip to false before the channel goes public**), otherwise a number-free "spread unusually wide" caution via `format_signal_message(report, spread_detail=False)` — routing in `_spread_detail_for`, test-enforced, committed default false. **Rate limits (free tier 50 req/h, 1000/day, checked 2026-07-22):** sampler 12/h + alert-batch refreshes ≤17/h + breach refreshes ≤12/h = worst case ≈41/h with ONE pair; adding pairs multiplies this — recheck before growing `scan_pairs`. - **Tiingo spread monitor** (optional; on only when a Tiingo key resolves — see Environment variables): a 5-min sampler task fetches top-of-book bid/ask per scanned pair (`tiingo_provider.py`), logs to `logs/spread_log.csv` (skipping the weekend-closed window), and caches the latest quote; alert sends refresh the quote first, and signal messages then show live bid/ask + a warning when spread ≥ 2× the pair's assumed round-trip cost (`WIDE_SPREAD_MULTIPLE`). **Warn-only BY DESIGN** — no historical bid/ask exists, so a suppressing gate could never be validated (live == backtest rule); signals never consume bid/ask. `/status` shows the live spread per pair. **License (free tier is internal-use-only):** raw bid/ask/spread numbers go to owner DMs always; the channel gets them ONLY while config.json `telegram.channel_spread_detail=true` (the channel is internal/testing as of 2026-07-22 — **flip to false before the channel goes public**), otherwise a number-free "spread unusually wide" caution via `format_signal_message(report, spread_detail=False)` — routing in `_spread_detail_for`, test-enforced, committed default false. **Rate limits (free tier 50 req/h, 1000/day, checked 2026-07-22):** sampler 12/h + alert-batch refreshes ≤17/h + breach refreshes ≤12/h = worst case ≈41/h with ONE pair; adding pairs multiplies this — recheck before growing `scan_pairs`.
- Signal messages include the full indicator breakdown, warnings, trade setup, **trailing-stop management plan** (arm price + trail distance, from Signal/pb params by `strategy`), and a risk disclaimer on every outbound message. - Signal messages include the full indicator breakdown, warnings, trade setup, **trailing-stop management plan** (arm price + trail distance, read from the `trail_arm_r`/`trail_dist_r` stamped on the setup by the producing per-TF config), and a risk disclaimer on every outbound message.
- Blocking scan work runs in `ThreadPoolExecutor(max_workers=4)`; one shared `ForexDataProvider` across all scans (cache reuse). - Blocking scan work runs in `ThreadPoolExecutor(max_workers=4)`; one shared `ForexDataProvider` across all scans (cache reuse).
- **Logging (hardened 2026-07-22)**: root-logger handlers (`force=True`) → console + `logs/telegram.log` (3-hour rotation, 240 backups = 30 days), full-date timestamps; every module logger (data_provider, tiingo_provider, news_engine, websockets, telegram.ext) lands in the same file. Failure surfaces all covered: candle loops and the spread sampler catch-and-continue per cycle (one bad scan can't kill a loop), breach tasks log exceptions via done-callback, PTB `add_error_handler` registered, and `sys.excepthook`/`threading.excepthook` (installed in `main()`) route uncaught crashes to the log. WS monitor keeps its exponential-backoff reconnect + `logger.exception`. - **Logging (hardened 2026-07-22)**: root-logger handlers (`force=True`) → console + `logs/telegram.log` (3-hour rotation, 240 backups = 30 days), full-date timestamps; every module logger (data_provider, tiingo_provider, news_engine, websockets, telegram.ext) lands in the same file. Failure surfaces all covered: candle loops and the spread sampler catch-and-continue per cycle (one bad scan can't kill a loop), breach tasks log exceptions via done-callback, PTB `add_error_handler` registered, and `sys.excepthook`/`threading.excepthook` (installed in `main()`) route uncaught crashes to the log. WS monitor keeps its exponential-backoff reconnect + `logger.exception`.
@@ -297,7 +297,7 @@ Replays history bar-by-bar through the **real `SignalAggregator.analyze_from_dat
- Same-bar SL+TP tie → SL first (conservative). Gap through SL/TP → filled at the bar open. - Same-bar SL+TP tie → SL first (conservative). Gap through SL/TP → filled at the bar open.
- **Trailing exits** (default on, from `signals.trail_arm_r/trail_dist_r`; XAU_PB trades carry per-trade 0.8/0.75 overrides): stop ratchets at bar close only, so a new stop can only fill from the next bar (conservative). Exit reasons: SL / TRAIL / TP1 (static mode) / TIMEOUT / NO_DATA. - **Trailing exits** (default on, from `signals.trail_arm_r/trail_dist_r`; XAU_PB trades carry per-trade 0.8/0.75 overrides): stop ratchets at bar close only, so a new stop can only fill from the next bar (conservative). Exit reasons: SL / TRAIL / TP1 (static mode) / TIMEOUT / NO_DATA.
- Session + weekend + weekday filters evaluate the **decision time = bar close** (`ts + primary_duration`) — exactly matching the live bot's wall-clock check. - Session + weekend + weekday filters evaluate the **decision time = bar close** (`ts + primary_duration`) — exactly matching the live bot's wall-clock check.
- The optional H4-ADX regime filter (`config.backtest.regime_filter`, default off) is applied in the replay by calling the live `_apply_regime_filter_live` method itself under the same guard — live and backtest cannot diverge if it's enabled (2026-07-21 parity fix; the unregistered-pair config fallback also applies the config.json overlay now). - The optional H4-ADX regime filter (`config.backtest.regime_filter`, default off) is applied in the replay by calling the live `_apply_regime_filter_live` method itself under the same guard, and the replay loads/slices the regime-filter TF whenever the gate is on (like live `analyze_pair`) — previously M5/M15 primaries silently no-opped the filter because H4 wasn't in their confirm maps. Live and backtest cannot diverge if it's enabled (2026-07-21 parity fix; the unregistered-pair config fallback also applies the config.json overlay now).
- Confirmation TFs are sliced to bars **fully closed** by decision time (no look-ahead). - Confirmation TFs are sliced to bars **fully closed** by decision time (no look-ahead).
**Performance:** `precompute_indicators` builds every indicator series once per TF (CuPy on GPU or NumPy — check the log line `Backend: CuPy GPU` vs `NumPy CPU fallback`; CuPy needs the `nvidia-*-cu12` pip wheels or it silently falls back). `GPUTechnicalEngine` subclasses the live engine and overrides only the `_compute_*` methods with cache lookups — outputs are byte-identical to the live engine (regression-tested). Bar replay: pure per-bar evaluation fans out over forked worker processes (`--workers`, 0 = auto: cores−2 capped 12; <5000 bars forces 1); the entry **cooldown (5 bars) is the only sequential rule**, applied in an ordered pass afterwards — parallel and sequential books are identical. ~300 bars/s/core, ~1700 bars/s at 10 workers. **Performance:** `precompute_indicators` builds every indicator series once per TF (CuPy on GPU or NumPy — check the log line `Backend: CuPy GPU` vs `NumPy CPU fallback`; CuPy needs the `nvidia-*-cu12` pip wheels or it silently falls back). `GPUTechnicalEngine` subclasses the live engine and overrides only the `_compute_*` methods with cache lookups — outputs are byte-identical to the live engine (regression-tested). Bar replay: pure per-bar evaluation fans out over forked worker processes (`--workers`, 0 = auto: cores−2 capped 12; <5000 bars forces 1); the entry **cooldown (5 bars) is the only sequential rule**, applied in an ordered pass afterwards — parallel and sequential books are identical. ~300 bars/s/core, ~1700 bars/s at 10 workers.
+7
View File
@@ -65,6 +65,13 @@ def load_dotenv(path: str = ENV_PATH) -> None:
key, value = key.strip(), value.strip() key, value = key.strip(), value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'": if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
value = value[1:-1] value = value[1:-1]
elif value[:1] not in "\"'":
# Unquoted value: whitespace-then-# starts an inline comment
# (KEY=abc # prod). Quote the value to keep a literal " #".
for sep in (" #", "\t#"):
cut = value.find(sep)
if cut != -1:
value = value[:cut].rstrip()
if key: if key:
os.environ.setdefault(key, value) os.environ.setdefault(key, value)
+13 -9
View File
@@ -851,15 +851,10 @@ class GPUTechnicalEngine(TechnicalEngine):
signal = SignalStrength.SELL signal = SignalStrength.SELL
conf = min(70, 50 + roc_abs * 10) conf = min(70, 50 + roc_abs * 10)
desc = f"ROC {roc_val:.2f}% — Falling momentum" desc = f"ROC {roc_val:.2f}% — Falling momentum"
elif roc_prev <= 0 < roc_val:
signal = SignalStrength.BUY
conf = 65
desc = f"ROC crossed above zero (+{roc_val:.2f}%)"
elif roc_prev >= 0 > roc_val:
signal = SignalStrength.SELL
conf = 65
desc = f"ROC crossed below zero ({roc_val:.2f}%)"
else: else:
# Zero-crosses land in the branches above by definition (a
# cross up is positive and rising, a cross down negative and
# falling) — only flat/fading momentum reaches here.
signal = SignalStrength.NEUTRAL signal = SignalStrength.NEUTRAL
conf = 30 conf = 30
desc = f"ROC {roc_val:+.2f}% — No clear momentum" desc = f"ROC {roc_val:+.2f}% — No clear momentum"
@@ -1690,6 +1685,13 @@ def run_backtest(
# ── Load all required timeframes ── # ── Load all required timeframes ──
timeframes = [bt_cfg.primary_tf] + list(bt_cfg.confirmation_tfs) timeframes = [bt_cfg.primary_tf] + list(bt_cfg.confirmation_tfs)
# Live parity: analyze_pair also fetches the regime-filter TF when the
# optional H4-ADX gate is enabled. Without it here, M5/M15 primaries
# (whose confirm maps lack H4) would make _apply_regime_filter_live
# silently no-op in the replay while live applies the gate.
flt = platform_cfg.backtest
if flt.regime_filter and flt.regime_filter_timeframe not in timeframes:
timeframes.append(flt.regime_filter_timeframe)
# Pad lookback so earliest TF has enough warmup for EMA200, ADX, etc. # Pad lookback so earliest TF has enough warmup for EMA200, ADX, etc.
pad_days = max(60, bt_cfg.warmup_bars // 4) pad_days = max(60, bt_cfg.warmup_bars // 4)
fetch_start = bt_cfg.start - timedelta(days=pad_days) fetch_start = bt_cfg.start - timedelta(days=pad_days)
@@ -1879,7 +1881,9 @@ def compute_metrics(trades: List[Trade]) -> Metrics:
gross_loss = float(-rs[losses_mask].sum()) if losses_mask.any() else 0.0 gross_loss = float(-rs[losses_mask].sum()) if losses_mask.any() else 0.0
pf = gross_profit / gross_loss if gross_loss > 0 else float("inf") pf = gross_profit / gross_loss if gross_loss > 0 else float("inf")
cum = np.cumsum(rs) # Prepend the flat starting equity so an opening losing streak counts
# as drawdown from 0, not from the first trade's already-negative cum.
cum = np.concatenate(([0.0], np.cumsum(rs)))
high_water = np.maximum.accumulate(cum) high_water = np.maximum.accumulate(cum)
drawdown = high_water - cum drawdown = high_water - cum
max_dd = float(drawdown.max()) if len(drawdown) else 0.0 max_dd = float(drawdown.max()) if len(drawdown) else 0.0
+8 -5
View File
@@ -12,9 +12,10 @@ Chart Patterns*) that are unambiguous from OHLC at bar close:
the interim peak (neckline) the interim peak (neckline)
double_top bearish reversal mirrored double_top bearish reversal mirrored
inv_hs bullish reversal inverse head & shoulders, break of inv_hs bullish reversal inverse head & shoulders, break of
the neckline (max of the two peaks the neckline (the LOWER of the two
flanking the head — conservative peaks flanking the head — the earlier
horizontal line, no sloped fits) horizontal trigger, no sloped fits;
validated as-is, see detector note)
hs_top bearish reversal head & shoulders top, mirrored hs_top bearish reversal head & shoulders top, mirrored
bull_flag bullish continuation strong pole, shallow ordered bull_flag bullish continuation strong pole, shallow ordered
retrace, close at a new high retrace, close at a new high
@@ -166,8 +167,10 @@ def _detect_hs(o, h, l, c, atr, cfg, bullish: bool) -> Optional[Dict]:
return None return None
if abs(s1 - s2) > cfg.cp_hs_shoulder_tol_atr * atr: if abs(s1 - s2) > cfg.cp_hs_shoulder_tol_atr * atr:
return None return None
# Neckline: the LOWER of the two interim peaks — the price must # Neckline: the LOWER of the two interim peaks — the EARLIER of
# clear both, so this is the conservative horizontal trigger. # the two horizontal triggers (max() would demand clearing both
# peaks). The whole validated CP book (M5 BUY-only) ran on this
# trigger — do not flip to max() without full re-validation.
p1 = float(h[i1 : i2 + 1].max()) p1 = float(h[i1 : i2 + 1].max())
p2 = float(h[i2 : i3 + 1].max()) p2 = float(h[i2 : i3 + 1].max())
neckline = min(p1, p2) neckline = min(p1, p2)
+25 -3
View File
@@ -67,6 +67,9 @@ class ForexDataProvider:
RATE_LIMIT = 55 RATE_LIMIT = 55
RATE_WINDOW = 60.0 RATE_WINDOW = 60.0
# Grace wait before the one retry when a fresh response at a candle
# boundary is missing the just-closed bar (feed latency).
FRESH_FETCH_RETRY_SECONDS = 3.0
def __init__(self, config: PlatformConfig): def __init__(self, config: PlatformConfig):
self.config = config self.config = config
@@ -130,6 +133,24 @@ class ForexDataProvider:
df = self._drop_forming_candle(df, timeframe) df = self._drop_forming_candle(df, timeframe)
df = self._validate_and_clean(df, symbol, timeframe) df = self._validate_and_clean(df, symbol, timeframe)
# The closed-bar guarantee must hold for FRESH responses too, not
# just cache hits: the scanner fires seconds after the boundary,
# and a lagging feed can answer without the just-closed bar — the
# scan would silently analyze (and session-check) the previous
# bar. Give the feed one short grace retry, then proceed loudly.
if (not df.empty and not self._has_latest_closed_bar(df, timeframe)):
time.sleep(self.FRESH_FETCH_RETRY_SECONDS)
retry = self._fetch_twelve_data(symbol, timeframe, outputsize)
retry = self._drop_forming_candle(retry, timeframe)
retry = self._validate_and_clean(retry, symbol, timeframe)
if retry is not None and not retry.empty:
df = retry
if not self._has_latest_closed_bar(df, timeframe):
logger.warning(
f"{symbol} {timeframe.value}: feed lagging — just-closed bar "
f"still missing after retry; this scan may run one bar behind"
)
if not self._is_data_fresh(df, symbol, timeframe): if not self._is_data_fresh(df, symbol, timeframe):
logger.warning( logger.warning(
f"STALE DATA from API: {symbol} {timeframe.value} — " f"STALE DATA from API: {symbol} {timeframe.value} — "
@@ -198,9 +219,10 @@ class ForexDataProvider:
def clear_cache(self) -> int: def clear_cache(self) -> int:
"""Wipe all cached data. Returns number of entries cleared.""" """Wipe all cached data. Returns number of entries cleared."""
count = len(self._cache) with self._cache_lock:
self._cache.clear() count = len(self._cache)
self._cache_timestamps.clear() self._cache.clear()
self._cache_timestamps.clear()
logger.info(f"Cache cleared: {count} entries wiped") logger.info(f"Cache cleared: {count} entries wiped")
return count return count
+3 -8
View File
@@ -638,15 +638,10 @@ class TechnicalEngine:
signal = SignalStrength.SELL signal = SignalStrength.SELL
conf = min(70, 50 + roc_abs * 10) conf = min(70, 50 + roc_abs * 10)
desc = f"ROC {roc_val:.2f}% — Falling momentum" desc = f"ROC {roc_val:.2f}% — Falling momentum"
elif roc_prev <= 0 < roc_val:
signal = SignalStrength.BUY
conf = 65
desc = f"ROC crossed above zero (+{roc_val:.2f}%)"
elif roc_prev >= 0 > roc_val:
signal = SignalStrength.SELL
conf = 65
desc = f"ROC crossed below zero ({roc_val:.2f}%)"
else: else:
# Zero-crosses land in the branches above by definition (a
# cross up is positive and rising, a cross down negative and
# falling) — only flat/fading momentum reaches here.
signal = SignalStrength.NEUTRAL signal = SignalStrength.NEUTRAL
conf = 30 conf = 30
desc = f"ROC {roc_val:+.2f}% — No clear momentum" desc = f"ROC {roc_val:+.2f}% — No clear momentum"
+4 -1
View File
@@ -221,6 +221,9 @@ def report_to_dict(report: SignalReport) -> dict:
"risk_reward": ts.risk_reward, "risk_reward": ts.risk_reward,
"risk_pips": round(ts.risk_pips, 1), "risk_pips": round(ts.risk_pips, 1),
"reward_pips": round(ts.reward_pips, 1), "reward_pips": round(ts.reward_pips, 1),
"strategy": ts.strategy,
"trail_arm_r": ts.trail_arm_r,
"trail_dist_r": ts.trail_dist_r,
} }
if report.primary_analysis: if report.primary_analysis:
d["indicators"] = { d["indicators"] = {
@@ -230,7 +233,7 @@ def report_to_dict(report: SignalReport) -> dict:
"description": ind.description, "description": ind.description,
} }
for name, ind in report.primary_analysis.indicators.items() for name, ind in report.primary_analysis.indicators.items()
if not name.startswith("_") if not name.startswith("_") and ind.weight != 0
} }
d["key_levels"] = { d["key_levels"] = {
k: round(v, 5) for k, v in report.primary_analysis.key_levels.items() k: round(v, 5) for k, v in report.primary_analysis.key_levels.items()
+26 -10
View File
@@ -15,6 +15,7 @@ import time
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
from dataclasses import dataclass, field from dataclasses import dataclass, field
from datetime import datetime, timezone from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
import requests import requests
@@ -281,14 +282,20 @@ class NewsEngine:
""" """
base, quote = self._parse_pair(pair_symbol) base, quote = self._parse_pair(pair_symbol)
# Check upcoming high-impact events # Check upcoming high-impact events. Grade from the SOONEST
# matching event, not the first listed — an unsorted feed could
# otherwise report "caution" for a far event while a nearer one
# warrants "danger".
soonest = None
for event in context.events_upcoming_1h: for event in context.events_upcoming_1h:
if event.is_high_impact and event.currency in (base, quote): if event.is_high_impact and event.currency in (base, quote):
mins = event.minutes_until if soonest is None or event.minutes_until < soonest.minutes_until:
if mins <= self.config.pre_event_buffer_minutes: soonest = event
return "danger", f"{event.currency} {event.event} in {mins:.0f}min" if soonest is not None:
else: mins = soonest.minutes_until
return "caution", f"{event.currency} {event.event} in {mins:.0f}min" if mins <= self.config.pre_event_buffer_minutes:
return "danger", f"{soonest.currency} {soonest.event} in {mins:.0f}min"
return "caution", f"{soonest.currency} {soonest.event} in {mins:.0f}min"
# Check for recent high-impact releases. A surprise print (actual # Check for recent high-impact releases. A surprise print (actual
# away from forecast) keeps the market unstable longer than an # away from forecast) keeps the market unstable longer than an
@@ -650,11 +657,20 @@ class NewsEngine:
if not date_str: if not date_str:
return None return None
# RFC 822/2822 forms, with or without seconds, numeric offsets AND
# named zones (GMT, EST/EDT, PST…). The old strptime "%Z" path
# accepted ANY abbreviation but attached no offset, so e.g. an EST
# pubDate was silently stamped UTC — a 5-hour error.
try:
dt = parsedate_to_datetime(date_str.strip())
if dt is not None:
if dt.tzinfo is None: # unknown zone name — assume UTC
return dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
except (TypeError, ValueError):
pass
formats = [ formats = [
"%a, %d %b %Y %H:%M:%S %z", # RFC 822 with numeric tz offset
"%a, %d %b %Y %H:%M:%S %Z", # RFC 822 with tz abbreviation (GMT/UTC)
"%a, %d %b %Y %H:%M %Z", # RFC 822 without seconds (MyFXBook)
"%a, %d %b %Y %H:%M %z", # RFC 822 without seconds, numeric offset
"%Y-%m-%dT%H:%M:%S%z", # ISO 8601 with offset "%Y-%m-%dT%H:%M:%S%z", # ISO 8601 with offset
"%Y-%m-%d %H:%M:%S", # ISO 8601 naive (assume UTC) "%Y-%m-%d %H:%M:%S", # ISO 8601 naive (assume UTC)
] ]
+39 -2
View File
@@ -50,6 +50,12 @@ class TradeSetup:
pip_value: float pip_value: float
# Entry archetype ("GENERIC", "XAU_PB") — selects trail params. # Entry archetype ("GENERIC", "XAU_PB") — selects trail params.
strategy: str = "GENERIC" strategy: str = "GENERIC"
# Trailing exit params stamped from the PRODUCING config, so per-TF
# factory overrides (e.g. M15 arm 0.70, M5 dist 0.85) reach every
# consumer. None = legacy/unstamped setup — formatters fall back to
# platform defaults; 0.0 means trailing is genuinely disabled.
trail_arm_r: Optional[float] = None
trail_dist_r: Optional[float] = None
@dataclass @dataclass
@@ -228,7 +234,12 @@ class SignalAggregator:
report.action_recommendation = "NO TRADE — Outside tradeable session" report.action_recommendation = "NO TRADE — Outside tradeable session"
report.trade_setup = None report.trade_setup = None
except Exception: except Exception:
pass # Fail-open by design (an unparseable timestamp shouldn't
# kill the scan) but never silently: this filter guards
# real money hours.
logger.exception(
f"{symbol}: live session filter errored — filter SKIPPED this scan"
)
return report return report
@@ -538,6 +549,7 @@ class SignalAggregator:
min_tp = (self.config.signals.round_trip_cost min_tp = (self.config.signals.round_trip_cost
* self.config.signals.min_tp_cost_multiple) * self.config.signals.min_tp_cost_multiple)
if 1.5 * sl_dist < min_tp: if 1.5 * sl_dist < min_tp:
_reject("spread_gate")
return return
report.trade_setup = TradeSetup( report.trade_setup = TradeSetup(
@@ -553,7 +565,10 @@ class SignalAggregator:
reward_pips=1.5 * sl_dist / pip_value, reward_pips=1.5 * sl_dist / pip_value,
pip_value=pip_value, pip_value=pip_value,
strategy="XAU_PB", strategy="XAU_PB",
trail_arm_r=self.config.signals.pb_trail_arm_r,
trail_dist_r=self.config.signals.pb_trail_dist_r,
) )
_reject("accepted") # funnel tick, mirrors the CP path
if report.signal_quality not in ("A+", "A", "B"): if report.signal_quality not in ("A+", "A", "B"):
report.signal_quality = "B" report.signal_quality = "B"
report.final_signal = (SignalStrength.BUY if is_buy report.final_signal = (SignalStrength.BUY if is_buy
@@ -629,6 +644,10 @@ class SignalAggregator:
).hour ).hour
except Exception: except Exception:
close_h = None close_h = None
logger.warning(
f"{report.symbol}: bar-close hour underivable — "
f"SELL hour gate skipped"
)
if close_h in cfg.sell_blocked_hours_utc: if close_h in cfg.sell_blocked_hours_utc:
_reject("sell_hour") _reject("sell_hour")
return return
@@ -683,6 +702,8 @@ class SignalAggregator:
reward_pips=height / pip_value, reward_pips=height / pip_value,
pip_value=pip_value, pip_value=pip_value,
strategy="XAU_CP", strategy="XAU_CP",
trail_arm_r=self.config.signals.trail_arm_r,
trail_dist_r=self.config.signals.trail_dist_r,
) )
if report.signal_quality not in ("A+", "A", "B"): if report.signal_quality not in ("A+", "A", "B"):
report.signal_quality = "B" report.signal_quality = "B"
@@ -814,7 +835,10 @@ class SignalAggregator:
f"DAY FILTER: {day_names.get(bar_wd, '?')} not in allowed days" f"DAY FILTER: {day_names.get(bar_wd, '?')} not in allowed days"
) )
except Exception: except Exception:
pass # Fail-open by design, but never silently (see live wrapper).
logger.exception(
f"{symbol}: session/day filter errored — filter SKIPPED"
)
# ── 6c. Transitioning regime quality boost ────────────────── # ── 6c. Transitioning regime quality boost ──────────────────
if report.market_regime == "transitioning" and report.signal_quality in ("B", "C", "D"): if report.market_regime == "transitioning" and report.signal_quality in ("B", "C", "D"):
@@ -922,6 +946,10 @@ class SignalAggregator:
).hour ).hour
except Exception: except Exception:
close_h = None close_h = None
logger.warning(
f"{report.symbol}: bar-close hour underivable — "
f"SELL hour gate skipped"
)
if close_h in self.config.xau_strategy.sell_blocked_hours_utc: if close_h in self.config.xau_strategy.sell_blocked_hours_utc:
report.warnings.append( report.warnings.append(
f"SELL FILTER: {close_h:02}:00 UTC entry blocked for shorts" f"SELL FILTER: {close_h:02}:00 UTC entry blocked for shorts"
@@ -1048,6 +1076,8 @@ class SignalAggregator:
risk_pips=round(sl_distance / pip_value, 1), risk_pips=round(sl_distance / pip_value, 1),
reward_pips=round(tp_distance / pip_value, 1), reward_pips=round(tp_distance / pip_value, 1),
pip_value=pip_value, pip_value=pip_value,
trail_arm_r=self.config.signals.trail_arm_r,
trail_dist_r=self.config.signals.trail_dist_r,
) )
def _check_mtf_agreement(self, report: SignalReport) -> bool: def _check_mtf_agreement(self, report: SignalReport) -> bool:
@@ -1165,6 +1195,11 @@ class SignalAggregator:
atr = primary.key_levels.get("atr", 0) atr = primary.key_levels.get("atr", 0)
if atr > 0 and score != 0: if atr > 0 and score != 0:
# Distance is measured from the FARTHEST of the 3 tracked
# swings (sorted()[0] = deepest low / highest high), so the
# penalty fires earlier than a nearest-swing rule would.
# Calibrated as-is in every validated book — changing the
# anchor is a signal-logic change requiring re-validation.
if score > 0: if score > 0:
swing_lows = sorted( swing_lows = sorted(
v for k, v in primary.key_levels.items() if k.startswith("swing_low_") v for k, v in primary.key_levels.items() if k.startswith("swing_low_")
@@ -1445,6 +1480,8 @@ class SignalAggregator:
risk_pips=risk_pips, risk_pips=risk_pips,
reward_pips=reward_pips, reward_pips=reward_pips,
pip_value=pip_value, pip_value=pip_value,
trail_arm_r=self.config.signals.trail_arm_r,
trail_dist_r=self.config.signals.trail_dist_r,
) )
def _collect_reasons(self, report: SignalReport): def _collect_reasons(self, report: SignalReport):
+41 -20
View File
@@ -398,12 +398,17 @@ def format_signal_message(report: SignalReport, spread_detail: bool = True) -> s
lines.append(f"⚖️ RR: 1:{ts.risk_reward:.1f}") lines.append(f"⚖️ RR: 1:{ts.risk_reward:.1f}")
# Trailing-stop management plan (matches backtested exit model: # Trailing-stop management plan (matches backtested exit model:
# TP1 is advisory, the trail is the exit). Pullback setups run # TP1 is advisory, the trail is the exit). Params are stamped on
# tighter stops and arm later. # the setup by the producing config so per-TF factory overrides
sig_cfg = PlatformConfig().signals # (M15 arm 0.70, M5 dist 0.85) are what subscribers see; only a
# legacy unstamped setup falls back to platform defaults.
is_pb = getattr(ts, "strategy", "GENERIC").endswith("_PB") is_pb = getattr(ts, "strategy", "GENERIC").endswith("_PB")
arm_r = sig_cfg.pb_trail_arm_r if is_pb else sig_cfg.trail_arm_r arm_r = getattr(ts, "trail_arm_r", None)
dist_r = sig_cfg.pb_trail_dist_r if is_pb else sig_cfg.trail_dist_r dist_r = getattr(ts, "trail_dist_r", None)
if arm_r is None or dist_r is None:
sig_cfg = PlatformConfig().signals
arm_r = sig_cfg.pb_trail_arm_r if is_pb else sig_cfg.trail_arm_r
dist_r = sig_cfg.pb_trail_dist_r if is_pb else sig_cfg.trail_dist_r
if arm_r > 0: if arm_r > 0:
risk_dist = abs(ts.entry_price - ts.stop_loss) risk_dist = abs(ts.entry_price - ts.stop_loss)
if ts.direction == "BUY": if ts.direction == "BUY":
@@ -574,6 +579,7 @@ _alert_min_quality = _APP.alert_min_quality
# signal doesn't re-send an identical message every candle. # signal doesn't re-send an identical message every candle.
_last_alert_fp: Dict[str, tuple] = {} _last_alert_fp: Dict[str, tuple] = {}
_last_alert_ts: Dict[str, float] = {} _last_alert_ts: Dict[str, float] = {}
_last_alert_setup: Dict[str, tuple] = {} # key → (entry, risk) of last ALERTED setup
_ALERT_REARM_SECONDS = 3600 # identical signal re-alerts after 1h _ALERT_REARM_SECONDS = 3600 # identical signal re-alerts after 1h
@@ -582,40 +588,54 @@ def _passes_quality(quality: str) -> bool:
def _alert_fingerprint(report: SignalReport) -> tuple: def _alert_fingerprint(report: SignalReport) -> tuple:
"""Alert identity for dedup. Includes a risk-relative entry bucket: """Alert identity for dedup: signal, quality, setup direction.
a persisting signal whose entry drifts a few ticks between candles
keeps the same fingerprint, while a setup that moved by half its risk Entry movement is checked separately in _should_alert against the
distance or more (new pullback, new swing anchor) re-alerts instead last ALERTED entry/risk (a stable reference). Deriving a price bucket
of being swallowed for the full re-arm hour.""" from the CURRENT SL distance here made the fingerprint churn on
normal ATR/anchor drift — near-duplicate alerts every candle."""
setup = report.trade_setup setup = report.trade_setup
entry_bucket = None
if setup is not None:
sl_dist = abs(setup.entry_price - setup.stop_loss)
entry_bucket = round(setup.entry_price / max(sl_dist * 0.5, 1e-9))
return ( return (
report.final_signal.name, report.final_signal.name,
report.signal_quality, report.signal_quality,
setup.direction if setup else None, setup.direction if setup else None,
entry_bucket,
) )
def _should_alert(key: str, report: SignalReport) -> bool: def _should_alert(key: str, report: SignalReport) -> bool:
"""True when this signal is new/changed for the key, or re-armed. """True when this signal is new/changed for the key, or re-armed.
A persisting identical signal dedups for the re-arm hour, but a setup
whose entry moved >= half the last-alerted risk distance (new
pullback/swing anchor) re-alerts immediately.
Pure check — call _commit_alert only after the message actually went Pure check — call _commit_alert only after the message actually went
out, so a failed Telegram send doesn't consume the alert for an hour. out, so a failed Telegram send doesn't consume the alert for an hour.
""" """
fp = _alert_fingerprint(report) fp = _alert_fingerprint(report)
return not ( if _last_alert_fp.get(key) != fp:
_last_alert_fp.get(key) == fp return True
and time.time() - _last_alert_ts.get(key, 0) < _ALERT_REARM_SECONDS if time.time() - _last_alert_ts.get(key, 0) >= _ALERT_REARM_SECONDS:
) return True
setup = report.trade_setup
last = _last_alert_setup.get(key)
if setup is not None and last is not None:
last_entry, last_risk = last
if abs(setup.entry_price - last_entry) >= 0.5 * max(last_risk, 1e-9):
return True
return False
def _commit_alert(key: str, report: SignalReport) -> None: def _commit_alert(key: str, report: SignalReport) -> None:
_last_alert_fp[key] = _alert_fingerprint(report) _last_alert_fp[key] = _alert_fingerprint(report)
_last_alert_ts[key] = time.time() _last_alert_ts[key] = time.time()
setup = report.trade_setup
if setup is not None:
_last_alert_setup[key] = (
setup.entry_price, abs(setup.entry_price - setup.stop_loss)
)
else:
_last_alert_setup.pop(key, None)
# ── Candle-boundary scheduler ──────────────────────────────────────── # ── Candle-boundary scheduler ────────────────────────────────────────
@@ -673,7 +693,8 @@ async def _scan_timeframe(
actionable = [ actionable = [
r for r in reports r for r in reports
if _passes_quality(r.signal_quality) if r.final_signal != SignalStrength.NEUTRAL # same gate as breach path
and _passes_quality(r.signal_quality)
and _should_alert(f"{r.symbol}_{tf.value}", r) and _should_alert(f"{r.symbol}_{tf.value}", r)
] ]
if actionable: if actionable:
+156 -3
View File
@@ -814,6 +814,18 @@ def _():
assert dt.tzinfo is not None assert dt.tzinfo is not None
@test("NewsEngine._parse_rss_date named zones carry their real offset")
def _():
# EST is UTC−5: 12:00 EST == 17:00 UTC. The old "%Z" path accepted the
# abbreviation but stamped UTC — a silent 5-hour shift.
dt = NewsEngine._parse_rss_date("Mon, 20 Jul 2026 12:00:00 EST")
assert dt is not None and dt.tzinfo is not None
assert dt.astimezone(timezone.utc).hour == 17
# GMT stays a no-op; without-seconds (MyFXBook) form still parses.
dt2 = NewsEngine._parse_rss_date("Mon, 20 Jul 2026 12:00 GMT")
assert dt2 is not None and dt2.astimezone(timezone.utc).hour == 12
@test("NewsEngine._parse_rss_date ISO format") @test("NewsEngine._parse_rss_date ISO format")
def _(): def _():
dt = NewsEngine._parse_rss_date("2025-05-12T14:30:00+00:00") dt = NewsEngine._parse_rss_date("2025-05-12T14:30:00+00:00")
@@ -899,6 +911,27 @@ def _():
assert risk == "caution" assert risk == "caution"
@test("NewsEngine.get_event_risk_level grades from the soonest event")
def _():
# Far event listed FIRST: an unsorted feed must still yield danger
# for the nearer event, not caution for the first match.
ne = NewsEngine(NewsConfig())
context = FundamentalContext()
context.events_upcoming_1h = [
EconomicEvent(
time=datetime.now(timezone.utc) + timedelta(minutes=55),
currency="USD", impact="high", event="GDP",
),
EconomicEvent(
time=datetime.now(timezone.utc) + timedelta(minutes=10),
currency="USD", impact="high", event="NFP",
),
]
risk, desc = ne.get_event_risk_level("XAU/USD", context)
assert risk == "danger"
assert "NFP" in desc
@test("NewsEngine.get_event_risk_level clear when no events") @test("NewsEngine.get_event_risk_level clear when no events")
def _(): def _():
ne = NewsEngine(NewsConfig()) ne = NewsEngine(NewsConfig())
@@ -1113,6 +1146,24 @@ def _():
assert ts.direction == "BUY" assert ts.direction == "BUY"
assert ts.risk_reward == 2.0 assert ts.risk_reward == 2.0
assert ts.entry_price > ts.stop_loss assert ts.entry_price > ts.stop_loss
assert ts.trail_arm_r is None and ts.trail_dist_r is None # unstamped default
@test("trade setups carry trail params from the producing config")
def _():
config = PlatformConfig()
config.signals.trail_arm_r = 0.70
config.signals.trail_dist_r = 0.85
agg = SignalAggregator(config)
report = SignalReport(symbol="XAU/USD", display_name="XAU/USD", timestamp="now")
report.final_signal = SignalStrength.BUY
report.final_score = 40.0
report.signal_quality = "A+"
report.current_price = 3300.0
ts = agg._generate_xau_atr_trade_setup(report, 10.0)
assert ts is not None, "A+ setup with ATR 10 should generate"
assert ts.trail_arm_r == 0.70, "setup must carry the producing config's arm"
assert ts.trail_dist_r == 0.85, "setup must carry the producing config's dist"
@test("SignalAggregator._check_mtf_agreement with 2 bullish") @test("SignalAggregator._check_mtf_agreement with 2 bullish")
@@ -1569,6 +1620,30 @@ def _():
assert "Indicators" in msg assert "Indicators" in msg
@test("format_signal_message renders stamped per-TF trail plan")
def _():
def _setup(**extra):
return TradeSetup(
pair="XAU/USD", display_name="XAU/USD", direction="BUY",
entry_price=3300.0, stop_loss=3290.0, take_profit_1=3315.0,
take_profit_2=3325.0, risk_reward=1.5, risk_pips=100,
reward_pips=150, pip_value=0.10, **extra,
)
report = SignalReport(symbol="XAU/USD", display_name="XAU/USD", timestamp="now")
report.final_signal = SignalStrength.BUY
report.signal_quality = "A"
# Stamped setup (M15-style overrides) must render the stamped values…
report.trade_setup = _setup(trail_arm_r=0.70, trail_dist_r=0.85)
msg = format_signal_message(report)
assert "+0.7R" in msg, "must render the stamped arm, not the 0.6 default"
assert "$8.50" in msg, "must render the stamped dist × risk, not the default"
# …while a legacy unstamped setup falls back to platform defaults.
report.trade_setup = _setup()
msg = format_signal_message(report)
assert "+0.6R" in msg
assert "$7.50" in msg
@test("_format_scan_summary produces valid text") @test("_format_scan_summary produces valid text")
def _(): def _():
config = PlatformConfig() config = PlatformConfig()
@@ -2116,6 +2191,10 @@ def _():
src = open(_gb.__file__, encoding="utf-8").read() src = open(_gb.__file__, encoding="utf-8").read()
assert "_apply_regime_filter_live(report, slice_dfs)" in src assert "_apply_regime_filter_live(report, slice_dfs)" in src
assert "backtest.regime_filter" in src assert "backtest.regime_filter" in src
# …and loads the regime-filter TF like live analyze_pair does, or the
# method silently no-ops for M5/M15 primaries (confirm maps lack H4).
assert "regime_filter_timeframe not in timeframes" in src, \
"replay must fetch/slice the regime-filter TF when the gate is on"
@test("disabled_indicators=['macd'] removes the pillar but keeps the histogram") @test("disabled_indicators=['macd'] removes the pillar but keeps the histogram")
@@ -2181,6 +2260,40 @@ def _():
break break
@test("Fresh fetch missing the just-closed bar retries once, then warns")
def _():
def bars_ending(last):
idx = pd.date_range(end=last, periods=300, freq="15min", tz="UTC")
v = np.linspace(3000.0, 3100.0, 300)
return pd.DataFrame({"Open": v, "High": v + 1, "Low": v - 1,
"Close": v, "Volume": 1000.0}, index=idx)
for _attempt in range(2):
provider = ForexDataProvider(PlatformConfig())
started = pd.Timestamp.now(tz="UTC").floor("15min")
last_closed = started - pd.Timedelta(minutes=15)
lagging = bars_ending(last_closed - pd.Timedelta(minutes=15))
complete = bars_ending(last_closed)
with patch("data_provider._is_forex_weekend", return_value=False), \
patch.object(ForexDataProvider, "FRESH_FETCH_RETRY_SECONDS", 0.0):
# (a) lagging first response → one retry returns the closed bar.
with patch.object(provider, "_fetch_twelve_data",
side_effect=[lagging.copy(), complete.copy()]) as mf:
served = provider.fetch_ohlcv("XAU/USD", Timeframe.M15)
# (b) complete first response → no retry at all.
provider._cache.clear()
provider._cache_timestamps.clear()
with patch.object(provider, "_fetch_twelve_data",
return_value=complete.copy()) as mf2:
provider.fetch_ohlcv("XAU/USD", Timeframe.M15)
if pd.Timestamp.now(tz="UTC").floor("15min") != started:
continue # boundary crossed mid-test — retry once
assert mf.call_count == 2, "lagging response must trigger exactly one retry"
assert served.index[-1] == last_closed, "retry result must be served"
assert mf2.call_count == 1, "boundary-complete response must not retry"
break
@test("Alert dedup: drift dedups, materially-moved setup re-alerts, no early commit") @test("Alert dedup: drift dedups, materially-moved setup re-alerts, no early commit")
def _(): def _():
import telegram_bot as tb import telegram_bot as tb
@@ -2201,17 +2314,23 @@ def _():
key = "TEST_ALERT_FP" key = "TEST_ALERT_FP"
tb._last_alert_fp.pop(key, None) tb._last_alert_fp.pop(key, None)
tb._last_alert_ts.pop(key, None) tb._last_alert_ts.pop(key, None)
r1 = rpt(3400.0, 3392.0) # risk $8 → $4 buckets tb._last_alert_setup.pop(key, None)
r1 = rpt(3400.0, 3392.0) # risk $8 → re-alert threshold $4
assert tb._should_alert(key, r1) assert tb._should_alert(key, r1)
assert tb._should_alert(key, r1), "_should_alert must not commit state itself" assert tb._should_alert(key, r1), "_should_alert must not commit state itself"
tb._commit_alert(key, r1) tb._commit_alert(key, r1)
assert not tb._should_alert(key, r1), "identical signal must dedup" assert not tb._should_alert(key, r1), "identical signal must dedup"
assert not tb._should_alert(key, rpt(3400.5, 3392.5)), \ assert not tb._should_alert(key, rpt(3400.5, 3392.5)), \
"sub-bucket drift must dedup" "sub-threshold drift must dedup"
assert not tb._should_alert(key, rpt(3401.0, 3392.6)), \
"SL-distance drift (ATR/anchor wobble) must not re-fingerprint"
assert tb._should_alert(key, rpt(3412.0, 3404.0)), \ assert tb._should_alert(key, rpt(3412.0, 3404.0)), \
"setup moved 1.5x risk — must re-alert" "setup moved 1.5x risk — must re-alert"
assert tb._should_alert(key, rpt(3395.5, 3387.5)), \
"setup moved half risk AGAINST the entry — must re-alert too"
tb._last_alert_fp.pop(key, None) tb._last_alert_fp.pop(key, None)
tb._last_alert_ts.pop(key, None) tb._last_alert_ts.pop(key, None)
tb._last_alert_setup.pop(key, None)
@test("Backtester --confirm-tfs defaults to None (factory's live map)") @test("Backtester --confirm-tfs defaults to None (factory's live map)")
@@ -2250,6 +2369,31 @@ def _():
assert t2.entry_time == idx[2] and t2.entry_price == 110.0 and t2.sl == 102.0 assert t2.entry_time == idx[2] and t2.entry_price == 110.0 and t2.sl == 102.0
@test("Max drawdown counts the decline from flat starting equity")
def _():
from gpu_backtest import Trade, compute_metrics
def mk(r):
t = Trade(pair="XAU/USD",
entry_time=pd.Timestamp("2026-01-05 10:00", tz="UTC"),
entry_price=3300.0, sl=3292.0, tp1=3312.0, tp2=3320.0,
direction="BUY", pip_value=0.10, risk_pips=80.0,
reward_pips=120.0, rr_planned=1.5, quality="B",
final_score=30.0, final_confidence=60.0,
market_regime="trending_up")
t.exit_reason = "SL" if r < 0 else "TP1"
t.r_multiple = r
return t
# Opening losing streak: equity 0 → −3R with no prior peak. The old
# high-water started at the first trade's cum (−1) and reported 2R.
m = compute_metrics([mk(-1.0), mk(-1.0), mk(-1.0)])
assert m.max_drawdown_r == 3.0, f"expected 3.0, got {m.max_drawdown_r}"
# Unchanged when a peak precedes the streak.
m2 = compute_metrics([mk(2.0), mk(-1.0), mk(-1.0), mk(-1.0)])
assert m2.max_drawdown_r == 3.0, f"expected 3.0, got {m2.max_drawdown_r}"
# ══════════════════════════════════════════════════════════════════════ # ══════════════════════════════════════════════════════════════════════
# MODULE 12: Chart-pattern entry (XAU_CP) # MODULE 12: Chart-pattern entry (XAU_CP)
# ══════════════════════════════════════════════════════════════════════ # ══════════════════════════════════════════════════════════════════════
@@ -2997,7 +3141,10 @@ def _():
"w", suffix=".env", delete=False, encoding="utf-8") "w", suffix=".env", delete=False, encoding="utf-8")
fh.write("# comment\n\nPLAIN=abc\nexport EXPORTED=def\n" fh.write("# comment\n\nPLAIN=abc\nexport EXPORTED=def\n"
'QUOTED="with spaces"\nSINGLE=\'sq\'\nEXISTING=from-dotenv\n' 'QUOTED="with spaces"\nSINGLE=\'sq\'\nEXISTING=from-dotenv\n'
"NOT A KV LINE\n") "NOT A KV LINE\n"
"COMMENTED=abc123 # prod key\n"
'QUOTED_HASH="keep #this"\n'
"HASH_IN_VALUE=abc#123\n")
fh.close() fh.close()
with patch.dict(os.environ, {"EXISTING": "from-shell"}, clear=True): with patch.dict(os.environ, {"EXISTING": "from-shell"}, clear=True):
load_dotenv(fh.name) load_dotenv(fh.name)
@@ -3007,6 +3154,12 @@ def _():
assert os.environ["SINGLE"] == "sq" assert os.environ["SINGLE"] == "sq"
assert os.environ["EXISTING"] == "from-shell", \ assert os.environ["EXISTING"] == "from-shell", \
"a real shell export must beat .env" "a real shell export must beat .env"
assert os.environ["COMMENTED"] == "abc123", \
"inline comment must be stripped, not absorbed into the value"
assert os.environ["QUOTED_HASH"] == "keep #this", \
"quoted values keep their hashes"
assert os.environ["HASH_IN_VALUE"] == "abc#123", \
"a hash with no preceding whitespace is part of the value"
load_dotenv("/nonexistent/.env") # missing file is a silent no-op load_dotenv("/nonexistent/.env") # missing file is a silent no-op