ForexBot2/CLAUDE.md

ForexBot2 — Codebase Reference

Signal-intelligence platform for XAU/USD (gold — live, validated) and EUR/USD (functional but OFF by default — no validated edge, see below). It fetches OHLCV from Twelve Data, scores 9 technical indicators per timeframe with regime-adaptive weights, checks multi-timeframe agreement and news/calendar risk, grades the signal (A+…F), and generates a full trade setup (entry, SL, TP1/TP2, trailing plan). Delivery: Telegram bot (auto-scanner + subscriber channel) and a CLI. A realism-focused backtester replays history through the exact same signal code.

This file is the authoritative documentation. DOCUMENTATION.md and USAGE.md are stale (May 2026) — do not trust them. PULLBACK_ENTRY_PLAN.txt documents the XAU_PB pullback strategy design. backtest/instructions_for_backtest.txt lists what every backtest report must record.

Read this whole file before changing signal logic. Every numeric threshold below was either backtested or is load-bearing for a backtested result. Rules for changes are at the bottom ("Rules for making changes").

---

Repository map

Path	Lines	Purpose
config.py	~374	All config dataclasses, PAIR_INFO registry, per-pair strategy configs. PlatformConfig defaults ARE the tuned XAU config.
data_provider.py	~347	Twelve Data SDK wrapper: fetch, rate-limit (55 credits/min), in-memory cache, validation/cleaning, freshness checks
engine.py	~1173	TechnicalEngine: computes all indicators on one DataFrame → TechnicalAnalysis (composite score, regime, key levels)
signal_aggregator.py	~1447	SignalAggregator: MTF analysis, news integration, gates/penalties, quality grade, trade-setup generation → SignalReport
news_engine.py	~680	Economic calendar (Forex Factory JSON + MyFXBook RSS fallback) + keyword RSS sentiment
telegram_bot.py	~1259	Async Telegram bot: candle-boundary auto-scanner, WebSocket level-breach monitor, owner lock, subscriber channel
pair_signals/__init__.py	52	PAIR_PRODUCERS registry + produce_signal()
pair_signals/xauusd.py	71	XAU config factory + per-timeframe overrides (docstring = validation record)
pair_signals/eurusd.py	158	EUR config factory + EUR regime weights (docstring = no-edge verdict + tuning history)
main.py	~348	CLI: python main.py --pair XAUUSD --timeframe H1 --detail/--json
test_all.py	~2361	QA suite: 179 tests in 10 modules, no live API needed. Must stay green.
backtest/gpu_backtest.py	~2231	Bar-replay backtester (CuPy→NumPy fallback) using the real SignalAggregator
backtest/diagnose_filters.py	609	Filter-diagnosis tooling
backtest/research/	—	Calendar research infra (ff_calendar_scraper.py, calendar_store.py) + result CSVs
backtest/*.csv	—	Verification trade books from past tuning campaigns (xau_*.csv, eur_verify.csv, …). Gitignored, local-only — used for manual before/after comparison, not by tests. eth/btc/gbp_verify.csv come from research whose pair code is NOT on main
backtest/cache/	—	Parquet OHLCV cache (raw bars)
test_ws_dual.py	108	Standalone WebSocket dual-connection probe
logs/telegram.log	—	Bot log (midnight rotation, 30 backups)

---

Pair status (do not confuse the two)

• XAU/USD: the tuned, validated product. It is the only entry in PlatformConfig.scan_pairs. All defaults in PlatformConfig/XauStrategyConfig are its tuned values.
• EUR/USD: fully wired (factory, gates, tests) but not in scan_pairs and its signals are informational only. Verdict (2026-07-11/12, recorded in pair_signals/eurusd.py docstring): no configuration passed validation on M5, M15, or H1 at a 1-pip round-trip cost; root cause is cost-to-volatility (EUR M15 ATR ≈ 10 pips → 1-pip spread taxes ~10% of ATR; gold's ATR/spread ratio is several times larger). Calendar-surprise and H1-hybrid avenues were also tested and closed. Do not re-open EUR tuning without new data (tick fills or H4/D1 horizons).

---

End-to-end pipeline (what happens on every scan)

Twelve Data REST (+ WebSocket ticks in the bot)
  → data_provider.fetch_ohlcv()        per timeframe: throttle, fetch, drop forming candle, validate/clean, cache
  → engine.TechnicalEngine.analyze()   per timeframe: 9 scored indicators + ATR context → regime → composite score
  → signal_aggregator.analyze_pair()   live entry point (weekend/day filters, fetch, staleness)
      └─ analyze_from_dataframes()     replayable core: MTF agreement → news → bonuses/penalties →
                                       final signal + quality grade → gates (session, SELL, M5, EUR, regime) →
                                       trade setup (3 generators) → reasons
  → SignalReport                       consumed by telegram_bot.py, main.py, backtest/gpu_backtest.py

Signal currency everywhere is the SignalStrength enum: STRONG_BUY=2, BUY=1, NEUTRAL=0, SELL=-1, STRONG_SELL=-2.

---

config.py — every knob

PAIR_INFO (market constants; keys are the canonical symbols "XAU/USD", "EUR/USD")

Field	XAU/USD	EUR/USD	Meaning
pip_value	0.10	0.0001	price units per pip
price_precision	2	5	decimals for SL/TP rounding & display
default_spread	0.30	0.0001	round-trip cost charged by the backtester when --spread not given

TechnicalConfig (indicator parameters)

EMA 9/21/50/200 (ema_mode: "4ema" default or "2ema" = 50/200 only); RSI 14 (OB 70 / OS 30 / neutral zone 40–60; rsi_mode: "regime_aware" default or "classic"); MACD 12/26/9 (macd_as_pillar=True — MACD is a scored indicator, not just divergence input); BB 20/2.0σ; ATR 14; ADX 14 (strong ≥25, very strong ≥50); divergence lookback 40; swing-S/R lookback 50; ROC period 10, threshold 1.0%; Keltner: EMA 20 mid ± 1.5 × ATR(10); ADX slope lookback 5, "emerging" threshold 3.0; disabled_indicators list; regime_weights_override dict (used by EUR).

SignalConfig (scoring & trade geometry — XAU-calibrated; EUR uses its own copies)

Field	Default	Meaning
mtf_agreement_threshold	2	timeframes (of primary + confirmations) that must agree
min_rr_ratio	0.5	reject setups below this RR
tp_tightening_factor	1.0	TP scale factor. Leave at 1.0: 0.8 was re-tested 2026-07-08 with spread+gap fills and nearly halved total R
round_trip_cost	0.30	XAU $ round trip
min_tp_cost_multiple	30.0	spread gate: TP1 distance must be ≥ cost × this (= $9 on XAU). Validated: sub-$10 targets lose
min_sl_pips	80.0	SL floor in pips ($8 on XAU)
tp_floor_rr	0.5	TP1 floor as fraction of SL distance (M15 factory overrides to 1.0)
trail_arm_r / trail_dist_r	0.6 / 0.75	trailing exit: once favorable excursion ≥ arm×risk, stop trails dist×risk behind best price (bar-close updates). TP1 becomes advisory. Validated on 12mo M5: +105R vs +48R static. 0 disables
pb_trail_arm_r / pb_trail_dist_r	0.8 / 0.75	trailing params for XAU_PB pullback trades (tighter stops → arm later)
weights, min_confidence, cooldown_bars, default_risk_pct, default_rr_ratio, atr_sl_multiplier	—	LEGACY / UNUSED outside tests. The engine has its own weights; backtest cooldown lives in BacktestConfig

NewsConfig

Tracked currencies ["USD","XAU","EUR"]; pre-event buffer 30 min, post-event 15 min (45 min after a surprise print); event_risk_caps_signals=False — XAU default: event risk warns but does NOT cap confidence (2026-07-11 audit: the caps suppressed trades worth +4.6..+7.2R/yr; XAU trades near USD releases outperform). EUR factory turns the cap back ON. Context cache TTL 600 s (must stay < pre-event buffer). 4 RSS feeds (investinglive, FXStreet, Google-News gold query, CNBC economy).

XauStrategyConfig (all XAU gates; enabled=True)

Field	Default	Notes
blocked_hours_utc	[0,1,2,10,11,13,16,21]	bar-close hours never traded. M5 factory appends 9 (post-London-open whipsaw, −28.9R/143 over 6 yr)
allowed_weekdays	[0,1,2,3,4,6]	Mon–Fri + Sun (Sydney open). weekend_filter=True: Fri 22:00 → Sun 22:00 UTC closed
min_atr_threshold	1.0	$1 ATR floor for swing-based setups
v2_sl_atr_multiplier	2.0	SL ATR floor in the generic setup generator
sl_atr_multiplier / tp_atr_multiplier	2.0 / 1.0	used by the XAU ATR setup generator (A+/A only)
sell_min_quality	["A+","A","B"]	SELLs below B are blocked
sell_vol_regimes	["high","extreme"]	SELLs only in these volatility regimes (quiet shorts fade vs gold's macro uptrend)
blocked_regimes / sell_blocked_regimes	[] / []	per-TF factory opt-ins. M15 factory sets blocked_regimes=["trending"] (mixed regime: strong ADX, EMAs unaligned — lost −6.8R/24)
M5 gates		m5_require_mtf=True; m5_min_abs_score=30; m5_min_confirmation_score=10; m5_sell_max_quality=["B","C"] (A-grade M5 SELLs enter at dip extremes and lose); m5_blocked_vol_regimes=["high"] (chop transition — 61% WR, negative total); m5_sell_min_rsi=25
M5 pullback (XAU_PB)		m5_pullback_enabled=True; min score 15 (lower than gate 1 BY DESIGN — mid-pullback scores weaker); ADX ≥ 20; RSI zones BUY (35,62) / SELL (38,65); retrace depth 0.8–6.0 ATR from the 40-bar swing extreme; band touch within 3 bars. Validated +38.3R/254 trades on 12mo

Legacy fields kept for config-shape compatibility: rsi_entry_threshold, min_ema200_tfs, allow_generic_trades, buy_threshold_boost, m5_confirmation_bars, h4_rsi_gate=False, h1_macd_gate=False, rsi_zone — not wired to current gates.

EurUsdStrategyConfig (EUR-local; nothing gold-calibrated leaks in)

Blocked hours [22,23,0,1,2,3]; weekdays Mon–Fri (M5 factory: Mon–Thu, Fridays bled −17R); min_atr_threshold=0.0003; SL: swing-buffer ×1.5 ATR, floor ×1.2 ATR (M5: 0.3/0.7), min 8 pips; min_rr_ratio=1.0; mature_move_threshold=35 (|score|>35 → tighter TP 1.2 ATR, else 1.8 ATR); blocked_regimes=["ranging","transitioning"]; min_quality_grades=["A+","A","B"]; min_entry_score=38 / min_entry_confidence=60 (M5 factory: 35/45); block_rsi_extended=True (no BUY at RSI>70, no SELL at RSI<30); costs: round_trip_cost=0.0001, min_tp_cost_multiple=10 (10-pip TP minimum), min_sl_pips=8. be_trigger_r/be_buffer_r are legacy (superseded by trailing exits).

FilterConfig (config.backtest — despite the name it applies LIVE too)

session_filter=True, window 03:00–23:00 UTC, blocked_hours_utc mirrors XAU's list (non-XAU pairs read this copy), allowed_weekdays Mon–Fri+Sun. regime_filter=False (optional H4-ADX≥25 gate, off).

PlatformConfig (top level)

symbol="XAU/USD", primary TF H1, confirmations [M15, H4], data_lookback_days=60, scan_pairs=[XAUUSD] (the single switch for what the bot scans), scan_timeframes=[M5,M15,H1,H4], candle_buffer_seconds=10, ws_cooldown_seconds=300, hardcoded twelve_data_api_key fallback.

---

engine.py — TechnicalEngine

Input contract: pandas DataFrame, UTC tz-aware DatetimeIndex, oldest-first, columns Open High Low Close Volume (capitalized). < 30 bars → empty NEUTRAL result. EMA200 needs ≥ 200 bars (otherwise EMA-trend indicator degrades gracefully).

Compute order matters: ATR first (EMA-ribbon and candlestick proximity use it) → EMA → RSI → BB → ADX → MACD (always computed, even if not a pillar — divergence needs the histogram) → ROC → Keltner (needs BB levels + MACD histogram) → divergence → structure S/R → candlesticks → regime classification → regime-aware RSI reinterpretation → composite.

Results land in TechnicalAnalysis.indicators: Dict[str, IndicatorSignal] (fields: name, value, signal, confidence 0-100, description, weight). Private series ride on the result object as underscore attrs (_rsi_series, _macd_histogram, _adx_slope, _kc_squeeze, _kc_squeeze_released) and are skipped by the composite scorer and all formatters (skip rule: name starts with _ or weight == 0).

The 9 scored indicators (+ ATR as context)

Key	Base weight	Signal logic (exact)
ema_trend	1.0	4-EMA mode: alignment score over checks (price>EMA9, 9>21, 21>50, 50>200), normalized to [-1,1]. >0.6 STRONG_BUY (conf ≤90), >0.2 BUY, mirrored sells, else NEUTRAL (40). Fast/medium cross ±10 conf. Ribbon (9/21/50 spread < 1 ATR) = "COMPRESSED"; compressed→expanded = +10 conf. Emits ema_fast/medium/slow/trend key levels
rsi	1.5	Wilder RSI (ewm α=1/14). Classic pass first: ≥70 SELL, ≤30 BUY, ≥60 BUY-momentum (45), ≤40 SELL-momentum (45), else NEUTRAL (30). Then overwritten by regime reinterpretation (below) when rsi_mode="regime_aware"
macd	1.5	Histogram = MACD line − signal line. Cross above 0 → BUY (conf 55+); accelerating (hist>prev>0) → STRONG_BUY; mirrored down; positive-but-fading / negative-but-fading → NEUTRAL (35); flat → NEUTRAL (30). Confidence scaled by
bollinger	1.5	Position = (close−lower)/(upper−lower). >1.0 SELL (65), <0.0 BUY (65), >0.8 SELL (45), <0.2 BUY (45), else NEUTRAL (30). Squeeze = width < 0.75 × 20-bar avg width → +10 conf. Emits bb_upper/lower/mid
adx	0.5	Wilder ADX/DI (ties zero BOTH DMs — fixes a systematic bearish nudge). ADX ≥ 25 → BUY/SELL by DI dominance, else NEUTRAL. Conf 40/65/80 by ADX band. ADX slope (5-bar diff): <25 with slope > 3.0 → "TREND EMERGING" +15 conf; also strengthening/weakening adjustments. Sets _adx_slope
roc	1.5	10-period %change. Positive & rising → BUY (STRONG if >2× threshold=2%); negative & falling → SELL; zero-cross → BUY/SELL (65); else NEUTRAL. Directionally redundant with MACD/Keltner/EMA (0.8–0.94 corr) but its weight is load-bearing: zeroing it backtested 13R WORSE — composite thresholds are calibrated with it. Do not remove without retuning thresholds
keltner	1.5	EMA20 mid ± 1.5×ATR(10). TTM squeeze = BB fully inside KC. Squeeze released (was→not) + MACD hist sign → STRONG_BUY/STRONG_SELL (80); squeeze active → NEUTRAL (60); close above/below channel → BUY/SELL continuation (65); inside → NEUTRAL (30). Emits kc_upper/lower/mid
divergence	1.5	RSI + MACD-hist vs price over 40 bars. Swing points (window 3) must align within ±2 bars to pair up; last two pairs: price LL + indicator HL = bullish, price HH + indicator LH = bearish. Any found → BUY/SELL/NEUTRAL-if-both, conf 70, value = count
candlestick	1.0	Last bar: Doji (body<10% range), Hammer/Hanging Man (lower wick>2×body), Shooting Star/Inverted Hammer, Engulfing (vs prev bar), Marubozu (body>85%). Majority vote → signal, conf 50; +20 if pattern sits within 0.5 ATR of a swing S/R level
atr	0.0 (context)	Wilder ATR. Volatility regime from percentile vs last 50 ATR values: >80 extreme, >60 high, >30 normal, else low. Emits atr, atr_pct

Structure S/R (not scored): swing highs/lows (window 3) over swing_sr_lookback (min 120 bars on 1min/5min TFs); up to 3 above and 3 below stored as swing_high_1..3 / swing_low_1..3 key levels.

Market regime classification (exact rules)

• ADX ≥ 25 and EMA-trend signal bullish → trending_up; bearish → trending_down; EMA mixed → trending (the "mixed" regime — momentum without directional structure; treated as direction-UNKNOWN everywhere; a past bullish-bias bug here cost −6.8R and is fixed)
• else ADX slope > 3.0 → transitioning (trend birth, ~5–10 bars before ADX crosses 25)
• else → ranging

Regime-aware RSI reinterpretation (rsi_mode="regime_aware", the default — classic mode backtested worse for gold)

Regime	RSI zones
trending_up	38–55 STRONG_BUY 75 ("pullback zone, ideal entry"); 30–38 BUY 60; 55–65 BUY 50; >65 NEUTRAL ("extended, wait"); <30 NEUTRAL ("possible reversal")
trending_down	mirrored: 45–62 STRONG_SELL 75; 62–70 SELL 60; 35–45 SELL 50; <35 / >70 NEUTRAL
transitioning / trending (mixed)	ONLY extremes: ≥70 SELL 50, ≤30 BUY 50, else NEUTRAL 30
ranging	≥70 SELL 70, ≤30 BUY 70, ≥60 SELL 45, ≤40 BUY 45, else NEUTRAL

Composite score

For each scored indicator: effective_weight = base_weight × regime_multiplier; contribution = signal.value × effective_weight × confidence/100. Composite = (Σ contributions / Σ effective_weights) × 50, clamped to ±100. Confidence = weight-averaged indicator confidence. Composite signal: ≥40 STRONG_BUY, ≥15 BUY, ≤−40 STRONG_SELL, ≤−15 SELL, else NEUTRAL.

Gold regime multipliers (defaults in _compute_composite; technical.regime_weights_override replaces the whole dict — EUR does this):

Indicator	trending_up/down/trending	ranging	transitioning
ema_trend	2.0	0.3	1.5
rsi	1.0	1.5	1.0
macd	2.0	1.5	2.5
bollinger	0.5	1.5	1.0
keltner	1.5	2.0	2.5
roc	1.5	0.5	2.0
candlestick	0.8	1.0	1.0
divergence	1.0	2.5	2.0
adx	0.8	0.5	1.0

---

signal_aggregator.py — SignalAggregator

Two entry points:

• analyze_pair(symbol) — LIVE: applies XAU/EUR weekend + allowed-weekday filters on wall clock, fetches all TFs (primary + confirmations), flags stale data (primary stale → signal suppressed to NEUTRAL/F), delegates to analyze_from_dataframes, then applies the live session check on the bar-CLOSE time (bar timestamp + candle duration): outside 03:00–23:00 or in blocked hours → NO TRADE. This is the exact predicate the backtester replays, so live == backtest.
• analyze_from_dataframes(symbol, dfs, context, use_news) — replayable core; treats each df as the visible history at decision time (caller must slice; no weekend filter here — the backtester applies it).

Pipeline inside analyze_from_dataframes (numbered as in the source)

1. Primary TF analysis → seeds report (score, confidence, regime, vol regime, price, timestamp = last bar's index label).
2. Confirmation TF analyses → mtf_signals / mtf_scores keyed by TF string ("5min", "1h", …).
3. MTF agreement: count BUY-ish vs SELL-ish across ALL TFs (primary included); agreement = count ≥ 2 in the primary's direction (two bullish confirms cannot "confirm" a bearish primary).
4. News (if use_news): event risk (clear/caution/danger) + pair sentiment (−1..+1). Backtests pass use_news=False.
5. Spread estimate = median(High−Low) of primary df.
6. _compute_final_signal (bonuses → event cap → penalties → thresholds → grade):
   • Bonuses: MTF agreement → +10 conf and score = 0.6×primary + 0.4×mean(all TF scores); news sentiment aligned (|sent|>0.2) → +5; in a directional trend, RSI in pullback zone → +5, BB position favorable → +5 (mixed trending regime gets NO directional bonus).
   • Event risk danger: warning always; confidence hard-capped at 30 ONLY if news.event_risk_caps_signals (False for XAU, True for EUR).
   • Penalty system (diminishing, replaces old additive stacking): collect, sort by size desc, apply with decay ×1.0, ×0.5, ×0.25…, total capped at 30. Penalties: mtf_disagreement 15; news_conflict 10; event_caution 15 (config-gated); counter_trend 15; extreme_vol 10; ema200_near 5 (price within 0.5% of EMA200 — env FB2_EMA_LO overrides); ema200_far 5 (>2.5%); exhaustion 8 (price >3.5 ATR beyond nearest swing); no_pullback 8 (>2.5 ATR from EMA21 in trend) / weak_pullback 5 (>2.0); rsi_extended 5 (RSI>65 with BUY / <35 with SELL in any strong-ADX regime, both directions in mixed trending); bb_overextended 8 (BB pos >0.85 BUY / <0.15 SELL).
   • Exhaustion hard block: ≥2 exhaustion-tagged penalties ({exhaustion, no_pullback, rsi_extended, bb_overextended} → report.exhaustion_count) AND momentum fading (ROC sign against score, or MACD signal against score) → confidence = 0.
   • Confidence clamped [0, 95]. Final signal: score ≥35 & conf ≥55 STRONG_BUY; ≥25 & ≥50 BUY; mirrored sells; else NEUTRAL.
   • Quality grade: |score|≥40 & conf≥65 & MTF → A+; ≥33 & ≥55 & MTF → A; ≥28 & ≥50 → B; ≥25 & ≥45 → C; ≥15 → D; else F.
7. Post-grade gates, in order:
   • 6b session filter (bar-label hour + weekday; XAU reads xau_strategy.blocked_hours_utc/allowed_weekdays, others read config.backtest.*) → NEUTRAL/F.
   • 6c transitioning boost: regime transitioning and grade B/C/D → A if |score|≥30 & conf≥50 & MTF; else B if |score|≥25 & conf≥45 (early trend entries).
   • 6d exhausted-trend cap: trending regime + exhaustion_count ≥2 + grade C/D → D, setup killed.
   • 6e XAU SELL quality gate: sell-direction & grade not in sell_min_quality → D.
   • 6f MTF opposition: ≥2 TFs pointing against the trade direction → D (never trade against consensus).
   • 6g XAU SELL volatility gate: sell-direction & vol regime not in {high, extreme} → D.
   • 6g2 XAU regime gates: regime in blocked_regimes (M15: mixed trending) → D; sell-direction & regime in sell_blocked_regimes → D.
   • 6h EUR gates (EUR only): blocked regimes (ranging/transitioning), quality < B, exhaustion count, session hours, |score| < min_entry_score, conf < min_entry_confidence, blocked vol regimes, indicator-agreement floor, RSI-extended block → each sets grade F with a warning.
   • 6i XAU M5 noise gates (_apply_m5_filters, primary TF = M5 only): (1) |score| ≥ 30; (2) MTF agreement required; (3) ≥1 confirmation TF with score ≥ ±10 in trade direction (NEUTRAL confirms don't count); (4) SELL grade must be B/C (A-grade M5 shorts lose −0.017R/trade); (5) vol regime high blocked; (6) SELL blocked when RSI < 25. Each failure → D, setup killed.
8. Trade setup: XAU + grade A+/A → _generate_xau_atr_trade_setup first; any pair + grade A+/A/B/C and no setup yet → _generate_trade_setup (swing-based).
9. Reasons: every non-neutral indicator description → bullish_reasons/bearish_reasons; EUR setups get an "EUR STRATEGY: …" marker inserted at reasons[0] (the backtester tags strategy from reasons[0]).
10. XAU_PB pullback (M5 XAU only, m5_pullback_enabled, no setup yet, grade ≠ F): _try_m5_pullback — see below.

Trade setup generators (exact geometry)

A. _generate_xau_atr_trade_setup (XAU, grade A+/A, ATR>0): TP quality scale A+ 2.0 / A 1.5 (A with exhaustion≥2 → 1.0). TP1 = entry ± tp_atr_multiplier(1.0)×scale×ATR; TP2 = 1.5×that; SL = entry ∓ 2.0×ATR. Gates: RR ≥ 0.5; TP1 distance ≥ $9 (spread gate). Prices rounded to 2 dp; pip 0.10 hardcoded — this generator is XAU-only by construction.

B. _generate_trade_setup (swing-based, any pair, grade A+..C): XAU requires ATR ≥ $1 (LOW ATR FILTER). Per-side:

• SL anchor = nearest swing low below price (BUY) / swing high above (SELL) ∓/± sl_swing_buffer×ATR (XAU 1.0, EUR = eur.sl_atr_multiplier); fallback (no swing) = (1+buffer)×ATR. SL distance floored at max(ATR × sl_atr_floor, min_sl_pips×pip) — XAU: max(2.0×ATR, $8); EUR: max(1.2×ATR, 8 pips) (M5 EUR: 0.7×ATR).
• TP1 = nearest swing beyond entry ± 0.3 ATR if it's within tp_swing_cap ATR (XAU 2.5) AND ≥ SL×RR-floor; else entry ± max(tp_mult×ATR, SL×RR-floor). XAU: tp_mult 1.5, RR-floor = max(min_rr 0.5, tp_floor_rr — 1.0 on M15). EUR: tp_mult 1.2 (mature move, |score|>35) or 1.8, RR-floor 1.0.
• TP2 = second swing if within 5 ATR, else entry ± max((tp_mult+1.5)×ATR, SL×floor×1.5); forced beyond TP1 by ≥1 ATR.
• Gates: RR ≥ min_rr; spread gate TP1 distance ≥ cost×multiple (XAU $9; EUR 10 pips).

C. _try_m5_pullback (XAU_PB): detection = EMA alignment ≥3 of 4 checks (9>21, 21>50, 50>200, close>200 — mirrored for SELL), ADX ≥ 20, price touched max(EMA21, KC-mid) within last 3 bars, resumption bar closed back in trend direction beyond EMA9, retrace depth 0.8–6.0 ATR from the 40-bar extreme, RSI in pullback zone (BUY 35–62 / SELL 38–65). Then gates: directional score ≥ 15; ≥1 confirmation TF ≥ ±10; not against MTF consensus; vol regime gates (same as breakout path). Geometry: SL = pullback 8-bar extreme ∓ 0.5 ATR (min $8); TP1 = 1.5×risk, TP2 = 2.5×risk; spread gate $9. Sets strategy="XAU_PB", grade min B, marker "XAU PULLBACK STRATEGY: …" at reasons[0]. Rejection funnel counters in class attr SignalAggregator.pb_funnel.

TradeSetup fields: pair, direction, entry, SL, TP1, TP2, RR, risk/reward pips, pip_value, strategy ("GENERIC" or "XAU_PB" — selects trailing params).

---

news_engine.py — NewsEngine

• Calendar: Forex Factory free JSON (nfs.faireconomy.media/ff_calendar_thisweek.json), 3 retries, honors 429 Retry-After with class-level backoff (≤600 s); fallback MyFXBook RSS. Never fabricates events — unavailable calendar = risk checks skipped that cycle.
• Caches are class-level (a fresh NewsEngine is built per scan): calendar + headlines, TTL = 600 s on success, 60 s on empty result; lock held across fetch so concurrent scans share one request.
• Event risk (get_event_risk_level): high-impact event for base/quote within 30 min → danger; within 60 min → caution; already-released high-impact → caution for 15 min after (45 min if actual ≠ forecast — "surprise"). EconomicEvent.surprise compares parsed numeric actual vs forecast (handles "%", "K/M/B" suffixes); direction is numeric-only (doesn't know if higher is bullish).
• Sentiment: keyword sets over headlines from 4 RSS feeds, deduped by title (Google News " - Publisher" suffix stripped), last 24 h only. Per-currency scores; GOLD_INVERSE_KEYWORDS (rate hike/hawkish/tightening/rate cut/dovish/easing/stimulus) flip sign for XAU (hawkish = dollar-bullish = gold-bearish). Currency detection via compiled word-boundary-ish regexes (bare "us"/"eu"/"uk" deliberately excluded). Pair sentiment = base − quote.

---

data_provider.py — ForexDataProvider

• Twelve Data SDK (TDClient.time_series(...).as_pandas()), symbols normalized to "XAU/USD" form; renames lowercase columns → capitalized; sorts oldest-first; localizes UTC.
• Rate limit: 55 credits/min sliding window (_throttle), blocks with sleep when exhausted.
• In-memory cache: key {symbol}_{tf}, TTL = candle duration (M5 300 s … H4 14400 s); stale cached data (freshness check) is evicted and refetched.
• _drop_forming_candle: Twelve Data includes the current partial candle; the scanner fires seconds after boundary, so the last bar is dropped if its close time is in the future. Without this every indicator would read a seconds-old stub and signals would repaint.
• Freshness (_is_data_fresh): last bar older than 3× candle duration → stale (warns; live path suppresses signal if the PRIMARY TF is stale). Weekends exempt (Fri 22:00→Sun 22:00 UTC).
• _validate_and_clean: drop all-NaN rows → ffill (limit 3) → drop NaN closes → sort + dedupe (keep last) → drop future timestamps (>now+5 min) → repair OHLC inconsistencies (High=max, Low=min of O/C/existing) → remove >10σ return outliers → warn if < ~210 bars.
• outputsize = max(300, min(days×bars_per_day+20, 5000)); TD caps at 5000/call (the backtester paginates by date).
• get_latest_price / get_spread_estimate (median M5 High−Low) helpers.

---

pair_signals/ — per-pair factories (ALWAYS use these, never bare PlatformConfig for a pair)

PAIR_PRODUCERS: Dict[symbol, factory]; factory signature make_config(primary_tf, confirmation_tfs=None) -> PlatformConfig. produce_signal(symbol, tf, data_provider) builds config → aggregator → analyze_pair. Registered: XAU/USD, EUR/USD.

A bare PlatformConfig() silently runs any pair with XAU defaults — always go through the factory (the bot, CLI, and backtester all do).

Default confirmation map (same in both factories, the bot, and backtest usage): M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1]. (Exception: main.py's CLI map uses M5→[M1,M15] — a CLI-only quirk.)

xauusd.py: PlatformConfig() defaults + per-TF overrides (each validated; full evidence in the module docstring):

• M5: append hour 9 UTC to blocked hours (6-yr replay: +207.1R vs +178.3R; M15-and-up keep hour 9 — it's positive there).
• M15: signals.tp_floor_rr = 1.0 (1R minimum targets: +116.6R vs +105.4R, better in train/val and 2024 OOS; hurts M5 — per-TF only) and xau_strategy.blocked_regimes = ["trending"] (mixed regime block: +65.7R vs +59.0R). A transitioning-SELL block was tested and REJECTED (bull-market artifact — failed 2024 OOS).
• The factory must stay equivalent to inline PlatformConfig(symbol=XAUUSD, …) + overrides — enforced by a test in test_all.py.

eurusd.py: sets symbol/display/pip, xau_strategy.enabled = False (belt-and-braces; XAU gates are also display-gated), news.event_risk_caps_signals = True, EUR session (7–20 h window, blocked [22,23,0-3], Mon–Fri), EUR regime-weight overrides (EUR_REGIME_WEIGHTS, and M5_REGIME_WEIGHTS + tighter trailing 0.45/0.5 + Mon–Thu + extra blocked hours [8,12,13,18,19] + swing lookback 20 on M5). Docstring = the complete no-edge verdict and closed research avenues.

---

telegram_bot.py — the bot

Run: python telegram_bot.py (env TELEGRAM_TOKEN, TELEGRAM_CHAT_ID, TELEGRAM_CHANNEL_ID; hardcoded fallbacks exist in the file).

• Owner lock: every command/button is wrapped in _owner_only — only TELEGRAM_CHAT_ID gets responses; others get a refusal. The bot is the owner's control panel; the product is the channel.
• Subscriber channel (TELEGRAM_CHANNEL_ID, -100…; "" disables): auto-scan signal alerts + breach alerts also post there (bot needs channel-admin). Monetization = Telegram Stars subscription invite links; Telegram handles membership, the bot has zero payment code. The channel feed keeps running even when the owner toggles DM auto-scan off (_feed_active() = DM chats OR channel).
• Auto-scan: boot-starts if TELEGRAM_CHAT_ID set (initial M15 scan populates the level cache first). Candle-boundary loops for M5/M15/H1/H4, firing at boundary + 10 s. Each scan runs all scan_pairs concurrently, updates the level cache, then broadcasts.
• Alert gating: quality threshold (_alert_min_quality, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle.
• WebSocket monitor: subscribes every scan_pairs symbol to TD quotes WS; 10 s heartbeat; exponential reconnect (≤60 s). On tick crossing any cached key level (swing_high/low_1..3, ema_trend, bb_upper/lower, kc_upper/lower; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown.
• Commands: /start (menu), /scan (buttons: per pair — M5/M15/H1/H4 + Scan All TFs), /autoscan (toggle DM copy), /status (loops, WS, live prices, level cache, cooldowns), /cache (age per pair/TF), /clearcache, /help.
• Signal messages include the full indicator breakdown, warnings, trade setup, trailing-stop management plan (arm price + trail distance, from Signal/pb params by strategy), and a risk disclaimer on every outbound message.
• Blocking scan work runs in ThreadPoolExecutor(max_workers=4); one shared ForexDataProvider across all scans (cache reuse). Logs to logs/telegram.log.

---

main.py — CLI

python main.py [--pair XAUUSD|EURUSD] [--timeframe M5|M15|M30|H1|H4|D1] [--detail] [--json] [--verbose]. Uses the pair factory, renders a compact terminal report (decision, setup, context, top-3 reasons per side, warnings) or full JSON via report_to_dict.

---

backtest/gpu_backtest.py — the backtester

Replays history bar-by-bar through the real SignalAggregator.analyze_from_dataframes — same gates, same setup math. News is disabled (use_news=False): backtests are pure technical.

Realism model (all validated fixes — do not regress):

• Entry at the next bar's open after the signal bar; SL/TP distances preserved relative to actual entry.
• Round-trip --spread subtracted from every trade's PnL before computing R (default = pair's default_spread: XAU $0.30, EUR 1 pip). Results before 2026-07-08 (+59.5R baseline era) charged no spread — not comparable.
• Trade resolution runs against the RAW price path (fetch_history(clean=False)): the parquet cache stores raw bars; cleaning (which drops >10σ news spikes that hit stops in reality) is applied only to the signal-side data. Caches written before this fix contain cleaned data — refetch to restore spike bars.
• Same-bar SL+TP tie → SL first (conservative). Gap through SL/TP → filled at the bar open.
• Trailing exits (default on, from signals.trail_arm_r/trail_dist_r; XAU_PB trades carry per-trade 0.8/0.75 overrides): stop ratchets at bar close only, so a new stop can only fill from the next bar (conservative). Exit reasons: SL / TRAIL / TP1 (static mode) / TIMEOUT / NO_DATA.
• Session + weekend + weekday filters evaluate the decision time = bar close (ts + primary_duration) — exactly matching the live bot's wall-clock check.
• Confirmation TFs are sliced to bars fully closed by decision time (no look-ahead).

Performance: precompute_indicators builds every indicator series once per TF (CuPy on GPU or NumPy — check the log line Backend: CuPy GPU vs NumPy CPU fallback; CuPy needs the nvidia-*-cu12 pip wheels or it silently falls back). GPUTechnicalEngine subclasses the live engine and overrides only the _compute_* methods with cache lookups — outputs are byte-identical to the live engine (regression-tested). Bar replay: pure per-bar evaluation fans out over forked worker processes (--workers, 0 = auto: cores−2 capped 12; <5000 bars forces 1); the entry cooldown (5 bars) is the only sequential rule, applied in an ordered pass afterwards — parallel and sequential books are identical. ~300 bars/s/core, ~1700 bars/s at 10 workers.

Strategy tags (from reasons[0] markers): XAU_PB, EUR, GENERIC (plus legacy XAU, GBP_PB recognized). --strategies filters the book by tag.

CLI:

python backtest/gpu_backtest.py --pair XAUUSD --days 365
python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-01 --primary-tf 15min --confirm-tfs 1h,4h
python backtest/gpu_backtest.py --pair EURUSD --primary-tf 5min --confirm-tfs 15min,1h --days 365
# key flags: --spread --quality A+,A,B,C --strategies --workers --max-hold-bars 300
#            --warmup-bars 250 --cooldown-bars 5 --no-cache --export-csv out.csv --detail

Report: summary metrics (total R, PF, Sharpe on R, max DD, MFE/MAE), breakdowns by quality/strategy/regime, trade log, and per-trade reasoning dumps (indicator-by-indicator agreement at entry — required by instructions_for_backtest.txt). --export-csv flattens every indicator's signal/value/agreement into columns.

Data: paginated TD fetches (5000-bar chunks) cached as parquet in backtest/cache/ keyed by symbol/TF/window; warmup pad ≥60 days before the window for EMA200.

---

test_all.py — QA suite

python test_all.py — 179 tests, 10 modules (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, EUR/USD support). No network needed (mocked). Uses a custom @test("name") decorator that runs immediately (NOT pytest — don't grep for def test_). Exit code non-zero on failure. Includes: pair-isolation tests (no gold logic on EUR configs and vice versa), the xauusd-factory ≡ inline-config equivalence test, and engine bug-fix regression tests (Module 9b). Any signal-logic change must keep this green and should add tests.

---

Environment variables

Variable	Purpose	Default
TWELVE_DATA_API_KEY	market data key	hardcoded fallback in config.py
TELEGRAM_TOKEN	bot token	hardcoded fallback in telegram_bot.py
TELEGRAM_CHAT_ID	owner chat id (boot auto-scan + owner lock)	hardcoded fallback
TELEGRAM_CHANNEL_ID	subscriber channel (-100…); "" disables the feed	hardcoded fallback
FB2_EMA_LO	EMA200 "near" penalty threshold in %	0.5

Dependencies (requirements.txt): numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets. Python 3.12. A venv/ exists in-repo.

---

Code conventions

• Type hints on public methods; dataclasses for all config and outputs.
• SignalStrength (−2..+2) is the universal signal currency.
• Key levels dict keys: swing_high_*, swing_low_*, ema_*, bb_*, kc_*, atr, atr_pct.
• Private per-indicator series ride on TechnicalAnalysis as _underscore attributes; anything _-prefixed or weight-0 is skipped by scorers/formatters.
• UTC everywhere; DataFrames indexed by tz-aware Datetime, oldest-first, capitalized OHLCV columns.
• Logging via module loggers; the bot adds a rotating file handler.

---

Rules for making changes (READ BEFORE EDITING SIGNAL LOGIC)

1. Never change a tuned value without re-validation. The acceptance rule used throughout: positive total R in BOTH the train and validation halves of a ≥12-month window, PF ≥ 1.3, at realistic cost (XAU $0.30 / EUR 1 pip round trip), with trailing exits and next-open fills. For EUR (and any marginal edge) additionally re-test on the 2024-06→2025-03 out-of-sample window — within-year splits alone have approved configs that failed OOS.
2. Per-timeframe overrides live in the pair factory (pair_signals/*.py), never in global defaults — the same knob has opposite effects on different TFs (proven twice: tp_floor_rr, hour-9 block). Document the validation evidence in the factory docstring.
3. Pair isolation is a hard invariant. XAU gates in the aggregator are display-gated (report.display_name == "XAU/USD"); EUR reads only EurUsdStrategyConfig for costs/geometry. Tests enforce zero XAU drift when EUR changes and vice versa.
4. Live == backtest parity: session/weekend/hour filters must evaluate bar-CLOSE time in both paths; xauusd.make_config must stay equivalent to the inline config the backtester/bot would build (test-enforced). If you add a live gate, add it to the replay path (or analyze_from_dataframes) in the same commit.
5. ROC stays weighted (see engine table) unless you retune composite thresholds and re-validate — removing it lost 13R.
6. Backtester realism knobs are results-critical: raw-path resolution, spread charging, next-open entry, SL-first ties, close-only trail ratchet. Any "optimization" that touches these invalidates comparability with the pinned books (backtest/*.csv, memory notes).
7. Don't zero/disable indicators via config in production paths — disabled_indicators exists for experiments; composite thresholds assume the full set.
8. The forming candle must stay dropped and analysis must only ever see closed bars, or live signals repaint.
9. Keep test_all.py green (python test_all.py); extend it for every new gate/knob.
10. Legacy/no-op config fields exist (flagged above) — do not build new logic on them; delete only with test updates.