Optimize EUR/USD to 78.7% WR on 2-year M15 backtest

Three-layer optimization:

1. Entry filters (signal_aggregator.py section 6e):
   - Block ranging regime
   - Only A+/A/B quality grades
   - min_entry_score=38, min_entry_confidence=60
   - Block RSI > 70 for BUY, RSI < 30 for SELL

2. Trade setup (config.py EurUsdStrategyConfig):
   - RR=0.50 with uniform quality scale
   - SL floor 1.2 ATR, TP 1.0 ATR

3. Breakeven trail (backtest/gpu_backtest.py):
   - When MFE reaches 0.3R, move SL to entry + 0.05R buffer
   - Converts 4 of 14 losses to +0.05R small wins
   - Bar-by-bar simulation integrated into simulate_trades()

Results (EUR/USD M15, 2024-05-18 to 2026-05-18):
  47 trades | 78.7% WR | +1.81R | PF 1.18 | MaxDD 3.35R
XAU/USD unchanged: 77.8% WR on 30-day, 68.7% WR on 2-year.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

.gitignore

@@ -0,0 +1,27 @@
+# Secrets
+.env
+.env.local
+
+# Editor / tooling
+.claude/
+.vscode/
+.idea/
+
+# Python
+__pycache__/
+*.pyc
+*.egg-info/
+.venv/
+venv/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Logs
+*.log
+
+# Backtesting data (keep scripts, ignore generated files)
+backtest/cache/
+backtest/*.csv
+backtest/__pycache__/

CLAUDE.md

@@ -0,0 +1,113 @@
+# ForexBot2
+
+XAU/USD signal intelligence platform. Analyzes gold across multiple timeframes using 9 technical indicators with regime-adaptive weights, news sentiment, and economic calendar data. Delivers signals via Telegram with full trade setups (entry, SL, TP).
+
+## Architecture
+
+```
+Twelve Data REST API + WebSocket
+        |
+  data_provider.py          (fetch, cache, validate OHLCV; rate-limited 55 calls/min)
+        |
+    engine.py                (9 indicators per timeframe → composite score)
+        |
+  signal_aggregator.py       (MTF agreement + news + final signal + trade setup)
+        |
+  +-----+--------+
+  |              |
+telegram_bot.py  main.py
+(Telegram alerts, (CLI single-pair
+ auto-scanner,    analysis)
+ WS monitor,
+ live dashboard)
+```
+
+## Core Files
+
+| File | Purpose |
+|------|---------|
+| `config.py` | All configuration dataclasses, XAU/USD pair definition, strategy config |
+| `data_provider.py` | Twelve Data SDK wrapper with in-memory cache and rate limiting |
+| `engine.py` | `TechnicalEngine` — computes EMA, RSI, BB, ADX, ATR, MACD, ROC, Keltner, divergences, candlestick patterns, swing S/R |
+| `signal_aggregator.py` | `SignalAggregator` — MTF analysis, news integration, diminishing penalty cap, trade setup generation |
+| `news_engine.py` | Economic calendar (Forex Factory JSON) + RSS sentiment (keyword-based) |
+| `telegram_bot.py` | Async Telegram bot with auto-scanner (M5/M15/H1/H4), WebSocket price monitor, live dashboard |
+| `main.py` | CLI entry point for XAU/USD analysis |
+| `test_all.py` | Comprehensive QA suite (~137 tests, no live API needed) |
+
+## Key Concepts
+
+**Signal Pipeline:** OHLCV → indicators → regime classification → regime-weighted composite score → MTF agreement check → news/event adjustment → diminishing penalty cap → final signal (STRONG_BUY/BUY/NEUTRAL/SELL/STRONG_SELL) → quality grade (A+/A/B/C/D/F) → trade setup if grade C+.
+
+**Regime-Adaptive Weights:** Market classified as `trending_up`, `trending_down`, `ranging`, or `transitioning` via ADX + EMA + ADX slope. Gold-specific weights per regime:
+- **Trending:** MACD (2.0), EMA (2.0), Divergence (2.0), Keltner (1.5), ROC (1.5)
+- **Ranging:** Divergence (2.5), Keltner (2.0), RSI (1.5), BB (1.5), MACD (1.5)
+- **Transitioning:** Keltner (2.5), MACD (2.5), ROC (2.0), Divergence (2.0), EMA (1.5)
+
+**Transitioning Regime:** ADX < 25 but ADX slope > 3.0 = trend emerging. Catches trend birth 5-10 bars before ADX crosses 25. Boosts quality grade for early entries.
+
+**Diminishing Penalty Cap:** Penalties sorted by magnitude, applied with exponential decay (1st at 100%, 2nd at 50%, 3rd at 25%). Hard cap at 30% total. Replaces old additive stacking that could gut 30-50% confidence.
+
+**Indicators (9 scored):** EMA Trend, RSI, Bollinger Bands, MACD, ADX, Keltner Channels (TTM Squeeze), ROC (Rate of Change), Divergence, Candlestick Patterns.
+
+**Trade Setup:** SL = nearest swing low/high ± 1×ATR buffer (min 1.5×ATR). TP = nearest swing level if RR≥1.2, else calculated from ATR. Minimum RR 0.5:1.
+
+## How to Run
+
+```bash
+# CLI (XAU/USD analysis)
+python main.py                    # H1 default
+python main.py --timeframe M15 --detail --json
+
+# Telegram bot (auto-scanner)
+TELEGRAM_TOKEN=<token> TELEGRAM_CHAT_ID=<id> python telegram_bot.py
+
+# Tests
+python test_all.py
+
+# Backtest
+python backtest/gpu_backtest.py --pair XAUUSD --days 365
+```
+
+## Environment Variables
+
+| Variable | Purpose | Default |
+|----------|---------|---------|
+| `TWELVE_DATA_API_KEY` | Market data API key | Hardcoded fallback in config |
+| `TELEGRAM_TOKEN` | Telegram bot token | Hardcoded fallback |
+| `TELEGRAM_CHAT_ID` | Auto-scan chat ID (enables boot auto-scan) | Hardcoded fallback |
+
+## Dependencies
+
+`numpy`, `pandas`, `requests`, `twelvedata`, `python-telegram-bot`, `websockets` — see `requirements.txt`.
+
+## Data Provider
+
+- **Source:** Twelve Data REST API via `twelvedata` SDK
+- **Rate limit:** 55 credits/min (Grow plan), enforced by sliding-window throttle
+- **Cache:** In-memory dict, TTL = candle duration of the timeframe
+- **Validation:** Drops NaN rows, forward-fills gaps (limit 3), fixes OHLC inconsistencies, removes outliers (>10σ), strips future timestamps, deduplicates
+- **WebSocket:** XAU/USD only
+
+## Telegram Bot
+
+- **Auto-scan:** M5 (data-only, no alerts), M15, H1, H4 at candle boundaries + 10s buffer
+- **Manual scan:** Buttons for M15/H1/H4 individual scans + "Scan All TFs" multi-timeframe summary
+- **WebSocket:** Streams live ticks for XAU/USD; triggers immediate M15 scan on key level breach (swing S/R, BB, KC, EMA200) with 5-min cooldown
+- **Live dashboard:** Edit-in-place message, 30s refresh, shows XAU/USD across timeframes
+- **Commands:** `/start`, `/scan`, `/autoscan`, `/status`, `/cache`, `/clearcache`, `/help`
+
+## Backtesting
+
+The `backtest/` directory contains `gpu_backtest.py` — a GPU-accelerated backtester (CuPy with NumPy fallback) that replays historical OHLCV through the real SignalAggregator. Pre-computes all indicator series including ROC, Keltner, ADX slope for 100x+ speedup. Uses parquet caching for data persistence.
+
+## Code Conventions
+
+- Python 3.12, type hints on public methods
+- Dataclasses for all config and output structures
+- `SignalStrength` enum (-2 to +2) is the universal signal currency
+- Indicators stored as `Dict[str, IndicatorSignal]` in `TechnicalAnalysis`
+- Key levels stored as `Dict[str, float]` — prefixed `swing_high_`, `swing_low_`, `ema_`, `bb_`, `kc_`, `atr`
+- Private indicator data attached as `result._rsi_series`, `result._macd_histogram`, `result._adx_slope`, `result._kc_squeeze` (underscore prefix, skipped by composite scorer)
+- Blocking signal computation runs in `ThreadPoolExecutor(max_workers=4)` to avoid blocking the async Telegram event loop
+- Shared `ForexDataProvider` instance across all scans for cache reuse

USAGE.md

@@ -0,0 +1,198 @@
+# ForexBot2 — USD/JPY Signal Intelligence
+
+A focused USD/JPY signal generator + walk-forward backtester. Live signals
+and backtests run through the **same** technical engine, so what you measure
+in a backtest is exactly what you'd execute live.
+
+## Setup
+
+```bash
+pip install -r requirements.txt
+```
+
+API key (optional — a default key is in `config.py`):
+
+```bash
+export TWELVE_DATA_API_KEY=your_key_here          # Linux/Mac
+$env:TWELVE_DATA_API_KEY="your_key_here"          # PowerShell
+```
+
+---
+
+## Live Signal
+
+```bash
+python main.py                          # H1, summary view
+python main.py --detail                 # Full indicator breakdown
+python main.py --timeframe H4 --detail  # H4 instead of H1
+python main.py --json                   # JSON output (for piping)
+```
+
+| Option        | Default | Description                                |
+|---------------|---------|--------------------------------------------|
+| `--timeframe` | `H1`    | `M5 M15 M30 H1 H4 D1`                      |
+| `--detail`    | off     | Indicator breakdown, key levels, reasons   |
+| `--json`      | off     | Machine-readable output                    |
+| `--verbose`   | off     | Debug logging                              |
+
+### Output
+
+```
+USD/JPY  │  Price: 149.230  │  Regime: trending_up  │  Vol: normal
+
+SIGNAL: BUY  │  Score: +28.4  │  Grade: B  │  Confidence: 62%
+Action: BUY — Decent setup, wait for confirmation
+
+Multi-Timeframe:  ✓ ALIGNED
+   M15: BUY          (+18.2)
+    H1: BUY          (+28.4)
+    H4: STRONG BUY   (+45.1)
+
+Trade Setup:
+  Direction:     BUY
+  Entry:         149.230
+  Stop Loss:     148.960  (27.0 pips)
+  Take Profit 1: 149.770  (54.0 pips)
+  Take Profit 2: 150.040
+  Risk:Reward:   1:2.0
+```
+
+---
+
+## Backtest (Walk-Forward)
+
+The backtester replays USD/JPY history bar-by-bar through the live signal
+pipeline. No lookahead, no parallel implementation — what passes in
+backtest passes live.
+
+```bash
+python main.py --backtest                          # Full available history
+python main.py --backtest --bars 2000              # Last 2000 H1 bars
+python main.py --backtest --start 2024-01-01 --end 2024-12-31
+python main.py --backtest --timeframe H4           # Backtest on H4
+python main.py --backtest --min-grade A            # Only A/A+ entries
+python main.py --backtest --journal                # Last 10 trades printed
+python main.py --backtest --journal-csv trades.csv # Full journal to CSV
+python main.py --backtest --regime-filter          # Skip choppy/ranging bars
+python main.py --backtest --regime-filter --regime-min-adx 35
+python main.py --backtest --session-filter         # Trade only 07-16 UTC
+python main.py --backtest --session-filter --session-start 8 --session-end 17
+python main.py --backtest --auto-session-filter    # Reactive session toggle
+python main.py --backtest --auto-session-filter --auto-session-lookback 15 \
+                          --auto-session-threshold-pct -0.5
+```
+
+### Regime filter
+
+The engine is a confluence-based trend-follower — it makes money in trending
+regimes and loses in chop. `--regime-filter` gates entries on H4 ADX:
+signals only fire when ADX ≥ `--regime-min-adx` (default 30). Sweet spot is
+ADX 30–35; tighter cuts trades faster than it improves quality.
+
+### Session filter
+
+`--session-filter` restricts entries to a UTC hour window (default 07-16,
+covering London + NY). Diagnostic on 2025-11 → 2026-05 showed 47% WR /
++127 pips inside that window vs 0% WR / -571 pips outside it. Override
+bounds with `--session-start` (inclusive) and `--session-end` (exclusive).
+
+Tradeoff: it's regime-specific medicine. In trending regimes (e.g. 2024)
+it cuts winning off-hours trades and gives back ~33% of the edge. Ships
+**default off** because the pure regime filter still wins cumulative
+across the 2024 / 2023 / 2026 windows.
+
+### Auto session toggle
+
+`--auto-session-filter` is a reactive variant: it tracks the rolling mean
+`pnl_pct` over the last N closed trades (`--auto-session-lookback`,
+default 10) and activates the 07-16 UTC window only when that mean drops
+below `--auto-session-threshold-pct` (default `-0.3`). Releases
+automatically once expectancy recovers.
+
+Less aggressive than the manual flag — preserves more trend edge but
+reacts slowly because the deque must fill with losses before kicking in.
+Use this when you don't want to manually toggle the regime medicine.
+
+### Execution model
+
+| Aspect             | Behavior                                                          |
+|--------------------|-------------------------------------------------------------------|
+| Entry              | Next bar's open (no current-bar lookahead)                        |
+| Stop Loss          | 1.5 × ATR from entry (configurable)                               |
+| Take Profit        | RR × stop distance (default 1:2)                                  |
+| Same-bar SL & TP   | Conservative: SL fills first                                      |
+| Spread cost        | Half-spread on entry + half-spread on exit (full round-trip)      |
+| Slippage           | 0.5 pips per fill (entry + exit)                                  |
+| Position size      | 1% of current equity risked per trade                             |
+| Concurrency        | One open position at a time                                       |
+| News in backtest   | Disabled — RSS/calendar can't be replayed without lookahead       |
+
+### Output
+
+```
+Backtest Summary — USD/JPY
+─────────────────────────────────────────────────
+Trades:           87  (52W / 35L)
+Win Rate:         59.8%
+Profit Factor:    1.84
+Expectancy:       +12.4 pips/trade  (+12.42 per trade)
+Initial Balance:  $10,000.00
+Final Balance:    $11,084.00
+Total Return:     +10.84%
+CAGR:             +18.92%
+Max Drawdown:     -4.21%
+Sharpe Ratio:     1.42
+Sortino Ratio:    2.18
+Avg Duration:     14.3 bars
+Win Streak:       7 | Loss Streak: 4
+─────────────────────────────────────────────────
+```
+
+---
+
+## Parameter Optimization (parallel)
+
+```bash
+python main.py --backtest --optimize
+```
+
+Runs a 3×3×3 grid (27 combos) over `atr_sl_multiplier`, `default_rr_ratio`,
+and `min_score` (signal-score threshold). Combos run in parallel across all
+CPU cores via `multiprocessing.Pool`. Twelve Data is hit once — workers
+share an on-disk parquet cache (`.cache/usdjpy_<tf>.parquet`).
+
+Output ranks combos by total return:
+
+```
+Top 10 Parameter Combinations (by total return):
+
+#   Return%     PF   WinR  Trades  MaxDD%  Sharpe  Params
+──────────────────────────────────────────────────────────────────────
+1   +14.21%   2.10  62.5%     64   -3.84   1.78    atr_sl_multiplier=1.5, default_rr_ratio=2.5, min_score=25.0
+2   +11.93%   1.92  58.7%     71   -4.11   1.55    atr_sl_multiplier=1.0, default_rr_ratio=2.0, min_score=15.0
+...
+```
+
+Customize the grid in `config.py → BacktestConfig.param_grid`.
+
+---
+
+## Files
+
+| File                  | Role                                              |
+|-----------------------|---------------------------------------------------|
+| `main.py`             | CLI entry point (live + backtest + optimize)      |
+| `config.py`           | All parameters in one place                       |
+| `data_provider.py`    | Twelve Data fetcher + parquet cache               |
+| `engine.py`           | Technical indicators (8 + divergence + patterns)  |
+| `signal_aggregator.py`| Multi-TF aggregation, grading, trade setup        |
+| `news_engine.py`      | RSS sentiment + economic calendar (live only)     |
+| `backtest.py`         | Walk-forward replay + parallel optimizer          |
+| `metrics.py`          | Win rate, PF, Sharpe, max DD, trade journal       |
+
+---
+
+## Disclaimer
+
+For educational and research use. Past performance does not guarantee
+future results. Always use proper risk management.

backtest/diagnose_filters.py

@@ -0,0 +1,609 @@
+"""
+ForexBot2 — Filter Rejection Diagnostic
+========================================
+Replays the same backtest loop as gpu_backtest.py but instead of
+collecting trades, counts how many M15 bars are rejected by each
+gate in the signal pipeline.
+
+Produces a summary table showing where potential trades are being
+lost, helping tune filter thresholds.
+
+Usage:
+    python backtest/diagnose_filters.py --pair XAUUSD --days 365
+    python backtest/diagnose_filters.py --pair XAUUSD --days 365 --sample-every 1
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import sys
+import time
+from collections import Counter, defaultdict
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple
+
+import numpy as np
+import pandas as pd
+
+# Ensure project root on sys.path.
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+from config import (  # noqa: E402
+    PAIR_INFO,
+    PlatformConfig,
+    SignalStrength,
+    Timeframe,
+)
+from signal_aggregator import SignalAggregator, SignalReport  # noqa: E402
+
+# Reuse GPU backtest infrastructure (data loading, indicator pre-compute, etc.)
+from gpu_backtest import (  # noqa: E402
+    BAR_SECONDS,
+    GPUTechnicalEngine,
+    PAIR_ALIASES,
+    BacktestConfig,
+    IndicatorCache,
+    fetch_history,
+    precompute_indicators,
+    _is_xau_blocked_by_calendar,
+)
+from data_provider import ForexDataProvider  # noqa: E402
+
+logger = logging.getLogger("diagnose_filters")
+
+
+# ═════════════════════════════════════════════════════════════════════
+#  OUTCOME CLASSIFICATION
+# ═════════════════════════════════════════════════════════════════════
+
+def classify_bar(
+    report: SignalReport,
+    is_xau: bool,
+) -> Tuple[str, List[str]]:
+    """Classify a bar's outcome and list which filters killed it.
+
+    Returns:
+        (outcome, [filter_names])
+
+    Outcomes:
+        "TRADE"           — quality C+ and trade_setup exists
+        "QUALITY_OK_NO_SETUP" — quality C+ but trade_setup is None
+        "LOW_QUALITY"     — signal reached BUY/SELL but quality D or F
+        "NEUTRAL"         — signal stayed NEUTRAL (score didn't reach +/-30)
+
+    Filter names are extracted from report.warnings.
+    """
+    filters_hit: List[str] = []
+
+    # Scan warnings for known filter tags
+    filter_tags = {
+        "H4 RSI GATE":       "H4_RSI_GATE",
+        "H1 MACD GATE":      "H1_MACD_GATE",
+        "RSI ZONE GATE":     "RSI_ZONE_GATE",
+        "XAU BUY FILTER":    "XAU_BUY_FILTER",
+        "SESSION FILTER":    "SESSION_FILTER",
+        "DAY FILTER":        "DAY_FILTER",
+        "LOW ATR FILTER":    "LOW_ATR_FILTER",
+        "SELL SUPPRESSED":   "SELL_SUPPRESSED",
+        "EXHAUSTION":        "EXHAUSTION_PENALTY",
+        "TREND EXHAUSTION":  "TREND_EXHAUSTION",
+        "NO PULLBACK":       "NO_PULLBACK_PENALTY",
+        "Weak pullback":     "WEAK_PULLBACK_PENALTY",
+        "Extended entry":    "EXTENDED_ENTRY_PENALTY",
+        "Counter-trend":     "COUNTER_TREND_PENALTY",
+        "EXTREME volatility":"EXTREME_VOL_PENALTY",
+        "NEAR EMA200":       "NEAR_EMA200_PENALTY",
+        "FAR FROM EMA200":   "FAR_EMA200_PENALTY",
+        "EVENT RISK":        "EVENT_RISK",
+        "REGIME FILTER":     "REGIME_FILTER",
+        "Timeframes CONFLICT": "MTF_CONFLICT",
+    }
+
+    for w in report.warnings:
+        for tag, name in filter_tags.items():
+            if tag in w:
+                filters_hit.append(name)
+
+    # Determine outcome
+    non_neutral = report.final_signal not in (SignalStrength.NEUTRAL,)
+    quality_ok = report.signal_quality in ("A+", "A", "B", "C")
+
+    if quality_ok and report.trade_setup is not None:
+        return "TRADE", filters_hit
+    elif quality_ok and report.trade_setup is None:
+        return "QUALITY_OK_NO_SETUP", filters_hit
+    elif non_neutral and not quality_ok:
+        return "LOW_QUALITY", filters_hit
+    else:
+        return "NEUTRAL", filters_hit
+
+
+# ═════════════════════════════════════════════════════════════════════
+#  SCORE DISTRIBUTION TRACKER
+# ═════════════════════════════════════════════════════════════════════
+
+class ScoreTracker:
+    """Tracks score/confidence distributions for analysis."""
+
+    def __init__(self):
+        self.scores: List[float] = []
+        self.confidences: List[float] = []
+        self.abs_scores: List[float] = []
+        # How many bars had score in various ranges
+        self.score_bins = Counter()
+        self.conf_bins = Counter()
+        # Track "near miss" bars — score close to thresholds
+        self.near_miss_score: int = 0  # score 20-29 (almost BUY/SELL)
+        self.near_miss_conf: int = 0   # confidence 45-54 (almost passing)
+
+    def record(self, score: float, confidence: float):
+        self.scores.append(score)
+        self.confidences.append(confidence)
+        self.abs_scores.append(abs(score))
+
+        # Score bins
+        a = abs(score)
+        if a < 10:
+            self.score_bins["0-9"] += 1
+        elif a < 20:
+            self.score_bins["10-19"] += 1
+        elif a < 30:
+            self.score_bins["20-29"] += 1
+            self.near_miss_score += 1
+        elif a < 40:
+            self.score_bins["30-39"] += 1
+        elif a < 50:
+            self.score_bins["40-49"] += 1
+        else:
+            self.score_bins["50+"] += 1
+
+        # Confidence bins
+        if confidence < 30:
+            self.conf_bins["0-29"] += 1
+        elif confidence < 45:
+            self.conf_bins["30-44"] += 1
+        elif confidence < 55:
+            self.conf_bins["45-54"] += 1
+            self.near_miss_conf += 1
+        elif confidence < 65:
+            self.conf_bins["55-64"] += 1
+        elif confidence < 75:
+            self.conf_bins["65-74"] += 1
+        else:
+            self.conf_bins["75+"] += 1
+
+
+# ═════════════════════════════════════════════════════════════════════
+#  MAIN DIAGNOSTIC LOOP
+# ═════════════════════════════════════════════════════════════════════
+
+def run_diagnosis(
+    pair: str,
+    start: datetime,
+    end: datetime,
+    sample_every: int = 4,
+    use_cache: bool = True,
+    primary_tf: Timeframe = Timeframe.M15,
+    confirm_tfs: Optional[List[Timeframe]] = None,
+    warmup_bars: int = 250,
+) -> None:
+    if confirm_tfs is None:
+        confirm_tfs = [Timeframe.H1, Timeframe.H4]
+
+    pair_info = PAIR_INFO[pair]
+    display = pair_info["display"]
+    is_xau = display == "XAU/USD"
+
+    platform_cfg = PlatformConfig(
+        symbol=pair,
+        display_name=display,
+        pip_value=pair_info["pip_value"],
+        primary_timeframe=primary_tf,
+        confirmation_timeframes=list(confirm_tfs),
+    )
+
+    provider = ForexDataProvider(platform_cfg)
+
+    # ── Load data ──
+    timeframes = [primary_tf] + list(confirm_tfs)
+    pad_days = max(60, warmup_bars // 4)
+    fetch_start = start - timedelta(days=pad_days)
+
+    dfs_full: Dict[Timeframe, pd.DataFrame] = {}
+    for tf in timeframes:
+        df = fetch_history(
+            provider, pair, tf,
+            start=fetch_start, end=end,
+            use_cache=use_cache,
+        )
+        if df.empty:
+            raise RuntimeError(f"No data for {pair} {tf.value}")
+        dfs_full[tf] = df
+
+    # ── Pre-compute indicators ──
+    t0 = time.perf_counter()
+    caches: Dict[str, IndicatorCache] = {
+        tf.value: precompute_indicators(dfs_full[tf], platform_cfg.technical, gpu_log=True)
+        for tf in timeframes
+    }
+    logger.info(f"Indicator pre-compute: {time.perf_counter() - t0:.2f}s")
+
+    # ── Build aggregator with GPU engine ──
+    aggregator = SignalAggregator(platform_cfg)
+    aggregator.tech_engine = GPUTechnicalEngine(platform_cfg.technical, caches)
+
+    # ── Walk primary TF ──
+    primary_df = dfs_full[primary_tf]
+    bt_start_ts = pd.Timestamp(start)
+    bt_end_ts = pd.Timestamp(end)
+
+    first_eligible = max(
+        warmup_bars,
+        int(primary_df.index.searchsorted(bt_start_ts)),
+    )
+    last_pos = min(
+        int(primary_df.index.searchsorted(bt_end_ts, side="right")),
+        len(primary_df),
+    )
+
+    primary_dur = pd.Timedelta(seconds=BAR_SECONDS[primary_tf])
+
+    # Counters
+    total_bars_in_window = 0
+    total_bars_sampled = 0
+    outcome_counts = Counter()
+    filter_counts = Counter()
+    # Filters that killed bars that were otherwise directional (non-NEUTRAL score)
+    directional_filter_counts = Counter()
+    # Pre-pipeline rejections (XAU calendar filters applied before analyze_from_dataframes)
+    pre_pipeline = Counter()
+
+    # Track signal directions
+    signal_dir_counts = Counter()
+    quality_counts = Counter()
+
+    # Score tracker
+    tracker = ScoreTracker()
+
+    # Track "would-have-been" trades: bars where the primary score was
+    # directional but got killed by a specific filter
+    filter_kill_breakdown = defaultdict(Counter)
+
+    logger.info(
+        f"Diagnosing {display} {primary_tf.value}: "
+        f"bars {first_eligible}-{last_pos} (every {sample_every}th bar)"
+    )
+
+    t_loop = time.perf_counter()
+    for pos in range(first_eligible, last_pos, sample_every):
+        total_bars_in_window += 1
+        ts = primary_df.index[pos]
+        decision_time = ts + primary_dur
+
+        # ── XAU calendar pre-filter (same as gpu_backtest.py) ──
+        if is_xau:
+            block = _is_xau_blocked_by_calendar(ts, platform_cfg)
+            if block:
+                pre_pipeline[block] += 1
+                continue
+
+        total_bars_sampled += 1
+
+        # ── Build sliced dfs (same logic as gpu_backtest.py) ──
+        slice_dfs: Dict[Timeframe, pd.DataFrame] = {}
+        for tf in timeframes:
+            full = dfs_full[tf]
+            bar_dur = pd.Timedelta(seconds=BAR_SECONDS[tf])
+            cutoff = decision_time - bar_dur
+            cut = int(full.index.searchsorted(cutoff, side="right"))
+            if cut == 0:
+                continue
+            slice_dfs[tf] = full.iloc[:cut]
+
+        if primary_tf not in slice_dfs:
+            continue
+
+        # ── Run the full signal pipeline ──
+        report: SignalReport = aggregator.analyze_from_dataframes(
+            symbol=pair,
+            dfs=slice_dfs,
+            context=None,
+            use_news=False,
+        )
+
+        # Record score/confidence
+        tracker.record(report.final_score, report.final_confidence)
+
+        # Signal direction
+        signal_dir_counts[report.final_signal.name] += 1
+        quality_counts[report.signal_quality] += 1
+
+        # Classify
+        outcome, filters_hit = classify_bar(report, is_xau)
+        outcome_counts[outcome] += 1
+
+        for f in filters_hit:
+            filter_counts[f] += 1
+
+        # If the primary score was directional (>= 15 or <= -15), the bar
+        # had some potential — track what killed it
+        if abs(report.primary_score) >= 15:
+            for f in filters_hit:
+                directional_filter_counts[f] += 1
+
+        # For bars that got quality D/F but had a directional signal,
+        # track which filter warnings appeared
+        if outcome in ("LOW_QUALITY", "QUALITY_OK_NO_SETUP"):
+            for f in filters_hit:
+                filter_kill_breakdown[outcome][f] += 1
+
+    elapsed = time.perf_counter() - t_loop
+
+    # ═════════════════════════════════════════════════════════════════
+    #  PRINT RESULTS
+    # ═════════════════════════════════════════════════════════════════
+
+    bar = "=" * 80
+    print(f"\n{bar}")
+    print(f"  FILTER REJECTION DIAGNOSTIC — {display}")
+    print(f"  {start.date()} to {end.date()} | {primary_tf.value} "
+          f"(sampled every {sample_every}th bar)")
+    print(f"{bar}")
+
+    # Scale counts if sampling
+    scale_note = ""
+    if sample_every > 1:
+        scale_note = f"  (sampled {total_bars_sampled} of ~{total_bars_in_window * sample_every} eligible bars)"
+    print(f"\n  Total bars in backtest window: {total_bars_in_window}")
+    if pre_pipeline:
+        for reason, count in sorted(pre_pipeline.items()):
+            print(f"  Pre-pipeline rejection ({reason}): {count}")
+    print(f"  Bars analyzed: {total_bars_sampled}{scale_note}")
+    print(f"  Analysis took: {elapsed:.1f}s "
+          f"({total_bars_sampled / max(elapsed, 0.001):.0f} bars/s)")
+
+    # ── Outcome breakdown ──
+    print(f"\n{'-' * 80}")
+    print("  OUTCOME BREAKDOWN")
+    print(f"{'-' * 80}")
+    outcome_order = ["TRADE", "QUALITY_OK_NO_SETUP", "LOW_QUALITY", "NEUTRAL"]
+    outcome_labels = {
+        "TRADE":               "Actual trades (quality C+ & setup exists)",
+        "QUALITY_OK_NO_SETUP": "Quality C+ but NO trade setup generated",
+        "LOW_QUALITY":         "Directional signal but quality D/F",
+        "NEUTRAL":             "NEUTRAL (score didn't reach +/-30 or conf < 55)",
+    }
+    for o in outcome_order:
+        count = outcome_counts.get(o, 0)
+        pct = count / max(total_bars_sampled, 1) * 100
+        label = outcome_labels.get(o, o)
+        print(f"  {count:6d} ({pct:5.1f}%)  {label}")
+
+    # ── Signal direction breakdown ──
+    print(f"\n{'-' * 80}")
+    print("  SIGNAL DIRECTION BREAKDOWN")
+    print(f"{'-' * 80}")
+    for name in ["STRONG_BUY", "BUY", "NEUTRAL", "SELL", "STRONG_SELL"]:
+        count = signal_dir_counts.get(name, 0)
+        pct = count / max(total_bars_sampled, 1) * 100
+        print(f"  {count:6d} ({pct:5.1f}%)  {name}")
+
+    # ── Quality grade breakdown ──
+    print(f"\n{'-' * 80}")
+    print("  QUALITY GRADE BREAKDOWN")
+    print(f"{'-' * 80}")
+    for grade in ["A+", "A", "B", "C", "D", "F"]:
+        count = quality_counts.get(grade, 0)
+        pct = count / max(total_bars_sampled, 1) * 100
+        print(f"  {count:6d} ({pct:5.1f}%)  Grade {grade}")
+
+    # ── Filter hit counts (all bars) ──
+    print(f"\n{'-' * 80}")
+    print("  FILTER HIT COUNTS (all bars, a bar can hit multiple filters)")
+    print(f"{'-' * 80}")
+    if filter_counts:
+        for name, count in filter_counts.most_common():
+            pct = count / max(total_bars_sampled, 1) * 100
+            print(f"  {count:6d} ({pct:5.1f}%)  {name}")
+    else:
+        print("  (no filters triggered)")
+
+    # ── Filters that killed directional bars (primary_score >= 15) ──
+    print(f"\n{'-' * 80}")
+    print("  FILTERS KILLING DIRECTIONAL BARS (|primary_score| >= 15)")
+    print(f"{'-' * 80}")
+    if directional_filter_counts:
+        for name, count in directional_filter_counts.most_common():
+            print(f"  {count:6d}  {name}")
+    else:
+        print("  (none)")
+
+    # ── Filter breakdown for LOW_QUALITY bars ──
+    if filter_kill_breakdown.get("LOW_QUALITY"):
+        print(f"\n{'-' * 80}")
+        print("  FILTERS ON LOW_QUALITY BARS (had BUY/SELL but quality D/F)")
+        print(f"{'-' * 80}")
+        for name, count in filter_kill_breakdown["LOW_QUALITY"].most_common():
+            print(f"  {count:6d}  {name}")
+
+    # ── Filter breakdown for QUALITY_OK_NO_SETUP bars ──
+    if filter_kill_breakdown.get("QUALITY_OK_NO_SETUP"):
+        print(f"\n{'-' * 80}")
+        print("  FILTERS ON QUALITY_OK_NO_SETUP BARS (quality C+ but no trade setup)")
+        print(f"{'-' * 80}")
+        for name, count in filter_kill_breakdown["QUALITY_OK_NO_SETUP"].most_common():
+            print(f"  {count:6d}  {name}")
+
+    # ── Score distribution ──
+    print(f"\n{'-' * 80}")
+    print("  SCORE DISTRIBUTION (|score|)")
+    print(f"{'-' * 80}")
+    for bin_name in ["0-9", "10-19", "20-29", "30-39", "40-49", "50+"]:
+        count = tracker.score_bins.get(bin_name, 0)
+        pct = count / max(total_bars_sampled, 1) * 100
+        marker = " <-- near miss (almost BUY/SELL)" if bin_name == "20-29" else ""
+        print(f"  {count:6d} ({pct:5.1f}%)  |score| {bin_name}{marker}")
+
+    # ── Confidence distribution ──
+    print(f"\n{'-' * 80}")
+    print("  CONFIDENCE DISTRIBUTION")
+    print(f"{'-' * 80}")
+    for bin_name in ["0-29", "30-44", "45-54", "55-64", "65-74", "75+"]:
+        count = tracker.conf_bins.get(bin_name, 0)
+        pct = count / max(total_bars_sampled, 1) * 100
+        marker = " <-- near miss (almost passing)" if bin_name == "45-54" else ""
+        print(f"  {count:6d} ({pct:5.1f}%)  confidence {bin_name}{marker}")
+
+    # ── Near misses ──
+    print(f"\n{'-' * 80}")
+    print("  NEAR MISSES")
+    print(f"{'-' * 80}")
+    print(f"  {tracker.near_miss_score:6d}  Bars with |score| 20-29 (threshold is 30)")
+    print(f"  {tracker.near_miss_conf:6d}  Bars with confidence 45-54 (threshold is 55)")
+
+    # ── Summary statistics ──
+    if tracker.scores:
+        scores_arr = np.array(tracker.scores)
+        conf_arr = np.array(tracker.confidences)
+        print(f"\n{'-' * 80}")
+        print("  SCORE/CONFIDENCE STATISTICS")
+        print(f"{'-' * 80}")
+        print(f"  Score:      mean={np.mean(scores_arr):+.1f}  "
+              f"median={np.median(scores_arr):+.1f}  "
+              f"std={np.std(scores_arr):.1f}  "
+              f"min={np.min(scores_arr):+.1f}  "
+              f"max={np.max(scores_arr):+.1f}")
+        print(f"  Confidence: mean={np.mean(conf_arr):.1f}  "
+              f"median={np.median(conf_arr):.1f}  "
+              f"std={np.std(conf_arr):.1f}  "
+              f"min={np.min(conf_arr):.1f}  "
+              f"max={np.max(conf_arr):.1f}")
+
+    # ── Blocked hours analysis (for XAU) ──
+    if is_xau:
+        blocked = set(platform_cfg.xau_strategy.blocked_hours_utc)
+        all_hours = set(range(24))
+        open_hours = sorted(all_hours - blocked)
+        print(f"\n{'-' * 80}")
+        print("  XAU HOUR/DAY CONFIG")
+        print(f"{'-' * 80}")
+        print(f"  Blocked hours:  {sorted(blocked)}")
+        print(f"  Open hours:     {open_hours}")
+        print(f"  Allowed days:   {platform_cfg.xau_strategy.allowed_weekdays} "
+              f"(0=Mon..4=Fri)")
+        print(f"  Open fraction:  {len(open_hours)/24:.0%} of hours x "
+              f"{len(platform_cfg.xau_strategy.allowed_weekdays)}/5 weekdays = "
+              f"{len(open_hours)/24 * len(platform_cfg.xau_strategy.allowed_weekdays)/5:.0%} of trading time")
+
+    # ── Interpretation / key takeaways ──
+    print(f"\n{bar}")
+    print("  KEY TAKEAWAYS")
+    print(f"{bar}")
+
+    neutral_pct = outcome_counts.get("NEUTRAL", 0) / max(total_bars_sampled, 1) * 100
+    low_q_pct = outcome_counts.get("LOW_QUALITY", 0) / max(total_bars_sampled, 1) * 100
+    trade_count = outcome_counts.get("TRADE", 0)
+
+    print(f"  - {neutral_pct:.1f}% of bars never reach a directional signal (NEUTRAL)")
+    print(f"  - {low_q_pct:.1f}% of bars get a BUY/SELL but confidence/quality too low")
+    print(f"  - {trade_count} bars produce actual trades")
+
+    if filter_counts:
+        top3 = filter_counts.most_common(3)
+        print(f"  - Top rejection filters:")
+        for name, count in top3:
+            print(f"      {name}: {count} bars hit")
+
+    if tracker.near_miss_score > 50:
+        print(f"  - SIGNIFICANT: {tracker.near_miss_score} bars had |score| 20-29 "
+              f"(lowering threshold from 30 to 25 would capture many)")
+
+    if tracker.near_miss_conf > 50:
+        print(f"  - SIGNIFICANT: {tracker.near_miss_conf} bars had confidence 45-54 "
+              f"(lowering threshold from 55 to 50 would capture many)")
+
+    print(f"\n{bar}\n")
+
+
+# ═════════════════════════════════════════════════════════════════════
+#  CLI
+# ═════════════════════════════════════════════════════════════════════
+
+def _resolve_pair(s: str) -> str:
+    s_up = s.upper().strip()
+    if s_up in PAIR_ALIASES:
+        return PAIR_ALIASES[s_up]
+    if s_up in PAIR_INFO:
+        return s_up
+    raise SystemExit(f"Unknown pair '{s}'. Valid: {', '.join(sorted(PAIR_ALIASES))}")
+
+
+def _resolve_tf(s: str) -> Timeframe:
+    for tf in Timeframe:
+        if tf.value == s:
+            return tf
+    raise SystemExit(f"Unknown timeframe '{s}'")
+
+
+def main(argv: Optional[List[str]] = None) -> int:
+    p = argparse.ArgumentParser(
+        description="Diagnose which filters reject the most bars in the backtest pipeline",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    p.add_argument("--pair", required=True, help="EURUSD | GBPUSD | USDJPY | XAUUSD")
+    p.add_argument("--days", type=int, default=365, help="Days of history")
+    p.add_argument("--start", help="ISO date YYYY-MM-DD (overrides --days)")
+    p.add_argument("--end", help="ISO date YYYY-MM-DD (defaults to today)")
+    p.add_argument("--primary-tf", default="15min", help="Primary TF")
+    p.add_argument("--confirm-tfs", default="1h,4h", help="Comma-separated confirmation TFs")
+    p.add_argument("--warmup-bars", type=int, default=250)
+    p.add_argument("--sample-every", type=int, default=4,
+                   help="Analyze every Nth bar (1=all, 4=25%% sample for speed)")
+    p.add_argument("--no-cache", action="store_true", help="Skip parquet cache")
+    p.add_argument("--log-level", default="INFO")
+    args = p.parse_args(argv)
+
+    logging.basicConfig(
+        level=args.log_level.upper(),
+        format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+    )
+
+    pair = _resolve_pair(args.pair)
+    primary_tf = _resolve_tf(args.primary_tf)
+    confirm_tfs = [_resolve_tf(s.strip()) for s in args.confirm_tfs.split(",") if s.strip()]
+
+    end = (
+        datetime.fromisoformat(args.end).replace(tzinfo=timezone.utc)
+        if args.end else datetime.now(timezone.utc)
+    )
+    if args.start:
+        start = datetime.fromisoformat(args.start).replace(tzinfo=timezone.utc)
+    else:
+        start = end - timedelta(days=args.days)
+
+    logger.info(
+        f"Diagnosing {PAIR_INFO[pair]['display']} | "
+        f"Window: {start.date()} -> {end.date()} | "
+        f"Sample: every {args.sample_every}th bar"
+    )
+
+    run_diagnosis(
+        pair=pair,
+        start=start,
+        end=end,
+        sample_every=args.sample_every,
+        use_cache=not args.no_cache,
+        primary_tf=primary_tf,
+        confirm_tfs=confirm_tfs,
+        warmup_bars=args.warmup_bars,
+    )
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

backtest/instructions_for_backtest.txt

@@ -0,0 +1,59 @@
+Note down entery and exit Why we took trade?
+What were each indicator signal on that specific time when we took trade
+How many indicators agreed at trade time
+Always backtest using engine.py and signal_aggregator.py
+Dont take trades on weekends and Fridays
+Keep TP small 1.3 Reward ratio
+At the end of test make table of results and print all values you observed.
+Trade context
+
+Time of day and trading session (Asian/London/NY/overlap)
+Day of week (track Mon/Tue/Wed/Thu performance separately)
+Market regime at entry: trending, ranging, or choppy (use ATR or ADX)
+Volatility level (ATR value or ATR percentile vs recent average)
+Pair/instrument traded
+News proximity (high-impact news within 30 min before/after entry)
+
+Trade execution details
+
+Spread at entry in pips
+Slippage on entry and exit (planned vs actual fill)
+Time in trade (minutes/hours from entry to exit)
+Maximum Adverse Excursion (MAE): how far against you it went
+Maximum Favorable Excursion (MFE): how far in profit before pullback
+
+Risk metrics
+
+Max consecutive losses (streak)
+Max consecutive wins (streak)
+Expectancy: (Win% x Avg Win) minus (Loss% x Avg Loss)
+Expectancy per hour in trade
+Recovery factor: Total R divided by Max DD
+
+Indicator/signal analysis
+
+Each indicator's standalone win rate when it agreed with the trade
+Confluence level performance (2 vs 3 vs 4+ indicators agreeing)
+Which indicator disagreed on losing trades (find the "canary")
+
+Decision quality
+
+Rate each setup A/B/C at entry, before knowing outcome
+Verify A-grade setups actually win more often
+Track system trades vs discretionary deviations separately
+
+Process notes
+
+Screenshot or chart snapshot at entry and exit
+Emotional state (calm, FOMO, revenge, tired)
+What would have happened if you held longer or exited sooner
+
+Aggregate analysis at end of backtest
+
+Equity curve, not just total R
+Performance breakdown by session, day, volatility regime
+Win rate over rolling 20-trade windows (check for edge decay)
+
+Critical extra
+"Do not trade" log: setups flagged but skipped, and what happened after
+

config.py

@@ -1,8 +1,6 @@
 """
-Forex Signal Intelligence Platform - Configuration
+ForexBot2 — Configuration (XAU/USD focused)
-===================================================
+==========================================
-Central configuration for all signal analysis parameters.
-Tuned by market regime and validated against historical performance.
 """
 
 from dataclasses import dataclass, field
@@ -37,160 +35,208 @@ class MarketSession(Enum):
     OVERLAP_LONDON_NY = "london_ny_overlap"
 
 
-# ── Major Forex Pairs ──────────────────────────────────────────────
+# ── Supported pairs ──────────────────────────────────────────────
-MAJOR_PAIRS = [
+XAUUSD = "XAU/USD"
-    "EURUSD=X", "GBPUSD=X", "USDJPY=X", "USDCHF=X",
+EURUSD = "EUR/USD"
-    "AUDUSD=X", "USDCAD=X", "NZDUSD=X",
-]
 
-CROSS_PAIRS = [
+PAIR_INFO = {
-    "EURGBP=X", "EURJPY=X", "GBPJPY=X", "AUDJPY=X",
+    XAUUSD: {"display": "XAU/USD", "pip_value": 0.10, "is_jpy_pair": False},
-    "EURCHF=X", "CADJPY=X",
+    EURUSD: {"display": "EUR/USD", "pip_value": 0.0001, "is_jpy_pair": False},
-]
-
-# Yahoo Finance symbol mapping for display
-SYMBOL_DISPLAY = {
-    "EURUSD=X": "EUR/USD", "GBPUSD=X": "GBP/USD",
-    "USDJPY=X": "USD/JPY", "USDCHF=X": "USD/CHF",
-    "AUDUSD=X": "AUD/USD", "USDCAD=X": "USD/CAD",
-    "NZDUSD=X": "NZD/USD", "EURGBP=X": "EUR/GBP",
-    "EURJPY=X": "EUR/JPY", "GBPJPY=X": "GBP/JPY",
-    "AUDJPY=X": "AUD/JPY", "EURCHF=X": "EUR/CHF",
-    "CADJPY=X": "CAD/JPY",
 }
 
 
 @dataclass
 class TechnicalConfig:
-    """Technical indicator parameters - carefully tuned for Forex."""
+    """Technical indicator parameters."""
-    # EMA periods (fast/medium/slow for trend detection)
     ema_fast: int = 9
     ema_medium: int = 21
     ema_slow: int = 50
     ema_trend: int = 200
 
-    # RSI
     rsi_period: int = 14
     rsi_overbought: float = 70.0
     rsi_oversold: float = 30.0
     rsi_neutral_upper: float = 60.0
     rsi_neutral_lower: float = 40.0
 
-    # MACD (12, 26, 9 is standard)
     macd_fast: int = 12
     macd_slow: int = 26
     macd_signal: int = 9
 
-    # Bollinger Bands
     bb_period: int = 20
     bb_std: float = 2.0
 
-    # ATR for volatility measurement
     atr_period: int = 14
 
-    # Stochastic
-    stoch_k: int = 14
-    stoch_d: int = 3
-    stoch_overbought: float = 80.0
-    stoch_oversold: float = 20.0
-
-    # ADX for trend strength
     adx_period: int = 14
     adx_strong_trend: float = 25.0
     adx_very_strong: float = 50.0
 
-    # Ichimoku
-    ichimoku_tenkan: int = 9
-    ichimoku_kijun: int = 26
-    ichimoku_senkou_b: int = 52
-
-    # Volume profile
-    volume_ma_period: int = 20
-
-    # Pivot points lookback
-    pivot_lookback: int = 1
-
-    # Divergence detection — how many bars back to scan for swing points
     divergence_lookback: int = 14
+    swing_sr_lookback: int = 50
+
+    disabled_indicators: List[str] = field(default_factory=list)
+
+    ema_mode: str = "4ema"          # "4ema" (9/21/50/200) or "2ema" (50/200)
+    rsi_mode: str = "classic"       # "classic" or "regime_aware"
+    macd_as_pillar: bool = True     # MACD histogram is a scored signal
+
+    # ROC (Rate of Change)
+    roc_period: int = 10
+    roc_threshold: float = 1.0     # % threshold for strong signal
+
+    # Keltner Channels
+    kc_period: int = 20
+    kc_atr_period: int = 10
+    kc_multiplier: float = 1.5
+
+    # ADX slope (trend acceleration detection)
+    adx_slope_lookback: int = 5
+    adx_slope_emerging_threshold: float = 3.0
+
+    regime_weights_override: Dict[str, Dict[str, float]] = field(default_factory=dict)
 
 
 @dataclass
 class SignalConfig:
-    """Signal scoring and filtering parameters."""
+    """Signal scoring & filtering."""
-    # Minimum confidence to emit a signal (0-100)
     min_confidence: float = 55.0
 
-    # Weight allocation for composite score
     weights: Dict[str, float] = field(default_factory=lambda: {
-        "trend": 0.25,          # EMA alignment, ADX
+        "ema_trend": 1.0,
-        "momentum": 0.20,       # RSI, Stochastic, MACD
+        "rsi": 1.5,
-        "volatility": 0.15,     # BB position, ATR regime
+        "bollinger": 1.5,
-        "price_action": 0.15,   # Candlestick patterns, S/R
+        "macd": 1.5,
-        "multi_timeframe": 0.15,# Higher TF confirmation
+        "keltner": 1.5,
-        "sentiment": 0.10,      # News/calendar events
+        "roc": 1.5,
+        "candlestick": 1.0,
+        "divergence": 1.5,
     })
 
-    # Multi-timeframe confirmation requirements
+    mtf_agreement_threshold: int = 2  # of 3 timeframes
-    mtf_agreement_threshold: int = 2  # out of 3 timeframes must agree
-
-    # Signal cooldown (prevent over-signalling same direction)
     cooldown_bars: int = 5
 
-    # Risk management defaults
+    default_risk_pct: float = 1.0
-    default_risk_pct: float = 1.0  # risk 1% per trade
+    default_rr_ratio: float = 2.0
-    default_rr_ratio: float = 2.0  # 1:2 risk-reward minimum
+    atr_sl_multiplier: float = 1.5
+    min_rr_ratio: float = 0.5
 
 
 @dataclass
 class NewsConfig:
-    """Economic calendar and sentiment configuration."""
+    """Economic calendar & sentiment."""
-    # High-impact event currencies
+    tracked_currencies: List[str] = field(default_factory=lambda: ["USD", "XAU", "EUR"])
-    tracked_currencies: List[str] = field(default_factory=lambda: [
+    min_impact: str = "medium"
-        "USD", "EUR", "GBP", "JPY", "CHF", "AUD", "CAD", "NZD"
-    ])
-
-    # Event impact levels to monitor
-    min_impact: str = "medium"  # "low", "medium", "high"
-
-    # Pre-event buffer (minutes before high-impact event = no new signals)
     pre_event_buffer_minutes: int = 30
-
-    # Post-event volatility window (minutes after event = cautious signals)
     post_event_buffer_minutes: int = 15
 
-    # News sentiment sources
     rss_feeds: List[str] = field(default_factory=lambda: [
-        "https://www.forexlive.com/feed/",
+        "https://investinglive.com/feed/",
         "https://www.fxstreet.com/rss",
     ])
 
 
+@dataclass
+class XauStrategyConfig:
+    """XAU/USD strategy config.
+    Legacy gates (H4 RSI, H1 MACD, M15 score) removed in favor of
+    new momentum indicators (ROC, Keltner squeeze, ADX slope).
+    """
+    enabled: bool = True
+    rsi_entry_threshold: float = 45.0
+    tp_atr_multiplier: float = 1.0
+    sl_atr_multiplier: float = 2.0
+    min_ema200_tfs: int = 2
+    allow_generic_trades: bool = True
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [0, 1, 2])
+    weekend_filter: bool = True
+    min_atr_threshold: float = 1.0
+    v2_sl_atr_multiplier: float = 2.5
+    buy_threshold_boost: int = 0
+    m5_confirmation_bars: int = 0
+    h4_rsi_gate: bool = False
+    h1_macd_gate: bool = False
+    rsi_zone: tuple = (40.0, 65.0)
+    allowed_weekdays: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 6])  # Mon-Fri + Sun (Sydney open)
+    sell_min_quality: List[str] = field(default_factory=lambda: ["A+", "A", "B"])
+
+
+@dataclass
+class EurUsdStrategyConfig:
+    """EUR/USD strategy config — separate from XAU for independent tuning."""
+    enabled: bool = True
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [22, 23, 0, 1, 2, 3])
+    weekend_filter: bool = True
+    allowed_weekdays: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4])
+    min_atr_threshold: float = 0.0003
+    sl_atr_multiplier: float = 1.5
+    tp_atr_multiplier: float = 1.0
+
+    sl_atr_floor: float = 1.2
+    min_rr_ratio: float = 0.5
+    mature_move_threshold: float = 35.0
+    tp_quality_scale: Dict[str, float] = field(default_factory=lambda: {
+        "A+": 1.0, "A": 1.0, "B": 1.0, "C": 1.0,
+    })
+    blocked_regimes: List[str] = field(default_factory=lambda: ["ranging"])
+    min_quality_grades: List[str] = field(default_factory=lambda: ["A+", "A", "B"])
+    max_exhaustion_tags: int = 99
+    block_near_ema200: bool = False
+    session_hours: List[int] = field(default_factory=list)
+    min_entry_score: float = 38.0
+    min_entry_confidence: float = 60.0
+    block_rsi_extended: bool = True
+    be_trigger_r: float = 0.3
+    be_buffer_r: float = 0.05
+
+
+@dataclass
+class FilterConfig:
+    """Regime and session filters for live signals."""
+    regime_filter: bool = False
+    regime_filter_timeframe: Timeframe = Timeframe.H4
+    regime_filter_min_adx: float = 25.0
+
+    session_filter: bool = True
+    session_start_hour: int = 3
+    session_end_hour: int = 23
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [0, 1, 2])
+
+    allowed_weekdays: List[int] = field(default_factory=lambda: [0, 1, 2, 3, 4, 6])  # Mon-Fri + Sun (Sydney open)
+
+
 @dataclass
 class PlatformConfig:
-    """Overall platform configuration."""
+    """Top-level configuration."""
     technical: TechnicalConfig = field(default_factory=TechnicalConfig)
     signals: SignalConfig = field(default_factory=SignalConfig)
     news: NewsConfig = field(default_factory=NewsConfig)
+    backtest: FilterConfig = field(default_factory=FilterConfig)
+    xau_strategy: XauStrategyConfig = field(default_factory=XauStrategyConfig)
+    eur_strategy: EurUsdStrategyConfig = field(default_factory=EurUsdStrategyConfig)
 
-    # Default pairs to monitor
+    symbol: str = XAUUSD
-    watch_pairs: List[str] = field(default_factory=lambda: MAJOR_PAIRS)
+    display_name: str = "XAU/USD"
+    pip_value: float = 0.10
 
-    # Analysis timeframes (primary + confirmation)
     primary_timeframe: Timeframe = Timeframe.H1
     confirmation_timeframes: List[Timeframe] = field(
         default_factory=lambda: [Timeframe.M15, Timeframe.H4]
     )
 
-    # Data source
+    data_lookback_days: int = 60
-    data_lookback_days: int = 60  # enough for 200 EMA on H1
 
-    # Twelve Data API key (overridden by TWELVE_DATA_API_KEY env var)
     twelve_data_api_key: str = "f563b22ffdfd4924955decbe1a8c8701"
 
-    # Dashboard refresh interval (seconds)
+    refresh_interval: int = 300
-    refresh_interval: int = 300  # 5 minutes
+
+    scan_pairs: List[str] = field(
+        default_factory=lambda: [XAUUSD]
+    )
+    scan_timeframes: List[Timeframe] = field(
+        default_factory=lambda: [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4]
+    )
+    candle_buffer_seconds: int = 10
+    ws_cooldown_seconds: int = 300
 
-    # Logging
     log_level: str = "INFO"
-    log_signals_to_file: bool = True
-    signal_log_path: str = "signals.log"

data_provider.py

@@ -1,41 +1,45 @@
 """
-Forex Signal Intelligence Platform - Data Provider
+ForexBot2 — Data Provider (Twelve Data SDK)
-====================================================
+==========================================================
-Retrieves and validates market data from Twelve Data API.
+Fetches and validates OHLCV via the official `twelvedata` Python SDK.
-Falls back to synthetic GBM data if the API is unavailable.
 """
 
-import hashlib
 import logging
 import os
 import time
 from collections import deque
-from datetime import datetime, timedelta, timezone
+from typing import Dict, Optional
-from typing import Dict, List, Optional
 
-import numpy as np
 import pandas as pd
-import requests
+from twelvedata import TDClient
 
-from config import PlatformConfig, Timeframe, SYMBOL_DISPLAY
+from config import PlatformConfig, Timeframe, XAUUSD, PAIR_INFO
 
 logger = logging.getLogger(__name__)
 
 
+def _is_forex_weekend(now=None) -> bool:
+    """Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC."""
+    if now is None:
+        now = pd.Timestamp.now(tz="UTC")
+    wd = now.weekday()
+    hour = now.hour
+    if wd == 4 and hour >= 22:
+        return True
+    if wd == 5:
+        return True
+    if wd == 6 and hour < 22:
+        return True
+    return False
+
+
 class DataIntegrityError(Exception):
     """Raised when data fails validation checks."""
     pass
 
 
 class ForexDataProvider:
-    """
+    """Twelve Data SDK client + on-disk parquet cache."""
-    Forex data provider backed by Twelve Data API.
-
-    Twelve Data has native 4h candles, eliminating the H1→H4 resampling
-    hack needed for Yahoo Finance. Free tier: 800 credits/day, 8 req/min.
-    """
-
-    TD_BASE_URL = "https://api.twelvedata.com"
 
     TD_INTERVALS = {
         Timeframe.M1:  "1min",
@@ -48,20 +52,19 @@ class ForexDataProvider:
         Timeframe.W1:  "1week",
     }
 
-    # Approximate bars per day for each timeframe (used to size outputsize)
     BARS_PER_DAY = {
-        Timeframe.M1:  1440,
+        Timeframe.M1: 1440, Timeframe.M5: 288, Timeframe.M15: 96,
-        Timeframe.M5:  288,
+        Timeframe.M30: 48, Timeframe.H1: 24, Timeframe.H4: 6,
-        Timeframe.M15: 96,
+        Timeframe.D1: 1, Timeframe.W1: 0.143,
-        Timeframe.M30: 48,
-        Timeframe.H1:  24,
-        Timeframe.H4:  6,
-        Timeframe.D1:  1,
-        Timeframe.W1:  0.143,
     }
 
-    # Free plan: 8 API credits per minute (each symbol = 1 credit)
+    CANDLE_SECONDS = {
-    RATE_LIMIT = 8
+        Timeframe.M1: 60, Timeframe.M5: 300, Timeframe.M15: 900,
+        Timeframe.M30: 1800, Timeframe.H1: 3600, Timeframe.H4: 14400,
+        Timeframe.D1: 86400, Timeframe.W1: 604800,
+    }
+
+    RATE_LIMIT = 55
     RATE_WINDOW = 60.0
 
     def __init__(self, config: PlatformConfig):
@@ -72,94 +75,97 @@ class ForexDataProvider:
         )
         self._cache: Dict[str, pd.DataFrame] = {}
         self._cache_timestamps: Dict[str, float] = {}
-        self._request_times: deque = deque()  # timestamps of recent API credits used
+        self._request_times: deque = deque()
-        self._session = requests.Session()
+        self._td: Optional[TDClient] = (
-        self._session.headers.update({
+            TDClient(apikey=self._api_key) if self._api_key else None
-            "User-Agent": "Mozilla/5.0 (compatible; ForexIntel/1.0)"
+        )
-        })
 
     @staticmethod
     def _to_td_symbol(symbol: str) -> str:
-        """Convert internal symbol (EURUSD=X) to Twelve Data format (EUR/USD)."""
+        if "/" in symbol:
+            return symbol
         clean = symbol.replace("=X", "")
         return f"{clean[:3]}/{clean[3:]}"
 
+    # ── Public API ─────────────────────────────────────────────────
+
     def fetch_ohlcv(
         self,
-        symbol: str,
+        symbol: str = XAUUSD,
-        timeframe: Timeframe,
+        timeframe: Timeframe = Timeframe.H1,
         lookback_days: Optional[int] = None,
     ) -> pd.DataFrame:
-        """
+        """Fetch OHLCV with in-memory caching for live mode."""
-        Fetch OHLCV data for a Forex pair with full validation.
-
-        Returns a DataFrame with columns: Open, High, Low, Close, Volume
-        Index is a timezone-aware DatetimeIndex in UTC.
-        """
         if lookback_days is None:
             lookback_days = self.config.data_lookback_days
 
         cache_key = f"{symbol}_{timeframe.value}"
         now = time.time()
 
+        cache_ttl = self.CANDLE_SECONDS.get(timeframe, self.config.refresh_interval)
         if (
             cache_key in self._cache_timestamps
-            and (now - self._cache_timestamps[cache_key]) < self.config.refresh_interval
+            and (now - self._cache_timestamps[cache_key]) < cache_ttl
             and cache_key in self._cache
         ):
             cached = self._cache.get(cache_key)
             if cached is not None and not cached.empty:
+                if self._is_data_fresh(cached, symbol, timeframe):
                     return cached
+                logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
+                del self._cache[cache_key]
+                del self._cache_timestamps[cache_key]
 
-        df = self._fetch_twelve_data(symbol, timeframe, lookback_days)
+        outputsize = max(
+            300,
+            min(int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20, 5000),
+        )
+        df = self._fetch_twelve_data(symbol, timeframe, outputsize)
         df = self._validate_and_clean(df, symbol, timeframe)
 
+        if not self._is_data_fresh(df, symbol, timeframe):
+            logger.warning(
+                f"STALE DATA from API: {symbol} {timeframe.value} — "
+                f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
+            )
+
         self._cache[cache_key] = df
         self._cache_timestamps[cache_key] = now
         return df
 
-    def prefetch_batch(
+    def _is_data_fresh(self, df: pd.DataFrame, symbol: str, timeframe: Timeframe) -> bool:
-        self,
+        """Check if the last candle is within acceptable age for this timeframe."""
-        symbols: List[str],
+        if df is None or df.empty:
-        timeframes: List[Timeframe],
+            return False
-        lookback_days: Optional[int] = None,
-    ) -> None:
-        """
-        Batch-fetch all symbols for each timeframe in a single API call per TF.
 
-        Twelve Data accepts comma-separated symbols, so 7 pairs × 3 timeframes
+        now = pd.Timestamp.now(tz="UTC")
-        = 3 requests instead of 21 — well within the free-tier 8 req/min limit.
-        Results are stored in the local cache; subsequent fetch_ohlcv calls
-        for the same symbol/TF will be served from cache.
-        """
-        if lookback_days is None:
-            lookback_days = self.config.data_lookback_days
 
-        for tf in timeframes:
+        if _is_forex_weekend(now):
-            try:
+            return True
-                self._batch_fetch_timeframe(symbols, tf, lookback_days)
-                time.sleep(1.0)  # brief pause between TF requests
-            except Exception as e:
-                logger.warning(f"Batch prefetch failed for {tf.value}: {e}")
 
-    def fetch_multiple(
+        last_bar = df.index[-1]
-        self,
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
-        symbols: List[str],
+        max_age_seconds = candle_seconds * 3
-        timeframe: Timeframe,
+        age_seconds = (now - last_bar).total_seconds()
-        lookback_days: Optional[int] = None,
-    ) -> Dict[str, pd.DataFrame]:
-        """Fetch data for multiple symbols, returning dict of DataFrames."""
-        results = {}
-        for symbol in symbols:
-            try:
-                df = self.fetch_ohlcv(symbol, timeframe, lookback_days)
-                results[symbol] = df
-            except Exception as e:
-                logger.warning(f"Failed to fetch {symbol}: {e}")
-        return results
 
-    def get_latest_price(self, symbol: str) -> Optional[float]:
+        if age_seconds > max_age_seconds:
-        """Return the most recent close price for a symbol."""
+            logger.warning(
+                f"{symbol} {timeframe.value}: last bar {last_bar.strftime('%Y-%m-%d %H:%M')} "
+                f"is {age_seconds / 60:.0f}m old (limit: {max_age_seconds / 60:.0f}m)"
+            )
+            return False
+
+        return True
+
+    def clear_cache(self) -> int:
+        """Wipe all cached data. Returns number of entries cleared."""
+        count = len(self._cache)
+        self._cache.clear()
+        self._cache_timestamps.clear()
+        logger.info(f"Cache cleared: {count} entries wiped")
+        return count
+
+    def get_latest_price(self, symbol: str = XAUUSD) -> Optional[float]:
         try:
             df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
             if df is not None and not df.empty:
@@ -168,47 +174,26 @@ class ForexDataProvider:
             pass
         return None
 
-    def get_spread_estimate(self, symbol: str) -> float:
+    def get_spread_estimate(self, symbol: str = XAUUSD) -> float:
-        """Estimate typical spread from recent M5 high-low ranges."""
         try:
             df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
             if df is not None and not df.empty:
                 return float((df["High"] - df["Low"]).median())
         except Exception:
             pass
+        info = PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])
+        return info["pip_value"]
 
-        fallback_spreads = {
+    def get_pair_info(self, symbol: str = XAUUSD) -> Dict:
-            "EURUSD=X": 0.00010, "GBPUSD=X": 0.00015,
+        info = PAIR_INFO.get(symbol, PAIR_INFO[XAUUSD])
-            "USDJPY=X": 0.010,   "USDCHF=X": 0.00015,
+        return {"symbol": symbol, **info}
-            "AUDUSD=X": 0.00015, "USDCAD=X": 0.00018,
-            "NZDUSD=X": 0.00020,
-        }
-        return fallback_spreads.get(symbol, 0.00020)
 
-    def get_pair_info(self, symbol: str) -> Dict:
+    # ── Private ────────────────────────────────────────────────────
-        """Return display name and pip value for a pair."""
-        display = SYMBOL_DISPLAY.get(symbol, symbol)
-        is_jpy = "JPY" in symbol
-        return {
-            "symbol": symbol,
-            "display": display,
-            "pip_value": 0.01 if is_jpy else 0.0001,
-            "is_jpy_pair": is_jpy,
-        }
-
-    # ── Private Methods ────────────────────────────────────────────
 
     def _throttle(self, credits: int = 1) -> None:
-        """
+        """Pro tier: 55 credits / minute. Block until budget allows."""
-        Block until `credits` API credits are available.
-
-        Twelve Data free tier: 8 credits/minute. Each symbol in any request
-        (single or batch) consumes 1 credit. This method enforces the limit
-        regardless of whether requests are batched or individual.
-        """
         while True:
             now = time.time()
-            # Evict credits older than the rate window
             while self._request_times and now - self._request_times[0] >= self.RATE_WINDOW:
                 self._request_times.popleft()
 
@@ -218,276 +203,76 @@ class ForexDataProvider:
                     self._request_times.append(time.time())
                 return
 
-            # Wait until the oldest credit expires
             wait = (self._request_times[0] + self.RATE_WINDOW) - now + 0.2
             logger.info(
                 f"Rate limit: {len(self._request_times)}/{self.RATE_LIMIT} credits used "
-                f"— waiting {wait:.1f}s for next window..."
+                f"— waiting {wait:.1f}s..."
             )
             time.sleep(wait)
 
-    def _batch_fetch_timeframe(
-        self, symbols: List[str], timeframe: Timeframe, lookback_days: int
-    ) -> None:
-        """
-        Single API call fetching all symbols for one timeframe.
-        Twelve Data returns a dict keyed by symbol when multiple are requested.
-        Parsed DataFrames are validated and stored in the cache.
-        """
-        if not self._api_key:
-            return
-
-        td_symbols = [self._to_td_symbol(s) for s in symbols]
-        interval = self.TD_INTERVALS[timeframe]
-        outputsize = min(
-            int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20, 5000
-        )
-        outputsize = max(outputsize, 300)
-
-        # Each symbol in a batch request costs 1 credit
-        self._throttle(credits=len(symbols))
-
-        resp = self._session.get(
-            f"{self.TD_BASE_URL}/time_series",
-            params={
-                "symbol": ",".join(td_symbols),
-                "interval": interval,
-                "outputsize": outputsize,
-                "apikey": self._api_key,
-                "timezone": "UTC",
-                "format": "json",
-            },
-            timeout=30,
-        )
-        resp.raise_for_status()
-        data = resp.json()
-
-        # If the whole batch request returned a top-level error, raise it
-        if isinstance(data, dict) and data.get("status") == "error":
-            raise DataIntegrityError(
-                f"Batch API error ({timeframe.value}): {data.get('message', 'unknown')}"
-            )
-
-        # Batch response: {"EUR/USD": {...}, "GBP/USD": {...}, ...}
-        now = time.time()
-        for symbol, td_sym in zip(symbols, td_symbols):
-            try:
-                sym_data = data.get(td_sym, {})
-                if sym_data.get("status") == "error":
-                    logger.warning(
-                        f"Batch fetch error for {td_sym}: {sym_data.get('message')}"
-                    )
-                    continue
-
-                values = sym_data.get("values")
-                if not values:
-                    logger.warning(f"No values in batch response for {td_sym}")
-                    continue
-
-                records = [
-                    {
-                        "Datetime": pd.Timestamp(v["datetime"], tz="UTC"),
-                        "Open":   float(v["open"]),
-                        "High":   float(v["high"]),
-                        "Low":    float(v["low"]),
-                        "Close":  float(v["close"]),
-                        "Volume": float(v.get("volume") or 0),
-                    }
-                    for v in reversed(values)
-                ]
-                df = pd.DataFrame(records).set_index("Datetime")
-                df.index.name = "Datetime"
-                df = self._validate_and_clean(df, symbol, timeframe)
-
-                cache_key = f"{symbol}_{timeframe.value}"
-                self._cache[cache_key] = df
-                self._cache_timestamps[cache_key] = now
-                logger.debug(f"Batch cached {len(df)} bars for {td_sym} @ {interval}")
-
-            except Exception as e:
-                logger.warning(f"Failed to parse batch data for {td_sym}: {e}")
-
     def _fetch_twelve_data(
-        self, symbol: str, timeframe: Timeframe, lookback_days: int
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start_date: Optional[str] = None,
+        end_date: Optional[str] = None,
     ) -> pd.DataFrame:
-        """
+        if self._td is None:
-        Fetch OHLCV from Twelve Data REST API.
+            raise DataIntegrityError(
-        Falls back to synthetic GBM data if API is unavailable or key is missing.
+                "No Twelve Data API key. Set TWELVE_DATA_API_KEY env var "
-        """
+                "or config.twelve_data_api_key."
-        if not self._api_key:
+            )
-            logger.warning("No Twelve Data API key — using synthetic data. "
-                           "Set TWELVE_DATA_API_KEY env var or config.twelve_data_api_key.")
-            return self._generate_synthetic_data(symbol, timeframe, lookback_days)
 
         td_symbol = self._to_td_symbol(symbol)
         interval = self.TD_INTERVALS[timeframe]
-        outputsize = min(
-            int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20,
-            5000,
-        )
-        outputsize = max(outputsize, 300)
 
         self._throttle(credits=1)
 
-        try:
+        kwargs = {
-            resp = self._session.get(
-                f"{self.TD_BASE_URL}/time_series",
-                params={
             "symbol": td_symbol,
             "interval": interval,
-                    "outputsize": outputsize,
+            "outputsize": min(outputsize, 5000),
-                    "apikey": self._api_key,
             "timezone": "UTC",
-                    "format": "json",
+        }
-                },
+        if start_date:
-                timeout=15,
+            kwargs["start_date"] = start_date
-            )
+        if end_date:
-            resp.raise_for_status()
+            kwargs["end_date"] = end_date
-            data = resp.json()
 
-            if data.get("status") == "error":
+        try:
+            df = self._td.time_series(**kwargs).as_pandas()
+        except Exception as e:
             raise DataIntegrityError(
-                    f"Twelve Data API error for {td_symbol}: {data.get('message', 'unknown')}"
+                f"Twelve Data fetch failed for {td_symbol}: {e}"
-                )
+            ) from e
 
-            values = data.get("values")
+        if df is None or df.empty:
-            if not values:
             raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
 
-            # Values arrive newest-first; reverse to chronological order
+        # SDK returns lowercase columns; engine expects capitalised.
-            records = [
+        df = df.rename(columns={
-                {
+            "open": "Open", "high": "High", "low": "Low",
-                    "Datetime": pd.Timestamp(v["datetime"], tz="UTC"),
+            "close": "Close", "volume": "Volume",
-                    "Open":   float(v["open"]),
+        })
-                    "High":   float(v["high"]),
+        for col in ("Open", "High", "Low", "Close", "Volume"):
-                    "Low":    float(v["low"]),
+            if col in df.columns:
-                    "Close":  float(v["close"]),
+                df[col] = pd.to_numeric(df[col], errors="coerce")
-                    "Volume": float(v.get("volume") or 0),
+        if "Volume" not in df.columns:
-                }
+            df["Volume"] = 0.0
-                for v in reversed(values)
-            ]
 
-            df = pd.DataFrame(records).set_index("Datetime")
+        # SDK returns newest-first; engine + indicators want oldest-first.
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
         df.index.name = "Datetime"
+
         logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
         return df
 
-        except DataIntegrityError:
-            raise
-        except Exception as e:
-            logger.info(f"Twelve Data fetch failed for {symbol} ({e}), using synthetic data.")
-            return self._generate_synthetic_data(symbol, timeframe, lookback_days)
-
-    def _generate_synthetic_data(
-        self, symbol: str, timeframe: Timeframe, lookback_days: int
-    ) -> pd.DataFrame:
-        """
-        Generate realistic synthetic Forex OHLCV using GBM with mean-reversion.
-        Used as a fallback when live data is unavailable.
-        """
-        pair_params = {
-            "EURUSD=X": {"price": 1.1350, "vol": 0.07, "drift": -0.001},
-            "GBPUSD=X": {"price": 1.3200, "vol": 0.08, "drift":  0.002},
-            "USDJPY=X": {"price": 144.50, "vol": 0.09, "drift":  0.003},
-            "USDCHF=X": {"price": 0.8850, "vol": 0.07, "drift": -0.001},
-            "AUDUSD=X": {"price": 0.6550, "vol": 0.10, "drift": -0.002},
-            "USDCAD=X": {"price": 1.3650, "vol": 0.07, "drift":  0.001},
-            "NZDUSD=X": {"price": 0.6100, "vol": 0.10, "drift": -0.001},
-            "EURGBP=X": {"price": 0.8590, "vol": 0.06, "drift":  0.000},
-            "EURJPY=X": {"price": 163.90, "vol": 0.09, "drift":  0.002},
-            "GBPJPY=X": {"price": 190.70, "vol": 0.11, "drift":  0.003},
-            "AUDJPY=X": {"price": 94.60,  "vol": 0.11, "drift":  0.001},
-            "EURCHF=X": {"price": 0.9380, "vol": 0.05, "drift":  0.000},
-            "CADJPY=X": {"price": 105.80, "vol": 0.09, "drift":  0.001},
-        }
-
-        params = pair_params.get(symbol, {"price": 1.0000, "vol": 0.08, "drift": 0.000})
-        base_price = params["price"]
-        annual_vol = params["vol"]
-        annual_drift = params["drift"]
-
-        tf_minutes = {
-            Timeframe.M1: 1, Timeframe.M5: 5, Timeframe.M15: 15,
-            Timeframe.M30: 30, Timeframe.H1: 60, Timeframe.H4: 240,
-            Timeframe.D1: 1440, Timeframe.W1: 10080,
-        }
-        bar_minutes = tf_minutes.get(timeframe, 60)
-        bars_per_year = (365.25 * 24 * 60) / bar_minutes
-        dt = 1.0 / bars_per_year
-
-        n_bars = min(int(lookback_days * 24 * 60 / bar_minutes), 2000)
-        n_bars = max(n_bars, 300)
-
-        seed_base = int(hashlib.md5(symbol.encode()).hexdigest()[:8], 16)
-        time_component = int(time.time()) // 3600
-        np.random.seed((seed_base + time_component) % (2**31))
-
-        bar_vol = annual_vol * np.sqrt(dt)
-        bar_drift = annual_drift * dt
-
-        regime_len = np.random.randint(20, 80)
-        trend_strength = np.random.normal(0, 0.3)
-
-        price = base_price
-        prices = [price]
-
-        for i in range(1, n_bars):
-            if i % regime_len == 0:
-                regime_len = np.random.randint(20, 80)
-                trend_strength = np.random.normal(0, 0.3)
-
-            mr_force = -0.001 * (price - base_price) / base_price
-            ret = bar_drift + trend_strength * dt + mr_force + bar_vol * np.random.randn()
-
-            if np.random.random() < 0.02:
-                ret *= np.random.uniform(2.0, 4.0)
-
-            price *= (1 + ret)
-            prices.append(price)
-
-        prices = np.array(prices)
-        opens = np.roll(prices, 1)
-        opens[0] = prices[0]
-
-        typical_range = bar_vol * base_price * 1.5
-        highs = np.maximum(prices, opens) + np.abs(np.random.normal(0, typical_range * 0.6, n_bars))
-        lows  = np.minimum(prices, opens) - np.abs(np.random.normal(0, typical_range * 0.6, n_bars))
-        highs = np.maximum(highs, np.maximum(opens, prices))
-        lows  = np.minimum(lows,  np.minimum(opens, prices))
-
-        volume = np.random.lognormal(mean=10, sigma=0.5, size=n_bars).astype(int)
-
-        freq_map = {
-            Timeframe.M1: "1min",  Timeframe.M5: "5min",
-            Timeframe.M15: "15min", Timeframe.M30: "30min",
-            Timeframe.H1: "1h",    Timeframe.H4: "4h",
-            Timeframe.D1: "D",     Timeframe.W1: "W",
-        }
-        freq = freq_map.get(timeframe, "1h")
-        end_time = pd.Timestamp.now(tz="UTC").floor(f"{bar_minutes}min")
-        index = pd.date_range(end=end_time, periods=n_bars, freq=freq, tz="UTC")
-
-        df = pd.DataFrame({
-            "Open": opens, "High": highs, "Low": lows,
-            "Close": prices, "Volume": volume,
-        }, index=index)
-        df.index.name = "Datetime"
-
-        logger.info(
-            f"Generated {n_bars} synthetic {timeframe.value} bars for "
-            f"{SYMBOL_DISPLAY.get(symbol, symbol)} @ {prices[-1]:.5f}"
-        )
-        return df
-
     def _validate_and_clean(
         self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
     ) -> pd.DataFrame:
-        """
-        Validate and clean OHLCV data.
-
-        Steps: drop all-NaN rows → ffill small gaps → remove future timestamps
-        → fix OHLC inconsistencies → remove >10-sigma outliers → sort & deduplicate.
-        """
         if df is None or df.empty:
             raise DataIntegrityError(f"Empty dataset for {symbol}")
 
@@ -497,6 +282,9 @@ class ForexDataProvider:
         df = df.ffill(limit=3)
         df = df.dropna(subset=["Close"])
 
+        df = df.sort_index()
+        df = df[~df.index.duplicated(keep="last")]
+
         now = pd.Timestamp.utcnow()
         future_mask = df.index > now + pd.Timedelta(minutes=5)
         if future_mask.any():
@@ -527,7 +315,6 @@ class ForexDataProvider:
 
         min_required = max(
             self.config.technical.ema_trend,
-            self.config.technical.ichimoku_senkou_b,
             200,
         ) + 10
 
@@ -537,9 +324,6 @@ class ForexDataProvider:
                 f"(need {min_required}). Some indicators will be partial."
             )
 
-        df = df.sort_index()
-        df = df[~df.index.duplicated(keep="last")]
-
         cleaned = original_len - len(df)
         if cleaned > 0:
             logger.info(f"{symbol}: Cleaned {cleaned}/{original_len} bars")

engine.py

@@ -83,22 +83,30 @@ class TechnicalEngine:
         )
 
         # ── Compute all indicators ─────────────────────────────────
-        self._compute_ema_signals(df, result)
+        disabled = set(self.config.disabled_indicators or [])
-        self._compute_rsi(df, result)
+
+        if "ema_trend"   not in disabled: self._compute_ema_signals(df, result)
+        if "rsi"         not in disabled: self._compute_rsi(df, result)
+        if "bollinger"   not in disabled: self._compute_bollinger(df, result)
+        if "adx"         not in disabled: self._compute_adx(df, result)
+        if "atr"         not in disabled: self._compute_atr(df, result)
+
         self._compute_macd(df, result)
-        self._compute_bollinger(df, result)
+
-        self._compute_stochastic(df, result)
+        if "roc"         not in disabled: self._compute_roc(df, result)
-        self._compute_adx(df, result)
+        if "keltner"     not in disabled: self._compute_keltner(df, result)
-        self._compute_atr(df, result)
+
-        self._compute_ichimoku(df, result)
+        if "divergence"  not in disabled: self._detect_divergences(df, result)
-        self._detect_divergences(df, result)
+        if "structure"   not in disabled: self._compute_structure_sr(df, result)
-        self._detect_candlestick_patterns(df, result)
+        if "candlestick" not in disabled: self._detect_candlestick_patterns(df, result)
-        self._compute_support_resistance(df, result)
-        self._compute_volume_analysis(df, result)
 
         # ── Classify market regime ─────────────────────────────────
         self._classify_regime(df, result)
 
+        # ── Regime-aware RSI reinterpretation ──────────────────────
+        if self.config.rsi_mode == "regime_aware" and "rsi" not in disabled:
+            self._reinterpret_rsi_for_regime(result)
+
         # ── Compute composite signal ───────────────────────────────
         self._compute_composite(result)
 
@@ -109,7 +117,87 @@ class TechnicalEngine:
     # ══════════════════════════════════════════════════════════════
 
     def _compute_ema_signals(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """EMA trend alignment analysis."""
+        """EMA trend alignment — supports 4-EMA and 2-EMA modes."""
+        if self.config.ema_mode == "2ema":
+            self._compute_ema_2(df, result)
+        else:
+            self._compute_ema_4(df, result)
+
+    def _compute_ema_2(self, df: pd.DataFrame, result: TechnicalAnalysis):
+        """EMA 50/200 trend alignment — cleaner, less noise on H1."""
+        c = self.config
+        close = df["Close"]
+
+        ema50 = close.ewm(span=c.ema_slow, adjust=False).mean()
+        ema200 = close.ewm(span=c.ema_trend, adjust=False).mean() if len(close) >= c.ema_trend else None
+        ema21 = close.ewm(span=c.ema_medium, adjust=False).mean()
+
+        latest = close.iloc[-1]
+        s = ema50.iloc[-1]
+        m = ema21.iloc[-1]
+        t = ema200.iloc[-1] if ema200 is not None and len(ema200) >= c.ema_trend else None
+
+        if t is None:
+            result.indicators["ema_trend"] = IndicatorSignal(
+                name="EMA Trend", value=0, signal=SignalStrength.NEUTRAL,
+                confidence=30, description="EMA200 data insufficient", weight=1.0,
+            )
+            result.key_levels["ema_medium"] = float(m)
+            result.key_levels["ema_slow"] = float(s)
+            return
+
+        # 3-level alignment: price vs EMA50, EMA50 vs EMA200
+        if latest > s > t:
+            signal = SignalStrength.STRONG_BUY
+            conf = 85
+            desc = "Full bullish: price > EMA50 > EMA200"
+        elif latest > s and s < t:
+            signal = SignalStrength.BUY
+            conf = 60
+            desc = "Emerging uptrend: price > EMA50, EMA50 < EMA200"
+        elif latest < s < t:
+            signal = SignalStrength.STRONG_SELL
+            conf = 85
+            desc = "Full bearish: price < EMA50 < EMA200"
+        elif latest < s and s > t:
+            signal = SignalStrength.SELL
+            conf = 60
+            desc = "Emerging downtrend: price < EMA50, EMA50 > EMA200"
+        else:
+            signal = SignalStrength.NEUTRAL
+            conf = 40
+            desc = "EMA alignment mixed"
+
+        # EMA50/200 crossover detection (recent 10 bars)
+        if len(ema50) >= 10 and ema200 is not None:
+            for lookback in range(2, min(11, len(ema50))):
+                prev_s = ema50.iloc[-lookback]
+                prev_t = ema200.iloc[-lookback]
+                if prev_s <= prev_t and s > t:
+                    desc += " | Golden cross (50/200)"
+                    conf = min(conf + 10, 95)
+                    break
+                elif prev_s >= prev_t and s < t:
+                    desc += " | Death cross (50/200)"
+                    conf = min(conf + 10, 95)
+                    break
+
+        normalized = 1.0 if signal == SignalStrength.STRONG_BUY else (
+            0.5 if signal == SignalStrength.BUY else (
+                -1.0 if signal == SignalStrength.STRONG_SELL else (
+                    -0.5 if signal == SignalStrength.SELL else 0.0)))
+
+        result.indicators["ema_trend"] = IndicatorSignal(
+            name="EMA Trend", value=normalized,
+            signal=signal, confidence=conf,
+            description=desc, weight=1.0,
+        )
+        result.key_levels["ema_trend"] = float(t)
+        result.key_levels["ema_medium"] = float(m)
+        result.key_levels["ema_slow"] = float(s)
+
+    def _compute_ema_4(self, df: pd.DataFrame, result: TechnicalAnalysis):
+        """Original 4-EMA (9/21/50/200) trend alignment analysis."""
         c = self.config
         close = df["Close"]
 
@@ -122,8 +210,6 @@ class TechnicalEngine:
         f, m, s = ema_fast.iloc[-1], ema_medium.iloc[-1], ema_slow.iloc[-1]
         t = ema_trend.iloc[-1] if ema_trend is not None and len(ema_trend) >= c.ema_trend else None
 
-        # Bullish: price > fast > medium > slow > trend
-        # Bearish: price < fast < medium < slow < trend
         alignment_score = 0
         checks = [(latest, f), (f, m), (m, s)]
         if t is not None:
@@ -154,7 +240,6 @@ class TechnicalEngine:
             signal = SignalStrength.NEUTRAL
             conf = 40
 
-        # Check for EMA crossover (recent)
         crossover_desc = ""
         if len(ema_fast) >= 3:
             prev_f, prev_m = ema_fast.iloc[-2], ema_medium.iloc[-2]
@@ -165,12 +250,32 @@ class TechnicalEngine:
                 crossover_desc = " | Death cross (fast/medium)"
                 conf = min(conf + 10, 95)
 
-        desc = f"EMA alignment: {alignment_score}/{max_checks}{crossover_desc}"
+        # EMA ribbon compression/expansion detection
+        ribbon_desc = ""
+        atr_val = result.key_levels.get("atr", 0)
+        ema_values = [f, m, s]
+        if atr_val > 0:
+            ribbon_width = (max(ema_values) - min(ema_values)) / atr_val
+            if ribbon_width < 1.0:
+                ribbon_desc = " | Ribbon COMPRESSED (trend change imminent)"
+            if len(ema_fast) >= 6:
+                prev_vals = [ema_fast.iloc[-5], ema_medium.iloc[-5], ema_slow.iloc[-5]]
+                prev_width = (max(prev_vals) - min(prev_vals)) / atr_val if atr_val > 0 else 0
+                if prev_width < 1.0 and ribbon_width >= 1.0:
+                    ribbon_desc = " | Ribbon EXPANDING (new trend starting)"
+                    conf = min(conf + 10, 95)
+
+        desc = f"EMA alignment: {alignment_score}/{max_checks}{crossover_desc}{ribbon_desc}"
         result.indicators["ema_trend"] = IndicatorSignal(
             name="EMA Trend", value=normalized,
             signal=signal, confidence=conf,
-            description=desc, weight=1.2,
+            description=desc, weight=1.0,
         )
+        if t is not None:
+            result.key_levels["ema_trend"] = float(t)
+        result.key_levels["ema_fast"] = float(f)
+        result.key_levels["ema_medium"] = float(m)
+        result.key_levels["ema_slow"] = float(s)
 
     def _compute_rsi(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """RSI with trend-adjusted interpretation."""
@@ -181,10 +286,10 @@ class TechnicalEngine:
         gain = delta.where(delta > 0, 0.0)
         loss = (-delta).where(delta < 0, 0.0)
 
-        avg_gain = gain.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period).mean()
+        avg_gain = gain.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period, adjust=False).mean()
-        avg_loss = loss.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period).mean()
+        avg_loss = loss.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period, adjust=False).mean()
 
-        rs = avg_gain / avg_loss.replace(0, np.nan)
+        rs = avg_gain / avg_loss.replace(0, 1e-10)
         rsi = 100 - (100 / (1 + rs))
         rsi_val = rsi.iloc[-1] if not rsi.empty else 50.0
 
@@ -215,14 +320,14 @@ class TechnicalEngine:
 
         result.indicators["rsi"] = IndicatorSignal(
             name="RSI", value=rsi_val, signal=signal,
-            confidence=conf, description=desc, weight=1.0,
+            confidence=conf, description=desc, weight=1.5,
         )
 
         # Attach RSI series for divergence detection (dynamic attribute, filtered by _prefix)
         result._rsi_series = rsi  # type: ignore[attr-defined]
 
     def _compute_macd(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """MACD with histogram momentum analysis."""
+        """MACD computation — always runs for divergence; optionally a signal pillar."""
         c = self.config
         close = df["Close"]
 
@@ -232,71 +337,72 @@ class TechnicalEngine:
         signal_line = macd_line.ewm(span=c.macd_signal, adjust=False).mean()
         histogram = macd_line - signal_line
 
-        macd_val = macd_line.iloc[-1]
+        result._macd_histogram = histogram  # type: ignore[attr-defined]
-        signal_val = signal_line.iloc[-1]
-        hist_val = histogram.iloc[-1]
 
-        # Check for crossover
+        if not c.macd_as_pillar:
-        crossover = False
+            return
-        crossover_type = ""
-        if len(macd_line) >= 2:
-            prev_macd = macd_line.iloc[-2]
-            prev_signal = signal_line.iloc[-2]
-            if prev_macd <= prev_signal and macd_val > signal_val:
-                crossover = True
-                crossover_type = "bullish"
-            elif prev_macd >= prev_signal and macd_val < signal_val:
-                crossover = True
-                crossover_type = "bearish"
 
-        # Histogram momentum (is it expanding or contracting?)
+        if len(histogram) < 3:
-        hist_momentum = ""
+            return
-        if len(histogram) >= 3:
-            h1, h2, h3 = histogram.iloc[-3], histogram.iloc[-2], histogram.iloc[-1]
-            if h3 > h2 > h1 and h3 > 0:
-                hist_momentum = "accelerating bullish"
-            elif h3 < h2 < h1 and h3 < 0:
-                hist_momentum = "accelerating bearish"
-            elif abs(h3) < abs(h2) and h3 * h2 > 0:
-                hist_momentum = "decelerating"
 
-        # Signal
+        hist_now = histogram.iloc[-1]
-        if crossover and crossover_type == "bullish":
+        hist_prev = histogram.iloc[-2]
-            signal = SignalStrength.STRONG_BUY
+        hist_prev2 = histogram.iloc[-3]
-            conf = 75
+
-            desc = f"MACD bullish crossover | Hist: {hist_val:.6f}"
+        if pd.isna(hist_now) or pd.isna(hist_prev):
-        elif crossover and crossover_type == "bearish":
+            return
-            signal = SignalStrength.STRONG_SELL
+
-            conf = 75
+        # Histogram zero-line crossover
-            desc = f"MACD bearish crossover | Hist: {hist_val:.6f}"
+        crossed_up = hist_prev <= 0 < hist_now
-        elif macd_val > signal_val and hist_val > 0:
+        crossed_down = hist_prev >= 0 > hist_now
-            signal = SignalStrength.BUY
+        # Momentum acceleration
-            conf = 55
+        accel_up = hist_now > 0 and hist_now > hist_prev > 0
-            desc = f"MACD above signal, positive histogram | {hist_momentum}"
+        accel_down = hist_now < 0 and hist_now < hist_prev < 0
-        elif macd_val < signal_val and hist_val < 0:
+        # Momentum fading
-            signal = SignalStrength.SELL
+        fading_up = hist_now > 0 and hist_now < hist_prev
-            conf = 55
+        fading_down = hist_now < 0 and hist_now > hist_prev
-            desc = f"MACD below signal, negative histogram | {hist_momentum}"
+
+        # Normalize histogram relative to recent range for confidence
+        hist_abs = histogram.abs()
+        if len(hist_abs) >= 20:
+            hist_pct = (abs(hist_now) / hist_abs.iloc[-20:].mean()) if hist_abs.iloc[-20:].mean() > 0 else 1.0
         else:
+            hist_pct = 1.0
+
+        if crossed_up or accel_up:
+            signal = SignalStrength.STRONG_BUY if accel_up else SignalStrength.BUY
+            conf = min(80, 55 + hist_pct * 10)
+            desc = "MACD histogram" + (" accelerating ↑" if accel_up else " crossed above zero")
+        elif crossed_down or accel_down:
+            signal = SignalStrength.STRONG_SELL if accel_down else SignalStrength.SELL
+            conf = min(80, 55 + hist_pct * 10)
+            desc = "MACD histogram" + (" accelerating ↓" if accel_down else " crossed below zero")
+        elif fading_up:
             signal = SignalStrength.NEUTRAL
             conf = 35
-            desc = f"MACD mixed | Hist: {hist_val:.6f}"
+            desc = f"MACD histogram positive but fading"
+        elif fading_down:
+            signal = SignalStrength.NEUTRAL
+            conf = 35
+            desc = f"MACD histogram negative but fading"
+        else:
+            signal = SignalStrength.NEUTRAL
+            conf = 30
+            desc = f"MACD histogram flat"
 
         result.indicators["macd"] = IndicatorSignal(
-            name="MACD", value=hist_val, signal=signal,
+            name="MACD", value=float(hist_now),
-            confidence=conf, description=desc, weight=1.1,
+            signal=signal, confidence=conf,
+            description=desc, weight=1.5,
         )
 
-        # Attach histogram for divergence detection
-        result._macd_histogram = histogram  # type: ignore[attr-defined]
-
     def _compute_bollinger(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """Bollinger Bands position and squeeze detection."""
         c = self.config
         close = df["Close"]
 
         sma = close.rolling(c.bb_period).mean()
-        std = close.rolling(c.bb_period).std()
+        std = close.rolling(c.bb_period).std(ddof=0)
         upper = sma + c.bb_std * std
         lower = sma - c.bb_std * std
 
@@ -351,74 +457,13 @@ class TechnicalEngine:
 
         result.indicators["bollinger"] = IndicatorSignal(
             name="Bollinger Bands", value=bb_position, signal=signal,
-            confidence=conf, description=desc, weight=0.9,
+            confidence=conf, description=desc, weight=1.5,
         )
 
         result.key_levels["bb_upper"] = upper_val
         result.key_levels["bb_lower"] = lower_val
         result.key_levels["bb_mid"] = sma_val
 
-    def _compute_stochastic(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Stochastic oscillator with crossover signals."""
-        c = self.config
-        high = df["High"]
-        low = df["Low"]
-        close = df["Close"]
-
-        lowest_low = low.rolling(c.stoch_k).min()
-        highest_high = high.rolling(c.stoch_k).max()
-
-        denom = highest_high - lowest_low
-        denom = denom.replace(0, np.nan)
-
-        k_line = 100 * (close - lowest_low) / denom
-        d_line = k_line.rolling(c.stoch_d).mean()
-
-        k_val = k_line.iloc[-1] if not k_line.empty else 50.0
-        d_val = d_line.iloc[-1] if not d_line.empty else 50.0
-
-        if pd.isna(k_val):
-            k_val = 50.0
-        if pd.isna(d_val):
-            d_val = 50.0
-
-        # Crossover detection
-        crossover = ""
-        if len(k_line) >= 2 and len(d_line) >= 2:
-            prev_k = k_line.iloc[-2]
-            prev_d = d_line.iloc[-2]
-            if not pd.isna(prev_k) and not pd.isna(prev_d):
-                if prev_k <= prev_d and k_val > d_val:
-                    crossover = " | Bullish %K/%D cross"
-                elif prev_k >= prev_d and k_val < d_val:
-                    crossover = " | Bearish %K/%D cross"
-
-        if k_val >= c.stoch_overbought and d_val >= c.stoch_overbought:
-            signal = SignalStrength.SELL
-            conf = 60
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f} — Overbought{crossover}"
-        elif k_val <= c.stoch_oversold and d_val <= c.stoch_oversold:
-            signal = SignalStrength.BUY
-            conf = 60
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f} — Oversold{crossover}"
-        elif crossover and "Bullish" in crossover:
-            signal = SignalStrength.BUY
-            conf = 55
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f}{crossover}"
-        elif crossover and "Bearish" in crossover:
-            signal = SignalStrength.SELL
-            conf = 55
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f}{crossover}"
-        else:
-            signal = SignalStrength.NEUTRAL
-            conf = 30
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f} — Neutral"
-
-        result.indicators["stochastic"] = IndicatorSignal(
-            name="Stochastic", value=k_val, signal=signal,
-            confidence=conf, description=desc, weight=0.8,
-        )
-
     def _compute_adx(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """ADX trend strength measurement."""
         c = self.config
@@ -439,12 +484,12 @@ class TechnicalEngine:
         plus_dm = plus_dm.where((plus_dm > minus_dm) & (plus_dm > 0), 0.0)
         minus_dm = minus_dm.where((minus_dm > plus_dm) & (minus_dm > 0), 0.0)
 
-        atr = tr.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean()
+        atr = tr.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean()
-        plus_di = 100 * (plus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean() / atr)
+        plus_di = 100 * (plus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean() / atr)
-        minus_di = 100 * (minus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean() / atr)
+        minus_di = 100 * (minus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean() / atr)
 
         dx = 100 * (plus_di - minus_di).abs() / (plus_di + minus_di).replace(0, np.nan)
-        adx = dx.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean()
+        adx = dx.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean()
 
         adx_val = adx.iloc[-1] if not adx.empty and not pd.isna(adx.iloc[-1]) else 0.0
         plus_di_last = plus_di.iloc[-1] if not plus_di.empty else np.nan
@@ -472,12 +517,33 @@ class TechnicalEngine:
             signal = SignalStrength.SELL if adx_val >= c.adx_strong_trend else SignalStrength.NEUTRAL
             direction = "bearish"
 
-        desc = f"ADX {adx_val:.1f} ({trend_strength} {direction}) | +DI={plus_di_val:.1f} -DI={minus_di_val:.1f}"
+        # ADX slope — trend acceleration detection
+        slope_lookback = self.config.adx_slope_lookback
+        adx_slope = 0.0
+        if len(adx) > slope_lookback:
+            prev_adx = adx.iloc[-slope_lookback - 1]
+            if not pd.isna(prev_adx):
+                adx_slope = adx_val - prev_adx
+
+        slope_desc = ""
+        emerging_threshold = self.config.adx_slope_emerging_threshold
+        if adx_val < self.config.adx_strong_trend and adx_slope > emerging_threshold:
+            slope_desc = " | TREND EMERGING (ADX rising fast)"
+            conf = min(conf + 15, 85)
+        elif adx_val >= self.config.adx_strong_trend and adx_slope > 0:
+            slope_desc = f" | Trend strengthening (slope +{adx_slope:.1f})"
+            conf = min(conf + 5, 85)
+        elif adx_val >= self.config.adx_strong_trend and adx_slope < -emerging_threshold:
+            slope_desc = f" | Trend weakening (slope {adx_slope:.1f})"
+            conf = max(conf - 10, 30)
+
+        desc = f"ADX {adx_val:.1f} ({trend_strength} {direction}) | +DI={plus_di_val:.1f} -DI={minus_di_val:.1f}{slope_desc}"
 
         result.indicators["adx"] = IndicatorSignal(
             name="ADX", value=adx_val, signal=signal,
-            confidence=conf, description=desc, weight=1.0,
+            confidence=conf, description=desc, weight=0.5,
         )
+        result._adx_slope = adx_slope  # type: ignore[attr-defined]
 
     def _compute_atr(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """ATR volatility measurement and regime classification."""
@@ -489,7 +555,7 @@ class TechnicalEngine:
         tr3 = (low - close.shift()).abs()
         tr = pd.concat([tr1, tr2, tr3], axis=1).max(axis=1)
 
-        atr = tr.ewm(alpha=1/c.atr_period, min_periods=c.atr_period).mean()
+        atr = tr.ewm(alpha=1/c.atr_period, min_periods=c.atr_period, adjust=False).mean()
         atr_val = atr.iloc[-1] if not atr.empty else 0.0
         if pd.isna(atr_val):
             atr_val = 0.0
@@ -523,90 +589,152 @@ class TechnicalEngine:
             weight=0.0,  # ATR is context, not a directional signal
         )
 
-    def _compute_ichimoku(self, df: pd.DataFrame, result: TechnicalAnalysis):
+    def _compute_roc(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Ichimoku Cloud analysis."""
+        """Rate of Change — momentum acceleration indicator."""
         c = self.config
-        high, low, close = df["High"], df["Low"], df["Close"]
+        close = df["Close"]
+        n = c.roc_period
 
-        if len(df) < c.ichimoku_senkou_b + 26:
+        if len(close) < n + 5:
             return
 
-        # Tenkan-sen (Conversion Line)
+        roc_val = ((close.iloc[-1] - close.iloc[-n - 1]) / close.iloc[-n - 1]) * 100
-        tenkan = (high.rolling(c.ichimoku_tenkan).max() +
+        roc_prev = ((close.iloc[-2] - close.iloc[-n - 2]) / close.iloc[-n - 2]) * 100
-                  low.rolling(c.ichimoku_tenkan).min()) / 2
 
-        # Kijun-sen (Base Line)
+        if pd.isna(roc_val) or pd.isna(roc_prev):
-        kijun = (high.rolling(c.ichimoku_kijun).max() +
-                 low.rolling(c.ichimoku_kijun).min()) / 2
-
-        latest = close.iloc[-1]
-        tenkan_val = tenkan.iloc[-1]
-        kijun_val = kijun.iloc[-1]
-
-        # Current (unshifted) cloud
-        sa = (tenkan.iloc[-1] + kijun.iloc[-1]) / 2
-        sb_raw = (high.rolling(c.ichimoku_senkou_b).max() +
-                  low.rolling(c.ichimoku_senkou_b).min()) / 2
-        sb = sb_raw.iloc[-1]
-
-        if pd.isna(sa) or pd.isna(sb) or pd.isna(tenkan_val) or pd.isna(kijun_val):
             return
 
-        cloud_top = max(sa, sb)
+        roc_rising = roc_val > roc_prev
-        cloud_bottom = min(sa, sb)
+        roc_abs = abs(roc_val)
-        cloud_is_green = sa > sb
+        threshold = c.roc_threshold
 
-        signals = []
+        if roc_val > 0 and roc_rising:
-        score = 0
+            if roc_abs > threshold * 2:
-
-        if latest > cloud_top:
-            signals.append("Price ABOVE cloud")
-            score += 2
-        elif latest < cloud_bottom:
-            signals.append("Price BELOW cloud")
-            score -= 2
-        else:
-            signals.append("Price IN cloud (indecision)")
-
-        if tenkan_val > kijun_val:
-            signals.append("TK cross bullish")
-            score += 1
-        elif tenkan_val < kijun_val:
-            signals.append("TK cross bearish")
-            score -= 1
-
-        if cloud_is_green:
-            signals.append("Green cloud")
-            score += 1
-        else:
-            signals.append("Red cloud")
-            score -= 1
-
-        if score >= 3:
                 signal = SignalStrength.STRONG_BUY
-            conf = 75
+                conf = min(80, 60 + roc_abs * 5)
-        elif score >= 1:
+                desc = f"ROC +{roc_val:.2f}% — Strong momentum acceleration"
+            else:
                 signal = SignalStrength.BUY
-            conf = 55
+                conf = min(70, 50 + roc_abs * 10)
-        elif score <= -3:
+                desc = f"ROC +{roc_val:.2f}% — Rising momentum"
+        elif roc_val < 0 and not roc_rising:
+            if roc_abs > threshold * 2:
                 signal = SignalStrength.STRONG_SELL
-            conf = 75
+                conf = min(80, 60 + roc_abs * 5)
-        elif score <= -1:
+                desc = f"ROC {roc_val:.2f}% — Strong bearish momentum"
+            else:
                 signal = SignalStrength.SELL
-            conf = 55
+                conf = min(70, 50 + roc_abs * 10)
+                desc = f"ROC {roc_val:.2f}% — Falling momentum"
+        elif roc_prev <= 0 < roc_val:
+            signal = SignalStrength.BUY
+            conf = 65
+            desc = f"ROC crossed above zero (+{roc_val:.2f}%)"
+        elif roc_prev >= 0 > roc_val:
+            signal = SignalStrength.SELL
+            conf = 65
+            desc = f"ROC crossed below zero ({roc_val:.2f}%)"
         else:
             signal = SignalStrength.NEUTRAL
-            conf = 35
+            conf = 30
+            desc = f"ROC {roc_val:+.2f}% — No clear momentum"
 
-        result.indicators["ichimoku"] = IndicatorSignal(
+        result.indicators["roc"] = IndicatorSignal(
-            name="Ichimoku", value=float(score), signal=signal,
+            name="ROC", value=roc_val, signal=signal,
-            confidence=conf,
+            confidence=conf, description=desc, weight=1.5,
-            description=f"Ichimoku [{score}/4]: {' | '.join(signals)}",
-            weight=1.1,
         )
 
-        result.key_levels["ichimoku_cloud_top"] = cloud_top
+    def _compute_keltner(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        result.key_levels["ichimoku_cloud_bottom"] = cloud_bottom
+        """Keltner Channels + Bollinger-Keltner squeeze detection (TTM Squeeze)."""
+        c = self.config
+        close = df["Close"]
+        high, low = df["High"], df["Low"]
+
+        if len(close) < max(c.kc_period, c.kc_atr_period) + 5:
+            return
+
+        kc_mid = close.ewm(span=c.kc_period, adjust=False).mean()
+
+        tr1 = high - low
+        tr2 = (high - close.shift()).abs()
+        tr3 = (low - close.shift()).abs()
+        tr = pd.concat([tr1, tr2, tr3], axis=1).max(axis=1)
+        kc_atr = tr.ewm(alpha=1/c.kc_atr_period, min_periods=c.kc_atr_period, adjust=False).mean()
+
+        kc_upper = kc_mid + c.kc_multiplier * kc_atr
+        kc_lower = kc_mid - c.kc_multiplier * kc_atr
+
+        latest = close.iloc[-1]
+        kc_upper_val = kc_upper.iloc[-1]
+        kc_lower_val = kc_lower.iloc[-1]
+        kc_mid_val = kc_mid.iloc[-1]
+
+        if pd.isna(kc_upper_val) or pd.isna(kc_lower_val):
+            return
+
+        bb_upper = result.key_levels.get("bb_upper")
+        bb_lower = result.key_levels.get("bb_lower")
+
+        is_squeeze = False
+        squeeze_released = False
+        if bb_upper is not None and bb_lower is not None:
+            is_squeeze = bb_lower > kc_lower_val and bb_upper < kc_upper_val
+
+            if len(close) >= 3:
+                bb_sma = close.rolling(c.bb_period).mean()
+                bb_std = close.rolling(c.bb_period).std(ddof=0)
+                if len(bb_sma) >= 2 and not pd.isna(bb_sma.iloc[-2]):
+                    prev_bb_upper = bb_sma.iloc[-2] + c.bb_std * bb_std.iloc[-2]
+                    prev_bb_lower = bb_sma.iloc[-2] - c.bb_std * bb_std.iloc[-2]
+                    prev_kc_upper = kc_upper.iloc[-2]
+                    prev_kc_lower = kc_lower.iloc[-2]
+                    was_squeeze = prev_bb_lower > prev_kc_lower and prev_bb_upper < prev_kc_upper
+                    squeeze_released = was_squeeze and not is_squeeze
+
+        macd_hist = getattr(result, "_macd_histogram", None)
+        macd_direction = 0
+        if macd_hist is not None and len(macd_hist) >= 1 and not pd.isna(macd_hist.iloc[-1]):
+            macd_direction = 1 if macd_hist.iloc[-1] > 0 else -1
+
+        if squeeze_released:
+            if macd_direction > 0:
+                signal = SignalStrength.STRONG_BUY
+                desc = "Keltner SQUEEZE RELEASED — bullish breakout"
+            elif macd_direction < 0:
+                signal = SignalStrength.STRONG_SELL
+                desc = "Keltner SQUEEZE RELEASED — bearish breakout"
+            else:
+                signal = SignalStrength.NEUTRAL
+                desc = "Keltner squeeze released — direction unclear"
+            conf = 80
+        elif is_squeeze:
+            signal = SignalStrength.NEUTRAL
+            conf = 60
+            desc = "Keltner SQUEEZE active — breakout imminent"
+        elif latest > kc_upper_val:
+            signal = SignalStrength.BUY
+            conf = 65
+            desc = f"Price above Keltner upper — strong uptrend continuation"
+        elif latest < kc_lower_val:
+            signal = SignalStrength.SELL
+            conf = 65
+            desc = f"Price below Keltner lower — strong downtrend continuation"
+        else:
+            signal = SignalStrength.NEUTRAL
+            conf = 30
+            desc = "Price within Keltner Channels"
+
+        result.indicators["keltner"] = IndicatorSignal(
+            name="Keltner", value=float(latest - kc_mid_val),
+            signal=signal, confidence=conf,
+            description=desc, weight=1.5,
+        )
+
+        result.key_levels["kc_upper"] = float(kc_upper_val)
+        result.key_levels["kc_lower"] = float(kc_lower_val)
+        result.key_levels["kc_mid"] = float(kc_mid_val)
+        result._kc_squeeze = is_squeeze  # type: ignore[attr-defined]
+        result._kc_squeeze_released = squeeze_released  # type: ignore[attr-defined]
 
     def _detect_divergences(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """
@@ -665,11 +793,39 @@ class TechnicalEngine:
                 name="Divergence", value=len(divergences),
                 signal=signal, confidence=70,
                 description=" | ".join(divergences),
-                weight=1.5,  # divergences are strong reversal signals
+                weight=1.5,
             )
             # Note: divergences are NOT added to pattern_signals to avoid
             # duplicate reason collection (the indicator entry covers them).
 
+    def _compute_structure_sr(self, df: pd.DataFrame, result: TechnicalAnalysis):
+        """Detect swing high/low structure levels for S/R and trade setup."""
+        lookback = min(self.config.swing_sr_lookback, len(df) - 1)
+        if lookback < 10:
+            return
+
+        price = df["Close"].iloc[-1]
+        high_window = df["High"].iloc[-lookback:]
+        low_window = df["Low"].iloc[-lookback:]
+
+        swing_highs = self._find_swing_points(high_window, mode="high", window=3)
+        swing_lows = self._find_swing_points(low_window, mode="low", window=3)
+
+        highs_above = sorted([v for _, v in swing_highs if v > price])
+        highs_below = sorted([v for _, v in swing_highs if v <= price], reverse=True)
+        lows_below = sorted([v for _, v in swing_lows if v < price], reverse=True)
+        lows_above = sorted([v for _, v in swing_lows if v >= price])
+
+        for i, val in enumerate(highs_above[:3], 1):
+            result.key_levels[f"swing_high_{i}"] = float(val)
+        for i, val in enumerate(lows_below[:3], 1):
+            result.key_levels[f"swing_low_{i}"] = float(val)
+
+        if not highs_above and highs_below:
+            result.key_levels["swing_high_1"] = float(highs_below[0])
+        if not lows_below and lows_above:
+            result.key_levels["swing_low_1"] = float(lows_above[0])
+
     def _detect_candlestick_patterns(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """Detect significant candlestick patterns on recent bars."""
         if len(df) < 5:
@@ -731,98 +887,127 @@ class TechnicalEngine:
             else:
                 signal = SignalStrength.NEUTRAL
 
+            conf = 50
+            atr_val = result.key_levels.get("atr", 0)
+            latest = c
+            proximity = atr_val * 0.5 if atr_val > 0 else 0
+
+            if proximity > 0:
+                swing_lows = sorted(
+                    [v for k, v in result.key_levels.items() if k.startswith("swing_low_")]
+                )
+                swing_highs = sorted(
+                    [v for k, v in result.key_levels.items() if k.startswith("swing_high_")]
+                )
+                near_support = any(abs(latest - sl) <= proximity for sl in swing_lows)
+                near_resistance = any(abs(latest - sh) <= proximity for sh in swing_highs)
+
+                if bullish > bearish and near_support:
+                    conf += 20
+                    patterns.append("(at support)")
+                elif bearish > bullish and near_resistance:
+                    conf += 20
+                    patterns.append("(at resistance)")
+
             result.indicators["candlestick"] = IndicatorSignal(
                 name="Candlestick", value=float(bullish - bearish),
-                signal=signal, confidence=50,
+                signal=signal, confidence=conf,
                 description=" | ".join(patterns),
-                weight=0.7,
+                weight=1.0,
             )
             result.pattern_signals.extend(patterns)
 
-    def _compute_support_resistance(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Calculate pivot points and key S/R levels."""
-        if len(df) < 2:
-            return
-
-        # Use previous bar for pivot calculation
-        prev_h = df["High"].iloc[-2]
-        prev_l = df["Low"].iloc[-2]
-        prev_c = df["Close"].iloc[-2]
-
-        pivot = (prev_h + prev_l + prev_c) / 3
-        r1 = 2 * pivot - prev_l
-        s1 = 2 * pivot - prev_h
-        r2 = pivot + (prev_h - prev_l)
-        s2 = pivot - (prev_h - prev_l)
-        r3 = prev_h + 2 * (pivot - prev_l)
-        s3 = prev_l - 2 * (prev_h - pivot)
-
-        result.key_levels.update({
-            "pivot": pivot,
-            "r1": r1, "r2": r2, "r3": r3,
-            "s1": s1, "s2": s2, "s3": s3,
-        })
-
-        # Signal based on price position relative to pivot
-        latest = df["Close"].iloc[-1]
-        if latest > r1:
-            desc = f"Price above R1 ({r1:.5f}), next target R2 ({r2:.5f})"
-        elif latest > pivot:
-            desc = f"Price above Pivot ({pivot:.5f}), testing R1 ({r1:.5f})"
-        elif latest > s1:
-            desc = f"Price below Pivot ({pivot:.5f}), support at S1 ({s1:.5f})"
-        else:
-            desc = f"Price below S1 ({s1:.5f}), support at S2 ({s2:.5f})"
-
-        result.indicators["pivot"] = IndicatorSignal(
-            name="Pivot Points", value=pivot,
-            signal=SignalStrength.BUY if latest > pivot else SignalStrength.SELL,
-            confidence=45, description=desc, weight=0.6,
-        )
-
-    def _compute_volume_analysis(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Volume analysis (limited on Forex but still useful as a filter)."""
-        c = self.config
-        vol = df.get("Volume")
-        if vol is None or vol.sum() == 0:
-            return
-
-        vol_ma = vol.rolling(c.volume_ma_period).mean()
-        current_vol = vol.iloc[-1]
-        avg_vol = vol_ma.iloc[-1]
-
-        if pd.isna(avg_vol) or avg_vol == 0:
-            return
-
-        vol_ratio = current_vol / avg_vol
-
-        if vol_ratio > 2.0:
-            desc = f"Volume {vol_ratio:.1f}x average — HIGH ACTIVITY"
-            conf = 55
-        elif vol_ratio > 1.3:
-            desc = f"Volume {vol_ratio:.1f}x average — above normal"
-            conf = 45
-        elif vol_ratio < 0.5:
-            desc = f"Volume {vol_ratio:.1f}x average — LOW (signals less reliable)"
-            conf = 25
-        else:
-            desc = f"Volume {vol_ratio:.1f}x average — normal"
-            conf = 35
-
-        result.indicators["volume"] = IndicatorSignal(
-            name="Volume", value=vol_ratio,
-            signal=SignalStrength.NEUTRAL,
-            confidence=conf, description=desc, weight=0.0,
-        )
-
     # ══════════════════════════════════════════════════════════════
     #  MARKET REGIME & COMPOSITE SCORING
     # ══════════════════════════════════════════════════════════════
 
+    def _reinterpret_rsi_for_regime(self, result: TechnicalAnalysis):
+        """Reinterpret RSI signal based on market regime — called after regime classification."""
+        rsi_ind = result.indicators.get("rsi")
+        if not rsi_ind:
+            return
+
+        rsi_val = rsi_ind.value
+        regime = result.market_regime
+
+        if regime in ("trending_up", "trending"):
+            if 38 <= rsi_val <= 55:
+                signal = SignalStrength.STRONG_BUY
+                conf = 75
+                desc = f"RSI {rsi_val:.1f} — Pullback zone in uptrend (ideal entry)"
+            elif 30 <= rsi_val < 38:
+                signal = SignalStrength.BUY
+                conf = 60
+                desc = f"RSI {rsi_val:.1f} — Deep pullback in uptrend"
+            elif 55 < rsi_val <= 65:
+                signal = SignalStrength.BUY
+                conf = 50
+                desc = f"RSI {rsi_val:.1f} — Momentum ok in uptrend"
+            elif rsi_val > 65:
+                signal = SignalStrength.NEUTRAL
+                conf = 30
+                desc = f"RSI {rsi_val:.1f} — Extended in uptrend, wait for pullback"
+            elif rsi_val < 30:
+                signal = SignalStrength.NEUTRAL
+                conf = 25
+                desc = f"RSI {rsi_val:.1f} — Oversold in uptrend, possible reversal"
+            else:
+                return
+        elif regime == "trending_down":
+            if 45 <= rsi_val <= 62:
+                signal = SignalStrength.STRONG_SELL
+                conf = 75
+                desc = f"RSI {rsi_val:.1f} — Bounce zone in downtrend (ideal short)"
+            elif 62 < rsi_val <= 70:
+                signal = SignalStrength.SELL
+                conf = 60
+                desc = f"RSI {rsi_val:.1f} — Deep bounce in downtrend"
+            elif 35 <= rsi_val < 45:
+                signal = SignalStrength.SELL
+                conf = 50
+                desc = f"RSI {rsi_val:.1f} — Momentum ok in downtrend"
+            elif rsi_val < 35:
+                signal = SignalStrength.NEUTRAL
+                conf = 30
+                desc = f"RSI {rsi_val:.1f} — Extended in downtrend, wait for bounce"
+            elif rsi_val > 70:
+                signal = SignalStrength.NEUTRAL
+                conf = 25
+                desc = f"RSI {rsi_val:.1f} — Overbought in downtrend, possible reversal"
+            else:
+                return
+        else:  # ranging
+            if rsi_val >= 70:
+                signal = SignalStrength.SELL
+                conf = 70
+                desc = f"RSI {rsi_val:.1f} — Overbought in range"
+            elif rsi_val <= 30:
+                signal = SignalStrength.BUY
+                conf = 70
+                desc = f"RSI {rsi_val:.1f} — Oversold in range"
+            elif rsi_val >= 60:
+                signal = SignalStrength.SELL
+                conf = 45
+                desc = f"RSI {rsi_val:.1f} — Upper range zone"
+            elif rsi_val <= 40:
+                signal = SignalStrength.BUY
+                conf = 45
+                desc = f"RSI {rsi_val:.1f} — Lower range zone"
+            else:
+                signal = SignalStrength.NEUTRAL
+                conf = 30
+                desc = f"RSI {rsi_val:.1f} — Mid-range, no edge"
+
+        result.indicators["rsi"] = IndicatorSignal(
+            name="RSI", value=rsi_val, signal=signal,
+            confidence=conf, description=desc, weight=rsi_ind.weight,
+        )
+
     def _classify_regime(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Classify the market regime (trending vs ranging)."""
+        """Classify the market regime (trending vs ranging vs transitioning)."""
         adx_sig = result.indicators.get("adx")
         ema_sig = result.indicators.get("ema_trend")
+        adx_slope = getattr(result, "_adx_slope", 0.0)
 
         if adx_sig and adx_sig.value >= self.config.adx_strong_trend:
             if ema_sig and ema_sig.signal in (SignalStrength.BUY, SignalStrength.STRONG_BUY):
@@ -831,6 +1016,8 @@ class TechnicalEngine:
                 result.market_regime = "trending_down"
             else:
                 result.market_regime = "trending"
+        elif adx_sig and adx_slope > self.config.adx_slope_emerging_threshold:
+            result.market_regime = "transitioning"
         else:
             result.market_regime = "ranging"
 
@@ -840,17 +1027,35 @@ class TechnicalEngine:
 
         Weighted average of all indicator signals, adjusted for:
         - Market regime (trend indicators weighted more in trends)
-        - Divergence boost (divergences override in ranging markets)
         - Confidence scaling
         """
         total_score = 0.0
         total_weight = 0.0
         total_confidence = 0.0
 
+        # Use override weights if provided, else gold-optimized defaults
+        if self.config.regime_weights_override:
+            regime_weights = self.config.regime_weights_override
+        else:
+            _trending = {
+                "ema_trend": 2.0, "rsi": 1.0, "macd": 2.0,
+                "bollinger": 0.5, "keltner": 1.5, "roc": 1.5,
+                "candlestick": 0.8, "divergence": 2.0, "adx": 0.8,
+            }
             regime_weights = {
-            "trending_up": {"ema_trend": 1.5, "adx": 1.3, "macd": 1.2, "rsi": 0.7, "bollinger": 0.6},
+                "trending_up": _trending,
-            "trending_down": {"ema_trend": 1.5, "adx": 1.3, "macd": 1.2, "rsi": 0.7, "bollinger": 0.6},
+                "trending_down": _trending,
-            "ranging": {"rsi": 1.4, "stochastic": 1.3, "bollinger": 1.4, "ema_trend": 0.6, "adx": 0.5},
+                "trending": _trending,
+                "ranging": {
+                    "ema_trend": 0.3, "rsi": 1.5, "macd": 1.5,
+                    "bollinger": 1.5, "keltner": 2.0, "roc": 0.5,
+                    "candlestick": 1.0, "divergence": 2.5, "adx": 0.5,
+                },
+                "transitioning": {
+                    "ema_trend": 1.5, "rsi": 1.0, "macd": 2.5,
+                    "bollinger": 1.0, "keltner": 2.5, "roc": 2.0,
+                    "candlestick": 1.0, "divergence": 2.0, "adx": 1.0,
+                },
             }
 
         active_regime_weights = regime_weights.get(result.market_regime, {})
@@ -863,7 +1068,7 @@ class TechnicalEngine:
             regime_mult = active_regime_weights.get(name, 1.0)
             weight *= regime_mult
 
-            signal_score = ind.signal.value  # -2 to +2
+            signal_score = ind.signal.value
             weighted_score = signal_score * weight * (ind.confidence / 100.0)
 
             total_score += weighted_score
@@ -871,16 +1076,14 @@ class TechnicalEngine:
             total_confidence += ind.confidence * weight
 
         if total_weight > 0:
-            normalized_score = (total_score / total_weight) * 50  # scale to -100..+100
+            normalized_score = (total_score / total_weight) * 50
             avg_confidence = total_confidence / total_weight
         else:
             normalized_score = 0
             avg_confidence = 0
 
-        # Clamp
         normalized_score = max(-100, min(100, normalized_score))
 
-        # Determine composite signal
         if normalized_score >= 40:
             result.composite_signal = SignalStrength.STRONG_BUY
         elif normalized_score >= 15:

main.py

@@ -1,35 +1,27 @@
 #!/usr/bin/env python3
 """
-Forex Signal Intelligence Platform
+ForexBot2 — XAU/USD Signal Intelligence
-====================================
+========================================
-Main entry point. Runs full analysis and outputs formatted signal reports.
-
 Usage:
-    python main.py                          # Analyze all major pairs
+    python main.py                              # Live signal on H1
-    python main.py --pairs EURUSD GBPUSD    # Specific pairs
+    python main.py --timeframe H4 --detail      # Detailed H4 signal
-    python main.py --timeframe H4           # Different primary TF
     python main.py --json                       # JSON output
-    python main.py --watch                  # Continuous monitoring mode
 """
 
 import argparse
 import json
 import logging
 import sys
-import time
 from datetime import datetime, timezone
-from typing import List
+from typing import Dict, List
 
-# Force UTF-8 output on Windows so box-drawing / Unicode chars render correctly
+# Force UTF-8 output on Windows so box-drawing chars render
 if hasattr(sys.stdout, "reconfigure"):
     sys.stdout.reconfigure(encoding="utf-8")
 if hasattr(sys.stderr, "reconfigure"):
     sys.stderr.reconfigure(encoding="utf-8")
 
-from config import (
+from config import PlatformConfig, Timeframe, SignalStrength, XAUUSD
-    PlatformConfig, Timeframe, SignalStrength,
-    MAJOR_PAIRS, CROSS_PAIRS,
-)
 from signal_aggregator import SignalAggregator, SignalReport
 
 
@@ -38,12 +30,10 @@ from signal_aggregator import SignalAggregator, SignalReport
 # ══════════════════════════════════════════════════════════════════
 
 class Colors:
-    """ANSI color codes for terminal output."""
     RESET = "\033[0m"
     BOLD = "\033[1m"
     DIM = "\033[2m"
     UNDERLINE = "\033[4m"
-
     RED = "\033[91m"
     GREEN = "\033[92m"
     YELLOW = "\033[93m"
@@ -53,16 +43,8 @@ class Colors:
     WHITE = "\033[97m"
     GRAY = "\033[90m"
 
-    BG_RED = "\033[41m"
-    BG_GREEN = "\033[42m"
-    BG_YELLOW = "\033[43m"
-    BG_BLUE = "\033[44m"
-    BG_MAGENTA = "\033[45m"
-    BG_CYAN = "\033[46m"
-
 
 def signal_color(signal: SignalStrength) -> str:
-    """Get color for a signal strength."""
     return {
         SignalStrength.STRONG_BUY:  Colors.GREEN + Colors.BOLD,
         SignalStrength.BUY:         Colors.GREEN,
@@ -72,8 +54,7 @@ def signal_color(signal: SignalStrength) -> str:
     }.get(signal, Colors.RESET)
 
 
-def quality_color(quality: str) -> str:
+def quality_color(q: str) -> str:
-    """Get color for signal quality grade."""
     return {
         "A+": Colors.GREEN + Colors.BOLD,
         "A":  Colors.GREEN,
@@ -81,196 +62,127 @@ def quality_color(quality: str) -> str:
         "C":  Colors.YELLOW,
         "D":  Colors.GRAY,
         "F":  Colors.DIM,
-    }.get(quality, Colors.RESET)
+    }.get(q, Colors.RESET)
 
 
-def score_bar(score: float, width: int = 30) -> str:
+def print_header(title: str = "XAU/USD SIGNAL INTELLIGENCE"):
-    """Create a visual score bar from -100 to +100."""
-    normalized = (score + 100) / 200  # 0 to 1
-    filled = int(normalized * width)
-    center = width // 2
-
-    bar = ""
-    for i in range(width):
-        if i == center:
-            bar += "│"
-        elif (score >= 0 and center < i <= filled) or \
-             (score < 0 and filled <= i < center):
-            bar += "█"
-        else:
-            bar += "░"
-
-    if score >= 0:
-        color = Colors.GREEN if score > 20 else Colors.GRAY
-    else:
-        color = Colors.RED if score < -20 else Colors.GRAY
-
-    return f"{color}{bar}{Colors.RESET}"
-
-
-def print_header():
-    """Print the platform header."""
     now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
-    width = 80
+    print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
-
+    print(f"{Colors.BOLD}{Colors.CYAN}  {title}{Colors.RESET}")
-    print(f"\n{Colors.CYAN}{'═' * width}{Colors.RESET}")
+    print(f"{Colors.GRAY}  Time: {now}{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ╔═══════════════════════════════════════════════════════╗{Colors.RESET}")
+    print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ║   FOREX SIGNAL INTELLIGENCE PLATFORM                 ║{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ║   Multi-Layer Signal Analysis Engine                  ║{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ╚═══════════════════════════════════════════════════════╝{Colors.RESET}")
-    print(f"{Colors.GRAY}  Scan Time: {now}{Colors.RESET}")
-    print(f"{Colors.CYAN}{'═' * width}{Colors.RESET}\n")
 
 
-def print_summary_table(reports: List[SignalReport]):
+# ── Live signal renderer ───────────────────────────────────────────
-    """Print a summary table of all pairs."""
-    print(f"{Colors.BOLD}{Colors.WHITE}  ┌─────────────┬───────────┬──────────────────────────────────┬────────┬────────────┐{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.WHITE}  │ Pair        │ Signal    │ Score                            │ Grade  │ Confidence │{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.WHITE}  ├─────────────┼───────────┼──────────────────────────────────┼────────┼────────────┤{Colors.RESET}")
 
-    for r in reports:
+_INDICATOR_PREFIXES = (
-        sig = r.final_signal.name.replace("_", " ")
+    "EMA Trend", "RSI", "Bollinger Bands", "Divergence",
-        sc = signal_color(r.final_signal)
+    "Candlestick", "ADX", "ATR", "News", "Regime",
-        qc = quality_color(r.signal_quality)
+)
-        bar = score_bar(r.final_score, 30)
-
-        pair = f"{r.display_name:<11}"
-        signal_str = f"{sc}{sig:<9}{Colors.RESET}"
-        grade_str = f"{qc}{r.signal_quality:<6}{Colors.RESET}"
-        conf_str = f"{r.final_confidence:5.1f}%"
-
-        # Add warning indicator
-        warn = ""
-        if r.event_risk == "danger":
-            warn = f" {Colors.RED}⚠{Colors.RESET}"
-        elif r.event_risk == "caution":
-            warn = f" {Colors.YELLOW}△{Colors.RESET}"
-
-        print(f"  │ {pair} │ {signal_str} │ {bar} │ {grade_str} │ {conf_str:>8}{warn}  │")
-
-    print(f"{Colors.BOLD}{Colors.WHITE}  └─────────────┴───────────┴──────────────────────────────────┴────────┴────────────┘{Colors.RESET}")
-
-    # Legend
-    print(f"\n{Colors.GRAY}  Legend: "
-          f"{Colors.GREEN}██ Bullish{Colors.GRAY} | "
-          f"{Colors.RED}██ Bearish{Colors.GRAY} | "
-          f"│ Center (0) | "
-          f"{Colors.RED}⚠ Event Risk{Colors.GRAY} | "
-          f"{Colors.YELLOW}△ Caution{Colors.RESET}")
 
 
-def print_detailed_report(report: SignalReport):
+def _compact_reason(reason: str) -> str:
-    """Print detailed analysis for a single pair."""
+    """Shorten a verbose reason string for the compact view."""
+    s = reason
+    if s.startswith("STRONG: "):
+        s = s[8:]
+    # Strip the leading "<Indicator>: " prefix — it's always an indicator name
+    for prefix in _INDICATOR_PREFIXES:
+        if s.startswith(prefix + ": "):
+            s = s[len(prefix) + 2:]
+            break
+    # Drop trailing detail after first ' | '
+    if " | " in s:
+        s = s.split(" | ", 1)[0]
+    return s.strip()
+
+
+def print_signal_report(report: SignalReport, detail: bool = False):
     sc = signal_color(report.final_signal)
     qc = quality_color(report.signal_quality)
     sig_name = report.final_signal.name.replace("_", " ")
 
-    print(f"\n{Colors.CYAN}{'─' * 80}{Colors.RESET}")
+    # ── 1. DECISION ────────────────────────────────────────────────
-    print(f"  {Colors.BOLD}{Colors.WHITE}{report.display_name}{Colors.RESET}"
+    mtf_tag = (f"{Colors.GREEN}MTF✓{Colors.RESET}" if report.mtf_agreement
-          f"  │  Price: {Colors.BOLD}{report.current_price:.5f}{Colors.RESET}"
+               else f"{Colors.YELLOW}MTF✗{Colors.RESET}")
-          f"  │  Regime: {Colors.CYAN}{report.market_regime}{Colors.RESET}"
+    print(f"  {sc}▶ {sig_name}{Colors.RESET}  "
-          f"  │  Vol: {report.volatility_regime}")
+          f"{qc}{report.signal_quality}{Colors.RESET} · "
+          f"{sc}{report.final_score:+.1f}{Colors.RESET} · "
+          f"{report.final_confidence:.0f}% · {mtf_tag}")
+    print(f"  {Colors.GRAY}{report.action_recommendation}{Colors.RESET}")
 
-    print(f"\n  {Colors.BOLD}SIGNAL:{Colors.RESET} {sc}{sig_name}{Colors.RESET}"
+    # ── 2. TRADE SETUP (if signal is actionable) ───────────────────
-          f"  │  Score: {sc}{report.final_score:+.1f}{Colors.RESET}"
+    if report.trade_setup:
-          f"  │  Grade: {qc}{report.signal_quality}{Colors.RESET}"
+        ts = report.trade_setup
-          f"  │  Confidence: {report.final_confidence:.0f}%")
+        dc = Colors.GREEN if ts.direction == "BUY" else Colors.RED
+        print(f"\n  {Colors.BOLD}Setup{Colors.RESET}  "
+              f"{dc}{ts.direction}{Colors.RESET} @ {Colors.BOLD}{ts.entry_price:.3f}{Colors.RESET}  "
+              f"→ SL {Colors.RED}{ts.stop_loss:.3f}{Colors.RESET} "
+              f"({ts.risk_pips:.1f}p)  "
+              f"→ TP {Colors.GREEN}{ts.take_profit_1:.3f}{Colors.RESET} "
+              f"({ts.reward_pips:.1f}p)  "
+              f"RR 1:{ts.risk_reward:.1f}")
 
-    print(f"  {Colors.BOLD}Action:{Colors.RESET} {report.action_recommendation}")
+    # ── 3. CONTEXT (price · regime · vol · MTF breakdown) ──────────
+    print(f"\n  {Colors.WHITE}{report.display_name}{Colors.RESET} "
+          f"{Colors.BOLD}{report.current_price:.3f}{Colors.RESET} · "
+          f"{Colors.CYAN}{report.market_regime}{Colors.RESET} · "
+          f"vol {report.volatility_regime}")
 
-    # Multi-timeframe breakdown
     if report.mtf_signals:
-        print(f"\n  {Colors.BOLD}Multi-Timeframe:{Colors.RESET}", end="")
+        parts = []
-        mtf_agree = f" {Colors.GREEN}✓ ALIGNED{Colors.RESET}" if report.mtf_agreement else f" {Colors.YELLOW}✗ DIVERGENT{Colors.RESET}"
-        print(mtf_agree)
         for tf, sig in report.mtf_signals.items():
             tf_sc = signal_color(sig)
-            tf_score = report.mtf_scores.get(tf, 0)
+            sc_val = report.mtf_scores.get(tf, 0)
-            print(f"    {tf:>5}: {tf_sc}{sig.name.replace('_', ' '):<12}{Colors.RESET} ({tf_score:+.1f})")
+            parts.append(f"{tf} {tf_sc}{sig.name.replace('_',' ')}{Colors.RESET} {sc_val:+.1f}")
+        print(f"  {Colors.GRAY}MTF{Colors.RESET}    " + "  ·  ".join(parts))
 
-    # Indicators detail
+    # ── 4. WHY — top 3 each side, compact ──────────────────────────
-    if report.primary_analysis:
+    bulls = [_compact_reason(r) for r in report.bullish_reasons[:3]]
-        print(f"\n  {Colors.BOLD}Indicators ({report.primary_analysis.timeframe}):{Colors.RESET}")
+    bears = [_compact_reason(r) for r in report.bearish_reasons[:3]]
+    if bulls or bears:
+        print()
+        rows = max(len(bulls), len(bears))
+        for i in range(rows):
+            left  = f"{Colors.GREEN}+{Colors.RESET} {bulls[i]}" if i < len(bulls) else ""
+            right = f"{Colors.RED}-{Colors.RESET} {bears[i]}" if i < len(bears) else ""
+            print(f"  {left:<60}  {right}")
+
+    # ── 5. WARNINGS / NEWS / EVENTS ────────────────────────────────
+    if report.warnings:
+        for w in report.warnings:
+            print(f"  {Colors.YELLOW}⚠{Colors.RESET}  {w}")
+    if report.event_description and report.event_risk != "clear":
+        print(f"  {Colors.YELLOW}📅{Colors.RESET} {report.event_description}")
+    if report.news_description and report.news_description != "Neutral news sentiment":
+        print(f"  {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
+
+    # ── 6. DETAIL (only with --detail) ─────────────────────────────
+    if detail and report.primary_analysis:
+        print(f"\n  {Colors.BOLD}Indicators ({report.primary_analysis.timeframe}){Colors.RESET}")
         for name, ind in report.primary_analysis.indicators.items():
             if name.startswith("_") or ind.weight == 0:
                 continue
             isc = signal_color(ind.signal)
-            ind_name = f"{ind.name:<18}"
+            print(f"    {ind.name:<16} {isc}{ind.signal.name.replace('_',' '):<11}{Colors.RESET} "
-            print(f"    {ind_name} {isc}{ind.signal.name.replace('_', ' '):<12}{Colors.RESET}"
+                  f"{ind.confidence:>3.0f}% {Colors.GRAY}│ {ind.description}{Colors.RESET}")
-                  f" ({ind.confidence:.0f}%) │ {Colors.GRAY}{ind.description}{Colors.RESET}")
 
-    # Bullish reasons
-    if report.bullish_reasons:
-        print(f"\n  {Colors.GREEN}▲ Bullish Factors:{Colors.RESET}")
-        for reason in report.bullish_reasons[:5]:
-            print(f"    {Colors.GREEN}+{Colors.RESET} {reason}")
-
-    # Bearish reasons
-    if report.bearish_reasons:
-        print(f"\n  {Colors.RED}▼ Bearish Factors:{Colors.RESET}")
-        for reason in report.bearish_reasons[:5]:
-            print(f"    {Colors.RED}-{Colors.RESET} {reason}")
-
-    # Warnings
-    if report.warnings:
-        print(f"\n  {Colors.YELLOW}⚠ Warnings:{Colors.RESET}")
-        for warning in report.warnings:
-            print(f"    {Colors.YELLOW}!{Colors.RESET} {warning}")
-
-    # News context
-    if report.news_description and report.news_description != "Neutral news sentiment":
-        print(f"\n  {Colors.MAGENTA}📰 News:{Colors.RESET} {report.news_description}")
-    if report.event_description and report.event_risk != "clear":
-        print(f"  {Colors.YELLOW}📅 Event:{Colors.RESET} {report.event_description}")
-
-    # Trade setup
-    if report.trade_setup:
-        ts = report.trade_setup
-        dir_color = Colors.GREEN if ts.direction == "BUY" else Colors.RED
-        print(f"\n  {Colors.BOLD}Trade Setup:{Colors.RESET}")
-        print(f"    Direction:     {dir_color}{ts.direction}{Colors.RESET}")
-        print(f"    Entry:         {ts.entry_price:.5f}")
-        print(f"    Stop Loss:     {Colors.RED}{ts.stop_loss:.5f}{Colors.RESET} ({ts.risk_pips:.1f} pips)")
-        print(f"    Take Profit 1: {Colors.GREEN}{ts.take_profit_1:.5f}{Colors.RESET} ({ts.reward_pips:.1f} pips)")
-        print(f"    Take Profit 2: {Colors.GREEN}{ts.take_profit_2:.5f}{Colors.RESET}")
-        print(f"    Risk:Reward:   1:{ts.risk_reward:.1f}")
-
-    # Key levels
-    if report.primary_analysis and report.primary_analysis.key_levels:
         levels = report.primary_analysis.key_levels
-        print(f"\n  {Colors.BOLD}Key Levels:{Colors.RESET}")
+        if levels:
-        level_names = ["r3", "r2", "r1", "pivot", "s1", "s2", "s3"]
+            swing_keys = sorted(
-        for lvl in level_names:
+                [k for k in levels if k.startswith("swing_")],
-            if lvl in levels:
+                key=lambda k: levels[k], reverse=True,
-                val = levels[lvl]
+            )
-                if val > report.current_price:
+            if swing_keys:
-                    color = Colors.RED  # resistance
+                print(f"\n  {Colors.BOLD}Structure Levels{Colors.RESET}")
-                else:
+                for k in swing_keys:
-                    color = Colors.GREEN  # support
+                    val = levels[k]
-                label = lvl.upper()
+                    color = Colors.RED if val > report.current_price else Colors.GREEN
-                print(f"    {label:>6}: {color}{val:.5f}{Colors.RESET}")
+                    label = k.replace("_", " ").title()
-
+                    print(f"    {label:>20}  {color}{val:.3f}{Colors.RESET}")
-
-def print_event_calendar(reports: List[SignalReport]):
-    """Print upcoming economic events affecting watched pairs."""
-    events_shown = set()
-    has_events = False
-
-    for report in reports:
-        if report.event_risk != "clear" and report.event_description not in events_shown:
-            if not has_events:
-                print(f"\n{Colors.YELLOW}{Colors.BOLD}  📅 ECONOMIC CALENDAR ALERTS{Colors.RESET}")
-                has_events = True
-            events_shown.add(report.event_description)
-            risk_color = Colors.RED if report.event_risk == "danger" else Colors.YELLOW
-            print(f"    {risk_color}● {report.event_description}{Colors.RESET} → affects {report.display_name}")
-
-    if not has_events:
-        print(f"\n{Colors.GRAY}  📅 No high-impact events in the near term{Colors.RESET}")
 
 
 def report_to_dict(report: SignalReport) -> dict:
-    """Convert a SignalReport to a JSON-serializable dict."""
     d = {
         "symbol": report.symbol,
         "display_name": report.display_name,
@@ -305,100 +217,60 @@ def report_to_dict(report: SignalReport) -> dict:
             "reward_pips": round(ts.reward_pips, 1),
         }
     if report.primary_analysis:
-        d["indicators"] = {}
+        d["indicators"] = {
-        for name, ind in report.primary_analysis.indicators.items():
+            name: {
-            if not name.startswith("_"):
-                d["indicators"][name] = {
                 "signal": ind.signal.name,
                 "confidence": round(ind.confidence, 1),
                 "description": ind.description,
             }
+            for name, ind in report.primary_analysis.indicators.items()
+            if not name.startswith("_")
+        }
         d["key_levels"] = {
             k: round(v, 5) for k, v in report.primary_analysis.key_levels.items()
         }
     return d
 
 
+
 # ══════════════════════════════════════════════════════════════════
-#  MAIN
+#  CLI
 # ══════════════════════════════════════════════════════════════════
 
 def parse_args():
     parser = argparse.ArgumentParser(
-        description="Forex Signal Intelligence Platform",
+        description="ForexBot2 — XAU/USD Signal Intelligence",
         formatter_class=argparse.RawDescriptionHelpFormatter,
         epilog="""
 Examples:
-  python main.py                        Analyze all major pairs (H1)
+  python main.py                          Live signal (H1, summary)
-  python main.py --pairs EURUSD GBPUSD  Analyze specific pairs
+  python main.py --detail                 Live signal with full breakdown
-  python main.py --timeframe H4         Use 4-hour as primary TF
+  python main.py --timeframe H4 --detail  H4 timeframe
-  python main.py --all                  Include cross pairs
+  python main.py --json                   JSON output
-  python main.py --json                 Output as JSON
-  python main.py --detail               Show detailed analysis per pair
-  python main.py --watch                Continuous monitoring (5min refresh)
         """,
     )
-    parser.add_argument(
+    parser.add_argument("--timeframe", "-tf", default="H1",
-        "--pairs", nargs="+", default=None,
-        help="Specific pairs to analyze (e.g., EURUSD GBPUSD)",
-    )
-    parser.add_argument(
-        "--timeframe", "-tf", default="H1",
                         choices=["M5", "M15", "M30", "H1", "H4", "D1"],
-        help="Primary analysis timeframe (default: H1)",
+                        help="Primary analysis timeframe (default: H1)")
-    )
+    parser.add_argument("--detail", action="store_true",
-    parser.add_argument(
+                        help="Show full indicator breakdown")
-        "--all", action="store_true",
+    parser.add_argument("--json", action="store_true",
-        help="Include cross pairs in analysis",
+                        help="Output as JSON")
-    )
+    parser.add_argument("--verbose", "-v", action="store_true",
-    parser.add_argument(
+                        help="Debug logging")
-        "--json", action="store_true",
+
-        help="Output results as JSON",
-    )
-    parser.add_argument(
-        "--detail", action="store_true",
-        help="Show detailed analysis for each pair",
-    )
-    parser.add_argument(
-        "--top", type=int, default=None,
-        help="Show only top N signals by strength",
-    )
-    parser.add_argument(
-        "--watch", action="store_true",
-        help="Continuous monitoring mode (refreshes every 5 min)",
-    )
-    parser.add_argument(
-        "--min-grade", default="F",
-        choices=["A+", "A", "B", "C", "D", "F"],
-        help="Minimum signal quality to display",
-    )
-    parser.add_argument(
-        "--verbose", "-v", action="store_true",
-        help="Enable debug logging",
-    )
     return parser.parse_args()
 
 
-def symbol_from_name(name: str) -> str:
+def build_config(args) -> PlatformConfig:
-    """Convert pair name like 'EURUSD' to Yahoo symbol 'EURUSD=X'."""
-    name = name.upper().replace("/", "").replace("_", "")
-    if not name.endswith("=X"):
-        name += "=X"
-    return name
-
-
-def run_analysis(args) -> List[SignalReport]:
-    """Run a single analysis cycle."""
     config = PlatformConfig()
 
-    # Configure timeframe
     tf_map = {
         "M5": Timeframe.M5, "M15": Timeframe.M15, "M30": Timeframe.M30,
         "H1": Timeframe.H1, "H4": Timeframe.H4, "D1": Timeframe.D1,
     }
     config.primary_timeframe = tf_map[args.timeframe]
 
-    # Configure confirmation timeframes based on primary
     tf_confirmations = {
         "M5":  [Timeframe.M1,  Timeframe.M15],
         "M15": [Timeframe.M5,  Timeframe.H1],
@@ -411,34 +283,28 @@ def run_analysis(args) -> List[SignalReport]:
         args.timeframe, [Timeframe.M15, Timeframe.H4]
     )
 
-    # Configure pairs
+    return config
-    if args.pairs:
-        config.watch_pairs = [symbol_from_name(p) for p in args.pairs]
-    elif args.all:
-        config.watch_pairs = MAJOR_PAIRS + CROSS_PAIRS
-    else:
-        config.watch_pairs = MAJOR_PAIRS
 
-    # Run analysis
+
+def run_live(args, config: PlatformConfig):
+    print(f"{Colors.GRAY}  Fetching market data and running analysis...{Colors.RESET}\n")
     aggregator = SignalAggregator(config)
-    reports = aggregator.analyze_all_pairs()
+    report = aggregator.analyze_pair()
 
-    # Filter by grade
+    if args.json:
-    grade_order = ["A+", "A", "B", "C", "D", "F"]
+        print(json.dumps(report_to_dict(report), indent=2))
-    min_idx = grade_order.index(args.min_grade) if args.min_grade in grade_order else len(grade_order) - 1
+        return
-    reports = [r for r in reports if grade_order.index(r.signal_quality) <= min_idx]
 
-    # Top N
+    print_header("XAU/USD SIGNAL INTELLIGENCE")
-    if args.top:
+    print_signal_report(report, detail=args.detail)
-        reports = reports[:args.top]
+    print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
-
+    print(f"{Colors.GRAY}  Disclaimer: For educational/research purposes only.{Colors.RESET}")
-    return reports
+    print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
 
 
 def main():
     args = parse_args()
 
-    # Setup logging
     log_level = logging.DEBUG if args.verbose else logging.WARNING
     logging.basicConfig(
         level=log_level,
@@ -446,63 +312,8 @@ def main():
         datefmt="%H:%M:%S",
     )
 
-    if args.watch:
+    config = build_config(args)
-        # Continuous monitoring mode
+    run_live(args, config)
-        print(f"{Colors.BOLD}Starting continuous monitoring mode (Ctrl+C to stop)...{Colors.RESET}")
-        while True:
-            try:
-                reports = run_analysis(args)
-                if args.json:
-                    output = [report_to_dict(r) for r in reports]
-                    print(json.dumps(output, indent=2))
-                else:
-                    print("\033[2J\033[H")  # clear screen
-                    print_header()
-                    print_summary_table(reports)
-                    print_event_calendar(reports)
-                    if args.detail:
-                        for r in reports:
-                            print_detailed_report(r)
-                    print(f"\n{Colors.GRAY}  Next refresh in 5 minutes...{Colors.RESET}")
-
-                time.sleep(300)
-            except KeyboardInterrupt:
-                print(f"\n{Colors.YELLOW}Monitoring stopped.{Colors.RESET}")
-                break
-    else:
-        # Single run
-        print(f"{Colors.GRAY}  Fetching market data and running analysis...{Colors.RESET}\n")
-        reports = run_analysis(args)
-
-        if args.json:
-            output = [report_to_dict(r) for r in reports]
-            print(json.dumps(output, indent=2))
-        else:
-            print_header()
-            if reports:
-                print_summary_table(reports)
-                print_event_calendar(reports)
-
-                if args.detail:
-                    for r in reports:
-                        print_detailed_report(r)
-                else:
-                    # Show detail for top signals only
-                    actionable = [r for r in reports if r.signal_quality in ("A+", "A", "B")]
-                    if actionable:
-                        print(f"\n{Colors.BOLD}{Colors.WHITE}  ═══ ACTIONABLE SIGNALS (Grade B or above) ═══{Colors.RESET}")
-                        for r in actionable:
-                            print_detailed_report(r)
-                    else:
-                        print(f"\n{Colors.GRAY}  No high-grade signals detected. "
-                              f"Use --detail for full breakdown or --min-grade C to see more.{Colors.RESET}")
-            else:
-                print(f"{Colors.YELLOW}  No data available. Check your internet connection.{Colors.RESET}")
-
-            print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
-            print(f"{Colors.GRAY}  Disclaimer: This tool is for educational/research purposes only.{Colors.RESET}")
-            print(f"{Colors.GRAY}  Past performance ≠ future results. Always use proper risk management.{Colors.RESET}")
-            print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
 
 
 if __name__ == "__main__":

news_engine.py

@@ -9,6 +9,8 @@ A perfect technical setup can be destroyed by an unexpected NFP print.
 """
 
 import logging
+import re
+import time
 import xml.etree.ElementTree as ET
 from dataclasses import dataclass, field
 from datetime import datetime, timedelta, timezone
@@ -81,17 +83,26 @@ class NewsEngine:
     # Keywords that indicate directional sentiment
     BULLISH_KEYWORDS = {
         "rate hike", "hawkish", "strong growth", "beat expectations",
-        "above forecast", "surplus", "expansion", "employment gains",
+        "above forecast", "above estimate", "above expectation",
+        "surplus", "expansion", "employment gains",
         "strong jobs", "gdp growth", "inflation rising", "tightening",
         "positive surprise", "record high", "bullish", "rally",
         "stronger than expected", "upbeat",
+        "beats", "surges", "soars", "jumps", "climbs", "rises",
+        "higher than expected", "better than expected",
+        "strong data", "robust", "resilient", "accelerates",
     }
 
     BEARISH_KEYWORDS = {
         "rate cut", "dovish", "recession", "slowdown", "contraction",
-        "below forecast", "deficit", "miss expectations", "weak data",
+        "below forecast", "below estimate", "below expectation",
+        "deficit", "miss expectations", "weak data",
         "unemployment rise", "negative surprise", "bearish", "selloff",
         "weaker than expected", "downbeat", "easing", "stimulus",
+        "misses", "slides", "drops", "falls", "plunges", "tumbles",
+        "weakens", "slumps", "declines", "retreats",
+        "lower than expected", "worse than expected",
+        "disappointing", "soft data",
     }
 
     # Currency-specific keywords for headline parsing
@@ -104,6 +115,7 @@ class NewsEngine:
         "AUD": ["aussie", "rba", "australian", "australia"],
         "CAD": ["loonie", "boc", "canadian", "canada"],
         "NZD": ["kiwi", "rbnz", "new zealand"],
+        "XAU": ["gold", "xau", "precious metal", "safe haven", "bullion"],
     }
 
     # High-impact event types (these move markets significantly)
@@ -114,6 +126,8 @@ class NewsEngine:
         "boe rate", "rba rate", "snb rate", "boc rate",
     }
 
+    _calendar_backoff_until: float = 0.0
+
     def __init__(self, config: NewsConfig):
         self.config = config
         self._session = requests.Session()
@@ -187,15 +201,21 @@ class NewsEngine:
 
         return context
 
+    @staticmethod
+    def _parse_pair(pair_symbol: str) -> Tuple[str, str]:
+        if "/" in pair_symbol:
+            parts = pair_symbol.split("/")
+            return parts[0], parts[1]
+        clean = pair_symbol.replace("=X", "")
+        return clean[:3], clean[3:]
+
     def get_event_risk_level(self, pair_symbol: str, context: FundamentalContext) -> Tuple[str, str]:
         """
         Get event risk level for a specific pair.
         Returns (risk_level, description).
         risk_level: "clear", "caution", "danger"
         """
-        clean = pair_symbol.replace("=X", "")
+        base, quote = self._parse_pair(pair_symbol)
-        base = clean[:3]
-        quote = clean[3:]
 
         # Check upcoming high-impact events
         for event in context.events_upcoming_1h:
@@ -222,9 +242,7 @@ class NewsEngine:
         Returns (score, description) where score is -1 to +1.
         Positive = bullish for the pair, negative = bearish.
         """
-        clean = pair_symbol.replace("=X", "")
+        base, quote = self._parse_pair(pair_symbol)
-        base = clean[:3]
-        quote = clean[3:]
 
         base_sent = context.news_sentiment.get(base, 0.0)
         quote_sent = context.news_sentiment.get(quote, 0.0)
@@ -247,18 +265,27 @@ class NewsEngine:
 
     # ── Private Methods ────────────────────────────────────────────
 
+    # Maps country names in MyFXBook titles to currency codes
+    _COUNTRY_TO_CURRENCY = {
+        "united states": "USD", "euro area": "EUR", "united kingdom": "GBP",
+        "japan": "JPY", "switzerland": "CHF", "australia": "AUD",
+        "canada": "CAD", "new zealand": "NZD",
+    }
+
     def _fetch_economic_calendar(self, currencies: List[str]) -> List[EconomicEvent]:
         """
         Fetch economic calendar events from a free API.
+        Tries Forex Factory first, falls back to MyFXBook RSS.
         Returns an empty list (with a log warning) if unavailable —
         we never generate fake events since that causes constant false alarms.
         """
+        for fetcher in (self._try_fetch_calendar_api, self._try_fetch_myfxbook_rss):
             try:
-            events = self._try_fetch_calendar_api()
+                events = fetcher()
                 if events:
                     return [e for e in events if e.currency in currencies]
             except Exception as e:
-            logger.debug(f"Calendar API fetch exception: {e}")
+                logger.debug(f"Calendar fetch exception ({fetcher.__name__}): {e}")
 
         logger.info(
             "Economic calendar data unavailable. "
@@ -268,13 +295,38 @@ class NewsEngine:
 
     def _try_fetch_calendar_api(self) -> List[EconomicEvent]:
         """Attempt to fetch from the Forex Factory free JSON calendar."""
-        events = []
+        now = time.time()
+        if now < NewsEngine._calendar_backoff_until:
+            wait = NewsEngine._calendar_backoff_until - now
+            logger.debug(f"Calendar API in backoff for {wait:.0f}s more")
+            return []
+
         url = "https://nfs.faireconomy.media/ff_calendar_thisweek.json"
+
+        for attempt in range(3):
+            try:
                 resp = self._session.get(url, timeout=10)
+            except requests.RequestException as e:
+                logger.debug(f"Calendar request failed (attempt {attempt + 1}): {e}")
+                if attempt < 2:
+                    time.sleep(2 ** attempt)
+                continue
+
+            if resp.status_code == 429:
+                retry_after = int(resp.headers.get("Retry-After", 300))
+                retry_after = min(retry_after, 600)
+                NewsEngine._calendar_backoff_until = time.time() + retry_after
+                logger.warning(
+                    f"Calendar API rate-limited (429), backing off {retry_after}s"
+                )
+                return []
+
             if resp.status_code != 200:
+                logger.debug(f"Calendar API returned {resp.status_code}")
                 return []
 
             data = resp.json()
+            events = []
             impact_map = {"High": "high", "Medium": "medium", "Low": "low"}
 
             for item in data:
@@ -300,6 +352,79 @@ class NewsEngine:
 
             return events
 
+        return []
+
+    def _try_fetch_myfxbook_rss(self) -> List[EconomicEvent]:
+        """Fallback calendar source: MyFXBook economic calendar RSS."""
+        url = "https://www.myfxbook.com/rss/forex-economic-calendar-events"
+        try:
+            resp = self._session.get(url, timeout=10)
+        except requests.RequestException as e:
+            logger.debug(f"MyFXBook calendar request failed: {e}")
+            return []
+
+        if resp.status_code != 200:
+            return []
+
+        try:
+            root = ET.fromstring(resp.content)
+        except ET.ParseError:
+            return []
+
+        events = []
+        for item in root.iter("item"):
+            try:
+                title_el = item.find("title")
+                pub_el = item.find("pubDate")
+                desc_el = item.find("description")
+                if title_el is None or title_el.text is None:
+                    continue
+
+                title = title_el.text.strip()
+
+                currency = ""
+                title_lower = title.lower()
+                for country, cur in self._COUNTRY_TO_CURRENCY.items():
+                    if title_lower.startswith(country):
+                        currency = cur
+                        title = title[len(country):].strip()
+                        break
+                if not currency:
+                    continue
+
+                event_time = self._parse_rss_date(
+                    pub_el.text if pub_el is not None else None
+                )
+                if event_time is None:
+                    continue
+
+                impact = "low"
+                desc_text = desc_el.text if desc_el is not None and desc_el.text else ""
+                impact_match = re.search(r"sprite-(high|medium|low)-impact", desc_text)
+                if impact_match:
+                    impact = impact_match.group(1)
+
+                cells = re.findall(r"<td[^>]*>(.*?)</td>", desc_text, re.DOTALL)
+                cells = [re.sub(r"<[^>]+>", "", c).strip() for c in cells]
+                previous = cells[2] if len(cells) > 2 and cells[2] else None
+                forecast = cells[3] if len(cells) > 3 and cells[3] else None
+                actual = cells[4] if len(cells) > 4 and cells[4] else None
+
+                events.append(EconomicEvent(
+                    time=event_time,
+                    currency=currency,
+                    impact=impact,
+                    event=title,
+                    forecast=forecast,
+                    previous=previous,
+                    actual=actual,
+                ))
+            except Exception:
+                continue
+
+        logger.debug(f"MyFXBook calendar: parsed {len(events)} events")
+        return events
+
     def _fetch_news_sentiment(self) -> List[NewsSentiment]:
         """Fetch and analyze news headlines for sentiment."""
         headlines = []
@@ -396,6 +521,8 @@ class NewsEngine:
         formats = [
             "%a, %d %b %Y %H:%M:%S %z",   # RFC 822 with numeric tz offset
             "%a, %d %b %Y %H:%M:%S %Z",   # RFC 822 with tz abbreviation (GMT/UTC)
+            "%a, %d %b %Y %H:%M %Z",      # RFC 822 without seconds (MyFXBook)
+            "%a, %d %b %Y %H:%M %z",      # RFC 822 without seconds, numeric offset
             "%Y-%m-%dT%H:%M:%S%z",         # ISO 8601 with offset
             "%Y-%m-%d %H:%M:%S",           # ISO 8601 naive (assume UTC)
         ]
@@ -413,15 +540,9 @@ class NewsEngine:
 
     @staticmethod
     def _get_affected_pairs(currency: str) -> List[str]:
-        """Get all major pairs affected by a currency event."""
+        _MAP = {
-        pairs = {
+            "USD": ["XAU/USD", "EUR/USD"],
-            "USD": ["EURUSD=X", "GBPUSD=X", "USDJPY=X", "USDCHF=X", "AUDUSD=X", "USDCAD=X", "NZDUSD=X"],
+            "XAU": ["XAU/USD"],
-            "EUR": ["EURUSD=X", "EURGBP=X", "EURJPY=X", "EURCHF=X"],
+            "EUR": ["EUR/USD"],
-            "GBP": ["GBPUSD=X", "EURGBP=X", "GBPJPY=X"],
-            "JPY": ["USDJPY=X", "EURJPY=X", "GBPJPY=X", "AUDJPY=X", "CADJPY=X"],
-            "CHF": ["USDCHF=X", "EURCHF=X"],
-            "AUD": ["AUDUSD=X", "AUDJPY=X"],
-            "CAD": ["USDCAD=X", "CADJPY=X"],
-            "NZD": ["NZDUSD=X"],
         }
-        return pairs.get(currency, [])
+        return _MAP.get(currency, [])

pair_signals/__init__.py

@@ -0,0 +1,52 @@
+"""
+Per-pair signal producers.
+
+Each pair module exposes:
+    make_config(primary_tf, confirmation_tfs) -> PlatformConfig
+
+To add a new pair: create pair_signals/<pair>.py with make_config(),
+then register it in this file.
+"""
+
+from typing import Optional, List, Dict, Callable
+
+from config import PlatformConfig, Timeframe
+from signal_aggregator import SignalAggregator, SignalReport
+from data_provider import ForexDataProvider
+
+PAIR_PRODUCERS: Dict[str, Callable] = {}
+
+
+def register_pair(symbol: str, config_factory: Callable):
+    PAIR_PRODUCERS[symbol] = config_factory
+
+
+def get_supported_pairs() -> List[str]:
+    return list(PAIR_PRODUCERS.keys())
+
+
+def produce_signal(
+    symbol: str,
+    primary_tf: Timeframe,
+    data_provider: ForexDataProvider,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+) -> SignalReport:
+    config_factory = PAIR_PRODUCERS.get(symbol)
+    if config_factory is None:
+        raise ValueError(
+            f"No signal producer registered for {symbol}. "
+            f"Available: {list(PAIR_PRODUCERS.keys())}"
+        )
+
+    config = config_factory(primary_tf, confirmation_tfs)
+    aggregator = SignalAggregator(config)
+    aggregator.data_provider = data_provider
+    return aggregator.analyze_pair(symbol=symbol)
+
+
+from pair_signals import xauusd as _xau_mod
+from pair_signals import eurusd as _eur_mod
+from config import XAUUSD, EURUSD
+
+register_pair(XAUUSD, _xau_mod.make_config)
+register_pair(EURUSD, _eur_mod.make_config)

pair_signals/eurusd.py

@@ -0,0 +1,58 @@
+"""EUR/USD signal producer config."""
+
+from typing import Optional, List
+from config import PlatformConfig, Timeframe, EURUSD
+
+_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+_TRENDING = {
+    "ema_trend": 2.0, "rsi": 1.0, "macd": 2.0,
+    "bollinger": 0.5, "keltner": 1.0, "roc": 1.5,
+    "candlestick": 0.5, "divergence": 2.0, "adx": 1.0,
+}
+
+EUR_REGIME_WEIGHTS = {
+    "trending_up": _TRENDING,
+    "trending_down": _TRENDING,
+    "trending": _TRENDING,
+    "ranging": {
+        "ema_trend": 0.3, "rsi": 2.0, "macd": 1.0,
+        "bollinger": 2.0, "keltner": 1.5, "roc": 0.5,
+        "candlestick": 0.8, "divergence": 2.5, "adx": 0.5,
+    },
+    "transitioning": {
+        "ema_trend": 1.5, "rsi": 1.5, "macd": 2.0,
+        "bollinger": 1.0, "keltner": 1.5, "roc": 2.0,
+        "candlestick": 0.8, "divergence": 2.0, "adx": 1.0,
+    },
+}
+
+
+def make_config(
+    primary_tf: Timeframe,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+) -> PlatformConfig:
+    config = PlatformConfig()
+    config.symbol = EURUSD
+    config.display_name = "EUR/USD"
+    config.pip_value = 0.0001
+    config.primary_timeframe = primary_tf
+    config.confirmation_timeframes = (
+        confirmation_tfs or _CONFIRMATIONS.get(primary_tf, [Timeframe.M15, Timeframe.H4])
+    )
+
+    config.xau_strategy.enabled = False
+
+    config.backtest.session_start_hour = 7
+    config.backtest.session_end_hour = 20
+    config.backtest.blocked_hours_utc = [22, 23, 0, 1, 2, 3]
+    config.backtest.allowed_weekdays = [0, 1, 2, 3, 4]
+
+    config.technical.regime_weights_override = EUR_REGIME_WEIGHTS
+
+    return config

pair_signals/xauusd.py

@@ -0,0 +1,23 @@
+"""XAU/USD signal producer config."""
+
+from typing import Optional, List
+from config import PlatformConfig, Timeframe
+
+_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+
+def make_config(
+    primary_tf: Timeframe,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+) -> PlatformConfig:
+    config = PlatformConfig()
+    config.primary_timeframe = primary_tf
+    config.confirmation_timeframes = (
+        confirmation_tfs or _CONFIRMATIONS.get(primary_tf, [Timeframe.M15, Timeframe.H4])
+    )
+    return config

requirements.txt

@@ -0,0 +1,6 @@
+numpy>=1.24
+pandas>=2.0
+requests>=2.28
+twelvedata>=1.2.0
+python-telegram-bot>=20.0
+websockets>=12.0

script.txt

@@ -1 +0,0 @@
-pip install pandas numpy requests python-dateutil -q