Fix live session filter overriding XAU strategy signals

The general session filter ran after analyze_from_dataframes() and
wiped out XAU oversold bounce signals at most hours. Now skips the
session filter when the XAU strategy has already fired — the strategy
has its own backtested hour filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

.gitignore

@@ -0,0 +1,25 @@
+# Secrets
+.env
+.env.local
+
+# Editor / tooling
+.claude/
+.vscode/
+.idea/
+
+# Python
+__pycache__/
+*.pyc
+*.egg-info/
+.venv/
+venv/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Logs
+*.log
+
+# Backtesting (scripts, data, results)
+backtest/

USAGE.md

@@ -0,0 +1,198 @@
+# ForexBot2 — USD/JPY Signal Intelligence
+
+A focused USD/JPY signal generator + walk-forward backtester. Live signals
+and backtests run through the **same** technical engine, so what you measure
+in a backtest is exactly what you'd execute live.
+
+## Setup
+
+```bash
+pip install -r requirements.txt
+```
+
+API key (optional — a default key is in `config.py`):
+
+```bash
+export TWELVE_DATA_API_KEY=your_key_here          # Linux/Mac
+$env:TWELVE_DATA_API_KEY="your_key_here"          # PowerShell
+```
+
+---
+
+## Live Signal
+
+```bash
+python main.py                          # H1, summary view
+python main.py --detail                 # Full indicator breakdown
+python main.py --timeframe H4 --detail  # H4 instead of H1
+python main.py --json                   # JSON output (for piping)
+```
+
+| Option        | Default | Description                                |
+|---------------|---------|--------------------------------------------|
+| `--timeframe` | `H1`    | `M5 M15 M30 H1 H4 D1`                      |
+| `--detail`    | off     | Indicator breakdown, key levels, reasons   |
+| `--json`      | off     | Machine-readable output                    |
+| `--verbose`   | off     | Debug logging                              |
+
+### Output
+
+```
+USD/JPY  │  Price: 149.230  │  Regime: trending_up  │  Vol: normal
+
+SIGNAL: BUY  │  Score: +28.4  │  Grade: B  │  Confidence: 62%
+Action: BUY — Decent setup, wait for confirmation
+
+Multi-Timeframe:  ✓ ALIGNED
+   M15: BUY          (+18.2)
+    H1: BUY          (+28.4)
+    H4: STRONG BUY   (+45.1)
+
+Trade Setup:
+  Direction:     BUY
+  Entry:         149.230
+  Stop Loss:     148.960  (27.0 pips)
+  Take Profit 1: 149.770  (54.0 pips)
+  Take Profit 2: 150.040
+  Risk:Reward:   1:2.0
+```
+
+---
+
+## Backtest (Walk-Forward)
+
+The backtester replays USD/JPY history bar-by-bar through the live signal
+pipeline. No lookahead, no parallel implementation — what passes in
+backtest passes live.
+
+```bash
+python main.py --backtest                          # Full available history
+python main.py --backtest --bars 2000              # Last 2000 H1 bars
+python main.py --backtest --start 2024-01-01 --end 2024-12-31
+python main.py --backtest --timeframe H4           # Backtest on H4
+python main.py --backtest --min-grade A            # Only A/A+ entries
+python main.py --backtest --journal                # Last 10 trades printed
+python main.py --backtest --journal-csv trades.csv # Full journal to CSV
+python main.py --backtest --regime-filter          # Skip choppy/ranging bars
+python main.py --backtest --regime-filter --regime-min-adx 35
+python main.py --backtest --session-filter         # Trade only 07-16 UTC
+python main.py --backtest --session-filter --session-start 8 --session-end 17
+python main.py --backtest --auto-session-filter    # Reactive session toggle
+python main.py --backtest --auto-session-filter --auto-session-lookback 15 \
+                          --auto-session-threshold-pct -0.5
+```
+
+### Regime filter
+
+The engine is a confluence-based trend-follower — it makes money in trending
+regimes and loses in chop. `--regime-filter` gates entries on H4 ADX:
+signals only fire when ADX ≥ `--regime-min-adx` (default 30). Sweet spot is
+ADX 30–35; tighter cuts trades faster than it improves quality.
+
+### Session filter
+
+`--session-filter` restricts entries to a UTC hour window (default 07-16,
+covering London + NY). Diagnostic on 2025-11 → 2026-05 showed 47% WR /
++127 pips inside that window vs 0% WR / -571 pips outside it. Override
+bounds with `--session-start` (inclusive) and `--session-end` (exclusive).
+
+Tradeoff: it's regime-specific medicine. In trending regimes (e.g. 2024)
+it cuts winning off-hours trades and gives back ~33% of the edge. Ships
+**default off** because the pure regime filter still wins cumulative
+across the 2024 / 2023 / 2026 windows.
+
+### Auto session toggle
+
+`--auto-session-filter` is a reactive variant: it tracks the rolling mean
+`pnl_pct` over the last N closed trades (`--auto-session-lookback`,
+default 10) and activates the 07-16 UTC window only when that mean drops
+below `--auto-session-threshold-pct` (default `-0.3`). Releases
+automatically once expectancy recovers.
+
+Less aggressive than the manual flag — preserves more trend edge but
+reacts slowly because the deque must fill with losses before kicking in.
+Use this when you don't want to manually toggle the regime medicine.
+
+### Execution model
+
+| Aspect             | Behavior                                                          |
+|--------------------|-------------------------------------------------------------------|
+| Entry              | Next bar's open (no current-bar lookahead)                        |
+| Stop Loss          | 1.5 × ATR from entry (configurable)                               |
+| Take Profit        | RR × stop distance (default 1:2)                                  |
+| Same-bar SL & TP   | Conservative: SL fills first                                      |
+| Spread cost        | Half-spread on entry + half-spread on exit (full round-trip)      |
+| Slippage           | 0.5 pips per fill (entry + exit)                                  |
+| Position size      | 1% of current equity risked per trade                             |
+| Concurrency        | One open position at a time                                       |
+| News in backtest   | Disabled — RSS/calendar can't be replayed without lookahead       |
+
+### Output
+
+```
+Backtest Summary — USD/JPY
+─────────────────────────────────────────────────
+Trades:           87  (52W / 35L)
+Win Rate:         59.8%
+Profit Factor:    1.84
+Expectancy:       +12.4 pips/trade  (+12.42 per trade)
+Initial Balance:  $10,000.00
+Final Balance:    $11,084.00
+Total Return:     +10.84%
+CAGR:             +18.92%
+Max Drawdown:     -4.21%
+Sharpe Ratio:     1.42
+Sortino Ratio:    2.18
+Avg Duration:     14.3 bars
+Win Streak:       7 | Loss Streak: 4
+─────────────────────────────────────────────────
+```
+
+---
+
+## Parameter Optimization (parallel)
+
+```bash
+python main.py --backtest --optimize
+```
+
+Runs a 3×3×3 grid (27 combos) over `atr_sl_multiplier`, `default_rr_ratio`,
+and `min_score` (signal-score threshold). Combos run in parallel across all
+CPU cores via `multiprocessing.Pool`. Twelve Data is hit once — workers
+share an on-disk parquet cache (`.cache/usdjpy_<tf>.parquet`).
+
+Output ranks combos by total return:
+
+```
+Top 10 Parameter Combinations (by total return):
+
+#   Return%     PF   WinR  Trades  MaxDD%  Sharpe  Params
+──────────────────────────────────────────────────────────────────────
+1   +14.21%   2.10  62.5%     64   -3.84   1.78    atr_sl_multiplier=1.5, default_rr_ratio=2.5, min_score=25.0
+2   +11.93%   1.92  58.7%     71   -4.11   1.55    atr_sl_multiplier=1.0, default_rr_ratio=2.0, min_score=15.0
+...
+```
+
+Customize the grid in `config.py → BacktestConfig.param_grid`.
+
+---
+
+## Files
+
+| File                  | Role                                              |
+|-----------------------|---------------------------------------------------|
+| `main.py`             | CLI entry point (live + backtest + optimize)      |
+| `config.py`           | All parameters in one place                       |
+| `data_provider.py`    | Twelve Data fetcher + parquet cache               |
+| `engine.py`           | Technical indicators (8 + divergence + patterns)  |
+| `signal_aggregator.py`| Multi-TF aggregation, grading, trade setup        |
+| `news_engine.py`      | RSS sentiment + economic calendar (live only)     |
+| `backtest.py`         | Walk-forward replay + parallel optimizer          |
+| `metrics.py`          | Win rate, PF, Sharpe, max DD, trade journal       |
+
+---
+
+## Disclaimer
+
+For educational and research use. Past performance does not guarantee
+future results. Always use proper risk management.

backtest/evaluate.py

@@ -0,0 +1,613 @@
+#!/usr/bin/env python3
+"""
+ForexBot2 — Historical Signal Evaluation
+=========================================
+Walks the engine over the last 7 days of market data for all scan pairs,
+records every signal and trade setup, then checks outcomes on M5 data.
+
+GPU-accelerated outcome evaluation via CuPy (RTX 4070 SUPER).
+Falls back to NumPy if CuPy/CUDA is unavailable.
+
+Usage:
+    python backtest/evaluate.py                  # full 7-day eval, M15+H1
+    python backtest/evaluate.py --tf H1          # single timeframe
+    python backtest/evaluate.py --days 3         # shorter window
+    python backtest/evaluate.py --no-cache       # re-download all data
+"""
+
+import argparse
+import csv
+import os
+import sys
+import time
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from dataclasses import dataclass, field
+from datetime import datetime, timezone, timedelta
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple
+
+import numpy as np
+import pandas as pd
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from config import (
+    PlatformConfig, Timeframe, SignalStrength,
+    EURUSD, USDJPY, GBPUSD, XAUUSD, PAIR_INFO,
+)
+from data_provider import ForexDataProvider
+from signal_aggregator import SignalAggregator
+
+# ── GPU setup ────────────────────────────────────────────────────────
+try:
+    if "CUDA_PATH" not in os.environ:
+        _nvidia_dir = os.path.join(
+            sys.prefix, "lib", "python3.12", "site-packages", "nvidia"
+        )
+        if os.path.isdir(_nvidia_dir):
+            os.environ["CUDA_PATH"] = _nvidia_dir
+    import cupy as cp
+    _test = cp.array([1.0, 2.0])
+    _ = cp.where(_test > 0.5)
+    GPU = True
+    GPU_NAME = cp.cuda.runtime.getDeviceProperties(0)["name"].decode()
+    print(f"GPU: {GPU_NAME} (CuPy {cp.__version__})")
+    del _test
+except Exception:
+    import numpy as cp  # type: ignore[no-redef]
+    GPU = False
+    print("GPU: not available — falling back to NumPy")
+
+# ── Constants ────────────────────────────────────────────────────────
+BACKTEST_DIR = Path(__file__).resolve().parent
+DATA_DIR = BACKTEST_DIR / "data"
+RESULTS_DIR = BACKTEST_DIR / "results"
+
+SCAN_PAIRS = [EURUSD, GBPUSD, XAUUSD]
+
+EVAL_TIMEFRAMES = [Timeframe.M15, Timeframe.H1]
+
+TF_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+ALL_TFS_NEEDED = {
+    Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4, Timeframe.D1,
+}
+
+OUTCOME_TF = Timeframe.M5
+
+MIN_LOOKBACK_BARS = 250
+
+
+@dataclass
+class Trade:
+    pair: str
+    display: str
+    timeframe: str
+    entry_time: pd.Timestamp
+    direction: str
+    entry_price: float
+    stop_loss: float
+    take_profit_1: float
+    take_profit_2: Optional[float]
+    risk_reward: float
+    risk_pips: float
+    reward_pips: float
+    signal_quality: str
+    confidence: float
+    score: float
+    regime: str
+    outcome: str = "OPEN"
+    exit_price: float = 0.0
+    exit_time: Optional[pd.Timestamp] = None
+    pnl_pips: float = 0.0
+    bars_held: int = 0
+
+
+# ══════════════════════════════════════════════════════════════════════
+#  DATA FETCHING & CACHING
+# ══════════════════════════════════════════════════════════════════════
+
+def _cache_path(symbol: str, tf: Timeframe) -> Path:
+    safe = symbol.replace("/", "_").replace("=", "")
+    return DATA_DIR / f"{safe}_{tf.value}.parquet"
+
+
+def fetch_all_data(
+    pairs: List[str],
+    timeframes,
+    lookback_days: int = 67,
+    use_cache: bool = True,
+) -> Dict[str, Dict[Timeframe, pd.DataFrame]]:
+    """Fetch OHLCV for all pair×TF combos, cache to parquet."""
+    config = PlatformConfig()
+    config.data_lookback_days = lookback_days
+    dp = ForexDataProvider(config)
+    data: Dict[str, Dict[Timeframe, pd.DataFrame]] = {}
+
+    total = len(pairs) * len(timeframes)
+    done = 0
+
+    for symbol in pairs:
+        data[symbol] = {}
+        display = PAIR_INFO.get(symbol, {}).get("display", symbol)
+        for tf in timeframes:
+            done += 1
+            cache = _cache_path(symbol, tf)
+            if use_cache and cache.exists():
+                mtime = datetime.fromtimestamp(cache.stat().st_mtime, tz=timezone.utc)
+                age_hours = (datetime.now(timezone.utc) - mtime).total_seconds() / 3600
+                if age_hours < 4:
+                    df = pd.read_parquet(cache)
+                    data[symbol][tf] = df
+                    print(f"  [{done}/{total}] {display} {tf.value} — cached ({len(df)} bars)")
+                    continue
+
+            print(f"  [{done}/{total}] {display} {tf.value} — fetching...", end="", flush=True)
+            try:
+                df = dp.fetch_ohlcv(symbol, tf, lookback_days=lookback_days)
+                data[symbol][tf] = df
+                try:
+                    df.to_parquet(cache)
+                except Exception:
+                    pass
+                print(f" {len(df)} bars")
+            except Exception as e:
+                print(f" FAILED: {e}")
+
+    return data
+
+
+# ══════════════════════════════════════════════════════════════════════
+#  WALK-FORWARD EVALUATION
+# ══════════════════════════════════════════════════════════════════════
+
+def walk_forward(
+    pair: str,
+    eval_tf: Timeframe,
+    all_data: Dict[Timeframe, pd.DataFrame],
+    eval_days: int,
+) -> List[Trade]:
+    """Walk through eval_days of candles, run the engine at each bar."""
+    primary_df = all_data.get(eval_tf)
+    if primary_df is None or primary_df.empty:
+        return []
+
+    conf_tfs = TF_CONFIRMATIONS.get(eval_tf, [Timeframe.M15, Timeframe.H4])
+    display = PAIR_INFO.get(pair, {}).get("display", pair)
+
+    cutoff = primary_df.index[-1] - pd.Timedelta(days=eval_days)
+    eval_indices = primary_df.index[primary_df.index >= cutoff]
+
+    if len(eval_indices) == 0:
+        return []
+
+    first_eval_pos = primary_df.index.get_loc(eval_indices[0])
+    if first_eval_pos < MIN_LOOKBACK_BARS:
+        first_eval_pos = MIN_LOOKBACK_BARS
+        eval_indices = primary_df.index[first_eval_pos:]
+
+    config = PlatformConfig()
+    config.primary_timeframe = eval_tf
+    config.confirmation_timeframes = conf_tfs
+    aggregator = SignalAggregator(config)
+
+    cooldown_bars = {
+        Timeframe.M5: 24, Timeframe.M15: 10,
+        Timeframe.H1: 5, Timeframe.H4: 3,
+    }.get(eval_tf, 6)
+
+    trades: List[Trade] = []
+    total = len(eval_indices)
+    bars_until_next = 0
+
+    for i, eval_time in enumerate(eval_indices):
+        if (i + 1) % max(1, total // 10) == 0:
+            print(f"    {display} {eval_tf.value}: {i+1}/{total} candles...", flush=True)
+
+        if bars_until_next > 0:
+            bars_until_next -= 1
+            continue
+
+        dfs: Dict[Timeframe, pd.DataFrame] = {}
+        dfs[eval_tf] = primary_df.loc[:eval_time]
+
+        for ctf in conf_tfs:
+            ctf_df = all_data.get(ctf)
+            if ctf_df is not None and not ctf_df.empty:
+                dfs[ctf] = ctf_df.loc[:eval_time]
+
+        try:
+            report = aggregator.analyze_from_dataframes(
+                symbol=pair, dfs=dfs, use_news=False,
+            )
+        except Exception:
+            continue
+
+        if report.trade_setup and report.signal_quality in ("A+", "A", "B", "C"):
+            ts = report.trade_setup
+            trades.append(Trade(
+                pair=pair,
+                display=display,
+                timeframe=eval_tf.value,
+                entry_time=eval_time,
+                direction=ts.direction,
+                entry_price=ts.entry_price,
+                stop_loss=ts.stop_loss,
+                take_profit_1=ts.take_profit_1,
+                take_profit_2=ts.take_profit_2,
+                risk_reward=ts.risk_reward,
+                risk_pips=ts.risk_pips,
+                reward_pips=ts.reward_pips,
+                signal_quality=report.signal_quality,
+                confidence=report.final_confidence,
+                score=report.final_score,
+                regime=report.market_regime,
+            ))
+            bars_until_next = cooldown_bars
+
+    return trades
+
+
+def _walk_forward_worker(args):
+    """Top-level worker for multiprocessing (must be picklable)."""
+    pair, tf, pair_data, eval_days = args
+    display = PAIR_INFO.get(pair, {}).get("display", pair)
+    trades = walk_forward(pair, tf, pair_data, eval_days)
+    return pair, tf, display, trades
+
+
+# ══════════════════════════════════════════════════════════════════════
+#  GPU-ACCELERATED OUTCOME EVALUATION
+# ══════════════════════════════════════════════════════════════════════
+
+def evaluate_outcomes(
+    trades: List[Trade],
+    m5_data: Dict[str, pd.DataFrame],
+    max_bars: int = 500,
+) -> List[Trade]:
+    """
+    For each trade, scan forward in M5 data to check if TP1 or SL hit first.
+    Uses GPU (CuPy) for vectorized parallel evaluation across all trades per pair.
+    """
+    if not trades:
+        return trades
+
+    pairs_in_trades = set(t.pair for t in trades)
+    label = "GPU" if GPU else "CPU"
+    print(f"\nEvaluating {len(trades)} trade outcomes on {label}...")
+
+    for pair in pairs_in_trades:
+        df = m5_data.get(pair)
+        if df is None or df.empty:
+            continue
+
+        pair_trades = [t for t in trades if t.pair == pair]
+        if not pair_trades:
+            continue
+
+        highs_np = df["High"].values.astype(np.float64)
+        lows_np = df["Low"].values.astype(np.float64)
+        closes_np = df["Close"].values.astype(np.float64)
+        timestamps = df.index
+
+        highs_gpu = cp.asarray(highs_np)
+        lows_gpu = cp.asarray(lows_np)
+
+        for trade in pair_trades:
+            start_mask = timestamps > trade.entry_time
+            start_indices = np.where(start_mask)[0]
+            if len(start_indices) == 0:
+                continue
+
+            start = int(start_indices[0])
+            end = min(start + max_bars, len(highs_gpu))
+
+            if start >= end:
+                continue
+
+            h_slice = highs_gpu[start:end]
+            l_slice = lows_gpu[start:end]
+
+            if trade.direction == "BUY":
+                tp_mask = h_slice >= trade.take_profit_1
+                sl_mask = l_slice <= trade.stop_loss
+            else:
+                tp_mask = l_slice <= trade.take_profit_1
+                sl_mask = h_slice >= trade.stop_loss
+
+            tp_hits = cp.where(tp_mask)[0]
+            sl_hits = cp.where(sl_mask)[0]
+
+            tp_idx = int(tp_hits[0]) if len(tp_hits) > 0 else max_bars + 1
+            sl_idx = int(sl_hits[0]) if len(sl_hits) > 0 else max_bars + 1
+
+            pip_val = PAIR_INFO.get(pair, {}).get("pip_value", 0.0001)
+
+            if tp_idx <= sl_idx and tp_idx < max_bars + 1:
+                trade.outcome = "WIN"
+                trade.bars_held = tp_idx + 1
+                real_idx = start + tp_idx
+                trade.exit_price = trade.take_profit_1
+                trade.exit_time = timestamps[real_idx]
+                trade.pnl_pips = trade.reward_pips
+            elif sl_idx < tp_idx and sl_idx < max_bars + 1:
+                trade.outcome = "LOSS"
+                trade.bars_held = sl_idx + 1
+                real_idx = start + sl_idx
+                trade.exit_price = trade.stop_loss
+                trade.exit_time = timestamps[real_idx]
+                trade.pnl_pips = -trade.risk_pips
+            else:
+                trade.outcome = "OPEN"
+                last_idx = end - 1
+                trade.exit_time = timestamps[last_idx]
+                exit_p = float(closes_np[last_idx])
+                trade.exit_price = exit_p
+                if trade.direction == "BUY":
+                    trade.pnl_pips = (exit_p - trade.entry_price) / pip_val
+                else:
+                    trade.pnl_pips = (trade.entry_price - exit_p) / pip_val
+                trade.bars_held = end - start
+
+    return trades
+
+
+# ══════════════════════════════════════════════════════════════════════
+#  REPORTING
+# ══════════════════════════════════════════════════════════════════════
+
+def print_report(trades: List[Trade], eval_days: int):
+    print("\n")
+    print("=" * 90)
+    print(f"  FOREXBOT2 — ENGINE EVALUATION REPORT ({eval_days}-DAY)")
+    print(f"  {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}")
+    print("=" * 90)
+
+    if not trades:
+        print("\n  No trades generated.\n")
+        return
+
+    closed = [t for t in trades if t.outcome in ("WIN", "LOSS")]
+    wins = [t for t in closed if t.outcome == "WIN"]
+    losses = [t for t in closed if t.outcome == "LOSS"]
+    still_open = [t for t in trades if t.outcome == "OPEN"]
+
+    # ── Overall ──────────────────────────────────────────────────
+    print(f"\n  OVERALL")
+    print(f"  {'─' * 50}")
+    print(f"  Total signals with trade setup: {len(trades)}")
+    print(f"  Closed: {len(closed)}  |  Still open: {len(still_open)}")
+
+    if closed:
+        win_rate = len(wins) / len(closed) * 100
+        total_pnl = sum(t.pnl_pips for t in closed)
+        avg_win = np.mean([t.pnl_pips for t in wins]) if wins else 0
+        avg_loss = np.mean([abs(t.pnl_pips) for t in losses]) if losses else 0
+        gross_profit = sum(t.pnl_pips for t in wins) if wins else 0
+        gross_loss = sum(abs(t.pnl_pips) for t in losses) if losses else 0
+        profit_factor = gross_profit / gross_loss if gross_loss > 0 else float("inf")
+        avg_rr = np.mean([t.risk_reward for t in closed])
+        avg_bars = np.mean([t.bars_held for t in closed])
+
+        print(f"  Win rate:       {win_rate:.1f}%  ({len(wins)}W / {len(losses)}L)")
+        print(f"  Profit factor:  {profit_factor:.2f}")
+        print(f"  Total P&L:      {total_pnl:+.1f} pips")
+        print(f"  Avg win:        +{avg_win:.1f} pips  |  Avg loss: -{avg_loss:.1f} pips")
+        print(f"  Avg R:R setup:  1:{avg_rr:.1f}")
+        print(f"  Avg hold:       {avg_bars:.0f} M5 bars ({avg_bars * 5:.0f} min)")
+
+        # Consecutive streaks
+        max_wins = max_losses = cur_wins = cur_losses = 0
+        for t in closed:
+            if t.outcome == "WIN":
+                cur_wins += 1
+                cur_losses = 0
+                max_wins = max(max_wins, cur_wins)
+            else:
+                cur_losses += 1
+                cur_wins = 0
+                max_losses = max(max_losses, cur_losses)
+        print(f"  Max streak:     {max_wins}W / {max_losses}L")
+
+    # ── By pair ──────────────────────────────────────────────────
+    print(f"\n  BY PAIR")
+    print(f"  {'─' * 50}")
+    print(f"  {'Pair':<12} {'Trades':>7} {'Closed':>7} {'Win%':>6} {'P&L':>10} {'PF':>7}")
+
+    for pair in SCAN_PAIRS:
+        display = PAIR_INFO.get(pair, {}).get("display", pair)
+        pt = [t for t in trades if t.pair == pair]
+        pc = [t for t in pt if t.outcome in ("WIN", "LOSS")]
+        pw = [t for t in pc if t.outcome == "WIN"]
+        if not pt:
+            continue
+        wr = (len(pw) / len(pc) * 100) if pc else 0
+        pnl = sum(t.pnl_pips for t in pc)
+        gp = sum(t.pnl_pips for t in pw) if pw else 0
+        gl = sum(abs(t.pnl_pips) for t in pc if t.outcome == "LOSS") or 1
+        pf = gp / gl if gl > 0 else 0
+        print(f"  {display:<12} {len(pt):>7} {len(pc):>7} {wr:>5.1f}% {pnl:>+9.1f}p {pf:>6.2f}")
+
+    # ── By timeframe ─────────────────────────────────────────────
+    print(f"\n  BY TIMEFRAME")
+    print(f"  {'─' * 50}")
+    print(f"  {'TF':<8} {'Trades':>7} {'Closed':>7} {'Win%':>6} {'P&L':>10} {'PF':>7}")
+
+    tfs = sorted(set(t.timeframe for t in trades))
+    for tf in tfs:
+        tt = [t for t in trades if t.timeframe == tf]
+        tc = [t for t in tt if t.outcome in ("WIN", "LOSS")]
+        tw = [t for t in tc if t.outcome == "WIN"]
+        if not tt:
+            continue
+        wr = (len(tw) / len(tc) * 100) if tc else 0
+        pnl = sum(t.pnl_pips for t in tc)
+        gp = sum(t.pnl_pips for t in tw) if tw else 0
+        gl = sum(abs(t.pnl_pips) for t in tc if t.outcome == "LOSS") or 1
+        pf = gp / gl if gl > 0 else 0
+        print(f"  {tf:<8} {len(tt):>7} {len(tc):>7} {wr:>5.1f}% {pnl:>+9.1f}p {pf:>6.2f}")
+
+    # ── By quality grade ─────────────────────────────────────────
+    print(f"\n  BY QUALITY GRADE")
+    print(f"  {'─' * 50}")
+    print(f"  {'Grade':<8} {'Trades':>7} {'Closed':>7} {'Win%':>6} {'P&L':>10} {'Avg RR':>7}")
+
+    for grade in ("A+", "A", "B", "C"):
+        gt = [t for t in trades if t.signal_quality == grade]
+        gc = [t for t in gt if t.outcome in ("WIN", "LOSS")]
+        gw = [t for t in gc if t.outcome == "WIN"]
+        if not gt:
+            continue
+        wr = (len(gw) / len(gc) * 100) if gc else 0
+        pnl = sum(t.pnl_pips for t in gc)
+        arr = np.mean([t.risk_reward for t in gt])
+        print(f"  {grade:<8} {len(gt):>7} {len(gc):>7} {wr:>5.1f}% {pnl:>+9.1f}p {arr:>6.1f}")
+
+    # ── By regime ────────────────────────────────────────────────
+    print(f"\n  BY MARKET REGIME")
+    print(f"  {'─' * 50}")
+    print(f"  {'Regime':<16} {'Trades':>7} {'Win%':>6} {'P&L':>10}")
+
+    for regime in sorted(set(t.regime for t in trades)):
+        rt = [t for t in trades if t.regime == regime]
+        rc = [t for t in rt if t.outcome in ("WIN", "LOSS")]
+        rw = [t for t in rc if t.outcome == "WIN"]
+        wr = (len(rw) / len(rc) * 100) if rc else 0
+        pnl = sum(t.pnl_pips for t in rc)
+        label = regime.replace("_", " ").title()
+        print(f"  {label:<16} {len(rt):>7} {wr:>5.1f}% {pnl:>+9.1f}p")
+
+    # ── Direction ────────────────────────────────────────────────
+    print(f"\n  BY DIRECTION")
+    print(f"  {'─' * 50}")
+    for d in ("BUY", "SELL"):
+        dt = [t for t in trades if t.direction == d]
+        dc = [t for t in dt if t.outcome in ("WIN", "LOSS")]
+        dw = [t for t in dc if t.outcome == "WIN"]
+        wr = (len(dw) / len(dc) * 100) if dc else 0
+        pnl = sum(t.pnl_pips for t in dc)
+        print(f"  {d:<8} {len(dt):>7} trades  {wr:>5.1f}% win  {pnl:>+9.1f}p")
+
+    print(f"\n{'=' * 90}\n")
+
+
+def save_trades_csv(trades: List[Trade], filename: str):
+    path = RESULTS_DIR / filename
+    with open(path, "w", newline="") as f:
+        writer = csv.writer(f)
+        writer.writerow([
+            "pair", "display", "timeframe", "entry_time", "direction",
+            "entry_price", "stop_loss", "take_profit_1", "risk_reward",
+            "risk_pips", "reward_pips", "signal_quality", "confidence",
+            "score", "regime", "outcome", "exit_price", "exit_time",
+            "pnl_pips", "bars_held",
+        ])
+        for t in trades:
+            writer.writerow([
+                t.pair, t.display, t.timeframe, t.entry_time, t.direction,
+                f"{t.entry_price:.5f}", f"{t.stop_loss:.5f}", f"{t.take_profit_1:.5f}",
+                f"{t.risk_reward:.2f}", f"{t.risk_pips:.1f}", f"{t.reward_pips:.1f}",
+                t.signal_quality, f"{t.confidence:.1f}", f"{t.score:.1f}",
+                t.regime, t.outcome, f"{t.exit_price:.5f}",
+                t.exit_time or "", f"{t.pnl_pips:.1f}", t.bars_held,
+            ])
+    print(f"Trades saved to {path}")
+
+
+# ══════════════════════════════════════════════════════════════════════
+#  MAIN
+# ══════════════════════════════════════════════════════════════════════
+
+def main():
+    parser = argparse.ArgumentParser(description="ForexBot2 signal engine evaluator")
+    parser.add_argument("--days", type=int, default=7, help="Evaluation window in days (default: 7)")
+    parser.add_argument("--tf", default=None, help="Single timeframe to evaluate (M15, H1)")
+    parser.add_argument("--no-cache", action="store_true", help="Re-download all data")
+    parser.add_argument("--max-outcome-bars", type=int, default=1000,
+                        help="Max M5 bars to scan for SL/TP hit (default: 1000 = ~83h)")
+    parser.add_argument("--pair", default=None,
+                        help="Single pair to evaluate (EURUSD, USDJPY, GBPUSD, XAUUSD)")
+    args = parser.parse_args()
+
+    eval_tfs = EVAL_TIMEFRAMES
+    if args.tf:
+        tf_map = {"M5": Timeframe.M5, "M15": Timeframe.M15, "H1": Timeframe.H1, "H4": Timeframe.H4}
+        if args.tf.upper() not in tf_map:
+            sys.exit(f"Invalid timeframe: {args.tf}. Choose from: {list(tf_map.keys())}")
+        eval_tfs = [tf_map[args.tf.upper()]]
+
+    pair_map = {"EURUSD": EURUSD, "USDJPY": USDJPY, "GBPUSD": GBPUSD, "XAUUSD": XAUUSD}
+    scan_pairs = SCAN_PAIRS
+    if args.pair:
+        key = args.pair.upper().replace("/", "")
+        if key not in pair_map:
+            sys.exit(f"Invalid pair: {args.pair}. Choose from: {list(pair_map.keys())}")
+        scan_pairs = [pair_map[key]]
+
+    DATA_DIR.mkdir(parents=True, exist_ok=True)
+    RESULTS_DIR.mkdir(parents=True, exist_ok=True)
+
+    # ── 1. Fetch data ────────────────────────────────────────────
+    lookback = args.days + 60
+    print(f"\n{'=' * 60}")
+    print(f"  Fetching {lookback} days of data for {len(scan_pairs)} pair(s)...")
+    print(f"{'=' * 60}")
+    t0 = time.time()
+    all_data = fetch_all_data(
+        scan_pairs, ALL_TFS_NEEDED,
+        lookback_days=lookback,
+        use_cache=not args.no_cache,
+    )
+    print(f"  Data fetched in {time.time() - t0:.1f}s")
+
+    # ── 2. Walk-forward evaluation (multi-core) ──────────────────
+    print(f"\n{'=' * 60}")
+    n_workers = min(os.cpu_count() or 4, 16)
+    print(f"  Walk-forward evaluation: {args.days} days, TFs: {[tf.value for tf in eval_tfs]}")
+    print(f"  Using {n_workers} CPU cores")
+    print(f"{'=' * 60}")
+    t1 = time.time()
+
+    jobs = []
+    for pair in scan_pairs:
+        pair_data = all_data.get(pair, {})
+        if not pair_data:
+            display = PAIR_INFO.get(pair, {}).get("display", pair)
+            print(f"  {display}: no data, skipping")
+            continue
+        for tf in eval_tfs:
+            jobs.append((pair, tf, pair_data, args.days))
+
+    all_trades: List[Trade] = []
+    with ProcessPoolExecutor(max_workers=n_workers) as pool:
+        futures = {pool.submit(_walk_forward_worker, job): job for job in jobs}
+        for future in as_completed(futures):
+            pair, tf, display, trades = future.result()
+            print(f"  {display} {tf.value}: {len(trades)} trade setups")
+            all_trades.extend(trades)
+
+    print(f"\n  Walk-forward complete in {time.time() - t1:.1f}s")
+    print(f"  Total trade setups: {len(all_trades)}")
+
+    # ── 3. Outcome evaluation (GPU) ──────────────────────────────
+    m5_data = {pair: all_data.get(pair, {}).get(Timeframe.M5) for pair in scan_pairs}
+    m5_data = {k: v for k, v in m5_data.items() if v is not None}
+
+    t2 = time.time()
+    all_trades = evaluate_outcomes(all_trades, m5_data, max_bars=args.max_outcome_bars)
+    print(f"  Outcomes evaluated in {time.time() - t2:.3f}s")
+
+    # ── 4. Report ────────────────────────────────────────────────
+    print_report(all_trades, args.days)
+
+    ts = datetime.now().strftime("%Y%m%d_%H%M%S")
+    save_trades_csv(all_trades, f"eval_{args.days}d_{ts}.csv")
+
+
+if __name__ == "__main__":
+    main()

config.py

@@ -1,8 +1,6 @@
 """
-Forex Signal Intelligence Platform - Configuration
-===================================================
-Central configuration for all signal analysis parameters.
-Tuned by market regime and validated against historical performance.
+ForexBot2 — Configuration
+==========================================
 """
 
 from dataclasses import dataclass, field
@@ -37,160 +35,160 @@ class MarketSession(Enum):
     OVERLAP_LONDON_NY = "london_ny_overlap"
 
 
-# ── Major Forex Pairs ──────────────────────────────────────────────
-MAJOR_PAIRS = [
-    "EURUSD=X", "GBPUSD=X", "USDJPY=X", "USDCHF=X",
-    "AUDUSD=X", "USDCAD=X", "NZDUSD=X",
-]
+# ── Supported pairs ──────────────────────────────────────────────
+USDJPY = "USDJPY=X"
+EURUSD = "EURUSD=X"
+GBPUSD = "GBPUSD=X"
+XAUUSD = "XAU/USD"
 
-CROSS_PAIRS = [
-    "EURGBP=X", "EURJPY=X", "GBPJPY=X", "AUDJPY=X",
-    "EURCHF=X", "CADJPY=X",
-]
-
-# Yahoo Finance symbol mapping for display
-SYMBOL_DISPLAY = {
-    "EURUSD=X": "EUR/USD", "GBPUSD=X": "GBP/USD",
-    "USDJPY=X": "USD/JPY", "USDCHF=X": "USD/CHF",
-    "AUDUSD=X": "AUD/USD", "USDCAD=X": "USD/CAD",
-    "NZDUSD=X": "NZD/USD", "EURGBP=X": "EUR/GBP",
-    "EURJPY=X": "EUR/JPY", "GBPJPY=X": "GBP/JPY",
-    "AUDJPY=X": "AUD/JPY", "EURCHF=X": "EUR/CHF",
-    "CADJPY=X": "CAD/JPY",
+PAIR_INFO = {
+    USDJPY: {"display": "USD/JPY", "pip_value": 0.01, "is_jpy_pair": True},
+    EURUSD: {"display": "EUR/USD", "pip_value": 0.0001, "is_jpy_pair": False},
+    GBPUSD: {"display": "GBP/USD", "pip_value": 0.0001, "is_jpy_pair": False},
+    XAUUSD: {"display": "XAU/USD", "pip_value": 0.10, "is_jpy_pair": False},
 }
 
+# Defaults (kept for backward compat)
+DISPLAY = "USD/JPY"
+PIP_VALUE = 0.01
+
 
 @dataclass
 class TechnicalConfig:
-    """Technical indicator parameters - carefully tuned for Forex."""
-    # EMA periods (fast/medium/slow for trend detection)
+    """Technical indicator parameters."""
     ema_fast: int = 9
     ema_medium: int = 21
     ema_slow: int = 50
     ema_trend: int = 200
 
-    # RSI
     rsi_period: int = 14
     rsi_overbought: float = 70.0
     rsi_oversold: float = 30.0
     rsi_neutral_upper: float = 60.0
     rsi_neutral_lower: float = 40.0
 
-    # MACD (12, 26, 9 is standard)
     macd_fast: int = 12
     macd_slow: int = 26
     macd_signal: int = 9
 
-    # Bollinger Bands
     bb_period: int = 20
     bb_std: float = 2.0
 
-    # ATR for volatility measurement
     atr_period: int = 14
 
-    # Stochastic
-    stoch_k: int = 14
-    stoch_d: int = 3
-    stoch_overbought: float = 80.0
-    stoch_oversold: float = 20.0
-
-    # ADX for trend strength
     adx_period: int = 14
     adx_strong_trend: float = 25.0
     adx_very_strong: float = 50.0
 
-    # Ichimoku
-    ichimoku_tenkan: int = 9
-    ichimoku_kijun: int = 26
-    ichimoku_senkou_b: int = 52
-
-    # Volume profile
-    volume_ma_period: int = 20
-
-    # Pivot points lookback
-    pivot_lookback: int = 1
-
-    # Divergence detection — how many bars back to scan for swing points
     divergence_lookback: int = 14
+    swing_sr_lookback: int = 50
+
+    disabled_indicators: List[str] = field(default_factory=lambda: ["divergence"])
+
+    ema_mode: str = "4ema"          # "4ema" (9/21/50/200) or "2ema" (50/200)
+    rsi_mode: str = "classic"       # "classic" or "regime_aware"
+    macd_as_pillar: bool = False    # promote MACD histogram to scored signal
+
+    regime_weights_override: Dict[str, Dict[str, float]] = field(default_factory=dict)
 
 
 @dataclass
 class SignalConfig:
-    """Signal scoring and filtering parameters."""
-    # Minimum confidence to emit a signal (0-100)
+    """Signal scoring & filtering."""
     min_confidence: float = 55.0
 
-    # Weight allocation for composite score
     weights: Dict[str, float] = field(default_factory=lambda: {
-        "trend": 0.25,          # EMA alignment, ADX
-        "momentum": 0.20,       # RSI, Stochastic, MACD
-        "volatility": 0.15,     # BB position, ATR regime
-        "price_action": 0.15,   # Candlestick patterns, S/R
-        "multi_timeframe": 0.15,# Higher TF confirmation
-        "sentiment": 0.10,      # News/calendar events
+        "ema_trend": 1.0,
+        "rsi": 1.5,
+        "bollinger": 1.5,
+        "candlestick": 1.0,
+        "divergence": 1.5,
     })
 
-    # Multi-timeframe confirmation requirements
-    mtf_agreement_threshold: int = 2  # out of 3 timeframes must agree
-
-    # Signal cooldown (prevent over-signalling same direction)
+    mtf_agreement_threshold: int = 2  # of 3 timeframes
     cooldown_bars: int = 5
 
-    # Risk management defaults
-    default_risk_pct: float = 1.0  # risk 1% per trade
-    default_rr_ratio: float = 2.0  # 1:2 risk-reward minimum
+    default_risk_pct: float = 1.0
+    default_rr_ratio: float = 2.0
+    atr_sl_multiplier: float = 1.5
+    min_rr_ratio: float = 1.2
 
 
 @dataclass
 class NewsConfig:
-    """Economic calendar and sentiment configuration."""
-    # High-impact event currencies
-    tracked_currencies: List[str] = field(default_factory=lambda: [
-        "USD", "EUR", "GBP", "JPY", "CHF", "AUD", "CAD", "NZD"
-    ])
-
-    # Event impact levels to monitor
-    min_impact: str = "medium"  # "low", "medium", "high"
-
-    # Pre-event buffer (minutes before high-impact event = no new signals)
+    """Economic calendar & sentiment."""
+    tracked_currencies: List[str] = field(default_factory=lambda: ["USD", "JPY", "EUR", "GBP", "XAU"])
+    min_impact: str = "medium"
     pre_event_buffer_minutes: int = 30
-
-    # Post-event volatility window (minutes after event = cautious signals)
     post_event_buffer_minutes: int = 15
 
-    # News sentiment sources
     rss_feeds: List[str] = field(default_factory=lambda: [
         "https://www.forexlive.com/feed/",
         "https://www.fxstreet.com/rss",
     ])
 
 
+@dataclass
+class XauStrategyConfig:
+    """XAU/USD oversold-bounce strategy (BUY only).
+    Backtested: 79.7% WR, PF 2.06 over 10 months (192 trades).
+    Entry: M15 RSI<30 + price above EMA200 on M15, H1, H4.
+    Exit: TP=0.7×ATR, SL=2.0×ATR, max hold 8 bars (2h).
+    """
+    enabled: bool = True
+    rsi_entry_threshold: float = 38.0  # Wilder RSI scale; equivalent to standard RSI ~30
+    tp_atr_multiplier: float = 0.7
+    sl_atr_multiplier: float = 2.0
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [11, 14, 15, 20])
+
+
+@dataclass
+class FilterConfig:
+    """Regime and session filters for live signals."""
+    regime_filter: bool = False
+    regime_filter_timeframe: Timeframe = Timeframe.H4
+    regime_filter_min_adx: float = 25.0
+
+    session_filter: bool = True
+    session_start_hour: int = 7
+    session_end_hour: int = 16
+    blocked_hours_utc: List[int] = field(default_factory=lambda: [0, 1, 2, 5, 6, 8, 12, 13, 14, 15, 16, 17, 21, 22, 23])
+
+    sell_filter: bool = True
+    sell_allowed_pairs: List[str] = field(default_factory=lambda: ["XAU/USD"])
+
+
 @dataclass
 class PlatformConfig:
-    """Overall platform configuration."""
+    """Top-level configuration."""
     technical: TechnicalConfig = field(default_factory=TechnicalConfig)
     signals: SignalConfig = field(default_factory=SignalConfig)
     news: NewsConfig = field(default_factory=NewsConfig)
+    backtest: FilterConfig = field(default_factory=FilterConfig)
+    xau_strategy: XauStrategyConfig = field(default_factory=XauStrategyConfig)
 
-    # Default pairs to monitor
-    watch_pairs: List[str] = field(default_factory=lambda: MAJOR_PAIRS)
+    symbol: str = USDJPY
+    display_name: str = DISPLAY
+    pip_value: float = PIP_VALUE
 
-    # Analysis timeframes (primary + confirmation)
     primary_timeframe: Timeframe = Timeframe.H1
     confirmation_timeframes: List[Timeframe] = field(
         default_factory=lambda: [Timeframe.M15, Timeframe.H4]
     )
 
-    # Data source
-    data_lookback_days: int = 60  # enough for 200 EMA on H1
+    data_lookback_days: int = 60
 
-    # Twelve Data API key (overridden by TWELVE_DATA_API_KEY env var)
     twelve_data_api_key: str = "f563b22ffdfd4924955decbe1a8c8701"
 
-    # Dashboard refresh interval (seconds)
-    refresh_interval: int = 300  # 5 minutes
+    refresh_interval: int = 300
+
+    scan_pairs: List[str] = field(
+        default_factory=lambda: [EURUSD, GBPUSD, XAUUSD]
+    )
+    scan_timeframes: List[Timeframe] = field(
+        default_factory=lambda: [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4]
+    )
+    candle_buffer_seconds: int = 10
+    ws_cooldown_seconds: int = 300
 
-    # Logging
     log_level: str = "INFO"
-    log_signals_to_file: bool = True
-    signal_log_path: str = "signals.log"

data_provider.py

@@ -1,23 +1,19 @@
 """
-Forex Signal Intelligence Platform - Data Provider
-====================================================
-Retrieves and validates market data from Twelve Data API.
-Falls back to synthetic GBM data if the API is unavailable.
+ForexBot2 — Data Provider (Twelve Data SDK)
+==========================================================
+Fetches and validates OHLCV via the official `twelvedata` Python SDK.
 """
 
-import hashlib
 import logging
 import os
 import time
 from collections import deque
-from datetime import datetime, timedelta, timezone
-from typing import Dict, List, Optional
+from typing import Dict, Optional
 
-import numpy as np
 import pandas as pd
-import requests
+from twelvedata import TDClient
 
-from config import PlatformConfig, Timeframe, SYMBOL_DISPLAY
+from config import PlatformConfig, Timeframe, USDJPY, PAIR_INFO
 
 logger = logging.getLogger(__name__)
 
@@ -28,14 +24,7 @@ class DataIntegrityError(Exception):
 
 
 class ForexDataProvider:
-    """
-    Forex data provider backed by Twelve Data API.
-
-    Twelve Data has native 4h candles, eliminating the H1→H4 resampling
-    hack needed for Yahoo Finance. Free tier: 800 credits/day, 8 req/min.
-    """
-
-    TD_BASE_URL = "https://api.twelvedata.com"
+    """Twelve Data SDK client + on-disk parquet cache."""
 
     TD_INTERVALS = {
         Timeframe.M1:  "1min",
@@ -48,20 +37,19 @@ class ForexDataProvider:
         Timeframe.W1:  "1week",
     }
 
-    # Approximate bars per day for each timeframe (used to size outputsize)
     BARS_PER_DAY = {
-        Timeframe.M1:  1440,
-        Timeframe.M5:  288,
-        Timeframe.M15: 96,
-        Timeframe.M30: 48,
-        Timeframe.H1:  24,
-        Timeframe.H4:  6,
-        Timeframe.D1:  1,
-        Timeframe.W1:  0.143,
+        Timeframe.M1: 1440, Timeframe.M5: 288, Timeframe.M15: 96,
+        Timeframe.M30: 48, Timeframe.H1: 24, Timeframe.H4: 6,
+        Timeframe.D1: 1, Timeframe.W1: 0.143,
     }
 
-    # Free plan: 8 API credits per minute (each symbol = 1 credit)
-    RATE_LIMIT = 8
+    CANDLE_SECONDS = {
+        Timeframe.M1: 60, Timeframe.M5: 300, Timeframe.M15: 900,
+        Timeframe.M30: 1800, Timeframe.H1: 3600, Timeframe.H4: 14400,
+        Timeframe.D1: 86400, Timeframe.W1: 604800,
+    }
+
+    RATE_LIMIT = 55
     RATE_WINDOW = 60.0
 
     def __init__(self, config: PlatformConfig):
@@ -72,94 +60,97 @@ class ForexDataProvider:
         )
         self._cache: Dict[str, pd.DataFrame] = {}
         self._cache_timestamps: Dict[str, float] = {}
-        self._request_times: deque = deque()  # timestamps of recent API credits used
-        self._session = requests.Session()
-        self._session.headers.update({
-            "User-Agent": "Mozilla/5.0 (compatible; ForexIntel/1.0)"
-        })
+        self._request_times: deque = deque()
+        self._td: Optional[TDClient] = (
+            TDClient(apikey=self._api_key) if self._api_key else None
+        )
 
     @staticmethod
     def _to_td_symbol(symbol: str) -> str:
-        """Convert internal symbol (EURUSD=X) to Twelve Data format (EUR/USD)."""
+        if "/" in symbol:
+            return symbol
         clean = symbol.replace("=X", "")
         return f"{clean[:3]}/{clean[3:]}"
 
+    # ── Public API ─────────────────────────────────────────────────
+
     def fetch_ohlcv(
         self,
-        symbol: str,
-        timeframe: Timeframe,
+        symbol: str = USDJPY,
+        timeframe: Timeframe = Timeframe.H1,
         lookback_days: Optional[int] = None,
     ) -> pd.DataFrame:
-        """
-        Fetch OHLCV data for a Forex pair with full validation.
-
-        Returns a DataFrame with columns: Open, High, Low, Close, Volume
-        Index is a timezone-aware DatetimeIndex in UTC.
-        """
+        """Fetch OHLCV with in-memory caching for live mode."""
         if lookback_days is None:
             lookback_days = self.config.data_lookback_days
 
         cache_key = f"{symbol}_{timeframe.value}"
         now = time.time()
 
+        cache_ttl = self.CANDLE_SECONDS.get(timeframe, self.config.refresh_interval)
         if (
             cache_key in self._cache_timestamps
-            and (now - self._cache_timestamps[cache_key]) < self.config.refresh_interval
+            and (now - self._cache_timestamps[cache_key]) < cache_ttl
             and cache_key in self._cache
         ):
             cached = self._cache.get(cache_key)
             if cached is not None and not cached.empty:
-                return cached
+                if self._is_data_fresh(cached, symbol, timeframe):
+                    return cached
+                logger.warning(f"{symbol} {timeframe.value}: cached data is stale, refetching")
+                del self._cache[cache_key]
+                del self._cache_timestamps[cache_key]
 
-        df = self._fetch_twelve_data(symbol, timeframe, lookback_days)
+        outputsize = max(
+            300,
+            min(int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20, 5000),
+        )
+        df = self._fetch_twelve_data(symbol, timeframe, outputsize)
         df = self._validate_and_clean(df, symbol, timeframe)
 
+        if not self._is_data_fresh(df, symbol, timeframe):
+            logger.warning(
+                f"STALE DATA from API: {symbol} {timeframe.value} — "
+                f"last bar {df.index[-1]} is too old. Signal quality may be degraded."
+            )
+
         self._cache[cache_key] = df
         self._cache_timestamps[cache_key] = now
         return df
 
-    def prefetch_batch(
-        self,
-        symbols: List[str],
-        timeframes: List[Timeframe],
-        lookback_days: Optional[int] = None,
-    ) -> None:
-        """
-        Batch-fetch all symbols for each timeframe in a single API call per TF.
+    def _is_data_fresh(self, df: pd.DataFrame, symbol: str, timeframe: Timeframe) -> bool:
+        """Check if the last candle is within acceptable age for this timeframe."""
+        if df is None or df.empty:
+            return False
 
-        Twelve Data accepts comma-separated symbols, so 7 pairs × 3 timeframes
-        = 3 requests instead of 21 — well within the free-tier 8 req/min limit.
-        Results are stored in the local cache; subsequent fetch_ohlcv calls
-        for the same symbol/TF will be served from cache.
-        """
-        if lookback_days is None:
-            lookback_days = self.config.data_lookback_days
+        now = pd.Timestamp.now(tz="UTC")
 
-        for tf in timeframes:
-            try:
-                self._batch_fetch_timeframe(symbols, tf, lookback_days)
-                time.sleep(1.0)  # brief pause between TF requests
-            except Exception as e:
-                logger.warning(f"Batch prefetch failed for {tf.value}: {e}")
+        if now.weekday() >= 5:
+            return True
 
-    def fetch_multiple(
-        self,
-        symbols: List[str],
-        timeframe: Timeframe,
-        lookback_days: Optional[int] = None,
-    ) -> Dict[str, pd.DataFrame]:
-        """Fetch data for multiple symbols, returning dict of DataFrames."""
-        results = {}
-        for symbol in symbols:
-            try:
-                df = self.fetch_ohlcv(symbol, timeframe, lookback_days)
-                results[symbol] = df
-            except Exception as e:
-                logger.warning(f"Failed to fetch {symbol}: {e}")
-        return results
+        last_bar = df.index[-1]
+        candle_seconds = self.CANDLE_SECONDS.get(timeframe, 3600)
+        max_age_seconds = candle_seconds * 3
+        age_seconds = (now - last_bar).total_seconds()
 
-    def get_latest_price(self, symbol: str) -> Optional[float]:
-        """Return the most recent close price for a symbol."""
+        if age_seconds > max_age_seconds:
+            logger.warning(
+                f"{symbol} {timeframe.value}: last bar {last_bar.strftime('%Y-%m-%d %H:%M')} "
+                f"is {age_seconds / 60:.0f}m old (limit: {max_age_seconds / 60:.0f}m)"
+            )
+            return False
+
+        return True
+
+    def clear_cache(self) -> int:
+        """Wipe all cached data. Returns number of entries cleared."""
+        count = len(self._cache)
+        self._cache.clear()
+        self._cache_timestamps.clear()
+        logger.info(f"Cache cleared: {count} entries wiped")
+        return count
+
+    def get_latest_price(self, symbol: str = USDJPY) -> Optional[float]:
         try:
             df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
             if df is not None and not df.empty:
@@ -168,47 +159,26 @@ class ForexDataProvider:
             pass
         return None
 
-    def get_spread_estimate(self, symbol: str) -> float:
-        """Estimate typical spread from recent M5 high-low ranges."""
+    def get_spread_estimate(self, symbol: str = USDJPY) -> float:
         try:
             df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
             if df is not None and not df.empty:
                 return float((df["High"] - df["Low"]).median())
         except Exception:
             pass
+        info = PAIR_INFO.get(symbol, PAIR_INFO[USDJPY])
+        return info["pip_value"]
 
-        fallback_spreads = {
-            "EURUSD=X": 0.00010, "GBPUSD=X": 0.00015,
-            "USDJPY=X": 0.010,   "USDCHF=X": 0.00015,
-            "AUDUSD=X": 0.00015, "USDCAD=X": 0.00018,
-            "NZDUSD=X": 0.00020,
-        }
-        return fallback_spreads.get(symbol, 0.00020)
+    def get_pair_info(self, symbol: str = USDJPY) -> Dict:
+        info = PAIR_INFO.get(symbol, PAIR_INFO[USDJPY])
+        return {"symbol": symbol, **info}
 
-    def get_pair_info(self, symbol: str) -> Dict:
-        """Return display name and pip value for a pair."""
-        display = SYMBOL_DISPLAY.get(symbol, symbol)
-        is_jpy = "JPY" in symbol
-        return {
-            "symbol": symbol,
-            "display": display,
-            "pip_value": 0.01 if is_jpy else 0.0001,
-            "is_jpy_pair": is_jpy,
-        }
-
-    # ── Private Methods ────────────────────────────────────────────
+    # ── Private ────────────────────────────────────────────────────
 
     def _throttle(self, credits: int = 1) -> None:
-        """
-        Block until `credits` API credits are available.
-
-        Twelve Data free tier: 8 credits/minute. Each symbol in any request
-        (single or batch) consumes 1 credit. This method enforces the limit
-        regardless of whether requests are batched or individual.
-        """
+        """Pro tier: 55 credits / minute. Block until budget allows."""
         while True:
             now = time.time()
-            # Evict credits older than the rate window
             while self._request_times and now - self._request_times[0] >= self.RATE_WINDOW:
                 self._request_times.popleft()
 
@@ -218,276 +188,76 @@ class ForexDataProvider:
                     self._request_times.append(time.time())
                 return
 
-            # Wait until the oldest credit expires
             wait = (self._request_times[0] + self.RATE_WINDOW) - now + 0.2
             logger.info(
                 f"Rate limit: {len(self._request_times)}/{self.RATE_LIMIT} credits used "
-                f"— waiting {wait:.1f}s for next window..."
+                f"— waiting {wait:.1f}s..."
             )
             time.sleep(wait)
 
-    def _batch_fetch_timeframe(
-        self, symbols: List[str], timeframe: Timeframe, lookback_days: int
-    ) -> None:
-        """
-        Single API call fetching all symbols for one timeframe.
-        Twelve Data returns a dict keyed by symbol when multiple are requested.
-        Parsed DataFrames are validated and stored in the cache.
-        """
-        if not self._api_key:
-            return
-
-        td_symbols = [self._to_td_symbol(s) for s in symbols]
-        interval = self.TD_INTERVALS[timeframe]
-        outputsize = min(
-            int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20, 5000
-        )
-        outputsize = max(outputsize, 300)
-
-        # Each symbol in a batch request costs 1 credit
-        self._throttle(credits=len(symbols))
-
-        resp = self._session.get(
-            f"{self.TD_BASE_URL}/time_series",
-            params={
-                "symbol": ",".join(td_symbols),
-                "interval": interval,
-                "outputsize": outputsize,
-                "apikey": self._api_key,
-                "timezone": "UTC",
-                "format": "json",
-            },
-            timeout=30,
-        )
-        resp.raise_for_status()
-        data = resp.json()
-
-        # If the whole batch request returned a top-level error, raise it
-        if isinstance(data, dict) and data.get("status") == "error":
-            raise DataIntegrityError(
-                f"Batch API error ({timeframe.value}): {data.get('message', 'unknown')}"
-            )
-
-        # Batch response: {"EUR/USD": {...}, "GBP/USD": {...}, ...}
-        now = time.time()
-        for symbol, td_sym in zip(symbols, td_symbols):
-            try:
-                sym_data = data.get(td_sym, {})
-                if sym_data.get("status") == "error":
-                    logger.warning(
-                        f"Batch fetch error for {td_sym}: {sym_data.get('message')}"
-                    )
-                    continue
-
-                values = sym_data.get("values")
-                if not values:
-                    logger.warning(f"No values in batch response for {td_sym}")
-                    continue
-
-                records = [
-                    {
-                        "Datetime": pd.Timestamp(v["datetime"], tz="UTC"),
-                        "Open":   float(v["open"]),
-                        "High":   float(v["high"]),
-                        "Low":    float(v["low"]),
-                        "Close":  float(v["close"]),
-                        "Volume": float(v.get("volume") or 0),
-                    }
-                    for v in reversed(values)
-                ]
-                df = pd.DataFrame(records).set_index("Datetime")
-                df.index.name = "Datetime"
-                df = self._validate_and_clean(df, symbol, timeframe)
-
-                cache_key = f"{symbol}_{timeframe.value}"
-                self._cache[cache_key] = df
-                self._cache_timestamps[cache_key] = now
-                logger.debug(f"Batch cached {len(df)} bars for {td_sym} @ {interval}")
-
-            except Exception as e:
-                logger.warning(f"Failed to parse batch data for {td_sym}: {e}")
-
     def _fetch_twelve_data(
-        self, symbol: str, timeframe: Timeframe, lookback_days: int
+        self,
+        symbol: str,
+        timeframe: Timeframe,
+        outputsize: int,
+        start_date: Optional[str] = None,
+        end_date: Optional[str] = None,
     ) -> pd.DataFrame:
-        """
-        Fetch OHLCV from Twelve Data REST API.
-        Falls back to synthetic GBM data if API is unavailable or key is missing.
-        """
-        if not self._api_key:
-            logger.warning("No Twelve Data API key — using synthetic data. "
-                           "Set TWELVE_DATA_API_KEY env var or config.twelve_data_api_key.")
-            return self._generate_synthetic_data(symbol, timeframe, lookback_days)
+        if self._td is None:
+            raise DataIntegrityError(
+                "No Twelve Data API key. Set TWELVE_DATA_API_KEY env var "
+                "or config.twelve_data_api_key."
+            )
 
         td_symbol = self._to_td_symbol(symbol)
         interval = self.TD_INTERVALS[timeframe]
-        outputsize = min(
-            int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20,
-            5000,
-        )
-        outputsize = max(outputsize, 300)
 
         self._throttle(credits=1)
 
+        kwargs = {
+            "symbol": td_symbol,
+            "interval": interval,
+            "outputsize": min(outputsize, 5000),
+            "timezone": "UTC",
+        }
+        if start_date:
+            kwargs["start_date"] = start_date
+        if end_date:
+            kwargs["end_date"] = end_date
+
         try:
-            resp = self._session.get(
-                f"{self.TD_BASE_URL}/time_series",
-                params={
-                    "symbol": td_symbol,
-                    "interval": interval,
-                    "outputsize": outputsize,
-                    "apikey": self._api_key,
-                    "timezone": "UTC",
-                    "format": "json",
-                },
-                timeout=15,
-            )
-            resp.raise_for_status()
-            data = resp.json()
-
-            if data.get("status") == "error":
-                raise DataIntegrityError(
-                    f"Twelve Data API error for {td_symbol}: {data.get('message', 'unknown')}"
-                )
-
-            values = data.get("values")
-            if not values:
-                raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
-
-            # Values arrive newest-first; reverse to chronological order
-            records = [
-                {
-                    "Datetime": pd.Timestamp(v["datetime"], tz="UTC"),
-                    "Open":   float(v["open"]),
-                    "High":   float(v["high"]),
-                    "Low":    float(v["low"]),
-                    "Close":  float(v["close"]),
-                    "Volume": float(v.get("volume") or 0),
-                }
-                for v in reversed(values)
-            ]
-
-            df = pd.DataFrame(records).set_index("Datetime")
-            df.index.name = "Datetime"
-            logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
-            return df
-
-        except DataIntegrityError:
-            raise
+            df = self._td.time_series(**kwargs).as_pandas()
         except Exception as e:
-            logger.info(f"Twelve Data fetch failed for {symbol} ({e}), using synthetic data.")
-            return self._generate_synthetic_data(symbol, timeframe, lookback_days)
+            raise DataIntegrityError(
+                f"Twelve Data fetch failed for {td_symbol}: {e}"
+            ) from e
 
-    def _generate_synthetic_data(
-        self, symbol: str, timeframe: Timeframe, lookback_days: int
-    ) -> pd.DataFrame:
-        """
-        Generate realistic synthetic Forex OHLCV using GBM with mean-reversion.
-        Used as a fallback when live data is unavailable.
-        """
-        pair_params = {
-            "EURUSD=X": {"price": 1.1350, "vol": 0.07, "drift": -0.001},
-            "GBPUSD=X": {"price": 1.3200, "vol": 0.08, "drift":  0.002},
-            "USDJPY=X": {"price": 144.50, "vol": 0.09, "drift":  0.003},
-            "USDCHF=X": {"price": 0.8850, "vol": 0.07, "drift": -0.001},
-            "AUDUSD=X": {"price": 0.6550, "vol": 0.10, "drift": -0.002},
-            "USDCAD=X": {"price": 1.3650, "vol": 0.07, "drift":  0.001},
-            "NZDUSD=X": {"price": 0.6100, "vol": 0.10, "drift": -0.001},
-            "EURGBP=X": {"price": 0.8590, "vol": 0.06, "drift":  0.000},
-            "EURJPY=X": {"price": 163.90, "vol": 0.09, "drift":  0.002},
-            "GBPJPY=X": {"price": 190.70, "vol": 0.11, "drift":  0.003},
-            "AUDJPY=X": {"price": 94.60,  "vol": 0.11, "drift":  0.001},
-            "EURCHF=X": {"price": 0.9380, "vol": 0.05, "drift":  0.000},
-            "CADJPY=X": {"price": 105.80, "vol": 0.09, "drift":  0.001},
-        }
+        if df is None or df.empty:
+            raise DataIntegrityError(f"Empty response from Twelve Data for {td_symbol}")
 
-        params = pair_params.get(symbol, {"price": 1.0000, "vol": 0.08, "drift": 0.000})
-        base_price = params["price"]
-        annual_vol = params["vol"]
-        annual_drift = params["drift"]
+        # SDK returns lowercase columns; engine expects capitalised.
+        df = df.rename(columns={
+            "open": "Open", "high": "High", "low": "Low",
+            "close": "Close", "volume": "Volume",
+        })
+        for col in ("Open", "High", "Low", "Close", "Volume"):
+            if col in df.columns:
+                df[col] = pd.to_numeric(df[col], errors="coerce")
+        if "Volume" not in df.columns:
+            df["Volume"] = 0.0
 
-        tf_minutes = {
-            Timeframe.M1: 1, Timeframe.M5: 5, Timeframe.M15: 15,
-            Timeframe.M30: 30, Timeframe.H1: 60, Timeframe.H4: 240,
-            Timeframe.D1: 1440, Timeframe.W1: 10080,
-        }
-        bar_minutes = tf_minutes.get(timeframe, 60)
-        bars_per_year = (365.25 * 24 * 60) / bar_minutes
-        dt = 1.0 / bars_per_year
-
-        n_bars = min(int(lookback_days * 24 * 60 / bar_minutes), 2000)
-        n_bars = max(n_bars, 300)
-
-        seed_base = int(hashlib.md5(symbol.encode()).hexdigest()[:8], 16)
-        time_component = int(time.time()) // 3600
-        np.random.seed((seed_base + time_component) % (2**31))
-
-        bar_vol = annual_vol * np.sqrt(dt)
-        bar_drift = annual_drift * dt
-
-        regime_len = np.random.randint(20, 80)
-        trend_strength = np.random.normal(0, 0.3)
-
-        price = base_price
-        prices = [price]
-
-        for i in range(1, n_bars):
-            if i % regime_len == 0:
-                regime_len = np.random.randint(20, 80)
-                trend_strength = np.random.normal(0, 0.3)
-
-            mr_force = -0.001 * (price - base_price) / base_price
-            ret = bar_drift + trend_strength * dt + mr_force + bar_vol * np.random.randn()
-
-            if np.random.random() < 0.02:
-                ret *= np.random.uniform(2.0, 4.0)
-
-            price *= (1 + ret)
-            prices.append(price)
-
-        prices = np.array(prices)
-        opens = np.roll(prices, 1)
-        opens[0] = prices[0]
-
-        typical_range = bar_vol * base_price * 1.5
-        highs = np.maximum(prices, opens) + np.abs(np.random.normal(0, typical_range * 0.6, n_bars))
-        lows  = np.minimum(prices, opens) - np.abs(np.random.normal(0, typical_range * 0.6, n_bars))
-        highs = np.maximum(highs, np.maximum(opens, prices))
-        lows  = np.minimum(lows,  np.minimum(opens, prices))
-
-        volume = np.random.lognormal(mean=10, sigma=0.5, size=n_bars).astype(int)
-
-        freq_map = {
-            Timeframe.M1: "1min",  Timeframe.M5: "5min",
-            Timeframe.M15: "15min", Timeframe.M30: "30min",
-            Timeframe.H1: "1h",    Timeframe.H4: "4h",
-            Timeframe.D1: "D",     Timeframe.W1: "W",
-        }
-        freq = freq_map.get(timeframe, "1h")
-        end_time = pd.Timestamp.now(tz="UTC").floor(f"{bar_minutes}min")
-        index = pd.date_range(end=end_time, periods=n_bars, freq=freq, tz="UTC")
-
-        df = pd.DataFrame({
-            "Open": opens, "High": highs, "Low": lows,
-            "Close": prices, "Volume": volume,
-        }, index=index)
+        # SDK returns newest-first; engine + indicators want oldest-first.
+        df = df.sort_index()
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
         df.index.name = "Datetime"
 
-        logger.info(
-            f"Generated {n_bars} synthetic {timeframe.value} bars for "
-            f"{SYMBOL_DISPLAY.get(symbol, symbol)} @ {prices[-1]:.5f}"
-        )
+        logger.debug(f"Fetched {len(df)} bars for {td_symbol} @ {interval}")
         return df
 
     def _validate_and_clean(
         self, df: pd.DataFrame, symbol: str, timeframe: Timeframe
     ) -> pd.DataFrame:
-        """
-        Validate and clean OHLCV data.
-
-        Steps: drop all-NaN rows → ffill small gaps → remove future timestamps
-        → fix OHLC inconsistencies → remove >10-sigma outliers → sort & deduplicate.
-        """
         if df is None or df.empty:
             raise DataIntegrityError(f"Empty dataset for {symbol}")
 
@@ -527,7 +297,6 @@ class ForexDataProvider:
 
         min_required = max(
             self.config.technical.ema_trend,
-            self.config.technical.ichimoku_senkou_b,
             200,
         ) + 10
 

engine.py

@@ -83,22 +83,27 @@ class TechnicalEngine:
         )
 
         # ── Compute all indicators ─────────────────────────────────
-        self._compute_ema_signals(df, result)
-        self._compute_rsi(df, result)
+        disabled = set(self.config.disabled_indicators or [])
+
+        if "ema_trend"   not in disabled: self._compute_ema_signals(df, result)
+        if "rsi"         not in disabled: self._compute_rsi(df, result)
+        if "bollinger"   not in disabled: self._compute_bollinger(df, result)
+        if "adx"         not in disabled: self._compute_adx(df, result)
+        if "atr"         not in disabled: self._compute_atr(df, result)
+
         self._compute_macd(df, result)
-        self._compute_bollinger(df, result)
-        self._compute_stochastic(df, result)
-        self._compute_adx(df, result)
-        self._compute_atr(df, result)
-        self._compute_ichimoku(df, result)
-        self._detect_divergences(df, result)
-        self._detect_candlestick_patterns(df, result)
-        self._compute_support_resistance(df, result)
-        self._compute_volume_analysis(df, result)
+
+        if "divergence"  not in disabled: self._detect_divergences(df, result)
+        if "structure"   not in disabled: self._compute_structure_sr(df, result)
+        if "candlestick" not in disabled: self._detect_candlestick_patterns(df, result)
 
         # ── Classify market regime ─────────────────────────────────
         self._classify_regime(df, result)
 
+        # ── Regime-aware RSI reinterpretation ──────────────────────
+        if self.config.rsi_mode == "regime_aware" and "rsi" not in disabled:
+            self._reinterpret_rsi_for_regime(result)
+
         # ── Compute composite signal ───────────────────────────────
         self._compute_composite(result)
 
@@ -109,7 +114,87 @@ class TechnicalEngine:
     # ══════════════════════════════════════════════════════════════
 
     def _compute_ema_signals(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """EMA trend alignment analysis."""
+        """EMA trend alignment — supports 4-EMA and 2-EMA modes."""
+        if self.config.ema_mode == "2ema":
+            self._compute_ema_2(df, result)
+        else:
+            self._compute_ema_4(df, result)
+
+    def _compute_ema_2(self, df: pd.DataFrame, result: TechnicalAnalysis):
+        """EMA 50/200 trend alignment — cleaner, less noise on H1."""
+        c = self.config
+        close = df["Close"]
+
+        ema50 = close.ewm(span=c.ema_slow, adjust=False).mean()
+        ema200 = close.ewm(span=c.ema_trend, adjust=False).mean() if len(close) >= c.ema_trend else None
+        ema21 = close.ewm(span=c.ema_medium, adjust=False).mean()
+
+        latest = close.iloc[-1]
+        s = ema50.iloc[-1]
+        m = ema21.iloc[-1]
+        t = ema200.iloc[-1] if ema200 is not None and len(ema200) >= c.ema_trend else None
+
+        if t is None:
+            result.indicators["ema_trend"] = IndicatorSignal(
+                name="EMA Trend", value=0, signal=SignalStrength.NEUTRAL,
+                confidence=30, description="EMA200 data insufficient", weight=1.0,
+            )
+            result.key_levels["ema_medium"] = float(m)
+            result.key_levels["ema_slow"] = float(s)
+            return
+
+        # 3-level alignment: price vs EMA50, EMA50 vs EMA200
+        if latest > s > t:
+            signal = SignalStrength.STRONG_BUY
+            conf = 85
+            desc = "Full bullish: price > EMA50 > EMA200"
+        elif latest > s and s < t:
+            signal = SignalStrength.BUY
+            conf = 60
+            desc = "Emerging uptrend: price > EMA50, EMA50 < EMA200"
+        elif latest < s < t:
+            signal = SignalStrength.STRONG_SELL
+            conf = 85
+            desc = "Full bearish: price < EMA50 < EMA200"
+        elif latest < s and s > t:
+            signal = SignalStrength.SELL
+            conf = 60
+            desc = "Emerging downtrend: price < EMA50, EMA50 > EMA200"
+        else:
+            signal = SignalStrength.NEUTRAL
+            conf = 40
+            desc = "EMA alignment mixed"
+
+        # EMA50/200 crossover detection (recent 10 bars)
+        if len(ema50) >= 10 and ema200 is not None:
+            for lookback in range(2, min(11, len(ema50))):
+                prev_s = ema50.iloc[-lookback]
+                prev_t = ema200.iloc[-lookback]
+                if prev_s <= prev_t and s > t:
+                    desc += " | Golden cross (50/200)"
+                    conf = min(conf + 10, 95)
+                    break
+                elif prev_s >= prev_t and s < t:
+                    desc += " | Death cross (50/200)"
+                    conf = min(conf + 10, 95)
+                    break
+
+        normalized = 1.0 if signal == SignalStrength.STRONG_BUY else (
+            0.5 if signal == SignalStrength.BUY else (
+                -1.0 if signal == SignalStrength.STRONG_SELL else (
+                    -0.5 if signal == SignalStrength.SELL else 0.0)))
+
+        result.indicators["ema_trend"] = IndicatorSignal(
+            name="EMA Trend", value=normalized,
+            signal=signal, confidence=conf,
+            description=desc, weight=1.0,
+        )
+        result.key_levels["ema_trend"] = float(t)
+        result.key_levels["ema_medium"] = float(m)
+        result.key_levels["ema_slow"] = float(s)
+
+    def _compute_ema_4(self, df: pd.DataFrame, result: TechnicalAnalysis):
+        """Original 4-EMA (9/21/50/200) trend alignment analysis."""
         c = self.config
         close = df["Close"]
 
@@ -122,8 +207,6 @@ class TechnicalEngine:
         f, m, s = ema_fast.iloc[-1], ema_medium.iloc[-1], ema_slow.iloc[-1]
         t = ema_trend.iloc[-1] if ema_trend is not None and len(ema_trend) >= c.ema_trend else None
 
-        # Bullish: price > fast > medium > slow > trend
-        # Bearish: price < fast < medium < slow < trend
         alignment_score = 0
         checks = [(latest, f), (f, m), (m, s)]
         if t is not None:
@@ -154,7 +237,6 @@ class TechnicalEngine:
             signal = SignalStrength.NEUTRAL
             conf = 40
 
-        # Check for EMA crossover (recent)
         crossover_desc = ""
         if len(ema_fast) >= 3:
             prev_f, prev_m = ema_fast.iloc[-2], ema_medium.iloc[-2]
@@ -169,8 +251,13 @@ class TechnicalEngine:
         result.indicators["ema_trend"] = IndicatorSignal(
             name="EMA Trend", value=normalized,
             signal=signal, confidence=conf,
-            description=desc, weight=1.2,
+            description=desc, weight=1.0,
         )
+        if t is not None:
+            result.key_levels["ema_trend"] = float(t)
+        result.key_levels["ema_fast"] = float(f)
+        result.key_levels["ema_medium"] = float(m)
+        result.key_levels["ema_slow"] = float(s)
 
     def _compute_rsi(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """RSI with trend-adjusted interpretation."""
@@ -181,10 +268,10 @@ class TechnicalEngine:
         gain = delta.where(delta > 0, 0.0)
         loss = (-delta).where(delta < 0, 0.0)
 
-        avg_gain = gain.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period).mean()
-        avg_loss = loss.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period).mean()
+        avg_gain = gain.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period, adjust=False).mean()
+        avg_loss = loss.ewm(alpha=1/c.rsi_period, min_periods=c.rsi_period, adjust=False).mean()
 
-        rs = avg_gain / avg_loss.replace(0, np.nan)
+        rs = avg_gain / avg_loss.replace(0, 1e-10)
         rsi = 100 - (100 / (1 + rs))
         rsi_val = rsi.iloc[-1] if not rsi.empty else 50.0
 
@@ -215,14 +302,14 @@ class TechnicalEngine:
 
         result.indicators["rsi"] = IndicatorSignal(
             name="RSI", value=rsi_val, signal=signal,
-            confidence=conf, description=desc, weight=1.0,
+            confidence=conf, description=desc, weight=1.5,
         )
 
         # Attach RSI series for divergence detection (dynamic attribute, filtered by _prefix)
         result._rsi_series = rsi  # type: ignore[attr-defined]
 
     def _compute_macd(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """MACD with histogram momentum analysis."""
+        """MACD computation — always runs for divergence; optionally a signal pillar."""
         c = self.config
         close = df["Close"]
 
@@ -232,71 +319,72 @@ class TechnicalEngine:
         signal_line = macd_line.ewm(span=c.macd_signal, adjust=False).mean()
         histogram = macd_line - signal_line
 
-        macd_val = macd_line.iloc[-1]
-        signal_val = signal_line.iloc[-1]
-        hist_val = histogram.iloc[-1]
+        result._macd_histogram = histogram  # type: ignore[attr-defined]
 
-        # Check for crossover
-        crossover = False
-        crossover_type = ""
-        if len(macd_line) >= 2:
-            prev_macd = macd_line.iloc[-2]
-            prev_signal = signal_line.iloc[-2]
-            if prev_macd <= prev_signal and macd_val > signal_val:
-                crossover = True
-                crossover_type = "bullish"
-            elif prev_macd >= prev_signal and macd_val < signal_val:
-                crossover = True
-                crossover_type = "bearish"
+        if not c.macd_as_pillar:
+            return
 
-        # Histogram momentum (is it expanding or contracting?)
-        hist_momentum = ""
-        if len(histogram) >= 3:
-            h1, h2, h3 = histogram.iloc[-3], histogram.iloc[-2], histogram.iloc[-1]
-            if h3 > h2 > h1 and h3 > 0:
-                hist_momentum = "accelerating bullish"
-            elif h3 < h2 < h1 and h3 < 0:
-                hist_momentum = "accelerating bearish"
-            elif abs(h3) < abs(h2) and h3 * h2 > 0:
-                hist_momentum = "decelerating"
+        if len(histogram) < 3:
+            return
 
-        # Signal
-        if crossover and crossover_type == "bullish":
-            signal = SignalStrength.STRONG_BUY
-            conf = 75
-            desc = f"MACD bullish crossover | Hist: {hist_val:.6f}"
-        elif crossover and crossover_type == "bearish":
-            signal = SignalStrength.STRONG_SELL
-            conf = 75
-            desc = f"MACD bearish crossover | Hist: {hist_val:.6f}"
-        elif macd_val > signal_val and hist_val > 0:
-            signal = SignalStrength.BUY
-            conf = 55
-            desc = f"MACD above signal, positive histogram | {hist_momentum}"
-        elif macd_val < signal_val and hist_val < 0:
-            signal = SignalStrength.SELL
-            conf = 55
-            desc = f"MACD below signal, negative histogram | {hist_momentum}"
+        hist_now = histogram.iloc[-1]
+        hist_prev = histogram.iloc[-2]
+        hist_prev2 = histogram.iloc[-3]
+
+        if pd.isna(hist_now) or pd.isna(hist_prev):
+            return
+
+        # Histogram zero-line crossover
+        crossed_up = hist_prev <= 0 < hist_now
+        crossed_down = hist_prev >= 0 > hist_now
+        # Momentum acceleration
+        accel_up = hist_now > 0 and hist_now > hist_prev > 0
+        accel_down = hist_now < 0 and hist_now < hist_prev < 0
+        # Momentum fading
+        fading_up = hist_now > 0 and hist_now < hist_prev
+        fading_down = hist_now < 0 and hist_now > hist_prev
+
+        # Normalize histogram relative to recent range for confidence
+        hist_abs = histogram.abs()
+        if len(hist_abs) >= 20:
+            hist_pct = (abs(hist_now) / hist_abs.iloc[-20:].mean()) if hist_abs.iloc[-20:].mean() > 0 else 1.0
         else:
+            hist_pct = 1.0
+
+        if crossed_up or accel_up:
+            signal = SignalStrength.STRONG_BUY if accel_up else SignalStrength.BUY
+            conf = min(80, 55 + hist_pct * 10)
+            desc = "MACD histogram" + (" accelerating ↑" if accel_up else " crossed above zero")
+        elif crossed_down or accel_down:
+            signal = SignalStrength.STRONG_SELL if accel_down else SignalStrength.SELL
+            conf = min(80, 55 + hist_pct * 10)
+            desc = "MACD histogram" + (" accelerating ↓" if accel_down else " crossed below zero")
+        elif fading_up:
             signal = SignalStrength.NEUTRAL
             conf = 35
-            desc = f"MACD mixed | Hist: {hist_val:.6f}"
+            desc = f"MACD histogram positive but fading"
+        elif fading_down:
+            signal = SignalStrength.NEUTRAL
+            conf = 35
+            desc = f"MACD histogram negative but fading"
+        else:
+            signal = SignalStrength.NEUTRAL
+            conf = 30
+            desc = f"MACD histogram flat"
 
         result.indicators["macd"] = IndicatorSignal(
-            name="MACD", value=hist_val, signal=signal,
-            confidence=conf, description=desc, weight=1.1,
+            name="MACD", value=float(hist_now),
+            signal=signal, confidence=conf,
+            description=desc, weight=1.5,
         )
 
-        # Attach histogram for divergence detection
-        result._macd_histogram = histogram  # type: ignore[attr-defined]
-
     def _compute_bollinger(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """Bollinger Bands position and squeeze detection."""
         c = self.config
         close = df["Close"]
 
         sma = close.rolling(c.bb_period).mean()
-        std = close.rolling(c.bb_period).std()
+        std = close.rolling(c.bb_period).std(ddof=0)
         upper = sma + c.bb_std * std
         lower = sma - c.bb_std * std
 
@@ -351,74 +439,13 @@ class TechnicalEngine:
 
         result.indicators["bollinger"] = IndicatorSignal(
             name="Bollinger Bands", value=bb_position, signal=signal,
-            confidence=conf, description=desc, weight=0.9,
+            confidence=conf, description=desc, weight=1.5,
         )
 
         result.key_levels["bb_upper"] = upper_val
         result.key_levels["bb_lower"] = lower_val
         result.key_levels["bb_mid"] = sma_val
 
-    def _compute_stochastic(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Stochastic oscillator with crossover signals."""
-        c = self.config
-        high = df["High"]
-        low = df["Low"]
-        close = df["Close"]
-
-        lowest_low = low.rolling(c.stoch_k).min()
-        highest_high = high.rolling(c.stoch_k).max()
-
-        denom = highest_high - lowest_low
-        denom = denom.replace(0, np.nan)
-
-        k_line = 100 * (close - lowest_low) / denom
-        d_line = k_line.rolling(c.stoch_d).mean()
-
-        k_val = k_line.iloc[-1] if not k_line.empty else 50.0
-        d_val = d_line.iloc[-1] if not d_line.empty else 50.0
-
-        if pd.isna(k_val):
-            k_val = 50.0
-        if pd.isna(d_val):
-            d_val = 50.0
-
-        # Crossover detection
-        crossover = ""
-        if len(k_line) >= 2 and len(d_line) >= 2:
-            prev_k = k_line.iloc[-2]
-            prev_d = d_line.iloc[-2]
-            if not pd.isna(prev_k) and not pd.isna(prev_d):
-                if prev_k <= prev_d and k_val > d_val:
-                    crossover = " | Bullish %K/%D cross"
-                elif prev_k >= prev_d and k_val < d_val:
-                    crossover = " | Bearish %K/%D cross"
-
-        if k_val >= c.stoch_overbought and d_val >= c.stoch_overbought:
-            signal = SignalStrength.SELL
-            conf = 60
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f} — Overbought{crossover}"
-        elif k_val <= c.stoch_oversold and d_val <= c.stoch_oversold:
-            signal = SignalStrength.BUY
-            conf = 60
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f} — Oversold{crossover}"
-        elif crossover and "Bullish" in crossover:
-            signal = SignalStrength.BUY
-            conf = 55
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f}{crossover}"
-        elif crossover and "Bearish" in crossover:
-            signal = SignalStrength.SELL
-            conf = 55
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f}{crossover}"
-        else:
-            signal = SignalStrength.NEUTRAL
-            conf = 30
-            desc = f"Stoch %K={k_val:.0f} %D={d_val:.0f} — Neutral"
-
-        result.indicators["stochastic"] = IndicatorSignal(
-            name="Stochastic", value=k_val, signal=signal,
-            confidence=conf, description=desc, weight=0.8,
-        )
-
     def _compute_adx(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """ADX trend strength measurement."""
         c = self.config
@@ -439,12 +466,12 @@ class TechnicalEngine:
         plus_dm = plus_dm.where((plus_dm > minus_dm) & (plus_dm > 0), 0.0)
         minus_dm = minus_dm.where((minus_dm > plus_dm) & (minus_dm > 0), 0.0)
 
-        atr = tr.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean()
-        plus_di = 100 * (plus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean() / atr)
-        minus_di = 100 * (minus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean() / atr)
+        atr = tr.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean()
+        plus_di = 100 * (plus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean() / atr)
+        minus_di = 100 * (minus_dm.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean() / atr)
 
         dx = 100 * (plus_di - minus_di).abs() / (plus_di + minus_di).replace(0, np.nan)
-        adx = dx.ewm(alpha=1/c.adx_period, min_periods=c.adx_period).mean()
+        adx = dx.ewm(alpha=1/c.adx_period, min_periods=c.adx_period, adjust=False).mean()
 
         adx_val = adx.iloc[-1] if not adx.empty and not pd.isna(adx.iloc[-1]) else 0.0
         plus_di_last = plus_di.iloc[-1] if not plus_di.empty else np.nan
@@ -476,7 +503,7 @@ class TechnicalEngine:
 
         result.indicators["adx"] = IndicatorSignal(
             name="ADX", value=adx_val, signal=signal,
-            confidence=conf, description=desc, weight=1.0,
+            confidence=conf, description=desc, weight=0.5,
         )
 
     def _compute_atr(self, df: pd.DataFrame, result: TechnicalAnalysis):
@@ -489,7 +516,7 @@ class TechnicalEngine:
         tr3 = (low - close.shift()).abs()
         tr = pd.concat([tr1, tr2, tr3], axis=1).max(axis=1)
 
-        atr = tr.ewm(alpha=1/c.atr_period, min_periods=c.atr_period).mean()
+        atr = tr.ewm(alpha=1/c.atr_period, min_periods=c.atr_period, adjust=False).mean()
         atr_val = atr.iloc[-1] if not atr.empty else 0.0
         if pd.isna(atr_val):
             atr_val = 0.0
@@ -523,91 +550,6 @@ class TechnicalEngine:
             weight=0.0,  # ATR is context, not a directional signal
         )
 
-    def _compute_ichimoku(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Ichimoku Cloud analysis."""
-        c = self.config
-        high, low, close = df["High"], df["Low"], df["Close"]
-
-        if len(df) < c.ichimoku_senkou_b + 26:
-            return
-
-        # Tenkan-sen (Conversion Line)
-        tenkan = (high.rolling(c.ichimoku_tenkan).max() +
-                  low.rolling(c.ichimoku_tenkan).min()) / 2
-
-        # Kijun-sen (Base Line)
-        kijun = (high.rolling(c.ichimoku_kijun).max() +
-                 low.rolling(c.ichimoku_kijun).min()) / 2
-
-        latest = close.iloc[-1]
-        tenkan_val = tenkan.iloc[-1]
-        kijun_val = kijun.iloc[-1]
-
-        # Current (unshifted) cloud
-        sa = (tenkan.iloc[-1] + kijun.iloc[-1]) / 2
-        sb_raw = (high.rolling(c.ichimoku_senkou_b).max() +
-                  low.rolling(c.ichimoku_senkou_b).min()) / 2
-        sb = sb_raw.iloc[-1]
-
-        if pd.isna(sa) or pd.isna(sb) or pd.isna(tenkan_val) or pd.isna(kijun_val):
-            return
-
-        cloud_top = max(sa, sb)
-        cloud_bottom = min(sa, sb)
-        cloud_is_green = sa > sb
-
-        signals = []
-        score = 0
-
-        if latest > cloud_top:
-            signals.append("Price ABOVE cloud")
-            score += 2
-        elif latest < cloud_bottom:
-            signals.append("Price BELOW cloud")
-            score -= 2
-        else:
-            signals.append("Price IN cloud (indecision)")
-
-        if tenkan_val > kijun_val:
-            signals.append("TK cross bullish")
-            score += 1
-        elif tenkan_val < kijun_val:
-            signals.append("TK cross bearish")
-            score -= 1
-
-        if cloud_is_green:
-            signals.append("Green cloud")
-            score += 1
-        else:
-            signals.append("Red cloud")
-            score -= 1
-
-        if score >= 3:
-            signal = SignalStrength.STRONG_BUY
-            conf = 75
-        elif score >= 1:
-            signal = SignalStrength.BUY
-            conf = 55
-        elif score <= -3:
-            signal = SignalStrength.STRONG_SELL
-            conf = 75
-        elif score <= -1:
-            signal = SignalStrength.SELL
-            conf = 55
-        else:
-            signal = SignalStrength.NEUTRAL
-            conf = 35
-
-        result.indicators["ichimoku"] = IndicatorSignal(
-            name="Ichimoku", value=float(score), signal=signal,
-            confidence=conf,
-            description=f"Ichimoku [{score}/4]: {' | '.join(signals)}",
-            weight=1.1,
-        )
-
-        result.key_levels["ichimoku_cloud_top"] = cloud_top
-        result.key_levels["ichimoku_cloud_bottom"] = cloud_bottom
-
     def _detect_divergences(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """
         Detect RSI and MACD divergences (powerful reversal signals).
@@ -665,11 +607,39 @@ class TechnicalEngine:
                 name="Divergence", value=len(divergences),
                 signal=signal, confidence=70,
                 description=" | ".join(divergences),
-                weight=1.5,  # divergences are strong reversal signals
+                weight=1.5,
             )
             # Note: divergences are NOT added to pattern_signals to avoid
             # duplicate reason collection (the indicator entry covers them).
 
+    def _compute_structure_sr(self, df: pd.DataFrame, result: TechnicalAnalysis):
+        """Detect swing high/low structure levels for S/R and trade setup."""
+        lookback = min(self.config.swing_sr_lookback, len(df) - 1)
+        if lookback < 10:
+            return
+
+        price = df["Close"].iloc[-1]
+        high_window = df["High"].iloc[-lookback:]
+        low_window = df["Low"].iloc[-lookback:]
+
+        swing_highs = self._find_swing_points(high_window, mode="high", window=3)
+        swing_lows = self._find_swing_points(low_window, mode="low", window=3)
+
+        highs_above = sorted([v for _, v in swing_highs if v > price])
+        highs_below = sorted([v for _, v in swing_highs if v <= price], reverse=True)
+        lows_below = sorted([v for _, v in swing_lows if v < price], reverse=True)
+        lows_above = sorted([v for _, v in swing_lows if v >= price])
+
+        for i, val in enumerate(highs_above[:3], 1):
+            result.key_levels[f"swing_high_{i}"] = float(val)
+        for i, val in enumerate(lows_below[:3], 1):
+            result.key_levels[f"swing_low_{i}"] = float(val)
+
+        if not highs_above and highs_below:
+            result.key_levels["swing_high_1"] = float(highs_below[0])
+        if not lows_below and lows_above:
+            result.key_levels["swing_low_1"] = float(lows_above[0])
+
     def _detect_candlestick_patterns(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """Detect significant candlestick patterns on recent bars."""
         if len(df) < 5:
@@ -731,94 +701,122 @@ class TechnicalEngine:
             else:
                 signal = SignalStrength.NEUTRAL
 
+            conf = 50
+            atr_val = result.key_levels.get("atr", 0)
+            latest = c
+            proximity = atr_val * 0.5 if atr_val > 0 else 0
+
+            if proximity > 0:
+                swing_lows = sorted(
+                    [v for k, v in result.key_levels.items() if k.startswith("swing_low_")]
+                )
+                swing_highs = sorted(
+                    [v for k, v in result.key_levels.items() if k.startswith("swing_high_")]
+                )
+                near_support = any(abs(latest - sl) <= proximity for sl in swing_lows)
+                near_resistance = any(abs(latest - sh) <= proximity for sh in swing_highs)
+
+                if bullish > bearish and near_support:
+                    conf += 20
+                    patterns.append("(at support)")
+                elif bearish > bullish and near_resistance:
+                    conf += 20
+                    patterns.append("(at resistance)")
+
             result.indicators["candlestick"] = IndicatorSignal(
                 name="Candlestick", value=float(bullish - bearish),
-                signal=signal, confidence=50,
+                signal=signal, confidence=conf,
                 description=" | ".join(patterns),
-                weight=0.7,
+                weight=1.0,
             )
             result.pattern_signals.extend(patterns)
 
-    def _compute_support_resistance(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Calculate pivot points and key S/R levels."""
-        if len(df) < 2:
-            return
-
-        # Use previous bar for pivot calculation
-        prev_h = df["High"].iloc[-2]
-        prev_l = df["Low"].iloc[-2]
-        prev_c = df["Close"].iloc[-2]
-
-        pivot = (prev_h + prev_l + prev_c) / 3
-        r1 = 2 * pivot - prev_l
-        s1 = 2 * pivot - prev_h
-        r2 = pivot + (prev_h - prev_l)
-        s2 = pivot - (prev_h - prev_l)
-        r3 = prev_h + 2 * (pivot - prev_l)
-        s3 = prev_l - 2 * (prev_h - pivot)
-
-        result.key_levels.update({
-            "pivot": pivot,
-            "r1": r1, "r2": r2, "r3": r3,
-            "s1": s1, "s2": s2, "s3": s3,
-        })
-
-        # Signal based on price position relative to pivot
-        latest = df["Close"].iloc[-1]
-        if latest > r1:
-            desc = f"Price above R1 ({r1:.5f}), next target R2 ({r2:.5f})"
-        elif latest > pivot:
-            desc = f"Price above Pivot ({pivot:.5f}), testing R1 ({r1:.5f})"
-        elif latest > s1:
-            desc = f"Price below Pivot ({pivot:.5f}), support at S1 ({s1:.5f})"
-        else:
-            desc = f"Price below S1 ({s1:.5f}), support at S2 ({s2:.5f})"
-
-        result.indicators["pivot"] = IndicatorSignal(
-            name="Pivot Points", value=pivot,
-            signal=SignalStrength.BUY if latest > pivot else SignalStrength.SELL,
-            confidence=45, description=desc, weight=0.6,
-        )
-
-    def _compute_volume_analysis(self, df: pd.DataFrame, result: TechnicalAnalysis):
-        """Volume analysis (limited on Forex but still useful as a filter)."""
-        c = self.config
-        vol = df.get("Volume")
-        if vol is None or vol.sum() == 0:
-            return
-
-        vol_ma = vol.rolling(c.volume_ma_period).mean()
-        current_vol = vol.iloc[-1]
-        avg_vol = vol_ma.iloc[-1]
-
-        if pd.isna(avg_vol) or avg_vol == 0:
-            return
-
-        vol_ratio = current_vol / avg_vol
-
-        if vol_ratio > 2.0:
-            desc = f"Volume {vol_ratio:.1f}x average — HIGH ACTIVITY"
-            conf = 55
-        elif vol_ratio > 1.3:
-            desc = f"Volume {vol_ratio:.1f}x average — above normal"
-            conf = 45
-        elif vol_ratio < 0.5:
-            desc = f"Volume {vol_ratio:.1f}x average — LOW (signals less reliable)"
-            conf = 25
-        else:
-            desc = f"Volume {vol_ratio:.1f}x average — normal"
-            conf = 35
-
-        result.indicators["volume"] = IndicatorSignal(
-            name="Volume", value=vol_ratio,
-            signal=SignalStrength.NEUTRAL,
-            confidence=conf, description=desc, weight=0.0,
-        )
-
     # ══════════════════════════════════════════════════════════════
     #  MARKET REGIME & COMPOSITE SCORING
     # ══════════════════════════════════════════════════════════════
 
+    def _reinterpret_rsi_for_regime(self, result: TechnicalAnalysis):
+        """Reinterpret RSI signal based on market regime — called after regime classification."""
+        rsi_ind = result.indicators.get("rsi")
+        if not rsi_ind:
+            return
+
+        rsi_val = rsi_ind.value
+        regime = result.market_regime
+
+        if regime in ("trending_up", "trending"):
+            if 38 <= rsi_val <= 55:
+                signal = SignalStrength.STRONG_BUY
+                conf = 75
+                desc = f"RSI {rsi_val:.1f} — Pullback zone in uptrend (ideal entry)"
+            elif 30 <= rsi_val < 38:
+                signal = SignalStrength.BUY
+                conf = 60
+                desc = f"RSI {rsi_val:.1f} — Deep pullback in uptrend"
+            elif 55 < rsi_val <= 65:
+                signal = SignalStrength.BUY
+                conf = 50
+                desc = f"RSI {rsi_val:.1f} — Momentum ok in uptrend"
+            elif rsi_val > 65:
+                signal = SignalStrength.NEUTRAL
+                conf = 30
+                desc = f"RSI {rsi_val:.1f} — Extended in uptrend, wait for pullback"
+            elif rsi_val < 30:
+                signal = SignalStrength.NEUTRAL
+                conf = 25
+                desc = f"RSI {rsi_val:.1f} — Oversold in uptrend, possible reversal"
+            else:
+                return
+        elif regime == "trending_down":
+            if 45 <= rsi_val <= 62:
+                signal = SignalStrength.STRONG_SELL
+                conf = 75
+                desc = f"RSI {rsi_val:.1f} — Bounce zone in downtrend (ideal short)"
+            elif 62 < rsi_val <= 70:
+                signal = SignalStrength.SELL
+                conf = 60
+                desc = f"RSI {rsi_val:.1f} — Deep bounce in downtrend"
+            elif 35 <= rsi_val < 45:
+                signal = SignalStrength.SELL
+                conf = 50
+                desc = f"RSI {rsi_val:.1f} — Momentum ok in downtrend"
+            elif rsi_val < 35:
+                signal = SignalStrength.NEUTRAL
+                conf = 30
+                desc = f"RSI {rsi_val:.1f} — Extended in downtrend, wait for bounce"
+            elif rsi_val > 70:
+                signal = SignalStrength.NEUTRAL
+                conf = 25
+                desc = f"RSI {rsi_val:.1f} — Overbought in downtrend, possible reversal"
+            else:
+                return
+        else:  # ranging
+            if rsi_val >= 70:
+                signal = SignalStrength.SELL
+                conf = 70
+                desc = f"RSI {rsi_val:.1f} — Overbought in range"
+            elif rsi_val <= 30:
+                signal = SignalStrength.BUY
+                conf = 70
+                desc = f"RSI {rsi_val:.1f} — Oversold in range"
+            elif rsi_val >= 60:
+                signal = SignalStrength.SELL
+                conf = 45
+                desc = f"RSI {rsi_val:.1f} — Upper range zone"
+            elif rsi_val <= 40:
+                signal = SignalStrength.BUY
+                conf = 45
+                desc = f"RSI {rsi_val:.1f} — Lower range zone"
+            else:
+                signal = SignalStrength.NEUTRAL
+                conf = 30
+                desc = f"RSI {rsi_val:.1f} — Mid-range, no edge"
+
+        result.indicators["rsi"] = IndicatorSignal(
+            name="RSI", value=rsi_val, signal=signal,
+            confidence=conf, description=desc, weight=rsi_ind.weight,
+        )
+
     def _classify_regime(self, df: pd.DataFrame, result: TechnicalAnalysis):
         """Classify the market regime (trending vs ranging)."""
         adx_sig = result.indicators.get("adx")
@@ -840,18 +838,29 @@ class TechnicalEngine:
 
         Weighted average of all indicator signals, adjusted for:
         - Market regime (trend indicators weighted more in trends)
-        - Divergence boost (divergences override in ranging markets)
         - Confidence scaling
         """
         total_score = 0.0
         total_weight = 0.0
         total_confidence = 0.0
 
-        regime_weights = {
-            "trending_up": {"ema_trend": 1.5, "adx": 1.3, "macd": 1.2, "rsi": 0.7, "bollinger": 0.6},
-            "trending_down": {"ema_trend": 1.5, "adx": 1.3, "macd": 1.2, "rsi": 0.7, "bollinger": 0.6},
-            "ranging": {"rsi": 1.4, "stochastic": 1.3, "bollinger": 1.4, "ema_trend": 0.6, "adx": 0.5},
-        }
+        # Use override weights if provided, else defaults
+        if self.config.regime_weights_override:
+            regime_weights = self.config.regime_weights_override
+        else:
+            _trending = {
+                "ema_trend": 2.0, "rsi": 1.5, "bollinger": 0.8,
+                "candlestick": 1.0, "divergence": 1.5,
+            }
+            regime_weights = {
+                "trending_up": _trending,
+                "trending_down": _trending,
+                "trending": _trending,
+                "ranging": {
+                    "ema_trend": 0.5, "rsi": 2.0, "bollinger": 2.0,
+                    "candlestick": 1.0, "divergence": 2.0,
+                },
+            }
 
         active_regime_weights = regime_weights.get(result.market_regime, {})
 
@@ -863,7 +872,7 @@ class TechnicalEngine:
             regime_mult = active_regime_weights.get(name, 1.0)
             weight *= regime_mult
 
-            signal_score = ind.signal.value  # -2 to +2
+            signal_score = ind.signal.value
             weighted_score = signal_score * weight * (ind.confidence / 100.0)
 
             total_score += weighted_score
@@ -871,23 +880,21 @@ class TechnicalEngine:
             total_confidence += ind.confidence * weight
 
         if total_weight > 0:
-            normalized_score = (total_score / total_weight) * 50  # scale to -100..+100
+            normalized_score = (total_score / total_weight) * 50
             avg_confidence = total_confidence / total_weight
         else:
             normalized_score = 0
             avg_confidence = 0
 
-        # Clamp
         normalized_score = max(-100, min(100, normalized_score))
 
-        # Determine composite signal
-        if normalized_score >= 40:
+        if normalized_score >= 45:
             result.composite_signal = SignalStrength.STRONG_BUY
-        elif normalized_score >= 15:
+        elif normalized_score >= 20:
             result.composite_signal = SignalStrength.BUY
-        elif normalized_score <= -40:
+        elif normalized_score <= -45:
             result.composite_signal = SignalStrength.STRONG_SELL
-        elif normalized_score <= -15:
+        elif normalized_score <= -20:
             result.composite_signal = SignalStrength.SELL
         else:
             result.composite_signal = SignalStrength.NEUTRAL

main.py

@@ -1,35 +1,27 @@
 #!/usr/bin/env python3
 """
-Forex Signal Intelligence Platform
-====================================
-Main entry point. Runs full analysis and outputs formatted signal reports.
-
+ForexBot2 — USD/JPY Signal Intelligence (single-pair)
+=======================================================
 Usage:
-    python main.py                          # Analyze all major pairs
-    python main.py --pairs EURUSD GBPUSD    # Specific pairs
-    python main.py --timeframe H4           # Different primary TF
-    python main.py --json                   # JSON output
-    python main.py --watch                  # Continuous monitoring mode
+    python main.py                              # Live signal on H1
+    python main.py --timeframe H4 --detail      # Detailed H4 signal
+    python main.py --json                       # JSON output
 """
 
 import argparse
 import json
 import logging
 import sys
-import time
 from datetime import datetime, timezone
-from typing import List
+from typing import Dict, List
 
-# Force UTF-8 output on Windows so box-drawing / Unicode chars render correctly
+# Force UTF-8 output on Windows so box-drawing chars render
 if hasattr(sys.stdout, "reconfigure"):
     sys.stdout.reconfigure(encoding="utf-8")
 if hasattr(sys.stderr, "reconfigure"):
     sys.stderr.reconfigure(encoding="utf-8")
 
-from config import (
-    PlatformConfig, Timeframe, SignalStrength,
-    MAJOR_PAIRS, CROSS_PAIRS,
-)
+from config import PlatformConfig, Timeframe, SignalStrength, USDJPY, DISPLAY
 from signal_aggregator import SignalAggregator, SignalReport
 
 
@@ -38,12 +30,10 @@ from signal_aggregator import SignalAggregator, SignalReport
 # ══════════════════════════════════════════════════════════════════
 
 class Colors:
-    """ANSI color codes for terminal output."""
     RESET = "\033[0m"
     BOLD = "\033[1m"
     DIM = "\033[2m"
     UNDERLINE = "\033[4m"
-
     RED = "\033[91m"
     GREEN = "\033[92m"
     YELLOW = "\033[93m"
@@ -53,224 +43,146 @@ class Colors:
     WHITE = "\033[97m"
     GRAY = "\033[90m"
 
-    BG_RED = "\033[41m"
-    BG_GREEN = "\033[42m"
-    BG_YELLOW = "\033[43m"
-    BG_BLUE = "\033[44m"
-    BG_MAGENTA = "\033[45m"
-    BG_CYAN = "\033[46m"
-
 
 def signal_color(signal: SignalStrength) -> str:
-    """Get color for a signal strength."""
     return {
-        SignalStrength.STRONG_BUY: Colors.GREEN + Colors.BOLD,
-        SignalStrength.BUY: Colors.GREEN,
-        SignalStrength.NEUTRAL: Colors.GRAY,
-        SignalStrength.SELL: Colors.RED,
+        SignalStrength.STRONG_BUY:  Colors.GREEN + Colors.BOLD,
+        SignalStrength.BUY:         Colors.GREEN,
+        SignalStrength.NEUTRAL:     Colors.GRAY,
+        SignalStrength.SELL:        Colors.RED,
         SignalStrength.STRONG_SELL: Colors.RED + Colors.BOLD,
     }.get(signal, Colors.RESET)
 
 
-def quality_color(quality: str) -> str:
-    """Get color for signal quality grade."""
+def quality_color(q: str) -> str:
     return {
         "A+": Colors.GREEN + Colors.BOLD,
-        "A": Colors.GREEN,
-        "B": Colors.CYAN,
-        "C": Colors.YELLOW,
-        "D": Colors.GRAY,
-        "F": Colors.DIM,
-    }.get(quality, Colors.RESET)
+        "A":  Colors.GREEN,
+        "B":  Colors.CYAN,
+        "C":  Colors.YELLOW,
+        "D":  Colors.GRAY,
+        "F":  Colors.DIM,
+    }.get(q, Colors.RESET)
 
 
-def score_bar(score: float, width: int = 30) -> str:
-    """Create a visual score bar from -100 to +100."""
-    normalized = (score + 100) / 200  # 0 to 1
-    filled = int(normalized * width)
-    center = width // 2
-
-    bar = ""
-    for i in range(width):
-        if i == center:
-            bar += "│"
-        elif (score >= 0 and center < i <= filled) or \
-             (score < 0 and filled <= i < center):
-            bar += "█"
-        else:
-            bar += "░"
-
-    if score >= 0:
-        color = Colors.GREEN if score > 20 else Colors.GRAY
-    else:
-        color = Colors.RED if score < -20 else Colors.GRAY
-
-    return f"{color}{bar}{Colors.RESET}"
-
-
-def print_header():
-    """Print the platform header."""
+def print_header(title: str = "USD/JPY SIGNAL INTELLIGENCE"):
     now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
-    width = 80
-
-    print(f"\n{Colors.CYAN}{'═' * width}{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ╔═══════════════════════════════════════════════════════╗{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ║   FOREX SIGNAL INTELLIGENCE PLATFORM                 ║{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ║   Multi-Layer Signal Analysis Engine                  ║{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.CYAN}  ╚═══════════════════════════════════════════════════════╝{Colors.RESET}")
-    print(f"{Colors.GRAY}  Scan Time: {now}{Colors.RESET}")
-    print(f"{Colors.CYAN}{'═' * width}{Colors.RESET}\n")
+    print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
+    print(f"{Colors.BOLD}{Colors.CYAN}  {title}{Colors.RESET}")
+    print(f"{Colors.GRAY}  Time: {now}{Colors.RESET}")
+    print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
 
 
-def print_summary_table(reports: List[SignalReport]):
-    """Print a summary table of all pairs."""
-    print(f"{Colors.BOLD}{Colors.WHITE}  ┌─────────────┬───────────┬──────────────────────────────────┬────────┬────────────┐{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.WHITE}  │ Pair        │ Signal    │ Score                            │ Grade  │ Confidence │{Colors.RESET}")
-    print(f"{Colors.BOLD}{Colors.WHITE}  ├─────────────┼───────────┼──────────────────────────────────┼────────┼────────────┤{Colors.RESET}")
+# ── Live signal renderer ───────────────────────────────────────────
 
-    for r in reports:
-        sig = r.final_signal.name.replace("_", " ")
-        sc = signal_color(r.final_signal)
-        qc = quality_color(r.signal_quality)
-        bar = score_bar(r.final_score, 30)
-
-        pair = f"{r.display_name:<11}"
-        signal_str = f"{sc}{sig:<9}{Colors.RESET}"
-        grade_str = f"{qc}{r.signal_quality:<6}{Colors.RESET}"
-        conf_str = f"{r.final_confidence:5.1f}%"
-
-        # Add warning indicator
-        warn = ""
-        if r.event_risk == "danger":
-            warn = f" {Colors.RED}⚠{Colors.RESET}"
-        elif r.event_risk == "caution":
-            warn = f" {Colors.YELLOW}△{Colors.RESET}"
-
-        print(f"  │ {pair} │ {signal_str} │ {bar} │ {grade_str} │ {conf_str:>8}{warn}  │")
-
-    print(f"{Colors.BOLD}{Colors.WHITE}  └─────────────┴───────────┴──────────────────────────────────┴────────┴────────────┘{Colors.RESET}")
-
-    # Legend
-    print(f"\n{Colors.GRAY}  Legend: "
-          f"{Colors.GREEN}██ Bullish{Colors.GRAY} | "
-          f"{Colors.RED}██ Bearish{Colors.GRAY} | "
-          f"│ Center (0) | "
-          f"{Colors.RED}⚠ Event Risk{Colors.GRAY} | "
-          f"{Colors.YELLOW}△ Caution{Colors.RESET}")
+_INDICATOR_PREFIXES = (
+    "EMA Trend", "RSI", "Bollinger Bands", "Divergence",
+    "Candlestick", "ADX", "ATR", "News", "Regime",
+)
 
 
-def print_detailed_report(report: SignalReport):
-    """Print detailed analysis for a single pair."""
+def _compact_reason(reason: str) -> str:
+    """Shorten a verbose reason string for the compact view."""
+    s = reason
+    if s.startswith("STRONG: "):
+        s = s[8:]
+    # Strip the leading "<Indicator>: " prefix — it's always an indicator name
+    for prefix in _INDICATOR_PREFIXES:
+        if s.startswith(prefix + ": "):
+            s = s[len(prefix) + 2:]
+            break
+    # Drop trailing detail after first ' | '
+    if " | " in s:
+        s = s.split(" | ", 1)[0]
+    return s.strip()
+
+
+def print_signal_report(report: SignalReport, detail: bool = False):
     sc = signal_color(report.final_signal)
     qc = quality_color(report.signal_quality)
     sig_name = report.final_signal.name.replace("_", " ")
 
-    print(f"\n{Colors.CYAN}{'─' * 80}{Colors.RESET}")
-    print(f"  {Colors.BOLD}{Colors.WHITE}{report.display_name}{Colors.RESET}"
-          f"  │  Price: {Colors.BOLD}{report.current_price:.5f}{Colors.RESET}"
-          f"  │  Regime: {Colors.CYAN}{report.market_regime}{Colors.RESET}"
-          f"  │  Vol: {report.volatility_regime}")
+    # ── 1. DECISION ────────────────────────────────────────────────
+    mtf_tag = (f"{Colors.GREEN}MTF✓{Colors.RESET}" if report.mtf_agreement
+               else f"{Colors.YELLOW}MTF✗{Colors.RESET}")
+    print(f"  {sc}▶ {sig_name}{Colors.RESET}  "
+          f"{qc}{report.signal_quality}{Colors.RESET} · "
+          f"{sc}{report.final_score:+.1f}{Colors.RESET} · "
+          f"{report.final_confidence:.0f}% · {mtf_tag}")
+    print(f"  {Colors.GRAY}{report.action_recommendation}{Colors.RESET}")
 
-    print(f"\n  {Colors.BOLD}SIGNAL:{Colors.RESET} {sc}{sig_name}{Colors.RESET}"
-          f"  │  Score: {sc}{report.final_score:+.1f}{Colors.RESET}"
-          f"  │  Grade: {qc}{report.signal_quality}{Colors.RESET}"
-          f"  │  Confidence: {report.final_confidence:.0f}%")
+    # ── 2. TRADE SETUP (if signal is actionable) ───────────────────
+    if report.trade_setup:
+        ts = report.trade_setup
+        dc = Colors.GREEN if ts.direction == "BUY" else Colors.RED
+        print(f"\n  {Colors.BOLD}Setup{Colors.RESET}  "
+              f"{dc}{ts.direction}{Colors.RESET} @ {Colors.BOLD}{ts.entry_price:.3f}{Colors.RESET}  "
+              f"→ SL {Colors.RED}{ts.stop_loss:.3f}{Colors.RESET} "
+              f"({ts.risk_pips:.1f}p)  "
+              f"→ TP {Colors.GREEN}{ts.take_profit_1:.3f}{Colors.RESET} "
+              f"({ts.reward_pips:.1f}p)  "
+              f"RR 1:{ts.risk_reward:.1f}")
 
-    print(f"  {Colors.BOLD}Action:{Colors.RESET} {report.action_recommendation}")
+    # ── 3. CONTEXT (price · regime · vol · MTF breakdown) ──────────
+    print(f"\n  {Colors.WHITE}{report.display_name}{Colors.RESET} "
+          f"{Colors.BOLD}{report.current_price:.3f}{Colors.RESET} · "
+          f"{Colors.CYAN}{report.market_regime}{Colors.RESET} · "
+          f"vol {report.volatility_regime}")
 
-    # Multi-timeframe breakdown
     if report.mtf_signals:
-        print(f"\n  {Colors.BOLD}Multi-Timeframe:{Colors.RESET}", end="")
-        mtf_agree = f" {Colors.GREEN}✓ ALIGNED{Colors.RESET}" if report.mtf_agreement else f" {Colors.YELLOW}✗ DIVERGENT{Colors.RESET}"
-        print(mtf_agree)
+        parts = []
         for tf, sig in report.mtf_signals.items():
             tf_sc = signal_color(sig)
-            tf_score = report.mtf_scores.get(tf, 0)
-            print(f"    {tf:>5}: {tf_sc}{sig.name.replace('_', ' '):<12}{Colors.RESET} ({tf_score:+.1f})")
+            sc_val = report.mtf_scores.get(tf, 0)
+            parts.append(f"{tf} {tf_sc}{sig.name.replace('_',' ')}{Colors.RESET} {sc_val:+.1f}")
+        print(f"  {Colors.GRAY}MTF{Colors.RESET}    " + "  ·  ".join(parts))
 
-    # Indicators detail
-    if report.primary_analysis:
-        print(f"\n  {Colors.BOLD}Indicators ({report.primary_analysis.timeframe}):{Colors.RESET}")
+    # ── 4. WHY — top 3 each side, compact ──────────────────────────
+    bulls = [_compact_reason(r) for r in report.bullish_reasons[:3]]
+    bears = [_compact_reason(r) for r in report.bearish_reasons[:3]]
+    if bulls or bears:
+        print()
+        rows = max(len(bulls), len(bears))
+        for i in range(rows):
+            left  = f"{Colors.GREEN}+{Colors.RESET} {bulls[i]}" if i < len(bulls) else ""
+            right = f"{Colors.RED}-{Colors.RESET} {bears[i]}" if i < len(bears) else ""
+            print(f"  {left:<60}  {right}")
+
+    # ── 5. WARNINGS / NEWS / EVENTS ────────────────────────────────
+    if report.warnings:
+        for w in report.warnings:
+            print(f"  {Colors.YELLOW}⚠{Colors.RESET}  {w}")
+    if report.event_description and report.event_risk != "clear":
+        print(f"  {Colors.YELLOW}📅{Colors.RESET} {report.event_description}")
+    if report.news_description and report.news_description != "Neutral news sentiment":
+        print(f"  {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
+
+    # ── 6. DETAIL (only with --detail) ─────────────────────────────
+    if detail and report.primary_analysis:
+        print(f"\n  {Colors.BOLD}Indicators ({report.primary_analysis.timeframe}){Colors.RESET}")
         for name, ind in report.primary_analysis.indicators.items():
             if name.startswith("_") or ind.weight == 0:
                 continue
             isc = signal_color(ind.signal)
-            ind_name = f"{ind.name:<18}"
-            print(f"    {ind_name} {isc}{ind.signal.name.replace('_', ' '):<12}{Colors.RESET}"
-                  f" ({ind.confidence:.0f}%) │ {Colors.GRAY}{ind.description}{Colors.RESET}")
+            print(f"    {ind.name:<16} {isc}{ind.signal.name.replace('_',' '):<11}{Colors.RESET} "
+                  f"{ind.confidence:>3.0f}% {Colors.GRAY}│ {ind.description}{Colors.RESET}")
 
-    # Bullish reasons
-    if report.bullish_reasons:
-        print(f"\n  {Colors.GREEN}▲ Bullish Factors:{Colors.RESET}")
-        for reason in report.bullish_reasons[:5]:
-            print(f"    {Colors.GREEN}+{Colors.RESET} {reason}")
-
-    # Bearish reasons
-    if report.bearish_reasons:
-        print(f"\n  {Colors.RED}▼ Bearish Factors:{Colors.RESET}")
-        for reason in report.bearish_reasons[:5]:
-            print(f"    {Colors.RED}-{Colors.RESET} {reason}")
-
-    # Warnings
-    if report.warnings:
-        print(f"\n  {Colors.YELLOW}⚠ Warnings:{Colors.RESET}")
-        for warning in report.warnings:
-            print(f"    {Colors.YELLOW}!{Colors.RESET} {warning}")
-
-    # News context
-    if report.news_description and report.news_description != "Neutral news sentiment":
-        print(f"\n  {Colors.MAGENTA}📰 News:{Colors.RESET} {report.news_description}")
-    if report.event_description and report.event_risk != "clear":
-        print(f"  {Colors.YELLOW}📅 Event:{Colors.RESET} {report.event_description}")
-
-    # Trade setup
-    if report.trade_setup:
-        ts = report.trade_setup
-        dir_color = Colors.GREEN if ts.direction == "BUY" else Colors.RED
-        print(f"\n  {Colors.BOLD}Trade Setup:{Colors.RESET}")
-        print(f"    Direction:     {dir_color}{ts.direction}{Colors.RESET}")
-        print(f"    Entry:         {ts.entry_price:.5f}")
-        print(f"    Stop Loss:     {Colors.RED}{ts.stop_loss:.5f}{Colors.RESET} ({ts.risk_pips:.1f} pips)")
-        print(f"    Take Profit 1: {Colors.GREEN}{ts.take_profit_1:.5f}{Colors.RESET} ({ts.reward_pips:.1f} pips)")
-        print(f"    Take Profit 2: {Colors.GREEN}{ts.take_profit_2:.5f}{Colors.RESET}")
-        print(f"    Risk:Reward:   1:{ts.risk_reward:.1f}")
-
-    # Key levels
-    if report.primary_analysis and report.primary_analysis.key_levels:
         levels = report.primary_analysis.key_levels
-        print(f"\n  {Colors.BOLD}Key Levels:{Colors.RESET}")
-        level_names = ["r3", "r2", "r1", "pivot", "s1", "s2", "s3"]
-        for lvl in level_names:
-            if lvl in levels:
-                val = levels[lvl]
-                if val > report.current_price:
-                    color = Colors.RED  # resistance
-                else:
-                    color = Colors.GREEN  # support
-                label = lvl.upper()
-                print(f"    {label:>6}: {color}{val:.5f}{Colors.RESET}")
-
-
-def print_event_calendar(reports: List[SignalReport]):
-    """Print upcoming economic events affecting watched pairs."""
-    events_shown = set()
-    has_events = False
-
-    for report in reports:
-        if report.event_risk != "clear" and report.event_description not in events_shown:
-            if not has_events:
-                print(f"\n{Colors.YELLOW}{Colors.BOLD}  📅 ECONOMIC CALENDAR ALERTS{Colors.RESET}")
-                has_events = True
-            events_shown.add(report.event_description)
-            risk_color = Colors.RED if report.event_risk == "danger" else Colors.YELLOW
-            print(f"    {risk_color}● {report.event_description}{Colors.RESET} → affects {report.display_name}")
-
-    if not has_events:
-        print(f"\n{Colors.GRAY}  📅 No high-impact events in the near term{Colors.RESET}")
+        if levels:
+            swing_keys = sorted(
+                [k for k in levels if k.startswith("swing_")],
+                key=lambda k: levels[k], reverse=True,
+            )
+            if swing_keys:
+                print(f"\n  {Colors.BOLD}Structure Levels{Colors.RESET}")
+                for k in swing_keys:
+                    val = levels[k]
+                    color = Colors.RED if val > report.current_price else Colors.GREEN
+                    label = k.replace("_", " ").title()
+                    print(f"    {label:>20}  {color}{val:.3f}{Colors.RESET}")
 
 
 def report_to_dict(report: SignalReport) -> dict:
-    """Convert a SignalReport to a JSON-serializable dict."""
     d = {
         "symbol": report.symbol,
         "display_name": report.display_name,
@@ -305,140 +217,94 @@ def report_to_dict(report: SignalReport) -> dict:
             "reward_pips": round(ts.reward_pips, 1),
         }
     if report.primary_analysis:
-        d["indicators"] = {}
-        for name, ind in report.primary_analysis.indicators.items():
-            if not name.startswith("_"):
-                d["indicators"][name] = {
-                    "signal": ind.signal.name,
-                    "confidence": round(ind.confidence, 1),
-                    "description": ind.description,
-                }
+        d["indicators"] = {
+            name: {
+                "signal": ind.signal.name,
+                "confidence": round(ind.confidence, 1),
+                "description": ind.description,
+            }
+            for name, ind in report.primary_analysis.indicators.items()
+            if not name.startswith("_")
+        }
         d["key_levels"] = {
             k: round(v, 5) for k, v in report.primary_analysis.key_levels.items()
         }
     return d
 
 
+
 # ══════════════════════════════════════════════════════════════════
-#  MAIN
+#  CLI
 # ══════════════════════════════════════════════════════════════════
 
 def parse_args():
     parser = argparse.ArgumentParser(
-        description="Forex Signal Intelligence Platform",
+        description="ForexBot2 — USD/JPY Signal Intelligence",
         formatter_class=argparse.RawDescriptionHelpFormatter,
         epilog="""
 Examples:
-  python main.py                        Analyze all major pairs (H1)
-  python main.py --pairs EURUSD GBPUSD  Analyze specific pairs
-  python main.py --timeframe H4         Use 4-hour as primary TF
-  python main.py --all                  Include cross pairs
-  python main.py --json                 Output as JSON
-  python main.py --detail               Show detailed analysis per pair
-  python main.py --watch                Continuous monitoring (5min refresh)
+  python main.py                          Live signal (H1, summary)
+  python main.py --detail                 Live signal with full breakdown
+  python main.py --timeframe H4 --detail  H4 timeframe
+  python main.py --json                   JSON output
         """,
     )
-    parser.add_argument(
-        "--pairs", nargs="+", default=None,
-        help="Specific pairs to analyze (e.g., EURUSD GBPUSD)",
-    )
-    parser.add_argument(
-        "--timeframe", "-tf", default="H1",
-        choices=["M5", "M15", "M30", "H1", "H4", "D1"],
-        help="Primary analysis timeframe (default: H1)",
-    )
-    parser.add_argument(
-        "--all", action="store_true",
-        help="Include cross pairs in analysis",
-    )
-    parser.add_argument(
-        "--json", action="store_true",
-        help="Output results as JSON",
-    )
-    parser.add_argument(
-        "--detail", action="store_true",
-        help="Show detailed analysis for each pair",
-    )
-    parser.add_argument(
-        "--top", type=int, default=None,
-        help="Show only top N signals by strength",
-    )
-    parser.add_argument(
-        "--watch", action="store_true",
-        help="Continuous monitoring mode (refreshes every 5 min)",
-    )
-    parser.add_argument(
-        "--min-grade", default="F",
-        choices=["A+", "A", "B", "C", "D", "F"],
-        help="Minimum signal quality to display",
-    )
-    parser.add_argument(
-        "--verbose", "-v", action="store_true",
-        help="Enable debug logging",
-    )
+    parser.add_argument("--timeframe", "-tf", default="H1",
+                        choices=["M5", "M15", "M30", "H1", "H4", "D1"],
+                        help="Primary analysis timeframe (default: H1)")
+    parser.add_argument("--detail", action="store_true",
+                        help="Show full indicator breakdown")
+    parser.add_argument("--json", action="store_true",
+                        help="Output as JSON")
+    parser.add_argument("--verbose", "-v", action="store_true",
+                        help="Debug logging")
+
     return parser.parse_args()
 
 
-def symbol_from_name(name: str) -> str:
-    """Convert pair name like 'EURUSD' to Yahoo symbol 'EURUSD=X'."""
-    name = name.upper().replace("/", "").replace("_", "")
-    if not name.endswith("=X"):
-        name += "=X"
-    return name
-
-
-def run_analysis(args) -> List[SignalReport]:
-    """Run a single analysis cycle."""
+def build_config(args) -> PlatformConfig:
     config = PlatformConfig()
 
-    # Configure timeframe
     tf_map = {
         "M5": Timeframe.M5, "M15": Timeframe.M15, "M30": Timeframe.M30,
         "H1": Timeframe.H1, "H4": Timeframe.H4, "D1": Timeframe.D1,
     }
     config.primary_timeframe = tf_map[args.timeframe]
 
-    # Configure confirmation timeframes based on primary
     tf_confirmations = {
-        "M5": [Timeframe.M1, Timeframe.M15],
-        "M15": [Timeframe.M5, Timeframe.H1],
+        "M5":  [Timeframe.M1,  Timeframe.M15],
+        "M15": [Timeframe.M5,  Timeframe.H1],
         "M30": [Timeframe.M15, Timeframe.H1],
-        "H1": [Timeframe.M15, Timeframe.H4],
-        "H4": [Timeframe.H1, Timeframe.D1],
-        "D1": [Timeframe.H4, Timeframe.W1],
+        "H1":  [Timeframe.M15, Timeframe.H4],
+        "H4":  [Timeframe.H1,  Timeframe.D1],
+        "D1":  [Timeframe.H4,  Timeframe.W1],
     }
     config.confirmation_timeframes = tf_confirmations.get(
         args.timeframe, [Timeframe.M15, Timeframe.H4]
     )
 
-    # Configure pairs
-    if args.pairs:
-        config.watch_pairs = [symbol_from_name(p) for p in args.pairs]
-    elif args.all:
-        config.watch_pairs = MAJOR_PAIRS + CROSS_PAIRS
-    else:
-        config.watch_pairs = MAJOR_PAIRS
+    return config
 
-    # Run analysis
+
+def run_live(args, config: PlatformConfig):
+    print(f"{Colors.GRAY}  Fetching market data and running analysis...{Colors.RESET}\n")
     aggregator = SignalAggregator(config)
-    reports = aggregator.analyze_all_pairs()
+    report = aggregator.analyze_pair()
 
-    # Filter by grade
-    grade_order = ["A+", "A", "B", "C", "D", "F"]
-    min_idx = grade_order.index(args.min_grade) if args.min_grade in grade_order else len(grade_order) - 1
-    reports = [r for r in reports if grade_order.index(r.signal_quality) <= min_idx]
+    if args.json:
+        print(json.dumps(report_to_dict(report), indent=2))
+        return
 
-    # Top N
-    if args.top:
-        reports = reports[:args.top]
-
-    return reports
+    print_header("USD/JPY SIGNAL INTELLIGENCE")
+    print_signal_report(report, detail=args.detail)
+    print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
+    print(f"{Colors.GRAY}  Disclaimer: For educational/research purposes only.{Colors.RESET}")
+    print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
 
 
 def main():
     args = parse_args()
 
-    # Setup logging
     log_level = logging.DEBUG if args.verbose else logging.WARNING
     logging.basicConfig(
         level=log_level,
@@ -446,63 +312,8 @@ def main():
         datefmt="%H:%M:%S",
     )
 
-    if args.watch:
-        # Continuous monitoring mode
-        print(f"{Colors.BOLD}Starting continuous monitoring mode (Ctrl+C to stop)...{Colors.RESET}")
-        while True:
-            try:
-                reports = run_analysis(args)
-                if args.json:
-                    output = [report_to_dict(r) for r in reports]
-                    print(json.dumps(output, indent=2))
-                else:
-                    print("\033[2J\033[H")  # clear screen
-                    print_header()
-                    print_summary_table(reports)
-                    print_event_calendar(reports)
-                    if args.detail:
-                        for r in reports:
-                            print_detailed_report(r)
-                    print(f"\n{Colors.GRAY}  Next refresh in 5 minutes...{Colors.RESET}")
-
-                time.sleep(300)
-            except KeyboardInterrupt:
-                print(f"\n{Colors.YELLOW}Monitoring stopped.{Colors.RESET}")
-                break
-    else:
-        # Single run
-        print(f"{Colors.GRAY}  Fetching market data and running analysis...{Colors.RESET}\n")
-        reports = run_analysis(args)
-
-        if args.json:
-            output = [report_to_dict(r) for r in reports]
-            print(json.dumps(output, indent=2))
-        else:
-            print_header()
-            if reports:
-                print_summary_table(reports)
-                print_event_calendar(reports)
-
-                if args.detail:
-                    for r in reports:
-                        print_detailed_report(r)
-                else:
-                    # Show detail for top signals only
-                    actionable = [r for r in reports if r.signal_quality in ("A+", "A", "B")]
-                    if actionable:
-                        print(f"\n{Colors.BOLD}{Colors.WHITE}  ═══ ACTIONABLE SIGNALS (Grade B or above) ═══{Colors.RESET}")
-                        for r in actionable:
-                            print_detailed_report(r)
-                    else:
-                        print(f"\n{Colors.GRAY}  No high-grade signals detected. "
-                              f"Use --detail for full breakdown or --min-grade C to see more.{Colors.RESET}")
-            else:
-                print(f"{Colors.YELLOW}  No data available. Check your internet connection.{Colors.RESET}")
-
-            print(f"\n{Colors.CYAN}{'═' * 80}{Colors.RESET}")
-            print(f"{Colors.GRAY}  Disclaimer: This tool is for educational/research purposes only.{Colors.RESET}")
-            print(f"{Colors.GRAY}  Past performance ≠ future results. Always use proper risk management.{Colors.RESET}")
-            print(f"{Colors.CYAN}{'═' * 80}{Colors.RESET}\n")
+    config = build_config(args)
+    run_live(args, config)
 
 
 if __name__ == "__main__":

mt5.md

@@ -0,0 +1,185 @@
+# MT5 Auto-Execution Integration Plan
+
+## Context
+ForexBot2 generates trade signals with full setups (entry, SL, TP) but currently only displays them in Telegram. The user wants fully automatic order execution via MetaTrader 5.
+
+**Constraint:** MT5 runs on a separate Windows machine. The Python `MetaTrader5` package requires Windows (COM interface). The bot runs on Linux. A network bridge is needed.
+
+**Broker:** TIO Markets (demo first, live later via config change).
+**Execution mode:** Fully automatic for actionable signals (A+/A/B grades).
+
+## Architecture: REST API Bridge
+
+```
+Linux (telegram_bot.py)                    Windows (mt5_bridge.py)
+========================                   ========================
+Signal fires with TradeSetup               FastAPI server running
+  -> mt5_client.execute_trade()              -> Wraps MetaTrader5 package
+     POST /api/order  ─────── HTTP ───────>  -> mt5.order_send()
+     <- {ticket: 12345}  <──── HTTP ──────   <- Returns ticket or error
+  -> Telegram: "ORDER PLACED"              -> Idempotency via order_id
+```
+
+**Why REST over WebSocket/file-sync:** Stateless, debuggable, retry-safe. For a bot placing 1-5 trades/day, HTTP latency (5-50ms LAN) is negligible.
+
+## Files to Create/Modify
+
+### New Files
+
+**1. `mt5_bridge/mt5_bridge.py`** — Windows-side FastAPI server (~200 lines)
+- Wraps `MetaTrader5` Python package
+- Endpoints: `/api/health`, `/api/account`, `/api/positions`, `/api/order` (POST), `/api/order/close` (POST)
+- Lot size calculation on bridge side (has access to `mt5.symbol_info()` for tick value, contract size, volume step)
+- Idempotency: stores `{order_id: ticket}` to prevent duplicate orders on retry
+- API key auth via `X-API-Key` header
+- MT5 credentials from env vars: `MT5_LOGIN`, `MT5_PASSWORD`, `MT5_SERVER`, `MT5_PATH`
+
+**2. `mt5_bridge/requirements.txt`** — Windows dependencies
+- `fastapi`, `uvicorn`, `MetaTrader5`, `pydantic`
+
+**3. `mt5_client.py`** — Linux-side async HTTP client (~150 lines)
+- `MT5ExecutionClient` class with `aiohttp`
+- `execute_trade(setup: TradeSetup, quality: str)` — sends order to bridge
+- Symbol mapping: `EURUSD=X` -> `EURUSD` (TIO Markets format, configurable)
+- Duplicate prevention: tracks open positions per symbol
+- Retry: 3 attempts, exponential backoff, same `order_id` for idempotency
+- Position sync: periodically calls `/api/positions` to stay in sync with MT5
+- Graceful degradation: if bridge offline, logs warning, bot continues signal-only
+
+### Modified Files
+
+**4. `config.py`** — Add `MT5Config` dataclass
+```python
+@dataclass
+class MT5Config:
+    enabled: bool = False
+    bridge_url: str = "http://192.168.1.100:8080"
+    bridge_api_key: str = ""
+    risk_pct: float = 1.0              # % of balance risked per trade
+    max_open_positions: int = 3
+    max_daily_trades: int = 10         # Circuit breaker
+    magic_number: int = 234000         # Identifies ForexBot2 orders
+    slippage_points: int = 20          # 2 pips for 5-digit brokers
+    auto_execute_grades: List[str]     # ["A+", "A", "B"]
+    symbol_map: Dict[str, str]         # Internal -> broker symbol
+```
+All overridable via env vars (`MT5_BRIDGE_URL`, `MT5_RISK_PCT`, etc.)
+
+**5. `telegram_bot.py`** — Wire execution into signal flow
+- In `_scan_timeframe()`: after sending Telegram messages, call `mt5_client.execute_trade()` for actionable signals
+- In `_handle_breach()`: same pattern after breach scan produces actionable signal
+- Add background task `_mt5_position_sync_loop()` for periodic position sync
+- Add Telegram commands: `/mt5status` (bridge health + account info), `/positions` (open trades)
+- Register new commands in `set_my_commands()`
+- Execution result messages sent to Telegram (ticket # on success, error on failure)
+
+**6. `requirements.txt`** — Add `aiohttp>=3.9` (async HTTP for Linux-side client)
+
+## Safety Layers
+
+1. **Demo account first** — `MT5_SERVER=TIOMarkets-Demo`
+2. **`enabled: False` default** — must explicitly opt-in via `MT5_ENABLED=true`
+3. **Duplicate prevention** — 3 layers: client-side tracking, bridge-side position check, idempotent `order_id`
+4. **Circuit breaker** — `max_daily_trades` stops execution if too many orders fire
+5. **Position limit** — `max_open_positions` caps concurrent exposure
+6. **Risk-based sizing** — lot size from `risk_pct` of account balance, calculated on bridge with live tick values
+7. **Graceful degradation** — bridge offline = signal-only mode, no crash
+
+## Position Sizing (calculated on Windows bridge)
+
+```
+risk_amount = balance * (risk_pct / 100)
+tick_value = mt5.symbol_info(symbol).trade_tick_value
+tick_size = mt5.symbol_info(symbol).trade_tick_size
+sl_ticks = abs(entry - stop_loss) / tick_size
+lot_size = risk_amount / (sl_ticks * tick_value)
+lot_size = round_down(lot_size, volume_step)
+lot_size = clamp(lot_size, volume_min, volume_max)
+```
+
+Bridge side is best because it has real-time `tick_value` (varies with exchange rates for cross pairs).
+
+## Symbol Mapping (TIO Markets)
+
+| Internal | MT5 Symbol |
+|----------|-----------|
+| `EURUSD=X` | `EURUSD` |
+| `USDJPY=X` | `USDJPY` |
+| `GBPUSD=X` | `GBPUSD` |
+| `XAU/USD` | `XAUUSD` |
+
+Configurable via `MT5_SYMBOL_MAP` env var (JSON) for broker variations.
+
+## Integration Points
+
+### In `_scan_timeframe()` (after line ~406):
+```python
+mt5 = _get_mt5_client()
+if mt5:
+    for report in actionable:
+        if report.trade_setup and report.signal_quality in mt5.config.auto_execute_grades:
+            result = await mt5.execute_trade(report.trade_setup, report.signal_quality)
+            # Send execution confirmation/error to Telegram
+```
+
+### In `_handle_breach()` (after line ~477):
+```python
+if report.trade_setup and report.signal_quality in mt5.config.auto_execute_grades:
+    mt5 = _get_mt5_client()
+    if mt5:
+        result = await mt5.execute_trade(report.trade_setup, report.signal_quality)
+        # Send execution confirmation/error to Telegram
+```
+
+## New Telegram Commands
+
+| Command | Description |
+|---------|-------------|
+| `/mt5status` | Bridge health, account balance/equity, open positions count |
+| `/positions` | List all open MT5 positions with P/L |
+
+## Implementation Order
+
+1. `config.py` — Add `MT5Config`
+2. `mt5_bridge/mt5_bridge.py` + `requirements.txt` — Windows bridge server
+3. `mt5_client.py` — Linux HTTP client
+4. `telegram_bot.py` — Wire execution + new commands
+5. `requirements.txt` — Add `aiohttp`
+
+## Verification
+
+1. Start bridge on Windows: `python mt5_bridge.py` -> verify `curl /api/health` returns MT5 connected
+2. Test order via curl: `POST /api/order` with demo credentials -> verify order appears in MT5
+3. Start bot on Linux with `MT5_ENABLED=true` -> verify `/mt5status` command works
+4. Wait for actionable signal (or manually trigger scan) -> verify order placed + Telegram confirmation
+5. Verify duplicate prevention: same signal shouldn't place second order
+6. Kill bridge -> verify bot continues in signal-only mode without crashing
+
+## Windows Bridge Setup (one-time)
+
+```bash
+# On Windows machine:
+pip install fastapi uvicorn MetaTrader5 pydantic
+
+# Set environment variables:
+set MT5_LOGIN=12345678
+set MT5_PASSWORD=your-password
+set MT5_SERVER=TIOMarkets-Demo
+set MT5_PATH=C:\Program Files\MetaTrader 5\terminal64.exe
+set BRIDGE_API_KEY=your-random-32-char-key
+set BRIDGE_PORT=8080
+
+# Run:
+python mt5_bridge.py
+```
+
+## Linux Bot Config (env vars)
+
+```bash
+MT5_ENABLED=true
+MT5_BRIDGE_URL=http://<windows-ip>:8080
+MT5_BRIDGE_API_KEY=your-random-32-char-key
+MT5_RISK_PCT=1.0
+MT5_MAX_OPEN_POSITIONS=3
+MT5_AUTO_EXECUTE_GRADES=A+,A,B
+```

news_engine.py

@@ -104,6 +104,7 @@ class NewsEngine:
         "AUD": ["aussie", "rba", "australian", "australia"],
         "CAD": ["loonie", "boc", "canadian", "canada"],
         "NZD": ["kiwi", "rbnz", "new zealand"],
+        "XAU": ["gold", "xau", "precious metal", "safe haven", "bullion"],
     }
 
     # High-impact event types (these move markets significantly)
@@ -187,15 +188,21 @@ class NewsEngine:
 
         return context
 
+    @staticmethod
+    def _parse_pair(pair_symbol: str) -> Tuple[str, str]:
+        if "/" in pair_symbol:
+            parts = pair_symbol.split("/")
+            return parts[0], parts[1]
+        clean = pair_symbol.replace("=X", "")
+        return clean[:3], clean[3:]
+
     def get_event_risk_level(self, pair_symbol: str, context: FundamentalContext) -> Tuple[str, str]:
         """
         Get event risk level for a specific pair.
         Returns (risk_level, description).
         risk_level: "clear", "caution", "danger"
         """
-        clean = pair_symbol.replace("=X", "")
-        base = clean[:3]
-        quote = clean[3:]
+        base, quote = self._parse_pair(pair_symbol)
 
         # Check upcoming high-impact events
         for event in context.events_upcoming_1h:
@@ -222,9 +229,7 @@ class NewsEngine:
         Returns (score, description) where score is -1 to +1.
         Positive = bullish for the pair, negative = bearish.
         """
-        clean = pair_symbol.replace("=X", "")
-        base = clean[:3]
-        quote = clean[3:]
+        base, quote = self._parse_pair(pair_symbol)
 
         base_sent = context.news_sentiment.get(base, 0.0)
         quote_sent = context.news_sentiment.get(quote, 0.0)
@@ -413,15 +418,11 @@ class NewsEngine:
 
     @staticmethod
     def _get_affected_pairs(currency: str) -> List[str]:
-        """Get all major pairs affected by a currency event."""
-        pairs = {
-            "USD": ["EURUSD=X", "GBPUSD=X", "USDJPY=X", "USDCHF=X", "AUDUSD=X", "USDCAD=X", "NZDUSD=X"],
-            "EUR": ["EURUSD=X", "EURGBP=X", "EURJPY=X", "EURCHF=X"],
-            "GBP": ["GBPUSD=X", "EURGBP=X", "GBPJPY=X"],
-            "JPY": ["USDJPY=X", "EURJPY=X", "GBPJPY=X", "AUDJPY=X", "CADJPY=X"],
-            "CHF": ["USDCHF=X", "EURCHF=X"],
-            "AUD": ["AUDUSD=X", "AUDJPY=X"],
-            "CAD": ["USDCAD=X", "CADJPY=X"],
-            "NZD": ["NZDUSD=X"],
+        _MAP = {
+            "USD": ["EURUSD=X", "USDJPY=X", "GBPUSD=X", "XAU/USD"],
+            "EUR": ["EURUSD=X"],
+            "JPY": ["USDJPY=X"],
+            "GBP": ["GBPUSD=X"],
+            "XAU": ["XAU/USD"],
         }
-        return pairs.get(currency, [])
+        return _MAP.get(currency, [])

requirements.txt

@@ -0,0 +1,6 @@
+numpy>=1.24
+pandas>=2.0
+requests>=2.28
+twelvedata>=1.2.0
+python-telegram-bot>=20.0
+websockets>=12.0

script.txt

@@ -1 +0,0 @@
-pip install pandas numpy requests python-dateutil -q

signal_aggregator.py

@@ -1,29 +1,19 @@
 """
-Forex Signal Intelligence Platform - Signal Aggregator
-========================================================
-Combines technical analysis across multiple timeframes with
-fundamental/news context to produce final actionable signals.
-
-This is the brain — it decides what's actually worth paying attention to.
-
-Key principles:
-1. Multi-timeframe agreement = higher confidence
-2. High-impact news overrides technicals
-3. Divergences between timeframes = caution
-4. No signal is better than a bad signal
+ForexBot2 — Signal Aggregator
+==============================
+Combines multi-timeframe technical analysis with optional news context
+into a graded actionable signal.
 """
 
 import logging
+import os
 from dataclasses import dataclass, field
 from datetime import datetime, timezone
 from typing import Dict, List, Optional
 
 import pandas as pd
 
-from config import (
-    PlatformConfig, SignalStrength, Timeframe,
-    SYMBOL_DISPLAY,
-)
+from config import PlatformConfig, SignalStrength, Timeframe
 from data_provider import ForexDataProvider
 from engine import TechnicalEngine, TechnicalAnalysis
 from news_engine import NewsEngine, FundamentalContext
@@ -33,10 +23,9 @@ logger = logging.getLogger(__name__)
 
 @dataclass
 class TradeSetup:
-    """A complete trade setup recommendation."""
     pair: str
     display_name: str
-    direction: str  # "BUY" or "SELL"
+    direction: str  # "BUY" / "SELL"
     entry_price: float
     stop_loss: float
     take_profit_1: float
@@ -49,47 +38,38 @@ class TradeSetup:
 
 @dataclass
 class SignalReport:
-    """Complete signal report for a single pair."""
     symbol: str
     display_name: str
     timestamp: str
 
-    # Composite signals
     primary_signal: SignalStrength = SignalStrength.NEUTRAL
     primary_score: float = 0.0
     primary_confidence: float = 0.0
 
-    # Multi-timeframe agreement
     mtf_agreement: bool = False
     mtf_signals: Dict[str, SignalStrength] = field(default_factory=dict)
     mtf_scores: Dict[str, float] = field(default_factory=dict)
 
-    # Technical details
     primary_analysis: Optional[TechnicalAnalysis] = None
     confirmation_analyses: List[TechnicalAnalysis] = field(default_factory=list)
 
-    # Fundamental context
     event_risk: str = "clear"
     event_description: str = ""
     news_sentiment: float = 0.0
     news_description: str = ""
 
-    # Final verdict
     final_signal: SignalStrength = SignalStrength.NEUTRAL
     final_score: float = 0.0
     final_confidence: float = 0.0
     action_recommendation: str = ""
-    signal_quality: str = ""  # "A+", "A", "B", "C", "D", "F"
+    signal_quality: str = ""  # A+, A, B, C, D, F
 
-    # Trade setup (only for actionable signals)
     trade_setup: Optional[TradeSetup] = None
 
-    # Key reasons
     bullish_reasons: List[str] = field(default_factory=list)
     bearish_reasons: List[str] = field(default_factory=list)
     warnings: List[str] = field(default_factory=list)
 
-    # Current price info
     current_price: float = 0.0
     spread_estimate: float = 0.0
     market_regime: str = ""
@@ -97,54 +77,180 @@ class SignalReport:
 
 
 class SignalAggregator:
-    """
-    Master signal aggregator.
-
-    Orchestrates data fetching, technical analysis across timeframes,
-    fundamental analysis, and produces final graded signals.
-    """
+    """Master aggregator for live signal analysis."""
 
     def __init__(self, config: PlatformConfig):
         self.config = config
         self.data_provider = ForexDataProvider(config)
         self.tech_engine = TechnicalEngine(config.technical)
-        self.news_engine = NewsEngine(config.news)
+        self._news_engine: Optional[NewsEngine] = None  # lazy
 
-    def analyze_all_pairs(self) -> List[SignalReport]:
-        """
-        Analyze all watched pairs and return sorted signal reports.
-        Returns reports sorted by signal quality (best first).
-        """
-        reports = []
-        context = self.news_engine.get_fundamental_context()
+    def _get_news_engine(self) -> NewsEngine:
+        if self._news_engine is None:
+            self._news_engine = NewsEngine(self.config.news)
+        return self._news_engine
 
-        # Batch-prefetch all pairs × all needed timeframes (3 API calls instead of 21)
-        all_timeframes = (
-            [self.config.primary_timeframe] + self.config.confirmation_timeframes
-        )
-        self.data_provider.prefetch_batch(self.config.watch_pairs, all_timeframes)
-
-        for symbol in self.config.watch_pairs:
-            try:
-                report = self.analyze_pair(symbol, context)
-                reports.append(report)
-            except Exception as e:
-                logger.error(f"Failed to analyze {symbol}: {e}")
-                continue
-
-        # Sort by absolute score (strongest signals first)
-        reports.sort(key=lambda r: abs(r.final_score), reverse=True)
-        return reports
+    # ── Public: live mode entry point ──────────────────────────────
 
     def analyze_pair(
         self,
-        symbol: str,
+        symbol: Optional[str] = None,
         context: Optional[FundamentalContext] = None,
     ) -> SignalReport:
-        """Complete analysis of a single pair across all layers."""
-        if context is None:
-            context = self.news_engine.get_fundamental_context()
+        """Live-mode entry: fetch all needed timeframes then delegate."""
+        if symbol is None:
+            symbol = self.config.symbol
 
+        timeframes = [self.config.primary_timeframe] + list(
+            self.config.confirmation_timeframes
+        )
+
+        bt = self.config.backtest
+        if bt.regime_filter and bt.regime_filter_timeframe not in timeframes:
+            timeframes.append(bt.regime_filter_timeframe)
+
+        dfs: Dict[Timeframe, pd.DataFrame] = {}
+        stale_tfs = []
+        for tf in timeframes:
+            try:
+                df = self.data_provider.fetch_ohlcv(symbol, tf)
+                dfs[tf] = df
+                if not self.data_provider._is_data_fresh(df, symbol, tf):
+                    stale_tfs.append(tf.value)
+            except Exception as e:
+                logger.warning(f"Fetch failed for {symbol} {tf.value}: {e}")
+
+        report = self.analyze_from_dataframes(
+            symbol=symbol,
+            dfs=dfs,
+            context=context,
+            use_news=True,
+        )
+
+        if stale_tfs:
+            report.warnings.append(f"STALE DATA on {', '.join(stale_tfs)} — signal may be unreliable")
+            if self.config.primary_timeframe.value in stale_tfs:
+                report.final_signal = SignalStrength.NEUTRAL
+                report.signal_quality = "F"
+                report.trade_setup = None
+                report.warnings.append("Primary timeframe stale — signal suppressed")
+
+        if bt.regime_filter:
+            self._apply_regime_filter_live(report, dfs)
+
+        if bt.session_filter:
+            xau_active = (
+                report.display_name == "XAU/USD"
+                and self.config.xau_strategy.enabled
+                and any("XAU STRATEGY" in r for r in report.bullish_reasons)
+            )
+            if not xau_active:
+                self._apply_session_filter_live(report)
+
+        return report
+
+    def _apply_session_filter_live(self, report: SignalReport) -> None:
+        """Skip signals outside the configured UTC trading window."""
+        bt = self.config.backtest
+        now_h = datetime.now(timezone.utc).hour
+        if not (bt.session_start_hour <= now_h < bt.session_end_hour):
+            report.warnings.append(
+                f"SESSION FILTER: {now_h:02}:00 UTC outside "
+                f"{bt.session_start_hour:02}:00-{bt.session_end_hour:02}:00 — skipping"
+            )
+            report.final_signal = SignalStrength.NEUTRAL
+            report.signal_quality = "F"
+            report.action_recommendation = (
+                f"NO TRADE — Outside session "
+                f"({bt.session_start_hour:02}-{bt.session_end_hour:02} UTC; now {now_h:02})"
+            )
+            report.trade_setup = None
+        else:
+            report.warnings.append(
+                f"Session: {now_h:02}:00 UTC inside trading window "
+                f"{bt.session_start_hour:02}-{bt.session_end_hour:02}"
+            )
+
+    def _apply_regime_filter_live(
+        self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
+    ) -> None:
+        """If higher-TF ADX is below threshold, downgrade signal to NEUTRAL."""
+        bt = self.config.backtest
+        regime_tf = bt.regime_filter_timeframe
+        df = dfs.get(regime_tf)
+        if df is None or df.empty:
+            report.warnings.append(
+                f"Regime filter requested but {regime_tf.value} data unavailable"
+            )
+            return
+
+        adx_value = self._latest_adx(df, self.config.technical.adx_period)
+        if adx_value is None:
+            report.warnings.append(
+                f"Regime filter: ADX could not be computed on {regime_tf.value}"
+            )
+            return
+
+        if adx_value < bt.regime_filter_min_adx:
+            report.warnings.append(
+                f"REGIME FILTER: {regime_tf.value} ADX={adx_value:.1f} < "
+                f"{bt.regime_filter_min_adx:.0f} — market choppy, skipping signal"
+            )
+            report.final_signal = SignalStrength.NEUTRAL
+            report.signal_quality = "F"
+            report.action_recommendation = (
+                f"NO TRADE — Regime filter (ADX {adx_value:.1f} on {regime_tf.value})"
+            )
+            report.trade_setup = None
+        else:
+            report.bullish_reasons.append(
+                f"Regime: {regime_tf.value} ADX={adx_value:.1f} confirms trending market"
+            ) if report.final_score > 0 else report.bearish_reasons.append(
+                f"Regime: {regime_tf.value} ADX={adx_value:.1f} confirms trending market"
+            )
+
+    @staticmethod
+    def _latest_adx(df: pd.DataFrame, period: int = 14) -> Optional[float]:
+        """Wilder ADX on a fresh df; returns the last finite value or None."""
+        if df is None or len(df) < period * 3:
+            return None
+        high, low, close = df["High"], df["Low"], df["Close"]
+        tr = pd.concat([
+            high - low,
+            (high - close.shift()).abs(),
+            (low - close.shift()).abs(),
+        ], axis=1).max(axis=1)
+        plus_dm = high.diff()
+        minus_dm = -low.diff()
+        plus_dm = plus_dm.where((plus_dm > minus_dm) & (plus_dm > 0), 0.0)
+        minus_dm = minus_dm.where((minus_dm > plus_dm) & (minus_dm > 0), 0.0)
+        atr_s = tr.ewm(alpha=1/period, min_periods=period, adjust=False).mean()
+        plus_di = 100 * (plus_dm.ewm(alpha=1/period, min_periods=period, adjust=False).mean() / atr_s)
+        minus_di = 100 * (minus_dm.ewm(alpha=1/period, min_periods=period, adjust=False).mean() / atr_s)
+        denom = (plus_di + minus_di).replace(0, float("nan"))
+        dx = 100 * (plus_di - minus_di).abs() / denom
+        adx = dx.ewm(alpha=1/period, min_periods=period, adjust=False).mean().dropna()
+        if adx.empty:
+            return None
+        return float(adx.iloc[-1])
+
+    # ── Public: bar-replayable entry point ─────────────────────────
+
+    def analyze_from_dataframes(
+        self,
+        symbol: str,
+        dfs: Dict[Timeframe, pd.DataFrame],
+        context: Optional[FundamentalContext] = None,
+        use_news: bool = True,
+    ) -> SignalReport:
+        """
+        Run the full analysis pipeline on already-fetched dataframes.
+
+        `dfs` must include the primary timeframe and ideally the
+        confirmation timeframes. Each df is treated as the visible history
+        at decision time — the backtester is responsible for slicing so
+        no future bars leak in.
+        """
         pair_info = self.data_provider.get_pair_info(symbol)
         display = pair_info["display"]
         pip_value = pair_info["pip_value"]
@@ -155,33 +261,33 @@ class SignalAggregator:
             timestamp=str(datetime.now(timezone.utc)),
         )
 
-        # ── 1. Primary Timeframe Analysis ──────────────────────────
-        try:
-            primary_df = self.data_provider.fetch_ohlcv(
-                symbol, self.config.primary_timeframe
-            )
-            primary_analysis = self.tech_engine.analyze(
-                primary_df, symbol, self.config.primary_timeframe.value
-            )
-            report.primary_analysis = primary_analysis
-            report.primary_signal = primary_analysis.composite_signal
-            report.primary_score = primary_analysis.composite_score
-            report.primary_confidence = primary_analysis.confidence
-            report.market_regime = primary_analysis.market_regime
-            report.volatility_regime = primary_analysis.volatility_regime
-            report.current_price = float(primary_df["Close"].iloc[-1])
-            report.mtf_signals[self.config.primary_timeframe.value] = primary_analysis.composite_signal
-            report.mtf_scores[self.config.primary_timeframe.value] = primary_analysis.composite_score
-        except Exception as e:
-            logger.error(f"Primary analysis failed for {symbol}: {e}")
-            report.warnings.append(f"Primary timeframe data unavailable: {e}")
+        # ── 1. Primary timeframe ───────────────────────────────────
+        primary_df = dfs.get(self.config.primary_timeframe)
+        if primary_df is None or primary_df.empty:
+            report.warnings.append("Primary timeframe data unavailable")
             report.signal_quality = "F"
             return report
 
-        # ── 2. Confirmation Timeframe Analyses ─────────────────────
+        primary_analysis = self.tech_engine.analyze(
+            primary_df, symbol, self.config.primary_timeframe.value
+        )
+        report.primary_analysis = primary_analysis
+        report.primary_signal = primary_analysis.composite_signal
+        report.primary_score = primary_analysis.composite_score
+        report.primary_confidence = primary_analysis.confidence
+        report.market_regime = primary_analysis.market_regime
+        report.volatility_regime = primary_analysis.volatility_regime
+        report.current_price = float(primary_df["Close"].iloc[-1])
+        report.timestamp = str(primary_df.index[-1])
+        report.mtf_signals[self.config.primary_timeframe.value] = primary_analysis.composite_signal
+        report.mtf_scores[self.config.primary_timeframe.value] = primary_analysis.composite_score
+
+        # ── 2. Confirmation timeframes ─────────────────────────────
         for tf in self.config.confirmation_timeframes:
+            cf_df = dfs.get(tf)
+            if cf_df is None or cf_df.empty:
+                continue
             try:
-                cf_df = self.data_provider.fetch_ohlcv(symbol, tf)
                 cf_analysis = self.tech_engine.analyze(cf_df, symbol, tf.value)
                 report.confirmation_analyses.append(cf_analysis)
                 report.mtf_signals[tf.value] = cf_analysis.composite_signal
@@ -189,35 +295,198 @@ class SignalAggregator:
             except Exception as e:
                 logger.warning(f"Confirmation TF {tf.value} failed for {symbol}: {e}")
 
-        # ── 3. Multi-Timeframe Agreement Check ─────────────────────
+        # ── 3. MTF agreement ───────────────────────────────────────
         report.mtf_agreement = self._check_mtf_agreement(report)
 
-        # ── 4. Fundamental Context ─────────────────────────────────
-        event_risk, event_desc = self.news_engine.get_event_risk_level(symbol, context)
-        report.event_risk = event_risk
-        report.event_description = event_desc
+        # ── 4. Fundamental context (optional) ──────────────────────
+        if use_news:
+            news = self._get_news_engine()
+            if context is None:
+                context = news.get_fundamental_context()
+            event_risk, event_desc = news.get_event_risk_level(symbol, context)
+            report.event_risk = event_risk
+            report.event_description = event_desc
+            sentiment, sent_desc = news.get_pair_sentiment(symbol, context)
+            report.news_sentiment = sentiment
+            report.news_description = sent_desc
+        else:
+            report.event_risk = "clear"
+            report.event_description = ""
+            report.news_sentiment = 0.0
+            report.news_description = ""
 
-        sentiment, sent_desc = self.news_engine.get_pair_sentiment(symbol, context)
-        report.news_sentiment = sentiment
-        report.news_description = sent_desc
+        # ── 5. Spread estimate from primary data (avoids extra API call) ─
+        try:
+            report.spread_estimate = float(
+                (primary_df["High"] - primary_df["Low"]).median()
+            )
+        except Exception:
+            report.spread_estimate = 0.0
 
-        # ── 5. Spread Estimate ─────────────────────────────────────
-        report.spread_estimate = self.data_provider.get_spread_estimate(symbol)
-
-        # ── 6. Compute Final Signal ────────────────────────────────
+        # ── 6. Final signal ────────────────────────────────────────
         self._compute_final_signal(report)
 
-        # ── 7. Generate Trade Setup (if signal is actionable) ──────
-        if report.signal_quality in ("A+", "A", "B"):
-            report.trade_setup = self._generate_trade_setup(report, pip_value)
+        # ── 6b. Session filter (works for both live and backtest) ─
+        if self.config.backtest.session_filter:
+            try:
+                bar_hour = pd.Timestamp(report.timestamp).hour
+                if bar_hour in self.config.backtest.blocked_hours_utc:
+                    report.final_signal = SignalStrength.NEUTRAL
+                    report.signal_quality = "F"
+                    report.warnings.append(f"SESSION FILTER: {bar_hour:02}:00 UTC blocked")
+            except Exception:
+                pass
 
-        # ── 8. Collect Reasons ─────────────────────────────────────
+        # ── 6c. XAU/USD oversold bounce strategy ─────────────────
+        if report.display_name == "XAU/USD" and self.config.xau_strategy.enabled:
+            if self._check_xau_oversold_bounce(report, dfs):
+                report.final_signal = SignalStrength.STRONG_BUY
+                report.final_score = 50.0
+                report.final_confidence = 85.0
+                report.signal_quality = "A+"
+                report.action_recommendation = (
+                    "STRONG BUY — XAU oversold bounce "
+                    "(RSI<30 + M15/H1/H4 all above EMA200)"
+                )
+                report.bullish_reasons.insert(
+                    0,
+                    "XAU STRATEGY: M15 RSI oversold + triple-TF bullish trend confirmation",
+                )
+                report.trade_setup = self._generate_xau_trade_setup(report, dfs)
+
+        # ── 7. Trade setup for actionable signals ──────────────────
+        if report.trade_setup is None and report.signal_quality in ("A+", "A", "B", "C"):
+            is_sell_direction = (
+                report.final_signal in (SignalStrength.SELL, SignalStrength.STRONG_SELL)
+                or (report.final_signal == SignalStrength.NEUTRAL and report.final_score < 0)
+            )
+            if is_sell_direction and self.config.backtest.sell_filter:
+                if report.display_name in self.config.backtest.sell_allowed_pairs:
+                    report.trade_setup = self._generate_trade_setup(report, pip_value)
+                elif self._forex_sell_qualifies(report):
+                    report.trade_setup = self._generate_trade_setup(report, pip_value)
+                else:
+                    report.trade_setup = None
+            else:
+                report.trade_setup = self._generate_trade_setup(report, pip_value)
+
+        # ── 8. Reasons ─────────────────────────────────────────────
         self._collect_reasons(report)
 
         return report
 
+    # ── Internals ──────────────────────────────────────────────────
+
+    def _forex_sell_qualifies(self, report: SignalReport) -> bool:
+        """Always returns False — forex SELL is structurally unprofitable.
+        0/4 trades won even under best conditions (trending_down + MTF + ADX>=25 + RSI<=50)."""
+        return False
+
+    def _check_xau_oversold_bounce(
+        self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
+    ) -> bool:
+        """Return True when XAU/USD oversold-bounce conditions are met.
+
+        Conditions (all must be true):
+        1. M15 RSI(14) < 30
+        2. M15 price > M15 EMA(200)
+        3. H1 price > H1 EMA(200)
+        4. H4 price > H4 EMA(200)
+        5. Current bar hour not in blocked list
+        """
+        cfg = self.config.xau_strategy
+
+        try:
+            bar_hour = pd.Timestamp(report.timestamp).hour
+            if bar_hour in cfg.blocked_hours_utc:
+                return False
+        except Exception:
+            return False
+
+        m15_analysis = None
+        h4_analysis = None
+        for ca in report.confirmation_analyses:
+            if ca.timeframe == Timeframe.M15.value:
+                m15_analysis = ca
+            elif ca.timeframe == Timeframe.H4.value:
+                h4_analysis = ca
+
+        h1_analysis = report.primary_analysis
+
+        if not m15_analysis or not h1_analysis or not h4_analysis:
+            return False
+
+        m15_rsi = m15_analysis.indicators.get("rsi")
+        if not m15_rsi or m15_rsi.value >= cfg.rsi_entry_threshold:
+            return False
+
+        m15_ema200 = m15_analysis.key_levels.get("ema_trend")
+        m15_df = dfs.get(Timeframe.M15)
+        if m15_df is None or m15_ema200 is None:
+            return False
+        m15_price = float(m15_df["Close"].iloc[-1])
+        if m15_price <= m15_ema200:
+            return False
+
+        h1_ema200 = h1_analysis.key_levels.get("ema_trend")
+        if h1_ema200 is None or report.current_price <= h1_ema200:
+            return False
+
+        h4_ema200 = h4_analysis.key_levels.get("ema_trend")
+        h4_df = dfs.get(Timeframe.H4)
+        if h4_df is None or h4_ema200 is None:
+            return False
+        h4_price = float(h4_df["Close"].iloc[-1])
+        if h4_price <= h4_ema200:
+            return False
+
+        return True
+
+    def _generate_xau_trade_setup(
+        self, report: SignalReport, dfs: Dict[Timeframe, pd.DataFrame]
+    ) -> Optional[TradeSetup]:
+        """Generate XAU-specific trade setup with backtested TP/SL."""
+        cfg = self.config.xau_strategy
+
+        m15_analysis = None
+        for ca in report.confirmation_analyses:
+            if ca.timeframe == Timeframe.M15.value:
+                m15_analysis = ca
+                break
+
+        if not m15_analysis:
+            return None
+
+        atr = m15_analysis.key_levels.get("atr", 0)
+        if atr <= 0:
+            return None
+
+        price = report.current_price
+        pip_value = 0.10
+
+        tp1 = price + cfg.tp_atr_multiplier * atr
+        sl = price - cfg.sl_atr_multiplier * atr
+        tp2 = price + cfg.tp_atr_multiplier * 2.0 * atr
+
+        sl_distance = price - sl
+        tp_distance = tp1 - price
+        rr = tp_distance / sl_distance if sl_distance > 0 else 0
+
+        return TradeSetup(
+            pair=report.symbol,
+            display_name=report.display_name,
+            direction="BUY",
+            entry_price=price,
+            stop_loss=round(sl, 2),
+            take_profit_1=round(tp1, 2),
+            take_profit_2=round(tp2, 2),
+            risk_reward=round(rr, 2),
+            risk_pips=round(sl_distance / pip_value, 1),
+            reward_pips=round(tp_distance / pip_value, 1),
+            pip_value=pip_value,
+        )
+
     def _check_mtf_agreement(self, report: SignalReport) -> bool:
-        """Check if multiple timeframes agree on direction."""
         signals = list(report.mtf_signals.values())
         if len(signals) < 2:
             return False
@@ -229,73 +498,157 @@ class SignalAggregator:
         return bullish >= threshold or bearish >= threshold
 
     def _compute_final_signal(self, report: SignalReport):
-        """
-        Compute the final signal by combining all layers.
-
-        Priority:
-        1. Event risk can downgrade or block signals
-        2. MTF agreement boosts confidence
-        3. News sentiment adds a small bias
-        4. Quality grade combines everything
-        """
         score = report.primary_score
         confidence = report.primary_confidence
 
-        # ── MTF adjustment ─────────────────────────────────────────
         if report.mtf_agreement:
-            # All timeframes agree — boost confidence and blend scores
-            confidence = min(confidence + 15, 95)
+            confidence += 10
             all_scores = list(report.mtf_scores.values())
             if all_scores:
                 mtf_avg = sum(all_scores) / len(all_scores)
                 score = score * 0.6 + mtf_avg * 0.4
         else:
-            # Timeframes disagree — reduce confidence
-            confidence *= 0.7
+            confidence -= 15
             if len(report.mtf_scores) >= 2:
                 scores = list(report.mtf_scores.values())
                 has_bull = any(s > 10 for s in scores)
                 has_bear = any(s < -10 for s in scores)
                 if has_bull and has_bear:
                     report.warnings.append("Timeframes CONFLICT — mixed signals across timeframes")
-                    confidence *= 0.6
 
-        # ── News sentiment adjustment ──────────────────────────────
         if abs(report.news_sentiment) > 0.2:
-            # Sentiment aligns with technical
             if (report.news_sentiment > 0 and score > 0) or \
                (report.news_sentiment < 0 and score < 0):
-                confidence = min(confidence + 5, 95)
-            # Sentiment conflicts with technical
+                confidence += 5
             elif (report.news_sentiment > 0 and score < -10) or \
                  (report.news_sentiment < 0 and score > 10):
                 report.warnings.append("News sentiment CONFLICTS with technical signal")
-                confidence *= 0.85
+                confidence -= 10
 
-        # ── Event risk adjustment ──────────────────────────────────
         if report.event_risk == "danger":
             report.warnings.append(f"EVENT RISK: {report.event_description}")
-            confidence *= 0.3  # Heavily penalize — event will disrupt price action
+            confidence = min(confidence, 30)
             report.action_recommendation = f"WAIT — {report.event_description}"
         elif report.event_risk == "caution":
             report.warnings.append(f"Upcoming event: {report.event_description}")
-            confidence *= 0.7
+            confidence -= 15
+
+        is_counter_trend = (
+            (report.market_regime == "trending_up" and score < 0) or
+            (report.market_regime == "trending_down" and score > 0)
+        )
+        if is_counter_trend:
+            confidence -= 15
+            report.warnings.append("Counter-trend signal — reduced confidence")
 
-        # ── Volatility regime adjustment ───────────────────────────
         if report.volatility_regime == "extreme":
             report.warnings.append("EXTREME volatility — wider stops needed, reduced position size")
-            confidence *= 0.8
+            confidence -= 10
         elif report.volatility_regime == "low":
             report.warnings.append("Low volatility — potential for breakout, tight ranges")
 
-        # ── Final signal determination ─────────────────────────────
-        if score >= 40 and confidence >= 65:
+        primary = report.primary_analysis
+        if primary is not None:
+            ema200 = primary.key_levels.get("ema_trend")
+            if ema200 and ema200 > 0 and report.current_price > 0:
+                ema_dist_pct = abs(report.current_price - ema200) / ema200 * 100.0
+                _lo = float(os.environ.get("FB2_EMA_LO", "0.5"))
+                _hi = float(os.environ.get("FB2_EMA_HI", "1.5"))
+                if ema_dist_pct < _lo:
+                    report.warnings.append(
+                        f"NEAR EMA200 ({ema_dist_pct:.2f}%) — chop risk"
+                    )
+                    confidence -= 10
+                elif ema_dist_pct > _hi:
+                    report.warnings.append(
+                        f"FAR FROM EMA200 ({ema_dist_pct:.2f}%) — over-extended"
+                    )
+                    confidence -= 10
+
+            atr = primary.key_levels.get("atr", 0)
+            if atr > 0 and score != 0:
+                is_buy_signal = score > 0
+                if is_buy_signal:
+                    swing_lows = sorted(
+                        v for k, v in primary.key_levels.items()
+                        if k.startswith("swing_low_")
+                    )
+                    if swing_lows and report.current_price - swing_lows[0] > 2.5 * atr:
+                        confidence -= 15
+                        report.warnings.append("TREND EXHAUSTION — price extended >2.5 ATR past swing low")
+                else:
+                    swing_highs = sorted(
+                        (v for k, v in primary.key_levels.items()
+                         if k.startswith("swing_high_")),
+                        reverse=True,
+                    )
+                    if swing_highs and swing_highs[0] - report.current_price > 2.5 * atr:
+                        confidence -= 15
+                        report.warnings.append("TREND EXHAUSTION — price extended >2.5 ATR past swing high")
+
+            ema_medium = primary.key_levels.get("ema_medium")
+            if atr > 0 and ema_medium and report.market_regime.startswith("trending"):
+                pullback_dist = abs(report.current_price - ema_medium) / atr
+                if pullback_dist > 2.0:
+                    confidence -= 20
+                    report.warnings.append(
+                        f"NO PULLBACK — price {pullback_dist:.1f} ATR from EMA21, chasing risk"
+                    )
+                elif pullback_dist > 1.5:
+                    confidence -= 10
+                    report.warnings.append(
+                        f"Weak pullback — price {pullback_dist:.1f} ATR from EMA21"
+                    )
+
+            if report.market_regime.startswith("trending"):
+                rsi_ind = primary.indicators.get("rsi")
+                bb_ind = primary.indicators.get("bollinger")
+
+                if rsi_ind:
+                    rsi_val = rsi_ind.value
+                    if report.market_regime in ("trending_up", "trending") and score > 0:
+                        if rsi_val > 65:
+                            confidence -= 15
+                            report.warnings.append(
+                                f"Extended entry — RSI {rsi_val:.0f} in uptrend (prefer pullback to 40-55)"
+                            )
+                        elif 38 <= rsi_val <= 55:
+                            confidence += 5
+                    elif report.market_regime == "trending_down" and score < 0:
+                        if rsi_val < 35:
+                            confidence -= 15
+                            report.warnings.append(
+                                f"Extended entry — RSI {rsi_val:.0f} in downtrend (prefer bounce to 45-62)"
+                            )
+                        elif 45 <= rsi_val <= 62:
+                            confidence += 5
+
+                if bb_ind:
+                    bb_pos = bb_ind.value
+                    if report.market_regime in ("trending_up", "trending") and score > 0:
+                        if bb_pos > 0.85:
+                            confidence -= 10
+                        elif bb_pos < 0.4:
+                            confidence += 5
+                    elif report.market_regime == "trending_down" and score < 0:
+                        if bb_pos < 0.15:
+                            confidence -= 10
+                        elif bb_pos > 0.6:
+                            confidence += 5
+
+        confidence = max(0.0, min(95.0, confidence))
+
+        # Final signal
+        if score > 38:
+            confidence -= 20
+            report.warnings.append("EXHAUSTION: score > 38 — trend overextended")
+        if score >= 40 and confidence >= 60:
             report.final_signal = SignalStrength.STRONG_BUY
-        elif score >= 15 and confidence >= 50:
+        elif score >= 30 and confidence >= 55:
             report.final_signal = SignalStrength.BUY
-        elif score <= -40 and confidence >= 65:
+        elif score <= -40 and confidence >= 60:
             report.final_signal = SignalStrength.STRONG_SELL
-        elif score <= -15 and confidence >= 50:
+        elif score <= -30 and confidence >= 55:
             report.final_signal = SignalStrength.SELL
         else:
             report.final_signal = SignalStrength.NEUTRAL
@@ -303,23 +656,34 @@ class SignalAggregator:
         report.final_score = score
         report.final_confidence = confidence
 
-        # ── Quality grade ──────────────────────────────────────────
+        # SELL suppression (pair-aware)
+        sell_suppressed = False
+        if self.config.backtest.sell_filter and report.final_signal in (
+            SignalStrength.SELL, SignalStrength.STRONG_SELL
+        ):
+            if report.display_name not in self.config.backtest.sell_allowed_pairs:
+                report.final_signal = SignalStrength.NEUTRAL
+                report.warnings.append(f"SELL SUPPRESSED for {report.display_name}")
+                sell_suppressed = True
+
+        # Quality grade
         abs_score = abs(score)
-        if abs_score >= 50 and confidence >= 75 and report.mtf_agreement:
+        if sell_suppressed:
+            report.signal_quality = "F"
+        elif abs_score >= 45 and confidence >= 70 and report.mtf_agreement:
             report.signal_quality = "A+"
-        elif abs_score >= 35 and confidence >= 65 and report.mtf_agreement:
+        elif abs_score >= 38 and confidence >= 60 and report.mtf_agreement:
             report.signal_quality = "A"
-        elif abs_score >= 25 and confidence >= 55:
+        elif abs_score >= 33 and confidence >= 55:
             report.signal_quality = "B"
-        elif abs_score >= 15 and confidence >= 45:
+        elif abs_score >= 30 and confidence >= 50:
             report.signal_quality = "C"
-        elif abs_score >= 10:
+        elif abs_score >= 15:
             report.signal_quality = "D"
         else:
             report.signal_quality = "F"
 
-        # ── Action recommendation ──────────────────────────────────
-        if not report.action_recommendation:  # not already set by event risk
+        if not report.action_recommendation:
             sig_name = report.final_signal.name.replace("_", " ")
             if report.signal_quality in ("A+", "A"):
                 report.action_recommendation = f"{sig_name} — High-quality setup, consider entry"
@@ -331,36 +695,98 @@ class SignalAggregator:
                 report.action_recommendation = "NO TRADE — No clear signal"
 
     def _generate_trade_setup(self, report: SignalReport, pip_value: float) -> Optional[TradeSetup]:
-        """Generate entry, SL, TP levels for actionable signals."""
         if not report.primary_analysis:
             return None
 
         price = report.current_price
         atr = report.primary_analysis.key_levels.get("atr", 0)
-
         if atr == 0:
             return None
 
-        is_buy = report.final_signal in (SignalStrength.BUY, SignalStrength.STRONG_BUY)
-        rr = self.config.signals.default_rr_ratio
+        min_sl_pips = 8.0 if pip_value < 0.01 else 80.0
+        min_sl_price = min_sl_pips * pip_value
 
-        # SL = 1.5x ATR from entry (adaptive to volatility)
-        sl_distance = atr * 1.5
-        tp1_distance = sl_distance * rr
-        tp2_distance = sl_distance * rr * 1.5
+        if report.final_signal in (SignalStrength.BUY, SignalStrength.STRONG_BUY):
+            is_buy = True
+        elif report.final_signal in (SignalStrength.SELL, SignalStrength.STRONG_SELL):
+            is_buy = False
+        else:
+            is_buy = report.final_score >= 0
+        levels = report.primary_analysis.key_levels
+        min_rr = self.config.signals.min_rr_ratio
+
+        mature_move = abs(report.final_score) > 30
+        tp_mult = 1.5 if mature_move else 2.0
+        tp_swing_cap = 2.5 if mature_move else 3.0
 
         if is_buy:
-            entry = price
-            sl = price - sl_distance
-            tp1 = price + tp1_distance
-            tp2 = price + tp2_distance
+            swing_lows = sorted(
+                [v for k, v in levels.items() if k.startswith("swing_low_") and v < price],
+                reverse=True,
+            )
+            if swing_lows:
+                sl_price = swing_lows[0] - atr * 1.0
+            else:
+                sl_price = price - atr * 2.0
+            min_sl = max(atr * 1.5, min_sl_price)
+            if price - sl_price < min_sl:
+                sl_price = price - min_sl
+
+            sl_distance = price - sl_price
+
+            swing_highs = sorted(
+                v for k, v in levels.items() if k.startswith("swing_high_") and v > price + atr * 0.3
+            )
+            if (swing_highs
+                    and swing_highs[0] - price <= atr * tp_swing_cap
+                    and swing_highs[0] - price >= sl_distance * min_rr):
+                tp1 = swing_highs[0]
+            else:
+                tp1 = price + max(atr * tp_mult, sl_distance * min_rr)
+            if len(swing_highs) >= 2 and swing_highs[1] - price <= atr * 5.0:
+                tp2 = swing_highs[1]
+            else:
+                tp2 = price + max(atr * (tp_mult + 1.5), sl_distance * min_rr * 1.5)
+
             direction = "BUY"
+            entry, sl = price, sl_price
         else:
-            entry = price
-            sl = price + sl_distance
-            tp1 = price - tp1_distance
-            tp2 = price - tp2_distance
+            swing_highs = sorted(
+                v for k, v in levels.items() if k.startswith("swing_high_") and v > price
+            )
+            if swing_highs:
+                sl_price = swing_highs[0] + atr * 1.0
+            else:
+                sl_price = price + atr * 2.0
+            min_sl = max(atr * 1.5, min_sl_price)
+            if sl_price - price < min_sl:
+                sl_price = price + min_sl
+
+            sl_distance = sl_price - price
+
+            swing_lows = sorted(
+                [v for k, v in levels.items() if k.startswith("swing_low_") and v < price - atr * 0.3],
+                reverse=True,
+            )
+            if (swing_lows
+                    and price - swing_lows[0] <= atr * tp_swing_cap
+                    and price - swing_lows[0] >= sl_distance * min_rr):
+                tp1 = swing_lows[0]
+            else:
+                tp1 = price - max(atr * tp_mult, sl_distance * min_rr)
+            if len(swing_lows) >= 2 and price - swing_lows[1] <= atr * 5.0:
+                tp2 = swing_lows[1]
+            else:
+                tp2 = price - max(atr * (tp_mult + 1.5), sl_distance * min_rr * 1.5)
+
             direction = "SELL"
+            entry, sl = price, sl_price
+
+        tp1_distance = abs(tp1 - entry)
+        rr = tp1_distance / sl_distance if sl_distance > 0 else 0
+
+        if rr < min_rr - 1e-9:
+            return None
 
         risk_pips = sl_distance / pip_value
         reward_pips = tp1_distance / pip_value
@@ -373,28 +799,19 @@ class SignalAggregator:
             stop_loss=sl,
             take_profit_1=tp1,
             take_profit_2=tp2,
-            risk_reward=rr,
+            risk_reward=round(rr, 2),
             risk_pips=risk_pips,
             reward_pips=reward_pips,
             pip_value=pip_value,
         )
 
     def _collect_reasons(self, report: SignalReport):
-        """
-        Collect bullish/bearish reasons from all indicators.
-
-        Each indicator's description is captured once via the indicators
-        dict. Pattern signals (candlestick) are already included in the
-        'candlestick' indicator description, so we do NOT re-process
-        pattern_signals to avoid duplicate entries.
-        """
         if not report.primary_analysis:
             return
 
         for name, ind in report.primary_analysis.indicators.items():
             if name.startswith("_") or ind.weight == 0:
                 continue
-
             if ind.signal in (SignalStrength.BUY, SignalStrength.STRONG_BUY):
                 prefix = "STRONG: " if ind.signal == SignalStrength.STRONG_BUY else ""
                 report.bullish_reasons.append(f"{prefix}{ind.name}: {ind.description}")
@@ -402,7 +819,6 @@ class SignalAggregator:
                 prefix = "STRONG: " if ind.signal == SignalStrength.STRONG_SELL else ""
                 report.bearish_reasons.append(f"{prefix}{ind.name}: {ind.description}")
 
-        # Add news context
         if report.news_sentiment > 0.1:
             report.bullish_reasons.append(f"News: {report.news_description}")
         elif report.news_sentiment < -0.1:

telegram_bot.py

@@ -0,0 +1,842 @@
+#!/usr/bin/env python3
+"""
+ForexBot2 — Telegram Bot
+========================
+Async Telegram interface with candle-boundary auto-scanner and
+real-time WebSocket level-breach monitor.
+
+Manual scan: 4 pairs × 3 timeframes (M15/H1/H4) via inline buttons.
+Auto-scan:   Starts automatically on boot if TELEGRAM_CHAT_ID is set.
+             /autoscan toggles for additional chats or to stop.
+             M5  fires at :00/:05/:10/… + 10s buffer (data only, no signals).
+             M15 fires at :00/:15/:30/:45 + 10s buffer.
+             H1  fires at :00 + 10s buffer.
+             H4  fires at :00 every 4 hours + 10s buffer.
+WebSocket:   Streams live ticks from TwelveData for EUR/USD & XAU/USD
+             (Grow plan supports only these 2 WS symbols).
+             When price crosses a key structure level (swing S/R, BB,
+             EMA200) cached from the last boundary scan, triggers an
+             immediate M15 scan and sends a Level Break Alert.
+             5-minute cooldown per pair prevents spam.
+             GBP/USD & USD/JPY are monitored via REST scans only.
+
+Plan: TwelveData Grow (55 credits/min, 8 trial WS credits).
+Credit budget (worst case, 4-hour boundary):
+    Unique fetches: 4 pairs × ~6 TFs = ~24 calls (rest cached) → fits in 55/min.
+    WebSocket: 2 symbols = 2 credits (one-time on connect).
+    Level breach scans: ~3 REST calls each, ~10/hour worst case.
+
+Usage:
+    python telegram_bot.py
+    TELEGRAM_TOKEN=<token> TELEGRAM_CHAT_ID=<id> python telegram_bot.py
+"""
+
+import asyncio
+import json
+import logging
+import os
+import sys
+import time
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime, timezone
+from typing import Dict, List, Optional
+
+import websockets
+
+from telegram import InlineKeyboardButton, InlineKeyboardMarkup, Update
+from telegram.constants import ParseMode
+from telegram.ext import (
+    Application,
+    CallbackQueryHandler,
+    CommandHandler,
+    ContextTypes,
+)
+
+from config import (
+    PlatformConfig, SignalStrength, Timeframe,
+    EURUSD, GBPUSD, XAUUSD, PAIR_INFO,
+)
+from data_provider import ForexDataProvider
+from signal_aggregator import SignalAggregator, SignalReport
+
+# ── Force UTF-8 on Windows ────────────────────────────────────────────
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8")
+
+logging.basicConfig(
+    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+    level=logging.INFO,
+    datefmt="%H:%M:%S",
+)
+logging.getLogger("httpx").setLevel(logging.WARNING)
+logging.getLogger("httpcore").setLevel(logging.WARNING)
+logger = logging.getLogger("telegram_bot")
+
+TOKEN = os.environ.get("TELEGRAM_TOKEN", "8228251480:AAFW2OAyQ2idMicql2t_CIXk1r9-qfRTfNE")
+CHAT_ID = os.environ.get("TELEGRAM_CHAT_ID", "6025899022")
+
+# ── Pairs & buttons ───────────────────────────────────────────────────
+
+SCAN_PAIRS = [EURUSD, GBPUSD, XAUUSD]
+
+CORRELATION_GROUPS = {
+    "usd_negative": {"pairs": {EURUSD, GBPUSD}, "label": "USD-weakness"},
+}
+
+_WS_TO_PAIR = {
+    "EUR/USD": EURUSD,
+    "XAU/USD": XAUUSD,
+}
+_WS_SYMBOLS = list(_WS_TO_PAIR.keys())
+
+_LEVEL_KEYS = (
+    "swing_high_1", "swing_high_2", "swing_high_3",
+    "swing_low_1", "swing_low_2", "swing_low_3",
+    "ema_trend",
+)
+
+SCAN_BUTTONS = [
+    {"cb": "scan_eurusd_m15", "symbol": EURUSD, "tf": Timeframe.M15, "label": "EUR/USD 15M"},
+    {"cb": "scan_eurusd_h1",  "symbol": EURUSD, "tf": Timeframe.H1,  "label": "EUR/USD 1H"},
+    {"cb": "scan_eurusd_h4",  "symbol": EURUSD, "tf": Timeframe.H4,  "label": "EUR/USD 4H"},
+{"cb": "scan_gbpusd_m15", "symbol": GBPUSD, "tf": Timeframe.M15, "label": "GBP/USD 15M"},
+    {"cb": "scan_gbpusd_h1",  "symbol": GBPUSD, "tf": Timeframe.H1,  "label": "GBP/USD 1H"},
+    {"cb": "scan_gbpusd_h4",  "symbol": GBPUSD, "tf": Timeframe.H4,  "label": "GBP/USD 4H"},
+    {"cb": "scan_xauusd_m15", "symbol": XAUUSD, "tf": Timeframe.M15, "label": "XAU/USD 15M"},
+    {"cb": "scan_xauusd_h1",  "symbol": XAUUSD, "tf": Timeframe.H1,  "label": "XAU/USD 1H"},
+    {"cb": "scan_xauusd_h4",  "symbol": XAUUSD, "tf": Timeframe.H4,  "label": "XAU/USD 4H"},
+]
+
+SCAN_LOOKUP = {b["cb"]: b for b in SCAN_BUTTONS}
+
+TF_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+AUTOSCAN_CONFIRMATIONS = {
+    Timeframe.M5:  [Timeframe.M15, Timeframe.H1],
+    Timeframe.M15: [Timeframe.M5, Timeframe.H1],
+    Timeframe.H1:  [Timeframe.M15, Timeframe.H4],
+    Timeframe.H4:  [Timeframe.H1, Timeframe.D1],
+}
+
+_executor = ThreadPoolExecutor(max_workers=2)
+
+
+# ── Shared data provider (single cache across all scans) ─────────────
+
+_shared_dp: Optional[ForexDataProvider] = None
+
+
+def _get_shared_dp() -> ForexDataProvider:
+    global _shared_dp
+    if _shared_dp is None:
+        _shared_dp = ForexDataProvider(PlatformConfig())
+    return _shared_dp
+
+
+# ── Signal formatting ─────────────────────────────────────────────────
+
+_SIGNAL_EMOJI = {
+    SignalStrength.STRONG_BUY:  "🟢",
+    SignalStrength.BUY:         "🟩",
+    SignalStrength.NEUTRAL:     "⬜",
+    SignalStrength.SELL:        "🟥",
+    SignalStrength.STRONG_SELL: "🔴",
+}
+
+_QUALITY_EMOJI = {
+    "A+": "🏆", "A": "🥇", "B": "🥈", "C": "🥉", "D": "⚠️", "F": "❌",
+}
+
+_SIGNAL_TAG = {
+    SignalStrength.STRONG_BUY:  "Strong Buy",
+    SignalStrength.BUY:         "Bullish",
+    SignalStrength.NEUTRAL:     "Neutral",
+    SignalStrength.SELL:        "Bearish",
+    SignalStrength.STRONG_SELL: "Strong Sell",
+}
+
+
+def _signal_label(sig: SignalStrength) -> str:
+    return sig.name.replace("_", " ")
+
+
+def _mtf_line(report: SignalReport) -> str:
+    if not report.mtf_signals:
+        return ""
+    parts = []
+    for tf, sig in report.mtf_signals.items():
+        score = report.mtf_scores.get(tf, 0)
+        parts.append(f"{tf}: {_signal_label(sig)} {score:+.1f}")
+    return " | ".join(parts)
+
+
+def _indicator_line(name: str, ind) -> str:
+    tag = _SIGNAL_TAG.get(ind.signal, "Neutral")
+    short = ind.description.split(" | ")[0] if " | " in ind.description else ind.description
+    return f"[{tag}] *{ind.name}*: {short}"
+
+
+def format_signal_message(report: SignalReport) -> str:
+    sig = report.final_signal
+    emoji = _SIGNAL_EMOJI.get(sig, "⬜")
+    qe = _QUALITY_EMOJI.get(report.signal_quality, "")
+    label = _signal_label(sig)
+    mtf_tag = "✅ MTF aligned" if report.mtf_agreement else "❌ MTF diverged"
+
+    lines: list[str] = []
+
+    now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
+    lines += [
+        f"🔍 *{report.display_name} Signal Analysis*  ({report.primary_analysis.timeframe if report.primary_analysis else ''})",
+        f"🕐 {now}",
+        "",
+        f"💰 Price: *{report.current_price:.5f}*" if report.current_price < 10 else f"💰 Price: *{report.current_price:.3f}*",
+        f"📊 Regime: {report.market_regime.replace('_', ' ').title()} | Vol: {report.volatility_regime}",
+        "",
+        "━━━━━━━━━━━━━━━━━━━━",
+        f"{emoji} *{label}*  {qe}{report.signal_quality}  |  Score: {report.final_score:+.1f}  |  {report.final_confidence:.0f}% conf",
+        f"_{report.action_recommendation}_",
+        f"{mtf_tag}",
+    ]
+
+    mtf = _mtf_line(report)
+    if mtf:
+        lines += ["", f"📈 *MTF*: {mtf}"]
+
+    if report.primary_analysis:
+        lines += ["", "━━━━━━━━━━━━━━━━━━━━", "📋 *Indicators*"]
+        for name, ind in report.primary_analysis.indicators.items():
+            if name.startswith("_") or ind.weight == 0:
+                continue
+            lines.append(_indicator_line(name, ind))
+
+    if report.warnings:
+        lines.append("")
+        for w in report.warnings:
+            lines.append(f"⚠️ {w}")
+
+    if report.trade_setup and sig not in (SignalStrength.NEUTRAL,):
+        ts = report.trade_setup
+        dir_emoji = "📈" if ts.direction == "BUY" else "📉"
+        lines += [
+            "",
+            "━━━━━━━━━━━━━━━━━━━━",
+            f"🚀 *TRADE SETUP*",
+            f"{dir_emoji} Direction: *{ts.direction}*",
+            f"▶ Entry:  `{ts.entry_price:.3f}`",
+            f"🛑 SL:     `{ts.stop_loss:.3f}`  ({ts.risk_pips:.1f} pips)",
+            f"🎯 TP1:    `{ts.take_profit_1:.3f}`  ({ts.reward_pips:.1f} pips)",
+        ]
+        if ts.take_profit_2:
+            lines.append(f"🎯 TP2:    `{ts.take_profit_2:.3f}`")
+        lines.append(f"⚖️  RR:     1:{ts.risk_reward:.1f}")
+
+    lines += ["", "━━━━━━━━━━━━━━━━━━━━",
+              "_Educational / research use only._"]
+
+    return "\n".join(lines)
+
+
+# ── Correlation filter ───────────────────────────────────────────────
+
+def _detect_correlated_exposure(reports: List[SignalReport]) -> List[str]:
+    warnings = []
+    for group_info in CORRELATION_GROUPS.values():
+        group_pairs = group_info["pairs"]
+        group_label = group_info["label"]
+        active = [
+            r for r in reports
+            if r.symbol in group_pairs
+            and r.final_signal not in (SignalStrength.NEUTRAL,)
+        ]
+        if len(active) >= 2:
+            directions = set()
+            for r in active:
+                if r.final_signal in (SignalStrength.BUY, SignalStrength.STRONG_BUY):
+                    directions.add("BUY")
+                else:
+                    directions.add("SELL")
+            if len(directions) == 1:
+                names = ", ".join(r.display_name for r in active)
+                warnings.append(
+                    f"⚠️ Same exposure ({group_label}): {names} both {directions.pop()}"
+                )
+    return warnings
+
+
+# ── Auto-scan summary format ─────────────────────────────────────────
+
+def _format_scan_summary(reports: List[SignalReport], tf: Timeframe) -> str:
+    now = datetime.now(timezone.utc).strftime("%H:%M UTC")
+    lines = [
+        f"📊 *{tf.value} Auto-Scan* | {now}",
+        "━━━━━━━━━━━━━━━━━━━━",
+    ]
+    for r in reports:
+        emoji = _SIGNAL_EMOJI.get(r.final_signal, "⬜")
+        sig = _signal_label(r.final_signal)
+        mtf = "✅" if r.mtf_agreement else "❌"
+        lines.append(
+            f"{emoji} *{r.display_name}*  {sig}  "
+            f"{_QUALITY_EMOJI.get(r.signal_quality, '')}{r.signal_quality}  "
+            f"{r.final_score:+.1f}  {r.final_confidence:.0f}%  {mtf}MTF"
+        )
+    lines.append("━━━━━━━━━━━━━━━━━━━━")
+
+    actionable = [r for r in reports if r.signal_quality in ("A+", "A", "B", "C")]
+    if actionable:
+        names = ", ".join(r.display_name for r in actionable)
+        lines.append(f"🔔 Actionable: {names}")
+    else:
+        lines.append("No actionable signals this cycle.")
+
+    corr_warnings = _detect_correlated_exposure(reports)
+    for w in corr_warnings:
+        lines.append(w)
+
+    return "\n".join(lines)
+
+
+# ── Keyboard helper ───────────────────────────────────────────────────
+
+def scan_keyboard() -> InlineKeyboardMarkup:
+    rows = []
+    for i in range(0, len(SCAN_BUTTONS), 4):
+        rows.append([
+            InlineKeyboardButton(b["label"], callback_data=b["cb"])
+            for b in SCAN_BUTTONS[i:i + 4]
+        ])
+    return InlineKeyboardMarkup(rows)
+
+
+# ── Signal runner (shared data provider for cache reuse) ──────────────
+
+def _run_signal_sync(
+    symbol: str,
+    timeframe: Timeframe,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+) -> SignalReport:
+    if confirmation_tfs is None:
+        confirmation_tfs = TF_CONFIRMATIONS.get(
+            timeframe, [Timeframe.M15, Timeframe.H4]
+        )
+    config = PlatformConfig()
+    config.primary_timeframe = timeframe
+    config.confirmation_timeframes = confirmation_tfs
+    aggregator = SignalAggregator(config)
+    aggregator.data_provider = _get_shared_dp()
+    return aggregator.analyze_pair(symbol=symbol)
+
+
+async def run_signal(
+    symbol: str,
+    timeframe: Timeframe,
+    confirmation_tfs: Optional[List[Timeframe]] = None,
+) -> SignalReport:
+    loop = asyncio.get_event_loop()
+    return await loop.run_in_executor(
+        _executor, _run_signal_sync, symbol, timeframe, confirmation_tfs
+    )
+
+
+# ── Candle-boundary scheduler ────────────────────────────────────────
+
+_autoscan_chats: set[int] = set()
+_autoscan_tasks: list[asyncio.Task] = []
+
+
+def _seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
+    """Seconds until the next candle close + buffer."""
+    now = datetime.now(timezone.utc)
+    total_minutes = now.hour * 60 + now.minute
+    elapsed_minutes = total_minutes % interval_minutes
+    elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
+    interval_seconds = interval_minutes * 60
+    remaining = interval_seconds - elapsed_seconds + buffer
+    if remaining <= buffer:
+        remaining += interval_seconds
+    return remaining
+
+
+async def _scan_timeframe(
+    bot,
+    tf: Timeframe,
+    confirmation_tfs: List[Timeframe],
+) -> None:
+    """Scan all pairs for one timeframe and broadcast results."""
+    reports: List[SignalReport] = []
+    for symbol in SCAN_PAIRS:
+        try:
+            report = await run_signal(symbol, tf, confirmation_tfs)
+            reports.append(report)
+        except Exception:
+            logger.exception(f"Auto-scan failed for {symbol} {tf.value}")
+
+    if not reports:
+        return
+
+    _update_level_cache(reports)
+
+    if tf == Timeframe.M5:
+        logger.info("M5 scan: data analysis only — no Telegram broadcast")
+        return
+
+    actionable = [r for r in reports if r.signal_quality in ("A+", "A", "B", "C")]
+
+    if not actionable:
+        logger.info(f"{tf.value} scan: no actionable signals — skipping Telegram message")
+        return
+
+    summary = _format_scan_summary(reports, tf)
+
+    for chat_id in list(_autoscan_chats):
+        try:
+            await bot.send_message(chat_id, summary, parse_mode=ParseMode.MARKDOWN)
+            for report in actionable:
+                detail = format_signal_message(report)
+                await bot.send_message(chat_id, detail, parse_mode=ParseMode.MARKDOWN)
+        except Exception:
+            logger.exception(f"Failed to send to chat {chat_id}")
+
+
+# ── WebSocket level-breach monitor ──────────────────────────────────
+
+_level_cache: Dict[str, Dict[str, float]] = {}
+_last_prices: Dict[str, float] = {}
+_scan_cooldowns: Dict[str, float] = {}
+
+_cfg = PlatformConfig()
+_WS_API_KEY = os.environ.get("TWELVE_DATA_API_KEY") or _cfg.twelve_data_api_key
+_WS_URL = f"wss://ws.twelvedata.com/v1/quotes/price?apikey={_WS_API_KEY}"
+_WS_COOLDOWN = _cfg.ws_cooldown_seconds
+
+
+def _update_level_cache(reports: List[SignalReport]) -> None:
+    for r in reports:
+        if r.primary_analysis and r.primary_analysis.key_levels:
+            levels = {
+                k: v for k, v in r.primary_analysis.key_levels.items()
+                if k in _LEVEL_KEYS
+            }
+            if levels:
+                _level_cache[r.symbol] = levels
+
+
+def _check_level_breach(symbol: str, price: float) -> Optional[str]:
+    levels = _level_cache.get(symbol)
+    if not levels:
+        return None
+
+    last = _last_prices.get(symbol)
+    if last is None:
+        return None
+
+    for name, level in levels.items():
+        if (last < level <= price) or (last > level >= price):
+            return name
+
+    return None
+
+
+async def _handle_breach(bot, ws_symbol: str, pair_symbol: str, price: float, level_name: str) -> None:
+    now = time.time()
+    if now - _scan_cooldowns.get(pair_symbol, 0) < _WS_COOLDOWN:
+        return
+
+    _scan_cooldowns[pair_symbol] = now
+    display = PAIR_INFO.get(pair_symbol, {}).get("display", ws_symbol)
+    label = level_name.replace("_", " ").title()
+    logger.info(f"Level breach: {display} crossed {label} at {price}")
+
+    try:
+        report = await run_signal(pair_symbol, Timeframe.M15)
+        _update_level_cache([report])
+        text = (
+            f"🔔 *Level Break — {display}*\n"
+            f"Price crossed *{label}* at `{price}`\n"
+            f"━━━━━━━━━━━━━━━━━━━━\n"
+            + format_signal_message(report)
+        )
+    except Exception as exc:
+        logger.exception(f"Breach scan failed for {display}")
+        text = f"🔔 *Level Break — {display}*\nPrice crossed {label} at `{price}`\n❌ Scan failed: {exc}"
+
+    for chat_id in list(_autoscan_chats):
+        try:
+            await bot.send_message(chat_id, text, parse_mode=ParseMode.MARKDOWN)
+        except Exception:
+            logger.exception(f"Failed to send breach alert to {chat_id}")
+
+
+async def _ws_monitor(bot) -> None:
+    backoff = 1
+    while _autoscan_chats:
+        try:
+            async with websockets.connect(_WS_URL) as ws:
+                subscribe = {
+                    "action": "subscribe",
+                    "params": {"symbols": ",".join(_WS_SYMBOLS)},
+                }
+                await ws.send(json.dumps(subscribe))
+                logger.info(f"WebSocket connected, subscribed to {len(_WS_SYMBOLS)} symbols")
+                backoff = 1
+
+                async for raw in ws:
+                    if not _autoscan_chats:
+                        break
+
+                    msg = json.loads(raw)
+                    if msg.get("event") != "price":
+                        continue
+
+                    ws_symbol = msg.get("symbol")
+                    pair_symbol = _WS_TO_PAIR.get(ws_symbol)
+                    if not pair_symbol:
+                        continue
+
+                    price = float(msg["price"])
+                    breached = _check_level_breach(pair_symbol, price)
+                    _last_prices[pair_symbol] = price
+
+                    if breached:
+                        asyncio.create_task(
+                            _handle_breach(bot, ws_symbol, pair_symbol, price, breached)
+                        )
+
+        except asyncio.CancelledError:
+            break
+        except Exception:
+            logger.exception(f"WebSocket error, reconnecting in {backoff}s")
+            await asyncio.sleep(backoff)
+            backoff = min(backoff * 2, 60)
+
+    logger.info("WebSocket monitor stopped")
+
+
+async def _candle_loop(bot, tf: Timeframe, interval_minutes: int) -> None:
+    label = tf.value
+    try:
+        while _autoscan_chats:
+            wait = _seconds_until_boundary(interval_minutes, buffer=10)
+            logger.info(f"{label} auto-scan: next in {wait:.0f}s")
+            await asyncio.sleep(wait)
+            if not _autoscan_chats:
+                break
+            logger.info(f"{label} auto-scan: firing")
+            await _scan_timeframe(
+                bot, tf, AUTOSCAN_CONFIRMATIONS[tf]
+            )
+    except asyncio.CancelledError:
+        pass
+    finally:
+        logger.info(f"{label} auto-scan loop stopped")
+
+
+_AUTOSCAN_LOOPS = [
+    (Timeframe.M5,  5),
+    (Timeframe.M15, 15),
+    (Timeframe.H1,  60),
+    (Timeframe.H4,  240),
+]
+
+
+_ws_task: Optional[asyncio.Task] = None
+
+
+def _start_scheduler(bot) -> None:
+    global _ws_task
+    alive = [t for t in _autoscan_tasks if not t.done()]
+    if not alive:
+        _autoscan_tasks.clear()
+        for tf, interval in _AUTOSCAN_LOOPS:
+            _autoscan_tasks.append(
+                asyncio.create_task(_candle_loop(bot, tf, interval))
+            )
+        logger.info("Candle-boundary scheduler started (M5/M15/H1/H4)")
+
+    if _ws_task is None or _ws_task.done():
+        _ws_task = asyncio.create_task(_ws_monitor(bot))
+        logger.info("WebSocket level-breach monitor started")
+
+
+def _stop_scheduler() -> None:
+    global _ws_task
+    for t in _autoscan_tasks:
+        t.cancel()
+    _autoscan_tasks.clear()
+    if _ws_task and not _ws_task.done():
+        _ws_task.cancel()
+    _ws_task = None
+    logger.info("Candle-boundary scheduler and WebSocket monitor stopped")
+
+
+# ── Handlers ──────────────────────────────────────────────────────────
+
+async def cmd_start(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    scanning = "running" if _autoscan_chats else "off"
+    text = (
+        "👋 *ForexBot2 — Signal Intelligence*\n\n"
+        "Pick a pair and timeframe to scan.\n"
+        "Signals include a full trade setup with entry, SL and TP.\n\n"
+        f"Auto-scan: *{scanning}* — /autoscan to toggle\n"
+        "/status — Bot & WebSocket status\n"
+        "/help — All commands"
+    )
+    await update.message.reply_text(
+        text,
+        parse_mode=ParseMode.MARKDOWN,
+        reply_markup=scan_keyboard(),
+    )
+
+
+async def cmd_help(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    text = (
+        "📖 *ForexBot2 Commands*\n\n"
+        "/start — Show scan buttons\n"
+        "/autoscan — Toggle auto-scan on/off\n"
+        "/status — Bot status, WS connection, live prices\n"
+        "/cache — Show when each pair/TF was last fetched\n"
+        "/clearcache — Wipe data cache, force fresh fetch\n"
+        "/help — This message\n\n"
+        "*Auto-scan* fires at candle close: M5/M15/H1/H4.\n"
+        "*WebSocket* monitors live prices and sends Level Break Alerts "
+        "when price crosses swing S/R, Bollinger bands, or EMA200."
+    )
+    await update.message.reply_text(text, parse_mode=ParseMode.MARKDOWN)
+
+
+async def cmd_status(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    lines = ["📊 *ForexBot2 Status*\n"]
+
+    if _autoscan_chats:
+        lines.append(f"✅ Auto-scan: *running* ({len(_autoscan_chats)} chat)")
+        tasks_alive = sum(1 for t in _autoscan_tasks if not t.done())
+        lines.append(f"   Candle loops: {tasks_alive}/{len(_AUTOSCAN_LOOPS)}")
+    else:
+        lines.append("⏹ Auto-scan: *stopped*")
+
+    if _ws_task and not _ws_task.done():
+        lines.append("📡 WebSocket: *connected*")
+    else:
+        lines.append("❌ WebSocket: *disconnected*")
+
+    if _last_prices:
+        lines.append("\n*Live Prices (WebSocket):*")
+        for ws_sym, pair in _WS_TO_PAIR.items():
+            price = _last_prices.get(pair)
+            display = PAIR_INFO.get(pair, {}).get("display", ws_sym)
+            if price is None:
+                lines.append(f"  {display}: waiting...")
+            elif price < 10:
+                lines.append(f"  {display}: `{price:.5f}`")
+            else:
+                lines.append(f"  {display}: `{price:.3f}`")
+    else:
+        lines.append("\n_No live prices yet (WS not streaming)_")
+
+    cached_pairs = len(_level_cache)
+    cached_levels = sum(len(v) for v in _level_cache.values())
+    if cached_levels:
+        lines.append(f"\n*Level Cache:* {cached_levels} levels across {cached_pairs} pairs")
+    else:
+        lines.append("\n*Level Cache:* empty — breach alerts start after first candle scan")
+
+    active_cooldowns = sum(
+        1 for v in _scan_cooldowns.values()
+        if time.time() - v < _WS_COOLDOWN
+    )
+    if active_cooldowns:
+        lines.append(f"*Cooldowns:* {active_cooldowns} pair(s) in cooldown")
+
+    await update.message.reply_text("\n".join(lines), parse_mode=ParseMode.MARKDOWN)
+
+
+async def cmd_cache(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    dp = _get_shared_dp()
+    if not dp._cache_timestamps:
+        await update.message.reply_text("📭 Cache is empty — no data fetched yet.")
+        return
+
+    now = time.time()
+    lines = ["📦 *Data Cache Status*\n"]
+
+    for pair in SCAN_PAIRS:
+        display = PAIR_INFO.get(pair, {}).get("display", pair)
+        pair_lines = []
+        for tf in [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4, Timeframe.D1]:
+            cache_key = f"{pair}_{tf.value}"
+            ts = dp._cache_timestamps.get(cache_key)
+            if ts is None:
+                continue
+            age_min = (now - ts) / 60
+            df = dp._cache.get(cache_key)
+            last_bar = df.index[-1].strftime("%H:%M") if df is not None and not df.empty else "?"
+            fresh = "✅" if age_min < dp.CANDLE_SECONDS.get(tf, 3600) / 60 else "⚠️"
+            pair_lines.append(f"  {tf.value:<5} {fresh} fetched `{age_min:.0f}m` ago | last bar `{last_bar}`")
+        if pair_lines:
+            lines.append(f"*{display}*")
+            lines.extend(pair_lines)
+
+    await update.message.reply_text("\n".join(lines), parse_mode=ParseMode.MARKDOWN)
+
+
+async def cmd_clearcache(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    dp = _get_shared_dp()
+    count = dp.clear_cache()
+    await update.message.reply_text(
+        f"🗑 Cache cleared ({count} entries wiped).\nNext scan will fetch fresh data from API.",
+        parse_mode=ParseMode.MARKDOWN,
+    )
+
+
+async def cmd_autoscan(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    chat_id = update.effective_chat.id
+
+    if chat_id in _autoscan_chats:
+        _autoscan_chats.discard(chat_id)
+        if not _autoscan_chats:
+            _stop_scheduler()
+        await update.message.reply_text(
+            "⏹ Auto-scan stopped.",
+            parse_mode=ParseMode.MARKDOWN,
+        )
+        return
+
+    _autoscan_chats.add(chat_id)
+    _start_scheduler(context.bot)
+
+    next_m5 = _seconds_until_boundary(5, buffer=10)
+    next_m15 = _seconds_until_boundary(15, buffer=10)
+    next_h1 = _seconds_until_boundary(60, buffer=10)
+    next_h4 = _seconds_until_boundary(240, buffer=10)
+    await update.message.reply_text(
+        "✅ *Auto-scan enabled*\n\n"
+        f"Scanning {len(SCAN_PAIRS)} pairs on M5/M15/H1/H4 at candle close.\n"
+        f"📡 Live price monitoring active — level-break alerts enabled.\n"
+        f"Next: M5 {next_m5 / 60:.0f}m | M15 {next_m15 / 60:.0f}m | "
+        f"H1 {next_h1 / 60:.0f}m | H4 {next_h4 / 60:.0f}m\n\n"
+        "Use /autoscan again to stop.",
+        parse_mode=ParseMode.MARKDOWN,
+    )
+
+
+async def cb_scan(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
+    query = update.callback_query
+    await query.answer()
+
+    btn = SCAN_LOOKUP.get(query.data)
+    if not btn:
+        return
+
+    pair_display = btn["label"]
+    await query.edit_message_text(
+        f"⏳ Scanning {pair_display}…",
+        parse_mode=ParseMode.MARKDOWN,
+    )
+
+    try:
+        report = await run_signal(btn["symbol"], btn["tf"])
+        text = format_signal_message(report)
+    except Exception as exc:
+        logger.exception("Signal scan failed")
+        text = f"❌ Scan failed: {exc}"
+
+    await query.edit_message_text(
+        text,
+        parse_mode=ParseMode.MARKDOWN,
+        reply_markup=scan_keyboard(),
+    )
+
+
+# ── Boot-time auto-start ──────────────────────────────────────────────
+
+async def _post_init(app: Application) -> None:
+    """Register command menu and start scheduler on boot."""
+    await app.bot.set_my_commands([
+        ("start", "Show scan buttons"),
+        ("autoscan", "Toggle auto-scan on/off"),
+        ("status", "Bot status, WS connection, live prices"),
+        ("cache", "Show data cache age per pair/TF"),
+        ("clearcache", "Wipe data cache, force fresh fetch"),
+        ("help", "All commands"),
+    ])
+
+    if not CHAT_ID:
+        logger.info("TELEGRAM_CHAT_ID not set — auto-scan disabled on boot. "
+                     "Use /autoscan in chat to enable.")
+        return
+
+    chat_id = int(CHAT_ID)
+    _autoscan_chats.add(chat_id)
+
+    logger.info("Running initial scan to populate level cache for WebSocket monitor...")
+    try:
+        init_reports = []
+        for symbol in SCAN_PAIRS:
+            report = await run_signal(symbol, Timeframe.M15)
+            init_reports.append(report)
+        _update_level_cache(init_reports)
+        logger.info(f"Level cache populated: {sum(len(v) for v in _level_cache.values())} levels")
+    except Exception:
+        logger.exception("Initial scan failed — WS will wait for first candle scan")
+
+    _start_scheduler(app.bot)
+
+    next_m5 = _seconds_until_boundary(5, buffer=10)
+    next_m15 = _seconds_until_boundary(15, buffer=10)
+    next_h1 = _seconds_until_boundary(60, buffer=10)
+    next_h4 = _seconds_until_boundary(240, buffer=10)
+    logger.info(
+        f"Auto-scan started for chat {chat_id} — "
+        f"M5 {next_m5:.0f}s, M15 {next_m15:.0f}s, H1 {next_h1:.0f}s, H4 {next_h4:.0f}s"
+    )
+
+    try:
+        await app.bot.send_message(
+            chat_id,
+            f"🤖 *ForexBot2 started*\n\n"
+            f"Auto-scanning {len(SCAN_PAIRS)} pairs on M5/M15/H1/H4.\n"
+            f"📡 Live price monitoring active — level-break alerts enabled.\n"
+            f"Next: M5 {next_m5 / 60:.0f}m | M15 {next_m15 / 60:.0f}m | "
+            f"H1 {next_h1 / 60:.0f}m | H4 {next_h4 / 60:.0f}m\n\n"
+            f"/autoscan to stop | Manual buttons via /start",
+            parse_mode=ParseMode.MARKDOWN,
+        )
+    except Exception:
+        logger.exception("Failed to send boot message")
+
+
+# ── Entry point ───────────────────────────────────────────────────────
+
+def main() -> None:
+    if not TOKEN:
+        sys.exit("TELEGRAM_TOKEN not set")
+
+    logger.info("Starting ForexBot2 Telegram bot…")
+
+    app = (
+        Application.builder()
+        .token(TOKEN)
+        .post_init(_post_init)
+        .build()
+    )
+
+    app.add_handler(CommandHandler("start", cmd_start))
+    app.add_handler(CommandHandler("help", cmd_help))
+    app.add_handler(CommandHandler("status", cmd_status))
+    app.add_handler(CommandHandler("autoscan", cmd_autoscan))
+    app.add_handler(CommandHandler("cache", cmd_cache))
+    app.add_handler(CommandHandler("clearcache", cmd_clearcache))
+    app.add_handler(CallbackQueryHandler(cb_scan, pattern="^scan_"))
+
+    logger.info("Bot running — press Ctrl+C to stop")
+    app.run_polling(allowed_updates=Update.ALL_TYPES)
+
+
+if __name__ == "__main__":
+    main()

ws_twelvedata.py

@@ -0,0 +1,55 @@
+"""Live tick stream from Twelve Data WebSocket API."""
+
+import json
+import asyncio
+import websockets
+from datetime import datetime, timezone
+
+
+API_KEY = "f563b22ffdfd4924955decbe1a8c8701"
+WS_URL = f"wss://ws.twelvedata.com/v1/quotes/price?apikey={API_KEY}"
+SYMBOLS = ["EUR/USD", "XAU/USD"]
+
+
+async def stream_ticks():
+    async with websockets.connect(WS_URL) as ws:
+        subscribe = {
+            "action": "subscribe",
+            "params": {
+                "symbols": ",".join(SYMBOLS),
+            },
+        }
+        await ws.send(json.dumps(subscribe))
+        print(f"Subscribed to: {', '.join(SYMBOLS)}")
+        print("-" * 60)
+
+        async for raw in ws:
+            msg = json.loads(raw)
+
+            if msg.get("event") == "subscribe-status":
+                print(f"Status: {msg.get('status')} — {msg.get('message', '')}")
+                continue
+
+            if msg.get("event") == "heartbeat":
+                continue
+
+            if msg.get("event") == "price":
+                ts = datetime.fromtimestamp(msg["timestamp"], tz=timezone.utc)
+                symbol = msg["symbol"]
+                price = msg["price"]
+                print(f"{ts:%H:%M:%S} | {symbol:<8} | {price}")
+
+
+async def main():
+    print(f"Connecting to Twelve Data WebSocket...")
+    print(f"API key: ...{API_KEY[-6:]}")
+    try:
+        await stream_ticks()
+    except KeyboardInterrupt:
+        print("\nStopped.")
+    except websockets.exceptions.ConnectionClosed as e:
+        print(f"\nConnection closed: {e}")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())