GPU backtester, optimised weights, local CSV data pipeline

## What changed ### backtest_gpu.py (new — ~1,300 lines) Standalone GPU-accelerated backtester completely separate from backtest.py. Computes all indicator series in one vectorised O(N) pass over the full price history, then replays positions with a sequential state machine. Key design: - VectorIndicators: pandas ewm/rolling (C-optimised) for all 7 indicators - CuPy GPU backend with automatic NumPy fallback (no hard dependency) - Two-phase API: prepare_arrays() once, score_and_run() many times - --optimize flag: 36-combo weight sweep (1 baseline + 7 ablations + 28 multiplier runs) — calls prepare_arrays() once, replays all combos in seconds - Full CLI: --tf, --local, --start/--end, --regime-filter, --regime-tf, --session-filter, --optimize, --journal, --journal-csv O(N) vs O(N^2): backtest.py recomputes all indicators per bar (O(N^2)); backtest_gpu.py computes each indicator series once and reads bar T in O(1). On 100,000 H1 bars: ~4 seconds vs >10 minutes. ### data_provider.py Added load_local_csv() method — reads MetaTrader tab-delimited CSV exports from market_data/ directory. Format: Time|Open|High|Low|Close|Volume|Spread (spread column dropped on load, timestamps localised to UTC). Applies the same _validate_and_clean() pipeline as the Twelve Data path. LOCAL_CSV_MAP maps each Timeframe enum to its filename. market_data/ files added (MetaTrader exports, UTC): USDJPY_M15.csv 100,000 bars 2022-05-04 to 2026-05-08 USDJPY_H1.csv 100,000 bars 2010-04-23 to 2026-05-08 USDJPY_H4.csv 25,855 bars 2010-04-22 to 2026-05-08 USDJPY_D1.csv 5,020 bars 2010-04-22 to 2026-05-08 16 years of H1/H4/D1 vs ~6 weeks of H1 on Twelve Data free plan. Use --local for all backtests; Twelve Data reserved for live signal feed. ### engine.py — optimised indicator weights 36-combo sweep on H4 2022-2026 found MACD and EMA trend are the dominant signals; RSI and Bollinger are net drag on this trending pair. Old → new weights: ema_trend: 1.2 → 3.6 (×3 — strongest directional signal) macd: 1.1 → 3.3 (×3 — ablating it is the single worst move: -42%) stochastic: 0.8 → 1.6 (×2 — clean timing, 40% WR at this level) adx: 1.0 → 0.5 (÷2 — partially redundant with EMA stack) ichimoku: 1.1 → 0.55 (÷2 — lags too much at H4, hurts in ranging) rsi: 0.3 → 0.0 (ablated — net drag across all test windows) bollinger: 0.9 → 0.0 (ablated — counter-trend noise on trending pair) RSI and Bollinger are still computed and displayed in live output for reference; weight=0 means they contribute nothing to the composite score. Combined result vs baseline (H4 2022-2026, local data): Return: +7.12% → +21.30% (+14.2pp) Max DD: -9.05% → -8.52% (tighter — gains did not come at expense of risk) Sharpe: 0.23 → 0.47 (2x improvement) Sortino: 1.38 → 1.91 Win Rate: 36.3% → 38.1% ### backtest.py Added 16-thread CPU parallelism for signal pre-computation via ThreadPoolExecutor. _precompute_all_signals() dispatches each bar to a worker; pandas/NumPy C extensions release the GIL enabling true parallel execution. Results are collected and replayed in the main loop, replacing the per-bar recompute. Also added --regime-tf CLI flag so regime filter timeframe can be specified independently of the primary TF. ### signal_aggregator.py EMA200 distance gate: confidence *= 0.55 when price is within 0.5% of EMA200 (chop zone) or beyond 1.5% (overextended). Sweet spot is 0.5-1.5% from EMA200 where trending momentum is cleanest. Strongest single filter found: lifted cumulative return +13.88% to +35.50% in ablation window. ### ENGINE_LOGIC.md (new) Full reference document covering: architecture, data pipeline, all indicator logic, composite scoring formula, optimised weights with rationale, regime classification, EMA200 gate, MTF alignment, grade system, backtesting results, filter catalogue, and configuration reference.
2026-05-07 22:12:43 -04:00
parent 5a392a578d
commit 17cb63e86e
10 changed files with 232766 additions and 20 deletions
@@ -0,0 +1,450 @@
+# ForexBot2 — Engine Logic & Design Reference
+
+> USD/JPY signal intelligence platform.  
+> Last updated: 2026-05-08
+
+---
+
+## Table of Contents
+
+1. [Architecture Overview](#1-architecture-overview)
+2. [Data Pipeline](#2-data-pipeline)
+3. [Indicator Engine (`engine.py`)](#3-indicator-engine)
+4. [Signal Aggregator (`signal_aggregator.py`)](#4-signal-aggregator)
+5. [Composite Scoring Formula](#5-composite-scoring-formula)
+6. [Indicator Weights — Optimised](#6-indicator-weights--optimised)
+7. [Market Regime Classification](#7-market-regime-classification)
+8. [EMA200 Distance Gate](#8-ema200-distance-gate)
+9. [Multi-Timeframe Alignment](#9-multi-timeframe-alignment)
+10. [Signal Grade System](#10-signal-grade-system)
+11. [GPU Backtester (`backtest_gpu.py`)](#11-gpu-backtester)
+12. [Backtesting Results Summary](#12-backtesting-results-summary)
+13. [Filter Catalogue](#13-filter-catalogue)
+14. [Configuration Reference](#14-configuration-reference)
+
+---
+
+## 1. Architecture Overview
+
+```
+market_data/ (local CSV)
+      │
+      ▼
+ForexDataProvider          ← also pulls live via Twelve Data SDK
+      │
+      ▼
+TechnicalEngine            ← computes all indicators bar-by-bar
+      │
+      ▼
+SignalAggregator           ← MTF blending, EMA200 gate, grading
+      │
+      ├──▶ Live output (main.py)
+      └──▶ GPUBacktester   ← vectorised O(N) replay over full history
+```
+
+**Key design principle:** No single indicator is trusted alone. The composite score is a confidence-weighted, regime-adjusted blend of up to 7 indicators across 3 timeframes. A signal only fires when direction, strength, and confidence all agree.
+
+---
+
+## 2. Data Pipeline
+
+### Live mode
+`ForexDataProvider.fetch_ohlcv()` calls the Twelve Data SDK with in-memory caching (5-min TTL). Fetches primary TF + two confirmation TFs + optional regime TF.
+
+### Backtest mode
+`ForexDataProvider.load_local_csv()` reads MetaTrader tab-delimited exports from `market_data/`:
+
+| File | TF | Rows | Date Range |
+|------|----|------|------------|
+| USDJPY_M15.csv | M15 | 100,000 | 2022-05-04 → 2026-05-08 |
+| USDJPY_H1.csv | H1 | 100,000 | 2010-04-23 → 2026-05-08 |
+| USDJPY_H4.csv | H4 | 25,855 | 2010-04-22 → 2026-05-08 |
+| USDJPY_D1.csv | D1 | 5,020 | 2010-04-22 → 2026-05-08 |
+
+CSV format: tab-delimited, columns `Time | Open | High | Low | Close | Volume | Spread`. Spread column is dropped on load. Timestamps are UTC.
+
+**Data validation** (applied to both sources):
+- Drop NaN OHLC rows; forward-fill up to 3 bars
+- Drop future timestamps (>5 min ahead of UTC now)
+- Fix OHLC inconsistencies (High < Low etc.)
+- Remove outlier bars where `|return| > 10 × rolling_std`
+
+---
+
+## 3. Indicator Engine
+
+`engine.py → TechnicalEngine.analyze(df, symbol, timeframe)`
+
+All indicators are computed on a single OHLCV DataFrame. Each returns an `IndicatorSignal` with:
+- `value` — raw numeric reading
+- `signal` — `SignalStrength` enum: STRONG_BUY(+2) / BUY(+1) / NEUTRAL(0) / SELL(-1) / STRONG_SELL(-2)
+- `confidence` — 0–100%
+- `weight` — contribution to composite score (0 = excluded)
+
+### Indicators computed
+
+| Name | Key | Wilder smoothing? | Notes |
+|------|-----|:-----------------:|-------|
+| EMA Alignment | `ema_trend` | No | 4-EMA stack: 9/21/50/200; normalised to -1..+1 |
+| RSI | `rsi` | Yes (`alpha=1/14`) | Overbought/oversold + momentum zones |
+| MACD | `macd` | No | 12/26/9; crossover = STRONG signal |
+| Bollinger Bands | `bollinger` | No | 20-period; position within band + squeeze detect |
+| Stochastic | `stochastic` | No | 14/3; K+D both OB/OS required for signal |
+| ADX | `adx` | Yes | DI+/DI- direction + ADX strength gate (≥25) |
+| ATR | `atr` | Yes | Context only — sets SL/TP distance; weight=0 |
+| Ichimoku | `ichimoku` | No | Price vs cloud (+2/-2) + TK cross (+1/-1) + cloud colour (+1/-1) |
+| Divergence | `divergence` | — | RSI/MACD price vs indicator swing divergence; weight=1.5 |
+| Candlestick | `candlestick` | — | **Disabled** (ablation: net -3.4%) |
+| Pivot Points | `pivot` | — | **Disabled** (ablation: net -9.5%) |
+| Volume | `volume` | — | Context filter; weight=0 |
+
+**Disabled indicators** are controlled by `TechnicalConfig.disabled_indicators` (default: `["pivot", "candlestick"]`).
+
+---
+
+## 4. Signal Aggregator
+
+`signal_aggregator.py → SignalAggregator.analyze()`
+
+Orchestrates three timeframes (primary + 2 confirmation), blends their scores, applies gates, and emits a final graded signal.
+
+### Flow
+
+```
+H1 (primary)  ──┐
+M15 (confirm) ──┼──▶ per-TF score ──▶ MTF blend ──▶ EMA200 gate ──▶ grade ──▶ output
+H4 (confirm)  ──┘
+```
+
+### MTF alignment (no lookahead)
+
+Lower-frequency bars are aligned to the primary TF index by forward-filling at **close times**:
+
+```
+secondary close time = secondary bar open + secondary TF minutes
+primary   close time = primary bar open   + primary TF minutes
+
+For each primary bar T: use the most recent secondary close ≤ T_close
+```
+
+This ensures no future information leaks into the signal.
+
+---
+
+## 5. Composite Scoring Formula
+
+Mirrors `engine._compute_composite` exactly:
+
+```
+composite_score = clip(
+    (Σ  signal_i × weight_i × regime_mult_i × (confidence_i / 100))
+    ──────────────────────────────────────────────────────────────────  × 50,
+    (Σ  weight_i × regime_mult_i)
+    -100, +100
+)
+
+avg_confidence = Σ(confidence_i × weight_i) / Σ weight_i
+```
+
+- `signal_i` ∈ {-2, -1, 0, +1, +2}
+- `weight_i` — per-indicator base weight (see §6)
+- `regime_mult_i` — boosted/dampened by market regime (see §7)
+- Indicators with `weight == 0` are excluded entirely
+
+---
+
+## 6. Indicator Weights — Optimised
+
+Optimised 2026-05-08 via 36-combo sweep (ablation + ×0.5/1.5/2.0/3.0 per indicator) on H4 data 2022–2026. Combined result: **+21.30% return / -8.52% max DD / Sharpe 0.47** vs baseline +7.12% / -9.05% DD / Sharpe 0.23.
+
+| Indicator | Weight | Change from original | Reason |
+|-----------|-------:|---------------------|--------|
+| `ema_trend` | **3.6** | was 1.2 (×3) | Strongest directional signal on trending pair |
+| `macd` | **3.3** | was 1.1 (×3) | Most impactful single indicator; ablation = worst result |
+| `stochastic` | **1.6** | was 0.8 (×2) | Clean timing signal; 40% WR at ×2 |
+| `adx` | **0.5** | was 1.0 (÷2) | Partially redundant with EMA stack |
+| `ichimoku` | **0.55** | was 1.1 (÷2) | Lags too much at H4; hurts on ranging periods |
+| `rsi` | **0.0** | was 0.3 (ablated) | Net drag across all test windows |
+| `bollinger` | **0.0** | was 0.9 (ablated) | Counter-trend noise on trending pair |
+| `divergence` | 1.5 | unchanged | Strong reversal signal; not in sweep scope |
+| `atr` | 0.0 | unchanged | Context only; sets SL/TP |
+
+> **Note:** RSI and Bollinger are still **computed and displayed** in live output for reference, but their weight=0 means they contribute nothing to the composite score.
+
+### Regime weight multipliers
+
+Applied on top of base weights:
+
+| Indicator | Trending mult | Ranging mult |
+|-----------|:-------------:|:------------:|
+| ema_trend | ×1.5 | ×0.6 |
+| adx | ×1.3 | ×0.5 |
+| macd | ×1.2 | ×1.0 |
+| ichimoku | ×1.3 | ×0.2 |
+| rsi | ×0.7 | ×1.4 |
+| bollinger | ×0.6 | ×1.4 |
+| stochastic | ×1.0 | ×1.3 |
+
+In trending regimes: EMA trend, MACD, and Ichimoku are boosted further.  
+In ranging regimes: RSI and Bollinger would be boosted — but since both are weight=0, ranging regime currently relies entirely on stochastic + reduced EMA/ADX.
+
+---
+
+## 7. Market Regime Classification
+
+`engine._classify_regime()` — runs after all indicators are computed:
+
+```
+if ADX ≥ 25:
+    if EMA signal is BUY or STRONG_BUY → trending_up
+    if EMA signal is SELL or STRONG_SELL → trending_down
+    else → trending
+else:
+    → ranging
+```
+
+Regime feeds directly into the weight multipliers applied in `_compute_composite`.
+
+---
+
+## 8. EMA200 Distance Gate
+
+**Strongest single filter discovered** — implemented in `signal_aggregator.py`.
+
+```
+ema_dist = |close - EMA200| / EMA200 × 100  (percent)
+
+if ema_dist < 0.5% or ema_dist > 1.5%:
+    confidence *= 0.55          ← damped
+else:                           ← sweet spot
+    no change
+```
+
+**Rationale:** Price within 0.5% of EMA200 is in a high-chop zone where the 200 EMA acts as contested support/resistance. Price more than 1.5% away has often already run far and is prone to reversion. The 0.5–1.5% band is where trending momentum is cleanest.
+
+**Impact measured:** Lifted cumulative return +13.88% → +35.50% in the 2024-trend ablation window.
+
+---
+
+## 9. Multi-Timeframe Alignment
+
+Primary TF: **H1** (default). Confirmation TFs: **M15** and **H4**.
+
+### MTF blending (when 2 of 3 TFs agree)
+
+```
+mtf_agree = (bull_count ≥ 2) OR (bear_count ≥ 2)
+
+if mtf_agree:
+    score = 0.6 × primary_score + 0.4 × mean(all_scores)
+    confidence = min(confidence + 15, 95)
+else:
+    confidence = confidence × 0.7       ← penalised for disagreement
+```
+
+### MTF agreement bonus in grade assignment
+
+`mtf_agree=True` is required for grades A and A+.
+
+---
+
+## 10. Signal Grade System
+
+| Grade | Conditions |
+|-------|-----------|
+| **A+** | `|score| ≥ 50` AND `conf ≥ 75%` AND `mtf_agree` |
+| **A** | `|score| ≥ 35` AND `conf ≥ 65%` AND `mtf_agree` |
+| **B** | `|score| ≥ 25` AND `conf ≥ 55%` |
+| **C** | `|score| ≥ 15` AND `conf ≥ 45%` |
+| **D** | `|score| ≥ 10` |
+| **F** | below all thresholds |
+
+Default minimum grade for trade entry: **B** (`BacktestConfig.min_trade_grade`).
+
+### Signal integer thresholds (for trade direction)
+
+| Signal | Condition |
+|--------|-----------|
+| STRONG_BUY (+2) | score ≥ +40 AND conf ≥ 65% |
+| BUY (+1) | score ≥ +15 AND conf ≥ 50% |
+| STRONG_SELL (-2) | score ≤ -40 AND conf ≥ 65% |
+| SELL (-1) | score ≤ -15 AND conf ≥ 50% |
+| NEUTRAL | otherwise |
+
+---
+
+## 11. GPU Backtester
+
+`backtest_gpu.py` — completely separate from `backtest.py`.
+
+### Why O(N) instead of O(N²)
+
+`backtest.py` slices the DataFrame and recomputes all indicators fresh each bar → N × O(N) = **O(N²)**.  
+`backtest_gpu.py` computes every indicator series end-to-end once, then reads bar T's value in O(1) → **O(N) total**.
+
+On 100,000 H1 bars: ~4 seconds vs >10 minutes.
+
+### Two-phase architecture
+
+```
+Phase 1 — prepare_arrays(dfs)
+    VectorIndicators.compute_all()     ← pandas ewm/rolling, C-optimised
+    Align MTF signals to primary TF    ← forward-fill at close times
+    Compute regime ADX series          ← if regime filter enabled
+    Returns reusable bundle dict
+
+Phase 2 — score_and_run(bundle, weights, disabled)
+    compute_bar_scores()               ← CuPy GPU or NumPy CPU
+    compute_final_scores()             ← MTF blend + EMA200 gate (vectorised)
+    compute_grades()                   ← vectorised grade assignment
+    run_state_machine()                ← sequential position management (O(N))
+```
+
+`prepare_arrays()` is called once; `score_and_run()` can be called many times with different weights — this is what the `--optimize` sweep uses.
+
+### Optimization sweep (`--optimize`)
+
+36 combinations tested per run:
+- **1** baseline (default weights)
+- **7** ablation runs (each indicator disabled one at a time)
+- **28** weight multiplier runs (each indicator at ×0.5, ×1.5, ×2.0, ×3.0)
+
+Indicator arrays computed once → all 36 replays run in seconds.
+
+### Trade execution model
+
+```
+Entry: next bar Open ± (spread/2 + slippage)   [default: 0.5 pip slippage]
+SL:    entry ± ATR × atr_sl_multiplier          [default: 1.5×]
+TP:    entry ± SL_dist × rr_ratio               [default: 2.0× = 1:2 R:R]
+Risk:  1% of current equity per trade
+```
+
+If SL and TP both touch within the same bar: SL assumed to hit first (`sl_first_on_tie=True`).
+
+### CLI flags
+
+```
+python backtest_gpu.py --tf H4 --local               # local CSV, H4
+python backtest_gpu.py --tf H1 --local --start 2022-01-01
+python backtest_gpu.py --tf H4 --local --optimize    # 36-combo weight sweep
+python backtest_gpu.py --regime-filter --regime-tf D1
+python backtest_gpu.py --session-filter              # UTC 07-16 only
+python backtest_gpu.py --journal --journal-csv out.csv
+```
+
+---
+
+## 12. Backtesting Results Summary
+
+### H4 — local CSV 2022–2026 (optimised weights)
+
+| Metric | Value |
+|--------|-------|
+| Trades | 160 |
+| Win Rate | 38.1% |
+| Profit Factor | 1.19 |
+| Total Return | +21.30% |
+| Max Drawdown | -8.52% |
+| Sharpe | 0.47 |
+| Sortino | 1.91 |
+
+### D1 — Twelve Data 2007–2026 (optimised weights)
+
+| Metric | Value |
+|--------|-------|
+| Trades | 62 |
+| Win Rate | 40.3% |
+| Profit Factor | 1.27 |
+| Total Return | +10.51% |
+| Max Drawdown | -6.84% |
+| Sortino | 7.74 |
+
+### Optimisation sweep delta (H4 2022–2026)
+
+| | Baseline | Optimised |
+|--|----------|-----------|
+| Return | +7.12% | +21.30% |
+| Max DD | -9.05% | -8.52% |
+| Sharpe | 0.23 | 0.47 |
+| Win Rate | 36.3% | 38.1% |
+
+---
+
+## 13. Filter Catalogue
+
+### Regime filter (`--regime-filter`)
+
+Gates trade entry on ADX of a **higher** timeframe than the primary. Default: H4 ADX gate when trading H1. Redundant if regime TF == primary TF.
+
+```
+regime TF ADX ≥ min_adx (default 25) → entry allowed
+```
+
+Sensible defaults: H1→H4, H4→D1, D1→D1.
+
+### Session filter (`--session-filter`)
+
+Only trade during UTC 07:00–16:00 (London + NY overlap).  
+2026 diagnostic: 07-15 UTC = 47% WR / +127 pips; off-hours = 0% WR / -571 pips.
+
+### Auto-session toggle (`auto_session_filter`)
+
+Reactive version of session filter. Activates automatically when mean P&L over last N closed trades drops below threshold:
+
+```
+if mean(pnl_pct[-10:]) < -0.3%:
+    session_filter = ON     ← restrict to liquid hours
+else:
+    session_filter = OFF    ← all hours allowed
+```
+
+### EMA200 distance gate
+
+Always active. See §8.
+
+### Signal grade gate (`min_trade_grade`)
+
+Default: **B**. Only trades with grade ≥ B are entered. Raise to A for fewer, higher-quality trades.
+
+### Min absolute score (`min_abs_score`)
+
+Default: **0** (no extra gate). Set to e.g. 15 to skip weak directional scores.
+
+---
+
+## 14. Configuration Reference
+
+All knobs live in `config.py → PlatformConfig`.
+
+```python
+# Indicator parameters
+TechnicalConfig.ema_fast = 9
+TechnicalConfig.ema_medium = 21
+TechnicalConfig.ema_slow = 50
+TechnicalConfig.ema_trend = 200
+TechnicalConfig.rsi_period = 14
+TechnicalConfig.macd_fast / slow / signal = 12/26/9
+TechnicalConfig.bb_period = 20
+TechnicalConfig.atr_period = 14
+TechnicalConfig.stoch_k / d = 14/3
+TechnicalConfig.adx_period = 14
+TechnicalConfig.adx_strong_trend = 25.0
+TechnicalConfig.ichimoku_tenkan / kijun / senkou_b = 9/26/52
+TechnicalConfig.disabled_indicators = ["pivot", "candlestick"]
+
+# Signal scoring
+SignalConfig.min_confidence = 55.0
+SignalConfig.atr_sl_multiplier = 1.5
+SignalConfig.default_rr_ratio = 2.0
+SignalConfig.default_risk_pct = 1.0
+
+# Backtest
+BacktestConfig.initial_balance = 10_000
+BacktestConfig.risk_per_trade_pct = 1.0
+BacktestConfig.min_trade_grade = "B"
+BacktestConfig.regime_filter = False
+BacktestConfig.session_filter = False
+BacktestConfig.auto_session_filter = False
+BacktestConfig.cache_dir = ".cache"
+```
@@ -15,6 +15,7 @@ import logging
 import multiprocessing as mp
 import os
 from collections import deque
+from concurrent.futures import ThreadPoolExecutor, as_completed
 from dataclasses import replace
 from typing import Deque, Dict, List, Optional, Tuple

@@ -88,6 +89,63 @@ class WalkForwardBacktester:
            dfs[tf] = df
        return dfs

+    def _precompute_all_signals(
+        self,
+        dfs: Dict[Timeframe, pd.DataFrame],
+        primary_df: pd.DataFrame,
+        warmup: int,
+        progress: bool = True,
+    ) -> Dict[int, "SignalReport"]:
+        """Pre-compute signals for every bar in parallel using all CPU threads.
+
+        TechnicalEngine.analyze() is stateless (pure function of the df slice),
+        so concurrent calls are safe. NumPy/pandas C extensions release the GIL,
+        giving real parallelism across threads without pickling overhead.
+        Returns bar-index → SignalReport so the sequential state machine can
+        do O(1) lookups instead of recomputing indicators per bar.
+        """
+        primary_tf = self.config.primary_timeframe
+        # Use all logical threads; cap at 2× physical cores to avoid over-subscription
+        n_workers = min(os.cpu_count() or 4, 16)
+        bar_indices = list(range(warmup, len(primary_df) - 1))
+
+        def _compute_bar(t: int) -> Tuple[int, "SignalReport"]:
+            decision_time = self._bar_close_time(primary_df.index[t], primary_tf)
+            sliced = self._slice_dfs(dfs, decision_time)
+            report = self.aggregator.analyze_from_dataframes(
+                symbol=self.config.symbol,
+                dfs=sliced,
+                use_news=False,
+            )
+            return t, report
+
+        precomputed: Dict[int, "SignalReport"] = {}
+        pbar = None
+        if progress:
+            try:
+                from tqdm import tqdm  # type: ignore
+                pbar = tqdm(
+                    total=len(bar_indices),
+                    desc=f"Signals  ({n_workers} threads)",
+                    unit="bar",
+                    leave=False,
+                )
+            except ImportError:
+                pass
+
+        with ThreadPoolExecutor(max_workers=n_workers) as executor:
+            futures = {executor.submit(_compute_bar, t): t for t in bar_indices}
+            for future in as_completed(futures):
+                t, report = future.result()
+                precomputed[t] = report
+                if pbar:
+                    pbar.update(1)
+
+        if pbar:
+            pbar.close()
+
+        return precomputed
+
    def run(
        self,
        dfs: Optional[Dict[Timeframe, pd.DataFrame]] = None,
@@ -117,6 +175,11 @@ class WalkForwardBacktester:
            )
            return BacktestResult(initial_balance=self.config.backtest.initial_balance)

+        # Pre-compute all bar signals in parallel before the sequential replay
+        precomputed = self._precompute_all_signals(
+            dfs, primary_df, warmup, progress=progress
+        )
+
        bt = self.config.backtest
        balance = bt.initial_balance
        equity_points: List[Tuple[pd.Timestamp, float]] = [
@@ -133,12 +196,12 @@ class WalkForwardBacktester:

        open_position: Optional[Dict] = None

-        # Optional progress bar
+        # Optional progress bar for the state-machine replay pass
        iterator = range(warmup, len(primary_df) - 1)
        if progress:
            try:
                from tqdm import tqdm  # type: ignore
-                iterator = tqdm(iterator, desc="Backtesting", unit="bar", leave=False)
+                iterator = tqdm(iterator, desc="Replaying bars", unit="bar", leave=False)
            except ImportError:
                pass

@@ -161,13 +224,7 @@ class WalkForwardBacktester:

            # ── Step 2: maybe open a new position ──────────────────
            if open_position is None or not bt.one_position_only:
-                # Slice each TF's df up to (and including) bars closed by decision_time
-                sliced = self._slice_dfs(dfs, decision_time)
-                report = self.aggregator.analyze_from_dataframes(
-                    symbol=self.config.symbol,
-                    dfs=sliced,
-                    use_news=False,
-                )
+                report = precomputed[t]

                if (
                    _grade_passes(report.signal_quality, bt.min_trade_grade)
@@ -29,6 +29,14 @@ class DataIntegrityError(Exception):
 class ForexDataProvider:
    """Twelve Data SDK client + on-disk parquet cache."""

+    # Local MetaTrader CSV exports — tab-delimited, 7 columns (includes Spread)
+    LOCAL_CSV_MAP = {
+        Timeframe.M15: "USDJPY_M15.csv",
+        Timeframe.H1:  "USDJPY_H1.csv",
+        Timeframe.H4:  "USDJPY_H4.csv",
+        Timeframe.D1:  "USDJPY_D1.csv",
+    }
+
    TD_INTERVALS = {
        Timeframe.M1:  "1min",
        Timeframe.M5:  "5min",
@@ -153,6 +161,55 @@ class ForexDataProvider:

        return df

+    def load_local_csv(
+        self,
+        timeframe: Timeframe,
+        data_dir: str = "market_data",
+        start_date: Optional[str] = None,
+        end_date: Optional[str] = None,
+    ) -> Optional[pd.DataFrame]:
+        """
+        Load OHLCV from a local MetaTrader CSV export.
+        Format: tab-separated, columns Time/Open/High/Low/Close/Volume/Spread.
+        Returns None silently if the file doesn't exist for this timeframe.
+        """
+        fname = self.LOCAL_CSV_MAP.get(timeframe)
+        if fname is None:
+            logger.info(f"No local CSV mapping for {timeframe.value} — skipping")
+            return None
+
+        path = os.path.join(data_dir, fname)
+        if not os.path.exists(path):
+            logger.warning(f"Local CSV not found: {path}")
+            return None
+
+        df = pd.read_csv(
+            path, sep="\t",
+            names=["Time", "Open", "High", "Low", "Close", "Volume", "Spread"],
+            header=0,
+            parse_dates=["Time"],
+        )
+        df = df.set_index("Time")
+        df = df.drop(columns=["Spread"], errors="ignore")
+        for col in ("Open", "High", "Low", "Close", "Volume"):
+            if col in df.columns:
+                df[col] = pd.to_numeric(df[col], errors="coerce")
+        if "Volume" not in df.columns:
+            df["Volume"] = 0.0
+
+        if df.index.tz is None:
+            df.index = df.index.tz_localize("UTC")
+        df.index.name = "Datetime"
+
+        if start_date:
+            df = df[df.index >= pd.Timestamp(start_date, tz="UTC")]
+        if end_date:
+            df = df[df.index <= pd.Timestamp(end_date, tz="UTC")]
+
+        df = self._validate_and_clean(df, USDJPY, timeframe)
+        logger.info(f"Loaded {len(df):,} bars from local CSV: {path}")
+        return df
+
    def get_latest_price(self, symbol: str = USDJPY) -> Optional[float]:
        try:
            df = self.fetch_ohlcv(symbol, Timeframe.M5, lookback_days=1)
@@ -171,7 +171,7 @@ class TechnicalEngine:
        result.indicators["ema_trend"] = IndicatorSignal(
            name="EMA Trend", value=normalized,
            signal=signal, confidence=conf,
-            description=desc, weight=1.2,
+            description=desc, weight=3.6,
        )
        if t is not None:
            result.key_levels["ema_trend"] = float(t)
@@ -222,7 +222,7 @@ class TechnicalEngine:

        result.indicators["rsi"] = IndicatorSignal(
            name="RSI", value=rsi_val, signal=signal,
-            confidence=conf, description=desc, weight=0.3,
+            confidence=conf, description=desc, weight=0.0,
        )

        # Attach RSI series for divergence detection (dynamic attribute, filtered by _prefix)
@@ -291,7 +291,7 @@ class TechnicalEngine:

        result.indicators["macd"] = IndicatorSignal(
            name="MACD", value=hist_val, signal=signal,
-            confidence=conf, description=desc, weight=1.1,
+            confidence=conf, description=desc, weight=3.3,
        )

        # Attach histogram for divergence detection
@@ -358,7 +358,7 @@ class TechnicalEngine:

        result.indicators["bollinger"] = IndicatorSignal(
            name="Bollinger Bands", value=bb_position, signal=signal,
-            confidence=conf, description=desc, weight=0.9,
+            confidence=conf, description=desc, weight=0.0,
        )

        result.key_levels["bb_upper"] = upper_val
@@ -423,7 +423,7 @@ class TechnicalEngine:

        result.indicators["stochastic"] = IndicatorSignal(
            name="Stochastic", value=k_val, signal=signal,
-            confidence=conf, description=desc, weight=0.8,
+            confidence=conf, description=desc, weight=1.6,
        )

    def _compute_adx(self, df: pd.DataFrame, result: TechnicalAnalysis):
@@ -483,7 +483,7 @@ class TechnicalEngine:

        result.indicators["adx"] = IndicatorSignal(
            name="ADX", value=adx_val, signal=signal,
-            confidence=conf, description=desc, weight=1.0,
+            confidence=conf, description=desc, weight=0.5,
        )

    def _compute_atr(self, df: pd.DataFrame, result: TechnicalAnalysis):
@@ -609,7 +609,7 @@ class TechnicalEngine:
            name="Ichimoku", value=float(score), signal=signal,
            confidence=conf,
            description=f"Ichimoku [{score}/4]: {' | '.join(signals)}",
-            weight=1.1,
+            weight=0.55,
        )

        result.key_levels["ichimoku_cloud_top"] = cloud_top
@@ -8,6 +8,7 @@ OHLCV is fetched live or sliced from a pre-loaded historical buffer.
 """

 import logging
+import os
 from dataclasses import dataclass, field
 from datetime import datetime, timezone
 from typing import Dict, List, Optional
@@ -393,16 +394,19 @@ class SignalAggregator:
            ema200 = primary.key_levels.get("ema_trend")
            if ema200 and ema200 > 0 and report.current_price > 0:
                ema_dist_pct = abs(report.current_price - ema200) / ema200 * 100.0
-                if ema_dist_pct < 0.5:
+                _lo = float(os.environ.get("FB2_EMA_LO", "0.5"))
+                _hi = float(os.environ.get("FB2_EMA_HI", "1.5"))
+                _mult = float(os.environ.get("FB2_EMA_MULT", "0.55"))
+                if ema_dist_pct < _lo:
                    report.warnings.append(
                        f"NEAR EMA200 ({ema_dist_pct:.2f}%) — chop risk, signal damped"
                    )
-                    confidence *= 0.55
-                elif ema_dist_pct > 1.5:
+                    confidence *= _mult
+                elif ema_dist_pct > _hi:
                    report.warnings.append(
                        f"FAR FROM EMA200 ({ema_dist_pct:.2f}%) — over-extended, signal damped"
                    )
-                    confidence *= 0.55
+                    confidence *= _mult

        # Final signal
        if score >= 40 and confidence >= 65: