GPU backtester, optimised weights, local CSV data pipeline
## What changed ### backtest_gpu.py (new — ~1,300 lines) Standalone GPU-accelerated backtester completely separate from backtest.py. Computes all indicator series in one vectorised O(N) pass over the full price history, then replays positions with a sequential state machine. Key design: - VectorIndicators: pandas ewm/rolling (C-optimised) for all 7 indicators - CuPy GPU backend with automatic NumPy fallback (no hard dependency) - Two-phase API: prepare_arrays() once, score_and_run() many times - --optimize flag: 36-combo weight sweep (1 baseline + 7 ablations + 28 multiplier runs) — calls prepare_arrays() once, replays all combos in seconds - Full CLI: --tf, --local, --start/--end, --regime-filter, --regime-tf, --session-filter, --optimize, --journal, --journal-csv O(N) vs O(N^2): backtest.py recomputes all indicators per bar (O(N^2)); backtest_gpu.py computes each indicator series once and reads bar T in O(1). On 100,000 H1 bars: ~4 seconds vs >10 minutes. ### data_provider.py Added load_local_csv() method — reads MetaTrader tab-delimited CSV exports from market_data/ directory. Format: Time|Open|High|Low|Close|Volume|Spread (spread column dropped on load, timestamps localised to UTC). Applies the same _validate_and_clean() pipeline as the Twelve Data path. LOCAL_CSV_MAP maps each Timeframe enum to its filename. market_data/ files added (MetaTrader exports, UTC): USDJPY_M15.csv 100,000 bars 2022-05-04 to 2026-05-08 USDJPY_H1.csv 100,000 bars 2010-04-23 to 2026-05-08 USDJPY_H4.csv 25,855 bars 2010-04-22 to 2026-05-08 USDJPY_D1.csv 5,020 bars 2010-04-22 to 2026-05-08 16 years of H1/H4/D1 vs ~6 weeks of H1 on Twelve Data free plan. Use --local for all backtests; Twelve Data reserved for live signal feed. ### engine.py — optimised indicator weights 36-combo sweep on H4 2022-2026 found MACD and EMA trend are the dominant signals; RSI and Bollinger are net drag on this trending pair. Old → new weights: ema_trend: 1.2 → 3.6 (×3 — strongest directional signal) macd: 1.1 → 3.3 (×3 — ablating it is the single worst move: -42%) stochastic: 0.8 → 1.6 (×2 — clean timing, 40% WR at this level) adx: 1.0 → 0.5 (÷2 — partially redundant with EMA stack) ichimoku: 1.1 → 0.55 (÷2 — lags too much at H4, hurts in ranging) rsi: 0.3 → 0.0 (ablated — net drag across all test windows) bollinger: 0.9 → 0.0 (ablated — counter-trend noise on trending pair) RSI and Bollinger are still computed and displayed in live output for reference; weight=0 means they contribute nothing to the composite score. Combined result vs baseline (H4 2022-2026, local data): Return: +7.12% → +21.30% (+14.2pp) Max DD: -9.05% → -8.52% (tighter — gains did not come at expense of risk) Sharpe: 0.23 → 0.47 (2x improvement) Sortino: 1.38 → 1.91 Win Rate: 36.3% → 38.1% ### backtest.py Added 16-thread CPU parallelism for signal pre-computation via ThreadPoolExecutor. _precompute_all_signals() dispatches each bar to a worker; pandas/NumPy C extensions release the GIL enabling true parallel execution. Results are collected and replayed in the main loop, replacing the per-bar recompute. Also added --regime-tf CLI flag so regime filter timeframe can be specified independently of the primary TF. ### signal_aggregator.py EMA200 distance gate: confidence *= 0.55 when price is within 0.5% of EMA200 (chop zone) or beyond 1.5% (overextended). Sweet spot is 0.5-1.5% from EMA200 where trending momentum is cleanest. Strongest single filter found: lifted cumulative return +13.88% to +35.50% in ablation window. ### ENGINE_LOGIC.md (new) Full reference document covering: architecture, data pipeline, all indicator logic, composite scoring formula, optimised weights with rationale, regime classification, EMA200 gate, MTF alignment, grade system, backtesting results, filter catalogue, and configuration reference.
This commit is contained in:
@@ -171,7 +171,7 @@ class TechnicalEngine:
|
||||
result.indicators["ema_trend"] = IndicatorSignal(
|
||||
name="EMA Trend", value=normalized,
|
||||
signal=signal, confidence=conf,
|
||||
description=desc, weight=1.2,
|
||||
description=desc, weight=3.6,
|
||||
)
|
||||
if t is not None:
|
||||
result.key_levels["ema_trend"] = float(t)
|
||||
@@ -222,7 +222,7 @@ class TechnicalEngine:
|
||||
|
||||
result.indicators["rsi"] = IndicatorSignal(
|
||||
name="RSI", value=rsi_val, signal=signal,
|
||||
confidence=conf, description=desc, weight=0.3,
|
||||
confidence=conf, description=desc, weight=0.0,
|
||||
)
|
||||
|
||||
# Attach RSI series for divergence detection (dynamic attribute, filtered by _prefix)
|
||||
@@ -291,7 +291,7 @@ class TechnicalEngine:
|
||||
|
||||
result.indicators["macd"] = IndicatorSignal(
|
||||
name="MACD", value=hist_val, signal=signal,
|
||||
confidence=conf, description=desc, weight=1.1,
|
||||
confidence=conf, description=desc, weight=3.3,
|
||||
)
|
||||
|
||||
# Attach histogram for divergence detection
|
||||
@@ -358,7 +358,7 @@ class TechnicalEngine:
|
||||
|
||||
result.indicators["bollinger"] = IndicatorSignal(
|
||||
name="Bollinger Bands", value=bb_position, signal=signal,
|
||||
confidence=conf, description=desc, weight=0.9,
|
||||
confidence=conf, description=desc, weight=0.0,
|
||||
)
|
||||
|
||||
result.key_levels["bb_upper"] = upper_val
|
||||
@@ -423,7 +423,7 @@ class TechnicalEngine:
|
||||
|
||||
result.indicators["stochastic"] = IndicatorSignal(
|
||||
name="Stochastic", value=k_val, signal=signal,
|
||||
confidence=conf, description=desc, weight=0.8,
|
||||
confidence=conf, description=desc, weight=1.6,
|
||||
)
|
||||
|
||||
def _compute_adx(self, df: pd.DataFrame, result: TechnicalAnalysis):
|
||||
@@ -483,7 +483,7 @@ class TechnicalEngine:
|
||||
|
||||
result.indicators["adx"] = IndicatorSignal(
|
||||
name="ADX", value=adx_val, signal=signal,
|
||||
confidence=conf, description=desc, weight=1.0,
|
||||
confidence=conf, description=desc, weight=0.5,
|
||||
)
|
||||
|
||||
def _compute_atr(self, df: pd.DataFrame, result: TechnicalAnalysis):
|
||||
@@ -609,7 +609,7 @@ class TechnicalEngine:
|
||||
name="Ichimoku", value=float(score), signal=signal,
|
||||
confidence=conf,
|
||||
description=f"Ichimoku [{score}/4]: {' | '.join(signals)}",
|
||||
weight=1.1,
|
||||
weight=0.55,
|
||||
)
|
||||
|
||||
result.key_levels["ichimoku_cloud_top"] = cloud_top
|
||||
|
||||
Reference in New Issue
Block a user