ForexBot2/ENGINE_LOGIC.md

ForexBot2 — Engine Logic & Design Reference

USD/JPY signal intelligence platform.
Last updated: 2026-05-08

---

Table of Contents

1. Architecture Overview
2. Data Pipeline
3. Indicator Engine (engine.py)
4. Signal Aggregator (signal_aggregator.py)
5. Composite Scoring Formula
6. Indicator Weights — Optimised
7. Market Regime Classification
8. EMA200 Distance Gate
9. Multi-Timeframe Alignment
10. Signal Grade System
11. GPU Backtester (backtest_gpu.py)
12. Backtesting Results Summary
13. Filter Catalogue
14. Configuration Reference

---

1. Architecture Overview

market_data/ (local CSV)
      │
      ▼
ForexDataProvider          ← also pulls live via Twelve Data SDK
      │
      ▼
TechnicalEngine            ← computes all indicators bar-by-bar
      │
      ▼
SignalAggregator           ← MTF blending, EMA200 gate, grading
      │
      ├──▶ Live output (main.py)
      └──▶ GPUBacktester   ← vectorised O(N) replay over full history

Key design principle: No single indicator is trusted alone. The composite score is a confidence-weighted, regime-adjusted blend of up to 7 indicators across 3 timeframes. A signal only fires when direction, strength, and confidence all agree.

---

2. Data Pipeline

Live mode

ForexDataProvider.fetch_ohlcv() calls the Twelve Data SDK with in-memory caching (5-min TTL). Fetches primary TF + two confirmation TFs + optional regime TF.

Backtest mode

ForexDataProvider.load_local_csv() reads MetaTrader tab-delimited exports from market_data/:

File	TF	Rows	Date Range
USDJPY_M15.csv	M15	100,000	2022-05-04 → 2026-05-08
USDJPY_H1.csv	H1	100,000	2010-04-23 → 2026-05-08
USDJPY_H4.csv	H4	25,855	2010-04-22 → 2026-05-08
USDJPY_D1.csv	D1	5,020	2010-04-22 → 2026-05-08

CSV format: tab-delimited, columns Time | Open | High | Low | Close | Volume | Spread. Spread column is dropped on load. Timestamps are UTC.

Data validation (applied to both sources):

• Drop NaN OHLC rows; forward-fill up to 3 bars
• Drop future timestamps (>5 min ahead of UTC now)
• Fix OHLC inconsistencies (High < Low etc.)
• Remove outlier bars where |return| > 10 × rolling_std

---

3. Indicator Engine

engine.py → TechnicalEngine.analyze(df, symbol, timeframe)

All indicators are computed on a single OHLCV DataFrame. Each returns an IndicatorSignal with:

• value — raw numeric reading
• signal — SignalStrength enum: STRONG_BUY(+2) / BUY(+1) / NEUTRAL(0) / SELL(-1) / STRONG_SELL(-2)
• confidence — 0–100%
• weight — contribution to composite score (0 = excluded)

Indicators computed

Name	Key	Wilder smoothing?	Notes
EMA Alignment	ema_trend	No	4-EMA stack: 9/21/50/200; normalised to -1..+1
RSI	rsi	Yes (alpha=1/14)	Overbought/oversold + momentum zones
MACD	macd	No	12/26/9; crossover = STRONG signal
Bollinger Bands	bollinger	No	20-period; position within band + squeeze detect
Stochastic	stochastic	No	14/3; K+D both OB/OS required for signal
ADX	adx	Yes	DI+/DI- direction + ADX strength gate (≥25)
ATR	atr	Yes	Context only — sets SL/TP distance; weight=0
Ichimoku	ichimoku	No	Price vs cloud (+2/-2) + TK cross (+1/-1) + cloud colour (+1/-1)
Divergence	divergence	—	RSI/MACD price vs indicator swing divergence; weight=1.5
Candlestick	candlestick	—	Disabled (ablation: net -3.4%)
Pivot Points	pivot	—	Disabled (ablation: net -9.5%)
Volume	volume	—	Context filter; weight=0

Disabled indicators are controlled by TechnicalConfig.disabled_indicators (default: ["pivot", "candlestick"]).

---

4. Signal Aggregator

signal_aggregator.py → SignalAggregator.analyze()

Orchestrates three timeframes (primary + 2 confirmation), blends their scores, applies gates, and emits a final graded signal.

Flow

H1 (primary)  ──┐
M15 (confirm) ──┼──▶ per-TF score ──▶ MTF blend ──▶ EMA200 gate ──▶ grade ──▶ output
H4 (confirm)  ──┘

MTF alignment (no lookahead)

Lower-frequency bars are aligned to the primary TF index by forward-filling at close times:

secondary close time = secondary bar open + secondary TF minutes
primary   close time = primary bar open   + primary TF minutes

For each primary bar T: use the most recent secondary close ≤ T_close

This ensures no future information leaks into the signal.

---

5. Composite Scoring Formula

Mirrors engine._compute_composite exactly:

composite_score = clip(
    (Σ  signal_i × weight_i × regime_mult_i × (confidence_i / 100))
    ──────────────────────────────────────────────────────────────────  × 50,
    (Σ  weight_i × regime_mult_i)
    -100, +100
)

avg_confidence = Σ(confidence_i × weight_i) / Σ weight_i

• signal_i ∈ {-2, -1, 0, +1, +2}
• weight_i — per-indicator base weight (see §6)
• regime_mult_i — boosted/dampened by market regime (see §7)
• Indicators with weight == 0 are excluded entirely

---

6. Indicator Weights — Optimised

Optimised 2026-05-08 via 36-combo sweep (ablation + ×0.5/1.5/2.0/3.0 per indicator) on H4 data 2022–2026. Combined result: +21.30% return / -8.52% max DD / Sharpe 0.47 vs baseline +7.12% / -9.05% DD / Sharpe 0.23.

Indicator	Weight	Change from original	Reason
ema_trend	3.6	was 1.2 (×3)	Strongest directional signal on trending pair
macd	3.3	was 1.1 (×3)	Most impactful single indicator; ablation = worst result
stochastic	1.6	was 0.8 (×2)	Clean timing signal; 40% WR at ×2
adx	0.5	was 1.0 (÷2)	Partially redundant with EMA stack
ichimoku	0.55	was 1.1 (÷2)	Lags too much at H4; hurts on ranging periods
rsi	0.0	was 0.3 (ablated)	Net drag across all test windows
bollinger	0.0	was 0.9 (ablated)	Counter-trend noise on trending pair
divergence	1.5	unchanged	Strong reversal signal; not in sweep scope
atr	0.0	unchanged	Context only; sets SL/TP

Note: RSI and Bollinger are still computed and displayed in live output for reference, but their weight=0 means they contribute nothing to the composite score.

Regime weight multipliers

Applied on top of base weights:

Indicator	Trending mult	Ranging mult
ema_trend	×1.5	×0.6
adx	×1.3	×0.5
macd	×1.2	×1.0
ichimoku	×1.3	×0.2
rsi	×0.7	×1.4
bollinger	×0.6	×1.4
stochastic	×1.0	×1.3

In trending regimes: EMA trend, MACD, and Ichimoku are boosted further.
In ranging regimes: RSI and Bollinger would be boosted — but since both are weight=0, ranging regime currently relies entirely on stochastic + reduced EMA/ADX.

---

7. Market Regime Classification

engine._classify_regime() — runs after all indicators are computed:

if ADX ≥ 25:
    if EMA signal is BUY or STRONG_BUY → trending_up
    if EMA signal is SELL or STRONG_SELL → trending_down
    else → trending
else:
    → ranging

Regime feeds directly into the weight multipliers applied in _compute_composite.

---

8. EMA200 Distance Gate

Strongest single filter discovered — implemented in signal_aggregator.py.

ema_dist = |close - EMA200| / EMA200 × 100  (percent)

if ema_dist < 0.5% or ema_dist > 1.5%:
    confidence *= 0.55          ← damped
else:                           ← sweet spot
    no change

Rationale: Price within 0.5% of EMA200 is in a high-chop zone where the 200 EMA acts as contested support/resistance. Price more than 1.5% away has often already run far and is prone to reversion. The 0.5–1.5% band is where trending momentum is cleanest.

Impact measured: Lifted cumulative return +13.88% → +35.50% in the 2024-trend ablation window.

---

9. Multi-Timeframe Alignment

Primary TF: H1 (default). Confirmation TFs: M15 and H4.

MTF blending (when 2 of 3 TFs agree)

mtf_agree = (bull_count ≥ 2) OR (bear_count ≥ 2)

if mtf_agree:
    score = 0.6 × primary_score + 0.4 × mean(all_scores)
    confidence = min(confidence + 15, 95)
else:
    confidence = confidence × 0.7       ← penalised for disagreement

MTF agreement bonus in grade assignment

mtf_agree=True is required for grades A and A+.

---

10. Signal Grade System

Grade	Conditions
A+	`
A	`
B	`
C	`
D	`
F	below all thresholds

Default minimum grade for trade entry: B (BacktestConfig.min_trade_grade).

Signal integer thresholds (for trade direction)

Signal	Condition
STRONG_BUY (+2)	score ≥ +40 AND conf ≥ 65%
BUY (+1)	score ≥ +15 AND conf ≥ 50%
STRONG_SELL (-2)	score ≤ -40 AND conf ≥ 65%
SELL (-1)	score ≤ -15 AND conf ≥ 50%
NEUTRAL	otherwise

---

11. GPU Backtester

backtest_gpu.py — completely separate from backtest.py.

Why O(N) instead of O(N²)

backtest.py slices the DataFrame and recomputes all indicators fresh each bar → N × O(N) = O(N²).
backtest_gpu.py computes every indicator series end-to-end once, then reads bar T's value in O(1) → O(N) total.

On 100,000 H1 bars: ~4 seconds vs >10 minutes.

Two-phase architecture

Phase 1 — prepare_arrays(dfs)
    VectorIndicators.compute_all()     ← pandas ewm/rolling, C-optimised
    Align MTF signals to primary TF    ← forward-fill at close times
    Compute regime ADX series          ← if regime filter enabled
    Returns reusable bundle dict

Phase 2 — score_and_run(bundle, weights, disabled)
    compute_bar_scores()               ← CuPy GPU or NumPy CPU
    compute_final_scores()             ← MTF blend + EMA200 gate (vectorised)
    compute_grades()                   ← vectorised grade assignment
    run_state_machine()                ← sequential position management (O(N))

prepare_arrays() is called once; score_and_run() can be called many times with different weights — this is what the --optimize sweep uses.

Optimization sweep (--optimize)

36 combinations tested per run:

• 1 baseline (default weights)
• 7 ablation runs (each indicator disabled one at a time)
• 28 weight multiplier runs (each indicator at ×0.5, ×1.5, ×2.0, ×3.0)

Indicator arrays computed once → all 36 replays run in seconds.

Trade execution model

Entry: next bar Open ± (spread/2 + slippage)   [default: 0.5 pip slippage]
SL:    entry ± ATR × atr_sl_multiplier          [default: 1.5×]
TP:    entry ± SL_dist × rr_ratio               [default: 2.0× = 1:2 R:R]
Risk:  1% of current equity per trade

If SL and TP both touch within the same bar: SL assumed to hit first (sl_first_on_tie=True).

CLI flags

python backtest_gpu.py --tf H4 --local               # local CSV, H4
python backtest_gpu.py --tf H1 --local --start 2022-01-01
python backtest_gpu.py --tf H4 --local --optimize    # 36-combo weight sweep
python backtest_gpu.py --regime-filter --regime-tf D1
python backtest_gpu.py --session-filter              # UTC 07-16 only
python backtest_gpu.py --journal --journal-csv out.csv

---

12. Backtesting Results Summary

H4 — local CSV 2022–2026 (optimised weights)

Metric	Value
Trades	160
Win Rate	38.1%
Profit Factor	1.19
Total Return	+21.30%
Max Drawdown	-8.52%
Sharpe	0.47
Sortino	1.91

D1 — Twelve Data 2007–2026 (optimised weights)

Metric	Value
Trades	62
Win Rate	40.3%
Profit Factor	1.27
Total Return	+10.51%
Max Drawdown	-6.84%
Sortino	7.74

Optimisation sweep delta (H4 2022–2026)

	Baseline	Optimised
Return	+7.12%	+21.30%
Max DD	-9.05%	-8.52%
Sharpe	0.23	0.47
Win Rate	36.3%	38.1%

---

13. Filter Catalogue

Regime filter (--regime-filter)

Gates trade entry on ADX of a higher timeframe than the primary. Default: H4 ADX gate when trading H1. Redundant if regime TF == primary TF.

regime TF ADX ≥ min_adx (default 25) → entry allowed

Sensible defaults: H1→H4, H4→D1, D1→D1.

Session filter (--session-filter)

Only trade during UTC 07:00–16:00 (London + NY overlap).
2026 diagnostic: 07-15 UTC = 47% WR / +127 pips; off-hours = 0% WR / -571 pips.

Auto-session toggle (auto_session_filter)

Reactive version of session filter. Activates automatically when mean P&L over last N closed trades drops below threshold:

if mean(pnl_pct[-10:]) < -0.3%:
    session_filter = ON     ← restrict to liquid hours
else:
    session_filter = OFF    ← all hours allowed

EMA200 distance gate

Always active. See §8.

Signal grade gate (min_trade_grade)

Default: B. Only trades with grade ≥ B are entered. Raise to A for fewer, higher-quality trades.

Min absolute score (min_abs_score)

Default: 0 (no extra gate). Set to e.g. 15 to skip weak directional scores.

---

14. Configuration Reference

All knobs live in config.py → PlatformConfig.

# Indicator parameters
TechnicalConfig.ema_fast = 9
TechnicalConfig.ema_medium = 21
TechnicalConfig.ema_slow = 50
TechnicalConfig.ema_trend = 200
TechnicalConfig.rsi_period = 14
TechnicalConfig.macd_fast / slow / signal = 12/26/9
TechnicalConfig.bb_period = 20
TechnicalConfig.atr_period = 14
TechnicalConfig.stoch_k / d = 14/3
TechnicalConfig.adx_period = 14
TechnicalConfig.adx_strong_trend = 25.0
TechnicalConfig.ichimoku_tenkan / kijun / senkou_b = 9/26/52
TechnicalConfig.disabled_indicators = ["pivot", "candlestick"]

# Signal scoring
SignalConfig.min_confidence = 55.0
SignalConfig.atr_sl_multiplier = 1.5
SignalConfig.default_rr_ratio = 2.0
SignalConfig.default_risk_pct = 1.0

# Backtest
BacktestConfig.initial_balance = 10_000
BacktestConfig.risk_per_trade_pct = 1.0
BacktestConfig.min_trade_grade = "B"
BacktestConfig.regime_filter = False
BacktestConfig.session_filter = False
BacktestConfig.auto_session_filter = False
BacktestConfig.cache_dir = ".cache"