Files
ForexBot2/DOCUMENTATION.md
T
harwinder 59a18b8bc6 Add project documentation and MT5 integration plan
DOCUMENTATION.md: Two-section guide covering the full project —
plain-English trader guide (indicators, scoring, trade setups) and
developer reference (architecture, data flow, every threshold/weight).

mt5.md: Architecture plan for automated order execution via MT5 bridge
(REST API on Windows, async client on Linux, risk-based lot sizing).
2026-05-11 19:50:21 -04:00

1012 lines
41 KiB
Markdown

# ForexBot2 — Complete Documentation
---
# SECTION 1: How It Works (For Traders & Everyone)
## What Is ForexBot2?
ForexBot2 is an automated forex analysis bot. It watches 4 currency pairs around the clock, analyzes them using 6 different technical indicators, checks the news, and sends you alerts on Telegram when it finds a good trading opportunity — complete with entry price, stop loss, and take profit levels.
Think of it as a tireless analyst who never sleeps, never gets emotional, and checks the charts every few minutes across multiple timeframes.
## What Pairs Does It Watch?
| Pair | What It Is |
|------|-----------|
| EUR/USD | Euro vs US Dollar — the world's most traded pair |
| GBP/USD | British Pound vs US Dollar |
| USD/JPY | US Dollar vs Japanese Yen |
| XAU/USD | Gold vs US Dollar |
## What Timeframes Does It Scan?
The bot looks at the same pair across multiple zoom levels, like looking at a map at different scales:
| Timeframe | What It Shows | How Often It Scans |
|-----------|--------------|-------------------|
| 5-Minute (M5) | Very short-term price action | Every 5 minutes — **data collection only**, no alerts |
| 15-Minute (M15) | Short-term moves | Every 15 minutes |
| 1-Hour (H1) | Intraday trends | Every hour |
| 4-Hour (H4) | Swing-level trends | Every 4 hours |
The M5 scan runs silently in the background — it gathers data to keep the bot's internal picture fresh, but it does not send you any messages. Only M15, H1, and H4 scans produce alerts.
## How Does It Decide What To Signal?
The bot uses a layered decision process. It is not based on a single indicator — it combines multiple tools, each looking at the market from a different angle.
### The 6 Active Indicators
#### 1. EMA Trend (Exponential Moving Averages)
**What it is:** An EMA is a smoothed line that follows the average price. The bot uses 4 EMAs with different speeds:
- **EMA 9** (fast) — reacts quickly to price changes
- **EMA 21** (medium) — smooths out noise
- **EMA 50** (slow) — shows the broader direction
- **EMA 200** (trend) — the "big picture" trend line that institutional traders watch
**How it works:** When the faster lines are above the slower lines, and price is above all of them, the market is in a clear uptrend. When everything is flipped (price below all lines, fast below slow), it is a downtrend.
**What the bot does with it:** It scores how well-aligned the EMAs are. Perfect bullish alignment = strong buy signal. Perfect bearish alignment = strong sell. Mixed or tangled EMAs = neutral. It also detects "golden crosses" (fast EMA crossing above medium — bullish) and "death crosses" (the opposite — bearish).
**Weight in the final decision:** Base weight of 1.0. In trending markets, this weight doubles to 2.0 because trends make EMAs more reliable. In ranging (sideways) markets, the weight drops to 0.5 because EMAs give false signals in chop.
---
#### 2. RSI (Relative Strength Index)
**What it is:** RSI measures how fast and how much the price has been going up vs down recently. It produces a number from 0 to 100.
**How to read it:**
- **Above 70** = "overbought" — price has risen too fast and may pull back
- **Below 30** = "oversold" — price has fallen too fast and may bounce
- **Between 40-60** = neutral territory
**What the bot does with it:** When RSI is above 70, the bot flags it as a sell signal (the move is exhausted). Below 30, it is a buy signal (the selling is overdone). In between, it looks at which side of 50 the RSI is on for a directional lean.
**Weight:** 1.5 base weight. Gets boosted to 2.0 in ranging markets (RSI works best when price bounces between support and resistance) and 1.5 in trending markets.
---
#### 3. Bollinger Bands
**What it is:** Bollinger Bands are three lines drawn around the price:
- A middle line (20-period moving average)
- An upper band (2 standard deviations above the average)
- A lower band (2 standard deviations below the average)
The bands expand when the market is volatile and contract when it is quiet.
**How to read it:**
- Price touching or breaking above the upper band = potentially overbought
- Price touching or breaking below the lower band = potentially oversold
- **Squeeze** (bands getting very tight) = a big move is coming, direction unknown yet
**What the bot does with it:** It calculates where the price sits within the bands (0 = at lower band, 1 = at upper band). Above 0.8 = sell signal. Below 0.2 = buy signal. It also detects squeezes (band width less than 75% of its recent average) and increases confidence when one is forming, because a breakout is imminent.
**Weight:** 1.5 base. Gets boosted to 2.0 in ranging markets (bands work best in ranges) and reduced to 0.8 in trending markets (price can "ride the band" for extended periods in a trend, making the overbought/oversold reading misleading).
---
#### 4. ADX (Average Directional Index)
**What it is:** ADX measures the *strength* of a trend, not its direction. It is a number from 0 to 100.
**How to read it:**
- **Below 25** = weak trend or ranging market
- **25-50** = strong trend
- **Above 50** = very strong trend
The *direction* comes from two companion lines: +DI (bullish pressure) and -DI (bearish pressure). Whichever is higher tells you the direction.
**What the bot does with it:** ADX has two roles. First, it helps classify the market as "trending" or "ranging" — this changes how much the bot trusts other indicators. Second, when the trend is strong (ADX above 25) and +DI is above -DI, it adds a buy signal; if -DI is winning, it adds a sell signal.
**Weight:** 0.5 — deliberately low because ADX confirms trend strength but is not great at timing entries.
---
#### 5. Divergence Detection (RSI + MACD)
**What it is:** Divergence happens when the price makes a new high (or low) but the indicator does not confirm it. This is one of the most powerful reversal signals in technical analysis.
**Types:**
- **Bullish divergence:** Price makes a lower low, but RSI or MACD makes a higher low. Translation: selling pressure is weakening even though price is still falling. A bounce is likely.
- **Bearish divergence:** Price makes a higher high, but the indicator makes a lower high. Translation: buying pressure is fading even though price is still rising. A pullback is likely.
**What the bot does with it:** It looks back 14 bars, finds swing points in both price and the indicators (RSI and MACD histogram), and checks if they disagree. If divergence is found, it flags it with 70% confidence.
**Weight:** 1.0 base. Gets boosted to 1.5 in ranging markets (divergences are strongest at range boundaries).
---
#### 6. Candlestick Patterns
**What it is:** Candlestick patterns are specific shapes formed by one or two candles that have historically predicted price direction.
**Patterns the bot detects:**
- **Doji** — tiny body (indecision, trend might reverse)
- **Hammer** — small body at top, long lower wick (buyers stepping in — bullish)
- **Hanging Man** — same shape as hammer but after an uptrend (bearish)
- **Shooting Star** — small body at bottom, long upper wick (sellers stepping in — bearish)
- **Inverted Hammer** — same shape as shooting star but after a downtrend (bullish)
- **Bullish Engulfing** — large bullish candle completely swallows previous bearish candle (strong reversal)
- **Bearish Engulfing** — large bearish candle completely swallows previous bullish candle (strong reversal)
- **Bullish Marubozu** — full-bodied bullish candle with almost no wicks (strong conviction buying)
- **Bearish Marubozu** — full-bodied bearish candle with almost no wicks (strong conviction selling)
**What the bot does with it:** It counts how many bullish patterns vs bearish patterns appeared in the last few candles. More bullish = buy signal. More bearish = sell signal. If a pattern appears right at a key support or resistance level, the confidence gets a 20% boost because patterns at key levels are more meaningful.
**Weight:** 1.5, same in both trending and ranging markets.
---
### Informational Indicators (Not Scored)
These indicators do not contribute to the buy/sell score but provide context:
- **ATR (Average True Range):** Measures volatility — how much the price moves on average per candle. Used for setting stop loss distances and classifying the market as low/normal/high/extreme volatility.
- **MACD Histogram:** Used internally by the divergence detector, not scored on its own.
- **Swing Structure (Support/Resistance):** Identifies the 3 nearest swing highs and 3 nearest swing lows. Used for setting take profit targets and stop losses.
### Dormant Indicators (Implemented But Not Active)
These are fully coded but currently switched off:
- **Stochastic Oscillator** — Similar to RSI but with a different calculation
- **Ichimoku Cloud** — Japanese trend-following system
- **Pivot Points** — Classic support/resistance levels
- **Volume Analysis** — Trading volume patterns
---
## How Are The Indicators Combined?
The bot does not just average the indicators. It uses a **regime-adaptive weighted scoring system**.
### Step 1: Classify the Market Regime
Using ADX and EMA alignment, the bot first determines whether the market is:
- **Trending up** — ADX above 25 and EMAs are bullishly aligned
- **Trending down** — ADX above 25 and EMAs are bearishly aligned
- **Ranging** — ADX below 25 (sideways/choppy market)
### Step 2: Adjust Indicator Weights
Depending on the regime, certain indicators get louder or quieter:
| Indicator | Trending Weight | Ranging Weight | Why |
|-----------|----------------|---------------|-----|
| EMA Trend | 2.0x | 0.5x | EMAs excel in trends, fail in chop |
| RSI | 1.5x | 2.0x | RSI overbought/oversold works in ranges |
| Bollinger | 0.8x | 2.0x | Mean reversion is a ranging strategy |
| Candlestick | 1.5x | 1.5x | Patterns work in both |
| Divergence | 1.0x | 1.5x | Divergences mark range boundaries |
| ADX | 1.0x | 1.0x | Confirms the regime itself |
### Step 3: Compute Composite Score
Each indicator produces a signal value (-2 to +2) and a confidence (0-100%). These are combined:
```
For each indicator:
weighted_contribution = signal_value x weight x regime_multiplier x (confidence / 100)
Composite Score = (sum of all contributions / sum of all weights) x 50
```
This produces a score from -100 (maximum bearish) to +100 (maximum bullish).
### Step 4: Map Score to Signal
| Score | Signal |
|-------|--------|
| +40 or higher | STRONG BUY |
| +15 to +39 | BUY |
| -14 to +14 | NEUTRAL |
| -15 to -39 | SELL |
| -40 or lower | STRONG SELL |
---
## Multi-Timeframe Confirmation
A signal on one timeframe alone is not enough. The bot cross-checks with other timeframes:
| Primary Timeframe | Confirmation Timeframes |
|-------------------|------------------------|
| M5 | M15 + H1 |
| M15 | M5 + H1 |
| H1 | M15 + H4 |
| H4 | H1 + Daily |
**Agreement rule:** If at least 2 out of 3 timeframes agree on direction, the signal gets a confidence boost (+10%). If they disagree, confidence is penalized (-15%) and a warning is shown.
When timeframes agree, the final score blends 60% from the primary timeframe and 40% from the average of confirmation timeframes.
---
## News & Economic Events
The bot monitors two types of fundamental data:
### Economic Calendar
- **Source:** Forex Factory (free, no API key needed)
- **What it watches:** High-impact events like interest rate decisions, NFP (jobs data), GDP, CPI (inflation), etc.
- **How it reacts:**
- If a major event is **within 30 minutes**: Confidence is hard-capped at 30% and the signal says "WAIT." Trading around major news releases is gambling, not analysis.
- If a major event is **within 60 minutes**: Confidence gets a 15% penalty.
- If a major event **just happened** (within 15 minutes ago): Caution flag applied.
### News Sentiment
- **Sources:** ForexLive and FXStreet RSS feeds (free)
- **How it works:** Headlines from the last 24 hours are scanned for bullish keywords ("rate hike", "hawkish", "beat expectations", etc.) and bearish keywords ("rate cut", "dovish", "recession", etc.). The bot counts how many of each appear and produces a sentiment score from -1.0 (very bearish) to +1.0 (very bullish).
- **How it affects signals:**
- News sentiment aligns with the technical signal: +5% confidence (confirmation)
- News sentiment conflicts with the technical signal: -10% confidence and a warning
---
## Signal Quality Grades
Every signal gets a letter grade based on its score, confidence, and multi-timeframe agreement:
| Grade | Requirements | Meaning |
|-------|-------------|---------|
| A+ | Score 45+, Confidence 70%+, MTF agrees | Highest conviction — strong setup across all dimensions |
| A | Score 30+, Confidence 60%+, MTF agrees | High conviction — solid setup with confirmation |
| B | Score 20+, Confidence 50%+ | Good setup — decent score but may lack MTF confirmation |
| C | Score 12+, Confidence 40%+ | Marginal setup — trade with caution |
| D | Score 8+ | Weak — interesting but not actionable |
| F | Below 8 | No signal — nothing to act on |
**Only grades A+, A, B, and C generate trade setups** with entry, SL, and TP. Grades D and F are informational only.
---
## Trade Setup: Entry, Stop Loss, Take Profit
When a signal qualifies (grade C or better), the bot calculates a complete trade setup:
### Stop Loss Placement
- **For BUY trades:** SL is placed below the nearest swing low, minus 1x ATR as a buffer. If that makes the SL too tight (less than 1.5x ATR from entry), it is widened to 1.5x ATR.
- **For SELL trades:** SL is placed above the nearest swing high, plus 1x ATR. Same minimum distance rule applies.
### Take Profit Placement
- **TP1 (primary target):** Placed at the nearest swing high (for buys) or swing low (for sells) — but only if that target gives a risk-reward ratio of at least 1.2:1. If no swing level qualifies, TP1 is calculated as `max(2x ATR, SL_distance x 1.2)` to guarantee the minimum risk-reward.
- **TP2 (extended target):** The next swing level beyond TP1, or a calculated target at approximately 3.5x ATR.
### Risk-Reward Filter
Every setup must have a risk-reward ratio of at least 1.2:1. If the math does not work out (target too close relative to stop loss), the setup is rejected — you will see the signal analysis but no trade setup.
### What "Mature Move" Means
If the signal score is above 30 (strong conviction), the bot treats it as a "mature move" and uses tighter take profit targets (1.5x ATR instead of 2x, capped at 2.5x ATR swing cap instead of 3x). The logic: in strong moves, it is better to take profit sooner because the move may be near exhaustion.
---
## Real-Time Price Monitoring (WebSocket)
For EUR/USD and XAU/USD, the bot also streams live prices via WebSocket (the other two pairs are monitored via periodic scans only — a limitation of the current data plan).
### Level Break Alerts
The bot caches key levels from every scan:
- 3 swing highs (resistance levels)
- 3 swing lows (support levels)
- Upper and lower Bollinger Bands
- EMA 200 (the big trend line)
When the live price crosses any of these levels, the bot:
1. Immediately runs a fresh M15 analysis
2. Sends a "Level Break Alert" to Telegram
3. Includes the full signal analysis and trade setup (if actionable)
4. Waits 5 minutes before alerting on the same pair again (prevents spam)
**Why this matters:** Key level breaks often trigger significant moves. The immediate re-scan catches the setup while it is fresh, rather than waiting for the next scheduled scan.
---
## Correlation Warnings
The bot tracks two correlation groups:
- **USD-weakness group:** EUR/USD and GBP/USD (both move inversely to USD)
- **Risk-sentiment group:** XAU/USD and USD/JPY (both respond to risk appetite)
If both pairs in a group signal the same direction, the bot warns you: "Same exposure — you may be doubling your risk on the same theme."
---
## Additional Confidence Adjustments
Beyond indicators, MTF, and news, the bot applies these final adjustments:
| Factor | Effect | Why |
|--------|--------|-----|
| Counter-trend signal | -15% confidence | Trading against the trend is riskier |
| Extreme volatility | -10% confidence | Wild markets produce unreliable signals |
| Near EMA 200 (within 0.5%) | -10% confidence | Price tends to chop around EMA 200 |
| Far from EMA 200 (beyond 1.5%) | -10% confidence | Overextended from the mean |
Final confidence is always clamped between 0% and 95% — the bot never claims 100% confidence.
---
## Telegram Interface
### Commands
| Command | What It Does |
|---------|-------------|
| `/start` | Shows the scan button grid (4 pairs x 3 timeframes) |
| `/autoscan` | Turns automatic scanning on or off |
| `/status` | Shows bot status, WebSocket connection, live prices, level cache |
| `/help` | Lists all available commands |
### Auto-Scan Behavior
- Starts automatically on boot if `TELEGRAM_CHAT_ID` is configured
- Scans all 4 pairs at each candle boundary (M5/M15/H1/H4)
- Only sends messages when actionable signals (grade A+/A/B/C) are found
- If no actionable signals are found, it stays silent (logged internally only)
### Message Format
Each signal message includes:
- Signal direction and strength (with color-coded emoji)
- Quality grade
- Composite score and confidence percentage
- Multi-timeframe agreement status
- Market regime and volatility classification
- Individual indicator breakdown
- Warnings (if any)
- Trade setup with entry, SL, TP1, TP2, and risk-reward ratio (if actionable)
---
# SECTION 2: Developer Reference
## Project Structure
```
ForexBot2/
config.py # All configuration dataclasses and constants
engine.py # Technical indicator computation engine
signal_aggregator.py # Multi-timeframe signal aggregation and trade setup
telegram_bot.py # Telegram interface, auto-scanner, WebSocket monitor
news_engine.py # Economic calendar and news sentiment
data_provider.py # Twelve Data API client with caching and rate limiting
main.py # CLI entry point for single-pair analysis
ws_twelvedata.py # Standalone WebSocket test utility
requirements.txt # Python dependencies
```
## Dependencies
```
numpy>=1.24 # Numerical computation
pandas>=2.0 # DataFrame handling for OHLCV data
requests>=2.28 # HTTP client for news/calendar APIs
twelvedata>=1.2.0 # Twelve Data SDK (REST OHLCV source)
python-telegram-bot>=20.0 # Async Telegram bot framework
websockets>=12.0 # WebSocket client for live price streaming
```
## Data Flow
```
Twelve Data REST API
|
data_provider.py
(fetch + cache + validate OHLCV)
|
engine.py
(compute 6 indicators per TF)
|
signal_aggregator.py
(MTF agreement + news + final signal + trade setup)
|
+------------+------------+
| |
telegram_bot.py main.py
(Telegram alerts + (CLI output)
WebSocket monitor)
```
## config.py
### Enums
```python
class Timeframe(Enum):
M1="1min", M5="5min", M15="15min", M30="30min",
H1="1h", H4="4h", D1="1d", W1="1wk"
class SignalStrength(Enum):
STRONG_BUY=2, BUY=1, NEUTRAL=0, SELL=-1, STRONG_SELL=-2
```
### Pair Definitions
```python
USDJPY = "USDJPY=X" # Yahoo-style suffix for forex
EURUSD = "EURUSD=X"
GBPUSD = "GBPUSD=X"
XAUUSD = "XAU/USD" # No suffix for commodities
PAIR_INFO = {
USDJPY: {"display": "USD/JPY", "pip_value": 0.01, "is_jpy_pair": True},
EURUSD: {"display": "EUR/USD", "pip_value": 0.0001, "is_jpy_pair": False},
GBPUSD: {"display": "GBP/USD", "pip_value": 0.0001, "is_jpy_pair": False},
XAUUSD: {"display": "XAU/USD", "pip_value": 0.10, "is_jpy_pair": False},
}
```
### Configuration Dataclasses
| Class | Key Fields |
|-------|-----------|
| `TechnicalConfig` | EMA periods (9/21/50/200), RSI (14, 70/30), BB (20, 2.0), MACD (12/26/9), ATR (14), ADX (14, 25/50), Stochastic (14/3, 80/20), Ichimoku (9/26/52), divergence_lookback=14, swing_sr_lookback=50, disabled_indicators=[] |
| `SignalConfig` | min_confidence=55, weights (ema=1.0, rsi=1.5, bb=1.5, candle=1.5, div=1.0), mtf_agreement_threshold=2, min_rr_ratio=1.2, atr_sl_multiplier=1.5 |
| `NewsConfig` | tracked_currencies=[USD,JPY,EUR,GBP,XAU], pre_event_buffer=30min, post_event_buffer=15min, rss_feeds=[forexlive, fxstreet] |
| `FilterConfig` | regime_filter=False (ADX on H4), session_filter=False (7-16 UTC) |
| `PlatformConfig` | Top-level container for all above + symbol, timeframes, scan_pairs, API key, ws_cooldown=300s |
---
## data_provider.py — ForexDataProvider
### Core Method: `fetch_ohlcv(symbol, timeframe, lookback_days)`
1. Checks in-memory cache — TTL equals the candle duration of the timeframe (e.g., 300s for M5)
2. If cached and fresh, returns immediately
3. Otherwise, calls Twelve Data SDK: `TDClient.time_series()`
4. Rate-limited to 55 API calls/minute (Grow plan limit)
5. Returns a pandas DataFrame with columns: `Open, High, Low, Close, Volume`
### Rate Limiting: `_throttle()`
Maintains a sliding window of timestamps. If 55 calls were made in the last 60 seconds, sleeps until a slot opens. This prevents hitting Twelve Data's rate limit.
### Validation: `_validate_and_clean(df)`
- Drops rows where all OHLC values are NaN
- Forward-fills gaps (max 3 consecutive)
- Removes future timestamps (clock drift protection)
- Fixes OHLC inconsistencies (e.g., High < Low)
- Removes statistical outliers (>10 standard deviations)
- Ensures minimum bar count for the longest indicator (200 + 10 = 210 bars)
- Deduplicates timestamps (keeps last)
### Symbol Conversion: `_to_td_symbol(symbol)`
```
"EURUSD=X" -> "EUR/USD" (strip =X, insert /)
"XAU/USD" -> "XAU/USD" (already has /, pass through)
```
### Constants
```python
RATE_LIMIT = 55
RATE_WINDOW = 60.0
BARS_PER_DAY = {M1: 1440, M5: 288, M15: 96, M30: 48, H1: 24, H4: 6, D1: 1}
CANDLE_SECONDS = {M1: 60, M5: 300, M15: 900, M30: 1800, H1: 3600, H4: 14400, D1: 86400}
```
---
## engine.py — TechnicalEngine
### Output Dataclasses
```python
@dataclass
class IndicatorSignal:
name: str # e.g. "ema_trend", "rsi"
value: float # Raw indicator value
signal: SignalStrength # BUY/SELL/NEUTRAL
confidence: float # 0-100
description: str # Human-readable explanation
weight: float = 1.0 # Importance in composite
@dataclass
class TechnicalAnalysis:
symbol: str
timeframe: str
timestamp: str
indicators: Dict[str, IndicatorSignal]
composite_signal: SignalStrength
composite_score: float # -100 to +100
confidence: float # 0-100
market_regime: str # "trending_up", "trending_down", "ranging"
volatility_regime: str # "low", "normal", "high", "extreme"
key_levels: Dict[str, float] # Swing highs/lows, BB, EMAs, ATR
pattern_signals: List[str] # Detected candlestick patterns
```
### Entry Point: `analyze(df, symbol, timeframe)`
1. Validates minimum 30 bars
2. Runs each indicator method (skips if in `disabled_indicators`)
3. Calls `_classify_regime()` based on ADX + EMA alignment
4. Calls `_compute_composite()` for weighted score
5. Returns `TechnicalAnalysis`
### Indicator Methods
Each method follows the same pattern:
1. Compute raw indicator values from the DataFrame
2. Apply thresholds to determine signal direction
3. Calculate confidence based on how extreme the reading is
4. Return an `IndicatorSignal` and store key levels
#### `_compute_ema_signals()` -> IndicatorSignal "ema_trend"
- Computes EMA 9, 21, 50, 200 using pandas `ewm()`
- Alignment score: checks ordering of price > fast > medium > slow > trend
- Normalizes to -1..+1
- Golden/death cross: checks if fast EMA crossed medium EMA in the last bar
- Stores key_levels: `ema_fast`, `ema_medium`, `ema_slow`, `ema_trend`
#### `_compute_rsi()` -> IndicatorSignal "rsi"
- Manual EMA-based RSI calculation (alpha = 1/period)
- Stores raw RSI value and series in `result._rsi_series` (used by divergence detector)
#### `_compute_macd_internal()` -> None
- Computes MACD line, signal line, and histogram
- Stores histogram in `result._macd_histogram` (consumed by divergence detector)
- Does not produce its own IndicatorSignal
#### `_compute_bollinger()` -> IndicatorSignal "bollinger"
- Middle band = 20-period SMA
- Upper/lower = middle +/- 2 standard deviations
- Position = (price - lower) / (upper - lower), range 0 to 1
- Squeeze = current band width < 75% of 20-bar average band width
- Stores key_levels: `bb_upper`, `bb_lower`, `bb_mid`
#### `_compute_adx()` -> IndicatorSignal "adx"
- Wilder's ADX calculation: True Range -> smoothed +DM/-DM -> +DI/-DI -> DX -> ADX
- Smoothing period = 14 bars
#### `_compute_atr()` -> IndicatorSignal "atr"
- True Range = max(H-L, |H-prevC|, |L-prevC|)
- ATR = EMA of True Range over 14 bars
- Percentile rank over last 50 bars determines volatility regime
- Stores key_levels: `atr`, `atr_pct` (ATR as % of price)
#### `_detect_divergences()` -> IndicatorSignal "divergence"
- `_find_swing_points(series, window=3)` -> lists of local highs and lows
- Checks RSI series and MACD histogram independently
- Bullish: price lower low + indicator higher low (in last 14 bars)
- Bearish: price higher high + indicator lower high
#### `_compute_structure_sr()` -> populates key_levels only
- Uses `_find_swing_points()` on High/Low series
- Stores nearest 3 swing highs above price and 3 swing lows below price
- key_levels: `swing_high_1/2/3`, `swing_low_1/2/3`
#### `_detect_candlestick_patterns()` -> IndicatorSignal "candlestick"
- Analyzes the last 3 candles
- Pattern detection uses body ratio, wick ratios, and multi-candle relationships
- Confidence boosted +20% (max 70%) if pattern occurs near a swing high/low
#### Dormant Methods (implemented, not called in `analyze()`)
- `_compute_stochastic()` — %K/%D crossovers, weight=2.5
- `_compute_ichimoku()` — cloud position + TK cross + cloud color, weight=0.3
- `_compute_support_resistance()` — pivot points (traditional), weight=0.8
- `_compute_volume_analysis()` — volume vs 20-bar MA, weight=0.0
### Regime Classification: `_classify_regime()`
```python
if adx >= 25 and ema_bullish: return "trending_up"
if adx >= 25 and ema_bearish: return "trending_down"
if adx >= 25: return "trending"
else: return "ranging"
```
### Composite Scoring: `_compute_composite()`
```python
regime_weights = {
"trending": {"ema_trend": 2.0, "rsi": 1.5, "bollinger": 0.8, "candlestick": 1.5, "divergence": 1.0},
"ranging": {"ema_trend": 0.5, "rsi": 2.0, "bollinger": 2.0, "candlestick": 1.5, "divergence": 1.5},
}
for indicator in active_indicators:
regime_mult = regime_weights[regime].get(indicator.name, 1.0)
contribution = signal_value * weight * regime_mult * (confidence / 100)
total += contribution
total_weight += weight * regime_mult
normalized_score = (total / total_weight) * 50 # Scale to -100..+100
```
Signal mapping: >=40 STRONG_BUY, >=15 BUY, <=-40 STRONG_SELL, <=-15 SELL, else NEUTRAL.
---
## signal_aggregator.py — SignalAggregator
### Output Dataclasses
```python
@dataclass
class TradeSetup:
pair: str # "EURUSD=X"
display_name: str # "EUR/USD"
direction: str # "BUY" or "SELL"
entry_price: float
stop_loss: float
take_profit_1: float
take_profit_2: float
risk_reward: float # e.g. 1.5
risk_pips: float
reward_pips: float
pip_value: float # From PAIR_INFO
@dataclass
class SignalReport:
symbol: str
display_name: str
timestamp: str
primary_signal / primary_score / primary_confidence
mtf_agreement: bool
mtf_signals: Dict[str, SignalStrength]
mtf_scores: Dict[str, float]
primary_analysis: TechnicalAnalysis
confirmation_analyses: List[TechnicalAnalysis]
event_risk / event_description # From news engine
news_sentiment / news_description # From news engine
final_signal / final_score / final_confidence
action_recommendation: str
signal_quality: str # "A+", "A", "B", "C", "D", "F"
trade_setup: Optional[TradeSetup]
bullish_reasons / bearish_reasons / warnings: List[str]
current_price / spread_estimate
market_regime / volatility_regime
```
### Entry Point: `analyze_pair(symbol, context)`
```
1. Determine timeframes: primary + confirmations (+ regime filter TF if enabled)
2. Fetch OHLCV for each timeframe via ForexDataProvider
3. Call analyze_from_dataframes()
4. Apply regime filter (if enabled): ADX on H4 < 25 -> NEUTRAL, quality F
5. Apply session filter (if enabled): outside 7-16 UTC -> NEUTRAL, quality F
6. Return SignalReport
```
### `analyze_from_dataframes(symbol, dfs, context, use_news)`
```
1. Run TechnicalEngine.analyze() on primary TF -> primary_analysis
2. Run TechnicalEngine.analyze() on each confirmation TF -> confirmation_analyses
3. Populate: primary_signal, primary_score, primary_confidence, market_regime, volatility_regime
4. Map MTF signals and scores
5. Check MTF agreement (>= 2 of 3 same direction)
6. Get current price from primary data (Close[-1])
7. Estimate spread from primary data (median of High-Low)
8. If use_news:
- Fetch FundamentalContext (economic calendar + RSS sentiment)
- Get event risk level for this pair
- Get pair sentiment (base currency - quote currency)
9. Compute final signal (_compute_final_signal)
10. If quality in (A+, A, B, C): generate trade setup
11. Collect bullish/bearish reasons from indicators
12. Return SignalReport
```
### `_compute_final_signal(report)`
Sequential confidence adjustments:
```
score = primary_score
confidence = primary_confidence
# 1. MTF Agreement
if mtf_agreement:
confidence += 10
score = 0.6 * primary_score + 0.4 * mean(mtf_scores)
else:
confidence -= 15
# 2. News sentiment
if sentiment aligns with score: confidence += 5
if sentiment conflicts with score: confidence -= 10
# 3. Event risk
if "danger": confidence = min(confidence, 30)
if "caution": confidence -= 15
# 4. Counter-trend
if signal opposes market_regime: confidence -= 15
# 5. Extreme volatility
if volatility_regime == "extreme": confidence -= 10
# 6. EMA 200 distance
ema_dist = |price - ema200| / price
if ema_dist < 0.005 or ema_dist > 0.015: confidence -= 10
# 7. Clamp
confidence = clamp(confidence, 0, 95)
```
Then determine final signal:
```python
if score >= 35 and confidence >= 60: STRONG_BUY
elif score >= 12 and confidence >= 45: BUY
elif score <= -35 and confidence >= 60: STRONG_SELL
elif score <= -12 and confidence >= 45: SELL
else: NEUTRAL
```
Quality grading:
```python
if |score| >= 45 and conf >= 70 and mtf_agreement: "A+"
elif |score| >= 30 and conf >= 60 and mtf_agreement: "A"
elif |score| >= 20 and conf >= 50: "B"
elif |score| >= 12 and conf >= 40: "C"
elif |score| >= 8: "D"
else: "F"
```
### `_generate_trade_setup(report, pip_value)`
```
Returns None if: no primary_analysis, ATR == 0, or RR < min_rr - 1e-9
mature_move = |score| > 30
tp_mult = 1.5 if mature else 2.0
tp_swing_cap = 2.5 if mature else 3.0
BUY path:
SL = nearest swing_low - 1.0 * ATR (min distance: 1.5 * ATR)
TP1 = nearest swing_high above price (if RR >= min_rr and dist <= tp_swing_cap * ATR)
else max(ATR * tp_mult, sl_distance * min_rr)
TP2 = next swing_high or max(ATR * (tp_mult + 1.5), sl_distance * min_rr * 1.5)
SELL path: mirror logic with swing_highs for SL and swing_lows for TP
Final RR check: tp1_distance / sl_distance >= min_rr (1.2) — reject if fails
```
---
## news_engine.py — NewsEngine
### Data Sources
| Source | URL | Type | Auth |
|--------|-----|------|------|
| Economic Calendar | `nfs.faireconomy.media/ff_calendar_thisweek.json` | REST JSON | None |
| ForexLive | `www.forexlive.com/feed/` | RSS XML | None |
| FXStreet | `www.fxstreet.com/rss` | RSS XML | None |
### Sentiment Keyword Lists
**Bullish (18 keywords):** rate hike, hawkish, strong growth, beat expectations, above forecast, surplus, expansion, employment gains, strong jobs, gdp growth, inflation rising, tightening, positive surprise, record high, bullish, rally, stronger than expected, upbeat
**Bearish (17 keywords):** rate cut, dovish, recession, slowdown, contraction, below forecast, deficit, miss expectations, weak data, unemployment rise, negative surprise, bearish, selloff, weaker than expected, downbeat, easing, stimulus
**High-Impact Events (15 types):** interest rate decision, nonfarm payrolls, non-farm, gdp, cpi, inflation, employment change, retail sales, pmi, fomc, ecb press conference, boe rate, rba rate, snb rate, boc rate
### Currency Detection Keywords
```python
CURRENCY_KEYWORDS = {
"USD": ["dollar", "fed", "fomc", "treasury", "us ", "american", "nonfarm", "nfp"],
"EUR": ["euro", "ecb", "eurozone", "eu ", "german", "french"],
"GBP": ["pound", "sterling", "boe", "bank of england", "uk ", "british"],
"JPY": ["yen", "boj", "bank of japan", "japanese"],
"XAU": ["gold", "xau", "precious metal", "safe haven", "bullion"],
# Also: CHF, AUD, CAD, NZD
}
```
### Key Methods
- `get_fundamental_context()` -> `FundamentalContext`: Fetches calendar + RSS, aggregates per-currency sentiment
- `get_event_risk_level(symbol, context)` -> `(str, str)`: Returns "danger"/"caution"/"clear" + description
- `get_pair_sentiment(symbol, context)` -> `(float, str)`: Returns `base_sentiment - quote_sentiment` and description
- `_analyze_headline_sentiment(headline)` -> `float`: `(bullish_count - bearish_count) / total`
### Pair Impact Mapping
```python
"USD" -> [EURUSD, USDJPY, GBPUSD, XAUUSD] # USD events affect all pairs
"EUR" -> [EURUSD]
"JPY" -> [USDJPY]
"GBP" -> [GBPUSD]
"XAU" -> [XAUUSD]
```
### Failure Mode
If the calendar API or RSS feeds are unreachable, the engine returns empty data — it never generates fake events or false sentiment. The signal pipeline continues without fundamental context (graceful degradation).
---
## telegram_bot.py
### Architecture
The bot runs a single async event loop with 3 concurrent subsystems:
```
Application (python-telegram-bot)
|
+-- Command handlers (/start, /help, /status, /autoscan)
+-- Callback handler (scan button presses)
+-- _candle_loop() x4 (M5, M15, H1, H4 async tasks)
+-- _ws_monitor() (WebSocket async task)
```
### Module-Level State
```python
_autoscan_chats: Set[int] # Chat IDs with active auto-scan
_autoscan_tasks: List[asyncio.Task] # Running candle loop tasks
_ws_task: Optional[asyncio.Task] # WebSocket monitor task
_level_cache: Dict[str, Dict[str, float]] # symbol -> {level_name: price}
_last_prices: Dict[str, float] # symbol -> last WS tick price
_scan_cooldowns: Dict[str, float] # symbol -> last breach scan timestamp
_executor: ThreadPoolExecutor(max_workers=2) # For blocking signal computation
```
### Boot Sequence: `_post_init(app)`
1. Register bot command menu via `set_my_commands()`
2. If `TELEGRAM_CHAT_ID` is set:
- Add chat to `_autoscan_chats`
- Run initial M15 scan for all pairs (populates level cache for WebSocket)
- Start scheduler: 4 candle loops + WebSocket monitor
- Send boot message with next scan times
### Signal Execution: `_run_signal_sync()` and `run_signal()`
`_run_signal_sync()` is a blocking function that runs in a thread pool:
```python
def _run_signal_sync(symbol, timeframe, confirmation_tfs):
config = PlatformConfig()
config.primary_timeframe = timeframe
config.confirmation_timeframes = confirmation_tfs or TF_CONFIRMATIONS[timeframe]
aggregator = SignalAggregator(config)
aggregator.data_provider = _get_shared_dp() # Shared instance for cache reuse
return aggregator.analyze_pair(symbol=symbol)
```
`run_signal()` is the async wrapper using `loop.run_in_executor()`.
### Auto-Scan Flow: `_scan_timeframe(bot, tf, confirmation_tfs)`
```python
1. For each symbol in SCAN_PAIRS:
report = await run_signal(symbol, tf, confirmation_tfs)
2. _update_level_cache(reports) # Refresh cached levels for WS breach detection
3. If tf == M5: return # Data only, no Telegram output
4. Filter actionable: quality in (A+, A, B, C)
5. If none actionable: log and return silently
6. Format scan summary (one-liner per pair)
7. For each autoscan chat:
Send summary message
Send detail message per actionable signal (includes trade setup)
```
### WebSocket Monitor: `_ws_monitor(bot)`
```python
while _autoscan_chats:
connect to wss://ws.twelvedata.com/v1/quotes/price?apikey=...
subscribe to ["EUR/USD", "XAU/USD"]
for each incoming message:
if event != "price": continue
map ws_symbol -> pair_symbol via _WS_TO_PAIR
breached = _check_level_breach(pair_symbol, price)
_last_prices[pair_symbol] = price
if breached:
asyncio.create_task(_handle_breach(...))
on error: exponential backoff (1s, 2s, 4s, ..., max 60s)
```
### Level Breach Detection: `_check_level_breach(symbol, price)`
```python
levels = _level_cache.get(symbol) # Cached from last scan
last = _last_prices.get(symbol) # Previous tick
for name, level in levels.items():
if (last < level <= price) or (last > level >= price):
return name # Price crossed this level
return None
```
### Breach Handler: `_handle_breach(bot, ws_symbol, pair_symbol, price, level_name)`
```python
1. Check 5-minute cooldown per pair
2. Run immediate M15 scan: run_signal(pair_symbol, Timeframe.M15)
3. Update level cache with fresh levels
4. Format: "Level Break -- {pair}\nPrice crossed {level} at {price}" + full signal message
5. Send to all autoscan chats
```
### Message Formatting
`format_signal_message(report)` builds a Markdown message:
```
Signal header (emoji, direction, quality, score, confidence, MTF status)
MTF breakdown per timeframe
Indicator details (arrow + name + description per indicator)
Warnings
Trade Setup (if present): direction, entry, SL, TP1, TP2, RR
Footer: "Educational / research use only."
```
`_format_scan_summary(reports, tf)` builds a compact multi-pair summary:
```
Timeframe Auto-Scan | HH:MM UTC
Per pair: emoji, signal, quality, score, confidence, MTF status
Actionable list
Correlation warnings (if any)
```
### Telegram Commands
| Handler | Function | Registration |
|---------|----------|-------------|
| `/start` | `cmd_start()` | Shows scan keyboard (12 buttons: 4 pairs x 3 TFs) |
| `/help` | `cmd_help()` | Lists all commands |
| `/status` | `cmd_status()` | Auto-scan state, WS state, live prices, level cache stats, cooldowns |
| `/autoscan` | `cmd_autoscan()` | Toggle: adds/removes chat from `_autoscan_chats`, starts/stops scheduler |
| Button press | `cb_scan()` | Runs single-pair single-TF scan, sends formatted result |
---
## main.py — CLI Entry Point
### Usage
```bash
python main.py # Default: USD/JPY H1
python main.py --timeframe M15 # USD/JPY M15
python main.py --detail # Include full indicator breakdown
python main.py --json # Output as JSON
python main.py --verbose # Debug logging
```
### Confirmation Timeframes (hardcoded per primary)
```python
M5: [M1, M15] M15: [M5, H1] M30: [M15, H1]
H1: [M15, H4] H4: [H1, D1] D1: [H4, W1]
```
### Output Format
Terminal output includes color-coded sections:
- Decision line with signal direction, quality grade, score, confidence
- Trade setup block (entry, SL, TP1, TP2, RR)
- Context section (price, regime, volatility, MTF breakdown)
- Top 3 bullish and bearish reasons
- Warnings, event risk, news sentiment
- Indicator details (with `--detail` flag)
- Key structure levels
---
## Key Design Decisions
1. **Thread pool for signal computation:** `TechnicalEngine.analyze()` and data fetching are CPU/IO-bound synchronous code. They run in a 2-thread `ThreadPoolExecutor` to avoid blocking the async Telegram event loop.
2. **Shared data provider:** A single `ForexDataProvider` instance is reused across all signal computations (`_get_shared_dp()`). This maximizes cache hits — when M15 and M5 scans both need H1 data, it is fetched once.
3. **WebSocket limited to 2 symbols:** The Twelve Data Grow plan (8 trial WS credits) only supports EUR/USD and XAU/USD via WebSocket. GBP/USD and USD/JPY are monitored exclusively through periodic REST scans.
4. **M5 is data-only:** The M5 scan runs every 5 minutes but only updates the level cache (used by WebSocket breach detection). It does not send Telegram messages — this reduces noise while keeping the internal picture current.
5. **Graceful degradation everywhere:** If news APIs fail, signals proceed without fundamental context. If WebSocket disconnects, it reconnects with exponential backoff. If a single pair's scan fails, others continue.
6. **Regime-adaptive weights:** Rather than fixed indicator weights, the composite scoring dynamically adjusts based on whether the market is trending or ranging. This is the single biggest factor in signal quality.
7. **Float tolerance on RR gate:** The trade setup RR check uses `rr < min_rr - 1e-9` instead of `rr < min_rr` to prevent edge-case rejection from floating-point rounding when the TP is calculated to exactly meet the minimum ratio.