Files
ForexBot2/DOCUMENTATION.md
T
harwinder 8ed6454ea5 Refocus on XAU/USD: add momentum indicators, tune TP, exhaustion block
Strip EUR/USD, GBP/USD, USD/JPY pairs and strategies from entire codebase.
Add 3 new scored indicators (ROC, Keltner/TTM Squeeze, ADX slope) and
enable MACD-as-pillar + divergence detection for early trend entry.
Introduce "transitioning" regime (ADX < 25, slope > 3.0) to catch trend
births before ADX confirms. Gold-specific composite weights per regime.

Tune TP multipliers: quality-tiered scaling (A+/A get 1.5-2.0x ATR via
XAU ATR method, B/C use swing-level targeting). Backtest: +24R total,
65% WR, PF 1.12 over 365 days.

Add momentum-confirmed exhaustion block: hard-reject trades when 2+
static exhaustion signals fire AND ROC/MACD confirm fading momentum.
Prevents late trend entries without killing strong trend trades.

Telegram: simplify buttons to timeframe-only labels, add "Scan All TFs"
multi-timeframe summary, add /scan command, fix BB/KC level breach keys,
fix price formatting (.2f for gold), fix deprecated asyncio call, fix
"1 pairs" grammar and 60s/30s dashboard refresh mismatch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-15 10:51:51 -04:00

1039 lines
43 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ForexBot2 — Complete Documentation
---
# SECTION 1: How It Works (For Traders & Everyone)
## What Is ForexBot2?
ForexBot2 is an automated XAU/USD (gold) signal intelligence platform. It analyzes gold across multiple timeframes using 9 technical indicators with regime-adaptive weights, checks the news, and sends you alerts on Telegram when it finds a good trading opportunity — complete with entry price, stop loss, and take profit levels.
Think of it as a tireless analyst who never sleeps, never gets emotional, and checks the charts every few minutes across multiple timeframes.
## What Does It Watch?
| Pair | What It Is |
|------|-----------|
| XAU/USD | Gold vs US Dollar |
## What Timeframes Does It Scan?
The bot looks at the same pair across multiple zoom levels, like looking at a map at different scales:
| Timeframe | What It Shows | How Often It Scans |
|-----------|--------------|-------------------|
| 5-Minute (M5) | Very short-term price action | Every 5 minutes — **data collection only**, no alerts |
| 15-Minute (M15) | Short-term moves | Every 15 minutes |
| 1-Hour (H1) | Intraday trends | Every hour |
| 4-Hour (H4) | Swing-level trends | Every 4 hours |
The M5 scan runs silently in the background — it gathers data to keep the bot's internal picture fresh, but it does not send you any messages. Only M15, H1, and H4 scans produce alerts.
## How Does It Decide What To Signal?
The bot uses a layered decision process. It is not based on a single indicator — it combines multiple tools, each looking at the market from a different angle.
### The 9 Active Indicators
#### 1. EMA Trend (Exponential Moving Averages)
**What it is:** An EMA is a smoothed line that follows the average price. The bot uses 4 EMAs with different speeds:
- **EMA 9** (fast) — reacts quickly to price changes
- **EMA 21** (medium) — smooths out noise
- **EMA 50** (slow) — shows the broader direction
- **EMA 200** (trend) — the "big picture" trend line that institutional traders watch
**How it works:** When the faster lines are above the slower lines, and price is above all of them, the market is in a clear uptrend. When everything is flipped (price below all lines, fast below slow), it is a downtrend.
**What the bot does with it:** It scores how well-aligned the EMAs are. Perfect bullish alignment = strong buy signal. Perfect bearish alignment = strong sell. Mixed or tangled EMAs = neutral. It also detects "golden crosses" (fast EMA crossing above medium — bullish) and "death crosses" (the opposite — bearish).
**Weight in the final decision:** Base weight of 1.0. In trending markets, this weight doubles to 2.0 because trends make EMAs more reliable. In ranging (sideways) markets, the weight drops to 0.5 because EMAs give false signals in chop.
---
#### 2. RSI (Relative Strength Index)
**What it is:** RSI measures how fast and how much the price has been going up vs down recently. It produces a number from 0 to 100.
**How to read it:**
- **Above 70** = "overbought" — price has risen too fast and may pull back
- **Below 30** = "oversold" — price has fallen too fast and may bounce
- **Between 40-60** = neutral territory
**What the bot does with it:** When RSI is above 70, the bot flags it as a sell signal (the move is exhausted). Below 30, it is a buy signal (the selling is overdone). In between, it looks at which side of 50 the RSI is on for a directional lean.
**Weight:** 1.5 base weight. Gets boosted to 2.0 in ranging markets (RSI works best when price bounces between support and resistance) and 1.5 in trending markets.
---
#### 3. Bollinger Bands
**What it is:** Bollinger Bands are three lines drawn around the price:
- A middle line (20-period moving average)
- An upper band (2 standard deviations above the average)
- A lower band (2 standard deviations below the average)
The bands expand when the market is volatile and contract when it is quiet.
**How to read it:**
- Price touching or breaking above the upper band = potentially overbought
- Price touching or breaking below the lower band = potentially oversold
- **Squeeze** (bands getting very tight) = a big move is coming, direction unknown yet
**What the bot does with it:** It calculates where the price sits within the bands (0 = at lower band, 1 = at upper band). Above 0.8 = sell signal. Below 0.2 = buy signal. It also detects squeezes (band width less than 75% of its recent average) and increases confidence when one is forming, because a breakout is imminent.
**Weight:** 1.5 base. Gets boosted to 2.0 in ranging markets (bands work best in ranges) and reduced to 0.8 in trending markets (price can "ride the band" for extended periods in a trend, making the overbought/oversold reading misleading).
---
#### 4. ADX (Average Directional Index)
**What it is:** ADX measures the *strength* of a trend, not its direction. It is a number from 0 to 100.
**How to read it:**
- **Below 25** = weak trend or ranging market
- **25-50** = strong trend
- **Above 50** = very strong trend
The *direction* comes from two companion lines: +DI (bullish pressure) and -DI (bearish pressure). Whichever is higher tells you the direction.
**What the bot does with it:** ADX has two roles. First, it helps classify the market as "trending" or "ranging" — this changes how much the bot trusts other indicators. Second, when the trend is strong (ADX above 25) and +DI is above -DI, it adds a buy signal; if -DI is winning, it adds a sell signal.
**Weight:** 0.5 — deliberately low because ADX confirms trend strength but is not great at timing entries.
---
#### 5. Divergence Detection (RSI + MACD)
**What it is:** Divergence happens when the price makes a new high (or low) but the indicator does not confirm it. This is one of the most powerful reversal signals in technical analysis.
**Types:**
- **Bullish divergence:** Price makes a lower low, but RSI or MACD makes a higher low. Translation: selling pressure is weakening even though price is still falling. A bounce is likely.
- **Bearish divergence:** Price makes a higher high, but the indicator makes a lower high. Translation: buying pressure is fading even though price is still rising. A pullback is likely.
**What the bot does with it:** It looks back 14 bars, finds swing points in both price and the indicators (RSI and MACD histogram), and checks if they disagree. If divergence is found, it flags it with 70% confidence.
**Weight:** 1.0 base. Gets boosted to 1.5 in ranging markets (divergences are strongest at range boundaries).
---
#### 6. Candlestick Patterns
**What it is:** Candlestick patterns are specific shapes formed by one or two candles that have historically predicted price direction.
**Patterns the bot detects:**
- **Doji** — tiny body (indecision, trend might reverse)
- **Hammer** — small body at top, long lower wick (buyers stepping in — bullish)
- **Hanging Man** — same shape as hammer but after an uptrend (bearish)
- **Shooting Star** — small body at bottom, long upper wick (sellers stepping in — bearish)
- **Inverted Hammer** — same shape as shooting star but after a downtrend (bullish)
- **Bullish Engulfing** — large bullish candle completely swallows previous bearish candle (strong reversal)
- **Bearish Engulfing** — large bearish candle completely swallows previous bullish candle (strong reversal)
- **Bullish Marubozu** — full-bodied bullish candle with almost no wicks (strong conviction buying)
- **Bearish Marubozu** — full-bodied bearish candle with almost no wicks (strong conviction selling)
**What the bot does with it:** It counts how many bullish patterns vs bearish patterns appeared in the last few candles. More bullish = buy signal. More bearish = sell signal. If a pattern appears right at a key support or resistance level, the confidence gets a 20% boost because patterns at key levels are more meaningful.
**Weight:** 1.5, same in both trending and ranging markets.
---
#### 7. MACD (Moving Average Convergence Divergence)
**What it is:** MACD tracks the difference between a fast (12-period) and slow (26-period) EMA, with a signal line (9-period EMA of the MACD). The histogram shows the difference between MACD and signal.
**How the bot uses it:** Histogram crossing above zero = bullish momentum building. Crossing below zero = bearish. Histogram acceleration (getting larger) confirms trend strength. MACD is gold's best leading indicator — it catches momentum shifts before price breaks key levels.
**Weight:** 2.0 in trending markets, 2.5 in transitioning markets, 1.5 in ranging markets.
---
#### 8. ROC (Rate of Change)
**What it is:** ROC measures the percentage change in price over the last N bars (default: 10). It quantifies momentum acceleration.
**How to read it:**
- ROC > 0 and rising = bullish momentum accelerating
- ROC crossing zero = directional shift
- |ROC| > 2% = strong momentum (possible exhaustion)
**What the bot does with it:** ROC catches the speed of trend moves. Combined with Keltner squeeze release, it identifies breakout moments. Zero-line crossovers provide high-confidence directional signals.
**Weight:** 1.5 in trending, 2.0 in transitioning, 0.5 in ranging.
---
#### 9. Keltner Channels (TTM Squeeze)
**What it is:** Keltner Channels are an EMA(20) midline with bands at ±1.5×ATR(10). When Bollinger Bands contract inside Keltner Channels, that is a "TTM Squeeze" — volatility compression preceding a big move.
**How to read it:**
- Squeeze ON = bands compressed, breakout imminent
- Squeeze RELEASED = explosion happening now, direction from MACD
- Price above upper channel = strong uptrend continuation
- Price below lower channel = strong downtrend continuation
**What the bot does with it:** The squeeze release is the single best range-to-trend transition signal. When the squeeze fires and MACD agrees on direction, the bot generates a high-confidence signal. This is the most important indicator in the "transitioning" regime.
**Weight:** 2.5 in transitioning (squeeze breakouts), 2.0 in ranging, 1.5 in trending.
---
### Informational Indicators (Not Scored)
These indicators do not contribute to the buy/sell score but provide context:
- **ATR (Average True Range):** Measures volatility — how much the price moves on average per candle. Used for setting stop loss distances and classifying the market as low/normal/high/extreme volatility.
- **ADX Slope:** Computed as ADX_now minus ADX_5_bars_ago. When ADX < 25 but slope > 3.0, the bot classifies the market as "transitioning" — a trend is being born. This catches trends 5-10 bars before traditional ADX threshold crossings.
- **EMA Ribbon Compression:** When the spread between EMA9/21/50 (normalized by ATR) is very tight, a trend change is imminent. The bot boosts EMA signal confidence when ribbon expansion is detected after compression.
- **Swing Structure (Support/Resistance):** Identifies the 3 nearest swing highs and 3 nearest swing lows. Used for setting take profit targets and stop losses.
---
## How Are The Indicators Combined?
The bot does not just average the indicators. It uses a **regime-adaptive weighted scoring system**.
### Step 1: Classify the Market Regime
Using ADX, EMA alignment, and ADX slope, the bot determines the market regime:
- **Trending up** — ADX above 25 and EMAs are bullishly aligned
- **Trending down** — ADX above 25 and EMAs are bearishly aligned
- **Transitioning** — ADX below 25 but ADX slope > 3.0 (trend is being born)
- **Ranging** — ADX below 25 and ADX slope ≤ 3.0 (sideways/choppy market)
### Step 2: Adjust Indicator Weights (Gold-Specific)
Depending on the regime, certain indicators get louder or quieter:
| Indicator | Trending | Ranging | Transitioning | Why |
|-----------|----------|---------|---------------|-----|
| EMA Trend | 2.0 | 0.3 | 1.5 | EMAs excel in trends, fail in chop |
| RSI | 1.0 | 1.5 | 1.0 | RSI OB/OS works in ranges; gold doesn't mean-revert cleanly in trends |
| Bollinger | 0.5 | 1.5 | 1.0 | Gold rides the band in trends |
| MACD | 2.0 | 1.5 | 2.5 | Gold's best leading indicator |
| Keltner | 1.5 | 2.0 | 2.5 | Squeeze breakouts catch range-to-trend transitions |
| ROC | 1.5 | 0.5 | 2.0 | Momentum acceleration matters most at trend birth |
| Candlestick | 0.8 | 1.0 | 1.0 | Patterns work everywhere |
| Divergence | 2.0 | 2.5 | 2.0 | Gold's best reversal detector |
| ADX | 0.8 | 0.5 | 1.0 | Confirms the regime itself |
### Step 3: Compute Composite Score
Each indicator produces a signal value (-2 to +2) and a confidence (0-100%). These are combined:
```
For each indicator:
weighted_contribution = signal_value x weight x regime_multiplier x (confidence / 100)
Composite Score = (sum of all contributions / sum of all weights) x 50
```
This produces a score from -100 (maximum bearish) to +100 (maximum bullish).
### Step 4: Map Score to Signal
| Score | Signal |
|-------|--------|
| +40 or higher | STRONG BUY |
| +15 to +39 | BUY |
| -14 to +14 | NEUTRAL |
| -39 to -15 | SELL |
| -40 or lower | STRONG SELL |
---
## Multi-Timeframe Confirmation
A signal on one timeframe alone is not enough. The bot cross-checks with other timeframes:
| Primary Timeframe | Confirmation Timeframes |
|-------------------|------------------------|
| M5 | M15 + H1 |
| M15 | M5 + H1 |
| H1 | M15 + H4 |
| H4 | H1 + Daily |
**Agreement rule:** If at least 2 out of 3 timeframes agree on direction, the signal gets a confidence boost (+10%). If they disagree, confidence is penalized (-15%) and a warning is shown.
When timeframes agree, the final score blends 60% from the primary timeframe and 40% from the average of confirmation timeframes.
---
## News & Economic Events
The bot monitors two types of fundamental data:
### Economic Calendar
- **Source:** Forex Factory (free, no API key needed)
- **What it watches:** High-impact events like interest rate decisions, NFP (jobs data), GDP, CPI (inflation), etc.
- **How it reacts:**
- If a major event is **within 30 minutes**: Confidence is hard-capped at 30% and the signal says "WAIT." Trading around major news releases is gambling, not analysis.
- If a major event is **within 60 minutes**: Confidence gets a 15% penalty.
- If a major event **just happened** (within 15 minutes ago): Caution flag applied.
### News Sentiment
- **Sources:** ForexLive and FXStreet RSS feeds (free)
- **How it works:** Headlines from the last 24 hours are scanned for bullish keywords ("rate hike", "hawkish", "beat expectations", etc.) and bearish keywords ("rate cut", "dovish", "recession", etc.). The bot counts how many of each appear and produces a sentiment score from -1.0 (very bearish) to +1.0 (very bullish).
- **How it affects signals:**
- News sentiment aligns with the technical signal: +5% confidence (confirmation)
- News sentiment conflicts with the technical signal: -10% confidence and a warning
---
## Signal Quality Grades
Every signal gets a letter grade based on its score, confidence, and multi-timeframe agreement:
| Grade | Requirements | Meaning |
|-------|-------------|---------|
| A+ | Score 45+, Confidence 70%+, MTF agrees | Highest conviction — strong setup across all dimensions |
| A | Score 30+, Confidence 60%+, MTF agrees | High conviction — solid setup with confirmation |
| B | Score 20+, Confidence 50%+ | Good setup — decent score but may lack MTF confirmation |
| C | Score 12+, Confidence 40%+ | Marginal setup — trade with caution |
| D | Score 8+ | Weak — interesting but not actionable |
| F | Below 8 | No signal — nothing to act on |
**Only grades A+, A, B, and C generate trade setups** with entry, SL, and TP. Grades D and F are informational only.
---
## Trade Setup: Entry, Stop Loss, Take Profit
When a signal qualifies (grade C or better), the bot calculates a complete trade setup:
### Stop Loss Placement
- **For BUY trades:** SL is placed below the nearest swing low, minus 1x ATR as a buffer. If that makes the SL too tight (less than 1.5x ATR from entry), it is widened to 1.5x ATR.
- **For SELL trades:** SL is placed above the nearest swing high, plus 1x ATR. Same minimum distance rule applies.
### Take Profit Placement
- **TP1 (primary target):** Placed at the nearest swing high (for buys) or swing low (for sells) — but only if that target gives a risk-reward ratio of at least 1.2:1. If no swing level qualifies, TP1 is calculated as `max(2x ATR, SL_distance x 1.2)` to guarantee the minimum risk-reward.
- **TP2 (extended target):** The next swing level beyond TP1, or a calculated target at approximately 3.5x ATR.
### Risk-Reward Filter
Every setup must have a risk-reward ratio of at least 1.2:1. If the math does not work out (target too close relative to stop loss), the setup is rejected — you will see the signal analysis but no trade setup.
### What "Mature Move" Means
If the signal score is above 30 (strong conviction), the bot treats it as a "mature move" and uses tighter take profit targets (1.5x ATR instead of 2x, capped at 2.5x ATR swing cap instead of 3x). The logic: in strong moves, it is better to take profit sooner because the move may be near exhaustion.
---
## Real-Time Price Monitoring (WebSocket)
The bot streams live XAU/USD prices via WebSocket for real-time level breach detection.
### Level Break Alerts
The bot caches key levels from every scan:
- 3 swing highs (resistance levels)
- 3 swing lows (support levels)
- Upper and lower Bollinger Bands
- EMA 200 (the big trend line)
When the live price crosses any of these levels, the bot:
1. Immediately runs a fresh M15 analysis
2. Sends a "Level Break Alert" to Telegram
3. Includes the full signal analysis and trade setup (if actionable)
4. Waits 5 minutes before alerting on the same pair again (prevents spam)
**Why this matters:** Key level breaks often trigger significant moves. The immediate re-scan catches the setup while it is fresh, rather than waiting for the next scheduled scan.
---
## Confidence Adjustments (Diminishing Penalty Cap)
Beyond indicators, MTF, and news, the bot applies penalties for risk factors. However, penalties use a **diminishing cap** to prevent over-suppression:
1. All penalties are collected and sorted by magnitude (largest first)
2. Applied with exponential decay: 1st penalty at 100%, 2nd at 50%, 3rd at 25%
3. **Hard cap: total penalty never exceeds 30%**
| Factor | Base Penalty | Why |
|--------|-------------|-----|
| MTF disagreement | 15% | Trading against other timeframes is riskier |
| Counter-trend signal | 15% | Trading against the trend is riskier |
| Event caution | 15% | Upcoming economic event |
| Extreme volatility | 10% | Wild markets produce unreliable signals |
| News conflict | 10% | News sentiment opposes signal |
| Exhaustion warning | 8% | Momentum may be exhausted |
| No pullback | 8% | Entering without pullback reduces RR |
| Near EMA 200 | 5% | Price tends to chop around EMA 200 |
| Far from EMA 200 | 5% | Overextended from the mean |
Final confidence is always clamped between 0% and 95% — the bot never claims 100% confidence.
---
## Telegram Interface
### Commands
| Command | What It Does |
|---------|-------------|
| `/start` | Shows the scan button grid (XAU/USD × 3 timeframes) |
| `/autoscan` | Turns automatic scanning on or off |
| `/status` | Shows bot status, WebSocket connection, live prices, level cache |
| `/help` | Lists all available commands |
### Auto-Scan Behavior
- Starts automatically on boot if `TELEGRAM_CHAT_ID` is configured
- Scans XAU/USD at each candle boundary (M5/M15/H1/H4)
- Only sends messages when actionable signals (grade A+/A/B/C) are found
- If no actionable signals are found, it stays silent (logged internally only)
### Message Format
Each signal message includes:
- Signal direction and strength (with color-coded emoji)
- Quality grade
- Composite score and confidence percentage
- Multi-timeframe agreement status
- Market regime and volatility classification
- Individual indicator breakdown
- Warnings (if any)
- Trade setup with entry, SL, TP1, TP2, and risk-reward ratio (if actionable)
---
# SECTION 2: Developer Reference
## Project Structure
```
ForexBot2/
config.py # All configuration dataclasses and constants
engine.py # Technical indicator computation engine
signal_aggregator.py # Multi-timeframe signal aggregation and trade setup
telegram_bot.py # Telegram interface, auto-scanner, WebSocket monitor
news_engine.py # Economic calendar and news sentiment
data_provider.py # Twelve Data API client with caching and rate limiting
main.py # CLI entry point for single-pair analysis
requirements.txt # Python dependencies
```
## Dependencies
```
numpy>=1.24 # Numerical computation
pandas>=2.0 # DataFrame handling for OHLCV data
requests>=2.28 # HTTP client for news/calendar APIs
twelvedata>=1.2.0 # Twelve Data SDK (REST OHLCV source)
python-telegram-bot>=20.0 # Async Telegram bot framework
websockets>=12.0 # WebSocket client for live price streaming (Telegram bot level-breach monitor)
```
## Data Flow
```
Twelve Data REST API
|
data_provider.py
(fetch + cache + validate OHLCV)
|
engine.py
(compute 9 indicators per TF)
|
signal_aggregator.py
(MTF agreement + news + final signal + trade setup)
|
+------------+------------+
| |
telegram_bot.py main.py
(Telegram alerts + (CLI output)
WebSocket monitor)
```
## config.py
### Enums
```python
class Timeframe(Enum):
M1="1min", M5="5min", M15="15min", M30="30min",
H1="1h", H4="4h", D1="1d", W1="1wk"
class SignalStrength(Enum):
STRONG_BUY=2, BUY=1, NEUTRAL=0, SELL=-1, STRONG_SELL=-2
```
### Pair Definitions
```python
XAUUSD = "XAU/USD"
XAUUSD = "XAU/USD" # No suffix for commodities
PAIR_INFO = {
XAUUSD: {"display": "XAU/USD", "pip_value": 0.10, "is_jpy_pair": False},
XAUUSD: {"display": "XAU/USD", "pip_value": 0.10, "is_jpy_pair": False},
}
```
### Configuration Dataclasses
| Class | Key Fields |
|-------|-----------|
| `TechnicalConfig` | EMA periods (9/21/50/200), RSI (14, 70/30), BB (20, 2.0), MACD (12/26/9), ATR (14), ADX (14, 25/50), Stochastic (14/3, 80/20), Ichimoku (9/26/52), divergence_lookback=14, swing_sr_lookback=50, disabled_indicators=[] |
| `SignalConfig` | min_confidence=55, weights (ema=1.0, rsi=1.5, bb=1.5, candle=1.5, div=1.0), mtf_agreement_threshold=2, min_rr_ratio=1.2, atr_sl_multiplier=1.5 |
| `NewsConfig` | tracked_currencies=[USD,JPY,EUR,GBP,XAU], pre_event_buffer=30min, post_event_buffer=15min, rss_feeds=[forexlive, fxstreet] |
| `FilterConfig` | regime_filter=False (ADX on H4), session_filter=False (7-16 UTC) |
| `PlatformConfig` | Top-level container for all above + symbol, timeframes, scan_pairs, API key, ws_cooldown=300s |
---
## data_provider.py — ForexDataProvider
### Core Method: `fetch_ohlcv(symbol, timeframe, lookback_days)`
1. Checks in-memory cache — TTL equals the candle duration of the timeframe (e.g., 300s for M5)
2. If cached and fresh, returns immediately
3. Otherwise, calls Twelve Data SDK: `TDClient.time_series()`
4. Rate-limited to 55 API calls/minute (Grow plan limit)
5. Returns a pandas DataFrame with columns: `Open, High, Low, Close, Volume`
### Rate Limiting: `_throttle()`
Maintains a sliding window of timestamps. If 55 calls were made in the last 60 seconds, sleeps until a slot opens. This prevents hitting Twelve Data's rate limit.
### Validation: `_validate_and_clean(df)`
- Drops rows where all OHLC values are NaN
- Forward-fills gaps (max 3 consecutive)
- Removes future timestamps (clock drift protection)
- Fixes OHLC inconsistencies (e.g., High < Low)
- Removes statistical outliers (>10 standard deviations)
- Ensures minimum bar count for the longest indicator (200 + 10 = 210 bars)
- Deduplicates timestamps (keeps last)
### Symbol Conversion: `_to_td_symbol(symbol)`
```
"XAU/USD" -> "XAU/USD" (pass-through for slash-format symbols)
"XAU/USD" -> "XAU/USD" (already has /, pass through)
```
### Constants
```python
RATE_LIMIT = 55
RATE_WINDOW = 60.0
BARS_PER_DAY = {M1: 1440, M5: 288, M15: 96, M30: 48, H1: 24, H4: 6, D1: 1}
CANDLE_SECONDS = {M1: 60, M5: 300, M15: 900, M30: 1800, H1: 3600, H4: 14400, D1: 86400}
```
---
## engine.py — TechnicalEngine
### Output Dataclasses
```python
@dataclass
class IndicatorSignal:
name: str # e.g. "ema_trend", "rsi"
value: float # Raw indicator value
signal: SignalStrength # BUY/SELL/NEUTRAL
confidence: float # 0-100
description: str # Human-readable explanation
weight: float = 1.0 # Importance in composite
@dataclass
class TechnicalAnalysis:
symbol: str
timeframe: str
timestamp: str
indicators: Dict[str, IndicatorSignal]
composite_signal: SignalStrength
composite_score: float # -100 to +100
confidence: float # 0-100
market_regime: str # "trending_up", "trending_down", "ranging"
volatility_regime: str # "low", "normal", "high", "extreme"
key_levels: Dict[str, float] # Swing highs/lows, BB, EMAs, ATR
pattern_signals: List[str] # Detected candlestick patterns
```
### Entry Point: `analyze(df, symbol, timeframe)`
1. Validates minimum 30 bars
2. Runs each indicator method (skips if in `disabled_indicators`)
3. Calls `_classify_regime()` based on ADX + EMA alignment
4. Calls `_compute_composite()` for weighted score
5. Returns `TechnicalAnalysis`
### Indicator Methods
Each method follows the same pattern:
1. Compute raw indicator values from the DataFrame
2. Apply thresholds to determine signal direction
3. Calculate confidence based on how extreme the reading is
4. Return an `IndicatorSignal` and store key levels
#### `_compute_ema_signals()` -> IndicatorSignal "ema_trend"
- Computes EMA 9, 21, 50, 200 using pandas `ewm()`
- Alignment score: checks ordering of price > fast > medium > slow > trend
- Normalizes to -1..+1
- Golden/death cross: checks if fast EMA crossed medium EMA in the last bar
- Stores key_levels: `ema_fast`, `ema_medium`, `ema_slow`, `ema_trend`
#### `_compute_rsi()` -> IndicatorSignal "rsi"
- Manual EMA-based RSI calculation (alpha = 1/period)
- Stores raw RSI value and series in `result._rsi_series` (used by divergence detector)
#### `_compute_macd_internal()` -> None
- Computes MACD line, signal line, and histogram
- Stores histogram in `result._macd_histogram` (consumed by divergence detector)
- Does not produce its own IndicatorSignal
#### `_compute_bollinger()` -> IndicatorSignal "bollinger"
- Middle band = 20-period SMA
- Upper/lower = middle +/- 2 standard deviations
- Position = (price - lower) / (upper - lower), range 0 to 1
- Squeeze = current band width < 75% of 20-bar average band width
- Stores key_levels: `bb_upper`, `bb_lower`, `bb_mid`
#### `_compute_adx()` -> IndicatorSignal "adx"
- Wilder's ADX calculation: True Range -> smoothed +DM/-DM -> +DI/-DI -> DX -> ADX
- Smoothing period = 14 bars
#### `_compute_atr()` -> IndicatorSignal "atr"
- True Range = max(H-L, |H-prevC|, |L-prevC|)
- ATR = EMA of True Range over 14 bars
- Percentile rank over last 50 bars determines volatility regime
- Stores key_levels: `atr`, `atr_pct` (ATR as % of price)
#### `_detect_divergences()` -> IndicatorSignal "divergence"
- `_find_swing_points(series, window=3)` -> lists of local highs and lows
- Checks RSI series and MACD histogram independently
- Bullish: price lower low + indicator higher low (in last 14 bars)
- Bearish: price higher high + indicator lower high
#### `_compute_structure_sr()` -> populates key_levels only
- Uses `_find_swing_points()` on High/Low series
- Stores nearest 3 swing highs above price and 3 swing lows below price
- key_levels: `swing_high_1/2/3`, `swing_low_1/2/3`
#### `_detect_candlestick_patterns()` -> IndicatorSignal "candlestick"
- Analyzes the last 3 candles
- Pattern detection uses body ratio, wick ratios, and multi-candle relationships
- Confidence boosted +20% (max 70%) if pattern occurs near a swing high/low
#### Dormant Methods (implemented, not called in `analyze()`)
- `_compute_stochastic()` — %K/%D crossovers, weight=2.5
- `_compute_ichimoku()` — cloud position + TK cross + cloud color, weight=0.3
- `_compute_support_resistance()` — pivot points (traditional), weight=0.8
- `_compute_volume_analysis()` — volume vs 20-bar MA, weight=0.0
### Regime Classification: `_classify_regime()`
```python
if adx >= 25 and ema_bullish: return "trending_up"
if adx >= 25 and ema_bearish: return "trending_down"
if adx >= 25: return "trending"
else: return "ranging"
```
### Composite Scoring: `_compute_composite()`
```python
regime_weights = {
"trending": {"ema_trend": 2.0, "rsi": 1.5, "bollinger": 0.8, "candlestick": 1.5, "divergence": 1.0},
"ranging": {"ema_trend": 0.5, "rsi": 2.0, "bollinger": 2.0, "candlestick": 1.5, "divergence": 1.5},
}
for indicator in active_indicators:
regime_mult = regime_weights[regime].get(indicator.name, 1.0)
contribution = signal_value * weight * regime_mult * (confidence / 100)
total += contribution
total_weight += weight * regime_mult
normalized_score = (total / total_weight) * 50 # Scale to -100..+100
```
Signal mapping: >=40 STRONG_BUY, >=15 BUY, <=-40 STRONG_SELL, <=-15 SELL, else NEUTRAL.
---
## signal_aggregator.py — SignalAggregator
### Output Dataclasses
```python
@dataclass
class TradeSetup:
pair: str # "XAU/USD"
display_name: str # "XAU/USD"
direction: str # "BUY" or "SELL"
entry_price: float
stop_loss: float
take_profit_1: float
take_profit_2: float
risk_reward: float # e.g. 1.5
risk_pips: float
reward_pips: float
pip_value: float # From PAIR_INFO
@dataclass
class SignalReport:
symbol: str
display_name: str
timestamp: str
primary_signal / primary_score / primary_confidence
mtf_agreement: bool
mtf_signals: Dict[str, SignalStrength]
mtf_scores: Dict[str, float]
primary_analysis: TechnicalAnalysis
confirmation_analyses: List[TechnicalAnalysis]
event_risk / event_description # From news engine
news_sentiment / news_description # From news engine
final_signal / final_score / final_confidence
action_recommendation: str
signal_quality: str # "A+", "A", "B", "C", "D", "F"
trade_setup: Optional[TradeSetup]
bullish_reasons / bearish_reasons / warnings: List[str]
current_price / spread_estimate
market_regime / volatility_regime
```
### Entry Point: `analyze_pair(symbol, context)`
```
1. Determine timeframes: primary + confirmations (+ regime filter TF if enabled)
2. Fetch OHLCV for each timeframe via ForexDataProvider
3. Call analyze_from_dataframes()
4. Apply regime filter (if enabled): ADX on H4 < 25 -> NEUTRAL, quality F
5. Apply session filter (if enabled): outside 7-16 UTC -> NEUTRAL, quality F
6. Return SignalReport
```
### `analyze_from_dataframes(symbol, dfs, context, use_news)`
```
1. Run TechnicalEngine.analyze() on primary TF -> primary_analysis
2. Run TechnicalEngine.analyze() on each confirmation TF -> confirmation_analyses
3. Populate: primary_signal, primary_score, primary_confidence, market_regime, volatility_regime
4. Map MTF signals and scores
5. Check MTF agreement (>= 2 of 3 same direction)
6. Get current price from primary data (Close[-1])
7. Estimate spread from primary data (median of High-Low)
8. If use_news:
- Fetch FundamentalContext (economic calendar + RSS sentiment)
- Get event risk level for this pair
- Get pair sentiment (base currency - quote currency)
9. Compute final signal (_compute_final_signal)
10. If quality in (A+, A, B, C): generate trade setup
11. Collect bullish/bearish reasons from indicators
12. Return SignalReport
```
### `_compute_final_signal(report)`
Sequential confidence adjustments:
```
score = primary_score
confidence = primary_confidence
# 1. MTF Agreement
if mtf_agreement:
confidence += 10
score = 0.6 * primary_score + 0.4 * mean(mtf_scores)
else:
confidence -= 15
# 2. News sentiment
if sentiment aligns with score: confidence += 5
if sentiment conflicts with score: confidence -= 10
# 3. Event risk
if "danger": confidence = min(confidence, 30)
if "caution": confidence -= 15
# 4. Counter-trend
if signal opposes market_regime: confidence -= 15
# 5. Extreme volatility
if volatility_regime == "extreme": confidence -= 10
# 6. EMA 200 distance
ema_dist = |price - ema200| / price
if ema_dist < 0.005 or ema_dist > 0.015: confidence -= 10
# 7. Clamp
confidence = clamp(confidence, 0, 95)
```
Then determine final signal:
```python
if score >= 35 and confidence >= 60: STRONG_BUY
elif score >= 12 and confidence >= 45: BUY
elif score <= -35 and confidence >= 60: STRONG_SELL
elif score <= -12 and confidence >= 45: SELL
else: NEUTRAL
```
Quality grading:
```python
if |score| >= 45 and conf >= 70 and mtf_agreement: "A+"
elif |score| >= 30 and conf >= 60 and mtf_agreement: "A"
elif |score| >= 20 and conf >= 50: "B"
elif |score| >= 12 and conf >= 40: "C"
elif |score| >= 8: "D"
else: "F"
```
### `_generate_trade_setup(report, pip_value)`
```
Returns None if: no primary_analysis, ATR == 0, or RR < min_rr - 1e-9
mature_move = |score| > 30
tp_mult = 1.5 if mature else 2.0
tp_swing_cap = 2.5 if mature else 3.0
BUY path:
SL = nearest swing_low - 1.0 * ATR (min distance: 1.5 * ATR)
TP1 = nearest swing_high above price (if RR >= min_rr and dist <= tp_swing_cap * ATR)
else max(ATR * tp_mult, sl_distance * min_rr)
TP2 = next swing_high or max(ATR * (tp_mult + 1.5), sl_distance * min_rr * 1.5)
SELL path: mirror logic with swing_highs for SL and swing_lows for TP
Final RR check: tp1_distance / sl_distance >= min_rr (1.2) — reject if fails
```
---
## news_engine.py — NewsEngine
### Data Sources
| Source | URL | Type | Auth |
|--------|-----|------|------|
| Economic Calendar | `nfs.faireconomy.media/ff_calendar_thisweek.json` | REST JSON | None |
| ForexLive | `www.forexlive.com/feed/` | RSS XML | None |
| FXStreet | `www.fxstreet.com/rss` | RSS XML | None |
### Sentiment Keyword Lists
**Bullish (18 keywords):** rate hike, hawkish, strong growth, beat expectations, above forecast, surplus, expansion, employment gains, strong jobs, gdp growth, inflation rising, tightening, positive surprise, record high, bullish, rally, stronger than expected, upbeat
**Bearish (17 keywords):** rate cut, dovish, recession, slowdown, contraction, below forecast, deficit, miss expectations, weak data, unemployment rise, negative surprise, bearish, selloff, weaker than expected, downbeat, easing, stimulus
**High-Impact Events (15 types):** interest rate decision, nonfarm payrolls, non-farm, gdp, cpi, inflation, employment change, retail sales, pmi, fomc, ecb press conference, boe rate, rba rate, snb rate, boc rate
### Currency Detection Keywords
```python
CURRENCY_KEYWORDS = {
"USD": ["dollar", "fed", "fomc", "treasury", "us ", "american", "nonfarm", "nfp"],
"EUR": ["euro", "ecb", "eurozone", "eu ", "german", "french"],
"GBP": ["pound", "sterling", "boe", "bank of england", "uk ", "british"],
"JPY": ["yen", "boj", "bank of japan", "japanese"],
"XAU": ["gold", "xau", "precious metal", "safe haven", "bullion"],
# Also: CHF, AUD, CAD, NZD
}
```
### Key Methods
- `get_fundamental_context()` -> `FundamentalContext`: Fetches calendar + RSS, aggregates per-currency sentiment
- `get_event_risk_level(symbol, context)` -> `(str, str)`: Returns "danger"/"caution"/"clear" + description
- `get_pair_sentiment(symbol, context)` -> `(float, str)`: Returns `base_sentiment - quote_sentiment` and description
- `_analyze_headline_sentiment(headline)` -> `float`: `(bullish_count - bearish_count) / total`
### Pair Impact Mapping
```python
"USD" -> [XAUUSD] # USD events affect gold
"XAU" -> [XAUUSD]
"XAU" -> [XAUUSD]
```
### Failure Mode
If the calendar API or RSS feeds are unreachable, the engine returns empty data — it never generates fake events or false sentiment. The signal pipeline continues without fundamental context (graceful degradation).
---
## telegram_bot.py
### Architecture
The bot runs a single async event loop with 3 concurrent subsystems:
```
Application (python-telegram-bot)
|
+-- Command handlers (/start, /help, /status, /autoscan)
+-- Callback handler (scan button presses)
+-- _candle_loop() x4 (M5, M15, H1, H4 async tasks)
+-- _ws_monitor() (WebSocket async task)
```
### Module-Level State
```python
_autoscan_chats: Set[int] # Chat IDs with active auto-scan
_autoscan_tasks: List[asyncio.Task] # Running candle loop tasks
_ws_task: Optional[asyncio.Task] # WebSocket monitor task
_level_cache: Dict[str, Dict[str, float]] # symbol -> {level_name: price}
_last_prices: Dict[str, float] # symbol -> last WS tick price
_scan_cooldowns: Dict[str, float] # symbol -> last breach scan timestamp
_executor: ThreadPoolExecutor(max_workers=2) # For blocking signal computation
```
### Boot Sequence: `_post_init(app)`
1. Register bot command menu via `set_my_commands()`
2. If `TELEGRAM_CHAT_ID` is set:
- Add chat to `_autoscan_chats`
- Run initial M15 scan for all pairs (populates level cache for WebSocket)
- Start scheduler: 4 candle loops + WebSocket monitor
- Send boot message with next scan times
### Signal Execution: `_run_signal_sync()` and `run_signal()`
`_run_signal_sync()` is a blocking function that runs in a thread pool:
```python
def _run_signal_sync(symbol, timeframe, confirmation_tfs):
config = PlatformConfig()
config.primary_timeframe = timeframe
config.confirmation_timeframes = confirmation_tfs or TF_CONFIRMATIONS[timeframe]
aggregator = SignalAggregator(config)
aggregator.data_provider = _get_shared_dp() # Shared instance for cache reuse
return aggregator.analyze_pair(symbol=symbol)
```
`run_signal()` is the async wrapper using `loop.run_in_executor()`.
### Auto-Scan Flow: `_scan_timeframe(bot, tf, confirmation_tfs)`
```python
1. For each symbol in SCAN_PAIRS:
report = await run_signal(symbol, tf, confirmation_tfs)
2. _update_level_cache(reports) # Refresh cached levels for WS breach detection
3. If tf == M5: return # Data only, no Telegram output
4. Filter actionable: quality in (A+, A, B, C)
5. If none actionable: log and return silently
6. Format scan summary (one-liner per pair)
7. For each autoscan chat:
Send summary message
Send detail message per actionable signal (includes trade setup)
```
### WebSocket Monitor: `_ws_monitor(bot)`
```python
while _autoscan_chats:
connect to wss://ws.twelvedata.com/v1/quotes/price?apikey=...
subscribe to ["XAU/USD"]
for each incoming message:
if event != "price": continue
map ws_symbol -> pair_symbol via _WS_TO_PAIR
breached = _check_level_breach(pair_symbol, price)
_last_prices[pair_symbol] = price
if breached:
asyncio.create_task(_handle_breach(...))
on error: exponential backoff (1s, 2s, 4s, ..., max 60s)
```
### Level Breach Detection: `_check_level_breach(symbol, price)`
```python
levels = _level_cache.get(symbol) # Cached from last scan
last = _last_prices.get(symbol) # Previous tick
for name, level in levels.items():
if (last < level <= price) or (last > level >= price):
return name # Price crossed this level
return None
```
### Breach Handler: `_handle_breach(bot, ws_symbol, pair_symbol, price, level_name)`
```python
1. Check 5-minute cooldown per pair
2. Run immediate M15 scan: run_signal(pair_symbol, Timeframe.M15)
3. Update level cache with fresh levels
4. Format: "Level Break -- {pair}\nPrice crossed {level} at {price}" + full signal message
5. Send to all autoscan chats
```
### Message Formatting
`format_signal_message(report)` builds a Markdown message:
```
Signal header (emoji, direction, quality, score, confidence, MTF status)
MTF breakdown per timeframe
Indicator details (arrow + name + description per indicator)
Warnings
Trade Setup (if present): direction, entry, SL, TP1, TP2, RR
Footer: "Educational / research use only."
```
`_format_scan_summary(reports, tf)` builds a compact multi-pair summary:
```
Timeframe Auto-Scan | HH:MM UTC
Per pair: emoji, signal, quality, score, confidence, MTF status
Actionable list
Actionable signals (if any)
```
### Telegram Commands
| Handler | Function | Registration |
|---------|----------|-------------|
| `/start` | `cmd_start()` | Shows scan keyboard (3 buttons: XAU/USD × 3 TFs) |
| `/help` | `cmd_help()` | Lists all commands |
| `/status` | `cmd_status()` | Auto-scan state, WS state, live prices, level cache stats, cooldowns |
| `/autoscan` | `cmd_autoscan()` | Toggle: adds/removes chat from `_autoscan_chats`, starts/stops scheduler |
| Button press | `cb_scan()` | Runs single-pair single-TF scan, sends formatted result |
---
## main.py — CLI Entry Point
### Usage
```bash
python main.py # Default: XAU/USD H1
python main.py --timeframe M15 # XAU/USD M15
python main.py --detail # Include full indicator breakdown
python main.py --json # Output as JSON
python main.py --verbose # Debug logging
```
### Confirmation Timeframes (hardcoded per primary)
```python
M5: [M1, M15] M15: [M5, H1] M30: [M15, H1]
H1: [M15, H4] H4: [H1, D1] D1: [H4, W1]
```
### Output Format
Terminal output includes color-coded sections:
- Decision line with signal direction, quality grade, score, confidence
- Trade setup block (entry, SL, TP1, TP2, RR)
- Context section (price, regime, volatility, MTF breakdown)
- Top 3 bullish and bearish reasons
- Warnings, event risk, news sentiment
- Indicator details (with `--detail` flag)
- Key structure levels
---
## Key Design Decisions
1. **Thread pool for signal computation:** `TechnicalEngine.analyze()` and data fetching are CPU/IO-bound synchronous code. They run in a 2-thread `ThreadPoolExecutor` to avoid blocking the async Telegram event loop.
2. **Shared data provider:** A single `ForexDataProvider` instance is reused across all signal computations (`_get_shared_dp()`). This maximizes cache hits — when M15 and M5 scans both need H1 data, it is fetched once.
3. **WebSocket for XAU/USD only:** The bot streams live prices via WebSocket for real-time level breach detection on gold.
4. **M5 is data-only:** The M5 scan runs every 5 minutes but only updates the level cache (used by WebSocket breach detection). It does not send Telegram messages — this reduces noise while keeping the internal picture current.
5. **Graceful degradation everywhere:** If news APIs fail, signals proceed without fundamental context. If WebSocket disconnects, it reconnects with exponential backoff. If a single pair's scan fails, others continue.
6. **Regime-adaptive weights:** Rather than fixed indicator weights, the composite scoring dynamically adjusts based on whether the market is trending or ranging. This is the single biggest factor in signal quality.
7. **Float tolerance on RR gate:** The trade setup RR check uses `rr < min_rr - 1e-9` instead of `rr < min_rr` to prevent edge-case rejection from floating-point rounding when the TP is calculated to exactly meet the minimum ratio.