Tiingo WebSocket streaming: re-enable the level-breach monitor
The monitor has been dead since the Oanda feed switch (Oanda's provider does not stream). New config.json scalar data_providers.streaming selects a tick-stream provider independent of the candle feed; telegram_bot resolves it via _get_streaming_dp() (separate instance, candle cache stays on the active provider) and falls back to the active provider when unset — behavior is byte-identical to before when the key is absent. TiingoDataProvider grows the two sanctioned streaming overrides: supports_streaming=True and iter_ticks() on wss://api.tiingo.com/fx (mid price only — the no-spread rule extends to the stream). Two deliberate divergences from the TD reference implementation, both because Tiingo authenticates inside the socket: server "E" messages raise TiingoDataError (a swallowed auth error would end the generator cleanly and reconnect-hammer with zero backoff), and on_connect fires on the first received message rather than after the subscribe send. Field order of quote frames was live-verified: ask is data[7], not data[6] as the docs table suggests (data[6] is askSize). make_data_provider(config, name=None) now takes an explicit provider name; keyless streaming providers disable the monitor with one warning instead of a traceback-per-minute loop. Tests 266 → 273 (WS parser, ticker round-trip, streaming config scalar, explicit-name dispatch, streaming-dp resolution, monitor auto-disable). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nsd8HcrV67GbCZXxZEn1Ma
This commit is contained in:
@@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
python test_all.py # QA suite: 266 tests, 18 modules, no network (mocked)
|
||||
python test_all.py # QA suite: 273 tests, 18 modules, no network (mocked)
|
||||
python main.py --pair XAUUSD --timeframe H1 --detail # CLI signal report (--json for full dump)
|
||||
python telegram_bot.py # the bot (refuses to boot without credentials)
|
||||
python oanda_daemon.py # auto-execution daemon on Oanda practice account
|
||||
@@ -30,7 +30,7 @@ Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy
|
||||
|
||||
## Configuration & credentials
|
||||
|
||||
Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`.
|
||||
Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -49,12 +49,12 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
|
||||
|
||||
### Layers
|
||||
|
||||
- **`providers/`** — one class per provider subclassing `BaseDataProvider` (`providers/base.py`), which owns everything the pipeline depends on: TTL+boundary cache, forming-candle drop, freshness check, validate/clean, paginated `fetch_range` (feeds the backtester from any provider), optional tick streaming. Subclasses implement only `_fetch_raw` + symbol/interval maps. Registry + `make_data_provider` in `providers/__init__.py`; config.json `data_providers.active` selects the live feed. A test enforces that subclasses don't override shared base methods. Tiingo is candles-only — do not re-add spread display from its free FX feed (two-tier indicative quotes, removed 2026-07-23).
|
||||
- **`providers/`** — one class per provider subclassing `BaseDataProvider` (`providers/base.py`), which owns everything the pipeline depends on: TTL+boundary cache, forming-candle drop, freshness check, validate/clean, paginated `fetch_range` (feeds the backtester from any provider), optional tick streaming. Subclasses implement only `_fetch_raw` + symbol/interval maps. Registry + `make_data_provider(config, name=None)` in `providers/__init__.py`; config.json `data_providers.active` selects the live candle feed and the optional `data_providers.streaming` scalar selects a different provider for the tick stream. A test enforces that subclasses don't override shared base methods (`supports_streaming`/`iter_ticks` are the sanctioned streaming override points). Tiingo streams mid-price ticks (`wss://api.tiingo.com/fx`) — do not re-add spread display from its free FX feed (two-tier indicative quotes, removed 2026-07-23); the no-spread rule extends to the WS stream.
|
||||
- **`engine.py`** — `TechnicalEngine`: 9 scored indicators (ema_trend, rsi, macd, bollinger, adx, roc, keltner, divergence, candlestick) + ATR as context. Regime classification (trending_up/down, `trending` = mixed/direction-UNKNOWN, transitioning, ranging) drives per-indicator weight multipliers and regime-aware RSI reinterpretation. Compute order matters (ATR first; MACD always computed even when not a pillar — divergence needs the histogram). Private series ride on the result as `_underscore` attrs; scorers/formatters skip anything `_`-prefixed or weight-0.
|
||||
- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
|
||||
- **`pair_signals/`** — per-pair config factories. **Always use the factory (`produce_signal` / `make_config`), never bare `PlatformConfig()`** — the factory applies the config.json overlay first, then the validated per-TF deltas (M5 hour-9 block, M15 trail arm 0.70, M15 SELL-hour block, etc.; evidence in the docstring). A test enforces factory ≡ inline-config equivalence. Confirmation map: M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1].
|
||||
- **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`).
|
||||
- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), WebSocket level-breach monitor (auto-disabled when the active provider has `supports_streaming=False` — Oanda does not stream). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
|
||||
- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
|
||||
- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown, trail ratchets at bar close exactly once per bar. Accepted divergence: one position total.
|
||||
- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user