Remove Tiingo spread monitor (indicative two-tier feed, unvalidatable)

Tiingo's free /tiingo/fx/top publishes only a two-tier indicative
spread for gold ($0.10 liquid hours / $0.70 rollover+news; verified on
273 production samples + live probes) — mid ± a posted increment, not
executable depth, so it can never validate the $0.30 backtest cost
assumption. Removed one day after shipping: tiingo_provider.py, the
sampler/quote plumbing in telegram_bot.py, the channel_spread_detail
config knob and tiingo_api_key resolver, and Module 14's Tiingo tests
(221 -> 210). CLAUDE.md documents the verdict: do not re-add a spread
display from Tiingo's free FX feed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BoZKi67Ct6QTeiFJmybXSw

CLAUDE.md

@@ -14,9 +14,8 @@ This file is the authoritative developer/agent reference. The human-facing docs
 |------|-------|---------|
 | `config.py` | ~374 | All config dataclasses, `PAIR_INFO` registry, per-pair strategy configs. **`PlatformConfig` defaults ARE the tuned XAU config.** |
 | `app_config.py` | ~290 | config.json loader/validation, env-precedence resolvers, repo-root `.env` loader (setdefault at import), `apply_pair_overrides` deployment overlay |
-| `config.json.example` | 27 | committed schema template; real gitignored config.json holds secrets + deployment settings |
+| `config.json.example` | 24 | committed schema template; real gitignored config.json holds secrets + deployment settings |
 | `data_provider.py` | ~347 | Twelve Data SDK wrapper: fetch, rate-limit (55 credits/min), in-memory cache, validation/cleaning, freshness checks |
-| `tiingo_provider.py` | ~160 | Tiingo `/tiingo/fx/top` bid/ask quotes (supplementary, NOT an OHLCV source): spread visibility on alerts (warn-only) + 5-min sampler → `logs/spread_log.csv`. No key → feature off |
 | `engine.py` | ~1173 | `TechnicalEngine`: computes all indicators on one DataFrame → `TechnicalAnalysis` (composite score, regime, key levels) |
 | `chart_patterns.py` | ~280 | Chart-pattern detectors for the XAU_CP archetype (pure functions, ATR-unit geometry, first-close-beyond triggers) |
 | `signal_aggregator.py` | ~1447 | `SignalAggregator`: MTF analysis, news integration, gates/penalties, quality grade, trade-setup generation → `SignalReport` |
@@ -25,13 +24,13 @@ This file is the authoritative developer/agent reference. The human-facing docs
 | `pair_signals/__init__.py` | 52 | `PAIR_PRODUCERS` registry + `produce_signal()` |
 | `pair_signals/xauusd.py` | 71 | XAU config factory + per-timeframe overrides (docstring = validation record) |
 | `main.py` | ~348 | CLI: `python main.py --pair XAUUSD --timeframe H1 --detail/--json` |
-| `test_all.py` | ~3100 | QA suite: **215 tests in 14 modules**, no live API needed. Must stay green. |
+| `test_all.py` | ~3100 | QA suite: **210 tests in 14 modules**, no live API needed. Must stay green. |
 | `backtest/gpu_backtest.py` | ~2231 | Bar-replay backtester (CuPy→NumPy fallback) using the real `SignalAggregator` |
 | `backtest/diagnose_filters.py` | 609 | Filter-diagnosis tooling |
 | `backtest/research/` | — | Calendar research infra (`ff_calendar_scraper.py`, `calendar_store.py`) + result CSVs |
 | `backtest/*.csv` | — | Verification trade books from past tuning campaigns (`xau_*.csv`, …). **Gitignored, local-only** — used for manual before/after comparison, not by tests |
 | `backtest/cache/` | — | Parquet OHLCV cache (raw bars) |
-| `logs/telegram.log` | — | Bot log (3-hour rotation × 240 backups = 30-day retention); `logs/spread_log.csv` = Tiingo spread samples |
+| `logs/telegram.log` | — | Bot log (3-hour rotation × 240 backups = 30-day retention) |
 
 ---
 
@@ -273,10 +272,10 @@ Run: `python telegram_bot.py` (credentials resolve env `TELEGRAM_TOKEN`/`TELEGRA
 - **Alert gating**: quality threshold (`_alert_min_quality`, default "C", menu cycles C→B→A) + fingerprint dedup per (symbol, TF) on (signal, quality, direction) with 1-hour re-arm — a persisting identical signal doesn't re-send every candle, but a setup whose entry moved ≥ half the last-ALERTED risk distance (new pullback/swing anchor; compared against the stored last-alert entry/risk, a stable reference immune to per-candle ATR drift) re-alerts. The fingerprint is committed only **after** a message reaches ≥1 target (`_commit_alert`), so a failed Telegram send doesn't consume the alert.
 - **WebSocket monitor**: subscribes every `scan_pairs` symbol to TD quotes WS; 10 s heartbeat; exponential reconnect (≤60 s). On tick crossing any cached key level (`swing_high/low_1..3`, `ema_trend`, `bb_upper/lower`, `kc_upper/lower`; cache TTL 1 h) → immediate M15 rescan → "LEVEL BREAK" alert if actionable+changed (same gates as auto-scan), 5-min per-pair cooldown.
 - **Commands**: `/start` (menu), `/scan` (buttons: per pair — M5/M15/H1/H4 + Scan All TFs), `/autoscan` (toggle DM copy), `/status` (loops, WS, live prices, level cache, cooldowns), `/cache` (age per pair/TF), `/clearcache`, `/help`.
-- **Tiingo spread monitor** (optional; on only when a Tiingo key resolves — see Environment variables): a 5-min sampler task fetches top-of-book bid/ask per scanned pair (`tiingo_provider.py`), logs to `logs/spread_log.csv` (skipping the weekend-closed window), and caches the latest quote; alert sends refresh the quote first, and signal messages then show live bid/ask + a warning when spread ≥ 2× the pair's assumed round-trip cost (`WIDE_SPREAD_MULTIPLE`). **Warn-only BY DESIGN** — no historical bid/ask exists, so a suppressing gate could never be validated (live == backtest rule); signals never consume bid/ask. `/status` shows the live spread per pair. **License (free tier is internal-use-only):** raw bid/ask/spread numbers go to owner DMs always; the channel gets them ONLY while config.json `telegram.channel_spread_detail=true` (the channel is internal/testing as of 2026-07-22 — **flip to false before the channel goes public**), otherwise a number-free "spread unusually wide" caution via `format_signal_message(report, spread_detail=False)` — routing in `_spread_detail_for`, test-enforced, committed default false. **Rate limits (free tier 50 req/h, 1000/day, checked 2026-07-22):** sampler 12/h + alert-batch refreshes ≤17/h + breach refreshes ≤12/h = worst case ≈41/h with ONE pair; adding pairs multiplies this — recheck before growing `scan_pairs`.
+- **Tiingo spread monitor: REMOVED 2026-07-23** (shipped 2026-07-22, removed next day). Tiingo's free `/tiingo/fx/top` publishes a two-tier *indicative* spread for gold — exactly $0.10 in liquid hours and exactly $0.70 in the 21:00–00:00 UTC rollover window / news moments (verified: 273-sample production log + live probes; only those two values ever occur). Quotes are mid ± a posted increment, not executable depth, so the feature could never validate the $0.30 backtest cost assumption. Do not re-add a spread display from Tiingo's free FX feed.
 - Signal messages include the full indicator breakdown, warnings, trade setup, **trailing-stop management plan** (arm price + trail distance, read from the `trail_arm_r`/`trail_dist_r` stamped on the setup by the producing per-TF config), and a risk disclaimer on every outbound message.
 - Blocking scan work runs in `ThreadPoolExecutor(max_workers=4)`; one shared `ForexDataProvider` across all scans (cache reuse).
-- **Logging (hardened 2026-07-22)**: root-logger handlers (`force=True`) → console + `logs/telegram.log` (3-hour rotation, 240 backups = 30 days), full-date timestamps; every module logger (data_provider, tiingo_provider, news_engine, websockets, telegram.ext) lands in the same file. Failure surfaces all covered: candle loops and the spread sampler catch-and-continue per cycle (one bad scan can't kill a loop), breach tasks log exceptions via done-callback, PTB `add_error_handler` registered, and `sys.excepthook`/`threading.excepthook` (installed in `main()`) route uncaught crashes to the log. WS monitor keeps its exponential-backoff reconnect + `logger.exception`.
+- **Logging (hardened 2026-07-22)**: root-logger handlers (`force=True`) → console + `logs/telegram.log` (3-hour rotation, 240 backups = 30 days), full-date timestamps; every module logger (data_provider, news_engine, websockets, telegram.ext) lands in the same file. Failure surfaces all covered: candle loops catch-and-continue per cycle (one bad scan can't kill a loop), breach tasks log exceptions via done-callback, PTB `add_error_handler` registered, and `sys.excepthook`/`threading.excepthook` (installed in `main()`) route uncaught crashes to the log. WS monitor keeps its exponential-backoff reconnect + `logger.exception`.
 
 ---
 
@@ -321,7 +320,7 @@ Data: paginated TD fetches (5000-bar chunks) cached as parquet in `backtest/cach
 
 ## test_all.py — QA suite
 
-`python test_all.py` — **215 tests, 14 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions, chart patterns, app_config, tiingo_provider). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11), config.json loading/precedence/validation coverage (Module 13), and Tiingo quote/spread-monitor + `.env` loader coverage (Module 14). **Any signal-logic change must keep this green and should add tests.**
+`python test_all.py` — **210 tests, 14 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions, chart patterns, app_config, .env loader + bot resilience). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11), config.json loading/precedence/validation coverage (Module 13), and `.env` loader + bot-resilience coverage (Module 14). **Any signal-logic change must keep this green and should add tests.**
 
 ---
 
@@ -335,7 +334,6 @@ Precedence for every credential/deployment setting: **env var → config.json
 | `TELEGRAM_TOKEN` | bot token | none — falls through to config.json `telegram.token`, then refuses to boot |
 | `TELEGRAM_CHAT_ID` | owner chat id (boot auto-scan + owner lock) | none — falls through to config.json `telegram.chat_id`, then refuses to boot |
 | `TELEGRAM_CHANNEL_ID` | subscriber channel (`-100…`); `""` disables the feed | none — falls through to config.json `telegram.channel_id` |
-| `TIINGO_API_KEY` (or `TIINGO`) | Tiingo bid/ask quote key for the spread monitor — optional | none — falls through to config.json `data_providers.tiingo.api_key`; absent = spread monitor off (never fatal) |
 | `FB2_EMA_LO` | EMA200 "near" penalty threshold in % | `0.5` |
 
 Dependencies (`requirements.txt`): numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets. Python 3.12. A `venv/` exists in-repo.

app_config.py

@@ -86,8 +86,7 @@ _VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
 _SCANNABLE_TFS = ("M5", "M15", "H1", "H4")
 
 _KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level"}
-_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality",
+_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
-                   "channel_spread_detail"}
 _KNOWN_SCANNING = {"pairs", "timeframes"}
 _KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
 
@@ -102,11 +101,6 @@ class AppConfig:
     telegram_chat_id: str = ""
     telegram_channel_id: str = ""          # "" = channel feed disabled
     alert_min_quality: str = "C"
-    # True = the channel is INTERNAL (testing) and may receive raw Tiingo
-    # bid/ask numbers. Default False: Tiingo's free tier is licensed
-    # internal-use-only, so a public/subscriber channel must only ever
-    # get the number-free wide-spread caution. Flip back before launch.
-    channel_spread_detail: bool = False
     active_provider: str = "twelve_data"
     provider_settings: Dict[str, Dict[str, Any]] = field(default_factory=dict)
     scan_pairs: List[str] = field(default_factory=lambda: [XAUUSD])
@@ -155,11 +149,6 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
             f'config.json: telegram.alert_min_quality "{quality}" '
             f"not in {_VALID_QUALITIES}")
     app.alert_min_quality = quality
-    spread_detail = tg.get("channel_spread_detail", app.channel_spread_detail)
-    if not isinstance(spread_detail, bool):
-        raise AppConfigError(
-            "config.json: telegram.channel_spread_detail must be a boolean")
-    app.channel_spread_detail = spread_detail
 
     dp = raw.get("data_providers")
     if dp is not None:
@@ -282,20 +271,6 @@ def provider_api_key(app: Optional[AppConfig] = None) -> str:
     return str(block.get("api_key", ""))
 
 
-def tiingo_api_key(app: Optional[AppConfig] = None) -> str:
-    """Key for the SUPPLEMENTARY Tiingo bid/ask quote provider (spread
-    monitor) — never a data_providers.active candidate; Twelve Data
-    stays the OHLCV source. Empty string = feature disabled.
-    Env TIINGO_API_KEY (or short TIINGO, e.g. from .env) →
-    config.json data_providers.tiingo.api_key → ""."""
-    app = app or get_app_config()
-    env = os.environ.get("TIINGO_API_KEY") or os.environ.get("TIINGO")
-    if env:
-        return env
-    block = app.provider_settings.get("tiingo", {})
-    return str(block.get("api_key", ""))
-
-
 # ── PlatformConfig overlay ────────────────────────────────────────────
 
 def apply_pair_overrides(

config.json.example

@@ -3,16 +3,12 @@
     "token": "123456789:REPLACE_WITH_YOUR_BOT_TOKEN",
     "chat_id": "REPLACE_WITH_OWNER_CHAT_ID",
     "channel_id": "-100REPLACE_WITH_CHANNEL_ID_OR_EMPTY_STRING",
-    "alert_min_quality": "C",
+    "alert_min_quality": "C"
-    "channel_spread_detail": false
   },
   "data_providers": {
     "active": "twelve_data",
     "twelve_data": {
       "api_key": "REPLACE_WITH_YOUR_TWELVE_DATA_API_KEY"
-    },
-    "tiingo": {
-      "api_key": ""
     }
   },
   "scanning": {

telegram_bot.py

@@ -67,12 +67,9 @@ from config import (
 )
 from app_config import (
     get_app_config, provider_api_key, telegram_token, telegram_chat_id,
-    telegram_channel_id, tiingo_api_key,
+    telegram_channel_id,
 )
 from data_provider import ForexDataProvider, make_data_provider
-from tiingo_provider import (
-    Quote, TiingoQuoteProvider, WIDE_SPREAD_MULTIPLE, log_quote,
-)
 from pair_signals import PAIR_PRODUCERS
 from signal_aggregator import SignalAggregator, SignalReport
 
@@ -92,9 +89,8 @@ _console_handler = logging.StreamHandler()
 _console_handler.setFormatter(_log_formatter)
 # 3-hour rotation × 240 backups = 30 days retention. Handlers sit on the
 # ROOT logger (force=True guarantees ownership even if something logged
-# first), so every module logger — data_provider, tiingo_provider,
+# first), so every module logger — data_provider, news_engine,
-# news_engine, signal_aggregator, websockets, telegram.ext — lands in
+# signal_aggregator, websockets, telegram.ext — lands in the same file.
-# the same file.
 _file_handler = logging.handlers.TimedRotatingFileHandler(
     os.path.join(_LOG_DIR, "telegram.log"),
     when="h", interval=3, backupCount=240, encoding="utf-8",
@@ -213,32 +209,6 @@ def _get_shared_dp() -> ForexDataProvider:
     return _shared_dp
 
 
-# ── Tiingo bid/ask spread monitor (optional; no key → feature off) ───
-# Warn-only BY DESIGN: there is no historical bid/ask to replay, so a
-# suppressing gate could never be validated (live == backtest rule).
-# Signals never consume bid/ask — this is execution-cost visibility.
-
-_TIINGO_KEY = tiingo_api_key()
-_tiingo: Optional[TiingoQuoteProvider] = (
-    TiingoQuoteProvider(_TIINGO_KEY) if _TIINGO_KEY else None
-)
-_last_quotes: Dict[str, Quote] = {}
-_QUOTE_MAX_AGE = 600          # cached quote stays displayable this long
-_SPREAD_SAMPLE_SECONDS = 300  # sampler cadence (→ logs/spread_log.csv)
-
-# config.json telegram.channel_spread_detail: True while the channel is
-# internal/testing → it gets raw bid/ask like the owner DM. MUST be
-# flipped back to false before the channel goes public (Tiingo free
-# tier is internal-use-only).
-_CHANNEL_SPREAD_DETAIL = _APP.channel_spread_detail
-
-
-def _spread_detail_for(chat_id: int) -> bool:
-    """Owner DMs always get raw quote numbers; the channel only while
-    config.json marks it internal."""
-    return chat_id != CHANNEL_ID or _CHANNEL_SPREAD_DETAIL
-
-
 # ── Signal formatting ─────────────────────────────────────────────────
 
 _SIGNAL_EMOJI = {
@@ -294,37 +264,6 @@ def _live_price_line(symbol: str) -> Optional[str]:
     return f"📡 Live: *{_fmt_price(price)}*  ({age:.0f}s tick)"
 
 
-def _spread_lines(symbol: str, include_quote: bool = True) -> List[str]:
-    """Live bid/ask + wide-spread warning for signal messages.
-    Empty when Tiingo is off or the cached quote is stale.
-
-    include_quote=False is the subscriber-channel variant: Tiingo's
-    free tier is licensed internal-use-only (no redistributing the
-    data), so raw bid/ask/spread numbers stay in the owner DM — the
-    channel gets only a number-free wide-spread caution."""
-    q = _last_quotes.get(symbol)
-    if q is None or q.age_seconds() > _QUOTE_MAX_AGE:
-        return []
-    assumed = PAIR_INFO.get(symbol, {}).get("default_spread", 0.0)
-    wide = assumed > 0 and q.spread >= WIDE_SPREAD_MULTIPLE * assumed
-    if not include_quote:
-        return [
-            "⚠️ Spread unusually wide right now — execution cost is "
-            "elevated, consider waiting for it to normalize"
-        ] if wide else []
-    lines = [
-        f"💱 Bid/Ask: {_fmt_price(q.bid)} / {_fmt_price(q.ask)}"
-        f"  (spread ${q.spread:.2f})"
-    ]
-    if wide:
-        lines.append(
-            f"⚠️ Wide spread: ${q.spread:.2f} ≈ {q.spread / assumed:.1f}× the "
-            f"${assumed:.2f} cost the strategy was validated at — "
-            f"execution is expensive right now"
-        )
-    return lines
-
-
 _DISCLAIMER = (
     "⚠️ _Not financial advice. For educational and research purposes only. "
     "Trading forex/CFDs carries a high risk of loss — do your own analysis "
@@ -332,7 +271,7 @@ _DISCLAIMER = (
 )
 
 
-def format_signal_message(report: SignalReport, spread_detail: bool = True) -> str:
+def format_signal_message(report: SignalReport) -> str:
     sig = report.final_signal
     emoji = _SIGNAL_EMOJI.get(sig, "⬜")
     qe = _QUALITY_EMOJI.get(report.signal_quality, "")
@@ -353,7 +292,6 @@ def format_signal_message(report: SignalReport, spread_detail: bool = True) -> s
     live = _live_price_line(report.symbol)
     if live:
         lines.append(live)
-    lines += _spread_lines(report.symbol, include_quote=spread_detail)
     lines += [
         f"📊 Regime: {regime_str} | Vol: {vol_str}",
         "",
@@ -551,24 +489,6 @@ async def run_signal(
     )
 
 
-async def _refresh_quotes(symbols: List[str]) -> None:
-    """Fetch fresh Tiingo quotes (blocking HTTP) off the event loop —
-    called just before alerts go out so the spread block reflects the
-    moment of entry, and by the 5-min sampler."""
-    if _tiingo is None:
-        return
-    loop = asyncio.get_running_loop()
-    for symbol in set(symbols):
-        try:
-            quote = await loop.run_in_executor(
-                _executor, _tiingo.get_quote, symbol)
-        except Exception as exc:
-            logger.warning(f"Tiingo quote refresh failed for {symbol}: {exc}")
-            continue
-        if quote is not None:
-            _last_quotes[symbol] = quote
-
-
 # ── Alert gating (quality threshold + dedup) ─────────────────────────
 
 _QUALITY_RANK = {"A+": 4, "A": 3, "B": 2, "C": 1}
@@ -697,13 +617,10 @@ async def _scan_timeframe(
         and _passes_quality(r.signal_quality)
         and _should_alert(f"{r.symbol}_{tf.value}", r)
     ]
-    if actionable:
-        await _refresh_quotes([r.symbol for r in actionable])
     sent_ok: Set[int] = set()
     for chat_id in _signal_targets():
         for i, report in enumerate(actionable):
-            detail = format_signal_message(
+            detail = format_signal_message(report)
-                report, spread_detail=_spread_detail_for(chat_id))
             try:
                 await bot.send_message(chat_id, detail, parse_mode=ParseMode.MARKDOWN)
                 sent_ok.add(i)
@@ -801,11 +718,9 @@ async def _handle_breach(bot, ws_symbol: str, pair_symbol: str, price: float, le
         f"⚡ *LEVEL BREAK* — {_escape_md(display)} crossed "
         f"*{_escape_md(label)}* @ `{_fmt_price(price)}`\n\n"
     )
-    await _refresh_quotes([pair_symbol])
     sent = False
     for chat_id in _signal_targets():
-        detail = header + format_signal_message(
+        detail = header + format_signal_message(report)
-            report, spread_detail=_spread_detail_for(chat_id))
         try:
             await bot.send_message(chat_id, detail, parse_mode=ParseMode.MARKDOWN)
             sent = True
@@ -927,37 +842,6 @@ async def _candle_loop(bot, tf: Timeframe, interval_minutes: int) -> None:
         logger.info(f"{label} auto-scan loop stopped")
 
 
-def _weekend_closed(now: datetime) -> bool:
-    """Fri 22:00 → Sun 22:00 UTC — the window the platform treats as
-    market-closed (same as the weekend filter)."""
-    wd, hr = now.weekday(), now.hour
-    return (wd == 4 and hr >= 22) or wd == 5 or (wd == 6 and hr < 22)
-
-
-async def _spread_sampler_loop() -> None:
-    """Every 5 min: fetch bid/ask for each scanned pair, cache it for
-    the alert formatter, and append to logs/spread_log.csv — the
-    dataset that validates (or refutes) the assumed $0.30 round-trip
-    cost every backtest charges."""
-    try:
-        while True:
-            try:
-                if not _weekend_closed(datetime.now(timezone.utc)):
-                    await _refresh_quotes(SCAN_PAIRS)
-                    for symbol in SCAN_PAIRS:
-                        q = _last_quotes.get(symbol)
-                        # Age guard: never log a stale row after a failed fetch.
-                        if q is not None and q.age_seconds() < _SPREAD_SAMPLE_SECONDS:
-                            log_quote(q)
-            except Exception:
-                logger.exception("Spread sampler: cycle failed")
-            await asyncio.sleep(_SPREAD_SAMPLE_SECONDS)
-    except asyncio.CancelledError:
-        pass
-    finally:
-        logger.info("Spread sampler stopped")
-
-
 _TF_INTERVAL_MINUTES = {
     Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.M30: 30,
     Timeframe.H1: 60, Timeframe.H4: 240, Timeframe.D1: 1440,
@@ -968,11 +852,10 @@ _AUTOSCAN_LOOPS = [
 
 
 _ws_task: Optional[asyncio.Task] = None
-_spread_task: Optional[asyncio.Task] = None
 
 
 def _start_scheduler(bot) -> None:
-    global _ws_task, _spread_task
+    global _ws_task
     alive = [t for t in _autoscan_tasks if not t.done()]
     if not alive:
         _autoscan_tasks.clear()
@@ -987,23 +870,15 @@ def _start_scheduler(bot) -> None:
         _ws_task = asyncio.create_task(_ws_monitor(bot))
         logger.info("WebSocket level-breach monitor started")
 
-    if _tiingo is not None and (_spread_task is None or _spread_task.done()):
-        _spread_task = asyncio.create_task(_spread_sampler_loop())
-        logger.info("Tiingo spread sampler started (every %ss)",
-                    _SPREAD_SAMPLE_SECONDS)
-
 
 def _stop_scheduler() -> None:
-    global _ws_task, _spread_task
+    global _ws_task
     for t in _autoscan_tasks:
         t.cancel()
     _autoscan_tasks.clear()
     if _ws_task and not _ws_task.done():
         _ws_task.cancel()
     _ws_task = None
-    if _spread_task and not _spread_task.done():
-        _spread_task.cancel()
-    _spread_task = None
     for t in list(_breach_tasks):
         t.cancel()
     _breach_tasks.clear()
@@ -1101,17 +976,6 @@ def _status_text() -> str:
     else:
         lines.append("❌ WebSocket: *disconnected*")
 
-    if _tiingo is not None:
-        for pair in SCAN_PAIRS:
-            q = _last_quotes.get(pair)
-            display = PAIR_INFO.get(pair, {}).get("display", pair)
-            if q is None:
-                lines.append(f"💱 Spread ({display}): waiting for first quote")
-            else:
-                lines.append(
-                    f"💱 Spread ({display}): `${q.spread:.2f}` "
-                    f"({q.age_seconds():.0f}s ago)")
-
     if _last_prices:
         lines.append("\n*Live Prices (WebSocket):*")
         for ws_sym, pair in _WS_TO_PAIR.items():

test_all.py

@@ -25,7 +25,7 @@ sys.path.insert(0, os.path.dirname(__file__))
 import json
 import app_config
 for _k in ("TELEGRAM_TOKEN", "TELEGRAM_CHAT_ID", "TELEGRAM_CHANNEL_ID",
-           "TWELVE_DATA_API_KEY", "TIINGO_API_KEY", "TIINGO"):
+           "TWELVE_DATA_API_KEY"):
     os.environ.pop(_k, None)
 _TEST_APP = app_config.AppConfig(
     provider_settings={"twelve_data": {"api_key": "test-key"}}
@@ -2751,7 +2751,7 @@ def _write_json(payload) -> str:
 
 _FULL_CFG = {
     "telegram": {"token": "t0k", "chat_id": "111", "channel_id": "-100222",
-                 "alert_min_quality": "B", "channel_spread_detail": True},
+                 "alert_min_quality": "B"},
     "data_providers": {"active": "twelve_data",
                        "twelve_data": {"api_key": "json-key"}},
     "scanning": {"pairs": ["XAU/USD"], "timeframes": ["M15", "H1"]},
@@ -2767,7 +2767,6 @@ def _():
     assert app.telegram_chat_id == "111"
     assert app.telegram_channel_id == "-100222"
     assert app.alert_min_quality == "B"
-    assert app.channel_spread_detail is True
     assert app.active_provider == "twelve_data"
     assert app.provider_settings["twelve_data"]["api_key"] == "json-key"
     assert app.scan_pairs == [XAUUSD]
@@ -2821,7 +2820,6 @@ def _():
         {"data_providers": {"active": "polygon",
                             "twelve_data": {"api_key": "k"}}},    # no block
         {"telegram": {"alert_min_quality": "Z"}},                 # bad quality
-        {"telegram": {"channel_spread_detail": "yes"}},           # not a bool
         {"log_level": "CHATTY"},                                  # bad level
     ]
     for payload in bad_payloads:
@@ -3004,135 +3002,11 @@ def _():
 
 
 # ══════════════════════════════════════════════════════════════════════
-#  MODULE 14: tiingo_provider.py (bid/ask spread monitor)
+#  MODULE 14: .env loader + telegram_bot resilience
 # ══════════════════════════════════════════════════════════════════════
-print("\n═══ Module 14: tiingo_provider.py ═══")
+print("\n═══ Module 14: .env loader + telegram_bot resilience ═══")
 
-from tiingo_provider import (
+from app_config import load_dotenv
-    Quote, TiingoQuoteProvider, WIDE_SPREAD_MULTIPLE, log_quote,
-    tiingo_ticker,
-)
-from app_config import load_dotenv, tiingo_api_key
-
-
-def _tiingo_response(payload):
-    resp = MagicMock()
-    resp.json.return_value = payload
-    resp.raise_for_status.return_value = None
-    return resp
-
-
-_TIINGO_ROW = [{
-    "ticker": "xauusd",
-    "quoteTimestamp": "2026-07-22T12:00:00+00:00",
-    "bidPrice": 2400.10, "bidSize": 1000000.0,
-    "askPrice": 2400.40, "askSize": 1000000.0,
-    "midPrice": 2400.25,
-}]
-
-
-@test("tiingo: ticker mapping XAU/USD → xauusd")
-def _():
-    assert tiingo_ticker("XAU/USD") == "xauusd"
-    assert tiingo_ticker("EUR/USD") == "eurusd"
-
-
-@test("tiingo: quote parsed — spread = ask − bid, canonical symbol")
-def _():
-    provider = TiingoQuoteProvider("k")
-    with patch("tiingo_provider.requests.get",
-               return_value=_tiingo_response(_TIINGO_ROW)) as mock_get:
-        q = provider.get_quote("XAU/USD")
-    assert q is not None
-    assert q.symbol == "XAU/USD"
-    assert abs(q.bid - 2400.10) < 1e-9 and abs(q.ask - 2400.40) < 1e-9
-    assert abs(q.spread - 0.30) < 1e-9
-    assert abs(q.mid - 2400.25) < 1e-9
-    assert q.quote_time is not None and q.quote_time.tzinfo is not None
-    assert q.age_seconds() < 5
-    params = mock_get.call_args.kwargs["params"]
-    assert params["tickers"] == "xauusd" and params["token"] == "k"
-
-
-@test("tiingo: mid falls back to (bid+ask)/2 when midPrice missing")
-def _():
-    row = [dict(_TIINGO_ROW[0], midPrice=None)]
-    provider = TiingoQuoteProvider("k")
-    with patch("tiingo_provider.requests.get",
-               return_value=_tiingo_response(row)):
-        q = provider.get_quote("XAU/USD")
-    assert q is not None and abs(q.mid - 2400.25) < 1e-9
-
-
-@test("tiingo: TTL cache dedupes calls; ttl=0 refetches")
-def _():
-    provider = TiingoQuoteProvider("k")
-    with patch("tiingo_provider.requests.get",
-               return_value=_tiingo_response(_TIINGO_ROW)) as mock_get:
-        provider.get_quote("XAU/USD")
-        provider.get_quote("XAU/USD")
-    assert mock_get.call_count == 1, "second call within TTL must hit cache"
-    provider = TiingoQuoteProvider("k", quote_ttl=0.0)
-    with patch("tiingo_provider.requests.get",
-               return_value=_tiingo_response(_TIINGO_ROW)) as mock_get:
-        provider.get_quote("XAU/USD")
-        provider.get_quote("XAU/USD")
-    assert mock_get.call_count == 2
-
-
-@test("tiingo: every failure path returns None (never raises)")
-def _():
-    provider = TiingoQuoteProvider("k", quote_ttl=0.0)
-    # network error
-    with patch("tiingo_provider.requests.get",
-               side_effect=Exception("boom")):
-        assert provider.get_quote("XAU/USD") is None
-    # HTTP error
-    resp = _tiingo_response(_TIINGO_ROW)
-    resp.raise_for_status.side_effect = Exception("401")
-    with patch("tiingo_provider.requests.get", return_value=resp):
-        assert provider.get_quote("XAU/USD") is None
-    # empty / non-list / null-price payloads
-    for payload in ([], {"detail": "unknown ticker"},
-                    [dict(_TIINGO_ROW[0], bidPrice=None)],
-                    [dict(_TIINGO_ROW[0], askPrice=None)]):
-        with patch("tiingo_provider.requests.get",
-                   return_value=_tiingo_response(payload)):
-            assert provider.get_quote("XAU/USD") is None, \
-                f"payload {payload!r} must yield None"
-
-
-@test("tiingo: log_quote writes header once then appends rows")
-def _():
-    q = Quote(symbol=XAUUSD, bid=2400.10, ask=2400.40, mid=2400.25,
-              spread=0.30, quote_time=None, fetched_at=time.time())
-    with tempfile.TemporaryDirectory() as td:
-        path = os.path.join(td, "sub", "spread_log.csv")
-        log_quote(q, path)
-        log_quote(q, path)
-        with open(path, encoding="utf-8") as fh:
-            lines = fh.read().strip().splitlines()
-    assert lines[0] == "ts_utc,symbol,bid,ask,mid,spread"
-    assert len(lines) == 3, "header + one row per log_quote call"
-    cells = lines[1].split(",")
-    assert cells[1] == XAUUSD and float(cells[5]) == 0.30
-
-
-@test("app_config: tiingo_api_key precedence env → env-short → json → ''")
-def _():
-    app = AppConfig(provider_settings={
-        "twelve_data": {"api_key": "td"},
-        "tiingo": {"api_key": "json-tiingo"},
-    })
-    with patch.dict(os.environ, {}, clear=True):
-        assert tiingo_api_key(app) == "json-tiingo"
-    with patch.dict(os.environ, {"TIINGO": "short-env"}, clear=True):
-        assert tiingo_api_key(app) == "short-env"
-    with patch.dict(os.environ, {"TIINGO": "short-env",
-                                 "TIINGO_API_KEY": "long-env"}, clear=True):
-        assert tiingo_api_key(app) == "long-env"
-    with patch.dict(os.environ, {}, clear=True):
-        assert tiingo_api_key(AppConfig()) == "", "no key anywhere = disabled"
 
 
 @test("app_config: load_dotenv parses lines, never overrides real env")
@@ -3163,90 +3037,6 @@ def _():
     load_dotenv("/nonexistent/.env")   # missing file is a silent no-op
 
 
-@test("telegram_bot: _spread_lines fresh/wide/stale/absent")
-def _():
-    import telegram_bot as tb
-    try:
-        # fresh, normal spread → bid/ask line only
-        tb._last_quotes[XAUUSD] = Quote(
-            symbol=XAUUSD, bid=2400.10, ask=2400.40, mid=2400.25,
-            spread=0.30, quote_time=None, fetched_at=time.time())
-        lines = tb._spread_lines(XAUUSD)
-        assert len(lines) == 1 and "Bid/Ask" in lines[0]
-        assert "spread $0.30" in lines[0]
-        # wide spread (≥ 2× the $0.30 assumption) → warning appended
-        tb._last_quotes[XAUUSD] = Quote(
-            symbol=XAUUSD, bid=2400.10, ask=2401.00, mid=2400.55,
-            spread=0.90, quote_time=None, fetched_at=time.time())
-        lines = tb._spread_lines(XAUUSD)
-        assert len(lines) == 2 and "Wide spread" in lines[1]
-        assert "3.0×" in lines[1]
-        # stale quote → nothing
-        tb._last_quotes[XAUUSD] = Quote(
-            symbol=XAUUSD, bid=2400.10, ask=2400.40, mid=2400.25,
-            spread=0.30, quote_time=None,
-            fetched_at=time.time() - tb._QUOTE_MAX_AGE - 1)
-        assert tb._spread_lines(XAUUSD) == []
-        # no quote at all → nothing
-        tb._last_quotes.pop(XAUUSD, None)
-        assert tb._spread_lines(XAUUSD) == []
-    finally:
-        tb._last_quotes.pop(XAUUSD, None)
-
-
-@test("telegram_bot: channel variant never leaks Tiingo numbers")
-def _():
-    # Tiingo free tier is internal-use-only: the subscriber channel may
-    # see a number-free caution, never bid/ask/spread values.
-    import telegram_bot as tb
-    try:
-        # normal spread → channel variant shows nothing at all
-        tb._last_quotes[XAUUSD] = Quote(
-            symbol=XAUUSD, bid=2400.10, ask=2400.40, mid=2400.25,
-            spread=0.30, quote_time=None, fetched_at=time.time())
-        assert tb._spread_lines(XAUUSD, include_quote=False) == []
-        # wide spread → one caution line with no prices in it
-        tb._last_quotes[XAUUSD] = Quote(
-            symbol=XAUUSD, bid=2400.10, ask=2401.00, mid=2400.55,
-            spread=0.90, quote_time=None, fetched_at=time.time())
-        lines = tb._spread_lines(XAUUSD, include_quote=False)
-        assert len(lines) == 1 and "unusually wide" in lines[0]
-        assert "2400" not in lines[0] and "0.90" not in lines[0] \
-            and "$" not in lines[0], "channel line must carry no data values"
-        # format_signal_message plumbs the flag through
-        config = PlatformConfig()
-        agg = SignalAggregator(config)
-        dfs = {
-            Timeframe.H1: make_ohlcv(300, seed=42),
-            Timeframe.M15: make_ohlcv(300, seed=43),
-            Timeframe.H4: make_ohlcv(300, seed=44),
-        }
-        report = agg.analyze_from_dataframes("XAU/USD", dfs, use_news=False)
-        owner_msg = tb.format_signal_message(report, spread_detail=True)
-        channel_msg = tb.format_signal_message(report, spread_detail=False)
-        assert "Bid/Ask" in owner_msg
-        assert "Bid/Ask" not in channel_msg
-    finally:
-        tb._last_quotes.pop(XAUUSD, None)
-
-
-@test("telegram_bot: _spread_detail_for routes by chat + internal flag")
-def _():
-    import telegram_bot as tb
-    saved_channel, saved_flag = tb.CHANNEL_ID, tb._CHANNEL_SPREAD_DETAIL
-    try:
-        tb.CHANNEL_ID = -100222
-        tb._CHANNEL_SPREAD_DETAIL = False    # default: channel is public
-        assert tb._spread_detail_for(111) is True, "owner DM gets raw quotes"
-        assert tb._spread_detail_for(-100222) is False, \
-            "public channel must get the number-free variant"
-        tb._CHANNEL_SPREAD_DETAIL = True     # internal-testing channel
-        assert tb._spread_detail_for(-100222) is True
-        assert tb._spread_detail_for(111) is True
-    finally:
-        tb.CHANNEL_ID, tb._CHANNEL_SPREAD_DETAIL = saved_channel, saved_flag
-
-
 @test("telegram_bot: log rotation is 3-hourly with 30-day retention")
 def _():
     import logging.handlers
@@ -3282,22 +3072,6 @@ def _():
         tb._autoscan_chats.clear()
 
 
-@test("telegram_bot: no Tiingo key → provider off, message unchanged")
-def _():
-    import telegram_bot as tb
-    assert tb._tiingo is None, "hermetic env has no Tiingo key"
-    config = PlatformConfig()
-    agg = SignalAggregator(config)
-    dfs = {
-        Timeframe.H1: make_ohlcv(300, seed=42),
-        Timeframe.M15: make_ohlcv(300, seed=43),
-        Timeframe.H4: make_ohlcv(300, seed=44),
-    }
-    report = agg.analyze_from_dataframes("XAU/USD", dfs, use_news=False)
-    msg = tb.format_signal_message(report)
-    assert "Bid/Ask" not in msg and "Wide spread" not in msg
-
-
 # ══════════════════════════════════════════════════════════════════════
 #  SUMMARY
 # ══════════════════════════════════════════════════════════════════════

tiingo_provider.py

@@ -1,157 +0,0 @@
-"""
-tiingo_provider.py — Tiingo top-of-book bid/ask quotes
-======================================================
-
-Supplementary LIVE-QUOTE provider, NOT an OHLCV source: Twelve Data
-remains the signal/backtest data provider. Tiingo's forex endpoint
-(GET /tiingo/fx/top) returns real bid/ask, which mid-price OHLCV
-cannot provide. Two uses:
-
-1. Spread visibility on alerts — signal messages show the live
-   bid/ask and warn when the spread is ≥ WIDE_SPREAD_MULTIPLE × the
-   assumed round-trip cost (PAIR_INFO ``default_spread``, $0.30 on
-   XAU) that every backtest charges.
-
-   Warn-only BY DESIGN: no historical bid/ask exists to replay, so a
-   suppressing gate could never be validated (live == backtest rule).
-   This sits in the same safety category as the staleness check.
-
-2. Spread logging (logs/spread_log.csv, 5-min sampler in the bot) —
-   builds the dataset to validate or refute the $0.30 round-trip
-   assumption against reality.
-
-Signals NEVER consume bid/ask — indicators stay on mid OHLCV. Every
-failure path returns None (alerts go out without the spread block
-rather than being delayed or dropped).
-
-Key resolution: env TIINGO_API_KEY (or TIINGO, e.g. from .env) →
-config.json data_providers.tiingo.api_key → "" (feature disabled).
-See app_config.tiingo_api_key.
-"""
-
-import logging
-import os
-import threading
-import time
-from dataclasses import dataclass
-from datetime import datetime, timezone
-from typing import Dict, Optional
-
-import requests
-
-logger = logging.getLogger(__name__)
-
-_TIINGO_FX_TOP_URL = "https://api.tiingo.com/tiingo/fx/top"
-
-# Spread ≥ this multiple of the pair's assumed round-trip cost is
-# flagged as "wide" on alerts (2× the $0.30 XAU assumption = $0.60).
-WIDE_SPREAD_MULTIPLE = 2.0
-
-# A quote younger than this is served from cache — dedupes bursts
-# (alert refresh right after a sampler tick) without hiding real moves.
-QUOTE_TTL_SECONDS = 10.0
-
-SPREAD_LOG_PATH = os.path.join(
-    os.path.dirname(os.path.abspath(__file__)), "logs", "spread_log.csv")
-_CSV_HEADER = "ts_utc,symbol,bid,ask,mid,spread\n"
-
-
-def tiingo_ticker(symbol: str) -> str:
-    """Canonical '"XAU/USD"' form → Tiingo's 'xauusd'."""
-    return symbol.replace("/", "").lower()
-
-
-@dataclass
-class Quote:
-    """One top-of-book snapshot. ``spread`` (ask − bid) IS the
-    round-trip cost a market order pays."""
-    symbol: str                     # canonical form, e.g. "XAU/USD"
-    bid: float
-    ask: float
-    mid: float
-    spread: float
-    quote_time: Optional[datetime]  # venue timestamp (UTC) if provided
-    fetched_at: float               # time.time() at fetch
-
-    def age_seconds(self) -> float:
-        return time.time() - self.fetched_at
-
-
-class TiingoQuoteProvider:
-    """Minimal /tiingo/fx/top client with a short TTL cache.
-
-    Lock held across the fetch (news_engine convention) so concurrent
-    scan threads share one request instead of racing the API.
-    """
-
-    def __init__(self, api_key: str, quote_ttl: float = QUOTE_TTL_SECONDS):
-        self.api_key = api_key
-        self.quote_ttl = quote_ttl
-        self._cache: Dict[str, Quote] = {}
-        self._lock = threading.Lock()
-
-    def get_quote(self, symbol: str) -> Optional[Quote]:
-        with self._lock:
-            cached = self._cache.get(symbol)
-            if cached is not None and cached.age_seconds() < self.quote_ttl:
-                return cached
-            quote = self._fetch(symbol)
-            if quote is not None:
-                self._cache[symbol] = quote
-            return quote
-
-    def _fetch(self, symbol: str) -> Optional[Quote]:
-        try:
-            resp = requests.get(
-                _TIINGO_FX_TOP_URL,
-                params={"tickers": tiingo_ticker(symbol),
-                        "token": self.api_key},
-                timeout=10,
-            )
-            resp.raise_for_status()
-            payload = resp.json()
-            if not isinstance(payload, list) or not payload:
-                logger.warning(
-                    f"Tiingo: unexpected payload for {symbol}: {payload!r:.200}")
-                return None
-            row = payload[0]
-            bid, ask = row.get("bidPrice"), row.get("askPrice")
-            if not isinstance(bid, (int, float)) or not isinstance(ask, (int, float)):
-                logger.warning(f"Tiingo: no bid/ask for {symbol}: {row!r:.200}")
-                return None
-            mid = row.get("midPrice")
-            if not isinstance(mid, (int, float)):
-                mid = (bid + ask) / 2
-            quote_time: Optional[datetime] = None
-            raw_ts = row.get("quoteTimestamp")
-            if isinstance(raw_ts, str):
-                try:
-                    quote_time = datetime.fromisoformat(
-                        raw_ts.replace("Z", "+00:00"))
-                except ValueError:
-                    pass
-            return Quote(
-                symbol=symbol, bid=float(bid), ask=float(ask),
-                mid=float(mid), spread=float(ask) - float(bid),
-                quote_time=quote_time, fetched_at=time.time(),
-            )
-        except Exception as exc:
-            logger.warning(f"Tiingo quote fetch failed for {symbol}: {exc}")
-            return None
-
-
-def log_quote(quote: Quote, path: str = SPREAD_LOG_PATH) -> None:
-    """Append one row to the spread log (creates dir + header on first
-    write). ~15 KB/day at 5-min sampling — no rotation needed."""
-    try:
-        os.makedirs(os.path.dirname(path), exist_ok=True)
-        is_new = not os.path.exists(path)
-        with open(path, "a", encoding="utf-8") as fh:
-            if is_new:
-                fh.write(_CSV_HEADER)
-            ts = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
-            fh.write(
-                f"{ts},{quote.symbol},{quote.bid:.5f},{quote.ask:.5f},"
-                f"{quote.mid:.5f},{quote.spread:.5f}\n")
-    except OSError as exc:
-        logger.warning(f"Spread log write failed: {exc}")