From 3f46505af4a380dd89ac67b8621f7cb21c70ba50 Mon Sep 17 00:00:00 2001 From: Harwinder Sekhon Date: Tue, 21 Jul 2026 13:25:21 -0400 Subject: [PATCH] Docs: config.json deployment overlay (CLAUDE.md repo map, env table, test count) Documents the app_config.py / config.json overlay shipped in Tasks 1-4: repo-map rows for app_config.py and config.json.example, env-var table precedence (env -> config.json -> none, hardcoded fallbacks removed 2026-07-21), PlatformConfig/pair_signals overlay call order, and the real 199-tests/13-modules suite count. Also fixes a stale SCAN_PAIRS comment in telegram_bot.py that still named PlatformConfig.scan_pairs as the deployment switch instead of config.json's scanning.pairs. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b --- CLAUDE.md | 24 ++++++++++++++++-------- telegram_bot.py | 8 +++++--- 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 1c07b76..899e6d8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,6 +13,8 @@ This file is the authoritative developer/agent reference. The human-facing docs | Path | Lines | Purpose | |------|-------|---------| | `config.py` | ~374 | All config dataclasses, `PAIR_INFO` registry, per-pair strategy configs. **`PlatformConfig` defaults ARE the tuned XAU config.** | +| `app_config.py` | ~244 | config.json loader/validation, env-precedence resolvers, `apply_pair_overrides` deployment overlay | +| `config.json.example` | 24 | committed schema template; real gitignored config.json holds secrets + deployment settings | | `data_provider.py` | ~347 | Twelve Data SDK wrapper: fetch, rate-limit (55 credits/min), in-memory cache, validation/cleaning, freshness checks | | `engine.py` | ~1173 | `TechnicalEngine`: computes all indicators on one DataFrame → `TechnicalAnalysis` (composite score, regime, key levels) | | `chart_patterns.py` | ~280 | Chart-pattern detectors for the XAU_CP archetype (pure functions, ATR-unit geometry, first-close-beyond triggers) | @@ -22,7 +24,7 @@ This file is the authoritative developer/agent reference. The human-facing docs | `pair_signals/__init__.py` | 52 | `PAIR_PRODUCERS` registry + `produce_signal()` | | `pair_signals/xauusd.py` | 71 | XAU config factory + per-timeframe overrides (docstring = validation record) | | `main.py` | ~348 | CLI: `python main.py --pair XAUUSD --timeframe H1 --detail/--json` | -| `test_all.py` | ~2100 | QA suite: **184 tests in 12 modules**, no live API needed. Must stay green. | +| `test_all.py` | ~2100 | QA suite: **199 tests in 13 modules**, no live API needed. Must stay green. | | `backtest/gpu_backtest.py` | ~2231 | Bar-replay backtester (CuPy→NumPy fallback) using the real `SignalAggregator` | | `backtest/diagnose_filters.py` | 609 | Filter-diagnosis tooling | | `backtest/research/` | — | Calendar research infra (`ff_calendar_scraper.py`, `calendar_store.py`) + result CSVs | @@ -108,7 +110,9 @@ Legacy fields kept for config-shape compatibility: `rsi_entry_threshold`, `min_e `session_filter=True`, window 03:00–23:00 UTC, `blocked_hours_utc` mirrors XAU's list (non-XAU pairs read this copy), `allowed_weekdays` Mon–Fri+Sun. `regime_filter=False` (optional H4-ADX≥25 gate, off). ### PlatformConfig (top level) -`symbol="XAU/USD"`, primary TF H1, confirmations [M15, H4], `data_lookback_days=60`, `scan_pairs=[XAUUSD]` (**the single switch for what the bot scans**), `scan_timeframes=[M5,M15,H1,H4]`, `candle_buffer_seconds=10`, `ws_cooldown_seconds=300`, hardcoded `twelve_data_api_key` fallback. +`symbol="XAU/USD"`, primary TF H1, confirmations [M15, H4], `data_lookback_days=60`, `scan_pairs=[XAUUSD]` (mirrors config.json `scanning.pairs` — see below), `scan_timeframes=[M5,M15,H1,H4]`, `candle_buffer_seconds=10`, `ws_cooldown_seconds=300`. + +Deployment settings (credentials, scan pairs/TFs, per-pair base blocked hours, alert quality, log level) overlay from config.json via `app_config.apply_pair_overrides`, applied in the pair factory BEFORE per-TF deltas. `scanning.pairs` in config.json is now the actual deployment switch for what the bot scans — `PlatformConfig.scan_pairs` just carries the resolved value. --- @@ -247,6 +251,8 @@ Two entry points: `PAIR_PRODUCERS: Dict[symbol, factory]`; factory signature `make_config(primary_tf, confirmation_tfs=None) -> PlatformConfig`. `produce_signal(symbol, tf, data_provider)` builds config → aggregator → `analyze_pair`. Registered: XAU/USD. +Inside `make_config`, call order is: `PlatformConfig()` defaults → `app_config.apply_pair_overrides(config)` (stamps config.json's non-credential deployment settings onto the config: base `xau_strategy`/`backtest` `blocked_hours_utc`, `scan_pairs`, `scan_timeframes`) → the validated per-TF deltas below. The overlay runs first so e.g. the M5 hour-9 append lands on top of the JSON base, not the other way around. + **A bare `PlatformConfig()` silently runs any pair with XAU defaults — always go through the factory** (the bot, CLI, and backtester all do). Default confirmation map (same in the factory, the bot, and backtest usage): M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1]. (Exception: `main.py`'s CLI map uses M5→[M1,M15] — a CLI-only quirk.) @@ -261,7 +267,7 @@ Default confirmation map (same in the factory, the bot, and backtest usage): M5 ## telegram_bot.py — the bot -Run: `python telegram_bot.py` (env `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`; hardcoded fallbacks exist in the file). +Run: `python telegram_bot.py` (credentials resolve env `TELEGRAM_TOKEN`/`TELEGRAM_CHAT_ID`/`TELEGRAM_CHANNEL_ID` → config.json `telegram.*` → refuses to boot; see Environment variables). - **Owner lock**: every command/button is wrapped in `_owner_only` — only `TELEGRAM_CHAT_ID` gets responses; others get a refusal. The bot is the owner's control panel; the product is the channel. - **Subscriber channel** (`TELEGRAM_CHANNEL_ID`, `-100…`; `""` disables): auto-scan signal alerts + breach alerts also post there (bot needs channel-admin). Monetization = Telegram Stars subscription invite links; Telegram handles membership, the bot has zero payment code. The channel feed keeps running even when the owner toggles DM auto-scan off (`_feed_active()` = DM chats OR channel). @@ -314,18 +320,20 @@ Data: paginated TD fetches (5000-bar chunks) cached as parquet in `backtest/cach ## test_all.py — QA suite -`python test_all.py` — **184 tests, 12 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions, chart patterns). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), and a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11). **Any signal-logic change must keep this green and should add tests.** +`python test_all.py` — **199 tests, 13 modules** (config, data_provider, engine, news_engine, signal_aggregator, main, telegram_bot, integration, pair_signals registry + engine-bug regressions, 2026-07-16 audit-fix regressions, chart patterns, app_config). No network needed (mocked). Uses a custom `@test("name")` decorator that runs immediately (NOT pytest — don't grep for `def test_`). Exit code non-zero on failure. Includes: the xauusd-factory ≡ inline-config equivalence test, engine bug-fix regression tests (Module 9b), a live-vs-GPU engine parity sweep incl. the EMA-ribbon path (Module 11), and config.json loading/precedence/validation coverage (Module 13). **Any signal-logic change must keep this green and should add tests.** --- ## Environment variables +Precedence for every credential/deployment setting: **env var → config.json → none**. Hardcoded fallbacks (previously in `config.py` / `telegram_bot.py`) were **REMOVED 2026-07-21**; the bot now refuses to boot (`telegram_bot.main`) and data fetches fail with a named error (`data_provider`) if a secret resolves to neither source. See `app_config.py` and `config.json.example`. + | Variable | Purpose | Default | |---|---|---| -| `TWELVE_DATA_API_KEY` | market data key | hardcoded fallback in `config.py` | -| `TELEGRAM_TOKEN` | bot token | hardcoded fallback in `telegram_bot.py` | -| `TELEGRAM_CHAT_ID` | owner chat id (boot auto-scan + owner lock) | hardcoded fallback | -| `TELEGRAM_CHANNEL_ID` | subscriber channel (`-100…`); `""` disables the feed | hardcoded fallback | +| `TWELVE_DATA_API_KEY` | market data key | none — falls through to config.json `data_providers.twelve_data.api_key`, then fails fast | +| `TELEGRAM_TOKEN` | bot token | none — falls through to config.json `telegram.token`, then refuses to boot | +| `TELEGRAM_CHAT_ID` | owner chat id (boot auto-scan + owner lock) | none — falls through to config.json `telegram.chat_id`, then refuses to boot | +| `TELEGRAM_CHANNEL_ID` | subscriber channel (`-100…`); `""` disables the feed | none — falls through to config.json `telegram.channel_id` | | `FB2_EMA_LO` | EMA200 "near" penalty threshold in % | `0.5` | Dependencies (`requirements.txt`): numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets. Python 3.12. A `venv/` exists in-repo. diff --git a/telegram_bot.py b/telegram_bot.py index 1b23633..1962a82 100755 --- a/telegram_bot.py +++ b/telegram_bot.py @@ -113,9 +113,11 @@ _CHANNEL_ID_RAW = telegram_channel_id() CHANNEL_ID: Optional[int] = int(_CHANNEL_ID_RAW) if _CHANNEL_ID_RAW else None # ── Pairs & buttons ─────────────────────────────────────────────────── -# The scanned pair list is config-driven: PlatformConfig.scan_pairs is -# the single switch for single- vs multi-pair operation. Buttons, WS -# subscriptions, and scan loops all derive from it. +# The scanned pair list is config-driven: `scanning.pairs` in config.json +# (read via app_config) is the single switch for single- vs multi-pair +# operation — `apply_pair_overrides` stamps it onto PlatformConfig.scan_pairs +# so the two stay in sync. Buttons, WS subscriptions, and scan loops all +# derive from it. SCAN_PAIRS = list(_APP.scan_pairs)