# Centralized config.json Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Move deployment settings (Telegram credentials, data-provider API keys/selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert quality boot default, log level) into a gitignored `config.json` loaded once at startup, and delete the hardcoded credentials from source.

**Architecture:** A new `app_config.py` module loads and validates `config.json` into an `AppConfig` dataclass (cached singleton), with resolver functions applying the precedence **env var → config.json → code default**. The pair factory calls `apply_pair_overrides()` on every fresh `PlatformConfig` *before* its validated per-timeframe deltas, so the bot, CLI, and backtester all pick up the overlay identically (live == backtest parity by construction). Spec: `docs/superpowers/specs/2026-07-21-centralized-config-design.md`.

**Tech Stack:** Python 3.12 stdlib only (`json`, `os`, `dataclasses`). No new dependencies.

## Global Constraints

- **Zero signal-logic change.** No indicator weight, threshold, gate, or geometry value may change. With no `config.json`, or one holding the example defaults, every produced `PlatformConfig` must be byte-identical (`dataclasses.asdict` equality) to today's factory output — pinned by a test.
- The test suite is `python test_all.py` — a custom harness (`@test("name")` decorator, tests execute at import time, exit code non-zero on failure). It is NOT pytest; there is no single-test selection. "Run the test" always means `python test_all.py` and checking the named line in the output. Baseline before this work: **184 passed, 0 failed**.
- The suite must be hermetic: it must pass identically whether or not the developer's machine has a `config.json`, and regardless of `TELEGRAM_*`/`TWELVE_DATA_API_KEY` env vars.
- Secrets get **no code default**. After Task 4 the only places credentials exist are `config.json` (gitignored) and env vars.
- `config.json.example` (already committed at repo root) is the schema reference. `config.json` is already in `.gitignore` — never commit it.
- Repo root: `/home/harwinder/Development/ForexBot2`. Use `venv/bin/python` if bare `python` lacks pandas/numpy.
- Every commit message ends with:
  ```
  Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
  Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
  ```
- Work on branch `centralized-config` (create from `main` at start of Task 1: `git checkout -b centralized-config`).

---

### Task 1: `app_config.py` — loader, validation, resolvers

**Files:**
- Create: `app_config.py`
- Modify: `test_all.py` (new Module 13 before the SUMMARY block, ~line 2495; hermetic header after line 19)
- Test: `test_all.py` (same file — this repo keeps all tests in one suite)

**Interfaces:**
- Consumes: `config.PAIR_INFO`, `config.Timeframe`, `config.XAUUSD`, `config.PlatformConfig` (all existing).
- Produces (later tasks rely on these exact names):
  - `class AppConfigError(Exception)`
  - `@dataclass class AppConfig` — fields: `telegram_token: str = ""`, `telegram_chat_id: str = ""`, `telegram_channel_id: str = ""`, `alert_min_quality: str = "C"`, `active_provider: str = "twelve_data"`, `provider_settings: Dict[str, Dict[str, Any]] = {}`, `scan_pairs: List[str] = [XAUUSD]`, `scan_timeframes: List[Timeframe] = [M5, M15, H1, H4]`, `pair_overrides: Dict[str, Dict[str, Any]] = {}`, `log_level: str = "INFO"`
  - `load_app_config(path: str = CONFIG_PATH) -> AppConfig`
  - `get_app_config() -> AppConfig` (cached), `set_app_config(app: Optional[AppConfig]) -> None` (test hook; `None` re-enables lazy load)
  - `telegram_token(app=None) -> str`, `telegram_chat_id(app=None) -> str`, `telegram_channel_id(app=None) -> str`, `provider_api_key(app=None) -> str`
  - `apply_pair_overrides(cfg: PlatformConfig, app: Optional[AppConfig] = None) -> PlatformConfig` (implemented here, wired in Task 2)

- [ ] **Step 1: Create branch**

```bash
cd /home/harwinder/Development/ForexBot2 && git checkout -b centralized-config
```

- [ ] **Step 2: Add the hermetic test header and Module 13 failing tests to `test_all.py`**

In `test_all.py`, directly after `sys.path.insert(0, os.path.dirname(__file__))` (line 19), add:

```python
# ── Hermetic app-config: the suite must not depend on the developer's
#    local config.json or TELEGRAM_*/TWELVE_DATA_API_KEY env vars. The
#    dummy TD key mirrors the pre-config.json behavior where a key was
#    always present (TDClient built, no network at construction).
import json
import app_config
for _k in ("TELEGRAM_TOKEN", "TELEGRAM_CHAT_ID", "TELEGRAM_CHANNEL_ID",
           "TWELVE_DATA_API_KEY"):
    os.environ.pop(_k, None)
_TEST_APP = app_config.AppConfig(
    provider_settings={"twelve_data": {"api_key": "test-key"}}
)
app_config.set_app_config(_TEST_APP)
```

Then insert Module 13 immediately BEFORE the `#  SUMMARY` comment block near the end of the file (search for `SUMMARY`):

```python
# ══════════════════════════════════════════════════════════════════════
#  MODULE 13: app_config.py (config.json overlay)
# ══════════════════════════════════════════════════════════════════════
print("\n═══ Module 13: app_config.py ═══")

import tempfile
from app_config import (
    AppConfig, AppConfigError, load_app_config, set_app_config,
    telegram_token, telegram_chat_id, telegram_channel_id,
    provider_api_key, apply_pair_overrides,
)


def _write_json(payload) -> str:
    fh = tempfile.NamedTemporaryFile(
        "w", suffix=".json", delete=False, encoding="utf-8")
    if isinstance(payload, str):
        fh.write(payload)
    else:
        json.dump(payload, fh)
    fh.close()
    return fh.name


_FULL_CFG = {
    "telegram": {"token": "t0k", "chat_id": "111", "channel_id": "-100222",
                 "alert_min_quality": "B"},
    "data_providers": {"active": "twelve_data",
                       "twelve_data": {"api_key": "json-key"}},
    "scanning": {"pairs": ["XAU/USD"], "timeframes": ["M15", "H1"]},
    "pairs": {"XAU/USD": {"blocked_hours_utc": [3, 4]}},
    "log_level": "WARNING",
}


@test("app_config: valid config.json maps every field")
def _():
    app = load_app_config(_write_json(_FULL_CFG))
    assert app.telegram_token == "t0k"
    assert app.telegram_chat_id == "111"
    assert app.telegram_channel_id == "-100222"
    assert app.alert_min_quality == "B"
    assert app.active_provider == "twelve_data"
    assert app.provider_settings["twelve_data"]["api_key"] == "json-key"
    assert app.scan_pairs == [XAUUSD]
    assert app.scan_timeframes == [Timeframe.M15, Timeframe.H1]
    assert app.pair_overrides[XAUUSD]["blocked_hours_utc"] == [3, 4]
    assert app.log_level == "WARNING"


@test("app_config: missing file returns pure defaults")
def _():
    app = load_app_config("/nonexistent/config.json")
    assert app == AppConfig(), "absent config.json must yield pure defaults"


@test("app_config: malformed JSON raises AppConfigError")
def _():
    path = _write_json("{not json")
    try:
        load_app_config(path)
        assert False, "should have raised"
    except AppConfigError as e:
        assert "not valid JSON" in str(e)


@test("app_config: unknown keys raise AppConfigError")
def _():
    for payload in (
        {"telgram": {}},                                   # top-level typo
        {"telegram": {"tokn": "x"}},                       # section typo
        {"pairs": {"XAU/USD": {"blocked_hours": [1]}}},    # pair-block typo
        {"scanning": {"pair": ["XAU/USD"]}},               # scanning typo
    ):
        try:
            load_app_config(_write_json(payload))
            assert False, f"should have raised for {payload}"
        except AppConfigError as e:
            assert "unknown key" in str(e).lower()


@test("app_config: invalid values raise AppConfigError")
def _():
    bad_payloads = [
        {"pairs": {"XAU/USD": {"blocked_hours_utc": [24]}}},      # hour range
        {"pairs": {"XAU/USD": {"blocked_hours_utc": "3"}}},       # not a list
        {"pairs": {"EUR/USD": {"blocked_hours_utc": [3]}}},       # unknown pair
        {"scanning": {"pairs": ["EUR/USD"]}},                     # unknown pair
        {"scanning": {"timeframes": ["M7"]}},                     # bad TF name
        {"scanning": {"timeframes": ["W1"]}},                     # unscannable TF
        {"data_providers": {"active": "polygon",
                            "twelve_data": {"api_key": "k"}}},    # no block
        {"telegram": {"alert_min_quality": "Z"}},                 # bad quality
        {"log_level": "CHATTY"},                                  # bad level
    ]
    for payload in bad_payloads:
        try:
            load_app_config(_write_json(payload))
            assert False, f"should have raised for {payload}"
        except AppConfigError:
            pass


@test("app_config: env precedence env → config.json → default")
def _():
    app = load_app_config(_write_json(_FULL_CFG))
    with patch.dict(os.environ, {}, clear=True):
        assert telegram_token(app) == "t0k"
        assert telegram_chat_id(app) == "111"
        assert telegram_channel_id(app) == "-100222"
        assert provider_api_key(app) == "json-key"
    with patch.dict(os.environ, {"TELEGRAM_TOKEN": "env-tok",
                                 "TELEGRAM_CHAT_ID": "999",
                                 "TWELVE_DATA_API_KEY": "env-key"}):
        assert telegram_token(app) == "env-tok"
        assert telegram_chat_id(app) == "999"
        assert provider_api_key(app) == "env-key"
    # channel: env set-but-empty means "disable feed" and must beat JSON
    with patch.dict(os.environ, {"TELEGRAM_CHANNEL_ID": ""}):
        assert telegram_channel_id(app) == ""
```

- [ ] **Step 3: Run to verify the new tests fail (import error)**

```bash
python test_all.py 2>&1 | head -5
```

Expected: `ModuleNotFoundError: No module named 'app_config'` (the hermetic header imports it before any test runs).

- [ ] **Step 4: Write `app_config.py`**

Create `app_config.py` with exactly this content:

```python
"""
app_config.py — deployment-settings overlay (config.json)
=========================================================

Loads the gitignored config.json at the repo root ONCE per process.
Deployment surface ONLY: Telegram credentials, data-provider selection
and API keys, scanned pairs/timeframes, per-pair base blocked hours,
alert-quality boot default, log level. Signal logic, weights,
thresholds and gates stay in config.py / pair_signals.

Precedence for every setting: env var → config.json → code default.
Secrets have NO code default — with neither env nor config.json the
bot refuses to start (telegram_bot.main) and data fetches fail with a
named error (data_provider).

Parity contract (test-enforced): with no config.json, or one holding
the config.json.example defaults, every produced PlatformConfig is
byte-identical to the pure-code defaults. apply_pair_overrides() runs
in the pair factory BEFORE the validated per-timeframe deltas, so the
bot, CLI and backtester all see the same overlay (live == backtest).

Spec: docs/superpowers/specs/2026-07-21-centralized-config-design.md
"""

import json
import logging
import os
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional

from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD

logger = logging.getLogger(__name__)

CONFIG_PATH = os.path.join(
    os.path.dirname(os.path.abspath(__file__)), "config.json")

_VALID_QUALITIES = ("A+", "A", "B", "C")
_VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
# Timeframes the bot's candle-boundary scheduler can run (W1 has no loop).
_SCANNABLE_TFS = ("M5", "M15", "M30", "H1", "H4", "D1")

_KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level"}
_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
_KNOWN_SCANNING = {"pairs", "timeframes"}
_KNOWN_PAIR_KEYS = {"blocked_hours_utc"}


class AppConfigError(Exception):
    """config.json is present but invalid — fail fast at boot."""


@dataclass
class AppConfig:
    telegram_token: str = ""
    telegram_chat_id: str = ""
    telegram_channel_id: str = ""          # "" = channel feed disabled
    alert_min_quality: str = "C"
    active_provider: str = "twelve_data"
    provider_settings: Dict[str, Dict[str, Any]] = field(default_factory=dict)
    scan_pairs: List[str] = field(default_factory=lambda: [XAUUSD])
    scan_timeframes: List[Timeframe] = field(default_factory=lambda: [
        Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4])
    pair_overrides: Dict[str, Dict[str, Any]] = field(default_factory=dict)
    log_level: str = "INFO"


def _check_keys(section: Dict[str, Any], known: set, where: str) -> None:
    unknown = sorted(set(section) - known)
    if unknown:
        raise AppConfigError(
            f"config.json: unknown key(s) {unknown} in {where}; "
            f"known keys: {sorted(known)}")


def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
    """Parse + validate config.json. Missing file → pure defaults."""
    app = AppConfig()
    if not os.path.exists(path):
        logger.warning(
            "config.json not found at %s — running on code defaults "
            "(secrets must come from environment variables)", path)
        return app
    try:
        with open(path, encoding="utf-8") as fh:
            raw = json.load(fh)
    except json.JSONDecodeError as e:
        raise AppConfigError(f"config.json is not valid JSON: {e}") from e
    if not isinstance(raw, dict):
        raise AppConfigError("config.json top level must be a JSON object")
    _check_keys(raw, _KNOWN_TOP, "top level")

    tg = raw.get("telegram", {})
    _check_keys(tg, _KNOWN_TELEGRAM, '"telegram"')
    app.telegram_token = str(tg.get("token", app.telegram_token))
    app.telegram_chat_id = str(tg.get("chat_id", app.telegram_chat_id))
    app.telegram_channel_id = str(
        tg.get("channel_id", app.telegram_channel_id))
    quality = tg.get("alert_min_quality", app.alert_min_quality)
    if quality not in _VALID_QUALITIES:
        raise AppConfigError(
            f'config.json: telegram.alert_min_quality "{quality}" '
            f"not in {_VALID_QUALITIES}")
    app.alert_min_quality = quality

    dp = raw.get("data_providers")
    if dp is not None:
        if not isinstance(dp, dict):
            raise AppConfigError('config.json: "data_providers" must be an object')
        app.active_provider = str(dp.get("active", app.active_provider))
        app.provider_settings = {}
        for name, block in dp.items():
            if name == "active":
                continue
            if not isinstance(block, dict):
                raise AppConfigError(
                    f'config.json: data_providers.{name} must be an object')
            app.provider_settings[name] = dict(block)
        if app.active_provider not in app.provider_settings:
            raise AppConfigError(
                f'config.json: data_providers.active "{app.active_provider}" '
                "has no matching provider block")

    scanning = raw.get("scanning", {})
    _check_keys(scanning, _KNOWN_SCANNING, '"scanning"')
    if "pairs" in scanning:
        pairs = scanning["pairs"]
        bad = [p for p in pairs if p not in PAIR_INFO]
        if bad:
            raise AppConfigError(
                f"config.json: unknown pair(s) {bad} in scanning.pairs; "
                f"known: {sorted(PAIR_INFO)}")
        app.scan_pairs = list(pairs)
    if "timeframes" in scanning:
        names = scanning["timeframes"]
        bad = [n for n in names if n not in _SCANNABLE_TFS]
        if bad:
            raise AppConfigError(
                f"config.json: unsupported timeframe(s) {bad} in "
                f"scanning.timeframes; supported: {list(_SCANNABLE_TFS)}")
        app.scan_timeframes = [Timeframe[n] for n in names]

    pair_section = raw.get("pairs", {})
    for symbol, overrides in pair_section.items():
        if symbol not in PAIR_INFO:
            raise AppConfigError(
                f'config.json: unknown pair "{symbol}" in pairs; '
                f"known: {sorted(PAIR_INFO)}")
        _check_keys(overrides, _KNOWN_PAIR_KEYS, f'pairs."{symbol}"')
        if "blocked_hours_utc" in overrides:
            hours = overrides["blocked_hours_utc"]
            valid = isinstance(hours, list) and all(
                isinstance(h, int) and not isinstance(h, bool)
                and 0 <= h <= 23 for h in hours)
            if not valid:
                raise AppConfigError(
                    f'config.json: pairs."{symbol}".blocked_hours_utc must '
                    "be a list of integers in 0-23")
    app.pair_overrides = {s: dict(o) for s, o in pair_section.items()}

    level = raw.get("log_level", app.log_level)
    if level not in _VALID_LOG_LEVELS:
        raise AppConfigError(
            f'config.json: log_level "{level}" not in {_VALID_LOG_LEVELS}')
    app.log_level = level
    return app


# ── Process-wide singleton ────────────────────────────────────────────

_cached: Optional[AppConfig] = None


def get_app_config() -> AppConfig:
    global _cached
    if _cached is None:
        _cached = load_app_config()
    return _cached


def set_app_config(app: Optional[AppConfig]) -> None:
    """Override the process-wide AppConfig (tests/experiments).
    Pass None to re-enable lazy loading from CONFIG_PATH."""
    global _cached
    _cached = app


# ── Resolution helpers: env var → config.json → default ───────────────

def telegram_token(app: Optional[AppConfig] = None) -> str:
    app = app or get_app_config()
    return os.environ.get("TELEGRAM_TOKEN") or app.telegram_token


def telegram_chat_id(app: Optional[AppConfig] = None) -> str:
    app = app or get_app_config()
    return os.environ.get("TELEGRAM_CHAT_ID") or app.telegram_chat_id


def telegram_channel_id(app: Optional[AppConfig] = None) -> str:
    """Env set-but-empty ("") means "disable the feed" and beats JSON,
    so this checks None-ness, not truthiness."""
    app = app or get_app_config()
    env = os.environ.get("TELEGRAM_CHANNEL_ID")
    return env if env is not None else app.telegram_channel_id


def provider_api_key(app: Optional[AppConfig] = None) -> str:
    """API key for the ACTIVE provider. The env override is per-provider
    (TWELVE_DATA_API_KEY applies only when twelve_data is active)."""
    app = app or get_app_config()
    if app.active_provider == "twelve_data":
        env = os.environ.get("TWELVE_DATA_API_KEY")
        if env:
            return env
    block = app.provider_settings.get(app.active_provider, {})
    return str(block.get("api_key", ""))


# ── PlatformConfig overlay ────────────────────────────────────────────

def apply_pair_overrides(
    cfg: PlatformConfig, app: Optional[AppConfig] = None
) -> PlatformConfig:
    """Stamp deployment settings from config.json onto a PlatformConfig.

    Called by the pair factories BEFORE their validated per-timeframe
    deltas, so e.g. the M5 hour-9 append lands on top of the JSON base.
    The pair's base blocked_hours_utc is set on BOTH mirrored copies
    (xau_strategy + backtest) so they cannot drift.
    """
    app = app or get_app_config()
    overrides = app.pair_overrides.get(cfg.symbol, {})
    if "blocked_hours_utc" in overrides:
        hours = list(overrides["blocked_hours_utc"])
        cfg.xau_strategy.blocked_hours_utc = hours
        cfg.backtest.blocked_hours_utc = list(hours)
    cfg.scan_pairs = list(app.scan_pairs)
    cfg.scan_timeframes = list(app.scan_timeframes)
    return cfg
```

- [ ] **Step 5: Run tests, verify Module 13 passes and total is 190**

```bash
python test_all.py 2>&1 | tail -25
```

Expected: `RESULTS: 190 passed, 0 failed, 190 total` (184 baseline + 6 new). All six `app_config:` lines show `✓`.

- [ ] **Step 6: Commit**

```bash
git add app_config.py test_all.py
git commit -m "Add app_config.py: config.json loader, validation, env-precedence resolvers"
```

---

### Task 2: Factory overlay wiring + parity pins

**Files:**
- Modify: `pair_signals/xauusd.py:81-90` (make_config), `test_all.py` (Module 9 equivalence test ~line 1851; Module 13 additions)

**Interfaces:**
- Consumes: `app_config.apply_pair_overrides`, `app_config.set_app_config`, `app_config.AppConfig`, `app_config.load_app_config` (Task 1).
- Produces: `make_config` now applies the overlay — every consumer (bot, CLI, backtester) inherits it with no further change. Test symbol `_TEST_APP` (Task 1) is the restore target for any test that swaps the singleton.

- [ ] **Step 1: Add failing tests (Module 13 additions)**

Append to Module 13 in `test_all.py`:

```python
@test("app_config: apply_pair_overrides stamps both blocked-hours mirrors")
def _():
    app = AppConfig(pair_overrides={XAUUSD: {"blocked_hours_utc": [3, 4]}},
                    scan_pairs=[XAUUSD],
                    scan_timeframes=[Timeframe.M15])
    cfg = apply_pair_overrides(PlatformConfig(), app)
    assert cfg.xau_strategy.blocked_hours_utc == [3, 4]
    assert cfg.backtest.blocked_hours_utc == [3, 4]
    assert cfg.scan_timeframes == [Timeframe.M15]
    # the two mirrors must be independent lists, not aliases
    cfg.xau_strategy.blocked_hours_utc.append(5)
    assert cfg.backtest.blocked_hours_utc == [3, 4]


@test("app_config: M5 factory appends hour 9 on top of a JSON base")
def _():
    from pair_signals import PAIR_PRODUCERS
    app = AppConfig(pair_overrides={XAUUSD: {"blocked_hours_utc": [3, 4]}},
                    provider_settings={"twelve_data": {"api_key": "test-key"}})
    set_app_config(app)
    try:
        cfg = PAIR_PRODUCERS[XAUUSD](Timeframe.M5, None)
    finally:
        set_app_config(_TEST_APP)
    assert cfg.xau_strategy.blocked_hours_utc == [3, 4, 9], \
        "validated M5 hour-9 delta must land on top of the JSON base"
    assert cfg.backtest.blocked_hours_utc == [3, 4], \
        "mirror carries the base only (per-TF delta is XAU-strategy-side)"


@test("app_config: config.json.example is valid and behavior-neutral")
def _():
    import dataclasses
    from pair_signals import PAIR_PRODUCERS
    example = load_app_config(
        os.path.join(os.path.dirname(__file__), "config.json.example"))
    for tf in (Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4):
        baseline = PAIR_PRODUCERS[XAUUSD](tf, None)   # under _TEST_APP
        set_app_config(example)
        try:
            with_example = PAIR_PRODUCERS[XAUUSD](tf, None)
        finally:
            set_app_config(_TEST_APP)
        assert dataclasses.asdict(baseline) == dataclasses.asdict(with_example), \
            f"example config.json must be byte-identical to defaults on {tf}"
```

- [ ] **Step 2: Run to verify the new tests fail**

```bash
python test_all.py 2>&1 | grep "app_config: M5 factory"
```

Expected: `✗ app_config: M5 factory appends hour 9 on top of a JSON base` (the factory doesn't apply the overlay yet, so blocked hours are the defaults `[0,1,2,10,11,13,16,21,9]`, not `[3,4,9]`).

- [ ] **Step 3: Wire the overlay into the factory**

In `pair_signals/xauusd.py`, change the import line and the top of `make_config`:

```python
from typing import Optional, List
from config import PlatformConfig, Timeframe
from app_config import apply_pair_overrides
```

and in `make_config`, immediately after `config = PlatformConfig()`:

```python
    config = PlatformConfig()
    # Deployment overlay (config.json) BEFORE the validated per-TF
    # deltas below, so e.g. the M5 hour-9 append lands on the JSON base.
    apply_pair_overrides(config)
    config.primary_timeframe = primary_tf
```

- [ ] **Step 4: Update the Module 9 equivalence test**

In `test_all.py` (~line 1851, `"xauusd.make_config equals inline PlatformConfig construction"`), the inline construction must apply the same overlay in the same order. After the `inline_cfg = PlatformConfig(...)` construction and BEFORE the per-TF override lines, add:

```python
        app_config.apply_pair_overrides(inline_cfg)
```

(`app_config` is already imported at the top of `test_all.py` from Task 1. Under `_TEST_APP` this is a no-op on blocked hours, but it keeps the equivalence definition exact: factory ≡ inline + overlay + per-TF deltas, whatever the active AppConfig.)

- [ ] **Step 5: Run tests, verify 193 passing**

```bash
python test_all.py 2>&1 | tail -25
```

Expected: `RESULTS: 193 passed, 0 failed, 193 total`. Watch specifically for `✓ xauusd.make_config equals inline PlatformConfig construction` (Module 9) and the three new Module 13 lines.

- [ ] **Step 6: Commit**

```bash
git add pair_signals/xauusd.py test_all.py
git commit -m "Wire config.json overlay into the pair factory (before per-TF deltas)"
```

---

### Task 3: Data provider — key from app_config, registry, delete hardcoded key

**Files:**
- Modify: `config.py:346` (delete field), `data_provider.py:72-76` (key resolution), `data_provider.py:265-269` (error message), `data_provider.py` end (registry), `signal_aggregator.py:107`, `backtest/gpu_backtest.py:1682`, `test_all.py` (Module 13 additions)

**Interfaces:**
- Consumes: `app_config.provider_api_key`, `app_config.get_app_config`, `app_config.AppConfigError` (Task 1).
- Produces: `data_provider.PROVIDER_REGISTRY: Dict[str, type]` and `data_provider.make_data_provider(config: PlatformConfig) -> ForexDataProvider` — Task 4 uses `make_data_provider` in `telegram_bot.py`.

- [ ] **Step 1: Add failing tests (Module 13 additions)**

```python
@test("data_provider: api key resolves env → config.json (no code default)")
def _():
    from data_provider import ForexDataProvider
    app = AppConfig(provider_settings={"twelve_data": {"api_key": "json-key"}})
    set_app_config(app)
    try:
        with patch.dict(os.environ, {}, clear=True):
            dp = ForexDataProvider(PlatformConfig())
            assert dp._api_key == "json-key"
        with patch.dict(os.environ, {"TWELVE_DATA_API_KEY": "env-key"}):
            dp = ForexDataProvider(PlatformConfig())
            assert dp._api_key == "env-key"
        set_app_config(AppConfig())        # no key anywhere
        with patch.dict(os.environ, {}, clear=True):
            dp = ForexDataProvider(PlatformConfig())
            assert dp._api_key is None and dp._td is None
    finally:
        set_app_config(_TEST_APP)


@test("data_provider: make_data_provider dispatches on active provider")
def _():
    from data_provider import ForexDataProvider, make_data_provider
    assert isinstance(make_data_provider(PlatformConfig()), ForexDataProvider)
    set_app_config(AppConfig(active_provider="polygon",
                             provider_settings={"polygon": {"api_key": "k"}}))
    try:
        make_data_provider(PlatformConfig())
        assert False, "unregistered provider must raise"
    except AppConfigError as e:
        assert "polygon" in str(e)
    finally:
        set_app_config(_TEST_APP)


@test("config.py: hardcoded twelve_data_api_key field is gone")
def _():
    assert not hasattr(PlatformConfig(), "twelve_data_api_key"), \
        "credentials must not live in committed source"
```

- [ ] **Step 2: Run to verify they fail**

```bash
python test_all.py 2>&1 | grep -A1 "hardcoded twelve_data"
```

Expected: `✗ config.py: hardcoded twelve_data_api_key field is gone` (plus the make_data_provider import failure).

- [ ] **Step 3: Implement**

`config.py`: delete line 346 (`twelve_data_api_key: str = "f563b22ffdfd4924955decbe1a8c8701"`).

`data_provider.py` — add to the local imports:

```python
from app_config import AppConfigError, get_app_config, provider_api_key
```

In `ForexDataProvider.__init__` replace the key resolution:

```python
        self.config = config
        self._api_key = provider_api_key() or None
```

In `_fetch_twelve_data`, update the no-key error:

```python
        if self._td is None:
            raise DataIntegrityError(
                "No data-provider API key. Set the TWELVE_DATA_API_KEY env "
                "var or data_providers.twelve_data.api_key in config.json."
            )
```

At the bottom of `data_provider.py` (after the class):

```python
# ── Provider registry ─────────────────────────────────────────────────
# config.json data_providers.active selects from here. Adding a provider
# later = one adapter class + one entry + one config block.
PROVIDER_REGISTRY: Dict[str, type] = {"twelve_data": ForexDataProvider}


def make_data_provider(config: PlatformConfig) -> ForexDataProvider:
    """Build the ACTIVE data provider (config.json: data_providers.active)."""
    name = get_app_config().active_provider
    cls = PROVIDER_REGISTRY.get(name)
    if cls is None:
        raise AppConfigError(
            f'Unknown data provider "{name}"; '
            f"registered: {sorted(PROVIDER_REGISTRY)}")
    return cls(config)
```

`signal_aggregator.py` line 18 + 107 — import and use the registry:

```python
from data_provider import ForexDataProvider, make_data_provider
```
```python
        self.data_provider = make_data_provider(config)
```

`backtest/gpu_backtest.py` line 1682 (and its import of `ForexDataProvider`):

```python
    provider = make_data_provider(platform_cfg)
```

- [ ] **Step 4: Run tests, verify 196 passing**

```bash
python test_all.py 2>&1 | tail -25
```

Expected: `RESULTS: 196 passed, 0 failed, 196 total`. If any Module 2 test fails on a missing key, it means it constructed `ForexDataProvider` outside `_TEST_APP` scope — fix the test to run under `_TEST_APP` (which supplies the dummy key), never by re-adding a code default.

- [ ] **Step 5: Commit**

```bash
git add config.py data_provider.py signal_aggregator.py backtest/gpu_backtest.py test_all.py
git commit -m "Data provider: key from config.json/env, provider registry, drop hardcoded key"
```

---

### Task 4: telegram_bot wiring + startup guard

**Files:**
- Modify: `telegram_bot.py` (credentials block ~93-105, logging ~88, SCAN_PAIRS ~113, alert quality ~478, WS key ~611-614, `_AUTOSCAN_LOOPS` ~797-802, scheduler log ~817, `_get_shared_dp` ~195, `main()` ~1261), `test_all.py` (Module 13 addition)

**Interfaces:**
- Consumes: `app_config.get_app_config`, resolvers, `data_provider.make_data_provider` (Tasks 1+3).
- Produces: nothing consumed later; this is the final wiring task.

- [ ] **Step 1: Add failing test (Module 13 addition)**

```python
@test("telegram_bot: credentials, quality, pairs, loops from app_config")
def _():
    import importlib
    import telegram_bot
    app = AppConfig(telegram_token="json-tok", telegram_chat_id="42",
                    telegram_channel_id="", alert_min_quality="B",
                    scan_timeframes=[Timeframe.M15, Timeframe.H1],
                    provider_settings={"twelve_data": {"api_key": "k"}})
    set_app_config(app)
    try:
        with patch.dict(os.environ, {}, clear=True):
            tb = importlib.reload(telegram_bot)
            assert tb.TOKEN == "json-tok"
            assert tb.OWNER_ID == 42
            assert tb.CHANNEL_ID is None, '"" channel id must disable the feed'
            assert tb._alert_min_quality == "B"
            assert [tf for tf, _ in tb._AUTOSCAN_LOOPS] == \
                [Timeframe.M15, Timeframe.H1]
            assert tb._AUTOSCAN_LOOPS == [(Timeframe.M15, 15),
                                          (Timeframe.H1, 60)]
    finally:
        set_app_config(_TEST_APP)
        importlib.reload(telegram_bot)


@test("telegram_bot: main() refuses to start without credentials")
def _():
    import telegram_bot
    if telegram_bot.TOKEN:      # only meaningful in the hermetic no-cred env
        return
    try:
        telegram_bot.main()
        assert False, "main() must refuse to start without token/chat_id"
    except SystemExit as e:
        assert "credentials" in str(e)
```

- [ ] **Step 2: Run to verify failure**

```bash
python test_all.py 2>&1 | grep "telegram_bot: credentials"
```

Expected: `✗ telegram_bot: credentials, quality, pairs, loops from app_config` (TOKEN still comes from the hardcoded fallback, not "json-tok").

- [ ] **Step 3: Implement the wiring**

`telegram_bot.py` — add to the local imports (after the `config` import block):

```python
from app_config import (
    get_app_config, telegram_token, telegram_chat_id, telegram_channel_id,
)
from data_provider import ForexDataProvider, make_data_provider
```

Immediately after the imports (before the logging block), add:

```python
_APP = get_app_config()
```

Change the `logging.basicConfig` line (~88) to honor the configured level:

```python
logging.basicConfig(level=getattr(logging, _APP.log_level, logging.INFO),
                    handlers=[_console_handler, _file_handler])
```

Replace the credentials block (lines ~93-105) — keep the existing comments about the owner lock and subscriber channel, change only the assignments:

```python
TOKEN = telegram_token()
CHAT_ID = telegram_chat_id()

# ... (owner-lock comment unchanged)
OWNER_ID: Optional[int] = int(CHAT_ID) if CHAT_ID else None

# ... (subscriber-channel comment unchanged)
_CHANNEL_ID_RAW = telegram_channel_id()
CHANNEL_ID: Optional[int] = int(_CHANNEL_ID_RAW) if _CHANNEL_ID_RAW else None
```

`SCAN_PAIRS` (~113):

```python
SCAN_PAIRS = list(_APP.scan_pairs)
```

`_get_shared_dp` (~195):

```python
        _shared_dp = make_data_provider(PlatformConfig())
```

Alert quality boot default (~478):

```python
_alert_min_quality = _APP.alert_min_quality
```

WS key (~611-614) — `provider_api_key` needs adding to the `app_config` import list:

```python
_cfg = PlatformConfig()
_WS_API_KEY = provider_api_key()
_WS_URL = f"wss://ws.twelvedata.com/v1/quotes/price?apikey={_WS_API_KEY}"
_WS_COOLDOWN = _cfg.ws_cooldown_seconds
```

`_AUTOSCAN_LOOPS` (~797-802) — derive from configured timeframes:

```python
_TF_INTERVAL_MINUTES = {
    Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.M30: 30,
    Timeframe.H1: 60, Timeframe.H4: 240, Timeframe.D1: 1440,
}
_AUTOSCAN_LOOPS = [
    (tf, _TF_INTERVAL_MINUTES[tf]) for tf in _APP.scan_timeframes
]
```

Scheduler log line (~817):

```python
        logger.info("Candle-boundary scheduler started (%s)",
                    "/".join(tf.name for tf, _ in _AUTOSCAN_LOOPS))
```

(The manual `/scan` buttons — `_TF_BUTTON_LABELS` — deliberately stay hardcoded M5/M15/H1/H4: they are the owner's on-demand UI, not the automated feed. Documented decision, not an omission.)

`main()` (~1261) — first lines of the function body:

```python
def main() -> None:
    if not TOKEN or not CHAT_ID:
        raise SystemExit(
            "telegram_bot: missing Telegram credentials. Set telegram.token "
            "and telegram.chat_id in config.json, or the TELEGRAM_TOKEN and "
            "TELEGRAM_CHAT_ID environment variables.")
```

- [ ] **Step 4: Run tests, verify 198 passing**

```bash
python test_all.py 2>&1 | tail -25
```

Expected: `RESULTS: 198 passed, 0 failed, 198 total`. If Module 7 tests break on `OWNER_ID` being `None`, they were relying on the hardcoded chat id — fix by having those tests set `telegram_bot.OWNER_ID`/`CHAT_ID` explicitly in their setup, never by re-adding fallbacks.

- [ ] **Step 5: Commit**

```bash
git add telegram_bot.py test_all.py
git commit -m "telegram_bot: credentials/quality/scan-loops from config.json, startup guard"
```

---

### Task 5: Migration, docs, final verification

**Files:**
- Create: `config.json` (LOCAL ONLY — gitignored, never committed)
- Modify: `CLAUDE.md` (repo map, env-var table, config section, test count)

**Interfaces:**
- Consumes: everything above.
- Produces: the deployed machine's real `config.json`; updated authoritative docs.

- [ ] **Step 1: Create the local config.json with the current live values**

These are the exact values being deleted from source (from pre-change `telegram_bot.py:93-105` and `config.py:346`), so the running bot behaves identically:

```bash
cat > /home/harwinder/Development/ForexBot2/config.json <<'EOF'
{
  "telegram": {
    "token": "8228251480:AAFW2OAyQ2idMicql2t_CIXk1r9-qfRTfNE",
    "chat_id": "6025899022",
    "channel_id": "-1003942265371",
    "alert_min_quality": "C"
  },
  "data_providers": {
    "active": "twelve_data",
    "twelve_data": { "api_key": "f563b22ffdfd4924955decbe1a8c8701" }
  },
  "scanning": {
    "pairs": ["XAU/USD"],
    "timeframes": ["M5", "M15", "H1", "H4"]
  },
  "pairs": {
    "XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
  },
  "log_level": "INFO"
}
EOF
git status --short   # config.json must NOT appear (gitignored)
```

- [ ] **Step 2: Boot smoke test (no network needed)**

```bash
python - <<'EOF'
from app_config import get_app_config, telegram_token, provider_api_key, apply_pair_overrides
from pair_signals import PAIR_PRODUCERS
from config import Timeframe, XAUUSD
app = get_app_config()
assert telegram_token(app), "token missing"
assert provider_api_key(app), "api key missing"
cfg = PAIR_PRODUCERS[XAUUSD](Timeframe.M5, None)
assert cfg.xau_strategy.blocked_hours_utc == [0, 1, 2, 10, 11, 13, 16, 21, 9]
print("boot smoke OK — live config.json loads and factory output unchanged")
EOF
```

Expected: `boot smoke OK — live config.json loads and factory output unchanged`.

- [ ] **Step 3: Update CLAUDE.md**

- Repository-map table: add rows for `app_config.py` ("config.json loader/validation, env-precedence resolvers, `apply_pair_overrides` deployment overlay") and `config.json.example` ("committed schema template; real gitignored config.json holds secrets + deployment settings").
- Environment-variables table: note precedence "env var → config.json → none"; state that hardcoded fallbacks were REMOVED 2026-07-21 and the bot refuses to boot without credentials.
- `PlatformConfig` section: delete the `twelve_data_api_key` mention ("hardcoded fallback"); add one line: "Deployment settings (credentials, scan pairs/TFs, per-pair base blocked hours, alert quality, log level) overlay from config.json via `app_config.apply_pair_overrides`, applied in the pair factory BEFORE per-TF deltas."
- `pair_signals` section: mention the overlay call order in `make_config`.
- Test-suite line: update "184 tests in 12 modules" → "198 tests in 13 modules" (verify against the actual final count from Step 4 before writing).

- [ ] **Step 4: Full-suite final verification**

```bash
python test_all.py 2>&1 | tail -15
```

Expected: `RESULTS: 198 passed, 0 failed, 198 total` (use the real number if it differs, and make CLAUDE.md match).

- [ ] **Step 5: Commit docs**

```bash
git add CLAUDE.md
git commit -m "Docs: config.json deployment overlay (CLAUDE.md repo map, env table, test count)"
```

- [ ] **Step 6: Merge/PR decision**

Implementation complete → follow superpowers:finishing-a-development-branch (options: merge `centralized-config` to main, or push the branch to `origin`/`gitea` for review). After merge, remind the user: **rotate the Telegram token and Twelve Data key** — the old values remain in git history on both remotes.