ForexBot2/docs/superpowers/specs/2026-07-21-centralized-config-design.md

# Centralized Configuration (config.json) — Design

**Date:** 2026-07-21
**Status:** Approved (user confirmed all four scoping decisions)

## Goal

Move deployment settings — Telegram credentials, data-provider API keys and
selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert
quality boot default, log level — into a single gitignored `config.json` at the
repo root, loaded once at startup.

**Hard constraint:** zero change to signal logic, indicator weights, thresholds,
or gates. With an unedited `config.json` (or none at all), every produced
`PlatformConfig` must be byte-identical to today's factory output. This is
test-enforced (see Testing).

## Scoping decisions (user-confirmed 2026-07-21)

1. **Secrets live in config.json** (gitignored). A committed
   `config.example.json` carries placeholders. Env vars still override.
   The hardcoded credential fallbacks in `config.py` and `telegram_bot.py`
   are **deleted** — on a machine with neither config.json nor env vars, the
   bot refuses to start with an explicit error instead of silently using
   committed credentials. (Recommend rotating the Telegram token and Twelve
   Data key after merge; both are exposed in git history.)
2. **Multi-provider is structure only**: a `data_providers` section with named
   provider blocks and an `active` selector. Only Twelve Data is implemented.
   No failover logic (nothing to fail over to yet).
3. **Blocked hours: base list per pair only.** The validated per-timeframe
   deltas (M5 appends hour 9; M15 SELL-block 15/17 UTC) stay in
   `pair_signals/xauusd.py` next to their validation evidence. The overlay is
   applied **before** those deltas and updates **both** mirrored copies
   (`xau_strategy.blocked_hours_utc` and `backtest.blocked_hours_utc`) so they
   cannot drift.
4. **Extra keys included:** scan timeframes, alert min quality (boot default),
   log level. Everything else (indicator params, RR geometry, M5 gates,
   CP knobs, news config) deliberately stays code-only.

## File schema

`config.json` (gitignored; `config.example.json` committed with placeholders
for secrets and today's tuned values for everything else):

```json
{
  "telegram": {
    "token": "<bot token>",
    "chat_id": "<owner chat id>",
    "channel_id": "<-100… or empty string to disable the channel feed>",
    "alert_min_quality": "C"
  },
  "data_providers": {
    "active": "twelve_data",
    "twelve_data": { "api_key": "<key>" }
  },
  "scanning": {
    "pairs": ["XAU/USD"],
    "timeframes": ["M5", "M15", "H1", "H4"]
  },
  "pairs": {
    "XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
  },
  "log_level": "INFO"
}
```

Timeframe strings use the `Timeframe` enum names (`M5`, `M15`, `H1`, `H4`, …).
Pair keys use canonical symbols (`XAU/USD`).

## Architecture

### New module: `app_config.py`

Single responsibility: read + validate `config.json`, expose typed accessors,
apply the pair-level overlay. No signal knowledge.

- `AppConfig` dataclass: `telegram` (token, chat_id, channel_id,
  alert_min_quality), `data_providers` (active name + per-provider dicts),
  `scan_pairs`, `scan_timeframes`, `pair_overrides`
  (symbol → {blocked_hours_utc}), `log_level`.
- `load_app_config(path=REPO_ROOT/"config.json") -> AppConfig` — module-level
  cached (loaded once per process). Missing file → `AppConfig` of pure
  defaults + one warning log. Malformed JSON, unknown top-level/section keys,
  unknown pair symbols, hours outside 0–23, unknown timeframe names, unknown
  `active` provider → raise `AppConfigError` with a named message (fail fast
  at boot).
- Precedence, applied inside the accessors:
  **env var → config.json → code default**. Secrets (`TELEGRAM_TOKEN`,
  `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`) have **no
  code default** after this change.
- `apply_pair_overrides(cfg: PlatformConfig, app: AppConfig) -> PlatformConfig`
  — sets the pair's base `blocked_hours_utc` on **both**
  `cfg.xau_strategy.blocked_hours_utc` and `cfg.backtest.blocked_hours_utc`,
  plus `cfg.scan_pairs` / `cfg.scan_timeframes`. Called by the pair factory
  (see below). A no-op when the pair has no override block.

### Wiring points (all consumers)

| Consumer | Change |
|---|---|
| `pair_signals/xauusd.py` | `make_config` calls `apply_pair_overrides` **before** the per-TF deltas (so M5 still appends hour 9 to whatever base the JSON sets, and mirrors stay in sync). |
| `data_provider.py` | API key resolution becomes `app_config` accessor for the **active** provider (env `TWELVE_DATA_API_KEY` still wins). `config.twelve_data_api_key` field and its hardcoded default are deleted. Missing key keeps today's behavior: provider constructed with `_td=None`, errors on first fetch. |
| `telegram_bot.py` | `TOKEN`, `CHAT_ID`, `CHANNEL_ID` come from `app_config` (hardcoded fallbacks deleted; missing token/chat_id → explicit startup error). `_alert_min_quality` boot default and `logging` level come from `app_config`. `SCAN_PAIRS` (today built from a bare `PlatformConfig()`) reads `app_config.scan_pairs` directly; `apply_pair_overrides` also stamps `cfg.scan_pairs`/`cfg.scan_timeframes` so the dataclass field stays truthful for any other reader. |
| `main.py` | No structural change — it uses the factory, which now applies the overlay. |
| `backtest/gpu_backtest.py` | No structural change — it uses the factory too, so an edited base blocked-hours list flows into replays identically to live (parity by construction). Data fetches pick up the provider key via `data_provider.py`. |
| `config.py` | `twelve_data_api_key` field removed. Everything else untouched. |

### Multi-provider structure

`data_provider.py` gains a minimal registry:
`PROVIDER_REGISTRY = {"twelve_data": ForexDataProvider}`. `active` selects
from it; an unknown name fails at load. Adding a provider later = one config
block + one adapter class registered here. No shared base class extracted now
(YAGNI — with one provider there is nothing to abstract over yet).

## Error handling summary

| Condition | Behavior |
|---|---|
| No config.json | Defaults + warning. Secrets must come from env or startup fails (bot) / fetch fails (provider). |
| Malformed JSON / bad values / unknown keys | `AppConfigError` at load, process exits with named message. |
| Missing Telegram token or chat_id at bot boot | Explicit startup error naming the missing key and both places it can be set. |
| Missing data-provider key | Same as today: provider builds, first fetch errors. |

## Migration

- Create the local `config.json` populated with the **current live values**
  (the ones being deleted from source) so the running bot keeps working
  unchanged. `.gitignore` gains `config.json`.
- Commit `config.example.json` with placeholders.
- CLAUDE.md: update the env-var table, repository map, and config section.
- Post-merge recommendation (user action): rotate the Telegram token and
  Twelve Data key, since both are in git history.

## Testing (`test_all.py`, new module)

1. **Parity pin (the critical one):** a config.json containing the example
   defaults, and no config.json at all, both produce factory output identical
   to today's (full dataclass equality across all four TFs).
2. Overlay mapping: JSON blocked hours land on **both** mirrored fields; M5
   factory still appends hour 9 on top of a custom base.
3. Precedence: env var beats config.json beats default, per key.
4. Validation: malformed JSON, bad hour, unknown pair/timeframe/provider each
   raise `AppConfigError`.
5. Missing-file behavior: defaults + no crash.
6. Existing factory-equivalence test stays green (equivalence is now defined
   over factory-with-overlay vs inline-config-with-overlay).

## Out of scope

Hot reload (config is read once at boot), a second concrete provider,
failover, moving any tuned strategy value beyond the base blocked-hours list,
news/indicator/geometry settings in JSON.