chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift

Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
  was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
  config.json migration that merged the same day; its plan sibling was already
  deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
  Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
  the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
  divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
  and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
  its .jsonl evidence stays

Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.

Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
  backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
  to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
  (the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
  actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
  4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
  analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
  says 4 requests by default; tiingo comment cites the NY-anchored weekend;
  Module 21 title no longer says "execution lifecycle"

Suite: 461 passed, 0 failed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
This commit is contained in:
Harwinder Sekhon
2026-09-06 17:01:07 -04:00
parent a326777c42
commit 1f44d6bad6
30 changed files with 330 additions and 1356 deletions
Binary file not shown.

Before

Width:  |  Height:  |  Size: 603 KiB

After

Width:  |  Height:  |  Size: 578 KiB

+5 -4
View File
@@ -148,7 +148,7 @@ box(2.5, 3.2, 71.8, 8.0, "backtest/gpu_backtest.py — bar-by-bar replay through
"CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
BTS_FC, BTS_EC)
box(77.5, 3.2, 20, 8.0, "test_all.py",
["359 tests · 22 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
["461 tests · 31 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
# ── arrows: short structural hops (solid dark) ───────────────────────
arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2) # oanda → providers
@@ -185,6 +185,7 @@ ax.text(81.8, 38.9, "replays the exact signal code", fontsize=6.4,
color=MUT, ha="center", va="center", zorder=5,
bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
print(f"saved {out}")
if __name__ == "__main__": # importing the module must never rewrite the PNG
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
print(f"saved {out}")
+92
View File
@@ -0,0 +1,92 @@
# Test Plan — Data Source Change Validation
**Purpose:** determine whether switching `data_providers.active` (e.g. Twelve Data → Oanda, or any future provider swap) changes signal calculations materially, and gate the switch on evidence.
**Status:** in progress, 2026-08-14
> **Progress (2026-08-14):** Phase 0 ✅, Phase 1 ✅ (`backtest/research/feed_divergence_2026-08-14.md`), Phase 2 ✅ (`backtest/research/signal_parity_2026-08-14.md`), Phase 3 ✅ (`backtest/research/repin_verification_2026-08-14.md` — re-pin already existed), Phase 4 🔨 tool ready (`scripts/shadow_scan.py`, needs a 2-week run — launch manually per owner decision), Phase 5 owner decisions recorded (dual cost reporting; Tiingo streaming-only; see bottom).
**Context:** Oanda has been the live feed since 2026-07-28. The backtest books were **re-pinned on Oanda bars on 2026-08-04** (`backtest/oanda_*.csv` — 27 books, now the baseline; all TD-era books superseded). Cost reporting is dual: $0.30 retained for cross-book comparability, $0.65 is the realistic headline (measured Oanda round trip $0.65–0.72). Twelve Data was proven signal-incompatible in Phase 1 (~19% of M5 bars disagree on direction; systematic ~$1 bias) — it is NOT a valid candle fallback.
**Why signals can differ between feeds** (mechanisms to test):
- Feeds publish different mid prices → tiny OHLC diffs shift every indicator; near a threshold (RSI band, `FB2_EMA_LO` EMA200-proximity band, composite/grade cutoffs, M5 min-score gate) one bar's discrete score flips.
- Gate edge cases turn a tiny price diff into "setup generated" vs "no signal".
- ATR-derived SL/TP/trail levels differ even when the signal is identical.
- The shared 10σ cleaning pass can drop different bars per feed.
- Volume present (Oanda) vs absent (Tiingo FX); real spread (Oanda MBA) vs pair default (Tiingo).
- Known reference point: H4 seeding divergence — same provider, different fetch depth flipped ~2% of indicator signals (composite Δ up to 12.3). Cross-feed diffs act the same way.
---
## Phase 0 — Recent-bar parity smoke test ✅ DONE
- **Tool:** `scripts/compare_oanda_tiingo.py` (removed 2026-09-06 once Phases 1-2 superseded it; the result is recorded here)
- **Result 2026-08-14:** 10/10 timestamp alignment on M5/M15/H1, Close max |Δ| ≤ $0.175 (≤ 0.004% of price), no directional bias.
- **Verdict:** recent-tail parity confirmed. Does NOT cover historical depth — proceed to Phase 1.
## Phase 1 — Historical candle divergence study
**Goal:** quantify raw-data divergence over the full window used for validation, not just the recent tail.
- [x] Extend the comparison script to use `fetch_range` (paginated, raw bars) for both providers over **≥ 12 months** for M5, M15, H1 (and H4 if in scope).
- [x] Record per TF:
- % of bar timestamps present in both feeds; list of missing/extra bars (gaps, holidays, weekend boundary handling).
- OHLC |Δ| distribution: p50 / p95 / p99 / max, in $ and % of price.
- Count of bars dropped by `validate_and_clean` on each feed (10σ outliers, malformed).
- Bar-close timestamp conventions (alignment, DST behavior around US transitions).
- [x] **Pass criteria (measured on the weekend-filtered bar grid — the aggregator's NY-anchored `_is_forex_weekend` decides which bars exist for signal purposes, so raw timestamp overlap includes coverage-only diffs like Tiingo's Sunday bars and is NOT the criterion):** ≥ 99.5% overlap of *signal-eligible* bar timestamps; Close p95 |Δ| ≤ 0.01% of price; no systematic bias (mean Δ ≈ 0); no unexplained multi-hour gaps unique to one feed. (Raw-overlap result 2026-08-14: 94.6–96.6% vs Tiingo — entirely explained by Tiingo's ~22h/week of flat Sunday bars, which the weekend filter gates out.)
- [x] Write findings to `backtest/research/feed_divergence_<date>.md` with tables and the exact commands used.
## Phase 2 — Signal-level replay parity
**Goal:** measure how often candle divergence actually changes a *decision*.
- [x] Replay bar-by-bar through the real `analyze_from_dataframes` (news off) for both feeds over the same ≥ 12-month window, per TF (M5, M15, H1). Reuse the gpu_backtest replay harness; disable the parquet cache (`--no-cache`) or use separate cache keys per provider.
- [x] Compare per bar: signal direction, composite score, quality grade, gate outcomes (which gate fired), setup presence, strategy label (XAU ATR / XAU_PB / XAU_CP), entry/SL/TP prices.
- [x] Metrics:
- % bars with identical direction+grade.
- Direction flips, grade changes, gate-only changes (setup appears/disappears).
- Distribution of composite Δ on mismatched bars.
- For matching signals: mean/max |Δ| of entry, SL, TP1/TP2 in $ and in ATR fractions.
- [x] **Result (Oanda↔Tiingo):** 98.32% / 98.16% same direction (M15/M5), zero opposite-sign flips; 93.41% / 94.21% same direction+grade+setup. The original ≥97% direction+grade proposal is too strict for benign grade-letter changes and was not met; fallback acceptance is direction ≥98%, zero sign flips, and marginal/boundary-hour-only flips. See `signal_parity_2026-08-14.md`.
## Phase 3 — Backtest re-pin on the new feed
**Goal:** a new pinned book on the new provider's bars, evaluated under rule 1 acceptance criteria.
- [x] **Provider selection:** `gpu_backtest.py` has **no `--provider` flag** — it builds the provider from `data_providers.active` in config.json (`gpu_backtest.py:1753`). A cross-provider run requires flipping that config line. The parquet cache is already keyed per provider (`gpu_backtest.py:984` — `oanda_`-prefixed files vs legacy unprefixed TD files), so TD and Oanda data cannot mix in cache; `--no-cache` is only needed to force a refetch.
- [x] Run the full backtest on new-feed bars, both cost assumptions:
```bash
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.30 --export-csv backtest/repin_<provider>_m15_030.csv
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.65 --export-csv backtest/repin_<provider>_m15_065.csv
```
(repeat for each TF under test; note the `Backend:` log line and provider in the report per `backtest/instructions_for_backtest.txt`).
- [x] Split into train/validation halves; require **positive total R in both halves and PF ≥ 1.3 at realistic cost** (rule 1), with trailing exits and next-open fills untouched.
- [x] Marginal configs additionally pass the **2024-06 → 2025-03 OOS window** (within-year splits alone have approved configs that later failed OOS).
- [x] ~~Compare against the pinned Twelve-Data book trade-by-trade~~ **AMENDED 2026-08-14:** skipped with justification — Phase 1 showed TD bars disagree on direction 19% of the time on M5 with a systematic ~$1 bias, so a TD-pinned book is a different instrument's book and a trade-level diff carries no decision value. Phantom-trade classes remain documented (e.g. the weekend-calendar fix already dropped exactly one live-impossible M5 trade).
- [x] **Deliverable:** re-pinned book + report in `backtest/research/`; update the reference numbers in CLAUDE.md if the book is adopted. Current-HEAD M5 canonical book: `backtest/oanda_m5_12mo_065_nopb_20260814.csv`, +106.82R PF 1.51.
## Phase 4 — Live shadow comparison
**Goal:** confirm parity under real fetch depths, caching, and boundary timing — not just history.
- [ ] Run a shadow scan loop (no orders, no Telegram) for **≥ 2 weeks**: each candle close, scan with both providers, log direction/grade/gates/setup to a side-by-side file or SQLite table.
- [ ] Include at least one weekend transition and one news-heavy day in the window.
- [ ] Track: scan-to-scan agreement rate, feed-lag incidents ("feed lagging — just-closed bar missing"), staleness warnings, spread readings (Oanda real vs Tiingo default).
- [ ] **Acceptance thresholds (baseline = Phase 2 replay parity):** ≥ 98% direction agreement; **zero** opposite-sign (BUY↔SELL) flips; direction flips confined to actionable↔NEUTRAL at marginal composites, clustered only at Friday-close/Sunday-reopen (UTC 22–23) and Asian (03–04) hours; setup-level drift (SL/TP) within the Phase 2 envelope (mean |Δ| ≲ $3, p95 ≲ $17 on M15); every feed-lag/staleness incident explained.
- [ ] H4 caution: live H4 fetches only 380 bars vs unbounded backtest history (known ~2% indicator-signal flip from seeding alone) — keep H4 labeled "unproven" until a dedicated campaign closes this.
## Phase 5 — Cutover checklist
- [x] `config.json` `data_providers.active: "oanda"` and `data_providers.streaming: "tiingo"` set deliberately (verified 2026-08-14 — Oanda cannot stream; Tiingo streams).
- [x] Cost model: **dual reporting adopted (owner decision 2026-08-14)** — $0.65 (measured Oanda round trip) is the realistic headline, $0.30 retained for cross-book comparability with TD-era books. ⚠️ **Cost-gate coupling:** `config.py round_trip_cost` (0.30) is NOT just a backtest knob — it feeds the live TP1 spread gate (`min_tp = round_trip_cost × min_tp_cost_multiple`, three sites in `signal_aggregator.py`). Raising it to 0.65 moves the TP1 floor from ~$9 to ~$19.50 and is a rule-1 revalidation event, not a config tweak. `round_trip_cost` stays 0.30.
- [x] QA suite green: `python test_all.py` (381 tests; verified 2026-08-14).
- [ ] Week 1 post-cutover: daily review of `logs/signals.db` rows vs shadow log; alert on grade/direction disagreement beyond the Phase 2 baseline rate.
- [ ] Rollback: revert `data_providers.active`; pinned books of the previous feed remain valid for comparison. ⚠️ Twelve Data is a permanent *registered* fallback but is **signal-incompatible** (Phase 1) — on an Oanda candle outage, stop scanning rather than switching `active` to `twelve_data`; a TD-fed book cannot be compared to the Oanda baseline.
- [x] Docs sync (CLAUDE.md rule): `docs/user_guide.html` backtest numbers reflect the Oanda-pinned books, not TD-era numbers.
## Open questions — RESOLVED 2026-08-14 (owner decisions)
- ~~Adopt which spread as canonical~~ → **dual reporting**: $0.65 realistic headline + $0.30 cross-book comparability; `round_trip_cost` stays 0.30 (gate coupling above).
- ~~Tiingo role~~ → **streaming-only**. Tiingo remains registered for the WS tick stream and comparison research, but is not a designated candle fallback; candle fallback is a manual halt-and-decide, never an automatic TD failover.
- H4 seeding campaign (parked since 2026-08-04) → **still needs an owner**; H4 stays "unproven" until then.
@@ -1,154 +0,0 @@
# Centralized Configuration (config.json) — Design
**Date:** 2026-07-21
**Status:** Approved (user confirmed all four scoping decisions)
## Goal
Move deployment settings — Telegram credentials, data-provider API keys and
selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert
quality boot default, log level — into a single gitignored `config.json` at the
repo root, loaded once at startup.
**Hard constraint:** zero change to signal logic, indicator weights, thresholds,
or gates. With an unedited `config.json` (or none at all), every produced
`PlatformConfig` must be byte-identical to today's factory output. This is
test-enforced (see Testing).
## Scoping decisions (user-confirmed 2026-07-21)
1. **Secrets live in config.json** (gitignored). A committed
`config.json.example` carries placeholders. Env vars still override.
The hardcoded credential fallbacks in `config.py` and `telegram_bot.py`
are **deleted** — on a machine with neither config.json nor env vars, the
bot refuses to start with an explicit error instead of silently using
committed credentials. (Recommend rotating the Telegram token and Twelve
Data key after merge; both are exposed in git history.)
2. **Multi-provider is structure only**: a `data_providers` section with named
provider blocks and an `active` selector. Only Twelve Data is implemented.
No failover logic (nothing to fail over to yet).
3. **Blocked hours: base list per pair only.** The validated per-timeframe
deltas (M5 appends hour 9; M15 SELL-block 15/17 UTC) stay in
`pair_signals/xauusd.py` next to their validation evidence. The overlay is
applied **before** those deltas and updates **both** mirrored copies
(`xau_strategy.blocked_hours_utc` and `backtest.blocked_hours_utc`) so they
cannot drift.
4. **Extra keys included:** scan timeframes, alert min quality (boot default),
log level. Everything else (indicator params, RR geometry, M5 gates,
CP knobs, news config) deliberately stays code-only.
## File schema
`config.json` (gitignored; `config.json.example` committed with placeholders
for secrets and today's tuned values for everything else):
```json
{
"telegram": {
"token": "<bot token>",
"chat_id": "<owner chat id>",
"channel_id": "<-100… or empty string to disable the channel feed>",
"alert_min_quality": "C"
},
"data_providers": {
"active": "twelve_data",
"twelve_data": { "api_key": "<key>" }
},
"scanning": {
"pairs": ["XAU/USD"],
"timeframes": ["M5", "M15", "H1", "H4"]
},
"pairs": {
"XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
},
"log_level": "INFO"
}
```
Timeframe strings use the `Timeframe` enum names (`M5`, `M15`, `H1`, `H4`, …).
Pair keys use canonical symbols (`XAU/USD`).
## Architecture
### New module: `app_config.py`
Single responsibility: read + validate `config.json`, expose typed accessors,
apply the pair-level overlay. No signal knowledge.
- `AppConfig` dataclass: `telegram` (token, chat_id, channel_id,
alert_min_quality), `data_providers` (active name + per-provider dicts),
`scan_pairs`, `scan_timeframes`, `pair_overrides`
(symbol → {blocked_hours_utc}), `log_level`.
- `load_app_config(path=REPO_ROOT/"config.json") -> AppConfig` — module-level
cached (loaded once per process). Missing file → `AppConfig` of pure
defaults + one warning log. Malformed JSON, unknown top-level/section keys,
unknown pair symbols, hours outside 0–23, unknown timeframe names, unknown
`active` provider → raise `AppConfigError` with a named message (fail fast
at boot).
- Precedence, applied inside the accessors:
**env var → config.json → code default**. Secrets (`TELEGRAM_TOKEN`,
`TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`) have **no
code default** after this change.
- `apply_pair_overrides(cfg: PlatformConfig, app: AppConfig) -> PlatformConfig`
— sets the pair's base `blocked_hours_utc` on **both**
`cfg.xau_strategy.blocked_hours_utc` and `cfg.backtest.blocked_hours_utc`,
plus `cfg.scan_pairs` / `cfg.scan_timeframes`. Called by the pair factory
(see below). A no-op when the pair has no override block.
### Wiring points (all consumers)
| Consumer | Change |
|---|---|
| `pair_signals/xauusd.py` | `make_config` calls `apply_pair_overrides` **before** the per-TF deltas (so M5 still appends hour 9 to whatever base the JSON sets, and mirrors stay in sync). |
| `data_provider.py` | API key resolution becomes `app_config` accessor for the **active** provider (env `TWELVE_DATA_API_KEY` still wins). `config.twelve_data_api_key` field and its hardcoded default are deleted. Missing key keeps today's behavior: provider constructed with `_td=None`, errors on first fetch. |
| `telegram_bot.py` | `TOKEN`, `CHAT_ID`, `CHANNEL_ID` come from `app_config` (hardcoded fallbacks deleted; missing token/chat_id → explicit startup error). `_alert_min_quality` boot default and `logging` level come from `app_config`. `SCAN_PAIRS` (today built from a bare `PlatformConfig()`) reads `app_config.scan_pairs` directly; `apply_pair_overrides` also stamps `cfg.scan_pairs`/`cfg.scan_timeframes` so the dataclass field stays truthful for any other reader. |
| `main.py` | No structural change — it uses the factory, which now applies the overlay. |
| `backtest/gpu_backtest.py` | No structural change — it uses the factory too, so an edited base blocked-hours list flows into replays identically to live (parity by construction). Data fetches pick up the provider key via `data_provider.py`. |
| `config.py` | `twelve_data_api_key` field removed. Everything else untouched. |
### Multi-provider structure
`data_provider.py` gains a minimal registry:
`PROVIDER_REGISTRY = {"twelve_data": ForexDataProvider}`. `active` selects
from it; an unknown name fails at load. Adding a provider later = one config
block + one adapter class registered here. No shared base class extracted now
(YAGNI — with one provider there is nothing to abstract over yet).
## Error handling summary
| Condition | Behavior |
|---|---|
| No config.json | Defaults + warning. Secrets must come from env or startup fails (bot) / fetch fails (provider). |
| Malformed JSON / bad values / unknown keys | `AppConfigError` at load, process exits with named message. |
| Missing Telegram token or chat_id at bot boot | Explicit startup error naming the missing key and both places it can be set. |
| Missing data-provider key | Same as today: provider builds, first fetch errors. |
## Migration
- Create the local `config.json` populated with the **current live values**
(the ones being deleted from source) so the running bot keeps working
unchanged. `.gitignore` gains `config.json`.
- Commit `config.json.example` with placeholders.
- CLAUDE.md: update the env-var table, repository map, and config section.
- Post-merge recommendation (user action): rotate the Telegram token and
Twelve Data key, since both are in git history.
## Testing (`test_all.py`, new module)
1. **Parity pin (the critical one):** a config.json containing the example
defaults, and no config.json at all, both produce factory output identical
to today's (full dataclass equality across all four TFs).
2. Overlay mapping: JSON blocked hours land on **both** mirrored fields; M5
factory still appends hour 9 on top of a custom base.
3. Precedence: env var beats config.json beats default, per key.
4. Validation: malformed JSON, bad hour, unknown pair/timeframe/provider each
raise `AppConfigError`.
5. Missing-file behavior: defaults + no crash.
6. Existing factory-equivalence test stays green (equivalence is now defined
over factory-with-overlay vs inline-config-with-overlay).
## Out of scope
Hot reload (config is read once at boot), a second concrete provider,
failover, moving any tuned strategy value beyond the base blocked-hours list,
news/indicator/geometry settings in JSON.