Harwinder Sekhon
c00ed4471b
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
183 lines
13 KiB
Markdown
183 lines
13 KiB
Markdown
# Signal-log audit → prediction-quality campaign — 2026-09-03
|
||
|
||
**Source evidence:** `logs/signals.db` 2026-08-13 → 2026-09-03 (14,683 rows, candle
|
||
snapshots from 08-18), replayed with the repo's own advisory outcome model
|
||
(`analyze_signal_log.py --stored-candles`; the replay reproduces
|
||
`advisory_outcome.py` to 5e-17 on the 49 panel-seeded rows), cross-checked
|
||
against the pinned Oanda books (`backtest/oanda_*_065*.csv`).
|
||
|
||
## Live-log findings that motivated the campaign
|
||
|
||
| View (3 weeks, $0.65 rt) | n | WR | R | PF |
|
||
|---|---|---|---|---|
|
||
| every unique setup event | 144 | 67% | +37.8 | 1.93 |
|
||
| opportunities (first per tf+direction, 60 min) | 66 | 67% | +18.3 | 1.92 |
|
||
| one-position-per-TF (5-bar cooldown) | 32 | 62% | +6.6 | 1.53 |
|
||
| direction BUY / SELL | 121 / 23 | 70% / 48% | +43.2 / −5.4 | 2.53 / 0.56 |
|
||
|
||
- Two rally days (08-19, 08-21) contribute essentially all of the R.
|
||
- SELL is negative in every view; in the books SELL is the weakest bucket on
|
||
every TF (H1 SELL −16.7R/5yr PF 0.77, −11.5R OOS PF 0.30; M15 SELL −11.7R OOS;
|
||
M5 SELL in `transitioning` −7.9R/12mo PF 0.21, −5.2R/5yr PF 0.49).
|
||
- Bollinger is anti-predictive on all four TFs (forward-return hit 0.26–0.50 at
|
||
12 bars) and "opposes" 93% of taken book trades; divergence is inverted on
|
||
M15/H1 live and trades where it opposed outperform in the books.
|
||
- Refuted live hypotheses (books disagree): ADX<20 gate, exhaustion gate,
|
||
NEAR-EMA200 gate, EXTREME-volatility gate, capping SL/ATR.
|
||
|
||
## Method (rule 1)
|
||
|
||
Harness: `backtest/gpu_backtest.py --pair XAUUSD --spread 0.65` on the CPU
|
||
backend (CuPy present but the GPU was not reachable; the M5 12mo control
|
||
reproduces the pinned `oanda_m5_12mo_065_nopb_20260814.csv` trade-for-trade
|
||
except the two trades on the end date, which `--end` excludes). Every candidate
|
||
is compared with a CONTROL produced on the same harness, never with the pinned
|
||
book directly. Windows (cache start = start − 62 days):
|
||
|
||
| window | `--start` | `--end` |
|
||
|---|---|---|
|
||
| 12mo (M15/H1) | 2025-08-04 | 2026-08-04 |
|
||
| 12mo (M5 canonical) | 2025-08-14 | 2026-08-14 |
|
||
| 5yr | 2021-08-01 | 2026-08-04 |
|
||
| OOS | 2024-06-01 | 2025-03-30 |
|
||
|
||
Acceptance: positive total R in both 12mo halves, PF ≥ 1.3, OOS not worse
|
||
than control, and a candidate that removes a net-positive bucket for a PF-only
|
||
gain is rejected (big-candle precedent). Runner:
|
||
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py` (scratchpad
|
||
tooling, not retained — both shipped gates are reproduced by a plain pair-factory
|
||
run at HEAD; see the parity recipe in `sizing_campaign_2026-09.md`).
|
||
|
||
Candidates (all `--set`, no code):
|
||
|
||
- **B1 SELL gating** — H1 `xau_strategy.sell_min_quality=[]` (block all H1
|
||
shorts); M5 `xau_strategy.sell_blocked_regimes=["transitioning"]`; M15
|
||
`xau_strategy.sell_min_quality=["B"]`.
|
||
- **B2 divergence off** — `technical.disabled_indicators=["divergence"]` on
|
||
M15 and H1 (M5 divergence-down was rejected 2026-08-04; not re-run).
|
||
- **B3 Bollinger weight 0 in trending regimes** — full
|
||
`technical.regime_weights_override` with bollinger 0.0 in
|
||
trending_up/trending_down/trending, ranging/transitioning unchanged; M5, M15, H1.
|
||
|
||
## Results
|
||
|
||
Every control reproduced its pinned book: M15 12mo (`ctl_m15_12mo`) is exact
|
||
at 397/+73.93R/PF 1.53; all three H1 controls are exact; the M5 12mo control
|
||
(`base_m5_12mo`) is exact except for the two trades that fall on the end
|
||
date, which `--end` excludes on this harness (see Method above).
|
||
|
||
| book | n | R | PF | DD | H1 | H2 | sellN | sellR | ΔR vs control | ΔPF |
|
||
|---|---|---|---|---|---|---|---|---|---|---|
|
||
| b1_h1_nosell_12mo | 91 | +22.2 | 1.79 | 4.1 | +20.6 | +1.6 | 0 | +0.0 | -1.8 | +0.25 |
|
||
| b1_h1_nosell_5yr | 335 | +49.1 | 1.46 | 13.2 | -5.0 | +54.1 | 0 | +0.0 | +16.7 | +0.28 |
|
||
| b1_h1_nosell_oos | 78 | +16.7 | 1.74 | 3.6 | +7.3 | +9.4 | 0 | +0.0 | +11.5 | +0.61 |
|
||
| b1_m15_sellB_12mo | 341 | +74.6 | 1.62 | 10.8 | +68.4 | +6.2 | 82 | +9.1 | +0.7 | +0.09 |
|
||
| b1_m15_sellB_5yr | 1378 | +119.1 | 1.22 | 41.4 | -21.3 | +140.3 | 335 | +3.7 | -8.9 | +0.02 |
|
||
| b1_m15_sellB_oos | 272 | +47.1 | 1.51 | 13.8 | +12.3 | +34.8 | 59 | -3.4 | +8.3 | +0.15 |
|
||
| b1_m5_selltrans_12mo | 557 | +112.2 | 1.56 | 9.9 | +55.9 | +56.4 | 75 | +14.8 | +7.9 | +0.06 |
|
||
| b1_m5_selltrans_5yr | 889 | +133.7 | 1.41 | 12.8 | -8.3 | +142.0 | 88 | +9.0 | +5.3 | +0.03 |
|
||
| b1_m5_selltrans_oos | 86 | +9.4 | 1.33 | 8.2 | -1.4 | +10.9 | 4 | +1.5 | +0.0 | +0.00 |
|
||
| b2_h1_nodiv_12mo | 149 | +28.6 | 1.62 | 6.5 | +23.6 | +5.0 | 49 | +5.5 | +4.6 | +0.07 |
|
||
| b2_h1_nodiv_5yr | 567 | +40.5 | 1.20 | 14.1 | -5.2 | +45.6 | 203 | -18.2 | +8.1 | +0.02 |
|
||
| b2_h1_nodiv_oos | 119 | +7.6 | 1.17 | 7.7 | -1.9 | +9.5 | 31 | -14.3 | +2.4 | +0.04 |
|
||
| b2_m15_nodiv_12mo | 439 | +81.3 | 1.51 | 13.1 | +72.3 | +8.9 | 147 | +7.0 | +7.4 | -0.02 |
|
||
| b2_m15_nodiv_5yr | 1771 | +139.9 | 1.20 | 47.4 | -16.2 | +156.1 | 578 | +11.7 | +11.9 | -0.01 |
|
||
| b2_m15_nodiv_oos | 341 | +52.3 | 1.43 | 17.1 | +15.6 | +36.7 | 93 | -10.0 | +13.5 | +0.07 |
|
||
| b3_h1_bb0_12mo | 154 | +31.2 | 1.64 | 9.4 | +22.8 | +8.4 | 52 | +6.7 | +7.3 | +0.09 |
|
||
| b3_h1_bb0_5yr | 583 | +37.3 | 1.18 | 18.3 | -7.4 | +44.7 | 206 | -20.4 | +4.9 | +0.00 |
|
||
| b3_h1_bb0_oos | 122 | +5.6 | 1.12 | 8.7 | -2.0 | +7.6 | 32 | -13.8 | +0.4 | -0.01 |
|
||
| b3_m15_bb0_12mo | 439 | +49.9 | 1.29 | 19.4 | +61.6 | -11.8 | 151 | +0.3 | -24.0 | -0.23 |
|
||
| b3_m15_bb0_5yr | 1790 | +126.6 | 1.18 | 42.3 | -1.8 | +128.5 | 592 | +17.4 | -1.3 | -0.03 |
|
||
| b3_m15_bb0_oos | 345 | +55.5 | 1.47 | 12.4 | +18.2 | +37.3 | 94 | -8.4 | +16.7 | +0.11 |
|
||
| b3_m5_bb0_12mo | 621 | +84.0 | 1.35 | 16.6 | +49.3 | +34.7 | 104 | +9.7 | -20.4 | -0.14 |
|
||
| b3_m5_bb0_5yr | 987 | +113.0 | 1.31 | 15.3 | -7.7 | +120.6 | 134 | +12.0 | -15.5 | -0.08 |
|
||
| b3_m5_bb0_oos | 93 | +10.6 | 1.35 | 5.6 | -2.9 | +13.5 | 9 | +3.7 | +1.2 | +0.01 |
|
||
| b12_h1_nosell_nodiv_12mo | 100 | +23.1 | 1.75 | 4.6 | +21.8 | +1.3 | 0 | +0.0 | -0.8 | +0.20 |
|
||
| b12_h1_nosell_nodiv_5yr | 364 | +58.7 | 1.52 | 12.9 | -4.1 | +62.8 | 0 | +0.0 | +26.3 | +0.34 |
|
||
| b12_h1_nosell_nodiv_oos | 88 | +21.9 | 1.92 | 4.8 | +9.4 | +12.5 | 0 | +0.0 | +16.8 | +0.79 |
|
||
| base_m5_12mo | 575 | +104.3 | 1.50 | 10.7 | +50.8 | +53.5 | 93 | +6.9 | — | — |
|
||
| ctl_h1_12mo | 138 | +24.0 | 1.55 | 8.0 | +19.4 | +4.6 | 47 | +1.8 | — | — |
|
||
| ctl_h1_5yr | 515 | +32.4 | 1.18 | 15.2 | -5.9 | +38.3 | 180 | -16.7 | — | — |
|
||
| ctl_h1_oos | 104 | +5.1 | 1.13 | 7.5 | -0.9 | +6.1 | 26 | -11.5 | — | — |
|
||
| ctl_m15_12mo | 397 | +73.9 | 1.53 | 10.8 | +66.7 | +7.2 | 138 | +8.5 | — | — |
|
||
| ctl_m15_5yr | 1578 | +128.0 | 1.21 | 35.8 | -3.4 | +131.4 | 535 | +12.6 | — | — |
|
||
| ctl_m15_oos | 299 | +38.8 | 1.36 | 15.5 | +5.9 | +32.9 | 86 | -11.7 | — | — |
|
||
| ctl_m5_5yr | 911 | +128.4 | 1.39 | 12.8 | -8.3 | +136.7 | 110 | +3.8 | — | — |
|
||
| ctl_m5_oos | 86 | +9.4 | 1.33 | 8.2 | -1.4 | +10.9 | 4 | +1.5 | — | — |
|
||
|
||
Raw table (n / R / PF / WR% / DD / H1 / H2 / PF-half-1 / PF-half-2 /
|
||
sellN / sellR / ΔR / ΔPF / Δn vs control; control and `base_m5_12mo`
|
||
rows carry no delta):
|
||
|
||
```
|
||
book n R PF WR% DD H1 H2 PF1 PF2 sellN sellR | ΔR ΔPF Δn
|
||
b1_h1_nosell_12mo 91 +22.2 1.79 62.6 4.1 +20.6 +1.6 2.10 1.18 0 +0.0 | -1.8 +0.25 -47
|
||
b1_h1_nosell_5yr 335 +49.1 1.46 58.2 13.2 -5.0 +54.1 0.87 1.78 0 +0.0 | +16.7 +0.28 -180
|
||
b1_h1_nosell_oos 78 +16.7 1.74 62.8 3.6 +7.3 +9.4 1.67 1.81 0 +0.0 | +11.5 +0.61 -26
|
||
b1_m15_sellB_12mo 341 +74.6 1.62 62.8 10.8 +68.4 +6.2 2.15 1.10 82 +9.1 | +0.7 +0.09 -56
|
||
b1_m15_sellB_5yr 1378 +119.1 1.22 57.0 41.4 -21.3 +140.3 0.91 1.46 335 +3.7 | -8.9 +0.02 -200
|
||
b1_m15_sellB_oos 272 +47.1 1.51 60.7 13.8 +12.3 +34.8 1.24 1.85 59 -3.4 | +8.3 +0.15 -27
|
||
b1_m5_selltrans_12mo 557 +112.2 1.56 53.7 9.9 +55.9 +56.4 1.54 1.58 75 +14.8 | +7.9 +0.06 -18
|
||
b1_m5_selltrans_5yr 889 +133.7 1.41 51.5 12.8 -8.3 +142.0 0.84 1.52 88 +9.0 | +5.3 +0.03 -22
|
||
b1_m5_selltrans_oos 86 +9.4 1.33 53.5 8.2 -1.4 +10.9 0.91 1.95 4 +1.5 | +0.0 +0.00 +0
|
||
b2_h1_nodiv_12mo 149 +28.6 1.62 62.4 6.5 +23.6 +5.0 1.93 1.24 49 +5.5 | +4.6 +0.07 +11
|
||
b2_h1_nodiv_5yr 567 +40.5 1.20 56.3 14.1 -5.2 +45.6 0.93 1.37 203 -18.2 | +8.1 +0.02 +52
|
||
b2_h1_nodiv_oos 119 +7.6 1.17 56.3 7.7 -1.9 +9.5 0.91 1.44 31 -14.3 | +2.4 +0.04 +15
|
||
b2_m15_nodiv_12mo 439 +81.3 1.51 60.8 13.1 +72.3 +8.9 1.90 1.11 147 +7.0 | +7.4 -0.02 +42
|
||
b2_m15_nodiv_5yr 1771 +139.9 1.20 55.8 47.4 -16.2 +156.1 0.94 1.39 578 +11.7 | +11.9 -0.01 +193
|
||
b2_m15_nodiv_oos 341 +52.3 1.43 59.2 17.1 +15.6 +36.7 1.23 1.68 93 -10.0 | +13.5 +0.07 +42
|
||
b3_h1_bb0_12mo 154 +31.2 1.64 60.4 9.4 +22.8 +8.4 1.89 1.36 52 +6.7 | +7.3 +0.09 +16
|
||
b3_h1_bb0_5yr 583 +37.3 1.18 54.5 18.3 -7.4 +44.7 0.90 1.35 206 -20.4 | +4.9 +0.00 +68
|
||
b3_h1_bb0_oos 122 +5.6 1.12 55.7 8.7 -2.0 +7.6 0.91 1.33 32 -13.8 | +0.4 -0.01 +18
|
||
b3_m15_bb0_12mo 439 +49.9 1.29 59.0 19.4 +61.6 -11.8 1.77 0.87 151 +0.3 | -24.0 -0.23 +42
|
||
b3_m15_bb0_5yr 1790 +126.6 1.18 56.1 42.3 -1.8 +128.5 0.99 1.31 592 +17.4 | -1.3 -0.03 +212
|
||
b3_m15_bb0_oos 345 +55.5 1.47 61.2 12.4 +18.2 +37.3 1.29 1.66 94 -8.4 | +16.7 +0.11 +46
|
||
b3_m5_bb0_12mo 621 +84.0 1.35 51.4 16.6 +49.3 +34.7 1.43 1.28 104 +9.7 | -20.4 -0.14 +46
|
||
b3_m5_bb0_5yr 987 +113.0 1.31 50.7 15.3 -7.7 +120.6 0.85 1.38 134 +12.0 | -15.5 -0.08 +76
|
||
b3_m5_bb0_oos 93 +10.6 1.35 54.8 5.6 -2.9 +13.5 0.85 2.17 9 +3.7 | +1.2 +0.01 +7
|
||
b12_h1_nosell_nodiv_12mo 100 +23.1 1.75 63.0 4.6 +21.8 +1.3 2.00 1.15 0 +0.0 | -0.8 +0.20 -38
|
||
b12_h1_nosell_nodiv_5yr 364 +58.7 1.52 59.6 12.9 -4.1 +62.8 0.90 1.87 0 +0.0 | +26.3 +0.34 -151
|
||
b12_h1_nosell_nodiv_oos 88 +21.9 1.92 63.6 4.8 +9.4 +12.5 1.93 1.92 0 +0.0 | +16.8 +0.79 -16
|
||
base_m5_12mo 575 +104.3 1.50 52.9 10.7 +50.8 +53.5 1.46 1.53 93 +6.9
|
||
ctl_h1_12mo 138 +24.0 1.55 60.9 8.0 +19.4 +4.6 1.84 1.22 47 +1.8
|
||
ctl_h1_5yr 515 +32.4 1.18 55.7 15.2 -5.9 +38.3 0.91 1.34 180 -16.7
|
||
ctl_h1_oos 104 +5.1 1.13 56.7 7.5 -0.9 +6.1 0.95 1.32 26 -11.5
|
||
ctl_m15_12mo 397 +73.9 1.53 62.2 10.8 +66.7 +7.2 1.95 1.10 138 +8.5
|
||
ctl_m15_5yr 1578 +128.0 1.21 56.5 35.8 -3.4 +131.4 0.99 1.37 535 +12.6
|
||
ctl_m15_oos 299 +38.8 1.36 59.2 15.5 +5.9 +32.9 1.10 1.66 86 -11.7
|
||
ctl_m5_5yr 911 +128.4 1.39 51.3 12.8 -8.3 +136.7 0.84 1.48 110 +3.8
|
||
ctl_m5_oos 86 +9.4 1.33 53.5 8.2 -1.4 +10.9 0.91 1.95 4 +1.5
|
||
```
|
||
|
||
## Decisions
|
||
|
||
- **SHIPPED** — M5 `xau_strategy.sell_blocked_regimes=["transitioning"]`
|
||
(b1_m5_selltrans): 12mo +7.9R, 5yr +5.3R, OOS unchanged (no such trades
|
||
in that window); positive in both 12mo halves on every window. H1
|
||
`xau_strategy.sell_min_quality=[]` (b1_h1_nosell, all H1 shorts blocked):
|
||
5yr +16.7R and OOS +11.5R more than pay for the 12mo cost of -1.8R; H1
|
||
shorts were -16.7R/5yr and -11.5R/OOS in the control. Both land in
|
||
`pair_signals/xauusd.py`.
|
||
- **REJECTED** — M15 `sell_min_quality=["B"]` (b1_m15_sellB): 5yr -8.9R
|
||
with max DD widening 35.8 -> 41.4R for a marginal +0.7R on 12mo; fails
|
||
the drawdown clause. All three B3 Bollinger-weight-0-in-trending-regimes
|
||
runs (b3_h1_bb0, b3_m15_bb0, b3_m5_bb0): negative on M5 (-20.4R/12mo,
|
||
-15.5R/5yr) and M15 (-24.0R/12mo, PF 1.29 -> still below the 12mo H2
|
||
going negative at -11.8R); H1's B3 5yr gain (+4.9R) doesn't clear the
|
||
bar once M5/M15 fail outright — Bollinger's constant-oppose vote stays
|
||
a load-bearing quality bar, consistent with the 2026-08-04 audit.
|
||
- **DECLINED** — B2-M15 (`technical.disabled_indicators=["divergence"]`
|
||
on M15, b2_m15_nodiv): R improves on every window (+7.4/+11.9/+13.5R)
|
||
but 5yr max DD widens 35.8 -> 47.4R while PF stays flat (1.21 -> 1.20) —
|
||
more risk for no risk-adjusted gain; owner call.
|
||
- **PARKED** — B2-H1 (`technical.disabled_indicators=["divergence"]` on
|
||
H1, b2_h1_nodiv, standalone: +4.6/+8.1/+2.4R, PF 1.62/1.20/1.17). The
|
||
combined run with the H1 no-short override (b12_h1_nosell_nodiv) passes
|
||
rule 1 and beats the no-shorts override alone on both 5yr and OOS: 12mo
|
||
100/+23.1R PF 1.75 (halves +21.8/+1.3), 5yr 364/+58.7R PF 1.52, OOS
|
||
88/+21.9R PF 1.92. Not shipped anyway: it would route through
|
||
`technical.disabled_indicators`, which CLAUDE.md rule 8 reserves for
|
||
experiments, not production paths. Owner decision; recommended vehicle
|
||
is an H1-only `regime_weights_override` with divergence weight 0 plus a
|
||
fresh validation run under that mechanism.
|