Harwinder Sekhon
c00ed4471b
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
3.5 KiB
Signal-Level Replay Parity — Oanda vs Tiingo (Phase 2)
Date: 2026-08-14
Tool: scripts/signal_parity_study.py (replays real analyze_from_dataframes, news off)
Window: 2025-08-14 → 2026-08-14 (12 months); M15 stride 8, M5 stride 24 → ~2,900 sampled decisions each
Preceding: Phase 1 price-diff study (feed_divergence_2026-08-14.md)
Headline
On the ~94–96% of bars the two feeds share, Oanda and Tiingo produce near-identical signals:
| Metric | M15 | M5 |
|---|---|---|
| same direction | 98.32% | 98.16% |
| same grade | 94.13% | 95.23% |
| same setup presence | 98.83% | 99.49% |
| same direction + grade + setup | 93.41% | 94.21% |
| opposite-sign flips (BUY↔SELL) | 0 | 0 |
| direction flips on actionable bars | 6.6% | 10.9% |
| grade changed on actionable bars | 14.2% | 7.8% |
| mean |Δ final_score| | 1.76 | 1.78 |
| mean |Δ primary_score| | 1.73 | 1.77 |
| same strategy tag | 100% | 100% |
Every direction flip is actionable↔NEUTRAL, never a sign reversal: the feeds never disagree on buy-vs-sell, only on whether a marginal bar crosses the action threshold. Flipping bars cluster at the Friday-close/Sunday-reopen boundary (UTC hours 22–23) and in the Asian hours (03–04).
Setup levels diverge even when the signal agrees
On bars where both feeds emit a setup with the same tag:
| M15 (n=119) | M5 (n=39) | |
|---|---|---|
| entry mean |Δ| | $0.12 | $0.10 |
| SL mean |Δ| | $1.83 | $2.79 |
| TP1 mean |Δ| | $2.47 | $1.52 |
| TP2 mean |Δ| | $3.74 | $2.46 |
| TP1 p95 |Δ| | $17.4 | $2.3 |
Entry price is the current close (≈ identical across feeds), but SL/TP are recomputed from ATR and recent swing extremes, which differ more — mean SL drift ~$2–3 (p95 up to $10–26 on M15), i.e. 20–40% of ATR. So even an identical signal places stops/targets at meaningfully different prices on a different feed.
Phase 2 pass/fail vs plan thresholds
| Criterion | Plan threshold | M15 / M5 | Verdict |
|---|---|---|---|
| identical direction+grade | ≥ 97% | 93.4% / 94.2% | marginal (below 97%, above 93%) |
| opposite-sign flips | (none specified) | 0 / 0 | PASS |
| flips clustered at marginal composites | no session/regime clustering | yes (boundary hours) | PASS |
The "identical direction+grade" metric lands slightly under the proposed 97% bar — but the shortfall is grade letters (A+ vs A vs B) changing on marginal composites, not direction. If the acceptance criterion is "no opposite-sign disagreement and ~98% direction identity", this passes comfortably.
Conclusion
Oanda ↔ Tiingo is signal-equivalent for direction (98% identical, zero sign flips) over 12 months. The residual deltas are:
- ~6% of decisions differ in grade letter on marginal composites (A+↔A↔B near thresholds).
- SL/TP levels drift ~$2–4 mean (up to ~$20 p95 on M15) even for identical signals — a real P&L difference at the trade level.
- Tiingo's extra Sunday bars (Phase 1) are gated by the aggregator's NY-anchored weekend filter, so they do not inject phantom signals, but they do slightly shift indicator history at Monday opens.
Net: switching Oanda→Tiingo changes a handful of marginal decisions and moves stop/target levels, but does not change what the system is doing (direction, strategy, setup presence). Oanda↔Twelve Data was not replayed — Phase 1 already showed 81% M5 bar-direction disagreement and a ~$3/$1 systematic gap, so a TD replay would trivially fail (that is the documented re-pin motivator).