chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
This commit is contained in:
Harwinder Sekhon
+1
-5
@@ -3,9 +3,6 @@
|
|||||||
.env.local
|
.env.local
|
||||||
config.json
|
config.json
|
||||||
|
|
||||||
# Oanda daemon state
|
|
||||||
oanda_state.json
|
|
||||||
|
|
||||||
# Editor / tooling
|
# Editor / tooling
|
||||||
.claude/
|
.claude/
|
||||||
.vscode/
|
.vscode/
|
||||||
@@ -17,6 +14,7 @@ __pycache__/
|
|||||||
*.egg-info/
|
*.egg-info/
|
||||||
.venv/
|
.venv/
|
||||||
venv/
|
venv/
|
||||||
|
.coverage
|
||||||
|
|
||||||
# OS
|
# OS
|
||||||
.DS_Store
|
.DS_Store
|
||||||
@@ -30,8 +28,6 @@ backtest/cache/
|
|||||||
backtest/*.csv
|
backtest/*.csv
|
||||||
backtest/research/*.parquet
|
backtest/research/*.parquet
|
||||||
backtest/research/duka_m5_boundary_spreads.csv
|
backtest/research/duka_m5_boundary_spreads.csv
|
||||||
backtest/trailgrid/
|
|
||||||
backtest/trailgrid5m/
|
|
||||||
backtest/__pycache__/
|
backtest/__pycache__/
|
||||||
.env.*
|
.env.*
|
||||||
logs/
|
logs/
|
||||||
|
|||||||
+4
-3
@@ -26,9 +26,10 @@ Two invariants, both load-bearing (CLAUDE.md rules 1 and 4):
|
|||||||
so replayed books can never see a panel field. Backtest parity holds
|
so replayed books can never see a panel field. Backtest parity holds
|
||||||
by construction.
|
by construction.
|
||||||
|
|
||||||
Cost model: every full panel spends 5 requests (4 personas + 1
|
Cost model: every full panel spends 4 requests (the personas; 5 with
|
||||||
synthesizer) against the shared DeepSeek daily cap (llm_client). Two
|
verdict_source="chair", which adds the synthesizer) against the shared
|
||||||
layers bound the spend on top of that cap:
|
DeepSeek daily cap (llm_client). Two layers bound the spend on top of
|
||||||
|
that cap:
|
||||||
|
|
||||||
- ``max_panels_per_day`` (config, default 20) — a UTC-day budget of
|
- ``max_panels_per_day`` (config, default 20) — a UTC-day budget of
|
||||||
panel *attempts*, mirroring the client's request cap.
|
panel *attempts*, mirroring the client's request cap.
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ analyze_signal_log.py — signal_log performance evaluator
|
|||||||
READ-ONLY analysis of logs/signals.db: how are the logged live signals
|
READ-ONLY analysis of logs/signals.db: how are the logged live signals
|
||||||
and the advisory LLM panel actually performing?
|
and the advisory LLM panel actually performing?
|
||||||
|
|
||||||
Three lenses:
|
Six lenses:
|
||||||
|
|
||||||
1. TRADE REPLAY — every logged row carrying a trade_setup is deduped
|
1. TRADE REPLAY — every logged row carrying a trade_setup is deduped
|
||||||
(telegram + dashboard log the same scan; a persisting signal re-fires
|
(telegram + dashboard log the same scan; a persisting signal re-fires
|
||||||
@@ -59,6 +59,7 @@ Usage:
|
|||||||
python analyze_signal_log.py --parity # surface parity only
|
python analyze_signal_log.py --parity # surface parity only
|
||||||
python analyze_signal_log.py --stored-candles # DB-only replay
|
python analyze_signal_log.py --stored-candles # DB-only replay
|
||||||
python analyze_signal_log.py --stored-candles --news # + news lens
|
python analyze_signal_log.py --stored-candles --news # + news lens
|
||||||
|
python analyze_signal_log.py --scan-outcomes # forward-move table
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
@@ -140,7 +141,8 @@ class SignalEvent:
|
|||||||
_COLS = ("id, logged_at, source, timeframe, signal, score, quality,"
|
_COLS = ("id, logged_at, source, timeframe, signal, score, quality,"
|
||||||
" price, market_regime, setup_json, panel_verdict,"
|
" price, market_regime, setup_json, panel_verdict,"
|
||||||
" panel_confidence, action")
|
" panel_confidence, action")
|
||||||
# Columns added by the 2026-09 observability work; absent on older DBs.
|
# Columns an older DB may lack (added by later schema versions); loaded
|
||||||
|
# only when PRAGMA table_info reports them.
|
||||||
_OPTIONAL_COLS = ("gate_reason", "actionable", "config_fingerprint",
|
_OPTIONAL_COLS = ("gate_reason", "actionable", "config_fingerprint",
|
||||||
"advisory_event_id", "advisory_outcome_status",
|
"advisory_event_id", "advisory_outcome_status",
|
||||||
"advisory_r_multiple", "scan_kind", "news_sentiment",
|
"advisory_r_multiple", "scan_kind", "news_sentiment",
|
||||||
|
|||||||
@@ -21,8 +21,6 @@ the config.json.example defaults, every produced PlatformConfig is
|
|||||||
byte-identical to the pure-code defaults. apply_pair_overrides() runs
|
byte-identical to the pure-code defaults. apply_pair_overrides() runs
|
||||||
in the pair factory BEFORE the validated per-timeframe deltas, so the
|
in the pair factory BEFORE the validated per-timeframe deltas, so the
|
||||||
bot, CLI and backtester all see the same overlay (live == backtest).
|
bot, CLI and backtester all see the same overlay (live == backtest).
|
||||||
|
|
||||||
Spec: docs/superpowers/specs/2026-07-21-centralized-config-design.md
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
|||||||
@@ -1,61 +1,21 @@
|
|||||||
Note down entery and exit Why we took trade?
|
Backtest reporting checklist — ForexBot2 (XAU/USD)
|
||||||
What were each indicator signal on that specific time when we took trade
|
==================================================
|
||||||
How many indicators agreed at trade time
|
Every backtest report must record:
|
||||||
Always backtest using engine.py and signal_aggregator.py
|
- The exact command line (window, --primary-tf, --spread, and any --set /
|
||||||
Dont take trades on weekends and Fridays
|
--quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
|
||||||
Keep TP small 1.3 Reward ratio
|
- Trades, total R, win rate, profit factor and max drawdown for the whole
|
||||||
At the end of test make table of results and print all values you observed.
|
window AND for both halves, at both cost conventions ($0.65 measured-Oanda
|
||||||
Trade context
|
headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
|
||||||
|
- Per trade (the exported CSV already carries these): entry/exit time and
|
||||||
Time of day and trading session (Asian/London/NY/overlap)
|
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
||||||
Day of week (track Mon/Tue/Wed/Thu performance separately)
|
indicator's signal / value / agreement at decision time.
|
||||||
Market regime at entry: trending, ranging, or choppy (use ATR or ADX)
|
- Breakdowns by session/hour, weekday, regime and volatility regime
|
||||||
Volatility level (ATR value or ATR percentile vs recent average)
|
(backtest/research/book_report.py) and, for any candidate change, the
|
||||||
Pair/instrument traded
|
removed/added/shifted trade diff against the control book
|
||||||
News proximity (high-impact news within 30 min before/after entry)
|
(backtest/research/book_diff.py).
|
||||||
|
- The md5 of the exported book whenever the run is a parity or regression check.
|
||||||
Trade execution details
|
Replays go through the real signal code (analyze_from_dataframes) — never a
|
||||||
|
re-implementation.
|
||||||
Spread at entry in pips
|
|
||||||
Slippage on entry and exit (planned vs actual fill)
|
|
||||||
Time in trade (minutes/hours from entry to exit)
|
|
||||||
Maximum Adverse Excursion (MAE): how far against you it went
|
|
||||||
Maximum Favorable Excursion (MFE): how far in profit before pullback
|
|
||||||
|
|
||||||
Risk metrics
|
|
||||||
|
|
||||||
Max consecutive losses (streak)
|
|
||||||
Max consecutive wins (streak)
|
|
||||||
Expectancy: (Win% x Avg Win) minus (Loss% x Avg Loss)
|
|
||||||
Expectancy per hour in trade
|
|
||||||
Recovery factor: Total R divided by Max DD
|
|
||||||
|
|
||||||
Indicator/signal analysis
|
|
||||||
|
|
||||||
Each indicator's standalone win rate when it agreed with the trade
|
|
||||||
Confluence level performance (2 vs 3 vs 4+ indicators agreeing)
|
|
||||||
Which indicator disagreed on losing trades (find the "canary")
|
|
||||||
|
|
||||||
Decision quality
|
|
||||||
|
|
||||||
Rate each setup A/B/C at entry, before knowing outcome
|
|
||||||
Verify A-grade setups actually win more often
|
|
||||||
Track system trades vs discretionary deviations separately
|
|
||||||
|
|
||||||
Process notes
|
|
||||||
|
|
||||||
Screenshot or chart snapshot at entry and exit
|
|
||||||
Emotional state (calm, FOMO, revenge, tired)
|
|
||||||
What would have happened if you held longer or exited sooner
|
|
||||||
|
|
||||||
Aggregate analysis at end of backtest
|
|
||||||
|
|
||||||
Equity curve, not just total R
|
|
||||||
Performance breakdown by session, day, volatility regime
|
|
||||||
Win rate over rolling 20-trade windows (check for edge decay)
|
|
||||||
|
|
||||||
Critical extra
|
|
||||||
"Do not trade" log: setups flagged but skipped, and what happened after
|
|
||||||
|
|
||||||
|
|
||||||
--- 2026-09-06: decision-time feature export ---
|
--- 2026-09-06: decision-time feature export ---
|
||||||
|
|||||||
@@ -95,7 +95,7 @@ Cost guardrails already in place: 5 requests/panel, `max_panels_per_day`
|
|||||||
injection (technical persona only), prompt-contract assertions,
|
injection (technical persona only), prompt-contract assertions,
|
||||||
panel row update, rowid hand-off + no-op paths). Existing attach tests
|
panel row update, rowid hand-off + no-op paths). Existing attach tests
|
||||||
stub `ap._dp` to stay off the network.
|
stub `ap._dp` to stay off the network.
|
||||||
- **Live dry test** (`panel_payload_drytest_2026-08-15.py`, fabricated
|
- **Live dry test** (`panel_payload_drytest_2026-08-15.py` — script removed 2026-09-06, raw replies kept in the `.jsonl` — fabricated
|
||||||
numbers only, 13 real DeepSeek requests total): **7/7 probes pass** —
|
numbers only, 13 real DeepSeek requests total): **7/7 probes pass** —
|
||||||
coherent-bullish → support@70; bearish-vs-BUY → oppose@85; genuinely
|
coherent-bullish → support@70; bearish-vs-BUY → oppose@85; genuinely
|
||||||
mixed → neutral@30; empty payload → neutral@20 (contract); no invented
|
mixed → neutral@30; empty payload → neutral@20 (contract); no invented
|
||||||
|
|||||||
@@ -164,7 +164,7 @@ def base_metrics(r: pd.Series, df: pd.DataFrame | None = None) -> dict:
|
|||||||
|
|
||||||
|
|
||||||
def _pf(r: pd.Series) -> float:
|
def _pf(r: pd.Series) -> float:
|
||||||
"""Profit factor, hour_analysis-compatible (loss side is r <= 0)."""
|
"""Profit factor (loss side is r <= 0)."""
|
||||||
pos = r[r > 0].sum()
|
pos = r[r > 0].sum()
|
||||||
neg = -r[r <= 0].sum()
|
neg = -r[r <= 0].sum()
|
||||||
return float(pos / max(neg, 1e-9))
|
return float(pos / max(neg, 1e-9))
|
||||||
|
|||||||
@@ -6,9 +6,8 @@ Usage:
|
|||||||
[--start YYYY-MM-DD --end YYYY-MM-DD] [--out DIR]
|
[--start YYYY-MM-DD --end YYYY-MM-DD] [--out DIR]
|
||||||
|
|
||||||
A "burst" is one bar whose range is >= K x the PRIOR bar's Wilder ATR(14) —
|
A "burst" is one bar whose range is >= K x the PRIOR bar's Wilder ATR(14) —
|
||||||
the same definition burst_profile.py uses (that module computes it inline
|
the same definition every burst study in this folder has used (spelled
|
||||||
inside burst_table() and exposes no reusable helper, so the formula is
|
out here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
||||||
replicated here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
|
||||||
TR.ewm(alpha=1/14, min_periods=14, adjust=False).mean(), ratio =
|
TR.ewm(alpha=1/14, min_periods=14, adjust=False).mean(), ratio =
|
||||||
(H-L) / ATR.shift(1)).
|
(H-L) / ATR.shift(1)).
|
||||||
|
|
||||||
@@ -60,7 +59,7 @@ def load_bars(parquet: str, start: str | None = None,
|
|||||||
|
|
||||||
|
|
||||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||||
"""Wilder ATR — identical to the inline formula in burst_profile.py."""
|
"""Wilder ATR (the definition every burst study in this folder uses)."""
|
||||||
h, l, c = df["High"], df["Low"], df["Close"]
|
h, l, c = df["High"], df["Low"], df["Close"]
|
||||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||||
axis=1).max(axis=1)
|
axis=1).max(axis=1)
|
||||||
|
|||||||
@@ -1,42 +0,0 @@
|
|||||||
"""Burst-candle frequency by UTC hour — the empirical "hedge fund hours".
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python backtest/research/burst_profile.py PARQUET_GLOB
|
|
||||||
|
|
||||||
Per UTC bar-close hour of the given bars: % of bars whose range is
|
|
||||||
>= K x the PRIOR bar's Wilder ATR(14), for K in {3,4,5}; mean bar
|
|
||||||
range; and mean signed drift over the 6 bars following a >=4x burst
|
|
||||||
(does price continue or snap back?). Analysis tooling only.
|
|
||||||
"""
|
|
||||||
import glob
|
|
||||||
import sys
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
|
|
||||||
def burst_table(parquet_glob: str, bar_minutes: int = 5):
|
|
||||||
files = sorted(glob.glob(parquet_glob))
|
|
||||||
if not files:
|
|
||||||
raise SystemExit(f"no parquet matches {parquet_glob}")
|
|
||||||
df = pd.concat([pd.read_parquet(f) for f in files]).sort_index()
|
|
||||||
df = df[~df.index.duplicated(keep="last")]
|
|
||||||
h, l, c = df["High"], df["Low"], df["Close"]
|
|
||||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
|
||||||
axis=1).max(axis=1)
|
|
||||||
atr = tr.ewm(alpha=1 / 14, min_periods=14, adjust=False).mean()
|
|
||||||
ratio = (h - l) / atr.shift(1)
|
|
||||||
close_hour = (df.index + pd.Timedelta(minutes=bar_minutes)).hour
|
|
||||||
fwd6 = c.shift(-6) - c # signed move over the 6 bars after this one
|
|
||||||
out = pd.DataFrame({
|
|
||||||
"bars": ratio.groupby(close_hour).size(),
|
|
||||||
**{f"pct_ge_{k}x": (ratio >= k).groupby(close_hour).mean() * 100
|
|
||||||
for k in (3, 4, 5)},
|
|
||||||
"mean_range": (h - l).groupby(close_hour).mean(),
|
|
||||||
"post4x_drift6": fwd6[ratio >= 4].groupby(close_hour[ratio >= 4]).mean(),
|
|
||||||
})
|
|
||||||
return out.round(3)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
print(burst_table(sys.argv[1],
|
|
||||||
int(sys.argv[2]) if len(sys.argv) > 2 else 5).to_string())
|
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
# Feed Divergence Study — Oanda vs Tiingo vs Twelve Data (Phase 1)
|
||||||
|
|
||||||
|
**Date:** 2026-08-14
|
||||||
|
**Test plan:** `docs/data_source_change_test_plan.md`
|
||||||
|
**Tool:** `scripts/feed_divergence_study.py` (fetch_range over 365d, parquet-cached)
|
||||||
|
**Window:** 2025-08-14 → 2026-08-14 (12 months); Oanda-vs-TD truncated at TD cache head (2026-07-23)
|
||||||
|
|
||||||
|
## Headline
|
||||||
|
|
||||||
|
Three mid-price feeds for the *same* XAU/USD instrument are **not** interchangeable:
|
||||||
|
|
||||||
|
| Pair | mean \|ΔClose\| | p95 \|ΔClose\| | direction agree (M5/M15/H1) | bias | extra bars |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| Oanda vs **Tiingo** | ~$0.13 | ~$0.29 | 97.7% / 98.3% / 98.5% | ~0 | Tiingo +3–5% (weekend) |
|
||||||
|
| Oanda vs **Twelve Data** | ~$3.0 | ~$11.9 | 81.0% / 88.7% / 92.7% | −$1.06 (TD high) | — |
|
||||||
|
|
||||||
|
- **Oanda ↔ Tiingo are near-identical** on shared bars (~1–2% of ATR on close), but Tiingo emits ~22 hours of Sunday bars/week that Oanda omits.
|
||||||
|
- **Oanda ↔ Twelve Data diverge massively** — ~$3 mean close gap (≈30% of M5 ATR, ~30% of M15 ATR), a systematic ~$1 bias (TD quoted ~$1 high), and 19% of M5 bars even disagree on bar *direction*. This is the documented "Oanda/TD divergence" that motivated re-pinning the books on Oanda bars.
|
||||||
|
|
||||||
|
## Oanda vs Tiingo — detailed
|
||||||
|
|
||||||
|
Both feeds: same bar grid, same head/tail timestamps. Tiingo simply has more bars (Sunday bars).
|
||||||
|
|
||||||
|
### Coverage (12 months)
|
||||||
|
|
||||||
|
| TF | Oanda bars | Tiingo bars | shared | Oanda-only | Tiingo-only |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| M5 | 70,901 | 73,241 | 70,807 (96.6%) | 94 (0.13%) | 2,434 (3.32%) |
|
||||||
|
| M15 | 23,640 | 24,922 | 23,615 (94.7%) | 25 (0.11%) | 1,307 (5.24%) |
|
||||||
|
| H1 | 5,912 | 6,246 | 5,909 (94.6%) | 3 (0.05%) | 337 (5.40%) |
|
||||||
|
| H4 | 1,597 | 1,657 | 1,597 (96.4%) | 0 | 60 (3.62%) |
|
||||||
|
|
||||||
|
- **Tiingo-only bars cluster exclusively on Sundays** (e.g. M15: 88/Sunday; H1: 22/Sunday; H4: 5/Sunday). Tiingo's FX endpoint resamples continuously and does **not** apply the FX weekend calendar — it emits ~22h of flat/stale Sunday bars that Oanda omits.
|
||||||
|
- Oanda-only bars cluster on holidays (2025-11-27 Thanksgiving, 2025-12-10, 2026-06-25) — a handful of bars, negligible.
|
||||||
|
|
||||||
|
### Per-bar OHLC diffs (Oanda − Tiingo, $)
|
||||||
|
|
||||||
|
| TF | Close mean | Close p95 | Close max | \|ΔClose\| / ATR14 (mean / p95) |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| M5 | $0.123 | $0.285 | $26.10 | 2.1% / 5.0% |
|
||||||
|
| M15 | $0.128 | $0.290 | $26.10 | 1.3% / 2.9% |
|
||||||
|
| H1 | $0.145 | $0.335 | $26.10 | 0.7% / 1.6% |
|
||||||
|
| H4 | $0.168 | $0.340 | $17.53 | 0.4% / 0.8% |
|
||||||
|
|
||||||
|
- Bias is ~zero on every TF (no systematic one-feed-high).
|
||||||
|
- Tail outliers (p99/max up to $26–88) occur during fast moves — the two feeds' bar extremes disagree on the exact tick, as expected.
|
||||||
|
- **Direction agreement is 97.7–98.4%**, return correlation 0.995–0.9997.
|
||||||
|
- Divergence is concentrated at the Friday-close / Sunday-reopen boundary (UTC hours 20–22; mean |ΔClose| ~2–4× the 08–19h baseline).
|
||||||
|
|
||||||
|
### Phase 1 pass/fail vs plan thresholds
|
||||||
|
|
||||||
|
| Criterion | Plan threshold | Result | Verdict |
|
||||||
|
|---|---|---|---|
|
||||||
|
| timestamp overlap | ≥ 99.5% | 94.6–96.6% | **FAIL** (Tiingo weekend bars) |
|
||||||
|
| Close p95 \|Δ\| | ≤ 0.01% of price | ~0.0067% | PASS |
|
||||||
|
| no systematic bias | mean Δ ≈ 0 | ~0 | PASS |
|
||||||
|
|
||||||
|
The overlap "failure" is a *coverage* difference, not a data-quality one: the shared bars agree tightly, but Tiingo's extra Sunday bars mean the feeds have a different bar *count*. The signal pipeline's own `_is_forex_weekend` filter (NY-anchored) gates those Sunday bars, so live impact depends on whether any code trusts the feed's raw bar set directly.
|
||||||
|
|
||||||
|
## Oanda vs Twelve Data — detailed (M5/M15/H1, cached TD parquets)
|
||||||
|
|
||||||
|
| TF | common | mean \|ΔClose\| | median \|ΔClose\| | p95 | max | bias (TD−Oanda) | dir agree |
|
||||||
|
|---|---|---|---|---|---|---|---|
|
||||||
|
| M5 | 66,246 | $2.999 | $1.118 | $11.934 | $101.85 | +1.05 | 80.96% |
|
||||||
|
| M15 | 22,090 | $2.951 | $1.051 | $11.848 | $101.85 | +1.08 | 88.71% |
|
||||||
|
| H1 | 5,526 | $2.902 | $0.985 | $11.908 | $70.94 | +1.06 | 92.67% |
|
||||||
|
|
||||||
|
- The ~$3 mean / ~$1 median / ~$12 p95 gap is **~23× larger** than the Oanda–Tiingo gap, and the ~$1 bias is systematic (Twelve Data quotes gold ~$1 high vs Oanda's true mid).
|
||||||
|
- M5 direction agreement of **81%** means one bar in five disagrees on direction — M5 signals on TD bars are effectively a *different instrument* than on Oanda bars.
|
||||||
|
- This matches the prior Oanda/TD M15 study of 2026-07-24 (`oanda_td_divergence.py`, removed 2026-09-06 as superseded by this report; mean |ΔClose| ~$3.3) and confirms the standing rule: **pinned Twelve-Data books are not comparable to Oanda live signals without a re-pin.**
|
||||||
|
|
||||||
|
## Conclusions for the data-source switch
|
||||||
|
|
||||||
|
1. **Oanda → Tiingo (or back) is signal-safe in magnitude** (price diffs ~1–2% of ATR, 98% direction agreement), *provided* the weekend-bar coverage difference is handled — Tiingo's Sunday bars must be gated by the aggregator's NY-anchored weekend filter, not by raw bar count.
|
||||||
|
2. **Oanda → Twelve Data is a recalibration, not a formality.** Phase 2/3 must quantify the actual signal/book delta; the $3/$1 bias alone guarantees a large fraction of marginal bars flip.
|
||||||
|
3. Re-pinning the books on Oanda bars (Phase 3) remains mandatory before any TD-pinned baseline is treated as a live expectation.
|
||||||
|
|
||||||
|
## Next steps
|
||||||
|
|
||||||
|
- Phase 2: signal-level replay parity (`analyze_from_dataframes` on both feeds) to turn price-diff stats into flip rates.
|
||||||
|
- Phase 3: backtest re-pin on Oanda bars at $0.30 and $0.65.
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
tf,a,b,a_bars,b_bars,common,a_only,b_only,dir_agree_pct,ret_corr,mean_abs_dclose,p95_abs_dclose,max_abs_dclose,dclose_pct_atr_mean
|
||||||
|
5min,oanda,tiingo,70901,73241,70807,94,2434,97.674000,0.994970,0.123070,0.285000,26.105000,2.147942
|
||||||
|
@@ -1,158 +0,0 @@
|
|||||||
"""Fetch XAU/USD M15 candles from the Oanda v20 REST API.
|
|
||||||
|
|
||||||
Mechanical data-fetching infrastructure only — no analysis here.
|
|
||||||
Paginates GET /v3/instruments/XAU_USD/candles from a start cursor
|
|
||||||
through "now" and writes the result to a parquet file.
|
|
||||||
|
|
||||||
Run from the repo root:
|
|
||||||
venv/bin/python backtest/research/fetch_oanda_m15.py
|
|
||||||
"""
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
from datetime import datetime, timedelta, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
import requests
|
|
||||||
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
|
||||||
|
|
||||||
from app_config import oanda_api_token, oanda_base_url # noqa: E402
|
|
||||||
|
|
||||||
INSTRUMENT = "XAU_USD"
|
|
||||||
GRANULARITY = "M15"
|
|
||||||
START = datetime(2025, 7, 1, 0, 0, 0, tzinfo=timezone.utc)
|
|
||||||
OUT_PATH = Path(__file__).parent / "oanda_xau_m15_20250701_now.parquet"
|
|
||||||
COUNT = 5000
|
|
||||||
REQUEST_SLEEP_S = 0.5
|
|
||||||
FRESHNESS_WINDOW = timedelta(minutes=30)
|
|
||||||
|
|
||||||
|
|
||||||
def _rfc3339(dt: datetime) -> str:
|
|
||||||
return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
|
||||||
|
|
||||||
|
|
||||||
def fetch_all_candles():
|
|
||||||
token = oanda_api_token()
|
|
||||||
base_url = oanda_base_url()
|
|
||||||
if not token:
|
|
||||||
raise RuntimeError(
|
|
||||||
"No Oanda API token resolved (env OANDA_API_TOKEN / "
|
|
||||||
"config.json oanda.api_token). Aborting.")
|
|
||||||
|
|
||||||
url = f"{base_url}/v3/instruments/{INSTRUMENT}/candles"
|
|
||||||
headers = {
|
|
||||||
"Authorization": f"Bearer {token}",
|
|
||||||
"Accept-Datetime-Format": "RFC3339",
|
|
||||||
}
|
|
||||||
|
|
||||||
cursor = START
|
|
||||||
include_first = True
|
|
||||||
all_rows = []
|
|
||||||
n_requests = 0
|
|
||||||
|
|
||||||
while True:
|
|
||||||
params = {
|
|
||||||
"granularity": GRANULARITY,
|
|
||||||
"price": "MBA",
|
|
||||||
"count": COUNT,
|
|
||||||
"dailyAlignment": 0,
|
|
||||||
"alignmentTimezone": "Etc/UTC",
|
|
||||||
"from": _rfc3339(cursor),
|
|
||||||
"includeFirst": "true" if include_first else "false",
|
|
||||||
}
|
|
||||||
|
|
||||||
resp = requests.get(url, headers=headers, params=params, timeout=30)
|
|
||||||
n_requests += 1
|
|
||||||
if resp.status_code != 200:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"Oanda API request failed: status={resp.status_code}\n"
|
|
||||||
f"URL: {resp.url}\n"
|
|
||||||
f"Response body: {resp.text}")
|
|
||||||
|
|
||||||
payload = resp.json()
|
|
||||||
candles = payload.get("candles", [])
|
|
||||||
|
|
||||||
for c in candles:
|
|
||||||
if not c.get("complete", False):
|
|
||||||
continue
|
|
||||||
mid = c["mid"]
|
|
||||||
bid = c["bid"]
|
|
||||||
ask = c["ask"]
|
|
||||||
all_rows.append({
|
|
||||||
"Datetime": pd.Timestamp(c["time"]).tz_convert("UTC")
|
|
||||||
if pd.Timestamp(c["time"]).tzinfo is not None
|
|
||||||
else pd.Timestamp(c["time"]).tz_localize("UTC"),
|
|
||||||
"Open": float(mid["o"]),
|
|
||||||
"High": float(mid["h"]),
|
|
||||||
"Low": float(mid["l"]),
|
|
||||||
"Close": float(mid["c"]),
|
|
||||||
"Volume": int(c["volume"]),
|
|
||||||
"bid_o": float(bid["o"]),
|
|
||||||
"bid_h": float(bid["h"]),
|
|
||||||
"bid_l": float(bid["l"]),
|
|
||||||
"bid_c": float(bid["c"]),
|
|
||||||
"ask_o": float(ask["o"]),
|
|
||||||
"ask_h": float(ask["h"]),
|
|
||||||
"ask_l": float(ask["l"]),
|
|
||||||
"ask_c": float(ask["c"]),
|
|
||||||
})
|
|
||||||
|
|
||||||
print(f" request {n_requests}: from={params['from']} "
|
|
||||||
f"got {len(candles)} candles (complete kept so far: "
|
|
||||||
f"{len(all_rows)})", file=sys.stderr)
|
|
||||||
|
|
||||||
if not candles:
|
|
||||||
break
|
|
||||||
|
|
||||||
last_time = pd.Timestamp(candles[-1]["time"])
|
|
||||||
if last_time.tzinfo is None:
|
|
||||||
last_time = last_time.tz_localize("UTC")
|
|
||||||
else:
|
|
||||||
last_time = last_time.tz_convert("UTC")
|
|
||||||
|
|
||||||
now = datetime.now(timezone.utc)
|
|
||||||
# Oanda's "count" includes the "from" boundary bar itself, so with
|
|
||||||
# includeFirst=false every full page comes back as count-1, not
|
|
||||||
# count (verified empirically: count=N, includeFirst=false always
|
|
||||||
# yields N-1 candles when a full page of data exists ahead of the
|
|
||||||
# cursor). Compare against the count actually expected for this
|
|
||||||
# request's includeFirst setting, not the raw requested count.
|
|
||||||
expected_full_page = COUNT if include_first else COUNT - 1
|
|
||||||
got_fewer_than_requested = len(candles) < expected_full_page
|
|
||||||
near_now = (now - last_time.to_pydatetime()) < FRESHNESS_WINDOW
|
|
||||||
|
|
||||||
if got_fewer_than_requested or near_now:
|
|
||||||
break
|
|
||||||
|
|
||||||
cursor = last_time.to_pydatetime()
|
|
||||||
include_first = False
|
|
||||||
time.sleep(REQUEST_SLEEP_S)
|
|
||||||
|
|
||||||
return all_rows, n_requests
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
print(f"Fetching {INSTRUMENT} {GRANULARITY} candles from "
|
|
||||||
f"{_rfc3339(START)} through now...", file=sys.stderr)
|
|
||||||
|
|
||||||
rows, n_requests = fetch_all_candles()
|
|
||||||
|
|
||||||
if not rows:
|
|
||||||
raise RuntimeError("No complete candles were fetched.")
|
|
||||||
|
|
||||||
df = pd.DataFrame(rows)
|
|
||||||
df = df.set_index("Datetime")
|
|
||||||
df = df.sort_index()
|
|
||||||
df = df[~df.index.duplicated(keep="last")]
|
|
||||||
|
|
||||||
df.to_parquet(OUT_PATH)
|
|
||||||
|
|
||||||
print(f"\nSaved {len(df)} rows to {OUT_PATH}", file=sys.stderr)
|
|
||||||
print(f"Requests made: {n_requests}", file=sys.stderr)
|
|
||||||
print(f"First timestamp: {df.index[0]}", file=sys.stderr)
|
|
||||||
print(f"Last timestamp: {df.index[-1]}", file=sys.stderr)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
"""Per-hour P&L breakdown of a backtest trade book.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python backtest/research/hour_analysis.py BOOK.csv BAR_MINUTES
|
|
||||||
|
|
||||||
Reports, per UTC hour (both entry hour and DECISION hour = entry_time
|
|
||||||
minus one bar, the clock blocked_hours_utc evaluates): trade count,
|
|
||||||
total R, mean R, win rate, profit factor. Also a per-(hour, direction)
|
|
||||||
table. Analysis tooling only — never imported by the live code.
|
|
||||||
"""
|
|
||||||
import sys
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
|
|
||||||
def hour_tables(book_csv: str, bar_minutes: int):
|
|
||||||
df = pd.read_csv(book_csv)
|
|
||||||
et = pd.to_datetime(df["entry_time"], utc=True, format="mixed")
|
|
||||||
df["entry_hour"] = et.dt.hour
|
|
||||||
df["decision_hour"] = (et - pd.Timedelta(minutes=bar_minutes)).dt.hour
|
|
||||||
|
|
||||||
def table(group_cols):
|
|
||||||
g = df.groupby(group_cols)["r_multiple"]
|
|
||||||
out = g.agg(count="count", total_r="sum", mean_r="mean",
|
|
||||||
wr=lambda r: (r > 0).mean() * 100)
|
|
||||||
out["pf"] = g.apply(
|
|
||||||
lambda r: r[r > 0].sum() / max(-r[r <= 0].sum(), 1e-9))
|
|
||||||
return out.round(2)
|
|
||||||
|
|
||||||
return (table("decision_hour"), table(["decision_hour", "direction"]),
|
|
||||||
table("entry_hour"))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
book, mins = sys.argv[1], int(sys.argv[2])
|
|
||||||
by_dec, by_dec_dir, by_ent = hour_tables(book, mins)
|
|
||||||
print(f"\n── {book} · by DECISION hour (bar-close clock) ──")
|
|
||||||
print(by_dec.to_string())
|
|
||||||
print(f"\n── by (decision hour, direction) ──")
|
|
||||||
print(by_dec_dir.to_string())
|
|
||||||
@@ -1,199 +0,0 @@
|
|||||||
"""
|
|
||||||
Oanda vs Twelve Data bar-divergence study — XAU/USD M15, 12 months.
|
|
||||||
====================================================================
|
|
||||||
Compares the mid-price OHLC bars the signal engine would see from each
|
|
||||||
provider over the same window. Purpose: quantify whether migrating the
|
|
||||||
signal feed from Twelve Data to Oanda is a formality (bars ~identical)
|
|
||||||
or a recalibration (bars differ at a scale that moves indicators).
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
--td Twelve Data parquet (raw bars from backtest/cache/)
|
|
||||||
--oanda Oanda parquet (from fetch_oanda_m5.py / fetch_oanda_m15.py)
|
|
||||||
--start / --end analysis window (UTC dates)
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
printed report
|
|
||||||
backtest/research/oanda_td_divergence_by_hour.csv
|
|
||||||
backtest/research/oanda_td_bar_diffs.parquet (per-bar diffs, common index)
|
|
||||||
"""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
RESEARCH_DIR = Path(__file__).parent
|
|
||||||
PIP = 0.10 # XAU pip in $
|
|
||||||
|
|
||||||
|
|
||||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
|
||||||
hl = df["High"] - df["Low"]
|
|
||||||
hc = (df["High"] - df["Close"].shift(1)).abs()
|
|
||||||
lc = (df["Low"] - df["Close"].shift(1)).abs()
|
|
||||||
tr = pd.concat([hl, hc, lc], axis=1).max(axis=1)
|
|
||||||
return tr.ewm(alpha=1.0 / period, adjust=False).mean()
|
|
||||||
|
|
||||||
|
|
||||||
def pct_stats(s: pd.Series) -> dict:
|
|
||||||
a = s.abs()
|
|
||||||
return {
|
|
||||||
"mean": a.mean(), "median": a.median(),
|
|
||||||
"p90": a.quantile(0.90), "p95": a.quantile(0.95),
|
|
||||||
"p99": a.quantile(0.99), "max": a.max(),
|
|
||||||
"bias(mean signed)": s.mean(),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def fmt_stats(name: str, s: pd.Series) -> str:
|
|
||||||
st = pct_stats(s)
|
|
||||||
return (f" {name:<6} mean ${st['mean']:.3f} med ${st['median']:.3f} "
|
|
||||||
f"p95 ${st['p95']:.3f} p99 ${st['p99']:.3f} max ${st['max']:.2f} "
|
|
||||||
f"bias {st['bias(mean signed)']:+.4f}")
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
ap = argparse.ArgumentParser()
|
|
||||||
ap.add_argument("--td", default=str(RESEARCH_DIR.parent / "cache" /
|
|
||||||
"XAU_USD_15min_20240522_20260723.parquet"))
|
|
||||||
ap.add_argument("--oanda", default=str(RESEARCH_DIR /
|
|
||||||
"oanda_xau_m15_20250701_now.parquet"))
|
|
||||||
ap.add_argument("--start", default="2025-07-24")
|
|
||||||
ap.add_argument("--end", default="2026-07-23")
|
|
||||||
args = ap.parse_args()
|
|
||||||
|
|
||||||
td = pd.read_parquet(args.td)
|
|
||||||
oa = pd.read_parquet(args.oanda)
|
|
||||||
|
|
||||||
start = pd.Timestamp(args.start, tz="UTC")
|
|
||||||
end = pd.Timestamp(args.end, tz="UTC") + pd.Timedelta(days=1)
|
|
||||||
start = max(start, td.index[0], oa.index[0])
|
|
||||||
end = min(end, td.index[-1], oa.index[-1])
|
|
||||||
td = td.loc[(td.index >= start) & (td.index <= end)]
|
|
||||||
oa = oa.loc[(oa.index >= start) & (oa.index <= end)]
|
|
||||||
|
|
||||||
print("=" * 78)
|
|
||||||
print("OANDA vs TWELVE DATA — XAU/USD M15 bar divergence")
|
|
||||||
print(f"Window: {start} -> {end} ({(end - start).days} days)")
|
|
||||||
print("=" * 78)
|
|
||||||
|
|
||||||
# ── Coverage ─────────────────────────────────────────────────────
|
|
||||||
common = td.index.intersection(oa.index)
|
|
||||||
td_only = td.index.difference(oa.index)
|
|
||||||
oa_only = oa.index.difference(td.index)
|
|
||||||
print(f"\nCOVERAGE")
|
|
||||||
print(f" Twelve Data bars : {len(td):>7,}")
|
|
||||||
print(f" Oanda bars : {len(oa):>7,}")
|
|
||||||
print(f" common : {len(common):>7,}")
|
|
||||||
print(f" TD-only : {len(td_only):>7,} "
|
|
||||||
f"({100 * len(td_only) / max(len(td), 1):.2f}% of TD)")
|
|
||||||
print(f" Oanda-only : {len(oa_only):>7,} "
|
|
||||||
f"({100 * len(oa_only) / max(len(oa), 1):.2f}% of Oanda)")
|
|
||||||
for label, idx in (("TD-only", td_only), ("Oanda-only", oa_only)):
|
|
||||||
if len(idx):
|
|
||||||
by_date = pd.Series(1, index=idx).groupby(idx.date).sum()
|
|
||||||
top = by_date.sort_values(ascending=False).head(8)
|
|
||||||
print(f" {label} bars cluster on: "
|
|
||||||
+ ", ".join(f"{d} ({n})" for d, n in top.items()))
|
|
||||||
by_hour = pd.Series(1, index=idx).groupby(idx.hour).sum()
|
|
||||||
top_h = by_hour.sort_values(ascending=False).head(6)
|
|
||||||
print(f" {label} by UTC hour: "
|
|
||||||
+ ", ".join(f"{h:02d}h ({n})" for h, n in top_h.items()))
|
|
||||||
|
|
||||||
t = td.loc[common]
|
|
||||||
o = oa.loc[common]
|
|
||||||
|
|
||||||
# ── Scale references ─────────────────────────────────────────────
|
|
||||||
atr = wilder_atr(t).dropna()
|
|
||||||
mean_price = t["Close"].mean()
|
|
||||||
print(f"\nSCALE REFERENCES (Twelve Data feed)")
|
|
||||||
print(f" mean close : ${mean_price:,.2f}")
|
|
||||||
print(f" mean bar range : ${(t['High'] - t['Low']).mean():.3f}")
|
|
||||||
print(f" mean ATR14 : ${atr.mean():.3f} median ${atr.median():.3f}")
|
|
||||||
|
|
||||||
# ── Per-field diffs (TD − Oanda mid) ─────────────────────────────
|
|
||||||
print(f"\nPER-BAR DIFFS (TD − Oanda, $; pip = $0.10)")
|
|
||||||
diffs = {}
|
|
||||||
for col in ("Open", "High", "Low", "Close"):
|
|
||||||
d = t[col] - o[col]
|
|
||||||
diffs[col] = d
|
|
||||||
print(fmt_stats(col, d))
|
|
||||||
|
|
||||||
dclose = diffs["Close"].abs()
|
|
||||||
print(f"\n |ΔClose| as % of mean ATR14 : "
|
|
||||||
f"mean {100 * dclose.mean() / atr.mean():.1f}% "
|
|
||||||
f"p95 {100 * dclose.quantile(0.95) / atr.mean():.1f}%")
|
|
||||||
|
|
||||||
print(f"\nTHRESHOLD EXCEEDANCE (|ΔClose|)")
|
|
||||||
for thr in (0.10, 0.30, 0.65, 1.00, 2.00, 5.00):
|
|
||||||
pct = 100 * (dclose > thr).mean()
|
|
||||||
print(f" > ${thr:<5.2f}: {pct:6.2f}% of bars ({(dclose > thr).sum():,})")
|
|
||||||
|
|
||||||
# ── Direction / return agreement ─────────────────────────────────
|
|
||||||
t_ret = t["Close"].pct_change()
|
|
||||||
o_ret = o["Close"].pct_change()
|
|
||||||
# only consecutive common bars (gap = one M15 step)
|
|
||||||
step_ok = pd.Series(common, index=common).diff().eq(pd.Timedelta(minutes=15))
|
|
||||||
tr, orr = t_ret[step_ok], o_ret[step_ok]
|
|
||||||
corr = tr.corr(orr)
|
|
||||||
t_dir = np.sign(t["Close"] - t["Open"])
|
|
||||||
o_dir = np.sign(o["Close"] - o["Open"])
|
|
||||||
both_nonflat = (t_dir != 0) & (o_dir != 0)
|
|
||||||
dir_agree = 100 * (t_dir[both_nonflat] == o_dir[both_nonflat]).mean()
|
|
||||||
print(f"\nAGREEMENT")
|
|
||||||
print(f" close-return correlation (consecutive bars): {corr:.5f} "
|
|
||||||
f"(n={step_ok.sum():,})")
|
|
||||||
print(f" bar direction (sign of close−open) agree : {dir_agree:.2f}%")
|
|
||||||
|
|
||||||
# ── By-hour breakdown ────────────────────────────────────────────
|
|
||||||
hours = common.hour
|
|
||||||
rows = []
|
|
||||||
for h in range(24):
|
|
||||||
m = hours == h
|
|
||||||
if not m.any():
|
|
||||||
continue
|
|
||||||
row = {"hour_utc": h, "bars": int(m.sum()),
|
|
||||||
"mean_abs_dclose": dclose[m].mean(),
|
|
||||||
"p95_abs_dclose": dclose[m].quantile(0.95),
|
|
||||||
"mean_abs_dhigh": diffs["High"][m].abs().mean(),
|
|
||||||
"mean_abs_dlow": diffs["Low"][m].abs().mean()}
|
|
||||||
if "bid_c" in o.columns:
|
|
||||||
row["oanda_median_spread"] = (
|
|
||||||
(o["ask_c"] - o["bid_c"])[m].median())
|
|
||||||
rows.append(row)
|
|
||||||
by_hour = pd.DataFrame(rows)
|
|
||||||
out_hour = RESEARCH_DIR / "oanda_td_divergence_by_hour.csv"
|
|
||||||
by_hour.to_csv(out_hour, index=False, float_format="%.4f")
|
|
||||||
print(f"\nBY UTC HOUR (written to {out_hour.name})")
|
|
||||||
print(by_hour.to_string(index=False,
|
|
||||||
float_format=lambda x: f"{x:.3f}"))
|
|
||||||
|
|
||||||
# ── By-month drift ───────────────────────────────────────────────
|
|
||||||
print(f"\nBY MONTH (mean |ΔClose|, $)")
|
|
||||||
by_month = dclose.groupby(common.to_period("M")).mean()
|
|
||||||
for p, v in by_month.items():
|
|
||||||
print(f" {p}: {v:.3f}")
|
|
||||||
|
|
||||||
# ── Oanda real spread (bonus, ties to Dukascopy study) ───────────
|
|
||||||
if "bid_c" in o.columns:
|
|
||||||
spread = o["ask_c"] - o["bid_c"]
|
|
||||||
liquid = ~o.index.hour.isin([21, 22, 23, 0])
|
|
||||||
print(f"\nOANDA REAL SPREAD (close bid/ask, $)")
|
|
||||||
print(f" all bars : median {spread.median():.3f} "
|
|
||||||
f"p25 {spread.quantile(0.25):.3f} p75 {spread.quantile(0.75):.3f}")
|
|
||||||
print(f" liquid hours : median {spread[liquid].median():.3f}")
|
|
||||||
print(f" 21-00 UTC : median {spread[~liquid].median():.3f}")
|
|
||||||
|
|
||||||
# ── Persist per-bar diffs ────────────────────────────────────────
|
|
||||||
out = pd.DataFrame({
|
|
||||||
"td_close": t["Close"], "oa_close": o["Close"],
|
|
||||||
"d_open": diffs["Open"], "d_high": diffs["High"],
|
|
||||||
"d_low": diffs["Low"], "d_close": diffs["Close"],
|
|
||||||
})
|
|
||||||
out_path = RESEARCH_DIR / "oanda_td_bar_diffs.parquet"
|
|
||||||
out.to_parquet(out_path)
|
|
||||||
print(f"\nPer-bar diffs written to {out_path}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
hour_utc,bars,mean_abs_dclose,p95_abs_dclose,mean_abs_dhigh,mean_abs_dlow,oanda_median_spread
|
|
||||||
0,1029,3.3483,13.9460,3.2906,3.4310,0.7900
|
|
||||||
1,1028,3.3483,12.8292,3.3477,3.6398,0.7700
|
|
||||||
2,1028,3.0288,10.8960,2.9469,3.1317,0.7700
|
|
||||||
3,1028,2.8783,11.1300,2.8419,2.9992,0.7600
|
|
||||||
4,1028,2.8232,10.4710,2.8165,2.8455,0.7400
|
|
||||||
5,1028,2.9268,10.6027,2.9234,3.0038,0.7400
|
|
||||||
6,1028,3.0466,10.9265,2.9487,3.0796,0.7200
|
|
||||||
7,1028,3.0286,11.3512,3.0301,3.0797,0.6700
|
|
||||||
8,1028,3.0223,11.1483,2.8989,3.0801,0.6600
|
|
||||||
9,1028,3.0460,11.6345,2.9307,3.0609,0.6600
|
|
||||||
10,1028,2.9280,10.9015,2.8647,3.0029,0.6600
|
|
||||||
11,1028,3.0340,11.4712,2.9591,2.9472,0.6600
|
|
||||||
12,1028,3.0986,11.3282,2.9412,3.1407,0.7000
|
|
||||||
13,1028,3.4014,12.3343,3.1077,3.4228,0.7300
|
|
||||||
14,1028,3.5074,11.3628,3.3960,3.6569,0.6900
|
|
||||||
15,1028,3.4514,11.8982,3.4124,3.6733,0.6700
|
|
||||||
16,1028,3.1275,12.5155,3.2038,3.3061,0.6700
|
|
||||||
17,1020,2.9826,12.6435,3.0064,3.1008,0.6900
|
|
||||||
18,1015,2.8721,12.5480,2.9101,2.9996,0.6900
|
|
||||||
19,1004,2.8637,12.1335,2.8824,2.9454,0.6900
|
|
||||||
20,996,2.8006,11.9325,2.8486,2.8577,0.7600
|
|
||||||
21,335,1.1085,4.8600,1.1212,1.2034,0.9400
|
|
||||||
22,676,4.0827,13.6787,4.8422,4.6761,0.8300
|
|
||||||
23,1027,3.2722,12.5725,3.5617,3.7747,0.8700
|
|
||||||
|
@@ -1,271 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
panel_payload_drytest_2026-08-15.py — live DeepSeek validation of the
|
|
||||||
enriched agent-panel payloads + hardened prompts (2026-08-15).
|
|
||||||
|
|
||||||
Spends ~6 REAL DeepSeek requests (against the shared 500/day cap) with
|
|
||||||
FABRICATED payloads — made-up prices/indicators, no real market data —
|
|
||||||
to verify the new technical-persona contract end to end:
|
|
||||||
|
|
||||||
P1 coherent bullish evidence -> technical persona SUPPORTS
|
|
||||||
P2 bearish evidence vs BUY brief -> technical persona OPPOSES
|
|
||||||
P3 conflicting evidence -> NEUTRAL or low confidence
|
|
||||||
P4 empty evidence -> NEUTRAL with confidence < 30
|
|
||||||
(the anti-hallucination contract)
|
|
||||||
P5 hallucination trap -> note invents no price levels
|
|
||||||
P6 output discipline -> <=20-word notes, exact JSON keys
|
|
||||||
P7 synthesizer over 4 supports -> AGREE verdict
|
|
||||||
|
|
||||||
Usage: venv/bin/python backtest/research/panel_payload_drytest_2026-08-15.py
|
|
||||||
Exit code 0 = all probes passed. Raw replies land in
|
|
||||||
backtest/research/panel_drytest_2026-08-15.jsonl
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
|
||||||
sys.path.insert(0, str(REPO_ROOT))
|
|
||||||
|
|
||||||
import agent_panel as ap # noqa: E402
|
|
||||||
from app_config import deepseek_api_key, get_app_config # noqa: E402
|
|
||||||
from llm_client import get_llm_client # noqa: E402
|
|
||||||
|
|
||||||
ARTIFACT = Path(__file__).with_suffix(".jsonl")
|
|
||||||
|
|
||||||
|
|
||||||
# ── Fabrication helpers (all numbers are invented) ───────────────────
|
|
||||||
|
|
||||||
def _staircase(start: float, step: float, n: int) -> pd.DataFrame:
|
|
||||||
"""n synthetic bars drifting `step` per bar; last close = start+n*step."""
|
|
||||||
rows, price = [], start
|
|
||||||
for _ in range(n):
|
|
||||||
o, c = price, price + step
|
|
||||||
rows.append({"Open": o, "Close": c,
|
|
||||||
"High": max(o, c) + abs(step) * 0.4,
|
|
||||||
"Low": min(o, c) - abs(step) * 0.4})
|
|
||||||
price = c
|
|
||||||
return pd.DataFrame(rows)
|
|
||||||
|
|
||||||
|
|
||||||
def _choppy(center: float, n: int) -> pd.DataFrame:
|
|
||||||
rows, price, sign = [], center, 1
|
|
||||||
for _ in range(n):
|
|
||||||
step = sign * 1.5
|
|
||||||
o, c = price, price + step
|
|
||||||
rows.append({"Open": o, "Close": c,
|
|
||||||
"High": max(o, c) + 0.6, "Low": min(o, c) - 0.6})
|
|
||||||
price, sign = c, -sign
|
|
||||||
return pd.DataFrame(rows)
|
|
||||||
|
|
||||||
|
|
||||||
_FAKE_PRICE = 2000.00 # deliberately made-up
|
|
||||||
_FAKE_ATR = 5.0
|
|
||||||
|
|
||||||
_BRIEF_BUY = {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY",
|
|
||||||
"quality": "A", "score": 38.5, "confidence": 62,
|
|
||||||
"price": _FAKE_PRICE, "market_regime": "trending_up",
|
|
||||||
"volatility_regime": "normal", "mtf_agreement": True}
|
|
||||||
|
|
||||||
_BULL_INDICATORS = {
|
|
||||||
"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0},
|
|
||||||
"rsi": {"signal": "BUY", "confidence": 65, "value": 58.0},
|
|
||||||
"macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25},
|
|
||||||
"bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7},
|
|
||||||
"adx": {"signal": "BUY", "confidence": 70, "value": 32.0},
|
|
||||||
"roc": {"signal": "BUY", "confidence": 55, "value": 0.35},
|
|
||||||
"keltner": {"signal": "BUY", "confidence": 65, "value": 3.1},
|
|
||||||
"candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0},
|
|
||||||
}
|
|
||||||
|
|
||||||
_BEAR_INDICATORS = {
|
|
||||||
"ema_trend": {"signal": "STRONG_SELL", "confidence": 85, "value": 0.1},
|
|
||||||
"rsi": {"signal": "SELL", "confidence": 70, "value": 74.0},
|
|
||||||
"macd": {"signal": "STRONG_SELL", "confidence": 80, "value": -1.4},
|
|
||||||
"bollinger": {"signal": "SELL", "confidence": 60, "value": 0.95},
|
|
||||||
"adx": {"signal": "SELL", "confidence": 70, "value": 35.0},
|
|
||||||
"roc": {"signal": "SELL", "confidence": 60, "value": -0.5},
|
|
||||||
"keltner": {"signal": "SELL", "confidence": 65, "value": -2.8},
|
|
||||||
"candlestick": {"signal": "SELL", "confidence": 55, "value": -2.0},
|
|
||||||
}
|
|
||||||
|
|
||||||
_MIXED_INDICATORS = {
|
|
||||||
"ema_trend": {"signal": "BUY", "confidence": 55, "value": 0.6},
|
|
||||||
"rsi": {"signal": "NEUTRAL", "confidence": 40, "value": 51.0},
|
|
||||||
"macd": {"signal": "SELL", "confidence": 55, "value": -0.3},
|
|
||||||
"bollinger": {"signal": "NEUTRAL", "confidence": 35, "value": 0.5},
|
|
||||||
"adx": {"signal": "NEUTRAL", "confidence": 40, "value": 18.0},
|
|
||||||
"roc": {"signal": "SELL", "confidence": 50, "value": -0.1},
|
|
||||||
"keltner": {"signal": "BUY", "confidence": 50, "value": 0.8},
|
|
||||||
"candlestick": {"signal": "NEUTRAL", "confidence": 30, "value": 0.0},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _tech_payload(indicators, mtf, candles_df=None, key_levels=None,
|
|
||||||
brief=None, exhaustion=0):
|
|
||||||
payload = {"brief": brief or dict(_BRIEF_BUY),
|
|
||||||
"indicators": indicators, "mtf": mtf,
|
|
||||||
"key_levels": key_levels or {}, "exhaustion": exhaustion}
|
|
||||||
if candles_df is not None:
|
|
||||||
# the REAL production normalizer, not a hand-rolled copy
|
|
||||||
payload["candles"] = {
|
|
||||||
"15min": ap._normalize_candles(candles_df, _FAKE_ATR,
|
|
||||||
len(candles_df))}
|
|
||||||
return payload
|
|
||||||
|
|
||||||
|
|
||||||
# ── Probe runner ─────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_RESULTS = []
|
|
||||||
|
|
||||||
|
|
||||||
def _record(probe, payload, reply, take, passed, detail):
|
|
||||||
_RESULTS.append({"probe": probe, "pass": bool(passed),
|
|
||||||
"detail": detail,
|
|
||||||
"stance": getattr(take, "stance", None),
|
|
||||||
"confidence": getattr(take, "confidence", None),
|
|
||||||
"note": getattr(take, "note", None)})
|
|
||||||
with ARTIFACT.open("a") as fh:
|
|
||||||
fh.write(json.dumps({"probe": probe, "payload": payload,
|
|
||||||
"raw_reply": reply}, default=str) + "\n")
|
|
||||||
mark = "PASS" if passed else "FAIL"
|
|
||||||
print(f"[{mark}] {probe}: {detail}")
|
|
||||||
if take is not None:
|
|
||||||
print(f" -> {take.stance} @ {take.confidence:.0f} "
|
|
||||||
f"| {take.note}")
|
|
||||||
|
|
||||||
|
|
||||||
def _ask(payload):
|
|
||||||
"""Real API call through the production client + production
|
|
||||||
validation (the same path run_panel uses)."""
|
|
||||||
reply = get_llm_client().chat_json(ap._TECH_SYSTEM, payload)
|
|
||||||
return reply, ap._validate_take("technical", reply)
|
|
||||||
|
|
||||||
|
|
||||||
def _want(probe: str) -> bool:
|
|
||||||
"""--only P3 runs just that probe (saves requests on re-runs)."""
|
|
||||||
if "--only" not in sys.argv:
|
|
||||||
return True
|
|
||||||
return sys.argv[sys.argv.index("--only") + 1] == probe
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
app = get_app_config()
|
|
||||||
if not deepseek_api_key(app):
|
|
||||||
print("ABORT: no DeepSeek API key (env DEEPSEEK_API_KEY / "
|
|
||||||
"config.json deepseek.api_key).")
|
|
||||||
return 2
|
|
||||||
if app.deepseek_settings.get("enabled", True) is False:
|
|
||||||
print("ABORT: deepseek.enabled is false in config.json.")
|
|
||||||
return 2
|
|
||||||
if ARTIFACT.exists() and "--only" not in sys.argv:
|
|
||||||
ARTIFACT.unlink() # full run = fresh wire log; --only appends
|
|
||||||
|
|
||||||
mtf_agree = {"5min": {"signal": "BUY", "score": 30.0},
|
|
||||||
"1h": {"signal": "STRONG_BUY", "score": 45.0}}
|
|
||||||
levels = {"resistance": 2012.0, "support": 1985.0}
|
|
||||||
|
|
||||||
# P1 — coherent bullish, with rising ATR staircase (last close 2000)
|
|
||||||
if _want("P1"):
|
|
||||||
p1 = _tech_payload(_BULL_INDICATORS, mtf_agree,
|
|
||||||
candles_df=_staircase(_FAKE_PRICE - 20 * 0.6,
|
|
||||||
0.6, 20),
|
|
||||||
key_levels=levels)
|
|
||||||
reply, take = _ask(p1)
|
|
||||||
_record("P1 coherent-bullish", p1, reply, take,
|
|
||||||
take is not None and take.stance == "support"
|
|
||||||
and take.confidence >= 50,
|
|
||||||
"expect support @ conf>=50")
|
|
||||||
|
|
||||||
# P2 — bearish everything against a BUY brief, falling staircase
|
|
||||||
if _want("P2"):
|
|
||||||
brief2 = dict(_BRIEF_BUY, market_regime="trending_down",
|
|
||||||
mtf_agreement=False)
|
|
||||||
mtf_sell = {"5min": {"signal": "SELL", "score": -28.0},
|
|
||||||
"1h": {"signal": "STRONG_SELL", "score": -41.0}}
|
|
||||||
p2 = _tech_payload(_BEAR_INDICATORS, mtf_sell, brief=brief2,
|
|
||||||
candles_df=_staircase(_FAKE_PRICE + 20 * 0.6,
|
|
||||||
-0.6, 20),
|
|
||||||
key_levels=levels, exhaustion=3)
|
|
||||||
reply, take = _ask(p2)
|
|
||||||
_record("P2 bearish-vs-BUY-brief", p2, reply, take,
|
|
||||||
take is not None and take.stance == "oppose",
|
|
||||||
"expect oppose (BUY brief fights all evidence)")
|
|
||||||
|
|
||||||
# P3 — genuinely mixed evidence: MTF neutral both ways, ranging
|
|
||||||
# regime, choppy candles. (An H1 SELL against a BUY is a *listed
|
|
||||||
# conflict* in the prompt — oppose is correct there, not mixed.)
|
|
||||||
if _want("P3"):
|
|
||||||
brief3 = dict(_BRIEF_BUY, market_regime="ranging", score=12.0,
|
|
||||||
confidence=35, mtf_agreement=False)
|
|
||||||
mtf_mixed = {"5min": {"signal": "NEUTRAL", "score": 4.0},
|
|
||||||
"1h": {"signal": "NEUTRAL", "score": -3.0}}
|
|
||||||
p3 = _tech_payload(_MIXED_INDICATORS, mtf_mixed, brief=brief3,
|
|
||||||
candles_df=_choppy(_FAKE_PRICE, 20))
|
|
||||||
reply, take = _ask(p3)
|
|
||||||
_record("P3 conflicting", p3, reply, take,
|
|
||||||
take is not None and (take.stance == "neutral"
|
|
||||||
or take.confidence < 60),
|
|
||||||
"expect neutral or conf<60 on genuinely mixed evidence")
|
|
||||||
|
|
||||||
# P4 — empty evidence: contract demands neutral @ conf<30
|
|
||||||
if _want("P4"):
|
|
||||||
p4 = _tech_payload({}, {}, key_levels=None)
|
|
||||||
reply, take = _ask(p4)
|
|
||||||
_record("P4 empty-evidence", p4, reply, take,
|
|
||||||
take is not None and take.stance == "neutral"
|
|
||||||
and take.confidence < 30,
|
|
||||||
"contract: thin/missing evidence -> neutral, conf<30")
|
|
||||||
|
|
||||||
# P5 — hallucination trap: no key_levels, no candles; the note must
|
|
||||||
# not cite any 3+-digit number absent from the payload
|
|
||||||
if _want("P5"):
|
|
||||||
p5 = _tech_payload(_BULL_INDICATORS, mtf_agree, key_levels=None)
|
|
||||||
reply, take = _ask(p5)
|
|
||||||
invented = []
|
|
||||||
if take is not None:
|
|
||||||
payload_blob = json.dumps(p5)
|
|
||||||
for num in re.findall(r"\d{3,}", take.note):
|
|
||||||
if num not in payload_blob:
|
|
||||||
invented.append(num)
|
|
||||||
_record("P5 hallucination-trap", p5, reply, take,
|
|
||||||
take is not None and not invented,
|
|
||||||
f"invented numbers in note: {invented or 'none'}")
|
|
||||||
|
|
||||||
# P6 — output discipline across the persona replies gathered above
|
|
||||||
notes = [r for r in _RESULTS if r["note"]]
|
|
||||||
wordy = [r["probe"] for r in notes if len(r["note"].split()) > 20]
|
|
||||||
_record("P6 output-discipline", {}, None, None,
|
|
||||||
notes and not wordy,
|
|
||||||
f"notes >20 words: {wordy or 'none'} "
|
|
||||||
f"(cap is 15; 20 = fail threshold)")
|
|
||||||
|
|
||||||
# P7 — synthesizer over four fabricated SUPPORT takes
|
|
||||||
if _want("P7"):
|
|
||||||
takes = [{"persona": p, "stance": "support", "confidence": c,
|
|
||||||
"note": "evidence aligned"}
|
|
||||||
for p, c in (("macro", 70), ("technical", 80),
|
|
||||||
("risk", 65), ("contrarian", 60))]
|
|
||||||
synth_payload = {"direction": "BUY", "takes": takes}
|
|
||||||
reply = get_llm_client().chat_json(ap._SYNTH_SYSTEM, synth_payload)
|
|
||||||
verdict = reply.get("verdict") if isinstance(reply, dict) else None
|
|
||||||
rationale = (reply or {}).get("rationale", "")
|
|
||||||
_record("P7 synthesizer", synth_payload, reply, None,
|
|
||||||
verdict == "AGREE" and len(str(rationale).split()) <= 40,
|
|
||||||
f"expect AGREE + <=40-word rationale; got {verdict!r} "
|
|
||||||
f"({len(str(rationale).split())} words)")
|
|
||||||
|
|
||||||
n_pass = sum(1 for r in _RESULTS if r["pass"])
|
|
||||||
n_api = sum(1 for r in _RESULTS if r["probe"] != "P6 output-discipline")
|
|
||||||
print(f"\n{n_pass}/{len(_RESULTS)} probes passed "
|
|
||||||
f"({n_api} API requests spent; "
|
|
||||||
f"raw wire log: {ARTIFACT.name})")
|
|
||||||
return 0 if n_pass == len(_RESULTS) else 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# Backtest Re-pin on Oanda bars — verification (Phase 3)
|
||||||
|
|
||||||
|
**Date:** 2026-08-14 (verification; original books were generated 2026-08-04, current-HEAD M5 canonical book exported 2026-08-14)
|
||||||
|
**Scope:** confirm the pinned Oanda books exist, satisfy rule 1, and quantify the Oanda↔TD delta that motivated them.
|
||||||
|
|
||||||
|
## Re-pin state
|
||||||
|
|
||||||
|
The re-pin is **already done** — full Oanda-bar books exist in `backtest/oanda_*.csv` for M5/M15/H1/H4, at both cost assumptions, with in-sample (12mo), out-of-sample (2024-06 → 2025-03), and 5-year variants. The shipped configs are `m5` (pullback OFF — `_nopb`) and `m15`.
|
||||||
|
|
||||||
|
### M5 canonical-reference reconciliation (2026-08-14)
|
||||||
|
|
||||||
|
The dated current-HEAD export `backtest/oanda_m5_12mo_065_nopb_20260814.csv` is the canonical M5 12-month reference going forward: **577 trades, +106.82R, PF 1.51**, with both 12-month halves positive (+50.80R / +56.02R; PF 1.46 / 1.55). It covers 2025-08-14 → 2026-08-14, runs the shipped M5 factory (XAU_PB off), charges $0.65 round trip, and uses next-open/raw-path/close-only-trail realism unchanged.
|
||||||
|
|
||||||
|
Earlier M5 snapshots (`oanda_m5_12mo_065_nopb.csv`: 586 / +98.11R / 1.45; `m5_12mo_shipped.csv`: 588 / +96.60R / 1.44; and the 555 / +98.64R ledger entry) remain historical evidence from different HEADs around the weekend-calendar and related QA work. They must not be cited as the current-HEAD reference.
|
||||||
|
|
||||||
|
### Rule-1 summary at realistic cost ($0.65 round trip)
|
||||||
|
|
||||||
|
| TF | 12mo in-sample | 12mo H1 / H2 | OOS (2024-06→2025-03) |
|
||||||
|
|---|---|---|---|
|
||||||
|
| M15 | +73.93R PF 1.53 | +66.71 / +7.23 (PF 1.10) | +38.77R PF 1.36 |
|
||||||
|
| M5 (shipped, nopb) | +106.82R PF 1.51 (current HEAD) | +50.80 / +56.02 (PF 1.46 / 1.55) | +9.52R PF 1.34 |
|
||||||
|
| H1 | +23.97R PF 1.55 | — | +5.15R PF 1.13 |
|
||||||
|
| H4 | +16.14R PF 2.28 | — | +1.31R PF 1.14 |
|
||||||
|
|
||||||
|
- **M5 (shipped)** passes rule 1 cleanly: both current-HEAD 12mo halves positive with PF ≥ 1.3, and OOS positive with PF 1.34.
|
||||||
|
- **M15** passes in-sample and OOS (PF 1.53 / 1.36) but its second half is soft (PF 1.10) — positive R, below the 1.3 bar. Pre-existing condition, not a re-pin artifact.
|
||||||
|
- **H1/H4** pass in-sample but OOS is marginal (PF 1.13–1.14) — consistent with the standing "H4 unproven for auto-execution" note.
|
||||||
|
|
||||||
|
## Oanda ↔ Twelve Data trade-level delta
|
||||||
|
|
||||||
|
Not worth a trade-by-trade book diff: Phase 1 already showed the two feeds disagree on **bar direction** 19% of the time on M5 (11% on M15), with a ~$3 mean / ~$1 systematic close gap. A TD-pinned book is a different instrument's book — this is the documented re-pin motivator, and the Oanda books above are the replacement baseline. A formal trade-level reconciliation would show near-total mismatch and add no decision value.
|
||||||
|
|
||||||
|
## Deliverable status
|
||||||
|
|
||||||
|
- [x] Re-pinned books exist on Oanda bars (12mo + OOS + 5yr, $0.30 + $0.65).
|
||||||
|
- [x] Rule 1 verified for the shipped TFs (M5 clean; M15 in-sample+OOS pass, H2 soft; H1/H4 OOS marginal).
|
||||||
|
- [x] Cost model: $0.65 matches measured Oanda spread (~$0.65–0.72); $0.30 retained for cross-book comparability.
|
||||||
|
- [x] Oanda↔TD delta quantified (Phase 1) — no further book diff needed.
|
||||||
|
|
||||||
|
Remaining Phase-3 follow-up: none. Owner decision 2026-08-14 retained dual reporting: $0.65 is the realistic headline while $0.30 remains for TD-era cross-book comparability and the live TP spread gate.
|
||||||
@@ -44,7 +44,9 @@ book directly. Windows (cache start = start − 62 days):
|
|||||||
Acceptance: positive total R in both 12mo halves, PF ≥ 1.3, OOS not worse
|
Acceptance: positive total R in both 12mo halves, PF ≥ 1.3, OOS not worse
|
||||||
than control, and a candidate that removes a net-positive bucket for a PF-only
|
than control, and a candidate that removes a net-positive bucket for a PF-only
|
||||||
gain is rejected (big-candle precedent). Runner:
|
gain is rejected (big-candle precedent). Runner:
|
||||||
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py`.
|
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py` (scratchpad
|
||||||
|
tooling, not retained — both shipped gates are reproduced by a plain pair-factory
|
||||||
|
run at HEAD; see the parity recipe in `sizing_campaign_2026-09.md`).
|
||||||
|
|
||||||
Candidates (all `--set`, no code):
|
Candidates (all `--set`, no code):
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# Signal-Level Replay Parity — Oanda vs Tiingo (Phase 2)
|
||||||
|
|
||||||
|
**Date:** 2026-08-14
|
||||||
|
**Tool:** `scripts/signal_parity_study.py` (replays real `analyze_from_dataframes`, news off)
|
||||||
|
**Window:** 2025-08-14 → 2026-08-14 (12 months); M15 stride 8, M5 stride 24 → ~2,900 sampled decisions each
|
||||||
|
**Preceding:** Phase 1 price-diff study (`feed_divergence_2026-08-14.md`)
|
||||||
|
|
||||||
|
## Headline
|
||||||
|
|
||||||
|
On the ~94–96% of bars the two feeds share, **Oanda and Tiingo produce near-identical signals**:
|
||||||
|
|
||||||
|
| Metric | M15 | M5 |
|
||||||
|
|---|---|---|
|
||||||
|
| same direction | 98.32% | 98.16% |
|
||||||
|
| same grade | 94.13% | 95.23% |
|
||||||
|
| same setup presence | 98.83% | 99.49% |
|
||||||
|
| same direction + grade + setup | 93.41% | 94.21% |
|
||||||
|
| opposite-sign flips (BUY↔SELL) | **0** | **0** |
|
||||||
|
| direction flips on actionable bars | 6.6% | 10.9% |
|
||||||
|
| grade changed on actionable bars | 14.2% | 7.8% |
|
||||||
|
| mean \|Δ final_score\| | 1.76 | 1.78 |
|
||||||
|
| mean \|Δ primary_score\| | 1.73 | 1.77 |
|
||||||
|
| same strategy tag | 100% | 100% |
|
||||||
|
|
||||||
|
Every direction flip is **actionable↔NEUTRAL**, never a sign reversal: the feeds never disagree on buy-vs-sell, only on *whether* a marginal bar crosses the action threshold. Flipping bars cluster at the Friday-close/Sunday-reopen boundary (UTC hours 22–23) and in the Asian hours (03–04).
|
||||||
|
|
||||||
|
## Setup levels diverge even when the signal agrees
|
||||||
|
|
||||||
|
On bars where **both** feeds emit a setup with the same tag:
|
||||||
|
|
||||||
|
| | M15 (n=119) | M5 (n=39) |
|
||||||
|
|---|---|---|
|
||||||
|
| entry mean \|Δ\| | $0.12 | $0.10 |
|
||||||
|
| SL mean \|Δ\| | $1.83 | $2.79 |
|
||||||
|
| TP1 mean \|Δ\| | $2.47 | $1.52 |
|
||||||
|
| TP2 mean \|Δ\| | $3.74 | $2.46 |
|
||||||
|
| TP1 p95 \|Δ\| | $17.4 | $2.3 |
|
||||||
|
|
||||||
|
Entry price is the current close (≈ identical across feeds), but **SL/TP are recomputed from ATR and recent swing extremes, which differ more** — mean SL drift ~$2–3 (p95 up to $10–26 on M15), i.e. 20–40% of ATR. So even an identical signal places stops/targets at meaningfully different prices on a different feed.
|
||||||
|
|
||||||
|
## Phase 2 pass/fail vs plan thresholds
|
||||||
|
|
||||||
|
| Criterion | Plan threshold | M15 / M5 | Verdict |
|
||||||
|
|---|---|---|---|
|
||||||
|
| identical direction+grade | ≥ 97% | 93.4% / 94.2% | **marginal** (below 97%, above 93%) |
|
||||||
|
| opposite-sign flips | (none specified) | 0 / 0 | PASS |
|
||||||
|
| flips clustered at marginal composites | no session/regime clustering | yes (boundary hours) | PASS |
|
||||||
|
|
||||||
|
The "identical direction+grade" metric lands slightly under the proposed 97% bar — but the shortfall is grade *letters* (A+ vs A vs B) changing on marginal composites, not direction. If the acceptance criterion is "no opposite-sign disagreement and ~98% direction identity", this passes comfortably.
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
Oanda ↔ Tiingo is **signal-equivalent for direction** (98% identical, zero sign flips) over 12 months. The residual deltas are:
|
||||||
|
|
||||||
|
1. **~6% of decisions differ in grade letter** on marginal composites (A+↔A↔B near thresholds).
|
||||||
|
2. **SL/TP levels drift ~$2–4 mean** (up to ~$20 p95 on M15) even for identical signals — a real P&L difference at the trade level.
|
||||||
|
3. **Tiingo's extra Sunday bars** (Phase 1) are gated by the aggregator's NY-anchored weekend filter, so they do not inject phantom signals, but they do slightly shift indicator history at Monday opens.
|
||||||
|
|
||||||
|
Net: switching Oanda→Tiingo changes a handful of marginal decisions and moves stop/target levels, but does not change *what the system is doing* (direction, strategy, setup presence). Oanda↔Twelve Data was not replayed — Phase 1 already showed 81% M5 bar-direction disagreement and a ~$3/$1 systematic gap, so a TD replay would trivially fail (that is the documented re-pin motivator).
|
||||||
@@ -3,8 +3,9 @@
|
|||||||
XAU/USD static dashboard publisher.
|
XAU/USD static dashboard publisher.
|
||||||
|
|
||||||
Standalone daemon that scans XAU/USD at candle closes and atomically
|
Standalone daemon that scans XAU/USD at candle closes and atomically
|
||||||
writes `snapshot.json` + `news.json` (plus a copy of
|
writes `snapshot.json` + `news.json` (plus copies of the static
|
||||||
`static/dashboard.html`) into an Apache-served directory. Apache does
|
`dashboard.html` / `.css` / `.js` and `news_summary.php`) into an
|
||||||
|
Apache-served directory. Apache does
|
||||||
all web serving — this process never opens a port, so a failure here
|
all web serving — this process never opens a port, so a failure here
|
||||||
can only ever mean a stale page.
|
can only ever mean a stale page.
|
||||||
|
|
||||||
|
|||||||
@@ -24,8 +24,9 @@ No new Python dependencies are required.
|
|||||||
## 2. Make the web folder writable
|
## 2. Make the web folder writable
|
||||||
|
|
||||||
The dashboard publishes straight into the portal docroot `/var/www/trade/`
|
The dashboard publishes straight into the portal docroot `/var/www/trade/`
|
||||||
(owner decision 2026-08-03) — it adds exactly four files there:
|
(owner decision 2026-08-03) — it adds exactly six files there:
|
||||||
`dashboard.html`, `news_summary.php`, `snapshot.json`, `news.json` (plus the
|
`dashboard.html`, `dashboard.css`, `dashboard.js`, `news_summary.php`,
|
||||||
|
`snapshot.json`, `news.json` (plus the
|
||||||
apache-owned `ai/` cache dir from §6). The daemon user must be able to write
|
apache-owned `ai/` cache dir from §6). The daemon user must be able to write
|
||||||
them:
|
them:
|
||||||
|
|
||||||
@@ -51,7 +52,7 @@ sudo restorecon -R /var/www/trade
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
venv/bin/python dashboard_publisher.py --once
|
venv/bin/python dashboard_publisher.py --once
|
||||||
ls /var/www/trade # dashboard.html snapshot.json news.json added next to the portal
|
ls /var/www/trade # dashboard.html/.css/.js news_summary.php snapshot.json news.json added next to the portal
|
||||||
```
|
```
|
||||||
|
|
||||||
Then open `http://<server>/dashboard.html` (path relative to however the
|
Then open `http://<server>/dashboard.html` (path relative to however the
|
||||||
@@ -141,8 +142,8 @@ curl -s "http://<server>/news_summary.php?id=$ID" # instant, from ai/<id>.json
|
|||||||
## Operational notes
|
## Operational notes
|
||||||
|
|
||||||
- **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
|
- **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
|
||||||
re-copies `static/dashboard.html` and `static/news_summary.php` into the
|
re-copies `static/dashboard.html`, `dashboard.css`, `dashboard.js` and
|
||||||
web folder at startup.
|
`news_summary.php` into the web folder at startup.
|
||||||
- **If the daemon dies**, systemd restarts it in 30 s; worst case the page
|
- **If the daemon dies**, systemd restarts it in 30 s; worst case the page
|
||||||
shows stale data with its own "updated N min ago" evidence. Apache and the
|
shows stale data with its own "updated N min ago" evidence. Apache and the
|
||||||
telegram bot are unaffected.
|
telegram bot are unaffected.
|
||||||
@@ -155,8 +156,5 @@ curl -s "http://<server>/news_summary.php?id=$ID" # instant, from ai/<id>.json
|
|||||||
DeepSeek-side spend alerting is the true backstop. Kept separate on
|
DeepSeek-side spend alerting is the true backstop. Kept separate on
|
||||||
purpose (independence rule: the PHP never imports/reads bot code or
|
purpose (independence rule: the PHP never imports/reads bot code or
|
||||||
config.json).
|
config.json).
|
||||||
- **Restart window**: for the first minutes after a daemon restart the new
|
|
||||||
page can meet a pre-upgrade news.json without headline ids — those rows
|
|
||||||
simply render non-clickable until the first news publish.
|
|
||||||
- **Credentials**: uses the same env/.env/config.json resolution as the rest
|
- **Credentials**: uses the same env/.env/config.json resolution as the rest
|
||||||
of the platform (Oanda candles feed; news needs no key; DeepSeek optional).
|
of the platform (Oanda candles feed; news needs no key; DeepSeek optional).
|
||||||
|
|||||||
Binary file not shown.
|
Before Width: | Height: | Size: 603 KiB After Width: | Height: | Size: 578 KiB |
@@ -148,7 +148,7 @@ box(2.5, 3.2, 71.8, 8.0, "backtest/gpu_backtest.py — bar-by-bar replay through
|
|||||||
"CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
|
"CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
|
||||||
BTS_FC, BTS_EC)
|
BTS_FC, BTS_EC)
|
||||||
box(77.5, 3.2, 20, 8.0, "test_all.py",
|
box(77.5, 3.2, 20, 8.0, "test_all.py",
|
||||||
["359 tests · 22 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
|
["461 tests · 31 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
|
||||||
|
|
||||||
# ── arrows: short structural hops (solid dark) ───────────────────────
|
# ── arrows: short structural hops (solid dark) ───────────────────────
|
||||||
arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2) # oanda → providers
|
arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2) # oanda → providers
|
||||||
@@ -185,6 +185,7 @@ ax.text(81.8, 38.9, "replays the exact signal code", fontsize=6.4,
|
|||||||
color=MUT, ha="center", va="center", zorder=5,
|
color=MUT, ha="center", va="center", zorder=5,
|
||||||
bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
|
bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
|
||||||
|
|
||||||
|
if __name__ == "__main__": # importing the module must never rewrite the PNG
|
||||||
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
|
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
|
||||||
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
|
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
|
||||||
print(f"saved {out}")
|
print(f"saved {out}")
|
||||||
|
|||||||
@@ -0,0 +1,92 @@
|
|||||||
|
# Test Plan — Data Source Change Validation
|
||||||
|
|
||||||
|
**Purpose:** determine whether switching `data_providers.active` (e.g. Twelve Data → Oanda, or any future provider swap) changes signal calculations materially, and gate the switch on evidence.
|
||||||
|
|
||||||
|
**Status:** in progress, 2026-08-14
|
||||||
|
|
||||||
|
> **Progress (2026-08-14):** Phase 0 ✅, Phase 1 ✅ (`backtest/research/feed_divergence_2026-08-14.md`), Phase 2 ✅ (`backtest/research/signal_parity_2026-08-14.md`), Phase 3 ✅ (`backtest/research/repin_verification_2026-08-14.md` — re-pin already existed), Phase 4 🔨 tool ready (`scripts/shadow_scan.py`, needs a 2-week run — launch manually per owner decision), Phase 5 owner decisions recorded (dual cost reporting; Tiingo streaming-only; see bottom).
|
||||||
|
**Context:** Oanda has been the live feed since 2026-07-28. The backtest books were **re-pinned on Oanda bars on 2026-08-04** (`backtest/oanda_*.csv` — 27 books, now the baseline; all TD-era books superseded). Cost reporting is dual: $0.30 retained for cross-book comparability, $0.65 is the realistic headline (measured Oanda round trip $0.65–0.72). Twelve Data was proven signal-incompatible in Phase 1 (~19% of M5 bars disagree on direction; systematic ~$1 bias) — it is NOT a valid candle fallback.
|
||||||
|
|
||||||
|
**Why signals can differ between feeds** (mechanisms to test):
|
||||||
|
|
||||||
|
- Feeds publish different mid prices → tiny OHLC diffs shift every indicator; near a threshold (RSI band, `FB2_EMA_LO` EMA200-proximity band, composite/grade cutoffs, M5 min-score gate) one bar's discrete score flips.
|
||||||
|
- Gate edge cases turn a tiny price diff into "setup generated" vs "no signal".
|
||||||
|
- ATR-derived SL/TP/trail levels differ even when the signal is identical.
|
||||||
|
- The shared 10σ cleaning pass can drop different bars per feed.
|
||||||
|
- Volume present (Oanda) vs absent (Tiingo FX); real spread (Oanda MBA) vs pair default (Tiingo).
|
||||||
|
- Known reference point: H4 seeding divergence — same provider, different fetch depth flipped ~2% of indicator signals (composite Δ up to 12.3). Cross-feed diffs act the same way.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 0 — Recent-bar parity smoke test ✅ DONE
|
||||||
|
|
||||||
|
- **Tool:** `scripts/compare_oanda_tiingo.py` (removed 2026-09-06 once Phases 1-2 superseded it; the result is recorded here)
|
||||||
|
- **Result 2026-08-14:** 10/10 timestamp alignment on M5/M15/H1, Close max |Δ| ≤ $0.175 (≤ 0.004% of price), no directional bias.
|
||||||
|
- **Verdict:** recent-tail parity confirmed. Does NOT cover historical depth — proceed to Phase 1.
|
||||||
|
|
||||||
|
## Phase 1 — Historical candle divergence study
|
||||||
|
|
||||||
|
**Goal:** quantify raw-data divergence over the full window used for validation, not just the recent tail.
|
||||||
|
|
||||||
|
- [x] Extend the comparison script to use `fetch_range` (paginated, raw bars) for both providers over **≥ 12 months** for M5, M15, H1 (and H4 if in scope).
|
||||||
|
- [x] Record per TF:
|
||||||
|
- % of bar timestamps present in both feeds; list of missing/extra bars (gaps, holidays, weekend boundary handling).
|
||||||
|
- OHLC |Δ| distribution: p50 / p95 / p99 / max, in $ and % of price.
|
||||||
|
- Count of bars dropped by `validate_and_clean` on each feed (10σ outliers, malformed).
|
||||||
|
- Bar-close timestamp conventions (alignment, DST behavior around US transitions).
|
||||||
|
- [x] **Pass criteria (measured on the weekend-filtered bar grid — the aggregator's NY-anchored `_is_forex_weekend` decides which bars exist for signal purposes, so raw timestamp overlap includes coverage-only diffs like Tiingo's Sunday bars and is NOT the criterion):** ≥ 99.5% overlap of *signal-eligible* bar timestamps; Close p95 |Δ| ≤ 0.01% of price; no systematic bias (mean Δ ≈ 0); no unexplained multi-hour gaps unique to one feed. (Raw-overlap result 2026-08-14: 94.6–96.6% vs Tiingo — entirely explained by Tiingo's ~22h/week of flat Sunday bars, which the weekend filter gates out.)
|
||||||
|
- [x] Write findings to `backtest/research/feed_divergence_<date>.md` with tables and the exact commands used.
|
||||||
|
|
||||||
|
## Phase 2 — Signal-level replay parity
|
||||||
|
|
||||||
|
**Goal:** measure how often candle divergence actually changes a *decision*.
|
||||||
|
|
||||||
|
- [x] Replay bar-by-bar through the real `analyze_from_dataframes` (news off) for both feeds over the same ≥ 12-month window, per TF (M5, M15, H1). Reuse the gpu_backtest replay harness; disable the parquet cache (`--no-cache`) or use separate cache keys per provider.
|
||||||
|
- [x] Compare per bar: signal direction, composite score, quality grade, gate outcomes (which gate fired), setup presence, strategy label (XAU ATR / XAU_PB / XAU_CP), entry/SL/TP prices.
|
||||||
|
- [x] Metrics:
|
||||||
|
- % bars with identical direction+grade.
|
||||||
|
- Direction flips, grade changes, gate-only changes (setup appears/disappears).
|
||||||
|
- Distribution of composite Δ on mismatched bars.
|
||||||
|
- For matching signals: mean/max |Δ| of entry, SL, TP1/TP2 in $ and in ATR fractions.
|
||||||
|
- [x] **Result (Oanda↔Tiingo):** 98.32% / 98.16% same direction (M15/M5), zero opposite-sign flips; 93.41% / 94.21% same direction+grade+setup. The original ≥97% direction+grade proposal is too strict for benign grade-letter changes and was not met; fallback acceptance is direction ≥98%, zero sign flips, and marginal/boundary-hour-only flips. See `signal_parity_2026-08-14.md`.
|
||||||
|
|
||||||
|
## Phase 3 — Backtest re-pin on the new feed
|
||||||
|
|
||||||
|
**Goal:** a new pinned book on the new provider's bars, evaluated under rule 1 acceptance criteria.
|
||||||
|
|
||||||
|
- [x] **Provider selection:** `gpu_backtest.py` has **no `--provider` flag** — it builds the provider from `data_providers.active` in config.json (`gpu_backtest.py:1753`). A cross-provider run requires flipping that config line. The parquet cache is already keyed per provider (`gpu_backtest.py:984` — `oanda_`-prefixed files vs legacy unprefixed TD files), so TD and Oanda data cannot mix in cache; `--no-cache` is only needed to force a refetch.
|
||||||
|
- [x] Run the full backtest on new-feed bars, both cost assumptions:
|
||||||
|
```bash
|
||||||
|
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.30 --export-csv backtest/repin_<provider>_m15_030.csv
|
||||||
|
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.65 --export-csv backtest/repin_<provider>_m15_065.csv
|
||||||
|
```
|
||||||
|
(repeat for each TF under test; note the `Backend:` log line and provider in the report per `backtest/instructions_for_backtest.txt`).
|
||||||
|
- [x] Split into train/validation halves; require **positive total R in both halves and PF ≥ 1.3 at realistic cost** (rule 1), with trailing exits and next-open fills untouched.
|
||||||
|
- [x] Marginal configs additionally pass the **2024-06 → 2025-03 OOS window** (within-year splits alone have approved configs that later failed OOS).
|
||||||
|
- [x] ~~Compare against the pinned Twelve-Data book trade-by-trade~~ **AMENDED 2026-08-14:** skipped with justification — Phase 1 showed TD bars disagree on direction 19% of the time on M5 with a systematic ~$1 bias, so a TD-pinned book is a different instrument's book and a trade-level diff carries no decision value. Phantom-trade classes remain documented (e.g. the weekend-calendar fix already dropped exactly one live-impossible M5 trade).
|
||||||
|
- [x] **Deliverable:** re-pinned book + report in `backtest/research/`; update the reference numbers in CLAUDE.md if the book is adopted. Current-HEAD M5 canonical book: `backtest/oanda_m5_12mo_065_nopb_20260814.csv`, +106.82R PF 1.51.
|
||||||
|
|
||||||
|
## Phase 4 — Live shadow comparison
|
||||||
|
|
||||||
|
**Goal:** confirm parity under real fetch depths, caching, and boundary timing — not just history.
|
||||||
|
|
||||||
|
- [ ] Run a shadow scan loop (no orders, no Telegram) for **≥ 2 weeks**: each candle close, scan with both providers, log direction/grade/gates/setup to a side-by-side file or SQLite table.
|
||||||
|
- [ ] Include at least one weekend transition and one news-heavy day in the window.
|
||||||
|
- [ ] Track: scan-to-scan agreement rate, feed-lag incidents ("feed lagging — just-closed bar missing"), staleness warnings, spread readings (Oanda real vs Tiingo default).
|
||||||
|
- [ ] **Acceptance thresholds (baseline = Phase 2 replay parity):** ≥ 98% direction agreement; **zero** opposite-sign (BUY↔SELL) flips; direction flips confined to actionable↔NEUTRAL at marginal composites, clustered only at Friday-close/Sunday-reopen (UTC 22–23) and Asian (03–04) hours; setup-level drift (SL/TP) within the Phase 2 envelope (mean |Δ| ≲ $3, p95 ≲ $17 on M15); every feed-lag/staleness incident explained.
|
||||||
|
- [ ] H4 caution: live H4 fetches only 380 bars vs unbounded backtest history (known ~2% indicator-signal flip from seeding alone) — keep H4 labeled "unproven" until a dedicated campaign closes this.
|
||||||
|
|
||||||
|
## Phase 5 — Cutover checklist
|
||||||
|
|
||||||
|
- [x] `config.json` `data_providers.active: "oanda"` and `data_providers.streaming: "tiingo"` set deliberately (verified 2026-08-14 — Oanda cannot stream; Tiingo streams).
|
||||||
|
- [x] Cost model: **dual reporting adopted (owner decision 2026-08-14)** — $0.65 (measured Oanda round trip) is the realistic headline, $0.30 retained for cross-book comparability with TD-era books. ⚠️ **Cost-gate coupling:** `config.py round_trip_cost` (0.30) is NOT just a backtest knob — it feeds the live TP1 spread gate (`min_tp = round_trip_cost × min_tp_cost_multiple`, three sites in `signal_aggregator.py`). Raising it to 0.65 moves the TP1 floor from ~$9 to ~$19.50 and is a rule-1 revalidation event, not a config tweak. `round_trip_cost` stays 0.30.
|
||||||
|
- [x] QA suite green: `python test_all.py` (381 tests; verified 2026-08-14).
|
||||||
|
- [ ] Week 1 post-cutover: daily review of `logs/signals.db` rows vs shadow log; alert on grade/direction disagreement beyond the Phase 2 baseline rate.
|
||||||
|
- [ ] Rollback: revert `data_providers.active`; pinned books of the previous feed remain valid for comparison. ⚠️ Twelve Data is a permanent *registered* fallback but is **signal-incompatible** (Phase 1) — on an Oanda candle outage, stop scanning rather than switching `active` to `twelve_data`; a TD-fed book cannot be compared to the Oanda baseline.
|
||||||
|
- [x] Docs sync (CLAUDE.md rule): `docs/user_guide.html` backtest numbers reflect the Oanda-pinned books, not TD-era numbers.
|
||||||
|
|
||||||
|
## Open questions — RESOLVED 2026-08-14 (owner decisions)
|
||||||
|
|
||||||
|
- ~~Adopt which spread as canonical~~ → **dual reporting**: $0.65 realistic headline + $0.30 cross-book comparability; `round_trip_cost` stays 0.30 (gate coupling above).
|
||||||
|
- ~~Tiingo role~~ → **streaming-only**. Tiingo remains registered for the WS tick stream and comparison research, but is not a designated candle fallback; candle fallback is a manual halt-and-decide, never an automatic TD failover.
|
||||||
|
- H4 seeding campaign (parked since 2026-08-04) → **still needs an owner**; H4 stays "unproven" until then.
|
||||||
@@ -1,154 +0,0 @@
|
|||||||
# Centralized Configuration (config.json) — Design
|
|
||||||
|
|
||||||
**Date:** 2026-07-21
|
|
||||||
**Status:** Approved (user confirmed all four scoping decisions)
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Move deployment settings — Telegram credentials, data-provider API keys and
|
|
||||||
selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert
|
|
||||||
quality boot default, log level — into a single gitignored `config.json` at the
|
|
||||||
repo root, loaded once at startup.
|
|
||||||
|
|
||||||
**Hard constraint:** zero change to signal logic, indicator weights, thresholds,
|
|
||||||
or gates. With an unedited `config.json` (or none at all), every produced
|
|
||||||
`PlatformConfig` must be byte-identical to today's factory output. This is
|
|
||||||
test-enforced (see Testing).
|
|
||||||
|
|
||||||
## Scoping decisions (user-confirmed 2026-07-21)
|
|
||||||
|
|
||||||
1. **Secrets live in config.json** (gitignored). A committed
|
|
||||||
`config.json.example` carries placeholders. Env vars still override.
|
|
||||||
The hardcoded credential fallbacks in `config.py` and `telegram_bot.py`
|
|
||||||
are **deleted** — on a machine with neither config.json nor env vars, the
|
|
||||||
bot refuses to start with an explicit error instead of silently using
|
|
||||||
committed credentials. (Recommend rotating the Telegram token and Twelve
|
|
||||||
Data key after merge; both are exposed in git history.)
|
|
||||||
2. **Multi-provider is structure only**: a `data_providers` section with named
|
|
||||||
provider blocks and an `active` selector. Only Twelve Data is implemented.
|
|
||||||
No failover logic (nothing to fail over to yet).
|
|
||||||
3. **Blocked hours: base list per pair only.** The validated per-timeframe
|
|
||||||
deltas (M5 appends hour 9; M15 SELL-block 15/17 UTC) stay in
|
|
||||||
`pair_signals/xauusd.py` next to their validation evidence. The overlay is
|
|
||||||
applied **before** those deltas and updates **both** mirrored copies
|
|
||||||
(`xau_strategy.blocked_hours_utc` and `backtest.blocked_hours_utc`) so they
|
|
||||||
cannot drift.
|
|
||||||
4. **Extra keys included:** scan timeframes, alert min quality (boot default),
|
|
||||||
log level. Everything else (indicator params, RR geometry, M5 gates,
|
|
||||||
CP knobs, news config) deliberately stays code-only.
|
|
||||||
|
|
||||||
## File schema
|
|
||||||
|
|
||||||
`config.json` (gitignored; `config.json.example` committed with placeholders
|
|
||||||
for secrets and today's tuned values for everything else):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"telegram": {
|
|
||||||
"token": "<bot token>",
|
|
||||||
"chat_id": "<owner chat id>",
|
|
||||||
"channel_id": "<-100… or empty string to disable the channel feed>",
|
|
||||||
"alert_min_quality": "C"
|
|
||||||
},
|
|
||||||
"data_providers": {
|
|
||||||
"active": "twelve_data",
|
|
||||||
"twelve_data": { "api_key": "<key>" }
|
|
||||||
},
|
|
||||||
"scanning": {
|
|
||||||
"pairs": ["XAU/USD"],
|
|
||||||
"timeframes": ["M5", "M15", "H1", "H4"]
|
|
||||||
},
|
|
||||||
"pairs": {
|
|
||||||
"XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
|
|
||||||
},
|
|
||||||
"log_level": "INFO"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Timeframe strings use the `Timeframe` enum names (`M5`, `M15`, `H1`, `H4`, …).
|
|
||||||
Pair keys use canonical symbols (`XAU/USD`).
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### New module: `app_config.py`
|
|
||||||
|
|
||||||
Single responsibility: read + validate `config.json`, expose typed accessors,
|
|
||||||
apply the pair-level overlay. No signal knowledge.
|
|
||||||
|
|
||||||
- `AppConfig` dataclass: `telegram` (token, chat_id, channel_id,
|
|
||||||
alert_min_quality), `data_providers` (active name + per-provider dicts),
|
|
||||||
`scan_pairs`, `scan_timeframes`, `pair_overrides`
|
|
||||||
(symbol → {blocked_hours_utc}), `log_level`.
|
|
||||||
- `load_app_config(path=REPO_ROOT/"config.json") -> AppConfig` — module-level
|
|
||||||
cached (loaded once per process). Missing file → `AppConfig` of pure
|
|
||||||
defaults + one warning log. Malformed JSON, unknown top-level/section keys,
|
|
||||||
unknown pair symbols, hours outside 0–23, unknown timeframe names, unknown
|
|
||||||
`active` provider → raise `AppConfigError` with a named message (fail fast
|
|
||||||
at boot).
|
|
||||||
- Precedence, applied inside the accessors:
|
|
||||||
**env var → config.json → code default**. Secrets (`TELEGRAM_TOKEN`,
|
|
||||||
`TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`) have **no
|
|
||||||
code default** after this change.
|
|
||||||
- `apply_pair_overrides(cfg: PlatformConfig, app: AppConfig) -> PlatformConfig`
|
|
||||||
— sets the pair's base `blocked_hours_utc` on **both**
|
|
||||||
`cfg.xau_strategy.blocked_hours_utc` and `cfg.backtest.blocked_hours_utc`,
|
|
||||||
plus `cfg.scan_pairs` / `cfg.scan_timeframes`. Called by the pair factory
|
|
||||||
(see below). A no-op when the pair has no override block.
|
|
||||||
|
|
||||||
### Wiring points (all consumers)
|
|
||||||
|
|
||||||
| Consumer | Change |
|
|
||||||
|---|---|
|
|
||||||
| `pair_signals/xauusd.py` | `make_config` calls `apply_pair_overrides` **before** the per-TF deltas (so M5 still appends hour 9 to whatever base the JSON sets, and mirrors stay in sync). |
|
|
||||||
| `data_provider.py` | API key resolution becomes `app_config` accessor for the **active** provider (env `TWELVE_DATA_API_KEY` still wins). `config.twelve_data_api_key` field and its hardcoded default are deleted. Missing key keeps today's behavior: provider constructed with `_td=None`, errors on first fetch. |
|
|
||||||
| `telegram_bot.py` | `TOKEN`, `CHAT_ID`, `CHANNEL_ID` come from `app_config` (hardcoded fallbacks deleted; missing token/chat_id → explicit startup error). `_alert_min_quality` boot default and `logging` level come from `app_config`. `SCAN_PAIRS` (today built from a bare `PlatformConfig()`) reads `app_config.scan_pairs` directly; `apply_pair_overrides` also stamps `cfg.scan_pairs`/`cfg.scan_timeframes` so the dataclass field stays truthful for any other reader. |
|
|
||||||
| `main.py` | No structural change — it uses the factory, which now applies the overlay. |
|
|
||||||
| `backtest/gpu_backtest.py` | No structural change — it uses the factory too, so an edited base blocked-hours list flows into replays identically to live (parity by construction). Data fetches pick up the provider key via `data_provider.py`. |
|
|
||||||
| `config.py` | `twelve_data_api_key` field removed. Everything else untouched. |
|
|
||||||
|
|
||||||
### Multi-provider structure
|
|
||||||
|
|
||||||
`data_provider.py` gains a minimal registry:
|
|
||||||
`PROVIDER_REGISTRY = {"twelve_data": ForexDataProvider}`. `active` selects
|
|
||||||
from it; an unknown name fails at load. Adding a provider later = one config
|
|
||||||
block + one adapter class registered here. No shared base class extracted now
|
|
||||||
(YAGNI — with one provider there is nothing to abstract over yet).
|
|
||||||
|
|
||||||
## Error handling summary
|
|
||||||
|
|
||||||
| Condition | Behavior |
|
|
||||||
|---|---|
|
|
||||||
| No config.json | Defaults + warning. Secrets must come from env or startup fails (bot) / fetch fails (provider). |
|
|
||||||
| Malformed JSON / bad values / unknown keys | `AppConfigError` at load, process exits with named message. |
|
|
||||||
| Missing Telegram token or chat_id at bot boot | Explicit startup error naming the missing key and both places it can be set. |
|
|
||||||
| Missing data-provider key | Same as today: provider builds, first fetch errors. |
|
|
||||||
|
|
||||||
## Migration
|
|
||||||
|
|
||||||
- Create the local `config.json` populated with the **current live values**
|
|
||||||
(the ones being deleted from source) so the running bot keeps working
|
|
||||||
unchanged. `.gitignore` gains `config.json`.
|
|
||||||
- Commit `config.json.example` with placeholders.
|
|
||||||
- CLAUDE.md: update the env-var table, repository map, and config section.
|
|
||||||
- Post-merge recommendation (user action): rotate the Telegram token and
|
|
||||||
Twelve Data key, since both are in git history.
|
|
||||||
|
|
||||||
## Testing (`test_all.py`, new module)
|
|
||||||
|
|
||||||
1. **Parity pin (the critical one):** a config.json containing the example
|
|
||||||
defaults, and no config.json at all, both produce factory output identical
|
|
||||||
to today's (full dataclass equality across all four TFs).
|
|
||||||
2. Overlay mapping: JSON blocked hours land on **both** mirrored fields; M5
|
|
||||||
factory still appends hour 9 on top of a custom base.
|
|
||||||
3. Precedence: env var beats config.json beats default, per key.
|
|
||||||
4. Validation: malformed JSON, bad hour, unknown pair/timeframe/provider each
|
|
||||||
raise `AppConfigError`.
|
|
||||||
5. Missing-file behavior: defaults + no crash.
|
|
||||||
6. Existing factory-equivalence test stays green (equivalence is now defined
|
|
||||||
over factory-with-overlay vs inline-config-with-overlay).
|
|
||||||
|
|
||||||
## Out of scope
|
|
||||||
|
|
||||||
Hot reload (config is read once at boot), a second concrete provider,
|
|
||||||
failover, moving any tuned strategy value beyond the base blocked-hours list,
|
|
||||||
news/indicator/geometry settings in JSON.
|
|
||||||
@@ -1,252 +0,0 @@
|
|||||||
================================================================================
|
|
||||||
MULTI-PROVIDER LLM LAYER - PLAN
|
|
||||||
(DeepSeek for news summary, Anthropic Claude for agent panel)
|
|
||||||
Written 2026-08-08. Plan only - no code has been changed.
|
|
||||||
================================================================================
|
|
||||||
|
|
||||||
WHY
|
|
||||||
---
|
|
||||||
Today every LLM feature (news enrichment + agent panel) goes through one
|
|
||||||
hardwired client: llm_client.py -> DeepSeekClient, a module singleton reached
|
|
||||||
via get_llm_client(). This plan describes how to route different features to
|
|
||||||
different providers - e.g. keep DeepSeek for cheap high-volume news scoring,
|
|
||||||
put the agent panel on Anthropic Claude for better reasoning.
|
|
||||||
|
|
||||||
The repo already solved this problem once for market data: the providers/
|
|
||||||
package (BaseDataProvider owns shared machinery, subclasses implement only
|
|
||||||
_fetch_raw, a registry + one config knob selects the provider). This plan
|
|
||||||
mirrors that proven pattern at the LLM layer, scaled down to a single module.
|
|
||||||
|
|
||||||
BLAST RADIUS (verified by grep)
|
|
||||||
-------------------------------
|
|
||||||
get_llm_client() call sites:
|
|
||||||
- news_engine.py ....... 4 sites
|
|
||||||
- agent_panel.py ....... 3 sites (+1 timeout getattr)
|
|
||||||
- telegram_bot.py ...... 1 site (/status health; reads private _count,
|
|
||||||
_count_day, _daily_cap fields)
|
|
||||||
- test_all.py .......... 41 locations stub via lc._client = _StubLLM(...)
|
|
||||||
Out of scope: static/news_summary.php (own PHP DeepSeek call + key file,
|
|
||||||
separate deployment).
|
|
||||||
|
|
||||||
DESIGN INVARIANTS (carried over unchanged)
|
|
||||||
------------------------------------------
|
|
||||||
1. Fail-safe contract: every call returns Optional[Dict]; any failure -> None;
|
|
||||||
callers keep their non-LLM fallbacks; LLM output never gates signals.
|
|
||||||
2. Single attempt, no retries (news cache cadence is the retry); per-provider
|
|
||||||
UTC daily cap; warn-once logging; health stamps (last_ok, last_error).
|
|
||||||
3. Static system prompts stay byte-identical constants - provider-neutral,
|
|
||||||
and they map to prompt caching on BOTH providers (DeepSeek prefix cache;
|
|
||||||
Anthropic cache_control ephemeral).
|
|
||||||
4. Test compatibility: all 41 existing lc._client stub sites keep working
|
|
||||||
with zero edits.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
1. llm_client.py REFACTOR (~195 -> ~430 lines, no new files)
|
|
||||||
================================================================================
|
|
||||||
|
|
||||||
1a. BaseLLMClient - shared machinery (extracted from today's DeepSeekClient)
|
|
||||||
- __init__(api_key, model, timeout, daily_cap, enabled)
|
|
||||||
- public: name ("deepseek"/"anthropic"), timeout property,
|
|
||||||
usage() -> (used_today, cap) # so telegram_bot stops reading privates
|
|
||||||
- owns: available property, _warn_once, _under_cap() (UTC-day counter,
|
|
||||||
lock), health stamps, and the template method:
|
|
||||||
|
|
||||||
def chat_json(self, system_prompt, user_payload):
|
|
||||||
if not self.available or not self._under_cap():
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return self._request(system_prompt, user_payload) # hook
|
|
||||||
except <provider-agnostic error mapping>:
|
|
||||||
# stamps last_error, returns None
|
|
||||||
|
|
||||||
- subclasses implement ONLY _request(system_prompt, payload) -> dict|None.
|
|
||||||
|
|
||||||
1b. DeepSeekClient(BaseLLMClient) - behavior-identical move
|
|
||||||
Current requests-based logic relocates into _request: same URL,
|
|
||||||
thinking:disabled, response_format:json_object, temperature 0,
|
|
||||||
max_tokens 2000, sort_keys=True payload, _parse_response.
|
|
||||||
Review criterion: Module 19 tests pass UNMODIFIED (they patch
|
|
||||||
client._session, which stays).
|
|
||||||
|
|
||||||
1c. AnthropicClient(BaseLLMClient) - new, official SDK
|
|
||||||
- Official `anthropic` Python SDK (add to requirements.txt), NOT raw
|
|
||||||
requests. LAZY IMPORT inside __init__: package missing -> warn-once +
|
|
||||||
client unavailable, so DeepSeek-only deployments never install it.
|
|
||||||
- Constructor: anthropic.Anthropic(api_key=..., timeout=<configured>,
|
|
||||||
max_retries=0) # house rule: single attempt
|
|
||||||
- _request:
|
|
||||||
resp = self._client.messages.create(
|
|
||||||
model=self._model, # default "claude-opus-5"
|
|
||||||
max_tokens=4000, # covers thinking + JSON reply
|
|
||||||
output_config={"effort": "low"}, # cheap structured takes
|
|
||||||
system=[{"type": "text", "text": system_prompt,
|
|
||||||
"cache_control": {"type": "ephemeral"}}],
|
|
||||||
messages=[{"role": "user",
|
|
||||||
"content": json.dumps(user_payload, sort_keys=True)}],
|
|
||||||
)
|
|
||||||
if resp.stop_reason == "refusal": # Opus 5: HTTP 200, check first
|
|
||||||
return None
|
|
||||||
text = first text block in resp.content
|
|
||||||
return self._parse_json(text) # tolerant: strip ```json fences,
|
|
||||||
# json.loads, dict check
|
|
||||||
- DELIBERATELY ABSENT (would 400 on Claude Opus 5): temperature (removed
|
|
||||||
on Opus 5 - do NOT copy from the DeepSeek client), budget_tokens,
|
|
||||||
prefills, response_format.
|
|
||||||
- Error mapping: APIConnectionError -> "network",
|
|
||||||
APIStatusError -> "http_<code>", parse fail -> "parse"
|
|
||||||
(same last_error kinds the /status display already understands).
|
|
||||||
- Defaults: model claude-opus-5 (config-overridable, e.g. claude-haiku-4-5
|
|
||||||
for ~5x cheaper), timeout 30s (Claude slower than V4-Flash), own
|
|
||||||
daily_request_cap 500.
|
|
||||||
|
|
||||||
1d. Registry + per-consumer routing
|
|
||||||
_PROVIDERS = {"deepseek": DeepSeekClient, "anthropic": AnthropicClient}
|
|
||||||
LLM_CONSUMERS = ("news", "agent_panel")
|
|
||||||
_clients = {} # keyed by PROVIDER name -> two consumers on the
|
|
||||||
# same provider share one instance and one cap
|
|
||||||
_client = None # kept: GLOBAL TEST OVERRIDE
|
|
||||||
|
|
||||||
def get_llm_client(consumer="news"):
|
|
||||||
if _client is not None: # tests: lc._client = stub
|
|
||||||
return _client # -> shadows ALL consumers
|
|
||||||
name = get_app_config().llm_routing.get(consumer, "deepseek")
|
|
||||||
create-once per provider, return _clients[name]
|
|
||||||
|
|
||||||
def reset_llm_client(): # clears _client AND _clients
|
|
||||||
|
|
||||||
Keeping _client as a documented test-override global is what makes the
|
|
||||||
41 existing stub sites work untouched.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
2. CONFIG (app_config.py + config.json.example)
|
|
||||||
================================================================================
|
|
||||||
- New top-level "anthropic" block, mirroring "deepseek":
|
|
||||||
{api_key, model, enabled, timeout_seconds, daily_request_cap}
|
|
||||||
plus anthropic_api_key() resolver: ANTHROPIC_API_KEY env -> config.json -> "".
|
|
||||||
- New top-level "llm" ROUTING block:
|
|
||||||
{"news": "deepseek", "agent_panel": "deepseek"}
|
|
||||||
Validation: keys within {news, agent_panel}; values within registered
|
|
||||||
provider names. Absent block -> {} -> everything on DeepSeek (today's
|
|
||||||
behavior; zero-migration default). AppConfig.llm_routing: Dict[str, str].
|
|
||||||
|
|
||||||
- The target state is then PURE CONFIG - no code changes on switch day:
|
|
||||||
|
|
||||||
"llm": { "news": "deepseek", "agent_panel": "anthropic" },
|
|
||||||
"anthropic": { "api_key": "", "model": "claude-opus-5", "enabled": true,
|
|
||||||
"timeout_seconds": 30, "daily_request_cap": 500 }
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
3. CONSUMER EDITS (mechanical, ~10 lines total)
|
|
||||||
================================================================================
|
|
||||||
- news_engine.py: 4 call sites -> get_llm_client("news")
|
|
||||||
- agent_panel.py: 3 call sites -> get_llm_client("agent_panel");
|
|
||||||
getattr(client, "_timeout", ...) -> public client.timeout
|
|
||||||
- telegram_bot.py _news_health_lines: iterate the DISTINCT providers actually
|
|
||||||
routed (news + panel); per provider print "<name>: used/cap calls today"
|
|
||||||
via the new usage() accessor instead of private fields. DeepSeek keeps its
|
|
||||||
"headlines scored" extra; cap/error warnings work off base-class stamps.
|
|
||||||
- NO changes to prompts, payload builders, caches, or the panel's own
|
|
||||||
max_panels_per_day budget - all provider-agnostic already.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
4. TESTS (new "Module 24: LLM provider routing", ~12 tests, ~200 lines)
|
|
||||||
================================================================================
|
|
||||||
Reuse _StubLLM; new _FakeAnthropic (fake messages.create returning a stub
|
|
||||||
response object) injected as client._client.
|
|
||||||
|
|
||||||
1. app_config: anthropic block parses/validates/defaults empty.
|
|
||||||
2. app_config: llm routing block - valid map round-trips; unknown consumer
|
|
||||||
key, unknown provider name, non-dict -> AppConfigError; absent -> {}.
|
|
||||||
3. anthropic_api_key resolves env -> config.json -> empty.
|
|
||||||
4. Base contract on BOTH subclasses (parametrized): keyless/disabled -> no
|
|
||||||
call; daily cap blocks + UTC-roll reset; failure stamps last_error,
|
|
||||||
success stamps last_ok.
|
|
||||||
5. AnthropicClient parse: clean JSON -> dict; ```json-fenced -> dict;
|
|
||||||
non-JSON -> None; stop_reason "refusal" -> None; no text block -> None.
|
|
||||||
6. AnthropicClient request shape: NO temperature key, max_retries=0,
|
|
||||||
cache_control present on system block (assert on captured kwargs).
|
|
||||||
7. AnthropicClient with `anthropic` package missing (simulated) ->
|
|
||||||
available False, chat_json None, no crash at import.
|
|
||||||
8. Routing: no llm block -> both consumers get the SAME DeepSeek singleton
|
|
||||||
(shared cap); routed config -> news gets DeepSeekClient while
|
|
||||||
agent_panel gets AnthropicClient; same-provider consumers share one
|
|
||||||
instance.
|
|
||||||
9. Global lc._client override shadows BOTH consumers (locks the 41 legacy
|
|
||||||
stub sites' semantics).
|
|
||||||
10. reset_llm_client() clears both the override and the per-provider cache.
|
|
||||||
11. End-to-end: routing agent_panel->anthropic, attach_panel reaches the
|
|
||||||
anthropic fake (5 calls) while a news call reaches the deepseek stub.
|
|
||||||
12. telegram _news_health_lines renders per-provider lines from usage()
|
|
||||||
without touching private fields.
|
|
||||||
|
|
||||||
GATE: Modules 19 and 23 must pass UNMODIFIED - the no-regression proof for
|
|
||||||
the refactor.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
5. DOCS
|
|
||||||
================================================================================
|
|
||||||
- CLAUDE.md: llm_client bullet -> provider-abstracted (BaseLLMClient;
|
|
||||||
deepseek + anthropic registered); config.json "llm" block routes consumers;
|
|
||||||
absent = all-DeepSeek. Add ANTHROPIC_API_KEY to env-var list. Test counts.
|
|
||||||
- config.json.example: both new blocks.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
COMMIT SEQUENCE (each commit green)
|
|
||||||
================================================================================
|
|
||||||
1. llm_client: extract BaseLLMClient; DeepSeekClient unchanged behavior
|
|
||||||
(+ base-contract tests) - pure refactor, Modules 19/23 green untouched.
|
|
||||||
2. config: anthropic block + llm consumer-routing block (+ config tests).
|
|
||||||
3. llm_client: AnthropicClient (official SDK, lazy import) + requirements.txt
|
|
||||||
(+ parse/shape tests).
|
|
||||||
4. routing: per-consumer get_llm_client + health lines - consumer call-site
|
|
||||||
edits, telegram usage() lines, routing/e2e tests.
|
|
||||||
5. docs: CLAUDE.md + example config.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
VERIFICATION
|
|
||||||
================================================================================
|
|
||||||
- python test_all.py green at every commit (~374 -> ~386).
|
|
||||||
- Live smoke A (no behavior change): default config, main.py --panel ->
|
|
||||||
still DeepSeek, /status unchanged.
|
|
||||||
- Live smoke B (the goal): llm.agent_panel="anthropic" + real
|
|
||||||
ANTHROPIC_API_KEY, run --panel -> one real Claude panel (5 requests,
|
|
||||||
validates JSON contract + latency + refusal-free path) while a news
|
|
||||||
refresh still hits DeepSeek; /status shows two provider lines.
|
|
||||||
- Backtest untouched by construction (no LLM in replay path); one 30-day
|
|
||||||
run as a formality.
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
EFFORT ESTIMATE
|
|
||||||
================================================================================
|
|
||||||
0 new files, 8 modified, ~520 lines, est. 3-5 hours.
|
|
||||||
|
|
||||||
llm_client.py base class + AnthropicClient + routing +235
|
|
||||||
test_all.py Module 24 +200
|
|
||||||
app_config.py anthropic + llm blocks, resolver +55
|
|
||||||
telegram_bot.py per-provider health lines +15
|
|
||||||
news_engine.py consumer tags on call sites +4
|
|
||||||
agent_panel.py consumer tags on call sites +4
|
|
||||||
config.json.example blocks +14
|
|
||||||
requirements.txt anthropic +1
|
|
||||||
CLAUDE.md docs +10
|
|
||||||
|
|
||||||
================================================================================
|
|
||||||
KEY DECISIONS EMBEDDED IN THIS PLAN
|
|
||||||
================================================================================
|
|
||||||
- Official Anthropic SDK, not raw requests (current API guidance for Python;
|
|
||||||
lazy import keeps it optional for DeepSeek-only installs).
|
|
||||||
- Default Claude model: claude-opus-5 ($5/$25 per MTok; a full 5-call panel
|
|
||||||
~ 8k in / 500 out ~ $0.05). Config-overridable to claude-haiku-4-5.
|
|
||||||
- Two Claude gotchas handled: temperature is REMOVED on Opus 5 (sending it
|
|
||||||
400s - must not be copied from the DeepSeek request body), and refusals
|
|
||||||
arrive as HTTP 200 with stop_reason "refusal" (checked before parsing,
|
|
||||||
mapped to the normal None/fallback path).
|
|
||||||
- Caps are per-provider, not per-consumer: two consumers on one provider
|
|
||||||
share its cap (today's semantics); splitting across two providers doubles
|
|
||||||
total headroom naturally.
|
|
||||||
- The panel's own max_panels_per_day budget is untouched - it sits above
|
|
||||||
whichever provider serves it.
|
|
||||||
- Not in scope: static/news_summary.php (own PHP client/key), streaming,
|
|
||||||
tool use, per-consumer model overrides (add a "model" key per consumer
|
|
||||||
later if ever needed).
|
|
||||||
================================================================================
|
|
||||||
+3
-2
@@ -65,8 +65,9 @@ class TiingoDataProvider(BaseDataProvider):
|
|||||||
}
|
}
|
||||||
|
|
||||||
# Window padding over the theoretical bar span: FX is closed ~2/7 of
|
# Window padding over the theoretical bar span: FX is closed ~2/7 of
|
||||||
# the week (Fri 22:00 → Sun 22:00 UTC), so a calendar window must be
|
# the week (Fri 17:00 → Sun 18:00 New York; see providers/base.py), so
|
||||||
# wider than `bars × candle_seconds` to yield the requested count.
|
# a calendar window must be wider than `bars × candle_seconds` to yield
|
||||||
|
# the requested count.
|
||||||
WINDOW_PAD_FACTOR = 1.6
|
WINDOW_PAD_FACTOR = 1.6
|
||||||
|
|
||||||
def __init__(self, config: PlatformConfig, api_key: Optional[str] = None):
|
def __init__(self, config: PlatformConfig, api_key: Optional[str] = None):
|
||||||
|
|||||||
@@ -1,116 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Scratchpad — Oanda vs Tiingo candle parity check
|
|
||||||
=================================================
|
|
||||||
Fetches the last N closed XAU/USD candles from Oanda and Tiingo for
|
|
||||||
M5/M15/H1 via the normal provider stack (make_data_provider →
|
|
||||||
fetch_ohlcv: forming-candle drop + validate/clean included, i.e. exactly
|
|
||||||
what the signal pipeline consumes) and reports how well they line up:
|
|
||||||
|
|
||||||
* last-bar timestamp per provider (feed-lag check)
|
|
||||||
* timestamp mismatches (bars present in one feed only)
|
|
||||||
* per-bar OHLC diff table for the common tail
|
|
||||||
* max/mean |diff| per column, with the Close diff also in % of price
|
|
||||||
|
|
||||||
Both feeds are mid-price (Oanda mid of MBA candles; Tiingo resampled
|
|
||||||
mid), so small diffs are expected — this measures how small.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
venv/bin/python scripts/compare_oanda_tiingo.py [--n 10]
|
|
||||||
"""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
|
||||||
|
|
||||||
from config import PlatformConfig, Timeframe, XAUUSD
|
|
||||||
from providers import make_data_provider
|
|
||||||
|
|
||||||
TIMEFRAMES = [Timeframe.M5, Timeframe.M15, Timeframe.H1]
|
|
||||||
OHLC = ["Open", "High", "Low", "Close"]
|
|
||||||
|
|
||||||
|
|
||||||
def fetch_tail(provider, tf, n):
|
|
||||||
"""Last n closed bars from one provider, or None on failure."""
|
|
||||||
try:
|
|
||||||
df = provider.fetch_ohlcv(symbol=XAUUSD, timeframe=tf)
|
|
||||||
except Exception as e: # scratchpad: report, don't crash
|
|
||||||
print(f" !! {provider.name} {tf.value} fetch failed: {e}")
|
|
||||||
return None
|
|
||||||
return df.tail(n)
|
|
||||||
|
|
||||||
|
|
||||||
def compare_tf(tf, oanda_df, tiingo_df, n):
|
|
||||||
print(f"\n{'=' * 78}\n{tf.value} — last {n} closed bars\n{'=' * 78}")
|
|
||||||
if oanda_df is None or tiingo_df is None:
|
|
||||||
print(" skipped (a provider failed)")
|
|
||||||
return
|
|
||||||
|
|
||||||
print(f" Oanda last bar: {oanda_df.index[-1]} "
|
|
||||||
f"Tiingo last bar: {tiingo_df.index[-1]}")
|
|
||||||
|
|
||||||
common = oanda_df.index.intersection(tiingo_df.index)
|
|
||||||
only_o = oanda_df.index.difference(tiingo_df.index)
|
|
||||||
only_t = tiingo_df.index.difference(oanda_df.index)
|
|
||||||
if len(only_o):
|
|
||||||
print(f" bars only in Oanda : {[str(t) for t in only_o]}")
|
|
||||||
if len(only_t):
|
|
||||||
print(f" bars only in Tiingo: {[str(t) for t in only_t]}")
|
|
||||||
if not len(common):
|
|
||||||
print(" NO common timestamps — feeds are on different bar grids!")
|
|
||||||
return
|
|
||||||
|
|
||||||
o = oanda_df.loc[common, OHLC]
|
|
||||||
t = tiingo_df.loc[common, OHLC]
|
|
||||||
diff = o - t
|
|
||||||
|
|
||||||
print(f"\n {len(common)}/{n} bars share timestamps "
|
|
||||||
f"(window {common[0]} → {common[-1]})\n")
|
|
||||||
|
|
||||||
# Per-bar Close table — the number that matters most.
|
|
||||||
hdr = f" {'bar close (UTC)':<20}{'Oanda C':>12}{'Tiingo C':>12}{'ΔC ($)':>10}"
|
|
||||||
print(hdr + f"{'ΔO':>8}{'ΔH':>8}{'ΔL':>8}")
|
|
||||||
for ts in common:
|
|
||||||
row = (f" {str(ts):<20}{o.loc[ts, 'Close']:>12.2f}"
|
|
||||||
f"{t.loc[ts, 'Close']:>12.2f}{diff.loc[ts, 'Close']:>10.3f}")
|
|
||||||
row += (f"{diff.loc[ts, 'Open']:>8.3f}{diff.loc[ts, 'High']:>8.3f}"
|
|
||||||
f"{diff.loc[ts, 'Low']:>8.3f}")
|
|
||||||
print(row)
|
|
||||||
|
|
||||||
print("\n summary over common bars:")
|
|
||||||
for col in OHLC:
|
|
||||||
d = diff[col].abs()
|
|
||||||
print(f" {col:<6} max |Δ| = {d.max():8.4f} mean |Δ| = {d.mean():8.4f}")
|
|
||||||
px = o["Close"].iloc[-1]
|
|
||||||
dc = diff["Close"].abs()
|
|
||||||
print(f" Close Δ as % of price: max {dc.max() / px * 100:.4f}% "
|
|
||||||
f"mean {dc.mean() / px * 100:.4f}%")
|
|
||||||
exact = (diff[OHLC].abs() < 1e-9).all(axis=1).sum()
|
|
||||||
print(f" bars with identical OHLC: {exact}/{len(common)}")
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
ap = argparse.ArgumentParser()
|
|
||||||
ap.add_argument("--n", type=int, default=10,
|
|
||||||
help="bars per timeframe to compare (default 10)")
|
|
||||||
args = ap.parse_args()
|
|
||||||
|
|
||||||
cfg = PlatformConfig()
|
|
||||||
oanda = make_data_provider(cfg, "oanda")
|
|
||||||
tiingo = make_data_provider(cfg, "tiingo")
|
|
||||||
|
|
||||||
for tf in TIMEFRAMES:
|
|
||||||
o_df = fetch_tail(oanda, tf, args.n)
|
|
||||||
t_df = fetch_tail(tiingo, tf, args.n)
|
|
||||||
compare_tf(tf, o_df, t_df, args.n)
|
|
||||||
|
|
||||||
print("\nDone. Note: Oanda=mid of MBA candles, Tiingo=resampled mid — "
|
|
||||||
"sub-$1 diffs on gold are normal feed divergence, not a bug.")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -11,7 +11,7 @@
|
|||||||
* - CACHE FOREVER: headlines are immutable, so each id costs at most
|
* - CACHE FOREVER: headlines are immutable, so each id costs at most
|
||||||
* one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
|
* one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
|
||||||
* the whitelist on purpose: summaries stay reachable after the
|
* the whitelist on purpose: summaries stay reachable after the
|
||||||
* headline ages out of the 24 h feed window.
|
* headline ages out of the 4 h feed window.
|
||||||
* - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
|
* - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
|
||||||
* errors (a perms mistake must not mean unmetered spend).
|
* errors (a perms mistake must not mean unmetered spend).
|
||||||
*
|
*
|
||||||
@@ -112,9 +112,7 @@ if (!is_array($news)) {
|
|||||||
}
|
}
|
||||||
$hit = null;
|
$hit = null;
|
||||||
foreach ($news['headlines'] ?? [] as $h) {
|
foreach ($news['headlines'] ?? [] as $h) {
|
||||||
// Fallback sha1(): a publisher that predates the id field still
|
$hid = $h['id'] ?? '';
|
||||||
// serves a valid whitelist during the restart window.
|
|
||||||
$hid = $h['id'] ?? sha1($h['headline'] ?? '');
|
|
||||||
if ($hid === $id) {
|
if ($hid === $id) {
|
||||||
$hit = $h;
|
$hit = $h;
|
||||||
break;
|
break;
|
||||||
|
|||||||
+2
-2
@@ -5289,9 +5289,9 @@ def _():
|
|||||||
|
|
||||||
|
|
||||||
# ══════════════════════════════════════════════════════════════════════
|
# ══════════════════════════════════════════════════════════════════════
|
||||||
# MODULE 21: 2026-08-04 audit-fix regressions (execution lifecycle)
|
# MODULE 21: 2026-08-04 audit-fix regressions (parity, providers, backtester)
|
||||||
# ══════════════════════════════════════════════════════════════════════
|
# ══════════════════════════════════════════════════════════════════════
|
||||||
print("\n═══ Module 21: 2026-08-04 audit fixes — execution lifecycle ═══")
|
print("\n═══ Module 21: 2026-08-04 audit fixes — parity, providers, backtester ═══")
|
||||||
|
|
||||||
|
|
||||||
@test("GPU parity: ALL indicators + composite identical on every prefix")
|
@test("GPU parity: ALL indicators + composite identical on every prefix")
|
||||||
|
|||||||
Reference in New Issue
Block a user