chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
This commit is contained in:
Harwinder Sekhon
+1
-5
@@ -3,9 +3,6 @@
|
||||
.env.local
|
||||
config.json
|
||||
|
||||
# Oanda daemon state
|
||||
oanda_state.json
|
||||
|
||||
# Editor / tooling
|
||||
.claude/
|
||||
.vscode/
|
||||
@@ -17,6 +14,7 @@ __pycache__/
|
||||
*.egg-info/
|
||||
.venv/
|
||||
venv/
|
||||
.coverage
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
@@ -30,8 +28,6 @@ backtest/cache/
|
||||
backtest/*.csv
|
||||
backtest/research/*.parquet
|
||||
backtest/research/duka_m5_boundary_spreads.csv
|
||||
backtest/trailgrid/
|
||||
backtest/trailgrid5m/
|
||||
backtest/__pycache__/
|
||||
.env.*
|
||||
logs/
|
||||
|
||||
+4
-3
@@ -26,9 +26,10 @@ Two invariants, both load-bearing (CLAUDE.md rules 1 and 4):
|
||||
so replayed books can never see a panel field. Backtest parity holds
|
||||
by construction.
|
||||
|
||||
Cost model: every full panel spends 5 requests (4 personas + 1
|
||||
synthesizer) against the shared DeepSeek daily cap (llm_client). Two
|
||||
layers bound the spend on top of that cap:
|
||||
Cost model: every full panel spends 4 requests (the personas; 5 with
|
||||
verdict_source="chair", which adds the synthesizer) against the shared
|
||||
DeepSeek daily cap (llm_client). Two layers bound the spend on top of
|
||||
that cap:
|
||||
|
||||
- ``max_panels_per_day`` (config, default 20) — a UTC-day budget of
|
||||
panel *attempts*, mirroring the client's request cap.
|
||||
|
||||
@@ -6,7 +6,7 @@ analyze_signal_log.py — signal_log performance evaluator
|
||||
READ-ONLY analysis of logs/signals.db: how are the logged live signals
|
||||
and the advisory LLM panel actually performing?
|
||||
|
||||
Three lenses:
|
||||
Six lenses:
|
||||
|
||||
1. TRADE REPLAY — every logged row carrying a trade_setup is deduped
|
||||
(telegram + dashboard log the same scan; a persisting signal re-fires
|
||||
@@ -59,6 +59,7 @@ Usage:
|
||||
python analyze_signal_log.py --parity # surface parity only
|
||||
python analyze_signal_log.py --stored-candles # DB-only replay
|
||||
python analyze_signal_log.py --stored-candles --news # + news lens
|
||||
python analyze_signal_log.py --scan-outcomes # forward-move table
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@@ -140,7 +141,8 @@ class SignalEvent:
|
||||
_COLS = ("id, logged_at, source, timeframe, signal, score, quality,"
|
||||
" price, market_regime, setup_json, panel_verdict,"
|
||||
" panel_confidence, action")
|
||||
# Columns added by the 2026-09 observability work; absent on older DBs.
|
||||
# Columns an older DB may lack (added by later schema versions); loaded
|
||||
# only when PRAGMA table_info reports them.
|
||||
_OPTIONAL_COLS = ("gate_reason", "actionable", "config_fingerprint",
|
||||
"advisory_event_id", "advisory_outcome_status",
|
||||
"advisory_r_multiple", "scan_kind", "news_sentiment",
|
||||
|
||||
@@ -21,8 +21,6 @@ the config.json.example defaults, every produced PlatformConfig is
|
||||
byte-identical to the pure-code defaults. apply_pair_overrides() runs
|
||||
in the pair factory BEFORE the validated per-timeframe deltas, so the
|
||||
bot, CLI and backtester all see the same overlay (live == backtest).
|
||||
|
||||
Spec: docs/superpowers/specs/2026-07-21-centralized-config-design.md
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
@@ -1,61 +1,21 @@
|
||||
Note down entery and exit Why we took trade?
|
||||
What were each indicator signal on that specific time when we took trade
|
||||
How many indicators agreed at trade time
|
||||
Always backtest using engine.py and signal_aggregator.py
|
||||
Dont take trades on weekends and Fridays
|
||||
Keep TP small 1.3 Reward ratio
|
||||
At the end of test make table of results and print all values you observed.
|
||||
Trade context
|
||||
|
||||
Time of day and trading session (Asian/London/NY/overlap)
|
||||
Day of week (track Mon/Tue/Wed/Thu performance separately)
|
||||
Market regime at entry: trending, ranging, or choppy (use ATR or ADX)
|
||||
Volatility level (ATR value or ATR percentile vs recent average)
|
||||
Pair/instrument traded
|
||||
News proximity (high-impact news within 30 min before/after entry)
|
||||
|
||||
Trade execution details
|
||||
|
||||
Spread at entry in pips
|
||||
Slippage on entry and exit (planned vs actual fill)
|
||||
Time in trade (minutes/hours from entry to exit)
|
||||
Maximum Adverse Excursion (MAE): how far against you it went
|
||||
Maximum Favorable Excursion (MFE): how far in profit before pullback
|
||||
|
||||
Risk metrics
|
||||
|
||||
Max consecutive losses (streak)
|
||||
Max consecutive wins (streak)
|
||||
Expectancy: (Win% x Avg Win) minus (Loss% x Avg Loss)
|
||||
Expectancy per hour in trade
|
||||
Recovery factor: Total R divided by Max DD
|
||||
|
||||
Indicator/signal analysis
|
||||
|
||||
Each indicator's standalone win rate when it agreed with the trade
|
||||
Confluence level performance (2 vs 3 vs 4+ indicators agreeing)
|
||||
Which indicator disagreed on losing trades (find the "canary")
|
||||
|
||||
Decision quality
|
||||
|
||||
Rate each setup A/B/C at entry, before knowing outcome
|
||||
Verify A-grade setups actually win more often
|
||||
Track system trades vs discretionary deviations separately
|
||||
|
||||
Process notes
|
||||
|
||||
Screenshot or chart snapshot at entry and exit
|
||||
Emotional state (calm, FOMO, revenge, tired)
|
||||
What would have happened if you held longer or exited sooner
|
||||
|
||||
Aggregate analysis at end of backtest
|
||||
|
||||
Equity curve, not just total R
|
||||
Performance breakdown by session, day, volatility regime
|
||||
Win rate over rolling 20-trade windows (check for edge decay)
|
||||
|
||||
Critical extra
|
||||
"Do not trade" log: setups flagged but skipped, and what happened after
|
||||
Backtest reporting checklist — ForexBot2 (XAU/USD)
|
||||
==================================================
|
||||
Every backtest report must record:
|
||||
- The exact command line (window, --primary-tf, --spread, and any --set /
|
||||
--quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
|
||||
- Trades, total R, win rate, profit factor and max drawdown for the whole
|
||||
window AND for both halves, at both cost conventions ($0.65 measured-Oanda
|
||||
headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
|
||||
- Per trade (the exported CSV already carries these): entry/exit time and
|
||||
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
||||
indicator's signal / value / agreement at decision time.
|
||||
- Breakdowns by session/hour, weekday, regime and volatility regime
|
||||
(backtest/research/book_report.py) and, for any candidate change, the
|
||||
removed/added/shifted trade diff against the control book
|
||||
(backtest/research/book_diff.py).
|
||||
- The md5 of the exported book whenever the run is a parity or regression check.
|
||||
Replays go through the real signal code (analyze_from_dataframes) — never a
|
||||
re-implementation.
|
||||
|
||||
|
||||
--- 2026-09-06: decision-time feature export ---
|
||||
|
||||
@@ -95,7 +95,7 @@ Cost guardrails already in place: 5 requests/panel, `max_panels_per_day`
|
||||
injection (technical persona only), prompt-contract assertions,
|
||||
panel row update, rowid hand-off + no-op paths). Existing attach tests
|
||||
stub `ap._dp` to stay off the network.
|
||||
- **Live dry test** (`panel_payload_drytest_2026-08-15.py`, fabricated
|
||||
- **Live dry test** (`panel_payload_drytest_2026-08-15.py` — script removed 2026-09-06, raw replies kept in the `.jsonl` — fabricated
|
||||
numbers only, 13 real DeepSeek requests total): **7/7 probes pass** —
|
||||
coherent-bullish → support@70; bearish-vs-BUY → oppose@85; genuinely
|
||||
mixed → neutral@30; empty payload → neutral@20 (contract); no invented
|
||||
|
||||
@@ -164,7 +164,7 @@ def base_metrics(r: pd.Series, df: pd.DataFrame | None = None) -> dict:
|
||||
|
||||
|
||||
def _pf(r: pd.Series) -> float:
|
||||
"""Profit factor, hour_analysis-compatible (loss side is r <= 0)."""
|
||||
"""Profit factor (loss side is r <= 0)."""
|
||||
pos = r[r > 0].sum()
|
||||
neg = -r[r <= 0].sum()
|
||||
return float(pos / max(neg, 1e-9))
|
||||
|
||||
@@ -6,9 +6,8 @@ Usage:
|
||||
[--start YYYY-MM-DD --end YYYY-MM-DD] [--out DIR]
|
||||
|
||||
A "burst" is one bar whose range is >= K x the PRIOR bar's Wilder ATR(14) —
|
||||
the same definition burst_profile.py uses (that module computes it inline
|
||||
inside burst_table() and exposes no reusable helper, so the formula is
|
||||
replicated here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
||||
the same definition every burst study in this folder has used (spelled
|
||||
out here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
||||
TR.ewm(alpha=1/14, min_periods=14, adjust=False).mean(), ratio =
|
||||
(H-L) / ATR.shift(1)).
|
||||
|
||||
@@ -60,7 +59,7 @@ def load_bars(parquet: str, start: str | None = None,
|
||||
|
||||
|
||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
"""Wilder ATR — identical to the inline formula in burst_profile.py."""
|
||||
"""Wilder ATR (the definition every burst study in this folder uses)."""
|
||||
h, l, c = df["High"], df["Low"], df["Close"]
|
||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||
axis=1).max(axis=1)
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
"""Burst-candle frequency by UTC hour — the empirical "hedge fund hours".
|
||||
|
||||
Usage:
|
||||
python backtest/research/burst_profile.py PARQUET_GLOB
|
||||
|
||||
Per UTC bar-close hour of the given bars: % of bars whose range is
|
||||
>= K x the PRIOR bar's Wilder ATR(14), for K in {3,4,5}; mean bar
|
||||
range; and mean signed drift over the 6 bars following a >=4x burst
|
||||
(does price continue or snap back?). Analysis tooling only.
|
||||
"""
|
||||
import glob
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def burst_table(parquet_glob: str, bar_minutes: int = 5):
|
||||
files = sorted(glob.glob(parquet_glob))
|
||||
if not files:
|
||||
raise SystemExit(f"no parquet matches {parquet_glob}")
|
||||
df = pd.concat([pd.read_parquet(f) for f in files]).sort_index()
|
||||
df = df[~df.index.duplicated(keep="last")]
|
||||
h, l, c = df["High"], df["Low"], df["Close"]
|
||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||
axis=1).max(axis=1)
|
||||
atr = tr.ewm(alpha=1 / 14, min_periods=14, adjust=False).mean()
|
||||
ratio = (h - l) / atr.shift(1)
|
||||
close_hour = (df.index + pd.Timedelta(minutes=bar_minutes)).hour
|
||||
fwd6 = c.shift(-6) - c # signed move over the 6 bars after this one
|
||||
out = pd.DataFrame({
|
||||
"bars": ratio.groupby(close_hour).size(),
|
||||
**{f"pct_ge_{k}x": (ratio >= k).groupby(close_hour).mean() * 100
|
||||
for k in (3, 4, 5)},
|
||||
"mean_range": (h - l).groupby(close_hour).mean(),
|
||||
"post4x_drift6": fwd6[ratio >= 4].groupby(close_hour[ratio >= 4]).mean(),
|
||||
})
|
||||
return out.round(3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(burst_table(sys.argv[1],
|
||||
int(sys.argv[2]) if len(sys.argv) > 2 else 5).to_string())
|
||||
@@ -0,0 +1,81 @@
|
||||
# Feed Divergence Study — Oanda vs Tiingo vs Twelve Data (Phase 1)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Test plan:** `docs/data_source_change_test_plan.md`
|
||||
**Tool:** `scripts/feed_divergence_study.py` (fetch_range over 365d, parquet-cached)
|
||||
**Window:** 2025-08-14 → 2026-08-14 (12 months); Oanda-vs-TD truncated at TD cache head (2026-07-23)
|
||||
|
||||
## Headline
|
||||
|
||||
Three mid-price feeds for the *same* XAU/USD instrument are **not** interchangeable:
|
||||
|
||||
| Pair | mean \|ΔClose\| | p95 \|ΔClose\| | direction agree (M5/M15/H1) | bias | extra bars |
|
||||
|---|---|---|---|---|---|
|
||||
| Oanda vs **Tiingo** | ~$0.13 | ~$0.29 | 97.7% / 98.3% / 98.5% | ~0 | Tiingo +3–5% (weekend) |
|
||||
| Oanda vs **Twelve Data** | ~$3.0 | ~$11.9 | 81.0% / 88.7% / 92.7% | −$1.06 (TD high) | — |
|
||||
|
||||
- **Oanda ↔ Tiingo are near-identical** on shared bars (~1–2% of ATR on close), but Tiingo emits ~22 hours of Sunday bars/week that Oanda omits.
|
||||
- **Oanda ↔ Twelve Data diverge massively** — ~$3 mean close gap (≈30% of M5 ATR, ~30% of M15 ATR), a systematic ~$1 bias (TD quoted ~$1 high), and 19% of M5 bars even disagree on bar *direction*. This is the documented "Oanda/TD divergence" that motivated re-pinning the books on Oanda bars.
|
||||
|
||||
## Oanda vs Tiingo — detailed
|
||||
|
||||
Both feeds: same bar grid, same head/tail timestamps. Tiingo simply has more bars (Sunday bars).
|
||||
|
||||
### Coverage (12 months)
|
||||
|
||||
| TF | Oanda bars | Tiingo bars | shared | Oanda-only | Tiingo-only |
|
||||
|---|---|---|---|---|---|
|
||||
| M5 | 70,901 | 73,241 | 70,807 (96.6%) | 94 (0.13%) | 2,434 (3.32%) |
|
||||
| M15 | 23,640 | 24,922 | 23,615 (94.7%) | 25 (0.11%) | 1,307 (5.24%) |
|
||||
| H1 | 5,912 | 6,246 | 5,909 (94.6%) | 3 (0.05%) | 337 (5.40%) |
|
||||
| H4 | 1,597 | 1,657 | 1,597 (96.4%) | 0 | 60 (3.62%) |
|
||||
|
||||
- **Tiingo-only bars cluster exclusively on Sundays** (e.g. M15: 88/Sunday; H1: 22/Sunday; H4: 5/Sunday). Tiingo's FX endpoint resamples continuously and does **not** apply the FX weekend calendar — it emits ~22h of flat/stale Sunday bars that Oanda omits.
|
||||
- Oanda-only bars cluster on holidays (2025-11-27 Thanksgiving, 2025-12-10, 2026-06-25) — a handful of bars, negligible.
|
||||
|
||||
### Per-bar OHLC diffs (Oanda − Tiingo, $)
|
||||
|
||||
| TF | Close mean | Close p95 | Close max | \|ΔClose\| / ATR14 (mean / p95) |
|
||||
|---|---|---|---|---|
|
||||
| M5 | $0.123 | $0.285 | $26.10 | 2.1% / 5.0% |
|
||||
| M15 | $0.128 | $0.290 | $26.10 | 1.3% / 2.9% |
|
||||
| H1 | $0.145 | $0.335 | $26.10 | 0.7% / 1.6% |
|
||||
| H4 | $0.168 | $0.340 | $17.53 | 0.4% / 0.8% |
|
||||
|
||||
- Bias is ~zero on every TF (no systematic one-feed-high).
|
||||
- Tail outliers (p99/max up to $26–88) occur during fast moves — the two feeds' bar extremes disagree on the exact tick, as expected.
|
||||
- **Direction agreement is 97.7–98.4%**, return correlation 0.995–0.9997.
|
||||
- Divergence is concentrated at the Friday-close / Sunday-reopen boundary (UTC hours 20–22; mean |ΔClose| ~2–4× the 08–19h baseline).
|
||||
|
||||
### Phase 1 pass/fail vs plan thresholds
|
||||
|
||||
| Criterion | Plan threshold | Result | Verdict |
|
||||
|---|---|---|---|
|
||||
| timestamp overlap | ≥ 99.5% | 94.6–96.6% | **FAIL** (Tiingo weekend bars) |
|
||||
| Close p95 \|Δ\| | ≤ 0.01% of price | ~0.0067% | PASS |
|
||||
| no systematic bias | mean Δ ≈ 0 | ~0 | PASS |
|
||||
|
||||
The overlap "failure" is a *coverage* difference, not a data-quality one: the shared bars agree tightly, but Tiingo's extra Sunday bars mean the feeds have a different bar *count*. The signal pipeline's own `_is_forex_weekend` filter (NY-anchored) gates those Sunday bars, so live impact depends on whether any code trusts the feed's raw bar set directly.
|
||||
|
||||
## Oanda vs Twelve Data — detailed (M5/M15/H1, cached TD parquets)
|
||||
|
||||
| TF | common | mean \|ΔClose\| | median \|ΔClose\| | p95 | max | bias (TD−Oanda) | dir agree |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| M5 | 66,246 | $2.999 | $1.118 | $11.934 | $101.85 | +1.05 | 80.96% |
|
||||
| M15 | 22,090 | $2.951 | $1.051 | $11.848 | $101.85 | +1.08 | 88.71% |
|
||||
| H1 | 5,526 | $2.902 | $0.985 | $11.908 | $70.94 | +1.06 | 92.67% |
|
||||
|
||||
- The ~$3 mean / ~$1 median / ~$12 p95 gap is **~23× larger** than the Oanda–Tiingo gap, and the ~$1 bias is systematic (Twelve Data quotes gold ~$1 high vs Oanda's true mid).
|
||||
- M5 direction agreement of **81%** means one bar in five disagrees on direction — M5 signals on TD bars are effectively a *different instrument* than on Oanda bars.
|
||||
- This matches the prior Oanda/TD M15 study of 2026-07-24 (`oanda_td_divergence.py`, removed 2026-09-06 as superseded by this report; mean |ΔClose| ~$3.3) and confirms the standing rule: **pinned Twelve-Data books are not comparable to Oanda live signals without a re-pin.**
|
||||
|
||||
## Conclusions for the data-source switch
|
||||
|
||||
1. **Oanda → Tiingo (or back) is signal-safe in magnitude** (price diffs ~1–2% of ATR, 98% direction agreement), *provided* the weekend-bar coverage difference is handled — Tiingo's Sunday bars must be gated by the aggregator's NY-anchored weekend filter, not by raw bar count.
|
||||
2. **Oanda → Twelve Data is a recalibration, not a formality.** Phase 2/3 must quantify the actual signal/book delta; the $3/$1 bias alone guarantees a large fraction of marginal bars flip.
|
||||
3. Re-pinning the books on Oanda bars (Phase 3) remains mandatory before any TD-pinned baseline is treated as a live expectation.
|
||||
|
||||
## Next steps
|
||||
|
||||
- Phase 2: signal-level replay parity (`analyze_from_dataframes` on both feeds) to turn price-diff stats into flip rates.
|
||||
- Phase 3: backtest re-pin on Oanda bars at $0.30 and $0.65.
|
||||
@@ -0,0 +1,2 @@
|
||||
tf,a,b,a_bars,b_bars,common,a_only,b_only,dir_agree_pct,ret_corr,mean_abs_dclose,p95_abs_dclose,max_abs_dclose,dclose_pct_atr_mean
|
||||
5min,oanda,tiingo,70901,73241,70807,94,2434,97.674000,0.994970,0.123070,0.285000,26.105000,2.147942
|
||||
|
@@ -1,158 +0,0 @@
|
||||
"""Fetch XAU/USD M15 candles from the Oanda v20 REST API.
|
||||
|
||||
Mechanical data-fetching infrastructure only — no analysis here.
|
||||
Paginates GET /v3/instruments/XAU_USD/candles from a start cursor
|
||||
through "now" and writes the result to a parquet file.
|
||||
|
||||
Run from the repo root:
|
||||
venv/bin/python backtest/research/fetch_oanda_m15.py
|
||||
"""
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
import requests
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from app_config import oanda_api_token, oanda_base_url # noqa: E402
|
||||
|
||||
INSTRUMENT = "XAU_USD"
|
||||
GRANULARITY = "M15"
|
||||
START = datetime(2025, 7, 1, 0, 0, 0, tzinfo=timezone.utc)
|
||||
OUT_PATH = Path(__file__).parent / "oanda_xau_m15_20250701_now.parquet"
|
||||
COUNT = 5000
|
||||
REQUEST_SLEEP_S = 0.5
|
||||
FRESHNESS_WINDOW = timedelta(minutes=30)
|
||||
|
||||
|
||||
def _rfc3339(dt: datetime) -> str:
|
||||
return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
def fetch_all_candles():
|
||||
token = oanda_api_token()
|
||||
base_url = oanda_base_url()
|
||||
if not token:
|
||||
raise RuntimeError(
|
||||
"No Oanda API token resolved (env OANDA_API_TOKEN / "
|
||||
"config.json oanda.api_token). Aborting.")
|
||||
|
||||
url = f"{base_url}/v3/instruments/{INSTRUMENT}/candles"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Accept-Datetime-Format": "RFC3339",
|
||||
}
|
||||
|
||||
cursor = START
|
||||
include_first = True
|
||||
all_rows = []
|
||||
n_requests = 0
|
||||
|
||||
while True:
|
||||
params = {
|
||||
"granularity": GRANULARITY,
|
||||
"price": "MBA",
|
||||
"count": COUNT,
|
||||
"dailyAlignment": 0,
|
||||
"alignmentTimezone": "Etc/UTC",
|
||||
"from": _rfc3339(cursor),
|
||||
"includeFirst": "true" if include_first else "false",
|
||||
}
|
||||
|
||||
resp = requests.get(url, headers=headers, params=params, timeout=30)
|
||||
n_requests += 1
|
||||
if resp.status_code != 200:
|
||||
raise RuntimeError(
|
||||
f"Oanda API request failed: status={resp.status_code}\n"
|
||||
f"URL: {resp.url}\n"
|
||||
f"Response body: {resp.text}")
|
||||
|
||||
payload = resp.json()
|
||||
candles = payload.get("candles", [])
|
||||
|
||||
for c in candles:
|
||||
if not c.get("complete", False):
|
||||
continue
|
||||
mid = c["mid"]
|
||||
bid = c["bid"]
|
||||
ask = c["ask"]
|
||||
all_rows.append({
|
||||
"Datetime": pd.Timestamp(c["time"]).tz_convert("UTC")
|
||||
if pd.Timestamp(c["time"]).tzinfo is not None
|
||||
else pd.Timestamp(c["time"]).tz_localize("UTC"),
|
||||
"Open": float(mid["o"]),
|
||||
"High": float(mid["h"]),
|
||||
"Low": float(mid["l"]),
|
||||
"Close": float(mid["c"]),
|
||||
"Volume": int(c["volume"]),
|
||||
"bid_o": float(bid["o"]),
|
||||
"bid_h": float(bid["h"]),
|
||||
"bid_l": float(bid["l"]),
|
||||
"bid_c": float(bid["c"]),
|
||||
"ask_o": float(ask["o"]),
|
||||
"ask_h": float(ask["h"]),
|
||||
"ask_l": float(ask["l"]),
|
||||
"ask_c": float(ask["c"]),
|
||||
})
|
||||
|
||||
print(f" request {n_requests}: from={params['from']} "
|
||||
f"got {len(candles)} candles (complete kept so far: "
|
||||
f"{len(all_rows)})", file=sys.stderr)
|
||||
|
||||
if not candles:
|
||||
break
|
||||
|
||||
last_time = pd.Timestamp(candles[-1]["time"])
|
||||
if last_time.tzinfo is None:
|
||||
last_time = last_time.tz_localize("UTC")
|
||||
else:
|
||||
last_time = last_time.tz_convert("UTC")
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
# Oanda's "count" includes the "from" boundary bar itself, so with
|
||||
# includeFirst=false every full page comes back as count-1, not
|
||||
# count (verified empirically: count=N, includeFirst=false always
|
||||
# yields N-1 candles when a full page of data exists ahead of the
|
||||
# cursor). Compare against the count actually expected for this
|
||||
# request's includeFirst setting, not the raw requested count.
|
||||
expected_full_page = COUNT if include_first else COUNT - 1
|
||||
got_fewer_than_requested = len(candles) < expected_full_page
|
||||
near_now = (now - last_time.to_pydatetime()) < FRESHNESS_WINDOW
|
||||
|
||||
if got_fewer_than_requested or near_now:
|
||||
break
|
||||
|
||||
cursor = last_time.to_pydatetime()
|
||||
include_first = False
|
||||
time.sleep(REQUEST_SLEEP_S)
|
||||
|
||||
return all_rows, n_requests
|
||||
|
||||
|
||||
def main():
|
||||
print(f"Fetching {INSTRUMENT} {GRANULARITY} candles from "
|
||||
f"{_rfc3339(START)} through now...", file=sys.stderr)
|
||||
|
||||
rows, n_requests = fetch_all_candles()
|
||||
|
||||
if not rows:
|
||||
raise RuntimeError("No complete candles were fetched.")
|
||||
|
||||
df = pd.DataFrame(rows)
|
||||
df = df.set_index("Datetime")
|
||||
df = df.sort_index()
|
||||
df = df[~df.index.duplicated(keep="last")]
|
||||
|
||||
df.to_parquet(OUT_PATH)
|
||||
|
||||
print(f"\nSaved {len(df)} rows to {OUT_PATH}", file=sys.stderr)
|
||||
print(f"Requests made: {n_requests}", file=sys.stderr)
|
||||
print(f"First timestamp: {df.index[0]}", file=sys.stderr)
|
||||
print(f"Last timestamp: {df.index[-1]}", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,40 +0,0 @@
|
||||
"""Per-hour P&L breakdown of a backtest trade book.
|
||||
|
||||
Usage:
|
||||
python backtest/research/hour_analysis.py BOOK.csv BAR_MINUTES
|
||||
|
||||
Reports, per UTC hour (both entry hour and DECISION hour = entry_time
|
||||
minus one bar, the clock blocked_hours_utc evaluates): trade count,
|
||||
total R, mean R, win rate, profit factor. Also a per-(hour, direction)
|
||||
table. Analysis tooling only — never imported by the live code.
|
||||
"""
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def hour_tables(book_csv: str, bar_minutes: int):
|
||||
df = pd.read_csv(book_csv)
|
||||
et = pd.to_datetime(df["entry_time"], utc=True, format="mixed")
|
||||
df["entry_hour"] = et.dt.hour
|
||||
df["decision_hour"] = (et - pd.Timedelta(minutes=bar_minutes)).dt.hour
|
||||
|
||||
def table(group_cols):
|
||||
g = df.groupby(group_cols)["r_multiple"]
|
||||
out = g.agg(count="count", total_r="sum", mean_r="mean",
|
||||
wr=lambda r: (r > 0).mean() * 100)
|
||||
out["pf"] = g.apply(
|
||||
lambda r: r[r > 0].sum() / max(-r[r <= 0].sum(), 1e-9))
|
||||
return out.round(2)
|
||||
|
||||
return (table("decision_hour"), table(["decision_hour", "direction"]),
|
||||
table("entry_hour"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
book, mins = sys.argv[1], int(sys.argv[2])
|
||||
by_dec, by_dec_dir, by_ent = hour_tables(book, mins)
|
||||
print(f"\n── {book} · by DECISION hour (bar-close clock) ──")
|
||||
print(by_dec.to_string())
|
||||
print(f"\n── by (decision hour, direction) ──")
|
||||
print(by_dec_dir.to_string())
|
||||
@@ -1,199 +0,0 @@
|
||||
"""
|
||||
Oanda vs Twelve Data bar-divergence study — XAU/USD M15, 12 months.
|
||||
====================================================================
|
||||
Compares the mid-price OHLC bars the signal engine would see from each
|
||||
provider over the same window. Purpose: quantify whether migrating the
|
||||
signal feed from Twelve Data to Oanda is a formality (bars ~identical)
|
||||
or a recalibration (bars differ at a scale that moves indicators).
|
||||
|
||||
Inputs:
|
||||
--td Twelve Data parquet (raw bars from backtest/cache/)
|
||||
--oanda Oanda parquet (from fetch_oanda_m5.py / fetch_oanda_m15.py)
|
||||
--start / --end analysis window (UTC dates)
|
||||
|
||||
Outputs:
|
||||
printed report
|
||||
backtest/research/oanda_td_divergence_by_hour.csv
|
||||
backtest/research/oanda_td_bar_diffs.parquet (per-bar diffs, common index)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
RESEARCH_DIR = Path(__file__).parent
|
||||
PIP = 0.10 # XAU pip in $
|
||||
|
||||
|
||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
hl = df["High"] - df["Low"]
|
||||
hc = (df["High"] - df["Close"].shift(1)).abs()
|
||||
lc = (df["Low"] - df["Close"].shift(1)).abs()
|
||||
tr = pd.concat([hl, hc, lc], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1.0 / period, adjust=False).mean()
|
||||
|
||||
|
||||
def pct_stats(s: pd.Series) -> dict:
|
||||
a = s.abs()
|
||||
return {
|
||||
"mean": a.mean(), "median": a.median(),
|
||||
"p90": a.quantile(0.90), "p95": a.quantile(0.95),
|
||||
"p99": a.quantile(0.99), "max": a.max(),
|
||||
"bias(mean signed)": s.mean(),
|
||||
}
|
||||
|
||||
|
||||
def fmt_stats(name: str, s: pd.Series) -> str:
|
||||
st = pct_stats(s)
|
||||
return (f" {name:<6} mean ${st['mean']:.3f} med ${st['median']:.3f} "
|
||||
f"p95 ${st['p95']:.3f} p99 ${st['p99']:.3f} max ${st['max']:.2f} "
|
||||
f"bias {st['bias(mean signed)']:+.4f}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--td", default=str(RESEARCH_DIR.parent / "cache" /
|
||||
"XAU_USD_15min_20240522_20260723.parquet"))
|
||||
ap.add_argument("--oanda", default=str(RESEARCH_DIR /
|
||||
"oanda_xau_m15_20250701_now.parquet"))
|
||||
ap.add_argument("--start", default="2025-07-24")
|
||||
ap.add_argument("--end", default="2026-07-23")
|
||||
args = ap.parse_args()
|
||||
|
||||
td = pd.read_parquet(args.td)
|
||||
oa = pd.read_parquet(args.oanda)
|
||||
|
||||
start = pd.Timestamp(args.start, tz="UTC")
|
||||
end = pd.Timestamp(args.end, tz="UTC") + pd.Timedelta(days=1)
|
||||
start = max(start, td.index[0], oa.index[0])
|
||||
end = min(end, td.index[-1], oa.index[-1])
|
||||
td = td.loc[(td.index >= start) & (td.index <= end)]
|
||||
oa = oa.loc[(oa.index >= start) & (oa.index <= end)]
|
||||
|
||||
print("=" * 78)
|
||||
print("OANDA vs TWELVE DATA — XAU/USD M15 bar divergence")
|
||||
print(f"Window: {start} -> {end} ({(end - start).days} days)")
|
||||
print("=" * 78)
|
||||
|
||||
# ── Coverage ─────────────────────────────────────────────────────
|
||||
common = td.index.intersection(oa.index)
|
||||
td_only = td.index.difference(oa.index)
|
||||
oa_only = oa.index.difference(td.index)
|
||||
print(f"\nCOVERAGE")
|
||||
print(f" Twelve Data bars : {len(td):>7,}")
|
||||
print(f" Oanda bars : {len(oa):>7,}")
|
||||
print(f" common : {len(common):>7,}")
|
||||
print(f" TD-only : {len(td_only):>7,} "
|
||||
f"({100 * len(td_only) / max(len(td), 1):.2f}% of TD)")
|
||||
print(f" Oanda-only : {len(oa_only):>7,} "
|
||||
f"({100 * len(oa_only) / max(len(oa), 1):.2f}% of Oanda)")
|
||||
for label, idx in (("TD-only", td_only), ("Oanda-only", oa_only)):
|
||||
if len(idx):
|
||||
by_date = pd.Series(1, index=idx).groupby(idx.date).sum()
|
||||
top = by_date.sort_values(ascending=False).head(8)
|
||||
print(f" {label} bars cluster on: "
|
||||
+ ", ".join(f"{d} ({n})" for d, n in top.items()))
|
||||
by_hour = pd.Series(1, index=idx).groupby(idx.hour).sum()
|
||||
top_h = by_hour.sort_values(ascending=False).head(6)
|
||||
print(f" {label} by UTC hour: "
|
||||
+ ", ".join(f"{h:02d}h ({n})" for h, n in top_h.items()))
|
||||
|
||||
t = td.loc[common]
|
||||
o = oa.loc[common]
|
||||
|
||||
# ── Scale references ─────────────────────────────────────────────
|
||||
atr = wilder_atr(t).dropna()
|
||||
mean_price = t["Close"].mean()
|
||||
print(f"\nSCALE REFERENCES (Twelve Data feed)")
|
||||
print(f" mean close : ${mean_price:,.2f}")
|
||||
print(f" mean bar range : ${(t['High'] - t['Low']).mean():.3f}")
|
||||
print(f" mean ATR14 : ${atr.mean():.3f} median ${atr.median():.3f}")
|
||||
|
||||
# ── Per-field diffs (TD − Oanda mid) ─────────────────────────────
|
||||
print(f"\nPER-BAR DIFFS (TD − Oanda, $; pip = $0.10)")
|
||||
diffs = {}
|
||||
for col in ("Open", "High", "Low", "Close"):
|
||||
d = t[col] - o[col]
|
||||
diffs[col] = d
|
||||
print(fmt_stats(col, d))
|
||||
|
||||
dclose = diffs["Close"].abs()
|
||||
print(f"\n |ΔClose| as % of mean ATR14 : "
|
||||
f"mean {100 * dclose.mean() / atr.mean():.1f}% "
|
||||
f"p95 {100 * dclose.quantile(0.95) / atr.mean():.1f}%")
|
||||
|
||||
print(f"\nTHRESHOLD EXCEEDANCE (|ΔClose|)")
|
||||
for thr in (0.10, 0.30, 0.65, 1.00, 2.00, 5.00):
|
||||
pct = 100 * (dclose > thr).mean()
|
||||
print(f" > ${thr:<5.2f}: {pct:6.2f}% of bars ({(dclose > thr).sum():,})")
|
||||
|
||||
# ── Direction / return agreement ─────────────────────────────────
|
||||
t_ret = t["Close"].pct_change()
|
||||
o_ret = o["Close"].pct_change()
|
||||
# only consecutive common bars (gap = one M15 step)
|
||||
step_ok = pd.Series(common, index=common).diff().eq(pd.Timedelta(minutes=15))
|
||||
tr, orr = t_ret[step_ok], o_ret[step_ok]
|
||||
corr = tr.corr(orr)
|
||||
t_dir = np.sign(t["Close"] - t["Open"])
|
||||
o_dir = np.sign(o["Close"] - o["Open"])
|
||||
both_nonflat = (t_dir != 0) & (o_dir != 0)
|
||||
dir_agree = 100 * (t_dir[both_nonflat] == o_dir[both_nonflat]).mean()
|
||||
print(f"\nAGREEMENT")
|
||||
print(f" close-return correlation (consecutive bars): {corr:.5f} "
|
||||
f"(n={step_ok.sum():,})")
|
||||
print(f" bar direction (sign of close−open) agree : {dir_agree:.2f}%")
|
||||
|
||||
# ── By-hour breakdown ────────────────────────────────────────────
|
||||
hours = common.hour
|
||||
rows = []
|
||||
for h in range(24):
|
||||
m = hours == h
|
||||
if not m.any():
|
||||
continue
|
||||
row = {"hour_utc": h, "bars": int(m.sum()),
|
||||
"mean_abs_dclose": dclose[m].mean(),
|
||||
"p95_abs_dclose": dclose[m].quantile(0.95),
|
||||
"mean_abs_dhigh": diffs["High"][m].abs().mean(),
|
||||
"mean_abs_dlow": diffs["Low"][m].abs().mean()}
|
||||
if "bid_c" in o.columns:
|
||||
row["oanda_median_spread"] = (
|
||||
(o["ask_c"] - o["bid_c"])[m].median())
|
||||
rows.append(row)
|
||||
by_hour = pd.DataFrame(rows)
|
||||
out_hour = RESEARCH_DIR / "oanda_td_divergence_by_hour.csv"
|
||||
by_hour.to_csv(out_hour, index=False, float_format="%.4f")
|
||||
print(f"\nBY UTC HOUR (written to {out_hour.name})")
|
||||
print(by_hour.to_string(index=False,
|
||||
float_format=lambda x: f"{x:.3f}"))
|
||||
|
||||
# ── By-month drift ───────────────────────────────────────────────
|
||||
print(f"\nBY MONTH (mean |ΔClose|, $)")
|
||||
by_month = dclose.groupby(common.to_period("M")).mean()
|
||||
for p, v in by_month.items():
|
||||
print(f" {p}: {v:.3f}")
|
||||
|
||||
# ── Oanda real spread (bonus, ties to Dukascopy study) ───────────
|
||||
if "bid_c" in o.columns:
|
||||
spread = o["ask_c"] - o["bid_c"]
|
||||
liquid = ~o.index.hour.isin([21, 22, 23, 0])
|
||||
print(f"\nOANDA REAL SPREAD (close bid/ask, $)")
|
||||
print(f" all bars : median {spread.median():.3f} "
|
||||
f"p25 {spread.quantile(0.25):.3f} p75 {spread.quantile(0.75):.3f}")
|
||||
print(f" liquid hours : median {spread[liquid].median():.3f}")
|
||||
print(f" 21-00 UTC : median {spread[~liquid].median():.3f}")
|
||||
|
||||
# ── Persist per-bar diffs ────────────────────────────────────────
|
||||
out = pd.DataFrame({
|
||||
"td_close": t["Close"], "oa_close": o["Close"],
|
||||
"d_open": diffs["Open"], "d_high": diffs["High"],
|
||||
"d_low": diffs["Low"], "d_close": diffs["Close"],
|
||||
})
|
||||
out_path = RESEARCH_DIR / "oanda_td_bar_diffs.parquet"
|
||||
out.to_parquet(out_path)
|
||||
print(f"\nPer-bar diffs written to {out_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,25 +0,0 @@
|
||||
hour_utc,bars,mean_abs_dclose,p95_abs_dclose,mean_abs_dhigh,mean_abs_dlow,oanda_median_spread
|
||||
0,1029,3.3483,13.9460,3.2906,3.4310,0.7900
|
||||
1,1028,3.3483,12.8292,3.3477,3.6398,0.7700
|
||||
2,1028,3.0288,10.8960,2.9469,3.1317,0.7700
|
||||
3,1028,2.8783,11.1300,2.8419,2.9992,0.7600
|
||||
4,1028,2.8232,10.4710,2.8165,2.8455,0.7400
|
||||
5,1028,2.9268,10.6027,2.9234,3.0038,0.7400
|
||||
6,1028,3.0466,10.9265,2.9487,3.0796,0.7200
|
||||
7,1028,3.0286,11.3512,3.0301,3.0797,0.6700
|
||||
8,1028,3.0223,11.1483,2.8989,3.0801,0.6600
|
||||
9,1028,3.0460,11.6345,2.9307,3.0609,0.6600
|
||||
10,1028,2.9280,10.9015,2.8647,3.0029,0.6600
|
||||
11,1028,3.0340,11.4712,2.9591,2.9472,0.6600
|
||||
12,1028,3.0986,11.3282,2.9412,3.1407,0.7000
|
||||
13,1028,3.4014,12.3343,3.1077,3.4228,0.7300
|
||||
14,1028,3.5074,11.3628,3.3960,3.6569,0.6900
|
||||
15,1028,3.4514,11.8982,3.4124,3.6733,0.6700
|
||||
16,1028,3.1275,12.5155,3.2038,3.3061,0.6700
|
||||
17,1020,2.9826,12.6435,3.0064,3.1008,0.6900
|
||||
18,1015,2.8721,12.5480,2.9101,2.9996,0.6900
|
||||
19,1004,2.8637,12.1335,2.8824,2.9454,0.6900
|
||||
20,996,2.8006,11.9325,2.8486,2.8577,0.7600
|
||||
21,335,1.1085,4.8600,1.1212,1.2034,0.9400
|
||||
22,676,4.0827,13.6787,4.8422,4.6761,0.8300
|
||||
23,1027,3.2722,12.5725,3.5617,3.7747,0.8700
|
||||
|
@@ -1,271 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
panel_payload_drytest_2026-08-15.py — live DeepSeek validation of the
|
||||
enriched agent-panel payloads + hardened prompts (2026-08-15).
|
||||
|
||||
Spends ~6 REAL DeepSeek requests (against the shared 500/day cap) with
|
||||
FABRICATED payloads — made-up prices/indicators, no real market data —
|
||||
to verify the new technical-persona contract end to end:
|
||||
|
||||
P1 coherent bullish evidence -> technical persona SUPPORTS
|
||||
P2 bearish evidence vs BUY brief -> technical persona OPPOSES
|
||||
P3 conflicting evidence -> NEUTRAL or low confidence
|
||||
P4 empty evidence -> NEUTRAL with confidence < 30
|
||||
(the anti-hallucination contract)
|
||||
P5 hallucination trap -> note invents no price levels
|
||||
P6 output discipline -> <=20-word notes, exact JSON keys
|
||||
P7 synthesizer over 4 supports -> AGREE verdict
|
||||
|
||||
Usage: venv/bin/python backtest/research/panel_payload_drytest_2026-08-15.py
|
||||
Exit code 0 = all probes passed. Raw replies land in
|
||||
backtest/research/panel_drytest_2026-08-15.jsonl
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
import agent_panel as ap # noqa: E402
|
||||
from app_config import deepseek_api_key, get_app_config # noqa: E402
|
||||
from llm_client import get_llm_client # noqa: E402
|
||||
|
||||
ARTIFACT = Path(__file__).with_suffix(".jsonl")
|
||||
|
||||
|
||||
# ── Fabrication helpers (all numbers are invented) ───────────────────
|
||||
|
||||
def _staircase(start: float, step: float, n: int) -> pd.DataFrame:
|
||||
"""n synthetic bars drifting `step` per bar; last close = start+n*step."""
|
||||
rows, price = [], start
|
||||
for _ in range(n):
|
||||
o, c = price, price + step
|
||||
rows.append({"Open": o, "Close": c,
|
||||
"High": max(o, c) + abs(step) * 0.4,
|
||||
"Low": min(o, c) - abs(step) * 0.4})
|
||||
price = c
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
def _choppy(center: float, n: int) -> pd.DataFrame:
|
||||
rows, price, sign = [], center, 1
|
||||
for _ in range(n):
|
||||
step = sign * 1.5
|
||||
o, c = price, price + step
|
||||
rows.append({"Open": o, "Close": c,
|
||||
"High": max(o, c) + 0.6, "Low": min(o, c) - 0.6})
|
||||
price, sign = c, -sign
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
_FAKE_PRICE = 2000.00 # deliberately made-up
|
||||
_FAKE_ATR = 5.0
|
||||
|
||||
_BRIEF_BUY = {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY",
|
||||
"quality": "A", "score": 38.5, "confidence": 62,
|
||||
"price": _FAKE_PRICE, "market_regime": "trending_up",
|
||||
"volatility_regime": "normal", "mtf_agreement": True}
|
||||
|
||||
_BULL_INDICATORS = {
|
||||
"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0},
|
||||
"rsi": {"signal": "BUY", "confidence": 65, "value": 58.0},
|
||||
"macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25},
|
||||
"bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7},
|
||||
"adx": {"signal": "BUY", "confidence": 70, "value": 32.0},
|
||||
"roc": {"signal": "BUY", "confidence": 55, "value": 0.35},
|
||||
"keltner": {"signal": "BUY", "confidence": 65, "value": 3.1},
|
||||
"candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0},
|
||||
}
|
||||
|
||||
_BEAR_INDICATORS = {
|
||||
"ema_trend": {"signal": "STRONG_SELL", "confidence": 85, "value": 0.1},
|
||||
"rsi": {"signal": "SELL", "confidence": 70, "value": 74.0},
|
||||
"macd": {"signal": "STRONG_SELL", "confidence": 80, "value": -1.4},
|
||||
"bollinger": {"signal": "SELL", "confidence": 60, "value": 0.95},
|
||||
"adx": {"signal": "SELL", "confidence": 70, "value": 35.0},
|
||||
"roc": {"signal": "SELL", "confidence": 60, "value": -0.5},
|
||||
"keltner": {"signal": "SELL", "confidence": 65, "value": -2.8},
|
||||
"candlestick": {"signal": "SELL", "confidence": 55, "value": -2.0},
|
||||
}
|
||||
|
||||
_MIXED_INDICATORS = {
|
||||
"ema_trend": {"signal": "BUY", "confidence": 55, "value": 0.6},
|
||||
"rsi": {"signal": "NEUTRAL", "confidence": 40, "value": 51.0},
|
||||
"macd": {"signal": "SELL", "confidence": 55, "value": -0.3},
|
||||
"bollinger": {"signal": "NEUTRAL", "confidence": 35, "value": 0.5},
|
||||
"adx": {"signal": "NEUTRAL", "confidence": 40, "value": 18.0},
|
||||
"roc": {"signal": "SELL", "confidence": 50, "value": -0.1},
|
||||
"keltner": {"signal": "BUY", "confidence": 50, "value": 0.8},
|
||||
"candlestick": {"signal": "NEUTRAL", "confidence": 30, "value": 0.0},
|
||||
}
|
||||
|
||||
|
||||
def _tech_payload(indicators, mtf, candles_df=None, key_levels=None,
|
||||
brief=None, exhaustion=0):
|
||||
payload = {"brief": brief or dict(_BRIEF_BUY),
|
||||
"indicators": indicators, "mtf": mtf,
|
||||
"key_levels": key_levels or {}, "exhaustion": exhaustion}
|
||||
if candles_df is not None:
|
||||
# the REAL production normalizer, not a hand-rolled copy
|
||||
payload["candles"] = {
|
||||
"15min": ap._normalize_candles(candles_df, _FAKE_ATR,
|
||||
len(candles_df))}
|
||||
return payload
|
||||
|
||||
|
||||
# ── Probe runner ─────────────────────────────────────────────────────
|
||||
|
||||
_RESULTS = []
|
||||
|
||||
|
||||
def _record(probe, payload, reply, take, passed, detail):
|
||||
_RESULTS.append({"probe": probe, "pass": bool(passed),
|
||||
"detail": detail,
|
||||
"stance": getattr(take, "stance", None),
|
||||
"confidence": getattr(take, "confidence", None),
|
||||
"note": getattr(take, "note", None)})
|
||||
with ARTIFACT.open("a") as fh:
|
||||
fh.write(json.dumps({"probe": probe, "payload": payload,
|
||||
"raw_reply": reply}, default=str) + "\n")
|
||||
mark = "PASS" if passed else "FAIL"
|
||||
print(f"[{mark}] {probe}: {detail}")
|
||||
if take is not None:
|
||||
print(f" -> {take.stance} @ {take.confidence:.0f} "
|
||||
f"| {take.note}")
|
||||
|
||||
|
||||
def _ask(payload):
|
||||
"""Real API call through the production client + production
|
||||
validation (the same path run_panel uses)."""
|
||||
reply = get_llm_client().chat_json(ap._TECH_SYSTEM, payload)
|
||||
return reply, ap._validate_take("technical", reply)
|
||||
|
||||
|
||||
def _want(probe: str) -> bool:
|
||||
"""--only P3 runs just that probe (saves requests on re-runs)."""
|
||||
if "--only" not in sys.argv:
|
||||
return True
|
||||
return sys.argv[sys.argv.index("--only") + 1] == probe
|
||||
|
||||
|
||||
def main() -> int:
|
||||
app = get_app_config()
|
||||
if not deepseek_api_key(app):
|
||||
print("ABORT: no DeepSeek API key (env DEEPSEEK_API_KEY / "
|
||||
"config.json deepseek.api_key).")
|
||||
return 2
|
||||
if app.deepseek_settings.get("enabled", True) is False:
|
||||
print("ABORT: deepseek.enabled is false in config.json.")
|
||||
return 2
|
||||
if ARTIFACT.exists() and "--only" not in sys.argv:
|
||||
ARTIFACT.unlink() # full run = fresh wire log; --only appends
|
||||
|
||||
mtf_agree = {"5min": {"signal": "BUY", "score": 30.0},
|
||||
"1h": {"signal": "STRONG_BUY", "score": 45.0}}
|
||||
levels = {"resistance": 2012.0, "support": 1985.0}
|
||||
|
||||
# P1 — coherent bullish, with rising ATR staircase (last close 2000)
|
||||
if _want("P1"):
|
||||
p1 = _tech_payload(_BULL_INDICATORS, mtf_agree,
|
||||
candles_df=_staircase(_FAKE_PRICE - 20 * 0.6,
|
||||
0.6, 20),
|
||||
key_levels=levels)
|
||||
reply, take = _ask(p1)
|
||||
_record("P1 coherent-bullish", p1, reply, take,
|
||||
take is not None and take.stance == "support"
|
||||
and take.confidence >= 50,
|
||||
"expect support @ conf>=50")
|
||||
|
||||
# P2 — bearish everything against a BUY brief, falling staircase
|
||||
if _want("P2"):
|
||||
brief2 = dict(_BRIEF_BUY, market_regime="trending_down",
|
||||
mtf_agreement=False)
|
||||
mtf_sell = {"5min": {"signal": "SELL", "score": -28.0},
|
||||
"1h": {"signal": "STRONG_SELL", "score": -41.0}}
|
||||
p2 = _tech_payload(_BEAR_INDICATORS, mtf_sell, brief=brief2,
|
||||
candles_df=_staircase(_FAKE_PRICE + 20 * 0.6,
|
||||
-0.6, 20),
|
||||
key_levels=levels, exhaustion=3)
|
||||
reply, take = _ask(p2)
|
||||
_record("P2 bearish-vs-BUY-brief", p2, reply, take,
|
||||
take is not None and take.stance == "oppose",
|
||||
"expect oppose (BUY brief fights all evidence)")
|
||||
|
||||
# P3 — genuinely mixed evidence: MTF neutral both ways, ranging
|
||||
# regime, choppy candles. (An H1 SELL against a BUY is a *listed
|
||||
# conflict* in the prompt — oppose is correct there, not mixed.)
|
||||
if _want("P3"):
|
||||
brief3 = dict(_BRIEF_BUY, market_regime="ranging", score=12.0,
|
||||
confidence=35, mtf_agreement=False)
|
||||
mtf_mixed = {"5min": {"signal": "NEUTRAL", "score": 4.0},
|
||||
"1h": {"signal": "NEUTRAL", "score": -3.0}}
|
||||
p3 = _tech_payload(_MIXED_INDICATORS, mtf_mixed, brief=brief3,
|
||||
candles_df=_choppy(_FAKE_PRICE, 20))
|
||||
reply, take = _ask(p3)
|
||||
_record("P3 conflicting", p3, reply, take,
|
||||
take is not None and (take.stance == "neutral"
|
||||
or take.confidence < 60),
|
||||
"expect neutral or conf<60 on genuinely mixed evidence")
|
||||
|
||||
# P4 — empty evidence: contract demands neutral @ conf<30
|
||||
if _want("P4"):
|
||||
p4 = _tech_payload({}, {}, key_levels=None)
|
||||
reply, take = _ask(p4)
|
||||
_record("P4 empty-evidence", p4, reply, take,
|
||||
take is not None and take.stance == "neutral"
|
||||
and take.confidence < 30,
|
||||
"contract: thin/missing evidence -> neutral, conf<30")
|
||||
|
||||
# P5 — hallucination trap: no key_levels, no candles; the note must
|
||||
# not cite any 3+-digit number absent from the payload
|
||||
if _want("P5"):
|
||||
p5 = _tech_payload(_BULL_INDICATORS, mtf_agree, key_levels=None)
|
||||
reply, take = _ask(p5)
|
||||
invented = []
|
||||
if take is not None:
|
||||
payload_blob = json.dumps(p5)
|
||||
for num in re.findall(r"\d{3,}", take.note):
|
||||
if num not in payload_blob:
|
||||
invented.append(num)
|
||||
_record("P5 hallucination-trap", p5, reply, take,
|
||||
take is not None and not invented,
|
||||
f"invented numbers in note: {invented or 'none'}")
|
||||
|
||||
# P6 — output discipline across the persona replies gathered above
|
||||
notes = [r for r in _RESULTS if r["note"]]
|
||||
wordy = [r["probe"] for r in notes if len(r["note"].split()) > 20]
|
||||
_record("P6 output-discipline", {}, None, None,
|
||||
notes and not wordy,
|
||||
f"notes >20 words: {wordy or 'none'} "
|
||||
f"(cap is 15; 20 = fail threshold)")
|
||||
|
||||
# P7 — synthesizer over four fabricated SUPPORT takes
|
||||
if _want("P7"):
|
||||
takes = [{"persona": p, "stance": "support", "confidence": c,
|
||||
"note": "evidence aligned"}
|
||||
for p, c in (("macro", 70), ("technical", 80),
|
||||
("risk", 65), ("contrarian", 60))]
|
||||
synth_payload = {"direction": "BUY", "takes": takes}
|
||||
reply = get_llm_client().chat_json(ap._SYNTH_SYSTEM, synth_payload)
|
||||
verdict = reply.get("verdict") if isinstance(reply, dict) else None
|
||||
rationale = (reply or {}).get("rationale", "")
|
||||
_record("P7 synthesizer", synth_payload, reply, None,
|
||||
verdict == "AGREE" and len(str(rationale).split()) <= 40,
|
||||
f"expect AGREE + <=40-word rationale; got {verdict!r} "
|
||||
f"({len(str(rationale).split())} words)")
|
||||
|
||||
n_pass = sum(1 for r in _RESULTS if r["pass"])
|
||||
n_api = sum(1 for r in _RESULTS if r["probe"] != "P6 output-discipline")
|
||||
print(f"\n{n_pass}/{len(_RESULTS)} probes passed "
|
||||
f"({n_api} API requests spent; "
|
||||
f"raw wire log: {ARTIFACT.name})")
|
||||
return 0 if n_pass == len(_RESULTS) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,40 @@
|
||||
# Backtest Re-pin on Oanda bars — verification (Phase 3)
|
||||
|
||||
**Date:** 2026-08-14 (verification; original books were generated 2026-08-04, current-HEAD M5 canonical book exported 2026-08-14)
|
||||
**Scope:** confirm the pinned Oanda books exist, satisfy rule 1, and quantify the Oanda↔TD delta that motivated them.
|
||||
|
||||
## Re-pin state
|
||||
|
||||
The re-pin is **already done** — full Oanda-bar books exist in `backtest/oanda_*.csv` for M5/M15/H1/H4, at both cost assumptions, with in-sample (12mo), out-of-sample (2024-06 → 2025-03), and 5-year variants. The shipped configs are `m5` (pullback OFF — `_nopb`) and `m15`.
|
||||
|
||||
### M5 canonical-reference reconciliation (2026-08-14)
|
||||
|
||||
The dated current-HEAD export `backtest/oanda_m5_12mo_065_nopb_20260814.csv` is the canonical M5 12-month reference going forward: **577 trades, +106.82R, PF 1.51**, with both 12-month halves positive (+50.80R / +56.02R; PF 1.46 / 1.55). It covers 2025-08-14 → 2026-08-14, runs the shipped M5 factory (XAU_PB off), charges $0.65 round trip, and uses next-open/raw-path/close-only-trail realism unchanged.
|
||||
|
||||
Earlier M5 snapshots (`oanda_m5_12mo_065_nopb.csv`: 586 / +98.11R / 1.45; `m5_12mo_shipped.csv`: 588 / +96.60R / 1.44; and the 555 / +98.64R ledger entry) remain historical evidence from different HEADs around the weekend-calendar and related QA work. They must not be cited as the current-HEAD reference.
|
||||
|
||||
### Rule-1 summary at realistic cost ($0.65 round trip)
|
||||
|
||||
| TF | 12mo in-sample | 12mo H1 / H2 | OOS (2024-06→2025-03) |
|
||||
|---|---|---|---|
|
||||
| M15 | +73.93R PF 1.53 | +66.71 / +7.23 (PF 1.10) | +38.77R PF 1.36 |
|
||||
| M5 (shipped, nopb) | +106.82R PF 1.51 (current HEAD) | +50.80 / +56.02 (PF 1.46 / 1.55) | +9.52R PF 1.34 |
|
||||
| H1 | +23.97R PF 1.55 | — | +5.15R PF 1.13 |
|
||||
| H4 | +16.14R PF 2.28 | — | +1.31R PF 1.14 |
|
||||
|
||||
- **M5 (shipped)** passes rule 1 cleanly: both current-HEAD 12mo halves positive with PF ≥ 1.3, and OOS positive with PF 1.34.
|
||||
- **M15** passes in-sample and OOS (PF 1.53 / 1.36) but its second half is soft (PF 1.10) — positive R, below the 1.3 bar. Pre-existing condition, not a re-pin artifact.
|
||||
- **H1/H4** pass in-sample but OOS is marginal (PF 1.13–1.14) — consistent with the standing "H4 unproven for auto-execution" note.
|
||||
|
||||
## Oanda ↔ Twelve Data trade-level delta
|
||||
|
||||
Not worth a trade-by-trade book diff: Phase 1 already showed the two feeds disagree on **bar direction** 19% of the time on M5 (11% on M15), with a ~$3 mean / ~$1 systematic close gap. A TD-pinned book is a different instrument's book — this is the documented re-pin motivator, and the Oanda books above are the replacement baseline. A formal trade-level reconciliation would show near-total mismatch and add no decision value.
|
||||
|
||||
## Deliverable status
|
||||
|
||||
- [x] Re-pinned books exist on Oanda bars (12mo + OOS + 5yr, $0.30 + $0.65).
|
||||
- [x] Rule 1 verified for the shipped TFs (M5 clean; M15 in-sample+OOS pass, H2 soft; H1/H4 OOS marginal).
|
||||
- [x] Cost model: $0.65 matches measured Oanda spread (~$0.65–0.72); $0.30 retained for cross-book comparability.
|
||||
- [x] Oanda↔TD delta quantified (Phase 1) — no further book diff needed.
|
||||
|
||||
Remaining Phase-3 follow-up: none. Owner decision 2026-08-14 retained dual reporting: $0.65 is the realistic headline while $0.30 remains for TD-era cross-book comparability and the live TP spread gate.
|
||||
@@ -44,7 +44,9 @@ book directly. Windows (cache start = start − 62 days):
|
||||
Acceptance: positive total R in both 12mo halves, PF ≥ 1.3, OOS not worse
|
||||
than control, and a candidate that removes a net-positive bucket for a PF-only
|
||||
gain is rejected (big-candle precedent). Runner:
|
||||
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py`.
|
||||
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py` (scratchpad
|
||||
tooling, not retained — both shipped gates are reproduced by a plain pair-factory
|
||||
run at HEAD; see the parity recipe in `sizing_campaign_2026-09.md`).
|
||||
|
||||
Candidates (all `--set`, no code):
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# Signal-Level Replay Parity — Oanda vs Tiingo (Phase 2)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Tool:** `scripts/signal_parity_study.py` (replays real `analyze_from_dataframes`, news off)
|
||||
**Window:** 2025-08-14 → 2026-08-14 (12 months); M15 stride 8, M5 stride 24 → ~2,900 sampled decisions each
|
||||
**Preceding:** Phase 1 price-diff study (`feed_divergence_2026-08-14.md`)
|
||||
|
||||
## Headline
|
||||
|
||||
On the ~94–96% of bars the two feeds share, **Oanda and Tiingo produce near-identical signals**:
|
||||
|
||||
| Metric | M15 | M5 |
|
||||
|---|---|---|
|
||||
| same direction | 98.32% | 98.16% |
|
||||
| same grade | 94.13% | 95.23% |
|
||||
| same setup presence | 98.83% | 99.49% |
|
||||
| same direction + grade + setup | 93.41% | 94.21% |
|
||||
| opposite-sign flips (BUY↔SELL) | **0** | **0** |
|
||||
| direction flips on actionable bars | 6.6% | 10.9% |
|
||||
| grade changed on actionable bars | 14.2% | 7.8% |
|
||||
| mean \|Δ final_score\| | 1.76 | 1.78 |
|
||||
| mean \|Δ primary_score\| | 1.73 | 1.77 |
|
||||
| same strategy tag | 100% | 100% |
|
||||
|
||||
Every direction flip is **actionable↔NEUTRAL**, never a sign reversal: the feeds never disagree on buy-vs-sell, only on *whether* a marginal bar crosses the action threshold. Flipping bars cluster at the Friday-close/Sunday-reopen boundary (UTC hours 22–23) and in the Asian hours (03–04).
|
||||
|
||||
## Setup levels diverge even when the signal agrees
|
||||
|
||||
On bars where **both** feeds emit a setup with the same tag:
|
||||
|
||||
| | M15 (n=119) | M5 (n=39) |
|
||||
|---|---|---|
|
||||
| entry mean \|Δ\| | $0.12 | $0.10 |
|
||||
| SL mean \|Δ\| | $1.83 | $2.79 |
|
||||
| TP1 mean \|Δ\| | $2.47 | $1.52 |
|
||||
| TP2 mean \|Δ\| | $3.74 | $2.46 |
|
||||
| TP1 p95 \|Δ\| | $17.4 | $2.3 |
|
||||
|
||||
Entry price is the current close (≈ identical across feeds), but **SL/TP are recomputed from ATR and recent swing extremes, which differ more** — mean SL drift ~$2–3 (p95 up to $10–26 on M15), i.e. 20–40% of ATR. So even an identical signal places stops/targets at meaningfully different prices on a different feed.
|
||||
|
||||
## Phase 2 pass/fail vs plan thresholds
|
||||
|
||||
| Criterion | Plan threshold | M15 / M5 | Verdict |
|
||||
|---|---|---|---|
|
||||
| identical direction+grade | ≥ 97% | 93.4% / 94.2% | **marginal** (below 97%, above 93%) |
|
||||
| opposite-sign flips | (none specified) | 0 / 0 | PASS |
|
||||
| flips clustered at marginal composites | no session/regime clustering | yes (boundary hours) | PASS |
|
||||
|
||||
The "identical direction+grade" metric lands slightly under the proposed 97% bar — but the shortfall is grade *letters* (A+ vs A vs B) changing on marginal composites, not direction. If the acceptance criterion is "no opposite-sign disagreement and ~98% direction identity", this passes comfortably.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Oanda ↔ Tiingo is **signal-equivalent for direction** (98% identical, zero sign flips) over 12 months. The residual deltas are:
|
||||
|
||||
1. **~6% of decisions differ in grade letter** on marginal composites (A+↔A↔B near thresholds).
|
||||
2. **SL/TP levels drift ~$2–4 mean** (up to ~$20 p95 on M15) even for identical signals — a real P&L difference at the trade level.
|
||||
3. **Tiingo's extra Sunday bars** (Phase 1) are gated by the aggregator's NY-anchored weekend filter, so they do not inject phantom signals, but they do slightly shift indicator history at Monday opens.
|
||||
|
||||
Net: switching Oanda→Tiingo changes a handful of marginal decisions and moves stop/target levels, but does not change *what the system is doing* (direction, strategy, setup presence). Oanda↔Twelve Data was not replayed — Phase 1 already showed 81% M5 bar-direction disagreement and a ~$3/$1 systematic gap, so a TD replay would trivially fail (that is the documented re-pin motivator).
|
||||
@@ -3,8 +3,9 @@
|
||||
XAU/USD static dashboard publisher.
|
||||
|
||||
Standalone daemon that scans XAU/USD at candle closes and atomically
|
||||
writes `snapshot.json` + `news.json` (plus a copy of
|
||||
`static/dashboard.html`) into an Apache-served directory. Apache does
|
||||
writes `snapshot.json` + `news.json` (plus copies of the static
|
||||
`dashboard.html` / `.css` / `.js` and `news_summary.php`) into an
|
||||
Apache-served directory. Apache does
|
||||
all web serving — this process never opens a port, so a failure here
|
||||
can only ever mean a stale page.
|
||||
|
||||
|
||||
@@ -24,8 +24,9 @@ No new Python dependencies are required.
|
||||
## 2. Make the web folder writable
|
||||
|
||||
The dashboard publishes straight into the portal docroot `/var/www/trade/`
|
||||
(owner decision 2026-08-03) — it adds exactly four files there:
|
||||
`dashboard.html`, `news_summary.php`, `snapshot.json`, `news.json` (plus the
|
||||
(owner decision 2026-08-03) — it adds exactly six files there:
|
||||
`dashboard.html`, `dashboard.css`, `dashboard.js`, `news_summary.php`,
|
||||
`snapshot.json`, `news.json` (plus the
|
||||
apache-owned `ai/` cache dir from §6). The daemon user must be able to write
|
||||
them:
|
||||
|
||||
@@ -51,7 +52,7 @@ sudo restorecon -R /var/www/trade
|
||||
|
||||
```bash
|
||||
venv/bin/python dashboard_publisher.py --once
|
||||
ls /var/www/trade # dashboard.html snapshot.json news.json added next to the portal
|
||||
ls /var/www/trade # dashboard.html/.css/.js news_summary.php snapshot.json news.json added next to the portal
|
||||
```
|
||||
|
||||
Then open `http://<server>/dashboard.html` (path relative to however the
|
||||
@@ -141,8 +142,8 @@ curl -s "http://<server>/news_summary.php?id=$ID" # instant, from ai/<id>.json
|
||||
## Operational notes
|
||||
|
||||
- **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
|
||||
re-copies `static/dashboard.html` and `static/news_summary.php` into the
|
||||
web folder at startup.
|
||||
re-copies `static/dashboard.html`, `dashboard.css`, `dashboard.js` and
|
||||
`news_summary.php` into the web folder at startup.
|
||||
- **If the daemon dies**, systemd restarts it in 30 s; worst case the page
|
||||
shows stale data with its own "updated N min ago" evidence. Apache and the
|
||||
telegram bot are unaffected.
|
||||
@@ -155,8 +156,5 @@ curl -s "http://<server>/news_summary.php?id=$ID" # instant, from ai/<id>.json
|
||||
DeepSeek-side spend alerting is the true backstop. Kept separate on
|
||||
purpose (independence rule: the PHP never imports/reads bot code or
|
||||
config.json).
|
||||
- **Restart window**: for the first minutes after a daemon restart the new
|
||||
page can meet a pre-upgrade news.json without headline ids — those rows
|
||||
simply render non-clickable until the first news publish.
|
||||
- **Credentials**: uses the same env/.env/config.json resolution as the rest
|
||||
of the platform (Oanda candles feed; news needs no key; DeepSeek optional).
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 603 KiB After Width: | Height: | Size: 578 KiB |
@@ -148,7 +148,7 @@ box(2.5, 3.2, 71.8, 8.0, "backtest/gpu_backtest.py — bar-by-bar replay through
|
||||
"CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
|
||||
BTS_FC, BTS_EC)
|
||||
box(77.5, 3.2, 20, 8.0, "test_all.py",
|
||||
["359 tests · 22 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
|
||||
["461 tests · 31 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
|
||||
|
||||
# ── arrows: short structural hops (solid dark) ───────────────────────
|
||||
arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2) # oanda → providers
|
||||
@@ -185,6 +185,7 @@ ax.text(81.8, 38.9, "replays the exact signal code", fontsize=6.4,
|
||||
color=MUT, ha="center", va="center", zorder=5,
|
||||
bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
|
||||
|
||||
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
|
||||
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
|
||||
print(f"saved {out}")
|
||||
if __name__ == "__main__": # importing the module must never rewrite the PNG
|
||||
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
|
||||
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
|
||||
print(f"saved {out}")
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
# Test Plan — Data Source Change Validation
|
||||
|
||||
**Purpose:** determine whether switching `data_providers.active` (e.g. Twelve Data → Oanda, or any future provider swap) changes signal calculations materially, and gate the switch on evidence.
|
||||
|
||||
**Status:** in progress, 2026-08-14
|
||||
|
||||
> **Progress (2026-08-14):** Phase 0 ✅, Phase 1 ✅ (`backtest/research/feed_divergence_2026-08-14.md`), Phase 2 ✅ (`backtest/research/signal_parity_2026-08-14.md`), Phase 3 ✅ (`backtest/research/repin_verification_2026-08-14.md` — re-pin already existed), Phase 4 🔨 tool ready (`scripts/shadow_scan.py`, needs a 2-week run — launch manually per owner decision), Phase 5 owner decisions recorded (dual cost reporting; Tiingo streaming-only; see bottom).
|
||||
**Context:** Oanda has been the live feed since 2026-07-28. The backtest books were **re-pinned on Oanda bars on 2026-08-04** (`backtest/oanda_*.csv` — 27 books, now the baseline; all TD-era books superseded). Cost reporting is dual: $0.30 retained for cross-book comparability, $0.65 is the realistic headline (measured Oanda round trip $0.65–0.72). Twelve Data was proven signal-incompatible in Phase 1 (~19% of M5 bars disagree on direction; systematic ~$1 bias) — it is NOT a valid candle fallback.
|
||||
|
||||
**Why signals can differ between feeds** (mechanisms to test):
|
||||
|
||||
- Feeds publish different mid prices → tiny OHLC diffs shift every indicator; near a threshold (RSI band, `FB2_EMA_LO` EMA200-proximity band, composite/grade cutoffs, M5 min-score gate) one bar's discrete score flips.
|
||||
- Gate edge cases turn a tiny price diff into "setup generated" vs "no signal".
|
||||
- ATR-derived SL/TP/trail levels differ even when the signal is identical.
|
||||
- The shared 10σ cleaning pass can drop different bars per feed.
|
||||
- Volume present (Oanda) vs absent (Tiingo FX); real spread (Oanda MBA) vs pair default (Tiingo).
|
||||
- Known reference point: H4 seeding divergence — same provider, different fetch depth flipped ~2% of indicator signals (composite Δ up to 12.3). Cross-feed diffs act the same way.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Recent-bar parity smoke test ✅ DONE
|
||||
|
||||
- **Tool:** `scripts/compare_oanda_tiingo.py` (removed 2026-09-06 once Phases 1-2 superseded it; the result is recorded here)
|
||||
- **Result 2026-08-14:** 10/10 timestamp alignment on M5/M15/H1, Close max |Δ| ≤ $0.175 (≤ 0.004% of price), no directional bias.
|
||||
- **Verdict:** recent-tail parity confirmed. Does NOT cover historical depth — proceed to Phase 1.
|
||||
|
||||
## Phase 1 — Historical candle divergence study
|
||||
|
||||
**Goal:** quantify raw-data divergence over the full window used for validation, not just the recent tail.
|
||||
|
||||
- [x] Extend the comparison script to use `fetch_range` (paginated, raw bars) for both providers over **≥ 12 months** for M5, M15, H1 (and H4 if in scope).
|
||||
- [x] Record per TF:
|
||||
- % of bar timestamps present in both feeds; list of missing/extra bars (gaps, holidays, weekend boundary handling).
|
||||
- OHLC |Δ| distribution: p50 / p95 / p99 / max, in $ and % of price.
|
||||
- Count of bars dropped by `validate_and_clean` on each feed (10σ outliers, malformed).
|
||||
- Bar-close timestamp conventions (alignment, DST behavior around US transitions).
|
||||
- [x] **Pass criteria (measured on the weekend-filtered bar grid — the aggregator's NY-anchored `_is_forex_weekend` decides which bars exist for signal purposes, so raw timestamp overlap includes coverage-only diffs like Tiingo's Sunday bars and is NOT the criterion):** ≥ 99.5% overlap of *signal-eligible* bar timestamps; Close p95 |Δ| ≤ 0.01% of price; no systematic bias (mean Δ ≈ 0); no unexplained multi-hour gaps unique to one feed. (Raw-overlap result 2026-08-14: 94.6–96.6% vs Tiingo — entirely explained by Tiingo's ~22h/week of flat Sunday bars, which the weekend filter gates out.)
|
||||
- [x] Write findings to `backtest/research/feed_divergence_<date>.md` with tables and the exact commands used.
|
||||
|
||||
## Phase 2 — Signal-level replay parity
|
||||
|
||||
**Goal:** measure how often candle divergence actually changes a *decision*.
|
||||
|
||||
- [x] Replay bar-by-bar through the real `analyze_from_dataframes` (news off) for both feeds over the same ≥ 12-month window, per TF (M5, M15, H1). Reuse the gpu_backtest replay harness; disable the parquet cache (`--no-cache`) or use separate cache keys per provider.
|
||||
- [x] Compare per bar: signal direction, composite score, quality grade, gate outcomes (which gate fired), setup presence, strategy label (XAU ATR / XAU_PB / XAU_CP), entry/SL/TP prices.
|
||||
- [x] Metrics:
|
||||
- % bars with identical direction+grade.
|
||||
- Direction flips, grade changes, gate-only changes (setup appears/disappears).
|
||||
- Distribution of composite Δ on mismatched bars.
|
||||
- For matching signals: mean/max |Δ| of entry, SL, TP1/TP2 in $ and in ATR fractions.
|
||||
- [x] **Result (Oanda↔Tiingo):** 98.32% / 98.16% same direction (M15/M5), zero opposite-sign flips; 93.41% / 94.21% same direction+grade+setup. The original ≥97% direction+grade proposal is too strict for benign grade-letter changes and was not met; fallback acceptance is direction ≥98%, zero sign flips, and marginal/boundary-hour-only flips. See `signal_parity_2026-08-14.md`.
|
||||
|
||||
## Phase 3 — Backtest re-pin on the new feed
|
||||
|
||||
**Goal:** a new pinned book on the new provider's bars, evaluated under rule 1 acceptance criteria.
|
||||
|
||||
- [x] **Provider selection:** `gpu_backtest.py` has **no `--provider` flag** — it builds the provider from `data_providers.active` in config.json (`gpu_backtest.py:1753`). A cross-provider run requires flipping that config line. The parquet cache is already keyed per provider (`gpu_backtest.py:984` — `oanda_`-prefixed files vs legacy unprefixed TD files), so TD and Oanda data cannot mix in cache; `--no-cache` is only needed to force a refetch.
|
||||
- [x] Run the full backtest on new-feed bars, both cost assumptions:
|
||||
```bash
|
||||
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.30 --export-csv backtest/repin_<provider>_m15_030.csv
|
||||
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.65 --export-csv backtest/repin_<provider>_m15_065.csv
|
||||
```
|
||||
(repeat for each TF under test; note the `Backend:` log line and provider in the report per `backtest/instructions_for_backtest.txt`).
|
||||
- [x] Split into train/validation halves; require **positive total R in both halves and PF ≥ 1.3 at realistic cost** (rule 1), with trailing exits and next-open fills untouched.
|
||||
- [x] Marginal configs additionally pass the **2024-06 → 2025-03 OOS window** (within-year splits alone have approved configs that later failed OOS).
|
||||
- [x] ~~Compare against the pinned Twelve-Data book trade-by-trade~~ **AMENDED 2026-08-14:** skipped with justification — Phase 1 showed TD bars disagree on direction 19% of the time on M5 with a systematic ~$1 bias, so a TD-pinned book is a different instrument's book and a trade-level diff carries no decision value. Phantom-trade classes remain documented (e.g. the weekend-calendar fix already dropped exactly one live-impossible M5 trade).
|
||||
- [x] **Deliverable:** re-pinned book + report in `backtest/research/`; update the reference numbers in CLAUDE.md if the book is adopted. Current-HEAD M5 canonical book: `backtest/oanda_m5_12mo_065_nopb_20260814.csv`, +106.82R PF 1.51.
|
||||
|
||||
## Phase 4 — Live shadow comparison
|
||||
|
||||
**Goal:** confirm parity under real fetch depths, caching, and boundary timing — not just history.
|
||||
|
||||
- [ ] Run a shadow scan loop (no orders, no Telegram) for **≥ 2 weeks**: each candle close, scan with both providers, log direction/grade/gates/setup to a side-by-side file or SQLite table.
|
||||
- [ ] Include at least one weekend transition and one news-heavy day in the window.
|
||||
- [ ] Track: scan-to-scan agreement rate, feed-lag incidents ("feed lagging — just-closed bar missing"), staleness warnings, spread readings (Oanda real vs Tiingo default).
|
||||
- [ ] **Acceptance thresholds (baseline = Phase 2 replay parity):** ≥ 98% direction agreement; **zero** opposite-sign (BUY↔SELL) flips; direction flips confined to actionable↔NEUTRAL at marginal composites, clustered only at Friday-close/Sunday-reopen (UTC 22–23) and Asian (03–04) hours; setup-level drift (SL/TP) within the Phase 2 envelope (mean |Δ| ≲ $3, p95 ≲ $17 on M15); every feed-lag/staleness incident explained.
|
||||
- [ ] H4 caution: live H4 fetches only 380 bars vs unbounded backtest history (known ~2% indicator-signal flip from seeding alone) — keep H4 labeled "unproven" until a dedicated campaign closes this.
|
||||
|
||||
## Phase 5 — Cutover checklist
|
||||
|
||||
- [x] `config.json` `data_providers.active: "oanda"` and `data_providers.streaming: "tiingo"` set deliberately (verified 2026-08-14 — Oanda cannot stream; Tiingo streams).
|
||||
- [x] Cost model: **dual reporting adopted (owner decision 2026-08-14)** — $0.65 (measured Oanda round trip) is the realistic headline, $0.30 retained for cross-book comparability with TD-era books. ⚠️ **Cost-gate coupling:** `config.py round_trip_cost` (0.30) is NOT just a backtest knob — it feeds the live TP1 spread gate (`min_tp = round_trip_cost × min_tp_cost_multiple`, three sites in `signal_aggregator.py`). Raising it to 0.65 moves the TP1 floor from ~$9 to ~$19.50 and is a rule-1 revalidation event, not a config tweak. `round_trip_cost` stays 0.30.
|
||||
- [x] QA suite green: `python test_all.py` (381 tests; verified 2026-08-14).
|
||||
- [ ] Week 1 post-cutover: daily review of `logs/signals.db` rows vs shadow log; alert on grade/direction disagreement beyond the Phase 2 baseline rate.
|
||||
- [ ] Rollback: revert `data_providers.active`; pinned books of the previous feed remain valid for comparison. ⚠️ Twelve Data is a permanent *registered* fallback but is **signal-incompatible** (Phase 1) — on an Oanda candle outage, stop scanning rather than switching `active` to `twelve_data`; a TD-fed book cannot be compared to the Oanda baseline.
|
||||
- [x] Docs sync (CLAUDE.md rule): `docs/user_guide.html` backtest numbers reflect the Oanda-pinned books, not TD-era numbers.
|
||||
|
||||
## Open questions — RESOLVED 2026-08-14 (owner decisions)
|
||||
|
||||
- ~~Adopt which spread as canonical~~ → **dual reporting**: $0.65 realistic headline + $0.30 cross-book comparability; `round_trip_cost` stays 0.30 (gate coupling above).
|
||||
- ~~Tiingo role~~ → **streaming-only**. Tiingo remains registered for the WS tick stream and comparison research, but is not a designated candle fallback; candle fallback is a manual halt-and-decide, never an automatic TD failover.
|
||||
- H4 seeding campaign (parked since 2026-08-04) → **still needs an owner**; H4 stays "unproven" until then.
|
||||
@@ -1,154 +0,0 @@
|
||||
# Centralized Configuration (config.json) — Design
|
||||
|
||||
**Date:** 2026-07-21
|
||||
**Status:** Approved (user confirmed all four scoping decisions)
|
||||
|
||||
## Goal
|
||||
|
||||
Move deployment settings — Telegram credentials, data-provider API keys and
|
||||
selection, scanned pairs, scan timeframes, per-pair base blocked hours, alert
|
||||
quality boot default, log level — into a single gitignored `config.json` at the
|
||||
repo root, loaded once at startup.
|
||||
|
||||
**Hard constraint:** zero change to signal logic, indicator weights, thresholds,
|
||||
or gates. With an unedited `config.json` (or none at all), every produced
|
||||
`PlatformConfig` must be byte-identical to today's factory output. This is
|
||||
test-enforced (see Testing).
|
||||
|
||||
## Scoping decisions (user-confirmed 2026-07-21)
|
||||
|
||||
1. **Secrets live in config.json** (gitignored). A committed
|
||||
`config.json.example` carries placeholders. Env vars still override.
|
||||
The hardcoded credential fallbacks in `config.py` and `telegram_bot.py`
|
||||
are **deleted** — on a machine with neither config.json nor env vars, the
|
||||
bot refuses to start with an explicit error instead of silently using
|
||||
committed credentials. (Recommend rotating the Telegram token and Twelve
|
||||
Data key after merge; both are exposed in git history.)
|
||||
2. **Multi-provider is structure only**: a `data_providers` section with named
|
||||
provider blocks and an `active` selector. Only Twelve Data is implemented.
|
||||
No failover logic (nothing to fail over to yet).
|
||||
3. **Blocked hours: base list per pair only.** The validated per-timeframe
|
||||
deltas (M5 appends hour 9; M15 SELL-block 15/17 UTC) stay in
|
||||
`pair_signals/xauusd.py` next to their validation evidence. The overlay is
|
||||
applied **before** those deltas and updates **both** mirrored copies
|
||||
(`xau_strategy.blocked_hours_utc` and `backtest.blocked_hours_utc`) so they
|
||||
cannot drift.
|
||||
4. **Extra keys included:** scan timeframes, alert min quality (boot default),
|
||||
log level. Everything else (indicator params, RR geometry, M5 gates,
|
||||
CP knobs, news config) deliberately stays code-only.
|
||||
|
||||
## File schema
|
||||
|
||||
`config.json` (gitignored; `config.json.example` committed with placeholders
|
||||
for secrets and today's tuned values for everything else):
|
||||
|
||||
```json
|
||||
{
|
||||
"telegram": {
|
||||
"token": "<bot token>",
|
||||
"chat_id": "<owner chat id>",
|
||||
"channel_id": "<-100… or empty string to disable the channel feed>",
|
||||
"alert_min_quality": "C"
|
||||
},
|
||||
"data_providers": {
|
||||
"active": "twelve_data",
|
||||
"twelve_data": { "api_key": "<key>" }
|
||||
},
|
||||
"scanning": {
|
||||
"pairs": ["XAU/USD"],
|
||||
"timeframes": ["M5", "M15", "H1", "H4"]
|
||||
},
|
||||
"pairs": {
|
||||
"XAU/USD": { "blocked_hours_utc": [0, 1, 2, 10, 11, 13, 16, 21] }
|
||||
},
|
||||
"log_level": "INFO"
|
||||
}
|
||||
```
|
||||
|
||||
Timeframe strings use the `Timeframe` enum names (`M5`, `M15`, `H1`, `H4`, …).
|
||||
Pair keys use canonical symbols (`XAU/USD`).
|
||||
|
||||
## Architecture
|
||||
|
||||
### New module: `app_config.py`
|
||||
|
||||
Single responsibility: read + validate `config.json`, expose typed accessors,
|
||||
apply the pair-level overlay. No signal knowledge.
|
||||
|
||||
- `AppConfig` dataclass: `telegram` (token, chat_id, channel_id,
|
||||
alert_min_quality), `data_providers` (active name + per-provider dicts),
|
||||
`scan_pairs`, `scan_timeframes`, `pair_overrides`
|
||||
(symbol → {blocked_hours_utc}), `log_level`.
|
||||
- `load_app_config(path=REPO_ROOT/"config.json") -> AppConfig` — module-level
|
||||
cached (loaded once per process). Missing file → `AppConfig` of pure
|
||||
defaults + one warning log. Malformed JSON, unknown top-level/section keys,
|
||||
unknown pair symbols, hours outside 0–23, unknown timeframe names, unknown
|
||||
`active` provider → raise `AppConfigError` with a named message (fail fast
|
||||
at boot).
|
||||
- Precedence, applied inside the accessors:
|
||||
**env var → config.json → code default**. Secrets (`TELEGRAM_TOKEN`,
|
||||
`TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`) have **no
|
||||
code default** after this change.
|
||||
- `apply_pair_overrides(cfg: PlatformConfig, app: AppConfig) -> PlatformConfig`
|
||||
— sets the pair's base `blocked_hours_utc` on **both**
|
||||
`cfg.xau_strategy.blocked_hours_utc` and `cfg.backtest.blocked_hours_utc`,
|
||||
plus `cfg.scan_pairs` / `cfg.scan_timeframes`. Called by the pair factory
|
||||
(see below). A no-op when the pair has no override block.
|
||||
|
||||
### Wiring points (all consumers)
|
||||
|
||||
| Consumer | Change |
|
||||
|---|---|
|
||||
| `pair_signals/xauusd.py` | `make_config` calls `apply_pair_overrides` **before** the per-TF deltas (so M5 still appends hour 9 to whatever base the JSON sets, and mirrors stay in sync). |
|
||||
| `data_provider.py` | API key resolution becomes `app_config` accessor for the **active** provider (env `TWELVE_DATA_API_KEY` still wins). `config.twelve_data_api_key` field and its hardcoded default are deleted. Missing key keeps today's behavior: provider constructed with `_td=None`, errors on first fetch. |
|
||||
| `telegram_bot.py` | `TOKEN`, `CHAT_ID`, `CHANNEL_ID` come from `app_config` (hardcoded fallbacks deleted; missing token/chat_id → explicit startup error). `_alert_min_quality` boot default and `logging` level come from `app_config`. `SCAN_PAIRS` (today built from a bare `PlatformConfig()`) reads `app_config.scan_pairs` directly; `apply_pair_overrides` also stamps `cfg.scan_pairs`/`cfg.scan_timeframes` so the dataclass field stays truthful for any other reader. |
|
||||
| `main.py` | No structural change — it uses the factory, which now applies the overlay. |
|
||||
| `backtest/gpu_backtest.py` | No structural change — it uses the factory too, so an edited base blocked-hours list flows into replays identically to live (parity by construction). Data fetches pick up the provider key via `data_provider.py`. |
|
||||
| `config.py` | `twelve_data_api_key` field removed. Everything else untouched. |
|
||||
|
||||
### Multi-provider structure
|
||||
|
||||
`data_provider.py` gains a minimal registry:
|
||||
`PROVIDER_REGISTRY = {"twelve_data": ForexDataProvider}`. `active` selects
|
||||
from it; an unknown name fails at load. Adding a provider later = one config
|
||||
block + one adapter class registered here. No shared base class extracted now
|
||||
(YAGNI — with one provider there is nothing to abstract over yet).
|
||||
|
||||
## Error handling summary
|
||||
|
||||
| Condition | Behavior |
|
||||
|---|---|
|
||||
| No config.json | Defaults + warning. Secrets must come from env or startup fails (bot) / fetch fails (provider). |
|
||||
| Malformed JSON / bad values / unknown keys | `AppConfigError` at load, process exits with named message. |
|
||||
| Missing Telegram token or chat_id at bot boot | Explicit startup error naming the missing key and both places it can be set. |
|
||||
| Missing data-provider key | Same as today: provider builds, first fetch errors. |
|
||||
|
||||
## Migration
|
||||
|
||||
- Create the local `config.json` populated with the **current live values**
|
||||
(the ones being deleted from source) so the running bot keeps working
|
||||
unchanged. `.gitignore` gains `config.json`.
|
||||
- Commit `config.json.example` with placeholders.
|
||||
- CLAUDE.md: update the env-var table, repository map, and config section.
|
||||
- Post-merge recommendation (user action): rotate the Telegram token and
|
||||
Twelve Data key, since both are in git history.
|
||||
|
||||
## Testing (`test_all.py`, new module)
|
||||
|
||||
1. **Parity pin (the critical one):** a config.json containing the example
|
||||
defaults, and no config.json at all, both produce factory output identical
|
||||
to today's (full dataclass equality across all four TFs).
|
||||
2. Overlay mapping: JSON blocked hours land on **both** mirrored fields; M5
|
||||
factory still appends hour 9 on top of a custom base.
|
||||
3. Precedence: env var beats config.json beats default, per key.
|
||||
4. Validation: malformed JSON, bad hour, unknown pair/timeframe/provider each
|
||||
raise `AppConfigError`.
|
||||
5. Missing-file behavior: defaults + no crash.
|
||||
6. Existing factory-equivalence test stays green (equivalence is now defined
|
||||
over factory-with-overlay vs inline-config-with-overlay).
|
||||
|
||||
## Out of scope
|
||||
|
||||
Hot reload (config is read once at boot), a second concrete provider,
|
||||
failover, moving any tuned strategy value beyond the base blocked-hours list,
|
||||
news/indicator/geometry settings in JSON.
|
||||
@@ -1,252 +0,0 @@
|
||||
================================================================================
|
||||
MULTI-PROVIDER LLM LAYER - PLAN
|
||||
(DeepSeek for news summary, Anthropic Claude for agent panel)
|
||||
Written 2026-08-08. Plan only - no code has been changed.
|
||||
================================================================================
|
||||
|
||||
WHY
|
||||
---
|
||||
Today every LLM feature (news enrichment + agent panel) goes through one
|
||||
hardwired client: llm_client.py -> DeepSeekClient, a module singleton reached
|
||||
via get_llm_client(). This plan describes how to route different features to
|
||||
different providers - e.g. keep DeepSeek for cheap high-volume news scoring,
|
||||
put the agent panel on Anthropic Claude for better reasoning.
|
||||
|
||||
The repo already solved this problem once for market data: the providers/
|
||||
package (BaseDataProvider owns shared machinery, subclasses implement only
|
||||
_fetch_raw, a registry + one config knob selects the provider). This plan
|
||||
mirrors that proven pattern at the LLM layer, scaled down to a single module.
|
||||
|
||||
BLAST RADIUS (verified by grep)
|
||||
-------------------------------
|
||||
get_llm_client() call sites:
|
||||
- news_engine.py ....... 4 sites
|
||||
- agent_panel.py ....... 3 sites (+1 timeout getattr)
|
||||
- telegram_bot.py ...... 1 site (/status health; reads private _count,
|
||||
_count_day, _daily_cap fields)
|
||||
- test_all.py .......... 41 locations stub via lc._client = _StubLLM(...)
|
||||
Out of scope: static/news_summary.php (own PHP DeepSeek call + key file,
|
||||
separate deployment).
|
||||
|
||||
DESIGN INVARIANTS (carried over unchanged)
|
||||
------------------------------------------
|
||||
1. Fail-safe contract: every call returns Optional[Dict]; any failure -> None;
|
||||
callers keep their non-LLM fallbacks; LLM output never gates signals.
|
||||
2. Single attempt, no retries (news cache cadence is the retry); per-provider
|
||||
UTC daily cap; warn-once logging; health stamps (last_ok, last_error).
|
||||
3. Static system prompts stay byte-identical constants - provider-neutral,
|
||||
and they map to prompt caching on BOTH providers (DeepSeek prefix cache;
|
||||
Anthropic cache_control ephemeral).
|
||||
4. Test compatibility: all 41 existing lc._client stub sites keep working
|
||||
with zero edits.
|
||||
|
||||
================================================================================
|
||||
1. llm_client.py REFACTOR (~195 -> ~430 lines, no new files)
|
||||
================================================================================
|
||||
|
||||
1a. BaseLLMClient - shared machinery (extracted from today's DeepSeekClient)
|
||||
- __init__(api_key, model, timeout, daily_cap, enabled)
|
||||
- public: name ("deepseek"/"anthropic"), timeout property,
|
||||
usage() -> (used_today, cap) # so telegram_bot stops reading privates
|
||||
- owns: available property, _warn_once, _under_cap() (UTC-day counter,
|
||||
lock), health stamps, and the template method:
|
||||
|
||||
def chat_json(self, system_prompt, user_payload):
|
||||
if not self.available or not self._under_cap():
|
||||
return None
|
||||
try:
|
||||
return self._request(system_prompt, user_payload) # hook
|
||||
except <provider-agnostic error mapping>:
|
||||
# stamps last_error, returns None
|
||||
|
||||
- subclasses implement ONLY _request(system_prompt, payload) -> dict|None.
|
||||
|
||||
1b. DeepSeekClient(BaseLLMClient) - behavior-identical move
|
||||
Current requests-based logic relocates into _request: same URL,
|
||||
thinking:disabled, response_format:json_object, temperature 0,
|
||||
max_tokens 2000, sort_keys=True payload, _parse_response.
|
||||
Review criterion: Module 19 tests pass UNMODIFIED (they patch
|
||||
client._session, which stays).
|
||||
|
||||
1c. AnthropicClient(BaseLLMClient) - new, official SDK
|
||||
- Official `anthropic` Python SDK (add to requirements.txt), NOT raw
|
||||
requests. LAZY IMPORT inside __init__: package missing -> warn-once +
|
||||
client unavailable, so DeepSeek-only deployments never install it.
|
||||
- Constructor: anthropic.Anthropic(api_key=..., timeout=<configured>,
|
||||
max_retries=0) # house rule: single attempt
|
||||
- _request:
|
||||
resp = self._client.messages.create(
|
||||
model=self._model, # default "claude-opus-5"
|
||||
max_tokens=4000, # covers thinking + JSON reply
|
||||
output_config={"effort": "low"}, # cheap structured takes
|
||||
system=[{"type": "text", "text": system_prompt,
|
||||
"cache_control": {"type": "ephemeral"}}],
|
||||
messages=[{"role": "user",
|
||||
"content": json.dumps(user_payload, sort_keys=True)}],
|
||||
)
|
||||
if resp.stop_reason == "refusal": # Opus 5: HTTP 200, check first
|
||||
return None
|
||||
text = first text block in resp.content
|
||||
return self._parse_json(text) # tolerant: strip ```json fences,
|
||||
# json.loads, dict check
|
||||
- DELIBERATELY ABSENT (would 400 on Claude Opus 5): temperature (removed
|
||||
on Opus 5 - do NOT copy from the DeepSeek client), budget_tokens,
|
||||
prefills, response_format.
|
||||
- Error mapping: APIConnectionError -> "network",
|
||||
APIStatusError -> "http_<code>", parse fail -> "parse"
|
||||
(same last_error kinds the /status display already understands).
|
||||
- Defaults: model claude-opus-5 (config-overridable, e.g. claude-haiku-4-5
|
||||
for ~5x cheaper), timeout 30s (Claude slower than V4-Flash), own
|
||||
daily_request_cap 500.
|
||||
|
||||
1d. Registry + per-consumer routing
|
||||
_PROVIDERS = {"deepseek": DeepSeekClient, "anthropic": AnthropicClient}
|
||||
LLM_CONSUMERS = ("news", "agent_panel")
|
||||
_clients = {} # keyed by PROVIDER name -> two consumers on the
|
||||
# same provider share one instance and one cap
|
||||
_client = None # kept: GLOBAL TEST OVERRIDE
|
||||
|
||||
def get_llm_client(consumer="news"):
|
||||
if _client is not None: # tests: lc._client = stub
|
||||
return _client # -> shadows ALL consumers
|
||||
name = get_app_config().llm_routing.get(consumer, "deepseek")
|
||||
create-once per provider, return _clients[name]
|
||||
|
||||
def reset_llm_client(): # clears _client AND _clients
|
||||
|
||||
Keeping _client as a documented test-override global is what makes the
|
||||
41 existing stub sites work untouched.
|
||||
|
||||
================================================================================
|
||||
2. CONFIG (app_config.py + config.json.example)
|
||||
================================================================================
|
||||
- New top-level "anthropic" block, mirroring "deepseek":
|
||||
{api_key, model, enabled, timeout_seconds, daily_request_cap}
|
||||
plus anthropic_api_key() resolver: ANTHROPIC_API_KEY env -> config.json -> "".
|
||||
- New top-level "llm" ROUTING block:
|
||||
{"news": "deepseek", "agent_panel": "deepseek"}
|
||||
Validation: keys within {news, agent_panel}; values within registered
|
||||
provider names. Absent block -> {} -> everything on DeepSeek (today's
|
||||
behavior; zero-migration default). AppConfig.llm_routing: Dict[str, str].
|
||||
|
||||
- The target state is then PURE CONFIG - no code changes on switch day:
|
||||
|
||||
"llm": { "news": "deepseek", "agent_panel": "anthropic" },
|
||||
"anthropic": { "api_key": "", "model": "claude-opus-5", "enabled": true,
|
||||
"timeout_seconds": 30, "daily_request_cap": 500 }
|
||||
|
||||
================================================================================
|
||||
3. CONSUMER EDITS (mechanical, ~10 lines total)
|
||||
================================================================================
|
||||
- news_engine.py: 4 call sites -> get_llm_client("news")
|
||||
- agent_panel.py: 3 call sites -> get_llm_client("agent_panel");
|
||||
getattr(client, "_timeout", ...) -> public client.timeout
|
||||
- telegram_bot.py _news_health_lines: iterate the DISTINCT providers actually
|
||||
routed (news + panel); per provider print "<name>: used/cap calls today"
|
||||
via the new usage() accessor instead of private fields. DeepSeek keeps its
|
||||
"headlines scored" extra; cap/error warnings work off base-class stamps.
|
||||
- NO changes to prompts, payload builders, caches, or the panel's own
|
||||
max_panels_per_day budget - all provider-agnostic already.
|
||||
|
||||
================================================================================
|
||||
4. TESTS (new "Module 24: LLM provider routing", ~12 tests, ~200 lines)
|
||||
================================================================================
|
||||
Reuse _StubLLM; new _FakeAnthropic (fake messages.create returning a stub
|
||||
response object) injected as client._client.
|
||||
|
||||
1. app_config: anthropic block parses/validates/defaults empty.
|
||||
2. app_config: llm routing block - valid map round-trips; unknown consumer
|
||||
key, unknown provider name, non-dict -> AppConfigError; absent -> {}.
|
||||
3. anthropic_api_key resolves env -> config.json -> empty.
|
||||
4. Base contract on BOTH subclasses (parametrized): keyless/disabled -> no
|
||||
call; daily cap blocks + UTC-roll reset; failure stamps last_error,
|
||||
success stamps last_ok.
|
||||
5. AnthropicClient parse: clean JSON -> dict; ```json-fenced -> dict;
|
||||
non-JSON -> None; stop_reason "refusal" -> None; no text block -> None.
|
||||
6. AnthropicClient request shape: NO temperature key, max_retries=0,
|
||||
cache_control present on system block (assert on captured kwargs).
|
||||
7. AnthropicClient with `anthropic` package missing (simulated) ->
|
||||
available False, chat_json None, no crash at import.
|
||||
8. Routing: no llm block -> both consumers get the SAME DeepSeek singleton
|
||||
(shared cap); routed config -> news gets DeepSeekClient while
|
||||
agent_panel gets AnthropicClient; same-provider consumers share one
|
||||
instance.
|
||||
9. Global lc._client override shadows BOTH consumers (locks the 41 legacy
|
||||
stub sites' semantics).
|
||||
10. reset_llm_client() clears both the override and the per-provider cache.
|
||||
11. End-to-end: routing agent_panel->anthropic, attach_panel reaches the
|
||||
anthropic fake (5 calls) while a news call reaches the deepseek stub.
|
||||
12. telegram _news_health_lines renders per-provider lines from usage()
|
||||
without touching private fields.
|
||||
|
||||
GATE: Modules 19 and 23 must pass UNMODIFIED - the no-regression proof for
|
||||
the refactor.
|
||||
|
||||
================================================================================
|
||||
5. DOCS
|
||||
================================================================================
|
||||
- CLAUDE.md: llm_client bullet -> provider-abstracted (BaseLLMClient;
|
||||
deepseek + anthropic registered); config.json "llm" block routes consumers;
|
||||
absent = all-DeepSeek. Add ANTHROPIC_API_KEY to env-var list. Test counts.
|
||||
- config.json.example: both new blocks.
|
||||
|
||||
================================================================================
|
||||
COMMIT SEQUENCE (each commit green)
|
||||
================================================================================
|
||||
1. llm_client: extract BaseLLMClient; DeepSeekClient unchanged behavior
|
||||
(+ base-contract tests) - pure refactor, Modules 19/23 green untouched.
|
||||
2. config: anthropic block + llm consumer-routing block (+ config tests).
|
||||
3. llm_client: AnthropicClient (official SDK, lazy import) + requirements.txt
|
||||
(+ parse/shape tests).
|
||||
4. routing: per-consumer get_llm_client + health lines - consumer call-site
|
||||
edits, telegram usage() lines, routing/e2e tests.
|
||||
5. docs: CLAUDE.md + example config.
|
||||
|
||||
================================================================================
|
||||
VERIFICATION
|
||||
================================================================================
|
||||
- python test_all.py green at every commit (~374 -> ~386).
|
||||
- Live smoke A (no behavior change): default config, main.py --panel ->
|
||||
still DeepSeek, /status unchanged.
|
||||
- Live smoke B (the goal): llm.agent_panel="anthropic" + real
|
||||
ANTHROPIC_API_KEY, run --panel -> one real Claude panel (5 requests,
|
||||
validates JSON contract + latency + refusal-free path) while a news
|
||||
refresh still hits DeepSeek; /status shows two provider lines.
|
||||
- Backtest untouched by construction (no LLM in replay path); one 30-day
|
||||
run as a formality.
|
||||
|
||||
================================================================================
|
||||
EFFORT ESTIMATE
|
||||
================================================================================
|
||||
0 new files, 8 modified, ~520 lines, est. 3-5 hours.
|
||||
|
||||
llm_client.py base class + AnthropicClient + routing +235
|
||||
test_all.py Module 24 +200
|
||||
app_config.py anthropic + llm blocks, resolver +55
|
||||
telegram_bot.py per-provider health lines +15
|
||||
news_engine.py consumer tags on call sites +4
|
||||
agent_panel.py consumer tags on call sites +4
|
||||
config.json.example blocks +14
|
||||
requirements.txt anthropic +1
|
||||
CLAUDE.md docs +10
|
||||
|
||||
================================================================================
|
||||
KEY DECISIONS EMBEDDED IN THIS PLAN
|
||||
================================================================================
|
||||
- Official Anthropic SDK, not raw requests (current API guidance for Python;
|
||||
lazy import keeps it optional for DeepSeek-only installs).
|
||||
- Default Claude model: claude-opus-5 ($5/$25 per MTok; a full 5-call panel
|
||||
~ 8k in / 500 out ~ $0.05). Config-overridable to claude-haiku-4-5.
|
||||
- Two Claude gotchas handled: temperature is REMOVED on Opus 5 (sending it
|
||||
400s - must not be copied from the DeepSeek request body), and refusals
|
||||
arrive as HTTP 200 with stop_reason "refusal" (checked before parsing,
|
||||
mapped to the normal None/fallback path).
|
||||
- Caps are per-provider, not per-consumer: two consumers on one provider
|
||||
share its cap (today's semantics); splitting across two providers doubles
|
||||
total headroom naturally.
|
||||
- The panel's own max_panels_per_day budget is untouched - it sits above
|
||||
whichever provider serves it.
|
||||
- Not in scope: static/news_summary.php (own PHP client/key), streaming,
|
||||
tool use, per-consumer model overrides (add a "model" key per consumer
|
||||
later if ever needed).
|
||||
================================================================================
|
||||
+3
-2
@@ -65,8 +65,9 @@ class TiingoDataProvider(BaseDataProvider):
|
||||
}
|
||||
|
||||
# Window padding over the theoretical bar span: FX is closed ~2/7 of
|
||||
# the week (Fri 22:00 → Sun 22:00 UTC), so a calendar window must be
|
||||
# wider than `bars × candle_seconds` to yield the requested count.
|
||||
# the week (Fri 17:00 → Sun 18:00 New York; see providers/base.py), so
|
||||
# a calendar window must be wider than `bars × candle_seconds` to yield
|
||||
# the requested count.
|
||||
WINDOW_PAD_FACTOR = 1.6
|
||||
|
||||
def __init__(self, config: PlatformConfig, api_key: Optional[str] = None):
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Scratchpad — Oanda vs Tiingo candle parity check
|
||||
=================================================
|
||||
Fetches the last N closed XAU/USD candles from Oanda and Tiingo for
|
||||
M5/M15/H1 via the normal provider stack (make_data_provider →
|
||||
fetch_ohlcv: forming-candle drop + validate/clean included, i.e. exactly
|
||||
what the signal pipeline consumes) and reports how well they line up:
|
||||
|
||||
* last-bar timestamp per provider (feed-lag check)
|
||||
* timestamp mismatches (bars present in one feed only)
|
||||
* per-bar OHLC diff table for the common tail
|
||||
* max/mean |diff| per column, with the Close diff also in % of price
|
||||
|
||||
Both feeds are mid-price (Oanda mid of MBA candles; Tiingo resampled
|
||||
mid), so small diffs are expected — this measures how small.
|
||||
|
||||
Usage:
|
||||
venv/bin/python scripts/compare_oanda_tiingo.py [--n 10]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from config import PlatformConfig, Timeframe, XAUUSD
|
||||
from providers import make_data_provider
|
||||
|
||||
TIMEFRAMES = [Timeframe.M5, Timeframe.M15, Timeframe.H1]
|
||||
OHLC = ["Open", "High", "Low", "Close"]
|
||||
|
||||
|
||||
def fetch_tail(provider, tf, n):
|
||||
"""Last n closed bars from one provider, or None on failure."""
|
||||
try:
|
||||
df = provider.fetch_ohlcv(symbol=XAUUSD, timeframe=tf)
|
||||
except Exception as e: # scratchpad: report, don't crash
|
||||
print(f" !! {provider.name} {tf.value} fetch failed: {e}")
|
||||
return None
|
||||
return df.tail(n)
|
||||
|
||||
|
||||
def compare_tf(tf, oanda_df, tiingo_df, n):
|
||||
print(f"\n{'=' * 78}\n{tf.value} — last {n} closed bars\n{'=' * 78}")
|
||||
if oanda_df is None or tiingo_df is None:
|
||||
print(" skipped (a provider failed)")
|
||||
return
|
||||
|
||||
print(f" Oanda last bar: {oanda_df.index[-1]} "
|
||||
f"Tiingo last bar: {tiingo_df.index[-1]}")
|
||||
|
||||
common = oanda_df.index.intersection(tiingo_df.index)
|
||||
only_o = oanda_df.index.difference(tiingo_df.index)
|
||||
only_t = tiingo_df.index.difference(oanda_df.index)
|
||||
if len(only_o):
|
||||
print(f" bars only in Oanda : {[str(t) for t in only_o]}")
|
||||
if len(only_t):
|
||||
print(f" bars only in Tiingo: {[str(t) for t in only_t]}")
|
||||
if not len(common):
|
||||
print(" NO common timestamps — feeds are on different bar grids!")
|
||||
return
|
||||
|
||||
o = oanda_df.loc[common, OHLC]
|
||||
t = tiingo_df.loc[common, OHLC]
|
||||
diff = o - t
|
||||
|
||||
print(f"\n {len(common)}/{n} bars share timestamps "
|
||||
f"(window {common[0]} → {common[-1]})\n")
|
||||
|
||||
# Per-bar Close table — the number that matters most.
|
||||
hdr = f" {'bar close (UTC)':<20}{'Oanda C':>12}{'Tiingo C':>12}{'ΔC ($)':>10}"
|
||||
print(hdr + f"{'ΔO':>8}{'ΔH':>8}{'ΔL':>8}")
|
||||
for ts in common:
|
||||
row = (f" {str(ts):<20}{o.loc[ts, 'Close']:>12.2f}"
|
||||
f"{t.loc[ts, 'Close']:>12.2f}{diff.loc[ts, 'Close']:>10.3f}")
|
||||
row += (f"{diff.loc[ts, 'Open']:>8.3f}{diff.loc[ts, 'High']:>8.3f}"
|
||||
f"{diff.loc[ts, 'Low']:>8.3f}")
|
||||
print(row)
|
||||
|
||||
print("\n summary over common bars:")
|
||||
for col in OHLC:
|
||||
d = diff[col].abs()
|
||||
print(f" {col:<6} max |Δ| = {d.max():8.4f} mean |Δ| = {d.mean():8.4f}")
|
||||
px = o["Close"].iloc[-1]
|
||||
dc = diff["Close"].abs()
|
||||
print(f" Close Δ as % of price: max {dc.max() / px * 100:.4f}% "
|
||||
f"mean {dc.mean() / px * 100:.4f}%")
|
||||
exact = (diff[OHLC].abs() < 1e-9).all(axis=1).sum()
|
||||
print(f" bars with identical OHLC: {exact}/{len(common)}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--n", type=int, default=10,
|
||||
help="bars per timeframe to compare (default 10)")
|
||||
args = ap.parse_args()
|
||||
|
||||
cfg = PlatformConfig()
|
||||
oanda = make_data_provider(cfg, "oanda")
|
||||
tiingo = make_data_provider(cfg, "tiingo")
|
||||
|
||||
for tf in TIMEFRAMES:
|
||||
o_df = fetch_tail(oanda, tf, args.n)
|
||||
t_df = fetch_tail(tiingo, tf, args.n)
|
||||
compare_tf(tf, o_df, t_df, args.n)
|
||||
|
||||
print("\nDone. Note: Oanda=mid of MBA candles, Tiingo=resampled mid — "
|
||||
"sub-$1 diffs on gold are normal feed divergence, not a bug.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -11,7 +11,7 @@
|
||||
* - CACHE FOREVER: headlines are immutable, so each id costs at most
|
||||
* one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
|
||||
* the whitelist on purpose: summaries stay reachable after the
|
||||
* headline ages out of the 24 h feed window.
|
||||
* headline ages out of the 4 h feed window.
|
||||
* - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
|
||||
* errors (a perms mistake must not mean unmetered spend).
|
||||
*
|
||||
@@ -112,9 +112,7 @@ if (!is_array($news)) {
|
||||
}
|
||||
$hit = null;
|
||||
foreach ($news['headlines'] ?? [] as $h) {
|
||||
// Fallback sha1(): a publisher that predates the id field still
|
||||
// serves a valid whitelist during the restart window.
|
||||
$hid = $h['id'] ?? sha1($h['headline'] ?? '');
|
||||
$hid = $h['id'] ?? '';
|
||||
if ($hid === $id) {
|
||||
$hit = $h;
|
||||
break;
|
||||
|
||||
+2
-2
@@ -5289,9 +5289,9 @@ def _():
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# MODULE 21: 2026-08-04 audit-fix regressions (execution lifecycle)
|
||||
# MODULE 21: 2026-08-04 audit-fix regressions (parity, providers, backtester)
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
print("\n═══ Module 21: 2026-08-04 audit fixes — execution lifecycle ═══")
|
||||
print("\n═══ Module 21: 2026-08-04 audit fixes — parity, providers, backtester ═══")
|
||||
|
||||
|
||||
@test("GPU parity: ALL indicators + composite identical on every prefix")
|
||||
|
||||
Reference in New Issue
Block a user