chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
This commit is contained in:
Harwinder Sekhon
@@ -1,61 +1,21 @@
|
||||
Note down entery and exit Why we took trade?
|
||||
What were each indicator signal on that specific time when we took trade
|
||||
How many indicators agreed at trade time
|
||||
Always backtest using engine.py and signal_aggregator.py
|
||||
Dont take trades on weekends and Fridays
|
||||
Keep TP small 1.3 Reward ratio
|
||||
At the end of test make table of results and print all values you observed.
|
||||
Trade context
|
||||
|
||||
Time of day and trading session (Asian/London/NY/overlap)
|
||||
Day of week (track Mon/Tue/Wed/Thu performance separately)
|
||||
Market regime at entry: trending, ranging, or choppy (use ATR or ADX)
|
||||
Volatility level (ATR value or ATR percentile vs recent average)
|
||||
Pair/instrument traded
|
||||
News proximity (high-impact news within 30 min before/after entry)
|
||||
|
||||
Trade execution details
|
||||
|
||||
Spread at entry in pips
|
||||
Slippage on entry and exit (planned vs actual fill)
|
||||
Time in trade (minutes/hours from entry to exit)
|
||||
Maximum Adverse Excursion (MAE): how far against you it went
|
||||
Maximum Favorable Excursion (MFE): how far in profit before pullback
|
||||
|
||||
Risk metrics
|
||||
|
||||
Max consecutive losses (streak)
|
||||
Max consecutive wins (streak)
|
||||
Expectancy: (Win% x Avg Win) minus (Loss% x Avg Loss)
|
||||
Expectancy per hour in trade
|
||||
Recovery factor: Total R divided by Max DD
|
||||
|
||||
Indicator/signal analysis
|
||||
|
||||
Each indicator's standalone win rate when it agreed with the trade
|
||||
Confluence level performance (2 vs 3 vs 4+ indicators agreeing)
|
||||
Which indicator disagreed on losing trades (find the "canary")
|
||||
|
||||
Decision quality
|
||||
|
||||
Rate each setup A/B/C at entry, before knowing outcome
|
||||
Verify A-grade setups actually win more often
|
||||
Track system trades vs discretionary deviations separately
|
||||
|
||||
Process notes
|
||||
|
||||
Screenshot or chart snapshot at entry and exit
|
||||
Emotional state (calm, FOMO, revenge, tired)
|
||||
What would have happened if you held longer or exited sooner
|
||||
|
||||
Aggregate analysis at end of backtest
|
||||
|
||||
Equity curve, not just total R
|
||||
Performance breakdown by session, day, volatility regime
|
||||
Win rate over rolling 20-trade windows (check for edge decay)
|
||||
|
||||
Critical extra
|
||||
"Do not trade" log: setups flagged but skipped, and what happened after
|
||||
Backtest reporting checklist — ForexBot2 (XAU/USD)
|
||||
==================================================
|
||||
Every backtest report must record:
|
||||
- The exact command line (window, --primary-tf, --spread, and any --set /
|
||||
--quality / --strategies / --confirm-tfs overrides) and the "Backend:" log line.
|
||||
- Trades, total R, win rate, profit factor and max drawdown for the whole
|
||||
window AND for both halves, at both cost conventions ($0.65 measured-Oanda
|
||||
headline, $0.30 cross-book comparability) — CLAUDE.md rule 1.
|
||||
- Per trade (the exported CSV already carries these): entry/exit time and
|
||||
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
||||
indicator's signal / value / agreement at decision time.
|
||||
- Breakdowns by session/hour, weekday, regime and volatility regime
|
||||
(backtest/research/book_report.py) and, for any candidate change, the
|
||||
removed/added/shifted trade diff against the control book
|
||||
(backtest/research/book_diff.py).
|
||||
- The md5 of the exported book whenever the run is a parity or regression check.
|
||||
Replays go through the real signal code (analyze_from_dataframes) — never a
|
||||
re-implementation.
|
||||
|
||||
|
||||
--- 2026-09-06: decision-time feature export ---
|
||||
|
||||
@@ -95,7 +95,7 @@ Cost guardrails already in place: 5 requests/panel, `max_panels_per_day`
|
||||
injection (technical persona only), prompt-contract assertions,
|
||||
panel row update, rowid hand-off + no-op paths). Existing attach tests
|
||||
stub `ap._dp` to stay off the network.
|
||||
- **Live dry test** (`panel_payload_drytest_2026-08-15.py`, fabricated
|
||||
- **Live dry test** (`panel_payload_drytest_2026-08-15.py` — script removed 2026-09-06, raw replies kept in the `.jsonl` — fabricated
|
||||
numbers only, 13 real DeepSeek requests total): **7/7 probes pass** —
|
||||
coherent-bullish → support@70; bearish-vs-BUY → oppose@85; genuinely
|
||||
mixed → neutral@30; empty payload → neutral@20 (contract); no invented
|
||||
|
||||
@@ -164,7 +164,7 @@ def base_metrics(r: pd.Series, df: pd.DataFrame | None = None) -> dict:
|
||||
|
||||
|
||||
def _pf(r: pd.Series) -> float:
|
||||
"""Profit factor, hour_analysis-compatible (loss side is r <= 0)."""
|
||||
"""Profit factor (loss side is r <= 0)."""
|
||||
pos = r[r > 0].sum()
|
||||
neg = -r[r <= 0].sum()
|
||||
return float(pos / max(neg, 1e-9))
|
||||
|
||||
@@ -6,9 +6,8 @@ Usage:
|
||||
[--start YYYY-MM-DD --end YYYY-MM-DD] [--out DIR]
|
||||
|
||||
A "burst" is one bar whose range is >= K x the PRIOR bar's Wilder ATR(14) —
|
||||
the same definition burst_profile.py uses (that module computes it inline
|
||||
inside burst_table() and exposes no reusable helper, so the formula is
|
||||
replicated here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
||||
the same definition every burst study in this folder has used (spelled
|
||||
out here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
||||
TR.ewm(alpha=1/14, min_periods=14, adjust=False).mean(), ratio =
|
||||
(H-L) / ATR.shift(1)).
|
||||
|
||||
@@ -60,7 +59,7 @@ def load_bars(parquet: str, start: str | None = None,
|
||||
|
||||
|
||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
"""Wilder ATR — identical to the inline formula in burst_profile.py."""
|
||||
"""Wilder ATR (the definition every burst study in this folder uses)."""
|
||||
h, l, c = df["High"], df["Low"], df["Close"]
|
||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||
axis=1).max(axis=1)
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
"""Burst-candle frequency by UTC hour — the empirical "hedge fund hours".
|
||||
|
||||
Usage:
|
||||
python backtest/research/burst_profile.py PARQUET_GLOB
|
||||
|
||||
Per UTC bar-close hour of the given bars: % of bars whose range is
|
||||
>= K x the PRIOR bar's Wilder ATR(14), for K in {3,4,5}; mean bar
|
||||
range; and mean signed drift over the 6 bars following a >=4x burst
|
||||
(does price continue or snap back?). Analysis tooling only.
|
||||
"""
|
||||
import glob
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def burst_table(parquet_glob: str, bar_minutes: int = 5):
|
||||
files = sorted(glob.glob(parquet_glob))
|
||||
if not files:
|
||||
raise SystemExit(f"no parquet matches {parquet_glob}")
|
||||
df = pd.concat([pd.read_parquet(f) for f in files]).sort_index()
|
||||
df = df[~df.index.duplicated(keep="last")]
|
||||
h, l, c = df["High"], df["Low"], df["Close"]
|
||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||
axis=1).max(axis=1)
|
||||
atr = tr.ewm(alpha=1 / 14, min_periods=14, adjust=False).mean()
|
||||
ratio = (h - l) / atr.shift(1)
|
||||
close_hour = (df.index + pd.Timedelta(minutes=bar_minutes)).hour
|
||||
fwd6 = c.shift(-6) - c # signed move over the 6 bars after this one
|
||||
out = pd.DataFrame({
|
||||
"bars": ratio.groupby(close_hour).size(),
|
||||
**{f"pct_ge_{k}x": (ratio >= k).groupby(close_hour).mean() * 100
|
||||
for k in (3, 4, 5)},
|
||||
"mean_range": (h - l).groupby(close_hour).mean(),
|
||||
"post4x_drift6": fwd6[ratio >= 4].groupby(close_hour[ratio >= 4]).mean(),
|
||||
})
|
||||
return out.round(3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(burst_table(sys.argv[1],
|
||||
int(sys.argv[2]) if len(sys.argv) > 2 else 5).to_string())
|
||||
@@ -0,0 +1,81 @@
|
||||
# Feed Divergence Study — Oanda vs Tiingo vs Twelve Data (Phase 1)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Test plan:** `docs/data_source_change_test_plan.md`
|
||||
**Tool:** `scripts/feed_divergence_study.py` (fetch_range over 365d, parquet-cached)
|
||||
**Window:** 2025-08-14 → 2026-08-14 (12 months); Oanda-vs-TD truncated at TD cache head (2026-07-23)
|
||||
|
||||
## Headline
|
||||
|
||||
Three mid-price feeds for the *same* XAU/USD instrument are **not** interchangeable:
|
||||
|
||||
| Pair | mean \|ΔClose\| | p95 \|ΔClose\| | direction agree (M5/M15/H1) | bias | extra bars |
|
||||
|---|---|---|---|---|---|
|
||||
| Oanda vs **Tiingo** | ~$0.13 | ~$0.29 | 97.7% / 98.3% / 98.5% | ~0 | Tiingo +3–5% (weekend) |
|
||||
| Oanda vs **Twelve Data** | ~$3.0 | ~$11.9 | 81.0% / 88.7% / 92.7% | −$1.06 (TD high) | — |
|
||||
|
||||
- **Oanda ↔ Tiingo are near-identical** on shared bars (~1–2% of ATR on close), but Tiingo emits ~22 hours of Sunday bars/week that Oanda omits.
|
||||
- **Oanda ↔ Twelve Data diverge massively** — ~$3 mean close gap (≈30% of M5 ATR, ~30% of M15 ATR), a systematic ~$1 bias (TD quoted ~$1 high), and 19% of M5 bars even disagree on bar *direction*. This is the documented "Oanda/TD divergence" that motivated re-pinning the books on Oanda bars.
|
||||
|
||||
## Oanda vs Tiingo — detailed
|
||||
|
||||
Both feeds: same bar grid, same head/tail timestamps. Tiingo simply has more bars (Sunday bars).
|
||||
|
||||
### Coverage (12 months)
|
||||
|
||||
| TF | Oanda bars | Tiingo bars | shared | Oanda-only | Tiingo-only |
|
||||
|---|---|---|---|---|---|
|
||||
| M5 | 70,901 | 73,241 | 70,807 (96.6%) | 94 (0.13%) | 2,434 (3.32%) |
|
||||
| M15 | 23,640 | 24,922 | 23,615 (94.7%) | 25 (0.11%) | 1,307 (5.24%) |
|
||||
| H1 | 5,912 | 6,246 | 5,909 (94.6%) | 3 (0.05%) | 337 (5.40%) |
|
||||
| H4 | 1,597 | 1,657 | 1,597 (96.4%) | 0 | 60 (3.62%) |
|
||||
|
||||
- **Tiingo-only bars cluster exclusively on Sundays** (e.g. M15: 88/Sunday; H1: 22/Sunday; H4: 5/Sunday). Tiingo's FX endpoint resamples continuously and does **not** apply the FX weekend calendar — it emits ~22h of flat/stale Sunday bars that Oanda omits.
|
||||
- Oanda-only bars cluster on holidays (2025-11-27 Thanksgiving, 2025-12-10, 2026-06-25) — a handful of bars, negligible.
|
||||
|
||||
### Per-bar OHLC diffs (Oanda − Tiingo, $)
|
||||
|
||||
| TF | Close mean | Close p95 | Close max | \|ΔClose\| / ATR14 (mean / p95) |
|
||||
|---|---|---|---|---|
|
||||
| M5 | $0.123 | $0.285 | $26.10 | 2.1% / 5.0% |
|
||||
| M15 | $0.128 | $0.290 | $26.10 | 1.3% / 2.9% |
|
||||
| H1 | $0.145 | $0.335 | $26.10 | 0.7% / 1.6% |
|
||||
| H4 | $0.168 | $0.340 | $17.53 | 0.4% / 0.8% |
|
||||
|
||||
- Bias is ~zero on every TF (no systematic one-feed-high).
|
||||
- Tail outliers (p99/max up to $26–88) occur during fast moves — the two feeds' bar extremes disagree on the exact tick, as expected.
|
||||
- **Direction agreement is 97.7–98.4%**, return correlation 0.995–0.9997.
|
||||
- Divergence is concentrated at the Friday-close / Sunday-reopen boundary (UTC hours 20–22; mean |ΔClose| ~2–4× the 08–19h baseline).
|
||||
|
||||
### Phase 1 pass/fail vs plan thresholds
|
||||
|
||||
| Criterion | Plan threshold | Result | Verdict |
|
||||
|---|---|---|---|
|
||||
| timestamp overlap | ≥ 99.5% | 94.6–96.6% | **FAIL** (Tiingo weekend bars) |
|
||||
| Close p95 \|Δ\| | ≤ 0.01% of price | ~0.0067% | PASS |
|
||||
| no systematic bias | mean Δ ≈ 0 | ~0 | PASS |
|
||||
|
||||
The overlap "failure" is a *coverage* difference, not a data-quality one: the shared bars agree tightly, but Tiingo's extra Sunday bars mean the feeds have a different bar *count*. The signal pipeline's own `_is_forex_weekend` filter (NY-anchored) gates those Sunday bars, so live impact depends on whether any code trusts the feed's raw bar set directly.
|
||||
|
||||
## Oanda vs Twelve Data — detailed (M5/M15/H1, cached TD parquets)
|
||||
|
||||
| TF | common | mean \|ΔClose\| | median \|ΔClose\| | p95 | max | bias (TD−Oanda) | dir agree |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| M5 | 66,246 | $2.999 | $1.118 | $11.934 | $101.85 | +1.05 | 80.96% |
|
||||
| M15 | 22,090 | $2.951 | $1.051 | $11.848 | $101.85 | +1.08 | 88.71% |
|
||||
| H1 | 5,526 | $2.902 | $0.985 | $11.908 | $70.94 | +1.06 | 92.67% |
|
||||
|
||||
- The ~$3 mean / ~$1 median / ~$12 p95 gap is **~23× larger** than the Oanda–Tiingo gap, and the ~$1 bias is systematic (Twelve Data quotes gold ~$1 high vs Oanda's true mid).
|
||||
- M5 direction agreement of **81%** means one bar in five disagrees on direction — M5 signals on TD bars are effectively a *different instrument* than on Oanda bars.
|
||||
- This matches the prior Oanda/TD M15 study of 2026-07-24 (`oanda_td_divergence.py`, removed 2026-09-06 as superseded by this report; mean |ΔClose| ~$3.3) and confirms the standing rule: **pinned Twelve-Data books are not comparable to Oanda live signals without a re-pin.**
|
||||
|
||||
## Conclusions for the data-source switch
|
||||
|
||||
1. **Oanda → Tiingo (or back) is signal-safe in magnitude** (price diffs ~1–2% of ATR, 98% direction agreement), *provided* the weekend-bar coverage difference is handled — Tiingo's Sunday bars must be gated by the aggregator's NY-anchored weekend filter, not by raw bar count.
|
||||
2. **Oanda → Twelve Data is a recalibration, not a formality.** Phase 2/3 must quantify the actual signal/book delta; the $3/$1 bias alone guarantees a large fraction of marginal bars flip.
|
||||
3. Re-pinning the books on Oanda bars (Phase 3) remains mandatory before any TD-pinned baseline is treated as a live expectation.
|
||||
|
||||
## Next steps
|
||||
|
||||
- Phase 2: signal-level replay parity (`analyze_from_dataframes` on both feeds) to turn price-diff stats into flip rates.
|
||||
- Phase 3: backtest re-pin on Oanda bars at $0.30 and $0.65.
|
||||
@@ -0,0 +1,2 @@
|
||||
tf,a,b,a_bars,b_bars,common,a_only,b_only,dir_agree_pct,ret_corr,mean_abs_dclose,p95_abs_dclose,max_abs_dclose,dclose_pct_atr_mean
|
||||
5min,oanda,tiingo,70901,73241,70807,94,2434,97.674000,0.994970,0.123070,0.285000,26.105000,2.147942
|
||||
|
@@ -1,158 +0,0 @@
|
||||
"""Fetch XAU/USD M15 candles from the Oanda v20 REST API.
|
||||
|
||||
Mechanical data-fetching infrastructure only — no analysis here.
|
||||
Paginates GET /v3/instruments/XAU_USD/candles from a start cursor
|
||||
through "now" and writes the result to a parquet file.
|
||||
|
||||
Run from the repo root:
|
||||
venv/bin/python backtest/research/fetch_oanda_m15.py
|
||||
"""
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
import requests
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from app_config import oanda_api_token, oanda_base_url # noqa: E402
|
||||
|
||||
INSTRUMENT = "XAU_USD"
|
||||
GRANULARITY = "M15"
|
||||
START = datetime(2025, 7, 1, 0, 0, 0, tzinfo=timezone.utc)
|
||||
OUT_PATH = Path(__file__).parent / "oanda_xau_m15_20250701_now.parquet"
|
||||
COUNT = 5000
|
||||
REQUEST_SLEEP_S = 0.5
|
||||
FRESHNESS_WINDOW = timedelta(minutes=30)
|
||||
|
||||
|
||||
def _rfc3339(dt: datetime) -> str:
|
||||
return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
|
||||
|
||||
def fetch_all_candles():
|
||||
token = oanda_api_token()
|
||||
base_url = oanda_base_url()
|
||||
if not token:
|
||||
raise RuntimeError(
|
||||
"No Oanda API token resolved (env OANDA_API_TOKEN / "
|
||||
"config.json oanda.api_token). Aborting.")
|
||||
|
||||
url = f"{base_url}/v3/instruments/{INSTRUMENT}/candles"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Accept-Datetime-Format": "RFC3339",
|
||||
}
|
||||
|
||||
cursor = START
|
||||
include_first = True
|
||||
all_rows = []
|
||||
n_requests = 0
|
||||
|
||||
while True:
|
||||
params = {
|
||||
"granularity": GRANULARITY,
|
||||
"price": "MBA",
|
||||
"count": COUNT,
|
||||
"dailyAlignment": 0,
|
||||
"alignmentTimezone": "Etc/UTC",
|
||||
"from": _rfc3339(cursor),
|
||||
"includeFirst": "true" if include_first else "false",
|
||||
}
|
||||
|
||||
resp = requests.get(url, headers=headers, params=params, timeout=30)
|
||||
n_requests += 1
|
||||
if resp.status_code != 200:
|
||||
raise RuntimeError(
|
||||
f"Oanda API request failed: status={resp.status_code}\n"
|
||||
f"URL: {resp.url}\n"
|
||||
f"Response body: {resp.text}")
|
||||
|
||||
payload = resp.json()
|
||||
candles = payload.get("candles", [])
|
||||
|
||||
for c in candles:
|
||||
if not c.get("complete", False):
|
||||
continue
|
||||
mid = c["mid"]
|
||||
bid = c["bid"]
|
||||
ask = c["ask"]
|
||||
all_rows.append({
|
||||
"Datetime": pd.Timestamp(c["time"]).tz_convert("UTC")
|
||||
if pd.Timestamp(c["time"]).tzinfo is not None
|
||||
else pd.Timestamp(c["time"]).tz_localize("UTC"),
|
||||
"Open": float(mid["o"]),
|
||||
"High": float(mid["h"]),
|
||||
"Low": float(mid["l"]),
|
||||
"Close": float(mid["c"]),
|
||||
"Volume": int(c["volume"]),
|
||||
"bid_o": float(bid["o"]),
|
||||
"bid_h": float(bid["h"]),
|
||||
"bid_l": float(bid["l"]),
|
||||
"bid_c": float(bid["c"]),
|
||||
"ask_o": float(ask["o"]),
|
||||
"ask_h": float(ask["h"]),
|
||||
"ask_l": float(ask["l"]),
|
||||
"ask_c": float(ask["c"]),
|
||||
})
|
||||
|
||||
print(f" request {n_requests}: from={params['from']} "
|
||||
f"got {len(candles)} candles (complete kept so far: "
|
||||
f"{len(all_rows)})", file=sys.stderr)
|
||||
|
||||
if not candles:
|
||||
break
|
||||
|
||||
last_time = pd.Timestamp(candles[-1]["time"])
|
||||
if last_time.tzinfo is None:
|
||||
last_time = last_time.tz_localize("UTC")
|
||||
else:
|
||||
last_time = last_time.tz_convert("UTC")
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
# Oanda's "count" includes the "from" boundary bar itself, so with
|
||||
# includeFirst=false every full page comes back as count-1, not
|
||||
# count (verified empirically: count=N, includeFirst=false always
|
||||
# yields N-1 candles when a full page of data exists ahead of the
|
||||
# cursor). Compare against the count actually expected for this
|
||||
# request's includeFirst setting, not the raw requested count.
|
||||
expected_full_page = COUNT if include_first else COUNT - 1
|
||||
got_fewer_than_requested = len(candles) < expected_full_page
|
||||
near_now = (now - last_time.to_pydatetime()) < FRESHNESS_WINDOW
|
||||
|
||||
if got_fewer_than_requested or near_now:
|
||||
break
|
||||
|
||||
cursor = last_time.to_pydatetime()
|
||||
include_first = False
|
||||
time.sleep(REQUEST_SLEEP_S)
|
||||
|
||||
return all_rows, n_requests
|
||||
|
||||
|
||||
def main():
|
||||
print(f"Fetching {INSTRUMENT} {GRANULARITY} candles from "
|
||||
f"{_rfc3339(START)} through now...", file=sys.stderr)
|
||||
|
||||
rows, n_requests = fetch_all_candles()
|
||||
|
||||
if not rows:
|
||||
raise RuntimeError("No complete candles were fetched.")
|
||||
|
||||
df = pd.DataFrame(rows)
|
||||
df = df.set_index("Datetime")
|
||||
df = df.sort_index()
|
||||
df = df[~df.index.duplicated(keep="last")]
|
||||
|
||||
df.to_parquet(OUT_PATH)
|
||||
|
||||
print(f"\nSaved {len(df)} rows to {OUT_PATH}", file=sys.stderr)
|
||||
print(f"Requests made: {n_requests}", file=sys.stderr)
|
||||
print(f"First timestamp: {df.index[0]}", file=sys.stderr)
|
||||
print(f"Last timestamp: {df.index[-1]}", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,40 +0,0 @@
|
||||
"""Per-hour P&L breakdown of a backtest trade book.
|
||||
|
||||
Usage:
|
||||
python backtest/research/hour_analysis.py BOOK.csv BAR_MINUTES
|
||||
|
||||
Reports, per UTC hour (both entry hour and DECISION hour = entry_time
|
||||
minus one bar, the clock blocked_hours_utc evaluates): trade count,
|
||||
total R, mean R, win rate, profit factor. Also a per-(hour, direction)
|
||||
table. Analysis tooling only — never imported by the live code.
|
||||
"""
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def hour_tables(book_csv: str, bar_minutes: int):
|
||||
df = pd.read_csv(book_csv)
|
||||
et = pd.to_datetime(df["entry_time"], utc=True, format="mixed")
|
||||
df["entry_hour"] = et.dt.hour
|
||||
df["decision_hour"] = (et - pd.Timedelta(minutes=bar_minutes)).dt.hour
|
||||
|
||||
def table(group_cols):
|
||||
g = df.groupby(group_cols)["r_multiple"]
|
||||
out = g.agg(count="count", total_r="sum", mean_r="mean",
|
||||
wr=lambda r: (r > 0).mean() * 100)
|
||||
out["pf"] = g.apply(
|
||||
lambda r: r[r > 0].sum() / max(-r[r <= 0].sum(), 1e-9))
|
||||
return out.round(2)
|
||||
|
||||
return (table("decision_hour"), table(["decision_hour", "direction"]),
|
||||
table("entry_hour"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
book, mins = sys.argv[1], int(sys.argv[2])
|
||||
by_dec, by_dec_dir, by_ent = hour_tables(book, mins)
|
||||
print(f"\n── {book} · by DECISION hour (bar-close clock) ──")
|
||||
print(by_dec.to_string())
|
||||
print(f"\n── by (decision hour, direction) ──")
|
||||
print(by_dec_dir.to_string())
|
||||
@@ -1,199 +0,0 @@
|
||||
"""
|
||||
Oanda vs Twelve Data bar-divergence study — XAU/USD M15, 12 months.
|
||||
====================================================================
|
||||
Compares the mid-price OHLC bars the signal engine would see from each
|
||||
provider over the same window. Purpose: quantify whether migrating the
|
||||
signal feed from Twelve Data to Oanda is a formality (bars ~identical)
|
||||
or a recalibration (bars differ at a scale that moves indicators).
|
||||
|
||||
Inputs:
|
||||
--td Twelve Data parquet (raw bars from backtest/cache/)
|
||||
--oanda Oanda parquet (from fetch_oanda_m5.py / fetch_oanda_m15.py)
|
||||
--start / --end analysis window (UTC dates)
|
||||
|
||||
Outputs:
|
||||
printed report
|
||||
backtest/research/oanda_td_divergence_by_hour.csv
|
||||
backtest/research/oanda_td_bar_diffs.parquet (per-bar diffs, common index)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
RESEARCH_DIR = Path(__file__).parent
|
||||
PIP = 0.10 # XAU pip in $
|
||||
|
||||
|
||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
hl = df["High"] - df["Low"]
|
||||
hc = (df["High"] - df["Close"].shift(1)).abs()
|
||||
lc = (df["Low"] - df["Close"].shift(1)).abs()
|
||||
tr = pd.concat([hl, hc, lc], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1.0 / period, adjust=False).mean()
|
||||
|
||||
|
||||
def pct_stats(s: pd.Series) -> dict:
|
||||
a = s.abs()
|
||||
return {
|
||||
"mean": a.mean(), "median": a.median(),
|
||||
"p90": a.quantile(0.90), "p95": a.quantile(0.95),
|
||||
"p99": a.quantile(0.99), "max": a.max(),
|
||||
"bias(mean signed)": s.mean(),
|
||||
}
|
||||
|
||||
|
||||
def fmt_stats(name: str, s: pd.Series) -> str:
|
||||
st = pct_stats(s)
|
||||
return (f" {name:<6} mean ${st['mean']:.3f} med ${st['median']:.3f} "
|
||||
f"p95 ${st['p95']:.3f} p99 ${st['p99']:.3f} max ${st['max']:.2f} "
|
||||
f"bias {st['bias(mean signed)']:+.4f}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--td", default=str(RESEARCH_DIR.parent / "cache" /
|
||||
"XAU_USD_15min_20240522_20260723.parquet"))
|
||||
ap.add_argument("--oanda", default=str(RESEARCH_DIR /
|
||||
"oanda_xau_m15_20250701_now.parquet"))
|
||||
ap.add_argument("--start", default="2025-07-24")
|
||||
ap.add_argument("--end", default="2026-07-23")
|
||||
args = ap.parse_args()
|
||||
|
||||
td = pd.read_parquet(args.td)
|
||||
oa = pd.read_parquet(args.oanda)
|
||||
|
||||
start = pd.Timestamp(args.start, tz="UTC")
|
||||
end = pd.Timestamp(args.end, tz="UTC") + pd.Timedelta(days=1)
|
||||
start = max(start, td.index[0], oa.index[0])
|
||||
end = min(end, td.index[-1], oa.index[-1])
|
||||
td = td.loc[(td.index >= start) & (td.index <= end)]
|
||||
oa = oa.loc[(oa.index >= start) & (oa.index <= end)]
|
||||
|
||||
print("=" * 78)
|
||||
print("OANDA vs TWELVE DATA — XAU/USD M15 bar divergence")
|
||||
print(f"Window: {start} -> {end} ({(end - start).days} days)")
|
||||
print("=" * 78)
|
||||
|
||||
# ── Coverage ─────────────────────────────────────────────────────
|
||||
common = td.index.intersection(oa.index)
|
||||
td_only = td.index.difference(oa.index)
|
||||
oa_only = oa.index.difference(td.index)
|
||||
print(f"\nCOVERAGE")
|
||||
print(f" Twelve Data bars : {len(td):>7,}")
|
||||
print(f" Oanda bars : {len(oa):>7,}")
|
||||
print(f" common : {len(common):>7,}")
|
||||
print(f" TD-only : {len(td_only):>7,} "
|
||||
f"({100 * len(td_only) / max(len(td), 1):.2f}% of TD)")
|
||||
print(f" Oanda-only : {len(oa_only):>7,} "
|
||||
f"({100 * len(oa_only) / max(len(oa), 1):.2f}% of Oanda)")
|
||||
for label, idx in (("TD-only", td_only), ("Oanda-only", oa_only)):
|
||||
if len(idx):
|
||||
by_date = pd.Series(1, index=idx).groupby(idx.date).sum()
|
||||
top = by_date.sort_values(ascending=False).head(8)
|
||||
print(f" {label} bars cluster on: "
|
||||
+ ", ".join(f"{d} ({n})" for d, n in top.items()))
|
||||
by_hour = pd.Series(1, index=idx).groupby(idx.hour).sum()
|
||||
top_h = by_hour.sort_values(ascending=False).head(6)
|
||||
print(f" {label} by UTC hour: "
|
||||
+ ", ".join(f"{h:02d}h ({n})" for h, n in top_h.items()))
|
||||
|
||||
t = td.loc[common]
|
||||
o = oa.loc[common]
|
||||
|
||||
# ── Scale references ─────────────────────────────────────────────
|
||||
atr = wilder_atr(t).dropna()
|
||||
mean_price = t["Close"].mean()
|
||||
print(f"\nSCALE REFERENCES (Twelve Data feed)")
|
||||
print(f" mean close : ${mean_price:,.2f}")
|
||||
print(f" mean bar range : ${(t['High'] - t['Low']).mean():.3f}")
|
||||
print(f" mean ATR14 : ${atr.mean():.3f} median ${atr.median():.3f}")
|
||||
|
||||
# ── Per-field diffs (TD − Oanda mid) ─────────────────────────────
|
||||
print(f"\nPER-BAR DIFFS (TD − Oanda, $; pip = $0.10)")
|
||||
diffs = {}
|
||||
for col in ("Open", "High", "Low", "Close"):
|
||||
d = t[col] - o[col]
|
||||
diffs[col] = d
|
||||
print(fmt_stats(col, d))
|
||||
|
||||
dclose = diffs["Close"].abs()
|
||||
print(f"\n |ΔClose| as % of mean ATR14 : "
|
||||
f"mean {100 * dclose.mean() / atr.mean():.1f}% "
|
||||
f"p95 {100 * dclose.quantile(0.95) / atr.mean():.1f}%")
|
||||
|
||||
print(f"\nTHRESHOLD EXCEEDANCE (|ΔClose|)")
|
||||
for thr in (0.10, 0.30, 0.65, 1.00, 2.00, 5.00):
|
||||
pct = 100 * (dclose > thr).mean()
|
||||
print(f" > ${thr:<5.2f}: {pct:6.2f}% of bars ({(dclose > thr).sum():,})")
|
||||
|
||||
# ── Direction / return agreement ─────────────────────────────────
|
||||
t_ret = t["Close"].pct_change()
|
||||
o_ret = o["Close"].pct_change()
|
||||
# only consecutive common bars (gap = one M15 step)
|
||||
step_ok = pd.Series(common, index=common).diff().eq(pd.Timedelta(minutes=15))
|
||||
tr, orr = t_ret[step_ok], o_ret[step_ok]
|
||||
corr = tr.corr(orr)
|
||||
t_dir = np.sign(t["Close"] - t["Open"])
|
||||
o_dir = np.sign(o["Close"] - o["Open"])
|
||||
both_nonflat = (t_dir != 0) & (o_dir != 0)
|
||||
dir_agree = 100 * (t_dir[both_nonflat] == o_dir[both_nonflat]).mean()
|
||||
print(f"\nAGREEMENT")
|
||||
print(f" close-return correlation (consecutive bars): {corr:.5f} "
|
||||
f"(n={step_ok.sum():,})")
|
||||
print(f" bar direction (sign of close−open) agree : {dir_agree:.2f}%")
|
||||
|
||||
# ── By-hour breakdown ────────────────────────────────────────────
|
||||
hours = common.hour
|
||||
rows = []
|
||||
for h in range(24):
|
||||
m = hours == h
|
||||
if not m.any():
|
||||
continue
|
||||
row = {"hour_utc": h, "bars": int(m.sum()),
|
||||
"mean_abs_dclose": dclose[m].mean(),
|
||||
"p95_abs_dclose": dclose[m].quantile(0.95),
|
||||
"mean_abs_dhigh": diffs["High"][m].abs().mean(),
|
||||
"mean_abs_dlow": diffs["Low"][m].abs().mean()}
|
||||
if "bid_c" in o.columns:
|
||||
row["oanda_median_spread"] = (
|
||||
(o["ask_c"] - o["bid_c"])[m].median())
|
||||
rows.append(row)
|
||||
by_hour = pd.DataFrame(rows)
|
||||
out_hour = RESEARCH_DIR / "oanda_td_divergence_by_hour.csv"
|
||||
by_hour.to_csv(out_hour, index=False, float_format="%.4f")
|
||||
print(f"\nBY UTC HOUR (written to {out_hour.name})")
|
||||
print(by_hour.to_string(index=False,
|
||||
float_format=lambda x: f"{x:.3f}"))
|
||||
|
||||
# ── By-month drift ───────────────────────────────────────────────
|
||||
print(f"\nBY MONTH (mean |ΔClose|, $)")
|
||||
by_month = dclose.groupby(common.to_period("M")).mean()
|
||||
for p, v in by_month.items():
|
||||
print(f" {p}: {v:.3f}")
|
||||
|
||||
# ── Oanda real spread (bonus, ties to Dukascopy study) ───────────
|
||||
if "bid_c" in o.columns:
|
||||
spread = o["ask_c"] - o["bid_c"]
|
||||
liquid = ~o.index.hour.isin([21, 22, 23, 0])
|
||||
print(f"\nOANDA REAL SPREAD (close bid/ask, $)")
|
||||
print(f" all bars : median {spread.median():.3f} "
|
||||
f"p25 {spread.quantile(0.25):.3f} p75 {spread.quantile(0.75):.3f}")
|
||||
print(f" liquid hours : median {spread[liquid].median():.3f}")
|
||||
print(f" 21-00 UTC : median {spread[~liquid].median():.3f}")
|
||||
|
||||
# ── Persist per-bar diffs ────────────────────────────────────────
|
||||
out = pd.DataFrame({
|
||||
"td_close": t["Close"], "oa_close": o["Close"],
|
||||
"d_open": diffs["Open"], "d_high": diffs["High"],
|
||||
"d_low": diffs["Low"], "d_close": diffs["Close"],
|
||||
})
|
||||
out_path = RESEARCH_DIR / "oanda_td_bar_diffs.parquet"
|
||||
out.to_parquet(out_path)
|
||||
print(f"\nPer-bar diffs written to {out_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,25 +0,0 @@
|
||||
hour_utc,bars,mean_abs_dclose,p95_abs_dclose,mean_abs_dhigh,mean_abs_dlow,oanda_median_spread
|
||||
0,1029,3.3483,13.9460,3.2906,3.4310,0.7900
|
||||
1,1028,3.3483,12.8292,3.3477,3.6398,0.7700
|
||||
2,1028,3.0288,10.8960,2.9469,3.1317,0.7700
|
||||
3,1028,2.8783,11.1300,2.8419,2.9992,0.7600
|
||||
4,1028,2.8232,10.4710,2.8165,2.8455,0.7400
|
||||
5,1028,2.9268,10.6027,2.9234,3.0038,0.7400
|
||||
6,1028,3.0466,10.9265,2.9487,3.0796,0.7200
|
||||
7,1028,3.0286,11.3512,3.0301,3.0797,0.6700
|
||||
8,1028,3.0223,11.1483,2.8989,3.0801,0.6600
|
||||
9,1028,3.0460,11.6345,2.9307,3.0609,0.6600
|
||||
10,1028,2.9280,10.9015,2.8647,3.0029,0.6600
|
||||
11,1028,3.0340,11.4712,2.9591,2.9472,0.6600
|
||||
12,1028,3.0986,11.3282,2.9412,3.1407,0.7000
|
||||
13,1028,3.4014,12.3343,3.1077,3.4228,0.7300
|
||||
14,1028,3.5074,11.3628,3.3960,3.6569,0.6900
|
||||
15,1028,3.4514,11.8982,3.4124,3.6733,0.6700
|
||||
16,1028,3.1275,12.5155,3.2038,3.3061,0.6700
|
||||
17,1020,2.9826,12.6435,3.0064,3.1008,0.6900
|
||||
18,1015,2.8721,12.5480,2.9101,2.9996,0.6900
|
||||
19,1004,2.8637,12.1335,2.8824,2.9454,0.6900
|
||||
20,996,2.8006,11.9325,2.8486,2.8577,0.7600
|
||||
21,335,1.1085,4.8600,1.1212,1.2034,0.9400
|
||||
22,676,4.0827,13.6787,4.8422,4.6761,0.8300
|
||||
23,1027,3.2722,12.5725,3.5617,3.7747,0.8700
|
||||
|
@@ -1,271 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
panel_payload_drytest_2026-08-15.py — live DeepSeek validation of the
|
||||
enriched agent-panel payloads + hardened prompts (2026-08-15).
|
||||
|
||||
Spends ~6 REAL DeepSeek requests (against the shared 500/day cap) with
|
||||
FABRICATED payloads — made-up prices/indicators, no real market data —
|
||||
to verify the new technical-persona contract end to end:
|
||||
|
||||
P1 coherent bullish evidence -> technical persona SUPPORTS
|
||||
P2 bearish evidence vs BUY brief -> technical persona OPPOSES
|
||||
P3 conflicting evidence -> NEUTRAL or low confidence
|
||||
P4 empty evidence -> NEUTRAL with confidence < 30
|
||||
(the anti-hallucination contract)
|
||||
P5 hallucination trap -> note invents no price levels
|
||||
P6 output discipline -> <=20-word notes, exact JSON keys
|
||||
P7 synthesizer over 4 supports -> AGREE verdict
|
||||
|
||||
Usage: venv/bin/python backtest/research/panel_payload_drytest_2026-08-15.py
|
||||
Exit code 0 = all probes passed. Raw replies land in
|
||||
backtest/research/panel_drytest_2026-08-15.jsonl
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
import agent_panel as ap # noqa: E402
|
||||
from app_config import deepseek_api_key, get_app_config # noqa: E402
|
||||
from llm_client import get_llm_client # noqa: E402
|
||||
|
||||
ARTIFACT = Path(__file__).with_suffix(".jsonl")
|
||||
|
||||
|
||||
# ── Fabrication helpers (all numbers are invented) ───────────────────
|
||||
|
||||
def _staircase(start: float, step: float, n: int) -> pd.DataFrame:
|
||||
"""n synthetic bars drifting `step` per bar; last close = start+n*step."""
|
||||
rows, price = [], start
|
||||
for _ in range(n):
|
||||
o, c = price, price + step
|
||||
rows.append({"Open": o, "Close": c,
|
||||
"High": max(o, c) + abs(step) * 0.4,
|
||||
"Low": min(o, c) - abs(step) * 0.4})
|
||||
price = c
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
def _choppy(center: float, n: int) -> pd.DataFrame:
|
||||
rows, price, sign = [], center, 1
|
||||
for _ in range(n):
|
||||
step = sign * 1.5
|
||||
o, c = price, price + step
|
||||
rows.append({"Open": o, "Close": c,
|
||||
"High": max(o, c) + 0.6, "Low": min(o, c) - 0.6})
|
||||
price, sign = c, -sign
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
_FAKE_PRICE = 2000.00 # deliberately made-up
|
||||
_FAKE_ATR = 5.0
|
||||
|
||||
_BRIEF_BUY = {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY",
|
||||
"quality": "A", "score": 38.5, "confidence": 62,
|
||||
"price": _FAKE_PRICE, "market_regime": "trending_up",
|
||||
"volatility_regime": "normal", "mtf_agreement": True}
|
||||
|
||||
_BULL_INDICATORS = {
|
||||
"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0},
|
||||
"rsi": {"signal": "BUY", "confidence": 65, "value": 58.0},
|
||||
"macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25},
|
||||
"bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7},
|
||||
"adx": {"signal": "BUY", "confidence": 70, "value": 32.0},
|
||||
"roc": {"signal": "BUY", "confidence": 55, "value": 0.35},
|
||||
"keltner": {"signal": "BUY", "confidence": 65, "value": 3.1},
|
||||
"candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0},
|
||||
}
|
||||
|
||||
_BEAR_INDICATORS = {
|
||||
"ema_trend": {"signal": "STRONG_SELL", "confidence": 85, "value": 0.1},
|
||||
"rsi": {"signal": "SELL", "confidence": 70, "value": 74.0},
|
||||
"macd": {"signal": "STRONG_SELL", "confidence": 80, "value": -1.4},
|
||||
"bollinger": {"signal": "SELL", "confidence": 60, "value": 0.95},
|
||||
"adx": {"signal": "SELL", "confidence": 70, "value": 35.0},
|
||||
"roc": {"signal": "SELL", "confidence": 60, "value": -0.5},
|
||||
"keltner": {"signal": "SELL", "confidence": 65, "value": -2.8},
|
||||
"candlestick": {"signal": "SELL", "confidence": 55, "value": -2.0},
|
||||
}
|
||||
|
||||
_MIXED_INDICATORS = {
|
||||
"ema_trend": {"signal": "BUY", "confidence": 55, "value": 0.6},
|
||||
"rsi": {"signal": "NEUTRAL", "confidence": 40, "value": 51.0},
|
||||
"macd": {"signal": "SELL", "confidence": 55, "value": -0.3},
|
||||
"bollinger": {"signal": "NEUTRAL", "confidence": 35, "value": 0.5},
|
||||
"adx": {"signal": "NEUTRAL", "confidence": 40, "value": 18.0},
|
||||
"roc": {"signal": "SELL", "confidence": 50, "value": -0.1},
|
||||
"keltner": {"signal": "BUY", "confidence": 50, "value": 0.8},
|
||||
"candlestick": {"signal": "NEUTRAL", "confidence": 30, "value": 0.0},
|
||||
}
|
||||
|
||||
|
||||
def _tech_payload(indicators, mtf, candles_df=None, key_levels=None,
|
||||
brief=None, exhaustion=0):
|
||||
payload = {"brief": brief or dict(_BRIEF_BUY),
|
||||
"indicators": indicators, "mtf": mtf,
|
||||
"key_levels": key_levels or {}, "exhaustion": exhaustion}
|
||||
if candles_df is not None:
|
||||
# the REAL production normalizer, not a hand-rolled copy
|
||||
payload["candles"] = {
|
||||
"15min": ap._normalize_candles(candles_df, _FAKE_ATR,
|
||||
len(candles_df))}
|
||||
return payload
|
||||
|
||||
|
||||
# ── Probe runner ─────────────────────────────────────────────────────
|
||||
|
||||
_RESULTS = []
|
||||
|
||||
|
||||
def _record(probe, payload, reply, take, passed, detail):
|
||||
_RESULTS.append({"probe": probe, "pass": bool(passed),
|
||||
"detail": detail,
|
||||
"stance": getattr(take, "stance", None),
|
||||
"confidence": getattr(take, "confidence", None),
|
||||
"note": getattr(take, "note", None)})
|
||||
with ARTIFACT.open("a") as fh:
|
||||
fh.write(json.dumps({"probe": probe, "payload": payload,
|
||||
"raw_reply": reply}, default=str) + "\n")
|
||||
mark = "PASS" if passed else "FAIL"
|
||||
print(f"[{mark}] {probe}: {detail}")
|
||||
if take is not None:
|
||||
print(f" -> {take.stance} @ {take.confidence:.0f} "
|
||||
f"| {take.note}")
|
||||
|
||||
|
||||
def _ask(payload):
|
||||
"""Real API call through the production client + production
|
||||
validation (the same path run_panel uses)."""
|
||||
reply = get_llm_client().chat_json(ap._TECH_SYSTEM, payload)
|
||||
return reply, ap._validate_take("technical", reply)
|
||||
|
||||
|
||||
def _want(probe: str) -> bool:
|
||||
"""--only P3 runs just that probe (saves requests on re-runs)."""
|
||||
if "--only" not in sys.argv:
|
||||
return True
|
||||
return sys.argv[sys.argv.index("--only") + 1] == probe
|
||||
|
||||
|
||||
def main() -> int:
|
||||
app = get_app_config()
|
||||
if not deepseek_api_key(app):
|
||||
print("ABORT: no DeepSeek API key (env DEEPSEEK_API_KEY / "
|
||||
"config.json deepseek.api_key).")
|
||||
return 2
|
||||
if app.deepseek_settings.get("enabled", True) is False:
|
||||
print("ABORT: deepseek.enabled is false in config.json.")
|
||||
return 2
|
||||
if ARTIFACT.exists() and "--only" not in sys.argv:
|
||||
ARTIFACT.unlink() # full run = fresh wire log; --only appends
|
||||
|
||||
mtf_agree = {"5min": {"signal": "BUY", "score": 30.0},
|
||||
"1h": {"signal": "STRONG_BUY", "score": 45.0}}
|
||||
levels = {"resistance": 2012.0, "support": 1985.0}
|
||||
|
||||
# P1 — coherent bullish, with rising ATR staircase (last close 2000)
|
||||
if _want("P1"):
|
||||
p1 = _tech_payload(_BULL_INDICATORS, mtf_agree,
|
||||
candles_df=_staircase(_FAKE_PRICE - 20 * 0.6,
|
||||
0.6, 20),
|
||||
key_levels=levels)
|
||||
reply, take = _ask(p1)
|
||||
_record("P1 coherent-bullish", p1, reply, take,
|
||||
take is not None and take.stance == "support"
|
||||
and take.confidence >= 50,
|
||||
"expect support @ conf>=50")
|
||||
|
||||
# P2 — bearish everything against a BUY brief, falling staircase
|
||||
if _want("P2"):
|
||||
brief2 = dict(_BRIEF_BUY, market_regime="trending_down",
|
||||
mtf_agreement=False)
|
||||
mtf_sell = {"5min": {"signal": "SELL", "score": -28.0},
|
||||
"1h": {"signal": "STRONG_SELL", "score": -41.0}}
|
||||
p2 = _tech_payload(_BEAR_INDICATORS, mtf_sell, brief=brief2,
|
||||
candles_df=_staircase(_FAKE_PRICE + 20 * 0.6,
|
||||
-0.6, 20),
|
||||
key_levels=levels, exhaustion=3)
|
||||
reply, take = _ask(p2)
|
||||
_record("P2 bearish-vs-BUY-brief", p2, reply, take,
|
||||
take is not None and take.stance == "oppose",
|
||||
"expect oppose (BUY brief fights all evidence)")
|
||||
|
||||
# P3 — genuinely mixed evidence: MTF neutral both ways, ranging
|
||||
# regime, choppy candles. (An H1 SELL against a BUY is a *listed
|
||||
# conflict* in the prompt — oppose is correct there, not mixed.)
|
||||
if _want("P3"):
|
||||
brief3 = dict(_BRIEF_BUY, market_regime="ranging", score=12.0,
|
||||
confidence=35, mtf_agreement=False)
|
||||
mtf_mixed = {"5min": {"signal": "NEUTRAL", "score": 4.0},
|
||||
"1h": {"signal": "NEUTRAL", "score": -3.0}}
|
||||
p3 = _tech_payload(_MIXED_INDICATORS, mtf_mixed, brief=brief3,
|
||||
candles_df=_choppy(_FAKE_PRICE, 20))
|
||||
reply, take = _ask(p3)
|
||||
_record("P3 conflicting", p3, reply, take,
|
||||
take is not None and (take.stance == "neutral"
|
||||
or take.confidence < 60),
|
||||
"expect neutral or conf<60 on genuinely mixed evidence")
|
||||
|
||||
# P4 — empty evidence: contract demands neutral @ conf<30
|
||||
if _want("P4"):
|
||||
p4 = _tech_payload({}, {}, key_levels=None)
|
||||
reply, take = _ask(p4)
|
||||
_record("P4 empty-evidence", p4, reply, take,
|
||||
take is not None and take.stance == "neutral"
|
||||
and take.confidence < 30,
|
||||
"contract: thin/missing evidence -> neutral, conf<30")
|
||||
|
||||
# P5 — hallucination trap: no key_levels, no candles; the note must
|
||||
# not cite any 3+-digit number absent from the payload
|
||||
if _want("P5"):
|
||||
p5 = _tech_payload(_BULL_INDICATORS, mtf_agree, key_levels=None)
|
||||
reply, take = _ask(p5)
|
||||
invented = []
|
||||
if take is not None:
|
||||
payload_blob = json.dumps(p5)
|
||||
for num in re.findall(r"\d{3,}", take.note):
|
||||
if num not in payload_blob:
|
||||
invented.append(num)
|
||||
_record("P5 hallucination-trap", p5, reply, take,
|
||||
take is not None and not invented,
|
||||
f"invented numbers in note: {invented or 'none'}")
|
||||
|
||||
# P6 — output discipline across the persona replies gathered above
|
||||
notes = [r for r in _RESULTS if r["note"]]
|
||||
wordy = [r["probe"] for r in notes if len(r["note"].split()) > 20]
|
||||
_record("P6 output-discipline", {}, None, None,
|
||||
notes and not wordy,
|
||||
f"notes >20 words: {wordy or 'none'} "
|
||||
f"(cap is 15; 20 = fail threshold)")
|
||||
|
||||
# P7 — synthesizer over four fabricated SUPPORT takes
|
||||
if _want("P7"):
|
||||
takes = [{"persona": p, "stance": "support", "confidence": c,
|
||||
"note": "evidence aligned"}
|
||||
for p, c in (("macro", 70), ("technical", 80),
|
||||
("risk", 65), ("contrarian", 60))]
|
||||
synth_payload = {"direction": "BUY", "takes": takes}
|
||||
reply = get_llm_client().chat_json(ap._SYNTH_SYSTEM, synth_payload)
|
||||
verdict = reply.get("verdict") if isinstance(reply, dict) else None
|
||||
rationale = (reply or {}).get("rationale", "")
|
||||
_record("P7 synthesizer", synth_payload, reply, None,
|
||||
verdict == "AGREE" and len(str(rationale).split()) <= 40,
|
||||
f"expect AGREE + <=40-word rationale; got {verdict!r} "
|
||||
f"({len(str(rationale).split())} words)")
|
||||
|
||||
n_pass = sum(1 for r in _RESULTS if r["pass"])
|
||||
n_api = sum(1 for r in _RESULTS if r["probe"] != "P6 output-discipline")
|
||||
print(f"\n{n_pass}/{len(_RESULTS)} probes passed "
|
||||
f"({n_api} API requests spent; "
|
||||
f"raw wire log: {ARTIFACT.name})")
|
||||
return 0 if n_pass == len(_RESULTS) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,40 @@
|
||||
# Backtest Re-pin on Oanda bars — verification (Phase 3)
|
||||
|
||||
**Date:** 2026-08-14 (verification; original books were generated 2026-08-04, current-HEAD M5 canonical book exported 2026-08-14)
|
||||
**Scope:** confirm the pinned Oanda books exist, satisfy rule 1, and quantify the Oanda↔TD delta that motivated them.
|
||||
|
||||
## Re-pin state
|
||||
|
||||
The re-pin is **already done** — full Oanda-bar books exist in `backtest/oanda_*.csv` for M5/M15/H1/H4, at both cost assumptions, with in-sample (12mo), out-of-sample (2024-06 → 2025-03), and 5-year variants. The shipped configs are `m5` (pullback OFF — `_nopb`) and `m15`.
|
||||
|
||||
### M5 canonical-reference reconciliation (2026-08-14)
|
||||
|
||||
The dated current-HEAD export `backtest/oanda_m5_12mo_065_nopb_20260814.csv` is the canonical M5 12-month reference going forward: **577 trades, +106.82R, PF 1.51**, with both 12-month halves positive (+50.80R / +56.02R; PF 1.46 / 1.55). It covers 2025-08-14 → 2026-08-14, runs the shipped M5 factory (XAU_PB off), charges $0.65 round trip, and uses next-open/raw-path/close-only-trail realism unchanged.
|
||||
|
||||
Earlier M5 snapshots (`oanda_m5_12mo_065_nopb.csv`: 586 / +98.11R / 1.45; `m5_12mo_shipped.csv`: 588 / +96.60R / 1.44; and the 555 / +98.64R ledger entry) remain historical evidence from different HEADs around the weekend-calendar and related QA work. They must not be cited as the current-HEAD reference.
|
||||
|
||||
### Rule-1 summary at realistic cost ($0.65 round trip)
|
||||
|
||||
| TF | 12mo in-sample | 12mo H1 / H2 | OOS (2024-06→2025-03) |
|
||||
|---|---|---|---|
|
||||
| M15 | +73.93R PF 1.53 | +66.71 / +7.23 (PF 1.10) | +38.77R PF 1.36 |
|
||||
| M5 (shipped, nopb) | +106.82R PF 1.51 (current HEAD) | +50.80 / +56.02 (PF 1.46 / 1.55) | +9.52R PF 1.34 |
|
||||
| H1 | +23.97R PF 1.55 | — | +5.15R PF 1.13 |
|
||||
| H4 | +16.14R PF 2.28 | — | +1.31R PF 1.14 |
|
||||
|
||||
- **M5 (shipped)** passes rule 1 cleanly: both current-HEAD 12mo halves positive with PF ≥ 1.3, and OOS positive with PF 1.34.
|
||||
- **M15** passes in-sample and OOS (PF 1.53 / 1.36) but its second half is soft (PF 1.10) — positive R, below the 1.3 bar. Pre-existing condition, not a re-pin artifact.
|
||||
- **H1/H4** pass in-sample but OOS is marginal (PF 1.13–1.14) — consistent with the standing "H4 unproven for auto-execution" note.
|
||||
|
||||
## Oanda ↔ Twelve Data trade-level delta
|
||||
|
||||
Not worth a trade-by-trade book diff: Phase 1 already showed the two feeds disagree on **bar direction** 19% of the time on M5 (11% on M15), with a ~$3 mean / ~$1 systematic close gap. A TD-pinned book is a different instrument's book — this is the documented re-pin motivator, and the Oanda books above are the replacement baseline. A formal trade-level reconciliation would show near-total mismatch and add no decision value.
|
||||
|
||||
## Deliverable status
|
||||
|
||||
- [x] Re-pinned books exist on Oanda bars (12mo + OOS + 5yr, $0.30 + $0.65).
|
||||
- [x] Rule 1 verified for the shipped TFs (M5 clean; M15 in-sample+OOS pass, H2 soft; H1/H4 OOS marginal).
|
||||
- [x] Cost model: $0.65 matches measured Oanda spread (~$0.65–0.72); $0.30 retained for cross-book comparability.
|
||||
- [x] Oanda↔TD delta quantified (Phase 1) — no further book diff needed.
|
||||
|
||||
Remaining Phase-3 follow-up: none. Owner decision 2026-08-14 retained dual reporting: $0.65 is the realistic headline while $0.30 remains for TD-era cross-book comparability and the live TP spread gate.
|
||||
@@ -44,7 +44,9 @@ book directly. Windows (cache start = start − 62 days):
|
||||
Acceptance: positive total R in both 12mo halves, PF ≥ 1.3, OOS not worse
|
||||
than control, and a candidate that removes a net-positive bucket for a PF-only
|
||||
gain is rejected (big-candle precedent). Runner:
|
||||
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py`.
|
||||
`<scratchpad>/campaign/run_campaign.sh`; summary: `summarize.py` (scratchpad
|
||||
tooling, not retained — both shipped gates are reproduced by a plain pair-factory
|
||||
run at HEAD; see the parity recipe in `sizing_campaign_2026-09.md`).
|
||||
|
||||
Candidates (all `--set`, no code):
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# Signal-Level Replay Parity — Oanda vs Tiingo (Phase 2)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Tool:** `scripts/signal_parity_study.py` (replays real `analyze_from_dataframes`, news off)
|
||||
**Window:** 2025-08-14 → 2026-08-14 (12 months); M15 stride 8, M5 stride 24 → ~2,900 sampled decisions each
|
||||
**Preceding:** Phase 1 price-diff study (`feed_divergence_2026-08-14.md`)
|
||||
|
||||
## Headline
|
||||
|
||||
On the ~94–96% of bars the two feeds share, **Oanda and Tiingo produce near-identical signals**:
|
||||
|
||||
| Metric | M15 | M5 |
|
||||
|---|---|---|
|
||||
| same direction | 98.32% | 98.16% |
|
||||
| same grade | 94.13% | 95.23% |
|
||||
| same setup presence | 98.83% | 99.49% |
|
||||
| same direction + grade + setup | 93.41% | 94.21% |
|
||||
| opposite-sign flips (BUY↔SELL) | **0** | **0** |
|
||||
| direction flips on actionable bars | 6.6% | 10.9% |
|
||||
| grade changed on actionable bars | 14.2% | 7.8% |
|
||||
| mean \|Δ final_score\| | 1.76 | 1.78 |
|
||||
| mean \|Δ primary_score\| | 1.73 | 1.77 |
|
||||
| same strategy tag | 100% | 100% |
|
||||
|
||||
Every direction flip is **actionable↔NEUTRAL**, never a sign reversal: the feeds never disagree on buy-vs-sell, only on *whether* a marginal bar crosses the action threshold. Flipping bars cluster at the Friday-close/Sunday-reopen boundary (UTC hours 22–23) and in the Asian hours (03–04).
|
||||
|
||||
## Setup levels diverge even when the signal agrees
|
||||
|
||||
On bars where **both** feeds emit a setup with the same tag:
|
||||
|
||||
| | M15 (n=119) | M5 (n=39) |
|
||||
|---|---|---|
|
||||
| entry mean \|Δ\| | $0.12 | $0.10 |
|
||||
| SL mean \|Δ\| | $1.83 | $2.79 |
|
||||
| TP1 mean \|Δ\| | $2.47 | $1.52 |
|
||||
| TP2 mean \|Δ\| | $3.74 | $2.46 |
|
||||
| TP1 p95 \|Δ\| | $17.4 | $2.3 |
|
||||
|
||||
Entry price is the current close (≈ identical across feeds), but **SL/TP are recomputed from ATR and recent swing extremes, which differ more** — mean SL drift ~$2–3 (p95 up to $10–26 on M15), i.e. 20–40% of ATR. So even an identical signal places stops/targets at meaningfully different prices on a different feed.
|
||||
|
||||
## Phase 2 pass/fail vs plan thresholds
|
||||
|
||||
| Criterion | Plan threshold | M15 / M5 | Verdict |
|
||||
|---|---|---|---|
|
||||
| identical direction+grade | ≥ 97% | 93.4% / 94.2% | **marginal** (below 97%, above 93%) |
|
||||
| opposite-sign flips | (none specified) | 0 / 0 | PASS |
|
||||
| flips clustered at marginal composites | no session/regime clustering | yes (boundary hours) | PASS |
|
||||
|
||||
The "identical direction+grade" metric lands slightly under the proposed 97% bar — but the shortfall is grade *letters* (A+ vs A vs B) changing on marginal composites, not direction. If the acceptance criterion is "no opposite-sign disagreement and ~98% direction identity", this passes comfortably.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Oanda ↔ Tiingo is **signal-equivalent for direction** (98% identical, zero sign flips) over 12 months. The residual deltas are:
|
||||
|
||||
1. **~6% of decisions differ in grade letter** on marginal composites (A+↔A↔B near thresholds).
|
||||
2. **SL/TP levels drift ~$2–4 mean** (up to ~$20 p95 on M15) even for identical signals — a real P&L difference at the trade level.
|
||||
3. **Tiingo's extra Sunday bars** (Phase 1) are gated by the aggregator's NY-anchored weekend filter, so they do not inject phantom signals, but they do slightly shift indicator history at Monday opens.
|
||||
|
||||
Net: switching Oanda→Tiingo changes a handful of marginal decisions and moves stop/target levels, but does not change *what the system is doing* (direction, strategy, setup presence). Oanda↔Twelve Data was not replayed — Phase 1 already showed 81% M5 bar-direction disagreement and a ~$3/$1 systematic gap, so a TD replay would trivially fail (that is the documented re-pin motivator).
|
||||
Reference in New Issue
Block a user