chore: keep only what the system needs — drop research tooling, study scripts, diagram, plan
Owner directive 2026-09-06: Dukascopy and similar research material is irrelevant to the running system; keep only code and files with a role in system functioning. Deleted (all in git history before this commit): - backtest/research/ tooling and data: book_report, book_diff, burst_map, burst_precursors, burst_indicator_map, open_trade_bursts, calendar_impact, watch_rule_eval, sizing_model, commit_ladder, llm_news_eval + llm_news_golden.csv, ff_calendar_scraper, duka_spread + duka_spread_by_hour.csv (the Dukascopy spread study), feed_divergence_oanda_vs_tiingo.csv, the panel dry-test JSONL, __init__.py - scripts/feed_divergence_study.py, scripts/signal_parity_study.py (completed studies; their reports keep the results) - docs/architecture.png + architecture_diagram.py (developer illustration) - docs/data_source_change_test_plan.md (Phase 4 shadow scan stays noted in CLAUDE.md) Kept: the campaign .md reports in backtest/research/ as the decision ledger CLAUDE.md cites (rule 1 evidence); scripts/shadow_scan.py (pending operational step); backtest/diagnose_filters.py (test-locked backtester diagnostic). Local, untracked material removed on this box: .superpowers/, walkforward/, baseline/, the commit-ladder and LLM-eval result folders, the Dukascopy boundary CSV; the sizing_2026-09 folder is pruned to the four 3-year baseline books CLAUDE.md names. .gitignore no longer lists the removed research outputs; the backtest instructions and CLAUDE.md no longer point at deleted tools. Suite: 461 passed, 0 failed (nothing imported the removed files). Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
This commit is contained in:
Harwinder Sekhon
@@ -26,12 +26,7 @@ Thumbs.db
|
||||
# Backtesting data (keep scripts, ignore generated files)
|
||||
backtest/cache/
|
||||
backtest/*.csv
|
||||
backtest/research/*.parquet
|
||||
backtest/research/duka_m5_boundary_spreads.csv
|
||||
backtest/__pycache__/
|
||||
.env.*
|
||||
logs/
|
||||
backtest/research/commit_ladder_2026-08/
|
||||
backtest/research/llm_news_eval_2026-08/
|
||||
|
||||
backtest/research/sizing_2026-09/
|
||||
|
||||
@@ -37,7 +37,7 @@ python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-0
|
||||
|
||||
**`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. Signal-log fixtures: use a WEEKDAY signal bar when a test exercises candle continuity — the outcome model treats weekend and 21:00 UTC gaps as scheduled, so Saturday-dated bars never produce a gap. `_candles_json` stores `df.tail(14)`, so a test snapshot longer than 14 bars silently drops its head. When patching weekend logic in tests, patch `providers.base._is_forex_weekend` — but know there are THREE independent signal-path weekend implementations (`providers/base.py`, `signal_aggregator.py`, and an inline copy in `backtest/gpu_backtest.py:_is_xau_blocked_by_calendar`) plus the advisory-only continuity copy in `advisory_outcome.py`; patching one does not reach the others.
|
||||
|
||||
Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets). scikit-learn and lightgbm are installed in the dev `venv/` for `backtest/research/` only — never add them to `requirements.txt` or import them from live code; anything shipped must be numpy-only.
|
||||
Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets). scikit-learn and lightgbm are installed in the dev `venv/` for ad-hoc research only — never add them to `requirements.txt` or import them from live code; anything shipped must be numpy-only.
|
||||
|
||||
## Configuration & credentials
|
||||
|
||||
@@ -96,11 +96,11 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
|
||||
## Current state notes (dated per entry)
|
||||
|
||||
- Oanda v20 is the live feed (`data_providers.active: "oanda"`). The 2026-08-04 re-pin made the 27 `backtest/oanda_*.csv` books the baseline; all TD-era books are superseded. (2026-08-14) The re-pin M5 book `oanda_m5_12mo_065_nopb_20260814.csv` (+106.82R PF 1.51) is itself SUPERSEDED by the 2026-09-03 book in the entry below; the M15 reference is unchanged since 2026-08-04: `oanda_m15_12mo_065.csv` (397 / +73.93R / PF 1.53). See `backtest/research/repin_verification_2026-08-14.md`.
|
||||
- Cost reporting is dual: $0.65 is the measured-Oanda realistic headline ($0.65-$0.72 round trip), while $0.30 remains for cross-book comparability and as the live TP spread-gate input. Raising `round_trip_cost` from $0.30 would move the TP floor from about $9 to $19.50 and requires full rule-1 revalidation. The $0.65 figure comes from the 2026-07-24 Dukascopy tick study (`backtest/research/duka_spread.py` + `duka_spread_by_hour.csv`, kept as the evidence driver although nothing imports it).
|
||||
- Cost reporting is dual: $0.65 is the measured-Oanda realistic headline ($0.65-$0.72 round trip), while $0.30 remains for cross-book comparability and as the live TP spread-gate input. Raising `round_trip_cost` from $0.30 would move the TP floor from about $9 to $19.50 and requires full rule-1 revalidation. The $0.65 figure comes from the 2026-07-24 Dukascopy tick study (9M ticks over 12 months; its tooling was removed 2026-09-06 — the conclusion lives here and in memory).
|
||||
- Twelve Data is signal-incompatible with Oanda (Phase 1: about 19% M5 bar-direction disagreement and a systematic ~$1 high bias; see `backtest/research/feed_divergence_2026-08-14.md`). On an Oanda candle outage, stop scanning rather than switching to TD candles.
|
||||
- Tiingo is streaming-only (`data_providers.streaming: "tiingo"`), not a designated candle fallback. Oanda-Tiingo signal parity is documented in `backtest/research/signal_parity_2026-08-14.md`; Phase 4's two-week live shadow scan remains a manual operational step.
|
||||
- Tiingo is streaming-only (`data_providers.streaming: "tiingo"`), not a designated candle fallback. Oanda-Tiingo signal parity is documented in `backtest/research/signal_parity_2026-08-14.md`; Phase 4's two-week live shadow scan (`scripts/shadow_scan.py`) remains a manual operational step.
|
||||
- Human-facing docs in `docs/` (`user_guide.html`, `beginner_guide.html` + PDF builds) must be kept in sync when signal behavior changes. As of 2026-09-06 the PDF builds are BEHIND the HTML (violent-move watch entry added without a PDF rebuild).
|
||||
- 2026-09-03 signal-log campaign shipped two new per-TF gates (`backtest/research/signal_log_campaign_2026-09.md`): M5 `sell_blocked_regimes=["transitioning"]` and H1 `sell_min_quality=[]` (all H1 shorts blocked). Current-HEAD M5 canonical book is now `backtest/oanda_m5_12mo_065_nopb_20260903.csv` (557 / +112.21R / PF 1.56, halves +55.9/+56.4); H1 reference `backtest/oanda_h1_12mo_065_20260903.csv` (91 / +22.2R / PF 1.79; 5yr +49.1R PF 1.46; OOS +16.7R PF 1.74).
|
||||
- 2026-09-06 stake-sizing campaign (`backtest/research/sizing_campaign_2026-09.md`): a walk-forward "second opinion" stake multiplier (meta-labelling on 65 decision-time features, 3-year Oanda books at $0.65) FAILED the pre-registered rule on M5 (clean null) and M15 (lift inside the label-shuffle null, explained by BUY-vs-SELL). Nothing that changes numbers shipped. Behaviour-neutral tooling remains: `position_sizing.build_features` + `--export-features` (default export stays 63 columns), `SignalReport.penalty_tags/penalty_total/momentum_fading`, `TradeSetup.stake_multiplier` (None; reserved), `backtest/research/{book_report,burst_map,sizing_model}.py`. Stage-B facts: M5 stretch measures are POSITIVE in both halves (do not re-propose exhaustion sizing on M5); M15 exhaustion tag is weakly negative (≈ +3R/yr, noise); bursts cluster at 12–15 UTC and 01 UTC bar-close and do NOT hurt entries.
|
||||
- 2026-09-06 stake-sizing campaign (`backtest/research/sizing_campaign_2026-09.md`): a walk-forward "second opinion" stake multiplier (meta-labelling on 65 decision-time features, 3-year Oanda books at $0.65) FAILED the pre-registered rule on M5 (clean null) and M15 (lift inside the label-shuffle null, explained by BUY-vs-SELL). Nothing that changes numbers shipped. Behaviour-neutral tooling remains: `position_sizing.build_features` + `--export-features` (default export stays 63 columns), `SignalReport.penalty_tags/penalty_total/momentum_fading`, `TradeSetup.stake_multiplier` (None; reserved) (the campaign's research scripts were removed 2026-09-06; git history has them). Stage-B facts: M5 stretch measures are POSITIVE in both halves (do not re-propose exhaustion sizing on M5); M15 exhaustion tag is weakly negative (≈ +3R/yr, noise); bursts cluster at 12–15 UTC and 01 UTC bar-close and do NOT hurt entries.
|
||||
- Auto-execution (oanda_daemon.py / oanda_executor.py) was removed 2026-09-03 — the owner is not auto-trading; last commit containing it is 31002cb. Book-parity rules that lived in its docstring (no broker TP, 5-bar cooldown, once-per-bar trail) remain the backtester's model.
|
||||
- 2026-09-06 stale-code cleanup (behaviour-neutral; suite green, M5/H1 12-month books byte-identical after every signal-side commit): `main.py --timeframe` now accepts M5/M15/H1/H4 and takes its confirmation TFs from the pair factory (the CLI's private table had paired M5 with M1+M15); `PlatformConfig.pip_value` is gone (`PAIR_INFO` is the source); removed as dead or superseded: `_get_affected_pairs`/`event_impact_on_pairs`, `SignalLogger.advance_with_bars`/`path`/`news_for`, `reset_llm_client`, the Telegram scan-summary/keyboard helpers and duplicate confirmation map, the 2026-07 Oanda-vs-TD research pager+study, `burst_profile.py`, `hour_analysis.py`, `scripts/compare_oanda_tiingo.py`, the never-implemented LLM-provider plan and the merged config.json spec. `docs/architecture_diagram.py` only writes the PNG when run as a script.
|
||||
- 2026-09-06 stale-code cleanup (behaviour-neutral; suite green, M5/H1 12-month books byte-identical after every signal-side commit): `main.py --timeframe` now accepts M5/M15/H1/H4 and takes its confirmation TFs from the pair factory (the CLI's private table had paired M5 with M1+M15); `PlatformConfig.pip_value` is gone (`PAIR_INFO` is the source); removed as dead or superseded: `_get_affected_pairs`/`event_impact_on_pairs`, `SignalLogger.advance_with_bars`/`path`/`news_for`, `reset_llm_client`, the Telegram scan-summary/keyboard helpers and duplicate confirmation map, the 2026-07 Oanda-vs-TD research pager+study, `burst_profile.py`, `hour_analysis.py`, `scripts/compare_oanda_tiingo.py`, the never-implemented LLM-provider plan and the merged config.json spec. Second pass the same day (owner directive: keep only what the system needs): every research script and data file under `backtest/research/` (book_report, book_diff, the burst tools, sizing_model, commit_ladder, llm_news_eval + golden CSV, ff_calendar_scraper, calendar_impact, watch_rule_eval, the Dukascopy spread study + CSVs, the panel dry-test JSONL), the two completed study scripts in `scripts/`, the architecture diagram + generator and the data-source test plan were deleted; the campaign `.md` reports stay as the decision ledger. Git history before `953c5ad` has all of it.
|
||||
|
||||
@@ -9,21 +9,18 @@ Every backtest report must record:
|
||||
- Per trade (the exported CSV already carries these): entry/exit time and
|
||||
price, exit reason, R multiple, MFE/MAE, entry reasons, and every
|
||||
indicator's signal / value / agreement at decision time.
|
||||
- Breakdowns by session/hour, weekday, regime and volatility regime
|
||||
(backtest/research/book_report.py) and, for any candidate change, the
|
||||
removed/added/shifted trade diff against the control book
|
||||
(backtest/research/book_diff.py).
|
||||
- Breakdowns by session/hour, weekday, regime and volatility regime, and,
|
||||
for any candidate change, the removed/added/shifted trade diff against the
|
||||
control book (the 5-bar cooldown displaces entries, so judge run-level
|
||||
numbers, never the added-trade bucket alone).
|
||||
- The md5 of the exported book whenever the run is a parity or regression check.
|
||||
Replays go through the real signal code (analyze_from_dataframes) — never a
|
||||
re-implementation.
|
||||
|
||||
|
||||
--- 2026-09-06: decision-time feature export ---
|
||||
- `--export-features` appends stake_multiplier, r_sized and the 65 `f_*` decision-time
|
||||
features (position_sizing.FEATURE_ORDER) AFTER the 63 pinned columns; default export is
|
||||
byte-identical to the pinned books. Use backtest/research/book_report.py for weekday /
|
||||
hour x weekday / session / streak / rolling-WR / feature-quintile (both halves) tables,
|
||||
backtest/research/burst_map.py for the burst-by-hour map, and
|
||||
backtest/research/sizing_model.py for the walk-forward + shuffle-control harness.
|
||||
byte-identical to the pinned books.
|
||||
- Any sizing report must show the FLAT book and the SIZED book side by side and the
|
||||
label-shuffle null; the 2026-09-06 campaign verdict (nothing ships) is the reference.
|
||||
label-shuffle null; the 2026-09-06 campaign verdict (nothing ships,
|
||||
backtest/research/sizing_campaign_2026-09.md) is the reference.
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
"""Research tooling: calendar sourcing/store and edge-screening utilities.
|
||||
|
||||
Nothing here is imported by the live bot; these modules support
|
||||
backtesting and edge validation only.
|
||||
"""
|
||||
@@ -1,86 +0,0 @@
|
||||
"""Trade-book diff: relaxed/gated run vs its control.
|
||||
|
||||
Usage:
|
||||
python backtest/research/book_diff.py CONTROL.csv CANDIDATE.csv BAR_MINUTES
|
||||
|
||||
Decision numbers are RUN-LEVEL (ΔtotalR, ΔPF, ΔmaxDD, Δtrades) — never
|
||||
the added-trade bucket alone: the 5-bar entry cooldown means new
|
||||
entries displace existing ones. The added/removed decomposition and the
|
||||
per-decision-hour histogram of added trades are diagnostics.
|
||||
|
||||
Shifted trades = removed+added pairs in the same direction within
|
||||
±5 bars (cooldown displacement, not a genuinely new/lost signal).
|
||||
"""
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def _metrics(df):
|
||||
r = df["r_multiple"]
|
||||
gains, losses = r[r > 0].sum(), -r[r <= 0].sum()
|
||||
cum = r.cumsum()
|
||||
return dict(trades=len(df), total_r=r.sum(),
|
||||
pf=gains / max(losses, 1e-9),
|
||||
wr=(r > 0).mean() * 100 if len(df) else 0.0,
|
||||
max_dd=(cum.cummax() - cum).max() if len(cum) else 0.0)
|
||||
|
||||
|
||||
def diff_books(control_csv, candidate_csv, bar_minutes, split="2026-01-29"):
|
||||
ctrl, cand = pd.read_csv(control_csv), pd.read_csv(candidate_csv)
|
||||
for df in (ctrl, cand):
|
||||
df["_t"] = pd.to_datetime(df["entry_time"], utc=True, format="mixed")
|
||||
df["_key"] = df["entry_time"].astype(str) + "|" + df["direction"]
|
||||
|
||||
added = cand[~cand["_key"].isin(ctrl["_key"])].copy()
|
||||
removed = ctrl[~ctrl["_key"].isin(cand["_key"])].copy()
|
||||
|
||||
# cooldown displacement: same-direction removed/added within ±5 bars
|
||||
tol = pd.Timedelta(minutes=5 * bar_minutes)
|
||||
shifted = 0
|
||||
rem_pool = removed.copy()
|
||||
for _, a in added.iterrows():
|
||||
m = rem_pool[(rem_pool["direction"] == a["direction"])
|
||||
& ((rem_pool["_t"] - a["_t"]).abs() <= tol)]
|
||||
if len(m):
|
||||
shifted += 1
|
||||
rem_pool = rem_pool.drop(m.index[0])
|
||||
|
||||
mc, mk = _metrics(ctrl), _metrics(cand)
|
||||
split_ts = pd.Timestamp(split, tz="UTC")
|
||||
halves = (cand[cand["_t"] < split_ts]["r_multiple"].sum(),
|
||||
cand[cand["_t"] >= split_ts]["r_multiple"].sum())
|
||||
hour_hist = None
|
||||
if len(added):
|
||||
added["dec_hour"] = (added["_t"]
|
||||
- pd.Timedelta(minutes=bar_minutes)).dt.hour
|
||||
hour_hist = added.groupby("dec_hour")["r_multiple"] \
|
||||
.agg(["count", "sum"]).round(2)
|
||||
return dict(
|
||||
control=mc, candidate=mk,
|
||||
delta_r=mk["total_r"] - mc["total_r"],
|
||||
delta_pf=mk["pf"] - mc["pf"],
|
||||
delta_dd=mk["max_dd"] - mc["max_dd"],
|
||||
delta_trades=mk["trades"] - mc["trades"],
|
||||
added_n=len(added), added_r=added["r_multiple"].sum(),
|
||||
removed_n=len(removed), removed_r=removed["r_multiple"].sum(),
|
||||
shifted=shifted, halves=halves, added_by_hour=hour_hist,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
d = diff_books(sys.argv[1], sys.argv[2], int(sys.argv[3]))
|
||||
c, k = d["control"], d["candidate"]
|
||||
print(f"control : {c['trades']:4d} trades {c['total_r']:+7.1f}R "
|
||||
f"PF {c['pf']:.2f} WR {c['wr']:.1f}% DD {c['max_dd']:.1f}")
|
||||
print(f"candidate: {k['trades']:4d} trades {k['total_r']:+7.1f}R "
|
||||
f"PF {k['pf']:.2f} WR {k['wr']:.1f}% DD {k['max_dd']:.1f}")
|
||||
print(f"ΔR {d['delta_r']:+.1f} ΔPF {d['delta_pf']:+.2f} "
|
||||
f"ΔDD {d['delta_dd']:+.1f} Δtrades {d['delta_trades']:+d}")
|
||||
print(f"added {d['added_n']} ({d['added_r']:+.1f}R) "
|
||||
f"removed {d['removed_n']} ({d['removed_r']:+.1f}R) "
|
||||
f"shifted≈{d['shifted']}")
|
||||
print(f"candidate halves: H1 {d['halves'][0]:+.1f}R H2 {d['halves'][1]:+.1f}R")
|
||||
if d["added_by_hour"] is not None:
|
||||
print("\nadded trades by decision hour:")
|
||||
print(d["added_by_hour"].to_string())
|
||||
@@ -1,581 +0,0 @@
|
||||
"""Full descriptive report of a backtest trade book (research only).
|
||||
|
||||
Usage:
|
||||
python backtest/research/book_report.py --book BOOK.csv --bar-minutes N \
|
||||
[--split YYYY-MM-DD|median] [--parquet-4h PATH] [--parquet-1d PATH] \
|
||||
[--out DIR]
|
||||
|
||||
Prints (and writes to DIR/report.txt) headline metrics for the whole book
|
||||
and for each half, group tables (weekday / decision hour / session / quality
|
||||
/ regime / vol_regime / direction / strategy / exit_reason / indicators_agree),
|
||||
win-loss streaks, a rolling win-rate summary, and a quantile-bucket screen of
|
||||
every numeric decision-time feature the book carries (`f_*` / `x_*` columns
|
||||
plus score, confidence, rr_planned, indicators_agree/oppose and `ind_*_value`).
|
||||
|
||||
The screen is deliberately half-split: a feature only "passes" when the
|
||||
top-bucket-minus-bottom-bucket mean-R gap has the SAME SIGN in both halves of
|
||||
the book and both extreme buckets are populated in both halves. Bin edges are
|
||||
fit on the WHOLE book (so the halves share edges and stay comparable) — that
|
||||
leaks a little cross-half information into the edges themselves, which is why
|
||||
the sign test, not the size of the gap, is the thing to trust.
|
||||
|
||||
Everything here is measurement, not simulation: it never re-runs the signal
|
||||
engine, it only describes a book that gpu_backtest.py already produced.
|
||||
Analysis tooling only — never imported by the live code.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import math
|
||||
import os
|
||||
from typing import Iterable
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
SESSIONS = (
|
||||
("asian", 3, 6),
|
||||
("london", 7, 12),
|
||||
("overlap", 13, 16),
|
||||
("ny", 17, 22),
|
||||
)
|
||||
|
||||
# Numeric non-`f_`/`x_` columns that are known at DECISION time and are
|
||||
# therefore legal screen candidates. bars_held / r_multiple / mfe_r / mae_r
|
||||
# are post-entry outcomes and must never appear here.
|
||||
BASE_SCREEN_COLS = ("score", "confidence", "rr_planned",
|
||||
"indicators_agree", "indicators_oppose")
|
||||
|
||||
|
||||
def session_of(hour: int) -> str:
|
||||
for name, lo, hi in SESSIONS:
|
||||
if lo <= hour <= hi:
|
||||
return name
|
||||
return "off"
|
||||
|
||||
|
||||
def feature_columns(df: pd.DataFrame) -> list[str]:
|
||||
"""Decision-time feature columns carried by this book, generically."""
|
||||
extra = [c for c in df.columns if c.startswith(("f_", "x_"))]
|
||||
base = [c for c in BASE_SCREEN_COLS if c in df.columns]
|
||||
ind = [c for c in df.columns
|
||||
if c.startswith("ind_") and c.endswith("_value")]
|
||||
out, seen = [], set()
|
||||
for c in extra + base + ind:
|
||||
if c not in seen:
|
||||
seen.add(c)
|
||||
out.append(c)
|
||||
return out
|
||||
|
||||
|
||||
# ───────────────────────────── loading ──────────────────────────────
|
||||
|
||||
def load_book(csv: str, bar_minutes: int = 5,
|
||||
split: str = "median") -> pd.DataFrame:
|
||||
"""Read a gpu_backtest trade book and add the decision-time columns.
|
||||
|
||||
`decision_time` = entry_time − one bar: entries fill at the NEXT bar's
|
||||
open, so the bar the signal actually decided on closed one bar earlier.
|
||||
That is the clock blocked_hours_utc and every session filter evaluate.
|
||||
`half` is 1 before the split instant and 2 at/after it.
|
||||
"""
|
||||
df = pd.read_csv(csv)
|
||||
for col in ("entry_time", "exit_time"):
|
||||
if col in df.columns:
|
||||
df[col] = pd.to_datetime(df[col], utc=True, format="mixed")
|
||||
df["decision_time"] = df["entry_time"] - pd.Timedelta(minutes=bar_minutes)
|
||||
df["decision_hour"] = df["decision_time"].dt.hour
|
||||
df["weekday"] = df["decision_time"].dt.weekday
|
||||
df["session"] = df["decision_hour"].map(session_of)
|
||||
df = df.sort_values("entry_time", kind="mergesort").reset_index(drop=True)
|
||||
df["half"] = _halves(df["entry_time"], split)
|
||||
return df
|
||||
|
||||
|
||||
def split_instant(entry_time: pd.Series, split: str) -> pd.Timestamp:
|
||||
if split == "median":
|
||||
return entry_time.median()
|
||||
return pd.Timestamp(split, tz="UTC")
|
||||
|
||||
|
||||
def _halves(entry_time: pd.Series, split: str) -> pd.Series:
|
||||
cut = split_instant(entry_time, split)
|
||||
return np.where(entry_time < cut, 1, 2)
|
||||
|
||||
|
||||
# ───────────────────────────── metrics ──────────────────────────────
|
||||
|
||||
def base_metrics(r: pd.Series, df: pd.DataFrame | None = None) -> dict:
|
||||
"""Headline stats for a series of R multiples.
|
||||
|
||||
max_dd uses the gpu_backtest.compute_metrics convention: a 0.0 is
|
||||
prepended to the cumulative-R curve so an opening losing streak counts
|
||||
as drawdown from flat, not from an already-negative first trade.
|
||||
pf/expectancy split on r > 0 / r < 0 (exact zeros are neither).
|
||||
"""
|
||||
r = pd.Series(r, dtype="float64").dropna()
|
||||
n = int(len(r))
|
||||
if n == 0:
|
||||
return {"n": 0, "total_r": 0.0, "mean_r": 0.0, "median_r": 0.0,
|
||||
"wr": 0.0, "pf": 0.0, "max_dd": 0.0, "sharpe": 0.0,
|
||||
"expectancy": 0.0, "expectancy_alt": 0.0, "recovery": 0.0,
|
||||
"avg_bars": float("nan"), "avg_mfe": float("nan"),
|
||||
"avg_mae": float("nan")}
|
||||
wins, losses = r[r > 0], r[r < 0]
|
||||
gross_win = float(wins.sum())
|
||||
gross_loss = float(-losses.sum())
|
||||
cum = np.concatenate(([0.0], np.cumsum(r.to_numpy())))
|
||||
max_dd = float((np.maximum.accumulate(cum) - cum).max())
|
||||
std = float(r.std(ddof=0))
|
||||
mean_r = float(r.mean())
|
||||
wr = len(wins) / n
|
||||
lr = len(losses) / n
|
||||
avg_win = float(wins.mean()) if len(wins) else 0.0
|
||||
avg_loss = float(-losses.mean()) if len(losses) else 0.0
|
||||
expectancy_alt = wr * avg_win - lr * avg_loss
|
||||
# Identity check: the textbook expectancy form must equal the plain mean
|
||||
# whenever no trade is exactly flat (and it does even then, since a 0 R
|
||||
# trade contributes 0 to both sides).
|
||||
assert math.isclose(expectancy_alt, mean_r, rel_tol=1e-9, abs_tol=1e-9), (
|
||||
f"expectancy mismatch: {expectancy_alt} vs mean {mean_r}")
|
||||
out = {
|
||||
"n": n,
|
||||
"total_r": float(r.sum()),
|
||||
"mean_r": mean_r,
|
||||
"median_r": float(r.median()),
|
||||
"wr": wr * 100.0,
|
||||
"pf": (gross_win / gross_loss) if gross_loss > 0 else float("inf"),
|
||||
"max_dd": max_dd,
|
||||
"sharpe": (mean_r / std) if std > 0 else 0.0,
|
||||
"expectancy": mean_r,
|
||||
"expectancy_alt": expectancy_alt,
|
||||
"recovery": (float(r.sum()) / max_dd) if max_dd > 0 else float("inf"),
|
||||
"avg_bars": float("nan"),
|
||||
"avg_mfe": float("nan"),
|
||||
"avg_mae": float("nan"),
|
||||
}
|
||||
if df is not None:
|
||||
for key, col in (("avg_bars", "bars_held"), ("avg_mfe", "mfe_r"),
|
||||
("avg_mae", "mae_r")):
|
||||
if col in df.columns:
|
||||
out[key] = float(pd.to_numeric(df[col],
|
||||
errors="coerce").mean())
|
||||
return out
|
||||
|
||||
|
||||
def _pf(r: pd.Series) -> float:
|
||||
"""Profit factor (loss side is r <= 0)."""
|
||||
pos = r[r > 0].sum()
|
||||
neg = -r[r <= 0].sum()
|
||||
return float(pos / max(neg, 1e-9))
|
||||
|
||||
|
||||
def group_table(df: pd.DataFrame, by, min_n: int = 10) -> pd.DataFrame:
|
||||
"""Per-group P&L with a half-1 / half-2 sign-stability flag."""
|
||||
keys = [by] if isinstance(by, str) else list(by)
|
||||
rows = []
|
||||
for key, sub in df.groupby(keys, dropna=False, sort=True):
|
||||
r = sub["r_multiple"]
|
||||
h1, h2 = sub[sub["half"] == 1]["r_multiple"], sub[sub["half"] == 2]["r_multiple"]
|
||||
m1 = float(h1.mean()) if len(h1) else float("nan")
|
||||
m2 = float(h2.mean()) if len(h2) else float("nan")
|
||||
stable = bool(len(h1) >= min_n and len(h2) >= min_n
|
||||
and np.sign(m1) != 0 and np.sign(m1) == np.sign(m2))
|
||||
rec = {}
|
||||
# pandas >= 2.2 hands back a 1-tuple even for a single group key.
|
||||
vals = key if isinstance(key, tuple) else (key,)
|
||||
for k, v in zip(keys, vals):
|
||||
rec[k] = v
|
||||
rec.update({
|
||||
"n": int(len(sub)),
|
||||
"total_r": round(float(r.sum()), 2),
|
||||
"mean_r": round(float(r.mean()), 3),
|
||||
"wr": round(float((r > 0).mean() * 100), 2),
|
||||
"pf": round(_pf(r), 2),
|
||||
"n_h1": int(len(h1)),
|
||||
"total_r_h1": round(float(h1.sum()), 2),
|
||||
"mean_r_h1": round(m1, 3) if len(h1) else float("nan"),
|
||||
"n_h2": int(len(h2)),
|
||||
"total_r_h2": round(float(h2.sum()), 2),
|
||||
"mean_r_h2": round(m2, 3) if len(h2) else float("nan"),
|
||||
"sign_stable": stable,
|
||||
})
|
||||
rows.append(rec)
|
||||
out = pd.DataFrame(rows)
|
||||
if not out.empty:
|
||||
out = out.set_index(keys)
|
||||
return out
|
||||
|
||||
|
||||
def streaks(df: pd.DataFrame) -> dict:
|
||||
"""Longest win/loss runs and the longest stretch under water (in trades)."""
|
||||
d = df.sort_values("entry_time", kind="mergesort")
|
||||
r = d["r_multiple"].to_numpy(dtype="float64")
|
||||
best_w = best_l = cur_w = cur_l = 0
|
||||
for x in r:
|
||||
if x > 0:
|
||||
cur_w, cur_l = cur_w + 1, 0
|
||||
elif x < 0:
|
||||
cur_l, cur_w = cur_l + 1, 0
|
||||
else:
|
||||
cur_w = cur_l = 0
|
||||
best_w, best_l = max(best_w, cur_w), max(best_l, cur_l)
|
||||
cum = np.concatenate(([0.0], np.cumsum(r)))
|
||||
peak = np.maximum.accumulate(cum)
|
||||
flat, longest = 0, 0
|
||||
for i in range(1, len(cum)):
|
||||
flat = 0 if cum[i] >= peak[i] else flat + 1
|
||||
longest = max(longest, flat)
|
||||
return {"max_win_streak": best_w, "max_loss_streak": best_l,
|
||||
"longest_flat_trades": longest}
|
||||
|
||||
|
||||
def rolling_wr(df: pd.DataFrame, window: int = 20) -> pd.Series:
|
||||
d = df.sort_values("entry_time", kind="mergesort")
|
||||
s = (d["r_multiple"] > 0).astype(float).rolling(window).mean() * 100
|
||||
s.index = d["entry_time"].to_numpy()
|
||||
return s
|
||||
|
||||
|
||||
def equity_curve(df: pd.DataFrame) -> pd.DataFrame:
|
||||
d = df.sort_values("entry_time", kind="mergesort")
|
||||
cum = d["r_multiple"].cumsum()
|
||||
peak = pd.concat([pd.Series([0.0]), cum]).cummax().iloc[1:]
|
||||
peak.index = cum.index
|
||||
return pd.DataFrame({"entry_time": d["entry_time"].to_numpy(),
|
||||
"cum_r": cum.to_numpy(),
|
||||
"drawdown": (peak - cum).to_numpy()})
|
||||
|
||||
|
||||
# ─────────────────────── quantile bucket screen ─────────────────────
|
||||
|
||||
def quantile_buckets(df: pd.DataFrame, col: str, q: int = 5,
|
||||
min_n: int = 10) -> pd.DataFrame:
|
||||
"""Equal-frequency buckets of `col` with per-half mean R.
|
||||
|
||||
Bin edges are fit on the WHOLE book (both halves share edges).
|
||||
Result carries gap_h1 / gap_h2 / sign_stable in `.attrs` and as
|
||||
repeated columns so the CSV is self-describing.
|
||||
"""
|
||||
v = pd.to_numeric(df.get(col), errors="coerce")
|
||||
d = df.loc[v.notna()].copy()
|
||||
d["_v"] = v.loc[v.notna()]
|
||||
empty = pd.DataFrame(columns=["bin", "lo", "hi", "n", "mean_r", "pf", "wr",
|
||||
"n_h1", "mean_r_h1", "n_h2", "mean_r_h2",
|
||||
"gap_h1", "gap_h2", "sign_stable"])
|
||||
empty.attrs = {"feature": col, "gap_h1": float("nan"),
|
||||
"gap_h2": float("nan"), "sign_stable": False, "n": 0,
|
||||
"mode": "empty"}
|
||||
if d.empty:
|
||||
return empty
|
||||
|
||||
nuniq = int(d["_v"].nunique())
|
||||
if nuniq < 3:
|
||||
mode = "values"
|
||||
d["_bin"] = d["_v"]
|
||||
order = sorted(d["_v"].unique())
|
||||
edges = {b: (float(b), float(b)) for b in order}
|
||||
else:
|
||||
mode = "qcut"
|
||||
binned = pd.qcut(d["_v"], q, duplicates="drop")
|
||||
d["_bin"] = binned
|
||||
order = list(binned.cat.categories)
|
||||
edges = {b: (float(b.left), float(b.right)) for b in order}
|
||||
|
||||
rows = []
|
||||
for b in order:
|
||||
sub = d[d["_bin"] == b]
|
||||
if sub.empty:
|
||||
continue
|
||||
r = sub["r_multiple"]
|
||||
h1 = sub[sub["half"] == 1]["r_multiple"]
|
||||
h2 = sub[sub["half"] == 2]["r_multiple"]
|
||||
lo, hi = edges[b]
|
||||
rows.append({
|
||||
"bin": str(b), "lo": round(lo, 4), "hi": round(hi, 4),
|
||||
"n": int(len(sub)), "mean_r": round(float(r.mean()), 3),
|
||||
"pf": round(_pf(r), 2),
|
||||
"wr": round(float((r > 0).mean() * 100), 2),
|
||||
"n_h1": int(len(h1)),
|
||||
"mean_r_h1": round(float(h1.mean()), 3) if len(h1) else float("nan"),
|
||||
"n_h2": int(len(h2)),
|
||||
"mean_r_h2": round(float(h2.mean()), 3) if len(h2) else float("nan"),
|
||||
})
|
||||
out = pd.DataFrame(rows)
|
||||
if out.empty:
|
||||
return empty
|
||||
|
||||
bot, top = out.iloc[0], out.iloc[-1]
|
||||
gap_h1 = float(top["mean_r_h1"] - bot["mean_r_h1"])
|
||||
gap_h2 = float(top["mean_r_h2"] - bot["mean_r_h2"])
|
||||
populated = min(bot["n_h1"], bot["n_h2"], top["n_h1"], top["n_h2"]) >= min_n
|
||||
signs_ok = (not math.isnan(gap_h1) and not math.isnan(gap_h2)
|
||||
and np.sign(gap_h1) != 0
|
||||
and np.sign(gap_h1) == np.sign(gap_h2))
|
||||
sign_stable = bool(populated and signs_ok and len(out) >= 2)
|
||||
out["gap_h1"] = round(gap_h1, 3) if not math.isnan(gap_h1) else float("nan")
|
||||
out["gap_h2"] = round(gap_h2, 3) if not math.isnan(gap_h2) else float("nan")
|
||||
out["sign_stable"] = sign_stable
|
||||
out.attrs = {"feature": col, "gap_h1": gap_h1, "gap_h2": gap_h2,
|
||||
"sign_stable": sign_stable, "n": int(len(d)), "mode": mode}
|
||||
return out
|
||||
|
||||
|
||||
def feature_screen(df: pd.DataFrame, q: int = 5, min_n: int = 10,
|
||||
columns: Iterable[str] | None = None) -> pd.DataFrame:
|
||||
"""Bucket screen over every decision-time numeric feature in the book."""
|
||||
cols = list(columns) if columns is not None else feature_columns(df)
|
||||
rows = []
|
||||
for c in cols:
|
||||
bk = quantile_buckets(df, c, q=q, min_n=min_n)
|
||||
a = bk.attrs
|
||||
stable = bool(a.get("sign_stable"))
|
||||
g1, g2 = a.get("gap_h1", float("nan")), a.get("gap_h2", float("nan"))
|
||||
min_abs = (min(abs(g1), abs(g2)) if stable else 0.0)
|
||||
rows.append({"feature": c,
|
||||
"gap_h1": round(g1, 3) if g1 == g1 else float("nan"),
|
||||
"gap_h2": round(g2, 3) if g2 == g2 else float("nan"),
|
||||
"min_abs_gap": round(float(min_abs), 3),
|
||||
"sign_stable": stable,
|
||||
"n": int(a.get("n", 0)),
|
||||
"bins": int(len(bk))})
|
||||
out = pd.DataFrame(rows)
|
||||
if out.empty:
|
||||
return out
|
||||
return out.sort_values(["min_abs_gap", "n"],
|
||||
ascending=[False, False]).reset_index(drop=True)
|
||||
|
||||
|
||||
# ────────────────────── higher-timeframe extension ──────────────────
|
||||
|
||||
def wilder_atr(bars: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
h, l, c = bars["High"], bars["Low"], bars["Close"]
|
||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||
axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1 / period, min_periods=period, adjust=False).mean()
|
||||
|
||||
|
||||
def _ext_series(parquet: str, minutes: int) -> pd.DataFrame:
|
||||
bars = pd.read_parquet(parquet).sort_index()
|
||||
bars = bars[~bars.index.duplicated(keep="last")]
|
||||
ext = ((bars["Close"] - bars["Close"].rolling(20).mean())
|
||||
/ wilder_atr(bars, 14))
|
||||
out = pd.DataFrame({
|
||||
"close_time": bars.index + pd.Timedelta(minutes=minutes),
|
||||
"ext": ext.to_numpy(),
|
||||
}).dropna().reset_index(drop=True)
|
||||
# Books parse to datetime64[us, UTC] (format="mixed"), parquet indexes are
|
||||
# [ns, UTC]; merge_asof refuses mismatched resolutions.
|
||||
out["close_time"] = out["close_time"].astype("datetime64[ns, UTC]")
|
||||
return out
|
||||
|
||||
|
||||
def add_htf_extension(df: pd.DataFrame, parquet_4h: str | None,
|
||||
parquet_1d: str | None) -> pd.DataFrame:
|
||||
"""Add direction-signed H4 / D1 stretch-from-mean in ATR units.
|
||||
|
||||
ext = (close − 20-bar SMA of close) / Wilder ATR(14) on that timeframe,
|
||||
taken from the LAST higher-TF bar whose CLOSE time (bar label + one bar)
|
||||
is <= decision_time — i.e. only information the live scan could have had.
|
||||
Signed +1 for BUY, −1 for SELL, so positive always means "extended in the
|
||||
direction of the trade".
|
||||
"""
|
||||
out = df.copy()
|
||||
sign = np.where(out["direction"].astype(str).str.upper().str.startswith("B"),
|
||||
1.0, -1.0)
|
||||
for col, parquet, minutes in (("x_h4_ext_atr", parquet_4h, 240),
|
||||
("x_d1_ext_atr", parquet_1d, 1440)):
|
||||
if not parquet:
|
||||
continue
|
||||
right = _ext_series(parquet, minutes)
|
||||
left = out[["decision_time"]].reset_index()
|
||||
left["decision_time"] = left["decision_time"].astype(
|
||||
"datetime64[ns, UTC]")
|
||||
left = left.sort_values("decision_time", kind="mergesort")
|
||||
merged = pd.merge_asof(left, right, left_on="decision_time",
|
||||
right_on="close_time", direction="backward")
|
||||
merged["_stale_h"] = ((merged["decision_time"] - merged["close_time"])
|
||||
.dt.total_seconds() / 3600.0)
|
||||
merged = merged.set_index("index").reindex(out.index)
|
||||
out[col] = merged["ext"].to_numpy() * sign
|
||||
out.attrs[f"{col}_max_stale_h"] = float(
|
||||
merged["_stale_h"].max()) if len(merged) else float("nan")
|
||||
return out
|
||||
|
||||
|
||||
# ──────────────────────────── reporting ─────────────────────────────
|
||||
|
||||
def _fmt(v, nd=3):
|
||||
if isinstance(v, float):
|
||||
if math.isinf(v):
|
||||
return "inf"
|
||||
if math.isnan(v):
|
||||
return "n/a"
|
||||
return f"{v:.{nd}f}"
|
||||
return str(v)
|
||||
|
||||
|
||||
def _metrics_block(name: str, m: dict) -> list[str]:
|
||||
return [
|
||||
f" {name:<10} n={m['n']:<5} totalR={_fmt(m['total_r'], 2):>8} "
|
||||
f"meanR={_fmt(m['mean_r'])} medR={_fmt(m['median_r'])} "
|
||||
f"WR={_fmt(m['wr'], 2)}% PF={_fmt(m['pf'], 2)}",
|
||||
f" {'':<10} maxDD={_fmt(m['max_dd'], 2)} recovery={_fmt(m['recovery'], 2)}"
|
||||
f" sharpe={_fmt(m['sharpe'])} expectancy={_fmt(m['expectancy'])}"
|
||||
f" (wr·avgW − lr·avgL = {_fmt(m['expectancy_alt'])})",
|
||||
f" {'':<10} avg bars={_fmt(m['avg_bars'], 1)} "
|
||||
f"avg MFE={_fmt(m['avg_mfe'])}R avg MAE={_fmt(m['avg_mae'])}R",
|
||||
]
|
||||
|
||||
|
||||
def build_report(df: pd.DataFrame, book: str, bar_minutes: int, split: str,
|
||||
out_dir: str, q: int = 5, min_n: int = 10) -> str:
|
||||
lines: list[str] = []
|
||||
add = lines.append
|
||||
cut = split_instant(df["entry_time"], split)
|
||||
add("=" * 78)
|
||||
add(f"BOOK REPORT {os.path.basename(book)}")
|
||||
add("=" * 78)
|
||||
add(f"trades : {len(df)}")
|
||||
add(f"entry window : {df['entry_time'].min()} -> {df['entry_time'].max()}")
|
||||
add(f"bar minutes : {bar_minutes} (decision_time = entry_time − 1 bar)")
|
||||
add(f"split rule : {split} -> cut at {cut} "
|
||||
f"(half 1 = before, half 2 = at/after)")
|
||||
add(f"half sizes : h1={int((df['half'] == 1).sum())} "
|
||||
f"h2={int((df['half'] == 2).sum())}")
|
||||
add("session map : asian 3-6, london 7-12, overlap 13-16, ny 17-22, "
|
||||
"else off (on DECISION hour, UTC)")
|
||||
add("bucket edges : quantile bins are fit on the WHOLE book; the two "
|
||||
"halves share the same edges.")
|
||||
feats = feature_columns(df)
|
||||
add(f"screen features : {len(feats)} "
|
||||
f"({sum(1 for c in feats if c.startswith(('f_', 'x_')))} f_/x_ columns)")
|
||||
for k, v in df.attrs.items():
|
||||
add(f"htf staleness : {k} = {_fmt(v, 1)} h (max age of the matched bar)")
|
||||
|
||||
add("")
|
||||
add("── HEADLINE METRICS ──")
|
||||
lines.extend(_metrics_block("full", base_metrics(df["r_multiple"], df)))
|
||||
for h in (1, 2):
|
||||
sub = df[df["half"] == h]
|
||||
lines.extend(_metrics_block(f"half {h}",
|
||||
base_metrics(sub["r_multiple"], sub)))
|
||||
|
||||
group_specs = [
|
||||
("weekday", "weekday"), ("decision_hour", "decision_hour"),
|
||||
("session", "session"), ("quality", "quality"), ("regime", "regime"),
|
||||
("vol_regime", "vol_regime"), ("direction", "direction"),
|
||||
("strategy", "strategy"), ("exit_reason", "exit_reason"),
|
||||
("indicators_agree", "indicators_agree"),
|
||||
]
|
||||
tables = {}
|
||||
for name, col in group_specs:
|
||||
if col not in df.columns:
|
||||
continue
|
||||
t = group_table(df, col, min_n=min_n)
|
||||
tables[name] = t
|
||||
add("")
|
||||
add(f"── BY {name.upper()} ──")
|
||||
add(t.to_string() if not t.empty else " (empty)")
|
||||
|
||||
# (weekday, decision_hour) pivots
|
||||
add("")
|
||||
add("── weekday x decision_hour : TOTAL R ──")
|
||||
wh = group_table(df, ["weekday", "decision_hour"], min_n=min_n)
|
||||
if not wh.empty:
|
||||
piv_r = wh["total_r"].unstack("decision_hour")
|
||||
piv_n = wh["n"].unstack("decision_hour")
|
||||
add(piv_r.fillna(0).round(2).to_string())
|
||||
add("")
|
||||
add("── weekday x decision_hour : N ──")
|
||||
add(piv_n.fillna(0).astype(int).to_string())
|
||||
tables["weekday_hour"] = wh
|
||||
piv_r.to_csv(os.path.join(out_dir, "groups_weekday_hour_total_r.csv"))
|
||||
piv_n.to_csv(os.path.join(out_dir, "groups_weekday_hour_n.csv"))
|
||||
|
||||
st = streaks(df)
|
||||
add("")
|
||||
add("── STREAKS ──")
|
||||
add(f" max consecutive wins : {st['max_win_streak']}")
|
||||
add(f" max consecutive losses : {st['max_loss_streak']}")
|
||||
add(f" longest flat stretch : {st['longest_flat_trades']} trades "
|
||||
f"between equity highs")
|
||||
|
||||
rw = rolling_wr(df, 20).dropna()
|
||||
add("")
|
||||
add("── ROLLING WIN RATE (20 trades) ──")
|
||||
if len(rw):
|
||||
add(f" min={_fmt(rw.min(), 2)}% median={_fmt(rw.median(), 2)}% "
|
||||
f"max={_fmt(rw.max(), 2)}% n={len(rw)}")
|
||||
add(" last 5: " + ", ".join(
|
||||
f"{ts:%Y-%m-%d %H:%M}={v:.2f}%" for ts, v in rw.tail(5).items()))
|
||||
else:
|
||||
add(" (fewer than 20 trades)")
|
||||
|
||||
scr = feature_screen(df, q=q, min_n=min_n)
|
||||
add("")
|
||||
add("── FEATURE SCREEN (top 25 by min |gap| across halves) ──")
|
||||
add(" gap = mean R(top bucket) − mean R(bottom bucket), per half; "
|
||||
"min_abs_gap is 0 unless the sign is stable.")
|
||||
add(scr.head(25).to_string(index=False) if not scr.empty else " (none)")
|
||||
|
||||
stable = scr[scr["sign_stable"]].head(10) if not scr.empty else scr
|
||||
add("")
|
||||
add(f"── BUCKET TABLES FOR THE TOP {len(stable)} SIGN-STABLE FEATURES ──")
|
||||
if stable.empty:
|
||||
add(" (no feature has a sign-stable top-vs-bottom gap)")
|
||||
for feat in stable["feature"]:
|
||||
bk = quantile_buckets(df, feat, q=q, min_n=min_n)
|
||||
add("")
|
||||
add(f" · {feat} (mode={bk.attrs['mode']}, n={bk.attrs['n']}, "
|
||||
f"gap_h1={_fmt(bk.attrs['gap_h1'])}, "
|
||||
f"gap_h2={_fmt(bk.attrs['gap_h2'])})")
|
||||
add(bk.to_string(index=False))
|
||||
bk.to_csv(os.path.join(out_dir, f"buckets_{feat}.csv"), index=False)
|
||||
|
||||
# CSV artefacts
|
||||
for name, t in tables.items():
|
||||
t.to_csv(os.path.join(out_dir, f"groups_{name}.csv"))
|
||||
scr.to_csv(os.path.join(out_dir, "feature_screen.csv"), index=False)
|
||||
equity_curve(df).to_csv(os.path.join(out_dir, "equity.csv"), index=False)
|
||||
all_buckets = []
|
||||
for c in feats:
|
||||
bk = quantile_buckets(df, c, q=q, min_n=min_n)
|
||||
if bk.empty:
|
||||
continue
|
||||
bk = bk.copy()
|
||||
bk.insert(0, "feature", c)
|
||||
all_buckets.append(bk)
|
||||
if all_buckets:
|
||||
pd.concat(all_buckets, ignore_index=True).to_csv(
|
||||
os.path.join(out_dir, "buckets.csv"), index=False)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("--book", required=True)
|
||||
ap.add_argument("--bar-minutes", type=int, required=True)
|
||||
ap.add_argument("--split", default="median",
|
||||
help="'median' (default) or YYYY-MM-DD")
|
||||
ap.add_argument("--parquet-4h", default=None)
|
||||
ap.add_argument("--parquet-1d", default=None)
|
||||
ap.add_argument("--q", type=int, default=5)
|
||||
ap.add_argument("--min-n", type=int, default=10)
|
||||
ap.add_argument("--out", default="backtest/research/book_report_out")
|
||||
args = ap.parse_args()
|
||||
|
||||
os.makedirs(args.out, exist_ok=True)
|
||||
df = load_book(args.book, args.bar_minutes, args.split)
|
||||
if args.parquet_4h or args.parquet_1d:
|
||||
df = add_htf_extension(df, args.parquet_4h, args.parquet_1d)
|
||||
text = build_report(df, args.book, args.bar_minutes, args.split,
|
||||
args.out, q=args.q, min_n=args.min_n)
|
||||
print(text)
|
||||
with open(os.path.join(args.out, "report.txt"), "w") as fh:
|
||||
fh.write(text + "\n")
|
||||
print(f"\n[written] {os.path.join(args.out, 'report.txt')}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,121 +0,0 @@
|
||||
"""What do the system's OWN indicators read right before a violent bar? Analysis tooling only.
|
||||
|
||||
Uses backtest.gpu_backtest.precompute_indicators (the same series the backtester feeds the
|
||||
engine) on 3 years of bars; every state is taken at t-1 (known before the bar). Reports burst
|
||||
rate (range >= 3x prior ATR14) for the NEXT bar and within the NEXT HOUR by indicator state,
|
||||
by regime, by volatility regime, by weekday x hour, day of month, calendar proximity — each in
|
||||
both halves — and whether the state predicts the burst's DIRECTION.
|
||||
|
||||
python backtest/research/burst_indicator_map.py PARQUET BAR_MINUTES [--start --end]
|
||||
"""
|
||||
import argparse, sys
|
||||
from pathlib import Path
|
||||
import numpy as np, pandas as pd
|
||||
ROOT = Path(__file__).resolve().parents[2]; sys.path.insert(0, str(ROOT)); sys.path.insert(0, str(ROOT / "backtest"))
|
||||
from gpu_backtest import precompute_indicators
|
||||
from config import TechnicalConfig
|
||||
|
||||
CAL = ROOT / "backtest" / "cache" / "calendar.parquet"
|
||||
|
||||
|
||||
def wilder_atr(df, period=14):
|
||||
pc = df["Close"].shift(1)
|
||||
tr = pd.concat([df["High"] - df["Low"], (df["High"] - pc).abs(), (df["Low"] - pc).abs()], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1 / period, min_periods=period, adjust=False).mean()
|
||||
|
||||
|
||||
def rate_table(d, col, target, half, order=None):
|
||||
base1 = d.loc[half == 1, target].mean(); base2 = d.loc[half == 2, target].mean()
|
||||
g = d.groupby(col, observed=True)
|
||||
out = pd.DataFrame({"n": g.size(),
|
||||
"h1_lift": d[half == 1].groupby(col, observed=True)[target].mean() / base1,
|
||||
"h2_lift": d[half == 2].groupby(col, observed=True)[target].mean() / base2,
|
||||
"rate%": 100 * g[target].mean()})
|
||||
if order is not None: out = out.reindex([o for o in order if o in out.index])
|
||||
out["stable"] = np.where((out.h1_lift > 1.15) & (out.h2_lift > 1.15), "HIGH", np.where((out.h1_lift < 0.85) & (out.h2_lift < 0.85), "low", ""))
|
||||
return out.round(2)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(); ap.add_argument("parquet"); ap.add_argument("bar_minutes", type=int)
|
||||
ap.add_argument("--start", default="2023-09-05"); ap.add_argument("--end", default="2026-09-05")
|
||||
a = ap.parse_args(); bm = a.bar_minutes
|
||||
px = pd.read_parquet(a.parquet)
|
||||
px = px[(px.index >= pd.Timestamp(a.start, tz="UTC") - pd.Timedelta(days=30)) & (px.index < pd.Timestamp(a.end, tz="UTC"))]
|
||||
ic = precompute_indicators(px, TechnicalConfig())
|
||||
d = pd.DataFrame(index=px.index)
|
||||
atr_prev = wilder_atr(px).shift(1)
|
||||
rng = px["High"] - px["Low"]
|
||||
d["burst"] = (rng >= 3.0 * atr_prev).astype(int)
|
||||
H = max(1, 60 // bm)
|
||||
d["next_hour"] = d["burst"][::-1].rolling(H, min_periods=1).max()[::-1].astype(int)
|
||||
d["next_bar"] = d["burst"]
|
||||
d["bdir"] = np.sign(px["Close"] - px["Open"])
|
||||
# states at t-1
|
||||
sh = lambda arr: pd.Series(np.asarray(arr, dtype=float), index=px.index).shift(1)
|
||||
rsi = sh(ic.rsi); adx = sh(ic.adx); adxs = sh(ic.adx_slope); hist = sh(ic.macd_hist); bbp = sh(ic.bb_position)
|
||||
bbw = sh(ic.bb_band_width); bbaw = sh(ic.bb_avg_width); kcsq = sh(ic.kc_squeeze); roc = sh(ic.roc)
|
||||
ef, em, es = sh(ic.ema_fast), sh(ic.ema_medium), sh(ic.ema_slow); et = sh(ic.ema_trend) if ic.ema_trend is not None else None
|
||||
close1 = px["Close"].shift(1); atr1 = sh(ic.atr)
|
||||
kcu, kcl = sh(ic.kc_upper), sh(ic.kc_lower)
|
||||
d["rsi"] = pd.cut(rsi, [0, 30, 40, 50, 60, 70, 100], labels=["<30", "30-40", "40-50", "50-60", "60-70", ">=70"])
|
||||
d["adx"] = pd.cut(adx, [0, 20, 25, 35, 50, 200], labels=["<20", "20-25", "25-35", "35-50", ">=50"])
|
||||
d["adx_slope"] = pd.cut(adxs, [-1e9, -3, 0, 3, 1e9], labels=["falling<-3", "-3..0", "0..3", "rising>3"])
|
||||
hist_prev = hist.shift(1)
|
||||
d["macd"] = np.select([(hist > 0) & (hist > hist_prev), (hist > 0), (hist < 0) & (hist < hist_prev), (hist < 0)],
|
||||
["pos&rising", "pos&fading", "neg&falling", "neg&fading"], default="flat")
|
||||
d["bb_pos"] = pd.cut(bbp, [-1e9, 0, 0.2, 0.8, 1.0, 1e9], labels=["<0 (below)", "0-0.2", "0.2-0.8", "0.8-1", ">1 (above)"])
|
||||
d["bb_squeeze"] = np.where(bbw < 0.75 * bbaw, "squeeze", "normal")
|
||||
d["kc_squeeze"] = np.where(kcsq > 0, "TTM squeeze", "no")
|
||||
d["kc_pos"] = np.select([close1 > kcu, close1 < kcl], ["above KC", "below KC"], default="inside")
|
||||
align = np.sign(close1 - ef) + np.sign(ef - em) + np.sign(em - es) + (np.sign(es - et) if et is not None else 0)
|
||||
d["ema_align"] = pd.cut(align, [-5, -3.5, -0.5, 0.5, 3.5, 5], labels=["fully bear(-4)", "mostly bear", "mixed", "mostly bull", "fully bull(+4)"])
|
||||
ribbon = (pd.concat([ef, em, es], axis=1).max(axis=1) - pd.concat([ef, em, es], axis=1).min(axis=1)) / atr1
|
||||
d["ribbon"] = np.where(ribbon < 1.0, "compressed(<1 ATR)", "expanded")
|
||||
ema_dir = np.where(align > 0, 1, np.where(align < 0, -1, 0))
|
||||
d["regime"] = np.select([(adx >= 25) & (ema_dir > 0), (adx >= 25) & (ema_dir < 0), adx >= 25, adxs > 3.0],
|
||||
["trending_up", "trending_down", "trending(mixed)", "transitioning"], default="ranging")
|
||||
atr_pct = atr1.rolling(50).apply(lambda w: (w[-1] > w).mean() * 100, raw=True)
|
||||
d["vol_regime"] = pd.cut(atr_pct, [-1, 30, 60, 80, 101], labels=["low", "normal", "high", "extreme"])
|
||||
d["roc"] = pd.cut(roc, [-1e9, -1, -0.3, 0.3, 1, 1e9], labels=["<-1%", "-1..-0.3", "-0.3..0.3", "0.3..1", ">1%"])
|
||||
d["ema200_dist"] = pd.cut((close1 - et) / et * 100 if et is not None else pd.Series(np.nan, index=px.index),
|
||||
[-1e9, -2.5, -0.5, 0.5, 2.5, 1e9], labels=["<-2.5%", "-2.5..-0.5", "near(±0.5%)", "0.5..2.5", ">2.5%"])
|
||||
# time
|
||||
ct = px.index + pd.Timedelta(minutes=bm)
|
||||
d["hour"] = ct.hour; d["weekday"] = ct.weekday; d["dom"] = pd.cut(ct.day, [0, 5, 10, 15, 20, 25, 31], labels=["1-5", "6-10", "11-15", "16-20", "21-25", "26-31"])
|
||||
d["first_friday"] = np.where((ct.weekday == 4) & (ct.day <= 7), "first Friday", "other")
|
||||
# calendar proximity (high-impact USD)
|
||||
cal = pd.read_parquet(CAL); cal["ts"] = pd.to_datetime(cal.dateline, unit="s", utc=True)
|
||||
hi = np.sort(cal[(cal.currency == "USD") & (cal.impactName.str.lower() == "high")].ts.values.astype("datetime64[ns]"))
|
||||
t = ct.values.astype("datetime64[ns]")
|
||||
nxt = np.searchsorted(hi, t) # next event at/after bar close
|
||||
mins_to = np.where(nxt < len(hi), (hi[np.minimum(nxt, len(hi) - 1)] - t) / np.timedelta64(1, "m"), 1e9)
|
||||
prv = nxt - 1
|
||||
mins_since = np.where(prv >= 0, (t - hi[np.maximum(prv, 0)]) / np.timedelta64(1, "m"), 1e9)
|
||||
d["calendar"] = np.select([mins_since <= 30, mins_to <= 15, mins_to <= 60, mins_to <= 180], ["0-30m AFTER hi-impact", "next 15m", "15-60m ahead", "1-3h ahead"], default="none within 3h")
|
||||
d = d[d.index >= pd.Timestamp(a.start, tz="UTC")].dropna(subset=["rsi", "adx", "bb_pos", "vol_regime"])
|
||||
half = pd.Series(np.where(np.arange(len(d)) < len(d) // 2, 1, 2), index=d.index)
|
||||
print(f"bars {len(d)} base next_bar {100*d.next_bar.mean():.3f}% next_hour {100*d.next_hour.mean():.2f}% (lift = rate / base; 'HIGH' = >1.15x in BOTH halves, 'low' = <0.85x in both)")
|
||||
groups = ["calendar", "regime", "vol_regime", "rsi", "adx", "adx_slope", "macd", "bb_pos", "bb_squeeze", "kc_squeeze", "kc_pos", "ema_align", "ribbon", "roc", "ema200_dist", "dom", "first_friday"]
|
||||
for target in ("next_bar", "next_hour"):
|
||||
print(f"\n################ TARGET: {target} ################")
|
||||
for g in groups:
|
||||
print(f"-- {g}"); print(rate_table(d, g, target, half).to_string())
|
||||
# weekday x hour top cells (next_bar)
|
||||
piv = d.pivot_table(index="weekday", columns="hour", values="next_bar", aggfunc="mean") * 100
|
||||
base = 100 * d.next_bar.mean()
|
||||
cells = piv.stack().sort_values(ascending=False)
|
||||
print(f"\n################ TOP 15 weekday x close-hour cells (next-bar burst pct, base {base:.2f}) ################")
|
||||
names = {0: "Mon", 1: "Tue", 2: "Wed", 3: "Thu", 4: "Fri", 6: "Sun"}
|
||||
for (wd, h), v in cells.head(15).items():
|
||||
n = int(((d.weekday == wd) & (d.hour == h)).sum())
|
||||
print(f" {names.get(wd, wd)} {h:02d}:xx {v:.2f}% ({v/base:.1f}x, n={n})")
|
||||
# direction: does the state predict the burst direction?
|
||||
b = d[d.burst == 1]
|
||||
print("\n################ BURST DIRECTION vs prior state (share of bursts going UP) ################")
|
||||
for g, up_states in (("ema_align", None), ("rsi", None), ("macd", None), ("regime", None)):
|
||||
print(f"-- {g}: " + ", ".join(f"{k}: {100*(v>0).mean():.0f}% up (n={len(v)})" for k, v in b.groupby(g, observed=True).bdir if len(v) >= 20))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,444 +0,0 @@
|
||||
"""Where and when gold moves violently — burst map of the raw bars.
|
||||
|
||||
Usage:
|
||||
python backtest/research/burst_map.py --parquet PATH --bar-minutes N \
|
||||
[--k 3,4,5] [--book BOOK.csv] [--within-bars 12] \
|
||||
[--start YYYY-MM-DD --end YYYY-MM-DD] [--out DIR]
|
||||
|
||||
A "burst" is one bar whose range is >= K x the PRIOR bar's Wilder ATR(14) —
|
||||
the same definition every burst study in this folder has used (spelled
|
||||
out here bar-for-bar: TR = max(H-L, |H-Cprev|, |L-Cprev|), ATR =
|
||||
TR.ewm(alpha=1/14, min_periods=14, adjust=False).mean(), ratio =
|
||||
(H-L) / ATR.shift(1)).
|
||||
|
||||
Reports:
|
||||
* burst frequency by (weekday, UTC bar-CLOSE hour), plus mean |1 h| and
|
||||
|4 h| absolute moves in $ and in ATR units on the same grid;
|
||||
* post-burst drift in the burst's own direction over 6/12/24/48 bars,
|
||||
overall and by session bucket — does a violent bar continue or snap back;
|
||||
* the largest trailing-60-minute close-to-close moves, de-overlapped;
|
||||
* if a trade book is given: how trades that were decided just after a burst,
|
||||
or that were OPEN when one printed, performed against the rest.
|
||||
|
||||
Every ATR-unit number here is normalised by the PRIOR bar's ATR — the same
|
||||
denominator as the burst test, so a "4x burst that ran another 4 ATR" reads
|
||||
directly. All hours are UTC and are the bar's CLOSE hour (label + one bar),
|
||||
matching the clock the live gates evaluate.
|
||||
|
||||
Analysis tooling only — never imported by the live code.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import math
|
||||
import os
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
try: # package import (backtest.research.burst_map)
|
||||
from backtest.research.book_report import base_metrics, load_book, session_of
|
||||
except ImportError: # run directly as a script from anywhere
|
||||
from book_report import base_metrics, load_book, session_of
|
||||
|
||||
SLOT_UNKNOWN = ""
|
||||
|
||||
|
||||
# ─────────────────────────── bar primitives ─────────────────────────
|
||||
|
||||
def load_bars(parquet: str, start: str | None = None,
|
||||
end: str | None = None) -> pd.DataFrame:
|
||||
"""Read a cached OHLCV parquet, de-duplicated and oldest-first."""
|
||||
df = pd.read_parquet(parquet).sort_index()
|
||||
df = df[~df.index.duplicated(keep="last")]
|
||||
if start:
|
||||
df = df[df.index >= pd.Timestamp(start, tz="UTC")]
|
||||
if end:
|
||||
df = df[df.index <= pd.Timestamp(end, tz="UTC")]
|
||||
return df
|
||||
|
||||
|
||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
"""Wilder ATR (the definition every burst study in this folder uses)."""
|
||||
h, l, c = df["High"], df["Low"], df["Close"]
|
||||
tr = pd.concat([h - l, (h - c.shift()).abs(), (l - c.shift()).abs()],
|
||||
axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1 / period, min_periods=period, adjust=False).mean()
|
||||
|
||||
|
||||
def burst_ratio(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
"""Bar range expressed in PRIOR-bar ATRs."""
|
||||
return (df["High"] - df["Low"]) / wilder_atr(df, period).shift(1)
|
||||
|
||||
|
||||
def burst_flags(df: pd.DataFrame, k: float, period: int = 14) -> pd.Series:
|
||||
"""True where the bar's range is >= k x the prior bar's Wilder ATR(14)."""
|
||||
return (burst_ratio(df, period) >= k).fillna(False)
|
||||
|
||||
|
||||
def burst_dir(df: pd.DataFrame) -> pd.Series:
|
||||
"""+1 up bar, −1 down bar, 0 doji (sign of Close − Open)."""
|
||||
return np.sign(df["Close"] - df["Open"]).astype(float)
|
||||
|
||||
|
||||
def close_times(df: pd.DataFrame, bar_minutes: int) -> pd.DatetimeIndex:
|
||||
"""Bar CLOSE timestamps (the parquet index labels bar OPEN time)."""
|
||||
return df.index + pd.Timedelta(minutes=bar_minutes)
|
||||
|
||||
|
||||
# ───────────────────── when does gold move? tables ──────────────────
|
||||
|
||||
def hour_weekday_table(df: pd.DataFrame, k: float, bar_minutes: int = 5,
|
||||
ks: tuple[float, ...] | None = None) -> dict:
|
||||
"""Burst frequency and absolute-move size on a (weekday, close-hour) grid.
|
||||
|
||||
Returns a dict of DataFrames (several tables are wanted, one frame each):
|
||||
burst_pct : weekday rows x close-hour cols, % of bars that are >= k x
|
||||
bursts (one frame; per-k frames live in `burst_pct_by_k`)
|
||||
absmove : (weekday, close-hour) rows; bars, mean |1 h move| and
|
||||
|4 h move| in $ and in prior-bar ATR units
|
||||
by_hour : the same stats collapsed to close-hour
|
||||
by_weekday : the same stats collapsed to weekday
|
||||
"""
|
||||
ks = tuple(ks) if ks else (k,)
|
||||
ct = close_times(df, bar_minutes)
|
||||
hour = pd.Series(ct.hour, index=df.index)
|
||||
wd = pd.Series(ct.weekday, index=df.index)
|
||||
atr_prev = wilder_atr(df).shift(1)
|
||||
|
||||
per_hour = max(1, int(round(60 / bar_minutes)))
|
||||
c = df["Close"]
|
||||
mv1 = (c - c.shift(per_hour)).abs()
|
||||
mv4 = (c - c.shift(4 * per_hour)).abs()
|
||||
stats = pd.DataFrame({
|
||||
"weekday": wd, "hour": hour,
|
||||
"abs1h_usd": mv1, "abs4h_usd": mv4,
|
||||
"abs1h_atr": mv1 / atr_prev, "abs4h_atr": mv4 / atr_prev,
|
||||
})
|
||||
for kk in ks:
|
||||
stats[f"burst_{kk:g}x"] = burst_flags(df, kk).astype(float) * 100.0
|
||||
|
||||
agg = {"abs1h_usd": "mean", "abs4h_usd": "mean",
|
||||
"abs1h_atr": "mean", "abs4h_atr": "mean"}
|
||||
agg.update({f"burst_{kk:g}x": "mean" for kk in ks})
|
||||
|
||||
grid = stats.groupby(["weekday", "hour"]).agg(agg)
|
||||
grid.insert(0, "bars", stats.groupby(["weekday", "hour"]).size())
|
||||
by_hour = stats.groupby("hour").agg(agg)
|
||||
by_hour.insert(0, "bars", stats.groupby("hour").size())
|
||||
by_wd = stats.groupby("weekday").agg(agg)
|
||||
by_wd.insert(0, "bars", stats.groupby("weekday").size())
|
||||
|
||||
pct_by_k = {kk: grid[f"burst_{kk:g}x"].unstack("hour") for kk in ks}
|
||||
return {
|
||||
"burst_pct": pct_by_k[k if k in pct_by_k else ks[0]].round(3),
|
||||
"burst_pct_by_k": {kk: v.round(3) for kk, v in pct_by_k.items()},
|
||||
"absmove": grid.round(3),
|
||||
"by_hour": by_hour.round(3),
|
||||
"by_weekday": by_wd.round(3),
|
||||
}
|
||||
|
||||
|
||||
def post_burst_drift(df: pd.DataFrame, k: float,
|
||||
horizons: tuple[int, ...] = (6, 12, 24, 48),
|
||||
bar_minutes: int = 5) -> pd.DataFrame:
|
||||
"""Signed drift IN the burst's direction over the next h bars, in ATRs.
|
||||
|
||||
A positive mean means violent bars continue; negative means they snap
|
||||
back. Split by the session bucket of the burst bar's close hour.
|
||||
"""
|
||||
flags = burst_flags(df, k)
|
||||
d = burst_dir(df)
|
||||
atr_prev = wilder_atr(df).shift(1)
|
||||
ct = close_times(df, bar_minutes)
|
||||
bucket = pd.Series([session_of(h) for h in ct.hour], index=df.index)
|
||||
c = df["Close"]
|
||||
|
||||
rows = []
|
||||
for h in horizons:
|
||||
fwd = (c.shift(-h) - c) * d / atr_prev
|
||||
sub = fwd[flags].dropna()
|
||||
buckets = bucket[flags].reindex(sub.index)
|
||||
rows.append({"horizon_bars": h, "bucket": "ALL", "n": int(len(sub)),
|
||||
"mean_atr": round(float(sub.mean()), 4) if len(sub) else float("nan"),
|
||||
"median_atr": round(float(sub.median()), 4) if len(sub) else float("nan"),
|
||||
"pct_positive": round(float((sub > 0).mean() * 100), 2) if len(sub) else float("nan")})
|
||||
for b, s in sub.groupby(buckets):
|
||||
rows.append({"horizon_bars": h, "bucket": b, "n": int(len(s)),
|
||||
"mean_atr": round(float(s.mean()), 4),
|
||||
"median_atr": round(float(s.median()), 4),
|
||||
"pct_positive": round(float((s > 0).mean() * 100), 2)})
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
def slot_label(weekday: int, hour: int, day: int) -> str:
|
||||
"""Time-slot label for a big move.
|
||||
|
||||
NOTE: there is no economic-calendar file in this repo, so these are
|
||||
TIME SLOTS, not confirmed events — "payroll window" means "the clock
|
||||
slot US payrolls normally print in", nothing more.
|
||||
"""
|
||||
if weekday == 4 and day <= 7 and hour in (12, 13):
|
||||
return "first-Friday 12:30 slot (payroll-report window)"
|
||||
if weekday == 2 and hour in (18, 19):
|
||||
return "Wednesday 18:00 slot (central-bank window)"
|
||||
if hour in (12, 13):
|
||||
return "12:30 slot (US data window)"
|
||||
if hour == 14:
|
||||
return "14:00 slot"
|
||||
if hour in (21, 22, 23):
|
||||
return "settlement/after-hours"
|
||||
if hour in (0, 1, 2):
|
||||
return "Asia open"
|
||||
return SLOT_UNKNOWN
|
||||
|
||||
|
||||
def largest_moves(df: pd.DataFrame, n: int = 40, window_minutes: int = 60,
|
||||
bar_minutes: int = 5) -> pd.DataFrame:
|
||||
"""The n biggest trailing-`window_minutes` close-to-close moves.
|
||||
|
||||
Overlapping windows are de-duplicated greedily: the largest move wins and
|
||||
anything whose close time is within +/- `window_minutes` of an already
|
||||
kept move is dropped, so one violent hour contributes one row.
|
||||
"""
|
||||
bars = max(1, int(round(window_minutes / bar_minutes)))
|
||||
c = df["Close"]
|
||||
move = c - c.shift(bars)
|
||||
atr_prev = wilder_atr(df).shift(1)
|
||||
ct = close_times(df, bar_minutes)
|
||||
tbl = pd.DataFrame({
|
||||
"close_time": ct, "move_usd": move.to_numpy(),
|
||||
"move_atr": (move / atr_prev).to_numpy(),
|
||||
}).dropna()
|
||||
tbl["abs_usd"] = tbl["move_usd"].abs()
|
||||
tbl = tbl.sort_values("abs_usd", ascending=False)
|
||||
|
||||
keep, taken = [], []
|
||||
gap = pd.Timedelta(minutes=window_minutes)
|
||||
for _, row in tbl.iterrows():
|
||||
t = row["close_time"]
|
||||
if any(abs(t - u) <= gap for u in taken):
|
||||
continue
|
||||
taken.append(t)
|
||||
keep.append(row)
|
||||
if len(keep) >= n:
|
||||
break
|
||||
out = pd.DataFrame(keep).reset_index(drop=True)
|
||||
if out.empty:
|
||||
return out
|
||||
out["weekday"] = out["close_time"].dt.weekday
|
||||
out["hour"] = out["close_time"].dt.hour
|
||||
out["direction"] = np.where(out["move_usd"] > 0, "UP", "DOWN")
|
||||
out["slot"] = [slot_label(w, h, t.day) for w, h, t
|
||||
in zip(out["weekday"], out["hour"], out["close_time"])]
|
||||
out["move_usd"] = out["move_usd"].round(2)
|
||||
out["move_atr"] = out["move_atr"].round(2)
|
||||
return out[["close_time", "weekday", "hour", "direction", "move_usd",
|
||||
"move_atr", "slot"]]
|
||||
|
||||
|
||||
# ───────────────────────── book <-> burst join ──────────────────────
|
||||
|
||||
def _stats(r: pd.Series) -> dict:
|
||||
m = base_metrics(r)
|
||||
return {"n": m["n"], "total_r": round(m["total_r"], 2),
|
||||
"mean_r": round(m["mean_r"], 3), "wr": round(m["wr"], 2),
|
||||
"pf": round(m["pf"], 2) if math.isfinite(m["pf"]) else float("inf")}
|
||||
|
||||
|
||||
def join_book_to_bursts(book_df: pd.DataFrame, bars: pd.DataFrame, k: float,
|
||||
within_bars: int = 12,
|
||||
bar_minutes: int = 5) -> dict:
|
||||
"""Split a trade book by its relationship to burst bars.
|
||||
|
||||
(a) `after_burst` : decision_time falls within `within_bars` bars after
|
||||
the most recent burst's close (bar distance is counted POSITIONALLY on
|
||||
the bar index, so weekend gaps cannot inflate it).
|
||||
(b) `burst_during` : at least one burst closed inside [entry, exit).
|
||||
(c) `burst_agrees` / `burst_opposes`: of (b), whether the FIRST burst that
|
||||
printed during the trade pushed with or against the trade direction.
|
||||
"""
|
||||
ct = close_times(bars, bar_minutes)
|
||||
ct_arr = ct.to_numpy()
|
||||
flags = burst_flags(bars, k).to_numpy()
|
||||
bdir = burst_dir(bars).to_numpy()
|
||||
burst_pos = np.flatnonzero(flags)
|
||||
|
||||
# Last burst position at or before each bar (−1 where none yet).
|
||||
last_burst = np.where(flags, np.arange(len(flags)), -1)
|
||||
last_burst = np.maximum.accumulate(last_burst)
|
||||
|
||||
dec = book_df["decision_time"].astype("datetime64[ns, UTC]").to_numpy()
|
||||
ent = book_df["entry_time"].astype("datetime64[ns, UTC]").to_numpy()
|
||||
exi = book_df["exit_time"].astype("datetime64[ns, UTC]").to_numpy()
|
||||
|
||||
# Position of the last bar that had CLOSED by decision_time.
|
||||
idx = np.searchsorted(ct_arr, dec, side="right") - 1
|
||||
valid = idx >= 0
|
||||
lb = np.where(valid, last_burst[np.clip(idx, 0, len(flags) - 1)], -1)
|
||||
bars_since = np.where((lb >= 0) & valid, idx - lb, np.iinfo(np.int32).max)
|
||||
after = pd.Series((bars_since >= 0) & (bars_since <= within_bars),
|
||||
index=book_df.index)
|
||||
|
||||
# Bursts printing inside the open trade.
|
||||
bct = ct_arr[burst_pos] if len(burst_pos) else np.array([], dtype=ct_arr.dtype)
|
||||
lo = np.searchsorted(bct, ent, side="left")
|
||||
hi = np.searchsorted(bct, exi, side="left")
|
||||
during = pd.Series(hi > lo, index=book_df.index)
|
||||
trade_sign = np.where(
|
||||
book_df["direction"].astype(str).str.upper().str.startswith("B"),
|
||||
1.0, -1.0)
|
||||
has = hi > lo
|
||||
first_dir = np.zeros(len(book_df), dtype=float)
|
||||
if len(burst_pos) and has.any():
|
||||
# hi > lo guarantees lo[has] indexes a real burst.
|
||||
first_dir[has] = bdir[burst_pos[lo[has]]]
|
||||
agrees = pd.Series(has & (first_dir * trade_sign > 0),
|
||||
index=book_df.index)
|
||||
opposes = pd.Series(has & (first_dir * trade_sign < 0),
|
||||
index=book_df.index)
|
||||
|
||||
r = book_df["r_multiple"]
|
||||
out = {
|
||||
"k": k, "within_bars": within_bars,
|
||||
"bursts_in_window": int(len(burst_pos)),
|
||||
"after_burst": _stats(r[after]),
|
||||
"not_after_burst": _stats(r[~after]),
|
||||
"burst_during": _stats(r[during]),
|
||||
"no_burst_during": _stats(r[~during]),
|
||||
"burst_agrees": _stats(r[agrees]),
|
||||
"burst_opposes": _stats(r[opposes]),
|
||||
"median_bars_since_burst": float(np.median(
|
||||
bars_since[bars_since < np.iinfo(np.int32).max])) if (
|
||||
bars_since < np.iinfo(np.int32).max).any() else float("nan"),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
# ──────────────────────────── reporting ─────────────────────────────
|
||||
|
||||
def _fmt_stats(name: str, s: dict) -> str:
|
||||
pf = "inf" if math.isinf(s["pf"]) else f"{s['pf']:.2f}"
|
||||
return (f" {name:<18} n={s['n']:<5} totalR={s['total_r']:>8.2f} "
|
||||
f"meanR={s['mean_r']:>7.3f} WR={s['wr']:>6.2f}% PF={pf}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("--parquet", required=True)
|
||||
ap.add_argument("--bar-minutes", type=int, required=True)
|
||||
ap.add_argument("--k", default="3,4,5",
|
||||
help="burst thresholds in prior-bar ATRs (default 3,4,5)")
|
||||
ap.add_argument("--book", default=None)
|
||||
ap.add_argument("--book-bar-minutes", type=int, default=None,
|
||||
help="bar minutes of the BOOK (default: --bar-minutes)")
|
||||
ap.add_argument("--within-bars", type=int, default=12)
|
||||
ap.add_argument("--start", default=None)
|
||||
ap.add_argument("--end", default=None)
|
||||
ap.add_argument("--n-moves", type=int, default=40)
|
||||
ap.add_argument("--out", default="backtest/research/burst_map_out")
|
||||
args = ap.parse_args()
|
||||
|
||||
ks = [float(x) for x in args.k.split(",") if x.strip()]
|
||||
os.makedirs(args.out, exist_ok=True)
|
||||
bars = load_bars(args.parquet, args.start, args.end)
|
||||
bm = args.bar_minutes
|
||||
primary = ks[0]
|
||||
|
||||
lines: list[str] = []
|
||||
add = lines.append
|
||||
add("=" * 78)
|
||||
add(f"BURST MAP {os.path.basename(args.parquet)}")
|
||||
add("=" * 78)
|
||||
add(f"bars : {len(bars)} ({bars.index[0]} -> {bars.index[-1]}"
|
||||
f", labels = bar OPEN time)")
|
||||
add(f"bar minutes : {bm} hours below are bar CLOSE hours (UTC)")
|
||||
add(f"burst definition : range >= K x PRIOR bar Wilder ATR(14); "
|
||||
f"K = {', '.join(f'{k:g}' for k in ks)}")
|
||||
add("ATR units : always normalised by the PRIOR bar's ATR")
|
||||
for k in ks:
|
||||
f = burst_flags(bars, k)
|
||||
add(f" K={k:g}: {int(f.sum())} burst bars "
|
||||
f"({float(f.mean()) * 100:.3f}% of bars)")
|
||||
|
||||
tabs = hour_weekday_table(bars, primary, bm, ks=tuple(ks))
|
||||
add("")
|
||||
add(f"── BURST % BY (weekday, close-hour), K={primary:g} ──")
|
||||
add(tabs["burst_pct"].fillna(0).round(2).to_string())
|
||||
add("")
|
||||
add("── MEAN ABSOLUTE MOVE BY CLOSE-HOUR (all weekdays) ──")
|
||||
add(tabs["by_hour"].to_string())
|
||||
add("")
|
||||
add("── MEAN ABSOLUTE MOVE BY WEEKDAY ──")
|
||||
add(tabs["by_weekday"].to_string())
|
||||
|
||||
add("")
|
||||
add(f"── POST-BURST DRIFT (in the burst's own direction), K={primary:g} ──")
|
||||
drift = post_burst_drift(bars, primary, bar_minutes=bm)
|
||||
add(drift.to_string(index=False))
|
||||
|
||||
add("")
|
||||
add(f"── LARGEST {args.n_moves} TRAILING-60-MIN MOVES (de-overlapped) ──")
|
||||
add(" slot labels are CLOCK SLOTS — no economic calendar exists in "
|
||||
"this repo, so no event is confirmed.")
|
||||
moves = largest_moves(bars, n=args.n_moves, bar_minutes=bm)
|
||||
add(moves.to_string(index=False))
|
||||
|
||||
join_rows = []
|
||||
if args.book:
|
||||
bbm = args.book_bar_minutes or bm
|
||||
book = load_book(args.book, bbm)
|
||||
add("")
|
||||
add(f"── BOOK JOIN {os.path.basename(args.book)} "
|
||||
f"({len(book)} trades, book bar = {bbm} min) ──")
|
||||
first, last = close_times(bars, bm)[0], close_times(bars, bm)[-1]
|
||||
outside = int(((book["decision_time"] < first)
|
||||
| (book["exit_time"] > last)).sum())
|
||||
if outside:
|
||||
add(f" !! {outside} trade(s) start before / end after the bar "
|
||||
f"window ({first} .. {last}) — their burst context is "
|
||||
f"partial. Widen --start/--end or use a longer parquet.")
|
||||
add(" CAUTION: group (b) is survivorship-shaped — a trade only sees "
|
||||
"a burst if it survived long enough to still be open, so its high "
|
||||
"PF is mostly 'winners last longer', not a burst edge.")
|
||||
for k in ks:
|
||||
j = join_book_to_bursts(book, bars, k, args.within_bars, bm)
|
||||
add(f" K={k:g} ({j['bursts_in_window']} bursts in window; "
|
||||
f"median bars since last burst = "
|
||||
f"{j['median_bars_since_burst']:.0f})")
|
||||
add(f" (a) decided within {args.within_bars} bars of a burst")
|
||||
add(_fmt_stats("after_burst", j["after_burst"]))
|
||||
add(_fmt_stats("not_after_burst", j["not_after_burst"]))
|
||||
add(" (b) a burst printed while the trade was open")
|
||||
add(_fmt_stats("burst_during", j["burst_during"]))
|
||||
add(_fmt_stats("no_burst_during", j["no_burst_during"]))
|
||||
add(" (c) of (b), first burst vs the trade's direction")
|
||||
add(_fmt_stats("burst_agrees", j["burst_agrees"]))
|
||||
add(_fmt_stats("burst_opposes", j["burst_opposes"]))
|
||||
for grp in ("after_burst", "not_after_burst", "burst_during",
|
||||
"no_burst_during", "burst_agrees", "burst_opposes"):
|
||||
join_rows.append({"k": k, "group": grp,
|
||||
"within_bars": args.within_bars,
|
||||
**j[grp]})
|
||||
|
||||
text = "\n".join(lines)
|
||||
print(text)
|
||||
with open(os.path.join(args.out, "burst_map.txt"), "w") as fh:
|
||||
fh.write(text + "\n")
|
||||
pd.concat([v.assign(k=kk) for kk, v in tabs["burst_pct_by_k"].items()]
|
||||
).to_csv(os.path.join(args.out, "hour_weekday_burst_pct.csv"))
|
||||
tabs["absmove"].to_csv(os.path.join(args.out, "hour_weekday_absmove.csv"))
|
||||
tabs["by_hour"].to_csv(os.path.join(args.out, "by_hour.csv"))
|
||||
tabs["by_weekday"].to_csv(os.path.join(args.out, "by_weekday.csv"))
|
||||
pd.concat([post_burst_drift(bars, k, bar_minutes=bm).assign(k=k)
|
||||
for k in ks]).to_csv(
|
||||
os.path.join(args.out, "post_burst_drift.csv"), index=False)
|
||||
moves.to_csv(os.path.join(args.out, "largest_moves.csv"), index=False)
|
||||
if join_rows:
|
||||
pd.DataFrame(join_rows).to_csv(
|
||||
os.path.join(args.out, "book_join.csv"), index=False)
|
||||
print(f"\n[written] {os.path.join(args.out, 'burst_map.txt')}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,127 +0,0 @@
|
||||
"""Are there signs BEFORE a violent bar? Analysis tooling only — never imported by the live code.
|
||||
|
||||
For every bar t, features are computed from bars <= t-1 only. Targets:
|
||||
next_bar : bar t itself is a burst (range >= K x prior Wilder ATR14)
|
||||
next_hour : any burst in bars t .. t+H-1
|
||||
Reports per-quintile burst rates (lift vs base rate) in both halves, the same
|
||||
within/outside the scheduled-data hours, and an out-of-sample logistic model
|
||||
(fit on half 1, scored on half 2): AUC + lift in the top 5% of scores.
|
||||
|
||||
python backtest/research/burst_precursors.py PARQUET BAR_MINUTES [--k 3] [--start ... --end ...]
|
||||
"""
|
||||
import argparse
|
||||
import numpy as np, pandas as pd
|
||||
|
||||
|
||||
def wilder_atr(df, period=14):
|
||||
pc = df["Close"].shift(1)
|
||||
tr = pd.concat([df["High"] - df["Low"], (df["High"] - pc).abs(), (df["Low"] - pc).abs()], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1 / period, min_periods=period, adjust=False).mean()
|
||||
|
||||
|
||||
def build(df, bar_minutes, k):
|
||||
d = df.copy()
|
||||
rng = d["High"] - d["Low"]
|
||||
atr_prev = wilder_atr(d).shift(1) # ATR known at t-1
|
||||
d["burst"] = (rng >= k * atr_prev).astype(int)
|
||||
H = max(1, 60 // bar_minutes)
|
||||
# burst within next H bars (t..t+H-1)
|
||||
d["next_hour"] = d["burst"][::-1].rolling(H, min_periods=1).max()[::-1].astype(int)
|
||||
d["next_bar"] = d["burst"]
|
||||
close_t = d.index + pd.Timedelta(minutes=bar_minutes)
|
||||
d["hour"] = close_t.hour
|
||||
r1 = rng.shift(1)
|
||||
d["squeeze6"] = r1.rolling(6).mean() / atr_prev
|
||||
d["squeeze12"] = r1.rolling(12).mean() / atr_prev
|
||||
d["last_bar_range"] = r1 / atr_prev
|
||||
sma = d["Close"].shift(1).rolling(20).mean(); sd = d["Close"].shift(1).rolling(20).std(ddof=0)
|
||||
bbw = (4 * sd / sma)
|
||||
d["bbw_pct500"] = bbw.rolling(500).rank(pct=True)
|
||||
v1 = d["Volume"].shift(1)
|
||||
d["vol_ratio"] = v1.rolling(3).mean() / v1.rolling(50).mean()
|
||||
d["drift12"] = (d["Close"].shift(1) - d["Close"].shift(13)).abs() / atr_prev
|
||||
d["atr_pct500"] = atr_prev.rolling(500).rank(pct=True)
|
||||
b = d["burst"].shift(1).fillna(0).astype(int)
|
||||
since = np.zeros(len(d)); cnt = 200
|
||||
bv = b.values
|
||||
for i in range(len(d)):
|
||||
cnt = 0 if bv[i] == 1 else min(200, cnt + 1)
|
||||
since[i] = cnt
|
||||
d["bars_since_burst"] = since
|
||||
return d.dropna(subset=["squeeze6", "squeeze12", "bbw_pct500", "vol_ratio", "drift12", "atr_pct500"])
|
||||
|
||||
|
||||
FEATS = ["squeeze6", "squeeze12", "last_bar_range", "bbw_pct500", "vol_ratio", "drift12", "atr_pct500", "bars_since_burst"]
|
||||
|
||||
|
||||
def quintile_table(d, feat, target, half):
|
||||
q = pd.qcut(d[feat], 5, labels=False, duplicates="drop")
|
||||
base1 = d.loc[half == 1, target].mean(); base2 = d.loc[half == 2, target].mean()
|
||||
rows = []
|
||||
for b in sorted(q.dropna().unique()):
|
||||
m = q == b
|
||||
lo, hi = d.loc[m, feat].min(), d.loc[m, feat].max()
|
||||
r1 = d.loc[m & (half == 1), target].mean(); r2 = d.loc[m & (half == 2), target].mean()
|
||||
rows.append((int(b), lo, hi, int(m.sum()), 100 * r1, r1 / base1, 100 * r2, r2 / base2))
|
||||
t = pd.DataFrame(rows, columns=["bin", "lo", "hi", "n", "h1_rate%", "h1_lift", "h2_rate%", "h2_lift"])
|
||||
return t.round(3)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(); ap.add_argument("parquet"); ap.add_argument("bar_minutes", type=int)
|
||||
ap.add_argument("--k", type=float, default=3.0); ap.add_argument("--start"); ap.add_argument("--end")
|
||||
a = ap.parse_args()
|
||||
df = pd.read_parquet(a.parquet)
|
||||
if a.start: df = df[df.index >= pd.Timestamp(a.start, tz="UTC")]
|
||||
if a.end: df = df[df.index < pd.Timestamp(a.end, tz="UTC")]
|
||||
d = build(df, a.bar_minutes, a.k)
|
||||
half = pd.Series(np.where(np.arange(len(d)) < len(d) // 2, 1, 2), index=d.index)
|
||||
print(f"bars {len(d)} K={a.k} base rate next_bar {100*d.next_bar.mean():.3f}% next_hour {100*d.next_hour.mean():.2f}%")
|
||||
print(f"half 1 ends {d.index[len(d)//2 - 1]}")
|
||||
# 1. the clock
|
||||
hr = d.groupby("hour")[["next_bar", "next_hour"]].mean() * 100
|
||||
hr["lift_next_bar"] = hr["next_bar"] / (100 * d.next_bar.mean())
|
||||
print("\n== THE CLOCK (bar-close hour UTC): burst rate % and lift ==")
|
||||
print(hr.round(2).T.to_string())
|
||||
sched = d["hour"].isin([1, 12, 13])
|
||||
print(f"\nscheduled hours (01,12,13) hold {100*d.loc[sched,'next_bar'].sum()/d.next_bar.sum():.1f}% of bursts in {100*sched.mean():.1f}% of bars")
|
||||
# 2. per-feature quintiles, both halves
|
||||
for target in ("next_bar", "next_hour"):
|
||||
print(f"\n== TARGET {target}: quintile burst rate (lift vs base) — ALL hours ==")
|
||||
for f in FEATS:
|
||||
t = quintile_table(d, f, target, half)
|
||||
print(f"-- {f}: lifts h1 {list(t.h1_lift)} h2 {list(t.h2_lift)} (bins lo..hi {t.lo.iloc[0]:.2f}..{t.hi.iloc[-1]:.2f})")
|
||||
# 3. same, OUTSIDE the scheduled hours (does anything predict the unscheduled ones?)
|
||||
u = d[~sched]; hu = half[~sched]
|
||||
print(f"\n== next_bar lifts OUTSIDE scheduled hours (base {100*u.next_bar.mean():.3f}%) ==")
|
||||
for f in FEATS:
|
||||
t = quintile_table(u, f, "next_bar", hu)
|
||||
print(f"-- {f}: h1 {list(t.h1_lift)} h2 {list(t.h2_lift)}")
|
||||
# 4. OOS logistic model: fit half 1, score half 2
|
||||
try:
|
||||
from sklearn.linear_model import LogisticRegression
|
||||
from sklearn.metrics import roc_auc_score
|
||||
X = d[FEATS].copy()
|
||||
for h in range(24): X[f"h{h}"] = (d["hour"] == h).astype(float)
|
||||
Xc = d[FEATS].copy() # clock-free variant
|
||||
for name, XX in (("features+clock", X), ("features only", Xc), ("clock only", X[[f"h{h}" for h in range(24)]])):
|
||||
mu, sd = XX[half == 1].mean(), XX[half == 1].std().replace(0, 1)
|
||||
Z = (XX - mu) / sd
|
||||
for target in ("next_bar", "next_hour"):
|
||||
m = LogisticRegression(C=0.1, max_iter=2000, class_weight="balanced").fit(Z[half == 1], d.loc[half == 1, target])
|
||||
p = m.predict_proba(Z[half == 2])[:, 1]; y = d.loc[half == 2, target]
|
||||
auc = roc_auc_score(y, p)
|
||||
top = p >= np.quantile(p, 0.95)
|
||||
print(f"OOS {name:15s} {target:9s}: AUC {auc:.3f} | top-5% of scores: burst rate {100*y[top].mean():.2f}% vs base {100*y.mean():.2f}% (lift {y[top].mean()/y.mean():.2f}x), captures {100*y[top].sum()/y.sum():.1f}% of bursts")
|
||||
except ImportError:
|
||||
print("sklearn missing — OOS model skipped")
|
||||
# 5. direction: does the burst continue the prior 12-bar drift?
|
||||
bt = d[d.burst == 1]
|
||||
prior = np.sign(bt["Close"].shift(1) - bt["Close"].shift(13)) if False else np.sign(d["Close"].shift(1) - d["Close"].shift(13))[d.burst == 1]
|
||||
bdir = np.sign(bt["Close"] - bt["Open"])
|
||||
agree = (prior == bdir)[(prior != 0) & (bdir != 0)]
|
||||
print(f"\nburst direction == prior-hour drift direction: {100*agree.mean():.1f}% of {len(agree)} bursts (50% = no information)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,111 +0,0 @@
|
||||
"""Do scheduled economic events move gold violently, and do they hurt OPEN trades? Analysis tooling only.
|
||||
|
||||
Needs backtest/cache/calendar.parquet (ff_calendar_scraper.py). Uses USD events by impact.
|
||||
python backtest/research/calendar_impact.py PARQUET_BARS BAR_MINUTES BOOK.csv ARM DIST [--start --end]
|
||||
"""
|
||||
import argparse, sys
|
||||
from pathlib import Path
|
||||
import numpy as np, pandas as pd
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from open_trade_bursts import wilder_atr, simulate, r_of, stats, K
|
||||
|
||||
CAL = Path(__file__).resolve().parent.parent / "cache" / "calendar.parquet"
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(); ap.add_argument("parquet"); ap.add_argument("bar_minutes", type=int)
|
||||
ap.add_argument("book"); ap.add_argument("arm", type=float); ap.add_argument("dist", type=float)
|
||||
ap.add_argument("--start", default="2023-09-05"); ap.add_argument("--end", default="2026-09-05")
|
||||
a = ap.parse_args(); bm = a.bar_minutes
|
||||
cal = pd.read_parquet(CAL)
|
||||
cal["ts"] = pd.to_datetime(cal["dateline"], unit="s", utc=True)
|
||||
cal = cal[(cal.ts >= pd.Timestamp(a.start, tz="UTC")) & (cal.ts < pd.Timestamp(a.end, tz="UTC"))]
|
||||
usd = cal[cal.currency == "USD"].copy()
|
||||
usd["impact"] = usd["impactName"].str.lower()
|
||||
print(f"calendar: {len(cal)} events in window, USD {len(usd)}: " + ", ".join(f"{k}={v}" for k, v in usd.impact.value_counts().items()))
|
||||
px = pd.read_parquet(a.parquet); px = px[(px.index >= pd.Timestamp(a.start, tz="UTC")) & (px.index < pd.Timestamp(a.end, tz="UTC"))]
|
||||
atr_prev = wilder_atr(px).shift(1)
|
||||
rng = px["High"] - px["Low"]; burst = (rng >= K * atr_prev).fillna(False)
|
||||
close = px["Close"]; idx = px.index
|
||||
H1 = 60 // bm
|
||||
move60 = (close.shift(-H1) - close).abs() / atr_prev # |move| over the next hour from bar OPEN-ish (label)
|
||||
move15 = (close.shift(-max(1, 15 // bm)) - close).abs() / atr_prev
|
||||
burst60 = burst[::-1].rolling(H1, min_periods=1).max()[::-1].astype(bool)
|
||||
# event -> bar containing it (label <= ts < label + bm)
|
||||
def bar_pos(ts):
|
||||
p = idx.searchsorted(ts, side="right") - 1
|
||||
return p if 0 <= p < len(idx) and (ts - idx[p]) < pd.Timedelta(minutes=bm) else -1
|
||||
usd["pos"] = [bar_pos(t) for t in usd.ts]
|
||||
usd = usd[usd.pos >= 0]
|
||||
usd["bar_range_atr"] = rng.values[usd.pos] / atr_prev.values[usd.pos]
|
||||
usd["burst_in_60m"] = burst60.values[usd.pos]
|
||||
usd["move60_atr"] = move60.values[usd.pos]; usd["move60_usd"] = (close.shift(-H1) - close).abs().values[usd.pos]
|
||||
usd["move15_atr"] = move15.values[usd.pos]
|
||||
# baseline: all bars, and bars in the same close-hours without any USD high/medium event within the hour
|
||||
chour = (idx + pd.Timedelta(minutes=bm)).hour
|
||||
ev_bars = set()
|
||||
for p in usd[usd.impact.isin(["high", "medium"])].pos: ev_bars.update(range(max(0, p - H1), min(len(idx), p + H1)))
|
||||
quiet = ~np.isin(np.arange(len(idx)), list(ev_bars))
|
||||
def base_row(mask, label):
|
||||
return dict(group=label, n=int(mask.sum()), bar_range_atr=float((rng / atr_prev)[mask].mean()), burst_in_60m_pct=100 * float(burst60[mask].mean()),
|
||||
move15_atr=float(move15[mask].mean()), move60_atr=float(move60[mask].mean()), move60_usd=float((close.shift(-H1) - close).abs()[mask].mean()))
|
||||
rows = [base_row(np.ones(len(idx), bool), "ALL bars"), base_row(quiet & np.isin(chour, [12, 13, 14]), "12-14 UTC bars with NO USD event")]
|
||||
for imp in ("high", "medium", "low"):
|
||||
g = usd[usd.impact == imp]
|
||||
rows.append(dict(group=f"USD {imp}-impact events", n=len(g), bar_range_atr=g.bar_range_atr.mean(), burst_in_60m_pct=100 * g.burst_in_60m.mean(),
|
||||
move15_atr=g.move15_atr.mean(), move60_atr=g.move60_atr.mean(), move60_usd=g.move60_usd.mean()))
|
||||
print("\n== HOW MUCH GOLD MOVES AROUND USD EVENTS (event bar = the bar containing the release) ==")
|
||||
print(pd.DataFrame(rows).round(2).to_string(index=False))
|
||||
hi = usd[usd.impact == "high"]
|
||||
top = hi.groupby("name").agg(n=("ts", "size"), bar_range_atr=("bar_range_atr", "mean"), burst60_pct=("burst_in_60m", lambda s: 100 * s.mean()),
|
||||
move60_usd=("move60_usd", "mean"), move60_atr=("move60_atr", "mean")).sort_values("n", ascending=False)
|
||||
print("\n== HIGH-IMPACT USD EVENTS BY NAME (n >= 6) ==")
|
||||
print(top[top.n >= 6].round(2).to_string())
|
||||
# what share of ALL bursts sit within ±30 min of a high/medium USD event?
|
||||
near = np.zeros(len(idx), bool)
|
||||
for p in usd[usd.impact.isin(["high", "medium"])].pos: near[max(0, p - 30 // bm):min(len(idx), p + 30 // bm + 1)] = True
|
||||
nearh = np.zeros(len(idx), bool)
|
||||
for p in hi.pos: nearh[max(0, p - 30 // bm):min(len(idx), p + 30 // bm + 1)] = True
|
||||
print(f"\nshare of ALL violent bars within ±30 min of a USD high-impact event: {100*burst[nearh].sum()/burst.sum():.1f}% (those bars are {100*nearh.mean():.1f}% of time)")
|
||||
print(f"share within ±30 min of a USD high- or medium-impact event: {100*burst[near].sum()/burst.sum():.1f}% (bars {100*near.mean():.1f}% of time)")
|
||||
# open trades through high-impact events
|
||||
book = pd.read_csv(a.book); book["entry_time"] = pd.to_datetime(book.entry_time, utc=True, format="mixed")
|
||||
bars = dict(O=px.Open.values, H=px.High.values, L=px.Low.values, C=px.Close.values, burst=burst.values, chour=chour.values)
|
||||
pos = idx.get_indexer(book.entry_time); ok = pos >= 0; book = book[ok].reset_index(drop=True); pos = pos[ok]
|
||||
ev_pos = np.sort(hi.pos.values)
|
||||
# flat mask: exit at the open of the bar that CONTAINS a high-impact event... use the bar before (exit ~5-15 min before)
|
||||
flat_mask = np.zeros(len(idx), bool); tight_mask = np.zeros(len(idx), bool)
|
||||
for p in ev_pos:
|
||||
if p - 1 >= 0: flat_mask[p - 1] = True
|
||||
tight_mask[max(0, p - 1):min(len(idx), p + H1)] = True
|
||||
res = {}
|
||||
for v, kw in (("base", {}), ("mask_flat", dict(flat_mask=flat_mask)), ("mask_tight", dict(tight_mask=tight_mask))):
|
||||
rs, fl, thru = [], [], []
|
||||
for i, t in book.iterrows():
|
||||
xp, held, reason, flag = simulate(bars, pos[i], t.direction, t.entry, t.sl, a.arm, a.dist, v, **kw)
|
||||
rs.append(r_of(t.direction, t.entry, t.sl, xp)); fl.append(flag)
|
||||
if v == "base":
|
||||
j0, j1 = pos[i], pos[i] + held
|
||||
k = ev_pos[(ev_pos > j0) & (ev_pos <= j1)]
|
||||
thru.append(k[0] if len(k) else -1)
|
||||
res[v] = (np.array(rs), np.array(fl))
|
||||
if v == "base": book["ev_pos"] = thru
|
||||
base = res["base"][0]; half = np.where(np.arange(len(book)) < len(book) // 2, 1, 2)
|
||||
hit = book.ev_pos.values >= 0
|
||||
print(f"\n== OPEN TRADES THAT HELD THROUGH A HIGH-IMPACT USD EVENT ({a.book.split('/')[-1]}) ==")
|
||||
print(f"trades through an event: {hit.sum()} of {len(book)} ({100*hit.mean():.1f}%) | mean R {base[hit].mean():+.3f} vs {base[~hit].mean():+.3f} for the rest | WR {100*(base[hit]>0).mean():.0f}% vs {100*(base[~hit]>0).mean():.0f}%")
|
||||
if hit.sum():
|
||||
d = np.where(book.direction == "BUY", 1.0, -1.0)
|
||||
m15 = np.array([ (close.shift(-max(1,15//bm)).values[p] - close.values[p]) if p >= 0 else np.nan for p in book.ev_pos])
|
||||
with_ = hit & (d * m15 > 0); against = hit & (d * m15 < 0)
|
||||
print(f" event's first 15-min move WITH the trade: n={with_.sum()} mean R {base[with_].mean():+.3f} | AGAINST: n={against.sum()} mean R {base[against].mean():+.3f} (WR {100*(base[against]>0).mean():.0f}%)")
|
||||
print("\n== PROTECTION AROUND HIGH-IMPACT USD EVENTS ==")
|
||||
b = stats(base)
|
||||
for v, (r, f) in res.items():
|
||||
s = stats(r); aff = f == 1
|
||||
d1 = r[half == 1].sum() - base[half == 1].sum(); d2 = r[half == 2].sum() - base[half == 2].sum()
|
||||
print(f"{v:11s} total {s['total']:+8.2f} Δ {s['total']-b['total']:+7.2f} PF {s['pf']:.2f} DD {s['dd']:.2f} affected {aff.sum():4d} (affected: base {base[aff].sum():+.2f} -> {r[aff].sum():+.2f}) | Δ h1 {d1:+.2f} / h2 {d2:+.2f}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,178 +0,0 @@
|
||||
"""Commit-ladder campaign driver (2026-08).
|
||||
|
||||
Question: of all signal-code states shipped in the last two months, which
|
||||
configuration is the most accurate? Historical in-era backtest numbers are
|
||||
not comparable (the backtester's realism model changed on 07-08/07-16 and
|
||||
the data feed switched to Oanda on 07-28), so every historical
|
||||
configuration is recreated on TODAY's harness via --set overrides and
|
||||
replayed on the same pinned Oanda bars. Bug-fix commits stay on; the
|
||||
ladder compares shipped tuning decisions only.
|
||||
|
||||
Usage:
|
||||
python backtest/research/commit_ladder.py # full matrix
|
||||
python backtest/research/commit_ladder.py --tf 5min # one timeframe
|
||||
python backtest/research/commit_ladder.py --windows 12mo,oos
|
||||
|
||||
Resumable: a run whose CSV already exists in the output dir is skipped.
|
||||
Outputs: backtest/research/commit_ladder_2026-08/<variant>_<window>.csv
|
||||
+ .log per run, summary.csv, and a printed ranking table.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
BT = ROOT / "backtest" / "gpu_backtest.py"
|
||||
OUT = ROOT / "backtest" / "research" / "commit_ladder_2026-08"
|
||||
|
||||
WINDOWS = {
|
||||
"12mo": ("2025-07-30", "2026-07-30"),
|
||||
"oos": ("2024-06-01", "2025-03-30"),
|
||||
"5yr": ("2021-07-30", "2026-07-30"),
|
||||
}
|
||||
SPLIT_12MO = pd.Timestamp("2026-01-29", tz="UTC") # book_diff.py's split
|
||||
|
||||
# (variant, "commit date what-it-represents", [--set overrides])
|
||||
VARIANTS: dict[str, list[tuple[str, str, list[str]]]] = {
|
||||
"5min": [
|
||||
("M5-A", "dad2432 07-09 gates+trailing+pullback",
|
||||
["xau_strategy.blocked_hours_utc=[0,1,2,10,11,13,16,21]",
|
||||
"xau_strategy.cp_enabled=false",
|
||||
"signals.trail_dist_r=0.75",
|
||||
"xau_strategy.m5_min_abs_score=30",
|
||||
"xau_strategy.m5_min_confirmation_score=10"]),
|
||||
("M5-B", "f3ba4d5 07-11 +hour-9 block",
|
||||
["xau_strategy.blocked_hours_utc=[0,1,2,10,11,13,16,21,9]",
|
||||
"xau_strategy.cp_enabled=false",
|
||||
"signals.trail_dist_r=0.75",
|
||||
"xau_strategy.m5_min_abs_score=30",
|
||||
"xau_strategy.m5_min_confirmation_score=10"]),
|
||||
("M5-C", "de6424e 07-16 +chart patterns",
|
||||
["xau_strategy.blocked_hours_utc=[0,1,2,10,11,13,16,21,9]",
|
||||
"signals.trail_dist_r=0.75",
|
||||
"xau_strategy.m5_min_abs_score=30",
|
||||
"xau_strategy.m5_min_confirmation_score=10"]),
|
||||
("M5-D", "3ee79b5 07-21 +trail dist 0.85",
|
||||
["xau_strategy.blocked_hours_utc=[0,1,2,10,11,13,16,21,9]",
|
||||
"xau_strategy.m5_min_abs_score=30",
|
||||
"xau_strategy.m5_min_confirmation_score=10"]),
|
||||
("M5-E", "0396970 07-30 relaxation (HEAD)", []),
|
||||
],
|
||||
"15min": [
|
||||
("M15-A", "987ec5b 07-09 pre-campaigns",
|
||||
["signals.tp_floor_rr=0.5",
|
||||
"xau_strategy.blocked_regimes=[]",
|
||||
"xau_strategy.sell_blocked_hours_utc=[]",
|
||||
"signals.trail_arm_r=0.60"]),
|
||||
("M15-B", "f3ba4d5/f88e39f 07-11 1R floor + regime block",
|
||||
["xau_strategy.sell_blocked_hours_utc=[]",
|
||||
"signals.trail_arm_r=0.60"]),
|
||||
("M15-C", "7d3fd95 07-16 +SELL hour 15/17 block",
|
||||
["signals.trail_arm_r=0.60"]),
|
||||
("M15-D", "bf8e5fb 07-21 trail arm 0.70 (HEAD)", []),
|
||||
],
|
||||
"1h": [
|
||||
("H1-A", "pre-d95d31a 07-08 TP-tightening model",
|
||||
["signals.min_rr_ratio=0.4",
|
||||
"signals.tp_tightening_factor=0.8"]),
|
||||
("H1-B", "HEAD", []),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def run_one(tf: str, name: str, overrides: list[str], wkey: str,
|
||||
spread: str | None = None) -> Path:
|
||||
suffix = f"_s{spread}" if spread else ""
|
||||
csv_path = OUT / f"{name}_{wkey}{suffix}.csv"
|
||||
log_path = OUT / f"{name}_{wkey}{suffix}.log"
|
||||
if csv_path.exists():
|
||||
print(f" [skip] {csv_path.name} exists")
|
||||
return csv_path
|
||||
start, end = WINDOWS[wkey]
|
||||
cmd = [sys.executable, str(BT), "--pair", "XAUUSD",
|
||||
"--primary-tf", tf, "--start", start, "--end", end,
|
||||
"--export-csv", str(csv_path)]
|
||||
if spread:
|
||||
cmd += ["--spread", spread]
|
||||
for ov in overrides:
|
||||
cmd += ["--set", ov]
|
||||
print(f" [run ] {csv_path.name} ...", flush=True)
|
||||
with open(log_path, "w") as lf:
|
||||
rc = subprocess.run(cmd, stdout=lf, stderr=subprocess.STDOUT,
|
||||
cwd=ROOT).returncode
|
||||
if rc != 0:
|
||||
raise SystemExit(f"backtest failed (rc={rc}) — see {log_path}")
|
||||
log = log_path.read_text()
|
||||
n_ov = log.count("--set override")
|
||||
if n_ov != len(overrides):
|
||||
raise SystemExit(
|
||||
f"{name}_{wkey}: expected {len(overrides)} --set overrides, "
|
||||
f"log shows {n_ov} — see {log_path}")
|
||||
if "Loading cached" not in log and wkey != "sanity":
|
||||
print(f" [warn] {log_path.name}: no 'Loading cached' line — "
|
||||
f"data was fetched live for at least one timeframe")
|
||||
return csv_path
|
||||
|
||||
|
||||
def metrics(csv_path: Path, wkey: str) -> dict:
|
||||
df = pd.read_csv(csv_path, parse_dates=["entry_time"])
|
||||
if df.empty:
|
||||
return {"trades": 0, "total_r": 0.0, "wr": 0.0, "pf": 0.0,
|
||||
"max_dd": 0.0, "h1_r": 0.0, "h2_r": 0.0, "years_pos": ""}
|
||||
if df["entry_time"].dt.tz is None:
|
||||
df["entry_time"] = df["entry_time"].dt.tz_localize("UTC")
|
||||
r = df["r_multiple"]
|
||||
pos, neg = r[r > 0].sum(), r[r < 0].sum()
|
||||
cum = r.cumsum()
|
||||
dd = (pd.concat([pd.Series([0.0]), cum]).cummax()[1:] - cum).max()
|
||||
out = {
|
||||
"trades": len(df),
|
||||
"total_r": round(r.sum(), 1),
|
||||
"wr": round((r > 0).mean() * 100, 1),
|
||||
"pf": round(pos / abs(neg), 2) if neg else float("inf"),
|
||||
"max_dd": round(dd, 1),
|
||||
"h1_r": "", "h2_r": "", "years_pos": "",
|
||||
}
|
||||
if wkey == "12mo":
|
||||
h1 = r[df["entry_time"] < SPLIT_12MO].sum()
|
||||
out["h1_r"] = round(h1, 1)
|
||||
out["h2_r"] = round(r.sum() - h1, 1)
|
||||
if wkey == "5yr":
|
||||
by_year = r.groupby(df["entry_time"].dt.year).sum()
|
||||
out["years_pos"] = f"{(by_year > 0).sum()}/{len(by_year)}"
|
||||
return out
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--tf", choices=list(VARIANTS), default=None)
|
||||
ap.add_argument("--windows", default="12mo,oos,5yr")
|
||||
args = ap.parse_args()
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
wkeys = [w.strip() for w in args.windows.split(",")]
|
||||
tfs = [args.tf] if args.tf else list(VARIANTS)
|
||||
|
||||
rows = []
|
||||
for tf in tfs:
|
||||
for name, label, overrides in VARIANTS[tf]:
|
||||
for wkey in wkeys:
|
||||
csv_path = run_one(tf, name, overrides, wkey)
|
||||
m = metrics(csv_path, wkey)
|
||||
rows.append({"tf": tf, "variant": name, "represents": label,
|
||||
"window": wkey, **m})
|
||||
print(f" {name} {wkey}: {m['trades']} trades, "
|
||||
f"{m['total_r']}R, PF {m['pf']}, WR {m['wr']}%")
|
||||
|
||||
summary = pd.DataFrame(rows)
|
||||
summary.to_csv(OUT / "summary.csv", index=False)
|
||||
print("\n=== SUMMARY (also in summary.csv) ===")
|
||||
print(summary.to_string(index=False))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,193 +0,0 @@
|
||||
"""Dukascopy XAU/USD tick download + bid/ask spread study.
|
||||
|
||||
Validates ForexBot2's $0.30 round-trip cost assumption (config:
|
||||
SignalConfig.round_trip_cost / PAIR_INFO default_spread). Round-trip
|
||||
cost of a market entry+exit = one ask-bid spread width (buy at ask,
|
||||
sell at bid), so the assumption holds iff typical spread ~ $0.30.
|
||||
|
||||
bi5 format: LZMA 'alone' stream of 20-byte big-endian records
|
||||
(ms_in_hour uint32, ask uint32, bid uint32, ask_vol f32, bid_vol f32);
|
||||
XAUUSD prices scaled x1000. URL months are 0-indexed.
|
||||
"""
|
||||
import lzma
|
||||
import sys
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import date, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import requests
|
||||
|
||||
SCALE = 1000.0
|
||||
CACHE = Path(__file__).parent / "tick_cache"
|
||||
CACHE.mkdir(exist_ok=True)
|
||||
URL = "https://datafeed.dukascopy.com/datafeed/XAUUSD/{y}/{m:02d}/{d:02d}/{h:02d}h_ticks.bi5"
|
||||
|
||||
# Bot's tradable bar-close hours: session 03:00-23:00 minus XAU blocked
|
||||
# hours [0,1,2,10,11,13,16,21] (M5 additionally blocks 9 — reported
|
||||
# separately below).
|
||||
BLOCKED = {0, 1, 2, 10, 11, 13, 16, 21}
|
||||
TRADABLE = sorted(set(range(3, 23)) - BLOCKED)
|
||||
|
||||
# ── Day sample ────────────────────────────────────────────────────────
|
||||
recent = [date(2026, 7, 6) + timedelta(days=i) for i in range(17)]
|
||||
recent = [d for d in recent if d.weekday() < 5] # Mon-Fri Jul 6-22
|
||||
monthly = [date(2025, 8, 13), date(2025, 9, 10), date(2025, 10, 15),
|
||||
date(2025, 11, 12), date(2025, 12, 10), date(2026, 1, 14),
|
||||
date(2026, 2, 11), date(2026, 3, 11), date(2026, 4, 15),
|
||||
date(2026, 5, 13), date(2026, 6, 10)] # mid-month Wednesdays
|
||||
nfp = [date(2026, 6, 5), date(2026, 7, 3)] # NFP Fridays
|
||||
sundays = [date(2026, 7, 12), date(2026, 7, 19)] # Sun 22-24 open
|
||||
DAYS = sorted(set(recent + monthly + nfp + sundays))
|
||||
assert all(d.weekday() < 5 for d in recent + monthly + nfp)
|
||||
|
||||
|
||||
_tls = __import__("threading").local()
|
||||
|
||||
|
||||
def _session() -> requests.Session:
|
||||
if not hasattr(_tls, "s"):
|
||||
_tls.s = requests.Session()
|
||||
_tls.s.headers["User-Agent"] = "Mozilla/5.0 (research; spread-study)"
|
||||
return _tls.s
|
||||
|
||||
|
||||
def fetch(day: date, hour: int) -> Path:
|
||||
p = CACHE / f"{day.isoformat()}_{hour:02d}.bi5"
|
||||
if p.exists():
|
||||
return p
|
||||
u = URL.format(y=day.year, m=day.month - 1, d=day.day, h=hour)
|
||||
last = None
|
||||
for attempt in range(4):
|
||||
try:
|
||||
r = _session().get(u, timeout=30)
|
||||
if r.status_code != 200:
|
||||
raise RuntimeError(f"{u} -> HTTP {r.status_code}")
|
||||
p.write_bytes(r.content)
|
||||
return p
|
||||
except (requests.ConnectionError, requests.Timeout, RuntimeError) as exc:
|
||||
last = exc
|
||||
__import__("time").sleep(1.5 * (attempt + 1))
|
||||
if hasattr(_tls, "s"):
|
||||
_tls.s.close()
|
||||
del _tls.s
|
||||
raise RuntimeError(f"giving up on {u}: {last}")
|
||||
|
||||
|
||||
def decode(p: Path, day: date, hour: int) -> pd.DataFrame:
|
||||
blob = p.read_bytes()
|
||||
if not blob:
|
||||
return pd.DataFrame()
|
||||
raw = lzma.decompress(blob)
|
||||
n = len(raw) // 20
|
||||
a = np.frombuffer(raw[: n * 20], dtype=">u4").reshape(n, 5)
|
||||
f = np.frombuffer(raw[: n * 20], dtype=">f4").reshape(n, 5)
|
||||
df = pd.DataFrame({
|
||||
"ms": a[:, 0].astype(np.int64),
|
||||
"ask": a[:, 1] / SCALE,
|
||||
"bid": a[:, 2] / SCALE,
|
||||
})
|
||||
df["spread"] = df["ask"] - df["bid"]
|
||||
df["date"], df["hour"] = day.isoformat(), hour
|
||||
df["weekday"] = day.weekday()
|
||||
return df
|
||||
|
||||
|
||||
def load_all() -> pd.DataFrame:
|
||||
jobs = [(d, h) for d in DAYS for h in range(24)]
|
||||
with ThreadPoolExecutor(max_workers=3) as ex:
|
||||
paths = list(ex.map(lambda j: fetch(*j), jobs))
|
||||
frames = [decode(p, d, h) for p, (d, h) in zip(paths, jobs)]
|
||||
df = pd.concat([f for f in frames if not f.empty], ignore_index=True)
|
||||
return df
|
||||
|
||||
|
||||
def wstats(g: pd.DataFrame) -> pd.Series:
|
||||
"""Tick stats + time-weighted mean (each tick's spread held until the
|
||||
next tick; last tick held to hour end)."""
|
||||
tw_num = tw_den = 0.0
|
||||
keys = [k for k in ("date", "hour") if k in g.columns]
|
||||
for _, grp in g.groupby(keys, sort=False):
|
||||
ms = grp["ms"].to_numpy()
|
||||
sp = grp["spread"].to_numpy()
|
||||
w = np.diff(np.append(ms, 3_600_000)).clip(min=0)
|
||||
tw_num += float((sp * w).sum())
|
||||
tw_den += float(w.sum())
|
||||
s = g["spread"]
|
||||
return pd.Series({
|
||||
"ticks": len(s),
|
||||
"tw_mean": tw_num / tw_den if tw_den else np.nan,
|
||||
"mean": s.mean(), "median": s.median(),
|
||||
"p75": s.quantile(0.75), "p90": s.quantile(0.90),
|
||||
"p99": s.quantile(0.99), "max": s.max(),
|
||||
"pct_le_030": (s <= 0.30).mean() * 100,
|
||||
"pct_ge_060": (s >= 0.60).mean() * 100,
|
||||
})
|
||||
|
||||
|
||||
def m5_boundary_spreads(df: pd.DataFrame) -> pd.DataFrame:
|
||||
"""Spread of the FIRST tick at/after each M5 boundary — what a
|
||||
next-open fill actually pays."""
|
||||
rows = []
|
||||
for (day, hour), grp in df.groupby(["date", "hour"], sort=False):
|
||||
ms = grp["ms"].to_numpy()
|
||||
sp = grp["spread"].to_numpy()
|
||||
wd = grp["weekday"].iat[0]
|
||||
for b in range(0, 3_600_000, 300_000):
|
||||
i = np.searchsorted(ms, b)
|
||||
if i < len(ms) and ms[i] - b <= 30_000: # tick within 30 s
|
||||
rows.append((day, wd, hour, b // 60_000, sp[i]))
|
||||
return pd.DataFrame(rows, columns=["date", "weekday", "hour", "minute", "spread"])
|
||||
|
||||
|
||||
def main() -> None:
|
||||
df = load_all()
|
||||
print(f"days={df['date'].nunique()} ticks={len(df):,} "
|
||||
f"price range {df['bid'].min():.0f}-{df['ask'].max():.0f}")
|
||||
|
||||
print("\n── Overall (all sampled hours) ──")
|
||||
print(wstats(df).round(3).to_string())
|
||||
|
||||
trad = df[df["hour"].isin(TRADABLE) & (df["weekday"] < 5)]
|
||||
print("\n── Bot-tradable hours (session 03-23 minus blocked, Mon-Fri) ──")
|
||||
print(wstats(trad).round(3).to_string())
|
||||
|
||||
roll = df[df["hour"].isin([21, 22, 23])]
|
||||
print("\n── Rollover window 21-24 UTC ──")
|
||||
print(wstats(roll).round(3).to_string())
|
||||
|
||||
print("\n── Per UTC hour ──")
|
||||
per_hour = df[df["weekday"] < 5].groupby("hour").apply(wstats, include_groups=False).round(3)
|
||||
per_hour["tradable"] = ["Y" if h in TRADABLE else "-" for h in per_hour.index]
|
||||
print(per_hour.to_string())
|
||||
per_hour.to_csv(Path(__file__).parent / "duka_spread_by_hour.csv")
|
||||
|
||||
print("\n── M5 next-open boundary fills ──")
|
||||
mb = m5_boundary_spreads(df)
|
||||
mb_trad = mb[mb["hour"].isin(TRADABLE) & (mb["weekday"] < 5)]
|
||||
for name, d in [("all hours", mb), ("tradable hours", mb_trad)]:
|
||||
s = d["spread"]
|
||||
print(f"{name}: n={len(s):,} mean={s.mean():.3f} median={s.median():.3f} "
|
||||
f"p90={s.quantile(0.9):.3f} p99={s.quantile(0.99):.3f} "
|
||||
f"%<=0.30={(s <= 0.30).mean()*100:.1f} %>=0.60={(s >= 0.60).mean()*100:.1f}")
|
||||
mb.to_csv(Path(__file__).parent / "duka_m5_boundary_spreads.csv", index=False)
|
||||
|
||||
print("\n── By month (tradable hours, tick median / tw_mean) ──")
|
||||
trad2 = trad.copy()
|
||||
trad2["month"] = trad2["date"].str[:7]
|
||||
bym = trad2.groupby("month").apply(wstats, include_groups=False).round(3)
|
||||
print(bym[["ticks", "tw_mean", "median", "p90", "pct_le_030", "pct_ge_060"]].to_string())
|
||||
|
||||
print("\n── NFP Fridays 12:00-15:00 UTC vs same hours other days ──")
|
||||
nfp_days = {"2026-06-05", "2026-07-03"}
|
||||
win = df[df["hour"].isin([12, 13, 14])]
|
||||
for name, d in [("NFP", win[win["date"].isin(nfp_days)]),
|
||||
("non-NFP", win[~win["date"].isin(nfp_days) & (win["weekday"] < 5)])]:
|
||||
s = d["spread"]
|
||||
print(f"{name}: n={len(s):,} tick-median={s.median():.3f} p90={s.quantile(0.9):.3f} "
|
||||
f"p99={s.quantile(0.99):.3f} max={s.max():.2f}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,25 +0,0 @@
|
||||
hour,ticks,tw_mean,mean,median,p75,p90,p99,max,pct_le_030,pct_ge_060,tradable
|
||||
0,362893.0,0.749,0.768,0.79,0.82,0.86,1.1,7.69,0.053,94.561,-
|
||||
1,540182.0,0.735,0.742,0.74,0.81,0.87,1.0,5.59,0.073,90.449,-
|
||||
2,431589.0,0.716,0.722,0.73,0.8,0.86,0.95,2.96,0.1,85.27,-
|
||||
3,343952.0,0.708,0.711,0.71,0.79,0.84,0.95,1.67,0.081,82.939,Y
|
||||
4,246262.0,0.704,0.702,0.71,0.78,0.82,0.94,1.86,0.098,82.627,Y
|
||||
5,388365.0,0.712,0.716,0.72,0.79,0.82,0.95,2.17,0.076,86.745,Y
|
||||
6,368999.0,0.65,0.651,0.65,0.69,0.74,0.94,2.32,0.079,71.668,Y
|
||||
7,313149.0,0.601,0.598,0.59,0.65,0.69,0.994,3.08,0.239,42.904,Y
|
||||
8,347276.0,0.612,0.616,0.6,0.66,0.74,1.0,3.04,0.308,48.349,Y
|
||||
9,321619.0,0.612,0.615,0.6,0.66,0.72,1.0,5.95,0.288,50.644,Y
|
||||
10,295271.0,0.611,0.618,0.6,0.66,0.71,1.0,9.36,0.164,49.253,-
|
||||
11,336963.0,0.619,0.624,0.6,0.67,0.79,1.0,4.97,0.203,50.178,-
|
||||
12,476773.0,0.66,0.697,0.63,0.72,0.877,1.367,14.98,0.257,59.633,Y
|
||||
13,651425.0,0.641,0.648,0.63,0.69,0.787,1.487,11.407,0.296,60.297,-
|
||||
14,673391.0,0.625,0.625,0.63,0.68,0.77,0.9,11.98,0.458,59.085,Y
|
||||
15,598398.0,0.636,0.638,0.64,0.7,0.78,0.91,4.49,0.433,66.296,Y
|
||||
16,485023.0,0.648,0.644,0.64,0.7,0.757,0.917,4.09,0.118,68.496,-
|
||||
17,450119.0,0.626,0.628,0.627,0.7,0.726,0.84,4.08,0.154,63.337,Y
|
||||
18,411538.0,0.626,0.63,0.637,0.7,0.71,0.79,4.16,0.107,67.734,Y
|
||||
19,435396.0,0.638,0.643,0.65,0.7,0.72,0.896,5.804,0.11,72.007,Y
|
||||
20,212654.0,0.684,0.665,0.66,0.7,0.76,1.217,7.15,0.226,70.215,Y
|
||||
21,31319.0,0.627,0.622,0.61,0.68,0.75,1.02,2.08,0.089,53.153,-
|
||||
22,108879.0,0.829,0.834,0.797,0.84,0.98,2.05,10.11,0.056,95.434,Y
|
||||
23,161079.0,1.032,0.913,0.8,0.89,1.307,2.77,13.174,0.092,95.41,-
|
||||
|
@@ -1,8 +1,8 @@
|
||||
# Feed Divergence Study — Oanda vs Tiingo vs Twelve Data (Phase 1)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Test plan:** `docs/data_source_change_test_plan.md`
|
||||
**Tool:** `scripts/feed_divergence_study.py` (fetch_range over 365d, parquet-cached)
|
||||
**Test plan:** `docs/data_source_change_test_plan.md` (removed 2026-09-06, in git history; the pending Phase 4 shadow scan is noted in CLAUDE.md)
|
||||
**Tool:** `scripts/feed_divergence_study.py` (script removed 2026-09-06, in git history) (fetch_range over 365d, parquet-cached)
|
||||
**Window:** 2025-08-14 → 2026-08-14 (12 months); Oanda-vs-TD truncated at TD cache head (2026-07-23)
|
||||
|
||||
## Headline
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
tf,a,b,a_bars,b_bars,common,a_only,b_only,dir_agree_pct,ret_corr,mean_abs_dclose,p95_abs_dclose,max_abs_dclose,dclose_pct_atr_mean
|
||||
5min,oanda,tiingo,70901,73241,70807,94,2434,97.674000,0.994970,0.123070,0.285000,26.105000,2.147942
|
||||
|
@@ -1,144 +0,0 @@
|
||||
"""Forex Factory calendar scraper -> backtest/cache/calendar.parquet.
|
||||
|
||||
Fetches weekly calendar pages and parses the event JSON embedded in
|
||||
`window.calendarComponentStates` (the `days:` array). Event `dateline`
|
||||
is a unix epoch, so timestamps carry no timezone ambiguity — verified
|
||||
2026-07-11 against known release times (NFP 12:30/13:30 UTC across DST,
|
||||
ECB 12:15/13:15) and against EUR/USD M15 volatility: the bar containing
|
||||
a high-impact event averages 3.5x normal range, and shifting event
|
||||
times by ±1h destroys that alignment.
|
||||
|
||||
Usage:
|
||||
python backtest/research/ff_calendar_scraper.py --start 2024-01-01 --end 2026-07-13
|
||||
python backtest/research/ff_calendar_scraper.py # refresh: last stored week -> today
|
||||
|
||||
Merges into the existing parquet on event (id, dateline), so refreshing
|
||||
is idempotent and also picks up revised `actual` values for recent weeks.
|
||||
Be polite: ~1.6s between requests (a full year is ~52 requests).
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import time
|
||||
from datetime import date, timedelta
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
import pandas as pd
|
||||
import requests
|
||||
|
||||
CACHE_DIR = Path(__file__).resolve().parent.parent / "cache"
|
||||
PARQUET = CACHE_DIR / "calendar.parquet"
|
||||
|
||||
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
|
||||
"(KHTML, like Gecko) Chrome/126.0 Safari/537.36")
|
||||
|
||||
FIELDS = ["id", "dateline", "name", "country", "currency", "impactName",
|
||||
"timeLabel", "timeMasked", "actual", "forecast", "previous",
|
||||
"revision", "actualBetterWorse"]
|
||||
|
||||
|
||||
def week_url(d: date) -> str:
|
||||
return (f"https://www.forexfactory.com/calendar?week="
|
||||
f"{d.strftime('%b').lower()}{d.day}.{d.year}")
|
||||
|
||||
|
||||
def extract_days(html: str) -> Optional[list]:
|
||||
"""Balanced-bracket extraction of the `days:` JSON array."""
|
||||
i = html.find("days:")
|
||||
if i < 0:
|
||||
return None
|
||||
j = html.find("[", i)
|
||||
depth, in_str, esc = 0, False, False
|
||||
for k in range(j, len(html)):
|
||||
ch = html[k]
|
||||
if in_str:
|
||||
if esc:
|
||||
esc = False
|
||||
elif ch == "\\":
|
||||
esc = True
|
||||
elif ch == '"':
|
||||
in_str = False
|
||||
else:
|
||||
if ch == '"':
|
||||
in_str = True
|
||||
elif ch == "[":
|
||||
depth += 1
|
||||
elif ch == "]":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return json.loads(html[j:k + 1])
|
||||
return None
|
||||
|
||||
|
||||
def fetch_week(sess: requests.Session, d: date, retries: int = 4) -> Optional[List[dict]]:
|
||||
for attempt in range(retries):
|
||||
try:
|
||||
r = sess.get(week_url(d), timeout=30)
|
||||
if r.status_code == 200:
|
||||
days = extract_days(r.text)
|
||||
if days is not None:
|
||||
return [{k: e.get(k) for k in FIELDS}
|
||||
for day in days for e in day.get("events", [])]
|
||||
except requests.RequestException:
|
||||
pass
|
||||
time.sleep(5 * (attempt + 1))
|
||||
return None
|
||||
|
||||
|
||||
def scrape(start: date, end: date) -> pd.DataFrame:
|
||||
sess = requests.Session()
|
||||
sess.headers["User-Agent"] = UA
|
||||
rows: List[dict] = []
|
||||
failed: List[date] = []
|
||||
d = start
|
||||
while d <= end:
|
||||
week_rows = fetch_week(sess, d)
|
||||
if week_rows is None:
|
||||
failed.append(d)
|
||||
print(f"FAILED week {d}")
|
||||
else:
|
||||
rows.extend(week_rows)
|
||||
d += timedelta(days=7)
|
||||
time.sleep(1.6)
|
||||
if failed:
|
||||
print(f"WARNING: {len(failed)} weeks failed: {failed}")
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
def merge_into_store(new: pd.DataFrame, path: Path = PARQUET) -> pd.DataFrame:
|
||||
if path.exists():
|
||||
old = pd.read_parquet(path)
|
||||
combined = pd.concat([old, new])
|
||||
else:
|
||||
combined = new
|
||||
# keep="last": a refresh overwrites stale rows (actuals fill in
|
||||
# after release, revisions land weeks later).
|
||||
combined = (combined.drop_duplicates(subset=["id", "dateline"], keep="last")
|
||||
.sort_values("dateline").reset_index(drop=True))
|
||||
combined.to_parquet(path, index=False)
|
||||
return combined
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--start", type=date.fromisoformat, default=None)
|
||||
ap.add_argument("--end", type=date.fromisoformat, default=date.today())
|
||||
args = ap.parse_args()
|
||||
|
||||
start = args.start
|
||||
if start is None:
|
||||
if PARQUET.exists():
|
||||
last = pd.read_parquet(PARQUET).dateline.max()
|
||||
start = pd.Timestamp(last, unit="s").date() - timedelta(days=7)
|
||||
else:
|
||||
start = date(2024, 1, 1)
|
||||
print(f"Scraping {start} -> {args.end}")
|
||||
new = scrape(start, args.end)
|
||||
combined = merge_into_store(new)
|
||||
print(f"Store: {len(combined)} events "
|
||||
f"({pd.Timestamp(combined.dateline.min(), unit='s')} -> "
|
||||
f"{pd.Timestamp(combined.dateline.max(), unit='s')}) -> {PARQUET}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,697 +0,0 @@
|
||||
"""
|
||||
DeepSeek news-analysis validation harness — XAU/USD.
|
||||
====================================================
|
||||
Research-only (no production code changes). Answers: is the DeepSeek
|
||||
layer's news analysis *correct*?
|
||||
|
||||
Part A (events): replays every high-impact USD release with a genuine
|
||||
surprise (2024-01 → 2026-07, from backtest/cache/calendar.parquet)
|
||||
through the REAL `NewsEngine._llm_event_direction` — real prompt, real
|
||||
payload, real validation — and scores its "gold bullish/bearish" calls
|
||||
against gold's actual move (Oanda M15 mids) at +15m/30m/1h/4h/24h.
|
||||
Baselines: mechanical surprise-sign doctrine, FF actualBetterWorse,
|
||||
always-bullish/bearish (2024-26 is a gold bull regime), coin flip.
|
||||
|
||||
Part B (headlines): a curated golden set (llm_news_golden.csv) through
|
||||
the REAL `NewsEngine._llm_headline_scores` in production-sized batches,
|
||||
vs the keyword scorer it overrides. Includes repeat-batch (determinism)
|
||||
and shuffled-order (cross-contamination) probes.
|
||||
|
||||
Implementation notes that keep the replay faithful to production:
|
||||
* `_llm_event_direction`/`_llm_headline_scores` resolve the client via
|
||||
`get_llm_client()`, so we seed the module singleton with a research
|
||||
client (raised daily cap: 586 events > production cap 500).
|
||||
* The event cache key omits the forecast, so a repeat `actual` would
|
||||
silently re-serve a direction computed for a different forecast —
|
||||
the cache is cleared before every call.
|
||||
* A "neutral" reply and a hard failure both return None; they are
|
||||
disambiguated by reading the event cache back after the call
|
||||
(absent = failure, "" = neutral, "bullish:/bearish: …" = call).
|
||||
* Raw request/response bytes are captured with a requests response
|
||||
hook — the request-building code path is untouched.
|
||||
|
||||
Usage (from repo root):
|
||||
venv/bin/python backtest/research/llm_news_eval.py events --limit 5
|
||||
venv/bin/python backtest/research/llm_news_eval.py events --determinism 20
|
||||
venv/bin/python backtest/research/llm_news_eval.py headlines
|
||||
venv/bin/python backtest/research/llm_news_eval.py report
|
||||
|
||||
Artifacts land in backtest/research/llm_news_eval_2026-08/ (JSONL replay
|
||||
logs are resume-safe: completed events are never re-paid). `report` is a
|
||||
pure offline function of the artifacts.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import random
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
import llm_client
|
||||
from app_config import deepseek_api_key, get_app_config
|
||||
from config import NewsConfig
|
||||
from llm_client import DEFAULT_MODEL, DeepSeekClient
|
||||
from news_engine import EconomicEvent, NewsEngine
|
||||
|
||||
RESEARCH_DIR = Path(__file__).parent
|
||||
ART_DIR = RESEARCH_DIR / "llm_news_eval_2026-08"
|
||||
CALENDAR = REPO_ROOT / "backtest" / "cache" / "calendar.parquet"
|
||||
BARS = (REPO_ROOT / "backtest" / "cache" /
|
||||
"oanda_XAU_USD_15min_20210531_20260804.parquet")
|
||||
GOLDEN = RESEARCH_DIR / "llm_news_golden.csv"
|
||||
EVENTS_JSONL = ART_DIR / "events_replay.jsonl"
|
||||
HEADLINES_JSONL = ART_DIR / "headlines_replay.jsonl"
|
||||
SCORED_CSV = ART_DIR / "events_scored.csv"
|
||||
|
||||
HORIZONS = [("15m", 15), ("30m", 30), ("1h", 60), ("4h", 240),
|
||||
("24h", 1440)]
|
||||
NEUTRAL_BAND = 0.15 # |xau| below this = neutral prediction (Part B)
|
||||
BATCH_CAP = 25 # production batch cap (news_engine.py:783)
|
||||
EXPECTED_CORPUS = 586
|
||||
|
||||
|
||||
# ── shared plumbing ──────────────────────────────────────────────────
|
||||
|
||||
class _Recorder:
|
||||
"""requests response hook capturing the exact wire traffic."""
|
||||
|
||||
def __init__(self):
|
||||
self.last = None
|
||||
|
||||
def hook(self, resp, **kwargs):
|
||||
body = resp.request.body
|
||||
if isinstance(body, bytes):
|
||||
body = body.decode("utf-8", "replace")
|
||||
self.last = {
|
||||
"request_body": body,
|
||||
"http_status": resp.status_code,
|
||||
"response_body": resp.text,
|
||||
"elapsed_s": round(resp.elapsed.total_seconds(), 3),
|
||||
}
|
||||
return resp
|
||||
|
||||
def pop(self):
|
||||
cap, self.last = self.last, None
|
||||
return cap or {}
|
||||
|
||||
|
||||
def install_research_client(timeout: float):
|
||||
"""Build a research DeepSeekClient (raised cap) and seed the module
|
||||
singleton so the production code paths use it. Returns (client,
|
||||
recorder)."""
|
||||
app = get_app_config()
|
||||
key = deepseek_api_key(app)
|
||||
ds = app.deepseek_settings
|
||||
if not key:
|
||||
sys.exit("No DeepSeek API key (env DEEPSEEK_API_KEY or "
|
||||
"config.json deepseek.api_key) — aborting before spend.")
|
||||
if ds.get("enabled") is False:
|
||||
sys.exit("deepseek.enabled is false in config.json — aborting.")
|
||||
client = DeepSeekClient(
|
||||
api_key=key,
|
||||
model=str(ds.get("model", DEFAULT_MODEL)),
|
||||
timeout=timeout,
|
||||
daily_cap=10 ** 6, # research: corpus 586 > production cap 500
|
||||
)
|
||||
rec = _Recorder()
|
||||
client._session.hooks["response"].append(rec.hook)
|
||||
with llm_client._client_lock:
|
||||
llm_client._client = client
|
||||
return client, rec
|
||||
|
||||
|
||||
def _s(x):
|
||||
"""Parquet str columns carry float NaN — EconomicEvent needs None."""
|
||||
return None if pd.isna(x) else str(x)
|
||||
|
||||
|
||||
def load_corpus():
|
||||
"""High-impact USD releases with a genuine surprise, through the
|
||||
production `EconomicEvent.surprise` gate. Asserts run before any
|
||||
API spend."""
|
||||
df = pd.read_parquet(CALENDAR)
|
||||
df = df[(df["impactName"] == "high") & (df["currency"] == "USD")]
|
||||
rows = []
|
||||
for r in df.itertuples():
|
||||
ev = EconomicEvent(
|
||||
time=datetime.fromtimestamp(int(r.dateline), tz=timezone.utc),
|
||||
currency="USD", impact="high", event=_s(r.name) or "",
|
||||
forecast=_s(r.forecast), previous=_s(r.previous),
|
||||
actual=_s(r.actual),
|
||||
)
|
||||
if ev.surprise not in ("positive", "negative"):
|
||||
continue
|
||||
rows.append({
|
||||
"ffid": int(r.id), "dateline": int(r.dateline),
|
||||
"utc_time": ev.time.isoformat(), "name": ev.event,
|
||||
"actual": ev.actual, "forecast": ev.forecast,
|
||||
"previous": ev.previous, "surprise": ev.surprise,
|
||||
"better_worse": int(r.actualBetterWorse), "ev": ev,
|
||||
})
|
||||
ids = [d["ffid"] for d in rows]
|
||||
if len(rows) != EXPECTED_CORPUS:
|
||||
sys.exit(f"corpus size {len(rows)} != {EXPECTED_CORPUS} — "
|
||||
"calendar.parquet changed; re-verify before spending.")
|
||||
if len(set(ids)) != len(ids):
|
||||
sys.exit("FF ids not unique — resume keying would break.")
|
||||
# Pin the actualBetterWorse mapping on two unambiguous NFP prints.
|
||||
nfp = {d["utc_time"][:10]: d["better_worse"] for d in rows
|
||||
if d["name"] == "Non-Farm Employment Change"}
|
||||
if nfp.get("2024-01-05") != 1 or nfp.get("2024-05-03") != 2:
|
||||
sys.exit("actualBetterWorse mapping pins failed (expected "
|
||||
"1=better-for-USD on 2024-01-05 beat, 2=worse on "
|
||||
"2024-05-03 miss).")
|
||||
rows.sort(key=lambda d: (d["dateline"], d["ffid"]))
|
||||
return rows
|
||||
|
||||
|
||||
def _load_jsonl(path: Path):
|
||||
recs = []
|
||||
if path.exists():
|
||||
with open(path) as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if line:
|
||||
recs.append(json.loads(line))
|
||||
return recs
|
||||
|
||||
|
||||
# ── Part A: event replay ─────────────────────────────────────────────
|
||||
|
||||
def _extract_reason(cap):
|
||||
try:
|
||||
content = json.loads(cap["response_body"])[
|
||||
"choices"][0]["message"]["content"]
|
||||
return json.loads(content).get("reason")
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def cmd_events(args):
|
||||
ART_DIR.mkdir(exist_ok=True)
|
||||
corpus = load_corpus()
|
||||
if args.limit:
|
||||
corpus = corpus[:args.limit]
|
||||
engine = NewsEngine(NewsConfig())
|
||||
existing = _load_jsonl(EVENTS_JSONL)
|
||||
done1 = {r["ffid"] for r in existing
|
||||
if r.get("run_pass") == 1 and r.get("llm_direction")}
|
||||
done2 = {r["ffid"] for r in existing
|
||||
if r.get("run_pass") == 2 and r.get("llm_direction")}
|
||||
pending = [d for d in corpus if d["ffid"] not in done1]
|
||||
print(f"corpus {len(corpus)} | already recorded {len(done1)} | "
|
||||
f"pending {len(pending)} | determinism re-calls "
|
||||
f"{args.determinism} | model calls are ~100 tokens each "
|
||||
f"(deepseek-v4-flash, static-prefix cached) → cost ≈ cents")
|
||||
if not pending and not args.determinism:
|
||||
print("nothing to do")
|
||||
return
|
||||
_, rec = install_research_client(args.timeout)
|
||||
|
||||
out = open(EVENTS_JSONL, "a")
|
||||
|
||||
def call_one(d, run_pass):
|
||||
# Cache key omits forecast → cleared per call so 196 colliding
|
||||
# corpus rows can't be served a stale direction.
|
||||
with NewsEngine._cache_lock:
|
||||
NewsEngine._llm_event_cache.clear()
|
||||
ev = d["ev"]
|
||||
engine._llm_event_direction(ev)
|
||||
key = (ev.event, ev.currency, str(ev.actual or ""))
|
||||
with NewsEngine._cache_lock:
|
||||
cached = NewsEngine._llm_event_cache.get(key)
|
||||
cap = rec.pop()
|
||||
if cached is None:
|
||||
direction, text = None, None # hard failure → retry
|
||||
elif cached == "":
|
||||
direction, text = "neutral", ""
|
||||
else:
|
||||
direction, text = cached.split(":", 1)[0], cached
|
||||
recd = {k: d[k] for k in (
|
||||
"ffid", "dateline", "utc_time", "name", "actual", "forecast",
|
||||
"previous", "surprise", "better_worse")}
|
||||
recd.update({
|
||||
"llm_direction": direction, "llm_text": text,
|
||||
"reason_raw": _extract_reason(cap), "run_pass": run_pass,
|
||||
"ts": datetime.now(timezone.utc).isoformat(), **cap,
|
||||
})
|
||||
out.write(json.dumps(recd) + "\n")
|
||||
out.flush()
|
||||
return direction
|
||||
|
||||
remaining = pending
|
||||
for npass in range(1, args.retry_passes + 1):
|
||||
if not remaining:
|
||||
break
|
||||
failed = []
|
||||
for i, d in enumerate(remaining):
|
||||
if call_one(d, 1) is None:
|
||||
failed.append(d)
|
||||
if args.sleep:
|
||||
time.sleep(args.sleep)
|
||||
if (i + 1) % 25 == 0:
|
||||
print(f" pass {npass}: {i + 1}/{len(remaining)} "
|
||||
f"({len(failed)} failed)")
|
||||
print(f"pass {npass} done: {len(remaining) - len(failed)} ok, "
|
||||
f"{len(failed)} failed")
|
||||
remaining = failed
|
||||
if remaining:
|
||||
print(f"WARNING: {len(remaining)} events still unrecorded after "
|
||||
f"{args.retry_passes} passes — re-run `events` to resume.")
|
||||
|
||||
if args.determinism:
|
||||
ok_ids = {x["ffid"] for x in remaining}
|
||||
done_now = [d for d in corpus if d["ffid"] not in ok_ids]
|
||||
done_now.sort(key=lambda d: (d["name"], d["dateline"]))
|
||||
stride = max(1, len(done_now) // args.determinism)
|
||||
sample = done_now[::stride][:args.determinism]
|
||||
todo = [d for d in sample if d["ffid"] not in done2]
|
||||
print(f"determinism probe: {len(todo)} re-calls "
|
||||
f"({len(sample) - len(todo)} already recorded)")
|
||||
for d in todo:
|
||||
call_one(d, 2)
|
||||
if args.sleep:
|
||||
time.sleep(args.sleep)
|
||||
out.close()
|
||||
print("events replay complete →", EVENTS_JSONL)
|
||||
|
||||
|
||||
# ── Part B: golden headlines ─────────────────────────────────────────
|
||||
|
||||
def _load_golden():
|
||||
g = pd.read_csv(GOLDEN)
|
||||
need = {"headline", "category", "expected_xau_sign", "rationale"}
|
||||
if not need <= set(g.columns):
|
||||
sys.exit(f"golden CSV missing columns {need - set(g.columns)}")
|
||||
if not g["expected_xau_sign"].isin([-1, 0, 1]).all():
|
||||
sys.exit("expected_xau_sign must be -1/0/+1")
|
||||
if not g["headline"].is_unique:
|
||||
sys.exit("duplicate headlines in golden CSV")
|
||||
return g
|
||||
|
||||
|
||||
def cmd_headlines(args):
|
||||
ART_DIR.mkdir(exist_ok=True)
|
||||
g = _load_golden()
|
||||
heads = list(g["headline"])
|
||||
_, rec = install_research_client(args.timeout)
|
||||
engine = NewsEngine(NewsConfig())
|
||||
out = open(HEADLINES_JSONL, "w") # tiny (~9 calls): fresh each run
|
||||
|
||||
def run_batch(texts, tag, batch_idx):
|
||||
scores = None
|
||||
for attempt in range(3):
|
||||
scores = engine._llm_headline_scores(texts)
|
||||
cap = rec.pop()
|
||||
if scores is not None:
|
||||
break
|
||||
print(f" {tag} batch {batch_idx} attempt {attempt + 1} "
|
||||
f"failed (http {cap.get('http_status')})")
|
||||
time.sleep(2.0)
|
||||
out.write(json.dumps({
|
||||
"run_tag": tag, "batch": batch_idx, "headlines": texts,
|
||||
"scores": scores,
|
||||
"ts": datetime.now(timezone.utc).isoformat(), **cap,
|
||||
}) + "\n")
|
||||
out.flush()
|
||||
return scores
|
||||
|
||||
batches = [heads[i:i + BATCH_CAP] for i in range(0, len(heads),
|
||||
BATCH_CAP)]
|
||||
for bi, b in enumerate(batches):
|
||||
ok = run_batch(b, "main", bi) is not None
|
||||
print(f"main batch {bi} ({len(b)} headlines): "
|
||||
f"{'ok' if ok else 'FAILED'}")
|
||||
time.sleep(args.sleep)
|
||||
run_batch(batches[0], "repeat", 0)
|
||||
print("repeat probe done")
|
||||
shuffled = heads[:]
|
||||
random.Random(20260806).shuffle(shuffled)
|
||||
for bi in range(0, len(shuffled), BATCH_CAP):
|
||||
run_batch(shuffled[bi:bi + BATCH_CAP], "shuffled",
|
||||
bi // BATCH_CAP)
|
||||
print("shuffled probe done →", HEADLINES_JSONL)
|
||||
out.close()
|
||||
|
||||
|
||||
# ── report: pure offline function of the artifacts ───────────────────
|
||||
|
||||
def _binom_p(k, n):
|
||||
"""Exact two-sided binomial p-value vs p=0.5 (no scipy)."""
|
||||
if n == 0:
|
||||
return float("nan")
|
||||
dist = [math.comb(n, i) for i in range(n + 1)]
|
||||
tot = float(2 ** n)
|
||||
pk = dist[k]
|
||||
return min(1.0, sum(c for c in dist if c <= pk) / tot)
|
||||
|
||||
|
||||
def _mcnemar_p(hit_a, hit_b):
|
||||
"""Exact McNemar via binomial on discordant pairs. Inputs: aligned
|
||||
boolean lists."""
|
||||
b = sum(1 for x, y in zip(hit_a, hit_b) if x and not y)
|
||||
c = sum(1 for x, y in zip(hit_a, hit_b) if not x and y)
|
||||
return b, c, _binom_p(min(b, c), b + c)
|
||||
|
||||
|
||||
def _latest_events(recs):
|
||||
"""Last pass-1 record per ffid (retries append; last wins)."""
|
||||
out = {}
|
||||
for r in recs:
|
||||
if r.get("run_pass") == 1:
|
||||
out[r["ffid"]] = r
|
||||
return list(out.values())
|
||||
|
||||
|
||||
def _fmt_rate(hits, n, label=""):
|
||||
if n == 0:
|
||||
return f"{label:<24} n=0"
|
||||
p = _binom_p(hits, n)
|
||||
return (f"{label:<24} {hits:>4}/{n:<4} = {hits / n * 100:5.1f}% "
|
||||
f"(binom p={p:.3f})")
|
||||
|
||||
|
||||
def report_events():
|
||||
recs = _latest_events(_load_jsonl(EVENTS_JSONL))
|
||||
if not recs:
|
||||
print("no event replay records — run `events` first")
|
||||
return
|
||||
bars = pd.read_parquet(BARS)
|
||||
closes = bars["Close"].to_numpy()
|
||||
close_ns = (bars.index + pd.Timedelta(minutes=15)).asi8
|
||||
|
||||
def anchor(t_ns, h_min):
|
||||
i = np.searchsorted(close_ns, t_ns, side="right") - 1
|
||||
if i < 0:
|
||||
return None
|
||||
stale_ns = max(45.0, h_min / 4.0) * 60e9
|
||||
if t_ns - close_ns[i] > stale_ns:
|
||||
return None
|
||||
return closes[i]
|
||||
|
||||
rows, skip = [], {h: 0 for h, _ in HORIZONS}
|
||||
base_skipped = 0
|
||||
for r in recs:
|
||||
t_ns = int(r["dateline"]) * 10 ** 9
|
||||
base = anchor(t_ns, 0)
|
||||
if base is None:
|
||||
base_skipped += 1
|
||||
continue
|
||||
row = dict(r)
|
||||
row.pop("request_body", None)
|
||||
row.pop("response_body", None)
|
||||
row["year"] = r["utc_time"][:4]
|
||||
row["base"] = base
|
||||
for hname, hmin in HORIZONS:
|
||||
p = anchor(t_ns + hmin * 60 * 10 ** 9, hmin)
|
||||
if p is None:
|
||||
skip[hname] += 1
|
||||
row[f"d_{hname}"] = np.nan
|
||||
else:
|
||||
row[f"d_{hname}"] = p - base
|
||||
rows.append(row)
|
||||
df = pd.DataFrame(rows)
|
||||
df.to_csv(SCORED_CSV, index=False)
|
||||
|
||||
n_fail = int((df["llm_direction"].isna()).sum())
|
||||
print("═" * 70)
|
||||
print("PART A — event gold-direction vs actual XAU move")
|
||||
print("═" * 70)
|
||||
print(f"events replayed: {len(df)} (baseline-price skipped: "
|
||||
f"{base_skipped}) | LLM hard failures: {n_fail}")
|
||||
print("horizon skip counts (stale/closed):",
|
||||
{k: v for k, v in skip.items() if v})
|
||||
dist = df["llm_direction"].value_counts(dropna=False).to_dict()
|
||||
print("LLM call distribution:", dist)
|
||||
|
||||
# Unconditional per-horizon noise scale (null): median |Δ| over all
|
||||
# 2024+ bar times, position-shifted.
|
||||
bars24 = bars[bars.index >= "2024-01-01"]
|
||||
null_med = {}
|
||||
for hname, hmin in HORIZONS:
|
||||
k = hmin // 15
|
||||
d = (bars24["Close"].shift(-k) - bars24["Close"]).abs().dropna()
|
||||
null_med[hname] = float(d.median())
|
||||
print("unconditional null median |Δ| ($):",
|
||||
{k: round(v, 2) for k, v in null_med.items()})
|
||||
|
||||
# Strategy call columns: +1 gold up, −1 gold down, 0/NaN excluded.
|
||||
df["c_llm"] = df["llm_direction"].map(
|
||||
{"bullish": 1, "bearish": -1}).fillna(0)
|
||||
df["c_doctrine"] = df["surprise"].map({"positive": -1, "negative": 1})
|
||||
df["c_ffbw"] = df["better_worse"].map({1: -1, 2: 1}).fillna(0)
|
||||
df["c_bull"] = 1
|
||||
df["c_bear"] = -1
|
||||
|
||||
strategies = [("LLM", "c_llm"), ("doctrine(surprise)", "c_doctrine"),
|
||||
("FF betterWorse", "c_ffbw"), ("always-bullish", "c_bull"),
|
||||
("always-bearish", "c_bear")]
|
||||
|
||||
for hname, _ in HORIZONS:
|
||||
dcol = f"d_{hname}"
|
||||
sub = df[df[dcol].notna()]
|
||||
print(f"\n── horizon {hname} (n scored bars={len(sub)}) ──")
|
||||
zeros = int((sub[dcol] == 0).sum())
|
||||
if zeros:
|
||||
print(f" Δ==0 count (scored as miss): {zeros}")
|
||||
for label, col in strategies:
|
||||
s = sub[sub[col] != 0]
|
||||
hits = int((np.sign(s[dcol]) == s[col]).sum())
|
||||
print(" " + _fmt_rate(hits, len(s), label))
|
||||
# noise-filtered LLM hit rate
|
||||
s = sub[(sub["c_llm"] != 0) &
|
||||
(sub[dcol].abs() >= null_med[hname])]
|
||||
hits = int((np.sign(s[dcol]) == s["c_llm"]).sum())
|
||||
print(" " + _fmt_rate(hits, len(s),
|
||||
f"LLM |Δ|≥null-med"))
|
||||
# event-window magnitude vs null
|
||||
mag = sub[dcol].abs().median()
|
||||
print(f" event-window median |Δ| = ${mag:.2f} "
|
||||
f"(null ${null_med[hname]:.2f})")
|
||||
# paired McNemar: LLM vs each baseline on common directional set
|
||||
for label, col in [("doctrine", "c_doctrine"),
|
||||
("FF betterWorse", "c_ffbw"),
|
||||
("always-bullish", "c_bull")]:
|
||||
s = sub[(sub["c_llm"] != 0) & (sub[col] != 0)]
|
||||
ha = list(np.sign(s[dcol]) == s["c_llm"])
|
||||
hb = list(np.sign(s[dcol]) == s[col])
|
||||
b, c, p = _mcnemar_p(ha, hb)
|
||||
print(f" McNemar LLM vs {label:<15} b={b:>3} c={c:>3} "
|
||||
f"p={p:.3f}")
|
||||
|
||||
# neutral-call flatness (15m)
|
||||
neu = df[(df["llm_direction"] == "neutral") & df["d_15m"].notna()]
|
||||
dire = df[(df["c_llm"] != 0) & df["d_15m"].notna()]
|
||||
if len(neu):
|
||||
print(f"\nneutral calls (n={len(neu)}): median |Δ_15m| "
|
||||
f"${neu['d_15m'].abs().median():.2f} vs directional "
|
||||
f"(n={len(dire)}) ${dire['d_15m'].abs().median():.2f} "
|
||||
f"(null ${null_med['15m']:.2f})")
|
||||
|
||||
# breakdowns at 1h
|
||||
dcol = "d_1h"
|
||||
sub = df[df[dcol].notna() & (df["c_llm"] != 0)]
|
||||
print("\n── LLM hit rate by event name (1h, top 10 by count) ──")
|
||||
for name, grp in sorted(sub.groupby("name"), key=lambda kv: -len(kv[1]))[:10]:
|
||||
hits = int((np.sign(grp[dcol]) == grp["c_llm"]).sum())
|
||||
print(" " + _fmt_rate(hits, len(grp), name[:24]))
|
||||
print("── LLM hit rate by year (1h) ──")
|
||||
for yr, grp in sub.groupby("year"):
|
||||
hits = int((np.sign(grp[dcol]) == grp["c_llm"]).sum())
|
||||
print(" " + _fmt_rate(hits, len(grp), yr))
|
||||
|
||||
# cluster view: shared datelines
|
||||
print("\n── same-timestamp clusters ──")
|
||||
clus = df.groupby("dateline").filter(lambda g: len(g) > 1)
|
||||
groups = list(clus.groupby("dateline"))
|
||||
consistent = 0
|
||||
net_hits, net_n = 0, 0
|
||||
for _, g in groups:
|
||||
signs = set(g.loc[g["c_llm"] != 0, "c_llm"])
|
||||
if len(signs) <= 1:
|
||||
consistent += 1
|
||||
net = int(g["c_llm"].sum())
|
||||
d = g["d_1h"].iloc[0]
|
||||
if net != 0 and pd.notna(d) and d != 0:
|
||||
net_n += 1
|
||||
if np.sign(d) == np.sign(net):
|
||||
net_hits += 1
|
||||
print(f"clusters: {len(groups)} datelines "
|
||||
f"({len(clus)} events, {len(clus) / max(len(df), 1) * 100:.0f}% of "
|
||||
f"corpus) | internally consistent: {consistent}/{len(groups)}")
|
||||
print(" " + _fmt_rate(net_hits, net_n, "cluster net-call (1h)"))
|
||||
|
||||
# determinism probe
|
||||
p2 = {r["ffid"]: r for r in _load_jsonl(EVENTS_JSONL)
|
||||
if r.get("run_pass") == 2 and r.get("llm_direction")}
|
||||
if p2:
|
||||
p1 = {r["ffid"]: r for r in recs}
|
||||
agree = same_reason = n = 0
|
||||
for fid, r2 in p2.items():
|
||||
r1 = p1.get(fid)
|
||||
if not r1 or not r1.get("llm_direction"):
|
||||
continue
|
||||
n += 1
|
||||
if r1["llm_direction"] == r2["llm_direction"]:
|
||||
agree += 1
|
||||
if r1.get("reason_raw") == r2.get("reason_raw"):
|
||||
same_reason += 1
|
||||
print(f"\ndeterminism probe: direction agreement {agree}/{n}, "
|
||||
f"identical reason text {same_reason}/{n}")
|
||||
|
||||
# stratified reason examples
|
||||
print("\n── reason samples (1h hit/miss) ──")
|
||||
sub = df[df[dcol].notna() & (df["c_llm"] != 0)].copy()
|
||||
sub["hit"] = np.sign(sub[dcol]) == sub["c_llm"]
|
||||
for verdict in (True, False):
|
||||
picked, seen = [], set()
|
||||
for _, r in sub[sub["hit"] == verdict].iterrows():
|
||||
if r["name"] in seen:
|
||||
continue
|
||||
seen.add(r["name"])
|
||||
picked.append(r)
|
||||
if len(picked) == 10:
|
||||
break
|
||||
tag = "HIT " if verdict else "MISS"
|
||||
for r in picked:
|
||||
print(f" [{tag}] {r['utc_time'][:16]} {r['name'][:30]:<30} "
|
||||
f"a={r['actual']} f={r['forecast']} → "
|
||||
f"{r['llm_direction']:<7} Δ1h={r[dcol]:+7.2f} "
|
||||
f"“{(r['reason_raw'] or '')[:60]}”")
|
||||
print(f"\nscored table → {SCORED_CSV}")
|
||||
|
||||
|
||||
def report_headlines():
|
||||
recs = _load_jsonl(HEADLINES_JSONL)
|
||||
if not recs:
|
||||
print("\n(no headline replay records — run `headlines` first)")
|
||||
return
|
||||
g = _load_golden()
|
||||
engine = NewsEngine(NewsConfig())
|
||||
|
||||
def collect(tag):
|
||||
m = {}
|
||||
for r in recs:
|
||||
if r["run_tag"] == tag and r["scores"]:
|
||||
for h, sc in zip(r["headlines"], r["scores"]):
|
||||
m[h] = tuple(sc)
|
||||
return m
|
||||
|
||||
main, shuf = collect("main"), collect("shuffled")
|
||||
print("\n" + "═" * 70)
|
||||
print("PART B — golden headline set: LLM vs keyword baseline")
|
||||
print("═" * 70)
|
||||
missing = [h for h in g["headline"] if h not in main]
|
||||
if missing:
|
||||
print(f"WARNING: {len(missing)} headlines missing from main run")
|
||||
|
||||
def cls(x):
|
||||
if x >= NEUTRAL_BAND:
|
||||
return 1
|
||||
if x <= -NEUTRAL_BAND:
|
||||
return -1
|
||||
return 0
|
||||
|
||||
rows = []
|
||||
for _, r in g.iterrows():
|
||||
h = r["headline"]
|
||||
if h not in main:
|
||||
continue
|
||||
llm_xau = main[h][1]
|
||||
kw = engine._analyze_headline_sentiment(h, for_currency="XAU")
|
||||
rows.append({
|
||||
"headline": h, "category": r["category"],
|
||||
"exp": int(r["expected_xau_sign"]),
|
||||
"llm_xau": llm_xau, "llm_cls": cls(llm_xau),
|
||||
"kw": kw, "kw_cls": cls(kw),
|
||||
})
|
||||
t = pd.DataFrame(rows)
|
||||
for who in ("llm", "kw"):
|
||||
acc = (t[f"{who}_cls"] == t["exp"]).mean()
|
||||
print(f"{who.upper():<4} 3-class accuracy (band ±{NEUTRAL_BAND}): "
|
||||
f"{(t[f'{who}_cls'] == t['exp']).sum()}/{len(t)} "
|
||||
f"= {acc * 100:.1f}%")
|
||||
ha = list(t["llm_cls"] == t["exp"])
|
||||
hb = list(t["kw_cls"] == t["exp"])
|
||||
b, c, p = _mcnemar_p(ha, hb)
|
||||
print(f"paired McNemar LLM vs keyword: b={b} c={c} p={p:.3f}")
|
||||
|
||||
print("\nper-category accuracy (LLM | keyword):")
|
||||
for cat, grp in t.groupby("category"):
|
||||
a1 = (grp["llm_cls"] == grp["exp"]).sum()
|
||||
a2 = (grp["kw_cls"] == grp["exp"]).sum()
|
||||
print(f" {cat:<18} {a1:>2}/{len(grp):<2} | {a2:>2}/{len(grp)}")
|
||||
|
||||
print("\nLLM confusion (rows=expected, cols=predicted -1/0/+1):")
|
||||
cm = pd.crosstab(t["exp"], t["llm_cls"]).reindex(
|
||||
index=[-1, 0, 1], columns=[-1, 0, 1], fill_value=0)
|
||||
print(cm.to_string())
|
||||
|
||||
print("\nLLM misclassified headlines:")
|
||||
for _, r in t[t["llm_cls"] != t["exp"]].iterrows():
|
||||
print(f" exp {r['exp']:+d} got {r['llm_xau']:+.2f} "
|
||||
f"[{r['category']}] {r['headline'][:70]}")
|
||||
|
||||
# repeat-batch determinism
|
||||
m_b0 = [r for r in recs if r["run_tag"] == "main" and r["batch"] == 0]
|
||||
r_b0 = [r for r in recs if r["run_tag"] == "repeat"]
|
||||
if m_b0 and r_b0 and m_b0[0]["scores"] and r_b0[0]["scores"]:
|
||||
a = np.array(m_b0[0]["scores"], dtype=float)
|
||||
b_ = np.array(r_b0[0]["scores"], dtype=float)
|
||||
exact = int((a == b_).all(axis=1).sum())
|
||||
print(f"\nrepeat-batch: {exact}/{len(a)} score pairs identical, "
|
||||
f"max |diff| = {np.abs(a - b_).max():.3f}")
|
||||
|
||||
# shuffled-order drift
|
||||
if shuf:
|
||||
both = [h for h in t["headline"] if h in shuf]
|
||||
drift = [abs(main[h][1] - shuf[h][1]) for h in both]
|
||||
if drift:
|
||||
print(f"shuffled-order xau drift over {len(both)} headlines: "
|
||||
f"mean {np.mean(drift):.3f}, max {np.max(drift):.3f}")
|
||||
|
||||
|
||||
def cmd_report(args):
|
||||
report_events()
|
||||
report_headlines()
|
||||
|
||||
|
||||
# ── CLI ──────────────────────────────────────────────────────────────
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[1])
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
e = sub.add_parser("events", help="replay release surprises")
|
||||
e.add_argument("--limit", type=int, default=0,
|
||||
help="smoke-test on first N events")
|
||||
e.add_argument("--determinism", type=int, default=0,
|
||||
help="re-call N stratified events a second time")
|
||||
e.add_argument("--sleep", type=float, default=0.3)
|
||||
e.add_argument("--retry-passes", type=int, default=3)
|
||||
e.add_argument("--timeout", type=float, default=30.0)
|
||||
e.set_defaults(fn=cmd_events)
|
||||
|
||||
h = sub.add_parser("headlines", help="golden-set probes")
|
||||
h.add_argument("--sleep", type=float, default=0.3)
|
||||
h.add_argument("--timeout", type=float, default=30.0)
|
||||
h.set_defaults(fn=cmd_headlines)
|
||||
|
||||
r = sub.add_parser("report", help="offline scoring report")
|
||||
r.set_defaults(fn=cmd_report)
|
||||
|
||||
args = ap.parse_args()
|
||||
args.fn(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,79 +0,0 @@
|
||||
headline,category,expected_xau_sign,rationale
|
||||
"Powell says restrictive policy needed for longer as inflation proves sticky",hawkish_fed,-1,"higher-for-longer rates lift real yields, gold-bearish"
|
||||
"Fed minutes show officials leaning toward two more rate hikes this year",hawkish_fed,-1,"more hikes = stronger dollar/yields"
|
||||
"Hot CPI print fuels bets on additional Fed tightening",hawkish_fed,-1,"tightening expectations weigh on gold"
|
||||
"Fed's Waller: higher-for-longer rates warranted by resilient economy",hawkish_fed,-1,"hawkish guidance"
|
||||
"FOMC raises rates 50bp, flags further increases ahead",hawkish_fed,-1,"outsized hike plus hawkish guidance"
|
||||
"Treasury yields spike as Fed pushes back on rate cut hopes",hawkish_fed,-1,"cut hopes dashed, yields up"
|
||||
"Core inflation accelerates, cementing case for hawkish Fed stance",hawkish_fed,-1,"hawkish repricing"
|
||||
"Fed dot plot shifts higher; markets price out easing for the year",hawkish_fed,-1,"easing priced out"
|
||||
"Fed cuts rates 25bp and signals more easing to come",dovish_fed,1,"easing cycle is gold-bullish"
|
||||
"Powell hints at rate cuts as inflation cools toward target",dovish_fed,1,"cut guidance"
|
||||
"FOMC minutes reveal growing support for policy easing",dovish_fed,1,"dovish minutes"
|
||||
"Fed's Williams says rate cuts appropriate later this year",dovish_fed,1,"cut guidance"
|
||||
"Soft inflation data boosts odds of a dovish Fed pivot",dovish_fed,1,"pivot odds up"
|
||||
"Fed pauses hikes, markets price aggressive easing cycle",dovish_fed,1,"easing priced in"
|
||||
"Central banks turn dovish as global growth falters",dovish_fed,1,"global easing backdrop"
|
||||
"Fed to restart bond purchases to support flagging economy",dovish_fed,1,"QE is gold-bullish"
|
||||
"Dollar index hits 12-month high on robust US economic data",strong_usd,-1,"strong dollar pressures gold"
|
||||
"US GDP grows 4.1% annualized, blowing past expectations",strong_usd,-1,"strong economy, hawkish implication"
|
||||
"Blockbuster jobs report sends dollar surging across the board",strong_usd,-1,"strong jobs lift dollar and yields"
|
||||
"US retail sales smash forecasts, underscoring consumer strength",strong_usd,-1,"strong data"
|
||||
"Greenback rallies as US data outshines Europe and Asia",strong_usd,-1,"dollar strength"
|
||||
"ISM services index jumps to two-year high on strong demand",strong_usd,-1,"strong data"
|
||||
"Consumer confidence surges to post-pandemic high in the US",strong_usd,-1,"strong data"
|
||||
"Dollar tumbles to eight-month low after dismal manufacturing data",weak_usd,1,"weak dollar lifts gold"
|
||||
"US economy adds far fewer jobs than expected; dollar slides",weak_usd,1,"weak jobs, dovish implication"
|
||||
"Greenback weakens as US retail sales unexpectedly contract",weak_usd,1,"weak data"
|
||||
"US GDP growth revised down sharply, stoking slowdown fears",weak_usd,1,"slowdown is dovish"
|
||||
"Dollar slumps as trade deficit balloons to record",weak_usd,1,"dollar weakness"
|
||||
"Soft services PMI adds to evidence of cooling US economy",weak_usd,1,"cooling economy"
|
||||
"US consumer sentiment plunges to two-year low",weak_usd,1,"weak data"
|
||||
"Real yields climb to multi-year highs as TIPS sell off",real_yields,-1,"rising real yields raise gold's opportunity cost"
|
||||
"10-year Treasury yield surges past 5% on supply glut",real_yields,-1,"yield spike"
|
||||
"Rising real interest rates dim appeal of non-yielding assets",real_yields,-1,"explicit opportunity-cost story"
|
||||
"Treasury yields sink as investors flock to bonds",real_yields,1,"falling yields support gold"
|
||||
"Real yields turn negative as inflation outpaces rates",real_yields,1,"negative real yields are gold's best backdrop"
|
||||
"Bond rally drives 10-year yield to lowest level in a year",real_yields,1,"falling yields"
|
||||
"Geopolitical tensions escalate in Middle East, investors seek safety",safe_haven,1,"safe-haven bid"
|
||||
"Stocks crater worldwide as banking crisis fears spread",safe_haven,1,"crisis flight to safety"
|
||||
"Escalating conflict drives flight to safe-haven assets",safe_haven,1,"explicit haven flow"
|
||||
"Investors dump risk assets after surprise sovereign default",safe_haven,1,"default shock"
|
||||
"Military escalation on NATO border rattles global markets",safe_haven,1,"geopolitical stress"
|
||||
"Wall Street plunges 4% in worst session since pandemic",safe_haven,1,"risk-off"
|
||||
"Currency crisis in emerging markets sparks contagion fears",safe_haven,1,"contagion, haven bid"
|
||||
"Nuclear rhetoric intensifies, sending investors to havens",safe_haven,1,"extreme geopolitical stress"
|
||||
"Ceasefire agreement lifts global markets, haven demand fades",risk_on,-1,"haven unwind"
|
||||
"Stocks roar to record highs as trade deal signed",risk_on,-1,"risk-on drains haven demand"
|
||||
"Risk appetite returns as banking fears subside",risk_on,-1,"risk-on"
|
||||
"Global equities rally on strong earnings and easing tensions",risk_on,-1,"risk-on"
|
||||
"Gold hits record high as central banks keep buying bullion",gold_direct,1,"direct demand"
|
||||
"Gold ETFs post biggest weekly inflows in three years",gold_direct,1,"investment inflows"
|
||||
"China's central bank extends gold buying spree to 18th month",gold_direct,1,"official-sector demand"
|
||||
"Miners struggle to meet gold demand as supply deficit widens",gold_direct,1,"supply deficit"
|
||||
"Gold slides 2% as investors book profits after record run",gold_direct,-1,"profit-taking pressure"
|
||||
"Gold tumbles below key support as ETF outflows accelerate",gold_direct,-1,"outflows"
|
||||
"Bullion demand weakens as Indian jewellery buying stalls",gold_direct,-1,"physical demand soft"
|
||||
"Speculators slash bullish gold bets as dollar strengthens",gold_direct,-1,"positioning unwind"
|
||||
"Fed holds rates steady, signals no rate hike on the horizon",negation_traps,1,"dovish hold; keyword layer sees the phrase rate hike and flips it gold-bearish"
|
||||
"Powell says no rate cuts until inflation shows sustained cooling",negation_traps,-1,"hawkish hold; keyword layer sees rate cut and calls it gold-bullish"
|
||||
"Recession fears fade as US data streak beats estimates",negation_traps,-1,"fading fear is risk-on; keywords see recession plus beats and cancel out"
|
||||
"Inflation fails to accelerate, easing pressure on the Fed",negation_traps,1,"disinflation opens door to cuts"
|
||||
"Fed's Bowman: not yet confident inflation is beaten, cuts premature",negation_traps,-1,"hawkish pushback phrased around negations"
|
||||
"Central bank stops short of hiking, but warns tightening may resume",negation_traps,-1,"hawkish hold with tightening bias"
|
||||
"No safe-haven bid for gold despite equity wobble, analysts say",negation_traps,-1,"explicitly negated haven demand"
|
||||
"Jobs report not as weak as feared, dollar recovers",negation_traps,-1,"less-bad data is dollar-positive"
|
||||
"US manufacturing output plunges as new orders dry up",inverted_metrics,1,"weak economy is dovish/gold-bullish; keyword layer reads plunges as bearish"
|
||||
"US factory activity slumps to three-year low",inverted_metrics,1,"weak data; keyword reads slumps as bearish"
|
||||
"Consumer spending falls for third straight month",inverted_metrics,1,"weak data; keyword reads falls as bearish"
|
||||
"Private payrolls growth slides to weakest pace since 2020",inverted_metrics,1,"weak jobs; keyword reads slides as bearish"
|
||||
"Jobless claims hit two-year high as layoffs spread",inverted_metrics,1,"higher claims = weaker economy = gold-bullish"
|
||||
"US unemployment rate hits 4.5%, highest since 2021",inverted_metrics,1,"higher unemployment = dovish = gold-bullish"
|
||||
"Lakers clinch playoff berth with overtime win over Suns",distractors,0,"irrelevant"
|
||||
"New Marvel film smashes box office records on opening weekend",distractors,0,"irrelevant"
|
||||
"Apple unveils thinner MacBook with faster chip lineup",distractors,0,"irrelevant"
|
||||
"Scientists discover new deep-sea species off Pacific coast",distractors,0,"irrelevant"
|
||||
"Heavy rain forecast for the Midwest through the weekend",distractors,0,"irrelevant"
|
||||
"Streaming service raises subscription prices in Europe",distractors,0,"irrelevant"
|
||||
"Electric vehicle startup recalls 12,000 cars over software glitch",distractors,0,"irrelevant"
|
||||
"Local elections see record turnout in several US states",distractors,0,"irrelevant"
|
||||
|
@@ -1,150 +0,0 @@
|
||||
"""What do violent bars do to OPEN trades, and can we protect them? Analysis tooling only.
|
||||
|
||||
Replays every trade of a book on the raw bars with the backtester's exit model (SL-first,
|
||||
gap-through at open, close-only trail ratchet, 300-bar timeout, $0.65 cost) and compares:
|
||||
base : reproduce the book (validation: must match r_multiple for almost every trade)
|
||||
flat_data : exit at the OPEN of the first bar whose close-hour enters {12,13} UTC (US data)
|
||||
flat_asia : same for close-hour {0,1} UTC (Asian open)
|
||||
tight_data : entering {12,13}, hold a protective stop 0.5R below the best price for the window
|
||||
tight_asia : same for {0,1}
|
||||
react_exit : a burst bar (range >= 3x prior ATR14) closes AGAINST the trade -> exit next open
|
||||
react_tight : after an opposing burst, protective stop 0.5R below the best price for the rest
|
||||
Plus a damage table: what happens to a trade when the first burst hits it (with vs against).
|
||||
|
||||
python backtest/research/open_trade_bursts.py BOOK.csv PARQUET BAR_MINUTES ARM DIST
|
||||
"""
|
||||
import sys
|
||||
import numpy as np, pandas as pd
|
||||
|
||||
COST = 0.65; MAX_BARS = 300; K = 3.0; PROT = 0.5
|
||||
|
||||
|
||||
def wilder_atr(df, period=14):
|
||||
pc = df["Close"].shift(1)
|
||||
tr = pd.concat([df["High"] - df["Low"], (df["High"] - pc).abs(), (df["Low"] - pc).abs()], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1 / period, min_periods=period, adjust=False).mean()
|
||||
|
||||
|
||||
def simulate(bars, i0, direction, entry, sl, arm, dist, variant, hours_flat=(), hours_tight=(), flat_mask=None, tight_mask=None):
|
||||
"""bars: dict of numpy arrays; i0 = index of the entry bar. Returns (exit_price, bars_held, reason, flag)."""
|
||||
O, H, L, C, burst, chour = bars["O"], bars["H"], bars["L"], bars["C"], bars["burst"], bars["chour"]
|
||||
d = 1.0 if direction == "BUY" else -1.0
|
||||
risk = abs(entry - sl); stop = sl; peak = entry; flag = 0
|
||||
n = len(O); tight = False; tight_until = -1; exit_next = False
|
||||
for j in range(i0, min(n, i0 + MAX_BARS)):
|
||||
if exit_next:
|
||||
return O[j], j - i0, "REACT", flag
|
||||
h = chour[j]
|
||||
in_flat = h in hours_flat; in_tight = h in hours_tight
|
||||
if variant.startswith("flat") and in_flat:
|
||||
return O[j], j - i0, "FLAT", 1
|
||||
if variant == "mask_flat" and flat_mask is not None and flat_mask[j]:
|
||||
return O[j], j - i0, "FLAT", 1
|
||||
if variant.startswith("tight") and in_tight:
|
||||
tight = True; tight_until = j; flag = 1
|
||||
if variant == "mask_tight" and tight_mask is not None and tight_mask[j]:
|
||||
tight = True; tight_until = j; flag = 1
|
||||
# stop check (SL first, gap-through at open)
|
||||
if d > 0:
|
||||
if O[j] <= stop: return O[j], j - i0, "SL", flag
|
||||
if L[j] <= stop: return stop, j - i0, "SL", flag
|
||||
else:
|
||||
if O[j] >= stop: return O[j], j - i0, "SL", flag
|
||||
if H[j] >= stop: return stop, j - i0, "SL", flag
|
||||
# update peak / trail at bar close
|
||||
peak = max(peak, H[j]) if d > 0 else min(peak, L[j])
|
||||
fav = d * (peak - entry)
|
||||
if fav >= arm * risk:
|
||||
stop = max(stop, peak - dist * risk) if d > 0 else min(stop, peak + dist * risk)
|
||||
if tight and (variant == "react_tight" or j <= tight_until or in_tight):
|
||||
stop = max(stop, peak - PROT * risk) if d > 0 else min(stop, peak + PROT * risk)
|
||||
if variant.startswith("react") and burst[j] and d * (C[j] - O[j]) < 0:
|
||||
flag = 1
|
||||
if variant == "react_exit": exit_next = True
|
||||
else: tight = True
|
||||
j = min(n - 1, i0 + MAX_BARS - 1)
|
||||
return C[j], j - i0, "TIMEOUT", flag
|
||||
|
||||
|
||||
def r_of(direction, entry, sl, exit_price):
|
||||
d = 1.0 if direction == "BUY" else -1.0
|
||||
return (d * (exit_price - entry) - COST) / abs(entry - sl)
|
||||
|
||||
|
||||
def stats(r):
|
||||
r = np.asarray(r); eq = np.concatenate([[0.0], np.cumsum(r)]); dd = (np.maximum.accumulate(eq) - eq).max()
|
||||
pf = r[r > 0].sum() / max(-r[r < 0].sum(), 1e-9)
|
||||
return dict(n=len(r), total=r.sum(), mean=r.mean(), wr=100 * (r > 0).mean(), pf=pf, dd=dd)
|
||||
|
||||
|
||||
def main():
|
||||
book_csv, pq, bm, arm, dist = sys.argv[1], sys.argv[2], int(sys.argv[3]), float(sys.argv[4]), float(sys.argv[5])
|
||||
book = pd.read_csv(book_csv); book["entry_time"] = pd.to_datetime(book["entry_time"], utc=True, format="mixed")
|
||||
px = pd.read_parquet(pq)
|
||||
atr_prev = wilder_atr(px).shift(1)
|
||||
bars = dict(O=px["Open"].values, H=px["High"].values, L=px["Low"].values, C=px["Close"].values,
|
||||
burst=((px["High"] - px["Low"]) >= K * atr_prev).fillna(False).values,
|
||||
chour=(px.index + pd.Timedelta(minutes=bm)).hour.values)
|
||||
idx = pd.Index(px.index)
|
||||
pos = idx.get_indexer(book["entry_time"])
|
||||
ok = pos >= 0
|
||||
print(f"{book_csv}: {len(book)} trades, {ok.sum()} matched to bars; arm {arm} dist {dist}")
|
||||
book = book[ok].reset_index(drop=True); pos = pos[ok]
|
||||
half = np.where(np.arange(len(book)) < len(book) // 2, 1, 2)
|
||||
variants = {"base": {}, "flat_data": dict(hours_flat=(12, 13)), "flat_asia": dict(hours_flat=(0, 1)),
|
||||
"tight_data": dict(hours_tight=(12, 13)), "tight_asia": dict(hours_tight=(0, 1)),
|
||||
"react_exit": {}, "react_tight": {}}
|
||||
results = {}
|
||||
for v, kw in variants.items():
|
||||
rs, flags = [], []
|
||||
for i, t in book.iterrows():
|
||||
xp, held, reason, flag = simulate(bars, pos[i], t.direction, t.entry, t.sl, arm, dist, v, **kw)
|
||||
rs.append(r_of(t.direction, t.entry, t.sl, xp)); flags.append(flag)
|
||||
results[v] = (np.array(rs), np.array(flags))
|
||||
base = results["base"][0]
|
||||
match = np.isclose(base, book["r_multiple"].values, atol=0.02).mean()
|
||||
print(f"validation: replay reproduces the book's r_multiple (±0.02) on {100*match:.1f}% of trades; "
|
||||
f"sum replay {base.sum():+.2f} vs book {book.r_multiple.sum():+.2f}")
|
||||
print("\n== PROTECTION VARIANTS (whole book; 'affected' = trades where the rule fired) ==")
|
||||
print(f"{'variant':12s} {'n':>5s} {'affected':>8s} {'total R':>9s} {'Δ vs base':>9s} {'PF':>5s} {'maxDD':>6s} {'WR%':>5s} | affected only: base R -> variant R | Δ h1 / Δ h2")
|
||||
b = stats(base)
|
||||
for v, (r, f) in results.items():
|
||||
s = stats(r); aff = f == 1
|
||||
d1 = r[half == 1].sum() - base[half == 1].sum(); d2 = r[half == 2].sum() - base[half == 2].sum()
|
||||
print(f"{v:12s} {s['n']:5d} {aff.sum():8d} {s['total']:+9.2f} {s['total']-b['total']:+9.2f} {s['pf']:5.2f} {s['dd']:6.2f} {s['wr']:5.1f} | "
|
||||
f"{base[aff].sum():+8.2f} -> {r[aff].sum():+8.2f} (n={aff.sum()}) | {d1:+.2f} / {d2:+.2f}")
|
||||
# damage table: first burst while open (base path), with vs against
|
||||
print("\n== DAMAGE: first violent bar that hits an OPEN trade (base exits) ==")
|
||||
rows = []
|
||||
for i, t in book.iterrows():
|
||||
d = 1.0 if t.direction == "BUY" else -1.0; risk = abs(t.entry - t.sl)
|
||||
xp, held, reason, _ = simulate(bars, pos[i], t.direction, t.entry, t.sl, arm, dist, "base")
|
||||
j0 = pos[i]; j1 = j0 + held
|
||||
for j in range(j0 + 1, j1 + 1): # bursts after the entry bar, up to and including the exit bar
|
||||
if bars["burst"][j]:
|
||||
against = d * (bars["C"][j] - bars["O"][j]) < 0
|
||||
mtm_before = (d * (bars["C"][j - 1] - t.entry) - COST) / risk
|
||||
mtm_after = (d * (bars["C"][j] - t.entry) - COST) / risk
|
||||
rows.append(dict(against=against, bars_open=j - j0, mtm_before=mtm_before, mtm_after=mtm_after,
|
||||
final=r_of(t.direction, t.entry, t.sl, xp), ended_here=(j == j1), hour=bars["chour"][j],
|
||||
sched=bars["chour"][j] in (0, 1, 12, 13)))
|
||||
break
|
||||
dmg = pd.DataFrame(rows)
|
||||
if len(dmg):
|
||||
g = dmg.groupby("against").agg(n=("final", "size"), bars_open_med=("bars_open", "median"),
|
||||
mtm_before=("mtm_before", "mean"), mtm_after=("mtm_after", "mean"),
|
||||
final=("final", "mean"), ended_on_that_bar_pct=("ended_here", lambda s: 100 * s.mean()),
|
||||
in_sched_hours_pct=("sched", lambda s: 100 * s.mean()))
|
||||
print(g.round(3).to_string())
|
||||
ag = dmg[dmg.against]
|
||||
print(f"\nopposing bursts: {len(ag)} trades ({100*len(ag)/len(book):.1f}% of the book); "
|
||||
f"mean R just before {ag.mtm_before.mean():+.3f} -> just after {ag.mtm_after.mean():+.3f} -> final {ag.final.mean():+.3f}; "
|
||||
f"{100*(ag.final<0).mean():.0f}% end as losers; {100*ag.ended_here.mean():.0f}% are stopped ON that bar")
|
||||
print(f" those with an open PROFIT before the burst (mtm>0): n={int((ag.mtm_before>0).sum())}, final mean {ag[ag.mtm_before>0].final.mean():+.3f}, "
|
||||
f"{100*(ag[ag.mtm_before>0].final<0).mean():.0f}% end as losers")
|
||||
print(f" by hour of the opposing burst (n, mean final R): " + ", ".join(f"{h:02d}:{n}/{m:+.2f}" for h, n, m in
|
||||
ag.groupby("hour").final.agg(["size", "mean"]).reset_index().itertuples(index=False)))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,7 +0,0 @@
|
||||
{"probe": "P1 coherent-bullish", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_up", "volatility_regime": "normal", "mtf_agreement": true}, "indicators": {"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0}, "rsi": {"signal": "BUY", "confidence": 65, "value": 58.0}, "macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25}, "bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7}, "adx": {"signal": "BUY", "confidence": 70, "value": 32.0}, "roc": {"signal": "BUY", "confidence": 55, "value": 0.35}, "keltner": {"signal": "BUY", "confidence": 65, "value": 3.1}, "candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0}}, "mtf": {"5min": {"signal": "BUY", "score": 30.0}, "1h": {"signal": "STRONG_BUY", "score": 45.0}}, "key_levels": {"resistance": 2012.0, "support": 1985.0}, "exhaustion": 0, "candles": {"15min": {"ref_close": 2000.0, "atr": 5.0, "bars": [[-2.4, -2.2, -2.4, -2.3], [-2.3, -2.1, -2.3, -2.2], [-2.2, -2.0, -2.2, -2.0], [-2.0, -1.9, -2.1, -1.9], [-1.9, -1.8, -2.0, -1.8], [-1.8, -1.6, -1.8, -1.7], [-1.7, -1.5, -1.7, -1.6], [-1.6, -1.4, -1.6, -1.4], [-1.4, -1.3, -1.5, -1.3], [-1.3, -1.2, -1.4, -1.2], [-1.2, -1.0, -1.2, -1.1], [-1.1, -0.9, -1.1, -1.0], [-1.0, -0.8, -1.0, -0.8], [-0.8, -0.7, -0.9, -0.7], [-0.7, -0.6, -0.8, -0.6], [-0.6, -0.4, -0.6, -0.5], [-0.5, -0.3, -0.5, -0.4], [-0.4, -0.2, -0.4, -0.2], [-0.2, -0.1, -0.3, -0.1], [-0.1, 0.0, -0.2, 0.0]]}}}, "raw_reply": {"stance": "support", "confidence": 70, "note": "The trend is up and all signs agree, so buying fits the flow."}}
|
||||
{"probe": "P2 bearish-vs-BUY-brief", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_down", "volatility_regime": "normal", "mtf_agreement": false}, "indicators": {"ema_trend": {"signal": "STRONG_SELL", "confidence": 85, "value": 0.1}, "rsi": {"signal": "SELL", "confidence": 70, "value": 74.0}, "macd": {"signal": "STRONG_SELL", "confidence": 80, "value": -1.4}, "bollinger": {"signal": "SELL", "confidence": 60, "value": 0.95}, "adx": {"signal": "SELL", "confidence": 70, "value": 35.0}, "roc": {"signal": "SELL", "confidence": 60, "value": -0.5}, "keltner": {"signal": "SELL", "confidence": 65, "value": -2.8}, "candlestick": {"signal": "SELL", "confidence": 55, "value": -2.0}}, "mtf": {"5min": {"signal": "SELL", "score": -28.0}, "1h": {"signal": "STRONG_SELL", "score": -41.0}}, "key_levels": {"resistance": 2012.0, "support": 1985.0}, "exhaustion": 3, "candles": {"15min": {"ref_close": 2000.0, "atr": 5.0, "bars": [[2.4, 2.4, 2.2, 2.3], [2.3, 2.3, 2.1, 2.2], [2.2, 2.2, 2.0, 2.0], [2.0, 2.1, 1.9, 1.9], [1.9, 2.0, 1.8, 1.8], [1.8, 1.8, 1.6, 1.7], [1.7, 1.7, 1.5, 1.6], [1.6, 1.6, 1.4, 1.4], [1.4, 1.5, 1.3, 1.3], [1.3, 1.4, 1.2, 1.2], [1.2, 1.2, 1.0, 1.1], [1.1, 1.1, 0.9, 1.0], [1.0, 1.0, 0.8, 0.8], [0.8, 0.9, 0.7, 0.7], [0.7, 0.8, 0.6, 0.6], [0.6, 0.6, 0.4, 0.5], [0.5, 0.5, 0.3, 0.4], [0.4, 0.4, 0.2, 0.2], [0.2, 0.3, 0.1, 0.1], [0.1, 0.2, -0.0, 0.0]]}}}, "raw_reply": {"stance": "oppose", "confidence": 85, "note": "The price is falling and every indicator says sell, so buying fights the trend."}}
|
||||
{"probe": "P3 conflicting", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 12.0, "confidence": 35, "price": 2000.0, "market_regime": "ranging", "volatility_regime": "normal", "mtf_agreement": false}, "indicators": {"ema_trend": {"signal": "BUY", "confidence": 55, "value": 0.6}, "rsi": {"signal": "NEUTRAL", "confidence": 40, "value": 51.0}, "macd": {"signal": "SELL", "confidence": 55, "value": -0.3}, "bollinger": {"signal": "NEUTRAL", "confidence": 35, "value": 0.5}, "adx": {"signal": "NEUTRAL", "confidence": 40, "value": 18.0}, "roc": {"signal": "SELL", "confidence": 50, "value": -0.1}, "keltner": {"signal": "BUY", "confidence": 50, "value": 0.8}, "candlestick": {"signal": "NEUTRAL", "confidence": 30, "value": 0.0}}, "mtf": {"5min": {"signal": "NEUTRAL", "score": 4.0}, "1h": {"signal": "NEUTRAL", "score": -3.0}}, "key_levels": {}, "exhaustion": 0, "candles": {"15min": {"ref_close": 2000.0, "atr": 5.0, "bars": [[0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0]]}}}, "raw_reply": {"stance": "neutral", "confidence": 30, "note": "Mixed signals with no clear direction, so cannot strongly support or oppose."}}
|
||||
{"probe": "P4 empty-evidence", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_up", "volatility_regime": "normal", "mtf_agreement": true}, "indicators": {}, "mtf": {}, "key_levels": {}, "exhaustion": 0}, "raw_reply": {"stance": "neutral", "confidence": 20, "note": "Not enough data to judge the trade direction."}}
|
||||
{"probe": "P5 hallucination-trap", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_up", "volatility_regime": "normal", "mtf_agreement": true}, "indicators": {"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0}, "rsi": {"signal": "BUY", "confidence": 65, "value": 58.0}, "macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25}, "bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7}, "adx": {"signal": "BUY", "confidence": 70, "value": 32.0}, "roc": {"signal": "BUY", "confidence": 55, "value": 0.35}, "keltner": {"signal": "BUY", "confidence": 65, "value": 3.1}, "candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0}}, "mtf": {"5min": {"signal": "BUY", "score": 30.0}, "1h": {"signal": "STRONG_BUY", "score": 45.0}}, "key_levels": {}, "exhaustion": 0}, "raw_reply": {"stance": "support", "confidence": 70, "note": "The trend is up and all signs agree, so buying fits the flow."}}
|
||||
{"probe": "P6 output-discipline", "payload": {}, "raw_reply": null}
|
||||
{"probe": "P7 synthesizer", "payload": {"direction": "BUY", "takes": [{"persona": "macro", "stance": "support", "confidence": 70, "note": "evidence aligned"}, {"persona": "technical", "stance": "support", "confidence": 80, "note": "evidence aligned"}, {"persona": "risk", "stance": "support", "confidence": 65, "note": "evidence aligned"}, {"persona": "contrarian", "stance": "support", "confidence": 60, "note": "evidence aligned"}]}, "raw_reply": {"verdict": "AGREE", "confidence": 85, "rationale": "All four analysts support buying, with high confidence and consistent evidence. No one opposes, so the panel clearly backs the trade."}}
|
||||
@@ -1,7 +1,7 @@
|
||||
# Signal-Level Replay Parity — Oanda vs Tiingo (Phase 2)
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Tool:** `scripts/signal_parity_study.py` (replays real `analyze_from_dataframes`, news off)
|
||||
**Tool:** `scripts/signal_parity_study.py` (script removed 2026-09-06, in git history) (replays real `analyze_from_dataframes`, news off)
|
||||
**Window:** 2025-08-14 → 2026-08-14 (12 months); M15 stride 8, M5 stride 24 → ~2,900 sampled decisions each
|
||||
**Preceding:** Phase 1 price-diff study (`feed_divergence_2026-08-14.md`)
|
||||
|
||||
|
||||
@@ -1,961 +0,0 @@
|
||||
"""Stage-D stake-sizing model driver (2026-09 campaign).
|
||||
|
||||
Walk-forward fit + evaluation of a per-signal stake multiplier on a trade
|
||||
book exported by ``gpu_backtest.py --export-features`` (63 base columns +
|
||||
``stake_multiplier``/``r_sized`` + 65 ``f_*`` decision-time features).
|
||||
|
||||
The protocol implemented here is the one pre-registered in
|
||||
``backtest/research/sizing_campaign_2026-09.md`` and is NOT to be varied to
|
||||
rescue a result:
|
||||
|
||||
* expanding walk-forward, 12-month burn-in, calendar-quarter refits, a
|
||||
``purge_bars × bar_minutes`` gap between the end of train and the start of
|
||||
the test quarter, standardisation fit on train only;
|
||||
* models: ridge on winsorised R (primary), L2 logistic on ``R > 0``
|
||||
(secondary), a one-feature ``f_is_buy`` simplicity baseline, an
|
||||
exhaustion-block-only ablation, and an optional lightgbm upper bound;
|
||||
* size map ``m = round_0.05(clip(normaliser × (0.5 + clip((z−lo)/(hi−lo),0,1)),
|
||||
0.5, 1.5))`` with ``lo``/``hi``/``normaliser`` calibrated on TRAIN
|
||||
predictions only;
|
||||
* a label-shuffle null (training labels permuted, test labels untouched);
|
||||
* six acceptance clauses, printed PASS/FAIL individually.
|
||||
|
||||
Usage:
|
||||
python backtest/research/sizing_model.py --book BOOK.csv --bar-minutes 5 \
|
||||
--features f_a,f_b,... [--model ridge|logistic] [--out DIR] \
|
||||
[--n-shuffle 50] [--skip-lgbm] \
|
||||
[--write-artifact PATH --tf 5min --confirm-tfs 15min,1h]
|
||||
|
||||
Analysis tooling only — never imported by the live code.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import subprocess
|
||||
import sys
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Sequence, Tuple
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(PROJECT_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from position_sizing import FEATURE_ORDER, FEATURE_VERSION # noqa: E402 (read-only import)
|
||||
|
||||
# ── Pre-registered constants (flag any change loudly) ──────────────────
|
||||
WINSOR_LO, WINSOR_HI = -1.5, 4.0
|
||||
LAMBDA_GRID: Tuple[float, ...] = (1.0, 10.0, 100.0, 1000.0)
|
||||
CV_FOLDS = 5
|
||||
MIN_TRAIN = 150
|
||||
MAP_Q_LO, MAP_Q_HI = 0.10, 0.90
|
||||
MAP_OUT_LO, MAP_OUT_HI = 0.5, 1.5
|
||||
MAP_CLIP_LO, MAP_CLIP_HI = 0.5, 1.5
|
||||
MAP_ROUND_TO = 0.05
|
||||
MEAN_MULT_TOL = 0.05
|
||||
DD_TOL = 1.10
|
||||
PF_FLOOR = 1.3
|
||||
NEWTON_MAX_ITER = 50
|
||||
NEWTON_TOL = 1e-8
|
||||
|
||||
#: The exhaustion-block ablation's fixed feature list.
|
||||
EXHAUSTION_FEATURES: Tuple[str, ...] = (
|
||||
"f_swing_dist_atr",
|
||||
"f_ema21_dist_atr",
|
||||
"f_ema200_dist_atr",
|
||||
"f_exhaustion_count",
|
||||
"f_penalty_total",
|
||||
"f_tag_bb_overextended",
|
||||
"f_tag_rsi_extended",
|
||||
"f_tag_no_pullback",
|
||||
"f_tag_exhaustion",
|
||||
)
|
||||
ISBUY_FEATURES: Tuple[str, ...] = ("f_is_buy",)
|
||||
|
||||
#: lightgbm never ships; running the full shuffle null through 200-tree fits
|
||||
#: costs minutes for a number nobody acts on, so its null is capped here.
|
||||
#: DEVIATION FROM THE PROTOCOL CONSTANT — printed in the summary when used.
|
||||
LGBM_MAX_SHUFFLE = 10
|
||||
|
||||
LGBM_PARAMS = dict(
|
||||
n_estimators=200,
|
||||
learning_rate=0.03,
|
||||
num_leaves=7,
|
||||
min_child_samples=30,
|
||||
subsample=0.8,
|
||||
colsample_bytree=0.8,
|
||||
reg_lambda=10,
|
||||
)
|
||||
|
||||
_LINES: List[str] = []
|
||||
|
||||
|
||||
def say(msg: str = "") -> None:
|
||||
"""Print and tee into the summary buffer."""
|
||||
print(msg)
|
||||
_LINES.append(msg)
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Book loading
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def load_book(path: Path) -> pd.DataFrame:
|
||||
"""Read a feature book, parse ``entry_time`` UTC, drop NaN-R rows."""
|
||||
df = pd.read_csv(path)
|
||||
if "entry_time" not in df.columns or "r_multiple" not in df.columns:
|
||||
raise SystemExit(f"{path}: not a trade book (no entry_time/r_multiple)")
|
||||
df["entry_time"] = pd.to_datetime(df["entry_time"], utc=True, format="mixed")
|
||||
n0 = len(df)
|
||||
bad = ~np.isfinite(pd.to_numeric(df["r_multiple"], errors="coerce").to_numpy(float))
|
||||
if bad.any():
|
||||
say(f"WARNING: dropping {int(bad.sum())}/{n0} rows with non-finite r_multiple")
|
||||
df = df.loc[~bad].copy()
|
||||
df["r_multiple"] = df["r_multiple"].astype(float)
|
||||
df = df.sort_values("entry_time", kind="mergesort").reset_index(drop=True)
|
||||
return df
|
||||
|
||||
|
||||
def resolve_features(df: pd.DataFrame, names: Sequence[str], label: str) -> List[str]:
|
||||
"""Validate a requested feature list against the book and FEATURE_ORDER."""
|
||||
missing = [n for n in names if n not in df.columns]
|
||||
if missing:
|
||||
raise SystemExit(f"{label}: columns absent from book: {', '.join(missing)}")
|
||||
unknown = [n for n in names if n.startswith("f_") and n not in FEATURE_ORDER]
|
||||
if unknown:
|
||||
say(f"NOTE: {label}: {', '.join(unknown)} not in position_sizing.FEATURE_ORDER")
|
||||
return list(names)
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
h = hashlib.sha256()
|
||||
with open(path, "rb") as fh:
|
||||
for chunk in iter(lambda: fh.read(1 << 20), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def git_commit() -> str:
|
||||
try:
|
||||
return subprocess.check_output(
|
||||
["git", "rev-parse", "HEAD"], cwd=PROJECT_ROOT, text=True).strip()
|
||||
except Exception: # pragma: no cover - research tool
|
||||
return "unknown"
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Walk-forward geometry
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def quarter_starts(first: pd.Timestamp, last: pd.Timestamp) -> List[pd.Timestamp]:
|
||||
"""Every Jan/Apr/Jul/Oct 1 00:00 UTC in ``[first, last]`` inclusive."""
|
||||
out: List[pd.Timestamp] = []
|
||||
year = first.year
|
||||
while True:
|
||||
for month in (1, 4, 7, 10):
|
||||
ts = pd.Timestamp(year=year, month=month, day=1, tz="UTC")
|
||||
if ts < first:
|
||||
continue
|
||||
if ts > last:
|
||||
return out
|
||||
out.append(ts)
|
||||
year += 1
|
||||
|
||||
|
||||
def quarter_label(ts: pd.Timestamp) -> str:
|
||||
return f"{ts.year}Q{(ts.month - 1) // 3 + 1}"
|
||||
|
||||
|
||||
def build_folds(times: np.ndarray, n_folds: int) -> List[np.ndarray]:
|
||||
"""Contiguous-by-time index blocks (rows are already time-sorted)."""
|
||||
n = len(times)
|
||||
edges = np.linspace(0, n, n_folds + 1).astype(int)
|
||||
return [np.arange(edges[i], edges[i + 1]) for i in range(n_folds)
|
||||
if edges[i + 1] > edges[i]]
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Standardisation
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def fit_standardiser(X: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
|
||||
"""Return (mean, scale, keep) from a training matrix.
|
||||
|
||||
NaNs are ignored when computing the mean (they are later filled with it).
|
||||
``scale`` is the population std with zeros replaced by 1.0; ``keep`` is
|
||||
False for zero-variance-in-train columns, which are dropped from the fit
|
||||
and carry a coefficient of 0.
|
||||
"""
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", category=RuntimeWarning)
|
||||
mean = np.nanmean(X, axis=0)
|
||||
std = np.nanstd(X, axis=0)
|
||||
mean = np.where(np.isfinite(mean), mean, 0.0)
|
||||
std = np.where(np.isfinite(std), std, 0.0)
|
||||
keep = std > 0
|
||||
scale = np.where(keep, std, 1.0)
|
||||
return mean, scale, keep
|
||||
|
||||
|
||||
def apply_standardiser(X: np.ndarray, mean: np.ndarray, scale: np.ndarray) -> np.ndarray:
|
||||
"""Fill NaN with the TRAIN mean, then standardise."""
|
||||
Z = np.array(X, dtype=float, copy=True)
|
||||
bad = ~np.isfinite(Z)
|
||||
if bad.any():
|
||||
Z[bad] = np.take(mean, np.where(bad)[1])
|
||||
return (Z - mean) / scale
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Models (numpy only)
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def _solve(A: np.ndarray, b: np.ndarray) -> np.ndarray:
|
||||
try:
|
||||
return np.linalg.solve(A, b)
|
||||
except np.linalg.LinAlgError: # pragma: no cover - degenerate design
|
||||
return np.linalg.lstsq(A, b, rcond=None)[0]
|
||||
|
||||
|
||||
def fit_ridge(X: np.ndarray, y: np.ndarray, lam: float) -> Tuple[np.ndarray, float]:
|
||||
"""Closed-form ridge with an UNPENALISED intercept.
|
||||
|
||||
Centring X and y is exactly equivalent to appending a 1-column carrying
|
||||
zero penalty. Returns ``(coef, intercept)`` with ``pred = intercept + X @ coef``.
|
||||
"""
|
||||
if X.shape[1] == 0:
|
||||
return np.zeros(0), float(y.mean()) if len(y) else 0.0
|
||||
xm = X.mean(axis=0)
|
||||
ym = float(y.mean())
|
||||
Xc = X - xm
|
||||
w = _solve(Xc.T @ Xc + lam * np.eye(X.shape[1]), Xc.T @ (y - ym))
|
||||
return w, ym - float(xm @ w)
|
||||
|
||||
|
||||
def fit_logistic(X: np.ndarray, y: np.ndarray, lam: float,
|
||||
max_iter: int = NEWTON_MAX_ITER,
|
||||
tol: float = NEWTON_TOL) -> Tuple[np.ndarray, float, float]:
|
||||
"""L2 logistic via Newton-Raphson, intercept unpenalised.
|
||||
|
||||
Returns ``(coef, intercept, grad_norm)`` where ``grad_norm`` is the
|
||||
infinity-norm of the penalised gradient at the returned point.
|
||||
"""
|
||||
n, p = X.shape
|
||||
A = np.hstack([np.ones((n, 1)), X])
|
||||
pen = np.zeros(p + 1)
|
||||
pen[1:] = lam
|
||||
beta = np.zeros(p + 1)
|
||||
# Warm-start the intercept at the empirical log-odds.
|
||||
frac = float(np.clip(y.mean(), 1e-6, 1 - 1e-6))
|
||||
beta[0] = math.log(frac / (1 - frac))
|
||||
grad = np.zeros(p + 1)
|
||||
for _ in range(max_iter):
|
||||
eta = A @ beta
|
||||
prob = 1.0 / (1.0 + np.exp(-np.clip(eta, -35, 35)))
|
||||
grad = A.T @ (y - prob) - pen * beta
|
||||
W = np.clip(prob * (1 - prob), 1e-10, None)
|
||||
H = A.T @ (A * W[:, None]) + np.diag(pen)
|
||||
step = _solve(H, grad)
|
||||
beta = beta + step
|
||||
if np.max(np.abs(step)) < tol:
|
||||
break
|
||||
eta = A @ beta
|
||||
prob = 1.0 / (1.0 + np.exp(-np.clip(eta, -35, 35)))
|
||||
grad = A.T @ (y - prob) - pen * beta
|
||||
return beta[1:], float(beta[0]), float(np.max(np.abs(grad)))
|
||||
|
||||
|
||||
def predict_linear(X: np.ndarray, coef: np.ndarray, intercept: float) -> np.ndarray:
|
||||
if X.shape[1] == 0:
|
||||
return np.full(len(X), intercept, dtype=float)
|
||||
return intercept + X @ coef
|
||||
|
||||
|
||||
def sigmoid(z: np.ndarray) -> np.ndarray:
|
||||
return 1.0 / (1.0 + np.exp(-np.clip(z, -35, 35)))
|
||||
|
||||
|
||||
def log_loss(y: np.ndarray, p: np.ndarray) -> float:
|
||||
p = np.clip(p, 1e-12, 1 - 1e-12)
|
||||
return float(-np.mean(y * np.log(p) + (1 - y) * np.log(1 - p)))
|
||||
|
||||
|
||||
def choose_lambda(X: np.ndarray, y: np.ndarray, kind: str,
|
||||
grid: Sequence[float] = LAMBDA_GRID,
|
||||
n_folds: int = CV_FOLDS) -> Tuple[float, Dict[float, float]]:
|
||||
"""5-fold time-contiguous CV inside the training window."""
|
||||
folds = build_folds(np.arange(len(y)), n_folds)
|
||||
if X.shape[1] == 0 or len(folds) < 2:
|
||||
return float(grid[0]), {}
|
||||
scores: Dict[float, float] = {}
|
||||
for lam in grid:
|
||||
vals: List[float] = []
|
||||
for hold in folds:
|
||||
mask = np.ones(len(y), dtype=bool)
|
||||
mask[hold] = False
|
||||
if mask.sum() < 2:
|
||||
continue
|
||||
if kind == "logistic":
|
||||
ytr = y[mask]
|
||||
if ytr.min() == ytr.max(): # single-class fold
|
||||
p = np.full(len(hold), float(np.clip(ytr.mean(), 1e-6, 1 - 1e-6)))
|
||||
else:
|
||||
c, b, _ = fit_logistic(X[mask], ytr, lam)
|
||||
p = sigmoid(predict_linear(X[hold], c, b))
|
||||
vals.append(log_loss(y[hold], p))
|
||||
else:
|
||||
c, b = fit_ridge(X[mask], y[mask], lam)
|
||||
resid = y[hold] - predict_linear(X[hold], c, b)
|
||||
vals.append(float(np.mean(resid ** 2)))
|
||||
scores[float(lam)] = float(np.mean(vals)) if vals else float("inf")
|
||||
best = min(scores, key=lambda k: scores[k])
|
||||
return best, scores
|
||||
|
||||
|
||||
def lgbm_available() -> bool:
|
||||
try:
|
||||
import lightgbm # noqa: F401
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def fit_lgbm(X: np.ndarray, y: np.ndarray, seed: int):
|
||||
import lightgbm as lgb
|
||||
model = lgb.LGBMRegressor(random_state=seed, verbose=-1, **LGBM_PARAMS)
|
||||
model.fit(X, y)
|
||||
return model
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Size map
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def round_to(x, step: float = MAP_ROUND_TO) -> np.ndarray:
|
||||
"""Round to the nearest ``step`` and kill float dust (0.05 → 2 dp)."""
|
||||
decimals = max(0, int(round(-math.log10(step)))) + 1
|
||||
return np.round(np.round(np.asarray(x, dtype=float) / step) * step, decimals)
|
||||
|
||||
|
||||
def calibrate_map(z_train: np.ndarray) -> Dict[str, float]:
|
||||
"""lo/hi/normaliser from TRAIN predictions only."""
|
||||
lo = float(np.quantile(z_train, MAP_Q_LO))
|
||||
hi = float(np.quantile(z_train, MAP_Q_HI))
|
||||
if not np.isfinite(lo) or not np.isfinite(hi) or hi <= lo:
|
||||
return {"lo": lo, "hi": hi, "normalizer": 1.0, "degenerate": True}
|
||||
raw = MAP_OUT_LO + (MAP_OUT_HI - MAP_OUT_LO) * np.clip((z_train - lo) / (hi - lo), 0, 1)
|
||||
mean_raw = float(np.mean(raw))
|
||||
norm = 1.0 / mean_raw if mean_raw > 0 else 1.0
|
||||
return {"lo": lo, "hi": hi, "normalizer": norm, "degenerate": False}
|
||||
|
||||
|
||||
def apply_map(z: np.ndarray, cal: Dict[str, float]) -> np.ndarray:
|
||||
z = np.asarray(z, dtype=float)
|
||||
if cal.get("degenerate"):
|
||||
return np.ones(len(z))
|
||||
lo, hi, norm = cal["lo"], cal["hi"], cal["normalizer"]
|
||||
raw = MAP_OUT_LO + (MAP_OUT_HI - MAP_OUT_LO) * np.clip((z - lo) / (hi - lo), 0, 1)
|
||||
return round_to(np.clip(raw * norm, MAP_CLIP_LO, MAP_CLIP_HI))
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Metrics
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def max_drawdown(r: np.ndarray) -> float:
|
||||
"""0-prepended cumulative-R drawdown (gpu_backtest.compute_metrics)."""
|
||||
r = np.asarray(r, dtype=float)
|
||||
if len(r) == 0:
|
||||
return 0.0
|
||||
cum = np.concatenate(([0.0], np.cumsum(r)))
|
||||
return float((np.maximum.accumulate(cum) - cum).max())
|
||||
|
||||
|
||||
def profit_factor(r: np.ndarray) -> float:
|
||||
r = np.asarray(r, dtype=float)
|
||||
gp = float(r[r > 0].sum()) if (r > 0).any() else 0.0
|
||||
gl = float(-r[r < 0].sum()) if (r < 0).any() else 0.0
|
||||
return gp / gl if gl > 0 else float("inf")
|
||||
|
||||
|
||||
def multiplier_hist(m: np.ndarray) -> pd.DataFrame:
|
||||
"""Counts in 0.1-wide bins from 0.5 to 1.5.
|
||||
|
||||
``np.histogram`` already closes the final bin on the right, so an
|
||||
exactly-1.5 multiplier lands in [1.4, 1.5] once and the counts sum to
|
||||
``len(m)``.
|
||||
"""
|
||||
edges = np.round(np.arange(MAP_OUT_LO, MAP_OUT_HI + 1e-9, 0.1), 2)
|
||||
counts, _ = np.histogram(np.asarray(m, dtype=float), bins=edges)
|
||||
counts = counts.astype(int)
|
||||
return pd.DataFrame({
|
||||
"bin_lo": edges[:-1], "bin_hi": edges[1:], "count": counts,
|
||||
"pct": np.where(len(m), counts / max(len(m), 1) * 100, 0.0),
|
||||
})
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Walk-forward driver
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def walk_forward(df: pd.DataFrame, features: Sequence[str], model: str,
|
||||
bar_minutes: int, burn_in_months: int, purge_bars: int,
|
||||
seed: int, y_train_override: Optional[np.ndarray] = None,
|
||||
verbose: bool = True) -> Dict:
|
||||
"""One full expanding walk-forward pass.
|
||||
|
||||
``y_train_override`` (full-book length, aligned to ``df``) replaces the
|
||||
TRAINING labels only — this is how the shuffle null is run. Test labels
|
||||
always come from the real ``r_multiple``.
|
||||
"""
|
||||
times = df["entry_time"].to_numpy()
|
||||
r_all = df["r_multiple"].to_numpy(float)
|
||||
r_train_src = r_all if y_train_override is None else np.asarray(y_train_override, float)
|
||||
X_all = df[list(features)].apply(pd.to_numeric, errors="coerce").to_numpy(float) \
|
||||
if features else np.zeros((len(df), 0))
|
||||
|
||||
book_start = df["entry_time"].iloc[0]
|
||||
book_end = df["entry_time"].iloc[-1]
|
||||
burn_end = book_start + pd.DateOffset(months=burn_in_months)
|
||||
purge = pd.Timedelta(minutes=purge_bars * bar_minutes)
|
||||
|
||||
qs = [q for q in quarter_starts(book_start, book_end) if q >= burn_end]
|
||||
rows: List[Dict] = []
|
||||
per_q: List[Dict] = []
|
||||
coef_rows: List[Dict] = []
|
||||
skipped: List[str] = []
|
||||
dropped_seen: set = set()
|
||||
|
||||
for q_start in qs:
|
||||
q_end = q_start + pd.DateOffset(months=3)
|
||||
train_mask = df["entry_time"] < (q_start - purge)
|
||||
test_mask = (df["entry_time"] >= q_start) & (df["entry_time"] < q_end)
|
||||
n_tr, n_te = int(train_mask.sum()), int(test_mask.sum())
|
||||
if n_tr < MIN_TRAIN or n_te == 0:
|
||||
skipped.append(f"{quarter_label(q_start)} (train={n_tr}, test={n_te})")
|
||||
continue
|
||||
|
||||
tr = np.flatnonzero(train_mask.to_numpy())
|
||||
te = np.flatnonzero(test_mask.to_numpy())
|
||||
Xtr_raw, Xte_raw = X_all[tr], X_all[te]
|
||||
mean, scale, keep = fit_standardiser(Xtr_raw)
|
||||
dropped = [features[i] for i in np.flatnonzero(~keep)]
|
||||
if verbose and dropped:
|
||||
tag = (quarter_label(q_start), tuple(dropped))
|
||||
if tag not in dropped_seen:
|
||||
dropped_seen.add(tag)
|
||||
say(f" WARNING {quarter_label(q_start)}: zero variance in train, "
|
||||
f"dropped from fit (coef 0): {', '.join(dropped)}")
|
||||
Xtr = apply_standardiser(Xtr_raw, mean, scale)[:, keep]
|
||||
Xte = apply_standardiser(Xte_raw, mean, scale)[:, keep]
|
||||
|
||||
r_tr = r_train_src[tr]
|
||||
lam: Optional[float] = None
|
||||
if model == "logistic":
|
||||
y_tr = (r_tr > 0).astype(float)
|
||||
lam, _ = choose_lambda(Xtr, y_tr, "logistic")
|
||||
coef, b, _ = fit_logistic(Xtr, y_tr, lam)
|
||||
z_tr = sigmoid(predict_linear(Xtr, coef, b))
|
||||
z_te = sigmoid(predict_linear(Xte, coef, b))
|
||||
elif model == "lgbm":
|
||||
y_tr = np.clip(r_tr, WINSOR_LO, WINSOR_HI)
|
||||
gbm = fit_lgbm(Xtr, y_tr, seed)
|
||||
z_tr = np.asarray(gbm.predict(Xtr), dtype=float)
|
||||
z_te = np.asarray(gbm.predict(Xte), dtype=float)
|
||||
coef, b = np.zeros(int(keep.sum())), 0.0
|
||||
else: # ridge / isbuy / exhaustion
|
||||
y_tr = np.clip(r_tr, WINSOR_LO, WINSOR_HI)
|
||||
lam, _ = choose_lambda(Xtr, y_tr, "ridge")
|
||||
coef, b = fit_ridge(Xtr, y_tr, lam)
|
||||
z_tr = predict_linear(Xtr, coef, b)
|
||||
z_te = predict_linear(Xte, coef, b)
|
||||
|
||||
cal = calibrate_map(z_tr)
|
||||
m_te = apply_map(z_te, cal)
|
||||
|
||||
if model != "lgbm":
|
||||
full_coef = np.zeros(len(features))
|
||||
full_coef[np.flatnonzero(keep)] = coef
|
||||
for name, c in zip(features, full_coef):
|
||||
coef_rows.append({"quarter": quarter_label(q_start), "feature": name,
|
||||
"coef_std": float(c), "lambda": lam,
|
||||
"dropped": name in dropped})
|
||||
coef_rows.append({"quarter": quarter_label(q_start), "feature": "_intercept",
|
||||
"coef_std": float(b), "lambda": lam, "dropped": False})
|
||||
|
||||
r_te = r_all[te]
|
||||
for i, idx in enumerate(te):
|
||||
rows.append({
|
||||
"entry_time": df["entry_time"].iloc[idx],
|
||||
"r_multiple": float(r_te[i]),
|
||||
"mult": float(m_te[i]),
|
||||
"r_sized": float(r_te[i] * m_te[i]),
|
||||
"quarter": quarter_label(q_start),
|
||||
"z": float(z_te[i]),
|
||||
})
|
||||
per_q.append({
|
||||
"quarter": quarter_label(q_start),
|
||||
"q_start": q_start, "n_train": n_tr, "n_test": n_te,
|
||||
"flat_r": float(r_te.sum()), "sized_r": float((r_te * m_te).sum()),
|
||||
"mean_m": float(m_te.mean()), "lambda": lam,
|
||||
"map_lo": cal["lo"], "map_hi": cal["hi"], "normalizer": cal["normalizer"],
|
||||
})
|
||||
|
||||
oos = pd.DataFrame(rows)
|
||||
return {"oos": oos, "quarters": pd.DataFrame(per_q),
|
||||
"coefs": pd.DataFrame(coef_rows), "skipped": skipped,
|
||||
"quarter_starts": qs}
|
||||
|
||||
|
||||
def evaluate(oos: pd.DataFrame) -> Dict:
|
||||
"""Headline OOS numbers for one model's walk-forward output."""
|
||||
if oos.empty:
|
||||
return {"n": 0, "flat_r": 0.0, "sized_r": 0.0, "lift": 0.0}
|
||||
r = oos["r_multiple"].to_numpy(float)
|
||||
rs = oos["r_sized"].to_numpy(float)
|
||||
m = oos["mult"].to_numpy(float)
|
||||
med = oos["entry_time"].median()
|
||||
first = (oos["entry_time"] < med).to_numpy()
|
||||
second = ~first
|
||||
return {
|
||||
"n": int(len(oos)),
|
||||
"flat_r": float(r.sum()), "sized_r": float(rs.sum()),
|
||||
"lift": float(rs.sum() - r.sum()),
|
||||
"flat_pf": profit_factor(r), "sized_pf": profit_factor(rs),
|
||||
"flat_dd": max_drawdown(r), "sized_dd": max_drawdown(rs),
|
||||
"mean_m": float(m.mean()), "min_m": float(m.min()), "max_m": float(m.max()),
|
||||
"median_time": med,
|
||||
"n_h1": int(first.sum()), "n_h2": int(second.sum()),
|
||||
"flat_h1": float(r[first].sum()), "flat_h2": float(r[second].sum()),
|
||||
"sized_h1": float(rs[first].sum()), "sized_h2": float(rs[second].sum()),
|
||||
}
|
||||
|
||||
|
||||
def acceptance(ev: Dict, real_lift: float, shuffle_p95: float) -> Dict:
|
||||
"""The six pre-registered clauses."""
|
||||
c1 = ev["sized_h1"] > ev["flat_h1"] and ev["sized_h2"] > ev["flat_h2"]
|
||||
c2 = ev["sized_r"] > ev["flat_r"]
|
||||
c3 = ev["sized_dd"] <= DD_TOL * ev["flat_dd"]
|
||||
c4 = (ev["sized_pf"] >= PF_FLOOR) and ev["sized_h1"] > 0 and ev["sized_h2"] > 0
|
||||
c5 = real_lift > shuffle_p95 if np.isfinite(shuffle_p95) else False
|
||||
c6 = (abs(ev["mean_m"] - 1.0) <= MEAN_MULT_TOL
|
||||
and ev["min_m"] >= MAP_CLIP_LO - 1e-9 and ev["max_m"] <= MAP_CLIP_HI + 1e-9)
|
||||
clauses = {"1_both_halves": bool(c1), "2_overall": bool(c2), "3_drawdown": bool(c3),
|
||||
"4_pf_and_halves": bool(c4), "5_beats_shuffle": bool(c5),
|
||||
"6_multiplier_bounds": bool(c6)}
|
||||
clauses["overall"] = bool(all(clauses.values()))
|
||||
return clauses
|
||||
|
||||
|
||||
def shuffle_null(df: pd.DataFrame, features: Sequence[str], model: str,
|
||||
bar_minutes: int, burn_in_months: int, purge_bars: int,
|
||||
n_shuffle: int, seed: int) -> np.ndarray:
|
||||
"""Lift distribution under permuted TRAINING labels (test labels real)."""
|
||||
rng = np.random.default_rng(seed)
|
||||
r_all = df["r_multiple"].to_numpy(float)
|
||||
lifts: List[float] = []
|
||||
for _ in range(n_shuffle):
|
||||
r_perm = r_all[rng.permutation(len(r_all))]
|
||||
res = walk_forward(df, features, model, bar_minutes, burn_in_months,
|
||||
purge_bars, seed, y_train_override=r_perm, verbose=False)
|
||||
ev = evaluate(res["oos"])
|
||||
lifts.append(ev["lift"])
|
||||
return np.asarray(lifts, dtype=float)
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Final fit + artifact
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def final_fit(df: pd.DataFrame, features: Sequence[str], model: str,
|
||||
seed: int) -> Dict:
|
||||
"""Fit on ALL trades; calibrate the map on in-sample predictions."""
|
||||
X_raw = df[list(features)].apply(pd.to_numeric, errors="coerce").to_numpy(float)
|
||||
r = df["r_multiple"].to_numpy(float)
|
||||
mean, scale, keep = fit_standardiser(X_raw)
|
||||
X = apply_standardiser(X_raw, mean, scale)[:, keep]
|
||||
if model == "logistic":
|
||||
y = (r > 0).astype(float)
|
||||
lam, _ = choose_lambda(X, y, "logistic")
|
||||
coef, b, gnorm = fit_logistic(X, y, lam)
|
||||
z = sigmoid(predict_linear(X, coef, b))
|
||||
target = "r_positive"
|
||||
else:
|
||||
y = np.clip(r, WINSOR_LO, WINSOR_HI)
|
||||
lam, _ = choose_lambda(X, y, "ridge")
|
||||
coef, b = fit_ridge(X, y, lam)
|
||||
gnorm = float("nan")
|
||||
z = predict_linear(X, coef, b)
|
||||
target = "r_winsorised"
|
||||
full_coef = np.zeros(len(features))
|
||||
full_coef[np.flatnonzero(keep)] = coef
|
||||
cal = calibrate_map(z)
|
||||
return {"mean": mean, "scale": scale, "keep": keep, "coef": full_coef,
|
||||
"intercept": float(b), "lambda": float(lam), "z": z,
|
||||
"mult": apply_map(z, cal), "map": cal, "target": target,
|
||||
"grad_norm": gnorm}
|
||||
|
||||
|
||||
def predict_multiplier(feature_dict: Dict[str, float], artifact: Dict) -> float:
|
||||
"""Runtime formula: standardise → linear score → size map → 0.05 grid."""
|
||||
names = artifact["features"]
|
||||
mean = artifact["mean"]
|
||||
scale = artifact["scale"]
|
||||
coef = artifact["coef"]
|
||||
z = float(artifact["intercept"])
|
||||
for i, name in enumerate(names):
|
||||
raw = feature_dict.get(name, mean[i])
|
||||
if raw is None or not np.isfinite(float(raw)):
|
||||
raw = mean[i]
|
||||
z += float(coef[i]) * (float(raw) - float(mean[i])) / float(scale[i])
|
||||
if artifact["model"] == "logistic":
|
||||
z = 1.0 / (1.0 + math.exp(-max(-35.0, min(35.0, z))))
|
||||
mp = artifact["map"]
|
||||
if mp["hi"] <= mp["lo"]:
|
||||
return 1.0
|
||||
frac = min(1.0, max(0.0, (z - mp["lo"]) / (mp["hi"] - mp["lo"])))
|
||||
raw_m = mp["out_lo"] + (mp["out_hi"] - mp["out_lo"]) * frac
|
||||
val = min(mp["clip_hi"], max(mp["clip_lo"], raw_m * mp["normalizer"]))
|
||||
step = mp["round_to"]
|
||||
return float(round(round(val / step) * step, 2))
|
||||
|
||||
|
||||
def build_artifact(fit: Dict, features: Sequence[str], model: str, df: pd.DataFrame,
|
||||
ev: Dict, clauses: Dict, real_lift: float, shuffle: np.ndarray,
|
||||
n_shuffle: int, book: Path, tf: str, confirm_tfs: List[str],
|
||||
burn_in_months: int, purge_bars: int, spread: float) -> Dict:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"feature_version": FEATURE_VERSION,
|
||||
"pair": "XAU/USD",
|
||||
"primary_tf": tf,
|
||||
"confirmation_tfs": confirm_tfs,
|
||||
"model": model,
|
||||
"target": fit["target"],
|
||||
"features": list(features),
|
||||
"mean": [float(v) for v in fit["mean"]],
|
||||
"scale": [float(v) for v in fit["scale"]],
|
||||
"coef": [float(v) for v in fit["coef"]],
|
||||
"intercept": float(fit["intercept"]),
|
||||
"map": {
|
||||
"kind": "clipped_linear_on_quantiles",
|
||||
"lo": float(fit["map"]["lo"]), "hi": float(fit["map"]["hi"]),
|
||||
"out_lo": MAP_OUT_LO, "out_hi": MAP_OUT_HI,
|
||||
"normalizer": float(fit["map"]["normalizer"]),
|
||||
"clip_lo": MAP_CLIP_LO, "clip_hi": MAP_CLIP_HI,
|
||||
"round_to": MAP_ROUND_TO,
|
||||
},
|
||||
"training": {
|
||||
"window": [df["entry_time"].iloc[0].isoformat(),
|
||||
df["entry_time"].iloc[-1].isoformat()],
|
||||
"spread": spread,
|
||||
"n_train_trades": int(len(df)),
|
||||
"n_oos_trades": int(ev.get("n", 0)),
|
||||
"burn_in_months": burn_in_months,
|
||||
"purge_bars": purge_bars,
|
||||
"refit": "calendar_quarter",
|
||||
"final_fit": "full_window",
|
||||
"lambda": float(fit["lambda"]),
|
||||
"git_commit": git_commit(),
|
||||
"book_sha256": sha256_file(book),
|
||||
},
|
||||
"acceptance": {
|
||||
**{k: v for k, v in clauses.items()},
|
||||
"flat_r": ev.get("flat_r"), "sized_r": ev.get("sized_r"),
|
||||
"sized_pf": ev.get("sized_pf"), "flat_dd": ev.get("flat_dd"),
|
||||
"sized_dd": ev.get("sized_dd"),
|
||||
"sized_h1": ev.get("sized_h1"), "sized_h2": ev.get("sized_h2"),
|
||||
"flat_h1": ev.get("flat_h1"), "flat_h2": ev.get("flat_h2"),
|
||||
"real_lift": float(real_lift),
|
||||
"shuffle_p95_lift": float(np.percentile(shuffle, 95)) if len(shuffle) else None,
|
||||
"n_shuffle": int(n_shuffle),
|
||||
"mean_multiplier": ev.get("mean_m"),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# Reporting
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def fmt(x, nd=2, width=0) -> str:
|
||||
if x is None:
|
||||
return "n/a".rjust(width)
|
||||
if isinstance(x, float) and not np.isfinite(x):
|
||||
return ("inf" if x > 0 else "-inf").rjust(width)
|
||||
s = f"{x:.{nd}f}" if isinstance(x, (int, float, np.floating)) else str(x)
|
||||
return s.rjust(width) if width else s
|
||||
|
||||
|
||||
def print_quarters(q: pd.DataFrame) -> None:
|
||||
say(f"{'quarter':<9}{'n_tr':>6}{'n_te':>6}{'flat R':>10}{'sized R':>10}"
|
||||
f"{'mean m':>9}{'lambda':>9}")
|
||||
for _, row in q.iterrows():
|
||||
say(f"{row['quarter']:<9}{int(row['n_train']):>6}{int(row['n_test']):>6}"
|
||||
f"{fmt(row['flat_r'], 2, 10)}{fmt(row['sized_r'], 2, 10)}"
|
||||
f"{fmt(row['mean_m'], 3, 9)}"
|
||||
f"{('n/a' if row['lambda'] is None else fmt(row['lambda'], 0)):>9}")
|
||||
|
||||
|
||||
def print_hist(m: np.ndarray) -> None:
|
||||
h = multiplier_hist(m)
|
||||
say(f"{'bin':<12}{'count':>8}{'pct':>8}")
|
||||
for _, row in h.iterrows():
|
||||
bar = "#" * int(round(row["pct"] / 2))
|
||||
say(f"{row['bin_lo']:.1f}–{row['bin_hi']:.1f}{'':<4}{int(row['count']):>8}"
|
||||
f"{row['pct']:>7.1f}% {bar}")
|
||||
|
||||
|
||||
def print_coef_signs(coefs: pd.DataFrame, label: str) -> None:
|
||||
"""Mean standardised coefficient + sign stability across quarters."""
|
||||
if coefs.empty:
|
||||
return
|
||||
d = coefs[coefs["feature"] != "_intercept"]
|
||||
say(f" standardised coefficients across quarters ({label}):")
|
||||
say(f" {'feature':<24}{'mean':>10}{'min':>10}{'max':>10}{'+q':>5}{'-q':>5} sign-stable")
|
||||
for name, grp in d.groupby("feature", sort=False):
|
||||
c = grp["coef_std"].to_numpy(float)
|
||||
pos, neg = int((c > 0).sum()), int((c < 0).sum())
|
||||
stable = "yes" if (pos == 0 or neg == 0) and (pos + neg) > 0 else "no"
|
||||
say(f" {name:<24}{c.mean():>10.4f}{c.min():>10.4f}{c.max():>10.4f}"
|
||||
f"{pos:>5}{neg:>5} {stable}")
|
||||
|
||||
|
||||
def print_comparison(table: List[Dict]) -> None:
|
||||
say(f"{'model':<12}{'n':>6}{'flat R':>10}{'sized R':>10}{'lift':>9}"
|
||||
f"{'PF':>7}{'DD':>8}{'mean m':>8}{'p95 null':>10}{'rank%':>7} pass")
|
||||
for row in table:
|
||||
say(f"{row['model']:<12}{row['n']:>6}{fmt(row['flat_r'], 2, 10)}"
|
||||
f"{fmt(row['sized_r'], 2, 10)}{fmt(row['lift'], 2, 9)}"
|
||||
f"{fmt(row['sized_pf'], 2, 7)}{fmt(row['sized_dd'], 2, 8)}"
|
||||
f"{fmt(row['mean_m'], 3, 8)}{fmt(row['shuffle_p95'], 2, 10)}"
|
||||
f"{fmt(row['rank_pct'], 0, 7)} {'PASS' if row['pass'] else 'FAIL'}")
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
# main
|
||||
# ══════════════════════════════════════════════════════════════════════
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--book", required=True, type=Path)
|
||||
ap.add_argument("--bar-minutes", required=True, type=int)
|
||||
ap.add_argument("--features", required=True,
|
||||
help="comma-separated f_* column names for the main model")
|
||||
ap.add_argument("--model", default="ridge", choices=("ridge", "logistic"))
|
||||
ap.add_argument("--burn-in-months", type=int, default=12)
|
||||
ap.add_argument("--purge-bars", type=int, default=300)
|
||||
ap.add_argument("--n-shuffle", type=int, default=50)
|
||||
ap.add_argument("--seed", type=int, default=20260905)
|
||||
ap.add_argument("--skip-lgbm", action="store_true")
|
||||
ap.add_argument("--out", type=Path, default=None)
|
||||
ap.add_argument("--write-artifact", type=Path, default=None)
|
||||
ap.add_argument("--tf", default=None, help="primary TF label for the artifact")
|
||||
ap.add_argument("--confirm-tfs", default="15min,1h")
|
||||
ap.add_argument("--spread", type=float, default=0.65,
|
||||
help="cost the book was produced at (recorded in the artifact)")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.write_artifact and not args.tf:
|
||||
raise SystemExit("--write-artifact requires --tf")
|
||||
|
||||
say("=" * 96)
|
||||
say("Stage-D stake-sizing walk-forward — RESEARCH ONLY")
|
||||
say("=" * 96)
|
||||
say(f"book : {args.book}")
|
||||
df = load_book(args.book)
|
||||
say(f"trades : {len(df)} "
|
||||
f"({df['entry_time'].iloc[0]} → {df['entry_time'].iloc[-1]})")
|
||||
say(f"bar minutes : {args.bar_minutes} purge = {args.purge_bars} bars = "
|
||||
f"{args.purge_bars * args.bar_minutes} min "
|
||||
f"({args.purge_bars * args.bar_minutes / 1440:.2f} d)")
|
||||
say(f"burn-in : {args.burn_in_months} months "
|
||||
f"(train-only until {df['entry_time'].iloc[0] + pd.DateOffset(months=args.burn_in_months)})")
|
||||
say(f"model / seed : {args.model} / {args.seed} shuffles: {args.n_shuffle}")
|
||||
say(f"winsor / lambda : [{WINSOR_LO}, {WINSOR_HI}] / {list(LAMBDA_GRID)} "
|
||||
f"CV folds {CV_FOLDS} (time-contiguous)")
|
||||
say(f"map : m = round_{MAP_ROUND_TO}(clip(norm × (0.5 + clip((z−lo)/(hi−lo),0,1)"
|
||||
f"), {MAP_CLIP_LO}, {MAP_CLIP_HI})) lo/hi = train q{MAP_Q_LO}/q{MAP_Q_HI}")
|
||||
say("")
|
||||
|
||||
main_feats = resolve_features(df, [f.strip() for f in args.features.split(",") if f.strip()],
|
||||
"--features")
|
||||
say(f"main features ({len(main_feats)}): {', '.join(main_feats)}")
|
||||
|
||||
specs: List[Tuple[str, List[str], str]] = [(args.model, main_feats, args.model)]
|
||||
for label, feats in (("isbuy", ISBUY_FEATURES), ("exhaustion", EXHAUSTION_FEATURES)):
|
||||
have = [f for f in feats if f in df.columns]
|
||||
if len(have) != len(feats):
|
||||
say(f"NOTE: {label} baseline missing "
|
||||
f"{', '.join(f for f in feats if f not in df.columns)} — using {len(have)}")
|
||||
if have:
|
||||
specs.append((label, have, "ridge"))
|
||||
use_lgbm = (not args.skip_lgbm) and lgbm_available()
|
||||
if not args.skip_lgbm and not use_lgbm:
|
||||
say("lightgbm not installed — upper bound skipped")
|
||||
if use_lgbm:
|
||||
all_f = [c for c in df.columns if c.startswith("f_")]
|
||||
specs.append(("lgbm", all_f, "lgbm"))
|
||||
say(f"lightgbm upper bound: {len(all_f)} f_* columns")
|
||||
say("")
|
||||
|
||||
results: Dict[str, Dict] = {}
|
||||
table: List[Dict] = []
|
||||
for label, feats, kind in specs:
|
||||
say("-" * 96)
|
||||
say(f"MODEL: {label} ({kind}, {len(feats)} feature(s))")
|
||||
say("-" * 96)
|
||||
res = walk_forward(df, feats, kind, args.bar_minutes, args.burn_in_months,
|
||||
args.purge_bars, args.seed, verbose=True)
|
||||
if res["skipped"]:
|
||||
say(f" skipped quarters: {'; '.join(res['skipped'])}")
|
||||
if res["oos"].empty:
|
||||
say(" no OOS quarters — nothing to evaluate")
|
||||
continue
|
||||
ev = evaluate(res["oos"])
|
||||
n_sh = args.n_shuffle if kind != "lgbm" else min(args.n_shuffle, LGBM_MAX_SHUFFLE)
|
||||
if kind == "lgbm" and n_sh != args.n_shuffle:
|
||||
say(f" NOTE (protocol deviation, flagged): lgbm shuffle null capped at "
|
||||
f"{n_sh} repeats (never ships)")
|
||||
sh = shuffle_null(df, feats, kind, args.bar_minutes, args.burn_in_months,
|
||||
args.purge_bars, n_sh, args.seed) if n_sh > 0 else np.array([])
|
||||
p95 = float(np.percentile(sh, 95)) if len(sh) else float("nan")
|
||||
rank = float((sh < ev["lift"]).mean() * 100) if len(sh) else float("nan")
|
||||
cl = acceptance(ev, ev["lift"], p95)
|
||||
|
||||
say(f" OOS quarters : {len(res['quarters'])} "
|
||||
f"({res['quarters']['quarter'].iloc[0]} → {res['quarters']['quarter'].iloc[-1]}), "
|
||||
f"{ev['n']} trades")
|
||||
say(f" flat R : {ev['flat_r']:+.2f} PF {fmt(ev['flat_pf'])} "
|
||||
f"maxDD {ev['flat_dd']:.2f}")
|
||||
say(f" sized R : {ev['sized_r']:+.2f} PF {fmt(ev['sized_pf'])} "
|
||||
f"maxDD {ev['sized_dd']:.2f}")
|
||||
say(f" lift : {ev['lift']:+.2f} R")
|
||||
say(f" halves (split {ev['median_time']}): "
|
||||
f"h1 flat {ev['flat_h1']:+.2f} / sized {ev['sized_h1']:+.2f} (n={ev['n_h1']}) | "
|
||||
f"h2 flat {ev['flat_h2']:+.2f} / sized {ev['sized_h2']:+.2f} (n={ev['n_h2']})")
|
||||
say(f" multiplier : mean {ev['mean_m']:.3f} min {ev['min_m']:.2f} "
|
||||
f"max {ev['max_m']:.2f}")
|
||||
say(f" shuffle null : n={len(sh)} mean {fmt(float(sh.mean()) if len(sh) else None)} "
|
||||
f"p95 {fmt(p95)} real-lift percentile rank {fmt(rank, 0)}%")
|
||||
say(" acceptance:")
|
||||
for k, v in cl.items():
|
||||
if k == "overall":
|
||||
continue
|
||||
say(f" {k:<22} {'PASS' if v else 'FAIL'}")
|
||||
say(f" {'OVERALL':<22} {'PASS' if cl['overall'] else 'FAIL'}")
|
||||
say(" per-quarter:")
|
||||
print_quarters(res["quarters"])
|
||||
print_coef_signs(res["coefs"], label)
|
||||
say("")
|
||||
|
||||
results[label] = {"res": res, "ev": ev, "shuffle": sh, "p95": p95,
|
||||
"rank": rank, "clauses": cl, "features": feats, "kind": kind}
|
||||
table.append({"model": label, "n": ev["n"], "flat_r": ev["flat_r"],
|
||||
"sized_r": ev["sized_r"], "lift": ev["lift"],
|
||||
"sized_pf": ev["sized_pf"], "sized_dd": ev["sized_dd"],
|
||||
"mean_m": ev["mean_m"], "shuffle_p95": p95, "rank_pct": rank,
|
||||
"pass": cl["overall"]})
|
||||
|
||||
if args.model not in results:
|
||||
raise SystemExit("the main model produced no OOS quarters — check burn-in / "
|
||||
"book window / MIN_TRAIN")
|
||||
|
||||
say("=" * 96)
|
||||
say("COMPARISON")
|
||||
say("=" * 96)
|
||||
print_comparison(table)
|
||||
say("")
|
||||
say(f"OOS multiplier histogram ({args.model}):")
|
||||
print_hist(results[args.model]["res"]["oos"]["mult"].to_numpy())
|
||||
say("")
|
||||
|
||||
main_label = args.model
|
||||
beats = None
|
||||
if main_label in results and "isbuy" in results:
|
||||
beats = results[main_label]["ev"]["lift"] > results["isbuy"]["ev"]["lift"]
|
||||
say(f"beats isbuy baseline: {'yes' if beats else 'no'} "
|
||||
f"({results[main_label]['ev']['lift']:+.2f} R vs "
|
||||
f"{results['isbuy']['ev']['lift']:+.2f} R)")
|
||||
say(f"OVERALL ACCEPTANCE ({main_label}): "
|
||||
f"{'PASS' if results[main_label]['clauses']['overall'] else 'FAIL'}")
|
||||
say("")
|
||||
|
||||
# ── artifact ──────────────────────────────────────────────────────
|
||||
artifact = None
|
||||
if args.write_artifact:
|
||||
say("-" * 96)
|
||||
say("FINAL FIT (all trades) + ARTIFACT")
|
||||
say("-" * 96)
|
||||
fit = final_fit(df, main_feats, args.model, args.seed)
|
||||
say(f" lambda (CV on full window): {fit['lambda']:g}")
|
||||
if np.isfinite(fit["grad_norm"]):
|
||||
say(f" logistic |grad|_inf at convergence: {fit['grad_norm']:.3e}")
|
||||
say(f" map lo/hi/normaliser: {fit['map']['lo']:.5f} / {fit['map']['hi']:.5f} / "
|
||||
f"{fit['map']['normalizer']:.5f}")
|
||||
artifact = build_artifact(
|
||||
fit, main_feats, args.model, df, results[main_label]["ev"],
|
||||
results[main_label]["clauses"], results[main_label]["ev"]["lift"],
|
||||
results[main_label]["shuffle"], args.n_shuffle, args.book, args.tf,
|
||||
[t.strip() for t in args.confirm_tfs.split(",") if t.strip()],
|
||||
args.burn_in_months, args.purge_bars, args.spread)
|
||||
# Self-check: the runtime formula must reproduce the in-sample map exactly.
|
||||
rows = df[main_feats].to_dict("records")
|
||||
repro = np.array([predict_multiplier(r, artifact) for r in rows])
|
||||
exact = bool(np.array_equal(repro, fit["mult"]))
|
||||
say(f" predict_multiplier reproduces in-sample multipliers exactly: "
|
||||
f"{'YES' if exact else 'NO'} (max |diff| "
|
||||
f"{float(np.max(np.abs(repro - fit['mult']))):.3e})")
|
||||
if not exact:
|
||||
raise SystemExit("predict_multiplier does not reproduce the final fit — abort")
|
||||
args.write_artifact.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.write_artifact.write_text(json.dumps(artifact, indent=2, default=str))
|
||||
say(f" wrote {args.write_artifact}")
|
||||
say("")
|
||||
|
||||
# ── outputs ───────────────────────────────────────────────────────
|
||||
if args.out:
|
||||
out = args.out
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
main_res = results[main_label]
|
||||
main_res["res"]["oos"].to_csv(out / "oos_trades.csv", index=False)
|
||||
main_res["res"]["quarters"].to_csv(out / "quarters.csv", index=False)
|
||||
coefs = pd.concat(
|
||||
[r["res"]["coefs"].assign(model=lbl) for lbl, r in results.items()
|
||||
if not r["res"]["coefs"].empty],
|
||||
ignore_index=True) if results else pd.DataFrame()
|
||||
coefs.to_csv(out / "coefs_by_quarter.csv", index=False)
|
||||
pd.DataFrame({
|
||||
"model": np.repeat([lbl for lbl in results], [len(r["shuffle"]) for r in results.values()]),
|
||||
"lift": np.concatenate([r["shuffle"] for r in results.values()])
|
||||
if results else np.array([]),
|
||||
}).to_csv(out / "shuffle_lifts.csv", index=False)
|
||||
multiplier_hist(main_res["res"]["oos"]["mult"].to_numpy()).to_csv(
|
||||
out / "multiplier_hist.csv", index=False)
|
||||
payload = {
|
||||
"book": str(args.book), "book_sha256": sha256_file(args.book),
|
||||
"git_commit": git_commit(), "bar_minutes": args.bar_minutes,
|
||||
"model": args.model, "features": main_feats, "seed": args.seed,
|
||||
"n_shuffle": args.n_shuffle, "burn_in_months": args.burn_in_months,
|
||||
"purge_bars": args.purge_bars,
|
||||
"beats_isbuy": beats,
|
||||
"models": {
|
||||
lbl: {
|
||||
"kind": r["kind"], "features": r["features"],
|
||||
"evaluation": {k: (None if isinstance(v, float) and not np.isfinite(v)
|
||||
else (str(v) if isinstance(v, pd.Timestamp) else v))
|
||||
for k, v in r["ev"].items()},
|
||||
"shuffle": {"n": int(len(r["shuffle"])),
|
||||
"mean": float(r["shuffle"].mean()) if len(r["shuffle"]) else None,
|
||||
"p95": None if not np.isfinite(r["p95"]) else r["p95"],
|
||||
"real_lift_percentile": None if not np.isfinite(r["rank"])
|
||||
else r["rank"]},
|
||||
"acceptance": r["clauses"],
|
||||
} for lbl, r in results.items()
|
||||
},
|
||||
"artifact": artifact,
|
||||
}
|
||||
(out / "results.json").write_text(json.dumps(payload, indent=2, default=str))
|
||||
(out / "summary.txt").write_text("\n".join(_LINES) + "\n")
|
||||
print(f"\nwrote outputs to {out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,78 +0,0 @@
|
||||
"""Evaluate the violent-move WATCH rule (as shipped in violent_watch.py) bar by bar on history. Analysis tooling only.
|
||||
Level per bar from: calendar (high-impact USD within 30 min ahead / 30 min after), clock slots, indicator readings.
|
||||
Reports share of bars per level, burst rate next bar / next hour per level (both halves), and hot-hours per day.
|
||||
python backtest/research/watch_rule_eval.py PARQUET BAR_MINUTES
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import numpy as np, pandas as pd
|
||||
ROOT = Path(__file__).resolve().parents[2]; sys.path.insert(0, str(ROOT)); sys.path.insert(0, str(ROOT / "backtest"))
|
||||
from gpu_backtest import precompute_indicators
|
||||
from config import TechnicalConfig
|
||||
|
||||
|
||||
def wilder_atr(df, period=14):
|
||||
pc = df["Close"].shift(1)
|
||||
tr = pd.concat([df["High"] - df["Low"], (df["High"] - pc).abs(), (df["Low"] - pc).abs()], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1 / period, min_periods=period, adjust=False).mean()
|
||||
|
||||
|
||||
def main():
|
||||
pq, bm = sys.argv[1], int(sys.argv[2])
|
||||
px = pd.read_parquet(pq); px = px[(px.index >= pd.Timestamp("2023-08-05", tz="UTC")) & (px.index < pd.Timestamp("2026-09-05", tz="UTC"))]
|
||||
ic = precompute_indicators(px, TechnicalConfig())
|
||||
sh = lambda a: pd.Series(np.asarray(a, dtype=float), index=px.index).shift(1)
|
||||
rsi, adxs, hist, bbp, kcu, kcl = sh(ic.rsi), sh(ic.adx_slope), sh(ic.macd_hist), sh(ic.bb_position), sh(ic.kc_upper), sh(ic.kc_lower)
|
||||
ef, em, es, et, adx, atr1 = sh(ic.ema_fast), sh(ic.ema_medium), sh(ic.ema_slow), sh(ic.ema_trend), sh(ic.adx), sh(ic.atr)
|
||||
c1 = px["Close"].shift(1); hp = hist.shift(1)
|
||||
atr_prev = wilder_atr(px).shift(1); rng = px["High"] - px["Low"]
|
||||
burst = (rng >= 3.0 * atr_prev).astype(int); H = max(1, 60 // bm)
|
||||
next_hour = burst[::-1].rolling(H, min_periods=1).max()[::-1].astype(int)
|
||||
align = np.sign(c1 - ef) + np.sign(ef - em) + np.sign(em - es) + np.sign(es - et)
|
||||
regime = np.select([(adx >= 25) & (align > 0), (adx >= 25) & (align < 0), adx >= 25, adxs > 3.0], ["trending_up", "trending_down", "trending", "transitioning"], default="ranging")
|
||||
atr_pct = atr1.rolling(50).apply(lambda w: (w[-1] > w).mean() * 100, raw=True); extreme = atr_pct > 80
|
||||
# readings
|
||||
bear = ((bbp < 0).astype(int) + (rsi < 30).astype(int) + (c1 < kcl).astype(int) + ((hist < 0) & (hist < hp) & (hp < 0)).astype(int))
|
||||
bull = ((bbp > 1).astype(int) + (rsi >= 70).astype(int) + (c1 > kcu).astype(int) + ((hist > 0) & (hist > hp) & (hp > 0)).astype(int))
|
||||
neutral = (adxs > 3).astype(int) + extreme.astype(int) + (regime == "transitioning").astype(int)
|
||||
readings = bear + bull + neutral
|
||||
# clock (now = bar close time)
|
||||
ct = px.index + pd.Timedelta(minutes=bm); wd = ct.weekday; hr = ct.hour
|
||||
clock = ((wd <= 4) & np.isin(hr, [12, 13, 14])) | (hr == 1) | ((wd == 6) & np.isin(hr, [22, 23]))
|
||||
# calendar
|
||||
cal = pd.read_parquet(ROOT / "backtest/cache/calendar.parquet"); cal["ts"] = pd.to_datetime(cal.dateline, unit="s", utc=True)
|
||||
hi = np.sort(cal[(cal.currency == "USD") & (cal.impactName.str.lower() == "high")].ts.values.astype("datetime64[ns]"))
|
||||
t = ct.values.astype("datetime64[ns]"); nx = np.searchsorted(hi, t)
|
||||
to_next = np.where(nx < len(hi), (hi[np.minimum(nx, len(hi) - 1)] - t) / np.timedelta64(1, "m"), 1e9)
|
||||
since = np.where(nx - 1 >= 0, (t - hi[np.maximum(nx - 1, 0)]) / np.timedelta64(1, "m"), 1e9)
|
||||
calendar = ((to_next > 0) & (to_next <= 30)) | (since <= 30)
|
||||
# FINAL RULE (2026-09-06, chosen from the variant sweep recorded in sizing_campaign_2026-09.md):
|
||||
# extreme = price-at-extreme readings only (outside Bollinger, RSI beyond 30/70, outside Keltner)
|
||||
# hot = calendar | (clock & extreme >= 1) -> ~8.6% of bars, next-bar odds 3-5x normal, ~4 hot hours/day
|
||||
# watch = clock | extreme >= 2 -> ~2x normal
|
||||
extreme = ((bbp < 0) | (bbp > 1)).astype(int) + ((rsi < 30) | (rsi >= 70)).astype(int) + ((c1 < kcl) | (c1 > kcu)).astype(int)
|
||||
level = np.where(calendar | (clock & (extreme >= 1)), "hot", np.where(clock | (extreme >= 2), "watch", "none"))
|
||||
d = pd.DataFrame({"level": level, "burst": burst.values, "next_hour": next_hour.values, "readings": readings.values, "clock": clock, "calendar": calendar}, index=px.index)
|
||||
d = d[d.index >= pd.Timestamp("2023-09-05", tz="UTC")]
|
||||
half = np.where(np.arange(len(d)) < len(d) // 2, 1, 2)
|
||||
b1, b2 = d.burst[half == 1].mean(), d.burst[half == 2].mean(); h1, h2 = d.next_hour[half == 1].mean(), d.next_hour[half == 2].mean()
|
||||
print(f"{Path(pq).name}: {len(d)} bars; base next-bar {100*d.burst.mean():.2f}%, next-hour {100*d.next_hour.mean():.1f}%")
|
||||
rows = []
|
||||
for lv in ("none", "watch", "hot"):
|
||||
m = d.level == lv
|
||||
rows.append(dict(level=lv, share_of_bars=100 * m.mean(), next_bar_pct=100 * d.burst[m].mean(), lift_h1=d.burst[m & (half == 1)].mean() / b1, lift_h2=d.burst[m & (half == 2)].mean() / b2,
|
||||
next_hour_pct=100 * d.next_hour[m].mean(), hour_lift_h1=d.next_hour[m & (half == 1)].mean() / h1, hour_lift_h2=d.next_hour[m & (half == 2)].mean() / h2,
|
||||
share_of_all_bursts=100 * d.burst[m].sum() / d.burst.sum()))
|
||||
print(pd.DataFrame(rows).round(2).to_string(index=False))
|
||||
hot = d[d.level == "hot"]
|
||||
kinds = pd.Series(np.where(hot.calendar, "calendar", "clock+extreme reading"), index=hot.index)
|
||||
print("hot bars by cause:", (100 * kinds.value_counts(normalize=True)).round(1).to_dict())
|
||||
hot_hours = hot.index.floor("h").unique()
|
||||
days = pd.Series(hot_hours.date).value_counts()
|
||||
print(f"distinct HOT hours per calendar day: mean {days.mean():.1f}, median {days.median():.0f}, max {days.max()} (≈ heads-up messages/day before the daily cap)")
|
||||
hb = d.burst.groupby(d.index.floor("h")).max(); hh = pd.Series(1, index=hot_hours)
|
||||
print(f"of all hours with a violent bar, the hour was flagged HOT at its start or during: {100*hb[hb==1].index.isin(hot_hours).mean():.1f}%")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 578 KiB |
@@ -1,191 +0,0 @@
|
||||
"""ForexBot2 architecture diagram → docs/architecture.png.
|
||||
|
||||
Regenerate after architecture changes: venv/bin/python docs/architecture_diagram.py
|
||||
(matplotlib only — no graphviz on the dev box)."""
|
||||
import os
|
||||
import matplotlib
|
||||
matplotlib.use("Agg")
|
||||
import matplotlib.pyplot as plt
|
||||
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
|
||||
|
||||
W, H = 100.0, 100.0
|
||||
fig, ax = plt.subplots(figsize=(16, 11.5))
|
||||
ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
|
||||
|
||||
INK = "#263238"; MUT = "#607D8B"; ARR = "#455A64"; HAUL = "#7B8FA1"
|
||||
|
||||
def box(x, y, w, h, title, lines, fc, ec, title_fs=8.2, body_fs=6.6, lw=1.3):
|
||||
ax.add_patch(FancyBboxPatch((x, y), w, h,
|
||||
boxstyle="round,pad=0.35,rounding_size=0.9",
|
||||
facecolor=fc, edgecolor=ec, linewidth=lw, zorder=2))
|
||||
cy = y + h - 1.9
|
||||
ax.text(x + w / 2, cy, title, ha="center", va="center", fontsize=title_fs,
|
||||
fontweight="bold", color=INK, zorder=3)
|
||||
cy -= 2.3
|
||||
for ln in lines:
|
||||
ax.text(x + w / 2, cy, ln, ha="center", va="center", fontsize=body_fs,
|
||||
color=INK, zorder=3)
|
||||
cy -= 1.85
|
||||
|
||||
def band(x, y, w, h, label, color):
|
||||
ax.add_patch(FancyBboxPatch((x, y), w, h,
|
||||
boxstyle="round,pad=0.3,rounding_size=1.2",
|
||||
facecolor="none", edgecolor=color, linewidth=1.0,
|
||||
linestyle=(0, (4, 3)), zorder=1))
|
||||
ax.text(x + 0.9, y + h - 1.4, label, ha="left", va="center",
|
||||
fontsize=7.2, fontweight="bold", color=color, zorder=3)
|
||||
|
||||
def arrow(p1, p2, color=ARR, lw=1.5, style="-|>", conn="arc3,rad=0.0",
|
||||
label=None, lx=0, ly=0, ls="solid", fs=6.4, rot=0):
|
||||
ax.add_patch(FancyArrowPatch(p1, p2, arrowstyle=style, mutation_scale=11,
|
||||
color=color, linewidth=lw, linestyle=ls,
|
||||
connectionstyle=conn, zorder=4))
|
||||
if label:
|
||||
mx, my = (p1[0] + p2[0]) / 2 + lx, (p1[1] + p2[1]) / 2 + ly
|
||||
ax.text(mx, my, label, fontsize=fs, color=MUT, ha="center",
|
||||
va="center", zorder=5, rotation=rot,
|
||||
bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
|
||||
|
||||
# ── title ────────────────────────────────────────────────────────────
|
||||
ax.text(2, 98.4, "ForexBot2 — XAU/USD Signal-Intelligence Platform",
|
||||
fontsize=15, fontweight="bold", color=INK, ha="left")
|
||||
ax.text(2, 96.0, "one pipeline, four delivery surfaces · closed bars only · live == backtest parity",
|
||||
fontsize=8.5, color=MUT, ha="left")
|
||||
|
||||
EXT_FC, EXT_EC = "#ECEFF1", "#90A4AE"
|
||||
DAT_FC, DAT_EC = "#E3F2FD", "#5C9BD1"
|
||||
NWS_FC, NWS_EC = "#E0F2F1", "#3D9C8F"
|
||||
COR_FC, COR_EC = "#FFFDF5", "#C9A227"
|
||||
REP_FC, REP_EC = "#FFE9A8", "#B8860B"
|
||||
DEL_FC, DEL_EC = "#E8F5E9", "#5FAF63"
|
||||
BTS_FC, BTS_EC = "#EDE7F6", "#8E6BC6"
|
||||
|
||||
# ── external services (top band) ─────────────────────────────────────
|
||||
band(1, 85.0, 98, 9.4, "EXTERNAL SERVICES", "#78909C")
|
||||
box(2.5, 86.2, 15.5, 6.4, "Oanda v20 API",
|
||||
["candles (live feed)", "practice orders"], EXT_FC, EXT_EC)
|
||||
box(19.5, 86.2, 15.5, 6.4, "Twelve Data · Tiingo",
|
||||
["registered fallbacks", "(Tiingo: tick stream)"], EXT_FC, EXT_EC)
|
||||
box(36.5, 86.2, 15.5, 6.4, "Forex Factory",
|
||||
["economic calendar", "MyFXBook RSS fallback"], EXT_FC, EXT_EC)
|
||||
box(53.5, 86.2, 12.5, 6.4, "News RSS ×4",
|
||||
["gold / macro", "headlines"], EXT_FC, EXT_EC)
|
||||
box(67.5, 86.2, 13.5, 6.4, "DeepSeek API",
|
||||
["V4-Flash LLM", "thinking: disabled"], EXT_FC, EXT_EC)
|
||||
box(82.5, 86.2, 15, 6.4, "Telegram API",
|
||||
["owner bot +", "subscriber channel"], EXT_FC, EXT_EC)
|
||||
|
||||
# ── data & context layer ─────────────────────────────────────────────
|
||||
box(2.5, 69.8, 33, 11.0, "providers/ (BaseDataProvider)",
|
||||
["single-flight fetch · TTL+boundary cache",
|
||||
"forming-candle DROP · validate/clean",
|
||||
"paginated fetch_range · optional tick stream",
|
||||
"active feed knob: data_providers.active = oanda"],
|
||||
DAT_FC, DAT_EC)
|
||||
box(38.5, 69.8, 33, 11.0, "news_engine.py + llm_client.py",
|
||||
["calendar events + keyword sentiment (gold-inverse)",
|
||||
"DeepSeek enrichment: batch headline scores,",
|
||||
"event gold-direction, market blurb — display-only,",
|
||||
"keyword fallback, daily request cap"],
|
||||
NWS_FC, NWS_EC)
|
||||
box(74.5, 69.8, 23, 11.0, "app_config.py",
|
||||
["credentials & deployment:",
|
||||
"env → config.json → refuse",
|
||||
"(no hardcoded fallbacks)",
|
||||
".env loaded at import"],
|
||||
"#FAFAFA", "#9E9E9E")
|
||||
|
||||
# ── signal core container ────────────────────────────────────────────
|
||||
ax.add_patch(FancyBboxPatch((2.5, 40.5), 95, 26.5,
|
||||
boxstyle="round,pad=0.35,rounding_size=1.2",
|
||||
facecolor=COR_FC, edgecolor=COR_EC, linewidth=1.6, zorder=1))
|
||||
ax.text(50, 65.2, "SIGNAL CORE — runs identically in live scans and backtests (closed bars only)",
|
||||
ha="center", fontsize=8.4, fontweight="bold", color="#8a6d1a", zorder=3)
|
||||
|
||||
box(4.5, 51.8, 28, 12.0, "engine.py — TechnicalEngine",
|
||||
["9 scored indicators + ATR context:",
|
||||
"EMA · RSI · MACD · BB · ADX · ROC ·",
|
||||
"Keltner · divergence · candlestick",
|
||||
"regime → adaptive weights → composite"],
|
||||
"white", COR_EC)
|
||||
box(4.5, 42.0, 28, 8.4, "pair_signals/xauusd.py — factory",
|
||||
["ALWAYS used: config.json overlay +",
|
||||
"validated per-TF deltas (hour blocks, trail…)"],
|
||||
"white", COR_EC)
|
||||
box(35.5, 42.0, 60, 20.7, "signal_aggregator.py — analyze_pair → analyze_from_dataframes",
|
||||
["multi-TF agreement (M5→[M15,H1] · M15→[M5,H1] · H1→[M15,H4] · H4→[H1,D1])",
|
||||
"→ news context (warn-only by default) → diminishing-decay penalties",
|
||||
"→ exhaustion hard block → quality grade A+…F",
|
||||
"→ ordered post-grade gate chain (session · XAU SELL vol/hour · regime · M5 noise)",
|
||||
"→ setup generators: XAU ATR (A+/A) · generic swing · XAU_PB · XAU_CP",
|
||||
"→ entry · SL · TP1/TP2 · trailing plan"],
|
||||
"white", COR_EC, body_fs=6.9)
|
||||
|
||||
# ── SignalReport hub ─────────────────────────────────────────────────
|
||||
box(31, 33.2, 38, 5.2, "SignalReport",
|
||||
["signal (−2…+2) · grade · trade setup · reasons · news summary"],
|
||||
REP_FC, REP_EC, title_fs=8.8)
|
||||
|
||||
# ── delivery surfaces ────────────────────────────────────────────────
|
||||
band(1, 13.2, 98, 17.2, "DELIVERY SURFACES", "#4C8F50")
|
||||
box(2.5, 15.0, 12, 12.6, "main.py",
|
||||
["CLI report", "--detail", "--json"], DEL_FC, DEL_EC)
|
||||
box(42.5, 15.0, 31, 12.6, "dashboard_publisher.py → Apache",
|
||||
["independent daemon · scans at candle closes",
|
||||
"atomic publish to /var/www/trade:",
|
||||
"dashboard.html · snapshot.json · news.json",
|
||||
"+ news_summary.php — on-demand AI",
|
||||
"gold-impact summary (whitelist · cache · cap)"],
|
||||
DEL_FC, DEL_EC)
|
||||
box(76, 15.0, 21.5, 12.6, "telegram_bot.py",
|
||||
["candle-boundary auto-scans", "quality threshold + dedup",
|
||||
"alerts + LLM context line", "WS level-breach monitor"],
|
||||
DEL_FC, DEL_EC)
|
||||
|
||||
# ── backtest / QA ────────────────────────────────────────────────────
|
||||
box(2.5, 3.2, 71.8, 8.0, "backtest/gpu_backtest.py — bar-by-bar replay through the SAME analyze_from_dataframes",
|
||||
["realism (results-critical): next-open entries · raw-path resolution · spread charged · SL-first ties · close-only trail",
|
||||
"CuPy GPU / NumPy fallback · parquet cache · fed by providers.fetch_range (news off)"],
|
||||
BTS_FC, BTS_EC)
|
||||
box(77.5, 3.2, 20, 8.0, "test_all.py",
|
||||
["461 tests · 31 modules", "no network (mocked)"], "#FAFAFA", "#9E9E9E")
|
||||
|
||||
# ── arrows: short structural hops (solid dark) ───────────────────────
|
||||
arrow((10, 86.2), (14, 81.2), label="OHLCV", lx=-3.4, ly=0.2) # oanda → providers
|
||||
arrow((27, 86.2), (24, 81.2), ls=(0, (3, 2)), lw=1.1) # fallbacks → providers
|
||||
arrow((44, 86.2), (48, 81.2)) # FF → news
|
||||
arrow((59.5, 86.2), (56, 81.2)) # RSS → news
|
||||
arrow((72, 86.2), (62, 81.2), label="LLM", lx=3.4, ly=0.6) # deepseek → news
|
||||
arrow((19, 69.8), (19, 64.2), label="DataFrames (UTC, closed bars)", lx=0.2, ly=0.9)
|
||||
arrow((33.5, 57.5), (35.5, 55.5)) # engine → aggregator
|
||||
arrow((32.5, 46.0), (35.5, 48.0)) # factory → aggregator
|
||||
arrow((55, 69.8), (55, 63.1), label="warn-only context", lx=0, ly=1.0)
|
||||
arrow((86, 69.8), (86, 67.3), ls=(0, (3, 2)), lw=1.1) # config → core
|
||||
arrow((50, 40.5), (50, 38.8)) # core → report
|
||||
arrow((36, 34.5), (8.5, 27.95), conn="arc3,rad=0.12") # report → cli
|
||||
arrow((60, 33.2), (58, 27.95)) # report → dashboard
|
||||
arrow((66, 34.5), (86.5, 27.95), conn="arc3,rad=-0.12") # report → telegram
|
||||
|
||||
# ── long-haul edges (thin dashed, routed through gutters) ────────────
|
||||
# telegram bot ↔ Telegram API — right-margin arc, clear of app_config
|
||||
arrow((95, 86.2), (95, 27.95), conn="arc3,rad=0.15", color=HAUL,
|
||||
lw=1.2, ls=(0, (4, 2)))
|
||||
ax.text(99.2, 57, "alerts / commands", fontsize=6.4, color=MUT,
|
||||
rotation=90, ha="center", va="center")
|
||||
# browser click → news_summary.php → DeepSeek
|
||||
arrow((66, 27.95), (73, 86.2), conn="arc3,rad=-0.18", color=HAUL,
|
||||
lw=1.2, ls=(0, (4, 2)),
|
||||
label="on-demand AI summary\n(browser click → PHP)", lx=2.5, ly=11.6)
|
||||
# providers.fetch_range → backtester — far-left margin channel
|
||||
arrow((1.5, 69.8), (1.5, 11.0), color=BTS_EC, lw=1.1, ls=(0, (3, 2)),
|
||||
label="fetch_range\n(history)", lx=3.6, ly=-1.2)
|
||||
# backtester replays the signal core — right gutter between boxes
|
||||
arrow((74.0, 11.6), (74.9, 40.3), color=HAUL, lw=1.3)
|
||||
ax.text(81.8, 38.9, "replays the exact signal code", fontsize=6.4,
|
||||
color=MUT, ha="center", va="center", zorder=5,
|
||||
bbox=dict(boxstyle="round,pad=0.15", fc="white", ec="none", alpha=0.9))
|
||||
|
||||
if __name__ == "__main__": # importing the module must never rewrite the PNG
|
||||
out = os.path.join(os.path.dirname(os.path.abspath(__file__)), "architecture.png")
|
||||
fig.savefig(out, dpi=200, bbox_inches="tight", facecolor="white")
|
||||
print(f"saved {out}")
|
||||
@@ -1,92 +0,0 @@
|
||||
# Test Plan — Data Source Change Validation
|
||||
|
||||
**Purpose:** determine whether switching `data_providers.active` (e.g. Twelve Data → Oanda, or any future provider swap) changes signal calculations materially, and gate the switch on evidence.
|
||||
|
||||
**Status:** in progress, 2026-08-14
|
||||
|
||||
> **Progress (2026-08-14):** Phase 0 ✅, Phase 1 ✅ (`backtest/research/feed_divergence_2026-08-14.md`), Phase 2 ✅ (`backtest/research/signal_parity_2026-08-14.md`), Phase 3 ✅ (`backtest/research/repin_verification_2026-08-14.md` — re-pin already existed), Phase 4 🔨 tool ready (`scripts/shadow_scan.py`, needs a 2-week run — launch manually per owner decision), Phase 5 owner decisions recorded (dual cost reporting; Tiingo streaming-only; see bottom).
|
||||
**Context:** Oanda has been the live feed since 2026-07-28. The backtest books were **re-pinned on Oanda bars on 2026-08-04** (`backtest/oanda_*.csv` — 27 books, now the baseline; all TD-era books superseded). Cost reporting is dual: $0.30 retained for cross-book comparability, $0.65 is the realistic headline (measured Oanda round trip $0.65–0.72). Twelve Data was proven signal-incompatible in Phase 1 (~19% of M5 bars disagree on direction; systematic ~$1 bias) — it is NOT a valid candle fallback.
|
||||
|
||||
**Why signals can differ between feeds** (mechanisms to test):
|
||||
|
||||
- Feeds publish different mid prices → tiny OHLC diffs shift every indicator; near a threshold (RSI band, `FB2_EMA_LO` EMA200-proximity band, composite/grade cutoffs, M5 min-score gate) one bar's discrete score flips.
|
||||
- Gate edge cases turn a tiny price diff into "setup generated" vs "no signal".
|
||||
- ATR-derived SL/TP/trail levels differ even when the signal is identical.
|
||||
- The shared 10σ cleaning pass can drop different bars per feed.
|
||||
- Volume present (Oanda) vs absent (Tiingo FX); real spread (Oanda MBA) vs pair default (Tiingo).
|
||||
- Known reference point: H4 seeding divergence — same provider, different fetch depth flipped ~2% of indicator signals (composite Δ up to 12.3). Cross-feed diffs act the same way.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Recent-bar parity smoke test ✅ DONE
|
||||
|
||||
- **Tool:** `scripts/compare_oanda_tiingo.py` (removed 2026-09-06 once Phases 1-2 superseded it; the result is recorded here)
|
||||
- **Result 2026-08-14:** 10/10 timestamp alignment on M5/M15/H1, Close max |Δ| ≤ $0.175 (≤ 0.004% of price), no directional bias.
|
||||
- **Verdict:** recent-tail parity confirmed. Does NOT cover historical depth — proceed to Phase 1.
|
||||
|
||||
## Phase 1 — Historical candle divergence study
|
||||
|
||||
**Goal:** quantify raw-data divergence over the full window used for validation, not just the recent tail.
|
||||
|
||||
- [x] Extend the comparison script to use `fetch_range` (paginated, raw bars) for both providers over **≥ 12 months** for M5, M15, H1 (and H4 if in scope).
|
||||
- [x] Record per TF:
|
||||
- % of bar timestamps present in both feeds; list of missing/extra bars (gaps, holidays, weekend boundary handling).
|
||||
- OHLC |Δ| distribution: p50 / p95 / p99 / max, in $ and % of price.
|
||||
- Count of bars dropped by `validate_and_clean` on each feed (10σ outliers, malformed).
|
||||
- Bar-close timestamp conventions (alignment, DST behavior around US transitions).
|
||||
- [x] **Pass criteria (measured on the weekend-filtered bar grid — the aggregator's NY-anchored `_is_forex_weekend` decides which bars exist for signal purposes, so raw timestamp overlap includes coverage-only diffs like Tiingo's Sunday bars and is NOT the criterion):** ≥ 99.5% overlap of *signal-eligible* bar timestamps; Close p95 |Δ| ≤ 0.01% of price; no systematic bias (mean Δ ≈ 0); no unexplained multi-hour gaps unique to one feed. (Raw-overlap result 2026-08-14: 94.6–96.6% vs Tiingo — entirely explained by Tiingo's ~22h/week of flat Sunday bars, which the weekend filter gates out.)
|
||||
- [x] Write findings to `backtest/research/feed_divergence_<date>.md` with tables and the exact commands used.
|
||||
|
||||
## Phase 2 — Signal-level replay parity
|
||||
|
||||
**Goal:** measure how often candle divergence actually changes a *decision*.
|
||||
|
||||
- [x] Replay bar-by-bar through the real `analyze_from_dataframes` (news off) for both feeds over the same ≥ 12-month window, per TF (M5, M15, H1). Reuse the gpu_backtest replay harness; disable the parquet cache (`--no-cache`) or use separate cache keys per provider.
|
||||
- [x] Compare per bar: signal direction, composite score, quality grade, gate outcomes (which gate fired), setup presence, strategy label (XAU ATR / XAU_PB / XAU_CP), entry/SL/TP prices.
|
||||
- [x] Metrics:
|
||||
- % bars with identical direction+grade.
|
||||
- Direction flips, grade changes, gate-only changes (setup appears/disappears).
|
||||
- Distribution of composite Δ on mismatched bars.
|
||||
- For matching signals: mean/max |Δ| of entry, SL, TP1/TP2 in $ and in ATR fractions.
|
||||
- [x] **Result (Oanda↔Tiingo):** 98.32% / 98.16% same direction (M15/M5), zero opposite-sign flips; 93.41% / 94.21% same direction+grade+setup. The original ≥97% direction+grade proposal is too strict for benign grade-letter changes and was not met; fallback acceptance is direction ≥98%, zero sign flips, and marginal/boundary-hour-only flips. See `signal_parity_2026-08-14.md`.
|
||||
|
||||
## Phase 3 — Backtest re-pin on the new feed
|
||||
|
||||
**Goal:** a new pinned book on the new provider's bars, evaluated under rule 1 acceptance criteria.
|
||||
|
||||
- [x] **Provider selection:** `gpu_backtest.py` has **no `--provider` flag** — it builds the provider from `data_providers.active` in config.json (`gpu_backtest.py:1753`). A cross-provider run requires flipping that config line. The parquet cache is already keyed per provider (`gpu_backtest.py:984` — `oanda_`-prefixed files vs legacy unprefixed TD files), so TD and Oanda data cannot mix in cache; `--no-cache` is only needed to force a refetch.
|
||||
- [x] Run the full backtest on new-feed bars, both cost assumptions:
|
||||
```bash
|
||||
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.30 --export-csv backtest/repin_<provider>_m15_030.csv
|
||||
python backtest/gpu_backtest.py --pair XAUUSD --days 365 --spread 0.65 --export-csv backtest/repin_<provider>_m15_065.csv
|
||||
```
|
||||
(repeat for each TF under test; note the `Backend:` log line and provider in the report per `backtest/instructions_for_backtest.txt`).
|
||||
- [x] Split into train/validation halves; require **positive total R in both halves and PF ≥ 1.3 at realistic cost** (rule 1), with trailing exits and next-open fills untouched.
|
||||
- [x] Marginal configs additionally pass the **2024-06 → 2025-03 OOS window** (within-year splits alone have approved configs that later failed OOS).
|
||||
- [x] ~~Compare against the pinned Twelve-Data book trade-by-trade~~ **AMENDED 2026-08-14:** skipped with justification — Phase 1 showed TD bars disagree on direction 19% of the time on M5 with a systematic ~$1 bias, so a TD-pinned book is a different instrument's book and a trade-level diff carries no decision value. Phantom-trade classes remain documented (e.g. the weekend-calendar fix already dropped exactly one live-impossible M5 trade).
|
||||
- [x] **Deliverable:** re-pinned book + report in `backtest/research/`; update the reference numbers in CLAUDE.md if the book is adopted. Current-HEAD M5 canonical book: `backtest/oanda_m5_12mo_065_nopb_20260814.csv`, +106.82R PF 1.51.
|
||||
|
||||
## Phase 4 — Live shadow comparison
|
||||
|
||||
**Goal:** confirm parity under real fetch depths, caching, and boundary timing — not just history.
|
||||
|
||||
- [ ] Run a shadow scan loop (no orders, no Telegram) for **≥ 2 weeks**: each candle close, scan with both providers, log direction/grade/gates/setup to a side-by-side file or SQLite table.
|
||||
- [ ] Include at least one weekend transition and one news-heavy day in the window.
|
||||
- [ ] Track: scan-to-scan agreement rate, feed-lag incidents ("feed lagging — just-closed bar missing"), staleness warnings, spread readings (Oanda real vs Tiingo default).
|
||||
- [ ] **Acceptance thresholds (baseline = Phase 2 replay parity):** ≥ 98% direction agreement; **zero** opposite-sign (BUY↔SELL) flips; direction flips confined to actionable↔NEUTRAL at marginal composites, clustered only at Friday-close/Sunday-reopen (UTC 22–23) and Asian (03–04) hours; setup-level drift (SL/TP) within the Phase 2 envelope (mean |Δ| ≲ $3, p95 ≲ $17 on M15); every feed-lag/staleness incident explained.
|
||||
- [ ] H4 caution: live H4 fetches only 380 bars vs unbounded backtest history (known ~2% indicator-signal flip from seeding alone) — keep H4 labeled "unproven" until a dedicated campaign closes this.
|
||||
|
||||
## Phase 5 — Cutover checklist
|
||||
|
||||
- [x] `config.json` `data_providers.active: "oanda"` and `data_providers.streaming: "tiingo"` set deliberately (verified 2026-08-14 — Oanda cannot stream; Tiingo streams).
|
||||
- [x] Cost model: **dual reporting adopted (owner decision 2026-08-14)** — $0.65 (measured Oanda round trip) is the realistic headline, $0.30 retained for cross-book comparability with TD-era books. ⚠️ **Cost-gate coupling:** `config.py round_trip_cost` (0.30) is NOT just a backtest knob — it feeds the live TP1 spread gate (`min_tp = round_trip_cost × min_tp_cost_multiple`, three sites in `signal_aggregator.py`). Raising it to 0.65 moves the TP1 floor from ~$9 to ~$19.50 and is a rule-1 revalidation event, not a config tweak. `round_trip_cost` stays 0.30.
|
||||
- [x] QA suite green: `python test_all.py` (381 tests; verified 2026-08-14).
|
||||
- [ ] Week 1 post-cutover: daily review of `logs/signals.db` rows vs shadow log; alert on grade/direction disagreement beyond the Phase 2 baseline rate.
|
||||
- [ ] Rollback: revert `data_providers.active`; pinned books of the previous feed remain valid for comparison. ⚠️ Twelve Data is a permanent *registered* fallback but is **signal-incompatible** (Phase 1) — on an Oanda candle outage, stop scanning rather than switching `active` to `twelve_data`; a TD-fed book cannot be compared to the Oanda baseline.
|
||||
- [x] Docs sync (CLAUDE.md rule): `docs/user_guide.html` backtest numbers reflect the Oanda-pinned books, not TD-era numbers.
|
||||
|
||||
## Open questions — RESOLVED 2026-08-14 (owner decisions)
|
||||
|
||||
- ~~Adopt which spread as canonical~~ → **dual reporting**: $0.65 realistic headline + $0.30 cross-book comparability; `round_trip_cost` stays 0.30 (gate coupling above).
|
||||
- ~~Tiingo role~~ → **streaming-only**. Tiingo remains registered for the WS tick stream and comparison research, but is not a designated candle fallback; candle fallback is a manual halt-and-decide, never an automatic TD failover.
|
||||
- H4 seeding campaign (parked since 2026-08-04) → **still needs an owner**; H4 stays "unproven" until then.
|
||||
@@ -1,259 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Phase 1 — historical candle divergence study (data source change test)
|
||||
=======================================================================
|
||||
Fetches the RAW closed-bar series for XAU/USD from two providers over a
|
||||
multi-month window via the normal provider stack (`fetch_range`, which
|
||||
paginates under the per-request bar cap and drops the forming candle),
|
||||
then quantifies how the feeds diverge:
|
||||
|
||||
* coverage: % shared timestamps, feed-only bars, where they cluster
|
||||
* OHLC |Δ| distribution (mean/median/p90/p95/p99/max, $ and bias)
|
||||
* |ΔClose| vs mean ATR14 (the scale the signal engine cares about)
|
||||
* threshold exceedance (how many bars differ by more than X $)
|
||||
* bar-direction agreement (sign of close−open)
|
||||
* close-return correlation on consecutive bars
|
||||
* per-hour and per-month drift
|
||||
|
||||
Both feeds are mid-price (Oanda mid of MBA candles; Tiingo resampled mid;
|
||||
Twelve Data its own mid). Frames are cached to parquet so re-runs are
|
||||
instant; pass --no-cache to force a fresh fetch.
|
||||
|
||||
Usage:
|
||||
venv/bin/python scripts/feed_divergence_study.py --a oanda --b tiingo \
|
||||
--tfs M15,H1,H4 --days 365
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from config import PlatformConfig, Timeframe, XAUUSD
|
||||
from providers import make_data_provider
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
CACHE_DIR = REPO / "backtest" / "cache"
|
||||
RESEARCH_DIR = REPO / "backtest" / "research"
|
||||
|
||||
_TF_LOOKUP = {t.value: t for t in Timeframe}
|
||||
_TF_LOOKUP.update({
|
||||
"M1": Timeframe.M1, "M5": Timeframe.M5, "M15": Timeframe.M15,
|
||||
"M30": Timeframe.M30, "H1": Timeframe.H1, "H4": Timeframe.H4,
|
||||
"D1": Timeframe.D1, "W1": Timeframe.W1,
|
||||
})
|
||||
|
||||
|
||||
def wilder_atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
|
||||
hl = df["High"] - df["Low"]
|
||||
hc = (df["High"] - df["Close"].shift(1)).abs()
|
||||
lc = (df["Low"] - df["Close"].shift(1)).abs()
|
||||
tr = pd.concat([hl, hc, lc], axis=1).max(axis=1)
|
||||
return tr.ewm(alpha=1.0 / period, adjust=False).mean()
|
||||
|
||||
|
||||
def fmt_stats(name: str, s: pd.Series) -> str:
|
||||
a = s.abs()
|
||||
return (f" {name:<6} mean ${a.mean():.3f} med ${a.median():.3f} "
|
||||
f"p90 ${a.quantile(0.90):.3f} p95 ${a.quantile(0.95):.3f} "
|
||||
f"p99 ${a.quantile(0.99):.3f} max ${a.max():.2f} "
|
||||
f"bias {s.mean():+.4f}")
|
||||
|
||||
|
||||
def cache_path(name: str, tf: Timeframe, start: pd.Timestamp,
|
||||
end: pd.Timestamp) -> Path:
|
||||
return (CACHE_DIR /
|
||||
f"{name}_XAU_USD_{tf.value}_{start:%Y%m%d}_{end:%Y%m%d}.parquet")
|
||||
|
||||
|
||||
def load_or_fetch(provider, name: str, tf: Timeframe, start, end,
|
||||
no_cache: bool):
|
||||
cp = cache_path(name, tf, start, end)
|
||||
if not no_cache and cp.exists():
|
||||
df = pd.read_parquet(cp)
|
||||
print(f" [{name}] loaded {len(df):,} bars from cache "
|
||||
f"{cp.name}")
|
||||
return df
|
||||
t0 = time.time()
|
||||
print(f" [{name}] fetching {tf.value} {start.date()} → {end.date()} ...")
|
||||
df = provider.fetch_range(XAUUSD, tf, start, end)
|
||||
print(f" [{name}] fetched {len(df):,} bars in {time.time() - t0:.1f}s")
|
||||
df.to_parquet(cp)
|
||||
print(f" [{name}] cached → {cp.name}")
|
||||
return df
|
||||
|
||||
|
||||
def compare(tf: Timeframe, a_name: str, b_name: str, a: pd.DataFrame,
|
||||
b: pd.DataFrame):
|
||||
print("\n" + "=" * 80)
|
||||
print(f"{tf.value} — {a_name.upper()} vs {b_name.upper()}, XAU/USD")
|
||||
print("=" * 80)
|
||||
|
||||
common = a.index.intersection(b.index)
|
||||
a_only = a.index.difference(b.index)
|
||||
b_only = b.index.difference(a.index)
|
||||
|
||||
print(f"\nCOVERAGE")
|
||||
print(f" {a_name} bars : {len(a):>8,} head {a.index[0]} tail {a.index[-1]}")
|
||||
print(f" {b_name} bars : {len(b):>8,} head {b.index[0]} tail {b.index[-1]}")
|
||||
print(f" shared : {len(common):>8,} "
|
||||
f"({100 * len(common) / max(len(a.index.union(b.index)), 1):.2f}% of union)")
|
||||
print(f" {a_name}-only : {len(a_only):>8,} "
|
||||
f"({100 * len(a_only) / max(len(a), 1):.2f}% of {a_name})")
|
||||
print(f" {b_name}-only : {len(b_only):>8,} "
|
||||
f"({100 * len(b_only) / max(len(b), 1):.2f}% of {b_name})")
|
||||
|
||||
if len(common) == 0:
|
||||
print(" !! no shared timestamps — feeds on different bar grids")
|
||||
return
|
||||
|
||||
for label, idx in ((f"{a_name}-only", a_only),
|
||||
(f"{b_name}-only", b_only)):
|
||||
if not len(idx):
|
||||
continue
|
||||
by_date = pd.Series(1, index=idx).groupby(idx.date).sum()
|
||||
top = by_date.sort_values(ascending=False).head(6)
|
||||
print(f" {label} clusters on: "
|
||||
+ ", ".join(f"{d} ({n})" for d, n in top.items()))
|
||||
|
||||
x = a.loc[common]
|
||||
y = b.loc[common]
|
||||
|
||||
atr = wilder_atr(x).dropna()
|
||||
mean_price = x["Close"].mean()
|
||||
print(f"\nSCALE REFERENCES ({a_name} feed)")
|
||||
print(f" mean close : ${mean_price:,.2f}")
|
||||
print(f" mean bar range: ${(x['High'] - x['Low']).mean():.3f}")
|
||||
print(f" mean ATR14 : ${atr.mean():.3f} median ${atr.median():.3f}")
|
||||
|
||||
print(f"\nPER-BAR DIFFS ({a_name} − {b_name}, $)")
|
||||
diffs = {}
|
||||
for col in ("Open", "High", "Low", "Close"):
|
||||
d = x[col] - y[col]
|
||||
diffs[col] = d
|
||||
print(fmt_stats(col, d))
|
||||
|
||||
dclose = diffs["Close"].abs()
|
||||
print(f"\n |ΔClose| as % of mean ATR14 : "
|
||||
f"mean {100 * dclose.mean() / atr.mean():.1f}% "
|
||||
f"p95 {100 * dclose.quantile(0.95) / atr.mean():.1f}%")
|
||||
|
||||
print(f"\nTHRESHOLD EXCEEDANCE (|ΔClose|)")
|
||||
for thr in (0.10, 0.30, 0.65, 1.00, 2.00, 5.00):
|
||||
pct = 100 * (dclose > thr).mean()
|
||||
print(f" > ${thr:<5.2f}: {pct:6.2f}% of bars ({(dclose > thr).sum():,})")
|
||||
|
||||
# direction agreement
|
||||
a_dir = np.sign(x["Close"] - x["Open"])
|
||||
b_dir = np.sign(y["Close"] - y["Open"])
|
||||
both_nonflat = (a_dir != 0) & (b_dir != 0)
|
||||
dir_agree = 100 * (a_dir[both_nonflat] == b_dir[both_nonflat]).mean()
|
||||
print(f"\nAGREEMENT")
|
||||
print(f" bar direction (sign of close−open) agree: {dir_agree:.2f}%"
|
||||
f" (n={both_nonflat.sum():,})")
|
||||
|
||||
step = pd.Series(common, index=common).diff().median()
|
||||
a_ret = x["Close"].pct_change()
|
||||
b_ret = y["Close"].pct_change()
|
||||
step_ok = pd.Series(common, index=common).diff().le(step + pd.Timedelta(seconds=1))
|
||||
corr = a_ret[step_ok].corr(b_ret[step_ok])
|
||||
print(f" close-return correlation (consecutive bars): {corr:.5f}")
|
||||
|
||||
# by-hour
|
||||
hours = common.hour
|
||||
rows = []
|
||||
for h in range(24):
|
||||
m = hours == h
|
||||
if not m.any():
|
||||
continue
|
||||
rows.append({"hour_utc": h, "bars": int(m.sum()),
|
||||
"mean_abs_dclose": dclose[m].mean(),
|
||||
"p95_abs_dclose": dclose[m].quantile(0.95)})
|
||||
by_hour = pd.DataFrame(rows)
|
||||
print(f"\nBY UTC HOUR (mean |ΔClose|, $)")
|
||||
print(by_hour.to_string(index=False,
|
||||
float_format=lambda v: f"{v:.3f}"))
|
||||
|
||||
print(f"\nBY MONTH (mean |ΔClose|, $)")
|
||||
months = dclose.groupby(common.tz_localize(None).to_period("M")).mean()
|
||||
for p, v in months.items():
|
||||
print(f" {p}: {v:.3f}")
|
||||
|
||||
return {
|
||||
"tf": tf.value, "a": a_name, "b": b_name,
|
||||
"a_bars": len(a), "b_bars": len(b), "common": len(common),
|
||||
"a_only": len(a_only), "b_only": len(b_only),
|
||||
"dir_agree_pct": round(dir_agree, 3),
|
||||
"ret_corr": round(corr, 5),
|
||||
"mean_abs_dclose": dclose.mean(),
|
||||
"p95_abs_dclose": dclose.quantile(0.95),
|
||||
"max_abs_dclose": dclose.max(),
|
||||
"dclose_pct_atr_mean": 100 * dclose.mean() / atr.mean(),
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--a", required=True)
|
||||
ap.add_argument("--b", required=True)
|
||||
ap.add_argument("--tfs", default="M15,H1,H4",
|
||||
help="comma-separated timeframe values (default M15,H1,H4)")
|
||||
ap.add_argument("--days", type=int, default=365)
|
||||
ap.add_argument("--end", default=None, help="UTC end date (default now)")
|
||||
ap.add_argument("--no-cache", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
tfs = []
|
||||
for s in args.tfs.split(","):
|
||||
s = s.strip()
|
||||
tf = _TF_LOOKUP.get(s)
|
||||
if tf is None:
|
||||
sys.exit(f"unknown timeframe '{s}'; use one of "
|
||||
f"{sorted(_TF_LOOKUP)}")
|
||||
tfs.append(tf)
|
||||
|
||||
end = pd.Timestamp(args.end, tz="UTC") if args.end \
|
||||
else pd.Timestamp.now(tz="UTC")
|
||||
start = end - pd.Timedelta(days=args.days)
|
||||
|
||||
cfg = PlatformConfig()
|
||||
pa = make_data_provider(cfg, args.a)
|
||||
pb = make_data_provider(cfg, args.b)
|
||||
|
||||
summary = []
|
||||
for tf in tfs:
|
||||
print(f"\n{'─' * 80}\nWINDOW {start.date()} → {end.date()} "
|
||||
f"({args.days} days), {tf.value}")
|
||||
try:
|
||||
a = load_or_fetch(pa, args.a, tf, start, end, args.no_cache)
|
||||
except Exception as e:
|
||||
print(f" !! {args.a} {tf.value} fetch failed: {e}")
|
||||
continue
|
||||
try:
|
||||
b = load_or_fetch(pb, args.b, tf, start, end, args.no_cache)
|
||||
except Exception as e:
|
||||
print(f" !! {args.b} {tf.value} fetch failed: {e}")
|
||||
continue
|
||||
r = compare(tf, args.a, args.b, a, b)
|
||||
if r:
|
||||
summary.append(r)
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("SUMMARY TABLE")
|
||||
print("=" * 80)
|
||||
sdf = pd.DataFrame(summary)
|
||||
print(sdf.to_string(index=False,
|
||||
float_format=lambda v: f"{v:.4f}"))
|
||||
out = RESEARCH_DIR / f"feed_divergence_{args.a}_vs_{args.b}.csv"
|
||||
if len(sdf):
|
||||
sdf.to_csv(out, index=False, float_format="%.6f")
|
||||
print(f"\nSummary written to {out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,246 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Phase 2 — signal-level replay parity (data source change test)
|
||||
===============================================================
|
||||
Replays the REAL decision core (`SignalAggregator.analyze_from_dataframes`,
|
||||
news off, use_news=False) over the same window for two providers and
|
||||
compares the *decisions* bar-by-bar: direction, composite/final score,
|
||||
quality grade, gate/setup presence, strategy tag, and entry/SL/TP levels.
|
||||
|
||||
This turns the Phase-1 price-diff stats into flip rates. Feeds are read
|
||||
from the Phase-1 parquet cache (scripts/feed_divergence_study.py writes
|
||||
them); pass --no-cache-fetch only if you want to fetch fresh via
|
||||
fetch_range first.
|
||||
|
||||
Slicing matches gpu_backtest.py exactly: decision at close of bar `pos`,
|
||||
each TF sliced to bars fully closed by that decision time, primary at the
|
||||
shared grid, so the two feeds are compared on identical timestamps.
|
||||
|
||||
Usage:
|
||||
venv/bin/python scripts/signal_parity_study.py --a oanda --b tiingo \
|
||||
--tf M15 --stride 8 --days 365
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from config import Timeframe, XAUUSD
|
||||
from pair_signals.xauusd import make_config
|
||||
from signal_aggregator import SignalAggregator
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
CACHE_DIR = REPO / "backtest" / "cache"
|
||||
RESEARCH_DIR = REPO / "backtest" / "research"
|
||||
|
||||
BAR_SECONDS = {
|
||||
Timeframe.M1: 60, Timeframe.M5: 300, Timeframe.M15: 900,
|
||||
Timeframe.M30: 1800, Timeframe.H1: 3600, Timeframe.H4: 14400,
|
||||
Timeframe.D1: 86400, Timeframe.W1: 604800,
|
||||
}
|
||||
_TF_LOOKUP = {t.value: t for t in Timeframe}
|
||||
_TF_LOOKUP.update({"M1": Timeframe.M1, "M5": Timeframe.M5,
|
||||
"M15": Timeframe.M15, "M30": Timeframe.M30,
|
||||
"H1": Timeframe.H1, "H4": Timeframe.H4,
|
||||
"D1": Timeframe.D1, "W1": Timeframe.W1})
|
||||
_CONFIRMS = {
|
||||
Timeframe.M5: [Timeframe.M15, Timeframe.H1],
|
||||
Timeframe.M15: [Timeframe.M5, Timeframe.H1],
|
||||
Timeframe.H1: [Timeframe.M15, Timeframe.H4],
|
||||
Timeframe.H4: [Timeframe.H1, Timeframe.D1],
|
||||
}
|
||||
|
||||
|
||||
def load_cached(name: str, tf: Timeframe, start: pd.Timestamp,
|
||||
end: pd.Timestamp) -> pd.DataFrame:
|
||||
cp = CACHE_DIR / f"{name}_XAU_USD_{tf.value}_{start:%Y%m%d}_{end:%Y%m%d}.parquet"
|
||||
if not cp.exists():
|
||||
sys.exit(f"missing cache {cp}; run scripts/feed_divergence_study.py "
|
||||
f"--a oanda --b {name} --tfs {tf.value} --days ... first")
|
||||
return pd.read_parquet(cp)
|
||||
|
||||
|
||||
def tag_of(report) -> str:
|
||||
first_bull = report.bullish_reasons[0] if report.bullish_reasons else ""
|
||||
first_bear = report.bearish_reasons[0] if report.bearish_reasons else ""
|
||||
if "XAU PULLBACK" in first_bull or "XAU PULLBACK" in first_bear:
|
||||
return "XAU_PB"
|
||||
if "XAU CHART PATTERN" in first_bull or "XAU CHART PATTERN" in first_bear:
|
||||
return "XAU_CP"
|
||||
if "XAU STRATEGY" in first_bull:
|
||||
return "XAU"
|
||||
return "GENERIC"
|
||||
|
||||
|
||||
def decide(agg, dfs, tss, stride, warmup) -> list:
|
||||
"""Replay decisions over the shared grid, returning one row per bar."""
|
||||
primary = dfs[agg.config.primary_timeframe]
|
||||
dur = pd.Timedelta(seconds=BAR_SECONDS[agg.config.primary_timeframe])
|
||||
|
||||
# positions of sampled bars in the primary frame
|
||||
primary_pos = primary.index.get_indexer(tss)
|
||||
rows = []
|
||||
for i in range(0, len(tss), stride):
|
||||
ts = tss[i]
|
||||
pos = int(primary_pos[i])
|
||||
if pos < warmup:
|
||||
continue
|
||||
slices = {}
|
||||
for tf, full in dfs.items():
|
||||
cutoff = ts # decision_time = ts + dur; bars closed by ts
|
||||
cut = int(full.index.searchsorted(cutoff, side="right"))
|
||||
if cut <= 0:
|
||||
continue
|
||||
slices[tf] = full.iloc[:cut]
|
||||
if agg.config.primary_timeframe not in slices:
|
||||
continue
|
||||
r = agg.analyze_from_dataframes(XAUUSD, slices, use_news=False)
|
||||
ts_setup = r.trade_setup
|
||||
rows.append({
|
||||
"ts": ts,
|
||||
"direction": r.final_signal.value,
|
||||
"final_score": r.final_score,
|
||||
"primary_score": r.primary_score,
|
||||
"grade": r.signal_quality,
|
||||
"has_setup": ts_setup is not None,
|
||||
"tag": tag_of(r) if ts_setup else "",
|
||||
"entry": ts_setup.entry_price if ts_setup else np.nan,
|
||||
"sl": ts_setup.stop_loss if ts_setup else np.nan,
|
||||
"tp1": ts_setup.take_profit_1 if ts_setup else np.nan,
|
||||
"tp2": ts_setup.take_profit_2 if ts_setup else np.nan,
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--a", required=True)
|
||||
ap.add_argument("--b", required=True)
|
||||
ap.add_argument("--tf", default="M15")
|
||||
ap.add_argument("--stride", type=int, default=1,
|
||||
help="sample every Nth shared bar (default 1)")
|
||||
ap.add_argument("--days", type=int, default=365)
|
||||
ap.add_argument("--end", default=None)
|
||||
ap.add_argument("--warmup", type=int, default=300)
|
||||
args = ap.parse_args()
|
||||
|
||||
tf = _TF_LOOKUP.get(args.tf)
|
||||
if tf is None:
|
||||
sys.exit(f"unknown tf {args.tf}")
|
||||
confirms = _CONFIRMS[tf]
|
||||
tfs = [tf] + confirms
|
||||
|
||||
end = pd.Timestamp(args.end, tz="UTC") if args.end \
|
||||
else pd.Timestamp.now(tz="UTC")
|
||||
start = end - pd.Timedelta(days=args.days)
|
||||
|
||||
cfg = make_config(tf, confirms)
|
||||
agg = SignalAggregator(cfg)
|
||||
|
||||
a_dfs, b_dfs = {}, {}
|
||||
for t in tfs:
|
||||
a_dfs[t] = load_cached(args.a, t, start, end)
|
||||
b_dfs[t] = load_cached(args.b, t, start, end)
|
||||
|
||||
# shared grid = primary timestamps present in both feeds
|
||||
shared = a_dfs[tf].index.intersection(b_dfs[tf].index)
|
||||
|
||||
t0 = time.time()
|
||||
print(f"replaying {len(shared)} shared {tf.value} bars "
|
||||
f"(stride {args.stride} → ~{len(shared)//args.stride}) on {args.a}...")
|
||||
a_rows = decide(agg, a_dfs, shared, args.stride, args.warmup)
|
||||
print(f" {args.a}: {len(a_rows)} decisions in {time.time()-t0:.1f}s")
|
||||
t0 = time.time()
|
||||
b_rows = decide(agg, b_dfs, shared, args.stride, args.warmup)
|
||||
print(f" {args.b}: {len(b_rows)} decisions in {time.time()-t0:.1f}s")
|
||||
|
||||
A = pd.DataFrame(a_rows).set_index("ts")
|
||||
B = pd.DataFrame(b_rows).set_index("ts")
|
||||
common = A.index.intersection(B.index)
|
||||
A, B = A.loc[common], B.loc[common]
|
||||
n = len(common)
|
||||
|
||||
print(f"\n{'=' * 78}")
|
||||
print(f"SIGNAL-LEVEL PARITY — {tf.value} — {args.a} vs {args.b} "
|
||||
f"({n:,} bars)")
|
||||
print("=" * 78)
|
||||
|
||||
def pct(x): return f"{100 * x:.2f}%"
|
||||
|
||||
same_dir = (A["direction"] == B["direction"])
|
||||
same_grade = (A["grade"] == B["grade"])
|
||||
same_setup = (A["has_setup"] == B["has_setup"])
|
||||
same_all = same_dir & same_grade & same_setup
|
||||
|
||||
print(f"\nIDENTITY")
|
||||
print(f" same direction : {pct(same_dir.mean())}")
|
||||
print(f" same grade : {pct(same_grade.mean())}")
|
||||
print(f" same setup presence : {pct(same_setup.mean())}")
|
||||
print(f" same dir+grade+setup : {pct(same_all.mean())}")
|
||||
|
||||
both_sig = (A["direction"] != 0) & (B["direction"] != 0)
|
||||
flips = both_sig & (A["direction"] != B["direction"])
|
||||
opp = flips & (A["direction"] == -B["direction"])
|
||||
print(f"\nDIRECTION")
|
||||
print(f" bars both actionable : {both_sig.sum():,} "
|
||||
f"({pct(both_sig.mean())})")
|
||||
print(f" direction flips : {flips.sum():,} "
|
||||
f"({pct(flips.sum()/max(both_sig.sum(),1))} of actionable)")
|
||||
print(f" opposite-sign flips : {opp.sum():,}")
|
||||
|
||||
grade_a = set(A.loc[A["has_setup"], "grade"])
|
||||
print(f"\nGRADE (of {args.a} setups, grades seen: {sorted(grade_a)})")
|
||||
changed = A.loc[both_sig, "grade"] != B.loc[both_sig, "grade"]
|
||||
print(f" grade changed on actionable bars: "
|
||||
f"{changed.sum():,} ({pct(changed.mean())})")
|
||||
|
||||
# score deltas on shared bars
|
||||
ds = (A["final_score"] - B["final_score"]).abs()
|
||||
dp = (A["primary_score"] - B["primary_score"]).abs()
|
||||
print(f"\nSCORE Δ ({args.a} − {args.b})")
|
||||
print(f" final_score mean {ds.mean():6.2f} p95 {ds.quantile(.95):6.2f} "
|
||||
f"max {ds.max():6.2f}")
|
||||
print(f" primary_score mean {dp.mean():6.2f} p95 {dp.quantile(.95):6.2f} "
|
||||
f"max {dp.max():6.2f}")
|
||||
|
||||
# setup price diffs on bars where BOTH generated a setup
|
||||
both_setup = A["has_setup"] & B["has_setup"]
|
||||
if both_setup.any():
|
||||
print(f"\nSETUP LEVELS (both-setup bars: {both_setup.sum():,})")
|
||||
for col, label in (("entry", "entry"), ("sl", "SL"), ("tp1", "TP1"),
|
||||
("tp2", "TP2")):
|
||||
d = (A.loc[both_setup, col] - B.loc[both_setup, col]).abs()
|
||||
print(f" {label:<6} mean ${d.mean():6.3f} p95 ${d.quantile(.95):7.3f} "
|
||||
f"max ${d.max():7.2f}")
|
||||
|
||||
# tag disagreement on both-setup bars
|
||||
if both_setup.any():
|
||||
same_tag = (A.loc[both_setup, "tag"] == B.loc[both_setup, "tag"])
|
||||
print(f" same strategy tag : {pct(same_tag.mean())}")
|
||||
|
||||
# where do flips cluster (hour)?
|
||||
flip_hours = A.loc[flips].index.hour.value_counts().sort_index()
|
||||
if len(flip_hours):
|
||||
print(f"\nFLIPS BY UTC HOUR")
|
||||
for h, c in flip_hours.items():
|
||||
print(f" {h:02d}: {c}")
|
||||
|
||||
out = {
|
||||
"tf": tf.value, "a": args.a, "b": args.b, "bars": n,
|
||||
"same_dir": same_dir.mean(), "same_grade": same_grade.mean(),
|
||||
"same_setup": same_setup.mean(), "same_all": same_all.mean(),
|
||||
"flip_rate_actionable": flips.sum() / max(both_sig.sum(), 1),
|
||||
"mean_abs_final_score_delta": ds.mean(),
|
||||
"mean_abs_primary_score_delta": dp.mean(),
|
||||
}
|
||||
print(f"\n{out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user