chore: repo hygiene — stale gitignore entries, finished plans, superseded research, doc drift
Removed (all recoverable from git history):
- llm_provider_switch_plan.txt: 2026-08-08 DeepSeek/Anthropic multi-provider plan that
was never implemented (llm_client.py is DeepSeek-only, no `anthropic` anywhere)
- docs/superpowers/specs/2026-07-21-centralized-config-design.md: spec of the
config.json migration that merged the same day; its plan sibling was already
deleted in b936d44. app_config's docstring pointer goes with it.
- scripts/compare_oanda_tiingo.py: Phase 0 recent-tail smoke test, superseded by the
Phase 1/2 study scripts (result recorded in the test plan)
- backtest/research/fetch_oanda_m15.py + oanda_td_divergence.py + its by-hour CSV:
the 2026-07-24 Oanda-vs-Twelve-Data study, superseded by the 2026-08-14 feed
divergence report (which keeps a one-line citation)
- backtest/research/burst_profile.py and hour_analysis.py: superseded by burst_map.py
and book_report.py
- backtest/research/panel_payload_drytest_2026-08-15.py: one-off live DeepSeek probe;
its .jsonl evidence stays
Added to git: the three 2026-08-14 research reports (feed divergence, re-pin
verification, signal parity), their CSV, and docs/data_source_change_test_plan.md -
CLAUDE.md cites them as standing evidence and their driver scripts were already
committed. The test plan no longer refers to auto-execution.
Hygiene:
- .gitignore: drop oanda_state.json (daemon removed 2026-09-03) and the vanished
backtest/trailgrid dirs; ignore .coverage
- docs/architecture_diagram.py only writes the PNG when run as a script (it used
to plot on import); test box refreshed to 461 tests / 31 modules; PNG regenerated
(the committed one still drew the removed daemon)
- backtest/instructions_for_backtest.txt: manual-journal head replaced by the
actual reporting checklist; the 2026-09-06 addendum is unchanged
- doc drift: README/publisher list all six published files; news_summary.php says
4 h (not 24 h) and drops the pre-id sha1 fallback whose window closed;
analyze_signal_log lists six lenses and --scan-outcomes; agent_panel cost model
says 4 requests by default; tiingo comment cites the NY-anchored weekend;
Module 21 title no longer says "execution lifecycle"
Suite: 461 passed, 0 failed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156YhogLytzj9P2iH1skGbL
This commit is contained in:
@@ -1,116 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Scratchpad — Oanda vs Tiingo candle parity check
|
||||
=================================================
|
||||
Fetches the last N closed XAU/USD candles from Oanda and Tiingo for
|
||||
M5/M15/H1 via the normal provider stack (make_data_provider →
|
||||
fetch_ohlcv: forming-candle drop + validate/clean included, i.e. exactly
|
||||
what the signal pipeline consumes) and reports how well they line up:
|
||||
|
||||
* last-bar timestamp per provider (feed-lag check)
|
||||
* timestamp mismatches (bars present in one feed only)
|
||||
* per-bar OHLC diff table for the common tail
|
||||
* max/mean |diff| per column, with the Close diff also in % of price
|
||||
|
||||
Both feeds are mid-price (Oanda mid of MBA candles; Tiingo resampled
|
||||
mid), so small diffs are expected — this measures how small.
|
||||
|
||||
Usage:
|
||||
venv/bin/python scripts/compare_oanda_tiingo.py [--n 10]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from config import PlatformConfig, Timeframe, XAUUSD
|
||||
from providers import make_data_provider
|
||||
|
||||
TIMEFRAMES = [Timeframe.M5, Timeframe.M15, Timeframe.H1]
|
||||
OHLC = ["Open", "High", "Low", "Close"]
|
||||
|
||||
|
||||
def fetch_tail(provider, tf, n):
|
||||
"""Last n closed bars from one provider, or None on failure."""
|
||||
try:
|
||||
df = provider.fetch_ohlcv(symbol=XAUUSD, timeframe=tf)
|
||||
except Exception as e: # scratchpad: report, don't crash
|
||||
print(f" !! {provider.name} {tf.value} fetch failed: {e}")
|
||||
return None
|
||||
return df.tail(n)
|
||||
|
||||
|
||||
def compare_tf(tf, oanda_df, tiingo_df, n):
|
||||
print(f"\n{'=' * 78}\n{tf.value} — last {n} closed bars\n{'=' * 78}")
|
||||
if oanda_df is None or tiingo_df is None:
|
||||
print(" skipped (a provider failed)")
|
||||
return
|
||||
|
||||
print(f" Oanda last bar: {oanda_df.index[-1]} "
|
||||
f"Tiingo last bar: {tiingo_df.index[-1]}")
|
||||
|
||||
common = oanda_df.index.intersection(tiingo_df.index)
|
||||
only_o = oanda_df.index.difference(tiingo_df.index)
|
||||
only_t = tiingo_df.index.difference(oanda_df.index)
|
||||
if len(only_o):
|
||||
print(f" bars only in Oanda : {[str(t) for t in only_o]}")
|
||||
if len(only_t):
|
||||
print(f" bars only in Tiingo: {[str(t) for t in only_t]}")
|
||||
if not len(common):
|
||||
print(" NO common timestamps — feeds are on different bar grids!")
|
||||
return
|
||||
|
||||
o = oanda_df.loc[common, OHLC]
|
||||
t = tiingo_df.loc[common, OHLC]
|
||||
diff = o - t
|
||||
|
||||
print(f"\n {len(common)}/{n} bars share timestamps "
|
||||
f"(window {common[0]} → {common[-1]})\n")
|
||||
|
||||
# Per-bar Close table — the number that matters most.
|
||||
hdr = f" {'bar close (UTC)':<20}{'Oanda C':>12}{'Tiingo C':>12}{'ΔC ($)':>10}"
|
||||
print(hdr + f"{'ΔO':>8}{'ΔH':>8}{'ΔL':>8}")
|
||||
for ts in common:
|
||||
row = (f" {str(ts):<20}{o.loc[ts, 'Close']:>12.2f}"
|
||||
f"{t.loc[ts, 'Close']:>12.2f}{diff.loc[ts, 'Close']:>10.3f}")
|
||||
row += (f"{diff.loc[ts, 'Open']:>8.3f}{diff.loc[ts, 'High']:>8.3f}"
|
||||
f"{diff.loc[ts, 'Low']:>8.3f}")
|
||||
print(row)
|
||||
|
||||
print("\n summary over common bars:")
|
||||
for col in OHLC:
|
||||
d = diff[col].abs()
|
||||
print(f" {col:<6} max |Δ| = {d.max():8.4f} mean |Δ| = {d.mean():8.4f}")
|
||||
px = o["Close"].iloc[-1]
|
||||
dc = diff["Close"].abs()
|
||||
print(f" Close Δ as % of price: max {dc.max() / px * 100:.4f}% "
|
||||
f"mean {dc.mean() / px * 100:.4f}%")
|
||||
exact = (diff[OHLC].abs() < 1e-9).all(axis=1).sum()
|
||||
print(f" bars with identical OHLC: {exact}/{len(common)}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--n", type=int, default=10,
|
||||
help="bars per timeframe to compare (default 10)")
|
||||
args = ap.parse_args()
|
||||
|
||||
cfg = PlatformConfig()
|
||||
oanda = make_data_provider(cfg, "oanda")
|
||||
tiingo = make_data_provider(cfg, "tiingo")
|
||||
|
||||
for tf in TIMEFRAMES:
|
||||
o_df = fetch_tail(oanda, tf, args.n)
|
||||
t_df = fetch_tail(tiingo, tf, args.n)
|
||||
compare_tf(tf, o_df, t_df, args.n)
|
||||
|
||||
print("\nDone. Note: Oanda=mid of MBA candles, Tiingo=resampled mid — "
|
||||
"sub-$1 diffs on gold are normal feed divergence, not a bug.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user