providers: keep real news bars (snap-back bad-print rule), Oanda retry ladder, scan_kind=failed

2026-09-07 signal-log inspection follow-up — three fixes that do not depend on sample size.

- providers/base.py: the signal-side 10σ outlier filter is now a bad-print filter. A >10σ
  close-to-close bar is dropped only when the NEXT close snaps back ≥80% of the way
  (OUTLIER_SIGMA / OUTLIER_REVERT_FRACTION); persisting moves and the newest bar are kept.
  The unconditional rule deleted ~36 real M5 news bars/yr in the backtester (whole-frame σ)
  and a different ~24/yr live (fetch-window σ) — e.g. the 2026-09-04 12:30 UTC NFP bar
  vanished from M5+M15 and the 12:35 scan was a byte-identical repeat of 12:30. 3-yr Oanda
  drops are now M5 2 / M15 1 / H1 0.
- providers/oanda.py: transient 401/5xx/network faults retry along RETRY_DELAYS_S=(2,5,10)s
  under RETRY_BUDGET_S=45s. 11 boundary scans in two days had lost their primary frame at
  xx:00:22–:26 UTC; one 2 s retry could not outlast the top-of-hour blip.
- signal_aggregator/_log_signal: a scan whose primary fetch failed logs scan_kind='failed'
  (price 0, no signal); analyze_signal_log.parity_report skips it like 'blocked'.
- test_all.py: 466 tests (+6): retry ladder walks all delays, 5xx blip recovers on the third
  attempt, retry budget stops the ladder, persisting/newest/partially-retraced 10σ bars kept,
  failed-scan row kind, parity skip.
- CLAUDE.md: cleaning rule, retry ladder, failed scan kind, dated current-state note.

Book impact at $0.65 (rule 1 still passes everywhere, both halves positive):
  12mo  M5 557/+112.21/PF1.56 → 560/+108.81/1.54   M15 397/+73.93/1.53 → 398/+74.61/1.53   H1 identical
  3yr   M5 857/+161.3/1.53 → 870/+153.6/1.50   M15 1086/+148.7/1.37 → 1092/+151.9/1.38
        H1 255/+70.5/1.98 → 253/+70.0/1.98   H4 113/+22.7/1.63 → 114/+21.7/1.59
Only trades on the previously-deleted event bars move; the pinned M5/M15 CSVs are no longer
byte-reproducible and should be regenerated at the next re-pin.

Claude-Session: https://claude.ai/code/session_01NDnBxJ2y8GRUM6ZNFJs1hZ
This commit is contained in:
Harwinder Sekhon
2026-09-07 11:07:29 -04:00
parent 35f1eb17b8
commit d2e679cd20
7 changed files with 248 additions and 36 deletions
+10 -2
View File
@@ -305,7 +305,10 @@ class SignalAggregator:
matches the context the signal just used; if it fails, the row
is still written, just without the news JSON blobs. ``blocked``
marks weekend/day early returns: no news fetch, no snapshots —
a ~1 KB liveness row (or nothing, per signal_log.log_blocked_scans)."""
a ~1 KB liveness row (or nothing, per signal_log.log_blocked_scans).
A scan whose PRIMARY fetch failed is logged as ``scan_kind``
``failed`` (price 0, no signal): liveness evidence that readers
skip like ``blocked``, so it never pollutes NEUTRAL statistics."""
context = None
if not blocked:
try:
@@ -313,7 +316,12 @@ class SignalAggregator:
except Exception:
logger.debug("signal log: news context unavailable",
exc_info=True)
extra = {"scan_kind": "blocked" if blocked else "scan"}
kind = "blocked" if blocked else "scan"
if not blocked and dataframes is not None:
primary = dataframes.get(self.config.primary_timeframe)
if primary is None or primary.empty:
kind = "failed"
extra = {"scan_kind": kind}
for key, fingerprint in (
("config_fingerprint", self.gate_config_fingerprint),
("engine_fingerprint", self.engine_config_fingerprint)):