harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

providers: floor every live fetch at 1000 bars (H4/D1 indicator-seeding parity)

Browse Source 
The engine's EMAs use ewm(adjust=False), so the first bar of a fetched
frame keeps weight (1-2/201)^n in the 200-bar trend EMA. With the 60-day
lookback the live H4 frame had 380 bars (seed weight 2.3%) and D1 300
(5%), while the backtester seeds from its whole cached history. Measured
on the deepest Oanda caches (267 H4 / 313 D1 bars sampled over two years,
raw and cleaned frames): ema_trend flipped on 2/267 H4 and 1/313 D1 bars
at those depths (composite off by up to 9.5 points) and on none from 600
bars up. MIN_LIVE_BARS = 1000 leaves a seed weight of e^-10; M5/M15
(5000) and H1 (1460) already exceed it and are unchanged. One request
either way. The backtester never calls fetch_ohlcv, so no book changes.

Tests: new Module 16 depth test; the Module 21 cache test's 300-bar
expectation becomes 1000. Suite 500 passed.

Claude-Session: https://claude.ai/code/session_01NQDjW8rXYrJR4JDWMWKUij
This commit is contained in:
Harwinder Sekhon
2026-09-10 07:55:20 -04:00
parent be4db8b2a7
commit 1bc6a27445
2 changed files with 52 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
providers/base.py
+21 -5
View File
@@ -96,6 +96,20 @@ class BaseDataProvider(ABC):
#: Hard per-request bar cap shared by Twelve Data and Oanda.
MAX_BARS_PER_REQUEST = 5000
#: Minimum bars per LIVE fetch on every timeframe (2026-09-10,
#: indicator-seeding parity with the backtester). The engine's EMAs
#: use ewm(adjust=False), so the first bar of the fetched frame keeps
#: weight (1-2/201)^n in the 200-bar trend EMA. With the 60-day
#: lookback, H4 fetched 380 bars (seed weight 2.3%) and D1 300 (5%)
#: while the backtester seeds from its whole cached history. Measured
#: on the deepest Oanda caches (267 H4 / 313 D1 bars sampled over two
#: years, raw and cleaned frames alike): ema_trend flipped on 2/267 H4
#: and 1/313 D1 bars at those depths (composite off by up to 9.5
#: points) and on none from 600 bars up; 1000 leaves a seed weight of
#: e^-10. M5/M15 (5000) and H1 (1460) already exceed it and are
#: unchanged. One request either way (cap 5000).
MIN_LIVE_BARS = 1000
# Grace wait before the one retry when a fresh response at a candle
# boundary is missing the just-closed bar (feed latency).
FRESH_FETCH_RETRY_SECONDS = 3.0
@@ -140,6 +154,12 @@ class BaseDataProvider(ABC):
# ── Public API ─────────────────────────────────────────────────
def _live_outputsize(self, timeframe: Timeframe, lookback_days: int) -> int:
"""Bars requested for one live fetch: the lookback in bars, floored
at MIN_LIVE_BARS (indicator seeding) and capped at the request cap."""
wanted = int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20
return max(self.MIN_LIVE_BARS, min(wanted, self.MAX_BARS_PER_REQUEST))
def fetch_ohlcv(
self,
symbol: str = XAUUSD,
@@ -151,11 +171,7 @@ class BaseDataProvider(ABC):
lookback_days = self.config.data_lookback_days
cache_key = f"{symbol}_{timeframe.value}"
outputsize = max(
300,
min(int(lookback_days * self.BARS_PER_DAY[timeframe]) + 20,
self.MAX_BARS_PER_REQUEST),
)
outputsize = self._live_outputsize(timeframe, lookback_days)
while True:
now = time.time()