audit: NY-anchored weekend calendar (all 3 copies) + FB2_EMA_LO crash-proofing
Owner decisions from the 2026-08-04 audit report: - spread-in-exit-triggers: DECLINED (books stay comparable; ledger in CLAUDE.md records the measured ~4-6% optimism). - weekend clock: FIXED. The market week is Fri 17:00 -> Sun 18:00 New York time, not Fri/Sun 22:00 UTC — during US DST the Oanda feed stops at 21:00 UTC (measured: 159/162 summer weekends). All three weekend implementations (providers/base, signal_aggregator, gpu_backtest calendar filter) now share the NY-anchored rule; a new test sweeps them hourly across both DST transitions and asserts agreement. No NTP needed — the host clock was fine, the constant was wrong. Ends the weekly false-"STALE DATA / feed lagging" hours (summer Fri 21-22 UTC, winter Sun 22-23 UTC). - FB2_EMA_LO (delegated): kept as a research knob but crash-proofed — an unparseable value degrades to the tuned default 0.5 with a warning instead of killing every scan. Documented in CLAUDE.md. Book gates: M15 12mo pinned book byte-identical (+78.7111R, 397 trades). M5 12mo drops exactly ONE trade (556 -> 555, +97.56 -> +98.64R): a phantom whose signal fired on the final summer-Friday bar (decision 21:00 UTC = NY 17:00 market close) and was "filled" at Sunday's reopen 49 hours later, -1.08R — an entry live trading can never take. The backtest no longer contains it; live == backtest parity improved. M15 unaffected because hour 21 is in its blocked set. Suite: 341/341 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn
This commit is contained in:
@@ -74,12 +74,12 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
|
|||||||
8. Don't add config fields nothing reads; don't disable indicators in production paths (`disabled_indicators` is for experiments only).
|
8. Don't add config fields nothing reads; don't disable indicators in production paths (`disabled_indicators` is for experiments only).
|
||||||
9. EUR/USD was removed 2026-07-16 after a no-edge verdict (cost-to-volatility; last commit with EUR support: `7d3fd95`). Do not re-add without new data. BTC/ETH work lives on unmerged branches only.
|
9. EUR/USD was removed 2026-07-16 after a no-edge verdict (cost-to-volatility; last commit with EUR support: `7d3fd95`). Do not re-add without new data. BTC/ETH work lives on unmerged branches only.
|
||||||
|
|
||||||
## Known open items from the 2026-08-04 QA audit (owner decisions pending)
|
## 2026-08-04 QA audit — owner decisions ledger
|
||||||
|
|
||||||
- **Weekend constant vs Oanda reality**: `_is_forex_weekend` hardcodes Fri 22:00→Sun 22:00 UTC, but Oanda's week is Fri 17:00→Sun 18:00 New York time — during US DST the feed stops at 21:00 UTC (159/162 summer weekends measured) and winter reopen is Sun 23:00. Result: two false-"STALE DATA/feed lagging" hours weekly (summer Fri 21–22, winter Sun 22–23). Fix = NY-anchored calendar in all three copies; books provably unaffected (no bars exist in those hours) but run a book gate anyway.
|
- **Weekend calendar: FIXED (owner-approved 2026-08-04).** All three `_is_forex_weekend` copies are NY-anchored (Fri 17:00 → Sun 18:00 ET; during US DST that is Fri 21:00 UTC — measured on 5yr of Oanda bars). Kills the weekly false-"STALE DATA/feed lagging" hours. Book impact, fully accounted: M15 byte-identical; M5 12mo dropped exactly ONE trade — a phantom whose signal fired on the final summer-Friday bar (close 21:00 UTC) and "filled" at Sunday's reopen 49 h later (−1.08R, live-impossible). M5 12mo reference at HEAD: 555 trades, +98.64R PF≈1.50 @ $0.65 (was 556/+97.56/1.48). A Module 21 test sweeps all three copies hourly across both DST transitions.
|
||||||
- **Spread in exit-trigger detection (rule 6)**: backtest stop triggers use raw mid High/Low; bid/ask-adjusted triggers would change 15/397 trades (−3.55R) at $0.30 and 30/397 (−4.24R) at $0.65 on the pinned M15 12mo book — a ~4–6% systematic optimism. Fixing invalidates all book comparability: conscious re-pin decision only.
|
- **Spread in exit-trigger detection (rule 6): owner DECLINED 2026-08-04.** Backtest stop triggers stay on raw mid High/Low; the measured optimism (~4–6% of total R: −3.55R @ $0.30, −4.24R @ $0.65 on the M15 12mo book) is accepted and books stay comparable. Re-open only as a conscious re-pin campaign.
|
||||||
- **H4 seeding divergence**: live H4 fetches only 380 bars; the backtest sees unbounded history — ~2% of sampled H4 bars flip an indicator signal (composite Δ up to 12.3). M5/M15/H1 are clean at their real fetch depths. H4 is already "unproven for auto-execution".
|
- **`FB2_EMA_LO`: HARDENED 2026-08-04 (delegated).** Research override for the EMA200-proximity band, default = tuned 0.5; an unparseable value now degrades to the default with a warning instead of crashing every scan. Documented here — it is a knob that exists.
|
||||||
- `FB2_EMA_LO` env var is an undocumented knob inside the scoring path (`signal_aggregator.py` EMA200-proximity penalty floor); an unparseable value crashes every analysis. Remove or config-ify only with rule-1 revalidation.
|
- **H4 seeding divergence: parked.** Live H4 fetches only 380 bars vs the backtest's unbounded history — ~2% of sampled H4 bars flip an indicator signal (composite Δ up to 12.3). M5/M15/H1 clean at their real fetch depths. Fold into any future H4 campaign; H4 remains "unproven for auto-execution".
|
||||||
|
|
||||||
## Current state notes (2026-07-30)
|
## Current state notes (2026-07-30)
|
||||||
|
|
||||||
|
|||||||
@@ -40,10 +40,13 @@ from dataclasses import dataclass, field
|
|||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Dict, List, Optional, Tuple
|
from typing import Dict, List, Optional, Tuple
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
|
_NY_TZ = ZoneInfo("America/New_York")
|
||||||
|
|
||||||
# Ensure project root on sys.path so this script can be invoked from anywhere.
|
# Ensure project root on sys.path so this script can be invoked from anywhere.
|
||||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||||
if str(PROJECT_ROOT) not in sys.path:
|
if str(PROJECT_ROOT) not in sys.path:
|
||||||
@@ -1450,18 +1453,22 @@ class BacktestConfig:
|
|||||||
def _is_xau_blocked_by_calendar(ts: pd.Timestamp, cfg: PlatformConfig) -> Optional[str]:
|
def _is_xau_blocked_by_calendar(ts: pd.Timestamp, cfg: PlatformConfig) -> Optional[str]:
|
||||||
"""Apply XAU weekend + allowed-weekday filter matching live `analyze_pair`.
|
"""Apply XAU weekend + allowed-weekday filter matching live `analyze_pair`.
|
||||||
|
|
||||||
Forex market closed: Friday 22:00 UTC -> Sunday 22:00 UTC (Sydney open).
|
Market closed Friday 17:00 -> Sunday 18:00 NEW YORK time (= Fri
|
||||||
Must match `_is_forex_weekend` in signal_aggregator.py exactly.
|
21:00 UTC in US summer, 22:00 in winter — matches measured Oanda
|
||||||
|
feed behavior; 2026-08-04 audit I43). Must match `_is_forex_weekend`
|
||||||
|
in signal_aggregator.py exactly (rule 4; Module 21 agreement test).
|
||||||
|
Note the weekday gates below stay on UTC weekday — they mirror the
|
||||||
|
live gates, which evaluate UTC weekdays.
|
||||||
"""
|
"""
|
||||||
xau = cfg.xau_strategy
|
xau = cfg.xau_strategy
|
||||||
if xau.weekend_filter:
|
if xau.weekend_filter:
|
||||||
wd = ts.weekday()
|
ny = ts.astimezone(_NY_TZ)
|
||||||
hour = ts.hour
|
wd = ny.weekday()
|
||||||
if wd == 4 and hour >= 22:
|
if wd == 4 and ny.hour >= 17:
|
||||||
return "WEEKEND_FILTER"
|
return "WEEKEND_FILTER"
|
||||||
if wd == 5:
|
if wd == 5:
|
||||||
return "WEEKEND_FILTER"
|
return "WEEKEND_FILTER"
|
||||||
if wd == 6 and hour < 22:
|
if wd == 6 and ny.hour < 18:
|
||||||
return "WEEKEND_FILTER"
|
return "WEEKEND_FILTER"
|
||||||
if xau.allowed_weekdays and ts.weekday() not in xau.allowed_weekdays:
|
if xau.allowed_weekdays and ts.weekday() not in xau.allowed_weekdays:
|
||||||
return "DAY_FILTER"
|
return "DAY_FILTER"
|
||||||
|
|||||||
+26
-8
@@ -24,6 +24,7 @@ import time
|
|||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
from typing import AsyncIterator, Callable, Dict, Optional, Sequence, Tuple
|
from typing import AsyncIterator, Callable, Dict, Optional, Sequence, Tuple
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
@@ -31,19 +32,36 @@ from config import PAIR_INFO, PlatformConfig, Timeframe, XAUUSD
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# The gold/FX market week is defined in New York time (see
|
||||||
|
# _is_forex_weekend); a fixed UTC hour is wrong for half the year.
|
||||||
|
_NY_TZ = ZoneInfo("America/New_York")
|
||||||
|
|
||||||
|
|
||||||
def _is_forex_weekend(now=None) -> bool:
|
def _is_forex_weekend(now=None) -> bool:
|
||||||
"""Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC."""
|
"""Gold/FX market closed for the weekend.
|
||||||
|
|
||||||
|
The market week is anchored to NEW YORK time — Friday 17:00 ET →
|
||||||
|
Sunday 18:00 ET — not a fixed UTC hour. During US DST the feed
|
||||||
|
actually stops at 21:00 UTC (measured on 5 years of Oanda bars:
|
||||||
|
159/162 summer Fridays; winter close 22:00, winter reopen Sunday
|
||||||
|
23:00 UTC). The old hardcoded Fri 22:00→Sun 22:00 UTC rule produced
|
||||||
|
two false-"STALE DATA/feed lagging" hours every week (2026-08-04
|
||||||
|
audit I43, fixed per owner decision).
|
||||||
|
|
||||||
|
Kept as three deliberate copies (here, signal_aggregator.py,
|
||||||
|
gpu_backtest._is_xau_blocked_by_calendar) — a Module 21 test asserts
|
||||||
|
they agree hour-by-hour across DST transitions.
|
||||||
|
"""
|
||||||
if now is None:
|
if now is None:
|
||||||
now = pd.Timestamp.now(tz="UTC")
|
now = pd.Timestamp.now(tz="UTC")
|
||||||
wd = now.weekday()
|
ny = now.astimezone(_NY_TZ)
|
||||||
hour = now.hour
|
wd = ny.weekday()
|
||||||
if wd == 4 and hour >= 22:
|
if wd == 4: # Friday
|
||||||
return True
|
return ny.hour >= 17
|
||||||
if wd == 5:
|
if wd == 5: # Saturday
|
||||||
return True
|
|
||||||
if wd == 6 and hour < 22:
|
|
||||||
return True
|
return True
|
||||||
|
if wd == 6: # Sunday
|
||||||
|
return ny.hour < 18
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+25
-8
@@ -10,9 +10,12 @@ import os
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from typing import Dict, List, Optional
|
from typing import Dict, List, Optional
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
|
_NY_TZ = ZoneInfo("America/New_York")
|
||||||
|
|
||||||
from chart_patterns import detect_chart_pattern
|
from chart_patterns import detect_chart_pattern
|
||||||
from config import PlatformConfig, SignalStrength, Timeframe
|
from config import PlatformConfig, SignalStrength, Timeframe
|
||||||
from providers import BaseDataProvider, make_data_provider
|
from providers import BaseDataProvider, make_data_provider
|
||||||
@@ -23,15 +26,19 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
|
|
||||||
def _is_forex_weekend(now_utc: datetime) -> bool:
|
def _is_forex_weekend(now_utc: datetime) -> bool:
|
||||||
"""Forex market closed: Friday 22:00 UTC → Sunday 22:00 UTC (Sydney open)."""
|
"""Gold/FX market closed for the weekend: Friday 17:00 → Sunday 18:00
|
||||||
wd = now_utc.weekday()
|
NEW YORK time (= Fri 21:00 UTC in US summer, 22:00 in winter —
|
||||||
hour = now_utc.hour
|
matches measured Oanda feed behavior; 2026-08-04 audit I43).
|
||||||
if wd == 4 and hour >= 22: # Friday after 22:00
|
Deliberate copy of providers.base._is_forex_weekend — a Module 21
|
||||||
return True
|
test asserts the copies agree across DST transitions."""
|
||||||
if wd == 5: # All Saturday
|
ny = now_utc.astimezone(_NY_TZ)
|
||||||
return True
|
wd = ny.weekday()
|
||||||
if wd == 6 and hour < 22: # Sunday before 22:00
|
if wd == 4: # Friday
|
||||||
|
return ny.hour >= 17
|
||||||
|
if wd == 5: # Saturday
|
||||||
return True
|
return True
|
||||||
|
if wd == 6: # Sunday
|
||||||
|
return ny.hour < 18
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
@@ -1253,7 +1260,17 @@ class SignalAggregator:
|
|||||||
ema200 = primary.key_levels.get("ema_trend")
|
ema200 = primary.key_levels.get("ema_trend")
|
||||||
if ema200 and ema200 > 0 and report.current_price > 0:
|
if ema200 and ema200 > 0 and report.current_price > 0:
|
||||||
ema_dist_pct = abs(report.current_price - ema200) / ema200 * 100.0
|
ema_dist_pct = abs(report.current_price - ema200) / ema200 * 100.0
|
||||||
|
# FB2_EMA_LO: research override for the EMA200-proximity
|
||||||
|
# penalty band; the tuned default is 0.5. An unparseable
|
||||||
|
# value must degrade to the default, never crash the scan
|
||||||
|
# (2026-08-04 audit; hardened per owner delegation).
|
||||||
|
try:
|
||||||
_lo = float(os.environ.get("FB2_EMA_LO", "0.5"))
|
_lo = float(os.environ.get("FB2_EMA_LO", "0.5"))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
logger.warning(
|
||||||
|
"FB2_EMA_LO=%r is not a number — using tuned "
|
||||||
|
"default 0.5", os.environ.get("FB2_EMA_LO"))
|
||||||
|
_lo = 0.5
|
||||||
if ema_dist_pct < _lo:
|
if ema_dist_pct < _lo:
|
||||||
report.warnings.append(f"NEAR EMA200 ({ema_dist_pct:.2f}%) — chop risk")
|
report.warnings.append(f"NEAR EMA200 ({ema_dist_pct:.2f}%) — chop risk")
|
||||||
penalties.append(("ema200_near", 5))
|
penalties.append(("ema200_near", 5))
|
||||||
|
|||||||
+59
@@ -6397,6 +6397,65 @@ def _():
|
|||||||
assert [p for p, _ in _gb._apply_cooldown(cands, 1)] == [10, 12, 15, 21, 24]
|
assert [p for p, _ in _gb._apply_cooldown(cands, 1)] == [10, 12, 15, 21, 24]
|
||||||
|
|
||||||
|
|
||||||
|
@test("weekend calendar is NY-anchored: summer close 21:00 UTC, winter reopen 23:00")
|
||||||
|
def _():
|
||||||
|
# 2026-08-04 audit I43, fixed per owner decision: the market week is
|
||||||
|
# Fri 17:00 -> Sun 18:00 NEW YORK time (measured on 5yr of Oanda
|
||||||
|
# bars), not a fixed UTC hour.
|
||||||
|
import providers.base as pbase
|
||||||
|
T = lambda s: pd.Timestamp(s, tz="UTC")
|
||||||
|
# US DST 2026: Mar 8 – Nov 1.
|
||||||
|
assert pbase._is_forex_weekend(T("2026-07-31 21:30")) is True, \
|
||||||
|
"summer Friday 21:30 UTC = 17:30 ET → market CLOSED (old code said open)"
|
||||||
|
assert pbase._is_forex_weekend(T("2026-07-31 20:30")) is False
|
||||||
|
assert pbase._is_forex_weekend(T("2026-01-02 21:30")) is False, \
|
||||||
|
"winter Friday 21:30 UTC = 16:30 ET → still open"
|
||||||
|
assert pbase._is_forex_weekend(T("2026-01-02 22:30")) is True
|
||||||
|
assert pbase._is_forex_weekend(T("2026-01-04 22:30")) is True, \
|
||||||
|
"winter Sunday 22:30 UTC = 17:30 ET → still closed (old code said open)"
|
||||||
|
assert pbase._is_forex_weekend(T("2026-01-04 23:30")) is False
|
||||||
|
assert pbase._is_forex_weekend(T("2026-08-02 21:30")) is True # summer Sun 17:30 ET
|
||||||
|
assert pbase._is_forex_weekend(T("2026-08-02 22:30")) is False # summer Sun 18:30 ET
|
||||||
|
assert pbase._is_forex_weekend(T("2026-08-01 03:00")) is True # Saturday
|
||||||
|
|
||||||
|
|
||||||
|
@test("weekend calendar: all three copies agree hourly across DST transitions")
|
||||||
|
def _():
|
||||||
|
import providers.base as pbase
|
||||||
|
import signal_aggregator as sa_mod
|
||||||
|
import gpu_backtest as _gb
|
||||||
|
cfg = PlatformConfig()
|
||||||
|
cfg.xau_strategy.weekend_filter = True
|
||||||
|
cfg.xau_strategy.allowed_weekdays = [] # isolate the weekend gate
|
||||||
|
# Sweep spring-forward (2026-03-06..11) and fall-back (2026-10-30..11-04).
|
||||||
|
for start in ("2026-03-06", "2026-10-30"):
|
||||||
|
for ts in pd.date_range(start, periods=5 * 24, freq="1h", tz="UTC"):
|
||||||
|
a = pbase._is_forex_weekend(ts)
|
||||||
|
b = sa_mod._is_forex_weekend(ts)
|
||||||
|
c = _gb._is_xau_blocked_by_calendar(ts, cfg) == "WEEKEND_FILTER"
|
||||||
|
assert a == b == c, f"{ts}: base={a} aggregator={b} backtest={c}"
|
||||||
|
|
||||||
|
|
||||||
|
@test("FB2_EMA_LO: garbage value degrades to tuned default, scan survives")
|
||||||
|
def _():
|
||||||
|
os.environ["FB2_EMA_LO"] = "not-a-number"
|
||||||
|
try:
|
||||||
|
config = PlatformConfig()
|
||||||
|
config.primary_timeframe = Timeframe.H1
|
||||||
|
config.confirmation_timeframes = [Timeframe.M15, Timeframe.H4]
|
||||||
|
agg = SignalAggregator(config)
|
||||||
|
dfs = {
|
||||||
|
Timeframe.H1: make_ohlcv(300, seed=42),
|
||||||
|
Timeframe.M15: make_ohlcv(300, seed=43),
|
||||||
|
Timeframe.H4: make_ohlcv(300, seed=44),
|
||||||
|
}
|
||||||
|
report = agg.analyze_from_dataframes("XAU/USD", dfs, use_news=False)
|
||||||
|
assert isinstance(report, SignalReport), \
|
||||||
|
"analysis must survive an unparseable FB2_EMA_LO"
|
||||||
|
finally:
|
||||||
|
os.environ.pop("FB2_EMA_LO", None)
|
||||||
|
|
||||||
|
|
||||||
# ══════════════════════════════════════════════════════════════════════
|
# ══════════════════════════════════════════════════════════════════════
|
||||||
# SUMMARY
|
# SUMMARY
|
||||||
# ══════════════════════════════════════════════════════════════════════
|
# ══════════════════════════════════════════════════════════════════════
|
||||||
|
|||||||
Reference in New Issue
Block a user