audit: scheduler cleanup, WS-log secret hygiene, simulator test coverage, CLAUDE.md sync (Batch G)

2026-08-04 Fable 5 QA audit, final code batch.

- Boundary math: removed the provably-dead `remaining <= buffer` guard
  from all three scheduler copies (518,400-instant frozen-clock sweep:
  never fired) and documented the exact-boundary-instant semantics.
- websockets/websockets.client loggers pinned to INFO so a DEBUG run
  can never write the TD key (connect URI) or Tiingo auth frame into
  the 30-day rotating log. (.env also chmod 600, untracked.)
- Cooldown pass factored into _apply_cooldown (pure, testable) — the
  only sequential replay rule finally has direct tests.
- New simulator behavior tests (previously ZERO coverage of the trade
  resolver): same-bar SL+TP tie → SL-first, gap-through-SL fills at
  the open, TIMEOUT pricing + round-trip spread charging, trailing
  ratchet arms at bar close and fills next-bar-only, cooldown ordering.
- CLAUDE.md synced to verified reality: 338 tests/21 modules; three
  weekend-implementation copies caveat; throttle claim corrected
  (single-flight, TD-only throttle); daemon trail/cooldown parity
  caveats (once-per-OBSERVED-bar, wall-clock cooldown) + OPEN_ONLY and
  reconcile notes; CP-path gate semantics documented as validated
  behavior; new "known open items" section for the owner-decision
  findings (weekend DST constant, spread-in-trigger rule-6 item, H4
  seeding divergence, FB2_EMA_LO).

Gates: suite 338/338 green; pinned M15 book and M5 HEAD baseline both
reproduced identically after the cooldown refactor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnZkzeFcTu9agFg8BDh2tn

CLAUDE.md

@@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
 ## Commands
 
 ```bash
-python test_all.py                      # QA suite: 302 tests, 20 modules, no network (mocked)
+python test_all.py                      # QA suite: 338 tests, 21 modules, no network (mocked)
 python main.py --pair XAUUSD --timeframe H1 --detail   # CLI signal report (--json for full dump)
 python telegram_bot.py                  # the bot (refuses to boot without credentials)
 python oanda_daemon.py                  # auto-execution daemon on Oanda practice account
@@ -26,7 +26,7 @@ python backtest/gpu_backtest.py --pair XAUUSD --start 2025-01-01 --end 2025-06-0
 #            --warmup-bars --cooldown-bars --no-cache --export-csv out.csv --detail
 ```
 
-**`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. When patching weekend logic in tests, patch `providers.base._is_forex_weekend`.
+**`test_all.py` is NOT pytest** — it uses a custom `@test("name")` decorator that runs each test immediately at import; don't grep for `def test_` or try `pytest test_all.py`. Run the whole file; it prints per-module results and exits non-zero on failure. There is no way to run a single test short of commenting modules out. Tests must stay green; every new gate/knob needs tests. When patching weekend logic in tests, patch `providers.base._is_forex_weekend` — but know there are THREE independent weekend implementations (`providers/base.py`, `signal_aggregator.py`, and an inline copy in `backtest/gpu_backtest.py:_is_xau_blocked_by_calendar`); patching the provider one does not reach the other two.
 
 Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy, pandas, requests, twelvedata, python-telegram-bot ≥20, websockets).
 
@@ -39,7 +39,8 @@ Precedence for every credential/deployment setting: **env var → config.json
 ### End-to-end pipeline (every scan)
 
 ```
-providers/<active>.fetch_ohlcv()       per TF: throttle, fetch, drop forming candle, validate/clean, cache
+providers/<active>.fetch_ohlcv()       per TF: single-flight fetch, drop forming candle, validate/clean, cache
+                                       (only Twelve Data rate-throttles; Oanda has no client throttle)
   → engine.TechnicalEngine.analyze()   per TF: 9 scored indicators + ATR context → regime → composite score
   → signal_aggregator.analyze_pair()   live entry point (weekend/day filters, fetch, staleness)
       └─ analyze_from_dataframes()     replayable core: MTF agreement → news → bonuses/penalties →
@@ -53,11 +54,11 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
 
 - **`providers/`** — one class per provider subclassing `BaseDataProvider` (`providers/base.py`), which owns everything the pipeline depends on: TTL+boundary cache, forming-candle drop, freshness check, validate/clean, paginated `fetch_range` (feeds the backtester from any provider), optional tick streaming. Subclasses implement only `_fetch_raw` + symbol/interval maps. Registry + `make_data_provider(config, name=None)` in `providers/__init__.py`; config.json `data_providers.active` selects the live candle feed and the optional `data_providers.streaming` scalar selects a different provider for the tick stream. A test enforces that subclasses don't override shared base methods (`supports_streaming`/`iter_ticks` are the sanctioned streaming override points). Tiingo streams mid-price ticks (`wss://api.tiingo.com/fx`) — do not re-add spread display from its free FX feed (two-tier indicative quotes, removed 2026-07-23); the no-spread rule extends to the WS stream.
 - **`engine.py`** — `TechnicalEngine`: 9 scored indicators (ema_trend, rsi, macd, bollinger, adx, roc, keltner, divergence, candlestick) + ATR as context. Regime classification (trending_up/down, `trending` = mixed/direction-UNKNOWN, transitioning, ranging) drives per-indicator weight multipliers and regime-aware RSI reinterpretation. Compute order matters (ATR first; MACD always computed even when not a pillar — divergence needs the histogram). Private series ride on the result as `_underscore` attrs; scorers/formatters skip anything `_`-prefixed or weight-0.
-- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
+- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. The PB/CP generators run AFTER the gate chain and force quality to "B"; the CP path deliberately accepts grade-F composites and re-checks only session/day/big-candle (via warning prefixes) plus its own SELL/vol/RSI gates — it does NOT re-check exhaustion or the M5 min-score/MTF gates. **This is validated behavior, not a bug** (2026-08-04 audit: 226/586 trades in the shipped M5 12mo book are XAU_CP, +38.9R, 104 of them exhaustion-flagged and net-positive) — changing it is a rule-1 revalidation event. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
 - **`pair_signals/`** — per-pair config factories. **Always use the factory (`produce_signal` / `make_config`), never bare `PlatformConfig()`** — the factory applies the config.json overlay first, then the validated per-TF deltas (M5 hour-9 block, M15 trail arm 0.70, M15 SELL-hour block, etc.; evidence in the docstring). A test enforces factory ≡ inline-config equivalence. Confirmation map: M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1].
 - **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that.
 - **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
-- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown, trail ratchets at bar close exactly once per bar. Accepted divergence: one position total.
+- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`).
 - **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. Deployment checklist: `deploy/README-dashboard.md`.
 - **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
 
@@ -73,6 +74,13 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
 8. Don't add config fields nothing reads; don't disable indicators in production paths (`disabled_indicators` is for experiments only).
 9. EUR/USD was removed 2026-07-16 after a no-edge verdict (cost-to-volatility; last commit with EUR support: `7d3fd95`). Do not re-add without new data. BTC/ETH work lives on unmerged branches only.
 
+## Known open items from the 2026-08-04 QA audit (owner decisions pending)
+
+- **Weekend constant vs Oanda reality**: `_is_forex_weekend` hardcodes Fri 22:00→Sun 22:00 UTC, but Oanda's week is Fri 17:00→Sun 18:00 New York time — during US DST the feed stops at 21:00 UTC (159/162 summer weekends measured) and winter reopen is Sun 23:00. Result: two false-"STALE DATA/feed lagging" hours weekly (summer Fri 21–22, winter Sun 22–23). Fix = NY-anchored calendar in all three copies; books provably unaffected (no bars exist in those hours) but run a book gate anyway.
+- **Spread in exit-trigger detection (rule 6)**: backtest stop triggers use raw mid High/Low; bid/ask-adjusted triggers would change 15/397 trades (−3.55R) at $0.30 and 30/397 (−4.24R) at $0.65 on the pinned M15 12mo book — a ~4–6% systematic optimism. Fixing invalidates all book comparability: conscious re-pin decision only.
+- **H4 seeding divergence**: live H4 fetches only 380 bars; the backtest sees unbounded history — ~2% of sampled H4 bars flip an indicator signal (composite Δ up to 12.3). M5/M15/H1 are clean at their real fetch depths. H4 is already "unproven for auto-execution".
+- `FB2_EMA_LO` env var is an undocumented knob inside the scoring path (`signal_aggregator.py` EMA200-proximity penalty floor); an unparseable value crashes every analysis. Remove or config-ify only with rule-1 revalidation.
+
 ## Current state notes (2026-07-30)
 
 - Oanda v20 is the live feed (`data_providers.active: "oanda"`), but the pinned backtest books are still Twelve-Data-pinned and the backtest cost assumption is $0.30 while measured real spreads are ~$0.65–0.72 — re-pin books on Oanda bars before treating feed-switch comparisons as valid (see Oanda/TD divergence study in memory).

backtest/gpu_backtest.py

@@ -1678,6 +1678,24 @@ def _reanchor_entries_to_raw(
         )
 
 
+def _apply_cooldown(
+    candidates: List[Tuple[int, Trade]], cooldown_bars: int
+) -> List[Tuple[int, Trade]]:
+    """Ordered cooldown pass over (bar_pos, Trade) candidates: drop any
+    candidate within `cooldown_bars` bars of the previously ACCEPTED
+    entry — exactly what a sequential replay loop would have skipped.
+    Candidates must already be sorted by position (stable merge from the
+    worker spans)."""
+    accepted: List[Tuple[int, Trade]] = []
+    last_entry_pos = -10_000
+    for pos, trade in candidates:
+        if pos - last_entry_pos < cooldown_bars:
+            continue
+        accepted.append((pos, trade))
+        last_entry_pos = pos
+    return accepted
+
+
 def run_backtest(
     bt_cfg: BacktestConfig,
     platform_cfg: Optional[PlatformConfig] = None,
@@ -1805,18 +1823,10 @@ def run_backtest(
         candidates.extend(cands)
     candidates.sort(key=lambda pc: pc[0])
 
-    # ── Phase 2: entry cooldown — the only sequential rule. A candidate
+    # ── Phase 2: entry cooldown — the only sequential rule.
-    # inside the cooldown window of the previously ACCEPTED entry is
+    accepted = _apply_cooldown(candidates, bt_cfg.cooldown_bars)
-    # dropped, exactly as the sequential loop skipped those bars.
+    trades: List[Trade] = [t for _, t in accepted]
-    trades: List[Trade] = []
+    signal_ts_list: List[pd.Timestamp] = [primary_df.index[p] for p, _ in accepted]
-    signal_ts_list: List[pd.Timestamp] = []
-    last_entry_pos = -10_000
-    for pos, trade in candidates:
-        if pos - last_entry_pos < bt_cfg.cooldown_bars:
-            continue
-        trades.append(trade)
-        signal_ts_list.append(primary_df.index[pos])
-        last_entry_pos = pos
 
     logger.info(
         f"Replay complete: {len(trades)} entries in "

dashboard_publisher.py

@@ -231,8 +231,11 @@ def seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
     elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
     interval_seconds = interval_minutes * 60
     remaining = interval_seconds - elapsed_seconds + buffer
-    if remaining <= buffer:
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
-        remaining += interval_seconds
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
     return remaining
 
 

oanda_daemon.py

@@ -139,8 +139,11 @@ def _seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
     elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
     interval_seconds = interval_minutes * 60
     remaining = interval_seconds - elapsed_seconds + buffer
-    if remaining <= buffer:
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
-        remaining += interval_seconds
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
     return remaining
 
 # ── Trade state persistence ───────────────────────────────────────────

telegram_bot.py

@@ -103,6 +103,12 @@ logging.basicConfig(level=getattr(logging, _APP.log_level, logging.INFO),
                     force=True)
 logging.getLogger("httpx").setLevel(logging.WARNING)
 logging.getLogger("httpcore").setLevel(logging.WARNING)
+# Never let DEBUG runs write credentials into the 30-day rotating log:
+# the websockets client logger emits the connect URI (Twelve Data puts
+# the API key in the query string) and frame contents (Tiingo's auth
+# frame carries the key) — 2026-08-04 audit, finding K47.
+logging.getLogger("websockets").setLevel(logging.INFO)
+logging.getLogger("websockets.client").setLevel(logging.INFO)
 logger = logging.getLogger("telegram_bot")
 
 TOKEN = telegram_token()
@@ -667,8 +673,11 @@ def _seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float:
     elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6
     interval_seconds = interval_minutes * 60
     remaining = interval_seconds - elapsed_seconds + buffer
-    if remaining <= buffer:
+    # NOTE: remaining > buffer always (elapsed < interval by construction);
-        remaining += interval_seconds
+    # a call at the exact boundary instant targets the NEXT boundary —
+    # steady-state loops wake at boundary+buffer, so only a process
+    # started in the boundary microsecond skips one scan (2026-08-04
+    # audit I42; the old `remaining <= buffer` guard was unreachable).
     return remaining
 
 

test_all.py

@@ -6316,6 +6316,87 @@ def _():
     _aio21.run(main())
 
 
+def _m21_sim_trade(direction="BUY", entry=2400.0, sl=2390.0, tp1=2410.0,
+                   ts="2026-01-05 10:00"):
+    import gpu_backtest as _gb
+    return _gb.Trade(pair="XAU/USD", entry_time=pd.Timestamp(ts, tz="UTC"),
+                     entry_price=entry, sl=sl, tp1=tp1, tp2=tp1 + 10,
+                     direction=direction, pip_value=0.1, risk_pips=100.0,
+                     reward_pips=100.0, rr_planned=1.0, quality="B",
+                     final_score=30.0, final_confidence=60.0,
+                     market_regime="trending_up")
+
+
+def _m21_sim_df(bars, ts="2026-01-05 10:00"):
+    idx = pd.date_range(ts, periods=len(bars), freq="15min", tz="UTC")
+    return pd.DataFrame(
+        {"Open": [b[0] for b in bars], "High": [b[1] for b in bars],
+         "Low": [b[2] for b in bars], "Close": [b[3] for b in bars],
+         "Volume": 1.0}, index=idx)
+
+
+@test("backtester: same-bar SL+TP tie resolves to SL (static path)")
+def _():
+    import gpu_backtest as _gb
+    t = _m21_sim_trade()
+    sim = _m21_sim_df([(2400, 2411, 2389, 2400)])   # both barriers inside bar 0
+    _gb.simulate_trades([t], sim, max_bars=5, spread=0.0, trail_arm_r=0.0)
+    assert t.exit_reason == "SL", f"tie must be SL-first, got {t.exit_reason}"
+    assert abs(t.exit_price - 2390.0) < 1e-9
+    assert abs(t.r_multiple - (-1.0)) < 1e-9
+
+
+@test("backtester: gap through SL fills at the open, not the stop")
+def _():
+    import gpu_backtest as _gb
+    t = _m21_sim_trade()
+    sim = _m21_sim_df([(2385, 2388, 2384, 2386)])   # opens below the 2390 stop
+    _gb.simulate_trades([t], sim, max_bars=5, spread=0.0, trail_arm_r=0.0)
+    assert t.exit_reason == "SL"
+    assert abs(t.exit_price - 2385.0) < 1e-9, "gap must fill at the open"
+    assert t.r_multiple < -1.0, "gap loss must exceed 1R"
+
+
+@test("backtester: TIMEOUT closes at the last window bar's close, spread charged")
+def _():
+    import gpu_backtest as _gb
+    t = _m21_sim_trade()
+    sim = _m21_sim_df([(2400, 2405, 2395, 2402)] * 3)   # never hits either side
+    _gb.simulate_trades([t], sim, max_bars=3, spread=0.30, trail_arm_r=0.0)
+    assert t.exit_reason == "TIMEOUT" and t.bars_held == 3
+    assert abs(t.exit_price - 2402.0) < 1e-9
+    expected_r = (2402.0 - 2400.0 - 0.30) / 10.0
+    assert abs(t.r_multiple - expected_r) < 1e-9, \
+        "R must be (pnl - round-trip spread) / risk"
+
+
+@test("backtester: trailing stop ratchets at bar close, fills next bar only")
+def _():
+    import gpu_backtest as _gb
+    t = _m21_sim_trade()
+    # bar0: runs to +1R (2410) — arms trail (0.6R=6), stop → 2410-7.5=2402.5;
+    # bar0 low 2398 must NOT trigger the new stop intra-bar.
+    # bar1: dips to 2402 → trailed stop 2402.5 hit → TRAIL exit.
+    sim = _m21_sim_df([(2400, 2410, 2398, 2409), (2409, 2409.5, 2402, 2403)])
+    _gb.simulate_trades([t], sim, max_bars=5, spread=0.0,
+                        trail_arm_r=0.6, trail_dist_r=0.75)
+    assert t.exit_reason == "TRAIL", f"got {t.exit_reason}"
+    assert t.bars_held == 2, "new stop may only fill from the NEXT bar"
+    assert abs(t.exit_price - 2402.5) < 1e-9
+    assert abs(t.r_multiple - 0.25) < 1e-9
+
+
+@test("backtester: cooldown pass drops candidates inside the window, in order")
+def _():
+    import gpu_backtest as _gb
+    cands = [(10, "A"), (12, "B"), (15, "C"), (21, "D"), (24, "E")]
+    out = _gb._apply_cooldown(cands, cooldown_bars=5)
+    assert [(p, t) for p, t in out] == [(10, "A"), (15, "C"), (21, "D")], \
+        f"cooldown must measure from the previously ACCEPTED entry, got {out}"
+    assert _gb._apply_cooldown([], 5) == []
+    assert [p for p, _ in _gb._apply_cooldown(cands, 1)] == [10, 12, 15, 21, 24]
+
+
 # ══════════════════════════════════════════════════════════════════════
 #  SUMMARY
 # ══════════════════════════════════════════════════════════════════════