Daily-trend veto on M15 + lot-size / already-open lines (campaign 2026-09-30), and the 2026-10-01 whole-codebase bug review
Two bodies of work that touch the same files, so one commit. Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule, written before the runs): - SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto "both"). A finished setup against a clear daily trend (closed D1 bars, Sunday stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier) is removed as the last step of the core. One definition (signal_aggregator.daily_trend_series) for live, the bar replay and the twin's host side; D1 is a required frame of the M15 scan; mirrored in cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R. - FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1 (−9.7R), buy-only and sell-only on M15, the wide-stop cap (xau_strategy.wide_stop_cap_atr stays 0.0, a research knob). - New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15 pins byte for byte. Twin and reference script MD5-equal on every window; recorder and feature-export parity re-proved; base-rate table rebuilt. - DISPLAY (books byte-identical): exposure.py — the lot size for trade_size.risk_per_trade (new config.json block, off by default, private to the owner) and the count of alerted same-direction plans still open (signals.alerted_at, read-only open_alerted_setups); the daily trend in words on Telegram, the CLI and the dashboard; signals.daily_trend logged. - providers/oanda: a date window with no bars is an empty page, not an error (a D1 pad starting on a Saturday could not be fetched). - Reports only, nothing shipped: seven context screens (no state nominated) and eight outside data sources (none adds a forecast to gold's own daily trend). - Independent verification (report section 5.13): every figure reproduced on both scripts, scoring identical to the previous commit on all four timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022; the neighbour clause is thin (a simple 100-day average misses the R line by 1.2R); "never adds a trade" is an observation, not a guarantee. Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects elsewhere, plus 4 found by an independent review of the fixes; all fixed, each with a test; no trade book changed: - telegram_bot: one _deliver_alert for the candle scan and the level break. The row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and un-stamped when no chat received it or the task is cancelled mid-stamp, so two alerts at one candle close count each other and no SQLite write runs on the event loop. The 5-bar cooldown counts bars of the scan's own frame (== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after the settlement pause) and is restored from signals.db at start-up. The lot-size line goes to the owner's own chat only (a group the owner ran /autoscan in used to receive it). /status gains a news-model line and escapes failure details. - signal_log: mark_alerted returns the stamp it wrote; unmark_alerted; last_alerted_bars (read-only). - advisory_outcome: candle continuity is judged bar by bar inside the walk; a hole after a trade's exit no longer voids it. The stored-candle replay resolves 108 of 111 logged setups instead of 99, all equal to the live resolver's R. - providers/oanda: an empty date window is accepted only when it is 5 days or shorter and no malformed candle was skipped; a longer one is retried, then refused, instead of being cached as history with a hole. - news_engine: blank RSS titles skipped; one unreadable headline no longer costs the local model its whole batch (checked on the real bundle); model health recorded; no DeepSeek audit call in a process that does not archive; event-direction cache keyed per release; peek_week_events never fetches. - llm_client: chat_json never raises and never logs the API key (requests quotes the Authorization value in InvalidHeader; error bodies are redacted). - signal_aggregator: the scan's news block is guarded; the replay path (use_news=False) assigns exactly the values it did before. - agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input (event timing over the cached calendar week, headlines newest first). scripts/panel_replay.py replays the logged payload per persona; scripts/label_scan_outcomes.py counts with an ISO cutoff. - test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34); tests that write Parquet print a notice and return without pyarrow. Verification after the last code change: suite 632 passed, 0 failed (also with both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37 green; static Python 3.9 / SQLite 3.34 check clean on every touched file. Evidence: backtest/research/daily_trend_campaign_2026-09-30.md, context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6, 8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
+74
-7
@@ -384,7 +384,7 @@ class NewsEngine:
|
||||
# writer runs inside _cached_fetch); the other two take the lock
|
||||
# around cache access but keep HTTP calls outside it.
|
||||
_llm_sent_cache: Dict[str, Tuple[float, float]] = {} # sha1 → (s, xau)
|
||||
_llm_event_cache: Dict[Tuple[str, str, str], str] = {}
|
||||
_llm_event_cache: Dict[Tuple[str, str, str, str, str], str] = {}
|
||||
_llm_summary_cache: Tuple[float, str] = (0.0, "")
|
||||
_LLM_SENT_CACHE_MAX = 400
|
||||
_LLM_EVENT_CACHE_MAX = 100
|
||||
@@ -609,6 +609,23 @@ class NewsEngine:
|
||||
key=lambda e: e.time,
|
||||
)
|
||||
|
||||
def peek_week_events(
|
||||
self, currencies: Optional[List[str]] = None
|
||||
) -> List[EconomicEvent]:
|
||||
"""get_week_events WITHOUT the refresh: whatever calendar the
|
||||
class-level cache holds right now, however old, filtered and sorted
|
||||
the same way. Never fetches and never waits on the cache lock, for
|
||||
a caller that must not add a network wait: the agent panel runs
|
||||
while an alert is waiting to go out, and a refetch with both
|
||||
calendar sources down takes about 40 seconds."""
|
||||
if currencies is None:
|
||||
currencies = self.config.tracked_currencies
|
||||
_stamp, events = NewsEngine._calendar_cache
|
||||
return sorted(
|
||||
(e for e in events if e.currency in currencies),
|
||||
key=lambda e: e.time,
|
||||
)
|
||||
|
||||
# ── Private Methods ────────────────────────────────────────────
|
||||
|
||||
# Maps country names in MyFXBook titles to currency codes
|
||||
@@ -871,6 +888,11 @@ class NewsEngine:
|
||||
continue
|
||||
|
||||
title = title_el.text or ""
|
||||
# A blank title carries nothing to score, show or
|
||||
# archive, and one such item made the local model
|
||||
# reject the whole batch (2026-10-01).
|
||||
if not title.strip():
|
||||
continue
|
||||
pub_date = self._parse_rss_date(
|
||||
pub_date_el.text if pub_date_el is not None else None
|
||||
)
|
||||
@@ -1173,11 +1195,27 @@ class NewsEngine:
|
||||
is re-sent to DeepSeek (fallback mode retries, audit sample);
|
||||
F. only score_source == "keyword" rows change; an llm row stands.
|
||||
"""
|
||||
bundle = news_model.get_news_model()
|
||||
if bundle is None or not headlines:
|
||||
if not headlines:
|
||||
return 0
|
||||
todo = [h for h in headlines if h.score_source == "keyword"]
|
||||
bundle = news_model.get_news_model()
|
||||
if bundle is None:
|
||||
self._record_source("news_model", False,
|
||||
"not loaded in this process")
|
||||
return 0
|
||||
# score_batch cleans every text first and raises on one that is
|
||||
# empty afterwards (" " is), which used to cost the whole
|
||||
# batch its scores; such a row keeps its keyword score instead.
|
||||
todo = [h for h in headlines if h.score_source == "keyword"
|
||||
and self._model_can_score(h.headline)]
|
||||
if not todo:
|
||||
# Nothing for the model this cycle (fallback mode with DeepSeek
|
||||
# healthy, or no readable headline). A failure an earlier batch
|
||||
# recorded says nothing about now and must not sit on /status
|
||||
# as if it were current; the next batch records its own outcome
|
||||
# and warns again if it fails.
|
||||
if self.source_is_failing(
|
||||
NewsEngine._source_health.get("news_model")):
|
||||
NewsEngine._source_health.pop("news_model", None)
|
||||
return 0
|
||||
try:
|
||||
results = news_model.score_batch(bundle, [h.headline for h in todo])
|
||||
@@ -1185,8 +1223,13 @@ class NewsEngine:
|
||||
raise ValueError(
|
||||
f"{len(results)} results for {len(todo)} headlines")
|
||||
except Exception as e:
|
||||
logger.debug(f"news model scoring failed, keyword scores stand: {e}")
|
||||
# Keyword scores stand for the cycle. Loud once (WARNING on the
|
||||
# first failure, INFO on recovery) and shown by /status: in
|
||||
# primary mode nothing else scores these headlines.
|
||||
self._record_source("news_model", False,
|
||||
f"{type(e).__name__}: {e}")
|
||||
return 0
|
||||
self._record_source("news_model", True, extra=f"{len(todo)} scored")
|
||||
for h, r in zip(todo, results):
|
||||
raw = float(r["score"]) # archive value
|
||||
shown = raw if r["relevant"] else 0.0 # display value
|
||||
@@ -1198,6 +1241,15 @@ class NewsEngine:
|
||||
h.score_source = "model"
|
||||
return len(todo)
|
||||
|
||||
@staticmethod
|
||||
def _model_can_score(text: str) -> bool:
|
||||
"""True when the local model's own cleaner accepts the text."""
|
||||
try:
|
||||
news_model.clean_headline(text)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _audit_with_deepseek(
|
||||
self, headlines: List[NewsSentiment]
|
||||
) -> List[NewsSentiment]:
|
||||
@@ -1207,10 +1259,15 @@ class NewsEngine:
|
||||
numbers. One batch per cycle at most; the pairs go into
|
||||
_llm_sent_cache so a headline is never audited twice. This is the
|
||||
only place DeepSeek is called for headlines in primary mode; with
|
||||
the cap at 0 (the shipped value) it returns [] without a call."""
|
||||
the cap at 0 (the shipped value) it returns [] without a call.
|
||||
The copies exist only to be archived, so a process that does not
|
||||
write the archive (the dashboard daemon, NEWS_ARCHIVE_ENABLED=0)
|
||||
makes no call either."""
|
||||
cap = app_config.news_model_audit_per_day()
|
||||
if cap <= 0 or not headlines or not get_llm_client().available:
|
||||
return []
|
||||
if not app_config.news_archive_enabled():
|
||||
return []
|
||||
today = datetime.now(timezone.utc).date().isoformat()
|
||||
if NewsEngine._audit_day != today:
|
||||
NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0
|
||||
@@ -1253,7 +1310,17 @@ class NewsEngine:
|
||||
close K/M-suffixed pairs backwards ~11% of the time (e.g. claims
|
||||
202K vs 217K called "higher than forecast") when left to compare
|
||||
the raw numbers itself."""
|
||||
key = (event.event, event.currency, str(event.actual or ""))
|
||||
# Forecast and release time are part of the key: the direction is
|
||||
# a reading of actual AGAINST forecast, and next month's release
|
||||
# can print the same actual against a different forecast. The time
|
||||
# is keyed in UTC so one instant is one key whatever offset the
|
||||
# feed printed it with.
|
||||
try:
|
||||
when = event.time.astimezone(timezone.utc).isoformat()
|
||||
except Exception:
|
||||
when = str(event.time)
|
||||
key = (event.event, event.currency, str(event.actual or ""),
|
||||
str(event.forecast or ""), when)
|
||||
with NewsEngine._cache_lock:
|
||||
if key in NewsEngine._llm_event_cache:
|
||||
return NewsEngine._llm_event_cache[key] or None
|
||||
|
||||
Reference in New Issue
Block a user