harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)

Browse Source 
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72%
agreement with DeepSeek on unseen days) replaces DeepSeek for headline
scoring. DeepSeek keeps event direction, the summary and the dashboard
endpoint.

- news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers,
  so the bot starts without them) + load-once, never-raising singleton.
- news_engine: _enrich_headline_scores mode switch (off/fallback/primary),
  _apply_model_sentiment (gold head only: xau_score + XAU entry, never
  USD; not-relevant shows 0.0; never touches _llm_sent_cache),
  _audit_with_deepseek (archive-only labels, deepseek_audit_per_day).
- Owner decision: primary mode writes nothing to logs/newsdata.db except
  audit copies — the model's output is not training data.
- app_config: news_model section (path/threads/mode/deepseek_audit_per_day)
  + NEWS_MODEL_MODE env override; validated at boot.
- news_archive: model rows carry teacher_model xau_transformer@<trained_at>.
- deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a
  commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers.
- test_all.py Module 38 (15 tests, one against the real bundle). Suite 581.
- CLAUDE.md, spec status note, handoff marked implemented.

Books unaffected: news never enters signals; no signal file touched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
2026-09-27 21:02:42 -04:00
parent 6af1237c2d
commit 0c56b4e155
13 changed files with 1966 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
news_engine.py
+186 -17
View File
@@ -8,6 +8,7 @@ Key principle: High-impact news events override technical signals.
A perfect technical setup can be destroyed by an unexpected NFP print.
"""
import copy
import hashlib
import html
import logging
@@ -23,7 +24,9 @@ from typing import Dict, List, Optional, Tuple
import requests
import app_config
import news_archive
import news_model
from config import NewsConfig
from llm_client import get_llm_client
@@ -222,18 +225,23 @@ class NewsSentiment:
publisher: Optional[str] = None # Google News <source> text, else None
# Which scorer produced sentiment_score / sentiment_by_currency:
# "keyword" (the built-in scorer, ~31% on the 2026-08-06 golden set),
# "llm" (DeepSeek, 78/78 on the same set) or "model" (a trained local
# artifact). Without it the two are indistinguishable in the archive
# and a model trained on the mix inherits the keyword scorer's errors.
# "llm" (DeepSeek, 78/78 on the same set) or "model" (the local XAU
# transformer, news_model.py, 72/78; since 2026-09-27). "model" means
# the model scored xau_score and XAU's per-currency entry; every
# other currency's entry, and sentiment_score when the headline names
# another currency, remain keyword-scored — the model has a gold head
# only. Without the tag the scorers are indistinguishable in the
# archive and a model trained on the mix inherits the keyword errors.
score_source: str = "keyword"
# DeepSeek's gold-specific score for THIS headline, whatever currencies
# it names. sentiment_by_currency["XAU"] carries it only when the
# headline matched a gold keyword; for a pure Fed / dollar / yields
# headline it used to be discarded, which left 930 of the first 1,546
# archived DeepSeek rows without a gold label (eval 2026-09-17).
# Nothing displayed or averaged reads it — the archive stores it as
# training data for a local scorer. None until DeepSeek has scored
# the headline (the keyword scorer never sets it).
# The gold-specific score for THIS headline (DeepSeek's, or the local
# model's raw signed score), whatever currencies it names.
# sentiment_by_currency["XAU"] carries it only when the headline
# matched a gold keyword; for a pure Fed / dollar / yields headline it
# used to be discarded, which left 930 of the first 1,546 archived
# DeepSeek rows without a gold label (eval 2026-09-17). Nothing
# displayed or averaged reads it — the archive stores it as training
# data. None until DeepSeek or the model has scored the headline (the
# keyword scorer never sets it).
xau_score: Optional[float] = None
@@ -384,6 +392,12 @@ class NewsEngine:
#: client's max_tokens (~10 tokens per headline, 2000 max).
_LLM_BATCH_CAP = 25
# primary-mode DeepSeek audit sample (news_model.deepseek_audit_per_day):
# how many model-scored headlines went to DeepSeek today, archive-only.
# Written under _cache_lock like _llm_sent_cache.
_audit_day: str = ""
_audit_sent_today: int = 0
# Per-source fetch health for status displays (telegram /status).
# name → {"ok": ts|None, "err": ts|None, "detail": str, "extra": str}.
# All writers run under _cache_lock (fetches happen inside
@@ -398,6 +412,14 @@ class NewsEngine:
self._session.headers.update({
"User-Agent": "Mozilla/5.0 (compatible; ForexIntel/1.0)"
})
# Create the local scorer's ONNX session off the scan thread so
# the first news cycle does not pay 3-5 s under _cache_lock.
# Idempotent per process (a NewsEngine is built for every scan).
try:
if app_config.news_model_mode() != "off":
news_model.warm_up_in_background()
except Exception as e: # never let the warm-up cost a scan
logger.debug(f"news model warm-up skipped: {e}")
def get_fundamental_context(
self, currencies: Optional[List[str]] = None
@@ -941,10 +963,13 @@ class NewsEngine:
# currency-matched headlines so display, per-currency means, the
# panel and the summary see exactly what they saw before.
scored = headlines + unmatched
mode = app_config.news_model_mode()
audit_rows: List[NewsSentiment] = []
try:
self._apply_llm_sentiment(scored)
audit_rows = self._enrich_headline_scores(scored, mode)
except Exception as e:
logger.debug(f"LLM sentiment failed, keyword scores stand: {e}")
logger.debug(f"headline sentiment enrichment failed, keyword "
f"scores stand: {e}")
# Archive BEFORE any downstream filtering. get_fundamental_context
# drops every headline inside ±NEUTRAL_HEADLINE_THRESHOLD, so a
@@ -955,16 +980,65 @@ class NewsEngine:
# arguments are evaluated BEFORE that function's own try/except, so
# a raising get_llm_client() would escape it. Archiving is a side
# effect — it must never cost us the headlines we just fetched.
#
# In "primary" mode the local model is the scorer and its output
# is not training data, so the cycle's rows are NOT archived
# (owner decision 2026-09-27: stop writing newsdata.db once the
# model scores). Only the DeepSeek audit copies, the one source of
# fresh labels, are stored; with deepseek_audit_per_day 0 the
# archive is frozen. "off" and "fallback" archive as before.
try:
news_archive.archive_headlines(
scored,
teacher_model=getattr(get_llm_client(), "_model", None),
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1)
teacher = getattr(get_llm_client(), "_model", None)
if mode == "primary":
if audit_rows:
news_archive.archive_headlines(
audit_rows, teacher_model=teacher,
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1)
logger.debug(
f"news archive: primary mode, {len(audit_rows)} DeepSeek "
f"audit rows stored, {len(scored)} model/keyword rows "
f"not archived")
else:
news_archive.archive_headlines(
scored, teacher_model=teacher,
teacher_prompt_sha1=_LLM_SENTIMENT_PROMPT_SHA1,
model_teacher=news_model.teacher_tag())
except Exception as e:
logger.debug(f"news archive skipped, headlines unaffected: {e}")
return headlines
def _enrich_headline_scores(
self, scored: List[NewsSentiment], mode: str
) -> List[NewsSentiment]:
"""Upgrade keyword scores on `scored` per the news_model mode.
Returns the DeepSeek audit copies to archive (primary mode only,
usually empty). Runs under _cache_lock like the scorers it calls.
| mode | DeepSeek for headlines | local model |
|----------|------------------------|----------------------------|
| off | every new headline | never |
| fallback | first, as before | rows DeepSeek left keyword |
| primary | audit sample only | every row |
primary with the model unavailable in this process (missing
files, onnxruntime not installed) degrades to the DeepSeek path
for the cycle, never straight to keywords; a model failure mid-
batch leaves keyword scores standing (see _apply_model_sentiment).
"""
if mode == "primary":
n = self._apply_model_sentiment(scored)
if n == 0 and news_model.get_news_model() is None:
self._apply_llm_sentiment(scored)
return []
return self._audit_with_deepseek(scored)
if mode == "fallback":
self._apply_llm_sentiment(scored)
self._apply_model_sentiment(scored)
return []
self._apply_llm_sentiment(scored)
return []
def _analyze_headline_sentiment(
self, headline: str, for_currency: Optional[str] = None
) -> float:
@@ -1074,6 +1148,101 @@ class NewsEngine:
h.xau_score = xau
h.score_source = "llm"
# ── Local XAU transformer (news_model.py, since 2026-09-27) ──────
def _apply_model_sentiment(self, headlines: List[NewsSentiment]) -> int:
"""Score with the local XAU transformer every headline that is
still keyword-scored. Returns the number scored. Never raises.
The model has a gold head only. It writes xau_score always, XAU's
per-currency score when XAU is mentioned, and the overall score
only when the headline names no other currency. USD/EUR/...
entries keep their keyword score: get_pair_sentiment subtracts
USD from XAU and must not see the same number on both sides.
Rules (spec 2026-09-27):
A. never write the model score into a non-XAU currency;
B. sentiment_score from the model only for [] or ["XAU"] — the
archive's `score` column must be honest about what the model
did and did not score;
C. xau_score is the raw signed score (training data, unfiltered);
D. displayed values are 0.0 when the model says not relevant — a
confident direction head can still give |score| ~0.2, which
NEUTRAL_HEADLINE_THRESHOLD (0.1) would show as "leaning";
E. NewsEngine._llm_sent_cache is never touched — it decides what
is re-sent to DeepSeek (fallback mode retries, audit sample);
F. only score_source == "keyword" rows change; an llm row stands.
"""
bundle = news_model.get_news_model()
if bundle is None or not headlines:
return 0
todo = [h for h in headlines if h.score_source == "keyword"]
if not todo:
return 0
try:
results = news_model.score_batch(bundle, [h.headline for h in todo])
if len(results) != len(todo):
raise ValueError(
f"{len(results)} results for {len(todo)} headlines")
except Exception as e:
logger.debug(f"news model scoring failed, keyword scores stand: {e}")
return 0
for h, r in zip(todo, results):
raw = float(r["score"]) # archive value
shown = raw if r["relevant"] else 0.0 # display value
h.xau_score = raw
if "XAU" in h.currencies_mentioned:
h.sentiment_by_currency["XAU"] = shown
if set(h.currencies_mentioned) <= {"XAU"}: # [] or ["XAU"]
h.sentiment_score = shown
h.score_source = "model"
return len(todo)
def _audit_with_deepseek(
self, headlines: List[NewsSentiment]
) -> List[NewsSentiment]:
"""primary mode: send up to news_model.deepseek_audit_per_day
model-scored headlines per UTC day to DeepSeek and return `llm`
COPIES for the archive. The displayed objects keep the model's
numbers. One batch per cycle at most; the pairs go into
_llm_sent_cache so a headline is never audited twice. This is the
only place DeepSeek is called for headlines in primary mode; with
the cap at 0 (the shipped value) it returns [] without a call."""
cap = app_config.news_model_audit_per_day()
if cap <= 0 or not headlines or not get_llm_client().available:
return []
today = datetime.now(timezone.utc).date().isoformat()
if NewsEngine._audit_day != today:
NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0
room = cap - NewsEngine._audit_sent_today
if room <= 0:
return []
cache = NewsEngine._llm_sent_cache
cand = [h for h in headlines if h.score_source == "model"
and hashlib.sha1(h.headline.encode()).hexdigest() not in cache]
batch = cand[:min(room, self._LLM_BATCH_CAP)]
if not batch:
return []
scores = self._llm_headline_scores([h.headline for h in batch])
if scores is None:
return []
NewsEngine._audit_sent_today += len(batch)
copies = []
for h, (overall, xau) in zip(batch, scores):
cache[hashlib.sha1(h.headline.encode()).hexdigest()] = (overall, xau)
c = copy.copy(h)
c.sentiment_score = overall
c.sentiment_by_currency = {
cur: (xau if cur == "XAU" else overall)
for cur in h.currencies_mentioned
}
c.xau_score = xau
c.score_source = "llm"
copies.append(c)
while len(cache) > self._LLM_SENT_CACHE_MAX:
cache.pop(next(iter(cache)))
return copies
def _llm_event_direction(self, event: EconomicEvent) -> Optional[str]:
"""Gold direction for a released surprise, e.g. "bearish: strong
jobs lift yields". None → caller keeps the numeric description.