Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)

The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72%
agreement with DeepSeek on unseen days) replaces DeepSeek for headline
scoring. DeepSeek keeps event direction, the summary and the dashboard
endpoint.

- news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers,
  so the bot starts without them) + load-once, never-raising singleton.
- news_engine: _enrich_headline_scores mode switch (off/fallback/primary),
  _apply_model_sentiment (gold head only: xau_score + XAU entry, never
  USD; not-relevant shows 0.0; never touches _llm_sent_cache),
  _audit_with_deepseek (archive-only labels, deepseek_audit_per_day).
- Owner decision: primary mode writes nothing to logs/newsdata.db except
  audit copies — the model's output is not training data.
- app_config: news_model section (path/threads/mode/deepseek_audit_per_day)
  + NEWS_MODEL_MODE env override; validated at boot.
- news_archive: model rows carry teacher_model xau_transformer@<trained_at>.
- deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a
  commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers.
- test_all.py Module 38 (15 tests, one against the real bundle). Suite 581.
- CLAUDE.md, spec status note, handoff marked implemented.

Books unaffected: news never enters signals; no signal file touched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
2026-09-27 21:02:42 -04:00
parent 6af1237c2d
commit 0c56b4e155
13 changed files with 1966 additions and 34 deletions
+15 -5
View File
@@ -324,9 +324,17 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
teacher_prompt_sha1: Optional[str] = None,
origin: str = "live",
first_seen: Optional[str] = None,
last_seen: Optional[str] = None) -> Optional[Dict[str, Any]]:
last_seen: Optional[str] = None,
model_teacher: Optional[str] = None) -> Optional[Dict[str, Any]]:
"""Flatten one ``NewsSentiment`` (or an equivalent object) into a row.
Provenance: an ``llm`` row stores ``teacher_model`` (the DeepSeek
model name); a ``model`` row stores ``model_teacher`` (the local
scorer's tag, ``xau_transformer@<trained_at>``, news_model.teacher_tag).
Both carry ``teacher_prompt_sha1``: for a model row it is the DeepSeek
prompt the model was distilled from, which is the truthful lineage.
Keyword rows carry neither.
Returns None when the object carries no usable headline text."""
text = getattr(headline, "headline", None)
if not isinstance(text, str) or not text.strip():
@@ -349,9 +357,10 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
getattr(headline, "currencies_mentioned", None)),
"keywords_json": _json_or_none(getattr(headline, "keywords", None)),
"score_source": source,
"teacher_model": teacher_model if source == "llm" else None,
"teacher_model": (teacher_model if source == "llm"
else model_teacher if source == "model" else None),
"teacher_prompt_sha1": (
teacher_prompt_sha1 if source == "llm" else None),
teacher_prompt_sha1 if source in ("llm", "model") else None),
"origin": origin,
"xau_score": _finite_score(getattr(headline, "xau_score", None)),
}
@@ -364,7 +373,8 @@ _archives_lock = threading.Lock()
def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
teacher_prompt_sha1: Optional[str] = None,
origin: str = "live") -> None:
origin: str = "live",
model_teacher: Optional[str] = None) -> None:
"""Persist a batch of scored headlines if the news_archive config
block enables it. NEVER raises — a logging failure must not break a
scan."""
@@ -377,7 +387,7 @@ def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
for headline in headlines:
row = build_row(headline, teacher_model=teacher_model,
teacher_prompt_sha1=teacher_prompt_sha1,
origin=origin)
origin=origin, model_teacher=model_teacher)
if row is not None:
rows.append(row)
if not rows: