Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72% agreement with DeepSeek on unseen days) replaces DeepSeek for headline scoring. DeepSeek keeps event direction, the summary and the dashboard endpoint. - news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers, so the bot starts without them) + load-once, never-raising singleton. - news_engine: _enrich_headline_scores mode switch (off/fallback/primary), _apply_model_sentiment (gold head only: xau_score + XAU entry, never USD; not-relevant shows 0.0; never touches _llm_sent_cache), _audit_with_deepseek (archive-only labels, deepseek_audit_per_day). - Owner decision: primary mode writes nothing to logs/newsdata.db except audit copies — the model's output is not training data. - app_config: news_model section (path/threads/mode/deepseek_audit_per_day) + NEWS_MODEL_MODE env override; validated at boot. - news_archive: model rows carry teacher_model xau_transformer@<trained_at>. - deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers. - test_all.py Module 38 (15 tests, one against the real bundle). Suite 581. - CLAUDE.md, spec status note, handoff marked implemented. Books unaffected: news never enters signals; no signal file touched.
This commit is contained in:
+15
-5
@@ -324,9 +324,17 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
|
||||
teacher_prompt_sha1: Optional[str] = None,
|
||||
origin: str = "live",
|
||||
first_seen: Optional[str] = None,
|
||||
last_seen: Optional[str] = None) -> Optional[Dict[str, Any]]:
|
||||
last_seen: Optional[str] = None,
|
||||
model_teacher: Optional[str] = None) -> Optional[Dict[str, Any]]:
|
||||
"""Flatten one ``NewsSentiment`` (or an equivalent object) into a row.
|
||||
|
||||
Provenance: an ``llm`` row stores ``teacher_model`` (the DeepSeek
|
||||
model name); a ``model`` row stores ``model_teacher`` (the local
|
||||
scorer's tag, ``xau_transformer@<trained_at>``, news_model.teacher_tag).
|
||||
Both carry ``teacher_prompt_sha1``: for a model row it is the DeepSeek
|
||||
prompt the model was distilled from, which is the truthful lineage.
|
||||
Keyword rows carry neither.
|
||||
|
||||
Returns None when the object carries no usable headline text."""
|
||||
text = getattr(headline, "headline", None)
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
@@ -349,9 +357,10 @@ def build_row(headline: Any, *, teacher_model: Optional[str] = None,
|
||||
getattr(headline, "currencies_mentioned", None)),
|
||||
"keywords_json": _json_or_none(getattr(headline, "keywords", None)),
|
||||
"score_source": source,
|
||||
"teacher_model": teacher_model if source == "llm" else None,
|
||||
"teacher_model": (teacher_model if source == "llm"
|
||||
else model_teacher if source == "model" else None),
|
||||
"teacher_prompt_sha1": (
|
||||
teacher_prompt_sha1 if source == "llm" else None),
|
||||
teacher_prompt_sha1 if source in ("llm", "model") else None),
|
||||
"origin": origin,
|
||||
"xau_score": _finite_score(getattr(headline, "xau_score", None)),
|
||||
}
|
||||
@@ -364,7 +373,8 @@ _archives_lock = threading.Lock()
|
||||
|
||||
def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
|
||||
teacher_prompt_sha1: Optional[str] = None,
|
||||
origin: str = "live") -> None:
|
||||
origin: str = "live",
|
||||
model_teacher: Optional[str] = None) -> None:
|
||||
"""Persist a batch of scored headlines if the news_archive config
|
||||
block enables it. NEVER raises — a logging failure must not break a
|
||||
scan."""
|
||||
@@ -377,7 +387,7 @@ def archive_headlines(headlines: Any, *, teacher_model: Optional[str] = None,
|
||||
for headline in headlines:
|
||||
row = build_row(headline, teacher_model=teacher_model,
|
||||
teacher_prompt_sha1=teacher_prompt_sha1,
|
||||
origin=origin)
|
||||
origin=origin, model_teacher=model_teacher)
|
||||
if row is not None:
|
||||
rows.append(row)
|
||||
if not rows:
|
||||
|
||||
Reference in New Issue
Block a user