From 520180d3bd1d8cb0c72f6372fed9616775dee657 Mon Sep 17 00:00:00 2001 From: Harwinder Sekhon Date: Sun, 27 Sep 2026 20:04:35 -0400 Subject: [PATCH] Spec: running the XAU transformer inside ForexBot2 (not implemented) Implementation specification for a future coding agent: vendored ONNX runtime module, config section, news_engine call site and scoring rules (gold only, never into USD), archive provenance, tests, ops checklist for both the bot and the dashboard daemon. Owner decisions recorded: primary mode with no DeepSeek for headlines, model files outside the repo. --- docs/FOREXBOT2_INTEGRATION_SPEC.md | 514 +++++++++++++++++++++++++++++ 1 file changed, 514 insertions(+) create mode 100644 docs/FOREXBOT2_INTEGRATION_SPEC.md diff --git a/docs/FOREXBOT2_INTEGRATION_SPEC.md b/docs/FOREXBOT2_INTEGRATION_SPEC.md new file mode 100644 index 0000000..00c389d --- /dev/null +++ b/docs/FOREXBOT2_INTEGRATION_SPEC.md @@ -0,0 +1,514 @@ +# Specification: run the ML-NEWS XAU transformer inside ForexBot2 + +Status: **specification only, not implemented.** Written 2026-09-27 so a coding agent can implement it +later without this conversation. Every file:line reference was checked against the repos on that date; +re-verify before editing, the numbers drift. + +Repos: +- **ML-NEWS** `~/Development/ML-NEWS` (git `ssh://git@192.168.0.11:222/harry/News-ML-Decoder.git`). + Produces the model. Nothing in ForexBot2 imports from it. +- **ForexBot2** `~/Development/ForexBot2`, live at `/home/Forex/ForexBot2`, deployed by `git pull`. + Consumes the model. + +--- + +## 1. Context and goal + +### 1.1 What the model is +`ML-NEWS/models/xau_transformer/` holds a fine-tuned `microsoft/deberta-v3-small` with two heads, +exported to ONNX: + +| File | Size | Purpose | +|---|---|---| +| `model.onnx` | 271 MB | encoder + heads; int8 embedding table, fp32 compute | +| `tokenizer.json` | 8 MB | HF `tokenizers` fast tokenizer | +| `meta.json` | 4 KB | thresholds, `trained_at`, `relevance_threshold`, `export_checks`, training stats | + +Inputs: one headline, ≤ 64 tokens. Outputs per headline (from `news_transformer.score_batch`): + +```python +{"headline": str, "p_relevant": float, "p_bullish": float, + "relevant": bool, # p_relevant >= meta["relevance_threshold"] (0.2 for the shipped bundle) + "direction": "bullish"|"bearish", + "label": "bullish"|"bearish"|"neutral", # neutral when not relevant + "score": float} # p_relevant * (2*p_bullish - 1), in [-1, 1] +``` + +`score` is on the same scale as DeepSeek's `xau` score that ForexBot2 already stores in +`xau_score`. The model was trained to copy DeepSeek (`deepseek-v4-flash`, prompt sha1 `d32e04c6…`, the +same prompt as `news_engine._LLM_SENTIMENT_PROMPT_SHA1`) on 17,245 headlines. Quality on +2026-09-27: golden set 72/78 (DeepSeek 78, keyword scorer 24), agreement with DeepSeek on unseen live +days ~72%, CPU latency 19 ms/headline on 1 thread, 9 ms on 4. Full numbers: `ML-NEWS/eval/RESULTS_2026-09-27.md`. + +**The model scores gold only.** It has no head for USD, EUR or anything else. This drives most of +the rules in §5. + +### 1.2 How ForexBot2 scores headlines today +`news_engine.py`, inside `NewsEngine._fetch_news_sentiment_raw` (called via +`_cached_fetch("_headlines_cache", …)` at :822, which holds `NewsEngine._cache_lock`, :629-647): + +1. RSS items become `NewsSentiment` objects (:891-910) with the **keyword scorer** filling + `sentiment_score` and `sentiment_by_currency[cur]` for each currency in `currencies_mentioned`, + `score_source="keyword"`, `xau_score=None`. +2. `scored = headlines + unmatched` (:944); `self._apply_llm_sentiment(scored)` (:945, in try/except). + `_apply_llm_sentiment` (:1032-1065) sends headlines not in `NewsEngine._llm_sent_cache` to DeepSeek + in batches of `_LLM_BATCH_CAP = 25` via `self._llm_headline_scores(texts)` (:1009) and, on success, + overwrites `sentiment_score`, `sentiment_by_currency` (XAU gets the gold score, every other currency + the overall score), sets `xau_score`, `score_source="llm"`. A failed batch stops sending; those + headlines stay keyword-scored and **stay out of the cache**, so they are re-sent next window. +3. `news_archive.archive_headlines(scored, teacher_model=…, teacher_prompt_sha1=…)` (:958-964). + The archive UPSERT (news_archive.py:203-260) keeps the higher-ranked source: + `_SOURCE_RANK = {"unknown": 0, "keyword": 1, "model": 2, "llm": 3}` (:107). `"model"` is already + reserved for "a trained local artifact" (news_engine.py:223-227). `build_row` (:323-354) stores + `teacher_model`/`teacher_prompt_sha1` only when `source == "llm"`. +4. Downstream consumers of the scores: `get_fundamental_context` (drops |score| < + `NEUTRAL_HEADLINE_THRESHOLD = 0.1`, :256), `get_pair_sentiment` (:553-557, computes + `XAU_mean − USD_mean`), `signal_aggregator.py:1677/1723` (acts on that when + `sentiment_affects_signals`), `agent_panel.py`, the dashboard. + +Two processes run this code: the bot (`venv/bin/python telegram_bot.py`, CLAUDE.md:30) and the +dashboard daemon (`dashboard_publisher.py:401-403` builds its own `NewsEngine`, `:503` calls +`get_fundamental_context()`; unit `deploy/gold-dashboard.service`). They share `config.json` and the +archive DB. + +### 1.3 Decisions taken by the owner (2026-09-27) +- **Mode `primary`, no DeepSeek for headline scoring.** The model scores every headline. DeepSeek stays + for everything else it does (`_llm_event_direction` for calendar surprises, the news summary, the + dashboard's on-demand `news_summary.php`). +- **Model files live outside the ForexBot2 tree**, path in `config.json`. +- Accepted consequence: the archive stops receiving fresh DeepSeek labels, so the ML-NEWS weekly + retrain and the "replay and compare" check lose their data source. Mitigation kept optional: + `deepseek_audit_per_day` (§4) labels a few headlines a day for the archive only. Default 0. + +### 1.4 Non-goals +- Removing the DeepSeek client or its non-headline uses. +- Scoring non-gold currencies with the model. +- Changing ML-NEWS training. (The one ML-NEWS change is §3.4, a runtime option.) +- Training from price moves. + +--- + +## 2. Target layout on the live server + +``` +/home/Forex/ForexBot2/ # git pull as today + news_model.py # NEW, vendored runtime (§3) + requirements.txt # + onnxruntime, tokenizers (§3.1) + app_config.py, news_engine.py, news_archive.py, test_all.py # edited (§4, §5, §6, §7) + config.json # + "news_model" section (§4) + deploy/README-news-model.md # NEW ops checklist (§8) + deploy/gold-dashboard.service # maybe + Environment=NEWS_MODEL_MODE=off (§8) +/home/Forex/models/xau_transformer/ # NEW, outside git, copied by hand + model.onnx tokenizer.json meta.json +``` + +Memory: each process that loads the model holds ~0.5–0.7 GB RSS for the ONNX session. Two processes +→ ~1.5 GB. `free -m` before deciding whether the dashboard daemon also runs it (§8 step 0). + +--- + +## 3. `news_model.py` (new, ForexBot2) + +### 3.1 Dependencies +Add to `requirements.txt`: +``` +onnxruntime>=1.30,<2 +tokenizers>=0.23,<1 +``` +Pin to the versions that verified the bundle in ML-NEWS `.venv` (check `pip show` there before +writing). `tokenizer.json` compatibility across older `tokenizers` releases is not guaranteed. +**Do not write the words "torch" or "transformers" anywhere in requirements.txt**, not even in a +comment: `test_all.py:11437-11462` greps the file for them and fails. + +### 3.2 Source +Copy `ML-NEWS/news_transformer.py` → `ForexBot2/news_model.py`. Keep verbatim: `clean_headline`, +`_sigmoid`, `load_model`, `score_batch`, `score_news`, constants `MAX_LENGTH = 64`, +`RELEVANT_MIN = 0.25`, `RELEVANT_MAX_IRRELEVANT = 0.10`. Drop the CLI `main` or keep it, either is fine. +Add a docstring line: `Vendored from ML-NEWS news_transformer.py at commit ; bundle trained_at +.` + +**Imports.** `onnxruntime` and `tokenizers` must stay imported *inside* `load_model` (as in +news_transformer.py:58-59). Module top imports only `numpy`, `json`, `logging`, `os`, `pathlib`, +`threading`. Reason: `news_engine.py` will `import news_model` at its top; if onnxruntime is missing on +a box, the bot must still start. + +### 3.3 Singleton +```python +_lock = threading.Lock() +_bundle: Optional[dict] = None +_failed: bool = False # tried and failed; don't retry, don't re-log + +def get_news_model() -> Optional[dict]: + """The loaded bundle, or None. Loads once per process. Never raises.""" + global _bundle, _failed + if _bundle is not None or _failed: + return _bundle + with _lock: + if _bundle is not None or _failed: + return _bundle + try: + import app_config + path = app_config.news_model_path() + threads = app_config.news_model_threads() + b = load_model(path, threads=threads) + logger.info("news model loaded from %s (trained_at %s, threshold %s, %d threads)", + path, b["meta"].get("trained_at"), b["meta"].get("relevance_threshold"), threads) + _bundle = b + except Exception as e: # ImportError, FileNotFoundError, bad JSON, ORT errors + logger.warning("news model unavailable, DeepSeek/keyword path stays: %s: %s", + type(e).__name__, e) + _failed = True + return _bundle + +def reset_for_tests() -> None: # clears _bundle/_failed; used by test_all.py +``` + +Optional: `warm_up_in_background()` that calls `get_news_model()` in a daemon thread at +`NewsEngine.__init__`, so the first news cycle isn't blocked by the ~3–5 s session creation under +`_cache_lock`. Nice to have, not required. + +### 3.4 Two changes inside `load_model` (also apply to ML-NEWS `news_transformer.py` so the copies stay identical) +```python +options = ort.SessionOptions() +if threads: + options.intra_op_num_threads = max(1, min(int(threads), os.cpu_count() or 1)) + options.inter_op_num_threads = 1 +# ORT intra-op threads spin-wait after each run by default; in a mostly idle +# bot process that is 4 cores at 100% between cycles. +options.add_session_config_entry("session.intra_op.allow_spinning", "0") +``` + +### 3.5 Contract `score_batch(bundle, headlines: list[str]) -> list[dict]` +Unchanged from ML-NEWS. Empty or whitespace-only headline raises `ValueError` from `clean_headline`; +callers pass already-non-empty `NewsSentiment.headline`, but §5 still wraps per-batch in try/except. +Batch size: `_apply_model_sentiment` sends all of a cycle's headlines in one call (typically 30–150); +`score_batch` pads to the longest in the batch, ≤ 64 tokens, so one call of 150 is ~1 s on 4 threads. + +--- + +## 4. Configuration + +### 4.1 `config.json` section (add to `config.json.example` too) +```json +"news_model": { + "path": "/home/Forex/models/xau_transformer", + "threads": 4, + "mode": "primary", + "deepseek_audit_per_day": 0 +} +``` + +| Key | Type | Default if absent | Meaning | +|---|---|---|---| +| `path` | str | required when mode ≠ off | directory with the three bundle files | +| `threads` | int ≥ 1 | 4 | ORT intra-op threads (measured: 1→19 ms, 4→9 ms, 16→13 ms) | +| `mode` | `"off"`/`"fallback"`/`"primary"` | `"off"` | see §5.2 | +| `deepseek_audit_per_day` | int ≥ 0 | 0 | primary mode only: headlines/day to also send to DeepSeek, archive-only | + +Whole section absent → `mode="off"` → today's behaviour exactly. + +Environment override: `NEWS_MODEL_MODE=off|fallback|primary` wins over config.json. Purpose: the +dashboard daemon's systemd unit can turn the model off for that process only. Pattern to copy: +`NEWS_ARCHIVE_ENABLED` in `app_config.py:574-583`. + +### 4.2 `app_config.py` edits (follow the `news_archive` section, :397-406, accessors :574/:588) +1. `_KNOWN_TOP` (:86-92): add `"news_model"`. **Required**; `_check_keys(raw, _KNOWN_TOP, "top level")` + at :181 makes boot fail on an unknown top-level key. +2. `_KNOWN_NEWS_MODEL = {"path", "threads", "mode", "deepseek_audit_per_day"}` next to + `_KNOWN_NEWS_ARCHIVE` (:113). +3. `AppConfig` dataclass (~:147): `news_model_settings: Dict[str, Any] = field(default_factory=dict)`. +4. In `load_app_config`, after the `news_archive` block: + ```python + nm = raw.get("news_model", {}) + _check_keys(nm, _KNOWN_NEWS_MODEL, '"news_model"') + if "mode" in nm and nm["mode"] not in ("off", "fallback", "primary"): + raise AppConfigError('config.json: news_model.mode must be "off", "fallback" or "primary"') + if "threads" in nm and (not isinstance(nm["threads"], int) or isinstance(nm["threads"], bool) or nm["threads"] < 1): + raise AppConfigError("config.json: news_model.threads must be a positive integer") + if "deepseek_audit_per_day" in nm and (not isinstance(nm["deepseek_audit_per_day"], int) or nm["deepseek_audit_per_day"] < 0): + raise AppConfigError("config.json: news_model.deepseek_audit_per_day must be a non-negative integer") + if nm.get("mode", "off") != "off" and not isinstance(nm.get("path"), str): + raise AppConfigError("config.json: news_model.path is required when mode is not off") + app.news_model_settings = dict(nm) + ``` +5. Accessors: + ```python + def news_model_mode() -> str: # env NEWS_MODEL_MODE overrides; default "off" + def news_model_path() -> Optional[str] + def news_model_threads() -> int # default 4 + def news_model_audit_per_day() -> int # default 0 + ``` +`NewsConfig` in `config.py:169` is **not** touched: tests construct it bare as `NewsConfig()`, and it +holds feed/threshold settings, not paths. `news_engine.py` reads the new settings through `app_config` +the same way it reaches DeepSeek through `get_llm_client()`. + +--- + +## 5. `news_engine.py` changes + +### 5.1 New method `_apply_model_sentiment` +Place next to `_apply_llm_sentiment` (:1032). Runs under `_cache_lock` like it. + +```python +def _apply_model_sentiment(self, headlines: List[NewsSentiment]) -> int: + """Score with the local XAU transformer every headline that is still + keyword-scored. Returns the number scored. Never raises. + + The model has a gold head only. It writes xau_score always, XAU's + per-currency score when XAU is mentioned, and the overall score only + when the headline names no other currency. USD/EUR/... entries keep + their keyword score: get_pair_sentiment subtracts USD from XAU and + must not see the same number on both sides. + """ + bundle = news_model.get_news_model() + if bundle is None or not headlines: + return 0 + todo = [h for h in headlines if h.score_source == "keyword"] + if not todo: + return 0 + try: + results = news_model.score_batch(bundle, [h.headline for h in todo]) + except Exception as e: + logger.debug(f"news model scoring failed, keyword scores stand: {e}") + return 0 + for h, r in zip(todo, results): + raw = float(r["score"]) # archive value, unmodified + shown = raw if r["relevant"] else 0.0 # display value, see rule D + h.xau_score = raw + if "XAU" in h.currencies_mentioned: + h.sentiment_by_currency["XAU"] = shown + if set(h.currencies_mentioned) <= {"XAU"}: # [] or ["XAU"] + h.sentiment_score = shown + h.score_source = "model" + return len(todo) +``` + +Rules, with the reason each exists: + +- **A. Never write the model score into a non-XAU currency.** `get_pair_sentiment` (:553-557) forms + `XAU_mean − USD_mean`; `signal_aggregator.py:1677/1723` acts on it. The DeepSeek path puts its + *overall* score into USD; the model has no overall score, so USD keeps the keyword number. +- **B. `sentiment_score` from the model only when the headline names no other currency.** The archive's + `score` column is `sentiment_score`, and the ML-NEWS replay eval reads it; for a gold-only headline it + should be the model's number, for "Fed hikes, dollar jumps" it stays keyword so the row is honest + about what the model did and didn't score. `score_source="model"` therefore means "XAU score is the + model's" — amend the dataclass comment at :223-227 to say so. +- **C. `xau_score` is always the raw signed score.** That column exists for training data + (:229-237); keep it unfiltered. +- **D. Display fields get 0.0 when `relevant` is False.** A headline the model calls not-relevant can + still carry |score| up to ~0.2 from a confident direction head; `NEUTRAL_HEADLINE_THRESHOLD = 0.1` + (:256) would let 0.15 through as "leaning bearish". Zeroing the shown value makes "neutral" mean + neutral. (0.1 happens to equal the model's irrelevant band `RELEVANT_MAX_IRRELEVANT`, so the two + notions agree.) +- **E. Do not touch `NewsEngine._llm_sent_cache`.** That cache decides what is re-sent to DeepSeek. In + `fallback` mode a headline that failed DeepSeek must stay uncached so it is retried next window and + its archive row upgrades `model` → `llm` through the rank rule (news_archive.py:203-260). +- **F. Only `score_source == "keyword"` rows are touched.** Never overwrite an `llm` row in memory. + +### 5.2 Call site (:944-947) +Replace +```python +scored = headlines + unmatched +try: + self._apply_llm_sentiment(scored) +except Exception as e: + logger.debug(f"LLM sentiment failed, keyword scores stand: {e}") +``` +with +```python +scored = headlines + unmatched +mode = app_config.news_model_mode() +try: + if mode == "primary": + n = self._apply_model_sentiment(scored) + if n == 0 and news_model.get_news_model() is None: + # model unavailable this process: degrade to DeepSeek, not to keywords + self._apply_llm_sentiment(scored) + else: + self._audit_with_deepseek(scored) # no-op when audit_per_day == 0 + elif mode == "fallback": + self._apply_llm_sentiment(scored) + self._apply_model_sentiment(scored) # only what DeepSeek left keyword-scored + else: + self._apply_llm_sentiment(scored) +except Exception as e: + logger.debug(f"headline sentiment enrichment failed, keyword scores stand: {e}") +``` +`import news_model` and `import app_config` at the module top of news_engine.py (news_model's top +imports only numpy, §3.2). + +Mode semantics: + +| mode | DeepSeek for headlines | model | when a step fails | +|---|---|---|---| +| `off` | as today | never | keyword scores stand (today) | +| `fallback` | first, as today | on rows DeepSeek left keyword-scored | keyword scores stand | +| `primary` | never (except audit) | on all rows | model missing → DeepSeek path for the cycle; model raises mid-batch → keyword | + +### 5.3 Audit sample (primary mode, optional feature, default off) +```python +_audit_day: str = "" # class attrs, like the caches +_audit_sent_today: int = 0 + +def _audit_with_deepseek(self, headlines): + cap = app_config.news_model_audit_per_day() + if cap <= 0 or not get_llm_client().available: + return + today = datetime.now(timezone.utc).date().isoformat() + if NewsEngine._audit_day != today: + NewsEngine._audit_day, NewsEngine._audit_sent_today = today, 0 + room = cap - NewsEngine._audit_sent_today + if room <= 0: + return + cand = [h for h in headlines if h.score_source == "model" + and hashlib.sha1(h.headline.encode()).hexdigest() not in NewsEngine._llm_sent_cache] + batch = cand[:min(room, self._LLM_BATCH_CAP)] + if not batch: + return + scores = self._llm_headline_scores([h.headline for h in batch]) + if scores is None: + return + NewsEngine._audit_sent_today += len(batch) + copies = [] + for h, (overall, xau) in zip(batch, scores): + NewsEngine._llm_sent_cache[hashlib.sha1(h.headline.encode()).hexdigest()] = (overall, xau) + c = copy.copy(h) # the displayed object keeps the model score + c.sentiment_score = overall + c.sentiment_by_currency = {cur: (xau if cur == "XAU" else overall) for cur in h.currencies_mentioned} + c.xau_score = xau + c.score_source = "llm" + copies.append(c) + self._pending_audit_rows = copies # archived by the call site right after `scored` +``` +The call site then archives `scored` first and `self._pending_audit_rows` second (same +`teacher_model`/`teacher_prompt_sha1` as today's DeepSeek rows); the rank rule upgrades those rows +to `llm`, the in-memory objects the bot displays keep the model's numbers. One batch per cycle at +most, ≤ `cap` per UTC day. This is the only place DeepSeek is called for headlines in primary mode. + +### 5.4 Archive call (:958-964) +Pass, in addition to today's kwargs: +```python +model_teacher=news_model.teacher_tag() # "xau_transformer@" or None +``` +where `teacher_tag()` reads the loaded bundle's meta (None if not loaded). See §6. + +### 5.5 Dataclass comment +`NewsSentiment.score_source` comment (:223-227): add `"model" = the local XAU transformer scored +xau_score (and XAU's per-currency score); other currencies remain keyword-scored.` + +--- + +## 6. `news_archive.py` changes + +No schema change. `_SOURCE_RANK` (:107) and `_STORED_RANK` (:203-206) already rank `model` = 2. + +- `archive_headlines(headlines, *, teacher_model=None, teacher_prompt_sha1=None, origin="live", + model_teacher=None)` — new kwarg, passed through to `build_row`. +- `build_row` (:323-354): + ```python + "teacher_model": (teacher_model if source == "llm" + else model_teacher if source == "model" else None), + "teacher_prompt_sha1": (teacher_prompt_sha1 if source in ("llm", "model") else None), + ``` + For a model row `teacher_prompt_sha1` is the DeepSeek prompt the model was distilled from (the same + constant the bot already passes), which is the truthful provenance. +- Effect: a retrain is visible in the data (`teacher_model` changes), and the UPSERT's + `COALESCE(excluded.teacher_model, headlines.teacher_model)` (:253-260) only applies when the new + rank ≥ stored rank, so an `llm` row's teacher is never replaced by a model tag. + +--- + +## 7. Tests (`test_all.py`) + +The suite is a plain script: `@test(name)` (:51-64) runs each function at definition time and +tallies; the file ends with `sys.exit(1 if FAIL > 0 else 0)`. Run: `venv/bin/python test_all.py`. +Stubbing patterns in use: instance-level `ne._llm_headline_scores = lambda texts: …` with +`lc._client = _StubLLM({...})` and `finally` cleanup (:5196-5215); module-level swap +`saved = _ne_mod.get_llm_client; _ne_mod.get_llm_client = stub; … finally restore` (:11391-11424). + +Add, using the same patterns and a fake bundle (`news_model.score_batch` swapped on the module): + +1. **model scores keyword rows only.** Two `NewsSentiment`s, one `score_source="llm"`; after + `_apply_model_sentiment`, the llm one is byte-identical, the keyword one has `score_source="model"`, + `xau_score` set, `sentiment_by_currency["XAU"]` = score, `["USD"]` unchanged (rule A). +2. **overall score rule (B).** `currencies_mentioned=["XAU"]` → `sentiment_score` = model score; + `["USD","XAU"]` → `sentiment_score` unchanged; `[]` → model score. +3. **relevance zeroing (D).** stub returns `relevant=False, score=-0.18` → `sentiment_by_currency["XAU"] + == 0.0`, `xau_score == -0.18`. +4. **cache untouched (E).** `_llm_sent_cache` empty before and after. +5. **failure isolation.** stub raises → all rows still `keyword`, method returns 0, no exception. + `news_model.get_news_model()` returning None → returns 0. +6. **primary degrades to DeepSeek.** `get_news_model` → None, `_llm_headline_scores` stubbed → rows + end up `llm`. +7. **audit sample.** `deepseek_audit_per_day=3`, 10 model rows → exactly 3 sent, displayed objects + still `model`, `_pending_audit_rows` has 3 `llm` copies; second call same day sends 0. +8. **config.** unknown key in `news_model` → `AppConfigError`; `mode="banana"` → error; section + absent → `news_model_mode() == "off"`; `NEWS_MODEL_MODE=off` env beats `"mode": "primary"`. +9. **archive.** `build_row` of a `model` headline with `model_teacher="xau_transformer@T"` stores it; + an `llm` row stores the DeepSeek teacher; existing rank tests still pass. +10. **import hygiene.** update the docstring of the test at :11437-11462 (it says numpy-only); add + an assertion that `requirements.txt` lists `onnxruntime` and `tokenizers`; keep its "no torch / + transformers" checks. +11. **integration, skipped when the bundle is absent** (`os.environ.get("NEWS_MODEL_DIR")` or the + configured path missing → print SKIP): load, `score_news(b, "Fed cuts rates, signals more + easing")["label"] == "bullish"`, `score_news(b, "Red Lobster endless shrimp losses")["label"] == + "neutral"`, latency of 20 calls < 1 s. + +`test_all.py:3675-3690` loads `config.json.example` for pair output only; adding the section there is +safe. + +--- + +## 8. Ops checklist → `deploy/README-news-model.md` (new) + +Covers **both** processes: the bot (`telegram_bot.py`) and the dashboard daemon +(`deploy/gold-dashboard.service`). + +0. `free -m`. Each model-loading process needs ~0.7 GB headroom. If only the bot should score with + the model, add `Environment=NEWS_MODEL_MODE=off` to `deploy/gold-dashboard.service` (the dashboard + then keeps using DeepSeek for its headlines; those `llm` rows land in the shared archive and act + as free audit labels). +1. `cd /home/Forex/ForexBot2 && git pull && venv/bin/pip install -r requirements.txt`. +2. `mkdir -p /home/Forex/models/xau_transformer` and copy `model.onnx`, `tokenizer.json`, `meta.json` + from the ML-NEWS machine (rsync, ~280 MB). Verify `sha256sum model.onnx` matches the source. + Owner: the account the bot runs as; read-only is enough. +3. Add the `news_model` section to `config.json` (§4.1) with `"mode": "primary"`. Leave the + `deepseek` section unchanged. +4. Smoke test without starting the bot: + ``` + venv/bin/python -c "import news_model as m; b=m.get_news_model(); print(m.score_news(b,'Gold jumps as dollar slides'))" + ``` + Expect `label: bullish`, and the "news model loaded" log line. +5. `venv/bin/python test_all.py` — green (integration test runs because the dir exists). +6. Restart the bot; restart the dashboard unit. Check: + - log: one "news model loaded" line per process, no "news model unavailable"; + - `top`: RSS ~+0.6 GB per process, idle CPU ~0% between scans (spinning disabled); + - archive: new rows have `score_source='model'`, `xau_score` set, `teacher_model` + `xau_transformer@…`; DeepSeek daily counter for headlines stays at 0 (audit off). +7. Rollback: set `"mode": "fallback"` or `"off"` (or the env var), restart. No data migration needed. +8. Retrain cadence: when ML-NEWS ships a new bundle, copy the three files over the old ones and + restart both processes. `meta.json.trained_at` changes, so the archive shows which model scored which + rows. If `deepseek_audit_per_day` is set, ML-NEWS's `eval/eval_archive.py --scorer transformer` on a + copy of the live archive gives the model-vs-DeepSeek agreement on those rows. + +--- + +## 9. Acceptance criteria (end-to-end) + +- With `mode="primary"` and DeepSeek enabled, one live news cycle archives only `model` rows (plus + ≤ `deepseek_audit_per_day` `llm` rows), and the cycle's added latency is < 1 s for ~100 headlines. +- With the model directory renamed away, both processes start, log one warning each, and the cycle's + rows are `llm` (DeepSeek path), not `keyword`. +- With `onnxruntime` uninstalled in a scratch venv, `python -c "import news_engine"` succeeds. +- A headline naming USD and XAU ends with `sentiment_by_currency["USD"]` equal to the keyword value + and `["XAU"]` equal to the model value. +- `mode="off"` reproduces today's archive rows exactly (`llm`/`keyword`, same teacher fields). +- `venv/bin/python test_all.py` exits 0. + +## 10. Known limits to state in the README +- The model copies DeepSeek's opinion (72% agreement on unseen live days, 72/78 golden); it does not + learn from price. DeepSeek itself is ~90% self-consistent. +- Gold only. Every other currency's sentiment remains the ~31%-accurate keyword scorer in primary + mode, which is what those currencies get today whenever DeepSeek is unavailable. +- With audit off, no new DeepSeek labels accrue; the ML-NEWS retrain loop needs either + `deepseek_audit_per_day > 0`, the dashboard left on DeepSeek, or a separate labelling run.