diff --git a/docs/FOREXBOT2_INTEGRATION_SPEC.md b/docs/FOREXBOT2_INTEGRATION_SPEC.md index 00c389d..c52f87a 100644 --- a/docs/FOREXBOT2_INTEGRATION_SPEC.md +++ b/docs/FOREXBOT2_INTEGRATION_SPEC.md @@ -1,8 +1,28 @@ # Specification: run the ML-NEWS XAU transformer inside ForexBot2 -Status: **specification only, not implemented.** Written 2026-09-27 so a coding agent can implement it -later without this conversation. Every file:line reference was checked against the repos on that date; -re-verify before editing, the numbers drift. +Status: **IMPLEMENTED 2026-09-27** (ForexBot2 branch `sql-integration`, on top of `6af1237`; suite +Module 38). The text below is the specification as written before the implementation; file:line +numbers refer to `6af1237` and have drifted. Where the implementation differs, this list wins: + +- **`primary` mode does not archive the cycle's rows.** Owner decision during implementation: once the + local model scores, `logs/newsdata.db` stops growing, because the model's own output is not training + data. Only the DeepSeek audit copies (`deepseek_audit_per_day`, shipped 0) are written. `off` and + `fallback` archive as before (a `fallback` model row carries `model_teacher`). §5.4 and §9 read + accordingly. +- `_audit_with_deepseek` **returns** the `llm` copies; there is no `_pending_audit_rows` attribute. + The mode switch of §5.2 lives in `NewsEngine._enrich_headline_scores(scored, mode)`, which returns + those copies, so the call site and the tests share one function. +- `config.json.example` ships `"mode": "off"`; the live server sets `primary` + (`deploy/README-news-model.md`). +- §8 step 0 was wrong: the dashboard daemon never writes the archive in any mode + (`set_db_writes_for_process(False)`), so its DeepSeek rows are not "free audit labels". Both + processes load the model (owner decision); `NEWS_MODEL_MODE=off` stays as the memory fallback. +- `warm_up_in_background()` is implemented (module-level once flag; a `NewsEngine` is built per scan). +- `news_model.get_news_model()` also treats a missing `path` as a load failure (one warning). +- `requirements.txt` floors: `onnxruntime>=1.30,<2`, `tokenizers>=0.23.2,<1` (the versions that + verified the bundle). +- ML-NEWS `news_transformer.py` received the §3.4 edits in the same session; the vendored copy differs + only in `load_model` having no default directory. Repos: - **ML-NEWS** `~/Development/ML-NEWS` (git `ssh://git@192.168.0.11:222/harry/News-ML-Decoder.git`). diff --git a/news_transformer.py b/news_transformer.py index 71b9c1f..7755808 100644 --- a/news_transformer.py +++ b/news_transformer.py @@ -25,6 +25,7 @@ from __future__ import annotations import argparse import html import json +import os import sys from pathlib import Path from typing import Any, Iterable @@ -68,8 +69,11 @@ def load_model(model_dir: str | Path = DEFAULT_MODEL_DIR, tokenizer.enable_padding(pad_id=int(meta["pad_token_id"]), pad_token=meta["pad_token"]) options = ort.SessionOptions() if threads: - options.intra_op_num_threads = threads + options.intra_op_num_threads = max(1, min(int(threads), os.cpu_count() or 1)) options.inter_op_num_threads = 1 + # ORT intra-op threads spin-wait after each run by default; in a mostly + # idle bot process that is 4 cores at 100% between cycles. + options.add_session_config_entry("session.intra_op.allow_spinning", "0") session = ort.InferenceSession(str(model_dir / meta.get("onnx_file", "model.onnx")), options, providers=["CPUExecutionProvider"]) return {"meta": meta, "tokenizer": tokenizer, "session": session}