Harwinder Sekhon
42025da0b0
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72% agreement with DeepSeek on unseen days) replaces DeepSeek for headline scoring. DeepSeek keeps event direction, the summary and the dashboard endpoint. - news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers, so the bot starts without them) + load-once, never-raising singleton. - news_engine: _enrich_headline_scores mode switch (off/fallback/primary), _apply_model_sentiment (gold head only: xau_score + XAU entry, never USD; not-relevant shows 0.0; never touches _llm_sent_cache), _audit_with_deepseek (archive-only labels, deepseek_audit_per_day). - Owner decision: primary mode writes nothing to logs/newsdata.db except audit copies — the model's output is not training data. - app_config: news_model section (path/threads/mode/deepseek_audit_per_day) + NEWS_MODEL_MODE env override; validated at boot. - news_archive: model rows carry teacher_model xau_transformer@<trained_at>. - deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers. - test_all.py Module 38 (15 tests, one against the real bundle). Suite 581. - CLAUDE.md, spec status note, handoff marked implemented. Books unaffected: news never enters signals; no signal file touched.
82 lines
5.1 KiB
Markdown
82 lines
5.1 KiB
Markdown
# Handoff: the local news model and ForexBot2 (2026-09-27)
|
||
|
||
**Status: IMPLEMENTED later the same day** (`news_model.py`, `news_model` config section, Module 38,
|
||
`deploy/README-news-model.md`). Two decisions were added during implementation: `primary` mode writes
|
||
nothing to `logs/newsdata.db` except the DeepSeek audit sample, and both processes load the model. The
|
||
authoritative record is the status note at the top of `docs/news_model_integration_spec_2026-09-27.md`
|
||
and the CLAUDE.md ledger row; this file is kept as the historical handoff and can be deleted.
|
||
|
||
Read this before touching headline scoring. It records what exists, what the owner decided, and what
|
||
was left to build at the time of writing.
|
||
|
||
## What exists
|
||
|
||
The owner's news model lives in a separate project, `~/Development/ML-NEWS`
|
||
(git `ssh://git@192.168.0.11:222/harry/News-ML-Decoder.git`). On 2026-09-27 it became a
|
||
DeBERTa-v3-small two-head transformer, exported to ONNX for CPU:
|
||
|
||
- Question 1: does this headline move the gold price? Question 2: if so, up or down?
|
||
- Trained to copy DeepSeek's gold score on 17,245 headlines: the Sep 8–22 `newsdata.db` archive plus a
|
||
2019–2026 Reuters / CNBC / MarketWatch crawl that DeepSeek labelled (560 calls).
|
||
- Golden set 72/78. DeepSeek 78, the keyword scorer 24, the 2026-09-17 TF-IDF model 39.
|
||
- Agreement with DeepSeek on unseen live-feed days about 72% (DeepSeek agrees with itself about 90%).
|
||
- 19 ms per headline on one CPU thread, 9 ms on four. Needs only `onnxruntime`, `tokenizers`, `numpy`.
|
||
- Bundle: `ML-NEWS/models/xau_transformer/{model.onnx (271 MB), tokenizer.json, meta.json}`.
|
||
- Full results: `~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md`.
|
||
|
||
It passed the four promotion criteria fixed in advance on 2026-09-22 (golden ≥ 55/78, distractors ≥ 6/8,
|
||
unseen-days agreement ≥ 62.8%, latency ≤ 20 ms). This supersedes the 2026-09-17 verdict "a fallback,
|
||
not a replacement" in the CLAUDE.md ledger.
|
||
|
||
## What the owner decided (2026-09-27)
|
||
|
||
1. **Use it as the primary headline scorer, with no DeepSeek calls for headline scoring.** DeepSeek
|
||
stays for everything else: calendar-event direction (`_llm_event_direction`), the news summary, the
|
||
dashboard's on-demand summary endpoint.
|
||
2. **Model files live outside the ForexBot2 tree**, for example `/home/Forex/models/xau_transformer`,
|
||
copied by hand, path set in `config.json`. Never in git. ForexBot2 keeps deploying by `git pull`.
|
||
3. **No shadow mode.** The archive already stores every headline with its score, so offline replay
|
||
covers the comparison.
|
||
4. Accepted consequence: with DeepSeek off for headlines, the archive stops receiving fresh DeepSeek
|
||
labels, which is what ML-NEWS retrains on. An optional config knob (`deepseek_audit_per_day`,
|
||
default 0) can label a few headlines a day for the archive only.
|
||
|
||
## What is left to build
|
||
|
||
Everything. The specification is `docs/news_model_integration_spec_2026-09-27.md` in this repo
|
||
(identical to `ML-NEWS/docs/FOREXBOT2_INTEGRATION_SPEC.md`). In one screen:
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `requirements.txt` | add `onnxruntime`, `tokenizers` (no "torch"/"transformers" anywhere in the file; a test greps for them) |
|
||
| `news_model.py` (new) | vendored copy of ML-NEWS `news_transformer.py` + a load-once singleton that never raises; onnxruntime imported lazily |
|
||
| `app_config.py` | optional `news_model` section: `path`, `threads`, `mode` (`off`/`fallback`/`primary`), `deepseek_audit_per_day`; `NEWS_MODEL_MODE` env override |
|
||
| `news_engine.py` | `_apply_model_sentiment` + a mode switch at the call site (~line 945); model score goes into `xau_score` and the XAU entry only, never into USD |
|
||
| `news_archive.py` | store `teacher_model` for `model` rows (the `model` rank already exists) |
|
||
| `test_all.py` | 11 tests listed in the spec |
|
||
| `deploy/README-news-model.md` (new) | ops for both the bot and the dashboard daemon |
|
||
| `config.json` (live) | the new section |
|
||
|
||
Rules that must survive any implementation:
|
||
|
||
- The model has a gold head only. **Never write its score into a non-XAU currency**:
|
||
`get_pair_sentiment` computes `XAU_mean − USD_mean` and the signal aggregator acts on it.
|
||
- A headline the model calls not-relevant shows 0.0, whatever the raw score; the raw score still goes
|
||
to `xau_score` for training data.
|
||
- Never write model results into `_llm_sent_cache`; that cache decides what is re-sent to DeepSeek.
|
||
- If the model fails to load, `primary` falls back to the DeepSeek path for that cycle, not to keywords.
|
||
- Both the bot and `dashboard_publisher.py` build a `NewsEngine`; each process that loads the model
|
||
holds about 0.6 GB. Decide per process with the env override.
|
||
- News never enters signals. CLAUDE.md §7 applies unchanged; no book is affected by any of this.
|
||
|
||
## How to verify when it is built
|
||
|
||
`venv/bin/python test_all.py` green; one news cycle in `primary` mode archives only `score_source='model'`
|
||
rows with `xau_score` set and zero DeepSeek headline calls; renaming the model directory away still
|
||
starts both processes and yields `llm` rows; `mode="off"` reproduces today's rows exactly.
|
||
|
||
## Re-verify before editing
|
||
|
||
Every file:line number in the spec was checked on 2026-09-27 against `6af1237` and drifts with each
|
||
commit. Check them first.
|