Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72% agreement with DeepSeek on unseen days) replaces DeepSeek for headline scoring. DeepSeek keeps event direction, the summary and the dashboard endpoint. - news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers, so the bot starts without them) + load-once, never-raising singleton. - news_engine: _enrich_headline_scores mode switch (off/fallback/primary), _apply_model_sentiment (gold head only: xau_score + XAU entry, never USD; not-relevant shows 0.0; never touches _llm_sent_cache), _audit_with_deepseek (archive-only labels, deepseek_audit_per_day). - Owner decision: primary mode writes nothing to logs/newsdata.db except audit copies — the model's output is not training data. - app_config: news_model section (path/threads/mode/deepseek_audit_per_day) + NEWS_MODEL_MODE env override; validated at boot. - news_archive: model rows carry teacher_model xau_transformer@<trained_at>. - deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers. - test_all.py Module 38 (15 tests, one against the real bundle). Suite 581. - CLAUDE.md, spec status note, handoff marked implemented. Books unaffected: news never enters signals; no signal file touched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
@@ -0,0 +1,185 @@
|
||||
# Deploying the local news model (live server)
|
||||
|
||||
Since 2026-09-27 ForexBot2 can score news headlines with the owner's ML-NEWS gold transformer
|
||||
instead of DeepSeek. The model runs on the CPU inside the bot process (and inside the dashboard
|
||||
daemon), needs two extra Python packages and a 280 MB model folder that lives **outside** the git
|
||||
tree. DeepSeek keeps its other jobs: the direction of a calendar surprise, the news summary and the
|
||||
dashboard's on-demand summary endpoint.
|
||||
|
||||
What the model is, in one paragraph: a DeBERTa-v3-small transformer with two heads (does this
|
||||
headline move gold? if so, up or down?) exported to ONNX, trained to copy DeepSeek's gold score on
|
||||
17,245 headlines. On the 78-headline golden set it scores 72 (DeepSeek 78, the old keyword scorer
|
||||
24); on unseen live days it agrees with DeepSeek about 72% of the time (DeepSeek agrees with itself
|
||||
about 90%). It scores one headline in about 10–20 ms on one CPU thread. It is **gold only**: every
|
||||
other currency's headline sentiment stays with the keyword scorer in `primary` mode.
|
||||
|
||||
Owner decisions this deployment implements:
|
||||
|
||||
- Mode `primary`: the model scores every headline; **no DeepSeek calls for headline scoring**.
|
||||
- **The news archive stops growing.** In `primary` mode the model's own output is not archived (it is
|
||||
not training data). Only the optional DeepSeek audit sample (`deepseek_audit_per_day`, shipped 0)
|
||||
writes rows. `logs/newsdata.db` stays as it is; nothing is migrated or deleted.
|
||||
- Both processes (bot and dashboard daemon) load the model so Telegram and the dashboard show the same
|
||||
headline scores. Each ONNX session holds about 0.6 GB of RAM.
|
||||
|
||||
## 0. Check memory
|
||||
|
||||
```bash
|
||||
free -m
|
||||
```
|
||||
|
||||
Each process that loads the model needs about 0.7 GB of headroom; two processes about 1.5 GB. If the
|
||||
box cannot spare that, keep the model on the bot only: uncomment the `Environment=NEWS_MODEL_MODE=off`
|
||||
line in `deploy/gold-dashboard.service` (step 5). The dashboard then keeps scoring its headlines with
|
||||
DeepSeek and its scores may differ from the bot's. The dashboard process never writes the archive in
|
||||
any mode, so this choice does not affect training data.
|
||||
|
||||
## 1. Code and dependencies
|
||||
|
||||
```bash
|
||||
cd /home/Forex/ForexBot2
|
||||
git pull
|
||||
venv/bin/pip install -r requirements.txt # adds onnxruntime and tokenizers
|
||||
```
|
||||
|
||||
`requirements.txt` must never contain the training stack (no torch, no transformers); the test suite
|
||||
checks that. The live box runs Python 3.9, where pip resolves onnxruntime 1.19.2 and tokenizers
|
||||
0.22.2; the bundle of 2026-09-27 scores identically there and on the dev box (1.30.0 / 0.23.2).
|
||||
|
||||
## 2. Copy the model files (outside git)
|
||||
|
||||
On the machine that holds ML-NEWS (`~/Development/ML-NEWS/models/xau_transformer/`), copy exactly
|
||||
three files. Skip the `torch/` and `torch_final/` subfolders; they are training checkpoints.
|
||||
|
||||
```bash
|
||||
ssh Forex@LIVE 'mkdir -p /home/Forex/models/xau_transformer'
|
||||
rsync -av --progress \
|
||||
~/Development/ML-NEWS/models/xau_transformer/{model.onnx,tokenizer.json,meta.json} \
|
||||
Forex@LIVE:/home/Forex/models/xau_transformer/
|
||||
```
|
||||
|
||||
Verify on the live box:
|
||||
|
||||
```bash
|
||||
sha256sum /home/Forex/models/xau_transformer/model.onnx
|
||||
# bundle of 2026-09-27: 9e7a64d3ae1df335059ce829cafcf6e4544b85a4f72cc88923c2e42c48e975a0
|
||||
sha256sum /home/Forex/models/xau_transformer/tokenizer.json
|
||||
# bundle of 2026-09-27: 045f93f80eb9ad18e559647563fb827dea59d93258fdb8b922de82bc53666d16
|
||||
python3 -c "import json; print(json.load(open('/home/Forex/models/xau_transformer/meta.json'))['trained_at'])"
|
||||
# 2026-09-27T23:15:30.403442+00:00
|
||||
```
|
||||
|
||||
The files should be owned by (or readable by) the account the bot runs as. Read-only is enough.
|
||||
|
||||
## 3. Configure
|
||||
|
||||
Add one block to `/home/Forex/ForexBot2/config.json` (any position; `config.json.example` shows it).
|
||||
Leave the `deepseek` block as it is.
|
||||
|
||||
```json
|
||||
"news_model": {
|
||||
"path": "/home/Forex/models/xau_transformer",
|
||||
"threads": 2,
|
||||
"mode": "primary",
|
||||
"deepseek_audit_per_day": 0
|
||||
}
|
||||
```
|
||||
|
||||
(`threads` is clamped to the core count; the live box has 2 cores, hence 2. Measured there: load
|
||||
0.8 s, 16 ms per headline, 633 MB resident for the session.)
|
||||
|
||||
| Key | Meaning |
|
||||
|---|---|
|
||||
| `path` | folder with `model.onnx`, `tokenizer.json`, `meta.json`; required when `mode` is not `off` |
|
||||
| `threads` | ONNX threads per process. Measured: 1 thread 19 ms per headline, 4 threads 9 ms, 16 threads 13 ms. 4 is right for a 4–8 core box |
|
||||
| `mode` | `off` = today's DeepSeek/keyword path, nothing loads. `fallback` = DeepSeek first, the model only for headlines DeepSeek failed on. `primary` = the model scores everything, DeepSeek never sees a headline (except the audit sample) |
|
||||
| `deepseek_audit_per_day` | `primary` only: headlines per UTC day also sent to DeepSeek and archived as `llm` rows (one call of up to 25 per cycle at most). They are training labels for the next ML-NEWS retrain; the bot never displays them. 0 = no DeepSeek headline calls at all |
|
||||
|
||||
The environment variable `NEWS_MODEL_MODE=off|fallback|primary` overrides `mode` for one process. It
|
||||
exists so the dashboard daemon's systemd unit can pick its own mode.
|
||||
|
||||
A config error (unknown key, bad mode, missing path) stops the bot at boot with a clear message, like
|
||||
every other `config.json` mistake.
|
||||
|
||||
## 4. Smoke test without starting the bot
|
||||
|
||||
```bash
|
||||
cd /home/Forex/ForexBot2
|
||||
venv/bin/python -c "
|
||||
import news_model as m
|
||||
b = m.get_news_model()
|
||||
print(m.score_news(b, 'Gold jumps as dollar slides'))
|
||||
print(m.score_news(b, 'Red Lobster endless shrimp losses'))"
|
||||
```
|
||||
|
||||
Expect a log line `news model loaded from /home/Forex/models/xau_transformer (trained_at 2026-09-27…,
|
||||
threshold 0.2, 4 threads)`, then `label: bullish` for the first headline and `label: neutral` for the
|
||||
second. If instead you see `news model unavailable, DeepSeek/keyword path stays: …`, the message names
|
||||
the cause (missing file, missing package, bad JSON). The bot would still start in that state and
|
||||
would score with DeepSeek for the whole process lifetime; fix the cause and restart.
|
||||
|
||||
Then run the suite. With the model folder configured, the integration test loads the real bundle:
|
||||
|
||||
```bash
|
||||
venv/bin/python test_all.py # expect RESULTS: … passed, 0 failed
|
||||
```
|
||||
|
||||
## 5. Restart both processes
|
||||
|
||||
Bot: restart it the way it is run on this box (nohup / screen / its unit). Dashboard:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart gold-dashboard
|
||||
```
|
||||
|
||||
If the dashboard must not load the model (step 0), first uncomment the
|
||||
`Environment=NEWS_MODEL_MODE=off` line in `deploy/gold-dashboard.service`, then
|
||||
`sudo cp deploy/gold-dashboard.service /etc/systemd/system/ && sudo systemctl daemon-reload` before
|
||||
the restart.
|
||||
|
||||
## 6. What to check afterwards
|
||||
|
||||
- **Logs** (`logs/telegram.log`, `logs/dashboard.log`): exactly one `news model loaded` line per
|
||||
process, no `news model unavailable` line. If the level is DEBUG you also see
|
||||
`news archive: primary mode, 0 DeepSeek audit rows stored, N model/keyword rows not archived` every
|
||||
news cycle.
|
||||
- **`top`**: each process about 0.6 GB more resident memory than before; CPU near 0% between scans
|
||||
(thread spinning is disabled in the runtime).
|
||||
- **DeepSeek spend**: the daily request count no longer moves with the headline count; only the event
|
||||
direction, the summary and the panel use it.
|
||||
- **Archive frozen** (the intended state with audit 0):
|
||||
|
||||
```bash
|
||||
sqlite3 -readonly logs/newsdata.db "SELECT MAX(first_seen), COUNT(*) FROM headlines;"
|
||||
```
|
||||
|
||||
The newest `first_seen` stops advancing after the restart. With `deepseek_audit_per_day` > 0 you
|
||||
see at most that many new rows a day, all `score_source = 'llm'`.
|
||||
- **Telegram / dashboard**: headline scores still appear; the wording and layout are unchanged, only
|
||||
the numbers now come from the model. A headline the model calls irrelevant to gold shows as neutral.
|
||||
|
||||
## 7. Rollback
|
||||
|
||||
Set `"mode": "fallback"` (DeepSeek first, the model only fills gaps and the archive grows again) or
|
||||
`"mode": "off"` (exactly the pre-2026-09-27 behaviour) in `config.json`, or export
|
||||
`NEWS_MODEL_MODE=off` for one process, and restart. No data migration in either direction.
|
||||
|
||||
## 8. Retraining cadence
|
||||
|
||||
When ML-NEWS ships a new bundle, copy the three files over the old ones (step 2), check the new
|
||||
`trained_at`, and restart both processes. Rows written in `fallback` mode carry
|
||||
`teacher_model = xau_transformer@<trained_at>`, so the archive shows which bundle scored them.
|
||||
|
||||
With `deepseek_audit_per_day` at 0 the archive receives no new DeepSeek labels, so the next retrain
|
||||
needs one of: a positive audit value for a few weeks, a spell in `fallback` mode, or a separate
|
||||
labelling run in ML-NEWS on freshly crawled headlines (`eval/eval_archive.py --scorer transformer` on a
|
||||
copy of the live archive gives the model-versus-DeepSeek agreement on whatever labelled rows exist).
|
||||
|
||||
## Known limits
|
||||
|
||||
- The model copies DeepSeek's opinion; it has not learned from price moves. DeepSeek itself is only
|
||||
about 90% consistent with itself, and the model agrees with it about 72% of the time.
|
||||
- Gold only. In `primary` mode every other currency's headline sentiment is the keyword scorer (24 of
|
||||
78 on the golden set), which is what those currencies got whenever DeepSeek was unavailable before.
|
||||
- News never enters signals (CLAUDE.md section 7). No trade, grade or backtest book changes with any of
|
||||
this; the model only changes the headline lines people read.
|
||||
Reference in New Issue
Block a user