ForexBot2/deploy/README-news-model.md

Deploying the local news model (live server)

Since 2026-09-27 ForexBot2 can score news headlines with the owner's ML-NEWS gold transformer instead of DeepSeek. The model runs on the CPU inside the bot process (and inside the dashboard daemon), needs two extra Python packages and a 280 MB model folder that lives outside the git tree. DeepSeek keeps its other jobs: the direction of a calendar surprise, the news summary and the dashboard's on-demand summary endpoint.

What the model is, in one paragraph: a DeBERTa-v3-small transformer with two heads (does this headline move gold? if so, up or down?) exported to ONNX, trained to copy DeepSeek's gold score on 17,245 headlines. On the 78-headline golden set it scores 72 (DeepSeek 78, the old keyword scorer 24); on unseen live days it agrees with DeepSeek about 72% of the time (DeepSeek agrees with itself about 90%). It scores one headline in about 10–20 ms on one CPU thread. It is gold only: every other currency's headline sentiment stays with the keyword scorer in primary mode.

Owner decisions this deployment implements:

• Mode primary: the model scores every headline; no DeepSeek calls for headline scoring.
• The news archive stops growing. In primary mode the model's own output is not archived (it is not training data). Only the optional DeepSeek audit sample (deepseek_audit_per_day, shipped 0) writes rows. logs/newsdata.db stays as it is; nothing is migrated or deleted.
• Both processes (bot and dashboard daemon) load the model so Telegram and the dashboard show the same headline scores. Each ONNX session holds about 0.6 GB of RAM.

0. Check memory

free -m

Each process that loads the model needs about 0.7 GB of headroom; two processes about 1.5 GB. If the box cannot spare that, keep the model on the bot only: uncomment the Environment=NEWS_MODEL_MODE=off line in deploy/gold-dashboard.service (step 5). The dashboard then keeps scoring its headlines with DeepSeek and its scores may differ from the bot's. The dashboard process never writes the archive in any mode, so this choice does not affect training data.

1. Code and dependencies

cd /home/Forex/ForexBot2
git pull
venv/bin/pip install -r requirements.txt      # adds onnxruntime and tokenizers

requirements.txt must never contain the training stack (no torch, no transformers); the test suite checks that. The live box runs Python 3.9, where pip resolves onnxruntime 1.19.2 and tokenizers 0.22.2; the bundle of 2026-09-27 scores identically there and on the dev box (1.30.0 / 0.23.2).

2. Copy the model files (outside git)

On the machine that holds ML-NEWS (~/Development/ML-NEWS/models/xau_transformer/), copy exactly three files. Skip the torch/ and torch_final/ subfolders; they are training checkpoints.

ssh Forex@LIVE 'mkdir -p /home/Forex/models/xau_transformer'
rsync -av --progress \
    ~/Development/ML-NEWS/models/xau_transformer/{model.onnx,tokenizer.json,meta.json} \
    Forex@LIVE:/home/Forex/models/xau_transformer/

Verify on the live box:

sha256sum /home/Forex/models/xau_transformer/model.onnx
# bundle of 2026-09-27: 9e7a64d3ae1df335059ce829cafcf6e4544b85a4f72cc88923c2e42c48e975a0
sha256sum /home/Forex/models/xau_transformer/tokenizer.json
# bundle of 2026-09-27: 045f93f80eb9ad18e559647563fb827dea59d93258fdb8b922de82bc53666d16
python3 -c "import json; print(json.load(open('/home/Forex/models/xau_transformer/meta.json'))['trained_at'])"
# 2026-09-27T23:15:30.403442+00:00

The files should be owned by (or readable by) the account the bot runs as. Read-only is enough.

3. Configure

Add one block to /home/Forex/ForexBot2/config.json (any position; config.json.example shows it). Leave the deepseek block as it is.

"news_model": {
  "path": "/home/Forex/models/xau_transformer",
  "threads": 2,
  "mode": "primary",
  "deepseek_audit_per_day": 0
}

(threads is clamped to the core count; the live box has 2 cores, hence 2. Measured there: load 0.8 s, 16 ms per headline, 633 MB resident for the session.)

Key	Meaning
path	folder with model.onnx, tokenizer.json, meta.json; required when mode is not off
threads	ONNX threads per process. Measured: 1 thread 19 ms per headline, 4 threads 9 ms, 16 threads 13 ms. 4 is right for a 4–8 core box
mode	off = today's DeepSeek/keyword path, nothing loads. fallback = DeepSeek first, the model only for headlines DeepSeek failed on. primary = the model scores everything, DeepSeek never sees a headline (except the audit sample)
deepseek_audit_per_day	primary only: headlines per UTC day also sent to DeepSeek and archived as llm rows (one call of up to 25 per cycle at most). They are training labels for the next ML-NEWS retrain; the bot never displays them. 0 = no DeepSeek headline calls at all

The environment variable NEWS_MODEL_MODE=off|fallback|primary overrides mode for one process. It exists so the dashboard daemon's systemd unit can pick its own mode.

A config error (unknown key, bad mode, missing path) stops the bot at boot with a clear message, like every other config.json mistake.

4. Smoke test without starting the bot

cd /home/Forex/ForexBot2
venv/bin/python -c "
import news_model as m
b = m.get_news_model()
print(m.score_news(b, 'Gold jumps as dollar slides'))
print(m.score_news(b, 'Red Lobster endless shrimp losses'))"

Expect a log line news model loaded from /home/Forex/models/xau_transformer (trained_at 2026-09-27…, threshold 0.2, 4 threads), then label: bullish for the first headline and label: neutral for the second. If instead you see news model unavailable, DeepSeek/keyword path stays: …, the message names the cause (missing file, missing package, bad JSON). The bot would still start in that state and would score with DeepSeek for the whole process lifetime; fix the cause and restart.

Then run the suite. With the model folder configured, the integration test loads the real bundle:

venv/bin/python test_all.py        # expect RESULTS: … passed, 0 failed

5. Restart both processes

Bot: restart it the way it is run on this box (nohup / screen / its unit). Dashboard:

sudo systemctl restart gold-dashboard

If the dashboard must not load the model (step 0), first uncomment the Environment=NEWS_MODEL_MODE=off line in deploy/gold-dashboard.service, then sudo cp deploy/gold-dashboard.service /etc/systemd/system/ && sudo systemctl daemon-reload before the restart.

6. What to check afterwards

• Logs (logs/telegram.log, logs/dashboard.log): exactly one news model loaded line per process, no news model unavailable line. If the level is DEBUG you also see news archive: primary mode, 0 DeepSeek audit rows stored, N model/keyword rows not archived every news cycle.

• top: each process about 0.6 GB more resident memory than before; CPU near 0% between scans (thread spinning is disabled in the runtime).

• DeepSeek spend: the daily request count no longer moves with the headline count; only the event direction, the summary and the panel use it.

• Archive frozen (the intended state with audit 0):

  sqlite3 -readonly logs/newsdata.db "SELECT MAX(first_seen), COUNT(*) FROM headlines;"
  

  The newest first_seen stops advancing after the restart. With deepseek_audit_per_day > 0 you see at most that many new rows a day, all score_source = 'llm'.

• Telegram / dashboard: headline scores still appear; the wording and layout are unchanged, only the numbers now come from the model. A headline the model calls irrelevant to gold shows as neutral.

7. Rollback

Set "mode": "fallback" (DeepSeek first, the model only fills gaps and the archive grows again) or "mode": "off" (exactly the pre-2026-09-27 behaviour) in config.json, or export NEWS_MODEL_MODE=off for one process, and restart. No data migration in either direction.

8. Retraining cadence

When ML-NEWS ships a new bundle, copy the three files over the old ones (step 2), check the new trained_at, and restart both processes. Rows written in fallback mode carry teacher_model = xau_transformer@<trained_at>, so the archive shows which bundle scored them.

With deepseek_audit_per_day at 0 the archive receives no new DeepSeek labels, so the next retrain needs one of: a positive audit value for a few weeks, a spell in fallback mode, or a separate labelling run in ML-NEWS on freshly crawled headlines (eval/eval_archive.py --scorer transformer on a copy of the live archive gives the model-versus-DeepSeek agreement on whatever labelled rows exist).

Known limits

• The model copies DeepSeek's opinion; it has not learned from price moves. DeepSeek itself is only about 90% consistent with itself, and the model agrees with it about 72% of the time.
• Gold only. In primary mode every other currency's headline sentiment is the keyword scorer (24 of 78 on the golden set), which is what those currencies got whenever DeepSeek was unavailable before.
• News never enters signals (CLAUDE.md section 7). No trade, grade or backtest book changes with any of this; the model only changes the headline lines people read.