harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)

Browse Source 
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72%
agreement with DeepSeek on unseen days) replaces DeepSeek for headline
scoring. DeepSeek keeps event direction, the summary and the dashboard
endpoint.

- news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers,
  so the bot starts without them) + load-once, never-raising singleton.
- news_engine: _enrich_headline_scores mode switch (off/fallback/primary),
  _apply_model_sentiment (gold head only: xau_score + XAU entry, never
  USD; not-relevant shows 0.0; never touches _llm_sent_cache),
  _audit_with_deepseek (archive-only labels, deepseek_audit_per_day).
- Owner decision: primary mode writes nothing to logs/newsdata.db except
  audit copies — the model's output is not training data.
- app_config: news_model section (path/threads/mode/deepseek_audit_per_day)
  + NEWS_MODEL_MODE env override; validated at boot.
- news_archive: model rows carry teacher_model xau_transformer@<trained_at>.
- deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a
  commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers.
- test_all.py Module 38 (15 tests, one against the real bundle). Suite 581.
- CLAUDE.md, spec status note, handoff marked implemented.

Books unaffected: news never enters signals; no signal file touched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
2026-09-27 21:02:42 -04:00
parent 6af1237c2d
commit 0c56b4e155
13 changed files with 1966 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deploy/README-dashboard.md
+3 -1
View File
@@ -19,7 +19,9 @@ git pull
venv/bin/python test_all.py # optional sanity: suite must be green
```
No new Python dependencies are required.
Two optional Python packages (`onnxruntime`, `tokenizers`) arrive with the local news model;
`venv/bin/pip install -r requirements.txt` after the pull. See `deploy/README-news-model.md`,
which also says when to keep this daemon on DeepSeek (`NEWS_MODEL_MODE=off` in the unit).
## 2. Make the web folder writable
deploy/README-news-model.md
+185
View File
@@ -0,0 +1,185 @@
# Deploying the local news model (live server)
Since 2026-09-27 ForexBot2 can score news headlines with the owner's ML-NEWS gold transformer
instead of DeepSeek. The model runs on the CPU inside the bot process (and inside the dashboard
daemon), needs two extra Python packages and a 280 MB model folder that lives **outside** the git
tree. DeepSeek keeps its other jobs: the direction of a calendar surprise, the news summary and the
dashboard's on-demand summary endpoint.
What the model is, in one paragraph: a DeBERTa-v3-small transformer with two heads (does this
headline move gold? if so, up or down?) exported to ONNX, trained to copy DeepSeek's gold score on
17,245 headlines. On the 78-headline golden set it scores 72 (DeepSeek 78, the old keyword scorer
24); on unseen live days it agrees with DeepSeek about 72% of the time (DeepSeek agrees with itself
about 90%). It scores one headline in about 10–20 ms on one CPU thread. It is **gold only**: every
other currency's headline sentiment stays with the keyword scorer in `primary` mode.
Owner decisions this deployment implements:
- Mode `primary`: the model scores every headline; **no DeepSeek calls for headline scoring**.
- **The news archive stops growing.** In `primary` mode the model's own output is not archived (it is
not training data). Only the optional DeepSeek audit sample (`deepseek_audit_per_day`, shipped 0)
writes rows. `logs/newsdata.db` stays as it is; nothing is migrated or deleted.
- Both processes (bot and dashboard daemon) load the model so Telegram and the dashboard show the same
headline scores. Each ONNX session holds about 0.6 GB of RAM.
## 0. Check memory
```bash
free -m
```
Each process that loads the model needs about 0.7 GB of headroom; two processes about 1.5 GB. If the
box cannot spare that, keep the model on the bot only: uncomment the `Environment=NEWS_MODEL_MODE=off`
line in `deploy/gold-dashboard.service` (step 5). The dashboard then keeps scoring its headlines with
DeepSeek and its scores may differ from the bot's. The dashboard process never writes the archive in
any mode, so this choice does not affect training data.
## 1. Code and dependencies
```bash
cd /home/Forex/ForexBot2
git pull
venv/bin/pip install -r requirements.txt # adds onnxruntime and tokenizers
```
`requirements.txt` must never contain the training stack (no torch, no transformers); the test suite
checks that. The live box runs Python 3.9, where pip resolves onnxruntime 1.19.2 and tokenizers
0.22.2; the bundle of 2026-09-27 scores identically there and on the dev box (1.30.0 / 0.23.2).
## 2. Copy the model files (outside git)
On the machine that holds ML-NEWS (`~/Development/ML-NEWS/models/xau_transformer/`), copy exactly
three files. Skip the `torch/` and `torch_final/` subfolders; they are training checkpoints.
```bash
ssh Forex@LIVE 'mkdir -p /home/Forex/models/xau_transformer'
rsync -av --progress \
~/Development/ML-NEWS/models/xau_transformer/{model.onnx,tokenizer.json,meta.json} \
Forex@LIVE:/home/Forex/models/xau_transformer/
```
Verify on the live box:
```bash
sha256sum /home/Forex/models/xau_transformer/model.onnx
# bundle of 2026-09-27: 9e7a64d3ae1df335059ce829cafcf6e4544b85a4f72cc88923c2e42c48e975a0
sha256sum /home/Forex/models/xau_transformer/tokenizer.json
# bundle of 2026-09-27: 045f93f80eb9ad18e559647563fb827dea59d93258fdb8b922de82bc53666d16
python3 -c "import json; print(json.load(open('/home/Forex/models/xau_transformer/meta.json'))['trained_at'])"
# 2026-09-27T23:15:30.403442+00:00
```
The files should be owned by (or readable by) the account the bot runs as. Read-only is enough.
## 3. Configure
Add one block to `/home/Forex/ForexBot2/config.json` (any position; `config.json.example` shows it).
Leave the `deepseek` block as it is.
```json
"news_model": {
"path": "/home/Forex/models/xau_transformer",
"threads": 2,
"mode": "primary",
"deepseek_audit_per_day": 0
}
```
(`threads` is clamped to the core count; the live box has 2 cores, hence 2. Measured there: load
0.8 s, 16 ms per headline, 633 MB resident for the session.)
| Key | Meaning |
|---|---|
| `path` | folder with `model.onnx`, `tokenizer.json`, `meta.json`; required when `mode` is not `off` |
| `threads` | ONNX threads per process. Measured: 1 thread 19 ms per headline, 4 threads 9 ms, 16 threads 13 ms. 4 is right for a 4–8 core box |
| `mode` | `off` = today's DeepSeek/keyword path, nothing loads. `fallback` = DeepSeek first, the model only for headlines DeepSeek failed on. `primary` = the model scores everything, DeepSeek never sees a headline (except the audit sample) |
| `deepseek_audit_per_day` | `primary` only: headlines per UTC day also sent to DeepSeek and archived as `llm` rows (one call of up to 25 per cycle at most). They are training labels for the next ML-NEWS retrain; the bot never displays them. 0 = no DeepSeek headline calls at all |
The environment variable `NEWS_MODEL_MODE=off|fallback|primary` overrides `mode` for one process. It
exists so the dashboard daemon's systemd unit can pick its own mode.
A config error (unknown key, bad mode, missing path) stops the bot at boot with a clear message, like
every other `config.json` mistake.
## 4. Smoke test without starting the bot
```bash
cd /home/Forex/ForexBot2
venv/bin/python -c "
import news_model as m
b = m.get_news_model()
print(m.score_news(b, 'Gold jumps as dollar slides'))
print(m.score_news(b, 'Red Lobster endless shrimp losses'))"
```
Expect a log line `news model loaded from /home/Forex/models/xau_transformer (trained_at 2026-09-27…,
threshold 0.2, 4 threads)`, then `label: bullish` for the first headline and `label: neutral` for the
second. If instead you see `news model unavailable, DeepSeek/keyword path stays: …`, the message names
the cause (missing file, missing package, bad JSON). The bot would still start in that state and
would score with DeepSeek for the whole process lifetime; fix the cause and restart.
Then run the suite. With the model folder configured, the integration test loads the real bundle:
```bash
venv/bin/python test_all.py # expect RESULTS: … passed, 0 failed
```
## 5. Restart both processes
Bot: restart it the way it is run on this box (nohup / screen / its unit). Dashboard:
```bash
sudo systemctl restart gold-dashboard
```
If the dashboard must not load the model (step 0), first uncomment the
`Environment=NEWS_MODEL_MODE=off` line in `deploy/gold-dashboard.service`, then
`sudo cp deploy/gold-dashboard.service /etc/systemd/system/ && sudo systemctl daemon-reload` before
the restart.
## 6. What to check afterwards
- **Logs** (`logs/telegram.log`, `logs/dashboard.log`): exactly one `news model loaded` line per
process, no `news model unavailable` line. If the level is DEBUG you also see
`news archive: primary mode, 0 DeepSeek audit rows stored, N model/keyword rows not archived` every
news cycle.
- **`top`**: each process about 0.6 GB more resident memory than before; CPU near 0% between scans
(thread spinning is disabled in the runtime).
- **DeepSeek spend**: the daily request count no longer moves with the headline count; only the event
direction, the summary and the panel use it.
- **Archive frozen** (the intended state with audit 0):
```bash
sqlite3 -readonly logs/newsdata.db "SELECT MAX(first_seen), COUNT(*) FROM headlines;"
```
The newest `first_seen` stops advancing after the restart. With `deepseek_audit_per_day` > 0 you
see at most that many new rows a day, all `score_source = 'llm'`.
- **Telegram / dashboard**: headline scores still appear; the wording and layout are unchanged, only
the numbers now come from the model. A headline the model calls irrelevant to gold shows as neutral.
## 7. Rollback
Set `"mode": "fallback"` (DeepSeek first, the model only fills gaps and the archive grows again) or
`"mode": "off"` (exactly the pre-2026-09-27 behaviour) in `config.json`, or export
`NEWS_MODEL_MODE=off` for one process, and restart. No data migration in either direction.
## 8. Retraining cadence
When ML-NEWS ships a new bundle, copy the three files over the old ones (step 2), check the new
`trained_at`, and restart both processes. Rows written in `fallback` mode carry
`teacher_model = xau_transformer@<trained_at>`, so the archive shows which bundle scored them.
With `deepseek_audit_per_day` at 0 the archive receives no new DeepSeek labels, so the next retrain
needs one of: a positive audit value for a few weeks, a spell in `fallback` mode, or a separate
labelling run in ML-NEWS on freshly crawled headlines (`eval/eval_archive.py --scorer transformer` on a
copy of the live archive gives the model-versus-DeepSeek agreement on whatever labelled rows exist).
## Known limits
- The model copies DeepSeek's opinion; it has not learned from price moves. DeepSeek itself is only
about 90% consistent with itself, and the model agrees with it about 72% of the time.
- Gold only. In `primary` mode every other currency's headline sentiment is the keyword scorer (24 of
78 on the golden set), which is what those currencies got whenever DeepSeek was unavailable before.
- News never enters signals (CLAUDE.md section 7). No trade, grade or backtest book changes with any of
this; the model only changes the headline lines people read.
deploy/gold-dashboard.service
+4
View File
@@ -15,6 +15,10 @@ Type=simple
User=Forex
WorkingDirectory=/home/Forex/ForexBot2
Environment=DASHBOARD_OUT_DIR=/var/www/trade
# The local news model (deploy/README-news-model.md) loads in this process
# too (~0.6 GB RSS) so the dashboard shows the bot's headline scores.
# Uncomment to keep this daemon on DeepSeek when memory is short:
#Environment=NEWS_MODEL_MODE=off
ExecStart=/home/Forex/ForexBot2/venv/bin/python dashboard_publisher.py
Restart=always
RestartSec=30