Local news model: ML-NEWS XAU transformer scores headlines (news_model.py, mode primary)
The owner's DeBERTa-v3-small two-head ONNX model (golden set 72/78, ~72% agreement with DeepSeek on unseen days) replaces DeepSeek for headline scoring. DeepSeek keeps event direction, the summary and the dashboard endpoint. - news_model.py: vendored ML-NEWS runtime (lazy onnxruntime/tokenizers, so the bot starts without them) + load-once, never-raising singleton. - news_engine: _enrich_headline_scores mode switch (off/fallback/primary), _apply_model_sentiment (gold head only: xau_score + XAU entry, never USD; not-relevant shows 0.0; never touches _llm_sent_cache), _audit_with_deepseek (archive-only labels, deepseek_audit_per_day). - Owner decision: primary mode writes nothing to logs/newsdata.db except audit copies — the model's output is not training data. - app_config: news_model section (path/threads/mode/deepseek_audit_per_day) + NEWS_MODEL_MODE env override; validated at boot. - news_archive: model rows carry teacher_model xau_transformer@<trained_at>. - deploy/README-news-model.md (ops for bot + dashboard daemon), unit gets a commented NEWS_MODEL_MODE=off line; requirements + onnxruntime, tokenizers. - test_all.py Module 38 (15 tests, one against the real bundle). Suite 581. - CLAUDE.md, spec status note, handoff marked implemented. Books unaffected: news never enters signals; no signal file touched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Harwinder Sekhon
@@ -23,7 +23,7 @@ cp config.json.example config.json # then fill i
|
||||
venv/bin/pip install cupy-cuda12x "nvidia-cuda-runtime-cu12==12.9.*" # optional, dev box: backtester GPU backend
|
||||
# (CuPy from pip needs that header wheel to compile kernels)
|
||||
venv/bin/python test_all.py # QA suite, no network; prints its own count
|
||||
# (566 tests / 37 modules as of 2026-09-27)
|
||||
# (581 tests / 38 modules as of 2026-09-27)
|
||||
venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail # one signal report in the terminal
|
||||
# --json full dump, --panel adds the advisory LLM panel,
|
||||
# --timeframe M5 | M15 | H1 | H4
|
||||
@@ -106,8 +106,9 @@ direction with the two confirmation timeframes. Agreement needs 2 of 3. Disagree
|
||||
penalty and blocks the top grades.
|
||||
|
||||
**6. News context (warning only).** `news_engine.py` reads the Forex Factory calendar (MyFXBook
|
||||
RSS as fallback) and four RSS feeds, scores headlines for gold, and can ask DeepSeek to improve
|
||||
the scores. This produces warnings and a text summary. **By design it never changes a signal.**
|
||||
RSS as fallback) and four RSS feeds and scores headlines for gold: the keyword scorer first, then
|
||||
the local XAU transformer (`news_model.py`, mode `primary`) or DeepSeek (modes `off` / `fallback`)
|
||||
overwrites those scores. This produces warnings and a text summary. **By design it never changes a signal.**
|
||||
The backtester runs with news off, so the live decision must be purely technical too.
|
||||
|
||||
**7. Penalties, then the signal and the grade.** When the timeframes agree, confidence gains 10
|
||||
@@ -270,11 +271,12 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; optional DeepSeek enrichment; display and warnings only |
|
||||
| `news_engine.py` | Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; headline scores upgraded by the local XAU model (`news_model.py`, mode `primary` live since 2026-09-27) or by DeepSeek (modes `off` / `fallback`, `_enrich_headline_scores`); DeepSeek keeps event direction and the summary; display and warnings only |
|
||||
| `llm_client.py` | Thin `requests` client for DeepSeek (`deepseek-v4-flash`, `thinking: disabled`, one attempt, UTC-day request cap). Every caller has a non-LLM fallback |
|
||||
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call |
|
||||
| `news_archive.py` | Durable headline corpus in `logs/newsdata.db` (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a `signals.db` rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and `logs/` is gitignored. Hooked at the end of `news_engine._fetch_news_sentiment_raw`, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: `xau_score` holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (`currencies_json` `[]`; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call. **Frozen in `news_model.mode` `primary` (live since 2026-09-27)**: the local model's output is not training data, so nothing is archived except the optional DeepSeek audit copies (`deepseek_audit_per_day`, shipped 0); `off` / `fallback` archive as before, a `fallback` model row carrying `teacher_model` `xau_transformer@<trained_at>` |
|
||||
| `agent_panel.py` | Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through `agent_panel.personas` for research). Attached post-hoc. v2 (2026-09-17, `PANEL_PROMPT_VERSION`): every persona sees a code-computed `context` block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from `agent_panel_base_rates.json`, event timing against the expected hold) and returns `p_win` plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, `verdict_source: "mean"`, default). `macro` (stance verbatim) and `chair` (synthesizer + `_apply_oppose_guard`) remain for comparison. The prompt version is logged and hashed. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half |
|
||||
| `violent_watch.py` | Display-only violent-move warning and heads-up. Attached post-hoc |
|
||||
| `news_model.py` | The local headline scorer (2026-09-27): vendored CPU runtime of the owner's ML-NEWS XAU transformer (DeBERTa-v3-small, two heads: moves gold? / up or down?, ONNX; `onnxruntime` + `tokenizers` imported lazily inside `load_model`, so the bot starts without them) plus a load-once, never-raising singleton (`get_news_model`, `teacher_tag`, `warm_up_in_background`). Gold head only: `news_engine._apply_model_sentiment` writes `xau_score` and XAU's per-currency entry, never USD. Bundle (280 MB) lives outside the tree at `news_model.path`. Spec and deviations: `docs/news_model_integration_spec_2026-09-27.md`; ops: `deploy/README-news-model.md` |
|
||||
|
||||
### Logging and evaluation
|
||||
|
||||
@@ -311,7 +313,7 @@ Arrows show who imports whom (generated from the code on 2026-09-15).
|
||||
| `app_config.py` | Loads `config.json` once per process; env var → config.json → refuse; validates keys |
|
||||
| `config.json.example` | The schema of `config.json` (which is gitignored) |
|
||||
| `deploy/README-dashboard.md`, `deploy/gold-dashboard.service` | Live-server checklist and systemd unit for the dashboard daemon |
|
||||
| `test_all.py` | The QA suite (36 modules; Module 36 is the GPU backend parity module, Module 37 the CUDA decision-stage parity module) |
|
||||
| `test_all.py` | The QA suite (38 modules; Module 36 is the GPU backend parity module, Module 37 the CUDA decision-stage parity module, Module 38 the local news model) |
|
||||
|
||||
---
|
||||
|
||||
@@ -374,6 +376,7 @@ gates live in `config.py` and the pair factory, never in `config.json`. Blocks (
|
||||
| `violent_watch` | `enabled`, `alert_line`, `headsup`, `max_headsup_per_day`, `headsup_min_level` (env `VIOLENT_WATCH_ENABLED`). Off by default |
|
||||
| `signal_log` | `enabled`, `path`, `log_blocked_scans` (env `SIGNAL_LOG_ENABLED`, `SIGNAL_LOG_PATH`, `SIGNAL_LOG_BLOCKED_SCANS`) |
|
||||
| `news_archive` | `enabled` (default true), `path` (env `NEWS_ARCHIVE_ENABLED`, `NEWS_ARCHIVE_PATH`) |
|
||||
| `news_model` | `path` (bundle directory, required when mode ≠ off), `threads` (default 4), `mode` (`off` default = DeepSeek/keyword path; `fallback` = model only where DeepSeek failed; `primary` = model scores every headline, no DeepSeek headline calls), `deepseek_audit_per_day` (default 0; primary only, archive-only DeepSeek labels). Env `NEWS_MODEL_MODE` overrides `mode` per process (the dashboard unit). Live: `primary` on both processes |
|
||||
| `log_level` | Logging level |
|
||||
|
||||
A legacy top-level `market_outlook` block is tolerated and ignored with one warning (the feature
|
||||
@@ -389,7 +392,9 @@ day, 30 kept), `logs/dashboard.log`. Read the databases with `mode=ro`, never `i
|
||||
## 6. Testing
|
||||
|
||||
- `venv/bin/python test_all.py` runs everything with mocked network. Expect
|
||||
`RESULTS: 566 passed, 0 failed, 566 total` (2026-09-27). On a box with a visible CUDA device
|
||||
`RESULTS: 581 passed, 0 failed, 581 total` (2026-09-27). Module 38's last test loads the
|
||||
real news-model bundle when `NEWS_MODEL_DIR` (or the configured `news_model.path`) exists and
|
||||
prints SKIP otherwise. On a box with a visible CUDA device
|
||||
Module 36 fails if the GPU backend falls back (the reason is in the message); with
|
||||
`FB2_BACKEND=cpu` or no CUDA the GPU-path tests print a notice and the NumPy twins are tested.
|
||||
Module 37 (the CUDA decision stage) adds about 85 s on the GPU because its references replay
|
||||
@@ -568,6 +573,13 @@ MD5s. Scorecards from it are in-book statistics, never decisions.
|
||||
Interior gaps in history are logged, never fatal, for the same reason.
|
||||
- Existing databases still contain an empty-going-forward `outlooks` table from the removed
|
||||
market-outlook feature. Nothing reads it.
|
||||
- The local news model has a gold head only. `_apply_model_sentiment` may write `xau_score`, XAU's
|
||||
per-currency entry and, for a headline naming no other currency, the overall score; USD and every
|
||||
other currency keep the keyword value (the pair sentiment is XAU − USD). It never touches
|
||||
`_llm_sent_cache`, and a not-relevant headline displays 0.0 while `xau_score` keeps the raw
|
||||
number. In `primary` mode nothing is archived but the DeepSeek audit copies, and the dashboard
|
||||
process never archives in any mode (its DB writes are off), so a retrain needs
|
||||
`deepseek_audit_per_day > 0`, a spell in `fallback`, or a labelling run in ML-NEWS.
|
||||
- The news score is a trailing echo of the last day's price move (r ≈ 0.67 with the past day,
|
||||
≈ 0.03 with the next). Both news gating flags stay off.
|
||||
- The level-breach rescan and the candle loop both skip the settlement-pause window
|
||||
@@ -632,6 +644,7 @@ One line per validated change or verdict, newest first. Full evidence in the nam
|
||||
|
||||
| Date | Decision | Where the evidence is |
|
||||
|---|---|---|
|
||||
| 2026-09-27 | Local news scorer, round two: the owner's ML-NEWS model is now a DeBERTa-v3-small two-head transformer (gold-relevant? then bullish/bearish), distilled from 17,245 DeepSeek gold labels (Sep 8–22 archive + a 2019–2026 Reuters/CNBC/MarketWatch crawl labelled by DeepSeek, 560 calls). Golden set 72/78 (DeepSeek 78, keywords 24, the 09-17 TF-IDF 39), agreement with DeepSeek on unseen live days ~72% (ceiling 90%), 19 ms/headline on one CPU thread as int8-embedding ONNX; passed the four promotion criteria fixed on 09-22. OWNER DECISIONS: it is to score headlines here in mode `primary` with NO DeepSeek for headline scoring (DeepSeek stays for event direction, summaries, the dashboard endpoint); model files live OUTSIDE this tree at a configured path (280 MB, never in git); shadow mode declined (the archive already allows offline replay). IMPLEMENTED the same day (branch `sql-integration`): `news_model.py` (vendored runtime + never-raising singleton, lazy `onnxruntime` / `tokenizers`), `news_model` config section + `NEWS_MODEL_MODE` env, `news_engine._enrich_headline_scores` / `_apply_model_sentiment` (gold-only rules A–F) / `_audit_with_deepseek`, archive provenance for `model` rows, `deploy/README-news-model.md`, Module 38 (15 tests, one against the real bundle: bullish / neutral labels, 7 ms per headline on the dev box). Two further owner decisions during implementation: **`primary` mode stops writing `newsdata.db`** (only the DeepSeek audit sample, shipped 0, is archived) and **both processes load the model**. Books unaffected (news never enters signals); no signal file touched, no parity event. ML-NEWS `news_transformer.py` received the same two `load_model` edits (commit `9496cd9` there) | `docs/news_model_integration_spec_2026-09-27.md`; `~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md` |
|
||||
| 2026-09-27 | Single "conviction" number vs the two-number grade (owner question: one number that is easier to understand and to decide on). Pre-registered; switch `signals.grading_mode` (`pair` default = the shipped rule, byte-identical on both scripts, all five pins reproduced; `min` = min(\|score\|, 0.6 × conf); `product` = \|score\| × conf / 100; thresholds from the control's corner points, no tuning; mirrored in K2, twin MD5-equal to the reference on both new paths). FAILED on M5 and M15: K-MIN moves 43–54 trades for −1.2R / −2.9R (a, b, e / a, b, d); K-PROD −29R on M15 with PF 1.29 and W3 DD 28.9R, neutral on M5. H1 / H4 K-PROD +11.9R / +17.0R on W3 by taking 40–100 extra low-score trades (H1 PF 1.98 → 1.75, DD doubles): a hypothesis, not evidence. Nothing ships; the knob stays research-only. Code review (high) before the runs: 6 findings fixed, 1 recorded. SHIPPED the display half of the question (owner request the same day): alerts lead with the grade and print score / confidence as words (`main.plain_words`, same line on Telegram, `/status`, CLI and the dashboard card); books untouched. Suite 566 | `backtest/research/conviction_campaign_2026-09-27.md` |
|
||||
| 2026-09-26 | Quant review + live log 2026-09-04 → 09-25 (100 setups replayed on the scans' stored candles): stop-hunt claim REFUTED (71% of stop-outs ran ≥ 1R past the stop, same as the books); −25.1R of the −37.8R raw live book were consecutive-bar re-entries the backtester's cooldown never takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week windows. SHIPPED: 5-bar alert cooldown; alert wording (trail first as the tested exit, TP1 a guide with its R, planned RR, validated-book line per class); M15 grade-A SELL block (`sell_min_quality` `A+, B`, PASSED a–f: W2 +154.15R/813 PF 1.53 vs +146.14/931 PF 1.43, OOS +68.5 vs +60.67, W3 DD +8%); 1R printed-target floor in every generator after the gates (`tp_print_floor_rr`, owner rule "TP1 at least 1:1"; label-only re-pin of all four timeframes). FAILED: M15 `min_rr_ratio` 1.0 (trade-identical, −2.0R); a pre-gate target floor (−24.6R). OWNER'S CALL: M5 `tp_floor_rr` 1.0 adds +48.9R on W2 with both halves up but max DD 10.1 → 15.9R (ladder 0.6–1.0 non-monotone); not shipped. Telegram / dashboard AI block rewritten for a new trader (owner rule): one plain headline against the usual odds, the chance "in 100", one reason for and one against, no verdict word or confidence figure; persona prompts ban trading jargon (`PANEL_PROMPT_VERSION` v2.1). Two delivery bugs from the DB checks fixed: the macro-source panel verdict never stored its probability (all v2 rows NULL); the level-breach rescan re-logged the 20:45 M15 bar three times every evening after the settlement pause (guard added to `_handle_breach`). Indicator grid and volume / daily-direction screens (Part C) not run | `backtest/research/rr_and_sell_campaign_2026-09-26.md` |
|
||||
| 2026-09-18 | Full-GPU research twin SHIPPED: `backtest/cudaBasedBacktesting.py` + `backtest/cuda_signal_kernels.py` (owner decisions: research only, `gpu_backtest.py` stays the base reference, every research flag supported, the one exemption from instructions item 6 recorded there). Proof: the five reference MD5s, four 6-month acceptance books (2026-03-18 → 2026-09-18, M5 and M15, $0.65 and $0.30), the M15 OOS `--export-features` CSV and `--record --record-bars` tables, and ten research-flag variants, all byte-identical to the reference script; Module 37 (16 tests) pins every stage against the live code. Books take 0.8–2.4 s instead of 13–115 s. Rule added: any signal change is a parity event for the twin | `backtest/research/cuda_backtester_2026-09-18.md` |
|
||||
|
||||
Reference in New Issue
Block a user