News-ML-Decoder/PROGRESS_2026-09-27.md

Session progress: 2026-09-27

Continues PROGRESS_2026-09-22.md. Full numbers in eval/RESULTS_2026-09-27.md.

DeepSeek calls this session: 560 (20 pilot + 200 full run + 140 precise top-up + 200 growth set; 15 failed), approved by the user in-session. 13,625 crawled headlines labelled; history labels now total 14,625.

Done, in order

1. Precise crawl resumed (crawl_google_news.py --query-set precise). It ran in the background the whole session and was still running at the end, at about 860 windows done today and 3,300 to go. Its log (logs/crawl_precise.log) is block-buffered and lags; use pgrep -f crawl_google_news and the crawl_log table.
2. Trained the transformer on the 3,620 labelled rows available before any DeepSeek call, swept 8 configs, exported, evaluated: golden 49/78, unseen days 65.7%. Failed the golden bar of 55. Written up as the "3,620 rows" column in the results file.
3. Pilot, 20 calls, precise pool only: 52% gold-relevant vs 7% for the broad pool. No hindsight bias worth worrying about. Log: logs/label_pilot2.log.
4. Full run, 200 calls, 30% broad share: 4,750 headlines. Log: logs/label_full_20260927.log.
5. Retrained on 8,870 rows, swept 5 configs (all within 1.2 val points of each other). Chosen on dev: lr 5e-5, 4 epochs, batch 64, best epoch 3.
   • Golden 62/78, distractors 8/8, Sep 18–22 agreement 71.9%, latency 19.0 ms. All four promotion criteria pass.
6. train.py --final on all 8,870 rows for 3 epochs, exported from torch_final. Shipped bundle models/xau_transformer/{model.onnx,tokenizer.json,meta.json}: golden 65/78, 0 decision flips, 18.9 ms.
7. Wrote eval/RESULTS_2026-09-27.md.
8. Round 2, at the user's request: 140 more calls labelled the remaining 3,400 precise headlines. Retrained on 12,270 rows (default config, no sweep). Non-final: golden 64/78, Sep 18–22 71.8%. Final on all rows, 4 epochs: golden 67/78, 18.9 ms. Re-exported; this is the shipped bundle now. The live-feed metric did not move (72.0 → 71.7%), so it has saturated on history labels.
9. Round 3, "teach it the two weak spots": added a growth query set to the crawler (20 intitle: queries: GDP, retail sales, ISM, PMI, jobless claims, consumer confidence/spending, manufacturing, "record high", "risk appetite", "Wall Street", "S&P 500", ceasefire, "trade deal", equities). Paused the precise crawl, crawled 1,880 windows (6,183 headlines), resumed the precise crawl. Added --query-set and a non-Latin filter to label_history.py. 200 calls labelled 4,975 growth headlines. Retrained on 17,245 rows: non-final golden 71/78, Sep 18–22 71.5%; final golden 72/78, strong_usd 7/7, risk_on 4/4, 18.8 ms. Re-exported; this is the shipped bundle now.

Where things are

Path	What
models/xau_transformer/	shipped ONNX bundle from round 3 torch_final (17,245 rows, golden 72/78)
models/xau_transformer/torch/	round 3 non-final weights (val numbers belong to these)
models/xau_transformer/torch_final/	round 3 final weights
session scratchpad sweep2/*_8870rows, sweep2/*_12270rows	round 1 and round 2 weights
models/xauusd_sentiment.joblib	TF-IDF, unchanged, still what ForexBot2 would load if it loaded anything
session scratchpad sweep/, sweep2/	all other sweep runs; may not survive a reboot
eval/results/golden_20260927_2315_transformer.*	round 3 final model golden eval (72/78)
eval/results/archive_20260927_2313_transformer.json	round 3 non-final archive eval
eval/results/golden_20260927_1949_*, archive_20260927_1947_*	round 2 equivalents
eval/results/golden_20260927_1931_transformer.*, archive_20260927_1927_*	round 1 equivalents

Known issues

• export_onnx.py prints PARITY FAILED on max |diff| (0.06–0.08 vs tolerance 0.05) even with 0 flips. Loosen the tolerance or check flips only.
• 14 of 340 label batches failed (DeepSeek timeouts / one 502); those headlines are simply unlabelled and will be picked up by the next run. About 570 precise headlines were unlabelled at the end, plus whatever the still-running crawl adds.
• 2025 labels have a mean xau of +0.12, higher than other years. Possible hindsight in DeepSeek's labels for the rally year.

Next

See "Next, revised" in the results file. The weak spots are fixed on the golden set. The step that now produces new information is shadow mode in ForexBot2; more history labels will not. Watch the gold-key live-feed number (83 → 79%) as history labels come to dominate the training set. The precise crawl is still running in the background and is free to leave alone.

Git

ML-NEWS is tracked in git since this session: ssh://git@192.168.0.11:222/harry/News-ML-Decoder.git, branch main. Ignored: both venvs, data/history.db, and the transformer weights (model.pt, *.onnx). The DeepSeek label cache and newsdata.db are tracked. Rebuild the weights with transformer/train.py then transformer/export_onnx.py.

ForexBot2 integration: specified, not built

docs/FOREXBOT2_INTEGRATION_SPEC.md is a full implementation spec for running the ONNX bundle inside ForexBot2 (vendored runtime module, news_model config section, news_engine call site and the gold-only scoring rules, archive provenance, tests, ops checklist for the bot and the dashboard daemon). Owner decisions on 2026-09-27: mode primary with no DeepSeek for headline scoring, model files outside the ForexBot2 tree. No ForexBot2 code was changed.