harry/News-ML-Decoder
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
01baef48ff51cfbfeb454453cfef1df8c2f1ef33
News-ML-Decoder/PROGRESS_2026-09-27.md
T
Harwinder Sekhon 01baef48ff Initial import: XAUUSD news sentiment models, evals, and data tooling 
TF-IDF baseline (news_sentiment.py, models/xauusd_sentiment.joblib) and the
two-head DeBERTa-v3-small transformer (transformer/, news_transformer.py)
with its Google News crawler and DeepSeek labeller (data_collect/), the
golden-set and archive evals (eval/), and the 2026-09-22 and 2026-09-27
results and progress notes.

Transformer weights (model.pt, model.onnx) and the crawl database are not
tracked; rebuild with transformer/train.py and transformer/export_onnx.py.
The DeepSeek label cache in data/ is tracked.
2026-09-27 19:22:30 -04:00

4.6 KiB
Raw Blame History

Session progress: 2026-09-27

Continues PROGRESS_2026-09-22.md. Full numbers in eval/RESULTS_2026-09-27.md.

DeepSeek calls this session: 560 (20 pilot + 200 full run + 140 precise top-up + 200 growth set; 15 failed), approved by the user in-session. 13,625 crawled headlines labelled; history labels now total 14,625.

Done, in order

  1. Precise crawl resumed (crawl_google_news.py --query-set precise). It ran in the background the whole session and was still running at the end, at about 860 windows done today and 3,300 to go. Its log (logs/crawl_precise.log) is block-buffered and lags; use pgrep -f crawl_google_news and the crawl_log table.
  2. Trained the transformer on the 3,620 labelled rows available before any DeepSeek call, swept 8 configs, exported, evaluated: golden 49/78, unseen days 65.7%. Failed the golden bar of 55. Written up as the "3,620 rows" column in the results file.
  3. Pilot, 20 calls, precise pool only: 52% gold-relevant vs 7% for the broad pool. No hindsight bias worth worrying about. Log: logs/label_pilot2.log.
  4. Full run, 200 calls, 30% broad share: 4,750 headlines. Log: logs/label_full_20260927.log.
  5. Retrained on 8,870 rows, swept 5 configs (all within 1.2 val points of each other). Chosen on dev: lr 5e-5, 4 epochs, batch 64, best epoch 3.
    • Golden 62/78, distractors 8/8, Sep 18–22 agreement 71.9%, latency 19.0 ms. All four promotion criteria pass.
  6. train.py --final on all 8,870 rows for 3 epochs, exported from torch_final. Shipped bundle models/xau_transformer/{model.onnx,tokenizer.json,meta.json}: golden 65/78, 0 decision flips, 18.9 ms.
  7. Wrote eval/RESULTS_2026-09-27.md.
  8. Round 2, at the user's request: 140 more calls labelled the remaining 3,400 precise headlines. Retrained on 12,270 rows (default config, no sweep). Non-final: golden 64/78, Sep 18–22 71.8%. Final on all rows, 4 epochs: golden 67/78, 18.9 ms. Re-exported; this is the shipped bundle now. The live-feed metric did not move (72.0 → 71.7%), so it has saturated on history labels.
  9. Round 3, "teach it the two weak spots": added a growth query set to the crawler (20 intitle: queries: GDP, retail sales, ISM, PMI, jobless claims, consumer confidence/spending, manufacturing, "record high", "risk appetite", "Wall Street", "S&P 500", ceasefire, "trade deal", equities). Paused the precise crawl, crawled 1,880 windows (6,183 headlines), resumed the precise crawl. Added --query-set and a non-Latin filter to label_history.py. 200 calls labelled 4,975 growth headlines. Retrained on 17,245 rows: non-final golden 71/78, Sep 18–22 71.5%; final golden 72/78, strong_usd 7/7, risk_on 4/4, 18.8 ms. Re-exported; this is the shipped bundle now.

Where things are

Path What
models/xau_transformer/ shipped ONNX bundle from round 3 torch_final (17,245 rows, golden 72/78)
models/xau_transformer/torch/ round 3 non-final weights (val numbers belong to these)
models/xau_transformer/torch_final/ round 3 final weights
session scratchpad sweep2/*_8870rows, sweep2/*_12270rows round 1 and round 2 weights
models/xauusd_sentiment.joblib TF-IDF, unchanged, still what ForexBot2 would load if it loaded anything
session scratchpad sweep/, sweep2/ all other sweep runs; may not survive a reboot
eval/results/golden_20260927_2315_transformer.* round 3 final model golden eval (72/78)
eval/results/archive_20260927_2313_transformer.json round 3 non-final archive eval
eval/results/golden_20260927_1949_*, archive_20260927_1947_* round 2 equivalents
eval/results/golden_20260927_1931_transformer.*, archive_20260927_1927_* round 1 equivalents

Known issues

  • export_onnx.py prints PARITY FAILED on max |diff| (0.06–0.08 vs tolerance 0.05) even with 0 flips. Loosen the tolerance or check flips only.
  • 14 of 340 label batches failed (DeepSeek timeouts / one 502); those headlines are simply unlabelled and will be picked up by the next run. About 570 precise headlines were unlabelled at the end, plus whatever the still-running crawl adds.
  • 2025 labels have a mean xau of +0.12, higher than other years. Possible hindsight in DeepSeek's labels for the rally year.

Next

See "Next, revised" in the results file. The weak spots are fixed on the golden set. The step that now produces new information is shadow mode in ForexBot2; more history labels will not. Watch the gold-key live-feed number (83 → 79%) as history labels come to dominate the training set. The precise crawl is still running in the background and is free to leave alone.

Reference in New Issue View Git Blame Copy Permalink