Harwinder Sekhon
01baef48ff
TF-IDF baseline (news_sentiment.py, models/xauusd_sentiment.joblib) and the two-head DeBERTa-v3-small transformer (transformer/, news_transformer.py) with its Google News crawler and DeepSeek labeller (data_collect/), the golden-set and archive evals (eval/), and the 2026-09-22 and 2026-09-27 results and progress notes. Transformer weights (model.pt, model.onnx) and the crawl database are not tracked; rebuild with transformer/train.py and transformer/export_onnx.py. The DeepSeek label cache in data/ is tracked.
35 lines
2.6 KiB
Plaintext
35 lines
2.6 KiB
Plaintext
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
|
WARNING huggingface_hub.utils._http: Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
|
/home/harwinder/Development/ML-NEWS/.venv-train/lib64/python3.12/site-packages/torch/jit/_script.py:1491: FutureWarning: `torch.jit.script` is deprecated. Please switch to `torch.compile` or `torch.export`.
|
|
warnings.warn(
|
|
3637 labelled rows -> 3620 unique headlines (0 golden-set duplicates dropped)
|
|
splits:
|
|
train n= 2402 relevant 757 irrelevant 1448 grey 197 bull 275 bear 482 {'history': 921, 'newsdata': 1481}
|
|
dev n= 200 relevant 54 irrelevant 132 grey 14 bull 22 bear 32 {'newsdata': 121, 'history': 79}
|
|
val n= 1018 relevant 378 irrelevant 501 grey 139 bull 171 bear 207 {'newsdata': 1018}
|
|
|
|
Loading weights: 0%| | 0/102 [00:00<?, ?it/s]
|
|
Loading weights: 100%|██████████| 102/102 [00:00<00:00, 1282.76it/s]
|
|
[transformers] [1mDebertaV2Model LOAD REPORT[0m from: microsoft/deberta-v3-small
|
|
Key | Status | |
|
|
----------------------------------------+------------+--+-
|
|
mask_predictions.classifier.weight | UNEXPECTED | |
|
|
lm_predictions.lm_head.LayerNorm.bias | UNEXPECTED | |
|
|
lm_predictions.lm_head.dense.weight | UNEXPECTED | |
|
|
mask_predictions.dense.weight | UNEXPECTED | |
|
|
mask_predictions.LayerNorm.weight | UNEXPECTED | |
|
|
mask_predictions.LayerNorm.bias | UNEXPECTED | |
|
|
lm_predictions.lm_head.bias | UNEXPECTED | |
|
|
lm_predictions.lm_head.dense.bias | UNEXPECTED | |
|
|
mask_predictions.dense.bias | UNEXPECTED | |
|
|
mask_predictions.classifier.bias | UNEXPECTED | |
|
|
lm_predictions.lm_head.LayerNorm.weight | UNEXPECTED | |
|
|
|
|
Notes:
|
|
- UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch.
|
|
epoch 1: loss 1.5514 dev 3-class 0.680 rel F1 0.780 dir acc 0.685 thr 0.3 (3s)
|
|
epoch 2: loss 0.9117 dev 3-class 0.770 rel F1 0.787 dir acc 0.852 thr 0.55 (6s)
|
|
epoch 3: loss 0.6081 dev 3-class 0.775 rel F1 0.813 dir acc 0.889 thr 0.2 (9s)
|
|
epoch 4: loss 0.4231 dev 3-class 0.775 rel F1 0.813 dir acc 0.870 thr 0.25 (12s)
|
|
best epoch 3, relevance threshold 0.2
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.639 macro-F1 0.635 rel P/R/F1 0.684/0.881/0.770 dir acc 0.886
|
|
wrote /home/harwinder/Development/ML-NEWS/models/xau_transformer/torch/model.pt and meta.json
|