Harwinder Sekhon
01baef48ff
TF-IDF baseline (news_sentiment.py, models/xauusd_sentiment.joblib) and the two-head DeBERTa-v3-small transformer (transformer/, news_transformer.py) with its Google News crawler and DeepSeek labeller (data_collect/), the golden-set and archive evals (eval/), and the 2026-09-22 and 2026-09-27 results and progress notes. Transformer weights (model.pt, model.onnx) and the crawl database are not tracked; rebuild with transformer/train.py and transformer/export_onnx.py. The DeepSeek label cache in data/ is tracked.
70 lines
6.2 KiB
Plaintext
70 lines
6.2 KiB
Plaintext
### small_e4_bs32 --epochs 4 --batch-size 32
|
|
8887 labelled rows -> 8870 unique headlines (0 golden-set duplicates dropped)
|
|
train n= 7460 relevant 2332 irrelevant 4492 grey 636 bull 1290 bear 1042 {'history': 5940, 'newsdata': 1520}
|
|
dev n= 392 relevant 119 irrelevant 245 grey 28 bull 72 bear 47 {'newsdata': 82, 'history': 310}
|
|
val n= 1018 relevant 378 irrelevant 501 grey 139 bull 171 bear 207 {'newsdata': 1018}
|
|
epoch 1: loss 1.0518 dev 3-class 0.786 rel F1 0.820 dir acc 0.882 thr 0.6 (12s)
|
|
epoch 2: loss 0.6078 dev 3-class 0.811 rel F1 0.877 dir acc 0.899 thr 0.65 (24s)
|
|
epoch 3: loss 0.3592 dev 3-class 0.816 rel F1 0.841 dir acc 0.908 thr 0.4 (35s)
|
|
epoch 4: loss 0.2141 dev 3-class 0.824 rel F1 0.865 dir acc 0.916 thr 0.25 (47s)
|
|
best epoch 4, relevance threshold 0.25
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.708 macro-F1 0.704 rel P/R/F1 0.788/0.876/0.830 dir acc 0.907
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep2/small_e4_bs32/model.pt and meta.json
|
|
### small_e4_bs64 --epochs 4
|
|
8887 labelled rows -> 8870 unique headlines (0 golden-set duplicates dropped)
|
|
train n= 7460 relevant 2332 irrelevant 4492 grey 636 bull 1290 bear 1042 {'history': 5940, 'newsdata': 1520}
|
|
dev n= 392 relevant 119 irrelevant 245 grey 28 bull 72 bear 47 {'newsdata': 82, 'history': 310}
|
|
val n= 1018 relevant 378 irrelevant 501 grey 139 bull 171 bear 207 {'newsdata': 1018}
|
|
epoch 1: loss 1.0966 dev 3-class 0.755 rel F1 0.783 dir acc 0.866 thr 0.35 (8s)
|
|
epoch 2: loss 0.5993 dev 3-class 0.816 rel F1 0.876 dir acc 0.899 thr 0.75 (15s)
|
|
epoch 3: loss 0.3881 dev 3-class 0.824 rel F1 0.869 dir acc 0.916 thr 0.6 (23s)
|
|
epoch 4: loss 0.2570 dev 3-class 0.821 rel F1 0.861 dir acc 0.916 thr 0.35 (30s)
|
|
best epoch 3, relevance threshold 0.6
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.720 macro-F1 0.715 rel P/R/F1 0.803/0.884/0.841 dir acc 0.942
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep2/small_e4_bs64/model.pt and meta.json
|
|
### small_e6_bs32 --epochs 6 --batch-size 32
|
|
8887 labelled rows -> 8870 unique headlines (0 golden-set duplicates dropped)
|
|
train n= 7460 relevant 2332 irrelevant 4492 grey 636 bull 1290 bear 1042 {'history': 5940, 'newsdata': 1520}
|
|
dev n= 392 relevant 119 irrelevant 245 grey 28 bull 72 bear 47 {'newsdata': 82, 'history': 310}
|
|
val n= 1018 relevant 378 irrelevant 501 grey 139 bull 171 bear 207 {'newsdata': 1018}
|
|
epoch 1: loss 1.0595 dev 3-class 0.786 rel F1 0.826 dir acc 0.832 thr 0.55 (12s)
|
|
epoch 2: loss 0.6070 dev 3-class 0.811 rel F1 0.845 dir acc 0.908 thr 0.35 (24s)
|
|
epoch 3: loss 0.3743 dev 3-class 0.811 rel F1 0.855 dir acc 0.891 thr 0.25 (35s)
|
|
epoch 4: loss 0.2223 dev 3-class 0.814 rel F1 0.854 dir acc 0.891 thr 0.2 (47s)
|
|
epoch 5: loss 0.1206 dev 3-class 0.814 rel F1 0.877 dir acc 0.908 thr 0.7 (59s)
|
|
epoch 6: loss 0.0744 dev 3-class 0.811 rel F1 0.851 dir acc 0.908 thr 0.25 (70s)
|
|
best epoch 4, relevance threshold 0.2
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.713 macro-F1 0.708 rel P/R/F1 0.811/0.876/0.842 dir acc 0.897
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep2/small_e6_bs32/model.pt and meta.json
|
|
### small_e8_bs64 --epochs 8
|
|
8887 labelled rows -> 8870 unique headlines (0 golden-set duplicates dropped)
|
|
train n= 7460 relevant 2332 irrelevant 4492 grey 636 bull 1290 bear 1042 {'history': 5940, 'newsdata': 1520}
|
|
dev n= 392 relevant 119 irrelevant 245 grey 28 bull 72 bear 47 {'newsdata': 82, 'history': 310}
|
|
val n= 1018 relevant 378 irrelevant 501 grey 139 bull 171 bear 207 {'newsdata': 1018}
|
|
epoch 1: loss 1.1464 dev 3-class 0.763 rel F1 0.787 dir acc 0.866 thr 0.45 (8s)
|
|
epoch 2: loss 0.6293 dev 3-class 0.796 rel F1 0.809 dir acc 0.882 thr 0.55 (15s)
|
|
epoch 3: loss 0.4127 dev 3-class 0.798 rel F1 0.838 dir acc 0.908 thr 0.7 (23s)
|
|
epoch 4: loss 0.2944 dev 3-class 0.801 rel F1 0.863 dir acc 0.891 thr 0.25 (30s)
|
|
epoch 5: loss 0.1823 dev 3-class 0.806 rel F1 0.838 dir acc 0.916 thr 0.25 (38s)
|
|
epoch 6: loss 0.1072 dev 3-class 0.804 rel F1 0.847 dir acc 0.916 thr 0.2 (46s)
|
|
epoch 7: loss 0.0660 dev 3-class 0.819 rel F1 0.873 dir acc 0.899 thr 0.5 (53s)
|
|
epoch 8: loss 0.0513 dev 3-class 0.821 rel F1 0.858 dir acc 0.916 thr 0.4 (60s)
|
|
best epoch 8, relevance threshold 0.4
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.709 macro-F1 0.704 rel P/R/F1 0.789/0.889/0.836 dir acc 0.905
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep2/small_e8_bs64/model.pt and meta.json
|
|
### small_e6_bs32_lr3e5 --epochs 6 --batch-size 32 --lr 3e-5
|
|
8887 labelled rows -> 8870 unique headlines (0 golden-set duplicates dropped)
|
|
train n= 7460 relevant 2332 irrelevant 4492 grey 636 bull 1290 bear 1042 {'history': 5940, 'newsdata': 1520}
|
|
dev n= 392 relevant 119 irrelevant 245 grey 28 bull 72 bear 47 {'newsdata': 82, 'history': 310}
|
|
val n= 1018 relevant 378 irrelevant 501 grey 139 bull 171 bear 207 {'newsdata': 1018}
|
|
epoch 1: loss 1.1073 dev 3-class 0.773 rel F1 0.803 dir acc 0.849 thr 0.6 (12s)
|
|
epoch 2: loss 0.6466 dev 3-class 0.814 rel F1 0.824 dir acc 0.899 thr 0.35 (23s)
|
|
epoch 3: loss 0.4186 dev 3-class 0.819 rel F1 0.856 dir acc 0.908 thr 0.55 (35s)
|
|
epoch 4: loss 0.2727 dev 3-class 0.814 rel F1 0.855 dir acc 0.908 thr 0.25 (47s)
|
|
epoch 5: loss 0.1877 dev 3-class 0.816 rel F1 0.835 dir acc 0.891 thr 0.2 (58s)
|
|
epoch 6: loss 0.1338 dev 3-class 0.821 rel F1 0.849 dir acc 0.899 thr 0.2 (70s)
|
|
best epoch 6, relevance threshold 0.2
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.714 macro-F1 0.711 rel P/R/F1 0.779/0.897/0.834 dir acc 0.926
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep2/small_e6_bs32_lr3e5/model.pt and meta.json
|
|
### SWEEP2 DONE
|