Harwinder Sekhon
01baef48ff
TF-IDF baseline (news_sentiment.py, models/xauusd_sentiment.joblib) and the two-head DeBERTa-v3-small transformer (transformer/, news_transformer.py) with its Google News crawler and DeepSeek labeller (data_collect/), the golden-set and archive evals (eval/), and the 2026-09-22 and 2026-09-27 results and progress notes. Transformer weights (model.pt, model.onnx) and the crawl database are not tracked; rebuild with transformer/train.py and transformer/export_onnx.py. The DeepSeek label cache in data/ is tracked.
70 lines
5.8 KiB
Plaintext
70 lines
5.8 KiB
Plaintext
### small_lr5e5_e8 --epochs 8
|
|
epoch 1: loss 1.6317 dev 3-class 0.680 rel F1 0.744 dir acc 0.630 thr 0.4 (3s)
|
|
epoch 2: loss 0.9693 dev 3-class 0.755 rel F1 0.758 dir acc 0.852 thr 0.5 (6s)
|
|
epoch 3: loss 0.5949 dev 3-class 0.735 rel F1 0.828 dir acc 0.852 thr 0.25 (9s)
|
|
epoch 4: loss 0.3509 dev 3-class 0.745 rel F1 0.868 dir acc 0.778 thr 0.45 (11s)
|
|
epoch 5: loss 0.2509 dev 3-class 0.755 rel F1 0.835 dir acc 0.870 thr 0.2 (14s)
|
|
epoch 6: loss 0.1451 dev 3-class 0.755 rel F1 0.867 dir acc 0.889 thr 0.65 (16s)
|
|
epoch 7: loss 0.1023 dev 3-class 0.760 rel F1 0.813 dir acc 0.889 thr 0.25 (19s)
|
|
epoch 8: loss 0.0671 dev 3-class 0.750 rel F1 0.881 dir acc 0.870 thr 0.65 (21s)
|
|
best epoch 7, relevance threshold 0.25
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.681 macro-F1 0.677 rel P/R/F1 0.721/0.876/0.791 dir acc 0.913
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/small_lr5e5_e8/model.pt and meta.json
|
|
### small_lr3e5_e6 --epochs 6 --lr 3e-5
|
|
epoch 1: loss 1.6456 dev 3-class 0.695 rel F1 0.766 dir acc 0.685 thr 0.65 (3s)
|
|
epoch 2: loss 1.0588 dev 3-class 0.745 rel F1 0.748 dir acc 0.815 thr 0.6 (6s)
|
|
epoch 3: loss 0.7237 dev 3-class 0.740 rel F1 0.810 dir acc 0.852 thr 0.3 (8s)
|
|
epoch 4: loss 0.5056 dev 3-class 0.765 rel F1 0.787 dir acc 0.833 thr 0.25 (11s)
|
|
epoch 5: loss 0.3863 dev 3-class 0.770 rel F1 0.803 dir acc 0.870 thr 0.3 (14s)
|
|
epoch 6: loss 0.3341 dev 3-class 0.750 rel F1 0.803 dir acc 0.870 thr 0.3 (17s)
|
|
best epoch 5, relevance threshold 0.3
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.646 macro-F1 0.646 rel P/R/F1 0.675/0.889/0.767 dir acc 0.870
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/small_lr3e5_e6/model.pt and meta.json
|
|
### small_lr2e5_e8 --epochs 8 --lr 2e-5
|
|
epoch 1: loss 1.7090 dev 3-class 0.680 rel F1 0.718 dir acc 0.685 thr 0.6 (3s)
|
|
epoch 2: loss 1.2253 dev 3-class 0.720 rel F1 0.773 dir acc 0.759 thr 0.65 (6s)
|
|
epoch 3: loss 0.8685 dev 3-class 0.740 rel F1 0.769 dir acc 0.852 thr 0.25 (9s)
|
|
epoch 4: loss 0.6338 dev 3-class 0.760 rel F1 0.787 dir acc 0.833 thr 0.25 (11s)
|
|
epoch 5: loss 0.4953 dev 3-class 0.760 rel F1 0.785 dir acc 0.870 thr 0.25 (14s)
|
|
epoch 6: loss 0.4410 dev 3-class 0.745 rel F1 0.781 dir acc 0.852 thr 0.35 (17s)
|
|
epoch 7: loss 0.3537 dev 3-class 0.760 rel F1 0.794 dir acc 0.870 thr 0.3 (19s)
|
|
epoch 8: loss 0.3262 dev 3-class 0.755 rel F1 0.820 dir acc 0.870 thr 0.35 (22s)
|
|
best epoch 4, relevance threshold 0.25
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.620 macro-F1 0.624 rel P/R/F1 0.638/0.897/0.746 dir acc 0.849
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/small_lr2e5_e8/model.pt and meta.json
|
|
### small_lr5e5_e4_s7 --epochs 4 --seed 7
|
|
epoch 1: loss 1.5257 dev 3-class 0.625 rel F1 0.774 dir acc 0.615 thr 0.75 (3s)
|
|
epoch 2: loss 1.0113 dev 3-class 0.685 rel F1 0.773 dir acc 0.785 thr 0.45 (6s)
|
|
epoch 3: loss 0.6608 dev 3-class 0.710 rel F1 0.812 dir acc 0.846 thr 0.7 (9s)
|
|
epoch 4: loss 0.4799 dev 3-class 0.720 rel F1 0.837 dir acc 0.862 thr 0.8 (12s)
|
|
best epoch 4, relevance threshold 0.8
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.629 macro-F1 0.610 rel P/R/F1 0.818/0.759/0.787 dir acc 0.857
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/small_lr5e5_e4_s7/model.pt and meta.json
|
|
### small_lr5e5_e4_bs32 --epochs 4 --batch-size 32
|
|
epoch 1: loss 1.4895 dev 3-class 0.730 rel F1 0.787 dir acc 0.833 thr 0.45 (4s)
|
|
epoch 2: loss 0.7746 dev 3-class 0.755 rel F1 0.784 dir acc 0.870 thr 0.25 (9s)
|
|
epoch 3: loss 0.4328 dev 3-class 0.760 rel F1 0.833 dir acc 0.833 thr 0.3 (13s)
|
|
epoch 4: loss 0.2717 dev 3-class 0.780 rel F1 0.803 dir acc 0.870 thr 0.2 (17s)
|
|
best epoch 4, relevance threshold 0.2
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.656 macro-F1 0.656 rel P/R/F1 0.683/0.894/0.774 dir acc 0.899
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/small_lr5e5_e4_bs32/model.pt and meta.json
|
|
### base_lr2e5_e4 --base microsoft/deberta-v3-base --epochs 4 --lr 2e-5 --batch-size 32
|
|
epoch 1: loss 1.5986 dev 3-class 0.675 rel F1 0.775 dir acc 0.648 thr 0.75 (7s)
|
|
epoch 2: loss 0.9843 dev 3-class 0.755 rel F1 0.778 dir acc 0.889 thr 0.45 (13s)
|
|
epoch 3: loss 0.5717 dev 3-class 0.760 rel F1 0.797 dir acc 0.889 thr 0.4 (19s)
|
|
epoch 4: loss 0.4177 dev 3-class 0.765 rel F1 0.800 dir acc 0.907 thr 0.45 (26s)
|
|
best epoch 4, relevance threshold 0.45
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.638 macro-F1 0.634 rel P/R/F1 0.678/0.873/0.763 dir acc 0.886
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/base_lr2e5_e4/model.pt and meta.json
|
|
### base_lr3e5_e6 --base microsoft/deberta-v3-base --epochs 6 --lr 3e-5 --batch-size 32
|
|
epoch 1: loss 1.5635 dev 3-class 0.705 rel F1 0.786 dir acc 0.741 thr 0.7 (6s)
|
|
epoch 2: loss 0.9028 dev 3-class 0.775 rel F1 0.800 dir acc 0.926 thr 0.65 (12s)
|
|
epoch 3: loss 0.4669 dev 3-class 0.770 rel F1 0.867 dir acc 0.870 thr 0.75 (19s)
|
|
epoch 4: loss 0.3115 dev 3-class 0.775 rel F1 0.840 dir acc 0.889 thr 0.55 (24s)
|
|
epoch 5: loss 0.1834 dev 3-class 0.780 rel F1 0.855 dir acc 0.889 thr 0.4 (30s)
|
|
epoch 6: loss 0.1201 dev 3-class 0.785 rel F1 0.879 dir acc 0.907 thr 0.65 (36s)
|
|
best epoch 6, relevance threshold 0.65
|
|
VAL (unseen 2026-09-18+, n=1018): 3-class 0.673 macro-F1 0.671 rel P/R/F1 0.750/0.857/0.800 dir acc 0.910
|
|
wrote /tmp/claude-1000/-home-harwinder-Development-ML-NEWS/2ebe8a2c-6870-461a-bffa-60b46627e57b/scratchpad/sweep/base_lr3e5_e6/model.pt and meta.json
|
|
### SWEEP DONE
|