Changed news engine code a little
This commit is contained in:
@@ -215,6 +215,42 @@ Telegram: edit `SCAN_TIMEFRAMES`.
|
||||
|
||||
---
|
||||
|
||||
## Suggested improvements (not yet implemented)
|
||||
|
||||
### Replace keyword sentiment with FinBERT
|
||||
|
||||
The current `news_engine._analyze_headline_sentiment()` uses keyword matching against `BULLISH_KEYWORDS` / `BEARISH_KEYWORDS` sets ([news_engine.py:82-95](news_engine.py#L82-L95)). This misses obvious context — e.g. "RBA signals pause", "Fed patient on rate cuts", "inflation cools" all score `0.00` because the lexicon doesn't include "pause", "patient", or "cools". Around 80% of headlines fetched currently score 0.
|
||||
|
||||
**Recommended fix: FinBERT (HuggingFace pretrained model on financial news).**
|
||||
|
||||
- Drop-in replacement for `_analyze_headline_sentiment()`.
|
||||
- Pretrained on financial headlines, ~70-75% accuracy on this specific domain.
|
||||
- One-time ~500MB model download; CPU inference ~50ms/headline (fine for ~30 headlines per fetch).
|
||||
- No API cost, no external dependency at runtime.
|
||||
|
||||
Implementation sketch (~30 lines):
|
||||
```python
|
||||
from transformers import pipeline
|
||||
_finbert = pipeline("sentiment-analysis", model="ProsusAI/finbert")
|
||||
|
||||
def _analyze_headline_sentiment(self, headline: str) -> float:
|
||||
result = _finbert(headline[:512])[0] # truncate to model max
|
||||
label, score = result["label"], result["score"]
|
||||
if label == "positive": return score
|
||||
if label == "negative": return -score
|
||||
return 0.0
|
||||
```
|
||||
|
||||
Add `transformers` and `torch` (or `torch-cpu`) to `requirements.txt`. Cache scored headlines by `hash(headline)` to avoid re-scoring across pairs in the same fetch. Add a try/except fallback to the current keyword logic if model load fails (graceful degradation).
|
||||
|
||||
**Realistic uplift on overall signal quality: 1-3% win-rate.** Sentiment is only 10% of the composite score ([config.py:139](config.py#L139)), so even doubling sentiment accuracy moves the needle modestly. Worth doing for technical correctness, not as a P&L optimization.
|
||||
|
||||
**Higher-tier alternative: Claude/GPT API per headline.** ~80-85% accurate, $0.001/headline (~$0.03 per /scan with caching). Adds external dependency and latency. Only worth it if FinBERT proves insufficient.
|
||||
|
||||
**Cheap alternative without ML: Expand keyword sets.** Add ~50-100 phrases to `BULLISH_KEYWORDS` / `BEARISH_KEYWORDS` ("pause", "patient", "data-dependent", "soft landing", "beat", "revised up" / "miss", "revised down", "warning", "fragile"). Gets ~50% → ~60% accuracy in 30 min, no new deps.
|
||||
|
||||
---
|
||||
|
||||
## Recent architectural decisions (kept short)
|
||||
|
||||
- **Removed synthetic data path** — explicit failure beats fake prices.
|
||||
|
||||
Reference in New Issue
Block a user