diff --git a/CLAUDE.md b/CLAUDE.md
index 10f1a01..15f95d1 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
## Commands
```bash
-python test_all.py # QA suite: 273 tests, 18 modules, no network (mocked)
+python test_all.py # QA suite: 286 tests, 19 modules, no network (mocked)
python main.py --pair XAUUSD --timeframe H1 --detail # CLI signal report (--json for full dump)
python telegram_bot.py # the bot (refuses to boot without credentials)
python oanda_daemon.py # auto-execution daemon on Oanda practice account
@@ -30,7 +30,7 @@ Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy
## Configuration & credentials
-Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`.
+Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`, `DEEPSEEK_API_KEY` (news LLM; the `deepseek` config.json block holds model/enabled/timeout/daily cap).
## Architecture
@@ -53,8 +53,8 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
- **`engine.py`** — `TechnicalEngine`: 9 scored indicators (ema_trend, rsi, macd, bollinger, adx, roc, keltner, divergence, candlestick) + ATR as context. Regime classification (trending_up/down, `trending` = mixed/direction-UNKNOWN, transitioning, ranging) drives per-indicator weight multipliers and regime-aware RSI reinterpretation. Compute order matters (ATR first; MACD always computed even when not a pillar — divergence needs the histogram). Private series ride on the result as `_underscore` attrs; scorers/formatters skip anything `_`-prefixed or weight-0.
- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
- **`pair_signals/`** — per-pair config factories. **Always use the factory (`produce_signal` / `make_config`), never bare `PlatformConfig()`** — the factory applies the config.json overlay first, then the validated per-TF deltas (M5 hour-9 block, M15 trail arm 0.70, M15 SELL-hour block, etc.; evidence in the docstring). A test enforces factory ≡ inline-config equivalence. Confirmation map: M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1].
-- **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`).
-- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
+- **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that.
+- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown, trail ratchets at bar close exactly once per bar. Accepted divergence: one position total.
- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
diff --git a/app_config.py b/app_config.py
index 72e639b..2709487 100644
--- a/app_config.py
+++ b/app_config.py
@@ -85,11 +85,15 @@ _VALID_LOG_LEVELS = ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")
# AUTOSCAN_CONFIRMATIONS entries in telegram_bot.py).
_SCANNABLE_TFS = ("M5", "M15", "H1", "H4")
-_KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level", "oanda"}
+_KNOWN_TOP = {"telegram", "data_providers", "scanning", "pairs", "log_level",
+ "oanda", "deepseek"}
_KNOWN_OANDA = {
"account_id", "api_token", "base_url", "lot_size",
"quality_gate", "max_hold_bars",
}
+_KNOWN_DEEPSEEK = {
+ "api_key", "model", "enabled", "timeout_seconds", "daily_request_cap",
+}
_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
_KNOWN_SCANNING = {"pairs", "timeframes"}
_KNOWN_PAIR_KEYS = {"blocked_hours_utc"}
@@ -113,6 +117,7 @@ class AppConfig:
Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4])
pair_overrides: Dict[str, Dict[str, Any]] = field(default_factory=dict)
oanda_settings: Dict[str, Any] = field(default_factory=dict)
+ deepseek_settings: Dict[str, Any] = field(default_factory=dict)
log_level: str = "INFO"
@@ -248,6 +253,29 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
"config.json: oanda.max_hold_bars must be a positive integer")
app.oanda_settings = dict(oanda)
+ deepseek = raw.get("deepseek", {})
+ _check_keys(deepseek, _KNOWN_DEEPSEEK, '"deepseek"')
+ if "enabled" in deepseek and not isinstance(deepseek["enabled"], bool):
+ raise AppConfigError(
+ "config.json: deepseek.enabled must be true or false")
+ if "timeout_seconds" in deepseek:
+ ts = deepseek["timeout_seconds"]
+ if isinstance(ts, bool) or not isinstance(ts, (int, float)) or ts <= 0:
+ raise AppConfigError(
+ "config.json: deepseek.timeout_seconds must be a positive "
+ "number")
+ if "daily_request_cap" in deepseek:
+ cap = deepseek["daily_request_cap"]
+ if isinstance(cap, bool) or not isinstance(cap, int) or cap < 1:
+ raise AppConfigError(
+ "config.json: deepseek.daily_request_cap must be a positive "
+ "integer")
+ if "model" in deepseek:
+ if not isinstance(deepseek["model"], str) or not deepseek["model"]:
+ raise AppConfigError(
+ "config.json: deepseek.model must be a non-empty string")
+ app.deepseek_settings = dict(deepseek)
+
level = raw.get("log_level", app.log_level)
if level not in _VALID_LOG_LEVELS:
raise AppConfigError(
@@ -339,6 +367,14 @@ def oanda_base_url(app: Optional[AppConfig] = None) -> str:
app.oanda_settings.get("base_url", "https://api-fxpractice.oanda.com"))
+# ── DeepSeek resolution helper (env → config.json → "") ──────────────
+
+def deepseek_api_key(app: Optional[AppConfig] = None) -> str:
+ app = app or get_app_config()
+ return os.environ.get("DEEPSEEK_API_KEY") or str(
+ app.deepseek_settings.get("api_key", ""))
+
+
# ── PlatformConfig overlay ────────────────────────────────────────────
def apply_pair_overrides(
diff --git a/config.json.example b/config.json.example
index 785607a..0e4eba6 100644
--- a/config.json.example
+++ b/config.json.example
@@ -33,5 +33,12 @@
"quality_gate": "C",
"max_hold_bars": 300
},
+ "deepseek": {
+ "api_key": "REPLACE_WITH_YOUR_DEEPSEEK_API_KEY_OR_EMPTY",
+ "model": "deepseek-v4-flash",
+ "enabled": true,
+ "timeout_seconds": 12,
+ "daily_request_cap": 500
+ },
"log_level": "INFO"
}
diff --git a/docs/user_guide.html b/docs/user_guide.html
index 5fd8793..2db2087 100644
--- a/docs/user_guide.html
+++ b/docs/user_guide.html
@@ -69,6 +69,7 @@
| Score / confidence | The regime-weighted composite of all nine indicators. Bigger absolute score = stronger consensus. |
| MTF aligned | The two confirmation timeframes agree with the trade direction. Signals without alignment are filtered out on M5. |
| Warnings ⚠️ | Context, not veto. Backtesting shows warned trades perform about the same — strong momentum often triggers warnings. |
+| Market context 📰 | An AI-written one-liner summarizing today's macro backdrop (headlines + high-impact releases) and its likely pull on gold. Informational only — it never blocks or grades a signal. |
| SL / TP1 / RR | Stop loss sits behind the nearest swing level with an ATR buffer. TP1 is the nearest opposing level. 1R = the entry-to-SL distance = the amount you risk. |
| Trail plan 🔄 | The real exit. Once price moves +0.6R in your favor (+0.8R for pullback setups), move the stop to trail 0.75R behind the best price and update it at each candle close. TP1 is advisory — let winners run. |
diff --git a/llm_client.py b/llm_client.py
new file mode 100644
index 0000000..4588120
--- /dev/null
+++ b/llm_client.py
@@ -0,0 +1,185 @@
+"""
+ForexBot2 — DeepSeek LLM client
+================================
+Thin requests-based client for DeepSeek's OpenAI-compatible chat API,
+used by `news_engine` for display/warning-only enrichment (headline
+sentiment, event surprise direction, market summary). Signals are NEVER
+gated on LLM output — every caller has a non-LLM fallback and treats a
+`None` return as "use the fallback".
+
+Design constraints:
+- One attempt per call, no retries: news runs inside scan cycles; the
+ news engine's 600 s context cache is the natural retry cadence.
+- A UTC-day request cap bounds spend even if a bug causes call storms.
+- Cost: DeepSeek caches identical prompt PREFIXES server-side (cache-hit
+ input is ~50x cheaper), so callers pass a STATIC system prompt (no
+ timestamps or interpolation) and put all variable content in the user
+ payload.
+
+This is a module-level singleton (`get_llm_client`) because the daily
+cap counter must survive the per-scan reconstruction of `NewsEngine`.
+"""
+
+import datetime as _dt
+import json
+import logging
+import threading
+from typing import Any, Dict, Optional
+
+import requests
+
+from app_config import deepseek_api_key, get_app_config
+
+logger = logging.getLogger(__name__)
+
+DEEPSEEK_URL = "https://api.deepseek.com/chat/completions"
+DEFAULT_MODEL = "deepseek-v4-flash"
+# 8s timed out on a measured cold-prefix-cache headline batch (2026-08-03);
+# warm-cache calls come back well under that. 12s buys cold-start headroom
+# while still bounding worst-case scan latency (≤3 calls per cache window).
+DEFAULT_TIMEOUT = 12.0
+DEFAULT_DAILY_CAP = 500
+
+
+class DeepSeekClient:
+ """JSON-in/JSON-out chat wrapper. All failures return None."""
+
+ def __init__(
+ self,
+ api_key: str,
+ model: str = DEFAULT_MODEL,
+ timeout: float = DEFAULT_TIMEOUT,
+ daily_cap: int = DEFAULT_DAILY_CAP,
+ enabled: bool = True,
+ ):
+ self._api_key = api_key or ""
+ self._model = model
+ self._timeout = timeout
+ self._daily_cap = daily_cap
+ self._enabled = enabled
+ self._session = requests.Session()
+ self._session.headers.update({
+ "Content-Type": "application/json",
+ "Authorization": f"Bearer {self._api_key}",
+ })
+ self._lock = threading.Lock()
+ self._count_day: Optional[_dt.date] = None
+ self._count = 0
+ self._warned: set = set()
+
+ @property
+ def available(self) -> bool:
+ return self._enabled and bool(self._api_key)
+
+ def _warn_once(self, kind: str, msg: str, *args: Any) -> None:
+ if kind in self._warned:
+ logger.debug(msg, *args)
+ else:
+ self._warned.add(kind)
+ logger.warning(msg, *args)
+
+ def _under_cap(self) -> bool:
+ """Count a request against the UTC-day cap; False = don't call."""
+ today = _dt.datetime.now(_dt.timezone.utc).date()
+ with self._lock:
+ if self._count_day != today:
+ self._count_day = today
+ self._count = 0
+ self._warned.discard("cap")
+ if self._count >= self._daily_cap:
+ self._warn_once(
+ "cap", "DeepSeek daily request cap (%d) exhausted — "
+ "falling back to keyword analysis until UTC midnight",
+ self._daily_cap)
+ return False
+ self._count += 1
+ return True
+
+ def chat_json(self, system_prompt: str,
+ user_payload: Dict[str, Any]) -> Optional[Dict[str, Any]]:
+ """One chat call; returns the model's JSON object or None."""
+ if not self.available or not self._under_cap():
+ return None
+ body = {
+ "model": self._model,
+ "messages": [
+ {"role": "system", "content": system_prompt},
+ {"role": "user",
+ "content": json.dumps(user_payload, sort_keys=True)},
+ ],
+ "response_format": {"type": "json_object"},
+ "temperature": 0,
+ # V4-Flash is a hybrid thinking model; without this flag it
+ # burns hundreds of reasoning tokens per call — measured
+ # 2026-08-03: a 25-headline batch blew straight through the
+ # timeout AND could exhaust max_tokens before the answer,
+ # returning empty content. The doctrine + few-shot examples
+ # in the system prompts carry the quality instead.
+ "thinking": {"type": "disabled"},
+ # Must fit the largest reply: a full sentiment batch is
+ # ~10 tokens per headline; 1000 truncated mid-JSON on a real
+ # RSS cycle (2026-08-03).
+ "max_tokens": 2000,
+ }
+ try:
+ resp = self._session.post(
+ DEEPSEEK_URL, json=body, timeout=self._timeout)
+ except requests.RequestException as e:
+ self._warn_once("network", "DeepSeek call failed: %s", e)
+ return None
+ if resp.status_code != 200:
+ self._warn_once(
+ f"http_{resp.status_code}",
+ "DeepSeek HTTP %s: %s", resp.status_code, resp.text[:200])
+ return None
+ try:
+ parsed = self._parse_response(resp.json())
+ except ValueError:
+ parsed = None
+ if parsed is None:
+ self._warn_once("parse", "DeepSeek response was not the "
+ "expected JSON object — ignoring")
+ return parsed
+
+ @staticmethod
+ def _parse_response(body: Any) -> Optional[Dict[str, Any]]:
+ """Extract the assistant's JSON object from a chat completion."""
+ try:
+ content = body["choices"][0]["message"]["content"]
+ except (KeyError, IndexError, TypeError):
+ return None
+ try:
+ obj = json.loads(content)
+ except (TypeError, ValueError):
+ return None
+ return obj if isinstance(obj, dict) else None
+
+
+# ── Process-wide singleton ────────────────────────────────────────────
+
+_client: Optional[DeepSeekClient] = None
+_client_lock = threading.Lock()
+
+
+def get_llm_client() -> DeepSeekClient:
+ global _client
+ with _client_lock:
+ if _client is None:
+ settings = get_app_config().deepseek_settings
+ _client = DeepSeekClient(
+ api_key=deepseek_api_key(),
+ model=str(settings.get("model", DEFAULT_MODEL)),
+ timeout=float(settings.get("timeout_seconds",
+ DEFAULT_TIMEOUT)),
+ daily_cap=int(settings.get("daily_request_cap",
+ DEFAULT_DAILY_CAP)),
+ enabled=settings.get("enabled", True) is not False,
+ )
+ return _client
+
+
+def reset_llm_client() -> None:
+ """Drop the singleton (tests / config reloads)."""
+ global _client
+ with _client_lock:
+ _client = None
diff --git a/main.py b/main.py
index 89d1142..c49a7e1 100755
--- a/main.py
+++ b/main.py
@@ -161,6 +161,8 @@ def print_signal_report(report: SignalReport, detail: bool = False):
print(f" {Colors.YELLOW}📅{Colors.RESET} {report.event_description}")
if report.news_description and report.news_description != "Neutral news sentiment":
print(f" {Colors.MAGENTA}📰{Colors.RESET} {report.news_description}")
+ if report.news_summary:
+ print(f" {Colors.MAGENTA}🧠{Colors.RESET} {report.news_summary}")
# ── 6. DETAIL (only with --detail) ─────────────────────────────
if detail and report.primary_analysis:
@@ -206,6 +208,7 @@ def report_to_dict(report: SignalReport) -> dict:
"event_description": report.event_description,
"news_sentiment": round(report.news_sentiment, 3),
"news_description": report.news_description,
+ "news_summary": report.news_summary,
"bullish_reasons": report.bullish_reasons,
"bearish_reasons": report.bearish_reasons,
"warnings": report.warnings,
diff --git a/news_engine.py b/news_engine.py
index 21109f0..291ebda 100755
--- a/news_engine.py
+++ b/news_engine.py
@@ -8,6 +8,7 @@ Key principle: High-impact news events override technical signals.
A perfect technical setup can be destroyed by an unexpected NFP print.
"""
+import hashlib
import logging
import re
import threading
@@ -21,10 +22,62 @@ from typing import Dict, List, Optional, Tuple
import requests
from config import NewsConfig
+from llm_client import get_llm_client
logger = logging.getLogger(__name__)
+# ── DeepSeek prompts ─────────────────────────────────────────────────
+# STATIC constants on purpose: DeepSeek prices a repeated byte-identical
+# prompt prefix at the cache-hit rate (~50x cheaper), so nothing here may
+# be interpolated or timestamped. All variable content travels in the
+# user payload. LLM output is display/warning-only — it never gates
+# signals (see NewsConfig.sentiment_affects_signals).
+
+_LLM_SENTIMENT_SYSTEM = """\
+You are a forex/gold news analyst. You receive JSON:
+{"headlines": ["...", ...]}
+For EACH headline, in the same order, score:
+- "s": overall market sentiment of the headline, -1.0 (very bearish)
+ to 1.0 (very bullish), 0.0 if irrelevant or neutral.
+- "xau": the effect on the GOLD (XAU/USD) price, -1.0 (gold down)
+ to 1.0 (gold up), 0.0 if irrelevant.
+Gold doctrine: hawkish Fed / rate hikes / rising real yields / strong
+dollar are gold-bearish; easing / rate cuts / safe-haven flows /
+geopolitical stress / weak dollar are gold-bullish.
+Handle negation correctly ("no rate hike" is not hawkish).
+Reply with ONLY JSON: {"scores": [[s, xau], ...]} — exactly one
+[s, xau] pair per input headline, in the same order, nothing else.
+
+Examples:
+"Fed signals faster rate hikes as inflation stays hot" -> [0.4, -0.7]
+"Gold surges to record high on safe-haven demand" -> [0.8, 0.9]
+"Dollar slips after soft jobs report; Treasury yields fall" -> [-0.4, 0.6]
+"""
+
+_LLM_EVENT_SYSTEM = """\
+You are a forex/gold analyst. You receive one released economic event as
+JSON: {"event", "currency", "actual", "forecast", "previous"}.
+Decide the likely effect of this RELEASE (actual vs forecast) on the
+GOLD (XAU/USD) price. A stronger-than-forecast USD economy or a hawkish
+implication is usually gold-bearish; weaker data or a dovish implication
+is usually gold-bullish. Mind inverted metrics: higher unemployment or
+jobless claims = weaker economy.
+Reply with ONLY JSON:
+{"gold_direction": "bullish"|"bearish"|"neutral", "reason": ""}
+"""
+
+_LLM_SUMMARY_SYSTEM = """\
+You are a forex/gold market-context writer. You receive JSON with recent
+"headlines" and today's high-impact "events" (released: true means the
+number is out). Write a 1-2 sentence plain-English summary of the
+current macro backdrop for a GOLD (XAU/USD) trader: the dominant theme
+and its likely pull on gold. No advice, no hedging boilerplate, max 45
+words.
+Reply with ONLY JSON: {"summary": "..."}
+"""
+
+
_RELEASE_VALUE_RE = re.compile(r"(-?\d+(?:\.\d+)?)\s*([kmbt])?", re.IGNORECASE)
_RELEASE_MULTIPLIERS = {"k": 1e3, "m": 1e6, "b": 1e9, "t": 1e12}
@@ -105,6 +158,7 @@ class FundamentalContext:
news_headlines: List[NewsSentiment] = field(default_factory=list)
event_impact_on_pairs: Dict[str, str] = field(default_factory=dict)
calendar_available: bool = False # True if live calendar data was fetched
+ llm_summary: str = "" # DeepSeek market-context blurb (display-only)
class NewsEngine:
@@ -185,6 +239,17 @@ class NewsEngine:
_headlines_cache: Tuple[float, List[NewsSentiment]] = (0.0, [])
_EMPTY_RESULT_TTL = 60.0 # retry sooner after a failed/empty fetch
+ # DeepSeek result caches. Headlines and releases are immutable, so a
+ # scored one never needs re-scoring; bounded by dropping the oldest
+ # insertion. _llm_sent_cache is only touched under _cache_lock (its
+ # writer runs inside _cached_fetch); the other two take the lock
+ # around cache access but keep HTTP calls outside it.
+ _llm_sent_cache: Dict[str, Tuple[float, float]] = {} # sha1 → (s, xau)
+ _llm_event_cache: Dict[Tuple[str, str, str], str] = {}
+ _llm_summary_cache: Tuple[float, str] = (0.0, "")
+ _LLM_SENT_CACHE_MAX = 400
+ _LLM_EVENT_CACHE_MAX = 100
+
def __init__(self, config: NewsConfig):
self.config = config
self._session = requests.Session()
@@ -264,6 +329,12 @@ class NewsEngine:
except Exception as e:
logger.warning(f"News sentiment fetch failed: {e}")
+ # 3. LLM market-context summary (display-only)
+ try:
+ context.llm_summary = self._llm_market_summary(context)
+ except Exception as e:
+ logger.debug(f"LLM market summary failed: {e}")
+
return context
@staticmethod
@@ -311,10 +382,17 @@ class NewsEngine:
)
if event.minutes_until > -window:
if surprised:
- return "caution", (
+ desc = (
f"{event.currency} {event.event} released — {surprise} surprise "
f"(actual {event.actual} vs forecast {event.forecast})"
)
+ # LLM enrichment touches the DESCRIPTION only —
+ # `surprise` is numeric-direction and does not know
+ # whether a beat is gold-bullish; the LLM does.
+ gold_dir = self._llm_event_direction(event)
+ if gold_dir:
+ desc += f" — gold {gold_dir}"
+ return "caution", desc
return "caution", f"{event.currency} {event.event} just released"
return "clear", "No significant events"
@@ -596,6 +674,11 @@ class NewsEngine:
except Exception as e:
logger.debug(f"Failed to fetch RSS from {feed_url}: {e}")
+ try:
+ self._apply_llm_sentiment(headlines)
+ except Exception as e:
+ logger.debug(f"LLM sentiment failed, keyword scores stand: {e}")
+
return headlines
def _analyze_headline_sentiment(
@@ -637,6 +720,130 @@ class NewsEngine:
return (bullish_score - bearish_score) / total
+ # ── DeepSeek enrichment (display/warning-only, keyword fallback) ──
+
+ def _llm_headline_scores(
+ self, texts: List[str]
+ ) -> Optional[List[Tuple[float, float]]]:
+ """One batched DeepSeek call → [(overall, xau), ...] aligned with
+ `texts`, or None on any failure/mismatch (caller keeps keyword
+ scores). Tests stub this method directly."""
+ result = get_llm_client().chat_json(
+ _LLM_SENTIMENT_SYSTEM, {"headlines": texts})
+ if not result:
+ return None
+ scores = result.get("scores")
+ if not isinstance(scores, list) or len(scores) != len(texts):
+ return None
+ out = []
+ for item in scores:
+ try:
+ s = max(-1.0, min(1.0, float(item[0])))
+ x = max(-1.0, min(1.0, float(item[1])))
+ except (TypeError, KeyError, ValueError, IndexError):
+ return None
+ out.append((s, x))
+ return out
+
+ def _apply_llm_sentiment(self, headlines: List[NewsSentiment]) -> None:
+ """Overwrite keyword sentiment on `headlines` with LLM scores.
+
+ Runs inside _cached_fetch, i.e. UNDER _cache_lock — cache access
+ is direct and the (rare) HTTP call blocks sibling scans at most
+ once per cache window, matching the existing shared-fetch design.
+ Only headlines missing from the cache are sent; a failed batch
+ changes nothing.
+ """
+ if not headlines or not get_llm_client().available:
+ return
+ cache = NewsEngine._llm_sent_cache
+ keyed = [(hashlib.sha1(h.headline.encode()).hexdigest(), h)
+ for h in headlines]
+ # Cap the batch so the reply fits the client's max_tokens — the
+ # overflow stays keyword-scored this cycle and, still uncached,
+ # goes out with the next window's batch.
+ new = [(k, h) for k, h in keyed if k not in cache][:25]
+ if new:
+ scores = self._llm_headline_scores([h.headline for _, h in new])
+ if scores is not None:
+ for (k, _), sc in zip(new, scores):
+ cache[k] = sc
+ while len(cache) > self._LLM_SENT_CACHE_MAX:
+ cache.pop(next(iter(cache)))
+ for k, h in keyed:
+ sc = cache.get(k)
+ if sc is None:
+ continue
+ overall, xau = sc
+ h.sentiment_score = overall
+ h.sentiment_by_currency = {
+ cur: (xau if cur == "XAU" else overall)
+ for cur in h.currencies_mentioned
+ }
+
+ def _llm_event_direction(self, event: EconomicEvent) -> Optional[str]:
+ """Gold direction for a released surprise, e.g. "bearish: strong
+ jobs lift yields". None → caller keeps the numeric description.
+ Cached per release; failures are NOT cached (retry next scan)."""
+ key = (event.event, event.currency, str(event.actual or ""))
+ with NewsEngine._cache_lock:
+ if key in NewsEngine._llm_event_cache:
+ return NewsEngine._llm_event_cache[key] or None
+ client = get_llm_client()
+ if not client.available:
+ return None
+ result = client.chat_json(_LLM_EVENT_SYSTEM, {
+ "event": event.event,
+ "currency": event.currency,
+ "actual": event.actual,
+ "forecast": event.forecast,
+ "previous": event.previous,
+ })
+ if not result:
+ return None
+ direction = result.get("gold_direction")
+ if direction not in ("bullish", "bearish", "neutral"):
+ return None
+ reason = str(result.get("reason", "")).strip()[:80]
+ text = "" if direction == "neutral" else (
+ f"{direction}: {reason}" if reason else direction)
+ with NewsEngine._cache_lock:
+ NewsEngine._llm_event_cache[key] = text
+ while len(NewsEngine._llm_event_cache) > self._LLM_EVENT_CACHE_MAX:
+ NewsEngine._llm_event_cache.pop(
+ next(iter(NewsEngine._llm_event_cache)))
+ return text or None
+
+ def _llm_market_summary(self, context: FundamentalContext) -> str:
+ """1-2 sentence macro blurb for alerts; "" when unavailable.
+ Cached (success OR failure) for the context-cache TTL so at most
+ one call is made per window."""
+ with NewsEngine._cache_lock:
+ ts, cached = NewsEngine._llm_summary_cache
+ if time.time() - ts < self.config.context_cache_ttl_seconds:
+ return cached
+ high_impact = [e for e in context.events_today if e.is_high_impact]
+ if not context.news_headlines and not high_impact:
+ return ""
+ client = get_llm_client()
+ if not client.available:
+ return ""
+ result = client.chat_json(_LLM_SUMMARY_SYSTEM, {
+ "headlines": [h.headline for h in context.news_headlines[:15]],
+ "events": [
+ {"event": e.event, "currency": e.currency,
+ "actual": e.actual, "forecast": e.forecast,
+ "released": not e.is_upcoming}
+ for e in high_impact[:10]
+ ],
+ })
+ summary = ""
+ if result:
+ summary = str(result.get("summary", "")).strip()[:300]
+ with NewsEngine._cache_lock:
+ NewsEngine._llm_summary_cache = (time.time(), summary)
+ return summary
+
def _extract_currencies(self, headline: str) -> List[str]:
"""Extract mentioned currencies from a headline."""
headline_lower = headline.lower()
diff --git a/signal_aggregator.py b/signal_aggregator.py
index b72974b..4bf9bba 100755
--- a/signal_aggregator.py
+++ b/signal_aggregator.py
@@ -79,6 +79,7 @@ class SignalReport:
event_description: str = ""
news_sentiment: float = 0.0
news_description: str = ""
+ news_summary: str = "" # LLM market-context blurb (display-only)
final_signal: SignalStrength = SignalStrength.NEUTRAL
final_score: float = 0.0
@@ -832,11 +833,13 @@ class SignalAggregator:
sentiment, sent_desc = news.get_pair_sentiment(symbol, context)
report.news_sentiment = sentiment
report.news_description = sent_desc
+ report.news_summary = context.llm_summary
else:
report.event_risk = "clear"
report.event_description = ""
report.news_sentiment = 0.0
report.news_description = ""
+ report.news_summary = ""
# ── 5. Spread estimate from primary data (avoids extra API call) ─
try:
diff --git a/telegram_bot.py b/telegram_bot.py
index 802c237..41813e6 100755
--- a/telegram_bot.py
+++ b/telegram_bot.py
@@ -395,6 +395,8 @@ def format_signal_message(report: SignalReport) -> str:
extra.append(f"📋 {ind_summary}")
for w in report.warnings:
extra.append(f"⚠️ {_escape_md(w)}")
+ if report.news_summary:
+ extra.append(f"📰 {_escape_md(report.news_summary)}")
if extra:
lines += [""] + extra
diff --git a/test_all.py b/test_all.py
index d552584..dc6edb4 100644
--- a/test_all.py
+++ b/test_all.py
@@ -26,7 +26,7 @@ sys.path.insert(0, os.path.dirname(__file__))
import json
import app_config
for _k in ("TELEGRAM_TOKEN", "TELEGRAM_CHAT_ID", "TELEGRAM_CHANNEL_ID",
- "TWELVE_DATA_API_KEY"):
+ "TWELVE_DATA_API_KEY", "DEEPSEEK_API_KEY"):
os.environ.pop(_k, None)
_TEST_APP = app_config.AppConfig(
provider_settings={"twelve_data": {"api_key": "test-key"}}
@@ -3044,6 +3044,42 @@ def _():
pass
+@test("app_config: deepseek block parses, validates, defaults empty")
+def _():
+ app = load_app_config(_write_json({"deepseek": {
+ "api_key": "sk-x", "model": "deepseek-v4-flash", "enabled": False,
+ "timeout_seconds": 5, "daily_request_cap": 100}}))
+ assert app.deepseek_settings["api_key"] == "sk-x"
+ assert app.deepseek_settings["enabled"] is False
+ assert load_app_config(_write_json({})).deepseek_settings == {}
+ for payload in (
+ {"deepseek": {"api_keyy": "typo"}}, # unknown key
+ {"deepseek": {"enabled": "yes"}}, # not a bool
+ {"deepseek": {"timeout_seconds": 0}}, # not positive
+ {"deepseek": {"timeout_seconds": True}}, # bool is not a number
+ {"deepseek": {"daily_request_cap": 2.5}}, # not an int
+ {"deepseek": {"model": ""}}, # empty model
+ {"deepseek": "oops"}, # not an object
+ ):
+ try:
+ load_app_config(_write_json(payload))
+ assert False, f"should have raised for {payload}"
+ except AppConfigError:
+ pass
+
+
+@test("app_config: deepseek_api_key resolves env → config.json → empty")
+def _():
+ from app_config import deepseek_api_key
+ app = load_app_config(_write_json({"deepseek": {"api_key": "json-ds"}}))
+ with patch.dict(os.environ, {}, clear=True):
+ assert deepseek_api_key(app) == "json-ds"
+ with patch.dict(os.environ, {"DEEPSEEK_API_KEY": "env-ds"}):
+ assert deepseek_api_key(app) == "env-ds"
+ with patch.dict(os.environ, {}, clear=True):
+ assert deepseek_api_key(AppConfig()) == ""
+
+
@test("config.py: hardcoded twelve_data_api_key field is gone")
def _():
assert not hasattr(PlatformConfig(), "twelve_data_api_key"), \
@@ -4657,6 +4693,278 @@ def _():
assert hits == ["M5_SELL_QUALITY", "SELL_HOUR", "TREND_EXHAUSTION"], hits
+# ══════════════════════════════════════════════════════════════════════
+# MODULE 19: DeepSeek news enrichment (llm_client + news_engine seams)
+# ══════════════════════════════════════════════════════════════════════
+
+print("\n═══ Module 19: DeepSeek news enrichment ═══")
+
+import datetime as _dt_m19
+import llm_client as lc
+from llm_client import DeepSeekClient
+from news_engine import (EconomicEvent, FundamentalContext, NewsEngine,
+ NewsSentiment)
+
+
+def _stub_resp(payload, status=200):
+ return type("R", (), {
+ "status_code": status,
+ "text": json.dumps(payload),
+ "json": lambda self: payload,
+ })()
+
+
+def _chat_body(content_obj):
+ return {"choices": [{"message": {"content": json.dumps(content_obj)}}]}
+
+
+def _counting_session(payload, status=200):
+ class S:
+ calls = 0
+
+ def post(self, url, json=None, timeout=None):
+ S.calls += 1
+ return _stub_resp(payload, status)
+ return S()
+
+
+@test("llm_client: response parsing and availability gating")
+def _():
+ parse = DeepSeekClient._parse_response
+ assert parse(_chat_body({"a": 1})) == {"a": 1}
+ assert parse({"choices": []}) is None
+ assert parse({"error": "x"}) is None
+ assert parse({"choices": [{"message": {"content": "not json"}}]}) is None
+ assert parse({"choices": [{"message": {"content": "[1,2]"}}]}) is None
+
+ # keyless / disabled clients never touch the network
+ for client in (DeepSeekClient(api_key=""),
+ DeepSeekClient(api_key="k", enabled=False)):
+ assert client.available is False
+ s = _counting_session(_chat_body({"a": 1}))
+ client._session = s
+ assert client.chat_json("sys", {"q": 1}) is None
+ assert type(s).calls == 0
+
+
+@test("llm_client: daily cap blocks requests and resets on UTC date roll")
+def _():
+ client = DeepSeekClient(api_key="k", daily_cap=2)
+ s = _counting_session(_chat_body({"ok": True}))
+ client._session = s
+ assert client.chat_json("sys", {}) == {"ok": True}
+ assert client.chat_json("sys", {}) == {"ok": True}
+ assert client.chat_json("sys", {}) is None, "third call must hit the cap"
+ assert type(s).calls == 2, "capped call must make no HTTP request"
+ # UTC date roll resets the counter
+ client._count_day = (_dt_m19.datetime.now(_dt_m19.timezone.utc).date()
+ - _dt_m19.timedelta(days=1))
+ assert client.chat_json("sys", {}) == {"ok": True}
+ assert type(s).calls == 3
+
+
+@test("llm_client: HTTP error and malformed body return None")
+def _():
+ client = DeepSeekClient(api_key="k")
+ client._session = _counting_session({"detail": "quota"}, status=429)
+ assert client.chat_json("sys", {}) is None
+ client._session = _counting_session({"unexpected": "shape"})
+ assert client.chat_json("sys", {}) is None
+
+
+class _StubLLM:
+ """Drop-in for the llm_client singleton."""
+ def __init__(self, reply):
+ self.reply = reply
+ self.calls = 0
+ available = True
+
+ def chat_json(self, system_prompt, user_payload):
+ self.calls += 1
+ return self.reply
+
+
+@test("NewsEngine: _llm_headline_scores validates, clamps, rejects mismatch")
+def _():
+ ne = NewsEngine(NewsConfig())
+ try:
+ lc._client = _StubLLM({"scores": [[5, -3], [0.2, 0.1]]})
+ assert ne._llm_headline_scores(["a", "b"]) == [(1.0, -1.0), (0.2, 0.1)]
+ lc._client = _StubLLM({"scores": [[0.1, 0.1]]})
+ assert ne._llm_headline_scores(["a", "b"]) is None, "length mismatch"
+ lc._client = _StubLLM({"scores": [["hot", 0], [0.3]]})
+ assert ne._llm_headline_scores(["a", "b"]) is None, "non-numeric/short"
+ lc._client = _StubLLM(None)
+ assert ne._llm_headline_scores(["a"]) is None, "failed call"
+ finally:
+ lc._client = None
+
+
+@test("NewsEngine: LLM sentiment overrides keyword scores per currency")
+def _():
+ now = _dt_m19.datetime.now(_dt_m19.timezone.utc)
+ ne = NewsEngine(NewsConfig())
+ NewsEngine._llm_sent_cache = {}
+ try:
+ lc._client = _StubLLM({"ok": True}) # available, unused (stubbed below)
+ h = NewsSentiment(
+ headline="Fed signals rate hike", source="s", timestamp=now,
+ currencies_mentioned=["USD", "XAU"], sentiment_score=1.0,
+ keywords=["rate hike"],
+ sentiment_by_currency={"USD": 1.0, "XAU": -1.0})
+ ne._llm_headline_scores = lambda texts: [(0.4, -0.7)] * len(texts)
+ ne._apply_llm_sentiment([h])
+ assert h.sentiment_score == 0.4
+ assert h.sentiment_by_currency == {"USD": 0.4, "XAU": -0.7}
+ assert h.keywords == ["rate hike"], "keywords stay keyword-derived"
+ finally:
+ NewsEngine._llm_sent_cache = {}
+ lc._client = None
+
+
+@test("NewsEngine: LLM failure keeps keyword sentiment untouched")
+def _():
+ now = _dt_m19.datetime.now(_dt_m19.timezone.utc)
+ ne = NewsEngine(NewsConfig())
+ NewsEngine._llm_sent_cache = {}
+ try:
+ lc._client = _StubLLM({"ok": True})
+ h = NewsSentiment(
+ headline="Fed signals rate hike", source="s", timestamp=now,
+ currencies_mentioned=["USD"], sentiment_score=0.6,
+ keywords=[], sentiment_by_currency={"USD": 0.6})
+ ne._llm_headline_scores = lambda texts: None
+ ne._apply_llm_sentiment([h])
+ assert h.sentiment_score == 0.6
+ assert h.sentiment_by_currency == {"USD": 0.6}
+ # unavailable client: not even the scores call happens
+ lc._client = None
+ ne._llm_headline_scores = lambda texts: (_ for _ in ()).throw(
+ AssertionError("must not be called when unavailable"))
+ ne._apply_llm_sentiment([h])
+ finally:
+ NewsEngine._llm_sent_cache = {}
+ lc._client = None
+
+
+@test("NewsEngine: LLM sentiment cache only sends new headlines")
+def _():
+ now = _dt_m19.datetime.now(_dt_m19.timezone.utc)
+ ne = NewsEngine(NewsConfig())
+ NewsEngine._llm_sent_cache = {}
+ sent_batches = []
+
+ def make(h):
+ return NewsSentiment(
+ headline=h, source="s", timestamp=now,
+ currencies_mentioned=["XAU"], sentiment_score=0.0,
+ keywords=[], sentiment_by_currency={"XAU": 0.0})
+
+ try:
+ lc._client = _StubLLM({"ok": True})
+
+ def scores(texts):
+ sent_batches.append(list(texts))
+ return [(0.5, 0.5)] * len(texts)
+
+ ne._llm_headline_scores = scores
+ ne._apply_llm_sentiment([make("headline one"), make("headline two")])
+ ne._apply_llm_sentiment([make("headline one"), make("headline two"),
+ make("headline three")])
+ assert sent_batches == [["headline one", "headline two"],
+ ["headline three"]], sent_batches
+ finally:
+ NewsEngine._llm_sent_cache = {}
+ lc._client = None
+
+
+@test("NewsEngine: LLM enriches surprise description, numeric fallback intact")
+def _():
+ now = _dt_m19.datetime.now(_dt_m19.timezone.utc)
+ ne = NewsEngine(NewsConfig())
+ NewsEngine._llm_event_cache = {}
+ released = EconomicEvent(
+ time=now - _dt_m19.timedelta(minutes=10), currency="USD",
+ impact="high", event="Nonfarm Payrolls",
+ forecast="200K", actual="300K")
+ ctx = FundamentalContext(events_today=[released])
+ try:
+ # failure first: description must equal today's numeric form
+ lc._client = _StubLLM(None)
+ risk, desc = ne.get_event_risk_level("XAU/USD", ctx)
+ assert risk == "caution"
+ assert desc.endswith("(actual 300K vs forecast 200K)"), desc
+ # then success: LLM direction appended to the description only
+ lc._client = _StubLLM({"gold_direction": "bearish",
+ "reason": "strong jobs lift yields"})
+ risk, desc = ne.get_event_risk_level("XAU/USD", ctx)
+ assert risk == "caution"
+ assert desc.endswith("— gold bearish: strong jobs lift yields"), desc
+ # bogus direction → numeric fallback
+ NewsEngine._llm_event_cache = {}
+ lc._client = _StubLLM({"gold_direction": "sideways"})
+ _, desc = ne.get_event_risk_level("XAU/USD", ctx)
+ assert desc.endswith("(actual 300K vs forecast 200K)"), desc
+ finally:
+ NewsEngine._llm_event_cache = {}
+ lc._client = None
+
+
+@test("NewsEngine: market summary cached across engine instances")
+def _():
+ now = _dt_m19.datetime.now(_dt_m19.timezone.utc)
+ NewsEngine._llm_summary_cache = (0.0, "")
+ ctx = FundamentalContext(news_headlines=[NewsSentiment(
+ headline="Gold steady", source="s", timestamp=now,
+ currencies_mentioned=["XAU"], sentiment_score=0.0, keywords=[])])
+ try:
+ stub = _StubLLM({"summary": "Soft dollar supports gold."})
+ lc._client = stub
+ assert NewsEngine(NewsConfig())._llm_market_summary(ctx) == \
+ "Soft dollar supports gold."
+ # second engine instance within the TTL: served from cache, no call
+ assert NewsEngine(NewsConfig())._llm_market_summary(ctx) == \
+ "Soft dollar supports gold."
+ assert stub.calls == 1
+ # failure caches "" so a broken API is retried next window, not per scan
+ NewsEngine._llm_summary_cache = (0.0, "")
+ lc._client = _StubLLM(None)
+ assert NewsEngine(NewsConfig())._llm_market_summary(ctx) == ""
+ assert NewsEngine._llm_summary_cache[1] == ""
+ finally:
+ NewsEngine._llm_summary_cache = (0.0, "")
+ lc._client = None
+
+
+@test("SignalReport.news_summary flows to Telegram, resets when news off")
+def _():
+ import telegram_bot as tb
+ config = PlatformConfig()
+ config.primary_timeframe = Timeframe.H1
+ config.confirmation_timeframes = [Timeframe.M15, Timeframe.H4]
+ agg = SignalAggregator(config)
+ dfs = {
+ Timeframe.H1: make_ohlcv(300, seed=42),
+ Timeframe.M15: make_ohlcv(300, seed=43),
+ Timeframe.H4: make_ohlcv(300, seed=44),
+ }
+ ctx = FundamentalContext(llm_summary="Test macro blurb.")
+ report = agg.analyze_from_dataframes(
+ "XAU/USD", dfs, context=ctx, use_news=True)
+ assert report.news_summary == "Test macro blurb."
+ assert "Test macro blurb." in tb.format_signal_message(report)
+ report = agg.analyze_from_dataframes(
+ "XAU/USD", dfs, context=ctx, use_news=False)
+ assert report.news_summary == ""
+
+
+@test("news defaults: LLM never gates signals (2026-07 audit lock)")
+def _():
+ cfg = NewsConfig()
+ assert cfg.event_risk_caps_signals is False
+ assert cfg.sentiment_affects_signals is False
+
+
# ══════════════════════════════════════════════════════════════════════
# SUMMARY
# ══════════════════════════════════════════════════════════════════════