dashboard: clickable news — detail dialogs + on-demand AI summary endpoint
- news_engine: RSS parser captures link/description/publisher (three new optional NewsSentiment fields); _clean_html_text/_safe_link helpers - news.json: headlines carry id = sha1(headline) + enrichment fields - static/news_summary.php (NEW): on-demand DeepSeek gold-impact summary, whitelist-gated against news.json ids, cache-forever in ai/, fail-closed file-based daily cap (150/day), key at /etc/gold-dashboard/deepseek.key; installed to the docroot by install_static - dashboard.html: newswire + calendar detail dialogs (score tiles, source description, article link, AI summary with loading/error states); delegated on stable containers, dialog keyed by id across polls - tests: +6 (357 total); docs: README-dashboard §6 one-time setup (key file, apache-owned ai/, SELinux httpd_can_network_connect), CLAUDE.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Aq1jZ6jBsBdEC2xT3u46SA
This commit is contained in:
Harwinder Sekhon
@@ -11,7 +11,7 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
python test_all.py # QA suite: 338 tests, 21 modules, no network (mocked)
|
||||
python test_all.py # QA suite: 359 tests, 22 modules, no network (mocked)
|
||||
python main.py --pair XAUUSD --timeframe H1 --detail # CLI signal report (--json for full dump)
|
||||
python telegram_bot.py # the bot (refuses to boot without credentials)
|
||||
python oanda_daemon.py # auto-execution daemon on Oanda practice account
|
||||
@@ -61,7 +61,7 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
|
||||
- **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that.
|
||||
- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
|
||||
- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`).
|
||||
- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. Deployment checklist: `deploy/README-dashboard.md`.
|
||||
- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. News/calendar rows are clickable (detail dialog); headlines carry `id` = sha1(headline) in news.json, and exactly one on-demand endpoint exists: `static/news_summary.php` (installed by `install_static`), which serves a DeepSeek gold-impact summary per headline — whitelist-gated against news.json ids, cache-forever in `ai/`, own fail-closed daily cap (150/day, file-based, separate from the publisher's), key at `/etc/gold-dashboard/deepseek.key`, never reads repo config. Deployment checklist: `deploy/README-dashboard.md`.
|
||||
- **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record.
|
||||
|
||||
## Rules for making changes (read before editing signal logic)
|
||||
|
||||
+24
-7
@@ -37,6 +37,7 @@ Out dir precedence: --out → DASHBOARD_OUT_DIR env → /var/www/trade
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import hashlib
|
||||
import logging
|
||||
import math
|
||||
import numbers
|
||||
@@ -76,6 +77,11 @@ SCAN_BUFFER_SECONDS = 10
|
||||
SNAPSHOT_FILE = "snapshot.json"
|
||||
NEWS_FILE = "news.json"
|
||||
PAGE_FILE = "dashboard.html"
|
||||
SUMMARY_PHP_FILE = "news_summary.php"
|
||||
# Repo files copied into the out dir at daemon start (git pull + restart
|
||||
# updates them). The PHP endpoint reads news.json in place, so it lives
|
||||
# next to the page.
|
||||
STATIC_FILES = [PAGE_FILE, SUMMARY_PHP_FILE]
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════
|
||||
@@ -180,9 +186,15 @@ def event_to_dict(ev) -> dict:
|
||||
|
||||
|
||||
def headline_to_dict(h) -> dict:
|
||||
# id uses the same sha1(headline) scheme as NewsEngine._llm_sent_cache;
|
||||
# it is the join key the PHP summary endpoint whitelists against.
|
||||
return {
|
||||
"id": hashlib.sha1(h.headline.encode("utf-8")).hexdigest(),
|
||||
"headline": h.headline,
|
||||
"source": h.source,
|
||||
"link": h.link,
|
||||
"description": h.description,
|
||||
"publisher": h.publisher,
|
||||
"timestamp": h.timestamp.astimezone(timezone.utc).isoformat()
|
||||
if h.timestamp else None,
|
||||
"currencies": list(h.currencies_mentioned),
|
||||
@@ -324,14 +336,19 @@ class DashboardPublisher:
|
||||
return self._news_engine
|
||||
|
||||
def install_static(self) -> None:
|
||||
"""Copy the repo's dashboard.html into the out dir (atomic), so
|
||||
git pull + restart also updates the page."""
|
||||
src = os.path.join(os.path.dirname(os.path.abspath(__file__)),
|
||||
"static", PAGE_FILE)
|
||||
"""Copy the repo's static files (page + PHP summary endpoint)
|
||||
into the out dir (atomic), so git pull + restart also updates
|
||||
them. The ai/ cache dir is NOT created here — the daemon runs as
|
||||
a different user than Apache; that dir is a one-time deploy step
|
||||
(deploy/README-dashboard.md)."""
|
||||
static_dir = os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)), "static")
|
||||
for name in STATIC_FILES:
|
||||
src = os.path.join(static_dir, name)
|
||||
if not os.path.exists(src):
|
||||
logger.warning(f"{src} missing — page not installed")
|
||||
return
|
||||
dst = os.path.join(self.out_dir, PAGE_FILE)
|
||||
logger.warning(f"{src} missing — not installed")
|
||||
continue
|
||||
dst = os.path.join(self.out_dir, name)
|
||||
shutil.copyfile(src, dst + ".tmp")
|
||||
os.replace(dst + ".tmp", dst)
|
||||
|
||||
|
||||
+66
-10
@@ -2,10 +2,12 @@
|
||||
|
||||
The dashboard is a **static publisher**: `dashboard_publisher.py` runs as its
|
||||
own daemon, scans XAU/USD at candle closes, and atomically writes
|
||||
`dashboard.html` + `snapshot.json` + `news.json` into a folder Apache already
|
||||
serves. Apache does all web serving — no new port, no Python web server, and
|
||||
the telegram bot process is completely unaffected by anything that happens
|
||||
here (see the module docstring for the independence rules).
|
||||
`dashboard.html` + `news_summary.php` + `snapshot.json` + `news.json` into a
|
||||
folder Apache already serves. Apache does all web serving — no new port, no
|
||||
Python web server, and the telegram bot process is completely unaffected by
|
||||
anything that happens here (see the module docstring for the independence
|
||||
rules). The one dynamic piece is `news_summary.php` (§6): an on-demand
|
||||
DeepSeek summary for a clicked headline, whitelist-gated against news.json.
|
||||
|
||||
All steps below run **on the live server** (`/home/Forex/ForexBot2`).
|
||||
|
||||
@@ -22,9 +24,10 @@ No new Python dependencies are required.
|
||||
## 2. Make the web folder writable
|
||||
|
||||
The dashboard publishes straight into the portal docroot `/var/www/trade/`
|
||||
(owner decision 2026-08-03) — it adds exactly three files there:
|
||||
`dashboard.html`, `snapshot.json`, `news.json`. The daemon user must be able
|
||||
to write them:
|
||||
(owner decision 2026-08-03) — it adds exactly four files there:
|
||||
`dashboard.html`, `news_summary.php`, `snapshot.json`, `news.json` (plus the
|
||||
apache-owned `ai/` cache dir from §6). The daemon user must be able to write
|
||||
them:
|
||||
|
||||
```bash
|
||||
# if the portal files can be owned by the daemon user:
|
||||
@@ -72,8 +75,9 @@ every 3 h (240 backups), same scheme as the bot.
|
||||
|
||||
## 5. Optional Apache polish
|
||||
|
||||
The page cache-busts its JSON fetches (`?t=<epoch>`), so no Apache config is
|
||||
strictly needed. To be explicit anyway:
|
||||
The page polls its JSON with conditional GETs (`cache: "no-cache"` → Apache
|
||||
answers unchanged files with 304), so no Apache config is strictly needed. To
|
||||
be explicit anyway:
|
||||
|
||||
```apache
|
||||
<Directory "/var/www/trade">
|
||||
@@ -87,15 +91,67 @@ The page is deliberately public (owner decision 2026-08-03). To restrict it
|
||||
later, add standard Apache basic auth (`htpasswd` + `Require valid-user`) on
|
||||
that directory — no code changes needed.
|
||||
|
||||
## 6. AI summary endpoint (PHP) — one-time setup
|
||||
|
||||
Clicking a newswire headline opens a detail dialog that fetches an on-demand
|
||||
DeepSeek gold-impact summary from `news_summary.php` (installed next to the
|
||||
page by the daemon). Owner decision 2026-08-07: a real on-demand endpoint on
|
||||
the public page, accepted with these guards baked into the PHP — the id must
|
||||
match a headline in the published news.json (arbitrary text can never reach
|
||||
the LLM), each headline's summary is cached forever in `ai/` (≤1 API call per
|
||||
headline, ever), and a file-based daily cap fails **closed** on I/O errors.
|
||||
|
||||
Requirements: `php` + the curl extension (`php -m | grep curl`).
|
||||
|
||||
```bash
|
||||
# 1. SELinux: allow httpd outbound HTTPS — WITHOUT this every summary call
|
||||
# dies as a curl timeout even with perfect perms (RHEL-family default).
|
||||
sudo setsebool -P httpd_can_network_connect 1
|
||||
|
||||
# 2. API key OUTSIDE the docroot, readable by Apache only:
|
||||
sudo install -d -o root -g root -m 755 /etc/gold-dashboard
|
||||
sudo sh -c 'umask 137; echo "sk-..." > /etc/gold-dashboard/deepseek.key'
|
||||
sudo chown root:apache /etc/gold-dashboard/deepseek.key # mode 640
|
||||
|
||||
# 3. apache-owned cache dir (the daemon runs as Forex and can't create it):
|
||||
sudo install -d -o apache -g apache -m 755 /var/www/trade/ai
|
||||
sudo semanage fcontext -a -t httpd_sys_rw_content_t '/var/www/trade/ai(/.*)?'
|
||||
sudo restorecon -Rv /var/www/trade/ai
|
||||
```
|
||||
|
||||
Knobs (Apache `SetEnv` in the vhost, both optional):
|
||||
`AI_SUMMARY_DAILY_CAP` (default 150), `DEEPSEEK_KEY_FILE` (default
|
||||
`/etc/gold-dashboard/deepseek.key`).
|
||||
|
||||
Verify from the shell — expect 400 / 404 / 200 / 200-from-cache:
|
||||
|
||||
```bash
|
||||
curl -s "http://<server>/news_summary.php?id=zzz" # {"error":"bad_id"}
|
||||
curl -s "http://<server>/news_summary.php?id=$(printf 'a%.0s' {1..40})" # unknown_id
|
||||
ID=$(python3 -c "import json;print(json.load(open('/var/www/trade/news.json'))['headlines'][0]['id'])")
|
||||
curl -s "http://<server>/news_summary.php?id=$ID" # real summary (2-12 s)
|
||||
curl -s "http://<server>/news_summary.php?id=$ID" # instant, from ai/<id>.json
|
||||
```
|
||||
|
||||
## Operational notes
|
||||
|
||||
- **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon
|
||||
re-copies `static/dashboard.html` into the web folder at startup.
|
||||
re-copies `static/dashboard.html` and `static/news_summary.php` into the
|
||||
web folder at startup.
|
||||
- **If the daemon dies**, systemd restarts it in 30 s; worst case the page
|
||||
shows stale data with its own "updated N min ago" evidence. Apache and the
|
||||
telegram bot are unaffected.
|
||||
- **DeepSeek**: the publisher process has its own daily request cap
|
||||
(`deepseek.daily_request_cap`, default 500) separate from the bot's —
|
||||
expect roughly up to ~430 extra calls/day worst case, V4-Flash pricing.
|
||||
- **Two independent DeepSeek meters** hit the same API key: the publisher's
|
||||
in-process cap above, and the PHP endpoint's file cap (`ai/.ht_cap.json`,
|
||||
default 150/day). They don't communicate — worst-case spend is their sum;
|
||||
DeepSeek-side spend alerting is the true backstop. Kept separate on
|
||||
purpose (independence rule: the PHP never imports/reads bot code or
|
||||
config.json).
|
||||
- **Restart window**: for the first minutes after a daemon restart the new
|
||||
page can meet a pre-upgrade news.json without headline ids — those rows
|
||||
simply render non-clickable until the first news publish.
|
||||
- **Credentials**: uses the same env/.env/config.json resolution as the rest
|
||||
of the platform (Oanda candles feed; news needs no key; DeepSeek optional).
|
||||
|
||||
@@ -9,6 +9,7 @@ A perfect technical setup can be destroyed by an unexpected NFP print.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import html
|
||||
import logging
|
||||
import re
|
||||
import threading
|
||||
@@ -103,6 +104,44 @@ def _parse_release_value(raw: Optional[str]) -> Optional[float]:
|
||||
return float(match.group(1)) * multiplier
|
||||
|
||||
|
||||
_DESC_MAX_CHARS = 500
|
||||
_MAX_LINK_CHARS = 2048
|
||||
|
||||
|
||||
def _clean_html_text(raw: Optional[str],
|
||||
limit: int = _DESC_MAX_CHARS) -> Optional[str]:
|
||||
"""RSS <description> → plain text: unescape entities, strip tags,
|
||||
collapse whitespace, truncate. Unescape and strip both run twice
|
||||
because feeds routinely double-encode (&lt;p&gt; wrapping
|
||||
real markup) — the second unescape can reveal fresh tags."""
|
||||
if not raw:
|
||||
return None
|
||||
text = html.unescape(raw)
|
||||
text = re.sub(r"<[^>]+>", " ", text)
|
||||
text = html.unescape(text)
|
||||
text = re.sub(r"<[^>]+>", " ", text)
|
||||
text = re.sub(r"\s+", " ", text).strip()
|
||||
if not text:
|
||||
return None
|
||||
if len(text) > limit:
|
||||
text = text[:limit].rstrip() + "…"
|
||||
return text
|
||||
|
||||
|
||||
def _safe_link(raw: Optional[str]) -> Optional[str]:
|
||||
"""Article URL for display: http(s) only (an RSS <link> is untrusted
|
||||
and lands in an href), bounded length (Google News redirect URLs run
|
||||
1-2 KB; anything past that is bloat, not a link)."""
|
||||
if not raw:
|
||||
return None
|
||||
link = raw.strip()
|
||||
if not link.lower().startswith(("http://", "https://")):
|
||||
return None
|
||||
if len(link) > _MAX_LINK_CHARS:
|
||||
return None
|
||||
return link
|
||||
|
||||
|
||||
@dataclass
|
||||
class EconomicEvent:
|
||||
"""Single economic calendar event."""
|
||||
@@ -155,6 +194,11 @@ class NewsSentiment:
|
||||
# Per-currency score: monetary-policy keywords flip sign for XAU
|
||||
# (hawkish is dollar-bullish but gold-bearish).
|
||||
sentiment_by_currency: Dict[str, float] = field(default_factory=dict)
|
||||
# Display-only enrichment (dashboard detail dialog). Optional with
|
||||
# defaults so existing constructor calls keep working.
|
||||
link: Optional[str] = None # article URL (Google News = redirect)
|
||||
description: Optional[str] = None # plain text, tags stripped, ≤500 chars
|
||||
publisher: Optional[str] = None # Google News <source> text, else None
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -742,6 +786,21 @@ class NewsEngine:
|
||||
currencies = self._extract_currencies(title)
|
||||
|
||||
if currencies:
|
||||
link_el = item.find("link")
|
||||
desc_el = item.find("description")
|
||||
src_el = item.find("source")
|
||||
# itertext() survives CDATA and stray child
|
||||
# elements where .text stops at the first child.
|
||||
desc = _clean_html_text(
|
||||
"".join(desc_el.itertext())
|
||||
if desc_el is not None else None)
|
||||
# Google News descriptions are often just the
|
||||
# headline again inside an anchor — worthless.
|
||||
if desc and desc.strip() == title.strip():
|
||||
desc = None
|
||||
publisher = None
|
||||
if src_el is not None:
|
||||
publisher = (src_el.text or "").strip() or None
|
||||
headlines.append(NewsSentiment(
|
||||
headline=title,
|
||||
source=feed_url,
|
||||
@@ -753,6 +812,10 @@ class NewsEngine:
|
||||
cur: self._analyze_headline_sentiment(title, cur)
|
||||
for cur in currencies
|
||||
},
|
||||
link=_safe_link(
|
||||
link_el.text if link_el is not None else None),
|
||||
description=desc,
|
||||
publisher=publisher,
|
||||
))
|
||||
|
||||
ok_feeds += 1
|
||||
|
||||
+212
-11
@@ -322,8 +322,8 @@ details.more .reasons { margin-top: 4px; }
|
||||
.rail-chip { cursor: pointer; }
|
||||
.rail-chip:hover, .rail-chip:focus-visible { border-color: var(--accent); }
|
||||
|
||||
/* ── Timeframe detail dialog ──────────────────────────────────── */
|
||||
#tf-detail {
|
||||
/* ── Detail dialogs (timeframe + news) ────────────────────────── */
|
||||
#tf-detail, #news-detail {
|
||||
/* the global `* { margin: 0 }` reset wipes the UA stylesheet's `margin: auto`,
|
||||
which is what centres a modal dialog — restore it explicitly. */
|
||||
position: fixed; inset: 0; margin: auto;
|
||||
@@ -333,9 +333,10 @@ details.more .reasons { margin-top: 4px; }
|
||||
background: var(--card); color: var(--ink);
|
||||
font-family: var(--sans); font-size: 16px;
|
||||
}
|
||||
#tf-detail::backdrop { background: rgba(var(--ink-rgb),0.5); }
|
||||
#tf-detail[open] { display: flex; flex-direction: column; }
|
||||
#detail-head {
|
||||
#news-detail { width: min(680px, 94vw); }
|
||||
#tf-detail::backdrop, #news-detail::backdrop { background: rgba(var(--ink-rgb),0.5); }
|
||||
#tf-detail[open], #news-detail[open] { display: flex; flex-direction: column; }
|
||||
#detail-head, #nd-head {
|
||||
flex: 0 0 auto;
|
||||
display: flex; align-items: center; gap: 10px; flex-wrap: wrap;
|
||||
padding: 14px 18px; background: var(--card2);
|
||||
@@ -347,13 +348,17 @@ details.more .reasons { margin-top: 4px; }
|
||||
}
|
||||
#detail-head .d-price { font-family: var(--mono); font-size: 19px; font-weight: 600; }
|
||||
#detail-head .d-sub { font-size: 13px; color: var(--muted); }
|
||||
#detail-close {
|
||||
#detail-close, #nd-close {
|
||||
margin-left: auto; font: inherit; font-size: 15px; line-height: 1;
|
||||
padding: 7px 13px; border-radius: 6px; cursor: pointer;
|
||||
background: var(--card); color: var(--ink2); border: 1px solid var(--line-strong);
|
||||
}
|
||||
#detail-close:hover { border-color: var(--accent); color: var(--accent-bright); }
|
||||
#detail-body { flex: 1 1 auto; min-height: 0; overflow-y: auto; padding: 4px 18px 20px; }
|
||||
#detail-close:hover, #nd-close:hover { border-color: var(--accent); color: var(--accent-bright); }
|
||||
#detail-body, #nd-body { flex: 1 1 auto; min-height: 0; overflow-y: auto; padding: 4px 18px 20px; }
|
||||
#nd-title {
|
||||
font-size: 12px; letter-spacing: 0.12em; text-transform: uppercase;
|
||||
color: var(--muted); font-weight: 600;
|
||||
}
|
||||
|
||||
.d-sec { margin-top: 20px; }
|
||||
.d-sec > h3 {
|
||||
@@ -435,6 +440,12 @@ details.more .reasons { margin-top: 4px; }
|
||||
.d-note b { color: var(--ink); }
|
||||
.sub-h { font-size: 12px; color: var(--muted); font-weight: 600; margin: 11px 0 5px; }
|
||||
|
||||
/* news detail dialog */
|
||||
.nd-headline { font-size: 18px; font-weight: 650; line-height: 1.35; margin-top: 14px; }
|
||||
.nd-meta { font-size: 12.5px; color: var(--muted); margin-top: 5px; }
|
||||
a.d-action { display: block; text-decoration: none; color: var(--accent-bright); }
|
||||
a.d-action:hover { border-color: var(--accent); }
|
||||
|
||||
/* ── News section ─────────────────────────────────────────────── */
|
||||
#news-wrap { display: grid; gap: 14px; grid-template-columns: minmax(0, 3fr) minmax(0, 2fr); }
|
||||
@media (max-width: 1100px) { #news-wrap { grid-template-columns: 1fr; } }
|
||||
@@ -467,6 +478,8 @@ details.more .reasons { margin-top: 4px; }
|
||||
}
|
||||
#cal-table tr.past { opacity: 0.55; }
|
||||
#cal-table tr.next-up td:first-child { box-shadow: inset 3px 0 0 var(--accent); padding-left: 7px; }
|
||||
#cal-table tr[data-ev-key] { cursor: pointer; }
|
||||
#cal-table tr[data-ev-key]:hover td { background: var(--accent-wash); }
|
||||
#cal-table .t { white-space: nowrap; color: var(--ink2); }
|
||||
#cal-table .cur { font-family: var(--mono); color: var(--muted); }
|
||||
#cal-table .ev { color: var(--ink); }
|
||||
@@ -514,6 +527,8 @@ details.more .reasons { margin-top: 4px; }
|
||||
padding: 7px 0; border-bottom: 1px solid var(--line);
|
||||
}
|
||||
.wire-item:last-child { border-bottom: none; }
|
||||
.wire-item[data-news-id] { cursor: pointer; padding: 7px 6px; margin: 0 -6px; border-radius: 6px; }
|
||||
.wire-item[data-news-id]:hover { background: var(--accent-wash); }
|
||||
.wire-score {
|
||||
font-family: var(--mono); font-size: 13px; font-weight: 700;
|
||||
padding: 2px 7px; border-radius: 5px; white-space: nowrap;
|
||||
@@ -569,6 +584,14 @@ footer {
|
||||
<div id="detail-body"></div>
|
||||
</dialog>
|
||||
|
||||
<dialog id="news-detail" aria-labelledby="nd-title">
|
||||
<div id="nd-head">
|
||||
<span id="nd-title">News detail</span>
|
||||
<button id="nd-close" type="button" aria-label="Close news detail">Close ✕</button>
|
||||
</div>
|
||||
<div id="nd-body"></div>
|
||||
</dialog>
|
||||
|
||||
<div class="eyebrow">Fundamentals</div>
|
||||
<div id="news-wrap">
|
||||
<section class="panel">
|
||||
@@ -1056,7 +1079,8 @@ function renderCalendar() {
|
||||
? `<span class="surp ${e.surprise}" title="actual ${e.surprise === "positive" ? "above" :
|
||||
e.surprise === "negative" ? "below" : "matches"} forecast (numeric only — not gold direction)">${
|
||||
e.surprise === "positive" ? "▲" : e.surprise === "negative" ? "▼" : "="}</span>` : "";
|
||||
rows += `<tr class="${past ? "past" : ""}${nextUp ? " next-up" : ""}">
|
||||
rows += `<tr class="${past ? "past" : ""}${nextUp ? " next-up" : ""}"
|
||||
data-ev-key="${esc(evKey(e))}" tabindex="0" role="button" aria-haspopup="dialog">
|
||||
<td class="t num">${esc(time)}</td>
|
||||
<td class="cur">${esc(e.currency)}</td>
|
||||
<td><span class="imp ${esc(e.impact)}">${e.impact === "medium" ? "Med" :
|
||||
@@ -1109,11 +1133,14 @@ function renderWire() {
|
||||
const scoreTxt = s == null ? "·" : (s > 0 ? "+" : "") + s.toFixed(1);
|
||||
let host = "";
|
||||
try { host = new URL(h.source).hostname.replace(/^www\./, ""); } catch {}
|
||||
return `<div class="wire-item">
|
||||
/* data-news-id only when the payload carries an id — after a daemon
|
||||
restart the new page briefly meets a pre-upgrade news.json */
|
||||
const open = h.id ? ` data-news-id="${h.id}" tabindex="0" role="button" aria-haspopup="dialog"` : "";
|
||||
return `<div class="wire-item"${open}>
|
||||
<span class="wire-score ${cls}" title="Headline sentiment ${esc(String(s ?? "n/a"))} (−1…+1)${
|
||||
h.currencies?.length ? " · " + esc(h.currencies.join(", ")) : ""}">${scoreTxt}</span>
|
||||
<span class="wire-h">${esc(h.headline)}</span>
|
||||
<span class="wire-meta">${esc(host)}${h.timestamp ? " · " + fmtAge(h.timestamp) : ""}</span></div>`;
|
||||
<span class="wire-meta">${esc(h.publisher || host)}${h.timestamp ? " · " + fmtAge(h.timestamp) : ""}</span></div>`;
|
||||
};
|
||||
const head = items.slice(0, 12).map(item).join("");
|
||||
const rest = items.slice(12);
|
||||
@@ -1122,6 +1149,179 @@ function renderWire() {
|
||||
<div class="wire">${rest.map(item).join("")}</div></details>` : "");
|
||||
}
|
||||
|
||||
/* ── news / calendar detail dialog ────────────────────────────── */
|
||||
const ndlg = $("news-detail");
|
||||
let ndState = null; // {kind:"wire", id, snap} | {kind:"cal", key, snap}
|
||||
const aiMemo = new Map(); // headline id -> {summary, generated_at} (successes only)
|
||||
const aiErr = new Map(); // headline id -> error code (cleared on reopen → retry)
|
||||
const aiPending = new Set();
|
||||
|
||||
/* stable across polls — the array index is not (new headlines shift it) */
|
||||
const evKey = e => `${e.time}|${e.currency}|${e.event}`;
|
||||
const findHeadline = id => (news?.headlines || []).find(h => h.id === id);
|
||||
const findEvent = key => (news?.events || []).find(e => evKey(e) === key);
|
||||
|
||||
const AI_ERR_MSG = {
|
||||
unknown_id: "This headline has left the live feed — no summary available.",
|
||||
cap_exceeded: "Daily AI budget reached — try again tomorrow.",
|
||||
no_key: "AI summaries are not configured on this server.",
|
||||
};
|
||||
|
||||
async function fetchAiSummary(id) {
|
||||
aiPending.add(id);
|
||||
try {
|
||||
const res = await fetch("news_summary.php?id=" + encodeURIComponent(id));
|
||||
const data = await res.json().catch(() => null);
|
||||
if (res.ok && data?.summary) aiMemo.set(id, data);
|
||||
else aiErr.set(id, data?.error || `http_${res.status}`);
|
||||
} catch {
|
||||
aiErr.set(id, "network");
|
||||
} finally {
|
||||
aiPending.delete(id);
|
||||
if (ndState?.kind === "wire" && ndState.id === id) renderNewsDlg();
|
||||
}
|
||||
}
|
||||
|
||||
function scoreTiles(h) {
|
||||
const cls = v => v > 0.05 ? "buy" : v < -0.05 ? "sell" : "";
|
||||
let tiles = tile("Overall sentiment", signed(h.score, 2), cls(h.score));
|
||||
const xau = h.score_by_currency?.XAU;
|
||||
if (xau != null) tiles += tile("Gold (XAU) effect", signed(xau, 2), cls(xau));
|
||||
for (const [cur, v] of Object.entries(h.score_by_currency || {})) {
|
||||
if (cur !== "XAU") tiles += tile(cur, signed(v, 2), cls(v));
|
||||
}
|
||||
let out = `<div class="d-stats">${tiles}</div>`;
|
||||
if (h.keywords?.length) {
|
||||
out += `<div class="nd-meta">Matched keywords: ${esc(h.keywords.join(", "))}</div>`;
|
||||
}
|
||||
out += `<div class="nd-meta">Scores range −1 (bearish) to +1 (bullish); the gold
|
||||
score flips monetary-policy keywords (hawkish = dollar-bullish, gold-bearish).</div>`;
|
||||
return out;
|
||||
}
|
||||
|
||||
function aiSection(id) {
|
||||
if (aiMemo.has(id)) {
|
||||
const m = aiMemo.get(id);
|
||||
return `<div class="d-note">${esc(m.summary)}</div>` + (m.generated_at
|
||||
? `<div class="nd-meta">generated ${fmtAge(m.generated_at)}</div>` : "");
|
||||
}
|
||||
if (aiPending.has(id)) return `<div class="d-note">Generating gold-impact summary…</div>`;
|
||||
if (aiErr.has(id)) {
|
||||
return `<div class="d-note">${esc(AI_ERR_MSG[aiErr.get(id)] || "AI summary unavailable right now.")}</div>`;
|
||||
}
|
||||
return `<div class="d-note">Generating gold-impact summary…</div>`;
|
||||
}
|
||||
|
||||
function wireDlgHTML() {
|
||||
const fresh = findHeadline(ndState.id);
|
||||
const h = fresh || ndState.snap;
|
||||
if (!h) return "";
|
||||
let host = "";
|
||||
try { host = new URL(h.source).hostname.replace(/^www\./, ""); } catch {}
|
||||
const when = h.timestamp
|
||||
? `${new Date(h.timestamp).toLocaleString(undefined,
|
||||
{dateStyle: "medium", timeStyle: "short"})} · ${fmtAge(h.timestamp)}` : "";
|
||||
let out = `<div class="nd-headline">${esc(h.headline)}</div>
|
||||
<div class="nd-meta">${esc(h.publisher || host)}${when ? " · " + when : ""}</div>`;
|
||||
if (!fresh) out += `<div class="nd-meta">No longer in the live feed — showing saved copy.</div>`;
|
||||
out += sec("Score", scoreTiles(h));
|
||||
if (h.description) out += sec("From the source", `<div class="d-note">${esc(h.description)}</div>`);
|
||||
out += sec("AI gold-impact summary", aiSection(ndState.id));
|
||||
if (h.link && /^https?:\/\//.test(h.link)) {
|
||||
out += `<a class="d-action" href="${esc(h.link)}" target="_blank"
|
||||
rel="noopener noreferrer">Read full article ↗</a>`;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function calDlgHTML() {
|
||||
const fresh = findEvent(ndState.key);
|
||||
const e = fresh || ndState.snap;
|
||||
if (!e) return "";
|
||||
const t = Date.parse(e.time) / 1000;
|
||||
const when = new Date(e.time).toLocaleString(undefined, {dateStyle: "full", timeStyle: "short"});
|
||||
const impact = `<span class="imp ${esc(e.impact)}">${e.impact === "medium" ? "Med"
|
||||
: e.impact.charAt(0).toUpperCase() + e.impact.slice(1)}</span>`;
|
||||
let out = `<div class="nd-headline">${esc(e.event)}</div>
|
||||
<div class="nd-meta">${esc(e.currency)} · ${when}</div>`;
|
||||
/* live countdown only for FUTURE events — tick()'s modulo wrap on a
|
||||
passed epoch would show a bogus restarting timer for released ones */
|
||||
const timing = t > nowServer()
|
||||
? `releases in <span class="num" data-close="${t}">--:--</span>`
|
||||
: `released ${fmtAge(e.time)}`;
|
||||
out += sec("Timing", `<div class="d-note">${impact} impact · ${timing}</div>`);
|
||||
const surpTxt = {positive: "actual above forecast", negative: "actual below forecast",
|
||||
inline: "actual matches forecast"}[e.surprise];
|
||||
out += sec("Numbers", `<div class="d-stats">${
|
||||
tile("Forecast", esc(e.forecast ?? "—")) +
|
||||
tile("Previous", esc(e.previous ?? "—")) +
|
||||
tile("Actual", esc(e.actual ?? "—"),
|
||||
e.surprise === "positive" ? "buy" : e.surprise === "negative" ? "sell" : "")
|
||||
}</div>` + (surpTxt
|
||||
? `<div class="nd-meta">${surpTxt} (numeric only — not gold direction)</div>` : ""));
|
||||
return out;
|
||||
}
|
||||
|
||||
function renderNewsDlg() {
|
||||
if (!ndState || !ndlg.open) return;
|
||||
const scroll = $("nd-body").scrollTop;
|
||||
$("nd-title").textContent = ndState.kind === "wire" ? "Newswire" : "Economic calendar";
|
||||
$("nd-body").innerHTML = ndState.kind === "wire" ? wireDlgHTML() : calDlgHTML();
|
||||
$("nd-body").scrollTop = scroll;
|
||||
tick();
|
||||
}
|
||||
|
||||
function openNewsDetail(id) {
|
||||
const h = findHeadline(id);
|
||||
if (!h) return;
|
||||
ndState = {kind: "wire", id, snap: h};
|
||||
aiErr.delete(id); // failures retry on reopen; successes are memoised
|
||||
if (!ndlg.open) ndlg.showModal();
|
||||
if (!aiMemo.has(id) && !aiPending.has(id)) fetchAiSummary(id);
|
||||
renderNewsDlg();
|
||||
$("nd-body").scrollTop = 0;
|
||||
$("nd-close").focus();
|
||||
}
|
||||
|
||||
function openCalDetail(key) {
|
||||
const e = findEvent(key);
|
||||
if (!e) return;
|
||||
ndState = {kind: "cal", key, snap: e};
|
||||
if (!ndlg.open) ndlg.showModal();
|
||||
renderNewsDlg();
|
||||
$("nd-body").scrollTop = 0;
|
||||
$("nd-close").focus();
|
||||
}
|
||||
|
||||
$("nd-close").addEventListener("click", () => ndlg.close());
|
||||
ndlg.addEventListener("close", () => { ndState = null; });
|
||||
ndlg.addEventListener("click", e => {
|
||||
if (e.target !== ndlg) return;
|
||||
const b = ndlg.getBoundingClientRect();
|
||||
if (e.clientX < b.left || e.clientX > b.right || e.clientY < b.top || e.clientY > b.bottom) ndlg.close();
|
||||
});
|
||||
|
||||
/* Delegated on the stable containers (renderWire/renderCalendar replace
|
||||
innerHTML every poll). Deliberately NOT wireOpeners(): its details/
|
||||
summary guard would swallow the 28 wire items nested in the
|
||||
"more headlines" <details>. A click on that toggle itself finds no
|
||||
[data-news-id] ancestor and falls through to the native behaviour. */
|
||||
function wireNewsOpeners(host, attr, open) {
|
||||
host.addEventListener("click", e => {
|
||||
const el = e.target.closest(`[${attr}]`);
|
||||
if (el) open(el.getAttribute(attr));
|
||||
});
|
||||
host.addEventListener("keydown", e => {
|
||||
if (e.key !== "Enter" && e.key !== " ") return;
|
||||
const el = e.target.closest?.(`[${attr}]`);
|
||||
if (!el || e.target !== el) return; // let inner controls keep their own keys
|
||||
e.preventDefault();
|
||||
open(el.getAttribute(attr));
|
||||
});
|
||||
}
|
||||
wireNewsOpeners($("wire"), "data-news-id", openNewsDetail);
|
||||
wireNewsOpeners($("cal-body"), "data-ev-key", openCalDetail);
|
||||
|
||||
/* ── polling ──────────────────────────────────────────────────── */
|
||||
async function fetchJSON(name) {
|
||||
/* no-cache = revalidate with the server every time (If-Modified-Since);
|
||||
@@ -1150,6 +1350,7 @@ async function poll() {
|
||||
$("banner-fail").style.display = "none";
|
||||
renderHero(); renderContext(); renderCards(); renderCalendar(); renderWire();
|
||||
renderDetail(); // keep an open detail view in sync with the new scan
|
||||
renderNewsDlg(); // same for an open news/calendar dialog
|
||||
tick(); // countdowns appear immediately, not after the first 1 s tick
|
||||
|
||||
/* one-shot refetch shortly after the next expected publish, so a
|
||||
|
||||
@@ -0,0 +1,245 @@
|
||||
<?php
|
||||
/**
|
||||
* On-demand gold-impact AI summary for one published headline.
|
||||
*
|
||||
* The ONLY dynamic piece of the otherwise-static dashboard. Serves
|
||||
* GET ?id=<sha1-of-headline> and returns {"summary": ..., "generated_at": ...}.
|
||||
*
|
||||
* Abuse model (the page is public):
|
||||
* - WHITELIST: the id must match a headline in the local news.json —
|
||||
* arbitrary text can never reach the LLM through this endpoint.
|
||||
* - CACHE FOREVER: headlines are immutable, so each id costs at most
|
||||
* one DeepSeek call, ever (ai/<id>.json). Cache is checked BEFORE
|
||||
* the whitelist on purpose: summaries stay reachable after the
|
||||
* headline ages out of the 24 h feed window.
|
||||
* - DAILY CAP: file-based UTC-day counter, FAIL CLOSED on counter I/O
|
||||
* errors (a perms mistake must not mean unmetered spend).
|
||||
*
|
||||
* DeepSeek doctrine (mirrors llm_client.py — deliberate local copy, the
|
||||
* dashboard never imports bot code and this file never reads the repo's
|
||||
* config.json):
|
||||
* - STATIC byte-identical system prompt (prefix-cache pricing).
|
||||
* - "thinking": disabled is MANDATORY for V4-Flash (else it burns the
|
||||
* timeout reasoning), temperature 0, json response_format.
|
||||
* - One attempt, no retries; every failure returns a JSON error the
|
||||
* page renders as a graceful fallback.
|
||||
*
|
||||
* Key file lives OUTSIDE the docroot: /etc/gold-dashboard/deepseek.key
|
||||
* (root:apache 640). Override paths/caps via Apache SetEnv:
|
||||
* DEEPSEEK_KEY_FILE, AI_SUMMARY_DAILY_CAP.
|
||||
*/
|
||||
|
||||
const NEWS_FILE = __DIR__ . '/news.json';
|
||||
const CACHE_DIR = __DIR__ . '/ai'; // apache-owned; created at deploy time
|
||||
const API_URL = 'https://api.deepseek.com/chat/completions';
|
||||
const MODEL = 'deepseek-v4-flash';
|
||||
const TIMEOUT = 12;
|
||||
const MAX_TOKENS = 200;
|
||||
const SUMMARY_MAX_CHARS = 400;
|
||||
|
||||
// .ht_ prefix: Apache's default <Files ".ht*"> rule denies these even
|
||||
// though they sit in the public docroot.
|
||||
const CAP_FILE = CACHE_DIR . '/.ht_cap.json';
|
||||
|
||||
const SYSTEM_PROMPT = <<<'PROMPT'
|
||||
You are a forex/gold news analyst. You receive one news item as JSON:
|
||||
{"headline", "description", "score", "xau_score", "keywords", "currencies"}.
|
||||
"xau_score" is the platform's automated gold-effect score for this item,
|
||||
-1.0 (gold down) to 1.0 (gold up); "score" is overall market sentiment.
|
||||
Both are automated and can be wrong — judge the news yourself.
|
||||
In 1-2 plain-English sentences (max 45 words) explain what this news
|
||||
means for the GOLD (XAU/USD) price and WHY it is bullish, bearish, or
|
||||
neutral. If your reading clearly contradicts xau_score, add "(the
|
||||
automated score disagrees)". Never write the words "xau_score" or
|
||||
"score" — a general reader sees this text.
|
||||
Gold doctrine: hawkish Fed / rate hikes / rising real yields / strong
|
||||
dollar are gold-bearish; easing / rate cuts / safe-haven flows /
|
||||
geopolitical stress / weak dollar are gold-bullish.
|
||||
Reply with ONLY JSON: {"summary": "..."}
|
||||
PROMPT;
|
||||
|
||||
function respond(int $status, array $obj, string $cache = 'no-store'): void
|
||||
{
|
||||
http_response_code($status);
|
||||
header('Content-Type: application/json; charset=utf-8');
|
||||
header('Cache-Control: ' . $cache);
|
||||
echo json_encode($obj, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
|
||||
exit;
|
||||
}
|
||||
|
||||
function atomic_write(string $path, string $data): bool
|
||||
{
|
||||
$tmp = $path . '.tmp.' . getmypid();
|
||||
if (@file_put_contents($tmp, $data) === false) {
|
||||
return false;
|
||||
}
|
||||
if (!@rename($tmp, $path)) {
|
||||
@unlink($tmp);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ── 1. id validation ────────────────────────────────────────────────
|
||||
$id = $_GET['id'] ?? '';
|
||||
if (!preg_match('/^[a-f0-9]{40}$/', $id)) {
|
||||
respond(400, ['error' => 'bad_id']);
|
||||
}
|
||||
|
||||
// ── 2. cache first (immutable headlines → summaries never change) ───
|
||||
$cachePath = CACHE_DIR . '/' . $id . '.json';
|
||||
$cached = @file_get_contents($cachePath);
|
||||
if ($cached !== false && $cached !== '') {
|
||||
http_response_code(200);
|
||||
header('Content-Type: application/json; charset=utf-8');
|
||||
header('Cache-Control: max-age=3600');
|
||||
echo $cached;
|
||||
exit;
|
||||
}
|
||||
|
||||
// ── 3. whitelist against the published feed ─────────────────────────
|
||||
$news = json_decode((string)@file_get_contents(NEWS_FILE), true);
|
||||
if (!is_array($news)) {
|
||||
respond(503, ['error' => 'no_feed']);
|
||||
}
|
||||
$hit = null;
|
||||
foreach ($news['headlines'] ?? [] as $h) {
|
||||
// Fallback sha1(): a publisher that predates the id field still
|
||||
// serves a valid whitelist during the restart window.
|
||||
$hid = $h['id'] ?? sha1($h['headline'] ?? '');
|
||||
if ($hid === $id) {
|
||||
$hit = $h;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ($hit === null) {
|
||||
respond(404, ['error' => 'unknown_id',
|
||||
'message' => 'Headline is no longer in the feed.']);
|
||||
}
|
||||
|
||||
// ── 4. daily cap (fail closed) ──────────────────────────────────────
|
||||
$dailyCap = (int)(getenv('AI_SUMMARY_DAILY_CAP') ?: 150);
|
||||
$capFh = @fopen(CAP_FILE, 'c+');
|
||||
if ($capFh === false || !flock($capFh, LOCK_EX)) {
|
||||
respond(503, ['error' => 'cap_io']);
|
||||
}
|
||||
$today = gmdate('Y-m-d');
|
||||
$cap = json_decode((string)stream_get_contents($capFh), true);
|
||||
if (!is_array($cap) || ($cap['day'] ?? '') !== $today) {
|
||||
$cap = ['day' => $today, 'count' => 0];
|
||||
}
|
||||
if ($cap['count'] >= $dailyCap) {
|
||||
flock($capFh, LOCK_UN);
|
||||
fclose($capFh);
|
||||
respond(429, ['error' => 'cap_exceeded']);
|
||||
}
|
||||
// Increment BEFORE the LLM call — attempts count against the cap
|
||||
// (same semantics as llm_client._under_cap).
|
||||
$cap['count']++;
|
||||
rewind($capFh);
|
||||
if (ftruncate($capFh, 0) === false
|
||||
|| fwrite($capFh, json_encode($cap)) === false) {
|
||||
flock($capFh, LOCK_UN);
|
||||
fclose($capFh);
|
||||
respond(503, ['error' => 'cap_io']);
|
||||
}
|
||||
fflush($capFh);
|
||||
flock($capFh, LOCK_UN);
|
||||
fclose($capFh);
|
||||
|
||||
// ── 5. API key ──────────────────────────────────────────────────────
|
||||
$keyFile = getenv('DEEPSEEK_KEY_FILE') ?: '/etc/gold-dashboard/deepseek.key';
|
||||
$apiKey = trim((string)@file_get_contents($keyFile));
|
||||
if ($apiKey === '') {
|
||||
respond(503, ['error' => 'no_key']);
|
||||
}
|
||||
|
||||
// ── 6. per-id stampede lock, then re-check the cache ────────────────
|
||||
$lockFh = @fopen(CACHE_DIR . '/.ht_lock_' . $id, 'c');
|
||||
if ($lockFh !== false) {
|
||||
flock($lockFh, LOCK_EX); // blocks while a concurrent winner generates
|
||||
$cached = @file_get_contents($cachePath);
|
||||
if ($cached !== false && $cached !== '') {
|
||||
flock($lockFh, LOCK_UN);
|
||||
fclose($lockFh);
|
||||
http_response_code(200);
|
||||
header('Content-Type: application/json; charset=utf-8');
|
||||
header('Cache-Control: max-age=3600');
|
||||
echo $cached;
|
||||
exit;
|
||||
}
|
||||
}
|
||||
|
||||
// ── 7. DeepSeek call (exact chat_json body shape) ───────────────────
|
||||
$userPayload = json_encode([
|
||||
'headline' => $hit['headline'] ?? '',
|
||||
'description' => $hit['description'] ?? null,
|
||||
'score' => $hit['score'] ?? null,
|
||||
'xau_score' => $hit['score_by_currency']['XAU'] ?? null,
|
||||
'keywords' => $hit['keywords'] ?? [],
|
||||
'currencies' => $hit['currencies'] ?? [],
|
||||
], JSON_UNESCAPED_UNICODE);
|
||||
|
||||
$body = json_encode([
|
||||
'model' => MODEL,
|
||||
'messages' => [
|
||||
['role' => 'system', 'content' => SYSTEM_PROMPT],
|
||||
['role' => 'user', 'content' => $userPayload],
|
||||
],
|
||||
'response_format' => ['type' => 'json_object'],
|
||||
'temperature' => 0,
|
||||
'thinking' => ['type' => 'disabled'],
|
||||
'max_tokens' => MAX_TOKENS,
|
||||
]);
|
||||
|
||||
$ch = curl_init(API_URL);
|
||||
curl_setopt_array($ch, [
|
||||
CURLOPT_POST => true,
|
||||
CURLOPT_POSTFIELDS => $body,
|
||||
CURLOPT_HTTPHEADER => ['Content-Type: application/json',
|
||||
'Authorization: Bearer ' . $apiKey],
|
||||
CURLOPT_RETURNTRANSFER => true,
|
||||
CURLOPT_TIMEOUT => TIMEOUT,
|
||||
]);
|
||||
$raw = curl_exec($ch);
|
||||
$httpCode = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
|
||||
curl_close($ch);
|
||||
|
||||
// ── 8. parse (any failure → 502, never cached) ──────────────────────
|
||||
$summary = null;
|
||||
if ($raw !== false && $httpCode === 200) {
|
||||
$resp = json_decode($raw, true);
|
||||
$content = $resp['choices'][0]['message']['content'] ?? '';
|
||||
$parsed = json_decode((string)$content, true);
|
||||
if (is_array($parsed) && is_string($parsed['summary'] ?? null)
|
||||
&& trim($parsed['summary']) !== '') {
|
||||
$summary = trim($parsed['summary']);
|
||||
if (mb_strlen($summary) > SUMMARY_MAX_CHARS) {
|
||||
$summary = mb_substr($summary, 0, SUMMARY_MAX_CHARS) . '…';
|
||||
}
|
||||
}
|
||||
}
|
||||
if ($lockFh !== false && $summary === null) {
|
||||
flock($lockFh, LOCK_UN);
|
||||
fclose($lockFh);
|
||||
}
|
||||
if ($summary === null) {
|
||||
respond(502, ['error' => 'llm_failed']);
|
||||
}
|
||||
|
||||
// ── 9. cache + respond ──────────────────────────────────────────────
|
||||
$out = json_encode(['summary' => $summary, 'generated_at' => gmdate('c')],
|
||||
JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE);
|
||||
if (!atomic_write($cachePath, $out)) {
|
||||
// Degrade to per-click generation rather than a broken feature —
|
||||
// the daily cap still bounds spend.
|
||||
error_log('news_summary.php: cache write failed for ' . $cachePath);
|
||||
}
|
||||
if ($lockFh !== false) {
|
||||
flock($lockFh, LOCK_UN);
|
||||
fclose($lockFh);
|
||||
}
|
||||
http_response_code(200);
|
||||
header('Content-Type: application/json; charset=utf-8');
|
||||
header('Cache-Control: max-age=3600');
|
||||
echo $out;
|
||||
+130
@@ -5295,6 +5295,97 @@ def _():
|
||||
NewsEngine._source_health = saved
|
||||
|
||||
|
||||
@test("NewsEngine: RSS parser captures link/description/publisher")
|
||||
def _():
|
||||
cfg = NewsConfig()
|
||||
cfg.rss_feeds = ["https://news.google.com/rss/search?q=gold"]
|
||||
ne = NewsEngine(cfg)
|
||||
saved = NewsEngine._source_health
|
||||
NewsEngine._source_health = {}
|
||||
# Google-News-shaped item: redirect <link>, HTML-escaped <description>,
|
||||
# <source> carrying the real publisher.
|
||||
rss = (b"<rss><channel><item>"
|
||||
b"<title>Gold surges as Fed signals rate cuts - Reuters</title>"
|
||||
b"<link>https://news.google.com/rss/articles/CBMiAbC?oc=5</link>"
|
||||
b"<description><a href=\"https://x\">Gold &amp; dollar "
|
||||
b"diverge as the Fed turns dovish.</a></description>"
|
||||
b"<source url=\"https://www.reuters.com\">Reuters</source>"
|
||||
b"</item></channel></rss>")
|
||||
|
||||
class _Sess:
|
||||
def get(self, url, timeout=None):
|
||||
return type("R", (), {"status_code": 200, "content": rss})()
|
||||
|
||||
ne._session = _Sess()
|
||||
try:
|
||||
heads = ne._fetch_news_sentiment_raw()
|
||||
assert len(heads) == 1, heads
|
||||
h = heads[0]
|
||||
assert h.link == "https://news.google.com/rss/articles/CBMiAbC?oc=5"
|
||||
# tags stripped, entities (incl. double-encoded &amp;) decoded
|
||||
assert h.description == "Gold & dollar diverge as the Fed turns dovish."
|
||||
assert h.publisher == "Reuters"
|
||||
finally:
|
||||
NewsEngine._source_health = saved
|
||||
|
||||
|
||||
@test("NewsEngine: enrichment fields default None / reject unsafe values")
|
||||
def _():
|
||||
cfg = NewsConfig()
|
||||
cfg.rss_feeds = ["https://feed.example/rss"]
|
||||
ne = NewsEngine(cfg)
|
||||
saved = NewsEngine._source_health
|
||||
NewsEngine._source_health = {}
|
||||
long_link = b"https://a.example/" + b"x" * 2100
|
||||
rss = (b"<rss><channel>"
|
||||
# bare item: no link/description/source at all
|
||||
b"<item><title>Dollar steady before Fed minutes</title></item>"
|
||||
# javascript: link + description that merely echoes the title
|
||||
b"<item><title>Gold slips on strong dollar data</title>"
|
||||
b"<link>javascript:alert(1)</link>"
|
||||
b"<description>Gold slips on strong dollar data</description></item>"
|
||||
# over-long link
|
||||
b"<item><title>Fed officials split on rate cuts</title>"
|
||||
b"<link>" + long_link + b"</link></item>"
|
||||
b"</channel></rss>")
|
||||
|
||||
class _Sess:
|
||||
def get(self, url, timeout=None):
|
||||
return type("R", (), {"status_code": 200, "content": rss})()
|
||||
|
||||
ne._session = _Sess()
|
||||
try:
|
||||
heads = ne._fetch_news_sentiment_raw()
|
||||
assert len(heads) == 3, heads
|
||||
by_title = {h.headline: h for h in heads}
|
||||
bare = by_title["Dollar steady before Fed minutes"]
|
||||
assert bare.link is None and bare.description is None \
|
||||
and bare.publisher is None
|
||||
js = by_title["Gold slips on strong dollar data"]
|
||||
assert js.link is None # non-http(s) scheme rejected
|
||||
assert js.description is None # identical to title → dropped
|
||||
assert by_title["Fed officials split on rate cuts"].link is None
|
||||
finally:
|
||||
NewsEngine._source_health = saved
|
||||
|
||||
|
||||
@test("NewsEngine: _clean_html_text strips/unescapes/truncates")
|
||||
def _():
|
||||
from news_engine import _clean_html_text, _safe_link
|
||||
assert _clean_html_text(None) is None
|
||||
assert _clean_html_text("") is None
|
||||
assert _clean_html_text("<p></p>") is None
|
||||
assert _clean_html_text("<p>Fed <b>holds</b>\n rates</p>") == "Fed holds rates"
|
||||
assert _clean_html_text("&lt;b&gt;x&lt;/b&gt; & y") == "x & y"
|
||||
long = _clean_html_text("a" * 600)
|
||||
assert len(long) == 501 and long.endswith("…")
|
||||
assert _safe_link(" https://a.example/x ") == "https://a.example/x"
|
||||
assert _safe_link("ftp://a.example/x") is None
|
||||
assert _safe_link("javascript:alert(1)") is None
|
||||
assert _safe_link("https://" + "x" * 2100) is None
|
||||
assert _safe_link(None) is None
|
||||
|
||||
|
||||
@test("/status renders news/calendar/AI health with ⚠️ degradations")
|
||||
def _():
|
||||
import telegram_bot as tb
|
||||
@@ -5540,6 +5631,45 @@ def _():
|
||||
assert payload["headlines"][1]["score"] is None # NaN → None
|
||||
assert payload["events"][0]["event"] == "ISM"
|
||||
assert payload["sentiment"] == {"XAU": 0.6, "USD": -0.1}
|
||||
# Every headline carries the 40-hex id the PHP endpoint whitelists on.
|
||||
import re as _re
|
||||
assert all(_re.fullmatch(r"[0-9a-f]{40}", h["id"])
|
||||
for h in payload["headlines"])
|
||||
|
||||
|
||||
@test("dashboard: headline_to_dict id matches LLM cache scheme + enrichment")
|
||||
def _():
|
||||
import hashlib
|
||||
h = NewsSentiment(
|
||||
headline="Gold surges", source="http://feed/b",
|
||||
timestamp=datetime(2026, 8, 7, 10, 0, tzinfo=timezone.utc),
|
||||
currencies_mentioned=["XAU"], sentiment_score=0.6,
|
||||
keywords=["surge"], sentiment_by_currency={"XAU": 0.6},
|
||||
link="https://x.example/a", description="Body.", publisher="Reuters")
|
||||
d = dpub.headline_to_dict(h)
|
||||
# Same sha1(headline) scheme as NewsEngine._llm_sent_cache — the id is
|
||||
# the join key news_summary.php validates against.
|
||||
assert d["id"] == hashlib.sha1(b"Gold surges").hexdigest()
|
||||
assert d["link"] == "https://x.example/a"
|
||||
assert d["description"] == "Body."
|
||||
assert d["publisher"] == "Reuters"
|
||||
# Pre-enrichment NewsSentiment (defaults) still serializes cleanly.
|
||||
bare = NewsSentiment(
|
||||
headline="Old", source="s", timestamp=None,
|
||||
currencies_mentioned=[], sentiment_score=0.0, keywords=[])
|
||||
d2 = dpub.headline_to_dict(bare)
|
||||
assert d2["link"] is None and d2["description"] is None \
|
||||
and d2["publisher"] is None
|
||||
json.dumps([d, d2], allow_nan=False)
|
||||
|
||||
|
||||
@test("dashboard: install_static installs page + PHP endpoint atomically")
|
||||
def _():
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
pub = dpub.DashboardPublisher(out_dir=tmp)
|
||||
pub.install_static()
|
||||
assert sorted(os.listdir(tmp)) == \
|
||||
["dashboard.html", "news_summary.php"] # both installed, no .tmp
|
||||
|
||||
|
||||
@test("dashboard: publish is atomic and leaves no temp file")
|
||||
|
||||
Reference in New Issue
Block a user