diff --git a/CLAUDE.md b/CLAUDE.md index 15f95d1..decfc88 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,10 +11,12 @@ Every numeric threshold in `config.py` and `pair_signals/xauusd.py` was backtest ## Commands ```bash -python test_all.py # QA suite: 286 tests, 19 modules, no network (mocked) +python test_all.py # QA suite: 296 tests, 20 modules, no network (mocked) python main.py --pair XAUUSD --timeframe H1 --detail # CLI signal report (--json for full dump) python telegram_bot.py # the bot (refuses to boot without credentials) python oanda_daemon.py # auto-execution daemon on Oanda practice account +python dashboard_publisher.py --once # web dashboard: one scan+publish cycle to $DASHBOARD_OUT_DIR + # (omit --once for the candle-close daemon; deploy/README-dashboard.md) # Backtests python backtest/gpu_backtest.py --pair XAUUSD --days 365 @@ -56,6 +58,7 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz - **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that. - **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups. - **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown, trail ratchets at bar close exactly once per bar. Accepted divergence: one position total. +- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. Deployment checklist: `deploy/README-dashboard.md`. - **`backtest/gpu_backtest.py`** — bar-by-bar replay through the real `analyze_from_dataframes` (news off). Realism model (do not regress): next-bar-open entries re-anchored to the raw price path, round-trip `--spread` charged per trade, trade resolution on RAW bars (cleaning applies only to signal-side data), same-bar SL+TP tie → SL first, gap fills at bar open, trail ratchet at bar close only. Indicators are precomputed once per TF (CuPy GPU or NumPy fallback — check the `Backend:` log line); replay fans out over worker processes and the 5-bar cooldown is applied in an ordered pass afterwards, so parallel and sequential books are identical. Parquet cache in `backtest/cache/` keyed by provider/symbol/TF/window. `backtest/*.csv` are gitignored pinned trade books from past campaigns — used for manual before/after comparison. `backtest/instructions_for_backtest.txt` lists what every backtest report must record. ## Rules for making changes (read before editing signal logic) diff --git a/dashboard_publisher.py b/dashboard_publisher.py new file mode 100644 index 0000000..e303050 --- /dev/null +++ b/dashboard_publisher.py @@ -0,0 +1,420 @@ +#!/usr/bin/env python3 +""" +XAU/USD static dashboard publisher. + +Standalone daemon that scans XAU/USD at candle closes and atomically +writes `snapshot.json` + `news.json` (plus a copy of +`static/dashboard.html`) into an Apache-served directory. Apache does +all web serving — this process never opens a port, so a failure here +can only ever mean a stale page. + +Independence rules (owner requirement — the telegram bot must be +immune to anything on the web side): + * Runs as its own process; never imported by, and never imports, + `telegram_bot` / `oanda_daemon` (telegram_bot resolves credentials + at import). The confirmation map and boundary math are therefore + duplicated here as local constants. + * The only shared-code dependency beyond the standard signal path is + the additive `NewsEngine.get_week_events` accessor, which no + production signal path calls. + +Scheduling: every M15/H1/H4 boundary is also an M5 boundary (all +intervals are multiples of 5, anchored to UTC midnight), so one +sequential loop sleeps to the next M5 boundary (+10 s buffer) and scans +whichever timeframes just closed. Due-classification uses the RAW wake +time — flooring 12:00:10 to the 5-minute grid yields the 12:00 boundary +that just closed. Do NOT subtract the buffer first: 12:00:10 − 15 s +floors to 11:55 and silently skips every H1/H4 scan (v1 bug, 2026-08-02). + +Scans run sequentially M5→M15→H1→H4 through one shared data provider so +later timeframes hit the warm boundary cache — that ordering IS the +duplicate-fetch dedup (providers have no in-flight dedup; Oanda has no +throttle). Keep it sequential. + +Run: python dashboard_publisher.py [--once] [--out DIR] +Out dir precedence: --out → DASHBOARD_OUT_DIR env → /var/www/html/gold-dashboard +""" + +import argparse +import json +import logging +import math +import numbers +import os +import shutil +import sys +import time +from datetime import datetime, timezone +from logging.handlers import TimedRotatingFileHandler +from typing import Dict, List, Optional + +from config import PlatformConfig, Timeframe, XAUUSD +from main import report_to_dict +from news_engine import NewsEngine +from pair_signals import produce_signal +from providers import make_data_provider +import providers.base as _providers_base + +logger = logging.getLogger("dashboard") + +DASHBOARD_TFS = [Timeframe.M5, Timeframe.M15, Timeframe.H1, Timeframe.H4] + +# Local copy of the bot's confirmation map (see independence rules above). +CONFIRMATIONS = { + Timeframe.M5: [Timeframe.M15, Timeframe.H1], + Timeframe.M15: [Timeframe.M5, Timeframe.H1], + Timeframe.H1: [Timeframe.M15, Timeframe.H4], + Timeframe.H4: [Timeframe.H1, Timeframe.D1], +} + +TF_INTERVAL_MINUTES = { + Timeframe.M5: 5, Timeframe.M15: 15, Timeframe.H1: 60, Timeframe.H4: 240, +} + +DEFAULT_OUT_DIR = "/var/www/html/gold-dashboard" +SCAN_BUFFER_SECONDS = 10 +SNAPSHOT_FILE = "snapshot.json" +NEWS_FILE = "news.json" +PAGE_FILE = "dashboard.html" + + +# ══════════════════════════════════════════════════════════════════ +# Pure layer (no network, fully covered by test_all.py) +# ══════════════════════════════════════════════════════════════════ + +def _safe_num(v): + """Coerce to a JSON-safe float; NaN/±inf become None.""" + try: + f = float(v) + except (TypeError, ValueError): + return v + if math.isnan(f) or math.isinf(f): + return None + return f + + +def _sanitize(obj): + """Recursively make a payload safe for json.dumps(allow_nan=False): + NaN/inf → None, numpy scalars → native ints/floats.""" + if isinstance(obj, bool): + return obj + if isinstance(obj, dict): + return {k: _sanitize(v) for k, v in obj.items()} + if isinstance(obj, (list, tuple)): + return [_sanitize(v) for v in obj] + if isinstance(obj, numbers.Integral): + return int(obj) + if isinstance(obj, numbers.Real): + return _safe_num(obj) + return obj + + +def analysis_to_dict(analysis) -> Optional[dict]: + """Serialize a TechnicalAnalysis. Iterates analysis.indicators only — + never vars()/asdict(), which would drag dynamic pandas attrs + (_rsi_series, _macd_histogram, …) into the payload. ATR (weight 0) + is surfaced separately as context, not in the directional grid.""" + if analysis is None: + return None + indicators = {} + atr_context = None + for name, ind in analysis.indicators.items(): + if name.startswith("_"): + continue + entry = { + "signal": ind.signal.name, + "value": _safe_num(ind.value), + "confidence": _safe_num(ind.confidence), + "weight": _safe_num(ind.weight), + "description": ind.description, + } + if name == "atr": + atr_context = entry + elif ind.weight != 0: + indicators[name] = entry + return { + "timeframe": analysis.timeframe, + "timestamp": str(analysis.timestamp), + "composite_signal": analysis.composite_signal.name, + "composite_score": _safe_num(analysis.composite_score), + "confidence": _safe_num(analysis.confidence), + "market_regime": analysis.market_regime, + "volatility_regime": analysis.volatility_regime, + "pattern_signals": list(analysis.pattern_signals), + "key_levels": {k: _safe_num(v) for k, v in analysis.key_levels.items()}, + "indicators": indicators, + "atr_context": atr_context, + } + + +def report_to_webapp_dict(report) -> dict: + """main.report_to_dict + the per-TF detail the dashboard needs.""" + d = report_to_dict(report) + d["primary_signal"] = report.primary_signal.name + d["primary_score"] = _safe_num(report.primary_score) + d["mtf_scores"] = {k: _safe_num(v) for k, v in report.mtf_scores.items()} + d["exhaustion_count"] = report.exhaustion_count + d["primary_analysis"] = analysis_to_dict(report.primary_analysis) + d["confirmation_analyses"] = [ + analysis_to_dict(a) for a in report.confirmation_analyses + ] + return _sanitize(d) + + +def event_to_dict(ev) -> dict: + """EconomicEvent → dict. surprise/minutes_until/is_high_impact are + properties, so asdict() would skip them — added explicitly. + actual/forecast/previous stay strings as published ('228K', '0.3%').""" + return { + "time": ev.time.astimezone(timezone.utc).isoformat(), + "currency": ev.currency, + "impact": ev.impact, + "event": ev.event, + "forecast": ev.forecast, + "previous": ev.previous, + "actual": ev.actual, + "surprise": ev.surprise, + "minutes_until": _safe_num(ev.minutes_until), + "is_high_impact": ev.is_high_impact, + } + + +def headline_to_dict(h) -> dict: + return { + "headline": h.headline, + "source": h.source, + "timestamp": h.timestamp.astimezone(timezone.utc).isoformat() + if h.timestamp else None, + "currencies": list(h.currencies_mentioned), + "score": _safe_num(h.sentiment_score), + "keywords": list(h.keywords), + "score_by_currency": { + k: _safe_num(v) for k, v in h.sentiment_by_currency.items() + }, + } + + +def build_news_payload(context, week_events, now: datetime) -> dict: + """news.json: full calendar week + scored headlines + LLM summary.""" + headlines = sorted( + context.news_headlines, + key=lambda h: h.timestamp or now, reverse=True, + ) + return _sanitize({ + "generated_at": now.isoformat(), + "llm_summary": context.llm_summary, + "sentiment": dict(context.news_sentiment), + "high_impact_warning": context.high_impact_warning, + "warning_message": context.warning_message, + "calendar_available": context.calendar_available, + "events": [event_to_dict(e) for e in week_events], + "headlines": [headline_to_dict(h) for h in headlines], + }) + + +def due_timeframes(wake_dt: datetime) -> List[Timeframe]: + """Timeframes whose candle just closed at the boundary this wake + belongs to. Takes the RAW wake time (boundary + buffer): flooring to + the 5-minute grid yields the boundary that just passed. Never + subtract the buffer first — see module docstring (v1 bug).""" + minutes = wake_dt.hour * 60 + wake_dt.minute + floored = minutes - (minutes % 5) + return [tf for tf in DASHBOARD_TFS if floored % TF_INTERVAL_MINUTES[tf] == 0] + + +def seconds_until_boundary(interval_minutes: int, buffer: int = 10) -> float: + # Same math as telegram_bot.py / oanda_daemon.py — kept local so the + # dashboard never imports bot modules (independence rules above). + now = datetime.now(timezone.utc) + total_minutes = now.hour * 60 + now.minute + elapsed_minutes = total_minutes % interval_minutes + elapsed_seconds = elapsed_minutes * 60 + now.second + now.microsecond / 1e6 + interval_seconds = interval_minutes * 60 + remaining = interval_seconds - elapsed_seconds + buffer + if remaining <= buffer: + remaining += interval_seconds + return remaining + + +def _empty_tf_entry() -> dict: + return {"status": "warming", "scanned_at": None, "scan_seconds": None, + "error": None, "error_at": None, "report": None} + + +def build_snapshot_payload(snapshots: Dict[str, dict], now: datetime) -> dict: + """snapshot.json: one entry per dashboard TF. Timeframes not yet + scanned appear as status="warming" so the page can render a + placeholder instead of a hole. server_time_epoch lets the client + compute its clock offset once so countdowns don't drift.""" + timeframes = {} + for tf in DASHBOARD_TFS: + interval = TF_INTERVAL_MINUTES[tf] + minutes = now.hour * 60 + now.minute + secs_into = (minutes % interval) * 60 + now.second + now.microsecond / 1e6 + next_close = now.timestamp() + (interval * 60 - secs_into) + entry = _empty_tf_entry() + entry.update(snapshots.get(tf.value) or {}) + entry["next_close_epoch"] = next_close + entry["next_scan_epoch"] = next_close + SCAN_BUFFER_SECONDS + timeframes[tf.value] = entry + return _sanitize({ + "symbol": XAUUSD, + "server_time_epoch": now.timestamp(), + "market_closed": _providers_base._is_forex_weekend(now), + "timeframes": timeframes, + }) + + +def publish(obj: dict, path: str) -> None: + """Atomic JSON write: temp file + os.replace, so Apache never serves + a torn file. allow_nan=False — a NaN slipping through would render + the page blank (JSON.parse rejects bare NaN).""" + data = json.dumps(obj, allow_nan=False, separators=(",", ":")) + tmp = path + ".tmp" + with open(tmp, "w", encoding="utf-8") as fh: + fh.write(data) + os.replace(tmp, path) + + +# ══════════════════════════════════════════════════════════════════ +# Runtime layer (constructed only from main(); never at import) +# ══════════════════════════════════════════════════════════════════ + +def _setup_logging(level: str = "INFO") -> None: + log_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "logs") + os.makedirs(log_dir, exist_ok=True) + log_file = os.path.join(log_dir, "dashboard.log") + + fmt = logging.Formatter( + "%(asctime)s [%(levelname)s] %(name)s: %(message)s", + datefmt="%Y-%m-%d %H:%M:%S") + + file_handler = TimedRotatingFileHandler( + log_file, when="h", interval=3, backupCount=240, utc=True) + file_handler.setFormatter(fmt) + + console_handler = logging.StreamHandler(sys.stdout) + console_handler.setFormatter(fmt) + + logging.basicConfig( + level=getattr(logging, level.upper(), logging.INFO), + handlers=[file_handler, console_handler], + force=True) + + +class DashboardPublisher: + def __init__(self, out_dir: str): + self.out_dir = out_dir + self.snapshots: Dict[str, dict] = {} + self.news_payload: Optional[dict] = None + self._dp = None + self._news_engine: Optional[NewsEngine] = None + + def _provider(self): + if self._dp is None: + self._dp = make_data_provider(PlatformConfig()) + return self._dp + + def _news(self) -> NewsEngine: + if self._news_engine is None: + self._news_engine = NewsEngine(PlatformConfig().news) + return self._news_engine + + def install_static(self) -> None: + """Copy the repo's dashboard.html into the out dir (atomic), so + git pull + restart also updates the page.""" + src = os.path.join(os.path.dirname(os.path.abspath(__file__)), + "static", PAGE_FILE) + if not os.path.exists(src): + logger.warning(f"{src} missing — page not installed") + return + dst = os.path.join(self.out_dir, PAGE_FILE) + shutil.copyfile(src, dst + ".tmp") + os.replace(dst + ".tmp", dst) + + def scan_tf(self, tf: Timeframe) -> None: + started = time.time() + try: + report = produce_signal(XAUUSD, tf, self._provider(), + CONFIRMATIONS[tf]) + self.snapshots[tf.value] = { + "status": "ok", + "scanned_at": datetime.now(timezone.utc).isoformat(), + "scan_seconds": round(time.time() - started, 2), + "error": None, + "error_at": None, + "report": report_to_webapp_dict(report), + } + logger.info(f"{tf.value}: {report.final_signal.name} " + f"q={report.signal_quality or '-'} " + f"({time.time() - started:.1f}s)") + except Exception as e: + # Keep the last good report; the page shows it with an + # error badge instead of going blank. + logger.exception(f"{tf.value} scan failed") + entry = self.snapshots.get(tf.value) or _empty_tf_entry() + entry["status"] = "error" + entry["error"] = f"{type(e).__name__}: {e}" + entry["error_at"] = datetime.now(timezone.utc).isoformat() + self.snapshots[tf.value] = entry + + def refresh_news(self) -> None: + try: + ne = self._news() + context = ne.get_fundamental_context() + week = ne.get_week_events() + self.news_payload = build_news_payload( + context, week, datetime.now(timezone.utc)) + except Exception: + logger.exception("news refresh failed — keeping last payload") + + def publish_all(self) -> None: + now = datetime.now(timezone.utc) + publish(build_snapshot_payload(self.snapshots, now), + os.path.join(self.out_dir, SNAPSHOT_FILE)) + if self.news_payload is not None: + publish(self.news_payload, os.path.join(self.out_dir, NEWS_FILE)) + + def run_cycle(self, tfs: List[Timeframe]) -> None: + for tf in tfs: # sequential — see module docstring (cache dedup) + self.scan_tf(tf) + self.refresh_news() + self.publish_all() + + def run_forever(self) -> None: + self.run_cycle(DASHBOARD_TFS) # populate the page at boot + while True: + delay = seconds_until_boundary(5, buffer=SCAN_BUFFER_SECONDS) + logger.info(f"next M5 boundary in {delay:.0f}s") + time.sleep(delay) + try: + due = due_timeframes(datetime.now(timezone.utc)) + logger.info(f"boundary wake — scanning {[t.value for t in due]}") + self.run_cycle(due) + except Exception: + logger.exception("cycle failed") + + +def main(argv=None) -> None: + ap = argparse.ArgumentParser(description="XAU/USD static dashboard publisher") + ap.add_argument("--once", action="store_true", + help="one scan+publish cycle, then exit") + ap.add_argument("--out", + default=os.environ.get("DASHBOARD_OUT_DIR", DEFAULT_OUT_DIR), + help="output directory served by Apache " + f"(default: $DASHBOARD_OUT_DIR or {DEFAULT_OUT_DIR})") + args = ap.parse_args(argv) + + _setup_logging() + os.makedirs(args.out, exist_ok=True) + pub = DashboardPublisher(args.out) + pub.install_static() + if args.once: + pub.run_cycle(DASHBOARD_TFS) + print(f"Published to {args.out}") + return + pub.run_forever() + + +if __name__ == "__main__": + main() diff --git a/deploy/README-dashboard.md b/deploy/README-dashboard.md new file mode 100644 index 0000000..3d951da --- /dev/null +++ b/deploy/README-dashboard.md @@ -0,0 +1,90 @@ +# Deploying the web dashboard (live LAMP server) + +The dashboard is a **static publisher**: `dashboard_publisher.py` runs as its +own daemon, scans XAU/USD at candle closes, and atomically writes +`dashboard.html` + `snapshot.json` + `news.json` into a folder Apache already +serves. Apache does all web serving — no new port, no Python web server, and +the telegram bot process is completely unaffected by anything that happens +here (see the module docstring for the independence rules). + +All steps below run **on the live server** (`/home/Forex/ForexBot2`). + +## 1. Get the code + +```bash +cd /home/Forex/ForexBot2 +git pull +venv/bin/python test_all.py # optional sanity: suite must be green +``` + +No new Python dependencies are required. + +## 2. Create the web folder + +```bash +sudo mkdir -p /var/www/html/gold-dashboard +sudo chown Forex: /var/www/html/gold-dashboard # the user the daemon runs as +``` + +RHEL/Alma/CentOS with SELinux enforcing — label the folder so httpd may read it +(the daemon writes as a normal user; only httpd is SELinux-confined): + +```bash +sudo semanage fcontext -a -t httpd_sys_content_t '/var/www/html/gold-dashboard(/.*)?' +sudo restorecon -R /var/www/html/gold-dashboard +``` + +## 3. Smoke-test once + +```bash +venv/bin/python dashboard_publisher.py --once +ls /var/www/html/gold-dashboard # dashboard.html snapshot.json news.json +``` + +Then open `http:///gold-dashboard/dashboard.html` — all four +timeframe cards, the calendar, and the newswire should populate. + +## 4. Install the daemon + +```bash +sudo cp deploy/gold-dashboard.service /etc/systemd/system/ +# edit User= if the repo owner is not "Forex" +sudo systemctl daemon-reload +sudo systemctl enable --now gold-dashboard +systemctl status gold-dashboard +tail -f logs/dashboard.log +``` + +The daemon scans all four timeframes at boot, then wakes at every M5 +boundary (+10 s) and rescans whichever timeframes just closed. Logs rotate +every 3 h (240 backups), same scheme as the bot. + +## 5. Optional Apache polish + +The page cache-busts its JSON fetches (`?t=`), so no Apache config is +strictly needed. To be explicit anyway: + +```apache + + + Header set Cache-Control "no-cache" + + +``` + +The page is deliberately public (owner decision 2026-08-03). To restrict it +later, add standard Apache basic auth (`htpasswd` + `Require valid-user`) on +that directory — no code changes needed. + +## Operational notes + +- **Updating**: `git pull` + `systemctl restart gold-dashboard`. The daemon + re-copies `static/dashboard.html` into the web folder at startup. +- **If the daemon dies**, systemd restarts it in 30 s; worst case the page + shows stale data with its own "updated N min ago" evidence. Apache and the + telegram bot are unaffected. +- **DeepSeek**: the publisher process has its own daily request cap + (`deepseek.daily_request_cap`, default 500) separate from the bot's — + expect roughly up to ~430 extra calls/day worst case, V4-Flash pricing. +- **Credentials**: uses the same env/.env/config.json resolution as the rest + of the platform (Oanda candles feed; news needs no key; DeepSeek optional). diff --git a/deploy/gold-dashboard.service b/deploy/gold-dashboard.service new file mode 100644 index 0000000..d186d79 --- /dev/null +++ b/deploy/gold-dashboard.service @@ -0,0 +1,23 @@ +# systemd unit for the XAU/USD static dashboard publisher. +# Install (as root, on the live server): +# cp deploy/gold-dashboard.service /etc/systemd/system/ +# systemctl daemon-reload && systemctl enable --now gold-dashboard +# See deploy/README-dashboard.md for the full checklist. + +[Unit] +Description=XAU/USD dashboard publisher (writes static JSON/HTML for Apache) +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +# Adjust to the account that owns /home/Forex/ForexBot2: +User=Forex +WorkingDirectory=/home/Forex/ForexBot2 +Environment=DASHBOARD_OUT_DIR=/var/www/html/gold-dashboard +ExecStart=/home/Forex/ForexBot2/venv/bin/python dashboard_publisher.py +Restart=always +RestartSec=30 + +[Install] +WantedBy=multi-user.target diff --git a/news_engine.py b/news_engine.py index 291ebda..921ced5 100755 --- a/news_engine.py +++ b/news_engine.py @@ -426,6 +426,24 @@ class NewsEngine: return net, desc + def get_week_events( + self, currencies: Optional[List[str]] = None + ) -> List[EconomicEvent]: + """Full cached calendar week (past and upcoming), filtered to + `currencies` (default: tracked_currencies), sorted by time. + + Read-only accessor for display surfaces (the web dashboard) — + rides the same class-level cache as signal scans, so it adds no + network I/O. No production signal path calls this. + """ + if currencies is None: + currencies = self.config.tracked_currencies + events = self._cached_fetch("_calendar_cache", self._fetch_calendar_raw) + return sorted( + (e for e in events if e.currency in currencies), + key=lambda e: e.time, + ) + # ── Private Methods ──────────────────────────────────────────── # Maps country names in MyFXBook titles to currency codes diff --git a/static/dashboard.html b/static/dashboard.html new file mode 100644 index 0000000..cd0e476 --- /dev/null +++ b/static/dashboard.html @@ -0,0 +1,773 @@ + + + + + + +Gold Desk — XAU/USD + + + + + +
+
+
XAU/USD
+
—
+
+
+
+
…
+
M5 close in --:--
+
updated —
+
+
+ + + + + + +
Timeframes
+
+ +
Fundamentals
+
+
+
+

Economic calendar

+ times local · USD events + +
+
+ + + + +
TimeCurImpactEventFcstPrevActual
Loading calendar…
+
+
+
+

Newswire

+
+
+
+ +
+ Signals are generated on closed candles only and refresh at candle boundaries + (M5/M15/H1/H4). Not financial advice — educational and research use only. +
+ + + + diff --git a/test_all.py b/test_all.py index dc6edb4..f8656ca 100644 --- a/test_all.py +++ b/test_all.py @@ -1102,6 +1102,40 @@ def _(): NewsEngine._calendar_cache = (0.0, []) +@test("NewsEngine.get_week_events filters, sorts, and rides the cache") +def _(): + NewsEngine._calendar_cache = (0.0, []) + try: + ne = NewsEngine(NewsConfig()) + calls = {"n": 0} + now = datetime.now(timezone.utc) + + def fake_fetch(): + calls["n"] += 1 + return [ + EconomicEvent(time=now + timedelta(hours=5), currency="USD", + impact="high", event="NFP"), + EconomicEvent(time=now - timedelta(hours=2), currency="USD", + impact="low", event="Past Event"), + EconomicEvent(time=now + timedelta(hours=1), currency="EUR", + impact="high", event="ECB Rate"), + ] + + ne._try_fetch_calendar_api = fake_fetch + ne._try_fetch_myfxbook_rss = lambda: [] + + events = ne.get_week_events() # defaults to tracked ["USD", "XAU"] + assert [e.event for e in events] == ["Past Event", "NFP"], \ + f"Expected USD-only sorted by time, got {[e.event for e in events]}" + + # Second call (and explicit currency widening) must not refetch. + wide = ne.get_week_events(currencies=["USD", "EUR"]) + assert calls["n"] == 1, f"Expected 1 fetch, got {calls['n']}" + assert [e.event for e in wide] == ["Past Event", "ECB Rate", "NFP"] + finally: + NewsEngine._calendar_cache = (0.0, []) + + @test("FundamentalContext defaults") def _(): ctx = FundamentalContext() @@ -4965,6 +4999,218 @@ def _(): assert cfg.sentiment_affects_signals is False +# ══════════════════════════════════════════════════════════════════════ +# MODULE 20: dashboard_publisher.py (static web dashboard) +# ══════════════════════════════════════════════════════════════════════ + +print("\n═══ Module 20: dashboard_publisher.py ═══") + +import tempfile +import dashboard_publisher as dpub +from engine import TechnicalAnalysis, IndicatorSignal + + +def make_fake_dashboard_report(): + """SignalReport with every serialization hazard: NaN value, inf key + level, weight-0 ATR, a poisoned '_secret' indicator, a dynamic + pandas attr, and a full TradeSetup.""" + analysis = TechnicalAnalysis( + symbol="XAU/USD", timeframe="1h", + timestamp="2026-08-03 10:00:00+00:00") + names = ["ema_trend", "rsi", "macd", "bollinger", "adx", + "roc", "keltner", "divergence", "candlestick"] + for i, name in enumerate(names): + analysis.indicators[name] = IndicatorSignal( + name=name, value=float(i), + signal=SignalStrength.BUY if i % 2 == 0 else SignalStrength.SELL, + confidence=60.0, description=f"{name} desc", weight=1.0) + analysis.indicators["rsi"].value = float("nan") + analysis.indicators["atr"] = IndicatorSignal( + name="atr", value=12.5, signal=SignalStrength.NEUTRAL, + confidence=0.0, description="ATR context", weight=0.0) + analysis.indicators["_secret"] = IndicatorSignal( + name="_secret", value=1.0, signal=SignalStrength.NEUTRAL, + confidence=0.0, description="must not serialize", weight=1.0) + analysis._rsi_series = pd.Series([1.0, 2.0]) # dynamic attr, must not leak + analysis.key_levels = {"atr": 12.5, "support": 3300.0, + "broken": float("inf")} + analysis.composite_signal = SignalStrength.BUY + analysis.composite_score = 42.0 + analysis.confidence = 61.5 + analysis.pattern_signals = ["BULLISH_ENGULFING"] + + report = SignalReport(symbol="XAU/USD", display_name="XAU/USD", + timestamp="2026-08-03 10:00:00+00:00") + report.final_signal = SignalStrength.BUY + report.final_score = 55.0 + report.final_confidence = 70.0 + report.signal_quality = "B" + report.primary_signal = SignalStrength.BUY + report.primary_score = 50.0 + report.primary_analysis = analysis + report.confirmation_analyses = [analysis] + report.mtf_signals = {"1h": SignalStrength.BUY, + "4h": SignalStrength.NEUTRAL} + report.mtf_scores = {"1h": 50.0, "4h": 10.0} + report.exhaustion_count = 1 + report.current_price = 3350.0 + report.trade_setup = TradeSetup( + pair="XAU/USD", display_name="XAU/USD", direction="BUY", + entry_price=3350.0, stop_loss=3340.0, take_profit_1=3360.0, + take_profit_2=3370.0, risk_reward=2.0, risk_pips=100.0, + reward_pips=200.0, pip_value=0.1, strategy="XAU_ATR", + trail_arm_r=1.0, trail_dist_r=0.85) + return report + + +@test("dashboard: report_to_webapp_dict round-trips allow_nan=False") +def _(): + d = dpub.report_to_webapp_dict(make_fake_dashboard_report()) + json.dumps(d, allow_nan=False) # raises if NaN/inf leaked + assert d["signal"] == "BUY" and d["primary_signal"] == "BUY" + assert d["mtf_scores"] == {"1h": 50.0, "4h": 10.0} + assert d["exhaustion_count"] == 1 + assert d["trade_setup"]["strategy"] == "XAU_ATR" + pa = d["primary_analysis"] + assert pa["composite_signal"] == "BUY" + assert pa["indicators"]["rsi"]["value"] is None # NaN → None + assert pa["key_levels"]["broken"] is None # inf → None + assert len(d["confirmation_analyses"]) == 1 + + +@test("dashboard: ATR is context not grid; _-keys excluded") +def _(): + pa = dpub.report_to_webapp_dict(make_fake_dashboard_report())["primary_analysis"] + assert "atr" not in pa["indicators"] + assert "_secret" not in pa["indicators"] + assert "_rsi_series" not in pa + assert pa["atr_context"]["value"] == 12.5 + assert len(pa["indicators"]) == 9 + + +@test("dashboard: missing divergence and weekend report tolerated") +def _(): + report = make_fake_dashboard_report() + del report.primary_analysis.indicators["divergence"] + pa = dpub.report_to_webapp_dict(report)["primary_analysis"] + assert len(pa["indicators"]) == 8 and "divergence" not in pa["indicators"] + + weekend = SignalReport(symbol="XAU/USD", display_name="XAU/USD", + timestamp="now") # primary_analysis=None + d = dpub.report_to_webapp_dict(weekend) + json.dumps(d, allow_nan=False) + assert d["primary_analysis"] is None + assert d["confirmation_analyses"] == [] + + +@test("dashboard: due_timeframes uses RAW wake time") +def _(): + def due(h, m): + wake = datetime(2026, 8, 3, h, m, 10, tzinfo=timezone.utc) + return [tf.value for tf in dpub.due_timeframes(wake)] + # The v1 regression: 12:00:10 must classify as the 12:00 boundary + # (all four due), not 11:55. + assert due(12, 0) == ["5min", "15min", "1h", "4h"] + assert due(13, 0) == ["5min", "15min", "1h"] + assert due(12, 15) == ["5min", "15min"] + assert due(12, 5) == ["5min"] + assert due(0, 0) == ["5min", "15min", "1h", "4h"] + + +@test("dashboard: build_snapshot_payload warming/error/next_close") +def _(): + now = datetime(2026, 8, 3, 12, 2, 30, tzinfo=timezone.utc) # Monday + with patch("providers.base._is_forex_weekend", return_value=False): + payload = dpub.build_snapshot_payload({}, now) + assert payload["symbol"] == "XAU/USD" + assert payload["market_closed"] is False + assert set(payload["timeframes"]) == {"5min", "15min", "1h", "4h"} + for entry in payload["timeframes"].values(): + assert entry["status"] == "warming" and entry["report"] is None + assert entry["next_close_epoch"] > payload["server_time_epoch"] + assert entry["next_scan_epoch"] == entry["next_close_epoch"] + 10 + # M5 closes at 12:05 → 150 s away from 12:02:30. + assert abs(payload["timeframes"]["5min"]["next_close_epoch"] + - (now.timestamp() + 150)) < 1e-6 + + # An errored TF keeps its stale report + error badge. + stale = {"status": "error", "scanned_at": "earlier", + "error": "ValueError: boom", "error_at": "now", + "report": {"signal": "BUY"}} + with patch("providers.base._is_forex_weekend", return_value=True): + payload = dpub.build_snapshot_payload({"5min": stale}, now) + assert payload["market_closed"] is True + m5 = payload["timeframes"]["5min"] + assert m5["status"] == "error" and m5["report"] == {"signal": "BUY"} + assert m5["error"] == "ValueError: boom" + json.dumps(payload, allow_nan=False) + + +@test("dashboard: event_to_dict includes computed properties") +def _(): + ev = EconomicEvent( + time=datetime(2026, 8, 7, 12, 30, tzinfo=timezone.utc), + currency="USD", impact="high", event="Non-Farm Payrolls", + forecast="180K", previous="147K", actual="228K") + d = dpub.event_to_dict(ev) + assert d["event"] == "Non-Farm Payrolls" + assert d["actual"] == "228K" # strings stay strings + assert d["surprise"] == "positive" # property, added explicitly + assert d["is_high_impact"] is True + assert isinstance(d["minutes_until"], float) + assert d["time"] == "2026-08-07T12:30:00+00:00" + json.dumps(d, allow_nan=False) + + +@test("dashboard: build_news_payload sorts headlines, survives NaN") +def _(): + now = datetime(2026, 8, 3, 12, 0, tzinfo=timezone.utc) + old = NewsSentiment( + headline="Old story", source="http://feed/a", + timestamp=now - timedelta(hours=5), + currencies_mentioned=["USD"], sentiment_score=float("nan"), + keywords=[], sentiment_by_currency={}) + new = NewsSentiment( + headline="Gold surges", source="http://feed/b", + timestamp=now - timedelta(minutes=10), + currencies_mentioned=["XAU"], sentiment_score=0.6, + keywords=["surge"], sentiment_by_currency={"XAU": 0.6}) + ctx = FundamentalContext( + news_headlines=[old, new], + news_sentiment={"XAU": 0.6, "USD": -0.1}, + llm_summary="Macro blurb.", calendar_available=True) + events = [EconomicEvent(time=now + timedelta(hours=3), + currency="USD", impact="medium", event="ISM")] + payload = dpub.build_news_payload(ctx, events, now) + json.dumps(payload, allow_nan=False) + assert payload["llm_summary"] == "Macro blurb." + assert [h["headline"] for h in payload["headlines"]] == \ + ["Gold surges", "Old story"] # newest first + assert payload["headlines"][1]["score"] is None # NaN → None + assert payload["events"][0]["event"] == "ISM" + assert payload["sentiment"] == {"XAU": 0.6, "USD": -0.1} + + +@test("dashboard: publish is atomic and leaves no temp file") +def _(): + with tempfile.TemporaryDirectory() as tmp: + path = os.path.join(tmp, "snapshot.json") + dpub.publish({"a": 1, "ok": True}, path) + with open(path, encoding="utf-8") as fh: + assert json.load(fh) == {"a": 1, "ok": True} + assert os.listdir(tmp) == ["snapshot.json"] # no .tmp left + + +@test("dashboard: seconds_until_boundary bounds and constants") +def _(): + for interval in (5, 15, 60, 240): + s = dpub.seconds_until_boundary(interval, buffer=10) + assert 10 < s <= interval * 60 + 10, f"{interval}min → {s}" + # Confirmation map mirrors the bot's (kept local by design). + assert dpub.CONFIRMATIONS[Timeframe.M5] == [Timeframe.M15, Timeframe.H1] + assert dpub.CONFIRMATIONS[Timeframe.H4] == [Timeframe.H1, Timeframe.D1] + + # ══════════════════════════════════════════════════════════════════════ # SUMMARY # ══════════════════════════════════════════════════════════════════════