agent_panel: ATR-normalized candle + indicator-value enrichment, hardened prompts, verdict logging
This commit is contained in:
Harwinder Sekhon
@@ -35,7 +35,7 @@ Python 3.12; a `venv/` exists in-repo. Dependencies in `requirements.txt` (numpy
|
||||
|
||||
## Configuration & credentials
|
||||
|
||||
Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`, `DEEPSEEK_API_KEY` (news LLM; the `deepseek` config.json block holds model/enabled/timeout/daily cap). The `agent_panel` config.json block (all-defaults = feature dark) controls the advisory LLM agent panel: `enabled`, per-surface `telegram`/`dashboard` toggles, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`.
|
||||
Precedence for every credential/deployment setting: **env var → config.json → refuse to run** (no hardcoded fallbacks). A gitignored repo-root `.env` is loaded at `app_config` import via `os.environ.setdefault`. `config.json` (gitignored; schema in `config.json.example`) holds secrets plus deployment settings: `data_providers.active` (the live-feed switch), `data_providers.streaming` (optional tick-stream provider for the level-breach monitor), `scanning.pairs`/`timeframes`, per-pair `blocked_hours_utc`, `log_level`. Key env vars: `TELEGRAM_TOKEN`, `TELEGRAM_CHAT_ID`, `TELEGRAM_CHANNEL_ID`, `TWELVE_DATA_API_KEY`, `TIINGO_API_KEY`, `OANDA_ACCOUNT_ID`/`OANDA_API_TOKEN`/`OANDA_BASE_URL`, `DEEPSEEK_API_KEY` (news LLM; the `deepseek` config.json block holds model/enabled/timeout/daily cap). The `agent_panel` config.json block (all-defaults = feature dark) controls the advisory LLM agent panel: `enabled`, per-surface `telegram`/`dashboard` toggles, `min_quality`, `max_panels_per_day`, `reuse_ttl_seconds`, plus the technical-payload enrichment knobs `candles_enabled`/`candle_bars_primary`/`candle_bars_confirm`.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -59,9 +59,9 @@ Signal currency everywhere is `SignalStrength` (−2..+2). DataFrames are UTC tz
|
||||
- **`engine.py`** — `TechnicalEngine`: 9 scored indicators (ema_trend, rsi, macd, bollinger, adx, roc, keltner, divergence, candlestick) + ATR as context. Regime classification (trending_up/down, `trending` = mixed/direction-UNKNOWN, transitioning, ranging) drives per-indicator weight multipliers and regime-aware RSI reinterpretation. Compute order matters (ATR first; MACD always computed even when not a pillar — divergence needs the histogram). Private series ride on the result as `_underscore` attrs; scorers/formatters skip anything `_`-prefixed or weight-0.
|
||||
- **`signal_aggregator.py`** — MTF agreement, diminishing-decay penalty system, exhaustion hard block, quality grading, then an ordered chain of post-grade gates (session, XAU SELL quality/vol/hour, regime blocks, M5 noise gates), then three setup generators: XAU ATR (A+/A), generic swing-based, plus the M5 pullback (`XAU_PB`) and chart-pattern (`XAU_CP`, `chart_patterns.py`) entry paths. The PB/CP generators run AFTER the gate chain and force quality to "B"; the CP path deliberately accepts grade-F composites and re-checks only session/day/big-candle (via warning prefixes) plus its own SELL/vol/RSI gates — it does NOT re-check exhaustion or the M5 min-score/MTF gates. **This is validated behavior, not a bug** (2026-08-04 audit: 226/586 trades in the shipped M5 12mo book are XAU_CP, +38.9R, 104 of them exhaustion-flagged and net-positive) — changing it is a rule-1 revalidation event. `TradeSetup.strategy` selects trailing params; trail params are stamped on the setup from the *producing* config so per-TF overrides reach formatters.
|
||||
- **`pair_signals/`** — per-pair config factories. **Always use the factory (`produce_signal` / `make_config`), never bare `PlatformConfig()`** — the factory applies the config.json overlay first, then the validated per-TF deltas (M5 hour-9 block, M15 trail arm 0.70, M15 SELL-hour block, etc.; evidence in the docstring). A test enforces factory ≡ inline-config equivalence. Confirmation map: M5→[M15,H1], M15→[M5,H1], H1→[M15,H4], H4→[H1,D1].
|
||||
- **`signal_log.py`** — SQLite signal-history logger (stdlib only). Hooked into `SignalAggregator.analyze_pair` only (the live entry point — the backtester never writes here, preserving parity), so every live scan from every surface (Telegram `telegram`, CLI `cli`, Oanda daemon `oanda_daemon`, dashboard `dashboard`) lands one row in `logs/signals.db`: signal/grade/price, MTF + trade-setup + reasons JSON, and the news at scan time (event risk + calendar events + scored headlines JSON, re-fetched via the NewsEngine class-level caches). Config.json `signal_log` block (`enabled`, `path`; env `SIGNAL_LOG_ENABLED`/`SIGNAL_LOG_PATH`). **Fail-safe by contract** — a logging failure can never break a scan. Panel fields are NOT logged (attach_panel runs post-hoc, after analyze_pair returns).
|
||||
- **`signal_log.py`** — SQLite signal-history logger (stdlib only). Hooked into `SignalAggregator.analyze_pair` only (the live entry point — the backtester never writes here, preserving parity), so every live scan from every surface (Telegram `telegram`, CLI `cli`, Oanda daemon `oanda_daemon`, dashboard `dashboard`) lands one row in `logs/signals.db`: signal/grade/price, MTF + trade-setup + reasons JSON, and the news at scan time (event risk + calendar events + scored headlines JSON, re-fetched via the NewsEngine class-level caches). Config.json `signal_log` block (`enabled`, `path`; env `SIGNAL_LOG_ENABLED`/`SIGNAL_LOG_PATH`). **Fail-safe by contract** — a logging failure can never break a scan. Panel fields arrive post-hoc: `log_signal` stashes the rowid on the report and `agent_panel.attach_panel` writes the verdict back onto the same row via `log_panel_update` (no rowid — backtest replay or logging disabled — is a silent no-op).
|
||||
- **`news_engine.py`** — Forex Factory calendar (MyFXBook RSS fallback) + RSS keyword sentiment with gold-inverse keywords. Class-level caches (fresh engine per scan). Optional DeepSeek enrichment via `llm_client.py` (requests-only, no SDK): batched headline sentiment overriding keyword scores, gold-direction on release surprises, and a market-summary blurb (`SignalReport.news_summary`) — **display/warning-only, keyword fallback on any failure, inert without a key**; prompts are static module constants (DeepSeek prefix-cache pricing) and requests send `thinking: disabled` (V4-Flash reasoning otherwise blows the timeout). By XAU-tuned default, event risk and sentiment **warn but do not gate signals** (`event_risk_caps_signals=False`, `sentiment_affects_signals=False`) — the LLM does not change that.
|
||||
- **`agent_panel.py`** — ai-hedge-fund-style advisory panel: 4 DeepSeek persona calls (macro/technical/risk/contrarian, concurrent on a panel-private pool) + 1 synthesizer producing an **ADVISORY-ONLY** consensus (`AGREE/NEUTRAL/OPPOSE` + confidence + rationale) written onto `SignalReport.panel_*` fields. Attached **post-hoc** by consumers (`telegram_bot` alert path, `main --panel`, `dashboard_publisher`) — never runs inside `analyze_from_dataframes` (backtest parity, test-locked) and no gate reads its fields. Static prompts (prefix caching); spend bounded by an exact per-bar cache, a reuse TTL keyed (symbol, TF, direction) so persisting signals don't re-spend every bar, and a `max_panels_per_day` UTC budget (default 20 ≈ ≤100 requests) on top of the shared 500/day client cap. `<2` valid persona takes skips the synthesizer; any failure degrades to no panel, never a changed signal.
|
||||
- **`agent_panel.py`** — ai-hedge-fund-style advisory panel: 4 DeepSeek persona calls (macro/technical/risk/contrarian, concurrent on a panel-private pool) + 1 synthesizer producing an **ADVISORY-ONLY** consensus (`AGREE/NEUTRAL/OPPOSE` + confidence + rationale) written onto `SignalReport.panel_*` fields. Attached **post-hoc** by consumers (`telegram_bot` alert path, `main --panel`, `dashboard_publisher`) — never runs inside `analyze_from_dataframes` (backtest parity, test-locked) and no gate reads its fields. Static prompts (prefix caching); spend bounded by an exact per-bar cache, a reuse TTL keyed (symbol, TF, direction) so persisting signals don't re-spend every bar, and a `max_panels_per_day` UTC budget (default 20 ≈ ≤100 requests) on top of the shared 500/day client cap. `<2` valid persona takes skips the synthesizer; any failure degrades to no panel, never a changed signal. The technical persona's payload is enriched (2026-08-15): raw indicator values plus ATR-normalized recent candles per TF (`candles_enabled`/`candle_bars_primary` 20/`candle_bars_confirm` 10 config keys; code does the arithmetic — raw prices are the model's weakest input, per `backtest/research/llm_news_eval_2026-08-06.md`). Prompts forbid training-data/outside knowledge and cap notes at 15 words / rationales at 30 to bound output tokens.
|
||||
- **`telegram_bot.py`** — owner-locked bot + subscriber channel; candle-boundary auto-scan loops (M5/M15/H1/H4, boundary+10 s), alert quality threshold + fingerprint dedup (committed only after a successful send), 📰 LLM market-context line on alerts when available, WebSocket level-breach monitor fed by the streaming provider (`data_providers.streaming`, falling back to the active provider; auto-disabled when the resolved provider can't stream or has no API key — Oanda does not stream, Tiingo does). Rotating log: `logs/telegram.log`, 3-h rotation × 240 backups.
|
||||
- **`oanda_daemon.py` / `oanda_executor.py`** — practice-account auto-execution. Book-parity rules (see daemon docstring): no broker-side TP (trailing model — TP1/TP2 advisory), 5-bar per-TF cooldown (wall-clock approximation, so it expires over weekends where the backtester would still count bars), trail ratchets at bar close once per *observed* bar (a missed/failed cycle skips those bars entirely — the intra-gap high/low never enters the peak). Accepted divergence: one position total; orders send `positionFill=OPEN_ONLY` so a state/broker divergence cancels the order instead of silently netting. State/broker re-sync runs at boot and after any suspicious fill (`_reconcile_positions`).
|
||||
- **`dashboard_publisher.py` / `static/dashboard.html`** — web dashboard as a **static publisher** (live LAMP server): standalone daemon scans at candle closes via `produce_signal` and atomically writes `snapshot.json`/`news.json`/`dashboard.html` to `$DASHBOARD_OUT_DIR` (Apache serves them; page is public by owner decision). **Hard independence rule: never imports, and is never imported by, `telegram_bot`/`oanda_daemon`** — confirmation map and boundary math are deliberate local copies, so a web-side failure can only mean a stale page. Due-TF classification uses the RAW wake time (flooring wake−buffer skips every H1/H4 scan — 2026-08-02 bug). News panel reads `NewsEngine.get_fundamental_context()` plus the additive `get_week_events()` accessor (display-only; no signal path calls it); as a separate process it has its own news caches and DeepSeek daily cap. News/calendar rows are clickable (detail dialog); headlines carry `id` = sha1(headline) in news.json, and exactly one on-demand endpoint exists: `static/news_summary.php` (installed by `install_static`), which serves a DeepSeek gold-impact summary per headline — whitelist-gated against news.json ids, cache-forever in `ai/`, own fail-closed daily cap (150/day, file-based, separate from the publisher's), key at `/etc/gold-dashboard/deepseek.key`, never reads repo config. Deployment checklist: `deploy/README-dashboard.md`.
|
||||
|
||||
+163
-20
@@ -32,10 +32,26 @@ layers bound the spend on top of that cap:
|
||||
|
||||
Prompts are STATIC module constants (DeepSeek prefix-cache pricing —
|
||||
see news_engine.py); all variable content travels in the user payload.
|
||||
|
||||
Payload enrichment (2026-08-15, owner-approved, advisory-only): the
|
||||
technical persona also receives raw indicator VALUES (with a units
|
||||
legend in the prompt) and ATR-NORMALIZED recent candles per timeframe
|
||||
(code does the arithmetic — raw prices are the model's weakest input;
|
||||
see backtest/research/llm_news_eval_2026-08-06.md). Sized by config
|
||||
``candle_bars_primary`` (default 20) / ``candle_bars_confirm``
|
||||
(default 10); ``candles_enabled: false`` reverts to the pre-enrichment
|
||||
payload. Prompts forbid outside/training-data knowledge and cap notes
|
||||
at 15 words / rationales at 30 (output tokens are the pricey side).
|
||||
|
||||
Verdict logging (2026-08-15): after a successful attach, the verdict is
|
||||
written back onto the scan's signals.db row via signal_log's rowid
|
||||
hand-off — building the "does OPPOSE predict losers?" dataset without
|
||||
changing any live behavior.
|
||||
"""
|
||||
|
||||
import datetime as _dt
|
||||
import logging
|
||||
import math
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
@@ -44,6 +60,7 @@ from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
|
||||
|
||||
from app_config import get_app_config
|
||||
from config import Timeframe
|
||||
from llm_client import DEFAULT_TIMEOUT, get_llm_client
|
||||
|
||||
if TYPE_CHECKING: # type hints only — keep the coupling one-way
|
||||
@@ -55,6 +72,8 @@ logger = logging.getLogger(__name__)
|
||||
DEFAULT_MIN_QUALITY = "B"
|
||||
DEFAULT_MAX_PANELS_PER_DAY = 20
|
||||
DEFAULT_REUSE_TTL_SECONDS = 3600.0
|
||||
DEFAULT_CANDLE_BARS_PRIMARY = 20
|
||||
DEFAULT_CANDLE_BARS_CONFIRM = 10
|
||||
|
||||
# Quality ranks for the min_quality gate. Deliberately a local copy —
|
||||
# telegram_bot's ranking is private and dashboard_publisher must not
|
||||
@@ -78,9 +97,11 @@ _TAKE_CONTRACT = """\
|
||||
Your review is advisory research on an already-generated signal; you are
|
||||
one voice on a panel, not the decision maker.
|
||||
Hard rules: use ONLY the data in the JSON you receive — do not invent
|
||||
numbers, events, or price levels. If the evidence your role needs is
|
||||
thin or missing, answer "neutral" with confidence below 30, unless your
|
||||
role's instructions say otherwise.
|
||||
numbers, events, or price levels. Your training data about gold, the
|
||||
markets, and current events is stale and OFF-LIMITS: never use outside
|
||||
knowledge; if a fact is not in the JSON, treat it as unknown. If the
|
||||
evidence your role needs is thin or missing, answer "neutral" with
|
||||
confidence below 30, unless your role's instructions say otherwise.
|
||||
Confidence scale (0-100): 80-100 strong one-sided evidence; 50-79 solid
|
||||
case with some doubt; 30-49 mixed evidence; below 30 little basis to
|
||||
judge.
|
||||
@@ -91,7 +112,7 @@ not "H4"; describe what an indicator MEANS ("the price has climbed
|
||||
unusually fast"), never its name or number.
|
||||
Reply with ONLY JSON:
|
||||
{"stance": "support"|"oppose"|"neutral", "confidence": <0-100>,
|
||||
"note": "<max 25 words>"}
|
||||
"note": "<max 15 words>"}
|
||||
"stance" says whether YOUR analysis supports the proposed trade
|
||||
direction, opposes it, or is genuinely mixed; "confidence" is how sure
|
||||
you are of your stance; "note" is your single most important point.
|
||||
@@ -125,15 +146,26 @@ and a soft jobs report -> {"stance": "support", "confidence": 75,
|
||||
_TECH_SYSTEM = """\
|
||||
You are the technical analyst on a gold (XAU/USD) trade-review panel.
|
||||
You receive JSON: a proposed trade brief ("brief") plus the technical
|
||||
evidence — per-"indicators" signal/confidence, multi-timeframe
|
||||
"mtf" signals and scores, "key_levels" versus the current price, and an
|
||||
evidence — per-"indicators" signal/confidence/value, multi-timeframe
|
||||
"mtf" signals and scores, "key_levels" versus the current price, an
|
||||
"exhaustion" count of consecutive same-direction bars flagged by the
|
||||
engine.
|
||||
engine, and optionally "candles".
|
||||
Indicator "value" units: rsi 0-100 (above 70 overbought, below 30
|
||||
oversold); adx 0-100 (above 25 = strong trend); macd = histogram in
|
||||
dollars (momentum); roc = percent; bollinger = position 0-1 (0 = lower
|
||||
band, 1 = upper band); keltner = dollar distance from the mid-line;
|
||||
ema_trend = 0-1 alignment score; divergence and candlestick = pattern
|
||||
counts.
|
||||
"candles" is per timeframe: {"ref_close", "atr", "bars"} — bars are
|
||||
oldest-first, each [open, high, low, close] expressed in ATR units
|
||||
relative to ref_close: 0.0 = the latest close, +1.0 = one ATR above it,
|
||||
-2.0 = two ATR below it. Read the SHAPE (drift, swings, wicks); never
|
||||
quote these as prices.
|
||||
Judge ONE thing: is the technical picture internally coherent for the
|
||||
proposed direction? Look for indicators fighting each other, the signal
|
||||
fighting the regime (e.g. a BUY in trending_down), higher timeframes
|
||||
disagreeing, price pressed into a key level, or an exhausted move being
|
||||
chased.
|
||||
disagreeing, price pressed into a key level, an exhausted move being
|
||||
chased, or the candle path contradicting the indicator readings.
|
||||
Stance rules:
|
||||
- support: the indicators, higher timeframes, and regime mostly agree
|
||||
with the proposed direction and none of the conflicts above appear.
|
||||
@@ -213,7 +245,7 @@ possible profit is too small and the price has already climbed
|
||||
unusually fast. Only the news backdrop supports buying."
|
||||
Reply with ONLY JSON:
|
||||
{"verdict": "AGREE"|"NEUTRAL"|"OPPOSE", "confidence": <0-100>,
|
||||
"rationale": "<max 40 words>"}
|
||||
"rationale": "<max 30 words>"}
|
||||
"""
|
||||
|
||||
|
||||
@@ -238,6 +270,16 @@ class PanelVerdict:
|
||||
|
||||
# ── Payload builders (pure functions; llm_client json-dumps sorted) ──
|
||||
|
||||
def _finite(x: Any) -> Optional[float]:
|
||||
"""float(x) or None for non-numeric / non-finite (NaN/inf breaks the
|
||||
strict-JSON wire format the client sends)."""
|
||||
try:
|
||||
f = float(x)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return f if math.isfinite(f) else None
|
||||
|
||||
|
||||
def _direction(report: "SignalReport") -> Optional[str]:
|
||||
"""BUY/SELL the panel should review; None = nothing to review."""
|
||||
if report.trade_setup is not None:
|
||||
@@ -286,21 +328,26 @@ def _macro_payload(report: "SignalReport",
|
||||
}
|
||||
|
||||
|
||||
def _technical_payload(report: "SignalReport") -> Dict[str, Any]:
|
||||
def _technical_payload(report: "SignalReport",
|
||||
candles: Optional[Dict[str, Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
indicators: Dict[str, Dict[str, Any]] = {}
|
||||
key_levels: Dict[str, float] = {}
|
||||
analysis = report.primary_analysis
|
||||
if analysis is not None:
|
||||
# Same visibility filter as main.report_to_dict: private series
|
||||
# ride as _-prefixed pseudo-indicators, weight-0 = context-only.
|
||||
indicators = {
|
||||
name: {"signal": ind.signal.name,
|
||||
"confidence": round(ind.confidence)}
|
||||
for name, ind in analysis.indicators.items()
|
||||
if not name.startswith("_") and ind.weight != 0
|
||||
}
|
||||
for name, ind in analysis.indicators.items():
|
||||
if name.startswith("_") or ind.weight == 0:
|
||||
continue
|
||||
entry: Dict[str, Any] = {"signal": ind.signal.name,
|
||||
"confidence": round(ind.confidence)}
|
||||
value = _finite(getattr(ind, "value", None))
|
||||
if value is not None:
|
||||
entry["value"] = round(value, 2)
|
||||
indicators[name] = entry
|
||||
key_levels = {k: round(v, 2) for k, v in analysis.key_levels.items()}
|
||||
return {
|
||||
payload: Dict[str, Any] = {
|
||||
"brief": _brief(report),
|
||||
"indicators": indicators,
|
||||
"mtf": {tf: {"signal": sig.name,
|
||||
@@ -309,6 +356,9 @@ def _technical_payload(report: "SignalReport") -> Dict[str, Any]:
|
||||
"key_levels": key_levels,
|
||||
"exhaustion": report.exhaustion_count,
|
||||
}
|
||||
if candles:
|
||||
payload["candles"] = candles
|
||||
return payload
|
||||
|
||||
|
||||
def _risk_payload(report: "SignalReport") -> Dict[str, Any]:
|
||||
@@ -344,6 +394,81 @@ def _contrarian_payload(report: "SignalReport") -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
# ── Candle context (ATR-normalized; technical persona only) ─────────
|
||||
#
|
||||
# Raw prices are the LLM's weakest input modality (measured in-repo:
|
||||
# numeric-comparison misreads, backtest/research/llm_news_eval_2026-08-06.md),
|
||||
# so code does the arithmetic and the model reads a normalized SHAPE:
|
||||
# each bar's [o,h,l,c] as offsets from the latest close in ATR units.
|
||||
# A panel-private provider singleton keeps the TTL cache warm across
|
||||
# attaches; any failure degrades to "no candles", never a failed panel.
|
||||
|
||||
_TF_BY_VALUE = {tf.value: tf for tf in Timeframe}
|
||||
|
||||
# Patchable for tests (mirrors the llm_client._client stub pattern).
|
||||
_dp: Any = None
|
||||
|
||||
|
||||
def _panel_provider() -> Any:
|
||||
global _dp
|
||||
if _dp is None:
|
||||
from config import PlatformConfig
|
||||
from providers import make_data_provider
|
||||
_dp = make_data_provider(PlatformConfig())
|
||||
return _dp
|
||||
|
||||
|
||||
def _normalize_candles(df: Any, atr: float, bars: int) -> Dict[str, Any]:
|
||||
"""Last `bars` closed candles as ATR-unit offsets from the final
|
||||
close: 0.0 = latest close, +1.0 = one ATR above it. Oldest-first."""
|
||||
tail = df.tail(bars)
|
||||
ref = float(tail["Close"].iloc[-1])
|
||||
out = []
|
||||
for _, row in tail.iterrows():
|
||||
out.append([round((float(row["Open"]) - ref) / atr, 1),
|
||||
round((float(row["High"]) - ref) / atr, 1),
|
||||
round((float(row["Low"]) - ref) / atr, 1),
|
||||
round((float(row["Close"]) - ref) / atr, 1)])
|
||||
return {"ref_close": round(ref, 2), "atr": round(float(atr), 2),
|
||||
"bars": out}
|
||||
|
||||
|
||||
def _candle_context(report: "SignalReport",
|
||||
settings: Dict[str, Any]) -> Optional[Dict[str, Any]]:
|
||||
"""Best-effort {"<tf>": _normalize_candles(...)} per analysis TF;
|
||||
None = omit the key from the payload. NEVER raises."""
|
||||
try:
|
||||
if not settings.get("candles_enabled", True):
|
||||
return None
|
||||
confirmations = list(report.confirmation_analyses or [])
|
||||
analyses = [report.primary_analysis] + confirmations
|
||||
counts = [int(settings.get("candle_bars_primary",
|
||||
DEFAULT_CANDLE_BARS_PRIMARY))] + [
|
||||
int(settings.get("candle_bars_confirm",
|
||||
DEFAULT_CANDLE_BARS_CONFIRM))
|
||||
] * len(confirmations)
|
||||
dp = _panel_provider()
|
||||
out: Dict[str, Any] = {}
|
||||
for analysis, n in zip(analyses, counts):
|
||||
if analysis is None:
|
||||
continue
|
||||
atr = _finite(getattr(
|
||||
(analysis.indicators or {}).get("atr"), "value", None))
|
||||
if not atr or atr <= 0:
|
||||
continue
|
||||
tf = _TF_BY_VALUE.get(analysis.timeframe)
|
||||
if tf is None:
|
||||
continue
|
||||
df = dp.fetch_ohlcv(report.symbol, tf)
|
||||
if df is None or len(df) < 2:
|
||||
continue
|
||||
out[analysis.timeframe] = _normalize_candles(df, atr, n)
|
||||
return out or None
|
||||
except Exception as e:
|
||||
logger.debug("agent panel: candle context unavailable: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
# ── Orchestration ────────────────────────────────────────────────────
|
||||
|
||||
# Panel-private pool: the 4 persona calls run concurrently WITHOUT
|
||||
@@ -385,6 +510,7 @@ def _ask_persona(persona: str, system: str,
|
||||
|
||||
def run_panel(report: "SignalReport",
|
||||
context: Optional["FundamentalContext"] = None,
|
||||
candles: Optional[Dict[str, Any]] = None,
|
||||
) -> Optional[PanelVerdict]:
|
||||
"""Run 4 personas + synthesizer; None = no verdict (fail-safe).
|
||||
|
||||
@@ -398,7 +524,7 @@ def run_panel(report: "SignalReport",
|
||||
return None
|
||||
jobs = (
|
||||
("macro", _MACRO_SYSTEM, _macro_payload(report, context)),
|
||||
("technical", _TECH_SYSTEM, _technical_payload(report)),
|
||||
("technical", _TECH_SYSTEM, _technical_payload(report, candles)),
|
||||
("risk", _RISK_SYSTEM, _risk_payload(report)),
|
||||
("contrarian", _CONTRARIAN_SYSTEM, _contrarian_payload(report)),
|
||||
)
|
||||
@@ -502,6 +628,20 @@ def _write_fields(report: "SignalReport", v: PanelVerdict) -> None:
|
||||
"confidence": t.confidence, "note": t.note} for t in v.takes]
|
||||
|
||||
|
||||
def _log_panel_verdict(report: "SignalReport") -> None:
|
||||
"""Persist the verdict onto the scan's signals.db row (fail-safe).
|
||||
|
||||
The row is inserted inside analyze_pair — BEFORE the panel attaches
|
||||
post-hoc — so panel columns arrive via this update. A report that was
|
||||
never logged (backtest replay, logging disabled) is a silent no-op.
|
||||
"""
|
||||
try:
|
||||
from signal_log import log_panel_update
|
||||
log_panel_update(report)
|
||||
except Exception:
|
||||
logger.debug("agent panel: verdict logging failed", exc_info=True)
|
||||
|
||||
|
||||
def attach_panel(report: "SignalReport",
|
||||
settings: Optional[Dict[str, Any]] = None,
|
||||
surface: str = "telegram",
|
||||
@@ -548,6 +688,7 @@ def attach_panel(report: "SignalReport",
|
||||
hit = prior
|
||||
if hit is not None:
|
||||
_write_fields(report, hit)
|
||||
_log_panel_verdict(report)
|
||||
return True
|
||||
|
||||
cap = int(settings.get("max_panels_per_day",
|
||||
@@ -567,7 +708,8 @@ def attach_panel(report: "SignalReport",
|
||||
logger.debug("agent panel: news context unavailable: %s", e)
|
||||
context = None
|
||||
|
||||
verdict = run_panel(report, context)
|
||||
candles = _candle_context(report, settings)
|
||||
verdict = run_panel(report, context, candles=candles)
|
||||
if verdict is None:
|
||||
return False
|
||||
_write_fields(report, verdict)
|
||||
@@ -576,6 +718,7 @@ def attach_panel(report: "SignalReport",
|
||||
while len(_cache) > _CACHE_MAX:
|
||||
_cache.popitem(last=False)
|
||||
_last_by_dir[dir_key] = verdict
|
||||
_log_panel_verdict(report)
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("agent panel: attach failed — signal unaffected")
|
||||
|
||||
@@ -97,6 +97,7 @@ _KNOWN_DEEPSEEK = {
|
||||
_KNOWN_AGENT_PANEL = {
|
||||
"enabled", "telegram", "dashboard", "min_quality",
|
||||
"max_panels_per_day", "reuse_ttl_seconds",
|
||||
"candles_enabled", "candle_bars_primary", "candle_bars_confirm",
|
||||
}
|
||||
_KNOWN_SIGNAL_LOG = {"enabled", "path"}
|
||||
_KNOWN_TELEGRAM = {"token", "chat_id", "channel_id", "alert_min_quality"}
|
||||
@@ -306,6 +307,18 @@ def load_app_config(path: str = CONFIG_PATH) -> AppConfig:
|
||||
raise AppConfigError(
|
||||
"config.json: agent_panel.reuse_ttl_seconds must be a "
|
||||
"non-negative number")
|
||||
if "candles_enabled" in panel \
|
||||
and not isinstance(panel["candles_enabled"], bool):
|
||||
raise AppConfigError(
|
||||
"config.json: agent_panel.candles_enabled must be true or false")
|
||||
for key in ("candle_bars_primary", "candle_bars_confirm"):
|
||||
if key in panel:
|
||||
n = panel[key]
|
||||
if isinstance(n, bool) or not isinstance(n, int) \
|
||||
or not 1 <= n <= 100:
|
||||
raise AppConfigError(
|
||||
f"config.json: agent_panel.{key} must be an integer "
|
||||
"between 1 and 100")
|
||||
app.agent_panel_settings = dict(panel)
|
||||
|
||||
slog = raw.get("signal_log", {})
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
# AI signal enrichment — analysis + build record (2026-08-15)
|
||||
|
||||
Owner questions:
|
||||
1. If the AI panel also received indicator values and (limited) candle data, would the system be more accurate?
|
||||
2. If AI reviewed every candle alongside the local score system, would prediction accuracy justify the token burn — or is there a middle ground?
|
||||
|
||||
## What the panel already received (pre-change baseline)
|
||||
|
||||
The four personas (`agent_panel.py`) already got a *distilled* picture:
|
||||
per-indicator signal+confidence, MTF signals/scores, key levels,
|
||||
exhaustion count, full trade-setup math, warnings, headlines/events/
|
||||
sentiment. Missing: raw indicator **values** and any **candle** data.
|
||||
|
||||
Cost guardrails already in place: 5 requests/panel, `max_panels_per_day`
|
||||
20 (≤100 requests/day), 3600 s reuse-TTL, per-bar exact cache, shared
|
||||
500 req/day DeepSeek cap, static prompts on the ~50× prefix cache.
|
||||
|
||||
## Evidence used
|
||||
|
||||
- **LLM is strong at text, weak at raw numerics** (measured in-repo,
|
||||
`llm_news_eval_2026-08-06.md`): headline sentiment 100% (78/78) vs
|
||||
keyword scorer 31%; but ~11% of event-direction calls misread close
|
||||
numeric comparisons until code pre-computed the comparison
|
||||
(`vs_forecast`) and the model only mapped it to doctrine.
|
||||
- **The book any filter would act on** (M5 canonical,
|
||||
`oanda_m5_12mo_065_nopb_20260814.csv`): 577 trades, +106.82R,
|
||||
PF 1.51, win rate 52.9%; A-grade 59.5%, B-grade 51.4% (+67.8R of the
|
||||
total), C-grade 43.8% (−0.23R, 16 trades). Average edge ≈ 0.185R/trade.
|
||||
|
||||
## Answer 1 — enrich the existing panel? YES (cheap, safe)
|
||||
|
||||
- A compact candle is ~15–25 tokens; 20 primary-TF + 10 confirm-TF bars
|
||||
≈ 500–1,500 tokens per persona call → cents/day at the 20-panel cap.
|
||||
**Tokens are not the constraint; the request cap is.**
|
||||
- Measured after the build (dry-test wire log): enriched technical
|
||||
payload with 20 candles ≈ **~380 tokens**; replies ≈ 25–35 tokens
|
||||
under the new 15-word note cap.
|
||||
- Caveats: (a) the panel is advisory-only — richer payloads improve the
|
||||
human's context, not the book's P&L; (b) raw prices are the model's
|
||||
weakest input, so candles are sent **ATR-normalized** (code does the
|
||||
arithmetic, LLM reads the shape) — the same pattern that fixed the
|
||||
event-direction misreads.
|
||||
|
||||
## Answer 2 — AI on every candle? NO
|
||||
|
||||
- **Feasibility**: M5+M15+H1+H4 ≈ 414 bars/day → one call/bar nearly
|
||||
exhausts the shared 500/day cap (news needs it too); a panel per bar
|
||||
≈ 2,070/day = 4× over. Most of that spend reviews HOLD bars.
|
||||
- **Parity (rule 4)**: an AI gate cannot be replayed by the backtester
|
||||
(news off; no recorded verdicts) — every pinned book becomes
|
||||
non-comparable, and model-version drift silently mutates the strategy.
|
||||
- **Edge**: the engine already compresses 380 bars into 9 tuned
|
||||
indicators + regime + MTF validated over 5 years; an LLM's generic
|
||||
prior on raw candles won't beat fitted thresholds, and the margin it
|
||||
would filter is ~0.185R/trade. B-grade holds +67.8R at 51.4% win —
|
||||
a blunt veto is net-negative.
|
||||
|
||||
## The middle ground (adopted)
|
||||
|
||||
1. **Enrich payloads** (built today, see below) — near-zero cost, no
|
||||
rule-1 exposure.
|
||||
2. **Log panel verdicts to signals.db** (built today) — creates the
|
||||
"does OPPOSE predict losers?" dataset with zero behavior change.
|
||||
3. **After 4–8 weeks**: join logged verdicts to outcomes; simulate an
|
||||
OPPOSE-veto against the pinned book offline. Only then consider any
|
||||
gating, as a full rule-1 revalidation.
|
||||
|
||||
## Build record (2026-08-15, owner-approved)
|
||||
|
||||
- `agent_panel.py`
|
||||
- Technical persona payload now carries indicator **values** (units
|
||||
legend added to the prompt) and **ATR-normalized candles** per TF:
|
||||
`{"ref_close", "atr", "bars": [[o,h,l,c]…]}` oldest-first, each price
|
||||
as `(price − ref_close) / atr`, 1 decimal. Window: 20 bars primary /
|
||||
10 per confirmation TF.
|
||||
- New config keys (validated in `app_config.py`, in
|
||||
`config.json.example`): `candles_enabled` (default true),
|
||||
`candle_bars_primary` (20), `candle_bars_confirm` (10).
|
||||
- Panel-private provider singleton (`_dp`, patchable in tests);
|
||||
TTL cache keeps fetches near-free; any failure → candles silently
|
||||
omitted, panel unaffected.
|
||||
- Prompt hardening: training data declared stale and OFF-LIMITS;
|
||||
"if a fact is not in the JSON, treat it as unknown"; thin evidence
|
||||
→ neutral < 30 (unchanged); note cap 25→**15 words**, rationale
|
||||
40→**30 words** (output-token savings).
|
||||
- `attach_panel` now calls `signal_log.log_panel_update(report)` after
|
||||
every successful attach (fresh, exact-cache, and reuse-TTL paths).
|
||||
- `signal_log.py`
|
||||
- `log()` returns the rowid; `log_signal` stashes it on the report as
|
||||
`_signal_log_rowid`; new `log_panel_update()` fills the 4 panel
|
||||
columns on that row post-hoc. No rowid (backtest replay, logging
|
||||
disabled, other process) → silent no-op. Fail-safe contract kept.
|
||||
- Tests: 381 → **388 passed** (7 new: candle normalization math, payload
|
||||
values/candles, context windows + ATR gating + disable flag, attach
|
||||
injection (technical persona only), prompt-contract assertions,
|
||||
panel row update, rowid hand-off + no-op paths). Existing attach tests
|
||||
stub `ap._dp` to stay off the network.
|
||||
- **Live dry test** (`panel_payload_drytest_2026-08-15.py`, fabricated
|
||||
numbers only, 13 real DeepSeek requests total): **7/7 probes pass** —
|
||||
coherent-bullish → support@70; bearish-vs-BUY → oppose@85; genuinely
|
||||
mixed → neutral@30; empty payload → neutral@20 (contract); no invented
|
||||
numbers in any note; all notes ≤ 15 words; synthesizer AGREE with a
|
||||
21-word rationale. Raw wire log:
|
||||
`backtest/research/panel_payload_drytest_2026-08-15.jsonl`.
|
||||
- Notable: an early P3 variant with H1 SELL against a BUY brief got
|
||||
oppose@60 — *correct* per the prompt (higher-TF disagreement is a
|
||||
listed conflict), so the probe was tightened to a genuinely mixed
|
||||
scenario. Documented so nobody "fixes" the prompt for it later.
|
||||
|
||||
## What this deliberately did NOT change
|
||||
|
||||
- No gate reads panel fields; the panel remains advisory-only.
|
||||
- No backtest-path change (`analyze_from_dataframes` untouched; parity
|
||||
test still locks panel fields empty in replay).
|
||||
- No verdict-based filtering — that decision waits for the signals.db
|
||||
dataset (4–8 weeks) and an offline veto replay against the pinned book.
|
||||
@@ -0,0 +1,7 @@
|
||||
{"probe": "P1 coherent-bullish", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_up", "volatility_regime": "normal", "mtf_agreement": true}, "indicators": {"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0}, "rsi": {"signal": "BUY", "confidence": 65, "value": 58.0}, "macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25}, "bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7}, "adx": {"signal": "BUY", "confidence": 70, "value": 32.0}, "roc": {"signal": "BUY", "confidence": 55, "value": 0.35}, "keltner": {"signal": "BUY", "confidence": 65, "value": 3.1}, "candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0}}, "mtf": {"5min": {"signal": "BUY", "score": 30.0}, "1h": {"signal": "STRONG_BUY", "score": 45.0}}, "key_levels": {"resistance": 2012.0, "support": 1985.0}, "exhaustion": 0, "candles": {"15min": {"ref_close": 2000.0, "atr": 5.0, "bars": [[-2.4, -2.2, -2.4, -2.3], [-2.3, -2.1, -2.3, -2.2], [-2.2, -2.0, -2.2, -2.0], [-2.0, -1.9, -2.1, -1.9], [-1.9, -1.8, -2.0, -1.8], [-1.8, -1.6, -1.8, -1.7], [-1.7, -1.5, -1.7, -1.6], [-1.6, -1.4, -1.6, -1.4], [-1.4, -1.3, -1.5, -1.3], [-1.3, -1.2, -1.4, -1.2], [-1.2, -1.0, -1.2, -1.1], [-1.1, -0.9, -1.1, -1.0], [-1.0, -0.8, -1.0, -0.8], [-0.8, -0.7, -0.9, -0.7], [-0.7, -0.6, -0.8, -0.6], [-0.6, -0.4, -0.6, -0.5], [-0.5, -0.3, -0.5, -0.4], [-0.4, -0.2, -0.4, -0.2], [-0.2, -0.1, -0.3, -0.1], [-0.1, 0.0, -0.2, 0.0]]}}}, "raw_reply": {"stance": "support", "confidence": 70, "note": "The trend is up and all signs agree, so buying fits the flow."}}
|
||||
{"probe": "P2 bearish-vs-BUY-brief", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_down", "volatility_regime": "normal", "mtf_agreement": false}, "indicators": {"ema_trend": {"signal": "STRONG_SELL", "confidence": 85, "value": 0.1}, "rsi": {"signal": "SELL", "confidence": 70, "value": 74.0}, "macd": {"signal": "STRONG_SELL", "confidence": 80, "value": -1.4}, "bollinger": {"signal": "SELL", "confidence": 60, "value": 0.95}, "adx": {"signal": "SELL", "confidence": 70, "value": 35.0}, "roc": {"signal": "SELL", "confidence": 60, "value": -0.5}, "keltner": {"signal": "SELL", "confidence": 65, "value": -2.8}, "candlestick": {"signal": "SELL", "confidence": 55, "value": -2.0}}, "mtf": {"5min": {"signal": "SELL", "score": -28.0}, "1h": {"signal": "STRONG_SELL", "score": -41.0}}, "key_levels": {"resistance": 2012.0, "support": 1985.0}, "exhaustion": 3, "candles": {"15min": {"ref_close": 2000.0, "atr": 5.0, "bars": [[2.4, 2.4, 2.2, 2.3], [2.3, 2.3, 2.1, 2.2], [2.2, 2.2, 2.0, 2.0], [2.0, 2.1, 1.9, 1.9], [1.9, 2.0, 1.8, 1.8], [1.8, 1.8, 1.6, 1.7], [1.7, 1.7, 1.5, 1.6], [1.6, 1.6, 1.4, 1.4], [1.4, 1.5, 1.3, 1.3], [1.3, 1.4, 1.2, 1.2], [1.2, 1.2, 1.0, 1.1], [1.1, 1.1, 0.9, 1.0], [1.0, 1.0, 0.8, 0.8], [0.8, 0.9, 0.7, 0.7], [0.7, 0.8, 0.6, 0.6], [0.6, 0.6, 0.4, 0.5], [0.5, 0.5, 0.3, 0.4], [0.4, 0.4, 0.2, 0.2], [0.2, 0.3, 0.1, 0.1], [0.1, 0.2, -0.0, 0.0]]}}}, "raw_reply": {"stance": "oppose", "confidence": 85, "note": "The price is falling and every indicator says sell, so buying fights the trend."}}
|
||||
{"probe": "P3 conflicting", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 12.0, "confidence": 35, "price": 2000.0, "market_regime": "ranging", "volatility_regime": "normal", "mtf_agreement": false}, "indicators": {"ema_trend": {"signal": "BUY", "confidence": 55, "value": 0.6}, "rsi": {"signal": "NEUTRAL", "confidence": 40, "value": 51.0}, "macd": {"signal": "SELL", "confidence": 55, "value": -0.3}, "bollinger": {"signal": "NEUTRAL", "confidence": 35, "value": 0.5}, "adx": {"signal": "NEUTRAL", "confidence": 40, "value": 18.0}, "roc": {"signal": "SELL", "confidence": 50, "value": -0.1}, "keltner": {"signal": "BUY", "confidence": 50, "value": 0.8}, "candlestick": {"signal": "NEUTRAL", "confidence": 30, "value": 0.0}}, "mtf": {"5min": {"signal": "NEUTRAL", "score": 4.0}, "1h": {"signal": "NEUTRAL", "score": -3.0}}, "key_levels": {}, "exhaustion": 0, "candles": {"15min": {"ref_close": 2000.0, "atr": 5.0, "bars": [[0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0], [0.0, 0.4, -0.1, 0.3], [0.3, 0.4, -0.1, 0.0]]}}}, "raw_reply": {"stance": "neutral", "confidence": 30, "note": "Mixed signals with no clear direction, so cannot strongly support or oppose."}}
|
||||
{"probe": "P4 empty-evidence", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_up", "volatility_regime": "normal", "mtf_agreement": true}, "indicators": {}, "mtf": {}, "key_levels": {}, "exhaustion": 0}, "raw_reply": {"stance": "neutral", "confidence": 20, "note": "Not enough data to judge the trade direction."}}
|
||||
{"probe": "P5 hallucination-trap", "payload": {"brief": {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY", "quality": "A", "score": 38.5, "confidence": 62, "price": 2000.0, "market_regime": "trending_up", "volatility_regime": "normal", "mtf_agreement": true}, "indicators": {"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0}, "rsi": {"signal": "BUY", "confidence": 65, "value": 58.0}, "macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25}, "bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7}, "adx": {"signal": "BUY", "confidence": 70, "value": 32.0}, "roc": {"signal": "BUY", "confidence": 55, "value": 0.35}, "keltner": {"signal": "BUY", "confidence": 65, "value": 3.1}, "candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0}}, "mtf": {"5min": {"signal": "BUY", "score": 30.0}, "1h": {"signal": "STRONG_BUY", "score": 45.0}}, "key_levels": {}, "exhaustion": 0}, "raw_reply": {"stance": "support", "confidence": 70, "note": "The trend is up and all signs agree, so buying fits the flow."}}
|
||||
{"probe": "P6 output-discipline", "payload": {}, "raw_reply": null}
|
||||
{"probe": "P7 synthesizer", "payload": {"direction": "BUY", "takes": [{"persona": "macro", "stance": "support", "confidence": 70, "note": "evidence aligned"}, {"persona": "technical", "stance": "support", "confidence": 80, "note": "evidence aligned"}, {"persona": "risk", "stance": "support", "confidence": 65, "note": "evidence aligned"}, {"persona": "contrarian", "stance": "support", "confidence": 60, "note": "evidence aligned"}]}, "raw_reply": {"verdict": "AGREE", "confidence": 85, "rationale": "All four analysts support buying, with high confidence and consistent evidence. No one opposes, so the panel clearly backs the trade."}}
|
||||
@@ -0,0 +1,271 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
panel_payload_drytest_2026-08-15.py — live DeepSeek validation of the
|
||||
enriched agent-panel payloads + hardened prompts (2026-08-15).
|
||||
|
||||
Spends ~6 REAL DeepSeek requests (against the shared 500/day cap) with
|
||||
FABRICATED payloads — made-up prices/indicators, no real market data —
|
||||
to verify the new technical-persona contract end to end:
|
||||
|
||||
P1 coherent bullish evidence -> technical persona SUPPORTS
|
||||
P2 bearish evidence vs BUY brief -> technical persona OPPOSES
|
||||
P3 conflicting evidence -> NEUTRAL or low confidence
|
||||
P4 empty evidence -> NEUTRAL with confidence < 30
|
||||
(the anti-hallucination contract)
|
||||
P5 hallucination trap -> note invents no price levels
|
||||
P6 output discipline -> <=20-word notes, exact JSON keys
|
||||
P7 synthesizer over 4 supports -> AGREE verdict
|
||||
|
||||
Usage: venv/bin/python backtest/research/panel_payload_drytest_2026-08-15.py
|
||||
Exit code 0 = all probes passed. Raw replies land in
|
||||
backtest/research/panel_drytest_2026-08-15.jsonl
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
import agent_panel as ap # noqa: E402
|
||||
from app_config import deepseek_api_key, get_app_config # noqa: E402
|
||||
from llm_client import get_llm_client # noqa: E402
|
||||
|
||||
ARTIFACT = Path(__file__).with_suffix(".jsonl")
|
||||
|
||||
|
||||
# ── Fabrication helpers (all numbers are invented) ───────────────────
|
||||
|
||||
def _staircase(start: float, step: float, n: int) -> pd.DataFrame:
|
||||
"""n synthetic bars drifting `step` per bar; last close = start+n*step."""
|
||||
rows, price = [], start
|
||||
for _ in range(n):
|
||||
o, c = price, price + step
|
||||
rows.append({"Open": o, "Close": c,
|
||||
"High": max(o, c) + abs(step) * 0.4,
|
||||
"Low": min(o, c) - abs(step) * 0.4})
|
||||
price = c
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
def _choppy(center: float, n: int) -> pd.DataFrame:
|
||||
rows, price, sign = [], center, 1
|
||||
for _ in range(n):
|
||||
step = sign * 1.5
|
||||
o, c = price, price + step
|
||||
rows.append({"Open": o, "Close": c,
|
||||
"High": max(o, c) + 0.6, "Low": min(o, c) - 0.6})
|
||||
price, sign = c, -sign
|
||||
return pd.DataFrame(rows)
|
||||
|
||||
|
||||
_FAKE_PRICE = 2000.00 # deliberately made-up
|
||||
_FAKE_ATR = 5.0
|
||||
|
||||
_BRIEF_BUY = {"pair": "XAU/USD", "timeframe": "15min", "direction": "BUY",
|
||||
"quality": "A", "score": 38.5, "confidence": 62,
|
||||
"price": _FAKE_PRICE, "market_regime": "trending_up",
|
||||
"volatility_regime": "normal", "mtf_agreement": True}
|
||||
|
||||
_BULL_INDICATORS = {
|
||||
"ema_trend": {"signal": "STRONG_BUY", "confidence": 90, "value": 1.0},
|
||||
"rsi": {"signal": "BUY", "confidence": 65, "value": 58.0},
|
||||
"macd": {"signal": "STRONG_BUY", "confidence": 80, "value": 1.25},
|
||||
"bollinger": {"signal": "BUY", "confidence": 60, "value": 0.7},
|
||||
"adx": {"signal": "BUY", "confidence": 70, "value": 32.0},
|
||||
"roc": {"signal": "BUY", "confidence": 55, "value": 0.35},
|
||||
"keltner": {"signal": "BUY", "confidence": 65, "value": 3.1},
|
||||
"candlestick": {"signal": "BUY", "confidence": 60, "value": 2.0},
|
||||
}
|
||||
|
||||
_BEAR_INDICATORS = {
|
||||
"ema_trend": {"signal": "STRONG_SELL", "confidence": 85, "value": 0.1},
|
||||
"rsi": {"signal": "SELL", "confidence": 70, "value": 74.0},
|
||||
"macd": {"signal": "STRONG_SELL", "confidence": 80, "value": -1.4},
|
||||
"bollinger": {"signal": "SELL", "confidence": 60, "value": 0.95},
|
||||
"adx": {"signal": "SELL", "confidence": 70, "value": 35.0},
|
||||
"roc": {"signal": "SELL", "confidence": 60, "value": -0.5},
|
||||
"keltner": {"signal": "SELL", "confidence": 65, "value": -2.8},
|
||||
"candlestick": {"signal": "SELL", "confidence": 55, "value": -2.0},
|
||||
}
|
||||
|
||||
_MIXED_INDICATORS = {
|
||||
"ema_trend": {"signal": "BUY", "confidence": 55, "value": 0.6},
|
||||
"rsi": {"signal": "NEUTRAL", "confidence": 40, "value": 51.0},
|
||||
"macd": {"signal": "SELL", "confidence": 55, "value": -0.3},
|
||||
"bollinger": {"signal": "NEUTRAL", "confidence": 35, "value": 0.5},
|
||||
"adx": {"signal": "NEUTRAL", "confidence": 40, "value": 18.0},
|
||||
"roc": {"signal": "SELL", "confidence": 50, "value": -0.1},
|
||||
"keltner": {"signal": "BUY", "confidence": 50, "value": 0.8},
|
||||
"candlestick": {"signal": "NEUTRAL", "confidence": 30, "value": 0.0},
|
||||
}
|
||||
|
||||
|
||||
def _tech_payload(indicators, mtf, candles_df=None, key_levels=None,
|
||||
brief=None, exhaustion=0):
|
||||
payload = {"brief": brief or dict(_BRIEF_BUY),
|
||||
"indicators": indicators, "mtf": mtf,
|
||||
"key_levels": key_levels or {}, "exhaustion": exhaustion}
|
||||
if candles_df is not None:
|
||||
# the REAL production normalizer, not a hand-rolled copy
|
||||
payload["candles"] = {
|
||||
"15min": ap._normalize_candles(candles_df, _FAKE_ATR,
|
||||
len(candles_df))}
|
||||
return payload
|
||||
|
||||
|
||||
# ── Probe runner ─────────────────────────────────────────────────────
|
||||
|
||||
_RESULTS = []
|
||||
|
||||
|
||||
def _record(probe, payload, reply, take, passed, detail):
|
||||
_RESULTS.append({"probe": probe, "pass": bool(passed),
|
||||
"detail": detail,
|
||||
"stance": getattr(take, "stance", None),
|
||||
"confidence": getattr(take, "confidence", None),
|
||||
"note": getattr(take, "note", None)})
|
||||
with ARTIFACT.open("a") as fh:
|
||||
fh.write(json.dumps({"probe": probe, "payload": payload,
|
||||
"raw_reply": reply}, default=str) + "\n")
|
||||
mark = "PASS" if passed else "FAIL"
|
||||
print(f"[{mark}] {probe}: {detail}")
|
||||
if take is not None:
|
||||
print(f" -> {take.stance} @ {take.confidence:.0f} "
|
||||
f"| {take.note}")
|
||||
|
||||
|
||||
def _ask(payload):
|
||||
"""Real API call through the production client + production
|
||||
validation (the same path run_panel uses)."""
|
||||
reply = get_llm_client().chat_json(ap._TECH_SYSTEM, payload)
|
||||
return reply, ap._validate_take("technical", reply)
|
||||
|
||||
|
||||
def _want(probe: str) -> bool:
|
||||
"""--only P3 runs just that probe (saves requests on re-runs)."""
|
||||
if "--only" not in sys.argv:
|
||||
return True
|
||||
return sys.argv[sys.argv.index("--only") + 1] == probe
|
||||
|
||||
|
||||
def main() -> int:
|
||||
app = get_app_config()
|
||||
if not deepseek_api_key(app):
|
||||
print("ABORT: no DeepSeek API key (env DEEPSEEK_API_KEY / "
|
||||
"config.json deepseek.api_key).")
|
||||
return 2
|
||||
if app.deepseek_settings.get("enabled", True) is False:
|
||||
print("ABORT: deepseek.enabled is false in config.json.")
|
||||
return 2
|
||||
if ARTIFACT.exists() and "--only" not in sys.argv:
|
||||
ARTIFACT.unlink() # full run = fresh wire log; --only appends
|
||||
|
||||
mtf_agree = {"5min": {"signal": "BUY", "score": 30.0},
|
||||
"1h": {"signal": "STRONG_BUY", "score": 45.0}}
|
||||
levels = {"resistance": 2012.0, "support": 1985.0}
|
||||
|
||||
# P1 — coherent bullish, with rising ATR staircase (last close 2000)
|
||||
if _want("P1"):
|
||||
p1 = _tech_payload(_BULL_INDICATORS, mtf_agree,
|
||||
candles_df=_staircase(_FAKE_PRICE - 20 * 0.6,
|
||||
0.6, 20),
|
||||
key_levels=levels)
|
||||
reply, take = _ask(p1)
|
||||
_record("P1 coherent-bullish", p1, reply, take,
|
||||
take is not None and take.stance == "support"
|
||||
and take.confidence >= 50,
|
||||
"expect support @ conf>=50")
|
||||
|
||||
# P2 — bearish everything against a BUY brief, falling staircase
|
||||
if _want("P2"):
|
||||
brief2 = dict(_BRIEF_BUY, market_regime="trending_down",
|
||||
mtf_agreement=False)
|
||||
mtf_sell = {"5min": {"signal": "SELL", "score": -28.0},
|
||||
"1h": {"signal": "STRONG_SELL", "score": -41.0}}
|
||||
p2 = _tech_payload(_BEAR_INDICATORS, mtf_sell, brief=brief2,
|
||||
candles_df=_staircase(_FAKE_PRICE + 20 * 0.6,
|
||||
-0.6, 20),
|
||||
key_levels=levels, exhaustion=3)
|
||||
reply, take = _ask(p2)
|
||||
_record("P2 bearish-vs-BUY-brief", p2, reply, take,
|
||||
take is not None and take.stance == "oppose",
|
||||
"expect oppose (BUY brief fights all evidence)")
|
||||
|
||||
# P3 — genuinely mixed evidence: MTF neutral both ways, ranging
|
||||
# regime, choppy candles. (An H1 SELL against a BUY is a *listed
|
||||
# conflict* in the prompt — oppose is correct there, not mixed.)
|
||||
if _want("P3"):
|
||||
brief3 = dict(_BRIEF_BUY, market_regime="ranging", score=12.0,
|
||||
confidence=35, mtf_agreement=False)
|
||||
mtf_mixed = {"5min": {"signal": "NEUTRAL", "score": 4.0},
|
||||
"1h": {"signal": "NEUTRAL", "score": -3.0}}
|
||||
p3 = _tech_payload(_MIXED_INDICATORS, mtf_mixed, brief=brief3,
|
||||
candles_df=_choppy(_FAKE_PRICE, 20))
|
||||
reply, take = _ask(p3)
|
||||
_record("P3 conflicting", p3, reply, take,
|
||||
take is not None and (take.stance == "neutral"
|
||||
or take.confidence < 60),
|
||||
"expect neutral or conf<60 on genuinely mixed evidence")
|
||||
|
||||
# P4 — empty evidence: contract demands neutral @ conf<30
|
||||
if _want("P4"):
|
||||
p4 = _tech_payload({}, {}, key_levels=None)
|
||||
reply, take = _ask(p4)
|
||||
_record("P4 empty-evidence", p4, reply, take,
|
||||
take is not None and take.stance == "neutral"
|
||||
and take.confidence < 30,
|
||||
"contract: thin/missing evidence -> neutral, conf<30")
|
||||
|
||||
# P5 — hallucination trap: no key_levels, no candles; the note must
|
||||
# not cite any 3+-digit number absent from the payload
|
||||
if _want("P5"):
|
||||
p5 = _tech_payload(_BULL_INDICATORS, mtf_agree, key_levels=None)
|
||||
reply, take = _ask(p5)
|
||||
invented = []
|
||||
if take is not None:
|
||||
payload_blob = json.dumps(p5)
|
||||
for num in re.findall(r"\d{3,}", take.note):
|
||||
if num not in payload_blob:
|
||||
invented.append(num)
|
||||
_record("P5 hallucination-trap", p5, reply, take,
|
||||
take is not None and not invented,
|
||||
f"invented numbers in note: {invented or 'none'}")
|
||||
|
||||
# P6 — output discipline across the persona replies gathered above
|
||||
notes = [r for r in _RESULTS if r["note"]]
|
||||
wordy = [r["probe"] for r in notes if len(r["note"].split()) > 20]
|
||||
_record("P6 output-discipline", {}, None, None,
|
||||
notes and not wordy,
|
||||
f"notes >20 words: {wordy or 'none'} "
|
||||
f"(cap is 15; 20 = fail threshold)")
|
||||
|
||||
# P7 — synthesizer over four fabricated SUPPORT takes
|
||||
if _want("P7"):
|
||||
takes = [{"persona": p, "stance": "support", "confidence": c,
|
||||
"note": "evidence aligned"}
|
||||
for p, c in (("macro", 70), ("technical", 80),
|
||||
("risk", 65), ("contrarian", 60))]
|
||||
synth_payload = {"direction": "BUY", "takes": takes}
|
||||
reply = get_llm_client().chat_json(ap._SYNTH_SYSTEM, synth_payload)
|
||||
verdict = reply.get("verdict") if isinstance(reply, dict) else None
|
||||
rationale = (reply or {}).get("rationale", "")
|
||||
_record("P7 synthesizer", synth_payload, reply, None,
|
||||
verdict == "AGREE" and len(str(rationale).split()) <= 40,
|
||||
f"expect AGREE + <=40-word rationale; got {verdict!r} "
|
||||
f"({len(str(rationale).split())} words)")
|
||||
|
||||
n_pass = sum(1 for r in _RESULTS if r["pass"])
|
||||
n_api = sum(1 for r in _RESULTS if r["probe"] != "P6 output-discipline")
|
||||
print(f"\n{n_pass}/{len(_RESULTS)} probes passed "
|
||||
f"({n_api} API requests spent; "
|
||||
f"raw wire log: {ARTIFACT.name})")
|
||||
return 0 if n_pass == len(_RESULTS) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+4
-1
@@ -46,7 +46,10 @@
|
||||
"dashboard": false,
|
||||
"min_quality": "B",
|
||||
"max_panels_per_day": 20,
|
||||
"reuse_ttl_seconds": 3600
|
||||
"reuse_ttl_seconds": 3600,
|
||||
"candles_enabled": true,
|
||||
"candle_bars_primary": 20,
|
||||
"candle_bars_confirm": 10
|
||||
},
|
||||
"signal_log": {
|
||||
"enabled": true,
|
||||
|
||||
+64
-4
@@ -10,6 +10,13 @@ Hooked ONLY into SignalAggregator.analyze_pair (the live entry point);
|
||||
the backtester calls analyze_from_dataframes directly and therefore
|
||||
never writes here (live == backtest parity — CLAUDE.md rule 4).
|
||||
|
||||
Panel fields arrive POST-HOC: the row is inserted during analyze_pair
|
||||
(before consumers attach the advisory panel), so log_signal stashes the
|
||||
rowid on the report and agent_panel.attach_panel calls
|
||||
log_panel_update() to fill panel_verdict / panel_confidence /
|
||||
panel_rationale / panel_takes_json on that same row. A report without a
|
||||
rowid (backtest replay, logging disabled) is a silent no-op.
|
||||
|
||||
Deployment settings come from the config.json "signal_log" block via
|
||||
app_config (env vars SIGNAL_LOG_ENABLED / SIGNAL_LOG_PATH win):
|
||||
|
||||
@@ -292,10 +299,28 @@ class SignalLogger:
|
||||
context: Any = None,
|
||||
source: str = "live",
|
||||
timeframe: str = "",
|
||||
) -> None:
|
||||
) -> int:
|
||||
"""Insert one scan row; returns the rowid (panel-update key)."""
|
||||
row = _report_row(report, context, source, timeframe)
|
||||
with self._lock:
|
||||
self._conn.execute(_INSERT, row)
|
||||
cur = self._conn.execute(_INSERT, row)
|
||||
self._conn.commit()
|
||||
return int(cur.lastrowid)
|
||||
|
||||
def update_panel(
|
||||
self,
|
||||
rowid: int,
|
||||
verdict: str,
|
||||
confidence: Optional[float],
|
||||
rationale: str,
|
||||
takes_json: str,
|
||||
) -> None:
|
||||
"""Fill the panel columns on an already-inserted scan row."""
|
||||
with self._lock:
|
||||
self._conn.execute(
|
||||
"UPDATE signals SET panel_verdict=?, panel_confidence=?,"
|
||||
" panel_rationale=?, panel_takes_json=? WHERE id=?",
|
||||
(verdict, confidence, rationale, takes_json, rowid))
|
||||
self._conn.commit()
|
||||
|
||||
def close(self) -> None:
|
||||
@@ -316,7 +341,9 @@ def log_signal(
|
||||
timeframe: str = "",
|
||||
) -> None:
|
||||
"""Persist one scan if the signal_log config block enables it.
|
||||
NEVER raises — a logging failure must not break a live scan."""
|
||||
Stashes the rowid on the report (``_signal_log_rowid``) so a post-hoc
|
||||
panel attach can update this same row. NEVER raises — a logging
|
||||
failure must not break a live scan."""
|
||||
try:
|
||||
if not app_config.signal_log_enabled():
|
||||
return
|
||||
@@ -326,12 +353,45 @@ def log_signal(
|
||||
if lg is None:
|
||||
lg = SignalLogger(path)
|
||||
_loggers[path] = lg
|
||||
lg.log(report, context=context, source=source, timeframe=timeframe)
|
||||
rowid = lg.log(report, context=context, source=source,
|
||||
timeframe=timeframe)
|
||||
try:
|
||||
report._signal_log_rowid = rowid
|
||||
except Exception:
|
||||
pass # exotic report object — panel update just won't happen
|
||||
except Exception:
|
||||
logger.warning("signal_log: failed to persist signal",
|
||||
exc_info=True)
|
||||
|
||||
|
||||
def log_panel_update(report: Any) -> None:
|
||||
"""Write post-hoc panel fields onto the row logged for this scan.
|
||||
|
||||
Matches on the private rowid hand-off from log_signal; a report that
|
||||
was never logged (backtest replay, logging disabled, logger in
|
||||
another process) is a silent no-op. NEVER raises."""
|
||||
try:
|
||||
rowid = getattr(report, "_signal_log_rowid", None)
|
||||
if rowid is None or not app_config.signal_log_enabled():
|
||||
return
|
||||
verdict = getattr(report, "panel_verdict", "") or ""
|
||||
if not verdict:
|
||||
return
|
||||
path = app_config.signal_log_path()
|
||||
with _loggers_lock:
|
||||
lg = _loggers.get(path)
|
||||
if lg is None:
|
||||
return
|
||||
lg.update_panel(
|
||||
int(rowid), verdict,
|
||||
_finite(getattr(report, "panel_confidence", None)),
|
||||
getattr(report, "panel_rationale", "") or "",
|
||||
_dumps(getattr(report, "panel_takes", None) or []))
|
||||
except Exception:
|
||||
logger.warning("signal_log: failed to persist panel verdict",
|
||||
exc_info=True)
|
||||
|
||||
|
||||
def reset_loggers_for_tests() -> None:
|
||||
"""Close and drop all cached loggers (test isolation)."""
|
||||
with _loggers_lock:
|
||||
|
||||
+224
-1
@@ -6952,10 +6952,14 @@ def _():
|
||||
app = load_app_config(_write_json({"agent_panel": {
|
||||
"enabled": True, "telegram": True, "dashboard": False,
|
||||
"min_quality": "A", "max_panels_per_day": 10,
|
||||
"reuse_ttl_seconds": 600}}))
|
||||
"reuse_ttl_seconds": 600, "candles_enabled": False,
|
||||
"candle_bars_primary": 30, "candle_bars_confirm": 12}}))
|
||||
assert app.agent_panel_settings["enabled"] is True
|
||||
assert app.agent_panel_settings["min_quality"] == "A"
|
||||
assert app.agent_panel_settings["max_panels_per_day"] == 10
|
||||
assert app.agent_panel_settings["candles_enabled"] is False
|
||||
assert app.agent_panel_settings["candle_bars_primary"] == 30
|
||||
assert app.agent_panel_settings["candle_bars_confirm"] == 12
|
||||
assert load_app_config(_write_json({})).agent_panel_settings == {}
|
||||
assert AppConfig().agent_panel_settings == {}
|
||||
|
||||
@@ -6971,6 +6975,11 @@ def _():
|
||||
{"agent_panel": {"max_panels_per_day": 0}}, # not positive
|
||||
{"agent_panel": {"max_panels_per_day": True}}, # bool is not an int
|
||||
{"agent_panel": {"reuse_ttl_seconds": -5}}, # negative
|
||||
{"agent_panel": {"candles_enabled": "yes"}}, # not a bool
|
||||
{"agent_panel": {"candle_bars_primary": 0}}, # not positive
|
||||
{"agent_panel": {"candle_bars_primary": 20.5}}, # not an int
|
||||
{"agent_panel": {"candle_bars_confirm": True}}, # bool is not an int
|
||||
{"agent_panel": {"candle_bars_confirm": 500}}, # absurdly large
|
||||
{"agent_panel": "oops"}, # not an object
|
||||
):
|
||||
try:
|
||||
@@ -6998,6 +7007,23 @@ class _PanelStub:
|
||||
return reply(user_payload) if callable(reply) else reply
|
||||
|
||||
|
||||
class _NullDP:
|
||||
"""Provider stub: candle fetch blows up → _candle_context degrades to
|
||||
None (and tests stay off the network)."""
|
||||
def fetch_ohlcv(self, *args, **kwargs):
|
||||
raise RuntimeError("no network in tests")
|
||||
|
||||
|
||||
class _FakeDP:
|
||||
"""Provider stub serving synthetic OHLCV per timeframe."""
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
|
||||
def fetch_ohlcv(self, symbol, timeframe, lookback_days=None):
|
||||
self.calls.append((symbol, timeframe))
|
||||
return make_ohlcv(60, seed=7)
|
||||
|
||||
|
||||
_TAKE_OK = {"stance": "support", "confidence": 70, "note": "fine"}
|
||||
_SYNTH_OK = {"verdict": "AGREE", "confidence": 75, "rationale": "panel backs it"}
|
||||
_ALL_OK = {ap._MACRO_SYSTEM: _TAKE_OK, ap._TECH_SYSTEM: _TAKE_OK,
|
||||
@@ -7148,6 +7174,7 @@ def _():
|
||||
try:
|
||||
stub = _PanelStub(dict(_ALL_OK))
|
||||
lc._client = stub
|
||||
ap._dp = _NullDP()
|
||||
report = _panel_report()
|
||||
assert ap.attach_panel(report, _PANEL_ON,
|
||||
context=FundamentalContext()) is True
|
||||
@@ -7175,6 +7202,7 @@ def _():
|
||||
assert len(stub.calls) == 10
|
||||
finally:
|
||||
lc._client = None
|
||||
ap._dp = None
|
||||
ap.reset_panel_state()
|
||||
|
||||
|
||||
@@ -7183,6 +7211,7 @@ def _():
|
||||
try:
|
||||
stub = _PanelStub(dict(_ALL_OK))
|
||||
lc._client = stub
|
||||
ap._dp = _NullDP()
|
||||
settings = {**_PANEL_ON, "max_panels_per_day": 1,
|
||||
"reuse_ttl_seconds": 0}
|
||||
assert ap.attach_panel(_panel_report(), settings,
|
||||
@@ -7201,6 +7230,7 @@ def _():
|
||||
assert ap.panel_health()["used_today"] == 1
|
||||
finally:
|
||||
lc._client = None
|
||||
ap._dp = None
|
||||
ap.reset_panel_state()
|
||||
|
||||
|
||||
@@ -7209,6 +7239,7 @@ def _():
|
||||
try:
|
||||
stub = _PanelStub(dict(_ALL_OK))
|
||||
lc._client = stub
|
||||
ap._dp = _NullDP()
|
||||
report = _panel_report()
|
||||
ctx = FundamentalContext()
|
||||
assert ap.attach_panel(report, {}, context=ctx) is False, \
|
||||
@@ -7233,6 +7264,7 @@ def _():
|
||||
"force bypasses enabled/surface/quality"
|
||||
finally:
|
||||
lc._client = None
|
||||
ap._dp = None
|
||||
ap.reset_panel_state()
|
||||
|
||||
|
||||
@@ -7311,6 +7343,7 @@ def _():
|
||||
raise RuntimeError("boom")
|
||||
try:
|
||||
lc._client = _Bomb()
|
||||
ap._dp = _NullDP()
|
||||
report = _panel_report()
|
||||
assert ap.attach_panel(report, _PANEL_ON,
|
||||
context=FundamentalContext()) is False
|
||||
@@ -7323,9 +7356,113 @@ def _():
|
||||
logging.disable(logging.NOTSET)
|
||||
finally:
|
||||
lc._client = None
|
||||
ap._dp = None
|
||||
ap.reset_panel_state()
|
||||
|
||||
|
||||
@test("agent_panel: candle normalization is ATR-relative to final close")
|
||||
def _():
|
||||
import pandas as pd
|
||||
df = pd.DataFrame({
|
||||
"Open": [100.0, 110.0], "High": [112.0, 115.0],
|
||||
"Low": [98.0, 108.0], "Close": [110.0, 112.0],
|
||||
"Volume": [1, 1]})
|
||||
out = ap._normalize_candles(df, atr=4.0, bars=2)
|
||||
assert out["ref_close"] == 112.0 and out["atr"] == 4.0
|
||||
assert out["bars"][0] == [-3.0, 0.0, -3.5, -0.5]
|
||||
assert out["bars"][1] == [-0.5, 0.8, -1.0, 0.0]
|
||||
capped = ap._normalize_candles(df, atr=4.0, bars=50)
|
||||
assert len(capped["bars"]) == 2, "window capped at available bars"
|
||||
|
||||
|
||||
@test("agent_panel: technical payload carries values + optional candles")
|
||||
def _():
|
||||
report = _panel_report()
|
||||
tech = ap._technical_payload(report)
|
||||
assert tech["indicators"], "real analysis must yield indicators"
|
||||
assert "value" in tech["indicators"]["rsi"]
|
||||
assert "value" in tech["indicators"]["adx"]
|
||||
assert "candles" not in tech, "candles only when provided"
|
||||
fake = {"1h": {"ref_close": 2400.0, "atr": 5.0,
|
||||
"bars": [[0.0, 0.5, -0.5, 0.2]]}}
|
||||
tech2 = ap._technical_payload(report, candles=fake)
|
||||
assert tech2["candles"] == fake
|
||||
json.dumps(tech2, sort_keys=True, allow_nan=False)
|
||||
|
||||
|
||||
@test("agent_panel: _candle_context windows, ATR gating, disable flag")
|
||||
def _():
|
||||
try:
|
||||
ap._dp = _FakeDP()
|
||||
report = _panel_report()
|
||||
settings = {**_PANEL_ON, "candle_bars_primary": 20,
|
||||
"candle_bars_confirm": 10}
|
||||
ctx = ap._candle_context(report, settings)
|
||||
tfs = [report.primary_analysis.timeframe] + [
|
||||
a.timeframe for a in report.confirmation_analyses]
|
||||
assert set(ctx) == set(tfs), f"one block per analysis TF: {set(ctx)}"
|
||||
assert len(ctx[tfs[0]]["bars"]) == 20, "primary window"
|
||||
assert len(ctx[tfs[1]]["bars"]) == 10, "confirm window"
|
||||
assert all(len(b) == 4 for b in ctx[tfs[0]]["bars"])
|
||||
json.dumps(ctx, allow_nan=False)
|
||||
|
||||
# disabled flag → omitted entirely, zero fetches
|
||||
ap._dp = _FakeDP()
|
||||
assert ap._candle_context(
|
||||
report, {**settings, "candles_enabled": False}) is None
|
||||
assert ap._dp.calls == [], "disabled must not fetch"
|
||||
|
||||
# degenerate ATR → that TF skipped, the rest survive
|
||||
ap._dp = _FakeDP()
|
||||
report.primary_analysis.indicators["atr"].value = 0.0
|
||||
ctx2 = ap._candle_context(report, settings)
|
||||
assert tfs[0] not in ctx2 and tfs[1] in ctx2
|
||||
|
||||
# provider explosion → None, never raises
|
||||
ap._dp = _NullDP()
|
||||
assert ap._candle_context(report, settings) is None
|
||||
finally:
|
||||
ap._dp = None
|
||||
|
||||
|
||||
@test("agent_panel: attach injects candles into the technical payload only")
|
||||
def _():
|
||||
try:
|
||||
ap._dp = _FakeDP()
|
||||
stub = _PanelStub(dict(_ALL_OK))
|
||||
lc._client = stub
|
||||
report = _panel_report()
|
||||
assert ap.attach_panel(report, _PANEL_ON,
|
||||
context=FundamentalContext()) is True
|
||||
by_prompt = dict(stub.calls)
|
||||
tech_payload = by_prompt[ap._TECH_SYSTEM]
|
||||
assert "candles" in tech_payload
|
||||
primary_tf = report.primary_analysis.timeframe
|
||||
assert len(tech_payload["candles"][primary_tf]["bars"]) == \
|
||||
ap.DEFAULT_CANDLE_BARS_PRIMARY
|
||||
assert "value" in tech_payload["indicators"]["rsi"]
|
||||
for sys_p in (ap._MACRO_SYSTEM, ap._RISK_SYSTEM,
|
||||
ap._CONTRARIAN_SYSTEM):
|
||||
assert "candles" not in by_prompt[sys_p], \
|
||||
"candles are technical-persona evidence only"
|
||||
finally:
|
||||
lc._client = None
|
||||
ap._dp = None
|
||||
ap.reset_panel_state()
|
||||
|
||||
|
||||
@test("agent_panel: prompts forbid outside knowledge and cap output words")
|
||||
def _():
|
||||
assert "OFF-LIMITS" in ap._TAKE_CONTRACT
|
||||
assert "max 15 words" in ap._TAKE_CONTRACT
|
||||
assert "max 30 words" in ap._SYNTH_SYSTEM
|
||||
assert "ATR units" in ap._TECH_SYSTEM
|
||||
for prompt in (ap._MACRO_SYSTEM, ap._TECH_SYSTEM, ap._RISK_SYSTEM,
|
||||
ap._CONTRARIAN_SYSTEM):
|
||||
assert ap._TAKE_CONTRACT in prompt, \
|
||||
"every persona must carry the anti-hallucination contract"
|
||||
|
||||
|
||||
print("\n═══ Module 24: signal_log.py (SQLite signal history) ═══")
|
||||
|
||||
|
||||
@@ -7441,6 +7578,92 @@ def _():
|
||||
os.remove(path)
|
||||
|
||||
|
||||
@test("signal_log: panel verdict updates the scan row post-hoc")
|
||||
def _():
|
||||
import sqlite3
|
||||
import signal_log as slog
|
||||
fd, path = tempfile.mkstemp(suffix=".db")
|
||||
os.close(fd)
|
||||
try:
|
||||
lg = slog.SignalLogger(path)
|
||||
report = _make_log_report()
|
||||
rowid = lg.log(report, context=None, source="test", timeframe="M5")
|
||||
assert rowid >= 1, "log() must return the rowid"
|
||||
report.panel_verdict = "OPPOSE"
|
||||
report.panel_confidence = 64.0
|
||||
report.panel_rationale = "risk too thin"
|
||||
report.panel_takes = [{"persona": "risk", "stance": "oppose",
|
||||
"confidence": 70.0, "note": "n"}]
|
||||
lg.update_panel(rowid, report.panel_verdict, 64.0,
|
||||
report.panel_rationale,
|
||||
slog._dumps(report.panel_takes))
|
||||
lg.close()
|
||||
conn = sqlite3.connect(path)
|
||||
row = conn.execute(
|
||||
"SELECT panel_verdict, panel_confidence, panel_rationale,"
|
||||
" panel_takes_json FROM signals WHERE id=?",
|
||||
(rowid,)).fetchone()
|
||||
conn.close()
|
||||
assert row[0] == "OPPOSE" and row[1] == 64.0
|
||||
assert row[2] == "risk too thin"
|
||||
assert json.loads(row[3])[0]["persona"] == "risk"
|
||||
finally:
|
||||
slog.reset_loggers_for_tests()
|
||||
os.remove(path)
|
||||
|
||||
|
||||
@test("signal_log: log_panel_update rowid hand-off; no-ops stay silent")
|
||||
def _():
|
||||
import sqlite3
|
||||
import signal_log as slog
|
||||
fd, path = tempfile.mkstemp(suffix=".db")
|
||||
os.close(fd)
|
||||
os.remove(path) # want a fresh file from SignalLogger
|
||||
try:
|
||||
os.environ["SIGNAL_LOG_PATH"] = path
|
||||
os.environ.pop("SIGNAL_LOG_ENABLED", None)
|
||||
# no rowid (backtest / unlogged report) → silent no-op
|
||||
slog.log_panel_update(_make_log_report())
|
||||
assert not os.path.exists(path), "no-op must not create the DB"
|
||||
|
||||
report = _make_log_report()
|
||||
slog.log_signal(report, source="test", timeframe="M5")
|
||||
assert getattr(report, "_signal_log_rowid", None) is not None, \
|
||||
"log_signal must stash the rowid on the report"
|
||||
|
||||
# empty verdict → no-op (row keeps its empty insert values)
|
||||
slog.log_panel_update(report)
|
||||
conn = sqlite3.connect(path)
|
||||
row = conn.execute(
|
||||
"SELECT panel_verdict, panel_takes_json FROM signals").fetchone()
|
||||
conn.close()
|
||||
assert row[0] in (None, "") and row[1] is None
|
||||
|
||||
report.panel_verdict = "AGREE"
|
||||
report.panel_confidence = 81.0
|
||||
report.panel_rationale = "all aligned"
|
||||
report.panel_takes = []
|
||||
slog.log_panel_update(report)
|
||||
conn = sqlite3.connect(path)
|
||||
row = conn.execute(
|
||||
"SELECT panel_verdict, panel_confidence, panel_rationale,"
|
||||
" panel_takes_json FROM signals").fetchone()
|
||||
conn.close()
|
||||
assert row[0] == "AGREE" and row[1] == 81.0
|
||||
assert row[2] == "all aligned"
|
||||
assert json.loads(row[3]) == []
|
||||
|
||||
# disabled → no-op, never raises
|
||||
os.environ["SIGNAL_LOG_ENABLED"] = "false"
|
||||
slog.log_panel_update(report)
|
||||
finally:
|
||||
os.environ.pop("SIGNAL_LOG_PATH", None)
|
||||
os.environ.pop("SIGNAL_LOG_ENABLED", None)
|
||||
slog.reset_loggers_for_tests()
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
|
||||
|
||||
@test("signal_log: null context → news JSON NULL, row still written")
|
||||
def _():
|
||||
import sqlite3
|
||||
|
||||
Reference in New Issue
Block a user