Harwinder Sekhon
b1e880be4f
- seed an advisory outcome on every setup row; same-setup re-logs within 60 min are DUPLICATE rows (advisory_event_id); resolver rebuilds bar streams across gaps - headlines/events interned into news_snapshots (DB 667 MB -> 262 MB); readers use signals_with_news / read_news and mode=ro (immutable=1 missed WAL rows) - post-gate truth columns: gate_reason, actionable, cleaned_warnings, scan_kind, config_fingerprint; SELL-hour gates fail closed - telegram: breach rescan reuses this bar's report; grade dropped from alert fingerprint (upgrade-only re-alert) - market_outlook: hour-bucket dedup (old rule skipped alternate hours), skip heartbeats - agent_panel: macro-anchored OPPOSE guard (opposed trades were +30R over 3 weeks) - analyze_signal_log --stored-candles/--parity; backfill + news migration scripts - tests: Module 25 (analyze_signal_log) + 27 (observability); 429 total Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
5.7 KiB
Signal-log live performance evaluation — 2026-08-18
Tool: analyze_signal_log.py (new, repo root). Read-only on
logs/signals.db (sqlite immutable=1); replays every logged trade
setup through the backtester's own simulate_trades on the RAW Oanda
price path (next-open fill, SL/TP distances preserved, SL-first ties,
gap-at-open fills, close-only trail ratchet with the per-setup stamped
trail params). Reproduce:
python analyze_signal_log.py --csv backtest/signal_log_eval_20260818.csv
Window & sample
- Log window: 2026-08-13 14:31 → 2026-08-18 14:13 UTC (5 days) — the entire signal_log history to date.
- 2,654 rows (telegram 2,180 / dashboard 474); M5 1,308, M15 1,189, H1 119, H4 38.
- 39 rows carry a
trade_setup→ 17 unique signal events after dedup (cross-source double-logs + persisting-signal re-fires collapse on (tf, direction, entry, SL, strategy) with a 60-min gap). - No real executions exist (
oanda_state.jsonempty, nooanda_daemonrows) — every outcome below is a hypothetical replayed fill, not a broker fill. - Cost: dual convention — $0.65 rt headline, $0.30 comparative (derived analytically, R is linear in cost).
Trade replay results
| First logged | TF | Signal | Grade | Strategy | Fill | Exit | Bars | R@0.65 |
|---|---|---|---|---|---|---|---|---|
| 08-13 15:30 | 5min | SELL | B | GENERIC | 4365.39 | TRAIL | 19 | −0.20 |
| 08-13 18:00 | 15min | SELL | B | GENERIC | 4356.05 | TRAIL | 44 | +0.91 |
| 08-14 07:45 | 5min | BUY | B | XAU_CP | 4339.72 | TRAIL | 60 | +2.56 |
| 08-14 12:15 | 15min | BUY | B | GENERIC | 4371.91 | TRAIL | 27 | +0.06 |
| 08-14 12:25 | 5min | STRONG_BUY | A | GENERIC | 4379.12 | TRAIL | 277 | +0.20 |
| 08-14 12:30 | 5min | STRONG_BUY | A | GENERIC | 4379.87 | TRAIL | 276 | +0.16 |
| 08-14 12:30 | 15min | BUY | B | GENERIC | 4379.87 | TRAIL | 63 | +0.18 |
| 08-17 08:00 | 1h | BUY | C | GENERIC | 4405.19 | TRAIL | 18 | −0.13 |
| 08-17 14:15 | 5min | STRONG_BUY | A | GENERIC | 4412.35 | TRAIL | 46 | −0.10 |
| 08-17 14:20 | 5min | STRONG_BUY | A | GENERIC | 4418.55 | TRAIL | 125 | −0.23 |
| 08-17 14:25 | 5min | STRONG_BUY | A | GENERIC | 4424.10 | SL | 151 | −1.02 |
| 08-17 14:30 | 15min | BUY | A | GENERIC | 4418.23 | SL | 17 | −1.03 |
| 08-17 14:45 | 15min | STRONG_BUY | A | GENERIC | 4417.49 | SL | 16 | −1.03 |
| 08-17 15:00 | 15min | STRONG_BUY | A | GENERIC | 4417.48 | SL | 15 | −1.03 |
| 08-17 15:00 | 5min | NEUTRAL | C | GENERIC | 4417.48 | TRAIL | 117 | −0.16 |
| 08-17 15:15 | 15min | STRONG_BUY | A | GENERIC | 4421.30 | SL | 14 | −1.04 |
| 08-17 15:30 | 15min | BUY | B | GENERIC | 4425.70 | SL | 85 | −1.02 |
Aggregate: 17 closed, 6 wins (35%), total −2.91R @0.65 / −2.67R @0.30, PF 0.58. By grade: A n=9 → −5.12R; B n=6 → +2.50R; C n=2 → −0.30R.
Reading: the book is two regimes. Everything through the 08-14 rally made money (+4.7R across 8 events, incl. the XAU_CP +2.56R); the 08-17 afternoon BUY/STRONG_BUY cluster chased the spike to ~4425 and got stopped as price faded — 9 events, −7.6R combined. One bad afternoon dominates the 5-day sample. Notable: a NEUTRAL-graded row carried a setup (post-gate CP/PB-style path) and lost small.
Advisory panel scorecard
Only 7 verdicts logged so far (2 NEUTRAL, 5 OPPOSE, 0 AGREE) — the panel's min-quality gate + daily budget keep it dark on most scans.
| Row | When | Signal | Verdict | Outcome basis | Impact |
|---|---|---|---|---|---|
| #1358 | 08-17 07:40 M5 | BUY B | NEUTRAL @55% | +4h move | −$8.19 in dir (pass was right) |
| #1361 | 08-17 07:45 M5 | BUY A | NEUTRAL @55% | +4h move | −$12.43 in dir (pass was right) |
| #1550 | 08-17 14:15 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 0.10R |
| #1553 | 08-17 14:25 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 1.02R |
| #1555 | 08-17 14:30 M15 | BUY A | OPPOSE @75% | replay | saved 1.03R |
| #1561 | 08-17 14:45 M15 | STRONG_BUY A | OPPOSE @75% | replay | saved 1.03R |
| #1588 | 08-17 15:30 M15 | BUY B | OPPOSE @75% | replay | saved 1.02R |
Every OPPOSE was correct: ≈ +4.2R would have been lost taking those five trades. The panel's rationale ("price climbed too fast, profit too small") correctly identified the 08-17 exhaustion spike that the composite scored A. Both NEUTRAL passes also preceded adverse moves. Caveat: n=7, one market episode, zero AGREE verdicts — the panel has only proven itself as a veto so far, never as a confirmation.
Directional hit-rate (non-setup signals)
27 deduped non-setup BUY/SELL rows, forward close-to-close on M5:
| Horizon | Hit rate | Mean move in direction |
|---|---|---|
| +1h | 48% | −$0.72 |
| +4h | 44% | −$2.30 |
| +24h | 7% | −$27.45 |
The 24h figure is dominated by the 08-17 buy-cluster reversal; not independently meaningful at n=27.
Method notes / caveats
- Small sample: 5 days, 17 events, one trending episode + one reversal. Directional read only; do not tune anything on this.
- Entry model = next bar open after the scan boundary (live scans run at boundary+10s), SL/TP distances preserved from the logged setup — identical to the backtester replay loop.
- Cross-source dedup verified: telegram+dashboard double-logs and 5-min re-fires of a persisting signal collapse to one event each.
- The stray repo-root
signals.db(828 rows, Aug 13–14, pre-panel schema) is an early duplicate subset oflogs/signals.db— excluded. - Tests: Module 25 in
test_all.pywas only added on 2026-09-03 (the 2026-08-18 claim of 7 tests was never true at HEAD). It now covers boundary flooring, clustering (sharedsignal_log.setup_key), trade building, the stored-candle replay books, surface parity, WAL-visible loading (mode=ro;immutable=1missed the newest rows) and hit-rate signing.