Harwinder Sekhon
b1e880be4f
- seed an advisory outcome on every setup row; same-setup re-logs within 60 min are DUPLICATE rows (advisory_event_id); resolver rebuilds bar streams across gaps - headlines/events interned into news_snapshots (DB 667 MB -> 262 MB); readers use signals_with_news / read_news and mode=ro (immutable=1 missed WAL rows) - post-gate truth columns: gate_reason, actionable, cleaned_warnings, scan_kind, config_fingerprint; SELL-hour gates fail closed - telegram: breach rescan reuses this bar's report; grade dropped from alert fingerprint (upgrade-only re-alert) - market_outlook: hour-bucket dedup (old rule skipped alternate hours), skip heartbeats - agent_panel: macro-anchored OPPOSE guard (opposed trades were +30R over 3 weeks) - analyze_signal_log --stored-candles/--parity; backfill + news migration scripts - tests: Module 25 (analyze_signal_log) + 27 (observability); 429 total Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL
114 lines
5.7 KiB
Markdown
114 lines
5.7 KiB
Markdown
# Signal-log live performance evaluation — 2026-08-18
|
||
|
||
**Tool:** `analyze_signal_log.py` (new, repo root). Read-only on
|
||
`logs/signals.db` (sqlite `immutable=1`); replays every logged trade
|
||
setup through the backtester's own `simulate_trades` on the RAW Oanda
|
||
price path (next-open fill, SL/TP distances preserved, SL-first ties,
|
||
gap-at-open fills, close-only trail ratchet with the per-setup stamped
|
||
trail params). Reproduce:
|
||
|
||
```bash
|
||
python analyze_signal_log.py --csv backtest/signal_log_eval_20260818.csv
|
||
```
|
||
|
||
## Window & sample
|
||
|
||
- Log window: **2026-08-13 14:31 → 2026-08-18 14:13 UTC (5 days)** —
|
||
the entire signal_log history to date.
|
||
- 2,654 rows (telegram 2,180 / dashboard 474); M5 1,308, M15 1,189,
|
||
H1 119, H4 38.
|
||
- 39 rows carry a `trade_setup` → **17 unique signal events** after
|
||
dedup (cross-source double-logs + persisting-signal re-fires collapse
|
||
on (tf, direction, entry, SL, strategy) with a 60-min gap).
|
||
- **No real executions exist** (`oanda_state.json` empty, no
|
||
`oanda_daemon` rows) — every outcome below is a hypothetical replayed
|
||
fill, not a broker fill.
|
||
- Cost: dual convention — **$0.65 rt headline**, $0.30 comparative
|
||
(derived analytically, R is linear in cost).
|
||
|
||
## Trade replay results
|
||
|
||
| First logged | TF | Signal | Grade | Strategy | Fill | Exit | Bars | R@0.65 |
|
||
|---|---|---|---|---|---|---|---|---|
|
||
| 08-13 15:30 | 5min | SELL | B | GENERIC | 4365.39 | TRAIL | 19 | −0.20 |
|
||
| 08-13 18:00 | 15min | SELL | B | GENERIC | 4356.05 | TRAIL | 44 | +0.91 |
|
||
| 08-14 07:45 | 5min | BUY | B | XAU_CP | 4339.72 | TRAIL | 60 | +2.56 |
|
||
| 08-14 12:15 | 15min | BUY | B | GENERIC | 4371.91 | TRAIL | 27 | +0.06 |
|
||
| 08-14 12:25 | 5min | STRONG_BUY | A | GENERIC | 4379.12 | TRAIL | 277 | +0.20 |
|
||
| 08-14 12:30 | 5min | STRONG_BUY | A | GENERIC | 4379.87 | TRAIL | 276 | +0.16 |
|
||
| 08-14 12:30 | 15min | BUY | B | GENERIC | 4379.87 | TRAIL | 63 | +0.18 |
|
||
| 08-17 08:00 | 1h | BUY | C | GENERIC | 4405.19 | TRAIL | 18 | −0.13 |
|
||
| 08-17 14:15 | 5min | STRONG_BUY | A | GENERIC | 4412.35 | TRAIL | 46 | −0.10 |
|
||
| 08-17 14:20 | 5min | STRONG_BUY | A | GENERIC | 4418.55 | TRAIL | 125 | −0.23 |
|
||
| 08-17 14:25 | 5min | STRONG_BUY | A | GENERIC | 4424.10 | SL | 151 | −1.02 |
|
||
| 08-17 14:30 | 15min | BUY | A | GENERIC | 4418.23 | SL | 17 | −1.03 |
|
||
| 08-17 14:45 | 15min | STRONG_BUY | A | GENERIC | 4417.49 | SL | 16 | −1.03 |
|
||
| 08-17 15:00 | 15min | STRONG_BUY | A | GENERIC | 4417.48 | SL | 15 | −1.03 |
|
||
| 08-17 15:00 | 5min | NEUTRAL | C | GENERIC | 4417.48 | TRAIL | 117 | −0.16 |
|
||
| 08-17 15:15 | 15min | STRONG_BUY | A | GENERIC | 4421.30 | SL | 14 | −1.04 |
|
||
| 08-17 15:30 | 15min | BUY | B | GENERIC | 4425.70 | SL | 85 | −1.02 |
|
||
|
||
**Aggregate: 17 closed, 6 wins (35%), total −2.91R @0.65 / −2.67R
|
||
@0.30, PF 0.58.** By grade: A n=9 → −5.12R; B n=6 → +2.50R; C n=2 →
|
||
−0.30R.
|
||
|
||
Reading: the book is two regimes. Everything through the 08-14 rally
|
||
made money (+4.7R across 8 events, incl. the XAU_CP +2.56R); the 08-17
|
||
afternoon BUY/STRONG_BUY cluster chased the spike to ~4425 and got
|
||
stopped as price faded — 9 events, −7.6R combined. One bad afternoon
|
||
dominates the 5-day sample. Notable: a NEUTRAL-graded row carried a
|
||
setup (post-gate CP/PB-style path) and lost small.
|
||
|
||
## Advisory panel scorecard
|
||
|
||
Only **7 verdicts** logged so far (2 NEUTRAL, 5 OPPOSE, 0 AGREE) — the
|
||
panel's min-quality gate + daily budget keep it dark on most scans.
|
||
|
||
| Row | When | Signal | Verdict | Outcome basis | Impact |
|
||
|---|---|---|---|---|---|
|
||
| #1358 | 08-17 07:40 M5 | BUY B | NEUTRAL @55% | +4h move | −$8.19 in dir (pass was right) |
|
||
| #1361 | 08-17 07:45 M5 | BUY A | NEUTRAL @55% | +4h move | −$12.43 in dir (pass was right) |
|
||
| #1550 | 08-17 14:15 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 0.10R |
|
||
| #1553 | 08-17 14:25 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 1.02R |
|
||
| #1555 | 08-17 14:30 M15 | BUY A | OPPOSE @75% | replay | saved 1.03R |
|
||
| #1561 | 08-17 14:45 M15 | STRONG_BUY A | OPPOSE @75% | replay | saved 1.03R |
|
||
| #1588 | 08-17 15:30 M15 | BUY B | OPPOSE @75% | replay | saved 1.02R |
|
||
|
||
**Every OPPOSE was correct: ≈ +4.2R would have been lost taking those
|
||
five trades.** The panel's rationale ("price climbed too fast, profit
|
||
too small") correctly identified the 08-17 exhaustion spike that the
|
||
composite scored A. Both NEUTRAL passes also preceded adverse moves.
|
||
Caveat: n=7, one market episode, zero AGREE verdicts — the panel has
|
||
only proven itself as a veto so far, never as a confirmation.
|
||
|
||
## Directional hit-rate (non-setup signals)
|
||
|
||
27 deduped non-setup BUY/SELL rows, forward close-to-close on M5:
|
||
|
||
| Horizon | Hit rate | Mean move in direction |
|
||
|---|---|---|
|
||
| +1h | 48% | −$0.72 |
|
||
| +4h | 44% | −$2.30 |
|
||
| +24h | 7% | −$27.45 |
|
||
|
||
The 24h figure is dominated by the 08-17 buy-cluster reversal; not
|
||
independently meaningful at n=27.
|
||
|
||
## Method notes / caveats
|
||
|
||
- **Small sample**: 5 days, 17 events, one trending episode + one
|
||
reversal. Directional read only; do not tune anything on this.
|
||
- Entry model = next bar open after the scan boundary (live scans run
|
||
at boundary+10s), SL/TP distances preserved from the logged setup —
|
||
identical to the backtester replay loop.
|
||
- Cross-source dedup verified: telegram+dashboard double-logs and
|
||
5-min re-fires of a persisting signal collapse to one event each.
|
||
- The stray repo-root `signals.db` (828 rows, Aug 13–14, pre-panel
|
||
schema) is an early duplicate subset of `logs/signals.db` — excluded.
|
||
- Tests: Module 25 in `test_all.py` was only added on 2026-09-03 (the
|
||
2026-08-18 claim of 7 tests was never true at HEAD). It now covers
|
||
boundary flooring, clustering (shared `signal_log.setup_key`), trade
|
||
building, the stored-candle replay books, surface parity, WAL-visible
|
||
loading (`mode=ro`; `immutable=1` missed the newest rows) and hit-rate
|
||
signing.
|