# Signal-log live performance evaluation — 2026-08-18

**Tool:** `analyze_signal_log.py` (new, repo root). Read-only on
`logs/signals.db` (sqlite `immutable=1`); replays every logged trade
setup through the backtester's own `simulate_trades` on the RAW Oanda
price path (next-open fill, SL/TP distances preserved, SL-first ties,
gap-at-open fills, close-only trail ratchet with the per-setup stamped
trail params). Reproduce:

```bash
python analyze_signal_log.py --csv backtest/signal_log_eval_20260818.csv
```

## Window & sample

- Log window: **2026-08-13 14:31 → 2026-08-18 14:13 UTC (5 days)** —
  the entire signal_log history to date.
- 2,654 rows (telegram 2,180 / dashboard 474); M5 1,308, M15 1,189,
  H1 119, H4 38.
- 39 rows carry a `trade_setup` → **17 unique signal events** after
  dedup (cross-source double-logs + persisting-signal re-fires collapse
  on (tf, direction, entry, SL, strategy) with a 60-min gap).
- **No real executions exist** (`oanda_state.json` empty, no
  `oanda_daemon` rows) — every outcome below is a hypothetical replayed
  fill, not a broker fill.
- Cost: dual convention — **$0.65 rt headline**, $0.30 comparative
  (derived analytically, R is linear in cost).

## Trade replay results

| First logged | TF | Signal | Grade | Strategy | Fill | Exit | Bars | R@0.65 |
|---|---|---|---|---|---|---|---|---|
| 08-13 15:30 | 5min | SELL | B | GENERIC | 4365.39 | TRAIL | 19 | −0.20 |
| 08-13 18:00 | 15min | SELL | B | GENERIC | 4356.05 | TRAIL | 44 | +0.91 |
| 08-14 07:45 | 5min | BUY | B | XAU_CP | 4339.72 | TRAIL | 60 | +2.56 |
| 08-14 12:15 | 15min | BUY | B | GENERIC | 4371.91 | TRAIL | 27 | +0.06 |
| 08-14 12:25 | 5min | STRONG_BUY | A | GENERIC | 4379.12 | TRAIL | 277 | +0.20 |
| 08-14 12:30 | 5min | STRONG_BUY | A | GENERIC | 4379.87 | TRAIL | 276 | +0.16 |
| 08-14 12:30 | 15min | BUY | B | GENERIC | 4379.87 | TRAIL | 63 | +0.18 |
| 08-17 08:00 | 1h | BUY | C | GENERIC | 4405.19 | TRAIL | 18 | −0.13 |
| 08-17 14:15 | 5min | STRONG_BUY | A | GENERIC | 4412.35 | TRAIL | 46 | −0.10 |
| 08-17 14:20 | 5min | STRONG_BUY | A | GENERIC | 4418.55 | TRAIL | 125 | −0.23 |
| 08-17 14:25 | 5min | STRONG_BUY | A | GENERIC | 4424.10 | SL | 151 | −1.02 |
| 08-17 14:30 | 15min | BUY | A | GENERIC | 4418.23 | SL | 17 | −1.03 |
| 08-17 14:45 | 15min | STRONG_BUY | A | GENERIC | 4417.49 | SL | 16 | −1.03 |
| 08-17 15:00 | 15min | STRONG_BUY | A | GENERIC | 4417.48 | SL | 15 | −1.03 |
| 08-17 15:00 | 5min | NEUTRAL | C | GENERIC | 4417.48 | TRAIL | 117 | −0.16 |
| 08-17 15:15 | 15min | STRONG_BUY | A | GENERIC | 4421.30 | SL | 14 | −1.04 |
| 08-17 15:30 | 15min | BUY | B | GENERIC | 4425.70 | SL | 85 | −1.02 |

**Aggregate: 17 closed, 6 wins (35%), total −2.91R @0.65 / −2.67R
@0.30, PF 0.58.** By grade: A n=9 → −5.12R; B n=6 → +2.50R; C n=2 →
−0.30R.

Reading: the book is two regimes. Everything through the 08-14 rally
made money (+4.7R across 8 events, incl. the XAU_CP +2.56R); the 08-17
afternoon BUY/STRONG_BUY cluster chased the spike to ~4425 and got
stopped as price faded — 9 events, −7.6R combined. One bad afternoon
dominates the 5-day sample. Notable: a NEUTRAL-graded row carried a
setup (post-gate CP/PB-style path) and lost small.

## Advisory panel scorecard

Only **7 verdicts** logged so far (2 NEUTRAL, 5 OPPOSE, 0 AGREE) — the
panel's min-quality gate + daily budget keep it dark on most scans.

| Row | When | Signal | Verdict | Outcome basis | Impact |
|---|---|---|---|---|---|
| #1358 | 08-17 07:40 M5 | BUY B | NEUTRAL @55% | +4h move | −$8.19 in dir (pass was right) |
| #1361 | 08-17 07:45 M5 | BUY A | NEUTRAL @55% | +4h move | −$12.43 in dir (pass was right) |
| #1550 | 08-17 14:15 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 0.10R |
| #1553 | 08-17 14:25 M5 | STRONG_BUY A | OPPOSE @80% | replay | saved 1.02R |
| #1555 | 08-17 14:30 M15 | BUY A | OPPOSE @75% | replay | saved 1.03R |
| #1561 | 08-17 14:45 M15 | STRONG_BUY A | OPPOSE @75% | replay | saved 1.03R |
| #1588 | 08-17 15:30 M15 | BUY B | OPPOSE @75% | replay | saved 1.02R |

**Every OPPOSE was correct: ≈ +4.2R would have been lost taking those
five trades.** The panel's rationale ("price climbed too fast, profit
too small") correctly identified the 08-17 exhaustion spike that the
composite scored A. Both NEUTRAL passes also preceded adverse moves.
Caveat: n=7, one market episode, zero AGREE verdicts — the panel has
only proven itself as a veto so far, never as a confirmation.

## Directional hit-rate (non-setup signals)

27 deduped non-setup BUY/SELL rows, forward close-to-close on M5:

| Horizon | Hit rate | Mean move in direction |
|---|---|---|
| +1h | 48% | −$0.72 |
| +4h | 44% | −$2.30 |
| +24h | 7% | −$27.45 |

The 24h figure is dominated by the 08-17 buy-cluster reversal; not
independently meaningful at n=27.

## Method notes / caveats

- **Small sample**: 5 days, 17 events, one trending episode + one
  reversal. Directional read only; do not tune anything on this.
- Entry model = next bar open after the scan boundary (live scans run
  at boundary+10s), SL/TP distances preserved from the logged setup —
  identical to the backtester replay loop.
- Cross-source dedup verified: telegram+dashboard double-logs and
  5-min re-fires of a persisting signal collapse to one event each.
- The stray repo-root `signals.db` (828 rows, Aug 13–14, pre-panel
  schema) is an early duplicate subset of `logs/signals.db` — excluded.
- Tests: Module 25 in `test_all.py` was only added on 2026-09-03 (the
  2026-08-18 claim of 7 tests was never true at HEAD). It now covers
  boundary flooring, clustering (shared `signal_log.setup_key`), trade
  building, the stored-candle replay books, surface parity, WAL-visible
  loading (`mode=ro`; `immutable=1` missed the newest rows) and hit-rate
  signing.