harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Quant review 2026-09-26: alert cooldown, plain-language alerts, M15 grade-A SELL block, 1R printed targets, panel/breach fixes

Browse Source 
Live signals.db review (2026-09-04 → 09-25, 100 unique setups replayed on the
scans' stored candles): the stop-hunt claim is refuted (71% of stop-outs ran
≥ 1R past the stop, the same as the books' 70–72%); −25.1R of the −37.8R raw
live book were consecutive-bar re-entries the backtester's 5-bar cooldown never
takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week
windows.

Delivery (book-neutral):
- telegram_bot: 5-bar alert cooldown per timeframe (_ALERT_COOLDOWN_BARS =
  backtest.cooldown_bars); setup block prints the trail first as the tested
  exit, TP1/TP2 as guides with their R, "Planned TP1 RR", and one validated-book
  line per setup class (agent_panel_base_rates.json); the informational 100-pip
  TP line is removed (owner request).
- AI block rewritten for a new trader: "AI check: the odds look better than /
  worse than / about usual", the chance "in 100" against the usual odds, one
  reason for and one against, no verdict word or confidence figure; rendered
  whenever the verdict carries a probability. agent_panel: persona prompts ban
  trading jargon (PANEL_PROMPT_VERSION v2.1-2026-09-26); the macro-source
  verdict now carries its persona's p_win (every live v2 row had
  panel_probability NULL). Dashboard and guides mirror the wording.
- telegram_bot: level-breach rescans skip the settlement-pause window (the
  20:45 M15 bar was re-logged at 22:01/22:06/22:11 every evening).

Signal (pre-registered rule-1 campaign, CUDA twin paired with gpu_backtest.py):
- M15 grade-A SELLs blocked (sell_min_quality ["A+","B"], M15 override 6):
  W2 813/+154.15R PF 1.53 vs 931/+146.14R PF 1.43, OOS +68.5 vs +60.67,
  W3 max DD 20.9 vs 19.3R. PASSED a–f.
- Printed target 1 floored at 1R in every generator (signals.tp_print_floor_rr),
  applied AFTER the min-RR and spread gates so no trade changes; mirrored in
  cuda_signal_kernels.py K2. Label-only re-pin of all four timeframes (only
  tp1/rr_planned columns move). A floor applied before the gate cost −24.6R and
  was rejected.
- FAILED: M15 min_rr_ratio 1.0 (trade-identical, −2.0R). Owner's call: M5
  tp_floor_rr 1.0 (+48.9R W2, max DD 10.1 → 15.9R), not shipped.

New pins: M5 W2 e599c8a8…, M15 W2 5b65ecf6…, M15 OOS 09b8a8e7…, H1 W3
a44fd19b…, H4 W3 f3587bae…; twin and reference MD5s equal on every window.
Base-rate table rebuilt from the new pins. Suite 558 passed.

Evidence: backtest/research/rr_and_sell_campaign_2026-09-26.md (plus the
2026-09-18 loss-anatomy and wrong-direction reports it builds on).
This commit is contained in:
Harwinder Sekhon
2026-09-26 21:06:05 -04:00
parent 266f204d35
commit 4e0a083955
16 changed files with 1695 additions and 201 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backtest/research/rr_and_sell_campaign_2026-09-26.md
+183
View File
@@ -0,0 +1,183 @@
# Risk-reward floor and M15 grade-A SELL block — pre-registered campaign, 2026-09-26
Owner questions (2026-09-26): plans sometimes show a target smaller than the stop (RR 0.75 / 0.5);
and "we still lose money". The live review of `logs/signals.db` (2026-09-04 → 2026-09-25, 100
unique setups replayed on the scans' own stored candles) found: no stop-hunt signature (52 stop-outs,
71% ran ≥ 1R past the stop, the same as the books' 70–72%); −25.1R of the −37.8R raw live book were
consecutive-bar re-entries the backtester's 5-bar cooldown never takes; the cooldown view
(−12.7R / 57) sits at the 3rd–5th percentile of three-week windows of the two-year books.
This file was written BEFORE any candidate run. Rules first, results appended below them.
## Harness
`venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --primary-tf <tf> --start <s>
--end <e> --spread <0.65|0.30> [--set ...] --export-csv <out>` (CUDA research twin; its parity
with `gpu_backtest.py` was re-proven today: the M15 and M5 two-year controls reproduce the pinned
MD5s `b4aa07e33e17e23774c486a6900d673b` and `49f8e602fc6ae769713c94ef7b257810`). Factory
configuration; `--set` is the only override. A passing candidate is re-run once on
`gpu_backtest.py` before it ships and the two MD5s must be equal.
Windows: W2 2024-09-05 → 2026-09-05 (halves split at 2025-09-05); OOS 2024-06-01 → 2025-03-30
(halves at the midpoint 2024-10-30); W3 2023-09-05 → 2026-09-05 (the owner's 3-year maximum,
halves at 2025-03-05; used for the max-DD clause). Costs $0.65 (headline) and $0.30.
## Candidates
| # | TF | `--set` | Hypothesis |
|---|---|---|---|
| B1 | M15 | `signals.min_rr_ratio=1.0` | Vetoes the grade-A XAU_ATR plan (RR 0.75 / 0.5); the signal falls through to the GENERIC swing setup whose M15 floor is already 1R. Every M15 plan then shows RR ≥ 1. Two-year book: XAU_ATR 174 / +5.7R / 24% wrong-direction vs GENERIC 757 / +140.5R / 16% |
| B2 | M5 | `signals.tp_floor_rr=1.0` | Retest of the 1R target floor on M5 (failed 2026-07-11 on Twelve Data bars at $0.30). Expected to fail again: it lifts low-ATR targets past the $9 spread gate |
| B3 | M15 | `xau_strategy.sell_min_quality=["A+","B"]` | Parked cycle-2 candidate: M15 grade-A SELLs 230 / −14.4R, negative in y1, y2, y3 and OOS (cycle_2026-09-10.md). Counter-fact on record: those shorts paid in 2021–22 |
| B4 | M15 | B1 + B3 | The shipped combination if both pass alone |
## Acceptance rule (CLAUDE.md rule 1, written before the runs)
A candidate PASSES only if, at $0.65, on W2: (a) total R ≥ control + 5R; (b) both halves ≥ the
control's halves − 1R (no half may fund the other); (c) PF ≥ 1.3 and ≥ control − 0.02; (d) OOS
total R ≥ control − 1R; (e) W3 max drawdown ≤ control + 10%; (f) the same sign of ΔR at $0.30.
A candidate that changes fewer than 20 trades on W2 is "no evidence", not a pass. B4 must pass
the rule on its own numbers, not by inheritance.
Judge run-level numbers, never the added/removed bucket alone (the 5-bar cooldown displaces
entries). Trade diff keyed on (entry time, direction).
## Results (all runs on the CUDA twin, `Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1)`, 0 cache fallbacks; controls reproduce the pinned MD5s)
| TF | Window | Cost | Candidate | Trades | Total R | H1 | H2 | WR | PF | Max DD | MD5 (8) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| M15 | W2 | $0.65 | control | 931 | +146.14 | +66.14 | +80.00 | 60.5% | 1.43 | 16.1 | b4aa07e3 |
| M15 | W2 | $0.65 | B1 | 931 | +144.16 | +65.09 | +79.07 | 60.3% | 1.42 | 16.1 | 80428e98 |
| M15 | W2 | $0.65 | **B3** | 813 | **+154.15** | +70.10 | +84.04 | 62.2% | 1.53 | 15.6 | 497520ef |
| M15 | W2 | $0.65 | B4 | 813 | +153.41 | +70.73 | +82.68 | 62.1% | 1.53 | 15.6 | 177b716d |
| M15 | W2 | $0.30 | control | 931 | +161.44 | +75.63 | +85.81 | 61.0% | 1.48 | 14.9 | 6bdea010 |
| M15 | W2 | $0.30 | B1 | 931 | +158.17 | +74.41 | +83.75 | 60.7% | 1.47 | 14.9 | 21dcfc9e |
| M15 | W2 | $0.30 | B3 | 813 | +167.53 | +78.61 | +88.91 | 62.6% | 1.59 | 14.5 | aff050f7 |
| M15 | W2 | $0.30 | B4 | 813 | +166.06 | +79.12 | +86.95 | 62.5% | 1.58 | 14.8 | 03a412b8 |
| M15 | OOS | $0.65 | control | 355 | +60.67 | +9.14 | +51.52 | 60.8% | 1.49 | 17.1 | 9fd624fb |
| M15 | OOS | $0.65 | B1 | 355 | +60.67 | +9.14 | +51.52 | 60.8% | 1.49 | 17.1 | 9fd624fb (identical) |
| M15 | OOS | $0.65 | B3 = B4 | 313 | +68.50 | +18.87 | +49.63 | 63.6% | 1.67 | 12.4 | 09b8a8e7 |
| M15 | OOS | $0.30 | control | 355 | +69.18 | +13.48 | +55.70 | 61.7% | 1.57 | 14.9 | 30a82817 |
| M15 | OOS | $0.30 | B3 = B4 | 313 | +76.21 | +22.92 | +53.29 | 64.5% | 1.77 | 11.1 | 5fc34cef |
| M15 | W3 | $0.65 | control | 1273 | +186.61 | +75.87 | +110.74 | 59.4% | 1.39 | 19.3 | 5194e5d4 |
| M15 | W3 | $0.65 | B1 | 1273 | +183.78 | +75.83 | +107.95 | 59.2% | 1.39 | 19.3 | 2d5cfc31 |
| M15 | W3 | $0.65 | B3 | 1105 | +195.79 | +76.87 | +118.92 | 61.1% | 1.49 | 20.9 | 8bed7199 |
| M15 | W3 | $0.65 | B4 | 1105 | +194.19 | +76.00 | +118.18 | 61.0% | 1.48 | 20.9 | 05201726 |
| M15 | W3 | $0.30 | control / B1 / B3 / B4 | 1273 / 1273 / 1105 / 1105 | +210.42 / +206.25 / +216.53 / +214.19 | | | | 1.45 / 1.44 / 1.55 / 1.54 | 17.7 / 17.7 / 19.6 / 19.6 | |
| M5 | W2 | $0.65 | control | 781 | +146.07 | +26.42 | +119.65 | 52.9% | 1.52 | 10.1 | 49f8e602 |
| M5 | W2 | $0.65 | B2 | 1105 | +194.94 | +54.11 | +140.83 | 53.6% | 1.50 | 15.9 | 351f9eee |
| M5 | W2 | $0.30 | control / B2 | 781 / 1105 | +162.09 / +220.32 | +31.71 / +65.22 | +130.37 / +155.10 | | 1.60 / 1.58 | 8.9 / 13.7 | |
| M5 | OOS | $0.65 | control | 86 | +9.47 | +0.01 | +9.46 | 53.5% | 1.33 | 8.2 | da497d74 |
| M5 | OOS | $0.65 | B2 | 235 | +13.17 | −6.01 | +19.18 | 52.3% | 1.17 | 12.3 | 3cece33b |
| M5 | OOS | $0.30 | control / B2 | 86 / 235 | +12.42 / +20.66 | +1.65 / −2.04 | +10.77 / +22.69 | | 1.46 / 1.28 | 6.7 / 9.5 | |
| M5 | W3 | $0.65 | control | 870 | +153.64 | +16.73 | +136.91 | 53.3% | 1.50 | 10.1 | 408de695 |
| M5 | W3 | $0.65 | B2 | 1333 | +223.87 | +47.49 | +176.38 | 53.4% | 1.48 | 15.9 | f89176d6 |
| M5 | W3 | $0.30 | control / B2 | 870 / 1333 | +172.40 / +256.20 | | | | 1.57 / 1.56 | 8.9 / 13.7 | |
Trade diffs on W2 at $0.65 (key = entry time + direction):
- B1: 0 added, 0 removed, 931 common. The 158 grade-A XAU_ATR setups become GENERIC (strategy
mix 915 / 16 vs 757 / 174); same bars, same direction, a swing-anchored stop instead of 2 × ATR.
- B3: 162 removed (−8.11R, all grade-A SELLs), 44 added (−0.11R, cooldown displacement), 769 common.
Direction mix 635 BUY / 178 SELL vs 635 / 296.
- B2: 392 added (+63.63R), 68 removed (+17.23R), 713 common.
### Verdicts against the pre-registered rule
| # | (a) ΔR ≥ +5 W2 | (b) halves | (c) PF | (d) OOS | (e) W3 DD ≤ +10% | (f) $0.30 sign | Verdict |
|---|---|---|---|---|---|---|---|
| B1 | −2.0 ✗ | −1.05 / −0.93 ✓ | 1.42 ✓ | 0 ✓ | 19.3 ✓ | −3.3 ✓ | **FAIL (a)**. Trade-identical; the GENERIC stop is not better than the 2 × ATR stop on the same grade-A signals (W3 −2.8R). Not shipped |
| B2 | +48.9 ✓ | +27.7 / +21.2 ✓ | 1.50 ✓ (−0.02) | +3.7 ✓ (but OOS PF 1.17, OOS H1 −6.0) | 15.9 vs 10.1 = +57% ✗ | +58.2 ✓ | **FAIL (e)**. Owner's call, see the ladder below |
| B3 | +8.0 ✓ | +4.0 / +4.0 ✓ | 1.53 ✓ | +7.8 ✓ | 20.9 vs 19.3 = +8% ✓ | +6.1 ✓ | **PASS a–f — SHIPPED** |
| B4 | +7.3 ✓ | ✓ | 1.53 ✓ | +7.8 ✓ | +8% ✓ | +4.6 ✓ | Passes, but 0.7R below B3 alone on W2 and 1.6R on W3; B1's part adds nothing. Not shipped |
B3 per year on W3 ($0.65, control → B3): 2023 (Sep–Dec) +13.2 → +10.8; 2024 +52.9 → +53.3;
2025 +77.3 → +81.2; 2026 (to Sep 5) +43.2 → +50.5. Removed grade-A SELLs by year: 2023 15 / +1.4R,
2024 76 / −1.3R, 2025 79 / −8.6R, 2026 60 / −5.8R. The max-DD episode is the same one in both books
(2023-12-01 → 2024-02-23), 19.3R → 20.9R. M15 SELLs remaining under B3 on W3: A+ 19 / +2.6R,
B 217 / −8.6R. The 5-year fact on the other side of the scale stands (shorts paid in 2021–22).
### M5 target-floor ladder (informational, run after B2's verdict; NOT pre-registered, NOT shipped)
`--set signals.tp_floor_rr=<x>` on M5, $0.65:
| Floor | W2 trades / R / PF / DD | OOS trades / R / PF / DD | W3 trades / R / PF / DD |
|---|---|---|---|
| 0.5 (shipped) | 781 / +146.1 / 1.52 / 10.1 | 86 / +9.5 / 1.33 / 8.2 | 870 / +153.6 / 1.50 / 10.1 |
| 0.6 | 868 / +159.1 / 1.51 / 10.2 | 105 / +1.5 / 1.04 / 12.5 | 979 / +164.5 / 1.46 / 17.1 |
| 0.75 | 975 / +192.4 / 1.57 / 13.4 | 147 / +13.5 / 1.31 / 9.4 | 1133 / +211.6 / 1.54 / 13.4 |
| 0.9 | 1053 / +209.3 / 1.58 / 13.5 | 194 / +22.8 / 1.41 / 7.5 | 1252 / +238.7 / 1.55 / 13.5 |
| 1.0 (B2) | 1105 / +194.9 / 1.50 / 15.9 | 235 / +13.2 / 1.17 / 12.3 | 1333 / +223.9 / 1.48 / 15.9 |
Reading: the 2026-07-11 verdict ("a 1R floor hurts M5") was made on Twelve Data bars at $0.30 with
the pre-relaxation gates and the pullback entries on; on Oanda bars at $0.65 with today's gates the
floor lifts small-ATR targets past the $9 spread gate and those trades earn (B2 added trades on W3:
554 / +90.8R, 53% winners, median stop $12 vs $20, 962 of 1,105 plans at exactly RR 1.0; positive
in every year; by volatility regime extreme +44.6R, low +32.0R, normal +14.2R). The cost is a deeper
drawdown in the same May–July 2026 episode (10.1R → 13.4–15.9R) and a non-monotone ladder (0.9 beats
1.0 on every window, 0.6 is worse than 0.5 out of sample), which is why nothing from it ships now.
If the owner wants M5 plans at RR ≥ 1 (the original request), the honest candidate is a fresh
pre-registered run at ONE value chosen before looking (0.75 or 1.0), with the DD clause waived
explicitly as on 2026-07-21, plus the cost ladder ($0.80, $1.00) because the added trades carry
$12 stops. Until then M5 plans keep 0.5R guides and the alert says so.
## Shipped (2026-09-26)
1. **A1 — alert cooldown** (`telegram_bot.py`): no new alert on a timeframe within 5 bars of the
last alerted setup's bar, `_ALERT_COOLDOWN_BARS = 5` = `backtest.cooldown_bars`. Book-neutral.
2. **A2 — alert wording** (`telegram_bot.py`, `main.py`, `docs/user_guide.html`): the trail is
printed first as the tested exit; TP1/TP2 are guides with their R; "Planned TP1 RR"; one
validated-book line per setup class from `agent_panel_base_rates.json`. Book-neutral.
3. **B3 — M15 grade-A SELL block** (`pair_signals/xauusd.py`, M15 override 6). Re-pins the M15
books (below).
4. **XAU_ATR target floor** (`signal_aggregator._generate_xau_atr_trade_setup`, mirrored in
`cuda_signal_kernels.py` K2): the gold ATR generator now honours `tp_floor_rr` like the GENERIC
one, compared as multipliers so the non-binding case is bit-identical. Binds only on M15 (floor
1R): grade-A plans print TP1 at 2 × ATR = 1R instead of 1.5 × ATR = 0.75R. TP1 is advisory in the
backtester, so the only book effect is through the $9 spread gate (parity MD5s below).
## Addendum (same day): printed-target floor extended to every generator
Owner rule: "make sure TP1 is at least 1:1". Shipped as `signals.tp_print_floor_rr = 1.0`
(effective floor `max(tp_floor_rr, tp_print_floor_rr)`), applied after the min-RR and spread gates in
the gold ATR, GENERIC and chart-pattern generators (Python and CUDA K2). Label-only by construction
and by measurement (twin books vs the previous pins, same code otherwise):
| Book | Rows | R before → after | Columns that differ | Rows with a new `tp1` | min `rr_planned` before → after |
|---|---|---|---|---|---|
| M5 W2 | 781 | +146.07 → +146.07 | `tp1`, `rr_planned` | 544 (GENERIC 408, XAU_CP 105, XAU_ATR 31) | 0.5 → 1.0 |
| M15 W2 | 813 | +154.15 → +154.15 | none | 0 | 1.0 → 1.0 |
| H1 W3 | 253 | +70.02 → +70.02 | `tp1`, `rr_planned` | 252 | 0.5 → 1.0 |
| H4 W3 | 114 | +21.68 → +21.68 | `tp1`, `rr_planned` | 108 | 0.5 → 1.0 |
New pins: M5 W2 `e599c8a8652f95b7fcdf5a7ef2b564f5`, H1 W3 `a44fd19b4067f46e5bd98d50cc3eff92`,
H4 W3 `f3587bae3c5dc3eda468ea3588b17dcb`; M15 W2 `5b65ecf6…` and OOS `09b8a8e7…` unchanged. Reference
script MD5s: see the parity lines appended below. The gated floor (`tp_floor_rr`, 0.5 on M5 / 1.0 on
M15) is untouched: raising it is the B2 question above, a book change, and remains the owner's call.
## Addendum 2 (same day): consistency checks on `signals.db` — two delivery bugs fixed
Read-only checks over the 8,453 rows (2026-09-04 → 09-25): scan latency (median 10–15 s after the
candle close on every timeframe; 2–3% of rows read the previous bar, all in 18–22 UTC around the
Labor Day gap and the settlement pause), `actionable` / setup consistency (clean), gate reasons for
actionable grades without a setup (all `SPREAD GATE`, small-ATR M5 bars), outcome states (97 resolved,
36 duplicates, 1 unresolvable for a candle gap), scan-outcome labelling (7,054 of 7,634 complete),
8 `failed` scans (Oanda "primary timeframe data unavailable", notably 06:11–06:17 UTC on both Sep 15
and Sep 16), dashboard-vs-bot disagreement on the same bar (4 bars of 3,300).
Two bugs, both fixed and tested, books untouched:
1. **Panel probability never stored under `verdict_source: "macro"`** (the live setting): all 33 v2
rows had `panel_probability` NULL although `panel_probs_json` carried the macro persona's `p_win`.
Cause: the macro-source `PanelVerdict` carried no `probability`, `_write_fields` wrote 0.0 and the
logger's `probability if probability else None` dropped it. Fix: the macro verdict's probability is
the macro take's `p_win` (`agent_panel.run_panel`). Without it the planned re-evaluation at 100 /
300 outcomes would have had no live probabilities to score.
2. **Breach rescans re-log the last pre-pause M15 bar.** Every evening the 20:45 UTC bar was logged
again at 22:01, 22:06 and 22:11 (Sep 17–24) by the tick-driven level-breach rescan: the first
post-pause bar closes at 22:15, so the frame still ends at 20:45. The candle loop already skips
that window (`_closed_bar_in_settlement`); `_handle_breach` now applies the same guard. Cosmetic
for alerts (dedup + cooldown), but it polluted the log's per-bar dedup and the parity reports.
Not bugs, recorded: 380 `blocked` weekend rows on Sep 20 carry no `bar_ts` by design; the 43× repeat of
the Sep 7 18:25 M5 bar is the known holiday gap (no holiday calendar); the H4 loop produced no setup
in three weeks (its two tradeable close hours).