ForexBot2/backtest/research/conviction_campaign_2026-09-27.md

Single "conviction" number vs the two-number grade — pre-registered campaign, 2026-09-27

Owner question (2026-09-27): "Can there be a combination of both [score and confidence] that is easier to understand and also for the system to make decisions?" Chosen route (owner): research a single decision number. This file was written BEFORE any candidate run. Rules first, results appended below them. Nothing ships from this campaign; a pass is a hypothesis for a per-timeframe override in a separate commit (CLAUDE.md rule 2).

What is tested

Today (signal_aggregator._compute_final_signal, gate 6c) a grade needs BOTH a score floor and a confidence floor: A+ |score| ≥ 40 & conf ≥ 65 & mtf; A 33 & 55 & mtf; B 28 & 50; C 25 & 45; D |score| ≥ 15; STRONG 35 & 55; BUY/SELL 25 & 50; 6c boost A 30 & 50 & mtf, B 25 & 45. Candidates replace the pair with one number. Thresholds are fixed analytically from the control's corner points — no grid, no tuning (a data-mined threshold would need its own holdout):

• K-MIN: conviction = min(|score|, 0.6 × confidence). Grades A+ ≥ 40 (mtf), A ≥ 33 (mtf), B ≥ 28, C ≥ 25, D |score| ≥ 15 unchanged, else F; STRONG ≥ 35, BUY/SELL ≥ 25; 6c boost A ≥ 30 (mtf), B ≥ 25. (The control's corner ratios score/conf are 0.615, 0.60, 0.56, 0.556, so 0.6 is the natural single scale.) Against the control: A+ needs conf ≥ 66.7 (was 65), B ≥ 46.7 (50), C ≥ 41.7 (45), STRONG ≥ 58.3 (55), BUY/SELL ≥ 41.7 (50); score floors unchanged.
• K-PROD: conviction = |score| × confidence / 100. A+ ≥ 26 (40×65), A ≥ 18.15 (33×55), B ≥ 14 (28×50), C ≥ 11.25 (25×45), D |score| ≥ 15, else F; STRONG ≥ 19.25 (35×55), BUY/SELL ≥ 12.5 (25×50); 6c A ≥ 15 (30×50), B ≥ 11.25 (25×45). A genuine trade-off: high confidence can lift a mid score and vice versa. Interaction on record: a product B/C with |score| < 25 is demoted to D on M5 by the live gate 1 (m5_min_abs_score 25); the gate stays, the count of demotions is reported (Step E). Property on record (code review, before the runs): the D rung stays |score| ≥ 15 in both candidates, and under the product form B/C can sit below it (B needs |score| ≥ 14.74 at the 95 confidence cap, C ≥ 11.85), so an M15/H1/H4 product book can hold a trade with |score| < 15. Kept as pre-registered (a score floor would make it a pair rule again); the count of |score| < 15 and < 25 trades per candidate book is reported.
• Geometric mean √(|score|·conf) is a monotone transform of the product (same partition at thresholds √T) — not a separate candidate.
• Direction stays sign(score); mtf agreement stays required for A+/A; the exhaustion hard block (conf = 0) gives conviction 0 in both forms, so NEUTRAL and at most D, as today.

Switch: signals.grading_mode ("pair" default = today's literal rule, "min", "product"), read in _compute_final_signal and gate 6c (signal_aggregator.py helpers _signal_pair, _quality_pair, _transitioning_boost, conviction, CONVICTION_THRESHOLDS), mirrored in K2 (backtest/cuda_signal_kernels.py, PI_grading_mode, PF_conv_* packed from the live module), refused by the twin for other values. pair is byte-identical to HEAD 1454ea1 (Step A proves it). Note: engine_config_fingerprint hashes SignalConfig, so the live fingerprint changes at the deploy of this field even in pair mode (label-only; split signal-log comparisons there).

Harness

Twin command template (research twin, CLAUDE.md §3/§9; seconds per book): venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --spread <0.65|0.30> --primary-tf <5min|15min|1h|4h> --start <s> --end <e> [--set signals.grading_mode=<min|product>] --export-csv <out>. Reference script (backtest/gpu_backtest.py, same flags, --workers 12) for the neutrality proof and every paired MD5. Factory configuration; --set is the only override.

Evidence order (all must hold before any candidate number is read):

1. Neutrality: with the new code and no --set, the five pins reproduce on BOTH scripts (M5 W2 e599c8a8652f95b7fcdf5a7ef2b564f5, M15 W2 5b65ecf6fad9a8da412749066fd50d36, M15 OOS 09b8a8e7f5833436c20aef60ca0a7ff7, H1 W3 a44fd19b4067f46e5bd98d50cc3eff92, H4 W3 f3587bae3c5dc3eda468ea3588b17dcb), plus the twin with an explicit --set signals.grading_mode=pair on M15 OOS.
2. Twin parity on the new paths: each candidate's M15 OOS $0.65 book is MD5-equal on both scripts.
3. A passing candidate is re-run once on gpu_backtest.py (W2, $0.65) and the two MD5s must be equal.

Windows: W2 2024-09-05 → 2026-09-05 (halves split at 2025-09-05); OOS 2024-06-01 → 2025-03-30 (halves at 2024-10-30); W3 2023-09-05 → 2026-09-05 (halves at 2025-03-05; max-DD clause); 5-year 2021-08-01 → 2026-08-04 (per-year clause; cache 20210531_20260804). Costs: $0.65 headline, $0.30 comparability. Books are written outside the tree (§9 keeps no CSVs in-tree).

Code review gate (owner instruction): the diff is reviewed with the code-review skill before Step A; outcome recorded here. Outcome (2026-09-27, code-review high, before Step A): 7 findings, 6 fixed in the same working tree — kernel mode codes pinned by name (GRADING_MODE_CODES) and checked against the live list; no zero-threshold fallback for a non-pair mode; the twin's refusal reads the kernel's map; conviction() refuses modes it has no formula for; one _final_signal_and_grade dispatch point; engine_config_fingerprint omits the switch under pair so live rows keep their fingerprint; CLAUDE.md test counts updated. One recorded, not changed: the D rung under the product form (above). Suite green after the fixes.

Candidates

#	TF	--set	Hypothesis
K-MIN	M5, M15 (H1/H4 for the record)	signals.grading_mode=min	one scale on the score axis; nearly the control, loosens B/C/BUY confidence, tightens A+
K-PROD	M5, M15 (H1/H4 for the record)	signals.grading_mode=product	genuine trade-off between the two axes; confidence can lift a mid score

Acceptance rule (CLAUDE.md rule 1, written before the runs)

A candidate PASSES only if, at $0.65, on W2: (a) total R ≥ control + 5R; (b) both halves ≥ the control's halves − 1R (no half may fund the other); (c) PF ≥ 1.3 and ≥ control − 0.02; (d) OOS total R ≥ control − 1R; (e) W3 max drawdown ≤ control + 10%; (f) the same sign of ΔR at $0.30. A candidate that changes fewer than 20 trades on W2 is "no evidence", not a pass.

Judge run-level numbers, never the added/removed bucket alone (the 5-bar cooldown displaces entries). Trade diff keyed on (entry time, direction).

Campaign clause (g): on the 5-year window at $0.65, no calendar year in which the control is positive goes below −5R under the candidate (trend-artefact guard, fixedtp_2026-08-05.md precedent); a W2 gain that lives in a single year is read as an artefact and noted.

The rule is judged independently on M5 W2 and M15 W2: the mode is read per config, so a pass ships as a per-timeframe factory override (rule 2) and a timeframe where it fails keeps pair. H1 and H4 are run on W3 for the record (ΔR at $0.65, trades changed); they are not ship candidates here — a positive H1/H4 result is a hypothesis for its own pre-registration. Nothing ships if no clause set passes; the field stays a research-only knob (precedent --fixed-rr-tp).

Run list

Step 0 suite green (Module 37 on the GPU) and the review gate. Step A neutrality (11 runs). Step B twin parity on the new paths (4 runs). Step C books on the twin: {M5 W2, M5 OOS, M5 W3, M15 W2, M15 OOS, M15 W3, H1 W3, H4 W3} × {$0.65, $0.30} × {control, min, product}. Step D per-year (5-year, M5 and M15, $0.65, control/min/product). Step E K-PROD M5 W2 $0.65 with --record DIR --record-bars (MD5 equal to the Step C book) to count m5_score demotions. Step F reference-script rerun for any passer.

Results

Reference script Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1), GPU cache fallbacks: 0; twin Backend: CuPy GPU (NVIDIA GeForce RTX 4070 SUPER, cupy 14.0.1), GPU cache fallbacks: 0 (no per-bar recompute path in cudaBasedBacktesting). Every run from the kept cache windows.

Step A — neutrality (new code, grading_mode at its default, no --set)

Book	Pin	gpu_backtest.py	cudaBasedBacktesting.py
M5 W2 $0.65	e599c8a8652f95b7fcdf5a7ef2b564f5	equal	equal
M15 W2 $0.65	5b65ecf6fad9a8da412749066fd50d36	equal	equal
M15 OOS $0.65	09b8a8e7f5833436c20aef60ca0a7ff7	equal	equal; also equal with an explicit --set signals.grading_mode=pair
H1 W3 $0.65	a44fd19b4067f46e5bd98d50cc3eff92	equal	equal
H4 W3 $0.65	f3587bae3c5dc3eda468ea3588b17dcb	equal	equal

The refactor (helpers + switch) is byte-neutral on both scripts: the five pins stand.

Step B — twin parity on the new code paths (M15 OOS $0.65)

Candidate	gpu_backtest.py	cudaBasedBacktesting.py
K-MIN	22726d8500383ce02e0a3cbf0412f6d8	22726d8500383ce02e0a3cbf0412f6d8 (equal)
K-PROD	d1ae11651d5d7c8066b7e8addc0c7f36	d1ae11651d5d7c8066b7e8addc0c7f36 (equal)

The twin's books below therefore count as evidence (instructions item 6, CUDA-twin addendum).

Books (all on the twin; the control rows reproduce the reference script's MD5s)

Halves: W2 at 2025-09-05, OOS at 2024-10-30, W3 at 2025-03-05. Trades / R from the 63-column book.

TF	Window	Cost	Candidate	Trades	Total R	H1	H2	WR	PF	Max DD	MD5 (8)
M5	W2	$0.65	control	781	+146.07	+26.42	+119.65	52.9%	1.52	10.1	e599c8a8
M5	W2	$0.65	K-MIN	812	+144.86	+24.73	+120.13	52.8%	1.50	11.5	2ab2b573
M5	W2	$0.65	K-PROD	805	+147.33	+29.27	+118.06	52.5%	1.52	11.6	a37dbb71
M5	W2	$0.30	control / K-MIN / K-PROD	781 / 812 / 805	+162.09 / +161.17 / +163.96	+31.71 / +30.15 / +34.70	+130.37 / +131.02 / +129.26		1.60 / 1.57 / 1.59	8.9 / 10.6 / 10.9	7ab48db9 / 64f26c30 / ed14d7ec
M5	OOS	$0.65	control	86	+9.47	+0.01	+9.46	53.5%	1.33	8.2	65c46e62
M5	OOS	$0.65	K-MIN	94	+14.32	+2.37	+11.94	56.4%	1.51	6.4	bec7c87f
M5	OOS	$0.65	K-PROD	91	+13.03	+0.10	+12.93	54.9%	1.45	4.5	217d0134
M5	OOS	$0.30	control / K-MIN / K-PROD	86 / 94 / 91	+12.42 / +17.34 / +16.05				1.46 / 1.66 / 1.58	6.7 / 4.8 / 4.1
M5	W3	$0.65	control	870	+153.64	+16.73	+136.91	53.3%	1.50	10.1	d20ec8a3
M5	W3	$0.65	K-MIN	905	+153.84	+20.64	+133.20	53.3%	1.48	11.5	5ad0c381
M5	W3	$0.65	K-PROD	896	+154.60	+20.12	+134.48	52.9%	1.49	11.6	466bda90
M5	W3	$0.30	control / K-MIN / K-PROD	870 / 905 / 896	+172.40 / +172.92 / +173.99				1.57 / 1.55 / 1.56	8.9 / 10.6 / 10.9
M15	W2	$0.65	control	813	+154.15	+70.10	+84.04	62.2%	1.53	15.6	5b65ecf6
M15	W2	$0.65	K-MIN	827	+151.25	+69.05	+82.21	61.9%	1.51	16.0	5c7918dc
M15	W2	$0.65	K-PROD	1106	+125.17	+62.26	+62.91	58.2%	1.29	28.9	ecfb62e5
M15	W2	$0.30	control / K-MIN / K-PROD	813 / 827 / 1106	+167.53 / +164.82 / +144.91	+78.61 / +77.62 / +74.30	+88.91 / +87.20 / +70.61		1.59 / 1.57 / 1.34	14.5 / 14.9 / 26.8	15486607 / 5ab311cf / 582019a7
M15	OOS	$0.65	control	313	+68.50	+18.87	+49.63	63.6%	1.67	12.4	09b8a8e7
M15	OOS	$0.65	K-MIN	314	+65.64	+19.20	+46.44	63.4%	1.64	12.4	22726d85
M15	OOS	$0.65	K-PROD	408	+53.98	+26.67	+27.32	59.1%	1.36	17.3	d1ae1165
M15	OOS	$0.30	control / K-MIN / K-PROD	313 / 314 / 408	+76.21 / +73.36 / +64.35				1.77 / 1.74 / 1.44	11.1 / 11.1 / 16.4
M15	W3	$0.65	control	1105	+195.79	+76.87	+118.92	61.1%	1.49	20.9	70195fb0
M15	W3	$0.65	K-MIN	1126	+191.25	+71.90	+119.35	60.7%	1.47	21.3	23edd8bc
M15	W3	$0.65	K-PROD	1478	+186.24	+84.70	+101.54	58.3%	1.33	28.9	0246cbb4
M15	W3	$0.30	control / K-MIN / K-PROD	1105 / 1126 / 1478	+216.53 / +212.32 / +215.54				1.55 / 1.53 / 1.39	19.6 / 20.0 / 26.8
H1	W3	$0.65	control	253	+70.02	+26.26	+43.76	64.4%	1.98	4.4	a44fd19b
H1	W3	$0.65	K-MIN	268	+74.81	+28.75	+46.06	64.9%	1.99	5.7	24674555
H1	W3	$0.65	K-PROD	351	+81.88	+30.68	+51.19	60.1%	1.75	8.5	2e8df9cc
H1	W3	$0.30	control / K-MIN / K-PROD	253 / 268 / 351	+72.74 / +77.64 / +85.91				2.03 / 2.04 / 1.80	4.3 / 5.6 / 8.3
H4	W3	$0.65	control	114	+21.68	+4.91	+16.77	57.9%	1.59	5.4	f3587bae
H4	W3	$0.65	K-MIN	117	+22.57	+4.91	+17.66	58.1%	1.59	5.4	3ed9a44e
H4	W3	$0.65	K-PROD	151	+38.69	+13.97	+24.72	58.9%	1.78	6.3	b696b617
H4	W3	$0.30	control / K-MIN / K-PROD	114 / 117 / 151	+22.52 / +23.43 / +39.88				1.61 / 1.62 / 1.81	5.0 / 5.0 / 6.0

Trade diffs at $0.65 (key = entry time + direction; W2 for M5/M15, W3 for H1/H4)

• M5 K-MIN: 37 added (+1.74R), 6 removed (+4.53R), 775 common (+143.12R vs +141.54R in the control: the cooldown shifts a few entries). Grade mix B 572 → 606, A 187 → 186, C 21 → 19. Changed 43.
• M5 K-PROD: 74 added (+16.51R), 50 removed (+10.12R), 731 common (+130.83R vs +135.95R). Grade mix A 187 → 263, A+ 1 → 3, C 21 → 8; XAU_ATR setups 32 → 57 (more A grades reach the ATR generator). Changed 124.
• M15 K-MIN: 34 added (−0.34R), 20 removed (+3.42R), 793 common (+151.59R vs +150.73R). A+ 22 → 10 (the A+ confidence floor rises to 66.7), B 443 → 465. Changed 54.
• M15 K-PROD: 544 added (+2.66R), 251 removed (+16.64R), 562 common (+122.51R vs +137.50R). The book grows 813 → 1106 (C 60 → 196, A+ 22 → 79); 295 candidate trades carry |score| < 25 and earn +5.11R between them. Changed 795.
• H1 (for the record): K-MIN +20 / −5, 248 common, ΔR +4.79; K-PROD +126 / −28, 225 common, ΔR +11.86 with PF 1.98 → 1.75 and max DD 4.4 → 8.5R; 63 trades with |score| < 25 (+7.38R).
• H4 (for the record): K-MIN +3 / −0, ΔR +0.89 (3 trades changed: no evidence); K-PROD +40 / −3, ΔR +17.01 with PF 1.59 → 1.78 and DD 5.4 → 6.3R, 28 trades with |score| < 25 (+15.41R).
• No candidate book on any timeframe holds a trade with |score| < 15 except the M5 chart-pattern entries the control also has (127 XAU_CP trades, forced grade B by design, identical in all three M5 books). The D-rung property recorded above did not materialise.
• Step E (M5 W2 $0.65 K-PROD with --record --record-bars, book MD5 a37dbb71… = the Step C book): 82 analysed bars carry the gate-1 warning M5 FILTER: |score| … < 25 minimum as their first warning, every one of them ending at grade D. That is the whole M5 gate-1 interaction.

Per year, 5-year window 2021-08-01 → 2026-08-04 at $0.65 (twin)

TF	Year	control n / R	K-MIN n / R	K-PROD n / R
M5	2021	12 / −1.99	14 / −0.28	13 / −0.55
M5	2022	50 / −2.18	54 / −0.43	52 / +1.29
M5	2023	61 / −2.75	62 / −2.10	61 / −3.57
M5	2024	102 / +0.97	108 / +4.05	106 / +4.19
M5	2025	332 / +40.54	346 / +38.87	342 / +41.63
M5	2026	348 / +92.49	361 / +87.15	358 / +85.73
M5	total	905 / +127.08 (PF 1.39, DD 13.5)	945 / +127.25 (1.37, 12.4)	932 / +128.72 (1.38, 12.2)
M15	2021	78 / −10.88	84 / −11.20	90 / −6.29
M15	2022	267 / −24.63	273 / −26.34	340 / −47.01
M15	2023	211 / +11.74	216 / +11.10	261 / +8.63
M15	2024	343 / +53.35	345 / +49.12	448 / +62.45
M15	2025	425 / +81.16	438 / +85.91	576 / +83.40
M15	2026	224 / +38.10	227 / +32.98	317 / +4.32
M15	total	1548 / +148.84 (PF 1.25, DD 48.3)	1583 / +141.58 (1.23, 48.7)	2032 / +105.50 (1.13, 77.4)

Verdicts against the pre-registered rule (W2 at $0.65 unless stated)

Candidate	TF	(a) ΔR ≥ +5	(b) halves ≥ ctrl − 1	(c) PF ≥ 1.3 & ≥ ctrl − 0.02	(d) OOS ≥ ctrl − 1	(e) W3 DD ≤ ctrl + 10%	(f) same sign at $0.30	(g) 5-yr years	changed	Verdict
K-MIN	M5	✗ −1.21	✗ H1 24.73 < 25.42	✓ 1.50	✓ +14.32 vs 9.47	✗ 11.5 > 11.1	✓ −0.92	✓	43	FAIL (a, b, e)
K-PROD	M5	✗ +1.26	✗ H2 118.06 < 118.65	✓ 1.52	✓ +13.03	✗ 11.6 > 11.1	✓ +1.87	✓	124	FAIL (a, b, e)
K-MIN	M15	✗ −2.90	✗ 69.05 < 69.10, 82.21 < 83.04	✓ 1.51	✗ 65.64 < 67.50	✓ 21.3 ≤ 23.0	✓ −2.71	✓	54	FAIL (a, b, d)
K-PROD	M15	✗ −28.98	✗	✗ 1.29	✗ 53.98	✗ 28.9 > 23.0	✓ −22.62	✓ (2026 +4.32, still positive)	795	FAIL (a–e)

Reading. K-MIN is the control with a slightly looser confidence floor on B/C/BUY and a tighter one on A+: it moves 43–54 trades on W2 and changes nothing of substance (−1.2R on M5, −2.9R on M15, PF within 0.02, OOS mixed). It is a re-labelling of the same rule with a small loss, not a better decision number. K-PROD is a different rule: on M15 it lets confidence substitute for score, grows the book by a third and gives back −29R with PF 1.29 and a W3 drawdown of 28.9R (5-year 77.4R, PF 1.13); on M5 it is neutral. On H1 and H4 the product rule adds R (+11.9R and +17.0R on W3) by taking 40–100 extra low-score trades; H1's PF drops 1.98 → 1.75 and its drawdown doubles, H4 stays "unproven" (114 → 151 trades). Both are hypotheses for their own pre-registration, not evidence.

Not shipped (2026-09-27)

No candidate passes on M5 or M15. The two-number rule stays as validated; signals.grading_mode remains a research-only knob (default pair, byte-identical books). Follow-ups:

• The "easier to understand" half of the owner's question is a display task: the grade already combines score, confidence and timeframe agreement, and the surfaces can lead with it and word the two raw numbers in plain language (no signal change, books untouched).
• H1 / H4 K-PROD (+11.9R / +17.0R on W3, more trades, lower H1 PF, higher DD) may be pre-registered on its own if the owner wants it; H4 has too few trades for rule 1 and H1's edge already lives in a small book (253 trades in 3 years).
• Do not re-run these two formulas on M5 or M15 with other thresholds: a tuned threshold would need its own holdout and the corner-point versions here bracket the plausible range.