Daily-trend veto on M15 + lot-size / already-open lines (campaign 2026-09-30), and the 2026-10-01 whole-codebase bug review
Two bodies of work that touch the same files, so one commit. Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule, written before the runs): - SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto "both"). A finished setup against a clear daily trend (closed D1 bars, Sunday stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier) is removed as the last step of the core. One definition (signal_aggregator.daily_trend_series) for live, the bar replay and the twin's host side; D1 is a required frame of the M15 scan; mirrored in cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R. - FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1 (−9.7R), buy-only and sell-only on M15, the wide-stop cap (xau_strategy.wide_stop_cap_atr stays 0.0, a research knob). - New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15 pins byte for byte. Twin and reference script MD5-equal on every window; recorder and feature-export parity re-proved; base-rate table rebuilt. - DISPLAY (books byte-identical): exposure.py — the lot size for trade_size.risk_per_trade (new config.json block, off by default, private to the owner) and the count of alerted same-direction plans still open (signals.alerted_at, read-only open_alerted_setups); the daily trend in words on Telegram, the CLI and the dashboard; signals.daily_trend logged. - providers/oanda: a date window with no bars is an empty page, not an error (a D1 pad starting on a Saturday could not be fetched). - Reports only, nothing shipped: seven context screens (no state nominated) and eight outside data sources (none adds a forecast to gold's own daily trend). - Independent verification (report section 5.13): every figure reproduced on both scripts, scoring identical to the previous commit on all four timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022; the neighbour clause is thin (a simple 100-day average misses the R line by 1.2R); "never adds a trade" is an observation, not a guarantee. Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects elsewhere, plus 4 found by an independent review of the fixes; all fixed, each with a test; no trade book changed: - telegram_bot: one _deliver_alert for the candle scan and the level break. The row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and un-stamped when no chat received it or the task is cancelled mid-stamp, so two alerts at one candle close count each other and no SQLite write runs on the event loop. The 5-bar cooldown counts bars of the scan's own frame (== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after the settlement pause) and is restored from signals.db at start-up. The lot-size line goes to the owner's own chat only (a group the owner ran /autoscan in used to receive it). /status gains a news-model line and escapes failure details. - signal_log: mark_alerted returns the stamp it wrote; unmark_alerted; last_alerted_bars (read-only). - advisory_outcome: candle continuity is judged bar by bar inside the walk; a hole after a trade's exit no longer voids it. The stored-candle replay resolves 108 of 111 logged setups instead of 99, all equal to the live resolver's R. - providers/oanda: an empty date window is accepted only when it is 5 days or shorter and no malformed candle was skipped; a longer one is retried, then refused, instead of being cached as history with a hole. - news_engine: blank RSS titles skipped; one unreadable headline no longer costs the local model its whole batch (checked on the real bundle); model health recorded; no DeepSeek audit call in a process that does not archive; event-direction cache keyed per release; peek_week_events never fetches. - llm_client: chat_json never raises and never logs the API key (requests quotes the Authorization value in InvalidHeader; error bodies are redacted). - signal_aggregator: the scan's news block is guarded; the replay path (use_news=False) assigns exactly the values it did before. - agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input (event timing over the cached calendar week, headlines newest first). scripts/panel_replay.py replays the logged payload per persona; scripts/label_scan_outcomes.py counts with an ISO cutoff. - test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34); tests that write Parquet print a notice and return without pyarrow. Verification after the last code change: suite 632 passed, 0 failed (also with both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37 green; static Python 3.9 / SQLite 3.34 check clean on every touched file. Evidence: backtest/research/daily_trend_campaign_2026-09-30.md, context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6, 8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7).
This commit is contained in:
Harwinder Sekhon
@@ -12,9 +12,13 @@ the repo and stays the reference:
|
||||
thread analyses the prefix [0..j].
|
||||
K2 fb2_aggregate signal_aggregator.analyze_from_dataframes end to end
|
||||
(MTF merge, penalties, grade, gates 6b-6i, the three
|
||||
setup generators, XAU_PB, XAU_CP, the stale-veto step)
|
||||
setup generators with the wide-stop cap, XAU_PB,
|
||||
XAU_CP, the stale-veto step, the daily-trend veto)
|
||||
plus the per-bar rules of gpu_backtest._replay_span.
|
||||
One thread per primary bar.
|
||||
One thread per primary bar. The daily trend itself
|
||||
arrives as a host flag (cudaBasedBacktesting.
|
||||
trend_flags): its pandas definition lives in
|
||||
signal_aggregator.daily_trend_series only.
|
||||
K3 fb2_features position_sizing.build_features (65 features), one
|
||||
thread per accepted trade (--export-features).
|
||||
K4 fb2_cooldown gpu_backtest._apply_cooldown (single thread).
|
||||
@@ -155,6 +159,8 @@ WARN_NAMES = (
|
||||
"rf_unavailable", # 33 regime filter: TF missing
|
||||
"rf_no_adx", # 34 regime filter: ADX None
|
||||
"rf_blocked", # 35 arg adx
|
||||
"trend_veto", # 36 daily-trend veto: arg setup direction (+1 / -1)
|
||||
"trend_unavailable", # 37 daily-trend veto on, no usable D1 bar
|
||||
)
|
||||
WARN_CODES = {n: i for i, n in enumerate(WARN_NAMES)}
|
||||
|
||||
@@ -206,6 +212,13 @@ SKIP_NAMES = {1: "WEEKEND_FILTER", 2: "DAY_FILTER", 3: "SESSION_WINDOW"}
|
||||
# 1 = min conviction, 2 = product conviction). Pinned by name, never by the
|
||||
# position in signal_aggregator.GRADING_MODES; pack_params checks the two agree.
|
||||
GRADING_MODE_CODES = {"pair": 0, "min": 1, "product": 2}
|
||||
# xau_strategy.daily_trend_veto (signal_aggregator.DAILY_TREND_MODES), pinned
|
||||
# by name like the grading modes; pack_params checks the two agree. The
|
||||
# per-bar trend itself is computed on the host (cudaBasedBacktesting.
|
||||
# trend_flags, from signal_aggregator.daily_trend_series): +1 / 0 / -1, and
|
||||
# DAILY_TREND_UNAVAILABLE where the D1 slice holds no usable bar.
|
||||
DAILY_TREND_CODES = {"off": 0, "both": 1, "buy": 2, "sell": 3}
|
||||
DAILY_TREND_UNAVAILABLE = 2
|
||||
|
||||
# ── Config packing ────────────────────────────────────────────────────
|
||||
PARAMS_F = (
|
||||
@@ -228,6 +241,7 @@ PARAMS_F = (
|
||||
# all 0.0 under "pair"): aplus, a, b, c, strong, buy, boost_a, boost_b
|
||||
"conv_aplus", "conv_a", "conv_b", "conv_c",
|
||||
"conv_strong", "conv_buy", "conv_boost_a", "conv_boost_b",
|
||||
"wide_stop_cap_atr", # 0.0 = off (GENERIC and XAU_CP stop cap, after the gates)
|
||||
)
|
||||
PARAMS_I = (
|
||||
"ema_trend_period", "ema_mode_2ema", "ema_partial_neutral", "rsi_regime_aware",
|
||||
@@ -245,6 +259,7 @@ PARAMS_I = (
|
||||
"quality_filter_mask", "strategies_mask", "primary_is_m5", "primary_bar_secs",
|
||||
"regime_filter", "n_confirm",
|
||||
"grading_mode", # 0 pair (shipped) / 1 min / 2 product
|
||||
"daily_trend_veto", # 0 off / 1 both / 2 buy / 3 sell (DAILY_TREND_CODES)
|
||||
)
|
||||
|
||||
# Fields of the four config dataclasses the kernels do NOT read, with the
|
||||
@@ -377,6 +392,12 @@ def pack_params(platform_cfg, bt_cfg) -> Tuple[np.ndarray, np.ndarray, np.ndarra
|
||||
for name, val in zip(("conv_aplus", "conv_a", "conv_b", "conv_c",
|
||||
"conv_strong", "conv_buy", "conv_boost_a", "conv_boost_b"), conv_t):
|
||||
f[name] = val
|
||||
f["wide_stop_cap_atr"] = x.wide_stop_cap_atr
|
||||
# daily-trend veto: the live module owns the mode list (same local import)
|
||||
from signal_aggregator import DAILY_TREND_MODES, daily_trend_mode_of
|
||||
assert set(DAILY_TREND_MODES) == set(DAILY_TREND_CODES), (
|
||||
f"signal_aggregator.DAILY_TREND_MODES {DAILY_TREND_MODES} != the kernel's {sorted(DAILY_TREND_CODES)}")
|
||||
daily_trend_veto = DAILY_TREND_CODES[daily_trend_mode_of(x)]
|
||||
from providers.base import BaseDataProvider # CANDLE_SECONDS (the aggregator's clock)
|
||||
bar_secs = BaseDataProvider.CANDLE_SECONDS[platform_cfg.primary_timeframe]
|
||||
strategies = list(bt_cfg.strategies_only or [])
|
||||
@@ -431,6 +452,7 @@ def pack_params(platform_cfg, bt_cfg) -> Tuple[np.ndarray, np.ndarray, np.ndarra
|
||||
"regime_filter": 1 if b.regime_filter else 0,
|
||||
"n_confirm": len(platform_cfg.confirmation_timeframes),
|
||||
"grading_mode": GRADING_MODE_CODES[grading_mode],
|
||||
"daily_trend_veto": daily_trend_veto,
|
||||
}
|
||||
pf = np.array([float(f[name]) for name in PARAMS_F], dtype=np.float64)
|
||||
pi = np.array([int(i[name]) for name in PARAMS_I], dtype=np.int32)
|
||||
@@ -480,6 +502,9 @@ def _defines() -> str:
|
||||
lines.append(f"#define Q_{name.replace('+', 'PLUS')} {code}")
|
||||
for name, code in STRATEGY_CODES.items():
|
||||
lines.append(f"#define STRAT_{name or 'NONE'} {code}")
|
||||
for name, code in DAILY_TREND_CODES.items():
|
||||
lines.append(f"#define TREND_{name} {code}")
|
||||
lines.append(f"#define FB2_TREND_UNAVAILABLE {DAILY_TREND_UNAVAILABLE}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
@@ -1512,6 +1537,7 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
|
||||
const double* __restrict__ rf_adx, const long long* __restrict__ rf_cut,
|
||||
const signed char* __restrict__ skip_reason, const signed char* __restrict__ label_hour,
|
||||
const signed char* __restrict__ label_wd, const signed char* __restrict__ close_hour,
|
||||
const signed char* __restrict__ daily_trend_p,
|
||||
const double* __restrict__ PF, const int* __restrict__ PI,
|
||||
const int first_pos, const int n_bars,
|
||||
double* __restrict__ OF2, int* __restrict__ OI2,
|
||||
@@ -1933,6 +1959,16 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
|
||||
if (!(rr < min_rr - 1e-9)) {
|
||||
if (tp1_distance < min_tp) { WARN(W_spread_gate_generic, tp1_distance); }
|
||||
else {
|
||||
// wide-stop cap AFTER the gates (signal_aggregator._wide_stop_cap, 0 = off)
|
||||
if (PF[PF_wide_stop_cap_atr] > 0.0 && xau_enabled && is_xau) {
|
||||
const double stop_floor = PYMAX(atr * sl_atr_floor, min_sl_price);
|
||||
const double cap_dist = PYMAX(PF[PF_wide_stop_cap_atr] * atr, stop_floor);
|
||||
if (sl_distance > cap_dist) {
|
||||
sl_distance = cap_dist;
|
||||
sl_price = is_buy ? (entry - cap_dist) : (entry + cap_dist);
|
||||
rr = tp1_distance / sl_distance;
|
||||
}
|
||||
}
|
||||
{ // printed-target floor AFTER the gates (signal_aggregator 2026-09-26)
|
||||
const double floor_dist = sl_distance * PF[PF_tp_print_floor_rr];
|
||||
if (tp1_distance < floor_dist) {
|
||||
@@ -2079,6 +2115,15 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
|
||||
}
|
||||
double rr = (sl_dist > 0.0) ? height / sl_dist : 0.0;
|
||||
if (!(rr < min_rr - 1e-9) && !(height < min_tp)) {
|
||||
// wide-stop cap AFTER the gates (signal_aggregator._wide_stop_cap, 0 = off)
|
||||
if (PF[PF_wide_stop_cap_atr] > 0.0 && xau_enabled && is_xau) {
|
||||
const double cap_dist = PYMAX(PF[PF_wide_stop_cap_atr] * atr, min_sl_price);
|
||||
if (sl_dist > cap_dist) {
|
||||
sl_dist = cap_dist;
|
||||
sl_ = is_buy ? (entry - cap_dist) : (entry + cap_dist);
|
||||
rr = height / sl_dist;
|
||||
}
|
||||
}
|
||||
{ // printed-target floor AFTER the gates (signal_aggregator 2026-09-26)
|
||||
double tp1_dist = height;
|
||||
const double floor_dist = sl_dist * PF[PF_tp_print_floor_rr];
|
||||
@@ -2103,7 +2148,22 @@ void fb2_aggregate(const double* __restrict__ open_p, const double* __restrict__
|
||||
}
|
||||
|
||||
// ── replay: the optional higher-TF regime filter (after analyze_from_dataframes) ──
|
||||
// strategy_pre_rf = the setup at step 11 (the stale spread-gate cleanup on
|
||||
// the host), i.e. BEFORE the daily-trend veto of step 12 and the filter.
|
||||
K2I(strategy_pre_rf) = strategy;
|
||||
|
||||
// ── 12. daily-trend veto (signal_aggregator._apply_daily_trend_veto) ──
|
||||
if (PI[PI_daily_trend_veto] != 0 && xau_enabled && is_xau) {
|
||||
const int td = daily_trend_p[pos];
|
||||
if (td == FB2_TREND_UNAVAILABLE) { WARN(W_trend_unavailable, 0.0); }
|
||||
else if (strategy != STRAT_NONE && td != 0) {
|
||||
const int mode = PI[PI_daily_trend_veto];
|
||||
const bool against = (direction > 0 && td < 0 && (mode == TREND_both || mode == TREND_buy))
|
||||
|| (direction < 0 && td > 0 && (mode == TREND_both || mode == TREND_sell));
|
||||
if (against) { WARN(W_trend_veto, (double)direction); quality = Q_D; strategy = STRAT_NONE; }
|
||||
}
|
||||
}
|
||||
|
||||
if (PI[PI_regime_filter]) {
|
||||
const long long cut = rf_cut[pos];
|
||||
if (cut == 0) { K2I(rf_state) = 1; WARN(W_rf_unavailable, 0.0); }
|
||||
@@ -2780,6 +2840,7 @@ def aggregate(primary_dev: Dict, primary_an: TFAnalysis,
|
||||
rf_adx, rf_cut,
|
||||
_dev(flags["skip_reason"], np.int8), _dev(flags["label_hour"], np.int8),
|
||||
_dev(flags["label_wd"], np.int8), _dev(flags["close_hour"], np.int8),
|
||||
_dev(flags["daily_trend"], np.int8),
|
||||
_dev(pf, np.float64), _dev(pi, np.int32),
|
||||
np.int32(first_pos), np.int32(n_bars), OF2, OI2, warn_code, warn_arg))
|
||||
cp.cuda.Device().synchronize()
|
||||
|
||||
Reference in New Issue
Block a user