sql-integration
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Harwinder Sekhon
|
2cc5915b00 |
Daily-trend veto on M15 + lot-size / already-open lines (campaign 2026-09-30), and the 2026-10-01 whole-codebase bug review
Two bodies of work that touch the same files, so one commit. Direction and setup-size campaign (2026-09-30 → 10-01, owner's RISK-FIRST rule, written before the runs): - SIGNAL: M15 daily-trend veto (factory override 7, xau_strategy.daily_trend_veto "both"). A finished setup against a clear daily trend (closed D1 bars, Sunday stubs dropped; close vs its 50-bar EMA AND vs the close 60 kept days earlier) is removed as the last step of the core. One definition (signal_aggregator.daily_trend_series) for live, the bar replay and the twin's host side; D1 is a required frame of the M15 scan; mirrored in cuda_signal_kernels.py K2. W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R. - FAILED, not shipped: the veto on M5 (W3 drawdown 10.1R → 15.2R) and on H1 (−9.7R), buy-only and sell-only on M15, the wide-stop cap (xau_strategy.wide_stop_cap_atr stays 0.0, a research knob). - New M15 pins: W2 dfc647ee…, OOS e87b6de1…, W3 ad3b0dbd…. M5 / H1 / H4 pins unchanged. --set xau_strategy.daily_trend_veto=off reproduces the previous M15 pins byte for byte. Twin and reference script MD5-equal on every window; recorder and feature-export parity re-proved; base-rate table rebuilt. - DISPLAY (books byte-identical): exposure.py — the lot size for trade_size.risk_per_trade (new config.json block, off by default, private to the owner) and the count of alerted same-direction plans still open (signals.alerted_at, read-only open_alerted_setups); the daily trend in words on Telegram, the CLI and the dashboard; signals.daily_trend logged. - providers/oanda: a date window with no bars is an empty page, not an error (a D1 pad starting on a Saturday could not be fetched). - Reports only, nothing shipped: seven context screens (no state nominated) and eight outside data sources (none adds a forecast to gold's own daily trend). - Independent verification (report section 5.13): every figure reproduced on both scripts, scoring identical to the previous commit on all four timeframes, no look-ahead. Caveats recorded there: this is a risk filter, not a more accurate direction call; nine tenths of the 2019 → 2026 gain is 2022; the neighbour clause is thin (a simple 100-day average misses the R line by 1.2R); "never adds a trade" is an observation, not a guarantee. Bug review of the whole codebase (2026-10-01). Signal path clean. 17 defects elsewhere, plus 4 found by an independent review of the fixes; all fixed, each with a test; no trade book changed: - telegram_bot: one _deliver_alert for the candle scan and the level break. The row is stamped alerted_at BEFORE the send, on a one-worker alert-db pool, and un-stamped when no chat received it or the task is cancelled mid-stamp, so two alerts at one candle close count each other and no SQLite write runs on the event loop. The 5-bar cooldown counts bars of the scan's own frame (== gpu_backtest._apply_cooldown; clock time released the 22:00 UTC bar after the settlement pause) and is restored from signals.db at start-up. The lot-size line goes to the owner's own chat only (a group the owner ran /autoscan in used to receive it). /status gains a news-model line and escapes failure details. - signal_log: mark_alerted returns the stamp it wrote; unmark_alerted; last_alerted_bars (read-only). - advisory_outcome: candle continuity is judged bar by bar inside the walk; a hole after a trade's exit no longer voids it. The stored-candle replay resolves 108 of 111 logged setups instead of 99, all equal to the live resolver's R. - providers/oanda: an empty date window is accepted only when it is 5 days or shorter and no malformed candle was skipped; a longer one is retried, then refused, instead of being cached as history with a hole. - news_engine: blank RSS titles skipped; one unreadable headline no longer costs the local model its whole batch (checked on the real bundle); model health recorded; no DeepSeek audit call in a process that does not archive; event-direction cache keyed per release; peek_week_events never fetches. - llm_client: chat_json never raises and never logs the API key (requests quotes the Authorization value in InvalidHeader; error bodies are redacted). - signal_aggregator: the scan's news block is guarded; the replay path (use_news=False) assigns exactly the values it did before. - agent_panel: PANEL_PROMPT_VERSION v2.2 — same prompt text, corrected input (event timing over the cached calendar week, headlines newest first). scripts/panel_replay.py replays the logged payload per persona; scripts/label_scan_outcomes.py counts with an ISO cutoff. - test_all.py: no ALTER TABLE … DROP COLUMN (the live box has SQLite 3.34); tests that write Parquet print a notice and return without pyarrow. Verification after the last code change: suite 632 passed, 0 failed (also with both Parquet engines blocked and FB2_BACKEND=cpu, the live box's view); the five reference MD5s reproduce on both backtesters, 0 cache fallbacks; Module 37 green; static Python 3.9 / SQLite 3.34 check clean on every touched file. Evidence: backtest/research/daily_trend_campaign_2026-09-30.md, context_screens_2026-10.md, new_inputs_2026-10.md; CLAUDE.md sections 2, 4, 6, 8, 9, 10 and 11; pair_signals/xauusd.py docstring (M15 override 7). |
||
|
Harwinder Sekhon
|
d81cdf845b |
Conviction campaign 2026-09-27 (FAILED, research knob kept) + plain-words alert wording
Owner question: can score and confidence be combined into one number that is easier to understand and for the system to decide on? Research (pre-registered, nothing ships to the signal path): - signals.grading_mode: "pair" (default, the shipped two-floor rule, byte- identical: all five reference MD5s reproduce on both scripts), "min" (min(|score|, 0.6*conf)), "product" (|score|*conf/100); thresholds fixed from the control's corner points, no tuning. Helpers _final_signal_and_grade / _transitioning_boost / conviction / CONVICTION_THRESHOLDS in signal_aggregator; mirrored in K2 (GRADING_MODE_CODES, PF_conv_* packed from the live module); twin refuses other values; M15 OOS MD5 equal on both scripts for both modes. - Verdict: FAILED on M5 and M15. K-MIN moves 43-54 trades for -1.2R / -2.9R (clauses a,b,e / a,b,d); K-PROD -29R on M15 (PF 1.29, W3 DD 28.9R), neutral on M5. H1/H4 product +11.9R / +17.0R on W3 from extra low-score trades (H1 PF 1.98 -> 1.75, DD doubles): hypothesis only. - Code review (high) before the runs: 6 findings fixed, 1 recorded (engine_config_fingerprint omits the switch under "pair", so live rows keep their fingerprint). Delivery (books untouched): alerts lead with the grade and print the two raw numbers in words (main.plain_words: "Direction: strong buy · Reliability: good · timeframes agree", cut-offs = the grade table's own) on Telegram, /status, the CLI and the dashboard card (plain_words in the snapshot); "MTF" label replaced by "Other timeframes"; guides updated. Suite 566 passed. Evidence: backtest/research/conviction_campaign_2026-09-27.md |
||
|
Harwinder Sekhon
|
4e0a083955 |
Quant review 2026-09-26: alert cooldown, plain-language alerts, M15 grade-A SELL block, 1R printed targets, panel/breach fixes
Live signals.db review (2026-09-04 → 09-25, 100 unique setups replayed on the scans' stored candles): the stop-hunt claim is refuted (71% of stop-outs ran ≥ 1R past the stop, the same as the books' 70–72%); −25.1R of the −37.8R raw live book were consecutive-bar re-entries the backtester's 5-bar cooldown never takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week windows. Delivery (book-neutral): - telegram_bot: 5-bar alert cooldown per timeframe (_ALERT_COOLDOWN_BARS = backtest.cooldown_bars); setup block prints the trail first as the tested exit, TP1/TP2 as guides with their R, "Planned TP1 RR", and one validated-book line per setup class (agent_panel_base_rates.json); the informational 100-pip TP line is removed (owner request). - AI block rewritten for a new trader: "AI check: the odds look better than / worse than / about usual", the chance "in 100" against the usual odds, one reason for and one against, no verdict word or confidence figure; rendered whenever the verdict carries a probability. agent_panel: persona prompts ban trading jargon (PANEL_PROMPT_VERSION v2.1-2026-09-26); the macro-source verdict now carries its persona's p_win (every live v2 row had panel_probability NULL). Dashboard and guides mirror the wording. - telegram_bot: level-breach rescans skip the settlement-pause window (the 20:45 M15 bar was re-logged at 22:01/22:06/22:11 every evening). Signal (pre-registered rule-1 campaign, CUDA twin paired with gpu_backtest.py): - M15 grade-A SELLs blocked (sell_min_quality ["A+","B"], M15 override 6): W2 813/+154.15R PF 1.53 vs 931/+146.14R PF 1.43, OOS +68.5 vs +60.67, W3 max DD 20.9 vs 19.3R. PASSED a–f. - Printed target 1 floored at 1R in every generator (signals.tp_print_floor_rr), applied AFTER the min-RR and spread gates so no trade changes; mirrored in cuda_signal_kernels.py K2. Label-only re-pin of all four timeframes (only tp1/rr_planned columns move). A floor applied before the gate cost −24.6R and was rejected. - FAILED: M15 min_rr_ratio 1.0 (trade-identical, −2.0R). Owner's call: M5 tp_floor_rr 1.0 (+48.9R W2, max DD 10.1 → 15.9R), not shipped. New pins: M5 W2 e599c8a8…, M15 W2 5b65ecf6…, M15 OOS 09b8a8e7…, H1 W3 a44fd19b…, H4 W3 f3587bae…; twin and reference MD5s equal on every window. Base-rate table rebuilt from the new pins. Suite 558 passed. Evidence: backtest/research/rr_and_sell_campaign_2026-09-26.md (plus the 2026-09-18 loss-anatomy and wrong-direction reports it builds on). |
||
|
Harwinder Sekhon
|
be4db8b2a7 |
xauusd M15: halve the RSI regime multiplier in every regime (M15 override 5, rule-1 validated)
Owner-approved 2026-09-09 after the pre-registered weight test (backtest/research/weight_test_2026-09-09.md). M15-only technical.regime_weights_override = the shipped table with rsi x 0.5 (0.5 trending family / 0.75 ranging / 0.5 transitioning), derived in the factory and pinned literally by the Module 9 equivalence test; a Module 33 test locks that M5/H1/H4 carry no override. Evidence, 2-year Oanda window 2024-09-05..2026-09-05 at $0.65: 793/+117.62R PF 1.41 -> 931/+146.14R PF 1.43 (fixed halves +27.9/+0.6R vs control); OOS 2024-06-01..2025-03-30 300/+40.59R PF 1.38 -> 355/+60.67R PF 1.49; 3-year +151.90R -> +186.61R (max DD 15.5 -> 19.3R, 2026 -4R). Parity: the factory run with no --set flag reproduces the validated candidate books byte-for- byte (2-yr MD5 d185f099..., OOS 6f8f24b4...). M15 canonical book re-pinned to backtest/oanda_m15_2yr_065_20260909.csv. Docs carry a one-line note. Claude-Session: https://claude.ai/code/session_01Nc9Ninzyi18QkWezrxeBvB |
||
|
Harwinder Sekhon
|
e79b533a60 |
violent-move watch: display-only warning line + owner heads-up (dark by default)
New violent_watch.py assesses each scan from the cached Forex Factory week events, the UTC clock and the report's own indicator states, and stamps SignalReport.watch_line post-hoc (telegram alerts, CLI, dashboard). A standalone heads-up goes to the owner on the fastest scanned timeframe when the level is "hot", once per UTC hour per trigger set, capped per day. Config block `violent_watch` (enabled False, alert_line, headsup, max_headsup_per_day, headsup_min_level); env VIOLENT_WATCH_ENABLED. Rule chosen from a 3-year bar-by-bar sweep (backtest/research/watch_rule_eval.py, recorded in sizing_campaign_2026-09.md): hot = high-impact USD release within 30 min (before or after) OR a hot clock slot (weekday 12-14 UTC, 01 UTC, Sunday 22-23 UTC) with at least one price-extreme reading (outside Bollinger bands, RSI <30/>=70, outside Keltner); ~8.6% of bars, next-bar odds 3-5x normal, ~4 hot hours/day. watch = clock slot alone or two price-extreme readings (~2x). Context readings (ADX rising, extreme volatility, transitioning, MACD acceleration) are phrases only - letting them raise the level flagged 27% of bars at ~2x and would have sent ~13 heads-ups/day (rejected). Direction sentence: DOWN when bearish readings dominate (70-78% on both TFs), UP on M5 only. Never a gate or exit: every open-trade protection tested lost money. Test-locked out of analyze_from_dataframes / engine / backtester. Module 30 (18 tests). Suite: 453 passed. Guide PDFs not rebuilt. Claude-Session: https://claude.ai/code/session_01RWV2yLt3YXHHGVYHSQU2EE |
||
|
Harwinder Sekhon
|
a3d7594f0c |
news: beginner-language summary, 4 h headline window, drop neutral headlines; --news log report
Verdict from the 3-week signal log (backtest/research/news_sentiment_value_2026-09-04.md): news_sentiment is a trailing-24h echo of price (r +0.67 with the past day, +0.03 with the next; most bullish 2 h after the 08-25 top, bearish 44 h later; most bearish at the 09-02 low, bullish 16 h later). Per-headline LLM labels are correct; the 12 h pool and the averaging are what destroy the information. Both gating flags stay False. - news_engine: _LLM_SUMMARY_SYSTEM rewritten for someone who has never traded (no acronyms, two worked examples); summary payload newest-first with the platform's own gold direction; reply cap 300 -> 400 chars. - config: headline_max_age_seconds 43200 -> 14400; RSS items without a parsable pubDate are dropped (they used to be stamped "now" and never aged out). - news_engine.get_fundamental_context drops headlines whose every relevant score is inside +-0.1 (_is_neutral_headline) before the per-currency mean; news_sentiment from this date is a mean over directional <=4 h headlines (log comparisons split here). - static: dashboard copy for the 4 h pool; news_summary.php prompt gets the same style rule. - analyze_signal_log: --news (under --stored-candles) prints sentiment lead/lag with split halves, sentiment at the window high/low with flip lags, and outcome buckets by direction x news x macro stance; LogRow gains news_sentiment/event_risk; _row_index shared. - tests: 406 -> 414 (age window, undated drop, neutral filter, summary prompt/payload/cap, --news helpers, load_rows round-trip); RSS fixtures now carry a pubDate. - docs: user/beginner guides, CLAUDE.md, deploy README. Claude-Session: https://claude.ai/code/session_0169BgFtWjp1SEAa5392Q9jH |
||
|
Harwinder Sekhon
|
9a8a2c1aa1 |
xauusd factory: block M5 shorts in transitioning regime, block all H1 shorts (rule-1 validated)
M5 sell_blocked_regimes=[transitioning]: 12mo +7.9R (PF 1.50->1.56, both halves up), 5yr +5.3R, OOS unchanged. H1 sell_min_quality=[]: 5yr +16.7R (PF 1.18->1.46), OOS +11.5R (PF 1.13->1.74), 12mo -1.8R with both halves positive. Campaign table, rejected and parked candidates: backtest/research/signal_log_campaign_2026-09.md Claude-Session: https://claude.ai/code/session_01FC4s6VCURFte4dub4xKxYL |
||
|
Harwinder Sekhon
|
26f0681ad1 |
Docs: user guide + beginner guide PDFs
Two printable guides in docs/ (HTML source + weasyprint-rendered A4 PDFs): a trader-facing user guide (signal anatomy, trail management, M5/M15/H1 comparison from the 12mo books, expectations, commands) and a plain-language beginner edition (glossary, worked sizing example, step-by-step trailing-stop walkthrough, H1-first guidance). |