ForexBot2/CLAUDE.md

ForexBot2 — developer guide

ForexBot2 is a signal-intelligence system for one market: gold against the US dollar (XAU/USD). It does not place trades. Every few minutes it downloads price candles, scores them with nine technical indicators, checks that several timeframes agree, grades the result (A+ down to F), and turns a good grade into a written trade plan (entry, stop loss, two targets, trailing rule). The plan is delivered to a Telegram bot, a command-line report, and a public web dashboard. A backtester replays history through the same signal code, so a backtest result is what the live scanner would have said on that day.

This file is the map of the system. Read section 2 first if you are new. Section 7 lists the rules you must follow before changing anything that affects signals. Section 10 lists the traps. Section 11 is the history of every validated change. The research reports behind those decisions live in backtest/research/*.md.

---

1. Quick start

python -m venv venv && venv/bin/pip install -r requirements.txt   # Python 3.12; a venv/ already exists on the dev box
cp config.json.example config.json                                 # then fill in the keys (see section 5)

venv/bin/pip install cupy-cuda12x "nvidia-cuda-runtime-cu12==12.9.*"   # optional, dev box: backtester GPU backend
                                                                   #   (CuPy from pip needs that header wheel to compile kernels)
venv/bin/python test_all.py                                        # QA suite, no network; prints its own count
                                                                   #   (632 tests / 40 modules as of 2026-10-01)
venv/bin/python main.py --pair XAUUSD --timeframe H1 --detail      # one signal report in the terminal
                                                                   #   --json full dump, --panel adds the advisory LLM panel,
                                                                   #   --timeframe M5 | M15 | H1 | H4
venv/bin/python telegram_bot.py                                    # the bot (refuses to start without credentials)
venv/bin/python dashboard_publisher.py --once                      # one scan + publish to $DASHBOARD_OUT_DIR
                                                                   #   (without --once it runs as a candle-close daemon)

# Backtests (timeframe names are 5min / 15min / 1h / 4h, not M5/M15)
venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --days 365
venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 \
    --primary-tf 15min --spread 0.65 --workers 12 --export-csv book.csv
venv/bin/python backtest/cudaBasedBacktesting.py --pair XAUUSD --start 2024-09-05 --end 2026-09-05 \
    --primary-tf 15min --spread 0.65 --export-csv book.csv   # research twin: same flags, every stage on
                                                            #   the GPU, seconds per book, same MD5 (section 9)

Backtest flags worth knowing: --spread (round-trip cost in dollars; the default is 0.30, the realistic Oanda figure is 0.65 and must be passed explicitly), --quality A+,A,B,C, --strategies, --workers, --max-hold-bars, --warmup-bars, --cooldown-bars, --no-cache, --export-csv out.csv, --detail, --set PATH=JSON (research override of any config value; an unknown path is a hard exit), --export-features (appends the 65 decision-time columns; never use it on a parity run because it changes the file), --fixed-rr-tp N (research only, not shipped), --set signals.grading_mode=min|product (research only, 2026-09-27: grade on one conviction number instead of the score + confidence floors; both FAILED rule 1 on M5 and M15, default pair is the shipped rule and byte-identical), --set xau_strategy.daily_trend_veto=off|both|buy|sell (2026-09-30: the M15 factory ships both; pass off to reproduce a pre-veto M15 book; buy / sell and every mode on M5 / H1 FAILED their rule), --set xau_strategy.wide_stop_cap_atr=K (research only: FAILED on 2026-09-30, ships 0 = off), --record DIR [--record-bars] (research recorder, see section 9), --backend auto|gpu|cpu (auto uses the CuPy GPU backend when CUDA works and falls back to NumPy with the reason in the Backend: line; gpu refuses to fall back; cpu forces the NumPy twins for a parity run; the env var FB2_BACKEND sets the default). Both backends produce byte-identical books (section 9).

test_all.py is not pytest. It uses its own @test("name") decorator that runs each test the moment the file is imported. Run the whole file; it prints per-module results and exits non-zero on any failure. There is no way to run one test on its own. Tests must stay green, and every new gate or setting needs a test.

---

2. How a scan works, from A to Z

A "scan" is one analysis of one timeframe. The bot runs one scan per timeframe at every candle close; the CLI and the dashboard run the same scan on demand. Everything below happens inside one scan, in this order.

1. Trigger. telegram_bot.py wakes 10 seconds after each candle boundary (M5 every 5 min, M15 every 15 min, H1 hourly, H4 every 4 hours). dashboard_publisher.py has its own copy of the same schedule. main.py runs once when you call it.

2. Build the configuration. pair_signals/xauusd.py (make_config) creates a PlatformConfig from config.py, applies the deployment overlay from config.json (app_config.py), then applies the validated per-timeframe overrides (section 8). Never build a bare PlatformConfig() for a real scan; always go through the factory. The confirmation timeframes come from the factory map: M5 → M15 + H1, M15 → M5 + H1, H1 → M15 + H4, H4 → H1 + D1. Since 2026-09-30 an M15 scan also needs the daily (D1) frame, not as a confirmation but as the input of the daily-trend veto (step 8); it is a required frame like the others.

3. Fetch candles. signal_aggregator.analyze_pair (the live entry point) first checks the calendar: on the forex weekend or a blocked weekday it returns a "blocked" scan and does nothing else. Otherwise it asks the active data provider (providers/, Oanda today) for the primary timeframe and each confirmation timeframe. The provider base class drops the still-forming candle, validates and cleans the bars, caches them until the next candle close, and checks freshness. If any required frame cannot be fetched after the retry ladder, the scan is logged as "failed" and no analysis runs.

4. Score each timeframe. engine.py (TechnicalEngine.analyze) computes ATR first, then nine scored indicators: EMA trend (9/21/50/200 ribbon), RSI, MACD, Bollinger bands, ADX, rate of change, Keltner channel, divergence, and candlestick patterns. Each indicator votes on a five-step scale (STRONG SELL, SELL, NEUTRAL, BUY, STRONG BUY) with a confidence. The engine classifies the market regime from ADX and the EMA ribbon: trending_up, trending_down, trending (strong trend, mixed direction), transitioning (ADX rising fast), or ranging. Each regime has its own table of per-indicator weight multipliers (regime_weight_table in engine.py is the single source of truth); the effective weight is the indicator's base weight × the regime multiplier. The composite score is the confidence-weighted average of the votes (vote value × effective weight × confidence), scaled by 50 and clipped to −100…+100. Divergence and candlestick only vote when a pattern exists; a missing vote is "no opinion", not zero. Three engine conventions matter when editing: MACD is always computed even when it is not a scored pillar, because divergence needs its histogram; RSI is reinterpreted per regime after classification (rsi_mode = "regime_aware"; treating the mixed trending regime as an uptrend was a −6.8R bias); and private helper series ride on the result as _underscore attributes, which every scorer and formatter skips along with weight-0 indicators, so a new indicator must follow that convention.

5. Multi-timeframe agreement. analyze_from_dataframes compares the primary timeframe's direction with the two confirmation timeframes. Agreement needs 2 of 3. Disagreement costs a penalty and blocks the top grades.

6. News context (warning only). news_engine.py reads the Forex Factory calendar (MyFXBook RSS as fallback) and four RSS feeds and scores headlines for gold: the keyword scorer first, then the local XAU transformer (news_model.py, mode primary) or DeepSeek (modes off / fallback) overwrites those scores. This produces warnings and a text summary. By design it never changes a signal. The backtester runs with news off, so the live decision must be purely technical too.

7. Penalties, then the signal and the grade. When the timeframes agree, confidence gains 10 points and the score is blended 60/40 with the mean of all timeframe scores. Then a list of penalties (opposing timeframes, stretched price, fading momentum, and so on) is subtracted from the confidence with diminishing weight: each further penalty counts half, and the total is capped at 30 points. Penalties never reduce the score itself. If two or more "exhaustion" penalties fire together with fading momentum, the trade is hard-blocked (confidence set to 0). Then:

Result	Rule
STRONG BUY / STRONG SELL	score ≥ 35 (≤ −35) and confidence ≥ 55
BUY / SELL	score ≥ 25 (≤ −25) and confidence ≥ 50
NEUTRAL	anything else
Grade A+	abs score ≥ 40, confidence ≥ 65, timeframes agree
Grade A	abs score ≥ 33, confidence ≥ 55, timeframes agree
Grade B	abs score ≥ 28, confidence ≥ 50
Grade C	abs score ≥ 25, confidence ≥ 45
Grade D	abs score ≥ 15
Grade F	below that

8. The gate chain. A series of checks can lower the grade or remove the setup. They run in this order (signal_aggregator.py, steps 6b to 6i): session and day filter (the bar-label hour and weekday against the blocked hours and allowed weekdays; the live wrapper and the backtester apply the same list to the bar-close time as well, see section 8); transitioning-regime boost (a B/C/D can become A or B when a trend is emerging); exhausted-trend cap (in a trending regime with two or more exhaustion tags, a C or D is pinned to D and loses its setup); SELL quality gate (shorts need a grade in sell_min_quality; H1's list is empty, so every H1 short is blocked; M15's list is A+, B, so every M15 grade-A short is blocked since 2026-09-26); multi-timeframe opposition gate; SELL volatility gate (shorts are allowed only when volatility is high or extreme; low- or normal-volatility shorts are blocked); regime blocks (M15 blocks the mixed trending regime; M5 blocks shorts in transitioning); SELL-hour gate (M15 blocks shorts at 15:00 and 17:00 UTC); big-candle gate (shipped off); and on M5 six extra noise gates (minimum score, required agreement, confirmation strength, SELL grade cap, chop-transition volatility, SELL RSI floor). Every gate that removes a setup writes a warning that names itself.

One more gate runs at the very end, after the setup generators and the late entry paths of step 9 (_apply_daily_trend_veto, step 12 in the code; M15 only since 2026-09-30): the daily-trend veto. The daily trend is read from closed D1 bars with the provider's Sunday stub bars dropped (signal_aggregator.daily_trend_series): +1 when the last daily close is above its 50-day EMA and above the close 60 kept days earlier, −1 when below both, else 0. A finished setup that goes against a non-zero state is removed (grade D, no setup, warning DAILY TREND FILTER: BUY against a falling daily trend — no trade). It runs after every archetype on purpose, so a chart-pattern setup is vetoed too. Where the veto is off (M5, H1, H4) the live wrapper still stamps report.daily_trend for the alert text and the signal log; nothing reads it back there.

9. Trade setup. Grades A+ and A get the gold ATR setup (_generate_xau_atr_trade_setup): stop 2 × ATR; target 1 at 2 × ATR for A+ and 1.5 × ATR for A (1 × ATR for an A carrying two or more exhaustion tags); target 2 at 1.5 × the target-1 distance. Other tradeable grades get the generic swing-based setup, which floors the stop at $8 (80 pips). Since 2026-09-26 every generator prints target 1 at no less than 1R (signals.tp_print_floor_rr, effective floor max(tp_floor_rr, tp_print_floor_rr)), applied after the min-RR and spread gates have judged the generator's own target, so which generator serves a signal, the stop and the books are unchanged; only the exported tp1 / rr_planned columns move (a floor applied before the gate cost −24.6R on the M15 two-year book, see the 2026-09-26 report). The planned ratio is a guide: the tested exit is the trailing stop. Both are checked against the spread gate: target 1 must be at least 30 × the round-trip cost ($9 at the $0.30 cost setting). Two extra entry paths run after the gates: the M5 pullback (XAU_PB, shipped off) and the chart-pattern entry (XAU_CP, chart_patterns.py: M5 only, buy-side patterns inverse head and shoulders, double bottom, bull flag). Both late paths run after the gate chain and force the grade to B, so an F-composite chart-pattern trade shows as a B in books and alerts. The chart-pattern path deliberately accepts weak composites and re-checks the session, day and big-candle rules, the multi-timeframe-opposition rule and the ATR floor, plus its own SELL, volatility and RSI checks; it does not re-check exhaustion or the M5 minimum-score and confirmation gates. That is validated behaviour, not a bug (section 11, 2026-08-04). A shipped setup always carries a BUY or SELL direction, even when the raw signal was NEUTRAL at grade C (confidence 45–50): the direction is the sign of the score. Every setup is stamped with its strategy name and its trailing parameters. Finally, a stale "costs would eat the edge" veto from an earlier generator is moved to cleaned_warnings when a later generator shipped a setup.

10. Delivery. The SignalReport goes to the surface that asked for it:

• Telegram (telegram_bot.py): an alert is sent only when the signal is not NEUTRAL, the grade reaches alert_min_quality, and a setup exists. Since 2026-09-26 no second alert goes out on a timeframe within 5 bars of the last alerted setup's bar (_ALERT_COOLDOWN_BARS, the backtester's entry cooldown; consecutive-bar re-entries were −25.1R of the −37.8R live book of 2026-09-04 → 09-25). Since 2026-10-01 the five bars are COUNTED in the scan's own primary frame (_primary_bars_since, the backtester's position difference; clock time is only the fallback for a report without frames) and the last alerted bar per timeframe is read back from signals.db at start-up (signal_log.last_alerted_bars → _restore_alert_cooldowns), so neither the settlement pause, the weekend nor a restart lets live alert where the books do not enter. After that, a fingerprint (signal + direction) stops repeats; a grade upgrade re-alerts. Since 2026-09-27 the header leads with the grade (Grade A · BUY · XAU/USD · M15) and the raw score / confidence line is replaced by words from main.plain_words (Direction: strong buy · Reliability: good · timeframes agree; the cut-offs are the grade table's own, so the words never disagree with the grade; the numbers stay in --json, --detail, the dashboard meters and the signal log). The /status summary and the dashboard card carry the same words (plain_words in the snapshot). The setup block prints the trailing rule first as the tested exit, TP1/TP2 as guides with their R, the "Planned TP1 RR", and one line of validated-book statistics for the setup class (_book_rate_line, from agent_panel_base_rates.json). The fingerprint is committed only after a successful send, so a failed send stays eligible next scan. Every send is logged with the signal-log row id. The level-breach monitor reuses the bar's cached M15 report (_cached_or_scan) instead of rescanning. Since 2026-09-30 three more display lines: the daily trend in words under the regime line (🧭 Daily trend: rising — this buy goes with it, main.daily_trend_words), and inside the setup block the lot size for the owner's risk limit (📏, off until trade_size.risk_per_trade is set) and the count of alerted same-direction plans still open (🧱). The size line is private: format_signal_message(report, channel=True) builds the copy without it unless trade_size.telegram_channel is true, and since 2026-10-01 _broadcast sends the owner copy to the owner's own chat only (_is_owner_chat): the channel and any other target (a group the owner used /autoscan or a scan button in; the owner lock is per user) get the copy without it. Both alert paths (candle scan, level break) end in one _deliver_alert: the panel, then on the one-worker alert-db pool the exposure lines are refreshed and the row is stamped alerted_at (signal_log.mark_alerted), then the send. Stamping BEFORE the send is what lets two alerts at the same candle close count each other; an alert that reaches no chat clears its own stamp (unmark_alerted) and commits nothing. No database write runs on the event loop.
• CLI (main.py): prints the report; --json dumps everything (including daily_trend, daily_trend_words, size_line, open_line).
• Dashboard (dashboard_publisher.py): writes snapshot.json, news.json and the static page into the Apache folder. Each card shows the daily-trend words; a plan shows the already-open line, and the size line only when trade_size.dashboard is true (the page is public).

Three display add-ons are attached after the scan by the surface, never inside the signal code. The first is exposure.attach_exposure (2026-09-30): the lot-size line and the already-open line, a leaf module that imports only app_config and signal_log. The other two are advisory: the LLM agent panel (agent_panel.py; since 2026-09-17 only the macro persona runs by default, agent_panel.personas: ["macro"], one DeepSeek call per panel — the technical, risk and contrarian personas carried no information in two live evaluations and stay available for research; the persona returns a probability that the trade closes profitably, anchored to the setup class's base rate from the validated books, shown as the "AI estimate"; with several personas configured the estimate is their equal-weight mean, verdict_source: "mean"; the older macro stance and chair synthesizer paths remain for comparison) and the violent-move watch (violent_watch.py, a "the next hour may be wild" line). No gate reads either of them. The live wrapper analyze_pair hands the fetched frames to the add-ons through a private report._frames attribute so the panel's price context sees exactly the bars the engine saw; the replayable core never sets it.

11. Logging. signal_log.py writes one row per live scan (from the bot and the CLI; the dashboard process never writes either database, section 3) into logs/signals.db (signal, grade, price, indicators, the last 14 candles per timeframe, the news at scan time, the setup, the gate that removed a setup, a gate-config fingerprint and an engine fingerprint; since 2026-09-30 also daily_trend on every real scan and alerted_at on rows whose alert reached a chat). The engine fingerprint hashes the scoring config, so it changes whenever a weight or threshold ships; signal-log comparisons across such a deploy must be split at the deploy time (the 2026-09-09 M15 RSI change is one). The gate fingerprint changes when a gate ships: the M15 daily-trend veto moved M15 rows from f9b2f801de54 to 8a8075688d67 (engine fingerprint unchanged), so M15 comparisons split there too. news_archive.py writes every headline and its score into logs/newsdata.db. Both are fail-safe: a logging error can never break a scan.

12. Measurement, later. Two models read the stored candles and score what happened next: advisory outcomes (advisory_outcome.py: replays each logged setup with next-open entry, stop first, close-only trailing; a cleaned-mid evidence model, not execution P&L) and scan outcomes (model-free forward moves 1 h / 4 h / 24 h after every scan). analyze_signal_log.py turns the database into reports. Since 2026-10-01 the outcome model checks candle continuity bar by bar as it walks: a hole in the stored candles AFTER a trade's exit no longer voids the trade, and a hole reached while it is open ends the walk with the progress kept (so the locked-profit floor is the same whether the stream arrives in one pass or many).

The backtester is the same pipeline on history. backtest/gpu_backtest.py loads candles from backtest/cache/*.parquet (fetched once through the same provider code), precomputes every indicator series over the whole window, and calls analyze_from_dataframes bar by bar with news off. When the daily-trend veto is on it also loads the D1 frame (1,400-day pad) and slices it as-of like every other frame, so the veto sees exactly the closed daily bars a live scan would. Trades are resolved on the raw price path with next-open entries, spread charged per trade, stop-first ties, gap fills at the open, and the trailing stop moved at bar close only. Time filters use bar-close time in both paths, so live and backtest agree. The replay fans out over worker processes; the 5-bar entry cooldown, the only cross-bar state, is applied in an ordered pass afterwards, which is why parallel and sequential books are identical.

---

3. File-by-file guide

Arrows show who imports whom (generated from the code on 2026-09-15).

Entry points and surfaces

File	Role	Uses	Used by
main.py	Command-line signal report (--detail, --json, --panel); the display helpers every surface shares (plain_words, daily_trend_words)	pair_signals, signal_aggregator, agent_panel, violent_watch, exposure, app_config	dashboard_publisher and telegram_bot (formatting helpers)
telegram_bot.py	Owner-locked bot: candle-close scan loops, alerts, breach monitor, heads-ups, /scan /status /news /autoscan /cache /clearcache /help /start	pair_signals, signal_aggregator, providers, agent_panel, violent_watch, exposure, news_engine, llm_client, signal_log, app_config	nobody (a top-level process)
exposure.py	Display-only lot-size line and already-open line (attach_exposure, 2026-09-30). lot_size floors to the lot step in Decimal so the risk limit is never exceeded; size_shared answers whether config.json shares the size with a public audience. A leaf: never raises, never read by the core	app_config, signal_log	main, telegram_bot, dashboard_publisher
dashboard_publisher.py	Static web publisher daemon; writes JSON + HTML into the Apache folder. Its own process with its own news caches and its own DeepSeek daily cap. Never writes signals.db or newsdata.db: main() calls app_config.set_db_writes_for_process(False), a process-wide switch every SQLite writer honours (owner decision 2026-09-17; the bot is the single writer because the dashboard is not always running). It does READ signals.db (mode=ro) for the already-open line since 2026-09-30	pair_signals, providers, main, agent_panel, violent_watch, exposure, news_engine, app_config	nobody. Hard rule: never imports and is never imported by telegram_bot (test-enforced)
static/dashboard.html, .css, .js	The public dashboard page (reads snapshot.json / news.json). Bootstrap 5 + jQuery + Chart.js + Font Awesome straight from a CDN, no build step; M5/M15/H1/H4 each get a card in one column grid, with a diverging Chart.js bar chart for direction and one per card for the indicator votes		copied by the publisher
static/news_summary.php	The one dynamic endpoint: on-demand DeepSeek summary for a clicked headline, whitelist-gated against news.json, own fail-closed daily cap, key read from /etc/gold-dashboard/deepseek.key (never repo config)		installed by the publisher
docs/user_guide.html, docs/beginner_guide.html	Human guides. Keep them in sync with signal behaviour. There are no PDF builds

Signal core (the part the backtester replays)

File	Role	Uses	Used by
config.py	Every tunable value: TechnicalConfig, SignalConfig, NewsConfig, XauStrategyConfig, FilterConfig, wrapped in PlatformConfig; Timeframe, SignalStrength, PAIR_INFO	nothing	everything
pair_signals/xauusd.py	The config factory: overlay + validated per-timeframe overrides + confirmation map. Its docstring is the evidence log for every override	config, app_config, engine	main, telegram_bot, dashboard_publisher, backtester
pair_signals/__init__.py	Registry of pair producers (PAIR_PRODUCERS) and produce_signal	config, signal_aggregator, providers, xauusd	the surfaces, backtester
engine.py	Indicators, regime, composite score (TechnicalEngine)	config	signal_aggregator, backtester, replay_record, pair factory
signal_aggregator.py	Multi-timeframe merge, penalties, grade, gate chain, setup generators, reasons, the daily-trend veto and its one definition (daily_trend_series / daily_trend / daily_trend_needed, shared by live, the replay and the twin's host side); analyze_pair (live) and analyze_from_dataframes (replayable core)	engine, chart_patterns, news_engine, providers, signal_log, config	main, telegram_bot, backtester, pair_signals, analyze_signal_log (agent_panel only for type hints)
chart_patterns.py	The XAU_CP pattern detector		signal_aggregator
position_sizing.py	Decision-time feature builder (65 columns) and the reserved stake multiplier (always 1.0). Bounded lookbacks keep live == backtest	config	backtester (--export-features)

Data providers

File	Role
providers/base.py	BaseDataProvider: cache with candle-duration TTL, forming-candle drop, validation and cleaning, freshness check, weekend and settlement-pause calendar, live fetch depth, paginated fetch_range for the backtester, optional tick stream
providers/oanda.py	Oanda v20 candles (the live feed since 2026-07-28); retry ladder 2 / 5 / 10 s under a 45 s budget
providers/twelve_data.py	Twelve Data candles + WebSocket quotes; registered fallback, but signal-incompatible with Oanda (see section 4)
providers/tiingo.py	Tiingo mid-price candles (research only) and the tick stream used by the breach monitor
providers/__init__.py	PROVIDER_REGISTRY and make_data_provider(config, name=None)

News and LLM

File	Role
news_engine.py	Calendar + RSS headline sentiment with gold-specific keywords; class-level caches; headline scores upgraded by the local XAU model (news_model.py, mode primary live since 2026-09-27) or by DeepSeek (modes off / fallback, _enrich_headline_scores); DeepSeek keeps event direction and the summary; display and warnings only
llm_client.py	Thin requests client for DeepSeek (deepseek-v4-flash, thinking: disabled, one attempt, UTC-day request cap). Every caller has a non-LLM fallback. chat_json never raises and never logs the API key (2026-10-01: a key with a line break made requests quote the Authorization value in its error text); an unserialisable payload is refused before the cap is counted
news_archive.py	Durable headline corpus in logs/newsdata.db (one row per distinct headline, upgrade-only scores, provenance of the scorer). Its own database on purpose: a signals.db rotation in 2026-09 destroyed a 3-week corpus of 4,054 headlines and logs/ is gitignored. Hooked at the end of news_engine._fetch_news_sentiment_raw, before the ±0.1 neutral-headline filter, so neutral headlines are kept for training; do not move the hook. Since 2026-09-17 the corpus is complete for a gold scorer: xau_score holds DeepSeek's gold score for every LLM-scored row (it used to survive only for gold-keyword headlines, so 930 of the first 1,546 rows had no gold label; older rows read NULL), headlines naming no tracked currency are scored and archived too (currencies_json []; they are never returned to the platform), and every new headline goes to DeepSeek in the cycle it arrives, 25 per call. Frozen in news_model.mode primary (live since 2026-09-27): the local model's output is not training data, so nothing is archived except the optional DeepSeek audit copies (deepseek_audit_per_day, shipped 0); off / fallback archive as before, a fallback model row carrying teacher_model xau_transformer@<trained_at>
agent_panel.py	Advisory persona panel (macro by default; technical / risk / contrarian and the chair available through agent_panel.personas for research). Attached post-hoc. v2 (2026-09-17, PANEL_PROMPT_VERSION): every persona sees a code-computed context block (prior move in ATR, distance from the 24 h extreme, candle size, stop/target in ATR, the class base rate from agent_panel_base_rates.json, event timing against the expected hold) and returns p_win plus one reason each way; stance is derived symmetrically (≥ 60 support, ≤ 40 oppose); the verdict is the equal-weight mean (quorum 3, verdict_source: "mean", default). macro (stance verbatim) and chair (synthesizer + _apply_oppose_guard) remain for comparison. The prompt version is logged and hashed. v2.2 (2026-10-01) changed the persona's INPUT, not the prompt text: the event timing reads the cached calendar week attach_panel hands to _price_context (NewsEngine.peek_week_events, which never fetches; a hold past UTC midnight sees tomorrow's release), and the ten headlines are the newest ten, not the first ten in feed order. Persona weights are not to be tuned before about 300 graded outcomes with a held-out half
violent_watch.py	Display-only violent-move warning and heads-up. Attached post-hoc
news_model.py	The local headline scorer (2026-09-27): vendored CPU runtime of the owner's ML-NEWS XAU transformer (DeBERTa-v3-small, two heads: moves gold? / up or down?, ONNX; onnxruntime + tokenizers imported lazily inside load_model, so the bot starts without them) plus a load-once, never-raising singleton (get_news_model, teacher_tag, warm_up_in_background). Gold head only: news_engine._apply_model_sentiment writes xau_score and XAU's per-currency entry, never USD. Bundle (280 MB) lives outside the tree at news_model.path. Spec and deviations: docs/news_model_integration_spec_2026-09-27.md; ops: deploy/README-news-model.md

Logging and evaluation

File	Role
signal_log.py	SQLite logger for every live scan (signals, news_snapshots, scan_outcomes; view signals_with_news); opportunistic scan-outcome labelling; advisory outcome seeding; mark_alerted (stamps alerted_at just before the alert is sent and returns the stamp it wrote), unmark_alerted (clears that stamp when no chat received the alert), the read-only open_alerted_setups (alerted plans whose outcome is still PENDING, one per event, stale rows dropped) and the read-only last_alerted_bars (newest alerted bar per timeframe, for the bot's cooldown after a restart)
advisory_outcome.py	The deterministic outcome model for logged setups. Continuity of the stored candles is judged one step at a time inside the walk (_next_bar_follows); _has_continuity stays all-or-nothing for the scan-outcome labeller
analyze_signal_log.py	Read-only reports on signals.db: trade replay, panel scorecard, parity between surfaces, news lead/lag, scan outcomes (--stored-candles, --news, --parity, --scan-outcomes)
scripts/backfill_signal_outcomes.py	Batch seeding / advancing of advisory outcomes on an imported database (--dry-run, --reseed-unresolvable)
scripts/label_scan_outcomes.py	Batch labelling of scan outcomes for a backlog (--dry-run)
scripts/panel_replay.py	Offline replay of the agent panel on logged, resolved setups: rebuilds each persona's payload (the exact logged one from panel_payloads_json per persona when the row has it, macro alone on live rows since 2026-09-17; else from the stored JSON blobs, interned news and the kept Oanda cache; the CSV column stored_payloads names the personas that replayed a logged payload), runs the CURRENT prompts, and scores Brier / calibration / constant-voter rate / counterfactual books against the logged v1 takes and the base-rate predictor. --dry-run, --max-calls, --limit, --out. Read-only on the DB; 4 DeepSeek calls per event
scripts/build_panel_base_rates.py	Builds agent_panel_base_rates.json (per timeframe × strategy × grade × direction: n, win rate, mean R, TP1 reach, median hold, book MD5) from the exported reference books. Re-run after any re-pin
agent_panel_base_rates.json	The base-rate table the panel anchors its probabilities to (rebuilt 2026-10-01 from the reference books of section 9, the M15 one with the daily-trend veto; generated_at and each book's MD5 are in the file)

Backtesting and research

File	Role
backtest/gpu_backtest.py	The backtester. The indicator precompute over the full history and the resolution of every trade run on the GPU when CUDA works (--backend, Backend: log line names the device or the fallback reason); the per-bar replay through analyze_from_dataframes is the live Python signal code in forked CPU workers by design and is 80–95% of wall time
backtest/gpu_kernels.py	CuPy backend state (lazy probe, never silent, sets CUDA_PATH to the pip header wheel) and the three CUDA kernels: EWM batch, pandas rolling mean/var replica, trade resolver. Each is a bit-for-bit twin of the NumPy/pandas code kept in gpu_backtest.py (--fmad=false, one thread per series or trade); the twins are pinned by Module 36 and by the reference MD5s
backtest/cudaBasedBacktesting.py	Research-only full-GPU twin of the backtester (2026-09-18): same CLI, same cache files, same 63-column book byte for byte, but the per-bar decision runs in CUDA kernels instead of the live Python code, so a book takes seconds (M5 two-year 2.4 s against 115 s). Reuses the loader, the precompute, the resolver, the metrics and the writers from gpu_backtest.py; the host only turns the kernels' numeric codes into the exact strings. No CPU path: --backend cpu is refused. Its numbers count as evidence only while the MD5 parity with gpu_backtest.py holds (section 7)
backtest/cuda_signal_kernels.py	The CUDA decision stage: K1 (engine.py plus the GPUTechnicalEngine overrides, one thread per bar), K2 (analyze_from_dataframes end to end with the gates, the setup generators, XAU_PB, XAU_CP, the regime filter and the replay's per-bar rules, one thread per primary bar), K3 (position_sizing.build_features), K4 (cooldown), K5 (prefix median), K6 (recorder bar rows); the config packer and the name lists the #defines are generated from. A leaf module that never imports the live code; pinned by Module 37
backtest/replay_record.py	Opt-in --record recorder: why every accepted trade was taken, plus calendar events. Book-neutral by contract
backtest/replay_scorecard.py	Read-only reader of recorded runs (DIR --check verifies its invariants)
scripts/ff_calendar_scraper.py	Builds backtest/cache/calendar.parquet (Forex Factory events) for the recorder
backtest/instructions_for_backtest.txt	What every backtest report must record. Follow it
backtest/research/*.md	The decision ledgers (one per campaign). Rule-1 evidence
backtest/cache/ (gitignored)	Candle caches keyed by provider, symbol, timeframe and padded window; only the Oanda windows listed in section 9 are kept

Configuration and deployment

File	Role
app_config.py	Loads config.json once per process; env var → config.json → refuse; validates keys
config.json.example	The schema of config.json (which is gitignored)
deploy/README-dashboard.md, deploy/gold-dashboard.service	Live-server checklist and systemd unit for the dashboard daemon
test_all.py	The QA suite (40 modules; Module 36 is the GPU backend parity module, Module 37 the CUDA decision-stage parity module, Module 38 the local news model, Module 39 the daily-trend veto and the wide-stop cap, Module 40 the trade-size / already-open / daily-trend display lines)

---

4. Data providers

All candle data goes through providers/base.py. Whatever the source, the pipeline sees the same thing: a UTC, oldest-first DataFrame with capitalised OHLCV columns, closed bars only. The base class owns the behaviour the signals depend on; a provider only implements _fetch_raw plus its symbol and interval maps. A test enforces that subclasses do not override the shared methods (supports_streaming and iter_ticks are the sanctioned streaming hooks).

Which provider does what today

Provider	Live candles	Tick stream	Notes
Oanda v20 (data_providers.active: "oanda")	yes, since 2026-07-28	no	Needs oanda.api_token and oanda.base_url; no account id. No client-side throttle. A legacy oanda.account_id key is tolerated and ignored
Tiingo (data_providers.streaming: "tiingo")	research only	yes (wss://api.tiingo.com/fx, mid-price)	Never show a spread from its free quote feed (two-tier indicative quotes)
Twelve Data	registered fallback	yes	About 19% of M5 bars disagree in direction with Oanda and highs sit about $1 too high. On an Oanda outage, stop scanning rather than switch feeds

Behaviour that matters for signals

• The forming candle is always dropped. Live signals would repaint otherwise.
• Cleaning is a bad-print filter, not a volatility filter: a bar more than 10σ from the previous close is removed only when the next close snaps back at least 80% of the way. Real news bars stay, and the newest bar is always kept. (OUTLIER_SIGMA, OUTLIER_REVERT_FRACTION.)
• Every live fetch asks for at least 1,000 bars (MIN_LIVE_BARS): M5 and M15 5,000, H1 1,460, H4 and D1 1,000. The 200-bar EMA needs that much history to match the backtester.
• Weekend: New-York anchored, Friday 17:00 to Sunday 18:00 New York time. There are three independent copies of this rule on the signal path (providers/base.py, signal_aggregator.py, and gpu_backtest._is_xau_blocked_by_calendar) plus one in advisory_outcome.py. Patch all of them, or none; a Module 21 test sweeps all four hour by hour across both daylight-saving transitions.
• The daily settlement pause (Monday to Thursday 17:00 New York, 21:00 UTC in summer) counts as a scheduled closure since 2026-09-13. A "stale data" warning at any other weekday hour is a real feed stall.
• The backtester fetches history through fetch_range (paginated, retries a failed page, logs an error for interior gaps longer than 3 bars) and stores it under backtest/cache/. A DATE window with no bars in it (a closed day at the head of a padded window) is an empty page, not a failure: until 2026-09-30 the Oanda provider raised on it and fetch_range aborted after three tries, so a D1 pad starting on a Saturday could not be fetched at all. Only a closure-sized window gets that reading (EMPTY_WINDOW_MAX, 5 days: a weekend joined to a holiday, and no malformed candle skipped); an empty answer for a longer window (a normal 15-day M5 page) raises, so fetch_range retries it and refuses the history after three tries instead of caching a hole (2026-10-01). A count request (the live path) with nothing in it still raises.
• Daily (D1) candles are UTC days (dailyAlignment=0). Oanda also emits a 1–2 hour stub bar labelled Sunday for the reopening (about 600 of them since 2015); it is a real bar for the H4 confirmation engine, and the daily-trend definition drops Sunday labels before it computes anything. Since 2026-09-30 D1 is a required frame of every M15 scan (live and backtest) and is fetched for display on M5 and H1 (cached until the next daily close; a failure there leaves the line empty and never fails the scan).

---

5. Configuration and credentials

Rule for every credential and deployment setting: environment variable → config.json → refuse to run. There are no hard-coded fallbacks. A gitignored .env at the repo root is loaded at import time and behaves like real environment variables.

config.json holds secrets and deployment settings only. Signal logic, weights, thresholds and gates live in config.py and the pair factory, never in config.json. Blocks (see config.json.example for the exact shape):

Block	What it controls
telegram	token, chat_id, channel_id, alert_min_quality (env TELEGRAM_TOKEN, TELEGRAM_CHAT_ID, TELEGRAM_CHANNEL_ID)
data_providers	active (live candle feed), streaming (tick feed), per-provider API keys (env TWELVE_DATA_API_KEY, TIINGO_API_KEY)
oanda	api_token, base_url (env OANDA_API_TOKEN, OANDA_BASE_URL)
scanning	pairs, timeframes
pairs."XAU/USD".blocked_hours_utc	The base list of blocked hours (default 0, 1, 2, 10, 11, 13, 16, 21)
deepseek	api_key, model, enabled, timeout_seconds, daily_request_cap (env DEEPSEEK_API_KEY)
agent_panel	enabled, telegram, dashboard, min_quality, max_panels_per_day, reuse_ttl_seconds, candle payload knobs, verdict_source (mean, the default since 2026-09-17; macro or chair for comparison), personas (default ["macro"]; any of macro / technical / risk / contrarian; chair needs two or more). All defaults = feature off
violent_watch	enabled, alert_line, headsup, max_headsup_per_day, headsup_min_level (env VIOLENT_WATCH_ENABLED). Off by default
signal_log	enabled, path, log_blocked_scans (env SIGNAL_LOG_ENABLED, SIGNAL_LOG_PATH, SIGNAL_LOG_BLOCKED_SCANS)
news_archive	enabled (default true), path (env NEWS_ARCHIVE_ENABLED, NEWS_ARCHIVE_PATH)
news_model	path (bundle directory, required when mode ≠ off), threads (default 4), mode (off default = DeepSeek/keyword path; fallback = model only where DeepSeek failed; primary = model scores every headline, no DeepSeek headline calls), deepseek_audit_per_day (default 0; primary only, archive-only DeepSeek labels). Env NEWS_MODEL_MODE overrides mode per process (the dashboard unit). Live: primary on both processes
trade_size	Lot-size guidance on the alert (display only, 2026-09-30): risk_per_trade (US dollars; 0 or absent = the size line is off), contract_size (ounces per lot, default 100), lot_step (default 0.01), telegram_channel and dashboard (default false: the size shows the owner's risk, so every Telegram chat other than the owner's own (telegram.chat_id) — the channel, a group — and the public page get it only when these are true). No env override
log_level	Logging level

A legacy top-level market_outlook block is tolerated and ignored with one warning (the feature was removed 2026-09-15). FB2_EMA_LO is a research-only environment knob for the EMA200 proximity band (default 0.5; an unparseable value falls back to the default with a warning).

Files the system writes: logs/signals.db, logs/newsdata.db, logs/telegram.log (one file per day, 30 kept), logs/dashboard.log. Read the databases with mode=ro, never immutable=1 (that mode cannot see rows still in the write-ahead log).

---

6. Testing

• venv/bin/python test_all.py runs everything with mocked network. Expect RESULTS: 632 passed, 0 failed, 632 total (2026-10-01). Module 38's last test loads the real news-model bundle when NEWS_MODEL_DIR (or the configured news_model.path) exists and prints SKIP otherwise. On a box with a visible CUDA device Module 36 fails if the GPU backend falls back (the reason is in the message); with FB2_BACKEND=cpu or no CUDA the GPU-path tests print a notice and the NumPy twins are tested. Module 37 (the CUDA decision stage) adds about 85 s on the GPU because its references replay two synthetic fixtures through the live signal code bar by bar; without CUDA each of its device tests prints a notice and only the arithmetic rule, packing, flag and source-guard tests run. The suite's hermetic app config points both database writers at a suite-private temp dir; before 2026-09-17 the RSS fixture tests wrote fixture headlines into the real logs/newsdata.db, and a Module 32 test now pins the isolation. One Twelve Data interval-map test reaches the network and can time out on a slow connection; a rerun clears it.
• The suite is meant to run on the live box too (Python 3.9, SQLite 3.34, no pyarrow, no CUDA): the four tests that must write a Parquet fixture print a notice and return when no Parquet engine is installed, and a test that needs an older database builds it forwards (_legacy_signals_db39: the base schema plus every additive column but the one under test), never with ALTER TABLE … DROP COLUMN (SQLite 3.35). Verified 2026-10-01 with both Parquet engines blocked and FB2_BACKEND=cpu: 0 failures.
• Test module numbers are cited from this file, so never renumber them. Module 10 and 26 are gaps.
• When a test needs the weekend rule, patch providers.base._is_forex_weekend and remember the other copies. Signal-log fixtures that exercise candle continuity must use a weekday bar. _candles_json stores the last 14 bars, so a longer fixture silently loses its head.
• A test walks the code with ast and forbids sklearn, scipy, lightgbm and torch in live modules and in requirements.txt. Research happens in the dev venv/; anything shipped is numpy-only.
• Behaviour-neutral changes are proven by regenerating the reference books (section 9) and showing the MD5s unchanged.

---

7. Rules for changing signal logic

1. Never change a tuned value without re-validation. Acceptance: positive total R in both halves of a window of at least 12 months, profit factor ≥ 1.3, at realistic cost ($0.65 round trip headline; also report $0.30 for comparability with older books), trailing exits, next-open fills. Marginal edges must also pass the 2024-06-01 → 2025-03-30 out-of-sample window. Backtest windows go back at most 3 years (owner preference, 2026-09-05). round_trip_cost stays at 0.30 because it also drives the live target floor (0.65 would move target 1's floor from about $9 to $19.50); changing it is its own rule-1 event. A change whose purpose is to cut the losing runs rather than add R may instead be judged by a risk-first rule (owner decision 2026-09-30), written before the runs: 3-year R at least 97% of the control, profit factor up by 0.10 or more, max drawdown not deeper, the 2019 → 2026 window better on R, on drawdown and in 2021 + 2022, an unseen holdout, both costs, and at least 3 of 4 neighbouring definitions holding. Under it the halves of a purely rising window are reported, not clauses. The owner chooses the rule per campaign; the default stays the acceptance above.
2. Per-timeframe overrides live in the pair factory, never in the global defaults. The same knob has had opposite effects on different timeframes several times. Record the evidence in the factory docstring.
3. Pair isolation. Gold-specific gates are keyed on the pair's display name so a future pair cannot inherit gold-calibrated logic by accident.
4. Live equals backtest. A new live gate goes into the replay path in the same commit. All time filters use bar-close time in both paths.
5. ROC stays weighted even though it correlates with MACD, Keltner and EMA. Removing it backtested 13R worse; the thresholds were calibrated with it.
6. The backtester's realism settings are results-critical (raw-path resolution, spread charging, next-open entry, stop-first ties, close-only trailing). Touching them breaks comparability with every recorded book.
7. The forming candle stays dropped.
8. Do not add config fields nothing reads. Do not disable indicators in production paths (disabled_indicators is for experiments only; a 0.0 regime multiplier in a per-timeframe regime_weights_override is equivalent for the composite and is the sanctioned vehicle for an indicator-off candidate).
9. EUR/USD was removed on 2026-07-16 after a no-edge verdict (last commit with it 7d3fd95). BTC/ETH work exists only on unmerged branches. Do not re-add either without new data.

Process rules that go with them: every backtest report follows backtest/instructions_for_backtest.txt; every candidate is compared with a control run on the same code and cache, never with an old book directly; the Backend: line (since 2026-09-18 it names the CUDA device or the fallback reason) and the "GPU cache fallbacks: 0" line are part of the evidence; keep docs/ in sync when behaviour changes. Any change to engine.py, signal_aggregator.py, chart_patterns.py, position_sizing.py, the recorder's row layout or the GPUTechnicalEngine overrides is also a parity event for the CUDA research twin (backtest/cudaBasedBacktesting.py): mirror it in backtest/cuda_signal_kernels.py in the same commit, keep Module 37 green, and regenerate at least the M15 OOS book with both scripts and show equal MD5s before a number from the twin is used as evidence. Until then its books are not evidence.

---

8. Validated configuration (what is actually running)

Defaults in config.py are the tuned gold configuration. The factory adds these per-timeframe overrides (each one has its validation record in the pair_signals/xauusd.py docstring):

Timeframe	Confirmations	Overrides on top of the defaults
M5	M15, H1	hour 9 UTC blocked and hour 21 unblocked; chart-pattern entries on (buy-side only: inverse head and shoulders, double bottom, bull flag); pullback entries off; partial EMA alignment scores neutral; trailing stop arms at 0.60R and trails 0.85R; shorts blocked in the transitioning regime
M15	M5, H1 (+ D1 for the veto)	target floor 1R; the mixed trending regime blocked; shorts blocked at 15:00 and 17:00 UTC; trailing stop arms at 0.70R and trails 0.75R; RSI weight halved in every regime; grade-A shorts blocked (sell_min_quality A+, B); daily-trend veto in both directions (daily_trend_veto both, since 2026-09-30)
H1	M15, H4	every short blocked (sell_min_quality empty)
H4	H1, D1	none. H4 remains "unproven" (few trades)

Shared defaults worth knowing: blocked hours 0, 1, 2, 10, 11, 13, 16, 21 UTC; trading days Monday to Friday plus Sunday evening; shorts need grade A+, A or B and high-or-extreme volatility; trailing stop arms at 0.6R and trails 0.75R; gated target floor 0.5R and printed target floor 1R; round-trip cost $0.30 with target 1 at least 30 × cost; the generic and chart-pattern setups floor the stop at $8; the gold ATR setup uses a 1 × ATR target before the grade scale (2× at A+, 1.5× at A) and a 2 × ATR stop; the H4 regime filter is off; news never gates; the daily-trend veto is off (M15 alone turns it on) and the wide-stop cap is off everywhere (wide_stop_cap_atr 0.0, a research knob that failed its rule).

The daily-trend veto failed the same rule on M5 (it deepens the 3-year losing run from 10.1R to 15.2R and loses the 2019 → 2026 window) and on H1 (−9.7R of +70.0R). Do not turn it on there without a new campaign; the daily trend is printed on those alerts as context only.

One semantic to know: blocked_hours_utc is applied to the bar label hour inside the core and to the bar close hour in the live wrapper and the backtester. So H1 can only trade bars closing 04–09, 15 and 18–20 UTC and H4 only bars closing 08:00 or 12:00. Both paths agree, and every book was validated that way. Do not "fix" one side alone.

---

9. Reference books and how to reproduce them

Pinned trade books are no longer kept in the tree (owner decision 2026-09-15; the old ones sit in ~/Development/ForexBot2_trash_2026-09-15/ until the owner deletes that folder). The gate is the MD5 of the exported 63-column CSV (md5sum on the whole file). Regenerate a book from the cache when a trade-level diff is needed; each run takes seconds to a few minutes.

Harness for every reference book: venv/bin/python backtest/gpu_backtest.py --pair XAUUSD --spread 0.65 --workers 12 --primary-tf <tf> --start <start> --end <end> --export-csv <out>, factory config, no --set, --quality, --strategies, --confirm-tfs or --export-features. All of these run offline from the kept cache windows (0 cache fallbacks). Since 2026-09-18 the CuPy GPU backend and --backend cpu both reproduce every MD5 below; earlier reports say NumPy CPU fallback because CuPy could not compile kernels on the dev box until then.

Book	Window	Trades / R / PF	MD5 of the CSV
M5 W2	2024-09-05 → 2026-09-05	781 / +146.07 / 1.52	e599c8a8652f95b7fcdf5a7ef2b564f5
M15 W2	2024-09-05 → 2026-09-05	597 / +155.89 / 1.78	dfc647ee02afd179dd9e031f344ed380
M15 OOS	2024-06-01 → 2025-03-30	258 / +63.78 / 1.80	e87b6de12f90f3b5a5d41a953ee9ff7a
H1 W3	2023-09-05 → 2026-09-05	253 / +70.0 / 1.98	a44fd19b4067f46e5bd98d50cc3eff92
H4 W3	2023-09-05 → 2026-09-05	114 / +21.7 / 1.59	f3587bae3c5dc3eda468ea3588b17dcb

Re-pinned 2026-09-30. M15 only: the daily-trend veto (M15 override 7). W2 216 trades removed, +1.7R, max drawdown 15.6R → 11.4R; OOS 55 removed, −4.7R (the accepted cost of the risk-first rule in a rising window), drawdown 12.4R → 9.1R; previous pins 5b65ecf6… (813 / +154.15 / 1.53) and 09b8a8e7… (313 / +68.50 / 1.67). M15 W3 with the veto 824 / +205.12 / 1.73, max drawdown 12.4R, ad3b0dbda0bf77b659caea1c53be6a50 (was 70195fb0…, 1105 / +195.79 / 1.49, drawdown 20.9R). The M5, H1 and H4 pins are unchanged. An M15 book needs the D1 frame of its window (1,400-day pad) in the cache; --set xau_strategy.daily_trend_veto=off reproduces the previous M15 pins byte for byte.

Re-pinned 2026-09-26. M15: the grade-A SELL block (M15 override 6, 162 trades removed, +8.0R); previous pins b4aa07e3… (931 / +146.14 / 1.43) and 9fd624fb… (355 / +60.67 / 1.49); M15 W3 (3-year) with the block 1105 / +195.79 / 1.49, 70195fb0e6865b3a46ee8a9aa8844e51. All four timeframes: the 1R printed-target floor, label-only (M5 W2: 544 rows change tp1 / rr_planned, H1 W3 252, H4 W3 108, M15 W2 80; trades and R identical to 49f8e602…, 7b848c91…, 186e677f… and to the M15 block-only book 497520ef…; every rr_planned is now ≥ 1.0).

Re-pinned 2026-09-17 for a label-only change: the exported strategy column now carries the generator's own stamp (XAU_ATR / XAU_PB / XAU_CP / GENERIC) instead of a tag inferred from the first reason's text, which had labelled every gold ATR setup GENERIC. All 62 other columns of all five books are byte-identical to the previous pins (29120a1e…, d185f099…, 6f8f24b4…, 3b45d12b…, e75adae1…); trades, R and PF are unchanged. --strategies accepts the live names (XAU still means XAU_ATR).

Other current figures at $0.65 (2026-10-01): M5 W3 (3-year) 870 / +153.64R / PF 1.50 / max drawdown 10.1R (d20ec8a3…); M15 W3 824 / +205.12R / PF 1.73 / 12.4R; M5 12-month (2025-08-14 → 2026-08-14) 560 / +108.81R / PF 1.54; M15 12-month (2025-08-04 → 2026-08-04) 266 / +92.3R / PF 2.12 / 8.3R; H1 12-month (2025-08-04 → 2026-08-04) 91 / +22.2R / PF 1.79. The long robustness window 2019-01-01 → 2026-08-07 (allowed as supporting evidence, owner decision 2026-09-30): M15 1473 / +223.33R / PF 1.42 / max drawdown 41.5R (aa5253aa…; by year +14.0, +45.1, −27.3, +3.2, +1.8, +53.5, +92.5, +40.6), M5 1052 / +128.10R / PF 1.34 / 18.4R (3f0c091c…; −1.0, −2.8, −1.7, −2.2, −2.8, +1.0, +40.5, +97.0: the M5 rules take few trades and earn nothing before 2025). The unseen holdout 2026-09-05 → 2026-10-01: M15 18 / −4.62R, M5 33 / −2.45R, H1 3 / −3.04R. The pins of the table above were reproduced on 2026-09-15 after the cache prune, on 2026-09-18 on both backends (backtest/research/gpu_backend_2026-09-18.md) and on 2026-10-01 with both scripts.

The CUDA research twin (backtest/cudaBasedBacktesting.py, section 3) reproduced all five MD5s above on 2026-09-18 in 0.8–2.4 s per book, plus the four 6-month acceptance books (2026-03-18 → 2026-09-18, M5 and M15 at $0.65 and $0.30: 86111c36…, e70d0252…, b8c98083…, e66b5fbd…, each equal to the reference script's), the M15 OOS --export-features CSV (6239d197…) and --record --record-bars tables, and ten research-flag variants (backtest/research/cuda_backtester_2026-09-18.md). Parity was re-proved for the 2026-09-30 change: the five pins above with both scripts, six --set books on W3 (veto both / buy / sell and the stop cap on M15, veto and cap on M5), the M15 OOS --export-features CSV with the veto on (732f396bb3ee6036b62750a8c755194e) and its --record --record-bars tables (backtest/research/daily_trend_campaign_2026-09-30.md).

Kept cache windows (backtest/cache/oanda_XAU_USD_<tf>_<paddedStart>_<end>.parquet; the padded start is the window start minus 62 days for M5/M15/H1, 234 for H4 and 1,400 for D1, so the D1 and H4 files of one run have different start dates than its M5 file): 3-year 20230705_20260905, 2-year 20240705_20260905, OOS 20240331_20250330, M5 12-month 20250613_20260814, M15/H1 12-month 20250603_20260804, 5-year 20210531_20260804, 7.6-year 20181031_20260807, the 2026-08-14 parity window 20250814_20260814, the 6-month CUDA acceptance window 20260115_20260918 (M5/M15/H1), their D1/H4 pads, and calendar.parquet. Added 2026-09-30 for the daily-trend veto (an M15 run needs the D1 frame of its window): 1d_20201105_20260905 (W2), 1d_20200801_20250330 (OOS), 1d_20150303_20260807 (the 7.6-year window), 1d_20211004_20260804 and 4h_20241213_20260804 (the M15 / H1 12-month window), and the holdout window 20260705_20261001 (M5 / M15 / H1) with 1d_20221105_20261001 and 4h_20260114_20261001. The M15 3-year run reads 1d_20191105_20260905, which H4 W3 already kept.

The research recorder (--record DIR [--record-bars]) writes Parquet files describing every accepted trade (indicator votes on all three timeframes, penalties, setup, outcome, calendar events). It never changes the book; pair every recorded run with a control run and show equal MD5s. Scorecards from it are in-book statistics, never decisions.

---

10. Things that bite

• Never build a bare PlatformConfig() for a real scan; use the factory. A test enforces factory equivalence with the inline construction the backtester uses.
• The backtester names timeframes 5min, 15min, 1h, 4h. M5 is a hard exit.
• --spread defaults to 0.30. Pass 0.65 for every headline result.
• --export-features changes the CSV; never use it on a parity run.
• The signal log's spread column is the median bar range, not a broker spread. actionable means grade ≥ C and a setup exists; the action text is written before the gate chain and can disagree with the final grade. scan_kind is blocked (weekend or day filter), failed (a required frame was missing) or normal; readers skip the first two. Several surfaces log the same bar, so readers dedup by the first row per (timeframe, bar_ts). Same-setup re-logs within 60 minutes are DUPLICATE rows; sum advisory R over RESOLVED rows only.
• Headlines and events in the signal log are content-addressed into news_snapshots; read through the signals_with_news view or signal_log.read_news.
• A non-empty cleaned_warnings_json is not a chart-pattern marker (a stale spread-gate veto lands there too since 2026-09-08); use setup_json.strategy.
• The --stored-candles scorecard in analyze_signal_log.py still advances outcomes without the locked-profit floor step that the logger and the backfill script use (owner decision pending), so the two measurement paths can disagree on gap rows. Those are now only the trades still OPEN at a hole (3 of 111 setups on the 2026-10-01 database copy; until then the replay also threw away every trade that had exited before a later hole, 12 of 111).
• There is no holiday awareness anywhere. On Labor Day 2026 Oanda published no bars for hours, the bot rescanned the stale bar 43 times, and the violent-move watch called the 12–14 UTC slot hot. Interior gaps in history are logged, never fatal, for the same reason.
• Existing databases still contain an empty-going-forward outlooks table from the removed market-outlook feature. Nothing reads it.
• The local news model has a gold head only. _apply_model_sentiment may write xau_score, XAU's per-currency entry and, for a headline naming no other currency, the overall score; USD and every other currency keep the keyword value (the pair sentiment is XAU − USD). It never touches _llm_sent_cache, and a not-relevant headline displays 0.0 while xau_score keeps the raw number. In primary mode nothing is archived but the DeepSeek audit copies, and the dashboard process never archives in any mode (its DB writes are off), so a retrain needs deepseek_audit_per_day > 0, a spell in fallback, or a labelling run in ML-NEWS.
• The news score is a trailing echo of the last day's price move (r ≈ 0.67 with the past day, ≈ 0.03 with the next). Both news gating flags stay off.
• The level-breach rescan and the candle loop both skip the settlement-pause window (_closed_bar_in_settlement); a rescan there only re-reads the last pre-pause bar.
• The alert cooldown counts bars of report._frames (set by analyze_pair). A report built by hand has no frames and falls back to clock time, which releases the first bars after the settlement pause or the weekend too early; that is the fallback, not the rule.
• An alert's row is stamped alerted_at BEFORE the send and the stamp is cleared when no chat received it, also when the alert's task is cancelled while the stamp is being written (a shutdown at that moment: the worker job cannot be stopped, so its future's done-callback undoes it). A cancellation in the MIDDLE of the send keeps the stamp, because a chat may already hold the message. Two residues, both accepted: if a send fails completely after another alert already counted the row, the other alert's "already open" line is one too high; and a process KILLED between the stamp and the send leaves a stamp on an alert nobody received (it also seeds the cooldown at the next start, which errs on the quiet side).
• news_engine._record_source("news_model", …) is the only sign that the local headline model failed a batch (WARNING once, /status line News model: not scoring); in primary mode nothing else scores those headlines for the cycle. The line describes the latest cycle that needed the model: a cycle with nothing for it to score (fallback mode with DeepSeek healthy) drops an older failure instead of leaving it on /status. /status failure details are exception text and are Markdown-escaped; an unpaired underscore made Telegram reject the reply.
• The agent panel's scorecards split at PANEL_PROMPT_VERSION v2.2 (2026-10-01): same prompt text, different input (calendar-week event timing, headlines newest first). _price_context never fetches the calendar itself; the offline replay passes an as-of context and must not see today's events.
• The Oanda empty-window rule cannot catch an anomalous empty answer for a window of 5 days or less (a short head page; every 3-day M1 page, a timeframe nothing fetches).
• The agent panel never runs inside analyze_from_dataframes, and its cached verdict is never reused when the evidence changed; the daily cap is the real spend bound.
• Live == backtest also means the backtester never writes to signals.db and never builds a NewsEngine.
• Twelve Data candles are not a drop-in replacement for Oanda (section 4).
• pyarrow (Parquet) is installed in the dev venv only; live code never touches Parquet.
• CuPy from pip (cupy-cuda12x) cannot compile a kernel without the CUDA headers; the nvidia-cuda-runtime-cu12 wheel provides them and backtest/gpu_kernels.py points CUDA_PATH at it. Until 2026-09-18 that failure was swallowed: every book was built on NumPy while the file was called a GPU backtester. Now the fallback reason is in the Backend: line and Module 36 fails on a box that has a visible GPU but no working backend. cupy is banned from the live roots by the same AST guard as scikit-learn.
• The CUDA kernels are bit-for-bit replicas of the CPU code: one thread per series (EWM, rolling) or per trade, compiled with --fmad=false. Do not "optimise" them with a parallel scan or a two-pass variance: the last bits change and a threshold can flip somewhere in 500k scans. A single sequential series is not faster on the GPU than pandas' C loop; the win is the batch across series and trades, and the replay itself stays on the CPU (same code as live).
• The replay workers are forked after the CUDA context exists and never touch the device (every cache array is host NumPy, asserted before the fork). Python 3.12 may warn about forking a multi-threaded process under -X dev; that is expected.
• backtest/cudaBasedBacktesting.py is a CUDA re-implementation of the decision stage, exempt from "replays go through the real signal code" only while its book MD5 equals gpu_backtest.py's. A signal change that is not mirrored in cuda_signal_kernels.py makes the twin drift silently: no test compares it with the live code except Module 37 and the paired MD5 runs. It has no CPU fallback, refuses --backend cpu, and every description, warning and reason string is produced on the host from the same f-string templates as the live code; the kernels emit codes and numbers only.
• The kernels keep Python's evaluation order on purpose: min/max as argument-order ternaries (never fmin/fmax), numpy's pairwise summation for mean/nanmean, pandas' online rolling and EWM recurrences, Python 3.12's Neumaier-compensated sum() for the multi-timeframe mean, round(x, 2) through an error-free product, atr_percentile = count / 50 * 100 (not count * 2: 15/50·100 = 30.000000000000004 flips "low" to "normal"). A cleaner formulation flips a threshold somewhere in 500k bars. The as-of alignment compares int64 nanoseconds; a mixed-unit index once returned slice length 0 for every confirmation frame.
• A printed-target change is not free: the spread gate judges the target, so a bigger target can switch a signal from the GENERIC swing stop to the XAU_ATR 2 × ATR stop. The 1R printed floor (tp_print_floor_rr) sits after the gates in all three generators for that reason; keep it there, and never feed the printed target back into a gate.
• "Stop hunting" was tested on the live log (2026-09-26): 71% of live stop-outs ran a full stop distance or more past the stop, the same as the books' 70–72%. The losers were real moves.
• The daily-trend veto has ONE definition (signal_aggregator.daily_trend_series; EMA span 50 and lookback 60 are module constants, validated literals). Live, the bar replay and the twin's host side (trend_flags) all call it; the kernel only receives a flag per bar. Changing the literals, the Sunday filter or the as-of rule (a daily bar labelled D counts from D + 1 day ≤ the primary bar's close) is a rule-1 event and a parity event. The live frame holds 1,000 daily bars and a backtest pad 1,400 days; the EMA seed has decayed below float precision in both, so the states agree.
• With the veto on, D1 is a REQUIRED frame of the M15 scan: a failed D1 fetch fails the scan (DATA UNAVAILABLE: 1d — scan failed), it does not fall back to "no veto". After a deploy watch that rate. Inside the core a missing D1 slice only adds the notice Daily trend filter requested but 1d data unavailable and ships the setup; the replay reaches that state only before the first daily bar of its pad.
• The veto is the LAST step of the core (after the chart-pattern path and the stale-veto cleanup), so gate_reason lists DAILY TREND FILTER ahead of the gate chain. The rule itself only removes a plan. In a book a removed entry also moves the 5-bar cooldown, which could free a later entry: that is possible in principle and happened 0 times in every book of the campaign (an observation, not a guarantee; count the added trades in any new comparison). The wide-stop cap never changes the trade set either (it is applied after the min-RR and spread gates); a smaller stop can lower the printed 1R target, never raise it.
• The control for any M15 candidate is now the factory config WITH the veto. Comparing against a pre-2026-09-30 M15 number needs --set xau_strategy.daily_trend_veto=off (reproduces 5b65ecf6… / 09b8a8e7… / 70195fb0…).
• The veto does not find big losers. The trades it removes are break-even after costs (−0.03R to −0.05R each against +0.15R to +0.25R for the kept ones); the gain is fewer flat trades and shorter losing runs. The SELL half is carried by the rising 3-year window and the BUY half by the falling years of 2019 → 2026; buy-only and sell-only both fail. 2021 is not improved and 2023 is worse: in a sideways year the daily trend flips late.
• The lot-size line reveals the owner's risk per trade. It is stamped per surface (exposure.attach_exposure) and the channel copy of an alert is built separately (format_signal_message(report, channel=True)); never print report.size_line on a public surface without checking exposure.size_shared. The dashboard snapshot is public.
• "Already open" is the advisory outcome model's view of ALERTED plans (alerted_at set, outcome PENDING), not a broker position. Rows alerted before the 2026-09-30 deploy carry no alerted_at, so the line starts empty and fills as alerts go out; a PENDING row older than the model's own time-out (300 bars, with a weekend allowance) is not shown. Stacked entries were not worse trades in the books, so the line is information, never a reason the code skips an alert.
• signals.daily_trend is the core's value on M15 and the live wrapper's display stamp on M5, H1 and H4 (_stamp_daily_trend_for_display, one extra cached D1 fetch on M5 / H1); it is NULL on blocked and failed scans. The stamp sits in analyze_pair, never in analyze_from_dataframes.
• The dashboard's due-timeframe check uses the raw wake time; flooring wake − buffer skipped every H1 and H4 scan once (2026-08-02).
• The agent panel's price context reads the frames analyze_pair stashes on report._frames. A report built by hand (scripts, tests) has no frames, so pass them to _price_context / run_panel explicitly or the context carries geometry and base rate only. scripts/panel_replay.py rebuilds them from the kept Oanda cache; a database copied from live is never migrated (read-only), so the replay tolerates missing v2 columns and rebuilds payloads from the stored blobs.
• The AI estimate is a calibrated base rate, not a filter: on the 37 resolved setups of 2026-09 the v2 personas sat within ±2 points of the class win rate (Brier 0.262 vs 0.263 for the base rate alone, 0.299 for the old macro verdict) and separated nothing. Do not gate on it; do not iterate prompts on that sample again (backtest/research/panel_v2_2026-09-17.md).

---

11. Decision ledger

One line per validated change or verdict, newest first. Full evidence in the named report.

Date	Decision	Where the evidence is
2026-10-01	Bug review of the whole codebase (owner request), then the fixes. Signal path clean: indicators, grading, gates, the daily-trend veto, the as-of logic and both backtesters checked, and no live module needs a newer Python or SQLite than the server has. 17 defects elsewhere, all FIXED, each with a test; books byte-identical (the five pins reproduced on both scripts, 0 cache fallbacks). Alert path: the "already open" line was stamped at scan time, so two alerts at one candle close never counted each other (the M5 and M15 books enter together 42 times in three years); the row is now stamped just before the send on a one-worker alert-db pool and un-stamped when no chat received it (_deliver_alert, one path for the candle scan and the level break; no SQLite write on the event loop any more). The 5-bar cooldown counted clock time and forgot its state at every restart: it now counts the scan's own bars like _apply_cooldown (the old rule released the 22:00 UTC bar after the settlement pause: 6 M15 and 3 M5 entries the books never take in three years) and is restored from signals.db at start-up. The lot-size line went to every target that was not the channel (a group the owner ran /autoscan in): now the owner's own chat only. Measurement: the outcome model rejected a whole candle stream for a hole anywhere in it; continuity is now judged bar by bar in the walk, so the stored-candle replay resolves 108 of 111 logged setups instead of 99, all 108 equal to the R the live resolver stored. History fetch: an empty answer is accepted only for a window of 5 days or less with no malformed candle (a longer one is retried, then refused). News: one unreadable headline no longer costs the local model its whole batch (verified on the real bundle), a failing batch is a WARNING and a /status line, no DeepSeek audit call in a process that does not archive, the event-direction cache is per release. LLM client: chat_json never raises and never logs the key; the scan's news block is guarded (M15 OOS pair re-proved, Module 37 green). Panel (PANEL_PROMPT_VERSION v2.2, a new scorecard era): event timing over the cached calendar week, headlines newest first; the replay script uses the logged macro payloads it used to ignore (39 of 110 events). Tests: no DROP COLUMN (SQLite 3.35), Parquet-writing tests skip without pyarrow, the suite passes with both Parquet engines blocked and no GPU. Also fixed on the way: /status failure details are Markdown-escaped. Independent review of these fixes (a second agent, 20,000-case differential fuzz of the outcome model among its checks): four further LOW defects, fixed the same day — an alert cancelled while its row was being stamped kept the stamp; the panel's calendar read could trigger a 40 s refetch (now peek_week_events); a stale model-failure line could stay on /status; an HTTP error body was logged without redaction. Independent verification of the 2026-09-30 direction change: every figure reproduced, no signal changed by this pass, caveats recorded in section 5.13 of the campaign report. Suite 632 / 40 modules	plan file ~/.claude/plans/you-are-a-quant-velvety-wave.md; sections 2, 4, 6 and 10 of this file
2026-10-01	New data sources, report only (owner request). Eight outside series tested against gold's own daily trend: the dollar (EUR/USD), silver, the US 10-year note, the real yield and the broad dollar index (FRED), the gold fund's share count, CFTC managed-money positioning, and Oanda's own client positioning (the position-book endpoint DOES answer on the practice API, with history to 2017; the plan had expected it gone). None shows a forward move of gold beyond gold's own trend (largest t-values 1.2 and 1.8); on the books only one flag, M5 trades against the broad dollar index (−11.2R over 435, losing in both eras), which is the 2025–2026 rally seen from another angle and does nothing on M15: WATCH, a hypothesis. Nothing ships, no feed added, all data stayed in the scratchpad	backtest/research/new_inputs_2026-10.md
2026-10-01	Context screens after the veto (owner request): weekly trend, four-hour trend, daily-trend strength, daily volatility state, place in the 20-day range, tick-volume surge, time to a high-impact US release; rule written first (a state needs 8% of both books and a loss in all four slices to earn a replay). NO state nominated on M15 or M5, no replay, nothing ships. Description: M15 has no losing pocket left in these views; M5 earns only where everything agrees (with the weekly and four-hour trend +0.16R to +0.24R per trade, against them about 0) and near the 20-day extreme in the trade's direction, so a "do not chase" filter would cut the part that pays	backtest/research/context_screens_2026-10.md
2026-09-30	Direction and setup-size campaign (owner request; RISK-FIRST rule approved by the owner, 2019 → 2026 allowed as supporting evidence). Daily-trend veto SHIPPED on M15 (override 7, daily_trend_veto both): a finished setup against a clear daily trend (close vs 50-day EMA and vs 60 days earlier, closed D1 bars, Sunday stubs dropped) is removed. PASSED every clause: W3 824 / +205.12R / PF 1.73 / max DD 12.4R vs 1105 / +195.79R / 1.49 / 20.9R; 2019 → 2026 1473 / +223.33R / 1.42 / 41.5R vs 2091 / +192.53R / 1.24 / 64.8R; unseen holdout −4.62R vs −6.63R; both costs; 3 of 4 neighbouring definitions. Accepted cost: W2 first half +61.8R vs +70.1R, old OOS window −4.7R. FAILED: buy-only and sell-only on M15, every mode on M5 (W3 drawdown 10.1R → 15.2R, long window worse) and on H1 (−9.7R). Wide-stop cap FAILED rule C (K = 6 +11.8R on W3 but K = 5 −18.6R on the long window: a peak, not a plateau); knob kept at 0.0. M15 books re-pinned (W2 dfc647ee…, OOS e87b6de1…), M5 / H1 / H4 pins unchanged, twin MD5-equal throughout, recorder parity re-proved. Display lines SHIPPED (books byte-identical): lot size for the owner's risk limit (trade_size in config.json, private by default), alerted same-direction plans still open (signals.alerted_at, read-only open_alerted_setups), the daily trend in words on every surface and in every logged row. Measured on the way: taking every M5 + M15 alert, two entries in three already have a same-direction plan open and up to 13 are open at once; those stacked entries are not worse trades. FIXED: the Oanda provider raised on an empty date window, so fetch_range aborted on a pad starting on a closed day. Stops and exits re-screened (25 variants): at a plateau, tighter is far worse. Suite 607 / 40 modules	backtest/research/daily_trend_campaign_2026-09-30.md; factory docstring M15 override 7
2026-09-27	Local news scorer, round two: the owner's ML-NEWS model is now a DeBERTa-v3-small two-head transformer (gold-relevant? then bullish/bearish), distilled from 17,245 DeepSeek gold labels (Sep 8–22 archive + a 2019–2026 Reuters/CNBC/MarketWatch crawl labelled by DeepSeek, 560 calls). Golden set 72/78 (DeepSeek 78, keywords 24, the 09-17 TF-IDF 39), agreement with DeepSeek on unseen live days ~72% (ceiling 90%), 19 ms/headline on one CPU thread as int8-embedding ONNX; passed the four promotion criteria fixed on 09-22. OWNER DECISIONS: it is to score headlines here in mode primary with NO DeepSeek for headline scoring (DeepSeek stays for event direction, summaries, the dashboard endpoint); model files live OUTSIDE this tree at a configured path (280 MB, never in git); shadow mode declined (the archive already allows offline replay). IMPLEMENTED the same day (branch sql-integration): news_model.py (vendored runtime + never-raising singleton, lazy onnxruntime / tokenizers), news_model config section + NEWS_MODEL_MODE env, news_engine._enrich_headline_scores / _apply_model_sentiment (gold-only rules A–F) / _audit_with_deepseek, archive provenance for model rows, deploy/README-news-model.md, Module 38 (15 tests, one against the real bundle: bullish / neutral labels, 7 ms per headline on the dev box). Two further owner decisions during implementation: primary mode stops writing newsdata.db (only the DeepSeek audit sample, shipped 0, is archived) and both processes load the model. Books unaffected (news never enters signals); no signal file touched, no parity event. ML-NEWS news_transformer.py received the same two load_model edits (commit 9496cd9 there)	docs/news_model_integration_spec_2026-09-27.md; ~/Development/ML-NEWS/eval/RESULTS_2026-09-27.md
2026-09-27	Single "conviction" number vs the two-number grade (owner question: one number that is easier to understand and to decide on). Pre-registered; switch signals.grading_mode (pair default = the shipped rule, byte-identical on both scripts, all five pins reproduced; min = min(|score|, 0.6 × conf); product = |score| × conf / 100; thresholds from the control's corner points, no tuning; mirrored in K2, twin MD5-equal to the reference on both new paths). FAILED on M5 and M15: K-MIN moves 43–54 trades for −1.2R / −2.9R (a, b, e / a, b, d); K-PROD −29R on M15 with PF 1.29 and W3 DD 28.9R, neutral on M5. H1 / H4 K-PROD +11.9R / +17.0R on W3 by taking 40–100 extra low-score trades (H1 PF 1.98 → 1.75, DD doubles): a hypothesis, not evidence. Nothing ships; the knob stays research-only. Code review (high) before the runs: 6 findings fixed, 1 recorded. SHIPPED the display half of the question (owner request the same day): alerts lead with the grade and print score / confidence as words (main.plain_words, same line on Telegram, /status, CLI and the dashboard card); books untouched. Suite 566	backtest/research/conviction_campaign_2026-09-27.md
2026-09-26	Quant review + live log 2026-09-04 → 09-25 (100 setups replayed on the scans' stored candles): stop-hunt claim REFUTED (71% of stop-outs ran ≥ 1R past the stop, same as the books); −25.1R of the −37.8R raw live book were consecutive-bar re-entries the backtester's cooldown never takes; the cooldown view sits at the 3rd–5th percentile of the books' three-week windows. SHIPPED: 5-bar alert cooldown; alert wording (trail first as the tested exit, TP1 a guide with its R, planned RR, validated-book line per class); M15 grade-A SELL block (sell_min_quality A+, B, PASSED a–f: W2 +154.15R/813 PF 1.53 vs +146.14/931 PF 1.43, OOS +68.5 vs +60.67, W3 DD +8%); 1R printed-target floor in every generator after the gates (tp_print_floor_rr, owner rule "TP1 at least 1:1"; label-only re-pin of all four timeframes). FAILED: M15 min_rr_ratio 1.0 (trade-identical, −2.0R); a pre-gate target floor (−24.6R). OWNER'S CALL: M5 tp_floor_rr 1.0 adds +48.9R on W2 with both halves up but max DD 10.1 → 15.9R (ladder 0.6–1.0 non-monotone); not shipped. Telegram / dashboard AI block rewritten for a new trader (owner rule): one plain headline against the usual odds, the chance "in 100", one reason for and one against, no verdict word or confidence figure; persona prompts ban trading jargon (PANEL_PROMPT_VERSION v2.1). Two delivery bugs from the DB checks fixed: the macro-source panel verdict never stored its probability (all v2 rows NULL); the level-breach rescan re-logged the 20:45 M15 bar three times every evening after the settlement pause (guard added to _handle_breach). Indicator grid and volume / daily-direction screens (Part C) not run	backtest/research/rr_and_sell_campaign_2026-09-26.md
2026-09-18	Full-GPU research twin SHIPPED: backtest/cudaBasedBacktesting.py + backtest/cuda_signal_kernels.py (owner decisions: research only, gpu_backtest.py stays the base reference, every research flag supported, the one exemption from instructions item 6 recorded there). Proof: the five reference MD5s, four 6-month acceptance books (2026-03-18 → 2026-09-18, M5 and M15, $0.65 and $0.30), the M15 OOS --export-features CSV and --record --record-bars tables, and ten research-flag variants, all byte-identical to the reference script; Module 37 (16 tests) pins every stage against the live code. Books take 0.8–2.4 s instead of 13–115 s. Rule added: any signal change is a parity event for the twin	backtest/research/cuda_backtester_2026-09-18.md
2026-09-18	GPU backend made real: CuPy had never compiled a kernel on the dev box (CUDA_PATH missing for the pip layout), so every book to date was NumPy. SHIPPED: backtest/gpu_kernels.py (lazy, loud backend probe; nvidia-cuda-runtime-cu12 header wheel; CUDA kernels for the EWM batch, the pandas rolling mean/var replica and the trade resolver, each a bit-for-bit twin of the CPU code), --backend auto/gpu/cpu + FB2_BACKEND, Module 36. All five reference MD5s reproduced on the GPU and with --backend cpu; precompute GPU == CPU on all 42 cached windows (2.34 M bars). Wall time roughly unchanged: the replay is CPU by design	backtest/research/gpu_backend_2026-09-18.md
2026-09-17	Local news scorer evaluated (the owner's scikit TF-IDF model in ~/Development/ML-NEWS, trained on 606 DeepSeek gold labels): golden set 39/78 vs DeepSeek 78/78 vs keywords 24/78; out-of-sample agreement with DeepSeek 64% (DeepSeek agrees with itself 90%); cleaning / negation marking / label weighting move nothing; backfilling the 961 missing gold labels lifts macro-headline agreement 38% → 52% but only reshuffles golden-set errors. Verdict: a fallback, not a replacement. SHIPPED to grow a complete corpus: xau_score archived for every headline, no-currency headlines scored and archived (never shown), every new headline sent in its cycle (25 per call). Dashboard process made database-read-only (set_db_writes_for_process); the bot is the single writer. Books unaffected (news never enters signals)	~/Development/ML-NEWS/eval/RESULTS_2026-09-17.md (evaluation scripts live outside the repo by owner instruction)
2026-09-17	Live review of two weeks of signals.db: live == backtest bar for bar (26/29 trades matched, R within 0.14); the raw advisory −7.15R is cluster re-entries the validated books never take (cooldown view −1.6R, backtester +1.1R on the window, an ordinary fortnight); composite score inversely related to the 24 h move on all timeframes (two-week bear reading, test in-book with --record-bars); big-candle gate re-tested and REJECTED again (removes +32R / +60R of net-positive trades). Agent panel v2 SHIPPED: probabilities anchored to book base rates, code-computed decision context, no news doctrine, mean aggregator (verdict_source: "mean"), prompt version logged, offline replay harness; owner decision the same day: only the macro persona runs live (personas: ["macro"], 1 call per panel; technical / risk / contrarian carried no information in either evaluation, kept for research); backtester strategy tag fixed (label-only, books re-pinned). Delivery findings: dashboard daemon on stale code since Sep 10, bot outage Sep 9	backtest/research/signal_log_review_2026-09-17.md, backtest/research/panel_v2_2026-09-17.md
2026-09-15	Backtest folder cleanup: diagnose_filters.py, shadow_scan.py, and the two one-time news scripts removed (last commit with them 931b265); every pinned CSV, the research output folders, the BTC/ETH cache snapshots, the Tiingo caches and 13 uncited Oanda windows moved to ~/Development/ForexBot2_trash_2026-09-15/ (346 MB) pending the owner's delete; all five reference MD5s reproduced afterwards. Suite 506 / 33	commit b0b71e1
2026-09-15	AI market-outlook feature removed (module, hourly Telegram brief, --outlook, outlooks writer, config block). Legacy config block tolerated with a warning. Books byte-identical	commit 931b265
2026-09-13	Bug audit, 11 delivery/measurement fixes, books byte-identical: NEUTRAL-signal C/B setups now alert (about 17% of the H1 12-month book's R, +3.7R of +22.2R, was never sent); a missing confirmation frame fails the scan; pandas utcnow replaced and pandas<4, numpy<3 pinned; advisory outcomes keep locked-profit floors (GAP_LOCKED); scan-outcome labeler bounded to 72 h; settlement pause is a scheduled closure; fetch_range retries pages and audits gaps; cache-fallback count logged and must be 0; partial regime_weights_override raises; the bot scans each bar once and attaches its log handler in main() only; the dashboard catches up missed timeframes. Verified sound and unchanged: composite maths, scan-outcome labels, outcome exits, Oanda alignment, no look-ahead	plan file ~/.claude/plans/you-are-python-software-joyful-teacup.md, commit 2836024
2026-09-10	Improvement cycle 1: 3-year baseline re-established; cost ladder shows no red flag (PF ≥ 1.3 everywhere at $0.80); the intraday edge is a fat right tail (top 5% of trades carry 94–96% of M5/M15 R). H1 divergence-to-zero REJECTED. M15 RSI × 0.5 sits on a stable plateau. SHIPPED: live fetches floor at 1,000 bars and the backtester pads history to match (books trade-identical). Parked cycle-2 candidate: block M15 grade-A SELLs (230 trades, −14.4R, negative in every year and OOS)	backtest/research/cycle_2026-09-10.md
2026-09-09	Weight test on the recorded replays: M15 RSI multiplier × 0.5 in every regime PASSED (2-year +117.6R → +146.1R, OOS +40.6 → +60.7R) and SHIPPED as an M15-only override; all M5 MACD candidates FAILED; M15 EMA × 2 and divergence-off FAILED	backtest/research/weight_test_2026-09-09.md
2026-09-09	Replay recorder shipped (--record, --record-bars), books byte-identical. First look is descriptive only: Bollinger opposes 94% of M5 entries; divergence-opposed M5 trades did best; bar-level baseline drifts +1.9 ATR/24 h on M5, so adjust for drift before judging BUY votes	backtest/research/replay_recording_2026-09-09.md
2026-09-08	Delivery fixes: stale spread-gate vetoes moved to cleaned_warnings (books identical; only the warning text on 20–33% of A-grade alerts changed); alerts require a setup; every send logged with its row id; telegram.log rotates daily × 30	commit history, 2026-09-08
2026-09-08	News archive shipped (logs/newsdata.db). Do not train a local headline scorer before about 4,000 headlines over 3+ weeks, time-split, headline text only, graded against the golden set (git show 1cebebf^:backtest/research/llm_news_golden.csv)	news_archive.py docstring, backtest/research/llm_news_eval_2026-08-06.md
2026-09-07	Signal-side outlier filter fixed (it deleted about 36 real M5 news bars a year in the backtester and a different 24 live). Book impact at $0.65, 12 months: M5 +112.2R → +108.8R, M15 +73.9R → +74.6R, H1 unchanged. Oanda retry ladder added; a scan that loses its primary frame is failed	CLAUDE.md history, commit 2026-09-07
2026-09-06	Stake-sizing campaign FAILED its pre-registered rule on M5 (clean null) and M15 (lift inside the shuffle null). Nothing changing numbers shipped; --export-features and the penalty tags remain. M5 stretch measures are positive in both halves (do not propose exhaustion sizing on M5); bursts cluster 12–15 UTC and 01 UTC and do not hurt entries; every open-trade burst protection lost money	backtest/research/sizing_campaign_2026-09.md
2026-09-06	Violent-move watch shipped dark (display only). Hot = high-impact USD release within ±30 min, or a hot clock slot plus a price extreme; flags 8.6% of bars at 3–5× odds. Context readings never raise the level	violent_watch.py docstring, sizing campaign addenda
2026-09-06	Stale-code cleanup: research tooling, Dukascopy study, diagrams and plans deleted (history before 953c5ad has them); main.py M5 confirmations fixed to M15 + H1; dead functions removed	commits of 2026-09-06
2026-09-04	News sentiment verdict: the score trails price, it does not lead it. Headline window 4 h, neutral headlines dropped, beginner-language summary shipped; gating flags stay off	backtest/research/news_sentiment_value_2026-09-04.md
2026-09-03	Signal-log campaign shipped two gates: M5 shorts blocked in transitioning (+7.9R/12 mo) and every H1 short blocked (5-year +16.7R, OOS +11.5R). Rejected: M15 SELL-B block, Bollinger-off. Parked: divergence-off. Panel verdict switched to the macro persona (owner decision); the chair's OPPOSE was anti-informative (opposed trades made +30R)	backtest/research/signal_log_campaign_2026-09.md, signal_log_eval_2026-08-18.md
2026-09-03	Auto-execution (Oanda daemon and executor) removed; the owner does not auto-trade (last commit with it 31002cb)	commit 31002cb
2026-08-15	Agent-panel technical persona gets ATR-normalised candles and raw indicator values	backtest/research/ai_signal_enrichment_2026-08-15.md
2026-08-14	Books re-pinned on Oanda bars; Twelve Data books superseded. Feed divergence and Oanda-Tiingo signal parity measured	repin_verification_2026-08-14.md, feed_divergence_2026-08-14.md, signal_parity_2026-08-14.md
2026-08-06	DeepSeek headline scoring validated (78/78 vs 24/78 for keywords); event misreads fixed with the vs_forecast field	backtest/research/llm_news_eval_2026-08-06.md
2026-08-05	Fixed-RR hard target vs trailing exit: NOT shipped (passes the letter of rule 1 on 2024+ windows but is a trend artefact; M15 negative in 2021 and 2023, drawdown doubles). --fixed-rr-tp stays as a research flag	backtest/research/fixedtp_2026-08-05.md
2026-08-04	QA audit. Weekend calendar made New-York anchored (removed one live-impossible phantom trade). Spread inside exit-trigger detection: owner DECLINED (4–6% optimism accepted, books stay comparable). FB2_EMA_LO hardened. M5 pullback entries disabled (PF 1.26 → 1.44 at $0.65). Partial EMA alignment set neutral on M5. XAU_CP gate bypass documented as validated behaviour (226 of 586 M5 trades, +38.9R)	memory files qa-audit-2026-08-04, m5-tuning-2026-08-04, m5-round2-2026-08-04
2026-07-30	M5 relaxation shipped (score 25, confidence 5, hour 21 unblocked); big-candle gate rejected (ships off)	memory relaxation-campaign-2026-07-30
2026-07-28	Oanda v20 becomes the live feed	commit d2d4fb4
2026-07-24	Real gold spread measured at about $0.65 round trip (9M Dukascopy ticks, 12 months); $0.30 kept only for cross-book comparability and the live target floor	memory dukascopy-spread-study
2026-07-21	M15 trailing arm 0.70 and M5 trailing distance 0.85 adopted	memory m15-trail-grid-2026-07-21
2026-07-16	M15 SELL-hour block (15 and 17 UTC) adopted; EUR/USD removed (no edge)	memory xau-m15-loss-analysis, eur-no-edge-verdict
2026-07-11	M15: mixed trending regime blocked, target floor 1R; M5: hour 9 blocked	pair factory docstring
2026-07-08	Backtester realism fixes (spread charging, gap fills, next-open entry, stop-first ties)	memory backtester-realism-fixes

Older memory notes live in ~/.claude/projects/-home-harwinder-Development-ForexBot2/memory/. The previous, longer version of this file is in git history before 2026-09-15.