One class per provider (TwelveData, Oanda, Tiingo), all subclassing
BaseDataProvider which owns cache, validation/cleaning, freshness +
boundary guarantees, fetch_range pagination, and streaming stubs —
subclasses implement only _fetch_raw. config.json data_providers.active
selects the live feed; data_provider.py / oanda_provider.py stay as
back-compat shims. Module 16 provider-abstraction tests (253 total green).
Claude-Session: https://claude.ai/code/session_01Mc7XKYyPySVXiUpcN82QEK
format_signal_message restyled (display-only, zero signal-logic impact):
verdict header first (signal - pair - TF - quality), trade setup moved
above the detail, per-indicator breakdown condensed to one arrow line
(_indicator_summary), heavy dividers dropped. New TP 100p line in every
setup block: entry +/- 100 pips ($10.00 on XAU) with its R-multiple vs
the stop (_INFO_TP_PIPS, formatter-only - not stamped on TradeSetup,
not in CLI/JSON/backtester). Trailing-plan logic unchanged (stamped
per-TF params, pullback fallback). Removed dead _indicator_line and
_SIGNAL_TAG. Tests 210 -> 212 (100p TP math BUY/SELL/no-setup, one-line
indicator summary).
Claude-Session: https://claude.ai/code/session_01BoZKi67Ct6QTeiFJmybXSw
Tiingo's free /tiingo/fx/top publishes only a two-tier indicative
spread for gold ($0.10 liquid hours / $0.70 rollover+news; verified on
273 production samples + live probes) — mid ± a posted increment, not
executable depth, so it can never validate the $0.30 backtest cost
assumption. Removed one day after shipping: tiingo_provider.py, the
sampler/quote plumbing in telegram_bot.py, the channel_spread_detail
config knob and tiingo_api_key resolver, and Module 14's Tiingo tests
(221 -> 210). CLAUDE.md documents the verdict: do not re-add a spread
display from Tiingo's free FX feed.
Claude-Session: https://claude.ai/code/session_01BoZKi67Ct6QTeiFJmybXSw
Five-agent audit of the signal path; all fixes are delivery/measurement
side. Regression-proven behavior-neutral on validated signals: the 5yr
M15 replay reproduces the pinned book exactly (+210.65R PF 1.35
WR 57.8% MaxDD 33.90R). Tests 215 -> 221.
- TradeSetup carries trail_arm_r/trail_dist_r stamped from the producing
config; Telegram trail plan reads them (was showing default 0.6/0.75
instead of the validated M15 arm 0.70 / M5 dist 0.85)
- Alert dedup: fingerprint = (signal, quality, direction); entry movement
compared against last-ALERTED entry/risk (old SL-dist bucket churned on
ATR drift -> near-duplicate alerts every candle)
- Fresh fetches enforce the closed-bar boundary check with one 3s grace
retry, then warn (feed lag no longer silently analyzes the prior bar)
- Backtest max DD includes the zero baseline (opening losing streaks
were undercounted; pinned M15 DD unchanged)
- Replay loads the regime-filter TF when the H4-ADX gate is on (was a
silent no-op for M5/M15 primaries)
- clear_cache under _cache_lock; dead ROC zero-cross branches removed
from both engines (provably unreachable); RSS dates via
parsedate_to_datetime (named zones no longer stamped UTC); event risk
grades from the soonest event; .env inline comments stripped;
auto-scan skips NEUTRAL like the breach path; report_to_dict honors
the weight-0 skip rule and emits strategy/trail params; PB funnel
counts spread_gate/accepted; session-filter fail-open paths log loudly
- Docs corrected to match validated code (H&S neckline = lower peak,
exhaustion anchors farthest tracked swing) with re-validation warnings
Claude-Session: https://claude.ai/code/session_01R5soRUeRBi8kHMB99csrJy
- Delete test_ws_dual.py (standalone probe, unused) and a stale tracked log
- config.py: drop MarketSession enum, is_jpy_pair, candle_buffer_seconds,
and the never-read legacy SignalConfig/XauStrategyConfig fields
- data_provider.py: drop uncalled get_latest_price/get_spread_estimate
- Remove unused imports (engine, telegram_bot, main, news_engine)
- test_all.py: drop/trim the corresponding defaults assertions (199 tests)
- CLAUDE.md updated to match
Zero behavior change: no production code read any removed field
(verified reference-by-reference); suite green at 199/199.
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
Documents the app_config.py / config.json overlay shipped in Tasks 1-4:
repo-map rows for app_config.py and config.json.example, env-var table
precedence (env -> config.json -> none, hardcoded fallbacks removed
2026-07-21), PlatformConfig/pair_signals overlay call order, and the
real 199-tests/13-modules suite count. Also fixes a stale SCAN_PAIRS
comment in telegram_bot.py that still named PlatformConfig.scan_pairs
as the deployment switch instead of config.json's scanning.pairs.
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
- Credentials, alert quality, scan pairs/timeframes, log level and
candle-loop intervals now resolve via app_config (env -> config.json
-> code default) instead of hardcoded fallbacks.
- _get_shared_dp() uses make_data_provider() for provider-registry
dispatch.
- main() refuses to start without a token/chat_id, raising SystemExit
with an actionable message instead of a bare sys.exit string.
- test_all.py Module 13: two new tests covering credential/quality/
loop wiring via a reload, and the startup guard.
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
Resolves the Twelve Data API key through app_config.provider_api_key()
(env → config.json → no default) instead of a hardcoded field on
PlatformConfig. Adds data_provider.PROVIDER_REGISTRY + make_data_provider()
so signal_aggregator and the backtester build the active provider through
one seam; telegram_bot.py gets the same one-line WS-key fix now (per Task 4's
already-planned edit for that line) so the suite doesn't crash on import
between this commit and Task 4's full wiring pass.
Claude-Session: https://claude.ai/code/session_01VPNF37MP9Kuv62B1VBrZ1b
Fixes all high+medium findings from the 2026-07-16 full-codebase audit.
High:
- data_provider: cache now rejects snapshots missing the latest closed
bar (_has_latest_closed_bar, weekend-exempt). The wall-clock TTL served
mid-candle snapshots at the next boundary, so live M15/H1 scans ran one
full bar behind and the bar-close session check evaluated the wrong hour.
- signal_aggregator: gate 6e (SELL quality) only acts on actionable
grades. It was re-grading session-blocked F reports to D, which
re-opened the M5 pullback path (entry check: quality != "F") on
session-blocked bars.
- gpu_backtest: port the live EMA-ribbon compression/expansion logic
(+10 conf on expansion) into the GPU engine. Its absence diverged
composite score/conf on 7-9% of bars and flipped signals on ~0.2%.
Medium:
- gpu_backtest: Bollinger precompute uses the exact pandas rolling calls
(cumsum E[x2]-E[x]2 lost precision at gold price levels; bb_position
error up to 7.6e-3 crossed signal thresholds, worst during squeezes).
- gpu_backtest: --confirm-tfs defaults to the pair factory's live
confirmation map instead of a hardcoded 1h,4h.
- gpu_backtest: entries re-anchor to the first RAW bar after the signal
bar, so >10-sigma spike bars dropped by cleaning are no longer skipped
at entry (favorable bias).
- gpu_backtest: per-TF warmup pad — D1 confirms get >=210 bars at window
start (62-day pad gave ~61 bars: no EMA200, unlike live).
- data_provider: thread-safety locks for cache + throttle (shared
provider across the bot's 4-thread pool); outlier filter keeps the
snap-back bar after a spike instead of punching a two-bar hole.
- telegram_bot: alert fingerprint gains a risk-relative entry bucket
(materially-moved setups re-alert) and is committed only after a
successful send (failed sends no longer consume the alert for 1h).
- engine: disabled_indicators=["macd"] now removes the MACD pillar from
scoring (histogram still computed for divergence).
Verification: test_all.py 187/187 (new Module 11: 8 audit-fix
regressions incl. a live-vs-GPU parity sweep with ribbon coverage);
real-data parity sweep over 813 slices of 14mo XAU M15: zero divergence
(pre-fix: 7-9% of bars diverged); both original bug repros no longer
reproduce; cached end-to-end backtest run passes with the live confirm
map. Books shift slightly vs pre-fix runs — comparability note added to
CLAUDE.md; pinned tuned numbers need re-validation under the fixed
backtester.
- TELEGRAM_CHANNEL_ID mirrors auto-scan signal + breach alerts to a
subscriber channel (Stars-subscription monetization; Telegram manages
membership). Feed keeps running even when the owner's DM auto-scan
is toggled off.
- _owner_only decorator locks all commands/buttons to TELEGRAM_CHAT_ID;
others get a refusal pointing at the channel.
- Every signal-bearing message (detailed signal, scan summary, all-TF
summary, breach alerts) now ends with a not-financial-advice /
educational-use disclaimer via a single _DISCLAIMER constant.
- telegram_bot: SCAN_PAIRS/_WS_TO_PAIR/SCAN_BUTTONS all derive from
PlatformConfig.scan_pairs (the single pair on/off switch); buttons and
keyboards are generated per pair; _run_signal_sync builds per-pair
configs via pair_signals factories (was: XAU defaults for any symbol);
scan-all works per pair; PB badge matches any *_PB tag; daily-rotating
file logs in logs/telegram.log (30-day retention)
- main.py: --pair argument (XAUUSD | EURUSD + aliases) routed through
pair_signals factories; price display is precision-aware (2dp gold,
5dp FX)
scan_pairs default stays [XAUUSD] until the EUR M5 re-tune validates.
Second entry pattern alongside the Keltner-breakout: enter on the
retrace to EMA21/KC-mid in an established trend (EMA >=3/4 aligned,
ADX>=20, retrace 0.8-6.0 ATR, resumption bar closed back in trend
direction, RSI in the pullback zone not the extended zone). Detector
lives in the aggregator so live and GPU-backtest paths share one
implementation. Own gates (composite floor 15, confirmation-TF
agreement, MTF opposition, vol regimes) with a diagnostic funnel.
PB stops anchor to the pullback swing (~$8 vs ~$25 breakout), so PB
trades get their own trail arm (0.8R vs 0.6R global) via per-trade
trail overrides in the backtester and strategy-aware Telegram alerts.
12mo M5 validation: PB alone +38.3R/254 trades, positive both halves
and all four full quarters; core breakout trades undisturbed
(+101.9R vs +105.0 baseline). Combined book: 549 trades (~10.6/wk),
+140.1R, PF 1.78, DD 11.3R vs 311 trades/+105.0R before.
Gates 5-6 in the M5 filter: block 'high' vol regime entries (chop
transition — 61% WR, negative total while low/normal/extreme all
positive) and block SELLs with RSI<25 (dip already extended, mean-
reverts before target). 365d: 419→311 trades, +41→+48.4R, DD 11.2→8.0R.
Replace the fixed RR-0.5 take-profit with a trailing stop: arm at
+0.6R favorable excursion, trail 0.75R behind best price, stop levels
update at bar close (conservative). The RR-0.5 target was the binding
constraint — winners kept running after TP. Whole trail parameter grid
(arm 0.4-1.2 × dist 0.5-1.25) lands +90-103R, a plateau not a spike;
also +91.7R on the pre-gate 419-trade population. TP1 stays advisory;
Telegram alerts include the arm price and trail distance. max_hold
200→300 bars (trailing exits need room; +10R).
12mo M5, same 311 trades: +105.0R total, PF 2.27, DD 7.9R, 61% WR,
train/val +54.6/+50.4, every quarter positive.
WebSocket:
- Send heartbeat every 10s — Twelve Data drops idle connections, so the
monitor previously died on weekends/quiet hours and churned reconnects
- Log failed symbol subscriptions from subscribe-status events
- Stamp fresh live tick price + age into every alert message
Alerts:
- Re-enable level-breach alerts: WS breach triggers M15 rescan and sends
the report when actionable (was scanning and discarding the result)
- Dedup: identical (signal, quality, direction) per pair/TF is sent once,
re-armed after 1h — no more repeat alerts every candle
- Configurable minimum alert quality (C+/B+/A), cycled from the menu
- Trade setup shows dollar risk/reward alongside pips
Interactive menu (/start, /scan):
- Timeframe scans, Scan All, auto-scan ON/OFF toggle, alert-quality
cycle, status, and help as inline buttons
- Markdown-safe edits: 'message not modified' swallowed, parse failures
fall back to plain text; exception text escaped
M5 already broadcast through the shared _scan_timeframe path but was
documented as data-only (docstring, CLAUDE.md, DOCUMENTATION.md, plus a
phantom `if tf == M5: return` step that never existed in code). Align all
docs/comments so M5 alerting at C+ grade is the intended, documented behavior.
Remove the live dashboard entirely per request: the 30s refresh loop,
per-scan refresh, boot send, and the dashboard-only stats layer
(_signal_stats / _stats_updated / _update_signal_stats and its call sites,
_format_stats_dashboard and its helper constants). Drop now-unused ZoneInfo
import and dashboard mentions from help/autoscan/boot messages and the test
suite. The WebSocket level-breach monitor and level cache are untouched.
Tests: 136 passed, 0 failed.
- Blocked hours reduced to 0-2 UTC only (daily rollover)
- M5 signals now broadcast to Telegram like M15/H1/H4
- Live dashboard shows New York time (auto EDT/EST via zoneinfo)
News engine: expand sentiment keywords (18%→60% headline coverage), add
MyFXBook RSS as fallback calendar source when Forex Factory returns 429,
add retry/backoff on calendar API rate limits, update stale ForexLive URL
to investinglive.com, add date formats without seconds for MyFXBook RSS.
Telegram bot: remove flood control, rate limiter, and dashboard failure
tracking — all sends go directly through bot.send_message/edit_message_text.
- Move index dedup before OHLC consistency fix in _validate_and_clean
to prevent crash when df.loc[dup_idx] returns Series instead of scalar
- Add global flood backoff: detect RetryAfter, pause all sends/edits
until ban expires instead of retrying and escalating the ban
- Stop dashboard edit-fail → new-message spam cascade: track consecutive
failures, only resend after 3 failed edits
- Increase dashboard refresh from 5s to 30s, min send interval to 1.5s
- Add comprehensive QA test suite (140 tests across all 7 modules)
Timeframe keys stored as enum values (15min/1h/4h) but dashboard
looked for M15/1h/4h — GBP/USD showed no price or signal text.
Now uses Timeframe enum directly with display labels. Reduced
dashboard refresh from 60s to 5s with throttle guard.
Replace per-scan Telegram messages with a single dashboard message
that edits in-place every 60 seconds. Each pair shows all TFs
horizontally with short signal labels (Buy/Sell/Neutral). WebSocket
live prices feed into the dashboard. Level-breach alerts disabled
(WS still updates cache silently). Removed /stats command.
- Drop USD/JPY from scan pairs (zero signals across all timeframes)
- Disable divergence indicator (zero impact in 14-config lab test)
- Parallelize evaluate.py with ProcessPoolExecutor (16 cores)
- Add --pair CLI filter to evaluate.py for single-pair backtests
- Add data freshness guard: stale cache auto-invalidated, primary TF
stale data suppresses signals to prevent trading on old data
- Add /cache command: shows fetch age and last bar per pair/TF
- Add /clearcache command: wipes in-memory cache for fresh API fetch
- Remove BB from WebSocket breach alerts (keep structural levels only)
- Show live prices only for WS-monitored pairs (EUR/USD, XAU/USD)
- Replace arrow emojis with text tags in indicator lines
- Trade setup TP now scales to SL distance ensuring min RR is met when
no viable swing target exists; skip swing targets with poor RR
- Add float tolerance to RR gate to prevent edge-case rejection
- M5 auto-scan runs for level cache only, no Telegram broadcast
- WebSocket limited to EUR/USD + XAU/USD (Grow plan constraint)
- Register bot command menu via set_my_commands on startup
- Fix H4 boundary calculation using total minutes (hour*60+min) not just minute
- Add /help command listing all bot commands
- Add /status command showing WS connection, live prices, level cache, cooldowns
- Run initial M15 scan on boot to populate level cache before WS starts
- Update /start message to mention new commands
XAU/USD is not available in TwelveData's symbol list.
Swapped to USD/CAD across config, scan pairs, Telegram buttons,
and added CAD to tracked news currencies.
Expand from 2 to 4 pairs (EUR/USD, USD/JPY, GBP/USD, XAU/USD) with
automatic scanning at M15 and H1 candle close boundaries. Scheduler
staggers H1 by 60s to stay within 8 credits/min; daily budget is
480 of 800 allowed. Cache TTL is now candle-duration-aware so cross-TF
confirmation data is reused without extra API calls. Spread estimate
uses primary-TF data instead of fetching M5 separately.
Add EURUSD pair with dynamic pair info lookup. Telegram bot now shows
4 scan buttons: EUR/USD 15M, EUR/USD 1H, USD/JPY 15M, USD/JPY 1H.
Each indicator displays directional arrows in the message output.
Price formatting adapts per pair (5dp for EUR, 3dp for JPY).
Introduces telegram_bot.py: python-telegram-bot v20+ async bot that
presents a persistent Scan Signal inline button. Each press fetches a
live USD/JPY signal via SignalAggregator and formats it with emoji,
MTF breakdown, top bull/bear reasons, and warnings. STRONG_BUY /
STRONG_SELL signals additionally render the full trade setup (entry,
SL, TP1, TP2, RR). Adds python-telegram-bot>=20.0 to requirements.txt.
Harwinder Sekhon