- data_provider: strip yfinance imports, _YF_FALLBACK_MAP, _YF_INTERVALS,
_YF_SESSION, _fetch_yfinance_fallback, _resample_h1_to_h4. Twelve Data
is the sole data source. Raises DataUnavailableError if a pair isn't
on the configured plan.
- config: remove XAU/USD and XAG/USD from SYMBOL_DISPLAY and PIP_VALUES.
PIP_VALUES is now empty (defaults: 0.01 JPY pairs, 0.0001 forex).
- signal_aggregator: clear _PER_SYMBOL_TFS overrides (was XAU/XAG-only).
- telegram_bot: replace XAUUSD with AUDUSD in SCAN_PAIRS. Auto-load .env
via python-dotenv so TELEGRAM_BOT_TOKEN/GEMINI_API_KEY are picked up.
- requirements: drop yfinance and curl_cffi dependencies.
End-to-end pipeline test passes: data freshness, news engine (Gemini live),
signal aggregator, and the /signal+/scan code paths in telegram_bot.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- New _PER_SYMBOL_TFS map in signal_aggregator routes gold/silver to
M15+H1+H4 confirmation; forex pairs untouched.
- H4 for XAU/XAG is resampled from yfinance H1 bars (yfinance has no
native 4h candle). New _resample_h1_to_h4 helper.
- Batch prefetch unions per-symbol TFs so future per-pair overrides
work without extra plumbing. XAU/XAG still excluded from Twelve Data
batch — H4 resample triggers a single yfinance H1 fetch.
- Telegram SCAN_TIMEFRAMES restored to [M15, H1] so /scan emits both
M15-primary and H1-primary cards. Shared cache means no extra credits.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Live server sits behind a TLS-inspecting proxy that re-signs HTTPS with
an untrusted root CA, breaking yfinance HTTPS calls. yfinance >= 0.2.40
requires a curl_cffi session (rejects requests.Session), so we build
one with verify=False and impersonate=chrome.
Trade-off: yfinance traffic is no longer authenticated against MITM.
Acceptable because the data is public market quotes and an attacker
can at worst inject bad prices. Session is yfinance-only and not reused
for any authenticated traffic.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- XAU/XAG always route to Yahoo Finance and skip Twelve Data entirely
(excluded from batch prefetch, served per-symbol via yfinance).
- Daily chart cleanly skipped for gold/silver — engine drops it from MTF.
- Telegram /scan limited to EURUSD, USDJPY, GBPUSD, XAUUSD.
- SCAN_TIMEFRAMES reduced to single H1 primary; engine still pulls
M15+H1+D1 internally via confirmation TFs (halves API credits per scan).
- Default confirmation TFs swapped H4 -> D1 in config and main.py.
- mtf_agreement_threshold kept at 2-of-3.
- CLAUDE.md cleaned up — old XAUUSD/synthetic-data bug reports removed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A single 5-minute refresh_interval applied to every timeframe wasted
API credits: a daily candle was being refetched every 5 minutes even
though it only changes once per day. Now each TF caches for roughly
its bar interval, so repeat scans within an hour skip D1 entirely
(saves ~60s of throttle wait) and within 30 min also skip H1.
config.py: new cache_ttl_per_tf dict on PlatformConfig with sensible
defaults for every timeframe; refresh_interval kept as fallback.
data_provider.py: _cache_ttl() helper used by both fetch_ohlcv and
prefetch_batch instead of the global refresh_interval.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- yfinance>=0.2.40 added as a hard dependency
- _fetch_yfinance_fallback(): pulls OHLCV from Yahoo for symbols not on
the Twelve Data free tier (XAUUSD → GC=F, XAGUSD → SI=F)
- Hooked into both batch and single-symbol fetch paths; logs as [yf]
to distinguish from Twelve Data [live]
- Fail-loud: if both providers return nothing, the pair is skipped with
the existing "data unavailable" message
- _analyze_all_tfs accepts an async progress callback; both /signal and
/scan now show coarse milestones via message edits:
"📡 Fetching market data..." → "🧠 Analyzing pairs..." → result
- prefetch_batch is now run in a thread so it doesn't block the event loop
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- telegram_bot: one ForexDataProvider shared across all SCAN_TIMEFRAMES;
combined prefetch of all unique TFs up front so the 4 concurrent scans
hit the warm cache instead of firing simultaneous API calls
- data_provider: skip prefetch_batch for timeframes already in cache
- data_provider: deduplicate "no data" warning per symbol (fires once, not once per TF)
- data_provider: [live] info log on successful fetch; outlier-removal downgraded to debug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
data_provider:
- Remove _generate_synthetic_data entirely — no more fake prices
- Raise DataUnavailableError instead of falling back to synthetic data
(no API key, daily limit hit, API returns no values, fetch failure)
- Move rate limiter to module-level globals (_rate_lock, _rate_request_times,
_rate_daily_credits) so all ForexDataProvider instances running in parallel
threads share the same 8-credit/min and 800-credit/day accounting
- _throttle now raises DataUnavailableError on daily limit (was silent return False)
- Remove _no_api_data tracking, numpy/hashlib imports
telegram_bot:
- Detect F-grade reports caused by data unavailability and show clean
"Data unavailable" line per TF instead of a garbled zero-price card
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- config.py: add XAU/USD and XAG/USD to SYMBOL_DISPLAY; add PIP_VALUES
map for commodities (gold pip = $0.01, not $0.0001)
- data_provider.py: use PIP_VALUES in get_pair_info so gold trade setup
shows correct pip counts instead of 269,000 pips for a $27 SL
- main.py: validate symbols in symbol_from_name — rejects non-alpha
inputs like '15M' with a helpful hint to use --timeframe instead
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
data_provider:
- Switch to Authorization header (recommended over apikey query param)
- Enforce 800 credits/day daily limit with warning at 80%
- Fall back to synthetic data when daily limit is reached
- Fix null OHLC field handling (API can return null for any field)
telegram_bot:
- Register commands with set_my_commands so Telegram shows / menu
- Add /credits command to show daily API credit usage and limits
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add telegram_bot.py: full bot interface with /scan, /signals, /pair,
/all, /tf, /watch, /unwatch, /status, /help commands; per-chat state;
background thread pool for analysis; auto message splitting
- Fix data_provider: warn once per symbol with no API data and cache
synthetic fallback to prevent redundant individual API calls
- Fix signal_aggregator: derive spread estimate from already-fetched
primary DataFrame instead of making a separate M5 API call per pair
- Add python-telegram-bot[job-queue]>=20.0 to requirements.txt
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Harwinder
harry