harry/ForexBot2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2a2ccb13dd08f01e2b79ab56213d45c459d51d23
ForexBot2/llm_provider_switch_plan.txt
T
Harwinder Sekhon b085b68d80 docs: multi-provider LLM layer plan (DeepSeek news / Anthropic panel) 
Reference plan only - no code changes. Describes the BaseLLMClient
refactor, AnthropicClient on the official SDK, per-consumer routing via
a config.json "llm" block, tests, commit sequence, and effort estimate
for the day we route the agent panel to Claude while news stays on
DeepSeek.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SisdfkxnXNL9iZsXxN3T15
2026-08-07 23:02:08 -04:00

253 lines
14 KiB
Plaintext
Raw Blame History

================================================================================
MULTI-PROVIDER LLM LAYER - PLAN
(DeepSeek for news summary, Anthropic Claude for agent panel)
Written 2026-08-08. Plan only - no code has been changed.
================================================================================
WHY
---
Today every LLM feature (news enrichment + agent panel) goes through one
hardwired client: llm_client.py -> DeepSeekClient, a module singleton reached
via get_llm_client(). This plan describes how to route different features to
different providers - e.g. keep DeepSeek for cheap high-volume news scoring,
put the agent panel on Anthropic Claude for better reasoning.
The repo already solved this problem once for market data: the providers/
package (BaseDataProvider owns shared machinery, subclasses implement only
_fetch_raw, a registry + one config knob selects the provider). This plan
mirrors that proven pattern at the LLM layer, scaled down to a single module.
BLAST RADIUS (verified by grep)
-------------------------------
get_llm_client() call sites:
- news_engine.py ....... 4 sites
- agent_panel.py ....... 3 sites (+1 timeout getattr)
- telegram_bot.py ...... 1 site (/status health; reads private _count,
_count_day, _daily_cap fields)
- test_all.py .......... 41 locations stub via lc._client = _StubLLM(...)
Out of scope: static/news_summary.php (own PHP DeepSeek call + key file,
separate deployment).
DESIGN INVARIANTS (carried over unchanged)
------------------------------------------
1. Fail-safe contract: every call returns Optional[Dict]; any failure -> None;
callers keep their non-LLM fallbacks; LLM output never gates signals.
2. Single attempt, no retries (news cache cadence is the retry); per-provider
UTC daily cap; warn-once logging; health stamps (last_ok, last_error).
3. Static system prompts stay byte-identical constants - provider-neutral,
and they map to prompt caching on BOTH providers (DeepSeek prefix cache;
Anthropic cache_control ephemeral).
4. Test compatibility: all 41 existing lc._client stub sites keep working
with zero edits.
================================================================================
1. llm_client.py REFACTOR (~195 -> ~430 lines, no new files)
================================================================================
1a. BaseLLMClient - shared machinery (extracted from today's DeepSeekClient)
- __init__(api_key, model, timeout, daily_cap, enabled)
- public: name ("deepseek"/"anthropic"), timeout property,
usage() -> (used_today, cap) # so telegram_bot stops reading privates
- owns: available property, _warn_once, _under_cap() (UTC-day counter,
lock), health stamps, and the template method:
def chat_json(self, system_prompt, user_payload):
if not self.available or not self._under_cap():
return None
try:
return self._request(system_prompt, user_payload) # hook
except <provider-agnostic error mapping>:
# stamps last_error, returns None
- subclasses implement ONLY _request(system_prompt, payload) -> dict|None.
1b. DeepSeekClient(BaseLLMClient) - behavior-identical move
Current requests-based logic relocates into _request: same URL,
thinking:disabled, response_format:json_object, temperature 0,
max_tokens 2000, sort_keys=True payload, _parse_response.
Review criterion: Module 19 tests pass UNMODIFIED (they patch
client._session, which stays).
1c. AnthropicClient(BaseLLMClient) - new, official SDK
- Official `anthropic` Python SDK (add to requirements.txt), NOT raw
requests. LAZY IMPORT inside __init__: package missing -> warn-once +
client unavailable, so DeepSeek-only deployments never install it.
- Constructor: anthropic.Anthropic(api_key=..., timeout=<configured>,
max_retries=0) # house rule: single attempt
- _request:
resp = self._client.messages.create(
model=self._model, # default "claude-opus-5"
max_tokens=4000, # covers thinking + JSON reply
output_config={"effort": "low"}, # cheap structured takes
system=[{"type": "text", "text": system_prompt,
"cache_control": {"type": "ephemeral"}}],
messages=[{"role": "user",
"content": json.dumps(user_payload, sort_keys=True)}],
)
if resp.stop_reason == "refusal": # Opus 5: HTTP 200, check first
return None
text = first text block in resp.content
return self._parse_json(text) # tolerant: strip ```json fences,
# json.loads, dict check
- DELIBERATELY ABSENT (would 400 on Claude Opus 5): temperature (removed
on Opus 5 - do NOT copy from the DeepSeek client), budget_tokens,
prefills, response_format.
- Error mapping: APIConnectionError -> "network",
APIStatusError -> "http_<code>", parse fail -> "parse"
(same last_error kinds the /status display already understands).
- Defaults: model claude-opus-5 (config-overridable, e.g. claude-haiku-4-5
for ~5x cheaper), timeout 30s (Claude slower than V4-Flash), own
daily_request_cap 500.
1d. Registry + per-consumer routing
_PROVIDERS = {"deepseek": DeepSeekClient, "anthropic": AnthropicClient}
LLM_CONSUMERS = ("news", "agent_panel")
_clients = {} # keyed by PROVIDER name -> two consumers on the
# same provider share one instance and one cap
_client = None # kept: GLOBAL TEST OVERRIDE
def get_llm_client(consumer="news"):
if _client is not None: # tests: lc._client = stub
return _client # -> shadows ALL consumers
name = get_app_config().llm_routing.get(consumer, "deepseek")
create-once per provider, return _clients[name]
def reset_llm_client(): # clears _client AND _clients
Keeping _client as a documented test-override global is what makes the
41 existing stub sites work untouched.
================================================================================
2. CONFIG (app_config.py + config.json.example)
================================================================================
- New top-level "anthropic" block, mirroring "deepseek":
{api_key, model, enabled, timeout_seconds, daily_request_cap}
plus anthropic_api_key() resolver: ANTHROPIC_API_KEY env -> config.json -> "".
- New top-level "llm" ROUTING block:
{"news": "deepseek", "agent_panel": "deepseek"}
Validation: keys within {news, agent_panel}; values within registered
provider names. Absent block -> {} -> everything on DeepSeek (today's
behavior; zero-migration default). AppConfig.llm_routing: Dict[str, str].
- The target state is then PURE CONFIG - no code changes on switch day:
"llm": { "news": "deepseek", "agent_panel": "anthropic" },
"anthropic": { "api_key": "", "model": "claude-opus-5", "enabled": true,
"timeout_seconds": 30, "daily_request_cap": 500 }
================================================================================
3. CONSUMER EDITS (mechanical, ~10 lines total)
================================================================================
- news_engine.py: 4 call sites -> get_llm_client("news")
- agent_panel.py: 3 call sites -> get_llm_client("agent_panel");
getattr(client, "_timeout", ...) -> public client.timeout
- telegram_bot.py _news_health_lines: iterate the DISTINCT providers actually
routed (news + panel); per provider print "<name>: used/cap calls today"
via the new usage() accessor instead of private fields. DeepSeek keeps its
"headlines scored" extra; cap/error warnings work off base-class stamps.
- NO changes to prompts, payload builders, caches, or the panel's own
max_panels_per_day budget - all provider-agnostic already.
================================================================================
4. TESTS (new "Module 24: LLM provider routing", ~12 tests, ~200 lines)
================================================================================
Reuse _StubLLM; new _FakeAnthropic (fake messages.create returning a stub
response object) injected as client._client.
1. app_config: anthropic block parses/validates/defaults empty.
2. app_config: llm routing block - valid map round-trips; unknown consumer
key, unknown provider name, non-dict -> AppConfigError; absent -> {}.
3. anthropic_api_key resolves env -> config.json -> empty.
4. Base contract on BOTH subclasses (parametrized): keyless/disabled -> no
call; daily cap blocks + UTC-roll reset; failure stamps last_error,
success stamps last_ok.
5. AnthropicClient parse: clean JSON -> dict; ```json-fenced -> dict;
non-JSON -> None; stop_reason "refusal" -> None; no text block -> None.
6. AnthropicClient request shape: NO temperature key, max_retries=0,
cache_control present on system block (assert on captured kwargs).
7. AnthropicClient with `anthropic` package missing (simulated) ->
available False, chat_json None, no crash at import.
8. Routing: no llm block -> both consumers get the SAME DeepSeek singleton
(shared cap); routed config -> news gets DeepSeekClient while
agent_panel gets AnthropicClient; same-provider consumers share one
instance.
9. Global lc._client override shadows BOTH consumers (locks the 41 legacy
stub sites' semantics).
10. reset_llm_client() clears both the override and the per-provider cache.
11. End-to-end: routing agent_panel->anthropic, attach_panel reaches the
anthropic fake (5 calls) while a news call reaches the deepseek stub.
12. telegram _news_health_lines renders per-provider lines from usage()
without touching private fields.
GATE: Modules 19 and 23 must pass UNMODIFIED - the no-regression proof for
the refactor.
================================================================================
5. DOCS
================================================================================
- CLAUDE.md: llm_client bullet -> provider-abstracted (BaseLLMClient;
deepseek + anthropic registered); config.json "llm" block routes consumers;
absent = all-DeepSeek. Add ANTHROPIC_API_KEY to env-var list. Test counts.
- config.json.example: both new blocks.
================================================================================
COMMIT SEQUENCE (each commit green)
================================================================================
1. llm_client: extract BaseLLMClient; DeepSeekClient unchanged behavior
(+ base-contract tests) - pure refactor, Modules 19/23 green untouched.
2. config: anthropic block + llm consumer-routing block (+ config tests).
3. llm_client: AnthropicClient (official SDK, lazy import) + requirements.txt
(+ parse/shape tests).
4. routing: per-consumer get_llm_client + health lines - consumer call-site
edits, telegram usage() lines, routing/e2e tests.
5. docs: CLAUDE.md + example config.
================================================================================
VERIFICATION
================================================================================
- python test_all.py green at every commit (~374 -> ~386).
- Live smoke A (no behavior change): default config, main.py --panel ->
still DeepSeek, /status unchanged.
- Live smoke B (the goal): llm.agent_panel="anthropic" + real
ANTHROPIC_API_KEY, run --panel -> one real Claude panel (5 requests,
validates JSON contract + latency + refusal-free path) while a news
refresh still hits DeepSeek; /status shows two provider lines.
- Backtest untouched by construction (no LLM in replay path); one 30-day
run as a formality.
================================================================================
EFFORT ESTIMATE
================================================================================
0 new files, 8 modified, ~520 lines, est. 3-5 hours.
llm_client.py base class + AnthropicClient + routing +235
test_all.py Module 24 +200
app_config.py anthropic + llm blocks, resolver +55
telegram_bot.py per-provider health lines +15
news_engine.py consumer tags on call sites +4
agent_panel.py consumer tags on call sites +4
config.json.example blocks +14
requirements.txt anthropic +1
CLAUDE.md docs +10
================================================================================
KEY DECISIONS EMBEDDED IN THIS PLAN
================================================================================
- Official Anthropic SDK, not raw requests (current API guidance for Python;
lazy import keeps it optional for DeepSeek-only installs).
- Default Claude model: claude-opus-5 ($5/$25 per MTok; a full 5-call panel
~ 8k in / 500 out ~ $0.05). Config-overridable to claude-haiku-4-5.
- Two Claude gotchas handled: temperature is REMOVED on Opus 5 (sending it
400s - must not be copied from the DeepSeek request body), and refusals
arrive as HTTP 200 with stop_reason "refusal" (checked before parsing,
mapped to the normal None/fallback path).
- Caps are per-provider, not per-consumer: two consumers on one provider
share its cap (today's semantics); splitting across two providers doubles
total headroom naturally.
- The panel's own max_panels_per_day budget is untouched - it sits above
whichever provider serves it.
- Not in scope: static/news_summary.php (own PHP client/key), streaming,
tool use, per-consumer model overrides (add a "model" key per consumer
later if ever needed).
================================================================================
Reference in New Issue View Git Blame Copy Permalink