================================================================================ MULTI-PROVIDER LLM LAYER - PLAN (DeepSeek for news summary, Anthropic Claude for agent panel) Written 2026-08-08. Plan only - no code has been changed. ================================================================================ WHY --- Today every LLM feature (news enrichment + agent panel) goes through one hardwired client: llm_client.py -> DeepSeekClient, a module singleton reached via get_llm_client(). This plan describes how to route different features to different providers - e.g. keep DeepSeek for cheap high-volume news scoring, put the agent panel on Anthropic Claude for better reasoning. The repo already solved this problem once for market data: the providers/ package (BaseDataProvider owns shared machinery, subclasses implement only _fetch_raw, a registry + one config knob selects the provider). This plan mirrors that proven pattern at the LLM layer, scaled down to a single module. BLAST RADIUS (verified by grep) ------------------------------- get_llm_client() call sites: - news_engine.py ....... 4 sites - agent_panel.py ....... 3 sites (+1 timeout getattr) - telegram_bot.py ...... 1 site (/status health; reads private _count, _count_day, _daily_cap fields) - test_all.py .......... 41 locations stub via lc._client = _StubLLM(...) Out of scope: static/news_summary.php (own PHP DeepSeek call + key file, separate deployment). DESIGN INVARIANTS (carried over unchanged) ------------------------------------------ 1. Fail-safe contract: every call returns Optional[Dict]; any failure -> None; callers keep their non-LLM fallbacks; LLM output never gates signals. 2. Single attempt, no retries (news cache cadence is the retry); per-provider UTC daily cap; warn-once logging; health stamps (last_ok, last_error). 3. Static system prompts stay byte-identical constants - provider-neutral, and they map to prompt caching on BOTH providers (DeepSeek prefix cache; Anthropic cache_control ephemeral). 4. Test compatibility: all 41 existing lc._client stub sites keep working with zero edits. ================================================================================ 1. llm_client.py REFACTOR (~195 -> ~430 lines, no new files) ================================================================================ 1a. BaseLLMClient - shared machinery (extracted from today's DeepSeekClient) - __init__(api_key, model, timeout, daily_cap, enabled) - public: name ("deepseek"/"anthropic"), timeout property, usage() -> (used_today, cap) # so telegram_bot stops reading privates - owns: available property, _warn_once, _under_cap() (UTC-day counter, lock), health stamps, and the template method: def chat_json(self, system_prompt, user_payload): if not self.available or not self._under_cap(): return None try: return self._request(system_prompt, user_payload) # hook except : # stamps last_error, returns None - subclasses implement ONLY _request(system_prompt, payload) -> dict|None. 1b. DeepSeekClient(BaseLLMClient) - behavior-identical move Current requests-based logic relocates into _request: same URL, thinking:disabled, response_format:json_object, temperature 0, max_tokens 2000, sort_keys=True payload, _parse_response. Review criterion: Module 19 tests pass UNMODIFIED (they patch client._session, which stays). 1c. AnthropicClient(BaseLLMClient) - new, official SDK - Official `anthropic` Python SDK (add to requirements.txt), NOT raw requests. LAZY IMPORT inside __init__: package missing -> warn-once + client unavailable, so DeepSeek-only deployments never install it. - Constructor: anthropic.Anthropic(api_key=..., timeout=, max_retries=0) # house rule: single attempt - _request: resp = self._client.messages.create( model=self._model, # default "claude-opus-5" max_tokens=4000, # covers thinking + JSON reply output_config={"effort": "low"}, # cheap structured takes system=[{"type": "text", "text": system_prompt, "cache_control": {"type": "ephemeral"}}], messages=[{"role": "user", "content": json.dumps(user_payload, sort_keys=True)}], ) if resp.stop_reason == "refusal": # Opus 5: HTTP 200, check first return None text = first text block in resp.content return self._parse_json(text) # tolerant: strip ```json fences, # json.loads, dict check - DELIBERATELY ABSENT (would 400 on Claude Opus 5): temperature (removed on Opus 5 - do NOT copy from the DeepSeek client), budget_tokens, prefills, response_format. - Error mapping: APIConnectionError -> "network", APIStatusError -> "http_", parse fail -> "parse" (same last_error kinds the /status display already understands). - Defaults: model claude-opus-5 (config-overridable, e.g. claude-haiku-4-5 for ~5x cheaper), timeout 30s (Claude slower than V4-Flash), own daily_request_cap 500. 1d. Registry + per-consumer routing _PROVIDERS = {"deepseek": DeepSeekClient, "anthropic": AnthropicClient} LLM_CONSUMERS = ("news", "agent_panel") _clients = {} # keyed by PROVIDER name -> two consumers on the # same provider share one instance and one cap _client = None # kept: GLOBAL TEST OVERRIDE def get_llm_client(consumer="news"): if _client is not None: # tests: lc._client = stub return _client # -> shadows ALL consumers name = get_app_config().llm_routing.get(consumer, "deepseek") create-once per provider, return _clients[name] def reset_llm_client(): # clears _client AND _clients Keeping _client as a documented test-override global is what makes the 41 existing stub sites work untouched. ================================================================================ 2. CONFIG (app_config.py + config.json.example) ================================================================================ - New top-level "anthropic" block, mirroring "deepseek": {api_key, model, enabled, timeout_seconds, daily_request_cap} plus anthropic_api_key() resolver: ANTHROPIC_API_KEY env -> config.json -> "". - New top-level "llm" ROUTING block: {"news": "deepseek", "agent_panel": "deepseek"} Validation: keys within {news, agent_panel}; values within registered provider names. Absent block -> {} -> everything on DeepSeek (today's behavior; zero-migration default). AppConfig.llm_routing: Dict[str, str]. - The target state is then PURE CONFIG - no code changes on switch day: "llm": { "news": "deepseek", "agent_panel": "anthropic" }, "anthropic": { "api_key": "", "model": "claude-opus-5", "enabled": true, "timeout_seconds": 30, "daily_request_cap": 500 } ================================================================================ 3. CONSUMER EDITS (mechanical, ~10 lines total) ================================================================================ - news_engine.py: 4 call sites -> get_llm_client("news") - agent_panel.py: 3 call sites -> get_llm_client("agent_panel"); getattr(client, "_timeout", ...) -> public client.timeout - telegram_bot.py _news_health_lines: iterate the DISTINCT providers actually routed (news + panel); per provider print ": used/cap calls today" via the new usage() accessor instead of private fields. DeepSeek keeps its "headlines scored" extra; cap/error warnings work off base-class stamps. - NO changes to prompts, payload builders, caches, or the panel's own max_panels_per_day budget - all provider-agnostic already. ================================================================================ 4. TESTS (new "Module 24: LLM provider routing", ~12 tests, ~200 lines) ================================================================================ Reuse _StubLLM; new _FakeAnthropic (fake messages.create returning a stub response object) injected as client._client. 1. app_config: anthropic block parses/validates/defaults empty. 2. app_config: llm routing block - valid map round-trips; unknown consumer key, unknown provider name, non-dict -> AppConfigError; absent -> {}. 3. anthropic_api_key resolves env -> config.json -> empty. 4. Base contract on BOTH subclasses (parametrized): keyless/disabled -> no call; daily cap blocks + UTC-roll reset; failure stamps last_error, success stamps last_ok. 5. AnthropicClient parse: clean JSON -> dict; ```json-fenced -> dict; non-JSON -> None; stop_reason "refusal" -> None; no text block -> None. 6. AnthropicClient request shape: NO temperature key, max_retries=0, cache_control present on system block (assert on captured kwargs). 7. AnthropicClient with `anthropic` package missing (simulated) -> available False, chat_json None, no crash at import. 8. Routing: no llm block -> both consumers get the SAME DeepSeek singleton (shared cap); routed config -> news gets DeepSeekClient while agent_panel gets AnthropicClient; same-provider consumers share one instance. 9. Global lc._client override shadows BOTH consumers (locks the 41 legacy stub sites' semantics). 10. reset_llm_client() clears both the override and the per-provider cache. 11. End-to-end: routing agent_panel->anthropic, attach_panel reaches the anthropic fake (5 calls) while a news call reaches the deepseek stub. 12. telegram _news_health_lines renders per-provider lines from usage() without touching private fields. GATE: Modules 19 and 23 must pass UNMODIFIED - the no-regression proof for the refactor. ================================================================================ 5. DOCS ================================================================================ - CLAUDE.md: llm_client bullet -> provider-abstracted (BaseLLMClient; deepseek + anthropic registered); config.json "llm" block routes consumers; absent = all-DeepSeek. Add ANTHROPIC_API_KEY to env-var list. Test counts. - config.json.example: both new blocks. ================================================================================ COMMIT SEQUENCE (each commit green) ================================================================================ 1. llm_client: extract BaseLLMClient; DeepSeekClient unchanged behavior (+ base-contract tests) - pure refactor, Modules 19/23 green untouched. 2. config: anthropic block + llm consumer-routing block (+ config tests). 3. llm_client: AnthropicClient (official SDK, lazy import) + requirements.txt (+ parse/shape tests). 4. routing: per-consumer get_llm_client + health lines - consumer call-site edits, telegram usage() lines, routing/e2e tests. 5. docs: CLAUDE.md + example config. ================================================================================ VERIFICATION ================================================================================ - python test_all.py green at every commit (~374 -> ~386). - Live smoke A (no behavior change): default config, main.py --panel -> still DeepSeek, /status unchanged. - Live smoke B (the goal): llm.agent_panel="anthropic" + real ANTHROPIC_API_KEY, run --panel -> one real Claude panel (5 requests, validates JSON contract + latency + refusal-free path) while a news refresh still hits DeepSeek; /status shows two provider lines. - Backtest untouched by construction (no LLM in replay path); one 30-day run as a formality. ================================================================================ EFFORT ESTIMATE ================================================================================ 0 new files, 8 modified, ~520 lines, est. 3-5 hours. llm_client.py base class + AnthropicClient + routing +235 test_all.py Module 24 +200 app_config.py anthropic + llm blocks, resolver +55 telegram_bot.py per-provider health lines +15 news_engine.py consumer tags on call sites +4 agent_panel.py consumer tags on call sites +4 config.json.example blocks +14 requirements.txt anthropic +1 CLAUDE.md docs +10 ================================================================================ KEY DECISIONS EMBEDDED IN THIS PLAN ================================================================================ - Official Anthropic SDK, not raw requests (current API guidance for Python; lazy import keeps it optional for DeepSeek-only installs). - Default Claude model: claude-opus-5 ($5/$25 per MTok; a full 5-call panel ~ 8k in / 500 out ~ $0.05). Config-overridable to claude-haiku-4-5. - Two Claude gotchas handled: temperature is REMOVED on Opus 5 (sending it 400s - must not be copied from the DeepSeek request body), and refusals arrive as HTTP 200 with stop_reason "refusal" (checked before parsing, mapped to the normal None/fallback path). - Caps are per-provider, not per-consumer: two consumers on one provider share its cap (today's semantics); splitting across two providers doubles total headroom naturally. - The panel's own max_panels_per_day budget is untouched - it sits above whichever provider serves it. - Not in scope: static/news_summary.php (own PHP client/key), streaming, tool use, per-consumer model overrides (add a "model" key per consumer later if ever needed). ================================================================================