How I Taught Claude Code to Offload Grunt Work to a Local 4B Model
Claude Code subagents cannot route to different models—they share the same base URL, making model isolation impossible through subagent configuration alone Setting ANTHROPIC_BASE_URL to a local llama.cpp server creates an all-or-nothing swap, downgrading the entire session rather than enabling hybrid routing The working pattern combines Claude Code Skills (markdown-based task descriptions) with Bash execution to delegate low-stakes text tasks to a local Qwen3.5-4B model while keeping Claude as t
Analysis
TL;DR
- Claude Code subagents cannot route to different models—they share the same base URL, making model isolation impossible through subagent configuration alone
- Setting ANTHROPIC_BASE_URL to a local llama.cpp server creates an all-or-nothing swap, downgrading the entire session rather than enabling hybrid routing
- The working pattern combines Claude Code Skills (markdown-based task descriptions) with Bash execution to delegate low-stakes text tasks to a local Qwen3.5-4B model while keeping Claude as the primary reasoning engine
- A critical bug involves Qwen3.5's reasoning mode: when
enable_thinkingis active, the model consumes all allocated tokens in<think>blocks without producing output in thecontentfield, requiring explicit disabling viachat_template_kwargs - The Skill + Bash composition is a general-purpose pattern extensible to any localhost service—embeddings models, OCR, fine-tuned domain models, or image generation—without requiring MCP servers or proxy infrastructure
Why It Matters
This article provides a practical, working blueprint for hybrid local-cloud LLM workflows that many AI practitioners have been seeking, demonstrating that complex proxy or MCP infrastructure is unnecessary for basic task delegation. The Skill + Bash pattern lowers the barrier to entry for developers who want to reduce API costs and leverage idle GPU resources while maintaining the quality of cloud models for complex reasoning tasks.
Technical Details
- Claude Code Skills architecture: Skills are markdown files stored at
~/.claude/skills/<name>/containing frontmatter withnameanddescriptionfields that instruct Claude when to invoke the skill, and a body defining the execution logic - llama.cpp server configuration: Runs Qwen3.5-4B GGUF (UD-Q4_K_XL quant, ~4GB VRAM) with
--jinjaflag for proper chat template handling and--port 8080, exposing an Anthropic-compatible/v1/messagesendpoint - Python bridge script (
ask_qwen.py): A 60-line standard-library-only script that accepts prompts via stdin or CLI arguments, constructs OpenAI-compatible chat completion requests, handles theenable_thinkingflag, and parses responses including thereasoning_contentvscontentdistinction - Reasoning trap bug: Qwen3.5's default
enable_thinking=Truecauses the model to fill the entire context window with chain-of-thought reasoning without closing the<think>block, resulting in emptycontentfields; fixed by settingchat_template_kwargs: {"enable_thinking": False} - Delegation criteria: Tasks must be text-in/text-out, self-contained, low-stakes, and tedious—explicitly excluding code edits, debugging, multi-step reasoning, and codebase-aware operations
Industry Insight
- The Skill + Bash pattern demonstrates that lightweight, composable tooling can outperform heavier infrastructure like MCP servers for simple routing use cases—practitioners should evaluate complexity requirements before adopting standardized protocols
- Local-first hybrid architectures that keep cloud models for reasoning and local models for grunt work represent a sustainable cost-performance balance, especially for developers on flat-rate API plans where marginal token savings are irrelevant but GPU utilization and offline capability matter
- The reasoning-mode gotcha highlights an emerging class of bugs as more models adopt extended thinking capabilities; developers integrating reasoning models into automated pipelines must explicitly control thinking flags and handle split response fields
Disclaimer: The above content is generated by AI and is for reference only.