I Built an AI Agent That Can Query My Kubernetes Cluster, But Never Break It 我构建了一个可以查询我的 Kubernetes 集群但永远不会破坏它的 AI 代理

TL;DR

• The article describes a safe, read-only AI agent for querying Kubernetes clusters using local LLMs (Qwen3 via Ollama) and fixed Python functions wrapping kubectl commands.
• Safety is enforced through two layers: the model only expresses intent by selecting from predefined tools (never generating raw commands), and Kubernetes RBAC restricts the ServiceAccount to get/list/watch permissions only.
• A key technical challenge was networking within Kubernetes pods, requiring direct IP addressing of the Docker bridge gateway and container names instead of localhost or published ports.
• Self-hosted smaller LLMs showed unreliability in tool-calling compared to hosted models, with issues including plain-text JSON output and incomplete calls, necessitating fallback parsers and careful schema management.

Why It Matters

This work provides a practical blueprint for deploying AI agents in sensitive infrastructure environments where safety is paramount. It demonstrates that robust security can be achieved not just by trusting the model's reasoning, but by architecting the system so that the model's capabilities are strictly bounded by both application-level function selection and infrastructure-level permission controls. This approach is highly relevant for DevOps teams looking to integrate AI assistants without introducing risk of accidental or malicious cluster modification.

Technical Details

• Architecture: Browser dashboard → FastAPI/ask endpoint → Local Ollama instance running Qwen3 → Fixed Python functions (get_pods, get_logs, describe_pod, get_deployments, get_services, get_nodes) → Kubernetes API → Model generates natural language response.
• Safety Mechanism 1 (Application Layer): The model never executes commands directly; it selects one of six predefined functions with parameters (e.g., pod name). Each function wraps a single hardcoded kubectl command template, preventing command injection or chaining.
• Safety Mechanism 2 (Infrastructure Layer): The agent runs under a Kubernetes ServiceAccount bound to a ClusterRole granting only get, list, and watch permissions on specific resources (pods, logs, deployments, services, nodes). Write operations and secret access are explicitly denied and verified via attempted write rejection.
• Networking Solution: Resolved pod-to-host communication by discovering the Docker bridge network gateway IP (via ip a on br-xxxxxx interface) and setting OLLAMA_URL to this IP. For Kubernetes API, connected directly to the control-plane container by name on port 6443 after attaching to the same Docker network.
• Model Limitations: Self-hosted LLMs exhibited inconsistent tool-calling behavior—sometimes outputting unstructured JSON instead of invoking the tool mechanism, or failing to complete valid tool requests. Schema mismatches (e.g., get_nodes registered backend but missing in sent schema) caused reasoning without execution capability. Model switching (llama3.2 → llama3.1 → qwen3) improved reliability.

Industry Insight

AI agents managing critical infrastructure must adopt a "defense-in-depth" strategy combining constrained model interfaces (intent-only selection over command generation) with strict least-privilege IAM policies at the platform level. Developers should anticipate significant non-model challenges when deploying agents inside containers—particularly networking complexities in orchestrated environments like Kubernetes—and validate permissions empirically rather than assuming configuration correctness. Additionally, self-hosted smaller models require rigorous testing and fallback logic for tool invocation due to inherent instability in structured output generation compared to large hosted alternatives.

TL;DR

• 项目构建了一个安全的AI代理，仅能查询Kubernetes集群状态且无法执行任何写入操作。
• 采用本地部署的Qwen3模型配合固定Python函数封装kubectl命令，实现意图与执行分离的安全架构。
• 通过RBAC权限限制和网络配置解决容器内服务通信问题，确保即使模型出错也不会破坏基础设施。
• 发现自托管小模型在工具调用可靠性上弱于云端大模型，需设计容错机制应对模型幻觉或调用失败。
• 强调DevOps AI工具“承认无知”比盲目猜测更重要，构建了多层防御（应用层+基础设施层）保障安全。

为什么值得看

该实践为AI Agent落地生产环境提供了可复用的安全范式，尤其解决了开发者对AI误操作基础设施的核心顾虑。其分层防护思路（代码约束+K8s RBAC）和工程细节（如Docker桥接网络IP定位）对构建可信AI运维工具有直接参考价值。

技术解析

• 安全架构设计：模型仅输出意图（如get_logs），不生成实际命令；所有操作被限制在6个预定义Python函数内，每个函数对应单一kubectl模板，杜绝命令注入风险。
• 权限隔离机制：Agent运行于K8s ServiceAccount下绑定只读ClusterRole（get/list/watch），经测试验证写操作会被K8s API Server主动拒绝，形成第二道防线。
• 网络连通性方案：解决Pod内访问宿主机Ollama的问题——通过ip a查找Docker桥接网卡的gateway IP（如br-xxxxx）设置OLLAMA_URL；K8s API Server则通过容器名直连6443端口而非暴露主机端口。
• 模型选型迭代：对比llama3.2/llama3.1/Qwen3发现工具调用稳定性差异，Qwen3表现最优；同时识别出“模型未收到完整tool schema”导致的虚假能力问题，需区分模型缺陷与应用bug。
• 容错策略：当模型无法回答时明确声明“不可知”，避免编造信息；对非结构化JSON响应添加fallback解析器处理模型输出异常。

行业启示

• AI Agent安全必须纵深防御：单纯依赖模型指令过滤不足，需结合基础设施级权限控制（如K8s RBAC）和应用层函数白名单，形成“模型意图→受限执行→系统拦截”的三层保护。
• 自托管模型的工具调用成熟度仍是瓶颈：当前开源小模型在结构化输出稳定性上落后于商业API，企业级应用应预留人工审核或降级回退机制，尤其在关键运维场景。
• 网络拓扑是AI Agent落地的隐形门槛：容器化环境中服务发现（如Pod访问宿主机服务）缺乏标准化方案，需提前规划网络策略（如共享Docker桥接网），否则将大幅增加调试成本。