AI News AI资讯 8h ago Updated 5h ago 更新于 5小时前 46

How to Secure AI Agents, MCP Servers, and LLM Apps in Production 如何在生产中保护 AI 代理、MCP 服务器和 LLM 应用的安全

Mend.io released a practical security framework for AI agents, MCP servers, and LLM applications, organized around three moves: see what matters, fix what matters faster, and protect AI in production Traditional AppSec fails for agentic AI because behavior emerges from models, prompts, context, and tools—not just code—creating novel failure modes like prompt injection through data and poisoned tool descriptions The framework introduces a five-layer attack surface map (interaction, agent, integra Mend.io发布实践指南,提出"看见-修复-保护"三阶段框架解决AI Agent、MCP服务器和LLM应用的安全治理缺口 传统AppSec假设失效,AI行为由模型、提示词、上下文和用户输入共同涌现,产生提示词注入、工具中毒等新型攻击面 提出五层攻击面映射(交互/Agent/集成/模型/代码)及AI-BOM扩展字段、12项配置检查清单等7个可复用工件 建立自动化分级处置流水线:证据充分的低风险项自动关闭,高风险和新型AI行为必须人工决策 运行时防护通过Python SDK或Docker API Server部署内外向Guardrails,结合AI红队形成持续改进闭环

62
Hot 热度
70
Quality 质量
65
Impact 影响力

Analysis 深度分析

TL;DR

  • Mend.io released a practical security framework for AI agents, MCP servers, and LLM applications, organized around three moves: see what matters, fix what matters faster, and protect AI in production
  • Traditional AppSec fails for agentic AI because behavior emerges from models, prompts, context, and tools—not just code—creating novel failure modes like prompt injection through data and poisoned tool descriptions
  • The framework introduces a five-layer attack surface map (interaction, agent, integration, model, code) and seven reusable artifacts including an extended AI-BOM, misconfiguration checklist, automation triage guidelines, and guardrail deployment patterns
  • Runtime protection combines inbound/outbound guardrails (available as Python SDK or Docker API Server), system prompt hardening patterns, and a continuous loop with AI red teaming
  • A maturity roadmap with four stages (Emerging to Leading) aligns to NIST AI RMF, OWASP AIMA, ISO/IEC 42001, and the EU AI Act via a 15-question self-assessment

Why It Matters

This framework addresses a critical gap as AI agents and MCP integrations proliferate in production faster than security programs can discover and govern them. It provides actionable, artifact-driven guidance rather than abstract principles, making it directly applicable for security teams, platform engineers, and AI developers who need to operationalize AI security. The emphasis on shadow agent discovery and evidence-backed automation triage reflects the practical realities of scaling AI security in enterprise environments.

Technical Details

  • Five-layer attack surface map: Interaction layer (prompt injection, context poisoning, data exfiltration), Agent layer (over-permissioned tools, unsafe defaults, goal hijacking), Integration layer (poisoned MCP tool descriptions, unscoped credentials, shadow servers), Model layer (EOL models, supply chain risk, unsafe generations), and Code layer (vulnerable AI-generated code, framework CVEs, malicious packages)
  • Seven reusable artifacts: (1) Five-layer attack surface map, (2) AI-BOM extension with nine fields per agent/MCP server, (3) 12-point misconfiguration checklist, (4) Automation triage decision framework, (5) Guardrail deployment patterns, (6) System prompt hardening validation checks, (7) 15-question maturity self-assessment
  • Discovery strategy: Five methods for hunting shadow agents—repository scanning for agentic signatures, network egress monitoring for model API calls, service account/API key audits, lightweight registration declarations, and continuous automated discovery
  • Guardrail deployment: Two delivery modes—an in-app Python SDK (Online or Offline) and a standalone Docker API Server requiring no code changes; minimal viable setup includes inbound guardrails (prompt injection, jailbreaks, out-of-policy requests) and outbound guardrails (credentials, PII, proprietary code, unsafe content)
  • Triage automation rules: Automated closure requires evidence trails; novel AI behaviors and risk acceptance decisions remain human-only; error rate sampling with retraining thresholds

Industry Insight

  • Organizations should prioritize discovering and inventorying shadow agents and unregistered MCP servers before investing in advanced guardrails—visibility precedes protection in agentic AI environments
  • The shift from code-centric to behavior-centric security requires retraining AppSec teams on AI-specific failure modes like tool poisoning and prompt injection through data, which bypass traditional vulnerability scanning
  • The framework's alignment with multiple regulatory standards (NIST AI RMF, OWASP AIMA, ISO/IEC 42001, EU AI Act) suggests a converging consensus on AI security governance, making these artifacts valuable for compliance-driven deployments

TL;DR

  • Mend.io发布实践指南,提出"看见-修复-保护"三阶段框架解决AI Agent、MCP服务器和LLM应用的安全治理缺口
  • 传统AppSec假设失效,AI行为由模型、提示词、上下文和用户输入共同涌现,产生提示词注入、工具中毒等新型攻击面
  • 提出五层攻击面映射(交互/Agent/集成/模型/代码)及AI-BOM扩展字段、12项配置检查清单等7个可复用工件
  • 建立自动化分级处置流水线:证据充分的低风险项自动关闭,高风险和新型AI行为必须人工决策
  • 运行时防护通过Python SDK或Docker API Server部署内外向Guardrails,结合AI红队形成持续改进闭环

为什么值得看

本文首次系统性地拆解了Agentic AI时代安全范式的根本转变——从"代码执行确定性"到"行为涌现不确定性",为安全团队提供了可落地的治理框架。对AI应用开发者、安全工程师和合规负责人而言,五层攻击面模型和AI-BOM扩展字段可直接用于资产盘点和风险量化。

技术解析

五层攻击面映射:交互层(用户输入/检索文档/Agent间消息→提示词注入/上下文中毒)、Agent层(系统提示词/配置/记忆/自主性设置→过度授权/目标劫持)、集成层(MCP服务器/工具定义/API→工具中毒/影子服务器)、模型层(基础模型/微调模型/嵌入→供应链风险/EOL模型)、代码层(AI生成代码/框架/SDK→漏洞/CVE)。

AI-BOM扩展与配置检查:在AI物料清单中增加身份、模型依赖、自主级别、工具权限、凭证范围、数据访问范围、MCP端点、提示词位置、最后审查时间等9个关键字段;配套12项配置检查清单,包括凭证最小化、高影响工具人工审批、系统提示词版本控制、MCP客户端认证、工具描述注入审查等。

自动化分级处置流水线:基于可达性、可利用上下文、业务上下文、Agent放大效应、修复可用性五维信号排序;明确自动化边界——证据充分的低风险项自动关闭,FP/TP评估抽样审查,Tier-3/高风险应用AI辅助人工决策,新型AI行为和风险接受必须人工记录。

运行时Guardrails部署:支持内嵌Python SDK(Online/Offline模式)和独立Docker API Server两种形态;最小可行配置包含内向Guardrails(拦截提示词注入/越权请求/Jailbreak)和外向Guardrails(拦截凭证/PII/专有代码/不安全内容);系统提示词加固遵循假设泄露、指令数据分离、限制爆炸半径、版本审查、对抗测试五原则。

行业启示

安全治理必须从"代码为中心"转向"行为为中心":传统漏洞扫描无法覆盖提示词注入、工具中毒等数据驱动攻击,安全团队需要建立覆盖模型供应链、Agent配置、MCP集成的全栈可见性,将AI-BOM纳入现有软件物料清单管理体系。

自动化与人工决策的边界需要明确定义:AI安全问题的复杂性决定了完全自动化不可行,但人工处理所有告警也不可持续;建议建立基于证据充分性和风险等级的分级处置机制,将人工精力集中在新型攻击模式和高风险决策上。

运行时防护与持续红队形成闭环:Guardrails不应是一次性部署,而应作为持续监控和迭代的基础设施;通过红队发现改进Guardrails规则,再通过Guardrails日志指导红队测试方向,形成自我强化的安全改进循环,同时对齐NIST AI RMF、OWASP AIMA和EU AI Act合规要求。

Disclaimer: The above content is generated by AI and is for reference only. 免责声明:以上内容由 AI 生成,仅供参考。

Security 安全 Agent Agent LLM 大模型 Deployment 部署