Introducing explicit prompt caching for OpenAI GPT-5.6 models on Amazon Bedrock 在Amazon Bedrock上为OpenAI GPT-5.6模型引入显式提示缓存

TL;DR

• OpenAI GPT-5.6 models (Sol, Terra, Luna) are now generally available on Amazon Bedrock with pay-per-token pricing and AWS security controls.
• Explicit prompt caching is introduced, allowing precise control over cached prompt portions, billed at a 90% discount for 30 minutes.
• The new models support reasoning effort levels (none to xhigh), streaming responses, function calling, and strict JSON schema output via the Responses API.

Why It Matters

This integration enhances cost efficiency and flexibility for AI practitioners by leveraging explicit prompt caching and tiered model capabilities on AWS. It also simplifies migration from earlier GPT models while maintaining compatibility with existing workflows through the OpenAI-compatible Responses API.

Technical Details

• Model Tiers: GPT-5.6 Sol (complex reasoning/agentic coding), Terra (balanced workloads), Luna (fast/high-volume tasks like classification/summarization).
• Explicit Prompt Caching: Users can specify which prompt parts to cache/reuse across requests, reducing costs by 90% for cached inputs valid for 30 minutes.
• Reasoning Effort Levels: Configurable from none (lowest latency) to xhigh (maximum reasoning), with medium as default; sampling parameters (temperature, top_p) only apply when effort=none.
• API Integration: Models use the OpenAI-compatible Responses API on Amazon Bedrock’s bedrock-mantle endpoint, supporting tool calls, structured JSON output, and streamed responses via standard SDK interfaces.
• Authentication: Short-term bearer tokens generated from AWS credentials using aws-bedrock-token-generator, eliminating long-lived secrets in code.

Industry Insight

Agentic workflows involving repeated system instructions or tool definitions will benefit significantly from explicit prompt caching, lowering operational costs and improving scalability. Organizations migrating from older GPT versions should start by retaining current reasoning effort levels before testing lower settings, as GPT-5.6 offers improved token efficiency without sacrificing quality.

TL;DR

• OpenAI GPT-5.6系列模型（Sol、Terra、Luna）现已在Amazon Bedrock全面可用，提供分层能力以满足不同业务需求。
• 引入显式提示缓存（explicit prompt caching），允许用户精确控制哪些提示部分被缓存并跨请求复用，节省高达90%成本。
• 支持推理努力级别调节（none至xhigh）、流式响应输出、函数调用及结构化JSON输出，增强灵活性与集成能力。
• 通过AWS凭证生成短期Bearer Token实现安全认证，兼容OpenAI Responses API，便于迁移现有工作负载。
• 模型按区域部署：Sol仅限美东两区，Terra/Luna额外覆盖美西俄勒冈区，需参考官方文档确认可用性。

为什么值得看

该更新为AI从业者提供了更经济、可控且高效的云端大模型接入方案，尤其适合需要高频重复输入或构建智能体（agentic）系统的企业场景。结合AWS的安全治理与计费体系，有助于降低LLM应用落地门槛并优化TCO。

技术解析

• 模型架构与分级：GPT-5.6家族分为三层——Sol用于复杂推理与编码任务，Terra适用于日常生产负载，Luna面向高速批量处理如分类摘要；三者均基于同一基础架构但针对特定用途调优。
• 显式提示缓存机制：开发者可手动指定prompt中需缓存的部分（如system instruction、tool schema等），缓存有效期30分钟，重用时享90%折扣；区别于隐式自动缓存，此功能提升成本透明度与资源利用率。
• 推理努力控制：新增reasoning.effort参数支持五个等级（none/low/medium/high/xhigh），默认medium；higher level分配更多计算资源解决难题，none则走低延迟路径；当设为none时仍保留temperature/top_p采样控制。
• API交互模式：采用OpenAI兼容的Responses API，支持同步创建、流式返回、工具调用（function calling）及严格JSON模式输出；示例代码展示如何通过Python SDK快速初始化客户端并发请求。
• 安全与部署细节：推荐使用aws-bedrock-token-generator从AWS IAM动态获取短命token避免密钥硬编码；模型端点位于bedrock-mantle，地域分布受限，部署前须核对Region支持情况。

行业启示

• 成本驱动型优化成为主流：显式缓存机制标志着LLM服务从“黑盒调用”转向“精细化成本管理”，未来更多平台或将推出类似可编程缓存策略以吸引高用量客户。
• 智能体开发标准化加速：统一的Responses API + 工具调用+结构化输出组合，降低了多步决策型应用（agent）的开发复杂度，推动AutoGen类框架向云厂商原生能力靠拢。
• 云厂商生态壁垒加深：AWS通过与OpenAI深度绑定（专属模型命名、权限整合、账单抵扣），强化其在企业级AI基础设施中的主导地位，迫使其他云平台加快差异化竞争节奏。