Set up OpenAI ChatGPT Codex with LiteLLM on Amazon ECS and Amazon Bedrock
OpenAI Codex can be configured to route model inference through a customer-operated LiteLLM gateway deployed on Amazon ECS, enabling centralized enterprise controls while keeping the task loop local on developer workstations LiteLLM serves as a shared control point for model authentication, routing, budgets, rate limits, and telemetry between Codex and Amazon Bedrock's OpenAI-compatible models The architecture uses AWS WAF, Application Load Balancer, Amazon RDS (PostgreSQL), Secrets Manager, KMS
Analysis
TL;DR
- OpenAI Codex can be configured to route model inference through a customer-operated LiteLLM gateway deployed on Amazon ECS, enabling centralized enterprise controls while keeping the task loop local on developer workstations
- LiteLLM serves as a shared control point for model authentication, routing, budgets, rate limits, and telemetry between Codex and Amazon Bedrock's OpenAI-compatible models
- The architecture uses AWS WAF, Application Load Balancer, Amazon RDS (PostgreSQL), Secrets Manager, KMS, and CloudWatch for a production-grade deployment
- Direct Bedrock access suits teams needing minimal complexity with native AWS IAM/CloudTrail, while LiteLLM adds value for cross-team consistency, scoped keys, hard budgets, and centralized routing policies
- The complete reference implementation is available in the guidance-codex repository with a LiteLLM on AWS quickstart guide
Why It Matters
This architecture addresses a critical gap as organizations scale AI coding agents from individual experimentation to enterprise-wide managed adoption, providing the governance layer that engineering leaders require. It demonstrates a practical pattern for combining OpenAI's Codex tooling with AWS's managed AI infrastructure while maintaining security, cost control, and operational visibility.
Technical Details
- Architecture: LiteLLM gateway runs on AWS Fargate (ECS) behind an Application Load Balancer and AWS WAF, with Amazon RDS (PostgreSQL) for state/usage/budget data, and AWS Secrets Manager + KMS for key storage
- Request Flow: Codex sends task context and tool definitions to the gateway's
/v1/responsesendpoint → ALB/WAF apply network controls → LiteLLM authenticates, checks policy, and invokes the Bedrock model via ECS task role → Bedrock returns text or function calls → Codex executes tools locally under sandbox/approval policy and loops back - Model Configuration: Validated with gateway alias
openai.gpt-5.5mapping tobedrock_mantle/openai.gpt-5.5in LiteLLM config; supports semantic continuation, streaming, and function calling validation - Prerequisites: AWS account with VPC/ECS/ELB/RDS/ECR/WAF/IAM/KMS/Secrets Manager/CloudWatch permissions, Bedrock model access, AWS CLI v2, Docker with Buildx, Codex CLI, Python 3, and ACM certificate for HTTPS
- Operational Trade-off: Customer owns gateway availability, database lifecycle, version upgrades, incident response, and capacity planning; the gateway does not receive a general-purpose shell or replace Codex's local approvals
Industry Insight
- Organizations should evaluate whether native AWS IAM + CloudTrail controls suffice before investing in a gateway layer; LiteLLM becomes justified when cross-team consistency, scoped virtual keys, and hard budgets are required
- The separation of local tool execution (Codex) from centralized model governance (LiteLLM gateway) represents a best-practice pattern for enterprise AI adoption that balances developer autonomy with organizational control
- Teams should plan for the operational burden of self-hosted gateways—availability, upgrades, and capacity planning—or consider managed alternatives like Portkey when internal resources are constrained
Disclaimer: The above content is generated by AI and is for reference only.