The setup involves deploying a customer-operated LiteLLM gateway on Amazon ECS using AWS Fargate. This gateway connects to an OpenAI model hosted on Amazon Bedrock. Requests are routed through Codex, which manages responses with scoped identities, budgets, rate limits, and telemetry.
Engineers can compare direct IAM Identity Center access with a managed Portkey deployment for managing access and security. This configuration allows for scalable, controlled interaction with OpenAI models within AWS infrastructure.
Such deployment options enable precise request management and monitoring, which are critical for production environments handling large-scale or sensitive AI workloads.