from-zero-to-ai-agent-building-with-azure-ai-agent-service

Imported from dirien/from-zero-to-ai-agent-building-with-azure-ai-agent-service (AGENTS.md). Install upstream with npx skills add dirien/from-zero-to-ai-agent-building-with-azure-ai-agent-service. Copyright stays with the author.

AGENTS.md

For AI coding agents working in this repo. Humans: read README.md instead.

What this repo is

A three-stage demo for a Pulumi/Microsoft Foundry talk: "From Prompt Agent to Production: AI Agents on Azure AI Foundry". Mirrors the structure of the AWS Bedrock AgentCore talk at /Users/dirien/Tools/repos/from-works-on-my-machine-to-production-ready-ai-agents-with-amazon-bedrock-agent-core.

Single fraud-detection narrative — the same agent grows from a local Python script to a containerized Foundry-hosted production agent.

Stages

Conventions

• Pulumi only for IaC. pulumi-azure-native + the bridged dirien/azurefoundry provider. No Bicep, no azapi, no Terraform native.
• Python everywhere. No TypeScript anywhere in this repo.
• swedencentral for everything (Foundry, ACR, Container Apps, AI Search). Foundry's Responses API + vector stores don't run in westeurope — only in australiaeast / eastus / eastus2 / francecentral / japaneast / swedencentral / uksouth. Sweden Central has the widest EU model coverage.
• uv for all Python package management. pyproject.toml per project, uv sync to install.
• ESC envs imported in Pulumi.<stack>.yaml (per-stack), never in Pulumi.yaml. Stage 2 + 3 each define their own Pulumi.dev.yaml with environment: [foundry-talk/dev].

Bridged provider

The TF-bridged dirien/azurefoundry provider supplies typed Pulumi resources for the Foundry data plane (agents, files, vector stores). It lives at:

• Source: https://github.com/dirien/terraform-provider-azurefoundry
• Local clone (for hacking): /Users/dirien/Tools/repos/terraform-provider-azurefoundry
• TF Registry: registry.terraform.io/dirien/azurefoundry
• Bridge command: pulumi package add terraform-provider registry.terraform.io/dirien/azurefoundry 0.8.4 (both Stage 2 and Stage 3 are pinned at 0.8.4 — needed for ToolboxV2, KnowledgeSource, KnowledgeBase, ProjectIndex, plus the headers + allowed_tools fields on mcp tool blocks).
• Resource catalogue (v0.8.4): Agent, AgentV2, File, FileV2, VectorStore, VectorStoreV2, MemoryStoreV2, ToolboxV2, KnowledgeSource, KnowledgeBase, ProjectIndex. Project connections still live on the ARM plane (use azure-native:cognitiveservices:ProjectConnection from pulumi-azure-native).

Provider config (typed):

• project_endpoint — required
• Auth: oidc_token + client_id + tenant_id (ESC OIDC, preferred); falls back to client_secret, api_key, use_azure_cli, or DefaultAzureCredential chain.
• Reads ARM_OIDC_TOKEN / ARM_CLIENT_ID / ARM_TENANT_ID from env (Pulumi ESC compatibility), so no provider block fields needed when running under pulumi env run ediri/pulumi-idp/auth.

Commands

ESC env imports are in each stack's Pulumi.dev.yaml, so the env auto-loads — no pulumi env run prefix needed for infra/. Stage 1's runner is pure Python (no Pulumi project) so it does need the prefix to inject env vars.

Gotchas

1. ESC OIDC token expires every hour. If az CLI commands start failing with AADSTS700024, re-issue via pulumi env run ediri/pulumi-idp/auth -- bash -c 'az login --service-principal --username "$ARM_CLIENT_ID" --federated-token "$ARM_OIDC_TOKEN" --tenant "$ARM_TENANT_ID"'. pulumi-azure-native and the bridged provider's OIDC path pick up the env vars natively and refresh on each run — only the az CLI caches. Same goes for any Python script using DefaultAzureCredential against https://ai.azure.com/.default — it'll fail with the same AADSTS700024 once the token expires.
2. Custom subdomain reservation. Deleting a Foundry cognitive account leaves a soft-delete with a 48-hour subdomain reservation. Purge it before recreating: az cognitiveservices account purge --name <name> --location <region> --resource-group <rg>.
3. RBAC propagation. Managed-identity role assignments take 10–30 min to flow through Foundry's access-check cache. Symptom: 401 from the Responses API immediately after pulumi up. Wait and retry — don't add more role assignments.
4. MAF doesn't load .env files automatically. Either set env vars in the shell (recommended — Pulumi ESC handles this) or call dotenv.load_dotenv() at the start of your script.
5. pulumi package add terraform-provider <name> defaults to registry.opentofu.org. Always prefix with registry.terraform.io/ for the bridged azurefoundry provider. Also: after a fresh upstream release, the TF registry sync is faster than Pulumi's resolver — you can see the version listed at https://registry.terraform.io/v1/providers/dirien/azurefoundry but pulumi package add still says "Could not resolve a version". Wait 30–60s and retry.
6. Stage 1 has no cloud dependency. Stage 1 runs entirely on Foundry Local — no Azure account, no Stage 2 prerequisite. Stage 2 and Stage 3 are independently deployable (each has its own resource group + Foundry account).
7. Pyright unresolved-import warnings for agent_framework, pulumi_azurefoundry, pulumi_azure_native, azure.ai.evaluation, azure.ai.projects are expected false positives — the venvs live under each stage's .venv and editor-side Pyright doesn't index them. Ignore.
8. project.name is the ARM path, not the bare project name. cognitiveservices.Project.name returns <account>/<project> (e.g. foundry-talk-dev/fraud-detection). When constructing the Foundry data-plane project_endpoint for the bridged provider, use the literal PROJECT_NAME constant, not project.name.apply(...). Symptom: every FileV2/VectorStoreV2/AgentV2 upload returns 404 "project does not exist".
9. westeurope does not support the Responses API and therefore no Foundry vector stores, no prompt agent kind. Supported regions (as of 2026-04): australiaeast / eastus / eastus2 / francecentral / japaneast / swedencentral / uksouth. Symptom: HTTP 400 "Vector stores are not supported in this region." See https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/responses#region-availability.
10. Hosted-agent runtime can return status: "failed" with empty output after a pulumi up that bumped the agent's version. The new agent version takes 2–5 min to warm up. Symptom: ./test.sh agents produces blank decisions, or ./test.sh eval shows decision_actual=UNKNOWN. Wait and retry. Also: a single corrupted session (agent_session_id) can poison itself permanently — every call against that exact scope returns 500 forever. Use --fresh on agents mode to suffix user-IDs with a per-run UUID, sidestepping any poisoned scope.
11. Toolbox preview opt-in header isn't auto-injected. When a kind=prompt agent is wired to consume an azurefoundry_toolbox_v2.consumer_endpoint, Foundry's runtime calls the toolbox MCP endpoint without the Foundry-Features: Toolboxes=V1Preview header → HTTP 403 "Access denied". Workaround: set the header explicitly on the agent's mcp tool block via the headers map (added in v0.8.1). Drop this once the runtime auto-injects.
12. CognitiveSearch project connection needs ApiKey auth (not AAD) for the portal Indexes tab to populate. AAD-auth connections work for KB enumeration but the portal's index lister uses the connection's stored credentials directly. Use authType=ApiKey with credentials.key = <search_admin_key> from search.list_admin_key_output(...). Plus the project MI needs both Search Index Data Contributor and Search Service Contributor on the search service (not just Search Index Data Reader — the docs are explicit about this for keyless retrieval at agent runtime).
13. azurefoundry_project_index is required for Foundry IQ → Indexes tab to populate. The portal calls listProjectIndexesResolver against the project, which walks a Foundry-managed catalog separate from the underlying Search indexes. Just having a KnowledgeBase doesn't auto-register an index. Bridged provider added ProjectIndex in v0.8.2; v0.8.3 fixed PUT→PATCH routing; v0.8.4 fixed the body shape (connectionName/indexName flat, not nested under azureSearch).
14. Portal users need both Azure AI User AND Azure AI Project Manager on the Foundry account. The portal's "no permission" page only mentions Azure AI User but several build-side surfaces (Evaluations, Red team, Guardrails, agent playground tracing) gate on Project Manager. Set via pulumi config set portalUserObjectIds "<oid1>,<oid2>" on the Stage 3 stack.
15. AI Red Teaming SDK has two paths — local PyRIT callback vs cloud-mode. The local path (RedTeam(...).scan(target=callable)) runs but does not render in the new Foundry portal Red team tab. For portal-visible runs, use the cloud-mode pattern: AzureAIAgentTarget + openai_client.evals.create() + project_client.beta.evaluation_taxonomies.create() + evals.runs.create(data_source.type=azure_ai_red_team). See advanced-foundry-deployment/red_team/run_redteam.py.
16. azure-ai-evaluation legacy evaluate() runner ignores async target callables silently. Async targets produce unawaited coroutines that the evaluators can't score → total: 0, passed: 0, failed: 0. Always use sync targets. Also: the bulk runner spawns parallel workers and threading-locks inside the target only serialise within a single worker — for true serial execution use the cloud-mode azure_ai_target_completions path instead.
17. Stage 3 governed agent's MCP tools cannot all be inlined in a Toolbox. Foundry's toolset API allows only mcp / web_search / azure_ai_search / openapi / a2a_preview / code_interpreter / file_search in toolboxes. function and memory_search must stay inline on the agent. Symptom on pulumi up: Tool type 'memory_search_preview' is not supported in toolsets.

Prerequisites for running

• Membership in the ediri Pulumi org (the pulumi-idp/auth ESC env lives there).
• Python 3.11+ with uv on PATH.
• pulumi CLI ≥ 3.231.
• az CLI 2.84+ on PATH (Stage 3 only — used by az acr build).

Do NOT

• Never commit secrets. Pulumi ESC or pulumi config set --secret only.
• Never introduce a third Pulumi provider. Azure-native + dirien/azurefoundry is the entire dependency set.
• Never add a fourth stage without updating both README.md and the table above.
• Never put environment: imports in Pulumi.yaml — always in the per-stack Pulumi.<stack>.yaml.