Imported from vhspace/mcp-common (cursor-marketplace/awx-mcp/skills/awx-automation/SKILL.md). Install upstream with npx skills add vhspace/mcp-common --skill awx-automation. Copyright stays with the author.

AWX Automation

Prefer CLI when shell access is available — ~90% fewer tokens.

CLI Path

IMPORTANT: The CLI wrapper auto-sources .env for credentials. Never manually source, export, or grep env vars — just run the command directly.

Discover flags: Run awx-cli <command> --help — not all commands support the same options.

If awx-cli is not on PATH, install with uvx --from awx-mcp awx-cli or run from the repo with uv run awx-cli.

MCP Path

Always pass fields to reduce MCP response size (~80-90% token savings).

Triage Failed Job

Use structured events, not raw stdout:

CLI:

awx-cli events <JOB_ID> --failed
awx-cli stdout <JOB_ID> --filter errors
awx-cli stdout <JOB_ID> --filter errors --host "failing-host*"

MCP:

awx_list_resources("job_events", filters={"failed": "true"}, parent_type="jobs", parent_id=<JOB_ID>, page_size=10)
awx_get_job_stdout(job_id=<JOB_ID>, filter="errors")
awx_get_job_stdout(job_id=<JOB_ID>, filter="errors", host="failing-host*")

If the error is Permission denied (publickey), the node likely needs bootstrapping — see Freshly Deployed Nodes (Bootstrap) below.

Cross-Tool Workflow

1. NetBox → resolve hostname (returns FQDN)
2. AWX → use short hostname in --limit (strip .cloud.together.ai). Verify with awx-cli hosts <INVENTORY_ID> if unsure.
3. Redfish → use oob_ip from NetBox for BMC operations

Freshly Deployed Nodes (Bootstrap)

MAAS-deployed machines only have the ubuntu user. AWX connects as the ansible user (credential id 3, ansible-service-account), so AWX jobs will fail with Permission denied (publickey) on new nodes until the ansible user is bootstrapped.

First, check whether AWX can already reach the node (no AWX credentials required):

awx-cli check-access <HOSTNAME>

If check-access reports OK, skip straight to AWX job templates. If it reports FAIL, run prep-awx-access.yaml from a local ansible host:

cd infra/ansible
ansible-playbook -i "HOSTNAME," prep-awx-access.yaml

For multiple nodes or inventory-based targeting:

CLUSTER_NAME="XYZ" ansible-playbook prep-awx-access.yaml --limit "node1,node2"

The playbook automatically tries your personal SSH user first and falls back to ubuntu — no extra flags needed. It applies the awx-ansible-user role which creates the ansible account and installs the authorized_keys.

Only proceed to AWX job templates (e.g. template 472, prep-ori-gpu-node.yaml) after this completes successfully.

Guided Prompts (MCP only)

• triage_failed_job(job_id) — step-by-step failure investigation
• launch_deployment(template_name) — find template, review survey, launch
• check_cluster_health() — ping, cluster status, metrics
• investigate_host(hostname) — cross-MCP: NetBox lookup then AWX investigation

Key Gotchas

• AWX inventories use short hostnames — NetBox returns FQDNs like host.cloud.together.ai but AWX inventories have just host. Strip the domain or run awx-cli hosts <ID> to check. Using a FQDN in --limit silently matches zero hosts.
• Use awx_launch_and_wait / awx-cli launch --wait over manual launch+poll
• Use job events for failure triage — structured data beats parsing stdout
• Field selection saves tokens (MCP) — pass fields to MCP tools; CLI --fields only works on templates, workflows, jobs, and list
• Filters use Django-style lookups — name__icontains, status, created__gt, etc.
• parent_type/parent_id — required for nested resources (e.g. events under a job)
• Transient errors retried — 429, 502, 503, 504 are automatically retried with backoff
• Freshly deployed nodes need bootstrap first — MAAS nodes only have ubuntu; run prep-awx-access.yaml before any AWX template (see "Freshly Deployed Nodes" above)
• The server can be flaky — if a tool call fails, retry once before giving up