huawei-cloud-cce-cci-bursting-deployer

Imported from pancake0001/huawei-cloudnative-skills (releases/container/cce-unscheduled/huawei-cloud-cce-cci-bursting-deployer/SKILL.md). Install upstream with npx skills add pancake0001/huawei-cloudnative-skills --skill huawei-cloud-cce-cci-bursting-deployer. Copyright stays with the author.

Huawei Cloud CCE-CCI Bursting Deployer

Overview

This skill configures CCE workloads to burst into CCI 2.0 serverless capacity for elastic scaling. It automates the full workflow: precheck cluster readiness including NodeCheck for physical-node headroom, ensure VPCEP dependencies for image pulling, install or update the virtual-kubelet addon, discover tenant-owned smoke images, deploy a smoke workload, verify the virtual node and pod status, and diagnose addon failures.

Architecture: python3 scripts/huawei-cloud.py → CCE/VPCEP/VPC API → Precheck → NodeCheck → VPCEP creation → Addon install → Image discovery → Smoke deployment → Verification → Diagnosis

Key Principle: Preview-first. Read-only checks (precheck, NodeCheck, verify, discover images, diagnose addon) run immediately, but VPCEP creation, addon installation, node pool changes, and smoke workload deployment require explicit user approval with confirm=true before execution.

Related Skills:

• huawei-cloud-cce-cluster-management - Cluster lifecycle, addon listing, kubeconfig retrieval
• huawei-cloud-cce-pod-failure-diagnoser - Pod failure diagnosis when bursting pods fail to reach Running
• huawei-cloud-cce-network-failure-diagnoser - Network diagnosis for VPCEP connectivity issues

Prerequisites

1. Python Requirements (MANDATORY)

• Python 3 installed (version >= 3.8)
• Run python3 --version to verify installation
• huawei-cloud.py script available in the scripts directory
• Required packages: huaweicloudsdkcce, huaweicloudsdkcore, huaweicloudsdkvpc, huaweicloudsdkvpcep, kubernetes

2. Credential Configuration

• Valid Huawei Cloud credentials (AK/SK mode)
• Security Rules:
  • Never expose AK/SK values in code, conversation, or commands
  • Never use echo commands to check credential environment variables
  • Use environment variables: HUAWEI_AK, HUAWEI_SK, HUAWEI_PROJECT_ID
  • Alternative env vars: HUAWEICLOUD_SDK_AK, HUAWEICLOUD_SDK_SK, HW_ACCESS_KEY, HW_SECRET_KEY
  • Prefer IAM users over root account for cloud operations
  • Never persist AK/SK in skill files, debug files, reports, or shell history

Configuration Method (Environment Variables Only):

export HUAWEI_AK=<your-ak>
export HUAWEI_SK=<your-sk>
export HUAWEI_PROJECT_ID=<your-project-id>

3. IAM Permission Requirements

Permission Failure Handling: When any command fails due to IAM permission errors, verify the permissions listed above, guide the user to create custom IAM policies, and pause execution until permissions are confirmed.

Core Commands

All tools are invoked through python3 scripts/huawei-cloud.py with key=value parameters.

1. Precheck (Read-Only)

python3 scripts/huawei-cloud.py huawei_precheck_cce_cci_bursting region=cn-north-4 cluster_id=<cluster-id> vpcep_subnet_id=<vpc-subnet-id>

Inspects cluster networking, resolves subnet roles, checks addon state, and reports any blocking issues. The cluster must be a Turbo/ENI cluster for CCI bursting.

2. Ensure VPCEP Dependencies

# Preview (no mutation)
python3 scripts/huawei-cloud.py huawei_ensure_cce_cci_vpcep region=cn-north-4 cluster_id=<cluster-id> vpcep_subnet_id=<vpc-subnet-id>

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_ensure_cce_cci_vpcep region=cn-north-4 cluster_id=<cluster-id> vpcep_subnet_id=<vpc-subnet-id> confirm=true

Creates or reuses SWR, SWR-API, and OBS-compatible VPCEP interface endpoints in the cluster VPC. If OBS information is missing, pass obs_endpoint_service_name obtained from the Huawei Cloud service ticket (do not guess).

3. Setup CCI Bursting

# Preview
python3 scripts/huawei-cloud.py huawei_setup_cce_cci_bursting region=cn-north-4 cluster_id=<cluster-id> vpcep_subnet_id=<vpc-subnet-id>

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_setup_cce_cci_bursting region=cn-north-4 cluster_id=<cluster-id> vpcep_subnet_id=<vpc-subnet-id> confirm=true

Ensures VPCEP dependencies, installs virtual-kubelet if absent, and configures CCI network parameters. Idempotent: updates existing addon configuration without uninstalling.

4. Verify Bursting Readiness (Read-Only)

python3 scripts/huawei-cloud.py huawei_verify_cce_cci_bursting region=cn-north-4 cluster_id=<cluster-id>

Checks addon state, virtual node readiness, and optional workload pod status. Returns ready=true when the virtual node is Ready and all workload pods are Running on it.

5. Deploy Smoke Workload

# Preview
python3 scripts/huawei-cloud.py huawei_deploy_cce_cci_smoke_workload region=cn-north-4 cluster_id=<cluster-id> replicas=2

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_deploy_cce_cci_smoke_workload region=cn-north-4 cluster_id=<cluster-id> replicas=2 confirm=true

Creates or patches a small Deployment forced onto CCI capacity using the bursting.cci.io/burst-to-cci: enforce label. Uses a regional SWR image by default.

6. Verify Smoke Workload (Read-Only)

python3 scripts/huawei-cloud.py huawei_verify_cce_cci_bursting region=cn-north-4 cluster_id=<cluster-id> namespace=cci2-burst-lab workload_name=cci2-burst-demo

Confirms test pods reach Running on the virtual node (bursting-node or virtual-kubelet).

7. Check Node Capacity (Read-Only)

python3 scripts/huawei-cloud.py huawei_check_cce_cci_node_capacity region=cn-north-4 cluster_id=<cluster-id>

Inspects physical-node addon headroom. The precheck includes NodeCheck automatically, warning when schedulable capacity is below the 2C/4GiB conservative baseline. Preview node pool expansion when the baseline is not met.

8. List Node Pools (Read-Only)

python3 scripts/huawei-cloud.py huawei_list_cce_nodepools region=cn-north-4 cluster_id=<cluster-id>

Lists CCE node pools in the cluster. Used after NodeCheck warns about insufficient headroom to identify existing pools for resize.

9. Resize Node Pool

# Preview
python3 scripts/huawei-cloud.py huawei_resize_cce_nodepool region=cn-north-4 cluster_id=<cluster-id> nodepool_id=<nodepool-id> node_count=3

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_resize_cce_nodepool region=cn-north-4 cluster_id=<cluster-id> nodepool_id=<nodepool-id> node_count=3 confirm=true

Resizes an existing node pool to increase physical-node capacity. Apply only after explicit user approval with confirm=true.

10. Create Node Pool

# Preview
python3 scripts/huawei-cloud.py huawei_create_cce_nodepool region=cn-north-4 cluster_id=<cluster-id> nodepool_name=<name> flavor=<flavor> availability_zone=<az> root_volume_size=40 root_volume_type=SSD

# Apply after user approval
python3 scripts/huawei-cloud.py huawei_create_cce_nodepool region=cn-north-4 cluster_id=<cluster-id> nodepool_name=<name> flavor=<flavor> availability_zone=<az> root_volume_size=40 root_volume_type=SSD confirm=true

Creates a new node pool. Prefer SSH keypair authentication. Apply only after explicit user approval with confirm=true.

11. Discover Smoke Images (Read-Only)

python3 scripts/huawei-cloud.py huawei_discover_cce_cci_smoke_images region=cn-north-4

Discover tenant-owned SWR basic images through namespace, repository, and tag queries. The smoke deployment automatically selects a discovered tenant-owned image when image is omitted.

12. Diagnose Bursting Addon (Read-Only)

python3 scripts/huawei-cloud.py huawei_diagnose_cce_cci_bursting_addon region=cn-north-4 cluster_id=<cluster-id>

Returns read-only addon diagnostics when verification fails or the virtual node does not become Ready. Inspect the returned diagnostics to identify region mismatches, missing project IDs, or configuration issues.

Parameter Reference

Common Parameters

Precheck Parameters

VPCEP Parameters

Setup Parameters

Smoke Workload Parameters

Verify Parameters

Subnet Role Reference

These are different ID namespaces. Never swap them. For a Turbo/ENI cluster, huawei_precheck_cce_cci_bursting resolves cci_subnet_id from spec.eni_network. Pass vpcep_subnet_id explicitly when a dedicated endpoint subnet is preferred.

Output Format

All tools return JSON with the following structure:

Precheck Output:

Verify Output:

Verification

Step-by-step Verification Checklist

1. Verify AK/SK credentials are configured via environment variables
2. Run huawei_precheck_cce_cci_bursting and confirm issues is empty
3. Verify the cluster is Turbo/ENI type (container_network_mode = eni)
4. Run huawei_ensure_cce_cci_vpcep preview and confirm VPCEP plan is correct
5. After user approval, apply VPCEP with confirm=true
6. Run huawei_setup_cce_cci_bursting preview, then apply with confirm=true
7. Run huawei_verify_cce_cci_bursting and confirm ready=true
8. Deploy smoke workload with preview, then confirm=true
9. Run final verification with namespace and workload_name parameters
10. Confirm all pods are Running on bursting-node or virtual-kubelet

Best Practices

1. Always precheck first: Run huawei_precheck_cce_cci_bursting before any mutation to identify blocking issues
2. Preview before apply: Always run mutation actions without confirm=true first, review the plan, then re-run with confirm=true after explicit user approval
3. Never swap subnet IDs: cci_subnet_id (Neutron UUID) and vpcep_subnet_id (VPC UUID) are different ID namespaces; swapping them causes addon failure
4. Prefer tenant-owned SWR images: Docker Hub images and public namespace images timeout in CCI capacity; prefer a tenant-owned regional SWR image via huawei_discover_cce_cci_smoke_images; public namespace images are fallback-only
5. Obtain OBS service name from service ticket: Never guess the obs_endpoint_service_name from a similar regional public service
6. Verify after each change: Run huawei_verify_cce_cci_bursting after each applied change to confirm progress
7. Setup is idempotent: huawei_setup_cce_cci_bursting updates existing addon configuration without uninstalling; safe to re-run
8. Reuse existing VPCEPs: The tool reuses accepted VPCEPs in the cluster VPC; no duplicate creation
9. Do not delete resources automatically: Never auto-delete VPCEPs, namespaces, workloads, addons, or addon ReplicaSets
10. Diagnose addon failures: When verification fails, run huawei_diagnose_cce_cci_bursting_addon for read-only diagnostics; never patch the internal bursting-status ConfigMap automatically
11. Node headroom is a warning: The 2C/4GiB NodeCheck is a conservative small-cluster warning, not a platform hard limit; size production resources with the official addon formula
12. Explicit project ID when needed: Pass project_id explicitly when IAM auto-resolution is unavailable

Reference Documents

Notes

• Preview-first by design — VPCEP creation, addon installation, node pool changes, and workload deployment return a preview without confirm=true; apply only after explicit user approval
• Idempotent setup — huawei_setup_cce_cci_bursting may update the existing virtual-kubelet addon configuration but never uninstalls it; it resolves and writes the regional project ID
• Turbo/ENI required — CCE to CCI bursting requires a Turbo cluster with ENI container network mode
• No credential persistence — AK/SK exists only during API calls; never written to disk, logs, or reports
• Cross-skill escalation — If bursting pods show CrashLoopBackOff or ImagePullBackOff, hand off to huawei-cloud-cce-pod-failure-diagnoser; if VPCEP connectivity fails, hand off to huawei-cloud-cce-network-failure-diagnoser
• Tenant-owned smoke images — Prefer a tenant-owned SWR image discovered via huawei_discover_cce_cci_smoke_images; public namespace images are fallback-only because CCI pulling through VPCEP may fail
• NodeCheck warning baseline — The 2C/4GiB value is a conservative small-cluster warning; size production addon resources with the official formula

Common Pitfalls