Imported from sca-templates/infra-kubernetes (
AGENTS.md). Install upstream withnpx skills add sca-templates/infra-kubernetes. Copyright stays with the author.
infra-kubernetes — Platform Guide
GitOps source of truth for the sca infrastructure platform on Kubernetes.
ArgoCD reconciles this repository: nothing is deployed by hand — every
change lands through git. The Docker Compose sibling repositories
(sca-templates/infra-*) remain the local-dev counterpart only, outside this
repo.
1. What this repo is
- Single source of truth for the platform components: environments, secret
projections. Services do not live here — each service owns its
deploy/in its own repo (app-repo-as-source, docs/onboarding-new-service.md). - ArgoCD app-of-apps: one root Application per environment renders two
ApplicationSets —
apps-<env>.yaml(platform catalog) andservices-<env>.yaml(services registry; present fordev/qa/prodonly, neverlocal). - A clean restart of the previous
infra-kubernetes(which churned infixcommits): a human-reviewed gate per logical change, rollback over forward-fix. - A template: it must be clonable standalone. Zero references to local paths outside the repo; sibling knowledge is always an external link.
Knowledge base: this guide is the rules of engagement; the file-accurate
documentation set lives in docs/ (start at docs/INDEX.md).
From Phase 1 onward the docs are the source of truth: phase prompts read
them first, execute against their gates, and update status.md /
architecture.md as components land. Prefer the docs over recollection.
2. Environment model
local is a platform-only sandbox (kind): the full catalog for iteration
and CI smoke, but no services. Services deploy from dev upward: dev/qa
promoted by syncing the ArgoCD Application of their own repo from the service
promote workflow, prod by bumping the version tag pin in the services
registry (chore(services) PR = go/no-go).
| Environment | Profile | Sync policy | Purpose |
|---|---|---|---|
local |
1 replica, full platform catalog, minimal resources | auto-sync + prune | Developer machine (kind); platform-only, no services |
dev |
reduced HA | auto-sync + prune (platform); services synced on demand by the promote workflow |
Shared integration; services via the promote workflow → <service>-dev app sync |
qa |
HA (3 replicas, PDBs, anti-affinity) | auto-sync, no prune (platform); services synced by the promote workflow against the qa approval gate |
Pre-production validation; services via the promote workflow → <service>-qa app sync |
prod |
full HA, real storage | manual sync | Production; services via the version pin in services-prod.yaml, go/no-go |
Sync policies follow ADR-003. Platform-component promotion is gated by the
promote-test; services promote by ArgoCD app syncs (dev/qa) and a verified
version bump (prod), and a service PR to its main only happens after dev
and qa (docs/workflow.md).
3. Component catalog
17 components + ArgoCD. Status column is the source of truth
(planned (Phase N) until the phase lands; deployed from then on). Chart
and image pins are set inside each phase and live under
infrastructure/<component>/ + envs/<env>/.
| Component | Namespace | Upstream chart | Role | Provenance | Status |
|---|---|---|---|---|---|
| cert-manager | cert-manager |
jetstack/cert-manager | In-cluster TLS issuers | — | deployed (Phase 1) |
| Vault | vault |
hashicorp/vault | Secrets SSOT (KV-v2 + k8s auth) | infra-vault | deployed (Phase 2) |
| External Secrets | external-secrets |
external-secrets/external-secrets | Projects Vault KV → native Secrets | — | deployed (Phase 3) |
| linkerd-crds | linkerd |
linkerd/linkerd-crds | Mesh CRDs | infra-linkerd | deployed (Phase 4) |
| CloudNativePG | cloudnative-pg |
cloudnative-pg/cloudnative-pg | PostgreSQL operator | — | planned (Phase 5) |
| Strimzi | strimzi |
strimzi/strimzi-kafka-operator | Kafka operator (KRaft) | infra-kafka | planned (Phase 6) |
| redis-operator | data |
ot-container-kit/redis-operator | Redis operator | infra-redis | planned (Phase 7) |
| Kong | kong |
kong/kong (DB-less) | Edge gateway, dedicated Application, no SSA | infra-kong | planned (Phase 8) |
| Linkerd control plane | linkerd |
linkerd/linkerd2 (script) | mTLS identity, golden signals | infra-linkerd | planned (Phase 9) |
| postgres-app (+ keycloak-db) | data |
CNPG Cluster CRs (raw) |
App + Keycloak databases; local only | infra-postgres | planned (Phase 10) |
| Kafka | data |
Kafka/KafkaNodePool CRs |
Event backbone, SCRAM | infra-kafka | planned (Phase 11) |
| Redis | data |
Redis CR |
Cache | infra-redis | planned (Phase 12) |
| Keycloak | keycloak |
bitnami/keycloak | OIDC IdP, realm sca |
infra-keycloak | planned (Phase 13) |
| kube-prometheus-stack | observability |
prometheus-community/kube-prometheus-stack | Metrics + Grafana + Alertmanager | infra-prometheus | planned (Phase 14) |
| Loki | loki |
grafana/loki | Log aggregation | infra-loki | planned (Phase 15) |
| Alloy | observability |
grafana/alloy | Ships pod logs to Loki | — | planned (Phase 15) |
| Tempo | tempo |
grafana/tempo | Distributed tracing (OTLP) | infra-tempo | planned (Phase 16) |
| MinIO | minio |
minio/minio | S3 stand-in (local): Velero | — | planned (Phase 17) |
| Velero | velero |
vmware-tanzu/velero | Cluster backup/restore | infra-velero | planned (Phase 18) |
| ArgoCD | argocd |
argoproj/argo-cd (Makefile) | GitOps engine | — | deployed (Phase 0.0) |
Intentional exclusions (per the restart plan): no Consul in-cluster
(ADR-001; native DNS + Linkerd replace it), no Unleash, no
KafkaConnect/Debezium, no kafka-ui, no linkerd-viz in-cluster. No latest
tags; this repo builds nothing. Services are not part of this catalog —
each service owns its deploy/ and is registered in argocd/services-<env>.yaml
(docs/onboarding-new-service.md).
4. Dependency order (sync-waves)
ArgoCD sync-wave annotations, integers spaced ≥10 apart. Operators/CRDs
before CRs; Vault before ESO syncs; datastores before consumers.
| Wave | Content |
|---|---|
| -30 | Root Application per env |
| -20 | ApplicationSet, namespaces, cert-manager (+CRDs) |
| -10 | Pure operators/CRDs: external-secrets, linkerd-crds, cloudnative-pg, strimzi, redis-operator |
| 0 | Vault |
| 10 | Vault seeded by bootstrap/seed-vault.sh after pod Running (CI/manual, not an ArgoCD hook) |
| 20 | Kong |
| 30 | Linkerd control plane (deployed by script) |
| 40 | Datastores: postgres-app, keycloak-db (CNPG), kafka, redis CRs |
| 50 | Consumers: keycloak |
| 60 | Observability: kube-prometheus-stack, loki, alloy, tempo |
| 60 | Service apps (registry services-<env>.yaml) — consumers of the catalog, wave set per service |
| 70 | MinIO (local only) |
| 80 | Velero |
The normative map with rationale lives in docs/architecture.md.
5. Image policy (strict)
Infrastructure uses official upstream images pinned by version. This repo
builds nothing. No floating tags, no latest, ever — enforced by the CI pin
guards (docs/security.md).
6. Commands
| Command | What it does |
|---|---|
make prereqs |
Install pinned kubectl/helm/kind into ~/.local/bin (idempotent, sha256-verified) — bootstrap/sca.sh prerrequisites |
make doctor |
Read-only platform health check (toolchain, PATH, docker, cluster, ArgoCD apps, .env seam, git source) |
make install-cli |
Symlink bootstrap/sca.sh → ~/.local/bin/sca for direct sca prereqs / sca doctor / sca version |
make cluster-up |
Create the local kind cluster from bootstrap/kind-config.yaml |
make cluster-down |
Delete the local kind cluster (keeps nothing) |
make bootstrap |
Install ArgoCD + apply the root Application for $ENV |
make status |
Cluster, ArgoCD apps and pod health overview |
make validate |
Static suite + live cluster checks |
make validate-static |
Static suite only: markdownlint, yamllint, YAML parse, bash -n |
make port-forward APP=<name> |
Reach a platform UI/API locally (argocd, vault, keycloak, grafana, prometheus, …) |
make clean |
Remove local state (.env, .secrets/, generated artifacts) |
The Makefile is a thin wrapper: after bootstrap, deployments happen
exclusively via git push → ArgoCD.
OS support: Linux (any distro) and macOS for the toolchain; Windows is
only supported via WSL2 (this platform is POSIX — no native Windows shell
support; sca fails fast with a pointer to WSL2).
7. Conventions (strict)
- English only: content, commits, PR descriptions.
- Conventional commits:
feat(platform): …,feat(vault): …,docs(readme): …. - Changes land via reviewed PRs. A component often ships as a single PR,
but the number of PRs and commits is a judgment call driven by the change —
the work is dynamic and grouping is not a rule. A blocked change rolls back
rather than being patched forward with
fixchains; nothing is deployed by hand aftermake bootstrap. - Changes land through PRs (initial population excepted, straight to
main). - Git writes are the user's: do not
git commit/git push/reset/amend or stage anything on your own. Draft commits/messages only; ask before any git mutation. The user commits and pushes. - Never commit
.env,.secrets/, kubeconfigs, unseal keys, tokens, dumps. Vault is the secrets SSOT;.secrets/lives only on disk. - Shared values changed must be mirrored across all
envs/<env>/<c>.yaml. - Respect sync-waves: a new component gets a wave ≥10 apart from its dependencies.
- From Phase 1, the commit that lands a component also updates
docs/roadmap.md(Work Log row) anddocs/architecture.md(Status, deviations log) /docs/status.md. Docs are updated as components land. The roadmap is tracked on the GitHub Projects board, Milestones and Issues and linked fromdocs/roadmap.md. - Consult the sca-docs notes below before touching topology, ports or networks.
8. Documentation to consult (always)
Fetch from the sca-docs vault (raw URLs, no local checkout assumed):
00-ecosystem/platform-overview.md— ecosystem vision00-ecosystem/conventions.md— naming, links, catalogs06-decisions/adr-001…adr-007— locked architecture decisions (ADR-001 no in-cluster Consul, ADR-003 sync policies)04-infrastructure/<component>.md— per-component canonical notes
Base URL: https://raw.githubusercontent.com/sca-templates/sca-docs/main/<path>
9. Troubleshooting quick refs
| Symptom | Probable cause | Fix |
|---|---|---|
ArgoCD app OutOfSync |
Drift or failed sync-wave | kubectl get application <name> -n argocd; check status.conditions; sync manually if prod |
ExternalSecret SecretSyncedError |
Vault path missing or k8s-auth role wrong | Check ClusterSecretStore vault Ready; verify path under secret/<service>/<env>; re-run bootstrap/seed-vault.sh (local) |
| Pods CrashLoopBackOff after a wave bump | Dependency started before its datastore/operator | Re-check wave assignment; consumers must be ≥10 above their dependency |
| kind cluster OOM | Host RAM exhausted | Stop the Compose stack before bootstrapping; 1-replica local profile |
ImagePullBackOff |
Bad pin or unreachable registry | Verify the pinned tag exists upstream; check node images; never float latest |
| ESO wedge (local) | Old long-refresh pattern | Do not restart ESO by hand; short refresh (~5 min) prevents recurrence |
Full runbooks live in docs/workflow.md.
CodeGraph
In repositories indexed by CodeGraph (a .codegraph/ directory exists at the repo root), reach for it BEFORE grep/find or reading files when you need to understand or locate code:
- MCP tool (when available):
codegraph_exploreanswers most code questions in one call — the relevant symbols' verbatim source plus the call paths between them, including dynamic-dispatch hops grep can't follow. Name a file or symbol in the query to read its current line-numbered source. If it's listed but deferred, load it by name via tool search. - Shell (always works):
codegraph explore "<symbol names or question>"prints the same output.
If there is no .codegraph/ directory, skip CodeGraph entirely — indexing is the user's decision.