diff --git a/docs/superpowers/specs/2026-07-15-argocd-offline-bootstrap-design.md b/docs/superpowers/specs/2026-07-15-argocd-offline-bootstrap-design.md new file mode 100644 index 0000000..cf5c0e5 --- /dev/null +++ b/docs/superpowers/specs/2026-07-15-argocd-offline-bootstrap-design.md @@ -0,0 +1,78 @@ +# Argo CD Offline Bootstrap + +## Goal + +Make Argo CD on the demo k3s server `192.168.1.173` able to start and reconcile from locally cached images while Harbor `192.168.1.119:8888` is offline, and provide a version-controlled installation path that preserves this behavior across server restarts, Argo CD reinstalls, and upgrades. + +## Current State and Root Cause + +Argo CD was installed with `kubectl apply`, then its images were redirected to Harbor. Most Argo CD containers still use `imagePullPolicy: Always`. When their pod sandboxes are recreated, kubelet contacts Harbor even though the exact images are cached locally. With Harbor offline, image resolution fails with `no route to host`, leaving the Argo CD control plane in `ErrImagePull` or `ImagePullBackOff`. + +The following exact images are already cached on `192.168.1.173`: + +- `192.168.1.119:8888/argoproj/argocd:v3.4.4` +- `192.168.1.119:8888/dexidp/dex:v2.45.0` +- `192.168.1.119:8888/library/redis:8.2.3-alpine` + +## Repository Design + +Create `bootstrap/argocd-offline/` in `mixingx.deploy` with: + +- A vendored copy of the official Argo CD `v3.4.4` install manifest, pinned to that release rather than `stable`. +- A `kustomization.yaml` that rewrites the Argo CD, Dex, and Redis images to the Harbor references above. +- Kustomize patches that set every Argo CD workload container and init container to `imagePullPolicy: IfNotPresent`. +- An idempotent `apply.sh` entry point for the demo k3s server. +- A README describing installation, verification, offline recovery, and upgrade rules. + +Vendoring the upstream manifest makes the bootstrap independent of GitHub availability. The supported install and upgrade path becomes: + +```bash +sudo ./apply.sh +``` + +Operators must not apply an unmodified upstream Argo CD manifest directly, because doing so can restore `Always` policies. + +## Apply Script Behavior + +The script will: + +1. Require root privileges and verify `k3s` and its embedded `kubectl` are available. +2. Render the local Kustomize overlay before making changes. +3. Reject the render if any workload container or init container still uses `imagePullPolicy: Always`. +4. Verify all three required image references exist in the local k3s containerd image store. +5. Apply the rendered resources to namespace `argocd`. +6. Wait for the Argo CD Deployments and application-controller StatefulSet to complete rollout. +7. Print pod state and fail if any Argo CD pod remains in an image-pull error. + +The script must not pull, retag, delete, or garbage-collect images. Missing cached images are a hard stop so the operator can restore Harbor or import the required image deliberately. + +## Live Recovery + +After repository validation, run the versioned bootstrap against `192.168.1.173`. Applying the overlay changes the pod templates from `Always` to `IfNotPresent`, causing Argo CD pods to restart using the exact locally cached images. No Harbor access is required for this recovery. + +Verify all Argo CD components become Ready and the existing MixingX Applications remain present. The bootstrap must not delete Applications or application workloads. + +## GitOps Publication + +1. Commit and push the `mixingx.deploy` `master` branch. +2. Update the `mixingx.hub` `mixingx-deploy` submodule pointer on `develop/v.1.0.1`. +3. Commit and push the Hub branch without staging existing `.zcode/plans/` files. + +## Upgrade Procedure + +For an Argo CD upgrade: + +1. Replace the vendored manifest with the official manifest for an exact release tag. +2. Update the three expected image versions and Harbor rewrites. +3. Render locally and verify there are no `Always` policies or unexpected external image references. +4. Ensure the new images are present in Harbor and cached or explicitly imported on `192.168.1.173` before application. +5. Apply only through the versioned bootstrap directory and verify rollouts. + +## Verification + +- The Kustomize render contains the expected Argo CD `v3.4.4`, Dex `v2.45.0`, and Redis `8.2.3-alpine` Harbor references. +- Every Deployment and StatefulSet container/initContainer has `imagePullPolicy: IfNotPresent`. +- No application container image is rewritten or modified. +- All Argo CD pods on `192.168.1.173` become Ready while Harbor remains unavailable. +- Existing Argo CD Applications remain intact. +- Repository branches and the Hub submodule pointer are synchronized with their remotes.