ArgoCD GitOps Deployment Blueprint

ArgoCD GitOps Deployment Blueprint

GitOps with ArgoCD sounds simple until you realize that a git merge to the deployment repository does not mean the cluster is in the desired state — it means ArgoCD will eventually try to reconcile. The gap between "merged" and "deployed" is where incidents live. At an enterprise I consulted for, a team merged a Kubernetes manifest with a typo in the resource limits (100m CPU instead of 1000m). ArgoCD synced it immediately, the pods started OOMKilling, and nobody noticed for 40 minutes because they assumed "merged means deployed successfully." This template adds sync validation, health checks, and drift alerting.

This GitOps pipeline combines a CI workflow (GitHub Actions or GitLab CI) that builds and tests the application with ArgoCD managing the deployment side. The CI pipeline updates manifests in a separate GitOps repo; ArgoCD detects the change and reconciles.

Pipeline Stages

• CI: build-and-test — Standard CI pipeline: lint, test, build container image, scan with Trivy, push to registry with digest tag.
• CI: update-manifests — Uses kustomize edit set image app=registry/app@sha256:abc123 to update the GitOps repo manifest with the new image digest. Commits to the GitOps repo's dev branch.
• ArgoCD: auto-sync dev — ArgoCD Application for dev is configured with automated: { prune: true, selfHeal: true }. Syncs within 3 minutes of the manifest change. Health check validates pod readiness.
• CI: promote-staging — After dev integration tests pass, the CI pipeline opens a PR from dev to staging branch in the GitOps repo. Manual review required.
• ArgoCD: sync staging — ArgoCD Application for staging with automated: { prune: false, selfHeal: false }. Requires manual sync click or CLI command after PR merge. Sync wave annotations control resource ordering.
• CI: promote-prod — PR from staging to main in the GitOps repo. Two required reviewers. Includes the diff of all Kubernetes manifests in the PR description.
• ArgoCD: sync prod — Manual sync with --prune=false for safety. Progressive delivery via Argo Rollouts: canary at 10%, analysis at 50%, full rollout at 100%. Automatic rollback on analysis failure.

Security Gates

• Image digest pinning — Manifests always reference images by SHA256 digest, never by mutable tag. ArgoCD verifies the digest at sync time.
• OPA/Kyverno admission — Cluster admission controller validates that synced resources meet security policies: no privileged containers, resource limits required, approved registries only.
• Git commit signing — Manifest changes in the GitOps repo require signed commits. ArgoCD can be configured to verify commit signatures before syncing.
• RBAC per Application — ArgoCD RBAC restricts which teams can sync which Applications. The frontend team cannot sync backend Applications.

What Breaks First

• ArgoCD sync stuck in "Progressing" — A deployment's readiness probe fails because the new image crashes on startup. ArgoCD shows "Progressing" indefinitely. Fix: set timeout.reconciliation in the Application spec and configure health check with progressDeadlineSeconds on the Deployment.
• Manifest drift from kubectl edit — Someone runs kubectl edit directly, causing ArgoCD to show "OutOfSync." With selfHeal: true, ArgoCD reverts the change, potentially undoing an emergency fix. Fix: disable selfHeal in production and use the GitOps repo for all changes, including emergency patches.
• Sync wave ordering failure — CRDs in wave 0, operators in wave 1, custom resources in wave 2. If the operator is not ready before wave 2 starts, the custom resources fail to apply. Fix: add health checks to the operator Application that verify CRD registration before proceeding.