Added bootstrap folder and gateway controller
This commit is contained in:
275
cluster/bootstrap/README.md
Normal file
275
cluster/bootstrap/README.md
Normal file
@@ -0,0 +1,275 @@
|
||||
# fastpass — Cluster Bootstrap
|
||||
|
||||
This directory contains resources and documentation for the **manual bootstrap
|
||||
sequence** required to bring up `fastpass` from a fresh Talos install.
|
||||
|
||||
Everything in this directory is applied **once by hand** before ArgoCD takes
|
||||
over. After Step 6 (app-of-apps), all further changes are made via Git.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- All 6 Talos nodes `Ready` (`kubectl get nodes`)
|
||||
- Cilium healthy (`cilium status --wait`)
|
||||
- IP pools applied (`kubectl get ciliumloadbalancerippool`)
|
||||
- `city-hall` has `helm`, `kubectl`, and `talosctl` installed
|
||||
- SSH keypair for ArgoCD → Gitea exists at `/home/wed/.ssh/argocd_gitea`
|
||||
- 1Password credentials file at `/home/wed/1password-credentials.json`
|
||||
- 1Password Connect token at `/home/wed/connect-token` (no trailing newline)
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Install Gateway API CRDs
|
||||
|
||||
The Gateway API CRDs are not bundled with Cilium — they must be installed
|
||||
independently before Cilium can register the GatewayClass controller.
|
||||
|
||||
```bash
|
||||
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
kubectl get crd gateways.gateway.networking.k8s.io
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Create the Cilium GatewayClass
|
||||
|
||||
The Cilium operator manages this object but does not auto-create it.
|
||||
|
||||
```bash
|
||||
kubectl apply -f cluster/bootstrap/gatewayclass.yaml
|
||||
```
|
||||
|
||||
Verify (may take a few seconds):
|
||||
|
||||
```bash
|
||||
kubectl get gatewayclass
|
||||
# Expected: cilium io.cilium/gateway-controller True
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Create the argocd namespace
|
||||
|
||||
Must be created before Helm — the Gitea repo secret targets this namespace.
|
||||
|
||||
```bash
|
||||
kubectl apply -f cluster/argocd/namespace.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — Create the Gitea repo secret
|
||||
|
||||
```bash
|
||||
kubectl create secret generic gitea-repo \
|
||||
--namespace argocd \
|
||||
--from-literal=type=git \
|
||||
--from-literal=url=git@gitea.mk-labs.cloud:rblundon/homelab.git \
|
||||
--from-file=sshPrivateKey=/home/wed/.ssh/argocd_gitea
|
||||
|
||||
kubectl label secret gitea-repo -n argocd \
|
||||
argocd.argoproj.io/secret-type=repository
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
kubectl -n argocd get secret gitea-repo
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — Install ArgoCD via Helm
|
||||
|
||||
```bash
|
||||
helm repo add argo https://argoproj.github.io/argo-helm
|
||||
helm repo update
|
||||
|
||||
helm upgrade --install argocd argo/argo-cd \
|
||||
--namespace argocd \
|
||||
--create-namespace \
|
||||
--version 7.8.23 \
|
||||
--values cluster/argocd/values.yaml \
|
||||
--wait
|
||||
```
|
||||
|
||||
Wait for all ArgoCD pods to be ready:
|
||||
|
||||
```bash
|
||||
kubectl -n argocd get pods --watch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 6 — Add Gitea to ArgoCD known hosts
|
||||
|
||||
```bash
|
||||
ssh-keyscan -t ed25519 gitea.mk-labs.cloud 2>/dev/null | \
|
||||
kubectl patch configmap argocd-ssh-known-hosts-cm -n argocd \
|
||||
--type merge \
|
||||
-p "{\"data\":{\"ssh_known_hosts\": \"$(ssh-keyscan -t ed25519 gitea.mk-labs.cloud 2>/dev/null)\"}}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 7 — Get the initial admin password
|
||||
|
||||
```bash
|
||||
kubectl -n argocd get secret argocd-initial-admin-secret \
|
||||
-o jsonpath='{.data.password}' | base64 -d; echo
|
||||
```
|
||||
|
||||
Access the UI via port-forward (ingress not yet deployed):
|
||||
|
||||
```bash
|
||||
kubectl port-forward svc/argocd-server -n argocd 8080:80 > /tmp/pf.log 2>&1 &
|
||||
```
|
||||
|
||||
Browse to http://localhost:8080
|
||||
|
||||
---
|
||||
|
||||
## Step 8 — Apply the app-of-apps
|
||||
|
||||
This is the final manual `kubectl apply`. It hands control to ArgoCD:
|
||||
|
||||
```bash
|
||||
kubectl apply -n argocd -f cluster/argocd/apps-of-apps.yaml
|
||||
```
|
||||
|
||||
From this point, all changes are made via Git. ArgoCD syncs `cluster/platform/`
|
||||
and `cluster/applications/` automatically.
|
||||
|
||||
Watch the sync:
|
||||
|
||||
```bash
|
||||
kubectl -n argocd get applications --watch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 9 — Approve CSRs
|
||||
|
||||
```bash
|
||||
kubectl get csr --no-headers | awk '{print $1}' | xargs kubectl certificate approve
|
||||
```
|
||||
|
||||
Run periodically until no pending CSRs remain.
|
||||
|
||||
---
|
||||
|
||||
## Step 10 — Bootstrap 1Password Connect secrets
|
||||
|
||||
Once `onepassword-connect` namespace exists (created by ArgoCD sync):
|
||||
|
||||
```bash
|
||||
# Verify namespace exists
|
||||
kubectl get namespace onepassword-connect
|
||||
|
||||
# Credentials secret — must be base64-encoded
|
||||
kubectl create secret generic op-credentials \
|
||||
--namespace onepassword-connect \
|
||||
--from-literal=1password-credentials.json=$(base64 -w 0 /home/wed/1password-credentials.json)
|
||||
|
||||
# Token secret — file must have no trailing newline
|
||||
kubectl create secret generic connect-token \
|
||||
--namespace onepassword-connect \
|
||||
--from-file=token=/home/wed/connect-token
|
||||
```
|
||||
|
||||
Restart Connect to pick up secrets:
|
||||
|
||||
```bash
|
||||
kubectl -n onepassword-connect rollout restart deployment onepassword-connect
|
||||
kubectl -n onepassword-connect rollout status deployment onepassword-connect
|
||||
```
|
||||
|
||||
Verify (should show only GET /health and GET /heartbeat 200s):
|
||||
|
||||
```bash
|
||||
kubectl -n onepassword-connect logs deployment/onepassword-connect -c connect-api 2>&1 | tail -10
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected platform sync order
|
||||
|
||||
| Wave | App |
|
||||
|------|----------------------|
|
||||
| 0 | apps-of-apps (root) |
|
||||
| 1 | external-secrets |
|
||||
| 1 | onepassword-connect |
|
||||
| 2 | nfs-csi |
|
||||
| 3 | cert-manager |
|
||||
| 4 | ingress-nginx |
|
||||
| 5 | external-dns |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**GatewayClass not ACCEPTED after applying:**
|
||||
Restart the Cilium operator — it may have started before the CRDs were present:
|
||||
```bash
|
||||
kubectl rollout restart deployment/cilium-operator -n kube-system
|
||||
kubectl get gatewayclass
|
||||
```
|
||||
|
||||
**ArgoCD can't reach Gitea:**
|
||||
```bash
|
||||
kubectl -n argocd logs deployment/argocd-repo-server | grep -i "error\|ssh"
|
||||
```
|
||||
Check that `gitea-repo` secret exists and known hosts configmap has `gitea.mk-labs.cloud`.
|
||||
|
||||
**App stuck in `Unknown` state:**
|
||||
```bash
|
||||
kubectl -n argocd get application <name> -o yaml | grep -A10 "conditions"
|
||||
```
|
||||
|
||||
**ArgoCD pruning itself:**
|
||||
The `argocd` namespace Applications must have `prune: false`. Never enable
|
||||
prune on any Application that manages the `argocd` namespace.
|
||||
|
||||
**Helm valueFiles field:**
|
||||
The correct field name is `valueFiles` (plural). `valuesFile` and `valuesFiles`
|
||||
are silently ignored — ArgoCD syncs green but values aren't applied.
|
||||
|
||||
**1Password Connect base64 error:**
|
||||
`illegal base64 data at input byte 0` means credentials were stored as raw JSON.
|
||||
Delete and recreate:
|
||||
```bash
|
||||
kubectl delete secret op-credentials -n onepassword-connect
|
||||
kubectl create secret generic op-credentials \
|
||||
--namespace onepassword-connect \
|
||||
--from-literal=1password-credentials.json=$(base64 -w 0 /home/wed/1password-credentials.json)
|
||||
kubectl -n onepassword-connect rollout restart deployment onepassword-connect
|
||||
```
|
||||
|
||||
**1Password Connect invalid Authorization header:**
|
||||
Token has a trailing newline. Recreate:
|
||||
```bash
|
||||
echo -n "your-token" > /home/wed/connect-token
|
||||
kubectl delete secret connect-token -n onepassword-connect
|
||||
kubectl create secret generic connect-token \
|
||||
--namespace onepassword-connect \
|
||||
--from-file=token=/home/wed/connect-token
|
||||
kubectl -n onepassword-connect rollout restart deployment onepassword-connect
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Post-bootstrap
|
||||
|
||||
Once ingress-nginx and cert-manager are synced:
|
||||
|
||||
- ArgoCD UI: https://argocd.local.mk-labs.cloud
|
||||
- Change admin password: `argocd account update-password`
|
||||
- Authentik OIDC: uncomment `oidc.config` in `cluster/argocd/values.yaml` once
|
||||
the ArgoCD provider is configured in Authentik
|
||||
- Migrate `gitea-repo` secret to ESO once 1Password Connect is healthy
|
||||
Reference in New Issue
Block a user