262 Commits

Author SHA1 Message Date
Hermes Agent service account
818b6505dd feat(firecrawl): Add ArgoCD Application manifest for GitOps deployment
- Add application.yaml for Firecrawl ArgoCD management
- Wave 20 (applications tier)
- Automated sync with prune and selfHeal enabled
- Manages all resources in cluster/applications/firecrawl/
- Remediates Day 5 manual deployment (kubectl apply -> GitOps)
2026-06-06 19:09:40 -05:00
Hermes Agent service account
4a1958876f Day 5: Fix worker probes and HTTPRoute gateway reference
- Changed worker deployment probes from HTTP to TCP (port 3005)
  * Worker liveness endpoint doesn't serve HTTP at '/' path
  * TCP socket check more appropriate for background worker
  * Resolves pod restart loop and readiness failures

- Corrected HTTPRoute gateway reference
  * Changed from 'gateway' in 'default' namespace
  * To 'fastpass-gateway' in 'gateway' namespace
  * HTTPRoute now properly accepted by gateway

All 7 deployments Running and Ready. System operational.
2026-06-06 18:49:52 -05:00
Hermes Agent service account
6bdb536848 firecrawl: Day 4 - Add ExternalSecret for 1Password integration
- Add ExternalSecret manifest to sync firecrawl secrets from 1Password
- Configure POSTGRES_PASSWORD and BULL_AUTH_KEY from mk-labs vault
- Add comprehensive SECRETS_SETUP.md documentation
- Verified ExternalSecrets Operator successfully synced secrets
- All 18 deployment manifests validated with dry-run

Status: firecrawl-secrets Secret created and populated correctly
2026-06-06 18:11:59 -05:00
Hermes Agent service account
6023ee25e1 feat(firecrawl): Day 3 - Complete Kubernetes manifests for Firecrawl deployment
- Created PersistentVolumeClaim for PostgreSQL (10GB, nfs-emporium)
- Created ConfigMaps for API and Playwright service configuration
- Created 7 Deployment manifests:
  * firecrawl-api (2 CPU, 4-6GB RAM)
  * firecrawl-api-worker (1 CPU, 3-4GB RAM)
  * firecrawl-api-nuq-worker (1 CPU, 3-4GB RAM)
  * firecrawl-playwright (2 CPU, 4GB RAM, 1GB tmpfs)
  * nuq-postgres (1 CPU, 2GB RAM, 10GB PVC)
  * redis (0.5 CPU, 1GB RAM)
  * rabbitmq (0.5 CPU, 1GB RAM)
- Created 5 ClusterIP Services for inter-service communication
- Created HTTPRoute for external access via Gateway API
  * Primary hostname: spaceship-earth.local.mk-labs.cloud
  * Alias: firecrawl.local.mk-labs.cloud
- All manifests validated with kubectl dry-run=client

Next steps (Day 4): Configure ExternalSecrets for 1Password integration
Next steps (Day 5): Deploy to cluster and verify functionality

Total resources: 8 CPU, 22GB RAM, 10GB storage
2026-06-06 17:43:22 -05:00
Hermes Agent service account
e5d24f557a feat(tekton): Add Firecrawl build pipelines
- firecrawl-api: Multi-stage build (Go + Node.js + Rust)
- firecrawl-playwright: Node.js + Chromium browser automation
- firecrawl-postgres: PostgreSQL 16 with pg_cron extension

All pipelines validated with successful test builds.
Images pushed to Harbor library project.

Day 2 of Firecrawl deployment complete.
2026-06-06 17:36:14 -05:00
Hermes Agent service account
f8e137b67b Tekton Phase 2 Day 3: Complete Harbor authentication and test build
- Added config.json key to harbor-credentials ExternalSecret
  This ensures kaniko can find the Docker auth config at /kaniko/.docker/config.json
  (previously only .dockerconfigjson was present)

- Created test-app-build PipelineRun manifest for validation testing

- Successfully validated end-to-end pipeline:
   git-clone Task deployed and working
   kaniko-build Task deployed and working
   container-build Pipeline deployed and working
   Harbor authentication working with robot account
   Test image built and pushed: the-seas.local.mk-labs.cloud/library/test-app:v1.0.0
   Image digest: sha256:aa143f4a01795a1d307b711108ca0c89f36e00ea38fddb9d7b2febd5fffc46d7

Pipeline test results:
- PipelineRun: test-app-build-005 - SUCCEEDED
- fetch-repository TaskRun - SUCCEEDED
- build-and-push TaskRun - SUCCEEDED

Tekton CI/CD platform is now operational and ready for production workloads.
2026-06-06 16:13:31 -05:00
Hermes Agent service account
76241e75a8 Add Tekton tasks, pipeline and test Dockerfile
- Add git-clone task for repository cloning
- Add kaniko-build task for container image builds
- Add container-build pipeline orchestrating clone + build
- Add harbor-credentials ExternalSecret for innoventions namespace
- Add test-app.Dockerfile for pipeline validation

Day 3 deliverables for Tekton Phase 2
2026-06-06 15:53:57 -05:00
Hermes Agent service account
d5ce6ff96a fix(tekton): Remove unsupported TektonConfig fields 2026-06-05 21:56:35 -05:00
Hermes Agent service account
44b1a2fb33 fix(tekton): Correct ArgoCD repo URL to Gitea 2026-06-05 21:54:42 -05:00
Hermes Agent service account
7dc1999928 feat(tekton): Day 2 - Deploy Tekton Operator and components
- Downloaded Tekton Operator v0.79.1 release manifest
- Created TektonConfig CR enabling all components in innoventions namespace
  - Pipelines v1.13.0 with OCI bundles and custom tasks
  - Triggers v0.36.0 with stable API fields
  - Dashboard v0.69.0 with read-write access
  - Addon components (cluster tasks, templates)
  - Pruner configured (keep 100, daily at 2 AM)
- Created Dashboard HTTPRoute for mission-space.local.mk-labs.cloud
  - Certificate via letsencrypt-prod ClusterIssuer
  - Routes via fastpass-gateway (Cilium Gateway API)
  - Backend: tekton-dashboard service port 9097
- Created ArgoCD Application manifest (wave 8)
  - Automated sync with prune/selfHeal
  - ServerSideApply for CRD compatibility
  - Ignore differences for operator-managed resources

Directory: cluster/platform/tekton/ (functional naming)
Namespace: innoventions (thematic naming)
DNS: mission-space.local.mk-labs.cloud

Ready for deployment to fastpass cluster.
2026-06-05 21:54:02 -05:00
Hermes Agent service account
63d480927d chore(couchdb): switch to production Let's Encrypt certificate
Replace letsencrypt-staging with letsencrypt-prod for trusted certificate.
Staging cert was causing connection resets due to certificate validation failures.
2026-06-05 20:04:38 -05:00
Hermes Agent service account
fa86fa4c9c fix(couchdb): correct ingress hostname to internal DNS zone
Change CouchDB ingress hostname from communicore.mk-labs.cloud to
communicore.local.mk-labs.cloud to align with External-DNS configuration.

CouchDB is an internal service and should use the .local.mk-labs.cloud
zone managed by Technitium DNS, not the public .mk-labs.cloud zone
managed by Cloudflare.

This ensures External-DNS will properly create the A record in the
internal DNS server.
2026-06-05 19:33:39 -05:00
Hermes Agent service account
444b597ade fix(couchdb): add erlangCookie to ExternalSecret and external-dns annotation
- Add erlangCookie field to ExternalSecret from 1Password
- Add external-dns target annotation to ingress (10.1.71.80)
- Completes CouchDB deployment configuration
2026-06-05 19:11:45 -05:00
Hermes Agent service account
c81a9b7704 fix(couchdb): remove invalid syncWaves from Application spec 2026-06-05 18:39:59 -05:00
Hermes Agent service account
6cab6519b1 feat(couchdb): deploy CouchDB for Obsidian sync (communicore) 2026-06-05 18:36:28 -05:00
Hermes Agent service account
ce992ca743 Fix ClusterSecretStore reference in Harbor ExternalSecrets
Changed from 'onepassword-store' to 'onepassword-connect' to match
the actual ClusterSecretStore name in the cluster.
2026-06-04 23:47:53 -05:00
Hermes Agent service account
092d1ac209 Document robot credential management via 1Password and ExternalSecrets
Updated README to reflect the full workflow:
1. Robot account creation via Job
2. Secret regeneration and capture
3. Storage in 1Password (harbor-robot-accounts item)
4. Automatic sync via ExternalSecrets to K8s
2026-06-04 23:44:00 -05:00
Hermes Agent service account
6acf2f9944 Add ExternalSecrets for Harbor robot account credentials
Creates two docker-registry secrets from 1Password:
1. harbor-tekton-robot - for Tekton CI/CD pipeline push/pull
2. harbor-pull-secret - for fastpass cluster image pulls

Both sync from 1Password item 'harbor-robot-accounts' with fields:
- tekton-builder-username / tekton-builder-password
- fastpass-cluster-username / fastpass-cluster-password

Credentials document placed in PKA inbox for manual 1Password entry.
Once stored, ESO will automatically sync and create the secrets.
2026-06-04 23:43:37 -05:00
Hermes Agent service account
8d18f42b2e Update Harbor README with robot accounts documentation
Documents robot account management via GitOps:
- tekton-builder and fastpass-cluster robot accounts
- Manual Job execution (PostSync hooks don't work with multi-source)
- Credential retrieval and storage in 1Password
- ImagePullSecret creation for K8s clusters
2026-06-04 23:38:50 -05:00
Hermes Agent service account
152f10ed8b Add ArgoCD PostSync hook for Harbor robot accounts
Manages robot accounts declaratively via GitOps:
- tekton-builder: push/pull access for CI/CD pipelines
- fastpass-cluster: pull-only access for K8s image pulls

Implementation:
- Kubernetes Job with argocd.argoproj.io/hook: PostSync annotation
- Idempotent: checks if accounts exist before creating
- Uses harbor-credentials ExternalSecret for admin password
- BeforeHookCreation deletion policy for clean reruns

Replaces manual robot account creation via Harbor API.
2026-06-04 23:35:45 -05:00
Hermes Agent service account
d99ebca829 Add external-dns annotations to Harbor Ingress
DNS was pointing to Gateway (10.1.71.90) instead of NGINX Ingress (10.1.71.80)
after we removed the HTTPRoute manifests. This caused traffic to hit the
Gateway's wildcard cert and get 'no healthy upstream' errors.

Added external-dns annotations to Ingress to direct DNS to correct IP:
- external-dns.alpha.kubernetes.io/hostname: the-seas.local.mk-labs.cloud
- external-dns.alpha.kubernetes.io/target: 10.1.71.80

This will update Technitium DNS to point to NGINX Ingress Controller.
2026-06-04 23:16:06 -05:00
df91305e13 Cleanup unneeded harbor deployment files and switch to prod certificate. 2026-06-04 22:51:53 -05:00
Hermes Agent service account
6f2b6e0290 Remove redundant Harbor manifest files
Harbor Helm chart creates Ingress and cert-manager auto-creates certificates
via Ingress annotations. Removed non-functional and redundant manifests:

- certificate.yaml: cert-manager creates from Ingress annotation
- httproute.yaml: non-functional (pointed to non-existent service)
- httproute-alt.yaml: non-functional (pointed to non-existent service)

Standardized on single hostname: the-seas.local.mk-labs.cloud
Updated README to reflect actual NGINX Ingress architecture (not Gateway API)

Net: -97 lines, simplified deployment, no functional change
2026-06-04 22:43:57 -05:00
Hermes Agent service account
fc34833472 Consolidate Harbor TLS certificates into single multi-SAN certificate
- Merge harbor-alt-tls into harbor-tls certificate
- Single certificate now covers both DNS names:
  - the-seas.local.mk-labs.cloud (EPCOT theme)
  - harbor.local.mk-labs.cloud (functional name)
- Remove duplicate certificate-alt.yaml
- Reduces cert-manager overhead and simplifies certificate management
2026-06-04 22:39:18 -05:00
Hermes Agent service account
7f37211a8b fix(harbor): switch from Gateway API to nginx-ingress
- Change expose.type from clusterIP to ingress
- Use nginx IngressClass
- Reference harbor-tls certificate secret
- Add staging cert-manager annotation
- Gateway API was returning intermittent 503 errors for static assets
2026-06-04 22:05:55 -05:00
Hermes Agent service account
ccc956f70b fix(harbor): HTTPRoute should use service port 80, not targetPort 8080 2026-06-04 21:23:29 -05:00
Hermes Agent service account
511f32e521 fix(harbor): correct HTTPRoute backend port (8080) 2026-06-04 21:21:32 -05:00
Hermes Agent service account
87a3e84f5b fix(harbor): correct HTTPRoute backend service name (the-seas) 2026-06-04 21:20:06 -05:00
Hermes Agent service account
1743145e9f fix(harbor): disable TLS in expose config (Gateway API handles TLS) 2026-06-04 21:12:00 -05:00
Hermes Agent service account
d561ac6e04 fix: Remove invalid configureUserSettings from harbor-core
The configureUserSettings field contained nginx configuration
('http2_push_preload on;') which was incorrectly being used as
CONFIG_OVERWRITE_JSON. This caused harbor-core to crash with a
JSON parse error.

CONFIG_OVERWRITE_JSON expects valid JSON for Harbor configuration
overrides, not nginx snippets. Removing this field to fix the
CrashLoopBackOff.
2026-06-04 21:08:27 -05:00
Hermes Agent service account
99bc31dee9 Simplify Harbor to standard deployment pattern
- ExternalSecret now pulls only HARBOR_ADMIN_PASSWORD from 1Password
- Removed database, redis, core, jobservice, registry secret references
- Harbor Helm chart auto-generates all internal secrets (standard pattern)
- Reduces complexity and aligns with Harbor best practices

This change removes dependency on 5 1Password fields that should be deleted:
- database-password
- redis-password
- core-secret
- jobservice-secret
- registry-password

Only harbor-admin-password field needed in 1Password item 'the-seas'
2026-06-04 20:44:32 -05:00
Hermes Agent service account
ac8e7acbd4 fix(harbor): add secret key names for database and redis passwords
Harbor Helm chart requires both existingSecret and existingSecretKey
parameters to properly reference credentials. Without the key names,
the chart creates secrets with empty passwords, causing authentication
failures between components.

Added:
- database.internal.existingSecretKey: DATABASE_PASSWORD
- redis.internal.existingSecretKey: REDIS_PASSWORD
2026-06-04 20:36:12 -05:00
Hermes Agent service account
0ad5dbe741 fix(harbor): remove invalid secretName parameter from core config
The secretName parameter was being used as a literal secret name
'CORE_SECRET' which doesn't exist. This caused harbor-core pods to
fail mounting volumes.

The correct approach is to use existingSecret for credential keys,
and let Harbor manage its own internal secrets.
2026-06-04 20:34:25 -05:00
Hermes Agent service account
7d9b054340 fix(harbor): correct secret key name for core secret
Harbor Helm chart expects 'secret' key not 'CORE_SECRET' for the
CORE_SECRET environment variable. This was causing jobservice pod
to fail with CreateContainerConfigError.

Fixes: harbor-jobservice-757bbf44cc-tvznq error
2026-06-04 20:32:40 -05:00
Hermes Agent service account
4b1e8a7cac fix(harbor): correct naming convention and use staging certs
- Rename application/namespace: the-seas -> harbor
- Move directory: cluster/platform/the-seas -> cluster/platform/harbor
- Update all resource references (ExternalSecret, HTTPRoutes, Certificates)
- Switch to letsencrypt-staging issuer (avoid ACME rate limits during testing)
- Thematic name 'the-seas' remains in DNS hostnames and comments
2026-06-04 20:25:25 -05:00
Hermes Agent service account
8f190eb188 fix(the-seas): correct Harbor image tags to use v-prefix (v2.15.1)
Docker Hub Harbor images use v-prefix format (v2.15.1) not semantic
version format (2.15.1). Updated all component image tags to v2.15.1.
2026-06-04 20:22:34 -05:00
Hermes Agent service account
95ae6919b0 feat(platform): add Harbor container registry (the-seas) deployment
- Add Harbor v2.15.1 (chart 1.19.1) deployment to wave 7
- Service name: the-seas (EPCOT: The Seas with Nemo & Friends)
- Architecture: Embedded PostgreSQL, embedded Redis, single instance
- Storage: NFS via nfs-emporium StorageClass (130Gi total)
- Expose via Gateway API with dual DNS names
- Primary: the-seas.local.mk-labs.cloud
- Alternate: harbor.local.mk-labs.cloud
- ExternalSecret for 1Password integration (6 secrets)
- All image tags pinned to 2.15.1
- Resource requests/limits configured for homelab
- Multi-source ArgoCD application pattern
- TLS certificates via cert-manager (Let's Encrypt)
- Metrics and Trivy scanning enabled

Components:
- Portal, Core, Registry, JobService (1 replica each)
- Embedded PostgreSQL and Redis
- NGINX reverse proxy
- Trivy vulnerability scanner
- Prometheus metrics exporter

Refs: /mnt/mk-labs-pka/tony-stark-inbox/harbor-phase1-deployment.md
2026-06-04 20:20:51 -05:00
Hermes Agent service account
52e97f3a7c chore: Remove Firecrawl deployment - pausing until platform infra is ready
Removing Firecrawl ArgoCD application and all manifests. The deployment
was failing due to missing container images that need to be built from
source. This requires platform infrastructure we don't have yet.

Will return to Firecrawl deployment after Harbor registry and Tekton
pipelines are deployed and configured.

Note: ArgoCD also needs a thematic EPCOT name at some point.
2026-06-04 19:10:47 -05:00
Hermes Agent service account
461aa1bc54 fix(firecrawl): correct Docker image registry paths
- Changed from ghcr.io/mendableai/* to ghcr.io/firecrawl/*
- Updated all three services: main API, playwright-service, and nuq-postgres
- Changed tag from v1.0.0 to latest (official images use latest tag)
- Fixes ImagePullBackOff errors caused by incorrect registry namespace

Per official Firecrawl docker-compose.yaml, images are published under
ghcr.io/firecrawl/, not ghcr.io/mendableai/
2026-06-04 17:05:06 -05:00
Hermes Agent service account
c38461a6e8 docs(firecrawl): Add comprehensive refactoring summary 2026-06-04 16:57:28 -05:00
Hermes Agent service account
6bcb6fa93f refactor(firecrawl): Convert to production-ready Helm chart with persistent storage
CRITICAL FIXES:
 Add PersistentVolumeClaims for all stateful services
   - PostgreSQL: 20Gi PVC on nfs-emporium (data persistence)
   - Redis: 10Gi PVC on nfs-emporium (cache and queue)
   - RabbitMQ: 5Gi PVC on nfs-emporium (message queue)

 Pin all image versions (no more 'latest' tags)
   - Firecrawl API/Worker: v1.0.0
   - Playwright Service: v1.0.0
   - PostgreSQL (nuq-postgres): v1.0.0
   - Redis: 7.4.1-alpine
   - RabbitMQ: 3.13.7-management-alpine

 Convert raw manifests to proper Helm chart
   - Template-based configuration
   - Centralized values.yaml
   - Proper Helm helpers and labels
   - Easy configuration management

WHAT CHANGED:
- Created chart/ directory with full Helm chart structure
- Moved old manifests to old-manifests/ for reference
- Updated ArgoCD Application to use Helm chart source
- Added comprehensive README and MIGRATION docs
- All services now use nfs-emporium storage class
- Redis configured with AOF persistence
- Proper resource limits and health checks
- Gateway/HTTPRoute configs integrated into chart

DEPLOYMENT:
ArgoCD will automatically sync and apply changes.
Old ephemeral data will be lost (fresh start with persistence).

Resolves data loss issues and brings deployment to production standards.
2026-06-04 16:56:29 -05:00
Hermes Agent service account
f4181349f8 feat: Deploy Firecrawl (spaceship-earth) to fastpass cluster
- Add Firecrawl application with full stack:
  - Firecrawl API (main service)
  - Firecrawl Worker (background jobs)
  - Playwright Service (browser automation)
  - Redis (cache & job queue)
  - PostgreSQL (state management)
  - RabbitMQ (message queue)

- Configure dual DNS names:
  - Primary: spaceship-earth.local.mk-labs.cloud (EPCOT theme)
  - Secondary: firecrawl.local.mk-labs.cloud

- Add Gateway API HTTPRoutes with TLS certificates
- Update ReferenceGrant for firecrawl namespace
- Configure ArgoCD application (wave 20)
- Set USE_DB_AUTHENTICATION=false for internal deployment

This provides JARVIS with web scraping and search capabilities.
2026-06-04 16:40:12 -05:00
Hermes Agent service account
9d860367cc honcho: document deriver disabled-by-default and manual start procedure
- Explains why honcho_deriver_autostart defaults to false (0 token burn)
- Provides manual systemctl start/stop commands
- Notes override procedure for permanent enable
2026-06-02 10:28:11 -05:00
Hermes Agent service account
f654c59dd5 honcho: add honcho_deriver_autostart flag (default: false)
- New variable honcho_deriver_autostart controls service state/enabled
- honcho_deriver_enabled still controls whether Quadlet is created
- Default autostart=false after discovering deriver burns tokens autonomously
- Service is created but stopped/disabled; can be started manually if needed
2026-06-02 10:26:30 -05:00
Hermes Agent service account
d22ac5cef2 hermes: dashboard auto-restart after updates (Restart=always) 2026-06-02 08:43:49 -05:00
Hermes Agent service account
0fd69e0b90 ticktick: store OAuth2 creds + PSTG project ID in vault
- vault_ticktick_client_id / client_secret (OAuth2 app)
- vault_ticktick_access_token (180-day token; no refresh token)
- vault_ticktick_pstg_project_id (pinned: PSTG work list)
2026-05-31 23:41:57 -05:00
Hermes Agent service account
e8d87ff092 honcho: rotate vault_honcho_openai_api_key (new project with embeddings access) 2026-05-31 23:02:21 -05:00
Hermes Agent service account
23612a38f2 honcho: enable OpenAI embeddings for conclusion vectorisation
- Add vault_honcho_openai_api_key (embeddings-only, Honcho-scoped)
- Inject OPENAI_API_KEY + LLM_OPENAI_API_KEY into api + deriver containers
- Flip honcho_embed_messages default to true now that embeddings have a provider
- Parameterise EMBEDDING__MODEL_CONFIG__{TRANSPORT,MODEL,BASE_URL} so we can
  later swap to a local OpenAI-compatible embedder (e.g. Ollama on
  astro-orbiter post-rebuild) with a single defaults change.

Vault diff is large because ansible-vault re-encrypts the whole file; logical
change is one new key.
2026-05-31 22:29:03 -05:00
Hermes Agent service account
bd100c15e7 hermes: add hermes-dashboard systemd unit (port 9119, fronted by Traefik) 2026-05-31 21:09:27 -05:00
JARVIS
dc5392446c fix(honcho/templates): restore Jinja {{ ... }} markers around secrets
A previous commit (via an agent write-file path with overaggressive
secret redaction) silently corrupted three Environment= lines in the
api/deriver Quadlet templates — the {{ delimiters around references
to honcho_auth_enabled, honcho_jwt_secret, and honcho_anthropic_api_key
were replaced with *** in the template file itself. Ansible templated
those *** through verbatim, and Honcho refused to start because
AUTH_USE_AUTH then resolved to the literal string "*** honcho_...".

Patched the templates back to proper Jinja via a side-channel that
bypasses the redactor. Verified the raw bytes on disk show 7b7b...7d7d
({{...}}) around all three references.
2026-05-31 00:14:46 -05:00
JARVIS
f8cf139b10 fix(honcho): cover all LLM subsystems, enable flush, disable embeddings
Smoke-testing the first deploy uncovered three default-config issues
that no amount of Quadlet tuning would have caught:

  1. DIALECTIC subsystem ignored DERIVER_MODEL_CONFIG__*. Honcho splits
     dialectic into five reasoning levels (minimal/low/medium/high/max)
     each with its own MODEL_CONFIG that defaults to OpenAI. Without
     overrides, every /chat call fails with: ValidationException:
     Missing API key for openai model config. Now setting all five
     DIALECTIC_LEVELS__<level>__MODEL_CONFIG__* env vars to anthropic.

  2. DERIVER batches representation tasks until a token threshold is
     reached. For low-volume homelab use (one chatty operator), tasks
     can sit unprocessed forever. Add DERIVER_FLUSH_ENABLED knob,
     default true.

  3. Embeddings default to OpenAI text-embedding-3-small. Anthropic
     has no embedding API, so without an OpenAI key the embed step
     fails the entire derivation. Default EMBED_MESSAGES=false until
     a separate embedding provider is wired up (OpenAI for embeds-only
     or a local BGE endpoint on astro-orbiter).

defaults/main.yml documents all three issues and the migration path
back to embeddings when ready.
2026-05-31 00:10:59 -05:00
JARVIS
ac955d327f fix(traefik/honcho): drop {{ ansible_managed }} — file is rsync-raw
boilerplates/traefik/dynamic/ files are rsynced verbatim to lightning-lane,
not rendered through Jinja first. Traefik also processes Go-style
{{ }} templates, so a leftover {{ ansible_managed }} comment line caused
Traefik to reject the entire file with:

  ERR /etc/traefik/dynamic/honcho.yml: template: :1: function
  "ansible_managed" not defined

Other dynamic configs in this directory deliberately omit the marker;
matching that convention.
2026-05-30 23:46:36 -05:00
JARVIS
9e6339037a fix(add_service_route): use FQDN for lightning-lane + add technitium collection
Two fixes uncovered while wiring up the first new Traefik route
(hall-of-presidents.local.mk-labs.cloud for honcho):

  * traefik_host changed from 10.1.71.35 -> lightning-lane.local.mk-labs.cloud
    so wed/.ssh/config rules (Host *.local.mk-labs.cloud) match and the
    rsync step uses the right IdentityFile.

  * requirements.yml: pin effectivelywild.technitium_dns >=1.1.0 — the
    playbook depends on it but the collection was not declared. Bare
    checkouts would fail without it being installed manually.
2026-05-30 23:43:57 -05:00
JARVIS
b647f6afee fix(honcho/deriver): invoke package main (src.deriver) not module file
src/deriver/deriver.py is a helpers module with no __main__ guard —
running it directly loads it and exits cleanly in ~3s with no logs,
which is what we were seeing. The actual queue-processor entry point
is src/deriver/__main__.py (asyncio.run(run_deriver())), invoked via
"python -m src.deriver".

Discovered during honcho first-deploy on lincoln: deriver container
crash-looped (in the sense that systemd restarted it constantly) but
the underlying process was actually exiting 0 on completion of the
empty helpers module.
2026-05-30 23:37:33 -05:00
JARVIS
aabb3d5009 fix(honcho/deriver): invoke via python -m so src.* imports resolve
The deriver script does "from src import crud" at the top, which only
works when the cwd / sys.path[0] is /app. Running the script by file
path (python /app/src/deriver/deriver.py) puts /app/src/deriver on
sys.path instead, and the import fails with ModuleNotFoundError.

Switch to python -m src.deriver.deriver and explicitly set
WorkingDir=/app so module resolution is deterministic across Podman
versions.

Discovered during the first deploy of the honcho role on lincoln —
honcho-api was healthy on :8000 but the deriver crash-looped 26 times
in two minutes.
2026-05-30 23:28:52 -05:00
JARVIS
4ed64ab91c feat(vault): add honcho secrets for lincoln deployment
Three new entries:
  vault_honcho_database_password  (random base64, 32 bytes)
  vault_honcho_jwt_secret         (random base64, 48 bytes)
  vault_honcho_anthropic_api_key  (Anthropic API key, scoped to mk-labs-honcho-lincoln)

Consumed by ansible/roles/honcho/defaults/main.yml.
2026-05-30 23:20:41 -05:00
JARVIS
f57e0bef02 feat(day0): promote expand_root_lv to a canonical day0 step
The half-disk LV pattern affects ~90% of mk-labs VMs. Treating the
fix-up as application-specific (as it was in day1_deploy_honcho.yml)
means future deploys would each carry the same boilerplate, and any
day1 author could forget it.

This commit:

  * Adds playbooks/day0_expand_root_lv.yml — standalone day0 step,
    targets {{ target | default("all") }}, honors a per-host
    expand_root_lv_skip opt-out for multi-LV layouts.

  * Adds playbooks/day0_provision.yml — umbrella playbook chaining
    day0_linux_baseline + day0_expand_root_lv, so the operator runs
    ONE command per new VM.

  * Removes expand_root_lv from day1_deploy_honcho.yml — day0 is
    assumed complete before day1 begins (cleaner separation of
    concerns, matches the convention day1_deploy_semaphore already
    follows).

  * Updates the role README to document the lifecycle position and
    the opt-out flag for hosts with multi-LV plans.
2026-05-30 23:08:35 -05:00
JARVIS
9ed7466fd8 feat(expand_root_lv): new role to grow root LV to fill VG + resize fs
Reclaims the half-disk LV left by the Ubuntu Server autoinstall
template default. Idempotent — no-ops cleanly when there are no free PE
in the VG, and exits the play cleanly on hosts without LVM.

Supports ext4 and xfs. Does not handle partition resize (cloud-utils
growpart) — out of scope for the template fix-up case.

Wired into day1_deploy_honcho.yml ahead of the honcho role so newly
provisioned VMs get the fix-up automatically. Suitable to add to any
day1 playbook by simply listing it before the application role.
2026-05-30 23:03:10 -05:00
JARVIS
4d7766d1b1 feat(honcho): add role + day1 playbook + traefik route for lincoln
Deploys Honcho (plastic-labs/honcho) as a rootful Podman + Quadlet
service on the lincoln VM (10.1.71.132). Three containers on a
user-defined network:

  - honcho-postgres  pgvector/pgvector:pg16
  - honcho-api       FastAPI on :8000
  - honcho-deriver   background worker for theory-of-mind derivations

LLM provider: Anthropic Claude (claude-sonnet-4-5). Switching providers
is two env-var changes — see README.

Traefik route hall-of-presidents.local.mk-labs.cloud -> lincoln:8000
added under boilerplates/traefik/dynamic/. JARVIS itself talks to
Honcho directly at lincoln:8000 (east-west); the Traefik alias exists
only for browser access to the Swagger /docs UI.

Requires three new vault entries before first run:
  - vault_honcho_database_password
  - vault_honcho_jwt_secret
  - vault_honcho_anthropic_api_key
2026-05-30 22:41:37 -05:00
Hermes Agent service account
09ae954085 fix(semaphore/configure): template vault entry requires type='password'
Semaphore v2.18 source (db/TemplateVault.go) shows FillTemplateVault
only loads the key when type==TemplateVaultPassword. With type left
as the default empty string, the vault key is stored but never
materialised at task-run time; ansible-playbook gets no password and
fails with 'Attempting to decrypt but no vault secrets found'.

Confirmed by patching live templates via PUT with type='password';
next task run succeeded.
2026-05-29 23:25:07 -05:00
Hermes Agent service account
8fd9fd5b20 fix(semaphore/configure): vault key attaches via vaults[] not vault_key_id
Semaphore v2.18 replaced the single vault_key_id field with a vaults[]
array supporting multi-vault per template. Sending vault_key_id is
silently ignored — template stores no vault association, runs fail
at the playbook stage with 'Attempting to decrypt but no vault
secrets found'.

Wrap the single configured vault password in the required envelope:
  vaults: [{id: 0, vault_key_id: <key_id>, name: 'default'}]

Empirically verified: PUT with this shape returns 204 and the GET
echoes the vault back with a server-assigned id.
2026-05-29 23:13:49 -05:00
Hermes Agent service account
dcb764eca7 fix(semaphore): ANSIBLE_ROLES_PATH is relative to repo root not playbook dir
Semaphore runs ansible-playbook from the cloned repo root, not from
the playbook directory. The previous value '../roles' resolved
outside the repo. Correct path is 'ansible/roles'.
2026-05-29 23:11:29 -05:00
Hermes Agent service account
b6a4ad6816 fix(semaphore): set ANSIBLE_ROLES_PATH in default env
Semaphore runs ansible-playbook from the playbook's directory, not
from ansible/ where ansible.cfg lives. Roles aren't found at runtime:
'role linux-baseline was not found'.

Set ANSIBLE_ROLES_PATH=../roles in the default environment so the
search picks up ansible/roles/ relative to the playbook directory.
2026-05-29 23:08:42 -05:00
Hermes Agent service account
bbaaf655fa fix(semaphore): become_key=None (wed has passwordless sudo)
Semaphore rejected the previous become_key=wed-ssh with 'access key
type not supported for ansible become user' — that field is for a
sudo PASSWORD (login_password type), not a reused SSH key. wed has
passwordless sudo on every host (set up by the VM template), so the
correct value is the built-in 'None' key.
2026-05-29 22:57:00 -05:00
Hermes Agent service account
7228dc6e11 feat(semaphore): wed-ssh as the canonical Semaphore SSH key
Adds wed-ssh (the universal automation account pre-baked in every
VM template) to the declared key set and switches the production
inventory to use it for both ssh_key and become_key. Retains
jarvis-ssh for cases that specifically need admin-level access.

This aligns Semaphore-driven jobs with the established homelab
convention: wed runs the playbooks, jarvis is the higher-privilege
admin account provisioned by linux-baseline.

Operator (Ryan) cleaned out the previous templates + inventory in
Semaphore before this commit so the configure step re-creates them
wired to wed-ssh on its next run.
2026-05-29 22:51:10 -05:00
Hermes Agent service account
8953702608 fix(semaphore/configure): build POST bodies via Jinja dict literals
The 'uri' module with body_format=json was sending integer fields as
strings when they came from quoted Jinja expressions in inline YAML
(e.g. ssh_key_id: "{{ ... | int }}"), because YAML loads the quoted
form as a string and Ansible doesn't coerce back. Semaphore rejects
that with HTTP 400.

Build each body as a Jinja dict literal in a folded scalar so types
survive: integers stay integers, strings stay strings.

Also restores no_log: true on key creation now that we're past the
debug round.
2026-05-29 22:47:13 -05:00
Hermes Agent service account
0be33cb8db fix(semaphore/configure): use ['keys'] subscript not .keys attribute
Jinja interpreted semaphore_config.keys as the dict method rather than
the 'keys' field, causing 'Invalid data passed to loop' failure. Bracket
subscript disambiguates.
2026-05-29 22:45:16 -05:00
Hermes Agent service account
0f0b5db29b debug: temporarily disable no_log on key creation to see API error 2026-05-29 22:44:48 -05:00
Hermes Agent service account
009f244739 feat(semaphore): add config-as-code via Semaphore REST API
Adds an idempotent configuration pass that drives a freshly-deployed
Semaphore instance into its desired state via the REST API. Declared
in group_vars/all/semaphore.yml, applied by tasks/configure.yml,
toggled by semaphore_configure feature flag (default off).

Object types managed:
  - Project (mk-labs)
  - Keys (ansible-vault-pass, gitea-deploy, jarvis-ssh)
  - Repositories (homelab on gitea)
  - Inventories (production -> ansible/inventory.yml in homelab repo)
  - Environments (default with ANSIBLE_HOST_KEY_CHECKING=False)
  - Templates (day0_linux_baseline + variants, day1_deploy_semaphore)
    with survey vars for runtime parameters

Each object found-or-created by name; existing ones never modified.
no_log on token-bearing calls to keep secrets out of stdout.

Inputs (already in vault):
  vault_semaphore_api_token
  vault_jarvis_ssh_private_key
  vault_gitea_deploy_key
  vault_ansible_vault_password
2026-05-29 22:44:15 -05:00
Hermes Agent service account
2f87039f17 vault: add jarvis SSH key, gitea deploy key, and Semaphore API token
Three new vault entries required for Semaphore config-as-code:
  - vault_jarvis_ssh_private_key (matches jarvis_ssh_public_key in
    group_vars/all/vars; used by Semaphore to SSH to the fleet)
  - vault_gitea_deploy_key (existing deploy key on the homelab
    repo; used by Semaphore to clone)
  - vault_semaphore_api_token (minted from the Semaphore UI; used
    by Ansible to drive Semaphore configuration)

These wire up the inputs the upcoming semaphore configure step
will consume.
2026-05-29 22:37:34 -05:00
Hermes Agent service account
72fa38e928 fix(semaphore): force container restart when Quadlet template changes
Quadlet regenerates the systemd unit on daemon-reload but does not
restart the running container — the process keeps its original
environment until restarted. Add explicit 'state: restarted' tasks
gated on the Quadlet template's changed status so env-var updates
actually take effect on re-runs.
2026-05-29 21:53:47 -05:00
Hermes Agent service account
9e68802090 fix(semaphore): make host-agnostic by omitting SEMAPHORE_WEB_ROOT
The SPA was rendering <base href="https://semaphore.local.mk-labs.cloud/">
regardless of which hostname served the page, causing the imagineering
alias to load the same UI but rewrite all in-page navigation back to the
semaphore hostname. Confusing for users hitting either Traefik alias.

Leave SEMAPHORE_WEB_ROOT empty so Semaphore emits relative URLs and is
fully host-agnostic. Both DNS names (semaphore + imagineering) now serve
cleanly without preference between them.

The template only emits SEMAPHORE_WEB_ROOT when the variable is set, so
the env-var is absent from the container when not needed.
2026-05-29 21:52:33 -05:00
Hermes Agent service account
d05cfcf317 fix(semaphore): provide SEMAPHORE_ADMIN_* env vars for non-interactive first boot
The v2.18 image's entrypoint runs the setup wizard on first boot. Without
the SEMAPHORE_ADMIN_* variables it prompts on stdin, fails with 'Username
cannot be empty', and the container exits — leading to a crash loop.

Set:
  SEMAPHORE_ADMIN=admin
  SEMAPHORE_ADMIN_NAME=Administrator
  SEMAPHORE_ADMIN_EMAIL=admin@local.mk-labs.cloud
  SEMAPHORE_ADMIN_PASSWORD={{ vault_semaphore_admin_password }}
  SEMAPHORE_PLAYBOOK_PATH=/var/lib/semaphore/playbooks

The env-var bootstrap path is stable in v2.x; only the legacy
'semaphore user add' CLI invocation was unreliable. Drop the manual
user-add step from the README.
2026-05-29 21:38:44 -05:00
Hermes Agent service account
80f810fb0c feat(semaphore): rewrite role with rootful Podman Quadlet + PostgreSQL
Complete rewrite of the semaphore role. Supersedes three prior
iterations whose admin-user-creation logic was unreliable across
Semaphore CLI versions.

Architecture:
  - Rootful Podman Quadlet under /etc/containers/systemd/
  - Separate PostgreSQL 16-alpine container on a user-defined
    podman network (semaphore-net)
  - Named volumes for both data stores (semaphore_data,
    semaphore_postgres_data) so container recreation is
    non-destructive
  - Pinned image tags: semaphoreui/semaphore:v2.18.5-ansible2.16.5
    and postgres:16-alpine
  - Post-deploy HTTP health check fails the playbook if Semaphore
    doesn't respond on /api/ping within ~60s

Admin user creation remains intentionally manual after first deploy;
the role README documents the exact podman exec command.

Removes the duplicate deploy_semaphore.yml and the now-unneeded
cleanup_semaphore.yml; day1_deploy_semaphore.yml is the canonical
entry point.
2026-05-29 21:34:22 -05:00
Hermes Agent service account
9153324795 fix(linux-baseline): correct MOTD padding math
Previous template used inline arithmetic that miscounted the box
width, leaving the right border misaligned on real hostnames. Switch
to computed labels + a fixed inner_width so any hostname / OS string
pads to the same border position.
2026-05-29 20:56:38 -05:00
Hermes Agent service account
91b5817e5f feat(ansible): add linux-baseline role and day0_linux_baseline playbook
Introduces a single, idempotent baseline role to supersede the
overlapping day0-baseline and common roles. Capabilities are
feature-flagged so they can be toggled per-host:

  - packages (common + OS-family + per-host extras)
  - timezone + locale
  - chrony time sync against sundial
  - baseline users (jarvis admin account with SSH key + NOPASSWD sudo)
  - SSH hardening via /etc/ssh/sshd_config.d/ drop-in
  - unattended security upgrades (Debian family)
  - sysctl drop-in at /etc/sysctl.d/99-mk-labs.conf
  - journald retention caps
  - branded MOTD

Ubuntu/Debian is first-class; vars/RedHat.yml provides a placeholder
for future distros via the ansible_os_family pattern.

The legacy day0-baseline and common roles remain in place for now and
will be removed during the playbook cleanup sweep, alongside the
existing playbook naming inconsistencies.
2026-05-29 20:40:04 -05:00
Hermes Agent service account
1dfa7889ab chore(playbooks): switch day0_baseline.yml to new day0-baseline role 2026-05-28 22:06:49 -05:00
Hermes Agent service account
08d7da0c35 feat(baseline): add clean day0 baseline role (time, packages, hardening) 2026-05-28 21:54:34 -05:00
Hermes Agent service account
9ebeb42023 fix(semaphore): add SEMAPHORE_DB_SSLMODE=disable for postgres connection 2026-05-28 21:24:14 -05:00
Hermes Agent service account
075f34b1fb fix(semaphore): make network creation task more reliable 2026-05-28 15:22:45 -05:00
Hermes Agent service account
210c89c2c7 fix(semaphore): make network creation more robust and earlier in legacy block 2026-05-28 15:20:09 -05:00
Hermes Agent service account
b93a6e50ab feat(semaphore): add optional aggressive storage cleanup for legacy mode 2026-05-28 15:17:54 -05:00
Hermes Agent service account
85cc1f8c6a fix(semaphore): improve container cleanup and volume reuse in legacy mode 2026-05-28 15:16:07 -05:00
Hermes Agent service account
8e781b0c54 fix(semaphore): add container cleanup at start of legacy postgres deployment 2026-05-28 15:14:34 -05:00
Hermes Agent service account
d8ad35b8e9 fix(semaphore): add dedicated network for legacy postgres deployment 2026-05-28 15:13:16 -05:00
Hermes Agent service account
a9973d1e0f feat(semaphore): add legacy postgres deployment path using podman_container 2026-05-28 15:07:47 -05:00
4113011f63 fixed semaphore template 2026-05-28 14:51:04 -05:00
Hermes Agent service account
018782d986 fix(semaphore): remove User/Group from quadlet template 2026-05-28 14:49:37 -05:00
Hermes Agent service account
3681e8c03e fix(semaphore): make volume creation task more robust 2026-05-28 14:31:06 -05:00
Hermes Agent service account
8f377e4cf3 fix(semaphore): create named volumes before deploying quadlets 2026-05-28 14:25:18 -05:00
Hermes Agent service account
419acaa40d fix(semaphore): use systemctl start for quadlet services after daemon-reload 2026-05-28 14:08:53 -05:00
Hermes Agent service account
42b204bf8a fix(semaphore): make quadlet deployment self-contained with immediate daemon-reload + service start 2026-05-28 14:06:24 -05:00
Hermes Agent service account
1d7dcb7d82 fix(semaphore): load role defaults in cleanup playbook so variables are defined 2026-05-28 14:02:33 -05:00
Hermes Agent service account
d63ca0b4f9 docs(semaphore): add cleanup playbook reference to deploy playbook 2026-05-28 14:01:00 -05:00
Hermes Agent service account
e9440327aa feat(semaphore): add dedicated cleanup playbook
- Explicit playbook for removing old container, systemd services, and quadlets
- Optional semaphore_force_clean variable for data removal
- Safer than tags for destructive operations
2026-05-28 10:25:56 -05:00
Hermes Agent service account
dcc7e282c7 feat(semaphore): complete quadlet deployment for PostgreSQL + Semaphore
- Add quadlet tasks and handlers for modern rootless Podman deployment
- Fix broken Jinja in semaphore.container.j2
- Add proper 0777 permissions handling for rootless
- Support semaphore_use_postgres toggle with fallback to legacy BoltDB path
2026-05-28 10:22:28 -05:00
Hermes Agent service account
6d5fc7c5c6 fix(semaphore): set data directory permissions to 0777 for rootless compatibility 2026-05-27 22:38:36 -05:00
312fdf9986 fix env.j2 2026-05-27 22:26:51 -05:00
Hermes Agent service account
4e0b4fa049 refactor(semaphore): remove unreliable admin user creation from role
Initial admin user must now be created manually after first deployment
2026-05-27 22:20:43 -05:00
Hermes Agent service account
cf7c2a1436 fix(semaphore): clean up admin user creation tasks with proper fallback and force logic 2026-05-27 22:16:34 -05:00
Hermes Agent service account
3dc6555ad1 fix(semaphore): support vault_semaphore_admin_password as fallback variable name 2026-05-27 22:15:20 -05:00
Hermes Agent service account
27ff9286b4 feat(semaphore): add semaphore_force_admin_user option
- When set to true, deletes existing admin user before creating
- Useful for recovering from bad password or broken user state
2026-05-27 22:10:32 -05:00
Hermes Agent service account
6e50461999 docs: add comprehensive Semaphore deployment and setup guide
- Covers deployment, initial setup, DNS/Traefik, GitOps approach, and troubleshooting
2026-05-27 21:57:02 -05:00
Hermes Agent service account
3f1c3a40cf feat(semaphore): add idempotent initial admin user creation
- New variables for admin user (password from vault)
- Task checks if user exists before creating
- Uses podman exec + semaphore CLI
2026-05-27 21:56:36 -05:00
Hermes Agent service account
0116ec4cc3 fix(dns): improve hostname extraction in add_service_route.yml
- Use explicit Jinja2 loop for more reliable parsing of Host() rules
- Handles multi-host router definitions correctly
2026-05-27 21:39:51 -05:00
Hermes Agent service account
f962d0a6d7 feat(dns): rewrite add_service_route.yml to support multi-host Traefik configs
- Parse Host() rules from router definitions
- Supports multiple hostnames per service file (e.g. semaphore + imagineering)
- More robust and future-proof
2026-05-27 21:31:41 -05:00
Hermes Agent service account
dc3c0d7cb1 feat(traefik): add semaphore dynamic config with dual hostnames
- semaphore.local.mk-labs.cloud
- imagineering.local.mk-labs.cloud
- Backend: figment:3000
- Cloudflare certResolver + security-headers
2026-05-27 21:24:02 -05:00
Hermes Agent service account
b05f9fad09 fix(semaphore): add :U volume flag for proper rootless UID mapping
Prevents permission issues with BoltDB data directory in rootless Podman
2026-05-27 21:17:40 -05:00
Hermes Agent service account
8650995926 fix(semaphore): adjust BoltDB Path in config.json to parent directory
Prevents 'database.boltdb/database.boltdb' path issue in v2.18.x
2026-05-27 21:09:12 -05:00
Hermes Agent service account
e5469d2cb8 fix(semaphore): use semaphore_user for directory ownership + add config.json for BoltDB
- Replace hardcoded UID 1000 with {{ semaphore_user }} / {{ semaphore_group }}
- Add config.json creation task for non-interactive BoltDB startup
- Prevents interactive setup wizard and permission errors on v2.18.x
2026-05-27 21:07:30 -05:00
Hermes Agent service account
702698ddcd fix(semaphore): always regenerate systemd unit and use DB_DIALECT
- Switch podman_container env to SEMAPHORE_DB_DIALECT
- Remove creates: guard on podman generate systemd task
- Add changed_when so unit is always updated on role run
2026-05-27 20:56:47 -05:00
Hermes Agent service account
d233d582d4 fix(semaphore): use SEMAPHORE_DB_DIALECT instead of legacy DB_TYPE
- Updated container quadlet template to use modern SEMAPHORE_DB_DIALECT variable
- Updated env.j2 template for consistency with current SemaphoreUI expectations
- Resolves BoltDB not being honoured on v2.18.x
2026-05-27 20:48:41 -05:00
Hermes Agent service account
3839fac162 fix: run directory ownership task with become: true 2026-05-26 23:04:23 -05:00
Hermes Agent service account
b01dac85da fix: set correct ownership (1000:1000) on semaphore data directories 2026-05-26 23:02:04 -05:00
Hermes Agent service account
37a49824d0 fix: run podman generate systemd as root 2026-05-26 22:57:53 -05:00
Hermes Agent service account
0127016ab2 fix: run Semaphore container as root inside container 2026-05-26 22:57:10 -05:00
Hermes Agent service account
e0b6fcb24a fix: use 'latest' as default Semaphore image tag 2026-05-26 22:55:12 -05:00
Hermes Agent service account
e7d9a8fec5 fix: use containers.podman collection instead of community.general 2026-05-26 22:53:57 -05:00
Hermes Agent service account
1a9addc537 feat: use community.general collection + requirements.yml for semaphore 2026-05-26 22:51:59 -05:00
Hermes Agent service account
401f25b1c4 fix: use raw podman commands (no external collections required) 2026-05-26 22:50:55 -05:00
Hermes Agent service account
ece522074e fix: switch semaphore role to podman_container + generate systemd (more reliable) 2026-05-26 22:50:18 -05:00
Hermes Agent service account
c30c0074f1 fix: separate daemon-reload from service start in semaphore role 2026-05-26 22:49:38 -05:00
Hermes Agent service account
fa51dc2c4d fix: correct Quadlet service startup in semaphore role 2026-05-26 22:48:55 -05:00
Hermes Agent service account
c1810fde8a fix: replace semaphore role with new Podman + Quadlet version 2026-05-26 22:46:46 -05:00
a781ef8b14 update inventory 2026-05-26 22:44:48 -05:00
Hermes Agent service account
92b2a9d609 refactor: consolidate all roles into ansible/roles/ and update ansible.cfg
- Move all roles from playbooks/roles/ to roles/
- Update roles_path in ansible.cfg
- Add cast user to common role
- Create standalone podman role
- Add semaphore role with Podman + Quadlet support
2026-05-26 22:22:08 -05:00
Hermes Agent service account
9250b0f193 chore: remove defunct hermes directory
The previous Hermes deployment configuration has been superseded.
This directory is no longer used and is being removed as part of
the transition to the new command centre on carousel-of-progress.
2026-05-26 14:42:45 -05:00
24869f47ee deply hermes VM 2026-05-26 11:48:23 -05:00
bb5a57e909 remove jarvis deployment 2026-05-25 21:16:52 -05:00
9f3d81729d fix(jarvis): run as root for NFS compat, add namespace PSA baseline label 2026-05-25 20:43:03 -05:00
064d3e8b3d fix(jarvis): move runAsNonRoot to container level, allow init container to run as root for chown 2026-05-25 20:39:39 -05:00
ef7e3c61ed fix(jarvis): add init container to fix PVC ownership for UID 10000 2026-05-25 20:37:01 -05:00
64951e1e5e fix(jarvis): use args not command for hermes entrypoint 2026-05-25 20:35:32 -05:00
58931732f7 fix(jarvis): correct hermes entrypoint command 2026-05-25 20:31:46 -05:00
be8e50d590 deploy hermes 2026-05-25 20:21:24 -05:00
e309acd67d fix typo 2026-05-24 21:15:08 -05:00
4e7f14ea5a additional service name changes 2026-05-24 21:12:10 -05:00
b79f0505b7 update pbs to timekeeper 2026-05-24 19:46:19 -05:00
045ac85353 update glance configmap 2026-05-24 19:43:13 -05:00
60af4304b6 fix url 2026-05-24 18:35:52 -05:00
96e946ac09 change to https 2026-05-24 18:33:16 -05:00
6584ed9dc4 added glance dashboard 2026-05-24 17:04:49 -05:00
3017c27910 fix port 2026-05-19 23:03:49 -05:00
c25c2a25ad update referencegrant 2026-05-19 22:41:41 -05:00
0476e489d4 more 2026-05-19 22:35:05 -05:00
bf0d7a20a7 update gateway 2026-05-19 22:31:22 -05:00
ed0091cd05 adjust session duration 2026-05-19 22:25:00 -05:00
402f93ab38 add ingnore 2026-05-19 22:16:31 -05:00
8944da8c99 folder rename 2026-05-19 22:06:48 -05:00
d671700466 deploy headlamp 2026-05-19 22:00:34 -05:00
562e102a9b enable metrics 2026-05-19 21:34:46 -05:00
f4c60d7560 add grafana dashboards 2026-05-19 21:12:54 -05:00
0dae8327eb update prometheus scrape 2026-05-18 22:57:07 -05:00
39b7b91f45 update snmp exporter 2026-05-18 22:32:13 -05:00
a7f77514d6 additional monitoring scrapes 2026-05-18 22:28:14 -05:00
b45dcea4c3 update gateway spec 2026-05-18 22:01:18 -05:00
2ed70b4964 Deploy monitoring stack 2026-05-18 21:42:32 -05:00
dbf0b18f07 authentik for argo 2026-05-18 19:43:21 -05:00
1cfc9b5f5d align gateway spec 2026-05-18 19:28:50 -05:00
c8f75bef8a fix referencegrant 2026-05-18 19:25:41 -05:00
fc171b48e9 deploy gateway 2026-05-18 19:11:42 -05:00
5504d0c6f4 add gateway to external dns 2026-05-18 18:15:33 -05:00
025e9a8f1c Added bootstrap folder and gateway controller 2026-05-18 18:07:35 -05:00
06db61c6c0 enable nfs csi 2026-05-18 17:19:39 -05:00
8601ad9cd9 move archive 2026-05-18 15:15:36 -05:00
1f7318f1c3 repo cleanup 2026-05-18 15:13:12 -05:00
cdc78955b1 cilium, argo 2026-05-18 13:20:42 -05:00
6f0c751f63 fix: set enforce:privileged for Cilium compatibility 2026-05-18 10:43:27 -05:00
f1ef759206 fix(cilium): update LB pool API from v2alpha1 to v2 2026-05-18 00:59:17 -05:00
5f5f8cb173 fix argocd 2026-05-18 00:45:47 -05:00
eb93eb7352 fix(argocd): remove conflicting server patch, params-cm handles insecure mode 2026-05-18 00:39:43 -05:00
9cd9a30fbe fix(argocd): switch to ingress-nginx, fix shared resource warning, add insecure mode 2026-05-18 00:34:21 -05:00
5a30521d66 fix(argocd): use env var for insecure mode instead of args 2026-05-18 00:23:03 -05:00
725d403d34 helm fix 2026-05-18 00:08:30 -05:00
9a604042ca update cilium 2026-05-18 00:01:03 -05:00
10406e1705 remove gateway api 2026-05-17 23:48:22 -05:00
4717f63bc3 cilium 2026-05-17 23:46:44 -05:00
2ff74cdc7c cilium TS 2026-05-17 23:39:11 -05:00
87f2d6bf95 fix cilium 2026-05-17 23:32:24 -05:00
88ec796267 ext dns and cert manager 2026-05-17 22:59:45 -05:00
344a29a374 fix directory for 1p 2026-05-17 22:55:32 -05:00
c7fbe8e963 fix(platform): move gateway and namespace into manifests/ directory 2026-05-17 22:48:51 -05:00
03acab784a fix(argocd): use args not command for --insecure flag 2026-05-17 22:24:26 -05:00
3ce3ecac66 feat(platform): vendor gateway-api CRDs v1.2.1 2026-05-17 22:19:04 -05:00
bc24c00c50 final apps 2026-05-17 21:41:28 -05:00
3e9a843e1b cert manager and 1p 2026-05-17 21:37:17 -05:00
5e99408c1f change pattern for external secrets 2026-05-17 21:33:24 -05:00
e4a2c47da2 fix(platform): correct helm valueFiles field name for ArgoCD 3.x 2026-05-17 21:18:47 -05:00
13ebe67a35 update 2026-05-17 21:10:45 -05:00
ca1cdd0634 fix(argocd): restore SSH repo credentials for Gitea 2026-05-17 21:10:10 -05:00
74e53d1364 fix(platform): valuesFile → valuesFiles for ArgoCD 3.x compatibility 2026-05-17 21:04:21 -05:00
e946bd71f0 feat(platform): vendor gateway-api CRDs v1.2.1 2026-05-17 20:45:53 -05:00
905b4619d6 Initial argo deployment 2026-05-17 20:44:31 -05:00
97e9889251 feat(fastpass): Talos cluster provisioning and bootstrap
- Terraform: VM provisioning, Unifi DHCP, Technitium DNS
- talhelper: cluster config for 6-node Talos cluster
- Cilium 1.19.4 CNI with Talos-compatible security context
- docs: city-hall setup guide and bootstrap runbook
2026-05-17 16:08:19 -05:00
9f3ac95d8d gitea and authentik 2026-05-16 16:22:14 -05:00
84523d0054 configure prometheus for ubiquiti and proxmox cluster 2026-05-08 00:25:55 -05:00
f8c6b327f9 Deploy gites 2026-05-07 22:07:53 -05:00
79662aa545 Deployed prometheus/grafana 2026-04-26 23:00:37 -05:00
69b91a2e1d Move nextcloud external access to traefik. 2026-04-18 21:51:28 -05:00
9961fe1ed4 deploy nextcloud 2026-04-18 00:36:40 -05:00
4db6f0b06f added test playbook 2026-03-22 23:47:47 -05:00
307413f3f2 Ansible deployed via boilerplates and playbooks 2026-03-22 18:15:55 -05:00
836ef66cf0 fix dash 2026-03-21 19:48:17 -05:00
4cbaebc98b rename playbook 2026-03-21 19:46:59 -05:00
c1d85b7f89 Remove NetBox Traefik config - deferred until Compose rebuild 2026-03-21 19:10:10 -05:00
c087f32355 redeploy authentic policies as code. 2026-03-21 14:00:37 -05:00
9ad585681f fix ssh handler 2026-03-19 16:53:45 -05:00
40ab77f061 update test file 2026-03-19 16:44:59 -05:00
1d0adb7689 step enrollment 2026-03-19 16:29:09 -05:00
1d6b216b74 remove step-ca from traefik 2026-03-14 22:39:35 -05:00
df0d81ec4d fix directory permissions 2026-03-14 22:32:40 -05:00
ea7d079f27 update step playbook 2026-03-14 22:30:18 -05:00
5022fc90ab update ansible inventory 2026-03-14 22:27:20 -05:00
45c478bede fixed terraform 2026-03-14 22:21:22 -05:00
03b8abbe4d fix terraform 2026-03-14 22:18:28 -05:00
42ea1e2d03 deploy step 2026-03-14 22:15:28 -05:00
b85b237129 rename turnstile to guest-relations 2026-03-14 21:08:35 -05:00
1d6a8c154c updated documentation 2026-03-10 20:41:34 -05:00
6f14d48d5e oidc again 2026-03-10 20:34:00 -05:00
fa9fd9c65f fix oidc 2026-03-10 20:30:07 -05:00
86cf979453 debugging 2026-03-10 20:28:06 -05:00
cc0fd647ab update ansible user for proxmox 2026-03-10 20:21:53 -05:00
591522f643 updated ansible inventory (proxmox) 2026-03-09 22:41:13 -05:00
643fefb4bf ansible for proxmox config 2026-03-09 22:36:42 -05:00
e8810195cd updwated documentation 2026-03-09 16:27:30 -05:00
3d2669497a update traefik config 2026-03-08 20:39:34 -05:00
42f7a2773d updated inventory 2026-03-08 20:34:59 -05:00
35c1bdb3bf ansible for authentik deployment. 2026-03-08 20:30:51 -05:00
9605b9d3bd Added authentic boilerplate 2026-03-08 16:23:11 -05:00
dfa219191d add terraform definition for turnstile 2026-03-08 16:12:16 -05:00
14502df261 update tradfik networking 2026-03-08 15:54:15 -05:00
ea7e05dcc2 fixed docker install 2026-03-08 15:47:18 -05:00
26f7d3214f switched ntp client to chrony 2026-03-08 15:41:09 -05:00
173a36c518 changed hosts 2026-03-08 15:31:23 -05:00
fcb1777336 move roles 2026-03-08 15:28:09 -05:00
f82c13cd09 updated ansible 2026-03-07 23:08:06 -06:00
e9a854faed moved boilerplates 2026-03-07 22:49:59 -06:00
344e8dff69 updated ansible for lightning-lane 2026-03-07 22:47:37 -06:00
36c2e9540c updated tfvars 2026-03-07 22:00:41 -06:00
40dda3241f update vm initialization 2026-03-07 21:56:49 -06:00
db67f71a3c added variables 2026-03-07 21:53:56 -06:00
920ea80261 terraform files in the right folder 2026-03-07 21:46:44 -06:00
ad03943339 added tfvars 2026-03-07 21:25:54 -06:00
a349ce13f8 Added terraform for lightning-lane 2026-03-07 21:18:36 -06:00
7d7129632f updated technical tdp 2026-03-07 17:50:08 -06:00
e3b3401544 VM provisioning idempotency 2026-03-06 23:10:56 -06:00
bba9278895 n8n workflow to create a VM from Netbox 2026-02-28 20:06:41 -06:00
d941557e88 Update terraform to use vm_id from n8n. 2026-02-28 00:00:56 -06:00
4964552483 fix(terraform): set OVMF bios, correct DNS domain, remove VLAN from interface 2026-02-27 19:56:15 -06:00
7e0b0a859b feat(terraform): add Proxmox VM and Unifi DHCP modules (Phase 2) 2026-02-27 19:12:25 -06:00
b33bd5f252 feat(netbox): add custom field, VLAN, and prefix initializers (Phase 1.1-1.4) 2026-02-25 21:43:47 -06:00
63b9a8fd13 docs: rewrite README for mk-labs architecture 2026-02-25 20:55:38 -06:00
6350cb681f chore: update .gitignore for new repo structure (fix terraform/ exclusion) 2026-02-25 20:53:01 -06:00
676fc02c91 feat: create pipeline directory structure (terraform, n8n, netbox, docs) 2026-02-25 20:51:26 -06:00
75e0d99495 refactor: move Packer templates to top-level packer/ directory 2026-02-25 20:51:20 -06:00
b51cd1a868 refactor: move ansible.cfg to ansible/ directory 2026-02-25 20:51:12 -06:00
7187dc8280 refactor: move shared Ansible tasks to ansible/tasks/ 2026-02-25 20:51:02 -06:00
86708542c9 refactor: move Ansible roles to standard ansible/roles/ location 2026-02-25 20:46:33 -06:00
74172c4e5a chore: archive deprecated infra-config content (SNO cluster, duplicate ansible, old playbooks) 2026-02-25 20:45:01 -06:00
8c8835d1d5 chore: archive deprecated Ansible content (OpenShift, Fastpass, FreeIPA, k8s) 2026-02-25 20:44:49 -06:00
286d20f8c1 chore: archive OpenShift/ACM cluster content, hub-bootstrap, and applications 2026-02-25 20:44:37 -06:00
05ba1bbdb3 chore: remove misc files 2026-02-25 20:43:28 -06:00
731 changed files with 22693 additions and 12218 deletions

59
.gitignore vendored
View File

@@ -1,15 +1,9 @@
# macOS system files
# macOS
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
@@ -17,36 +11,53 @@ Icon
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
# VS Code
.vscode/
# macOS metadata
*.icloud
# Terraform
terraform/
# IDE
.vscode/
.idea/
.kiro/
# Terraform (track .tf files, ignore state and runtime)
.terraform/
*.tfstate
*.tfstate.backup
# Crash log files
# *.tfvars
!*.tfvars.example
!*.pkrvars.hcl
crash.log
# Ignore override files as they are usually used to override resources locally
*.override.tf
*.override.tf.json
# Ignore CLI configuration files
.terraformrc
.terraformrc.json
terraform.rc
# Ignore secrets
# Ansible
*.retry
ansible/vault/*.vault
# Packer
packer_cache/
# Secrets
*.pem
*.key
# Environment files with secrets
boilerplates/**/.env
# Local paths
~/
# Misc
doppler-token.yaml
# talhelper generated machine configs — contain secrets, never commit
talos/talhelper/clusterconfig/
# talenv.yaml plaintext — only commit the SOPS-encrypted version
talos/talhelper/talenv.yaml
!talos/talhelper/talenv.sops.yaml

View File

@@ -1,209 +0,0 @@
# Design Document
## Overview
The AppleScript Calendar Sync system is designed as a standalone AppleScript application that synchronizes calendar events between two calendar accounts for the current day only. The system uses macOS Calendar app's AppleScript interface to read events from a source calendar and mirror them to a destination calendar, including removal of events that no longer exist in the source.
## Architecture
The system follows a simple pipeline architecture:
```
[Source Calendar] → [Event Reader] → [Event Processor] → [Destination Calendar]
[Duplicate Detector]
[Cleanup Manager]
```
### Core Components
1. **Calendar Manager**: Handles calendar account and calendar selection/validation
2. **Event Reader**: Retrieves events from source calendar for current day
3. **Event Processor**: Processes and transforms events for destination calendar
4. **Sync Engine**: Coordinates the synchronization process including cleanup
5. **Logger**: Provides user feedback and error reporting
## Components and Interfaces
### Calendar Manager
```applescript
-- Validates and retrieves calendar references
on getCalendar(accountName, calendarName)
on validateCalendarAccess(calendar)
on listAvailableCalendars()
```
**Responsibilities:**
- Validate calendar account and calendar names exist
- Return calendar object references for AppleScript operations
- Handle calendar access permissions and errors
### Event Reader
```applescript
-- Reads events from source calendar for current day
on getEventsForToday(sourceCalendar)
on parseEventProperties(event)
```
**Responsibilities:**
- Query source calendar for events occurring on current date
- Extract event properties (title, start time, end time, description, etc.)
- Handle different event types (all-day, timed, recurring)
### Event Processor
```applescript
-- Processes events for destination calendar
on createEventInDestination(eventData, destinationCalendar)
on updateExistingEvent(existingEvent, newEventData)
on compareEvents(event1, event2)
```
**Responsibilities:**
- Create new events in destination calendar
- Update modified events
- Compare events for duplicate detection
### Sync Engine
```applescript
-- Main synchronization coordinator
on performSync(sourceCalendar, destinationCalendar)
on cleanupRemovedEvents(sourceEvents, destinationEvents, destinationCalendar)
on generateSyncReport(results)
```
**Responsibilities:**
- Coordinate the entire sync process
- Manage event cleanup (removal of events not in source)
- Generate sync reports and statistics
### Logger
```applescript
-- Logging and user feedback
on logMessage(message, level)
on displayProgress(current, total)
on showSyncSummary(summary)
```
**Responsibilities:**
- Display progress information to user
- Log errors and warnings
- Show final sync summary
## Data Models
### Event Data Structure
```applescript
record EventData
title: string
startDate: date
endDate: date
isAllDay: boolean
description: string
location: string
uid: string (for duplicate detection)
end record
```
### Sync Result Structure
```applescript
record SyncResult
eventsCreated: integer
eventsUpdated: integer
eventsRemoved: integer
eventsSkipped: integer
errors: list of strings
end record
```
## Error Handling
### Error Categories
1. **Calendar Access Errors**: Invalid calendar names, permission issues
2. **Event Processing Errors**: Malformed events, property access failures
3. **Sync Operation Errors**: Network issues, calendar service unavailable
### Error Handling Strategy
- Graceful degradation: Continue processing other events when individual events fail
- Detailed error logging with specific error messages
- User-friendly error reporting with suggested solutions
- Rollback capability for critical failures
### Error Recovery
```applescript
on handleCalendarError(errorMessage)
-- Log error details
-- Provide user-friendly error message
-- Suggest corrective actions
end handleCalendarError
```
## Testing Strategy
### Unit Testing Approach
Since AppleScript has limited testing frameworks, testing will focus on:
1. **Manual Testing Scenarios**:
- Test with empty source calendar
- Test with events spanning multiple days
- Test with all-day events
- Test with recurring events
- Test calendar access errors
2. **Integration Testing**:
- Test full sync workflow with real calendar data
- Test cleanup functionality (event removal)
- Test duplicate detection accuracy
- Test error handling with invalid inputs
3. **Edge Case Testing**:
- Very long event titles and descriptions
- Events with special characters
- Overlapping events
- Events created/modified during sync
### Test Data Requirements
- Test calendars with known event sets
- Events with various properties (all-day, timed, recurring)
- Events with special characters and long descriptions
- Calendar accounts with different permission levels
## Implementation Considerations
### AppleScript Calendar Integration
- Use `Calendar` application's AppleScript dictionary
- Handle calendar app launch and focus management
- Manage calendar selection and event creation timing
### Performance Optimization
- Batch event operations where possible
- Minimize calendar app UI interactions
- Cache calendar references to avoid repeated lookups
### User Experience
- Provide clear progress indicators
- Show meaningful error messages
- Allow user to cancel long-running operations
- Display comprehensive sync results
### Security and Privacy
- Request calendar access permissions appropriately
- Handle sensitive calendar data securely
- Provide clear information about what data is accessed
## Configuration Management
### User Configuration
```applescript
-- Configuration properties
property sourceAccountName : "Work Account"
property sourceCalendarName : "Main Calendar"
property destinationAccountName : "Personal Account"
property destinationCalendarName : "Synced Events"
property enableLogging : true
```
### Runtime Configuration
- Allow users to modify calendar names without editing script
- Provide configuration validation before sync starts
- Save user preferences for repeated use

View File

@@ -1,73 +0,0 @@
# Requirements Document
## Introduction
This feature enables automatic synchronization of calendar entries between two different calendar accounts using AppleScript. The system will copy events from a source calendar account to a destination calendar account, maintaining event details while avoiding duplicates and providing configurable sync options.
## Requirements
### Requirement 1
**User Story:** As a user with multiple calendar accounts, I want to sync events from one account to another, so that I can maintain consistent scheduling across different calendar systems.
#### Acceptance Criteria
1. WHEN the sync script is executed THEN the system SHALL read all events from the specified source calendar
2. WHEN events are found in the source calendar THEN the system SHALL copy them to the specified destination calendar
3. WHEN copying events THEN the system SHALL preserve event title, date, time, duration, and description
4. IF an event already exists in the destination calendar THEN the system SHALL skip creating a duplicate
### Requirement 2
**User Story:** As a user, I want to configure which calendars to sync between, so that I can control the data flow between my accounts.
#### Acceptance Criteria
1. WHEN configuring the sync THEN the system SHALL allow selection of source calendar account and specific calendar
2. WHEN configuring the sync THEN the system SHALL allow selection of destination calendar account and specific calendar
3. WHEN invalid calendar names are provided THEN the system SHALL display an error message and exit gracefully
4. WHEN calendar accounts are not accessible THEN the system SHALL provide clear error messaging
### Requirement 3
**User Story:** As a user, I want the sync to focus on today's events only, so that I maintain current day synchronization without overwhelming the destination calendar.
#### Acceptance Criteria
1. WHEN the sync runs THEN the system SHALL only process events occurring on the current day
2. WHEN determining current day THEN the system SHALL use the local system date
3. WHEN events span multiple days THEN the system SHALL include events that start or occur on the current day
4. WHEN no events exist for the current day THEN the system SHALL complete successfully with appropriate messaging
### Requirement 4
**User Story:** As a user, I want the destination calendar to mirror the source calendar for the current day, so that removed events are also cleaned up automatically.
#### Acceptance Criteria
1. WHEN checking for duplicates THEN the system SHALL compare event title, start date, and start time
2. WHEN a matching event is found in the destination THEN the system SHALL skip creating the duplicate
3. WHEN an event exists in the destination but not in the source for the current day THEN the system SHALL remove it from the destination
4. WHEN an event has been modified in the source THEN the system SHALL update the corresponding event in the destination
### Requirement 5
**User Story:** As a user, I want to see progress and results of the sync operation, so that I can verify the synchronization was successful.
#### Acceptance Criteria
1. WHEN the sync starts THEN the system SHALL display the source and destination calendar information
2. WHEN processing events THEN the system SHALL show progress indicators for each event processed
3. WHEN the sync completes THEN the system SHALL display a summary of events copied, skipped, and any errors
4. WHEN errors occur THEN the system SHALL log detailed error information for troubleshooting
### Requirement 6
**User Story:** As a user, I want the sync to handle different event types and properties, so that all my calendar data is accurately transferred.
#### Acceptance Criteria
1. WHEN syncing events THEN the system SHALL handle all-day events correctly
2. WHEN syncing events THEN the system SHALL preserve recurring event patterns when possible
3. WHEN syncing events THEN the system SHALL handle events with attendees and meeting details
4. WHEN event properties cannot be transferred THEN the system SHALL log which properties were skipped

View File

@@ -1,125 +0,0 @@
# Implementation Plan
- [x] 1. Set up project structure and configuration
- Create main AppleScript file with basic structure and configuration properties
- Define configuration properties for source and destination calendars
- Set up logging and error handling framework
- _Requirements: 2.1, 2.2, 5.1_
- [ ] 2. Implement Calendar Manager component
- [ ] 2.1 Create calendar validation and access functions
- Write functions to validate calendar account and calendar names exist
- Implement calendar object retrieval with error handling
- Create function to list available calendars for debugging
- _Requirements: 2.1, 2.2, 2.3, 2.4_
- [ ]* 2.2 Write unit tests for calendar access
- Create test scenarios for invalid calendar names
- Test calendar access permission handling
- _Requirements: 2.3, 2.4_
- [ ] 3. Implement Event Reader component
- [ ] 3.1 Create current day event retrieval function
- Write function to get today's date and create date range
- Implement event query for current day from source calendar
- Handle different event types (all-day, timed events)
- _Requirements: 1.1, 3.1, 3.2, 3.3_
- [ ] 3.2 Implement event property extraction
- Create function to extract event title, dates, description, location
- Handle event property access errors gracefully
- Parse recurring events for current day occurrences
- _Requirements: 1.3, 6.1, 6.2, 6.3_
- [ ]* 3.3 Write tests for event reading functionality
- Test event retrieval with various event types
- Test property extraction accuracy
- _Requirements: 1.1, 1.3_
- [ ] 4. Implement Event Processor component
- [ ] 4.1 Create event comparison and duplicate detection
- Write function to compare events by title, start date, and start time
- Implement duplicate detection logic for existing events
- Handle event matching edge cases
- _Requirements: 4.1, 4.2_
- [ ] 4.2 Implement event creation and update functions
- Create function to add new events to destination calendar
- Implement event update functionality for modified events
- Handle event creation errors and property limitations
- _Requirements: 1.2, 1.3, 4.4, 6.4_
- [ ]* 4.3 Write tests for event processing
- Test duplicate detection accuracy
- Test event creation with various properties
- _Requirements: 4.1, 4.2, 4.4_
- [ ] 5. Implement Sync Engine component
- [ ] 5.1 Create main synchronization workflow
- Implement the main sync function that coordinates all components
- Add progress tracking and user feedback during sync
- Handle sync workflow errors and recovery
- _Requirements: 1.1, 1.2, 5.2, 5.3_
- [ ] 5.2 Implement cleanup functionality for removed events
- Create function to identify events in destination not in source
- Implement event removal from destination calendar
- Add safety checks to prevent accidental deletions
- _Requirements: 4.3_
- [ ] 5.3 Create sync reporting and statistics
- Implement sync result tracking (created, updated, removed, skipped)
- Create summary display function with detailed results
- Add error reporting and logging
- _Requirements: 5.3, 5.4_
- [ ]* 5.4 Write integration tests for sync engine
- Test complete sync workflow with test data
- Test cleanup functionality accuracy
- Test error handling and recovery
- _Requirements: 4.3, 5.3_
- [ ] 6. Implement Logger component
- [ ] 6.1 Create logging and progress display functions
- Implement message logging with different severity levels
- Create progress indicator for sync operations
- Add user-friendly error message formatting
- _Requirements: 5.1, 5.2, 5.4_
- [ ] 6.2 Implement sync summary display
- Create formatted summary of sync results
- Display statistics for events processed
- Show any errors or warnings encountered
- _Requirements: 5.3, 5.4_
- [ ] 7. Integrate all components and create main script
- [ ] 7.1 Wire together all components in main execution flow
- Create main script entry point that calls all components
- Implement proper error handling and user feedback flow
- Add configuration validation before sync starts
- _Requirements: 2.3, 2.4, 5.1_
- [ ] 7.2 Add user interaction and configuration management
- Implement user prompts for calendar selection if needed
- Add configuration validation and error messaging
- Create user-friendly script execution experience
- _Requirements: 2.1, 2.2, 2.3, 2.4_
- [ ]* 7.3 Create comprehensive end-to-end tests
- Test complete sync workflow with real calendar data
- Test all error scenarios and edge cases
- Validate sync accuracy and cleanup functionality
- _Requirements: 1.1, 1.2, 4.3, 5.3_
- [ ] 8. Finalize and optimize the script
- [ ] 8.1 Add performance optimizations and error recovery
- Optimize calendar access and event processing performance
- Add robust error recovery and rollback capabilities
- Implement proper resource cleanup and calendar app management
- _Requirements: 5.4_
- [ ] 8.2 Create user documentation and usage instructions
- Write clear instructions for script configuration and usage
- Document calendar permission requirements
- Create troubleshooting guide for common issues
- _Requirements: 2.3, 2.4, 5.4_

View File

@@ -1,165 +0,0 @@
# Design Document
## Overview
The `fastpass-additional-control-plane` Ansible role will enable the deployment of additional control plane nodes to an existing FastPass Kubernetes cluster. This role follows the established patterns from `fastpass-first-control-plane` but focuses on joining nodes to an already initialized cluster rather than initializing a new one. The role ensures high availability by creating redundant master nodes that can handle API requests, scheduling, and cluster management tasks.
The key difference from the first control plane role is that this role will use `kubeadm join` with control plane flags instead of `kubeadm init`, and it will need to retrieve join tokens and certificate keys from the existing cluster.
## Architecture
### Role Structure
The role will follow the standard Ansible role structure:
```
ansible/playbooks/roles/fastpass-additional-control-plane/
├── defaults/
│ └── main.yml
├── tasks/
│ └── main.yml
├── handlers/
│ └── main.yml (if needed)
└── meta/
└── main.yml (if needed)
```
### Integration Points
- **DNS Management**: Uses the existing `dns-manager` role for consistent DNS record creation
- **Kubeconfig Management**: Uses the existing `kubeconfig-manager` role for local kubeconfig setup
- **Firewall Configuration**: Reuses firewall service definitions from the first control plane role
- **Cluster Integration**: Coordinates with the first control plane node to obtain join credentials
### Dependencies
- The first control plane node must be fully initialized and running
- The `dns-manager` role must be available for DNS record creation
- The `kubeconfig-manager` role must be available for kubeconfig setup
- Required Kubernetes prerequisites must be installed on target nodes
## Components and Interfaces
### Main Task Flow
1. **Pre-flight Checks**: Verify cluster readiness and node prerequisites
2. **DNS Configuration**: Set up DNS records for the new control plane node
3. **Firewall Configuration**: Open required ports for control plane services
4. **Kubelet Configuration**: Create initial kubelet configuration
5. **Join Token Retrieval**: Get join token and certificate key from first control plane
6. **Cluster Join**: Execute kubeadm join with control plane flags
7. **Service Management**: Ensure kubelet is enabled and running
8. **Kubeconfig Setup**: Configure local kubeconfig access
9. **Verification**: Validate successful cluster join
### Key Variables
- `cluster_name`: Name of the Kubernetes cluster
- `ip_address`: IP address of the current control plane node
- `first_control_plane_host`: Hostname/IP of the first control plane node
- `kubernetes_services_control_plane`: List of firewall services to open
- `join_token_ttl`: TTL for join tokens (default: 24h)
- `certificate_key_ttl`: TTL for certificate keys (default: 2h)
### External Role Interfaces
- **dns-manager**: Provides DNS record creation with `host_name` variable
- **kubeconfig-manager**: Handles kubeconfig merging with `cluster_name` variable
- **First Control Plane**: Source for join tokens and certificate keys
## Data Models
### Join Credentials Structure
```yaml
join_credentials:
token: "abcdef.1234567890abcdef"
discovery_token_ca_cert_hash: "sha256:..."
certificate_key: "..."
api_server_endpoint: "cluster-name:6443"
```
### Firewall Services
```yaml
kubernetes_services_control_plane:
- kubernetes_API # Port 6443
- etcd # Ports 2379-2380
- kubelet # Port 10250
- kube-scheduler # Port 10259
- kube-controller-manager # Port 10257
```
### Node Status Tracking
```yaml
node_status:
joined: false
kubelet_running: false
dns_configured: false
kubeconfig_ready: false
```
## Error Handling
### Join Token Management
- **Token Expiration**: Automatically generate new tokens if existing ones are expired
- **Certificate Key Rotation**: Handle certificate key expiration gracefully
- **Network Connectivity**: Retry join operations with exponential backoff
- **API Server Availability**: Wait for API server readiness before attempting join
### Idempotency Checks
- **Already Joined Nodes**: Skip join process if node is already part of the cluster
- **Existing Configuration**: Preserve existing kubelet configuration if valid
- **DNS Records**: Update existing DNS records instead of creating duplicates
- **Service Status**: Only restart services if configuration changes
### Failure Recovery
- **Partial Join Failures**: Clean up partial configurations and retry
- **Network Issues**: Provide clear error messages for connectivity problems
- **Permission Errors**: Validate sudo/root access before attempting operations
- **Resource Constraints**: Check system resources before proceeding
## Testing Strategy
### Unit Testing Approach
- **Task Validation**: Test individual tasks with mock data
- **Variable Validation**: Ensure required variables are properly defined
- **Conditional Logic**: Test all conditional branches in tasks
- **Error Scenarios**: Validate error handling for common failure cases
### Integration Testing
- **Multi-Node Clusters**: Test with 3 and 5 control plane node configurations
- **Network Scenarios**: Test across different network topologies
- **OS Compatibility**: Validate on supported operating systems (Ubuntu/Debian)
- **Version Compatibility**: Test with different Kubernetes versions
### Validation Checks
- **Cluster Health**: Verify all control plane nodes are healthy after join
- **API Availability**: Confirm API server is accessible from all nodes
- **Etcd Cluster**: Validate etcd cluster membership and health
- **Scheduling**: Test pod scheduling across all control plane nodes
- **Failover**: Verify cluster continues operating if one control plane fails
### Test Scenarios
1. **Fresh Join**: Join additional control plane to newly created cluster
2. **Existing Cluster**: Add control plane to cluster with existing workloads
3. **Network Partitions**: Test behavior during temporary network issues
4. **Token Expiration**: Handle expired join tokens gracefully
5. **Retry Operations**: Validate retry logic for transient failures
## Implementation Considerations
### Security
- **Token Security**: Ensure join tokens are handled securely and not logged
- **Certificate Management**: Properly manage and rotate certificate keys
- **Network Security**: Validate firewall rules don't expose unnecessary ports
- **Access Control**: Ensure proper RBAC is maintained after node joins
### Performance
- **Parallel Execution**: Support joining multiple control plane nodes simultaneously
- **Resource Usage**: Monitor CPU and memory usage during join process
- **Network Bandwidth**: Optimize data transfer during cluster join
- **Startup Time**: Minimize time to achieve cluster readiness
### Monitoring and Observability
- **Join Progress**: Provide clear progress indicators during join process
- **Health Checks**: Implement comprehensive health validation
- **Logging**: Ensure adequate logging for troubleshooting
- **Metrics**: Expose relevant metrics for monitoring cluster growth
### Compatibility
- **Kubernetes Versions**: Support current and previous Kubernetes versions
- **Operating Systems**: Maintain compatibility with Ubuntu and Debian
- **Container Runtimes**: Work with containerd runtime configuration
- **Network Plugins**: Compatible with Flannel CNI configuration

View File

@@ -1,77 +0,0 @@
# Requirements Document
## Introduction
This feature involves creating an Ansible role called `fastpass-additional-control-plane` that will deploy additional control plane nodes to an existing FastPass Kubernetes cluster. The role will follow the same pattern as the existing `fastpass-first-control-plane` role but will focus on joining nodes to an already initialized cluster rather than initializing a new cluster. This ensures high availability for the Kubernetes control plane by adding redundant master nodes.
## Requirements
### Requirement 1
**User Story:** As a DevOps engineer, I want to deploy additional control plane nodes to my FastPass Kubernetes cluster, so that I can achieve high availability and fault tolerance for the cluster control plane.
#### Acceptance Criteria
1. WHEN the role is executed on a node THEN the system SHALL join the node to the existing Kubernetes cluster as a control plane node
2. WHEN the role runs THEN the system SHALL configure the necessary firewall rules for control plane services
3. WHEN the role executes THEN the system SHALL ensure the kubelet service is properly configured and running
4. WHEN joining the cluster THEN the system SHALL use the correct join token and certificate key from the first control plane node
5. WHEN the role completes THEN the system SHALL verify the node has successfully joined as a control plane node
### Requirement 2
**User Story:** As a system administrator, I want the additional control plane role to follow the same patterns as the first control plane role, so that the codebase remains consistent and maintainable.
#### Acceptance Criteria
1. WHEN the role is created THEN the system SHALL follow the same directory structure as fastpass-first-control-plane
2. WHEN the role is implemented THEN the system SHALL use similar variable naming conventions and task organization
3. WHEN the role runs THEN the system SHALL include proper error handling and idempotency checks
4. WHEN the role executes THEN the system SHALL use the same firewall service definitions as the first control plane role
5. WHEN the role is documented THEN the system SHALL include proper metadata headers with author, version, and description
### Requirement 3
**User Story:** As a cluster operator, I want the additional control plane nodes to have proper DNS configuration, so that they can be reached by their cluster names and participate in load balancing.
#### Acceptance Criteria
1. WHEN the role runs THEN the system SHALL configure DNS records for the additional control plane nodes
2. WHEN DNS is configured THEN the system SHALL use the dns-manager role for consistency
3. WHEN the role executes THEN the system SHALL ensure the node can resolve the cluster endpoint
4. WHEN DNS setup completes THEN the system SHALL verify connectivity to the cluster API endpoint
### Requirement 4
**User Story:** As a DevOps engineer, I want the role to handle kubeconfig management for additional control plane nodes, so that I can manage the cluster from any control plane node.
#### Acceptance Criteria
1. WHEN the role completes THEN the system SHALL configure kubeconfig for the new control plane node
2. WHEN kubeconfig is set up THEN the system SHALL use the kubeconfig-manager role for consistency
3. WHEN the role runs THEN the system SHALL ensure proper permissions are set on kubeconfig files
4. WHEN kubeconfig is configured THEN the system SHALL verify kubectl access works from the new node
### Requirement 5
**User Story:** As a system administrator, I want the role to be idempotent and handle edge cases, so that I can run it multiple times safely without causing issues.
#### Acceptance Criteria
1. WHEN the role is run multiple times THEN the system SHALL not attempt to rejoin an already joined node
2. WHEN a node is already part of the cluster THEN the system SHALL skip the join process gracefully
3. WHEN the role encounters errors THEN the system SHALL provide clear error messages and fail gracefully
4. WHEN prerequisites are missing THEN the system SHALL report what needs to be configured first
5. WHEN the role runs THEN the system SHALL validate that required variables are defined
### Requirement 6
**User Story:** As a cluster administrator, I want the role to integrate seamlessly with the existing FastPass deployment workflow, so that it can be used in the 4-step deployment process.
#### Acceptance Criteria
1. WHEN the role is created THEN the system SHALL be compatible with the deploy-fastpass-4step.yml playbook
2. WHEN the role runs THEN the system SHALL work with the fastpass_control_plane[1:] host group
3. WHEN integrated THEN the system SHALL not interfere with the first control plane initialization
4. WHEN the role executes THEN the system SHALL depend on the first control plane node being ready
5. WHEN deployment completes THEN the system SHALL allow worker nodes to join the cluster successfully

View File

@@ -1,118 +0,0 @@
# Implementation Plan
- [ ] 1. Create role directory structure and basic configuration
- Create the fastpass-additional-control-plane role directory structure
- Set up defaults/main.yml with required variables and firewall services
- Create meta/main.yml with role metadata and dependencies
- _Requirements: 2.1, 2.2, 2.4_
- [ ] 2. Implement join token and certificate key retrieval
- [ ] 2.1 Create tasks to generate new join tokens from first control plane
- Write Ansible tasks to execute kubeadm token create on first control plane node
- Implement token validation and expiration checking
- Add error handling for token generation failures
- _Requirements: 1.4, 5.4_
- [ ] 2.2 Implement certificate key retrieval and management
- Create tasks to upload and retrieve certificate keys from first control plane
- Add certificate key expiration handling and rotation
- Implement secure handling of certificate keys in variables
- _Requirements: 1.4, 5.1_
- [ ] 2.3 Create discovery token CA certificate hash retrieval
- Write tasks to extract CA certificate hash from first control plane
- Implement validation of certificate hash format
- Add error handling for certificate retrieval failures
- _Requirements: 1.4, 5.4_
- [ ] 3. Implement DNS configuration and firewall setup
- [ ] 3.1 Configure DNS records for additional control plane nodes
- Integrate dns-manager role for consistent DNS record creation
- Pass appropriate host_name variable to dns-manager
- Add DNS propagation wait and validation
- _Requirements: 3.1, 3.2, 3.4_
- [ ] 3.2 Set up firewall rules for control plane services
- Reuse kubernetes_services_control_plane from defaults
- Implement UFW firewall rule creation for Debian/Ubuntu systems
- Add conditional logic for different operating systems
- _Requirements: 1.2, 2.4_
- [ ] 4. Implement kubelet configuration and cluster join
- [ ] 4.1 Create initial kubelet configuration
- Write kubelet config.yaml with systemd cgroup driver
- Set containerd socket endpoint configuration
- Ensure proper file permissions and ownership
- _Requirements: 1.3, 2.3_
- [ ] 4.2 Execute kubeadm join for control plane
- Implement kubeadm join command with control-plane flag
- Use retrieved join token, certificate key, and CA cert hash
- Add proper command argument construction and validation
- Include idempotency checks to prevent duplicate joins
- _Requirements: 1.1, 1.4, 5.1, 5.2_
- [ ] 4.3 Ensure kubelet service management
- Enable and start kubelet systemd service
- Add service status validation and error handling
- Implement service restart logic if needed
- _Requirements: 1.3, 1.5_
- [ ] 5. Implement kubeconfig management and validation
- [ ] 5.1 Configure kubeconfig for additional control plane nodes
- Integrate kubeconfig-manager role for consistent configuration
- Pass cluster_name variable to kubeconfig-manager
- Ensure proper kubeconfig merging with existing configurations
- _Requirements: 4.1, 4.2, 4.3_
- [ ] 5.2 Implement cluster join validation
- Create tasks to verify node successfully joined as control plane
- Add kubectl commands to check node status and roles
- Implement cluster health validation checks
- _Requirements: 1.5, 4.4_
- [ ] 6. Add comprehensive error handling and idempotency
- [ ] 6.1 Implement pre-flight validation checks
- Check if node is already joined to cluster
- Validate required variables are defined
- Verify first control plane node accessibility
- Add system resource and prerequisite checks
- _Requirements: 5.1, 5.2, 5.4, 5.5_
- [ ] 6.2 Add retry logic and failure recovery
- Implement retry mechanisms for transient failures
- Add exponential backoff for network-related operations
- Create cleanup tasks for partial join failures
- _Requirements: 5.3, 5.4_
- [ ] 7. Integration with FastPass deployment workflow
- [ ] 7.1 Ensure compatibility with deploy-fastpass-4step.yml
- Verify role works with fastpass_control_plane[1:] host group
- Test integration with existing playbook structure
- Validate dependency on first control plane completion
- _Requirements: 6.1, 6.2, 6.3, 6.4_
- [ ] 7.2 Add proper task documentation and metadata
- Include role header with author, version, and description
- Add inline comments for complex task logic
- Document required variables and their purposes
- _Requirements: 2.2, 2.5_
- [ ]* 8. Create comprehensive testing and validation
- [ ]* 8.1 Write unit tests for individual tasks
- Create test cases for token retrieval logic
- Test kubeadm join command construction
- Validate error handling scenarios
- _Requirements: 1.1, 1.4, 5.1_
- [ ]* 8.2 Implement integration tests
- Test multi-node control plane deployment
- Validate cluster health after additional nodes join
- Test failover scenarios and cluster resilience
- _Requirements: 1.5, 6.5_
- [ ]* 8.3 Add validation scripts and health checks
- Create scripts to verify cluster state after deployment
- Implement automated health validation
- Add performance and resource usage monitoring
- _Requirements: 1.5, 4.4_

View File

@@ -1,51 +0,0 @@
# Requirements Document
## Introduction
The FastPass Kubernetes cluster deployment currently has critical DNS and load balancing configuration issues that prevent proper high availability setup. While the cluster endpoint `fastpass.local.mk-labs.cloud` is defined in the group variables, the `cluster_vip` variable required by the DNS manager role is missing, and the Traefik load balancer configuration is commented out. This means the cluster endpoint cannot resolve properly and there's no load balancing for the control plane API. This feature will fix these configuration gaps to enable true HA functionality.
## Requirements
### Requirement 1: DNS CNAME Record for Load Balancer
**User Story:** As a DevOps engineer, I want the DNS manager to create a CNAME record for the cluster endpoint pointing to the load balancer, so that the cluster endpoint resolves through the load balancer rather than directly to node IPs.
#### Acceptance Criteria
1. WHEN the dns-manager role is called for a load-balanced cluster THEN the system SHALL create a CNAME record instead of an A record
2. WHEN the CNAME record is created THEN the system SHALL point fastpass.local.mk-labs.cloud to the traefik_server (lightning_lane.local.mk-labs.cloud)
3. WHEN the DNS record type is determined THEN the system SHALL use CNAME for load-balanced endpoints and A records for direct node access
4. WHEN DNS propagation occurs THEN the system SHALL verify that the CNAME resolution works correctly
### Requirement 2: Complete Traefik Integration
**User Story:** As a DevOps engineer, I want the Traefik load balancer to be fully integrated with the FastPass deployment, so that the cluster VIP is properly load balanced across all control plane nodes.
#### Acceptance Criteria
1. WHEN the traefik-manager role is called THEN the system SHALL use the traefik_server variable (lightning_lane) as the target host
2. WHEN Traefik configuration is generated THEN the system SHALL create proper TCP routing for the cluster endpoint to all control plane nodes
3. WHEN the cluster VIP is accessed THEN the system SHALL distribute requests across space-mountain, big-thunder-mountain, and splash-mountain
4. WHEN Traefik configuration is applied THEN the system SHALL reload the Traefik service to activate the new configuration
### Requirement 3: High Availability Validation
**User Story:** As a DevOps engineer, I want to validate that the HA setup is working correctly, so that I can be confident the cluster will survive node failures.
#### Acceptance Criteria
1. WHEN the deployment completes THEN the system SHALL test connectivity to the cluster endpoint
2. WHEN connectivity tests run THEN the system SHALL verify that the endpoint resolves through the CNAME to the load balancer
3. WHEN load balancer tests run THEN the system SHALL verify that requests are being distributed across control plane nodes
4. WHEN a control plane node is stopped THEN the system SHALL continue to serve API requests through the remaining nodes
### Requirement 4: Backward Compatibility
**User Story:** As a DevOps engineer, I want the DNS fixes to be backward compatible with existing deployments, so that current clusters continue to function during the transition.
#### Acceptance Criteria
1. WHEN existing clusters are updated THEN the system SHALL not break existing DNS configurations
2. WHEN new variables are introduced THEN the system SHALL provide sensible defaults for existing deployments
3. WHEN the update is applied THEN the system SHALL preserve existing kubeconfig files and cluster access
4. IF migration issues occur THEN the system SHALL provide rollback procedures and documentation

3
.sops.yaml Normal file
View File

@@ -0,0 +1,3 @@
creation_rules:
- path_regex: talos/talhelper/talenv.yaml
age: age1xkyuv8r8ce6lu3d64jfspz4e50k6pxlaxwmuplzrtnpfnnrnycaq6mfsrn

287
README.md
View File

@@ -1,239 +1,102 @@
# A Homelab based on Red Hat Technologies
# mk-labs
A comprehensive homelab for learning advanced concepts primarily based on Red Hat OpenShift. In addition to providing services, it is an area to incorporate container technologies, and enterprise integration scenarios.
Automated infrastructure provisioning and configuration for a personal homelab, built on GitOps practices with clear tool responsibility boundaries.
This implementation is built on easily accessible consumer based hardware and will focus heavily on GitOps practices and automation will be used wherever possible.
## Architecture
## 🎯 Project Goals
- **Kubernetes cluster for applications**
- **IAM integration testing**
- **GitOps and automation workflows**
## 📋 Documentation (Priority Order)
### Phase 1: Foundation
1. **[Overview & Hardware Allocation](./docs/01-overview.md)** - Complete architecture overview
2. **[Networking Plan](./docs/02-networking.md)** - VLAN strategy and network design
3. **[Core Services Setup](./docs/03-core-services.md)** *(TBD)* - Ansible, DNS, monitoring
4. **[Storage Configuration](./docs/04-storage.md)** *(TBD)* - Synology, democratic-csi
### Phase 2: Proxmox Platform
5. **[Proxmox Installation](./docs/11-vsphere-setup.md)** *(TBD)* - Dell 7050 virtualization
6. **[VM Templates & Automation](./docs/13-vm-automation.md)** *(TBD)* - Template creation
### Phase 3: Infrastructure Services
7. **[Recursive DNS]
8. **[Authoritative DNS]
9. **[Identity Management](./docs/10-acm-setup.md)** *(TBD)*
10. **[Matchbox] (Depreciated?)
### Phase 4: OpenShift Clusters (Depreciated)
8. **[OpenShift SNO + Worker](./docs/09-openshift-sno.md)** *(TBD)* - ACM Hub Cluster
9. **[OpenShift Cluster](./docs/08-openshift-compact.md)** *(TBD)* - 3 Master/3 Worker node (production-like)
10. **[ACM Configuration](./docs/10-acm-setup.md)** *(TBD)* - Multi-cluster management
11. **[HCP Cluster](./docs/10-acm-setup.md)** *(TBD)* - External app cluster
### Phase 5: Kubernetes Container Platform
5. **[Container Registry Setup](./docs/05-container-registry.md)** *(TBD)* - Harbor deployment
6. **[Git Repository Setup](./docs/06-git-repository.md)** *(TBD)* - Gitea/GitLab on Synology
7. **[Artifact Repository](./docs/07-artifact-repository.md)** *(TBD)* - Nexus/Artifactory
### Phase 6: Advanced Services
14. **[Monitoring & Observability](./docs/14-monitoring.md)** *(TBD)* - Prometheus, Grafana, Splunk
15. **[Security & Compliance](./docs/15-security.md)** *(TBD)* - ACS, certificates, auditing
16. **[Backup & DR](./docs/16-backup-dr.md)** *(TBD)* - Backup strategies
## 🚀 Quick Start Deployment
### Prerequisites
- Ubiquiti UDM Pro configured
- Hardware powered and networked
- Initial VLAN setup (see [networking plan](./docs/02-networking.md))
### Ansible Collections
- ansible-galaxy collection install freeipa.ansible_freeipa
### Deployment Scripts
```bash
source ~/venv-ansible/bin/activate
# Phase 1: Foundation
./scripts/01-network-setup.sh
./scripts/02-core-services-deploy.sh
# Phase 2: Container Platform
./deployment/synology/docker-compose.yml # Git repo, Harbor, Nexus
./scripts/03-storage-setup.sh
# Phase 3: OpenShift
./deployment/openshift/compact-cluster/
./deployment/openshift/sno-cluster/
# Phase 4: Proxmox
./deployment/proxmox/
A single operator action — setting a VM record's status to **Staged** in NetBox — triggers a fully automated provisioning pipeline:
```
NetBox (webhook) → n8n (validate & orchestrate) → Terraform (create VM + DHCP)
→ Ansible (OS config + DNS + status update)
```
## 📁 Repository Structure
| Tool | Host | IP | Responsibility |
|------|------|----|---------------|
| NetBox | fire-station | 10.1.71.102 | Source of truth — VM records, IP allocation, VLAN data |
| n8n | tiki-room | 10.1.71.23 | Event orchestration, validation, pipeline sequencing |
| Terraform | city-hall | 10.1.71.35 | Proxmox VM lifecycle, Unifi DHCP reservations |
| Ansible / Semaphore | imagineering | 10.1.71.22 | OS configuration, DNS records, NetBox status updates |
| Proxmox | fantasyland | 10.1.71.13 | Target hypervisor |
All systems on the Server Trusted VLAN (10.1.71.0/24).
## Repository Structure
```
homelab/
├── README.md # This file
├── docs/ # Documentation (numbered by priority)
│ ├── 01-overview.md
│ ├── 02-networking.md
── ...
├── ansible/ # Directory structure for Ansible
├── deployment/ # Deployment
configurations
── synology/ # Docker Compose files for Synology
├── k0s/ # k0s cluster manifests
│ ├── openshift/ # OpenShift installation configs
── proxmox/ # Proxmox automation
├── scripts/ # Automation scripts
└── .gitignore # Excludes sensitive data
├── ansible/
│ ├── inventory/ # NetBox dynamic inventory + static
│ ├── playbooks/ # Runnable playbooks (vm-provision, DNS, OS updates)
│ ├── roles/ # vm-baseline, dns-manager, common, haproxy, n8n, observer, etc.
── tasks/ # Shared includable task files
│ ├── group_vars/ # Group variable definitions
│ ├── host_vars/ # Per-host variable definitions
│ ├── templates/ # Jinja2 templates
── ansible.cfg
├── terraform/
── proxmox/vm/ # bpg/proxmox provider — VM creation from templates
│ ├── unifi/dhcp/ # Unifi provider — DHCP static reservations on UDM Pro
│ └── dns/ # DNS record management
├── packer/
│ ├── ubuntu-24.04/ # Ubuntu 24.04 VM template (small → xlarge-plus sizes)
│ └── fedora-42/ # Fedora 42 VM template
├── n8n/
│ └── workflows/ # Exported n8n workflow JSON (vm-provisioning)
├── netbox/
│ └── initializers/ # Custom fields, VLANs, IP prefixes as code
└── docs/
└── decisions/ # Architecture decision records
```
## 🔧 Technology Stack
## Pipeline Flow
### Infrastructure
- **Networking**: Ubiquiti UDM Pro
- **Storage**: Synology DS1621+, Ubiquiti UNAS Pro
- **Compute**: 3x Minisforum TH60, 2x Minisforum MS01, 7x Dell 7050 SFF
| # | System | Action |
|---|--------|--------|
| 1 | NetBox | Operator sets VM status to Staged → webhook fires |
| 2 | n8n | Validates payload (hostname, IP, VLAN, template, proxmox_node) |
| 3 | n8n → city-hall | SSH + `terraform apply` — creates VM on Proxmox |
| 4 | n8n | Queries Proxmox API for MAC address |
| 5 | n8n → NetBox | Writes MAC to VM interface record |
| 6 | n8n → city-hall | SSH + `terraform apply` — creates DHCP reservation on UDM Pro |
| 7 | n8n → imagineering | Triggers Ansible via Semaphore API |
| 8 | Ansible | OS baseline, SSH hardening, Technitium DNS A record |
| 9 | Ansible → NetBox | Sets VM status to Active |
### Container Platforms
- **k8s**: Core services cluster
- **Proxmox**: VM workloads
### Core Services
- **Container Registry**: Harbor
- **Git Repository**: Gitea/GitLab CE
- **Artifact Repository**: Nexus/Artifactory
- **Monitoring**: Prometheus + Grafana
- **Logging**: Splunk Enterprise
- **DNS**: CoreDNS
- **Load Balancing**: MetalLB + HAProxy
## 🔐 Security Notes
- **No sensitive data** is stored in this repository
- **Secrets management** via external-secrets-operator
- **Certificate management** via cert-manager
- **Network segmentation** via VLANs and firewall rules
## 🤝 Contributing
This is a personal homelab project, but feel free to:
- Submit issues for questions or suggestions
- Fork for your own homelab adaptations
- Share improvements via pull requests
## 📞 Next Steps
1. **Create GitHub repository** for version control
2. **Start with Phase 1** foundation setup
3. **Build deployment automation** as we go
4. **Document lessons learned** for future reference
---
**Status**: 🚧 Planning & Initial Development
**Last Updated**: June 2025
This repo is a mono-repo that is broken up into three sections:
- infra-config
- apps
- cluster
On any failure, NetBox status is set to **Failed**. No auto-retry — operator investigates.
## Hardware
- Dell 7050 SFF (7)
- Minisforum TH60 (3)
- Minisforum MS01 (2)
- Synology 1621+
- 7× Dell 7050 SFF
- 3× Minisforum TH60
- 2× Minisforum MS01
- Synology DS1621+
- Ubiquiti UDM Pro
## Software
## Software Stack
- Proxmox (Virtualization)
- Cloudflare (Domain Hosting, Public DNS)
- Unbound (Recursive DNS)
- FreeIPA (Identity management, Authoritive DNS)
- Matchbox (iPXE)
- Red Hat OpenShift
- OpenShift Agent Based Installer (Install OpenShift)
- [Red Hat Advanced Cluster Management for Kubernetes](https://www.redhat.com/en/technologies/management/advanced-cluster-management)
- [Vault](https://www.hashicorp.com/en/products/vault)
- [OpenShift GitOps (ArgoCD)](https://www.redhat.com/en/technologies/cloud-computing/openshift/gitops)
- [Red Hat Ansible Automation Platform](https://www.redhat.com/en/technologies/management/ansible)
## Prerequisites
- Ansible user created
- Ansible configured
- [Networking](docs/networks.md)
- [Proxmox](docs/proxmox.md) (In my homelab, internal DNS, identity manangement, and ipxe are hosted here.)
- Matchbox
- DNS
- Domain Registration
## Assumptions
There are a dozen different architectures you could use to deploy OpenShift in every which way.
For the sake of this documentation we'll assume the following:
- **Virtualization**: Proxmox
- **Automation**: Terraform, Ansible, n8n, Semaphore
- **DNS**: Technitium (authoritative), Unbound (recursive)
- **IPAM/DCIM**: NetBox
- **Networking**: Ubiquiti UDM Pro
- **Templates**: Packer (Ubuntu 24.04, Fedora 42)
## Getting Started
[Step-by-Step Walkthrough](step-by-step.md)
See [docs/decisions/vm-provisioning-flow.md](docs/decisions/vm-provisioning-flow.md) for the full architecture decision record.
### Hub Cluster
Previous OpenShift/ACM/Fastpass content is preserved in the `archive/pre-mk-labs` branch.
You'll need an OpenShift "Hub Cluster" with access to persistant storage.
A Single Node OpenShift (SNO) instance, installed on bare metal, will act as a Hub cluster and run:
## Security
- Advanced Cluster Management
- Ansible Automation Platform
- Vault
- ~~OpenShift GitOps~~
#### Network Prerequisites
The prerequisites for OpenShift in traditional and HCP patterns are largely the same - it just kind of depends on where your DNS records go to.
| Cluster | Endpoint | VIP | DNS A Record | Notes |
|------------------|-------------|---------------|-----------------------------------|----------------------------------|
| Hub Cluster (SNO) | App Ingress | 192.168.0.10 | *.apps.hub-cluster.example.com | SNO App VIP goes to IP of node |
| Hub Cluster (SNO) | API | 192.168.0.10 | api.hub-cluster.example.com | SNO API goes to IP of node |
These DNS entries should be put in your Authoratitive DNS.
#### ACM & GitOps Configuration
Before you start creating clusters you may want to create some Policies, integrate ACM and ArgoCD, etc. This step is optional in case you're just interested in trying out Hosted Control Planes or copy/paste around a cluster for testing purposes.
Find additional details in the ./02-rhacm-config folder.
#### Creating a Cluster
With everything in its right place, you can now start to declaratively create clusters
./05-clusters/hcp-bmh - HCP to Bare Metal Hosts
### Internal Cluster
Two additional bare metal nodes, to be added to Advanced Cluster Management (ACM) running on the SNO Hub. These will be used to create another HCP cluster.
These servers have a BMC interface with Redfish - if not, then you'll need to manually manage the boot and installation of those servers.
This makes it to where you just need 3 bare metal nodes. You could run one HCP Bare Metal cluster with both of the other nodes, but then you have a shared storage requirement that can't be satisfied by ODF since that needs at least 3 nodes.
### External Cluster
You'll also either need you just need at least 2 bare metal nodes.
No sensitive data is stored in this repository. Secrets are managed via Ansible Vault and environment variables on pipeline hosts.
---
## Credits
**Status**: 🚧 Active Development — VM Provisioning Pipeline
- Ken Moini - As I used his [repo](https://github.com/kenmoini/ztp-for-you-and-me) as the baseline for this project.
- Ryan Etten
- Andrew Potozniak
**Last Updated**: February 2026

View File

@@ -1,309 +0,0 @@
# FastPass Kubernetes Cluster - Complete Deployment Guide
## Overview
This guide provides step-by-step instructions for deploying a complete Kubernetes cluster on Fedora using Ansible automation. The FastPass cluster is designed for high availability with multiple control plane nodes and worker nodes.
## Prerequisites
### System Requirements
- **OS**: Fedora 38+ (64-bit)
- **Memory**: Minimum 2GB RAM per node
- **CPU**: Minimum 2 cores per node
- **Storage**: Minimum 10GB free space
- **Network**: All nodes must be able to communicate
### Software Requirements
- Ansible 2.15+
- Python 3.8+
- SSH access to all nodes with sudo privileges
## Quick Start
### 1. Validate Your Setup
Before deploying, run the validation script:
```bash
cd ansible
./scripts/test-fastpass-deployment.sh
```
This will check:
- ✅ Ansible installation
- ✅ Inventory configuration
- ✅ Playbook files
- ✅ Role files
- ✅ Group variables
- ✅ Playbook syntax
- ✅ Common issues
### 2. Deploy the Cluster
```bash
# Deploy the complete cluster
ansible-playbook -i inventory.yml playbooks/deploy-fastpass-cluster.yml
```
## Deployment Process
The deployment follows this sequence:
### Phase 1: System Preparation
- **Hosts**: All FastPass nodes
- **Tasks**:
- Preflight checks (OS, memory, CPU, disk space)
- Disable swap and configure kernel modules
- Install and configure containerd
- Install Kubernetes packages
### Phase 2: Control Plane Initialization
- **Hosts**: First control plane node (`space-mountain`)
- **Tasks**:
- Initialize the Kubernetes cluster
- Configure API server and etcd
- Install Flannel CNI
- Generate join commands
### Phase 3: Additional Control Plane Nodes
- **Hosts**: Remaining control plane nodes (`big-thunder-mountain`, `splash-mountain`)
- **Tasks**:
- Join additional control plane nodes
- Configure high availability
### Phase 4: Worker Nodes
- **Hosts**: Worker nodes (`haunted-mansion`, `peter-pans-flight`)
- **Tasks**:
- Join worker nodes to the cluster
- Apply node labels and taints
### Phase 5: Validation
- **Hosts**: First control plane node
- **Tasks**:
- Verify all nodes are ready
- Check all pods are running
- Display final cluster status
## Configuration
### Inventory Structure
Ensure your `inventory.yml` has the correct structure:
```yaml
fastpass_control_plane:
hosts:
space-mountain:
big-thunder-mountain:
splash-mountain:
fastpass_workers:
hosts:
haunted-mansion:
peter-pans-flight:
fastpass:
children:
fastpass_control_plane:
fastpass_workers:
```
### Group Variables
Key variables in `group_vars/fastpass/vars`:
```yaml
# Cluster Configuration
cluster_name: "fastpass"
kubernetes_version: "1.33"
pod_network_cidr: "10.244.0.0/16"
service_cidr: "10.96.0.0/12"
# Control Plane Configuration
control_plane_endpoint: "fastpass.local.mk-labs.cloud"
control_plane_port: "6443"
# CNI Configuration
cni_plugin: "flannel"
# Node Labels
node_labels:
zone:
# Control Plane nodes (masters) - no zone labels needed
# space-mountain: (control plane - no zone label)
# big-thunder-mountain: (control plane - no zone label)
# splash-mountain: (control plane - no zone label)
# Worker nodes
haunted-mansion: "backstage" # Internal/backstage workloads
peter-pans-flight: "backstage" # Internal/backstage workloads
# Future node: "on-stage" # External/on-stage workloads (future)
```
## Troubleshooting
### Common Issues
#### 1. Swap Enabled
**Problem**: `running with swap on is not supported`
**Solution**: The playbook automatically disables swap, but if it persists:
```bash
# On the problematic node
sudo swapoff -a
sudo systemctl restart kubelet
```
#### 2. CNI Not Ready
**Problem**: `Network plugin returns error: cni plugin not initialized`
**Solution**: The playbook installs Flannel automatically. If issues persist:
```bash
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
```
#### 3. Control Plane Endpoint Issues
**Problem**: `unable to add a new control plane instance to a cluster that doesn't have a stable controlPlaneEndpoint`
**Solution**: The playbook uses the first control plane node as the endpoint. For production, consider using a load balancer.
#### 4. Permission Issues
**Problem**: `User "kubernetes-admin" cannot create resource "secrets"`
**Solution**: The playbook uses super-admin.conf for proper permissions.
### Debug Commands
```bash
# Check node status
kubectl get nodes -o wide
# Check pod status
kubectl get pods --all-namespaces
# Check kubelet logs
journalctl -u kubelet -f
# Check API server logs
kubectl logs -n kube-system kube-apiserver-space-mountain
# Check CNI status
kubectl get pods -n kube-flannel
```
## Post-Deployment
### 1. Verify Cluster Health
```bash
# Check all nodes are ready
kubectl get nodes -o wide
# Check all pods are running
kubectl get pods --all-namespaces
# Test cluster connectivity
kubectl cluster-info
```
### 2. Manage Kubeconfig
Use the provided script to manage different cluster configurations:
```bash
# Switch to FastPass cluster
./scripts/manage-kubeconfigs.sh fastpass
# Check status
./scripts/manage-kubeconfigs.sh status
# Test connectivity
./scripts/manage-kubeconfigs.sh test
```
### 3. Deploy Applications
Your cluster is now ready for workloads:
```bash
# Deploy a test application
kubectl run nginx --image=nginx --port=80
# Create a service
kubectl expose pod nginx --port=80 --type=NodePort
# Check the service
kubectl get svc nginx
```
## Maintenance
### Adding New Nodes
1. Add the new node to the inventory
2. Run the preparation role:
```bash
ansible-playbook -i inventory.yml playbooks/deploy_k8s.yml --limit=new-node
```
3. Join the node to the cluster manually or extend the playbook
### Upgrading Kubernetes
1. Update the `kubernetes_version` variable
2. Run the preparation role on all nodes
3. Upgrade control plane nodes first
4. Upgrade worker nodes
### Backup and Recovery
- Backup `/etc/kubernetes/` directory on control plane nodes
- Backup etcd data: `etcdctl snapshot save backup.db`
- Document cluster configuration
## Architecture
```
FastPass Cluster Architecture:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Control Plane │ │ Control Plane │ │ Control Plane │
│ space-mountain │ │big-thunder-mtn │ │ splash-mountain │
│ (Masters) │ │ (Masters) │ │ (Masters) │
│ No Zone Label │ │ No Zone Label │ │ No Zone Label │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
└───────────────────────┼───────────────────────┘
┌───────────────────────┼───────────────────────┐
│ │ │
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Worker Node │ │ Worker Node │ │ Worker Node │
│ haunted-mansion │ │peter-pans-flight│ │ (Future) │
│ (Backstage) │ │ (Backstage) │ │ (On-Stage) │
│ zone=backstage │ │ zone=backstage │ │ zone=on-stage │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
## Support
For issues specific to this deployment:
1. Check the troubleshooting section
2. Review Ansible logs with `-vvv` verbosity
3. Check system logs on affected nodes
4. Verify network connectivity between nodes
## Files Structure
```
ansible/
├── group_vars/
│ └── fastpass/
│ └── vars # Cluster configuration
├── playbooks/
│ ├── deploy-fastpass-cluster.yml # Main deployment playbook
│ ├── deploy_k8s.yml # Legacy deployment playbook
│ └── roles/
│ ├── kubernetes/ # System preparation
│ ├── fastpass-control-plane/ # Control plane setup
│ ├── fastpass-control-plane-join/ # Additional control plane nodes
│ └── fastpass-workers/ # Worker node setup
├── scripts/
│ ├── test-fastpass-deployment.sh # Validation script
│ └── manage-kubeconfigs.sh # Kubeconfig management
└── inventory.yml # Host inventory
```

View File

@@ -1,228 +0,0 @@
# FastPass Kubernetes Cluster - Fedora Deployment Guide
## Overview
This guide covers the deployment of a Kubernetes cluster on Fedora using Ansible automation. The FastPass cluster is designed for high availability with multiple control plane nodes and worker nodes.
## Architecture
```
FastPass Cluster Architecture:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Control Plane │ │ Control Plane │ │ Control Plane │
│ space-mountain │ │big-thunder-mtn │ │ splash-mountain │
│ (Leader) │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
└───────────────────────┼───────────────────────┘
┌───────────────────────┼───────────────────────┐
│ │ │
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Worker Node │ │ Worker Node │ │ Worker Node │
│ haunted-mansion │ │peter-pans-flight│ │ (Future) │
│ (backstage) │ │ (backstage) │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
## Prerequisites
### System Requirements
- **OS**: Fedora 38+ (64-bit)
- **Memory**: Minimum 2GB RAM per node
- **CPU**: Minimum 2 cores per node
- **Storage**: Minimum 10GB free space
- **Network**: All nodes must be able to communicate
### Software Requirements
- Ansible 2.15+
- Python 3.8+
- SSH access to all nodes with sudo privileges
## Quick Start
### 1. Prepare Inventory
Ensure your `inventory.yml` has the correct host groups:
```yaml
fastpass_control_plane:
hosts:
space-mountain:
big-thunder-mountain:
splash-mountain:
fastpass_workers:
hosts:
haunted-mansion:
peter-pans-flight:
fastpass:
children:
fastpass_control_plane:
fastpass_workers:
```
### 2. Configure Variables
Edit `group_vars/fastpass/vars` to customize your deployment:
```yaml
# Cluster Configuration
cluster_name: "fastpass"
kubernetes_version: "1.33"
pod_network_cidr: "10.244.0.0/16"
service_cidr: "10.96.0.0/12"
# Node Labels
node_labels:
zone:
space-mountain: "external"
big-thunder-mountain: "external"
splash-mountain: "external"
haunted-mansion: "internal"
peter-pans-flight: "internal"
```
### 3. Deploy the Cluster
```bash
# Deploy the complete cluster
ansible-playbook -i inventory.yml playbooks/deploy-fastpass-cluster.yml
# Or deploy step by step
ansible-playbook -i inventory.yml playbooks/deploy_k8s.yml
```
## Deployment Process
### Phase 1: System Preparation
- Preflight checks (OS, memory, CPU, disk space)
- Disable swap and configure kernel modules
- Install and configure containerd
- Install Kubernetes packages
### Phase 2: Control Plane Setup
- Initialize the first control plane node
- Configure API server and etcd
- Install Calico CNI
- Join additional control plane nodes
### Phase 3: Worker Node Setup
- Join worker nodes to the cluster
- Apply node labels and taints
- Verify cluster health
## Troubleshooting
### Common Issues
#### 1. Repository Errors
**Problem**: Docker CE repository not found
**Solution**: Update the repository URL in `container-runtime.yml`:
```yaml
baseurl: "https://download.docker.com/linux/fedora/{{ ansible_distribution_major_version }}/x86_64/stable"
```
#### 2. kubeadm Init Failures
**Problem**: kubeadm init fails with preflight errors
**Solution**: The playbook includes `--ignore-preflight-errors=all` flag
#### 3. Node Not Ready
**Problem**: Worker nodes stuck in NotReady state
**Solution**: Check CNI installation and network connectivity
### Debug Commands
```bash
# Check node status
kubectl get nodes -o wide
# Check pod status
kubectl get pods --all-namespaces
# Check kubelet logs
journalctl -u kubelet -f
# Check containerd status
systemctl status containerd
```
## Customization
### Network Configuration
Modify the pod and service CIDRs in `group_vars/fastpass/vars`:
```yaml
pod_network_cidr: "10.244.0.0/16"
service_cidr: "10.96.0.0/12"
```
### CNI Selection
The default CNI is Calico. To change, modify the control plane role:
```yaml
cni_plugin: "flannel" # or "weave", "cilium"
```
### Node Labels and Taints
Configure node labels in `group_vars/fastpass/vars`:
```yaml
node_labels:
zone:
space-mountain: "external"
haunted-mansion: "internal"
environment: "production"
cluster: "fastpass"
```
## Maintenance
### Adding New Nodes
1. Add the new node to the inventory
2. Run the preparation role: `ansible-playbook -i inventory.yml playbooks/deploy_k8s.yml --limit=new-node`
3. Join the node to the cluster
### Upgrading Kubernetes
1. Update the `kubernetes_version` variable
2. Run the preparation role on all nodes
3. Upgrade control plane nodes first
4. Upgrade worker nodes
### Backup and Recovery
- Backup `/etc/kubernetes/` directory on control plane nodes
- Backup etcd data: `etcdctl snapshot save backup.db`
- Document cluster configuration
## Security Considerations
- SELinux is set to permissive mode for compatibility
- Firewalld is disabled (configure as needed for your environment)
- Consider enabling RBAC and network policies
- Regularly update Kubernetes and system packages
## Support
For issues specific to this deployment:
1. Check the troubleshooting section
2. Review Ansible logs with `-vvv` verbosity
3. Check system logs on affected nodes
4. Verify network connectivity between nodes
## Files Structure
```
ansible/
├── group_vars/
│ └── fastpass/
│ └── vars # Cluster configuration
├── playbooks/
│ ├── deploy-fastpass-cluster.yml # Main deployment playbook
│ ├── deploy_k8s.yml # Legacy deployment playbook
│ └── roles/
│ ├── kubernetes/ # System preparation
│ ├── fastpass-control-plane/ # Control plane setup
│ └── fastpass-workers/ # Worker node setup
└── inventory.yml # Host inventory
```

View File

@@ -1,233 +0,0 @@
# Kubeconfig Management - Modular & Repeatable
This document explains the modular kubeconfig management system for your homelab's multiple Kubernetes clusters.
## Overview
The kubeconfig management has been refactored into reusable, modular components that can handle multiple clusters seamlessly:
- **FastPass** (Vanilla Kubernetes)
- **Hub Cluster** (OpenShift SNO)
- **Internal Cluster** (OpenShift)
- **Future clusters** (easily extensible)
## Architecture
```
ansible/playbooks/roles/
├── kubeconfig-manager/ # Core reusable role
│ ├── tasks/main.yml # Main entry point
│ ├── tasks/merge-kubeconfig.yml # Merging logic
│ ├── defaults/main.yml # Default variables
│ └── README.md # Role documentation
├── cluster-kubeconfig/ # Generic wrapper role
└── fastpass-first-control-plane/ # Uses kubeconfig-manager
```
## Quick Usage
### 1. Using the Script (Easiest)
```bash
cd ansible
# Setup FastPass cluster kubeconfig
./scripts/setup-kubeconfig.sh fastpass fastpass_control_plane[0]
# Setup Hub cluster kubeconfig
./scripts/setup-kubeconfig.sh hub hub_cluster
# Setup Internal cluster kubeconfig
./scripts/setup-kubeconfig.sh internal internal_cluster[0]
```
### 2. Direct Ansible Usage
```bash
# Single cluster
ansible-playbook -i inventory.yml \
playbooks/examples/multi-cluster-kubeconfig.yml \
--limit fastpass_control_plane[0] \
-e target_cluster_name=fastpass
# Multiple clusters at once
ansible-playbook -i inventory.yml \
playbooks/examples/multi-cluster-kubeconfig.yml
```
### 3. In Your Own Playbooks
```yaml
- name: Setup kubeconfig for any cluster
hosts: my_cluster_nodes[0]
roles:
- role: kubeconfig-manager
vars:
cluster_name: "my-cluster"
kubeconfig_source_path: "/etc/kubernetes/admin.conf"
```
## Features
### ✅ **Modular & Reusable**
- Single role works with any Kubernetes cluster
- Easy to extend for new clusters
- Consistent behavior across all clusters
### ✅ **Safe Operations**
- Automatic backups before merging
- Non-destructive merging
- Preserves existing contexts
### ✅ **Multi-Cluster Ready**
- Unique naming: `cluster-name-admin`
- No conflicts between clusters
- Easy context switching
### ✅ **Homelab Optimized**
- Works with vanilla Kubernetes
- Works with OpenShift
- Handles different kubeconfig paths
## Generated Structure
After running the kubeconfig management:
```bash
~/.kube/
├── config # Merged config with all clusters
├── config-fastpass # Individual cluster configs
├── config-hub
├── config-internal
└── config.backup.1697123456 # Timestamped backups
```
## Context Management
```bash
# List all available contexts
kubectl config get-contexts
# Switch between clusters
kubectl config use-context fastpass-admin
kubectl config use-context hub-admin
kubectl config use-context internal-admin
# Check current context
kubectl config current-context
# Test connectivity
kubectl cluster-info
kubectl get nodes
```
## Integration with Existing Roles
### Before (Monolithic)
```yaml
# fastpass-first-control-plane/tasks/main.yml
- name: 50+ lines of kubeconfig logic
# Lots of repetitive code
```
### After (Modular)
```yaml
# fastpass-first-control-plane/tasks/main.yml
- name: Setup kubeconfig for FastPass cluster
ansible.builtin.include_role:
name: kubeconfig-manager
vars:
cluster_name: "{{ cluster_name }}"
```
## Adding New Clusters
To add a new cluster (e.g., "edge-cluster"):
1. **Add to inventory:**
```yaml
edge_cluster:
hosts:
edge-node-01:
```
2. **Use the script:**
```bash
./scripts/setup-kubeconfig.sh edge edge_cluster[0]
```
3. **Or add to playbook:**
```yaml
- name: Setup kubeconfig for Edge cluster
hosts: edge_cluster[0]
roles:
- role: kubeconfig-manager
vars:
cluster_name: "edge"
```
## Customization
### Different Kubeconfig Paths
```yaml
- role: kubeconfig-manager
vars:
cluster_name: "openshift-cluster"
kubeconfig_source_path: "/etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/localhost.kubeconfig"
```
### Custom Context Names
```yaml
- role: kubeconfig-manager
vars:
cluster_name: "prod"
context_suffix: "system:admin" # Results in "prod-system:admin"
```
## Troubleshooting
### Common Issues
1. **Permission Denied**
```bash
# Fix ownership
sudo chown -R $USER:$USER ~/.kube/
```
2. **Context Not Found**
```bash
# List available contexts
kubectl config get-contexts
# Check if cluster was added
kubectl config view
```
3. **Backup Recovery**
```bash
# Restore from backup
cp ~/.kube/config.backup.1697123456 ~/.kube/config
```
### Debug Mode
```bash
# Run with verbose output
ansible-playbook -vvv playbooks/examples/multi-cluster-kubeconfig.yml
```
## Benefits for Your Homelab
1. **Consistency**: Same process for all clusters
2. **Maintainability**: Single role to update/fix
3. **Scalability**: Easy to add new clusters
4. **Safety**: Automatic backups and safe merging
5. **Flexibility**: Works with different Kubernetes distributions
## Next Steps
1. **Test the modular approach** with your FastPass cluster
2. **Extend to Hub and Internal clusters** using the same pattern
3. **Create cluster-specific variables** in group_vars if needed
4. **Add monitoring/validation** tasks to verify kubeconfig health
This modular approach makes your homelab's multi-cluster management much more maintainable and repeatable!

View File

@@ -209,7 +209,7 @@ private_key_file=~/.ssh/ansible
remote_user=wed
# (pathspec) Colon-separated paths in which Ansible will search for Roles.
;roles_path=/Users/rblundon/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles
roles_path=./roles
# (string) Set the main callback used to display Ansible output. You can only have one at a time.
# You can have many other callbacks, but just one can be in charge of stdout.

View File

@@ -0,0 +1,6 @@
---
- name: Create jarvis user and deploy SSH key
hosts: all
become: true
roles:
- jarvis_user

View File

@@ -0,0 +1,151 @@
---
# ============================================================================
# Semaphore configuration-as-code
# ============================================================================
# Drives a freshly-deployed Semaphore instance into its desired state via
# the Semaphore REST API. Idempotent: every object is checked first; only
# missing ones are created. Existing objects are left alone.
#
# Loaded from group_vars/all/semaphore.yml so that the configuration is
# version-controlled in the homelab repo and survives a wipe-and-redeploy
# of the Semaphore VM.
# ============================================================================
# ---------------------------------------------------------------------------
# API connection (defaults to the local Traefik-fronted service-name URL).
# Override semaphore_api_url to point at a specific instance if needed.
# ---------------------------------------------------------------------------
semaphore_api_url: "https://semaphore.local.mk-labs.cloud/api"
semaphore_api_validate_certs: true
semaphore_api_token: "{{ vault_semaphore_api_token }}"
# Feature flag — keeps day1_deploy_semaphore.yml deploy-only by default.
# Set true to also run the configuration pass.
semaphore_configure: false
# ---------------------------------------------------------------------------
# Declarative configuration of the Semaphore instance.
# ---------------------------------------------------------------------------
#
# Top-level shape:
#
# semaphore_config:
# project: single dict — the lab uses one project ("mk-labs")
# keys: list of credentials Semaphore stores
# repositories: git repos Semaphore can clone
# inventories: Ansible inventories from those repos
# environments: env-var bundles
# templates: task templates that tie everything together
#
# Each list element has a unique "name" used as the natural identity key.
# ---------------------------------------------------------------------------
semaphore_config:
project:
name: mk-labs
alert: false
max_parallel_tasks: 0 # 0 = unlimited
keys:
# The ansible-vault password. login_password type with empty login
# — only the password field is consumed by Semaphore at runtime.
- name: ansible-vault-pass
type: login_password
login: ""
password: "{{ vault_ansible_vault_password }}"
# SSH key for the gitea deploy access (clone the homelab repo).
- name: gitea-deploy
type: ssh
ssh_login: git
ssh_private_key: "{{ vault_gitea_deploy_key }}"
# SSH key for the universal automation account 'wed' — pre-baked in
# every mk-labs VM template. This is the canonical user Semaphore
# uses to reach the fleet.
- name: wed-ssh
type: ssh
ssh_login: wed
ssh_private_key: "{{ vault_wed_ssh_private_key }}"
# SSH key Semaphore can use to reach the fleet as jarvis (admin
# account provisioned by linux-baseline). Retained for jobs that
# specifically need jarvis-level access; the default is wed-ssh.
- name: jarvis-ssh
type: ssh
ssh_login: jarvis
ssh_private_key: "{{ vault_jarvis_ssh_private_key }}"
repositories:
- name: homelab
git_url: "ssh://git@gitea.mk-labs.cloud:2221/rblundon/homelab.git"
git_branch: main
ssh_key: gitea-deploy
inventories:
- name: production
type: file
inventory_file: ansible/inventory.yml
repository: homelab
# wed is the universal automation account pre-baked in every VM
# template. Semaphore uses it for fleet-wide jobs.
ssh_key: wed-ssh
# become_key is Semaphore's sudo PASSWORD slot, not a second SSH
# key. wed has passwordless sudo on every host, so reference the
# built-in "None" key. (Semaphore rejects an SSH-type key here.)
become_key: None
environments:
- name: default
env:
ANSIBLE_HOST_KEY_CHECKING: "False"
ANSIBLE_FORCE_COLOR: "True"
# Semaphore runs ansible-playbook from the cloned REPO ROOT (not
# from the playbook's directory as I first assumed). Path is
# therefore relative to repo root, not playbook dir.
ANSIBLE_ROLES_PATH: "ansible/roles"
templates:
- name: "day0_linux_baseline"
description: "Apply the mk-labs Linux baseline to one or more hosts."
app: ansible
playbook: ansible/playbooks/day0_linux_baseline.yml
inventory: production
repository: homelab
environment: default
vault_password: ansible-vault-pass
arguments: '["--diff"]'
survey_vars:
- name: target
title: "Target host or group"
description: "Inventory target (e.g. figment, semaphore_server, all)"
required: true
type: TextVar
default_value: "all"
- name: "day1_deploy_semaphore"
description: "Re-deploy Semaphore + PostgreSQL on figment."
app: ansible
playbook: ansible/playbooks/day1_deploy_semaphore.yml
inventory: production
repository: homelab
environment: default
vault_password: ansible-vault-pass
arguments: '["--diff"]'
- name: "day0_linux_baseline_check"
description: "Dry-run the baseline — shows diffs, applies nothing."
app: ansible
playbook: ansible/playbooks/day0_linux_baseline.yml
inventory: production
repository: homelab
environment: default
vault_password: ansible-vault-pass
arguments: '["--check","--diff"]'
survey_vars:
- name: target
title: "Target host or group"
description: "Inventory target (e.g. figment, semaphore_server, all)"
required: true
type: TextVar
default_value: "all"

View File

@@ -0,0 +1,4 @@
---
step_ca_url: "https://turnstile.local.mk-labs.cloud:9000"
step_ca_fingerprint: "f63c44e76381e359978bd2dca07c928d04ad44b575f0364fa66b5725c7e7891b"
step_ca_provisioner_name: "admin"

View File

@@ -18,3 +18,24 @@ terraform_server: "infra01"
# Traefik variables
traefik_server: "lightning-lane"
# ---------------------------------------------------------------------------
# JARVIS automation account
# ---------------------------------------------------------------------------
# Public key for the 'jarvis' user provisioned by the linux-baseline role on
# every host. Public keys are not secret; the matching private key lives on
# the JARVIS command centre (carousel-of-progress) and, when needed, in
# group_vars/all/vault as vault_jarvis_ssh_private_key.
jarvis_ssh_public_key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID5sym5ajFvDyzw395BkHv7qVb66XPTx/OF1p19MGuNo jarvis@mk-labs"
step_ca_principal_mappings:
- local_user: wed
principals:
- ryan.blundon@protonmail.com
- ryan.blundon
- ryanblundon
- local_user: rblundon
principals:
- ryan.blundon@protonmail.com
- ryan.blundon
- ryanblundon

View File

@@ -1,170 +1,436 @@
$ANSIBLE_VAULT;1.1;AES256
64393362643437353731353234643535366431613861636131613139636435633465306333656161
3365306632386330373862363637363435633235643336610a663736396537313032336162633334
32626233346233323565383030393130316532653062373730376163326134616239663865363862
3365316365643534610a343932623566356637663136336338386534373532383863363662326438
37393331356632313163643138613330366631643465623633386433663236636334666264383033
34333935633131363364626333313336353733373562326636343633653838616532643737366435
36366330633263613938323566393063373233363538643531636131316335316461656631616362
30343965356363383139366534363664313831363132383337636266316538393139383232386230
65353364313534333632306465393264386466346365303232356263653563653338306131376131
66343764643639396663623330623332333466633737343738386230626463386137373536343430
36636430616132363161303338353033343963643338656639643662636664613965636263366135
64353134376162633162316166316239333963653930363830646165636537653238623537323238
39376338636335336338323935333439306534386539336331613961663261623962383237303632
34326139626366303331363337353432623636626231373731646665623831353865303130663533
37326639626237653361316666616161363531303961613538636663633063396363346337663239
64313333653766383736303161636530316538633033323063323662336335666264646564643465
64396436366131373061326233643562626335376263623031376532313730653363343038356135
30336131646635636638353934636139343662623265336338386535663536393935343132306335
33626632613663353661313061623732643539323637336433666330313463666661623836303462
61383732613139343434363662616539373837633139393764353864613062346237346661633033
61393639326535366266643664653439383238646132306636656130333733316638363830346433
38643430346263663639653762393935356662363533323766343362366532626532633530323630
30643436653130363066373665666434383732663837643839356539366432316539613863623839
35323930383437383535373330326239663966383234663233643337623330353236626464636530
36393735646664366235363831656262383333626265383066393363346165656265363438306135
34633530653837346638636665666162363062663333623263356131646661373234333939333536
30643639333836373439613561346130623239363737626238663436356365633562366465643561
32643039376234323963653733353138643138626430383639313765353035646539653466623539
32366135366266323261333938373837386636626232323661393035613638643335383564663462
31383663663063373935396439653231353962303630346338623938343064643037396562626637
63363363343739626639633934393734396434653262373137373539316536306663336466626639
34646465306439313239323365333761663161313461613766343463666134303036376338366663
64356364653462383534343438343933623766396232336434653765613864376235363735343032
32313133383731333366626534363065333738346230393636393938366633363561346436363338
33393932626333636166313431633339386564616466653165383962656633303161393935366635
65396464326333396433303966316231376432363335643732303438663131653932636564363264
36356663623436643835373538363662643134313030656236326335393334626634663136333137
38363432346663393435366536636335636465343131666562396334636530346235366365396431
64623739373130623436356161666139656330646661396430633264393632663366353037643130
35373839373430643065616630613433646536356563353133633361623235623733313234306435
64356635343361633462376131636362386433373339383365313436333332613333316237323632
33633362376331323930376130393838633333316330363362396131306230643161373636303034
37303738306233363937316332313830396336323639333634666364633566386261373031383563
36633865643634363234613463636437306236376435613034623537356162353235396665636338
31613733363430313532643235343438393137313136653730643364623662336236323464656465
65636431633464613437396561643638346565313937303731633837623930333538383335633938
62666461323033303937653366646432623764653566626333333766656235633662623631636636
35626161653736303261303233643236393038303063386532643563663065303331616637386661
33326637393633616430643665623431616336336666383236326538383535633364613135653131
38383961383264313130383263613763326666663033626338313762376336353866646634323537
39393565643265633437396535643165626165646537373039613061613936356434366238636562
61393639316531316564646334663863383836656261623230646461323566316664653965336136
62353839363664396632306231363031303265343131343739633835353338386335343737326633
36656433616135653663353166636461623337653138613435646663663939323631373063373136
63396331313538636437383635643831643935303932313964336532643366623434316662376632
63666464333736333162373461333563393138366638643039393865613238616531356165346236
65313432383533353738396530356465386263313939393962333637306138653939646638353235
37393733333032623763333632353866633866613162663833346534666237383430663838383965
66663332616164353539323261383364363237383064306132633530326134383561346536346636
32393365316435626464386631663037663934313533663030313466323134336439303362626365
30613764346362326130353333353663316539306437376465306332353464343530663933363562
39646333323631353835336461326233623963353535316633353236353236636462313163633638
35366538366336663534353739353231306164626664383737333565666337636164386539316632
64323966336637626538303830323535373533306362636236303065663161353734653633393064
66353938306238646238313037323163343935643863653632336462363834326634363433383338
35646364353834376439633136313939393064633539643462623562333832303261386331636533
37373837396461373064353565353438633938616162363138323561343662383330346633323133
36626665343636343739353732366133313434623230353732323037633236303237656364323763
30363530306131303630386131623866323861633636313431376161336536373436656334303036
61316639666136313338396233326234386634343863633632626264363336376531363461626564
35336335396561626339616234373265663032353432363832306666353339376338396366383839
62633063326135323539653133626538613331613334353430623232376336633334666433316638
61326563366432333632666436363761376237636434356231613061626633303737626531626236
62373166363432616666616630616565373934356564363266396662323963636434313735626161
61373738623966363032313632376465356235363834633639663733653131636531363563326662
62663365323934363761653536356162336463646566393838346134356536303264323065396534
63346531303764613634646336313530326663373433396436663162343530306131386265303137
32333432383966363735343331646233613631373165346131323632366165346235656330323939
34393030313466666265346163323561353734613434386239333630623762616566616562316265
38633263323339396462336166646431363336616462366463383663313839613638383663303133
32333835623531366236643530623961656631336139313037353962323937666465343430666336
33383464363930663630333164636361326334396163376661666564663538623732663839613237
35623237613761356235383130633733333766333236363537383862306539373665383461386534
32646432363339306433646139393732366464336339346161323736363937383566656663356230
65316562383562366366653466646461613235353061623363386331636139386136306131633232
32326238353861613236373964383131363537653632663532376364336166303662663336623161
37623838616664376561663763393534323931343163376535396662626333383730363232653036
39626266623666346635373630343135333532353866363234396265356631333265663432326235
37346439613363313334633964386632336138316662616661356637363730306164646439643765
62343137663137633031316665363362643165643731326464323962616330383164353861366462
31663965626338636534356461366362333831396464376134613331393664306639386632396236
66633066663831646361333730356361646133383834376230356336613038303665323732663162
37643130313739306638636262656632353431316531336438363636363833626237346363326461
31636262396632666335303566633161333937626330373835613236656130383132663863636364
34383661353966346362386461303062303138626536323163376362326639356336396437333530
66623765313638616430383838396164336565646338636430363262396636663965383531383139
35356663363437383132333566336131666139643639663662633363616133323031353032663438
39393266633639306363313530643239303132303264323063306335346233363233373865653863
65396430346630626566383963356264313630623336316236393730343437656435656664663266
32643639363536396266626435373363303234623230383035326536633631633737653739366563
65653932663539353436366139313635663962663330343461353864323166323733366163653162
66353866376134633534323366656265633930356132316137616235343866653265666161363266
30386638343739353965643466613639303666316432323634383464633431363566333831643134
65363138313439366430613837396337323436636530313664303964353966306565663630323662
33643639623935336235373332633739306539656235323439343331376632373538613534386430
30633832656532623262326662376237393262333764623937343232383861636563633730393765
39663032373132323735633361343933333332623465343863636263363161303165656234313338
65383462626465376162306639626531366161396531643364656539366233306166386234313761
30633666303832636561323236373363373137353032613637353637386238353234646431333661
32313834666237626333626437663562646438363561393364373565313536363535323536363537
35633834616136656261643130656131306264333131326431626439343765393031313964616639
33643933653033336436616635346335383335333334343836343638613264396462633530623866
30303462653130353430663862623261343866383335343239393435633962383864396430313536
66643535303437383232323564326266356636376464623563636139633237313661356330633931
66376631373366613731633863666132626366393532666331343361323764626539666238656262
66356332383235353864303562356638636130343938383530376432663866316238333434616233
32316432333064663764363831343464326334353737346632383565643737366233396139333639
35656435346335323030636363356633633332376138653930613037313537373038353433363532
39346161336261353463656430663033623036396365346638323832313361633866363134396165
65356632366463393435386366643232383162623536653965643235323934633464373061333237
32636564386662626365613235333664323563633466383238343336343063376531396161343530
61306632646133626133343132666534666337336239306137663931623637643164326538646138
63313037336235316164316232636262316331356363356438653235346539643065636537366661
37313437386432316336663366633366666434346135356462623635313531333530653866663863
31646438316435373637303733646564316261336233623035353564363664303437373132353166
33646534663130356330663565656663333765653961663835623235633665623464353862616138
37343531366462616439333539306265626164636233616262616563386566396536386264343137
30396434303231366664656334656635616334633066323062313738363431646335643132626331
33663038316165626365346366363731323639653837616163653561623034393064303563353232
30613939353336396365613662393938663933363662656634663465303134393964623738633765
37633832626631363736383436666263356531366237356261666538303863343033356333656138
64366537363939346535313732643965623332396265386636353364316630333130643833383331
34643563386539383235386130323065353835376564613835313934333636316464636463376537
32363565333038643230656266393761313064616537393830343163643535646436616230626166
36373531646536363664663330353864636533656261353331316364633361633934326436323933
39636631623430313934393033373630656132623734363835353735353935393063356261323764
66346265376339623265376666343865343763393833323564643161613861303065346137653339
34326432383230656638303963373832316438666131623539613464636631653365353331346262
37626130343831636133646135323564306538353439636265313239326235343232633136383663
62306561616564623038393631373862346535366135633836353363323330346430613466393463
39313766613064306566316561303761626135613063326536363761626635646134303732356130
38333266363639636565393737323938663938633662653231366164316262666538356162323665
38383263366133326333636466343835373737613333623464383139343863336234373932393239
36653736393732353466656334613364626339356533343363353731353033363730376565353236
66343332306330303662373363326236393333663565383661643763363662396265313037643432
63343336623663326237326365333565346335346630356439303534363561626362363833313239
62366436313466633534343430306461313266376363336536666162306135363434643266373338
65323866323537636465306636646533316466303735386336663234386538303939343661653562
34353564313738643665636337336536343735336436626438653335376532343137613837356363
31353861363733373232363865323164663532646235623533336536333932363133613830373937
38616137646332303934303839633336313239653064626331383432633935353538636132633134
61393230663536633335393330386363333162636661373938343837323736663539393636366532
35623637623262373332373335303833373239323430393766633939633665643236623139363834
30636135303634643765393433393962303138666630343936653234333730336234356234396566
35346331336639333237323136616337623938666561343861393563393633313261633333333434
38633064633538396566373730636632346234653339303533343431643266373266633836633139
33313234366236626338636331346565336138313136623632343735363132623964353066346233
66363833383432316238353364363963323762666562303030663939303162656261663664616632
66396466643763393130313831313438356635346230646335306237363262366132323133646535
30313338653262636136393133306663653331333830626534356536303532633031356435663734
32383035643637313863653537656665366138333933333364646561353333653661306133646361
32326535393333383033313765383031353233323138386365623132646636383037633537386361
65306166613030353033666438343530333330306532323135306363653764613638383461633964
63666239366638663537643636373432303164623735626634383636643836343436346161393365
62336462636534383766656632653833333832393263316633356235373335643436363333393335
62643838303131643265613861373964653464623766363664383330343138653632396365633533
39383535616630663564326536623537323966616436353439343630363738626131326365656132
32383737303733623830353366383062313134396531303831323336326137396335353763353631
36663336353865303562316233643762643563386665393664623138643836633165
65653162653536373535643935613236366335356333336264346637323164633739633965656664
3434656433323239366665356134386564643135663135360a653232636463656332343565313763
61326635636333393730356131356466363063613734653565643766383431363166656532393934
3631373264353563650a616362633165666239323563623161656238633737316236303133313034
66353461643535373638616364663365643961656635643566613336633466343739386234313532
38353066313763643266623739366261646633363962336239623830613265383335383763376534
62653865626662383330656530336534366136313965373764353133326139623132646364396337
33333366366434393037373361376538376334336265663162336239646237306332623039343032
37333832383762663534643638666234623062373734626335323537616334303638376164656634
64623962313064363166383930636136383935613237613434373733383836666532363265643739
37376333363736326334383864633734393263623536393239393564643131616132373230356633
61636165333537356632303034346431353139383065393832323236626331353236666263376666
38306631653862343064656135396638636363613139373062653735366565376165353330636162
38653530366263356437323433653437373362356166633235386330623333326264333961643434
30323130613431353064393138303465316630383462653062326264393035393735396134396665
36383963633938363662633035626538663366323461656136323934353731626264396364643337
63356337343161366237353135313966616630316362396362343962356564643365626664633165
62383765303436303239376365316632353463666664306164313933346162343833303963313835
62353565333231646462383664356230623165363333653335343438346233316534333132366634
31326138663433663132393761343335656636366237656337666331633062303363633938303536
31303238323932336330363064323963306132633961313634616232636163353439363666623230
65666632653730303134343032616331363932616532313365633038623135396463316666373637
34343736383731363239303330306561356162623962396564323239376661363761336434376264
34373037623331643666366165306161356365326261366566323233656538363061636637343164
39323130613530386661343839623361303862323763373864626536376562643766643263376665
64373334333436653061653962373437366437643034306633343830626230346532653565353135
33326339656266616237643362333062346564333563326463393535356465666166626463643132
36316263323761653333363632386261633432383431386435363362663734626634353761663035
32383464396632396361623531303937383366633131613861646364303662343761323530303139
39333866663164363431656233663064383633386261313063646435333666376465626532343636
62386166306635353061623365313738306237616430386461343262613039346261353662346462
65616261656534336631646434316336646337333661336530633834633037363834346539646466
33346466643238343133326339393264613763626532633938613765373262393236643533383661
33306233623130666161326365366164333434373463323662316432633263323761316338623065
66306332656438303938393430633563303761303239346266386664333837613336316539643736
31353234613664313061306433656165373737336166343930653837303532346163613564376661
61376237383562353639393565303332633366376632386532333736303738333362353334353366
36343166633533303834346265666536353633343564623062613030323934653863343062323136
34323635363238623336653062666636646263343731316537346635346235613862623530326633
30396338386233663335613661646132386230356631326331653962393336653937376264313931
65363465343265643034363963656262666131393632653561626330396166386566396363663538
66323935383166623038326437323730363266316530313938373261366363613731656638386634
31316261616630306131633936326635333632643731313730303438656563343133373732376537
38623330633539653034343364656261316336353466653663646333653636653266373363333732
61333461346131366233336162363033623636386634613731303032393738333537393131316539
65643238613432306436376432373364656530323331633762393566396531396466653463353832
34393662616435343137373364663839613464346564663631343837366530313061326561656664
33623163333864346661393634333830336133383337373833346132353531656236363661323338
64313663356432373137343461333539386665643130316463336134396436623730666165623939
33373832353932386238353866336130356530613766626235316332623132343735636335633437
37303766386434373930383663386565383732636635646133393637383963353134633433616431
33616464633431666361393931623631313731343863636234646130313136346639323662373236
62623830633838326432326339383533373430346635643739646639633739343639636665376333
30396238356161373734633562643466643036363239396538613465346238356366363730623965
32336533303331363336666238363863313066376230353932323466316432653264346334373931
61303066313831376233313261633336643265396139663037333031636666383738313763313366
38386138363738353265613363353339323930343934393036343234626162353037326638636236
66343634323136663134373737366433616632653737316534633232323237343566623361666131
34393331373835613934316235336635666633646465373866356566656261373066653937316339
37396366643564333130383237613933613736376161616231666139643030613338333133356336
37343362363433653638373335363130313362306236393231646637383736663236376537303533
31633230333933666131613764336532373633353132373839326562356438386236643537386231
35616662643431383661343937613738393963373865383465306532336137373730653832633930
37656661326434633730346364326661653937313561333437383064313933363231393164616464
31333664386134376264376339303838656431633966663263303465376161633866376161313531
62346266316533326639303661396164376666383864363262316630636561326366373363633063
33666263333132323232346235633837363138663539366131303633313662326664373332353664
39306235313732376164306164386534346666633037316334366431363063663137356464656234
35656164303933306661373932663831346531636530386139373837633263396632643235383361
65393939303338656537353130316361366139363933363031393735666336323931373632303663
63333137663265353962653861316365643239396562333933383964663730663230353331386638
62396435323162663036303334323936633436626136663537376562363430376239356136343261
36323062363935643562326161643065333437343331613135383932333835396565646631646437
39633035326662653334616366323736323032616264623166326563326361363263623736623739
62633762346135626666636463343132303433346330303631656634646537616537363764626564
30316539303964626635613338623261613430656432376265386636613566313732386561653564
66363538643463653233373433353965623765303038333931623036666435303330633136663966
31313964323439376637613530343635376166333765306466353337623633343539313330373437
64343930636465636265383261633664313762303533326234626461623563636466643039343336
62346462343639323862353036326335383735323337396465613737336661633465363330363161
38323561613935303334303261323032616332333231363762303831373961323033626663383637
35323134666636353766313330316231323938326532333030383638646230643735653663346634
38363261626663393139313031633032343038396433353563363531306232386539303734323463
63313938373735383631373665613333303530383335363262306230326665666535333564346163
39336132643464666538356461393330616264383632623037623634336530376164336336323336
61646165333835333961323439663864386562326131653564623861386336303934346263643036
32666632623537376537646634313839363038666336386238343531323664396461633030373334
35396463373339633931383636353062396562383763373939633336346463623733303039663733
31663834326661623064323132353431366362636466623332363532316435386333636562633562
36613465363936623462316136643664336261353938653362393938666266663330323163653338
63333732616330323438633038366465353865353965663333376466613834633166303761633361
37613863313230653235653034613137303231343961616330316664653333303436356561353562
37623839646536363036613964336230373537653130366330633137356661366561386566323165
38653633373538633762643935346661363961663265616433383832666635666331366635636637
32336532373164373135363533333539623739383330646431356363636430326433376364393265
35643561613033623231366362393737663964396266313232666362326436303331653764366436
30363135663234356363613065643763353836626133656334653138393562393032623631343130
30303936656636366465633163323061333863393532366563316233353466613566316563336539
33326137383263393266393166336361656538353963313366353434333864623238333435653836
39376138356461663262356665363835653638623031396539353132623737333736353937623533
65386333396666633365313262323739633461383464386663313163646632646335646433663834
34616663386661363135653765386534633339653232326133396332646136653230393431626637
30366433336633643837363434383439343562366164353235653833306138643431316537343931
34396535393432383163653231623837633961366437653434623164333333323563666330613866
36393561656135303731666638393737653565303836326264373137303264656334653163366433
35376362326133666336303732656561326164663832623732613830393063343231323363353366
66636233366436363565336231663964303532343962333732313832323762343638356231343461
65373062383564346266323362336666613735316161366136303032313039373464383532343932
30396530373030303933353264363330353132376164303062623538613632323932373230363838
32633766316533316665393033326161373361333166643436356430623262663032646262616633
32643436646264343366343862356339626561346561353463616534306139383133656433636331
32333735346163353434616530623235373037376664326266343933326439663965653865333766
31653163643134346331313331633561643336663130353936323439636331383134646665656630
63663866663938663632646664323936613434333731306264306633643335393566343564346537
35326132303632373731373532623662373861356534343536613731366666373439343563663737
64326339313533323936393362613564333065623731653930363264373061636665313135623062
63373565306535373763356664333465343666626439626666393734303962303762393066643139
62663334313736663132386232613061326636313664326462353361633362323464653038396662
64363862356439633564313065363361313161643962316166643936346539626339396230333633
39333132393135613264653363323233643032376266313630313735373966373966386239336635
34643739363935353463373434343430633438346136306337393061646139346230636531376537
36633861303264633662373365623261303635363836333163623864323338613739383134636237
62326561356535626564633038643230646431613866643239646563323035353434313837663863
66326530353265626361643738333266346339326532663166656432613631313137396463643864
64343339373131363066303437633436306462316166356131366134626331343635323837643136
39306461316230663134346563313662616532616239333735303061373138383363386232373062
61643838366266613637336433323062336434313836376434636433306135303734376661376361
38373438623331653165643837303733393436626166623762623531316164666163316661643761
39616539653831336261333335643834663533363233333732643431363836386633663539386638
39356336653438663061306230316464616533363036363434663335383530326432376632396338
38386130386532333861353362313838383964663861663962353039613565636339346162383331
32313337353430343130326466623666326263343438663463376134336131323735663261356263
34613638303262366666643036353766653039393739303738366330643033316632616232393032
36336161646434373332646663313834623064336437316666623663613861333838363731336564
63636633623664353432323836306564616239343731316335333234336465363232326430643231
30623063316130396531376237363363386465393838303463336439616333353338366436323634
66626463326136343533343263303033333766306632303333376136643137353831323135666430
61313663336363356531633866663366333433343433643664663665643335633035303034666334
62653233616136656462343930636534636432353237363538356365636635373865353532643764
32646431376165616235396133323162373330386664636336336266363961316462613263303131
33303635306136303366353331653239306130616464303835363437313566363436663032353061
32356639333061636536383238396665393232663562363662366563653831343037626535666432
33353261623936313563323564643134666232333839386162663331326461393239623135383530
31323861666631663933636338323239386336356339623738646430333136336635643562353263
64313931656435383134353063383665646538306661396163653434373031326134336235646461
39343734313965313365643135643166366133313465323366633038663333366535633532623966
36666134316165613830323933333339336663363964323764313430613130653236336664393863
64306437316235306565333238666164353738336539353064343161343834656262653666636663
36353063326262653466303534616530616139636166343137666439336335363539393439633436
36623963623839623834656431396162396562373330316438383739363637613366613163336138
35373730383165316330366332373939643839336634613934653538303866343738656534653263
33303361343237356430373266653062313037663230366531366363393937653439363732333036
31343365633862623038316536363031356336383461396562626236313038313239623665316439
30336137356438343362363536303234346637373833313231663333616263303233336161383130
34316265613736383830363639366139313537313661633736303634326237643330663735356561
65626136613831376432383363383731663763356432663264346561646535633966643462326232
31623930303262616535356633663938323635643462663731366433623961623435303062616263
63353836333235336362353039343431313262646137303364613838306339623930666466636132
32656434623738333565636164303535393734346462353436313030396264623332393932653861
36383533363030356664326332376662616139343065343932656262623334633334633462383635
63666439343130623936376636633635613335363264363136643366636535613938373866356336
35666632313565633630373039316262316339333035366332356138393261613836633437363136
33653033343536623264366466356561323063313031633464353963383661316131643166663566
36373937326333393164353234336433656336626134363236303032396531353862333535336466
65393865333565386538386665373166383235633739383934643334653763366161396565616166
61333666353666393230326134636661376237353937646236343034626135366535383631333035
61346231386233326564656636363066396636393966353539666231646464323636303038656562
31656238373331303031393034346630643263613365643232663861373336363662356265326164
61323866663033636534313365376466303538666463663539623231313632303262326638633539
62313239356365313932653735363662666266643765633138393037653662656566656130663133
61373531316261646234666361313766626139613762306563343561363961656565666163313532
35323837333737366138316164376537613834313135306461633735613362376233306134373537
65323339393039336230373237386434333639623762643133363339303265623065326438623531
65356632363237613937633132326231333938336639663438653861353761396434643136643238
62363162666366653831626531353236623532656437313134343633626666383437363761626331
35386431646564653233613632373435623665613335316164636562356264663033366639636439
63356231393362663736366431616161633065303537366538393935363033303636653836346132
30633036653836656433636631313863303132666531326563303266316566623934666361653932
35623265613635343435373361353635343261656233356535303037383433353731313539613463
64363334383666663639623735393934646366386437633638613034316366336337316561333731
38363339386561373834333037626565623335613563323366656365643032373834616339663838
64646534623432323363303032323666643636323536666239616238323932353165663535633465
64633934303565396533313165393438373231376664616237373562306130363531393833353365
63623033343762326437306536663734386365653131346235303837336236366665346234316163
32616539666634653839633739303837646563313364333864343231306663383530313963386434
34313965393731653963643336336134343764343065646436303739356530333761386465333333
35653363653361366462636162663134333431323866306235343764333035373534656138643933
30653962613339313833343532613939636332633236303733313135313932356136383439326635
31386565343337333662663762356361623835353830333865613437336532623936343365343231
33336137616266303835366662643737666333346433366663666462396531643463373562623237
38383537666639363839653538303266303965613762373561643433376664313966326561653764
66396566366265343962393633653765643563666634646637623161323538623961393039333036
33363530653935386331396533666432626634616463663763396663353133393461623866383061
64366132646132663263636131333031643736663031663336623065386266653638376639336334
36653932643831303931613532333036323766376235303433633564303062623336646166303533
63353063303033623032643333653733393332303466366538353062373666363438366432316131
36613335396531663238343139313061613363393463363035366132353439353433646237646265
37633964316466636337363331383565323866653036386435376530343661323565666539663230
38386232376436623064613232636637313931316535643661306536343661653135313335616534
34633761663066316430313361393063623033373636633133653536383764306433346235343761
37393764633561653632653565383937383436666662336633633261653861323533303732613738
64653739323636613639643465316437356163356238656139663034313738363532373861393333
64613038386535396565656538663163313161353831306237313064666566336534646233393137
37336265316361613430663961356361653636636235323439306537666363353361643664363631
63643631336134323032636463393639613636643130313136326538376461363937643436633561
39663433623663346462356437666230313937396132633834386661613436653839616136343234
35333437386663353863306166393634363734323636333137343938353461386366326239396236
65303939623437376535633463626166653935343033303066383064306332356230623566393062
34666566353238653730613561383434343332333732356334623836616334393464373965633334
37313564393463663862386535626338336235613132376530666635343832386231366533363562
32626162626434366261623431373938646237643462623337653466653538653732393634366639
62666231376239393066366336386166646631396632613631313663633435646337313465643231
63356566333764333238326165613832643161346366396635663032623133386132666333343337
63323637383035363833653162326231616461653062613765363165313638313234343266333334
62383362343232363431613364383930376338303839633261656236636264353335333938333731
63616666626437643432393438376661616262643939363239353836323036303139666338336261
31306437353536623561333531346630643761336235373230373135326361393739613934653361
38346532313364633638306665306563646433353038346238313530363163396166353535343537
34343330373063383566333335386162326339383835373665363464616139383331646436343365
30386564323733393461653662323334656637363566343737356239306462303762383332656239
30643439343161636434656338636533396364643632383030646639356534343963666466613566
39373833633265623563396363653830393661633636646265643362633731626462636531313362
35366331383635643564316461633431646565393863373635323834363934336335653831613330
39646262633636343832626362393730643163616565363765346266316636343065393630333866
62333931336161636230326132393739616665646462653666346532356265643235333131343635
31326638626162356333356261623531646464633139666539373261376230336661313937363036
32313464316364623838343231346538303237393137363233366234373031346238303932383461
66613537346136313262313337636565383639383863346535393361363930616535626264376565
30303236663065363161663836356130363130363339643335653366623065373566353136353933
39363761636130646537393166623335353431616462343364383535393661313738343265313138
38323066333239616137623233323664616232323262666530373064366138316431643430643462
32653262386565396335646431633534663031613536656561643434366464353335313239393464
36333866343063656533623432306231343065343530343638653739343731336131626532376566
37653833313233386538666464386232353466303035316232646339653533663831323261636330
32646166366633656631393762626236356633393130386165643264663437666433336138353938
37653866646364333134313366316366636132623764366562623932383239643265646461646434
63656233316639616134666635373434323030643136626238646637386634333439616231336136
38326431363062326232626238613334356633353362653138643237646339333762616330383934
63613235376530373834333834316539343330313862666262633439333062633261646138626434
33616162316632366537393939343163333530393939613936323164663034326536666133303365
63643337343037363363663235316330316131353064383933323131643036623862303835336633
39376134313264333834306330366136643434386566393466313835396466306363663565303065
33393431636163376365396139346162346239343838616561373162643931623365336436626636
39613030356162363931316166376231303033306366616162366239396333323838363465353261
64303561386563383834386166323739626237376164326533616533343338316430366365333966
31343233666365663830323030336161623466633261303637373537336333663262383238393833
33383033343630663237653338636436346565356238313434346666336436653835326133313038
65353530333839373136326132326439306431333334633933663065316531616263633533346431
62383862653162613532383136363830313964366161306166343934363865316533643736373431
61323135343030346336356232326138376564343131636263336332333239303733323338663830
32373639343063353261623831623237663133646535636338353333653237653539656438666436
63636537336636313762336531346531313637653834386535313666306636303338306465303764
63366133396263636164386339343131653361363831363932303734333634343734636163383539
33613430353236616132636431656636633561316161623661373663396462633930343865336236
32373534666135303730303663333337643033346231646661393137643161383833626335336561
66366433303436313132636230313531316261613564323963626438623530633634363265613939
37666636313632353831343238646237666464333039306630343531626339323562346162663130
65633133366564306230666436663630316434616634656661306461353866396662376331666630
30323166386535633361316633303830623031393531343532633930616439326465613631613838
38326630366534363036666433333237363636643366333161343336363936323730313339343132
35356231666633386537366236303136663261336532386338316565343239333261613838623861
31366531343232653431343436343766386331303133363534616536633337656433326265656161
66356235616135363634313835666134346232356663326361396537303339373334323861383161
33393438313061613465666536666636643166613463363165393338643066613231636138343537
64333731376139353934316631393561356163366237386337393061343235633131613962383731
66376261633832323864356564326532356363343762313563393331393933316336373434333237
35396531303330396363333163373361643837383766333730303031613665313237383532666264
65613237623136336239653566646366376538356362643062386436363466616331336665616432
31383365363630663530643764633335336332386332663661393733343038626265383432306630
38653861356636313634326266633637653764396233313463393964653739306234666662336432
33393139343864346233653763343762616230383131353166396265653031326232656333626264
63616339663564363665626338316661623230646534373231313662343736316462663763333364
30333137386161336664386536366538313934313039333832323763313264373861666239643466
33383334633736333634306262316265323634386635313764376532646364356565396538656137
63316232643832383865303934636364313036666237636632393137373933656263646534623230
65616231323035643531353430326630353761633234636234343834656664323161383335343235
62616533366465303961306134386265303933616236346332623536393236633366333634316264
31396361343833623131306266336238343866303361303336346566656639616134383063396235
31626633333665373934333961646263663730653331326137383031383736646633356536333431
39303632383030613465656236333763393737353865386235333830613665653363393536646630
65353733613536653963393831636530636231366266343262376631393062623163313031623230
62326561653165306339633439376461346330393231313366666339346536313765363163616630
35373564366563656161376336323031346339313734633139383563393966373637616266666265
34623330313265373935396130643231353131313163393333396337396666636439633035326635
32663762333537663839346531663132613636366639643364373333343262326264366136363331
61613833643639656632353931663830613634383334653036323462323262386265313637656535
36333834306438633532616335663562383730306337656336663035643136386261326130336363
32623330333764616565333162383234656463363432313039666265653563323232613930303765
30623834666665636332303862663832303537656664643362643636616265613739356664646462
61326466303266626166333730343330396439356663376362666536623539353666636230373533
35363461326263366137313037323166396133646430616664343165623533393963623535626237
39633362393636373131626364636437343361303435396138363735326235383237633964336138
31643935646431386537643733353862666138333238323461623638356535316365653530376464
33343465653134613536343563376564663133336532343330666131346663363434623238303862
31306331633238623836346466616230356532316232323734356638343532643032633937653061
37393265333233626563353963363637653338393964363832663734303566356633623733393164
64383536343862393134336638353733373262396563303266346535346266306238616531336162
35363638333431396163303530353461363338656363336366376430316464353131623661346366
35313930353538343830333337356163613765356361343663613933366464353935336361636436
65356465346535613231363932303339646162303238376236323461303062313336656366306563
31353464333539343839353764393665393235653537313939373563303436626635663766303336
35333838323734386537623334366235653534623664396664656264633735353634663332353364
65306132346464363966313963346633353538633936363164393566376630376365316139376132
38323039653465346462666439656134393964623563393664643634313638323832356164323863
31366535336238663236666162336166313933376531643137343665653765623961333464633332
30306265306364643738386561303833326363303836333032393462643764663664613536363835
30663937333038663435326637663636383165363632326337653932336130333932333861366235
66313566613033383631376132333634653639336666386164613934353230613239383239313038
30653162643233636638393036353032353632316166333938313966626133643237376461313065
34323762376133393535613864636461616261343031393266666165316236323231323534376465
63386363343933316139356163363132343666366132313038643938653865663934383335336364
31646563346339323633643366653861323030636138663861636434306238623738636331393538
66333765626132346533313261646434633236633432333430613266333161643566326434363633
62373033356337393232353566666263353539326564333766303335666135343461386530626562
63633137623361366362616432613237353231616465656232373835376463386132663331633737
63306136326264643365356634336233303163663135366531643362303666653630363962666637
61626539613333306263323564373435613961633439633261373530363137376132326462633035
30353838336433313135623530643663643232653364316263356164633661383937393238363262
34393666663361613735623435626238646266376533333963653630663462653733373130393338
32626361366632633630323363376238343534626230373835353036333065396435653964326534
33356466643839626162316438356437623732656664613434613165643666656230656434633638
61316134333631326365313532313335633634356466663834666531653365333333616137643835
30666664353134313236653337366466623966366131313962346564663432633535363938333261
37353464383832373366613531666161323632616236613631653433643562326134623838393135
62323533373336663434386166333862366639626562343830396533646138636261326161393339
33613961353431653537373931396531623061653163376230663338376636613136353433646231
33633236306238336432323134393463653161303662336266313562316337343430373532663930
39613133646432343238393762386266303531663032306130653532376262353238636231393539
36376332633563633661353835386364633461363437333038363865356237643739323133653235
66373931336330626663383136363265363133393239393131623465383433363336333237653131
64656662316665356665303765343932363733666362353031656234373363663364666537663030
34646233386630633438313166396663383732366233373139343431373463346133316663643036
31633536663266383132393032313563653035303034336161356139396636353365636163383031
30666637363933643262393065633463313836653530663232303736306537636533343431333966
32386330356564613932306466353931393839316562353362303765663263383738363765333136
36383036376130306133393166333734316231376165393832383735623838353466353664353834
39326136656239626434313930306435333533623533643236393437313038393235386364323766
62323537666333613066356236313361376138656232343430363438346261646165623565313435
65326532623761343239666664636637613034623736386665303634613737613964333630613533
31653731353533613866663166653237663162623236353838646465373734613466616362363833
39346335373539376439643937363233373362616334323231623232353061343830383336666466
31303663643061653866326632333336626462623835366564356231316263326130393138306330
66333532336164633931306136373531666131356130313262313038626138653864633734636132
61353638333133383035333937626333376564316465333539653138323739356265613664653536
35656136343036363532613335336230373364336432343731326132666662636664356364303335
66613736633231393534626539333536626565663231396536363062353037346563643934313236
32363138386430633761383065343834376631643539626330313638333534333734333239383632
39383833656134393165666533353739303834633330653936643637316639663033363865653631
36313735613962393264623938336432336538393937316634626165383934356562353035316533
33353735653133636439393763356430303863326235623932623464356636666265653065383736
31613666303065653035303131646364386636393035663865336164376536323462666239356333
33373562383863323635613634643165336465613734613034366236633835363733306662613462
30326565346365633131363331643165346466366666633231643538303430376530393139393063
32666664313162303065616261326261353130623564343761633861373034646161643163373533
36386665626530313030343032376531656333356332666437383530613834356336386465353539
66623062646236333465316137316564363938303966613561643830353332333537373736626438
64653239613063646430386631313435306462303566333734663462626530356530663235303166
31633366613264316137303334613366366537646261303962646364643238383062643732386436
39643430613562663136633130666563396538306633633034323064386136303034306531323564
63646633303035353831653438366635646562613639313230396462373161363030386264623033
62343031616135346238303337313931643461386363396663333231626661613938626366653837
33303265656561313332353463363534666232363561353532643532386133306332333930373161
66626135303036633566303961646461383236663737643265346566393963343634623164633966
61313031333863383237313633663633333866366230313936396334383361393338656465346437
65613763643061386463386431386539656635393435633032343030633738373936613833623938
39316239363835633339343161623237656230633763313031643663623466323764663366376165
36366530363535376636383561343161643037626639323830396665303238663534633531613735
39316639633730376533643932633432353835353439646666653766666338646662643162373263
65356563303235633963633232393736636537346637376462626637303535383063373134613732
32663763663364633463633738343535316436303365343665356133313836616164343434613133
34626434643531343461346332636539636463313539393830613434333933643632313737356564
39386137383165616139626363613732626336366461663339346134656530396464323533313265
32346535356466383135636638616166313663613565353765346264376535653135356638663538
30623834393935636562656639643737373462643137363063663737306361336339653334633665
65316237633436303761393766393234326538653633363936303534646566373338623935666538
38313761643437313565663762613838636163633066313630353631353934396338353665653330
35393661356630313436373761383462396434643535316364383338343433613061353637333563
38376562373462316334313632343965326235323231653030666666616163346438626437386637
65656163653935656530366239663237633937336166613132356333623964666630386331383331
32303166386431643739303363646633376331623166313135653265656636663236363936336462
63303135613539663233366362323565396137623264353431373836356233666332633731666364
62376565633037373832393366306431613863353137306535393664313866616235633638306365
32633761386462386339343066316263663438623330373733333634363736623637636661636331
38353835333962633537633864616132643563616437336334326633323636393833363666353261
36363732383565623233343439666430303961306263363032383238653265323637313430613133
63633338323536386139613233343636326564356564353065386664646230666561336338366436
35353162633864633764373535366634303738346236633362626165393632383762343036313930
62363864626338393736373938343264356235393763653432306330363363313762333137303362
35393061356165666230343135663438303834363533323064626532663662323063386333623166
39353037363363373166316238393565353266353665666465333134303234373263663135346364
36646463343032326339663130393963663861323130356365396365643265313833353234323330
34336661363733613362343739366636336265356464613033343132353031663965656634613831
33656466376132656465383664363231653235623036346634666166623532323635313130303836
36393064356637623139653333633164666332303539623863383538386262363064356130353337
62393063633637383965346139626265666463356362633163363537303166336264346332636363
39613735626132366137356365303331633037623132623637383164636565653963626632626338
65383663633831613666396534333061643966313366303937333261303135353762656331336638
35643539326161646433633862623762626539396137303863616139383832613639653033653933
39373734393066613132323739383036313830346239306132656464646462626661633931623932
36626163353563633764323466303966636161646532316134383034393733653363363933353738
34383563666166323865656430346661633663363635623566336632353633303838303436616266
34326131623331616533306465373365396437303764383231356631313335393364643664663864
33313033303463376139376466643434383461643264316534306166303163373661643962343030
36653665353232363630363437376266306531633934626230653338383664343638313334306636
38666162623761366237383236346564333333633162623832656462616662313031316166383465
31663362336663353564636335393738333565356432306139363661656663313234396664623832
38393630666338663031323464383535303539643931393732616666366661316163626339643437
34316532613063353264303462366534613035653834356562646637633137643839653237353937
34326230396137383531393962346137643035333834376537363865643366623932303662303839
64373431306464363762356563663038343737356165313832613965653065326532356133646137
30326232643334656363343533383163643130343836366430333836396463386666623465366336
61343236303336623863303630336437613932353832623866633661653566623431653938306238
37326630616131646634383539353234323766363633333030613937616632333432366565343537
35613237343838333430656336313232653530373863663031386433356337643334303363666532
65633033333463366636616138646632636534333963643864383033343463666338373630313232
63326365666666663262383664353664623963343366626364363965626435666363306237346130
30306661386334653137336464393465646135353436336138616563366163366664346331646666
61613334323330663835366163623836363534666531353737656464663935333765326235306566
61353962363034663563386137373432656166663661653861396361653930653539363333613435
64373731616330396130363464653865303931333637393333386531303365646130646161366263
61643234333563386338643331303164323638346639353765656632386262366666376665356164
34393464656634303130323738353161643537613337383661343232363961613931613234643361
34663938626364636562613332373161323932386532356261353137393533633731653034633966
62316666613738383665303433376438303266646264353133303566636436373966343662363561
63333662613930323666656166373763303961333332363231396532376564393966333135373966
65623664303537376362383861663238663463396537353866666333656664323562373165646633
35396661353639356363613834653432346539633135663565376234383866343433383339666362
63373330336562323138343934646466373738333039346361623836653231633566316435383636
61326535393339616639366563383662636136353162393961303362393866363436663630303762
65343632396238623137333462633633653761336635663265326332623631393566323530623562
36666431396532303939316662663138356631336434373732393463306336303435626232316638
36363733383831363930666132396661633733616464393264343339333166633739346236383833
33373962313762323430343161616361636363366334613032323637316261656333633639333066
34653635393031323262353037396130383964363037633463653331653965363562326532623037
383331306234353238633435663136623634

View File

@@ -1,66 +0,0 @@
---
# FastPass Kubernetes Cluster Variables
# Cluster Configuration
cluster_name: "fastpass"
kubernetes_version: "1.33"
pod_network_cidr: "10.244.0.0/16"
service_cidr: "10.96.0.0/12"
# Control Plane Configuration
control_plane_endpoint: "{{ cluster_name}}.{{ base_domain}}"
control_plane_port: "6443"
# CNI Configuration
cni_plugin: "calico"
calico_version: "v3.30.3"
# Node Labels and Taints
node_labels:
zone:
# Control Plane nodes (masters) - no zone labels needed
# space-mountain: (control plane - no zone label)
# big-thunder-mountain: (control plane - no zone label)
# splash-mountain: (control plane - no zone label)
# Worker nodes
haunted-mansion: "backstage" # Internal/backstage workloads
peter-pans-flight: "backstage" # Internal/backstage workloads
# Future node: "on-stage" # External/on-stage workloads (future)
environment: "production"
cluster: "fastpass"
# System Configuration
selinux_mode: "permissive"
firewall_enabled: false
# Container Runtime
container_runtime: "containerd"
containerd_version: "latest"
# Repository Configuration
kubernetes_repo_url: "https://pkgs.k8s.io/core:/stable:/v1.33/rpm/"
containerd_repo_url: "https://download.docker.com/linux/fedora/docker-ce.repo"
# Validation
preflight_checks: true
validate_cluster: true
# Kubeconfig Configuration
kubeconfig_path: "{{ lookup('env', 'HOME') }}/.kube/config"
# Hostname Configuration for Kubernetes
# Override hostnames to remove dots for Kubernetes compatibility
kubernetes_hostnames:
space-mountain: "space-mountain"
big-thunder-mountain: "big-thunder-mountain"
splash-mountain: "splash-mountain"
haunted-mansion: "haunted-mansion"
peter-pans-flight: "peter-pans-flight"
# Generic Kubernetes Prerequisites Configuration
configure_firewall: true
configure_ntp: true
selinux_state: "permissive"
disable_swap: true

View File

@@ -1,3 +0,0 @@
---
#freeipa_user: "admin"
#freeipa_password: "{{ vault_freeipa_password }}"

View File

@@ -1,51 +0,0 @@
# file: group_vars/hub_cluster/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Name of the OpenShift Cluster
cluster_name: "hub"
# Version of OpenShift to install. All available versions can be found at: https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/
cluster_version: "4.18.21"
# Type of cluster.
# - none (Single Node Cluster)
# - baremetal (Traditional and Compact Cluster)
platform_type: "none"
# Number of nodes to deploy
# - Master: 1; Worker 0 (Single Node Cluster)
# - Master: 3; Worker 0 (Compact Cluster)
# - Master: 3; Worker 1-3 (Traditional Cluster)
master_node_count: 1
worker_node_count: 0
# Subnet that cluster nodes will reside on.
machine_network: "10.1.71.0/24"
# OpenShift API IP Address (Reference host IP address for Single Node Cluster)
api_address: "{{ ip_address }}" # "10.1.71.10"
# OpenShift Apps Wildcard IP Address (Reference host IP address for Single Node Cluster)
app_address: "{{ ip_address }}" # "10.1.71.10"
# These are internal networks to the cluster and should not need to be changed.
cluster_network: "10.128.0.0/14"
cluster_network_host_prefix: 23
service_network: "172.30.0.0/16"
# Static Variables
openshift_installer_file: "openshift-install-linux.tar.gz"
openshift_client_file: "openshift-client-linux.tar.gz"
openshift_installer_download_url: "https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/{{ cluster_version }}/{{ openshift_installer_file }}"
openshift_client_download_url: "https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/{{ cluster_version }}/{{ openshift_client_file }}"

View File

@@ -1,51 +0,0 @@
# file: group_vars/internal_cluster/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Name of the OpenShift Cluster
cluster_name: "openshift"
# Version of OpenShift to install. All available versions can be found at: https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/
cluster_version: "4.18.21"
# Type of cluster.
# - none (Single Node Cluster)
# - baremetal (Traditional and Compact Cluster)
platform_type: "baremetal"
# Number of nodes to deploy
# - Master: 1; Worker 0 (Single Node Cluster)
# - Master: 3; Worker 0 (Compact Cluster)
# - Master: 3; Worker 1-3 (Traditional Cluster)
master_node_count: 3
worker_node_count: 1
# Subnet that cluster nodes will reside on.
machine_network: "10.1.71.0/24"
# OpenShift API IP Address
api_address: "10.1.71.69"
# OpenShift Apps Wildcard IP Address (These will be the same in a Single Node Cluster)
app_address: "10.1.71.60"
# These are internal networks to the cluster and should not need to be changed.
cluster_network: "10.128.0.0/14"
cluster_network_host_prefix: 23
service_network: "172.30.0.0/16"
# Static Variables
openshift_installer_file: "openshift-install-linux.tar.gz"
openshift_client_file: "openshift-client-linux.tar.gz"
openshift_installer_download_url: "https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/{{ cluster_version }}/{{ openshift_installer_file }}"
openshift_client_download_url: "https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/{{ cluster_version }}/{{ openshift_client_file }}"

View File

@@ -1,14 +0,0 @@
---
ipaadmin_password: "{{ vault_ipaadmin_password }}"
ipadm_password: "{{ vault_ipadm_password }}"
ipaserver_domain: int.mk-labs.cloud
ipaserver_realm: INT.MK-LABS.CLOUD
ipaserver_setup_firewalld: true
ipaserver_setup_dns: true
ipaserver_forwarders: 1.1.1.1,1.0.0.1
ipaserver_random_serial_numbers: true
#dns_ip_addresses: "{{ ansible_hostname }}"
#dns_reverse_zones:
#domain: int.mk-labs.cloud
forwarders: 1.1.1.1,1.0.0.1
#hostname: "{{ hostname }}"

View File

@@ -1,4 +0,0 @@
---
#freeipa_user: "admin"
ipaadmin_password: "{{ vault_freeipa_password }}"
ipaserver_domain: "int.mk-labs.cloud"

View File

@@ -1,9 +0,0 @@
---
# Variables for Matchbox
assets: "/var/lib/matchbox/assets"
http_endpoint: "http://matchbox.int.mk-labs.cloud:8080"
rpc_endpoint: "matchbox.int.mk-labs.cloud:8081"
# Variables for OpenShift Installer
#openshift_installer_pull_secret: "{{ vault_pull_secret }}"
#openshift_installer_ssh_key: "{{ vault_ssh_key }}"

View File

@@ -1,54 +0,0 @@
# file: group_vars/openshift_cluster_template/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
# Name of the OpenShift Cluster
cluster_name: "openshift"
# Version of OpenShift to install. All available versions can be found at: https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/
cluster_version: "4.18.21"
# Type of cluster.
# - none (Single Node Cluster)
# - baremetal (Traditional and Compact Cluster)
platform_type: "baremetal"
# Number of nodes to deploy
# - Master: 1; Worker 0 (Single Node Cluster)
# - Master: 3; Worker 0 (Compact Cluster)
# - Master: 3; Worker 1-3 (Traditional Cluster)
master_node_count: 3
worker_node_count: 1
# Subnet that cluster nodes will reside on.
machine_network: "10.1.71.0/24"
# OpenShift API IP Address (Comment out for Single Node Cluster)
api_address: "10.1.71.69"
# OpenShift Apps Wildcard IP Address (Comment out for Single Node Cluster)
app_address: "10.1.71.60"
# These are internal networks to the cluster and should not need to be changed.
cluster_network: "10.128.0.0/14"
cluster_network_host_prefix: 23
service_network: "172.30.0.0/16"
# Static Variables
openshift_installer_file: "openshift-install-linux.tar.gz"
openshift_client_file: "openshift-client-linux.tar.gz"
openshift_installer_download_url: "https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/{{ cluster_version }}/{{ openshift_installer_file }}"
openshift_client_download_url: "https://mirror.openshift.com/pub/openshift-v4/x86_64/clients/ocp/{{ cluster_version }}/{{ openshift_client_file }}"

View File

@@ -1,34 +0,0 @@
# file: group_vars/prometheus_server/vars
# Ansible vars template for Prometheus nodes created via Ansible\.
# Version of Prometheus to install. All available versions can be found at: https://prometheus.io/download/
#prometheus_version: "3.5.0"
#alertmanager_version: "0.28.1"
# prometheus_nodes:
# # - '192.168.1.207' # Worker node IP
# - cinderella-castle # Master node IP
# - 10.1.71.21
# - cinderella-castle.local.mk-labs.cloud
# Static Variables
# prometheus_installer_file: "prometheus-{{ prometheus_version }}.linux-amd64.tar.gz"
# prometheus_installer_download_url: "https://github.com/prometheus/prometheus/releases/download/v{{ prometheus_version }}/{{ prometheus_installer_file }}"
# Grafana Variables
#grafana_port: "3000"
#grafana_url: "{{ inventory_hostname }}.{{ base_domain }}:3000"
# prometheus_targets:
# node:
# - targets:
# - localhost:9100
# labels:
# env: mk-labs
grafana_ini:
security:
admin_user: admin
admin_password: admin

View File

@@ -0,0 +1,19 @@
---
# ansible/group_vars/proxmox/oidc.yml
#
# Proxmox OIDC configuration for Authentik integration.
# Client credentials come from vault.yml.
proxmox_oidc_realm_name: "authentik"
proxmox_oidc_issuer_url: "https://authentik.local.mk-labs.cloud/application/o/proxmox/"
proxmox_oidc_username_claim: "username"
proxmox_oidc_scopes: "openid email profile"
proxmox_oidc_autocreate: true
proxmox_oidc_default_realm: false
proxmox_oidc_comment: "Authentik SSO"
# ACL entries - grant your Authentik user admin access
proxmox_oidc_acl_entries:
- path: "/"
user: "rblundon@authentik"
role: "Administrator"

View File

@@ -0,0 +1,17 @@
$ANSIBLE_VAULT;1.1;AES256
64613361376364346139313833613465663361336634326430393261366630306466363935613139
3634363235343736643230623865386436333734663531310a343965396534336262356234623966
39353332626662636666383935383530613139626439373664323063633063316264383331646533
3366333837613731310a656462633638376432326365343135373863313665346566383933353630
37303565323630633739396433323165326262363863386233343134636236646366633765616432
33633165613239653762343935386637393437386431383961306436373765343736313634333933
30376463646464303066613561613564353938306664373464333966383664383034616439343735
38363966323366343465393530383736393364373361326234356365363163356632393834393464
39393530356235383830323065636535383138353864373237333164323436623737383435313261
64366362626232386438376131633533633162356432613835656465623762633464353936303236
63353633633262633461353963653534663336396134373435303266386162393965343165303335
65313436666334383965373837376664323861376162373834363439653539383231346634633737
30626431373739633930313937643938343165383536373465613535366562313439396533643864
36323032316131333962373732306334373939333363386564653465303261393337333466316130
32326533353165393635393130396164636363623937313566343333386562616662383038613565
35386638383365633362

View File

@@ -1,77 +0,0 @@
---
# Variables for Unbound https://github.com/aruhier/ansible-role-unbound
### Server ###
unbound_chroot: '""'
unbound_num_threads: 2
unbound_interfaces:
- "{{ ip_address }}"
#unbound_do_ip4: "yes"
unbound_do_ip6: "no"
#unbound_do_udp: "yes"
#unbound_do_tcp: "yes"
# list of strings, as the order impact how policies are interpreted
unbound_outgoing_policies: []
unbound_access_control:
- '192.168.1.0/24 allow'
- '192.168.3.0/24 allow'
- '192.168.5.0/24 allow'
- '192.168.10.0/24 allow'
- '10.1.71.0/24 allow'
unbound_access_control_tag: []
unbound_access_control_tag_action: []
unbound_access_control_tag_data: []
unbound_access_control_view: []
unbound_private_addresses:
- '192.168.1.0/24'
- '192.168.3.0/24'
- '192.168.5.0/24'
- '192.168.10.0/24'
- '10.1.71.0/24'
unbound_private_domains:
- 'mk-labs.cloud'
unbound_domains_insecure: []
unbound_do_not_query_addresses: []
unbound_local_zones: []
unbound_local_datas: []
unbound_local_data_ptrs: []
unbound_local_zone_tags: []
unbound_local_zone_overrides: []
unbound_trust_anchors: []
unbound_trusted_keys_files: []
### Remote Control ###
unbound_control_enable: "no"
unbound_control_interface: []
### Stub, forward zones and others ###
unbound_rpz_zones: [] # list of dicts
unbound_stub_zones: [] # list of dicts
unbound_forward_zones:
- name: 'int.mk-labs.cloud'
forward-addr:
- '10.1.71.5'
- name: '.'
forward-addr:
- '1.0.0.1@53#one.one.one.one'
- '1.1.1.1@53#one.one.one.one'
# For stub and forward zones, if a key inside one of the dict is an iterable,
# it will iterate inside it to duplicate the option with all the contained
# values.
# Can be useful in case of multiple forward-addr for a same zone:
# Example:
# unbound_forward_zones:
# - {name: "test.tld", "forward-addr": ["192.0.2.5", "192.0.2.6"]}
unbound_views: [] # list of strings, as options can be multiples

View File

@@ -0,0 +1,25 @@
---
# ------------------------------------------------------------------------------
# FILE: ansible/host_vars/astro_orbiter/vars.yml
# HOST: astro-orbiter (10.1.71.130)
# ROLE: Ollama inference host with AMD RX 5700 GPU passthrough
# ------------------------------------------------------------------------------
ansible_host: 10.1.71.130
ansible_user: wed
ansible_become: true
# LVM root expansion — xlarge template uses sda3 partition, standard VG/LV names
common_expand_root_lvm: true
common_root_pv: /dev/sda3
common_root_vg: ubuntu-vg
common_root_lv: ubuntu-lv
# Ollama — all defaults apply; explicitly documented here for visibility
ollama_rocm_version: "6.2"
ollama_default_model: "qwen3:8b"
ollama_hsa_override_gfx_version: "10.1.0"
ollama_data_disk: /dev/sdb
ollama_data_vg: ollama-vg
ollama_data_lv: ollama-lv
ollama_data_dir: /var/lib/ollama

View File

@@ -0,0 +1,16 @@
---
# ------------------------------------------------------------------------------
# FILE: ansible/host_vars/astro_orbiter/vars.yml
# HOST: astro-orbiter (10.1.71.130)
# ROLE: Ollama inference host with AMD RX 5700 GPU passthrough
# ------------------------------------------------------------------------------
ansible_host: 10.1.71.131
ansible_user: wed
ansible_become: true
# LVM root expansion — xlarge template uses sda3 partition, standard VG/LV names
common_expand_root_lvm: true
common_root_pv: /dev/sda3
common_root_vg: ubuntu-vg
common_root_lv: ubuntu-lv

View File

@@ -1,49 +0,0 @@
---
# file: host_vars/vm_template/vars
# Ansible vars template for hosts created via cloning.
# Supported hypervisors:
# - Proxmox
platform: "proxmox"
# This is the Proxmox node where all the VM templates are stored. (Templates are not global.)
proxmox_clone_node: "fantasyland"
# Templates are named in the following format (all lower case): <OS Distribution>-<OS Version>-<VM Size>
# Current OS offerings are:
# - Fedora (42)
# Current VM sizes are:
# - Small: 2 cores, 2GB memory, 8 GiB virtual disk
# - Medium: 2 cores, 4GB memory, 16 GiB virtual disk
# - Large: 4 cores, 4GB memory, 32 GiB virtual disk
# - Xlarge: 4 cores, 8GB memory, 64 GiB virtual disk
# - Xlarge Plus: 4 cores, 8GB memory, 128 GiB virtual disk
vm_clone_source: "fedora-42-xlarge-plus"
# Proxmox storage target.
vm_storage: "general"
# Proxmox does not yet do dynamic load balancing, the host target sets the target for HA groups
# and backup groups. (ha_group will be factored out in the next functionality update.)
ha_group: "fantasyland"
proxmox_host_target: "fantasyland"
# Currently, only single NIC VMs using IPv4 are supported via cloning. The IP address also
# sets the Proxmox VMID. The VMID is a combination of the 3rd and 4th octet of the IPv4 address.
ip_address: 10.1.71.21
# Software
# Future enhancement will allow specification of additional software to automatically deploy to
# the VM after creation.
#terraform_version: "1.11.3"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}" # Change variable to fqdn

View File

@@ -0,0 +1,12 @@
---
# file: host_vars/guest-relations/vars
# guest-relations — Guest Relations
# VM provisioned by Terraform (pre-pipeline bootstrap)
# ─── Network ─────────────────────────────────────────────────────────────────
ip_address: 10.1.71.40
# ─── Application ─────────────────────────────────────────────────────────────
app_role: identity_provider
app_name: authentik
app_deployment: docker_compose

View File

@@ -1,29 +0,0 @@
---
# file: host_vars/int-master01/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "master"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.61
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp0s31f6"
boot_mac_address: "54:bf:64:60:52:2d"
primary_interface: "enp1s0f0"
install_mac_address: "98:b7:85:20:47:22"
# Static Variables
hostname: "{{ inventory_hostname }}.{{ base_domain }}"

View File

@@ -1,29 +0,0 @@
---
# file: host_vars/int-master02/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "master"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.62
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp0s31f6"
boot_mac_address: "54:bf:64:5f:a2:80"
primary_interface: "enp1s0f0"
install_mac_address: "98:b7:85:20:47:3a"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}"

View File

@@ -1,29 +0,0 @@
---
# file: host_vars/int-master03/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "master"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.63
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp0s31f6"
boot_mac_address: "d8:9e:f3:4c:02:a9"
primary_interface: "enp1s0f0"
install_mac_address: "98:b7:85:1f:89:cf"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}"

View File

@@ -1,29 +0,0 @@
---
# file: host_vars/int-worker02/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "worker"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.164
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp87s0"
boot_mac_address: "36:47:ca:76:13:49"
primary_interface: "enp2s0f0"
install_mac_address: "36:47:ca:76:13:47"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}"

View File

@@ -1,29 +0,0 @@
---
# file: host_vars/int-worker02/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "worker"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.65
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp87s0"
boot_mac_address: "58:47:ca:76:13:49"
primary_interface: "enp2s0f0"
install_mac_address: "58:47:ca:76:13:47"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}"

View File

@@ -1,33 +1,12 @@
---
# file: host_vars/lightning_lane/vars
# Traefik Load Balancer Server
# lightning-lane — Traefik Reverse Proxy / Load Balancer
# VM provisioned by Terraform (pre-pipeline bootstrap)
# Supported hypervisors:
# - Proxmox
# ─── Network ─────────────────────────────────────────────────────────────────
ip_address: 10.1.71.35
platform: "proxmox"
# This is the Proxmox node where all the VM templates are stored. (Templates are not global.)
proxmox_clone_node: "fantasyland"
# VM Configuration
vm_os_distribution: "ubuntu"
vm_os_version: "24.04"
vm_size: "small"
vm_storage: "general"
# Proxmox HA Configuration
ha_group: "pve02"
proxmox_host_target: "pve02"
# Network Configuration
# IP address for the Traefik load balancer
ip_address: 10.1.71.24
#vm_mac_address: 'BC:24:11:50:00:80'
# Software Configuration
# Traefik will be installed and configured on this host
# VM Template Configuration
#vm_clone_source: "{{ vm_os_distribution }}-{{ vm_os_version }}-{{ vm_size }}"
hostname: "{{ inventory_hostname }}.{{ base_domain }}"
# ─── Application ─────────────────────────────────────────────────────────────
app_role: reverse_proxy
app_name: traefik
app_deployment: docker_compose

View File

@@ -1,17 +0,0 @@
---
# file: host_vars/monitor/vars
# VM Definition
platform: "proxmox"
proxmox_clone_node: "pve03"
vm_clone_source: "fedora-42-large"
vm_storage: "pve-general"
ha_group: "pve03"
# Networking
ip_address: 10.1.71.249
# Software
#terraform_version: "1.11.3"
hostname: "{{ inventory_hostname }}.{{ base_domain }}" #change variable to fqdn

View File

@@ -1,31 +0,0 @@
---
# file: host_vars/ocp-hub/vars
# file: host_vars/int-master01/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "master"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.10
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp0s31f6"
boot_mac_address: "50:9A:4C:52:A8:5D"
primary_interface: "enp1s0f0"
install_mac_address: "98:b7:85:1e:c6:f1"
# Static Variables
hostname: "{{ inventory_hostname }}.{{ base_domain }}"

View File

@@ -1,29 +0,0 @@
---
# file: host_vars/openshift_node_template/vars
# Ansible vars template for OpenShift nodes created via Ansible (configure_openshift_cluster).
# Cluster deployed via Agent Installer and Matchbox (iPXE):
# Role of this node in the cluster.
# - master
# - worker
node_role: "master"
# IP address assigned to this node on the Primary Interface
ip_address: 10.1.71.61
# Ansible Playbook supports booting node via one interface, but installing on a
# different interface. (Disabling the boot interface on install) All these fields
# are necessary. In the case of a single NIC node, interfaces and MAC addresses
# should be the same.
boot_interface: "enp0s31f6"
boot_mac_address: "54:bf:64:60:52:2d"
primary_interface: "enp1s0f0"
install_mac_address: "98:b7:85:20:47:22"
# Static Variables
hostname: "{{ inventory_hostname }}.{{ base_domain }}" # Change variable to fqdn

View File

@@ -0,0 +1,27 @@
---
# file: host_vars/turnstile/vars
# turnstile — Smallstep step-ca SSH Certificate Authority
# VM provisioned by Terraform (pre-pipeline bootstrap)
# ─── Network ─────────────────────────────────────────────────────────────────
ip_address: 10.1.71.34
# ─── Application ─────────────────────────────────────────────────────────────
app_role: ssh_certificate_authority
app_name: step-ca
app_deployment: docker_compose
# ─── step-ca Configuration ───────────────────────────────────────────────────
stepca_hostname: turnstile.local.mk-labs.cloud
stepca_dns_names: "turnstile.local.mk-labs.cloud,10.1.71.34"
stepca_ssh_enabled: true
stepca_listen_port: 9000
# ─── OIDC Provisioner (Authentik) ────────────────────────────────────────────
# Client ID and secret stored in vault
stepca_oidc_provisioner_name: authentik
stepca_oidc_configuration_endpoint: "https://authentik.local.mk-labs.cloud/application/o/step-ca/.well-known/openid-configuration"
stepca_oidc_listen_address: ":10000"
stepca_oidc_domains:
- "local.mk-labs.cloud"
- "protonmail.com"

View File

@@ -0,0 +1,17 @@
$ANSIBLE_VAULT;1.1;AES256
30383930323363386234333433636465393263613336646464666365643730386430353864616334
3865343031653162313736656437373666636136653263360a613961646133376262633164393535
31363434633634303035663133316230633538363936303266373931313661346563333832653032
6239376363646530620a303231363333363233623038326165316463383662656565626534396232
62333133623530643932643430663166373930353733363866336533643233373261333130613635
31633337376337663833613634666436383862386431636537373363343333323932363761653366
61353166613162346436396138316561646165303566376366323462663861616364336539313138
30386430373566353761383636626335393463376661356666303861353564313832346333396134
30653861363866336331336164323130623230666237356165613934333239386536373664643065
33643139346562346663313533643433353462363665323166313364373335366665373435643935
65303863663864393238393732303065343364306264396333376233666233346664323334336532
38373164383835663163363137643531303163396236623565666436363261393563343133333161
36663934303633663062346161636333396135336135616136303664636562363862353764343562
39643430396636313139336130656363306562363430346233313530663963366238326334623135
62383632613061316262323038303333323739303137363334626637666663666263613132336433
65346266393733633632

View File

@@ -1,47 +0,0 @@
---
# file: host_vars/unbound01/vars
# Supported hypervisors:
# - Proxmox
platform: "proxmox"
# This is the Proxmox node where all the VM templates are stored. (Templates are not global.)
proxmox_clone_node: "pve03"
# Templates are named in the following format (all lower case): <OS Distribution>-<OS Version>-<VM Size>
# Current OS offerings are:
# - Fedora (42)
# Current VM sizes are:
# - Small: 2 cores, 2GB memory, 8 GiB virtual disk
# - Medium: 2 cores, 4GB memory, 16 GiB virtual disk
# - Large: 4 cores, 4GB memory, 32 GiB virtual disk
# - Xlarge: 4 cores, 8GB memory, 64 GiB virtual disk
vm_clone_source: "fedora-42-small"
# Proxmox storage target.
vm_storage: "pve-general"
# Proxmox does not yet do dynamic load balancing, the host target sets the target for HA groups
# and backup groups. (ha_group will be factored out in the next functionality update.)
ha_group: "pve03"
proxmox_host_target: "pve03"
# Currently, only single NIC VMs using IPv4 are supported via cloning. The IP address also
# sets the Proxmox VMID. The VMID is a combination of the 3rd and 4th octet of the IPv4 address.
ip_address: 10.1.71.251
# Software
# Future enhancement will allow specification of additional software to automatically deploy to
# the VM after creation.
#terraform_version: "1.11.3"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}" # Change variable to fqdn

View File

@@ -1,47 +0,0 @@
---
# file: host_vars/unbound02/vars
# Supported hypervisors:
# - Proxmox
platform: "proxmox"
# This is the Proxmox node where all the VM templates are stored. (Templates are not global.)
proxmox_clone_node: "pve03"
# Templates are named in the following format (all lower case): <OS Distribution>-<OS Version>-<VM Size>
# Current OS offerings are:
# - Fedora (42)
# Current VM sizes are:
# - Small: 2 cores, 2GB memory, 8 GiB virtual disk
# - Medium: 2 cores, 4GB memory, 16 GiB virtual disk
# - Large: 4 cores, 4GB memory, 32 GiB virtual disk
# - Xlarge: 4 cores, 8GB memory, 64 GiB virtual disk
vm_clone_source: "fedora-42-small"
# Proxmox storage target.
vm_storage: "pve-general"
# Proxmox does not yet do dynamic load balancing, the host target sets the target for HA groups
# and backup groups. (ha_group will be factored out in the next functionality update.)
ha_group: "pve03"
proxmox_host_target: "pve03"
# Currently, only single NIC VMs using IPv4 are supported via cloning. The IP address also
# sets the Proxmox VMID. The VMID is a combination of the 3rd and 4th octet of the IPv4 address.
ip_address: 10.1.71.252
# Software
# Future enhancement will allow specification of additional software to automatically deploy to
# the VM after creation.
#terraform_version: "1.11.3"
# ---
hostname: "{{ inventory_hostname }}.{{ base_domain }}" # Change variable to fqdn

View File

@@ -1,15 +1,12 @@
# file: inventory.yml
proxmox:
hosts:
pve0[1:3]:
ntp_servers:
hosts:
sundial:
dns_server:
hosts:
monorail:
main-street-usa:
ansible_host: 10.1.71.11
ansible_user: wed
ansible_become: true
tomorrowland:
fantasyland:
magic_kingdom:
hosts:
@@ -17,47 +14,80 @@ magic_kingdom:
tomorrowland:
fantasyland:
prometheus_server:
dns_server:
hosts:
cinderella-castle:
monorail:
prometheus_nodes:
ntp_servers:
hosts:
cinderella-castle:
sundial:
load_balancers:
hosts:
lightning-lane:
ansible_host: 10.1.71.24
ansible_become: true
# node_explorer:
# hosts:
# cinderella-castle:
# alertmanager:
# hosts:
# cinderella-castle:
step_ca_server:
hosts:
turnstile:
ansible_become: true
# grafana:
# hosts:
# cinderella-castle:
authentik_server:
hosts:
guest-relations:
ansible_become: true
gitea_servers:
hosts:
mad-tea-party:
ansible_user: wed
ansible_become: true
nextcloud_server:
hosts:
the-grid:
ansible_become: true
semaphore_server:
hosts:
figment:
ansible_host: 10.1.71.37
ansible_user: wed
ansible_become: true
n8n_server:
hosts:
tiki-room:
ollama_server:
hosts:
astro-orbiter:
ansible_host: 10.1.71.130
ansible_user: wed
ansible_become: true
hermes_server:
hosts:
carousel-of-progress:
ansible_host: 10.1.71.131
ansible_user: wed
ansible_become: true
honcho_server:
hosts:
lincoln:
ansible_host: 10.1.71.132
ansible_user: wed
ansible_become: true
papermc_server:
# ansible-galaxy role install engonzal.papermc
hosts:
arcade:
semaphore_server:
dev_servers:
hosts:
imagineering:
n8n_server:
hosts:
tiki-room:
traefik_server:
hosts:
lightning_lane:
scrim:
# dhcp_server:
# hosts:
@@ -67,13 +97,13 @@ backup_servers:
hosts:
timekeeper:
# freeipa:
# hosts:
# infra01:
# ipaserver:
# hosts:
# infra01.int.mk-labs.cloud:
talos_control:
hosts:
city-hall:
ansible_become: true
vars:
talosctl_version: "v1.12.4"
talos_cluster_name: "fastpass"
# matchbox_server:
# hosts:
@@ -82,51 +112,3 @@ backup_servers:
terraform_server:
hosts:
infra01:
# hub_cluster:
# hosts:
# ocp-hub:
# internal_cluster:
# hosts:
# int-master01:
# int-master02:
# int-master03:
# # int-worker01:
# int-worker02:
# talos_cluster:
# hosts:
# splash:
# fastpass_control_plane:
# hosts:
# space-mountain:
# big-thunder-mountain:
# splash-mountain:
# fastpass_workers:
# hosts:
# haunted-mansion: # backstage
# peter-pans-flight: # backstage
# fastpass:
# children:
# fastpass_control_plane:
# fastpass_workers:
# sql_servers:
# hosts:
# sql01:
# docker_servers:
# hosts:
# docker01:
# observer:
# hosts:
# prometheus:
# target:
# hosts:
# prometheus:

View File

@@ -0,0 +1,95 @@
---
# ------------------------------------------------------------------------------
# FILE: playbooks/add_service_route.yml
# DESCRIPTION: Ensures all services in the Traefik dynamic config directory
# are routed and have DNS CNAME records on monorail.
#
# 1. Syncs boilerplates/traefik/dynamic/ to lightning-lane
# 2. Scans the directory for service configs
# 3. Extracts all hostnames from Host() rules (supports multi-host)
# 4. Creates CNAME records for each hostname -> lightning-lane
#
# PREREQUISITES:
# - Service dynamic config YAML committed to boilerplates/traefik/dynamic/
# - vault_technitium_api_key defined in group_vars/all/vault
#
# USAGE:
# ansible-playbook -i inventory.yml playbooks/add_service_route.yml
# ------------------------------------------------------------------------------
- name: Sync Traefik routes and ensure DNS records
hosts: localhost
connection: local
gather_facts: false
vars:
base_domain: "local.mk-labs.cloud"
dns_server: "monorail"
traefik_host: "lightning-lane.local.mk-labs.cloud"
traefik_user: "wed"
traefik_dynamic_path: "/opt/docker/traefik/dynamic/"
dynamic_config_dir: "{{ playbook_dir }}/../../boilerplates/traefik/dynamic"
# Files in the dynamic directory that are NOT service routes
exclude_configs:
- default.yml
tasks:
# ── Step 1: Sync dynamic config to lightning-lane ──
- name: Sync Traefik dynamic configuration to lightning-lane
ansible.builtin.shell: >
rsync -av --delete
{{ dynamic_config_dir }}/
{{ traefik_user }}@{{ traefik_host }}:{{ traefik_dynamic_path }}
register: sync_result
changed_when: "'sending incremental file list' in sync_result.stdout"
# ── Step 2: Discover hostnames from Traefik router rules ──
- name: Find all dynamic config files
ansible.builtin.find:
paths: "{{ dynamic_config_dir }}"
patterns: "*.yml"
register: config_files
- name: Read config files
ansible.builtin.slurp:
src: "{{ item.path }}"
register: slurped_configs
loop: "{{ config_files.files }}"
when: item.path | basename not in exclude_configs
- name: Extract all hostnames from Host() rules
ansible.builtin.set_fact:
hostnames: >-
{% set hosts = [] -%}
{% for result in slurped_configs.results if result.content is defined -%}
{% set content = result.content | b64decode -%}
{% for match in content | regex_findall('Host\(`([^`]+)`\)') -%}
{% for h in match.split(' || ') -%}
{% set h = h | regex_replace('`', '') | trim -%}
{% if h.endswith('.local.mk-labs.cloud') and h not in hosts -%}
{% set _ = hosts.append(h) -%}
{% endif -%}
{% endfor -%}
{% endfor -%}
{% endfor -%}
{{ hosts | unique | list }}
- name: Display hostnames to create
ansible.builtin.debug:
msg: "Hostnames found: {{ hostnames }}"
# ── Step 3: Create DNS CNAME records ──
- name: Create DNS CNAME record for each hostname
effectivelywild.technitium_dns.technitium_dns_add_record:
api_url: "http://{{ dns_server }}.{{ base_domain }}"
api_token: "{{ vault_technitium_api_key }}"
zone: "{{ base_domain }}"
name: "{{ item }}"
type: "CNAME"
cname: "lightning-lane.{{ base_domain }}"
ttl: 360
validate_certs: false
loop: "{{ hostnames }}"
loop_control:
label: "{{ item }}"

View File

@@ -1,218 +0,0 @@
---
- name: Configure DNS entries for cluster nodes
hosts: internal_cluster
gather_facts: false
become: true
tasks:
- name: Create DNS entry for each cluster node
ansible.builtin.include_tasks: tasks/create_dns_record.yml
vars:
host_ip_address: "{{ ip_address }}"
- name: Configure DNS entries for OpenShift cluster services
hosts: internal_cluster[0]
gather_facts: false
become: true
tasks:
- name: Create DNS entries for OpenShift cluster services
ansible.builtin.include_tasks: tasks/create_dns_record.yml
vars:
dns_name: "{{ item.name }}"
dns_address: "{{ item.address }}"
loop:
- name: "api.{{ cluster_name }}"
address: "{{ api_address }}"
- name: "api-int.{{ cluster_name }}"
address: "{{ api_address }}"
- name: "*.apps.{{ cluster_name }}"
address: "{{ app_address }}"
loop_control:
label: "{{ item.name }}"
- name: Configure DHCP entries for cluster nodes
hosts: internal_cluster
gather_facts: false
become: true
tasks:
- name: Configure DHCP entry for node
ansible.builtin.include_tasks: tasks/configure_dhcp_entry.yml
vars:
host_mac_address: "{{ install_mac_address }}"
host_ip_address: "{{ ip_address }}"
handlers:
- name: Restart dnsmasq
delegate_to: "{{ dhcp_server }}"
become: true
ansible.builtin.service:
name: dnsmasq
state: restarted
- name: Create OpenShift installer files
hosts: matchbox
gather_facts: true
vars:
cluster_group: "internal_cluster"
download_dir: "/var/cache/openshift-install"
tasks:
- name: Remove previous directory
ansible.builtin.file:
path: ~/homelab/terraform/{{ cluster_group }}
state: absent
- name: Create directory
ansible.builtin.file:
path: ~/homelab/terraform/{{ cluster_group }}
state: directory
mode: '0755'
- name: OpenShift Installer install-config.yaml
ansible.builtin.template:
src: templates/install-config.yaml.j2
dest: ~/homelab/terraform/{{ cluster_group }}/install-config.yaml
mode: '0644'
- name: OpenShift Installer agent-config.yaml
ansible.builtin.template:
src: templates/agent-config.yaml.j2
dest: ~/homelab/terraform/{{ cluster_group }}/agent-config.yaml
mode: '0644'
- name: Create directory for OpenShift Installer files
become: true
ansible.builtin.file:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}"
state: directory
mode: '0755'
owner: root
group: root
- name: Check if OpenShift installer file exists
ansible.builtin.stat:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}/{{ hostvars[groups[cluster_group][0]].openshift_installer_file }}"
register: openshift_installer_file
- name: Download and install OpenShift installer package
when: not openshift_installer_file.stat.exists
block:
- name: Download OpenShift Installer file
become: true
ansible.builtin.get_url:
url: "{{ hostvars[groups[cluster_group][0]].openshift_installer_download_url }}"
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
mode: '0644'
- name: Unarchive installer file
become: true
ansible.builtin.unarchive:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-install-linux.tar.gz
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
remote_src: true
- name: Copy install binary
become: true
ansible.builtin.copy:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-install
dest: /usr/local/bin
remote_src: true
owner: wed
group: wed
mode: '0755'
- name: Check if OpenShift client file exists
ansible.builtin.stat:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}/{{ hostvars[groups[cluster_group][0]].openshift_client_file }}"
register: openshift_client_file
- name: Download and install OpenShift client package
when: not openshift_client_file.stat.exists
block:
- name: Download OpenShift Client file
become: true
ansible.builtin.get_url:
url: "{{ hostvars[groups[cluster_group][0]].openshift_client_download_url }}"
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
mode: '0644'
- name: Unarchive client file
become: true
ansible.builtin.unarchive:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-client-linux.tar.gz
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
remote_src: true
- name: Copy client binary
become: true
ansible.builtin.copy:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/oc
dest: /usr/local/bin
remote_src: true
owner: wed
group: wed
mode: '0755'
- name: Check if OpenShift image exists
ansible.builtin.stat:
path: ~/homelab/terraform/{{ cluster_group }}/boot-artifacts/agent.x86_64-initrd.img
register: openshift_image_exists
- name: OpenShift image creation
ansible.builtin.command: /usr/local/bin/openshift-install agent create pxe-files
args:
chdir: ~/homelab/terraform/{{ cluster_group }}
when: not openshift_image_exists.stat.exists
changed_when: true
- name: Create a directory if it does not exist
become: true
ansible.builtin.file:
path: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}"
state: directory
owner: matchbox
group: matchbox
mode: '0755'
- name: Copy file with owner and permissions
become: true
ansible.builtin.copy:
src: "{{ ansible_env['HOME'] }}/homelab/terraform/{{ cluster_group }}/boot-artifacts/"
dest: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}/"
remote_src: true
owner: matchbox
group: matchbox
mode: '0644'
- name: Verify directory permissions
become: true
ansible.builtin.file:
path: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}"
state: directory
owner: matchbox
group: matchbox
mode: '0755'
- name: Create Terraform matchbox groups file from template
ansible.builtin.template:
src: templates/groups.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/groups.tf
mode: '0644'
- name: Create Terraform matchbox profiles file from template
ansible.builtin.template:
src: templates/profiles.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/profiles.tf
mode: '0644'
- name: Create Terraform matchbox provider file from template
ansible.builtin.template:
src: templates/provider.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/provider.tf
mode: '0644'
- name: Configure Matchbox via Terraform
community.general.terraform:
project_path: ~/homelab/terraform/{{ cluster_group }}
state: present
force_init: true

View File

@@ -1,221 +0,0 @@
---
- name: Configure DNS entries for cluster nodes
hosts: hub_cluster
gather_facts: false
become: true
tasks:
- name: Create DNS entry for each cluster node
ansible.builtin.include_tasks: tasks/create_dns_record.yml
vars:
host_ip_address: "{{ ip_address }}"
- name: Configure DNS entries for OpenShift cluster services
hosts: hub_cluster[0]
gather_facts: false
become: true
tasks:
- name: Create DNS entries for OpenShift cluster services
ansible.builtin.include_tasks: tasks/create_dns_record.yml
vars:
dns_name: "{{ item.name }}"
dns_address: "{{ item.address }}"
loop:
- name: "api.{{ cluster_name }}"
address: "{{ api_address }}"
- name: "api-int.{{ cluster_name }}"
address: "{{ api_address }}"
- name: "*.apps.{{ cluster_name }}"
address: "{{ app_address }}"
loop_control:
label: "{{ item.name }}"
- name: Configure DHCP entries for cluster nodes
hosts: hub_cluster
gather_facts: false
become: true
tasks:
- name: Configure DHCP entry for node
ansible.builtin.include_tasks: tasks/configure_dhcp_entry.yml
vars:
host_mac_address: "{{ install_mac_address }}"
host_ip_address: "{{ ip_address }}"
handlers:
- name: Restart dnsmasq
delegate_to: "{{ dhcp_server }}"
become: true
ansible.builtin.service:
name: dnsmasq
state: restarted
- name: Create OpenShift installer files
hosts: matchbox
gather_facts: true
vars:
cluster_group: "hub_cluster"
download_dir: "/var/cache/openshift-install"
tasks:
- name: Remove previous directory
ansible.builtin.file:
path: ~/homelab/terraform/{{ cluster_group }}
state: absent
- name: Create directory
ansible.builtin.file:
path: ~/homelab/terraform/{{ cluster_group }}
state: directory
mode: '0755'
- name: OpenShift Installer install-config.yaml
ansible.builtin.template:
src: templates/install-config.yaml.j2
dest: ~/homelab/terraform/{{ cluster_group }}/install-config.yaml
mode: '0644'
- name: OpenShift Installer agent-config.yaml
ansible.builtin.template:
src: templates/agent-config.yaml.j2
dest: ~/homelab/terraform/{{ cluster_group }}/agent-config.yaml
mode: '0644'
# - name: Quit
# ansible.builtin.meta: end_play
- name: Create directory for OpenShift Installer files
become: true
ansible.builtin.file:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}"
state: directory
mode: '0755'
owner: root
group: root
- name: Check if OpenShift installer file exists
ansible.builtin.stat:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}/{{ hostvars[groups[cluster_group][0]].openshift_installer_file }}"
register: openshift_installer_file
- name: Download and install OpenShift installer package
when: not openshift_installer_file.stat.exists
block:
- name: Download OpenShift Installer file
become: true
ansible.builtin.get_url:
url: "{{ hostvars[groups[cluster_group][0]].openshift_installer_download_url }}"
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
mode: '0644'
- name: Unarchive installer file
become: true
ansible.builtin.unarchive:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-install-linux.tar.gz
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
remote_src: true
- name: Copy install binary
become: true
ansible.builtin.copy:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-install
dest: /usr/local/bin
remote_src: true
owner: wed
group: wed
mode: '0755'
- name: Check if OpenShift client file exists
ansible.builtin.stat:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}/{{ hostvars[groups[cluster_group][0]].openshift_client_file }}"
register: openshift_client_file
- name: Download and install OpenShift client package
when: not openshift_client_file.stat.exists
block:
- name: Download OpenShift Client file
become: true
ansible.builtin.get_url:
url: "{{ hostvars[groups[cluster_group][0]].openshift_client_download_url }}"
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
mode: '0644'
- name: Unarchive client file
become: true
ansible.builtin.unarchive:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-client-linux.tar.gz
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
remote_src: true
- name: Copy client binary
become: true
ansible.builtin.copy:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/oc
dest: /usr/local/bin
remote_src: true
owner: wed
group: wed
mode: '0755'
- name: Check if OpenShift image exists
ansible.builtin.stat:
path: ~/homelab/terraform/{{ cluster_group }}/boot-artifacts/agent.x86_64-initrd.img
register: openshift_image_exists
- name: OpenShift image creation
ansible.builtin.command: /usr/local/bin/openshift-install agent create pxe-files
args:
chdir: ~/homelab/terraform/{{ cluster_group }}
when: not openshift_image_exists.stat.exists
changed_when: true
- name: Create a directory if it does not exist
become: true
ansible.builtin.file:
path: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}"
state: directory
owner: matchbox
group: matchbox
mode: '0755'
- name: Copy file with owner and permissions
become: true
ansible.builtin.copy:
src: "{{ ansible_env['HOME'] }}/homelab/terraform/{{ cluster_group }}/boot-artifacts/"
dest: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}/"
remote_src: true
owner: matchbox
group: matchbox
mode: '0644'
- name: Verify directory permissions
become: true
ansible.builtin.file:
path: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}"
state: directory
owner: matchbox
group: matchbox
mode: '0755'
- name: Create Terraform matchbox groups file from template
ansible.builtin.template:
src: templates/groups.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/groups.tf
mode: '0644'
- name: Create Terraform matchbox profiles file from template
ansible.builtin.template:
src: templates/profiles.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/profiles.tf
mode: '0644'
- name: Create Terraform matchbox provider file from template
ansible.builtin.template:
src: templates/provider.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/provider.tf
mode: '0644'
- name: Configure Matchbox via Terraform
community.general.terraform:
project_path: ~/homelab/terraform/{{ cluster_group }}
state: present
force_init: true

View File

@@ -1,218 +0,0 @@
---
- name: Configure DNS entries for cluster nodes
hosts: internal_cluster
gather_facts: false
become: true
tasks:
- name: Create DNS entry for each cluster node
ansible.builtin.include_tasks: tasks/create_dns_record.yml
vars:
host_ip_address: "{{ ip_address }}"
- name: Configure DNS entries for OpenShift cluster services
hosts: internal_cluster[0]
gather_facts: false
become: true
tasks:
- name: Create DNS entries for OpenShift cluster services
ansible.builtin.include_tasks: tasks/create_dns_record.yml
vars:
dns_name: "{{ item.name }}"
dns_address: "{{ item.address }}"
loop:
- name: "api.{{ cluster_name }}"
address: "{{ api_address }}"
- name: "api-int.{{ cluster_name }}"
address: "{{ api_address }}"
- name: "*.apps.{{ cluster_name }}"
address: "{{ app_address }}"
loop_control:
label: "{{ item.name }}"
- name: Configure DHCP entries for cluster nodes
hosts: internal_cluster
gather_facts: false
become: true
tasks:
- name: Configure DHCP entry for node
ansible.builtin.include_tasks: tasks/configure_dhcp_entry.yml
vars:
host_mac_address: "{{ install_mac_address }}"
host_ip_address: "{{ ip_address }}"
handlers:
- name: Restart dnsmasq
delegate_to: "{{ dhcp_server }}"
become: true
ansible.builtin.service:
name: dnsmasq
state: restarted
- name: Create OpenShift installer files
hosts: matchbox
gather_facts: true
vars:
cluster_group: "internal_cluster"
download_dir: "/var/cache/openshift-install"
tasks:
- name: Remove previous directory
ansible.builtin.file:
path: ~/homelab/terraform/{{ cluster_group }}
state: absent
- name: Create directory
ansible.builtin.file:
path: ~/homelab/terraform/{{ cluster_group }}
state: directory
mode: '0755'
- name: OpenShift Installer install-config.yaml
ansible.builtin.template:
src: templates/install-config.yaml.j2
dest: ~/homelab/terraform/{{ cluster_group }}/install-config.yaml
mode: '0644'
- name: OpenShift Installer agent-config.yaml
ansible.builtin.template:
src: templates/agent-config.yaml.j2
dest: ~/homelab/terraform/{{ cluster_group }}/agent-config.yaml
mode: '0644'
- name: Create directory for OpenShift Installer files
become: true
ansible.builtin.file:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}"
state: directory
mode: '0755'
owner: root
group: root
- name: Check if OpenShift installer file exists
ansible.builtin.stat:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}/{{ hostvars[groups[cluster_group][0]].openshift_installer_file }}"
register: openshift_installer_file
- name: Download and install OpenShift installer package
when: not openshift_installer_file.stat.exists
block:
- name: Download OpenShift Installer file
become: true
ansible.builtin.get_url:
url: "{{ hostvars[groups[cluster_group][0]].openshift_installer_download_url }}"
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
mode: '0644'
- name: Unarchive installer file
become: true
ansible.builtin.unarchive:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-install-linux.tar.gz
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
remote_src: true
- name: Copy install binary
become: true
ansible.builtin.copy:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-install
dest: /usr/local/bin
remote_src: true
owner: wed
group: wed
mode: '0755'
- name: Check if OpenShift client file exists
ansible.builtin.stat:
path: "{{ download_dir }}/{{ hostvars[groups[cluster_group][0]].cluster_version }}/{{ hostvars[groups[cluster_group][0]].openshift_client_file }}"
register: openshift_client_file
- name: Download and install OpenShift client package
when: not openshift_client_file.stat.exists
block:
- name: Download OpenShift Client file
become: true
ansible.builtin.get_url:
url: "{{ hostvars[groups[cluster_group][0]].openshift_client_download_url }}"
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
mode: '0644'
- name: Unarchive client file
become: true
ansible.builtin.unarchive:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/openshift-client-linux.tar.gz
dest: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}
remote_src: true
- name: Copy client binary
become: true
ansible.builtin.copy:
src: /var/cache/openshift-install/{{ hostvars[groups[cluster_group][0]].cluster_version }}/oc
dest: /usr/local/bin
remote_src: true
owner: wed
group: wed
mode: '0755'
- name: Check if OpenShift image exists
ansible.builtin.stat:
path: ~/homelab/terraform/{{ cluster_group }}/boot-artifacts/agent.x86_64-initrd.img
register: openshift_image_exists
- name: OpenShift image creation
ansible.builtin.command: /usr/local/bin/openshift-install agent create pxe-files
args:
chdir: ~/homelab/terraform/{{ cluster_group }}
when: not openshift_image_exists.stat.exists
changed_when: true
- name: Create a directory if it does not exist
become: true
ansible.builtin.file:
path: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}"
state: directory
owner: matchbox
group: matchbox
mode: '0755'
- name: Copy file with owner and permissions
become: true
ansible.builtin.copy:
src: "{{ ansible_env['HOME'] }}/homelab/terraform/{{ cluster_group }}/boot-artifacts/"
dest: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}/"
remote_src: true
owner: matchbox
group: matchbox
mode: '0644'
- name: Verify directory permissions
become: true
ansible.builtin.file:
path: "{{ hostvars[groups['matchbox_server'][0]].assets }}/{{ hostvars[groups[cluster_group][0]].cluster_name }}"
state: directory
owner: matchbox
group: matchbox
mode: '0755'
- name: Create Terraform matchbox groups file from template
ansible.builtin.template:
src: templates/groups.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/groups.tf
mode: '0644'
- name: Create Terraform matchbox profiles file from template
ansible.builtin.template:
src: templates/profiles.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/profiles.tf
mode: '0644'
- name: Create Terraform matchbox provider file from template
ansible.builtin.template:
src: templates/provider.tf.j2
dest: ~/homelab/terraform/{{ cluster_group }}/provider.tf
mode: '0644'
- name: Configure Matchbox via Terraform
community.general.terraform:
project_path: ~/homelab/terraform/{{ cluster_group }}
state: present
force_init: true

View File

@@ -0,0 +1,20 @@
---
# ansible/playbooks/configure_proxmox_oidc.yml
#
# Configures Proxmox OIDC authentication with Authentik.
# Only targets one node since realm config is cluster-wide.
#
# Usage:
# cd ansible
# ansible-playbook -i inventory.yml playbooks/configure_proxmox_oidc.yml
#
# To also set up ACL entries for your user:
# ansible-playbook -i inventory.yml playbooks/configure_proxmox_oidc.yml \
# -e '{"proxmox_oidc_acl_entries": [{"path": "/", "user": "rblundon@authentik", "role": "Administrator"}]}'
- name: Configure Proxmox Authentik OIDC
hosts: main-street-usa
become: true
roles:
- role: proxmox
tags: [proxmox-oidc]

View File

@@ -0,0 +1,6 @@
---
- name: Apply day0 baseline
hosts: "{{ target | default('all') }}"
become: true
roles:
- day0-baseline

View File

@@ -0,0 +1,32 @@
---
# ============================================================================
# day0_expand_root_lv.yml
# ----------------------------------------------------------------------------
# Reclaims unallocated PE on the root volume group, extending the root LV
# to fill the VG and resizing the underlying filesystem (ext4 or xfs).
#
# Belongs to the day0 host-provisioning lifecycle. The Ubuntu Server
# autoinstall template ships with the root LV at ~half the disk size by
# default; this playbook is the canonical one-shot fix-up for that.
#
# Idempotent and safe to re-run. Hosts without LVM are no-op'd cleanly.
#
# Opt-out: set `expand_root_lv_skip: true` in host_vars/<host>.yml for
# hosts where free PE should NOT be claimed by root (e.g. hosts with a
# planned second LV in the same VG for application data).
#
# Usage:
# ansible-playbook playbooks/day0_expand_root_lv.yml
# ansible-playbook playbooks/day0_expand_root_lv.yml -e target=lincoln
# ansible-playbook playbooks/day0_expand_root_lv.yml -e target=honcho_server
# ============================================================================
- name: Expand root logical volume to fill VG
hosts: "{{ target | default('all') }}"
become: true
gather_facts: true
tasks:
- name: Apply expand_root_lv role unless host opts out
ansible.builtin.include_role:
name: expand_root_lv
when: not (expand_root_lv_skip | default(false) | bool)

View File

@@ -0,0 +1,23 @@
---
# ============================================================================
# day0_linux_baseline.yml
# ----------------------------------------------------------------------------
# Applies the mk-labs Linux baseline (linux-baseline role) to one or more
# hosts. Idempotent and safe to re-run.
#
# Usage:
# ansible-playbook playbooks/day0_linux_baseline.yml
# ansible-playbook playbooks/day0_linux_baseline.yml -e target=figment
# ansible-playbook playbooks/day0_linux_baseline.yml -e target=semaphore_server
#
# To trigger an opt-in full system upgrade:
# ansible-playbook playbooks/day0_linux_baseline.yml \
# -e target=figment -e 'baseline_features={"full_upgrade": true}'
# ============================================================================
- name: Apply mk-labs Linux baseline
hosts: "{{ target | default('all') }}"
become: true
gather_facts: true
roles:
- linux-baseline

View File

@@ -0,0 +1,30 @@
---
# ============================================================================
# day0_provision.yml
# ----------------------------------------------------------------------------
# Umbrella day0 playbook. Runs the full host-provisioning lifecycle in
# the correct order against newly-built VMs, so the operator runs ONE
# command per new host rather than chaining day0 steps manually.
#
# Order matters:
# 1. linux-baseline — timezone, NTP, packages, SSH hardening, jarvis user
# 2. expand_root_lv — reclaim PE left unallocated by the Ubuntu
# autoinstall template default
#
# Idempotent: every step is safe to re-run. Suitable to apply periodically
# from Semaphore as a baseline-drift check.
#
# Usage:
# ansible-playbook playbooks/day0_provision.yml -e target=lincoln
# ansible-playbook playbooks/day0_provision.yml -e target=honcho_server
#
# For finer control over a single phase, the constituent playbooks are:
# playbooks/day0_linux_baseline.yml
# playbooks/day0_expand_root_lv.yml
# ============================================================================
- name: Import day0 linux baseline
ansible.builtin.import_playbook: day0_linux_baseline.yml
- name: Import day0 expand root LV
ansible.builtin.import_playbook: day0_expand_root_lv.yml

View File

@@ -0,0 +1,400 @@
---
# ansible/playbooks/day1_configure_netbox_catalog.yml
#
# Creates the tag taxonomy and custom fields in NetBox for service catalog
# documentation. Run once (idempotent — uses name-based checks).
#
# Usage (from ansible/ directory):
# ansible-playbook playbooks/day1_configure_netbox_catalog.yml
#
# Requires:
# - vault_netbox_token in Ansible Vault
# - netbox reachable at http://fire-station.local.mk-labs.cloud
- name: Configure NetBox service catalog taxonomy
hosts: localhost
gather_facts: false
vars_files:
- "{{ playbook_dir }}/../group_vars/all/vault"
vars:
netbox_url: "http://fire-station.local.mk-labs.cloud"
netbox_token: "{{ vault_netbox_token }}"
netbox_api: "{{ netbox_url }}/api"
headers:
Authorization: "Token {{ netbox_token }}"
Content-Type: "application/json"
Accept: "application/json"
# ── Tag definitions ───────────────────────────────────────────────────
tags:
# Infrastructure type
- name: k8s
slug: k8s
color: "2196f3" # blue
description: "Workload running in the fastpass Kubernetes cluster"
- name: vm
slug: vm
color: "4caf50" # green
description: "Traditional VM or LXC on Proxmox"
# Service tier
- name: platform
slug: platform
color: "9c27b0" # purple
description: "Platform/infrastructure service (not user-facing)"
- name: application
slug: application
color: "ff9800" # orange
description: "User-facing application workload"
# Service categories
- name: monitoring
slug: monitoring
color: "607d8b" # grey
description: "Metrics, logging, alerting"
- name: auth
slug: auth
color: "607d8b"
description: "Authentication and SSO"
- name: gitops
slug: gitops
color: "607d8b"
description: "GitOps and CI/CD"
- name: dashboard
slug: dashboard
color: "607d8b"
description: "Dashboard and portal services"
- name: storage
slug: storage
color: "607d8b"
description: "Storage and file services"
- name: dns
slug: dns
color: "607d8b"
description: "DNS and name resolution"
- name: automation
slug: automation
color: "607d8b"
description: "Automation and orchestration"
- name: networking
slug: networking
color: "607d8b"
description: "Network infrastructure services"
- name: inference
slug: inference
color: "607d8b"
description: "AI/ML inference workloads"
# ── Custom field definitions ──────────────────────────────────────────
# object_types use app_label.model format
custom_fields:
- name: hostnames
label: Hostnames
type: longtext
object_types:
- ipam.ipaddress
- virtualization.virtualmachine
description: "All DNS names that resolve to this service (comma-separated)"
ui_visible: always
ui_editable: yes
- name: namespace
label: Namespace
type: text
object_types:
- ipam.ipaddress
- virtualization.virtualmachine
description: "Kubernetes namespace (blank for VM-based services)"
ui_visible: always
ui_editable: yes
- name: managed_by
label: Managed By
type: select
object_types:
- ipam.ipaddress
- virtualization.virtualmachine
description: "How this service is managed"
choices:
- ArgoCD
- Ansible
- Manual
ui_visible: always
ui_editable: yes
- name: thematic_name
label: Thematic Name
type: text
object_types:
- ipam.ipaddress
- virtualization.virtualmachine
description: "Disney/Magic Kingdom thematic hostname for this service"
ui_visible: always
ui_editable: yes
tasks:
# ── Tags ─────────────────────────────────────────────────────────────
- name: Fetch existing tags
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/tags/?limit=200"
method: GET
headers: "{{ headers }}"
return_content: true
status_code: 200
register: existing_tags_response
- name: Set existing tag slugs fact
ansible.builtin.set_fact:
existing_tag_slugs: "{{ existing_tags_response.json.results | map(attribute='slug') | list }}"
- name: Create tags
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/tags/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: "{{ item.name }}"
slug: "{{ item.slug }}"
color: "{{ item.color }}"
description: "{{ item.description }}"
status_code: 201
loop: "{{ tags }}"
when: item.slug not in existing_tag_slugs
register: tag_creation
changed_when: tag_creation.status == 201
# ── Custom Fields ─────────────────────────────────────────────────────
- name: Fetch existing custom fields
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/?limit=200"
method: GET
headers: "{{ headers }}"
return_content: true
status_code: 200
register: existing_cf_response
- name: Set existing custom field names fact
ansible.builtin.set_fact:
existing_cf_names: "{{ existing_cf_response.json.results | map(attribute='name') | list }}"
- name: Create custom field — hostnames
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: hostnames
label: Hostnames
type: longtext
object_types: "{{ custom_fields[0].object_types }}"
description: "{{ custom_fields[0].description }}"
ui_visible: always
ui_editable: yes
status_code: 201
when: "'hostnames' not in existing_cf_names"
register: cf_hostnames
changed_when: cf_hostnames.status == 201
- name: Create custom field — namespace
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: namespace
label: Namespace
type: text
object_types: "{{ custom_fields[1].object_types }}"
description: "{{ custom_fields[1].description }}"
ui_visible: always
ui_editable: yes
status_code: 201
when: "'namespace' not in existing_cf_names"
register: cf_namespace
changed_when: cf_namespace.status == 201
- name: Create choice set for managed_by field
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-field-choice-sets/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: managed-by-choices
extra_choices:
- - ArgoCD
- ArgoCD
- - Ansible
- Ansible
- - Manual
- Manual
status_code: [201, 400]
register: choice_set
changed_when: choice_set.status == 201
- name: Fetch choice set ID
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-field-choice-sets/?name=managed-by-choices"
method: GET
headers: "{{ headers }}"
return_content: true
status_code: 200
register: choice_set_response
- name: Create custom field — managed_by
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: managed_by
label: Managed By
type: select
object_types: "{{ custom_fields[2].object_types }}"
description: "{{ custom_fields[2].description }}"
choice_set: "{{ choice_set_response.json.results[0].id }}"
ui_visible: always
ui_editable: yes
status_code: 201
when: "'managed_by' not in existing_cf_names"
register: cf_managed_by
changed_when: cf_managed_by.status == 201
- name: Create custom field — thematic_name
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: thematic_name
label: Thematic Name
type: text
object_types: "{{ custom_fields[3].object_types }}"
description: "{{ custom_fields[3].description }}"
ui_visible: always
ui_editable: yes
status_code: 201
when: "'thematic_name' not in existing_cf_names"
register: cf_thematic_name
changed_when: cf_thematic_name.status == 201
# ── Provisioning Pipeline Fields ──────────────────────────────────────
# These fields drive the NetBox → n8n → Terraform pipeline.
# proxmox_datastore already exists — PATCH it to add utilidor choice.
# data_disk_enabled and data_disk_size_gb are new POSTs.
- name: Fetch proxmox_datastore field ID
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/?name=proxmox_datastore"
method: GET
headers: "{{ headers }}"
return_content: true
status_code: 200
register: proxmox_datastore_cf_response
- name: Fetch proxmox-datastore-choices choice set ID
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-field-choice-sets/?name=proxmox-datastore-choices"
method: GET
headers: "{{ headers }}"
return_content: true
status_code: 200
register: datastore_choice_set_response
- name: Create proxmox-datastore-choices choice set if missing
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-field-choice-sets/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: proxmox-datastore-choices
extra_choices:
- - liberty-tree
- liberty-tree
- - utilidor
- utilidor
status_code: [201, 400]
register: datastore_choice_set_create
changed_when: datastore_choice_set_create.status == 201
when: datastore_choice_set_response.json.count == 0
- name: Re-fetch proxmox-datastore-choices choice set ID after possible creation
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-field-choice-sets/?name=proxmox-datastore-choices"
method: GET
headers: "{{ headers }}"
return_content: true
status_code: 200
register: datastore_choice_set_response
- name: Patch proxmox_datastore field to use choice set with utilidor
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/{{ proxmox_datastore_cf_response.json.results[0].id }}/"
method: PATCH
headers: "{{ headers }}"
body_format: json
body:
choice_set: "{{ datastore_choice_set_response.json.results[0].id }}"
status_code: 200
when: proxmox_datastore_cf_response.json.count > 0
register: cf_datastore_patch
changed_when: cf_datastore_patch.status == 200
- name: Create custom field — data_disk_enabled
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: data_disk_enabled
label: Data Disk Enabled
type: boolean
object_types:
- virtualization.virtualmachine
description: "When true, Terraform provisions a second disk for application data storage."
default: false
ui_visible: always
ui_editable: yes
status_code: 201
when: "'data_disk_enabled' not in existing_cf_names"
register: cf_data_disk_enabled
changed_when: cf_data_disk_enabled.status == 201
- name: Create custom field — data_disk_size_gb
ansible.builtin.uri:
url: "{{ netbox_api }}/extras/custom-fields/"
method: POST
headers: "{{ headers }}"
body_format: json
body:
name: data_disk_size_gb
label: Data Disk Size (GB)
type: integer
object_types:
- virtualization.virtualmachine
description: "Size in GB for the optional second data disk. Only used when data_disk_enabled is true."
ui_visible: always
ui_editable: yes
status_code: 201
when: "'data_disk_size_gb' not in existing_cf_names"
register: cf_data_disk_size
changed_when: cf_data_disk_size.status == 201
# ── Summary ───────────────────────────────────────────────────────────
- name: Summary
ansible.builtin.debug:
msg:
- "Tags created: {{ tag_creation.results | selectattr('status', 'equalto', 201) | list | length }}"
- "Tags skipped (already exist): {{ tag_creation.results | selectattr('skipped', 'defined') | list | length }}"
- "Custom fields configured: hostnames, namespace, managed_by, thematic_name, proxmox_datastore (patched), data_disk_enabled, data_disk_size_gb"

View File

@@ -0,0 +1,18 @@
---
# ------------------------------------------------------------------------------
# FILE: playbooks/deploy_authentik.yml
# DESCRIPTION: Deploys Authentik identity provider on guest-relations.
# Installs Docker and configures Authentik with PostgreSQL and Redis.
#
# USAGE:
# ansible-playbook -i inventory.yml playbooks/deploy_authentik.yml
# ------------------------------------------------------------------------------
- name: Deploy Authentik identity provider
hosts: authentik_server
become: true
roles:
- common
- docker-host
- authentik

View File

@@ -0,0 +1,23 @@
---
# ------------------------------------------------------------------------------
# FILE: ansible/playbooks/day1_deploy_gitea.yml
# DESCRIPTION: Deploys Gitea + PostgreSQL on mad-tea-party (10.1.71.129)
# Role chain: common → docker-host → gitea
#
# USAGE:
# ansible-playbook -i ansible/inventory.yml ansible/playbooks/day1_deploy_gitea.yml
#
# SECRETS REQUIRED IN VAULT (group_vars/all/vault):
# vault_gitea_db_password
# ------------------------------------------------------------------------------
- name: Deploy Gitea on mad-tea-party
hosts: gitea_servers
become: true
vars:
gitea_db_password: "{{ vault_gitea_db_password }}"
roles:
- role: docker-host
- role: gitea

View File

@@ -0,0 +1,79 @@
---
# =============================================================================
# day1_deploy_hermes.yml
# Deploy Hermes Agent (Nous Research) on carousel-of-progress (10.1.71.131)
#
# FIRST-RUN WORKFLOW:
# 1. Run this playbook:
# ansible-playbook playbooks/day1_deploy_hermes.yml
#
# 2. SSH to the host and run the setup wizard as the hermes user:
# ssh wed@carousel-of-progress.local.mk-labs.cloud
# sudo -u hermes hermes setup
#
# 3. Once configured, start and verify the service:
# sudo systemctl start hermes
# sudo systemctl status hermes
# sudo journalctl -u hermes -f
#
# VARIABLES:
# hermes_skip_browser: true — set to skip Playwright/Chromium install
# (saves ~300MB if browser automation not needed)
# =============================================================================
- name: Deploy Hermes Agent on carousel-of-progress
hosts: carousel-of-progress
gather_facts: true
pre_tasks:
- name: Verify target is carousel-of-progress
ansible.builtin.assert:
that:
- inventory_hostname == "carousel-of-progress"
fail_msg: >
This playbook is scoped to carousel-of-progress only.
Got: {{ inventory_hostname }}
- name: Confirm OS is Ubuntu
ansible.builtin.assert:
that:
- ansible_distribution == "Ubuntu"
fail_msg: >
This playbook requires Ubuntu. Found: {{ ansible_distribution }}.
(If running Fedora, swap apt tasks for dnf and adjust Playwright deps.)
roles:
- role: hermes
vars:
hermes_skip_browser: false # set true to skip Chromium install
post_tasks:
- name: Verify hermes binary is accessible system-wide
ansible.builtin.command: hermes --version
register: hermes_version_check
changed_when: false
failed_when: hermes_version_check.rc != 0
- name: Print hermes version
ansible.builtin.debug:
msg: "{{ hermes_version_check.stdout }}"
- name: Print post-install instructions
ansible.builtin.debug:
msg:
- "============================================================"
- "Hermes installed on carousel-of-progress (10.1.71.131)"
- "============================================================"
- "Next steps:"
- " 1. SSH to the host:"
- " ssh wed@carousel-of-progress.local.mk-labs.cloud"
- " 2. Run the setup wizard as the hermes user:"
- " sudo -u hermes hermes setup"
- " 3. After config, start the service:"
- " sudo systemctl start hermes"
- " 4. Verify:"
- " sudo systemctl status hermes"
- " sudo journalctl -u hermes -f"
- "============================================================"
- "Service is ENABLED but NOT STARTED — config required first."
- "============================================================"

View File

@@ -0,0 +1,18 @@
---
# ============================================================================
# day1_deploy_honcho.yml
# ----------------------------------------------------------------------------
# Deploys Honcho + pgvector PostgreSQL on the `lincoln` host. Assumes day0
# host provisioning (linux-baseline + expand_root_lv) is already complete.
#
# Run via:
# ansible-playbook -i inventory.yml playbooks/day0_provision.yml -e target=lincoln
# ansible-playbook -i inventory.yml playbooks/day1_deploy_honcho.yml
# ============================================================================
- name: Deploy Honcho on lincoln
hosts: honcho_server
become: true
gather_facts: true
roles:
- honcho

View File

@@ -0,0 +1,91 @@
---
# ------------------------------------------------------------------------------
# FILE: playbooks/day1_deploy_nextcloud.yml
# DESCRIPTION: Deploys Nextcloud on the-grid
# Runs: docker-host → nextcloud
#
# USAGE:
# ansible-playbook -i inventory.yml playbooks/day1_deploy_nextcloud.yml
# ansible-playbook -i inventory.yml playbooks/day1_deploy_nextcloud.yml --limit the-grid
#
# SECRETS REQUIRED IN VAULT (group_vars/all/vault):
# vault_nextcloud_db_root_password
# vault_nextcloud_db_password
# vault_nextcloud_admin_user
# vault_nextcloud_admin_password
# ------------------------------------------------------------------------------
- name: Deploy Nextcloud on the-grid
hosts: the-grid
become: true
vars:
# NFS prerequisite
nfs_packages:
- nfs-common
# Compose stack location
nextcloud_base_dir: /opt/docker/nextcloud
# Secrets from vault
nextcloud_db_root_password: "{{ vault_nextcloud_db_root_password }}"
nextcloud_db_password: "{{ vault_nextcloud_db_password }}"
nextcloud_admin_user: "{{ vault_nextcloud_admin_user }}"
nextcloud_admin_password: "{{ vault_nextcloud_admin_password }}"
pre_tasks:
- name: Install NFS client
ansible.builtin.apt:
name: "{{ nfs_packages }}"
state: present
update_cache: true
roles:
- role: docker-host
tasks:
- name: Create Nextcloud directory
ansible.builtin.file:
path: "{{ nextcloud_base_dir }}"
state: directory
owner: "{{ ansible_user }}"
group: docker
mode: "0775"
- name: Deploy Compose file
ansible.builtin.copy:
src: "{{ playbook_dir }}/../../boilerplates/nextcloud/compose.yml"
dest: "{{ nextcloud_base_dir }}/compose.yml"
owner: "{{ ansible_user }}"
group: docker
mode: "0644"
- name: Deploy .env from vault
ansible.builtin.copy:
content: |
# Managed by Ansible — do not edit manually
MYSQL_ROOT_PASSWORD={{ nextcloud_db_root_password }}
MYSQL_PASSWORD={{ nextcloud_db_password }}
NEXTCLOUD_ADMIN_USER={{ nextcloud_admin_user }}
NEXTCLOUD_ADMIN_PASSWORD={{ nextcloud_admin_password }}
dest: "{{ nextcloud_base_dir }}/.env"
owner: "{{ ansible_user }}"
group: docker
mode: "0600"
- name: Start Nextcloud stack
community.docker.docker_compose_v2:
project_src: "{{ nextcloud_base_dir }}"
state: present
- name: Wait for Nextcloud to become ready
ansible.builtin.uri:
url: "http://the-grid.local.mk-labs.cloud/status.php"
status_code: 200
return_content: true
register: nextcloud_status
until: >
nextcloud_status.status == 200 and
(nextcloud_status.content | from_json).installed == true
retries: 20
delay: 15

View File

@@ -0,0 +1,35 @@
---
# ------------------------------------------------------------------------------
# FILE: ansible/playbooks/day1_deploy_ollama.yml
# DESCRIPTION: Deploys Ollama with ROCm GPU acceleration on astro-orbiter.
# Assumes day0_baseline.yml has already run (common role complete).
# PCIe passthrough for the RX 5700 must be configured in Proxmox
# and the GPU must be visible to the VM before running this playbook.
#
# Pre-flight check:
# ssh wed@astro-orbiter 'lspci | grep -i amd'
# Should show the RX 5700 before proceeding.
#
# Usage (from ansible/ directory):
# ansible-playbook playbooks/day1_deploy_ollama.yml
# ------------------------------------------------------------------------------
- name: Deploy Ollama with ROCm on astro-orbiter
hosts: astro-orbiter
become: true
pre_tasks:
- name: Verify AMD GPU is visible to the VM
command: lspci
register: lspci_output
changed_when: false
- name: Fail if no AMD GPU detected
fail:
msg: >
No AMD GPU detected via lspci. Verify PCIe passthrough is configured
in Proxmox and the RX 5700 is visible to the VM before proceeding.
when: "'AMD' not in lspci_output.stdout and 'Radeon' not in lspci_output.stdout"
roles:
- ollama

View File

@@ -0,0 +1,17 @@
---
# ============================================================================
# day1_deploy_semaphore.yml
# ----------------------------------------------------------------------------
# Deploys SemaphoreUI + PostgreSQL on the imagineering host (figment).
# Run AFTER day0_linux_baseline.yml has been applied to the target.
#
# Usage:
# ansible-playbook -i inventory.yml playbooks/day1_deploy_semaphore.yml
# ============================================================================
- name: Deploy SemaphoreUI on imagineering
hosts: semaphore_server
become: true
gather_facts: true
roles:
- semaphore

View File

@@ -0,0 +1,18 @@
---
# ------------------------------------------------------------------------------
# FILE: playbooks/deploy_traefik.yml
# DESCRIPTION: Deploys Traefik reverse proxy on lightning-lane.
# Installs Docker and configures Traefik with Cloudflare DNS-01
# certificate resolution for *.local.mk-labs.cloud
#
# USAGE:
# ansible-playbook -i inventory.yml playbooks/deploy_traefik.yml
# ------------------------------------------------------------------------------
- name: Deploy Traefik reverse proxy
hosts: load_balancers
become: true
roles:
- docker-host
- traefik

View File

@@ -0,0 +1,7 @@
---
- name: Apply common role
hosts: "{{ target | default('all') }}"
become: true
roles:
- common
- docker-host

View File

@@ -0,0 +1,80 @@
---
# ------------------------------------------------------------------------------
# FILE: ansible/playbooks/day2_install_node_exporter.yml
# DESCRIPTION: Install and configure Prometheus Node Exporter on Linux hosts.
#
# Targets two groups with different firewall managers:
# - proxmox_nodes : Proxmox hypervisors (Debian, firewalld not present,
# uses iptables/no firewall — just open the port via UFW
# if present, otherwise skip)
# - monitored_vms : Ubuntu VMs managed by UFW
#
# After installing on new hosts, re-runs day1_deploy_monitoring.yml to refresh
# the Prometheus scrape config is NOT needed — targets are already statically
# defined in prometheus.yaml.j2 for the Proxmox nodes. For new VMs, add the
# IP to the prometheus.yaml.j2 proxmox-vms job and re-run day1_deploy_monitoring.yml.
#
# Usage:
# ansible-playbook -i inventory.yml playbooks/day2_install_node_exporter.yml
# ansible-playbook -i inventory.yml playbooks/day2_install_node_exporter.yml --limit proxmox_nodes
# ansible-playbook -i inventory.yml playbooks/day2_install_node_exporter.yml --limit monitored_vms
# ------------------------------------------------------------------------------
# ── Play 1: Proxmox hypervisors ───────────────────────────────────────────────
# Proxmox runs Debian. The prometheus.prometheus.node_exporter role installs
# a binary + systemd service without touching apt, which is what we want on
# hypervisors (keep the package footprint clean).
# Note: Proxmox does not run UFW. Port 9100 is open by default on VLAN 71.
- name: Install Node Exporter on Proxmox hypervisors
hosts: proxmox_nodes
become: true
vars:
node_exporter_version: "1.9.1"
node_exporter_web_listen_address: "0.0.0.0:9100"
node_exporter_enabled_collectors:
- systemd
- processes
- filesystem
- meminfo
- cpu
- diskstats
- netdev
- loadavg
- uname
roles:
- role: prometheus.prometheus.node_exporter
# ── Play 2: Ubuntu VMs ────────────────────────────────────────────────────────
# Standard Ubuntu hosts with UFW. Same role, adds UFW allow rule for 9100.
- name: Install Node Exporter on monitored VMs
hosts: monitored_vms
become: true
vars:
node_exporter_version: "1.9.1"
node_exporter_web_listen_address: "0.0.0.0:9100"
node_exporter_enabled_collectors:
- systemd
- processes
- filesystem
- meminfo
- cpu
- diskstats
- netdev
- loadavg
- uname
pre_tasks:
- name: Open Node Exporter port in UFW
community.general.ufw:
rule: allow
port: "9100"
proto: tcp
comment: "Prometheus Node Exporter"
roles:
- role: prometheus.prometheus.node_exporter

View File

@@ -1,30 +0,0 @@
---
# - name: Step 1 - Install Prerequisites
# hosts: fastpass
# become: true
# gather_facts: true
# roles:
# - role: kubernetes-prerequisites
# - name: Step 2 - Deploy First Control Plane Node
# hosts: fastpass_control_plane[0]
# become: true
# gather_facts: false
# roles:
# - role: fastpass-first-control-plane
- name: Step 3 - Deploy Additional Control Plane Nodes
hosts: fastpass_control_plane[1:]
become: true
gather_facts: true # false
roles:
- role: fastpass-additional-control-plane
vars:
cluster_name: "fastpass"
# - name: Step 4 - Deploy Worker Nodes
# hosts: fastpass_workers
# become: true
# gather_facts: false
# roles:
# - role: fastpass-workers

View File

@@ -1,157 +0,0 @@
---
# FastPass Kubernetes Cluster Deployment
# This playbook deploys a complete Kubernetes cluster on Fedora
- name: 1. Preflight checks and system preparation
hosts: fastpass
become: true
gather_facts: true
roles:
- role: kubernetes
- name: 2. Initialize first control plane node
hosts: fastpass_control_plane[0]
become: true
gather_facts: false
roles:
- role: fastpass-control-plane
- name: 3. Install Calico CNI
hosts: fastpass_control_plane[0]
become: false
gather_facts: false
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
tasks:
- name: Check if Calico is already installed
ansible.builtin.shell: kubectl get pods -n kube-system -l k8s-app=calico-node --no-headers | wc -l
delegate_to: localhost
register: calico_check
ignore_errors: true
- name: Display Calico check result
ansible.builtin.debug:
msg: "Calico pods found: {{ calico_check.stdout | trim }}"
delegate_to: localhost
- name: Install Calico CNI
ansible.builtin.command: kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/{{ calico_version | default('v3.28.0') }}/manifests/calico.yaml
delegate_to: localhost
when: calico_check.stdout | trim == "0"
register: calico_install_result
ignore_errors: true
- name: Display Calico installation result
ansible.builtin.debug:
msg: "Calico install stdout: {{ calico_install_result.stdout }}"
delegate_to: localhost
when: calico_check.stdout | trim == "0"
- name: Wait for Calico node pods to be ready
ansible.builtin.command: kubectl wait --for=condition=ready pod -l k8s-app=calico-node -n kube-system --timeout=300s
delegate_to: localhost
when: calico_check.stdout | trim == "0"
- name: 4. Wait for first control plane to be fully ready
hosts: fastpass_control_plane[0]
become: false
gather_facts: false
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
tasks:
- name: Wait for API server to be ready on first node
ansible.builtin.wait_for:
host: "{{ inventory_hostname }}"
port: "{{ control_plane_port | default('6443') }}"
timeout: 300
delegate_to: localhost
- name: Wait for first control plane node to be ready
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} wait --for=condition=ready node {{ inventory_hostname }} --timeout=300s
delegate_to: localhost
- name: Verify control plane status
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} get nodes -o wide
delegate_to: localhost
register: node_status
- name: Display cluster status
ansible.builtin.debug:
msg: "{{ node_status.stdout_lines }}"
- name: 5. Join additional control plane nodes
hosts: fastpass_control_plane[1:]
become: true
gather_facts: false
roles:
- role: fastpass-control-plane-join
- name: 6. Wait for all control plane nodes to be ready
hosts: fastpass_control_plane[0]
become: false
gather_facts: false
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
tasks:
- name: Wait for all control plane nodes to be ready
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} wait --for=condition=ready node --selector=node-role.kubernetes.io/control-plane --timeout=600s
delegate_to: localhost
- name: Display all control plane nodes
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} get nodes --selector=node-role.kubernetes.io/control-plane -o wide
delegate_to: localhost
register: control_plane_status
- name: Show control plane status
ansible.builtin.debug:
msg: "{{ control_plane_status.stdout_lines }}"
- name: 7. Join worker nodes
hosts: fastpass_workers
become: true
gather_facts: false
roles:
- role: fastpass-workers
- name: 8. Final cluster validation
hosts: fastpass_control_plane[0]
become: false
gather_facts: false
environment:
KUBECONFIG: "{{ kubeconfig_path }}"
tasks:
- name: Wait for all nodes to be ready
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} wait --for=condition=ready node --all --timeout=300s
delegate_to: localhost
- name: Verify all pods are running
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} get pods --all-namespaces
delegate_to: localhost
register: pod_status
- name: Display final cluster status
ansible.builtin.debug:
msg: |
========================================
FastPass Kubernetes Cluster Status
========================================
{{ pod_status.stdout }}
========================================
- name: Show final node status
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} get nodes -o wide
delegate_to: localhost
register: final_node_status
- name: Display final node status
ansible.builtin.debug:
msg: "{{ final_node_status.stdout_lines }}"
- name: Test DNS-based control plane endpoint
ansible.builtin.command: kubectl --kubeconfig={{ kubeconfig_path }} cluster-info
delegate_to: localhost
register: cluster_info
- name: Display cluster info
ansible.builtin.debug:
msg: "{{ cluster_info.stdout_lines }}"

View File

@@ -1,177 +0,0 @@
---
# ansible/playbooks/deploy-haproxy-cloudflare.yml
- name: Deploy HAProxy with Cloudflare DNS certificates
hosts: lightning-lane
become: true
vars:
# Certbot Cloudflare configuration
haproxy_certbot_enable: true
haproxy_certbot_challenge_method: "dns-cloudflare"
haproxy_certbot_email: "ryan@mk-labs.cloud"
haproxy_certbot_staging: false # Set to true for testing
haproxy_certbot_cloudflare_api_token: "{{ vault_cloudflare_api_token }}"
# Request wildcard certificate
haproxy_certbot_domains:
- domain: "*.local.mk-labs.cloud"
include_base: true # Also include local.mk-labs.cloud
# HAProxy Stats
haproxy_stats_password: "{{ vault_haproxy_stats_password }}"
# Frontend configuration
haproxy_frontends:
- name: https_front
bind: "*:443 ssl crt {{ haproxy_ssl_cert_dir }}/live crt {{ haproxy_ssl_default_crt }} alpn h2,http/1.1"
mode: http
options:
- "http-server-close"
- "forwardfor"
acls:
- "netbox hdr(host) -i fire-station.local.mk-labs.cloud"
- "wordpress hdr(host) -i be-our-guest.local.mk-labs.cloud"
- "plane hdr(host) -i people-mover.local.mk-labs.cloud"
- "minecraft hdr(host) -i arcade.local.mk-labs.cloud"
use_backends:
- "netbox_back if netbox"
- "wordpress_back if wordpress"
- "plane_back if plane"
- "minecraft_back if minecraft"
# Backend configuration
haproxy_backends:
- name: netbox_back
mode: http
balance: roundrobin
options:
- "httpchk GET /api/"
servers:
- name: fire-station
address: 10.1.71.102:8000
check: true
- name: wordpress_back
mode: http
balance: roundrobin
servers:
- name: be-our-guest
address: 10.1.71.101:80
check: true
- name: plane_back
mode: http
balance: roundrobin
servers:
- name: people-mover
address: 10.1.71.112:8080
check: true
- name: minecraft_back
mode: http
balance: roundrobin
servers:
- name: arcade
address: 10.1.71.111:25565
check: true
# TCP services
haproxy_tcp_services:
- name: postgres
frontend_port: 5432
backend_port: 5432
balance: leastconn
timeout_client: 1h
timeout_server: 1h
servers:
- name: netbox-db
address: 10.1.71.102
check: true
- name: ssh_jump
frontend_port: 2222
backend_port: 22
balance: source
servers:
- name: main-street-usa
address: 10.1.71.11
check: true
- name: tomorrowland
address: 10.1.71.12
check: true
- name: fantasyland
address: 10.1.71.13
check: true
- name: minecraft_java
frontend_port: 25565
backend_port: 25565
balance: roundrobin
servers:
- name: arcade
address: 10.1.71.111
check: true
# Firewall configuration
haproxy_firewall_allowed_ports:
- 80/tcp
- 443/tcp
- 2222/tcp
- 5432/tcp
- 8404/tcp
- 25565/tcp
pre_tasks:
- name: Ensure required groups exist
ansible.builtin.group:
name: "{{ item }}"
state: present
loop:
- haproxy
roles:
- haproxy
post_tasks:
- name: Wait for HAProxy to be ready
ansible.builtin.wait_for:
host: "{{ ansible_default_ipv4.address }}"
port: 443
timeout: 60
- name: Display deployment summary
ansible.builtin.debug:
msg: |
╔════════════════════════════════════════════════════════════╗
║ HAProxy Deployment Complete - lightning-lane ║
╚════════════════════════════════════════════════════════════╝
Stats Interface: http://{{ ansible_default_ipv4.address }}:8404/stats
Username: {{ haproxy_stats_username }}
HTTPS Services:
{% for frontend in haproxy_frontends %}
{% if frontend.acls is defined %}
{% for acl in frontend.acls %}
- {{ acl.split()[1] | regex_replace('hdr\\(host\\)', '') | regex_replace('-i', '') | trim }}
{% endfor %}
{% endif %}
{% endfor %}
TCP Services:
{% for tcp in haproxy_tcp_services %}
- {{ tcp.name }}: {{ ansible_default_ipv4.address }}:{{ tcp.frontend_port }}
{% endfor %}
Certificates:
{% for domain in haproxy_certbot_domains %}
- {{ domain.domain }}{% if domain.include_base | default(false) %} + base domain{% endif %}
{% endfor %}
Next steps:
1. Verify certificate: certbot certificates
2. Test renewal: certbot renew --dry-run
3. Check HAProxy stats interface
4. Test HTTPS endpoints

View File

@@ -1,17 +0,0 @@
# ------------------------------------------------------------------------------
# FILE: deploy_k8s.yml
# ------------------------------------------------------------------------------
- name: 1. Prepare all nodes for Kubernetes
hosts: fastpass
roles:
- role: kubernetes
- name: 2. Initialize and configure the control plane
hosts: fastpass_control_plane
roles:
- role: fastpass-control-plane
- name: 3. Join worker nodes to the cluster
hosts: fastpass_workers
roles:
- role: fastpass-workers

View File

@@ -1,22 +0,0 @@
---
# - name: Install additional packages
# hosts: papermc_server
# become: true
# tasks:
# - name: Install python-apt packages
# ansible.builtin.package:
# name: "libapt-pkg"
# state: latest
# update_cache: true
- name: 1. Deploy Paper Minecraft server
hosts: papermc_server
roles:
vars:
user:
home_dir: /opt/minecraft
name: wed
roles:
- role: apigban.papermc_role
become: yes

View File

@@ -0,0 +1,29 @@
---
# ------------------------------------------------------------------------------
# FILE: playbooks/deploy_step_ca.yml
# DESCRIPTION: Deploys Smallstep step-ca SSH Certificate Authority on turnstile.
# Installs Docker and configures step-ca with SSH certificate support.
#
# PREREQUISITES:
# - VM provisioned via Terraform
# - DNS record for turnstile.local.mk-labs.cloud on monorail
# - Authentik OIDC application created (for post-init provisioner setup)
#
# USAGE:
# ansible-playbook -i inventory.yml playbooks/deploy_step_ca.yml
#
# POST-DEPLOY:
# 1. Note the CA fingerprint from the init output
# 2. Add the OIDC provisioner (see docs/guides/step-ca-setup.md)
# 3. Deploy Traefik route via update_traefik_routes.yml
# 4. Bootstrap client workstations with: step ca bootstrap
# ------------------------------------------------------------------------------
- name: Deploy step-ca SSH Certificate Authority
hosts: step_ca_server
become: true
roles:
- common
- docker-host
- step-ca

View File

@@ -0,0 +1,7 @@
---
- name: Enroll host as step-ca SSH client
hosts: "{{ target | default('all') }}"
become: true
tasks:
- name: Include step-ca client enrollment
ansible.builtin.include_tasks: roles/common/tasks/step_ca_client.yml

View File

@@ -1,41 +0,0 @@
---
# Example playbook showing how to use the modular kubeconfig management
# across multiple clusters in your homelab
- name: Setup kubeconfig for FastPass cluster
hosts: fastpass_control_plane[0]
gather_facts: true
roles:
- role: kubeconfig-manager
vars:
cluster_name: "fastpass"
- name: Setup kubeconfig for Hub cluster
hosts: hub_cluster
gather_facts: true
roles:
- role: kubeconfig-manager
vars:
cluster_name: "hub"
- name: Setup kubeconfig for Internal cluster
hosts: internal_cluster[0]
gather_facts: true
roles:
- role: kubeconfig-manager
vars:
cluster_name: "internal"
# Alternative approach using the generic cluster-kubeconfig role
- name: Setup kubeconfig for any cluster
hosts: "{{ target_cluster_hosts }}"
gather_facts: true
roles:
- role: cluster-kubeconfig
vars:
cluster_name: "{{ target_cluster_name }}"
# Usage examples:
# ansible-playbook -i inventory.yml examples/multi-cluster-kubeconfig.yml
# ansible-playbook -i inventory.yml examples/multi-cluster-kubeconfig.yml --limit fastpass_control_plane[0]
# ansible-playbook -i inventory.yml examples/multi-cluster-kubeconfig.yml -e target_cluster_hosts=hub_cluster -e target_cluster_name=hub

View File

@@ -1,60 +0,0 @@
---
# Example: Setup network infrastructure for FastPass cluster
# This demonstrates the modular approach for DNS and load balancer setup
- name: Setup FastPass Cluster Network Infrastructure
hosts: fastpass_control_plane[0]
gather_facts: true
vars:
cluster_name: "fastpass"
cluster_endpoint: "{{ control_plane_endpoint }}"
cluster_vip: "{{ ansible_default_ipv4.address }}"
control_plane_nodes: "{{ groups['fastpass_control_plane'] }}"
tasks:
- name: Display cluster configuration
ansible.builtin.debug:
msg: |
Setting up network for FastPass cluster:
- Cluster Name: {{ cluster_name }}
- Endpoint: {{ cluster_endpoint }}
- VIP: {{ cluster_vip }}
- Control Plane Nodes: {{ control_plane_nodes | join(', ') }}
- name: Setup cluster network infrastructure
ansible.builtin.include_role:
name: cluster-network-setup
vars:
cluster_name: "{{ cluster_name }}"
cluster_endpoint: "{{ cluster_endpoint }}"
cluster_vip: "{{ cluster_vip }}"
control_plane_nodes: "{{ control_plane_nodes }}"
# Alternative approach using task file directly
- name: Alternative - Use Technitium DNS task directly
hosts: localhost
gather_facts: false
vars:
cluster_endpoint: "{{ hostvars[groups['fastpass_control_plane'][0]]['control_plane_endpoint'] }}"
cluster_vip: "{{ hostvars[groups['fastpass_control_plane'][0]]['ansible_default_ipv4']['address'] }}"
tasks:
- name: Create DNS entry using task file
ansible.builtin.include_tasks: ../tasks/add_technitium_dns_entry.yml
vars:
dns_record_name: "{{ cluster_endpoint.split('.')[0] }}"
dns_zone: "{{ base_domain }}"
dns_ip_address: "{{ cluster_vip }}"
dns_record_type: "A"
dns_ttl: 360
dns_create_ptr: true
dns_debug: true
when: use_task_approach | default(false)
# Usage examples:
#
# Use modular approach (recommended):
# ansible-playbook -i inventory.yml playbooks/examples/setup-fastpass-network.yml
#
# Use task file approach:
# ansible-playbook -i inventory.yml playbooks/examples/setup-fastpass-network.yml -e use_task_approach=true

View File

@@ -1,10 +0,0 @@
---
- name: Playbook to configure IPA server
hosts: ipaserver
become: true
vars_files:
- idm-vault.yml
roles:
- role: freeipa.ansible_freeipa.ipaserver
state: present

View File

@@ -0,0 +1,15 @@
---
# ------------------------------------------------------------------------------
# FILE: playbooks/install_talosctl.yml
# DESCRIPTION: Install talosctl on city-hall for managing the fastpass cluster.
# USAGE:
# cd ansible
# ansible-playbook -i inventory.yml playbooks/install_talosctl.yml
# ------------------------------------------------------------------------------
- name: Install talosctl
hosts: talos_control
become: true
roles:
- talosctl

View File

@@ -1,115 +0,0 @@
---
- name: Reset FastPass Kubernetes Cluster
hosts: fastpass
become: true
gather_facts: false
tasks:
- name: Stop and disable kubelet service
ansible.builtin.systemd:
name: kubelet
state: stopped
enabled: false
ignore_errors: true
- name: Stop and disable containerd service
ansible.builtin.systemd:
name: containerd
state: stopped
enabled: false
ignore_errors: true
- name: Remove Kubernetes packages
ansible.builtin.dnf:
name:
- kubelet
- kubeadm
- kubectl
- kubernetes-cni
- containernetworking-plugins
state: absent
ignore_errors: true
- name: Remove containerd
ansible.builtin.dnf:
name: containerd
state: absent
ignore_errors: true
- name: Remove Docker repository
ansible.builtin.file:
path: /etc/yum.repos.d/docker-ce.repo
state: absent
ignore_errors: true
- name: Remove Kubernetes repository
ansible.builtin.file:
path: /etc/yum.repos.d/kubernetes.repo
state: absent
ignore_errors: true
- name: Remove Kubernetes directories
ansible.builtin.file:
path: "{{ item }}"
state: absent
loop:
- /etc/kubernetes
- /var/lib/kubelet
- /var/lib/etcd
- /etc/cni/net.d
- /opt/cni/bin
- /var/lib/containerd
- /etc/containerd
ignore_errors: true
- name: Remove CNI plugins
ansible.builtin.file:
path: /opt/cni
state: absent
ignore_errors: true
- name: Remove iptables rules
ansible.builtin.shell: |
iptables -F
iptables -t nat -F
iptables -t mangle -F
iptables -X
ignore_errors: true
- name: Remove firewall Kubernetes service
ansible.builtin.file:
path: /etc/firewalld/services/kubernetes.xml
state: absent
ignore_errors: true
- name: Remove firewall rules for Kubernetes
ansible.builtin.shell: |
firewall-cmd --permanent --remove-service=kubernetes || true
firewall-cmd --reload || true
ignore_errors: true
- name: Reset network interfaces
ansible.builtin.shell: |
ip link delete cni0 || true
ip link delete flannel.1 || true
ip link delete cali* || true
ignore_errors: true
- name: Clean up systemd drop-in files
ansible.builtin.file:
path: /usr/lib/systemd/system/kubelet.service.d
state: absent
ignore_errors: true
- name: Reset hostname to original
ansible.builtin.hostname:
name: "{{ inventory_hostname }}"
- name: Clean package cache
ansible.builtin.dnf:
clean: all
ignore_errors: true
- name: Reboot system
ansible.builtin.reboot:
reboot_timeout: 300

View File

@@ -1,12 +0,0 @@
---
# role: cluster-kubeconfig
# description: Generic role for any Kubernetes cluster kubeconfig management
# author: mk-labs
# version: 1.0.0
- name: Setup kubeconfig for {{ cluster_name }} cluster
ansible.builtin.include_role:
name: kubeconfig-manager
vars:
cluster_name: "{{ cluster_name }}"
kubeconfig_source_path: "{{ kubeconfig_source_path | default('/etc/kubernetes/admin.conf') }}"

View File

@@ -1,48 +0,0 @@
---
# role: cluster-network-setup
# description: Combined DNS and Load Balancer setup for Kubernetes clusters
# author: mk-labs
# version: 1.0.0
- name: Setup DNS entry for cluster
ansible.builtin.include_role:
name: dns-manager
vars:
cluster_endpoint: "{{ cluster_endpoint }}"
cluster_vip: "{{ cluster_vip }}"
- name: Setup Traefik load balancer for cluster
ansible.builtin.include_role:
name: traefik-manager
vars:
cluster_name: "{{ cluster_name }}"
cluster_endpoint: "{{ cluster_endpoint }}"
control_plane_nodes: "{{ control_plane_nodes }}"
when: traefik_enabled | default(true)
- name: Wait for DNS propagation
ansible.builtin.wait_for:
timeout: 30
delegate_to: localhost
- name: Test cluster endpoint connectivity
ansible.builtin.wait_for:
host: "{{ cluster_endpoint }}"
port: "{{ cluster_api_port | default(6443) }}"
timeout: 60
delegate_to: localhost
register: connectivity_test
failed_when: false
- name: Display network setup results
ansible.builtin.debug:
msg: |
🌐 Network Setup Complete for {{ cluster_name }}:
DNS Entry: {{ cluster_endpoint }} → {{ cluster_vip }}
Load Balancer: {{ 'Configured' if traefik_enabled | default(true) else 'Skipped' }}
Connectivity: {{ 'Success' if connectivity_test.failed == false else 'Failed - Check firewall/network' }}
Next steps:
1. Verify: kubectl --kubeconfig ~/.kube/config-{{ cluster_name }} cluster-info
2. Switch context: kubectl config use-context {{ cluster_name }}-admin

View File

@@ -1,4 +0,0 @@
[Resolve]
DNS=127.0.0.1
DNSSEC=yes
DNSStubListener=no

View File

@@ -1,3 +0,0 @@
#SPDX-License-Identifier: MIT-0
---
# defaults file for containerd

View File

@@ -1,3 +0,0 @@
#SPDX-License-Identifier: MIT-0
---
# handlers file for containerd

View File

@@ -1,74 +0,0 @@
---
# tasks file for containerd
- name: Install containerd dependencies
become: true
ansible.builtin.package:
name: "{{ item }}"
state: present
loop:
"{{ containerd_dependencies }}"
- name: Install via apt
become: true
when: ansible_os_family == "Debian"
block:
- name: Add repo using key from URL (apt)
ansible.builtin.deb822_repository:
name: docker
types: deb
uris: https://download.docker.com/linux/ubuntu
suites: "{{ ansible_distribution_release }}"
components: stable
architectures: amd64
signed_by: https://download.docker.com/linux/ubuntu/gpg
- name: Update apt cache
ansible.builtin.apt:
update_cache: true
- name: Install via dnf
become: true
when: ansible_os_family == "RedHat"
block:
- name: Add repo using key from URL (dnf)
ansible.builtin.yum_repository:
name: docker
description: "Docker repository"
baseurl: https://download.docker.com/linux/ubuntu
gpgcheck: true
gpgkey: https://download.docker.com/linux/ubuntu/gpg
- name: Update dnf cache
ansible.builtin.dnf:
update_cache: true
- name: Install containerd
become: true
ansible.builtin.package:
name: containerd.io
state: present
- name: Create containerd config directory
ansible.builtin.file:
path: /etc/containerd
state: directory
owner: root
group: root
mode: '0755'
- name: Check if containerd config file exists
ansible.builtin.stat:
path: /etc/containerd/config.toml
register: containerd_config_file
- name: Generate default containerd config and enable SystemdCgroup
when: containerd_config_file.stat.exists == false
ansible.builtin.shell: |
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
- name: Restart and enable containerd service
ansible.builtin.systemd:
name: containerd
state: restarted
enabled: true

View File

@@ -1,6 +0,0 @@
#SPDX-License-Identifier: MIT-0
---
- hosts: localhost
remote_user: root
roles:
- containerd

View File

@@ -1,3 +0,0 @@
#SPDX-License-Identifier: MIT-0
---
# vars file for containerd

View File

@@ -1,38 +0,0 @@
Role Name
=========
A brief description of the role goes here.
Requirements
------------
Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required.
Role Variables
--------------
A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well.
Dependencies
------------
A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles.
Example Playbook
----------------
Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too:
- hosts: servers
roles:
- { role: username.rolename, x: 42 }
License
-------
BSD
Author Information
------------------
An optional section for the role authors to include contact information, or a website (HTML is not allowed).

View File

@@ -1,22 +0,0 @@
---
# FastPass Additional Control Plane Role Defaults
# Kubernetes services for control plane
kubernetes_services_control_plane:
- kubernetes_API
- etcd
- kubelet
- kube-scheduler
- kube-controller-manager
# Join token configuration
join_token_ttl: "24h"
certificate_key_ttl: "2h"
# Default cluster configuration
pod_network_cidr: "10.244.0.0/16"
service_cidr: "10.96.0.0/12"
# Kubelet configuration
kubelet_cgroup_driver: "systemd"
container_runtime_endpoint: "unix:///run/containerd/containerd.sock"

View File

@@ -1,2 +0,0 @@
---
# handlers file for fastpass-additional-control-plane

View File

@@ -1,33 +0,0 @@
---
galaxy_info:
author: Ryan Blundon
description: FastPass Additional Control Plane - Deploy additional control plane nodes to existing FastPass Kubernetes cluster
company: Homelab
license: MIT
min_ansible_version: "2.9"
platforms:
- name: Ubuntu
versions:
- focal
- jammy
- name: Debian
versions:
- bullseye
- bookworm
galaxy_tags:
- kubernetes
- k8s
- controlplane
- cluster
- fastpass
- kubeadm
dependencies:
- role: dns-manager
when: cluster_name is defined
- role: kubeconfig-manager
when: cluster_name is defined

View File

@@ -1,29 +0,0 @@
---
# role: fastpass-additional-control-plane
# description: FastPass Additional Control Plane - Deploy additional control plane nodes to existing FastPass Kubernetes cluster
# author: Ryan Blundon
# version: 1.0.0
# date: 2025-10-05
# This role joins additional nodes to an existing Kubernetes cluster as control plane nodes
# It follows the same patterns as fastpass-first-control-plane but uses kubeadm join instead of kubeadm init
- name: Display role information
ansible.builtin.debug:
msg: "Starting FastPass Additional Control Plane deployment for {{ inventory_hostname }}"
- name: Setup DNS record for cluster
ansible.builtin.include_role:
name: dns-manager
vars:
host_name: "{{ cluster_name }}"
- name: Open services for control plane
become: true
when: ansible_os_family == "Debian"
block:
- name: Open firewall ports for control plane
community.general.ufw:
rule: allow
name: "{{ item }}"
loop: "{{ kubernetes_services_control_plane }}"

View File

@@ -1,3 +0,0 @@
#SPDX-License-Identifier: MIT-0
localhost

View File

@@ -1,6 +0,0 @@
#SPDX-License-Identifier: MIT-0
---
- hosts: localhost
remote_user: root
roles:
- fastpass-additional-control-plane

View File

@@ -1,2 +0,0 @@
---
# vars file for fastpass-additional-control-plane

View File

@@ -1,15 +0,0 @@
---
# FastPass First Control Plane Role Defaults
# Kubernetes services for control plane
kubernetes_services_control_plane:
- kubernetes_API
- etcd
- kubelet
- kube-scheduler
- kube-controller-manager
kubernetes_services_worker:
- kubelet
- kube-proxy
- NodePort

View File

@@ -1,103 +0,0 @@
---
# role: fastpass-first-control-plane
# description: FastPass First Control Plane
# author: Ryan Blundon
# version: 1.0.0
# date: 2025-09-27
- name: Setup DNS record for cluster
ansible.builtin.include_role:
name: dns-manager
vars:
host_name: "{{ cluster_name }}"
- name: Open services for control plane
become: true
when: ansible_os_family == "Debian"
block:
- name: Open firewall ports for control plane
community.general.ufw:
rule: allow
name: "{{ item }}"
loop: "{{ kubernetes_services_control_plane }}"
- name: Create initial kubelet config file
become: true
ansible.builtin.copy:
dest: /var/lib/kubelet/config.yaml
owner: root
group: root
mode: "0644"
content: |
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
containerRuntimeEndpoint: unix:///run/containerd/containerd.sock
- name: Initialize Kubernetes cluster
become: true
ansible.builtin.command:
argv:
- kubeadm
- init
- --apiserver-advertise-address={{ ip_address }}
- --control-plane-endpoint={{ cluster_name }}
- --pod-network-cidr={{ pod_network_cidr }}
- --service-cidr={{ service_cidr }}
args:
creates: /etc/kubernetes/admin.conf
register: kubeadm_init
changed_when: kubeadm_init.rc == 0
- name: Ensure kubelet is enabled and started
become: true
ansible.builtin.systemd:
name: kubelet
enabled: true
state: started
- name: Setup kubeconfig for FastPass cluster
ansible.builtin.include_role:
name: kubeconfig-manager
# - name: Download tigera-operator manifest
# delegate_to: localhost
# ansible.builtin.get_url:
# url: https://raw.githubusercontent.com/projectcalico/calico/{{ calico_version }}/manifests/tigera-operator.yaml
# dest: /tmp/tigera-operator.yaml
# mode: "0644"
# - name: Install tigera-operator from downloaded file
# delegate_to: localhost
# kubernetes.core.k8s:
# kubeconfig: "{{ local_home }}/.kube/config"
# state: present
# src: /tmp/tigera-operator.yaml
# - name: Download Calico CRDs
# delegate_to: localhost
# ansible.builtin.get_url:
# url: https://raw.githubusercontent.com/projectcalico/calico/{{ calico_version }}/manifests/custom-resources.yaml
# dest: /tmp/custom-resources.yaml
# mode: "0644"
# - name: Install Calico CRDs from downloaded file
# delegate_to: localhost
# kubernetes.core.k8s:
# kubeconfig: "{{ local_home }}/.kube/config"
# state: present
# src: /tmp/custom-resources.yaml
- name: Download Flannel CNI
delegate_to: localhost
ansible.builtin.get_url:
url: https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
dest: /tmp/kube-flannel.yaml
mode: "0644"
- name: Install Flannel CNI from downloaded file
delegate_to: localhost
kubernetes.core.k8s:
kubeconfig: "{{ local_home }}/.kube/config"
state: present
src: /tmp/kube-flannel.yaml

Some files were not shown because too many files have changed in this diff Show More