- minecraft-prometheus-exporter v3.1.2 plugin (port 9225)
- ServiceMonitor for Prometheus scraping
- Grafana dashboard ConfigMap (ID 20659, Minecraft server stats)
- metrics port added to Service and Deployment containerPorts
- PaperMC 26.1.2 via itzg/minecraft-server:2026.7.0
- Namespace: minecraft, Service: journey-into-imagination
- TCP port 10182 (non-standard) via ingress-nginx tcp forwarding
- Pure Storage CSI PVC (pure-block, 50Gi) for world data
- World seed hardcoded: -5177989977648707969
- RCON password via ExternalSecret + 1Password Connect
- SleepMost v5.6.2 plugin for single-player sleep
- Whitelist off at launch, toggle-ready
- ExternalDNS annotations for internal Technitium record
- Manual steps: UniFi port forward WAN:10182→10.1.71.80:10182,
Cloudflare A record + SRV for journey-into-imagination.mk-labs.cloud
Fixes 413 Entity Too Large error in Obsidian LiveSync sync operations.
Applies to both internal (communicore.local) and public (communicore.mk-labs.cloud) ingress routes.
- Deploy Portworx Operator + CSI driver via ArgoCD
- Support both iSCSI block and NFS file storage from FlashArray
- Integrate with 1Password External Secrets for FlashArray credentials
- Include comprehensive deployment documentation and validation script
- Storage classes: pure-block (iSCSI) and pure-file (NFS)
- Talos Linux compatible with iSCSI/multipath configuration
- Production gateway on astro-orbiter VM working correctly
- K8s deployment ready but cannot complete Signal registration
- Signal servers reject WebSocket connections from K8s cluster network path
- Document migration procedure for when astro-orbiter is decommissioned
- Remove --account flag to let signal-cli auto-detect registered accounts
- Multi-account mode will find the registered +126****8840 account automatically
- Use bbernhard/signal-cli-rest-api:latest (known working image)
- Override command to run /usr/bin/signal-cli directly in daemon HTTP mode
- Bypasses REST API wrapper to expose native JSON-RPC endpoint at /api/v1/rpc
- Switch from non-existent GitLab registry image to official asamk/signal-cli:v0.14.5
- Maintain data path at /home/.local/share/signal-cli (where data was copied from astro-orbiter)
- Remove unnecessary command override, let entrypoint handle signal-cli execution
- Replace bbernhard/signal-cli-rest-api wrapper with native signal-cli 0.14.5
- Run signal-cli daemon in HTTP mode matching astro-orbiter working config
- Expose JSON-RPC API at /api/v1/rpc for Hermes gateway compatibility
- Switch health probes from HTTP /v1/health to TCP port check
- Maintain existing PVC mount path /home/.local/share/signal-cli
This fixes the missing /api/v1/rpc endpoint that Hermes requires for
Signal message delivery.
- Move manifests from cluster/platform/connections to cluster/applications/signal-cli
- Add Ingress for connections.local.mk-labs.cloud with cert-manager TLS
- Update ArgoCD application path to cluster/applications/signal-cli
- Configure letsencrypt-prod cluster issuer for automatic TLS certificates
This enables external HTTPS access to the Signal CLI REST API for Hermes
notifications with automatic certificate management.
Deployment name: connections → signal-cli-rest-api (what it IS)
Service name: connections (unchanged - Epcot-themed role)
Updated labels throughout deployment and service selector to match.
- Set namespace pod-security.kubernetes.io/enforce to baseline
- Remove restrictive container securityContext
- Allows signal-cli-rest-api container to run its entrypoint script
which requires user/group modification capabilities
- Add container securityContext to satisfy PodSecurity policy
- Set SIGNAL_CLI_UID/GID to 0 to disable user modification attempts
- Fixes CrashLoopBackOff due to groupmod permission denied
The signal-cli-rest-api container's entrypoint script requires
privileges to modify user/group settings. Removing securityContext
allows the container to run with its default settings.
- Deploy signal-cli-rest-api 0.85 for Hermes Signal notifications
- Replace broken astro-orbiter VM (10.1.71.130:8080) with K8s service
- ArgoCD-managed GitOps deployment in connections namespace
- NFS-backed persistent storage for signal-cli state
- Fixes UNREGISTERED_FAILURE affecting midday market cron job
- Epcot-themed service (communication pavilion concept)
Service endpoint: http://connections.connections.svc.cluster.local:8080
Per Tony's recommendation — eliminates the explicit include filter.
New manifests go in templates/ and are picked up automatically by ArgoCD.
No filter to update when adding future resources.
Moved: namespace.yaml, externalsecret.yaml, ingress-public.yaml -> templates/
ExternalDNS v0.15.1 does not support --target as a CLI flag.
Remove the extraArgs stanza from external-dns-cloudflare values.yaml
and instead add the target annotation directly on the CouchDB ingress:
external-dns.alpha.kubernetes.io/target: ingress.mk-labs.cloud
This achieves the same result (Cloudflare CNAME -> ingress.mk-labs.cloud)
without crashing the controller.
- external-dns-cloudflare: add excludeDomains: [local.mk-labs.cloud] to stop
internal subdomain records from leaking to Cloudflare
- external-dns-cloudflare: replace hostname annotationFilter with opt-in model
(external-dns.alpha.kubernetes.io/public=true) so only explicitly tagged
services get public Cloudflare records
- external-dns-cloudflare: add extraArgs --target=lb.mk-labs.cloud
- external-dns (Technitium/rfc2136): add extraArgs
--target=lightning-lane.local.mk-labs.cloud for internal records
- couchdb: add external-dns.alpha.kubernetes.io/public: 'true' annotation —
first service to opt in to public DNS; will create communicore.mk-labs.cloud
pointing to lb.mk-labs.cloud via Cloudflare ExternalDNS
URGENT: Cloudflare was creating records for local.mk-labs.cloud hosts.
Ryan: manually delete any *.local.mk-labs.cloud records currently in Cloudflare
(look for communicore.local.mk-labs.cloud and any other local.* entries).
Deploy a second ExternalDNS instance targeting Cloudflare to manage
public DNS records in the mk-labs.cloud zone. The existing Technitium
(rfc2136) instance handling local.mk-labs.cloud is unchanged.
Components:
- application.yaml: ArgoCD Application, wave 6, namespace external-dns-cloudflare
- values.yaml: Cloudflare provider, domainFilters: mk-labs.cloud, txtOwnerId: fastpass
- externalsecret.yaml: ExternalSecret pulling CF_API_TOKEN from 1Password
PREREQUISITE (manual): Ryan must create the following in 1Password before
the ExternalSecret will sync:
Item name: cloudflare-external-dns
Field name: api-token
Value: Cloudflare API token with DNS Edit on mk-labs.cloud
Until then, the ExternalSecret will show SecretSyncedError — expected.
Playwright service doesn't expose a health endpoint at /, causing HTTP
probes to fail with 404. Switch to tcpSocket probes which simply verify
the port is listening. Service is already confirmed running on port 3000.
- Changed worker deployment probes from HTTP to TCP (port 3005)
* Worker liveness endpoint doesn't serve HTTP at '/' path
* TCP socket check more appropriate for background worker
* Resolves pod restart loop and readiness failures
- Corrected HTTPRoute gateway reference
* Changed from 'gateway' in 'default' namespace
* To 'fastpass-gateway' in 'gateway' namespace
* HTTPRoute now properly accepted by gateway
All 7 deployments Running and Ready. System operational.
