- Deploy Portworx Operator + CSI driver via ArgoCD
- Support both iSCSI block and NFS file storage from FlashArray
- Integrate with 1Password External Secrets for FlashArray credentials
- Include comprehensive deployment documentation and validation script
- Storage classes: pure-block (iSCSI) and pure-file (NFS)
- Talos Linux compatible with iSCSI/multipath configuration
- Production gateway on astro-orbiter VM working correctly
- K8s deployment ready but cannot complete Signal registration
- Signal servers reject WebSocket connections from K8s cluster network path
- Document migration procedure for when astro-orbiter is decommissioned
- Remove --account flag to let signal-cli auto-detect registered accounts
- Multi-account mode will find the registered +126****8840 account automatically
- Use bbernhard/signal-cli-rest-api:latest (known working image)
- Override command to run /usr/bin/signal-cli directly in daemon HTTP mode
- Bypasses REST API wrapper to expose native JSON-RPC endpoint at /api/v1/rpc
- Switch from non-existent GitLab registry image to official asamk/signal-cli:v0.14.5
- Maintain data path at /home/.local/share/signal-cli (where data was copied from astro-orbiter)
- Remove unnecessary command override, let entrypoint handle signal-cli execution
- Replace bbernhard/signal-cli-rest-api wrapper with native signal-cli 0.14.5
- Run signal-cli daemon in HTTP mode matching astro-orbiter working config
- Expose JSON-RPC API at /api/v1/rpc for Hermes gateway compatibility
- Switch health probes from HTTP /v1/health to TCP port check
- Maintain existing PVC mount path /home/.local/share/signal-cli
This fixes the missing /api/v1/rpc endpoint that Hermes requires for
Signal message delivery.
- Move manifests from cluster/platform/connections to cluster/applications/signal-cli
- Add Ingress for connections.local.mk-labs.cloud with cert-manager TLS
- Update ArgoCD application path to cluster/applications/signal-cli
- Configure letsencrypt-prod cluster issuer for automatic TLS certificates
This enables external HTTPS access to the Signal CLI REST API for Hermes
notifications with automatic certificate management.
Deployment name: connections → signal-cli-rest-api (what it IS)
Service name: connections (unchanged - Epcot-themed role)
Updated labels throughout deployment and service selector to match.
- Set namespace pod-security.kubernetes.io/enforce to baseline
- Remove restrictive container securityContext
- Allows signal-cli-rest-api container to run its entrypoint script
which requires user/group modification capabilities
- Add container securityContext to satisfy PodSecurity policy
- Set SIGNAL_CLI_UID/GID to 0 to disable user modification attempts
- Fixes CrashLoopBackOff due to groupmod permission denied
The signal-cli-rest-api container's entrypoint script requires
privileges to modify user/group settings. Removing securityContext
allows the container to run with its default settings.
- Deploy signal-cli-rest-api 0.85 for Hermes Signal notifications
- Replace broken astro-orbiter VM (10.1.71.130:8080) with K8s service
- ArgoCD-managed GitOps deployment in connections namespace
- NFS-backed persistent storage for signal-cli state
- Fixes UNREGISTERED_FAILURE affecting midday market cron job
- Epcot-themed service (communication pavilion concept)
Service endpoint: http://connections.connections.svc.cluster.local:8080
Per Tony's recommendation — eliminates the explicit include filter.
New manifests go in templates/ and are picked up automatically by ArgoCD.
No filter to update when adding future resources.
Moved: namespace.yaml, externalsecret.yaml, ingress-public.yaml -> templates/
ExternalDNS v0.15.1 does not support --target as a CLI flag.
Remove the extraArgs stanza from external-dns-cloudflare values.yaml
and instead add the target annotation directly on the CouchDB ingress:
external-dns.alpha.kubernetes.io/target: ingress.mk-labs.cloud
This achieves the same result (Cloudflare CNAME -> ingress.mk-labs.cloud)
without crashing the controller.
- external-dns-cloudflare: add excludeDomains: [local.mk-labs.cloud] to stop
internal subdomain records from leaking to Cloudflare
- external-dns-cloudflare: replace hostname annotationFilter with opt-in model
(external-dns.alpha.kubernetes.io/public=true) so only explicitly tagged
services get public Cloudflare records
- external-dns-cloudflare: add extraArgs --target=lb.mk-labs.cloud
- external-dns (Technitium/rfc2136): add extraArgs
--target=lightning-lane.local.mk-labs.cloud for internal records
- couchdb: add external-dns.alpha.kubernetes.io/public: 'true' annotation —
first service to opt in to public DNS; will create communicore.mk-labs.cloud
pointing to lb.mk-labs.cloud via Cloudflare ExternalDNS
URGENT: Cloudflare was creating records for local.mk-labs.cloud hosts.
Ryan: manually delete any *.local.mk-labs.cloud records currently in Cloudflare
(look for communicore.local.mk-labs.cloud and any other local.* entries).
Deploy a second ExternalDNS instance targeting Cloudflare to manage
public DNS records in the mk-labs.cloud zone. The existing Technitium
(rfc2136) instance handling local.mk-labs.cloud is unchanged.
Components:
- application.yaml: ArgoCD Application, wave 6, namespace external-dns-cloudflare
- values.yaml: Cloudflare provider, domainFilters: mk-labs.cloud, txtOwnerId: fastpass
- externalsecret.yaml: ExternalSecret pulling CF_API_TOKEN from 1Password
PREREQUISITE (manual): Ryan must create the following in 1Password before
the ExternalSecret will sync:
Item name: cloudflare-external-dns
Field name: api-token
Value: Cloudflare API token with DNS Edit on mk-labs.cloud
Until then, the ExternalSecret will show SecretSyncedError — expected.
Playwright service doesn't expose a health endpoint at /, causing HTTP
probes to fail with 404. Switch to tcpSocket probes which simply verify
the port is listening. Service is already confirmed running on port 3000.
- Changed worker deployment probes from HTTP to TCP (port 3005)
* Worker liveness endpoint doesn't serve HTTP at '/' path
* TCP socket check more appropriate for background worker
* Resolves pod restart loop and readiness failures
- Corrected HTTPRoute gateway reference
* Changed from 'gateway' in 'default' namespace
* To 'fastpass-gateway' in 'gateway' namespace
* HTTPRoute now properly accepted by gateway
All 7 deployments Running and Ready. System operational.
- Added config.json key to harbor-credentials ExternalSecret
This ensures kaniko can find the Docker auth config at /kaniko/.docker/config.json
(previously only .dockerconfigjson was present)
- Created test-app-build PipelineRun manifest for validation testing
- Successfully validated end-to-end pipeline:
✅ git-clone Task deployed and working
✅ kaniko-build Task deployed and working
✅ container-build Pipeline deployed and working
✅ Harbor authentication working with robot account
✅ Test image built and pushed: the-seas.local.mk-labs.cloud/library/test-app:v1.0.0
✅ Image digest: sha256:aa143f4a01795a1d307b711108ca0c89f36e00ea38fddb9d7b2febd5fffc46d7
Pipeline test results:
- PipelineRun: test-app-build-005 - SUCCEEDED
- fetch-repository TaskRun - SUCCEEDED
- build-and-push TaskRun - SUCCEEDED
Tekton CI/CD platform is now operational and ready for production workloads.
Replace letsencrypt-staging with letsencrypt-prod for trusted certificate.
Staging cert was causing connection resets due to certificate validation failures.
Change CouchDB ingress hostname from communicore.mk-labs.cloud to
communicore.local.mk-labs.cloud to align with External-DNS configuration.
CouchDB is an internal service and should use the .local.mk-labs.cloud
zone managed by Technitium DNS, not the public .mk-labs.cloud zone
managed by Cloudflare.
This ensures External-DNS will properly create the A record in the
internal DNS server.
Updated README to reflect the full workflow:
1. Robot account creation via Job
2. Secret regeneration and capture
3. Storage in 1Password (harbor-robot-accounts item)
4. Automatic sync via ExternalSecrets to K8s
Creates two docker-registry secrets from 1Password:
1. harbor-tekton-robot - for Tekton CI/CD pipeline push/pull
2. harbor-pull-secret - for fastpass cluster image pulls
Both sync from 1Password item 'harbor-robot-accounts' with fields:
- tekton-builder-username / tekton-builder-password
- fastpass-cluster-username / fastpass-cluster-password
Credentials document placed in PKA inbox for manual 1Password entry.
Once stored, ESO will automatically sync and create the secrets.
Documents robot account management via GitOps:
- tekton-builder and fastpass-cluster robot accounts
- Manual Job execution (PostSync hooks don't work with multi-source)
- Credential retrieval and storage in 1Password
- ImagePullSecret creation for K8s clusters
