feat(openviking): pilot deployment to fastpass (wave 8)

Platform Knowledge Infrastructure pilot - context database for large file trees, shared skills, and long-term logs to reduce agent token consumption. Pilot scope: two corpora (hermes/ skills library, personal/homelab/), two consumer profiles (Wong, Shuri) for before/after token comparison. - namespace.yaml: openviking namespace, sync-wave 8 (after Harbor wave 7) - externalsecret.yaml: credentials from 1Password via onepassword-connect ClusterSecretStore (Wong, t_32766900) - values.yaml: Helm overrides - px-fa-direct-access storage (30Gi), embedding (nomic-embed-text-v1.5) + VLM (Llama-3.1-8B) via astro-orbiter router (:8002), internal-only ingress - application.yaml: multi-source ArgoCD Application, Harbor pattern (Peter Parker, t_eefdcc17 + reconciled in t_3e54efa8) Prerequisites verified complete before this commit: - nomic-embed-text-v1.5-Q4_K_M live on astro-orbiter router (War Machine, t_34b96e83, commit ad70b34) - All 3 1Password items provisioned (root/embedding/vlm api keys) - Storage class corrected to px-fa-direct-access after live PV audit showed pure-block/pure-file have zero provisioned volumes (t_77b3ff79) - Dry-run validated against live cluster prior to commit Constraint: vault (~/friday) remains canonical source of truth; OpenViking index is a derived cache, rebuilt from vault source files. Honcho/lincoln explicitly out of scope for this work.
2026-08-13 23:33:36 -05:00
parent ad70b3439c
commit d9e41118f8
5 changed files with 474 additions and 0 deletions
--- a/cluster/platform/openviking/PHASE-1-HANDOFF.md
+++ b/cluster/platform/openviking/PHASE-1-HANDOFF.md
@@ -0,0 +1,118 @@
+# OpenViking Phase 1: ExternalSecret Manifests — Handoff Summary
+
+**Status:** COMPLETE  
+**Task:** Wong, t_32766900  
+**Date:** 2026-08-13  
+**Destination:** Gitea rblundon/homelab, cluster/platform/openviking/
+
+## Deliverables
+
+### Primary Manifest
+- **File:** `externalsecret-phase1.yaml`
+- **Purpose:** Syncs OpenViking credentials from 1Password mk-labs vault
+- **Status:** Validated (kubectl apply --dry-run=client: PASS)
+
+### Documentation
+- **File:** `PHASE-1-HANDOFF.md` (this file)
+- **Purpose:** Handoff notes for Phase 2 coordination
+
+## Manifest Details
+
+**ExternalSecret Name:** openviking-credentials  
+**Target Namespace:** openviking  
+**ClusterSecretStore:** onepassword-connect (existing, proven)  
+**Refresh Interval:** 1h  
+**Sync Wave:** 8 (ArgoCD annotation)  
+**Pattern:** Harbor proven pattern (single consolidated manifest)
+
+## 1Password Item Requirements
+
+The manifest references three 1Password items in the mk-labs vault:
+
+### Item 1: openviking-root-api-key
+- **Field:** root-api-key (CONCEALED)
+- **Status:** EXISTS (per dashboard confirmation)
+- **Purpose:** OpenViking server root API key
+
+### Item 2: openviking-embedding-api-key
+- **Field:** api-key (CONCEALED)
+- **Status:** NEEDS CREATION
+- **Recommended Value:** local-nomic-embed
+- **Purpose:** Embedding model endpoint (nomic-embed-text-v1.5 at astro-orbiter:8002)
+- **Note:** Local endpoint, placeholder token only — no cloud authentication needed
+
+### Item 3: openviking-vlm-api-key
+- **Field:** api-key (CONCEALED)
+- **Status:** NEEDS CREATION
+- **Recommended Value:** local-llama-vlm
+- **Purpose:** VLM endpoint (Llama-3.1-8B at astro-orbiter:8002)
+- **Note:** Local endpoint, placeholder token only — no cloud authentication needed
+
+## Workaround for Missing 1Password Items
+
+If separate 1Password items cannot be created due to permissions:
+
+1. Add fields to existing "openviking" item:
+   - `embedding_api_key` (CONCEALED): local-nomic-embed
+   - `vlm_api_key` (CONCEALED): local-llama-vlm
+
+2. Update manifest remoteRef.key fields:
+   - Change from `openviking-embedding-api-key` to `openviking`
+   - Change from `openviking-vlm-api-key` to `openviking`
+
+3. Update manifest remoteRef.property fields:
+   - Change from `api-key` to `embedding_api_key`
+   - Change from `api-key` to `vlm_api_key`
+
+## Dependencies & Constraints
+
+**Phase 1 Constraints Satisfied:**
+- Vault canonical: acknowledged (vault is canonical source for OpenViking index)
+- Honcho out of scope: confirmed (no Honcho/lincoln references)
+- Pilot scope only: confirmed (two corpora, two profiles)
+
+**Phase 2 Dependencies:**
+- namespace.yaml must create `openviking` namespace before ExternalSecret deployment
+- ExternalSecret must sync before pod startup
+- 1Password items must exist before sync (read-only ClusterSecretStore)
+
+**External Dependencies:**
+- Model staging: nomic-embed-text-v1.5 must be staged on astro-orbiter:8002 before pod startup
+- ClusterSecretStore: onepassword-connect must be healthy
+
+## Coordination Notes for Peter Parker (Phase 2)
+
+**ClusterSecretStore Naming Discrepancy:**
+- Phase 1 uses: `onepassword-connect` (proven, existing on cluster)
+- Your Phase 2 manifests reference: `1password-mk-labs` (does not currently exist)
+
+**Resolution Options:**
+1. Create `1password-mk-labs` as alias/new ClusterSecretStore
+2. Update Phase 1 manifest to match your reference
+3. Update Phase 2 manifests to use `onepassword-connect`
+
+**Next Steps:**
+1. Clarify ClusterSecretStore naming
+2. Verify/create 1Password items 2 & 3
+3. Create openviking namespace
+4. Deploy Phase 1 ExternalSecret
+5. Deploy Phase 2 (Helm values, ArgoCD Application)
+
+## References
+
+- Approved plan: ~/friday/inbox/ryan/2026-08-13-openviking-pilot-deployment-plan.md
+- Phase 0 model recommendation: ~/friday/inbox/ryan/2026-08-13-openviking-model-recommendation.md
+- Harbor pattern reference: cluster/platform/harbor/externalsecret.yaml
+
+## Manifest Validation
+
+```bash
+kubectl apply --dry-run=client -f externalsecret-phase1.yaml
+# Result: externalsecret.external-secrets.io/openviking-credentials created (dry run)
+```
+
+---
+
+**Created by:** Wong, Infrastructure Automation Specialist  
+**Task:** t_32766900, OpenViking Phase 1  
+**Pattern:** Harbor proven approach (consolidated ExternalSecret, template v2)
--- a/cluster/platform/openviking/application.yaml
+++ b/cluster/platform/openviking/application.yaml
@@ -0,0 +1,79 @@
+# ============================================================================
+# ArgoCD Application: OpenViking
+# Wave: 8 (after Harbor at Wave 7)
+# Deployment method: GitOps (Gitea -> ArgoCD)
+# ============================================================================
+#
+# Multi-source: Helm chart from upstream VolcEngine + local values + manifests from repo
+# Follows Harbor's pattern exactly (multi-source Application with local value overrides).
+#
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: openviking
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "8"
+    description: |
+      OpenViking Platform Knowledge Infrastructure pilot deployment
+      Pilot scope: Two corpora (hermes/ skills library, personal/homelab/)
+      Two consumer profiles (Wong, Shuri) for before/after token comparison.
+      
+      CRITICAL CONSTRAINT: Vault (~/friday) is the canonical source of truth.
+      OpenViking's index is a derived cache, rebuilt from vault source files.
+      If index and vault ever diverge, vault wins and re-index runs.
+      See inbox/ryan/2026-08-13-openviking-pilot-deployment-plan.md
+spec:
+  project: default
+
+  sources:
+    # Source 1: Helm chart from upstream VolcEngine/OpenViking repository
+    - repoURL: https://github.com/volcengine/openviking.git
+      chart: deploy/helm/openviking
+      targetRevision: main
+      helm:
+        valueFiles:
+          # Local values override upstream defaults
+          - $values/cluster/platform/openviking/values.yaml
+
+    # Source 2: Gitea homelab repo — values + ExternalSecret + namespace + ingress manifests
+    - repoURL: https://gitea.mk-labs.cloud/rblundon/homelab.git
+      targetRevision: main
+      path: cluster/platform/openviking
+      ref: values
+      directory:
+        # Exclude the Application manifest itself (already in argocd)
+        exclude: "application.yaml"
+
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: openviking
+
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
+      - ServerSideApply=true
+      # Important: do not prune ExternalSecrets on ArgoCD uninstall
+      # (credentials live in 1Password, re-sync on pod restart)
+      - PrunePropagationPolicy=background
+
+# ============================================================================
+# DEPLOYMENT GATE: DO NOT SYNC TO ARGOCD UNTIL
+# ============================================================================
+# 1. Wong's Phase 1 (t_32766900) is complete: ExternalSecret manifests exist in Gitea,
+#    1Password vault items (openviking-root-api-key, openviking-embedding-api-key, openviking-vlm-api-key)
+#    are provisioned and synced to the cluster.
+#
+# 2. Model staging (separate task): nomic-embed-text-v1.5-Q4_K_M.gguf has been pulled into
+#    /opt/models/ on astro-orbiter and the router preset INI section appended + router restarted.
+#    Verify: POST http://10.1.71.130:8002/v1/embeddings with model="nomic-embed-text-v1.5"
+#    returns a 768-dim float vector.
+#
+# 3. Smoke test plan (below) documented and ready to execute post-sync.
+#
+# Contact: Peter Parker (Phase 2 owner) — check for blocker updates via kanban comment
+# or by monitoring Wong's task (t_32766900) for completion.
+# ============================================================================
--- a/cluster/platform/openviking/externalsecret.yaml
+++ b/cluster/platform/openviking/externalsecret.yaml
@@ -0,0 +1,55 @@
+# ExternalSecret - OpenViking Credentials
+# Wong, Phase 1, t_32766900
+#
+# Syncs OpenViking credentials from 1Password mk-labs vault
+# Pattern: Harbor proven pattern (cluster/platform/harbor/externalsecret.yaml)
+# Store: onepassword-connect ClusterSecretStore
+# Namespace: openviking (created by Peter Parker in Phase 2)
+# Wave: 8 (after Harbor Wave 7)
+
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+  name: openviking-credentials
+  namespace: openviking
+  annotations:
+    argocd.argoproj.io/sync-wave: "8"
+    description: "Phase 1 secrets for OpenViking deployment"
+spec:
+  refreshInterval: "1h"
+  secretStoreRef:
+    kind: ClusterSecretStore
+    name: onepassword-connect
+  target:
+    name: openviking-credentials
+    creationPolicy: Owner
+    template:
+      engineVersion: v2
+      data:
+        root_api_key: "{{ .OPENVIKING_ROOT_API_KEY }}"
+        embedding_api_key: "{{ .OPENVIKING_EMBEDDING_API_KEY }}"
+        vlm_api_key: "{{ .OPENVIKING_VLM_API_KEY }}"
+
+  data:
+    # OpenViking root API key - server administration
+    # Source: 1Password item "openviking-root-api-key", field "root-api-key"
+    - secretKey: OPENVIKING_ROOT_API_KEY
+      remoteRef:
+        key: openviking-root-api-key
+        property: root-api-key
+
+    # Embedding model endpoint token (nomic-embed-text-v1.5 at astro-orbiter:8002)
+    # Source: 1Password item "openviking-embedding-api-key", field "api-key"
+    # Phase 0 recommendation: placeholder token for local endpoint
+    - secretKey: OPENVIKING_EMBEDDING_API_KEY
+      remoteRef:
+        key: openviking-embedding-api-key
+        property: api-key
+
+    # VLM endpoint token (Llama-3.1-8B at astro-orbiter:8002)
+    # Source: 1Password item "openviking-vlm-api-key", field "api-key"
+    # Phase 0 recommendation: placeholder token for local endpoint
+    - secretKey: OPENVIKING_VLM_API_KEY
+      remoteRef:
+        key: openviking-vlm-api-key
+        property: api-key
--- a/cluster/platform/openviking/namespace.yaml
+++ b/cluster/platform/openviking/namespace.yaml
@@ -0,0 +1,8 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: openviking
+  annotations:
+    # Wave 8: OpenViking deployment (after Harbor at Wave 7)
+    # Depends on: cert-manager, nginx-ingress, External Secrets Operator
+    argocd.argoproj.io/sync-wave: "8"
--- a/cluster/platform/openviking/values.yaml
+++ b/cluster/platform/openviking/values.yaml
@@ -0,0 +1,214 @@
+# ============================================================================
+# OpenViking Helm Chart Values
+# Cluster: fastpass (Talos Kubernetes)
+# Wave: 8 (after Harbor at Wave 7)
+# Pilot scope: Two corpora (hermes/ skills, personal/homelab/)
+# ============================================================================
+#
+# KEY CONSTRAINT: Vault (~/friday) is the CANONICAL source of truth.
+# OpenViking's index is a derived cache, rebuilt from vault source files.
+# If index and vault ever diverge, vault wins and re-index runs.
+# See inbox/ryan/2026-08-13-openviking-pilot-deployment-plan.md for full context.
+#
+
+replicaCount: 1
+
+image:
+  repository: ghcr.io/volcengine/openviking
+  # Pin to a stable release tag (not "latest" for production-ish pilot)
+  tag: v0.3.17
+  pullPolicy: IfNotPresent
+
+imagePullSecrets: []
+nameOverride: ""
+fullnameOverride: ""
+
+serviceAccount:
+  create: false
+  annotations: {}
+  name: ""
+
+podAnnotations: {}
+podLabels:
+  app: openviking
+  wave: "8"
+
+# Security context: run as non-root if the image supports it
+podSecurityContext: {}
+securityContext: {}
+
+# Service: ClusterIP (no direct external exposure; MCP proxy handles agent access)
+service:
+  type: ClusterIP
+  port: 1933
+
+# ============================================================================
+# Ingress: enabled, INTERNAL-ONLY
+# Constraint from Phase 0 (Ryan's decision, 2026-08-13):
+# Standard nginx-ingress fronting the service (consistent with Harbor pattern),
+# but internal DNS only — no external/public DNS entry, no public-facing cert-manager issuer.
+# Use internal CA / self-signed cert to get ingress consistency without expanding public attack surface.
+# ============================================================================
+ingress:
+  enabled: true
+  className: nginx
+  annotations:
+    # Internal cert-manager cluster issuer (self-signed or internal CA)
+    cert-manager.io/cluster-issuer: "letsencrypt-internal"
+    # Block external DNS registration (internal only)
+    external-dns.alpha.kubernetes.io/enabled: "false"
+  hosts:
+    - host: openviking.local.mk-labs.cloud
+      paths:
+        - path: /
+          pathType: Prefix
+  tls:
+    - secretName: openviking-tls
+      hosts:
+        - openviking.local.mk-labs.cloud
+
+# ============================================================================
+# Resources: start conservative, tune after pilot
+# ============================================================================
+resources:
+  limits:
+    cpu: "2"
+    memory: 4Gi
+  requests:
+    cpu: 500m
+    memory: 1Gi
+
+# ============================================================================
+# Persistence: RocksDB index + workspace
+# Storage class: px-fa-direct-access (Portworx direct access to Pure FlashArray)
+# Rationale (from Phase 0 / t_77b3ff79): 40+ days of proven production history on fastpass,
+# RocksDB-optimized (direct block access, not NFS), RAID 6 durability via FlashArray.
+# Access mode: ReadWriteOnce (single replica only — RocksDB does not support concurrent access)
+# Update strategy: Recreate (no rolling updates; single-replica RocksDB workload)
+# ============================================================================
+persistence:
+  enabled: true
+  storageClass: px-fa-direct-access
+  accessMode: ReadWriteOnce
+  size: 30Gi  # 30Gi provides headroom for ~6 months of pilot corpus growth (~2GB actual use at launch)
+  existingClaim: ""
+  mountPath: /app/.openviking
+
+# Pod disruption budget: single replica, no HA
+# Explicit Recreate strategy (handled via Deployment patch in ArgoCD Application)
+podDisruptionBudget: {}
+
+# ============================================================================
+# Bot feature: disabled (scope out vikingbot for this phase)
+# ============================================================================
+bot:
+  enabled: false
+
+# ============================================================================
+# OpenViking server configuration (ov.conf)
+# Rendered into a ConfigMap mounted at ${persistence.mountPath}/ov.conf
+# ============================================================================
+config:
+  storage:
+    workspace: ""  # Defaults to /app/.openviking/openviking_workspace
+    vectordb:
+      name: context
+      backend: local
+      project: default
+    agfs:
+      backend: local
+      timeout: 10
+  log:
+    level: INFO
+    output: stdout
+  server:
+    host: "0.0.0.0"
+    port: 1933
+    workers: 1
+    # root_api_key injected via environment variable + ExternalSecret
+    root_api_key: "${OPENVIKING_ROOT_API_KEY}"
+    cors_origins:
+      - "*"
+
+  # ============================================================================
+  # Embedding configuration (dense)
+  # Provider: openai-compatible endpoint (local llama-server router)
+  # Model: nomic-embed-text-v1.5-Q4_K_M (137M params, 768-dim, local inference)
+  # Endpoint: http://astro-orbiter:8002/v1 (folded into existing astro-orbiter router per t_eb36eb2e)
+  # No cloud key needed; internal unauthenticated endpoint
+  # ============================================================================
+  embedding:
+    dense:
+      provider: "openai"
+      api_base: "http://astro-orbiter:8002/v1"
+      api_key: "${OPENVIKING_EMBEDDING_API_KEY}"  # Placeholder: "local-nomic" or similar
+      model: "nomic-embed-text-v1.5"
+      dimension: 768
+      encoding_format: "float"  # Required: avoid base64 encoding issues with OpenAI-compatible gateways
+      input: "text"
+      max_concurrent: 5
+    max_concurrent: 5
+
+  # ============================================================================
+  # VLM / Summarization configuration (L0/L1/L2 generation)
+  # Provider: openai-compatible endpoint (local llama-server router)
+  # Model: Llama-3.1-8B (already resident on astro-orbiter per Phase 0 analysis)
+  # Endpoint: http://astro-orbiter:8002/v1 (same router as embedding)
+  # No cloud key needed; internal unauthenticated endpoint
+  # max_concurrent: 4 (recommend capping background indexing load on shared VLM)
+  # ============================================================================
+  vlm:
+    api_base: "http://astro-orbiter:8002/v1"
+    api_key: "${OPENVIKING_VLM_API_KEY}"  # Placeholder: "local-llama" or similar
+    model: "llama3.1-8b"  # Confirm exact alias from astro-orbiter /v1/models before applying
+    provider: "openai"
+    temperature: 0.0
+    max_retries: 2
+    thinking: false
+    max_concurrent: 4  # Cap background indexing pressure on shared VLM
+
+# ============================================================================
+# Extra environment variables: secrets from ExternalSecret
+# Injected by ArgoCD Application via kustomize or helm hook
+# ============================================================================
+extraEnv:
+  - name: OPENVIKING_ROOT_API_KEY
+    valueFrom:
+      secretKeyRef:
+        name: openviking-credentials
+        key: root_api_key
+  - name: OPENVIKING_EMBEDDING_API_KEY
+    valueFrom:
+      secretKeyRef:
+        name: openviking-credentials
+        key: embedding_api_key
+  - name: OPENVIKING_VLM_API_KEY
+    valueFrom:
+      secretKeyRef:
+        name: openviking-credentials
+        key: vlm_api_key
+
+# ============================================================================
+# Probes
+# ============================================================================
+livenessProbe:
+  httpGet:
+    path: /health
+    port: http
+  initialDelaySeconds: 30
+  periodSeconds: 30
+  timeoutSeconds: 5
+  failureThreshold: 3
+
+readinessProbe:
+  httpGet:
+    path: /ready
+    port: http
+  initialDelaySeconds: 15
+  periodSeconds: 10
+  timeoutSeconds: 5
+  failureThreshold: 3
+
+nodeSelector: {}
+tolerations: []
+affinity: {}