homelab/cluster/platform/tekton/README.md

# Innoventions - Tekton CI/CD Platform

**Service Theme:** Innoventions (EPCOT Innovation Hub)  
**Dashboard DNS:** mission-space.local.mk-labs.cloud (Mission: SPACE control center)  
**Namespace:** innoventions  
**Wave:** 8 (after Harbor wave 7)  
**Phase:** Platform Buildout Phase 2

---

## Overview

Tekton Pipelines provides the CI/CD brain of the mk-labs platform. It builds containers from Git repositories (Gitea) and pushes images to Harbor registry, enabling automated workflows triggered by webhooks.

**Key capabilities:**
- Container image builds from source (Kaniko)
- Automated webhook triggers from Gitea
- Reusable task library for common operations
- Web dashboard for pipeline monitoring and management
- Integration with Harbor registry for image storage

---

## Component Versions

**Installation Date:** June 6, 2026  
**Deployed By:** Rocket Raccoon (CI/CD Specialist)

| Component | Version | Notes |
|-----------|---------|-------|
| Tekton Operator | v0.79.1 | Latest stable (May 2026) |
| Tekton Pipelines | v1.13.0 | Managed by operator |
| Tekton Triggers | v0.36.0 | Managed by operator |
| Tekton Dashboard | v0.69.0 | Managed by operator |
| Kaniko Executor | v1.24.0 | For rootless container builds |

**Version pinning policy:** All versions explicitly pinned. No `latest` tags.

**Compatibility verified:** 
- Tekton Operator v0.79.1 supports Kubernetes 1.27+
- Fastpass cluster running Talos Kubernetes 1.30+
- Compatible with existing Gateway API (wave 4)

---

## Architecture

### Installation Method: Tekton Operator

Tekton is deployed via the official Tekton Operator, not Helm charts or raw manifests.

**Rationale:**
- Official Tekton project support
- Automated component lifecycle management
- Proven upgrade path
- Single TektonConfig CR manages all components

**Pattern:**
1. ArgoCD deploys operator manifest (operator-install.yaml)
2. ArgoCD applies TektonConfig CR (tektonconfig.yaml)
3. Operator deploys and manages Pipelines, Triggers, Dashboard

### Components Enabled

Via TektonConfig `profile: all`:

- **Pipelines:** Core task/pipeline execution engine
- **Triggers:** Webhook EventListener for Git events
- **Dashboard:** Web UI for monitoring and management
- **Chains:** Disabled (planned for Phase 3 - image signing)

### Integration Points

**Harbor Container Registry:**
- URL: the-seas.local.mk-labs.cloud
- Robot account: tekton-builder
- Projects: library, platform, applications
- Credentials: ExternalSecret from 1Password

**Gitea Source Control:**
- Webhook integration via EventListener
- Clone tasks pull from Gitea repos
- Webhook secret validation

**Gateway API:**
- Dashboard exposed via HTTPRoute
- TLS certificate from cert-manager
- Hostname: mission-space.local.mk-labs.cloud

**Storage:**
- PVC-backed workspaces using nfs-emporium StorageClass
- 7-day retention for completed PipelineRuns
- Auto-cleanup to manage disk usage

---

## Files in This Directory

```
cluster/platform/tekton/
├── application.yaml                # ArgoCD Application definition
├── tektonconfig.yaml               # TektonConfig CR (what to install)
├── operator-install.yaml           # Tekton Operator static manifest
├── externalsecret.yaml             # Harbor credentials from 1Password
├── dashboard-httproute.yaml        # Gateway API route for Dashboard
├── rbac.yaml                       # ServiceAccount + RBAC for pipelines
└── README.md                       # This file
```

### application.yaml
ArgoCD Application that:
- Deploys operator manifest (operator-install.yaml)
- Applies TektonConfig and other resources
- Sync wave 8 (after Harbor)
- Auto-sync with prune/selfHeal

### tektonconfig.yaml
CustomResource defining:
- Components to install (Pipelines, Triggers, Dashboard)
- Target namespace: innoventions
- Performance tuning (HA disabled for homelab)
- Retention policy (7 days)

### operator-install.yaml
Static YAML from: https://github.com/tektoncd/operator/releases/v0.79.1
- CRDs for TektonConfig, TektonPipeline, etc.
- Operator Deployment
- RBAC (ClusterRole, ServiceAccount)

### externalsecret.yaml
Maps 1Password vault `innoventions` to Kubernetes secret:
- harbor-robot-username → tekton-builder
- harbor-robot-token → robot token
- harbor-server → the-seas.local.mk-labs.cloud
- Creates docker-registry secret for Kaniko

### dashboard-httproute.yaml
Gateway API HTTPRoute:
- Hostname: mission-space.local.mk-labs.cloud
- Backend: tekton-dashboard service (port 9097)
- TLS via cert-manager

### rbac.yaml
ServiceAccount `tekton-pipeline` with permissions:
- Create/delete pods and PVCs
- Get/list/watch configmaps and secrets
- Required for pipeline execution

---

## Reusable Pipeline Library

Tasks and Pipelines live in `cluster/tekton/` for reuse across the homelab:

**cluster/tekton/tasks/**
- git-clone.yaml - Clone repository from Gitea
- kaniko-build.yaml - Build + push container image

**cluster/tekton/pipelines/**
- container-build.yaml - Complete build workflow

**cluster/tekton/triggers/**
- eventlistener.yaml - Webhook receiver
- triggerbinding.yaml - Extract Git event data
- triggertemplate.yaml - Instantiate PipelineRun

---

## Access & Usage

### Dashboard Access

**URL:** https://mission-space.local.mk-labs.cloud

**Features:**
- View PipelineRuns and TaskRuns
- Monitor execution logs
- Manually trigger pipelines
- View task/pipeline definitions

### CLI Access (tkn)

Install Tekton CLI:
```bash
brew install tektoncd-cli  # macOS
# or download from https://github.com/tektoncd/cli
```

**Common commands:**
```bash
# List pipelines
tkn pipeline list -n innoventions

# Run a pipeline manually
tkn pipeline start container-build \
  --param git-url=https://gitea.mk-labs.cloud/rblundon/my-app.git \
  --param git-revision=main \
  --param image-name=the-seas.local.mk-labs.cloud/library/my-app \
  --param image-tag=v1.0 \
  --workspace name=shared-workspace,claimName=my-workspace \
  --showlog \
  --namespace innoventions

# Watch logs
tkn pipelinerun logs -f -n innoventions

# List recent runs
tkn pipelinerun list -n innoventions
```

---

## Testing & Verification

### Health Checks

**Operator status:**
```bash
kubectl get tektonconfig -n innoventions
kubectl get pods -n innoventions
```

**Expected pods:**
- tekton-operator-*
- tekton-pipelines-controller-*
- tekton-pipelines-webhook-*
- tekton-triggers-controller-*
- tekton-triggers-webhook-*
- tekton-dashboard-*

**ArgoCD sync:**
```bash
kubectl get applications -n argocd innoventions
```

### Manual Pipeline Test

1. Create test workspace:
```bash
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-workspace
  namespace: innoventions
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: nfs-emporium
EOF
```

2. Run container-build pipeline (see CLI commands above)

3. Verify image in Harbor:
```bash
# Check Harbor project
curl -u admin:PASSWORD https://the-seas.local.mk-labs.cloud/api/v2.0/projects/library/repositories
```

### Webhook Integration Test

1. Configure Gitea webhook:
   - URL: http://el-gitea-listener.innoventions.svc.cluster.local:8080
   - Secret: (from 1Password vault `innoventions`)
   - Events: Push

2. Push to repository:
```bash
git commit --allow-empty -m "Test webhook trigger"
git push
```

3. Watch for automatic PipelineRun:
```bash
kubectl get pipelinerun -n innoventions -w
```

---

## Troubleshooting

### Operator Not Deploying Components

**Check TektonConfig status:**
```bash
kubectl describe tektonconfig config -n innoventions
```

**Check operator logs:**
```bash
kubectl logs -l app=tekton-operator -n innoventions
```

### Harbor Push Fails

**Verify secret exists:**
```bash
kubectl get secret harbor-credentials -n innoventions -o yaml
```

**Test credentials manually:**
```bash
# Extract credentials
kubectl get secret harbor-credentials -n innoventions -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d

# Test login
docker login the-seas.local.mk-labs.cloud -u robot\$tekton-builder -p TOKEN
```

### Webhook Not Triggering

**Check EventListener logs:**
```bash
kubectl logs -l eventlistener=gitea-listener -n innoventions -f
```

**Verify webhook secret matches:**
- Gitea webhook secret
- 1Password vault `innoventions` field `gitea-webhook-secret`

**Check webhook delivery in Gitea UI:**
- Repository → Settings → Webhooks → Recent Deliveries

### Dashboard Not Accessible

**Check HTTPRoute:**
```bash
kubectl get httproute -n innoventions
kubectl describe httproute tekton-dashboard -n innoventions
```

**Check certificate:**
```bash
kubectl get certificate -n innoventions
```

**Check DNS resolution:**
```bash
dig mission-space.local.mk-labs.cloud
```

---

## Dependencies

**Required (must be deployed first):**
- Wave 1: ExternalSecrets Operator
- Wave 2: 1Password Connect
- Wave 3: cert-manager
- Wave 4: Gateway API
- Wave 5: NFS CSI Driver
- Wave 7: Harbor container registry

**Integrates with:**
- Gitea (source control)
- Harbor (image registry)
- ArgoCD (future: Image Updater in Phase 3)

---

## Future Enhancements (Phase 3+)

**Tekton Chains (Image Signing):**
- Enable in TektonConfig
- Sign images with Cosign
- Store signatures in Rekor transparency log
- Verify signatures in ArgoCD

**Advanced Tasks:**
- Code linting (golangci-lint, flake8)
- Unit test execution
- Security scanning (Trivy, Grype)
- Helm chart publishing
- Multi-arch builds

**Notifications:**
- Slack integration for build status
- Email alerts on failures
- GitHub/Gitea commit status updates

**Monitoring:**
- Prometheus metrics from Tekton controllers
- Grafana dashboards for pipeline analytics
- Alert rules for failed builds

---

## References

**Tekton Documentation:**
- Operator: https://github.com/tektoncd/operator
- Pipelines: https://tekton.dev/docs/pipelines/
- Triggers: https://tekton.dev/docs/triggers/
- Dashboard: https://github.com/tektoncd/dashboard

**Task Catalog:**
- Official: https://github.com/tektoncd/catalog
- Community: https://hub.tekton.dev/

**Integration Guides:**
- Kaniko: https://github.com/GoogleContainerTools/kaniko
- Harbor: https://goharbor.io/docs/
- Gateway API: https://gateway-api.sigs.k8s.io/

---

## Maintenance

**Upgrade Procedure:**
1. Check operator release notes for breaking changes
2. Update operator-install.yaml to new version
3. Commit and push to trigger ArgoCD sync
4. Operator will manage component upgrades
5. Monitor pod restarts and logs

**Cleanup Old PipelineRuns:**
Automatic via TektonConfig retention policy (7 days).

Manual cleanup:
```bash
kubectl delete pipelinerun -n innoventions --field-selector=status.conditions[0].reason=Succeeded
```

**Backup:**
Export task/pipeline definitions:
```bash
kubectl get tasks,pipelines,triggers -n innoventions -o yaml > tekton-backup.yaml
```

---

**Deployed:** June 6, 2026  
**Owner:** Platform Team  
**Contact:** Rocket Raccoon (CI/CD Specialist)  
**Status:** ✅ OPERATIONAL