homelab/cluster/tekton/pipelines/README.md

# Tekton Pipeline Library

Reusable Tekton Pipelines for common workflows across the mk-labs homelab.

---

## Overview

Pipelines orchestrate multiple Tasks into a complete workflow. They define the sequence, data flow, and conditions for task execution.

**Key characteristics:**
- Compose multiple tasks into workflows
- Define parameters passed to tasks
- Manage workspaces shared between tasks
- Support parallel and sequential execution
- Can be triggered manually or via webhooks

---

## Available Pipelines

### container-build.yaml

Complete container image build workflow from Git source to Harbor registry.

**What it does:**
1. Clone Git repository (git-clone task)
2. Build container image with Kaniko (kaniko-build task)
3. Push image to Harbor registry

**Parameters:**
- `git-url` - Git repository URL
- `git-revision` - Branch, tag, or commit (default: main)
- `image-name` - Target image name (without tag)
- `image-tag` - Image tag (default: latest)

**Workspaces:**
- `shared-workspace` - Shared between clone and build steps

**Usage (manual):**
```bash
tkn pipeline start container-build \
  --param git-url=https://gitea.mk-labs.cloud/rblundon/my-app.git \
  --param git-revision=main \
  --param image-name=the-seas.local.mk-labs.cloud/library/my-app \
  --param image-tag=v1.0.0 \
  --workspace name=shared-workspace,volumeClaimTemplateFile=workspace-template.yaml \
  --showlog \
  --namespace innoventions
```

**Usage (from another pipeline):**
```yaml
tasks:
  - name: build-my-app
    taskRef:
      name: container-build
    params:
      - name: git-url
        value: $(params.repo-url)
      - name: image-tag
        value: $(params.version)
```

---

## Pipeline Execution Model

**Sequential tasks:**
```yaml
tasks:
  - name: first
    taskRef:
      name: some-task
  - name: second
    runAfter:
      - first
    taskRef:
      name: another-task
```

**Parallel tasks:**
```yaml
tasks:
  - name: lint
    taskRef:
      name: golangci-lint
  - name: test
    taskRef:
      name: go-test
  # Both run in parallel
```

**Conditional execution:**
```yaml
tasks:
  - name: deploy-to-prod
    when:
      - input: "$(params.environment)"
        operator: in
        values: ["production"]
    taskRef:
      name: deploy
```

---

## Creating New Pipelines

Basic pipeline structure:

```yaml
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: my-pipeline
  namespace: innoventions
spec:
  params:
    - name: my-param
      description: Parameter description
      default: default-value
  workspaces:
    - name: shared-data
      description: Workspace shared between tasks
  tasks:
    - name: step1
      taskRef:
        name: some-task
      params:
        - name: task-param
          value: $(params.my-param)
      workspaces:
        - name: output
          workspace: shared-data
    
    - name: step2
      runAfter:
        - step1
      taskRef:
        name: another-task
      workspaces:
        - name: source
          workspace: shared-data
```

**Best practices:**
- Keep pipelines focused on a single workflow
- Use meaningful task names that describe the action
- Document parameters and expected values
- Consider workspace size requirements
- Test manually before setting up webhooks

---

## Workspace Management

Pipelines require workspace storage for sharing data between tasks.

**Option 1: VolumeClaimTemplate (recommended for webhooks)**
```yaml
workspaces:
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: nfs-emporium
```

**Option 2: Existing PVC (for manual runs)**
```yaml
workspaces:
  - name: shared-workspace
    persistentVolumeClaim:
      claimName: my-workspace-pvc
```

**Option 3: EmptyDir (for ephemeral data)**
```yaml
workspaces:
  - name: shared-workspace
    emptyDir: {}
```

---

## Monitoring Pipeline Runs

**List recent runs:**
```bash
tkn pipelinerun list -n innoventions
```

**Get run details:**
```bash
tkn pipelinerun describe BUILD-NAME -n innoventions
```

**Follow logs:**
```bash
tkn pipelinerun logs BUILD-NAME -f -n innoventions
```

**Delete old runs:**
```bash
kubectl delete pipelinerun -n innoventions --field-selector=status.conditions[0].reason=Succeeded
```

---

## Pipeline Templates

Create parameterized templates for common patterns:

```yaml
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: template-container-build
  namespace: innoventions
  labels:
    type: template
spec:
  params:
    - name: git-url
    - name: git-revision
      default: main
    - name: harbor-project
      default: library
    - name: app-name
    - name: version
  tasks:
    - name: clone-source
      taskRef:
        name: git-clone
      params:
        - name: url
          value: $(params.git-url)
        - name: revision
          value: $(params.git-revision)
      workspaces:
        - name: output
          workspace: workspace
    
    - name: build-and-push
      runAfter:
        - clone-source
      taskRef:
        name: kaniko-build
      params:
        - name: image
          value: the-seas.local.mk-labs.cloud/$(params.harbor-project)/$(params.app-name):$(params.version)
      workspaces:
        - name: source
          workspace: workspace
  
  workspaces:
    - name: workspace
```

---

## Integration with Triggers

Pipelines can be automatically triggered by webhooks via Tekton Triggers.

See: `cluster/tekton/triggers/` for EventListener, TriggerBinding, and TriggerTemplate configurations.

**Flow:**
1. Git push to Gitea
2. Gitea sends webhook to EventListener
3. TriggerBinding extracts event data (repo URL, commit SHA)
4. TriggerTemplate creates PipelineRun with extracted parameters
5. Pipeline executes automatically

---

## Future Pipeline Ideas

**Multi-stage builds:**
- Build → Test → Scan → Push
- Conditional deployment based on test results

**Multi-arch builds:**
- Build for amd64 and arm64
- Create multi-arch manifest

**Monorepo support:**
- Detect changed directories
- Build only affected services

**Advanced workflows:**
- Helm chart linting and publishing
- Documentation generation
- Release artifact creation
- Changelog generation

---

## Troubleshooting

**Pipeline fails to start:**
- Check task references exist
- Verify workspace configuration
- Check RBAC permissions

**Tasks timeout:**
- Increase timeout in pipeline spec
- Check for stuck processes in task logs

**Workspace errors:**
- Verify StorageClass exists
- Check NFS CSI driver status
- Ensure PVC can be created

**Image push fails:**
- Verify harbor-credentials secret exists
- Test credentials manually with docker login
- Check Harbor project permissions

---

## References

**Tekton Pipeline Docs:**
- Pipelines: https://tekton.dev/docs/pipelines/pipelines/
- PipelineRuns: https://tekton.dev/docs/pipelines/pipelineruns/
- Workspaces: https://tekton.dev/docs/pipelines/workspaces/

**Examples:**
- Official examples: https://github.com/tektoncd/pipeline/tree/main/examples
- Community examples: https://github.com/tektoncd/catalog

---

**Location:** cluster/tekton/pipelines/  
**Deployed to:** innoventions namespace  
**Owner:** Platform Team