homelab/cluster/tekton/pipelines/README.md

Tekton Pipeline Library

Reusable Tekton Pipelines for common workflows across the mk-labs homelab.

---

Overview

Pipelines orchestrate multiple Tasks into a complete workflow. They define the sequence, data flow, and conditions for task execution.

Key characteristics:

• Compose multiple tasks into workflows
• Define parameters passed to tasks
• Manage workspaces shared between tasks
• Support parallel and sequential execution
• Can be triggered manually or via webhooks

---

Available Pipelines

container-build.yaml

Complete container image build workflow from Git source to Harbor registry.

What it does:

1. Clone Git repository (git-clone task)
2. Build container image with Kaniko (kaniko-build task)
3. Push image to Harbor registry

Parameters:

• git-url - Git repository URL
• git-revision - Branch, tag, or commit (default: main)
• image-name - Target image name (without tag)
• image-tag - Image tag (default: latest)

Workspaces:

• shared-workspace - Shared between clone and build steps

Usage (manual):

tkn pipeline start container-build \
  --param git-url=https://gitea.mk-labs.cloud/rblundon/my-app.git \
  --param git-revision=main \
  --param image-name=the-seas.local.mk-labs.cloud/library/my-app \
  --param image-tag=v1.0.0 \
  --workspace name=shared-workspace,volumeClaimTemplateFile=workspace-template.yaml \
  --showlog \
  --namespace innoventions

Usage (from another pipeline):

tasks:
  - name: build-my-app
    taskRef:
      name: container-build
    params:
      - name: git-url
        value: $(params.repo-url)
      - name: image-tag
        value: $(params.version)

---

Pipeline Execution Model

Sequential tasks:

tasks:
  - name: first
    taskRef:
      name: some-task
  - name: second
    runAfter:
      - first
    taskRef:
      name: another-task

Parallel tasks:

tasks:
  - name: lint
    taskRef:
      name: golangci-lint
  - name: test
    taskRef:
      name: go-test
  # Both run in parallel

Conditional execution:

tasks:
  - name: deploy-to-prod
    when:
      - input: "$(params.environment)"
        operator: in
        values: ["production"]
    taskRef:
      name: deploy

---

Creating New Pipelines

Basic pipeline structure:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: my-pipeline
  namespace: innoventions
spec:
  params:
    - name: my-param
      description: Parameter description
      default: default-value
  workspaces:
    - name: shared-data
      description: Workspace shared between tasks
  tasks:
    - name: step1
      taskRef:
        name: some-task
      params:
        - name: task-param
          value: $(params.my-param)
      workspaces:
        - name: output
          workspace: shared-data
    
    - name: step2
      runAfter:
        - step1
      taskRef:
        name: another-task
      workspaces:
        - name: source
          workspace: shared-data

Best practices:

• Keep pipelines focused on a single workflow
• Use meaningful task names that describe the action
• Document parameters and expected values
• Consider workspace size requirements
• Test manually before setting up webhooks

---

Workspace Management

Pipelines require workspace storage for sharing data between tasks.

Option 1: VolumeClaimTemplate (recommended for webhooks)

workspaces:
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: nfs-emporium

Option 2: Existing PVC (for manual runs)

workspaces:
  - name: shared-workspace
    persistentVolumeClaim:
      claimName: my-workspace-pvc

Option 3: EmptyDir (for ephemeral data)

workspaces:
  - name: shared-workspace
    emptyDir: {}

---

Monitoring Pipeline Runs

List recent runs:

tkn pipelinerun list -n innoventions

Get run details:

tkn pipelinerun describe BUILD-NAME -n innoventions

Follow logs:

tkn pipelinerun logs BUILD-NAME -f -n innoventions

Delete old runs:

kubectl delete pipelinerun -n innoventions --field-selector=status.conditions[0].reason=Succeeded

---

Pipeline Templates

Create parameterized templates for common patterns:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: template-container-build
  namespace: innoventions
  labels:
    type: template
spec:
  params:
    - name: git-url
    - name: git-revision
      default: main
    - name: harbor-project
      default: library
    - name: app-name
    - name: version
  tasks:
    - name: clone-source
      taskRef:
        name: git-clone
      params:
        - name: url
          value: $(params.git-url)
        - name: revision
          value: $(params.git-revision)
      workspaces:
        - name: output
          workspace: workspace
    
    - name: build-and-push
      runAfter:
        - clone-source
      taskRef:
        name: kaniko-build
      params:
        - name: image
          value: the-seas.local.mk-labs.cloud/$(params.harbor-project)/$(params.app-name):$(params.version)
      workspaces:
        - name: source
          workspace: workspace
  
  workspaces:
    - name: workspace

---

Integration with Triggers

Pipelines can be automatically triggered by webhooks via Tekton Triggers.

See: cluster/tekton/triggers/ for EventListener, TriggerBinding, and TriggerTemplate configurations.

Flow:

1. Git push to Gitea
2. Gitea sends webhook to EventListener
3. TriggerBinding extracts event data (repo URL, commit SHA)
4. TriggerTemplate creates PipelineRun with extracted parameters
5. Pipeline executes automatically

---

Future Pipeline Ideas

Multi-stage builds:

• Build → Test → Scan → Push
• Conditional deployment based on test results

Multi-arch builds:

• Build for amd64 and arm64
• Create multi-arch manifest

Monorepo support:

• Detect changed directories
• Build only affected services

Advanced workflows:

• Helm chart linting and publishing
• Documentation generation
• Release artifact creation
• Changelog generation

---

Troubleshooting

Pipeline fails to start:

• Check task references exist
• Verify workspace configuration
• Check RBAC permissions

Tasks timeout:

• Increase timeout in pipeline spec
• Check for stuck processes in task logs

Workspace errors:

• Verify StorageClass exists
• Check NFS CSI driver status
• Ensure PVC can be created

Image push fails:

• Verify harbor-credentials secret exists
• Test credentials manually with docker login
• Check Harbor project permissions

---

References

Tekton Pipeline Docs:

• Pipelines: https://tekton.dev/docs/pipelines/pipelines/
• PipelineRuns: https://tekton.dev/docs/pipelines/pipelineruns/
• Workspaces: https://tekton.dev/docs/pipelines/workspaces/

Examples:

• Official examples: https://github.com/tektoncd/pipeline/tree/main/examples
• Community examples: https://github.com/tektoncd/catalog

---

Location: cluster/tekton/pipelines/
Deployed to: innoventions namespace
Owner: Platform Team