rblundon/homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Squashed 'acm-hub-bootstrap/' content from commit aea23ee

Browse Source 
git-subtree-dir: acm-hub-bootstrap
git-subtree-split: aea23ee88fafd5cab0e2d9189f872159c5f16b8e
This commit is contained in:
Ryan Blundon
2025-03-30 15:44:27 -05:00
commit b9b83d1fea
53 changed files with 7445 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
sealed-secrets-key.yaml
slack-token-key.yaml
*-private.yaml
eso-token-*.yaml

38
README.md Normal file
View File

@@ -0,0 +1,38 @@
### Introduction
This repository is used to bootstrap the RHACM Hub cluster as well as hold the ACM configuration
I use in my homelab environment. It is referenced by the [cluster-config](https://github.com/gnunn-gitops/cluster-config) repo for the `local.hub` cluster's ACM applications (Hub, Policies and Observerability).
From the perspective of bootstrapping the Hub there are a couple of different ways to go. You could
manually install OpenShift GitOps and have it bootstrap everything on the Hub including RHACM. However while
I did consider that approach I'm using ACM to bootstrap my managed clusters and I wanted to use the exact same workflow
if I could for the Hub. This is because I wanted to avoid having to create a tweaked workflow just for the Hub.
### Bootstrap Process
The bootstrap process, encapsulated in the `bootstrap.sh` script, is as follows:
1. Install the RHACM operator and wait for it to complete
2. Install the MultiClusterHub custom resource and wait for it to be in the Running phase
3. Deploy all of the RHACM policies including the policy that deploys the OpenShift
GitOps operator with the bootstrap application
4. Label the Hub cluster, `local-cluster`, with the label `gitops=local.hub`. This trigger the
GitOps policy we deployed in step #3 to install OpenShift GitOps on the Hub cluster and
deploy the bootstrap application pointing to the repo and path needed for this cluster, i.e.
`local.home`
At this point the GitOps operator deploys everything the Hub cluster requires including storage, operators, etc
as well as other ACM components. In particular, the components that the script deployed (Hub + Policies) have
Argo applications that will take over managing these.
With the script bootstrapping the Hub because a one command affair. Once the command completes successfully
I walk away for 20-30 minutes while everything gets sorted out.
### Side-note on LVM Operator
If you are running a SNO cluster like I am and using the LVM Operator make sure that whatever device you are
using for storage with it is free. For example, if you are doing a reinstall of the hub (or other cluster), after
the cluster is provisioned but before you provision Day 2 ops ssh to the cluster and run `lsblk` to make sure
the device is empty with no partitions.
A reminder for future me. :)

45
bootstrap.sh Executable file
View File

@@ -0,0 +1,45 @@
#!/bin/bash
LANG=C
SLEEP_SECONDS=30
echo ""
echo "Installing RHACM Operator."
kustomize build github.com/redhat-cop/gitops-catalog/advanced-cluster-management/operator/overlays/release-2.9 | oc apply -f -
echo "Pause $SLEEP_SECONDS seconds for the creation of the rhacm-operator..."
sleep $SLEEP_SECONDS
echo "Waiting for operator to start"
until oc get deployment multiclusterhub-operator -n open-cluster-management
do
sleep 10;
done
echo "Installing the MultiClusterHub"
kustomize build github.com/redhat-cop/gitops-catalog/advanced-cluster-management/instance/base | oc apply -f -
echo "Waiting for hub to be installed"
until [[ $(oc get multiclusterhub multiclusterhub -n open-cluster-management -o jsonpath='{.status.phase}') == 'Running' ]]
do
echo "Waiting for hub, current status:"
oc get multiclusterhub multiclusterhub -n open-cluster-management
sleep 10
done
echo "Installing policies and initial secrets"
kustomize build bootstrap/secrets/base | oc apply -f -
kustomize build bootstrap/policies/overlays/default --enable-alpha-plugins | oc apply -f -
echo "Labeling cluster with 'gitops: local.home'"
oc label managedcluster local-cluster gitops=local.hub --overwrite=true
echo "Check policy compliance with the following command:"
echo " oc get policy -A"
echo "Once GitOps configuration is complete you may need to run the following to fix Unknown status\n"
echo " oc delete secret local-cluster-import -n local-cluster\n"

1
bootstrap/install-gitops/README.md Normal file
View File

@@ -0,0 +1 @@
Experimental, not used at the momentgit add

58
bootstrap/install-gitops/setup.sh Normal file
View File

@@ -0,0 +1,58 @@
#!/bin/bash
LANG=C
SLEEP_SECONDS=45
echo ""
echo "Installing GitOps Operator."
kustomize build ../../components/policies/gitops/base/manifests/gitops-subscription | oc apply -f -
echo "Pause $SLEEP_SECONDS seconds for the creation of the gitops-operator..."
sleep $SLEEP_SECONDS
echo "Waiting for operator to start"
until oc get deployment gitops-operator-controller-manager -n openshift-operators
do
sleep 5;
done
echo "Waiting for openshift-gitops namespace to be created"
until oc get ns openshift-gitops
do
sleep 5;
done
echo "Waiting for deployments to start"
until oc get deployment cluster -n openshift-gitops
do
sleep 5;
done
echo "Waiting for all pods to be created"
deployments=(cluster kam openshift-gitops-applicationset-controller openshift-gitops-redis openshift-gitops-repo-server openshift-gitops-server)
for i in "${deployments[@]}";
do
echo "Waiting for deployment $i";
oc rollout status deployment $i -n openshift-gitops
done
echo "Apply overlay to override default instance"
# echo "Create default instance of gitops operator"
kustomize build ../../components/policies/gitops/base/manifests/gitops-instance/base |
yq -y ".spec.repo.env |= map(select(.name == \"INFRASTRUCTURE_ID\").value=\"$INFRASTRUCTURE_ID\")" |
yq -y ".spec.repo.env |= map(select(.name == \"CLUSTER_ID\").value=\"$CLUSTER_ID\")" |
yq -y ".spec.repo.env |= map(select(.name == \"CLUSTER_NAME\").value=\"$CLUSTER_NAME\")" |
yq -y ".spec.repo.env |= map(select(.name == \"SUB_DOMAIN\").value=\"$SUB_DOMAIN\")" |
oc apply -f -
sleep 10
echo "Waiting for all pods to redeploy"
deployments=(cluster kam openshift-gitops-applicationset-controller openshift-gitops-redis openshift-gitops-repo-server openshift-gitops-server)
for i in "${deployments[@]}";
do
echo "Waiting for deployment $i";
oc rollout status deployment $i -n openshift-gitops
done
echo "GitOps Operator ready"

8
bootstrap/policies/overlays/default/kustomization.yaml Normal file
View File

@@ -0,0 +1,8 @@
namespace: acm-policies
namespace: acm-policies
resources:
# - ../../../../components/policies/compliance/base
- ../../../../components/policies/cert-expiration/base
- ../../../../components/policies/gitops/base

1
bootstrap/secrets/base/README.md Normal file
View File

@@ -0,0 +1 @@
Bootstrap tokens for ESO to access Doppler projects, note that these secrets are not in git.

4
bootstrap/secrets/base/hub-secrets-namespace.yaml Normal file
View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: hub-secrets

10
bootstrap/secrets/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,10 @@
resources:
- namespace.yaml
- hub-secrets-namespace.yaml
# Note these secrets are not stored in git and only applied during bootstrap process
# They are the tokens for accessing the Doppler secrets SaaS service where secrets are stored
- eso-token-cluster-push-secret.yaml
- eso-token-cluster-home-secret.yaml
- eso-token-cluster-hub-secret.yaml
- eso-token-cluster-rhdp-secret.yaml
- eso-token-cluster-microshift-secret.yaml

8
bootstrap/secrets/base/namespace.yaml Normal file
View File

@@ -0,0 +1,8 @@
apiVersion: v1
kind: Namespace
metadata:
annotations:
openshift.io/description: ACM Policies
openshift.io/display-name: ACM Policies
labels:
name: acm-policies

12
components/apps/acm/overlays/hub/kustomization.yaml Normal file
View File

@@ -0,0 +1,12 @@
resources:
- github.com/redhat-cop/gitops-catalog/advanced-cluster-management/operator/overlays/release-2.12
- github.com/redhat-cop/gitops-catalog/advanced-cluster-management/instance/base
patches:
- patch: |-
- op: replace
path: /spec/channel
value: 'release-2.13'
target:
kind: Subscription
name: advanced-cluster-management

31
components/apps/acm/overlays/observability/custom-metrics-allowlist.yaml Normal file
View File

@@ -0,0 +1,31 @@
kind: ConfigMap
apiVersion: v1
metadata:
name: observability-metrics-custom-allowlist
namespace: open-cluster-management-observability
data:
metrics_list.yaml: |
names:
- argocd_cluster_info
- argocd-server-metrics
- argocd_app_info
- argocd_app_sync_total
- argocd_app_reconcile_count
- argocd_app_reconcile_bucket
- argocd_app_k8s_request_total
- argocd_kubectl_exec_pending
- argocd-metrics
- argocd_cluster_api_resource_objects
- argocd_cluster_api_resources
- argocd_cluster_events_total
- argocd_git_request_total
- argocd_git_request_duration_seconds_bucket
- argocd-repo-server
- argocd_redis_request_total
- process_start_time_seconds
- workqueue_depth
- go_memstats_heap_alloc_bytes
- process_cpu_seconds_total
- go_goroutines
- go_gc_duration_seconds
- grpc_server_handled_total

4212
components/apps/acm/overlays/observability/gitops-dashboard.json Normal file
View File

File diff suppressed because it is too large Load Diff

46
components/apps/acm/overlays/observability/kustomization.yaml Normal file
View File

@@ -0,0 +1,46 @@
namespace: open-cluster-management-observability
generatorOptions:
disableNameSuffixHash: true
labels:
grafana-custom-dashboard: "true"
resources:
- github.com/redhat-cop/gitops-catalog/advanced-cluster-management/instance/observability
- custom-metrics-allowlist.yaml
configMapGenerator:
- name: gitops-dashboard
files:
- gitops.json=gitops-dashboard.json
patches:
- patch: |-
- op: replace
path: /spec/storageConfig/storageClass
value: lvms-vg1
- op: replace
path: /spec/observabilityAddonSpec/interval
value: 60
- op: add
path: /spec/advanced
value:
alertmanager:
replicas: 1
grafana:
replicas: 1
observatoriumAPI:
replicas: 1
query:
replicas: 1
queryFrontend:
replicas: 1
queryFrontendMemcached:
replicas: 1
rbacQueryProxy:
replicas: 1
receive:
replicas: 1
target:
kind: MultiClusterObservability
name: observability

3
components/clusters/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,3 @@
resources:
- local-home.yaml
- local-hub.yaml

18
components/clusters/base/local-home.yaml Normal file
View File

@@ -0,0 +1,18 @@
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
annotations:
open-cluster-management/created-via: other
labels:
cloud: Other
cluster.open-cluster-management.io/clusterset: dev
gitops: local.home
color: "0066CC"
name: home
vendor: OpenShift
name: home
spec:
hubAcceptsClient: true
leaseDurationSeconds: 60
managedClusterClientConfigs:
- url: https://api.home.ocplab.com:6443

20
components/clusters/base/local-hub.yaml Normal file
View File

@@ -0,0 +1,20 @@
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
annotations:
open-cluster-management/created-via: other
labels:
cloud: Other
cluster.open-cluster-management.io/clusterset: default
gitops: local.hub
color: "3E8635"
local-cluster: "true"
name: local-cluster
velero.io/exclude-from-backup: "true"
vendor: OpenShift
name: local-cluster
spec:
hubAcceptsClient: true
leaseDurationSeconds: 60
managedClusterClientConfigs:
- url: https://api.hub.ocplab.com:6443

2
components/policies/cert-expiration/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,2 @@
resources:
- policy-cert-expiration.yaml

50
components/policies/cert-expiration/base/policy-cert-expiration.yaml Normal file
View File

@@ -0,0 +1,50 @@
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: certificate-expiration
annotations:
policy.open-cluster-management.io/categories: SC System and Communications Protection
policy.open-cluster-management.io/standards: NIST SP 800-53
policy.open-cluster-management.io/controls: SC-8 Transmission Confidentiality and Integrity
spec:
disabled: false
remediationAction: inform
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: CertificatePolicy
metadata:
name: policy-certificate
spec:
namespaceSelector:
include:
- default
exclude:
- kube-*
remediationAction: inform
severity: low
minimumDuration: 300h
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: certificate-expiration-placement
spec:
clusterSelector:
matchExpressions: []
clusterConditions:
- status: "True"
type: ManagedClusterConditionAvailable
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: certificate-expiration-placement
placementRef:
name: certificate-expiration-placement
apiGroup: apps.open-cluster-management.io
kind: PlacementRule
subjects:
- name: certificate-expiration
apiGroup: policy.open-cluster-management.io
kind: Policy

2
components/policies/compliance/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,2 @@
resources:
- scan-nist-800-53-policy.yaml

55
components/policies/compliance/base/scan-nist-800-53-policy.yaml Normal file
View File

@@ -0,0 +1,55 @@
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: scan-nist-800-53
annotations:
policy.open-cluster-management.io/standards: NIST-CSF
policy.open-cluster-management.io/categories: PR.IP Information Protection Processes and Procedures
policy.open-cluster-management.io/controls: PR.IP-1 Baseline Configuration
spec:
remediationAction: enforce
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: scan-nist-800-53
spec:
remediationAction: inform
severity: high
object-templates:
- complianceType: mustnothave
objectDefinition:
apiVersion: compliance.openshift.io/v1alpha1
kind: ComplianceCheckResult
metadata:
namespace: openshift-compliance
labels:
compliance.openshift.io/check-status: FAIL
compliance.openshift.io/suite: cis-compliance
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-scan-nist-800-53
placementRef:
name: placement-scan-nist-800-53
kind: PlacementRule
apiGroup: apps.open-cluster-management.io
subjects:
- name: scan-nist-800-53
kind: Policy
apiGroup: policy.open-cluster-management.io
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: placement-scan-nist-800-53
spec:
clusterConditions:
- status: 'True'
type: ManagedClusterConditionAvailable
clusterSelector:
matchExpressions:
- {key: vendor, operator: In, values: ["OpenShift"]}

7
components/policies/gatekeeper/config-policy-namespace.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: v1
kind: Namespace
metadata:
annotations:
openshift.io/description: ACM Configuration Policies
openshift.io/display-name: ACM Configuration Policies
name: acm-config-policies

131
components/policies/gatekeeper/gatekeeper-exclude-namespaces.yaml Normal file
View File

@@ -0,0 +1,131 @@
# Sample policy to configure gatekeeper to exclude namespaces from certain processes for all constraints in the cluster
# See: https://github.com/open-policy-agent/gatekeeper/tree/release-3.3#exempting-namespaces-from-gatekeeper
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: policy-gatekeeper-config-exclude-namespaces
annotations:
policy.open-cluster-management.io/standards: NIST SP 800-53
policy.open-cluster-management.io/categories: CM Configuration Management
policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
remediationAction: enforce
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-config-exclude-namespaces
spec:
remediationAction: enforce # will be overridden by remediationAction in parent policy
severity: low
object-templates:
- complianceType: mustonlyhave
objectDefinition:
apiVersion: config.gatekeeper.sh/v1alpha1
kind: Config
metadata:
name: config
namespace: openshift-gatekeeper-system
spec:
match:
- excludedNamespaces:
- hive
- kube-node-lease
- kube-public
- kube-storage-version-migrator-operator
- kube-system
- open-cluster-management
- open-cluster-management-hub
- open-cluster-management-agent
- open-cluster-management-agent-addon
- openshift
- openshift-apiserver
- openshift-apiserver-operator
- openshift-authentication
- openshift-authentication-operator
- openshift-cloud-credential-operator
- openshift-cluster-csi-drivers
- openshift-cluster-machine-approver
- openshift-cluster-node-tuning-operator
- openshift-cluster-samples-operator
- openshift-cluster-storage-operator
- openshift-cluster-version
- openshift-compliance
- openshift-config
- openshift-config-managed
- openshift-config-operator
- openshift-console
- openshift-console-operator
- openshift-console-user-settings
- openshift-controller-manager
- openshift-controller-manager-operator
- openshift-dns
- openshift-dns-operator
- openshift-etcd
- openshift-etcd-operator
- openshift-gatekeeper-operator
- openshift-gatekeeper-system
- openshift-image-registry
- openshift-infra
- openshift-ingress
- openshift-ingress-canary
- openshift-ingress-operator
- openshift-insights
- openshift-kni-infra
- openshift-kube-apiserver
- openshift-kube-apiserver-operator
- openshift-kube-controller-manager
- openshift-kube-controller-manager-operator
- openshift-kube-scheduler
- openshift-kube-scheduler-operator
- openshift-kube-storage-version-migrator
- openshift-kube-storage-version-migrator-operator
- openshift-kubevirt-infra
- openshift-machine-api
- openshift-machine-config-operator
- openshift-marketplace
- openshift-monitoring
- openshift-multus
- openshift-network-diagnostics
- openshift-network-operator
- openshift-node
- openshift-oauth-apiserver
- openshift-openstack-infra
- openshift-operators
- openshift-operator-lifecycle-manager
- openshift-ovirt-infra
- openshift-ovn-kubernetes
- openshift-sdn
- openshift-service-ca
- openshift-service-ca-operator
- openshift-user-workload-monitoring
- openshift-vsphere-infra
processes:
- '*'
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-policy-gatekeeper-config-exclude-namespaces
placementRef:
name: placement-policy-gatekeeper-config-exclude-namespaces
kind: PlacementRule
apiGroup: apps.open-cluster-management.io
subjects:
- name: policy-gatekeeper-config-exclude-namespaces
kind: Policy
apiGroup: policy.open-cluster-management.io
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: placement-policy-gatekeeper-config-exclude-namespaces
spec:
clusterConditions:
- status: "True"
type: ManagedClusterConditionAvailable
clusterSelector:
matchExpressions:
- {key: policy, operator: In, values: ["opa"]}

139
components/policies/gatekeeper/gatekeeper-install-policy.yaml Normal file
View File

@@ -0,0 +1,139 @@
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: policy-gatekeeper-operator
annotations:
policy.open-cluster-management.io/standards: NIST SP 800-53
policy.open-cluster-management.io/categories: CM Configuration Management
policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
remediationAction: enforce
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-operator-ns
spec:
remediationAction: enforce
severity: high
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: v1
kind: Namespace
metadata:
name: openshift-gatekeeper-operator
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-operator-catalog-source
spec:
remediationAction: enforce
severity: high
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: gatekeeper-operator
namespace: openshift-gatekeeper-operator
spec:
displayName: Gatekeeper Operator Upstream
publisher: github.com/font/gatekeeper-operator
sourceType: grpc
image: 'quay.io/gatekeeper/gatekeeper-operator-bundle-index:v0.2.1'
updateStrategy:
registryPoll:
interval: 45m
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-operator-group
spec:
remediationAction: enforce
severity: high
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: gatekeeper-operator
namespace: openshift-gatekeeper-operator
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper-operator-subscription
spec:
remediationAction: enforce
severity: high
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: gatekeeper-operator-sub
namespace: openshift-gatekeeper-operator
spec:
channel: stable
name: gatekeeper-operator
source: gatekeeper-operator
sourceNamespace: openshift-gatekeeper-operator
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: gatekeeper
spec:
remediationAction: enforce
severity: high
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: operator.gatekeeper.sh/v1alpha1
kind: Gatekeeper
metadata:
name: gatekeeper
spec:
audit:
auditChunkSize: 500
logLevel: INFO
replicas: 1
validatingWebhook: Enabled
mutatingWebhook: Disabled
webhook:
emitAdmissionEvents: Enabled
logLevel: INFO
replicas: 2
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-policy-gatekeeper-operator
placementRef:
name: placement-policy-gatekeeper-operator
kind: PlacementRule
apiGroup: apps.open-cluster-management.io
subjects:
- name: policy-gatekeeper-operator
kind: Policy
apiGroup: policy.open-cluster-management.io
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: placement-policy-gatekeeper-operator
spec:
clusterConditions:
- status: "True"
type: ManagedClusterConditionAvailable
clusterSelector:
matchExpressions:
- {key: policy, operator: In, values: ["opa"]}

8
components/policies/gitops/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,8 @@
namespace: acm-policies
resources:
- placement.yaml
- managedclustersetbinding.yaml
generators:
- policy-generator-config.yaml

7
components/policies/gitops/base/managedclustersetbinding.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: cluster.open-cluster-management.io/v1beta2
kind: ManagedClusterSetBinding
metadata:
name: global
namespace: acm-policies
spec:
clusterSet: global

39
components/policies/gitops/base/manifests/eso/base/secret-store.yaml Normal file
View File

@@ -0,0 +1,39 @@
apiVersion: v1
kind: Secret
metadata:
name: eso-token-cluster
namespace: external-secrets
data:
dopplerToken: '{{hub (fromSecret "acm-policies" (printf "eso-token-cluster-%s" .ManagedClusterName) "dopplerToken") hub}}'
---
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
name: doppler-cluster
spec:
provider:
doppler:
auth:
secretRef:
dopplerToken:
name: eso-token-cluster
key: dopplerToken
namespace: external-secrets
conditions:
- namespaceSelector:
matchLabels:
openshift.io/cluster-monitoring: "true"
- namespaces:
- "cert-manager"
- "dev-tools"
- "gitops"
- "openshift-config"
- "openshift-gitops"
- "openshift-monitoring"
- "openshift-pipelines"
- "stackrox"
- "stackrox-secured-cluster-service"
- "tenant-secrets"
- "trusted-profile-analyzer"

4
components/policies/gitops/base/manifests/gitops-alerting/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,4 @@
namespace: openshift-gitops
resources:
- prometheus-rules.yaml

31
components/policies/gitops/base/manifests/gitops-alerting/base/prometheus-rules.yaml Normal file
View File

@@ -0,0 +1,31 @@
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: argocd-health-alerts
annotations:
policy.open-cluster-management.io/disable-templates: "true"
spec:
groups:
- name: ArgoCD
rules:
- alert: ArgoCDHealthAlert
annotations:
message: ArgoCD application {{ $labels.name }} is not healthy
expr: argocd_app_info{namespace="openshift-gitops", health_status!~"Healthy|Suspended|Progressing|Degraded"} > 0
for: 5m
labels:
severity: warning
- alert: ArgoCDDegradedAlert
annotations:
message: ArgoCD application {{ $labels.name }} is degraded
expr: argocd_app_info{namespace="openshift-gitops", health_status="Degraded"} > 0
for: 5m
labels:
severity: critical
- alert: ArgoCDStuckAlert
annotations:
message: ArgoCD application {{ $labels.name }} is stuck in progressing for more than 10m
expr: argocd_app_info{namespace="openshift-gitops", health_status="Progressing"} > 0
for: 10m
labels:
severity: warning

27
components/policies/gitops/base/manifests/gitops-bootstrap/base/bootstrap-app.yaml Normal file
View File

@@ -0,0 +1,27 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cluster-config-bootstrap
namespace: openshift-gitops
labels:
gitops.ownedBy: cluster-config
spec:
destination:
namespace: openshift-gitops
server: https://kubernetes.default.svc
project: bootstrap
source:
path: bootstrap/overlays/{{ fromClusterClaim "gitops" }}
repoURL: https://github.com/gnunn-gitops/cluster-config-pins.git
targetRevision: 'HEAD'
syncPolicy:
automated:
prune: false
selfHeal: false
ignoreDifferences:
- group: argoproj.io
kind: Application
managedFieldsManagers:
- argocd-server
jsonPointers:
- /spec/syncPolicy/automated

16
components/policies/gitops/base/manifests/gitops-bootstrap/base/bootstrap-appproject.yaml Normal file
View File

@@ -0,0 +1,16 @@
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: bootstrap
namespace: openshift-gitops
spec:
clusterResourceWhitelist:
- group: '*'
kind: '*'
description: Project for bootstrap cluster app
destinations:
- namespace: '*'
server: https://kubernetes.default.svc
sourceRepos:
- https://github.com/gnunn-gitops/cluster-config
- https://github.com/gnunn-gitops/cluster-config-pins

3
components/policies/gitops/base/manifests/gitops-bootstrap/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,3 @@
resources:
- bootstrap-appproject.yaml
- bootstrap-app.yaml

389
components/policies/gitops/base/manifests/gitops-instance/base/argocd.yaml Normal file
View File

@@ -0,0 +1,389 @@
apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
name: openshift-gitops
namespace: openshift-gitops
spec:
resourceTrackingMethod: annotation
applicationSet: {}
extraConfig:
exec.enabled: "true"
resource.respectRBAC: "normal"
resource.ignoreResourceUpdatesEnabled: 'true'
resource.compareoptions: |
ignoreAggregatedRoles: true
resource.customizations.ignoreResourceUpdates.external-secrets.io_ExternalSecret: |
jsonPointers:
- /status/refreshTime
resource.customizations.ignoreResourceUpdates.ocs.openshift.io_StorageCluster: |
jsonPointers:
- /status
- /metadata/resourceVersion
resource.customizations.ignoreResourceUpdates.ocs.openshift.io_StorageSystem: |
jsonPointers:
- /status
- /metadata/resourceVersion
resource.customizations.ignoreResourceUpdates.noobaa.io_Noobaa: |
jsonPointers:
- /status
- /metadata/resourceVersion
resource.customizations.ignoreResourceUpdates.noobaa.io_BackingStore: |
jsonPointers:
- /status
- /metadata/resourceVersion
accounts.admin: apiKey, login
ui.cssurl: 'https://gnunn-gitops.github.io/cluster-config/themes/{{ fromClusterClaim "gitops" }}/custom-cluster.css'
kustomizeBuildOptions: "--enable-helm --enable-alpha-plugins"
oidcConfig: |
name: Keycloak
issuer: https://sso.ocplab.com/realms/ocplab
clientID: argocd
clientSecret: $oidc.keycloak.clientSecret
requestedScopes: ["openid", "profile", "email", "groups"]
controller:
extraCommandArgs:
- '--persist-resource-health=false'
resources:
limits:
memory: 4Gi
requests:
cpu: 1000m
memory: 3Gi
monitoring:
enabled: true
repo:
sidecarContainers:
- name: setenv-cmp-plugin
command: [/var/run/argocd/argocd-cmp-server]
env:
- name: KUSTOMIZE_PLUGIN_HOME
value: /etc/kustomize/plugin
- name: INFRASTRUCTURE_ID
value: '{{ (lookup "config.openshift.io/v1" "Infrastructure" "" "cluster").status.infrastructureName }}'
- name: CLUSTER_ID
value: '{{ (lookup "config.openshift.io/v1" "ClusterVersion" "" "version").spec.clusterID }}'
- name: CLUSTER_GITOPS_NAME
value: '{{ fromClusterClaim "gitops" }}'
- name: CLUSTER_NAME
value: '{{ fromClusterClaim "name" }}'
- name: SUB_DOMAIN
value: '{{ (lookup "config.openshift.io/v1" "Ingress" "openshift-ingress" "cluster").spec.domain }}'
- name: BASE_DOMAIN
value: '{{ (lookup "config.openshift.io/v1" "DNS" "" "cluster").spec.baseDomain }}'
- name: COLOR
value: '{{ default "0066CC" (fromClusterClaim "color") }}'
image: quay.io/gnunn/tools:latest
imagePullPolicy: Always
securityContext:
runAsNonRoot: true
volumeMounts:
- mountPath: /var/run/argocd
name: var-files
- mountPath: /home/argocd/cmp-server/plugins
name: plugins
- mountPath: /tmp
name: tmp
- mountPath: /home/argocd/cmp-server/config/plugin.yaml
subPath: plugin.yaml
name: setenv-cmp-plugin
volumes:
- configMap:
name: setenv-cmp-plugin
name: setenv-cmp-plugin
resources:
limits:
cpu: '1'
memory: 1.5Gi
requests:
cpu: 250m
memory: 768Mi
redis:
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
server:
insecure: true
# host: 'openshift-gitops-server-openshift-gitops.{{ (lookup "config.openshift.io/v1" "Ingress" "openshift-ingress" "cluster").spec.domain }}'
route:
enabled: true
tls:
termination: edge
insecureEdgeTerminationPolicy: Redirect
notifications:
enabled: true
resourceActions:
- group: compliance.openshift.io
kind: ComplianceScan
action: |
discovery.lua: |
local actions = {}
local enabled = false
if obj.status ~= nil and obj.status.phase == "DONE" then
enabled = true
end
actions["rescan"] = {["disabled"] = not(enabled)}
return actions
definitions:
- name: rescan
action.lua: |
if obj.metadata.annotations == nil then
obj.metadata.annotations = {}
end
obj.metadata.annotations["compliance.openshift.io/rescan"] = ""
return obj
resourceHealthChecks:
- group: argoproj.io
kind: Application
check: |
hs = {}
hs.status = "Progressing"
hs.message = ""
if obj.status ~= nil then
if obj.status.health ~= nil then
hs.status = obj.status.health.status
hs.message = obj.status.health.message
end
end
return hs
- group: argoproj.io
kind: RolloutManager
check: |
hs = {}
if obj.status ~= nil then
if obj.status.conditions ~= nil then
for _, condition in ipairs(obj.status.conditions) do
hs.message = condition.message
break
end
end
if obj.status.phase ~= nil then
if obj.status.phase == "Failure" then
hs.status = "Degraded"
return hs
elseif obj.status.phase == "Available" then
hs.status = "Healthy"
return hs
elseif obj.status.phase == "Pending" then
hs.status = "Progressing"
return hs
end
end
hs.status = "Progressing"
hs.message = "Waiting for operator to update status"
return hs
end
- group: operators.coreos.com
kind: Subscription
check: |
health_status = {}
if obj.status ~= nil then
if obj.status.conditions ~= nil then
numDegraded = 0
numPending = 0
numSuspended = 0
msg = ""
for i, condition in pairs(obj.status.conditions) do
msg = msg .. i .. ": " .. condition.type .. " | " .. condition.status .. " | " .. (condition.reason and condition.reason or '') .. "\n"
if condition.type == "InstallPlanPending" and condition.status == "True" then
if condition.reason == "RequiresApproval" then
numSuspended = numSuspended + 1
else
numPending = numPending + 1
end
elseif (condition.type == "InstallPlanMissing" and condition.reason ~= "ReferencedInstallPlanNotFound") then
numDegraded = numDegraded + 1
elseif (condition.type == "CatalogSourcesUnhealthy" or condition.type == "InstallPlanFailed") and condition.status == "True" then
numDegraded = numDegraded + 1
elseif (condition.type == "ResolutionFailed" and condition.reason ~= "ConstraintsNotSatisfiable") then
numDegraded = numDegraded + 1
elseif (condition.type == 'ChannelDeprecated' and condition.status == "True") then
numDegraded = numDegraded + 1
end
end
if numDegraded > 0 then
health_status.status = "Degraded"
health_status.message = msg
return health_status
elseif numSuspended > 0 then
health_status.status = "Suspended"
health_status.message = msg
return health_status
elseif numPending > 0 then
health_status.status = "Progressing"
health_status.message = "An install plan for a subscription is pending installation"
return health_status
else
health_status.status = "Healthy"
health_status.message = msg
return health_status
end
end
end
health_status.status = "Progressing"
health_status.message = "An install plan for a subscription is pending installation"
return health_status
- group: operators.coreos.com
kind: InstallPlan
check: |
hs = {}
if obj.status ~= nil then
if obj.status.phase ~= nil then
if obj.status.phase == "Complete" then
hs.status = "Healthy"
hs.message = obj.status.phase
return hs
elseif obj.status.phase == "RequiresApproval" then
hs.status = "Suspended"
hs.message = obj.status.phase
return hs
else
hs.status = "Progressing"
hs.message = obj.status.phase
return hs
end
end
end
hs.status = "Progressing"
hs.message = "Waiting for InstallPlan to complete"
return hs
- group: platform.stackrox.io
kind: Central
check: |
hs = {}
if obj.status ~= nil and obj.status.conditions ~= nil then
for i, condition in ipairs(obj.status.conditions) do
if condition.status == "True" or condition.reason == "InstallSuccessful" or condition.reason == "UpgradeSuccessful" then
hs.status = "Healthy"
hs.message = "Install Successful"
return hs
end
end
end
hs.status = "Progressing"
hs.message = "Waiting for Central to deploy."
return hs
- group: image.openshift.io
kind: ImageStream
check: |
hs = {}
hs.status = "Progressing"
hs.message = ""
if obj.status ~= nil then
if obj.status.tags ~= nil then
numTags = 0
for _ , item in pairs(obj.status.tags) do
numTags = numTags + 1
numItems = 0
if item.tags ~= nil then
for _ , item in pairs(item.tags) do
numItems = numItems + 1
end
if numItems == 0 then
return hs
end
end
end
if numTags > 0 then
hs.status = "Healthy"
hs.message = "ImageStream has tags resolved"
return hs
end
end
end
return hs
- group: build.openshift.io
kind: Build
check: |
hs = {}
if obj.status ~= nil then
if obj.status.phase ~= nil then
if obj.status.phase == "Complete" then
hs.status = "Healthy"
hs.message = obj.status.phase
return hs
end
end
end
hs.status = "Progressing"
hs.message = "Waiting for Build to complete"
return hs
- kind: PersistentVolumeClaim
check: |
hs = {}
if obj.status ~= nil then
if obj.status.phase ~= nil then
if obj.status.phase == "Pending" then
hs.status = "Healthy"
hs.message = obj.status.phase
return hs
end
if obj.status.phase == "Bound" then
hs.status = "Healthy"
hs.message = obj.status.phase
return hs
end
end
end
hs.status = "Progressing"
hs.message = "Waiting for PVC"
return hs
resourceIgnoreDifferences:
resourceIdentifiers:
- group: route.openshift.io
kind: Route
customization:
jsonPointers:
- /status/ingress
- /metadata/annotations
- group: quay.redhat.com
kind: QuayRegistry
customization:
jsonPointers:
- /status/ingress
- group: cluster.open-cluster-management.io
kind: ManagedCluster
customization:
jsonPointers:
- /spec/managedClusterClientConfigs
resourceExclusions: |
- apiGroups:
- tekton.dev
clusters:
- '*'
kinds:
- TaskRun
- PipelineRun
- apiGroups:
- operator.tekton.dev
clusters:
- '*'
kinds:
- TektonAddon
- TektonInstallerSet
- apiGroups:
- compliance.openshift.io
kinds:
- ComplianceCheckResult
- ComplianceRemediation
- apiGroups:
- policy.open-cluster-management.io
kinds:
- ConfigurationPolicy
- apiGroups:
- noobaa.io
kinds:
- NooBaa
- BucketClass
ha:
enabled: false
rbac:
defaultPolicy: 'role:none'
policy: |
p, role:none, *, *, */*, deny
g, system:cluster-admins, role:admin
g, cluster-admins, role:admin
scopes: "[groups]"

3
components/policies/gitops/base/manifests/gitops-instance/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,3 @@
resources:
- setenv-cmp-plugin-cm.yaml
- argocd.yaml

19
components/policies/gitops/base/manifests/gitops-instance/base/setenv-cmp-plugin-cm.yaml Normal file
View File

@@ -0,0 +1,19 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: setenv-cmp-plugin
namespace: openshift-gitops
data:
plugin.yaml: |
apiVersion: argoproj.io/v1alpha1
kind: ConfigManagementPlugin
metadata:
name: setenv-cmp-plugin
spec:
init:
command: [sh, -c, 'echo "Initializing setenv-plugin-cmp..."']
generate:
command:
- sh
- "-c"
- "set -o pipefail && kustomize build --enable-helm --enable-alpha-plugins . | envsub"

2
components/policies/gitops/base/manifests/gitops-notifications/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,2 @@
resources:
- notifications-config.yaml

256
components/policies/gitops/base/manifests/gitops-notifications/base/notifications-cm.yaml Normal file
View File

@@ -0,0 +1,256 @@
# No longer needed with NotificationsConfig introduced in 1.12
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/managed-by: openshift-gitops
app.kubernetes.io/name: argocd-notifications-cm
app.kubernetes.io/part-of: argocd
annotation:
policy.open-cluster-management.io/disable-templates: "true"
name: argocd-notifications-cm
namespace: openshift-gitops
data:
service.slack: |
token: $slack-token
template.app-deployed: |-
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#18be52",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
},
{
"title": "Revision",
"value": "{{.app.status.sync.revision}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
template.app-health-degraded: |-
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link": "{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#f4c030",
"fields": [
{
"title": "Health Status",
"value": "{{.app.status.health.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
template.app-sync-failed: |-
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#E96D76",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
template.app-sync-running: |-
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#0DADEA",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
template.app-sync-status-unknown: |-
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#E96D76",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
template.app-sync-succeeded: |-
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#18be52",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
# trigger.on-created: |-
# - description: Application is created.
# oncePer: app.metadata.name
# send:
# - app-created
# when: "true"
# trigger.on-deleted: |-
# - description: Application is deleted.
# oncePer: app.metadata.name
# send:
# - app-deleted
# when: app.metadata.deletionTimestamp != nil
trigger.on-deployed: |-
- description: Application is synced and healthy. Triggered once per commit.
oncePer: app.status.operationState.syncResult.revision
send:
- app-deployed
when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status
== 'Healthy'
trigger.on-health-degraded: |-
- description: Application has degraded
send:
- app-health-degraded
when: app.status.health.status == 'Degraded'
trigger.on-sync-failed: |-
- description: Application syncing has failed
send:
- app-sync-failed
when: app.status.operationState.phase in ['Error', 'Failed']
trigger.on-sync-running: |-
- description: Application is being synced
send:
- app-sync-running
when: app.status.operationState.phase in ['Running']
trigger.on-sync-status-unknown: |-
- description: Application status is 'Unknown'
end:
- app-sync-status-unknown
when: app.status.sync.status == 'Unknown'
trigger.on-sync-succeeded: |-
- description: Application syncing has succeeded
send:
- app-sync-succeeded
when: app.status.operationState.phase in ['Succeeded']

540
components/policies/gitops/base/manifests/gitops-notifications/base/notifications-config.yaml Normal file
View File

@@ -0,0 +1,540 @@
apiVersion: argoproj.io/v1alpha1
kind: NotificationsConfiguration
metadata:
name: default-notifications-configuration
namespace: openshift-gitops
spec:
services:
service.slack: |
token: $slack-token
templates:
template.app-created: |-
email:
subject: Application {{.app.metadata.name}} has been created.
message: Application {{.app.metadata.name}} has been created.
teams:
title: Application {{.app.metadata.name}} has been created.
template.app-deleted: |-
email:
subject: Application {{.app.metadata.name}} has been deleted.
message: Application {{.app.metadata.name}} has been deleted.
teams:
title: Application {{.app.metadata.name}} has been deleted.
template.app-deployed: |-
email:
subject: New version of an application {{.app.metadata.name}} is up and running.
message: |
{{if eq .serviceType "slack"}}:white_check_mark:{{end}} Application {{.app.metadata.name}} is now running new version of deployments manifests.
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#18be52",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
},
{
"title": "Revision",
"value": "{{.app.status.sync.revision}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
teams:
facts: |
[{
"name": "Sync Status",
"value": "{{.app.status.sync.status}}"
},
{
"name": "Repository",
"value": "{{.app.spec.source.repoURL}}"
},
{
"name": "Revision",
"value": "{{.app.status.sync.revision}}"
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"name": "{{$c.type}}",
"value": "{{$c.message}}"
}
{{end}}
]
potentialAction: |-
[{
"@type":"OpenUri",
"name":"Operation Application",
"targets":[{
"os":"default",
"uri":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}"
}]
},
{
"@type":"OpenUri",
"name":"Open Repository",
"targets":[{
"os":"default",
"uri":"{{.app.spec.source.repoURL | call .repo.RepoURLToHTTPS}}"
}]
}]
themeColor: '#000080'
title: New version of an application {{.app.metadata.name}} is up and running.
template.app-health-degraded: |-
email:
subject: Application {{.app.metadata.name}} has degraded.
message: |
{{if eq .serviceType "slack"}}:exclamation:{{end}} Application {{.app.metadata.name}} has degraded.
Application details: {{.context.argocdUrl}}/applications/{{.app.metadata.name}}.
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link": "{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#f4c030",
"fields": [
{
"title": "Health Status",
"value": "{{.app.status.health.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
teams:
facts: |
[{
"name": "Health Status",
"value": "{{.app.status.health.status}}"
},
{
"name": "Repository",
"value": "{{.app.spec.source.repoURL}}"
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"name": "{{$c.type}}",
"value": "{{$c.message}}"
}
{{end}}
]
potentialAction: |
[{
"@type":"OpenUri",
"name":"Open Application",
"targets":[{
"os":"default",
"uri":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}"
}]
},
{
"@type":"OpenUri",
"name":"Open Repository",
"targets":[{
"os":"default",
"uri":"{{.app.spec.source.repoURL | call .repo.RepoURLToHTTPS}}"
}]
}]
themeColor: '#FF0000'
title: Application {{.app.metadata.name}} has degraded.
template.app-sync-failed: |-
email:
subject: Failed to sync application {{.app.metadata.name}}.
message: |
{{if eq .serviceType "slack"}}:exclamation:{{end}} The sync operation of application {{.app.metadata.name}} has failed at {{.app.status.operationState.finishedAt}} with the following error: {{.app.status.operationState.message}}
Sync operation details are available at: {{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true .
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#E96D76",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
teams:
facts: |
[{
"name": "Sync Status",
"value": "{{.app.status.sync.status}}"
},
{
"name": "Failed at",
"value": "{{.app.status.operationState.finishedAt}}"
},
{
"name": "Repository",
"value": "{{.app.spec.source.repoURL}}"
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"name": "{{$c.type}}",
"value": "{{$c.message}}"
}
{{end}}
]
potentialAction: |-
[{
"@type":"OpenUri",
"name":"Open Operation",
"targets":[{
"os":"default",
"uri":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true"
}]
},
{
"@type":"OpenUri",
"name":"Open Repository",
"targets":[{
"os":"default",
"uri":"{{.app.spec.source.repoURL | call .repo.RepoURLToHTTPS}}"
}]
}]
themeColor: '#FF0000'
title: Failed to sync application {{.app.metadata.name}}.
template.app-sync-running: |-
email:
subject: Start syncing application {{.app.metadata.name}}.
message: |
The sync operation of application {{.app.metadata.name}} has started at {{.app.status.operationState.startedAt}}.
Sync operation details are available at: {{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true .
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#0DADEA",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
teams:
facts: |
[{
"name": "Sync Status",
"value": "{{.app.status.sync.status}}"
},
{
"name": "Started at",
"value": "{{.app.status.operationState.startedAt}}"
},
{
"name": "Repository",
"value": "{{.app.spec.source.repoURL}}"
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"name": "{{$c.type}}",
"value": "{{$c.message}}"
}
{{end}}
]
potentialAction: |-
[{
"@type":"OpenUri",
"name":"Open Operation",
"targets":[{
"os":"default",
"uri":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true"
}]
},
{
"@type":"OpenUri",
"name":"Open Repository",
"targets":[{
"os":"default",
"uri":"{{.app.spec.source.repoURL | call .repo.RepoURLToHTTPS}}"
}]
}]
title: Start syncing application {{.app.metadata.name}}.
template.app-sync-status-unknown: |-
email:
subject: Application {{.app.metadata.name}} sync status is 'Unknown'
message: |
{{if eq .serviceType "slack"}}:exclamation:{{end}} Application {{.app.metadata.name}} sync is 'Unknown'.
Application details: {{.context.argocdUrl}}/applications/{{.app.metadata.name}}.
{{if ne .serviceType "slack"}}
{{range $c := .app.status.conditions}}
* {{$c.message}}
{{end}}
{{end}}
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#E96D76",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
teams:
facts: |
[{
"name": "Sync Status",
"value": "{{.app.status.sync.status}}"
},
{
"name": "Repository",
"value": "{{.app.spec.source.repoURL}}"
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"name": "{{$c.type}}",
"value": "{{$c.message}}"
}
{{end}}
]
potentialAction: |-
[{
"@type":"OpenUri",
"name":"Open Application",
"targets":[{
"os":"default",
"uri":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}"
}]
},
{
"@type":"OpenUri",
"name":"Open Repository",
"targets":[{
"os":"default",
"uri":"{{.app.spec.source.repoURL | call .repo.RepoURLToHTTPS}}"
}]
}]
title: Application {{.app.metadata.name}} sync status is 'Unknown'
template.app-sync-succeeded: |-
email:
subject: Application {{.app.metadata.name}} has been successfully synced.
message: |
{{if eq .serviceType "slack"}}:white_check_mark:{{end}} Application {{.app.metadata.name}} has been successfully synced at {{.app.status.operationState.finishedAt}}.
Sync operation details are available at: {{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true .
slack:
attachments: |
[{
"title": "{{ .app.metadata.name}}",
"title_link":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}",
"color": "#18be52",
"fields": [
{
"title": "Sync Status",
"value": "{{.app.status.sync.status}}",
"short": true
},
{
"title": "Repository",
"value": "{{.app.spec.source.repoURL}}",
"short": true
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"title": "{{$c.type}}",
"value": "{{$c.message}}",
"short": true
}
{{end}}
]
}]
deliveryPolicy: Post
groupingKey: ""
notifyBroadcast: false
teams:
facts: |
[{
"name": "Sync Status",
"value": "{{.app.status.sync.status}}"
},
{
"name": "Synced at",
"value": "{{.app.status.operationState.finishedAt}}"
},
{
"name": "Repository",
"value": "{{.app.spec.source.repoURL}}"
}
{{range $index, $c := .app.status.conditions}}
{{if not $index}},{{end}}
{{if $index}},{{end}}
{
"name": "{{$c.type}}",
"value": "{{$c.message}}"
}
{{end}}
]
potentialAction: |-
[{
"@type":"OpenUri",
"name":"Operation Details",
"targets":[{
"os":"default",
"uri":"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}?operation=true"
}]
},
{
"@type":"OpenUri",
"name":"Open Repository",
"targets":[{
"os":"default",
"uri":"{{.app.spec.source.repoURL | call .repo.RepoURLToHTTPS}}"
}]
}]
themeColor: '#000080'
title: Application {{.app.metadata.name}} has been successfully synced
triggers:
trigger.on-created: |-
- description: Application is created.
oncePer: app.metadata.name
send:
- app-created
when: "true"
trigger.on-deleted: |-
- description: Application is deleted.
oncePer: app.metadata.name
send:
- app-deleted
when: app.metadata.deletionTimestamp != nil
trigger.on-deployed: |-
- description: Application is synced and healthy. Triggered once per commit.
oncePer: app.status.operationState.syncResult.revision
send:
- app-deployed
when: app.status.operationState.phase in ['Succeeded'] and app.status.health.status
== 'Healthy'
trigger.on-health-degraded: |-
- description: Application has degraded
send:
- app-health-degraded
when: app.status.health.status == 'Degraded'
trigger.on-sync-failed: |-
- description: Application syncing has failed
send:
- app-sync-failed
when: app.status.operationState.phase in ['Error', 'Failed']
trigger.on-sync-running: |-
- description: Application is being synced
send:
- app-sync-running
when: app.status.operationState.phase in ['Running']
trigger.on-sync-status-unknown: |-
- description: Application status is 'Unknown'
send:
- app-sync-status-unknown
when: app.status.sync.status == 'Unknown'
trigger.on-sync-succeeded: |-
- description: Application syncing has succeeded
send:
- app-sync-succeeded
when: app.status.operationState.phase in ['Succeeded']

20
components/policies/gitops/base/manifests/gitops-secrets/base/argocd-external-secret.yaml Normal file
View File

@@ -0,0 +1,20 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: argocd-secret
namespace: openshift-gitops
spec:
data:
- remoteRef:
conversionStrategy: Default
decodingStrategy: None
key: OIDC_ARGOCD
secretKey: oidc.keycloak.clientSecret
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: doppler-cluster
target:
creationPolicy: Merge
deletionPolicy: Retain
name: argocd-secret

3
components/policies/gitops/base/manifests/gitops-secrets/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,3 @@
resources:
- argocd-external-secret.yaml
- notifications-secret.yaml

17
components/policies/gitops/base/manifests/gitops-secrets/base/notifications-secret.yaml Normal file
View File

@@ -0,0 +1,17 @@
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: argocd-notifications-secret
namespace: openshift-gitops
spec:
secretStoreRef:
kind: ClusterSecretStore
name: doppler-cluster
target:
name: argocd-notifications-secret
creationPolicy: Merge
deletionPolicy: Retain
data:
- secretKey: slack-token
remoteRef:
key: GITOPS_SLACK_TOKEN

12
components/policies/gitops/base/manifests/gitops-subscription/base/cluster-admin-rolebinding.yaml Normal file
View File

@@ -0,0 +1,12 @@
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: argocd-application-controller-cluster-admin
subjects:
- kind: ServiceAccount
name: openshift-gitops-argocd-application-controller
namespace: openshift-gitops
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin

39
components/policies/gitops/base/manifests/gitops-subscription/base/gitops-controller-rbac.yaml Normal file
View File

@@ -0,0 +1,39 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
name: gitops-controller
aggregationRule:
clusterRoleSelectors:
- matchLabels:
gitops/aggregate-to-controller: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gitops-controller-admin
labels:
gitops/aggregate-to-controller: "true"
aggregationRule:
clusterRoleSelectors:
- matchLabels:
rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules: []
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gitops-controller-view
labels:
gitops/aggregate-to-controller: "true"
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- watch

53
components/policies/gitops/base/manifests/gitops-subscription/base/gitops-server-rbac.yaml Normal file
View File

@@ -0,0 +1,53 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gitops-server
rules:
- verbs:
- get
- patch
- delete
apiGroups:
- '*'
resources:
- '*'
- verbs:
- create
- get
- list
- watch
- update
- patch
- delete
apiGroups:
- ''
resources:
- secrets
- configmaps
- verbs:
- create
- get
- list
- watch
- update
- delete
- patch
apiGroups:
- argoproj.io
resources:
- applications
- appprojects
- applicationsets
- verbs:
- create
- list
apiGroups:
- ''
resources:
- events
- apiGroups:
- ""
resources:
- pods/exec
verbs:
- create

8
components/policies/gitops/base/manifests/gitops-subscription/base/kustomization.yaml Normal file
View File

@@ -0,0 +1,8 @@
resources:
- sub-namespace.yaml
- sub-operatorgroup.yaml
- subscription.yaml
- namespace.yaml
- gitops-controller-rbac.yaml
- gitops-server-rbac.yaml
- cluster-admin-rolebinding.yaml

6
components/policies/gitops/base/manifests/gitops-subscription/base/namespace.yaml Normal file
View File

@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
labels:
openshift.io/cluster-monitoring: "true"
name: openshift-gitops

6
components/policies/gitops/base/manifests/gitops-subscription/base/sub-namespace.yaml Normal file
View File

@@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
labels:
openshift.io/cluster-monitoring: "true"
name: openshift-gitops-operator

7
components/policies/gitops/base/manifests/gitops-subscription/base/sub-operatorgroup.yaml Normal file
View File

@@ -0,0 +1,7 @@
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: gitops-operators
namespace: openshift-gitops-operator
spec:
upgradeStrategy: Default

19
components/policies/gitops/base/manifests/gitops-subscription/base/subscription.yaml Normal file
View File

@@ -0,0 +1,19 @@
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: openshift-gitops-operator
namespace: openshift-gitops-operator
spec:
config:
env:
- name: ARGOCD_CLUSTER_CONFIG_NAMESPACES
value: openshift-gitops, gitops
- name: CONTROLLER_CLUSTER_ROLE
value: gitops-controller
- name: SERVER_CLUSTER_ROLE
value: gitops-server
channel: gitops-1.15
installPlanApproval: Automatic
name: openshift-gitops-operator
source: redhat-operators
sourceNamespace: openshift-marketplace

28
components/policies/gitops/base/placement.yaml Normal file
View File

@@ -0,0 +1,28 @@
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: Placement
metadata:
name: placement-policy-gitops
namespace: acm-policies
spec:
clusterSets:
- global
predicates:
- requiredClusterSelector:
labelSelector:
matchExpressions:
- key: gitops
operator: Exists
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-policy-gitops
namespace: acm-policies
placementRef:
apiGroup: cluster.open-cluster-management.io
kind: Placement
name: placement-policy-gitops
subjects:
- apiGroup: policy.open-cluster-management.io
kind: PolicySet
name: gitops

60
components/policies/gitops/base/policy-generator-config.yaml Normal file
View File

@@ -0,0 +1,60 @@
apiVersion: policy.open-cluster-management.io/v1
kind: PolicyGenerator
metadata:
name: gitops-policy-generator
policyDefaults:
namespace: acm-policies
remediationAction: enforce
placementBindingDefaults:
name: "binding-policy-gitops"
policies:
- name: policy-eso-secret-store
configurationPolicyAnnotations:
apps.open-cluster-management.io/reconcile-option: replace
manifests:
- path: manifests/eso/base/
- name: policy-gitops-subscription
remediationAction: enforce
manifests:
- path: manifests/gitops-subscription/base/
- name: policy-gitops-instance
# Needed to fix issue with merging lists resulting in duplicates
configurationPolicyAnnotations:
apps.open-cluster-management.io/reconcile-option: replace
complianceType: "mustonlyhave"
manifests:
- path: manifests/gitops-instance/base/
- name: policy-gitops-bootstrap
manifests:
- path: manifests/gitops-bootstrap/base/
- name: policy-gitops-notifications
remediationAction: enforce
manifests:
- path: manifests/gitops-notifications/base/
configurationPolicyAnnotations:
policy.open-cluster-management.io/disable-templates: "true"
- name: policy-gitops-alerting
remediationAction: enforce
manifests:
- path: manifests/gitops-alerting/base/
configurationPolicyAnnotations:
policy.open-cluster-management.io/disable-templates: "true"
# Put ESO secrets into separate policy since it's chicken and egg
# Not being used since I have a global policy that does this but included for example
# - name: policy-gitops-secrets
# remediationAction: enforce
# manifests:
# - path: manifests/gitops-secrets/base/
policySets:
- name: "gitops"
description: "Policy for bootstrapping cluster with gitops"
policies:
- policy-eso-secret-store
- policy-gitops-subscription
- policy-gitops-instance
- policy-gitops-bootstrap
- policy-gitops-notifications
- policy-gitops-alerting
generatePolicySetPlacement: false
placement:
name: placement-policy-gitops

459
components/policies/opa/gatekeeper-probes-livenessnotset.yaml Normal file
View File

@@ -0,0 +1,459 @@
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: policy-gatekeeper-livenessprobe
annotations:
policy.open-cluster-management.io/standards: NIST SP 800-53
policy.open-cluster-management.io/categories: CM Configuration Management
policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
remediationAction: enforce
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-containerlivenessprobenotset
spec:
remediationAction: enforce
severity: low
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
creationTimestamp: null
name: containerlivenessprobenotset
spec:
crd:
spec:
names:
kind: ContainerLivenessprobeNotset
targets:
- libs:
- |
package lib.konstraint
default is_gatekeeper = false
is_gatekeeper {
has_field(input, "review")
has_field(input.review, "object")
}
object = input {
not is_gatekeeper
}
object = input.review.object {
is_gatekeeper
}
format(msg) = gatekeeper_format {
is_gatekeeper
gatekeeper_format = {"msg": msg}
}
format(msg) = msg {
not is_gatekeeper
}
name = object.metadata.name
kind = object.kind
has_field(obj, field) {
obj[field]
}
missing_field(obj, field) = true {
obj[field] == ""
}
missing_field(obj, field) = true {
not has_field(obj, field)
}
is_service {
lower(kind) == "service"
}
is_statefulset {
lower(kind) == "statefulset"
}
is_daemonset {
lower(kind) == "daemonset"
}
is_deployment {
lower(kind) == "deployment"
}
is_pod {
lower(kind) == "pod"
}
is_namespace {
lower(kind) == "namespace"
}
is_workload {
containers[_]
}
pod_containers(pod) = all_containers {
keys = {"containers", "initContainers"}
all_containers = [c | keys[k]; c = pod.spec[k][_]]
}
containers[container] {
pods[pod]
all_containers = pod_containers(pod)
container = all_containers[_]
}
containers[container] {
all_containers = pod_containers(object)
container = all_containers[_]
}
container_images[image] {
containers[container]
image = container.image
}
container_images[image] {
image = object.spec.image
}
split_image(image) = [image, "latest"] {
not contains(image, ":")
}
split_image(image) = [image_name, tag] {
[image_name, tag] = split(image, ":")
}
pods[pod] {
is_statefulset
pod = object.spec.template
}
pods[pod] {
is_daemonset
pod = object.spec.template
}
pods[pod] {
is_deployment
pod = object.spec.template
}
pods[pod] {
is_pod
pod = object
}
volumes[volume] {
pods[pod]
volume = pod.spec.volumes[_]
}
mem_multiple("E") = 1000000000000000000000 { true }
mem_multiple("P") = 1000000000000000000 { true }
mem_multiple("T") = 1000000000000000 { true }
mem_multiple("G") = 1000000000000 { true }
mem_multiple("M") = 1000000000 { true }
mem_multiple("k") = 1000000 { true }
mem_multiple("") = 1000 { true }
mem_multiple("m") = 1 { true }
mem_multiple("Ki") = 1024000 { true }
mem_multiple("Mi") = 1048576000 { true }
mem_multiple("Gi") = 1073741824000 { true }
mem_multiple("Ti") = 1099511627776000 { true }
mem_multiple("Pi") = 1125899906842624000 { true }
mem_multiple("Ei") = 1152921504606846976000 { true }
get_suffix(mem) = suffix {
not is_string(mem)
suffix := ""
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) > 0
suffix := substring(mem, count(mem) - 1, -1)
mem_multiple(suffix)
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) > 1
suffix := substring(mem, count(mem) - 2, -1)
mem_multiple(suffix)
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) > 1
not mem_multiple(substring(mem, count(mem) - 1, -1))
not mem_multiple(substring(mem, count(mem) - 2, -1))
suffix := ""
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) == 1
not mem_multiple(substring(mem, count(mem) - 1, -1))
suffix := ""
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) == 0
suffix := ""
}
canonify_mem(orig) = new {
is_number(orig)
new := orig * 1000
}
canonify_mem(orig) = new {
not is_number(orig)
suffix := get_suffix(orig)
raw := replace(orig, suffix, "")
re_match("^[0-9]+$", raw)
new := to_number(raw) * mem_multiple(suffix)
}
canonify_storage(orig) = new {
is_number(orig)
new := orig
}
canonify_storage(orig) = new {
not is_number(orig)
suffix := get_suffix(orig)
raw := replace(orig, suffix, "")
re_match("^[0-9]+$", raw)
new := to_number(raw) * mem_multiple(suffix)
}
canonify_cpu(orig) = new {
is_number(orig)
new := orig * 1000
}
canonify_cpu(orig) = new {
not is_number(orig)
endswith(orig, "m")
new := to_number(replace(orig, "m", ""))
}
canonify_cpu(orig) = new {
not is_number(orig)
not endswith(orig, "m")
re_match("^[0-9]+$", orig)
new := to_number(orig) * 1000
}
dropped_capability(container, cap) {
container.securityContext.capabilities.drop[_] == cap
}
added_capability(container, cap) {
container.securityContext.capabilities.add[_] == cap
}
no_read_only_filesystem(c) {
not has_field(c, "securityContext")
}
no_read_only_filesystem(c) {
has_field(c, "securityContext")
not has_field(c.securityContext, "readOnlyRootFilesystem")
}
priviledge_escalation_allowed(c) {
not has_field(c, "securityContext")
}
priviledge_escalation_allowed(c) {
has_field(c, "securityContext")
has_field(c.securityContext, "allowPrivilegeEscalation")
}
- |-
package lib.openshift
import data.lib.konstraint
is_deploymentconfig {
lower(konstraint.object.apiVersion) == "apps.openshift.io/v1"
lower(konstraint.object.kind) == "deploymentconfig"
}
is_route {
lower(konstraint.object.apiVersion) == "route.openshift.io/v1"
lower(konstraint.object.kind) == "route"
}
is_workload_kind {
is_deploymentconfig
}
is_workload_kind {
konstraint.is_statefulset
}
is_workload_kind {
konstraint.is_daemonset
}
is_workload_kind {
konstraint.is_deployment
}
is_all_kind {
is_workload_kind
}
is_all_kind {
konstraint.is_service
}
is_all_kind {
is_route
}
pods[pod] {
is_deploymentconfig
pod = konstraint.object.spec.template
}
pods[pod] {
pod = konstraint.pods[_]
}
containers[container] {
pods[pod]
all_containers = konstraint.pod_containers(pod)
container = all_containers[_]
}
containers[container] {
container = konstraint.containers[_]
}
rego: |-
package ocp.bestpractices.container_livenessprobe_notset
import data.lib.konstraint
import data.lib.openshift
violation[msg] {
openshift.is_workload_kind
container := openshift.containers[_]
konstraint.missing_field(container, "livenessProbe")
obj := konstraint.object
msg := konstraint.format(sprintf("%s/%s: container '%s' has no livenessProbe. See: https://docs.openshift.com/container-platform/latest/applications/application-health.html", [obj.kind, obj.metadata.name, container.name]))
}
target: admission.k8s.gatekeeper.sh
- complianceType: musthave
objectDefinition:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: ContainerLivenessprobeNotset
metadata:
name: containerlivenessprobenotset
spec:
enforcementAction: dryrun
match:
kinds:
- apiGroups:
- apps.openshift.io
- apps
kinds:
- DeploymentConfig
- DaemonSet
- Deployment
- StatefulSet
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-audit-liveness
spec:
remediationAction: inform # will be overridden by remediationAction in parent policy
severity: low
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: ContainerLivenessprobeNotset
metadata:
name: containerlivenessprobenotset
status:
totalViolations: 0
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-admission-liveness
spec:
remediationAction: inform # will be overridden by remediationAction in parent policy
severity: low
object-templates:
- complianceType: mustnothave
objectDefinition:
apiVersion: v1
kind: Event
metadata:
namespace: openshift-gatekeeper-system # set it to the actual namespace where gatekeeper is running if different
annotations:
constraint_action: deny
constraint_kind: ContainerLivenessprobeNotset
constraint_name: containerlivenessprobenotset
event_type: violation
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-policy-gatekeeper-containerlivenessprobenotset
placementRef:
name: placement-policy-gatekeeper-containerlivenessprobenotset
kind: PlacementRule
apiGroup: apps.open-cluster-management.io
subjects:
- name: policy-gatekeeper-livenessprobe
kind: Policy
apiGroup: policy.open-cluster-management.io
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: placement-policy-gatekeeper-containerlivenessprobenotset
spec:
clusterConditions:
- status: "True"
type: ManagedClusterConditionAvailable
clusterSelector:
matchExpressions:
- { key: policy, operator: In, values: ["opa"] }

459
components/policies/opa/gatekeeper-probes-readinessnotset.yaml Normal file
View File

@@ -0,0 +1,459 @@
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
name: policy-gatekeeper-readinessprobe
annotations:
policy.open-cluster-management.io/standards: NIST SP 800-53
policy.open-cluster-management.io/categories: CM Configuration Management
policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
remediationAction: enforce
disabled: false
policy-templates:
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-containerreadinessprobenotset
spec:
remediationAction: enforce
severity: low
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
creationTimestamp: null
name: containerreadinessprobenotset
spec:
crd:
spec:
names:
kind: ContainerReadinessprobeNotset
targets:
- libs:
- |
package lib.konstraint
default is_gatekeeper = false
is_gatekeeper {
has_field(input, "review")
has_field(input.review, "object")
}
object = input {
not is_gatekeeper
}
object = input.review.object {
is_gatekeeper
}
format(msg) = gatekeeper_format {
is_gatekeeper
gatekeeper_format = {"msg": msg}
}
format(msg) = msg {
not is_gatekeeper
}
name = object.metadata.name
kind = object.kind
has_field(obj, field) {
obj[field]
}
missing_field(obj, field) = true {
obj[field] == ""
}
missing_field(obj, field) = true {
not has_field(obj, field)
}
is_service {
lower(kind) == "service"
}
is_statefulset {
lower(kind) == "statefulset"
}
is_daemonset {
lower(kind) == "daemonset"
}
is_deployment {
lower(kind) == "deployment"
}
is_pod {
lower(kind) == "pod"
}
is_namespace {
lower(kind) == "namespace"
}
is_workload {
containers[_]
}
pod_containers(pod) = all_containers {
keys = {"containers", "initContainers"}
all_containers = [c | keys[k]; c = pod.spec[k][_]]
}
containers[container] {
pods[pod]
all_containers = pod_containers(pod)
container = all_containers[_]
}
containers[container] {
all_containers = pod_containers(object)
container = all_containers[_]
}
container_images[image] {
containers[container]
image = container.image
}
container_images[image] {
image = object.spec.image
}
split_image(image) = [image, "latest"] {
not contains(image, ":")
}
split_image(image) = [image_name, tag] {
[image_name, tag] = split(image, ":")
}
pods[pod] {
is_statefulset
pod = object.spec.template
}
pods[pod] {
is_daemonset
pod = object.spec.template
}
pods[pod] {
is_deployment
pod = object.spec.template
}
pods[pod] {
is_pod
pod = object
}
volumes[volume] {
pods[pod]
volume = pod.spec.volumes[_]
}
mem_multiple("E") = 1000000000000000000000 { true }
mem_multiple("P") = 1000000000000000000 { true }
mem_multiple("T") = 1000000000000000 { true }
mem_multiple("G") = 1000000000000 { true }
mem_multiple("M") = 1000000000 { true }
mem_multiple("k") = 1000000 { true }
mem_multiple("") = 1000 { true }
mem_multiple("m") = 1 { true }
mem_multiple("Ki") = 1024000 { true }
mem_multiple("Mi") = 1048576000 { true }
mem_multiple("Gi") = 1073741824000 { true }
mem_multiple("Ti") = 1099511627776000 { true }
mem_multiple("Pi") = 1125899906842624000 { true }
mem_multiple("Ei") = 1152921504606846976000 { true }
get_suffix(mem) = suffix {
not is_string(mem)
suffix := ""
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) > 0
suffix := substring(mem, count(mem) - 1, -1)
mem_multiple(suffix)
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) > 1
suffix := substring(mem, count(mem) - 2, -1)
mem_multiple(suffix)
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) > 1
not mem_multiple(substring(mem, count(mem) - 1, -1))
not mem_multiple(substring(mem, count(mem) - 2, -1))
suffix := ""
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) == 1
not mem_multiple(substring(mem, count(mem) - 1, -1))
suffix := ""
}
get_suffix(mem) = suffix {
is_string(mem)
count(mem) == 0
suffix := ""
}
canonify_mem(orig) = new {
is_number(orig)
new := orig * 1000
}
canonify_mem(orig) = new {
not is_number(orig)
suffix := get_suffix(orig)
raw := replace(orig, suffix, "")
re_match("^[0-9]+$", raw)
new := to_number(raw) * mem_multiple(suffix)
}
canonify_storage(orig) = new {
is_number(orig)
new := orig
}
canonify_storage(orig) = new {
not is_number(orig)
suffix := get_suffix(orig)
raw := replace(orig, suffix, "")
re_match("^[0-9]+$", raw)
new := to_number(raw) * mem_multiple(suffix)
}
canonify_cpu(orig) = new {
is_number(orig)
new := orig * 1000
}
canonify_cpu(orig) = new {
not is_number(orig)
endswith(orig, "m")
new := to_number(replace(orig, "m", ""))
}
canonify_cpu(orig) = new {
not is_number(orig)
not endswith(orig, "m")
re_match("^[0-9]+$", orig)
new := to_number(orig) * 1000
}
dropped_capability(container, cap) {
container.securityContext.capabilities.drop[_] == cap
}
added_capability(container, cap) {
container.securityContext.capabilities.add[_] == cap
}
no_read_only_filesystem(c) {
not has_field(c, "securityContext")
}
no_read_only_filesystem(c) {
has_field(c, "securityContext")
not has_field(c.securityContext, "readOnlyRootFilesystem")
}
priviledge_escalation_allowed(c) {
not has_field(c, "securityContext")
}
priviledge_escalation_allowed(c) {
has_field(c, "securityContext")
has_field(c.securityContext, "allowPrivilegeEscalation")
}
- |-
package lib.openshift
import data.lib.konstraint
is_deploymentconfig {
lower(konstraint.object.apiVersion) == "apps.openshift.io/v1"
lower(konstraint.object.kind) == "deploymentconfig"
}
is_route {
lower(konstraint.object.apiVersion) == "route.openshift.io/v1"
lower(konstraint.object.kind) == "route"
}
is_workload_kind {
is_deploymentconfig
}
is_workload_kind {
konstraint.is_statefulset
}
is_workload_kind {
konstraint.is_daemonset
}
is_workload_kind {
konstraint.is_deployment
}
is_all_kind {
is_workload_kind
}
is_all_kind {
konstraint.is_service
}
is_all_kind {
is_route
}
pods[pod] {
is_deploymentconfig
pod = konstraint.object.spec.template
}
pods[pod] {
pod = konstraint.pods[_]
}
containers[container] {
pods[pod]
all_containers = konstraint.pod_containers(pod)
container = all_containers[_]
}
containers[container] {
container = konstraint.containers[_]
}
rego: |-
package ocp.bestpractices.container_readinessprobe_notset
import data.lib.konstraint
import data.lib.openshift
violation[msg] {
openshift.is_workload_kind
container := openshift.containers[_]
konstraint.missing_field(container, "readinessProbe")
obj := konstraint.object
msg := konstraint.format(sprintf("%s/%s: container '%s' has no readinessProbe. See: https://docs.openshift.com/container-platform/4.4/applications/application-health.html", [obj.kind, obj.metadata.name, container.name]))
}
target: admission.k8s.gatekeeper.sh
- complianceType: musthave
objectDefinition:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: ContainerReadinessprobeNotset
metadata:
name: containerreadinessprobenotset
spec:
enforcementAction: dryrun
match:
kinds:
- apiGroups:
- apps.openshift.io
- apps
kinds:
- DeploymentConfig
- DaemonSet
- Deployment
- StatefulSet
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-audit-readiness
spec:
remediationAction: inform # will be overridden by remediationAction in parent policy
severity: low
object-templates:
- complianceType: musthave
objectDefinition:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: ContainerReadinessprobeNotset
metadata:
name: containerreadinessprobenotset
status:
totalViolations: 0
- objectDefinition:
apiVersion: policy.open-cluster-management.io/v1
kind: ConfigurationPolicy
metadata:
name: policy-gatekeeper-admission-readiness
spec:
remediationAction: inform # will be overridden by remediationAction in parent policy
severity: low
object-templates:
- complianceType: mustnothave
objectDefinition:
apiVersion: v1
kind: Event
metadata:
namespace: openshift-gatekeeper-system # set it to the actual namespace where gatekeeper is running if different
annotations:
constraint_action: deny
constraint_kind: ContainerReadinessprobeNotset
constraint_name: containerreadinessprobenotset
event_type: violation
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
name: binding-policy-gatekeeper-containerreadinessprobenotset
placementRef:
name: placement-policy-gatekeeper-containerreadinessprobenotset
kind: PlacementRule
apiGroup: apps.open-cluster-management.io
subjects:
- name: policy-gatekeeper-readinessprobe
kind: Policy
apiGroup: policy.open-cluster-management.io
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
name: placement-policy-gatekeeper-containerreadinessprobenotset
spec:
clusterConditions:
- status: "True"
type: ManagedClusterConditionAvailable
clusterSelector:
matchExpressions:
- { key: policy, operator: In, values: ["opa"] }