homelab/cluster/applications/couchdb/README.md

# CouchDB Deployment

**Application Name:** couchdb  
**Thematic Name:** communicore (DNS only)  
**Purpose:** Obsidian sync backend for mk-labs  
**Deployment Method:** Helm chart via ArgoCD  

## Overview

This deployment provides a CouchDB instance for Obsidian sync across multiple devices in the mk-labs homelab. It replaces the previous NFS-based shared vault with a proper sync solution.

## Architecture

- **Helm Chart:** Apache CouchDB official chart (v4.6.3, CouchDB 3.5.1)
- **Cluster:** Single-node deployment (suitable for homelab)
- **Storage:** 20Gi persistent volume using nfs-emporium StorageClass
- **Secrets:** ExternalSecrets Operator pulling from 1Password
- **TLS:** cert-manager with letsencrypt-staging (switch to prod after validation)
- **Ingress:** nginx-ingress with external-dns

## DNS Configuration

- **Primary:** communicore.mk-labs.cloud (thematic name)
- Managed automatically by external-dns via ingress annotations

## Prerequisites

### 1Password Vault Setup

Create a new item in 1Password (mk-labs vault):

```
Item Name: couchdb
Type: Login

Fields:
  admin-password: (generate strong password, 32 characters)
  cookie-auth-secret: (generate random string, 32 characters)
```

Generate the secrets:
```bash
# Admin password
openssl rand -base64 32

# Cookie auth secret
openssl rand -base64 32
```

Store both values in the 1Password item before deploying.

## Deployment

### Initial Deployment

The deployment is managed entirely through GitOps:

1. Commit and push the manifests to the homelab repository
2. ArgoCD will automatically detect and sync the application
3. Monitor the deployment:

```bash
# Watch ArgoCD sync status
kubectl get application -n argocd couchdb -w

# Watch pod creation
kubectl get pods -n couchdb -w

# Check deployment status
kubectl get all -n couchdb
```

### Verification Steps

1. **Check ExternalSecret sync:**
```bash
kubectl get externalsecret -n couchdb
kubectl get secret -n couchdb couchdb-admin
```

2. **Verify pod health:**
```bash
kubectl get pods -n couchdb
kubectl logs -n couchdb -l app=couchdb
```

3. **Test internal access:**
```bash
kubectl port-forward -n couchdb svc/couchdb-svc-couchdb 5984:5984
curl -u admin:PASSWORD http://localhost:5984/
# Should return: {"couchdb":"Welcome","version":"3.5.1"}
```

4. **Check certificate:**
```bash
kubectl get certificate -n couchdb
kubectl describe certificate -n couchdb couchdb-tls
```

5. **Test external access:**
```bash
# Once DNS propagates (may take 2-5 minutes)
curl -k https://communicore.mk-labs.cloud/
# Should show CouchDB welcome message
```

### Switch to Production Certificates

After successful validation (24-48 hours stable), switch to production certificates:

```bash
cd ~/git/homelab/cluster/applications/couchdb

# Update values.yaml
sed -i 's/letsencrypt-staging/letsencrypt-prod/g' values.yaml

# Commit and push
git add values.yaml
git commit -m "feat(couchdb): switch to production TLS certificates"
git push

# Delete old staging certificate and secret to trigger reissuance
kubectl delete certificate -n couchdb couchdb-tls
kubectl delete secret -n couchdb couchdb-tls

# Verify new certificate
kubectl get certificate -n couchdb
```

## Obsidian Configuration

### Plugin Installation

On each device (workstation, desktop, carousel-of-progress):

1. Install "Self-hosted LiveSync" plugin in Obsidian
2. Enable the plugin
3. Configure as follows:

### Connection Settings

```
Remote Database URL: https://communicore.mk-labs.cloud
Remote Database: obsidian-sync
Username: admin
Password: (from 1Password couchdb item)
End-to-end Encryption: ✓ Enabled
Passphrase: (create a strong passphrase, store in 1Password)
```

### Initial Sync (IMPORTANT)

**ONLY ONE DEVICE** should perform the initial upload:

1. **On Workstation (primary device):**
   - Open Obsidian settings → Self-hosted LiveSync
   - Click "Rebuild Everything"
   - Wait for upload to complete (may take several minutes)

2. **On Other Devices (desktop, carousel):**
   - Open Obsidian settings → Self-hosted LiveSync
   - Click "Fetch from Remote"
   - Wait for download to complete

### Vault Locations

As documented in communicore-decisions.md:

- **Workstation:** ~/friday
- **Desktop:** ~/friday
- **Carousel-of-progress:** (to be determined by JARVIS)

## Troubleshooting

### Pod Not Starting

```bash
# Check events
kubectl describe pod -n couchdb <pod-name>

# Check logs
kubectl logs -n couchdb <pod-name>

# Common issues:
# - Secret not synced: check ExternalSecret status
# - PVC pending: verify storage class and NFS CSI driver
```

### Certificate Issues

```bash
# Check certificate status
kubectl get certificate -n couchdb
kubectl describe certificate -n couchdb couchdb-tls

# Check cert-manager logs
kubectl logs -n cert-manager -l app=cert-manager
```

### DNS Not Resolving

```bash
# Check external-dns logs
kubectl logs -n external-dns -l app.kubernetes.io/name=external-dns

# Verify ingress annotation
kubectl get ingress -n couchdb -o yaml | grep external-dns
```

### Connection Refused in Obsidian

1. Verify external access works from command line first
2. Check that the database exists:
```bash
curl -u admin:PASSWORD https://communicore.mk-labs.cloud/_all_dbs
```
3. If obsidian-sync database doesn't exist, create it:
```bash
curl -u admin:PASSWORD -X PUT https://communicore.mk-labs.cloud/obsidian-sync
```

## Backup Strategy

CouchDB data is stored on nfs-emporium persistent volume. The NFS backend (Synology at 10.1.71.9) should handle snapshots.

Additional backup options:
- CouchDB replication to backup instance
- Scheduled exports to S3/Minio
- Git backup of vault using Obsidian Git plugin

## Rollback

If deployment fails or causes issues:

```bash
# Delete the ArgoCD application
kubectl delete application -n argocd couchdb

# Delete the namespace (will cascade delete all resources)
kubectl delete namespace couchdb

# Remove from git
cd ~/git/homelab
git rm -rf cluster/applications/couchdb
git commit -m "rollback: remove couchdb deployment"
git push
```

## Maintenance

### Upgrade CouchDB Version

1. Update chart version in application.yaml
2. Update image tag in values.yaml
3. Test in staging first
4. Commit and push to trigger ArgoCD sync

### Rotate Admin Password

1. Update password in 1Password
2. ExternalSecret will automatically sync within 1h (or trigger manually)
3. Update Obsidian plugin configuration on all devices

## References

- [CouchDB Documentation](https://docs.couchdb.org/)
- [Helm Chart Repository](https://github.com/apache/couchdb-helm)
- [Self-hosted LiveSync Plugin](https://github.com/vrtmrz/obsidian-livesync)
- [communicore-decisions.md](file:///mnt/mk-labs-pka/inbox/team/communicore-decisions.md)

## Status

- **Created:** 2026-06-05
- **Status:** Ready for deployment
- **Next Steps:** 
  1. Create 1Password item with credentials
  2. Commit and push to homelab repo
  3. Monitor ArgoCD sync
  4. Validate external access
  5. Configure Obsidian on all devices