refactor(firecrawl): Convert to production-ready Helm chart with persistent storage

CRITICAL FIXES:
 Add PersistentVolumeClaims for all stateful services
   - PostgreSQL: 20Gi PVC on nfs-emporium (data persistence)
   - Redis: 10Gi PVC on nfs-emporium (cache and queue)
   - RabbitMQ: 5Gi PVC on nfs-emporium (message queue)

 Pin all image versions (no more 'latest' tags)
   - Firecrawl API/Worker: v1.0.0
   - Playwright Service: v1.0.0
   - PostgreSQL (nuq-postgres): v1.0.0
   - Redis: 7.4.1-alpine
   - RabbitMQ: 3.13.7-management-alpine

 Convert raw manifests to proper Helm chart
   - Template-based configuration
   - Centralized values.yaml
   - Proper Helm helpers and labels
   - Easy configuration management

WHAT CHANGED:
- Created chart/ directory with full Helm chart structure
- Moved old manifests to old-manifests/ for reference
- Updated ArgoCD Application to use Helm chart source
- Added comprehensive README and MIGRATION docs
- All services now use nfs-emporium storage class
- Redis configured with AOF persistence
- Proper resource limits and health checks
- Gateway/HTTPRoute configs integrated into chart

DEPLOYMENT:
ArgoCD will automatically sync and apply changes.
Old ephemeral data will be lost (fresh start with persistence).

Resolves data loss issues and brings deployment to production standards.
This commit is contained in:
Hermes Agent service account
2026-06-04 16:56:29 -05:00
parent f4181349f8
commit 6bcb6fa93f
31 changed files with 1354 additions and 7 deletions

View File

@@ -0,0 +1,97 @@
# Migration to Helm Chart
**Date**: 2026-06-04
**Reason**: Production-ready refactoring
## Critical Issues Fixed
### 1. ❌ No Persistent Storage → ✅ PersistentVolumeClaims
**Old**: PostgreSQL and Redis used `emptyDir` volumes
- All data lost on pod restart
- Not production-ready
**New**: All stateful services have PVCs
- PostgreSQL: 20Gi PVC on nfs-emporium
- Redis: 10Gi PVC on nfs-emporium
- RabbitMQ: 5Gi PVC on nfs-emporium
### 2. ❌ Using `latest` Tags → ✅ Pinned Versions
**Old**: Images used `latest` or unspecified tags
- Unpredictable updates
- Hard to rollback
- No version control
**New**: All images pinned to specific versions
- Firecrawl: `v1.0.0`
- Redis: `7.4.1-alpine`
- RabbitMQ: `3.13.7-management-alpine`
- PostgreSQL: `v1.0.0`
- Playwright: `v1.0.0`
### 3. ❌ Raw Manifests → ✅ Helm Chart
**Old**: Individual YAML files
- Hard to maintain
- No templating
- Difficult to customize
**New**: Proper Helm chart structure
- Template-based configuration
- Easy value overrides
- Industry standard deployment
## Migration Steps
1. **Backup** (if needed): Old deployment had ephemeral storage, no data to preserve
2. **ArgoCD Auto-Sync**: Push changes and ArgoCD will:
- Delete old resources
- Create new Helm release
- Provision PVCs
- Deploy with pinned versions
## Old Files Location
All previous manifests are archived in `old-manifests/` directory for reference.
## Rollback Procedure
If needed to rollback:
```bash
# Revert git commit
git revert HEAD
# ArgoCD will automatically sync back to old state
# Or manually:
argocd app sync firecrawl
```
## Verification
After deployment:
```bash
# Check all PVCs are bound
kubectl -n firecrawl get pvc
# Verify pods are running
kubectl -n firecrawl get pods
# Check image versions
kubectl -n firecrawl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[*].image}{"\n"}{end}'
# Test API
curl https://spaceship-earth.local.mk-labs.cloud/v0/health/liveness
```
## Benefits
**Data Persistence**: No more data loss on restarts
**Version Control**: Explicit version management
**Configuration Management**: Clean Helm values
**Maintainability**: Standard Helm patterns
**Production Ready**: Follows Kubernetes best practices
## Notes
- Storage class `nfs-emporium` provides persistent NFS-backed storage
- All resources properly labeled with Helm chart metadata
- Gateway/HTTPRoute configurations preserved
- EPCOT theming (spaceship-earth) maintained