homelab/cluster/applications/firecrawl/chart/README.md

# Firecrawl Helm Chart

Production-ready Helm chart for deploying Firecrawl web scraping service.

## Overview

Firecrawl is a web scraping and search service that provides JARVIS with web context capabilities. This deployment includes all necessary components with proper persistent storage and pinned image versions.

## Components

### Application Services
- **Firecrawl API** - Main REST API service
  - Port: 3002
  - Health checks: `/v0/health/liveness`, `/v0/health/readiness`
  - Resources: 4Gi memory, 2 CPU cores

- **Firecrawl Worker** - Background job processor
  - Processes crawl jobs from queue
  - Resources: 3Gi memory, 1 CPU core

- **Playwright Service** - Browser automation
  - Port: 3000
  - Headless Chrome/Firefox for JavaScript rendering
  - Resources: 2Gi memory, 1 CPU core

### Infrastructure Services
- **PostgreSQL** (nuq-postgres) - State management
  - Port: 5432
  - Custom image with queue extensions
  - **Persistent Storage**: 20Gi PVC on nfs-emporium

- **Redis** - Cache and job queue
  - Port: 6379
  - AOF persistence enabled
  - **Persistent Storage**: 10Gi PVC on nfs-emporium

- **RabbitMQ** - Message queue
  - AMQP Port: 5672
  - Management UI: 15672
  - **Persistent Storage**: 5Gi PVC on nfs-emporium

## Production Features

### ✅ Persistent Storage
All stateful services use PersistentVolumeClaims backed by NFS storage:
- PostgreSQL: 20Gi (database state)
- Redis: 10Gi (cache and job queue)
- RabbitMQ: 5Gi (message queue data)

### ✅ Pinned Image Versions
All container images use specific version tags (no `latest`):
- Firecrawl API/Worker: `ghcr.io/mendableai/firecrawl:v1.0.0`
- Playwright Service: `ghcr.io/mendableai/firecrawl/playwright-service:v1.0.0`
- PostgreSQL: `ghcr.io/mendableai/firecrawl/nuq-postgres:v1.0.0`
- Redis: `redis:7.4.1-alpine`
- RabbitMQ: `rabbitmq:3.13.7-management-alpine`

### ✅ Helm Configuration Management
Proper Helm chart structure with:
- Configurable values via `values.yaml`
- Template helpers for consistent naming
- Resource requests and limits
- Health checks and probes
- Service discovery

## Configuration

### Key Values

```yaml
# Image versions (pinned)
image:
  repository: ghcr.io/mendableai/firecrawl
  tag: "v1.0.0"

# Persistent storage
postgres.persistence:
  enabled: true
  storageClass: "nfs-emporium"
  size: 20Gi

redis.persistence:
  enabled: true
  storageClass: "nfs-emporium"
  size: 10Gi

rabbitmq.persistence:
  enabled: true
  storageClass: "nfs-emporium"
  size: 5Gi

# Performance tuning
config:
  NUM_WORKERS_PER_QUEUE: "8"
  CRAWL_CONCURRENT_REQUESTS: "10"
  MAX_CONCURRENT_JOBS: "5"
```

See `values.yaml` for all configuration options.

## Deployment

### ArgoCD (Recommended)
This application is managed by ArgoCD in wave 20 (applications tier):

```bash
# Check application status
kubectl -n argocd get application firecrawl

# Sync manually if needed
argocd app sync firecrawl

# View application details
argocd app get firecrawl
```

### Manual Helm Deployment
```bash
# From repository root
cd cluster/applications/firecrawl

# Install
helm install firecrawl ./chart -n firecrawl --create-namespace

# Upgrade
helm upgrade firecrawl ./chart -n firecrawl

# Uninstall
helm uninstall firecrawl -n firecrawl
```

## Access

### External Access (via Gateway API)
- **Primary**: https://spaceship-earth.local.mk-labs.cloud
- **Secondary**: https://firecrawl.local.mk-labs.cloud

DNS automatically configured via ExternalDNS pointing to Gateway LB (10.1.71.90).

### Internal Access
```
http://firecrawl-api.firecrawl.svc.cluster.local:3002
```

### API Usage Example
```bash
curl -X POST https://spaceship-earth.local.mk-labs.cloud/v0/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'
```

## Monitoring

### View Pods
```bash
kubectl -n firecrawl get pods
```

### Check Logs
```bash
# API logs
kubectl -n firecrawl logs -l app.kubernetes.io/component=api -f

# Worker logs
kubectl -n firecrawl logs -l app.kubernetes.io/component=worker -f

# PostgreSQL logs
kubectl -n firecrawl logs -l app.kubernetes.io/component=postgres -f

# Redis logs
kubectl -n firecrawl logs -l app.kubernetes.io/component=redis -f
```

### Check Storage
```bash
# View PVCs
kubectl -n firecrawl get pvc

# Check PVC details
kubectl -n firecrawl describe pvc firecrawl-postgres-pvc
kubectl -n firecrawl describe pvc firecrawl-redis-pvc
kubectl -n firecrawl describe pvc firecrawl-rabbitmq-pvc
```

## Troubleshooting

### Pods Not Starting
```bash
kubectl -n firecrawl describe pod <pod-name>
kubectl -n firecrawl get events --sort-by='.lastTimestamp'
```

### Storage Issues
```bash
# Check PVC binding
kubectl -n firecrawl get pvc

# Verify NFS storage class
kubectl get storageclass nfs-emporium -o yaml

# Check NFS CSI driver
kubectl -n kube-system get pods -l app=csi-nfs-controller
```

### Database Connection Issues
```bash
# Test PostgreSQL connectivity from API pod
kubectl -n firecrawl exec -it deployment/firecrawl-api -- \
  sh -c 'apt-get update && apt-get install -y postgresql-client && \
  psql -h firecrawl-postgres -U postgres -d postgres -c "SELECT 1"'

# Test Redis connectivity
kubectl -n firecrawl exec -it deployment/firecrawl-api -- \
  sh -c 'apt-get update && apt-get install -y redis-tools && \
  redis-cli -h firecrawl-redis PING'
```

### Gateway Routing Issues
```bash
# Verify ReferenceGrant
kubectl -n gateway get referencegrant allow-httproutes -o yaml

# Check HTTPRoutes
kubectl -n firecrawl get httproute

# View Gateway status
kubectl -n gateway get gateway fastpass-gateway -o yaml
```

## Upgrading

### Update Image Versions
1. Edit `values.yaml` and update image tags
2. Commit and push changes
3. ArgoCD will automatically sync
4. Or manually: `helm upgrade firecrawl ./chart -n firecrawl`

### Migrate from Old Deployment
The old raw manifest deployment will be automatically replaced by ArgoCD:
1. Commit this Helm chart
2. Push to repository
3. ArgoCD detects the change and applies Helm chart
4. Old resources are pruned
5. New resources with PVCs are created

**Note**: Old PostgreSQL/Redis data was ephemeral and will be lost. Fresh start with persistent storage.

## Storage Architecture

### NFS Provisioner
- **Storage Class**: `nfs-emporium`
- **Provisioner**: `nfs.csi.k8s.io`
- **NFS Server**: emporium (Synology DS1621+) - 10.1.71.9
- **NFS Export**: `/volume1/fastpass`
- **Reclaim Policy**: Delete
- **Binding Mode**: Immediate

### PVC Structure
Each PVC gets a unique subdirectory:
```
/volume1/fastpass/firecrawl/firecrawl-postgres-pvc/
/volume1/fastpass/firecrawl/firecrawl-redis-pvc/
/volume1/fastpass/firecrawl/firecrawl-rabbitmq-pvc/
```

## Theme

Part of the EPCOT-themed namespace with primary hostname **spaceship-earth** representing Firecrawl's role as the information and knowledge hub for JARVIS.

## Maintenance

### Backup Recommendations
```bash
# PostgreSQL backup (manual)
kubectl -n firecrawl exec deployment/firecrawl-postgres -- \
  pg_dump -U postgres postgres > firecrawl-backup-$(date +%Y%m%d).sql

# NFS snapshots (via Synology)
# Use Synology Snapshot Replication for point-in-time backups
```

### Scaling
```bash
# Scale API replicas
helm upgrade firecrawl ./chart -n firecrawl \
  --set api.replicaCount=2

# Scale workers
helm upgrade firecrawl ./chart -n firecrawl \
  --set worker.replicaCount=3
```

## Future Enhancements
- [ ] Add Prometheus metrics and ServiceMonitor
- [ ] Implement resource quotas and LimitRanges
- [ ] Add PodDisruptionBudgets for HA
- [ ] Configure NetworkPolicies for security
- [ ] Add external secrets management (HashiCorp Vault or Sealed Secrets)
- [ ] Implement horizontal pod autoscaling
- [ ] Add extract worker deployment
- [ ] Add nuq worker deployment

## Support

For issues or questions:
- Check ArgoCD application status
- Review pod logs
- Verify storage bindings
- Consult Firecrawl documentation: https://docs.firecrawl.dev