6023ee25e1
- Created PersistentVolumeClaim for PostgreSQL (10GB, nfs-emporium) - Created ConfigMaps for API and Playwright service configuration - Created 7 Deployment manifests: * firecrawl-api (2 CPU, 4-6GB RAM) * firecrawl-api-worker (1 CPU, 3-4GB RAM) * firecrawl-api-nuq-worker (1 CPU, 3-4GB RAM) * firecrawl-playwright (2 CPU, 4GB RAM, 1GB tmpfs) * nuq-postgres (1 CPU, 2GB RAM, 10GB PVC) * redis (0.5 CPU, 1GB RAM) * rabbitmq (0.5 CPU, 1GB RAM) - Created 5 ClusterIP Services for inter-service communication - Created HTTPRoute for external access via Gateway API * Primary hostname: spaceship-earth.local.mk-labs.cloud * Alias: firecrawl.local.mk-labs.cloud - All manifests validated with kubectl dry-run=client Next steps (Day 4): Configure ExternalSecrets for 1Password integration Next steps (Day 5): Deploy to cluster and verify functionality Total resources: 8 CPU, 22GB RAM, 10GB storage
46 lines
1.1 KiB
YAML
46 lines
1.1 KiB
YAML
apiVersion: v1
|
|
kind: ConfigMap
|
|
metadata:
|
|
name: firecrawl-config
|
|
namespace: firecrawl
|
|
labels:
|
|
app.kubernetes.io/name: firecrawl
|
|
app.kubernetes.io/component: configuration
|
|
app.kubernetes.io/part-of: firecrawl
|
|
data:
|
|
# Service URLs (Kubernetes internal DNS)
|
|
REDIS_URL: "redis://redis:6379"
|
|
REDIS_RATE_LIMIT_URL: "redis://redis:6379"
|
|
POSTGRES_HOST: "nuq-postgres"
|
|
POSTGRES_PORT: "5432"
|
|
POSTGRES_USER: "postgres"
|
|
POSTGRES_DB: "postgres"
|
|
NUQ_RABBITMQ_URL: "amqp://rabbitmq:5672"
|
|
PLAYWRIGHT_MICROSERVICE_URL: "http://playwright-service:3000/scrape"
|
|
|
|
# Server Configuration
|
|
HOST: "0.0.0.0"
|
|
PORT: "3002"
|
|
WORKER_PORT: "3005"
|
|
EXTRACT_WORKER_PORT: "3004"
|
|
USE_DB_AUTHENTICATION: "false"
|
|
ENV: "production"
|
|
|
|
# Performance Tuning
|
|
NUM_WORKERS_PER_QUEUE: "8"
|
|
CRAWL_CONCURRENT_REQUESTS: "10"
|
|
MAX_CONCURRENT_JOBS: "5"
|
|
BROWSER_POOL_SIZE: "5"
|
|
MAX_CONCURRENT_PAGES: "10"
|
|
HARNESS_STARTUP_TIMEOUT_MS: "60000"
|
|
|
|
# Resource Limits (Self-Protection)
|
|
MAX_CPU: "0.8"
|
|
MAX_RAM: "0.8"
|
|
|
|
# Logging
|
|
LOGGING_LEVEL: "info"
|
|
|
|
# Security
|
|
ALLOW_LOCAL_WEBHOOKS: "false"
|