Add shared nginx-proxy role; deploy Hermes in Docker; rename Paperclip

nginx-proxy (new role):
- Shared nginx + certbot under /opt/nginx, replacing the copies that
  lived inside the Paperclip compose project
- Apps publish via tasks_from: add_vhost (cert marker -> issue cert ->
  write vhost -> nginx -t -> reload); a vhost that fails nginx -t is
  rolled back
- Certs issued one per domain via Cloudflare DNS challenge; domains
  registered as marker files in cert-domains/; optional LE staging
- Stock nginx.conf plus conf.d (directory mounts, not single-file
  mounts); catch-all server drops unknown Host/SNI
- Containers renamed to nginx and certbot

hermes_docker (new role):
- Official Hermes image, host networking, gateway and dashboard bound
  to loopback and published through nginx-proxy
- Dashboard login via basic-auth provider, secrets from the vault
- Main model set from hermes_model (fantasmic endpoint, key from vault);
  config.yaml overrides are merged from a 0600 staged file that is
  removed afterwards, so the key never appears in a command line
- Data dir owner-only (HERMES_HOME_MODE=0700)

paperclip-docker:
- Remove nginx, certbot and docker-compose download tasks; deploy with
  community.docker.docker_compose_v2 (idempotent, remove_orphans)
- Stop passing the Cloudflare token and certbot email to the Paperclip
  container via env_file
- Bind Postgres to 127.0.0.1 (host networking exposed it on all
  interfaces); stop chowning pgdata, which the postgres image owns
- Handler recreates the paperclip service so .env changes apply
- Domain m-v.wdw-labs.cloud -> cranium-command.wdw-labs.cloud

Playbooks:
- Add day1_deploy_nginx_proxy.yml and day1_deploy_hermes_docker.yml
- deploy_paperclip_stack.yml: run nginx-proxy before paperclip-docker,
  CLI-based container checks (no Docker SDK dependency)
- Remove deploy-hermes.yml

Host vars: add nginx_proxy_certbot_email and nginx_proxy_certbot_staging
(staging stays on until the switch to real certs).

ansible/roles/hermes_docker/defaults/main.yml

@@ -0,0 +1,91 @@
+---
+hermes_image: nousresearch/hermes-agent:latest
+# missing = only pull if absent (no surprise upgrades on redeploy).
+# Set to "always" when you actually want to upgrade the image.
+hermes_pull_policy: missing
+
+hermes_container_name: hermes
+hermes_project_dir: /opt/hermes
+hermes_data_dir: "{{ hermes_project_dir }}/data"
+# The container's hermes user. The image's start-up hook chowns the data
+# volume to it; the role only needs the UID to stage files it can read.
+hermes_uid: 10000
+# config.yaml, .env and sessions live here and can hold API keys, so keep it
+# owner-only. Passed to the container as HERMES_HOME_MODE (see compose).
+hermes_data_dir_mode: "0700"
+hermes_secrets_env_file: "{{ hermes_project_dir }}/hermes-secrets.env"
+
+# Same network namespace as Paperclip and nginx
+hermes_network_mode: host
+hermes_memory_limit: 8G
+hermes_cpu_limit: "4.0"
+# Playwright/Chromium needs shared memory (upstream troubleshooting note)
+hermes_shm_size: 1g
+
+# --- Gateway / API server (loopback only; Paperclip reaches it locally) ---
+hermes_gateway_port: 8642
+hermes_api_server_enabled: true
+hermes_api_server_host: 127.0.0.1
+# Leave empty to run keyless on loopback. If you set one, Paperclip (or any
+# other client) must send it as a bearer token.
+# e.g. hermes_api_server_key: "{{ vault_hermes_api_server_key }}"
+hermes_api_server_key: ""
+
+# --- Dashboard (bound to loopback, published through nginx) ---
+hermes_dashboard_enabled: true
+hermes_dashboard_host: 127.0.0.1
+hermes_dashboard_port: 9119
+hermes_domain: haunted-mansion.wdw-labs.cloud
+# Publish the dashboard through the nginx-proxy role (cert + vhost + reload).
+# Requires day1_deploy_nginx_proxy.yml to have run first.
+hermes_publish_vhost: true
+hermes_public_url: "https://{{ hermes_domain }}"
+hermes_dashboard_username: admin
+
+# Vault variables this role requires (see README block at bottom of file)
+hermes_required_vault_vars:
+  - vault_hermes_dashboard_password_hash
+  - vault_hermes_dashboard_secret
+
+# --- Preflight ---
+hermes_preflight_ports:
+  - "{{ hermes_gateway_port }}"
+  - "{{ hermes_dashboard_port }}"
+hermes_wait_timeout: 120
+
+# --- config.yaml overrides (deep-merged, only the keys listed here) ---
+# nginx runs with host networking, so it reaches the dashboard from loopback.
+# Model settings are NOT managed by default (your wizard-written config stays
+# untouched). To manage them, add a model block, for example:
+#   model:
+#     provider: custom
+#     model: <served-model-name>
+#     base_url: http://10.1.71.130:8000/v1
+#     api_key: "{{ vault_vllm_api_key }}"
+hermes_config_overrides:
+  dashboard:
+    trusted_proxies:
+      - "127.0.0.1"
+
+# Main model (written to config.yaml under model:). Leave empty to manage it
+# by hand with `hermes model` or the dashboard. Example:
+#   hermes_model:
+#     provider: custom
+#     default: <served model name from the endpoint's /v1/models>
+#     base_url: http://fantasmic.wdw-labs.cloud:8000/v1
+#     api_key: "{{ vault_fantasmic_api_key }}"
+#     # context_length: 65536   # only if Hermes complains about the context window
+# The api_key lands in config.yaml in plain text (owner-only, see hermes_data_dir_mode).
+hermes_model: {}
+
+# What the merge task actually applies: base overrides + the model block.
+hermes_effective_overrides: >-
+  {{ hermes_config_overrides
+     | combine({'model': hermes_model} if (hermes_model | length > 0) else {}, recursive=True) }}
+
+# Required vault vars:
+#   vault_hermes_dashboard_password_hash  - output of hash_password() (see below)
+#   vault_hermes_dashboard_secret         - openssl rand -base64 32
+# Generate the hash with:
+#   docker exec -it -w /opt/hermes hermes /opt/hermes/.venv/bin/python -c \
+#     "from plugins.dashboard_auth.basic import hash_password; print(hash_password('PW'))"

ansible/roles/hermes_docker/files/merge_config.py

@@ -0,0 +1,40 @@
+"""Deep-merge overrides (a JSON file, path in argv[1]) into config.yaml.
+
+Run inside the hermes container as the hermes user. Only keys present in the
+overrides are touched. Prints CHANGED when the file was rewritten.
+"""
+import json
+import os
+import sys
+
+import yaml
+
+PATH = "/opt/data/config.yaml"
+with open(sys.argv[1]) as fh:
+    overrides = json.load(fh)
+
+if os.path.exists(PATH):
+    with open(PATH) as fh:
+        current = yaml.safe_load(fh) or {}
+else:
+    current = {}
+
+
+def merge(dst, src):
+    changed = False
+    for key, val in src.items():
+        if isinstance(val, dict) and isinstance(dst.get(key), dict):
+            changed |= merge(dst[key], val)
+        elif dst.get(key) != val:
+            dst[key] = val
+            changed = True
+    return changed
+
+
+if merge(current, overrides):
+    with open(PATH, "w") as fh:
+        yaml.safe_dump(current, fh, sort_keys=False)
+    print("CHANGED")
+else:
+    print("UNCHANGED")
+sys.exit(0)

ansible/roles/hermes_docker/handlers/main.yml

@@ -0,0 +1,5 @@
+---
+# Config.yaml is read at start, so a plain restart is enough.
+# Compose/env changes are handled by docker compose itself (it recreates).
+- name: Restart hermes
+  ansible.builtin.command: "docker restart {{ hermes_container_name }}"

ansible/roles/hermes_docker/tasks/main.yml

@@ -0,0 +1,148 @@
+---
+- name: Preflight | Required vault variables are defined
+  ansible.builtin.assert:
+    that: "{{ item }} is defined and ({{ item }} | length) > 0"
+    fail_msg: "{{ item }} is not defined. Add it to your ansible vault."
+    quiet: true
+  loop: "{{ hermes_required_vault_vars }}"
+  tags: [hermes, preflight]
+
+- name: Preflight | Confirm docker CLI is available
+  ansible.builtin.command: docker --version
+  changed_when: false
+  tags: [hermes, preflight]
+
+- name: Preflight | Get existing container state
+  ansible.builtin.command: "docker inspect -f '{% raw %}{{.State.Running}}{% endraw %}' {{ hermes_container_name }}"
+  register: hermes_existing
+  failed_when: false
+  changed_when: false
+  tags: [hermes, preflight]
+
+# Only enforce free ports when our container is not already running,
+# so redeploys stay idempotent.
+- name: Preflight | Ensure required ports are free
+  ansible.builtin.wait_for:
+    host: 127.0.0.1
+    port: "{{ item }}"
+    state: stopped
+    timeout: 3
+  loop: "{{ hermes_preflight_ports }}"
+  when: hermes_existing.stdout | default('') | trim != 'true'
+  tags: [hermes, preflight]
+
+- name: Create project directory
+  ansible.builtin.file:
+    path: "{{ hermes_project_dir }}"
+    state: directory
+    mode: "0755"
+  tags: [hermes]
+
+# No owner here: the image's start-up hook chowns the volume to the hermes user.
+- name: Create data directory (owner-only)
+  ansible.builtin.file:
+    path: "{{ hermes_data_dir }}"
+    state: directory
+    mode: "{{ hermes_data_dir_mode }}"
+  tags: [hermes]
+
+# Compose reads this as root on the host; the container never sees the file.
+- name: Template secrets env file
+  ansible.builtin.template:
+    src: hermes-secrets.env.j2
+    dest: "{{ hermes_secrets_env_file }}"
+    owner: root
+    group: root
+    mode: "0600"
+  no_log: true
+  tags: [hermes]
+
+- name: Template docker compose file
+  ansible.builtin.template:
+    src: docker-compose.yml.j2
+    dest: "{{ hermes_project_dir }}/docker-compose.yml"
+    mode: "0644"
+  tags: [hermes]
+
+- name: Deploy hermes
+  community.docker.docker_compose_v2:
+    project_src: "{{ hermes_project_dir }}"
+    pull: "{{ hermes_pull_policy }}"
+    state: present
+  tags: [hermes]
+
+- name: Verify gateway health
+  ansible.builtin.uri:
+    url: "http://127.0.0.1:{{ hermes_gateway_port }}/health"
+    status_code: 200
+  register: hermes_health
+  until: hermes_health.status == 200
+  retries: "{{ (hermes_wait_timeout | int) // 5 }}"
+  delay: 5
+  when: hermes_api_server_enabled | bool
+  tags: [hermes, verify]
+
+- name: Wait for dashboard port
+  ansible.builtin.wait_for:
+    host: "{{ hermes_dashboard_host }}"
+    port: "{{ hermes_dashboard_port }}"
+    timeout: "{{ hermes_wait_timeout }}"
+  when: hermes_dashboard_enabled | bool
+  tags: [hermes, verify]
+
+# Overrides can contain secrets (the model api_key), so they are staged in a
+# 0600 file owned by the container user instead of travelling in a command
+# line (visible in `ps`), and removed again afterwards. The merge script runs
+# inside the container with the image's own venv (PyYAML is available there),
+# as the hermes user, so config.yaml stays owned by it.
+- name: Apply config.yaml overrides
+  when: hermes_effective_overrides | length > 0
+  tags: [hermes, config]
+  block:
+    - name: Stage overrides file
+      ansible.builtin.copy:
+        content: "{{ hermes_effective_overrides | to_json }}"
+        dest: "{{ hermes_data_dir }}/.ansible-overrides.json"
+        owner: "{{ hermes_uid }}"
+        group: "{{ hermes_uid }}"
+        mode: "0600"
+      changed_when: false   # transient file; only the merge result should count as a change
+      no_log: true
+
+    - name: Merge into config.yaml
+      ansible.builtin.command:
+        argv:
+          - docker
+          - exec
+          - -i
+          - --user
+          - hermes
+          - "{{ hermes_container_name }}"
+          - /opt/hermes/.venv/bin/python
+          - "-"
+          - /opt/data/.ansible-overrides.json
+        stdin: "{{ lookup('ansible.builtin.file', 'merge_config.py') }}"
+      register: hermes_config_merge
+      changed_when: "'CHANGED' in hermes_config_merge.stdout and 'UNCHANGED' not in hermes_config_merge.stdout"
+      notify: Restart hermes
+
+  always:
+    - name: Remove staged overrides file
+      ansible.builtin.file:
+        path: "{{ hermes_data_dir }}/.ansible-overrides.json"
+        state: absent
+      changed_when: false
+
+- name: Publish dashboard through nginx-proxy
+  ansible.builtin.include_role:
+    name: nginx-proxy
+    tasks_from: add_vhost
+  vars:
+    vhost_name: hermes
+    vhost_domain: "{{ hermes_domain }}"
+    vhost_upstream_port: "{{ hermes_dashboard_port }}"
+    vhost_proxy_timeout: 3600s   # long-lived dashboard WebSockets
+  when:
+    - hermes_publish_vhost | bool
+    - hermes_dashboard_enabled | bool
+  tags: [hermes, nginx]

ansible/roles/hermes_docker/templates/docker-compose.yml.j2

@@ -0,0 +1,31 @@
+# Managed by Ansible (hermes_docker role)
+services:
+  hermes:
+    image: {{ hermes_image }}
+    container_name: {{ hermes_container_name }}
+    restart: unless-stopped
+    network_mode: {{ hermes_network_mode }}
+    command: gateway run
+    shm_size: "{{ hermes_shm_size }}"
+    env_file:
+      - {{ hermes_secrets_env_file }}
+    environment:
+      HERMES_HOME_MODE: "{{ hermes_data_dir_mode }}"
+{% if hermes_dashboard_enabled | bool %}
+      HERMES_DASHBOARD: "1"
+      HERMES_DASHBOARD_HOST: "{{ hermes_dashboard_host }}"
+      HERMES_DASHBOARD_PORT: "{{ hermes_dashboard_port }}"
+      HERMES_DASHBOARD_PUBLIC_URL: "{{ hermes_public_url }}"
+{% endif %}
+{% if hermes_api_server_enabled | bool %}
+      API_SERVER_ENABLED: "true"
+      API_SERVER_HOST: "{{ hermes_api_server_host }}"
+      API_SERVER_PORT: "{{ hermes_gateway_port }}"
+{% endif %}
+    volumes:
+      - {{ hermes_data_dir }}:/opt/data
+    deploy:
+      resources:
+        limits:
+          memory: {{ hermes_memory_limit }}
+          cpus: "{{ hermes_cpu_limit }}"

ansible/roles/hermes_docker/templates/hermes-secrets.env.j2

@@ -0,0 +1,8 @@
+# Managed by Ansible (hermes_docker role) - do not edit
+# Single quotes keep compose from interpolating '$' inside hashes/secrets.
+HERMES_DASHBOARD_BASIC_AUTH_USERNAME={{ hermes_dashboard_username }}
+HERMES_DASHBOARD_BASIC_AUTH_PASSWORD_HASH='{{ vault_hermes_dashboard_password_hash }}'
+HERMES_DASHBOARD_BASIC_AUTH_SECRET='{{ vault_hermes_dashboard_secret }}'
+{% if hermes_api_server_key | length > 0 %}
+API_SERVER_KEY='{{ hermes_api_server_key }}'
+{% endif %}