rblundon/homelab
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update terraform to use vm_id from n8n.

Browse Source
This commit is contained in:
Ryan Blundon
2026-02-28 00:00:56 -06:00
parent 4964552483
commit d941557e88
8 changed files with 267 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
.ansible/.lock Normal file
View File

BIN
docs/mk-labs-vm-provisioning-prd.docx Normal file
View File

Binary file not shown.

66
docs/n8n-setup.md Normal file
View File

@@ -0,0 +1,66 @@
# n8n Setup & Dependencies
**Host:** tiki-room (10.1.71.23)
**Install method:** Proxmox LXC via helper script
**Version:** Community Edition
## Community Node Dependencies
The VM provisioning pipeline requires the following n8n community node:
### n8n-nodes-globals
- **Package:** [n8n-nodes-globals](https://github.com/umanamente/n8n-nodes-globals)
- **Purpose:** Stores secrets (API tokens, credentials) as encrypted global constants in n8n's credential store. Referenced by pipeline workflows to pass secrets to Terraform and NetBox API calls without persisting them on disk.
- **Install:**
```bash
cd /usr/lib/node_modules/n8n
npm install n8n-nodes-globals
systemctl restart n8n
```
- **Version at time of writing:** v1.1.0
### Configuration
After installing the community node, create a **Global Constants** credential in n8n with the following key-value pairs:
| Key | Description |
|-----|-------------|
| `PROXMOX_API_URL` | Proxmox API endpoint (e.g., `https://fantasyland.local.mk-labs.cloud:8006`) |
| `PROXMOX_API_TOKEN` | Proxmox API token in format `user@realm!token-name=secret` |
| `UNIFI_USERNAME` | UDM Pro admin username |
| `UNIFI_PASSWORD` | UDM Pro admin password |
| `UNIFI_API_URL` | UDM Pro API URL (e.g., `https://10.1.0.1:443`) |
| `NETBOX_URL` | NetBox base URL (e.g., `https://fire-station`) |
| `NETBOX_API_TOKEN` | NetBox REST API token |
These constants are encrypted at rest in n8n's database and are accessible in any workflow via the Global Constants node. This avoids storing secrets on city-hall's filesystem or in Terraform `.tfvars` files.
## n8n Credentials
In addition to global constants, the following n8n credentials are used by pipeline nodes:
| Credential Type | Name | Used By | Details |
|----------------|------|---------|---------|
| SSH | city-hall SSH | Terraform apply nodes | SSH access to 10.1.71.35 for running Terraform commands |
| Header Auth | NetBox API Token | HTTP Request nodes | `Authorization: Token <token>` for NetBox API calls |
## Secrets Management Roadmap
The current approach (n8n Global Constants) is a Phase 1 solution. Future plans:
- **Phase 6+:** Evaluate HashiCorp Vault or equivalent for centralized secrets management
- **Note:** n8n Community Edition may not support external secrets store integrations — verify before planning migration
## Webhook Configuration
The pipeline is triggered by a NetBox webhook:
| Setting | Value |
|---------|-------|
| Webhook URL | `http://tiki-room:5678/webhook/vm-provision` |
| HTTP Method | POST |
| Content Type | `application/json` |
| SSL Verification | Disabled (internal VLAN) |
See NetBox Event Rule configuration in `netbox/initializers/custom_fields.yml` for the trigger conditions.

182
docs/vm-provisioning-flow.md Normal file
View File

@@ -0,0 +1,182 @@
# ADR: VM Provisioning Pipeline Flow
**Status:** Accepted
**Date:** February 2026
**Supersedes:** PRD v1.0 Section 3.2 (adds DHCP step, adjusts tool boundaries)
**Last Updated:** February 28, 2026 — revised flow for DHCP-before-boot, HA affinity via Proxmox API, secrets via n8n globals
## Context
The mk-labs homelab needs an automated VM provisioning pipeline triggered by a single operator action. The original PRD defined the flow; this ADR documents the adjusted architecture where n8n owns all orchestration including DHCP reservation (previously Ansible-rendered Terraform templates).
## Decision
### System Context
| Tool | Host | IP Address | Role in Pipeline |
|------|------|-----------|-----------------|
| NetBox | fire-station | 10.1.71.102 | Source of truth — VM records, IP allocation, VLAN data, webhook emitter |
| n8n | tiki-room | 10.1.71.23 | Event orchestration, payload validation, pipeline sequencing |
| Terraform | city-hall | 10.1.71.35 | Proxmox VM lifecycle + Unifi DHCP reservations (repo: /opt/git/homelab) |
| Ansible / Semaphore | imagineering | 10.1.71.22 | OS configuration, DNS record creation, NetBox status updates |
| Proxmox | fantasyland | 10.1.71.13 | Target hypervisor for all new VM deployments |
### Status State Machine
```
PLANNED →(operator)→ STAGED →(pipeline)→ ACTIVE
|
on error
↓
FAILED
```
- **Planned**: All fields populated, operator not yet ready. No trigger.
- **Staged**: Operator validated and ready. Triggers pipeline.
- **Active**: Set by Ansible on successful OS config + DNS registration.
- **Failed**: Set by n8n or Ansible on any error. Manual investigation.
### Provisioning Flow (Revised)
| # | System | Action | Detail |
|---|--------|--------|--------|
| **1** | NetBox | Operator sets VM status to Staged | Triggers webhook to n8n on tiki-room |
| **2** | n8n | Receive & validate webhook payload | Check: hostname, IP, template, proxmox_node, proxmox_datastore all present. Status must be exactly Staged. On failure → set NetBox status to Failed, stop. |
| **3** | n8n → city-hall | SSH to city-hall, run `terraform apply` (proxmox/vm) | Creates VM clone on fantasyland in **stopped** state. Uses `-state=states/{hostname}.tfstate` for per-VM state isolation. Passes secrets from n8n Global Constants via `-var` flags. |
| **4** | n8n | Parse Terraform output for MAC address | Reads `mac_address` from `terraform output -json`. |
| **5** | n8n → NetBox | Write MAC address to VM interface | PATCH to NetBox API to update VM interface record with MAC. |
| **6** | n8n → city-hall | SSH to city-hall, run `terraform apply` (unifi/dhcp) | Creates static DHCP reservation on UDM Pro. Uses `-state=states/{hostname}.tfstate`. MAC and IP passed as variables. |
| **7** | n8n → Proxmox API | Add VM to HA resource + node affinity rule | POST to `/cluster/ha/resources` to register VM as HA resource. PUT to `/cluster/ha/rules/{prefer_node}` to add VM to the target node's affinity rule. |
| **8** | n8n → Proxmox API | Start VM | POST to `/nodes/{node}/qemu/{vmid}/status/start`. VM boots and receives IP via DHCP (reservation already in place). |
| **9** | n8n → imagineering | Call Semaphore API to trigger Ansible | Launches vm-provision playbook with VM details as extra vars. |
| **10** | Ansible | OS configuration & DNS record creation | Applies vm-baseline role: packages, SSH keys, Technitium DNS A record creation. |
| **11** | Ansible → NetBox | Set VM status to Active | Final step of playbook updates NetBox VM status via API. |
| **ERR** | n8n / Ansible | Any step fails | NetBox VM status set to Failed. Pipeline halts. No cleanup — operator investigates manually. |
### Key Design Decisions
**VM created in stopped state (Step 3)**
Cloud-init uses `ip=dhcp`. The DHCP reservation must exist on the UDM Pro before the VM's first boot, otherwise the VM gets a random DHCP lease instead of its assigned IP. The VM stays stopped until after the Unifi DHCP reservation is confirmed (Step 6).
**HA affinity via Proxmox API, not Terraform (Step 7)**
Proxmox 9.x replaced HA groups with HA node affinity rules. The bpg/proxmox Terraform provider does not yet support these (tracked in bpg/terraform-provider-proxmox#2097). n8n calls the Proxmox API directly to add the VM to the appropriate `prefer_{node}` affinity rule. This also handles VM migration from fantasyland (clone source) to the target node.
**Per-VM Terraform state files (Steps 3, 6)**
Each VM gets its own state file via `-state=states/{hostname}.tfstate` to avoid state conflicts when managing multiple VMs. The `states/` directory is in `.gitignore`.
**Secrets via n8n Global Constants**
All secrets (Proxmox API token, Unifi credentials, NetBox API token) are stored in n8n's encrypted credential store using the `n8n-nodes-globals` community node. Secrets are passed to Terraform via `-var` flags in SSH commands. No secrets are stored on city-hall's filesystem. See `docs/n8n-setup.md` for details.
### Tool Responsibility Boundary (Adjusted)
| Tool | Owns | Does Not Own |
|------|------|-------------|
| **Terraform** | Proxmox VM lifecycle (bpg/proxmox), UDM Pro DHCP reservations (unifi provider) | OS config, DNS records, application deployment, NetBox data, HA placement |
| **Ansible** | OS configuration, DNS A records (Technitium API), NetBox status updates | Infrastructure provisioning, DHCP reservations, event orchestration |
| **n8n** | Event orchestration, API glue, pipeline sequencing, HA affinity rule management, triggering all Terraform and Ansible steps, secrets distribution | Configuration management, infrastructure state |
| **NetBox** | Source of truth for all assets | Execution of any automation — it only holds data |
### Dependencies
| Component | Dependency | Purpose |
|-----------|-----------|---------|
| n8n | n8n-nodes-globals (community node) | Encrypted secrets storage for pipeline credentials |
| Terraform (proxmox/vm) | bpg/proxmox provider >= 0.74.0 | Proxmox VM lifecycle management |
| Terraform (unifi/dhcp) | paultyng/unifi provider >= 0.41.0 | UDM Pro DHCP static reservations |
| Proxmox | API token: terraform@pve!terraform-token | Terraform + n8n API access (requires VM.GuestAgent.Audit for Proxmox 9) |
| NetBox | Event Rule + Webhook (VM Staged Trigger) | Pipeline trigger on status change to Staged |
| NetBox | REST API token | n8n reads/writes VM data |
### Change from PRD v1.0
**What changed (original):** DHCP reservation moved from "existing Terraform/UDM Pro integration" (out of scope) to an explicit pipeline step owned by n8n.
**What changed (this revision):**
1. VM is created in **stopped** state. DHCP reservation is created before VM boots.
2. HA node affinity rule assignment added as an explicit step (Proxmox API, not Terraform).
3. VM start is a separate explicit step after HA rule assignment.
4. Secrets managed via n8n Global Constants instead of filesystem-based `.tfvars` files.
5. Per-VM Terraform state files via `-state` flag.
**Why:** Cloud-init uses `ip=dhcp`, so the DHCP reservation must exist before first boot. The bpg/proxmox Terraform provider doesn't support Proxmox 9 HA affinity rules, so n8n handles placement via API. Secrets in n8n's credential store keeps city-hall stateless.
### Git Repository Structure
```
homelab/
├── ansible/
│ ├── inventory/
│ │ ├── netbox.yml # NetBox dynamic inventory plugin
│ │ └── static.yml # Legacy static inventory
│ ├── playbooks/
│ │ ├── vm-provision.yml # Main playbook (Semaphore trigger)
│ │ ├── add_technitium_dns_entry.yml
│ │ ├── add_dhcp_reservation.yml
│ │ ├── update-linux-os.yml
│ │ └── set_hostname.yml
│ ├── roles/
│ │ ├── vm-baseline/ # NEW — packages, SSH, DNS, NetBox status
│ │ ├── dns-manager/ # Existing Technitium DNS role
│ │ ├── common/ # Base system config
│ │ ├── haproxy/
│ │ ├── n8n/
│ │ ├── observer/
│ │ └── time-sync/
│ ├── tasks/ # Shared task files
│ ├── group_vars/
│ ├── host_vars/
│ ├── templates/
│ ├── scripts/
│ ├── ansible.cfg
│ └── requirements.yml
│
├── terraform/
│ ├── proxmox/
│ │ └── vm/
│ │ ├── main.tf # bpg/proxmox VM resource (creates stopped)
│ │ ├── variables.tf # hostname, ip, template, proxmox_node
│ │ ├── outputs.tf # vm_id, mac_address
│ │ ├── providers.tf
│ │ ├── terraform.tfvars.example
│ │ └── states/ # Per-VM state files (.gitignore)
│ ├── unifi/
│ │ └── dhcp/
│ │ ├── main.tf # Unifi user resource (DHCP reservation)
│ │ ├── variables.tf # hostname, mac, ip
│ │ ├── providers.tf
│ │ ├── terraform.tfvars.example
│ │ └── states/ # Per-VM state files (.gitignore)
│ └── dns/
│ ├── dns.tf # Existing DNS terraform
│ └── provider.tf
│
├── packer/
│ ├── ubuntu-24.04/ # Ubuntu template build
│ └── fedora-42/ # Fedora template build
│
├── n8n/
│ └── workflows/
│ └── vm-provisioning.json # Exported n8n workflow
│
├── netbox/
│ └── initializers/
│ ├── custom_fields.yml
│ ├── vlans.yml
│ └── prefixes.yml
│
├── docs/
│ ├── decisions/
│ │ ├── vm-provisioning-flow.md # This document
│ │ └── tool-responsibility-matrix.md
│ └── n8n-setup.md # n8n dependencies and configuration
│
├── README.md
├── LICENSE
└── .gitignore
```
## Consequences
- **Positive:** Single visual workflow in n8n shows entire pipeline. Ansible simplified to pure configuration management. Terraform usage consistent (always invoked via n8n SSH, never rendered from templates). Secrets never stored on disk outside n8n's encrypted database. Per-VM state files prevent conflicts.
- **Negative:** n8n becomes a critical dependency — if tiki-room is down, no provisioning. Two separate `terraform apply` calls per VM (Proxmox + Unifi) instead of one. HA rule management is imperative API calls rather than declarative Terraform.
- **Mitigated by:** n8n workflow JSON committed to Git. Both Terraform modules can be run manually from city-hall if n8n is unavailable. HA rules can be managed via `ha-manager` CLI on any Proxmox node.

14
terraform/proxmox/vm/main.tf
View File

@@ -1,12 +1,7 @@
# ─── VM ID Derivation ────────────────────────────────────────────────────────
# Convention carried from existing Ansible playbooks:
# ─── VM ID Convention ────────────────────────────────────────────────────────
# VM ID = {third_octet}{fourth_octet:03d}
# Example: 10.1.71.35 → 71035
locals {
ip_octets = split(".", var.ip_address)
vm_id = tonumber("${local.ip_octets[2]}${format("%03d", tonumber(local.ip_octets[3]))}")
}
# Computed by n8n and passed as a variable to keep the logic in one place.
# ─── Template Lookup ─────────────────────────────────────────────────────────
# Find the template VM ID by name on the clone node
@@ -29,8 +24,7 @@ data "proxmox_virtual_environment_vms" "templates" {
resource "proxmox_virtual_environment_vm" "vm" {
name = var.hostname
bios = "ovmf"
vm_id = local.vm_id
vm_id = var.vm_id
node_name = var.target_node
description = "Provisioned by mk-labs pipeline on ${timestamp()}"
@@ -65,7 +59,7 @@ resource "proxmox_virtual_environment_vm" "vm" {
network_device {
bridge = var.bridge
model = "virtio"
# vlan_id = var.vlan_id
vlan_id = var.vlan_id
}
# VM stays STOPPED after clone.

2
terraform/proxmox/vm/outputs.tf
View File

@@ -1,6 +1,6 @@
output "vm_id" {
description = "Proxmox VM ID of the created VM."
value = proxmox_virtual_environment_vm.vm.vm_id
value = var.vm_id
}
output "mac_address" {

16
terraform/proxmox/vm/terraform.tfvars.example
View File

@@ -1,13 +1,11 @@
# Copy this file to terraform.tfvars and fill in the values.
# terraform.tfvars is in .gitignore — never commit secrets.
proxmox_api_url = "https://fantasyland.mk-labs.net:8006"
proxmox_api_token = "terraform@pve!terraform-token=your-token-secret-here"
# These values come from n8n via -var flags at runtime:
# All values come from n8n via -var flags at runtime.
# Secrets are stored in n8n Global Constants — no terraform.tfvars file needed.
#
# vm_id = 71200
# hostname = "test-vm"
# ip_address = "10.1.71.100"
# ip_address = "10.1.71.200"
# target_node = "fantasyland"
# template_name = "ubuntu-24.04-small"
# datastore = "liberty-tree"
# vlan_id = 71
# proxmox_api_url = "https://fantasyland.local.mk-labs.cloud:8006"
# proxmox_api_token = "terraform@pve!terraform-token=your-token-secret-here"

9
terraform/proxmox/vm/variables.tf
View File

@@ -2,7 +2,7 @@
variable "proxmox_api_url" {
type = string
description = "Proxmox API endpoint URL (e.g., https://fantasyland.local.mk-labs.cloud:8006)"
description = "Proxmox API endpoint URL (e.g., https://fantasyland.mk-labs.net:8006)"
}
variable "proxmox_api_token" {
@@ -13,6 +13,11 @@ variable "proxmox_api_token" {
# ─── VM Identity ─────────────────────────────────────────────────────────────
variable "vm_id" {
type = number
description = "Proxmox VM ID. Computed by n8n from IP: {third_octet}{fourth_octet:03d} (e.g., 10.1.71.200 → 71200)."
}
variable "hostname" {
type = string
description = "VM hostname. Used as the Proxmox VM name and cloud-init hostname."
@@ -26,7 +31,7 @@ variable "ip_address" {
variable "dns_servers" {
type = list(string)
description = "DNS servers for the VM."
default = ["10.1.71.1"]
default = ["10.1.71.102", "10.1.71.1"]
}
variable "search_domain" {