fix(llm-inference): bitsandbytes int4 OOM — pending switch to llama.cpp+GGUF

bitsandbytes quantizes on-the-fly: loads full bf16 weights (~54GB RAM peak)
before compressing to int4. Kills the 40GB OptiPlex on torch.compile warmup.

Fix in next commit: switch serve phase to llama.cpp + GGUF Q4_K_M.
Pre-quantized weights load directly — peak RAM ~16GB, no compile overhead.

ansible/roles/llm-inference/defaults/main.yml

@@ -20,8 +20,9 @@ llm_hf_model: google/gemma-2-27b-it
 # vLLM serve
 llm_serve_port: 8000
 llm_serve_host: "0.0.0.0"
-llm_gpu_memory_utilization: "0.90"
-llm_max_model_len: 8192
+llm_quantization: "bitsandbytes"   # int4 — fits 27B in 24GB VRAM (~14GB vs ~54GB bf16)
+llm_gpu_memory_utilization: "0.92" # higher utilization to give KV cache room
+llm_max_model_len: 4096            # safe ceiling given 15.96GB model + int4
 
 # Monitoring
 llm_gpu_exporter_version: "1.13.1"