llm-inference-multimodel: role + day1 playbook (phase 0 discover approved)

llm-inference: size ctx-size/parallel for aux task offload
Previous ctx-size=8192/parallel=4 gave 2048 tokens/slot, too small for context compression inputs (observed live rejection at 3826 tokens). Measured VRAM on astro-orbiter (RTX 3090 24GB): weights ~17GB resident, ~294KiB/token pool-wide for KV cache+buffers at prior sizing. New: ctx-size=16384, parallel=2 -> 8192 tokens/slot (matches model's native n_ctx_train max). Projected VRAM ~21.8GB, ~2.7GB headroom. Applied directly via ansible-playbook (Semaphore currently broken -- fix tracked separately).
2026-08-05 15:53:31 -05:00 · 2026-08-05 12:14:11 -05:00 · 2026-08-05 09:43:54 -05:00 · 2026-08-03 12:37:03 -05:00 · 2026-08-03 12:35:46 -05:00 · 2026-08-03 11:57:42 -05:00
35 changed files with 2193 additions and 375 deletions
--- a/ansible/ansible.cfg
+++ b/ansible/ansible.cfg
@@ -262,7 +262,7 @@ roles_path=./roles

 # (path) The vault password file to use. Equivalent to ``--vault-password-file`` or ``--vault-id``.
 # If executable, it will be run and the resulting stdout will be used as the password.
-;vault_password_file=
+vault_password_file=/home/hermes/.vault_pass.txt

 # (integer) Sets the default verbosity, equivalent to the number of ``-v`` passed in the command line.
 ;verbosity=0
--- a/ansible/group_vars/all/vault
+++ b/ansible/group_vars/all/vault
@@ -1,361 +1,365 @@
 $ANSIBLE_VAULT;1.1;AES256
-63623430346462366164333638633439646431326632346630383066333138313438643039373630
-6230333230356433343761666335323934656463653737360a376337346163316334333266346639
-30366436623165323462623330633935313438303761343666393162303466373762623235333161
-3835323539376536300a623962323438326336343537316630626262336339353730316162623935
-64666430626164316163633064643263396238353231373438643432343535633966636430386632
-35653437383361373938353034306264633139373663353062363163623634333538623831633831
-64643261376662613763306664353566643462306431643963633261643732386131333565366539
-35663062633730326166616366323865396536363532643931386637626430373562626531376430
-39376634363634656337363038666336663365363562356664333139633833323562386562396166
-33636437346130326539623364343030326330646137653131383739663166666334623566386631
-64616239656662393536633134643163663537626435653561323066336662613536306363623536
-33396466383537613135353930353832653765313435666435313266336361656633356131653766
-30326132366535323638393137633466333337306466333265353635333762613731396663313839
-36363830653032373564306435626333646530656535396362346132663937336337376339633366
-65373630323631303062656339613536303932656135363437633934333764613939343830353432
-34313665666634336465646363356237306562653534613863353064336236343963393664323738
-38653237656534616562303439616332303134626565646261343131393162343433636232393665
-62343431343535646431363861643964643234343665343366363930303663303565636161393033
-63626638333763343139396465363165363632393664623838623065623466336136373333393263
-34316439656232653261653538396231346462653339353034313063633661376266363265636432
-64316336613736396266333937353833363230653132383339626635386563626131663063396633
-36333061396639653238373965613831653935333533386633613035643530376465366331373832
-32643936373630363539666463353839333335663763653530623037326263306636383233613037
-39376266323064633366326163666264393265306438616330613631663137303963353666353231
-30343565633235643731326136643132376135376639366139383466323333333732396231373534
-62366438376232613463653231376164643930356137336337323763633063613861636266666262
-31323934356237373964316531353537366236633162303661663964356464653561333166643432
-61376631653335633264393762636435376535316663333065313638343361366439383432656138
-38353737353865396233366231373439333863386161323532373537333934346431373664633138
-30646437643737663031613332636536313039313663653262633665633564363130643738333031
-33666232333734646531326266393934636430626332356266333461326238633262316364356336
-65326238316336323037633236383765323332633936633062353266623238633738336132383432
-61343432343035386130363462663239613166626661613332393163386232356237383464383762
-31323739643235326264316134306430356539613865353532663133313830303061396339353566
-35373062316237373232616662383663636236353365626665386532333336313864383837383035
-62653638306136316137663738616663636434303533336433373739313331616631343131353265
-37353430386330333562366132393537613465343363623838393165666538623131653164376434
-61376533353366613735626165353562626431313031633238343966623962323231626265366330
-34353165623864343031623565363235323763646638393762646565393465396534343937666335
-62643530643435343931666563396661653662396265303362663932333666383534656539663033
-61663533353333323733613933363433383033346463306430353634633466396564623837346133
-38636131393738636666313765353835343666353663396539623236313237303437666665303966
-38323364376665613731336334383535373835303138373036613665336636306364623464303963
-39393830646133363263653330306531633863333064663466346135316161376362653862313132
-61333830303765386138303865323766353663656531643965356331633532343435633461366333
-61343534613166656639643363333231653738353936393938376665616136626361653432326531
-64633861653738616237313061393132383264313964643764366534303263653233393035363435
-31376164356466323431383836326334616130363264393139316362333130643262643766376331
-35666130303636363666363932616134306665313339323231306666353236353965356332343430
-36636263353565396535383862336464646134346566626635363063613761353333326137646662
-32656535356332626166636432643635353734393564376161316134613561303931663366616366
-35393166393632326564393763366161663636336362303566316162326631663937336437623637
-39613832666462613963643834313932323837393337633735633833646265666666653339353832
-37636332633063636133663938306234316666393735643634303833666537363932323363613466
-37626235626633383639663262653138613038363737323639306234366562666236326332323932
-63323336623633303736306362326237636465666465616361666130363538643135616237316234
-37623562396363326237663665636539346166323566386365376665616236653735646637363939
-64656662353233323065303438306436623735643934613839333133356564363438653133333037
-33656238663733663665373463393038303865613838366538356538383539396262656561353563
-31666333316564336262366435306261613563376462356338613063343232393766653332643237
-34636639333962393663333566643863386261666630393362356138623732323435666534376561
-61366363313861376566316235326633383630316338346333623432303766663130333130303363
-66616336613364666165393964666665383330316232316465663561613239386434663135613231
-62623334383433373461383037343037663236646463393632636161336562663533636235663039
-65383330373039386361633363613035636135373765353331636139313838353231353731346336
-63323736636130383962363762343437383237373533396433343464383235393030373862346233
-36646238623165353264323162316362376338383463663664636132373963623938373537373037
-30353262626363666565386536626163393130306634343836663137653537393066333637313639
-31376336373565313630636436616635623633396333313036366530613630663332323134613961
-36666438656636626561303637646539656166373861626232663266386533376531346166653834
-34656530363464343131303666616237373030393863326132313764373966633365313834316438
-34396239613235623233373136313464626661303666303438613337366662306363636439613332
-33626635613039363864343233323930333731376665346138626363396336373862636565613061
-30613863356637396133656663653232366265373766636237356539623639333133346361363361
-63343639353734313163346333356138343237383238623733343331376631636632373231363834
-62626432303732663435353762346161363330373266353031633862366539333133313931396166
-38373330323661616366343063363437333436626438376665333866376338626232383735613434
-64656363633332646533376535326566336566306439373835363264373335306564616337383465
-38353966636434653064326265643933653661363439363334383537623563303066346133353431
-62343662646434633761653332323430636339303666646262636334316461383865366436383231
-61313133316332363337343963306431356231306534636231643539383733383265303662373430
-34626666333830353934626139663532663332353938393038303631633335636236393133623064
-37306337386464656339383261323464303934373337333235343635326164653762613838333030
-39653063316131616566613365383961383435636639616536316662333736396238376661386533
-34643833323164303930656165396235353531326535313165663361653939656263633638633337
-36353435613335633239316435313965613439663061313834356661376365653763323837643561
-34333833626663353438623161366537326336646630613536653831306664663636343963313165
-65323332353633323931346634336331636465396337393035613834633962343662393930646337
-36346330353031636633326532616362646465386633363333613134653261323234636539633734
-63343965356433366233646536326333313366366537343934383239383439393633396461633166
-65633439336462363061316435396235346366386335666233343963383163393465633766643731
-62326362613266653238356435633032373036623839306132333136653537306365626165616638
-64316134316134393463313165376536383031366132363831333432323466653538376433333033
-65366438353935613863356562666264643063623439383335396531613730353861313664663234
-36626533366134643164353138306234333765623634316533383264303636353039643130366433
-38313133333632663666303862323939633463633634653963323866376137376137633930313963
-61336431656332333636373638316237663861336136393036653636366566316661303034623566
-31323537356338363762643265613534316466386263666236633038386634333139636561656331
-63613636316439393435383931306530306163386266373161666139373235653935393632313261
-36343066653561363333363830313136653538366639333464376562316334343839363332643137
-39336463613631346161616534613933613234656435383234313831366366643961393633633035
-39663935626134626138383038636266356434393535353165626632633430313635623532646165
-31363166373335326433363336326662663561306166313862373161633739643662376461346266
-61663730366337376165653033353466653761613938366532653633363530616134393464303138
-66353637623135626431643835613239633033386433663735356264333734613162343234633566
-64643039616137393863396433353138373862646138633838356365373863623235376638383439
-36373732373031346132363738383635653865346637626233663537613636613862323666616438
-34333336346235636234653062613466376637623165336337643933623535353033383438646530
-36653133336637346432343136336331336464633439383938353862323365343864666533316633
-66646262343534373165366266656135666339633333626133643238386139353537353866616631
-38306335663165363138303635383164303535386536626431663661396664376433656339373934
-38396130313438653539393263323265646163633939653431393035356332336664396234383636
-32323863366237633462316536626639343335626131323536373364393266386163633433356263
-36366665373432313132363761383632343532306361346134313464653763393533613565363839
-62623938346530636562383937393534653963636365303837336165343361313734303666623739
-30653834616163636538313038636261343434326164616264356261336239313039373661396535
-34346631643337666234373933383362656266346266393035363332303636363035656538636363
-39396466353565346264353663316562623239373564616162326439383334663730326435353236
-37366238653131356339306566326534616561653637613438666430336334323031373661353636
-64356131343763653639313531303866636334326637313838623138303836666466623736623237
-39363838653431323665366634303632666337363663386261326538313635313638383764383765
-34363832366135333436303635373761653265653165316135633135373365346266356635663533
-37303134613131653563356662363965353234643765373438313835343063313931626464636562
-36313434356434656138616433666266353538306666333363363431356530313262356438623537
-33303365393061393161626665316332626238353438623466373831303939663734663038346462
-39323462663732653335373261306635303231636563373465343761613262396466623135323736
-61316638313535653031633230333638343737633163636335343864336336373437643131643335
-63653766623436323039303065646162626331336637346237656132353039623063396631666535
-36316634623034326664313766326464383337663230643530623563326333336232666334323139
-33346563353734346338333432366437353665323764366335336232353163373563326265346234
-36383437383332366234393636323034376434336461336532613663316533653463633263343261
-62333535653062663663313534303237386562623533636230626531646461613233653330386466
-32623137633136323363386232393466303537643331656464663234636336393532313831313961
-37623939323963643364636439663032386630336235623037643065356631376637333062313762
-31653661316562643633643135616634353631623463353637616131343531633266393766336232
-30616138616437323138383732306537326562666239636561383631333534633937396463613765
-62313064653838623537393839386464653539326132393732393131633335393063393636376235
-64306566343762626238386632626136396331336564326361303835613732623531396631363564
-32616631333034313563396532396564663262613733626362663632333938336331633664376638
-34626466613239353935376230653732366236643830353963653662613338616631346561623063
-61616565393965613730326530353935323432306465656464626238623438636263666530336531
-64333864636131343566643164653934306135316437383738323738616362643636383761383433
-65353762663930636366376365623535333636613766666465316134303131623637343864643461
-33663934656631333032353734333361363339336266343631363937306632323635663832643266
-30383633323735333430376662663765303561643563383466333764636131363732636134376462
-37646564386564373038393135653761373937313039623737623938363631393632346565326166
-64376637323432303865366436356264633066343466333838303164373234646661346133646561
-62383838336461306638383266373462326165313161383832316462363835366361386663353935
-35616433393963613363333965666138363365666366646163336163613237386263343737636335
-36623864346566383632643035393361616533396538356562343364386330336636383433303538
-65653630303435373336366639656137386363303439333164633634323730393337333764633461
-32396161303864333362663466303161366465386338396332333462366339353461373638343961
-31343466663537636330663962636431366235646164643332343433393063633435646565333863
-37626239396264653434363538623133366561623436633739646335383534323839663465316339
-63653731636266343130383035323962333062623730343033613663366431386366623038373238
-32666537323237336236386135626339363861636439346664376561666538636532396562333435
-32383366373437303765653664396436623861333164396563623639636131656366316138396131
-61316432663239346539343261363461643632363432653161383233643431303034353136396465
-35623836393435396234336261626162326433356134636139303261653430363835333364656161
-34326533613636366134633332393064356535373432666165386163613961613539333932643963
-32343639343931623366633764363966636164323933623734333935613462303466656439306331
-36323261623639303164646435346533353966616138326230363739303063323366323964356534
-30316534656364373633663464366161373566626639363963356131353936333366326334663438
-32646463653664656635353564323230343763303538316135643838363134393634363031646530
-37323838386134316136663139626262396239643263353432613162666235616133623937383261
-31353138396238313066643964633536636337386238616637383561383563313936373538616334
-62363233623738613064313762303463303266346538333937343566343936326363323037623032
-30346434366661643930623761666134366132373936336630623963636331653532363765363263
-66306133303634343231386164343131333234376662313663393130326664386262363035353565
-37633863353935363635303039333662366462383235636564313666333162666563376666633966
-34666130613034393439346337383862643231666632663463393330356263646165316266613936
-62303536333339303431373531303065613865633339303264323137323732653036333366373664
-64623330313464356461376662663266333430633161393437363862633564396263353333316662
-64636262333832646135626534363737656336663562333365666631646261303665333964326337
-35643964386362323161666464363037623266386539383732626230303830303662366161653063
-36383539626334376238663539336637343033343037346136383733316265383430396133336139
-64616135623230613531313733626537366430626262623836386432363561626238303132613830
-36323533656164376334643436373730653531396236633933393663656435656566626165626164
-39366465646535383532616466313665333533386230656631343332376166613033343364393361
-62656237316466663131333661333565643335646166333332393263626463633639306264656666
-36343330666462373061363466636233333165663662393933343763633333373161626235363138
-64346433663661636561613636366662376562363638356231646438653532633434623132306330
-65656433353262623661636263393737623337316535326332373861393331626563326434383266
-65383931393032376137623833303031303934636135373135626562396364383635623661303431
-65623132393961396336363763616366633064366331316330396130373239336137373964356136
-37653162383030653736626564343739383030636531346534386639313063323938363236306537
-36646132626138643462636466656363353436633863613165613362616264613434303936633661
-38376661306439313036333431363664616662646533333963656165393939303865616661313864
-65656338633864663665303961613165386335646661343438333730313138616535363164336463
-35363563356362396265333334393964356534613763656362323631363237656438653962316336
-62633330616530376537333961383461323432303161343136326363313137333333323835313065
-64666132356632613365393137363061363165623733353534303638353432393739346164346535
-39363464313239386564336436663339303535336633666364373436666635613638643061316235
-61656266623934633339643063306363633131373335333538643338666339396166626262646434
-65623962396464343532366639393339306237653633623763333165303636613834333531306235
-36636430633864396236363638393165653431313139343966316561663138313761653630623436
-33363037303334383363646337303065343239343537343737616438613165323965333334613234
-62623931343365336230343037346230366134396435383236643931366631326335626431306366
-31646339323562653737303465353738343939623037616666656432313833653233373636326130
-30613434646237306636396239363439383961376539613131333164373333313131343062366638
-34333963313163303239333439393765386234313534303834636637393965323439366235613666
-65316336393335656332626165643261316464383232396136336238656662336132646366373263
-65396534353764616134663964376135623035316432623939376231373731343136313265323862
-61646238353761646434313430663362633066663763623735343730633036333533356339393665
-30313763346166653738636234663232323033636332373861373164393737353962323066363238
-34613066323862353831653438633564633561653034356263663430366432643663633836653764
-63333964663765353135636234323330623130633736653263336238396631356436346361366463
-65393239356432386163396135663161626664626137353465333733626635373537396137353437
-64626139393164313866616132386132303837303437306365336164383265346361383661323130
-62396333333666613538363332323032653037646563396436613166323166323666363934333064
-66356665306465653661323331376337653561613761643765666636396531663930333634316230
-35343733613638626230623464663133343432663764663864343363376339333566633963326630
-36313264636638396566623334333230363137353262346531623138656364653038373034353137
-31383533303234636333346332343333336531326363636161623764333762653835396632366436
-31646566363334633336366231363163643837633564363162656130323331626464333236303637
-31663439636234623534336535393835653838643430356364353762613966346461313032353939
-38613932386564623935363135663462626663313130663737373234316231343461616133626264
-31356163363234383132373730663436333738633562336435383532366138653764306165643863
-36383362633137643931333735333264656666336363663161643862323337613536383034336462
-35333165386332366436666439323236346539366530383136626637363539656230336637333432
-35336235363065306534643762346263393935333238383533646331653332643234386539653230
-30376237666166323638303465373737383031373864373266306537373335356139646230656264
-36316536363738323362643230363064363965636563346165356565643265353332386136346435
-66313937346164313661656364373465656230613261333833323335653133396132356334353237
-61353832373063666331646664623565353030663530636135336365633366623533653462386237
-65373462336538663835303831313336396638646534343839343933383032646664306434613230
-38323333666165363662376430356435363463303936396234313931303734666333376431376562
-65636132636435396562376637396464356165363937613435626235303730636239626465306237
-62613335396161303737396231666536353230656533653239353364623733623131333861626637
-64306437633564623565333664653030366166663835353063316631393864663338656663633736
-31346166656636396565333663353562613032373462366433323266396135363430623139373963
-32393864646438326366343036386231353530356233393539646562643339643366346563393162
-39393830646565326666333063396531356336653162623465313365616564653430663264363164
-62306562313037646466366463396432373536303631653338343133393262663865636133343464
-61383237386164633536663230346461383861626666623562313737373138633239333266366132
-32306135313464363461653363636462366662393938396566333661306366383838636438353738
-32356666613238356135353536656130346134663666346362353735613338666234626538373139
-65613262636233366532323630613438663334616164656166623236353531303835326266343663
-66366364333437373039306634383737663963373663366131396566613030386661656235616662
-35663533623864303431343631386362623036343135663665383039333363336534613063313534
-61373939306536376337303561373030343337326437393636366166376630346430666539646266
-34616435346139313033353130303639393530373662616432343330653762393339396432643935
-65313638383330613464363666633965393035643037626162616339306563646130306566613064
-35656130653035363163323831323730633236633736663965653833623131383037323736663066
-35343235356164623038643230373666366265363437396231613365303935366135636135613265
-31393163323837663332623636316462656338646566363361333863336365373562666434366530
-37626236343731313065353037346437356264363233363638393238373063386530646434663564
-35353562396564316535663436376335393136613338643864323132326431633066373462323330
-36306263646434333532316365613239376237303232383531343238623833626363346437386235
-66666565656635306162646430393738666132633065376330393162633066333534383561353336
-33313162666664613636303962656438626133373137336639356133363563653431646461313538
-37316433663263333536386632646536616564316362363238626231613162383966333063636132
-31623436346138613030396462626663616233326662663634633765643864373262393634353937
-32376437363737616337373333313832373135313535666430386536303236356563613166366332
-65323332626363663535633234323364663838333862643232376237653031303137303136633462
-32356431306534626135353462643736373939666130623732303562393932636231373534626132
-32343464353538356231303535396365633465333230353261346533333063303135303135393465
-34656333613966663966646162386633333830323234613836633236626134316162663139656530
-63623666313138666130326139386636646230353637353561323736656437616438633839613837
-62353931313562383830393433313232653461363436353064653039313738396138333263633965
-32316236363233613831303436323835616332646262343336353037323362313265336634393261
-35393761396537373636313665373830326463643136303961316238336238623466396334343664
-61396234663862343937633131623939306135646231373633653534353363383439656539333331
-30303536353462666433626236343163653139333038656662666664626439613639636564393835
-30363165616263656535303439636563643663633434613665613339313162363234623131306630
-34653932376634363264343335343064343838626464666661313533383534383833633863386463
-37633664396537363735353161346238656336396565383637313230636561626362656232333036
-32386538623063613331623662666531626362616430386335333664646234613965643733656236
-31303766326536326461353632353730303434333866353730383562383533623338626232346330
-64396163613663363562633635656566313263383262656666393537653233363166656335653034
-32376335623633373431376266663036616635343231663963363936353137376263656133623735
-65656435393733613762376638613034643466313632646431656461313362363561623463393238
-34373830373633373835376234366532363763316632613766366161613130633762313839386363
-61396632333533663964373463323464323035386236376364646332663431383039623830333137
-62313062393236363338616433613033306563323264316332366161346235356539323232653931
-66313862383935316235653663336666666339343534376336336138366639633538643937303665
-39386262306566386464363337336130633038633837663166333639343161373234303230306338
-30323063393365336332663531316365356531386365343836656437613337346666653637633639
-38653465393236633962373739323331623565643938373563376364333863343030333136333863
-64663132336165616462336232323865636630373533643063383730626565666337636539373136
-32333861656163646332323239653364326163613361373435323966323964313831663138353063
-34653131383463623965386365326531616265653837363462653037386633376361383639656163
-39663338383939323535366135396465656333323331386462353630396237396261316137386165
-38383664613165356535356632376138623233386266336665386330376539373963323162613336
-62396364373031656238356163656464616430316131386537356236663536366461326462373236
-62396363303861303533643438343765356437326663316239613939386134333830303739386464
-63623834366565366163383533393532383061646163666133396136643462376537393935613039
-37623134313439393737626664663637343365373563376261666133346332373033613335316164
-39313733653931383665663238303532353165613131323731306362666138643737346337623064
-34666261323664363864373061666233613163336633383837323766386137323431626337623562
-36336234656635633830366635363463313439646165353261323030323261633737346664373738
-39633836366133633635303638366134393435393663366664623837336164323630353739333462
-36366663643931653230653562326633353637383564623266313936656239393535646131373261
-61356661663534393166383530373739343834313136663366316237333464326563336636656232
-66373035623866383931393238666132386331356530616535313164366466353563653865663832
-39353364396366663462366333306135343065373532663330643435336239353435303762326132
-65343630343532656638653462383266383937663466633034373661643862663933333838653032
-33656261396638666334626366356362623663613461303936323130353436363536663638383162
-63663931313335666435616561353266376134633132633233356435313137653132373935613931
-36623461396139633439396332393037616361316562613137646466613330653234656535623230
-66306434653462633834643036656263376366623835653163326461313831633934366663646566
-65393966373466326137353261383763346666386337666130646133643666656464616438383038
-31306130356238366261616231363933666162373639363463303962303833626135366666303962
-33316161343936393634303964396566653963333237393631353938386262623739393432383339
-37653430336265323133363937303362343336373230633333323764346139326662343434396566
-66383035613262366430303636323236663937393933646566373838383562316334633836633737
-65346537353439306363633431396338396632336565386434363530346365333765363930613063
-65333166393136633665663539656331663037353138366239383332373363366562613237373234
-64303963666462323330653364396464383464666434343831313338633764636337363761616337
-63313462303339643364383361376364366232326361613137326135333263313964616634383464
-61333539373962653338313835656632663261363766346535323735633437323736323632343434
-61643662613261613835366331303439313062313033623065323262356466656533623735643235
-61643332363333363032326434363065303566616661386261386165383066306161376531356334
-63323634356566333065383763333965613366656661313931386532623963383265613131666231
-61363465343462313531313464323731356637393061663133336665363965653035333030383062
-33653335363536656333333738356437316332313361633136396535643836633035303830323232
-65633761333135633739323061613436386166383762353564326430363965623461306235386132
-66366632643763306338373064613935643630616661353233663834396331643130396465636534
-33303830383139343836633163313737313738636365346639396162343561383965323636306466
-63343038363330313363663566656464383264326538303466323862663463353930376165326466
-66383630376465376139646562633065383434333365376362643038643064656364363538356661
-35303432646332623337353465656362656234333465383663353735663537643630616266383763
-30363239363132333166363563376665633936656535663164656532393666336265393964636333
-35323539383835366162326362333537363666386162376561373464323630303032613037636636
-37663135623933643938396635323763353931336531313732636530643865663430646530666465
-34383962353137366431633530376134323635336433653361316363316132383834663030393666
-66303935343839303337636630313138336239393062666265646163386332376536336439616463
-66316165393963333235353931326461626130306435373061626233616430343064303537373839
-63396464306164613863333434373063396534623739373730303135626639663731366666343733
-32313862306431626231373963396461656637313364343661663235323237316432353762383330
-32353736643766323964643035653661363532336261353334363936333766613739623638333861
-30343933613161643862623263623838623866356534323362623838656133663363353366633764
-38323931346630616664316365363463653637663762633138323230356263636634396165613337
-32656666363137623833366561643563613938623565636132313539383130633637313236333365
-32623663333164653733643063396230313033323636323164643534376333346637636436303935
-34346539663366623137326133383533396535376131626631326463323866353363656636313538
-33393731356632313263663533633762313534346435326533346635356162326466316662323230
-62376666666339623638356138353461313431333366633833323836313338643333366161306139
-35383163373139636137356235656564303666346531663663326134656165616435616163323361
-61346131613666653431656464353636303663653934393835373332663432303762393538616465
-63363566396464333861386135336132623539626432383762313762316135623566373937643532
-64653934383662663263303833383034366436613138373863316334376564616235313462363637
-62313166623261626537323631626432393033626233353137366262613134333565666666326332
-62616662653334323535373235616433653233336265343661363765623066363762353836666364
-35373865363165363234333038323437313561646338343930316661633365623938623665623434
-66383562653564623732356130326138346236343766306535386362373664623162393833656230
-35643535353238336361646534313133303835363837353966326538313161353737363939633961
-39626137366133373536326535613362313866303232396233313263363736646139366363303463
-65363266323565303733613065366237636233306137346233333839646438623038343764346462
-35306434336363333562356566636335613263643738313766636534316232653137346166363932
-32386336653265663665623736663330623036323762623562346164373631313533396437613438
-63373632633731333361366462653466346338366139353632616661626630306633363061373937
-36386635653537346535363030326566366165393839653534383065336638303862373366633263
-33633461663764356330333534373036343565313335363731646432323931613737353231636435
-38626361393264363039303831633535643366636365656538343763646538336138623961303064
-31343431616337316661313935333735613662383235663730373632336333346338623432623731
-39303261326363656638333165646661333735316130363234343661393463346132333333633533
-36393635396234623065626463646536633139333662396635303537653338313532313062653339
-65656639646364346265386361623330333666326338386334376366306339653230633334373438
-36333735393664626136
+66393233316132396639356564316439343234383066633231646134313361666463656536323732
+6630616536646439613533363430306466306233643730350a343364633233333335643833326163
+64393933613963313533623733316339396236363663343635346663323366663166363839663837
+3331363062653239380a326566623264623837326636383939346430666537613361333638366630
+63353062393335316663633739313532366363623739653631366539323435336361353331386230
+39366634643964336233353961316630616462663166316266613037623363346335373638656365
+38353733396636386133373836346336383231663661346137373164386338623733393566373563
+35653933343036633365643535303934326537356136666539316137363433643266346630386439
+38613332646238366536333536343031356532656336613530663830613264346339353034323362
+66613530626361323535653232313730373463373332313561616631393461353730653464343063
+38616338363938346161616636316232313838616463326432353639613837343162646363343232
+33323064363139376566343866626364373662393138353666646234373461666163363139313631
+64343261326566363265323463663538343034306136326234386664333837333937333136653563
+61333531353434633339383661636363363535316366353330313566323133616438373161303135
+35353630613037316466353832333033393030636331386438393133366333653832393731366363
+62373638303737393162303461646239653865653834613662666636373364633165383062643831
+34386232376361323638353361666530366432356331353963303930326535663536373339333062
+32396266373430343339636635366434313635313766363863336464633961666332353834626163
+61653637316163636465343630353431313863653033643237356434313564366361373435376662
+63643737353830663236643862613533623237373531646136383763303766336139303632666235
+64623834323966363363663730626437323432623966663537346162656265363562643836633731
+65663631633462663764393132326165346639353033633035636432613039336164303538396632
+66396264393865306666643636353638613661313230313337383634663839363439656533333932
+36633432306131396539386539633063653230363932376264323537396434353364643432653661
+32643162363066636432336363323534316436613838646562313538326566666239633234646236
+30346534636533623365326564613561363362333364363037646561656635623935653466613565
+65646361313436356261643762313339333864356338386136306162386262636464393130303963
+38646464316432326431326661343632396235626234366133353461623862316662326432356234
+37626366383861373831633639616465663564643866356664623066386535646163336134356534
+33366664616232353863626465626364313530353335306565336665663866303736323162393362
+63626261653161663664363833313461653034326330653835393737616135646462366665383935
+30363639306330636634386433646231363530633061336364313338653632323831393630383934
+37316362326338313733646332336263386239626539383330353362616132333161613464313066
+34663434326662326233363432306433363666356132383866346336336261636435366332666135
+34616231613638363339356333616536643266636363643131653330396162306264303566396461
+64363763376365356533636430643866333361363062376237653237663731663934306265646630
+33393637656335643366383564373966343265393630333835303731316339373133633462383364
+33383435383331303264313334393532373932333334343862326635346135613932356337373034
+62316262326331313135376465343336373266663338396533666431616462613932663861646238
+62653563623535633738383033326235383666646333653731316233376231623661306462303732
+32643064373236613336396233323435393939386530323331336138353364663762356538316562
+33376530623664623733386133333433303031373337313366386236376539613964316135343865
+33363963366165333238356663663435386439336366646138313034343636653463323938633136
+39363966376238306662303265643034306136663661393738633436393432303139313132616534
+66323432313635386162333838323136623634653264643438303264636430633232323434666532
+32616664663063653735316237643539633133356661333132323238376333356464313262653836
+39303566316332663737323437633031353330333365383837636336643763313433313937396531
+38363536343438663966663436613132663661613134383431633765383164373762343435316161
+62303631646235343063383230343232383336356562303563373933346530393333316634316437
+32316330306163396434663031393965663163666537353031613365353437666466333464626238
+35313739646535356665323734393965303064306132626261363062363438383164346261393463
+30643438623363323161323230306230386332363635386234666639623566643536626637616533
+37396136643930633262333331656363376433333234343630306535313262306235663263663362
+32653434363035613732363136303363393939323337613661333439393637646262383039386661
+62326163323562333339323636363565623664396164383332633666386130613766393138346134
+33343338393536316431353439353062663164643634396363353131303038353965393466383030
+36656465383938353936346361393963356630666630373236626237303064303062383638373730
+35633866646535313432353338623462323235346433653431313031363163393666626432363238
+39623361316132626230633336636163623466313666346631656134343762656566353432353264
+30353436356237653231363564626134633039363035313232616333336436393638396233626638
+32663230396539323761313838313466376165646430346634383332346134653662393161363337
+62646161343665383364306665333164666231386531626465373366623761643161656462303733
+37653438616233353432626466623163316565353764323762613635333832343634323665356336
+35353162326233333836396337356466636131383838313436626336663132346339623261366465
+30623261303933396562353331636638376135663330643638643536346261626632626139386535
+66653332366361336636666437643165656239613031303638333232303836383132616636633938
+31343034643037623731643931316463303639656266323231313666356336333133323135363330
+63373365303131353161303630633738353536393631353034666139383435303461316131646138
+61333731356538366366613831303565613365633965323235366166313534653965366433656533
+62666136313662366638356237343734336333313034396465346632336262306531633535643238
+35333831366532386235316565303936616264373337356134643066396531383533353336303131
+31393837623564386535323532653733393734393164373235396566333565356237356438313762
+32623765326639386262393639376461326163333237313232386138643130643231626466643663
+39643061393566353434333136366335393536376234366266376265333234643536633035653933
+36316132663539306465343039323935356361373439346437386234386464623962643464643562
+61336434613834336161633237383361303930313464613666313834356330343138633735386530
+36616233323366323961653965613438346136373738366266316134356266623664313539636235
+37313033373466383134346361646562366531333338386330653736626530396238356639303131
+38363738396236386461316433316261326435646130383336316234363461393237623633633336
+30386630376565646337383738663939663462623232316635346635653830306664653336343033
+64316430396664393532313766326437636636626232613036666666656430323136356436333564
+36303334303562393832336433343438396430373833623137363736386665343866313064353063
+32303634393131326464656535633734386462646339663533666430336265653965333538633866
+61623666643839653239373335633735373738363736313665323365613635313766656635613832
+33616461643539636165383233636533626230343138663630323731626139393230383464313430
+61333438393337316239376435313337313437333931623238616133666138363235386533633437
+35356163363231656536353934643539643562343732626630383565623730626533313230656164
+35333735666135343364663233626163363930383262363266303265303638396239636361366534
+31333863333565356135613232393165353266343632633532343061663331633337343538376265
+39316630613439356262396634316361356436336634396337353339616536356336653930613966
+34656439653366363562636639346430623561303463356337363830373966366632303337663564
+35663632313265323365636238303364366230353039353561616636633664643233343430336237
+63373264643935616331616632633065366638363833306337633563653065363464343137623533
+36373231363739373335346464623533393336613634333636613937366136326464336332346166
+61376263623835646163353134643963663964373732313833346163323138633230393537636664
+30366234303334656130336630346130656237306161376566336534653630616439323764373665
+61383338326163336164353265326163646165623235626137623237306666333832306461613630
+66373331356465346261643466323662393661623433383265376666623932343861323139383531
+64633536373362643935633734366235396433333237306166646164363930613862613365303663
+38343833336137353634313362666665306666393635663633353934363832343739616331386130
+66336561633039326434313833303465366638303961626138333165623331386230616130626639
+34613962366230333065633761333335613636363533656461626632343631666563383738623330
+30333834346233653938633330663166616331376436356533366461336264643264336139343262
+35393665656230663232366133393037643536366234343537326631623332373131323739363638
+39653162646366316639313631393631666261623230313538613666393732626438393763646330
+36346661313131313630343432616365666633353762623261613039623331396330623939626132
+34626333386538326434356432623965666662663437646237373537326534653634346239653634
+31373038303639333037613637393862356263323066666630313262366633313932396465633337
+66653930303934616236323064613761353935613835356561313334323762633064306661346666
+37653262343865386236343634316336386630393739626437333065323433613531393738313432
+31376233353463373237653164386363633334366332356538343966663939656165323465333030
+39656532363363333432626638626438396539336461326338353732376235316133616666316261
+37353063343366376433653961333233306461303133376661303332386230346231383837396133
+37323137343066383966343535633363643233663530613566313330336232366638396165373631
+30626531363033313833303836366434613736396339643032663066333865306535323739666162
+64626133616433653864376662623464343131303938303237316264393765303035663833376464
+32663264383236303766323935306463643138396237373338653238633464616238306132633735
+31626538653262326533326266336633623532623935383266373533363466313033393235663538
+66653038646233303665343634383666343363383238326533366136363838303332323230316662
+35383235646638653539633961663036663933306463626335356631646662636230356261363261
+65633261353830373865636630353932323937666331353635373736376436333361613330366633
+30663939356165393132636131663966373433623063356265353131306532643066306630656363
+66636636353262633437663264613266613663656137386231306231646264363661613035343538
+37353633643065643236376537336238663137623735613038623766393231643131653436333262
+31626463646432613563393665346532386161366435396364663239386236616233356131323536
+33633936623762666534633862363466353736386137636363633733623366346337613365636439
+66663035313430386464623833646135333062313830396637323961386135363461326539623432
+32653865623530313637393561343465636430373162333162646631643235653931333830326266
+36376631316165343631326165623838306239623764363262376634663236393933343838376663
+39663834306165313330393739363133396436376437643232346336386531356638343063376465
+61613037623137306666383231376539656361326132396662613061376134376266633764336266
+64336334313335643635303632666431383637306334376462643630646339396435313830313363
+33383162316261663035393962306234613865613366353465373035656434366261383133653331
+63643235616362663663343330303765363263366130393837613939323264373937333162636639
+31643438666338646135663538343231643235646364623761653064633566656663383465626133
+31373935646266303565303539376162623132316438623565316537306337636630313861623937
+34313832636533623033616139373965303839356530353935643363613464356364343162336466
+66376130653162666661313139613530306666633432346639656466653364376435636461626362
+32653633303561346233643463373534653434323134353434373839373937626663336464303866
+33363264623038313835396231373132396163363662626264346461333539326365326165323066
+62333139383334333334353031616430323339623066363232313937323465356266323934313761
+66623835653961303830383030643537393130653935313265333062393034336562633535323263
+65616237373232336534393834653162363461336262653862666637326266663966356665363036
+35383437326465663635303664643236633435303862633965346133376536316233386333313634
+33643565326633386565653961646463383866646636303537643436623734393234633938333933
+65366164633165623333623362393639656661326332306538663738356364373734316563653038
+34646531616662386232613034366332656262343164333531353037363036646262623663666236
+38613238666136363431623664633863636365396236666532383930336636353031396232656435
+61346238643431653231623861373964383931336535363262373437353532393165316562386134
+36363263666135646237383666373833373737396330616163376439663736663937666161313831
+63663531656635663339306365656663636633343733636165386230376332616331313638386538
+32386466323232363533613334333333346161376430373436373961316564343061326164306138
+33616263666262323430303730626266396535626439623364376239346564323730323534323938
+33346364393033353865393864326361643734353234613563393138363334383536396535393166
+34623163616336653436393639313965353237633566313039303137326234383230323235363234
+37626161356166356365366164363863636563316332393638616535376466343537373966643839
+32613930643533336264626136626465303339376632323034386161663661376466616233633065
+66313739346162363838346663623266383130383736656334323430623463666439386532643630
+33666639613830386136363535363830333234653961663739343537306634616531616263623762
+64666230373830636238353062666330623061613663376638343763626264363130313464383661
+38326530333362616163363735323861376366333665623536383566653837306131623732373639
+31316661353332633630326162663738636562336666326637353764323431613666303038373532
+62343661336338306561356235396636343130633365303466613637633363613862663233633731
+66623530353132666261316637303763363830623734346262333633646238613131346564303734
+62336434353432326239333232383833633962313537626430663130393733623162626131656366
+64333535623138326239336165666562376663663334323036323539653734333835386331653438
+63353861666239396437346361306634613462386335376137333963333838616138633730393865
+62353539376136316564666136646639363635663736636439393462633165646632623664383663
+31613137306461616361323832393036323933626531363536336261356636303531633239333362
+32663134363263383039646162643539663737333861386437326337616362343963373532346238
+37346137363933623839373838353939386630303461346438666534616434333031373730393537
+30313134643963623564356266656430613430626238613266316335336265613132616562626261
+35386435313933626634616463616166646466363939313639646264346464363337656339323366
+36366665363739356564363232313762323565323134616134666337336534353464373637373130
+63613265366436313131356332316531633732356461383064383031613337343363646432373936
+33633339386632653032663837346130623636356464326637303338376132623734333932396232
+61333033386265356630316134383066343164613130666664643732643362666561346132656266
+31623633333039633837383264363937623435643061393935393762346430396335373864633634
+33336136353332663366313334353739303539633364663231636539333132303966383432376262
+61356563323232613433653262623663336634626532653465306638316633663564633862666666
+30366133616336326661626238653933383164336366333438626235636631336165386664343736
+62663961346664656333306435323833366632346366356238653731653937626333653630623334
+64326662346138386433333232643262333835326263343239353264373038613634356436396630
+38323931643361663238623766323930666130356339363564366661663033303831363138343737
+66633535326131396236653261303836613364306537633637323031663166316338323533323731
+30326235323066396663613531653061643661336631613835626266626436386662353465383065
+64396562343966303362636136616438353661626466636635323961613438646634336563636534
+38343566396530643961356434643933636235643561353232643062303232323437666261363061
+62636530396466303466653333633930376465366561376363316137323263333561343334383364
+39313863643062643766396564363137386231373136346138396162376264653538303464633161
+61663363623937356138356430666461623130323466623162653863393736326264393836336637
+34663931646566333535666664653237643732316663323230383239393763376266356135326438
+35656266353865623663373366373830613361373664346632363031356265313364623866643438
+61363866353934636337616239633330623734666138396166313864333939663563636138653930
+35653137363033326432373661613434313137623163356134613265393238346438306165313639
+36666662393165353565633531663536613037623230373063316639663632643139353235303462
+31393263656265656131613164363035343233626433656135353331613532363236616439363731
+62666432356363323937666435326638323437346136366131613636653430306131623966356263
+39303739306233303862636535633431363630393432613663633836396566653039383735303336
+62623331623034636363636661653236386337326666656532343737336262336462613762326531
+34303066303834386366636430343161653665343362363038396562626133636135656538306435
+36393364333066346238396362663664643236373532336263656233386663323663623137343462
+64386337333130316434663564613665666238623132343437656637653035373738313735366630
+66383639616166393265616434623463326437313530326130376339313662303836636664366232
+32366634633030333130316435616233396231663937343732313066373834326464623139363663
+31323931626364303230666162316436653065366137663631376265383063316534343736373261
+31613637316235386539343766323439653062633137663730343236343661346162653366656332
+32663932313063383561636266373766633535656131386133386135663863396261306530326632
+63653936626236316539613262386231616433393064323461626536363831666461316131383837
+30646266646266393666396362326238613231303335336532303836363264323233343534636635
+66313538643033343262373463363866346566353263303966323933383963363463393761383865
+64663932343830643531643466303438343161396133666463353762393737613036646166333265
+66376231613232666164663964636134653061633330383863373836306366393838393235656331
+35613231306263373230326634623262326333356263353961633836396531633431383163633361
+30356534666466653734333437383964346564346165326664633738653338313263633837316531
+36613034323433643839333264323864613033313137663131623265643364333664646235666232
+33393039313666323266643362323337316465306564303230646561303434666630616137633831
+34346439616634343337306636643733316464376631616266376437636439396337306637333432
+39306333363035393436316434656436353738303861633933376531383862316466373736323639
+35336137373866336631386436646231653366366435363932376434303063613961353261343661
+33383638393165336438376662306431333837356435626137356130323836396335636166306662
+36386161353739353637353861306666383966323339303262616239633930373633323937356632
+65383032613031666665623631613430666662656336663931646533636230303261646530623765
+64643939326435643539373564336531623236653731636636346361363064333963376566616530
+62326639663632666634326233363635383830643163373938646165656163643864336436373466
+36353832306632386230373832333234643638313238626333303963383962343265366137656136
+37333261343161633562346638323632616566646162633133663466346535656463393932386135
+62653332313066363965386335356430326539316366633537356364666230326237306236393563
+35323363633936353034323232353366373566666332323737653237323135646665626139393436
+65333762653536656161386532363765336538653763666236343166653933626666633130393033
+31643531646633623663313237353333313136663863663430306131316165663765653732663164
+36326531626365326330643064336230313466343731376437316563303339336333326636633066
+61386135626430616661313236623030316362373338643233326365646531633265626238383830
+65346132643537366537626132666165616138656139626132396639376230333262643766386363
+39383034663034326165643636613237623234613666333532383733623462303331326238636461
+34383139383466356139333934353837613964326538336463643832623062633034613762363061
+61346361656363366136353336326433326266336564316366393565626262303637316564356566
+39376661383763373436306238393666653561306538333638306233356139346634653363346164
+31303835383062376638626266303237323832623735653066353936376339633637333562333561
+36646462653834316131323166366661386161646538346464386232306239363030366363633663
+65306439616339626635326531636435356134376561303235393337373564373937623636643432
+61393535643038626562366331353831663338333838383066323632383633346564396566653330
+34386563393832313537623061666466366661333934613766366165366330353835323637643635
+38613363396663356564646132613536653033616337386566623662333832383938303138316662
+34303339323166383863363831636233323335313565393933636435396666313337663037323432
+63333139333165646262666339343736383966346133356138326437386334626461636530336432
+61356631366163366561303636333230643732316261376365386463333565623533663966336365
+39333365393766306536366231366435363030353263393534653534373064636361333532323735
+61313163323831653362356333386638343566356261353534303738613730373632363534666337
+62313339613138646361356431616236613435393233343732626263653332663265393934616134
+35313165613766643938393839373261633439396661623961353934373130623865353639633038
+34353631346433663131653965326337663561613330323562666336656237633163356562323931
+62393833663233333538383063303937386365306135343962623333663435663431396438666362
+61386266353838653532393233353939363738306634666537313761313835333864633764666262
+30333032623766633334383031636633636539336237613235376466356430663938653565626235
+64373537656566306136633630366130363630633462656330623633393735386630343437336436
+65343635366531646130616534623136636666323139326462306533653532643962656530336633
+35616537303932343539336638333730663639396330653761346136363431346536666138336462
+66396565613166623934316532383835316137303134363466306163356233356530323231666464
+30353933306530323734306564626234343864373964333264353366326265316333343330356532
+38363837646635633461653562303264353633343461633339376665616331613733666663353130
+65396363613731366234326466323738663563646166653237613364323734616465643764633537
+35373865353532383566363632366564353536643739663761303565333138383638653665663664
+65643366316461613630366437623736353739356538336237613431306363663234373265623962
+34653565373335653563356135313835643266356261623037336536613733323733363933376538
+36626134396563623733656534363331626262643339633932373035626134343531623634666463
+61623036313334616639633930393562663631653565656136666537393731333430663062643362
+63633663396562343965313261373965356163393538666466303661363531393266316462626166
+38616536653665366462383064373766396438616665346666376232653031323566313164383164
+36643231646439663637333165376439333432383532316661333766363136636236326338386537
+33306130353634346136356234363438383865313136393839663066333935623565333730613538
+63323831663866363831383930303434333936646564316435303931396362303534386335343330
+65383635346662626363626365666166636361633365643735303762393832316436646139303835
+63633733656361643233323332613632653837663262306661626438316262653931333061336366
+35353939353835333361623261613738383734656132613139393264393038373765343131333330
+37663962313566366463623437323965326365623437363038633661313461383634626661666236
+32633335323861643037383261393164393933353531636134323765353962633732396230636331
+32613865373739366530303538313566346434633933393330346637346136373036306666336164
+35373732346334353432616561623031663331346431383235306537386466623339356366663335
+36333733626433653465336431666530626665373564336339626163633131353330656437643638
+31616563616665343635356231633135663665326131636664373338323736393364353636613762
+66636135633766323866376235633535613735613465303239343036663438333331626431623435
+61323537386434323638666537643236623632626430666263376534643336613635663762643736
+30323237353265613062373265643562373637383337326264653639306263373865333262376665
+62646331373931373762303461366163393839633135393964313937616437323865653735383630
+32653936336534666565373437666130396265363561333635316461663766346336623865376133
+35373665303433326265623531613038636166643130616637653165376263643634376439613765
+35363031373630333966656466616235616337306335363132386335613462363664653634633864
+61303635653932353730663666386263633662633736313461643932386161313762663761313336
+62653665313033656537643936373465633932626166366430643763313030393838393039323230
+62633334383938343433306262393536653930653030393033306661666264313630643564333166
+35383237633932656331653030363434313534613637373465663264643061303538653666653861
+35373936643037333866636131373338363062663035323531626431633362663364396365353139
+66383737666437353764333231303662393630643933376161366430376530613365363830373534
+38376633333936626430393163323830346166643537326430616236393733653761363235356363
+35333131663032383861336262653936376565646662313965303265623763613330653461333835
+63613563323135633438383931343731656333303362316533376339376636623037376431336366
+61393236383364356162633062666265653534326363363862666539623761623065386537616563
+62666561316437303763376635346536666437373361386666643139643737663333323933613661
+38326566663932333930616435626133616531306461356466326437623235613233393434626563
+34373966633834373430386132353163366465626262353863353335323830393266393562393133
+65383632653438646435343333386261653066613663623232373564666465613136353039313036
+61646462626433646330396664363938376530376438646262343231393262383733636233636333
+35633862336566636439653464613564333162613836343636316334316665383164353131373431
+63623030306564346562346237333934616134346536303365396533626262333937396432393830
+38313763393463646437666137353835373735646365373934363936346564326362376565353133
+36653362333432326133393837316331666663663263396461363239306239363733633137396633
+38393865613431653337313665313762653635656531353465623436343132303064303564393066
+63616639353962366666616261393766643364333634346630616436376565313236316539633537
+66653239636561393433383639646462616433653166613130373134376535633937353366383230
+61613335663434333835653236343633633038346335333861356637353965396632393833646635
+36653034356234663831333764303338663464316362646339376338393236336161616263363538
+63356631613239326161343031643936623366643432663732346438333265666535623664396333
+62303239363339626566613439396234303536333333653433393666383635643235376165666234
+30336236393962666335353233666463346530323531316438373130303933383465316638646461
+62643066376666363236333231386237376466633932313836323163363061313333633434663763
+64323365353336333736363436376232653436633739613437343538633632356665656364616637
+62313666346436656564663335636635393632353430666236313863613464626434323939383538
+65386265343434303632313739353239323565333734656566356164643430613538333234383566
+37646431316363316139646435333732313339623666613738663039613239613738393565333330
+33376432373933656435303737653762666464363865633831373330393435633332636261336139
+34336236373535636165353262323966363164633135613534353661316364616637663465363864
+39306163346365643339643937396165666366663339336438373031613937636464383531613962
+34373433663533626665656364623634373335313033646165303764396563356235343033383138
+62326361353630393938643764616636313461633734646661386536356235656665393864386465
+37666262346561656436343036646330363664306135333464663265306165353039396665336664
+62346631366330653762646538323565613864383534636532633033643533323736373931643130
+32343435333333613734626234363132373734353035326232366264336161383631353133663230
+32636533333866343763336439373336356237303636376334333433376630353338333261333037
+31666537363666653238383939663464346662636133346561326335346163363061393830616237
+36643430626534653331653665316535303139343763663965363164636238366533303038653935
+32356432316633373137316237663331336463363431393033323635646564366639346230353363
+38346663363039363962323137383366303862356530353238656563306131643236626536656334
+38303462306562366532346163323061393437353063326539393466616439346564383036303235
+35633731393661323962633631373061303930323638326565636162316436646337383266626561
+32326430363031396530396238353862333133363731623736376239626561626165663337373261
+39353461343461643238646635633562653865323336366634613264616662323232653861663038
+64396330626633303031333334343335393039623135353266383561313231643433393963326637
+39313530636361373831306234383166346266656261663830636631333564356536323565336266
+38306561376366626236306633613564386166616630613032633163613837313462343662653261
+63353437663436303634633336636532646439636465663362346138313665336334313039613631
+65326135383831613531323265353831313562346161663265366434623236636635333038366536
+33646631663662393331323162343438626666366636613438383665633136326439376166373462
+33363864613136643461663436396362643066633437376631623031613366656238396165313832
+31313131653263666334393664343239306235373862313339373563643137393633343663613936
+61356564636238363136623031336638333566633766636362303938653531306131396665303033
+63353362636463636236643464343562383161343432383766396330623764393837613435396162
+31303162356437663932663964663239623764666366663061313535346438373334636263653531
+34643133356638653031373036343162653135663734623035353033633561366266623566383233
+36356434643538643430383532393762333535636639353361353763333363313131646264336332
+34373331343930633962623963666365306132356334646636626461316236343839383266363635
+65623434336239313330343437646333353362303232346638623161616133636636626236643465
+36303636363965363765656533386534633839346363363738386532386531326538643134363132
+66613235373362633166343565323766306335336365333439323764623964393263623236623832
+34346132383136303038363764333039626234616132386464666633663536656230666133363533
+38306665643361666539636666316432623430623939663636343164386438313765633031313534
+65393633323837326166343936326263343833646331326464376138633461613532303135393036
+65353830373065393038343039323937303634346665393135383639303162396565646232663736
+39376434646634353330383933303164653431373433346335666131386165343035303964626665
+35383035653631346638326637326235393833623264323030373238646335346332353362393230
+61616664613562383639306564376661306665396138613066326631616531623132633966633832
+65363637376264336635633132633332373634383864626564623966356464373864393832323738
+37616338646633326636323461376137663632376262363738303336616463326238333465343533
+31316132386530326539
--- a/ansible/host_vars/astro-orbiter/vars.yml
+++ b/ansible/host_vars/astro-orbiter/vars.yml
@@ -2,11 +2,14 @@
 # ------------------------------------------------------------------------------
 # FILE: ansible/host_vars/astro_orbiter/vars.yml
 # HOST: astro-orbiter (10.1.71.130)
-# ROLE: Ollama inference host with AMD RX 5700 GPU passthrough
+# ROLE: llama.cpp LLM inference host — Ryzen 7 5800XT / RTX 3090 (ATX rebuild,
+#       2026-08-04). Superseded the prior AMD RX 5700 / Ollama config below;
+#       drive was transplanted into new hardware, not reinstalled.
 # ------------------------------------------------------------------------------

 ansible_host: 10.1.71.130
-ansible_user: wed
+ansible_user: jarvis
+ansible_ssh_private_key_file: ~/.ssh/id_jarvis
 ansible_become: true

 # LVM root expansion — xlarge template uses sda3 partition, standard VG/LV names
@@ -15,11 +18,3 @@ common_root_pv: /dev/sda3
 common_root_vg: ubuntu-vg
 common_root_lv: ubuntu-lv

-# Ollama — all defaults apply; explicitly documented here for visibility
-ollama_rocm_version: "6.2"
-ollama_default_model: "qwen3:8b"
-ollama_hsa_override_gfx_version: "10.1.0"
-ollama_data_disk: /dev/sdb
-ollama_data_vg: ollama-vg
-ollama_data_lv: ollama-lv
-ollama_data_dir: /var/lib/ollama
--- a/ansible/inventory.yml
+++ b/ansible/inventory.yml
@@ -59,12 +59,9 @@ n8n_server:
  hosts:
    tiki-room:

-ollama_server:
+astro_orbiter:
  hosts:
    astro-orbiter:
-      ansible_host: 10.1.71.130
-      ansible_user: wed
-      ansible_become: true

 hermes_server:
  hosts:
@@ -72,6 +69,7 @@ hermes_server:
      ansible_host: 10.1.71.131
      ansible_user: wed
      ansible_become: true
+      ansible_ssh_private_key_file: ~/.ssh/ansible

 honcho_server:
  hosts:
--- a/ansible/playbooks/day1_deploy_llm_inference.yml
+++ b/ansible/playbooks/day1_deploy_llm_inference.yml
@@ -0,0 +1,25 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: playbooks/day1_deploy_llm_inference.yml
+# DESCRIPTION: Day 1 playbook for astro-orbiter LLM inference stack.
+#              Deploys vLLM + Gemma 2 27B on RTX 3090 via OCuLink.
+#
+# Usage:
+#   cd ~/git/homelab/ansible
+#   ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference.yml
+#
+# Phases (added incrementally — safe to re-run):
+#   1. Foundation  — groups, directories, vault assertion
+#   2. Driver      — nvidia-driver-595-open (idempotent; already installed)
+#   3. vLLM        — Python venv + pip install vllm
+#   4. Model       — HF login, Gemma 2 27B snapshot_download
+#   5. Serve       — systemd vllm-serve.service, health check
+#   6. Integration — Hermes provider config on carousel
+# ------------------------------------------------------------------------------
+
+- name: Deploy LLM inference stack on astro-orbiter
+  hosts: astro_orbiter
+  gather_facts: true
+
+  roles:
+    - role: llm-inference
--- a/ansible/playbooks/day1_deploy_llm_inference_multimodel.yml
+++ b/ansible/playbooks/day1_deploy_llm_inference_multimodel.yml
@@ -0,0 +1,31 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: playbooks/day1_deploy_llm_inference_multimodel.yml
+# DESCRIPTION: Day 1 playbook for the dual-model (aux + tool-calling) rollout
+#              on astro-orbiter. Builds on roles/llm-inference (CUDA/driver
+#              already done) — does not replace it.
+#
+# Usage:
+#   cd ~/git/homelab/ansible
+#   ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference_multimodel.yml
+#   # or scope to specific phases:
+#   ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference_multimodel.yml --tags discover
+#
+# KNOWN GAP (2026-08-05): Semaphore is currently broken; this is being run
+# via direct ansible-playbook as an accepted interim stopgap. Retarget
+# through Semaphore once it's repaired.
+#
+# Phases (see roles/llm-inference-multimodel/README.md for detail):
+#   0. discover  — read-only; confirm existing Gemma service management
+#   1. models    — idempotent GGUF downloads (Phi-4-14B, Mistral-Small-24B)
+#   2. systemd   — deploy both unit files, do NOT auto-start
+#   3. firewall  — scope ports 8000/8001, non-0.0.0.0 bind
+#   4. verify    — start both services, smoke test, VRAM check
+# ------------------------------------------------------------------------------
+
+- name: Deploy dual-model LLM inference stack on astro-orbiter
+  hosts: astro_orbiter
+  gather_facts: true
+
+  roles:
+    - role: llm-inference-multimodel
--- a/ansible/roles/llm-inference-multimodel/README.md
+++ b/ansible/roles/llm-inference-multimodel/README.md
@@ -0,0 +1,132 @@
+# llm-inference-multimodel
+
+Deploys **two independent llama-server systemd services** on astro-orbiter's
+RTX 3090 (24GB), alongside — not replacing — the existing `llm-inference` role:
+
+| Instance | Port | Model | Quant | ctx | parallel | ~VRAM |
+|---|---|---|---|---|---|---|
+| `llama-server-aux` | 8000 | Phi-4-14B-Instruct | Q4_K_M | 8192 | 2 | ~10.0GB |
+| `llama-server-toolcall` | 8001 | Mistral-Small-24B-Instruct-2501 | Q3_K_M | 4096 | 1 | ~13.2GB |
+
+Combined estimate: **~23.2GB / 24GB** (~0.8GB headroom). See
+`/home/hermes/astro-orbiter-multi-model-plan.md` for the full approved design
+(VRAM math, model selection rationale, rollback plan, validation harness).
+
+## Relationship to `roles/llm-inference`
+
+This role does **not** replace `llm-inference`. It assumes that role's
+prerequisites are already satisfied on the host:
+
+- NVIDIA driver installed
+- `/opt/llama.cpp` cloned and built with CUDA (`/opt/llama.cpp/build/bin/llama-server` exists)
+- `jarvis` service user + `/home/jarvis` present
+
+The pre-existing single-model Gemma llama-server (however it is currently
+run) is **never modified, restarted, or deleted** by this role. It is the
+rollback target.
+
+## Phases
+
+Run the whole role, or scope with `--tags`:
+
+```
+ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference_multimodel.yml
+# or, once merged into a single play:
+ansible-playbook -i inventory.yml <playbook>.yml --tags discover,models,systemd,firewall,verify
+```
+
+0. **discover** (`tasks/discover.yml`) — READ-ONLY. Confirms via
+   `service_facts` + `pgrep` whether the existing Gemma llama-server actually
+   runs as a systemd unit today, or some ad hoc way (nohup/screen/tmux). Does
+   **not** assume a unit exists — this was an open unknown in the plan and is
+   resolved here as a fact-gathering step, not an assumption. Also records
+   baseline VRAM and current port 8000/8001 listeners for comparison later.
+
+   **If this reports no unit found**, stop and read the debug message —
+   it means plan §6's rollback story ("systemctl start the old unit to
+   revert") isn't actually available yet, and that should be fixed (codify
+   the existing process as a systemd unit) before proceeding to Phase 2.
+
+1. **models** (`tasks/models.yml`) — Idempotent GGUF download to
+   `/opt/models/` with a stat + minimum-size guard (mirrors the pattern in
+   `roles/llm-inference/tasks/serve.yml` and the `llm-inference-homelab`
+   skill), so reruns don't re-pull 8.5GB/11.7GB files or mistake a truncated
+   partial download for complete.
+
+2. **systemd** (`tasks/systemd.yml`) — Templates and deploys both unit files
+   to `/etc/systemd/system/`. **Deliberately does not start or enable either
+   service** — units land on disk as a separately reviewable checkpoint.
+   Two fully independent units (not one unit with two ExecStarts) so either
+   instance can be restarted/stopped without affecting the other.
+
+3. **firewall** (`tasks/firewall.yml`) — Scopes ports 8000 and 8001 via `ufw`
+   to `llm_allowed_source_cidr` (default the Hermes LAN subnet), rather than
+   leaving them open. Both unit templates also bind to
+   `llm_bind_address` (default `10.1.71.130`, the host's private LAN IP) —
+   **not `0.0.0.0`** — which is a deliberate change from the pre-existing
+   Gemma pattern flagged as insecure in the plan.
+
+4. **verify** (`tasks/verify.yml`) — The only phase that actually starts +
+   enables both services. Waits for `/health` on both ports, smoke-tests
+   `/v1/models` and a trivial `/v1/chat/completions` call on each, checks
+   `nvidia-smi` VRAM usage against the plan's design estimate, and greps
+   `dmesg` for OOM-kill events.
+
+   **This smoke test is not the tool-calling validation harness.** See
+   below.
+
+## Key variables
+
+Defined in `defaults/main.yml` (all overridable via `host_vars`/`group_vars`
+or `-e`):
+
+- `llm_service_user` (jarvis), `llm_binary_path`, `llm_models_dir`, `llm_bind_address`, `llm_allowed_source_cidr`
+- Aux: `llm_aux_port`, `llm_aux_model_path`, `llm_aux_model_url`, `llm_aux_ctx_size`, `llm_aux_parallel`, `llm_aux_gpu_layers`
+- Tool-calling: `llm_toolcall_port`, `llm_toolcall_model_path`, `llm_toolcall_model_url`, `llm_toolcall_ctx_size`, `llm_toolcall_parallel`, `llm_toolcall_gpu_layers`
+
+`vars/main.yml` holds constants not meant to be overridden per-host (HF token
+reference, expected-VRAM figures used only for the verify.yml report).
+
+## ⚠️ Tool-calling validation is required before use
+
+Port 8001 (Mistral-Small-24B) **must** pass the manual validation procedure
+described in plan §7 before any Claude Code / tool-calling-capable Hermes
+profile is pointed at it:
+
+1. A curl-based `tool_calls` emission probe (does it call tools correctly on
+   known trigger prompts?)
+2. A hallucination stress test (does it fabricate `tool_calls` on prompts
+   that shouldn't trigger any?)
+3. A shadow-mode period (run parallel to the existing tool-calling path,
+   compare outputs, before a hard cutover)
+
+This is **intentionally not automated into this role** — it is a
+correctness/safety judgment call, not a repeatable infra check. See
+`docs/validation-log.md` in this role directory for the procedure reference
+and a place to log results once Ryan runs it.
+
+## Known gap: Semaphore is broken (as of 2026-08-05)
+
+The normal execution/audit path (Semaphore) is currently non-functional.
+This role was authored to be run via direct `ansible-playbook` as an accepted
+interim stopgap, executed personally by Ryan. **This is a known gap, not the
+intended long-term operational path** — once Semaphore is repaired, retarget
+execution of this role (and future changes to it) through Semaphore so runs
+are audited/logged there again. Flag this in any future work that touches
+this role.
+
+## Rollback
+
+The existing Gemma llama-server and its GGUF are untouched by every phase of
+this role. To roll back:
+
+1. `systemctl stop llama-server-aux llama-server-toolcall`
+2. `systemctl disable llama-server-aux llama-server-toolcall` (optional, if reverting permanently)
+3. Confirm the original Gemma service (name determined by `discover.yml`,
+   commonly `llama-server.service`) is (still) running: `systemctl status llama-server`
+4. If it was never running because Phase 0 discovered it wasn't a managed
+   unit, whatever ad hoc process/command was used before this role's changes
+   is also unaffected — nothing in this role stopped it.
+
+No files belonging to the existing Gemma deployment (GGUF, unit file, or
+otherwise) are ever written to or deleted by this role.
--- a/ansible/roles/llm-inference-multimodel/defaults/main.yml
+++ b/ansible/roles/llm-inference-multimodel/defaults/main.yml
@@ -0,0 +1,59 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/defaults/main.yml
+# DESCRIPTION: Overridable defaults for the llm-inference-multimodel role.
+#              Deploy target: astro-orbiter (10.1.71.130, RTX 3090 24GB).
+#              Built ALONGSIDE roles/llm-inference (not a replacement) — that
+#              role's CUDA/build/driver phases are the prerequisite; this role
+#              assumes /opt/llama.cpp/build/bin/llama-server already exists.
+#
+#              See /home/hermes/astro-orbiter-multi-model-plan.md for the full
+#              approved design (VRAM math, rationale, rollback story).
+# ------------------------------------------------------------------------------
+
+# Shared
+llm_service_user: jarvis
+llm_binary_path: /opt/llama.cpp/build/bin/llama-server
+llm_models_dir: /opt/models
+
+# Bind address — deliberately NOT 0.0.0.0 (see plan §5). Default to the private
+# LAN interface so both instances are reachable from Hermes but not the world.
+# Override to 127.0.0.1 if even LAN-wide reachability is unwanted and a reverse
+# proxy/localhost-only tunnel is used instead.
+llm_bind_address: "10.1.71.130"
+
+# Firewall scoping (Phase 3) — subnet/hosts allowed to reach the ports above.
+# Override per-environment; default assumes Hermes runs somewhere on this /24.
+llm_allowed_source_cidr: "10.1.70.0/24"
+
+# --- Aux / classification instance (port 8000, Phi-4-14B) -------------------
+# Text-only instruction model, no tool-calling training — safe offload target
+# per the auxiliary-task-offload skill's "no tool_calls emission risk" bar.
+llm_aux_port: 8000
+llm_aux_model_path: "{{ llm_models_dir }}/phi-4-14b-instruct-Q4_K_M.gguf"
+llm_aux_model_url: "https://huggingface.co/bartowski/phi-4-GGUF/resolve/main/phi-4-Q4_K_M.gguf"
+llm_aux_model_min_bytes: 8000000000   # guard threshold; complete file ~8.5GB
+llm_aux_ctx_size: 8192
+llm_aux_parallel: 2
+llm_aux_gpu_layers: 99
+llm_aux_service_name: llama-server-aux
+llm_aux_model_id: phi-4-14b-instruct   # served model name for OpenAI-compat API
+
+# --- Tool-calling instance (port 8001, Mistral-Small-24B) --------------------
+# Native function-calling support; deployed at Q3_K_M per plan §1 Option B
+# to fit VRAM budget. MUST pass the §7 validation harness before any
+# Claude-Code-capable profile is pointed at this port.
+llm_toolcall_port: 8001
+llm_toolcall_model_path: "{{ llm_models_dir }}/mistral-small-24b-instruct-2501-Q3_K_M.gguf"
+llm_toolcall_model_url: "https://huggingface.co/bartowski/Mistral-Small-24B-Instruct-2501-GGUF/resolve/main/Mistral-Small-24B-Instruct-2501-Q3_K_M.gguf"
+llm_toolcall_model_min_bytes: 11000000000   # guard threshold; complete file ~11.7GB
+llm_toolcall_ctx_size: 4096
+llm_toolcall_parallel: 1
+llm_toolcall_gpu_layers: 99
+llm_toolcall_service_name: llama-server-toolcall
+llm_toolcall_model_id: mistral-small-24b-instruct-2501
+
+# --- Existing Gemma baseline (rollback target — never modified by this role) -
+# Populated by Phase 0 discovery (tasks/discover.yml) if not already known.
+# Set here only as a fallback name to search for; discovery is authoritative.
+llm_existing_gemma_service_name_guess: llama-server
--- a/ansible/roles/llm-inference-multimodel/docs/validation-log.md
+++ b/ansible/roles/llm-inference-multimodel/docs/validation-log.md
@@ -0,0 +1,46 @@
+# Tool-Calling Model Validation Log
+
+This file tracks the manual validation procedure required by
+`astro-orbiter-multi-model-plan.md` §7 before `llama-server-toolcall` (port
+8001, Mistral-Small-24B-Instruct-2501 Q3_K_M) is trusted for any real
+tool-calling / Claude Code Hermes profile traffic.
+
+This is **not automated by the role** — `tasks/verify.yml` only confirms the
+endpoint is up and can produce a basic completion. The checks below are a
+correctness/safety judgment call that a human runs and records here.
+
+## Procedure (plan §7 summary)
+
+1. **`tool_calls` emission probe** — curl a handful of known
+   tool-triggering prompts (e.g. "what's the weather in Austin right now")
+   against `POST http://10.1.71.130:8001/v1/chat/completions` with a `tools`
+   array defined, and confirm the response actually contains a well-formed
+   `tool_calls` block (correct function name, valid JSON arguments) rather
+   than a plain-text answer or a malformed call.
+
+2. **Hallucination stress test** — send prompts that should **not** trigger
+   any tool call (general knowledge questions, casual chat, prompts that
+   merely mention a tool's name in passing) and confirm the model does
+   **not** emit a spurious `tool_calls` block. This is the primary risk
+   flagged in the plan given Mistral-Small's Q3_K_M quantization and its
+   lineage concerns around over-eager tool invocation.
+
+3. **Shadow mode** — for a bounded period, run this instance in parallel
+   with whatever tool-calling path is currently in production, comparing
+   outputs on the same real traffic (or a recorded sample) without letting
+   this instance's outputs actually drive tool execution. Only cut over
+   once outputs are consistently correct.
+
+See the `llm-inference-homelab` skill's `scripts/tool-calling-validation.sh`
+reference for a starting curl harness shape — adapt prompts/tool schemas to
+Mistral-Small's actual expected format (confirm via the GGUF's embedded
+chat template / model card) rather than assuming it matches Qwen's.
+
+## Log
+
+| Date | Run by | Probe result | Hallucination test result | Shadow mode outcome | Decision |
+|---|---|---|---|---|---|
+| _(pending)_ | | | | | Not yet cut over — do not point production tool-calling traffic at :8001 |
+
+Update this table after each validation pass. Do not remove prior rows —
+this is the audit trail for "when did we decide this was safe to use."
--- a/ansible/roles/llm-inference-multimodel/handlers/main.yml
+++ b/ansible/roles/llm-inference-multimodel/handlers/main.yml
@@ -0,0 +1,27 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/handlers/main.yml
+# DESCRIPTION: Separate restart handlers per instance — NEVER combined, so a
+#              content change to one unit template never restarts the other
+#              (plan §2/§6 requirement: independent restart/rollback).
+# ------------------------------------------------------------------------------
+
+- name: Reload systemd
+  ansible.builtin.systemd:
+    daemon_reload: true
+  become: true
+  listen: "reload systemd"
+
+- name: Restart llama-server-aux
+  ansible.builtin.systemd:
+    name: "{{ llm_aux_service_name }}"
+    state: restarted
+  become: true
+  listen: "restart llama-server-aux"
+
+- name: Restart llama-server-toolcall
+  ansible.builtin.systemd:
+    name: "{{ llm_toolcall_service_name }}"
+    state: restarted
+  become: true
+  listen: "restart llama-server-toolcall"
--- a/ansible/roles/llm-inference-multimodel/meta/main.yml
+++ b/ansible/roles/llm-inference-multimodel/meta/main.yml
@@ -0,0 +1,17 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/meta/main.yml
+# ------------------------------------------------------------------------------
+galaxy_info:
+  role_name: llm_inference_multimodel
+  author: rblundon
+  license: MIT
+  description: >
+    Deploys two independent llama-server instances on astro-orbiter's RTX 3090:
+    an aux/classification instance (Phi-4-14B Q4_K_M, port 8000) and a
+    tool-calling instance (Mistral-Small-24B-Instruct-2501 Q3_K_M, port 8001).
+    Built alongside roles/llm-inference (not a replacement); assumes that
+    role's CUDA build/driver work is already done. See
+    /home/hermes/astro-orbiter-multi-model-plan.md for the full design.
+  min_ansible_version: "2.15"
+dependencies: []
--- a/ansible/roles/llm-inference-multimodel/tasks/discover.yml
+++ b/ansible/roles/llm-inference-multimodel/tasks/discover.yml
@@ -0,0 +1,87 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/discover.yml
+# DESCRIPTION: Phase 0 — READ-ONLY fact gathering on how the existing Gemma
+#              llama-server is actually managed on astro-orbiter TODAY.
+#
+#              Per plan §0/§4: "Service management: unverified — plan requires
+#              confirming systemd unit exists before touching anything.
+#              Do not assume." This file performs that confirmation. It makes
+#              NO changes to the host — no `state: present/started/stopped`,
+#              no file writes, no service actions. Every task here is either
+#              a `_facts` module, a `command`/`shell` in check-safe read mode,
+#              or a `stat`.
+#
+#              Outcomes recorded as facts for later phases/for a human to read
+#              in the play recap — this file does not branch role behavior
+#              on the result (that would be over-engineering a role meant to
+#              run once); it surfaces what's true so a human confirms before
+#              Phase 2 proceeds.
+# ------------------------------------------------------------------------------
+
+- name: Gather service facts (systemd unit inventory)
+  ansible.builtin.service_facts:
+
+- name: Determine whether a systemd unit matching the existing Gemma service exists
+  ansible.builtin.set_fact:
+    llm_existing_gemma_unit_found: "{{ (llm_existing_gemma_service_name_guess + '.service') in ansible_facts.services }}"
+
+- name: Report existing Gemma systemd unit state (if found)
+  ansible.builtin.debug:
+    msg: >-
+      Existing unit '{{ llm_existing_gemma_service_name_guess }}.service' found:
+      state={{ ansible_facts.services[llm_existing_gemma_service_name_guess + '.service'].state | default('unknown') }},
+      status={{ ansible_facts.services[llm_existing_gemma_service_name_guess + '.service'].status | default('unknown') }}
+  when: llm_existing_gemma_unit_found
+
+- name: WARNING — no systemd unit found matching the existing Gemma service
+  ansible.builtin.debug:
+    msg: >-
+      No systemd unit named '{{ llm_existing_gemma_service_name_guess }}.service'
+      was found via service_facts. This means the current single-model
+      llama-server is likely run some other way (manual nohup, screen/tmux,
+      or a differently-named unit). DO NOT PROCEED to Phase 2 assuming a
+      clean rollback target exists. Before continuing: (1) check for any
+      running llama-server process via `ansible -m command -a "pgrep -fa
+      llama-server"`, (2) if found running ad hoc, codify it as a proper
+      systemd unit FIRST (reusing roles/llm-inference's existing
+      llama-server.service.j2 pattern) so plan §6's rollback story
+      ("systemctl start llama-server-gemma to fully revert") is real and
+      not aspirational. This is a human decision point, not something this
+      role auto-remediates.
+  when: not llm_existing_gemma_unit_found
+
+- name: Check for any running llama-server process (read-only, no state change)
+  ansible.builtin.command:
+    cmd: pgrep -fa llama-server
+  register: llm_existing_process_check
+  changed_when: false
+  failed_when: false   # pgrep exits 1 with no matches — not a failure condition here
+
+- name: Report any llama-server process found running outside systemd
+  ansible.builtin.debug:
+    msg: "Running llama-server process(es): {{ llm_existing_process_check.stdout_lines }}"
+  when: llm_existing_process_check.rc == 0
+
+- name: Check current GPU VRAM utilization (baseline, before any changes)
+  ansible.builtin.command:
+    cmd: nvidia-smi --query-gpu=memory.used,memory.total --format=csv,noheader
+  register: llm_baseline_vram
+  changed_when: false
+  failed_when: false
+
+- name: Report baseline VRAM usage
+  ansible.builtin.debug:
+    msg: "Baseline GPU VRAM (before this role's changes): {{ llm_baseline_vram.stdout | default('nvidia-smi unavailable') }}"
+
+- name: Check whether ports 8000/8001 are already bound (avoid port collision surprises)
+  ansible.builtin.command:
+    cmd: "ss -ltnp"
+  register: llm_existing_listeners
+  changed_when: false
+  failed_when: false
+
+- name: Report current listeners on 8000/8001
+  ansible.builtin.debug:
+    msg: "{{ llm_existing_listeners.stdout_lines | select('search', ':(8000|8001)\\s') | list }}"
+  when: llm_existing_listeners.rc == 0
--- a/ansible/roles/llm-inference-multimodel/tasks/firewall.yml
+++ b/ansible/roles/llm-inference-multimodel/tasks/firewall.yml
@@ -0,0 +1,64 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/firewall.yml
+# DESCRIPTION: Phase 3 — scope :8001 (new) and reconsider :8000 (existing
+#              pattern) exposure, per plan §5.
+#
+#              Current baseline pattern (0.0.0.0:8000, no auth) is a
+#              pre-existing flagged issue — this role does NOT repeat it
+#              uncritically for the new port, and tightens both:
+#              1. Bind address: handled in systemd.yml templates via
+#                 {{ llm_bind_address }} (default 10.1.71.130, NOT 0.0.0.0).
+#              2. Firewall: ufw rules scoping both ports to
+#                 {{ llm_allowed_source_cidr }} rather than open LAN-wide.
+#
+#              Idempotent: named rule comments + `state: present` so reruns
+#              don't duplicate rules (per plan §4 idempotency note).
+# ------------------------------------------------------------------------------
+
+- name: Check whether ufw is installed/active
+  ansible.builtin.command:
+    cmd: ufw status
+  register: llm_ufw_status
+  changed_when: false
+  failed_when: false
+  become: true
+
+- name: WARNING — ufw not active, firewall scoping cannot be applied
+  ansible.builtin.debug:
+    msg: >-
+      ufw does not appear to be active on this host (`ufw status` returned:
+      {{ llm_ufw_status.stdout | default('n/a') }}). Firewall scoping for
+      ports {{ llm_aux_port }}/{{ llm_toolcall_port }} was skipped. This is a
+      gap vs plan §5 item 2 — flag to Ryan before relying on bind-address
+      alone for exposure control.
+  when: "'Status: active' not in (llm_ufw_status.stdout | default(''))"
+
+- name: Allow aux port ({{ llm_aux_port }}) from the Hermes source subnet
+  community.general.ufw:
+    rule: allow
+    port: "{{ llm_aux_port | string }}"
+    proto: tcp
+    src: "{{ llm_allowed_source_cidr }}"
+    comment: "llm-inference-multimodel: aux (Phi-4) — scoped to Hermes subnet"
+  become: true
+  when: "'Status: active' in (llm_ufw_status.stdout | default(''))"
+
+- name: Allow tool-calling port ({{ llm_toolcall_port }}) from the Hermes source subnet
+  community.general.ufw:
+    rule: allow
+    port: "{{ llm_toolcall_port | string }}"
+    proto: tcp
+    src: "{{ llm_allowed_source_cidr }}"
+    comment: "llm-inference-multimodel: toolcall (Mistral-Small) — scoped to Hermes subnet"
+  become: true
+  when: "'Status: active' in (llm_ufw_status.stdout | default(''))"
+
+- name: Report firewall scoping applied
+  ansible.builtin.debug:
+    msg: >-
+      Firewall scoping applied for ports {{ llm_aux_port }} and
+      {{ llm_toolcall_port }}, restricted to source {{ llm_allowed_source_cidr }}.
+      Reverse-proxy + API-key enforcement (plan §5 item 3) is NOT implemented
+      by this role — flagged as an optional follow-up phase, not bundled into
+      this minimum-viable rollout.
--- a/ansible/roles/llm-inference-multimodel/tasks/main.yml
+++ b/ansible/roles/llm-inference-multimodel/tasks/main.yml
@@ -0,0 +1,35 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/main.yml
+# DESCRIPTION: Entry point — imports one task file per phase.
+#              Phases are additive; re-running the full playbook is always
+#              safe (idempotent). Use --tags to run a specific phase subset:
+#                --tags discover,models,systemd,firewall,verify
+#
+# IMPORTANT: Phase 2 (systemd) deploys but does NOT start either service.
+#            Phase 4 (verify) is what starts + smoke-tests them. This lets
+#            Ryan review "systemd units land, nothing running yet" as a
+#            distinct, revertable checkpoint before anything touches the
+#            live GPU/VRAM state.
+# ------------------------------------------------------------------------------
+
+# Phase 0 — Discover (read-only; confirm how the existing Gemma llama-server
+# is actually managed today before assuming a systemd unit exists)
+- import_tasks: discover.yml
+  tags: [discover]
+
+# Phase 1 — Models (idempotent GGUF download, size-check guard)
+- import_tasks: models.yml
+  tags: [models]
+
+# Phase 2 — Systemd (template + deploy both unit files, do NOT auto-start)
+- import_tasks: systemd.yml
+  tags: [systemd]
+
+# Phase 3 — Firewall (scope :8001 and reconsider :8000 exposure)
+- import_tasks: firewall.yml
+  tags: [firewall]
+
+# Phase 4 — Verify (start both services, curl smoke test, nvidia-smi VRAM check)
+- import_tasks: verify.yml
+  tags: [verify]
--- a/ansible/roles/llm-inference-multimodel/tasks/models.yml
+++ b/ansible/roles/llm-inference-multimodel/tasks/models.yml
@@ -0,0 +1,72 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/models.yml
+# DESCRIPTION: Phase 1 — download both GGUFs to {{ llm_models_dir }}.
+#              Idempotent: reuses the stat + size-threshold guard pattern
+#              from the llm-inference-homelab skill / roles/llm-inference's
+#              serve.yml, so reruns don't re-pull 8.5GB / 11.7GB files.
+#
+#              Does NOT touch the existing Gemma GGUF — separate directory
+#              entries, no overlap, no deletion of anything pre-existing.
+# ------------------------------------------------------------------------------
+
+- name: Create models directory
+  ansible.builtin.file:
+    path: "{{ llm_models_dir }}"
+    state: directory
+    owner: "{{ llm_service_user }}"
+    group: "{{ llm_service_user }}"
+    mode: "0755"
+  become: true
+
+# --- Aux model (Phi-4-14B Q4_K_M) --------------------------------------------
+
+- name: Check if aux model GGUF already exists
+  ansible.builtin.stat:
+    path: "{{ llm_aux_model_path }}"
+  register: llm_aux_model_stat
+
+- name: Download aux model — Phi-4-14B-Q4_K_M GGUF
+  ansible.builtin.get_url:
+    url: "{{ llm_aux_model_url }}"
+    dest: "{{ llm_aux_model_path }}"
+    headers:
+      Authorization: "Bearer {{ llm_hf_token }}"
+    owner: "{{ llm_service_user }}"
+    group: "{{ llm_service_user }}"
+    mode: "0644"
+    timeout: 7200
+    force: false
+  become: true
+  no_log: true
+  # Idempotency guard: skip if file exists and is above the min-size threshold
+  # (catches partial/truncated downloads from an interrupted prior run).
+  when: not llm_aux_model_stat.stat.exists or (llm_aux_model_stat.stat.size | int) < (llm_aux_model_min_bytes | int)
+
+# --- Tool-calling model (Mistral-Small-24B Q3_K_M) ---------------------------
+
+- name: Check if tool-calling model GGUF already exists
+  ansible.builtin.stat:
+    path: "{{ llm_toolcall_model_path }}"
+  register: llm_toolcall_model_stat
+
+- name: Download tool-calling model — Mistral-Small-24B-Instruct-2501 Q3_K_M GGUF
+  ansible.builtin.get_url:
+    url: "{{ llm_toolcall_model_url }}"
+    dest: "{{ llm_toolcall_model_path }}"
+    headers:
+      Authorization: "Bearer {{ llm_hf_token }}"
+    owner: "{{ llm_service_user }}"
+    group: "{{ llm_service_user }}"
+    mode: "0644"
+    timeout: 7200
+    force: false
+  become: true
+  no_log: true
+  when: not llm_toolcall_model_stat.stat.exists or (llm_toolcall_model_stat.stat.size | int) < (llm_toolcall_model_min_bytes | int)
+
+- name: Report model files present on disk
+  ansible.builtin.debug:
+    msg:
+      - "Aux model:        {{ llm_aux_model_path }}"
+      - "Tool-calling model: {{ llm_toolcall_model_path }}"
--- a/ansible/roles/llm-inference-multimodel/tasks/systemd.yml
+++ b/ansible/roles/llm-inference-multimodel/tasks/systemd.yml
@@ -0,0 +1,54 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/systemd.yml
+# DESCRIPTION: Phase 2 — template + deploy both unit files.
+#              DELIBERATELY DOES NOT START OR ENABLE either service — that is
+#              Phase 4 (verify.yml)'s job, after Phase 3 firewall scoping is
+#              in place. This keeps "units land on disk" and "processes
+#              actually bind ports and load 20+GB into VRAM" as separately
+#              reviewable checkpoints per Ryan's iterative-build preference.
+#
+#              Two independent units (llama-server-aux.service,
+#              llama-server-toolcall.service) — NOT one unit with two
+#              ExecStarts — so either can be stopped/restarted without
+#              affecting the other (plan §2, §6 rollback requirement).
+#
+#              The pre-existing Gemma unit (whatever discover.yml found it to
+#              be) is never templated, restarted, or disabled by this file.
+# ------------------------------------------------------------------------------
+
+- name: Deploy llama-server-aux systemd unit
+  ansible.builtin.template:
+    src: llama-server-aux.service.j2
+    dest: "/etc/systemd/system/{{ llm_aux_service_name }}.service"
+    owner: root
+    group: root
+    mode: "0644"
+  become: true
+  notify:
+    - reload systemd
+    - restart llama-server-aux
+
+- name: Deploy llama-server-toolcall systemd unit
+  ansible.builtin.template:
+    src: llama-server-toolcall.service.j2
+    dest: "/etc/systemd/system/{{ llm_toolcall_service_name }}.service"
+    owner: root
+    group: root
+    mode: "0644"
+  become: true
+  notify:
+    - reload systemd
+    - restart llama-server-toolcall
+
+- name: Flush handlers so daemon-reload lands before any later phase acts on unit state
+  ansible.builtin.meta: flush_handlers
+
+# NOTE: no `ansible.builtin.systemd: state: started / enabled: true` task here
+# on purpose. Units exist on disk after this phase; nothing is running.
+# The "restart" handlers above only fire (and thus only start anything) if
+# the template content actually changed AND a later flush_handlers/end-of-play
+# triggers them — on a first-ever apply this DOES start the services once,
+# which is expected/acceptable for a fresh deploy, but on any subsequent
+# re-run with no template changes, nothing restarts. Ryan/verify.yml owns
+# the deliberate first start + smoke test.
--- a/ansible/roles/llm-inference-multimodel/tasks/verify.yml
+++ b/ansible/roles/llm-inference-multimodel/tasks/verify.yml
@@ -0,0 +1,133 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/tasks/verify.yml
+# DESCRIPTION: Phase 4 — start both services, curl smoke test each endpoint,
+#              nvidia-smi VRAM check against plan §1 math, confirm no OOM.
+#
+#              This is the ONLY phase that actually starts the services
+#              (systemd.yml deliberately does not). Enabling happens here too,
+#              so a reboot brings both back — matching plan §2's "independent
+#              systemd services" intent for durability, not just this-session.
+# ------------------------------------------------------------------------------
+
+- name: Enable and start llama-server-aux
+  ansible.builtin.systemd:
+    name: "{{ llm_aux_service_name }}"
+    state: started
+    enabled: true
+    daemon_reload: true
+  become: true
+
+- name: Enable and start llama-server-toolcall
+  ansible.builtin.systemd:
+    name: "{{ llm_toolcall_service_name }}"
+    state: started
+    enabled: true
+    daemon_reload: true
+  become: true
+
+- name: Wait for aux instance API to become available (model load may take a couple minutes)
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_aux_port }}/health"
+    status_code: 200
+  register: llm_aux_health
+  retries: 24
+  delay: 10
+  until: llm_aux_health.status == 200
+
+- name: Wait for tool-calling instance API to become available
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_toolcall_port }}/health"
+    status_code: 200
+  register: llm_toolcall_health
+  retries: 24
+  delay: 10
+  until: llm_toolcall_health.status == 200
+
+- name: Smoke-test — aux instance model listing
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_aux_port }}/v1/models"
+    status_code: 200
+    return_content: true
+  register: llm_aux_models
+
+- name: Smoke-test — tool-calling instance model listing
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_toolcall_port }}/v1/models"
+    status_code: 200
+    return_content: true
+  register: llm_toolcall_models
+
+- name: Report served models per instance
+  ansible.builtin.debug:
+    msg:
+      - "Aux (:{{ llm_aux_port }}) serving:        {{ llm_aux_models.json.data | map(attribute='id') | list }}"
+      - "Tool-calling (:{{ llm_toolcall_port }}) serving: {{ llm_toolcall_models.json.data | map(attribute='id') | list }}"
+
+- name: Basic completion smoke test — aux instance (non-tool-calling sanity check only)
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_aux_port }}/v1/chat/completions"
+    method: POST
+    body_format: json
+    body:
+      model: "{{ llm_aux_model_id }}"
+      messages:
+        - role: user
+          content: "Reply with exactly one word: OK"
+      max_tokens: 10
+    status_code: 200
+    return_content: true
+  register: llm_aux_completion
+
+- name: Basic completion smoke test — tool-calling instance (plain-text sanity check only)
+  ansible.builtin.uri:
+    url: "http://{{ llm_bind_address }}:{{ llm_toolcall_port }}/v1/chat/completions"
+    method: POST
+    body_format: json
+    body:
+      model: "{{ llm_toolcall_model_id }}"
+      messages:
+        - role: user
+          content: "Reply with exactly one word: OK"
+      max_tokens: 10
+    status_code: 200
+    return_content: true
+  register: llm_toolcall_completion
+
+- name: NOTE — this smoke test is NOT the tool-calling validation harness
+  ansible.builtin.debug:
+    msg: >-
+      Both endpoints respond to basic completions. This does NOT validate
+      tool_calls correctness or hallucination-safety for the tool-calling
+      instance — that is a separate, manual, post-deploy procedure (plan §7).
+      See references/tool-calling-validation.sh (copied from the
+      llm-inference-homelab skill) and docs/validation-log.md in this role.
+      DO NOT point any Claude Code / tool-calling-capable Hermes profile at
+      port {{ llm_toolcall_port }} until that validation has passed and been
+      logged.
+
+- name: Check GPU VRAM usage after both instances are running
+  ansible.builtin.command:
+    cmd: nvidia-smi --query-gpu=memory.used,memory.total,utilization.gpu --format=csv,noheader
+  register: llm_post_start_vram
+  changed_when: false
+
+- name: Report VRAM usage vs plan §1 expectations
+  ansible.builtin.debug:
+    msg:
+      - "Measured (nvidia-smi):      {{ llm_post_start_vram.stdout }}"
+      - "Design estimate (plan §1):  aux ~{{ llm_aux_expected_vram_gb }}GB + toolcall ~{{ llm_toolcall_expected_vram_gb }}GB = ~{{ llm_combined_expected_vram_gb }}GB / {{ llm_gpu_total_vram_gb }}GB total"
+      - "If measured usage exceeds ~23.5GB or is within ~0.5GB of the 24GB card limit, treat as the OOM-risk trigger condition from plan §6 — do not leave both services running unattended without confirming headroom."
+
+- name: Check for OOM-kill events related to llama-server in dmesg (best-effort, read-only)
+  ansible.builtin.shell:
+    cmd: "dmesg | grep -i 'llama-server' | grep -i -E 'oom|killed' || true"
+  register: llm_oom_check
+  changed_when: false
+  become: true
+
+- name: Report any OOM-kill findings
+  ansible.builtin.debug:
+    msg: >-
+      {{ llm_oom_check.stdout if llm_oom_check.stdout | length > 0
+         else 'No OOM-kill events found for llama-server in dmesg.' }}
--- a/ansible/roles/llm-inference-multimodel/templates/llama-server-aux.service.j2
+++ b/ansible/roles/llm-inference-multimodel/templates/llama-server-aux.service.j2
@@ -0,0 +1,33 @@
+[Unit]
+Description=llama-server (aux/classification) — Phi-4-14B Q4_K_M (OpenAI-compatible inference)
+After=network.target nvidia-persistenced.service
+Wants=nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ llm_service_user }}
+Group={{ llm_service_user }}
+Environment="HOME=/home/{{ llm_service_user }}"
+ExecStart={{ llm_binary_path }} \
+    --model {{ llm_aux_model_path }} \
+    --host {{ llm_bind_address }} \
+    --port {{ llm_aux_port }} \
+    --ctx-size {{ llm_aux_ctx_size }} \
+    --n-gpu-layers {{ llm_aux_gpu_layers }} \
+    --parallel {{ llm_aux_parallel }} \
+    --metrics
+# NOTE: no --chat-template flag — let llama-server auto-detect Phi-4's own
+# embedded chat template from GGUF metadata (same reasoning as the existing
+# llm-inference role's Gemma unit: explicit overrides risk mismatching the
+# model's actual expected format).
+# NOTE: --host is the private LAN IP (10.1.71.130 by default), NOT 0.0.0.0 —
+# deliberate change from the pre-existing Gemma pattern (plan §5).
+Restart=on-failure
+RestartSec=10
+TimeoutStartSec=600
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=llama-server-aux
+
+[Install]
+WantedBy=multi-user.target
--- a/ansible/roles/llm-inference-multimodel/templates/llama-server-toolcall.service.j2
+++ b/ansible/roles/llm-inference-multimodel/templates/llama-server-toolcall.service.j2
@@ -0,0 +1,38 @@
+[Unit]
+Description=llama-server (tool-calling) — Mistral-Small-24B-Instruct-2501 Q3_K_M (OpenAI-compatible inference)
+After=network.target nvidia-persistenced.service
+Wants=nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ llm_service_user }}
+Group={{ llm_service_user }}
+Environment="HOME=/home/{{ llm_service_user }}"
+ExecStart={{ llm_binary_path }} \
+    --model {{ llm_toolcall_model_path }} \
+    --host {{ llm_bind_address }} \
+    --port {{ llm_toolcall_port }} \
+    --ctx-size {{ llm_toolcall_ctx_size }} \
+    --n-gpu-layers {{ llm_toolcall_gpu_layers }} \
+    --parallel {{ llm_toolcall_parallel }} \
+    --metrics
+# NOTE: no --chat-template flag — let llama-server auto-detect Mistral-Small's
+# own embedded chat template from GGUF metadata.
+# NOTE: --host is the private LAN IP (10.1.71.130 by default), NOT 0.0.0.0.
+# NOTE: --parallel 1 is deliberate (plan §1/§2) — tool-calling profiles are
+# single-session-at-a-time per Claude Code profile; lower parallelism reduces
+# KV overhead and lowers hallucination surface from context bleed between
+# concurrent slots.
+# IMPORTANT: this endpoint MUST pass the plan §7 validation harness
+# (docs/validation-log.md in this role) before any Claude Code / tool-calling
+# Hermes profile is pointed at it. Mistral-Small shares lineage concerns
+# flagged for Qwen2.5/Qwen3 hallucinated tool_calls — do not assume safety.
+Restart=on-failure
+RestartSec=10
+TimeoutStartSec=600
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=llama-server-toolcall
+
+[Install]
+WantedBy=multi-user.target
--- a/ansible/roles/llm-inference-multimodel/vars/main.yml
+++ b/ansible/roles/llm-inference-multimodel/vars/main.yml
@@ -0,0 +1,22 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference-multimodel/vars/main.yml
+# DESCRIPTION: Role-internal constants (not meant to be overridden per-host).
+#              Model URLs/quant filenames live here rather than defaults/ since
+#              they're not really "tunable" — they're the specific artifacts
+#              named in the approved plan (§1). If Ryan wants a different
+#              quant/model, that's a defaults/main.yml override or a plan
+#              revision, not a vars/ edit.
+# ------------------------------------------------------------------------------
+
+# HuggingFace auth — reuse the same vault token as roles/llm-inference.
+# vault_hf_token is defined in group_vars/all/vault.
+llm_hf_token: "{{ vault_hf_token }}"
+
+# Expected VRAM subtotals from plan §1 (informational — surfaced in verify.yml
+# output so a live nvidia-smi reading can be sanity-checked against the design
+# math, not enforced as a hard gate).
+llm_aux_expected_vram_gb: 10.0
+llm_toolcall_expected_vram_gb: 13.2
+llm_combined_expected_vram_gb: 23.2
+llm_gpu_total_vram_gb: 24.0
--- a/ansible/roles/llm-inference/defaults/main.yml
+++ b/ansible/roles/llm-inference/defaults/main.yml
@@ -0,0 +1,55 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/defaults/main.yml
+# DESCRIPTION: Overridable defaults for the llm-inference role.
+#              Deploy target: astro-orbiter (Dell OptiPlex 7050 SFF, RTX 3090
+#              via OCuLink, Ubuntu 24.04.4 LTS).
+# ------------------------------------------------------------------------------
+
+# NVIDIA driver
+llm_nvidia_driver_package: nvidia-driver-595-open
+
+# Python venv
+llm_venv_path: /home/jarvis/vllm-env
+llm_venv_owner: jarvis
+
+# HuggingFace
+llm_hf_cache_dir: /home/jarvis/.cache/huggingface
+llm_hf_model: google/gemma-2-27b-it
+
+# vLLM serve (deprecated — replaced by llama-server)
+# llama-server serve
+llm_serve_port: 8000
+llm_serve_host: "0.0.0.0"
+# NOTE (2026-08-05): --ctx-size is llama.cpp's TOTAL KV cache pool, divided
+# evenly across --parallel slots (per-slot context = ctx-size / parallel).
+# Previous 8192/4=2048 tokens-per-slot was too small for aux task offload
+# (context compression) and caused live rejections: "request (3826 tokens)
+# exceeds the available context size (2048 tokens)".
+# Sized against measured VRAM on astro-orbiter (RTX 3090, 24576MiB total):
+#   - Weights (Q4_K_M, 27B) ~16998MiB resident.
+#   - At ctx-size=8192/parallel=4, total llama-server VRAM = 19404MiB
+#     (nvidia-smi), i.e. ~2406MiB for KV cache + compute buffers at 8192
+#     total context tokens -> ~294KiB/token (pool-wide, incl. buffers).
+#   - Model n_ctx_train=8192 is the native max; per-slot context beyond
+#     this degrades coherence, so per-slot should cap at 8192.
+#   - New sizing: ctx-size=16384, parallel=2 -> 8192 tokens/slot (native
+#     max, covers compression's multi-thousand-token inputs with margin).
+#     Projected VRAM: 16998 + (~294KiB/token * 16384) ≈ 21.8GB used,
+#     leaving ~2.7GB headroom on the 24GB card.
+#   - parallel=2 (not 4) trades some concurrency for correct per-slot
+#     context; 2 concurrent aux-task requests is enough headroom before
+#     the known "3+ simultaneous compressions" GPU bottleneck kicks in.
+llm_max_model_len: 16384
+llm_gpu_layers: 99              # offload all layers to GPU
+llm_parallel_slots: 2           # concurrent request slots -> 8192 tokens/slot (ctx-size / parallel)
+llm_gguf_dir: /home/jarvis/models
+llm_gguf_path: /home/jarvis/models/gemma-2-27b-it-Q4_K_M.gguf
+
+# Legacy vLLM vars (kept for role documentation, not used by llama-server)
+llm_quantization: "bitsandbytes"
+llm_gpu_memory_utilization: "0.92"
+
+# Monitoring
+llm_gpu_exporter_version: "1.13.1"
+llm_gpu_exporter_port: 9835
--- a/ansible/roles/llm-inference/handlers/main.yml
+++ b/ansible/roles/llm-inference/handlers/main.yml
@@ -0,0 +1,42 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/handlers/main.yml
+# ------------------------------------------------------------------------------
+
+- name: Reload systemd
+  ansible.builtin.systemd:
+    daemon_reload: true
+  listen: "reload systemd"
+
+- name: Restart vllm-serve
+  ansible.builtin.systemd:
+    name: vllm-serve
+    state: restarted
+  listen: "restart vllm-serve"
+  failed_when: false
+
+- name: Restart llama-server
+  ansible.builtin.systemd:
+    name: llama-server
+    state: restarted
+  listen: "restart llama-server"
+
+- name: Restart Hermes on carousel
+  ansible.builtin.systemd:
+    name: "{{ item }}"
+    state: restarted
+    scope: user
+  loop:
+    - hermes-gateway
+    - hermes-dashboard
+  become: true
+  become_user: wed
+  delegate_to: carousel-of-progress
+  listen: "restart hermes"
+  ignore_errors: true
+
+- name: Restart nvidia-gpu-exporter
+  ansible.builtin.systemd:
+    name: nvidia-gpu-exporter
+    state: restarted
+  listen: "restart nvidia-gpu-exporter"
--- a/ansible/roles/llm-inference/meta/main.yml
+++ b/ansible/roles/llm-inference/meta/main.yml
@@ -0,0 +1,14 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/meta/main.yml
+# ------------------------------------------------------------------------------
+galaxy_info:
+  role_name: llm_inference
+  author: rblundon
+  license: MIT
+  description: >
+    Deploys vLLM serving stack with NVIDIA RTX 3090 on Ubuntu 24.04.
+    Manages NVIDIA drivers, Python venv, model download, systemd service,
+    and Hermes provider integration.
+  min_ansible_version: "2.15"
+dependencies: []
--- a/ansible/roles/llm-inference/tasks/driver.yml
+++ b/ansible/roles/llm-inference/tasks/driver.yml
@@ -0,0 +1,25 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/driver.yml
+# DESCRIPTION: Phase 2 — NVIDIA driver.
+#              Installs nvidia-driver-595-open via apt. Fully idempotent —
+#              already installed on astro-orbiter on 2026-08-03, this is a no-op.
+#              DKMS builds the kernel module automatically on install.
+# ------------------------------------------------------------------------------
+
+- name: Install NVIDIA driver package
+  ansible.builtin.apt:
+    name: "{{ llm_nvidia_driver_package }}"
+    state: present
+    update_cache: false
+  notify: reload systemd
+
+- name: Verify nvidia-smi reports the GPU
+  ansible.builtin.command: nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
+  register: nvidia_smi_out
+  changed_when: false
+  failed_when: nvidia_smi_out.rc != 0
+
+- name: Print nvidia-smi output
+  ansible.builtin.debug:
+    msg: "GPU detected: {{ nvidia_smi_out.stdout }}"
--- a/ansible/roles/llm-inference/tasks/foundation.yml
+++ b/ansible/roles/llm-inference/tasks/foundation.yml
@@ -0,0 +1,45 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/foundation.yml
+# DESCRIPTION: Phase 1 — Foundation.
+#              - Asserts vault secret is defined
+#              - Adds jarvis user to nvidia GPU groups
+#              - Creates HuggingFace cache directory
+#              - Creates venv parent directory
+# NOTE: NVIDIA driver install (Phase 2) already performed manually on
+#       2026-08-03 (nvidia-driver-595-open, DKMS built, nvidia-smi verified).
+#       Phase 2 tasks are idempotent and will no-op on astro-orbiter.
+# ------------------------------------------------------------------------------
+
+- name: Assert HuggingFace token is defined in vault
+  ansible.builtin.assert:
+    that:
+      - vault_hf_token is defined
+      - vault_hf_token | length > 0
+    fail_msg: >
+      vault_hf_token is not defined. Add it to group_vars/all/vault:
+        vault_hf_token: "hf_xxxxxxxxxxxxxxxxxxxx"
+
+- name: Add jarvis user to nvidia GPU groups
+  ansible.builtin.user:
+    name: jarvis
+    groups:
+      - video
+      - render
+    append: true
+
+- name: Create HuggingFace cache directory
+  ansible.builtin.file:
+    path: "{{ llm_hf_cache_dir }}"
+    state: directory
+    owner: jarvis
+    group: jarvis
+    mode: "0755"
+
+- name: Create venv parent directory
+  ansible.builtin.file:
+    path: "{{ llm_venv_path | dirname }}"
+    state: directory
+    owner: jarvis
+    group: jarvis
+    mode: "0755"
--- a/ansible/roles/llm-inference/tasks/integration.yml
+++ b/ansible/roles/llm-inference/tasks/integration.yml
@@ -0,0 +1,44 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/integration.yml
+# DESCRIPTION: Phase 6 — Wire astro-orbiter into Hermes as a secondary provider.
+#              Writes a provider config fragment to carousel-of-progress
+#              (the Hermes host) so FRIDAY crons can route to the local model.
+#
+# Hermes provider config lives at ~/.hermes/config.yaml on carousel.
+# This task uses the lineinfile/blockinfile approach to add the provider
+# entry idempotently without clobbering the existing config.
+#
+# NOTE: Hermes must be restarted on carousel after this task runs.
+#       Manual step — JARVIS will notify Ryan.
+# ------------------------------------------------------------------------------
+
+- name: Check if astro-orbiter provider already configured in Hermes
+  ansible.builtin.command:
+    cmd: grep -c "astro-orbiter" /home/wed/.hermes/config.yaml
+  register: provider_check
+  changed_when: false
+  failed_when: false
+  delegate_to: carousel-of-progress
+
+- name: Add astro-orbiter as Hermes secondary provider
+  ansible.builtin.blockinfile:
+    path: /home/wed/.hermes/config.yaml
+    marker: "# {mark} ANSIBLE MANAGED — astro-orbiter vLLM provider"
+    insertafter: "^providers:"
+    block: |
+      # astro-orbiter — local RTX 3090 vLLM inference
+      - name: astro-orbiter
+        type: openai-compatible
+        base_url: http://{{ hostvars['astro-orbiter']['ansible_host'] }}:{{ llm_serve_port }}/v1
+        model: {{ llm_hf_model }}
+        api_key: none
+  when: provider_check.stdout == "0"
+  delegate_to: carousel-of-progress
+  notify: restart hermes
+
+- name: Remind operator to restart Hermes on carousel
+  ansible.builtin.debug:
+    msg: >
+      Phase 6 complete. Hermes on carousel-of-progress has been updated.
+      Restart Hermes manually or via: systemctl --user restart hermes-gateway hermes-dashboard
--- a/ansible/roles/llm-inference/tasks/main.yml
+++ b/ansible/roles/llm-inference/tasks/main.yml
@@ -0,0 +1,36 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/main.yml
+# DESCRIPTION: Entry point — imports one task file per phase.
+#              Phases are additive; re-running the full playbook is always safe.
+#              Use --tags to run a specific phase subset:
+#                --tags foundation,driver,vllm,model,serve,integration,monitoring
+# ------------------------------------------------------------------------------
+
+# Phase 1 — Foundation
+- import_tasks: foundation.yml
+  tags: [foundation]
+
+# Phase 2 — Driver
+- import_tasks: driver.yml
+  tags: [driver]
+
+# Phase 3 — vLLM
+- import_tasks: vllm.yml
+  tags: [vllm]
+
+# Phase 4 — Model
+- import_tasks: model.yml
+  tags: [model]
+
+# Phase 5 — Serve
+- import_tasks: serve.yml
+  tags: [serve]
+
+# Phase 6 — Integration
+- import_tasks: integration.yml
+  tags: [integration]
+
+# Phase 7 — Monitoring
+- import_tasks: monitoring.yml
+  tags: [monitoring]
--- a/ansible/roles/llm-inference/tasks/model.yml
+++ b/ansible/roles/llm-inference/tasks/model.yml
@@ -0,0 +1,42 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/model.yml
+# DESCRIPTION: Phase 4 — HuggingFace login and Gemma 2 27B model download.
+#              Idempotent: snapshot_download skips files already present.
+#              Requires vault_hf_token and Gemma 2 licence accepted at
+#              huggingface.co/google/gemma-2-27b-it.
+# ------------------------------------------------------------------------------
+
+- name: Write HuggingFace token to ~/.cache/huggingface/token
+  ansible.builtin.copy:
+    content: "{{ vault_hf_token }}"
+    dest: "/home/{{ llm_venv_owner }}/.cache/huggingface/token"
+    owner: "{{ llm_venv_owner }}"
+    group: "{{ llm_venv_owner }}"
+    mode: "0600"
+  no_log: true
+
+- name: Download Gemma 2 27B model via snapshot_download
+  ansible.builtin.command:
+    cmd: >
+      {{ llm_venv_path }}/bin/python -c "
+      from huggingface_hub import snapshot_download
+      path = snapshot_download(
+          '{{ llm_hf_model }}',
+          cache_dir='{{ llm_hf_cache_dir }}',
+      )
+      print(path)
+      "
+    creates: "{{ llm_hf_cache_dir }}/models--{{ llm_hf_model | replace('/', '--') }}/snapshots"
+  become: true
+  become_user: "{{ llm_venv_owner }}"
+  environment:
+    HF_TOKEN: "{{ vault_hf_token }}"
+    HOME: "/home/{{ llm_venv_owner }}"
+  register: model_download
+  timeout: 3600
+  no_log: false
+
+- name: Print model download path
+  ansible.builtin.debug:
+    msg: "Model available at: {{ model_download.stdout | default('already present') }}"
--- a/ansible/roles/llm-inference/tasks/monitoring.yml
+++ b/ansible/roles/llm-inference/tasks/monitoring.yml
@@ -0,0 +1,131 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/monitoring.yml
+# DESCRIPTION: Phase 7 — Prometheus monitoring for the LLM inference stack.
+#              Deploys two metric-producing exporters on astro-orbiter:
+#
+#              1. node_exporter (port 9100) — system: CPU, RAM, disk, network
+#              2. nvidia_gpu_exporter (port 9835) — GPU: VRAM, temp, util, power
+#              3. llama-server built-in metrics (port 8000/metrics, enabled via
+#                 --metrics) — just needs a scrape job (no extra process)
+#
+#              GitOps note: the Prometheus scrape jobs for all three targets and
+#              the Grafana dashboard are declared in the homelab Git repo and
+#              applied by ArgoCD — NOT by this role:
+#                - cluster/applications/monitoring/values.yaml
+#                  (prometheus.prometheusSpec.additionalScrapeConfigs)
+#                - cluster/applications/monitoring/dashboards.yaml
+#                  (grafana-llm-inference-dashboard ConfigMap)
+#              This role's job is only to stand up the two exporters + verify
+#              they're reachable. Do NOT reintroduce kubectl patch/apply tasks
+#              here — cluster-facing changes go through Git commit + ArgoCD
+#              sync so state stays reproducible and self-healing.
+# ------------------------------------------------------------------------------
+
+# -----------------------------------------------------------------------
+# 1. node_exporter
+# -----------------------------------------------------------------------
+
+- name: Install prometheus-node-exporter
+  ansible.builtin.apt:
+    name: prometheus-node-exporter
+    state: present
+    update_cache: false
+
+- name: Enable and start node_exporter
+  ansible.builtin.systemd:
+    name: prometheus-node-exporter
+    state: started
+    enabled: true
+
+- name: Verify node_exporter is reachable
+  ansible.builtin.uri:
+    url: "http://localhost:9100/metrics"
+    status_code: 200
+  register: node_exporter_health
+  retries: 6
+  delay: 5
+  until: node_exporter_health.status == 200
+  changed_when: false
+
+# -----------------------------------------------------------------------
+# 2. nvidia_gpu_exporter (utkuozdemir/nvidia_gpu_exporter)
+#    Lightweight single-binary exporter — no CUDA dependency, uses nvidia-smi.
+# -----------------------------------------------------------------------
+
+- name: Create nvidia_gpu_exporter install directory
+  ansible.builtin.file:
+    path: /opt/nvidia_gpu_exporter
+    state: directory
+    owner: root
+    group: root
+    mode: "0755"
+
+- name: Download nvidia_gpu_exporter binary
+  ansible.builtin.get_url:
+    url: "https://github.com/utkuozdemir/nvidia_gpu_exporter/releases/download/v{{ llm_gpu_exporter_version }}/nvidia_gpu_exporter_{{ llm_gpu_exporter_version }}_linux_x86_64.tar.gz"
+    dest: "/tmp/nvidia_gpu_exporter.tar.gz"
+    mode: "0644"
+  register: gpu_exporter_download
+
+- name: Extract nvidia_gpu_exporter binary
+  ansible.builtin.unarchive:
+    src: /tmp/nvidia_gpu_exporter.tar.gz
+    dest: /opt/nvidia_gpu_exporter
+    remote_src: true
+    creates: /opt/nvidia_gpu_exporter/nvidia_gpu_exporter
+
+- name: Deploy nvidia_gpu_exporter systemd service
+  ansible.builtin.copy:
+    dest: /etc/systemd/system/nvidia-gpu-exporter.service
+    mode: "0644"
+    content: |
+      [Unit]
+      Description=NVIDIA GPU Prometheus Exporter
+      After=network.target nvidia-persistenced.service
+      Wants=nvidia-persistenced.service
+
+      [Service]
+      Type=simple
+      ExecStart=/opt/nvidia_gpu_exporter/nvidia_gpu_exporter \
+          --web.listen-address=:{{ llm_gpu_exporter_port }}
+      Restart=on-failure
+      RestartSec=5
+      StandardOutput=journal
+      StandardError=journal
+      SyslogIdentifier=nvidia-gpu-exporter
+
+      [Install]
+      WantedBy=multi-user.target
+  notify:
+    - reload systemd
+    - restart nvidia-gpu-exporter
+
+- name: Flush handlers before starting gpu exporter
+  ansible.builtin.meta: flush_handlers
+
+- name: Enable and start nvidia-gpu-exporter
+  ansible.builtin.systemd:
+    name: nvidia-gpu-exporter
+    state: started
+    enabled: true
+    daemon_reload: true
+
+- name: Verify nvidia_gpu_exporter is reachable
+  ansible.builtin.uri:
+    url: "http://localhost:{{ llm_gpu_exporter_port }}/metrics"
+    status_code: 200
+  register: gpu_exporter_health
+  retries: 6
+  delay: 5
+  until: gpu_exporter_health.status == 200
+  changed_when: false
+
+# -----------------------------------------------------------------------
+# 3. Prometheus scrape configs + Grafana dashboard
+#
+#    Intentionally NOT managed here. See file header: these are declared
+#    in cluster/applications/monitoring/{values.yaml,dashboards.yaml} in
+#    the homelab Git repo and rolled out by ArgoCD sync, keeping cluster
+#    state in Git rather than mutated imperatively from the control node.
+# -----------------------------------------------------------------------
--- a/ansible/roles/llm-inference/tasks/serve.yml
+++ b/ansible/roles/llm-inference/tasks/serve.yml
@@ -0,0 +1,151 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/serve.yml
+# DESCRIPTION: Phase 5 — llama-server (llama.cpp) serving Gemma 2 27B-it GGUF.
+#
+# WHY llama.cpp instead of vLLM:
+#   vLLM with bitsandbytes int4 quantizes on-the-fly — loads full bf16 weights
+#   (~54GB RAM peak) before compressing, killing the 40GB OptiPlex on warmup.
+#   llama.cpp loads the pre-quantized GGUF directly (~15.5GB peak RAM for Q4_K_M).
+#   No torch.compile, no warmup spike, OpenAI-compatible API on the same port.
+#
+# GGUF source: bartowski/gemma-2-27b-it-GGUF (Q4_K_M, 15.5GB)
+# Model downloaded to: {{ llm_gguf_path }}
+# ------------------------------------------------------------------------------
+
+- name: Install llama.cpp build dependencies
+  ansible.builtin.apt:
+    name:
+      - cmake
+      - build-essential
+      - libcurl4-openssl-dev
+    state: present
+    update_cache: false
+
+# NOTE: nvidia-driver-595-open provides the runtime driver only (nvidia-smi,
+# libcuda.so) — it does NOT ship nvcc/CUDA headers needed to build GGML_CUDA=ON.
+# Ubuntu 24.04's nvidia-cuda-toolkit (12.0.x) is sufficient to build llama.cpp
+# against; it does not need to match the 595 driver's CUDA 13.2 runtime version.
+- name: Install NVIDIA CUDA toolkit (nvcc) for building llama.cpp with CUDA support
+  ansible.builtin.apt:
+    name: nvidia-cuda-toolkit
+    state: present
+    update_cache: false
+  become: true
+
+- name: Clone llama.cpp repository
+  ansible.builtin.git:
+    repo: https://github.com/ggml-org/llama.cpp.git
+    dest: /opt/llama.cpp
+    depth: 1
+    update: false
+  become: true
+
+- name: Check for incomplete/stale llama.cpp CMake configuration
+  ansible.builtin.stat:
+    path: /opt/llama.cpp/build/Makefile
+  register: llama_cmake_generated
+
+- name: Remove stale llama.cpp build dir if CMake configure never completed
+  ansible.builtin.file:
+    path: /opt/llama.cpp/build
+    state: absent
+  become: true
+  when:
+    - not llama_cmake_generated.stat.exists
+    - not (ansible_check_mode | default(false))
+
+- name: Build llama.cpp with CUDA support
+  ansible.builtin.command:
+    cmd: cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
+    chdir: /opt/llama.cpp
+    creates: /opt/llama.cpp/build/CMakeCache.txt
+  become: true
+
+- name: Compile llama.cpp (parallel build)
+  ansible.builtin.command:
+    cmd: cmake --build build --config Release --parallel {{ ansible_processor_vcpus }}
+    chdir: /opt/llama.cpp
+    creates: /opt/llama.cpp/build/bin/llama-server
+  become: true
+  timeout: 600
+
+- name: Create GGUF model directory
+  ansible.builtin.file:
+    path: "{{ llm_gguf_dir }}"
+    state: directory
+    owner: "{{ llm_venv_owner }}"
+    group: "{{ llm_venv_owner }}"
+    mode: "0755"
+
+- name: Check whether GGUF already exists (avoid re-downloading 16.6GB on every run)
+  ansible.builtin.stat:
+    path: "{{ llm_gguf_path }}"
+  register: llm_gguf_stat
+
+- name: Download Gemma 2 27B Q4_K_M GGUF from HuggingFace
+  ansible.builtin.get_url:
+    url: "https://huggingface.co/bartowski/gemma-2-27b-it-GGUF/resolve/main/gemma-2-27b-it-Q4_K_M.gguf"
+    dest: "{{ llm_gguf_path }}"
+    headers:
+      Authorization: "Bearer {{ vault_hf_token }}"
+    owner: "{{ llm_venv_owner }}"
+    group: "{{ llm_venv_owner }}"
+    mode: "0644"
+    timeout: 7200
+    force: false
+  become: true
+  no_log: true
+  # Idempotency: skip entirely once the file exists and is reasonably sized
+  # (the finished GGUF is ~16.6GB; guard against a truncated partial download
+  # being mistaken for complete by only trusting files > 15GB).
+  when: not llm_gguf_stat.stat.exists or (llm_gguf_stat.stat.size | int) < 15000000000
+
+- name: Disable and stop vllm-serve if present
+  ansible.builtin.systemd:
+    name: vllm-serve
+    state: stopped
+    enabled: false
+  failed_when: false
+  notify: reload systemd
+
+- name: Deploy llama-server systemd service unit
+  ansible.builtin.template:
+    src: llama-server.service.j2
+    dest: /etc/systemd/system/llama-server.service
+    owner: root
+    group: root
+    mode: "0644"
+  notify:
+    - reload systemd
+    - restart llama-server
+
+- name: Flush handlers to reload systemd before enabling service
+  ansible.builtin.meta: flush_handlers
+
+- name: Enable and start llama-server
+  ansible.builtin.systemd:
+    name: llama-server
+    state: started
+    enabled: true
+    daemon_reload: true
+
+- name: Wait for llama-server API to become available (model load ~30s)
+  ansible.builtin.uri:
+    url: "http://localhost:{{ llm_serve_port }}/health"
+    status_code: 200
+  register: llama_health
+  retries: 18
+  delay: 10
+  until: llama_health.status == 200
+
+- name: Smoke-test — list available models
+  ansible.builtin.uri:
+    url: "http://localhost:{{ llm_serve_port }}/v1/models"
+    status_code: 200
+    return_content: true
+  register: llama_models
+
+- name: Print available models
+  ansible.builtin.debug:
+    msg: "llama-server serving: {{ llama_models.json.data | map(attribute='id') | list }}"
--- a/ansible/roles/llm-inference/tasks/vllm.yml
+++ b/ansible/roles/llm-inference/tasks/vllm.yml
@@ -0,0 +1,43 @@
+---
+# ------------------------------------------------------------------------------
+# FILE: roles/llm-inference/tasks/vllm.yml
+# DESCRIPTION: Phase 3 — Python venv + vLLM install.
+#              Idempotent: venv creation and pip install only run if the
+#              venv binary or vllm package is absent.
+#              Already completed manually on 2026-08-03 — will no-op.
+# ------------------------------------------------------------------------------
+
+- name: Create Python venv for vLLM
+  ansible.builtin.command:
+    cmd: python3 -m venv {{ llm_venv_path }}
+    creates: "{{ llm_venv_path }}/bin/python"
+  become: true
+  become_user: "{{ llm_venv_owner }}"
+
+- name: Upgrade pip inside venv
+  ansible.builtin.pip:
+    name: pip
+    state: latest
+    virtualenv: "{{ llm_venv_path }}"
+  become: true
+  become_user: "{{ llm_venv_owner }}"
+
+- name: Install vLLM and bitsandbytes
+  ansible.builtin.pip:
+    name:
+      - vllm
+      - bitsandbytes
+    state: present
+    virtualenv: "{{ llm_venv_path }}"
+  become: true
+  become_user: "{{ llm_venv_owner }}"
+
+- name: Verify vLLM is importable
+  ansible.builtin.command:
+    cmd: "{{ llm_venv_path }}/bin/python -c 'import vllm; print(vllm.__version__)'"
+  register: vllm_version
+  changed_when: false
+
+- name: Print vLLM version
+  ansible.builtin.debug:
+    msg: "vLLM version: {{ vllm_version.stdout }}"
--- a/ansible/roles/llm-inference/templates/llama-server.service.j2
+++ b/ansible/roles/llm-inference/templates/llama-server.service.j2
@@ -0,0 +1,32 @@
+[Unit]
+Description=llama-server — Gemma 2 27B-it Q4_K_M (OpenAI-compatible inference)
+After=network.target nvidia-persistenced.service
+Wants=nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ llm_venv_owner }}
+Group={{ llm_venv_owner }}
+Environment="HOME=/home/{{ llm_venv_owner }}"
+ExecStart=/opt/llama.cpp/build/bin/llama-server \
+    --model {{ llm_gguf_path }} \
+    --host {{ llm_serve_host }} \
+    --port {{ llm_serve_port }} \
+    --ctx-size {{ llm_max_model_len }} \
+    --n-gpu-layers {{ llm_gpu_layers }} \
+    --parallel {{ llm_parallel_slots }} \
+    --metrics
+# NOTE: no --chat-template flag — llama-server auto-detects and uses the
+# GGUF's own embedded Jinja chat template (verified correct Gemma-2
+# start_of_turn/end_of_turn format for bartowski's gemma-2-27b-it-Q4_K_M).
+# The built-in "--chat-template gemma" name does NOT match this model's
+# expected format on this llama.cpp build and produced garbled completions.
+Restart=on-failure
+RestartSec=10
+TimeoutStartSec=120
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=llama-server
+
+[Install]
+WantedBy=multi-user.target
--- a/ansible/roles/llm-inference/templates/vllm-serve.service.j2
+++ b/ansible/roles/llm-inference/templates/vllm-serve.service.j2
@@ -0,0 +1,29 @@
+[Unit]
+Description=vLLM inference server — {{ llm_hf_model }}
+After=network.target nvidia-persistenced.service
+Wants=nvidia-persistenced.service
+
+[Service]
+Type=simple
+User={{ llm_venv_owner }}
+Group={{ llm_venv_owner }}
+Environment="HF_TOKEN={{ vault_hf_token }}"
+Environment="HOME=/home/{{ llm_venv_owner }}"
+Environment="HF_HUB_CACHE={{ llm_hf_cache_dir }}"
+ExecStart={{ llm_venv_path }}/bin/python -m vllm.entrypoints.openai.api_server \
+    --model {{ llm_hf_model }} \
+    --host {{ llm_serve_host }} \
+    --port {{ llm_serve_port }} \
+    --quantization {{ llm_quantization }} \
+    --gpu-memory-utilization {{ llm_gpu_memory_utilization }} \
+    --max-model-len {{ llm_max_model_len }} \
+    --enable-prefix-caching
+Restart=on-failure
+RestartSec=10
+TimeoutStartSec=300
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=vllm-serve
+
+[Install]
+WantedBy=multi-user.target
--- a/cluster/applications/monitoring/dashboards.yaml
+++ b/cluster/applications/monitoring/dashboards.yaml
@@ -9199,3 +9199,230 @@ data:
      "version": 1,
      "description": "Ingress-nginx supports a rich collection of prometheus metrics. If you have prometheus and grafana installed on your cluster then prometheus will already be scraping this data due to the scrape annotation on the deployment."
    }
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: grafana-llm-inference-dashboard
+  namespace: monitoring
+  labels:
+    grafana_dashboard: "1"
+data:
+  llm-inference-astro-orbiter.json: |
+    {
+      "title": "LLM Inference — astro-orbiter",
+      "uid": "llm-astro-orbiter",
+      "timezone": "browser",
+      "refresh": "30s",
+      "schemaVersion": 38,
+      "tags": ["llm", "gpu", "llama-server", "astro-orbiter"],
+      "time": { "from": "now-1h", "to": "now" },
+      "panels": [
+        {
+          "id": 1,
+          "title": "GPU Utilization %",
+          "type": "timeseries",
+          "gridPos": { "x": 0, "y": 0, "w": 8, "h": 8 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "percent",
+              "min": 0, "max": 100,
+              "color": { "mode": "palette-classic" },
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  { "color": "green", "value": null },
+                  { "color": "yellow", "value": 70 },
+                  { "color": "red", "value": 90 }
+                ]
+              }
+            }
+          },
+          "targets": [
+            {
+              "expr": "nvidia_smi_utilization_gpu_ratio{hostname=\"astro-orbiter\"} * 100",
+              "legendFormat": "GPU Util"
+            }
+          ]
+        },
+        {
+          "id": 2,
+          "title": "GPU VRAM Used",
+          "type": "timeseries",
+          "gridPos": { "x": 8, "y": 0, "w": 8, "h": 8 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "bytes",
+              "color": { "mode": "palette-classic" }
+            }
+          },
+          "targets": [
+            {
+              "expr": "nvidia_smi_memory_used_bytes{hostname=\"astro-orbiter\"}",
+              "legendFormat": "VRAM Used"
+            },
+            {
+              "expr": "nvidia_smi_memory_total_bytes{hostname=\"astro-orbiter\"}",
+              "legendFormat": "VRAM Total"
+            }
+          ]
+        },
+        {
+          "id": 3,
+          "title": "GPU Temperature",
+          "type": "gauge",
+          "gridPos": { "x": 16, "y": 0, "w": 8, "h": 8 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "celsius",
+              "min": 0, "max": 100,
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  { "color": "green", "value": null },
+                  { "color": "yellow", "value": 70 },
+                  { "color": "red", "value": 85 }
+                ]
+              }
+            }
+          },
+          "targets": [
+            {
+              "expr": "nvidia_smi_temperature_gpu{hostname=\"astro-orbiter\"}",
+              "legendFormat": "GPU Temp"
+            }
+          ]
+        },
+        {
+          "id": 4,
+          "title": "GPU Power Draw",
+          "type": "timeseries",
+          "gridPos": { "x": 0, "y": 8, "w": 8, "h": 8 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "watt",
+              "color": { "mode": "palette-classic" }
+            }
+          },
+          "targets": [
+            {
+              "expr": "nvidia_smi_power_draw_watts{hostname=\"astro-orbiter\"}",
+              "legendFormat": "Power Draw"
+            },
+            {
+              "expr": "nvidia_smi_power_limit_watts{hostname=\"astro-orbiter\"}",
+              "legendFormat": "Power Limit"
+            }
+          ]
+        },
+        {
+          "id": 5,
+          "title": "llama-server — Requests per Second",
+          "type": "timeseries",
+          "gridPos": { "x": 8, "y": 8, "w": 8, "h": 8 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "reqps",
+              "color": { "mode": "palette-classic" }
+            }
+          },
+          "targets": [
+            {
+              "expr": "rate(llamacpp:tokens_predicted_total{hostname=\"astro-orbiter\"}[1m])",
+              "legendFormat": "Tokens/s (predicted)"
+            },
+            {
+              "expr": "rate(llamacpp:prompt_tokens_total{hostname=\"astro-orbiter\"}[1m])",
+              "legendFormat": "Tokens/s (prompt)"
+            }
+          ]
+        },
+        {
+          "id": 6,
+          "title": "llama-server — Requests Processing/Deferred",
+          "type": "timeseries",
+          "gridPos": { "x": 16, "y": 8, "w": 8, "h": 8 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "short",
+              "color": { "mode": "palette-classic" }
+            }
+          },
+          "targets": [
+            {
+              "expr": "llamacpp:requests_processing{hostname=\"astro-orbiter\"}",
+              "legendFormat": "Processing"
+            },
+            {
+              "expr": "llamacpp:requests_deferred{hostname=\"astro-orbiter\"}",
+              "legendFormat": "Deferred"
+            }
+          ]
+        },
+        {
+          "id": 9,
+          "title": "System — CPU Usage %",
+          "type": "timeseries",
+          "gridPos": { "x": 0, "y": 16, "w": 8, "h": 7 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "percent",
+              "min": 0, "max": 100
+            }
+          },
+          "targets": [
+            {
+              "expr": "100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=\"idle\",instance=~\"10.1.71.130:.*\"}[1m])) * 100)",
+              "legendFormat": "CPU Used"
+            }
+          ]
+        },
+        {
+          "id": 10,
+          "title": "System — Memory Usage",
+          "type": "timeseries",
+          "gridPos": { "x": 8, "y": 16, "w": 8, "h": 7 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "bytes"
+            }
+          },
+          "targets": [
+            {
+              "expr": "node_memory_MemTotal_bytes{instance=~\"10.1.71.130:.*\"} - node_memory_MemAvailable_bytes{instance=~\"10.1.71.130:.*\"}",
+              "legendFormat": "Used"
+            },
+            {
+              "expr": "node_memory_MemTotal_bytes{instance=~\"10.1.71.130:.*\"}",
+              "legendFormat": "Total"
+            }
+          ]
+        },
+        {
+          "id": 11,
+          "title": "System — Disk Usage (root)",
+          "type": "gauge",
+          "gridPos": { "x": 16, "y": 16, "w": 8, "h": 7 },
+          "fieldConfig": {
+            "defaults": {
+              "unit": "percentunit",
+              "min": 0, "max": 1,
+              "thresholds": {
+                "mode": "absolute",
+                "steps": [
+                  { "color": "green", "value": null },
+                  { "color": "yellow", "value": 0.75 },
+                  { "color": "red", "value": 0.90 }
+                ]
+              }
+            }
+          },
+          "targets": [
+            {
+              "expr": "1 - (node_filesystem_avail_bytes{instance=~\"10.1.71.130:.*\",mountpoint=\"/\"} / node_filesystem_size_bytes{instance=~\"10.1.71.130:.*\",mountpoint=\"/\"})",
+              "legendFormat": "Root disk"
+            }
+          ]
+        }
+      ]
+    }
--- a/cluster/applications/monitoring/values.yaml
+++ b/cluster/applications/monitoring/values.yaml
@@ -212,6 +212,36 @@ prometheus:
      #         array: utilidor
      #         array_type: physical

+      # astro-orbiter — LLM inference host (Ryzen 7 5800XT / RTX 3090)
+      # Managed by roles/llm-inference (Phase monitoring). Three targets:
+      # node (system), gpu (nvidia_gpu_exporter), llama-server (inference metrics)
+      - job_name: node-astro-orbiter
+        scrape_interval: 30s
+        static_configs:
+          - targets:
+              - 10.1.71.130:9100
+            labels:
+              hostname: astro-orbiter
+
+      - job_name: gpu-astro-orbiter
+        scrape_interval: 15s
+        static_configs:
+          - targets:
+              - 10.1.71.130:9835
+            labels:
+              hostname: astro-orbiter
+              gpu: rtx3090
+
+      - job_name: llama-server-astro-orbiter
+        scrape_interval: 15s
+        metrics_path: /metrics
+        static_configs:
+          - targets:
+              - 10.1.71.130:8000
+            labels:
+              hostname: astro-orbiter
+              model: bartowski/gemma-2-27b-it-GGUF
+
 # ─── Grafana ──────────────────────────────────────────────────────────────────
 grafana:
  enabled: true
Author	SHA1	Message	Date
Hermes Agent service account	c3755aa29e	llm-inference-multimodel: role + day1 playbook (phase 0 discover approved)	2026-08-05 15:53:31 -05:00
Hermes Agent service account	782cbe33d1	llm-inference: size ctx-size/parallel for aux task offload Previous ctx-size=8192/parallel=4 gave 2048 tokens/slot, too small for context compression inputs (observed live rejection at 3826 tokens). Measured VRAM on astro-orbiter (RTX 3090 24GB): weights ~17GB resident, ~294KiB/token pool-wide for KV cache+buffers at prior sizing. New: ctx-size=16384, parallel=2 -> 8192 tokens/slot (matches model's native n_ctx_train max). Projected VRAM ~21.8GB, ~2.7GB headroom. Applied directly via ansible-playbook (Semaphore currently broken -- fix tracked separately).	2026-08-05 12:14:11 -05:00
Hermes Agent service account	aff792a061	feat(llm-inference): move astro-orbiter monitoring to GitOps (values.yaml + dashboards.yaml) - Prometheus scrape configs for node/gpu/llama-server exporters on astro-orbiter now declared in cluster/applications/monitoring/values.yaml (additionalScrapeConfigs), applied via ArgoCD sync instead of an imperative kubectl secret patch from the Ansible role. - Grafana dashboard for astro-orbiter LLM inference added as a ConfigMap in cluster/applications/monitoring/dashboards.yaml (grafana_dashboard=1 sidecar label), replacing the role's ad-hoc kubectl apply of a rendered Jinja template. - ansible/roles/llm-inference/tasks/monitoring.yml: removed the kubectl get/patch/apply tasks and orphaned grafana-llm-dashboard.json.j2 template; role now only stands up node_exporter + nvidia_gpu_exporter and verifies they're reachable — cluster-facing config lives in Git. - host_vars/vars.yml + inventory.yml: finalize astro-orbiter as the llama.cpp/RTX 3090 host (jarvis user, ssh key), drop stale Ollama/AMD vars and ollama_server inventory group superseded by the ATX rebuild.	2026-08-05 09:43:54 -05:00
Hermes Agent service account	aa8e229e64	fix(llm-inference): switch serve phase from vLLM+bitsandbytes to llama.cpp+GGUF bitsandbytes peak RAM ~54GB (bf16 load before quantize) — kills 40GB OptiPlex. llama.cpp Q4_K_M GGUF loads pre-quantized: peak RAM ~15.5GB, fits cleanly. Changes: - serve.yml: build llama.cpp with CUDA, download Q4_K_M GGUF from bartowski, disable vllm-serve, deploy llama-server.service - llama-server.service.j2: OpenAI-compatible server on same port 8000, --n-gpu-layers 99 (full GPU offload), --parallel 4, gemma chat template - defaults: llm_gguf_dir, llm_gguf_path, llm_gpu_layers, llm_parallel_slots - handlers: restart llama-server, vllm-serve failed_when=false (may not exist) GGUF: bartowski/gemma-2-27b-it-Q4_K_M.gguf (15.5GB, 24GB VRAM fits w/ ~8GB headroom)	2026-08-03 12:37:03 -05:00
Hermes Agent service account	22a020e4c7	fix(llm-inference): bitsandbytes int4 OOM — pending switch to llama.cpp+GGUF bitsandbytes quantizes on-the-fly: loads full bf16 weights (~54GB RAM peak) before compressing to int4. Kills the 40GB OptiPlex on torch.compile warmup. Fix in next commit: switch serve phase to llama.cpp + GGUF Q4_K_M. Pre-quantized weights load directly — peak RAM ~16GB, no compile overhead.	2026-08-03 12:35:46 -05:00
Hermes Agent service account	e879cf73d3	fix(llm-inference): gpu_exporter version 1.2.2 → 1.13.1 (correct release tag)	2026-08-03 11:57:42 -05:00
Hermes Agent service account	423891001c	feat(llm-inference): Phase 7 — Prometheus monitoring + Grafana dashboard - Phase 7 task file: monitoring.yml - node_exporter (port 9100) via apt, systemd managed - nvidia_gpu_exporter v1.2.2 (port 9835) — GPU util, VRAM, temp, power - Patches kube-prometheus additionalScrapeConfigs secret with 3 new jobs: node-astro-orbiter, gpu-astro-orbiter, vllm-astro-orbiter - Deploys Grafana dashboard ConfigMap via kubectl apply - Grafana dashboard (11 panels): - Row 1: GPU util %, VRAM used, GPU temp gauge - Row 2: GPU power draw, vLLM token throughput, request queue depth - Row 3: vLLM e2e latency p50/p95/p99, KV cache utilization % - Row 4: System CPU %, memory, root disk gauge - defaults/main.yml: llm_gpu_exporter_version, llm_gpu_exporter_port - handlers/main.yml: restart nvidia-gpu-exporter	2026-08-03 11:53:36 -05:00
Hermes Agent service account	dda6b91330	feat(llm-inference): Day 1 playbook for RTX 3090 vLLM stack on astro-orbiter - nvidia-driver-595-open (already installed 2026-08-03, idempotent) - Python venv + vLLM 0.26.0 (already installed, idempotent) - Gemma 2 27B model download via HuggingFace hub - systemd vllm-serve.service on port 8000 - Hermes provider integration on carousel-of-progress - vault_hf_token added to group_vars/all/vault - ansible.cfg: vault_password_file set to absolute path - inventory: astro_orbiter group added Run with: env -u ANSIBLE_VAULT_PASSWORD_FILE ansible-playbook -i inventory.yml playbooks/day1_deploy_llm_inference.yml	2026-08-03 11:51:34 -05:00