feat(jmri): headless JMRI server with Leviton layout power monitor and X11 GUI mode

- Stable udev device symlinks (/dev/jmri/nce, /dev/jmri/loconet, /dev/jmri/lcc)
- jmri-monitor: polls Leviton Decora Smart switch to start/stop JMRI automatically
  - Quiet hours 1-10 AM (no polling)
  - 30s off-delay before shutdown
- LCRR config cloned from Gitea (ssh://gitea.mk-labs.cloud:2221/rblundon/LCRR.git)
- ~/.jmri symlinked to LCRR repo for GitOps config management
- jmri-gui: X11 remote GUI access (PanelPro/DecoderPro) via ssh -X as jmri user
  - Stops daemon, launches GUI, restarts daemon on exit if layout still on
- jmri user gets login shell + SSH key for GUI sessions
- Full JRE installed (openjdk-21-jre) for AWT/X11 support
This commit is contained in:
Hermes Agent service account
2026-07-29 00:43:23 -05:00
parent a5433dcb5b
commit d974c75d7c
26 changed files with 2460 additions and 459 deletions

160
COUCHDB-ERLANGCOOKIE-FIX.md Normal file
View File

@@ -0,0 +1,160 @@
# CouchDB erlangCookie Fix - Implementation Guide
## Summary
**Problem**: CouchDB deployment fails because `erlangCookie` is missing from the ExternalSecret configuration.
**Decision**: Externalize `erlangCookie` to 1Password (pragmatic approach)
**Rationale**:
- ExternalSecret architecture requires ownership of the entire secret
- Mixing externalized and chart-generated fields in the same secret is not supported
- Single-node deployment makes erlangCookie rotation unnecessary
- This is an acceptable deviation from the pure Harbor pattern given the architectural constraints
## Implementation Steps
### 1. Generate erlangCookie Value
```bash
openssl rand -hex 20
```
Example output: `f4e3c2b1a9d8e7f6c5b4a3d2e1f0a9b8c7d6e5f4`
### 2. Add to 1Password
- **Vault**: `mk-labs`
- **Item**: `couchdb`
- **Field Name**: `erlang-cookie`
- **Field Type**: password (concealed)
- **Value**: `<paste generated value from step 1>`
### 3. Update ExternalSecret Configuration
File: `cluster/applications/couchdb/externalsecret.yaml`
```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: couchdb-credentials
namespace: couchdb
labels:
app.kubernetes.io/name: couchdb
app.kubernetes.io/part-of: mk-labs
spec:
refreshInterval: 1h
secretStoreRef:
kind: ClusterSecretStore
name: onepassword-connect
target:
name: couchdb-admin
creationPolicy: Owner
template:
engineVersion: v2
data:
adminUsername: "admin"
adminPassword: "{{ .adminPassword }}"
cookieAuthSecret: "{{ .cookieAuthSecret }}"
erlangCookie: "{{ .erlangCookie }}" # ← ADD THIS LINE
data:
- secretKey: adminPassword
remoteRef:
key: couchdb
property: admin-password
- secretKey: cookieAuthSecret
remoteRef:
key: couchdb
property: cookie-auth-secret
- secretKey: erlangCookie # ← ADD THIS BLOCK
remoteRef:
key: couchdb
property: erlang-cookie
```
### 4. Update values.yaml Documentation (Optional)
File: `cluster/applications/couchdb/values.yaml`
Update the comment block at line 9-10:
```yaml
# Admin credentials managed via ExternalSecret
# See externalsecret.yaml for 1Password integration
#
# NOTE: erlangCookie is externalized to 1Password for architectural
# simplicity (ExternalSecret ownership model). In a pure Harbor pattern,
# this would be chart-generated, but single-node deployment makes this
# acceptable. The erlangCookie is treated as an immutable infrastructure
# secret (generate once, never rotate).
createAdminSecret: false
extraSecretName: "couchdb-admin"
```
### 5. Commit and Push
```bash
cd ~/git/homelab
git add cluster/applications/couchdb/externalsecret.yaml
git add cluster/applications/couchdb/values.yaml # if modified
git commit -m "fix(couchdb): add erlangCookie to ExternalSecret from 1Password"
git push origin main
```
### 6. Verify Deployment
```bash
# Watch ExternalSecret sync
kubectl get externalsecret -n couchdb couchdb-credentials -w
# Wait for: SecretSynced
# Verify secret created with all four keys
kubectl get secret -n couchdb couchdb-admin -o yaml
# Should contain: adminUsername, adminPassword, cookieAuthSecret, erlangCookie
# Watch ArgoCD sync
kubectl get application -n argocd couchdb -w
# Wait for: Healthy/Synced
# Watch pod startup
kubectl get pods -n couchdb -w
# Wait for: Running
# Test CouchDB access
kubectl port-forward -n couchdb svc/couchdb-svc-couchdb 5984:5984 &
curl http://localhost:5984/
# Expected: {"couchdb":"Welcome","version":"3.5.1"}
```
## Why Not Follow Harbor Pattern Exactly?
**Harbor Pattern**: Only user-facing credentials externalized, internal secrets chart-generated.
**CouchDB Constraint**: ExternalSecret uses `creationPolicy: Owner`, which takes full ownership of the target secret. This prevents the Helm chart from adding auto-generated fields to the same secret.
**Options Considered**:
1.**Externalize erlangCookie** (SELECTED) - Works with current architecture
2. ❌ Chart auto-generation - Conflicts with ExternalSecret ownership
3. ❌ Dual-secret approach - Requires Helm chart customization
4. ❌ Disable ExternalSecret - Loses 1Password integration for admin password
**Decision**: Pragmatic approach wins. erlangCookie is treated as an infrastructure secret (generate once, never rotate), which is acceptable for a single-node deployment.
## Secret Classification
| Secret | Type | 1Password? | Rationale |
|------------------|---------------|------------|------------------------------------|
| adminUsername | User-facing | No* | Static value, hardcoded in template |
| adminPassword | User-facing | ✅ YES | User login credential |
| cookieAuthSecret | Gray area | ✅ YES | Session security, periodic rotation |
| erlangCookie | Internal | ✅ YES** | Architectural constraint |
\* Hardcoded in ExternalSecret template (not fetched from 1Password)
\*\* Pragmatic deviation from Harbor pattern due to ExternalSecret architecture
## References
- Full analysis: `/home/hermes/couchdb-erlangcookie-analysis.txt`
- Harbor pattern: `/home/hermes/harbor-simplification-complete.txt`
- CouchDB Helm chart: `apache/couchdb` v4.6.3

View File

@@ -39,3 +39,7 @@ step_ca_principal_mappings:
- ryan.blundon@protonmail.com
- ryan.blundon
- ryanblundon
# Leviton My Leviton API
leviton_email: "{{ vault_leviton_email }}"
leviton_password: "{{ vault_leviton_password }}"

View File

@@ -1,436 +1,441 @@
$ANSIBLE_VAULT;1.1;AES256
65653162653536373535643935613236366335356333336264346637323164633739633965656664
3434656433323239366665356134386564643135663135360a653232636463656332343565313763
61326635636333393730356131356466363063613734653565643766383431363166656532393934
3631373264353563650a616362633165666239323563623161656238633737316236303133313034
66353461643535373638616364663365643961656635643566613336633466343739386234313532
38353066313763643266623739366261646633363962336239623830613265383335383763376534
62653865626662383330656530336534366136313965373764353133326139623132646364396337
33333366366434393037373361376538376334336265663162336239646237306332623039343032
37333832383762663534643638666234623062373734626335323537616334303638376164656634
64623962313064363166383930636136383935613237613434373733383836666532363265643739
37376333363736326334383864633734393263623536393239393564643131616132373230356633
61636165333537356632303034346431353139383065393832323236626331353236666263376666
38306631653862343064656135396638636363613139373062653735366565376165353330636162
38653530366263356437323433653437373362356166633235386330623333326264333961643434
30323130613431353064393138303465316630383462653062326264393035393735396134396665
36383963633938363662633035626538663366323461656136323934353731626264396364643337
63356337343161366237353135313966616630316362396362343962356564643365626664633165
62383765303436303239376365316632353463666664306164313933346162343833303963313835
62353565333231646462383664356230623165363333653335343438346233316534333132366634
31326138663433663132393761343335656636366237656337666331633062303363633938303536
31303238323932336330363064323963306132633961313634616232636163353439363666623230
65666632653730303134343032616331363932616532313365633038623135396463316666373637
34343736383731363239303330306561356162623962396564323239376661363761336434376264
34373037623331643666366165306161356365326261366566323233656538363061636637343164
39323130613530386661343839623361303862323763373864626536376562643766643263376665
64373334333436653061653962373437366437643034306633343830626230346532653565353135
33326339656266616237643362333062346564333563326463393535356465666166626463643132
36316263323761653333363632386261633432383431386435363362663734626634353761663035
32383464396632396361623531303937383366633131613861646364303662343761323530303139
39333866663164363431656233663064383633386261313063646435333666376465626532343636
62386166306635353061623365313738306237616430386461343262613039346261353662346462
65616261656534336631646434316336646337333661336530633834633037363834346539646466
33346466643238343133326339393264613763626532633938613765373262393236643533383661
33306233623130666161326365366164333434373463323662316432633263323761316338623065
66306332656438303938393430633563303761303239346266386664333837613336316539643736
31353234613664313061306433656165373737336166343930653837303532346163613564376661
61376237383562353639393565303332633366376632386532333736303738333362353334353366
36343166633533303834346265666536353633343564623062613030323934653863343062323136
34323635363238623336653062666636646263343731316537346635346235613862623530326633
30396338386233663335613661646132386230356631326331653962393336653937376264313931
65363465343265643034363963656262666131393632653561626330396166386566396363663538
66323935383166623038326437323730363266316530313938373261366363613731656638386634
31316261616630306131633936326635333632643731313730303438656563343133373732376537
38623330633539653034343364656261316336353466653663646333653636653266373363333732
61333461346131366233336162363033623636386634613731303032393738333537393131316539
65643238613432306436376432373364656530323331633762393566396531396466653463353832
34393662616435343137373364663839613464346564663631343837366530313061326561656664
33623163333864346661393634333830336133383337373833346132353531656236363661323338
64313663356432373137343461333539386665643130316463336134396436623730666165623939
33373832353932386238353866336130356530613766626235316332623132343735636335633437
37303766386434373930383663386565383732636635646133393637383963353134633433616431
33616464633431666361393931623631313731343863636234646130313136346639323662373236
62623830633838326432326339383533373430346635643739646639633739343639636665376333
30396238356161373734633562643466643036363239396538613465346238356366363730623965
32336533303331363336666238363863313066376230353932323466316432653264346334373931
61303066313831376233313261633336643265396139663037333031636666383738313763313366
38386138363738353265613363353339323930343934393036343234626162353037326638636236
66343634323136663134373737366433616632653737316534633232323237343566623361666131
34393331373835613934316235336635666633646465373866356566656261373066653937316339
37396366643564333130383237613933613736376161616231666139643030613338333133356336
37343362363433653638373335363130313362306236393231646637383736663236376537303533
31633230333933666131613764336532373633353132373839326562356438386236643537386231
35616662643431383661343937613738393963373865383465306532336137373730653832633930
37656661326434633730346364326661653937313561333437383064313933363231393164616464
31333664386134376264376339303838656431633966663263303465376161633866376161313531
62346266316533326639303661396164376666383864363262316630636561326366373363633063
33666263333132323232346235633837363138663539366131303633313662326664373332353664
39306235313732376164306164386534346666633037316334366431363063663137356464656234
35656164303933306661373932663831346531636530386139373837633263396632643235383361
65393939303338656537353130316361366139363933363031393735666336323931373632303663
63333137663265353962653861316365643239396562333933383964663730663230353331386638
62396435323162663036303334323936633436626136663537376562363430376239356136343261
36323062363935643562326161643065333437343331613135383932333835396565646631646437
39633035326662653334616366323736323032616264623166326563326361363263623736623739
62633762346135626666636463343132303433346330303631656634646537616537363764626564
30316539303964626635613338623261613430656432376265386636613566313732386561653564
66363538643463653233373433353965623765303038333931623036666435303330633136663966
31313964323439376637613530343635376166333765306466353337623633343539313330373437
64343930636465636265383261633664313762303533326234626461623563636466643039343336
62346462343639323862353036326335383735323337396465613737336661633465363330363161
38323561613935303334303261323032616332333231363762303831373961323033626663383637
35323134666636353766313330316231323938326532333030383638646230643735653663346634
38363261626663393139313031633032343038396433353563363531306232386539303734323463
63313938373735383631373665613333303530383335363262306230326665666535333564346163
39336132643464666538356461393330616264383632623037623634336530376164336336323336
61646165333835333961323439663864386562326131653564623861386336303934346263643036
32666632623537376537646634313839363038666336386238343531323664396461633030373334
35396463373339633931383636353062396562383763373939633336346463623733303039663733
31663834326661623064323132353431366362636466623332363532316435386333636562633562
36613465363936623462316136643664336261353938653362393938666266663330323163653338
63333732616330323438633038366465353865353965663333376466613834633166303761633361
37613863313230653235653034613137303231343961616330316664653333303436356561353562
37623839646536363036613964336230373537653130366330633137356661366561386566323165
38653633373538633762643935346661363961663265616433383832666635666331366635636637
32336532373164373135363533333539623739383330646431356363636430326433376364393265
35643561613033623231366362393737663964396266313232666362326436303331653764366436
30363135663234356363613065643763353836626133656334653138393562393032623631343130
30303936656636366465633163323061333863393532366563316233353466613566316563336539
33326137383263393266393166336361656538353963313366353434333864623238333435653836
39376138356461663262356665363835653638623031396539353132623737333736353937623533
65386333396666633365313262323739633461383464386663313163646632646335646433663834
34616663386661363135653765386534633339653232326133396332646136653230393431626637
30366433336633643837363434383439343562366164353235653833306138643431316537343931
34396535393432383163653231623837633961366437653434623164333333323563666330613866
36393561656135303731666638393737653565303836326264373137303264656334653163366433
35376362326133666336303732656561326164663832623732613830393063343231323363353366
66636233366436363565336231663964303532343962333732313832323762343638356231343461
65373062383564346266323362336666613735316161366136303032313039373464383532343932
30396530373030303933353264363330353132376164303062623538613632323932373230363838
32633766316533316665393033326161373361333166643436356430623262663032646262616633
32643436646264343366343862356339626561346561353463616534306139383133656433636331
32333735346163353434616530623235373037376664326266343933326439663965653865333766
31653163643134346331313331633561643336663130353936323439636331383134646665656630
63663866663938663632646664323936613434333731306264306633643335393566343564346537
35326132303632373731373532623662373861356534343536613731366666373439343563663737
64326339313533323936393362613564333065623731653930363264373061636665313135623062
63373565306535373763356664333465343666626439626666393734303962303762393066643139
62663334313736663132386232613061326636313664326462353361633362323464653038396662
64363862356439633564313065363361313161643962316166643936346539626339396230333633
39333132393135613264653363323233643032376266313630313735373966373966386239336635
34643739363935353463373434343430633438346136306337393061646139346230636531376537
36633861303264633662373365623261303635363836333163623864323338613739383134636237
62326561356535626564633038643230646431613866643239646563323035353434313837663863
66326530353265626361643738333266346339326532663166656432613631313137396463643864
64343339373131363066303437633436306462316166356131366134626331343635323837643136
39306461316230663134346563313662616532616239333735303061373138383363386232373062
61643838366266613637336433323062336434313836376434636433306135303734376661376361
38373438623331653165643837303733393436626166623762623531316164666163316661643761
39616539653831336261333335643834663533363233333732643431363836386633663539386638
39356336653438663061306230316464616533363036363434663335383530326432376632396338
38386130386532333861353362313838383964663861663962353039613565636339346162383331
32313337353430343130326466623666326263343438663463376134336131323735663261356263
34613638303262366666643036353766653039393739303738366330643033316632616232393032
36336161646434373332646663313834623064336437316666623663613861333838363731336564
63636633623664353432323836306564616239343731316335333234336465363232326430643231
30623063316130396531376237363363386465393838303463336439616333353338366436323634
66626463326136343533343263303033333766306632303333376136643137353831323135666430
61313663336363356531633866663366333433343433643664663665643335633035303034666334
62653233616136656462343930636534636432353237363538356365636635373865353532643764
32646431376165616235396133323162373330386664636336336266363961316462613263303131
33303635306136303366353331653239306130616464303835363437313566363436663032353061
32356639333061636536383238396665393232663562363662366563653831343037626535666432
33353261623936313563323564643134666232333839386162663331326461393239623135383530
31323861666631663933636338323239386336356339623738646430333136336635643562353263
64313931656435383134353063383665646538306661396163653434373031326134336235646461
39343734313965313365643135643166366133313465323366633038663333366535633532623966
36666134316165613830323933333339336663363964323764313430613130653236336664393863
64306437316235306565333238666164353738336539353064343161343834656262653666636663
36353063326262653466303534616530616139636166343137666439336335363539393439633436
36623963623839623834656431396162396562373330316438383739363637613366613163336138
35373730383165316330366332373939643839336634613934653538303866343738656534653263
33303361343237356430373266653062313037663230366531366363393937653439363732333036
31343365633862623038316536363031356336383461396562626236313038313239623665316439
30336137356438343362363536303234346637373833313231663333616263303233336161383130
34316265613736383830363639366139313537313661633736303634326237643330663735356561
65626136613831376432383363383731663763356432663264346561646535633966643462326232
31623930303262616535356633663938323635643462663731366433623961623435303062616263
63353836333235336362353039343431313262646137303364613838306339623930666466636132
32656434623738333565636164303535393734346462353436313030396264623332393932653861
36383533363030356664326332376662616139343065343932656262623334633334633462383635
63666439343130623936376636633635613335363264363136643366636535613938373866356336
35666632313565633630373039316262316339333035366332356138393261613836633437363136
33653033343536623264366466356561323063313031633464353963383661316131643166663566
36373937326333393164353234336433656336626134363236303032396531353862333535336466
65393865333565386538386665373166383235633739383934643334653763366161396565616166
61333666353666393230326134636661376237353937646236343034626135366535383631333035
61346231386233326564656636363066396636393966353539666231646464323636303038656562
31656238373331303031393034346630643263613365643232663861373336363662356265326164
61323866663033636534313365376466303538666463663539623231313632303262326638633539
62313239356365313932653735363662666266643765633138393037653662656566656130663133
61373531316261646234666361313766626139613762306563343561363961656565666163313532
35323837333737366138316164376537613834313135306461633735613362376233306134373537
65323339393039336230373237386434333639623762643133363339303265623065326438623531
65356632363237613937633132326231333938336639663438653861353761396434643136643238
62363162666366653831626531353236623532656437313134343633626666383437363761626331
35386431646564653233613632373435623665613335316164636562356264663033366639636439
63356231393362663736366431616161633065303537366538393935363033303636653836346132
30633036653836656433636631313863303132666531326563303266316566623934666361653932
35623265613635343435373361353635343261656233356535303037383433353731313539613463
64363334383666663639623735393934646366386437633638613034316366336337316561333731
38363339386561373834333037626565623335613563323366656365643032373834616339663838
64646534623432323363303032323666643636323536666239616238323932353165663535633465
64633934303565396533313165393438373231376664616237373562306130363531393833353365
63623033343762326437306536663734386365653131346235303837336236366665346234316163
32616539666634653839633739303837646563313364333864343231306663383530313963386434
34313965393731653963643336336134343764343065646436303739356530333761386465333333
35653363653361366462636162663134333431323866306235343764333035373534656138643933
30653962613339313833343532613939636332633236303733313135313932356136383439326635
31386565343337333662663762356361623835353830333865613437336532623936343365343231
33336137616266303835366662643737666333346433366663666462396531643463373562623237
38383537666639363839653538303266303965613762373561643433376664313966326561653764
66396566366265343962393633653765643563666634646637623161323538623961393039333036
33363530653935386331396533666432626634616463663763396663353133393461623866383061
64366132646132663263636131333031643736663031663336623065386266653638376639336334
36653932643831303931613532333036323766376235303433633564303062623336646166303533
63353063303033623032643333653733393332303466366538353062373666363438366432316131
36613335396531663238343139313061613363393463363035366132353439353433646237646265
37633964316466636337363331383565323866653036386435376530343661323565666539663230
38386232376436623064613232636637313931316535643661306536343661653135313335616534
34633761663066316430313361393063623033373636633133653536383764306433346235343761
37393764633561653632653565383937383436666662336633633261653861323533303732613738
64653739323636613639643465316437356163356238656139663034313738363532373861393333
64613038386535396565656538663163313161353831306237313064666566336534646233393137
37336265316361613430663961356361653636636235323439306537666363353361643664363631
63643631336134323032636463393639613636643130313136326538376461363937643436633561
39663433623663346462356437666230313937396132633834386661613436653839616136343234
35333437386663353863306166393634363734323636333137343938353461386366326239396236
65303939623437376535633463626166653935343033303066383064306332356230623566393062
34666566353238653730613561383434343332333732356334623836616334393464373965633334
37313564393463663862386535626338336235613132376530666635343832386231366533363562
32626162626434366261623431373938646237643462623337653466653538653732393634366639
62666231376239393066366336386166646631396632613631313663633435646337313465643231
63356566333764333238326165613832643161346366396635663032623133386132666333343337
63323637383035363833653162326231616461653062613765363165313638313234343266333334
62383362343232363431613364383930376338303839633261656236636264353335333938333731
63616666626437643432393438376661616262643939363239353836323036303139666338336261
31306437353536623561333531346630643761336235373230373135326361393739613934653361
38346532313364633638306665306563646433353038346238313530363163396166353535343537
34343330373063383566333335386162326339383835373665363464616139383331646436343365
30386564323733393461653662323334656637363566343737356239306462303762383332656239
30643439343161636434656338636533396364643632383030646639356534343963666466613566
39373833633265623563396363653830393661633636646265643362633731626462636531313362
35366331383635643564316461633431646565393863373635323834363934336335653831613330
39646262633636343832626362393730643163616565363765346266316636343065393630333866
62333931336161636230326132393739616665646462653666346532356265643235333131343635
31326638626162356333356261623531646464633139666539373261376230336661313937363036
32313464316364623838343231346538303237393137363233366234373031346238303932383461
66613537346136313262313337636565383639383863346535393361363930616535626264376565
30303236663065363161663836356130363130363339643335653366623065373566353136353933
39363761636130646537393166623335353431616462343364383535393661313738343265313138
38323066333239616137623233323664616232323262666530373064366138316431643430643462
32653262386565396335646431633534663031613536656561643434366464353335313239393464
36333866343063656533623432306231343065343530343638653739343731336131626532376566
37653833313233386538666464386232353466303035316232646339653533663831323261636330
32646166366633656631393762626236356633393130386165643264663437666433336138353938
37653866646364333134313366316366636132623764366562623932383239643265646461646434
63656233316639616134666635373434323030643136626238646637386634333439616231336136
38326431363062326232626238613334356633353362653138643237646339333762616330383934
63613235376530373834333834316539343330313862666262633439333062633261646138626434
33616162316632366537393939343163333530393939613936323164663034326536666133303365
63643337343037363363663235316330316131353064383933323131643036623862303835336633
39376134313264333834306330366136643434386566393466313835396466306363663565303065
33393431636163376365396139346162346239343838616561373162643931623365336436626636
39613030356162363931316166376231303033306366616162366239396333323838363465353261
64303561386563383834386166323739626237376164326533616533343338316430366365333966
31343233666365663830323030336161623466633261303637373537336333663262383238393833
33383033343630663237653338636436346565356238313434346666336436653835326133313038
65353530333839373136326132326439306431333334633933663065316531616263633533346431
62383862653162613532383136363830313964366161306166343934363865316533643736373431
61323135343030346336356232326138376564343131636263336332333239303733323338663830
32373639343063353261623831623237663133646535636338353333653237653539656438666436
63636537336636313762336531346531313637653834386535313666306636303338306465303764
63366133396263636164386339343131653361363831363932303734333634343734636163383539
33613430353236616132636431656636633561316161623661373663396462633930343865336236
32373534666135303730303663333337643033346231646661393137643161383833626335336561
66366433303436313132636230313531316261613564323963626438623530633634363265613939
37666636313632353831343238646237666464333039306630343531626339323562346162663130
65633133366564306230666436663630316434616634656661306461353866396662376331666630
30323166386535633361316633303830623031393531343532633930616439326465613631613838
38326630366534363036666433333237363636643366333161343336363936323730313339343132
35356231666633386537366236303136663261336532386338316565343239333261613838623861
31366531343232653431343436343766386331303133363534616536633337656433326265656161
66356235616135363634313835666134346232356663326361396537303339373334323861383161
33393438313061613465666536666636643166613463363165393338643066613231636138343537
64333731376139353934316631393561356163366237386337393061343235633131613962383731
66376261633832323864356564326532356363343762313563393331393933316336373434333237
35396531303330396363333163373361643837383766333730303031613665313237383532666264
65613237623136336239653566646366376538356362643062386436363466616331336665616432
31383365363630663530643764633335336332386332663661393733343038626265383432306630
38653861356636313634326266633637653764396233313463393964653739306234666662336432
33393139343864346233653763343762616230383131353166396265653031326232656333626264
63616339663564363665626338316661623230646534373231313662343736316462663763333364
30333137386161336664386536366538313934313039333832323763313264373861666239643466
33383334633736333634306262316265323634386635313764376532646364356565396538656137
63316232643832383865303934636364313036666237636632393137373933656263646534623230
65616231323035643531353430326630353761633234636234343834656664323161383335343235
62616533366465303961306134386265303933616236346332623536393236633366333634316264
31396361343833623131306266336238343866303361303336346566656639616134383063396235
31626633333665373934333961646263663730653331326137383031383736646633356536333431
39303632383030613465656236333763393737353865386235333830613665653363393536646630
65353733613536653963393831636530636231366266343262376631393062623163313031623230
62326561653165306339633439376461346330393231313366666339346536313765363163616630
35373564366563656161376336323031346339313734633139383563393966373637616266666265
34623330313265373935396130643231353131313163393333396337396666636439633035326635
32663762333537663839346531663132613636366639643364373333343262326264366136363331
61613833643639656632353931663830613634383334653036323462323262386265313637656535
36333834306438633532616335663562383730306337656336663035643136386261326130336363
32623330333764616565333162383234656463363432313039666265653563323232613930303765
30623834666665636332303862663832303537656664643362643636616265613739356664646462
61326466303266626166333730343330396439356663376362666536623539353666636230373533
35363461326263366137313037323166396133646430616664343165623533393963623535626237
39633362393636373131626364636437343361303435396138363735326235383237633964336138
31643935646431386537643733353862666138333238323461623638356535316365653530376464
33343465653134613536343563376564663133336532343330666131346663363434623238303862
31306331633238623836346466616230356532316232323734356638343532643032633937653061
37393265333233626563353963363637653338393964363832663734303566356633623733393164
64383536343862393134336638353733373262396563303266346535346266306238616531336162
35363638333431396163303530353461363338656363336366376430316464353131623661346366
35313930353538343830333337356163613765356361343663613933366464353935336361636436
65356465346535613231363932303339646162303238376236323461303062313336656366306563
31353464333539343839353764393665393235653537313939373563303436626635663766303336
35333838323734386537623334366235653534623664396664656264633735353634663332353364
65306132346464363966313963346633353538633936363164393566376630376365316139376132
38323039653465346462666439656134393964623563393664643634313638323832356164323863
31366535336238663236666162336166313933376531643137343665653765623961333464633332
30306265306364643738386561303833326363303836333032393462643764663664613536363835
30663937333038663435326637663636383165363632326337653932336130333932333861366235
66313566613033383631376132333634653639336666386164613934353230613239383239313038
30653162643233636638393036353032353632316166333938313966626133643237376461313065
34323762376133393535613864636461616261343031393266666165316236323231323534376465
63386363343933316139356163363132343666366132313038643938653865663934383335336364
31646563346339323633643366653861323030636138663861636434306238623738636331393538
66333765626132346533313261646434633236633432333430613266333161643566326434363633
62373033356337393232353566666263353539326564333766303335666135343461386530626562
63633137623361366362616432613237353231616465656232373835376463386132663331633737
63306136326264643365356634336233303163663135366531643362303666653630363962666637
61626539613333306263323564373435613961633439633261373530363137376132326462633035
30353838336433313135623530643663643232653364316263356164633661383937393238363262
34393666663361613735623435626238646266376533333963653630663462653733373130393338
32626361366632633630323363376238343534626230373835353036333065396435653964326534
33356466643839626162316438356437623732656664613434613165643666656230656434633638
61316134333631326365313532313335633634356466663834666531653365333333616137643835
30666664353134313236653337366466623966366131313962346564663432633535363938333261
37353464383832373366613531666161323632616236613631653433643562326134623838393135
62323533373336663434386166333862366639626562343830396533646138636261326161393339
33613961353431653537373931396531623061653163376230663338376636613136353433646231
33633236306238336432323134393463653161303662336266313562316337343430373532663930
39613133646432343238393762386266303531663032306130653532376262353238636231393539
36376332633563633661353835386364633461363437333038363865356237643739323133653235
66373931336330626663383136363265363133393239393131623465383433363336333237653131
64656662316665356665303765343932363733666362353031656234373363663364666537663030
34646233386630633438313166396663383732366233373139343431373463346133316663643036
31633536663266383132393032313563653035303034336161356139396636353365636163383031
30666637363933643262393065633463313836653530663232303736306537636533343431333966
32386330356564613932306466353931393839316562353362303765663263383738363765333136
36383036376130306133393166333734316231376165393832383735623838353466353664353834
39326136656239626434313930306435333533623533643236393437313038393235386364323766
62323537666333613066356236313361376138656232343430363438346261646165623565313435
65326532623761343239666664636637613034623736386665303634613737613964333630613533
31653731353533613866663166653237663162623236353838646465373734613466616362363833
39346335373539376439643937363233373362616334323231623232353061343830383336666466
31303663643061653866326632333336626462623835366564356231316263326130393138306330
66333532336164633931306136373531666131356130313262313038626138653864633734636132
61353638333133383035333937626333376564316465333539653138323739356265613664653536
35656136343036363532613335336230373364336432343731326132666662636664356364303335
66613736633231393534626539333536626565663231396536363062353037346563643934313236
32363138386430633761383065343834376631643539626330313638333534333734333239383632
39383833656134393165666533353739303834633330653936643637316639663033363865653631
36313735613962393264623938336432336538393937316634626165383934356562353035316533
33353735653133636439393763356430303863326235623932623464356636666265653065383736
31613666303065653035303131646364386636393035663865336164376536323462666239356333
33373562383863323635613634643165336465613734613034366236633835363733306662613462
30326565346365633131363331643165346466366666633231643538303430376530393139393063
32666664313162303065616261326261353130623564343761633861373034646161643163373533
36386665626530313030343032376531656333356332666437383530613834356336386465353539
66623062646236333465316137316564363938303966613561643830353332333537373736626438
64653239613063646430386631313435306462303566333734663462626530356530663235303166
31633366613264316137303334613366366537646261303962646364643238383062643732386436
39643430613562663136633130666563396538306633633034323064386136303034306531323564
63646633303035353831653438366635646562613639313230396462373161363030386264623033
62343031616135346238303337313931643461386363396663333231626661613938626366653837
33303265656561313332353463363534666232363561353532643532386133306332333930373161
66626135303036633566303961646461383236663737643265346566393963343634623164633966
61313031333863383237313633663633333866366230313936396334383361393338656465346437
65613763643061386463386431386539656635393435633032343030633738373936613833623938
39316239363835633339343161623237656230633763313031643663623466323764663366376165
36366530363535376636383561343161643037626639323830396665303238663534633531613735
39316639633730376533643932633432353835353439646666653766666338646662643162373263
65356563303235633963633232393736636537346637376462626637303535383063373134613732
32663763663364633463633738343535316436303365343665356133313836616164343434613133
34626434643531343461346332636539636463313539393830613434333933643632313737356564
39386137383165616139626363613732626336366461663339346134656530396464323533313265
32346535356466383135636638616166313663613565353765346264376535653135356638663538
30623834393935636562656639643737373462643137363063663737306361336339653334633665
65316237633436303761393766393234326538653633363936303534646566373338623935666538
38313761643437313565663762613838636163633066313630353631353934396338353665653330
35393661356630313436373761383462396434643535316364383338343433613061353637333563
38376562373462316334313632343965326235323231653030666666616163346438626437386637
65656163653935656530366239663237633937336166613132356333623964666630386331383331
32303166386431643739303363646633376331623166313135653265656636663236363936336462
63303135613539663233366362323565396137623264353431373836356233666332633731666364
62376565633037373832393366306431613863353137306535393664313866616235633638306365
32633761386462386339343066316263663438623330373733333634363736623637636661636331
38353835333962633537633864616132643563616437336334326633323636393833363666353261
36363732383565623233343439666430303961306263363032383238653265323637313430613133
63633338323536386139613233343636326564356564353065386664646230666561336338366436
35353162633864633764373535366634303738346236633362626165393632383762343036313930
62363864626338393736373938343264356235393763653432306330363363313762333137303362
35393061356165666230343135663438303834363533323064626532663662323063386333623166
39353037363363373166316238393565353266353665666465333134303234373263663135346364
36646463343032326339663130393963663861323130356365396365643265313833353234323330
34336661363733613362343739366636336265356464613033343132353031663965656634613831
33656466376132656465383664363231653235623036346634666166623532323635313130303836
36393064356637623139653333633164666332303539623863383538386262363064356130353337
62393063633637383965346139626265666463356362633163363537303166336264346332636363
39613735626132366137356365303331633037623132623637383164636565653963626632626338
65383663633831613666396534333061643966313366303937333261303135353762656331336638
35643539326161646433633862623762626539396137303863616139383832613639653033653933
39373734393066613132323739383036313830346239306132656464646462626661633931623932
36626163353563633764323466303966636161646532316134383034393733653363363933353738
34383563666166323865656430346661633663363635623566336632353633303838303436616266
34326131623331616533306465373365396437303764383231356631313335393364643664663864
33313033303463376139376466643434383461643264316534306166303163373661643962343030
36653665353232363630363437376266306531633934626230653338383664343638313334306636
38666162623761366237383236346564333333633162623832656462616662313031316166383465
31663362336663353564636335393738333565356432306139363661656663313234396664623832
38393630666338663031323464383535303539643931393732616666366661316163626339643437
34316532613063353264303462366534613035653834356562646637633137643839653237353937
34326230396137383531393962346137643035333834376537363865643366623932303662303839
64373431306464363762356563663038343737356165313832613965653065326532356133646137
30326232643334656363343533383163643130343836366430333836396463386666623465366336
61343236303336623863303630336437613932353832623866633661653566623431653938306238
37326630616131646634383539353234323766363633333030613937616632333432366565343537
35613237343838333430656336313232653530373863663031386433356337643334303363666532
65633033333463366636616138646632636534333963643864383033343463666338373630313232
63326365666666663262383664353664623963343366626364363965626435666363306237346130
30306661386334653137336464393465646135353436336138616563366163366664346331646666
61613334323330663835366163623836363534666531353737656464663935333765326235306566
61353962363034663563386137373432656166663661653861396361653930653539363333613435
64373731616330396130363464653865303931333637393333386531303365646130646161366263
61643234333563386338643331303164323638346639353765656632386262366666376665356164
34393464656634303130323738353161643537613337383661343232363961613931613234643361
34663938626364636562613332373161323932386532356261353137393533633731653034633966
62316666613738383665303433376438303266646264353133303566636436373966343662363561
63333662613930323666656166373763303961333332363231396532376564393966333135373966
65623664303537376362383861663238663463396537353866666333656664323562373165646633
35396661353639356363613834653432346539633135663565376234383866343433383339666362
63373330336562323138343934646466373738333039346361623836653231633566316435383636
61326535393339616639366563383662636136353162393961303362393866363436663630303762
65343632396238623137333462633633653761336635663265326332623631393566323530623562
36666431396532303939316662663138356631336434373732393463306336303435626232316638
36363733383831363930666132396661633733616464393264343339333166633739346236383833
33373962313762323430343161616361636363366334613032323637316261656333633639333066
34653635393031323262353037396130383964363037633463653331653965363562326532623037
383331306234353238633435663136623634
61346566636664323837633233623331666232363731633634366538333962363966366433636232
3363303535353934306538656332633436303638373933610a326364636563386437386466653335
61326564366131383062363037663965343330663431396331343831326133393233366439646538
3435366135636162300a326161313661633134393666623964323761343139323630353938313864
38393339343231636566386534363831616237343531373835326662376162373264633038353330
33663539646165656261326663306332313932333761383937633265373330346134333532616138
36623861356264643963366138646233343436643066396430653663323039333165306536383565
66346634656131323066353837313836356162346330376232363835653331323736383839356533
36636536313630623730333631383166383032363633643263613335633938616235656166356164
37376563633634666465663332633635653736393135353538346366386630393765316364616463
64343563646233623330633765326537383739323835306566613730323630376339346139613163
39623132633830393635313537616661326536643662323237353337316234663931626261306639
32376434363964363437373165613062333966653331373664383633656130313764626639353164
61363261636362326261383430666235316538383933643365663830336665633164613435303065
65313761623032373332623932623765616263633433663466366633363333356330393161346536
32373966366330363565663632306130656161323234303666313636353166353064636266386239
38363935313564323930396531336566313066386630626431346633323331633436326237633862
33363334346264343537333065336536343264336538643862386336666635383766333235323034
65313764343637343361323064333864343464356336376333613361356530343765626166366337
30326266316135353263323131326664326635616231303831353036333439383633376161383662
34613337376237393537343932326330323564313931376261393163653464316232373238393535
38376637666533363166646336376665363538643363666537393232643762383130366165373037
39636531336664633539653162393536306531313637663039636431353365396466363236376432
66633064316161623739623138386664623165363762663637353435373863656661393362353330
62333163366438343364626432643232396133643065636133626339323935623061323761643331
38393466353833643432643265623431323436346638626661323036303334303239383838396335
33353031373435306266303466366264363737326562326666363066646362623335306465363666
34343862336264626134336338626330396130393833383434363834336133393032613237393031
62633133663433653662373636356231396630626463633432326665326461396530613162373366
34363462353865336166313639663438653839376531623533376335323863383632376433343432
63316239333566356362306133323332353937366237616237326566323363646261376434373939
65623031396139396164653939393162393930326530656135363564363565643133616261666436
39313366383334633036373832323462623161666563303336383131363163373233616435663065
33316266626162363063366539323462346464393932393165643161623964373539393964636139
34343538663533343931613439663436383531393939333239613234386465343430303833353133
64636338343032373230346562363338303166616166666438646262393333313633656162303764
61386164353835383233343535636132643831623062393539336233626235303032663464386266
66366664346364623533613832373631313336336530386162363861303333306563353439333236
65316161383538373733633137633065633362363936373264333034646135366434656534383030
35303838326361313163636432643638373832396165626532623261366332613562383261346661
32386164366264356536653439353763346539643934393466643166333761366436613165623064
38623530316264333939383932313133613730383632663830366365343937313036373439663930
66613034633532303065333935623264636534393638386433346336353765316264346662313337
34613732356265363839343434633563303665313030613535343336363432323366623234623738
38396439376335653731346464363533313161343162653930656137626337623961336232623165
37663863313532626638303937326237353235393866303063313532343164303165636262646434
33316363343265336462626630376665336162336661616436656135366137633834326465303964
62643435393537613836656334616236333131653133653065636361343261643032646163633462
38666465633531383235613732366131366637336638393639346463663830626539653538636165
34633238396233646337383936633438393039306461633631363161656465336238333434633030
37346334396538386437323337393562353463336130363837393033353834386631396164386466
39343338383634663338373536656363623234313339343363636432303937316462376139333138
36386331616639376331393963356138666463306666306436643237303764346330393332626465
62666235656334323231626635333338363736663063656664326139346363323039393565363966
36326662353631646135623832333266366139313662316333376237326433626434336531366666
63323062393030656639353634336339343034316237373932376466653930616465346436383664
31343339663333386533323261393164363039386666323766343337363033366239386266393362
35653333666432376337653332393930383561626431376364326439633331353033613361376333
36633965616237333662613931366533366636393432393863323330383431326635386466646565
31633136373466663161643965396434333437626636626237383630663730663531636139666166
63613964646133666361303430643766333063643732393062333631616132323762623735613566
62613362366536386565636538656138366532643932373163353437636562353737343566613330
34366436323063356433303863383961303130666637366235353264356433313066616262313532
37653165356230313165323330666339376337656332366531383566353536633831363430353764
61383533636263393336313631393766353638353862363666323163356631396234326262626634
64633438333438633235666361373333316265326237393134346539343361353461666664306239
30643939633462303463626237653135333863336530326438313233343232323634633935653134
65626337313266356462323039393564356638353263643566313264616239353036323063623333
32636561373166653739396635633039623464656536653531356365393362343132343631343433
30623438613137333063653061376434336233666633366530333634346136663831643866386132
66383865633531356533316437383933656439353262626138623231613532643834383238323438
63616238653338656536316662306565326233646136366634666330666263356235636231393133
62373665333336336166363936656636373865666132646465656563653464633338623831616436
38663730353764386164653166666136336665383439313135346139373239636137333035323862
62366231323434363864656532323462633061663765616365366335376533336661656536353537
37383466356438343539363935653534336332386130306235353565383933316137313233383832
30663663346339396236633733633464393631376436353833663462316533646534383134356165
33616139663236653661383063323865633339343836643134396364356230323135363731626236
65333338656430376466616137303561356433393733373961313862663430656431343363363635
63663561396438646536346237653562663661353438363237616539393335386338363132623937
35613838616134633334623939333466646131363663653961646138613064366361646235346130
64353130656532333664376635633137333434343965633361333830366232356434346263646232
33336635323062653161346138373636613638623431383538343436323737316335306334366339
37303033646363636637363333323533363037616536303737653064333230313761313366666664
39643861393636356663396431346139646562383035396461323165663665626238653536353133
38346564383761636537333961656538656366333530383831306533623532303635333539616165
65303537663934626561666335643361333339316434373263613637663037613033633930313437
62376233613638636234303264353339653963383633303761316339626434303466376431343664
62616433623431353164383735313835383935333538636432386562616438623062643866333137
65366263656631616334303538363239316632373936663431643239303034373831623765663662
65646133393263373639393664373265376664356434303131396465646161616464623734316436
30396265396536333731363535613035333066616136306538623165306462333561306337386131
38313135663030333439643361633864663061323738313862326131623637663634323131373263
31383730653838656630306138386435643435343162626638656338333131363137363063616432
33326636383530343335613765346634316530376635636233356364373066343835376633356137
61356336616138383536653161363930306364313035313135656634363632353731393934656562
32643362636466656562666466613364333739393736313130656137396437373763323138353362
65353661636537623230623666373534646635626538616431323966326230396637326335326239
30373534616362666435346263646238316637646330336635346437333537643630303062303338
66626237303830386164656531356439656162656633303564316636663639383662306462373430
30616465353461366532356339313734386137623566653262656365643136303065353834646365
66353637306139316132386239306163343564356464316537656237623734326363333137366463
63633864373431353064393561653366653233616233616536373837353065353330643863663361
63653336633135636564373632653561393862633034393266613337343132363034616139366665
32313536643066376638303465616632316533636530626165663561383562623133383834376362
37353633636561326263376366373430656562306633666431316664313832353039366437396436
36333233313765363833316536363132623633336439653734623964373461396433343536316635
33356266663630376662373632323436313431376435646561336665366634333135316334303863
66653461386632303362366535346434616438653135613264666334363363663633666130313734
36376664636363623965363630326236346566393961326332613638636163666562346661363663
37663939306338643134316163333233363164616232353063323134373565616532373636653964
34363237353234393632326438346132353036313232313231663762323131303830613532386262
65663836636266616661373939383937363330623665656139613239396231326631613461396435
30343064366431343735383135623135643165376464343138393131666235323832623034333466
65663932623335323361313930343330636232626661323464623236313262303638353933613630
38646365373164633131653430616338643163383639643134376464336537363861316137353063
62623965633339303838663434306161633733373365383863633462303161643965326433333832
62383731376261303364373931376631626138656535363638376236393133636261626234616137
30333637656139343535636534346239373631633966616462636637306639336630336337343832
66373730373161613935393230623934653530666464636363653066353566333730336261386138
39626537306532633937353761303765393530316236373432373734373965623238303430623632
35396435343065346136616632376533633465346336366137663836323234613735306537373631
39653632303133633032316634323966333762623130613161323064346562346163633361323333
37613735666361366530356363393262393233346239653137303330336161613461666330636530
30366531633538636433323831646433646230376335646535643831316238323233383534626139
37343635383333633034346436376433303634383336316535306236323061623365303432326632
63343036623136363966373363373864623262646462333332616635623831383338666438333962
35326534656537386636363234316332653232626663623261343737636361633338343933373066
39346537653261356366656135303961356162363763623532613631636239633432393266623966
31303434343438643563303463326564393632653938656661366630323636643364333664306461
37616563663630643462653634666166633333393466373462363937336333303963303735666234
65666632343036646536393965643837313234353537663839353032323362663237323338623139
32656663313266393763343133356438303637306633306231323033643535346637386332643934
36653638623334343030363330643237636635633464336561316331383432636438333839336535
37346436393066626634623136366664383734653936393335626636383736343266653863396331
34303234393663653438333466313962313731666165366233313162613437633430333562343163
36663330656639643633633336643436636564366533383235383262313865656538366531303337
66353732626333623031326261306633313666393130373562626564343539313338366364656336
61353837633765646564653936366236303463363531336537333965666437303536643135393062
38343836316265396665386664326532616632346265613334353466643165663463613661333134
33623333343037333133656434376537303462343061393838336438346635363732393337313532
66346366663538373237666565643662383665623130346466316663346162313964653830373630
65663134303466303630353764636639613661613939396438633234636437363937356465326132
38653132613066356633333434366265303235363664636432323566643838636266363762633064
30383332373631326635613266366331656664393934353731653063366537616435396432663731
36376437326237303263393334386263396334316333316132653765343564373935353563353531
63396539613739316539656332623262636136363839353339353331306439323966656530383564
34336362653064656138363861353331306164663834626662646263623561393764303864306565
33366230303663303839313364353430363034316437326135393530366335363065303335636564
31336135333163643864396264383437633736336639333539656661653638333936643030396339
64646336323338376530653735613739333939363366363333623036663861613061366531643034
36303037363361333561313961343730363533383435376362333739363863643365616166613036
64663134666166383663656338323933386362393162636132306331633065323139663162383632
38373264376666363964653138623232313732626665386333643435663531373532346539636636
34303839653338366330613734646636303163643532373030613239313239646630366438386662
36616263373562626435346638383836336635386661396565653836633339353938383164373137
39323364666636356561343139373830383865633466656532336361336363363964353933613133
64343833343330353433376166323531393066356437333132346133643364383562393731333234
39363830653233386566356130356439353535353636656362616534326336616163316233353962
32663361363836646166326663633730333436376230663235393738396639616331303066626662
31643139303835303431303535393536643163393264386463323862303439353536366131373033
31653234323165373562313063636436313538393639663962393139323231303863336165383830
32363535616361666533316132343965343661373762323064333161383463343731356161326333
32623464666562626531343664643933376162646631313166643365666465383035613464643762
30663065616166613531343339383662303438653531333866396561346637623664376266373839
31643233366363646632663635343161303033323061353737666233663561623462663837643236
64613363383631663039613031616133386130363730333361393739356361376165353036326237
34383533316436333465313939306437383337663864643935346336353935303664663866663564
34346362313730306132373936663764633662646466333135343332393765373333333363376135
61303839363236373066643066363135306266333364666132323033393836356531313232363864
61653565323263306263653535306634313662366331643663643966353164326161383530613730
65303738336561363161336330373832393062613130303131633265303037396230313839306661
62373938613034383966336137303431613161343937343461663066343436626563623366646132
32386238353662663337363363393039653330383331636430343632343064383565626338313030
65643333316431616637643162353634323333313165343833636465363461333063306538396636
34343832633565383961656537373562636433323566663830326136353765303230393939356132
62356130396264633930356362653066363163623631613434326464383361643262373936636461
61616631363335353366323865316439396335373966393464626335353631383137376532346262
62343930633430333335623639643431306533396531333666653030643161356339353933643837
33343734373435303235646239343838363439333433333437643162326539326261656330643938
38306131343464613330636636396266353638616230333263323334353862363738623362633034
38626566303336393163366265306564646436623630656533633630653836333035616364623031
35363735343835306132646664613261353230336537646161643463316566343430373461326233
62396563343939643432663062633064383738383834336634323635396461383830363063636432
36666566396535363835303034363732653433623665353437626533633032333838316564323834
30356132613563616333313039666330666565666534393139653065663739363061306365666135
63396262393464653566626635643365356533623761643763663662383832393164326462313837
35616233396165363434633434613031663739343432366263303734343635336136323830333163
33626461646661343564303134366139666639383630663438396139306566643636623431396332
35633232333064383161383933333931636561313035343561643534656635363432333266376536
37373537336239363166306430376132323862333732393164323964363966666262346564613631
39616438623830386266643634663562396336666134323137366233363061613332376661326435
66313565346663343538383235356232346438383835623364353334376565393464393966366538
65376334623438663562623930616266353961393430353361623034306231326530303438316339
34386462666166363534643764633563613934366164303531343366383464643832353739316433
36326364663737353562396261363736393562393964633539363031353432356238333461383730
39386437663434326635373465643234373034303963396434656161633839613731326365613736
36343664386565313535343635376532386331643264613563613735373631626466313964383639
65663761663033333932336665383631636563363561646635383965393039393363346166613861
66393466326634633438343632376538326634353937633562333137373565343339626635376133
36626638393932623837336162343831303436643535643232616432363061346165373965323365
63386632336532656635643863623132393931383838346263636339646332373665396632653361
32346330306362343538353833313833383334366666613436343066636661366238333764353863
37333534366536313464383364303335356232363866363737376665616438666533633066373164
61636362363237313362633865333630386430623431656437646437396131363035616434343136
33353839353765623531323061306465653336623333393837363538653830326561396137613138
64646133626130323234383763623935323433336535343631383535356437333864633330393134
66623530653636623565353830613538616433613538383632303539386362383636376465393036
36373930326430373336666138653334616530653930336364353464613163656131323566393933
62646265393539313432653462616431636232326131313239363337396635336131383534353338
61356535663130336266376431666533353864393766313962623563313131336533656461373836
34653833353730363433663435313039363861386336643937626430326563353230386630356437
63396636393561376330316232343964666337663532353438303964613365303632323263633161
36323033313935636239633232333632373637613439633332666665356331363532373138393861
32326239616338366364613765393665396461633034346436623037666133623565613266376536
37633966626636323762623965653831646434356232626338636332656661333464623261393933
38643631313063336161663830343835366336346632636663353561623835303130313163366635
38633038366234303334336432653534353734346236346638356135633234323364633937666162
62323132393032303139393139633663303138646133626364626533356465323464613338363533
37306362346431306164613637303965373132323633613439396166383134323162313164323431
34383136323636636539353264396439613231653663636638343636626562353861383736306331
31303564313861306666666332303136343330363063313964353331346433626366303935383066
35386261633137613331303630613137376434663836383930366263666262656666356139343164
35616334373931343336636637356339343266613161623561336264623866633834626664663333
66346231386433383163653437353336613731646435626431366663666465313864613837343365
36633962326633313735393238373166636435656661373732386637626264323363623632353631
61613062653164353761313730613936356632306438643431306332326363616464343237386539
63393866393461663135616235643465373838336566336330346662616130666130396364343634
64333335656565623662646136313766623031363037366535386461393430646231353536346332
64313431326164306266666539366264616139303065626536333664623737336134643761333536
66333331303831316137646137316636633161353766393136363038653765376465623561373761
30633331306565393365323735613239393338326635626136363933303862343465623166313539
38393533376630313666643335623165336165633933323036643539623136333938303233353063
30373032633036663963323831646234653537646230643363663238356631303434373265303735
39613263383261306332346264646562636432663238663536653934386630393534653832333332
39316530373861323830303136663462346164386261636666356235633563646264316636363066
39353464346131643239363736383335643132383862303435366230353631613330333133396336
34373532613134613231636636373132643462353865363439356336383266616631363139656265
65333332663139393135643366623266333561313538646639666636303038663962626438663934
35386463386635613236313666306266353064663666333861643932633862336236333435363539
66356130646235636637626339356632376638316231656266303637623965636135636333643065
34396633613536336163306561363332356430616334393133396639663731653863336531313564
35653333356232396232613864313834656533323034396134313061313731613732386366343134
35663836313639663038653036633338336666633066396136663964646137636262313762306261
31313266643663373736393637323539343035636464336563636162343830313831383631313865
66633337613632343363356331663639616231653033663962376131646533313237333536663438
39353034383661396237623438623361323232653135653430313738633835613961323735343365
63376131386339643330303938343765393365656466393965363838343431373763616139393066
37323037396165636630663733333661626462643437616633383132396532396263373936343538
30333732303235376435666565313435356137383538313039626462363066373864303633616638
61363438396634613862303965653937356662373231346539653339333034343961623435393739
62393430633365323034646233393537306266326234643339623339663764343935386430383461
65396434363634363439336137613837316532373931626434323366393564646630313330366162
36653532306439653964373631653061363664663039633134633233346166376261323938363438
65653637313432663962646536376634333735616531653133653061313534623163323738643466
65363538643336353338346632323565313234333463396464663732613332646237653332623866
38666535386337386361333565646633366538333333346336363636333237316462636365383939
61393631356163313933643230343133643539323834363437306465653438396565393532636464
62343263353638343565653065373461386461363736313931313163656138656333383733373532
62356563643663373765336334306466303437343733316431623665626465666633373463366136
63663738663431306532616431373031666462346131623665376235663634336265303035346432
37616536663965643236346264656361613732386239653234613932326462613637666231396531
33316162353038306535386661626464316130343436353230383934346263323738643731373534
36616537626437393530666162336338383563333636656230646235353266373532653431633539
34646531303630373034323932303863623536666261326463303031636666363738363664653738
31376335316239373437363431626630613161663962663934643431303966323237616339393132
33366661383263396632633838393536383335343761306461313638613236326266396363623634
37623165613935373730663139303737373864663035323030643264663932353465666138303264
35326665656538353762643637393664636137313034643637643730313336356464383131343463
30623232363838373338366163326538316465643930656230323831666462353037633131643961
39393238386637313632663336373661636230386632346432366633356139376430646338316235
66393663366232366230326262633638653863396635323539363465383535303433643061633839
61666431353737393939646638653435343039353463616165653630373130383238333336303533
36343331313235376263303633326139613734663264353330643666633234323730366134663262
66643730373761323439636161623135646337383163353361633663376466343839623437613662
62633837653464316430626266646338333530646234623834643739303738366165646235323232
63663433336664376665623734613965626464653832643366353432633437643739396265376531
66636436336135343833356334636531376131316463343162353362356439386139346365333938
35646431306532663936616264626538666638336130313133323830313966353161666236323735
36313666346638363865336331313163346635613735363634366265373363303033616663363563
63353530636566303733343962653138363532326339666230666661393766333863363539373565
32363733613164373762646633373833373466326633383339626431323462306663353235346664
62653361323631623063313435663735333638616366613634363166656431303031353833646563
39313630366430613234643361666365386462346338323330646366366335656136643462663166
63316135306437376139643337646639303261666434316266323465373832333534373730636465
34313832633839396634316561346633643734373763666136363039653136353866343132326430
37303737383030616635343539386337336236376236383961333336363030376333666432613362
61366365316364613639326236333738313936393535646636383261356635303238303938383130
33613264396339646136346631643938356138653535636636393837346430396433313661613337
31323836323162323632343462666433633537623763653566643231656134353235383131663466
62313730366338646132626338623934646364316637343364653236316566666338363339363731
38373832316239613961646432373864306235656339373963643938343664346238373761663830
35663732396430666536313239386465373235393466643930383439613862313337313638386662
34623033653238333064343861633132313762646534646263663566633264393638343730313939
63353438333331383861343135326439303965333834646430393531383762623765666237316165
65346263386636363638663530643633336331623734343231626363353165323562616138373331
33393464333464666235666539333663663061383335343933373066333966346131653361333935
62643363653236626330656561383635656431303231353731396466363232613462323938643730
63356531616364346264653666613564613239646465376238336337383938313035366636396638
65303566343738343731656338663832396330623066646233376436323334633339343162383966
35343164626365386162666162346366656539383334346238336131616338393261306537303534
62343164386462393039393763303566346163646330336264623462353464306232363934343630
38396565613830346634653934653737636262646438643866376138633065303239633538373432
32653562343666313361383331636261313331363639633465383161393063336335343938653933
39663865613236393037333031613964633961326332316662666531353839343937393761636164
63643230623332626266323061386665333862303532313530653534343762383066643736336466
36626564633265303330666331353163396130376331333136386638306233343138613664626537
31626236643637363766636635333465396233663239353865306638303331363937663936613563
30306166393033336164333432626239333338653837336234393536343438326436343434373433
62613165633736353838313439366434323664333832366132663766613236643430313764343032
32636630393032343130623438333166306564366562373764656135653730346237343737346638
66333462383865333733316237663833323036643230616531353630306565353236363734643839
38616532626537613336366637386633653065383135643135363332353731353663323365653763
34623433343135636338303662633966343965303461313136373333396538626166633330343635
61633034356135623639623636626463373339633665366362343064313262333061666637323163
38646266613461393533663061376132363234376164393939663763643962633361663665336239
64356633323730303262383766386635613361316139353331346466643661333331336161636137
63616230396361626133373934613164336335376565656230623030393861383234346138303564
39326134383063393765336461306237653839336531363538383666313832353732663536343134
38663535316333656164396562303036653137653366393462663238613632383230626130346266
33653136383237303139316133636230373866316164336663613666373565643733663739323362
37633734356234626432646566333038303465333563323931396630386430633032626536623765
38623961373063323464353465316432393134633432343465633030363539363865383661646138
64373436336132356535646164336631626639383765386166326532643436393638653431623533
65666639346664613266616364663033363730373637343962393533656463653661613737303837
31633262653361373036383636323766653538346532353038393835303931323264376535373062
36663631316234363063393938633034306465373966333062663162653266386237663137653033
66346435313334653962373935316537376132353962653535366564306132636534306164666431
38363634333635373966353636633836373865666237656336626139353130626332383338313238
62616632616337363664616165336563616161386662306535666634613965613039356165396239
32316364623534643662323334326461386335666530373662353931363131306364626638636430
61363533613638616632396463343239633865643438373837333535336662366432373738393361
31663064623431383336343034653734306635323764613031353862646663626339323764313731
33313132613138633364646532633334643663663339633264343365363231633034666265376161
37313261616230616664396138306264653139643438306662313462386439336434373664313532
61396631663663353334653961363663393330353830306130313065366330653761393436326535
38326163366564373236386431366436633138326561303237613965353266346361303063613666
66333637313061383138373661643964326162633137626163383936386132346636386335353134
37393661666465366331383164366666666431386166396337386630643438643564336139656634
61613232653433336330376433653062333062663034306630653538303337343661326634363463
35626130353561353436386261663335623964633564303365316166306237306634633464653565
61353532613631653037643162666366313339633034646435316338353034366630303736386639
66396236646132366265653536386435333066306461303334646535643835393730643338376633
37303161613837383838323961343834343562666532366136303337363832363765326637653934
66396331333938633034303761313566656533346336393531326235393862623462306434396438
33393539306234626163626664393037666533326133336464323831323132633035313561303034
38343332306532303939363038383934376136333661343435313238356536396537333066613665
31623564306435313466343133643730313262623765346464616263313831313164353162636334
64666133333330646666663336613332373664653932613834383739393762326636303466363134
33393164326232353065396361636539626531656337353363666233343139333263383664666531
63663539303165643564356261346433373438396532396439363330623035353636386531356437
30326231333333393533376366333062386464323565306463336164636434303137386661393139
65336135313462363237343861313165393862393233656561663662333162306539316365393063
62623837353437373236653965646131383262653733303038306134626430623961376366383331
62383366336637626531653532633664616537333032386434306330616439373430383664623638
65336666303263613965376664336262353139396535646662316233343764396661616131346237
62666561373866646564383634313639383636653861666164393263306339616163383965643366
34656538383934313136316231643934663963643737353637633032313930623266323466653032
36636339373034336463313862353638313538316465373132616166323863373832626330383937
30313565373731663663336466363239303734336632396636346139636335646162353166643330
30323530613736613736623238393233316635333437333938313932623834643363616239343032
34626636323461303438376635633637636232386333636139626565303635656630343062353639
39396264386638326666303438623334336163343938336566353034656638323633343837376433
65366232303838643832623332663435636530363638636433303336356531633131356266396334
37313864393761623333653638383766366261353334653066316563386437383432343964616535
30306134383166666630636666316630383030346130353865386539303537363539636161643733
62343530396563393464656566363739313666613762613565386331366365393338313237636231
65326630313630363139626161373730373962656134363733633530376261613239323631623433
61396631333736383731363135383664663632643939376334633966373231363430326663323061
31666134396439346166346430363739626665663133323930363663303164616337316233656333
39303833653366353931343936623362346163363766346461653264353733363439643439336435
32656665623532666465663834363564666634653038616361393665303766326436306533613439
64616334393539643662303036626131623137383164303566623430646337353264343030303935
33313164643439643662393632323433366238306261623335656430396437396535653335343965
31616265363437356263616539343232333539393133323334633730366263386135393864393162
33653161623735363335323838353231326166376365656661373965353433336466623134386230
61653733393737643832303562663933353935626437316538393133613063316566646565323034
32643235303038656362663561666539633432383636333463396162353035323531623133656563
65376362643432643933393063393266663563663636616261663638626238653633303630383062
35363134323131323236623563353035306430343534643962333530383836666565646163346139
65633239396437346432346233643463333831663636323833386338386430373430336536313463
31386131653831623832346564343137326163326261306463666563643839653864353764353861
61663539646431633531653465333336663733663264336565386662626335376231643563356337
65653439613164306639353862656264333733386235333732643730623639333937396236616466
36303332626163323932356338313763363563326139633836343637663832633262323065326435
31643134303737316635336534613038313362396335636164656436623764656537303732643730
35613936303762623765656265663362613736636531633335666230643335633161303738663465
61636165376538313839646133653262343530393837313465343330626362626164366165323831
34396134616138363562313934626239353135336335656331383035393763306166666533643465
34383436356162323631373164373565313563656664663762343630383161646633356434643631
64306239373363343161663030343734366239356237393033643931646434393237363239333631
66343963633266613138353433383338343737636238346433366361643436363233373034656231
37633030623232613030633832306337663336616632333730373639323261646432616531366634
63663837623235383438363533313330663135323661626236333761323165663638623930653631
37633933383562396331326264623866353966633932663961643032383363613333323937383538
34653536313866336530343832336564353036376234383538323732306565636234623832376562
64626166366164626436663564353539323765663365303332373138373034333064626164336439
64333639313832656439613532326330336630613131386463663934653539663763646138343435
37636161623363613039666334626164643464313439376330653837363661393431376632326137
34383134386563343337363037616431366239623461356563613039393936653330303830303534
30373463363265653439633736396664623661646237633537373465316362383965333738333831
37373735636362376265376363386566313663343961623064343363366339373236316434653737
35356436316230626631623130386631636664623762613438313632613134396131336534393931
61663639653166376465323162313064623836623366393464386365313739626636303265343463
30336665636238343930303539313930626439326462343865326531616239323036306230363634
33626531393332393332663137303938613163653637376264663863643163346635616564326337
65393335383534323038656464353963316164396263663533643464623561343337623661663635
39663331633239336432366466623666313036393661623966396630623431626465646437363634
31623536346231373534656262336466336465353261663433363134633030393533326539666661
39623931313339646134363337343233346334366262646638613039363730613433313863396564
64643436653637356563393131666465663332326665373937643764376231336331616361323530
35303038323739316265363863316139363033386538346139636262626633303563353737363739
35656463646364323964366533656339643466323965323661306166393933353237643834373264
66623533623837306238393234666331636566616535323231393364643039323064343433393533
66653333633931303761376162666530386161336534343932633933616139306430326430326566
63656534343362323830643831643836303331643364396566626662333439646531616465623963
64616361396635393339643035616536643435353738383437303832626135316631633166613434
64366335373365653763353436333961663265643739646536343564383062396337333362363235
38326236363564313437343832383961303032643831363563646664373338616363623935656538
35376233326434623339376463346338636162306461656434626135623263613632393135306531
38326434636637366633336337663739633232386531336162333939333132336362363430363965
64316337353637656439343233343362666130663165613335633363353339633537323436323334
61343838383365346466336332643065353836366463613736353231343665623430373232333133
34616261653862353062376231356438363038643238333833353464653138303734373735303135
37663338363737666461646161653434636237313332343838346335313363616134653262633362
33653236343930316231636534613666373061373332653434356530343334313434366137636364
64393035383635646363373732636166383931343362356666643836313065346365396162393237
64623064643831363161653034316162363935356563316536393531643164643466323432323139
63643533373431653435383136663665316131343632383366396366376237346265303634326232
37653331666434316263653531323861386339326130396563663864373364663831336465326665
35383534633433356134636632353333346633383335383039626432323932333433303663383364
34633133323166313032393031346536333863306163333462623063663239666563653365643631
33623238343335333932303836396662323832366139616334653639653838303662313162643335
63396164353038636432653332376536323366343765383034366633666639353936316436613130
33633763396366363565636532613738663963363932373061313635376338626665616131346532
65323564333465616664393965633761653962616137663336333430663433646562333364633361
31613736366462333131356131303063386633386664366166653136333266663161663332353737
31623139663130363735336233353135323166613964653839313137316239393431306239343137
65643431353936616433336263353439383763663833333031653033353464653762656433393436
32376666306637636134

View File

@@ -0,0 +1,7 @@
---
# main-street-station — JMRI / LCRR server
jmri_profile_id: "My_JMRI_Railroad.3f178885"
jmri_lcrr_repo: "ssh://git@gitea.mk-labs.cloud:2221/rblundon/LCRR.git"
jmri_leviton_email: "{{ leviton_email }}"
jmri_leviton_password: "{{ leviton_password }}"
jmri_ssh_authorized_key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINnSM/9fO8rz/amqkyoGUzUKNNzzmtSXPwOCr1O9zKNO ansible"

View File

@@ -9,17 +9,45 @@ jmri_user: jmri
jmri_group: jmri
jmri_home: /home/jmri
# Profile — override per-host in inventory
# Find on the old box: ls ~/.jmri/profiles/
# Profile — set per-host in host_vars
jmri_profile_id: ""
# Config restore source (path to the backed-up .jmri directory)
# LCRR git repo (set per-host in host_vars; leave blank to skip clone)
jmri_lcrr_repo: ""
# SSH key for jmri user (for jmri-gui X11 access — set per-host in host_vars)
jmri_ssh_authorized_key: ""
# SSH key for jmri user → Gitea
jmri_gitea_key: /home/jmri/.ssh/id_ed25519_gitea
# Config restore source (legacy tar-based restore — leave blank to skip)
jmri_config_src: ""
# Ports (for documentation / firewall rules)
jmri_json_port: 12080
jmri_withrottle_port: 12090
# USB serial device for NCE (set per-host in inventory)
# e.g. /dev/ttyUSB0 or /dev/ttyACM0
jmri_serial_device: /dev/ttyUSB0
# ---------------------------------------------------------------------------
# Phase 2 — Layout power monitor (Leviton Decora Smart Wi-Fi)
# ---------------------------------------------------------------------------
jmri_leviton_email: "" # set via group_vars (vault-backed)
jmri_leviton_password: "" # set via group_vars (vault-backed)
jmri_leviton_switch_name: "Layout"
jmri_monitor_poll_interval: 30 # seconds between polls (active hours)
jmri_monitor_quiet_start: 1 # hour (24h) to stop polling
jmri_monitor_quiet_end: 10 # hour (24h) to resume polling
jmri_monitor_stop_delay: 30 # seconds of OFF state before stopping JMRI
# ---------------------------------------------------------------------------
# USB device identification (ENV{ID_SERIAL} from udev — the segment of the
# by-id symlink name before the -ifNN suffix)
# ---------------------------------------------------------------------------
# LocoBuffer-NG (LocoNet bridge) → /dev/jmri/loconet
jmri_dev_loconet_serial: "RR-CirKits_LocoBuffer-NG_CDC_ACM_SERIAL_DEVICE_AA5700218A"
# NCE command station → /dev/jmri/nce ← triggers service start/stop
jmri_dev_nce_serial: "Microchip_Technology_Inc._Simple_CDC_Device_Demo"
# FTDI LCC interface → /dev/jmri/lcc
jmri_dev_lcc_serial: "ftdi_usb_serial_converter_ftDYQHZX"

View File

@@ -1,9 +1,23 @@
---
- name: Reload udev
ansible.builtin.command: udevadm control --reload-rules
changed_when: false
- name: Reload systemd
ansible.builtin.systemd:
daemon_reload: true
- name: Restart jmri-monitor
ansible.builtin.systemd:
name: jmri-monitor
state: restarted
- name: Restart jmri
ansible.builtin.systemd:
name: jmri
state: restarted
- name: Restart sshd
ansible.builtin.systemd:
name: ssh
state: restarted

View File

@@ -1,7 +1,9 @@
---
- name: Install Java runtime (headless)
- name: Install Java runtime (full — required for GUI mode)
ansible.builtin.apt:
name: openjdk-21-jre-headless
name:
- openjdk-21-jre
- openjdk-21-jdk
state: present
update_cache: true
@@ -16,17 +18,147 @@
name: "{{ jmri_user }}"
group: "{{ jmri_group }}"
home: "{{ jmri_home }}"
shell: /usr/sbin/nologin
shell: /bin/bash
system: true
create_home: true
state: present
- name: Deploy SSH authorized key for jmri user
ansible.posix.authorized_key:
user: "{{ jmri_user }}"
key: "{{ jmri_ssh_authorized_key }}"
state: present
when: jmri_ssh_authorized_key | length > 0
- name: Add JMRI user to dialout group (serial device access)
ansible.builtin.user:
name: "{{ jmri_user }}"
groups: dialout
append: true
# ---------------------------------------------------------------------------
# Phase 1 — Stable udev device symlinks
# Creates /dev/jmri/nce, /dev/jmri/loconet, /dev/jmri/lcc
# ---------------------------------------------------------------------------
- name: Create /dev/jmri symlink directory (static — survives reboots)
ansible.builtin.file:
path: /etc/jmri
state: directory
owner: root
group: root
mode: '0755'
- name: Deploy udev rules for stable JMRI device symlinks
ansible.builtin.template:
src: 99-jmri-devices.rules.j2
dest: /etc/udev/rules.d/99-jmri-devices.rules
owner: root
group: root
mode: '0644'
notify: Reload udev
# ---------------------------------------------------------------------------
# Phase 2 — LocoNet traffic monitor
# Installs jmri-monitor: watches /dev/jmri/loconet, starts/stops jmri.service
# ---------------------------------------------------------------------------
- name: Install python3-venv (monitor virtualenv support)
ansible.builtin.apt:
name: python3-venv
state: present
- name: Create virtualenv for jmri-monitor
ansible.builtin.command:
cmd: python3 -m venv /opt/jmri-monitor
creates: /opt/jmri-monitor/bin/python3
- name: Install decora_wifi into jmri-monitor virtualenv
ansible.builtin.pip:
name: decora_wifi
state: present
virtualenv: /opt/jmri-monitor
- name: Deploy jmri-monitor script
ansible.builtin.template:
src: jmri-monitor.py.j2
dest: /usr/local/bin/jmri-monitor
owner: root
group: root
mode: '0755'
notify: Restart jmri-monitor
- name: Deploy jmri-monitor systemd unit
ansible.builtin.template:
src: jmri-monitor.service.j2
dest: /etc/systemd/system/jmri-monitor.service
owner: root
group: root
mode: '0644'
notify:
- Reload systemd
- Restart jmri-monitor
- name: Enable jmri-monitor service
ansible.builtin.systemd:
name: jmri-monitor
enabled: true
state: started
daemon_reload: true
# ---------------------------------------------------------------------------
# Phase 2b — LCRR config repo (clone/pull .jmri from Gitea)
# ---------------------------------------------------------------------------
- name: Ensure jmri .ssh directory exists
ansible.builtin.file:
path: /home/jmri/.ssh
state: directory
owner: "{{ jmri_user }}"
group: "{{ jmri_group }}"
mode: '0700'
- name: Deploy jmri SSH config for Gitea
ansible.builtin.copy:
dest: /home/jmri/.ssh/config
owner: "{{ jmri_user }}"
group: "{{ jmri_group }}"
mode: '0600'
content: |
Host gitea.mk-labs.cloud
HostName gitea.mk-labs.cloud
User git
Port 2221
IdentityFile {{ jmri_gitea_key }}
StrictHostKeyChecking accept-new
- name: Clone LCRR config repo if not present
ansible.builtin.git:
repo: "{{ jmri_lcrr_repo }}"
dest: "{{ jmri_home }}/LCRR"
version: main
accept_hostkey: true
key_file: "{{ jmri_gitea_key }}"
update: false
become_user: "{{ jmri_user }}"
when: jmri_lcrr_repo | length > 0
notify: Restart jmri
- name: Remove auto-generated .jmri dir if it exists (will be replaced by symlink)
ansible.builtin.file:
path: "{{ jmri_home }}/.jmri"
state: absent
when:
- jmri_lcrr_repo | length > 0
- name: Link .jmri config from LCRR repo
ansible.builtin.file:
src: "{{ jmri_home }}/LCRR/.jmri"
dest: "{{ jmri_home }}/.jmri"
state: link
owner: "{{ jmri_user }}"
group: "{{ jmri_group }}"
force: true
when: jmri_lcrr_repo | length > 0
notify: Restart jmri
- name: Check if JMRI is already installed at correct version
ansible.builtin.stat:
path: "{{ jmri_install_dir }}/JmriFaceless"
@@ -79,9 +211,60 @@
- Reload systemd
- Restart jmri
- name: Enable and start JMRI service
- name: Enable jmri service (monitor manages start/stop — do not start directly)
ansible.builtin.systemd:
name: jmri
enabled: true
state: started
enabled: false
daemon_reload: true
# ---------------------------------------------------------------------------
# Phase 3 — X11 remote GUI access
# ---------------------------------------------------------------------------
- name: Install xauth (required for SSH X11 forwarding)
ansible.builtin.apt:
name: xauth
state: present
- name: Remove old jmri X11 sshd drop-in if present (renamed)
ansible.builtin.file:
path: /etc/ssh/sshd_config.d/20-jmri-x11.conf
state: absent
notify: Restart sshd
- name: Deploy sshd drop-in to enable X11 forwarding (must load before hardening)
ansible.builtin.copy:
dest: /etc/ssh/sshd_config.d/09-jmri-x11.conf
owner: root
group: root
mode: '0644'
content: |
# Allow X11 forwarding for JMRI GUI sessions (jmri role)
# Must be numbered below 10-mk-labs-hardening.conf — first match wins.
X11Forwarding yes
X11UseLocalhost yes
notify: Restart sshd
- name: Deploy jmri-gui script
ansible.builtin.template:
src: jmri-gui.j2
dest: /usr/local/bin/jmri-gui
owner: root
group: root
mode: '0755'
- name: Deploy sudoers drop-in for jmri-gui (wed can manage jmri service)
ansible.builtin.copy:
dest: /etc/sudoers.d/jmri-gui
owner: root
group: root
mode: '0440'
validate: 'visudo -cf %s'
content: |
# jmri user can manage its own service (for jmri-gui interactive sessions)
jmri ALL=(root) NOPASSWD: /usr/bin/systemctl start jmri.service
jmri ALL=(root) NOPASSWD: /usr/bin/systemctl stop jmri.service
jmri ALL=(root) NOPASSWD: /opt/jmri-monitor/bin/python3
# wed retains service control for automation/admin use
wed ALL=(root) NOPASSWD: /usr/bin/systemctl start jmri.service
wed ALL=(root) NOPASSWD: /usr/bin/systemctl stop jmri.service
wed ALL=(root) NOPASSWD: /opt/jmri-monitor/bin/python3

View File

@@ -0,0 +1,25 @@
# /etc/udev/rules.d/99-jmri-devices.rules
# Managed by Ansible — do not edit manually.
#
# Creates stable /dev/jmri/* symlinks for all three JMRI serial interfaces.
# Uses ID_SERIAL (full serial string) to match by-id symlink identity,
# so device numbering (ttyACM0, ttyACM1, etc.) doesn't matter after reboots.
# Ownership: root:dialout mode 0660 — jmri user is a member of dialout.
# NCE command station (Microchip CDC)
# Appears when layout is powered on; disappears when layout powers off.
SUBSYSTEM=="tty", ENV{ID_SERIAL}=="{{ jmri_dev_nce_serial }}", \
SYMLINK+="jmri/nce", \
GROUP="dialout", MODE="0660"
# RR-CirKits LocoBuffer-NG (LocoNet bridge)
# USB-powered — always present when server is running.
SUBSYSTEM=="tty", ENV{ID_SERIAL}=="{{ jmri_dev_loconet_serial }}", \
SYMLINK+="jmri/loconet", \
GROUP="dialout", MODE="0660"
# FTDI USB serial (LCC interface)
# USB-powered — always present when server is running.
SUBSYSTEM=="tty", ENV{ID_SERIAL}=="{{ jmri_dev_lcc_serial }}", \
SYMLINK+="jmri/lcc", \
GROUP="dialout", MODE="0660"

View File

@@ -0,0 +1,111 @@
#!/bin/bash
# jmri-gui — launch JMRI GUI over SSH -X (X11 forwarding)
# Managed by Ansible — do not edit manually.
#
# Usage (connect as jmri user):
# ssh -X jmri@main-street-station jmri-gui panelpro
# ssh -X jmri@main-street-station jmri-gui decoderpro
# ssh -X jmri@main-street-station jmri-gui status
#
# ~/.ssh/config recommended:
# Host main-street-station
# User jmri
# ForwardX11 yes
set -euo pipefail
JMRI_DIR="{{ jmri_install_dir }}"
JMRI_PROFILE="{{ jmri_profile_id }}"
JMRI_SERVICE="jmri.service"
MONITOR_SERVICE="jmri-monitor.service"
usage() {
echo "Usage: jmri-gui <panelpro|decoderpro|status>"
exit 1
}
check_display() {
if [ -z "${DISPLAY:-}" ]; then
echo "ERROR: No DISPLAY set. Connect with: ssh -X jmri@main-street-station"
exit 1
fi
}
status() {
echo "=== JMRI daemon ==="
systemctl status "$JMRI_SERVICE" --no-pager -l 2>&1 | head -8
echo ""
echo "=== Layout monitor ==="
systemctl status "$MONITOR_SERVICE" --no-pager -l 2>&1 | head -5
}
launch() {
local app="$1"
local binary
case "$app" in
panelpro) binary="PanelPro" ;;
decoderpro) binary="DecoderPro" ;;
*) usage ;;
esac
check_display
# Stop the daemon if running
if systemctl is-active --quiet "$JMRI_SERVICE" 2>/dev/null; then
echo "Stopping JMRI daemon..."
sudo systemctl stop "$JMRI_SERVICE"
fi
echo "Launching $binary (profile: $JMRI_PROFILE)..."
echo "Close the window to return to daemon mode."
echo ""
# SSH sets DISPLAY to localhost:N (TCP) — translate to unix socket form
# so Java's AWT xauth lookup matches the forwarded cookie.
export DISPLAY=$(echo "$DISPLAY" | sed 's/localhost:/:/')
# Force AWT out of headless mode
export JMRI_OPTIONS="-Djava.awt.headless=false"
# Run directly — we are already the jmri user, DISPLAY is set by SSH
"$JMRI_DIR/$binary" --profile="$JMRI_PROFILE"
echo ""
echo "$binary closed."
# Restart daemon if layout switch is still on
if systemctl is-active --quiet "$MONITOR_SERVICE" 2>/dev/null; then
if sudo /opt/jmri-monitor/bin/python3 - <<'EOF'
from decora_wifi import DecoraWiFiSession
from decora_wifi.models.residential_account import ResidentialAccount
import sys
session = DecoraWiFiSession()
person = session.login("{{ jmri_leviton_email }}", "{{ jmri_leviton_password }}")
perms = person.get_residential_permissions()
for perm in perms:
acct_id = perm.data.get('residentialAccountId')
if not acct_id:
continue
acct = ResidentialAccount(session, acct_id)
acct.refresh()
for r in acct.get_residences():
for s in r.get_iot_switches():
if s.data.get('name') == '{{ jmri_leviton_switch_name }}':
sys.exit(0 if s.data.get('power') == 'ON' else 1)
sys.exit(1)
EOF
then
echo "Layout is ON — restarting JMRI daemon."
sudo systemctl start "$JMRI_SERVICE"
else
echo "Layout is OFF — JMRI daemon will not restart."
fi
fi
}
case "${1:-}" in
panelpro|decoderpro) launch "$1" ;;
status) status ;;
*) usage ;;
esac

View File

@@ -0,0 +1,156 @@
#!/opt/jmri-monitor/bin/python3
"""
jmri-monitor — Leviton Decora Smart Wi-Fi layout power monitor
Managed by Ansible — do not edit manually.
Polls the Leviton cloud API for the "{{ jmri_leviton_switch_name }}" switch state.
- Switch ON after being OFF → systemctl start jmri.service
- Switch OFF for {{ jmri_monitor_stop_delay }}s → systemctl stop jmri.service
Quiet hours {{ jmri_monitor_quiet_start }}:00{{ jmri_monitor_quiet_end }}:00: no polling (layout assumed off).
"""
import subprocess
import time
import logging
import sys
from datetime import datetime
LEVITON_EMAIL = "{{ jmri_leviton_email }}"
LEVITON_PASSWORD = "{{ jmri_leviton_password }}"
SWITCH_NAME = "{{ jmri_leviton_switch_name }}"
POLL_INTERVAL = {{ jmri_monitor_poll_interval }}
QUIET_START = {{ jmri_monitor_quiet_start }}
QUIET_END = {{ jmri_monitor_quiet_end }}
STOP_DELAY = {{ jmri_monitor_stop_delay }}
JMRI_SERVICE = "jmri.service"
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [jmri-monitor] %(levelname)s: %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
stream=sys.stdout,
)
log = logging.getLogger(__name__)
def systemctl(action):
try:
result = subprocess.run(
["systemctl", action, JMRI_SERVICE],
capture_output=True, text=True, timeout=30,
)
if result.returncode == 0:
log.info("systemctl %s %s: OK", action, JMRI_SERVICE)
else:
log.warning("systemctl %s %s: %s", action, JMRI_SERVICE, result.stderr.strip())
except Exception as e:
log.error("systemctl %s failed: %s", action, e)
def is_quiet_hours():
hour = datetime.now().hour
if QUIET_START < QUIET_END:
return QUIET_START <= hour < QUIET_END
else:
# wraps midnight e.g. 236
return hour >= QUIET_START or hour < QUIET_END
def get_switch_state():
"""Returns True if switch is ON, False if OFF, None on error."""
try:
from decora_wifi import DecoraWiFiSession
from decora_wifi.models.residential_account import ResidentialAccount
session = DecoraWiFiSession()
person = session.login(LEVITON_EMAIL, LEVITON_PASSWORD)
if not person:
log.error("Leviton login failed")
return None
perms = person.get_residential_permissions()
for perm in perms:
acct_id = perm.data.get('residentialAccountId')
if not acct_id:
continue
acct = ResidentialAccount(session, acct_id)
acct.refresh()
for residence in acct.get_residences():
for switch in residence.get_iot_switches():
if switch.data.get('name') == SWITCH_NAME:
state = switch.data.get('power', 'OFF')
session.call_api('/Person/logout', {}, 'post')
return state == 'ON'
all_names = []
for perm in perms:
acct_id = perm.data.get('residentialAccountId')
if acct_id:
acct = ResidentialAccount(session, acct_id)
acct.refresh()
for r in acct.get_residences():
all_names += [s.data.get('name') for s in r.get_iot_switches()]
log.warning("Switch '%s' not found — available: %s", SWITCH_NAME, all_names)
session.call_api('/Person/logout', {}, 'post')
return None
except ImportError:
log.error("decora_wifi not installed")
return None
except Exception as e:
log.error("Error querying Leviton API: %s", e)
return None
def main():
log.info("Layout power monitor starting")
log.info("Switch: '%s' Poll: %ds Quiet: %02d:00%02d:00 Stop delay: %ds",
SWITCH_NAME, POLL_INTERVAL, QUIET_START, QUIET_END, STOP_DELAY)
layout_on = False
off_since = None
while True:
if is_quiet_hours():
log.debug("Quiet hours — sleeping 60s")
# If layout was on when quiet hours started, stop JMRI
if layout_on:
log.info("Quiet hours began — stopping JMRI")
layout_on = False
off_since = None
systemctl("stop")
time.sleep(60)
continue
state = get_switch_state()
if state is True:
off_since = None
if not layout_on:
log.info("Layout switch ON — starting JMRI")
layout_on = True
systemctl("start")
elif state is False:
if layout_on:
if off_since is None:
off_since = time.monotonic()
log.info("Layout switch OFF — waiting %ds before stopping JMRI", STOP_DELAY)
elif time.monotonic() - off_since >= STOP_DELAY:
log.info("Layout switch OFF for %ds — stopping JMRI", STOP_DELAY)
layout_on = False
off_since = None
systemctl("stop")
else:
off_since = None
else:
# API error — don't change state, try again next poll
log.warning("Could not determine switch state — retrying in %ds", POLL_INTERVAL)
time.sleep(POLL_INTERVAL)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,20 @@
[Unit]
Description=JMRI LocoNet Traffic Monitor
Documentation=https://www.jmri.org/
# Must start after udev has processed devices
After=systemd-udev-settle.service
[Service]
Type=simple
# Runs as root — needs to call systemctl start/stop jmri.service
User=root
ExecStart=/usr/local/bin/jmri-monitor
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=jmri-monitor
[Install]
WantedBy=multi-user.target

View File

@@ -1,26 +1,21 @@
[Unit]
Description=JMRI Server (JmriFaceless)
Documentation=https://www.jmri.org/
After=network.target
# Give USB devices time to enumerate after boot
After=dev-{{ jmri_serial_device | basename }}.device
Wants=dev-{{ jmri_serial_device | basename }}.device
# jmri-monitor starts and stops this service based on LocoNet traffic.
# Do NOT enable this unit directly — it is managed by jmri-monitor.service.
After=network.target systemd-udev-settle.service
[Service]
Type=simple
User={{ jmri_user }}
Group={{ jmri_group }}
WorkingDirectory={{ jmri_install_dir }}
# JmriFaceless requires a profile ID
# Find yours: ls ~/.jmri/profiles/
ExecStart={{ jmri_install_dir }}/JmriFaceless --profile={{ jmri_profile_id }}
# Restart on failure, not on clean exit (e.g. intentional shutdown from Jython)
Restart=on-failure
RestartSec=10
# Monitor owns lifecycle — do not auto-restart
Restart=no
# Give hardware time to settle before retry
# Give hardware time to settle on start
TimeoutStartSec=30
# Logging — view with: journalctl -u jmri -f
@@ -32,4 +27,5 @@ SyslogIdentifier=jmri
SupplementaryGroups=dialout
[Install]
WantedBy=multi-user.target
# Intentionally no WantedBy — started only by jmri-monitor
WantedBy=

View File

@@ -0,0 +1,337 @@
# Firecrawl Architecture Diagram
## High-Level Overview
```
┌─────────────────────────────────────────────────────────────────────────┐
│ External Access │
│ spaceship-earth.local.mk-labs.cloud / firecrawl.local.mk-labs.cloud │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Gateway API │ │
│ │ (HTTPRoute) │ │
│ │ TLS Termination │ │
│ └────────┬─────────┘ │
└─────────────────────────────────┼──────────────────────────────────────┘
┌─────────────────────────────────┼──────────────────────────────────────┐
│ Firecrawl Namespace │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ API Service │ │
│ │ (ClusterIP) │ │
│ │ Port 3002 │ │
│ └────────┬─────────┘ │
│ │ │
│ ┌──────────────────┼──────────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────┐ ┌─────────────┐ ┌──────────────────┐ │
│ │ API Deployment │ │ Worker │ │ NUQ Worker │ │
│ │ (firecrawl-api)│ │ Deployment │ │ Deployment │ │
│ │ │ │(firecrawl- │ │ (firecrawl-api) │ │
│ │ Entrypoint: │ │ api) │ │ │ │
│ │ dist/src/ │ │ │ │ Entrypoint: │ │
│ │ index.js │ │ Entrypoint: │ │ dist/src/ │ │
│ │ │ │ dist/src/ │ │ services/worker/│ │
│ │ 4-6GB / 2 CPU │ │ services/ │ │ nuq-worker.js │ │
│ │ │ │ queue- │ │ │ │
│ │ Replicas: 1 │ │ worker.js │ │ 3-4GB / 1 CPU │ │
│ │ │ │ │ │ │ │
│ │ Health: │ │ 3-4GB/1 CPU │ │ Replicas: 1 │ │
│ │ /v0/health/* │ │ │ │ │ │
│ │ │ │ Replicas: 1 │ │ │ │
│ └────────┬────────┘ └──────┬──────┘ └────────┬─────────┘ │
│ │ │ │ │
│ └─────────┬────────┴──────────────────┘ │
│ │ │
│ ┌─────────┼──────────────┬──────────────┐ │
│ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │
│ ┌────────────┐ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │
│ │ Playwright │ │ Redis │ │PostgreSQL│ │ RabbitMQ │ │
│ │ Service │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ Deployment │ │Deployment│ │StatefulSet│ │ Deployment │ │
│ │ (Harbor) │ │(Upstream)│ │ (Harbor) │ │ (Upstream) │ │
│ │ │ │ │ │ │ │ │ │
│ │ Service: │ │ Service: │ │ Service: │ │ Service: │ │
│ │ 3000 │ │ 6379 │ │ 5432 │ │ 5672, 15672 │ │
│ │ │ │ │ │ │ │ │ │
│ │ 4GB/2 CPU │ │ 1GB/0.5 │ │ 2GB/1 CPU│ │ 1GB/0.5 CPU │ │
│ │ │ │ │ │ │ │ │ │
│ │ tmpfs: │ │ │ │ PVC: │ │ Healthcheck: │ │
│ │ 1GB │ │ │ │ 10GB │ │ Required │ │
│ └────────────┘ └──────────┘ └──────────┘ └───────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────┐ │
│ │ NFS PVC │ │
│ │ 10GB │ │
│ │(nfs- │ │
│ │emporium) │ │
│ └──────────┘ │
│ │
│ Configuration: │
│ ┌──────────────┐ ┌────────────────┐ │
│ │ ConfigMap │ │ ExternalSecret │ │
│ │ (firecrawl- │ │ (firecrawl- │ │
│ │ config) │ │ secrets) │ │
│ │ │ │ │ │
│ │ - URLs │ │ ┌──────────┐ │ │
│ │ - Ports │ │ │1Password │ │ │
│ │ - Tuning │ │ │ Vault │ │ │
│ │ │ │ └────┬─────┘ │ │
│ └──────────────┘ │ │ │ │
│ │ ▼ │ │
│ │ - postgres- │ │
│ │ password │ │
│ │ - bull-auth- │ │
│ │ key │ │
│ └────────────────┘ │
└────────────────────────────────────────────────────────────────────────┘
```
## Service Communication Flow
### API Request Flow
```
External User
Gateway API (TLS termination)
API Service (port 3002)
├─────► Playwright Service (browser automation)
│ └─► Returns rendered HTML/Markdown
├─────► Redis (queue jobs, cache results)
├─────► PostgreSQL (store job metadata)
└─────► RabbitMQ (publish job events)
```
### Background Job Processing Flow
```
API receives request
Job queued in Redis
RabbitMQ notifies workers
├─────► Worker picks up job
│ └─► Processes scraping tasks
└─────► NUQ Worker picks up database jobs
└─► Processes queue from PostgreSQL
```
### Database Queue Flow (NUQ)
```
Job created in PostgreSQL (nuq.queue_scrape table)
NUQ Worker polls for jobs (prefetch)
├─► Status: queued → active
├─► Worker processes job
│ └─► Calls Playwright or direct fetch
└─► Status: active → completed/failed
└─► Results stored in returnvalue column
```
## Build Pipeline Flow
```
GitHub: mendableai/firecrawl
Tekton Pipeline (innoventions namespace)
├─────► firecrawl-api-build
│ │
│ ├─► Git Clone Task
│ │
│ ├─► Kaniko Build Task
│ │ └─► Multi-stage: Go → Node → Runtime
│ │
│ └─► Push to Harbor
│ └─► the-seas.local.mk-labs.cloud/applications/firecrawl-api:latest
├─────► firecrawl-playwright-build
│ │
│ ├─► Git Clone Task
│ │
│ ├─► Kaniko Build Task
│ │ └─► Node.js + Chromium install
│ │
│ └─► Push to Harbor
│ └─► .../firecrawl-playwright:latest
└─────► firecrawl-postgres-build
├─► Git Clone Task
├─► Kaniko Build Task
│ └─► postgres:16 + pg_cron + nuq.sql
└─► Push to Harbor
└─► .../firecrawl-postgres:latest
```
## Deployment Flow (ArgoCD)
```
Gitea Repository (homelab)
└─► cluster/applications/firecrawl/
ArgoCD Application (sync)
├─► Wave 0: Namespace
├─► Wave 1: ConfigMap, ExternalSecret
├─► Wave 2: PostgreSQL StatefulSet + PVC
│ Redis Deployment
│ RabbitMQ Deployment
├─► Wave 3: Playwright Deployment
│ (waits for infrastructure)
├─► Wave 4: API Deployment
│ Worker Deployments
│ (waits for all dependencies)
└─► Wave 5: Services, HTTPRoute
```
## Resource Distribution
```
Total Cluster Capacity: ~48 CPU / ~96GB RAM (6 nodes)
Firecrawl Allocation:
┌────────────────────────────────────┐
│ API: 2 CPU / 4-6GB │ ████████████
│ Worker: 1 CPU / 3-4GB │ ██████
│ NUQ Worker: 1 CPU / 3-4GB │ ██████
│ Playwright: 2 CPU / 4GB │ ████████████
│ PostgreSQL: 1 CPU / 2GB │ ██████
│ Redis: 0.5 CPU / 1GB │ ███
│ RabbitMQ: 0.5 CPU / 1GB │ ███
├────────────────────────────────────┤
│ TOTAL: 8 CPU / 22GB RAM │
└────────────────────────────────────┘
Percentage of cluster: ~17% CPU, ~23% RAM
Headroom available: ✅ Excellent
```
## Data Flow
### Scrape Request Example
```
1. User → POST /v1/scrape {"url": "https://example.com"}
2. API validates request
3. API creates job in PostgreSQL (nuq.queue_scrape)
4. API queues job in Redis
5. RabbitMQ notifies workers
6. Worker picks up job
7. Worker calls Playwright service
│ └─► Playwright launches Chromium
│ └─► Renders page (handles JS)
│ └─► Returns HTML
8. Worker converts HTML → Markdown (Go library)
9. Worker stores result in PostgreSQL (returnvalue column)
10. Worker updates job status: completed
11. API returns result to user
└─► {"markdown": "...", "html": "...", "metadata": {...}}
```
---
## Network Policies (Future Enhancement)
```
firecrawl namespace:
├─► Ingress Rules:
│ ├─ Allow: Gateway API → API Service (port 3002)
│ └─ Deny: All other external traffic
├─► Egress Rules:
│ ├─ Allow: API → Playwright (port 3000)
│ ├─ Allow: API → Redis (port 6379)
│ ├─ Allow: API → PostgreSQL (port 5432)
│ ├─ Allow: API → RabbitMQ (port 5672)
│ ├─ Allow: All → Internet (for web scraping)
│ └─ Deny: All other cluster traffic
└─► Inter-Pod Rules:
├─ Allow: API → All infrastructure services
├─ Allow: Workers → All infrastructure services
├─ Deny: PostgreSQL → Internet (security)
└─ Deny: Redis → Internet (security)
```
---
## Monitoring & Observability (Future Enhancement)
```
Prometheus Metrics:
├─► API Metrics (port 3002/metrics)
│ ├─ Request rate
│ ├─ Response times
│ ├─ Job queue depth
│ └─ Error rates
├─► Worker Metrics (port 3005/metrics)
│ ├─ Jobs processed
│ ├─ Processing times
│ └─ Success/failure rates
├─► PostgreSQL Metrics
│ ├─ Connection pool usage
│ ├─ Query performance
│ └─ Table sizes
└─► Playwright Metrics
├─ Browser pool usage
├─ Page load times
└─ Chromium memory usage
Grafana Dashboards:
├─ Firecrawl Overview
├─ Job Processing Metrics
├─ Service Health
└─ Resource Utilization
```
---
**Diagram Version:** 1.0
**Last Updated:** June 6, 2026
**Created By:** Rocket Raccoon (CI/CD Specialist)

View File

@@ -0,0 +1,412 @@
# Firecrawl Environment Variables Reference
Complete reference for all environment variables used by Firecrawl services.
---
## Required Variables (CRITICAL)
These variables MUST be set for Firecrawl to function.
### Server Configuration
```yaml
HOST: "0.0.0.0" # Listen address
PORT: "3002" # Main API port
WORKER_PORT: "3005" # Worker liveness check port
EXTRACT_WORKER_PORT: "3004" # Extract worker port
```
### Database (PostgreSQL)
```yaml
POSTGRES_USER: "postgres" # Database username
POSTGRES_PASSWORD: "<SECRET>" # 🔐 Database password (from 1Password)
POSTGRES_DB: "postgres" # Database name
POSTGRES_HOST: "nuq-postgres" # K8s service name
POSTGRES_PORT: "5432" # PostgreSQL port
```
### Redis (Queue & Cache)
```yaml
REDIS_URL: "redis://redis:6379"
REDIS_RATE_LIMIT_URL: "redis://redis:6379"
```
### RabbitMQ (Message Broker)
```yaml
NUQ_RABBITMQ_URL: "amqp://rabbitmq:5672"
```
### Playwright Service
```yaml
PLAYWRIGHT_MICROSERVICE_URL: "http://playwright-service:3000/scrape"
```
### Authentication
```yaml
USE_DB_AUTHENTICATION: "false" # Set "true" for production with Supabase
```
---
## Security Variables (REQUIRED)
### Admin UI Protection
```yaml
BULL_AUTH_KEY: "<SECRET>" # 🔐 Queue admin UI password (from 1Password)
# URL: /admin/{BULL_AUTH_KEY}/queues
```
### API Testing
```yaml
TEST_API_KEY: "<SECRET>" # 🔐 Optional - API key for testing (from 1Password)
```
---
## Optional Variables (Features)
### AI Features (JSON Format, Extract API)
```yaml
# OpenAI Configuration
OPENAI_API_KEY: "<SECRET>" # 🔐 OpenAI API key (from 1Password)
OPENAI_BASE_URL: "" # Custom OpenAI-compatible endpoint
MODEL_NAME: "" # Override default model (e.g., gpt-4)
MODEL_EMBEDDING_NAME: "" # Override embedding model
# Ollama (Alternative to OpenAI)
OLLAMA_BASE_URL: "" # E.g., http://localhost:11434/api
# When using Ollama, set:
# MODEL_NAME: "deepseek-r1:7b"
# MODEL_EMBEDDING_NAME: "nomic-embed-text"
```
### Proxy Configuration
```yaml
PROXY_SERVER: "" # Full URL (http://0.1.2.3:1234) or IP:port
PROXY_USERNAME: "" # 🔐 Proxy username (from 1Password)
PROXY_PASSWORD: "" # 🔐 Proxy password (from 1Password)
```
### Search API Configuration
```yaml
# By default, uses Google search
# Optionally use SearXNG instead:
SEARXNG_ENDPOINT: "" # E.g., http://your.searxng.server
SEARXNG_ENGINES: "" # Comma-separated engine list
SEARXNG_CATEGORIES: "" # Comma-separated categories
```
### Monitoring & Logging
```yaml
LOGGING_LEVEL: "info" # debug, info, warn, error
SLACK_WEBHOOK_URL: "" # 🔐 Slack webhook for alerts (from 1Password)
```
### Supabase Integration (Advanced)
```yaml
# Note: Not currently configurable for self-hosted instances
SUPABASE_ANON_TOKEN: "" # For DB authentication
SUPABASE_URL: "" # Supabase project URL
SUPABASE_SERVICE_TOKEN: "" # Supabase service role key
```
---
## Performance Tuning
### Worker Pools
```yaml
NUM_WORKERS_PER_QUEUE: "8" # Number of workers per queue
CRAWL_CONCURRENT_REQUESTS: "10" # Concurrent crawl requests
MAX_CONCURRENT_JOBS: "5" # Maximum concurrent jobs
```
### Browser Pool (Playwright)
```yaml
BROWSER_POOL_SIZE: "5" # Browser instance pool size
MAX_CONCURRENT_PAGES: "10" # Max concurrent pages per browser
```
### Resource Limits (Self-Protection)
```yaml
MAX_CPU: "0.8" # 0.0-1.0, reject jobs above threshold
MAX_RAM: "0.8" # 0.0-1.0, reject jobs above threshold
```
### Timeouts
```yaml
HARNESS_STARTUP_TIMEOUT_MS: "60000" # 60 seconds startup timeout
```
---
## Playwright-Specific Variables
Set on `playwright-service` pods only:
```yaml
PORT: "3000" # Playwright service port
PROXY_SERVER: "" # Same as API proxy (if needed)
PROXY_USERNAME: "" # Same as API proxy (if needed)
PROXY_PASSWORD: "" # Same as API proxy (if needed)
ALLOW_LOCAL_WEBHOOKS: "false" # Security: block local webhooks
BLOCK_MEDIA: "" # Optional: block media resources
MAX_CONCURRENT_PAGES: "10" # Browser concurrency (same as API)
```
---
## Environment-Specific Variables
### Development
```yaml
ENV: "local"
```
### Production
```yaml
ENV: "production"
```
---
## FlyIO-Specific Variables (Not Used in K8s)
These variables are set by FlyIO platform and not needed for Kubernetes deployment:
```yaml
FLY_PROCESS_GROUP: "app" # Not used in K8s
```
---
## Deprecated / Unused Variables
Variables found in examples but not required for self-hosted deployment:
```yaml
AUTUMN_SECRET_KEY: "" # Mendable platform-specific
SELF_HOSTED_WEBHOOK_URL: "" # Custom webhook endpoint
LLAMAPARSE_API_KEY: "" # PDF parsing service (optional)
```
---
## Variable Precedence
1. **ExternalSecret** (from 1Password) - Highest priority for secrets
2. **ConfigMap** - Non-sensitive configuration
3. **Deployment env:** - Direct environment variables (override)
4. **Dockerfile defaults** - Lowest priority
---
## ConfigMap Example
Non-sensitive variables suitable for ConfigMap:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: firecrawl-config
namespace: firecrawl
data:
# Service URLs
REDIS_URL: "redis://redis:6379"
REDIS_RATE_LIMIT_URL: "redis://redis:6379"
POSTGRES_HOST: "nuq-postgres"
POSTGRES_PORT: "5432"
POSTGRES_USER: "postgres"
POSTGRES_DB: "postgres"
NUQ_RABBITMQ_URL: "amqp://rabbitmq:5672"
PLAYWRIGHT_MICROSERVICE_URL: "http://playwright-service:3000/scrape"
# Server Configuration
HOST: "0.0.0.0"
PORT: "3002"
WORKER_PORT: "3005"
EXTRACT_WORKER_PORT: "3004"
USE_DB_AUTHENTICATION: "false"
ENV: "production"
# Performance Tuning
NUM_WORKERS_PER_QUEUE: "8"
CRAWL_CONCURRENT_REQUESTS: "10"
MAX_CONCURRENT_JOBS: "5"
BROWSER_POOL_SIZE: "5"
MAX_CONCURRENT_PAGES: "10"
HARNESS_STARTUP_TIMEOUT_MS: "60000"
# Logging
LOGGING_LEVEL: "info"
# Security
ALLOW_LOCAL_WEBHOOKS: "false"
```
---
## ExternalSecret Example
Sensitive variables synced from 1Password:
```yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: firecrawl-secrets
namespace: firecrawl
spec:
refreshInterval: 1h
secretStoreRef:
name: onepassword-connect
kind: ClusterSecretStore
target:
name: firecrawl-secrets
creationPolicy: Owner
data:
# Required secrets
- secretKey: POSTGRES_PASSWORD
remoteRef:
key: firecrawl
property: postgres-password
- secretKey: BULL_AUTH_KEY
remoteRef:
key: firecrawl
property: bull-auth-key
# Optional secrets (uncomment when added to 1Password)
# - secretKey: OPENAI_API_KEY
# remoteRef:
# key: firecrawl
# property: openai-api-key
# - secretKey: TEST_API_KEY
# remoteRef:
# key: firecrawl
# property: test-api-key
```
---
## Usage in Deployments
### Combining ConfigMap and Secret
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
namespace: firecrawl
spec:
template:
spec:
containers:
- name: api
image: the-seas.local.mk-labs.cloud/applications/firecrawl-api:latest
envFrom:
# Load all non-sensitive variables
- configMapRef:
name: firecrawl-config
# Load all secrets
- secretRef:
name: firecrawl-secrets
env:
# Override specific variables if needed
- name: FLY_PROCESS_GROUP
value: "app"
```
### Playwright-Specific ConfigMap
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: playwright-service-config
namespace: firecrawl
data:
PORT: "3000"
ALLOW_LOCAL_WEBHOOKS: "false"
MAX_CONCURRENT_PAGES: "10"
```
---
## Validation Checklist
Before deployment, ensure:
- [ ] `POSTGRES_PASSWORD` set in 1Password
- [ ] `BULL_AUTH_KEY` set in 1Password (strong random value)
- [ ] All service URLs use correct K8s service names
- [ ] Ports match service definitions (3002, 3000, 5432, 6379, 5672)
- [ ] `USE_DB_AUTHENTICATION` is "false" (Supabase not available)
- [ ] `ENV` is "production" (not "local")
- [ ] Worker pool sizes appropriate for cluster capacity
- [ ] `ALLOW_LOCAL_WEBHOOKS` is "false" (security)
---
## Troubleshooting
### "Supabase client is not configured"
**Expected warning** - Supabase is not available for self-hosted instances. Safe to ignore.
### "You're bypassing authentication"
**Expected warning** - When `USE_DB_AUTHENTICATION=false`. Normal for self-hosted deployment.
### Connection refused errors
**Check:**
- Service names match environment variables
- Services are running: `kubectl get svc -n firecrawl`
- Pods are ready: `kubectl get pods -n firecrawl`
### Build failures
**Check:**
- Required build args are set
- Kaniko has sufficient memory (4GB for API build)
---
## Security Best Practices
1. **Generate strong secrets:**
```bash
# Generate BULL_AUTH_KEY
openssl rand -base64 32
# Generate POSTGRES_PASSWORD
openssl rand -base64 24
```
2. **Never expose PostgreSQL externally:**
- Use ClusterIP service only
- No LoadBalancer or NodePort
- Access via kubectl port-forward for maintenance
3. **Protect admin UI:**
- Keep BULL_AUTH_KEY secret
- Don't commit to git
- Rotate periodically
4. **Use ExternalSecrets:**
- Never put secrets in ConfigMaps
- Never commit secrets to git
- Store all sensitive data in 1Password
---
## References
- **Upstream .env Template:** `/tmp/firecrawl/SELF_HOST.md`
- **Docker Compose Reference:** `/tmp/firecrawl/docker-compose.yaml`
- **K8s Example ConfigMap:** `/tmp/firecrawl/examples/kubernetes/cluster-install/configmap.yaml`
---
**Last Updated:** June 6, 2026
**Maintained By:** Rocket Raccoon (CI/CD Specialist)

View File

@@ -0,0 +1,375 @@
# Firecrawl - Web Scraping & Crawling Service
**Project:** Platform Buildout - Firecrawl Deployment
**Service Name:** Spaceship Earth (EPCOT themed)
**DNS:** spaceship-earth.local.mk-labs.cloud (primary), firecrawl.local.mk-labs.cloud (alias)
**Namespace:** firecrawl
**Owner:** Rocket Raccoon (CI/CD Specialist)
**Status:** 🚧 IN PROGRESS - Day 1 Complete
---
## Overview
Firecrawl is a self-hosted web scraping and crawling API that converts URLs to LLM-ready content (Markdown, JSON, HTML). This deployment enables JARVIS web search capability.
**Upstream:** https://github.com/mendableai/firecrawl
**License:** AGPLv3 (open source)
### Capabilities
- **Search:** Search the web and get full page content
- **Scrape:** Convert URLs to markdown, HTML, screenshots, or structured JSON
- **Crawl:** Scrape all URLs of a website with a single request
- **Interact:** Click, scroll, write, wait before extracting (JS-heavy sites)
- **Map:** Discover all URLs on a website
---
## Architecture
### Services (5 components)
| Service | Image | CPU | Memory | Storage | Purpose |
|---------|-------|-----|--------|---------|---------|
| **API** | Harbor: firecrawl-api:latest | 2.0 | 4-6GB | - | Main REST API |
| **Worker** | Harbor: firecrawl-api:latest | 1.0 | 3-4GB | - | Background job processor |
| **NUQ Worker** | Harbor: firecrawl-api:latest | 1.0 | 3-4GB | - | Database queue worker |
| **Playwright** | Harbor: firecrawl-playwright:latest | 2.0 | 4GB | 1GB tmpfs | Browser automation |
| **PostgreSQL** | Harbor: firecrawl-postgres:latest | 1.0 | 2GB | 10GB PVC | Data storage |
| **Redis** | Upstream: redis:alpine | 0.5 | 1GB | - | Queue & cache |
| **RabbitMQ** | Upstream: rabbitmq:3-management | 0.5 | 1GB | - | Message broker |
**Total Resources:** ~8 CPU, ~22GB RAM, 11GB storage
### Service Dependencies
```
API Service ─┬─► Redis (queue/cache)
├─► PostgreSQL (data storage)
├─► RabbitMQ (message broker) ⚠️ HEALTH CHECK REQUIRED
└─► Playwright Service (browser automation)
Worker ──────┬─► Redis
├─► PostgreSQL
└─► RabbitMQ
NUQ Worker ──┴─► PostgreSQL
```
**Startup Order:**
1. Redis, PostgreSQL, RabbitMQ (infrastructure)
2. Playwright Service
3. API, Workers (after all dependencies ready)
---
## Build Strategy
**Hybrid Approach:** Build custom images via Tekton, use upstream for infrastructure.
### Custom Builds (Tekton → Harbor)
1. **firecrawl-api** (Multi-stage: Go + Node.js + Rust)
- Source: `apps/api/Dockerfile`
- Registry: `the-seas.local.mk-labs.cloud/applications/firecrawl-api:latest`
- Build time: ~15 minutes (first), ~5 minutes (cached)
- Used by: API, Worker, NUQ Worker (different entrypoints)
2. **firecrawl-playwright** (Node.js + Chromium)
- Source: `apps/playwright-service-ts/Dockerfile`
- Registry: `the-seas.local.mk-labs.cloud/applications/firecrawl-playwright:latest`
- Build time: ~10 minutes
3. **firecrawl-postgres** (PostgreSQL + pg_cron + init script)
- Source: `apps/nuq-postgres/Dockerfile`
- Registry: `the-seas.local.mk-labs.cloud/applications/firecrawl-postgres:latest`
- Build time: ~3 minutes
- Note: Custom build required for pg_cron extension and nuq.sql schema
### Upstream Images
- **Redis:** `redis:alpine`
- **RabbitMQ:** `rabbitmq:3-management`
---
## Configuration
### Required Secrets (1Password)
Managed via ExternalSecret → 1Password vault item: `firecrawl`
- `POSTGRES_PASSWORD` - Database password (CRITICAL)
- `BULL_AUTH_KEY` - Queue admin UI authentication (CRITICAL)
- `OPENAI_API_KEY` - Optional, for AI features
- `TEST_API_KEY` - Optional, for testing
### ConfigMap (Non-sensitive)
- Service URLs (Redis, PostgreSQL, RabbitMQ, Playwright)
- Port configuration (3002 API, 3005 Worker)
- Performance tuning (worker pools, concurrency limits)
- Logging level
### Environment Variables Reference
See [ENVIRONMENT_VARIABLES.md](./ENVIRONMENT_VARIABLES.md) for complete list.
---
## Deployment Timeline
### Day 1 (June 6) - Investigation & Planning ✅
- Repository analysis complete
- Architecture decisions finalized
- Environment configuration researched
- Day 2 plan created
### Day 2 (June 7) - Tekton Pipelines 🚧
- Create 3 build pipelines (API, Playwright, PostgreSQL)
- Test builds and push to Harbor
- Validate image integrity
### Day 3 (June 8) - Kubernetes Manifests
- Create Deployments, Services, StatefulSets
- Configure ConfigMaps and ExternalSecrets
- Set up HTTPRoute for ingress
### Day 4 (June 9) - Secrets & Configuration
- Create 1Password vault item
- Configure ExternalSecret sync
- Validate configuration
### Day 5 (June 10) - Deployment & Testing
- ArgoCD Application creation
- Deploy to cluster
- Service health validation
- API functionality testing
### Day 6 (June 11) - JARVIS Integration
- Configure JARVIS environment variables
- Test web search functionality
- End-to-end validation
- Documentation delivery
**Target Completion:** June 12, 2026
---
## Access & URLs
**Primary Access:**
- API: https://spaceship-earth.local.mk-labs.cloud
- Alias: https://firecrawl.local.mk-labs.cloud
- Queue UI: https://spaceship-earth.local.mk-labs.cloud/admin/[BULL_AUTH_KEY]/queues
**Health Endpoints:**
- Liveness: https://spaceship-earth.local.mk-labs.cloud/v0/health/liveness
- Readiness: https://spaceship-earth.local.mk-labs.cloud/v0/health/readiness
**Internal Services (cluster-only):**
- Playwright: http://playwright-service.firecrawl.svc:3000
- PostgreSQL: postgresql://nuq-postgres.firecrawl.svc:5432
- Redis: redis://redis.firecrawl.svc:6379
- RabbitMQ: amqp://rabbitmq.firecrawl.svc:5672
- RabbitMQ Mgmt: http://rabbitmq.firecrawl.svc:15672
---
## API Usage Examples
### Scrape a URL
```bash
curl -X POST https://spaceship-earth.local.mk-labs.cloud/v1/scrape \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com"
}'
```
### Search the Web
```bash
curl -X POST https://spaceship-earth.local.mk-labs.cloud/v1/search \
-H 'Content-Type: application/json' \
-d '{
"query": "kubernetes best practices"
}'
```
### Crawl a Website
```bash
curl -X POST https://spaceship-earth.local.mk-labs.cloud/v1/crawl \
-H 'Content-Type: application/json' \
-d '{
"url": "https://docs.example.com"
}'
```
---
## Directory Structure
```
cluster/applications/firecrawl/
├── README.md # This file
├── ENVIRONMENT_VARIABLES.md # Complete env var reference
├── namespace.yaml # Namespace definition
├── configmap.yaml # Non-sensitive configuration
├── externalsecret.yaml # 1Password secret sync
├── postgresql/
│ ├── statefulset.yaml # PostgreSQL StatefulSet
│ ├── service.yaml # PostgreSQL Service
│ └── pvc.yaml # Persistent Volume Claim
├── redis/
│ ├── deployment.yaml # Redis Deployment
│ └── service.yaml # Redis Service
├── rabbitmq/
│ ├── deployment.yaml # RabbitMQ Deployment
│ └── service.yaml # RabbitMQ Service
├── playwright/
│ ├── deployment.yaml # Playwright Deployment
│ └── service.yaml # Playwright Service
├── api/
│ ├── deployment.yaml # API Deployment
│ └── service.yaml # API Service
├── workers/
│ ├── worker-deployment.yaml # Queue Worker Deployment
│ └── nuq-worker-deployment.yaml # NUQ Worker Deployment
├── ingress/
│ └── httproute.yaml # Gateway API HTTPRoute
└── argocd/
└── application.yaml # ArgoCD Application manifest
```
---
## Tekton Pipelines
Build pipelines located in `cluster/tekton/pipelines/`:
- `firecrawl-api-build.yaml` - API service build
- `firecrawl-playwright-build.yaml` - Playwright service build
- `firecrawl-postgres-build.yaml` - PostgreSQL build
**Trigger Builds:**
```bash
# API build
kubectl create -f cluster/tekton/pipelines/firecrawl-api-build.yaml
# Playwright build
kubectl create -f cluster/tekton/pipelines/firecrawl-playwright-build.yaml
# PostgreSQL build
kubectl create -f cluster/tekton/pipelines/firecrawl-postgres-build.yaml
```
**Monitor Builds:**
```bash
# List pipeline runs
tkn pipelinerun list -n innoventions
# Watch logs
tkn pipelinerun logs -f <pipelinerun-name> -n innoventions
```
---
## Troubleshooting
### API Pod Not Starting
**Check:**
1. RabbitMQ health status (API depends on healthy RabbitMQ)
2. Environment variables (ConfigMap and Secret)
3. Database connectivity (PostgreSQL)
**Commands:**
```bash
kubectl logs -n firecrawl deployment/api
kubectl describe pod -n firecrawl -l app=api
kubectl get externalsecret -n firecrawl
```
### Build Failures
**Check:**
1. Harbor connectivity
2. Harbor credentials secret
3. Build resource limits (increase if OOM)
**Commands:**
```bash
tkn pipelinerun describe <name> -n innoventions
kubectl logs -n innoventions <kaniko-pod>
```
### Database Connection Errors
**Check:**
1. PostgreSQL pod status
2. PVC binding
3. Init script execution
**Commands:**
```bash
kubectl logs -n firecrawl statefulset/nuq-postgres
kubectl exec -it -n firecrawl nuq-postgres-0 -- psql -U postgres -d postgres -c '\d nuq.queue_scrape'
```
---
## Monitoring
**Resource Usage:**
```bash
kubectl top pods -n firecrawl
```
**Service Health:**
```bash
# API liveness
curl -k https://spaceship-earth.local.mk-labs.cloud/v0/health/liveness
# API readiness
curl -k https://spaceship-earth.local.mk-labs.cloud/v0/health/readiness
# Playwright health
kubectl exec -n firecrawl deployment/playwright-service -- curl localhost:3000/health
```
**Queue Status:**
Navigate to: https://spaceship-earth.local.mk-labs.cloud/admin/[BULL_AUTH_KEY]/queues
---
## Security Notes
1. **PostgreSQL Credentials:** Stored in 1Password, synced via ExternalSecret
2. **Admin UI:** Protected by BULL_AUTH_KEY (in URL path)
3. **Database Port:** NOT exposed outside cluster (ClusterIP only)
4. **TLS:** All external traffic encrypted via cert-manager certificates
5. **RBAC:** Service accounts scoped to firecrawl namespace
---
## References
- **Upstream Docs:** https://docs.firecrawl.dev
- **GitHub:** https://github.com/mendableai/firecrawl
- **Self-Hosting Guide:** https://github.com/mendableai/firecrawl/blob/main/SELF_HOST.md
- **K8s Examples:** `/tmp/firecrawl/examples/kubernetes/`
- **Mission Brief:** `~/friday/inbox/agents/rocket/FIRECRAWL-DEPLOYMENT-MISSION-BRIEF.md`
---
## Changelog
### 2026-06-06 - Day 1 Complete
- Initial repository structure created
- Architecture decisions finalized
- Build strategy documented
- Environment variables researched
- Ready for Day 2 (pipeline creation)
---
**Contact:** Rocket Raccoon (CI/CD Specialist)
**Project Manager:** Pepper Potts
**Cluster:** fastpass (Talos Kubernetes)
**Last Updated:** June 6, 2026

View File

@@ -0,0 +1,103 @@
# ------------------------------------------------------------------------------
# democratic-csi Helm Values for Pure FlashArray (utilidor)
#
# Backend: Pure Storage FlashArray via freenas-iscsi driver
# Target: utilidor.local.mk-labs.cloud (10.1.71.5)
#
# Note: The freenas-iscsi driver works with Pure API v2+
# Pure FlashArray exposes an API compatible with FreeNAS/TrueNAS
# ------------------------------------------------------------------------------
csiDriver:
name: "org.democratic-csi.iscsi-pure"
storageClasses:
- name: pure-iscsi-block
defaultClass: false
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
parameters:
fsType: ext4
mountOptions: []
secrets:
provisioner-secret:
controller-publish-secret:
node-stage-secret:
node-publish-secret:
controller-expand-secret:
driver:
config:
driver: freenas-iscsi
instance_id: mk-labs-utilidor
httpConnection:
protocol: https
host: utilidor.local.mk-labs.cloud
port: 443
# Pure API token from pure-exporter secret
apiKey: "6157dea0-4c0d-b14b-6e6a-453aa649355d"
allowInsecure: true
apiVersion: 2
zfs:
# Pure volumes are created in a pod/volume group context
# This maps to Pure's volume naming scheme
datasetParentName: csi-volumes
detachedSnapshotsDatasetParentName: csi-snapshots
datasetEnableQuotas: false
datasetEnableReservation: false
datasetPermissionsMode: "0777"
datasetPermissionsUser: 0
datasetPermissionsGroup: 0
iscsi:
targetPortal: "10.1.71.5:3260"
targetPortals: []
# Pure FlashArray iSCSI configuration
interface: default
namePrefix: "csi-"
nameSuffix: ""
targetGroups:
# Pure uses host groups for iSCSI targets
# Leave empty to auto-select
extentCommentTemplate: "Kubernetes CSI volume {{ parameters.[csi.storage.k8s.io/pvc/namespace] }}/{{ parameters.[csi.storage.k8s.io/pvc/name] }}"
extentInsecureTpc: true
extentXenCompat: false
extentDisablePhysicalBlocksize: true
extentBlocksize: 512
extentRpm: "SSD"
extentAvailThreshold: 0
controller:
enabled: true
replicaCount: 3
strategy: deployment
priorityClassName: system-cluster-critical
externalAttacher:
enabled: true
externalProvisioner:
enabled: true
externalResizer:
enabled: true
externalSnapshotter:
enabled: false
node:
enabled: true
rbac:
enabled: true
driver:
logLevel: info
# Talos requires hostNetwork for iSCSI operations
hostNetwork: true
hostIPC: true
dnsPolicy: ClusterFirstWithHostNet

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
{"nodes": [{"id": "home_hermes_git_homelab_packer_fedora_42_scripts_cleanup_sh", "label": "cleanup.sh", "file_type": "code", "source_file": "packer/fedora-42/scripts/cleanup.sh", "source_location": "L1", "metadata": {"language": "bash", "kind": "file"}}, {"id": "home_hermes_git_homelab_packer_fedora_42_scripts_cleanup_sh__entry", "label": "cleanup.sh script", "file_type": "code", "source_file": "packer/fedora-42/scripts/cleanup.sh", "source_location": "L1", "metadata": {"language": "bash", "kind": "bash_entrypoint"}}], "edges": [{"source": "home_hermes_git_homelab_packer_fedora_42_scripts_cleanup_sh", "target": "home_hermes_git_homelab_packer_fedora_42_scripts_cleanup_sh__entry", "relation": "contains", "confidence": "EXTRACTED", "source_file": "packer/fedora-42/scripts/cleanup.sh", "source_location": "L1", "weight": 1.0}]}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
{"nodes": [{"id": "home_hermes_git_homelab_ansible_scripts_migrate_dns_references_sh", "label": "migrate-dns-references.sh", "file_type": "code", "source_file": "ansible/scripts/migrate-dns-references.sh", "source_location": "L1", "metadata": {"language": "bash", "kind": "file"}}, {"id": "home_hermes_git_homelab_ansible_scripts_migrate_dns_references_sh__entry", "label": "migrate-dns-references.sh script", "file_type": "code", "source_file": "ansible/scripts/migrate-dns-references.sh", "source_location": "L1", "metadata": {"language": "bash", "kind": "bash_entrypoint"}}, {"id": "scripts_migrate_dns_references_print_status", "label": "print_status()", "file_type": "code", "source_file": "ansible/scripts/migrate-dns-references.sh", "source_location": "L15", "metadata": {"language": "bash", "kind": "bash_function"}}], "edges": [{"source": "home_hermes_git_homelab_ansible_scripts_migrate_dns_references_sh", "target": "home_hermes_git_homelab_ansible_scripts_migrate_dns_references_sh__entry", "relation": "contains", "confidence": "EXTRACTED", "source_file": "ansible/scripts/migrate-dns-references.sh", "source_location": "L1", "weight": 1.0}, {"source": "home_hermes_git_homelab_ansible_scripts_migrate_dns_references_sh", "target": "scripts_migrate_dns_references_print_status", "relation": "defines", "confidence": "EXTRACTED", "source_file": "ansible/scripts/migrate-dns-references.sh", "source_location": "L15", "weight": 1.0}, {"source": "home_hermes_git_homelab_ansible_scripts_migrate_dns_references_sh__entry", "target": "scripts_migrate_dns_references_print_status", "relation": "calls", "confidence": "EXTRACTED", "source_file": "ansible/scripts/migrate-dns-references.sh", "source_location": "L34", "weight": 1.0, "context": "call"}]}

View File

@@ -0,0 +1 @@
{"nodes": [{"id": "home_hermes_git_homelab_packer_ubuntu_24_04_scripts_cleanup_sh", "label": "cleanup.sh", "file_type": "code", "source_file": "packer/ubuntu-24.04/scripts/cleanup.sh", "source_location": "L1", "metadata": {"language": "bash", "kind": "file"}}, {"id": "home_hermes_git_homelab_packer_ubuntu_24_04_scripts_cleanup_sh__entry", "label": "cleanup.sh script", "file_type": "code", "source_file": "packer/ubuntu-24.04/scripts/cleanup.sh", "source_location": "L1", "metadata": {"language": "bash", "kind": "bash_entrypoint"}}], "edges": [{"source": "home_hermes_git_homelab_packer_ubuntu_24_04_scripts_cleanup_sh", "target": "home_hermes_git_homelab_packer_ubuntu_24_04_scripts_cleanup_sh__entry", "relation": "contains", "confidence": "EXTRACTED", "source_file": "packer/ubuntu-24.04/scripts/cleanup.sh", "source_location": "L1", "weight": 1.0}]}

File diff suppressed because one or more lines are too long

1
graphify-out/cache/stat-index.json vendored Normal file

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,56 @@
#!/bin/bash
# verify-multipath-config.sh
# Verify that multipath.conf is properly configured on Talos worker nodes
# Run from city-hall after applying the fixed configuration
set -e
NODE="${1:-10.1.71.69}" # Default to jungle-cruise
echo "=== Verifying multipath configuration on node $NODE ==="
echo
echo "1. Checking ext-multipathd service status..."
talosctl services --nodes "$NODE" | grep -E "(STATE|multipathd)" || echo " ⚠️ Service not found"
echo
echo "2. Checking if /etc/multipath.conf exists in ext-multipathd container..."
if talosctl exec --nodes "$NODE" --service ext-multipathd -- cat /etc/multipath.conf &>/dev/null; then
echo " ✅ File exists! Contents:"
talosctl exec --nodes "$NODE" --service ext-multipathd -- cat /etc/multipath.conf | head -20
echo " ... (truncated)"
else
echo " ❌ File does NOT exist"
echo " This means the ExtensionServiceConfig is still not being applied."
exit 1
fi
echo
echo "3. Checking multipath-tools extension..."
talosctl get extensions --nodes "$NODE" | grep -E "(NAME|multipath)" || echo " ⚠️ Extension not found"
echo
echo "4. Checking for multipath devices..."
echo " Running: multipath -ll"
if talosctl exec --nodes "$NODE" -- multipath -ll 2>/dev/null; then
echo " ✅ Multipath devices detected"
else
echo " ⚠️ No multipath devices (may be normal if iSCSI not connected yet)"
fi
echo
echo "5. Checking ext-multipathd logs (last 20 lines)..."
talosctl logs --nodes "$NODE" --service ext-multipathd --tail 20
echo
echo "=== Verification complete ==="
echo
echo "Expected results:"
echo " ✅ ext-multipathd service: Running"
echo " ✅ /etc/multipath.conf: Exists with Pure FlashArray config"
echo " ✅ multipath-tools extension: Loaded"
echo " ✅ multipath -ll: Shows Pure devices (if iSCSI connected)"
echo
echo "If /etc/multipath.conf does NOT exist:"
echo " - Verify talhelper genconfig was run"
echo " - Check clusterconfig/fastpass-jungle-cruise.yaml for 'kind: ExtensionServiceConfig'"
echo " - Ensure it's NOT inside quotes (should be structured YAML, not a string)"