From 8a960ffc73cf9edbfe78adaacbb38666822b3c80 Mon Sep 17 00:00:00 2001 From: Andrew Stoltz Date: Thu, 23 Apr 2026 15:59:50 -0500 Subject: [PATCH] feat(fc-distribution): K8s manifest for Phase 1 edition publisher MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds apps/fc-distribution/{fc-distribution.yaml,kustomization.yaml,README.md}. Ships the FlowerCore.Distribution service (Blazor + REST + MCP) backed by Synology NFS for SQLite catalog + content-addressed blob root. Contents: - Namespace fc-distribution - 3x OnePasswordItem (FlowerCore Code Signing CA informational + per-edition signing keys for kiosk-standard and aistation-field) - Deployment: localhost/fc-distribution:v202604232000 (already imported to rke2-server via ctr), pinned to rke2-server nodeSelector because Synology NFS ACL restricts writes to that node, emptyDir for /tmp + /app/logs, inline NFS for /data (subPath distribution/data) and /blobs (subPath distribution/blobs), Secret volume mounts for /signing/. readOnlyRootFilesystem + runAsUser 1654 + drop ALL capabilities. Probes: startup + readiness on /healthz, liveness on tcpSocket (defense against future auth middleware accidentally gating /healthz). - Service (ClusterIP :80 -> container :8080) - Certificate (cert-manager ClusterIssuer step-ca-acme, dist.iamworkin.lan, 90d / 30d renew). pfSense Unbound override dist.iamworkin.lan -> 10.0.56.200 already in place (req'd for HTTP-01). - IngressRoute (Traefik websecure, Host rule on dist.iamworkin.lan) Env var keys align with the scaffold: FlowerCore__Database__ConnectionStrings__Sqlite FlowerCore__Distribution__Blobs__Root FlowerCore__Distribution__Signing__EditionCerts____{CertPath,KeyPath} Consumer: ProvisioningAgent (USB-side, Phase 2) — see FlowerCore.Notes/docs/infrastructure/usb-provisioning-architecture.md. Co-Authored-By: Claude Opus 4.7 (1M context) --- apps/fc-distribution/README.md | 91 +++++++ apps/fc-distribution/fc-distribution.yaml | 283 ++++++++++++++++++++++ apps/fc-distribution/kustomization.yaml | 9 + 3 files changed, 383 insertions(+) create mode 100644 apps/fc-distribution/README.md create mode 100644 apps/fc-distribution/fc-distribution.yaml create mode 100644 apps/fc-distribution/kustomization.yaml diff --git a/apps/fc-distribution/README.md b/apps/fc-distribution/README.md new file mode 100644 index 0000000..9d946f5 --- /dev/null +++ b/apps/fc-distribution/README.md @@ -0,0 +1,91 @@ +# fc-distribution — staged deployment (Phase 1, USB provisioning) + +**Status:** manifests staged, **NOT YET APPLIED**. Image must be built + +imported and signing 1Password items confirmed before `git push`. + +- Architecture: [`../../../FlowerCore.Notes/docs/infrastructure/usb-provisioning-architecture.md`](../../../FlowerCore.Notes/docs/infrastructure/usb-provisioning-architecture.md) +- Repo: `D:\git\FlowerCore\FlowerCore.Distribution\` (`README.md`, `CLAUDE.md`) +- Shared lib: `FlowerCore.Common` -> `FlowerCore.Shared.Distribution` + +`FlowerCore.Distribution` publishes signed edition manifests (ECDSA P-256 +over canonical JSON) and serves the SHA-256 content-addressed blob store +that USB builders pull from. The verifier embeds the `IAmWorkin ACME CA +Root CA` as the trust anchor; per-edition leaf signing material lives in +1Password and is mounted into the pod read-only. + +## Deployment order (do NOT skip / reorder) + +### 1. pfSense Unbound DNS — DONE 2026-04-23 + +`dist.iamworkin.lan -> 10.0.56.200` was added to pfSense Unbound out of band. +Verify before push: + +```bash +nslookup dist.iamworkin.lan 10.0.56.1 # expect 10.0.56.200 +python bluejay-infra/scripts/check-pfsense-dns.py +``` + +If this is missing, cert-manager HTTP-01 will silently back off ~2h. See +memory `feedback_pfsense_dns_required_for_acme.md`. + +### 2. 1Password items required in vault `IAmWorkin` + +| Item title | Item id | Used as | +|---|---|---| +| `FlowerCore Code Signing CA` | (existing) | Informational handle only — root CA is baked into the image at build time, not mounted | +| `FlowerCore Edition Signing Key - edition:kiosk-standard` | `3hf33egdvnni6jyuws3r737mqe` | Mounted at `/signing/kiosk-standard/` | +| `FlowerCore Edition Signing Key - edition:aistation-field` | `ccxrtsan5samfq4pfuczymacrq` | Mounted at `/signing/aistation-field/` | + +Each edition item must publish three field labels (the operator turns +field labels into Secret keys verbatim): + +- `certificate.pem` — leaf certificate +- `private-key.pem` — ECDSA P-256 private key +- `chain.pem` — leaf + intermediate (referenced by the env var as the + cert-path; the verifier uses this for signature path validation) + +### 3. Build + import the image to rke2-server + +The Pod is pinned to `rke2-server` because the Synology NFS export +`/volume1/kubernetes` only allows that node. Importing to the agents is +optional until the ACL is widened. + +```bash +# From BLUEJAY-WS, in D:\git\FlowerCore\FlowerCore.Distribution +TAG="v$(date +%Y%m%d%H%M)" +dotnet.exe publish -c Release -o deploy/app \ + src/FlowerCore.Distribution.Web/FlowerCore.Distribution.Web.csproj +podman build -t localhost/fc-distribution:$TAG -f deploy/Dockerfile.deploy deploy +podman save localhost/fc-distribution:$TAG -o /tmp/fc-distribution.tar +scp /tmp/fc-distribution.tar rke2-server:/tmp/ +ssh rke2-server "sudo /var/lib/rancher/rke2/bin/ctr -a /run/k3s/containerd/containerd.sock -n k8s.io images import /tmp/fc-distribution.tar" +``` + +### 4. Bump the image tag + push + +Edit `fc-distribution.yaml`, replace `localhost/fc-distribution:v202604231530` +with the tag from step 3, then: + +```bash +cd D:/git/FlowerCore/bluejay-infra +python scripts/check-pfsense-dns.py +git add apps/fc-distribution/ +git commit -m "feat(fc-distribution): deploy Phase 1 manifest publisher" +git push +``` + +ArgoCD picks up within ~3 minutes and creates `infra-fc-distribution`. + +### 5. Verify + +```bash +fcadmin_ssh noc1 ' + kubectl -n argocd get application infra-fc-distribution + kubectl -n fc-distribution get certificate,pod,secret + curl -sk -m 8 -o /dev/null -w "HTTP %{http_code}\n" https://dist.iamworkin.lan/healthz +' +``` + +Expect: Certificate `Ready: True` within ~60s, `/healthz` HTTP 200, both +`edition-kiosk-standard` and `edition-aistation-field` Secrets present +with `certificate.pem`, `private-key.pem`, `chain.pem` keys. diff --git a/apps/fc-distribution/fc-distribution.yaml b/apps/fc-distribution/fc-distribution.yaml new file mode 100644 index 0000000..895d5c0 --- /dev/null +++ b/apps/fc-distribution/fc-distribution.yaml @@ -0,0 +1,283 @@ +# FlowerCore.Distribution — edition manifest publisher + content-addressed blob store. +# Phase 1 of the USB provisioning architecture: signed edition manifests +# (ECDSA P-256 over canonical JSON) published per edition, plus a SHA-256 +# content-addressed blob store that USB builders pull from. +# +# Architecture: FlowerCore.Notes/docs/infrastructure/usb-provisioning-architecture.md +# Repo: FlowerCore.Distribution/{README.md,CLAUDE.md} +# Shared lib: FlowerCore.Common -> FlowerCore.Shared.Distribution +# (manifest schema, canonical JSON, ECDSA P-256 sign/verify) +# +# Deployment order (see bluejay-infra/README.md and apps/fc-distribution/README.md): +# 1. pfSense Unbound DNS override for dist.iamworkin.lan -> 10.0.56.200 +# (DONE 2026-04-23 — verify with `python bluejay-infra/scripts/check-pfsense-dns.py`). +# 2. 1Password items must exist in vault `IAmWorkin`: +# - `FlowerCore Code Signing CA` (informational) +# - `FlowerCore Edition Signing Key - edition:kiosk-standard` (3hf33egdvnni6jyuws3r737mqe) +# - `FlowerCore Edition Signing Key - edition:aistation-field` (ccxrtsan5samfq4pfuczymacrq) +# Each edition item is expected to publish three field labels: +# certificate.pem, private-key.pem, chain.pem +# 3. Synology NFS export `/volume1/kubernetes` is currently restricted to +# rke2-server (10.0.56.11). Pod is pinned via nodeSelector below. The +# app writes to subPaths `distribution/data` and `distribution/blobs`. +# 4. Build + import image: localhost/fc-distribution:v +# Import to rke2-server via `ctr images import` (NFS-pinned, no need +# for the agents until ACL is widened — see guacamole pattern). +# 5. Bump the image tag below and git push; ArgoCD ApplicationSet picks up +# within ~3 minutes and creates `infra-fc-distribution`. +# +# NOTE on the root trust anchor: +# The verifier needs an embedded root CA (`IAmWorkin ACME CA Root CA`). +# That root is shipped INSIDE the published image (Phase 2 build step +# bakes it into the bundle), NOT mounted from a Secret here. The +# `codesigning-root-cert` OnePasswordItem below is informational only — +# it gives operators a quick handle to the CA item from the cluster. +--- +apiVersion: v1 +kind: Namespace +metadata: + name: fc-distribution + labels: + app.kubernetes.io/part-of: flowercore +--- +# Informational handle to the FlowerCore Code Signing CA item in 1Password. +# Not consumed by the pod at runtime — the root trust anchor is baked into +# the published image. Operators can `kubectl -n fc-distribution get secret +# codesigning-root-cert` to discover the CA item URL/admin handle. +apiVersion: onepassword.com/v1 +kind: OnePasswordItem +metadata: + name: codesigning-root-cert + namespace: fc-distribution +spec: + itemPath: "vaults/IAmWorkin/items/FlowerCore Code Signing CA" +--- +# Edition signing key + leaf cert + chain for edition:kiosk-standard. +# 1Password item id: 3hf33egdvnni6jyuws3r737mqe +# Operator syncs each field to a Secret key of the same name. Mounted +# read-only at /signing/kiosk-standard inside the pod. +apiVersion: onepassword.com/v1 +kind: OnePasswordItem +metadata: + name: edition-kiosk-standard + namespace: fc-distribution +spec: + itemPath: "vaults/IAmWorkin/items/FlowerCore Edition Signing Key - edition:kiosk-standard" +--- +# Edition signing key + leaf cert + chain for edition:aistation-field. +# 1Password item id: ccxrtsan5samfq4pfuczymacrq +apiVersion: onepassword.com/v1 +kind: OnePasswordItem +metadata: + name: edition-aistation-field + namespace: fc-distribution +spec: + itemPath: "vaults/IAmWorkin/items/FlowerCore Edition Signing Key - edition:aistation-field" +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: fc-distribution + namespace: fc-distribution + labels: + app.kubernetes.io/name: fc-distribution + app.kubernetes.io/part-of: flowercore +spec: + replicas: 1 + revisionHistoryLimit: 3 + strategy: + # NFS-backed SQLite + blob store on a single node. Recreate avoids any + # multi-attach overlap on the same NFS subPath during rollout. + type: Recreate + selector: + matchLabels: + app.kubernetes.io/name: fc-distribution + template: + metadata: + labels: + app.kubernetes.io/name: fc-distribution + app.kubernetes.io/part-of: flowercore + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "8080" + prometheus.io/path: "/metrics" + spec: + # Synology NFS export `/volume1/kubernetes` ACL only allows rke2-server + # (10.0.56.11) right now. Until the ACL is widened in DSM (admin only), + # this Pod must run on rke2-server or NFS mounts will be access-denied. + nodeSelector: + kubernetes.io/hostname: rke2-server + securityContext: + runAsNonRoot: true + fsGroup: 1654 + fsGroupChangePolicy: OnRootMismatch + containers: + - name: web + # Placeholder tag — bump to the image you built + imported to + # rke2-server before applying. Build with: + # dotnet.exe publish -c Release -o deploy/app \ + # src/FlowerCore.Distribution.Web/FlowerCore.Distribution.Web.csproj + # podman build -t localhost/fc-distribution:v -f deploy/Dockerfile.deploy deploy + image: localhost/fc-distribution:v202604232000 + imagePullPolicy: Never + ports: + - containerPort: 8080 + name: http + env: + - name: ASPNETCORE_URLS + value: "http://+:8080" + - name: ASPNETCORE_ENVIRONMENT + value: "Production" + - name: DOTNET_SYSTEM_GLOBALIZATION_INVARIANT + value: "false" + # SQLite connection (catalog + data-protection keys via FlowerCoreDbContext). + # Read by Data/DatabaseProviderExtensions.cs in precedence order; Sqlite key wins. + - name: FlowerCore__Database__Provider + value: "Sqlite" + - name: FlowerCore__Database__ConnectionStrings__Sqlite + value: "Data Source=/data/distribution.db" + # Content-addressed blob root (SHA-256 sharded on disk). + # Bound by Services/NfsPvcBlobProvider.cs under FlowerCore:Distribution:Blobs. + - name: FlowerCore__Distribution__Blobs__Root + value: "/blobs" + # Per-edition signing material — paths into the read-only + # secret mounts below. Field labels in 1Password (and therefore + # Secret key names) are: certificate.pem, private-key.pem, chain.pem + - name: FlowerCore__Distribution__Signing__EditionCerts__kiosk-standard__CertPath + value: "/signing/kiosk-standard/chain.pem" + - name: FlowerCore__Distribution__Signing__EditionCerts__kiosk-standard__KeyPath + value: "/signing/kiosk-standard/private-key.pem" + - name: FlowerCore__Distribution__Signing__EditionCerts__aistation-field__CertPath + value: "/signing/aistation-field/chain.pem" + - name: FlowerCore__Distribution__Signing__EditionCerts__aistation-field__KeyPath + value: "/signing/aistation-field/private-key.pem" + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + cpu: 500m + memory: 512Mi + # /healthz is exposed by the scaffold (StartupGateMiddleware-aware). + # Liveness uses tcpSocket as a cheap fallback in case a future + # middleware change accidentally gates /healthz behind auth + # (memory: feedback_k8s_probes_behind_auth_middleware). + startupProbe: + httpGet: + path: /healthz + port: 8080 + initialDelaySeconds: 5 + periodSeconds: 5 + failureThreshold: 30 + readinessProbe: + httpGet: + path: /healthz + port: 8080 + periodSeconds: 10 + failureThreshold: 3 + livenessProbe: + tcpSocket: + port: 8080 + initialDelaySeconds: 30 + periodSeconds: 30 + failureThreshold: 3 + securityContext: + runAsNonRoot: true + runAsUser: 1654 + runAsGroup: 1654 + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + capabilities: + drop: + - ALL + volumeMounts: + - name: sqlite + mountPath: /data + subPath: distribution/data + - name: blobs + mountPath: /blobs + subPath: distribution/blobs + - name: tmp + mountPath: /tmp + - name: logs + mountPath: /app/logs + - name: kiosk-standard + mountPath: /signing/kiosk-standard + readOnly: true + - name: aistation-field + mountPath: /signing/aistation-field + readOnly: true + volumes: + # Synology NFS at /volume1/kubernetes — same export pattern as + # apps/guacamole/guacamole.yaml (recordings volume). Pinned by + # ACL to rke2-server. Never mount the subpath as nfs.path — + # always mount the export root and use volumeMount.subPath. + - name: sqlite + nfs: + server: 10.0.58.3 + path: /volume1/kubernetes + - name: blobs + nfs: + server: 10.0.58.3 + path: /volume1/kubernetes + - name: tmp + emptyDir: {} + - name: logs + emptyDir: {} + - name: kiosk-standard + secret: + secretName: edition-kiosk-standard + defaultMode: 0400 + - name: aistation-field + secret: + secretName: edition-aistation-field + defaultMode: 0400 +--- +apiVersion: v1 +kind: Service +metadata: + name: fc-distribution + namespace: fc-distribution + labels: + app.kubernetes.io/name: fc-distribution + app.kubernetes.io/part-of: flowercore +spec: + type: ClusterIP + selector: + app.kubernetes.io/name: fc-distribution + ports: + - name: http + port: 80 + targetPort: 8080 +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: fc-distribution-tls + namespace: fc-distribution +spec: + secretName: fc-distribution-tls-secret + issuerRef: + name: step-ca-acme + kind: ClusterIssuer + dnsNames: + - dist.iamworkin.lan + duration: 2160h # 90d + renewBefore: 720h # 30d +--- +apiVersion: traefik.io/v1alpha1 +kind: IngressRoute +metadata: + name: fc-distribution + namespace: fc-distribution +spec: + entryPoints: + - websecure + routes: + - match: Host(`dist.iamworkin.lan`) + kind: Rule + services: + - name: fc-distribution + port: 80 + tls: + secretName: fc-distribution-tls-secret diff --git a/apps/fc-distribution/kustomization.yaml b/apps/fc-distribution/kustomization.yaml new file mode 100644 index 0000000..e32b4f2 --- /dev/null +++ b/apps/fc-distribution/kustomization.yaml @@ -0,0 +1,9 @@ +# ArgoCD's bluejay-infra ApplicationSet uses a directory generator and does +# not require kustomization.yaml (existing apps like fc-llm-bridge and +# guacamole have none). This file is included anyway as a single source of +# truth for the resource list and to make `kubectl kustomize` previews work +# from a working copy. +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - fc-distribution.yaml