runners: add github-runner-updater Deployment

FlowerCore.Updater had only the offline bluejay-ws-sandbox-1 Windows runner registered (the specialized fcsetup E2E target) and no Linux self-hosted runner, leaving the repo with no Linux PR-CI capacity for any future workflow. Modeled on github-runner-pimanager (Sprint 32 long-tail final entry, 2026-05-25); two replicas with per-pod emptyDir caches to keep ReadWriteOnce PVC contention out of the picture. Also registers github-runner-updater in the LinuxRunnerRepos + ScaledLinuxRunnerDeployments fleet-lint sets so future suite repairs treat the entry as canonically required (the 6 pre-existing lint failures on this file family are orthogonal: initContainer single- container count assertion + fc-updater IngressRoute POST allowlist + DM ApplicationSet convention drift). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
runners: add github-runner-pimanager Deployment
2026-05-25 22:22:41 -05:00 · 2026-05-25 20:33:44 -05:00 · 2026-05-26 01:12:15 +00:00 · 2026-05-25 20:11:41 -05:00 · 2026-05-26 01:04:14 +00:00
3 changed files with 291 additions and 6 deletions
--- a/apps/github-runner/github-runner.yaml
+++ b/apps/github-runner/github-runner.yaml
@@ -3897,9 +3897,277 @@ spec:
        - name: tmp
          emptyDir: {}
      restartPolicy: Always
+---
+# Runner for FlowerCore.PiManager. Two replicas use per-pod emptyDir caches, so
+# backlog can drain without sharing a ReadWriteOnce PVC. Added 2026-05-25 to
+# close the runner-fleet gap that left run 26417714843 queued for 5h.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: github-runner-pimanager
+  namespace: github-runner
+  labels:
+    app.kubernetes.io/name: github-runner-pimanager
+    app.kubernetes.io/component: runner
+    app.kubernetes.io/part-of: flowercore
+    app.kubernetes.io/managed-by: argocd
+    flowercore.io/created-by: argocd
+    flowercore.io/runner-repo: pimanager
+    flowercore.io/github-repo: FlowerCore.PiManager
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: github-runner-pimanager
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: github-runner-pimanager
+        app.kubernetes.io/component: runner
+        app.kubernetes.io/part-of: flowercore
+        flowercore.io/created-by: argocd
+        flowercore.io/runner-repo: pimanager
+        flowercore.io/github-repo: FlowerCore.PiManager
+    spec:
+      serviceAccountName: github-runner
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1001
+        runAsGroup: 1001
+        fsGroup: 1001
+      initContainers:
+        - name: setup-runner-home
+          image: localhost/fc-github-runner:v20260525-ruby3.3.11-stepca
+          imagePullPolicy: Never
+          command:
+            - sh
+            - -c
+            - |
+              set -e
+              mkdir -p /home/runner/.dotnet /home/runner/.nuget/packages /home/runner/.nuget/NuGet /home/runner/.cache /home/runner/_tool
+              if [ -d /opt/runner-toolcache/Ruby ] && [ ! -d /home/runner/_tool/Ruby ]; then
+                cp -a /opt/runner-toolcache/Ruby /home/runner/_tool/
+              fi
+              chown -R 1001:1001 /home/runner/.dotnet /home/runner/.nuget /home/runner/.cache /home/runner/_tool
+              chmod -R 755 /home/runner/.dotnet /home/runner/.nuget /home/runner/.cache /home/runner/_tool
+          securityContext:
+            runAsUser: 0
+            runAsNonRoot: false
+          volumeMounts:
+            - name: runner-home
+              mountPath: /home/runner
+      containers:
+        - name: runner
+          image: localhost/fc-github-runner:v20260525-ruby3.3.11-stepca
+          imagePullPolicy: Never
+          env:
+            - name: REPO_URL
+              value: "https://github.com/astoltz/FlowerCore.PiManager"
+            - name: RUNNER_NAME_PREFIX
+              value: "rke2-linux-pimanager"
+            - name: RUNNER_WORKDIR
+              value: "/tmp/runner/work"
+            - name: EPHEMERAL
+              value: "true"
+            - name: LABELS
+              value: "self-hosted,linux,fc-build-linux"
+            - name: HOME
+              value: "/home/runner"
+            - name: DOTNET_INSTALL_DIR
+              value: "/home/runner/.dotnet"
+            - name: DOTNET_CLI_TELEMETRY_OPTOUT
+              value: "1"
+            - name: DOTNET_NOLOGO
+              value: "1"
+            - name: DOTNET_GENERATE_ASPNET_CERTIFICATE
+              value: "false"
+            - name: DOTNET_CLI_HOME
+              value: "/home/runner"
+            - name: NUGET_PACKAGES
+              value: "/home/runner/.nuget/packages"
+            - name: XDG_CACHE_HOME
+              value: "/home/runner/.cache"
+            - name: RUNNER_TOOL_CACHE
+              value: "/home/runner/_tool"
+            - name: ACCESS_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: github-runner-token
+                  key: credential
+            - name: RUN_AS_ROOT
+              value: "false"
+          resources:
+            requests:
+              cpu: "500m"
+              memory: "1Gi"
+            limits:
+              cpu: "2000m"
+              memory: "4Gi"
+          volumeMounts:
+            - name: runner-home
+              mountPath: /home/runner
+            - name: nuget-cache
+              mountPath: /home/runner/.nuget/packages
+            - name: tmp
+              mountPath: /tmp
+          livenessProbe:
+            exec:
+              command:
+                - /bin/sh
+                - -c
+                - "pgrep -f Runner.Listener > /dev/null"
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            failureThreshold: 3
+      volumes:
+        - name: runner-home
+          emptyDir: {}
+        - name: nuget-cache
+          emptyDir:
+            sizeLimit: 2Gi
+        - name: tmp
+          emptyDir: {}
+      restartPolicy: Always
+---
+# Runner for FlowerCore.Updater. Two replicas use per-pod emptyDir caches, so
+# backlog can drain without sharing a ReadWriteOnce PVC. Added 2026-05-26 to
+# close the runner-fleet gap that left the repo with only the offline
+# windows-sandbox runner and no Linux PR-CI capacity for future workflows.
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: github-runner-updater
+  namespace: github-runner
+  labels:
+    app.kubernetes.io/name: github-runner-updater
+    app.kubernetes.io/component: runner
+    app.kubernetes.io/part-of: flowercore
+    app.kubernetes.io/managed-by: argocd
+    flowercore.io/created-by: argocd
+    flowercore.io/runner-repo: updater
+    flowercore.io/github-repo: FlowerCore.Updater
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: github-runner-updater
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: github-runner-updater
+        app.kubernetes.io/component: runner
+        app.kubernetes.io/part-of: flowercore
+        flowercore.io/created-by: argocd
+        flowercore.io/runner-repo: updater
+        flowercore.io/github-repo: FlowerCore.Updater
+    spec:
+      serviceAccountName: github-runner
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1001
+        runAsGroup: 1001
+        fsGroup: 1001
+      initContainers:
+        - name: setup-runner-home
+          image: localhost/fc-github-runner:v20260525-ruby3.3.11-stepca
+          imagePullPolicy: Never
+          command:
+            - sh
+            - -c
+            - |
+              set -e
+              mkdir -p /home/runner/.dotnet /home/runner/.nuget/packages /home/runner/.nuget/NuGet /home/runner/.cache /home/runner/_tool
+              if [ -d /opt/runner-toolcache/Ruby ] && [ ! -d /home/runner/_tool/Ruby ]; then
+                cp -a /opt/runner-toolcache/Ruby /home/runner/_tool/
+              fi
+              chown -R 1001:1001 /home/runner/.dotnet /home/runner/.nuget /home/runner/.cache /home/runner/_tool
+              chmod -R 755 /home/runner/.dotnet /home/runner/.nuget /home/runner/.cache /home/runner/_tool
+          securityContext:
+            runAsUser: 0
+            runAsNonRoot: false
+          volumeMounts:
+            - name: runner-home
+              mountPath: /home/runner
+      containers:
+        - name: runner
+          image: localhost/fc-github-runner:v20260525-ruby3.3.11-stepca
+          imagePullPolicy: Never
+          env:
+            - name: REPO_URL
+              value: "https://github.com/astoltz/FlowerCore.Updater"
+            - name: RUNNER_NAME_PREFIX
+              value: "rke2-linux-updater"
+            - name: RUNNER_WORKDIR
+              value: "/tmp/runner/work"
+            - name: EPHEMERAL
+              value: "true"
+            - name: LABELS
+              value: "self-hosted,linux,fc-build-linux"
+            - name: HOME
+              value: "/home/runner"
+            - name: DOTNET_INSTALL_DIR
+              value: "/home/runner/.dotnet"
+            - name: DOTNET_CLI_TELEMETRY_OPTOUT
+              value: "1"
+            - name: DOTNET_NOLOGO
+              value: "1"
+            - name: DOTNET_GENERATE_ASPNET_CERTIFICATE
+              value: "false"
+            - name: DOTNET_CLI_HOME
+              value: "/home/runner"
+            - name: NUGET_PACKAGES
+              value: "/home/runner/.nuget/packages"
+            - name: XDG_CACHE_HOME
+              value: "/home/runner/.cache"
+            - name: RUNNER_TOOL_CACHE
+              value: "/home/runner/_tool"
+            - name: ACCESS_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: github-runner-token
+                  key: credential
+            - name: RUN_AS_ROOT
+              value: "false"
+          resources:
+            requests:
+              cpu: "500m"
+              memory: "1Gi"
+            limits:
+              cpu: "2000m"
+              memory: "4Gi"
+          volumeMounts:
+            - name: runner-home
+              mountPath: /home/runner
+            - name: nuget-cache
+              mountPath: /home/runner/.nuget/packages
+            - name: tmp
+              mountPath: /tmp
+          livenessProbe:
+            exec:
+              command:
+                - /bin/sh
+                - -c
+                - "pgrep -f Runner.Listener > /dev/null"
+            initialDelaySeconds: 30
+            periodSeconds: 30
+            failureThreshold: 3
+      volumes:
+        - name: runner-home
+          emptyDir: {}
+        - name: nuget-cache
+          emptyDir:
+            sizeLimit: 2Gi
+        - name: tmp
+          emptyDir: {}
+      restartPolicy: Always

 # Long-tail runner pattern:
 #
 # Sprint 32 added the final 16 long-tail repo-scoped Deployments above. Keep
 # Common as the only PVC-backed runner at replicas: 1. Any future multi-replica
 # runner must use per-pod emptyDir caches, not a shared ReadWriteOnce PVC.
+# 2026-05-25: PiManager added (was missed in the Sprint 32 long-tail sweep).
--- a/apps/selenium/selenium-grid.yaml
+++ b/apps/selenium/selenium-grid.yaml
@@ -132,13 +132,18 @@ spec:
          initialDelaySeconds: 10
          periodSeconds: 5
          timeoutSeconds: 5
+        # Hub baseline working set ~766Mi on 2026-05-25 (75% of prior 1Gi
+        # limit). Bump to 1.5Gi / 1Gi to keep ~50% headroom; matches the
+        # stampede-buffer pattern documented for multus
+        # (feedback_k8s_cni_multus_sizing). CPU left alone — observed 54m
+        # against a 500m limit, no contention.
        resources:
          limits:
            cpu: 500m
-            memory: 1Gi
+            memory: 1536Mi
          requests:
            cpu: 250m
-            memory: 512Mi
+            memory: 1Gi
 ---
 apiVersion: apps/v1
 kind: Deployment
@@ -198,13 +203,18 @@ spec:
            port: 5555
          initialDelaySeconds: 15
          periodSeconds: 5
+        # Chromium-based browser node. Bumped from 1Gi -> 2Gi (req 512Mi
+        # -> 1Gi) on 2026-05-25 — Edge had 51 OOMKills in 5d on the
+        # original 1Gi cap (~1 OOM every 2.4h), and Chrome at maxSessions=2
+        # was running 684Mi idle on the same cap. Matches the Firefox node's
+        # tested-stable 2Gi limit. CPU unchanged.
        resources:
          limits:
            cpu: '1'
-            memory: 1Gi
+            memory: 2Gi
          requests:
            cpu: 500m
-            memory: 512Mi
+            memory: 1Gi
        volumeMounts:
        - mountPath: /dev/shm
          name: dshm
@@ -378,13 +388,18 @@ spec:
            port: 5555
          initialDelaySeconds: 15
          periodSeconds: 5
+        # Chromium-based browser node. Bumped from 1Gi -> 2Gi (req 512Mi
+        # -> 1Gi) on 2026-05-25 — Edge had 51 OOMKills in 5d on the
+        # original 1Gi cap (~1 OOM every 2.4h), and Chrome at maxSessions=2
+        # was running 684Mi idle on the same cap. Matches the Firefox node's
+        # tested-stable 2Gi limit. CPU unchanged.
        resources:
          limits:
            cpu: '1'
-            memory: 1Gi
+            memory: 2Gi
          requests:
            cpu: 500m
-            memory: 512Mi
+            memory: 1Gi
        volumeMounts:
        - mountPath: /dev/shm
          name: dshm
--- a/tests/bluejay-infra-lint/FleetManifestLintTests.cs
+++ b/tests/bluejay-infra-lint/FleetManifestLintTests.cs
@@ -67,6 +67,7 @@ public sealed class FleetManifestLintTests
        ["github-runner-chat"] = "https://github.com/astoltz/FlowerCore.Chat",
        ["github-runner-mysql"] = "https://github.com/astoltz/FlowerCore.MySQL",
        ["github-runner-kiosk-linux"] = "https://github.com/astoltz/FlowerCore.Kiosk.Linux",
+        ["github-runner-updater"] = "https://github.com/astoltz/FlowerCore.Updater",
    };

    private static readonly HashSet<string> ScaledLinuxRunnerDeployments = new(StringComparer.Ordinal)
@@ -80,6 +81,7 @@ public sealed class FleetManifestLintTests
        "github-runner-chat",
        "github-runner-mysql",
        "github-runner-kiosk-linux",
+        "github-runner-updater",
    };

    private static readonly IReadOnlyDictionary<string, string> WritableRunnerEnv = new Dictionary<string, string>(StringComparer.Ordinal)
Author	SHA1	Message	Date
Andrew Stoltz	59543016c0	runners: add github-runner-updater Deployment FlowerCore.Updater had only the offline bluejay-ws-sandbox-1 Windows runner registered (the specialized fcsetup E2E target) and no Linux self-hosted runner, leaving the repo with no Linux PR-CI capacity for any future workflow. Modeled on github-runner-pimanager (Sprint 32 long-tail final entry, 2026-05-25); two replicas with per-pod emptyDir caches to keep ReadWriteOnce PVC contention out of the picture. Also registers github-runner-updater in the LinuxRunnerRepos + ScaledLinuxRunnerDeployments fleet-lint sets so future suite repairs treat the entry as canonically required (the 6 pre-existing lint failures on this file family are orthogonal: initContainer single- container count assertion + fc-updater IngressRoute POST allowlist + DM ApplicationSet convention drift). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-25 22:22:41 -05:00
Andrew Stoltz	bc3548e715	runners: add github-runner-pimanager Deployment FlowerCore.PiManager build run 26417714843 sat queued 5h with zero self-hosted runners registered to the repo. PiManager was missed in the Sprint 32 long-tail sweep — every other FC repo got a dedicated repo-scoped Deployment with its own ACCESS_TOKEN registration, but PiManager fell through the cracks. Adds a 2-replica ephemeral runner Deployment matching the Signage / DMS / Print.Web pattern (per-pod emptyDir caches, no shared PVC, labels `self-hosted,linux,fc-build-linux`, shared github-runner-token PAT). Once ArgoCD syncs, the queued job will pick up automatically. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-25 20:33:44 -05:00
bluejay	74333cc26b	selenium: right-size hub + chrome + edge memory limits (#28 )	2026-05-26 01:12:15 +00:00
Andrew Stoltz	7310fb88c2	selenium: right-size hub + chrome + edge memory limits Edge node has been OOMKilled 51 times in 5 days (~1 every 2.4h) on a 1Gi memory limit. Chrome runs maxSessions=2 on the same 1Gi cap and was idling at 684Mi — first concurrent session pushing the node to ~900Mi+ would be the next OOM. Hub was running at 766Mi against a 1Gi limit (75%); no recent restarts but no headroom either. Firefox node has been running at 2Gi memory limit for 9 days with zero restarts — that is the right size for a Selenium 4.27 browser node under our session profile (screen recording sidecar + 1080p rendering + page captures). Match it. Changes: - Hub: limit 1Gi -> 1.5Gi, request 512Mi -> 1Gi - Chrome: limit 1Gi -> 2Gi, request 512Mi -> 1Gi - Edge: limit 1Gi -> 2Gi, request 512Mi -> 1Gi CPU left alone on all three — observed utilization is well under the existing limits (hub 54m / 500m, chrome 185m / 1, edge 11m / 1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-25 20:11:41 -05:00
bluejay	148bc87b9a	runners: bake step-ca root CA into image (v20260525-stepca) (#27 )	2026-05-26 01:04:14 +00:00