bluejay/bluejay-infra
1
0
Fork 0
Code Issues Pull Requests 14 Actions Packages Projects Releases Wiki Activity

runners: fleet-wide right-size CPU requests from 500m to 100m

Browse Source 
All 33 runner Deployments now request 100m CPU instead of 500m,
freeing roughly 50 idle pods × 400m = ~20 cores back to the cluster.
Observed CPU usage on idle runners is ~1m via kubectl top; the 500m
request was a 500× over-provision that was eating allocatable CPU
and blocking new workload scheduling — WorldBuilder runner could not
be scheduled even at the new 100m request because the pre-existing
fleet held the cluster at 99% requested.

Burst headroom preserved by limits.cpu: 2000m unchanged. TtsReader
keeps its 8Gi memory limit from the 2026-05-25 OOMKill fix; only
the CPU request line moves.

Recreate strategy on each deployment means a brief offline window
per runner during rollout; in-flight CI jobs complete on the
existing container before the new spec takes effect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Stoltz
2026-05-26 10:09:24 -05:00
parent 5cbc1a06b1
commit 2b420ce8a4
1 changed files with 30 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
apps/github-runner/github-runner.yaml
View File

@@ -241,7 +241,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -397,7 +397,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -533,7 +533,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -665,7 +665,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -797,7 +797,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -929,7 +929,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1064,7 +1064,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1196,7 +1196,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1328,7 +1328,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1460,7 +1460,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1594,7 +1594,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1732,7 +1732,7 @@ spec:
# symptoms surfaced. 8Gi gives ~30% headroom over peak observed. # symptoms surfaced. 8Gi gives ~30% headroom over peak observed.
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "2Gi" memory: "2Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -1867,7 +1867,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2001,7 +2001,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2135,7 +2135,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2269,7 +2269,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2402,7 +2402,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2536,7 +2536,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2669,7 +2669,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2802,7 +2802,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -2935,7 +2935,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3068,7 +3068,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3201,7 +3201,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3335,7 +3335,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3469,7 +3469,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3603,7 +3603,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3737,7 +3737,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -3871,7 +3871,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -4004,7 +4004,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"
@@ -4138,7 +4138,7 @@ spec:
value: "false" value: "false"
resources: resources:
requests: requests:
cpu: "500m" cpu: "100m"
memory: "1Gi" memory: "1Gi"
limits: limits:
cpu: "2000m" cpu: "2000m"