Compare commits
1 Commits
sprint42/c
...
sprint39/c
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2896b60d3c |
@@ -103,6 +103,7 @@ curl -sk -X DELETE https://dns.iamworkin.lan/api/v1/servers/<serverId>/zones/iam
|
|||||||
- **Public read-only hosts**: if a public host fronts a service that also exposes admin writes internally, add a Traefik route match like `Host(...) && (Method(GET) || Method(HEAD))` on the public edge instead of trusting the app to reject unsafe methods.
|
- **Public read-only hosts**: if a public host fronts a service that also exposes admin writes internally, add a Traefik route match like `Host(...) && (Method(GET) || Method(HEAD))` on the public edge instead of trusting the app to reject unsafe methods.
|
||||||
- **Public read-write allowlist hosts**: if a public host accepts a tightly bounded write surface (e.g. bootstrap-JWT POST), pin the allowlist as `(Method(GET) || Method(HEAD) || Method(POST) || Method(OPTIONS))`. PUT/PATCH/DELETE must still 404 at the route. Track A's `updatecenter.iamworkin.lan` / `updates.iamworkin.lan` are the canonical example. The lint test enforces this invariant.
|
- **Public read-write allowlist hosts**: if a public host accepts a tightly bounded write surface (e.g. bootstrap-JWT POST), pin the allowlist as `(Method(GET) || Method(HEAD) || Method(POST) || Method(OPTIONS))`. PUT/PATCH/DELETE must still 404 at the route. Track A's `updatecenter.iamworkin.lan` / `updates.iamworkin.lan` are the canonical example. The lint test enforces this invariant.
|
||||||
- **Traefik VIP netpols**: when a `NetworkPolicy` allows `10.0.56.200`, also allow the post-DNAT backend ports (`8443` for TLS plus `8080` or `8000` for HTTP) or Calico will drop the rewritten flow.
|
- **Traefik VIP netpols**: when a `NetworkPolicy` allows `10.0.56.200`, also allow the post-DNAT backend ports (`8443` for TLS plus `8080` or `8000` for HTTP) or Calico will drop the rewritten flow.
|
||||||
|
- **RemoteDesktop isolation**: `apps/fc-desktop/network-policies.yaml` intentionally keeps desktop pod egress to named CoreDNS, `intranet-web:5300/TCP`, and noc1 step-ca `10.0.56.10:9000/9443` only. Guacamole display egress is owned separately by `apps/guacamole/guacamole.yaml` through `guacd-desktop-egress` on `5901/TCP`.
|
||||||
- **Auth-safe probes**: services behind API-key or global auth middleware should prefer `tcpSocket` probes unless `/health` is explicitly exempted before the middleware runs.
|
- **Auth-safe probes**: services behind API-key or global auth middleware should prefer `tcpSocket` probes unless `/health` is explicitly exempted before the middleware runs.
|
||||||
- **ArgoCD must use internal Gitea URL**: `http://gitea-clusterip.gitea.svc.cluster.local:3000/bluejay/bluejay-infra.git`, not the external HTTPS URL (step-ca cert isn't trusted by ArgoCD). The `ApplicationSet` and any hand-created `Application` must both use the internal URL.
|
- **ArgoCD must use internal Gitea URL**: `http://gitea-clusterip.gitea.svc.cluster.local:3000/bluejay/bluejay-infra.git`, not the external HTTPS URL (step-ca cert isn't trusted by ArgoCD). The `ApplicationSet` and any hand-created `Application` must both use the internal URL.
|
||||||
|
|
||||||
|
|||||||
@@ -1,263 +0,0 @@
|
|||||||
# fc-build-windows runner gate
|
|
||||||
|
|
||||||
Status: OPEN-WITH-OPERATOR-ACTION as of 2026-05-20.
|
|
||||||
|
|
||||||
This directory is intentionally not a live runner deployment. It records the
|
|
||||||
exact gate for bringing up the Windows self-hosted runner fleet without faking
|
|
||||||
capacity in GitHub or Kubernetes.
|
|
||||||
|
|
||||||
## Lane evidence
|
|
||||||
|
|
||||||
- `D:\git\FlowerCore\FlowerCore.Notes\docs\dashboards\decisions-waiting.html`
|
|
||||||
lines 15078-15085: Q-MR-82 says the Updater Windows Sandbox E2E run is
|
|
||||||
queued and `bluejay-ws-sandbox-1` is offline.
|
|
||||||
- `D:\git\FlowerCore\FlowerCore.Notes\memory\project_morning_routine_8_2026_05_20.md`:
|
|
||||||
Morning Routine #8 carries Q-MR-82 as the fleet-wide Windows runner gap.
|
|
||||||
- `D:\git\FlowerCore\FlowerCore.Notes\docs\standards\sprint-37-codex-dispatch-log-2026-05-19.md`
|
|
||||||
lines 76, 84-85, and 97: keep BLUEJAY-WS out of runner plans, merge Linux
|
|
||||||
runner expansion separately, and keep true Windows-only workflows parked on
|
|
||||||
the Windows runner host substrate path.
|
|
||||||
- `D:\git\FlowerCore\FlowerCore.Notes\docs\ai-agents\codex-prompts\2026-05-20-xxxxl-sprint-42-orchestrator-briefs.md`
|
|
||||||
lane Cx-5: land a deployment only if a Windows runner image/substrate is
|
|
||||||
ready; otherwise commit an operator-action gate.
|
|
||||||
- `D:\git\FlowerCore\FlowerCore.Notes\memory\feedback_bluejay_ws_never_a_github_runner.md`:
|
|
||||||
BLUEJAY-WS is operator-only territory; Windows runners belong on a dedicated
|
|
||||||
KubeVirt Windows VM such as `ci1` or a sibling VM.
|
|
||||||
|
|
||||||
## Live probe summary
|
|
||||||
|
|
||||||
Commands run on 2026-05-20 from `D:\git\FlowerCore\bluejay-infra`:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
$env:KUBECONFIG="$env:USERPROFILE\.kube\rke2.yaml"
|
|
||||||
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"`t"}{.metadata.labels.kubernetes\.io/os}{"`n"}{end}'
|
|
||||||
```
|
|
||||||
|
|
||||||
Result: `rke2-agent1`, `rke2-agent2`, and `rke2-server` all report
|
|
||||||
`kubernetes.io/os=linux`. There is no Windows Kubernetes node, so Windows
|
|
||||||
containers on RKE2 cannot satisfy `fc-build-windows`.
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
kubectl -n kubevirt-vms get vm,vmi,pods -o wide
|
|
||||||
```
|
|
||||||
|
|
||||||
Result: KubeVirt is healthy and `ci1` is `Running` / `Ready=True` on
|
|
||||||
`rke2-agent1` with VMI IP `10.42.103.35`.
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
virtctl --kubeconfig $env:USERPROFILE\.kube\rke2.yaml port-forward vm/ci1.kubevirt-vms 15985:5985
|
|
||||||
```
|
|
||||||
|
|
||||||
Result during port tests: `dial tcp 10.42.103.35:5985: connect: no route to
|
|
||||||
host`. The same result was seen for RDP 3389 and SSH 22. The VM exists, but it
|
|
||||||
is not remotely reachable for runner bootstrap from this lane.
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gh api /repos/astoltz/FlowerCore.Updater/actions/runners `
|
|
||||||
--jq '.runners[]? | {name,status,busy,labels:[.labels[].name]}'
|
|
||||||
gh run list --repo astoltz/FlowerCore.Updater `
|
|
||||||
--workflow "Updater Windows Sandbox E2E" --limit 5
|
|
||||||
```
|
|
||||||
|
|
||||||
Result: GitHub has one Updater runner, `bluejay-ws-sandbox-1`, with
|
|
||||||
`status=offline`; run `26150689447` is still `queued`.
|
|
||||||
|
|
||||||
## Feasibility classification
|
|
||||||
|
|
||||||
### Option A: Windows containers on RKE2
|
|
||||||
|
|
||||||
Not feasible without operator-physical infrastructure work. Kubernetes Windows
|
|
||||||
containers require a Windows node. The current cluster has Linux-only RKE2
|
|
||||||
nodes.
|
|
||||||
|
|
||||||
### Option B: KubeVirt Windows VM
|
|
||||||
|
|
||||||
Partially present, not deployable from this lane.
|
|
||||||
|
|
||||||
`apps/kubevirt-vms/ci1.yaml` already defines a Windows Server 2025 KubeVirt VM
|
|
||||||
using `localhost/fc-win-server-2025:v1`, and the live VM is running. However:
|
|
||||||
|
|
||||||
- the guest is not reachable over RDP, WinRM, or SSH through `virtctl
|
|
||||||
port-forward`;
|
|
||||||
- the current root disk is a `containerDisk`, so runner installation inside the
|
|
||||||
running guest is not a durable fleet state unless the first-boot automation
|
|
||||||
re-registers on every boot or the VM is moved to a persistent PVC-backed
|
|
||||||
disk;
|
|
||||||
- FC.Updater `Updater Windows Sandbox E2E` uses
|
|
||||||
`[self-hosted, windows, windows-sandbox]`, while `fc-build-windows` build jobs
|
|
||||||
use `[self-hosted, windows, fc-build-windows]`. Do not advertise
|
|
||||||
`windows-sandbox` until Windows Sandbox has been proven in the guest.
|
|
||||||
|
|
||||||
### Option C: bluejay-ws-sandbox-1
|
|
||||||
|
|
||||||
Operator-only emergency fallback. GitHub shows it registered but offline. The
|
|
||||||
current memory says BLUEJAY-WS must not be a fleet runner host, so this lane
|
|
||||||
does not start or re-register it. If the operator deliberately overrides the
|
|
||||||
policy to drain an emergency queue, start the existing visible runner console
|
|
||||||
from the BLUEJAY-WS desktop and treat that as temporary break-glass, not the
|
|
||||||
permanent Q-MR-82 closure.
|
|
||||||
|
|
||||||
## Operator action plan
|
|
||||||
|
|
||||||
### 1. Pick the Windows host class
|
|
||||||
|
|
||||||
Use `ci1` or a sibling Windows Server 2025 VM for WPF build/test jobs that need
|
|
||||||
`fc-build-windows`.
|
|
||||||
|
|
||||||
Use a Windows 11 Pro/Enterprise KubeVirt VM for Updater or WorldBuilder
|
|
||||||
Windows Sandbox gates, unless Windows Sandbox support is explicitly proven on
|
|
||||||
the selected guest. The workflow labels must match the real capability:
|
|
||||||
|
|
||||||
- WPF build runner: `self-hosted,windows,fc-build-windows,ci1`
|
|
||||||
- Sandbox runner: `self-hosted,windows,windows-sandbox,ci-sandbox1`
|
|
||||||
|
|
||||||
### 2. Make the VM reachable and durable
|
|
||||||
|
|
||||||
From BLUEJAY-WS:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
$env:KUBECONFIG="$env:USERPROFILE\.kube\rke2.yaml"
|
|
||||||
kubectl -n kubevirt-vms get vm,vmi,pods -o wide
|
|
||||||
virtctl --kubeconfig $env:KUBECONFIG vnc ci1 -n kubevirt-vms
|
|
||||||
virtctl --kubeconfig $env:KUBECONFIG port-forward vm/ci1.kubevirt-vms 13389:3389
|
|
||||||
virtctl --kubeconfig $env:KUBECONFIG port-forward vm/ci1.kubevirt-vms 15985:5985
|
|
||||||
```
|
|
||||||
|
|
||||||
Before runner registration, fix the current port-forward failure. The expected
|
|
||||||
state is that RDP or WinRM accepts a connection through the control plane.
|
|
||||||
|
|
||||||
For durability, either:
|
|
||||||
|
|
||||||
- move the runner VM to a persistent PVC-backed root disk; or
|
|
||||||
- keep `containerDisk` and bake first-boot runner registration into the sysprep
|
|
||||||
flow using a non-expiring credential lookup path.
|
|
||||||
|
|
||||||
Do not install a runner by hand into a transient VM and call Q-MR-82 closed.
|
|
||||||
|
|
||||||
### 3. Install runner prerequisites inside the VM
|
|
||||||
|
|
||||||
Run in an elevated PowerShell session in the Windows runner guest:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
winget install Microsoft.DotNet.SDK.10 --silent
|
|
||||||
winget install Microsoft.DotNet.DesktopRuntime.8 --silent
|
|
||||||
winget install Microsoft.PowerShell --silent
|
|
||||||
winget install Git.Git --silent
|
|
||||||
winget install Microsoft.VisualStudio.2022.BuildTools --silent
|
|
||||||
winget install Google.Chrome --silent
|
|
||||||
```
|
|
||||||
|
|
||||||
For a Sandbox-capable runner only:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
Enable-WindowsOptionalFeature -Online -FeatureName Containers-DisposableClientVM -All
|
|
||||||
Restart-Computer -Force
|
|
||||||
```
|
|
||||||
|
|
||||||
After reboot:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
Get-CimInstance -ClassName Win32_OptionalFeature -Filter "Name='Containers-DisposableClientVM'"
|
|
||||||
Test-Path C:\Windows\System32\WindowsSandbox.exe
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Register repo-scoped GitHub runners
|
|
||||||
|
|
||||||
The `astoltz` account uses repo-scoped runners. Generate a fresh one-hour
|
|
||||||
registration token per repo immediately before `config.cmd`.
|
|
||||||
|
|
||||||
From a trusted operator shell with `gh` authenticated:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
$repos = @(
|
|
||||||
"FlowerCore.Updater",
|
|
||||||
"FlowerCore.WorldBuilder",
|
|
||||||
"FlowerCore.DeviceManagement"
|
|
||||||
)
|
|
||||||
|
|
||||||
foreach ($repo in $repos) {
|
|
||||||
$token = gh api -X POST "/repos/astoltz/$repo/actions/runners/registration-token" --jq .token
|
|
||||||
$repoSlug = $repo.ToLowerInvariant().Replace("flowercore.", "").Replace(".", "-")
|
|
||||||
$runnerDir = "C:\fc-ghr\$repoSlug-fc-build-windows"
|
|
||||||
|
|
||||||
New-Item -ItemType Directory -Force -Path $runnerDir | Out-Null
|
|
||||||
Set-Location $runnerDir
|
|
||||||
|
|
||||||
if (-not (Test-Path ".\config.cmd")) {
|
|
||||||
Invoke-WebRequest `
|
|
||||||
-Uri "https://github.com/actions/runner/releases/download/v2.323.0/actions-runner-win-x64-2.323.0.zip" `
|
|
||||||
-OutFile "actions-runner.zip"
|
|
||||||
Add-Type -AssemblyName System.IO.Compression.FileSystem
|
|
||||||
[System.IO.Compression.ZipFile]::ExtractToDirectory((Resolve-Path actions-runner.zip), $runnerDir)
|
|
||||||
}
|
|
||||||
|
|
||||||
.\config.cmd `
|
|
||||||
--url "https://github.com/astoltz/$repo" `
|
|
||||||
--token $token `
|
|
||||||
--name "ci1-$repoSlug-fc-build-windows" `
|
|
||||||
--labels "self-hosted,windows,fc-build-windows,ci1" `
|
|
||||||
--work "_work" `
|
|
||||||
--unattended `
|
|
||||||
--replace
|
|
||||||
|
|
||||||
.\svc.ps1 install
|
|
||||||
.\svc.ps1 start
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
For Updater Sandbox E2E, register only after the guest proves Sandbox support,
|
|
||||||
and use `windows-sandbox` labels:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
$token = gh api -X POST "/repos/astoltz/FlowerCore.Updater/actions/runners/registration-token" --jq .token
|
|
||||||
.\config.cmd `
|
|
||||||
--url "https://github.com/astoltz/FlowerCore.Updater" `
|
|
||||||
--token $token `
|
|
||||||
--name "ci-sandbox1-updater" `
|
|
||||||
--labels "self-hosted,windows,windows-sandbox,ci-sandbox1" `
|
|
||||||
--work "_work" `
|
|
||||||
--unattended `
|
|
||||||
--replace
|
|
||||||
```
|
|
||||||
|
|
||||||
Keep registration tokens out of Git and logs. The durable credential source for
|
|
||||||
automation should be the existing 1Password item named `GitHub PAT (Runner
|
|
||||||
Registration)`, used only to mint short-lived repo registration tokens.
|
|
||||||
|
|
||||||
### 5. Verify GitHub and workflow pickup
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
gh api /repos/astoltz/FlowerCore.Updater/actions/runners `
|
|
||||||
--jq '.runners[] | select(.labels[].name == "windows-sandbox") | {name,status,busy,labels:[.labels[].name]}'
|
|
||||||
|
|
||||||
gh api /repos/astoltz/FlowerCore.DeviceManagement/actions/runners `
|
|
||||||
--jq '.runners[] | select(.labels[].name == "fc-build-windows") | {name,status,busy,labels:[.labels[].name]}'
|
|
||||||
|
|
||||||
gh run list --repo astoltz/FlowerCore.Updater `
|
|
||||||
--workflow "Updater Windows Sandbox E2E" --limit 3
|
|
||||||
```
|
|
||||||
|
|
||||||
Q-MR-82 can be marked resolved only after the Updater run moves from `queued` to
|
|
||||||
`in_progress` or `completed` on an online runner, or after the affected WPF
|
|
||||||
build repos show online `fc-build-windows` repo-scoped runners and their queued
|
|
||||||
jobs start.
|
|
||||||
|
|
||||||
## Break-glass BLUEJAY-WS command
|
|
||||||
|
|
||||||
Only if the operator explicitly overrides the "BLUEJAY-WS is not a runner"
|
|
||||||
policy to drain a queue:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
Set-Location C:\fc-ghr\updater-sandbox
|
|
||||||
.\run.cmd
|
|
||||||
```
|
|
||||||
|
|
||||||
If a Windows service exists:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
Get-Service 'actions.runner.*'
|
|
||||||
Start-Service 'actions.runner.*'
|
|
||||||
```
|
|
||||||
|
|
||||||
This does not close Q-MR-82 permanently. It is a temporary queue drain until a
|
|
||||||
dedicated VM runner is online.
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
resources:
|
|
||||||
- operator-gate-configmap.yaml
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: fc-build-windows-operator-gate
|
|
||||||
namespace: kubevirt-vms
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: fc-build-windows
|
|
||||||
app.kubernetes.io/component: operator-gate
|
|
||||||
app.kubernetes.io/part-of: github-runner
|
|
||||||
flowercore.io/q-card: Q-MR-82
|
|
||||||
annotations:
|
|
||||||
flowercore.io/outcome: OPEN-WITH-OPERATOR-ACTION
|
|
||||||
flowercore.io/live-runner: "false"
|
|
||||||
data:
|
|
||||||
outcome: OPEN-WITH-OPERATOR-ACTION
|
|
||||||
gate.md: |
|
|
||||||
Do not treat this ConfigMap as runner capacity.
|
|
||||||
|
|
||||||
Current probe, 2026-05-20:
|
|
||||||
- RKE2 nodes are linux-only; Windows containers require a Windows node.
|
|
||||||
- KubeVirt `ci1` is Running/Ready, but RDP 3389, WinRM 5985, and SSH 22
|
|
||||||
through `virtctl port-forward` return `connect: no route to host`.
|
|
||||||
- GitHub Updater runner list has only `bluejay-ws-sandbox-1`, status
|
|
||||||
offline. Updater Windows Sandbox E2E run 26150689447 remains queued.
|
|
||||||
|
|
||||||
Required operator action:
|
|
||||||
1. Make a dedicated Windows VM reachable and durable.
|
|
||||||
2. Install .NET 10 SDK, .NET 8 Desktop Runtime, Git, VS Build Tools, and
|
|
||||||
PowerShell 7.
|
|
||||||
3. Register repo-scoped runners with short-lived GitHub registration tokens.
|
|
||||||
4. Add `fc-build-windows` labels only to WPF build-capable guests.
|
|
||||||
5. Add `windows-sandbox` labels only after Sandbox support is proven.
|
|
||||||
registration-token-pattern.ps1: |
|
|
||||||
$repo = "FlowerCore.Updater"
|
|
||||||
$token = gh api -X POST "/repos/astoltz/$repo/actions/runners/registration-token" --jq .token
|
|
||||||
$runnerDir = "C:\fc-ghr\updater-fc-build-windows"
|
|
||||||
|
|
||||||
New-Item -ItemType Directory -Force -Path $runnerDir | Out-Null
|
|
||||||
Set-Location $runnerDir
|
|
||||||
|
|
||||||
# Install the Actions runner package here if config.cmd is absent.
|
|
||||||
.\config.cmd `
|
|
||||||
--url "https://github.com/astoltz/$repo" `
|
|
||||||
--token $token `
|
|
||||||
--name "ci1-updater-fc-build-windows" `
|
|
||||||
--labels "self-hosted,windows,fc-build-windows,ci1" `
|
|
||||||
--work "_work" `
|
|
||||||
--unattended `
|
|
||||||
--replace
|
|
||||||
|
|
||||||
.\svc.ps1 install
|
|
||||||
.\svc.ps1 start
|
|
||||||
verification.ps1: |
|
|
||||||
gh api /repos/astoltz/FlowerCore.Updater/actions/runners `
|
|
||||||
--jq '.runners[] | {name,status,busy,labels:[.labels[].name]}'
|
|
||||||
|
|
||||||
gh run list --repo astoltz/FlowerCore.Updater `
|
|
||||||
--workflow "Updater Windows Sandbox E2E" --limit 3
|
|
||||||
|
|
||||||
$env:KUBECONFIG="$env:USERPROFILE\.kube\rke2.yaml"
|
|
||||||
kubectl -n kubevirt-vms get vm,vmi,pods -o wide
|
|
||||||
@@ -20,9 +20,12 @@
|
|||||||
# 1) desktop-isolation — Browser Lab session pods.
|
# 1) desktop-isolation — Browser Lab session pods.
|
||||||
#
|
#
|
||||||
# Locks down pods labeled `app.kubernetes.io/name=remote-desktop` (every
|
# Locks down pods labeled `app.kubernetes.io/name=remote-desktop` (every
|
||||||
# session pod regardless of template). Allows guacd ingress for the VNC/RDP
|
# session pod regardless of template). Allows guacd ingress for the display
|
||||||
# display lane and remotedesktop-web's pre-handoff probing. Egress: NFS to
|
# lane and remotedesktop-web's pre-handoff probing. Egress is deliberately
|
||||||
# Synology, DNS, Traefik (cluster + LB VIP), Intranet (Browser Lab home).
|
# narrow: named CoreDNS, direct Intranet web, and noc1 step-ca only. There is
|
||||||
|
# no broad Traefik/VIP or internet egress from desktop sessions. If a future
|
||||||
|
# Browser Lab path needs a public-style host, prefer an explicit Service rule
|
||||||
|
# or include the post-DNAT backend port per the Traefik VIP lint.
|
||||||
apiVersion: networking.k8s.io/v1
|
apiVersion: networking.k8s.io/v1
|
||||||
kind: NetworkPolicy
|
kind: NetworkPolicy
|
||||||
metadata:
|
metadata:
|
||||||
@@ -65,51 +68,22 @@ spec:
|
|||||||
- port: 5901
|
- port: 5901
|
||||||
protocol: TCP
|
protocol: TCP
|
||||||
egress:
|
egress:
|
||||||
# NFS to Synology
|
# CoreDNS only. The old to: [] DNS rule accidentally allowed any DNS
|
||||||
|
# listener in any namespace or routed network.
|
||||||
- to:
|
- to:
|
||||||
- ipBlock:
|
|
||||||
cidr: 10.0.58.3/32
|
|
||||||
ports:
|
|
||||||
- port: 2049
|
|
||||||
protocol: TCP
|
|
||||||
- port: 2049
|
|
||||||
protocol: UDP
|
|
||||||
- port: 111
|
|
||||||
protocol: TCP
|
|
||||||
- port: 111
|
|
||||||
protocol: UDP
|
|
||||||
- to:
|
|
||||||
- ipBlock:
|
|
||||||
cidr: 10.0.58.3/32
|
|
||||||
ports:
|
|
||||||
- port: 445
|
|
||||||
protocol: TCP
|
|
||||||
- to: []
|
|
||||||
ports:
|
|
||||||
- port: 53
|
|
||||||
protocol: UDP
|
|
||||||
- port: 53
|
|
||||||
protocol: TCP
|
|
||||||
- to:
|
|
||||||
- ipBlock:
|
|
||||||
cidr: 10.0.56.200/32
|
|
||||||
- ipBlock:
|
|
||||||
cidr: 10.43.33.87/32
|
|
||||||
- namespaceSelector:
|
- namespaceSelector:
|
||||||
matchLabels:
|
matchLabels:
|
||||||
kubernetes.io/metadata.name: traefik-system
|
kubernetes.io/metadata.name: kube-system
|
||||||
podSelector:
|
podSelector:
|
||||||
matchLabels:
|
matchLabels:
|
||||||
app.kubernetes.io/name: traefik
|
k8s-app: kube-dns
|
||||||
ports:
|
ports:
|
||||||
- port: 80
|
- port: 53
|
||||||
protocol: TCP
|
protocol: UDP
|
||||||
- port: 443
|
- port: 53
|
||||||
protocol: TCP
|
|
||||||
- port: 8000
|
|
||||||
protocol: TCP
|
|
||||||
- port: 8443
|
|
||||||
protocol: TCP
|
protocol: TCP
|
||||||
|
# Browser Lab home / internal docs target. Use the real service port
|
||||||
|
# directly rather than public Traefik host aliases.
|
||||||
- to:
|
- to:
|
||||||
- namespaceSelector:
|
- namespaceSelector:
|
||||||
matchLabels:
|
matchLabels:
|
||||||
@@ -120,6 +94,17 @@ spec:
|
|||||||
ports:
|
ports:
|
||||||
- port: 5300
|
- port: 5300
|
||||||
protocol: TCP
|
protocol: TCP
|
||||||
|
# noc1 step-ca ACME endpoint. The lane brief called out 9000/TCP; the live
|
||||||
|
# ACME directory currently answers on 9443/TCP, so both stay pinned to the
|
||||||
|
# same host rather than reopening Traefik or internet egress.
|
||||||
|
- to:
|
||||||
|
- ipBlock:
|
||||||
|
cidr: 10.0.56.10/32
|
||||||
|
ports:
|
||||||
|
- port: 9000
|
||||||
|
protocol: TCP
|
||||||
|
- port: 9443
|
||||||
|
protocol: TCP
|
||||||
---
|
---
|
||||||
# 2) fc-desktop-default-deny — namespace-wide catch-all.
|
# 2) fc-desktop-default-deny — namespace-wide catch-all.
|
||||||
#
|
#
|
||||||
@@ -330,3 +315,11 @@ spec:
|
|||||||
protocol: UDP
|
protocol: UDP
|
||||||
- port: 53
|
- port: 53
|
||||||
protocol: TCP
|
protocol: TCP
|
||||||
|
- to:
|
||||||
|
- ipBlock:
|
||||||
|
cidr: 10.0.56.10/32
|
||||||
|
ports:
|
||||||
|
- port: 9000
|
||||||
|
protocol: TCP
|
||||||
|
- port: 9443
|
||||||
|
protocol: TCP
|
||||||
|
|||||||
33
apps/fc-devicemgmt/argocd-application.yaml
Normal file
33
apps/fc-devicemgmt/argocd-application.yaml
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
# Explicit ArgoCD Application shape for bootstrap/review.
|
||||||
|
#
|
||||||
|
# The live bluejay-infra ApplicationSet already discovers apps/* directories
|
||||||
|
# and creates this same Application name (`infra-fc-devicemgmt`) automatically.
|
||||||
|
# Keep repoURL on the internal Gitea ClusterIP URL; ArgoCD does not trust the
|
||||||
|
# external step-ca HTTPS endpoint.
|
||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: infra-fc-devicemgmt
|
||||||
|
namespace: argocd
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: fc-devicemgmt
|
||||||
|
app.kubernetes.io/part-of: flowercore
|
||||||
|
app.kubernetes.io/managed-by: argocd
|
||||||
|
flowercore.io/tenant-id: system
|
||||||
|
flowercore.io/created-by: bluejay-infra
|
||||||
|
spec:
|
||||||
|
project: default
|
||||||
|
source:
|
||||||
|
repoURL: http://gitea-clusterip.gitea.svc.cluster.local:3000/bluejay/bluejay-infra.git
|
||||||
|
targetRevision: main
|
||||||
|
path: apps/fc-devicemgmt
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc
|
||||||
|
namespace: fc-devicemgmt
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
|
- ServerSideApply=true
|
||||||
@@ -254,6 +254,68 @@ spec:
|
|||||||
targetPort: 4822
|
targetPort: 4822
|
||||||
name: guacd
|
name: guacd
|
||||||
---
|
---
|
||||||
|
# Guacd display egress isolation.
|
||||||
|
#
|
||||||
|
# Guacamole web talks to guacd on TCP/4822. Guacd then opens the desktop
|
||||||
|
# display connection to the per-session pod. Keep that second hop at raw VNC
|
||||||
|
# 5901/TCP for the current RemoteDesktop Browser Lab/openSUSE images. Do not
|
||||||
|
# grant guacd broad fc-desktop namespace egress; desktop-to-desktop lateral
|
||||||
|
# paths remain blocked by apps/fc-desktop/network-policies.yaml.
|
||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: NetworkPolicy
|
||||||
|
metadata:
|
||||||
|
name: guacd-desktop-egress
|
||||||
|
namespace: guacamole
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/part-of: remotedesktop
|
||||||
|
app.kubernetes.io/component: display-isolation
|
||||||
|
spec:
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app: guacd
|
||||||
|
policyTypes:
|
||||||
|
- Ingress
|
||||||
|
- Egress
|
||||||
|
ingress:
|
||||||
|
- from:
|
||||||
|
- podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app: guacamole
|
||||||
|
ports:
|
||||||
|
- port: 4822
|
||||||
|
protocol: TCP
|
||||||
|
egress:
|
||||||
|
- to:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: kube-system
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
k8s-app: kube-dns
|
||||||
|
ports:
|
||||||
|
- port: 53
|
||||||
|
protocol: UDP
|
||||||
|
- port: 53
|
||||||
|
protocol: TCP
|
||||||
|
# kubectl-proxy sidecar reaches the Kubernetes API; keep it explicit
|
||||||
|
# because this NetworkPolicy selects the whole guacd pod.
|
||||||
|
- to: []
|
||||||
|
ports:
|
||||||
|
- port: 443
|
||||||
|
protocol: TCP
|
||||||
|
- port: 6443
|
||||||
|
protocol: TCP
|
||||||
|
- to:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: fc-desktop
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: remote-desktop
|
||||||
|
ports:
|
||||||
|
- port: 5901
|
||||||
|
protocol: TCP
|
||||||
|
---
|
||||||
# Guacamole Web Application
|
# Guacamole Web Application
|
||||||
apiVersion: apps/v1
|
apiVersion: apps/v1
|
||||||
kind: Deployment
|
kind: Deployment
|
||||||
|
|||||||
93
tests/bluejay-infra-lint/RemoteDesktopNetworkPolicyTests.cs
Normal file
93
tests/bluejay-infra-lint/RemoteDesktopNetworkPolicyTests.cs
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
using FluentAssertions;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace BluejayInfraLint.Tests;
|
||||||
|
|
||||||
|
[Trait("Category", "Unit")]
|
||||||
|
public sealed class RemoteDesktopNetworkPolicyTests
|
||||||
|
{
|
||||||
|
private static readonly ManifestInventory Inventory = ManifestInventory.Load();
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void LiveDesktopIsolation_AllowsOnlyCoreDnsIntranetAndStepCaEgress()
|
||||||
|
{
|
||||||
|
var policy = NetworkPolicy("fc-desktop", "desktop-isolation");
|
||||||
|
var ports = policy.EgressPorts().ToHashSet(StringComparer.Ordinal);
|
||||||
|
|
||||||
|
ports.Should().BeEquivalentTo("53", "5300", "9000", "9443");
|
||||||
|
policy.AllScalars().Should().Contain(new[]
|
||||||
|
{
|
||||||
|
"kube-system",
|
||||||
|
"kube-dns",
|
||||||
|
"intranet",
|
||||||
|
"intranet-web",
|
||||||
|
"10.0.56.10/32"
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void LiveDesktopIsolation_RemovesInternetNfsAndTraefikEgress()
|
||||||
|
{
|
||||||
|
var policy = NetworkPolicy("fc-desktop", "desktop-isolation");
|
||||||
|
var scalars = policy.AllScalars().ToList();
|
||||||
|
var ports = policy.EgressPorts().ToHashSet(StringComparer.Ordinal);
|
||||||
|
|
||||||
|
scalars.Should().NotContain(new[] { "10.0.58.3/32", "10.0.56.200/32", "10.43.33.87/32", "traefik-system" });
|
||||||
|
ports.Should().NotContain(new[] { "80", "443", "445", "111", "2049", "8000", "8080", "8443" });
|
||||||
|
policy.MappingSequence("spec", "egress")
|
||||||
|
.Should()
|
||||||
|
.NotContain(rule => EgressRuleHasEmptyTo(rule), "desktop sessions must not use to: [] internet-style egress");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void LiveGuacdIsolation_AllowsRawVncToDesktopPodsOnly()
|
||||||
|
{
|
||||||
|
var policy = NetworkPolicy("guacamole", "guacd-desktop-egress");
|
||||||
|
var scalars = policy.AllScalars().ToList();
|
||||||
|
var ports = policy.EgressPorts().ToHashSet(StringComparer.Ordinal);
|
||||||
|
|
||||||
|
ports.Should().Contain("5901");
|
||||||
|
scalars.Should().Contain(new[] { "fc-desktop", "remote-desktop" });
|
||||||
|
ports.Should().NotContain(new[] { "3000", "3001", "3389", "80", "8080", "8443" });
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void LiveGuacdIsolation_KeepsGuacamoleWebIngressOnGuacdPort()
|
||||||
|
{
|
||||||
|
var policy = NetworkPolicy("guacamole", "guacd-desktop-egress");
|
||||||
|
|
||||||
|
policy.Scalar("spec", "podSelector", "matchLabels", "app").Should().Be("guacd");
|
||||||
|
policy.AllScalars().Should().Contain(new[] { "guacamole", "4822" });
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void HelperSmoke_FindsExpectedRemoteDesktopPolicies()
|
||||||
|
{
|
||||||
|
NetworkPolicy("fc-desktop", "desktop-isolation").Name.Should().Be("desktop-isolation");
|
||||||
|
NetworkPolicy("guacamole", "guacd-desktop-egress").Name.Should().Be("guacd-desktop-egress");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void HelperSmoke_EgressPortExtractionKeepsDistinctPorts()
|
||||||
|
{
|
||||||
|
var ports = NetworkPolicy("fc-desktop", "desktop-isolation")
|
||||||
|
.EgressPorts()
|
||||||
|
.ToHashSet(StringComparer.Ordinal);
|
||||||
|
|
||||||
|
ports.Should().HaveCount(4);
|
||||||
|
ports.Should().Contain(new[] { "53", "5300", "9000", "9443" });
|
||||||
|
}
|
||||||
|
|
||||||
|
private static ManifestDocument NetworkPolicy(string ns, string name)
|
||||||
|
=> Inventory.Documents.Single(document =>
|
||||||
|
document.Kind == "NetworkPolicy"
|
||||||
|
&& string.Equals(document.Namespace, ns, StringComparison.Ordinal)
|
||||||
|
&& string.Equals(document.Name, name, StringComparison.Ordinal));
|
||||||
|
|
||||||
|
private static bool EgressRuleHasEmptyTo(YamlDotNet.RepresentationModel.YamlMappingNode rule)
|
||||||
|
=> rule.Children.Any(entry =>
|
||||||
|
entry.Key is YamlDotNet.RepresentationModel.YamlScalarNode key
|
||||||
|
&& string.Equals(key.Value, "to", StringComparison.Ordinal)
|
||||||
|
&& entry.Value is YamlDotNet.RepresentationModel.YamlSequenceNode sequence
|
||||||
|
&& sequence.Children.Count == 0);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user