Kubernetes on a Mac, one microVM per pod

$ uname -sr; kubectl exec web -- uname -sr
Darwin 27.0.0
Linux 6.18.59

The first line is the Mac. The second is a pod on it, and that pod runs its own Linux kernel. The Mac is the Kubernetes node, and there is no Linux VM in between that hosts the node.

This is Briglet. It runs the Kubernetes control plane and a node agent as macOS processes on Apple silicon, and boots every pod as a microVM through hull and the hvi microVMM. It is open source under Apache-2.0, and it is experimental: it implements a subset of what the kubelet does, and the repository lists the gaps.

Why the Mac is the node

Kubernetes on a Mac usually means a Linux VM that runs the node, and every pod's containers share that VM's kernel. Apple's container framework takes the same path: the Kubernetes plugin it previewed at KubeCon North America this week boots each node as a Linux VM and provisions it with kubeadm, kind images and Cilium. The isolation boundary stays around the node. We wanted it around each pod instead. So the control plane and the node agent run on the Mac, and each pod gets a VM of its own, with its own kernel.

A Kubernetes node registers itself and renews a Lease. It runs the pods bound to it and reports their status. It mounts volumes, runs probes, restarts containers, and serves logs and exec. kube-proxy turns Services into routing. In Briglet each of these is a piece of Go code on the Mac.

From a clean Mac to a cluster

The recording below, from November 4, starts on a MacBook Pro with no Go, no kubectl and no Briglet. Long builds and the first image pull are cut short; the commands and their output are real.

hack/fetch-prereqs.sh fetches etcd, kube-apiserver and kubectl from the kubebuilder envtest release. Kubernetes has no darwin builds of kube-controller-manager and kube-scheduler, so the script builds both from the v1.37.0 source. For the pod VMs it takes hvi and vz-runner from the hull v0.1.0 release, the one Brig v0.5.1 installs, and builds a hull CLI whose gateway also routes Services. Then briglet up starts everything as user processes, each one only after the previous one reports healthy:

$ bin/briglet up
etcd                     ready in  0.75s (pid 55805, started)
kube-apiserver           ready in  3.01s (pid 55807, started)
addons                   applied 7 objects (CoreDNS registry.k8s.io/coredns/coredns:v1.14.6 at 10.96.0.10)
kube-controller-manager  ready in  1.51s (pid 55831, started)
kube-scheduler           ready in  1.26s (pid 55833, started)
gateway                  ready in  0.26s (pid 55835, started)
node                     ready in  4.26s (pid 55837, started)
briglet up took 11.18s; node mac is Ready

A pod is a microVM

When the scheduler binds a pod to the node, the agent allocates the pod an address, stages its volumes in a pod directory, pulls its images through hull, and runs hull run with the pod's address and shares. CPUs and memory come from the containers' limits.

The VM boots a small Linux image and runs briglet-guest, a static supervisor. It gives each container a mount and pid namespace over an overlay of its image, with the kubelet's default capabilities and no_new_privs. It runs init containers, sidecars and restarts with the kubelet's back-off; a restart keeps the VM, its kernel and its address. Logs and status go to the shared pod directory, so the agent serves kubectl logs without entering the VM.

Each pod shows up as a VM in hull:

$ hull ps | grep -e STATUS -e demo_
ID                     STATUS   EXIT  PID    IP          CREATED
demo_client_75356a93   stopped  -            10.87.0.14  2026-10-11 13:42:28
demo_web_8a0785ab      running  -     86130  10.87.0.13  2026-10-11 13:42:10

On a Mac Studio, three passes of each step:

These runs used Briglet's overlay-block root filesystem and pulled images through mirror.gcr.io. Cold passes start from a wiped state directory and include the image pull; in the default mode, a first nginx:alpine pull on the MacBook Pro took about 50 s. These are samples on one machine, not a benchmark.

Services without kube-proxy

There is no CNI plugin and no kube-proxy. Each pod VM's network goes to hull's gateway, a user-space process on the Mac. The node agent watches Services and EndpointSlices and gives the gateway a Service table, one row per ClusterIP and port, with the ready endpoints behind it. A NodePort becomes a listener on the Mac.

A client pod reads the Service by name, through CoreDNS and the gateway:

$ kubectl run client -n demo --image=busybox --restart=Never -- sh -c 'wget -qO- http://web | head -4'
pod/client created
$ kubectl logs -n demo client
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>

The Mac reaches the same pod through a NodePort:

$ kubectl get svc -n demo web-np
NAME     TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
web-np   NodePort   10.105.57.171   <none>        80:30642/TCP   2s
$ curl -s 127.0.0.1:30642 | grep -i nginx | head -3
<title>Welcome to nginx!</title>
<h1>Welcome to nginx!</h1>
<p>If you see this page, nginx is successfully installed and working.

nginx logs both requests from 10.87.0.1, the gateway, because Service traffic is proxied:

10.87.0.1 - - [11/Oct/2026:10:42:27 +0000] "GET / HTTP/1.1" 200 896 "-" "Wget" "-"
10.87.0.1 - - [11/Oct/2026:10:42:46 +0000] "GET / HTTP/1.1" 200 896 "-" "curl/8.7.1" "-"

The Mac has no route to pod addresses; it reaches pods through NodePorts.

A second Mac

briglet join creates credentials for a second node agent, and the controller manager gives each node half of the pod network. Pod addresses are not routed between Macs. A pod on one Mac reaches a pod on the other through a Service with a node port, and kubectl logs and exec work on both.

What the conformance tests say

We run the Kubernetes conformance tests with Sonobuoy and the v1.37.1 conformance image, in parallel. In the latest run, on one Mac Studio node, 401 of 433 executed tests pass, in 1 h 29 min. The 29 [Serial] tests are not part of a parallel run and still need one of their own. Briglet is not Kubernetes-certified, and a node whose pods are VMs cannot pass every test.

Each of the 32 failures matches a pattern in the skip list, with a reason per group:

• 18 are unsupported by design.
• 13 need feature gates this control plane does not turn on (PodCertificateRequest, ClusterTrustBundle).
• 1 needs two untainted nodes.

The design group lists what a VM per pod gives up:

# --- Unsupported by design: a pod is a VM and the node is a Mac ---
# hostNetwork, hostPID and hostIPC have no meaning on this node.
\[sig-node\] KubeletManagedEtcHosts
# No cgroups inside the VM, so there is no in-place resize or OOMKilled.
\[sig-node\] Pod InPlace Resize Container
# sysctls apply to the whole VM, not to one container.
\[sig-node\] Sysctls \[LinuxOnly\]
# The gateway routes Services round-robin per connection; no session affinity.
\[sig-network\] Services should (be able to switch|have) session affinity

The first full run passed 339. Dropping hull's --wait-ip, which waited for a lease that a pod with a static address never gets, took it to 392. Nine fixes to pod status, probes, Memory emptyDirs, privileged pods and OIDC discovery took it to 401.

What is not there yet

Three limits come with the design. A pod cannot share the host's namespaces. Resources are enforced per VM, not per container. The Mac reaches pods only through NodePorts.

Some gaps are work still to do. The agent does not serve attach, port-forward, stats or metrics, so kubectl run --rm -it returns 404. A cluster holds two nodes, and each Mac needs its own CoreDNS replica. On a second Mac the binaries need an ad-hoc signature and Local Network access, or their connections to the other Mac fail. If a node's gateway restarts, its running pod VMs lose their network until they are recreated.

Try it

You need an Apple silicon Mac with Homebrew and the Xcode command line tools:

brew install go kubernetes-cli
git clone https://github.com/brig-sh/briglet && cd briglet
hack/fetch-prereqs.sh
make
bin/briglet up
export KUBECONFIG=~/.briglet/admin.kubeconfig
kubectl run web --image=nginx:alpine --port=80
kubectl wait --for=condition=Ready pod/web --timeout=180s
kubectl exec web -- uname -sr

Set BRIGLET_REGISTRY_MIRROR=mirror.gcr.io before briglet up if Docker Hub's pull limit gets in the way. briglet down stops the cluster and keeps its state; briglet down --wipe removes it. docs/setup.md covers each step and a second Mac.

Briglet, hull and hvi are open source under Apache-2.0, and hull is based on urunc, a CNCF Sandbox project. We wrote about the VMM under every pod in why we wrote our own microVMM. Issues and conformance runs on other Macs are welcome on the Briglet repository.