AirLLM: A 70B Model on a 12 GB GPU, and What That Really Means
AirLLM runs models far larger than your VRAM by streaming them layer by layer from disk. It really does put a 70B model on a consumer GPU. A realistic look at how it works, how to …
sander@sneekes:~$ whoami
Product Owner. DevOps Engineer. AI enthusiast.
Kubernetes | GitOps | AI Agents | Self-Hosted Everything
AirLLM runs models far larger than your VRAM by streaming them layer by layer from disk. It really does put a 70B model on a consumer GPU. A realistic look at how it works, how to …
Upgrading my Talos cluster used to be an evening of copy-pasted talosctl commands. Now it is a pull request. One evening with two Claude sessions: a break-glass script, tuppr as an …
One long day turning observed Hubble traffic into reviewed CiliumNetworkPolicies: a Go app that only reads the cluster and only writes to Git, three namespaces in enforce by …
Kinomap would not connect properly to the tablet on our cross-trainer, so instead of fighting it we wrote our own: a Kotlin/Compose app that speaks Bluetooth FTMS to the machine, a …
How HolmesGPT and its health-check operator landed on my Talos cluster in one evening: a wrapper chart, an Argo CD app, secrets from OpenBao, LLMs through LiteLLM — and the four …