nixgpu

Share one GPU between Kubernetes, containers, and your desktop

By priority, reactively, with no budgets, no quotas, and no reservations. For cards with an in-kernel DRM driver (AMD today; Intel is structurally compatible; NVIDIA excluded until an in-tree driver stack matures). One physical card, many consumers — k8s pods, an LLM server, an image generator, a video transcoder, and the interactive desktop — all co-residing when they fit, yielding by priority when they don't.

Status: pre-alpha, fully dogfooded

!

CONTRACT.md is the first real artifact

The behavior contract — what the platform must do — is written and stable. It was extracted from a production system running all of it on a single 16 GiB RDNA2 card: continuous batching, VCN-parallel video, reactive kill-reclaim, desktop spill-shedding, all verified live.

!

Fully dogfooded: production runs all four modules

device-tokens, priority-ladder, pressure-watcher, and ondemand-front were adopted back into the production cluster they were extracted from — the generalized forms are live-verified on the real card, scheduling, guarding, and fronting real tenants today. Only the optional kernel module is still to come.

!

No invented benchmarks

Nothing on this page claims a tested number that isn't in CONTRACT.md. The contract's behaviors (B1–B15) are the honest spec; most of them double as the public test suite once the landed modules are re-verified live.

!

Stress-tested under real contention

The token cap and pressure-watcher were run today through deliberately adversarial multi-tenant contention — an oversubscribed card, chaos-fired concurrent tenants, and a real LLM chat completion served under load — with zero card resets throughout; this is the strongest evidence yet for the contract's core claims, though it's same-day validation, not long-term soak.

Modules

device-tokens (landed)

Split one card into parallel scheduling lanes (compute + vcn media engine) via a generic device plugin, with co-scheduling under a small concurrency ceiling per lane.

priority-ladder (landed)

The PriorityClass set (desktop > interactive > besteffort) that defines who yields first. Priority is set by intent, not hardwired to an app.

pressure-watcher (landed)

The one hand-rolled piece: a small host-native DaemonSet that watches VRAM pressure and starvation and reclaims by scaling the lowest-priority tenant to zero. Includes desktop GTT-spill detection and a device-plugin registration-zombie guard, both battle-tested in production.

ondemand-front (landed)

A scale-to-zero front (Sablier + Caddy) serving one honest status page while a pod is not-Ready: cold start, GPU contention, and desktop-in-use are the same wait-state, announced the same way.

kernel (optional, planned)

DRM cgroup (dmem) accounting and TTM eviction-order patches for kernels that lack them. The watcher core runs on stock kernels reading sysfs — this module is only for filling gaps on older kernels.

The pitch

Everyone sharing a single consumer GPU hits the same wall: compute VRAM is pinned — the kernel cannot evict or swap it. Static partitioning wastes the card; per-app VRAM budgets are a fool's errand; and nothing turnkey exists for consumer AMD/Intel silicon (MIG-class isolation is datacenter-only).

Co-reside whatever fits

Nothing is declared, capped, or reserved. When the card is full and a higher-priority task needs VRAM, the lowest-priority compute tenant is scaled to zero — the only way to free pinned VRAM.

The desktop is just the top-priority tenant

Its graphics VRAM spills to GTT under pressure; the watcher reads that global signal from sysfs and sheds k8s tenants lowest-first. No host-side agent, no reservation. Video is separate silicon: the VCN/media engine gets its own scheduling lane and is never evicted for compute VRAM pressure.

Requirements (deliberate, not negotiable)

GPU driverIn-kernel DRM driver required (AMD today; Intel structurally compatible). NVIDIA excluded until an in-tree driver stack matures.
Delivery pathDeclarative GitOps only — nixidy-rendered manifests synced by Argo CD, the spine shipped by the sibling nixk3s project. Hand-applied YAML is not supported.
ScopeSharing substrate only. The shared LLM serving lane ships in the sibling nixllm project (implementing contract behaviors B4/B10/B14/B15); other tenants ship as app modules in the sibling nixapps project.