Container Images

Reference documentation for the container image build system. This covers the contents of every service image, release configuration file formats, constraint override tooling, and local build instructions. The eight Python service images (keystone, horizon, glance, placement, barbican, neutron, cinder, nova), nova-compute, the Tempest image and the release-independent ovn, keystone-federation-proxy, backup-shifter, libvirt, openstack-hypervisor-operator and kvm-node-agent are built by the root flake from images/<name>/image.nix, and the operator images from nix/lib/operator-image.nix, as Nix Image Build describes; the sections below name what each of them carries.

Image Hierarchy

No Dockerfile is left. Every image is an output of the root flake: an image under images/ is built from images/<name>/image.nix, an operator image from nix/lib/operator-image.nix (Nix Image Build).

The figure shows which file of the repository feeds which step of a service image build. Before the build, hack/nix-lock.sh turns the release pins, the constraint overrides, the patches and the base pins into the lock under nix/locks/<release>/<name>/, and the lock is committed (Python environments and locks). nix build reads that lock: it fetches the source by commit and hash, builds the venv and assembles the image. nova-compute is a second build from the nova lock.

ovn compiles Open vSwitch and OVN from pinned upstream commits; keystone-federation-proxy compiles mod_auth_openidc from one; openstack-hypervisor-operator and kvm-node-agent compile a Go program from a pinned upstream commit with the patches under images/<name>/patches/ and run it on the distroless-equivalent image of nix/lib/static-image.nix; libvirt runs the libvirt of nix/pkgs/libvirt.nix with the QEMU of images/libvirt/qemu.nix. Each is described under Release-independent images.

Service Images

CI builds, verifies and publishes these images through the Build Images Workflow. The figure shows the three jobs every run executes, the pull request path, which ends after the loaded image is scanned and verified, and the push path, which pushes by digest and then merges, attests and signs. The service images build from Nix through the build-nix-image action, and their unit tests run as Nix builds. PR vs Push Behavior lists the differences aspect by aspect.

keystone

Location: images/keystone/image.nix, built from the lock nix/locks/<release>/keystone/ (Nix Image Build)

The venv holds keystone with its ldap extra and uwsgi from the base pins of nix/base-requirements.txt. pyproject.nix generates no PBR wsgi_scripts, so image.nix writes the uWSGI entry script /var/lib/openstack/bin/keystone-wsgi-public; its callable matches the wsgi_scripts entry of keystone's setup.cfg. The keystone-operator launches uWSGI with --wsgi-file on that path.

Runtime packages: none beyond the venv. python-ldap brings libldap and libsasl in its closure. The image carries no Apache module, because keystone runs under uWSGI.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Keystone dependencies
• keystone-manage CLI available via PATH

Build and verify:

nix build .#keystone-2026_1-image && ./result | docker load
bash tests/container-images/verify_keystone.sh keystone:2026.1

OCI labels: the five labels of Labels and reproducibility; org.opencontainers.image.version is the keystone ref of the lock.

horizon

Location: images/horizon/image.nix, built from the lock nix/locks/<release>/horizon/

The horizon=== pin in upper-constraints.txt is stripped before the lock is resolved (see Constraint Overrides). Two parts of the image are horizon's own:

• The static assets are built with the image. A derivation copies openstack_dashboard out of the venv into a writable tree, replaces local/local_settings.py with build-time settings, and runs collectstatic --noinput and compress --force (django-compressor offline compression). The assets are written to /var/lib/openstack/horizon-static with the offline manifest at dashboard/manifest.json. The build-time settings use the in-process cache backend: horizon wraps the Angular template-cache preloads in a cache block (horizon/_scripts.html), and with an unreachable Memcached django-compressor skips that block and every page render fails at runtime with OfflineGenerationError
• The packaged openstack_dashboard/local/local_settings.py is a link to /etc/openstack-dashboard/local_settings.py, where the horizon-operator mounts the rendered Django settings ConfigMap. The image creates /etc/openstack-dashboard/, owned by root; the link dangles until the ConfigMap is mounted

Runtime packages: none beyond the venv; the pymemcache session-cache client and uwsgi come from the base pins of nix/base-requirements.txt.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Serves via uWSGI loading openstack_dashboard.wsgi directly (the module ships application) — no hand-written wsgi script, and static assets are served through uwsgi --static-map /static=/var/lib/openstack/horizon-static
• i18n message catalogs are not compiled (compilemessages needs gettext at build time); the dashboard renders in English. Deferred until a locale requirement lands

Unit tests: horizon ships no .stestr.conf, and its Django suite runs under pytest. nix/lib/service-tests.nix branches on .stestr.conf presence and delegates to horizon's upstream tools/unit_tests.sh driver in the pytest path, without a database. Horizon 27.0.0 (2026.2) no longer ships that driver, so for it the run calls pytest four times the way that tag's tox.ini does, once per project with its own Django settings module (openstack_auth, horizon, openstack_dashboard and its plugin tests).

Build and verify:

nix build .#horizon-2026_1-image && ./result | docker load
bash tests/container-images/verify_horizon.sh horizon:2026.1

glance

Location: images/glance/image.nix, built from the lock nix/locks/<release>/glance/

The operator runs the API under uWSGI on every supported release with the hand-shipped /var/lib/openstack/bin/glance-wsgi-api entry script, which image.nix installs from images/glance/glance-wsgi-api with mode 0555 (the path the glance-operator's --wsgi-file flag references). Glance's stock module path (glance.wsgi.api:application) is unusable under the operator's config layout: wsgi_app.init_app() ignores sys.argv (and so uWSGI's --pyargv) and reads only $OS_GLANCE_CONFIG_DIR/glance-api.conf, so the shim redirects config discovery to the two mounted --config-dir roots instead.

The venv holds glance with the two pip_packages of releases/<release>/extra-packages.yaml: glance_store[s3] (the S3 store driver's extra lives on glance_store, not glance, and pulls boto3, botocore and s3transfer, all pinned in upper-constraints.txt) and lhafile (optional LHA-archive support for the image_decompression image-import plugin, pinned in overrides/<release>/constraints.txt). Its glance-api and glance-manage console scripts come from setup.cfg.

The image stays config-free: the glance-operator mounts glance-api.conf, glance-api-paste.ini, and policy, and provides the staging/tasks paths as emptyDir mounts.

Runtime packages:

Package	Purpose
qemu-utils	qemu-img, which the image_conversion image-import plugin runs on the staged image (qemu-img info, qemu-img convert)
rados, rbd	The modules of nix/pkgs/ceph-bindings.nix. glance_store/_drivers/rbd.py imports both behind a guard, and configure_add raises BadStoreConfiguration ("The required libraries(rbd and rados) are not available") without them. The store runs no ceph or rbd binary, so the image carries no ceph CLI

Ceph bindings in the virtualenv: the Ceph bindings have no PyPI wheels. nix/pkgs/ceph-bindings.nix builds the two extension modules for Python 3.12 from the ceph source of nixpkgs-images, against its librados and librbd. nix/lib/service-image.nix puts their directory at /var/lib/openstack/ceph-bindings and writes ceph-bindings.pth with that single line into the venv's own site-packages. The venv's bin/python pins sys.prefix to the venv's store path, so the .pth has to live in the venv itself. Python's site module appends the directory to sys.path, and rados and rbd resolve. The cinder image carries the same two modules.

How a patch reaches the image: hack/nix-lock.sh lists every patches/<service>/<release>/*.patch in the patches array of the lock's sources.json, and nix/lib/python-env.nix applies them with pkgs.applyPatches to the fetched source before the venv and the unit tests are built from it. Adding or retiring a patch is followed by make nix-lock; nix-lock-check fails until the lock names the new set. A patch keeps its Applies-to line and its assertion in the verify script.

Source patch:patches/glance/2026.1/0001-normalize-scheme-prefixed-s3-host-in-location-repair.patch strips the http:// or https:// prefix of s3_store_host in _construct_s3_url (glance/common/store_utils.py). Upstream's S3 credential-rotation repair, _update_s3_location_and_store_id, compares every S3 image location against a URL that function builds from the raw option value. The location URLs the S3 driver stores carry only the bare authority, because Store._set_url_prefix and StoreLocation strip the prefix, so the two never match when the option has one. The glance-operator always renders a prefix: the GlanceBackend CRD requires ^https?:// on spec.s3.host, and boto3 needs the scheme in endpoint_url for a non-AWS endpoint such as Garage. Unpatched, every API request that touches an S3 image logs "S3 URL mismatch for image ..., updating URL" and rewrites the location row, and a genuine credential rotation cannot be told apart from the permanent false positive. The patch strips the prefix the same way the store does. No upstream test pins the old behaviour: S3CredentialUpdateTestCase in glance/tests/unit/common/test_utils.py gives every mocked store the bare host s3.amazonaws.com, and the single-store S3 tests in glance/tests/unit/test_store_image.py never reach _construct_s3_url. The patch therefore carries no test hunk. Upstream status: not yet proposed.

Glance 33.0.0 (2026.2) carries no twin of that patch, because it removed _construct_s3_url together with the comparison it fed. Its _update_s3_location_credentials runs on every image read and strips the credentials a legacy location URL embeds, rewriting the location to the credential-free form the S3 driver of glance_store 5.7.0 writes. The rewrite is lazy: a location no 2026.2 pod reads keeps its embedded credentials. During a 2026.1 → 2026.2 rolling update the patched 2026.1 pods, whose glance_store 5.4.0 reads the credentials from the URL, put them back on each read and log "S3 URL mismatch", so a location can flip back and forth until the last 2026.1 pod is gone.

Source patch:patches/glance/2026.2/0001-test-new-image-with-location-do-not-rely-on-dns-resolution.patch carries upstream commit a37c43135e (master, 2026-09-28), cherry-picked to stable/2026.2 as a3e5f490b7 (2026-09-30), until a glance tag contains it; 33.0.0 is the newest 33.x tag as of 2026-10-10. The test TestImageFactory.test_new_image_with_location in glance/tests/unit/test_store_image.py builds an image whose location is http://storeurl.com/container/<uuid>. Glance 33.0.0 added an SSRF filter for HTTP(S) locations: _check_location_uri (glance/location.py) calls store_utils.validate_external_location, which hands http and https locations to validate_uri (glance/common/utils.py), and get_validated_address resolves the host with socket.getaddrinfo. The domain storeurl.com answers NXDOMAIN since about 2026-10-09, the lookup raises socket.gaierror, and the test fails with BadStoreUri: Invalid location before any store backend runs. A suite that resolves a public host cannot pass without DNS either. The patch mocks glance.common.utils.socket.getaddrinfo for that one test and returns a single fake address; the hunk is the upstream one verbatim, so the patch reverse-applies on the first tag that contains the commit and is retired on that Renovate bump. It changes a test module only, nothing in the built image, so verify_glance.sh carries no assertion for it. There is no 2026.1 twin: glance 32.0.0 has no validate_uri, so the test never resolves the host there.

tests/container-images/verify_glance.sh Test 12 drives the repair the built image carries, under an http:// and an https:// host. On 2026.1 it calls _update_s3_location_and_store_id: a location the driver's StoreLocation wrote has to stay unchanged while the credentials match, and still has to be rewritten once the access key rotates. On 2026.2 it calls _update_s3_location_credentials: a location the driver wrote carries no credentials and stays unchanged, a legacy location with embedded credentials becomes the driver-written URL, and a second pass changes nothing. An image that carries neither function fails the test.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Glance dependencies and the S3 store driver
• glance-manage and glance-api CLIs available via PATH

Unit tests: glance ships a .stestr.conf, so nix/lib/service-tests.nix runs its suite under stestr (the default path, as for keystone). No release carries an exclude file. The first CI run of the 33.0.0 suite counted 2,423 tests with the one test_new_image_with_location failure the source patch above removes; a run of the patched tree in the Docker build container (local, arm64, 2026-10-10) counted 2,423 tests and passed with no failure: 2,422 passed, 1 skipped.

Image contract check: tests/container-images/verify_glance.sh is the hard gate — it verifies the CLIs, importability, the uWSGI entry script, that uwsgi imports glance, the S3 store driver's boto3 resolution, non-root execution, and the absence of build tools. On 2026.1, its Test 12 fails against an image built without the source patch above. Test 13 covers the RBD store's bindings with the checks of tests/lib/ceph_bindings.sh, which the cinder and nova-compute scripts share, with the nix origin: ceph-bindings.pth holds its single line, rados and rbd import and resolve below the store path of nix/pkgs/ceph-bindings.nix, and /usr/lib/python3/dist-packages is absent from the virtualenv's sys.path. The store's guard attributes rbd and rados must not be None, and command -v ceph and command -v rbd must fail, so an image.nix edit that adds the ceph CLI to the glance image fails the build.

Build and verify:

nix build .#glance-2026_1-image && ./result | docker load
bash tests/container-images/verify_glance.sh glance:2026.1

placement

Location: images/placement/image.nix, built from the lock nix/locks/<release>/placement/ of the project openstack-placement

The image's one service-specific part is the WSGI entry script: upstream ships no usable one for either release. 15.0.0 (2026.1) packages with pyproject.toml and declares no WSGI script at all. 16.0.0 (2026.2) declares only placement-manage and placement-status under [project.scripts] and no WSGI script either. image.nix therefore writes /var/lib/openstack/bin/placement-api, a two-line entry that calls placement.wsgi.init_application(), under the upstream name for every release.

The image stays config-free: placement locates its configuration via the OS_PLACEMENT_CONFIG_DIR environment variable set by the operator.

Runtime packages: none beyond the venv.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Placement dependencies
• placement-manage and placement-status CLIs available via PATH
• The uWSGI entry script at /var/lib/openstack/bin/placement-api

Image contract check: tests/container-images/verify_placement.sh is the hard gate — it verifies the CLIs, importability, the uWSGI entry script (present, executable, parsable, and its import target resolvable), that uwsgi runs and imports placement, non-root execution, and the absence of build tools.

Build and verify:

nix build .#placement-2026_1-image && ./result | docker load
bash tests/container-images/verify_placement.sh placement:2026.1

barbican

Location: images/barbican/image.nix, built from the lock nix/locks/<release>/barbican/

The image ships no WSGI entry script at all. Both releases ship barbican/wsgi/api.py with a module-level application, so the barbican-operator launches uWSGI with the stock module path barbican.wsgi.api:application against config mounted at /etc/barbican/. Placement's entry script is written by hand because upstream ships no usable one, and glance carries the glance-wsgi-api shim because its stock module path ignores the operator's config layout. Barbican needs neither.

The image stays config-free: the barbican-operator mounts barbican.conf and barbican-api-paste.ini under /etc/barbican/.

Runtime packages: none beyond the venv.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Barbican dependencies, including the in-tree vault secret-store plugin and castellan
• barbican-manage and barbican-status CLIs available via PATH
• No WSGI entry script; the service is launched via the barbican.wsgi.api:application module path

Unit tests: barbican ships a .stestr.conf, so nix/lib/service-tests.nix runs its suite under stestr (the default path, as for keystone).

Image contract check: tests/container-images/verify_barbican.sh is the hard gate — it verifies the CLIs, importability, that the vault_plugin and castellan.drivers vault stevedore entry points load, that barbican.wsgi.api resolves and binds application to something other than the None sentinel, that uwsgi runs and imports barbican, non-root execution, and the absence of build tools. Both plugin halves are loaded through their entry-point metadata rather than imported by module path, because that is how barbican and castellan resolve them at runtime — an import would go green off the .py file on disk while the runtime lookup raises stevedore NoMatches. The WSGI check inspects the module instead of importing it: importing barbican.wsgi.api executes get_api_wsgi_script(), which reads paste config from /etc/barbican/barbican-api-paste.ini, absent from the bare image. So it pairs importlib.util.find_spec for the module path with an ast.parse of the module source for the application symbol. Both pinned releases open with application = None and only rebind it inside a threading.Lock() block, so the check walks into nested statement bodies but stops at every function, class and lambda boundary — uWSGI looks application up as a module global, and a binding in a nested scope is a different symbol. A module whose only module-level binding is that sentinel is rejected; otherwise uWSGI binds application to None and every request fails.

Build and verify:

nix build .#barbican-2026_1-image && ./result | docker load
bash tests/container-images/verify_barbican.sh barbican:2026.1

neutron

Location: images/neutron/image.nix, built from the lock nix/locks/<release>/neutron/

The image ships no WSGI entry script. No release ships a neutron-server script. neutron/wsgi/api.py is byte-identical at 28.0.1 (2026.1) apart from one argument 29.0.0 (2026.2) adds, prog='neutron-api', to the boot_server call. Both tags bind a module-level application inside a threading.Lock() block, so the neutron-operator launches uWSGI with --module neutron.wsgi.api and lets the process find its configuration through the OS_NEUTRON_CONFIG_DIR and OS_NEUTRON_CONFIG_FILES environment variables. The venv carries the console scripts declared in pyproject.toml (28.0.1): neutron-db-manage, neutron-status, neutron-periodic-workers, neutron-ovn-maintenance-worker, neutron-ovn-metadata-agent and neutron-ovn-db-sync-util.

The image stays config-free. The neutron-operator supplies the configuration through OS_NEUTRON_CONFIG_DIR and OS_NEUTRON_CONFIG_FILES, and either points [DEFAULT] api_paste_config at the shipped package data file /var/lib/openstack/etc/neutron/api-paste.ini or mounts that file to /etc/neutron/api-paste.ini. That file is byte-identical at both tags.

Runtime packages:

Package	Purpose
haproxy	The metadata agent spawns haproxy -f <cfg> per network (neutron/agent/metadata/driver_base.py)
iproute2	ip, which that agent uses for ip netns exec (neutron/agent/linux/ip_lib.py)
procps	sysctl, which that agent runs through ip netns exec in every namespace it creates (IpNetnsCommand.add in neutron/agent/linux/ip_lib.py); without it no network gets its haproxy
openvswitch	ovsdb-client, which pre_fork_initialize shells out to (neutron/common/ovn/utils.py); without it every API worker dies with [Errno 2] No such file or directory: 'ovsdb-client'

The four come from nixpkgs-images, and their binaries are on the image's PATH. The OVSDB wire protocol is schema-independent, so the client talks to the OVN 26.03 server.

The virtualenv holds ovs 3.7.0 (2026.1) and 3.7.1 (2026.2), whose ovs/dns_resolve.py resolves no hostname without the unbound Python module. The image ships no unbound module, so import unbound inside the image raises ModuleNotFoundError. Per decision D9 of issue #898, OVNCentral publishes IP addresses in status.dbAddress and status.internalDbAddress, and the neutron-operator renders [ovn] ovn_nb_connection and ovn_sb_connection from those.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Neutron dependencies
• The console scripts listed above available via PATH
• No WSGI entry script; the service is launched via the neutron.wsgi.api module path

Unit tests: neutron ships a .stestr.conf, so nix/lib/service-tests.nix runs its suite under stestr (the default path, as for keystone), with releases/<release>/test-excludes/neutron.txt and the sandbox exclude list. No release file excludes a test: the first runs at 27.0.3, 28.0.1 and 29.0.0 (22,093 tests) hit no environment-dependent failure. At roughly 21,000 tests the suite is the largest in the tree.

Image contract check: tests/container-images/verify_neutron.sh is the hard gate. Its 12 tests cover neutron-db-manage --help and neutron-status --help, the importability of neutron and of the ovs and ovsdbapp client libraries, the presence of api-paste.ini, and --help on the four companion console scripts. ovsdb-client --version naming Open vSwitch, haproxy -v, ip -V and sysctl --version prove the four runtime packages. The remaining tests check non-root execution, the absence of build tools, and that uwsgi runs and imports neutron. The WSGI check inspects the module instead of importing it: importing neutron.wsgi.api executes server.boot_server(api.api_server), which reads the configuration the operator mounts and the bare image does not carry. So it pairs importlib.util.find_spec for the module path with an ast.parse of the module source, and rejects a module whose only module-level binding of application is the None sentinel. Pointed at a barbican image, the script fails in each of its first nine tests.

Build and verify:

nix build .#neutron-2026_1-image && ./result | docker load
bash tests/container-images/verify_neutron.sh neutron:2026.1

cinder

Location: images/cinder/image.nix, built from the lock nix/locks/<release>/cinder/

The image ships no WSGI entry script. cinder/wsgi/api.py is byte-identical at 28.0.0 (2026.1) and 29.0.0 (2026.2), and both tags bind a module-level application inside a threading.Lock() block. The cinder-operator launches uWSGI with --module cinder.wsgi.api:application and --pyargv "--config-dir <dir>", because initialize_application() reads CONF(sys.argv[1:]).

The venv carries the nine console scripts declared in pyproject.toml (28.0.0): cinder-api, cinder-backup, cinder-manage, cinder-rootwrap, cinder-rtstool, cinder-scheduler, cinder-status, cinder-volume and cinder-volume-usage-audit; at 29.0.0 it carries the s3, gcs and swift extras of releases/2026.2/extra-packages.yaml. image.nix adds the five state directories under /var/lib/cinder, empty and owned by UID/GID 42424 together with /var/lib/cinder itself, and installs cinder-amqp-ready at /var/lib/openstack/bin/cinder-amqp-ready with mode 0555. The root helper's sudo comes from nix/lib/service-image.nix.

The image stays config-free. The package data files api-paste.ini, resource_filters.json, rootwrap.conf and rootwrap.d/volume.filters land under /var/lib/openstack/etc/cinder/ at both tags, declared under [tool.setuptools.data-files] in pyproject.toml at 28.0.0 and 29.0.0. Nothing in images/cinder/image.nix copies etc/cinder/ by hand, and the contract script asserts the four files. The cinder-operator points api_paste_config and resource_query_filters_file at those absolute paths.

Runtime packages:

Package	Purpose
nfs-utils	mount.nfs, which NfsDriver.do_setup probes for through the root helper (cinder/volume/drivers/nfs.py) and raises NfsException without. Under the restricted posture the pod never mounts and the probe's non-zero exit is tolerated, but the binary has to exist. nixpkgs installs it into bin/, so the image links /usr/sbin/mount.nfs, the path the probe names, to it
qemu-utils	qemu-img, which create-from-image, clone and extend shell out to (fetch_to_raw, resize_image, convert_image in cinder/image/image_utils.py), along with the LUKS qcow2 pre-create in cinder/volume/drivers/remotefs.py
ceph-client	ceph, rbd and rados, the client output of nixpkgs' ceph. The RBD volume driver (cinder/volume/drivers/rbd.py) runs ceph mon dump --format=json in _get_mon_addrs, rbd import in _create_encrypted_volume and _copy_image_to_volume, rbd export in copy_volume_to_image and rbd status in _get_image_status. The Ceph backup driver (cinder/backup/drivers/ceph.py) pipes rbd export-diff into rbd import-diff in _rbd_diff_transfer
rados, rbd	The modules of nix/pkgs/ceph-bindings.nix. Both drivers import them behind a guard, and check_for_setup_error raises "rados and rbd python libraries not found" without them (VolumeBackendAPIException in the volume driver, BackupDriverException in the backup driver)
sudo	The root helper is sudo cinder-rootwrap and the image carries no sudoers entry. sudo comes from nix/lib/service-image.nix

Ceph client: the image writes the ceph-bindings.pth of the glance section. The client is Ceph 20.2.4 (Tentacle) from nixpkgs-images. The image creates no ceph system user and no /var/lib/ceph or /var/log/ceph, because nothing reads them: the ceph.conf the cinder-operator renders for an RBD backend sets log_file = /dev/null and an admin socket under /tmp. The cipher pin belongs to the lab's Ceph (decision D7 of issue #1338), and the image does not set it.

Source patch:patches/cinder/2026.1/0001-nfs-run-qemu-img-info-as-the-service-user.patch and its 2026.2 twin flip the one run_as_root=True in NfsDriver._qemu_img_info to run_as_root=False. _qemu_img_info_base in cinder/volume/drivers/remotefs.py then follows nas_secure_file_operations for that call, like every other file operation of the driver. Upstream forced root in commit dbc9c7ca59 (2017) for files a Nova instance had attached; under the restricted posture of decision D3 of issue #979 there is no root to fall back to, and every file is owned by the service user.

The flip is necessary but not sufficient, and the cinder-operator of issue #987 owes the other half. _qemu_img_info_base resolves run_as_root or self._execute_as_root, and _execute_as_root starts out True: only NfsDriver.set_nas_security_options clears it, and only when nas_secure_file_operations resolves to true. The option defaults to auto, which resolves to true only for a new install that can write .cinderSecureEnvIndicator onto the share, and to false for every existing one. The operator therefore has to write nas_secure_file_operations = true explicitly; without it qemu-img info still goes through sudo cinder-rootwrap, which has no sudoers entry, and every create-from-image, clone and extend fails. The patches reach the image as the glance section describes: the lock's sources.json lists them, and nix/lib/python-env.nix applies them to the fetched source of the image, the unit tests and the e2e images alike. Cinder's test_copy_volume_from_snapshot keeps expecting run_as_root=True and stays green, because its test driver never calls set_nas_security_options and _execute_as_root keeps its default True, so the patch carries no test hunk. Upstream status: not yet proposed.

patches/cinder/2026.1/0002-create-from-image-run-qemu-img-as-the-service-user.patch and its 2026.2 twin flip two more forced-root qemu-img info calls. Both sit above the NFS driver on the create-from-image path and neither goes through it, so the flip in 0001 never reaches them. CreateVolumeFromSpecTask._create_from_image_cache_or_download (cinder/volume/flows/manager/create_volume.py) inspects the image it has just downloaded with image_utils.qemu_img_info(tmp_image), whose run_as_root defaults to True. fetch_verify_image (cinder/image/image_utils.py) inspects the same file through get_qemu_data(image_id, has_meta, format_raw, dest, True), which passes the flag positionally. In both cases the file is the temporary copy the service itself wrote into image_conversion_dir, owned by UID 42424, so reading it as root gains nothing. The two calls fail differently where sudo cinder-rootwrap has no sudoers entry: the manager call raises the ProcessExecutionError uncaught and every create-from-image ends in error status, while fetch_verify_image swallows it. get_qemu_data reads a failing qemu-img as "qemu-img is not installed", returns None for a raw image and skips the backing-file and data-file checks the function exists for, and raises ImageUnacceptable for every other disk format. Three upstream tests in cinder/tests/unit/test_image_utils.py pin the fetch_verify_image argument and move with it, so this patch does carry test hunks; no upstream test pins the manager call. tests/container-images/verify_cinder.sh asserts both call sites against the built image. Upstream status: not yet proposed.

patches/cinder/2026.1/0003-tests-collect-garbage-before-the-backup-tpool-size-tests.patch is test-only. BackupTestCase.test_default_tpool_size and test_tpool_size assert that eventlet's native thread pool is empty before and after they build a BackupManager. The Ceph backup driver tests leave os-brick RBDVolumeIOWrapper objects to the garbage collector, and a collected wrapper closes itself: close() flushes, the flush reaches the image through a tpool.Proxy, and that starts the pool's 20 threads. A collection between the two assertions fails the test with "Second list contains 20 additional elements", as it did once in test-service-images (cinder, 2025.2) on 2026-10-01. Both tests now start with gc.collect() and tpool.killall(), so every such finalizer has run before the first assertion; the assertions are unchanged. The patch changes nothing at runtime. Upstream status: not yet proposed; master replaced both tests when the backup service moved to native threads (c07c49c586). Cinder 29.0.0 carries that commit, so 2026.2 needs no twin.

patches/cinder/2026.1/0004-scheduler-fall-back-to-the-service-registry-for-backup-hosts.patch and its 2026.2 twin patches/cinder/2026.2/0003-scheduler-fall-back-to-the-service-registry-for-backup-hosts.patch (the same hunks, cut from 29.0.0) let the scheduler place a backup before the backup service has reported itself. Since the backup-host-selection series (upstream acfc87e201 and its successors, Stein), HostManager._get_available_backup_service_host in cinder/scheduler/host_manager.py picks the host from backup_service_states, which only the capability report a backup service publishes over an RPC fanout fills: once from init_host, then every backup_driver_stats_polling_interval (60 s). A scheduler that starts in the same second as the backup service misses the first report, and a first report sent while the backup driver still initializes carries backend_state: False; either way the scheduler holds no usable backup service until the next report, and create_backup in cinder/scheduler/manager.py answers every backup in between with Service not found for creating backup, the backup goes to error, and the volume returns to available. CI run 39046784189 hit that window 21 s after the processes started, in cinder-backup-nfs step 6, and a user who backs up a volume in the first minute after a cold start of the control plane hits it the same way. With the patch, _get_any_available_backup_service falls back to the service registry (objects.ServiceList.get_all_by_topic for cinder-backup, filtered by availability zone and is_up) when no report names a ready service and no driver was asked for, which is what restore and delete read in cinder/backup/api.py and what the scheduler read before the reports existed; a report, once received, keeps precedence. No upstream test pins the capability-only behaviour, so the patch adds six tests to HostManagerTestCase in cinder/tests/unit/scheduler/test_host_manager.py for the fallback, its availability-zone and heartbeat filters, the cluster name, the driver case and the empty registry. tests/container-images/verify_cinder.sh asserts both halves against the built image with mocks: an empty report set resolves to the registered service, and a report wins without consulting the registry. Upstream status: not yet proposed.

Readiness probe: images/cinder/cinder-amqp-ready is the exec readiness probe of the cinder-scheduler, cinder-volume and cinder-backup processes (decision D2 of issue #979). None of the three serves an HTTP port, so readiness here is "a process of this container holds a socket established to the message broker port", the semantics of kolla's healthcheck_port. The script reads /proc/net/tcp and /proc/net/tcp6 and skips a table that does not exist. It exits 0 and prints established to broker port <n> when a row in state 01 has a remote port equal to CINDER_AMQP_PORT (default 5672) and an inode one of the container's own processes holds; otherwise it exits 1 and prints no established connection to broker port <n>. The inode match is what keeps the answer local: /proc/net/tcp* is scoped to the network namespace every container of a pod shares, so the row alone would let one healthy connection report ready for every co-located cinder process. A process counts as the container's own when it shares the probe's mount namespace, which a container keeps to itself whatever the pod spec says: shareProcessNamespace: true lists the mates' processes in /proc, and the probe walks past them. The probe cannot see a broker that died without a FIN or an RST — that socket stays ESTABLISHED until the TCP keepalive expires, long after oslo.messaging's heartbeat gave up on it — so like healthcheck_port it answers "the connection exists", not "the broker answers". A CINDER_AMQP_PORT that is not a port number — not a number at all, or a number outside 1–65535, such as the 0 an unset field renders as — exits with a one-line message naming the variable but not its value, and not with a traceback: kubelet copies an exec probe's output verbatim into the Unhealthy event, and the key this misconfiguration is confused with carries the broker password. Refusing the out-of-range value is what keeps it from reading as a broker outage, the message a port no row can match would print for the life of the deployment. It needs no capability and no writable filesystem. The cinder-operator of issue #987 wires it as readinessProbe.exec.command: ["/var/lib/openstack/bin/cinder-amqp-ready"].

State directories: /var/lib/cinder is [DEFAULT] state_path. mnt and backup_mount are the two os-brick mount bases (nfs_mount_point_base, backup_mount_point_base), conversion is image_conversion_dir, tmp is [oslo_concurrency] lock_path, and coordination is the tooz file:// directory (decisions D2 and D12 of issue #979). The cinder-operator mounts an emptyDir over /var/lib/cinder under readOnlyRootFilesystem. The five directories in the image are what a plain docker run gets.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Cinder dependencies
• The nine console scripts and cinder-amqp-ready available via PATH
• No WSGI entry script; the API is launched via the cinder.wsgi.api:application module path
• sudo present with no sudoers entry

Unit tests: cinder ships a .stestr.conf, so nix/lib/service-tests.nix runs its suite under stestr. Both releases carry an exclude file. The 2026.1 file records the 13 TestFormatInspectors failures of its first run (they build their fixture images with qemu-img create and died with exit status 127) and does not exclude them: six of the 13 are the safety checks between a tenant-uploaded image and the volume host, and this run is the only gate in the pipeline that runs them, so the test run carries qemu-utils instead. releases/2026.1/test-excludes/cinder.txt excludes one test, test_put_container_disabled, which passes upstream only because tox runs unprivileged and os.makedirs raises PermissionError, whereas the Docker build container of the first runs ran it as root; the first run of the 28.0.0 suite counted 18,076 tests. releases/2026.2/test-excludes/cinder.txt keeps that one pattern, and the first run of the 29.0.0 suite counted 19,003 tests with no failure (18,984 passed, 19 skipped).

Image contract check: tests/container-images/verify_cinder.sh is the hard gate. Its 16 test functions cover cinder-manage --version and cinder-status --help, the importability of cinder and of cinder.wsgi.wsgi together with the driver and backend libraries (os_brick, castellan's Barbican key manager, boto3, tooz, taskflow, oslo_privsep), the four package data files, and --help on cinder-scheduler, cinder-volume, cinder-backup and cinder-api. mount.nfs being present and executable, qemu-img --version, sudo --version and a refused sudo -n true prove the runtime tools. The Ceph test, which runs right after it as test 16, runs the bindings checks of the glance Test 13. After cinder.objects.register_all(), the guard attributes rados and rbd of cinder.volume.drivers.rbd and cinder.backup.drivers.ceph must not be None. ceph --version and rbd --version must print a line starting with ceph version ; the script logs that line for the cipher revisit of decision D7 of issue #1338 and asserts no version. rbd help export-diff and rbd help import-diff must exit 0, and ceph --conf /nonexistent --connect-timeout 1 mon dump must fail fast with error calling conf_read_file, which is what a cinder-volume pod with a broken ceph.conf shows. An image without ceph-bindings.pth fails the .pth and import assertions while the tools still run. The probe test covers its presence and executability, the exit 1 that names port 5672 in a bare container, that it honours CINDER_AMQP_PORT=1, that a CINDER_AMQP_PORT carrying a URL is refused without a traceback, the exit 0 against a connection the container itself holds, and the exit 1 against the same connection seen from a second container joined to its network namespace. The two patch tests assert resolved values rather than the source text. The first reads the value _qemu_img_info_base resolves: run_as_root has to come out False once _execute_as_root is False, and True on the constructor default, which is the configuration dependency the operator has to satisfy. The second covers the create-from-image path: the run_as_root with which the volume manager (CreateVolumeFromSpecTask._create_from_image_cache_or_download) and image_utils.fetch_verify_image inspect the downloaded image has to come out False. The remaining tests check non-root execution, the absence of build tools, that uwsgi runs and imports cinder, and the five state directories, empty and owned by 42424 along with their parent. The WSGI check inspects cinder.wsgi.api instead of importing it: an import runs initialize_application() at module level and dies with oslo_service.wsgi.ConfigNotFound in a bare image. So it pairs importlib.util.find_spec with an ast.parse of the module source, and rejects a module whose only module-level binding of application is the None sentinel. Pointed at a neutron image, the script fails tests 1 to 11, 15 and 16, and passes only the three shared checks (non-root, no build tools, uwsgi).

Build and verify:

nix build .#cinder-2026_1-image && ./result | docker load
bash tests/container-images/verify_cinder.sh cinder:2026.1

nova

Location: images/nova/image.nix and images/nova/novnc.json, built from the lock nix/locks/<release>/nova/

The image carries the noVNC console assets nova-novncproxy serves. It is built from nova 33.0.0 (2026.1) and 34.0.0 (2026.2) and ships no WSGI entry script. The nova-operator of issue #1017 launches uWSGI on the module paths nova.wsgi.osapi_compute:application for the compute API and nova.wsgi.metadata:application for the metadata API, and passes the configuration through OS_NOVA_CONFIG_DIR and OS_NOVA_CONFIG_FILES (decision D1 of issue #1014). Both modules build their application at import time through wsgi_app.init_application, so importing either one inside a bare image raises ConfigFilesNotFoundError. At 33.0.0 both also call monkey_patch.patch(backend='threading') at import.

The venv carries pyroute2, the one pip_packages entry of the 2026.2 release, and the eleven console scripts declared in [project.scripts] of pyproject.toml (33.0.0): nova-compute, nova-conductor, nova-manage, nova-novncproxy, nova-policy, nova-rootwrap, nova-rootwrap-daemon, nova-scheduler, nova-serialproxy, nova-spicehtml5proxy and nova-status. image.nix adds the noVNC tree at /usr/share/novnc, the two state directories under /var/lib/nova, empty and owned by UID/GID 42424 together with /var/lib/nova itself, and nova-amqp-ready at /var/lib/openstack/bin/nova-amqp-ready with mode 0555.

The image stays config-free. The package data files api-paste.ini, rootwrap.conf and rootwrap.d/compute.filters land under /var/lib/openstack/etc/nova/ at both tags, declared under [tool.setuptools.data-files] in pyproject.toml at 33.0.0 and 34.0.0. api-paste.ini and rootwrap.conf are byte-identical at both tags. Nothing in images/nova/image.nix copies etc/nova/ by hand, and the contract script asserts the three files.

Runtime packages:

Package	Purpose
sudo	The root helper nova-rootwrap is a compute-side tool, and the image carries no sudoers entry. sudo comes from nix/lib/service-image.nix

qemu-utils and libvirt0 stay out (decision D14 of issue #1014). The control-plane roles convert no images, and the fake driver of issue #1018 needs no libvirt: nova.virt.fake imports without it.

noVNC pin: images/nova/novnc.json holds version, the upstream tag v1.7.0, rev, the commit that tag resolves to, and hash, the NAR hash of that commit's tree, which make nix-lock writes. v1.7.0 is an annotated tag, so the pin names the peeled commit, not the tag object. The build fetches by commit and hash with fetchFromGitHub and compares the version of the fetched package.json with the pin. A tree that disagrees fails the build with novnc.json: version <tag> does not match package.json, so a bump that moved one key and left the other behind stops there instead of shipping old assets under a new version.

Nine paths are copied: app/, core/, vendor/, vnc.html, vnc_lite.html, defaults.json, mandatory.json, package.json and LICENSE.txt. vnc_lite.html is the page [vnc] novncproxy_base_url names (decision D8 of issue #1014; the tree ships no vnc_auto.html) and it imports core/rfb.js. vnc.html loads app/ui.js, which reads defaults.json, mandatory.json and package.json, the version the console shows. vendor/ carries pako, the inflate library core/ imports. LICENSE.txt is the licence notice of the assets: MPL-2.0 for the core library, plus the licences it lists for the bundled parts. The tests, the docs, po/, snap/, utils/ and the lint configuration are not copied.

The fetch is a fixed-output derivation, so it needs no GitHub token. Renovate tracks version and rev through one custom manager (github-tags on novnc/noVNC, regex versioning for the leading v), whose single matchStrings entry spans the two adjacent keys, so the tag and the commit move in one PR. Renovate leaves hash stale, so nix-lock-check fails the PR until make nix-lock is pushed to its branch. Majors are disabled; minors and patches wait the 3-day cooldown and are not automerged, because the console page is user-facing and no e2e suite loads it before issue #1018. Digest updates are disabled too: a tag moved upstream to another commit is not a release, and the pin stays on the reviewed commit. tests/unit/renovate/novnc_pin_custommanager_test.sh replays the regex over novnc.json, checks the three rules, and resolves refs/tags/v1.7.0^{} upstream when it has network access, which is where a moved tag shows up.

Readiness probe: images/nova/nova-amqp-ready is the exec readiness probe of the nova-scheduler and nova-conductor processes (decision D1 of issue #1014). Neither serves an HTTP port, so readiness here is "a process of this container holds a socket established to the message broker port", the semantics of kolla's healthcheck_port. The script is cinder's cinder-amqp-ready with NOVA_AMQP_PORT in place of CINDER_AMQP_PORT, same default of 5672 and same semantics; the two images are independent build contexts, so each carries its own copy, and tests/unit/images/amqp_ready_probe_copies_test.sh fails when the code of the two copies drifts apart. It reads /proc/net/tcp and /proc/net/tcp6, skips a table that does not exist, and exits 0 with established to broker port <n> when a row in state 01 has a remote port equal to NOVA_AMQP_PORT and an inode one of the container's own processes holds. Otherwise it exits 1 with no established connection to broker port <n>. It walks the processes' descriptors only once a table holds such a row, and stops at the first socket of its own, because the walk is the expensive half on a service running a worker per CPU. The inode match is what keeps the answer local: /proc/net/tcp* is scoped to the network namespace every container of a pod shares, so the row alone would let one healthy connection report ready for every co-located nova process. A process counts as the container's own when it shares the probe's mount namespace, which a container keeps to itself whatever the pod spec says. A NOVA_AMQP_PORT that is not a port number, including the 0 an unset field renders as, exits with a one-line message naming the variable but not its value: kubelet copies an exec probe's output verbatim into the Unhealthy event, and the key this misconfiguration is confused with carries the broker password. Like healthcheck_port, the probe answers "the connection exists", not "the broker answers": a broker that died without a FIN or an RST leaves the socket ESTABLISHED until the TCP keepalive expires. It needs no capability and no writable filesystem. The nova-operator wires it as readinessProbe.exec.command: ["/var/lib/openstack/bin/nova-amqp-ready"].

State directories: /var/lib/nova is [DEFAULT] state_path, where a compute stores its node identity compute_id. tmp is [oslo_concurrency] lock_path, and instances is instances_path, whose default $state_path/instances a compute expects to exist. The nova-operator mounts an emptyDir over /var/lib/nova, so the two directories in the image are what a plain docker run gets.

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (gcc, cc, uv and python3-config are not on PATH, and /nix/store holds no compiler, binutils or uv package)
• Virtualenv at /var/lib/openstack with all Nova dependencies
• The eleven console scripts and nova-amqp-ready available via PATH
• No WSGI entry script; the two APIs are launched via the nova.wsgi.osapi_compute:application and nova.wsgi.metadata:application module paths
• The noVNC console assets under /usr/share/novnc
• sudo present with no sudoers entry

Unit tests: nova ships a .stestr.conf, so nix/lib/service-tests.nix runs its suite under stestr. It sets OS_NOVA_DISABLE_EVENTLET_PATCHING=False for every service, the value nova's tox py3 env uses: at 33.0.0 nova/cmd/scheduler.py selects the threading backend at import while nova/tests/unit/__init__.py has already selected eventlet, and oslo.service raises BackendAlreadySelected during stestr discovery, before any exclude list applies. The other services ignore the variable. Both releases carry an exclude file that excludes nothing. The first runs counted 16,803 tests at 33.0.0, with 63 skips and 2 expected failures, and 16,613 at 34.0.0 with 67 skips and 2 expected failures; none hit an environment-dependent failure.

Image contract check: tests/container-images/verify_nova.sh is the hard gate. Its 13 tests cover nova-manage --version and nova-status --help, the importability of nova and of nova.virt.fake, os_vif, os_brick, websockify, oslo_privsep and oslo_limit, together with the ModuleNotFoundError an import libvirt has to raise. Then come the three package data files, the eleven console scripts as executables, and --help on nova-scheduler, nova-conductor, nova-novncproxy and nova-compute. The noVNC test asserts that the copied files are present and readable by the service user and that the package.json version equals the version it reads out of images/nova/novnc.json; with that version edited to v9.9.9 it is the only test that fails, reporting 1.7.0 against 9.9.9. The probe test covers its presence and executability, the exit 1 that names port 5672 in a bare container, that it honours NOVA_AMQP_PORT=1, that a NOVA_AMQP_PORT carrying a transport URL and one carrying 0 are both refused without a traceback, the exit 0 against a connection the container itself holds and past a descriptor that vanished mid-scan, that a connection to 5671 counts with NOVA_AMQP_PORT=5671 and one to 5672 does not, and the exit 1 against the same connection seen from a second container joined to its network namespace, with and without a shared PID namespace, once that container has been shown the connection's row. The connection is held against a listener that never accepts, so the container owns the client end alone and a probe that read a row's local port would fail the ready checks. uwsgi --version, uwsgi importing nova, the absence of qemu-img, sudo --version and a refused sudo -n true prove the runtime wiring. The remaining tests check non-root execution, the absence of build tools, and the two state directories, empty and owned by 42424 along with their parent. The WSGI check inspects the two modules instead of importing them: an import runs wsgi_app.init_application at module level and dies with ConfigFilesNotFoundError in a bare image. So it pairs importlib.util.find_spec for each module path with an ast.parse of the module source, and rejects a module whose only module-level binding of application is the None sentinel. Both release images pass all 57 assertions. Pointed at a cinder image the script exits 1 with 37 of them failing: every test but the non-root and build-tool ones fails, while the wrong image still satisfies the uwsgi and sudo halves of the runtime test and the libvirt-absence half of the import test.

Build and verify:

nix build .#nova-2026_1-image && ./result | docker load
bash tests/container-images/verify_nova.sh nova:2026.1

nova-compute

Location: images/nova-compute/image.nix and images/nova-compute/sudoers, built from the lock nix/locks/<release>/nova/ with its nova-compute dependency group

nova-compute runs from this image on a compute cluster's hypervisor nodes. It is built from nova's own lock (33.0.0 for 2026.1, 34.0.0 for 2026.2), so nova-compute:2026.1 and nova:2026.1 carry the same nova with the same patches and constraints, and it is published under the same four tags as ghcr.io/c5c3/nova (see Tag Schema). On top of nova the image carries the libvirt binding, qemu-img, the host tools for iSCSI, multipath and NVMe that os-brick (the library nova attaches volumes with) runs, mount.nfs for the NFS exports nova mounts itself, cryptsetup, genisoimage, the Ceph client, and a rootwrap and sudo posture that lets the unprivileged openstack user start nova's privileged helpers. The nova control-plane image carries none of it (decision D14 of issue #1014). Its consumer is the NovaCompute node pool, which runs it under the tag of the Nova's installed release.

The image has a directory of its own instead of being part of images/nova/image.nix. hack/ci-generate-cleanup-matrix.sh turns every images/<name>/ directory into a package the nightly GHCR cleanup prunes. The only other packages it knows are the <name>-operator images, one per operators/<name>/ directory with a go.mod. And verify_nova.sh would share an image definition with an image it must not describe.

image.nix builds the image on nix/lib/service-image.nix under the name nova-compute, with the venv env.venvWith [ "nova-compute" ]: nova's default dependencies plus the nova-compute group, which hack/nix-lock.sh writes from the nova-compute block of extra-packages.yaml (libvirt-python at both releases). upper-constraints.txt fixes the binding's version: 12.0.0 at 2026.1, 12.6.0 at 2026.2. PyPI ships it as an sdist only, and nix/lib/python-env.nix builds it against the libvirt of nix/pkgs/libvirt.nix, the derivation the libvirt image runs (checks.<system>.libvirt-pairing holds the two together). The nova image's venv and nova's unit-test venv stay without it.

There is no noVNC tree and no nova-amqp-ready: the NovaCompute pod runs no probe, because the service state Nova reports is its health signal.

Runtime packages (the packages of image.nix, the same list in both releases):

Package	What runs it
qemu-utils	qemu-img create (nova/virt/libvirt/utils.py), qemu-img info and convert (nova/privsep/qemu.py), and qemu-nbd
openiscsi	iscsiadm (os_brick/initiator/connectors/iscsi.py) and iscsid; a root link puts iscsiadm at /sbin/iscsiadm, which os-brick reads at import
multipath-tools	multipath, multipathd and kpartx (os_brick/initiator/linuxscsi.py)
nvme-cli	nvme (os_brick/initiator/connectors/nvmeof.py, os_brick/privileged/nvmeof.py)
lsscsi	lsscsi (os_brick/initiator/linuxscsi.py)
nfs-utils	mount.nfs, the helper mount -t nfs runs, and rpc.statd. nova mounts a Cinder NFS export itself, without os-brick: LibvirtNFSVolumeDriver (nova/virt/libvirt/volume/nfs.py) goes through nova/virt/libvirt/volume/mount.py to mount in nova/privsep/fs.py, below [libvirt] nfs_mount_point_base (default $state_path/mnt). nixpkgs installs the helpers into bin/; root links put them at /sbin, where util-linux looks for mount.<type>, and mount.nfs also at /usr/sbin, noble's path
cryptsetup.bin	cryptsetup (os_brick/encryptors/luks.py) for encrypted volumes; the compute contract renders [key_manager] backend = barbican when Barbican is enabled
cdrkit	genisoimage, the default of [DEFAULT] mkisofs_cmd (nova/conf/configdrive.py), which builds config drives
ceph-client	ceph and rbd (nova/storage/rbd_utils.py): ceph mon dump in get_mon_addrs, rbd import in import_image, rbd export in export_image and ceph df in get_pool_info
util-linux.bin, e2fsprogs.bin, kmod, sg3_utils, lvm2.bin	The tools noble's package closure carried that nova's privsep and os-brick run by name: mount, umount, blockdev, findmnt, lsblk, blkid, losetup, mkswap, mkfs, nsenter, mkfs.ext4, e2fsck, resize2fs, modprobe (with xz and zstd), sg_scan and dmsetup
rados, rbd	The modules of nix/pkgs/ceph-bindings.nix. nova/storage/rbd_utils.py imports them behind a guard, and RBDDriver.__init__ raises RuntimeError('rbd python libraries not found') through _check_for_import_failure without them

Two more root links come from nixpkgs' systemd without adding it to PATH: /lib/udev/scsi_id (get_scsi_wwn in os_brick/initiator/linuxscsi.py) and /usr/bin/udevadm. nixpkgs' udev ships no binaries, and the full systemd on PATH would bring reboot, halt and init. Measured with nix path-info -S on x86_64-linux, the store paths of the 2026.1 image hold about 3.06 GB against 0.57 GB for the nova image; Ceph, systemd, libvirt and QEMU's tools make most of the difference.

Node identities: nixpkgs' packages run no postinst, but openiscsi installs an /etc/iscsi/initiatorname.iscsi it generated at build time. os-brick reports the initiator name and the host NQN to Cinder as the node's identity, so a baked file would give every compute node the same IQN or NQN, and a baked /etc/multipath.conf can disagree with the host's multipathd. The image removes /etc/iscsi/initiatorname.iscsi, /etc/nvme/hostnqn, /etc/nvme/hostid and /etc/multipath.conf, and iscsiadm --version and nvme version still run without them.

Ceph client: the image writes the ceph-bindings.pth of the glance section. The client is Ceph 20.2.4 (Tentacle) from nixpkgs-images, as in the cinder image, and the image creates no ceph system user. The NovaCompute still offers no imagesType: rbd, because it carries no Ceph credential contract (see the NovaCompute reference).

Rootwrap and sudo posture: nova's root helper is sudo nova-rootwrap <[DEFAULT] rootwrap_config>, and rootwrap_config defaults to /etc/nova/rootwrap.conf. nova-compute hands that helper to every privsep context it starts: os-brick's, nova's sys_admin_pctxt and os-vif's. A privsep context is a root daemon oslo.privsep starts through privsep-helper, and the helper is how it gets root. The compute contract renders no [privsep*], [workarounds] or rootwrap_config key, so nova's default helper is the one that runs. The rootwrap.conf nova installs points filters_path and exec_dirs at system directories, where neither compute.filters nor privsep-helper lives, and the programs of this image live in /nix/store. The image closes both gaps:

• /etc/nova/rootwrap.conf, written by nix/lib/rootwrap-conf.nix, sets filters_path=/var/lib/openstack/etc/nova/rootwrap.d and exec_dirs=/var/lib/openstack/bin, followed by the store bin directory of every runtime package. The file is root-owned with mode 0644. It records where this image keeps its binaries, which is image plumbing, so the config-free rule of the nova image does not cover it
• /etc/sudoers.d/nova-compute, mode 0440, sets a global secure_path that starts with /var/lib/openstack/bin and holds one rule: openstack ALL = (root) NOPASSWD: /var/lib/openstack/bin/nova-rootwrap /etc/nova/rootwrap.conf *. visudo -csf checks it at build time. The strict flag makes a reference to an undefined alias fatal, such as a NOPASSWD that lost its colon and reads as an alias name; plain visudo -c only warns about it

oslo.rootwrap always admits privsep-helper run as root, whatever the filters say, so one rule covers every privsep daemon nova-compute starts, and sudo -n true stays refused. Through privsep-helper the rule is root-equivalent, which is what nova's own design grants its service user. The pod that runs this image is privileged on a hypervisor node anyway. Because secure_path is global, a consumer that runs nova-compute as root resolves the same helper. sudo runs the PAM stack of nix/lib/service-image.nix, whose account phase permits: the image has no /etc/shadow, and the one rule names a nologin service user with no password ageing to check.

Open vSwitch client: os-vif plugs OVS ports through [os_vif_ovs] ovsdb_interface, whose default native is the ovsdbapp IDL. ovsdbapp is a Python library already in the virtualenv (2.16.1 at 2026.1, 2.19.0 at 2026.2), and nova runs no ovs-* binary itself. The image therefore installs no OVS package. Pointing [os_vif_ovs] ovsdb_connection at the host's OVSDB socket is the consumer's configuration.

What the image expects from its pod (the NovaCompute node contract is the pod spec that meets it):

• Configuration mounted below /etc/nova, such as /etc/nova/compute.conf.d, and never a volume at /etc/nova itself, which would hide rootwrap.conf
• Privilege escalation allowed (allowPrivilegeEscalation not false), since sudo is a setuid binary
• The host's /etc/iscsi, /etc/nvme and /etc/multipath* mounted, so os-brick reports the node's own identity and agrees with the host's multipathd
• /var/lib/nova mounted from the host (a hostPath, never an emptyDir): it holds compute_id, the node identity nova-compute writes on first start and must find again after every pod recreation, and instances below it. Without it the next start writes a new node UUID, which collides with the existing ComputeNode record of the host
• For a consumer that runs nova-compute as the image's openstack user: /var/lib/nova, /var/lib/nova/instances and /var/lib/nova/tmp on the host owned by 42424:42424 before nova-compute starts. The NovaCompute pod runs nova-compute as root, because a stock host's libvirt socket is root:libvirt 0660 with a host-specific group ID, so it needs no ownership change. It creates instances itself, root:root 0755, in its create-instances-dir init container (see the node contract). A non-root consumer needs the owner change. The mount hides the image's own directories, the kubelet creates a missing hostPath directory as root:root 0755, and fsGroup does not apply to a hostPath. On a directory nova-compute cannot write, the first start fails to write compute_id and exits with InvalidNodeConfiguration. An init container running as root sets the owner with install -d -o 42424 -g 42424 /var/lib/nova /var/lib/nova/instances /var/lib/nova/tmp, which leaves everything below the three directories alone. Never chown -R: the instance disks below instances belong to libvirt and QEMU
• [DEFAULT] state_path = /var/lib/nova in the compute configuration, so compute_id lands on that mount rather than in nova's default $pybasedir

Final image properties:

• Runs as openstack user (UID 42424, GID 42424)
• Contains no build tools (no gcc, cc, uv, python3-config or pkg-config on PATH, and no compiler, binutils or uv package in /nix/store)
• Virtualenv at /var/lib/openstack with Nova, its dependencies and libvirt-python
• sudo present with the one nova-rootwrap rule

Image contract check: tests/container-images/verify_nova_compute.sh runs inline on pull requests and in verify-nova-compute-image on push. Its eleven tests, of which test 10 runs right after test 4:

1. nova-compute, nova-manage, nova-rootwrap and privsep-helper are executable, and nova-compute --help exits 0.
2. import libvirt succeeds, libvirt.getVersion() is at least 12000000, since the binding links the libvirt the libvirt image runs, and the installed libvirt-python equals the effective pin of the image's release. The release is NOVA_COMPUTE_RELEASE, which both CI jobs set from matrix.release; unset, it is the one whose source-refs.yaml names the image's nova, which needs a tag pin. The pin comes from overrides/<release>/constraints.txt when that file carries one and from upper-constraints.txt otherwise, the order hack/nix-lock.sh merges them in. A nova version that no release or more than one release carries, a -libvirt-python override and a missing pin each fail with a message naming the value.
3. nova.virt.libvirt.driver, the os-brick iSCSI and NVMe connectors, the LUKS encryptor and vif_plug_ovs.ovsdb.impl_idl import.
4. The eleven host tools run, ceph --version and rbd --version among them, one assertion per tool, so a missing package names itself.
5. The four identity files are absent.
6. The posture holds: the two rootwrap.conf lines, compute.filters, six trusted paths that the service user cannot write, one NOPASSWD entry in sudo -n -l and a refused sudo -n true. sudo -n nova-rootwrap /etc/nova/rootwrap.conf id exits 99 with Unauthorized command: id, as the service user and as root. privsep-helper run through rootwrap exits 1 with ConfigFilesNotFoundError, which proves rootwrap found it through exec_dirs (without that entry rootwrap exits 96 with Executable not found).
7. The container runs as openstack.
8. The build-tools check of tests/lib/build_tools.sh passes, and pkg-config is not on PATH.
9. The state directories match verify_nova.sh test 13.
10. The bindings checks of the glance Test 13 pass. nova.storage.rbd_utils.rbd and nova.storage.rbd_utils.rados, the attributes RBDDriver.__init__ checks, are not None. rbd help export and rbd help import exit 0, and the ceph --version line starts with ceph version ; the script logs that line and asserts no version.
11. The tools nova and os-brick run as root resolve: one command -v probe per tool of the noble closure, /sbin/mount.nfs, /sbin/iscsiadm and /lib/udev/scsi_id are executable, and modprobe --version lists +ZSTD and +XZ, so the host's compressed modules load.

The 2026.1 and 2026.2 images pass all 90 assertions. Pointed at the nova control-plane image, the script exits 1: test 2 reports ModuleNotFoundError: No module named 'libvirt', and tests 4, 6, 10 and 11 fail on the missing tools, posture and bindings.

Release-independent images

These images ship software from outside the OpenStack release matrix. They have no key in releases/*/source-refs.yaml and no lock under nix/locks/, and are built once per upstream version instead of once per release.

ovn

Location: images/ovn/source.json (the pin), images/ovn/ovn.nix (the build), images/ovn/image.nix (the image)

OVN and Open vSwitch daemons and client tools, compiled by Nix from pinned upstream commits and shipped on the FHS façade of nix/lib/fhs-layout.nix (Nix Image Build).

Property	Value
Base image	none; the root flake builds the image from nixpkgs-images
Version pin	"version": "v26.03.2" in images/ovn/source.json, content-pinned by rev and hash
Open vSwitch	The commit the ovs submodule gitlink names at that OVN commit, ovs.rev with ovs.hash
User	openstack (UID 42424, GID 42424), the fakeNss entry of the façade
Entrypoint	None; every workload names the daemon it runs

Build (images/ovn/ovn.nix):

• Fetches OVN at rev and OVS at ovs.rev with fetchFromGitHub, each by commit and NAR hash, and puts the OVS tree where the empty submodule directory was
• Builds with the requirements of OVN's Documentation/intro/install/general.rst: autoconf, automake, libtool, pkg-config, OpenSSL, unbound and libcap_ng, plus the python3 the OVS code generation needs
• Builds OVS, then OVN against the OVS source tree (--with-ovs-source). Both configure with --prefix=$out --localstatedir=/var --sysconfdir=/etc
• Drops the headers, the static libraries, the man pages, $out/etc, the OVS Python package, the bugtool plugins and every script whose first line names python3; disallowedRequisites keeps python3 out of the closure
• Fails in its installCheckPhase when ovn-northd --version names another version than the pin

Image (images/ovn/image.nix):

• The façade with /usr/share/ovn and /usr/share/openvswitch linked into the store path of the build, the build itself, and the packages listed below
• /var/run links to /run. /run/openvswitch, /run/ovn, /var/lib/openvswitch, /var/lib/ovn, /var/log/openvswitch, /var/log/ovn, /etc/openvswitch and /etc/ovn exist, all owned by 42424
• The OCI labels title ovn, description "OVN and Open vSwitch daemons and client tools built from pinned upstream sources", licenses Apache-2.0, vendor SAP SE and version 26.03.2; User 42424:42424

Installed paths:

Path	Contents
/bin	ovn-northd, ovn-controller, ovn-nbctl, ovn-sbctl, ovn-appctl, ovsdb-tool, ovsdb-client, ovsdb-server, ovs-vswitchd, ovs-vsctl, ovs-appctl, ovs-ofctl, and the commands of the packages below
/usr/share/ovn	OVN's OVSDB schemas and scripts/ovn-ctl, a link into the store
/usr/share/openvswitch	The OVS OVSDB schemas and scripts/ovs-ctl, a link into the store

Runtime packages (nixpkgs):

Package	Purpose
coreutils, findutils, gnugrep, gnused, gawk	The commands ovn-ctl, ovs-ctl and the operator's scripts run (id, install, date, seq, find, grep, sed, awk)
hostname	hostname -f, which the database reconciler runs
iproute2	ip, which the chassis DaemonSets of issue #903 use to inspect interfaces
iputils	ping, which the OVN datapath e2e probes send across the Geneve tunnel; they run as root, since a store path carries no file capability
kmod	modprobe, which those DaemonSets use to load host kernel modules from the /lib/modules they mount
glibc	ldd, which verify_ovn.sh runs on the daemons

OpenSSL, unbound and libcap_ng are in the image as the libraries the daemons link against.

Version output: OVN programs print <prog> 26.03.2 on --version, followed by Open vSwitch Library 3.7.0, the OVS revision they were built against. OVS programs print <prog> (Open vSwitch) 3.7.0. Those two numbers agreeing is what the single pin buys, and tests/container-images/verify_ovn.sh compares them.

Non-root by default: ovsdb-server, ovn-northd and the client tools need no privileges, so the image runs as openstack. The chassis pods of issue #903 set runAsUser: 0 and the capabilities they need in their own pod spec.

Version pin: Open vSwitch has no version of its own to choose. It follows the ovs submodule gitlink (decision D1 of issue #898): one version line to bump, one Renovate rule. OVN's Documentation/intro/install/general.rst notes under Build Requirements that the submodule is "not recommended to be used as a source for OVS build"; ovn-kubernetes builds its OVS from that gitlink in dist/images/Dockerfile.fedora, and this image does the same.

version names a git tag, which is a mutable ref: upstream can move or delete it. The build therefore never resolves the tag. It fetches rev, the commit the tag names, and ovs.rev, the commit the ovs gitlink names at rev, each by its NAR hash. make nix-lock reads the gitlink out of a shallow, blobless fetch of rev and writes ovs.rev and both hashes, and the nix-lock-check job fails while the committed pin differs from what it writes. Neither a moved OVN tag nor a changed gitlink can alter the image without a change in this repository, and a rev that does not build the tagged version fails the build's version guard.

A Renovate customManager tracks ovn-org/ovn github-tags on the version and rev keys of images/ovn/source.json, so a tag pull request moves both. Major and minor bumps are disabled to hold the image on the 26.03 LTS line, and digest updates are disabled too. Patch bumps wait a three-day cooldown and are not automerged: the pull request fails nix-lock-check until someone runs make nix-lock on its branch, which writes the hashes and the OVS commit. See Dependency Management.

Image contract check: tests/container-images/verify_ovn.sh runs nine tests against a built image. Five cover the software: the pinned OVN version on ovn-northd, ovn-controller, ovn-nbctl and ovn-sbctl, the OVS daemons and clients, the shipped OVS matching the revision OVN links against, ovn-ctl and ovs-ctl running under the image's shell, and ovsdb-tool create building a database from each of ovn-nb.ovsschema, ovn-sb.ovsschema and vswitch.ovsschema — the schemas are the runtime artifact this image exists to serve, and no other assertion here reads them. The other four cover the packaging: ldd free of unresolved libraries with libssl.so.3 linked into the TLS-speaking daemons, no build toolchain and no development headers (assert_no_build_tools of tests/lib/build_tools.sh lists the store too), UID 42424 with writable state directories, and modprobe, ip, ping, hostname and find on PATH, with hostname -f exiting 0.

keystone-federation-proxy

Location: images/keystone-federation-proxy/source.json (the pin), images/keystone-federation-proxy/mod_auth_openidc.nix (the module build), images/keystone-federation-proxy/httpd-base.conf (the Apache scaffold), images/keystone-federation-proxy/image.nix (the image)

The Apache reverse proxy that terminates OIDC and SAML in front of Keystone, built by Nix. The keystone operator projects it into the Keystone API pod when an OIDC or SAML KeystoneIdentityBackend is attached.

• mod_auth_openidc is compiled from the upstream commit source.json pins, with Redis support through hiredis and without brotli and jq, and upstream's make check runs in the build.
• Apache and mod_auth_mellon come from the nixpkgs-images revision flake.lock pins.
• Both modules are built with their modules/ directory stripped, and a closure guard fails the build when the debug references to httpd's dev output or gcc come back.

The image is the FHS façade of nix/lib/fhs-layout.nix without a Python environment, plus coreutils. /usr/sbin/apache2 is nixpkgs' httpd, the command the operator runs, and /etc/keystone-federation-proxy/httpd-base.conf loads the modules by their store paths, mod_unixd and mod_log_config included. /etc/keystone-federation-proxy/ with its conf.d/ and metadata/ mount points, /var/www/html and /tmp (mode 1777) are real directories. The image runs as UID 42424 and sets no Cmd and no Entrypoint; the operator renders the apache2 -DFOREGROUND command, and the scaffold listens on port 5050.

The move from the Docker image changed three versions:

Component	Docker image (Ubuntu noble)	Nix image
Apache httpd	2.4.58	2.4.69
mod_auth_mellon	0.19.0	0.19.1
mod_auth_openidc	2.4.15.1	2.4.20.3

mod_auth_openidc 2.4.20 stretches OIDCCryptoPassphrase with PBKDF2, so the session and state cookies the Docker image issued fail to decrypt once on the Nix image. A browser in the middle of a login or with a websso session logs in again once; bearer-token CLI authentication is unaffected. A backend with OAuth 2.0 introspection logs upstream's startup warning that the OAuth 2.0 Resource Server functionality is deprecated in favour of mod_oauth2. A ControlPlane on the default ghcr.io/c5c3/keystone-federation-proxy:latest picks the image up on the next pod start.

nix build .#keystone-federation-proxy-image
./result | docker load

# Run the full image contract check
bash tests/container-images/verify_keystone_federation_proxy.sh keystone-federation-proxy:latest

Nix Image Build describes the module build and its guards. The build, verification and tag scheme are described in build-keystone-federation-proxy / merge-keystone-federation-proxy-image.

backup-shifter

Location: images/backup-shifter/image.nix

The rclone shifter for the OVN database backups, built by Nix: the FHS façade of nix/lib/fhs-layout.nix without a Python environment, plus rclone and coreutils. It is the shifter container of the OVNCentral backup CronJob and runs only when spec.backup.s3 is set, copying the northbound and southbound snapshots the backup init container left on the PVC to the configured bucket. rclone comes from the nixpkgs-images revision flake.lock pins, so the image has no version pin of its own. The image sets no Cmd and no Entrypoint: the operator renders the rclone copy command and the RCLONE_S3_* environment onto the CronJob container.

nix build .#backup-shifter-image
./result | docker load

# Run the full image contract check
bash tests/container-images/verify_backup_shifter.sh backup-shifter:latest

Nix Image Build describes the flake, the shared library and the local commands. The build, verification and tag scheme are described in build-backup-shifter / merge-backup-shifter-image.

libvirt

Location: images/libvirt/image.nix and images/libvirt/qemu.nix, with the libvirt of nix/pkgs/libvirt.nix

The libvirt daemon and QEMU for hypervisor nodes whose host image has neither, built by the root flake as libvirt-image (Hypervisor images). libvirt and QEMU come from the nixpkgs-images revision flake.lock pins, so the image has no release axis, and a re-lock moves both: libvirt 12.2.0 and QEMU 10.2.4 at the current pin. The Docker image the lab ran before #1407 carried noble's libvirt 10.0.0 and QEMU 8.2.2. The hypervisor package of issue #1142 runs the image privileged as a DaemonSet beside nova-compute. The libvirt-e2e DaemonSet of the e2e-nova-libvirt CI job is its second consumer: it runs the image on a kind node, where QEMU emulates the guest without KVM.

Content	Why it is in the image
nix/pkgs/libvirt.nix	libvirtd, virtlogd, virsh and virt-admin, each linked into /bin. Built with --sysconfdir=/etc, so libvirtd reads /etc/libvirt/libvirtd.conf and /etc/libvirt/qemu.conf, keeps its secrets below /etc/libvirt/secrets and finds its TLS files below /etc/pki. nixpkgs' wrapper puts dnsmasq, iptables, nftables, lvm2 and the other helpers libvirt runs on the daemon's PATH
images/libvirt/qemu.nix	qemu-system-x86_64, also at /usr/bin/qemu-system-x86_64, and qemu-img. The x86_64-softmmu target alone, with the rbd block driver for a Cinder volume on Ceph; built without libiscsi and curl, and without the GTK, SDL, OpenGL, PulseAudio, PipeWire, JACK, SPICE and smartcard parts noble kept in Recommends
dmidecode	libvirt reads the host's SMBIOS (firmware system information) data with it
iproute2	ip
kmod	modprobe and rmmod, for the vhost_net and NFS client modules of the host's module tree; nixpkgs builds kmod with xz and zstd and searches /lib/modules last
systemd	/usr/bin/systemctl and /usr/bin/systemd-run, the commands the lab's libvirtd.sh starts libvirtd with
coreutils, findutils, gnugrep, gnused, gawk, getent	the commands of the lab scripts (sha256sum, comm, cut, sort, tr, awk, grep) and of the contract check (getent group)

QEMU is built without modules, so every block driver is compiled in, and the choice is made at configure time: qemu-img --help lists rbd and none of curl, http, https, ftp, ftps, iscsi, nfs or ssh. QEMU would otherwise connect to a URL that a disk or a backing file names, from the node's address. QEMU's NBD client is compiled in, so a disk or backing file that names an NBD server still makes QEMU connect to it; Nova rejects a downloaded Glance image that has a backing file. The rbd driver links the librbd of Ceph 20.2.4 and adds ceph.lib, about 1 GB, to the closure; the store paths of the image hold about 1.99 GB on x86_64-linux.

Firmware: the UEFI firmware is QEMU's own edk2 build. /usr/share/qemu holds QEMU's share/qemu, where libvirt's qemu_datadir looks for firmware descriptors, with copies of the descriptors that name /usr/share/qemu/<file> instead of the store path: libvirt records the descriptor's path in a UEFI guest's XML, and the next image has another store path. /usr/share/OVMF/OVMF_CODE_4M.fd and /usr/share/OVMF/OVMF_VARS_4M.fd, the names noble's ovmf package used, link to edk2-x86_64-code.fd and edk2-i386-vars.fd.

libvirt and nova-compute: the nova-compute image builds libvirt-python against the same nix/pkgs/libvirt.nix derivation, so the client and the daemon are one libvirt, and checks.<system>.libvirt-pairing fails nix flake check when either side takes another one. tests/container-images/verify_libvirt.sh checks that libvirtd --version reports 12; the major moves with the nixpkgs-images pin. libvirtd prints the name it was started by, and nixpkgs' wrapper starts it under the wrapper's path, so the line reads /sbin/libvirtd (libvirt) 12.2.0 when PATH finds it.

What the image does not carry: the image has no Cmd and no Entrypoint. The consumer renders libvirtd.conf, qemu.conf and the start command. /etc/libvirt is a writable copy of libvirt's default configuration, qemu.conf with noble's mode 0600, and keeps the definition of the default NAT network at /etc/libvirt/qemu/networks/default.xml without its autostart link. The consumer runs libvirtd in the host's network namespace, where that network would create virbr0 and NAT rules on the node.

Runs as root: libvirtd has to run as root, so the image sets no User and creates no openstack user (see Design Deviations). fakeNss writes the guest accounts noble's packages created, with noble's IDs: the libvirt-qemu user (UID 64055, primary group kvm) and the libvirt (103) and kvm (104) groups. nixpkgs compiles root as libvirt's default QEMU user, and the lab and the e2e suite set user = "root" in qemu.conf. The consumer sets the socket's ownership in its own configuration.

The same applies to /dev/kvm. A privileged container gets its own device node with the host's group ID and mode. Debian and Ubuntu hosts create it as root:kvm with mode 0660, and their kvm group ID usually differs from the image's, so QEMU cannot open the device and a guest with virt_type=kvm does not start. Before it starts libvirtd, the consumer runs chown root:kvm /dev/kvm && chmod 0660 /dev/kvm on the container's own device node. On a hostPath mount of the host's /dev the same command would change the host's device.

Tags: the image streams as libvirt:latest. CI publishes ghcr.io/c5c3/libvirt as <sha>, and on main as latest and <libvirt-version>-r<N>, such as 12.2.0-r57, where N counts the commits under images/libvirt, nix/pkgs/libvirt.nix, nix/lib/image-defaults.nix and flake.lock (see the tag table). Every run of build-images.yaml on main that is not a pull request rebuilds the image and moves latest to the new manifest. The keeper tag is created once per value and never moved, so it stays on the first build that had it, and Retention keeps every manifest that carries one, the <libvirt-package-version>-r<N> tags of the Docker-built image included. A manifest left with its <sha> tag alone is deleted once it is older than 24 hours. A consumer therefore pins the keeper tag by digest, as the lab does with ghcr.io/c5c3/libvirt:<tag>@sha256:<digest>.

nix build .#libvirt-image && ./result | docker load

# Run the full image contract check
bash tests/container-images/verify_libvirt.sh libvirt:latest

The contract check runs without --privileged. It checks the libvirt 12 version line, the QEMU and qemu-img versions, the tools on PATH and the commands of the lab scripts (modprobe --version listing +ZSTD and +XZ among them), the firmware files and the x86_64 descriptor, root, the guest accounts, the network definition without its autostart link, and the absence of build tools, git and python3. Its daemon smoke test names a log file in /etc/libvirt/libvirtd.conf, starts libvirtd in the container, waits at most 30 seconds for /run/libvirt/libvirt-sock, and asks the QEMU driver for its version and its x86_64 domain capabilities: the emulator path, the efi firmware value, and a written log file, which proves that libvirtd reads /etc/libvirt. Its rbd test reads the driver list of qemu-img --help and runs qemu-img info rbd:volumes/x: librados runs and fails with error connecting, because no ceph.conf names a monitor. Its secret test makes the calls of the lab's ceph-secret.sh (Lab hypervisors) against libvirtd: it defines a private, ephemeral ceph secret, sets its value with virsh secret-set-value --file from a base64 key with a trailing newline, and checks that secret-get-value refuses it, that a second secret, not private, reads the key back unchanged, and that /etc/libvirt/secrets stays empty.

Its build, verification and tag scheme are described in build-libvirt / merge-libvirt-image.

openstack-hypervisor-operator

Location: images/openstack-hypervisor-operator/source.json (the pin), manager.nix and patch-checks.sh (the build), image.nix (the image)

openstack-hypervisor-operator (hvo) from cobaltcore-dev, compiled by Nix from a pinned commit of its upstream main branch with the patches under images/openstack-hypervisor-operator/patches/ applied. The hypervisor package of issue #1142 deploys it with the upstream Helm chart of the same commit: deploy/lab/metal-stack/hypervisor/hvo-release.yaml runs it on the metal-stack lab (see Lab hypervisors).

Property	Value
Build	buildGo127Module of nixpkgs-images (Nix Image Build)
Runtime base	none; nix/lib/static-image.nix, the stand-in for gcr.io/distroless/static:nonroot
Version pin	rev in images/openstack-hypervisor-operator/source.json, a 40-character commit of upstream main (hack/ci-resolve-hvo-commit.sh prints it), with hash and vendorHash
User	65532:65532, the nonroot user of the distroless base
Entrypoint	/usr/bin/manager
Label	io.c5c3.upstream-commit, set to the pinned commit

Build (manager.nix):

• Fetches the pinned commit of https://github.com/cobaltcore-dev/openstack-hypervisor-operator by commit and NAR hash, so no branch or tag decides what is compiled, and downloads the Go modules into a fixed-output derivation checked against vendorHash
• Applies every *.patch under patches/, in name order, to the module download and the build alike. A patch that does not apply fails the patch phase
• Runs patch-checks.sh in its checkPhase, in the patched tree with the vendored modules. check_live_migrate_opts fails when Go code outside the tests and outside vendor/ still names gophercloud's servers.LiveMigrateOpts, whose BlockMigration cannot carry "auto". The grep matches the bare type name, so a literal, a var, new(), a pointer and an aliased import all fail. Only a // outside a double-quoted string before the name lets the line pass, so a comment passes while a / in a string or a division does not. That catches a live migration a pin move adds or reshapes, and a re-cut patch that lost its controller.go hunk; a grep error fails the build too
• Runs TestLiveMigrateAutoBody, the test patch 0001 brings, and fails unless the log shows --- PASS: TestLiveMigrateAutoBody. go test -run exits 0 with [no tests to run] when the test is missing, so require_pass is what catches a patch that lost its test hunk
• Runs TestHypervisorCreatedWithDefaultHighAvailability and TestTraitsInSyncSetsTraitsUpdated, the tests of patches 0002 and 0003, in one go test of ./internal/controller/, and fails unless the log shows the top-level --- PASS: line of each. The -run expression does not match the package's Ginkgo entry point, TestControllers in suite_test.go, so the run needs no envtest binary. The space after each test name in the grep keeps a subtest's PASS line from matching
• Runs TestServiceClientInterface, the test of patch 0004, in a go test of ./internal/openstack/, and fails unless the log shows --- PASS: TestServiceClientInterface with the same trailing space. The -run expression does not match that package's Ginkgo entry point, TestOpenstack in suite_test.go, so the suite does not run
• Builds ./cmd with CGO_ENABLED=0 and upstream's ldflags, with the version set to sha-<commit>, the tag the image is published under, and installs it as bin/manager. manager --version therefore prints manager sha-<commit> (linux/<arch>) <commit>. Upstream's generate step is skipped: the generated files are committed

nix log .#openstack-hypervisor-operator-image.manager shows the --- PASS: lines of a build.

Why main: main has the required --agent-namespaces flag, the offboarding taint and parallel migrations in an Eviction (--eviction-concurrency). The latest tag, v1.2.3, has none of them. Upstream publishes a chart for every main commit, version 1.2.3+sha-<short> with appVersion sha-<full commit>, and that chart renders the image as <repository>:<appVersion>. Upstream pushes no image under that tag for a main commit, only a moving latest. This image is published as sha-<commit>, so the chart of the pinned commit runs it with only the repository overridden.

Source patches: four files under images/openstack-hypervisor-operator/patches/, applied in order. Each brings a plain Go test outside upstream's Ginkgo suite, which the build step runs.

0001-eviction-let-nova-choose-block-migration.patch. Upstream's liveMigrate (internal/controller/eviction/controller.go) asks Nova for a live migration with block_migration: false. Nova refuses that for a server on the hypervisor's local disks with InvalidSharedStorage. Every server on the metal-stack lab boots from a local disk, so the Eviction cannot move it. From compute microversion 2.25 on, Nova accepts the string "auto" and chooses block migration per server. gophercloud types LiveMigrateOpts.BlockMigration as *bool, so the patch adds a LiveMigrateOptsBuilder of its own that sends {"os-migrateLive": {"block_migration": "auto", "host": null}}, plus the plain Go test TestLiveMigrateAutoBody that pins the body; no upstream test pins it.

0002-hypervisor-make-the-high-availability-default-configurable.patch. Upstream's HypervisorController.Reconcile (internal/controller/hypervisor_controller.go) creates every Hypervisor with spec.highAvailability: true, and onboarding then waits for the HaEnabled condition, which only SAP's kvm-ha-service sets. The patch adds the flag --default-high-availability (default true, upstream's behaviour) and writes its value on create; an existing Hypervisor keeps its value. TestHypervisorCreatedWithDefaultHighAvailability reconciles a Node with the fake client and asserts true for the default and false once the flag's variable is false. The lab release sets the flag to false.

0003-traits-report-traitsupdated-when-nothing-differs.patch. Upstream's TraitsController.Reconcile (internal/controller/traits_controller.go) sets TraitsUpdated only on the path that calls Placement, so only when a custom trait differs, while onboarding and the aggregates controller wait for TraitsUpdated=True. The patch sets the condition, reason Succeeded and the message Custom traits are in sync, when nothing differs, without calling Placement and without writing status.traits. TestTraitsInSyncSetsTraitsUpdated runs the controller with no Placement client: the condition in Handover, no second write on a repeat, and no condition in Testing.

0004-openstack-select-the-catalog-interface-with-os-interface.patch. Upstream's ServiceClientFromProvider (internal/openstack/service_client.go) builds every OpenStack client of the operator from an empty gophercloud.EndpointOpts, so hvo reads the catalog's public endpoints and no other. The patch reads the environment variable OS_INTERFACE: public, internal or admin selects that interface, and unset or empty keeps public, upstream's behaviour. Any other value, Internal included, is an error before the catalog is read, and hvo exits at start. An error of the catalog lookup, gophercloud's ErrEndpointNotFound when the catalog has no endpoint of that interface, is returned unchanged. TestServiceClientInterface builds a compute client on a provider whose endpoint locator records the options it gets: for each of the three values, for unset and empty, for Internal, internalURL and internal with a leading space or a trailing newline (no lookup happens) and for a locator error. The lab release sets internal, whose endpoints are the in-cluster Service URLs.

Each patch header records Upstream status: not submitted. Their author submits them upstream, and issue #1066 tracks them until main carries them.

Tags: CI publishes ghcr.io/c5c3/openstack-hypervisor-operator as sha-<hvo-commit>-<sha> on every push, and on main also as sha-<hvo-commit>, upstream-<hvo-commit> and latest (see the tag table). The upstream chart resolves sha-<hvo-commit>. That shape is a build artifact to Retention, which would delete it once a newer pin holds latest, while a chart of the older pin still names it. upstream-<hvo-commit> sits on the same manifest and is a keeper tag, so every pin that reached main stays pullable under both tags.

Pin moves: Renovate tracks the rev key of images/openstack-hypervisor-operator/source.json as a git-refs digest of upstream main, weekly and without automerge (see Dependency Management). The pull request fails nix-lock-check until make nix-lock writes hash and vendorHash for the new commit. Outside the Nix build and hack/nix-lock.sh, hack/ci-resolve-hvo-commit.sh is the only parser of the pin in CI. A new commit on which a patch no longer applies fails the build in the patch phase; that patch is then re-cut against the new commit. Once upstream carries a patch's change, the patch is dropped together with its checks in patch-checks.sh: for 0001 check_live_migrate_opts and the TestLiveMigrateAutoBody run, for 0002 and 0003 the test's name in the controller go test run and its require_pass, and that run itself once neither test is left; for 0004 the TestServiceClientInterface run and its require_pass. With the last patch gone, patches/ can go too. A pin move also checks what Connect a Compute Cluster, Drain a Compute Node and the table Namespaces on a compute cluster say about hvo against the new commit, and updates the commit the two guides name.

Image contract check: tests/container-images/verify_hvo.sh runs five tests against a built image. manager --version names sha-<pin> and ends with the pin. manager --help lists -agent-namespaces and -eviction-concurrency, which v1.2.3 does not have, and -default-high-availability, which only patch 0002 brings. The image runs /usr/bin/manager as 65532:65532, and its io.c5c3.upstream-commit label equals the pin. The script reads the usage text instead of starting the binary without flags: upstream logs its --agent-namespaces is required error before it installs a logger, so that message never reaches the output. The fifth test, test_static_image_layout of tests/lib/static_image.sh, checks the files and the configuration of the distroless base that nix/lib/static-image.nix stands in for.

Its build, verification and tag scheme are described in build-hvo / merge-hvo-image / verify-hvo-image.

kvm-node-agent

Location: images/kvm-node-agent/source.json (the pin), manager.nix and patch-checks.sh (the build), image.nix (the image)

kvm-node-agent (kna) from cobaltcore-dev, compiled by Nix from a pinned commit of its upstream main branch with the patches under images/kvm-node-agent/patches/ applied. The agent runs on every hypervisor node: it reports libvirt's state into the node's Hypervisor and installs the node's libvirt TLS files under the host's /etc/pki. deploy/lab/metal-stack/hypervisor/kna-release.yaml runs it with the upstream Helm chart of the same commit (see Lab hypervisors).

Property	Value
Build	buildGo127Module of nixpkgs-images (Nix Image Build)
Runtime base	none; nix/lib/static-image.nix, the stand-in for gcr.io/distroless/static:nonroot
Version pin	rev in images/kvm-node-agent/source.json, a 40-character commit of upstream main (hack/ci-resolve-kna-commit.sh prints it), with hash and vendorHash
User	0:0 (see Design Deviations)
Entrypoint	/usr/bin/manager
Label	io.c5c3.upstream-commit, set to the pinned commit

Build (manager.nix):

• Fetches the pinned commit of https://github.com/cobaltcore-dev/kvm-node-agent by commit and NAR hash, downloads the Go modules against vendorHash and applies every *.patch under patches/, as for openstack-hypervisor-operator
• Runs patch-checks.sh in its checkPhase. The script refuses to run as root and runs the two tests patch 0001 brings in one go test of ./internal/certificates/: TestUpdateTLSCertificateKeyMode and TestUpdateTLSCertificateKeyGroupNotPermitted. The build fails unless the log shows each test's --- PASS: line followed by a space, which a subtest's line does not match. A test binary exits 0 with testing: warning: no tests to run when a test is missing, so require_pass is what catches a patch that lost its test file. The second test skips as root and prints --- SKIP; Nix builds as a build user, and refuse_root fails a build that runs as root before any test. tests/unit/images/kna_patch_test_guard_test.sh runs both functions against such logs and a stubbed id
• Builds ./cmd with CGO_ENABLED=0 and upstream's ldflags without the build date, with the version set to sha-<commit>, the tag the image is published under, and installs it as bin/manager. manager --version therefore prints manager version sha-<commit> (); the parentheses hold the unset build date. Upstream's generate step is skipped: the generated files are committed

Why main: upstream publishes a chart for every main commit, version 0.2.0+sha-<short> with appVersion sha-<full commit>, and that chart renders the image as <repository>:<appVersion>. Upstream pushes no image under that tag: ghcr.io/cobaltcore-dev/kvm-node-agent:sha-<commit> answers 404. A fix the agent needs on hosts outside SAP ships here as a patch while it is proposed upstream. This image is published as sha-<commit>, so the chart of the pinned commit runs it with only the repository overridden.

Source patch:images/kvm-node-agent/patches/0001-certificates-restrict-private-key-file-modes.patch. Upstream's UpdateTLSCertificate (internal/certificates/manage_libvirt.go) writes every file of the node's TLS Secret with mode 0644, the private key included (#1175). Every local user of the node can then read the key that authenticates it to every libvirtd of its migration domain. With the patch:

• libvirt/private/serverkey.pem, qemu/server-key.pem and ch/server-key.pem get mode 0600, and the six certificate files keep 0644. A key target that a later upstream commit adds gets 0600 too.
• When the environment variable PKI_KEY_GROUP holds a numeric group ID, qemu/server-key.pem and ch/server-key.pem get that group and mode 0640. With native TLS migration, QEMU opens its key itself, as the user it runs as, and Cloud Hypervisor reads its own the same way. libvirt/private/serverkey.pem keeps mode 0600, but it holds the same key, and libvirt's client key links to it: the group can read the key the node authenticates with to the libvirtd of its peers as well. Only a number is accepted, because the distroless image has no group database, and an invalid value fails the update before any file is written. The agent has to belong to the group or hold CAP_CHOWN; otherwise the update fails with operation not permitted and replaces no file. Unset or empty, the variable changes no group. The upstream chart has no value for it, so a host that needs it sets it with a post-renderer.
• Every file is written to a temporary file in its own directory, and the temporary files are renamed over their targets only once all of them are written. os.WriteFile keeps the mode of a file that already exists, so the rename is what tightens a key an earlier version left at 0644. It needs write permission on the directory and no CAP_FOWNER, and no reader sees a partly written key. A write that fails leaves the files of the previous Secret in place instead of a new certificate beside an old key.

The agent writes the files only when the Secret's resourceVersion changes, so a node keeps the keys it has until its certificate is reissued. TestUpdateTLSCertificateKeyMode walks the PKI directory in eight subtests and fails for any file with the key's content that is not a key target, for a key file with another mode or group, and for any other file whose mode is not 0644. TestUpdateTLSCertificateKeyGroupNotPermitted sets a group the process does not belong to and expects a failed update that leaves every file of the previous Secret in place and no temporary file behind. A root process may give a file any group, so that test skips as root, and the build runs it as a Nix build user. The patch header records Upstream status: not submitted. Its author submits it upstream; #1175 stays open until upstream main carries the change, and issue #1066 tracks it.

Tags: CI publishes ghcr.io/c5c3/kvm-node-agent as sha-<kna-commit>-<sha> on every push, and on main also as sha-<kna-commit>, upstream-<kna-commit> and latest (see the tag table). The upstream chart resolves sha-<kna-commit>, and upstream-<kna-commit> keeps that manifest through Retention, as for openstack-hypervisor-operator.

Pin moves: Renovate tracks the rev key of images/kvm-node-agent/source.json as a git-refs digest of upstream main, weekly and without automerge (see Dependency Management). The pull request fails nix-lock-check until make nix-lock writes hash and vendorHash for the new commit. Outside the Nix build and hack/nix-lock.sh, hack/ci-resolve-kna-commit.sh is the only parser of the pin in CI. The lab's chart ref in deploy/lab/metal-stack/hypervisor/sources.yaml moves with the pin by hand, and tests/unit/deploy/metal_stack_hypervisor_test.sh fails while the chart's short SHA differs from it. A new commit on which the patch no longer applies fails the build in the patch phase; the patch is then re-cut against the new commit. Once upstream carries the change, the patch goes together with its two test runs in patch-checks.sh, and with the last patch patches/ can go too.

Image contract check: tests/container-images/verify_kna.sh runs five tests against a built image. The first line of manager --version is manager version sha-<pin> (). manager --help exits 0 and lists -health-probe-bind-address, the flag the chart passes; a binary that needed a C library would not start on the static base at all. The image runs /usr/bin/manager as 0:0, and its io.c5c3.upstream-commit label equals the pin. The fifth test is the static-image check of verify_hvo.sh.

Its build, verification and tag scheme are described in build-kna / merge-kna-image / verify-kna-image.

Release Configuration

All release-specific configuration lives under releases/<release>/ (e.g., releases/2026.1/). These files are the single source of truth for what gets built. Adding a new service or updating a version requires editing only these files and running make nix-lock, which writes the locks under nix/locks/ from them.

source-refs.yaml

Location: releases/<release>/source-refs.yaml

Maps each OpenStack component to a git ref (tag, branch, or commit SHA) specifying the version to build.

Format:

# SPDX-FileCopyrightText: Copyright 2026 SAP SE or an SAP affiliate company
#
# SPDX-License-Identifier: Apache-2.0

keystone: "28.0.0"

Each key is a service name matching the image directory under images/. Values are quoted strings representing git refs — typically release tags (e.g., "28.0.0").

To add a new service, add a single line: <service>: "<git-ref>".

upper-constraints.txt

Location: releases/<release>/upper-constraints.txt

Contains pinned Python dependency versions from the OpenStack requirements repository (stable/<release> branch). This file is committed as-is from the upstream repository to enable Renovate tracking and git diff for constraint changes.

Format:

cryptography===44.0.0
oslo.limit===2.8.0
keystonemiddleware===10.9.0

Each line pins a single package using the === (arbitrary equality) operator. This is the standard format used by OpenStack's global requirements process.

Source: https://raw.githubusercontent.com/openstack/requirements/stable/<release>/upper-constraints.txt

extra-packages.yaml

Location: releases/<release>/extra-packages.yaml

Defines per-service Python extras and additional pip packages that are not part of the core OpenStack package. hack/nix-lock.sh reads both into each lock under nix/locks/.

Format:

# SPDX-FileCopyrightText: Copyright 2026 SAP SE or an SAP affiliate company
#
# SPDX-License-Identifier: Apache-2.0

keystone:
  pip_extras:
    - ldap
  pip_packages: []

Key	Purpose
<service>.pip_extras	Bare Python extra names combined with the project name to form the dependency of the lock (e.g. keystone[ldap]).
<service>.pip_packages	Additional pip packages locked and installed alongside the service. Use an empty list ([]) when none are needed.

No block carries apt_packages: every image names its runtime packages in images/<name>/image.nix, and verify_release_config.sh fails a block that carries the key.

To add packages for a new service, add a new top-level key matching the service name with a pip_extras list, then run make nix-lock.

nova-compute is the one key that is not a service. It configures the image of images/nova-compute/, which is built from the nova lock and has no source-refs.yaml key of its own: hack/nix-lock.sh writes its pip_extras and pip_packages into the nova-compute dependency group of the nova lock. verify_release_config.sh checks the block, image.nix and the group through its DERIVED_IMAGES list.

Constraint Overrides

The constraint override system allows selective modification of individual package pins in upper-constraints.txt without replacing the entire file. This is useful for applying security fixes or version bumps for individual packages.

Override Format

Override files are placed at overrides/<release>/constraints.txt. Each line is one of three types:

Syntax	Action	Example
package===version	Set the pin for package: an existing pin is removed and the line is appended, so a package without a pin gets one	cryptography===44.0.1
-package	Remove package from constraints entirely	-oslo.messaging
# comment or blank	Skipped (no action)	# Security fix for CVE-2025-1234

Example override file (overrides/2026.1/constraints.txt):

# Security fix: bump cryptography for CVE-2025-1234
cryptography===44.0.1

# Remove oslo.messaging pin to allow newer version
-oslo.messaging

Real-world use — the horizon self-pin: upper-constraints.txt pins horizon=== itself (unlike keystone, which never appears there). A source install with --constraint refuses to install the horizon source tree against its own pin, so overrides/<release>/constraints.txt ships a -horizon removal line for every release. The git ref in source-refs.yaml stays the single source of truth for what is built, independent of the upstream pin.

Script Usage

Location: scripts/apply-constraint-overrides.sh

# Apply overrides for the 2026.1 release
./scripts/apply-constraint-overrides.sh 2026.1

Behavior:

Condition	Result
overrides/<release>/constraints.txt exists	Each line is processed: a package===version line deletes any existing pin with sed and is appended to the file, a -package line deletes the pin with sed
overrides/<release>/constraints.txt does not exist	Script exits with code 0, no changes made (idempotent)

The script reads releases/<release>/upper-constraints.txt relative to the current working directory (must be invoked from the repository root) and modifies it in-place. It uses GNU sed -i for modifications (default on Ubuntu/CI runners — BSD sed is not supported).

Arguments:

Argument	Required	Description
<release>	Yes	Release identifier (e.g., 2026.1), used to locate overrides/<release>/constraints.txt

Local Build Instructions

Every image builds with Nix:

nix build .#keystone-2026_1-image && ./result | docker load
bash tests/container-images/verify_keystone.sh keystone:2026.1

Every <name>-<rel_>-image attribute of Nix Image Build works the same way.

Building nova-compute locally

The compute image builds from the nova lock of the release, with the libvirt-python of its nova-compute group:

nix build .#nova-compute-2026_1-image && ./result | docker load

# Run the full image contract check
NOVA_COMPUTE_RELEASE=2026.1 bash tests/container-images/verify_nova_compute.sh nova-compute:2026.1

For 2026.2, build .#nova-compute-2026_2-image and check nova-compute:2026.2. Without NOVA_COMPUTE_RELEASE the contract script finds the release from the nova version inside the image. The first build per nixpkgs-images revision compiles libvirt, which libvirt-python builds against.

Building libvirt locally

nix build .#libvirt-image && ./result | docker load

# Run the full image contract check
bash tests/container-images/verify_libvirt.sh libvirt:latest

The first build per nixpkgs-images revision compiles libvirt and QEMU.

Building ovn locally

The build needs no source checkout and no token: the root flake fetches OVN at the pinned rev and Open vSwitch at the pinned ovs.rev by their hashes (Nix Image Build).

nix build .#ovn-image && ./result | docker load   # Loaded image: ovn:26.03.2

# Print the pin the build used (26.03.2, without the leading v)
hack/ci-resolve-ovn-version.sh

# Run the full image contract check
bash tests/container-images/verify_ovn.sh ovn:$(hack/ci-resolve-ovn-version.sh)

Both projects are compiled from source, so the first build takes a while unless the Cachix cache or the local store holds the build of the pin. nix build .#ovn-image.ovn builds OVS and OVN without the image.

Building openstack-hypervisor-operator locally

The build needs no source checkout and no token: the root flake fetches the pinned commit, applies the patches and runs their checks itself.

nix build .#openstack-hypervisor-operator-image && ./result | docker load

# Print the pinned upstream commit
hack/ci-resolve-hvo-commit.sh

# Run the full image contract check
bash tests/container-images/verify_hvo.sh \
  openstack-hypervisor-operator:sha-$(hack/ci-resolve-hvo-commit.sh)

nix log .#openstack-hypervisor-operator-image.manager prints the --- PASS: lines of TestLiveMigrateAutoBody, TestHypervisorCreatedWithDefaultHighAvailability, TestTraitsInSyncSetsTraitsUpdated and TestServiceClientInterface.

Building kvm-node-agent locally

The build needs no source checkout and no token: the root flake fetches the pinned commit, applies the patch and runs its two tests itself.

nix build .#kvm-node-agent-image && ./result | docker load

# Print the pinned upstream commit
hack/ci-resolve-kna-commit.sh

# Run the full image contract check
bash tests/container-images/verify_kna.sh kvm-node-agent:sha-$(hack/ci-resolve-kna-commit.sh)

nix log .#kvm-node-agent-image.manager prints the --- PASS: TestUpdateTLSCertificateKeyMode and --- PASS: TestUpdateTLSCertificateKeyGroupNotPermitted lines.

Design Deviations

The implementation deviates from the original design document in four areas, documented with # DEVIATION comments in the affected Nix expressions:

Generic openstack user instead of per-service users:

The original design's Keystone Dockerfile example creates a per-service user (e.g., groupadd keystone / useradd keystone). The implementation uses a single generic openstack user (UID/GID 42424) shared by all service images. This reduces complexity: no service image needs its own user creation step.

nix/lib/service-image.nix carries the full # DEVIATION paragraph: the Nix-built service and Tempest images run as the user that nix/lib/fhs-layout.nix writes. Each service's image.nix carries a short # DEVIATION comment pointing at it (images/keystone/image.nix, images/horizon/image.nix, images/glance/image.nix, images/placement/image.nix, images/barbican/image.nix, images/neutron/image.nix, images/cinder/image.nix, images/nova/image.nix, images/nova-compute/image.nix).

images/ovn/image.nix carries the comment for the other half of the same decision. It is built without nix/lib/service-image.nix, so it carries the openstack user and group itself, through the fakeNss entry of the FHS façade: the distro packages would install separate openvswitch and ovn users, and the source build carries no such packaging. The backup-shifter image states the same decision in the header comment of images/backup-shifter/image.nix, also through the façade, because rclone brings no service account at all. The keystone-federation-proxy image states it in images/keystone-federation-proxy/image.nix: the sidecar runs as the same user as the keystone container it fronts. One identity across all non-root images keeps the pod security contexts uniform, and in the OVNCentral backup CronJob it lets the backup init container and the shifter container share a single pod security context.

Root instead of the openstack user (libvirt):

images/libvirt/image.nix keeps root and creates no openstack user, and carries a # DEVIATION comment saying so. libvirtd has to run as root to manage domains, devices and cgroups on the node. fakeNss writes the guest accounts noble's libvirt packages created, with noble's IDs: the libvirt-qemu user and the libvirt and kvm groups.

Distroless and UID 65532 instead of the openstack user (openstack-hypervisor-operator):

images/openstack-hypervisor-operator/image.nix creates no openstack user, and carries a # DEVIATION comment saying so. The operator is a static Go binary, so it runs on nix/lib/static-image.nix, the stand-in for the gcr.io/distroless/static:nonroot base and the image the CobaltCore operator images run on (nix/lib/operator-image.nix), as that base's nonroot user, UID 65532. Upstream's own image is Alpine with UID 4200. Its chart sets only runAsNonRoot: true and no command, which any non-root user and the image's Entrypoint satisfy.

Distroless and UID 0 instead of the openstack user (kvm-node-agent):

images/kvm-node-agent/image.nix creates no openstack user, and carries a # DEVIATION comment saying so. The agent is a static Go binary on the same stand-in for the gcr.io/distroless/static:nonroot base, but it runs as root (User 0:0) instead of that base's nonroot user. kna authenticates to the host's system bus and starts units there, and a Debian host's dbus-daemon drops the connection of a UID it cannot resolve (#1167). Upstream's own image is Alpine with USER 42438:42438, a user that exists on SAP's hosts only. Every consumer of the image therefore runs the agent as root unless its pod sets runAsUser: the upstream chart sets none for the manager, and runAsNonRoot: true rejects the image.