Skip to content

Adopt an External Keystone

This guide walks a platform owner through putting an existing Keystone installation under operator management without deploying anything into it.

A ControlPlane in External mode is service-less: it projects no MariaDB, no Memcached, and no Keystone workload. It manages the identity plane against the Keystone API you already run — minting and rotating the admin application credential, provisioning declarative service accounts, and importing your existing service catalog rather than writing to it. Your installation keeps serving tokens throughout, and deleting the ControlPlane leaves it untouched.

This is the first step of a staged adoption. Taking over the database and the Keystone deployment itself are later, separate phases — see Brownfield Keystone Adoption.

Prerequisites

Devstack

This guide is written against the Quick Start (ControlPlane).

bash
KIND_HOST_PORT=8443 WITH_CONTROLPLANE=true make deploy-infra

Follow it through its final Verify step, so the operators and the shared infrastructure (OpenBao, External Secrets, K-ORC) are running. This guide adds a second, External-mode ControlPlane in its own namespace; the managed controlplane the tutorial creates in openstack stays as it is.

Beyond the devstack, adopting a real installation needs:

  • A reachable Keystone API. K-ORC dials it from inside the cluster. Nothing restricts the operator's egress by default, but on a cluster that enforces a restrictive egress policy you must explicitly allow traffic from the K-ORC namespace to the endpoint and port.
  • The admin password, as a Secret in the ControlPlane's namespace. The operator never invents it and never rotates it — you supply it and rotate it at the installation (see step 6).
  • A CA bundle Secret, if the endpoint uses a private CA. The key defaults to ca.crt. An IP-based authURL needs an IP SAN in the certificate; a hostname resolves through cluster DNS.
  • A spec.region that matches your catalog. The region and the selected interface must both exist in the external catalog, or the control plane fails loudly rather than importing nothing.

Standing in a brownfield Keystone on kind

On kind there is no pre-existing installation to adopt, so create one. These are the same fixtures the e2e suite uses: a plain, operator-free Keystone in namespace brownfield-keystone, serving http://keystone.brownfield-keystone.svc:5000/v3.

bash
kubectl apply -f tests/e2e/c5c3/external-keystone/00-fixture-keystone.yaml
kubectl -n brownfield-keystone rollout status deploy/keystone --timeout=5m

kubectl apply -f tests/e2e/c5c3/external-keystone/01-fixture-catalog-setup-job.yaml
kubectl -n brownfield-keystone wait --for=condition=Complete \
  job/keystone-fixture-setup --timeout=5m

The setup Job makes this look like a real installation rather than a fresh bootstrap:

  • a non-default admin identity — user brownfield-admin, project platform-admin, domain heimdall — so nothing relies on the admin/Default names a bootstrap would leave behind;
  • a duplicate identity-type service (keystone-legacy alongside keystone), which is what forces the catalog disambiguation in step 2.

Against a real installation, skip this section entirely.

Steps

1. Create the namespace and the admin-password Secret

The ControlPlane needs its own namespace (one ControlPlane per namespace), and the admin password of the existing installation as a Secret in it.

bash
kubectl create namespace brownfield

# Prompt without echo, then pipe the value in on stdin so the admin password
# never appears on argv (visible in /proc/<pid>/cmdline for the life of the
# process) or in your shell history file.
read -rs -p 'Keystone admin password: ' PW; echo
printf '%s' "$PW" | kubectl -n brownfield create secret generic brownfield-admin-password \
  --from-file=password=/dev/stdin
unset PW

Type your installation's real admin password at the prompt. On the kind devstack it is the fixture password brownfield_admin_fixture_pw_0, which the setup Job of the previous section gave brownfield-admin.

Nothing else in this guide reads the password directly — everything downstream authenticates with the application credential the operator mints from it.

2. Apply the External-mode ControlPlane

This is the manifest the e2e suite applies, imported from the suite itself. The walkthrough and the suite share the namespace brownfield, the ControlPlane name controlplane-external, and the admin-password Secret from step 1, so there is one manifest here rather than a hand-kept copy of one — see Tested by.

yaml
apiVersion: c5c3.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane-external
  namespace: brownfield
spec:
  openStackRelease: "2025.2"
  services:
    keystone:
      mode: External
      external:
        authURL: http://keystone.brownfield-keystone.svc:5000/v3
        catalog:
          identityServiceName: keystone
  korc:
    adminCredential:
      cloudCredentialsRef:
        cloudName: admin
        secretName: k-orc-clouds-yaml
      passwordSecretRef:
        name: brownfield-admin-password
        key: password
      userName: brownfield-admin
      projectName: platform-admin
      domainName: heimdall
      applicationCredential:
        restricted: true
        rotation:
          mode: PasswordDriven

That ControlPlane is the only thing in the file — the suite keeps its fixture admin-password Secret in a sibling file so applying this one cannot overwrite the Secret you filled in step 1 — so on the devstack apply it as it stands:

bash
kubectl apply -f tests/e2e/c5c3/external-keystone/02-controlplane-external.yaml

Against a real installation, copy it out and set external.authURL, spec.region, and the admin identity to yours.

Field by field:

FieldWhy
mode: ExternalSelects the service-less path. No Keystone workload is deployed and no child CR is projected.
external.authURLThe identity endpoint the operator manages against.
external.endpointTypeWhich catalog interface to authenticate against. Omitted here, so it defaults to public — the interface that is normally reachable from outside the installation.
external.caBundleSecretRefThe private-CA bundle, when the endpoint needs one. Omitted here: the kind fixture is plain HTTP — devstack only, see the warning below.
external.catalog.identityServiceNameDisambiguates the identity-service import. Only needed when your catalog holds more than one identity-type service — as the fixture does.
korc.adminCredential.cloudCredentialsRefWhere the minted credential is materialized — the clouds.yaml Secret and cloud entry read back in step 4. Spelled out here, but k-orc-clouds-yaml / admin are the defaults.
korc.adminCredential.passwordSecretRefThe Secret from step 1.
userName / projectName / domainNameThe admin identity to authenticate as. They default to admin / admin / Default; the fixture uses a non-default identity, so all three are set.
applicationCredential.restrictedKeeps the minted credential least-privilege — it cannot mint further application credentials. Spelled out here, but true is the default.
applicationCredential.rotation.modePasswordDriven keys the re-mint on a hash of the admin password — see step 6. Spelled out here, but it is the default.

spec.openStackRelease stays required but is advisory in this mode: no images are deployed, so it only has to match your installation at a future managed takeover.

authURL must be https:// against a real installation

The CRD admits http:// so the kind fixture can run without certificates. It is the only reason. Over plain HTTP the admin password travels in the clear on every mint — and the minted application credential's id and secret come back in the clear — to anything on the path between K-ORC and the endpoint. There is no handshake to fail, so TLSVerificationFailed never fires: the failure mode is a silent success.

Against anything but a throwaway devstack, use https:// and supply the private CA via external.caBundleSecretRef. Pairing that ref with an http:// authURL is rejected at admission — a CA bundle a plaintext endpoint never consults would only manufacture false confidence.

Adoption means a new CR, never a flipped one

The webhook forbids spec.infrastructure, services.horizon, and every managed-only Keystone knob in External mode — it names the offending field. It also rejects mode transitions in both directions: Managed → External is refused, and External → Managed is reserved for the phase-3 takeover. So you adopt an installation by creating a new External-mode ControlPlane, never by flipping an existing managed one.

3. What Ready means in this mode

Nothing is deployed, so the sub-reconcilers that would deploy something report Status=True with reason ExternallyManagedInfrastructureReady, DBCredentialsReady, AdminPasswordReady, and KeystoneReady. They are not evidence that anything converged.

The conditions that carry real signal are KORCReady, AdminCredentialReady, and CatalogReady. All three are proxied through K-ORC: the operator holds no OpenStack client of its own, so "can we reach and authenticate against your Keystone?" is answered by whether the imports and the application-credential mint succeed. ServiceAccountsReady is not among them here: it folds the registrations the ControlPlane projects for its own built-in services, and an External-mode plane manages none, so it reads True with reason NoServiceRegistrationsProjected whatever your registrations are doing. Their readiness is reported on each KeystoneService CR, as step 5 shows.

bash
kubectl -n brownfield get controlplane controlplane-external
kubectl -n brownfield get controlplane controlplane-external \
  -o jsonpath='{range .status.conditions[*]}{.type}{"\t"}{.status}{"\t"}{.reason}{"\n"}{end}'

When something is wrong, KORCReady names the failure class:

ReasonWhat it meansWhat to do
AuthenticationFailedKeystone rejected the admin credential (HTTP 401).The password in the Secret is not (or is no longer) the installation's admin password.
EndpointUnreachableauthURL could not be dialled — DNS, connection refused, timeout.Check the URL, cluster DNS, and your egress policy.
TLSVerificationFailedThe endpoint's certificate did not verify.Supply the private CA via external.caBundleSecretRef. A rotated bundle only takes effect after K-ORC's provider cache expires (roughly half the token lifetime).
CatalogEndpointMismatchAuthentication worked, but the requested interface/region is absent from the catalog.Correct external.endpointType or spec.region.
ImportStalledAn import is waiting for a resource that "will be created externally" — but in this mode every import target already exists, so it never resolves.The operator is looking in the wrong place; the message names endpoint_type and region.
CredentialDriftThe installation changed underneath the CR.Reconcile the CR with reality. Drift is surfaced, never remediated — the operator does not write to your installation.

Imports resolve once. If an imported object is later replaced in Keystone, that surfaces as drift rather than the operator silently re-pointing at the new one.

The full reason vocabulary for every condition is in the ControlPlane CRD reference.

4. Verify

First, converge:

bash
kubectl -n brownfield wait --for=condition=Ready \
  controlplane/controlplane-external --timeout=10m

Then prove the ControlPlane deployed nothing — this is the whole point of the mode:

bash
kubectl -n brownfield get mariadbs,memcacheds,keystones,deployments
# No resources found in brownfield namespace.

Finally, prove the minted credential actually authenticates against your installation — not merely that the operator called itself Ready. Read it from the materialized Secret:

bash
kubectl -n brownfield get secret k-orc-clouds-yaml \
  -o jsonpath='{.data.clouds\.yaml}' | base64 -d

Use that clouds.yaml with an OpenStack client (openstack token issue, openstack catalog list). The e2e suite does this in a Job; see Tested by.

Read the credential from Kubernetes, not the OpenBao UI

On kind, OpenBao enforces mTLS, so browsing to it or using the bao CLI needs a client certificate. The materialized Secret above is the supported handle.

5. Register a service account

Service users of other OpenStack services (nova, glance, …) are declared with a KeystoneService CR, one per service. The ControlPlane itself carries no account fields: the registration owns its Keystone user and project, the operator-generated OpenBao-backed password, and the teardown of all three.

This is the registration the e2e suite creates in the ControlPlane's own namespace, declaring a catalog entry and an account:

yaml
apiVersion: c5c3.io/v1alpha1
kind: KeystoneService
metadata:
  name: nova
  namespace: brownfield
spec:
  controlPlaneRef:
    name: controlplane-external
  catalog:
    serviceType: compute
    endpoints:
    - interface: public
      url: "http://nova-api.brownfield.svc.cluster.local:8774/v2.1"
    - interface: internal
      url: "http://nova-api.brownfield.svc.cluster.local:8774/v2.1"
  account:
    project:
      name: service-nova
      create: true
    roles:
    - service

The semantics that matter:

  • The CR's own metadata keys everything. metadata.name names the child resources and the consumer Secret, and metadata.namespace is the delivery namespace. account.userName defaults to metadata.name.

  • account.project.create: true makes the registration create service-nova and own it, so the project is deleted again at teardown. Set create: false to reference a project that already exists: it is then imported unmanaged, never created and never deleted.

  • Neither adopt flag is set. Both default to the fail-loudly posture, so a pre-existing Keystone user of the same name surfaces ServiceAccountCollision and a pre-existing catalog row of the same type and name surfaces ServiceCollision, instead of either being taken over silently. The fixture's catalog holds identity rows only, so the compute entry collides with nothing.

    Setting account.adopt: true is explicit consent to take over such a user. An adopted user becomes operator-owned, so it is deleted at teardown. The one identity adopt does not unlock is the ControlPlane's own admin identity, which the reconciler refuses outright.

    Adoption does not apply the new password on the first pass

    With the currently pinned K-ORC, a newly adopted user keeps its pre-existing password even though the registration reports AccountReady=True and the consumer Secret carries the operator's generated one. Rotate the account once with a CredentialRotation to force the first real write. Tracked in #920.

  • account.roles are projected — each becomes an unmanaged K-ORC Role import plus a managed RoleAssignment binding the role to the user on the project, and the account is not Ready until every assignment lands in Keystone.

A registration in the ControlPlane's own namespace, or in one of its dedicated service namespaces, is admitted as it stands; anywhere else the namespace has to be listed in spec.korc.serviceRegistrations.allowedNamespaces on the ControlPlane first. Register a Service the ControlPlane Does Not Manage walks that flow.

The password is generated into OpenBao at openstack/keystone/{namespace}/{name}/service-accounts/credentials — keyed on the KeystoneService CR — and materialized as a stable consumer Secret named after it:

bash
kubectl -n brownfield get secret nova-credentials \
  -o jsonpath='{.data.clouds\.yaml}' | base64 -d

Readiness is reported on the registration itself (Ready, with AccountReady and CatalogReady beneath it), so one lagging registration is attributable without decoding the ControlPlane's aggregate condition.

6. Rotating the admin password, out-of-band

The admin password belongs to your installation, so it rotates there — the operator has no database access and no rotation job in this mode.

After rotating it at the installation, update the referenced Secret — again off the command line, since this one carries the fresh production password.

Writing that Secret is not bookkeeping: it is the trigger. The operator keys the mint on a hash of the admin password, so the write re-mints the application credential — and a re-mint is destructive-first: K-ORC deletes and revokes the credential that is currently working before it tries the fresh one. So the value has to be proven against Keystone before it is written. Type it twice, then issue a token with it; the Secret is only written if both readings agree and your installation accepts the password.

bash
read -rs -p 'new Keystone admin password: ' PW; echo
read -rs -p 'confirm: ' PW2; echo

# The admin identity the ControlPlane authenticates as (spec.korc.adminCredential).
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_URL=http://keystone.brownfield-keystone.svc:5000/v3
export OS_USERNAME=brownfield-admin
export OS_PROJECT_NAME=platform-admin
export OS_USER_DOMAIN_NAME=heimdall OS_PROJECT_DOMAIN_NAME=heimdall

if [ "$PW" != "$PW2" ]; then
  echo 'passwords do not match — Secret left untouched, nothing rotated' >&2
elif ! OS_PASSWORD="$PW" openstack token issue >/dev/null 2>&1; then
  echo 'password does not authenticate — Secret left untouched, credential intact' >&2
else
  printf '%s' "$PW" | kubectl -n brownfield create secret generic brownfield-admin-password \
    --from-file=password=/dev/stdin --dry-run=client -o yaml | kubectl -n brownfield replace -f -
fi
unset PW PW2

Run the token check from wherever authURL is reachable — that is the network position K-ORC dials from, so a pass there answers the same question the operator is about to ask. On the kind devstack the endpoint is a cluster-internal Service, so run it in a pod (kubectl run --rm -i --image=… -- openstack token issue), as the suite does.

Neither gate is ceremony, and equality alone is not enough. Comparing two readings catches a divergent typo and nothing else. It passes just as happily when you mistype the same thing twice, when you type the old password from muscle memory, and when you update the Secret before you actually rotated at the installation. Each of those changes the hash, so the re-mint fires all the same — revoking the working credential and then failing 401 against a password Keystone never accepted. You are left with no valid admin credential at all, KORCReady on AuthenticationFailed, and every import and service-account projection driven by it broken until a human notices and redoes the step. Asking Keystone first is the difference between a rotation and an outage.

Forgetting the Secret update is the classic drift. The old password stops authenticating, KORCReady goes to AuthenticationFailed, and the operator reports it — it will not go hunting for the new password.

To force a re-mint without a password change, request one. reMint: true is what makes it a rotation: without it the reconciler falls back to the password-hash check, finds the hash unchanged, and reports Ready=True with reason NoRotationNeeded having rotated nothing. The CR binds to the ControlPlane by namespace — one ControlPlane per namespace — so there is no reference field:

yaml
apiVersion: c5c3.io/v1alpha1
kind: CredentialRotation
metadata:
  name: rotate-admin
  namespace: brownfield
spec:
  target: adminApplicationCredential
  reMint: true

The nudge is one-shot per spec generation, so a reMint: true left in the spec fires once per edit rather than on every resync.

A service-account password rotates the same way, with target: serviceAccountPassword and keystoneService: <name> naming the KeystoneService registration in the CredentialRotation's namespace — and there reMint is not merely advisable but required: that path has no password-hash auto-detect at all, so without it the request is a guaranteed no-op.

spec.serviceAccount was replaced by spec.keystoneService

The field named an account declared on the ControlPlane; it now names a KeystoneService CR. CredentialRotation is v1alpha1, so the rename is applied in place and there is no conversion: an existing CR carrying serviceAccount loses the value on the next write and reports Ready=False reason MissingKeystoneService. Re-create it against a KeystoneService registration.

The minted credential must never be copied

A re-mint is delete + recreate, and the previous credential is revoked at the Keystone level the moment it happens — any client still holding it starts getting 404 Could not find Application Credential, with no grace period.

So every consumer must read the credential from the materialized Secret (or its OpenBao path) at use time. A copy pasted into another Secret, a config file, or an environment variable dies without warning on the next rotation.

7. OpenBao paths in this mode

External mode never seeds a bootstrap path: there is no operator-generated admin password to seed. Only the application-credential and service-account paths exist, and both are created on first push — there is no per-ControlPlane OpenBao preparation to do, because the operator provisions the per-tenant store itself.

The full per-mode path catalog is in OpenBao paths per ControlPlane mode.

8. Deletion — zero blast radius

Deleting the ControlPlane tears down only what it created or adopted:

bash
kubectl -n brownfield delete controlplane controlplane-external
  • The admin application credential is revoked (K-ORC's finalizer), and the OpenBao-backed Secrets are removed.
  • Managed service-account users and projects are deleted from Keystone — including any you marked adopt: true.
  • Every unmanaged import is untouched: the admin user, the domain, the catalog services and endpoints, and any project referenced with project.create: false.
  • Your Keystone keeps serving tokens throughout.

Ordering is held by the c5c3.io/orc-teardown finalizer, so the credential is revoked against a still-reachable Keystone before the CR leaves etcd.

See also

Tested by

bash
chainsaw test --test-dir tests/e2e/c5c3/external-keystone

or make e2e-external-keystone. The suite stands up the brownfield Keystone, adopts it with an External-mode ControlPlane, authenticates an OpenStack client with the minted credential, rotates it, and asserts the imports survive deletion.

The suite runs in namespace brownfield with the ControlPlane controlplane-external — the names this walkthrough uses — so step 2 imports the suite's own manifest rather than restating it.