OpenStack Resource Controller (K-ORC)

Repository: github.com/k-orc/openstack-resource-controllerRuns in: Control Plane Cluster (Deployment)

K-ORC (Kubernetes OpenStack Resource Controller) enables declarative management of OpenStack resources via Kubernetes CRDs. It follows the Infrastructure-as-Code pattern and integrates seamlessly into GitOps workflows.

Supported OpenStack Services (in CobaltCore):

Service	Resources
Keystone (Identity)	Domain, Project, Role, Group, Service, Endpoint, User, ApplicationCredential

Architecture:

┌───────────────────────────────────────────────────────────────────┐
│                    K-ORC (Control Plane Cluster)                  │
├───────────────────────────────────────────────────────────────────┤
│                                                                   │
│  ┌─────────────────────────────────────────────────────────────┐  │
│  │                    K-ORC Manager Pod                        │  │
│  │                    (orc-system Namespace)                   │  │
│  │                                                             │  │
│  │  Reconciliation Controllers:                                │  │
│  │  └── Keystone: Domain, Project, Role, Group, Service,       │  │
│  │               Endpoint, User, ApplicationCredential         │  │
│  └─────────────────────────────────────────────────────────────┘  │
│                              │                                    │
│                              │ Gophercloud SDK                    │
│                              ▼                                    │
│  ┌─────────────────────────────────────────────────────────────┐  │
│  │             Keystone Identity API                           │  │
│  └─────────────────────────────────────────────────────────────┘  │
│                                                                   │
└───────────────────────────────────────────────────────────────────┘

Key Features:

• 8 CRD types for declarative Keystone resource management
• Management Policies: managed (full lifecycle) or unmanaged (read-only import)
• Import existing resources via filters (Name, Tags, ID)
• Credential Management via clouds.yaml in Kubernetes Secrets
• Dependency Management: Automatic ordering (e.g., Network before Subnet)
• Finalizers: Safe deletion with cleanup in OpenStack

Credential Management for K-ORC:

K-ORC requires a clouds.yaml secret for authentication against OpenStack APIs. The c5c3-operator creates K-ORC ApplicationCredential CRs. K-ORC creates the Application Credentials in Keystone and writes the result to a Kubernetes Secret. A PushSecret writes this secret to OpenBao, from where it's provided via ESO in the target namespace:

# PushSecret: K-ORC Application Credential → OpenBao
# Watches the Secret written by K-ORC and pushes it to OpenBao
apiVersion: external-secrets.io/v1alpha1
kind: PushSecret
metadata:
  name: k-orc-app-credential
  namespace: openstack
spec:
  secretStoreRefs:
    - name: openbao-cluster-store
      kind: ClusterSecretStore
  selector:
    secret:
      name: k-orc-app-credential  # Written by K-ORC
  data:
    - match:
        secretKey: clouds.yaml
        remoteRef:
          remoteKey: kv-v2/data/openstack/k-orc/app-credential
          property: clouds.yaml

---
# ExternalSecret: OpenBao → k-orc-clouds-yaml in orc-system
# Reads the Application Credential from OpenBao and creates the
# Secret that K-ORC mounts as clouds.yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: k-orc-clouds-yaml
  namespace: orc-system
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: openbao-cluster-store
    kind: ClusterSecretStore
  target:
    name: k-orc-clouds-yaml
    creationPolicy: Owner  # ESO owns the lifecycle of this Secret
  data:
    - secretKey: clouds.yaml
      remoteRef:
        key: kv-v2/data/openstack/k-orc/app-credential
        property: clouds.yaml

The resulting Kubernetes Secret in orc-system (created and managed by ESO):

# Auto-generated by ESO — do not edit manually
apiVersion: v1
kind: Secret
metadata:
  name: k-orc-clouds-yaml
  namespace: orc-system
  # Owned by ExternalSecret (ESO manages lifecycle)
  ownerReferences:
    - apiVersion: external-secrets.io/v1beta1
      kind: ExternalSecret
      name: k-orc-clouds-yaml
type: Opaque
data:
  clouds.yaml: |
    clouds:
      openstack:
        auth_type: v3applicationcredential
        auth:
          auth_url: https://identity.example.com
          application_credential_id: "abc123..."
          application_credential_secret: "xyz789..."
        region_name: RegionOne

Example: Create Keystone Project:

apiVersion: openstack.k-orc.cloud/v1alpha1
kind: Project
metadata:
  name: customer-project
spec:
  cloudCredentialsRef:
    cloudName: openstack
    secretName: k-orc-clouds-yaml  # References generated secret
  managementPolicy: managed
  resource:
    description: "Customer project"
    enabled: true
    tags: ["customer-a", "production"]

Note: The cloudCredentialsRef.secretName references a secret provided via the path K-ORC → PushSecret → OpenBao → ESO. Credentials are automatically rotated via the CredentialRotation CRD.

Use Cases in CobaltCore:

• Declarative management of Keystone services, endpoints, and users
• Automated provisioning of service accounts and application credentials
• Declarative management of Keystone domains, projects, and roles
• GitOps integration for Identity-as-Code
• Multi-tenant setup with reproducible configurations

Management Policies:

• managed: K-ORC creates, updates, and deletes the OpenStack resource (full lifecycle). Finalizers prevent Kubernetes deletion until the OpenStack resource is removed. Use for: All bootstrap resources, new service registrations.

• unmanaged: K-ORC imports an existing OpenStack resource as read-only via filters (name, tags, ID). K-ORC does not modify or delete the resource. Status reflects the external state. Use for: Brownfield deployments (see Brownfield Integration), shared resources managed by other tools.

Deployment (HelmRelease):

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: k-orc
  namespace: flux-system
spec:
  interval: 1h
  url: https://k-orc.github.io/openstack-resource-controller

---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: openstack-resource-controller
  namespace: orc-system
spec:
  interval: 30m
  chart:
    spec:
      chart: openstack-resource-controller
      version: ">=0.1.0"
      sourceRef:
        kind: HelmRepository
        name: k-orc
        namespace: flux-system
  values:
    # clouds.yaml Secret — provided via OpenBao + ESO
    globalCloudConfig:
      secretName: k-orc-clouds-yaml  # Created by ExternalSecret from OpenBao
  install:
    crds: CreateReplace
  upgrade:
    crds: CreateReplace

Bootstrap Resources (Chicken-and-Egg Problem):

The Keystone Bootstrap Job creates foundational resources (Admin User, Default Domain, Service Project, Roles) directly via the Keystone API — before K-ORC has credentials. These resources must be imported into K-ORC (managementPolicy: unmanaged) so that K-ORC can reference them when creating dependent resources (Services, Endpoints, Users).

When spec.korc.bootstrapResources is configured in the ControlPlane CR, the c5c3-operator creates K-ORC CRs with managementPolicy: unmanaged to import existing bootstrap resources:

• domains: Default domain (created by Keystone Bootstrap Job)
• projects: Service project, admin project (created by Keystone Bootstrap Job)

Only after these imports are visible in K-ORC can the c5c3-operator create new resources (Services, Endpoints, Service Users, Application Credentials) with managementPolicy: managed.

Troubleshooting:

Common issues and diagnostic commands:

Issue	Diagnostic	Resolution
K-ORC cannot reach Keystone	kubectl logs -n orc-system -l app=openstack-resource-controller	Verify clouds.yaml secret and Keystone endpoint
clouds.yaml secret missing	kubectl get secret k-orc-clouds-yaml -n orc-system	Check ESO/PushSecret pipeline
Service already exists	Check K-ORC CRD status conditions	Use managementPolicy: unmanaged to import
CRD stuck in Creating	kubectl describe <crd-kind> <name> -n openstack	Check dependency ordering and Keystone availability

Further Reading

• C5C3 Operator — Creates K-ORC CRs for service catalog management
• Service Operators — Services whose credentials K-ORC manages
• Credential Lifecycle — Full credential flow including K-ORC
• Brownfield Integration — Using unmanaged policy for existing resources