Headless Service Reconciliation

Reference documentation for the headless Service reconciliation logic that creates and updates a v1 Service with clusterIP: None for each Memcached CR.

Source: internal/controller/service.go, internal/controller/memcached_controller.go

Overview

When a Memcached CR is created or updated, the reconciler ensures a matching headless Service exists in the same namespace with the same name. The headless Service enables DNS-based pod discovery — clients such as pymemcache can resolve individual pod IPs via DNS SRV records for consistent hashing. The Service is constructed from the CR spec using a pure builder function, then applied via controllerutil.CreateOrUpdate for idempotent create/update semantics. A controller owner reference on the Service enables automatic garbage collection when the Memcached CR is deleted.

The headless Service is always created for every Memcached CR. It is not opt-in.

Service Construction

constructService(mc *Memcached, svc *Service) sets the desired state of the headless Service in-place. It is called within the controllerutil.CreateOrUpdate mutate function so that both creation and updates use identical logic.

ClusterIP

The Service is always headless:

svc.Spec.ClusterIP = corev1.ClusterIPNone  // "None"

Setting clusterIP: None means Kubernetes does not allocate a virtual IP. Instead, DNS queries for the Service name return A records for each ready pod IP, enabling direct pod-to-pod communication.

Note: The ClusterIP field is immutable after Service creation. Because constructService sets it in the mutate function, it is applied during the initial Create call and left unchanged on subsequent Updates.

Port Configuration

The Service exposes a single port:

corev1.ServicePort{
    Name:       "memcached",
    Port:       11211,
    TargetPort: intstr.FromString("memcached"),
    Protocol:   corev1.ProtocolTCP,
}

Field	Value	Description
`name`	`memcached`	Named port for DNS SRV record lookup
`port`	`11211`	Memcached default TCP port
`targetPort`	`memcached` (named)	References the container port name on the pod
`protocol`	`TCP`	Memcached uses TCP

The targetPort uses the named port memcached (matching the container port defined in the Deployment) rather than a numeric value, so the Service remains valid even if the container port number changes.

Note: The metrics port (9150) is not included in this Service. It will be added in Phase 4 (ServiceMonitor Reconciliation) when monitoring is enabled.

Labels

The Service uses the same standard Kubernetes recommended labels as the Deployment, generated by labelsForMemcached(name):

Label Key	Value	Purpose
`app.kubernetes.io/name`	`memcached`	Identifies the application
`app.kubernetes.io/instance`	`<cr-name>`	Distinguishes instances of the same application
`app.kubernetes.io/managed-by`	`memcached-operator`	Identifies the managing controller

These labels are set on the Service's metadata.labels and on spec.selector, ensuring the Service selects pods managed by the same Memcached CR instance.

Selector

The Service selector uses the same label set as the Deployment's spec.selector.matchLabels. This ensures the headless Service resolves to the exact same pods that the Deployment manages:

svc.Spec.Selector = labelsForMemcached(mc.Name)

Custom Annotations

Custom annotations from spec.service.annotations are applied to the Service metadata. This supports integration with external tools such as Prometheus scraping, load balancer configuration, or service mesh injection.

if mc.Spec.Service != nil && len(mc.Spec.Service.Annotations) > 0 {
    svc.Annotations = mc.Spec.Service.Annotations
} else {
    svc.Annotations = nil
}

CR State	Service Annotations
`spec.service` is nil	No annotations (nil)
`spec.service.annotations` empty	No annotations (nil)
`spec.service.annotations` set	Exact copy of the annotation map

Annotations are replaced entirely on each reconciliation. The operator does not merge with existing annotations — whatever is in spec.service.annotations becomes the Service's annotation set.

Reconciliation Method

reconcileService(ctx, mc *Memcached) on MemcachedReconciler ensures the headless Service matches the desired state:

func (r *MemcachedReconciler) reconcileService(ctx context.Context, mc *memcachedv1alpha1.Memcached) error {
    logger := log.FromContext(ctx)

    svc := &corev1.Service{
        ObjectMeta: metav1.ObjectMeta{
            Name:      mc.Name,
            Namespace: mc.Namespace,
        },
    }

    result, err := controllerutil.CreateOrUpdate(ctx, r.Client, svc, func() error {
        constructService(mc, svc)
        return controllerutil.SetControllerReference(mc, svc, r.Scheme)
    })
    if err != nil {
        return fmt.Errorf("reconciling Service: %w", err)
    }

    logger.Info("Service reconciled", "name", svc.Name, "operation", result)
    return nil
}

CreateOrUpdate Behavior

controllerutil.CreateOrUpdate performs a server-side get-or-create:

Scenario	Mutate Function Called	API Operation	`result` Value
Service does not exist	Yes	Create	`controllerutil.OperationResultCreated`
Service exists, spec differs	Yes	Update	`controllerutil.OperationResultUpdated`
Service exists, spec matches	Yes	(no-op)	`controllerutil.OperationResultNone`

The mutate function runs before every create or update, ensuring the Service always reflects the current CR spec. External drift (manual edits) is corrected on the next reconciliation cycle.

Owner Reference

controllerutil.SetControllerReference adds an owner reference to the Service's metadata:

Field	Value
`apiVersion`	`memcached.c5c3.io/v1alpha1`
`kind`	`Memcached`
`name`	`<cr-name>`
`uid`	`<cr-uid>`
`controller`	`true`
`blockOwnerDeletion`	`true`

This enables:

Garbage collection: Deleting the Memcached CR automatically deletes the owned Service via Kubernetes' owner reference cascade.
Watch filtering: The Owns(&corev1.Service{}) watch on the controller maps Service events back to the owning Memcached CR for reconciliation.

Error Handling

Error Scenario	Behavior
API server unreachable	Error returned, controller-runtime requeues with backoff
Service create/update fails	Error wrapped with context, returned for requeue
Owner reference conflict	Error from `SetControllerReference`, returned for requeue

Errors are wrapped with fmt.Errorf("reconciling Service: %w", err) to provide context in logs while preserving the original error for apierrors.IsXxx() checks upstream.

Reconcile Integration

The Reconcile method calls reconcileService after reconcileDeployment:

func (r *MemcachedReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    memcached := &memcachedv1alpha1.Memcached{}
    if err := r.Get(ctx, req.NamespacedName, memcached); err != nil {
        if apierrors.IsNotFound(err) {
            return ctrl.Result{}, nil
        }
        return ctrl.Result{}, err
    }

    if err := r.reconcileDeployment(ctx, memcached); err != nil {
        return ctrl.Result{}, err
    }

    if err := r.reconcileService(ctx, memcached); err != nil {
        return ctrl.Result{}, err
    }

    return ctrl.Result{}, nil
}

Scenario	Return Value	Effect
CR not found (deleted)	`ctrl.Result{}, nil`	No requeue; owner ref cascade handles Service cleanup
CR fetch fails	`ctrl.Result{}, err`	Requeue with exponential backoff
Deployment reconcile fails	`ctrl.Result{}, err`	Requeue; Service reconcile skipped
Service reconcile succeeds	`ctrl.Result{}, nil`	No requeue
Service reconcile fails	`ctrl.Result{}, err`	Requeue with exponential backoff

Reconciliation Flow

text

  Memcached CR created/updated
            │
            ▼
  ┌─────────────────────────────┐
  │  Reconcile                  │
  │  1. Fetch Memcached CR      │
  │  2. If NotFound → return    │
  │  3. If error → requeue      │
  └────────────┬────────────────┘
               │
               ▼
  ┌─────────────────────────────┐
  │  reconcileDeployment        │
  │  (see deployment-           │
  │   reconciliation.md)        │
  └────────────┬────────────────┘
               │
               ▼
  ┌─────────────────────────────┐
  │  reconcileService           │
  │                             │
  │  CreateOrUpdate:            │
  │    ┌──────────────────────┐ │
  │    │ Mutate function      │ │
  │    │  constructService    │ │
  │    │  SetControllerRef    │ │
  │    └──────────────────────┘ │
  │                             │
  │  Service (headless)         │
  │  ├─ Name: <cr-name>        │
  │  ├─ Namespace: <cr-ns>     │
  │  ├─ ClusterIP: None        │
  │  ├─ Port: 11211/TCP        │
  │  ├─ Selector: std labels   │
  │  ├─ Annotations: from spec │
  │  └─ OwnerRef → Memcached CR│
  └─────────────────────────────┘

CRD Spec: ServiceSpec

The spec.service field on the Memcached CR controls Service customization:

// ServiceSpec defines configuration for the headless Service.
type ServiceSpec struct {
    Annotations map[string]string `json:"annotations,omitempty,omitzero"`
}

Field	Type	Required	Default	Description
`spec.service`	`*ServiceSpec`	No	`nil`	Service configuration block
`spec.service.annotations`	`map[string]string`	No	`nil`	Custom annotations for the Service

Service Manifest Examples

Minimal CR

yaml

apiVersion: memcached.c5c3.io/v1alpha1
kind: Memcached
metadata:
  name: my-cache
  namespace: default
spec: {}

Produces:

yaml

apiVersion: v1
kind: Service
metadata:
  name: my-cache
  namespace: default
  labels:
    app.kubernetes.io/name: memcached
    app.kubernetes.io/instance: my-cache
    app.kubernetes.io/managed-by: memcached-operator
  ownerReferences:
    - apiVersion: memcached.c5c3.io/v1alpha1
      kind: Memcached
      name: my-cache
      controller: true
      blockOwnerDeletion: true
spec:
  clusterIP: None
  selector:
    app.kubernetes.io/name: memcached
    app.kubernetes.io/instance: my-cache
    app.kubernetes.io/managed-by: memcached-operator
  ports:
    - name: memcached
      port: 11211
      targetPort: memcached
      protocol: TCP

CR with Custom Annotations

yaml

apiVersion: memcached.c5c3.io/v1alpha1
kind: Memcached
metadata:
  name: my-cache
  namespace: default
spec:
  service:
    annotations:
      prometheus.io/scrape: "true"
      prometheus.io/port: "11211"

Produces a Service identical to the minimal example, with the addition of:

yaml

metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "11211"

Headless Service Reconciliation ​

Overview ​

Service Construction ​

ClusterIP ​

Port Configuration ​

Labels ​

Selector ​

Custom Annotations ​

Reconciliation Method ​

CreateOrUpdate Behavior ​

Owner Reference ​

Error Handling ​

Reconcile Integration ​

Reconciliation Flow ​

CRD Spec: ServiceSpec ​

Service Manifest Examples ​

Minimal CR ​

CR with Custom Annotations ​

Headless Service Reconciliation

Overview

Service Construction

ClusterIP

Port Configuration

Labels

Selector

Custom Annotations

Reconciliation Method

CreateOrUpdate Behavior

Owner Reference

Error Handling

Reconcile Integration

Reconciliation Flow

CRD Spec: ServiceSpec

Service Manifest Examples

Minimal CR

CR with Custom Annotations