Building Production-Grade
Kubernetes Operators
with Java

Ashish Thakur
Senior Software Engineer · Red Hat

About Me

Ashish Thakur

Senior Software Engineer at Red Hat

Contributor to Fabric8 Kubernetes Client — the industry-standard Java client for Kubernetes & OpenShift

> 1M monthly downloads 3.7k stars 11 years

Building block for Quarkus, Spring Cloud Kubernetes, Apache Flink, and JOSDK (CNCF Operator Framework)

Connect

  • Twitter/X: @ashish___thakur
  • LinkedIn: linkedin.com/in/ashish-thakur111
  • GitHub: github.com/ash-thakur-rh
  • Blog: ashthakur.in/blog

The Problem

Day 0 — Design & Plan

  • Architecture decisions & tooling
  • Capacity planning & infrastructure

Day 1 — Deploy

kubectl apply -f deployment.yaml

Install, configure, and launch. It works.

Day 2 — Operate & Maintain

  • Upgrades & zero-downtime rollouts
  • Scaling, backup, restore, failover
  • Configuration drift & recovery

Human operators can't be on-call 24/7 →

Encode operational knowledge into software

What are Kubernetes Operators?

Definition

Software extensions that manage cluster and non-cluster resources on behalf of Kubernetes.

The Goal

Automate complex Day 2 operations — upgrades, failover, scaling — via custom controllers.

Declare

User specifies what they want via a Custom Resource

Watch

Controller detects the desired state change

Reconcile

Controller drives current state → desired state

Declarative — the user requests what, the operator figures out how

Operator vs. Controller

Controller

Observe → Compare → Act loop on built-in K8s resources. No CRDs needed.

Fabric8 K8s Client can build either — controllers or operators.

e.g. ReplicaSet, DaemonSet, StatefulSet

Operator

Controller + CRDs — manages a single app's full lifecycle with domain knowledge.

JOSDK provides all the boilerplate you'd rewrite each time with Fabric8 Client directly.

e.g. Strimzi (Kafka), Prometheus Operator

Every Operator is a Controller, but not every Controller is an Operator.

Fabric8 Client powers both — JOSDK adds the operator boilerplate: reconciliation loop, retries, event handling, dependent resources.

Real-World Example

Without an Operator — You manage everything

  • Write Deployment YAML
  • Write Service YAML
  • Write ConfigMap YAML
  • Create Secret with DB credentials
  • Deploy PostgreSQL StatefulSet
  • Configure Ingress, HPA

6+ YAML files, manual and error-prone

With an Operator — You declare intent

apiVersion: example.io/v1
kind: MicroService
metadata:
  name: petclinic
spec:
  image: spring-petclinic:3.5.6
  replicas: 2
  database: { databaseName: petclinicdb }
  exposed: true

The operator auto-creates Deployment, Service, ConfigMap, Secret, Ingress, and HPA behind the scenes.

The Java Stack

Why Java? Massive enterprise adoption. Existing teams. Mature ecosystem.

Your Operator
Business logic & domain knowledge
Java Operator SDK (JOSDK)
Reconciliation, events, retries, dependent resources
Fabric8 Kubernetes Client
Fluent API, CRUD, watch, CRDs, OpenShift support
Kubernetes API
REST API & etcd

Fabric8 Kubernetes Client

The industry-standard Java client for Kubernetes & OpenShift

Multiple HTTP Clients

OkHttp, Vert.x, JDK HttpClient, Jetty

Fluent DSL

Chainable, readable API

OpenShift First-Class

Full support, unlike official client

CRD Support

Code gen + Maven plugin

Extensions

Knative, Tekton, Istio

Watch & Informers

Real-time events, SharedInformers, caching

Built-in Testing — Not available in official K8s client

Mock Server, CRUD mode, and Kube API Test Server — fast, robust testing for your operators without a real cluster. In the AI-driven development era, rapid feedback loops matter more than ever.

Fabric8 in Action

Fluent DSL for CRUD, watching, and custom resources

// List pods by label
client.pods().inNamespace("production")
    .withLabel("app", "petclinic").list();

// Create a resource
client.resource(deployment).create();

// Watch for real-time events
client.pods().watch(new Watcher<Pod>() {
    public void eventReceived(Action action, Pod pod) {
        log.info("{}: {}", action, pod.getMetadata().getName());
    }
});

// Type-safe Custom Resources
client.resources(MicroService.class)
    .inNamespace("production").withName("petclinic").get();

From Client to Framework

Fabric8 is the client. Building a production operator needs more:

Reconciliation LoopWatch → compare → act cycle with retry logic
Event HandlingDeduplication, batching, scheduling
🔗
Dependent ResourcesDeclarative secondary resource management
📊
ObservabilityHealth checks, metrics, structured logging

→ Java Operator SDK (JOSDK)

JOSDK — Key Concepts

Reconciler

Implement reconcile() — framework handles the loop, retries, and error recovery.

Dependent Resources

Declaratively manage secondary resources. Auto-creates, updates, garbage-collects.

Event Sources

Custom triggers beyond K8s API — external systems, timers, conditions.

CNCF Project 929 stars Quarkus + Spring Boot

Custom Resource Definition

Define your CRD as a Java class — Fabric8 K8s Client CRD Generator generates the YAML schema automatically

@Group("example.io") @Version("v1") @ShortNames("ms")
public class MicroService extends
    CustomResource<MicroServiceSpec, MicroServiceStatus>
    implements Namespaced { }

public class MicroServiceSpec {
    private String image;
    private int replicas;
    private DatabaseSpec database;
    private boolean exposed;
    private String hostname;
}

Custom Resource — What Users Write

Simple

apiVersion: example.io/v1
kind: MicroService
metadata:
  name: petclinic
spec:
  image: spring-petclinic:3.5.6
  replicas: 2
  exposed: true

Production — same CR, more spec

spec:
  image: spring-petclinic:4.0.3
  replicas: 2
  database: { databaseName: petclinicdb }
  autoscaling:
    minReplicas: 2
    maxReplicas: 8
  exposed: true
  hostname: petclinic.example.com

One simple resource — the operator creates Deployments, Services, ConfigMaps, database, HPA, Ingress automatically.

Full Custom Resource — Production Example

Operator-managed database, autoscaling, ingress — all from one CR

apiVersion: example.io/v1
kind: MicroService
metadata:
  name: petclinic-prod
  namespace: production
spec:
  image: quay.io/ash-thakur/spring-petclinic:4.0.3-arm64
  replicas: 2
  containerPort: 8080
  database:                              # Operator provisions PostgreSQL
    databaseName: petclinicdb
    username: petclinic
    storageSize: 5Gi
  autoscaling: { minReplicas: 2, maxReplicas: 8, targetCPUUtilizationPercentage: 20 }
  exposed: true
  hostname: petclinic.apps.mycluster.example.com

Auto-generates DB credentials Secret, StatefulSet, headless Service — no manual kubectl create secret.

Implementing the Reconciler

@Workflow declares the dependency graph — reconcile() just reads status

@Workflow(dependents = {
    @Dependent(type = ConfigMapDependentResource.class),
    @Dependent(type = DeploymentDependentResource.class,
               dependsOn = "ConfigMap"),
    @Dependent(type = ServiceDependentResource.class,
               dependsOn = "Deployment"),
    @Dependent(type = IngressDependentResource.class,
               reconcilePrecondition = ExposedCondition.class)
})
@ControllerConfiguration
public class MicroServiceReconciler
    implements Reconciler<MicroService> {

    public UpdateControl<MicroService> reconcile(
            MicroService ms, Context<MicroService> ctx) {
        int ready = ctx.getSecondaryResource(Deployment.class)
            .map(d -> d.getStatus().getReadyReplicas())
            .orElse(0);
        return UpdateControl.patchStatus(
            buildStatusPatch(ms, ready));
    }
}

Dependent Resources

Define desired state — the framework diffs against reality and reconciles

public class DeploymentDependentResource
    extends CRUDKubernetesDependentResource
             <Deployment, MicroService> {

    protected Deployment desired(MicroService ms,
                        Context<MicroService> ctx) {
        return new DeploymentBuilder()
            .withNewMetadata()
                .withName(ms.getMetadata().getName())
                .withNamespace(ms.getMetadata().getNamespace())
            .endMetadata()
            .withNewSpec()
                .withReplicas(ms.getSpec().getReplicas())
                .withNewSelector()
                  .withMatchLabels(Map.of("app",
                    ms.getMetadata().getName()))
                .endSelector()
                // ... template, container, probes ...
            .endSpec().build();
    }
}

Patterns and Best Practices

Idempotency is Key — Reconcile must be safe to run multiple times with the same result.
Always Reconcile All Resources — Check actual state of all components, not just the triggering event.
Stateless Logic — Use K8s Status sub-resources to track progress, not in-memory state.
Leverage Caching — Use Informers and Event Sources to avoid excessive API calls.

Error Handling

Write errors to status subresource — cleanup uses owner references for automatic GC

@Override
public ErrorStatusUpdateControl<MicroService> updateErrorStatus(
        MicroService ms, Context<MicroService> ctx, Exception e) {
    log.error("Error reconciling {}/{}: {}",
        ms.getMetadata().getNamespace(),
        ms.getMetadata().getName(), e.getMessage());
    MicroService patch = buildStatusPatch(ms, Phase.ERROR, 0);
    patch.getStatus()
         .setMessage("Reconciliation error: " + e.getMessage());
    return ErrorStatusUpdateControl.patchStatus(patch);
}

@Override
public DeleteControl cleanup(MicroService ms,
                              Context<MicroService> ctx) {
    // Owner references handle garbage collection automatically
    return DeleteControl.defaultDelete();
}

Who’s Using It in Production?

Keycloak Operator

Official, Quarkus + JOSDK

Apache Flink

K8s Operator — "market leader"

Apache Spark

K8s Operator for Spark

Strimzi

Kafka on Kubernetes

Debezium

Change Data Capture

Glasskube

K8s package manager

Resources

DEMO

Thanks!

Ashish Thakur
Senior Software Engineer · Red Hat

github.com/ash-thakur-rh