Scheduling

[WCR]: Worker Cluster Resource [WCRs]: Worker Cluster Resources

One of the most powerful Kratix features is the ability Platform teams have to fully control scheduling of work across Kubernetes Cluster. Scheduling happens in two stages:

1. Determining which Promises are installed in which Clusters (Scheduling Promises)
2. Determining where the workloads will run following a Resource Requests (Scheduling Workloads)

The following sections in this page document those stages. For hands-on scheduling guides, check the Adding a new Worker Cluster and Compound Promise pages.

Scheduling Promises

By default, Worker Cluster Resources (WCRs) for every Promise on the platform will be installed on all Clusters registered with the platform. In other words, all registered Clusters will be ready to run workloads for all Promises. When a new Cluster is registered, Kratix will automatically install the WCRs for the Promises that are already installed in the Platform.

Platform teams can, however, control which Clusters receive which Promises by using a combination of Cluster labels and Promise Cluster selectors.

The labels in the Cluster document are the standard Kubernetes labels: simple, arbitrary, key/value pairs. In the example below, the Cluster object is being created with a label environment with value dev:

worker-cluster-2.yaml

apiVersion: platform.kratix.io/v1alpha1
kind: Cluster
metadata:
  labels:
    environment: dev
  # ...

In the Promise document, the clusterSelector label selector defines in which Clusters the Promise resources should be installed. The clusterSelector is a equality-based selector, i.e., it will only match Clusters that have keys/values that match. You can add multiple key/value pairs to the clusterSelector, but note that it will only match when the Cluster has a matching label for all the selectors.

jenkins-promise.yaml

apiVersion: platform.kratix.io/v1alpha1
kind: Promise
metadata:
  name: jenkins-promise
spec:
  clusterSelector:
    environment: dev
  workerClusterResources:
  # ...

If a Promise has no clusterSelector, it will be applied to all Clusters. If a Cluster has no labels, only Promises with no clusterSelector will be applied.

The table below contains a few examples:

Cluster Label	Promise Selector	Match?
no label	no selector	✅
env: dev	no selector	✅
env: dev	env: dev	✅
env: dev zone:eu	env: dev	✅
env: dev zone:eu	env: dev zone:eu	✅
env: dev	env: prod	⛔️
env: dev	env: dev zone:eu	⛔️
no label	env: dev	⛔️

Scheduling Workloads

When a new Resource Request comes in, Kratix reacts by triggering the Request Pipeline, as defined in the Promise's spec.xaasRequestPipeline. The output of the pipeline will be then written to the Repository, which will in turn be applied to one of the registered Worker Clusters.

By default, Kratix will randomly select a registered Cluster to instantiate the workload. If the Promise has spec.clusterSelector set, the workload will be scheduled to a Cluster that has matching labels.

It is possible to dynamically determine where workloads will go at the Pipeline stage. The section below documents the process.

Dynamic Scheduling

Kratix mounts a metadata directory at the root of the pipeline's container when instantiating the Request Pipeline. At scheduling time, Kratix will look for a cluster-selectors.yaml file in that directory with arbitrary key/value pairs and will add those to what is already present in the Promise's spec.clusterSelector when filtering Clusters.

That means there's no overwrite of keys. For example, if the Promise defines a clusterSelector: { env: dev } and the Pipeline defines env: prod, Kratix will look for Clusters with labels: env=dev && env=prod ¹.

The table below contains a few examples:

Cluster Label	Promise Selector	cluster-selectors.yaml	Match?
no label	no selector	no_selector	✅
env: dev	no selector	no_selector	✅
env: dev	env: dev	no_selector	✅
env: dev zone: eu	env: dev	zone: eu	✅
env: dev zone: eu	no selector	zone: eu	✅
env: dev	env: dev	env: prod	⛔️
env: dev	env: prod	env: dev	⛔️
env: dev	env: dev zone: eu	no_selector	⛔️
no label	no_selector	env: dev	⛔️

In the event that more than one cluster matches the resulting labels, Kratix will randomly select within the available matching registered Clusters. If you prefer to be certain of a single cluster match, it is suggested that you add a unique identifier to all clusters (e.g. clusterName) so that there can only ever be a single match.

Compound Promises

Compound Promises are Promises that, in its WCRs, contain other Promises. That ability allows Platform teams deliver entire stacks on demand, instead of simple databases or services.

To enable this functionality, the following needs to be true:

• The Platform cluster must register itself as a Worker cluster
• The GitOps toolkit must be installed in the Platform cluster
• The Compound Promise must instruct Kratix to install its WCR (i.e. the other Promises) in the Platform cluster
• Optionally, the sub-Promises may instruct Kratix to install their WCR outside the Platform cluster

For detailed instruction on the above, please check the Compound Promises guide.

---

1. This will always return empty, since a label key can only hold a single value.↩