Multi cluster
System setup
This guide will show how to use Kratix on two Kubernetes clusters. Install the prerequisites listed below if they aren't already on your system. If you are using pre-existing clusters or want to use a different tool to provision your clusters, skip requirements 1 and 2.
kind
CLI / Kubernetes-in-Docker (KinD): Used to create and manage local Kubernetes clusters in Docker. See the quick start guide to install.docker
CLI / Docker: Used to orchestrate containers.kind
(above) requires that you have Docker installed and configured. See Get Docker to install.kubectl
/ Kubernetes command-line tool: The CLI for Kubernetes — allows you to run commands against Kubernetes clusters. See the install guide.
To get setup locally quickly with KinD clusters you can use the ./scripts/quick-start.sh
from the root of the Kratix repository. This provisions
an in-cluster MinIO
to use as the backing State Store.
Alternatively you can provide the --git
flag to create it with an in-cluster Gitea
instance instead.
Set up platform cluster
If you are not using a pre-existing cluster, create your platform cluster locally using KinD:
kind create cluster --image kindest/node:v1.27.3 --name platform
# set PLATFORM to point to the platform cluster context
export PLATFORM="kind-platform"
Install cert-manager
Kratix requires cert-manager to be installed in the Platform cluster. If you already have it installed, skip to the next section.
To install it, run:
kubectl --context $PLATFORM apply --filename https://github.com/cert-manager/cert-manager/releases/download/v1.12.0/cert-manager.yaml
Make sure that cert-manager
is ready before installing Kratix:
$ kubectl --context $PLATFORM get pods --namespace cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-7476c8fcf4-r8cnd 1/1 Running 0 19s
cert-manager-cainjector-bdd866bd4-7d8zp 1/1 Running 0 19s
cert-manager-webhook-5655dcfb4b-54r49 1/1 Running 0 19s
Install Kratix
If your are using your own pre-existing cluster, set the PLATFORM
environment
variable to the name of the kubectl context used to communicate to it. Install Kratix
on the platform cluster.
# Install Kratix
kubectl apply --context $PLATFORM --filename https://github.com/syntasso/kratix/releases/latest/download/kratix.yaml
You can also install and configure Kratix with Helm. For more information, see the Helm Chart documentation.
Set up State Store
Kratix uses GitOps to provision resources on the worker clusters. You can configure Kratix with multiple different GitOps repositories by creating State Stores. Kratix supports Bucket State Store and Git State Store.
If your are using local KinD clusters you can install MinIO or Gitea as an in-cluster State Store
- Bucket (on KinD)
- Git (on KinD)
- Custom
# Install MinIO and register it as a BucketStateStore
kubectl apply --context $PLATFORM --filename https://raw.githubusercontent.com/syntasso/kratix/main/config/samples/minio-install.yaml
kubectl apply --context $PLATFORM --filename https://raw.githubusercontent.com/syntasso/kratix/main/config/samples/platform_v1alpha1_bucketstatestore.yaml
# Install Gitea and register it as a Git State Store
kubectl apply --context $PLATFORM --filename https://raw.githubusercontent.com/syntasso/kratix/main/hack/platform/gitea-install.yaml
kubectl apply --context $PLATFORM --filename https://raw.githubusercontent.com/syntasso/kratix/main/config/samples/platform_v1alpha1_gitstatestore.yaml
If your aren't using KinD clusters you will need to create your own State Store that is accessible by the platform and worker cluster. Follow the docs for creating State Stores.
Set up a worker cluster
Create worker cluster
If you are not using a pre-existing cluster, create your platform cluster locally using KinD:
kind create cluster --image kindest/node:v1.27.3 --name worker
# set WORKER to point to the worker cluster context
export WORKER="kind-worker"
If you are using your own pre-existing cluster, set the WORKER
environment
variable to the name of the kubectl context used to communicate to it.
Install Flux
Install Flux. Flux will be used to pull down resources to the cluster from the State Store
# Install flux on the worker
kubectl apply --context $WORKER --filename https://raw.githubusercontent.com/syntasso/kratix/main/hack/destination/gitops-tk-install.yaml
Configure Flux
Now that Flux is installed, we need to configure it to pull down from the Kratix State Store. Follow the steps below that match the State Store you created previously:
- Bucket (on KinD)
- Git (on KinD)
- Custom
# Configure Flux to pull down from MinIO
kubectl apply --context $WORKER --filename https://raw.githubusercontent.com/syntasso/kratix/main/hack/destination/gitops-tk-resources.yaml
# Configure Flux to pull down from Gitea
kubectl apply --context $WORKER --filename https://raw.githubusercontent.com/syntasso/kratix/main/hack/destination/gitops-tk-resources-git.yaml
You will need to manual configure the Flux resources to use the Git/Bucket created.
- Bucket: Download and modify this example configuration to use the endpoint, bucket and credentials for your Bucket.
- Git: Download and modify this example configuration to use the url, branch, and credentials for your Git Repository.
Register cluster as a Destination with Kratix
The final step is to tell Kratix that the cluster is ready to receive workloads. Follow the steps below that match the State Store you created previously:
- Bucket (on KinD)
- Git (on KinD)
- Custom
# Configure Flux to pull down from MinIO
kubectl apply --context $PLATFORM --filename https://raw.githubusercontent.com/syntasso/kratix/main/config/samples/platform_v1alpha1_worker.yaml
# Install flux on the worker and configure it to pull down from MinIO
curl https://raw.githubusercontent.com/syntasso/kratix/main/config/samples/platform_v1alpha1_worker.yaml | \
sed "s_BucketStateStore_GitStateStore_g" | \
kubectl apply --context $PLATFORM --filename -
You will need to manual configure the Destination resources to use created State Store.
Download and modify this example configuration,
replacing the spec.StateStoreRef.name
, spec.StateStoreRef.namespace
and spec.StateStoreRef.kind
to match the previously created State Store
Flux will eventually reconcile the clusters state, making the worker
cluster ready
to receive workloads. You can verify its readiness by observing the kratix-worker-system
namespace appearing in the worker
cluster:
$ kubectl --context $WORKER get namespaces
NAME STATUS AGE
...
kratix-worker-system Active 1m
...
🎉 Congratulations! Kratix is now installed! Jump to Installing and using a Promise to spin up your first as-a-service offering.