Skip to content

About the exalsius operator

The exalsius-operator is the open-source Kubernetes operator underneath the exalsius platform. It lets infrastructure teams describe managed Kubernetes clusters, and the application workspaces running on them, as Kubernetes resources. Everything on this page works with plain kubectl; the exls CLI and the exalsius API build on the same resources but are not required.

Describes exalsius-operator v0.12.1

The pages in this section correspond to release v0.12.1 of the exalsius-operator repository.

New to the operator? The operator quickstart turns two SSH-reachable Linux machines into a management cluster, a provisioned child cluster, and a running Jupyter workspace in about 20 minutes.

Already running k0rdent? Install the operator with its Helm chart and follow the chart's "After installing" section.

Why an operator

Running AI workloads on rented or scattered GPU machines means repeating the same work: provision a cluster, wire up networking, install the GPU stack, deploy the application, publish a URL, and tear everything down again. The operator turns that work into three declarative resources, so the desired state lives in the Kubernetes API and a controller keeps reality in line with it.

The operator does not provision machines or install Helm charts itself. It translates its resources into k0rdent objects and reports k0rdent's progress back on the resource you created.

About k0rdent

k0rdent is an open-source project for managing many Kubernetes clusters from one, built entirely on standard open-source tooling. You install its core component, the k0rdent Cluster Manager (KCM), on a Kubernetes cluster of your own. That cluster becomes the management cluster, and from it k0rdent creates and manages other clusters on public clouds, private infrastructure, or bare machines.

The operator relies on k0rdent for two jobs:

  • Provisioning clusters. k0rdent drives Cluster API. A ClusterTemplate is a Helm chart describing a cluster blueprint, a Credential holds the access that blueprint needs, and a ClusterDeployment asks for one cluster built from the two.
  • Delivering services to those clusters. A ServiceTemplate wraps a Helm chart, and a ServiceSet asks for it to be installed on a managed cluster. k0rdent hands the installation to Sveltos.

Building on k0rdent rather than beside it is a deliberate trade. Every infrastructure provider k0rdent supports is available to a Colony through that provider's cluster template, and the operator stays small because it never talks to a cloud API or runs Helm itself. In exchange, the operator is only useful on a cluster where k0rdent is already running.

The operator quickstart installs k0rdent for you on a fresh machine. To add the operator to a cluster of your own, install k0rdent first, with its Sveltos state management provider (ksm-projectsveltos) enabled. The k0rdent documentation covers that:

The quickstart is verified with KCM 1.10.0.

The three resources

flowchart LR
  colony[Colony] --> cd[ClusterDeployment<br/>k0rdent] --> child[Child cluster<br/>hosted control plane, Cilium]
  wsc[WorkspaceClass] --> st[ServiceTemplate<br/>k0rdent, wraps a Helm chart]
  wsd[WorkspaceDeployment] --> ss[ServiceSet<br/>k0rdent] --> rel[Helm release in ws-&lt;name&gt;<br/>on the child cluster]
  wsd -. pins .-> wsc
  wsd -. targets .-> cd
  • A Colony lists the clusters you want provisioned, possibly across several cloud providers or on-premise sites. One cluster can span machines in different datacenters, because its control plane is hosted and Cilium encrypts traffic between nodes with WireGuard.
  • A WorkspaceClass is a catalog entry for a type of workspace, such as a Jupyter notebook or an inference server. Platform administrators publish classes; users pick from them. It plays the same role as a StorageClass: it describes what is available, not a running instance.
  • A WorkspaceDeployment is a user's request for one instance of a class on one specific cluster. Its status carries the phase, conditions, and access URLs, so a client only ever watches this one resource.

All three live on the management cluster, where k0rdent and the operator run. The only thing that lands on a child cluster is the workspace itself: a Helm release in its own ws-<name> namespace.

What the operator does

  • Provisions and groups clusters. Each Colony entry becomes a k0rdent ClusterDeployment built from a cluster template. The operator aggregates the child kubeconfigs into one Secret per Colony.
  • Publishes a GPU inventory. The Colony status lists, per child cluster, which GPU offerings exist and how many of each. The inventory describes what exists; free capacity is computed live when a workspace is admitted.
  • Publishes a workspace catalog. A WorkspaceClass pins the chart to deploy, the default resources, the prerequisites the target cluster needs first, the access endpoints, and the configuration prompts users see. Classes are versioned and immutable, so a running workspace's definition never changes underneath it. Ready-made classes live in the exalsius-workspace-hub.
  • Deploys workspaces. For each WorkspaceDeployment the operator merges class defaults with the user's overrides, installs missing prerequisites once per cluster, and checks GPU capacity. A workspace whose GPUs are busy moves to Waiting and proceeds on its own once capacity frees up; a workspace asking for GPUs the cluster does not have is Failed.
  • Publishes workspace URLs. When a child cluster is attached to a per-tenant regional cluster, the operator attaches routes to the tenant's Gateway and reports the resulting URLs on the WorkspaceDeployment. Without one, workspaces are reached with kubectl port-forward.
  • Tears down. Deleting a WorkspaceDeployment uninstalls the release and its namespace; deleting a Colony removes its clusters.

Concepts and architecture explains each of these in depth, with diagrams of both reconcilers.

How the operator relates to the CLI and the platform

The operator is one component of the exalsius stack. The exalsius API turns requests from the exls CLI and the hosted platform into the resources above, and adds what the operator leaves out: multi-cloud node acquisition, cost-aware placement, and tenant management.

The two layers use different words for the same things:

In the CLI and platform In the operator
A cluster A child cluster: one entry of a Colony, represented by a k0rdent ClusterDeployment
A workspace type, such as jupyter or llm-inference A WorkspaceClass
A workspace A WorkspaceDeployment
The node pool Not an operator concept; the exalsius API manages nodes

Choose the layer by what you want to operate. The CLI and platform give you the full journey without running a management cluster yourself; start with the CLI quickstart. The operator gives you the same cluster and workspace model on infrastructure you control end to end.

Where to go next