Deployment Basics

vCluster Kubernetes Distributions

vCluster is able to use different Kubernetes distributions as a virtual cluster backend. By default, vCluster will use vanilla k8s as the virtual Kubernetes cluster.

warning

Changing the kubernetes distribution of vCluster is not supported, once you deployed a virtual cluster.

Vanilla k8s

In order to use vanilla k8s as a backing cluster, create a vCluster with the following command:

vcluster create my-vcluster

Connect to the vCluster and start using it:

kubectl get ns
...

For more configuration options, please take a look at the k8s distro reference

Compatibility matrix showing which host K8s version (left column) is supported by which vCluster K8s distro versions.

	v1.29.0	v1.28.4	v1.27.8	v1.26.11
1.29	✅	🆗	🆗	🆗
1.28	🆗	✅	🆗	🆗
1.27	🆗	🆗	✅	🆗
1.26	🆗	🆗	🆗	✅

Legend:

⚠️ : known issue

✅ : recommended

🆗 : likely compatible

k3s

k3s is a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances.

In order to use k3s as backing cluster, create a vCluster with the following command:

vcluster create my-vcluster --set controlPlane.distro.k3s.enabled=true

Start using it:

kubectl get ns
...

For more configuration options, please take a look at the k8s distro reference

Compatibility matrix showing which host K8s version (left column) is supported by which vCluster K3s distro versions.

	v1.29.0-k3s1	v1.28.5-k3s1	v1.27.9-k3s1	v1.26.12-k3s1
1.29	✅	🆗	🆗	🆗
1.28	🆗	✅	🆗	🆗
1.27	🆗	🆗	✅	🆗
1.26	🆗	🆗	🆗	✅

Legend:

⚠️ : known issue

✅ : recommended

🆗 : likely compatible

k0s

k0s is an all-inclusive Kubernetes distribution, which is configured with all of the features needed to build a Kubernetes cluster and packaged as a single binary for ease of use. vCluster supports k0s as backing virtual Kubernetes cluster.

In order to use k0s as backing cluster, create a vCluster with the following command:

vcluster create my-vcluster --set controlPlane.distro.k0s.enabled=true

Start using it:

kubectl get ns
...

Please note that dual stack networking is not supported with k0s, you will be able to deploy it on a dual stack host cluster, but it will not have all the dual stack features.

For more configuration options, please take a look at the k8s distro reference

Compatibility matrix showing which host K8s version (left column) is supported by which vCluster k0s distro versions.

	v1.29.1-k0s.0	v1.28.2-k0s.0	v1.27.6-k0s.0	v1.26.9-k0s.0
1.29	✅	🆗	🆗	🆗
1.28	🆗	✅	🆗	🆗
1.27	🆗	🆗	✅	🆗
1.26	🆗	🆗	🆗	✅

Legend:

⚠️ : known issue

✅ : recommended

🆗 : likely compatible

Data Persistence

vCluster allows you to use either etcd or a relational database via KINE as a backend. This feature provides flexibility to vCluster operators. The available datastore options allow you to select a datastore that best fits your use case.

vCluster supports the following datastore options:

• Embedded SQLite (default with Persistent Volume)
• PostgreSQL
• MySQL
• MariaDB
• etcd

warning

Changing the backing store of vCluster is not supported once deployed.

Configuration Options

Embedded SQLite (Default)

This is the default, so you don't need to configure anything. If you want to explicitly set this option, you can use:

controlPlane:
  backingStore:
    database:
      embedded:
        enabled: true

Embedded SQLite (No PV)

By default vCluster wil use a persistent volume claim to store the data in. You can also instead use an emptyDir to store the virtual cluster data.

In order to use an emptyDir to store the data instead of a persistent volume, please create a values.yaml with the following contents:

controlPlane:
  statefulSet:
    persistence:
      volumeClaim:
        enabled: true

Then upgrade or recreate the vCluster with:

vcluster create my-vcluster -n my-vcluster --upgrade -f values.yaml

Potential Data Loss

This method should only be used for testing purposes, as data will be lost upon pod recreation.

Embedded Etcd

This is a pro only feature, but allows you to deploy etcd within each vCluster that allows you to use HA (which embedded SQLite doesn't allow):

controlPlane:
  backingStore:
    etcd:
      embedded:
        enabled: true

Deployed Etcd

This will deploy an etcd instance alongside vCluster that is used as a backing store:

controlPlane:
  backingStore:
    etcd:
      deploy:
        enabled: true

MySQL / MariaDB

In its most common form, the option for MySQL and MariaDB has the following format:

controlPlane:
  backingStore:
    database:
      external:
        enabled: true
        dataSource: mysql://username:password@tcp(hostname:3306)/database-name

If you specify a database name and it does not exist, the server will attempt to create it.

Postgresql

In its most common form, the option for PostgreSQL has the following format:

controlPlane:
  backingStore:
    database:
      external:
        enabled: true
        dataSource: postgres://username:password@hostname:port/database-name

More advanced configuration parameters are available. For more information on these, please see https://godoc.org/github.com/lib/pq.

If you specify a database name and it does not exist, the server will attempt to create it.

Deploy vCluster

Deploy a vCluster instance called my-vcluster to namespace team-x. The installation instructions use vCluster CLI, but there are other installation options as well.

1. Install the vCluster CLI.

   Homebrew

   brew install loft-sh/tap/vcluster-experimental

   If you installed the CLI using brew install vcluster, you should brew uninstall vcluster and then install the experimental version. The binaries in the tap are signed using the Sigstore framework for enhanced security.

   Mac (Intel/AMD)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.1/vcluster-darwin-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Mac (Silicon/ARM)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.1/vcluster-darwin-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Linux (AMD)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.1/vcluster-linux-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Linux (ARM)

   curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.1/vcluster-linux-arm64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

   Windows Powershell

   md -Force "$Env:APPDATA\vcluster"; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Tls,Tls11,Tls12';
   Invoke-WebRequest -URI "https://github.com/loft-sh/vcluster/releases/download/v0.20.0-beta.1/vcluster-windows-amd64.exe" -o $Env:APPDATA\vcluster\vcluster.exe;
   $env:Path += ";" + $Env:APPDATA + "\vcluster";
   [Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::User);

   Reboot Required

   You may need to reboot your computer to use the CLI due to changes to the PATH variable (see below).

   Check Environment Variable $PATH

   Line 4 of this install script adds the install directory %APPDATA%\vcluster to the $PATHenvironment variable. This is only effective for the current Powershell session, i.e. when opening a new terminal window,vcluster may not be found.

   Make sure to add the folder %APPDATA%\vcluster to the PATH environment variable after installing vcluster CLI via Powershell. Afterward, a reboot might be necessary.

   Alternatively, you can download the binary for your platform from the GitHub Releases page and add this binary to your PATH.

   To confirm that vCluster CLI is successfully installed, test via:

   vcluster --version

2. (Optional) Configure vCluster with vcluster.yaml.

   Create a file called vcluster.yaml with extra configuration for vCluster. Refer to the vcluster.yaml reference docs to explore all configuration options.

3. Deploy vCluster

   vCluster CLI

   1. Deploy vCluster.

      Default config

      vcluster create my-vcluster --namespace team-x

      Custom config

      Include your vcluster-yaml config file.

      vcluster create my-vcluster --namespace team-x --values vcluster.yaml

      When the installation finishes, you are automatically connected to the virtual cluster and can run kubectl commands within the virtual cluster.

   Helm

   1. Create the "team-x" namespace.

      kubectl create namespace team-x

   2. Deploy vCluster.

      Since the v0.20 release is a beta, you need to specify the version or Helm uses the latest GA version.

      Default config

      helm upgrade --install my-vcluster vcluster \
      --repo https://charts.loft.sh \
      --namespace team-x \
      --repository-config='' \
      --version 0.20.0-beta.6

      Custom config

      Include your vcluster-yaml config file.

      helm upgrade --install my-vcluster vcluster \
      --values vcluster.yaml \
      --repo https://charts.loft.sh \
      --namespace team-x \
      --repository-config='' \
      --version 0.20.0-beta.6

   kubectl

   This uses Helm to generate the Kubernetes deployment files before applying with kubectl. Since the v0.20 release is a beta, you need to specify the version or Helm uses the latest GA version.

   1. Create the "team-x" namespace.

      kubectl create namespace team-x

   2. Deploy vCluster.

      Default config

      helm template my-vcluster vcluster --repo https://charts.loft.sh --version 0.20.0-beta.6 -n team-x | kubectl apply -f -

      Custom config

      Include your vcluster-yaml config file.

      helm template my-vcluster vcluster --repo https://charts.loft.sh --version 0.20.0-beta.6 -n team-x -f vcluster.yaml | kubectl apply -f -

   Terraform

   1. Create a main.tf file.

      Since the v0.20 release is a beta, you need to specify the version or Helm uses the latest GA version.

      Default config

      provider "helm" {
        kubernetes {
          config_path = "~/.kube/config"
        }
      }
      
      resource "helm_release" "my_vcluster" {
        name             = "my-vcluster"
        namespace        = "team-x"
        create_namespace = true
      
        repository       = "https://charts.loft.sh"
        chart            = "vcluster"
        version          = "0.20.0-beta.6"
      }

      Custom config

      Include your vcluster-yaml config file.

      provider "helm" {
        kubernetes {
          config_path = "~/.kube/config"
        }
      }
      
      resource "helm_release" "my_vcluster" {
        name             = "my-vcluster"
        namespace        = "team-x"
        create_namespace = true
      
        repository       = "https://charts.loft.sh"
        chart            = "vcluster"
        version          = "0.20.0-beta.6"
      
        values = [
          file("${path.module}/vcluster.yaml")
        ]
      }

   2. Install the required Helm provider.

      terraform init

   3. Generate a plan.

      terraform plan

      Verify that the provider can access your cluster and that the proposed changes are correct.

   4. Deploy vCluster.

      terraform apply

   Argo CD

   To deploy vCluster using ArgoCD, you need the following files:

   • vcluster.yaml for your vCluster configuration. This is optional if you are deploying with default configuration.
   • my-vcluster.yaml for your ArgoCD Application definition.

   1. Create the ArgoCD Application file my-vcluster.yaml.

      Deployment uses the vCluster Helm chart. Since the v0.20 release is a beta, you need to specify the targetRevision or Helm uses the latest GA version.

      Default config

      apiVersion: argoproj.io/v1alpha1
      kind: Application
      metadata:
        name: my-vcluster
        namespace: argocd
      spec:
        project: default
        source:
          chart: vcluster
          repoURL: https://charts.loft.sh
          targetRevision: 0.20.0-beta.6
          helm:
            releaseName: my-vcluster
        destination:
          server: https://kubernetes.default.svc
          namespace: team-x

      Custom config

      Use your vcluster.yaml file to pass the chart values.

      apiVersion: argoproj.io/v1alpha1
      kind: Application
      metadata:
        name: my-vcluster
        namespace: argocd
      spec:
        project: default
        source:
          chart: vcluster
          repoURL: https://charts.loft.sh
          targetRevision: 0.20.0-beta.6
          helm:
            releaseName: my-vcluster
            valueFiles:
              - vcluster.yaml
        destination:
          server: https://kubernetes.default.svc
          namespace: team-x

   2. Commit and push these files to your configured ArgoCD repository and synchronize it with your configured cluster.

   Cluster API

   1. Install the clusterctl CLI.

   2. Install the vCluster provider.

      clusterctl init --infrastructure vcluster

   3. Generate the required manifests and apply using clusterctl generate cluster and kubectl.

      Deployment uses the vCluster Helm chart. Since the v0.20 release is a beta, you need to export the chart version, or Helm uses the latest GA version.

      Default config

      export CLUSTER_NAME=my-vcluster
      export CLUSTER_NAMESPACE=team-x
      export KUBERNETES_VERSION=1.29.3
      export CHART_VERSION=0.20.0-beta.6
      
      kubectl create namespace ${CLUSTER_NAMESPACE}
      
      clusterctl generate cluster ${CLUSTER_NAME} \
      --infrastructure vcluster \
      --kubernetes-version ${KUBERNETES_VERSION} \
      --target-namespace ${CLUSTER_NAMESPACE} | kubectl apply -f -

      Custom config

      Include your vcluster-yaml config file.

      export CLUSTER_NAME=my-vcluster
      export CLUSTER_NAMESPACE=team-x
      export KUBERNETES_VERSION=1.29.3
      export HELM_VALUES=$(cat vcluster.yaml)
      export CHART_VERSION=0.20.0-beta.6
      
      kubectl create namespace ${CLUSTER_NAMESPACE}
      
      clusterctl generate cluster ${CLUSTER_NAME} \
        --infrastructure vcluster \
        --kubernetes-version ${KUBERNETES_VERSION} \
        --target-namespace ${CLUSTER_NAMESPACE} | kubectl apply -f -

   4. Execute the following command to wait for the vCluster custom resource to report a ready status.

      kubectl wait --for=condition=ready vcluster -n $CLUSTER_NAMESPACE $CLUSTER_NAME --timeout=300s

   Learn more about Cluster API Provider for vCluster