21.30. kube-lib - Kubernetes Library

The following documentation is for Kubernetes Library (kube-lib) content package at version v4.8.0-alpha00.25+g8e224a5bec085c80968ec27e0332c339a65d0659.

License: Kube-Lib is APLv2

This document provides information on how to use the Digital Rebar Kube-Lib content add-on. Use of this content supports common functions needed for other Kubernetes workflows like KRIB and EdgeLab.

21.30.1. Digital Rebar Kube-Lib

Kube-Lib is a library of standard Kubernetes (and k3s) installation processes leveraging v4.6 cluster pattern and other advanced features of Digital Rebar.

Note: Kube-Lib replaces install components in KRIB and edge-lab.

21.30.1.1. Local vs Online Requirements

By default, Kube-Lib uses online components; it will attempt to download and store local copies when posssible. For that reason, our goal is to look for local copies first. Depending on the use-case, that may allow operators to use Kube-Lib is Air-Gap environments by pre-populating resources.

21.30.1.2. Kube-Lib Basics

Kube-Lib is a Content Pack addition to Digital Rebar Provision. It uses the Multi-Machine Cluster Pattern v4.6+ which provides atomic guarantees. This allows for Kubernetes leader(s) to be dynamically elected, forcing all other nodes to wait until the kubeadm on the elected leader to generate an installation token for the rest of the nodes. Once the Kubernetes leader is bootstrapped, the Digital Rebar system facilitates the security token hand-off to rest of the cluster so they can join without any operator intervention.

21.30.1.3. Elected -vs- Specified Leaders

By default, the Kube-Lib process will dynamically elect a leader(s) for the Kubernetes cluster. This leader is simply selected by the cluster manager and the rest of the cluster will coalesce around the elected leader(s).

If you wish to specify a specific machines to be the designated leaders using cluster/leader to true, you can do so by setting a Param on the target machine(s).

21.30.2. Operating Kube-Lib

21.30.2.1. Use kubectl - from anywhere

Once the Kubernetes cluster build has been completed, you may use the kubectl command to both verify and manage the cluster. You will need to download the conf file with the appropriate tokens and information to connect to and authenticate your kubectl connections. Below is an example of doing this:

# get the Admin configuration and tokens
drpcli profiles get cluster01 param kube-lib/kube-conf --expand > kube.conf
export KUBECONFIG=~/Downloads/kube.conf
kubectl get nodes

21.30.3. Object Specific Documentation

21.30.3.1. profiles

The content package provides the following profiles.

21.30.3.1.1. helm-openfaas

Overrides Array of Helm charts to install OpenFaaS

21.30.3.2. stages

The content package provides the following stages.

21.30.3.2.1. kube-lib-dashboard

Installs and runs Kubernetes Dashboard after a cluster has been constructed. This stage is idempotent and can be run multiple times.

Note: will skip if the dashboard version does not start with v

21.30.3.2.2. kube-lib-helm

Installs and runs Helm Charts after a cluster has been constructed.

This stage is idempotent and can be run multiple times. This allows operators to create workflows with multiple instances of this stage. The charts to run are determined by the edge-lab/helm-charts parameter.

Unless helm is uploaded to file helm/helm, this stage requires internet access.

21.30.3.2.3. kube-lib-k3s-cluster

Coordinates operations across multiple machines to install k3s using the v4.6 cluster manager patter

  • Downloads k3s to cache
  • Uses kube-lib-k3s-leader to install the leader
  • Uses kube-lib-k3s-worker to install the workers

21.30.3.2.4. kube-lib-k3s-machine-install

Installs k3s using v4.6 cluster pattern Will install leader if cluster/leader: true Will install worker if cluster/leader: false

21.30.3.3. tasks

The content package provides the following tasks.

21.30.3.3.1. kube-lib-helm-install

Installs Helm on the leader. This uses the Digital Rebar Cluster pattern so cluster/profile and cluster/leader must be set.

21.30.3.3.2. kube-lib-k3s-cluster

This uses the v4.6+ cluster install pattern so it MUST be started from the cluster manager, not from one of the members of the cluster.

If the cluster/filter does not match any machines then the task fail and allow operators to create the correct selectors.

For operational guidelines, see Multi-Machine Cluster Pattern v4.6+

21.30.3.3.3. kube-lib-k3s-machine-install

Install k3s on machines based on v4.6 cluster pattern. Generally, this is started by the kube-lib-k3s-cluster workflow and not called directly.

cluster/leader: true becomes the cluster leader. others become workers. cluster/manager: no operation

For operational guidelines, see Multi-Machine Cluster Pattern v4.6+

21.30.3.3.4. kube-lib-kubectl-install

Installs KubeCTL on machine.

Looks up from ‘kube-lib/kubernetes-download-url’ using ‘kube-lib/kubernetes-version’

Saves binary to files/kubernetes/kubectl-[arch] so only download is required.

Air-Gap: populate files/kubernetes/kubectl-[arch] to avoid download.

21.30.3.3.5. kube-lib-dashboard-install

Installs Dashboard on the Cluster. Saves install token into k3s/dashboard-token Note: will skip if the dashboard version does not start with v

To start the dashboard, use kubectl proxy then open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

21.30.3.3.6. kube-lib-helm-charts

Runs Helm based on Charts defined in kube-lib/helm-carts.

There are a wide range of options available that are defined in the Param

This uses the Digital Rebar Cluster pattern so cluster/profile and cluster/leader must be set.

The task is designed to be idempotent; however, some helm operations are not.

21.30.3.4. workflows

The content package provides the following workflows.

21.30.3.4.1. k3s-cluster-build

Builds a k3s cluster and secondary actions on all machines in lab Uses the task-library cluster building components

Downloads from the Rancher Github (k3s/download-url) unless binary is already present as /files/k3s/k3s

21.30.3.4.2. k3s-machine-install

Installs k3s on a machine (can be either worker or leader) Designed to be run by k3s-cluster-builder

21.30.3.5. params

The content package provides the following params.

21.30.3.5.1. kube-lib/api-port

The API bindPort number for the kubernetes/k3s cluster . Defaults to ‘6443’.

21.30.3.5.2. kube-lib/dashboard-token

Param is set (output) by the Dashboard install process

To start the dashboard, use kubectl proxy then open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

Note: this is stored as a secure object so the decode is required

21.30.3.5.3. kube-lib/dashboard-version

Version of Dashboard to install.

Note: Required “v” prefix or task Will skip Dashboard install.

Override the default in cluster or global profile to use a different version.

21.30.3.5.4. kube-lib/helm-version

Allows operators to determine the version of Helm to install Includes the download URLs and sha256sums Uses task-lib download-utilities format

21.30.3.5.5. kube-lib/kube-config

Param is set (output) by the cluster building process

To use the file, use one of the following * save to $HOME/.kube * export KUBECONFIG=[file] * pass –kubeconfig=[kube.config] to the kubectl or other tools.

If your cluster is using the default cluster01 then tou can retrieve the kube.config file using: drpcli profiles get cluster01 param kube-lib/kube-config –decode > kube.config

Note: this is stored as a secure object so the decode is required

21.30.3.5.6. kube-lib/kubectl-version

Allows operators to determine the version of kubectl to install Includes the download URLs and sha256sums

21.30.3.5.7. kube-lib/node-token

Param is set (output) by the leader during cluster building process HAS NO DEFAULT: param is used as a timing gate between the Leader and Workers

21.30.3.5.8. kube-lib/helm-charts

21.30.3.5.9. Install Helm Charts

Array of charts to install via Helm. The list will be followed in order.

Work is idempotent: No action is taken if charts are already installed.

Fields: chart and name are required.

Options exist to inject additional control flags into helm install instructions:

  • name: name of the chart (required)
  • chart: reference of the chart (required) - may rely on repo, path or other helm install [chart] standard
  • namespace: kubernetes namespace to use for chart (defaults to none)
  • params: map of parameters to include in the helm install (optional). Keys and values are converted to –[key] [value] in the install instruction.
  • set: array of values to set in the helm install (optional). values are converted to –set [value] in the install instruction.
  • targz (optional) provides a location for a tar.gz file containing charts to install. Path is relative.
  • repos (optional) adds the requested repos to helm using helm repo add before installing helm. syntax is [repo name]: [repo path].
  • sleep (optional): time in seconds to wait after install (defaults to none)
  • postkubectl (optional) map of kubectl [request] commands to run after the helm install - stored using the map key
  • prekubectl (optional) array of kubectl [request] commands to run before the helm install
  • git (optional): array of urls for cloning repos into local path

NOT YET PORTED FROM KRIB * wait: wait for name (and namespace if provided) to be running before next action * templates (optional) map of DRP templates keyed to the desired names (must be uploaded!) to render before doing other work. * templatesbefore (optional) expands the provided template files inline before the helm install happens. * templatesafter (optional) expands the provided template files inline after the helm install happens

example:

[
  {
    "chart": "stable/mysql",
    "name": "mysql"
  }, {
    "chart": "openfaas/openfaas",
    "name": "openfaas",
    "namespace": "openfaas",
    "git": [
      "https://github.com/openfaas/faas-netes/"
    ],
    "repos": {
      "openfaas":"https://openfaas.github.io/faas-netes/"
      },
    "prekubectl": [
       "apply -f ./faas-netes/namespaces.yml",
       "apply -f ./faas-netes/yaml_amd64"
    ],
    "postkubectl": {
      "openfaas/password": "-n openfaas get secret basic-auth -o jsonpath=\"{.data.basic-auth-password}\" | base64 --decode"
    },
    "set": [
      "functionNamespace=openfaas-fn",
      "generateBasicAuth=true"
    ],
    "sleep": 10
  }
]

21.30.3.5.10. kube-lib/k3s-version

Allows operators to determine the version of k3s to install Includes the download URLs and sha256sums

21.30.3.5.11. kube-lib/kubernetes-version

Allows operators to specify the version of Kubernetes to install

to get latest, curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt

21.30.3.5.12. openfaas/password

Param is set (output) by the leader during OpenFaaS helm chart install.