23.18. edge-lab - Edge Lab

The following documentation is for Edge Lab (edge-lab) content package at version v4.6.0-beta01.97+g1e33864277dc9bbb839f71f1eff994a0c4f05c23.

Please see http://edgelab.digital for complete the reference architecture and additional documentation.

Assumptions:

  • the leader node have internet access for downloading containerd and k3s binaries
  • all nodes have have internet access for downloading containers
  • drpcli has been installed on client system

Requirements:

  • Edge Lab requires Digital Rebar Platform tip (post v4.2) due to RPi install requirements.
  • K3s has been migrated to use the standardized kube-lib content pack instead of a specialized k3s installation. This also uses the v4.6 cluster pattern where the DRP server acts as the cluster manager.

23.18.1. Building & Installing DRP on Edge Lab

Please see the documentation in https://github.com/digitalrebar/edgelab for building the physical environment.

23.18.2. Setup Edge Lab Content

Note: make sure that you export RS_ENDPOINT to your endpoint!

Edge Lab (edge-lab) is available in the RackN Catalog, you can install it using drpcli catalog item install edge-lab This was done for you as part of the bootstrapping process.

To manually create and upload:

  1. clone edge-lab with DRP content: git clone https://github.com/digitalrebar/provision-content
  2. got to the edge-lab content: cd provision-contentedge-lab
  3. build the content pack: drpcli contents bundle edge-lab.yaml
  4. install the edge-lab content pack: drpcli contents upload edge-lab.yaml

If you make changes, you can repeat steps 3 and 4 to upload the latest content.

23.18.3. Setup K3s Cluster via Kube-Lib

This process uses the standard component_kube_lib module. Use catalog item install kube-lib if it is missing.

Recommended, install kubectl has been installed on your client system.

Note

You _must_ have attached storage in each node for the k3s cluster to operate! This could be an SDcard or USB drive. If the partitions are not xfs, the code will wipe the drive!

By default, the cluster/filter will select ALL machines in the edge-lab cluster. To build your cluster:

  1. select the edge-lab-manager machine (which is also the DRP server)
  2. apply the k3-cluster-build workflow
  3. wait for workflow to complete on cluster manager
  1. this will select and start the k3s install on the leader
  2. after the leader is installed, this will install k3s on all the remaining machines as workders
  1. use drpcli profiles get ``edge-lab-manager` param kube-lib/kube-config –decode > kube.conf` to get the kube.config file. You can also download from the cluster’s profile in the UX.

To access on the DRP server or cluster:

  1. test with kubectl –kubeconfig=kube.conf get nodes

To access from another system, use an SSH tunnel:

  1. setup an ssh tunnel: ssh root@[host wifi ip] -L 6443:[leader node ip]:6443
  2. modify your admin.conf file to use 127.0.0.1 instead of the leader node ip
  3. test with kubectl –kubeconfig=kube.conf get nodes

23.18.4. Access Kubernetes Dashboard

The Kubernetes Dashboard v2 is automatically installed by the k3s-cluster-manager workflow. Admin and service accounts are created for access.

To access the Dashboard:

#. use kubectl –kubeconfig=kube.conf proxy to create a network path to your cluster. #. get the Dashboard authentication token drpcli profiles get edge-lab-manager param kube-lib/dashboard-token –decode #. open the Dashboard using the proxy: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

23.18.5. Install OpenFaaS via Helm

The k3s install above will automatically include Helm v3 and run charts defined in the kube-lib/helm-charts param array.

OpenFaaS is provided as an example in the helm-openfaas profile. To include OpenFaaS in the k3s install, simply assign the helm-openfaas profile on cluster leader based on your architecture (you can assign it to all cluster machines and it will be ignore on the non-leaders).

Since the helm stages are idempotent, either include the profile before running the k3s-install workflow or re-run the workflow.

The OpenFaaS helm chart install will write its password to the cluster profile using openfaas/password. To retrieve the password use drpcli profiles get cluster01 param openfaas/password –decode.

23.18.6. Machine Reset

Setting a machine’s workflow to edge-lab-reboot-to-discover will force the machine through a soft reboot process and back to discovery.

To indicate that the machine has been rebooted, the icon is set to the red recycling image. The icon will remain until it get’s reset.

The machine is avilable once it is marked Runnable again.

23.18.7. Edge Lab Bootstrap

This content pack includes the steps needed to bootstrap the Edge Lab server. This includes a workflow, stage, and tasks to:

  1. Download the required isos (sledgehammer rpi, but others as well).
  2. Set the default preferences to make Edge Lab work.
  3. Set up an ssh key pair if one does not exist and update the global profile with it. This can be used to access the clients from the server.
  4. Set up the storage attach Param for the Edge Lab
  5. Create the default subnet for the Edge Lab.

The start_me.sh script will install the edge-lab content pack and set the DRP server’s workflow to the bootstrap workflow. The status of bootstrapping can be tracked in the jobs for the DRP server.

23.18.8. Object Specific Documentation

23.18.8.1. workflows

The content package provides the following workflows.

23.18.8.1.1. bootstrap-edge-lab

Bootstrap Edge Lab server.

  • Base Bootstrap * Make sure discovery bootenvs are loaded for operation. * Setup an ssh key pair and install it to the global profile. * Set the discovery preferences. * Update the Global Profiles
  • Edge Bootstrap * Edge Lab DHCP Subnet

Do NOT lock the bootstrap - it’s used for the cluster manager too.

23.18.8.1.2. edge-lab-bootstrap

23.18.8.1.2.1. DEPRECATED in favor of bootstrap-edge-lab

Bootstrap Edge Lab server.

  • Base Bootstrap * Make sure discovery bootenvs are loaded for operation. * Setup an ssh key pair and install it to the global profile. * Set the discovery preferences. * Update the Global Profiles
  • Edge Bootstrap * Edge Lab DHCP Subnet

Do NOT lock the bootstrap - it’s used for the cluster manager too.

23.18.8.1.3. edge-lab-reboot-to-discover

Reboot system back into discover

23.18.8.2. params

The content package provides the following params.

23.18.8.2.1. edge-lab/gpio

If a SQUID LED is attached to the RPi, this will turn on different colors. It is very helpful for validating workflows during development

You can supply an hash with specific number of the GPIO pins to set and the 0/1 value to set

Three pin mappings to colors have been pre-wired:

  • 18 = red
  • 23 = green
  • 24 = blue

See: https://github.com/simonmonk/squid https://www.raspberrypi.org/forums/viewtopic.php?f=5&t=5185#p68739

23.18.8.2.2. edge-lab/nfs-share

Path for shared NFS storage for cluster Typically created during bootstrap on DRP host

23.18.8.2.3. edge-lab/reset-workflow

Workflow to set before rebooting system.

23.18.8.3. stages

The content package provides the following stages.

23.18.8.3.1. edge-lab-bootstrap

Edge Lab bootstrap stage to build out the edge lab server.

The following things are done:

  • Run the regular bootstrap
  • Set useful global profile variables
  • Make default subnet for the Edge Lab environment
  • Upload all required content

23.18.8.3.2. edge-lab-reboot-to-discover

Reboot a node back into discover-base

23.18.8.4. tasks

The content package provides the following tasks.

23.18.8.4.1. edge-lab-bootstrap-subnet

A task to set the default subnet for Edge Lab.

If running in a cloud provide, subnet is created but disabled.

23.18.8.4.2. edge-lab-gpio

If a SQUID LED is attached to the RPi, this will turn on different colors. It is very helpful for validating workflows during development

You can supply an array with specific number of the GPIO pins to set.

Three pin mappings to colors have been pre-wired:

  • 18 = red
  • 23 = green
  • 24 = blue

See: https://github.com/simonmonk/squid https://www.raspberrypi.org/forums/viewtopic.php?f=5&t=5185#p68739

23.18.8.4.3. edge-lab-reboot-workflow

A task to reboot to a workflow specifed by the edge-lab/reboot-workflow.

23.18.8.4.4. edge-lab-setup-nfs

Install NFS server and creates a share Primary usecase is to support shared storage for a k3s cluster

23.18.8.4.5. edge-lab-bootstrap-content-check

A task to update content packs for Edge Lab.

23.18.8.4.6. edge-lab-bootstrap-global-profile

A task to set the default subnet for Edge Lab.

If running in a cloud provide, subnet is created but disabled.