Skip to content

title: How dr-provision Boots Machines tags: - explanation - architecture

How dr-provision Boots Machines

As part of its role as a system provisioner, dr-provision manages all aspects of how managed machines boot into all of the environments necessary to perform initial discovery and configuration, install the desired OS, run the desired final workload, and decomission the machine.

To make all of this work, there are several systems that are configured on the fly based on a combination of boot environments, machine parameters, and system wide configuration settings.

DHCP Operations

The DHCP server is responsible for pointing Machines at the proper bootloader when they request a PXE or HTTP boot and dr-provision thinks they should boot over the network.

Static File Services

The static TFTP, HTTP, and HTTPS servers perform most of the heavy lifting for implementing network boot and OS installation. To make this task easier, the static file services have the ability to mount, unmount, and replace arbitrary sets of virtual filesystems that are overlayed on top the files stored on-disk under the tftpboot directory. These can include the contents of archives that dr-provision can read natively, transparent proxies to remote HTTP(s) servers, and virtual files provided by rendering templates.

Objects and Params

Machine

dr-provision manages how a Machine should boot by paying attention to the following fields on the Machine object:

  • BootEnv, which refers to the boot environment dr-provision wants to boot the Machine into. As you may imagine, the BootEnv field is directly responsible for the bulk of the information needed to manage how a Machine will boot.
  • HardwareAddrs, which records the hardware addresses of the network devices present in the Machine. HardwareAddrs is the key field that the DHCP server uses to determine how to respond to incoming PXE or HTTP boot requests.

BootEnv

dr-provision uses BootEnv objects to manage how a Machine should boot. The most critical fields on this Object are: * Loaders, which controls what OS provided bootloaders should be used for various system types. * Templates, which contains the templates that should be rendered and provided by the static file system when a Machine is using this BootEnv. All BootEnvs must provide bootloader configuration templates for the bootloaders we support at locations they expect to find them at. BootEnvs that handle OS installation must also provide templates for all configuration files needed by the OS install process. * OS.Name, which contains the name of the operating system the BootEnv provides. This is used to pick out the proper entries in the package-repositories Param, and is used to set the Machine OS field when the Machine is set to a new BootEnv. * OS.SupportedArchitectures, which contains architecture specific values for each architecture this BootEnv can handle. These fields are: * IsoFile, which is the name of the ISO file (or other archive) that can be used to provide the files needed to boot into this environment over the network. If this bootenv is used to install an OS, then an install repository should also be present on the file. It is recommended that all BootEnvs that are network bootable specify an IsoFile, although it is not mandatory with a properly configured package-repositories Param. * Sha256, which should contain a SHA256 checksum for the IsoFile. If present, it will be used to validate that the IsoFile is not corrupt when it is uploaded to dr-provision. * IsoUrl, which contains a URL that can be used by external tooling to download the IsoFile and upload it to dr-provision. * Kernel, which should contain a partial path to where dr-provision should provide the kernel used to boot this BootEnv. This path should not begin with a /, and it should be relative to where the kernel resides on the IsoFile (if specified) or the install repository for the .OS.Name (if using package-repositories). Boot environments must define a Kernel if they are intended to be booted to over the network. * Initrds, which should contain a list of partial paths to the initrds that dr-provision needs to provide for this BootEnv. It has the same restrictions as the Kernel. * BootParams, which should contain a templatized version of the arguments passed to the Kernel at boot. * Kernel, which is a legacy amd64 specific field. It is superceded by the OS.SupportedArchitectures["amd64"].Kernel field. * Initrds, which is a legacy amd64 specific field. It is superceded by the OS.SupportedArchitectures["amd64"].Loaders field * BootParams, which is a legacy amd64 specific field. It is superceded by the OS.SupportedArchitectures["amd64"].BootParams field

Params

bootenv-customize

bootenv-customize is used to customize various fields in a BootEnv on a per-machine or a global basis. It consists of a map of bootenv Name to structures with the following fields: * Loaders, which is merged into the Loaders field on the BootEnv. * Templates, which is merged into the Templates field on the BootEnv on a template Name by Name basis. * OS.Name, which overrides the OS.Name field on the BootEnv. * OS.SupportedArchitectures, which gets merged into the OS.SupportedArchitectures field.

bootenv-customize is applied according to the usual merging and application rules for Params.

package-repositories

package-repositories serves two purposes:

  1. Provide a central location to define what remote package repositories should be used to install packages from when a task needs to install a package or when installing an operating system.
  2. Provide a remote location that can be used to provide a kernel, initrds, and bootloaders when booting a Machine into a BootEnv when dr-provision does not have the files available locally.

package-repositories consists of a list of repository definition objects, which have the following fields:

  • tag, which provides a non-unique identifier for this repository. It is used as part of the name when rendering the repository definition into a format suitable for the final package manager, if the package manager has the concept of a named repository. Aside from that, it is primarily for human consumption.
  • repoType, which refers to the package management system that can access this repository definition. If set, it must be set to one of yum, zypp, or apt. If not set, it is inferred based on the the contents of the os field.
  • os, which consists of a list of operating systems that this repository is applicable to. When dr-provision is selecting repositories for a Machine, it matches entries in this list to the OS field on the Machine object.
  • arch, which determines what system architecture the packages present in the repository are for. If the repository provides packages for multiple architectures, or if architecture will be determined by package manager URL variables, this value should be set to any.
  • url, which contains a URL to the root of the package repository.
  • bootloc, which contains a URL to a location we can find bootloaders, kernels, and initrds for this repo. It is only meaningful if installSource is true, and if not set url will be used instead.
  • installSource, which should be set if the repository can be used to boot a machine. If installSource is true, the following constraints must also be true:
  • The os field must contain exactly one entry.
  • The url and bootloc fields must not contain any package manager specific wildcards.
  • arch must not be any.
  • securitySource, which should be set if the repo provides mandatory package updates that should be applied during OS installation. Mainly used by Debian and its derivatives.
  • distribution, which is the name of the distribution this repo refers to. For Debian and derivatives, it must be the codename of the distribution. For everything else, it should be the major number of the release version.
  • components, which is a list of component sub-repositories to pull packages from. What this field should contain depends on the repository type and the details of the package manager.

How They tie Together

Providing Local Content

Whenever an archive is uploaded to dr-provision via drpcli isos upload artifactName.ext or the equivalent UX operation, the contents of the file will be made available at mounts/isos/artifactName.ext. This may happen directly using dr-provision's built-in ability serve files from ISOs or uncompressed tarballs directly, or via extracting the installed files using bsdtar.

Changing a BootEnv for a Machine

At system startup or when a machine Bootenv changes, dr-provision performs the following steps:

  1. Merge any bootenv-customize values into the BootEnv based on the usual Param precedence order. The resulting finalized BootEnv will be cached and used for further operations.
  2. Create a new set of virtual filesystem mounts for the machine
  3. Determine whether dr-provision can boot the BootEnv using local assets.
  4. If there is an expanded artifact at mounts/isos/IsoFile, we can. Add a link from machines/uuid/boot to mounts/isos/IsoFile to the new set of virtual filesystems.
  5. Otherwise, see if there is a repository that can be used as an installSource for the bootenv OS.Name. If there is, add a transparent proxy from machines/uuid/boot to the calculated boot location for the repository to new set of virtual filesystems.
  6. If neither can happen, refuse to change the bootenv on the machine and return an error.
  7. For each Template the BootEnv provides, render the template. If it fails to render, refuse to change the bootenv on the machine and return an error. Otherwise, add a link from the template path to the new template to the new set of virtual filesystems.
  8. Replace the virtual filesystem mounts owned by the Machine with the new set of virtual filesystems. This operation is atomic, and once it is finished the Machine can boot into the new BootEnv.

Rendering Package Repositories