22.27. image-deploy - Image Deploy

The following documentation is for Image Deploy (image-deploy) content package at version v4.8.0-alpha00.23+g0d8d59e518bbf87d75d8b1d2b105750e54b3edbb.

The image-deploy plugin deploys images to systems. These images can be tarballs that get untarred onto defined partition-based filesystems or raw disk images that get blown onto the disk. In either case, additional files can be injected into the images for post-install operations.

The provided stages and workflows also know how to drive cloud-init and provided the information required to let cloud-init startup the post-install process. This includes injecting DRPCLI-based runners.

The image-deploy system is built upon and extends the curtin project. Curtin is driven by a configuration YAML file that is built dynamically based upon the value of parameters. These parameters drive the actual imaging process.

22.27.1. Getting Started

The ‘image-deploy-base’ Workflow has been defined as a getting started reference example. It contains the mose basic components needed for image provisioning.

22.27.2. Image-Deploy Image Construction

image-deploy requires an image. This image can be in many different formats. The format type defines what additional operational parameters that need to be set. These images can be built using RackN’s image-builder content pack, using Hashicorp’s Packer tool (examples can be found here: FILL_IN), or other tools that generate images. The images have some specific requirements that must be met for success.

The first requirement is that all images must have a /curtin directory in the top-level directory of the base image file. If the image is a raw disk image, this directory must be backed into that. If the image is a rootfs tarball, the directory must be added as part of that tarball. This must be created by the base image or first image applied.

The second requirement is that all drivers must be added to the system in the base image or additional tarball packages. The image-deploy components do NOT handle driver updates or installs.

The third requirement is for raw disk images. The disk image must be constructed for the the boot-style of the target hardware. If the hard is going to boot in UEFI mode, then the disk image must be formatted with EFI partitions that hold EFI bootloaders. The same applies for legacy systems. There are disk layouts that can work for both.

A final item to consider during image construction is how will the next phase of operation start. This means what will run the next phase of operation. For windows, this could be an unattend.xml that is part of the image or injected by additional parameters. Or it could be cloud-init is installed and set to automatically start. The image-deploy system has the ability configure cloud-init meta-data and user-data with default to start a DRP runner on boot. The system will also inject scripts to install a runner for the OS type in question. These could also be called to install the agent as well. Depending upon the choice of startup method, different things need to be done. Some involve adding things to the image.

22.27.3. Image-Deploy Operations

Once an image id built, the system needs to know where it is, what type of image it is, what OS it contains, and what startup process to use. These are all defined as parameters that can be added to a machine. Because all of this information is consistent across all machines using this image, it is best practice to create a profile that contains all these parameters.

To define the location of the image file, you must use ONE of the following parameters. If both are used, the system will use the image-deploy/image-file value.

  • image-deploy/image-file - This is a path relative to the tftpboot directory of the running DRP endpoint. e.g. If the image has been uploaded with the files API to images/my-win-image.raw.gz, then the value of this parameter would be files/images/my-win-image.tar.gz.
  • image-deploy/image-url - This is a URL that references the image. This must be an http or https URL that does NOT require authentication.

To define the type and content of the image, all of these values must be specified:

  • image-deploy/image-type - This is the curtin image type value. These types are define in the parameter and validated by the systems. Some examples are: dd-gz, tgz, and others.
  • image-deploy/image-os - This is the type of OS being installed. Current values are are linux or windows. The value of basic is deprecated and maps to windows.
  • image-deploy/image-os-subtype - This is the subtype of the image-deploy/image-os value. This is ignored currently for windows, but for linux, this is used to indicate a raw image. The raw value indicates that the disk is a full disk image and curtin should not run any hooks to alter the image state.

Additional parameters can be used to override and expand the system.

22.27.4. Image-Deploy Customizations

With the basic parameters set in a profile, additional parameters can be added to drive further customization and enhancements.

22.27.4.1. Partitioning

When using a raw disk image, partitioning is not used other than to select the hard disk. By default, this will be specified by the image-deploy/install-disk parameter with a value of /dev/sda. This value can be overridden to direct the install to a different location.

When using a rootfs and other tarballs, the system will use the image-deploy-install-disk as the default disk and use a simple GPT-based filesystem layout chosen by curtin. A custom partition table can be added by using the curtin/partitions parameter. This object-based parameter will be converted to yaml and injected into the curtin-config yaml.

The curtin/partitions parameter is an object that defines the disks, partitions, filesystems, and mount points for system. The definition of the object can be found here: https://curtin.readthedocs.io/en/latest/topics/storage.html.

22.27.4.2. Additional Content

Once the base image and the RackN content tarball are added to the system, additional tarballs can be added to the main disk. These are defined in the image-deploy/additional-tarballs parameter. This is a list of objects that define file locations or URLs, and format (like the image-deploy/image-type parameter. The assumption is that these tarballs will be applied at the root level of the mounted images. These MUST be built prior to the curtin-deploy task is executed.

The image-deploy tasks build one tarball and apply it for all systems. This is the RackN Content tarball. This contains at a minimum an OS appropriate runner, the curtin hooks needed to complete installation if required, an install script to install the DRPCLI as a runner, and optionally cloud-init configuration pieces.

The following pieces are installed and their locations:

  • drpcli(.exe) - For linux, this is installed to /usr/local/bin. For windows, this is /curtin.
  • curtin hooks - These will go into /curtin and can be ignored.

For window’s systems, there is an additional file added if specified. If image-deploy/windows-unattend-template is defined, it should contain the name of a template that is loaded into the system. This template will be expanded and stored in the newly imaged filesystem at the location specified by image-deploy/windows-unattend-path. The default path is Windows/Panther/unattend.xml. This can be used to inject a dynamic unattend xml file for startup usage. This allows for a more flexible image.

For further flexibility, the system can be configured to configure cloud-init or cloud-base as well.

22.27.4.3. Cloud-Init / Cloud-Base

The image-deploy system will attempt to configure cloud-init (or Cloud-Base the windows version of cloud-init) by default. To turn this off, set the image-deploy/use-cloud-init parameter to false.

For linux systems, the system will configure cloud-init to use a data source to find meta-data and user-data. These will be injected locally by default into the /curtin directory. The image-deploy/cloud-init-base-url parameter allows this to be overridden. This can be other directory locations and the files will be placed accordingly. The system also allows for a remote file location through specific URLs or parameter expansion. A common use would be to specific, {{.Machine.Url}}/. This expansion would map to files provided by the image-deploy-cloud-init stage.

For windows systems, the system will configure cloud-base to use the custom RackN data source that works similar to the data sources used by linux. The same parameters control the location and contents of the meta-data and user-data files.

The contents of the meta-data file includes the hostname, ssh access keys, admin-username, and admin-password. Hostname is specified by the Machine.Name field. The ssh access keys are pulled from the access-keys parameter. The admin-username and admin-password come from the parameters image-deploy/admin-username and image-deploy/admin-password respectively if specified.

The final piece of cloud-init data is the user-data file. If not specified, the default action will be to install and start the DRP runner as a last action of cloud-init. Other user-data content can be provided by filling in the string parameter, cloud-init/user-data. For best results, one should include a final run command to install and start the agent.

For linux,

#cloud-config
runcmd:
  - /curtin/drpcli-install.sh

For windows,

#cloud-config
runcmd:
  - /curtin/drpcli-install.bat

22.27.5. Problem Determination

To debug issues, you can set the curtin/debug parameter to true and addtional curtin logs will be generated.

22.27.6. Object Specific Documentation

22.27.6.1. params

The content package provides the following params.

22.27.6.1.1. curtin/partitions

The partition configuration for a Machine which is deployed via the Root Tarball (tgz image type). You must include the storage, version, and config stanzas in the Param. See examples below for details.

This is a Curtin YAML format definition. For complete documentation on the data structure values and options, please see the documentation at:

If unspecified, the default action to setup a simple partition table suitable for linux rootfs installs (single / root filesystem as big as full disk size, no swapfile located in root as /swap.img).

Examples can be found in the Image Deploy Storage Examples. Here is an external link to the Examples (please note that this location may change in the future):

22.27.6.1.2. image-deploy/image-os-subtype

While image-deploy/image-os defines the overall OS type. This variable can be used to define the sub-type.

This is really only used for linux and it supports raw and other.

raw is used to support a raw image that does not need hooks.

other is used to attempt to support other linux (not debian-based or redhat-based).

22.27.6.1.3. image-deploy/swapfile-maxsize

The maximum size of the curtin created swap file. This will default to 8GB if unspecified.

The value of this string is a size string, ##KB, ##MB, ##GB, or ##TB. The B is optional.

22.27.6.1.4. cloud-init/user-data

This is the data to return to the node on cloud init calls.

This Param should contain either the cloud-config YAML user-data or a BASH script. You must pass content as a multi-line string, and it should conform to cloud-init correct syntax and formats. Reference cloud-config examples:

Note that several capabilities covered in cloud-config examples may be available directly in the image-deploy plugin via other Params configurations.

Digital Rebar Platform provides the user-data and metadata as a local set of files, written in to the deployed image. This is typically written in to the /curtin directory, where cloud-init will consume them.

22.27.6.1.5. curtin/network/bootmac

The MAC Address the system booted from. Used as the interface mac in the network config.

22.27.6.1.6. curtin/network/gateway

The default gateway address for the network.

  • example: 10.10.10.1

22.27.6.1.7. image-deploy/admin-username

If specified, this can be used to set the admin username. This should be used in conjunction with image-deploy/admin-password.

If not specified, the default will be set to rocketskates.

22.27.6.1.8. image-deploy/image-url

The URL location for the image for image-deploy is to install. This must be a URL that is accessible from the Machine being deployed at the time of deployment.

22.27.6.1.9. image-deploy/other-dracut-path

The path to the dracut executable used by the linux/other os-type/os-sub-type pair.

22.27.6.1.10. image-deploy/other-initrd-path

The path to the dracut file that dracut should create.

The ${kv} string should be used to indicate the version of the kernel.

22.27.6.1.11. image-deploy/swapfile-size

The exact size of the curtin created swap file. This will default to something upto the max size.

The default if unspecified is to create a file bounded by the value of image-deploy/swapfile-maxsize or 8GB.

The value of this string is a size string, ##KB, ##MB, ##GB, or ##TB. The B is optional.

To turn off creation of the swapfile, set this value to 0.

22.27.6.1.12. curtin/prereqs

List of packages needed to install and run curtin. Packages come from the image-deploy plugin and can be found at:

See Schema default for an example.

22.27.6.1.13. curtin/skip-finalize-windows

Skip running the Windows finalize tasks. Default value is false (the finalize tasks SHOULD run).

22.27.6.1.14. id-test-index

If unset, the tests have not started. If set, it is the index of the current test.

22.27.6.1.15. image-deploy/windows-unattend-template

The template to use for unattend.xml file.

22.27.6.1.16. image-deploy/windows-license-key

The license key for the windows operating system as a string.

22.27.6.1.17. curtin/network/template

A Digital Rebar Platform Template to include in to the cloud-config data to define the network configuration of the machine.

22.27.6.1.18. image-deploy/cloud-init-base-url

This parameter defines the base path to the user-data and meta-data files. The valid values are file:///path, http[s]://hostname:port/path, or just path.

The default is for local mode, file:///curtin/.

For remote usage, the value should be `{{.Machine.Url}}/`. This works with the image-deploy-cloud-init stage.

The path MUST end in a slash.

22.27.6.1.19. image-deploy/skip-grub2-configure

Whether to skip trying to reconfigure grub.

If you are deploying a raw image type, you MUST set this value to true as the raw image already (should) contain a valid boot loader installed.

22.27.6.1.20. image-deploy/image-type

The file image may be in many formats. This tells image-deploy what method to use to install.

Validated methods are:

  • tgz - Tar gzip compress filesystem
  • dd-tgz - tar file containing a single gzip compressed disk image
  • dd-txz - tar file containing a single XV compressed disk image`
  • dd-tbz - tar file containing a single BZ compressed disk image`
  • dd-tar - tar file containing a single uncompressed disk image`
  • dd-bz2 - tar file containing a single BZ2 compressed disk image`
  • dd-gz - raw disk image that is gzip compressed
  • dd-xz - raw disk image that is XZ compressed`
  • dd-raw - raw disk image that is uncompressed

Note

For the most up to date list, see the Param type definition below on your DRP Endpoint.

22.27.6.1.21. curtin/version

The version of curtin to install and use. It comes from the image-deploy plugin. The version comes from http://endpoint:8091/files/plugin_providers/image-deploy/curtin-<version>.tar.gz

  • example: 20.2-rackn
  • example: 20.2-rackn3
  • example: master

22.27.6.1.22. id-test-tests

A list of profiles to test through.

22.27.6.1.23. image-deploy/additional-tarballs

A list of additional tarballs to install into the image during deployment time. This Param is an array of objects that define what tarballs to inject in to the final deployed image. This can be used to add configuration data, content, scripts, etc. inside the deployed system.

You can control the order in which multiple tarball files will be unrolled by setting the index value in a numerical order.

Example:

  • This example serves the my-tarball.tar file from the DRP HTTP Files space; which will then be un-tarred in the target Machine filesystem.
[
  {
    "drpserved": true,
    "format": "tar",
    "index": 1,
    "uri": "files/my-tarball.tar"
  }
]

The values are defined as follows:

  • drpserved: will automatically fill in the full HTTP URI path to the DRP Files server if set to true. Do not specify a leading forward slash (/). If false, specify a full HTTP URI to the tarball file artifact.
  • format: a Curtin file format type (eg tar, or tgz). See Sources in note below.
  • uri: the URI of the resources, either a full HTTP URI path if drpserved is false, or a relative URI path with no leading forward slash, on the DRP HTTP Files server.
  • index: a unique numerical value to define the order in which the tarballs should be unrolled, as files in subsequent tarballs may potentially overwrite files in earlier tarballs.

Note

Sources definition for Curtin configs is documented at: https://curtin.readthedocs.io/en/latest/topics/config.html#sources

22.27.6.1.24. image-deploy/admin-password

If specified, this can be used to set the admin password. This should be used in conjunction with image-deploy/admin-username.

If not specified, the default will be set to RocketSkates.

22.27.6.1.25. image-deploy/install-disk

The disk that the image-deploy system should install to. The default value is /dev/sda. If the disks in the system are iterated with different device names, then you must adjust this Param appropriately. For example, virtual machines may iterate their disks to the Sledgehammer environment as /dev/vda, so you would set this Param value to /dev/vda in that case.

For troubleshooting, boot the Machine in to Sledgehammer (eg via the discover-base workflow; or any other appropriate workflow), and use lsblk to iterate and identify the device names of the disks.

22.27.6.1.26. curtin/debug-deploy-pause

If set to true, then the curtin-deploy task will pause for debugging the deployment environment. The pause will last a total of 30 minutes, after which provisioning will continue.

To force return to normal operation, remove the control file, and the pause loop will escape; continuing provisioning.

  • rm -f /tmp/curtin-debug-pause-deploy

22.27.6.1.27. curtin/network/nameservers

A list of nameservers to use, in array format.

Example:
[ "1.1.1.1", "1.0.0.1", "8.8.8.8" ]

22.27.6.1.28. image-deploy/image-file

The file image for image-deploy to install. This will have the Provisioner’s URL prepended to the filename.

You may use the drpcli isos upload ... command to place an image into the system. If this method is used, please prepend ‘isos/’ to your filename. You MUST NOT have any other directories in the filename; they must reside in the base isos/ directory.

  • example: isos/centos-7.tgz

You may use the drpcli files upload ... command to place an image into the system. If this method is used, prepend ‘files/’ to your filename. You MAY use additional directories in the filename.

  • example: files/images/windows/windows-server-2019.raw.gz

You may also just place the image file under the DRP tftpboot directory. If this method is used, use the path relative to the tftpboot directory.

  • example: my_dir/some_stuff/ubuntu.tgz

In the above example, this might resolve to the following path on the Digital Rebar Platform filesystem, based on a default “production” install type:

  • /var/lib/dr-provision/tftpboot/my_dir/some_stuff/

22.27.6.1.29. image-deploy/use-cloud-init

Set to true (the default) for the image deploy system to attempt to configure cloud-init to use a meta-data/user-data pair of files. These rely on other paraemeters to fill in the secondary files.

The type of cloud-init access is controlled by the parameter, image-deploy/cloud-init-service.

22.27.6.1.30. curtin/debug

Enable enhanced debug output for curtin deployed installs via the image-deploy plugin.

22.27.6.1.31. curtin/network/ifname

The interface name to use inside the curtin network section.

Must be the same interface as the curtin/network/bootmac defined interface.

  • example: enp1s0f0
  • example: eth0

22.27.6.1.32. image-deploy/swapfile-name

The path to the linux swap file created by curtin.

If unspecified, the path will default to /swap.img.

22.27.6.1.33. image-deploy/skip-fstab

Skip rewriting the fstab on the target image.

22.27.6.1.34. image-deploy/skip-network-configure

Whether to skip basic network configuration. Should be set to true if your image already has a working network interface configuration.

22.27.6.1.35. image-deploy/windows-unattend-path

The path to the Windows unattend.xml file. The default value is Windows/Panther/unattend.xml.

22.27.6.1.36. cloud-init/x509-certs

This contains all the x509 certs for the system as a single string. Used to configure access to HTTPS (TLS Enabled) services that may not be recognized by as CA signed certificates.

22.27.6.1.37. curtin/network/cidr

The numeric IP address and netmask for this Machine, in CIDR notation.

  • example: 10.10.10.10/24

22.27.6.1.38. image-deploy/image-os

The file image contains the install contents for an operating system. This variable defines the image OS at a high level.

Valid values are:

  • esxi
  • linux
  • windows

22.27.6.2. stages

The content package provides the following stages.

22.27.6.2.1. id-test-init-test

Run another test stage

22.27.6.2.2. id-test-next-test

Run another test stage

22.27.6.2.3. image-deploy

Deploy an image within sledgehammer

22.27.6.2.4. image-deploy-cloud-init

Stage to handle serving a cloud-init file

22.27.6.2.5. image-deploy-no-erase

Deploy an image within sledgehammer without wiping disks first

22.27.6.3. tasks

The content package provides the following tasks.

22.27.6.3.1. cloud-init-wait

This will cause the system to wait for cloud-init to finish and start the runner before advancing to far into the task list.

The task works for linux, darwin, and windows OSes.

22.27.6.3.2. curtin-deploy

A task to install an image through curtin.

22.27.6.3.3. curtin-install

A task to install curtin.

22.27.6.3.4. id-test-init-test

A task to pick the init test.

22.27.6.3.5. id-test-next-test

A task to pick the next test.

22.27.6.4. workflows

The content package provides the following workflows.

22.27.6.4.1. image-deploy-base

Provides the minimal workflow needed to install an image using the image deploy process. Cloud-init defaults includes starting the runner and ssh-access.

The three Params are required

  • Location of image: either image-deploy/image-file or image-deploy/image-url
  • Operating system: image-deploy/image-os
  • Image format: image-deploy/image-type