26.29. pool

The following documentation is for pool content package at version v0.0.0.

The pool plugin adds a new object to the system, the Pool. The pools can be accessed through the normal API calls or CLI. The Pool can be created, destroyed, and updated like normal objects.

Additionally, there are 4 actions on the pool object and 1 on the machine object to help use pools. These actions add and remove machines to and from pools. Also, allocate and return machines. These allow machines to be reserved within the pool and then unreserved. The machine object gains a return to free action.

Allocation and Return actions can add or remove parameters, add or remove profiles, and set a workflow.

26.29.1. Basic Pool Operations

Before reviewing the actions used, it’s helpful to understand the basic model used by the Pool Plugin. Fundamentally, all pool activities are based on generally available Digital Rebar Provision features such as parameter storage, indexes and filters. For this reason you can always monitor (and adjust) pool activity from the primary API.

26.29.1.1. Target Users

While we’ve worked to make the Pool Plugin easy to manually use from the DRPCLI, the target use case for the Pool Plugin is as an external integration point by another system. It is important to understand and use Digital Rebar Provision workflows, profiles and params correctly when making pooling requests. The pooling system is designed to streamline and integrate with normal processes instead of replacing them.

26.29.1.2. Creating Pools

All machines when created or the pool plugin is configured are placed in a default pool unless already in another pool. Machines can only be in one pool at a time. Pooling is effectively a simple hierarchy with default that the root: you can only add from default and remove to default.

Machines are always in a pool. There is no concept of “unpooled” machines once the Pool Plugin is installed.

26.29.1.3. Allocating Machines

Once machines are in a pool, operators can allocate from and return to a pool (including the default pool). The allocation process does _not_ remove machines from the pool; instead, it is marking machines in the pool as free or in-use.

26.29.1.4. Pool Status

A Machine has a status with in the Pool. The actions alter that status along with the Machines operation. There are six status levels in the Pool.

26.29.1.4.1. Free

Free is the starting state of all machines. This indicates that is a member of the pool and available for allocation. This is the only status that will have the pool/allocated flag to false. Free is also the final stage of a return operation.

26.29.1.4.2. Building

Building represents that a machine has been allocated and as part of the allocation process a target stage has been specified to wait for with the pool/wait-for-stage action parameter. The pool plugin is watching for the machine to enter this stage before moving the system to InUse. Building removes the machine from being able to be allocated.

26.29.1.4.3. HoldBuild

If the machine becomes not Runnable while the pool plugin is waiting for a target stage, the machine will be moved to this status. This indicates a possible error has occurred and the machine should be evaluated. NOTE: The system does NOT currently check the machine’s current task for failure, only if the machine is Runnable.

26.29.1.4.4. InUse

InUse is the complete status for allocation. If no wait-for-stage is provided, the machine is directly placed in this status. This state presents a completed allocation.

26.29.1.4.5. Destroying

Destroying represents that a machine has been requested to return to the Free state, but is not there yet. The pool/wait-for-stage was specified and the machine has not reached that stage. Upon reaching that stage, the machine will change status to Free.

26.29.1.4.6. HoldDestroy

This status is the same HoldBuild status, but for the return process.

26.29.1.5. Safe Workflows

The Pooling Plugin does not limit operational choices to ensure Workflows complete before allocation. Imposing workflow requirements would constrict important use cases; however, operators may want to ensure that machines that are not (re)allocated in the middle of workflows.

To protect workflow integrity, operators can simply add an additional Param (e.g.: pool/ready) into their workflow stages that set ready to false at the start of a workflow and true at the end of the workflow. This parameter can then be used as a filter when requesting machines for allocation.

An even ligher weight approach would be to filter based on a target Stage such as sledgehammer-wait.

26.29.1.6. Behind the Scenes

The pool and allocation states for machines are stored as simple Param data on the machine itself as well as in the pool object. The pool plugin will use the pool object as the source of truth when evaluating the machine states. The overal storage in Digital Rebar Provision is important because it means that all state storage is integrated into Digital Rebar Provision state so there is no risk of conflicting state (as could happen with Terraform or other tools).

  • The pool name is stored in the pool/name Param with a default of default
  • The pool allocated state is stored in the pool/allocated Param with a default of false.
  • The pool status state is stored in the pool/status Param with a default of Free.

While dangerous and NOT recommended, changing or removing these Params manually (outside of using the pool actions) will can change the pool status for the Pool Plugin. The Pool Plugin will attempt to correct these values. Manually, altering the pool object is also NOT recommended, but can be used to force states and will generate updates to the machine over time.

26.29.2. Actions

The following actions are added to the pool object with the exception of the returnToPool which is added to the machine object.

NOTE: Listing and membership can be seen through the normal list and show API calls on the pool object.

26.29.2.1. addMachines

addMachines takes machines from the default pool and moves it to the pool object. The action requires either a list of machines specified by the pool/machine-list or the parameter pool/all-machines set to true indicating all unallocated machines in the default pool.

The machines are added in the Free status.

The command returns an array of objects which contained

  • Status = The status of the most recent action.
  • Allocated = A boolean that represents if the machine is allocated in that pool or not.
  • Name = The name of the machine
  • UUID = The UUID of the machine

26.29.2.2. removeMachines

removeMachines takes Free machines from the pool specified and puts them in default pool. The action requires either a list of machines specified by the pool/machine-list or the parameter pool/all-machines set to true indicating all unallocated machines in the pool specified by pool/name.

The machines are added in the Free status and must be Free in the specified pool.

The command returns an array of objects which contained

  • Status = The status of the most recent action.
  • Allocated = A boolean that represents if the machine is allocated in that pool or not.
  • Name = The name of the machine
  • UUID = The UUID of the machine

26.29.2.3. allocateMachines

allocateMachines allocates a number of machines from the pool and allows for those machines to have a set of parameters applied to the machine, profiles set on the machine, and a workflow requested as part of the allocation. A filter can be applied to inidicate which machines should be considered.

The following parameters can be passed into the action to alter behavior.

  • pool/count - indicates the number of machines to allocate
  • pool/partial-success - indicates if the allocation should succeed if the count can not be satisfied.
  • pool/filter - is a map of field or parameter names with the function to apply on that item. This works like the machines list API call.
  • pool/new-workflow - indicates the workflow for the machine to start once allocated.
  • pool/add-profiles - a list of profiles to add to the machine during allocation.
  • pool/add-parameters - a map of parameters that should be added to the machine during allocation.
  • pool/remove-profiles - a list of profiles to remove from a machine during allocation.
  • pool/remove-parameters - a list of parameters to clear from the machine during allocation.
  • pool/wait-for-stage - a stage the machines should achieve before marking the machine complete.
  • pool/blocking - a boolean that indicates if the system should wait
  • pool/wait-timeout - an integer number of seconds to wait for all machines to achieve the stage specified by pool/wait-for-stage.

The pool/remove-profiles and pool/remove-parameters are removed before the pool/add-profiles and pool/add-parameters are applied. The pool/blocking boolean indicates if the command should wait for the machines to get to the pool/wait-for-stage. The pool/wait-timeout applies for the total time to wait for the set of machines to achieve the stage specified by pool/wait-for-stage. It is NOT applied serially for each machine in the allocation. If pool/wait-timeout is not specified, the command will wait indefinitely.

The command returns an array of objects which contained

  • Status = The status of the most recent status.
  • Allocated = A boolean that represents if the machine is allocated in that pool or not.
  • Name = The name of the machine
  • UUID = The UUID of the machine

26.29.2.4. returnMachines

returnMachines deallocates a machine or set of machines back to the pool. The machines are specified similarly to the poolAddMachines and poolRemoveMachines actions with the pool/all-machines flag or the pool/machine-list list.

Machines must be in the InUse status to qualify for deallocation.

The following parameters can be passed into the action to alter behavior.

  • pool/new-workflow - indicates the workflow for the machine to start once deallocated.
  • pool/add-profiles - a list of profiles to add to the machine during deallocation.
  • pool/add-parameters - a map of parameters that should be added to the machine during deallocation.
  • pool/remove-profiles - a list of profiles to remove from a machine during deallocation.
  • pool/remove-parameters - a list of parameters to clear from the machine during deallocation.
  • pool/wait-for-stage - a stage the machines should achieve before marking the machine complete.
  • pool/blocking - a boolean that indicates if the system should wait
  • pool/wait-timeout - an integer number of seconds to wait for all machines to achieve the stage specified by pool/wait-for-stage.

The pool/remove-profiles and pool/remove-parameters are removed before the pool/add-profiles and pool/add-parameters are applied. The pool/blocking boolean indicates if the command should wait for the machines to get to the pool/wait-for-stage. The pool/wait-timeout applies for the total time to wait for the set of machines to achieve the stage specified by pool/wait-for-stage. It is NOT applied serially for each machine in the allocation. If pool/wait-timeout is not specified, the command will wait indefinitely.

The command returns an array of objects which contained

  • Status = The status of the most recent allocate or deallocate action.
  • Allocated = A boolean that represents if the machine is allocated in that pool or not.
  • Name = The name of the machine
  • UUID = The UUID of the machine

26.29.2.5. returnToPool

This machine object action allows for the machine to return itself to the pool with the same parameters as the returnMachines action on the pool object. The pool is assumed based upon the machine’s current pool. This action only operates on one machine. The command returns “Success” or an error object.

26.29.3. params

The content package provides the following params.

26.29.3.1. pool/all-machines

This parameter is used on the poolAllocateMachines, poolReturnMachines, poolAddMachines, and poolRemoveMachines actions to indicated all the machines applicable for the command should be manipulated.

For poolAllocateMachines, it implies all machines in the pool not already allocated. For poolReturnMachines, it implies all machines in the pool that are allocated. For poolAddMachines, it implies all machines not allocated in the default pool. For poolRemoveMachines, it implies all machines not allocated in the referenced pool.

26.29.3.2. pool/allocated

This parameter indicates if the machines has been allocated in that pool.

26.29.3.3. pool/status

This string parameter provides the status of the machine with in the pool.

This can be:

  • unallocated - indicates an unallocated machine
  • allocated - indicates an allocated machine that may or may not have run through a workflow.
  • complete - indicates an allocated machine that has completed to the specified stage.
  • timeout - indicates an allocated machine that timed out waiting for the specified stage.
  • error - indicates an allocated machine that errored while waiting for the specified stage.

26.29.3.4. pool/wait-for-stage

This string parameter indicates which stage the pool plugin should wait for the machine to achieve before marking the machine complete.

26.29.3.5. pool/add-profiles

This parameter is used on the poolAllocateMachines, poolReturnMachines, and returnToPool actions to add profiles to the machine as part of the action. The parameter is a list of profile names that are appended to the list of profiles on the machine unless already present.

26.29.3.6. pool/blocking

This boolean parameter defines if allocate or return actions should block until all nodes achieve the wait-stage.

26.29.3.7. pool/force

This boolean parameter when used with the returnMachines action will cause the machine to be immediately placed in the Free status. All other waiting, blocking, and timeouts will be ignored. The adding and removing of profiles, parameters, and workflow will be done.

26.29.3.8. pool/partial-success

This boolean parameter defines if the allocation should fail an unwind on errors or continue with the machines accrued so far. This can be used in conjunction with the pool/count parameter to allocate all currently unallocated machines in a pool. The returned structure by poolAllocateMachines will only have the allocated machines.

26.29.3.9. pool/count

This parameter indicates the number of machines allocate from a pool. It defaults to 1.

26.29.3.10. pool/filter

This parameter allows for filtering which machines should be allocated from the pool.

The parameter is a map of strings. The key is the parameter or field on the machine to test. The value is the lookup function that should be applied on that field.

For example:
inventory/data/CPUS: Eq(3)

This would filter allocation for machines with 3 CPUs being reported.

26.29.3.11. pool/remove-profiles

This parameter defines list of profile names that should be removed from the machine before allocation or deallocation. These profiles are removed before the pool/add-profiles are applied.

26.29.3.12. pool/wait-timeout

This integer parameter defines the number of seconds the allocation operation should wait for machines to get to the stage specified by pool/wait-for-stage. The default is 7200 seconds.

26.29.3.13. pool/add-parameters

This parameter is used on the poolAllocateMachines, poolReturnMachines, and returnToPool actions to add parameters to the machine as part of the action. The parameter is a map of undefined types. They should match the types of the parameters contained within.

26.29.3.14. pool/machine-list

This parameter defines a list of machines that the operation should work on. This used for the poolAddMachines, poolRemoveMachines, and poolReturnMachines actions.

The list is an array of either machine UUIDs or Key:Value syntax, e.g. “Name:fred.domain.local”

26.29.3.15. pool/new-workflow

This string parameter defines the workflow the machine should be placed in upon allocation from the pool or deallocation back to the pool.

26.29.3.16. pool/remove-parameters

This parameter defines a set of parameter to remove the machine before allocation or deallocation. This parameter is a list of parameter names to remove. These are removed before the pool/add-parameters are applied.