v4.8 [December 2021]

Release Date: December 7, 2021

Release Themes: Usability, Resource Management Abstractions, UX View Panels, Cloud Usecases

In addition to bug fixes and performance improvements, the release includes several customer-driven features.

Important Notices

New for this release:

• Workflows relating to Clusters and Cloud_Wrappers have been significantly changed. Operators using those patterns should carefully review their implementation before upgrading.
• Cleanup operation offers safer option than destroy by enabling pre-destroy actions by defining [on-delete-workflow]{.title-ref}. Operators should migrate destroy to cleanup where possible.
• Portal UX editing panels on several screens have been signficantly changed. UXViews ability to limit machine field view/edit is no longer enabled in v4.8.
• Tech Preview status for new Resource Abstraction Models (work order, resource broker and cluster) reflects that these new features will likely undergo additional change before the next release cycle and should not be used for production systems.
• Context Containers need to be rebuilt for v4.8+ Docker Context so as to NOT inject a DRPCLI copy. To ensure that the DRPCLI version matches the DRP endpoint, the Docker Context will now automatically attach the correct DRPCLI to containers during the starting process. See rs_release_v48_contexts.

From prior releases:

• v4.7 added port 8090 to the list of ports _required for provisioning operations. Please verify that port 8090 (default, this can be changed) is accessible for Digital Rebar endpoints.
• v4.7 changed the install zip format, the API-based upgrade of DRP to v4.7+ requires usage of most recent https://portal.RackN.io (v4.7 for self-hosted UX users) or the use of DRPCLI v4.6.7+. The v4.7 install.sh upgrade process also includes theses changes.
• v4.6 changed mananging of signed certificates, the process for updating and importing certificates has changed to require using the DRPCLI or API to update certificates. See rs_cert_ops for details.

Vulnerabilities

The following vulnerabilities were reported

• _rs_cve_2022_46382
• _rs_cve_2022_46383

Deprecations

• pre-Universal Bootstrap (bootstrap-base, bootstrap-advanced, bootstrap-contexts) workflows and related stages will be removed in v4.9. This functionality has been migrated to bootstrap-universal.

Removals

• The capability for Machine Panel field restriction for UX Views has been removed in v4.8. If you are using this capability, please let RackN know.
• The single machine behavior of Cloud_Wrapper has been removed. The v4.8 Cloud_Wrapper works strictly with the Resource Broker model.
• The pre v4.6 cluster patterns have been removed. The v4.8 Cluster utilities work strictly with the Cluster model.
• All pre-v4.8 cluster tasks and support have been removed and will not be supported. Migrated to Clusters abstraction.
• All pre-v4.8 cloud-wrapper behaviors have been removed and will not be supported. Migrated to Resource Brokers abstraction.

Technology Preview

This release contains significant Resource Management Abstraction enhancments that extend Digital Rebar from primarily managing single machine instances to coordinating groups of machines (cluster), managing platform lifecycle (cluster), and providing a portal way to interact with cloud and physical systems (resource broker). To allow these new abstraction to work in both lifecycle (aka workflow) and service modes, we\'ve introduced a new job scheduling concept called Work Orders that can be applied to all three instance types: machine, cluster and resource broker.

With these new models available in v4.8, RackN will be building reference applications that leverage the new concepts as part of the v4.9 release. The Tech Preview status for these new models will be removed for v4.9.

USAGE NOTE: To get the best experience with these evolving resource abstractions, please use the RackN UX hosted from https://tip.rackn.io. This version of the UX will be frequently updated during the post v4.8 development cycle to to showcase the new abstractions.

For more details, see

• rs_release_v48_resource_brokers
• rs_release_v48_workorders
• rs_release_v48_terraform

SAML v2 authentication providers enabled

Digital Rebar endpoint can register external identify providers (this is a new configurable model). During login by the UX, users with external providers will be presented with them as additional options for authentication instead of using the internal Digital Rebar user authentication. The authentication process with the SAML provider creates a token that is then used for Digital Rebar access via the UX. The token may also be retrieved from the UX and used with DRPCLI requests.

This provides a practical approach for operators looking to add two factor authentication (TFA) to their implementation.

For details see: rs_saml_ops

External Secrets

Extend the Digital Rebar parameters to dynamically retrieve information from external sources. Params identify a location external to Digital Rebar for parameter information instead of the Digital Rebar server\'s interal encrypted secret storage. This allows operators to leverage centralized secret vaults for critical information and reduces the risk associated with duplicating and distributing sensitive information.

To use external secrets, operators need to install the Vault plugin provider. This allows them to define one or more external sources for Param data. The value of the existing Param is then updated to use a [LookupUri]{.title-ref} instead of storing the information directly. To maintain performance, Digital Rebar caches the secret in memory after retrieval. The cache duration is configurable per Param.

UX refactoring for View Panels

The way the UX handles form input has been signficantly refactored. The new forms are editing in place with changes being sumbitted as single field PATCH requests (the old form used PATCH in bulk). This change eliminates the need to \"edit\" then \"save\" the form during normal operation. Errors generated by the API are immediately returned to the user at the field being edited.

The new forms also feature additional decomposition of values into tabs to reduce information overload on the previous single page forms.

NOTE: We have regressed the UX Views ability to high individual fields on the machines panel. We are evaluating if this feature is still needed.

Context Images and Bootstrap Updated

Context images have been updated to add newer tool versions, provide simpler image location awareness and remove the need to add drp tools and entrypoint to a container. This allows using standard containers to create contexts without modification.

Universal has an updated bootstrap-context task and included in the universal-bootstrap workflow. It is important to run the bootstrap-contexts task found in task-library after updating from earlier versions of DRP. The simplest way to do this is to run the universal-bootstrap workflow on the endpoint.

Please see rs_contexts for details.

Cluster and Resource Broker Models (Tech Preview)

Resource Abstractions allow Digital Rebar operators to manage infrastructure a higher level of control such as a cluster or cloud platform level. That allows operators to create IaC automation that functions at a much higher level in the management topology such as self-building vCenter and Kubernetes clusters. It also enables portable multi-cloud pipelines.

New models for Cluster and Resource Broker were added in this release to improve coordination of multi-machine operation. These new models are functionally specialized machine types that run in context containers managed by Digital Rebar. While they are similar to machine objects in many ways, they have distinct APIs and behaviors. The most notable change is that both types have a _required tracking profile whose name matchines the object name.

Clusters are used to coordinate operations over groups of machines. All machines in a cluster will reference the cluster\'s tracking profile using the [cluster/tags]{.title-ref} param instead of the legacy [cluster/profile]{.title-ref} params. Since clusters have their own workflow and lifecycle; they are able act outside of their machines\' lifecycle. This includes building, resizing or destroy clusters, managing platforms from their platform API, and other non-machine actions.

Resource Brokers provide a critical abstraction for handling machine lifecycles in a general purpose away. Automation, such as a Cluster workflow, can make a non-differentiated request to a Resource Broker (generally via a work order) to allocate or release machines. Since the implementation is left to the broker, Digital Rebar operators are able to manage the details of how and where the resources are provided. For example, brokers can be used to abstract different clouds or cloud regions. They can also be used to front pools of physical machines to allow cloud-like allocation for bare metal infrastructure.

Work Orders (Tech Preview)

Work Orders allow Digital Rebar operators to schedule jobs for infrastructure without making lifecycle changes to the infrastructure. This allows operators to perform one off or scheduled tasks on sytems that do not change the systems configuraiton. For example, running Helm charts against a cluster, creating VLANs on a switch, requesting a storage LUN, or performing a routine compliance scan.

Digital Rebar v4.8 create a new operational mode for the Digital Rebar runner that allows tasks to be performed asynchronously outside of Workflows. This allows machines, clusters and resource brokers to be addressed as services asynchronously.

Work Orders have their own high level object contructs in Digital Rebar that allow them to be managed differently than Workflows. Since they are asynchronous, they can be scheduled and requested from multiple sources in parallel; however, they are logged as jobs. There remains only one type of job tracking and history.

Unlike Workflows, Work Orders can be performed in parallel by the DRP runner / agent; however, a runner can only be in either Workflow or Work Order mode at a time.

Terraform and Cloud-Wrapper Rewrite (Tech Preview)

Cloud Wrapper templates were updated to enable better integration into universal workflow and simplify importing existing Terraform plans.

1. to use cloud-init instead of relying on Ansible for join-up where possible.
2. to allow creating many machines from a single plan (uses Resource Brokers)
3. improve controls by assigning pipelines and workflows after Terraform creates instances
4. automatic synchronization when Terraform creates and destroys instances
5. improved debugging and troubleshooting

WARNING: These changes break functionality for previous users. Due to known limitations with the previous approach, RackN is not supporting the prior Cloud_Wrapper or Terraform implementations. Contact RackN for advice on migration.

Other Items of Note

• API [machines play]{.title-ref} command can be used to start or restart a Workflow. Restarts previously required clearing and setting the Machine.Workflow.
• Addition of \"cleanup\" as a safer alternative to \"destroy\" by allowing select objects to execute operations before being destroyed when the [on-delete-workflow]{.title-ref} is defined. Operators should migrate all destroy operations to use cleanup.
• Machines Start API streamlines restarting workflows by eliminating the need to clear and reset and run a Workflow. Operators are recommended to use this approach for starting Workflows in v4.8+. The UX will automatically detect and use Start if available.
• Scale enhancements (back ported to v4.7.11+) enable systems to maintain performance with more than >10,0000 _active machines.