23.23. flexiflow - FlexiFlow¶
The following documentation is for FlexiFlow (flexiflow) content package at version v4.6.0-beta01.97+g1e33864277dc9bbb839f71f1eff994a0c4f05c23.
The FlexiFlow system allows an operator to dynamically inject new tasks in to an existing workflow based on a Parameter. This allows for other content to build up Tasks to add an existing workflow flexibly. Some examples may include adding specific tasks based on classification results from the Classify system, or other paths.
Stage Based Manipulation:
The Tasks that are defined in the parameter (specified by the
flexiflow/list-parameter) will be composed and added to the task list.
These will be dynamically inserted between the
Composed means the value of that parameter is the aggregation of all occurances of that parameter at all levels of the system, (e.g. From the parameters on the machine, then profiles on machines, then parameters on the stage, …). This allows multiple resources to provide the FlexiFlow tasks that will eventually be run on the system. Typically, Parameter orders of precedence would override earlier occurrences of the Parameter values. FlexiFlow does not follow that standard path.
The Param that is referenced by
flexiflow/list-parameter should be a
simple array type Param. It lists each Task that should be executed in
the listed sequence by the FlexiFlow process at that Stage in the
Workflow Based Manipulation
The Workflow that is defined in the Param named
be run when the stage of the same name is run. This allows the operator
to jump in to another Workflow in an existing workflow.
23.23.1. Operating FlexiFlow Stage Based Manipulation¶
Here is a brief outline of example usage of the FlexiFlow system utilizing
the RackN defined stage. In this example, we will define tasks to run in
126.96.36.199. Using the Stage and Task Injection¶
In this example we will create a “my-flexiflow-discovery” workflow.
Meta: color: grey icon: magic Name: my-flexiflow-discovery Description: My flexiFlow Discovery Workflow Stages: - discover - ipmi-inventory - raid-inventory - network-lldp - inventory - classify - flexiflow-stage - sledgehammer-wait
Note that the
flexiflow-stage is run after the
classify stage. A
FlexiFlow stage can be added anywhere that conceptually makes sense.
188.8.131.52. Using the FlexiFlow Stage¶
The FlexiFlow system is comprised of a single stage named
The stage must contain ONLY the
flexiflow-start stage will dynamically inject the desired tasks
in to the Workflow for the operator. DO NOT add any tasks between these
start and stop tasks.
You can choose to use the provided
flexiflow-stage, and customize
it’s use by adding Params to the Machine that define the Tasks that will run,
or you can create a custom Stage. The provided FlexiFlow stage is used the
same way that a custom named stage would be used.
To define the custom list of tasks to execute, simply follow the below steps:
- Add the
flexiflow/list-parameterparam to the Machine
- The value of which is another Parameter that is an Array of Strings that lists tasks to add
- An example is
superflex(which would be the Array of Strings)
- Insure you create a Param with Array of Strings as the type definition (eg
superflex, defined as Array of Strings)
- Add the Param (eg
superflex) to the Machine with a list of tasks in the array
If you create a custom stage, you can optionally embed the Params that control defining the list of Tasks to insert and/or the actual task list to insert.
Here is an example Stage in YAML that defines which specific Param will contain
the list of tasks to be executed. Note that we are not defining the actual
list of task in this example. Presumably a previous Stage (possibly Classify)
would build up the value of
superflex tasks to execute dynamically.
--- Name: "my-superflex-stage" Description: "Perform tasks defined by the 'superflex' Param." Documentation: | The Param 'superflex' is an array that will contain the list of flexiflow tasks to run. Params: flexiflow/list-parameter: "superflex" Tasks: - "flexiflow-start" - "flexiflow-stop" Meta: color: "orange" icon: "magic" title: "RackN Content"
Note that as the
Documentation field says; the
for this Stage is defined as
superflex. This Parameter must be defined
as an array; which is a list of the Tasks to execute during this Stage. The
Param can be defined on the Machine in all of the normal ways (directly as a
Param, as part of a Profile, or as a Param in the Global Profile, dynamically by
other content, etc).
Here is an example stage that defines the custom FlexiFlow stage, which sets
the control Parameter name to
superflex, and also defines the Tasks that
superflex will reference and run.
--- Name: "my-superflex-stage-v2" Description: "Perform tasks to run that are defined by the 'superflex' Param." Documentation: | The Param 'superflex' is an array that will contain the list of additional tasks to run. The actual tasks that are run are also listed in this Stage. Params: flexiflow/list-parameter: superflex superflex: - task1 - task2 - task3 Tasks: - "flexiflow-start" - "flexiflow-stop" Meta: color: "orange" icon: "magic" title: "RackN Content"
- ..note:: Warning!! The Params that are used in FlexiFlow MUST BE TYPE DEFINED correctly
- and added to the system before they are used. This use case pattern does NOT support using adhoc Params.
184.108.40.206. Create Parameter Definition¶
It is critical that you correctly type define the Parameter that defines the list of tasks to run in advance of using the Parameter. If you do not, you will receive a warning message and the task injection will fail.
An example type definition for the Parameter (using our
above), would look like:
--- Name: "superflex" Description: "Defines the flexiflow list parameter." Documentation: | This param defines the list of Tasks to execute for flexiflow during the stage run. If left empty, no tasks will be run. Schema: type: "array" items: type: "string" default:  Meta: color: "blue" icon: "magic" title: "RackN Content"
220.127.116.11. Possible Errors¶
Here is a list of some of the possible errors you may encounter.
Failed to render actions: : template: :35:30: executing "flexiflow_start.sh" at <.ComposeParam>: error calling ComposeParam: Cannot compose freeform param superflex Updated job for flexiflow-stage:flexiflow:flexiflow-start to incomplete
In this case, the
superflex list parameter must be defined in advance of using it,
the operator has attempted to use the Parameter as an adhoc param. Create a Param
type definition as specified in the Create Parameter Definition section above.
23.23.2. Operating FlexiFlow Workflow Based Manipulation¶
The FlexiFlow content also allows you to add a Stage to a given workflow. At that Stage,
the system will reference the Param named
flexiflow/workflow. The value of the
param should be the name of an existing Workflow. That workflow will then be executed
on the system.
Please review the
test-flexiflow-workflow-main Workflow which will dynamically add the
test workflow named
test-flexiflow-workflow-include. This provides a working example
of using the include workflow capability.
23.23.3. Object Specific Documentation¶
The content package provides the following params.
This defines the parameter that the
will use to reference the list of Tasks to dynamically execute. It
essentially operates as a “control pointer” to the real list, allowing
for flexible usage of a single defined Param to point to a variable list
The Param that it points to must be type defined in the system, and it must be an Array of Strings.
Indicate that we should go back to local disk.
Param for testing flexiflow system. This Param contains an Array of
Strings that will define which tasks to dynamically add to the
The content package provides the following stages.
Just provides a marker (decorator) that the flexiflow main stage has started.
Just provides a marker (decorator) that the flexiflow main stage has stopped.
flexiflow/workflow Parameter to
which should then be dynamically included in the
This Stage implements the FlexiFlow capability. When the task
is run, it will reference the paramtger defined in the
Param. The Param that is referenced must be defined as an Array of Strings. Each
array element must be an existing Task on the system. The list of tasks will
be dynamically injected in to this stage and run.
Do not insert any tasks between the start and stop tasks.
This task requires that the Param
flexiflow/list-parameter exists on the
machine to run successfully.
Allows the operator to set the
flexiflow/workflow Parameter to the
name of an existing workflow. This will then be added dynamically to
the system to be executed immediately at this position in the original
This Stage tests the FlexiFlow capability. It defines that the
flexiflow/list-parameter is set to
subsequently sets that Paramter to use the test tasks named
The content package provides the following tasks.
Just markes the beginning of the test flexiflow workflow.
Just markes the ending of the test flexiflow workflow.
This is an example test task that just prints a message and exits successfully.
This is an example test task that just prints a message and exits successfully.
This task begins the FlexiFlow stage. No other tasks should preceed it on
the stage, and the only other task that should follow is the
Tasks will be dynamically injected in to the task list after this task, if they have been specified by the appropriate control Params.
This task ends the FlexiFLow stage. No other tasks should follow it on
the stage, and the only other task that should preceed is the
Tasks will be dynamically injected in to the workflow if they have been specified by the appropriate control Params prior to this task.