Flow Control step

The Flow Control step contains a set of options that allow you to configure how documents are processed within a process execution.

The Parallel Processing options allow part of a process execution to be executed in parallel. You enable parallel processing by setting the Number of Threads or Processes to a value greater than 1.

When documents get to the Flow Control step, they are separated into the number of threads defined, and each thread is executed in parallel from the current Flow Control step through to completion. Adding this step to a process can help speed up "slow" or "complex" steps and spread memory to multiple computing processes.

note

Some of these options require you to use a Runtime cluster to enhance the power of your document flow. Consult with a representative to learn more about how these options can be enabled in your account.

Parallel processing is disabled when you are using test mode. You must also ensure that parallel processing is enabled in your account.

Adding Flow Control step to process

1. Drag the Flow Control step to the process. The Flow Control step dialog opens.

2. On the General tab, in the Display Name field, enter the name to describe the step. If you do not enter the name, “Flow Control” appears on the step.

3. In the Parallel Processing Type field, select one of the parallel processing options:

   • Threads — If selected, two or more "processing" tasks run concurrently. Multiple threads can run within the same computing process. It is set by default.

     Note: Boomi recommends using threads for the parallel processing types to avoid a possible issue with document caches clearing properly when multiple JVMs are used.

   • Processes — If selected, instances of a Java Virtual Machine, consist of one or more threads, that are executed sequentially by a computer system that has the ability to run several JVMs concurrently (“forked execution”). This setting is more of a request than a hard requirement. If the Runtime on which the process is running has no way to execute in a separate process, and it falls back to separate threads.

   You must have Runtime clusters enabled in your account to select Processes under Parallel Processing Type.

4. In the Number of Threads field, enter the number of threads or processes that process the documents in parallel to completion.

   • If you enter the number less than or equal to 1 (or undefined), no parallel processing is performed.
   • If your process is running on a Runtime cloud, the maximum number of units that you can use is 10.

5. Select one of the Batch options:

   • No Document Batching - If selected, does not break document sets but adheres to the defined parallel processing options.
   • Run each document individually - If selected, executes each document to completion, one document at a time. This setting changes the document processing after this step. Each input document to the Flow Control step is executed to completion, one at a time (in all subsequent steps). Parallel processing and/or a Runtime cluster are not required to add this step to a process. This option slows down any process, including processes whose mode is set to General rather than to Low Latency. Use this option sparingly.
   • Run as batches of x documents - If selected, breaks the entire document set into batches based on the defined document count and executes each batch to completion, one batch at a time.

6. On the Notes tab,

   • Enter the title of the note in the Title field.
   • Enter the note in the Note field. 7 . Click OK

Flow Control step example:

The following examples illustrate the use of Flow Control step with Run each document individually option and Parallel Processing option.

Run each document individually option

Run each Document individually option

An example of a Flow Control step that uses the Run Each Document Individually option.

• Start step (Query) — Retrieves 9 documents

• Run Each Document Individually is selected:

  • Flow Control step — Passes through document 1

  • Map step — Maps document 1

  • Connector (Send) — Sends document 1

  • Flow Control step — Passes through document 2

  • Map step — Maps document 2

  • Connector (Send) — Sends document 2

  In the similar way, documents 3–9 are passed, mapped and sent individually through the process.

note

If the Run each document individually option and Parallel Processing option are both enabled in the same Flow Control step, the Run each document individually handling occurs within the context of parallel processing. The Run each document individually option slows down any process, including processes set to General mode rather than to Low Latency. Use this option sparingly.

Parallel Processing option

Parallel Processing option

An example of a Flow Control step that uses the Parallel Processing option.

• Start step (Query) — Retrieves 9 documents

• Parallel Processing Type is Threads and Number of Threads is 2:

  • Flow Control step

    • Creates group 1 with documents 1-5

    • Creates group 2 with documents 6-9

  • Group 1 (concurrently with Group 2)

    • Map step — Maps documents 1-5

    • Connector (Send) — Sends documents 1-5

  • Group 2 (concurrently with Group 1)

    • Map step — Maps documents 6-9

    • Connector (Send) — Sends documents 6-9

Flow Control step and fiber executions

Fibers are the string of steps in the process flow that follow a Flow Control step with Parallel Processing options defined. For example, setting Number of Threads to two, spawns two fiber executions.

When a process execution reaches a Flow Control step with Number of Threads or Processes set to a number greater than one, the current (controlling) process execution essentially pauses. The current collection of documents is divided according to the desired number of threads, and a "fiber" execution is started for each thread. The Flow Control step then waits for each of the fiber executions to complete and handles the results. The controlling process never executes the steps that follow the Flow Control step because they were all executed by the fiber executions.

Fiber executions are very similar to "inline" process executions (see Process Call), except that the "child" processes start mid-process — at the Flow Control step from which the fiber execution was launched.