Skip to main content
Feedback

Adding variables

Variables in Data Integration enhances the creation of connections, automates procedures, and transfers values between Data Flows.

Data Integration denotes variables with curly brackets { }. You can use this format to save data as a variable or to write or reference a variable throughout the Data Integration platform.

note
  • Variables can accommodate up to 20,000 characters.
  • Multi-value variables can accommodate as many as 10,000 values.
  • Variable names can contain only letters, digits, or underscores; do not use other characters.
  • Data Integration truncates values that exceed these limitations.

Variable types:

  • Data Flow Variables
  • Environment Variables

Data Flow variables

You use Data Flow Variables only within a specific Data Flow. You can define multiple variables and use them in both Logic and Action Data Flows.

Procedure

  1. Navigate to Data Integration Console.
  2. Create a new Logic Flow or Action Flow.
  3. Click the Variables tab in the top right-hand corner.
  4. Click +Add Variable to add a new Data Flow variable.
  5. To create an Array, use square brackets for the variable value, and then select the Contains Multiple Values checkbox.
  6. Click Apply Changes.
note

If you select the Clear Value On Start checkbox, the variable value updates to its original value when you create a new Data Flow, regardless of any changes during the Data Flow.

Encrypted variables

You can encrypt sensitive values stored in Data Flow variables to use in Python steps. Select the Encrypt option to encrypt and hide the variable value. Data Integration cannot decrypt encrypted variables, and you cannot save new values to them using inline Python code.

note

To create the same variable with the same name, delete the encrypted variable and create a new one.

After saving the Data Flow, you cannot view the value entered in the encrypted variable. Use the un-hide icon to confirm the entered value before saving the Data Flow.

note
  • You can use Encrypted Variables only within a Logic Data Flow, and only in Python logic steps.
  • To create an Array, use square brackets and save it as the variable value, then select the Contains Multiple Values checkbox.
  • Data Integration does not support Clear Value On Start in Encrypted variables.

Environment variables

You can use Environment variables throughout the platform and configure them by clicking the Variables tab from the Data Integration menu. These variables apply in Data Flows only when you define them as Data Flow variables.

note
  • Environment variable names can contain only letters, digits, or underscores** and must start with a letter; do not use other characters.
  • You can modify values for environment variables in the Variables tab of the main menu, not within the Data Flow.

Procedure

  1. Navigate to the Data Integration Console.

  2. Click the Variables tab from the menu.

  3. Click + Add Variable in the top right-hand corner.

  4. Add a value to the variable.

note
  • To create an Array, use square brackets to save it as the variable value.
  • Use a comma (,) to separate multiple email addresses or Slack entries.

  1. After creating an environment variable, go to your Data Flow and select the Variables tab in the right-hand corner.

  2. Set up a new Data Flow variable and enter the Environment variable name inside curly brackets to use the value (Test_Data Flow_Variable's value is 10).

  3. Click Apply Changes.

For more information on the techniques of using Variables, refer to Using Source to Target Variables in Logic Data Flow and Using Variables in REST Action Data Flows topics.

Variable naming limitation

Data Integration validates variable names and prevents the use of reserved names.

Reserved variable names (cannot use):

  • run_id
  • env_id
  • account_id

Variable character limitations

Variable LimitationSupportedDetails
Character limit for VariablesVariables can accommodate up to 20,000 characters. The system truncates any values exceeding this limit.
Variables containing multiple values for specific platformsYesSupported: Yes (Google AdWords, BingAds, Facebook Ads - Accounts List Only). Variables can contain multiple values, but this feature applies only to specific platforms such as Google AdWords, Bing Ads, and Facebook Ads, specifically for managing account lists. Multi-value variables can accommodate up to 10,000 values. The system truncates values beyond this limit.
Using environment Variables in custom query extraction settings for database migrationYesUse Variables in the Custom Query extraction settings for database migration purposes.
Using environment Variables in bucket source incremental extraction templateYesYou can use environment variables within the Bucket Source incremental extraction template for data extraction processes.
Passing environment variables into Action flowNoData Integration does not support passing Environment variables directly into the Action flow.
Parsing action output into Logic Variables from REST API JSONNoYou cannot parse the output of an action directly into a logic variable from a REST API response in JSON format.
Using environment Variables in REST Action API URLNoData Integration does not support using Environment variables within the API URL for REST actions.
Source-to-Target Flow Source and Target connectionNoData Integration does not support using variables in the connections for Source to Target Flow.
Using Variables in REST API request bodyNoCurrently, you cannot use variables directly in the body of a REST API request.
Using Variables in table filter settingsNoThe filter within Table Settings does not support using variables and does not accept SQL queries.
On this Page