Skip to main content
Feedback

Working with environments

An Environment serves as a self-contained workspace for authorized users, containing Data Flows, Connections, Variables, and Data Flow groups managed by the Account Administrator.

Use Environments to migrate content between workspaces. Each environment remains fully isolated and has its own set of users, letting users to be part of multiple Environments.

Environment supports multiple teams to use the Data Integration platform simultaneously without accessing unauthorized projects. Administrator can easily update Environments and user permissions as needed, as statuses change.

note

Only Administrator can manage Environments. They have permission to create, edit, deploy packages, and undo changes. For more information, refer to the user roles and capabilities topic.

Variables

The platform uses environment variables throughout, and each Environment maintains its own set of variables.

You can do this by:

  • Changing the Environment variable values.
  • Setting new variables directly in the display table.
  • Editing and deleting Variables as needed.

Settings

The Settings tab includes options to customize your work with Environments. You can apply your preferences and changes to all packages and the default settings for new ones.

A Package merges Data Integration objects from one Environment into another.

note

When deploying an Environment:

  • You can overwrite Data Flow variables, while Environment variables can maintain their original values.
  • The toggle in the Variable Deployment section of the Settings tab applies only to Environment variables.

Deployments

A Deployment Package is a grouping of Data Integration objects, such as Data Flows, Connections, Data Flow Groups, and Variables used to move configurations from a Source Environment to a Target Environment.

A deployment will deploy all the objects in the package from the designated Source Environment to the designated Target Environment.

note

Deployments migrate object configurations only. No datasets are transferred to the Target Environment.

Roles and permissions

Users with the Admin or Deployment Manager role can create Deployment Packages and manage deployments across Environments. For more information, refer to the user roles and capabilities topic.

Why use deployments with environment variables

Without environment variables, deploying a Data Flow that connects to an external data source, such as Databricks or Snowflake means the Connection in the Target Environment still points to the Source Environment's database. You need to manually update the host, credentials, or catalog every time you deploy.

Environment variables solve this. By storing connection details as variables and referencing them inside Connections, each Environment automatically resolves the correct value for its own data source. You deploy once, the right credentials are already in place.

Example scenario: A Data Flow in DEV connects to a Databricks workspace using {databricks_host} and {databricks_catalog} as variables. DEV resolves these to the DEV workspace values; PROD resolves them to the PROD workspace values. When the Data Flow is deployed from DEV to PROD, no Connection update is needed.

To set this up:

  1. In each Environment, go to the Variables tab and create variables with the same names but environment-specific values — for example, databricks_host with the DEV host in DEV and the PROD host in PROD.
  2. In the Connection editor, reference those variables using the {variable_name} syntax in the relevant fields (such as Host or Catalog).
  3. When deploying, keep Add Credentials to Connections set to OFF so the Target Environment's credentials are not overwritten.

Create a deployment package

Creating a deployment package is a three-step wizard: Set Up, Object Selection, and Settings.

Step 1: Set up

  1. Navigate to Environments > Deployments and click Add Package.
  2. Enter a name for the package.
  3. Set the Source Environment (where the objects currently exist) and the Target Environment (where they will be deployed).

Step 2: Object Selection

Select the Data Integration objects to include. Use the tabs to browse by type — Rivers, Blueprints, Connections, Dataframes, Variables, and Groups. Each tab shows a count of selected versus available objects. Use the Search, Data Flow Type, and Data Flow Group filters to narrow results.

Step 3: Settings

Configure settings for this package, then save and run the deployment.

note

Settings configured here apply to this package only. To update defaults for all new packages, go to Environment Settings.

Deployment settings

Notifications

Enable Send Notification on Deployment and enter an Email address to receive an email each time this package is deployed.

Data Flow Deployment

  • Deploy New Data Flows to a Target (default: OFF): Deploys only Data Flows that don't already exist in the Target Environment.
  • Add Related Data Flows (default: ON): Includes all Data Flows linked or related to the ones you selected.
  • Add Related Connection (default: ON): Includes all Connections associated with the selected Data Flows.
  • Add Related Groups (default: ON): Includes all Groups related to the selected Data Flows.
  • Overwrite Dynamic Parameters (default: OFF): Overwrites all previous Dynamic Parameters in the Target Environment.

Connection Deployment

  • Deploy New Connections to a Target (default: ON): Deploys only Connections that are new to the Target Environment, avoiding existing Connections.
  • Add Credentials to Connections (default: OFF): Includes Connection registration credentials (login details, tokens, passwords) in the deployment.
  • Add Related File Zone Connections From Connections (default: ON): Deploys all custom File Zone Connections for the selected Connections.

Group Deployment

  • Add Related Data Flows (default: OFF): Deploys all Data Flows linked or related to the Groups you selected.

Environment Variables Deployment

  • Deploy New Environment Variables to a Target (default: OFF): Deploys only Variables that are new to the Target Environment, avoiding existing Variables.
  • Deploy Environment Variables With Their Values (default: ON): Deploys the original values for each deployed variable.

DataFrame Deployment

  • Deploy New DataFrames to a Target (default: OFF): Deploys only DataFrames that are new to the Target Environment, avoiding existing DataFrames.

Initial connection setup across environments

When setting up Environments for the first time, ensure that Connections in each Environment point to the correct resources with the correct credentials before deploying Data Flows. The steps below prevent credentials from overwritten in future deployments.

Step 1: First deployment: deploy the Connection without credentials

Deploy the Connection from the Source Environment with Add Credentials to Connections set to OFF. This creates a blank Connection in the Target Environment without writing any credentials.

Step 2: Enter environment-specific credentials

Navigate to the Target Environment and manually enter the environment-specific credentials into the blank Connection. Repeat for each Target Environment.

Step 3: Future deployments: use the recommended settings

For all subsequent deployments, use these settings to preserve environment-specific credentials:

  • Add Related Connection (ON): Keeps deployed Data Flows linked to their Connections in the Target Environment.
  • Deploy New Connections to a Target (ON): Only new Connections are deployed; existing Connections are not overwritten.
  • Add Credentials to Connections (ON): Includes credentials for Data Flows whose Source Connections share the same credentials across environments.
On this Page