Skip to main content
Feedback

Configuring sub rivers

Sub Rivers let you embed smaller and specialized processes within the larger River. The sub-processes can inherit configurations and settings from the main River or use unique parameters. You can build flexible, modular data pipelines that efficiently handle complex workflows.

Consider an agency that uses Data Integration to manage Facebook Ads data for multiple customers. Instead of duplicating the entire River configuration for each customer, the agency creates a Sub River for each customer, letting them tailor the data pulled specifically for that customer while maintaining common definitions across all Sub Rivers, providing scalability and reducing the need for duplication.

Procedure

  1. Navigate to the Data Integration console.
  2. Click Rivers from the left-hand menu.
  3. Click ADD River and select Source to Target River or choose an existing River.
  4. Select the Custom Report option in the "Source" tab.
  5. Select a Target, navigate to the Settings tab, and turn on the "Sub Rivers". A Sub Rivers tab appears in the River editor when enabled.
  6. In the Sub Rivers tab, click New Sub River to create your first sub river - "Sub River 1". You can rename it as needed.

image.png

Supported Sources:
  • Shopify
  • Klaviyo
  • Stripe
  • Google Ads
  • Google Ad Manager
  • Google Analytics
  • YouTube Reporting
  • Bing Ads
  • Facebook Social
  • Facebook Ads
  • Jira
  • Twitter Ads
  • LinkedIn Social
  • LinkedIn Ads
  • Mavenlink
  • Snapchat Marketing
  • Taboola
  • Instagram Social
  • Active Campaigns
  • Hubspot
  • REST (via Actions)
  • TikTok
Supported Targets:
  • Snowflake
  • Google BigQuery
  • Azure Synapse
  • Amazon Redshift
  • Databricks SQL
  • Azure Blob Storage
  1. Selecting the "Source Connection" parameters adds them to the "Sub Rivers" list, letting each Sub river use a different connection from the main River.

Each Sub River can have a separate Data Integration source connection while pulling data.

image.png

  1. For each parameter within the Sub river, do one of the following:
    a. Use the value inherited from the main River.
    b. Enter a custom value unique to that Sub river.

  2. Each Sub River runs on its own schedule or inherits the main River’s schedule. Click the clock icon next to the parameter drop-down to configure Sub River-level scheduling.

Limitation

  • You cannot Deploy a River with Sub Rivers between Environments.
  • You cannot create versions for Sub Rivers.
  • Copying a River with Sub Rivers does not duplicate the Sub Rivers, but only the primary configuration.
On this Page