Skip to main content
Feedback

Snowflake as a source walkthrough

You can start establishing a new Data Flow within Data Integration, proceeding to the selection of extraction modes and the administration of your data storage.

You can effectively harness Data Integration capabilities to extract data from Snowflake and seamlessly merge it into your data ecosystem.

Prerequisite

Ensure to create the right connection for Snowflake Source within Data Integration.

Data Flow modes

When using Snowflake as a Source, you can select between two Data Flow modes:

  • Multi-Tables (Standard Extraction)
  • Custom query.

Multi-tables (Standard Extraction)

This mode in Data Integration combines data from different tables into a single schema before transferring it to the destination. It establishes table relationships to ensure consistent linking and loading. Data Integration Multi-Tables Data Flow mode mainly employs SQL queries for transformations, with scheduling or manual triggering options.

For further details about using Multi-Tables (Standard Extraction), refer to the Databases Data Flow Modes topic.

Configuring a custom incremental load

You can run incremental loads on any column with incremental values in Snowflake. This feature is available only in multi-table mode.

Procedure

  1. Navigate to your table settings in the Schema tab.
  2. In the Incremental section, enter the column name you want to use.
  3. Select one of the following incremental types:
    • Running Number
    • Timestamp
    • Epoch
  4. Set your start and end values.
  5. Click Save and run your Data Flow.

For more information, refer to the Multi-Tables Standard Extraction.

Custom query

Data Integration Custom Query Data Flow mode empowers you to input data into the platform via personalized SQL queries, offering utmost control over data loading and transformations. You can specify data and transformations precisely, using SQL, pulling from databases or data warehouses like Snowflake. Data is then scheduled for automatic or on-demand loading into Data Integration, ensuring real-time data access.

note

When opting for the Incremental Extract mode, you have only two options: Datetime and Running number.

For further details about using Custom Query, refer to the Databases Data Flow Modes topic.

On this Page