Skip to main content
Feedback

Snowflake as a Source walkthrough

This document provides a comprehensive walkthrough in Snowflake as a Source, starting with establishing a new River within Data Integration, then proceeding to the selection of extraction modes and the administration of your data storage.
By the end of this guide, you will have a clear understanding of how to effectively harness Data Integration capabilities to extract data from Snowflake and seamlessly merge it into your data ecosystem.

Prerequisite

Please ensure the creation of a correct connection for Snowflake Source within Data Integration.

River modes

When using Snowflake as a Source, you have the choice to select between two River modes:

  • Multi-Tables (Standard Extraction)
  • Custom query.

Multi-tables (Standard Extraction)

This mode in Data Integration combines data from various tables into a single schema before transferring it to the destination. It establishes table relationships to ensure consistent linking and loading. Data Integration Multi-Tables River mode mainly employs SQL queries for transformations, with scheduling or manual triggering options.

To obtain further details about using Multi-Tables (Standard Extraction), kindly refer to our documentation on Databases River Modes.

Configuring a custom incremental load

You can run incremental loads on any column with incremental values in Snowflake. This feature is available only in multi-table mode.

Follow these steps to configure a custom incremental load:

  1. Navigate to your table settings in the Schema tab.
  2. In the Incremental Load section, enter the column name you want to use.
  3. Select one of the following incremental methods:
    • Running Number
    • Timestamp
    • Float
  4. Set your start and end values.
  5. Click Save and run your River.

For more information, refer to the Multi-Tables Standard Extraction.

Custom query

Data Integration Custom Query River mode empowers users to input data into the platform via personalized SQL queries, offering utmost control over data loading and transformations. Users can specify data and transformations precisely, using SQL, pulling from databases or data warehouses like Snowflake. Data is then scheduled for automatic or on-demand loading into Data Integration, ensuring real-time data access.

note

When opting for the Incremental Extract mode, it's crucial to recognize that you have a choice of only 2 options: Datetime and Running number.

To obtain further details about using Custom Query, kindly refer to our documentation on Databases River Modes.

On this Page