Skip to main content
Feedback

Google Cloud Storage (GCS) walkthrough

Prerequisites

Google Cloud Storage Connection.

Pulling data from GCS using Data Integration

  1. Select Create New River from the top right of the Data Integration screen.

  2. Choose Data Source to Target as your river type.

  3. In the General Info tab, name your river, describe it and choose a group. Next, navigate to the Source tab.

  4. Find Google Cloud Storage under Storage in the list of data sources and select it.

  5. Under Source Connection, select the connection you created, or create a new one.

  6. Select the desired Bucket Name from the list.

  7. Choose an Extract Method.

  8. Define the Prefix.

  9. Define the File Pattern.

  10. Define the File type.

  11. Define what to do After pulling - files will be:

  12. (Optional): Configure the Number of files to pull.

Extract methods

Run all:

  • Returns data from all times. Given the provided Bucket, Prefix, and File Pattern.

Incremental run: by modified timestamp

  • Pulls data in the date range between the start and end date provided, including the end date. Given the provided Bucket, Prefix, and File Pattern.
  • You must select a start date.
  • Leaving the end date empty will pull data according to the current time of the River's run.
  • Select a timezone offset to send dates considering the offset.
  • Use the Last Days Back For Each Run to expend the start date and pull data a given number of days back before the chosen start date.

Incremental run: by template

  • Templates give you the option to run over folders and load the files according to the folder order. You just need to choose your template type and write your template structure.
  • Timestamp Template - Use {} and proper timestamp part in order to define the folder format.
  • Epoc time template - Use {e} (for an epoc) or {ee} (for an epoc in milliseconds) in order to define the folder to be able to run on
  • epoc time.
  • Enter the desired Start Value (required) and End Value (optional) Note:
    • This method is valid for whole library and not available for files.
    • The Start Date won't be advanced if a River run is unsuccessful. If you don't want this default setting, click More Options and select the checkbox to advance the start date even if the River run is unsuccessful (Not recommended).

The time-dependent report include Campaigns, Email templates, and Events.

Results after the pull

image.png

Options:

  • Move to archive path
  • Remain in original place
  • Deleted
note

The action chosen will remain true for successful and failed rivers. For example, if something fails on the upload to the target we still move it to archive path from source.

On this Page