New Source to Target experience
The Source to Target Data Flow in Data Integration lets you extract data from a source system and load it into a cloud data warehouse or storage target. Data Integration automatically detects the incoming data structure and generates the corresponding target tables and columns, no manual schema setup required.
How it works
Every Source to Target Data Flow follows the same four-step structure, regardless of the source type:
- Set up the source: select the connector and configure source-specific settings.
- Select a target: choose the destination warehouse or storage and set the loading mode.
- Configure the schema: review column mapping, set data types and modes.
- Schedule and run: run immediately or set a recurring schedule.
The target configuration, loading modes, and schema settings are consistent across all source types. Only the source setup steps differ.
Supported sources
Data Integration supports five source types in a Source to Target Data Flow:
| Source type | Connect to |
|---|---|
| Application | SaaS business applications, such as Salesforce, HubSpot, Google Ads, Facebook Ads, Shopify, and 180+ others |
| Database | Relational databases, such as MySQL, PostgreSQL, Oracle, SQL Server, BigQuery, Snowflake, Redshift, MongoDB, and more |
| Storage and files | Cloud and on-premise file storage, such as Amazon S3, Azure Blob Storage, Google Cloud Storage, SFTP, and more |
| REST API | Any external REST API endpoint with supported authentication (API key, OAuth, Basic Auth) |
| Events | Webhook-based event push, any system that can send HTTP POST requests with a JSON body |
Refer to Selecting the source type to identify which one applies to your data origin and follow the setup guide.
Supported targets
After extracting data from your source, you can load it into the following target systems:
| Target | Notes |
|---|---|
| Amazon Redshift | Supports clustering |
| Amazon S3 | File storage |
| Amazon Athena | Query-based storage |
| Azure Blob Storage | File storage |
| Azure Synapse Analytics | Enterprise data warehousing and big data analytics |
| Databricks SQL | Lakehouse architecture built on Delta Lake |
| Google BigQuery | Supports clustering and repeated fields |
| Google Cloud Storage | File storage |
| PostgreSQL RDS/Aurora | Relational database management storage |
| Snowflake | Supports clustering and Upsert-Merge |
| Firebolt | High-performance, cloud data warehousing |
| Treasure Data | Customer Data Platform (CDP) and data management |
Key capabilities across targets:
- Loading modes: Overwrite, Append only, or Upsert-Merge.
- Schema mapping: Auto-detect source schema or manually configure columns.
- Cluster keys: Supported in Snowflake, BigQuery, and Redshift.
- Expressions: Apply SQL-based expressions during load.
Target capabilities such as Upsert-Merge, clustering, and repeated fields vary by platform.
Before you begin
Before creating a Source to Target Data Flow, ensure you have the following:
- An active Data Integration account.
- Valid credentials for the source system you want to connect to.
- Access to a supported cloud data warehouse or storage target.
Next steps
Choose your source type and follow the setup instructions:
