Events

An Event Flow uses a Webhook source to receive data that an external system pushes in real time. Whenever a defined event occurs in the source system, it sends an HTTP POST request with a JSON payload to a Data Integration-generated endpoint URL. Data Integration receives the payload and loads it into your target immediately.

Use this Flow type when you need event-driven, real-time data ingestion. For example, capturing order events, user actions, or system alerts the moment they happen, rather than pulling data on a schedule.

Example: An e-commerce platform wants real-time order updates in its analytics warehouse. Whenever a new order is created, the platform triggers a webhook that sends a JSON payload, such as order ID, customer details, items, total, and timestamp which Data Integration loads directly into Snowflake.

Flow: External system (HTTP POST) > Webhook endpoint > Source to Target > Snowflake

note

An Event flow receives data that is pushed to it. Unlike other source flow types, there is no scheduled polling, the external system is responsible for triggering the webhook whenever an event occurs.

Before you begin

Before creating an Event flow, ensure you have the following:

• An active Data Integration account.
• Access to the external system that will send events, with permission to configure a webhook or HTTP POST callback URL in that system.
• The external system must support sending HTTP POST requests with a JSON body. Refer to Events in the Sources section for webhook setup details.
• Access to a supported cloud data warehouse or storage target.

Step 1: Create a Data Flow

1. Navigate to the Data Integration Console.
2. Click Data Flows in the left-hand menu.
3. Click Create Data Flow and select Source to Target Data Flow, or open an existing data flow.

Step 2: Set up the source

1. In the Source tab, select Events (Webhook) as the source type.
2. Click Enable to activate the webhook.

Data Integration generates a unique Webhook Endpoint URL. Copy this URL. You will register it in your external system as the destination for event payloads.

Webhook endpoint URL

The generated endpoint URL is unique to this data flow. Your external system sends HTTP POST requests to this URL each time a triggering event occurs. Data Integration parses the incoming JSON payload and loads the data into the configured target.

note

Keep the Webhook Endpoint URL confidential. Any system with access to this URL can send data to your flow. If the URL is exposed or compromised, disable the webhook and re-enable it to generate a new URL.

CORS configuration (optional)

If events are sent from a browser-based client. For example, a JavaScript application running in the user's browser, configure Cross-Origin Resource Sharing (CORS) to allow the browser to make cross-origin POST requests.

In the Access-Control-Allow-Origin field, enter the domain from which browser clients will send requests (for example, https://your-app.com). Leave this field empty if events are sent from a server-side system or backend service, where CORS does not apply.

Step 3: Select a target

In the Target tab, select the cloud destination where Data Integration loads the incoming event data.

Data Integration supports all major cloud data warehouses and cloud file storage systems as targets.

Select or create a connection for your target. Then define the Database, Schema, and Target Table where the event data will be stored. Data Integration automatically detects available databases and schemas.

Loading modes

Event flows most commonly use Append only mode, since each webhook payload represents a discrete event that should be preserved as a separate row. Choose the mode that fits your use case:

Mode	Behavior	When to use
Append only	Adds each incoming payload as a new row without modifying existing data.	Event logs, order events, user actions. Any scenario where every event must be retained. Recommended for most event flows.
Upsert-Merge	Inserts new records and updates existing ones based on a defined merge key.	When payloads represent state updates and you want the target to reflect the latest state per key. For example, order status changes keyed on order ID.
Overwrite	Replaces all existing data in the target table with the current run's data.	Rarely suitable for event flows.

If you select Upsert-Merge, define your merge key in the Schema tab (Step 4).

note

For file storage targets, specify the bucket and file path where Data Integration stores the data. The connection's default bucket is used if you do not specify one.

Step 4: Configure the schema

The Schema tab lets you review and adjust the target table structure before the flow processes live events. Data Integration auto-detects the schema from the structure of the incoming JSON payload.

Click Auto-Mapping to populate target field names and data types automatically.

note

Send a test event from your external system before configuring the schema. Data Integration uses a received payload sample to detect field names and data types. Without a sample payload, Auto-Mapping cannot populate the schema.

Column mapping

Field	Description
Target field name	Rename the column as it appears in the target table.
Data type	Change or cast the data type during loading.
Mode	Controls how the column handles values. Options: Nullable, Required, or Repeated.
Primary key	Mark one or more fields as the merge key for Upsert-Merge flows.
Cluster key	Define a partition or cluster field, if supported by the target warehouse.
Expression	Write SQL to create a new field or transform an existing one using target warehouse syntax.

Column modes

Mode	Description
Nullable	The column can contain null values. Not every row needs a value for this field. Use for optional event attributes.
Required	The column must always contain a value. Null values are not allowed. Use for event IDs, timestamps, and mandatory fields.
Repeated	The column stores multiple values per row as an array or list. Use for nested arrays in JSON payloads such as item lists or tag arrays.

Defining primary keys (upsert-merge only)

If you selected Upsert-Merge as the loading mode, click the key icon next to a field to mark it as the primary key. To create a composite key, click multiple fields. The combination of those field values acts as the unique identifier for each row.

note

• Data Integration does not include column descriptions in the target metadata. Document field descriptions separately if your data governance process requires them.
• Column names cannot contain dots (.). Dots cause errors during data processing. Rename any affected fields before the flow processes live events.

Step 5: Activate and monitor the Flow

Activate the webhook

An Event flow does not run on a polling schedule. Once saved and enabled, Data Integration listens continuously at the webhook endpoint URL. Each incoming HTTP POST payload triggers a load to the target automatically.

To start receiving live events:

1. Copy the Webhook Endpoint URL from the Source tab.
2. Register the URL in your external system as the webhook destination or callback URL.
3. Trigger a test event from the external system to confirm the connection is working.

You can also click Run at the bottom of the page to manually trigger the flow and verify the schema and target configuration before registering the webhook with your external system.

Set up notifications

In the Settings tab, enter your email address to receive alerts for load failures or warnings. Enable On Warning to also receive alerts when a payload is received but contains no data. To receive alerts in Slack, connect a dedicated Slack channel to Data Integration's email alert system.

Monitor incoming events

After the flow is active, open the Activities tab in the right-hand menu to review processing logs. Each webhook payload processed by Data Integration appears as a separate run entry showing the number of records loaded and any errors encountered.

Common issues to watch for:

Issue	Likely cause	Resolution
Schema mismatch	The external system sent a payload with new or renamed fields that do not map to the target schema.	Update the column mapping in the Schema tab to accommodate the new fields.
Load failure	A payload was received but failed to load into the target.	Review the error details in the Activities log. The payload is not re-sent automatically. The external system must resend the event.
Empty run	A run shows zero records loaded.	The payload was received but contained no data. Review the JSON structure sent by the external system.

note

A Data Flow supports up to 500 tables.

Related topics

• Selecting the source type — Compare all five source types to confirm Events is the right choice for your use case.
• Sub data flows — Embed sub-processes within a parent Event flow to handle multiple event schemas in one flow.