Data Integration metadata

note

Currently only available in Source to target Rivers.

Data Integration supports custom metadata that describes the data origin. Data Integration metadata currently includes:

Name	Type	Description	Default
_rivery_last_update	Datetime	The time the columns was last updated or inserted (Note: At time of the insert to DB statement)	Yes
_rivery_run_id	String	The run ID that updated the columns or inserted it	Yes
_rivery_river_id	String	The River ID that the column originated from	Yes
_rivery_river_name	String	The River name that the column originated from (Note: The name at the time of the run)	No
_rivery_env_id	String	The River environment ID that the column originated from	No
_rivery_env_name	String	The River environment name that the column originated from	No

note

Data Integration metadata cannot include the following characters: ' ` " / If the variable value contains one of those, please replace it.

How to use Data Integration metadata

Source to target Rivers include the following metadata columns in each output table by default:

_rivery_last_update _rivery_run_id _rivery_river_id

note

• Currently available for Predefined Reports supported API sources and Multi-Table supported Database sources.
• Any database could be the target.
• Logic Rivers currently do not support this feature.

To enable this feature, navigate to the target tab and set the "Add Data Integration Metadata" toggle to true (under Additional Options).

Metadata syntax

In each of the corresponding databases, the following statement will be executed for the '_data_integration_last_update' expression:

Name	Statement	Type
Google Big Query	CURRENT_TIMESTAMP()	TIMESTAMP
Amazon Redshift	GETDATE()	TIMESTAMP
Snowflake	current_timestamp()	TIMESTAMP
Azure Synapse	CURRENT_TIMESTAMP	TIMESTAMP
Databricks	current_timestamp()	TIMESTAMP
Postgres	now()	TIMESTAMP

In custom expressions

Data Integration Metadata can also be used in custom expressions for target mapping. When creating a river, simply go to the schema tab and enter any of the supported expressions, followed by any valid name.

As a result, the following metadata will be added to the target table: