Azure Blob Storage walkthrough
Azure Blob storage is Microsoft's cloud object storage solution. Blob storage is designed to accommodate large amounts of unstructured data. Unstructured data is data that does not adhere to a particular data model or definition, such as text or binary data.
Source details
Connection
To connect Azure Blob storage with your destination, refer to the Azure Blob storage connection topic. Choose a Source Connection after creating one.
Container name
After creating a Container, click the curved arrow next to Container Name on the right side of the row to select a name from the drop-down list.
Extraction modes
In Azure Blob storage, there are three types of extraction modes:
- All
- Incremental load by file modified timestamp.
- Incremental run: by template.
All
Select All to retrieve all data, regardless of time period.
Incremental load by file modified timestamp
Incremental Load by File Modified Timestamp allows you to control the date range of your data:
- Start Date is mandatory.
- Data can be retrieved for the date range specified between the start and end dates.
- If you leave the end date blank, the data is pulled at the current time of the river's run.
- Date timezone: UTC.
Incremental run: by template
Templates let you run over folders and load files in the order in which they were created. Choose a template type (Timestamp or Epoc time) and a data date range:
- Start Value is mandatory.
- Data can be retrieved for the date range specified between the start and end values.
- If you leave the end date blank, the data is pulled at the current time of the river's run.
- Date timezone: UTC.
- Splitting the requests into smaller chunks can improve the performance of the connection and reduce the time it takes to pull the data.
- You need to pull the date column to figure out when each record in the results was created.
