YAML Builder

The YAML Builder provides a low-code interface for creating Blueprint connectors. This tool lets you to visually configure a connector and automatically generate the necessary YAML, which eliminates manual coding.

A Blueprint connector defines the following:

• Authentication: Methods for connecting to an external API.
• User inputs: Parameters that you provide when you configure the connector.
• API operations: The sequence and logic of the API calls.
• Data extraction: Specific information to extract and save from API responses.

Blueprint flow

The following flow shows how the Blueprint components connect.

Variables and storage

To save data from an API response, you must configure the following variable:

• variables_metadata to declare the variable and define its storage location.
• variables_output to extract the data and identify what to store.

tip

Consider variables_metadata as creating a container, and variables_output as filling that container with data.

Prerequisites

Before building a connector, ensure you have the following:

• API documentation: Identify the endpoints, methods, and response formats for your specific requirements from the API documentation.
• Authentication credentials: Gather the API key, OAuth credentials, or username and password for the connection.
• Sample API response: Use a sample response to write JSON paths for data extraction.

Navigating to YAML builder

1. Log in to the Data Integration console.
2. From the left-hand menu, click Blueprints > + Add Blueprint.
3. Click Use our YAML Builder.

Expand and configure the required settings in the left pane:

Step 1: Connector configuration

The connector configuration defines the connection settings for the API. You must configure this section before adding individual steps.

Configuration settings

Setting	Purpose	Example
Connector Name	Identifies the connector.	Salesforce API.
Base URL	Root URL for all API calls.	https://api.example.com/v1.
Default Headers	Headers sent with every request.	Content-Type: application/json.
Variables Metadata	Declares the variables used to store output data.	Refer to Variables Metadata for more information.
Variables Storages	Defines the physical storage location for data.	file_system.

Configuring variables Metadata

Before a step can save data, you must declare the variable and its storage destination in the connector configuration.

connector:
  name: "My Connector"
  base_url: "https://api.example.com"
  
  # STEP 1: Declare the variable "bucket"
  variables_metadata:
    final_output_file:          # Variable name
      format: json              # Data format
      storage_name: results_dir # Which storage to use
  
  # STEP 2: Define the storage location
  variables_storages:
    - name: results_dir
      type: file_system

When a step extracts data and saves it to a variable such as final_output_file, the system references the variables_metadata to determine the following:

• The data format. For example, JSON.
• The storage destination. For example, results_dir mapping to file_system.

Interface parameters

Interface parameters are the inputs that you provide when you configure a connector. Use these parameters to collect credentials, filters, and other dynamic values.

Parameter types

The following table describes the available parameter types:

Type	Description	Use case
String	Simple text input.	Domain names or filter values.
Authentication	Credential collection.	API tokens, OAuth, or Basic authentication.
Date Range	Date picker with start and end values.	Time-bounded data fetching.
List	Drop-down selection.	Predefined options.

Authentication types

When you select the Authentication parameter type, you must select an authentication method that corresponds to the requirements of the external API:

Authentication type	Usage
Bearer	Used for token-based APIs.
Basic HTTP	Used for APIs requiring a username and password.
API Key	Used for static keys passed in a header or query string.
OAuth2	Used for modern APIs requiring token refresh capabilities.

Referencing parameters in steps

To use interface parameters in your connector logic, you must reference them using the {parameter_name} syntax. Do not use percent signs for interface parameter references.

The specific syntax used for a reference depends on the Parameter Type selected:

• Standard parameters — For types such as String, use the exact name defined in the interface parameters section. For example, {domain}.
• Date Range parameters — This type requires a specific property reference. You must specify either .start_date or .end_date to retrieve the correct value from the range. For example, {date_range.start_date}.

Use the following example to reference a standard string and a date range in a YAML configuration:

interface_parameters:
  section:
    source:
      - name: "domain"          # Parameter name
        type: "string"
      - name: "date_range"
        type: "date_range"
        format: "YYYY-mm-DD"

connector:
  base_url: "https://{domain}.api.example.com"  # Reference: {domain}

steps:
  - endpoint: "{{%BASE_URL%}}/data"
    query_params:
      from: "{date_range.start_date}"           # Reference: {param.start_date}
      to: "{date_range.end_date}"               # Reference: {param.end_date}

Workflow steps

A step defines the API calls to execute and their sequence. You can configure a step to perform the following actions:

• Make an HTTP request (REST step).
• Iterate through data (Loop step).
• Extract and save data (Variables output).

Step types

The following table describes the available step types:

• REST: Execute a single API call.
• Loop: Iterate over an array and execute nested steps for each item.

REST Step

Configuring a REST step

You can define the parameters for your REST step, such as the step name, HTTP method, and API endpoint. This example shows the YAML code generated when you configure a standard GET request:

steps:
  - name: "Get Users"
    description: "Fetch all users from API"
    type: rest
    http_method: GET
    endpoint: "{{%BASE_URL%}}/users"
    query_params:
      status: "active"
    headers:
      Accept: "application/json"

Configuring variable outputs

Use the variable outputs section to extract data from API responses. You must define the following fields for each extraction:

Field	Purpose
response_location	Where in response: data (body), header, or status_code.
variable_name	Which variable to store data in (must be declared in variables_metadata).
variable_format	Format: json or text.
transformation_layers	How to extract/transform the data.

The following example shows how to configure the extraction of user IDs from a JSON response:

variables_output:
  - response_location: data
    variable_name: user_ids              # Must match variables_metadata
    variable_format: json
    transformation_layers:
      - type: extract_json
        from_type: json
        json_path: "$.users[*].id"       # Extract just the IDs

Connecting steps with variables

Variables pass data between steps. The following is an example to extract data in an initial step and reference it in a subsequent loop step:

steps:
  # Step 1: Fetch list and extract IDs
  - name: "Get User IDs"
    type: rest
    http_method: GET
    endpoint: "{{%BASE_URL%}}/users"
    variables_output:
      - variable_name: user_ids          # Save IDs here
        transformation_layers:
          - json_path: "$.users[*].id"

  # Step 2: Loop through the IDs
  - name: "Get User Details"
    type: loop
    loop:
      variable_name: user_ids            # Read from Step 1's output
      item_name: user_id
    steps:
      - name: "Fetch Details"
        endpoint: "{{%BASE_URL%}}/users/{{%user_id%}}"  # Use current ID

Loop Step

Configuring Loop step

Use loops to iterate over arrays of data. You can configure loops for the following two scenarios.

Iterating over data from a previous step Configure this when an initial step retrieves a list and a subsequent step must process each item.

steps:
  # Step 1: Get the list
  - name: "Get Projects"
    type: rest
    endpoint: "{{%BASE_URL%}}/projects"
    variables_output:
      - variable_name: project_ids
        transformation_layers:
          - json_path: "$.projects[*].id"    # Extract IDs only

  # Step 2: Loop through each ID
  - name: "Get Project Details"
    type: loop
    loop:
      type: data
      variable_name: project_ids             # Array from Step 1
      item_name: project_id                  # Current item alias
      add_to_results: true
    steps:
      - name: "Fetch Details"
        type: rest
        endpoint: "{{%BASE_URL%}}/projects/{{%project_id%}}"
        variables_output:
          - variable_name: final_output_file
            overwrite_storage: false         # Append, don't replace

important

The item_name represents the entire current value. If you require properties like an ID, you must extract them in the transformation layer first.

Iterating over external data Configure this scenario when the connector receives data from a source River. This action must occur in the first step of your configuration.

steps:
  - name: "Process External IDs"
    type: loop
    loop:
      type: data
      variable_name: "{ext.source_ids}"      # External variable syntax
      item_name: item_id
    steps:
      - name: "Fetch Item"
        endpoint: "{{%BASE_URL%}}/items/{{%item_id%}}"

warning

The {ext.} prefix is required for external variables. You can only use this prefix in the first step.

Step 5. Configuring pagination

Configure pagination to automatically retrieve all pages when an API returns paginated data.

The following table describes the available pagination types:

Type	Description	Usage
Page	Increments the page number (1, 2, 3).	Use when the API uses ?page=N.
Offset	Skips records (0, 50, 100).	Use when the API uses ?offset=N&limit=M.
Cursor	Uses a token from the response.	Use when the API returns a next_cursor token.

The following is an example:

pagination:
  type: page
  location: qs                    # Query string
  parameters:
    - name: page
      value: 1                    # Start at page 1
      increment_by: 1             # Add 1 each request
    - name: per_page
      value: 100                  # Page size (static)
  break_conditions:
    - name: "No More Data"
      condition:
        type: empty_json_path
        key_json_path: "$.data"   # Stop when data is empty

Break conditions

You must add break conditions to prevent infinite loops. The following table describes common break conditions:

Condition	Description
empty_json_path	Stops when a specified JSON path returns empty or null.
page_size_break	Stops when the returned items are less than the requested page size.
boolean_break	Stops when a field evaluates to true or false. For example, has_more: false.
total_items_reached	Stops when the cumulative count equals the total items available.

Referencing variables

Use the following syntax to reference variables within your configuration:

Context	Syntax	Example
Base URL	{{%BASE_URL%}}	{{%BASE_URL%}}/users
Internal Variable	{{%variable_name%}}	{{%user_id%}}
Loop Item	{{%item_name%}}	{{%order_id%}}
Interface Parameter	{param_name}	{domain}
Date Range Start	{param.start_date}	{dates.start_date}
Date Range End	{param.end_date}	{dates.end_date}
External Variable	{ext.variable_name}	{ext.source_ids}

Example: Complete Blueprint configuration

The following example is a complete Blueprint that retrieves projects and their associated tasks.

Click to expand the complete YAML configuration

# 1. USER INPUTS - What the user provides
interface_parameters:
  section:
    source:
      - name: "domain"
        type: "string"
      - name: "api_credentials"
        type: "authentication"
        auth_type: "bearer"
        fields:
          - name: "bearer_token"
            is_encrypted: true

# 2. CONNECTOR - API settings and storage declaration
connector:
  name: "Project Manager API"
  base_url: "https://{domain}.projectapi.com/v1"
  default_headers:
    Content-Type: "application/json"
  
  # Declare where output data goes
  variables_metadata:
    final_output_file:
      format: json
      storage_name: results_dir
  variables_storages:
    - name: results_dir
      type: file_system

# 3. STEPS - The workflow
steps:
  # Step 1: Get project IDs
  - name: "Get Projects"
    type: rest
    http_method: GET
    endpoint: "{{%BASE_URL%}}/projects"
    pagination:
      type: page
      location: qs
      parameters:
        - name: page
          value: 1
          increment_by: 1
      break_conditions:
        - name: "No More"
          condition:
            type: empty_json_path
            key_json_path: "$.projects"
    variables_output:
      - response_location: data
        variable_name: project_ids
        variable_format: json
        transformation_layers:
          - type: extract_json
            from_type: json
            json_path: "$.projects[*].id"

  # Step 2: Loop through projects, get tasks
  - name: "Get Tasks"
    type: loop
    loop:
      type: data
      variable_name: project_ids
      item_name: project_id
      add_to_results: true
      ignore_errors: true
    steps:
      - name: "Fetch Project Tasks"
        type: rest
        http_method: GET
        endpoint: "{{%BASE_URL%}}/projects/{{%project_id%}}/tasks"
        variables_output:
          - response_location: data
            variable_name: final_output_file
            variable_format: json
            overwrite_storage: false
            transformation_layers:
              - type: extract_json
                from_type: json
                json_path: "$.tasks[*]"

Best practices

Follow these guidelines to optimize your connector configuration:

• Declare variables: Add entries to variables_metadata before you reference them in steps.
• Extract specific data: Use JSON paths to extract only the required data.
• Add break conditions: Always include break conditions to prevent infinite pagination loops.
• Configure retry strategies: Handle HTTP 429 and 5xx errors gracefully.
• Use meaningful names: Assign descriptive variable names. For example, use user_ids instead of data.
• Test JSON paths: Verify your paths against actual API responses.
• Mark sensitive fields: Always set is_encrypted: true for tokens, passwords, and other sensitive data.

Troubleshooting

Problem	Cause	Solution
Data is not saving	Variable is not declared in variables_metadata.	Add the missing variable declaration.
Loop is not iterating	The variable is empty, or the path is incorrect.	Make sure your transformation is successfully extracting an array.
External loop fails	The variable name is missing the {ext.} prefix.	Add the {ext.} prefix to the variable name.
Pagination never stops	There is no break condition set.	Add a break condition to stop the pagination.
Authentication fails	The auth type is incorrect, or required fields are missing.	Check your auth configuration to ensure it matches the API's requirements.

Related topics

• Refer Authentication Methods for detailed authentication configuration.
• Refer Interface Parameters for parameter types.
• Refer Pagination for pagination configuration.
• Refer Configuring a POST Request for POST example.