Pagination

This guide explains how to configure automatic pagination for Boomi Blueprint connectors. Use pagination to retrieve data from APIs that return results across multiple responses. When configured, the connector automatically sends consecutive requests until all data is retrieved.

Pagination ensures your integration can handle large datasets without manual looping.

important

Pagination parameters can only be sent via the query string (qs) or request body (body). Sending pagination parameters via request headers is not supported. However, you can read pagination cursors and tokens from response headers.

Pagination types

Select a pagination type based on the API's requirements:

Type	Description	Common APIs
page	Increments a page number with each request.	Jira, many REST APIs
offset	Increments a starting record count.	Slack, database-style APIs
cursor	Uses a unique token provided in the previous response.	GitHub, Salesforce, Stripe

Pagination locations

Define where the connector should place pagination parameters in the outgoing request:

Location	Description
qs	Parameters are added to the URL query string.
body	Parameters are added to the request body (common for POST).
url	The next-page token is embedded directly in a provided URL.

Page-based pagination

Increments a specific page number, for example, 1, 2, 3 for each subsequent request.

pagination:
  type: "page"
  location: "qs"
  parameters:
    - name: "page"
      value: 1              # Starting page
      increment_by: 1       # Amount to add per request
    - name: "per_page"
      value: 100            # Static page size
  break_conditions:
    - name: "No More Data"
      condition:
        type: "empty_json_path"
        key_json_path: "$.data"

How it works

The connector automatically increments the page value for each sequential request while keeping the per_page size static:

1. Request 1: ?page=1&per_page=100
2. Request 2: ?page=2&per_page=100
3. Request 3: ?page=3&per_page=100

The cycle repeats until a break condition is met.

Example: Jira issues

Example: Jira issues

steps:
  - name: "Get All Issues"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/search"
    query_params:
      jql: "project = PROJ"
    pagination:
      type: "page"
      location: "qs"
      parameters:
        - name: "startAt"
          value: 0
          increment_by: 100
        - name: "maxResults"
          value: 100
      break_conditions:
        - name: "All Issues Retrieved"
          condition:
            type: "total_items_reached"
            total_items_json_path: "$.total"
            current_items_json_path: "$.issues"
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"
        transformation_layers:
          - type: "extract_json"
            from_type: "json"
            json_path: "$.issues[*]"

Offset-based pagination

Increments a starting record index by the page size. For example, 0, 100, 200.

Example: Slack conversations

In this example, the connector increments the offset by 200 for every request.

steps:
  - name: "Get All Messages"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/conversations.history"
    pagination:
      type: "offset"
      location: "qs"
      parameters:
        - name: "offset"
          value: 0
          increment_by: 200
        - name: "limit"
          value: 200
      break_conditions:
        - name: "Less Than Page Size"
          condition:
            type: "page_size_break"
            page_size_param_name: "limit"
            items_json_path: "$.messages"

Cursor-based pagination

Uses a next page token provided by the API to fetch the next set of results.

Cursor from response body

Extract the next-page cursor from the response body:

pagination:
  type: "cursor"
  location: "qs"
  parameters:
    - name: "cursor"
      value: ""              # Empty on first request
      cursor_json_path: "$.next_cursor"    # Where to find next cursor
    - name: "limit"
      value: 100             # Items per request (static)
  break_conditions:
    - name: "No More Pages"
      condition:
        type: "empty_json_path"
        key_json_path: "$.next_cursor"

How it works

The connector sends the initial request, extracts the unique token provided in the API's response, and injects that exact token into the next request:

1. First request: ?limit=100 (no cursor)
2. Response contains: {"data": [...], "next_cursor": "abc123"}
3. Second request: ?cursor=abc123&limit=100
4. Response contains: {"data": [...], "next_cursor": "def456"}
5. Continues until next_cursor is empty or null

Cursor from response header

Extract the next-page cursor from response headers:

pagination:
  type: "cursor"
  location: "qs"
  parameters:
    - name: "cursor"
      value: ""
      cursor_header_key: "X-Next-Cursor"   # Header containing next cursor
    - name: "limit"
      value: 100
  break_conditions:
    - name: "No Cursor Header"
      condition:
        type: "empty_header"
        header_key: "X-Next-Cursor"

RFC 8288 link header pagination

For APIs using standard Link headers (GitHub, many modern APIs):

pagination:
  type: "cursor"
  location: "url"                # Full URL in Link header
  parameters:
    - name: "page"
      value: ""
      cursor_header_key: "link"  # Standard Link header
      link_header_type: "rfc8288"
      link_rel: "next"           # Follow "next" relation
  break_conditions:
    - name: "No Next Link"
      condition:
        type: "empty_header"
        header_key: "link"

Example: GitHub repositories

steps:
  - name: "Get All Repos"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/user/repos"
    query_params:
      per_page: "100"
    pagination:
      type: "cursor"
      location: "url"
      parameters:
        - name: "page"
          value: ""
          cursor_header_key: "link"
          link_header_type: "rfc8288"
          link_rel: "next"
      break_conditions:
        - name: "No More Pages"
          condition:
            type: "empty_header"
            header_key: "link"
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"

Example: Stripe customers

steps:
  - name: "Get All Customers"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/customers"
    pagination:
      type: "cursor"
      location: "qs"
      parameters:
        - name: "starting_after"
          value: ""
          cursor_json_path: "$.data[-1].id"   # Last item's ID
        - name: "limit"
          value: 100
      break_conditions:
        - name: "No More Data"
          condition:
            type: "boolean_break"
            key_json_path: "$.has_more"
            break_value: false
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"
        transformation_layers:
          - type: "extract_json"
            from_type: "json"
            json_path: "$.data[*]"

Break conditions

Break conditions determine when the pagination loop stops. You must include at least one condition to prevent infinite loops.

Type	Description
empty_json_path	Stop when JSONPath returns empty/null
page_size_break	Stop when items returned < page size
total_items_reached	Stop when total count reached
boolean_break	Stop when boolean field equals value
string_equal	Stop when string field equals value
number_break	Stop when numeric condition met
empty_header	Stop when response header is empty

Expand to see supported break conditions

Empty JSON Path

Stop when a specific path in the response is empty or null:

break_conditions:
  - name: "No More Data"
    condition:
      type: "empty_json_path"
      key_json_path: "$.data"

Page size break

Stop when fewer items are returned than the page size (indicates last page):

break_conditions:
  - name: "Less Than Page Size"
    condition:
      type: "page_size_break"
      page_size_param_name: "limit"     # Name of your limit parameter
      items_json_path: "$.items"        # Path to items array

Total items reached

Stop when the cumulative count of retrieved items equals the total:

break_conditions:
  - name: "All Items Retrieved"
    condition:
      type: "total_items_reached"
      total_items_json_path: "$.total"        # Total count from API
      current_items_json_path: "$.results"    # Items array path

Boolean break

Stop when a boolean field equals a specific value:

break_conditions:
  - name: "No More Pages"
    condition:
      type: "boolean_break"
      key_json_path: "$.has_more"
      break_value: false                # Stop when has_more is false

String equal

Stop when a string field equals a specific value:

break_conditions:
  - name: "End Status Reached"
    condition:
      type: "string_equal"
      key_json_path: "$.status"
      break_value: "complete"

Empty header

Stop when a response header is empty or missing:

break_conditions:
  - name: "No Next Page Header"
    condition:
      type: "empty_header"
      header_key: "X-Next-Page"

Multiple break conditions

Use multiple conditions for safety. Pagination stops when ANY condition is met:

Example of multiple break conditions

break_conditions:
  # Primary: API indicates no more data
  - name: "No More Data"
    condition:
      type: "boolean_break"
      key_json_path: "$.has_next"
      break_value: false
  
  # Safety: Empty results
  - name: "Empty Results"
    condition:
      type: "empty_json_path"
      key_json_path: "$.data"
  
  # Safety: Page size check
  - name: "Less Than Requested"
    condition:
      type: "page_size_break"
      page_size_param_name: "limit"
      items_json_path: "$.data"

Pagination with POST requests

For APIs that use POST requests, configure the pagination location to target the request body.

Example

steps:
  - name: "Search Records"
    type: "rest"
    http_method: "POST"
    endpoint: "{{%BASE_URL%}}/search"
    headers:
      Content-Type: "application/json"
    body:
      query: "status:active"
      fields: ["id", "name", "email"]
    pagination:
      type: "offset"
      location: "body"               # Pagination in request body
      parameters:
        - name: "offset"
          value: 0
          increment_by: 50
        - name: "limit"
          value: 50
      break_conditions:
        - name: "All Retrieved"
          condition:
            type: "page_size_break"
            page_size_param_name: "limit"
            items_json_path: "$.results"
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"
        transformation_layers:
          - type: "extract_json"
            from_type: "json"
            json_path: "$.results[*]"

Pagination configuration examples

Here are a few complete pagination configuration:

Simple page pagination

steps:
  - name: "Get All Users"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/users"
    pagination:
      type: "page"
      location: "qs"
      parameters:
        - name: "page"
          value: 1
          increment_by: 1
        - name: "size"
          value: 100
      break_conditions:
        - name: "Empty Page"
          condition:
            type: "empty_json_path"
            key_json_path: "$.users"
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"
        transformation_layers:
          - type: "extract_json"
            from_type: "json"
            json_path: "$.users[*]"

Cursor with total count

steps:
  - name: "Get All Orders"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/orders"
    pagination:
      type: "cursor"
      location: "qs"
      parameters:
        - name: "page_token"
          value: ""
          cursor_json_path: "$.next_page_token"
        - name: "page_size"
          value: 250
      break_conditions:
        - name: "No Next Token"
          condition:
            type: "empty_json_path"
            key_json_path: "$.next_page_token"
        - name: "Total Reached"
          condition:
            type: "total_items_reached"
            total_items_json_path: "$.total_orders"
            current_items_json_path: "$.orders"
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"
        transformation_layers:
          - type: "extract_json"
            from_type: "json"
            json_path: "$.orders[*]"

Link header pagination

steps:
  - name: "Get All Issues"
    type: "rest"
    http_method: "GET"
    endpoint: "{{%BASE_URL%}}/repos/owner/repo/issues"
    query_params:
      state: "all"
      per_page: "100"
    pagination:
      type: "cursor"
      location: "url"
      parameters:
        - name: "page"
          value: ""
          cursor_header_key: "link"
          link_header_type: "rfc8288"
          link_rel: "next"
      break_conditions:
        - name: "No Next Link"
          condition:
            type: "empty_header"
            header_key: "link"
    variables_output:
      - response_location: "data"
        variable_name: "final_output_file"
        variable_format: "json"

Best practices

• Always include break conditions: Do not configure pagination without defining at least one break condition. Without a break condition, your connector will continuously request subsequent pages—creating an infinite loop that can exhaust your API limits and system resources.

  Example: Break condition

  # WRONG: Risk of infinite loop
  pagination:
    type: "page"
    location: "qs"
    parameters:
      - name: "page"
        value: 1
        increment_by: 1
  
  # CORRECT: Has break condition
  pagination:
    type: "page"
    location: "qs"
    parameters:
      - name: "page"
        value: 1
        increment_by: 1
    break_conditions:
      - name: "Empty Results"
        condition:
          type: "empty_json_path"
          key_json_path: "$.data"

• Implement multiple safety Conditions: Add backup break conditions such as page_size_break or empty_json_path to handle unexpected API behaviors.

  Example

  Add backup break conditions for reliability:

  break_conditions:
    - name: "Primary: Empty Data"
      condition:
        type: "empty_json_path"
        key_json_path: "$.results"
    - name: "Safety: Page Size Check"
      condition:
        type: "page_size_break"
        page_size_param_name: "limit"
        items_json_path: "$.results"

• Maximize page sizes: Set your parameters to the maximum page size permitted by the API.

  Example: Set appropriate page sizes

  Use the maximum page size allowed by the API to minimize requests:

  parameters:
    - name: "per_page"
      value: 100      # Use API's maximum allowed

• Match offsets to increments: When using offset pagination, ensure your increment_by value matches your limit value to prevent skipping records or pulling duplicates.

  Example

  For offset pagination, ensure increment matches page size:

  parameters:
    - name: "offset"
      value: 0
      increment_by: 100     # Must match limit
    - name: "limit"
      value: 100            # Same as increment_by

• Review API documentation: Different APIs utilize highly variable pagination patterns. Identify required parameters, for example, startAt vs. offset before configuration.

  Example

  Different APIs use different pagination patterns:

  API	Pagination type	Key parameters
  Jira	Page (offset)	startAt, maxResults
  GitHub	Cursor (Link)	per_page, Link header
  Stripe	Cursor	starting_after, limit
  Salesforce	Cursor	nextRecordsUrl
  Slack	Cursor	cursor, limit

Troubleshooting

Pagination is not stopping

• Verify your break condition's json_path accurately targets the final response payload.
• Test the JSONPath expression against a sample API response manually.
• Add secondary break conditions as fail-safes.

The connector retrieves duplicate records

• Confirm whether the endpoint uses 1-based or 0-based page numbering.
• Check that your increment_by value accurately reflects the page size limits.
• Ensure your cursor parameter is updating correctly across sequential requests.

The connector is skipping pages

• Verify that your initial starting value is correct. For example, starting at 0 instead of 1.
• Check that your increment_by parameter does not exceed the page size.

The cursor is not found

• Verify your cursor_json_path precisely matches the response payload structure.
• Check if the API nests the cursor differently on the first page compared to subsequent pages.
• If the cursor is not in the body, try using cursor_header_key to locate it in the response headers.