Skip to main content
Feedback

Bright Data - Partner operation

The Bright Data — Partner connector operation defines how to interact with your Bright Data account and represents a specific action (Execute) to perform against a specific object type, such as Dataset, Scraper, Unlocker, or SERP.

Actions

Create a separate action and operation component for each action/object combination required for your integration. The Bright Data — Partner connector uses the Execute action across all its object types to streamline data retrieval and job submission.

Available actions

The following list introduces the object types managed by the connector using the Execute action:

  • Dataset: Access to the marketplace dataset filtering and retrieval. Handles both filtering
    and snapshot downloads with automatic polling.
  • Scraper: Web scraping job submission and asynchronous result retrieval.
  • Unlocker: Web unlocking and proxy services for accessing protected content.
  • SERP: Search engine results page data from various search engines (Google, Bing, etc.).

Operations

Each operation implements a specific task such as triggering a collection or querying a dataset.

The Bright Data — Partner connector operations return data in JSON format. If the underlying API returns non-JSON (e.g., HTML/CSV), the connector wraps the raw content into a JSON structure for consistent processing in Boomi.

Execute action

Options tab

Connector action: The Connector action corresponds to the action you want the operation to take in Bright Data — Partner connector. For this connector, it is set to Execute.

Object: The object can be imported using the Import button. You can choose from Dataset,
Scraper, Unlocker, or SERP.

Request profile: The profile is set to a flexible XML/XSD schema that accepts dynamic content structures, allowing for varied API interactions.

Response profile: This is a JSON profile returned by Bright Data — Partner connector.

**Tracking direction: Select the document tracking direction for the operation, either Input
Documents or Output Documents.
**Error behavior: Select Error Behavior if you want to be able to act on failed operations in
your process instead of only reporting them on the Manage page.
**Request Payload: (Optional) A field to define a document-level override for the request.
This can be JSON, URL query style, or newline-separated key=value pairs.

Object-Specific behavior

Dataset

  • Mode A: Dataset Filter: Used when no snapshot_id is provided. Filters marketplace datasets based on provided query parameters.

  • Mode B: Dataset Snapshot: Used when a snapshot_id (starting with snap_) is provided. The connector automatically polls every 10 seconds (up to 25 retries) until the snapshot is ready.

Scraper

  • Mode A: Trigger Collection: Used when no snapshot_id is provided. Submits a new scraping job and returns a snapshot_id.

  • Mode B: Scraper Snapshot: Used when a snapshot_id (starting with s_) is provided. Polls for completion and downloads results.

Unlocker

  • Requires a url parameter to bypass website restrictions.

  • Supports parameters like method, country, and body.

SERP

  • Requires engine and query parameters.

  • Supports multiple engines (comma-separated) in a single request.

Dynamic operation properties

The Bright Data — Partner connector allows you to set dynamic operation properties for the following properties to override default values:

  • Request Payload
On this Page