Bitbucket predefined reports
The Bitbucket source in Data Integration comes with 12 predefined reports in two groups: content data, with the full text of your documentation files, and engineering data, with 11 reports on repositories, pull requests, commits, pipelines, and deployments. Every per-repository report reads only the repositories you select under Repositories when you build the data flow.
Content data
Content data is optimized for Boomi Knowledge Hub. To load your documentation files into Knowledge Hub, choose Knowledge Hub as the target and select Documentation Files. See the Bitbucket walkthrough.
| Report | Description | Sync type |
|---|---|---|
| Documentation Files | Retrieves one row per documentation file (Markdown, MDX, reStructuredText, and AsciiDoc by default) on each selected repository's main branch, or on the branch you set. Each row has the file's repo_slug and path, its full text in content, and last_modified and last_commit_hash from the last commit that changed it. | Full refresh |
Engineering data
| Report | Description | Sync type |
|---|---|---|
| Repositories | Retrieves the workspace's repositories, including project, main branch, size, language, and visibility. | Incremental |
| Projects | Retrieves the workspace's projects. | Incremental |
| Workspace Members | Retrieves the members of the workspace. Email addresses aren't included. | Full refresh |
| Pull Requests | Retrieves pull requests in every state (open, merged, declined, and superseded), with title, description, author, source and destination branches, and dates. | Incremental |
| Pull Request Activity | Retrieves one row per pull request event: an update (opened, pushed, merged, declined), an approval, a change request, or a comment. Comments include their text, the file an inline comment is on, and the parent comment. | Incremental |
| Commits | Retrieves the commit history of each repository's main branch, or of the branch you set, with message, author, date, and parents. | Full refresh |
| Branches | Retrieves branches with their latest commit and merge strategies. | Full refresh |
| Pipelines | Retrieves Bitbucket Pipelines runs with their state, result, trigger, target branch or tag, and duration. | Full refresh |
| Pipeline Steps | Retrieves one row per step of each pipeline run, with its state, result, and duration. | Full refresh |
| Deployments | Retrieves deployments with their environment, state, and release. | Full refresh |
| Environments | Retrieves deployment environments. environment_type tells Test, Staging, and Production apart. | Full refresh |
The incremental reports send their start date to Bitbucket, which returns only records updated on or after it. For Pull Request Activity, the date applies to the pull request. Full refresh reports reload everything on every run.
Table Source Settings and merge keys
Each report's options appear under Table Source Settings. Bitbucket applies them itself, so they also cut API calls. When you load into a data warehouse, use Upsert-Merge with the merge keys below.
Content data
| Report | Table Source Settings | Merge key |
|---|---|---|
| Documentation Files | Folder, File filter, Folder depth, Branch | repo_slug, path |
- Folder: the folder to read, ending with a slash, for example
docs/. Leave it blank to read the whole repository. A narrow folder keeps runs short. - File filter: which files count as documentation. The default,
path ~ ".md" OR path ~ ".rst" OR path ~ ".adoc", also matches.mdx. To add plain text, appendOR path ~ ".txt". - Folder depth: how many folder levels below Folder to read. The default is 10. Lower it if a run fails with HTTP 555.
- Branch: the branch to read in every selected repository. Leave it blank for each repository's main branch. A repository without that branch is skipped.
Engineering data
| Report | Table Source Settings | Merge key |
|---|---|---|
| Repositories | Your role | uuid |
| Projects | None | uuid |
| Workspace Members | None | user_uuid |
| Pull Requests | States | repo_slug, id |
| Pull Request Activity | States | repo_slug, pull_request_id, activity_type, activity_date, actor_uuid, comment_id |
| Commits | Branch, Path, Exclude history of | repo_slug, hash |
| Branches | Filter, Sort | repo_slug, name |
| Pipelines | Branch, Status, Trigger, Ref type, Pipeline type, Pipeline name | uuid |
| Pipeline Steps | Same as Pipelines | uuid |
| Deployments | None | uuid |
| Environments | None | uuid |
- Your role (Repositories): only repositories where the connection's user is a member, contributor, admin, or owner.
- States (Pull Requests, Pull Request Activity): only pull requests in the selected states. Leave it blank for all four: open, merged, declined and superseded.
- Path and Exclude history of (Commits): only commits that changed a file or folder, or leave out the history of a branch, tag, or commit, for example an old release tag.
- Filter and Sort (Branches): a Bitbucket query (BBQL) on branch fields, for example
name ~ "release/", and the order to return branches in. - Branch, Status, Trigger, Ref type, Pipeline type, and Pipeline name (Pipelines, Pipeline Steps): only runs that match, for example failed runs on
main.
Joining engineering reports
- Pipeline Steps to Pipelines on
pipeline_uuid. - Deployments to Environments on
environment_uuid. - Pull Request Activity to Pull Requests on
repo_slugandpull_request_id.pull_request_idis text, so cast it to joinpull_requests.id.
Notes
Content data
- Documentation Files reloads every matching file on every run, at about 2 API calls per file.
- Each run returns only the files that exist at that moment. Nothing marks a file that was deleted or renamed in Bitbucket, so a merge keeps that file's earlier row.
- A documentation file that isn't UTF-8, or that fails to download, is skipped. The other files still load.
Engineering data
- Commits reads the whole history of the branch on every run. On busy repositories, set Exclude history of or Path to keep it small.
- Pipeline Steps costs 1 API call per pipeline run. On busy repositories, set a Branch or Status.
- Pull Request Activity reads the whole activity of each pull request updated in the window, not only the new events.
If you need a report or object that isn't currently supported, contact support so it can be evaluated for a future release.