diff --git a/api-usage-docs/service-amazon-advertising-api.md b/api-usage-docs/service-amazon-advertising-api.md index a592e41..3c48f93 100644 --- a/api-usage-docs/service-amazon-advertising-api.md +++ b/api-usage-docs/service-amazon-advertising-api.md @@ -242,3 +242,24 @@ Authorization: Bearer } } ``` + +--- + +### Marketing Stream S3 + +Product `110` registers Amazon Marketing Stream datasets for delivery through shared, regional S3 and SQS resources managed by Openbridge. Authenticate each request with an Openbridge bearer token. + +| Method | Endpoint | Success | +|---|---|---| +| GET | `/service/amzadv/list-adv-v2/{remote_identity_id}` | `200 OK` with an array of advertiser accounts. Supports the optional `advertiser_id` query filter. | +| GET | `/service/amzadv/stream-s3/list-datasets` | `200 OK` with the dataset list at `attributes.datasets`. | +| POST | `/service/amzadv/stream-s3/{remote_identity_id}` | `200 OK` with the Amazon subscription IDs. | +| PATCH | `/service/amzadv/stream-s3/update/{remote_identity_id}/{subscription_id}` | `200 OK` with replacement Amazon subscription IDs. | +| DELETE | `/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id}` | `204 No Content` after archiving the Amazon streams stored on the Openbridge subscription. | +| DELETE | `/service/amzadv/stream-s3/delete-orphaned/{remote_identity_id}/{advertiser_id}` | `204 No Content` after archiving the advertiser's streams eligible for cleanup. Requires that the remote identity have no active product `110` subscriptions. | + +Create and update run synchronously. Send the advertiser ID and full dataset list in `data.attributes.advertiser_id` and `data.attributes.datasets`. Both operations return a top-level `advertiser_id` and a dataset-to-ID map at `attributes.amz_subscription_ids`. Update archives the old Amazon streams before registering the requested datasets. + +Store `advertiser_id` and the JSON-stringified `amz_subscription_ids` map in the product `110` pipeline's `product_parameters`. See the [Marketing Stream S3 tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline.md) for request examples and the full pipeline lifecycle. + +For the `/stream-v2` customer-managed AWS workflow, see the [legacy tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md). diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md b/tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md new file mode 100644 index 0000000..274e082 --- /dev/null +++ b/tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md @@ -0,0 +1,510 @@ +# Amazon Advertising Marketing Stream Pipeline (Legacy) + +> **Legacy:** This guide covers the customer-managed AWS flow using `/stream-v2` and product `87`. For new pipelines, see [Marketing Stream v2](./amazon-advertising-marketing-stream-pipeline.md), which uses `/stream-s3` and requires no customer AWS account or CloudFormation template. + +## Table of Contents + +- [Overview](#overview) +- [Prerequisites](#prerequisites) +- [Step 1 — Choose or create a remote identity](#step-1--choose-or-create-a-remote-identity) +- [Step 2 — Select a profile type](#step-2--select-a-profile-type) +- [Step 3 — Select a profile](#step-3--select-a-profile) + - [Sponsored Ads profiles (seller/vendor)](#sponsored-ads-profiles-sellervendor) + - [DSP profiles](#dsp-profiles) +- [Step 4 — (DSP only) Select an advertiser](#step-4--dsp-only-select-an-advertiser) +- [Step 5 — Select marketing stream datasets](#step-5--select-marketing-stream-datasets) +- [Step 6 — Provide an AWS IAM Role ARN and generate SQS queues](#step-6--provide-an-aws-iam-role-arn-and-generate-sqs-queues) +- [Step 7 — Create the pipeline subscription](#step-7--create-the-pipeline-subscription) + - [product_parameters](#product_parameters) + - [Full example request (Sponsored Ads)](#full-example-request-sponsored-ads) + - [Full example request (DSP)](#full-example-request-dsp) +- [Updating a marketing stream subscription](#updating-a-marketing-stream-subscription) +- [Deleting a marketing stream subscription](#deleting-a-marketing-stream-subscription) + +--- + +## Overview + +This tutorial walks through creating an Amazon Advertising Marketing Stream pipeline subscription end-to-end. The Marketing Stream product (product ID `87`) delivers near-real-time advertising performance data via Amazon's Marketing Stream API into SQS queues provisioned by Openbridge. + +Three profile-type flows are supported: + +| Profile type | Description | +|---|---| +| **Sponsored Ads** (`ads`) | Seller or Vendor accounts running Sponsored Products, Sponsored Brands, or Sponsored Display campaigns | +| **DSP** (`dsp`) | Demand-Side Platform accounts | +| **Managed DSP** (`managedDsp`) | DSP manager accounts acting on behalf of advertisers | + +The flow mirrors the Openbridge website wizard: choose an identity, select a profile type, pick a profile (and advertiser for DSP), choose datasets, provide an AWS IAM Role ARN to provision SQS queues, and finally create the pipeline subscription. + +--- + +## Prerequisites + +- A JWT access token — see [Authentication API](../api-usage-docs/authentication-api.md) +- Your account ID and user ID — see [Identity Configuration Step 1](./identity-configuration.md#step-1--look-up-your-account-and-user-ids) +- An active Amazon Advertising account +- An AWS IAM Role ARN with permissions for Openbridge to provision SQS queues on your behalf +- A storage destination already configured — see [Subscription Configuration Step 3](./subscription-configuration.md#step-3--identify-your-storage-destination) + +--- + +## Step 1 — Choose or create a remote identity + +The Marketing Stream product requires an **Amazon Advertising** identity (remote identity type `14`). + +### Create a new identity + +If you do not have an identity yet, follow the [Identity Configuration tutorial](./identity-configuration.md) to create one via the OAuth flow. Amazon Advertising identities are region-specific — use the correct region for your marketplace: + +| Region identifier | Region name | +|---|---| +| `na` | North America | +| `eu` | Europe | +| `fe` | Far East | + +### List existing identities + +To find an existing Amazon Advertising identity, list remote identities filtered by type: + +```http +GET https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0 +Authorization: Bearer +``` + +**Example response:** + +```json +{ + "links": { + "first": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", + "last": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", + "next": "", + "prev": "" + }, + "data": [ + { + "type": "RemoteIdentity", + "id": "112", + "attributes": { + "name": "My Advertising Account", + "created_at": "2024-01-15T12:00:00", + "modified_at": "2024-06-01T08:30:00", + "remote_unique_id": "amzn1.account.AEHZ5", + "account_id": 1, + "user_id": 1, + "invalid_identity": 0, + "region": "na", + "email": "advertiser@example.com" + } + } + ], + "meta": { + "pagination": { + "page": 1, + "pages": 1, + "count": 1 + } + } +} +``` + +Record the identity `id` (e.g., `112`) and `region` (e.g., `na`) — you will need both in subsequent steps. + +See the [Remote Identity API](../api-usage-docs/remote-identity-api.md) for full endpoint details. + +--- + +## Step 2 — Select a profile type + +Before fetching profiles, decide which profile type applies to your use case: + +| Profile type value | Use case | +|---|---| +| `ads` | Seller or Vendor accounts (Sponsored Products, Sponsored Brands, Sponsored Display) | +| `dsp` | DSP advertiser accounts | +| `managedDsp` | DSP manager accounts operating on behalf of advertisers | + +This choice determines the wizard flow, the available datasets, and how the SQS queue payload is constructed. Record your `profile_type` — you will use it in Steps 3, 5, and 6. + +--- + +## Step 3 — Select a profile + +### Sponsored Ads profiles (seller/vendor) + +For `ads` profile types, fetch seller and vendor profiles: + +```http +GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=seller,vendor +Authorization: Bearer +``` + +Replace `{remote_identity_id}` with the identity ID from Step 1. + +**Example response:** + +```json +[ + { + "id": 4463883966959342, + "type": "AmazonAdvertisingProfile", + "attributes": { + "country_code": "US", + "currency_code": "USD", + "daily_budget": 10.0, + "timezone": "America/Los_Angeles", + "account_info": { + "id": "ENTITY1234567890", + "type": "AmazonAdvertisingProfileAccountInfo", + "attributes": { + "marketplace_country": "US", + "marketplace_string_id": "ATVPDKIKX0DER", + "name": "My Seller Account", + "type": "seller", + "subType": "", + "valid_payment_method": true + } + } + } + } +] +``` + +Record the profile `id` (e.g., `4463883966959342`). For Sponsored Ads, the `advertiser_id` meta field will be set to `NOT_APPLICABLE`. + +> **Note:** Profiles with `subType` of `AMAZON_ATTRIBUTION` are excluded and cannot be used with Marketing Stream. + +### DSP profiles + +For `dsp` profile types, fetch DSP profiles: + +```http +GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp +Authorization: Bearer +``` + +For `managedDsp` profile types, fetch managed DSP accounts: + +```http +GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp&is_manager=true +Authorization: Bearer +``` + +For managed DSP accounts, the response includes a `dsp_advertiser_id` field. Record this value — it is used as the `advertiser_id` in subsequent steps. For managed DSP, the `profile_id` meta field will be set to `NOT_APPLICABLE`. + +See the [Service API: Amazon Advertising](../api-usage-docs/service-amazon-advertising-api.md) for full endpoint details. + +--- + +## Step 4 — (DSP only) Select an advertiser + +This step applies only to the `dsp` profile type. If your profile type is `ads` or `managedDsp`, skip to Step 5. + +Fetch the advertisers available under your DSP profile: + +```http +GET https://service.api.openbridge.io/service/amzadv/list-adv/{remote_identity_id}/{profile_id} +Authorization: Bearer +``` + +Replace `{remote_identity_id}` with your identity ID and `{profile_id}` with the DSP profile ID from Step 3. + +**Example response:** + +```json +[ + { + "id": "ADV123456", + "type": "AmazonAdvertisingAdvertiser", + "attributes": { + "name": "My DSP Advertiser", + "country": "US", + "currency": "USD", + "timezone": "America/Los_Angeles", + "url": "https://example.com" + } + } +] +``` + +Record the advertiser `id` (e.g., `ADV123456`) — you will use it as the `advertiser_id` in the SQS payload and subscription meta. + +--- + +## Step 5 — Select marketing stream datasets + +Fetch the available marketing stream datasets for your region and profile type: + +```http +GET https://service.api.openbridge.io/service/amzadv/stream-v2/list-datasets?region={region}&profile_type={profile_type} +Authorization: Bearer +``` + +Replace `{region}` with your identity's region (e.g., `na`) and `{profile_type}` with the type from Step 2 (e.g., `ads` or `dsp`). + +The response returns a list of dataset names. Below are the known datasets and their internal payload names: + +### Sponsored Ads datasets + +| Dataset name | Payload name | +|---|---| +| `sp-traffic` | `amzn_stream2_sp_traffic` | +| `sp-conversion` | `amzn_stream2_sp_conversion` | +| `budget-usage` | `amzn_stream2_budget_usage` | +| `sd-traffic` | `amzn_stream2_sd_traffic` | +| `sd-conversion` | `amzn_stream2_sb_conversion` | +| `sb-traffic` | `amzn_stream2_sb_traffic` | +| `sb-conversion` | `amzn_stream2_sd_conversion` | +| `sb-clickstream` | `amzn_stream2_sb_clickstream` | +| `sb-rich-media` | `amzn_stream2_sb_rich_media` | +| `sp-budget-recommendations` | `amzn_stream2_sp_budget_recommendations` | +| `sponsored-ads-campaign-diagnostics-recommendations` | `amzn_stream2_campaign_recommendations` | +| `campaigns` | `amzn_stream2_campaigns` | +| `adgroups` | `amzn_stream2_adgroups` | +| `ads` | `amzn_stream2_ads` | +| `targets` | `amzn_stream2_targets` | + +### DSP datasets + +| Dataset name | Payload name | +|---|---| +| `adsp-traffic` | `amzn_stream2_adsp_traffic` | +| `adsp-conversion` | `amzn_stream2_adsp_conversion` | +| `adsp-clickstream` | `amzn_stream2_adsp_clickstream` | +| `adsp-rich-media` | `amzn_stream2_adsp_rich_media` | +| `adsp-campaigns` | `amzn_stream2_adsp_campaigns` | +| `adsp-campaign-flights` | `amzn_stream2_adsp_campaign_flights` | +| `adsp-adgroups` | `amzn_stream2_adsp_adgroups` | + +Choose the datasets you want to subscribe to and record them as an array (e.g., `["sp-traffic", "sp-conversion", "budget-usage"]`). You will pass this array in Step 6 and as a stringified JSON array in the subscription meta in Step 7. + +--- + +## Step 6 — Provide an AWS IAM Role ARN and generate SQS queues + +The Marketing Stream product requires an AWS IAM Role ARN that grants Openbridge permission to create and manage SQS queues in your AWS account. Openbridge uses this role to provision dedicated SQS queues for each selected dataset. + +### Create the SQS queues + +``` +POST https://service.api.openbridge.io/service/amzadv/stream-v2/{remote_identity_id} +``` + +Replace `{remote_identity_id}` with the identity ID from Step 1. + +**Example request (Sponsored Ads):** + +```http +POST https://service.api.openbridge.io/service/amzadv/stream-v2/112 +Authorization: Bearer +Content-Type: application/json +``` + +```json +{ + "data": { + "type": "Service", + "attributes": { + "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "datasets": ["sp-traffic", "sp-conversion", "budget-usage"], + "profile_type": "ads", + "profile_id": "4463883966959342" + } + } +} +``` + +**Example request (DSP):** + +```json +{ + "data": { + "type": "Service", + "attributes": { + "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "datasets": ["adsp-traffic", "adsp-conversion"], + "profile_type": "dsp", + "advertiser_id": "ADV123456" + } + } +} +``` + +| Field | Description | +|---|---| +| `role_arn` | AWS IAM Role ARN granting Openbridge permissions to manage SQS queues | +| `datasets` | Array of dataset names from Step 5 | +| `profile_type` | `"ads"` for Sponsored Ads, `"dsp"` for DSP and Managed DSP | +| `profile_id` | Required when `profile_type` is `"ads"` — the profile ID from Step 3 | +| `advertiser_id` | Required when `profile_type` is `"dsp"` — the advertiser ID from Step 4 (or the `dsp_advertiser_id` for managed DSP) | + +### Async response handling + +This endpoint returns an **asynchronous response**. The initial response includes a `Location` header with a polling URL. Poll this URL until you receive an HTTP `200` response with the queue data. + +- **Poll interval:** 2 seconds +- **Timeout:** up to 13 minutes (the queue provisioning process can take several minutes) + +**Example success response:** + +```json +{ + "data": { + "type": "AmazonAdvertisingStreamV2", + "attributes": { + "queue_urls": { + "sp-traffic": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123", + "sp-conversion": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123", + "budget-usage": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123" + }, + "dataset_params": { + "sp-traffic": {"stream_id": "str_abc123"}, + "sp-conversion": {"stream_id": "str_def456"}, + "budget-usage": {"stream_id": "str_ghi789"} + } + } + } +} +``` + +Save the `queue_urls` and `dataset_params` values from the response — they are required for the pipeline subscription meta in Step 7. + +--- + +## Step 7 — Create the pipeline subscription + +With all the pieces gathered from the previous steps, create the pipeline subscription for product `87` (Amazon Advertising Marketing Stream). + +``` +POST https://subscriptions.api.openbridge.io/v2/sub +``` + +For general detail on subscription creation, see the [Subscription Configuration tutorial](./subscription-configuration.md). + +### product_parameters + +This product requires 8 keys in `product_parameters`. Remote identity is set via the top-level `remote_identity` field (see Step 1) rather than a `product_parameters` key. + +| Key | Source | Description | +|---|---|---| +| `profile_id` | Step 3 | The advertising profile ID. Set to `"NOT_APPLICABLE"` for managed DSP. | +| `profile_type` | Step 2 | `"ads"`, `"dsp"`, or `"managedDsp"` | +| `advertiser_id` | Step 4 | The advertiser ID. Set to `"NOT_APPLICABLE"` for Sponsored Ads (`ads`). | +| `aws_iam_role_arn` | Step 6 | The AWS IAM Role ARN provided in the SQS generation request | +| `queue_urls` | Step 6 | Stringified `queue_urls` map from the SQS generation response (JSON string) | +| `dataset_params` | Step 6 | Stringified `dataset_params` map from the SQS generation response (JSON string) | +| `selected_tables` | Step 5 | Stringified array of selected dataset names (JSON string) | +| `stage_ids` | Step 5 | Stringified array of stage IDs corresponding to the selected datasets (JSON string) | + +> **Note:** The `stage_ids` are resolved by mapping each selected dataset name to its corresponding payload stage via the [Products API payload definitions](../api-usage-docs/products-api.md). Each dataset name maps to a payload name (see the tables in Step 5), and the payload's `stage_id` is used. + +### Full example request (Sponsored Ads) + +```json +{ + "data": { + "type": "Subscription", + "attributes": { + "account": 1, + "user": 1, + "product": 87, + "name": "My Marketing Stream - Sponsored Ads", + "status": "active", + "date_start": "2024-06-01T00:00:00Z", + "remote_identity": 112, + "storage_group": 1, + "product_parameters": { + "profile_id": "4463883966959342", + "profile_type": "ads", + "advertiser_id": "NOT_APPLICABLE", + "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "queue_urls": "{\"sp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123\",\"sp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123\",\"budget_usage\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123\"}", + "dataset_params": "{\"sp-traffic\":{\"stream_id\":\"str_abc123\"},\"sp-conversion\":{\"stream_id\":\"str_def456\"},\"budget-usage\":{\"stream_id\":\"str_ghi789\"}}", + "selected_tables": "[\"sp-traffic\",\"sp-conversion\",\"budget-usage\"]", + "stage_ids": "[101,102,103]" + } + } + } +} +``` + +### Full example request (DSP) + +```json +{ + "data": { + "type": "Subscription", + "attributes": { + "account": 1, + "user": 1, + "product": 87, + "name": "My Marketing Stream - DSP", + "status": "active", + "date_start": "2024-06-01T00:00:00Z", + "remote_identity": 112, + "storage_group": 1, + "product_parameters": { + "profile_id": "4463883966959342", + "profile_type": "dsp", + "advertiser_id": "ADV123456", + "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "queue_urls": "{\"adsp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-traffic-abc123\",\"adsp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-conversion-abc123\"}", + "dataset_params": "{\"adsp-traffic\":{\"stream_id\":\"str_xyz123\"},\"adsp-conversion\":{\"stream_id\":\"str_xyz456\"}}", + "selected_tables": "[\"adsp-traffic\",\"adsp-conversion\"]", + "stage_ids": "[201,202]" + } + } + } +} +``` + +Replace the placeholder values (`account`, `user`, `remote_identity`, `storage_group`, and the `product_parameters` values) with values from your account and the previous steps. + +--- + +## Updating a marketing stream subscription + +To update an existing marketing stream subscription (e.g., to add or remove datasets), use two requests: + +**1. Update the SQS queues via Service API:** + +``` +PATCH https://service.api.openbridge.io/service/amzadv/stream-v2/update/{remote_identity_id}/{subscription_id} +``` + +The request body is the same structure as the create endpoint — include the `role_arn`, updated `datasets`, `profile_type`, and `profile_id` or `advertiser_id`. This is an async operation with the same polling behavior as the create endpoint. + +**2. Update the pipeline subscription:** + +``` +PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +``` + +Update the `queue_urls`, `dataset_params`, `selected_tables`, `stage_ids`, and any other changed keys in `product_parameters` with the new values from the update response (partial merge — omitted keys keep their stored value). + +See the [Subscriptions API (v2)](../api-usage-docs/subscriptions-api.md) for full PATCH documentation. + +--- + +## Deleting a marketing stream subscription + +To delete a marketing stream pipeline, mark the pipeline subscription as invalid: + +``` +PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +``` + +```json +{ + "data": { + "type": "Subscription", + "id": "12345", + "attributes": { + "status": "invalid" + } + } +} +``` + +See the [Subscription Configuration tutorial](./subscription-configuration.md#deleting-a-subscription) for more detail on subscription status management. diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index 167220f..1fbb4d9 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -1,300 +1,43 @@ -# Amazon Advertising Marketing Stream Pipeline +# Amazon Advertising Marketing Stream S3 Pipeline -## Table of Contents - -- [Overview](#overview) -- [Prerequisites](#prerequisites) -- [Step 1 — Choose or create a remote identity](#step-1--choose-or-create-a-remote-identity) -- [Step 2 — Select a profile type](#step-2--select-a-profile-type) -- [Step 3 — Select a profile](#step-3--select-a-profile) - - [Sponsored Ads profiles (seller/vendor)](#sponsored-ads-profiles-sellervendor) - - [DSP profiles](#dsp-profiles) -- [Step 4 — (DSP only) Select an advertiser](#step-4--dsp-only-select-an-advertiser) -- [Step 5 — Select marketing stream datasets](#step-5--select-marketing-stream-datasets) -- [Step 6 — Provide an AWS IAM Role ARN and generate SQS queues](#step-6--provide-an-aws-iam-role-arn-and-generate-sqs-queues) -- [Step 7 — Create the pipeline subscription](#step-7--create-the-pipeline-subscription) - - [product_parameters](#product_parameters) - - [Full example request (Sponsored Ads)](#full-example-request-sponsored-ads) - - [Full example request (DSP)](#full-example-request-dsp) -- [Updating a marketing stream subscription](#updating-a-marketing-stream-subscription) -- [Deleting a marketing stream subscription](#deleting-a-marketing-stream-subscription) - ---- - -## Overview - -This tutorial walks through creating an Amazon Advertising Marketing Stream pipeline subscription end-to-end. The Marketing Stream product (product ID `87`) delivers near-real-time advertising performance data via Amazon's Marketing Stream API into SQS queues provisioned by Openbridge. - -Three profile-type flows are supported: - -| Profile type | Description | -|---|---| -| **Sponsored Ads** (`ads`) | Seller or Vendor accounts running Sponsored Products, Sponsored Brands, or Sponsored Display campaigns | -| **DSP** (`dsp`) | Demand-Side Platform accounts | -| **Managed DSP** (`managedDsp`) | DSP manager accounts acting on behalf of advertisers | - -The flow mirrors the Openbridge website wizard: choose an identity, select a profile type, pick a profile (and advertiser for DSP), choose datasets, provide an AWS IAM Role ARN to provision SQS queues, and finally create the pipeline subscription. - ---- +Marketing Stream S3 (product `110`) delivers Amazon dataset files through shared, regional S3 and SQS resources managed by Openbridge. Setup requires an Amazon Advertising connection and an Openbridge storage destination. For the customer-managed AWS workflow, see the [legacy Marketing Stream tutorial](./amazon-advertising-marketing-stream-pipeline-legacy.md). ## Prerequisites -- A JWT access token — see [Authentication API](../api-usage-docs/authentication-api.md) -- Your account ID and user ID — see [Identity Configuration Step 1](./identity-configuration.md#step-1--look-up-your-account-and-user-ids) -- An active Amazon Advertising account -- An AWS IAM Role ARN with permissions for Openbridge to provision SQS queues on your behalf -- A storage destination already configured — see [Subscription Configuration Step 3](./subscription-configuration.md#step-3--identify-your-storage-destination) - ---- - -## Step 1 — Choose or create a remote identity - -The Marketing Stream product requires an **Amazon Advertising** identity (remote identity type `14`). - -### Create a new identity - -If you do not have an identity yet, follow the [Identity Configuration tutorial](./identity-configuration.md) to create one via the OAuth flow. Amazon Advertising identities are region-specific — use the correct region for your marketplace: - -| Region identifier | Region name | -|---|---| -| `na` | North America | -| `eu` | Europe | -| `fe` | Far East | - -### List existing identities - -To find an existing Amazon Advertising identity, list remote identities filtered by type: - -```http -GET https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0 -Authorization: Bearer -``` - -**Example response:** - -```json -{ - "links": { - "first": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", - "last": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", - "next": "", - "prev": "" - }, - "data": [ - { - "type": "RemoteIdentity", - "id": "112", - "attributes": { - "name": "My Advertising Account", - "created_at": "2024-01-15T12:00:00", - "modified_at": "2024-06-01T08:30:00", - "remote_unique_id": "amzn1.account.AEHZ5", - "account_id": 1, - "user_id": 1, - "invalid_identity": 0, - "region": "na", - "email": "advertiser@example.com" - } - } - ], - "meta": { - "pagination": { - "page": 1, - "pages": 1, - "count": 1 - } - } -} -``` - -Record the identity `id` (e.g., `112`) and `region` (e.g., `na`) — you will need both in subsequent steps. - -See the [Remote Identity API](../api-usage-docs/remote-identity-api.md) for full endpoint details. - ---- - -## Step 2 — Select a profile type - -Before fetching profiles, decide which profile type applies to your use case: - -| Profile type value | Use case | -|---|---| -| `ads` | Seller or Vendor accounts (Sponsored Products, Sponsored Brands, Sponsored Display) | -| `dsp` | DSP advertiser accounts | -| `managedDsp` | DSP manager accounts operating on behalf of advertisers | - -This choice determines the wizard flow, the available datasets, and how the SQS queue payload is constructed. Record your `profile_type` — you will use it in Steps 3, 5, and 6. - ---- - -## Step 3 — Select a profile - -### Sponsored Ads profiles (seller/vendor) - -For `ads` profile types, fetch seller and vendor profiles: - -```http -GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=seller,vendor -Authorization: Bearer -``` +- An [Openbridge JWT access token](../api-usage-docs/authentication-api.md). +- Your [Openbridge account and user IDs](./identity-configuration.md#step-1--look-up-your-account-and-user-ids). +- An Amazon Advertising [remote identity](./identity-configuration.md) (type `14`) with access to the advertiser. +- A configured [storage destination](./subscription-configuration.md#step-3--identify-your-storage-destination). -Replace `{remote_identity_id}` with the identity ID from Step 1. - -**Example response:** - -```json -[ - { - "id": 4463883966959342, - "type": "AmazonAdvertisingProfile", - "attributes": { - "country_code": "US", - "currency_code": "USD", - "daily_budget": 10.0, - "timezone": "America/Los_Angeles", - "account_info": { - "id": "ENTITY1234567890", - "type": "AmazonAdvertisingProfileAccountInfo", - "attributes": { - "marketplace_country": "US", - "marketplace_string_id": "ATVPDKIKX0DER", - "name": "My Seller Account", - "type": "seller", - "subType": "", - "valid_payment_method": true - } - } - } - } -] -``` - -Record the profile `id` (e.g., `4463883966959342`). For Sponsored Ads, the `advertiser_id` meta field will be set to `NOT_APPLICABLE`. - -> **Note:** Profiles with `subType` of `AMAZON_ATTRIBUTION` are excluded and cannot be used with Marketing Stream. - -### DSP profiles - -For `dsp` profile types, fetch DSP profiles: - -```http -GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp -Authorization: Bearer -``` - -For `managedDsp` profile types, fetch managed DSP accounts: +## Step 1 — Find your advertiser and datasets ```http -GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp&is_manager=true +GET https://service.api.openbridge.io/service/amzadv/list-adv-v2/112 Authorization: Bearer ``` -For managed DSP accounts, the response includes a `dsp_advertiser_id` field. Record this value — it is used as the `advertiser_id` in subsequent steps. For managed DSP, the `profile_id` meta field will be set to `NOT_APPLICABLE`. - -See the [Service API: Amazon Advertising](../api-usage-docs/service-amazon-advertising-api.md) for full endpoint details. +Replace `112` with your remote identity ID. The response is an array of advertiser accounts. Use the selected account's `advertiserAccountId` as the string `advertiser_id`. To look up a known ID, append `?advertiser_id={advertiserAccountId}`. ---- - -## Step 4 — (DSP only) Select an advertiser - -This step applies only to the `dsp` profile type. If your profile type is `ads` or `managedDsp`, skip to Step 5. - -Fetch the advertisers available under your DSP profile: +Before registering datasets, verify that the ID matches a seller or vendor profile's top-level `id` from `GET /service/amzadv/profiles-only/112?profile_types=seller,vendor&is_manager=false`. The Subscriptions API checks this match when you save product `110`. For a manager identity, query with `is_manager=true` and include `"is_manager": "true"` in the pipeline's `product_parameters`. [Amazon Advertising profile lookup](../api-usage-docs/service-amazon-advertising-api.md#list-profiles) ```http -GET https://service.api.openbridge.io/service/amzadv/list-adv/{remote_identity_id}/{profile_id} +GET https://service.api.openbridge.io/service/amzadv/stream-s3/list-datasets Authorization: Bearer ``` -Replace `{remote_identity_id}` with your identity ID and `{profile_id}` with the DSP profile ID from Step 3. - -**Example response:** - ```json -[ - { - "id": "ADV123456", - "type": "AmazonAdvertisingAdvertiser", - "attributes": { - "name": "My DSP Advertiser", - "country": "US", - "currency": "USD", - "timezone": "America/Los_Angeles", - "url": "https://example.com" - } +{ + "type": "AMSDatasets", + "attributes": { + "datasets": ["ads-performance-v1", "ads-converted-product-v1"] } -] -``` - -Record the advertiser `id` (e.g., `ADV123456`) — you will use it as the `advertiser_id` in the SQS payload and subscription meta. - ---- - -## Step 5 — Select marketing stream datasets - -Fetch the available marketing stream datasets for your region and profile type: - -```http -GET https://service.api.openbridge.io/service/amzadv/stream-v2/list-datasets?region={region}&profile_type={profile_type} -Authorization: Bearer -``` - -Replace `{region}` with your identity's region (e.g., `na`) and `{profile_type}` with the type from Step 2 (e.g., `ads` or `dsp`). - -The response returns a list of dataset names. Below are the known datasets and their internal payload names: - -### Sponsored Ads datasets - -| Dataset name | Payload name | -|---|---| -| `sp-traffic` | `amzn_stream2_sp_traffic` | -| `sp-conversion` | `amzn_stream2_sp_conversion` | -| `budget-usage` | `amzn_stream2_budget_usage` | -| `sd-traffic` | `amzn_stream2_sd_traffic` | -| `sd-conversion` | `amzn_stream2_sb_conversion` | -| `sb-traffic` | `amzn_stream2_sb_traffic` | -| `sb-conversion` | `amzn_stream2_sd_conversion` | -| `sb-clickstream` | `amzn_stream2_sb_clickstream` | -| `sb-rich-media` | `amzn_stream2_sb_rich_media` | -| `sp-budget-recommendations` | `amzn_stream2_sp_budget_recommendations` | -| `sponsored-ads-campaign-diagnostics-recommendations` | `amzn_stream2_campaign_recommendations` | -| `campaigns` | `amzn_stream2_campaigns` | -| `adgroups` | `amzn_stream2_adgroups` | -| `ads` | `amzn_stream2_ads` | -| `targets` | `amzn_stream2_targets` | - -### DSP datasets - -| Dataset name | Payload name | -|---|---| -| `adsp-traffic` | `amzn_stream2_adsp_traffic` | -| `adsp-conversion` | `amzn_stream2_adsp_conversion` | -| `adsp-clickstream` | `amzn_stream2_adsp_clickstream` | -| `adsp-rich-media` | `amzn_stream2_adsp_rich_media` | -| `adsp-campaigns` | `amzn_stream2_adsp_campaigns` | -| `adsp-campaign-flights` | `amzn_stream2_adsp_campaign_flights` | -| `adsp-adgroups` | `amzn_stream2_adsp_adgroups` | - -Choose the datasets you want to subscribe to and record them as an array (e.g., `["sp-traffic", "sp-conversion", "budget-usage"]`). You will pass this array in Step 6 and as a stringified JSON array in the subscription meta in Step 7. - ---- - -## Step 6 — Provide an AWS IAM Role ARN and generate SQS queues - -The Marketing Stream product requires an AWS IAM Role ARN that grants Openbridge permission to create and manage SQS queues in your AWS account. Openbridge uses this role to provision dedicated SQS queues for each selected dataset. - -### Create the SQS queues - -``` -POST https://service.api.openbridge.io/service/amzadv/stream-v2/{remote_identity_id} +} ``` -Replace `{remote_identity_id}` with the identity ID from Step 1. - -**Example request (Sponsored Ads):** +## Step 2 — Register the datasets with Amazon ```http -POST https://service.api.openbridge.io/service/amzadv/stream-v2/112 +POST https://service.api.openbridge.io/service/amzadv/stream-s3/112 Authorization: Bearer Content-Type: application/json ``` @@ -304,131 +47,39 @@ Content-Type: application/json "data": { "type": "Service", "attributes": { - "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "datasets": ["sp-traffic", "sp-conversion", "budget-usage"], - "profile_type": "ads", - "profile_id": "4463883966959342" - } - } -} -``` - -**Example request (DSP):** - -```json -{ - "data": { - "type": "Service", - "attributes": { - "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "datasets": ["adsp-traffic", "adsp-conversion"], - "profile_type": "dsp", - "advertiser_id": "ADV123456" + "advertiser_id": "589711999947333868", + "datasets": ["ads-performance-v1"] } } } ``` -| Field | Description | -|---|---| -| `role_arn` | AWS IAM Role ARN granting Openbridge permissions to manage SQS queues | -| `datasets` | Array of dataset names from Step 5 | -| `profile_type` | `"ads"` for Sponsored Ads, `"dsp"` for DSP and Managed DSP | -| `profile_id` | Required when `profile_type` is `"ads"` — the profile ID from Step 3 | -| `advertiser_id` | Required when `profile_type` is `"dsp"` — the advertiser ID from Step 4 (or the `dsp_advertiser_id` for managed DSP) | - -### Async response handling - -This endpoint returns an **asynchronous response**. The initial response includes a `Location` header with a polling URL. Poll this URL until you receive an HTTP `200` response with the queue data. - -- **Poll interval:** 2 seconds -- **Timeout:** up to 13 minutes (the queue provisioning process can take several minutes) - -**Example success response:** +Send at least one supported dataset. The Service API selects the shared S3 bucket and notification queue based on the remote identity's region. Successful registration returns `200 OK` in the same request. ```json { - "data": { - "type": "AmazonAdvertisingStreamV2", - "attributes": { - "queue_urls": { - "sp-traffic": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123", - "sp-conversion": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123", - "budget-usage": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123" - }, - "dataset_params": { - "sp-traffic": {"stream_id": "str_abc123"}, - "sp-conversion": {"stream_id": "str_def456"}, - "budget-usage": {"stream_id": "str_ghi789"} - } + "type": "ServiceAmzAdvBrandMetricsStream", + "advertiser_id": "589711999947333868", + "attributes": { + "amz_subscription_ids": { + "ads-performance-v1": "AMAZON_STREAM_SUBSCRIPTION_ID" } } } ``` -Save the `queue_urls` and `dataset_params` values from the response — they are required for the pipeline subscription meta in Step 7. - ---- +Save the complete map at `attributes.amz_subscription_ids`. Keep each dataset key exactly as returned. Registrations are made one dataset at a time, so a failed multi-dataset request can leave Amazon subscriptions that need cleanup before retrying. -## Step 7 — Create the pipeline subscription +## Step 3 — Create the Openbridge pipeline -With all the pieces gathered from the previous steps, create the pipeline subscription for product `87` (Amazon Advertising Marketing Stream). +Send the advertiser ID and the JSON-stringified dataset-to-subscription-ID map as `product_parameters`. Use the remote identity from Step 1 and your Openbridge storage group. The two required product parameters are `advertiser_id` and `amz_subscription_ids`. The Subscriptions API stores the map as a string, and the Service API reads it back for updates and deletes. [Subscriptions API](../api-usage-docs/subscriptions-api.md) -``` +```http POST https://subscriptions.api.openbridge.io/v2/sub +Authorization: Bearer +Content-Type: application/json ``` -For general detail on subscription creation, see the [Subscription Configuration tutorial](./subscription-configuration.md). - -### product_parameters - -This product requires 8 keys in `product_parameters`. Remote identity is set via the top-level `remote_identity` field (see Step 1) rather than a `product_parameters` key. - -| Key | Source | Description | -|---|---|---| -| `profile_id` | Step 3 | The advertising profile ID. Set to `"NOT_APPLICABLE"` for managed DSP. | -| `profile_type` | Step 2 | `"ads"`, `"dsp"`, or `"managedDsp"` | -| `advertiser_id` | Step 4 | The advertiser ID. Set to `"NOT_APPLICABLE"` for Sponsored Ads (`ads`). | -| `aws_iam_role_arn` | Step 6 | The AWS IAM Role ARN provided in the SQS generation request | -| `queue_urls` | Step 6 | Stringified `queue_urls` map from the SQS generation response (JSON string) | -| `dataset_params` | Step 6 | Stringified `dataset_params` map from the SQS generation response (JSON string) | -| `selected_tables` | Step 5 | Stringified array of selected dataset names (JSON string) | -| `stage_ids` | Step 5 | Stringified array of stage IDs corresponding to the selected datasets (JSON string) | - -> **Note:** The `stage_ids` are resolved by mapping each selected dataset name to its corresponding payload stage via the [Products API payload definitions](../api-usage-docs/products-api.md). Each dataset name maps to a payload name (see the tables in Step 5), and the payload's `stage_id` is used. - -### Full example request (Sponsored Ads) - -```json -{ - "data": { - "type": "Subscription", - "attributes": { - "account": 1, - "user": 1, - "product": 87, - "name": "My Marketing Stream - Sponsored Ads", - "status": "active", - "date_start": "2024-06-01T00:00:00Z", - "remote_identity": 112, - "storage_group": 1, - "product_parameters": { - "profile_id": "4463883966959342", - "profile_type": "ads", - "advertiser_id": "NOT_APPLICABLE", - "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "queue_urls": "{\"sp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123\",\"sp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123\",\"budget_usage\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123\"}", - "dataset_params": "{\"sp-traffic\":{\"stream_id\":\"str_abc123\"},\"sp-conversion\":{\"stream_id\":\"str_def456\"},\"budget-usage\":{\"stream_id\":\"str_ghi789\"}}", - "selected_tables": "[\"sp-traffic\",\"sp-conversion\",\"budget-usage\"]", - "stage_ids": "[101,102,103]" - } - } - } -} -``` - -### Full example request (DSP) - ```json { "data": { @@ -436,73 +87,61 @@ This product requires 8 keys in `product_parameters`. Remote identity is set via "attributes": { "account": 1, "user": 1, - "product": 87, - "name": "My Marketing Stream - DSP", + "product": 110, + "name": "My Marketing Stream", "status": "active", - "date_start": "2024-06-01T00:00:00Z", + "date_start": "2026-11-18T00:00:00Z", "remote_identity": 112, "storage_group": 1, "product_parameters": { - "profile_id": "4463883966959342", - "profile_type": "dsp", - "advertiser_id": "ADV123456", - "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "queue_urls": "{\"adsp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-traffic-abc123\",\"adsp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-conversion-abc123\"}", - "dataset_params": "{\"adsp-traffic\":{\"stream_id\":\"str_xyz123\"},\"adsp-conversion\":{\"stream_id\":\"str_xyz456\"}}", - "selected_tables": "[\"adsp-traffic\",\"adsp-conversion\"]", - "stage_ids": "[201,202]" + "advertiser_id": "589711999947333868", + "amz_subscription_ids": "{\"ads-performance-v1\":\"AMAZON_STREAM_SUBSCRIPTION_ID\"}" } } } } ``` -Replace the placeholder values (`account`, `user`, `remote_identity`, `storage_group`, and the `product_parameters` values) with values from your account and the previous steps. - ---- - -## Updating a marketing stream subscription - -To update an existing marketing stream subscription (e.g., to add or remove datasets), use two requests: - -**1. Update the SQS queues via Service API:** - -``` -PATCH https://service.api.openbridge.io/service/amzadv/stream-v2/update/{remote_identity_id}/{subscription_id} -``` - -The request body is the same structure as the create endpoint — include the `role_arn`, updated `datasets`, `profile_type`, and `profile_id` or `advertiser_id`. This is an async operation with the same polling behavior as the create endpoint. +Replace the example IDs and start date with your values. Save the Openbridge subscription ID from the `201 Created` response. -**2. Update the pipeline subscription:** +## Update the dataset selection +```http +PATCH https://service.api.openbridge.io/service/amzadv/stream-s3/update/{remote_identity_id}/{subscription_id} +Authorization: Bearer +Content-Type: application/json ``` -PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} -``` - -Update the `queue_urls`, `dataset_params`, `selected_tables`, `stage_ids`, and any other changed keys in `product_parameters` with the new values from the update response (partial merge — omitted keys keep their stored value). - -See the [Subscriptions API (v2)](../api-usage-docs/subscriptions-api.md) for full PATCH documentation. ---- +Use the Openbridge subscription ID and send the same body as in Step 2 with the existing advertiser ID and the **full desired dataset list**. Check the list against the supported IDs before sending it. The service archives every Amazon subscription in the stored map before registering the requested datasets, including datasets you keep. This can interrupt delivery, and a failed update can leave the previous streams archived. -## Deleting a marketing stream subscription +After `200 OK`, send `PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id}` with `data.type` set to `Subscription`, `data.id` set to the Openbridge subscription ID, and the new JSON-stringified map in `data.attributes.product_parameters.amz_subscription_ids`. Replace the entire map so subsequent updates and deletes use the current Amazon IDs. -To delete a marketing stream pipeline, mark the pipeline subscription as invalid: +## Delete the pipeline +```http +DELETE https://service.api.openbridge.io/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id} +Authorization: Bearer ``` -PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} -``` + +The Service API archives the Amazon streams using the pipeline's stored metadata. After `204 No Content`, mark the Openbridge subscription invalid with `PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id}` and this body. [Subscription status management](./subscription-configuration.md#deleting-a-subscription) ```json { "data": { "type": "Subscription", "id": "12345", - "attributes": { - "status": "invalid" - } + "attributes": {"status": "invalid"} } } ``` -See the [Subscription Configuration tutorial](./subscription-configuration.md#deleting-a-subscription) for more detail on subscription status management. +## Clean up orphaned Amazon streams + +Use the cleanup endpoint if a registration left Amazon streams without an active Openbridge pipeline. + +```http +DELETE https://service.api.openbridge.io/service/amzadv/stream-s3/delete-orphaned/{remote_identity_id}/{advertiser_id} +Authorization: Bearer +``` + +This endpoint archives **all** streams returned for the advertiser except those already archived or suspended. It returns `204 No Content` on success and `400 Bad Request` if any active product `110` subscription exists for the remote identity.