From d0c724b308f750b6dec0410d2f9fe550543d5a4b Mon Sep 17 00:00:00 2001 From: Devin Roberts Date: Tue, 13 Oct 2026 12:32:11 -0400 Subject: [PATCH 1/6] add AMS S3 docs --- .../service-amazon-advertising-api.md | 20 + ...tising-marketing-stream-pipeline-legacy.md | 510 ++++++++++++++++++ ...n-advertising-marketing-stream-pipeline.md | 484 ++++------------- 3 files changed, 633 insertions(+), 381 deletions(-) create mode 100644 tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md diff --git a/api-usage-docs/service-amazon-advertising-api.md b/api-usage-docs/service-amazon-advertising-api.md index a592e41..75f88ba 100644 --- a/api-usage-docs/service-amazon-advertising-api.md +++ b/api-usage-docs/service-amazon-advertising-api.md @@ -242,3 +242,23 @@ Authorization: Bearer } } ``` + +--- + +### Marketing Stream v2 + +Marketing Stream v2 uses `/service/amzadv/stream-s3` and Openbridge-managed delivery resources. No customer AWS account, IAM role, or CloudFormation template is required. Follow the [Marketing Stream v2 pipeline tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline.md) for requests, async response handling, and subscription metadata. + +| Method | Path | Purpose | +|---|---|---| +| GET | `/service/amzadv/list-adv-v2/{remote_identity_id}` | List advertiser accounts; optional `advertiser_id` filter | +| POST | `/service/amzadv/stream-s3/{remote_identity_id}` | Provision resources and register selected datasets | +| GET | `/service/amzadv/stream/poll-async/{request_id}` | Poll the `Location` returned by create or update | +| PATCH | `/service/amzadv/stream-s3/update/{remote_identity_id}/{co_id}/{subscription_id}` | Internal update worker; requires an existing ConsoleOps request | +| DELETE | `/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id}` | Archive registered Amazon streams | + +Create and the update handler accept `data.attributes.advertiser_id` and a nonempty `data.attributes.datasets` array containing `ads-performance-v1` and/or `ads-converted-product-v1`. There is currently no routed managed S3 dataset-list endpoint. + +For the older `/stream-v2` customer-managed AWS flow, see the [legacy tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md). + +The customer-facing async update handler is currently not routed. The update worker listed above is not a customer entry point; see the tutorial for this limitation. diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md b/tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md new file mode 100644 index 0000000..274e082 --- /dev/null +++ b/tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md @@ -0,0 +1,510 @@ +# Amazon Advertising Marketing Stream Pipeline (Legacy) + +> **Legacy:** This guide covers the customer-managed AWS flow using `/stream-v2` and product `87`. For new pipelines, see [Marketing Stream v2](./amazon-advertising-marketing-stream-pipeline.md), which uses `/stream-s3` and requires no customer AWS account or CloudFormation template. + +## Table of Contents + +- [Overview](#overview) +- [Prerequisites](#prerequisites) +- [Step 1 — Choose or create a remote identity](#step-1--choose-or-create-a-remote-identity) +- [Step 2 — Select a profile type](#step-2--select-a-profile-type) +- [Step 3 — Select a profile](#step-3--select-a-profile) + - [Sponsored Ads profiles (seller/vendor)](#sponsored-ads-profiles-sellervendor) + - [DSP profiles](#dsp-profiles) +- [Step 4 — (DSP only) Select an advertiser](#step-4--dsp-only-select-an-advertiser) +- [Step 5 — Select marketing stream datasets](#step-5--select-marketing-stream-datasets) +- [Step 6 — Provide an AWS IAM Role ARN and generate SQS queues](#step-6--provide-an-aws-iam-role-arn-and-generate-sqs-queues) +- [Step 7 — Create the pipeline subscription](#step-7--create-the-pipeline-subscription) + - [product_parameters](#product_parameters) + - [Full example request (Sponsored Ads)](#full-example-request-sponsored-ads) + - [Full example request (DSP)](#full-example-request-dsp) +- [Updating a marketing stream subscription](#updating-a-marketing-stream-subscription) +- [Deleting a marketing stream subscription](#deleting-a-marketing-stream-subscription) + +--- + +## Overview + +This tutorial walks through creating an Amazon Advertising Marketing Stream pipeline subscription end-to-end. The Marketing Stream product (product ID `87`) delivers near-real-time advertising performance data via Amazon's Marketing Stream API into SQS queues provisioned by Openbridge. + +Three profile-type flows are supported: + +| Profile type | Description | +|---|---| +| **Sponsored Ads** (`ads`) | Seller or Vendor accounts running Sponsored Products, Sponsored Brands, or Sponsored Display campaigns | +| **DSP** (`dsp`) | Demand-Side Platform accounts | +| **Managed DSP** (`managedDsp`) | DSP manager accounts acting on behalf of advertisers | + +The flow mirrors the Openbridge website wizard: choose an identity, select a profile type, pick a profile (and advertiser for DSP), choose datasets, provide an AWS IAM Role ARN to provision SQS queues, and finally create the pipeline subscription. + +--- + +## Prerequisites + +- A JWT access token — see [Authentication API](../api-usage-docs/authentication-api.md) +- Your account ID and user ID — see [Identity Configuration Step 1](./identity-configuration.md#step-1--look-up-your-account-and-user-ids) +- An active Amazon Advertising account +- An AWS IAM Role ARN with permissions for Openbridge to provision SQS queues on your behalf +- A storage destination already configured — see [Subscription Configuration Step 3](./subscription-configuration.md#step-3--identify-your-storage-destination) + +--- + +## Step 1 — Choose or create a remote identity + +The Marketing Stream product requires an **Amazon Advertising** identity (remote identity type `14`). + +### Create a new identity + +If you do not have an identity yet, follow the [Identity Configuration tutorial](./identity-configuration.md) to create one via the OAuth flow. Amazon Advertising identities are region-specific — use the correct region for your marketplace: + +| Region identifier | Region name | +|---|---| +| `na` | North America | +| `eu` | Europe | +| `fe` | Far East | + +### List existing identities + +To find an existing Amazon Advertising identity, list remote identities filtered by type: + +```http +GET https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0 +Authorization: Bearer +``` + +**Example response:** + +```json +{ + "links": { + "first": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", + "last": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", + "next": "", + "prev": "" + }, + "data": [ + { + "type": "RemoteIdentity", + "id": "112", + "attributes": { + "name": "My Advertising Account", + "created_at": "2024-01-15T12:00:00", + "modified_at": "2024-06-01T08:30:00", + "remote_unique_id": "amzn1.account.AEHZ5", + "account_id": 1, + "user_id": 1, + "invalid_identity": 0, + "region": "na", + "email": "advertiser@example.com" + } + } + ], + "meta": { + "pagination": { + "page": 1, + "pages": 1, + "count": 1 + } + } +} +``` + +Record the identity `id` (e.g., `112`) and `region` (e.g., `na`) — you will need both in subsequent steps. + +See the [Remote Identity API](../api-usage-docs/remote-identity-api.md) for full endpoint details. + +--- + +## Step 2 — Select a profile type + +Before fetching profiles, decide which profile type applies to your use case: + +| Profile type value | Use case | +|---|---| +| `ads` | Seller or Vendor accounts (Sponsored Products, Sponsored Brands, Sponsored Display) | +| `dsp` | DSP advertiser accounts | +| `managedDsp` | DSP manager accounts operating on behalf of advertisers | + +This choice determines the wizard flow, the available datasets, and how the SQS queue payload is constructed. Record your `profile_type` — you will use it in Steps 3, 5, and 6. + +--- + +## Step 3 — Select a profile + +### Sponsored Ads profiles (seller/vendor) + +For `ads` profile types, fetch seller and vendor profiles: + +```http +GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=seller,vendor +Authorization: Bearer +``` + +Replace `{remote_identity_id}` with the identity ID from Step 1. + +**Example response:** + +```json +[ + { + "id": 4463883966959342, + "type": "AmazonAdvertisingProfile", + "attributes": { + "country_code": "US", + "currency_code": "USD", + "daily_budget": 10.0, + "timezone": "America/Los_Angeles", + "account_info": { + "id": "ENTITY1234567890", + "type": "AmazonAdvertisingProfileAccountInfo", + "attributes": { + "marketplace_country": "US", + "marketplace_string_id": "ATVPDKIKX0DER", + "name": "My Seller Account", + "type": "seller", + "subType": "", + "valid_payment_method": true + } + } + } + } +] +``` + +Record the profile `id` (e.g., `4463883966959342`). For Sponsored Ads, the `advertiser_id` meta field will be set to `NOT_APPLICABLE`. + +> **Note:** Profiles with `subType` of `AMAZON_ATTRIBUTION` are excluded and cannot be used with Marketing Stream. + +### DSP profiles + +For `dsp` profile types, fetch DSP profiles: + +```http +GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp +Authorization: Bearer +``` + +For `managedDsp` profile types, fetch managed DSP accounts: + +```http +GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp&is_manager=true +Authorization: Bearer +``` + +For managed DSP accounts, the response includes a `dsp_advertiser_id` field. Record this value — it is used as the `advertiser_id` in subsequent steps. For managed DSP, the `profile_id` meta field will be set to `NOT_APPLICABLE`. + +See the [Service API: Amazon Advertising](../api-usage-docs/service-amazon-advertising-api.md) for full endpoint details. + +--- + +## Step 4 — (DSP only) Select an advertiser + +This step applies only to the `dsp` profile type. If your profile type is `ads` or `managedDsp`, skip to Step 5. + +Fetch the advertisers available under your DSP profile: + +```http +GET https://service.api.openbridge.io/service/amzadv/list-adv/{remote_identity_id}/{profile_id} +Authorization: Bearer +``` + +Replace `{remote_identity_id}` with your identity ID and `{profile_id}` with the DSP profile ID from Step 3. + +**Example response:** + +```json +[ + { + "id": "ADV123456", + "type": "AmazonAdvertisingAdvertiser", + "attributes": { + "name": "My DSP Advertiser", + "country": "US", + "currency": "USD", + "timezone": "America/Los_Angeles", + "url": "https://example.com" + } + } +] +``` + +Record the advertiser `id` (e.g., `ADV123456`) — you will use it as the `advertiser_id` in the SQS payload and subscription meta. + +--- + +## Step 5 — Select marketing stream datasets + +Fetch the available marketing stream datasets for your region and profile type: + +```http +GET https://service.api.openbridge.io/service/amzadv/stream-v2/list-datasets?region={region}&profile_type={profile_type} +Authorization: Bearer +``` + +Replace `{region}` with your identity's region (e.g., `na`) and `{profile_type}` with the type from Step 2 (e.g., `ads` or `dsp`). + +The response returns a list of dataset names. Below are the known datasets and their internal payload names: + +### Sponsored Ads datasets + +| Dataset name | Payload name | +|---|---| +| `sp-traffic` | `amzn_stream2_sp_traffic` | +| `sp-conversion` | `amzn_stream2_sp_conversion` | +| `budget-usage` | `amzn_stream2_budget_usage` | +| `sd-traffic` | `amzn_stream2_sd_traffic` | +| `sd-conversion` | `amzn_stream2_sb_conversion` | +| `sb-traffic` | `amzn_stream2_sb_traffic` | +| `sb-conversion` | `amzn_stream2_sd_conversion` | +| `sb-clickstream` | `amzn_stream2_sb_clickstream` | +| `sb-rich-media` | `amzn_stream2_sb_rich_media` | +| `sp-budget-recommendations` | `amzn_stream2_sp_budget_recommendations` | +| `sponsored-ads-campaign-diagnostics-recommendations` | `amzn_stream2_campaign_recommendations` | +| `campaigns` | `amzn_stream2_campaigns` | +| `adgroups` | `amzn_stream2_adgroups` | +| `ads` | `amzn_stream2_ads` | +| `targets` | `amzn_stream2_targets` | + +### DSP datasets + +| Dataset name | Payload name | +|---|---| +| `adsp-traffic` | `amzn_stream2_adsp_traffic` | +| `adsp-conversion` | `amzn_stream2_adsp_conversion` | +| `adsp-clickstream` | `amzn_stream2_adsp_clickstream` | +| `adsp-rich-media` | `amzn_stream2_adsp_rich_media` | +| `adsp-campaigns` | `amzn_stream2_adsp_campaigns` | +| `adsp-campaign-flights` | `amzn_stream2_adsp_campaign_flights` | +| `adsp-adgroups` | `amzn_stream2_adsp_adgroups` | + +Choose the datasets you want to subscribe to and record them as an array (e.g., `["sp-traffic", "sp-conversion", "budget-usage"]`). You will pass this array in Step 6 and as a stringified JSON array in the subscription meta in Step 7. + +--- + +## Step 6 — Provide an AWS IAM Role ARN and generate SQS queues + +The Marketing Stream product requires an AWS IAM Role ARN that grants Openbridge permission to create and manage SQS queues in your AWS account. Openbridge uses this role to provision dedicated SQS queues for each selected dataset. + +### Create the SQS queues + +``` +POST https://service.api.openbridge.io/service/amzadv/stream-v2/{remote_identity_id} +``` + +Replace `{remote_identity_id}` with the identity ID from Step 1. + +**Example request (Sponsored Ads):** + +```http +POST https://service.api.openbridge.io/service/amzadv/stream-v2/112 +Authorization: Bearer +Content-Type: application/json +``` + +```json +{ + "data": { + "type": "Service", + "attributes": { + "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "datasets": ["sp-traffic", "sp-conversion", "budget-usage"], + "profile_type": "ads", + "profile_id": "4463883966959342" + } + } +} +``` + +**Example request (DSP):** + +```json +{ + "data": { + "type": "Service", + "attributes": { + "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "datasets": ["adsp-traffic", "adsp-conversion"], + "profile_type": "dsp", + "advertiser_id": "ADV123456" + } + } +} +``` + +| Field | Description | +|---|---| +| `role_arn` | AWS IAM Role ARN granting Openbridge permissions to manage SQS queues | +| `datasets` | Array of dataset names from Step 5 | +| `profile_type` | `"ads"` for Sponsored Ads, `"dsp"` for DSP and Managed DSP | +| `profile_id` | Required when `profile_type` is `"ads"` — the profile ID from Step 3 | +| `advertiser_id` | Required when `profile_type` is `"dsp"` — the advertiser ID from Step 4 (or the `dsp_advertiser_id` for managed DSP) | + +### Async response handling + +This endpoint returns an **asynchronous response**. The initial response includes a `Location` header with a polling URL. Poll this URL until you receive an HTTP `200` response with the queue data. + +- **Poll interval:** 2 seconds +- **Timeout:** up to 13 minutes (the queue provisioning process can take several minutes) + +**Example success response:** + +```json +{ + "data": { + "type": "AmazonAdvertisingStreamV2", + "attributes": { + "queue_urls": { + "sp-traffic": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123", + "sp-conversion": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123", + "budget-usage": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123" + }, + "dataset_params": { + "sp-traffic": {"stream_id": "str_abc123"}, + "sp-conversion": {"stream_id": "str_def456"}, + "budget-usage": {"stream_id": "str_ghi789"} + } + } + } +} +``` + +Save the `queue_urls` and `dataset_params` values from the response — they are required for the pipeline subscription meta in Step 7. + +--- + +## Step 7 — Create the pipeline subscription + +With all the pieces gathered from the previous steps, create the pipeline subscription for product `87` (Amazon Advertising Marketing Stream). + +``` +POST https://subscriptions.api.openbridge.io/v2/sub +``` + +For general detail on subscription creation, see the [Subscription Configuration tutorial](./subscription-configuration.md). + +### product_parameters + +This product requires 8 keys in `product_parameters`. Remote identity is set via the top-level `remote_identity` field (see Step 1) rather than a `product_parameters` key. + +| Key | Source | Description | +|---|---|---| +| `profile_id` | Step 3 | The advertising profile ID. Set to `"NOT_APPLICABLE"` for managed DSP. | +| `profile_type` | Step 2 | `"ads"`, `"dsp"`, or `"managedDsp"` | +| `advertiser_id` | Step 4 | The advertiser ID. Set to `"NOT_APPLICABLE"` for Sponsored Ads (`ads`). | +| `aws_iam_role_arn` | Step 6 | The AWS IAM Role ARN provided in the SQS generation request | +| `queue_urls` | Step 6 | Stringified `queue_urls` map from the SQS generation response (JSON string) | +| `dataset_params` | Step 6 | Stringified `dataset_params` map from the SQS generation response (JSON string) | +| `selected_tables` | Step 5 | Stringified array of selected dataset names (JSON string) | +| `stage_ids` | Step 5 | Stringified array of stage IDs corresponding to the selected datasets (JSON string) | + +> **Note:** The `stage_ids` are resolved by mapping each selected dataset name to its corresponding payload stage via the [Products API payload definitions](../api-usage-docs/products-api.md). Each dataset name maps to a payload name (see the tables in Step 5), and the payload's `stage_id` is used. + +### Full example request (Sponsored Ads) + +```json +{ + "data": { + "type": "Subscription", + "attributes": { + "account": 1, + "user": 1, + "product": 87, + "name": "My Marketing Stream - Sponsored Ads", + "status": "active", + "date_start": "2024-06-01T00:00:00Z", + "remote_identity": 112, + "storage_group": 1, + "product_parameters": { + "profile_id": "4463883966959342", + "profile_type": "ads", + "advertiser_id": "NOT_APPLICABLE", + "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "queue_urls": "{\"sp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123\",\"sp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123\",\"budget_usage\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123\"}", + "dataset_params": "{\"sp-traffic\":{\"stream_id\":\"str_abc123\"},\"sp-conversion\":{\"stream_id\":\"str_def456\"},\"budget-usage\":{\"stream_id\":\"str_ghi789\"}}", + "selected_tables": "[\"sp-traffic\",\"sp-conversion\",\"budget-usage\"]", + "stage_ids": "[101,102,103]" + } + } + } +} +``` + +### Full example request (DSP) + +```json +{ + "data": { + "type": "Subscription", + "attributes": { + "account": 1, + "user": 1, + "product": 87, + "name": "My Marketing Stream - DSP", + "status": "active", + "date_start": "2024-06-01T00:00:00Z", + "remote_identity": 112, + "storage_group": 1, + "product_parameters": { + "profile_id": "4463883966959342", + "profile_type": "dsp", + "advertiser_id": "ADV123456", + "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", + "queue_urls": "{\"adsp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-traffic-abc123\",\"adsp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-conversion-abc123\"}", + "dataset_params": "{\"adsp-traffic\":{\"stream_id\":\"str_xyz123\"},\"adsp-conversion\":{\"stream_id\":\"str_xyz456\"}}", + "selected_tables": "[\"adsp-traffic\",\"adsp-conversion\"]", + "stage_ids": "[201,202]" + } + } + } +} +``` + +Replace the placeholder values (`account`, `user`, `remote_identity`, `storage_group`, and the `product_parameters` values) with values from your account and the previous steps. + +--- + +## Updating a marketing stream subscription + +To update an existing marketing stream subscription (e.g., to add or remove datasets), use two requests: + +**1. Update the SQS queues via Service API:** + +``` +PATCH https://service.api.openbridge.io/service/amzadv/stream-v2/update/{remote_identity_id}/{subscription_id} +``` + +The request body is the same structure as the create endpoint — include the `role_arn`, updated `datasets`, `profile_type`, and `profile_id` or `advertiser_id`. This is an async operation with the same polling behavior as the create endpoint. + +**2. Update the pipeline subscription:** + +``` +PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +``` + +Update the `queue_urls`, `dataset_params`, `selected_tables`, `stage_ids`, and any other changed keys in `product_parameters` with the new values from the update response (partial merge — omitted keys keep their stored value). + +See the [Subscriptions API (v2)](../api-usage-docs/subscriptions-api.md) for full PATCH documentation. + +--- + +## Deleting a marketing stream subscription + +To delete a marketing stream pipeline, mark the pipeline subscription as invalid: + +``` +PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +``` + +```json +{ + "data": { + "type": "Subscription", + "id": "12345", + "attributes": { + "status": "invalid" + } + } +} +``` + +See the [Subscription Configuration tutorial](./subscription-configuration.md#deleting-a-subscription) for more detail on subscription status management. diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index 167220f..99242d2 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -1,300 +1,71 @@ -# Amazon Advertising Marketing Stream Pipeline +# Amazon Advertising Marketing Stream S3 Pipeline -## Table of Contents +Marketing Stream S3 uses **`/service/amzadv/stream-s3`** in the Service API. The managed S3 delivery flow uses this endpoint prefix; `/stream-v2` belongs to the [legacy tutorial](./amazon-advertising-marketing-stream-pipeline-legacy.md). -- [Overview](#overview) -- [Prerequisites](#prerequisites) -- [Step 1 — Choose or create a remote identity](#step-1--choose-or-create-a-remote-identity) -- [Step 2 — Select a profile type](#step-2--select-a-profile-type) -- [Step 3 — Select a profile](#step-3--select-a-profile) - - [Sponsored Ads profiles (seller/vendor)](#sponsored-ads-profiles-sellervendor) - - [DSP profiles](#dsp-profiles) -- [Step 4 — (DSP only) Select an advertiser](#step-4--dsp-only-select-an-advertiser) -- [Step 5 — Select marketing stream datasets](#step-5--select-marketing-stream-datasets) -- [Step 6 — Provide an AWS IAM Role ARN and generate SQS queues](#step-6--provide-an-aws-iam-role-arn-and-generate-sqs-queues) -- [Step 7 — Create the pipeline subscription](#step-7--create-the-pipeline-subscription) - - [product_parameters](#product_parameters) - - [Full example request (Sponsored Ads)](#full-example-request-sponsored-ads) - - [Full example request (DSP)](#full-example-request-dsp) -- [Updating a marketing stream subscription](#updating-a-marketing-stream-subscription) -- [Deleting a marketing stream subscription](#deleting-a-marketing-stream-subscription) - ---- - -## Overview - -This tutorial walks through creating an Amazon Advertising Marketing Stream pipeline subscription end-to-end. The Marketing Stream product (product ID `87`) delivers near-real-time advertising performance data via Amazon's Marketing Stream API into SQS queues provisioned by Openbridge. - -Three profile-type flows are supported: - -| Profile type | Description | -|---|---| -| **Sponsored Ads** (`ads`) | Seller or Vendor accounts running Sponsored Products, Sponsored Brands, or Sponsored Display campaigns | -| **DSP** (`dsp`) | Demand-Side Platform accounts | -| **Managed DSP** (`managedDsp`) | DSP manager accounts acting on behalf of advertisers | - -The flow mirrors the Openbridge website wizard: choose an identity, select a profile type, pick a profile (and advertiser for DSP), choose datasets, provide an AWS IAM Role ARN to provision SQS queues, and finally create the pipeline subscription. - ---- +Amazon delivers dataset files to S3, with queue notifications used by Openbridge to process the files into your configured storage destination. Openbridge provisions and manages these delivery resources in its own AWS account. **Customers do not need an AWS account, an IAM role ARN, or a CloudFormation template.** ## Prerequisites -- A JWT access token — see [Authentication API](../api-usage-docs/authentication-api.md) -- Your account ID and user ID — see [Identity Configuration Step 1](./identity-configuration.md#step-1--look-up-your-account-and-user-ids) -- An active Amazon Advertising account -- An AWS IAM Role ARN with permissions for Openbridge to provision SQS queues on your behalf -- A storage destination already configured — see [Subscription Configuration Step 3](./subscription-configuration.md#step-3--identify-your-storage-destination) - ---- +- A [JWT access token](../api-usage-docs/authentication-api.md). +- Your [account and user IDs](./identity-configuration.md#step-1--look-up-your-account-and-user-ids). +- An Amazon Advertising remote identity (type `14`) with access to the advertiser account. +- A configured [storage destination](./subscription-configuration.md#step-3--identify-your-storage-destination). ## Step 1 — Choose or create a remote identity -The Marketing Stream product requires an **Amazon Advertising** identity (remote identity type `14`). - -### Create a new identity - -If you do not have an identity yet, follow the [Identity Configuration tutorial](./identity-configuration.md) to create one via the OAuth flow. Amazon Advertising identities are region-specific — use the correct region for your marketplace: - -| Region identifier | Region name | -|---|---| -| `na` | North America | -| `eu` | Europe | -| `fe` | Far East | - -### List existing identities - -To find an existing Amazon Advertising identity, list remote identities filtered by type: +Follow the [Identity Configuration tutorial](./identity-configuration.md) to connect an Amazon Advertising account through OAuth, or list existing valid identities: ```http GET https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0 Authorization: Bearer ``` -**Example response:** - -```json -{ - "links": { - "first": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", - "last": "https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0&page=1", - "next": "", - "prev": "" - }, - "data": [ - { - "type": "RemoteIdentity", - "id": "112", - "attributes": { - "name": "My Advertising Account", - "created_at": "2024-01-15T12:00:00", - "modified_at": "2024-06-01T08:30:00", - "remote_unique_id": "amzn1.account.AEHZ5", - "account_id": 1, - "user_id": 1, - "invalid_identity": 0, - "region": "na", - "email": "advertiser@example.com" - } - } - ], - "meta": { - "pagination": { - "page": 1, - "pages": 1, - "count": 1 - } - } -} -``` - -Record the identity `id` (e.g., `112`) and `region` (e.g., `na`) — you will need both in subsequent steps. - -See the [Remote Identity API](../api-usage-docs/remote-identity-api.md) for full endpoint details. - ---- - -## Step 2 — Select a profile type - -Before fetching profiles, decide which profile type applies to your use case: - -| Profile type value | Use case | -|---|---| -| `ads` | Seller or Vendor accounts (Sponsored Products, Sponsored Brands, Sponsored Display) | -| `dsp` | DSP advertiser accounts | -| `managedDsp` | DSP manager accounts operating on behalf of advertisers | - -This choice determines the wizard flow, the available datasets, and how the SQS queue payload is constructed. Record your `profile_type` — you will use it in Steps 3, 5, and 6. - ---- - -## Step 3 — Select a profile - -### Sponsored Ads profiles (seller/vendor) - -For `ads` profile types, fetch seller and vendor profiles: - -```http -GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=seller,vendor -Authorization: Bearer -``` - -Replace `{remote_identity_id}` with the identity ID from Step 1. - -**Example response:** - -```json -[ - { - "id": 4463883966959342, - "type": "AmazonAdvertisingProfile", - "attributes": { - "country_code": "US", - "currency_code": "USD", - "daily_budget": 10.0, - "timezone": "America/Los_Angeles", - "account_info": { - "id": "ENTITY1234567890", - "type": "AmazonAdvertisingProfileAccountInfo", - "attributes": { - "marketplace_country": "US", - "marketplace_string_id": "ATVPDKIKX0DER", - "name": "My Seller Account", - "type": "seller", - "subType": "", - "valid_payment_method": true - } - } - } - } -] -``` - -Record the profile `id` (e.g., `4463883966959342`). For Sponsored Ads, the `advertiser_id` meta field will be set to `NOT_APPLICABLE`. - -> **Note:** Profiles with `subType` of `AMAZON_ATTRIBUTION` are excluded and cannot be used with Marketing Stream. +Record the identity's `id`. The service uses its `region` (`na`, `eu`, or `fe`) to select the Amazon API and AWS delivery region; you do not send a region in the create request. -### DSP profiles +## Step 2 — Select an advertiser account -For `dsp` profile types, fetch DSP profiles: +Use the new advertiser-account lookup, which does not require a profile ID: ```http -GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp +GET https://service.api.openbridge.io/service/amzadv/list-adv-v2/112 Authorization: Bearer ``` -For `managedDsp` profile types, fetch managed DSP accounts: +Replace `112` with your remote identity ID. The service returns a flat array of Amazon `advertiserAccounts` records and collects all pages from Amazon. Record the selected record's `advertiserAccountId` as `advertiser_id`. To filter the lookup, append `?advertiser_id={advertiserAccountId}`. -```http -GET https://service.api.openbridge.io/service/amzadv/profiles-only/{remote_identity_id}?profile_types=dsp&is_manager=true -Authorization: Bearer -``` - -For managed DSP accounts, the response includes a `dsp_advertiser_id` field. Record this value — it is used as the `advertiser_id` in subsequent steps. For managed DSP, the `profile_id` meta field will be set to `NOT_APPLICABLE`. +This flow uses advertiser account IDs for all requests. The legacy profile-type selection (`ads`, `dsp`, `managedDsp`), profile lookup, and profile-scoped advertiser lookup are no longer setup steps. -See the [Service API: Amazon Advertising](../api-usage-docs/service-amazon-advertising-api.md) for full endpoint details. +## Step 3 — Select datasets and find the product ---- +The new service accepts these dataset IDs: -## Step 4 — (DSP only) Select an advertiser +| Dataset ID | +|---| +| `ads-performance-v1` | +| `ads-converted-product-v1` | -This step applies only to the `dsp` profile type. If your profile type is `ads` or `managedDsp`, skip to Step 5. +Choose a nonempty array, such as `["ads-performance-v1", "ads-converted-product-v1"]`. The reviewed service code has a dataset-list method but no `/stream-s3/list-datasets` route; the legacy `/stream-v2/list-datasets` returns datasets for the old flow. -Fetch the advertisers available under your DSP profile: +Find the Marketing Stream v2 product in the [Products API](../api-usage-docs/products-api.md): ```http -GET https://service.api.openbridge.io/service/amzadv/list-adv/{remote_identity_id}/{profile_id} +GET https://subscriptions.api.openbridge.io/product?remote_identity_type=14&name__icontains=Marketing Authorization: Bearer ``` -Replace `{remote_identity_id}` with your identity ID and `{profile_id}` with the DSP profile ID from Step 3. - -**Example response:** - -```json -[ - { - "id": "ADV123456", - "type": "AmazonAdvertisingAdvertiser", - "attributes": { - "name": "My DSP Advertiser", - "country": "US", - "currency": "USD", - "timezone": "America/Los_Angeles", - "url": "https://example.com" - } - } -] -``` - -Record the advertiser `id` (e.g., `ADV123456`) — you will use it as the `advertiser_id` in the SQS payload and subscription meta. - ---- - -## Step 5 — Select marketing stream datasets - -Fetch the available marketing stream datasets for your region and profile type: +Follow pagination and confirm the product's name and `required_meta_fields`. Record its numeric `id` and retrieve its payload definitions: ```http -GET https://service.api.openbridge.io/service/amzadv/stream-v2/list-datasets?region={region}&profile_type={profile_type} +GET https://service.api.openbridge.io/service/products/product/{product_id}/payloads?stage_id__gte=1000 Authorization: Bearer ``` -Replace `{region}` with your identity's region (e.g., `na`) and `{profile_type}` with the type from Step 2 (e.g., `ads` or `dsp`). - -The response returns a list of dataset names. Below are the known datasets and their internal payload names: - -### Sponsored Ads datasets - -| Dataset name | Payload name | -|---|---| -| `sp-traffic` | `amzn_stream2_sp_traffic` | -| `sp-conversion` | `amzn_stream2_sp_conversion` | -| `budget-usage` | `amzn_stream2_budget_usage` | -| `sd-traffic` | `amzn_stream2_sd_traffic` | -| `sd-conversion` | `amzn_stream2_sb_conversion` | -| `sb-traffic` | `amzn_stream2_sb_traffic` | -| `sb-conversion` | `amzn_stream2_sd_conversion` | -| `sb-clickstream` | `amzn_stream2_sb_clickstream` | -| `sb-rich-media` | `amzn_stream2_sb_rich_media` | -| `sp-budget-recommendations` | `amzn_stream2_sp_budget_recommendations` | -| `sponsored-ads-campaign-diagnostics-recommendations` | `amzn_stream2_campaign_recommendations` | -| `campaigns` | `amzn_stream2_campaigns` | -| `adgroups` | `amzn_stream2_adgroups` | -| `ads` | `amzn_stream2_ads` | -| `targets` | `amzn_stream2_targets` | - -### DSP datasets - -| Dataset name | Payload name | -|---|---| -| `adsp-traffic` | `amzn_stream2_adsp_traffic` | -| `adsp-conversion` | `amzn_stream2_adsp_conversion` | -| `adsp-clickstream` | `amzn_stream2_adsp_clickstream` | -| `adsp-rich-media` | `amzn_stream2_adsp_rich_media` | -| `adsp-campaigns` | `amzn_stream2_adsp_campaigns` | -| `adsp-campaign-flights` | `amzn_stream2_adsp_campaign_flights` | -| `adsp-adgroups` | `amzn_stream2_adsp_adgroups` | - -Choose the datasets you want to subscribe to and record them as an array (e.g., `["sp-traffic", "sp-conversion", "budget-usage"]`). You will pass this array in Step 6 and as a stringified JSON array in the subscription meta in Step 7. - ---- - -## Step 6 — Provide an AWS IAM Role ARN and generate SQS queues - -The Marketing Stream product requires an AWS IAM Role ARN that grants Openbridge permission to create and manage SQS queues in your AWS account. Openbridge uses this role to provision dedicated SQS queues for each selected dataset. - -### Create the SQS queues - -``` -POST https://service.api.openbridge.io/service/amzadv/stream-v2/{remote_identity_id} -``` - -Replace `{remote_identity_id}` with the identity ID from Step 1. +Resolve the stages for your selected datasets from these definitions. Do not reuse legacy product `87`, legacy payload mappings, or example stage IDs. The new product ID and dataset-to-payload mappings are not established by the reviewed service implementation; its `109` reference is explicitly marked TODO. -**Example request (Sponsored Ads):** +## Step 4 — Provision managed delivery resources ```http -POST https://service.api.openbridge.io/service/amzadv/stream-v2/112 +POST https://service.api.openbridge.io/service/amzadv/stream-s3/112 Authorization: Bearer Content-Type: application/json ``` @@ -304,130 +75,81 @@ Content-Type: application/json "data": { "type": "Service", "attributes": { - "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "datasets": ["sp-traffic", "sp-conversion", "budget-usage"], - "profile_type": "ads", - "profile_id": "4463883966959342" + "advertiser_id": "ADVERTISER_ACCOUNT_ID", + "datasets": ["ads-performance-v1", "ads-converted-product-v1"] } } } ``` -**Example request (DSP):** +Use the advertiser account ID from Step 2. These are the two setup fields consumed by the new implementation; omit `role_arn`, `aws_account_id`, `profile_id`, and `profile_type`. + +### Poll the asynchronous operation + +The service returns `202 Accepted` and a `Location` header: + +```http +Location: https://service.api.openbridge.io/service/amzadv/stream/poll-async/{request_id} +``` ```json { "data": { - "type": "Service", - "attributes": { - "role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "datasets": ["adsp-traffic", "adsp-conversion"], - "profile_type": "dsp", - "advertiser_id": "ADV123456" - } + "type": "AmazonAdvertisingBrandMetricsStream", + "id": "ADVERTISER_ACCOUNT_ID", + "attributes": {"status": "CREATING"} } } ``` -| Field | Description | -|---|---| -| `role_arn` | AWS IAM Role ARN granting Openbridge permissions to manage SQS queues | -| `datasets` | Array of dataset names from Step 5 | -| `profile_type` | `"ads"` for Sponsored Ads, `"dsp"` for DSP and Managed DSP | -| `profile_id` | Required when `profile_type` is `"ads"` — the profile ID from Step 3 | -| `advertiser_id` | Required when `profile_type` is `"dsp"` — the advertiser ID from Step 4 (or the `dsp_advertiser_id` for managed DSP) | - -### Async response handling - -This endpoint returns an **asynchronous response**. The initial response includes a `Location` header with a polling URL. Poll this URL until you receive an HTTP `200` response with the queue data. +Send authenticated `GET` requests to the returned `Location` using the same user token. Poll while the HTTP status is `202`, for example every two seconds with an application timeout. Continue only after `200 OK`; handle terminal errors before creating the pipeline subscription. The shared polling path remains `/stream/poll-async`, for managed S3 delivery operations. -- **Poll interval:** 2 seconds -- **Timeout:** up to 13 minutes (the queue provisioning process can take several minutes) - -**Example success response:** +The completed response has **top-level `attributes`**, unlike the initial `data.attributes` envelope. Example for one selected dataset: ```json { - "data": { - "type": "AmazonAdvertisingStreamV2", - "attributes": { - "queue_urls": { - "sp-traffic": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123", - "sp-conversion": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123", - "budget-usage": "https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123" - }, - "dataset_params": { - "sp-traffic": {"stream_id": "str_abc123"}, - "sp-conversion": {"stream_id": "str_def456"}, - "budget-usage": {"stream_id": "str_ghi789"} + "type": "ServiceAmzAdvBrandMetricsStream", + "advertiser_id": "ADVERTISER_ACCOUNT_ID", + "attributes": { + "queue_urls": { + "ads-performance-v1": "https://sqs.us-east-1.amazonaws.com/123456789012/example-queue" + }, + "dataset_params": { + "ads-performance-v1": { + "amz_subscription_id": "AMAZON_STREAM_SUBSCRIPTION_ID", + "bucket": "example-openbridge-managed-bucket", + "queue": "arn:aws:sqs:us-east-1:123456789012:example-queue" } } } } ``` -Save the `queue_urls` and `dataset_params` values from the response — they are required for the pipeline subscription meta in Step 7. - ---- +Save both maps exactly as returned, including the hyphenated dataset keys. `queue_urls` contains queue URLs; `dataset_params` contains each Amazon subscription ID, bucket name, and queue ARN. These are Openbridge-managed resources, not customer AWS resources. The response type names shown above are the names currently emitted by the service. -## Step 7 — Create the pipeline subscription +## Step 5 — Create the pipeline subscription -With all the pieces gathered from the previous steps, create the pipeline subscription for product `87` (Amazon Advertising Marketing Stream). +Create the pipeline through the [Subscriptions API (v2)](../api-usage-docs/subscriptions-api.md): -``` +```http POST https://subscriptions.api.openbridge.io/v2/sub +Authorization: Bearer +Content-Type: application/json ``` -For general detail on subscription creation, see the [Subscription Configuration tutorial](./subscription-configuration.md). - -### product_parameters - -This product requires 8 keys in `product_parameters`. Remote identity is set via the top-level `remote_identity` field (see Step 1) rather than a `product_parameters` key. +Prepare `product_parameters` using the selected product's catalog requirements: -| Key | Source | Description | -|---|---|---| -| `profile_id` | Step 3 | The advertising profile ID. Set to `"NOT_APPLICABLE"` for managed DSP. | -| `profile_type` | Step 2 | `"ads"`, `"dsp"`, or `"managedDsp"` | -| `advertiser_id` | Step 4 | The advertiser ID. Set to `"NOT_APPLICABLE"` for Sponsored Ads (`ads`). | -| `aws_iam_role_arn` | Step 6 | The AWS IAM Role ARN provided in the SQS generation request | -| `queue_urls` | Step 6 | Stringified `queue_urls` map from the SQS generation response (JSON string) | -| `dataset_params` | Step 6 | Stringified `dataset_params` map from the SQS generation response (JSON string) | -| `selected_tables` | Step 5 | Stringified array of selected dataset names (JSON string) | -| `stage_ids` | Step 5 | Stringified array of stage IDs corresponding to the selected datasets (JSON string) | - -> **Note:** The `stage_ids` are resolved by mapping each selected dataset name to its corresponding payload stage via the [Products API payload definitions](../api-usage-docs/products-api.md). Each dataset name maps to a payload name (see the tables in Step 5), and the payload's `stage_id` is used. - -### Full example request (Sponsored Ads) +| Key | Value | +|---|---| +| `advertiser_id` | Advertiser account ID from Step 2 | +| `queue_urls` | JSON-stringified complete `queue_urls` map from Step 4 | +| `dataset_params` | JSON-stringified complete `dataset_params` map from Step 4 | +| `selected_tables` | JSON-stringified array of selected dataset IDs | +| `stage_ids` | JSON-stringified array of corresponding stage IDs from Step 3 | -```json -{ - "data": { - "type": "Subscription", - "attributes": { - "account": 1, - "user": 1, - "product": 87, - "name": "My Marketing Stream - Sponsored Ads", - "status": "active", - "date_start": "2024-06-01T00:00:00Z", - "remote_identity": 112, - "storage_group": 1, - "product_parameters": { - "profile_id": "4463883966959342", - "profile_type": "ads", - "advertiser_id": "NOT_APPLICABLE", - "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "queue_urls": "{\"sp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-traffic-abc123\",\"sp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-sp-conversion-abc123\",\"budget_usage\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-budget-usage-abc123\"}", - "dataset_params": "{\"sp-traffic\":{\"stream_id\":\"str_abc123\"},\"sp-conversion\":{\"stream_id\":\"str_def456\"},\"budget-usage\":{\"stream_id\":\"str_ghi789\"}}", - "selected_tables": "[\"sp-traffic\",\"sp-conversion\",\"budget-usage\"]", - "stage_ids": "[101,102,103]" - } - } - } -} -``` +Set `remote_identity` at the top level. An AWS IAM role parameter is not part of this setup. Confirm any additional required metadata against the selected product before submitting. -### Full example request (DSP) +The following is a request **template** for one dataset. Replace the product placeholder with the numeric product ID and the stage placeholder inside the JSON string with the actual numeric stage ID. Replace all account, destination, advertiser, and resource examples with your values: ```json { @@ -436,61 +158,63 @@ This product requires 8 keys in `product_parameters`. Remote identity is set via "attributes": { "account": 1, "user": 1, - "product": 87, - "name": "My Marketing Stream - DSP", + "product": "", + "name": "My Marketing Stream v2", "status": "active", - "date_start": "2024-06-01T00:00:00Z", + "date_start": "2026-10-13T00:00:00Z", "remote_identity": 112, "storage_group": 1, "product_parameters": { - "profile_id": "4463883966959342", - "profile_type": "dsp", - "advertiser_id": "ADV123456", - "aws_iam_role_arn": "arn:aws:iam::123456789012:role/openbridge-marketing-stream-role", - "queue_urls": "{\"adsp_traffic\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-traffic-abc123\",\"adsp_conversion\":\"https://sqs.us-east-1.amazonaws.com/123456789012/ob-stream2-adsp-conversion-abc123\"}", - "dataset_params": "{\"adsp-traffic\":{\"stream_id\":\"str_xyz123\"},\"adsp-conversion\":{\"stream_id\":\"str_xyz456\"}}", - "selected_tables": "[\"adsp-traffic\",\"adsp-conversion\"]", - "stage_ids": "[201,202]" + "advertiser_id": "ADVERTISER_ACCOUNT_ID", + "queue_urls": "{\"ads-performance-v1\":\"https://sqs.us-east-1.amazonaws.com/123456789012/example-queue\"}", + "dataset_params": "{\"ads-performance-v1\":{\"amz_subscription_id\":\"AMAZON_STREAM_SUBSCRIPTION_ID\",\"bucket\":\"example-openbridge-managed-bucket\",\"queue\":\"arn:aws:sqs:us-east-1:123456789012:example-queue\"}}", + "selected_tables": "[\"ads-performance-v1\"]", + "stage_ids": "[]" } } } } ``` -Replace the placeholder values (`account`, `user`, `remote_identity`, `storage_group`, and the `product_parameters` values) with values from your account and the previous steps. - ---- +For multiple datasets, include every selected dataset in both response maps and both selection arrays. ## Updating a marketing stream subscription -To update an existing marketing stream subscription (e.g., to add or remove datasets), use two requests: +The current service URL configuration exposes only the update worker route: -**1. Update the SQS queues via Service API:** - -``` -PATCH https://service.api.openbridge.io/service/amzadv/stream-v2/update/{remote_identity_id}/{subscription_id} +```http +PATCH https://service.api.openbridge.io/service/amzadv/stream-s3/update/{remote_identity_id}/{co_id}/{subscription_id} ``` -The request body is the same structure as the create endpoint — include the `role_arn`, updated `datasets`, `profile_type`, and `profile_id` or `advertiser_id`. This is an async operation with the same polling behavior as the create endpoint. +This route calls `update_stack` and requires an existing ConsoleOps request ID (`co_id`). It is not the customer-facing async entry point. The `update` handler that creates that request and returns a polling `Location` is currently not routed. A customer-facing dataset update cannot be started through the documented public endpoints until that route is restored; do not create a `co_id` or call the worker directly. -**2. Update the pipeline subscription:** +When the customer-facing update entry point is available, its handler expects the stored `advertiser_id` and the **full desired dataset list**, not just additions, using the same body as Step 4. The implementation archives **all existing Amazon stream subscriptions** and registers the requested datasets again, including retained datasets. Allow for a delivery interruption. -``` +After a successful managed resource update, update the pipeline: + +```http PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +Authorization: Bearer +Content-Type: application/json ``` -Update the `queue_urls`, `dataset_params`, `selected_tables`, `stage_ids`, and any other changed keys in `product_parameters` with the new values from the update response (partial merge — omitted keys keep their stored value). - -See the [Subscriptions API (v2)](../api-usage-docs/subscriptions-api.md) for full PATCH documentation. - ---- +Send `data.type: "Subscription"`, `data.id` equal to the subscription ID, and `data.attributes.product_parameters` containing the newly stringified `queue_urls`, `dataset_params`, `selected_tables`, and `stage_ids`. Replace both resource maps with the complete successful response, because Amazon stream subscription IDs can change. Omitted product parameter keys keep their stored values. ## Deleting a marketing stream subscription -To delete a marketing stream pipeline, mark the pipeline subscription as invalid: +First archive the registered Amazon streams while their metadata is still available: +```http +DELETE https://service.api.openbridge.io/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id} +Authorization: Bearer ``` + +After `204 No Content`, mark the pipeline subscription invalid: + +```http PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +Authorization: Bearer +Content-Type: application/json ``` ```json @@ -498,11 +222,9 @@ PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} "data": { "type": "Subscription", "id": "12345", - "attributes": { - "status": "invalid" - } + "attributes": {"status": "invalid"} } } ``` -See the [Subscription Configuration tutorial](./subscription-configuration.md#deleting-a-subscription) for more detail on subscription status management. +The reviewed delete handler archives Amazon stream subscriptions; it does not delete the managed S3 bucket or SQS queues. See [subscription status management](./subscription-configuration.md#deleting-a-subscription) for pipeline behavior. From beac7ed39b73283c05b77ccc9e843203a776b1c9 Mon Sep 17 00:00:00 2001 From: Devin Roberts Date: Tue, 13 Oct 2026 12:37:57 -0400 Subject: [PATCH 2/6] update AMS S3 product ID in docs --- .../service-amazon-advertising-api.md | 2 +- ...n-advertising-marketing-stream-pipeline.md | 23 +++++++------------ 2 files changed, 9 insertions(+), 16 deletions(-) diff --git a/api-usage-docs/service-amazon-advertising-api.md b/api-usage-docs/service-amazon-advertising-api.md index 75f88ba..44e92ba 100644 --- a/api-usage-docs/service-amazon-advertising-api.md +++ b/api-usage-docs/service-amazon-advertising-api.md @@ -247,7 +247,7 @@ Authorization: Bearer ### Marketing Stream v2 -Marketing Stream v2 uses `/service/amzadv/stream-s3` and Openbridge-managed delivery resources. No customer AWS account, IAM role, or CloudFormation template is required. Follow the [Marketing Stream v2 pipeline tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline.md) for requests, async response handling, and subscription metadata. +Marketing Stream v2 (product ID `110`) uses `/service/amzadv/stream-s3` and Openbridge-managed delivery resources. No customer AWS account, IAM role, or CloudFormation template is required. Follow the [Marketing Stream v2 pipeline tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline.md) for requests, async response handling, and subscription metadata. | Method | Path | Purpose | |---|---|---| diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index 99242d2..141a23e 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -35,7 +35,7 @@ Replace `112` with your remote identity ID. The service returns a flat array of This flow uses advertiser account IDs for all requests. The legacy profile-type selection (`ads`, `dsp`, `managedDsp`), profile lookup, and profile-scoped advertiser lookup are no longer setup steps. -## Step 3 — Select datasets and find the product +## Step 3 — Select datasets The new service accepts these dataset IDs: @@ -46,21 +46,14 @@ The new service accepts these dataset IDs: Choose a nonempty array, such as `["ads-performance-v1", "ads-converted-product-v1"]`. The reviewed service code has a dataset-list method but no `/stream-s3/list-datasets` route; the legacy `/stream-v2/list-datasets` returns datasets for the old flow. -Find the Marketing Stream v2 product in the [Products API](../api-usage-docs/products-api.md): +Marketing Stream v2 uses **product ID `110`**. Retrieve its [payload definitions](../api-usage-docs/products-api.md) to resolve the stage IDs for your selected datasets: ```http -GET https://subscriptions.api.openbridge.io/product?remote_identity_type=14&name__icontains=Marketing +GET https://service.api.openbridge.io/service/products/product/110/payloads?stage_id__gte=1000 Authorization: Bearer ``` -Follow pagination and confirm the product's name and `required_meta_fields`. Record its numeric `id` and retrieve its payload definitions: - -```http -GET https://service.api.openbridge.io/service/products/product/{product_id}/payloads?stage_id__gte=1000 -Authorization: Bearer -``` - -Resolve the stages for your selected datasets from these definitions. Do not reuse legacy product `87`, legacy payload mappings, or example stage IDs. The new product ID and dataset-to-payload mappings are not established by the reviewed service implementation; its `109` reference is explicitly marked TODO. +Resolve the stages for your selected datasets from these definitions. Use the payload mappings and stage IDs for product `110`. ## Step 4 — Provision managed delivery resources @@ -137,7 +130,7 @@ Authorization: Bearer Content-Type: application/json ``` -Prepare `product_parameters` using the selected product's catalog requirements: +Prepare `product_parameters` for product `110`: | Key | Value | |---|---| @@ -147,9 +140,9 @@ Prepare `product_parameters` using the selected product's catalog requirements: | `selected_tables` | JSON-stringified array of selected dataset IDs | | `stage_ids` | JSON-stringified array of corresponding stage IDs from Step 3 | -Set `remote_identity` at the top level. An AWS IAM role parameter is not part of this setup. Confirm any additional required metadata against the selected product before submitting. +Set `remote_identity` at the top level. An AWS IAM role parameter is not part of this setup. Confirm any additional required metadata against product `110` before submitting. -The following is a request **template** for one dataset. Replace the product placeholder with the numeric product ID and the stage placeholder inside the JSON string with the actual numeric stage ID. Replace all account, destination, advertiser, and resource examples with your values: +The following is a request **template** for one dataset. Replace the stage placeholder inside the JSON string with the actual numeric stage ID. Replace all account, destination, advertiser, and resource examples with your values: ```json { @@ -158,7 +151,7 @@ The following is a request **template** for one dataset. Replace the product pla "attributes": { "account": 1, "user": 1, - "product": "", + "product": 110, "name": "My Marketing Stream v2", "status": "active", "date_start": "2026-10-13T00:00:00Z", From 40b697eebeb8d1206e6b905c4e1e840361f41daa Mon Sep 17 00:00:00 2001 From: Devin Roberts Date: Thu, 15 Oct 2026 11:04:43 -0400 Subject: [PATCH 3/6] refactor: update AMS docs to reflect queue changes --- .../service-amazon-advertising-api.md | 2 + ...n-advertising-marketing-stream-pipeline.md | 58 +++++++++++-------- 2 files changed, 35 insertions(+), 25 deletions(-) diff --git a/api-usage-docs/service-amazon-advertising-api.md b/api-usage-docs/service-amazon-advertising-api.md index 44e92ba..0800ae4 100644 --- a/api-usage-docs/service-amazon-advertising-api.md +++ b/api-usage-docs/service-amazon-advertising-api.md @@ -259,6 +259,8 @@ Marketing Stream v2 (product ID `110`) uses `/service/amzadv/stream-s3` and Open Create and the update handler accept `data.attributes.advertiser_id` and a nonempty `data.attributes.datasets` array containing `ads-performance-v1` and/or `ads-converted-product-v1`. There is currently no routed managed S3 dataset-list endpoint. +Completed operations return a shared `sqs_notification_url` and `bucket`, plus `amz_subscription_ids` keyed by dataset ID. They no longer return per-dataset `queue_urls` or `dataset_params`. Product `110` requires `advertiser_id`, `sqs_notification_url`, `selected_tables`, and `profile_type`; see the tutorial for wire encoding, additional lifecycle metadata, and the current resource validation limitation. + For the older `/stream-v2` customer-managed AWS flow, see the [legacy tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md). The customer-facing async update handler is currently not routed. The update worker listed above is not a customer entry point; see the tutorial for this limitation. diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index 141a23e..9fcabf9 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -33,7 +33,7 @@ Authorization: Bearer Replace `112` with your remote identity ID. The service returns a flat array of Amazon `advertiserAccounts` records and collects all pages from Amazon. Record the selected record's `advertiserAccountId` as `advertiser_id`. To filter the lookup, append `?advertiser_id={advertiserAccountId}`. -This flow uses advertiser account IDs for all requests. The legacy profile-type selection (`ads`, `dsp`, `managedDsp`), profile lookup, and profile-scoped advertiser lookup are no longer setup steps. +This flow uses advertiser account IDs for all requests. Profile lookup and profile-scoped advertiser lookup are no longer setup steps. The service request does not use `profile_type`, but the product `110` subscription schema still requires it; retain the account type (`ads`, `dsp`, or `managedDsp`) for Step 5. ## Step 3 — Select datasets @@ -97,28 +97,28 @@ Location: https://service.api.openbridge.io/service/amzadv/stream/poll-async/{re Send authenticated `GET` requests to the returned `Location` using the same user token. Poll while the HTTP status is `202`, for example every two seconds with an application timeout. Continue only after `200 OK`; handle terminal errors before creating the pipeline subscription. The shared polling path remains `/stream/poll-async`, for managed S3 delivery operations. -The completed response has **top-level `attributes`**, unlike the initial `data.attributes` envelope. Example for one selected dataset: +The completed response uses the following `data` envelope: ```json { - "type": "ServiceAmzAdvBrandMetricsStream", - "advertiser_id": "ADVERTISER_ACCOUNT_ID", - "attributes": { - "queue_urls": { - "ads-performance-v1": "https://sqs.us-east-1.amazonaws.com/123456789012/example-queue" - }, - "dataset_params": { - "ads-performance-v1": { - "amz_subscription_id": "AMAZON_STREAM_SUBSCRIPTION_ID", - "bucket": "example-openbridge-managed-bucket", - "queue": "arn:aws:sqs:us-east-1:123456789012:example-queue" + "data": { + "type": "ServiceAmzAdvBrandMetricsStream", + "advertiser_id": "589711999947333868", + "attributes": { + "sqs_notification_url": "https://sqs.us-east-1.amazonaws.com/699059249070/ams-dev-ri6073-589711999947333868-s3-notifications", + "bucket": "openbridge-ams-us-east-1-589711999947333868", + "amz_subscription_ids": { + "ads-performance-v1": "amzn1.ads-stream.sub1.4VzvRZFtnSqGFKLhHomypB", + "ads-converted-product-v1": "amzn1.ads-stream.sub1.6sGplvMSZ73EW51WDlnM6w" } } } } ``` -Save both maps exactly as returned, including the hyphenated dataset keys. `queue_urls` contains queue URLs; `dataset_params` contains each Amazon subscription ID, bucket name, and queue ARN. These are Openbridge-managed resources, not customer AWS resources. The response type names shown above are the names currently emitted by the service. +All selected datasets share `sqs_notification_url` and `bucket`. Save those values and the complete `amz_subscription_ids` map, preserving the hyphenated dataset keys. The service no longer returns `queue_urls`, `dataset_params`, or a per-dataset queue ARN. + +The service worker and shared polling handler currently store and return the completed object without the outer `data` wrapper. Clients supporting that implementation should also accept the unwrapped object (`response.attributes` instead of `response.data.attributes`). ## Step 5 — Create the pipeline subscription @@ -134,13 +134,19 @@ Prepare `product_parameters` for product `110`: | Key | Value | |---|---| -| `advertiser_id` | Advertiser account ID from Step 2 | -| `queue_urls` | JSON-stringified complete `queue_urls` map from Step 4 | -| `dataset_params` | JSON-stringified complete `dataset_params` map from Step 4 | -| `selected_tables` | JSON-stringified array of selected dataset IDs | -| `stage_ids` | JSON-stringified array of corresponding stage IDs from Step 3 | +| `advertiser_id` | Required nonempty alphanumeric string from Step 2 | +| `selected_tables` | Required JSON-stringified nonempty array of unique dataset ID strings | +| `stage_ids` | Optional JSON-stringified nonempty array of unique integer stage IDs from Step 3 | +| `profile_type` | Required string: `ads`, `dsp`, or `managedDsp` | +| `sqs_notification_url` | Required nonempty string: the shared notification queue URL from Step 4 | +| `bucket` | Save the shared bucket name from Step 4 | +| `amz_subscription_ids` | Save the complete map from Step 4 as a JSON string; required by the service update/delete handlers | -Set `remote_identity` at the top level. An AWS IAM role parameter is not part of this setup. Confirm any additional required metadata against product `110` before submitting. +Set `remote_identity` at the top level. An AWS IAM role parameter is not part of this setup. The product schema allows additional properties, so retain `bucket` and `amz_subscription_ids` alongside its required fields. `queue_urls` and `dataset_params` are no longer required by the schema. + +The schema defines `selected_tables` and `stage_ids` as arrays, but the v2 subscription validator requires JSON-encoded strings in the request and decodes them before validation. `sqs_notification_url` is a plain URL string, copied directly from the service response. Although its schema description still refers to a stringified dataset map, the service returns one shared URL. + +**Integration limitation:** the product `110` resource validator still reads the removed `queue_urls` and `dataset_params` fields. The template below satisfies the corrected schema after selection-array decoding, but that validator must also be aligned with the new fields for end-to-end subscription creation. The following is a request **template** for one dataset. Replace the stage placeholder inside the JSON string with the actual numeric stage ID. Replace all account, destination, advertiser, and resource examples with your values: @@ -158,9 +164,11 @@ The following is a request **template** for one dataset. Replace the stage place "remote_identity": 112, "storage_group": 1, "product_parameters": { - "advertiser_id": "ADVERTISER_ACCOUNT_ID", - "queue_urls": "{\"ads-performance-v1\":\"https://sqs.us-east-1.amazonaws.com/123456789012/example-queue\"}", - "dataset_params": "{\"ads-performance-v1\":{\"amz_subscription_id\":\"AMAZON_STREAM_SUBSCRIPTION_ID\",\"bucket\":\"example-openbridge-managed-bucket\",\"queue\":\"arn:aws:sqs:us-east-1:123456789012:example-queue\"}}", + "advertiser_id": "589711999947333868", + "profile_type": "ads", + "sqs_notification_url": "https://sqs.us-east-1.amazonaws.com/123456789012/example-queue", + "bucket": "example-openbridge-managed-bucket", + "amz_subscription_ids": "{\"ads-performance-v1\":\"AMAZON_STREAM_SUBSCRIPTION_ID\"}", "selected_tables": "[\"ads-performance-v1\"]", "stage_ids": "[]" } @@ -169,7 +177,7 @@ The following is a request **template** for one dataset. Replace the stage place } ``` -For multiple datasets, include every selected dataset in both response maps and both selection arrays. +For multiple datasets, include every selected dataset in `amz_subscription_ids` and `selected_tables`, and resolve the corresponding `stage_ids` if supplied. Keep the single shared `sqs_notification_url` and `bucket`. ## Updating a marketing stream subscription @@ -191,7 +199,7 @@ Authorization: Bearer Content-Type: application/json ``` -Send `data.type: "Subscription"`, `data.id` equal to the subscription ID, and `data.attributes.product_parameters` containing the newly stringified `queue_urls`, `dataset_params`, `selected_tables`, and `stage_ids`. Replace both resource maps with the complete successful response, because Amazon stream subscription IDs can change. Omitted product parameter keys keep their stored values. +Send `data.type: "Subscription"`, `data.id` equal to the subscription ID, and `data.attributes.product_parameters` containing the refreshed `sqs_notification_url`, `bucket`, and JSON-stringified `amz_subscription_ids`, plus JSON-stringified `selected_tables` and optional `stage_ids`. Preserve the required `advertiser_id` and `profile_type`. Replace the complete Amazon subscription ID map, because IDs can change. The product validation limitation in Step 5 also applies to updates. Omitted product parameter keys keep their stored values. ## Deleting a marketing stream subscription From f401e892b16d84326af185feb41fea91faf49c10 Mon Sep 17 00:00:00 2001 From: Devin Roberts Date: Thu, 15 Oct 2026 16:18:13 -0400 Subject: [PATCH 4/6] refactor: fix step 3 in AMS setup --- .../service-amazon-advertising-api.md | 3 +- ...n-advertising-marketing-stream-pipeline.md | 39 +++++++++++-------- 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/api-usage-docs/service-amazon-advertising-api.md b/api-usage-docs/service-amazon-advertising-api.md index 0800ae4..54ef887 100644 --- a/api-usage-docs/service-amazon-advertising-api.md +++ b/api-usage-docs/service-amazon-advertising-api.md @@ -252,12 +252,13 @@ Marketing Stream v2 (product ID `110`) uses `/service/amzadv/stream-s3` and Open | Method | Path | Purpose | |---|---|---| | GET | `/service/amzadv/list-adv-v2/{remote_identity_id}` | List advertiser accounts; optional `advertiser_id` filter | +| GET | `/service/amzadv/stream-s3/list-datasets` | List valid dataset IDs as a JSON array | | POST | `/service/amzadv/stream-s3/{remote_identity_id}` | Provision resources and register selected datasets | | GET | `/service/amzadv/stream/poll-async/{request_id}` | Poll the `Location` returned by create or update | | PATCH | `/service/amzadv/stream-s3/update/{remote_identity_id}/{co_id}/{subscription_id}` | Internal update worker; requires an existing ConsoleOps request | | DELETE | `/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id}` | Archive registered Amazon streams | -Create and the update handler accept `data.attributes.advertiser_id` and a nonempty `data.attributes.datasets` array containing `ads-performance-v1` and/or `ads-converted-product-v1`. There is currently no routed managed S3 dataset-list endpoint. +Create and the update handler accept `data.attributes.advertiser_id` and a nonempty `data.attributes.datasets` array containing `ads-performance-v1` and/or `ads-converted-product-v1`. Retrieve the valid IDs with `GET /service/amzadv/stream-s3/list-datasets`. Product `110` uses stage ID `1000` for every dataset selection (`stage_ids: "[1000]"` when supplied). Completed operations return a shared `sqs_notification_url` and `bucket`, plus `amz_subscription_ids` keyed by dataset ID. They no longer return per-dataset `queue_urls` or `dataset_params`. Product `110` requires `advertiser_id`, `sqs_notification_url`, `selected_tables`, and `profile_type`; see the tutorial for wire encoding, additional lifecycle metadata, and the current resource validation limitation. diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index 9fcabf9..e33ca96 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -37,23 +37,30 @@ This flow uses advertiser account IDs for all requests. Profile lookup and profi ## Step 3 — Select datasets -The new service accepts these dataset IDs: - -| Dataset ID | -|---| -| `ads-performance-v1` | -| `ads-converted-product-v1` | - -Choose a nonempty array, such as `["ads-performance-v1", "ads-converted-product-v1"]`. The reviewed service code has a dataset-list method but no `/stream-s3/list-datasets` route; the legacy `/stream-v2/list-datasets` returns datasets for the old flow. - -Marketing Stream v2 uses **product ID `110`**. Retrieve its [payload definitions](../api-usage-docs/products-api.md) to resolve the stage IDs for your selected datasets: +Marketing Stream S3 uses **product ID `110`** with stage ID **`1000`** for every dataset selection. Retrieve the valid dataset IDs: ```http -GET https://service.api.openbridge.io/service/products/product/110/payloads?stage_id__gte=1000 +GET https://service.api.openbridge.io/service/amzadv/stream-s3/list-datasets Authorization: Bearer ``` -Resolve the stages for your selected datasets from these definitions. Use the payload mappings and stage IDs for product `110`. +The endpoint returns a JSON array: + +```json +{ + "data": { + "type": "AMSDatasets", + "attributes": { + "datasets": [ + "ads-performance-v1", + "ads-converted-product-v1" + ] + } + } +} +``` + +The stage ID remains `1000` whether you select one dataset or both. All selected datasets use the shared notification queue and bucket returned in Step 4. ## Step 4 — Provision managed delivery resources @@ -136,7 +143,7 @@ Prepare `product_parameters` for product `110`: |---|---| | `advertiser_id` | Required nonempty alphanumeric string from Step 2 | | `selected_tables` | Required JSON-stringified nonempty array of unique dataset ID strings | -| `stage_ids` | Optional JSON-stringified nonempty array of unique integer stage IDs from Step 3 | +| `stage_ids` | Optional; when supplied, use `"[1000]"` for this product | | `profile_type` | Required string: `ads`, `dsp`, or `managedDsp` | | `sqs_notification_url` | Required nonempty string: the shared notification queue URL from Step 4 | | `bucket` | Save the shared bucket name from Step 4 | @@ -148,7 +155,7 @@ The schema defines `selected_tables` and `stage_ids` as arrays, but the v2 subsc **Integration limitation:** the product `110` resource validator still reads the removed `queue_urls` and `dataset_params` fields. The template below satisfies the corrected schema after selection-array decoding, but that validator must also be aligned with the new fields for end-to-end subscription creation. -The following is a request **template** for one dataset. Replace the stage placeholder inside the JSON string with the actual numeric stage ID. Replace all account, destination, advertiser, and resource examples with your values: +The following is a request **template** for one dataset. Replace all account, destination, advertiser, and resource examples with your values: ```json { @@ -170,14 +177,14 @@ The following is a request **template** for one dataset. Replace the stage place "bucket": "example-openbridge-managed-bucket", "amz_subscription_ids": "{\"ads-performance-v1\":\"AMAZON_STREAM_SUBSCRIPTION_ID\"}", "selected_tables": "[\"ads-performance-v1\"]", - "stage_ids": "[]" + "stage_ids": "[1000]" } } } } ``` -For multiple datasets, include every selected dataset in `amz_subscription_ids` and `selected_tables`, and resolve the corresponding `stage_ids` if supplied. Keep the single shared `sqs_notification_url` and `bucket`. +For multiple datasets, include every selected dataset in `amz_subscription_ids` and `selected_tables`, and keep `stage_ids` as `"[1000]"`. Keep the single shared `sqs_notification_url` and `bucket`. ## Updating a marketing stream subscription From b22caaa5c0a12cccd9c2a9863aab669e8eb21611 Mon Sep 17 00:00:00 2001 From: Devin Roberts Date: Mon, 19 Oct 2026 08:30:30 -0400 Subject: [PATCH 5/6] fix: remove defunct `profile_type` SPM requirement --- tutorials/amazon-advertising-marketing-stream-pipeline.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index e33ca96..200412a 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -33,8 +33,6 @@ Authorization: Bearer Replace `112` with your remote identity ID. The service returns a flat array of Amazon `advertiserAccounts` records and collects all pages from Amazon. Record the selected record's `advertiserAccountId` as `advertiser_id`. To filter the lookup, append `?advertiser_id={advertiserAccountId}`. -This flow uses advertiser account IDs for all requests. Profile lookup and profile-scoped advertiser lookup are no longer setup steps. The service request does not use `profile_type`, but the product `110` subscription schema still requires it; retain the account type (`ads`, `dsp`, or `managedDsp`) for Step 5. - ## Step 3 — Select datasets Marketing Stream S3 uses **product ID `110`** with stage ID **`1000`** for every dataset selection. Retrieve the valid dataset IDs: @@ -82,7 +80,7 @@ Content-Type: application/json } ``` -Use the advertiser account ID from Step 2. These are the two setup fields consumed by the new implementation; omit `role_arn`, `aws_account_id`, `profile_id`, and `profile_type`. +Use the advertiser account ID from Step 2. These are the two setup fields consumed by the new implementation; omit `role_arn`, `aws_account_id` and `profile_id`. ### Poll the asynchronous operation @@ -144,7 +142,6 @@ Prepare `product_parameters` for product `110`: | `advertiser_id` | Required nonempty alphanumeric string from Step 2 | | `selected_tables` | Required JSON-stringified nonempty array of unique dataset ID strings | | `stage_ids` | Optional; when supplied, use `"[1000]"` for this product | -| `profile_type` | Required string: `ads`, `dsp`, or `managedDsp` | | `sqs_notification_url` | Required nonempty string: the shared notification queue URL from Step 4 | | `bucket` | Save the shared bucket name from Step 4 | | `amz_subscription_ids` | Save the complete map from Step 4 as a JSON string; required by the service update/delete handlers | @@ -172,7 +169,6 @@ The following is a request **template** for one dataset. Replace all account, de "storage_group": 1, "product_parameters": { "advertiser_id": "589711999947333868", - "profile_type": "ads", "sqs_notification_url": "https://sqs.us-east-1.amazonaws.com/123456789012/example-queue", "bucket": "example-openbridge-managed-bucket", "amz_subscription_ids": "{\"ads-performance-v1\":\"AMAZON_STREAM_SUBSCRIPTION_ID\"}", @@ -206,7 +202,7 @@ Authorization: Bearer Content-Type: application/json ``` -Send `data.type: "Subscription"`, `data.id` equal to the subscription ID, and `data.attributes.product_parameters` containing the refreshed `sqs_notification_url`, `bucket`, and JSON-stringified `amz_subscription_ids`, plus JSON-stringified `selected_tables` and optional `stage_ids`. Preserve the required `advertiser_id` and `profile_type`. Replace the complete Amazon subscription ID map, because IDs can change. The product validation limitation in Step 5 also applies to updates. Omitted product parameter keys keep their stored values. +Send `data.type: "Subscription"`, `data.id` equal to the subscription ID, and `data.attributes.product_parameters` containing the refreshed `sqs_notification_url`, `bucket`, and JSON-stringified `amz_subscription_ids`, plus JSON-stringified `selected_tables` and optional `stage_ids`. Preserve the required `advertiser_id`. Replace the complete Amazon subscription ID map, because IDs can change. The product validation limitation in Step 5 also applies to updates. Omitted product parameter keys keep their stored values. ## Deleting a marketing stream subscription From bf19e4dfa34fa63d8829bbfb0f294f5c2ee270eb Mon Sep 17 00:00:00 2001 From: Devin Roberts Date: Wed, 18 Nov 2026 14:46:46 -0500 Subject: [PATCH 6/6] docs: updated AMS S3 docs --- .../service-amazon-advertising-api.md | 26 ++- ...n-advertising-marketing-stream-pipeline.md | 171 +++++------------- 2 files changed, 54 insertions(+), 143 deletions(-) diff --git a/api-usage-docs/service-amazon-advertising-api.md b/api-usage-docs/service-amazon-advertising-api.md index 54ef887..3c48f93 100644 --- a/api-usage-docs/service-amazon-advertising-api.md +++ b/api-usage-docs/service-amazon-advertising-api.md @@ -245,23 +245,21 @@ Authorization: Bearer --- -### Marketing Stream v2 +### Marketing Stream S3 -Marketing Stream v2 (product ID `110`) uses `/service/amzadv/stream-s3` and Openbridge-managed delivery resources. No customer AWS account, IAM role, or CloudFormation template is required. Follow the [Marketing Stream v2 pipeline tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline.md) for requests, async response handling, and subscription metadata. +Product `110` registers Amazon Marketing Stream datasets for delivery through shared, regional S3 and SQS resources managed by Openbridge. Authenticate each request with an Openbridge bearer token. -| Method | Path | Purpose | +| Method | Endpoint | Success | |---|---|---| -| GET | `/service/amzadv/list-adv-v2/{remote_identity_id}` | List advertiser accounts; optional `advertiser_id` filter | -| GET | `/service/amzadv/stream-s3/list-datasets` | List valid dataset IDs as a JSON array | -| POST | `/service/amzadv/stream-s3/{remote_identity_id}` | Provision resources and register selected datasets | -| GET | `/service/amzadv/stream/poll-async/{request_id}` | Poll the `Location` returned by create or update | -| PATCH | `/service/amzadv/stream-s3/update/{remote_identity_id}/{co_id}/{subscription_id}` | Internal update worker; requires an existing ConsoleOps request | -| DELETE | `/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id}` | Archive registered Amazon streams | +| GET | `/service/amzadv/list-adv-v2/{remote_identity_id}` | `200 OK` with an array of advertiser accounts. Supports the optional `advertiser_id` query filter. | +| GET | `/service/amzadv/stream-s3/list-datasets` | `200 OK` with the dataset list at `attributes.datasets`. | +| POST | `/service/amzadv/stream-s3/{remote_identity_id}` | `200 OK` with the Amazon subscription IDs. | +| PATCH | `/service/amzadv/stream-s3/update/{remote_identity_id}/{subscription_id}` | `200 OK` with replacement Amazon subscription IDs. | +| DELETE | `/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id}` | `204 No Content` after archiving the Amazon streams stored on the Openbridge subscription. | +| DELETE | `/service/amzadv/stream-s3/delete-orphaned/{remote_identity_id}/{advertiser_id}` | `204 No Content` after archiving the advertiser's streams eligible for cleanup. Requires that the remote identity have no active product `110` subscriptions. | -Create and the update handler accept `data.attributes.advertiser_id` and a nonempty `data.attributes.datasets` array containing `ads-performance-v1` and/or `ads-converted-product-v1`. Retrieve the valid IDs with `GET /service/amzadv/stream-s3/list-datasets`. Product `110` uses stage ID `1000` for every dataset selection (`stage_ids: "[1000]"` when supplied). +Create and update run synchronously. Send the advertiser ID and full dataset list in `data.attributes.advertiser_id` and `data.attributes.datasets`. Both operations return a top-level `advertiser_id` and a dataset-to-ID map at `attributes.amz_subscription_ids`. Update archives the old Amazon streams before registering the requested datasets. -Completed operations return a shared `sqs_notification_url` and `bucket`, plus `amz_subscription_ids` keyed by dataset ID. They no longer return per-dataset `queue_urls` or `dataset_params`. Product `110` requires `advertiser_id`, `sqs_notification_url`, `selected_tables`, and `profile_type`; see the tutorial for wire encoding, additional lifecycle metadata, and the current resource validation limitation. +Store `advertiser_id` and the JSON-stringified `amz_subscription_ids` map in the product `110` pipeline's `product_parameters`. See the [Marketing Stream S3 tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline.md) for request examples and the full pipeline lifecycle. -For the older `/stream-v2` customer-managed AWS flow, see the [legacy tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md). - -The customer-facing async update handler is currently not routed. The update worker listed above is not a customer entry point; see the tutorial for this limitation. +For the `/stream-v2` customer-managed AWS workflow, see the [legacy tutorial](../tutorials/amazon-advertising-marketing-stream-pipeline-legacy.md). diff --git a/tutorials/amazon-advertising-marketing-stream-pipeline.md b/tutorials/amazon-advertising-marketing-stream-pipeline.md index 200412a..1fbb4d9 100644 --- a/tutorials/amazon-advertising-marketing-stream-pipeline.md +++ b/tutorials/amazon-advertising-marketing-stream-pipeline.md @@ -1,66 +1,40 @@ # Amazon Advertising Marketing Stream S3 Pipeline -Marketing Stream S3 uses **`/service/amzadv/stream-s3`** in the Service API. The managed S3 delivery flow uses this endpoint prefix; `/stream-v2` belongs to the [legacy tutorial](./amazon-advertising-marketing-stream-pipeline-legacy.md). - -Amazon delivers dataset files to S3, with queue notifications used by Openbridge to process the files into your configured storage destination. Openbridge provisions and manages these delivery resources in its own AWS account. **Customers do not need an AWS account, an IAM role ARN, or a CloudFormation template.** +Marketing Stream S3 (product `110`) delivers Amazon dataset files through shared, regional S3 and SQS resources managed by Openbridge. Setup requires an Amazon Advertising connection and an Openbridge storage destination. For the customer-managed AWS workflow, see the [legacy Marketing Stream tutorial](./amazon-advertising-marketing-stream-pipeline-legacy.md). ## Prerequisites -- A [JWT access token](../api-usage-docs/authentication-api.md). -- Your [account and user IDs](./identity-configuration.md#step-1--look-up-your-account-and-user-ids). -- An Amazon Advertising remote identity (type `14`) with access to the advertiser account. +- An [Openbridge JWT access token](../api-usage-docs/authentication-api.md). +- Your [Openbridge account and user IDs](./identity-configuration.md#step-1--look-up-your-account-and-user-ids). +- An Amazon Advertising [remote identity](./identity-configuration.md) (type `14`) with access to the advertiser. - A configured [storage destination](./subscription-configuration.md#step-3--identify-your-storage-destination). -## Step 1 — Choose or create a remote identity - -Follow the [Identity Configuration tutorial](./identity-configuration.md) to connect an Amazon Advertising account through OAuth, or list existing valid identities: - -```http -GET https://remote-identity.api.openbridge.io/ri?remote_identity_type=14&invalid_identity=0 -Authorization: Bearer -``` - -Record the identity's `id`. The service uses its `region` (`na`, `eu`, or `fe`) to select the Amazon API and AWS delivery region; you do not send a region in the create request. - -## Step 2 — Select an advertiser account - -Use the new advertiser-account lookup, which does not require a profile ID: +## Step 1 — Find your advertiser and datasets ```http GET https://service.api.openbridge.io/service/amzadv/list-adv-v2/112 Authorization: Bearer ``` -Replace `112` with your remote identity ID. The service returns a flat array of Amazon `advertiserAccounts` records and collects all pages from Amazon. Record the selected record's `advertiserAccountId` as `advertiser_id`. To filter the lookup, append `?advertiser_id={advertiserAccountId}`. - -## Step 3 — Select datasets +Replace `112` with your remote identity ID. The response is an array of advertiser accounts. Use the selected account's `advertiserAccountId` as the string `advertiser_id`. To look up a known ID, append `?advertiser_id={advertiserAccountId}`. -Marketing Stream S3 uses **product ID `110`** with stage ID **`1000`** for every dataset selection. Retrieve the valid dataset IDs: +Before registering datasets, verify that the ID matches a seller or vendor profile's top-level `id` from `GET /service/amzadv/profiles-only/112?profile_types=seller,vendor&is_manager=false`. The Subscriptions API checks this match when you save product `110`. For a manager identity, query with `is_manager=true` and include `"is_manager": "true"` in the pipeline's `product_parameters`. [Amazon Advertising profile lookup](../api-usage-docs/service-amazon-advertising-api.md#list-profiles) ```http GET https://service.api.openbridge.io/service/amzadv/stream-s3/list-datasets Authorization: Bearer ``` -The endpoint returns a JSON array: - ```json { - "data": { - "type": "AMSDatasets", - "attributes": { - "datasets": [ - "ads-performance-v1", - "ads-converted-product-v1" - ] - } + "type": "AMSDatasets", + "attributes": { + "datasets": ["ads-performance-v1", "ads-converted-product-v1"] } } ``` -The stage ID remains `1000` whether you select one dataset or both. All selected datasets use the shared notification queue and bucket returned in Step 4. - -## Step 4 — Provision managed delivery resources +## Step 2 — Register the datasets with Amazon ```http POST https://service.api.openbridge.io/service/amzadv/stream-s3/112 @@ -73,61 +47,32 @@ Content-Type: application/json "data": { "type": "Service", "attributes": { - "advertiser_id": "ADVERTISER_ACCOUNT_ID", - "datasets": ["ads-performance-v1", "ads-converted-product-v1"] + "advertiser_id": "589711999947333868", + "datasets": ["ads-performance-v1"] } } } ``` -Use the advertiser account ID from Step 2. These are the two setup fields consumed by the new implementation; omit `role_arn`, `aws_account_id` and `profile_id`. - -### Poll the asynchronous operation - -The service returns `202 Accepted` and a `Location` header: - -```http -Location: https://service.api.openbridge.io/service/amzadv/stream/poll-async/{request_id} -``` - -```json -{ - "data": { - "type": "AmazonAdvertisingBrandMetricsStream", - "id": "ADVERTISER_ACCOUNT_ID", - "attributes": {"status": "CREATING"} - } -} -``` - -Send authenticated `GET` requests to the returned `Location` using the same user token. Poll while the HTTP status is `202`, for example every two seconds with an application timeout. Continue only after `200 OK`; handle terminal errors before creating the pipeline subscription. The shared polling path remains `/stream/poll-async`, for managed S3 delivery operations. - -The completed response uses the following `data` envelope: +Send at least one supported dataset. The Service API selects the shared S3 bucket and notification queue based on the remote identity's region. Successful registration returns `200 OK` in the same request. ```json { - "data": { - "type": "ServiceAmzAdvBrandMetricsStream", - "advertiser_id": "589711999947333868", - "attributes": { - "sqs_notification_url": "https://sqs.us-east-1.amazonaws.com/699059249070/ams-dev-ri6073-589711999947333868-s3-notifications", - "bucket": "openbridge-ams-us-east-1-589711999947333868", - "amz_subscription_ids": { - "ads-performance-v1": "amzn1.ads-stream.sub1.4VzvRZFtnSqGFKLhHomypB", - "ads-converted-product-v1": "amzn1.ads-stream.sub1.6sGplvMSZ73EW51WDlnM6w" - } + "type": "ServiceAmzAdvBrandMetricsStream", + "advertiser_id": "589711999947333868", + "attributes": { + "amz_subscription_ids": { + "ads-performance-v1": "AMAZON_STREAM_SUBSCRIPTION_ID" } } } ``` -All selected datasets share `sqs_notification_url` and `bucket`. Save those values and the complete `amz_subscription_ids` map, preserving the hyphenated dataset keys. The service no longer returns `queue_urls`, `dataset_params`, or a per-dataset queue ARN. - -The service worker and shared polling handler currently store and return the completed object without the outer `data` wrapper. Clients supporting that implementation should also accept the unwrapped object (`response.attributes` instead of `response.data.attributes`). +Save the complete map at `attributes.amz_subscription_ids`. Keep each dataset key exactly as returned. Registrations are made one dataset at a time, so a failed multi-dataset request can leave Amazon subscriptions that need cleanup before retrying. -## Step 5 — Create the pipeline subscription +## Step 3 — Create the Openbridge pipeline -Create the pipeline through the [Subscriptions API (v2)](../api-usage-docs/subscriptions-api.md): +Send the advertiser ID and the JSON-stringified dataset-to-subscription-ID map as `product_parameters`. Use the remote identity from Step 1 and your Openbridge storage group. The two required product parameters are `advertiser_id` and `amz_subscription_ids`. The Subscriptions API stores the map as a string, and the Service API reads it back for updates and deletes. [Subscriptions API](../api-usage-docs/subscriptions-api.md) ```http POST https://subscriptions.api.openbridge.io/v2/sub @@ -135,25 +80,6 @@ Authorization: Bearer Content-Type: application/json ``` -Prepare `product_parameters` for product `110`: - -| Key | Value | -|---|---| -| `advertiser_id` | Required nonempty alphanumeric string from Step 2 | -| `selected_tables` | Required JSON-stringified nonempty array of unique dataset ID strings | -| `stage_ids` | Optional; when supplied, use `"[1000]"` for this product | -| `sqs_notification_url` | Required nonempty string: the shared notification queue URL from Step 4 | -| `bucket` | Save the shared bucket name from Step 4 | -| `amz_subscription_ids` | Save the complete map from Step 4 as a JSON string; required by the service update/delete handlers | - -Set `remote_identity` at the top level. An AWS IAM role parameter is not part of this setup. The product schema allows additional properties, so retain `bucket` and `amz_subscription_ids` alongside its required fields. `queue_urls` and `dataset_params` are no longer required by the schema. - -The schema defines `selected_tables` and `stage_ids` as arrays, but the v2 subscription validator requires JSON-encoded strings in the request and decodes them before validation. `sqs_notification_url` is a plain URL string, copied directly from the service response. Although its schema description still refers to a stringified dataset map, the service returns one shared URL. - -**Integration limitation:** the product `110` resource validator still reads the removed `queue_urls` and `dataset_params` fields. The template below satisfies the corrected schema after selection-array decoding, but that validator must also be aligned with the new fields for end-to-end subscription creation. - -The following is a request **template** for one dataset. Replace all account, destination, advertiser, and resource examples with your values: - ```json { "data": { @@ -162,64 +88,42 @@ The following is a request **template** for one dataset. Replace all account, de "account": 1, "user": 1, "product": 110, - "name": "My Marketing Stream v2", + "name": "My Marketing Stream", "status": "active", - "date_start": "2026-10-13T00:00:00Z", + "date_start": "2026-11-18T00:00:00Z", "remote_identity": 112, "storage_group": 1, "product_parameters": { "advertiser_id": "589711999947333868", - "sqs_notification_url": "https://sqs.us-east-1.amazonaws.com/123456789012/example-queue", - "bucket": "example-openbridge-managed-bucket", - "amz_subscription_ids": "{\"ads-performance-v1\":\"AMAZON_STREAM_SUBSCRIPTION_ID\"}", - "selected_tables": "[\"ads-performance-v1\"]", - "stage_ids": "[1000]" + "amz_subscription_ids": "{\"ads-performance-v1\":\"AMAZON_STREAM_SUBSCRIPTION_ID\"}" } } } } ``` -For multiple datasets, include every selected dataset in `amz_subscription_ids` and `selected_tables`, and keep `stage_ids` as `"[1000]"`. Keep the single shared `sqs_notification_url` and `bucket`. - -## Updating a marketing stream subscription - -The current service URL configuration exposes only the update worker route: - -```http -PATCH https://service.api.openbridge.io/service/amzadv/stream-s3/update/{remote_identity_id}/{co_id}/{subscription_id} -``` - -This route calls `update_stack` and requires an existing ConsoleOps request ID (`co_id`). It is not the customer-facing async entry point. The `update` handler that creates that request and returns a polling `Location` is currently not routed. A customer-facing dataset update cannot be started through the documented public endpoints until that route is restored; do not create a `co_id` or call the worker directly. +Replace the example IDs and start date with your values. Save the Openbridge subscription ID from the `201 Created` response. -When the customer-facing update entry point is available, its handler expects the stored `advertiser_id` and the **full desired dataset list**, not just additions, using the same body as Step 4. The implementation archives **all existing Amazon stream subscriptions** and registers the requested datasets again, including retained datasets. Allow for a delivery interruption. - -After a successful managed resource update, update the pipeline: +## Update the dataset selection ```http -PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} +PATCH https://service.api.openbridge.io/service/amzadv/stream-s3/update/{remote_identity_id}/{subscription_id} Authorization: Bearer Content-Type: application/json ``` -Send `data.type: "Subscription"`, `data.id` equal to the subscription ID, and `data.attributes.product_parameters` containing the refreshed `sqs_notification_url`, `bucket`, and JSON-stringified `amz_subscription_ids`, plus JSON-stringified `selected_tables` and optional `stage_ids`. Preserve the required `advertiser_id`. Replace the complete Amazon subscription ID map, because IDs can change. The product validation limitation in Step 5 also applies to updates. Omitted product parameter keys keep their stored values. +Use the Openbridge subscription ID and send the same body as in Step 2 with the existing advertiser ID and the **full desired dataset list**. Check the list against the supported IDs before sending it. The service archives every Amazon subscription in the stored map before registering the requested datasets, including datasets you keep. This can interrupt delivery, and a failed update can leave the previous streams archived. -## Deleting a marketing stream subscription +After `200 OK`, send `PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id}` with `data.type` set to `Subscription`, `data.id` set to the Openbridge subscription ID, and the new JSON-stringified map in `data.attributes.product_parameters.amz_subscription_ids`. Replace the entire map so subsequent updates and deletes use the current Amazon IDs. -First archive the registered Amazon streams while their metadata is still available: +## Delete the pipeline ```http DELETE https://service.api.openbridge.io/service/amzadv/stream-s3/delete/{remote_identity_id}/{subscription_id} Authorization: Bearer ``` -After `204 No Content`, mark the pipeline subscription invalid: - -```http -PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id} -Authorization: Bearer -Content-Type: application/json -``` +The Service API archives the Amazon streams using the pipeline's stored metadata. After `204 No Content`, mark the Openbridge subscription invalid with `PATCH https://subscriptions.api.openbridge.io/v2/sub/{subscription_id}` and this body. [Subscription status management](./subscription-configuration.md#deleting-a-subscription) ```json { @@ -231,4 +135,13 @@ Content-Type: application/json } ``` -The reviewed delete handler archives Amazon stream subscriptions; it does not delete the managed S3 bucket or SQS queues. See [subscription status management](./subscription-configuration.md#deleting-a-subscription) for pipeline behavior. +## Clean up orphaned Amazon streams + +Use the cleanup endpoint if a registration left Amazon streams without an active Openbridge pipeline. + +```http +DELETE https://service.api.openbridge.io/service/amzadv/stream-s3/delete-orphaned/{remote_identity_id}/{advertiser_id} +Authorization: Bearer +``` + +This endpoint archives **all** streams returned for the advertiser except those already archived or suspended. It returns `204 No Content` on success and `400 Bad Request` if any active product `110` subscription exists for the remote identity.