feat(datasource intercom): read-only Intercom datasource, phase 1 by christophebrun-forest · Pull Request #387 · ForestAdmin/agent-ruby

christophebrun-forest · 2026-09-08T14:42:36Z

Integration branch of phase 1 of PRD-1111 — four lots already reviewed and merged here, now going to main together:

Lot	PR	Ticket
1 — foundation, tickets and conversations	#383	PRD-1112
2 — search filters and per-endpoint operator tables	#384	PRD-1118
2.5 — relations of the reference tier	#385	PRD-1145
4 — contacts, companies, promotion of the relations	#386	PRD-1120

A new gem, forest_admin_datasource_intercom, publishing nine read-only collections over the Intercom REST API. Everything outside those four lots — writes on tickets and conversations (lot 3), writes on contacts and companies (lot 4b, PRD-1152), notes, tags and segments (lot 5), the bounded group-by (lot 6), and actions, which will ship as plugins the agent opts into rather than from the collections — is out of scope and refused by name rather than half-served.

The rule the whole package is built on

A page that looks filtered, sorted, counted or resolved without being so is the one answer this datasource must not give. Every limit of the API is either translated exactly or refused with a message naming what to change, before the request leaves the process:

a condition a collection cannot express is refused, naming the lot or the endpoint restriction behind it;
a group-by is refused rather than computed over the pages a walk happened to collect;
a list or a count over a set of ids larger than a bulk read fetches is refused rather than answered with the rows, or the number, the truncation left;
a relation whose fan-out exceeds what the target resolves is refused rather than answered for the first slice of a page and left nil for the rest;
a pages.next whose cursor cannot be read is refused rather than taken for the last page;
a nesting depth past 2 or a group past 15 conditions is refused with a message naming what to simplify — Intercom answers a 400 naming neither;
a projection reaching through two relations is refused, rather than resolved for the first hop and quietly missing the column the second named;
a date filter that names no day — a time of day on its own, a month with no date — is refused rather than completed from the server's clock, which would make the answer depend on when the request ran;
a date bound inside a day — future, after_x_hours_ago, before_x_hours_ago, a time of day — is refused, since Intercom would answer the whole UTC day. The one exception is an upper bound at the present (past, the *_to_date family) on a column that holds past instants, where the wider day adds no row;
a body that is empty, or a reference list cut at the page cap, is never cached as the workspace's answer: the first is refused, the second re-read;
every route out of a pagination walk that is short of what was asked for is logged, naming the window it stopped in: the two caps, and the two defensive stops Intercom does not trigger today.

The single thing that cannot be refused is a sort on the endpoints that ignore one: Intercom accepts it and drops it silently (measured), so those columns are not sortable and a requested order is reported in the log. POST /contacts/search is the only endpoint that honours a sort, and the only collection with sortable columns — everywhere else, and on its own read-by-id route, an order asked for is reported rather than dropped.

The collections

Collection	Endpoint	Pagination
`IntercomConversation`	`GET /conversations`, `POST /conversations/search`	cursor
`IntercomTicket`	`POST /tickets/search`, `GET /tickets/{id}`	cursor
`IntercomContact`	`GET /contacts`, `POST /contacts/search`, `GET /companies/{id}/contacts`	cursor + id bulk read
`IntercomCompany`	`POST /companies/list`	offset
`IntercomAdmin`, `IntercomTeam`, `IntercomTicketType`, `IntercomTicketState`	one call each	read whole
`IntercomTeamMembership`	synthesized from `GET /teams`	read whole

Nineteen relations are declared across the set — assignee, team, state, type, contact, company, owner, plus the two directions of the team membership — and a condition through a relation is resolved by the target itself and rewritten into a foreign-key condition, or refused by name where the endpoint filters no such key.

What a relation costs differs per target, and the README carries the table: the reference tier resolves a whole page for one request, IntercomContact for one request per fifteen distinct contacts, and IntercomCompany for one request per distinct account — Intercom has no bulk read for a company. That last one is therefore the only relation with a ceiling: past MAX_RELATION_READS distinct accounts on a single read it is refused by name. Every page size a list view offers sits under it; an export, which batches a thousand rows, does not, and has to leave the column out.

The operator table is data, and it says where each row comes from

The table is a committed YAML file, one row per column per endpoint, each carrying its provenance (measured / spec), and every filter_operators in the package is derived from it — no collection writes one by hand, so a column cannot advertise a filter the translator would refuse. Every row is validated when the file loads, since the file is edited by hand from what the probe measures: an unknown operator or type, a column filed as both filterable and refused, and operators that publish nothing for the column's type all fail at boot rather than producing a schema nobody can explain. That is the mechanism, and it holds whatever the rows say.

What the rows say today is mostly spec: 18 of 89 are measured, and no endpoint has been probed end to end — all three carry measured_at: null. exe/forest_admin_intercom_probe is what turns a candidate into a measurement, one search per (field, operator) cell against a real workspace, and running it is the first thing to do on the customer's. The README ranks the rows worth watching; the top of that list is admin_assignee_id / team_assignee_id, typed string here where Intercom documents Integer — they carry the filter an ops team reaches for first.

Several measured points contradict Intercom's own documentation, and are recorded as such:

date filters are truncated in UTC, not in the workspace timezone — which is what made today return nothing, since the two bounds the toolkit rewrites it into read as "from tomorrow" and "before today";
x-ratelimit-limit is 1667 and describes the 10-second window, not the documented 10 000 a minute — the limiter paces on the instantaneous rate from the response headers;
/tickets/search refuses company_id with invalid_field although every ticket carries one;
per_page=200 is refused with invalid_per_page instead of being clamped;
a ticket carries no statistics block, so closed_at and last_responder are derived from the parts, which arrive complete in the search response.

Privacy

A conversation body is raw personal data. Nothing logs one: logs carry operations, counts, statuses and Intercom's request id. A response that fails to parse is named rather than quoted — a JSON parser opens its message with the characters it choked on, and on a 200 those are the payload. display_as=plaintext on every conversation read, and the regional host is a first-class configuration parameter.

Outside the package

Two changes in forest_admin_datasource_toolkit, both in Utils::Collection and both needed by the relation traversal:

get_field_schema raises ValidationError instead of ForestException on an unknown column, an unknown relation or a relation of the wrong type — each names a field the request asked for, which is a 400 the caller can read, not a 500 saying the agent broke. Nothing stops being caught: ValidationError descends from ForestException, so every rescue ForestAdminDatasourceToolkit::Exceptions::ForestException in the repo still catches these three. The observable change is the status — Http::ErrorTranslator maps a toolkit ValidationError onto 400 — and the toolkit, customizer and agent suites are green on it;
list_relation compacts a related list: a through row whose target the foreign collection no longer answers yielded a nil where every consumer down to the JSON:API serializer reads a row by key.

aggregate_relation counting through those same nil targets is still open, and is mergeable on main independently.

Verification

713 examples, 0 failures, 100% line coverage on the package;
bundle exec rubocop clean across the repository, 997 files;
HTTP stubbed with WebMock, every fixture hand-written from the OpenAPI 2.16 specification, none captured from a workspace;
the suite passes identically under UTC, Europe/Paris, America/New_York, Asia/Tokyo and Pacific/Auckland;
the package is registered in the five places a new gem has to be: version.rb format, gemspec MFA opt-out, .rubocop.yml, .releaserc.js, and both lists of build.yml.

Known, and documented rather than discovered

A boot-time attribute read that fails for any reason other than a 403 fails the boot. A token missing the permission still costs only the attribute columns; a timeout, a 5xx or a revoked token no longer lets a pod publish a schema missing columns its siblings have.
A relation reads its target undecorated: a permission scope or segment on the target does not narrow what a relation resolves — the behaviour of a native datasource joining a table, written down in the README.
The related list of an account is refused as soon as a permission scope or a segment is defined on IntercomContact. GET /companies/{id}/contacts returns the account's contacts whole and narrows nothing, and /contacts/search filters no company field, so there is no request that takes both halves. The refusal names the condition it could not carry alongside the account. Resolving it properly means reading the account's contact ids first and handing id IN [...] plus the rest of the tree to /contacts/search, which that endpoint does answer — and which is not in this lot.
A row whose foreign key is null matches no relation filter, negated ones included: a ticket with no assignee is not "assigned to someone other than Marie".
Ticket attributes are display-only (R7): an attribute is filtered through an id that differs per ticket type, so a union column has no single id to translate to. One collection per ticket type stays available if the customer says the trade is worth it — the ids are already kept per type.
The company_id column of a contact names the first of its accounts, and the company relation resolves that same one, while company_id equals X returns every contact of X — so an account's related list can show a contact whose company points elsewhere. The column is the payload's reading, the filter the account endpoint's.

The README carries the whole of it: collections, filtering per column with one reason per refusal, relations and what each costs, pagination tiers, rate limiting and configuration.

🤖 Generated with Claude Code

Note

Add read-only Intercom datasource (phase 1): collections, client, and query translation

Adds a new forest_admin_datasource_intercom package exposing read-only Intercom collections: Admin, Team, TeamMembership, TicketType, TicketState, Contact, Company, Conversation, and Ticket, registered in datasource.rb
Adds pagination bases (CursorCollection, OffsetCollection, FetchAllCollection) and a CursorWalker with caps of 50 pages and 7,500 records per walk
Adds an HTTP client with rate limiting, retries, reference caching, and per-page read deduplication, plus filter/condition-tree translation to Intercom search queries with operator limits (max depth 2, max 15 values per list)
Changes the toolkit's collection validators to raise ValidationError instead of ForestException for missing fields/relations, and drops nil through-relation targets in collection.rb
Behavioral Change: toolkit consumers previously rescuing ForestException for invalid collection fields must now rescue ValidationError; free-text search and unsupported aggregations on tickets/companies raise UnsupportedOperatorError

Changes since #387 opened

Added refusal of date bounds that fall inside a day in the ForestAdminDatasourceIntercom::Query::DayBounds class and passed the leaf and field through ForestAdminDatasourceIntercom::Query::FilterValue. [0f2525d]
Replaced truncation of oversized id lists with refusal in the Contact and RecordsById collections, and added an empty-Or short-circuit in OffsetCollection. [0f2525d]
Changed ForestAdminDatasourceIntercom::Client and ForestAdminDatasourceIntercom::Cache to avoid caching partial lists and to change error handling. [0f2525d]
Restricted boot-time degradation in DataAttributesIntrospector and TicketAttributesIntrospector to HTTP 403. [0f2525d]
Left names unknown (nil) when lookups fail in the Admin and Team collections. [0f2525d]
Changed ForestAdminDatasourceIntercom::Collections::Ticket.embed_timeline to keep timeline nil without parts and to warn on truncation. [0f2525d]
Updated documentation and comments. [0f2525d]
Changed the conditions under which ForestAdminDatasourceIntercom::Query::DayBounds emits the UTC-day truncation warning, and removed the local-date comparison. [e425693]
Added tests to the ForestAdminDatasourceIntercom::Query::FilterValue spec for a caller west of UTC. [e425693]
Scoped ForestAdminDatasourceIntercom::Client.fetch_all results to the read scope [a1dbbdb]
Changed ForestAdminDatasourceToolkit::Utils::Collection.list_relation to keep nil entries for through relations [a1dbbdb]
Extended narrowed company filter detection in ForestAdminDatasourceIntercom::Collections::Contact [a1dbbdb]
Updated inline comments for the category column in Ticket and TicketType schemas [a1dbbdb]

^{Macroscope summarized 5823c4b.}

PRD-1112, first lot of the Intercom datasource. Six read-only collections: conversations and tickets on the cursor tier, admins, teams, ticket types and ticket states read whole. Rows, record details, exact counts and the conversation thread work. Server-side filtering (lot 2), writes and actions (lot 3), contacts and companies (lot 4) are refused with a message naming the lot that will answer them, rather than approximated. 254 specs, 100% line coverage, rubocop clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…ables (lot 2) (#384) * fix(intercom): page id lookups and report explicit sorts Two findings from the lot 1 review, both places where a cursor collection answers something other than what it was asked for. An `id in [...]` read returned every record it fetched whatever page was asked for, so page 1 and page 2 of a related-record list rendered the same rows. The window is now cut out of the ids before they are read, which also spares the requests the discarded records cost: Intercom reads them one request each. `default_pk_sort?` read a symbol-keyed `false` as an absent key, so an explicit `?sort=-id` was taken for the ascending default the agent injects, and the warning about Intercom silently ignoring a sort never fired. Refs PRD-1118 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…385) * feat(datasource): intercom relations of the reference tier Declares the relations between the collections this datasource already serves, and answers them (lot 2.5, PRD-1145). Publishing a many-to-one is not free: the agent marks it filterable as soon as any column of its target is, so the interface offers `assignee:name` the moment the relation exists. The traversal is therefore the bulk of this change, not the declarations. * eight many-to-one relations towards Admin, Team, TicketState and TicketType, whose targets all shipped with lot 1; * IntercomTeamMembership, synthesized from GET /teams: Intercom carries the membership on both sides and exposes no resource for the pair, while a many-to-many needs a collection to travel through. Read-only, Intercom writing none; * admin_names and team_names replace the arrays of ids on Team and Admin, read only when a projection asks for them; * one label per object on a ticket: state_category and state_external_label are dropped, both being a hop away on the state relation and neither having ever been filterable; * a projection through a relation reads its target once per page; a condition through one is resolved by the target, exactly, and becomes a condition on the foreign key. Over fifteen matching ids, or on a key the endpoint does not filter, it is refused by name rather than approximated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…he relations (lot 4) (#386) * feat(intercom): contacts, companies and contact relations Publishes the two collections phase 2 is walked through, and turns the denormalized contact columns of lot 1 into a relation now that their target exists. Contacts read through the one endpoint of the API that sorts, so the measured table gained a sortable flag and the cursor tier a server-side order; the match-all predicate Tickets already sent is what routes a list view asking for an order through the search. Companies read through the third pagination tier, by offset, which is the one place the window a list view asks for maps onto what Intercom takes. In exchange it is looked up rather than searched, and anything past the two published lookups is refused by name. Contact and company custom attributes are typed from GET /data_attributes and carry api_writable for the lot that writes, though every column here is read-only. Resolving a relation condition is now bounded on the cursor tier: the target is read one record past what a group may hold, rather than walking a whole collection to refuse the fan-out afterwards. contact_email and contact_ids give way to contact_id and the contact relation, one readable label plus navigation, which is the rule lot 2.5 set for the ticket labels. No saved view can rest on them: both were already refused server-side. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

qltysh · 2026-09-08T14:43:44Z

47 new issues

Tool	Category	Rule	Count
qlty	Structure	Function with high complexity (count = 6): print_diff	24
qlty	Structure	Function with many parameters (count = 6): list_page	20
qlty	Duplication	Found 17 lines of identical code in 5 locations (mass = 76)	1
qlty	Structure	High total complexity (count = 90)	1
qlty	Structure	Function with many returns (count = 4): entry_for	1

qltysh · 2026-09-08T14:43:46Z

+  gem 'rspec', '~> 3.0'
+  gem 'simplecov', '~> 0.22', require: false
+  gem 'webmock', '~> 3.0'
+end


Found 17 lines of identical code in 4 locations (mass = 76) [qlty:identical-code]

qltysh · 2026-09-08T14:43:47Z

+      end
+    end
+  end
+end


High total complexity (count = 67) [qlty:file-complexity]

qltysh · 2026-09-08T14:43:47Z

+    # `list_key` is the key the endpoint puts its records under, and it is not
+    # `data` everywhere: `/tickets/search` answers under `tickets` (measured), so
+    # a caller names its own and `data` stays the fallback.
+    def list_page(path, per_page:, starting_after: nil, params: {}, list_key: 'data', boot: false)


Function with many parameters (count = 6): list_page [qlty:function-parameters]

qltysh · 2026-09-08T14:43:47Z

+    # collections that read them never send one, and the parameter is here for
+    # the one that does. `{ field:, ascending: }`, translated to Intercom's own
+    # spelling on the way out.
+    def search_page(path, query:, per_page:, starting_after: nil, list_key: 'data', params: {}, sort: nil)


Function with many parameters (count = 7): search_page [qlty:function-parameters]

qltysh · 2026-09-08T14:43:48Z

+      return [0, nil] if wait <= 0
+      return [0, first_warning? ? wait : nil] if wait > @max_wait
+
+      [wait, nil]


Function with high complexity (count = 7): plan_wait [qlty:function-complexity]

qltysh · 2026-09-08T14:43:48Z

+      @limit     = limit if limit
+      return if remaining.nil?
+
+      @remaining = new_window || @remaining.nil? ? remaining : [remaining, @remaining].min


Function with high complexity (count = 7): record [qlty:function-complexity]

qltysh · 2026-09-08T14:43:48Z

+        add(union, name, column, definition)
+      end
+
+      def add(union, name, column, definition)


Function with many parameters (count = 4): add [qlty:function-parameters]

Review pass on PR #387. Correctness * a relation read slices the fan-out by what the target resolves in one read, and refuses past what it resolves at all, instead of resolving the first 25 rows of a page and leaving the rest nil; * the contacts of an account with a scope or a segment filtered alongside are refused with a message naming that condition, rather than with the table reason telling the operator to read the account's contacts, which is what they asked for; * the `id in [...]` route reports the order it cannot apply, refuses a count larger than a bulk read fetches, and deduplicates its ids; * the cursor walk logs the two defensive stops it took silently; * a date filter naming no day, a time of day or a month, is refused rather than completed from the clock of whichever host ran it; * the offset tier bounds a windowed read by a record budget instead of by the page cap meant for a read with no window of its own; * a projection reaching through two relations is refused by name, like the filter that reaches that deep. Consistency * the four enrichment hooks read the projection through `column_asked?`, so an all-columns read fills the columns it publishes; * `search_fields.yml` fails at load on a column filed as both filterable and refused, and on operators that publish nothing for its type, which is what its own docstring promised; * `active_support` and `active_support/time` are required rather than relied on through an incidental require of the toolkit; * a base_url carrying credentials is masked in every `inspect`, and a cleartext one is reported as carrying the bearer in clear; * `RecordsById` and `AttributeNaming` hold what the paginated tiers and the two introspectors were duplicating, messages included. Docs: nine collections rather than eleven, 19 of 85 table rows measured, free-text search on contacts too, what each relation really costs, and every limit above. 560 examples, 0 failures, 100% line coverage; rubocop clean on 912 files; toolkit, customizer and agent suites green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

qltysh · 2026-09-09T12:26:53Z

+        !limit.nil? && count > limit
+      end
+
+      def read_targets(caller, collection, target, ids, wanted)


Function with many parameters (count = 5): read_targets [qlty:function-parameters]

qltysh · 2026-09-09T12:26:53Z

+      # caller: a page that looks like the whole answer and is not is the
+      # failure this datasource exists to avoid, so there is no route out of
+      # this walk that is short and quiet.
+      def cut_short?(page, seen_cursors, pages, records:, window:)


Function with many parameters (count = 5): cut_short? [qlty:function-parameters]

Second review pass on PR #387. Four defects, each one a place where the package promised something it did not do. * a lookup naming no record is an empty page rather than a 500. Intercom answers `GET /companies?name=` with a 404 where a search endpoint would answer an empty list, and nothing read it: an operator typing an account name that matches nothing broke the page instead of seeing no row. Read the way a record read by its id already was, and only a 404 -- anything else still raises. The related list of an account Intercom no longer answers for reads the same way; * the pinned API version is checked at boot. `Client#me` reads the header Intercom echoes the served version in, and nothing called it: the check the configuration claimed ran was code that never ran, while a workspace serving another version answers payloads of another shape. Called from the datasource on the boot connection, degrading to a warning like the three attribute reads; * the table names every column of the collections it answers for. Seven columns had no row at all -- `state_id` among them, which the README ranks second on the list to probe -- so they fell back on the generic "takes no filter on it", the one refusal an operator cannot act on; and `state_external_label` outlived by two lots the column it refused. A spec now reads the schema against the table in both directions, which is what the reasons are worth; * the probe ships with the gem. `bin/` is excluded from `spec.files`, so the tool the README makes the first step against a customer's workspace reached nobody who had not cloned the repository. It moves to `exe/forest_admin_intercom_probe` -- and with it its guard, RubyGems loading an executable from a stub of its own, where `$PROGRAM_NAME == __FILE__` would have shipped a command that exits without doing anything. Consistency * `ContactIdentity` reads the Contacts path off the table rather than writing it a second time; * the account-contacts route reports an order it cannot apply with the route as the reason and not the column: this is the one collection Intercom does sort; * the attribute families of the table stop being data nothing reads. `Endpoint#attribute_refusal` is validated at load and read by the translator, so a scope filtering on a workspace attribute is refused with the R7 arbitration rather than with the message for a column nobody declared -- those columns being named by the workspace, no row can name them; * the README figures for the measured rows are derived from the file by a spec: 18 of 89, and they cannot drift from it again. 579 examples, 0 failures, 100% line coverage; rubocop clean on 912 files; toolkit, agent and customizer suites green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

qltysh · 2026-09-10T12:51:00Z

+        end
+
+        print_operators(field, column, result[:operators])
+      end


Function with high complexity (count = 6): print_diff [qlty:function-complexity]

qltysh · 2026-09-10T12:51:00Z

+      MAX_DEPTH = 2
+      MAX_GROUP_SIZE = 15
+
+      def self.call(condition_tree, endpoint:, collection:, timezone: nil, attribute_columns: [])


Function with many parameters (count = 5): call [qlty:function-parameters]

qltysh · 2026-09-10T12:51:00Z

+      # which no row of the table can name: they are discovered at boot. They
+      # share one refusal, carried by the endpoint, and passing them here is
+      # what lets it be read instead of the generic message.
+      def initialize(endpoint:, collection:, timezone: nil, attribute_columns: [])


Function with many parameters (count = 4): initialize [qlty:function-parameters]

POST /companies/list paginates by offset and advertises its next page by number, never by cursor. Routing it through the shared page builder made next_cursor refuse that shape, so both the list and the count answered 500. The offset tier now builds its page without reading pages.next, which it never used, and the company fixtures carry the advertisement the real endpoint sends on every page but the last. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…to two (#391) * perf(datasource): cut a page of Intercom tickets from seven requests to two Intercom joins nothing, so a ticket list projecting five relations was seven sequential round trips: the search itself, the contacts read twice over, and the four workspace lists re-read on every page. Three changes, none of which serves a record from a store: * the reference lists -- teammates, teams, ticket types, ticket states -- are held for reference_cache_ttl (60s, 0 to disable). They are what four of a ticket's five relations resolve through and they change a few times a year; what the window trades is bounded staleness, and nothing else is cached. * the reads issued while one page is built are deduplicated for the length of that call and no longer. The contacts of a page were asked for twice, once for the contact_name column and once for the contact relation, from two callers that cannot see each other. * connections are reused. Seven requests to one host meant seven TLS handshakes on Faraday's default adapter; the client now builds on faraday-net_http_persistent, and falls back with a warning where it cannot be loaded rather than failing to boot. Measured in ticket_list_budget_spec: 7 requests before, 5 on the first page after a boot, 2 on every page after it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

qltysh · 2026-09-15T10:20:12Z

+        end
+
+        entry
+      end


Function with high complexity (count = 6): read [qlty:function-complexity]

qltysh · 2026-09-15T10:20:12Z

+      refuse_adapter!(adapter) unless name.respond_to?(:to_sym)
+      refuse_adapter!(adapter) unless options.nil? || namable_hash?(options)
+
+      options.nil? ? name.to_sym : [name.to_sym, options.to_h { |key, value| [key.to_sym, value] }]


Function with high complexity (count = 6): normalize_adapter [qlty:function-complexity]

Pylon landed on main and registered itself in the three files a package registers in, which is where this branch registers Intercom. The conflict was those three and nothing else. Both entries kept in all of them: the release sed, the gem push and the git assets of .releaserc.js, the test matrix and the coverage file list of build.yml, and the three cop excludes of .rubocop.yml. Verified by parsing each file back rather than by reading the diff, since a lost entry here fails silently: the sed no-ops and the gem is never published, with nothing red in CI to say so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

qltysh · 2026-09-15T11:16:06Z

Coverage Impact

⬆️ Merging this pull request will increase total coverage on main by 0.5%.

Modified Files with Diff Coverage (44)

Rating	File	% Diff	Uncovered Line #s
	...rcom/lib/forest_admin_datasource_intercom/collections/admin.rb	100.0%
	...ercom/lib/forest_admin_datasource_intercom/query/day_bounds.rb	100.0%
	...b/forest_admin_datasource_intercom/pagination/cursor_walker.rb	100.0%
	...rcom/lib/forest_admin_datasource_intercom/query/caller_zone.rb	100.0%
	...t_admin_datasource_intercom/query/condition_tree_translator.rb	100.0%
	...m/lib/forest_admin_datasource_intercom/collections/timeline.rb	100.0%
	...intercom/lib/forest_admin_datasource_intercom/configuration.rb	100.0%
	...st_admin_datasource_intercom/collections/contact/serializer.rb	100.0%
	...orest_admin_datasource_intercom/collections/team_membership.rb	100.0%
	...dmin_datasource_intercom/collections/ticket/derived_columns.rb	100.0%
	...om/lib/forest_admin_datasource_intercom/collections/contact.rb	100.0%
	...m/lib/forest_admin_datasource_intercom/query/operator_table.rb	100.0%
	...admin_datasource_intercom/collections/conversation/timeline.rb	100.0%
	...com/lib/forest_admin_datasource_intercom/query/filter_value.rb	100.0%
	...ercom/lib/forest_admin_datasource_intercom/collections/team.rb	100.0%
	...asource_intercom/lib/forest_admin_datasource_intercom/cache.rb	100.0%
	...b/forest_admin_datasource_intercom/collections/conversation.rb	100.0%
	...min_datasource_intercom/collections/conversation/serializer.rb	100.0%
	.../lib/forest_admin_datasource_intercom/collections/relations.rb	100.0%
	..._admin_datasource_intercom/collections/fetch_all_collection.rb	100.0%
	...ib/forest_admin_datasource_intercom/collections/ticket_type.rb	100.0%
	...rest_admin_datasource_intercom/exe/forest_admin_intercom_probe	100.0%
	...st_admin_datasource_intercom/collections/company/serializer.rb	100.0%
	...est_admin_datasource_intercom/collections/custom_attributes.rb	100.0%
	..._intercom/lib/forest_admin_datasource_intercom/rate_limiter.rb	100.0%
	.../forest_admin_datasource_intercom/collections/records_by_id.rb	100.0%
	...rest_admin_datasource_intercom/collections/contact_identity.rb	100.0%
	...ce_intercom/lib/forest_admin_datasource_intercom/datasource.rb	100.0%
	...om/lib/forest_admin_datasource_intercom/collections/company.rb	100.0%
	...orest_admin_datasource_intercom/collections/base_collection.rb	100.0%
	...in_datasource_intercom/lib/forest_admin_datasource_intercom.rb	100.0%
	...est_admin_datasource_intercom/collections/cursor_collection.rb	100.0%
	...om/lib/forest_admin_datasource_intercom/query/search_fields.rb	100.0%
	...source_intercom/lib/forest_admin_datasource_intercom/client.rb	100.0%
	...est_admin_datasource_intercom/collections/offset_collection.rb	100.0%
	...est_admin_datasource_intercom/collections/ticket/serializer.rb	100.0%
	...b/forest_admin_datasource_intercom/collections/ticket_state.rb	100.0%
	...com/lib/forest_admin_datasource_intercom/collections/ticket.rb	100.0%
	...oolkit/lib/forest_admin_datasource_toolkit/utils/collection.rb	100.0%
	..._intercom/lib/forest_admin_datasource_intercom/retry_policy.rb	100.0%
	...ib/forest_admin_datasource_intercom/schema/attribute_naming.rb	100.0%
	...n_datasource_intercom/schema/ticket_attributes_introspector.rb	100.0%
	...min_datasource_intercom/schema/data_attributes_introspector.rb	100.0%
	...urce_intercom/lib/forest_admin_datasource_intercom/throttle.rb	100.0%
	Total	100.0%

🚦 See full report on Qlty Cloud »

🛟 Help

Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.
Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.
File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)
- Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

Follows the repository relicensing in #401: the gemspec still declared GPL-3.0 and the package shipped no LICENSE file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… causes Both re-raises in must_succeed chained the rescued error as cause, which full_message and error trackers print: a parsing error quotes the payload, and Faraday's error carries the request with its bearer token. The APIError keeps only the status and the body, so with no cause the request is no longer reachable from it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…elations aggregate read relation paths such as team:name on the flat records, so every row fell into one nil group: a team leaderboard showed one blank bar holding the total. The relations the aggregation names are now nested in first, as list does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…int filters Ticket#state, Ticket#previous_state and Contact#company were offered as filters the datasource then refused. Each many-to-one now sets is_filterable from the same check the runtime refusal uses, so the agent refuses the traversal before it reaches the datasource. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…mpanies by name Neither collection declared search, so the customizer emulated one as an equality on every String column: a ticket search came back as an empty page, a company search as an error about a filter nobody wrote. Both now declare search, tickets reach their existing refusal and the offset tier refuses one before reading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

linear-code · 2026-10-29T08:18:41Z

PRD-1858

…arch takes A ticket returns its category as Customer, Back-office or Tracker, while /tickets/search takes request, task and tracker. A filter on the label the column shows was sent unchanged and matched nothing. Rows of search_fields.yml can now carry a values map from what the column shows to what the search takes; a value outside it is refused by name. The fixtures used the search vocabulary in responses, which is why no spec caught it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The three tiers read a page limit of 0 like no limit at all, and walked the collection up to its caps. The toolkit's Page#apply and the cursor walker both read it as an empty window; a single page_limit helper now keeps nil and 0 apart, and no request is sent for an empty window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Counting a set of ids read every contact it named, up to three searches of a hundred full payloads. Each slice is now one search asked for a single record, and the count is the sum of their total_count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Companies, conversations and tickets read by id cost one request each, issued one after another: up to 100 for a related list of companies, 10 for a page of conversation timelines. Client#fetch_records now reads them concurrent_reads at a time, keeping the order of the ids, the 404 as nil, the read scope on the calling thread, and the failure of the earliest id once every worker is joined. concurrent_reads defaults to 1, the previous behaviour: every read in flight holds a connection of the pool, so widening it is left to the integrator, sized against pool_size and the server's thread count as the README explains. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Measured on a real workspace: Intercom refuses an IN naming more than 50 values with a 400 naming neither the field nor the condition. Contacts were read, counted and resolved 100 ids per search, so any page naming more than 50 contacts failed. They now go 50 at a time, and an in or not in filter of more than 50 values is refused before it is sent, with a message naming the limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Measured on a real workspace: past 50 values Intercom refuses the list's length, and past 15 it refuses the composite query each value counts as an element of. The limit holds per list, on IN and NIN alike, wherever the list stands: an IN of 15 next to another condition, or two lists of 10 and 6 in nested groups, are accepted. Contacts are now read, counted and resolved 15 ids per search, and a longer list is refused before it is sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

qltysh · 2026-10-29T15:07:02Z

+        found[id] = record
+        scope[record_key(path, id, params)] = record if scope && record
+      end
+      ids.map { |id| found[id] }


Function with high complexity (count = 5): fetch_records [qlty:function-complexity]

qltysh · 2026-10-29T15:07:02Z

+
+    def record_key(path, id, params) = ['record', path, id, params]
+
+    def read_record(path, id, params, boot: false)


Function with many parameters (count = 4): read_record [qlty:function-parameters]

qltysh · 2026-10-29T15:07:02Z

+          end
+        end
+      end.each(&:join)
+      outcomes


Function with high complexity (count = 7): run_workers [qlty:function-complexity]

qltysh · 2026-10-29T15:07:02Z

+                   open_timeout: 5, timeout: 30, retry_policy: RetryPolicy.new,
+                   rate_limiter: RateLimiter.new, boot_open_timeout: 3, boot_timeout: 10,
+                   boot_retry_policy: RetryPolicy.boot,
+                   reference_cache_ttl: DEFAULT_REFERENCE_CACHE_TTL, adapter: nil,


Function with many parameters (count = 14): initialize [qlty:function-parameters]

…op base_path Configuration#base_path was called by nothing. The adapter fallback could not run for the default adapter, a hard dependency of the gem, and for a configured adapter it hid the misconfiguration behind a slower default. An adapter the process cannot load is now refused at boot, like every other invalid option. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…Collection Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Every comment block of eight lines or more is cut to three at most, keeping what the code cannot say: the measured Intercom behaviours and their figures, the workarounds, the concurrency constraints and the ticket references. Comment lines go from 2148 to 1270; no code line changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

matthv

Reviewed the whole diff (tests: 696 examples green, 100% line / 89% branch coverage). The refusals the description promises hold up and are tested with their messages. The inline comments are the spots where a result can still look filtered, complete or resolved without being so, plus two comment/convention points.

A few facts in the PR description don't match the code:

"19 of 85 are measured": search_fields.yml has 89 rows (fields + refused) with 18 measured, which is what the README and search_fields_spec.rb:180 say.
bin/probe_search_fields doesn't exist; the script is exe/forest_admin_intercom_probe.
"IntercomContact for one request per hundred distinct contacts": it's per 15 (Contact::IDS_PER_READ = FilterValue::MAX_LIST_SIZE), as the README says.
"the script that rewrites it": the probe writes evidence and says to merge it by hand (exe/forest_admin_intercom_probe:24-27). search_fields.rb:3 and search_fields.yml:16-17 repeat the same claim.
The lot mapping (writes/actions in lot 5) disagrees with the README's roadmap (lot 3 for writes and actions, lot 5 for notes/tags/segments, lot 6 for group-by).

Addresses the review of #387: - refuse a date bound inside a day, except an upper bound at the present on a column that holds past instants (past and the *_to_date family) - refuse a list by more ids than one read fetches, as the count already did - refuse an empty body instead of reading it as an empty list - cache no reference list cut at the page cap - map only transport failures to APIError, so a bug is not logged as an outage - degrade the boot introspection on a 403 only, and log the error message - leave admin_names / team_names nil when the names cannot be read - leave a ticket timeline nil when the payload carries no parts, and warn when Intercom truncated it at 500 parts - answer an Or [] tree on companies with no rows and a count of 0 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ions The client refuses an adapter it cannot load rather than falling back, the probe writes evidence merged by hand rather than rewriting the table, and actions will ship as plugins, never declared by a collection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

christophebrun-forest · 2026-11-03T17:15:42Z

Thanks for the review, matthv. Every inline point is answered in its own thread: 22474de8 for the behaviour, 0f2525d4 for the comments.

Description: corrected on all five points. It now says 18 of 89 measured rows, exe/forest_admin_intercom_probe, one request per fifteen contacts, a table edited by hand from the probe's evidence, and a lot mapping that matches the README (3 for writes on tickets and conversations, 4b for contacts and companies, 5 for notes, tags and segments, 6 for group-by, actions as plugins). The verification figures are updated too: 713 examples, 100% line coverage, rubocop clean on 997 files, green under the five timezones.

qlty: these findings are advisory (qlty check reports no blocking issue, and the repo has no qlty config). The parameter-count findings sit on Configuration#initialize and the injectable constructors, which is the public configuration surface and stays keyword-based, as in the other datasource packages. The complexity findings are left for a follow-up rather than mixed into this fix.

qltysh · 2026-11-03T17:16:00Z

+        end
+      end
+
+      [records, true]


Function with high complexity (count = 10): collect_pages [qlty:function-complexity]

qltysh · 2026-11-03T17:16:00Z

+      # mapping it would have the callers that degrade log it as a transient Intercom outage.
+      raise unless RetryPolicy.connection_failure?(e)
+
+      raise APIError, "Intercom API call failed: #{operation}: #{e.class}: #{e.message}", cause: nil


Function with high complexity (count = 5): must_succeed [qlty:function-complexity]

qltysh · 2026-11-03T17:16:00Z

+
+        refuse_condition!(filter.condition_tree) unless filter&.condition_tree.nil?
+
+        listed_records(filter)


Function with high complexity (count = 5): fetch_records [qlty:function-complexity]

qltysh · 2026-11-03T17:16:00Z

+
+        refuse_condition!(filter.condition_tree) unless filter&.condition_tree.nil?
+
+        exact_count(read_offset_page(page: 1, per_page: 1))


Function with high complexity (count = 5): count_records [qlty:function-complexity]

qltysh · 2026-11-03T17:16:00Z

+        @now = now
+      end
+
+      def call(seconds, spelling, leaf, field)


Function with many parameters (count = 4): call [qlty:function-parameters]

matthv · 2026-11-04T10:02:17Z

Forest Validator · Verify fixes: ✅ 1 closed

When (UTC)	Commit	Step	Result
04/11 13:15	`a1dbbdb`	Verify fixes	✅ 1 closed · partial list, no finding read · run `387-20261104131409-matthv-verify`

1 earlier step

When (UTC)	Commit	Step	Result
04/11 10:02	`0f2525d`	Verify fixes	10 closed · 1 reopened · run `387-20261104095319-matthv-verify`

matthv

Changes requested after /verify-fixes run 387-20261104095319-matthv-verify: 1 reopened, 0 regressions.

Reopened: Sub-day date filters return a superset, usually with no warning.

A bound on the caller's midnight in a negative-offset zone has the same local and UTC date, so the widening went unreported. Warn whenever a bound is not on a UTC midnight, except the upper bound at the present. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

qltysh · 2026-11-04T10:18:46Z

+        @now = now
+      end
+
+      def call(seconds, spelling, leaf, field)


Function with many parameters (count = 4): call [qlty:function-parameters]

qltysh · 2026-11-04T10:18:46Z

+
+        return day - UTC_DAY if spelling == '>'
+
+        seconds == day ? seconds : day + UTC_DAY


Function with high complexity (count = 5): call [qlty:function-complexity]

A company_id equality sent with a search skipped the account route and reached the translator, which blamed company_id for a filter the operator never wrote. It now gets the account refusal, naming the search. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

list_relation dropped through rows whose target was nil while aggregate_relation still counted them, for every datasource. The toolkit is back to main, and the Intercom client now reads a reference list once per page, so TeamMembership and its admin relation see the same /admins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The category is read as its label (Customer, Back-office, Tracker); only the search takes request, task and tracker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

qltysh · 2026-11-04T11:30:56Z

+        causes = []
+        causes << "condition on #{others.uniq.join(", ")}" unless others.empty?
+        causes << "search #{filter.search.to_s.strip.inspect}" unless blank_search?(filter)
+        causes.empty? ? 'condition filtered alongside it' : causes.join(' and the ')


Function with high complexity (count = 5): narrowing_cause [qlty:function-complexity]

qltysh · 2026-11-04T11:30:56Z

+        @now = now
+      end
+
+      def call(seconds, spelling, leaf, field)


Function with many parameters (count = 4): call [qlty:function-parameters]

qltysh · 2026-11-04T11:30:56Z

+
+        return day - UTC_DAY if spelling == '>'
+
+        seconds == day ? seconds : day + UTC_DAY


Function with high complexity (count = 5): call [qlty:function-complexity]

christophebrun-forest · 2026-11-04T11:32:49Z

@matthv the reopened finding (Sub-day date filters return a superset) is addressed in e425693: report_utc_day now warns whenever a bound is not on a UTC midnight, on either side of UTC, except the < at the present that past and *_to_date end on. The America/New_York context in filter_value_spec pins today and a passed upper bound.

Three more commits came out of a self-review since your pass:

aead429: a search typed in a Company's contacts list now gets the account refusal, naming the search, instead of a translator error blaming company_id.
a2702d9: Utils::Collection#list_relation is back to main (no more filter_map), so many-to-many lists and counts agree again for every datasource. On the Intercom side, Client#fetch_all is now deduplicated per page, so TeamMembership and its admin relation read the same /admins.
a1dbbdb: the ticket category comments were stale; fixed.

Could you take another look?

matthv

Approved after /verify-fixes run 387-20261104131409-matthv-verify: all 11 of my review threads are closed, none reopened.

# [1.49.0](v1.48.0...v1.49.0) (2026-11-04) ### Features * **datasource intercom:** read-only Intercom datasource, phase 1 ([#387](#387)) ([8b30efc](8b30efc)), closes [#383](#383) [#384](#384) [#385](#385) [#386](#386)

forest-bot · 2026-11-04T14:32:44Z

🎉 This PR is included in version 1.49.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

christophebrun-forest and others added 4 commits September 1, 2026 17:43

qltysh Bot reviewed Sep 8, 2026

View reviewed changes

macroscopeapp Bot reviewed Sep 8, 2026

View reviewed changes

qltysh Bot reviewed Sep 9, 2026

View reviewed changes

macroscopeapp Bot reviewed Sep 9, 2026

View reviewed changes

qltysh Bot reviewed Sep 10, 2026

View reviewed changes

macroscopeapp Bot reviewed Sep 10, 2026

View reviewed changes

christophebrun-forest and others added 2 commits September 14, 2026 16:49

qltysh Bot reviewed Sep 15, 2026

View reviewed changes

macroscopeapp Bot reviewed Sep 15, 2026

View reviewed changes

Comment thread packages/forest_admin_datasource_intercom/lib/forest_admin_datasource_intercom/configuration.rb Resolved

Comment thread ...datasource_intercom/lib/forest_admin_datasource_intercom/collections/fetch_all_collection.rb Resolved

macroscopeapp Bot reviewed Sep 15, 2026

View reviewed changes

christophebrun-forest and others added 6 commits October 28, 2026 17:42

chore: merge main into the Intercom branch

f108132

chore(datasource): license the Intercom gem under Apache-2.0

8a185e4

Follows the repository relicensing in #401: the gemspec still declared GPL-3.0 and the package shipped no LICENSE file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

macroscopeapp Bot reviewed Oct 28, 2026

View reviewed changes

Comment thread ...in_datasource_intercom/lib/forest_admin_datasource_intercom/collections/offset_collection.rb Resolved

Comment thread ...orest_admin_datasource_intercom/lib/forest_admin_datasource_intercom/query/search_fields.yml Resolved

christophebrun-forest and others added 2 commits October 29, 2026 09:24

christophebrun-forest and others added 4 commits October 29, 2026 15:46

qltysh Bot reviewed Oct 29, 2026

View reviewed changes

christophebrun-forest and others added 3 commits October 29, 2026 16:13

docs(toolkit): trim the comments the Intercom branch added to Utils::…

6e86afa

…Collection Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

matthv reviewed Nov 3, 2026

View reviewed changes

christophebrun-forest and others added 2 commits November 3, 2026 18:13

qltysh Bot reviewed Nov 3, 2026

View reviewed changes

matthv requested changes Nov 4, 2026

View reviewed changes

qltysh Bot reviewed Nov 4, 2026

View reviewed changes

christophebrun-forest mentioned this pull request Nov 4, 2026

fix: load every review thread and review of a PR past 100 matthv/pr-radar#30

Merged

christophebrun-forest and others added 3 commits November 4, 2026 12:29

docs(datasource): correct the Intercom category comments

a1dbbdb

The category is read as its label (Customer, Back-office, Tracker); only the search takes request, task and tracker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

qltysh Bot reviewed Nov 4, 2026

View reviewed changes

christophebrun-forest requested a review from matthv November 4, 2026 11:32

matthv approved these changes Nov 4, 2026

View reviewed changes

christophebrun-forest merged commit 8b30efc into main Nov 4, 2026
35 checks passed

christophebrun-forest deleted the feat/datasource-intercom branch November 4, 2026 14:30

forest-bot added the released label Nov 4, 2026

+                      end
+                      entry
+                    end


		def record_key(path, id, params) = ['record', path, id, params]

		def read_record(path, id, params, boot: false)


		refuse_condition!(filter.condition_tree) unless filter&.condition_tree.nil?

		listed_records(filter)


		refuse_condition!(filter.condition_tree) unless filter&.condition_tree.nil?

		exact_count(read_offset_page(page: 1, per_page: 1))


		return day - UTC_DAY if spelling == '>'

		seconds == day ? seconds : day + UTC_DAY

Conversation

christophebrun-forest commented Sep 8, 2026 • edited by macroscopeapp Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The rule the whole package is built on

The collections

The operator table is data, and it says where each row comes from

Privacy

Outside the package

Verification

Known, and documented rather than discovered

Add read-only Intercom datasource (phase 1): collections, client, and query translation

Changes since #387 opened

Uh oh!

qltysh Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

47 new issues

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qltysh Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qltysh Bot Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

qltysh Bot Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qltysh Bot Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

christophebrun-forest commented Sep 8, 2026 •

edited by macroscopeapp Bot

Loading

qltysh Bot commented Sep 8, 2026 •

edited

Loading

qltysh Bot commented Sep 15, 2026 •

edited

Loading

matthv commented Nov 4, 2026 •

edited

Loading