Repository navigation
feat: add SQL_QUERY dashboard filter type to the schema - #3672
karl-power wants to merge 3 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 96a5282 The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🟡 Tier 3 — StandardIntroduces new logic, modifies core functionality, or touches areas with non-trivial risk. Why this tier:
Review process: Full human review — logic, architecture, edge cases. Stats
|
Add a variable-only dashboard filter whose dropdown values come from the first column of a ClickHouse query run on a connection. - Add SqlQueryDashboardFilterSchema and the isSqlQueryFilter guard - Validate the template's macros and variable references, rejecting $__sourceTable and $__filters - Export the filter's connection as a name in dashboard templates - Keep the type out of the external API and MCP for now Refs HDX-5524
989bc97 to
f94e9ab
Compare
E2E Test Results✅ All tests passed • 466 passed • 1 skipped • 1091s
Tests ran across 4 shards in parallel. |
Duplicate code analysis1 new clone introduced by this PR.
If one of these duplicates a helper that already exists, import it instead jscpd compared this branch against |
| if (isStaticListFilter(filter)) return filter; | ||
|
|
||
| if (isSqlQueryFilter(filter)) { | ||
| filter.connection = |
There was a problem hiding this comment.
🟠 major — Export swaps the SQL_QUERY connection ID for a name, but import never maps it back
In packages/app/src/DBDashboardImportPage.tsx, the Mapping effect (~L634) and onSubmit (~L1044-1064) only resolve connection names for raw SQL tiles. An exported SQL_QUERY filter is saved with connection: 'Production DB' (a name) where an ID is expected. Also, a connection missing at export time is written as '', which fails connection: z.string().min(1), so DashboardTemplateSchema rejects the whole file. Fix: add a per-filter connection mapping (match name to ID, require a selection, write connection: id on submit) in this PR, or keep the ID on export until import support lands.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
There was a problem hiding this comment.
Fixed in 93a610b: the import page now maps SQL query filter connections by name, like raw SQL tiles, and blocks the import until each one is mapped.
| return (filters ?? []).filter( | ||
| (filter): filter is ExternalCompatibleDashboardFilter => | ||
| !isAdhocFilter(filter), | ||
| !isAdhocFilter(filter) && !isSqlQueryFilter(filter), |
There was a problem hiding this comment.
🟠 major — External API/MCP updates silently delete SQL_QUERY filters they hide
GET hides SQL_QUERY filters through getExternalCompatibleFilters. But PUT (routers/external-api/v2/dashboards.ts:3632) and MCP saveDashboard.ts:386-415 replace filters with only the request's filters, so a read-modify-write round trip deletes a SQL_QUERY filter and breaks any tile that uses its $variable. Fix: carry existing internal-only filters whose IDs are not in the request into the saved list, as MCP already does for hidden PromQL filters, including the variable-name clash check.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
There was a problem hiding this comment.
existing behaviour for hidden filters. a follow up PR will expose SQL filters anyway
| filterName: filter.name, | ||
| language: 'sql', | ||
| where: filter.sqlTemplate, | ||
| detail, |
There was a problem hiding this comment.
🔵 minor — The new SQL_QUERY validation message is never shown to the user
The only caller, DBDashboardImportPage.tsx:339-343, renders {filterName}: invalid {language} query: {where} and ignores detail. The user sees the whole template marked as an invalid SQL query (even though the SQL is never parsed) and never sees the reason, such as "$__sourceTable cannot be used". Fix: render warning.detail when it is set.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
There was a problem hiding this comment.
Fixed in 96a5282: the import warning now shows detail under each filter line, for all filter types.
PR Review
4 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 3 minor 4 posted as inline comment(s) on the changed lines. Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides. |
Deep Review✅ No critical issues found. This is a clean, well-tested schema-only change. The findings below are consistency gaps and latent risks that become live when follow-up PRs wire up SQL_QUERY creation and value lookup — worth tracking now or filing as follow-ups. 🟡 P2 — recommended
🔵 P3 nitpicks (4)
Reviewers (7): correctness, testing, maintainability, project-standards, api-contract, kieran-typescript, previous-comments. Testing gaps:
No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond |
| tooltip: 'Sources the condition keys and values come from', | ||
| label: getSourceNames(filter.sources, sources), | ||
| }; | ||
| case 'SQL_QUERY': |
There was a problem hiding this comment.
🟠 major — Editing a SQL_QUERY filter opens a broken form, and saving it always fails
The list shows every filter with an edit pencil (line 184). For a SQL_QUERY filter, toFormValues in filterFormState.ts drops connection and sqlTemplate, DashboardFilterEditForm.tsx falls through to QueryExpressionFilterEditForm, and the type picker has no SQL_QUERY option. toSavedFilter then reaches the final DashboardFilterSchema.parse, which throws, and the user only sees "This filter is incomplete and could not be saved." Until the editor PR lands, hide or disable the edit action for isSqlQueryFilter(filter), or carry the SQL fields through toFormValues/toSavedFilter.
Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond
|
|
||
| if (isSqlQueryFilter(filter)) { | ||
| filter.connection = | ||
| connections.find(conn => conn.id === filter.connection)?.name ?? ''; |
There was a problem hiding this comment.
🔵 minor — Exporting a filter whose connection was deleted produces a template that cannot be imported
When the connection can't be found, the code falls back to ''. But SqlQueryDashboardFilterSchema.connection is z.string().min(1), so DashboardTemplateSchema rejects the exported file and the whole import fails. A raw-SQL tile's connection is plain z.string(), so tiles don't have this problem. Fall back to a non-empty placeholder (for example the original ID) so the import's unmapped-connection row shows up, or relax the template schema.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| ]); | ||
|
|
||
| // Propagate connection mapping changes to other RawSQL tiles with the same input connection | ||
| // Propagate connection mapping changes to other RawSQL tiles and SQL query |
There was a problem hiding this comment.
🔵 minor — The rewritten connection-propagation effect has no test for the new tile↔filter propagation
This effect now copies a mapping between raw-SQL tiles and SQL query filters that share an input connection, and it tracks two prev-refs. No test in DBDashboardImportPage.test.tsx changes one mapping and checks that the other updates. The new tests only cover auto-mapping and the missing-connection error. Add a test with a template that has a raw-SQL tile and a SQL_QUERY filter on the same unknown connection: select a connection on one, then assert the other gets it and a dirty field isn't overwritten.
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
| {warning.filterName}: invalid {warning.language} query:{' '} | ||
| {warning.where} | ||
| </Text> | ||
| {warning.detail && ( |
There was a problem hiding this comment.
🔵 minor — New warning detail text uses the raw palette color c="yellow"
The Repository conventions section 'Semantic component variants' says to use <Text variant="warning"> for status text instead of raw palette colors such as c="yellow", and to migrate nearby usages you touch. Use variant="warning" on the new detail Text (and on the sibling Text this diff rewraps).
Advisory. Fix if it is a small defect in code this PR changes; otherwise reply in the thread. Do not widen the PR or touch files it did not already change. How to respond
Summary
Adds the
SQL_QUERYdashboard filter type to the schema: a variable-only filter whose dropdown values come from the first column of a free-form ClickHouse query. This is the first of four PRs; it adds the data model and validation only, so the filter can't be created or used in the UI yet.SqlQueryDashboardFilterSchemastores aconnectionID and asqlTemplate, withisBroadcastEnabled: falseandisVariableEnabled: truefixed, since there's no column to broadcast.isSqlQueryFiltertype guard.validateDashboardFilterQueriesexpands the template with empty variable selections and flags$__sourceTable/$__filters(the filter has no source) and any macro or variable expansion error. The SQL itself isn't parsed; ClickHouse reports syntax errors when it runs.ADHOC, until they expose it.How to test on Vercel preview
N/A — non-UI change
References