Skip to content

feat: portable text search with contains, startsWith and endsWith - #114

Merged
Makisuo merged 5 commits into
MapleTechLabs:mainfrom
artile:feat/text-search
Nov 2, 2026
Merged

Makisuo merged 5 commits into
MapleTechLabs:mainfrom
artile:feat/text-search

Conversation

@artile

@artile artile commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

A .like() pattern built from user text treats % and _ in that text as wildcards. A search for "50%" also finds "500", and there is no portable way to say "case-insensitive contains".

Closes #83.
Part of #52.

Changes

  • New string methods on Expr: .contains(value), .startsWith(value), .endsWith(value). The value is text, not a pattern. %, _ and \ in it are escaped with a backslash, and the helper adds the wildcards.
  • A string value is escaped in JS, then bound (Postgres) or written as a literal (ClickHouse). A param or expression value is escaped in SQL with three replace() calls, because its value is known only at run time.
  • { insensitive: true } (TextMatchOptions) writes ILIKE on both dialects. On ClickHouse 26.8, ILIKE folds UTF-8. On Postgres it folds by the database LC_CTYPE (ASCII only in a C locale, as in PGlite). The docs say this.
  • Postgres gets an explicit ESCAPE '\'. ClickHouse uses the backslash as its LIKE escape already, so it gets no clause (this also keeps older ClickHouse versions working).
  • New export escapeLike(text): the escape step on its own, for a pattern you build for .like().
  • Tests: compile-shape tests (src/ch/text-match.test.ts), type tests (expr.test-d.ts), a docs example test, and three new core cases (text-match, text-match-param, text-match-insensitive). The core cases run live on Postgres (PGlite) and ClickHouse, on rows that contain %, _ and \. Each case has a row that an unescaped pattern would also match, so a missing escape fails the case.
  • Docs: a "Text search" section in docs/expressions.md, and docs/reference.md updates. Changeset: minor.

Left out

  • SQLite, MySQL and D1. Upstream has no such dialect yet. SQLite needs the ESCAPE clause (it has no default escape) and its LIKE ignores case for ASCII only. MySQL needs ESCAPE '\\' and its case rules follow the collation. The escape clause is written only for non-ClickHouse dialects, so a new dialect gets it, but insensitive on those dialects needs a decision (ILIKE does not exist there; lower() on both sides is one choice).
  • No lower()-based fallback for Postgres. ILIKE keeps the Postgres semantics users expect, and a lower() rewrite would not use the same indexes (pg_trgm supports ILIKE directly).
  • No ClickHouse positionCaseInsensitiveUTF8 / startsWith() rendering. LIKE / ILIKE with escaping gives the same results, ClickHouse already rewrites LIKE '%x%' to a substring search, and one SQL shape for both dialects is simpler to review.
  • No notContains and friends. CH.not(...) covers it.

Validation

Run in the PR worktree, with a live ClickHouse 26.8.2.7 (the CI version) and EFFECT_ORM_CLICKHOUSE_* set, so no suite is skipped:

  • bun run build: pass.
  • bun run typecheck: pass.
  • bunx vitest run --testTimeout=60000 --hookTimeout=60000: 61 files passed, 1059 tests passed, 0 skipped. (The long timeouts are only for a loaded machine.)
  • node scripts/check-doc-citations.mjs: ok (25 files).
  • node scripts/check-exports-documented.mjs: ok (289 /clickhouse, 207 /postgres exports).
  • node scripts/check-doc-examples.mjs: ok (21 files typechecked).
  • bun scripts/check-package.ts: isolated tarball and CommonJS consumer ok.

Upstream CI for a fork PR waits for a maintainer to approve the run.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

artile and others added 3 commits October 25, 2026 17:36
The value is text, not a pattern: %, _ and the backslash are escaped.
A string is escaped before it is bound; a param or expression is escaped
in SQL with replace(). { insensitive: true } writes ILIKE. Postgres gets
ESCAPE '\'; ClickHouse already escapes with a backslash. escapeLike()
is the escape step on its own.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 42 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a9f26f78-77b0-4f5b-8d1a-5ed17e6c237e
Commits

Reviewing files that changed from the base of the PR and between b57faf1 and c1a79e1.

Files ignored due to path filters: 1
  • tests/__snapshots__/core-sql.test.ts.snap is excluded by !**/*.snap
Files selected for processing: 15
  • .changeset/text-search.md
  • docs/expressions.md
  • docs/reference.md
  • docs/sqlite.md
  • src/ch/dialect.ts
  • src/ch/expr.test-d.ts
  • src/ch/expr.ts
  • src/ch/param.ts
  • src/ch/text-match.test.ts
  • src/core.ts
  • src/docs-examples.test.ts
  • src/expr.ts
  • src/sql/sql-syntax.ts
  • src/sqlite/dialect.ts
  • tests/core-cases.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Makisuo and others added 2 commits November 2, 2026 16:39
Resolves src/ch/expr.ts: keeps main's likeEscape and SQLite ILIKE refusal
beside the text-match helpers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SQLite's LIKE ignores ASCII case and it has no ILIKE, so contains/startsWith/
endsWith were case-insensitive there (and the insensitive form failed to
compile). A dialect with likeIgnoresCase now gets GLOB (escaping *, ? and [)
for a case-sensitive match and lower(x) LIKE lower(pattern) ESCAPE '\' for an
insensitive one. The LIKE escape clause follows main's likeEscape dialect
flag, so Postgres, whose LIKE escapes with a backslash already, no longer gets
an explicit ESCAPE.

The text-match core cases now run on SQLite and D1 too, with rows that a
case-insensitive LIKE or an unescaped GLOB would also match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Makisuo

Makisuo commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Maintainer pass for merge-readiness (head c1a79e1):

Changes

  • Merged origin/main. The conflict was in src/ch/expr.ts. feat: SQLite and Cloudflare D1 query dialects (#49, fuzzed) #119 added likeEscape (ESCAPE '\' only where LIKE has no default escape) and made SQLite refuse ILIKE. The merge commit keeps both of those beside the helpers.
  • SQLite/D1 semantics: SQLite's LIKE ignores ASCII case, so on main contains("a_") would also match "A_B", and { insensitive: true } failed to compile (no ILIKE). I added a likeIgnoresCase syntax flag, set on the SQLite dialect and inherited by D1. Where it is set:
    • a case-sensitive match is x GLOB pattern, with GLOB's *, ? and [ escaped in brackets: in JS for a string, and with replace() in SQL for a param or expression;
    • an insensitive match is lower(x) LIKE lower(pattern) ESCAPE '\'. This does not depend on PRAGMA case_sensitive_like. Like PGlite's C locale, SQLite's built-in lower() folds ASCII only, and the docs say so.
  • The escape clause now follows main's likeEscape flag, the same as .like(). Postgres no longer gets an explicit ESCAPE '\', because its LIKE already uses the backslash as the escape character.
  • Core cases: the texts fixture now has a SQLite/D1 form, so text-match, text-match-param and text-match-insensitive run on all four dialects (plus clickhouse-join-nulls). I added rows that a case-insensitive LIKE (A_B) or an unescaped GLOB (x, whom) would also match, and conditions for [x] and o?. Snapshots are updated.
  • Updated src/ch/text-match.test.ts, the docs example test, docs/expressions.md, docs/sqlite.md and the changeset.

Verification

  • bun run build and bun run typecheck pass.
  • bun run test against live ClickHouse 26.8: 74 files and 1468 tests pass, and the doc citation, export catalog and doc example checks pass.
  • With -t text-match, the three cases pass live on ClickHouse, clickhouse-join-nulls, Postgres (PGlite), SQLite (node:sqlite) and D1 (Miniflare).
  • FUZZ_SCALE=0.3 bunx vitest run tests/fuzz: 268/269 pass. The one failure was a generated-fixtures property on clickhouse-join-nulls that passed on rerun. In the targeted reruns that did fail, the error was ClickHouse "Division by zero", the known flake that test(fuzz): fix three flaky generated-data properties #131 fixes. The fuzz generators do not use these helpers.

Makisuo added a commit that referenced this pull request Nov 2, 2026
…te migrate tests 30s

ClickHouse 26.2 returns 3 for `-5 % -4` under UNION ALL (26.8: -1).
Record it as a known issue scoped to a negative constant divisor.

pg-enums-uniques starts a PGlite per test, which timed out at 5s on CI
runners busy with the fuzz suite (#110, #113, #114, #131).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Makisuo
Makisuo merged commit bf82171 into MapleTechLabs:main Nov 2, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Portable text search: case-insensitive contains and LIKE escaping

2 participants