Skip to content

Tests: import data/loinc-testing.db into its own file and rename it into place; suites copy it - #393

Merged
grahamegrieve merged 1 commit into
HealthIntersections:mainfrom
jmandel:fix/loinc-test-db-race
Nov 14, 2026
Merged

grahamegrieve merged 1 commit into
HealthIntersections:mainfrom
jmandel:fix/loinc-test-db-race

Conversation

@jmandel

@jmandel jmandel commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #384, where you found that cs-loinc.test.js re-imports data/loinc-testing.db while cs-loinc-classtype, cs-worker-readonly and cluster-worker-readonly copy it.

I ran those four suites together in 4 jest workers (npx jest -w 4 ...), 10 times on each side:

runs that passed what failed
main (38d3ffe) 0 of 10 cs-worker-readonly and cluster-worker-readonly every time (4 tests, SQLITE_ERROR: no such table: Codes / Languages). They copied the file while it was deleted or half re-imported.
this PR 10 of 10 none (91 of 91 tests each run)

In the main runs the file already existed from an earlier run. On a fresh checkout, the first main run also failed cs-loinc-classtype, because the file didn't exist yet. On this branch the first run started with no data/ directory, and it passed.

The change

New tests/loinc-testing-db.js:

  • importLoincTestingDb() imports the test subset into a temporary file next to data/loinc-testing.db, sets journal_mode = DELETE so the whole database is in that one file, and renames it into place. A suite copying it then gets the old database or the new one, never part of one. It also removes any -wal/-shm that an older run left next to the file.
  • loincTestingDbCopy(dir) copies the file into dir, importing it first if no suite has done so yet. Any suite can therefore run alone, or first.

The suites:

  • cs-loinc.test.js imports through the helper. It no longer deletes the file first, and it opens the file read-only for its table checks. Its provider tests (LOINC Provider) now run on a copy.
  • cs-loinc-classtype, cs-worker-readonly and cluster-worker-readonly get their copies from the helper.
  • ancestors-of and closure-providers used to open the shared file in place. They now use copies too, which they delete afterwards. Loading a LOINC provider read-write sets WAL and creates indexes, so it was writing to the file while other suites copied it.
  • The import now goes to the repository's data/, where every reader already looked. Before, cs-loinc.test.js wrote to FHIRSMITH_DATA_DIR when that was set, and the readers then found a stale file or none. With it set, main's tests/cs run fails cs-loinc, cs-loinc-classtype and cs-worker-readonly.

jest has no globalSetup here (maxWorkers: 1, setupFilesAfterEnv only), so I made the import lazy rather than adding a global step to every run.

Runs (on 38d3ffe)

  • The race runs above, 10 on each side.
  • tests/cs + tests/tx + tests/server/cluster-worker-readonly.test.js, except the test cases, with FHIRSMITH_DATA_DIR set: 3,240 passed, 87 skipped, 38 failed. Main has 116 failures in that harness. The 78 tests that pass now are in cs-loinc (71), cs-worker-readonly (4) and cs-loinc-classtype (3). The 38 failures are on main too:
    • cs-ndc (37) has the same data-directory mismatch for ndc-testing.db, which I left alone.
    • cs-rxnorm-archive-index (1) expects a chmod'ed file to be read-only, which it isn't under the harness's unshare -r user namespace.
  • tests/tx/test-cases.test.js: 5,251 of 5,251 passed, with validator 7.0.5.

🤖 Generated with Claude Code

…amed into place (HealthIntersections#384)

4 suites in 4 jest workers, 10 runs: 0/10 passed before (cs-worker-readonly
and cluster-worker-readonly copied a half-written file: "no such table:
Codes"), 10/10 after.

- tests/loinc-testing-db.js: importLoincTestingDb imports to a temporary
  file next to data/loinc-testing.db, sets journal_mode DELETE, renames it
  into place; loincTestingDbCopy copies it, importing first if it's missing
- cs-loinc.test.js imports through it (no longer deleting the file first)
  and opens it read-only; its provider tests use a copy
- cs-loinc-classtype, cs-worker-readonly, cluster-worker-readonly,
  ancestors-of and closure-providers use copies: a provider loading the
  shared file set WAL and made indexes in it while others copied it
- the import goes to the repository's data/ (where every reader looked),
  not FHIRSMITH_DATA_DIR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@grahamegrieve grahamegrieve left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok thanks

@grahamegrieve
grahamegrieve merged commit 1030b58 into HealthIntersections:main Nov 14, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants