Skip to content

LOINC: say once at load when the database has no idx_codes_description - #392

Merged
grahamegrieve merged 1 commit into
HealthIntersections:mainfrom
jmandel:feat/loinc-index-hint
Nov 14, 2026
Merged

grahamegrieve merged 1 commit into
HealthIntersections:mainfrom
jmandel:feat/loinc-index-hint

Conversation

@jmandel

@jmandel jmandel commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #384, as you asked there: the server now notices an existing LOINC database without idx_codes_description and says so once at startup.

On a copy of LOINC 2.83 (imported before #384), it logs one warning with the command. After that command runs (1.1 s), it logs nothing.

LOINC: /path/to/Loinc_2.83.db has no index on Codes.Description, so a filter that names a part (SCALE_TYP = Doc) reads every code the first time. Add it once, while the server isn't using the file: sqlite3 /path/to/Loinc_2.83.db "CREATE INDEX IF NOT EXISTS idx_codes_description ON Codes(Description COLLATE NOCASE)"

The change

  • LoincServicesFactory.load looks for the index in sqlite_master. If it isn't there, the factory logs one warning (module tx-loinc) with the sqlite3 command, quoting the path for the shell when it needs it.
  • It warns once per file per process, however many times the file is loaded.
  • A tx-worker (readOnly) doesn't check, because the master loads the same file and gives the warning. So a cluster logs it once, not once per worker.
  • The server doesn't create the index itself, because the hint is what you asked for. (The load does already create three other indexes when the file is writable.)
  • The CHANGELOG entry from LOINC: look up a filter's part name or regex once per factory; index Codes.Description #384 now has a sentence about the warning.

--check (#390)

#390 is still open, so this only does the startup warning. Its checkLibrary reads the library yml and checks the sources' types, prefixes and local files. It doesn't open any databases, and there's no per-provider hook. Once #390 is merged, a loinc: source whose file is already in the terminology cache could get the same sqlite_master check there. That would mean opening each LOINC database during --check.

Tests

tests/cs/cs-loinc-index-hint.test.js (4 tests) loads a database that has the importer's tables but no rows and no indexes, the way a database imported before the index looks. The whole suite runs in 1.8 s, while a full test import alone takes about 4.5 s. In one full tests/cs + tests/tx run, a full import placed late in the run stalled for over 100 s. The tests check that:

  • without the index, two loads give one warning, and the command in it has the path quoted (the path contains a space);
  • with the index, nothing is logged;
  • a read-only (tx-worker) load logs nothing;
  • running the command from the warning adds the index, and a later load of the file logs nothing.

If loads aren't de-duplicated, the first test fails.

Runs (on 38d3ffe)

  • tests/cs/cs-loinc-index-hint.test.js: 4 of 4 passed.
  • tests/cs + tests/tx, except the test cases: 3,165 passed, 87 skipped, 116 failed. Main (the $batch-validate-code: one prefetchCodes per code system version for the whole batch #378 follow-up branch, which only adds a test) gives 116 failures in the same 5 suites in the same harness:
    • cs-loinc, cs-loinc-classtype, cs-worker-readonly and cs-ndc find no fixture database when FHIRSMITH_DATA_DIR is set. The LOINC fix is in the companion PR for the test database race.
    • cs-rxnorm-archive-index expects a chmod'ed file to be read-only, which it isn't under the harness's unshare -r user namespace.
  • tests/tx/test-cases.test.js: 5,251 of 5,251 passed, with validator 7.0.5.

🤖 Generated with Claude Code

HealthIntersections#384)

On a copy of LOINC 2.83 imported before HealthIntersections#384: one warning, with the
command; after running that command (1.1 s), none.

LoincServicesFactory.load looks for the index in sqlite_master and, when
it's missing, logs once per file:

  LOINC: <file> has no index on Codes.Description, so a filter that names
  a part (SCALE_TYP = Doc) reads every code the first time. Add it once,
  while the server isn't using the file: sqlite3 <file> "CREATE INDEX IF
  NOT EXISTS idx_codes_description ON Codes(Description COLLATE NOCASE)"

A tx-worker (readOnly) doesn't check: the master loads the same file.
4 tests: warns once without the index (path quoted for the shell), silent
with it, silent in a worker, and the logged command silences it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@grahamegrieve grahamegrieve left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok thanks

@grahamegrieve
grahamegrieve merged commit 2241d83 into HealthIntersections:main Nov 14, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants