Skip to content

fix: rewrite preprocessor as standalone host exe, regenerate headers - #213

Closed
samooth wants to merge 1 commit into
vrischmann:masterfrom
samooth:fix/preprocessor-rewrite
Closed

samooth wants to merge 1 commit into
vrischmann:masterfrom
samooth:fix/preprocessor-rewrite

Conversation

@samooth

@samooth samooth commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

The header preprocessor (which generates c/loadable-ext-*.h) was broken since the 0.16 Io.

Reader migration — readAlloc always failed, dump() was never called, and CI never exercised the consumer path.

Rewrite as a standalone host executable that takes paths via stdin.

Also bumps SQLite to 3.53.4, regenerates the committed headers, and fixes the CI matrix.

Depends on PR2 (feat/compat-0.16-0.17) for the cross-compile legs.

@vrischmann

Copy link
Copy Markdown
Owner

Why exactly do you change it to a standalone executable ?

@samooth
samooth force-pushed the fix/preprocessor-rewrite branch from 2c36fff to 22d264b Compare October 3, 2026 21:33
@samooth

samooth commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Good question. The standalone executable was an intermediate attempt, not the final approach.

The inline custom build step was broken in three ways: readAlloc always failed against the new 0.16/0.17 Io.Reader API, processor.dump() was never called (so the output was just a 47-byte comment banner), and the consumer path never depended on the preprocess step. I first tried moving the logic into a small host executable to sidestep the cross-version Build-step/Io API differences and to pass paths via stdin (avoiding argv quoting issues on Windows/POSIX).

After fixing the inline step instead — readFileAlloc, actually calling dump(), and adding the dependency — the standalone executable was no longer needed, so the later commits reverted to the upstream-style inline PreprocessStep. The current PR head no longer builds or uses a standalone executable; I'll squash/reword the commits so the history doesn't imply otherwise.

The build-time header preprocessor (which generates c/loadable-ext-*.h
for loadable extensions) was broken in three ways:

1. readOriginalData used readAlloc, which always failed against the
   0.16/0.17 Io.Reader API.
2. processor.dump() was never called, so the output header was just a
   comment banner.
3. the consumer path was not wired into the build, so zig build test
   never exercised it.

Fix the inline PreprocessStep to use readFileAlloc, call dump(), and
depend on the generated headers. Bump SQLite to 3.53.4, regenerate the
committed c/loadable-ext-*.h headers, and update the CI matrix.
@samooth
samooth force-pushed the fix/preprocessor-rewrite branch from aa12c3b to b49eecd Compare November 3, 2026 23:43
@vrischmann vrischmann closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants