feat: add synthetic test data and measure the search #69
+439
−7
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds a separate script that fills the catalog with fake items, so the search can be tested at a size where indexes matter. Closes #25.
What's in it
npm run db:synthetic -- add [--count N]adds 100 000 items by default (75 % elements, 25 % vaults), withSYN-ids in synthetic groups of their own (SYN-UB-1001and so on), and the search tables for them.npm run db:synthetic -- removedeletes them again. It only touches rows with aSYN-id.SYN-rows alone (it already did).addruns in one transaction and refuses to run when synthetic data already exists.syntheticData.ts) copies descriptions from the real catalog, so the word distribution is realistic, and gives the same rows every time. It has tests without a database.EXPLAIN (ANALYZE, BUFFERS)results.Measured (100 562 entries, 1.4 million words, PostgreSQL 18)
Page of 20 with indexes: 1–2 ms for browsing, filters and one-letter searches, 10 ms for "kip" together with a filter. Without indexes: 12–52 ms. A page 501 into a result costs the same as the first page with indexes (about 2 ms). The full tables are in the README.
Testing
npm run lint,tsc --noEmitand the 52 tests pass against a database (0 skipped).addis refused, that the seed leaves the rows alone, and thatremovebrings the tables back to the real catalog (562 entries, 8 896 words).Notes
docs/AI_USAGE.md.🤖 Generated with Claude Code