Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 28 additions & 16 deletions README.md

Large diffs are not rendered by default.

16 changes: 8 additions & 8 deletions docs/AI_USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,10 +258,8 @@
- **What Claude did:** Wrote the client, the cache policy for `searchCatalog`, the
authorization link (one refresh and one repeat on an expired token, one refresh at a
time between tabs), the session, the error messages, the Vite proxy and the tests.
Checked it against the running backend: the first page, "load more" and the errors.
That check showed that a plain query with `after` is answered from the cache because
`after` is not part of the key, so the pages must be loaded with `fetchMore`. This is
now in the README and `docs/API.md`.
The original cache design accumulated pages; the pagination work below replaces it
with a current-page cache policy.

## Synthetic test data (`backend/src/seed/synthetic.ts`)

Expand Down Expand Up @@ -481,11 +479,13 @@
- **What the team did:** Decided that making the page work is done in parts, each with its own issue and pull request, and that the first part is the search field and the results (#117). Chose that the pull request is stacked on the design (#114) until that is merged.
- **What Claude did:** Wrote the query `searchCatalog` and the page with the results, the loading, error and empty states, the debounce and the search in the address. Moved the example rows to a fixture for the tests. Wrote the tests on the real Apollo Client with a stub for `fetch`, and checked them by breaking five behaviors one at a time (the tests fail without each of them). Checked in a browser against the real backend: the first 20 of 562, a search, no result, a search from the address, and 9 words.

## Load more elements (`src/components/elements/ElementsPage.tsx`, `src/lib/searchOperations.ts`)
## Cursor pagination on Elements (`src/pages/ElementsPage.tsx`, `src/lib/searchOperations.ts`)

- **Tool:** Claude Code
- **What the team did:** Decided that "Load more elements" is disabled when everything is shown ("Showing N of N"), and that all the work to make the page work is one issue (#110), in several pull requests.
- **What Claude did:** Wrote the query for the next page (without `totalCount`, which the API counts only when it is asked), the button with its states (loading, disabled when everything is shown, error with a new try) and the tests, and checked the tests by breaking six behaviors one at a time. Checked in a browser against the real backend: 20, 40 and 60 of 562, one result (disabled at once), and 36 of 36 for a search that has two pages.
- **What the team did:** Replaced "Load more" with Previous/Next pagination to keep only one page of catalog rows in the DOM and Apollo cache. Chose cursor history in the URL so reload, shared links, and browser Back can restore a page without changing the backend's keyset API.
- **What Claude did:** Reused the complete catalog query for every page, kept the page number and cursor stack in the URL, replaced the cached connection and garbage-collected old normalized rows, added accessible focus/status updates and retry behavior, and preserved the scroll position across an element detail visit. Added component/cache coverage including ten pages and an E2E page-2 → detail → Back journey with a no-refetch assertion. The ten-page test measured 20 table rows, 20 card entries, and 20 cached catalog rows after paging; the prior accumulating design would hold 200 result rows and render 400 responsive DOM entries for the same ten pages. Updated README, API, and sustainability documentation, including why there are no numbered pages or "last page" (keyset cursors cannot jump, and offset paging gets slower with depth). Found and fixed two problems in the run of the e2e tests: the e2e sort checks counted an empty table (shown while the next page loads) as sorted, and the flow expected a full second page for a filter that has between 21 and 40 results.
- **Review round (team feedback with Claude Code review):** The team found that the list was replaced by "Loading elements…" while the next page loaded (with a 700 ms slow API the controls disappeared and the page jumped), that the text under the list said the page three times and that "Showing 2 of 562" now meant only the page, that `totalCount` was counted on every page although it is the same for the whole search, that an `eslint-disable` for `react-hooks` had been added, and that the address grew by about 100 characters per page (3067 characters on page 29). The team chose to keep the cursor stack in `location.state` and accept that a shared link loses the Previous history. Claude kept the old rows (dimmed, `aria-busy`) and the controls while a page loads, made one line "Showing 21–40 of 562" with the page number for screen readers, asked for the count once per search with `@include(if: $withTotal)`, removed the `eslint-disable` (the effect that resets a bad page address depends on `setSearchParams` only), put one cursor in the address and the rest in the history entry with "First page" for a link without history, and fixed a sort change that left the cursor in the address. Added component tests (one count per search, Previous through the history, a shared link, `?page=3` without a cursor) and e2e tests (a reload keeps Previous and the count, a link in a new tab, and a 700 ms slow next page that keeps the rows and the controls).
- **Limits:** The e2e tests were run on the 562-entry seed (the synthetic rows were removed for the run). A screen reader was not used.

## The count of the search stops at 1001 (`backend/src/catalog/search.ts`)

Expand Down Expand Up @@ -580,4 +580,4 @@
- The result did not make the code shorter: the files under `src/components/gymnasts/` have more lines than before, because of the new files and their comments. What changed is that each decision now has one place.
- Changed outside the Gymnast page: the element page (colors, the badge, "Try in a routine", "Add to gymnast"), "Your gymnasts" (the wrapping of the title and the list), `variables.css` (`--card-hover`), and two shared parts that other pages use too: `Button` (the `icon` prop, which changes nothing when it is not given) and `DifficultyBadge` (its colors now follow the theme, also in the results of the Elements page).
- Not done on purpose: moving the error messages of the other pages (the elements results, the element page, the login, "Your gymnasts") to the three layers, because those files belong to other pull requests and the change would conflict with them. The menus of the Elements page still have their own copies of the menu classes that `menuStyles.ts` now holds.
- **Limits:** The end-to-end tests need the database and the backend, and are run by hand: the course git server has no runners. A screen reader was not used: that has to be done by a person. Lighthouse was started on a production build in the last round, but the run did not finish (the computer was low on memory), so there are no Lighthouse scores from that round. "Your gymnasts" was fixed after the last check in a browser and has not been measured again, and the results table of the Elements page, which uses the changed badge, has not been looked at in dark mode. The test gymnast was made with the test login in the local database and deleted again.
- **Limits:** The end-to-end tests need the database and the backend, and are run by hand: the course git server has no runners. A screen reader was not used: that has to be done by a person. Lighthouse was started on a production build in the last round, but the run did not finish (the computer was low on memory), so there are no Lighthouse scores from that round. "Your gymnasts" was fixed after the last check in a browser and has not been measured again, and the results table of the Elements page, which uses the changed badge, has not been looked at in dark mode. The test gymnast was made with the test login in the local database and deleted again.
7 changes: 4 additions & 3 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ The requirement is that the solution works with any amount of data. The catalog
- **Filter and sort need indexes.** `schema.prisma` declares an index for each sort field combined with `id` (`codeSort`, `difficulty`, and `hasName` with `nameSort`), and for the filter columns (apparatus, group). Without them the database sorts the whole catalog for every request. The name sort puts named items first in both directions (`hasName` descending), so ascending and descending each have their own index. The apparatus filter is often combined with a sort, so there are also indexes on apparatus with `difficulty` and with `codeSort`. The difficulty sort always has an apparatus filter, so it uses the index on apparatus and `difficulty`. The difficulty filter without an apparatus uses the index on `difficulty`. Without them, a filtered page walks past the rows of the other apparatus first, which is slow in one direction (for example vault sorted by ascending difficulty).
- **Sort by name needs a rule for items without a name.** Most items have no name. A cursor does not accept `nulls` placement, so the sort uses a plain `hasName` field and a `nameSort` field that is never `null` (see Search without raw SQL).
- **The text search must not scan the table.** A search for `%word%` cannot use an ordinary index. The search uses a word table with a B-tree index and a prefix match, which finds the rows that have a word starting with the search term without reading every row. A search that is correct but reads every row works on 500 rows and not on 5 million.
- **`totalCount` is only calculated when the client asks for it.** Counting a large result takes time: about 30 ms for the whole catalog and about 100 ms for a search in a test with 225 000 entries, and the cost grows with the catalog. The resolver for the field runs only if `totalCount` is in the query, so a client that scrolls without showing a total does not pay for it. The client should therefore ask for `totalCount` only with the first page of a new search or filter, and not with the pages that are loaded while scrolling. Together with the debounce on the search field, this keeps the count from running on every keystroke. **The count is capped:** it stops at 1001 (`MAX_TOTAL_COUNT` is 1000, and the database counts one more), so the cost does not grow with the catalog. A `totalCount` above 1000 means that there are more than 1000 matches, and the interface shows "1000+". The schema is the same (`Int!`), only the meaning of a value above 1000 is new.
- **`totalCount` is only calculated when the client asks for it.** Counting a large result takes time: about 30 ms for the whole catalog and about 100 ms for a search in a test with 225 000 entries. The resolver for the field runs only if `totalCount` is in the query. A client that does not need a total can omit it, or use `@include` to request it only on the first page. The Elements page does that: the count is the same on every page of a search, so it asks with `totalCount @include(if: $withTotal)` on the first page only and carries the number to the later pages. **The count is capped:** it stops at 1001 (`MAX_TOTAL_COUNT` is 1000, and the database counts one more), so the cost does not grow with the catalog. A `totalCount` above 1000 means that there are more than 1000 matches, and the interface shows "1000+". The schema is the same (`Int!`), only the meaning of a value above 1000 is new.
- **Elements and vaults are different tables, but one list.** A query cannot combine two tables into one sorted and paged result. The data model keeps them apart because they work differently, so the list is read from `CatalogEntry`, a read-only table with the shared columns that the seed script fills. It never gets out of date because the catalog never changes between seed runs.
- **Variants are few.** A variant group has two or three members, so `variants` does not need paging.

Expand Down Expand Up @@ -330,7 +330,7 @@ The scripts are in `backend/package.json`: `dev` and `start` run `src/index.ts`
- **`imageUrl`** is `IMAGE_BASE_URL`, the apparatus (the first part of the id) and the file name. Moving the drawings only changes `IMAGE_BASE_URL`.
- **`value` and `dValue`** are `Decimal` in the database and are converted to numbers at the end, where they are sent.
- **A search result is a catalog entry** (one per code). The element or vault behind each result is loaded in one batch by DataLoader, so a page costs a fixed number of queries (4–6 in the tests) whether it has 5 or 100 results.
- **`totalCount` is only calculated when the query asks for it**, also through fragments, and not when `@skip` or `@include` leaves it out. A client that wants the count only on the first page writes `totalCount @include(if: $firstPage)`.
- **`totalCount` is only calculated when the query asks for it**, also through fragments, and not when `@skip` or `@include` leaves it out. A client that wants the count only on the first page writes `totalCount @include(if: $withTotal)`, as the Elements page does.
- **Errors.** `BadUserInputError` and the other errors in `backend/src/errors.ts` get their code in `extensions.code`. Errors from GraphQL itself (an invalid query) keep their code. Anything else gives `INTERNAL_SERVER_ERROR` with the message "Internal server error": the details go to the log and never to the client. `formatError` reads `originalError` directly instead of using Apollo's `unwrapResolverError`, because that function fails when two copies of the `graphql` package are loaded.
- **A request body above 100 kB is refused** (413). The CORS allow-list is `CORS_ORIGIN`; a browser on another address cannot call the API.
- **`graphql` is pinned to version 16.** Apollo Server 5 needs `^16`, and the latest release is 17.
Expand Down Expand Up @@ -362,7 +362,8 @@ These are the choices for the implementation. The server is built with them (iss

Apollo Client is set up in `src/lib/apollo/` (issue #75). See "Calling the API from the frontend" in the README for how to use it.

- **`cache.ts`.** `searchCatalog` is a Relay connection. The key of the field is the search, filter and sort, so all the pages of one search are one list. The merge function puts a page with `after` after the pages already there, skips a page that arrives twice (by cursor), starts over when the first page (no `after`) arrives again, and keeps `totalCount` when a later page does not have it. `possibleTypes` lets a fragment on `CatalogItem` match `Element` and `Vault`. Elements and vaults are stored once by `__typename` and `id`, also when they are in several lists. Because `after` is not in the key, a plain query with `after` is a cache hit; `fetchMore` is the way to load the next page.
- **`cache.ts`.** `searchCatalog` is a Relay connection. The cache keeps one current page; its read policy only returns that page when search, filter, sort, page size, and `after` all match. A new page replaces the prior connection, and the Elements page runs Apollo garbage collection so normalized catalog rows no longer referenced by the current page can be removed. Every page asks for the same result fields, so a cached page is complete; only `totalCount` is left out after the first page (`@include(if: $withTotal)`), and a page that is read without it does not need it. When a later page arrives without the count, the merge keeps the count of the same search. The address stores the page number and the cursor of that page (`?page=3&after=<cursor>`); a page number without its cursor is not sufficient to restore a keyset page, so `?page=3` alone opens page 1. `possibleTypes` lets a fragment on `CatalogItem` match `Element` and `Vault`.
- **Previous / Next and "Page N", no numbered pages (issue #162).** Going back needs no API change: the cursor that starts a page is the `endCursor` of the page before it, so the client keeps the stack of earlier cursors (text only, no rows) in the history entry (`location.state`) and asks for the earlier page again. Jumping to page N, a list of numbered pages or "last page" cannot be done with keyset cursors: it would need offset paging, which gets slower the deeper the page (the database reads and drops every row before the offset), or a `before`/`last` extension of the API. "Page X of Y" would also only be exact below the `totalCount` cap of 1001. The team chose Previous / Next with the current page number, which keeps every page as cheap as the first and needs no backend change. Only the cursor of the current page is in the address (about 100 characters), so the address does not grow with the page; the stack would have been about 3 kB on page 29. The price is that a shared link to page 5 has no history: it opens page 5 and offers "First page" instead of "Previous". Back, Forward and a reload keep the history entry.
- **`authLink.ts`, `session.ts`.** Adds `Authorization: Bearer <access token>`, and on `UNAUTHENTICATED` refreshes the session once and repeats the request once. The login operations (`Login`, `RefreshSession`, `Logout`) never start a refresh. Parts of the page that refresh at the same moment share one request, and tabs follow each other through `navigator.locks`, with the refresh token read inside the lock. A refresh token that the server refuses ends the login; a network error keeps it for a later try. The access token is kept in memory, and the refresh token in `localStorage` (see the decision in "Identifying the user").
- **`api.ts`.** The `login`, `refreshSession` and `logout` calls are plain `fetch` requests, not Apollo operations, because the refresh is called by the link itself.
- **`errors.ts`.** One place for the messages of the error codes.
Expand Down
Loading