IT2810-H26 · xeniag · Nov 10, 2026 · Nov 10, 2026 · Nov 10, 2026
@@ -589,3 +589,10 @@
   - Changed outside the Gymnast page: the element page (colors, the badge, "Try in a routine", "Add to gymnast"), "Your gymnasts" (the wrapping of the title and the list), `variables.css` (`--card-hover`), and two shared parts that other pages use too: `Button` (the `icon` prop, which changes nothing when it is not given) and `DifficultyBadge` (its colors now follow the theme, also in the results of the Elements page).
   - Not done on purpose: moving the error messages of the other pages (the elements results, the element page, the login, "Your gymnasts") to the three layers, because those files belong to other pull requests and the change would conflict with them. The menus of the Elements page still have their own copies of the menu classes that `menuStyles.ts` now holds.
 - **Limits:** The end-to-end tests need the database and the backend, and are run by hand: the course git server has no runners. A screen reader was not used: that has to be done by a person. Lighthouse was started on a production build in the last round, but the run did not finish (the computer was low on memory), so there are no Lighthouse scores from that round. "Your gymnasts" was fixed after the last check in a browser and has not been measured again, and the results table of the Elements page, which uses the changed badge, has not been looked at in dark mode. The test gymnast was made with the test login in the local database and deleted again.
+
+## A shorter README (`README.md`, `docs/`)
+
+- **Tool:** Claude Code
+- **What the team did:** Took the peer feedback that the README (about 59 kB) is too long to find your way in, wrote issue #182, and asked for the reasoning to be kept.
+- **What Claude did:** Moved four long sections unchanged to their own documents (`ELEMENTS_PAGE.md`, `DATABASE_SETUP.md`, `CATALOG_SEARCH.md`, `E2E_TESTS.md`) with a short summary and a link in the README (now about half the size). Added a "Where is what" table that points from each feature to the file where it starts, and a list of contents at the top of `docs/API.md`. A script checked that every relative link and anchor in the changed files works.
+- **Limits:** The text of the moved sections was not shortened or checked again, so what was outdated in them is still outdated (that is issue #148). A first run of the move emptied `README.md` because a formatting command was cut off in a pipe; it was restored from git and run again.
@@ -4,6 +4,8 @@ This document describes the API between the React client and the backend: why we
 
 The schema itself is in [`backend/src/schema.graphql`](../backend/src/schema.graphql). It is a draft: the backend is not implemented yet. The data behind it is described in [DATA_MODEL.md](DATA_MODEL.md).
 
+**Contents:** [Why GraphQL](#why-graphql-and-not-rest) · [The schema and the database](#the-schema-and-the-database-are-different) · [Overview of the schema](#overview-of-the-schema) · [Search, filter, sort and pagination](#search-filter-sort-and-pagination) · [Scalability](#scalability) · [Images](#images) · [Identifying the user](#identifying-the-user) · [Error handling](#error-handling) · [The server](#the-server) · [Tools](#tools) · [Configuration](#configuration) · [Testing the API](#testing-the-api) · [Open questions](#open-questions)
+
 ## Why GraphQL and not REST
 
 GraphQL is required by the assignment, but it also suits this application:

@@ -0,0 +1,58 @@
+# Catalog search
+
+The catalog search is in `backend/src/catalog/search.ts`. It finds elements and vaults by the start of their words, with filters, sorting and cursor pagination, using Prisma only. How it works and why is described in "Search without raw SQL" in [docs/API.md](./API.md#search-without-raw-sql).
+
+The tests in `backend/src/catalog/` run with `npm test`. The search tests use your local database and need the seeded catalog, so run `npm run db:seed` first. They only read from the database.
+
+**Without `DATABASE_URL` in `backend/.env`, the database tests are skipped.** Vitest then reports them as skipped and the run still passes, so a passing run without a database does not mean the search was tested. Check that the output says 0 skipped.
+
+## Synthetic test data and measurements
+
+The real catalog has about 560 items, which is too small to see whether the search scales. `npm run db:synthetic` adds fake items for that:
+
+```bash
+npm run db:synthetic -- add               # 100 000 items (use --count for another number)
+npm run db:synthetic -- remove            # deletes all synthetic data again
+```
+
+The items copy descriptions from the real catalog, so the words are spread like in real data. They are spread over the four apparatus (vault items on VT, elements on the others), about 5 % have a name, and the elements have a difficulty value from 0.1 to 1.0. Every id starts with `SYN-` and every item is in a synthetic group of its own (`SYN-UB-1001` and so on), so the real catalog is never mixed up with them. The same input gives the same rows. `npm run db:seed` never adds synthetic data, and it leaves the `SYN-` rows alone. `add` stops if synthetic data already exists, and runs in one transaction, so it either adds everything or nothing. `remove` deletes only rows with a `SYN-` id. Use a local database, never the one on the virtual machine.
+
+**Measured on 2026-10-17** with 100 562 catalog entries and 1 415 215 words (562 entries are real), PostgreSQL 18 in a Docker container on a laptop, after `ANALYZE`. The time is the median of 7 calls to `searchCatalog` from Node, including Prisma, with 20 items per page and without `totalCount`. "Without indexes" means that all indexes on `catalog_entries` and `catalog_words` except the primary keys were dropped (and created again afterwards).
+
+| Query                             | With indexes (ms) | Without indexes (ms) |
+| --------------------------------- | ----------------: | -------------------: |
+| browse, sort CODE                 |               1.6 |                 20.8 |
+| browse, sort NAME                 |               1.2 |                 20.2 |
+| browse, sort DIFFICULTY (UB)      |               1.1 |                 12.6 |
+| filter apparatus UB               |               1.4 |                 12.5 |
+| filter difficulty 0.1/0.2 (BB)    |               1.1 |                 12.6 |
+| search "a" (one letter)           |               1.6 |                 48.2 |
+| search "t" (one letter)           |               1.4 |                 47.7 |
+| search "forward" (common word)    |               1.3 |                 48.0 |
+| search "kip" + UB                 |              10.1 |                 35.1 |
+| search "forward turn" (two words) |               1.8 |                 52.1 |
+| search "syn5003" (rare word)      |               1.1 |                 29.6 |
+| search "1001.5" (code)            |               4.6 |                 48.7 |
+
+Pages far into a result (100 items per page; the cursor for page N comes from following the pages before it):
+
+| Query             | Page | With indexes (ms) | Without indexes (ms) |
+| ----------------- | ---: | ----------------: | -------------------: |
+| browse, sort CODE |    1 |               1.5 |                 18.9 |
+| browse, sort CODE |  101 |               2.2 |                 39.2 |
+| browse, sort CODE |  501 |               1.9 |                 34.5 |
+| search "a"        |    1 |               1.8 |                 49.8 |
+| search "a"        |  101 |               4.0 |                 75.7 |
+| search "a"        |  501 |               1.5 |                 31.1 |
+
+`EXPLAIN (ANALYZE, BUFFERS)` on the queries Prisma sends shows that the indexes are used:
+
+- **Page 501 of the list sorted by code:** an index scan on `catalog_entries_code_sort_id_idx` reads 99 rows and takes 0.11 ms. A page far into the list costs the same as the first page, because the cursor is a condition on the index and nothing is skipped (no `OFFSET`).
+- **A one-letter search ("a"):** an index scan on `catalog_entries_code_sort_id_idx` walks the entries in order and checks each one against `catalog_words` through its primary key (63 entries checked for 20 results). It stops as soon as the page is full, so the search does not read all 1.4 million words.
+- **Without the indexes** the same queries read the whole table: 12–52 ms for a page, and 30–76 ms for a page far into a result, and the time grows with the number of items.
+
+The tables take 63 MB (`catalog_entries`) and 222 MB (`catalog_words`) with their indexes, 185 MB of it in indexes. That is the price of the speed, at this size. Adding the synthetic data takes about 70 seconds for 100 000 items.
+
+**Reading the numbers.** These are one machine and one run, so use them to compare, not as promises. The slowest case with indexes is a search for a common word together with a filter ("kip" on uneven bars, 10 ms), where PostgreSQL has to check many words. A one-letter search that is only a trailing "s" ("s") finds nothing on purpose, because the trailing "s" is removed from a search word.
+
+Run `ANALYZE` after adding or removing many rows, or PostgreSQL plans from old statistics (`npm run db:synthetic` does not run it). To repeat the measurement, call `searchCatalog` from a small script and use `EXPLAIN (ANALYZE, BUFFERS)` on the queries Prisma logs (`log: [{ emit: 'event', level: 'query' }]`). The application code never uses raw SQL, but `EXPLAIN` is typed by hand in `psql`.
@@ -0,0 +1,144 @@
+# Database (local setup)
+
+The backend uses PostgreSQL 18. Everyone installs it directly on their own machine, without Docker, and everyone uses version 18 so that migrations behave the same for all of us.
+
+## 1. Install PostgreSQL 18
+
+**Mac**, with one of these:
+
+- **Postgres.app** (recommended): download it from [postgresapp.com](https://postgresapp.com/), move it to Applications and open it. Choose PostgreSQL 18 if you are asked, and click _Initialize_. To use `psql` in the terminal, run this once and open a new terminal:
+
+  ```bash
+  sudo mkdir -p /etc/paths.d && echo /Applications/Postgres.app/Contents/Versions/latest/bin | sudo tee /etc/paths.d/postgresapp
+  ```
+
+- **Homebrew:**
+
+  ```bash
+  brew install postgresql@18
+  brew services start postgresql@18
+  ```
+
+**Windows:** download the installer from EDB through [postgresql.org/download/windows](https://www.postgresql.org/download/windows/) and choose version 18. Keep the default port 5432, and write down the password you choose for the `postgres` user. `psql` is available as _SQL Shell (psql)_ in the Start menu.
+
+## 2. Create a user and a database
+
+Open `psql` as the administrator:
+
+- Mac: `psql postgres`
+- Windows: _SQL Shell (psql)_, log in as `postgres`
+
+Then create a user and a database, both called `cop`. Choose your own password:
+
+```sql
+CREATE USER cop WITH PASSWORD 'choose-a-password' CREATEDB;
+CREATE DATABASE cop OWNER cop;
+```
+
+The user needs `CREATEDB` because `prisma migrate dev` creates a temporary _shadow database_ to check the migrations, and deletes it again afterwards.
+
+## 3. Connect the backend to the database
+
+Copy the example file and fill in your password:
+
+```bash
+cp backend/.env.example backend/.env
+```
+
+On Windows: `copy backend\.env.example backend\.env`.
+
+Open `backend/.env` and replace `your-password` in `DATABASE_URL`. The format is `postgresql://USER:PASSWORD@HOST:PORT/DATABASE?schema=public`. The file is ignored by Git, so your password is never committed.
+
+## 4. Install and run
+
+From the repository root:
+
+```bash
+npm ci
+npm run generate
+npm run db:migrate
+npm run db:seed
+```
+
+`npm run db:migrate` creates the tables in your database by applying the migrations in `backend/prisma/migrations/`. Run it again whenever someone has added a new migration.
+
+| Command                       | What it does                                                                                    |
+| ----------------------------- | ----------------------------------------------------------------------------------------------- |
+| `npm run generate`            | Runs both generators below. `typecheck` and `build` in `backend` run it first                   |
+| `npm run db:generate`         | Generates the Prisma client from `backend/prisma/schema.prisma`                                 |
+| `npm run codegen`             | Generates the TypeScript types for the resolvers from `backend/src/schema.graphql`              |
+| `npm run db:migrate`          | Creates a migration from changes in the schema and applies it to your database                  |
+| `npm run db:deploy`           | Applies existing migrations without creating new ones. Used on the server                       |
+| `npm run db:seed`             | Loads the catalog from `data/` into the database. See "Seeding" below                           |
+| `npm run db:synthetic -- add` | Adds 100 000 fake items for scalability tests. `remove` deletes them. See "Synthetic test data" |
+| `npm run db:studio`           | Opens Prisma Studio in the browser, to look at the data in the database                         |
+
+## 5. Check the connection
+
+To check that the backend can reach your database:
+
+1. Make sure PostgreSQL is running. In Postgres.app the server should show as running. With Homebrew, run `brew services start postgresql@18`. On Windows the service starts automatically.
+2. Open a terminal in the repository root and go to the backend folder:
+
+   ```bash
+   cd backend
+   ```
+
+3. Run:
+
+   ```bash
+   npx prisma migrate status
+   ```
+
+4. Look for these lines in the output. The first one means that the connection works, and the last one that your tables are up to date:
+
+   ```
+   Datasource "db": PostgreSQL database "cop", schema "public" at "localhost:5432"
+   ...
+   Database schema is up to date!
+   ```
+
+5. Go back to the repository root:
+
+   ```bash
+   cd ..
+   ```
+
+If the output says "Following migration have not yet been applied", the connection works but your tables are missing or out of date. Go back to the repository root and run `npm run db:migrate`. If the connection fails, Prisma prints an error such as "Can't reach database server" or "Authentication failed" instead.
+
+## 6. Seeding
+
+The seed script loads the catalog (apparatus, element groups, variant groups, elements and vaults) from the JSON files in `data/` into your database, and rebuilds the search tables. Run it from the repository root:
+
+```bash
+npm run db:seed
+```
+
+Run it again:
+
+- **after migrations**, when you have run `npm run db:migrate` or `npm run db:deploy`
+- **whenever `data/` changes**, for example after a new run of the extraction script
+
+It is safe to run as often as you like:
+
+- **Everything happens in one transaction.** If something fails, nothing is changed.
+- **A second run with the same data changes nothing.** It prints a table of what was inserted, updated, marked as removed and deleted.
+- **It checks the data first.** If something in `data/` breaks a rule in `docs/DATA_MODEL.md`, for example a variant group with different codes or a missing image, it stops with a message that names the row and the rule, and changes nothing.
+- **It never touches user data.** Coaches, gymnasts, routines and their lists are left alone.
+- **Removed elements are kept while they are in use.** An element or vault that is no longer in `data/` is marked as removed. It is deleted only if no routine or gymnast list uses it.
+
+The code is in `backend/src/seed/`. The rules for search words, including the written-out abbreviations such as `handstand` for `hstd`, are in `backend/src/catalog/words.ts`. The search uses the same rules.
+
+## 7. Look at the data
+
+To see what is in your database, open Prisma Studio from the repository root:
+
+```bash
+npm run db:studio
+```
+
+The terminal prints the address, for example "Prisma Studio is running at: http://localhost:51212". The port can change between runs. Open the address in your browser if it does not open by itself. The page has a list of all the tables. Click a table to see its rows, filter and sort them, and follow the relations, for example from an element to its variant group or from a routine to its items. Stop it with Ctrl+C in the terminal.
+
+Prisma Studio uses the connection in `backend/.env`, so there is nothing more to set up. You can also edit and delete rows there, so be careful: changes to the catalog are overwritten the next time you run `npm run db:seed`, and changes to user data are not undone.
+
+To write your own SQL queries, connect with `psql -h localhost -U cop -d cop` (on Windows, use _SQL Shell (psql)_) and enter the password from `backend/.env`. The tables have snake_case names in the database, such as `catalog_entries` and `element_group_id`. `\dt` lists the tables, `\d elements` shows the columns of a table, and `\q` quits.