diff --git a/README.md b/README.md index ba33285..8a96b57 100644 --- a/README.md +++ b/README.md @@ -12,13 +12,14 @@ The code in `src/` is the Vite starter project with one example test. The toolin ## What to review -| Document | Describes | -| ---------------------------------------------- | ---------------------------------------------------------------------------------------- | -| [docs/UI_PLANS.md](docs/UI_PLANS.md) | The user groups, user stories and pages of the application | -| [docs/design/README.md](docs/design/README.md) | Mockups of five of the pages. Mock designs only, not the final design | -| [docs/DATA_MODEL.md](docs/DATA_MODEL.md) | The entities, how they relate, the constraints and the reasoning behind the main choices | -| [docs/examples/](docs/examples/) | Invented example data that shows how each entity in the data model is structured | -| [docs/AI_USAGE.md](docs/AI_USAGE.md) | How AI was used in the project | +| Document | Describes | +| ---------------------------------------------- | ----------------------------------------------------------------------------------------- | +| [docs/UI_PLANS.md](docs/UI_PLANS.md) | The user groups, user stories and pages of the application | +| [docs/design/README.md](docs/design/README.md) | Mockups of five of the pages. Mock designs only, not the final design | +| [docs/DATA_MODEL.md](docs/DATA_MODEL.md) | The entities, how they relate, the constraints and the reasoning behind the main choices | +| [docs/API.md](docs/API.md) | The GraphQL API: why GraphQL, the schema, search and pagination, login and error handling | +| [docs/examples/](docs/examples/) | Invented example data that shows how each entity in the data model is structured | +| [docs/AI_USAGE.md](docs/AI_USAGE.md) | How AI was used in the project | ## The dataset @@ -42,6 +43,19 @@ For coaches, who log in with Google: The D-value of a routine is calculated from its elements and is not stored. Composition requirements and connection values are not included in this version. +## Why GraphQL and not REST + +GraphQL is required by the assignment, but it also suits this application. The full description of the API is in [docs/API.md](docs/API.md), and the schema is in [`backend/src/schema.graphql`](backend/src/schema.graphql). + +- **The data is nested.** A page often needs an apparatus with its groups and the items in them, or a gymnast with its routines and their elements. With REST that means several requests, or one endpoint for each page. With GraphQL the client asks for exactly the nested fields it needs in one request. +- **Less data over the network.** A list of search results only needs a code, a description and a value, while a detail page needs more. The client chooses the fields, so we do not send what is not shown. This is part of our sustainability choices. +- **One typed contract.** The schema describes every query, argument and field. We can generate TypeScript types from it, so a change in the API shows up as a type error in the client and not as a bug at runtime. +- **Search, filter, sort and pagination in one query.** `searchCatalog` takes all of them as typed arguments and runs on the server over the whole data set. Pages are fetched with cursors, which keeps every page fast whatever the size of the catalog. + +The costs are that HTTP caching is harder than with REST, because all requests go to one endpoint, and that the server must guard against expensive queries. We handle the last one with an upper limit on the page size, and we avoid repeated database queries for nested fields with DataLoader. + +How the API stays fast when the catalog grows (cursors, indexes, text search, limits and caching) is described in the scalability section of [docs/API.md](docs/API.md#scalability). + ## Technology In the repository now: diff --git a/backend/src/schema.graphql b/backend/src/schema.graphql new file mode 100644 index 0000000..13ad6d7 --- /dev/null +++ b/backend/src/schema.graphql @@ -0,0 +1,239 @@ +# GraphQL schema draft: the catalog (apparatus, groups, elements, vaults, search) +# and the user data (coaches, gymnasts, routines, login). +# +# ERROR HANDLING +# Errors are thrown as GraphQL errors with a machine-readable +# `extensions.code`. The client switches on the code, never on the message. +# +# UNAUTHENTICATED The operation needs a logged-in coach. +# NOT_FOUND The id does not exist, or belongs to another coach. +# Both cases give the same answer, so that nobody can find +# out which ids exist (constraint 9 in docs/DATA_MODEL.md). +# NAME_TAKEN The coach already has a routine with this name for this +# gymnast and apparatus (constraint 6). +# INVALID_ITEMS The items do not fit the routine: a vault outside vault, +# an element on vault, more than one vault, or an item from +# another apparatus (constraints 7 and 8). +# BAD_USER_INPUT Empty name, malformed id, `first` out of range. +# LIMIT_REACHED The coach has reached a limit: 200 gymnasts, 50 routines +# per gymnast, or 50 items in a routine. +# +# Why this way: +# - `me` is the only operation where "not logged in" is a normal answer. The +# app asks on every page load, so it returns null instead of an error. +# Every other protected operation throws UNAUTHENTICATED. +# - Codes are part of the contract, but the schema stays small. Typed result +# unions (Routine | NameTaken | ...) were considered, but they add a type +# and a fragment per error for a prototype with few failure cases. +# - Failures that the user can fix (NAME_TAKEN, INVALID_ITEMS) carry a +# readable message that the UI can show as it is. + +"""One of the four apparatus: VT, UB, BB or FX.""" +type Apparatus { + id: ID! + name: String! + groups: [ElementGroup!]! +} + +"""A group of elements or vaults on one apparatus, e.g. "1.000 Mounts".""" +type ElementGroup { + id: ID! + number: Int! + name: String! + apparatus: Apparatus! +} + +"""Fields shared by everything in the catalog: elements and vaults.""" +interface CatalogItem { + id: ID! + code: String! + description: String! + """The gymnast the item is named after, if any.""" + name: String + """Full URL of the drawing, ready to use as the src of an .""" + imageUrl: String! + elementGroup: ElementGroup! +} + +"""One element on uneven bars, balance beam or floor.""" +type Element implements CatalogItem { + id: ID! + code: String! + description: String! + name: String + """Full URL of the drawing, ready to use as the src of an .""" + imageUrl: String! + elementGroup: ElementGroup! + """Difficulty value, 0.1-1.0.""" + value: Float! + """Difficulty letter A-J, calculated from value.""" + difficultyLetter: String! + """Other ways of performing the same element (same code, same value).""" + variants: [Element!]! +} + +"""One vault from the vault table.""" +type Vault implements CatalogItem { + id: ID! + code: String! + description: String! + name: String + """Full URL of the drawing, ready to use as the src of an .""" + imageUrl: String! + elementGroup: ElementGroup! + """Fixed difficulty score, e.g. 2.6.""" + dValue: Float! + variants: [Vault!]! +} + +"""What the user can sort the catalog by.""" +enum CatalogSortField { + CODE + DIFFICULTY + NAME +} + +enum SortDirection { + ASC + DESC +} + +input CatalogSort { + field: CatalogSortField! = CODE + direction: SortDirection! = ASC +} + +input CatalogFilter { + apparatusId: ID + elementGroupId: ID + """Applies to value for elements and dValue for vaults.""" + minDifficulty: Float + maxDifficulty: Float + """Only items named after a gymnast.""" + namedOnly: Boolean +} + +type CatalogEdge { + """Opaque position marker. Pass it as `after` to get the next page.""" + cursor: String! + node: CatalogItem! +} + +type PageInfo { + hasNextPage: Boolean! + endCursor: String +} + +type CatalogConnection { + edges: [CatalogEdge!]! + pageInfo: PageInfo! + """ + Number of matches in the whole result set, not only this page. Counting is + expensive on a large result, so the server only counts when the client asks + for this field. + """ + totalCount: Int! +} + +type Query { + apparatus: [Apparatus!]! + elementGroup(id: ID!): ElementGroup + element(id: ID!): Element + vault(id: ID!): Vault + catalogItem(id: ID!): CatalogItem + + """The logged-in coach, or null when nobody is logged in.""" + me: Coach + """Throws UNAUTHENTICATED or NOT_FOUND.""" + gymnast(id: ID!): Gymnast + """Throws UNAUTHENTICATED or NOT_FOUND.""" + routine(id: ID!): Routine + """ + The D-score of a list of items, for the calculator. Open to everyone and + nothing is saved. Throws INVALID_ITEMS when the items do not fit together. + At most 50 ids, otherwise BAD_USER_INPUT. + """ + calculateDScore(itemIds: [ID!]!): Float! + + searchCatalog( + """ + One word, case-insensitive. Finds items with a word that starts with the + term, in the code, name or description. Empty or missing means no text + filter. + """ + search: String + filter: CatalogFilter + sort: CatalogSort + """Page size, 1-100. A value outside the range gives BAD_USER_INPUT.""" + first: Int! = 20 + after: String + ): CatalogConnection! +} + +"""An ISO 8601 date and time, e.g. 2026-10-01T12:00:00Z.""" +scalar DateTime + +"""A logged-in user. Google's own id is never exposed.""" +type Coach { + id: ID! + name: String! + email: String! + createdAt: DateTime! + """All the coach's gymnasts. Limited to 200 per coach by the backend.""" + gymnasts: [Gymnast!]! +} + +type Gymnast { + id: ID! + name: String! + """ + Optionally only the routines on one apparatus. Limited to 50 per gymnast + by the backend. + """ + routines(apparatusId: ID): [Routine!]! +} + +type Routine { + id: ID! + name: String! + gymnast: Gymnast! + apparatus: Apparatus! + """The elements, or the single vault, in the order of the routine.""" + items: [CatalogItem!]! + """Sum of the eight highest values, each code counted once. Calculated.""" + dScore: Float! +} + +type AuthPayload { + """Session token the client sends in the Authorization header.""" + token: String! + coach: Coach! +} + +input CreateRoutineInput { + gymnastId: ID! + apparatusId: ID! + name: String! +} + +type Mutation { + """Verifies the Google token and creates the coach on the first login.""" + login(googleToken: String!): AuthPayload! + + """Throws LIMIT_REACHED.""" + createGymnast(name: String!): Gymnast! + renameGymnast(id: ID!, name: String!): Gymnast! + """Also deletes the gymnast's routines. Returns the id of the deleted gymnast.""" + deleteGymnast(id: ID!): ID! + + """Throws NAME_TAKEN or LIMIT_REACHED.""" + createRoutine(input: CreateRoutineInput!): Routine! + """Throws NAME_TAKEN.""" + renameRoutine(id: ID!, name: String!): Routine! + deleteRoutine(id: ID!): ID! + """ + Replaces the whole list. The client sends the ids in the wanted order. + More than 50 ids throws LIMIT_REACHED. Throws INVALID_ITEMS. + """ + setRoutineItems(routineId: ID!, itemIds: [ID!]!): Routine! +} diff --git a/docs/AI_USAGE.md b/docs/AI_USAGE.md index 9541e73..34a2bba 100644 --- a/docs/AI_USAGE.md +++ b/docs/AI_USAGE.md @@ -42,3 +42,16 @@ Add a short README in docs/examples/ saying the data is invented, and link to the folder from DATA_MODEL.md. ``` + +## GraphQL schema (`backend/src/schema.graphql` and `docs/API.md`) + +- **Tool:** Claude Code +- **What the team did:** Had never used GraphQL before, so the schema was built + step by step in a conversation: catalog types first, then search and + pagination, then user data and errors. The team decided the direction, + for example schema-first, cursor pagination and how errors are reported, and + asked for the reasoning behind each choice. +- **What Claude did:** Wrote the draft schema and `docs/API.md`, explained each + GraphQL concept as it was introduced, and checked that the schema parses + with the `graphql` library. Part of the reasoning in the error handling + section comes from Claude's suggestions, which the team accepted. diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..67a81bc --- /dev/null +++ b/docs/API.md @@ -0,0 +1,297 @@ +# GraphQL API + +This document describes the API between the React client and the backend: why we chose GraphQL, what the schema contains, how search and pagination work, how users are identified, and how errors are reported. + +The schema itself is in [`backend/src/schema.graphql`](../backend/src/schema.graphql). It is a draft: the backend is not implemented yet. The data behind it is described in [DATA_MODEL.md](DATA_MODEL.md). + +## Why GraphQL and not REST + +GraphQL is required by the assignment, but it also suits this application: + +- **The data is nested.** A page often needs an apparatus, its groups and the items in them, or a gymnast with its routines and their elements. With REST this is either several requests or one endpoint per page. With GraphQL the client asks for exactly the nested fields it needs in one request. +- **Less data over the network.** A list of search results only needs a code, a description and a value. A detail page needs more. The client chooses the fields, so we do not send fields that are not shown. This is also part of our sustainability choices. +- **One typed contract.** The schema describes every query, argument and field. We can generate TypeScript types from it for the client, so a change in the API shows up as a type error and not as a bug at runtime. + +## The schema and the database are different + +The GraphQL schema is not a copy of the database tables. It describes what the client needs: + +| Database / data model | GraphQL | +| ---------------------------------- | --------------------------------------------------------------- | +| `Routine.elementIds` (list of ids) | `Routine.items` (the elements themselves, in order) | +| `VariantGroup` | `Element.variants` and `Vault.variants` | +| `Coach.googleId` | not exposed | +| D-score, difficulty letter | calculated fields: `Routine.dScore`, `Element.difficultyLetter` | + +Resolvers on the server translate between the two. Prisma is used inside the resolvers to read from PostgreSQL. The client never talks to Prisma or the database. + +## Overview of the schema + +### Catalog (open to everyone) + +- `Apparatus` and `ElementGroup` describe the structure of the Code of Points. +- `CatalogItem` is an interface with the fields that elements and vaults share (`id`, `code`, `description`, `name`, `imageUrl`, `elementGroup`). +- `Element` and `Vault` implement it. An element has `value` and `difficultyLetter`. A vault has `dValue`. Both have `variants`. +- `searchCatalog` searches, filters, sorts and pages over elements and vaults together. + +Fields that only exist on one type are fetched with a fragment: + +```graphql +{ + catalogItem(id: "VT-1.22-a") { + id + name + ... on Vault { + dValue + } + ... on Element { + value + } + } +} +``` + +### User data (coaches only) + +- `Coach`, `Gymnast` and `Routine`, with `Routine.items` in the order of the routine and a calculated `Routine.dScore`. +- `login(googleToken)` verifies the Google token and returns a session token and the coach. +- Mutations create, rename and delete gymnasts and routines. `setRoutineItems` replaces the whole list of items in a routine, so the client sends the ids in the wanted order and the server does not need separate add, remove and move operations. +- `calculateDScore(itemIds)` is for the calculator, which works without logging in. It calculates the D-score and saves nothing. + +## Search, filter, sort and pagination + +All of it happens in one query, `searchCatalog`, on the server and in the database. The client never filters or sorts only the part of the result it has loaded. + +```graphql +query { + searchCatalog( + search: "salto" + filter: { apparatusId: "BB", minDifficulty: 0.4 } + sort: { field: DIFFICULTY, direction: DESC } + first: 20 + after: "" + ) { + totalCount + pageInfo { + hasNextPage + endCursor + } + edges { + node { + id + code + description + ... on Element { + value + } + } + } + } +} +``` + +- **Search** is one word and is not case sensitive. It finds items with a word that starts with the term. An empty search means no text filter. The client should wait until the user stops typing (debounce) before it sends the query. +- **Filter** by apparatus, group, difficulty range, and whether the item is named after a gymnast. +- **Sort** by code, difficulty or name, ascending or descending. `DIFFICULTY` means `value` for elements and `dValue` for vaults. +- **Pagination** uses cursors (the Relay connection pattern: `edges`, `node`, `cursor`, `pageInfo`). `totalCount` is the number of matches in the whole result set. + +### Why cursors and not page numbers + +The requirement is that the solution works with any amount of data. With offset pagination the database has to count past all earlier rows for every page, and rows can be skipped or shown twice if data changes between two requests. A cursor says "continue after this row", so the cost does not grow with the page number, and it fits infinite scroll. The cursor is opaque to the client, which only passes the `endCursor` back as `after`. + +## Scalability + +The requirement is that the solution works with any amount of data. The catalog is the part that can grow, so that is where the design must hold. Below is what the API does about it, and what the backend must do. + +### The catalog + +- **Every list is paged with cursors**, and the page size has an upper limit of 100 in the schema documentation (`first`: 1–100). A client cannot ask for the whole catalog in one request. +- **A page costs the same on page 1 and page 10 000.** The cursor holds the sort values and the `id` of the last row, and the database continues after it with an index. Offset pagination would read and throw away all earlier rows. We check this with `EXPLAIN ANALYZE` (see Search without raw SQL). +- **Sorting must be stable.** Many items have the same difficulty, so `id` must always be the last sort key. Without it rows can be skipped or repeated between two pages. +- **Filter and sort need indexes.** `schema.prisma` declares an index for each sort field combined with `id` (`codeSort`, `difficulty`, `hasName` with `nameSort`), and for the filter columns (apparatus, group). Without them the database sorts the whole catalog for every request. +- **Sort by name needs a rule for items without a name.** Most items have no name. A cursor does not accept `nulls` placement, so the sort uses a plain `hasName` field and a `nameSort` field that is never `null` (see Search without raw SQL). +- **The text search must not scan the table.** A search for `%word%` cannot use an ordinary index. The search uses a word table with a B-tree index and a prefix match, which finds the rows that have a word starting with the search term without reading every row. A search that is correct but reads every row works on 500 rows and not on 5 million. +- **`totalCount` is only calculated when the client asks for it.** Counting a large result takes time. The resolver for the field runs only if `totalCount` is in the query, so a client that scrolls without showing a total does not pay for it. +- **Elements and vaults are different tables, but one list.** A query cannot combine two tables into one sorted and paged result. The data model keeps them apart because they work differently, so the list is read from `CatalogEntry`, a read-only table with the shared columns that the seed script fills. It never gets out of date because the catalog never changes between seed runs. +- **Variants are few.** A variant group has two or three members, so `variants` does not need paging. + +### Protecting the server + +- **No N+1 queries.** A list of 20 items that each asks for `elementGroup` and `variants` would give 41 database queries. DataLoader collects the lookups of one request into one query for each kind. This is needed for `elementGroup`, `variants`, `Gymnast.routines`, `Routine.items` and `Routine.apparatus`. +- **Limit the depth and cost of a query.** The schema has cycles (`Apparatus` → `groups` → `apparatus` → …), so a client could ask for a very deep query. The server must reject queries deeper than a fixed depth (for example 6) and, if needed, set a cost limit. +- **Limit the size of the arguments.** `calculateDScore` and `setRoutineItems` take at most 50 ids. + +### User data + +The lists of user data are not paged. They are small by their nature: a coach has some tens of gymnasts, and a gymnast has a few routines on each apparatus. The backend enforces a limit (200 gymnasts per coach, 50 routines per gymnast) so they stay small, and the limits are written in the schema. We considered cursor connections for them as well and decided against it for now. The reasons, and when to change, are in [USER_LIST_PAGINATION.md](USER_LIST_PAGINATION.md). + +### Caching + +- **On the client**, Apollo Client stores the result by type and `id`. The ids of elements and vaults are unique across both types (`UB-1.101-a`, `VT-1.22-a`), so an item that is loaded in a list is reused in the detail view without a new request. +- **On the server**, the catalog is read-only, so search results and items can be cached in memory and cleared when the seed script runs. User data must not be shared between coaches in a cache. +- **Images** are static files with long cache headers (see Images). + +### What this does not cover + +These choices keep the API and the queries cheap. A single Node process and a single PostgreSQL server on the virtual machine is the limit of what we run. For more traffic the same API can run on several Node processes, since the server keeps no state between requests (the session is a token). + +## Images + +The database stores only the file name of a drawing (the `image` field in the data model). The API gives a full URL in `imageUrl`, ready to use as the `src` of an ``. The backend builds it from a base URL in an environment variable and the file name. + +- The data does not depend on where the images are. If they move to another folder or server, we change one variable and no data. +- The client does not need to know where the images are. +- The name `imageUrl` makes it clear that this is a URL. For GraphQL, a file name and a URL are both just a string, so the name is the only thing that tells them apart. +- The images are static files served by Apache and cached by the browser. They are not sent through GraphQL, which keeps the responses small. + +The file names are not exposed in the API. The to-do list for images, the virtual machine and the frontend is in IMAGES.md in the docs folder. + +## Identifying the user + +Coaches log in with Google. The client sends the Google token to `login`. The server verifies it, creates the coach on the first login, and returns its own session token. The client sends that token in the `Authorization` header with every request. The Google token itself is not stored, only the Google id (see "Google login and `googleId`" in [DATA_MODEL.md](DATA_MODEL.md)). + +The catalog, `searchCatalog` and `calculateDScore` do not need a login. Everything about gymnasts and routines does, and a coach can only reach their own data. + +## Error handling + +Errors are thrown from the resolvers as GraphQL errors with a machine-readable code in `extensions.code`. The client reacts to the code and never to the text of the message. + +| Code | When | +| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `UNAUTHENTICATED` | The operation needs a logged-in coach. | +| `NOT_FOUND` | The id does not exist, or belongs to another coach. | +| `NAME_TAKEN` | The gymnast already has a routine with this name on this apparatus (constraint 6). | +| `INVALID_ITEMS` | The items do not fit the routine: a vault outside vault, an element on vault, more than one vault, or an item from another apparatus (constraints 7 and 8). | +| `BAD_USER_INPUT` | Empty name, malformed id, or `first` out of range. | +| `LIMIT_REACHED` | The coach has reached a limit: 200 gymnasts, 50 routines per gymnast, or 50 items in a routine. See [USER_LIST_PAGINATION.md](USER_LIST_PAGINATION.md). | + +An error response looks like this: + +```json +{ + "data": null, + "errors": [ + { + "message": "A routine with this name already exists", + "extensions": { "code": "NAME_TAKEN" } + } + ] +} +``` + +### Why we chose this + +- **`me` returns `null` when nobody is logged in.** The app asks `me` on every page load, and "not logged in" is a normal state for everyone who only uses the catalog. If it were an error, normal use would trigger the error handling and logging. Every other protected operation throws `UNAUTHENTICATED`, because there a missing login is a real problem that the UI must react to, for example by sending the user to the login page. +- **`NOT_FOUND` also for other coaches' data.** A separate "forbidden" answer would tell a user which ids exist. The same answer in both cases follows constraint 9 (a coach can only read and change their own gymnasts and routines) without leaking anything. +- **Plain GraphQL errors and not typed result unions.** A union such as `Routine | NameTaken | InvalidItems` makes the compiler force the client to handle each error, but it adds a type and a fragment per error to every mutation. For a prototype with a few failure cases this costs more than it gives. If the number of error cases grows, the unions can be introduced for single mutations without changing the rest of the schema. +- **Readable messages for errors the user can fix.** `NAME_TAKEN` and `INVALID_ITEMS` have messages that the UI can show as they are. + +The price of this choice is that the types do not force the client to handle the errors. We handle that with tests of the error cases (see the testing section in the README when it exists). + +## Tools + +These are the choices for the implementation. They are proposals until the backend is built, and the team can change them. + +### Server and types + +- **Apollo Server** for the GraphQL server. It has the most documentation, it works with Apollo Client (which we use for the cache and for the local state in the frontend), and it has Apollo Sandbox for trying queries in the browser. +- **Schema first.** We write `schema.graphql` by hand and the resolvers follow it. The file is easy to read for someone who reviews the project, and the schema is the contract between the frontend and the backend. The alternative, generating the schema from TypeScript code (code first), hides the contract in the code. +- **GraphQL Code Generator** makes the TypeScript types for the resolvers (backend) and for the queries (frontend) from the same schema file. A change in the schema then shows up as a type error in both places. + +### Search without raw SQL + +**The backend never uses raw SQL.** No `$queryRaw`, `$executeRaw` or their `Unsafe` versions. Every query goes through Prisma Client, which sends all values as parameters. This removes SQL injection by the way the code is built, and not by careful escaping. A lint rule (`no-restricted-properties` on those names) makes the build fail if someone adds a raw query. Indexes are declared in `schema.prisma` with `@@index`, and the migrations are generated by Prisma. + +The requirement is search on single words that is not case sensitive, together with filter, sort and cursors over the whole catalog. Prisma can do this if the data is shaped for it. Two things stop Prisma from doing it on the element and vault tables directly: a query cannot combine two tables into one sorted list, and a cursor only works when every sort field is a plain field of the model (a sort with `nulls` placement, or on a related record, makes it throw an error). + +**The catalog entry model.** `CatalogEntry` is a read-only table, filled by the seed script, with one row for every element and every vault: + +| Field | Content | +| ------------------------------- | ------------------------------------------------------------------------------- | +| `id` | The id of the element or vault, for example `UB-1.101-a` | +| `type` | `ELEMENT` or `VAULT` | +| `code`, `name`, `description` | Copied from the element or vault | +| `difficulty` | `value` for an element and `dValue` for a vault | +| `apparatusId`, `elementGroupId` | For the filters | +| `hasName`, `nameSort` | Whether the item is named after a gymnast, and the name in lowercase (or empty) | +| `codeSort` | The code with each part padded to the same length, so `2.1` sorts before `10.1` | + +The element and vault tables stay as the data model describes them and are the source for the detail view. The entry table is for the list. `searchCatalog` reads the entries, and the fields that only exist on one type (`value`, `dValue`, `imageUrl`, `variants`) are loaded from the element and vault tables with DataLoader. + +**Sort without `null`.** Most items have no name, so sorting by name cannot put `null` last, because a cursor does not accept `nulls` placement. The sort is `hasName` descending, then `nameSort`, then `id`. All three are plain fields and no value is `null`, so named items come first in both directions and the cursor works. + +**The word index.** A second table, `CatalogWord`, has one row for each distinct word of each entry: the entry id and the word in lowercase. The seed script fills it by splitting the code, the name and the description into words (letters and digits; a code like `3.304` is kept as one word). It has a B-tree index on `word`. The search is a Prisma filter: + +```ts +where: { + words: { + some: { + word: { + startsWith: term + } + } + } +} +``` + +- The term is cleaned to lowercase letters, digits and `.` before use. This also removes `%` and `_`, which mean something in a `LIKE` pattern. +- Both the stored words and the term are lowercase, so the search is not case sensitive without `mode: 'insensitive'`. That mode becomes `ILIKE`, which an ordinary index cannot serve. +- `startsWith` finds the rows that have a word starting with the term, so "sal" finds "salto". It works from the first letter. +- The filter, sort and cursor use the same `findMany` call, and `totalCount` uses `count` with the same `where`. + +**What the index needs.** A B-tree index can serve `LIKE 'sal%'` only if the database uses the `C` collation, or the index uses `text_pattern_ops`. The simple way is to create the database with the `C` collation (a setting when the database is created on the virtual machine, which goes in the setup instructions). This must be checked, not assumed (see below). + +**What it costs.** + +- It does not find the middle of a word ("alto" does not find "salto"), and it does not forgive spelling mistakes. This is what the requirement asks for (single words), and a search engine could add more later. +- There is no ordering by relevance. The user sorts by code, difficulty or name, which all have indexes and give a stable cursor. +- The tables hold the same words again, so the database is larger. The catalog is read-only and small compared to the cost of a slow search, and the seed script keeps the tables in step with the catalog. If the catalog changes, the seed script runs again. + +**To verify before we rely on it.** Fill a test database with a catalog many times larger than the real one (for example a million entries made from the real words), and run `EXPLAIN ANALYZE` on the queries that Prisma sends for the search, the filter and the sort, with and without a cursor. Check that the indexes are used, that a page far into the result is as fast as the first, and that a one-letter term still returns in a reasonable time. If the prefix search is too slow, the fallbacks that do not need raw SQL are to require two letters in the interface, or to limit how many words a one-letter term can match. We do not fall back to raw SQL. + +**Why not Prisma's `search` filter.** Full text search for PostgreSQL in Prisma is a preview feature (`fullTextSearchPostgres`). The documentation does not say if prefix matching is supported, and the index for it cannot be declared in `schema.prisma`, which would mean raw SQL in a migration. We do not use it. + +**The cursor.** The cursor given to the client is an opaque text that holds the sort field, the values of the sort fields for the last row, and its `id`. The server decodes and checks it before Prisma gets it. A cursor that cannot be decoded, or that was made for another sort field, gives `BAD_USER_INPUT`. + +**The database user is limited.** The backend connects with a user that can read and write the tables of the application, and cannot create users or databases. Migrations are run with a different user. + +The alternative for much larger systems is a search engine such as Meilisearch or Elasticsearch. It would be a second service on the virtual machine to install, start and keep in step with the database, and the assignment asks for the database on the virtual machine. For a catalog of this size it adds work and no benefit, so we do not use it. + +### N+1 queries + +We use **DataLoader**, which collects the lookups of one request and sends one query. Prisma already batches `findUnique` calls made at the same time, so the loaders matter most for lists of related rows, which Prisma does not combine: `Gymnast.routines`, `Element.variants` and `Routine.items`. A loader must return the rows in the same order as the ids it got (for `Routine.items`, in the order of the routine). Create the loaders for each request, so that data does not leak between users. + +## Configuration + +The settings are read from environment variables, and an example file with all names (but without secrets) is part of the repository. The names are proposals: + +| Variable | Used for | +| ------------------ | -------------------------------------------------------- | +| `PORT` | The port of the server, `3001` | +| `DATABASE_URL` | The PostgreSQL connection for Prisma | +| `IMAGE_BASE_URL` | The address where the drawings are served (see Images) | +| `GOOGLE_CLIENT_ID` | Checks that a Google token was made for this application | +| `SESSION_SECRET` | The key that signs the session tokens | + +The course staff must be able to start the backend, so the README (or a file readable with `sudo` on the virtual machine) must list how to set these. + +## Testing the API + +Vitest runs the API tests against a test database, with queries sent to the server. The cases that matter most for this API: + +- **Search:** not case sensitive, no text gives all items, and elements and vaults appear in the same list. +- **Pagination:** every item is returned once when we follow the cursors to the end. This must also hold when many items have the same sort value, and for items without a name when sorting by name. `hasNextPage` is `false` on the last page. +- **Filter and sort:** done over the whole catalog, not only the loaded page. +- **Errors:** each code in the table is returned for the right case. A coach that asks for another coach's gymnast gets `NOT_FOUND`, and a request without a login gets `UNAUTHENTICATED` (but `me` gives `null`). +- **Injection:** a search term, an `apparatusId` and a cursor that contain SQL (for example a quote followed by DROP TABLE) are handled as plain values and give no results or `BAD_USER_INPUT`, and the tables are still there afterwards. A cursor made for another sort field is rejected. The lint rule against raw queries passes. +- **Constraints 6–8:** a routine name that exists, a vault in a beam routine, more than one vault, and an item from the wrong apparatus. +- **Limits:** `first` of 0 and 101, more than 50 ids, the 201st gymnast, and a query that is too deep. +- **The D-score:** the eight highest values, each code counted once, and the single vault on vault. + +The end-to-end tests (a long sequence of actions in the browser) are described separately in the README when they exist. + +## Open questions + +- Should the lists of gymnasts and routines become cursor connections? Decided against for now, with the triggers for changing, in [USER_LIST_PAGINATION.md](USER_LIST_PAGINATION.md). +- How long does a session token live, and how does the client renew it? A short life limits the damage if a token is stolen, but makes the coach log in again more often. +- What is the `alt` text of the drawings? See [IMAGES.md](IMAGES.md) when it is merged. diff --git a/docs/DATA_MODEL.md b/docs/DATA_MODEL.md index 252cb66..73dd29b 100644 --- a/docs/DATA_MODEL.md +++ b/docs/DATA_MODEL.md @@ -224,3 +224,11 @@ Visitors who are not logged in should also be able to try out routines, but they ### The difficulty letter is calculated The difficulty letter (A–J) and the value (0.1–1.0) always match one to one, so we only store `value`. The letter is calculated when it is shown, and filtering by difficulty is done on `value`. + +### The image is stored as a file name + +The `image` field holds only the file name, for example `UB-1.101-a.webp`, and not a full URL. The GraphQL API turns it into a URL (`imageUrl`) by adding the address where the images are served. If the images move, no data needs to change. + +### The search uses two derived tables + +The catalog is searched through two extra tables that are not part of the diagram above, because they hold no new information: `CatalogEntry` (one row for each element and vault, with the shared columns and the sort keys) and `CatalogWord` (the lowercase words of each entry, for the word search). The seed script fills both from the catalog, and they are read-only. They exist because a database query cannot combine the element and vault tables into one sorted list with a cursor, and because the backend does not use raw SQL. The fields and the reasoning are in [API.md](API.md#search-without-raw-sql). diff --git a/docs/USER_LIST_PAGINATION.md b/docs/USER_LIST_PAGINATION.md new file mode 100644 index 0000000..233a7d7 --- /dev/null +++ b/docs/USER_LIST_PAGINATION.md @@ -0,0 +1,61 @@ +# Should the lists of user data use cursors? + +This document records a decision about the GraphQL schema in [`backend/src/schema.graphql`](../backend/src/schema.graphql): whether the lists of gymnasts and routines should be cursor connections, like the search in the catalog, or stay plain lists. + +## The question + +The catalog is paged with cursors (`searchCatalog`), because it can grow without limit. Three lists of user data are plain lists: + +| Field | Type | Limit set by the backend | +| ------------------ | ----------------- | ------------------------ | +| `Coach.gymnasts` | `[Gymnast!]!` | 200 per coach | +| `Gymnast.routines` | `[Routine!]!` | 50 per gymnast | +| `Routine.items` | `[CatalogItem!]!` | 50 per routine | + +The requirement is that the solution works with any amount of data, so we need to know whether these limits are a weakness. + +## Decision + +**No. The lists stay plain lists with limits, for now.** We revisit it if one of the triggers at the end of this document happens. + +## Why not cursors + +1. **The size is bounded by the domain, not by the amount of data in the system.** A coach has some tens of gymnasts, a gymnast has a few routines on each apparatus, and a routine has about eight counted elements. The system grows by getting more coaches, and that does not make any single list longer. Every query for user data starts with the coach's id, which has an index, so the cost depends on that coach's own rows and not on the size of the tables. The catalog is different: it is the same for everyone, and its size is only limited by the Code of Points. +2. **The user interface needs the whole list.** The gymnast overview, the choice of gymnast when an element is added to a routine, and the order of the elements in a routine all work on the full list. A routine is ordered and edited as a whole (`setRoutineItems` replaces the list), so paging it would make editing harder, not easier. With a cursor connection, the dropdown would have to load more pages on scroll, or the client would have to fetch all the pages anyway. +3. **Less code and fewer places to make mistakes.** A connection needs edge and page-info types, cursor handling in the backend, a cache merge rule in Apollo Client for each paged field, a "load more" interaction in the interface, and tests of all of it. For a field that holds 20 rows this is cost without a benefit. It matters more here because the team is new to GraphQL. +4. **Limits give a safe upper bound.** With a limit, a list cannot grow so large that it slows the server or the client. Without paging and without a limit it could, so the limit is what makes the plain list acceptable. + +## Why cursors could be better + +These are the real arguments for changing, and they are the reason this is a decision and not a given: + +1. **The limit is a restriction on the user.** A coach who reaches 200 gymnasts cannot add more. For one coach in a gymnastics club this is very unlikely, but it is a rule that the assignment's "any amount of data" does not mention. +2. **Changing later breaks the clients.** Changing `gymnasts: [Gymnast!]!` to `gymnasts(first, after): GymnastConnection!` changes the shape of the answer. All queries that use the field must be rewritten. This is cheap while the schema is a draft and nobody has built on it, and it gets more expensive with every component that uses the field. With generated TypeScript types the compiler shows every place that must change, which limits the damage. +3. **Someone who reads the schema may expect it.** A reviewer who sees cursors on the catalog and plain lists on the user data may ask why. That is why this document exists. + +## Triggers for changing + +Change `Coach.gymnasts` (and then `Gymnast.routines` if needed) to a cursor connection if: + +- we add accounts that share gymnasts between several coaches, for example a club that sees all its gymnasts, because then the list is no longer bounded by one coach, or +- real use or the staff's testing shows coaches who need more than the limit, or +- the interface gets a search or a sort over the gymnasts that must run on the server. + +`Routine.items` stays a plain list in any case, because it is ordered and edited as a whole. + +How it would look: + +```graphql +type Coach { + gymnasts(first: Int! = 50, after: String): GymnastConnection! +} +``` + +It would use the same `edges`, `node`, `cursor` and `pageInfo` as the catalog, with `id` as the last sort key. + +## What the backend must do + +- Enforce the limits when a gymnast or routine is created, and when items are set on a routine. The error code is `LIMIT_REACHED`, with a message the interface can show. +- Put an index on `Gymnast.coachId` and `Routine.gymnastId`, and on `Routine.apparatusId` together with `gymnastId`, so the lists are fast. +- Load the lists with DataLoader (see [API.md](API.md)) so a page that shows several gymnasts does not send one query for each. +- Test the limits: the 200th gymnast works, the 201st gives `LIMIT_REACHED`, and a coach cannot reach another coach's lists.