diff --git a/.gitignore b/.gitignore index e38fcfd..d1a9d78 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,7 @@ scripts/extract-cop/en_1.1.-WAG-Code-of-Points-2025-2028.pdf scripts/extract-cop/review.html scripts/extract-cop/.venv __pycache__ + +# Playwright (end-to-end tests) +playwright-report/ +test-results/ diff --git a/README.md b/README.md index 54480c4..a56c9c8 100644 --- a/README.md +++ b/README.md @@ -21,18 +21,18 @@ The calculator and About pages are placeholders. Adding items from the catalog t ## What to review -| Review | What to look at | -| -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| [Running application](http://it2810-29.idi.ntnu.no/project2) | Search, load more, element details, and the Your gymnasts test login | -| [ElementsPage tests](src/components/elements/ElementsPage.test.tsx) | Search, result paging, URL state, loading, and error behavior | -| [Element detail tests](src/components/elements/ElementDetailPage.test.tsx) | Catalog detail rendering and error/loading states | -| [Gymnast page tests](src/components/gymnasts/YourGymnastsPage.test.tsx) | Test login, loading, profile creation, and error handling | -| [Backend catalog tests](backend/src/catalog/search.test.ts) | Search, filters, sorting, pagination, and count limits | -| [Backend server tests](backend/src/server/server.test.ts) | GraphQL API behavior; database-backed tests require a migrated and seeded database | -| [docs/API.md](docs/API.md) | API design, schema, search, pagination, authentication, and errors | -| [docs/AI_USAGE.md](docs/AI_USAGE.md) | How AI was used and how the work was reviewed | -| [docs/ACCESSIBILITY.md](docs/ACCESSIBILITY.md) | Accessibility choices, testing, and known open points | -| [docs/SUSTAINABILITY.md](docs/SUSTAINABILITY.md) | Sustainable design choices, measurements, and what remains unmeasured | +| Review | What to look at | +| -------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| [Running application](http://it2810-29.idi.ntnu.no/project2) | Search, load more, element details, and the Your gymnasts test login | +| [ElementsPage tests](src/components/elements/ElementsPage.test.tsx) | Search, result paging, URL state, loading, and error behavior | +| [Element detail tests](src/components/elements/ElementDetailPage.test.tsx) | Catalog detail rendering and error/loading states | +| [Gymnast page tests](src/components/gymnasts/YourGymnastsPage.test.tsx) | Test login, loading, profile creation, and error handling | +| [Backend catalog tests](backend/src/catalog/search.test.ts) | Search, filters, sorting, pagination, and count limits | +| [Backend server tests](backend/src/server/server.test.ts) | GraphQL API behavior; database-backed tests require a migrated and seeded database | +| [docs/API.md](docs/API.md) | API design, schema, search, pagination, authentication, and errors | +| [docs/AI_USAGE.md](docs/AI_USAGE.md) | How AI was used and how the work was reviewed | +| [docs/ACCESSIBILITY.md](docs/ACCESSIBILITY.md) | Accessibility choices, testing, and known open points | +| [docs/SUSTAINABILITY.md](docs/SUSTAINABILITY.md) | Sustainable design choices, measurements, and what remains unmeasured | ### How to try it @@ -526,16 +526,58 @@ The tables take 63 MB (`catalog_entries`) and 222 MB (`catalog_words`) with thei Run `ANALYZE` after adding or removing many rows, or PostgreSQL plans from old statistics (`npm run db:synthetic` does not run it). To repeat the measurement, call `searchCatalog` from a small script and use `EXPLAIN (ANALYZE, BUFFERS)` on the queries Prisma logs (`log: [{ emit: 'event', level: 'query' }]`). The application code never uses raw SQL, but `EXPLAIN` is typed by hand in `psql`. +## End-to-end tests + +The assignment asks for "a form of automated end-to-end testing" of "a longer sequence of interactions", and for testing of the API. The component tests (Vitest) and the API tests in `backend/src/server/` already exist; the end-to-end tests drive the **whole application in a real browser**, against the real backend and the real database, the way a user does. They are in `e2e/` and use [Playwright](https://playwright.dev), with [axe-core](https://github.com/dequelabs/axe-core-npm/tree/develop/packages/playwright) for the accessibility checks. + +| File | What it tests | +| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `e2e/elements-flow.e2e.ts` | One long flow: search, filter by apparatus and group, "Load more", sort, open an element, go back, "Clear filters". It checks the address, the results, the title and the focus at each step. | +| `e2e/vault.e2e.ts` | The vault difficulty bands, and sorting vault by its D-score in both directions. | +| `e2e/atypical.e2e.ts` | Atypical use: an address with filters that do not exist, a search without a result, the 80-character limit, a structure that fails to load (the request is made to fail), the disabled group filter, and keyboard-only use. | +| `e2e/accessibility.e2e.ts` | axe-core on the page in its main states: filtered and sorted, a menu open, the reason for a disabled filter shown, vault, no result, dark mode, a phone (375 px) and reflow at 320 px. A violation fails the test. | +| `e2e/api.e2e.ts` | The GraphQL API over HTTP: search, filter, sort, cursors, the error codes and `Apparatus.difficultyValues`. The same requests can be sent to the virtual machine. | + +### How to run them + +They need the database to be running and seeded (see "Database (local setup)" above), and a browser: + +```bash +npm ci +npm run generate # the Prisma and GraphQL types, once +npx playwright install chromium # the browser, once (about 150 MB) +npm run e2e +``` + +`npm run e2e` starts the backend and the frontend itself (and uses them if they are already running), runs the tests, and prints the result. `npm run e2e:report` opens the report with a screenshot and a trace of every test that failed. The tests only read the catalog, so they can be run again and again. + +Settings, all optional, as environment variables: + +| Variable | Meaning | +| --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `E2E_BASE_URL` | An address that is already running, for example `http://it2810-29.idi.ntnu.no/project2`. Nothing is started, and the same tests check the real deployment (including the Apache proxy of `/project2/graphql`). | +| `E2E_FRONTEND_PORT`, `E2E_BACKEND_PORT` | The ports that the tests start the app on, 5173 and 3001 by default. Change them if another program uses them, or the tests would use that program. | +| `E2E_CHANNEL` | A browser that is already installed, for example `msedge`, instead of the Chromium that Playwright downloads. | +| `E2E_API_URL` | The GraphQL address of `e2e/api.e2e.ts`, if it is not the one that follows from the others. | + +### Choices and limits + +- **Playwright, not Cypress.** It is made for end-to-end tests, drives real Chromium, waits for the page by itself (a test says what it expects, and Playwright retries until it is true), and has an official axe-core integration. The tests are TypeScript like the rest of the project, and the files end in `.e2e.ts` so that Vitest does not pick them up. +- **The tests depend on the seeded catalog** (the data in `data/`). They avoid exact numbers where a relation is enough ("fewer than the whole catalog", "every code starts with the number of the group"), but they do assume that uneven bars, floor and vault exist with their groups and difficulty values. +- **The accessibility tests are automated checks, not a full audit.** axe-core finds what a program can find. What needs a person, such as a screen reader (NVDA, VoiceOver) and whether the order of the focus makes sense, has to be tested by hand. `region` is turned off for a state with a menu open: the menus are in a portal (the Base UI default), which axe reports as best practice, not as a WCAG failure. +- **They are not run by a server on every pull request.** The course git server has no runners (see issue #76), so they are run by the person who makes the change, and before each delivery against the virtual machine with `E2E_BASE_URL`. + ## Project structure -| Folder | Contents | -| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `backend/` | The backend package: TypeScript, Prisma, the seed script, the catalog search, the GraphQL server and the GraphQL schema | -| `data/` | The catalog data extracted from the Code of Points: JSON files and one image per element | -| `docs/` | Project plans, data model, API, accessibility, sustainability, AI usage, and catalog extraction documentation | -| `public/` | Static files served as they are, such as the favicon | -| `scripts/extract-cop/` | The Python script that extracts `data/` from the Code of Points PDF | -| `src/` | The React frontend, shared UI components, API operations, Apollo Client, and frontend tests | +| Folder | Contents | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| `backend/` | The backend package: TypeScript, Prisma, the seed script, the catalog search, the GraphQL server and the GraphQL schema | +| `data/` | The catalog data extracted from the Code of Points: JSON files and one image per element | +| `docs/` | Project plans, data model, API, accessibility, sustainability, AI usage, and catalog extraction documentation | +| `e2e/` | The end-to-end tests (Playwright): whole flows in a real browser, accessibility checks and the API over HTTP. See "End-to-end tests" | +| `public/` | Static files served as they are, such as the favicon | +| `scripts/extract-cop/` | The Python script that extracts `data/` from the Code of Points PDF | +| `src/` | The React frontend, shared UI components, API operations, Apollo Client, and frontend tests | ## Working in this repository diff --git a/docs/AI_USAGE.md b/docs/AI_USAGE.md index cc57f4d..5a70fb7 100644 --- a/docs/AI_USAGE.md +++ b/docs/AI_USAGE.md @@ -507,3 +507,10 @@ - **Accessibility check of the page** with axe-core, Lighthouse, keyboard-only use, contrast, target sizes, reflow, text spacing and reduced motion. Fixed what was in this pull request (focus lost after "Clear filters", the reason for a disabled filter not visible on touch, the page title) and listed the rest. A screen reader was not used: that has to be done by a person. - **Fixed the review point:** when the structure of the filters cannot be loaded and the address has filters, the page shows the error with "Try again" and does not search. - **What was not done by Claude:** nothing was posted or replied to on git without being asked, and the choice to offer bands for vault, the wording of the sort list and which findings to fix were the team's. + +## End-to-end tests (`e2e/`, `playwright.config.ts`) + +- **Tool:** Claude Code +- **What the team did:** Read the requirement of the assignment ("a form of automated end-to-end testing ... a longer sequence of interactions, and testing of the API") and wrote issue #139. Decided that the tests are run against the virtual machine too, and that atypical use and accessibility are part of them. +- **What Claude did:** Checked what the requirement means and what the repository already had (component tests and API tests, but no test that drives the whole application). Chose Playwright (ready-made, waits for the page by itself) with the official axe-core integration, and wrote the configuration so that the tests start the app themselves or run against any address (`E2E_BASE_URL`). Wrote one long flow, the vault bands, atypical use (wrong addresses, no result, a failing request, keyboard only, the disabled filter), axe-core checks of the page in its main states, and tests of the API over HTTP. Ran them against the real backend and the real catalog with Playwright's own Chromium and with Edge, and broke the application on purpose in four places to check that a test fails each time. Wrote the README section, including what the tests do not cover (a screen reader, and a run on every pull request, because the course server has no runners). +- **What was found:** The first run had 4 failing tests, all in the tests and not in the application: Playwright does not click a control that is `aria-disabled`, one expectation was wrong, and a check ran before the new results had arrived (the page keeps the old results on the screen until the new ones are there). diff --git a/docs/API.md b/docs/API.md index 1a76b23..6479d9f 100644 --- a/docs/API.md +++ b/docs/API.md @@ -493,7 +493,7 @@ Vitest runs the API tests against a test database, with queries sent to the serv **Decision: the test files run one at a time (`fileParallelism: false` in `vite.config.ts`).** The database tests share one local database, and some of them write to it (the login tests) while others check that the row counts do not change, so two files at the same time would break each other. The price is that the frontend tests also run one file at a time. We keep it because the suite is still small and fast (the frontend and non-database tests finish in about a second), so the loss is small for now. Running only the database tests one at a time (for example as a separate Vitest project, or with a database per test file) would remove the price, but it is more setup than the gain is worth now. Revisit it when the suite gets slow. -The end-to-end tests (a long sequence of actions in the browser) are described separately in the README when they exist. +The end-to-end tests (a long sequence of actions in the browser, and the API over HTTP against any address, also the virtual machine) are in `e2e/` and are described in "End-to-end tests" in the README. ## Open questions diff --git a/e2e/accessibility.e2e.ts b/e2e/accessibility.e2e.ts new file mode 100644 index 0000000..d1ed5ad --- /dev/null +++ b/e2e/accessibility.e2e.ts @@ -0,0 +1,148 @@ +// axe-core checks the Elements page against WCAG 2.0, 2.1 and 2.2 level A and AA, in the +// states where something is different: filtered and sorted, a menu open, the reason for a +// disabled filter shown, dark mode and a phone. A violation fails the test. What axe cannot +// decide (a screen reader, the order of the focus) is described in the README. + +import AxeBuilder from '@axe-core/playwright' +import { expect, test, type Page } from '@playwright/test' +import { openMenu, resultsLoaded } from './helpers.ts' + +const TAGS = ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa', 'wcag22aa'] + +/** + * `region` is axe's best-practice rule that all content is in a landmark. The menus are in + * a portal at the end of the document (the Base UI default, so that they are not clipped), + * so it is turned off for a state with a menu open. It is not a WCAG failure. + */ +async function violations(page: Page, options: { menuOpen?: boolean } = {}) { + const builder = new AxeBuilder({ page }).withTags(TAGS) + if (options.menuOpen) builder.disableRules(['region']) + const { violations } = await builder.analyze() + return violations.map((v) => ({ + rule: v.id, + impact: v.impact, + nodes: v.nodes.map((n) => n.target.join(' ')), + })) +} + +const FILTERED = + 'elements?apparatus=UB&group=UB-6&difficulty=A,B&sort=difficulty&dir=desc' + +test.describe('light mode', () => { + test('the page with the whole catalog', async ({ page }) => { + await page.goto('elements') + await resultsLoaded(page) + expect(await violations(page)).toEqual([]) + }) + + test('filtered and sorted', async ({ page }) => { + await page.goto(FILTERED) + await resultsLoaded(page) + expect(await violations(page)).toEqual([]) + }) + + test('with a filter menu open', async ({ page }) => { + await page.goto(FILTERED) + await resultsLoaded(page) + await openMenu(page, /^Difficulty/) + expect(await violations(page, { menuOpen: true })).toEqual([]) + }) + + test('with the sort menu open', async ({ page }) => { + await page.goto(FILTERED) + await resultsLoaded(page) + await openMenu(page, /Sort by/) + expect(await violations(page, { menuOpen: true })).toEqual([]) + }) + + test('with the reason for the disabled group filter shown', async ({ + page, + }) => { + await page.goto('elements') + await resultsLoaded(page) + await page + .getByRole('button', { name: 'Element group' }) + .click({ force: true }) + await expect( + page.getByText('Choose an apparatus to choose a group.').first(), + ).toBeVisible() + expect(await violations(page)).toEqual([]) + }) + + test('vault, with the bands', async ({ page }) => { + await page.goto('elements?apparatus=VT&difficulty=2,3') + await resultsLoaded(page) + expect(await violations(page)).toEqual([]) + }) + + test('a search without a result', async ({ page }) => { + await page.goto('elements?search=zzzqqqxxx') + await expect(page.getByText(/^No elements match/)).toBeVisible() + expect(await violations(page)).toEqual([]) + }) +}) + +test.describe('dark mode', () => { + test.beforeEach(async ({ page }) => { + // The app keeps the choice of theme here (App.tsx). + await page.addInitScript(() => localStorage.setItem('theme', 'dark')) + }) + + test('filtered and sorted', async ({ page }) => { + await page.goto(FILTERED) + await resultsLoaded(page) + await expect(page.locator('html')).toHaveAttribute('data-theme', 'dark') + expect(await violations(page)).toEqual([]) + }) + + test('with a menu open', async ({ page }) => { + await page.goto(FILTERED) + await resultsLoaded(page) + await openMenu(page, /^Difficulty/) + expect(await violations(page, { menuOpen: true })).toEqual([]) + }) +}) + +test.describe('on a phone', () => { + test.use({ viewport: { width: 375, height: 800 }, hasTouch: true }) + + test('filtered and sorted, without sideways scrolling', async ({ page }) => { + await page.goto(FILTERED) + await resultsLoaded(page) + expect(await violations(page)).toEqual([]) + const overflow = await page.evaluate( + () => + document.documentElement.scrollWidth > + document.documentElement.clientWidth, + ) + expect(overflow).toBe(false) + }) + + test('the controls are at least 40 px tall, so they can be pressed with a finger', async ({ + page, + }) => { + await page.goto('elements') + await resultsLoaded(page) + const heights = await page + .locator('[role=group] button, button[aria-haspopup]') + .evaluateAll((buttons) => + buttons.map((b) => Math.round(b.getBoundingClientRect().height)), + ) + expect(heights.length).toBeGreaterThanOrEqual(4) + expect(Math.min(...heights)).toBeGreaterThanOrEqual(40) + }) +}) + +test('the page reflows at 320 px wide (400 % zoom) without sideways scrolling', async ({ + page, +}) => { + await page.setViewportSize({ width: 320, height: 700 }) + await page.goto(FILTERED) + await resultsLoaded(page) + const overflow = await page.evaluate( + () => + document.documentElement.scrollWidth > + document.documentElement.clientWidth, + ) + expect(overflow).toBe(false) +}) diff --git a/e2e/api.e2e.ts b/e2e/api.e2e.ts new file mode 100644 index 0000000..c903858 --- /dev/null +++ b/e2e/api.e2e.ts @@ -0,0 +1,199 @@ +// The GraphQL API over HTTP, as the browser (or anyone else) calls it. The detailed cases +// of every query and error are tested in backend/src/server/ against the database; these +// check the contract in docs/API.md on whatever address is being tested, so the same +// requests can be sent to the virtual machine with E2E_BASE_URL. + +import { expect, test, type APIRequestContext } from '@playwright/test' +import { apiUrl } from './env.ts' + +type GqlError = { message: string; extensions?: { code?: string } } +type Result = { data: T | null; errors?: GqlError[] } + +async function gql( + request: APIRequestContext, + query: string, + variables: Record = {}, +): Promise> { + const response = await request.post(apiUrl, { + data: { query, variables }, + headers: { 'content-type': 'application/json' }, + }) + return (await response.json()) as Result +} + +const SEARCH = ` + query ($search: String, $filter: CatalogFilter, $sort: CatalogSort, $first: Int! = 20, $after: String) { + searchCatalog(search: $search, filter: $filter, sort: $sort, first: $first, after: $after) { + totalCount + pageInfo { hasNextPage endCursor } + edges { cursor node { id code ... on Element { value } ... on Vault { dValue } } } + } + }` + +type Node = { id: string; code: string; value?: number; dValue?: number } +type Page = { + searchCatalog: { + totalCount: number + pageInfo: { hasNextPage: boolean; endCursor: string | null } + edges: { cursor: string; node: Node }[] + } +} + +const errorCode = (result: Result) => + result.errors?.[0]?.extensions?.code + +test('the server is up and answers a query', async ({ request }) => { + const result = await gql<{ apparatus: { id: string }[] }>( + request, + '{ apparatus { id } }', + ) + expect(result.errors).toBeUndefined() + expect(result.data!.apparatus.map((a) => a.id)).toEqual([ + 'VT', + 'UB', + 'BB', + 'FX', + ]) +}) + +test('each apparatus lists the difficulty values that exist, lowest first', async ({ + request, +}) => { + const { data } = await gql<{ + apparatus: { id: string; difficultyValues: number[] }[] + }>(request, '{ apparatus { id difficultyValues } }') + for (const { id, difficultyValues } of data!.apparatus) { + expect(difficultyValues.length).toBeGreaterThan(0) + expect(difficultyValues).toEqual( + [...difficultyValues].sort((a, b) => a - b), + ) + // Vault has D-scores, the others the element values 0.1-1.0. + expect(difficultyValues.every((v) => (id === 'VT' ? v > 1 : v <= 1))).toBe( + true, + ) + } +}) + +test('the search is case insensitive and finds the start of words', async ({ + request, +}) => { + const lower = await gql(request, SEARCH, { search: 'salto' }) + const upper = await gql(request, SEARCH, { search: 'SALTO' }) + expect(lower.data!.searchCatalog.totalCount).toBeGreaterThan(0) + expect(upper.data!.searchCatalog.totalCount).toBe( + lower.data!.searchCatalog.totalCount, + ) + const prefix = await gql(request, SEARCH, { search: 'sal' }) + expect(prefix.data!.searchCatalog.totalCount).toBeGreaterThanOrEqual( + lower.data!.searchCatalog.totalCount, + ) +}) + +test('filtering runs over the whole catalog, and the count is of the whole result', async ({ + request, +}) => { + const all = await gql(request, SEARCH, { first: 5 }) + const bars = await gql(request, SEARCH, { + first: 100, + filter: { apparatusId: 'UB', difficultyValues: [0.1, 0.2] }, + }) + const page = bars.data!.searchCatalog + expect(page.totalCount).toBeLessThan(all.data!.searchCatalog.totalCount) + expect(page.edges.length).toBe(page.totalCount) + expect(page.edges.every((e) => e.node.id.startsWith('UB-'))).toBe(true) + expect(page.edges.every((e) => [0.1, 0.2].includes(e.node.value!))).toBe(true) +}) + +test('vault is filtered on its D-score, and elements are not mixed in', async ({ + request, +}) => { + const { data } = await gql<{ + apparatus: { id: string; difficultyValues: number[] }[] + }>(request, '{ apparatus { id difficultyValues } }') + const values = data!.apparatus.find((a) => a.id === 'VT')!.difficultyValues + const chosen = [values[1], values[2]] + const found = await gql(request, SEARCH, { + first: 100, + filter: { apparatusId: 'VT', difficultyValues: chosen }, + }) + const edges = found.data!.searchCatalog.edges + expect(edges.length).toBeGreaterThan(0) + expect(edges.every((e) => chosen.includes(e.node.dValue!))).toBe(true) + expect(edges.every((e) => e.node.id.startsWith('VT-'))).toBe(true) +}) + +test('sorting is over the whole result, in both directions', async ({ + request, +}) => { + for (const direction of ['ASC', 'DESC']) { + const found = await gql(request, SEARCH, { + first: 100, + filter: { apparatusId: 'BB' }, + sort: { field: 'DIFFICULTY', direction }, + }) + const values = found.data!.searchCatalog.edges.map((e) => e.node.value!) + const sorted = [...values].sort((a, b) => + direction === 'ASC' ? a - b : b - a, + ) + expect(values).toEqual(sorted) + } +}) + +test('pages follow each other by cursor without skipping or repeating', async ({ + request, +}) => { + const seen: string[] = [] + let after: string | null = null + let pages = 0 + do { + const result: Result = await gql(request, SEARCH, { + first: 100, + after, + filter: { apparatusId: 'FX' }, + }) + const page = result.data!.searchCatalog + seen.push(...page.edges.map((e) => e.node.id)) + after = page.pageInfo.hasNextPage ? page.pageInfo.endCursor : null + pages++ + } while (after && pages < 20) + expect(seen.length).toBeGreaterThan(0) + expect(new Set(seen).size).toBe(seen.length) +}) + +test('a request that cannot be answered is refused with a code, not an internal error', async ({ + request, +}) => { + const cases: [string, Record][] = [ + ['unknown apparatus', { filter: { apparatusId: 'XX' } }], + [ + 'difficulty above the range for vault', + { filter: { apparatusId: 'VT', difficultyValues: [10.1] } }, + ], + [ + 'a value that is not in steps of 0.1', + { filter: { difficultyValues: [0.15] } }, + ], + ['difficulty sort without an apparatus', { sort: { field: 'DIFFICULTY' } }], + ['a page that is too big', { first: 101 }], + ['a search that is too long', { search: 'a'.repeat(81) }], + ] + for (const [label, variables] of cases) { + const result = await gql(request, SEARCH, variables) + expect(errorCode(result), label).toBe('BAD_USER_INPUT') + } +}) + +test('an invalid query is refused with a code, and does not show an internal error', async ({ + request, +}) => { + const result = await gql(request, '{ noSuchField }') + expect(errorCode(result)).toBe('GRAPHQL_VALIDATION_FAILED') +}) + +test('the data of a coach needs a login', async ({ request }) => { + const result = await gql<{ gymnast: unknown }>( + request, + '{ gymnast(id: "x") { id } }', + ) + expect(errorCode(result)).toBe('UNAUTHENTICATED') +}) diff --git a/e2e/atypical.e2e.ts b/e2e/atypical.e2e.ts new file mode 100644 index 0000000..cccd69d --- /dev/null +++ b/e2e/atypical.e2e.ts @@ -0,0 +1,176 @@ +// Atypical use: what the page does with an address that is wrong, a search without a +// result, a request that fails, and a user who never touches the mouse. + +import { expect, test } from '@playwright/test' +import { choose, counts, resultsLoaded, expectSearch } from './helpers.ts' + +test('an address with filters that do not exist is cleaned, and gives no error', async ({ + page, +}) => { + await page.goto('elements?apparatus=XX&group=ZZ&difficulty=Q,9.9') + await resultsLoaded(page) + // The API would answer BAD_USER_INPUT for these, so they are never sent. + await expect(page.getByRole('alert')).toHaveCount(0) + expect((await counts(page)).shown).toBe(20) + await expect(page.getByRole('button', { name: 'Clear filters' })).toBeHidden() +}) + +test('a group of another apparatus is left out, and a sort that needs an apparatus is the default', async ({ + page, +}) => { + await page.goto('elements?apparatus=UB&group=VT-1&sort=difficulty&dir=desc') + await resultsLoaded(page) + await expect(page.getByRole('alert')).toHaveCount(0) + await expect( + page.getByRole('button', { name: 'Apparatus: Uneven bars' }), + ).toBeVisible() + + await page.goto('elements?sort=difficulty&dir=desc') + await resultsLoaded(page) + await expect(page.getByRole('alert')).toHaveCount(0) + await expect( + page.getByRole('button', { name: /Sort by Code: low to high/ }), + ).toBeVisible() +}) + +test('a search without a result says so, and has nothing to load', async ({ + page, +}) => { + await page.goto('elements') + await resultsLoaded(page) + await page.getByRole('searchbox').fill('zzzqqqxxx') + await expect(page.getByText('No elements match “zzzqqqxxx”.')).toBeVisible() + await expect( + page.getByRole('button', { name: 'Load more elements' }), + ).toHaveCount(0) + + // With a filter the message says that the filters are part of it. + await page.goto('elements?search=zzzqqqxxx&apparatus=FX') + await expect( + page.getByText('No elements match “zzzqqqxxx” with these filters.'), + ).toBeVisible() +}) + +test('the search field takes at most 80 characters, the most the API accepts', async ({ + page, +}) => { + await page.goto('elements') + const field = page.getByRole('searchbox') + await field.fill('a'.repeat(120)) + expect((await field.inputValue()).length).toBe(80) + await expect(page.getByRole('alert')).toHaveCount(0) +}) + +test('a structure that fails to load is an error with a retry, not unfiltered results', async ({ + page, +}) => { + let failing = true + const searches: string[] = [] + await page.route('**/graphql', async (route) => { + const body = route.request().postDataJSON() as { + operationName?: string + variables?: { filter?: unknown } + } + if (body.operationName === 'CatalogStructure' && failing) { + return route.fulfill({ + status: 500, + contentType: 'application/json', + body: JSON.stringify({ + errors: [ + { message: 'boom', extensions: { code: 'INTERNAL_SERVER_ERROR' } }, + ], + }), + }) + } + if (body.operationName === 'SearchCatalog') + searches.push(JSON.stringify(body.variables?.filter)) + return route.continue() + }) + + await page.goto('elements?apparatus=UB&difficulty=A') + await expect(page.getByRole('alert')).toBeVisible() + // No results under an address that says Uneven bars, and nothing was searched. + await expect(page.getByText(/^Showing/)).toHaveCount(0) + expect(searches).toEqual([]) + await expectSearch(page, '?apparatus=UB&difficulty=A') + + // The server is back: trying again loads the structure and searches with the filters. + failing = false + await page.getByRole('button', { name: 'Try again' }).click() + await resultsLoaded(page) + expect(searches).toEqual(['{"apparatusId":"UB","difficultyValues":[0.1]}']) +}) + +test('a filter that cannot be used is disabled but reachable, and says why', async ({ + page, +}) => { + await page.goto('elements') + await resultsLoaded(page) + const group = page.getByRole('button', { name: 'Element group' }) + + // aria-disabled and not the disabled attribute: the keyboard can still reach it. + await expect(group).toHaveAttribute('aria-disabled', 'true') + await expect(group).not.toHaveAttribute('disabled', /.*/) + await group.focus() + await expect(group).toBeFocused() + + // The reason is read with the button, and it is shown when the button is pressed too (a + // tooltip does not open on a tap). + const reason = 'Choose an apparatus to choose a group.' + await expect(group).toHaveAccessibleDescription(reason) + // force: Playwright does not click what is aria-disabled, but a person can press it. + await group.click({ force: true }) + await expect(page.getByText(reason).first()).toBeVisible() + await expect(page.getByRole('menu')).toHaveCount(0) + await page.keyboard.press('Escape') +}) + +test('the filters and the sort can be used with the keyboard only', async ({ + page, +}) => { + await page.goto('elements') + await resultsLoaded(page) + + // The search field, then the three filters, in the order they are on the page. + await page.getByRole('searchbox').focus() + await page.keyboard.press('Tab') + const apparatus = page.getByRole('button', { name: /^Apparatus/ }) + await expect(apparatus).toBeFocused() + + // Enter opens the menu, the arrows move, Enter chooses, and the focus comes back. + await page.keyboard.press('Enter') + await expect(page.getByRole('menu')).toBeVisible() + await page.keyboard.press('ArrowDown') // the first item is "All apparatus" + await page.keyboard.press('ArrowDown') + await page.keyboard.press('Enter') + await expect(page.getByRole('menu')).toBeHidden() + await expectSearch(page, /^\?apparatus=(VT|UB|BB|FX)$/) + await expect(apparatus).toBeFocused() + + // Escape closes a menu without choosing, and the focus is back on its button. + await page.keyboard.press('Space') + await expect(page.getByRole('menu')).toBeVisible() + await page.keyboard.press('Escape') + await expect(page.getByRole('menu')).toBeHidden() + await expect(apparatus).toBeFocused() + + // The difficulty menu takes several choices with Space and stays open. + await page.getByRole('button', { name: /^Difficulty/ }).focus() + await page.keyboard.press('Enter') + await expect(page.getByRole('menu')).toBeVisible() + await page.keyboard.press('ArrowDown') + await page.keyboard.press('Space') + await page.keyboard.press('ArrowDown') + await page.keyboard.press('Space') + await expect(page.getByRole('menu')).toBeVisible() + await expect( + page.getByRole('menuitemcheckbox', { checked: true }), + ).toHaveCount(2) + await page.keyboard.press('Escape') + + // And the sort, which is one list. + await page.getByRole('button', { name: /Sort by/ }).focus() + await page.keyboard.press('Enter') + await choose(page, 'menuitemradio', 'Name: A to Z') + await expect(page).toHaveURL(/sort=name/) +}) diff --git a/e2e/elements-flow.e2e.ts b/e2e/elements-flow.e2e.ts new file mode 100644 index 0000000..69d27c5 --- /dev/null +++ b/e2e/elements-flow.e2e.ts @@ -0,0 +1,115 @@ +// A longer sequence of interactions on the Elements page, as a coach would do it: search, +// narrow down with the filters, sort, load more, open an element, come back and clear. +// It checks the address, the results and the title at each step. It needs the seeded +// catalog of the repository (562 entries). + +import { expect, test } from '@playwright/test' +import { + choose, + counts, + isDescending, + openMenu, + resultsLoaded, + rowCodes, + rowValues, + search, + expectSearch, +} from './helpers.ts' + +test('search, filter, sort, load more, open an element and come back, then clear', async ({ + page, +}) => { + // 1. The page opens with the whole catalog, 20 at a time. + await page.goto('elements') + await expect(page).toHaveTitle('Elements – WAG CoP Planner') + const all = await counts(page) + expect(all.shown).toBe(20) + expect(all.total).toBeGreaterThan(100) + + // 2. A search narrows it down, and is in the address after the user stops typing. + await page.getByRole('searchbox').fill('salto') + await expect(page).toHaveURL(/search=salto/) + await expect + .poll(async () => (await counts(page)).total) + .toBeLessThan(all.total) + + // 3. The apparatus filter narrows it more, and its menu closes after one choice. + await openMenu(page, /^Apparatus/) + await choose(page, 'menuitemradio', 'Floor exercise') + await expect(page.getByRole('menu')).toBeHidden() + await expectSearch(page, '?search=salto&apparatus=FX') + await expect( + page.getByRole('button', { name: 'Apparatus: Floor exercise' }), + ).toBeVisible() + await expect.poll(async () => (await counts(page)).total).toBeGreaterThan(20) + const floor = await counts(page) + + // 4. "Load more" adds the next page to the list, until everything is shown. + const before = (await rowCodes(page)).length + await page.getByRole('button', { name: 'Load more elements' }).click() + await expect( + page.getByText(`Showing ${floor.total} of ${floor.total}`), + ).toBeVisible() + expect((await rowCodes(page)).length).toBeGreaterThan(before) + await expect( + page.getByRole('button', { name: 'Load more elements' }), + ).toBeDisabled() + + // 5. Sorting by difficulty starts from the first page, in the order that was asked for. + await openMenu(page, /Sort by/) + await choose(page, 'menuitemradio', 'Difficulty: high to low') + await expect(page).toHaveURL(/sort=difficulty&dir=desc/) + await expect(page.getByText(`Showing 20 of ${floor.total}`)).toBeVisible() + expect(isDescending(await rowValues(page))).toBe(true) + + // 6. The group filter lists the groups of the apparatus, and every result is in the group. + await openMenu(page, 'Element group') + await choose(page, 'menuitemradio', 'Saltos backward') + await expectSearch(page, /group=FX-5/) + // The old results stay on the page until the new ones arrive, so wait for them. + await expect + .poll(async () => { + const codes = await rowCodes(page) + return codes.length > 0 && codes.every((code) => code.startsWith('5.')) + }) + .toBe(true) + expect(isDescending(await rowValues(page))).toBe(true) + const narrowed = await counts(page) + const narrowedAddress = search(page) + + // 7. An element opens on its own page, with a title that names it. + await page.locator('table tbody tr a').first().click() + await expect(page).toHaveURL(/\/elements\/FX-5\./) + await expect(page.getByRole('heading', { level: 1 })).toBeVisible() + await expect(page).toHaveTitle( + /^Element 5\.\S+.* – Floor exercise – WAG CoP Planner$/, + ) + + // 8. Back finds the same result: the address has everything, and so do the buttons. + await page.goBack() + await resultsLoaded(page) + await expectSearch(page, narrowedAddress) + expect(await counts(page)).toEqual(narrowed) + await expect( + page.getByRole('button', { name: 'Element group: Saltos backward' }), + ).toBeVisible() + await expect( + page.getByRole('button', { name: /Sort by Difficulty: high to low/ }), + ).toBeVisible() + await expect(page).toHaveTitle('Elements – WAG CoP Planner') + + // 9. "Clear filters" takes the filters away and keeps the search. The button goes away + // with them, so the focus moves to the first filter and not to the top of the page. + await page.getByRole('button', { name: 'Clear filters' }).click() + await expectSearch(page, /^(?!.*apparatus)/) + await expectSearch(page, /^(?!.*group)/) + await expectSearch(page, /search=salto/) + await expect(page.getByRole('button', { name: /^Apparatus/ })).toBeFocused() + // Difficulty can only be sorted within an apparatus, so the sort is the default again. + await expect( + page.getByRole('button', { name: /Sort by Code: low to high/ }), + ).toBeVisible() + await expect + .poll(async () => (await counts(page)).total) + .toBeGreaterThan(floor.total) +}) diff --git a/e2e/env.ts b/e2e/env.ts new file mode 100644 index 0000000..0abd701 --- /dev/null +++ b/e2e/env.ts @@ -0,0 +1,25 @@ +// Where the end-to-end tests run. The defaults start the app from this repository on the +// usual ports. Everything can be changed with environment variables, so the same tests can +// also check an address that is already up, for example the virtual machine: +// +// E2E_BASE_URL=http://it2810-29.idi.ntnu.no/project2 npm run e2e + +/** The port of the frontend (Vite) and of the backend when the tests start them. */ +export const frontendPort = Number(process.env['E2E_FRONTEND_PORT'] ?? 5173) +export const backendPort = Number(process.env['E2E_BACKEND_PORT'] ?? 3001) + +/** An address that is already running. Not set: the tests start the app themselves. */ +export const externalBaseUrl = process.env['E2E_BASE_URL']?.replace(/\/+$/, '') + +/** The address of the app, with a final slash so that `page.goto('elements')` works. */ +export const baseURL = `${externalBaseUrl ?? `http://localhost:${frontendPort}/project2`}/` + +/** + * The GraphQL endpoint for the tests of the API. On the virtual machine Apache sends + * `/project2/graphql` to the backend, and locally the tests call the backend directly. + */ +export const apiUrl = + process.env['E2E_API_URL'] ?? + (externalBaseUrl + ? `${externalBaseUrl}/graphql` + : `http://localhost:${backendPort}/graphql`) diff --git a/e2e/helpers.ts b/e2e/helpers.ts new file mode 100644 index 0000000..d3a1037 --- /dev/null +++ b/e2e/helpers.ts @@ -0,0 +1,71 @@ +import { expect, type Locator, type Page } from '@playwright/test' + +/** The line under the results: "Showing 20 of 562" (or "1000+"). */ +const showing = (page: Page): Locator => page.getByText(/^Showing \d+ of /) + +/** Waits for the results and gives the numbers of "Showing 20 of 562". */ +export async function counts(page: Page) { + const text = (await showing(page).first().textContent({ timeout: 15_000 }))! + const [, shown, total] = /Showing (\d+) of (\d+)/.exec(text)! + return { shown: Number(shown), total: Number(total) } +} + +/** The page has loaded the first results. */ +export async function resultsLoaded(page: Page) { + await expect(showing(page).first()).toBeVisible({ timeout: 15_000 }) +} + +/** + * The difficulty of each row in the table, as the number at the end of its last cell: the + * element value (0.3) or the D-score of a vault (3.2). + */ +export async function rowValues(page: Page): Promise { + const cells = await page + .locator('table tbody tr td:last-child') + .allTextContents() + return cells.map((cell) => Number(/(\d+(?:\.\d+)?)\s*$/.exec(cell)![1])) +} + +/** The code at the start of each row: "6.210". */ +export async function rowCodes(page: Page): Promise { + const cells = await page + .locator('table tbody tr td:first-child') + .allTextContents() + return cells.map((cell) => cell.trim()) +} + +/** Opens the menu of a filter or of the sort by its button, and waits for it. */ +export async function openMenu(page: Page, button: string | RegExp) { + await page.getByRole('button', { name: button }).click() + await expect(page.getByRole('menu')).toBeVisible() +} + +/** Chooses an item of the open menu. A single choice closes the menu, several do not. */ +export async function choose( + page: Page, + role: 'menuitemradio' | 'menuitemcheckbox', + name: string, +) { + await page.getByRole(role, { name, exact: true }).click() +} + +/** + * Waits until the address is what it should be. The address changes a moment after a click, + * so a test that reads it at once can be too early. A string must be the whole address + * ("?apparatus=UB&difficulty=A,B"), and an expression is tested on it. + */ +export async function expectSearch(page: Page, expected: string | RegExp) { + const poll = expect.poll(() => search(page), { timeout: 10_000 }) + if (typeof expected === 'string') await poll.toBe(expected) + else await poll.toMatch(expected) +} + +/** The address as the test sees it, decoded, without the path: "?apparatus=UB&difficulty=A,B". */ +export function search(page: Page): string { + return decodeURIComponent(new URL(page.url()).search) +} + +export const isDescending = (values: number[]) => + values.every((v, i) => i === 0 || v <= values[i - 1]) +export const isAscending = (values: number[]) => + values.every((v, i) => i === 0 || v >= values[i - 1]) diff --git a/e2e/vault.e2e.ts b/e2e/vault.e2e.ts new file mode 100644 index 0000000..4060f38 --- /dev/null +++ b/e2e/vault.e2e.ts @@ -0,0 +1,106 @@ +// Vault has D-scores (1.6-6.4) and not the element values 0.1-1.0, so its difficulty filter +// offers bands of one whole number, and its difficulty sort is by the score. + +import { expect, test } from '@playwright/test' +import { + choose, + counts, + isAscending, + isDescending, + openMenu, + resultsLoaded, + rowValues, + expectSearch, +} from './helpers.ts' + +test('the difficulty filter of vault offers bands in order, and filters on the score', async ({ + page, +}) => { + await page.goto('elements?apparatus=VT') + await resultsLoaded(page) + const all = await counts(page) + + // The bands, lowest first, and not the 23 separate D-values. + await openMenu(page, 'Difficulty') + const bands = await page.getByRole('menuitemcheckbox').allTextContents() + expect(bands.length).toBeGreaterThan(3) + expect(bands.length).toBeLessThan(12) + expect(bands.every((band) => /^\d\.0–\d\.9$/.test(band))).toBe(true) + expect(bands).toEqual([...bands].sort()) + + // What each band is on its own, to compare with the two together. The rows are in the + // order of the codes, so the first page does not have to hold a vault of each band, and + // the number of vaults is what tells that nothing is missing or extra. + const [first, second] = [bands[1], bands[2]] + const [low, high] = [first, second].map((band) => Number(band.slice(0, 1))) + const apart: number[] = [] + for (const band of [low, high]) { + await page.goto(`elements?apparatus=VT&difficulty=${band}`) + await resultsLoaded(page) + apart.push((await counts(page)).total) + } + + // Two bands: the menu stays open between the choices, and both end up in the address. + await page.goto('elements?apparatus=VT') + await resultsLoaded(page) + await openMenu(page, 'Difficulty') + await choose(page, 'menuitemcheckbox', second) + await choose(page, 'menuitemcheckbox', first) + await expect(page.getByRole('menu')).toBeVisible() + await page.keyboard.press('Escape') + await expectSearch(page, `?apparatus=VT&difficulty=${low},${high}`) + await expect( + page.getByRole('button', { name: `Difficulty: ${first}, ${second}` }), + ).toBeVisible() + + // The old results stay on the page until the new ones arrive, so wait for the total. + await expect + .poll(async () => (await counts(page)).total) + .toBe(apart[0] + apart[1]) + expect(apart[0] + apart[1]).toBeLessThan(all.total) + + // Every vault in the result has a score inside the two bands. + const values = await rowValues(page) + expect(values.length).toBeGreaterThan(0) + expect(values.every((v) => v >= low && v < high + 1)).toBe(true) +}) + +test('vault is sorted by its D-score in both directions', async ({ page }) => { + await page.goto('elements?apparatus=VT') + await resultsLoaded(page) + + await openMenu(page, /Sort by/) + await choose(page, 'menuitemradio', 'Difficulty: high to low') + await expect(page).toHaveURL(/sort=difficulty&dir=desc/) + await expect.poll(async () => isDescending(await rowValues(page))).toBe(true) + const highest = (await rowValues(page))[0] + expect(highest).toBeGreaterThan(5) + + await openMenu(page, /Sort by/) + await choose(page, 'menuitemradio', 'Difficulty: low to high') + await expect.poll(async () => isAscending(await rowValues(page))).toBe(true) + expect((await rowValues(page))[0]).toBeLessThan(highest) +}) + +test('the chosen difficulty is dropped when the apparatus changes from vault to bars', async ({ + page, +}) => { + await page.goto('elements?apparatus=VT&difficulty=2') + await resultsLoaded(page) + await expect( + page.getByRole('button', { name: 'Difficulty: 2.0–2.9' }), + ).toBeVisible() + + await openMenu(page, /^Apparatus/) + await choose(page, 'menuitemradio', 'Uneven bars') + // The bands are D-scores and the bars have the letters A-J, so the two do not carry over. + await expectSearch(page, '?apparatus=UB') + // The page shows the new apparatus a moment after the address changes: wait for it, or the + // menu is read with the bands of vault. + await expect( + page.getByRole('button', { name: 'Apparatus: Uneven bars' }), + ).toBeVisible() + await openMenu(page, 'Difficulty') + const letters = await page.getByRole('menuitemcheckbox').allTextContents() + expect(letters.join('')).toBe('ABCDEFGHIJ') +}) diff --git a/package-lock.json b/package-lock.json index 090b820..f73e602 100644 --- a/package-lock.json +++ b/package-lock.json @@ -27,12 +27,14 @@ "rxjs": "^7.8.2" }, "devDependencies": { + "@axe-core/playwright": "^4.14.0", "@babel/core": "^7.29.7", "@eslint/js": "^10.0.1", + "@playwright/test": "^1.64.1", "@rolldown/plugin-babel": "^0.2.3", "@tailwindcss/vite": "^4.3.3", - "@testing-library/jest-dom": "^7.0.1", "@testing-library/dom": "^10.4.2", + "@testing-library/jest-dom": "^7.0.1", "@testing-library/react": "^16.3.3", "@types/babel__core": "^7.20.5", "@types/cors": "^2.8.19", @@ -472,6 +474,22 @@ "dev": true, "license": "MIT" }, + "node_modules/@axe-core/playwright": { + "version": "4.14.0", + "resolved": "https://registry.npmjs.org/@axe-core/playwright/-/playwright-4.14.0.tgz", + "integrity": "sha512-SGUhG5PIHd2eMdpgBF2PyQLOhX+qyDR2PT4cmoS4Qk2ak+jX/rrSvst9l0PGv5WTHolKpUMvVmoHpr4cZgf1RQ==", + "dev": true, + "license": "MPL-2.0", + "dependencies": { + "axe-core": "~4.14.0" + }, + "engines": { + "node": ">=22.12.0" + }, + "peerDependencies": { + "playwright-core": ">= 1.0.0" + } + }, "node_modules/@babel/code-frame": { "version": "7.29.7", "resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz", @@ -3344,6 +3362,22 @@ "url": "https://github.com/sponsors/oxc-project" } }, + "node_modules/@playwright/test": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.64.1.tgz", + "integrity": "sha512-IaYGml/nSOiZWROfPtuIL7neuu9qDOZ3TNvNljUIrhDMxiGTgPjzj/g9p9RxpS2tSepPDRY+UwAaEeiKtUvIAg==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "playwright": "1.64.1" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=20" + } + }, "node_modules/@prisma/adapter-pg": { "version": "7.10.1", "resolved": "https://registry.npmjs.org/@prisma/adapter-pg/-/adapter-pg-7.10.1.tgz", @@ -4731,83 +4765,6 @@ "node": ">=6.9.0" } }, - "node_modules/@testing-library/react": { - "version": "16.3.3", - "resolved": "https://registry.npmjs.org/@testing-library/react/-/react-16.3.3.tgz", - "integrity": "sha512-Uo193NgQbPMz6lrrhtRQQFcMC6Re/ELLFbbuVL30WDlZxlpZf9/lMHTAVxPRLw1q1iu9OJmR1c2BLiENRstdBg==", - "dev": true, - "license": "MIT", - "dependencies": { - "@babel/runtime": "^7.12.5" - }, - "engines": { - "node": ">=18" - }, - "peerDependencies": { - "@testing-library/dom": "^10.0.0", - "@types/react": "^18.0.0 || ^19.0.0", - "@types/react-dom": "^18.0.0 || ^19.0.0", - "react": "^18.0.0 || ^19.0.0", - "react-dom": "^18.0.0 || ^19.0.0" - }, - "peerDependenciesMeta": { - "@types/react": { - "optional": true - }, - "@types/react-dom": { - "optional": true - } - } - }, - "node_modules/@testing-library/react/node_modules/@babel/runtime": { - "version": "7.29.10", - "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.10.tgz", - "integrity": "sha512-NpugTzbKgGrNe45Ui32cN8vCFFMxVtP/9rtfwv0nP5FRDAjvYBxJcLVpYA9mr1jxP6cnIT7x9LUzuDTbWYYNRA==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">=6.9.0" - } - }, - "node_modules/@types/aria-query": { - "version": "5.0.4", - "resolved": "https://registry.npmjs.org/@types/aria-query/-/aria-query-5.0.4.tgz", - "integrity": "sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw==", - "dev": true, - "license": "MIT" - }, - "node_modules/@types/babel__core": { - "version": "7.20.5", - "resolved": "https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz", - "integrity": "sha512-qoQprZvz5wQFJwMDqeseRXWv3rqMvhgpbXFfVyWhbx9X47POIA6i/+dXefEmZKoAgOaTdaIgNSMqMIU61yRyzA==", - "dev": true, - "license": "MIT", - "peer": true, - "dependencies": { - "@babel/code-frame": "^7.10.4", - "@babel/runtime": "^7.12.5", - "@types/aria-query": "^5.0.1", - "aria-query": "5.3.0", - "dom-accessibility-api": "^0.5.9", - "lz-string": "^1.5.0", - "picocolors": "1.1.1", - "pretty-format": "^27.0.2" - }, - "engines": { - "node": ">=18" - } - }, - "node_modules/@testing-library/dom/node_modules/@babel/runtime": { - "version": "7.29.10", - "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.10.tgz", - "integrity": "sha512-NpugTzbKgGrNe45Ui32cN8vCFFMxVtP/9rtfwv0nP5FRDAjvYBxJcLVpYA9mr1jxP6cnIT7x9LUzuDTbWYYNRA==", - "dev": true, - "license": "MIT", - "peer": true, - "engines": { - "node": ">=6.9.0" - } - }, "node_modules/@testing-library/jest-dom": { "version": "7.0.1", "resolved": "https://registry.npmjs.org/@testing-library/jest-dom/-/jest-dom-7.0.1.tgz", @@ -4887,8 +4844,7 @@ "resolved": "https://registry.npmjs.org/@types/aria-query/-/aria-query-5.0.4.tgz", "integrity": "sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw==", "dev": true, - "license": "MIT", - "peer": true + "license": "MIT" }, "node_modules/@types/babel__core": { "version": "7.20.5", @@ -6324,6 +6280,16 @@ "node": ">= 6.0.0" } }, + "node_modules/axe-core": { + "version": "4.14.0", + "resolved": "https://registry.npmjs.org/axe-core/-/axe-core-4.14.0.tgz", + "integrity": "sha512-9WTZxEjsZ7b13TH8JPmbV2z8CHbl80/2hm3XPEG4JgNdQLK81IBRXmSxHfMAOkSqQeRxT/0dwNDz2GOm3zzpcQ==", + "dev": true, + "license": "MPL-2.0", + "engines": { + "node": ">=4" + } + }, "node_modules/babel-plugin-react-compiler": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/babel-plugin-react-compiler/-/babel-plugin-react-compiler-1.0.0.tgz", @@ -10866,6 +10832,35 @@ "devOptional": true, "license": "MIT" }, + "node_modules/playwright": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.64.1.tgz", + "integrity": "sha512-HCrVhHr5QQuwYoswEUGTb7gGp522yflbv/ST8nF4sFWGuLjQCt17Q/EubpGGSNKYn1tsTCPZxqHhHCfVURu7UA==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "playwright-core": "1.64.1" + }, + "bin": { + "playwright": "cli.js" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/playwright-core": { + "version": "1.64.1", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.64.1.tgz", + "integrity": "sha512-BC4s8y6SEVA6Q1rSxZTto8hI1KUI3xXlHGZgvGqP4IqPdI/a6FX9H29IDNz7vKo9l21b6LfYosScPL6M3xxyDQ==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "playwright-core": "cli.js" + }, + "engines": { + "node": ">=20" + } + }, "node_modules/possible-typed-array-names": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz", diff --git a/package.json b/package.json index 881035d..9353481 100644 --- a/package.json +++ b/package.json @@ -24,7 +24,9 @@ "db:synthetic": "npm run synthetic -w backend --", "test:run": "vitest run", "test:ui": "vitest --ui", - "test:coverage": "vitest run --coverage" + "test:coverage": "vitest run --coverage", + "e2e": "playwright test", + "e2e:report": "playwright show-report" }, "dependencies": { "@apollo/client": "^4.3.4", @@ -43,12 +45,14 @@ "rxjs": "^7.8.2" }, "devDependencies": { + "@axe-core/playwright": "^4.14.0", "@babel/core": "^7.29.7", "@eslint/js": "^10.0.1", + "@playwright/test": "^1.64.1", "@rolldown/plugin-babel": "^0.2.3", "@tailwindcss/vite": "^4.3.3", - "@testing-library/jest-dom": "^7.0.1", "@testing-library/dom": "^10.4.2", + "@testing-library/jest-dom": "^7.0.1", "@testing-library/react": "^16.3.3", "@types/babel__core": "^7.20.5", "@types/cors": "^2.8.19", diff --git a/playwright.config.ts b/playwright.config.ts new file mode 100644 index 0000000..4e501d5 --- /dev/null +++ b/playwright.config.ts @@ -0,0 +1,66 @@ +// End-to-end tests: Playwright drives a real browser through the running application. See +// "End-to-end tests" in the README. The tests are in `e2e/` and end in `.e2e.ts`, so that +// Vitest (which runs the component tests) does not pick them up. + +import { defineConfig, devices } from '@playwright/test' +import { + backendPort, + baseURL, + externalBaseUrl, + frontendPort, +} from './e2e/env.ts' + +export default defineConfig({ + testDir: 'e2e', + testMatch: '**/*.e2e.ts', + // The tests only read the catalog, but the servers are shared, so one at a time keeps a + // failure easy to read. + fullyParallel: false, + workers: 1, + forbidOnly: Boolean(process.env['CI']), + retries: process.env['CI'] ? 1 : 0, + reporter: [['list'], ['html', { open: 'never' }]], + use: { + baseURL, + // Screenshots and a trace of the test that failed, to see what happened. + trace: 'retain-on-failure', + screenshot: 'only-on-failure', + }, + projects: [ + { + name: 'chromium', + use: { + ...devices['Desktop Chrome'], + // For example E2E_CHANNEL=msedge to use the Edge that Windows has, and not download + // a browser. Not set: the Chromium that `npx playwright install chromium` gives. + channel: process.env['E2E_CHANNEL'] || undefined, + }, + }, + ], + // Start the backend and the frontend, unless E2E_BASE_URL says that an address is up. + // The database must be running and seeded (see "Database (local setup)" in the README). + webServer: externalBaseUrl + ? undefined + : [ + { + command: 'npm run start -w backend', + url: `http://localhost:${backendPort}/health`, + env: { + PORT: String(backendPort), + CORS_ORIGIN: `http://localhost:${frontendPort}`, + IMAGE_BASE_URL: `http://localhost:${backendPort}/images`, + }, + reuseExistingServer: !process.env['CI'], + timeout: 60_000, + }, + { + command: `npm run dev -- --host localhost --port ${frontendPort} --strictPort`, + url: baseURL, + env: { + VITE_API_URL: `http://localhost:${backendPort}/graphql`, + }, + reuseExistingServer: !process.env['CI'], + timeout: 60_000, + }, + ], +}) diff --git a/tsconfig.e2e.json b/tsconfig.e2e.json new file mode 100644 index 0000000..04b8614 --- /dev/null +++ b/tsconfig.e2e.json @@ -0,0 +1,24 @@ +{ + "compilerOptions": { + "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.e2e.tsbuildinfo", + "target": "es2023", + "lib": ["ES2023", "DOM"], + "types": ["node"], + "skipLibCheck": true, + + /* Bundler mode */ + "module": "nodenext", + "allowImportingTsExtensions": true, + "verbatimModuleSyntax": true, + "moduleDetection": "force", + "noEmit": true, + + /* Linting */ + "strict": true, + "noUnusedLocals": true, + "noUnusedParameters": true, + "erasableSyntaxOnly": true, + "noFallthroughCasesInSwitch": true + }, + "include": ["playwright.config.ts", "e2e"] +} diff --git a/tsconfig.json b/tsconfig.json index 313ae6a..36109fc 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -6,6 +6,9 @@ }, { "path": "./tsconfig.node.json" + }, + { + "path": "./tsconfig.e2e.json" } ], "compilerOptions": {