Skip to content

Docs: Write setup and startup documentation #76

Merged
merged 3 commits into from
Oct 15, 2026
+177 −88
Merged

Docs: Write setup and startup documentation #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8e226bc
Docs: Write setup and startup documentation
asgeija Oct 15, 2026
106cbd5
Fix: Replace dead dataset link
asgeija Oct 15, 2026
81cda94
Docs: Address review feedback on rollback notes
asgeija Oct 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 28 additions & 13 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,22 @@

- **Name:** UFOund it
- **What it is:** UFOund it, is a webapp that displays UFO sightings.
- **Dataset:** [NUFORC UFO Sightings (Kaggle)](https://www.kaggle.com/datasets/NUFORC/ufo-sightings), CSV, approx 80,000 reports, approx 15mb.
- **Dataset:** [NUFORC](https://nuforc.org/) UFO sightings, CSV in `backend/data/ufo_sightings_scrubbed.csv` (the Kaggle page is gone), approx 80,000 reports, approx 15mb.
- **Current state:** The backend is an Apollo Server with `sightings`, `sighting` and `createSighting`, backed by MySQL on the VM. The search and detail pages fetch sightings through Apollo Client, with search, sort and "load more" pagination done in the database. A temporary footer shows whether the backend is reachable.
- **Course context:** NTNU IT2810 project. Must be reproducible, accessible and sustainable.

## Tech stack

| Area | Choice |
| -------------------- | ------------------------------------------------------------------------- |
| Runtime | Node.js 22 or newer (`engines` in package.json) |
| Frontend | React, TypeScript, Vite, React Router |
| API | GraphQL (custom backend, Node + TypeScript, Apollo Server) |
| GraphQL client | Apollo Client |
| Database | MySQL (leaning, not final), run directly on the VM |
| Testing | Vitest + React Testing Library (components), e2e tool and API testing TBD |
| Linting / formatting | ESLint, Prettier |
| UI libraries | Leaflet with react-leaflet (OpenStreetMap tiles, no API key) |
| Area | Choice |
| -------------------- | ------------------------------------------------------------------------------------- |
| Runtime | Node.js 22 or newer (`engines` in package.json) |
| Frontend | React, TypeScript, Vite, React Router |
| API | GraphQL (custom backend, Node + TypeScript, Apollo Server) |
| GraphQL client | Apollo Client |
| Database | MySQL, run directly on the VM |
| Testing | Vitest + React Testing Library (components), API tests with node --test, e2e tool TBD |
| Linting / formatting | ESLint, Prettier |
| UI libraries | Leaflet with react-leaflet (OpenStreetMap tiles, no API key) |

## Commands

Expand All @@ -32,6 +32,13 @@ npm install
# Start frontend dev server (proxies /project2/graphql to the backend)
npm run dev

# Set up the database on the VM (once). Installs MySQL, creates the tables and
# imports the dataset. Full steps are in README.md.
bash backend/db-installation/my-sql-download.sh
python3 backend/data/cleanAndInsertDatatoDb.py verify
python3 backend/data/cleanAndInsertDatatoDb.py clean
python3 backend/data/cleanAndInsertDatatoDb.py insert

# Start backend dev server (GraphQL on http://localhost:3001, restarts on save)
npm run dev:backend

Expand Down Expand Up @@ -73,7 +80,7 @@ npm run test:watch
```
frontend/
src/
components/ Reusable UI (Banner, SearchBar, SightingList, SightingMap, BackendStatus)
components/ Reusable UI (Banner, SearchBar, SightingList, SightingMap, LocationPicker, BackendStatus)
pages/ Route-level components (SearchPage, SightingDetailPage, SubmitSightingPage)
hooks/ Custom hooks (debounce, sighting search, single sighting)
utils/ Pure helpers (formatting)
Expand All @@ -85,9 +92,15 @@ backend/ Node + TypeScript GraphQL server (Apollo Server 5)
index.ts Starts the server on PORT (default 3001)
schema.ts GraphQL type definitions and resolvers
validate-sighting.ts Server-side validation for createSighting
schema.test.ts, schema.db.test.ts API tests (faked and real database)
db/ MySQL connection pool and sighting queries
db-design/ create-tables.sql, the database schema
db-installation/ Script that installs MySQL on the VM
data/ NUFORC CSV and the script that cleans and imports it
dist/ Build output from `npm run build` (git-ignored)
# docs/
.github/workflows/ CI/CD (checks, then deploy on merge to main)
deploy/ Apache, systemd and sudoers files for the VM
docs/ ci-cd.md: CI, deploy, VM setup and rollback
```

## Code style
Expand Down Expand Up @@ -144,6 +157,8 @@ Never:
- **Backend port:** 3001
- **GraphQL endpoint:** The frontend calls `/project2/graphql` on its own origin. Apache must proxy it to `http://localhost:3001/graphql`, like the Vite dev proxy does locally. `VITE_GRAPHQL_URL` overrides the URL at build time if needed.
- **How to deploy:** Automatic. The `deploy` job in CI deploys every push to `main` that passes `checks`, and GitHub tracks it as the `production` environment. The frontend goes to `/var/www/html/project2`, the backend runs from `/srv/ufound-it` as the systemd service `ufound-it-backend`. One-time VM setup is in `docs/ci-cd.md`.
- **Backend settings on the VM:** `/srv/ufound-it/backend/.env`, readable with `sudo` (variables in `backend/.env.example`). CI never changes it, and it must never be committed.
- **Bad deploy:** roll back as described in `docs/ci-cd.md`.

## AI usage documentation

Expand Down
194 changes: 131 additions & 63 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,116 @@ Repository for team IT2810-H26-T25 in IT2810-H26.
1. [Running the Project](#running-the-project)
2. [Working in this repository](#working-in-this-repository)
3. [Project Description](#project-description)
4. [Planned Functionality](#planned-functionality)
4. [Functionality](#functionality)
5. [GraphQL API Design](#graphql-api-design)
6. [Stored User-Generated Data](#stored-user-generated-data)
7. [Technology Choices](#technology-choices)
8. [AI Usage](#ai-usage)

## Running the Project

To get a feel for how the application will look, you can run the project
locally. You need [Node.js](https://nodejs.org/) and npm installed. From the
repository root:
The project has three parts: a MySQL database, a GraphQL backend and a React
frontend. The live version runs at http://it2810-25.idi.ntnu.no/project2/.

### Requirements

- [Node.js](https://nodejs.org/) 22 or newer, with npm.
- For the database: MySQL 8 and Python 3 on a Linux machine (we use the course
VM).

### 1. Clone and install

```bash
git clone <repo-url>
cd <repo-folder>
npm install
```

### 2. Set up the database (once, on the VM)

The scripts need `sudo` and the `mysql` client. Run them from the repository
root:

```bash
# Installs MySQL, creates the database project_2_db and the user project_2_app
# with a random password, and creates the tables. The password is saved in
# ~/.config/project_2/mysql.cnf.
bash backend/db-installation/my-sql-download.sh

# Checks and cleans the NUFORC CSV (about 80,000 rows), then imports it.
python3 backend/data/cleanAndInsertDatatoDb.py verify
python3 backend/data/cleanAndInsertDatatoDb.py clean
python3 backend/data/cleanAndInsertDatatoDb.py insert
```

`insert` reads the password from `~/.config/project_2/mysql.cnf`. Use
`--mysql-config` to point it at another file.

### 3. Start the backend

The backend reads its database settings from `backend/.env`, which is not
committed. Copy `backend/.env.example` to `backend/.env` and set `DB_PASSWORD`
to the password from `~/.config/project_2/mysql.cnf`. Then:

```bash
npm run dev:backend
```

GraphQL is now served on http://localhost:3001/graphql. To run the built
version, as on the VM, use `npm run build` and then `npm run start -w backend`.

**Running locally against the VM's database.** Instead of installing MySQL,
open an SSH tunnel and keep the terminal open (needs the NTNU network or VPN):

```bash
ssh -L 3306:localhost:3306 <user>@it2810-25.idi.ntnu.no
```

Then use `DB_HOST=localhost` in `backend/.env` as usual.

### 4. Start the frontend

In a second terminal, from the repository root:

```bash
npm run dev
```

Then open the address Vite prints in the terminal, usually
http://localhost:5173. For this first submission there is no backend or
database to set up, because the app runs on example data stored in the
frontend. Other useful commands are `npm run build`, `npm run lint` and
`npm run test`.
Open the address Vite prints, usually http://localhost:5173/project2/. Vite
forwards `/project2/graphql` to the backend, so no CORS setup is needed. The
footer shows "Backend: unavailable" if the backend is not running or is on
another port. If port 3001 is taken, run `PORT=3002 npm run dev:backend` and
`BACKEND_PORT=3002 npm run dev`.

### Other commands

| Command | What it does |
| ---------------------- | ---------------------------------------------------------- |
| `npm run build` | Type-checks and builds the frontend and the backend. |
| `npm run lint` | ESLint for the frontend and the backend. |
| `npm run format:check` | Prettier check for the whole repository. |
| `npm run test` | Frontend and backend tests. No backend or database needed. |

The API tests that run against the real database are described in
[AGENTS.md](AGENTS.md#commands).

### Starting the backend on the VM

Everything needed to start the backend is in this repository and on the VM:

- The backend runs as the systemd service `ufound-it-backend` from
`/srv/ufound-it`, on port 3001. Apache forwards
`/project2/graphql` to it. Check it with
`sudo systemctl status ufound-it-backend` and
`journalctl -u ufound-it-backend -n 50`.
- The database settings are in `/srv/ufound-it/backend/.env` (readable with
`sudo`). The variables are listed in `backend/.env.example`. The database
password is also in `~/.config/project_2/mysql.cnf` for the account that ran
the install script.
- To start the backend by hand instead:
`cd /srv/ufound-it/backend && /srv/ufound-it/node --env-file=.env dist/index.js`.
- How the VM is set up, and how every merge to `main` is deployed, is in
[docs/ci-cd.md](docs/ci-cd.md).

## Working in this repository

Expand Down Expand Up @@ -57,10 +145,10 @@ UFOund it is a web application for exploring reported UFO sightings. You can
see where people have observed UFOs, what the objects looked like and what the
observers wrote about them.

The data comes from the
[UFO Sightings dataset on Kaggle](https://www.kaggle.com/datasets/NUFORC/ufo-sightings),
which contains about 80,000 reports collected by the National UFO Reporting
Center (NUFORC). Each report has a date and time, city, state, country, the
The data is a dataset of about 80,000 reports collected by the National UFO
Reporting Center ([NUFORC](https://nuforc.org/)). We got it from Kaggle, but
the dataset page has been removed, so the CSV file we use is kept in this
repository as `backend/data/ufo_sightings_scrubbed.csv`. Each report has a date and time, city, state, country, the
shape of the object, how long the sighting lasted, a free text comment from the
observer, the date it was posted and the latitude and longitude of the
location.
Expand All @@ -72,42 +160,39 @@ shapes and dates are natural things to sort and filter on. With 80,000 rows the
dataset is also large enough that searching, sorting and paging have to happen
on the server, which is what the course asks for.

For this first submission the application runs on a small set of hand written
example sightings stored in the frontend. There is no backend or database yet.
The sightings are stored in a MySQL database on the course VM and served
through a GraphQL API. Search, sorting and paging happen in the database, so
the browser only loads what it shows.

## Planned Functionality

**Featured sighting.** The landing page shows a featured sighting of the day
with its location on a map, its shape and the observer's comment.
## Functionality

**Search.** A search field finds sightings by city, shape or words in the
comment. The search is case insensitive and waits until the user stops typing
before it runs, so it does not send a request for every keystroke. This already
works on the example data.
before it runs, so it does not send a request for every keystroke.

**List view.** Search results are shown as a list where each sighting has a
small map, the location, date, shape, duration and comment. Clicking anywhere on
a sighting opens its detail page. Since a search can match thousands of
sightings, the list will either be split into pages or load more results as the
user scrolls. We have not decided which yet.
small map, the location, date, shape, duration and comment. The list loads 20
sightings at a time, and a "Load more" button fetches the next page. Clicking
anywhere on a sighting opens its detail page.

**Detail view.** Each sighting has its own page with a larger map that can be
dragged and zoomed, along with all the information about the report. The page
has its own URL, so a sighting can be shared or bookmarked.

**Sorting and filtering.** Results can be sorted by newest or oldest first, or
alphabetically by city or shape. The search and sort choices are kept in the
URL, so they are still there when the user returns from a detail page or
refreshes.
**Sorting.** Results can be sorted by newest or oldest first, or
alphabetically by city or shape. The sorting runs in the database, so it covers
all results and not only the ones loaded. The search and sort choices are kept
in the URL, so they are still there when the user returns from a detail page
or refreshes.

**Adding sightings.** Users will be able to report their own sightings, which
are then stored in the database and shown alongside the ones from the dataset.
**Adding sightings.** Users can report their own sightings with a form that
includes a map to pick the location. The server validates the input, and the
sighting is stored in the database and shown alongside the ones from the
dataset.

Right now the search, sorting and filtering run in the browser on the example
data, so we can try out how it feels. In the finished version this will be
replaced by GraphQL queries against the backend, so that sorting and filtering
always cover the whole dataset and not only what has been loaded in the
browser.
**Not done yet.** The API can filter by shape, country, state and date range,
but the interface has no filter controls yet. End-to-end tests are also still
missing.

## GraphQL API Design

Expand All @@ -131,15 +216,13 @@ what it shows, and Apollo Client can merge the pages into one list. The trade-of

## Stored User-Generated Data

The main data users create are their own sighting reports. A report will have
The main data users create are their own sighting reports. A report has
the same fields as the sightings in the dataset, which means location, date and
time, shape, duration and a comment. These are stored permanently in the
database.

We have not yet decided whether users will log in. Without accounts, reports
are stored without being tied to a user. With accounts, users could see and
manage their own reports, and we could later add comments or up votes on
sightings, which is an idea we would like to explore if there is time.
There are no user accounts, so reports are stored without being tied to a
user. They are marked as user submitted, to tell them apart from the dataset. (You cannot tell this in the frontend yet)

## Technology Choices

Expand All @@ -148,32 +231,17 @@ navigation between the search page and the detail pages. Maps are made with
Leaflet through react-leaflet, using map tiles from OpenStreetMap, which is free
and needs no API key.

The backend will be a Node.js server written in TypeScript with a GraphQL API.
We plan to use Apollo Server on the backend and Apollo Client in the frontend,
since Apollo is well documented and handles caching of query results for us.
For the database we are leaning towards MySQL, but this is not final. The
backend and database will run directly on the course virtual machine, without
Docker.
The backend is a Node.js server written in TypeScript with a GraphQL API. We
use Apollo Server on the backend and Apollo Client in the frontend, since
Apollo is well documented and handles caching of query results for us. The data
is stored in MySQL. The backend and database run directly on the course virtual machine.

We will test components with Vitest and React Testing Library, and add end to
end tests and API tests as the application grows. ESLint and Prettier keep the
We test components with Vitest and React Testing Library, and the GraphQL API
with Node's built-in test runner, both against a faked database and against the
real one. End to end tests are still to come. ESLint and Prettier keep the
code consistent, and every pull request is reviewed by another group member
before it is merged into main.

## Running the backend against the VM database

The backend reads its database settings from `backend/.env`, which is not
committed. To run it locally against the database on the course VM:

1. Open an SSH tunnel and keep the terminal open. `ssh -L 3306:localhost:3306 <user>@it2810-25.idi.ntnu.no`
(needs NTNU network or VPN).
2. Copy `backend/.env.example` to `backend/.env` and fill in `DB_PASSWORD` and
`DB_NAME`
3. Run `npm run dev:backend`.

On the VM the same `.env` file is used with `DB_HOST=localhost`, and the
backend is started with `npm run start -w backend`.

## AI Usage

We use AI coding assistants, mainly Claude Code, to generate code, fix bugs and
Expand Down
Loading