Skip to content
Merged

Dev #17

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,13 @@
node_modules
dist
coverage
.env
.env.*
!.env.example
*.log
playwright-report
test-results
.DS_Store
.vscode/*
!.vscode/extensions.json
.idea
7 changes: 7 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
dist
coverage
node_modules
pnpm-lock.yaml
documents
README.md
.github
6 changes: 6 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"singleQuote": true,
"semi": true,
"printWidth": 100,
"trailingComma": "all"
}
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ Datasettet har rundt 900 000 spill, så søk, filtrering og sortering må skje i

Datasettet lastes ned og importeres i en PostgreSQL-database som kjører på serveren. Klienten henter ikke data direkte fra databasen, men går via et GraphQL-API på serveren.

For oppsett av database se egen [dokumentasjon](db/database-setup.md).

For databasens layout (dvs. tabeller, kolonner, etc.) se [database-layout](db/database-layout.md).

For bruk av databasen se [database-usage](db/database-usage.md).

## Skisse

![Skisse av applikasjonen](documents/game-showcase.png)
Expand Down
2 changes: 2 additions & 0 deletions backend/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
DATABASE_URL=postgresql://gameapp:<FYLL-INN-PASSORD>@localhost:5432/gamesdb
PORT=3001
24 changes: 0 additions & 24 deletions backend/.gitignore

This file was deleted.

18 changes: 18 additions & 0 deletions backend/eslint.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import js from '@eslint/js';
import globals from 'globals';
import tseslint from 'typescript-eslint';
import eslintConfigPrettier from 'eslint-config-prettier/flat';
import { defineConfig, globalIgnores } from 'eslint/config';

export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.ts'],
extends: [js.configs.recommended, tseslint.configs.recommended],
languageOptions: {
ecmaVersion: 2023,
globals: globals.node,
},
},
eslintConfigPrettier,
]);
9 changes: 0 additions & 9 deletions backend/eslint.config.ts

This file was deleted.

24 changes: 10 additions & 14 deletions backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,34 +4,30 @@
"description": "",
"main": "index.js",
"scripts": {
"test": "vitest run",
"dev": "tsx watch src/index.ts",
"build": "tsc",
"start": "node dist/index.js",
"lint": "eslint ."
},
"keywords": [],
"author": "",
"license": "ISC",
"devEngines": {
"packageManager": {
"name": "pnpm",
"version": "12.6.0",
"onFail": "download"
}
},
"packageManager": "pnpm@12.6.0",
"type": "module",
"dependencies": {
"@apollo/server": "^5.5.1",
"dotenv": "^18.0.5",
"graphql": "^17.0.2",
"graphql-tag": "^2.12.7",
"pg": "^8.23.0"
"pg": "^8.23.1"
},
"devDependencies": {
"@eslint/js": "^10.0.1",
"@types/node": "^24.19.0",
"@types/pg": "^8.23.1",
"eslint": "^10.11.0",
"eslint-config-prettier": "^10.1.8",
"globals": "^17.12.0",
"jiti": "^2.7.0",
"typescript-eslint": "^8.70.1",
"vitest": "^5.0.2"
"tsx": "^4.23.15",
"typescript": "~6.0.3",
"typescript-eslint": "^8.71.0"
}
}
2 changes: 0 additions & 2 deletions backend/pnpm-workspace.yaml

This file was deleted.

2 changes: 2 additions & 0 deletions backend/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
// placeholder, erstattes av apollo server senere
console.log('backend');
14 changes: 14 additions & 0 deletions backend/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"compilerOptions": {
"target": "ES2023",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"outDir": "dist",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src"]
}
47 changes: 47 additions & 0 deletions db/database-layout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@

#### Database-layout

Under er utskrift fra postgres lokalt på vm. Eksemplene viser henholdsvis alle tabellene i databasen vår `gamesdb`, og alle kolonnene som tabellen `games` inneholder.

```shell
gamesdb-> \dt
List of tables
Schema | Name | Type | Owner
--------+-----------------+-------+---------
public | developers | table | gameapp
public | game_developers | table | gameapp
public | game_genres | table | gameapp
public | game_platforms | table | gameapp
public | game_publishers | table | gameapp
public | game_stores | table | gameapp
public | game_tags | table | gameapp
public | games | table | gameapp
public | genres | table | gameapp
public | platforms | table | gameapp
public | publishers | table | gameapp
public | reviews | table | gameapp
public | stores | table | gameapp
public | tags | table | gameapp
(14 rows)

gamesdb-> \d games
Table "public.games"
Column | Type | Collation | Nullable | Default
------------------+---------+-----------+----------+---------
id | bigint | | not null |
slug | text | | |
name | text | | not null |
released | date | | |
background_image | text | | |
rating | real | | |
ratings_count | integer | | not null | 0
added | integer | | not null | 0
metacritic | integer | | |
playtime | integer | | |
esrb_rating | text | | |
description | text | | |
website | text | | |
dominant_color | text | | |

```

73 changes: 73 additions & 0 deletions db/database-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Database-oppsett

PostgreSQL kjører direkte på VM-en og er kun tilgjengelig fra `localhost`. Dataene kommer fra [RAWG-datasettet på Hugging Face](https://huggingface.co/datasets/atalaydenknalbant/video-games-dataset), filtrert til spill med registrert brukerengasjement (`added > 0`, ca. 120 000 av 899 000 rader)

## 1. Installer Postgres

```bash
sudo apt install postgresql postgresql-contrib
sudo systemctl enable --now postgresql
```

Installerer postgres, sørger for at postgres.service kjører på startup (`enable`), og starter postgres.service (`--now`).

## 2. Opprett rolle, database og extension

```bash
sudo -u postgres psql
```

```sql
CREATE USER gameapp WITH PASSWORD '<passord>';
CREATE DATABASE gamesdb OWNER gameapp;
\c gamesdb
CREATE EXTENSION IF NOT EXISTS pg_trgm;
\q
```

`pg_trgm` gir indeksert, case-insensitivt delstrengsøk (`ILIKE '%term%'`).

## 3. Verifiser at Postgres kun lytter lokalt

```bash
sudo ss -tlnp | grep 5432
```

Skal kun vise `127.0.0.1:5432` og `[::1]:5432`.

## 4. Last ned datasettet

```bash
pip install -U huggingface_hub
hf download atalaydenknalbant/video-games-dataset videogames_data.parquet \
--repo-type dataset --local-dir ~/data
```

Parquet-filen (ca. 1 GB) brukes i stedet for JSONL (4,3 GB) og CSV (2 GB).

## 5. Kjør importen

```bash
cd backend/db
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
export DATABASE_URL='postgresql://gameapp:<passord>@localhost:5432/gamesdb'
python load.py
```

`load.py` leser Parquet-filen med DuckDB, deler opp flerverdi-felter til oppslags- og koblingstabeller, laster alt med `COPY`, oppretter indekser etter lasting og kjører `ANALYZE`. Filstien kan overstyres med `PARQUET_PATH`. Finnes tabellene allerede, avbryter scriptet. `python load.py --force` sletter og laster alt på nytt, inkludert reviews.

| Fil | Innhold |
| --- | --- |
| `schema.sql` | Tabeller: `games`, oppslagstabeller, koblingstabeller, `reviews` |
| `indexes.sql` | Trigram-indeks for søk, keyset-indekser for sortering, omvendte indekser for filtrering |
| `load.py` | Import fra Parquet til Postgres |

## 6. Verifiser

```bash
psql "$DATABASE_URL" -c "SELECT count(*) FROM games;"
psql "$DATABASE_URL" -c "EXPLAIN ANALYZE SELECT id, name FROM games WHERE name ILIKE '%zelda%' LIMIT 20;"
```

Første skal gi ca. 120 067. Den andre skal vise `Bitmap Index Scan on games_name_trgm` (målt til 0,7 ms).
29 changes: 29 additions & 0 deletions db/database-usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

#### Database-usage

Under er noen generelle kommandoer som kan brukes i postgres for navigere databasen:

```
\l list databaser
\c gamesdb bytt database
\dt list tabeller
\d games kolonner, typer, indekser og constraints for en tabell
\d+ games samme, med størrelse og beskrivelser
\di list indekser
\dx list extensions (pg_trgm)
\du list roller
\x veksle utvidet visning (én kolonne per linje, nyttig for lange rader)
\timing vis kjøretid for hver spørring
\q avslutt
```

Se på data i tabellen (spørringer):
```SQL
// viser første 5 spillene i games-tabellen
SELECT * FROM games LIMIT 5;

// viser metacritic score for spill med id 4291
SELECT metacritic FROM games WHERE id = 4291;

```
Loading
Loading