Skip to content

60 ronny dataimporting #67

Jump to bottom
Merged
merged 3 commits into from
Oct 15, 2026
Merged

60 ronny dataimporting #67

merged 3 commits into from
Oct 15, 2026
+473 −169

Conversation

ronnenga
Copy link
Member

Summary

Improved the Google Books API importing system and prepared it for larger database imports.

Changes

  • Added configurable subjects, languages, and import limits.
  • Implemented pagination and round-robin importing across subjects.
  • Added MongoDB progress tracking to resume imports across executions.
  • Added English-language filtering and duplicate-safe database updates.
  • Improved API error handling, retry reporting, and request limits.
  • Added a lightweight sample importer for local frontend development.
  • Renamed book and shelf service files for clearer naming.

Testing

  • Successfully built the TypeScript backend.
  • Tested importing books into local MongoDB.
  • Verified saved import progress across multiple runs.
  • Verified duplicate handling and API request limits.

Notes

  • Automatic scheduling and deployment to the VM are not yet implemented.

@ronnenga ronnenga requested a review from a team as a code owner October 15, 2026 09:22
@ronnenga ronnenga linked an issue Oct 15, 2026 that may be closed by this pull request
Improve Google Data Importing #60
Closed
ilsea
ilsea requested changes Oct 15, 2026
View reviewed changes
Copy link
Member

@ilsea ilsea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comments to show what files are for

@ilsea ilsea merged commit f9e4746 into main Oct 15, 2026
@ilsea ilsea deleted the 60-ronny-dataimporting branch October 15, 2026 09:36
Sign in to join this conversation on GitHub.
Reviewers

@ilsea ilsea ilsea approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Improve Google Data Importing
2 participants
@ronnenga @ilsea