Merged issue 46 with develop #59
Merged
+553
−20
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added a web-scraper that will scrape relevant information for the charity about-us page. Currently supports scraping description String.
Added method to get the URL of the logo. Converted WebDriverWait object to an object parameter, and initialize it in the constructor.
Changed driver and wait initialization to be for each method instead of constructor. Added a method for getting categories from charity URL.
Added method updateKeyNumbers() that gets the value of the 3 key number values by IK for the given charity.
Fixed updateDescription so it doesn't output some of the text twice, and it now returns the String.
Changed methods to return their values, and made the url a object parameter.
Added a web-scraper that will scrape relevant information for the charity about-us page. Currently supports scraping description String.
Added method to get the URL of the logo. Converted WebDriverWait object to an object parameter, and initialize it in the constructor.
Changed driver and wait initialization to be for each method instead of constructor. Added a method for getting categories from charity URL.
Added method updateKeyNumbers() that gets the value of the 3 key number values by IK for the given charity.
Fixed updateDescription so it doesn't output some of the text twice, and it now returns the String.
Changed methods to return their values, and made the url a object parameter.
Fixed method to get description to be more robust, and it now correctly spaces paragraphs. Made scraping methods private, and a public method to run them all. Restructured the class for easier testing with mockito.
Edited methods to return strings instead for easier uploading to database.
Removed unused collections import statement.
Changed description parameter to only contain URL to pass to the URLCharityScraper
Added 4 new columns: description, logoURL, categories, and key_values to the create table method. Creates and calls a URLCharityScraper object on updating API values that populates these new columns using the URL for the charity given from the API.
Changed getDescription test to use getURL instead.
Changed getDescription test to use getURL instead.
Test file for URLCharityScraper with as much coverage as possible.
…tion' of git.ntnu.no:cathrkri/systemutviklingTeam6 into 46-mvp-webscrabing-av-innsamlingskontrollen-for-description # Conflicts: # helpmehelpapplication/src/main/java/ntnu/systemutvikling/team6/scraper/URLCharityScraper.java
Added parameters for the new SQL entries to the Charity object.
Adds the scraped parameters directly to the charity object.
Fixed getDescription test.
Temp fix to make tests work by using a valid charity URL.
Temp fix to make tests work.
Temp fix to make tests work.
roaraf
deleted the
46-mvp-webscrabing-av-innsamlingskontrollen-for-description
branch
Sign in
to join this conversation on GitHub.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.