|
|
d7baf39087
|
Implement exponential backoff for handling Reddit API rate limits in _fetch_data method
|
2026-01-17 22:14:26 +00:00 |
|
|
|
193ff43975
|
Refactor dataset creation to use post_to_dict for improved data structure and limit API calls to 400
|
2026-01-17 22:14:15 +00:00 |
|
|
|
1d2865470b
|
Add comment parsing to _parse_posts
|
2026-01-17 18:20:21 +00:00 |
|
|
|
db21e86b8e
|
Fix post ID extraction in _parse_thread method
|
2026-01-17 16:18:04 +00:00 |
|
|
|
09d12ae173
|
Add logging to reddit api class
|
2026-01-17 16:12:18 +00:00 |
|
|
|
38cf57e198
|
Include Ireland posts in dataset creation
|
2026-01-17 16:05:42 +00:00 |
|
|
|
ed3d89fd27
|
Refactor post fetching to use ThreadPoolExecutor for improved concurrency
|
2026-01-17 16:05:37 +00:00 |
|
|
|
d44b247bda
|
rename dataset output to "posts.json"
|
2026-01-17 14:52:32 +00:00 |
|
|
|
d5e6b7a895
|
Refactor post detail fetching into separate _parse_thread method
|
2026-01-17 14:51:57 +00:00 |
|
|
|
610bab67d5
|
Add boards.ie to dataset creation & add logging config
|
2026-01-17 14:43:56 +00:00 |
|
|
|
b8ed409e04
|
implement slight efficiency gain in board.ie pagination
|
2026-01-17 14:43:14 +00:00 |
|
|
|
0523c1a091
|
Refactor logging to use class logger in BoardsAPI
|
2026-01-17 14:37:28 +00:00 |
|
|
|
a1c1e1e0d8
|
patch broken title scrape
|
2026-01-17 14:28:16 +00:00 |
|
|
|
9eec7b00e3
|
Implement BoardsAPI to fetch new category posts and their details
|
2026-01-17 14:25:43 +00:00 |
|
|
|
c3a81d8b01
|
update requirements.txt
|
2026-01-17 13:59:43 +00:00 |
|
|
|
ad416d4966
|
Add Comment DTO
|
2026-01-17 13:59:35 +00:00 |
|
|
|
47e71113f6
|
Merge branch 'main' of github:ThisBirchWood/ethnograph-view
|
2026-01-15 12:43:53 +00:00 |
|
|
|
b0e079599a
|
Rename fetch data script & add check for empty posts
|
2026-01-13 19:06:00 +00:00 |
|
|
|
538ea9fe12
|
Remove database connection and schema setup from the project
|
2026-01-13 19:01:18 +00:00 |
|
|
|
73a19f3ce3
|
Add script to orchestrate dataset creation
|
2026-01-13 18:59:42 +00:00 |
|
|
|
e58c18bf99
|
add json files and vscode workspaces to gitignore
|
2026-01-13 18:57:29 +00:00 |
|
|
|
d4fb78aac4
|
Add pagination to new_subreddit method to bypass 100 post limit
|
2026-01-13 18:46:43 +00:00 |
|
|
|
05874d233f
|
Implement subreddit search method for new posts
|
2026-01-13 18:39:55 +00:00 |
|
|
|
b5624035ec
|
rename reddit_connecter to reddit_api
|
2026-01-13 14:45:20 +00:00 |
|
|
|
7c01c335fa
|
remove base_connector and remove non-subreddit specific methods
Project will focus on specific communities, not enact a reddit-wide search
|
2026-01-13 14:19:43 +00:00 |
|
|
|
62823bfd44
|
update requirements.txt
|
2026-01-12 15:31:14 +00:00 |
|
|
|
0cc95c5358
|
add ID field to post dto
|
2026-01-11 20:36:19 +00:00 |
|
|
|
68642709b7
|
add rudimentary sentiment analysis endpoint to calculate average sentiment of posts
|
2026-01-11 17:31:37 +00:00 |
|
|
|
4d459f2035
|
update main.py to launch flask app
|
2026-01-11 17:21:09 +00:00 |
|
|
|
195188dcd7
|
update User-agent header in _fetch_data method and add __exit__ method to Database class
|
2026-01-11 15:30:34 +00:00 |
|
|
|
b5a2b01402
|
remove debug print statements from fetch_subreddit function
|
2026-01-11 15:11:49 +00:00 |
|
|
|
2a8e3fd4db
|
update README to clarify requirements
|
2026-01-11 15:11:09 +00:00 |
|
|
|
4b8aebd312
|
add fetch_subreddit endpoint to retrieve and insert top posts from a specified subreddit
|
2026-01-11 15:07:44 +00:00 |
|
|
|
d3c985ba1f
|
update posts table schema to include title and author_username fields
|
2026-01-11 14:44:34 +00:00 |
|
|
|
5e1bccb2a8
|
add execute_many method to Database class and update fetch_reddit endpoint to insert posts into database
|
2026-01-11 14:44:22 +00:00 |
|
|
|
1e9eb11aa1
|
update docker-compose to drop volume on restart
For faster development of database schema
|
2026-01-11 14:44:12 +00:00 |
|
|
|
c12e1f7623
|
add database schema and update docker-compose for initialization
|
2026-01-11 14:22:46 +00:00 |
|
|
|
550bba62f7
|
add Reddit post search endpoint
|
2026-01-11 13:46:11 +00:00 |
|
|
|
1907760f1a
|
add docker-compose file and configure PostgreSQL service; update database credentials in app.py
|
2026-01-11 13:38:52 +00:00 |
|
|
|
fd437db4eb
|
add basic flask server and requirements
|
2026-01-11 13:35:41 +00:00 |
|
|
|
19dff39dff
|
add requirements section to readme
|
2026-01-11 13:35:25 +00:00 |
|
|
|
2034cf6537
|
update readme
|
2026-01-11 13:17:37 +00:00 |
|
|
|
6adfc2faf5
|
first commit
|
2025-11-10 19:30:19 +00:00 |
|
|
|
11755358a4
|
first commit
|
2025-11-10 19:30:05 +00:00 |
|
|
|
be61a9c7ae
|
ADD requirements.txt & Database class
|
2025-11-06 15:45:09 +00:00 |
|
|
|
a9778f26b2
|
ADD: Subreddit search
|
2025-11-06 14:41:46 +00:00 |
|
|
|
6533bc2f13
|
PATCH: Broken URL handling
|
2025-11-06 14:28:41 +00:00 |
|
|
|
9f5c6a9647
|
Initial Commit
|
2025-11-06 14:15:50 +00:00 |
|
