|
|
cc12f97c20
|
fix: increase post and comment fetch limits
|
2026-02-09 21:20:57 +00:00 |
|
|
|
ec91904481
|
refactor(dataset creation): update API methods to return only posts
|
2026-02-09 21:20:08 +00:00 |
|
|
|
e72d336de4
|
remove /data subdir
Dataset is now created in the pwd. Avoids issues if the folder didn't exist
|
2026-01-27 12:36:01 +00:00 |
|
|
|
7d94494fe2
|
youtube connector returns posts and comments in a flat manner
|
2026-01-24 20:19:15 +00:00 |
|
|
|
152264bda9
|
separate comment and post data structures
This allows for a flat data structure, benefical to data analysis
|
2026-01-22 15:53:47 +00:00 |
|
|
|
3c4aad77ef
|
update number of fetched comments and videos from youtube
|
2026-01-22 15:29:55 +00:00 |
|
|
|
187401c5eb
|
Implement YouTube API integration for video and comment fetching
|
2026-01-19 20:50:17 +00:00 |
|
|
|
9c66ec8b82
|
Save to jsonl file after every fetch
Reduces errors and lost data
|
2026-01-19 20:22:47 +00:00 |
|
|
|
193ff43975
|
Refactor dataset creation to use post_to_dict for improved data structure and limit API calls to 400
|
2026-01-17 22:14:15 +00:00 |
|
|
|
38cf57e198
|
Include Ireland posts in dataset creation
|
2026-01-17 16:05:42 +00:00 |
|
|
|
d44b247bda
|
rename dataset output to "posts.json"
|
2026-01-17 14:52:32 +00:00 |
|
|
|
610bab67d5
|
Add boards.ie to dataset creation & add logging config
|
2026-01-17 14:43:56 +00:00 |
|
|
|
47e71113f6
|
Merge branch 'main' of github:ThisBirchWood/ethnograph-view
|
2026-01-15 12:43:53 +00:00 |
|
|
|
b0e079599a
|
Rename fetch data script & add check for empty posts
|
2026-01-13 19:06:00 +00:00 |
|
