fix(connectors): category / search fields breaking

Ideally category and search are fully optional, however some sites break if one or the other is not provided.

Unfortuntely `boards.ie` has a different page type for searches and I'm not bothered to implement a scraper from scratch.

In addition, removed comment limit options.
This commit is contained in:
2026-03-11 21:16:26 +00:00
parent 12cbc24074
commit 01d6bd0164
4 changed files with 48 additions and 38 deletions

View File

@@ -46,8 +46,7 @@ def fetch_and_process_dataset(self,
raw_posts = connector.get_new_posts_by_search(
search=search,
category=category,
post_limit=limit,
comment_limit=limit
post_limit=limit
)
posts.extend(post.to_dict() for post in raw_posts)