Add comment limit to _parse_comments method in BoardsAPI

Some boards.ie threads have thousands of comments which is slow to fetch with pagination
This commit is contained in:
2026-01-19 20:23:11 +00:00
parent 9c66ec8b82
commit 85388ef6aa

View File

@@ -105,11 +105,11 @@ class BoardsAPI:
return post
def _parse_comments(self, url: str, post_id: str) -> list[Comment]:
def _parse_comments(self, url: str, post_id: str, comment_limit: int = 500) -> list[Comment]:
comments = []
current_url = url
while current_url:
while current_url and len(comments) < comment_limit:
html = self._fetch_page(current_url)
page_comments = self._parse_page_comments(html, post_id)
comments.extend(page_comments)