dylan/crosspost
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Add comment limit to _parse_comments method in BoardsAPI

Browse Source 
Some boards.ie threads have thousands of comments which is slow to fetch with pagination
This commit is contained in:
Dylan De Faoite
2026-01-19 20:23:11 +00:00
parent 9c66ec8b82
commit 85388ef6aa
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
connectors/boards_api.py
View File

@@ -105,11 +105,11 @@ class BoardsAPI:
return post return post
def _parse_comments(self, url: str, post_id: str) -> list[Comment]: def _parse_comments(self, url: str, post_id: str, comment_limit: int = 500) -> list[Comment]:
comments = [] comments = []
current_url = url current_url = url
while current_url: while current_url and len(comments) < comment_limit:
html = self._fetch_page(current_url) html = self._fetch_page(current_url)
page_comments = self._parse_page_comments(html, post_id) page_comments = self._parse_page_comments(html, post_id)
comments.extend(page_comments) comments.extend(page_comments)