Commit Graph

8 Commits

Author SHA1 Message Date
376773a0cc style: run python linter & prettifier on backend code 2026-03-25 19:34:43 +00:00
01d6bd0164 fix(connectors): category / search fields breaking
Ideally category and search are fully optional, however some sites break if one or the other is not provided.

Unfortuntely `boards.ie` has a different page type for searches and I'm not bothered to implement a scraper from scratch.

In addition, removed comment limit options.
2026-03-11 21:16:26 +00:00
2ab74d922a feat(api): support per-source search, category and limit configuration 2026-03-10 23:15:33 +00:00
dc330b87b9 fix(celery): process dataset directly in fetch task
Calling the original `process_dataset` function led to issues with JSON serialisation.
2026-03-10 22:17:00 +00:00
a65c4a461c fix(api): flask delegates dataset fetch to celery 2026-03-10 19:17:41 +00:00
e20d0689e8 fix(celery): adjust try-catch logic to improve error handling
Capturing the instantiation of the database and dataset manager objects inside the try-catch will cause errors if something else fails.

If an exception occurs and the dataset_manager is not initialised, the code inside the catch block will fail.
2026-03-04 21:18:59 +00:00
eb4187c559 feat(api): add status returns for NLP processing 2026-03-03 13:46:37 +00:00
3a58705635 feat: add celery & redis for background data processing 2026-03-03 12:27:14 +00:00