Use source-specific polling settings in collection

- Read max_posts from source.max_posts (fallback to 100)
- Read fetch_comments from source settings
- Allows customizing collection per source

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2025-10-11 18:36:32 -05:00
parent 5d3b01926c
commit 99d51fe14a

View File

@@ -161,14 +161,14 @@ class PollingService:
end_iso = end_date.isoformat()
try:
# Call the existing collect_platform function
# Call the existing collect_platform function using source settings
posts_collected = collect_platform(
platform=source.platform,
community=source.source_id,
start_date=start_iso,
end_date=end_iso,
max_posts=100, # Default limit
fetch_comments=True,
max_posts=source.max_posts or 100,
fetch_comments=source.fetch_comments if hasattr(source, 'fetch_comments') else True,
index=index,
dirs=dirs
)