Use source-specific polling settings in collection
- Read max_posts from source.max_posts (fallback to 100) - Read fetch_comments from source settings - Allows customizing collection per source 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -161,14 +161,14 @@ class PollingService:
|
|||||||
end_iso = end_date.isoformat()
|
end_iso = end_date.isoformat()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Call the existing collect_platform function
|
# Call the existing collect_platform function using source settings
|
||||||
posts_collected = collect_platform(
|
posts_collected = collect_platform(
|
||||||
platform=source.platform,
|
platform=source.platform,
|
||||||
community=source.source_id,
|
community=source.source_id,
|
||||||
start_date=start_iso,
|
start_date=start_iso,
|
||||||
end_date=end_iso,
|
end_date=end_iso,
|
||||||
max_posts=100, # Default limit
|
max_posts=source.max_posts or 100,
|
||||||
fetch_comments=True,
|
fetch_comments=source.fetch_comments if hasattr(source, 'fetch_comments') else True,
|
||||||
index=index,
|
index=index,
|
||||||
dirs=dirs
|
dirs=dirs
|
||||||
)
|
)
|
||||||
|
|||||||
Reference in New Issue
Block a user