feat: strict remote-only filtering with positive remote evidence
LinkedIn's f_WT=2 filter is nearly ignored for guest scraping and guest pages expose no structured workplace-type field, so text filtering is the only option: - Require positive remote evidence (title/location/description must say remote) before keeping a job; silence now means drop, not keep. Configurable via scraper.require_remote_evidence. - Catch more hybrid/on-site phrasings: "Hybrid: <city>", "office-based", "N days onsite/in office", "return to office", titles with hybrid/onsite (without tripping on "hybrid cloud"). - Append "Remote" to search keywords when remote_only is on, since keyword matching surfaces far more genuinely remote posts than the broken filter. Verified against live LinkedIn postings: three known hybrid/on-site leaks dropped, known remote post kept. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -49,6 +49,11 @@ scraper:
|
||||
location: "Canada"
|
||||
work_types: ["remote"]
|
||||
remote_only: true # also drop jobs whose location/description reveals hybrid or on-site
|
||||
# LinkedIn's remote filter is unreliable for logged-out scraping, so on top of
|
||||
# dropping visible hybrid/on-site jobs, require positive proof of remote
|
||||
# (title/location/description must say remote). Strict: jobs that never state
|
||||
# they're remote are dropped. Set false to keep unverified jobs.
|
||||
require_remote_evidence: true
|
||||
pages_per_keyword: 3
|
||||
delay_between_pages: [3, 6]
|
||||
delay_between_keywords: [5, 10]
|
||||
|
||||
Reference in New Issue
Block a user