You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following up on #22 — splitting out the smaller changes into their own PRs (#25, #26, #27), and opening this issue first for Firecrawl specifically, as you asked, since it's a bigger change and adds a paid dependency.
The idea: JSearch and Adzuna each index one source. Firecrawl instead runs a real web search, so Dossier doesn't need a bespoke scraper per job board (which breaks the moment a site's markup changes, and risks its terms of use). Concretely:
A fourth JobSourceAdapter, discovery-only: it returns postings from a search, and reuses the existing guarded fetchPosting() for each posting's full text, exactly the way Adzuna's adapter already does.
Two Firecrawl /v2/search calls per page — one query biased toward a fixed list of job-board domains (LinkedIn, Indeed, Naukri, Glassdoor, Greenhouse, Lever, Ashby, Workday, SmartRecruiters, …), one with no site restriction at all — merged and de-duplicated by URL. The open query catches a posting on a board outside that list, or a company's own careers page; the biased one keeps results relevant when the open query is noisy.
Needs a Firecrawl API key (DOSSIER_FIRECRAWL_KEY, or saved in Settings → Job search, same pattern as the other sources' keys).
One Prisma migration, for the new PostingSource enum value.
It's the first job source that costs money per search rather than offering a free tier sized for personal use (Firecrawl has a free trial, then pay-per-search).
It's the first source where Dossier doesn't control the result shape at all — whatever Firecrawl's search index returns, including sites Dossier wouldn't otherwise touch — so the usual "Dossier only talks to sources you choose" framing needs a sentence acknowledging that a web search is broader by nature.
A working reference implementation already exists on my fork (rahulrathore95/dossier, src/lib/job-search/firecrawl.ts, with tests) if you want to look at the actual code before we agree on the approach. Happy to adjust the design based on your take, then open the PR once we're aligned.
Activity
rahulrathore95 commentedon Oct 28, 2026
Closing — I'm going to keep working on my own fork rather than contributing this upstream. Thanks for engaging with it.