Download
Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites.
Features
- Easily crawl websites to extract information
- Supports setting crawl depth and controlling the number of pages
- Handles redirects and follows links intelligently
- Allows filtering of URLs using custom rules
- Supports asynchronous crawling for better performance
- Customizable callbacks for processing crawled pages
Project Samples
Categories
LibrariesLicense
MIT LicenseFollow Spatie Crawler
Other Useful Business Software
Catch Bugs Before Your Customers Do
Move from alert to fix before users notice. AppSignal monitors errors, performance bottlenecks, host health, and uptime—all from one dashboard. Instant notifications on deployments, anomaly triggers for memory spikes or error surges, and seamless log management. Works out of the box with Rails, Django, Express, Phoenix, Next.js, and dozens more. Starts at $23/month with no hidden fees.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of Spatie Crawler!
