Crawler Detect Files

CrawlerDetect is a PHP class for detecting bots/crawlers/spiders

This is an exact mirror of the Crawler Detect project, hosted at https://github.com/JayBizzle/Crawler-Detect. SourceForge is not affiliated with Crawler Detect. For more information, see the SourceForge Open Source Mirror Directory.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
README.md	2025-09-30	423 Bytes	0
v1.3.6 source code.tar.gz	2025-09-30	32.5 kB	0
v1.3.6 source code.zip	2025-09-30	42.1 kB	0
Totals: 3 Items		75.0 kB	0

What's Changed

Added Mailpit to raw files by @loadkpi in https://github.com/JayBizzle/Crawler-Detect/pull/568
GitLab detection by @dormadekhin in https://github.com/JayBizzle/Crawler-Detect/pull/573

New Contributors

@dormadekhin made their first contribution in https://github.com/JayBizzle/Crawler-Detect/pull/573

Full Changelog: https://github.com/JayBizzle/Crawler-Detect/compare/v1.3.5...v1.3.6

Source: README.md, updated 2025-09-30

Other Useful Business Software

Gen AI apps are built with MongoDB Atlas

The database for AI-powered applications.

MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.

Start Free

Simple, Secure Domain Registration

Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.

Recommended Projects

RobotsTxt
This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is...
Spatie Crawler
Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl...
OpenSearchServer Search Engine
OpenSearchServer is a powerful, enterprise-class, search engine program. Using the web user interface, the crawlers (web, file, database, etc.) and the client libraries (REST/API , Ruby, Rails, Node.js, PHP, Perl) you will be able to integrate quickly and easily advanced full-text search...
DNS Crawler
DNS Crawler is a lightweight, Python-based utility designed for efficient batch processing and assessment of internet domain names. It reads from a list of domains formatted as: domain_name <tab> or ; optional_comment and generates a detailed, Excel-compatible CSV report with columns...
RobotsDisallowed
RobotsDisallowed is a public catalog that tracks websites and organizations explicitly blocking AI and web-scraping crawlers in their robots.txt or related mechanisms. It focuses on documenting the growing trend of content owners asserting control over how their data is used for model training...

Crawler Detect Files

CrawlerDetect is a PHP class for detecting bots/crawlers/spiders

Get an email when there's a new version of Crawler Detect

What's Changed

New Contributors