X-Crawl

A high-performance web crawling and scraping framework for Node.js, designed for large-scale data extraction.

Features

Supports concurrent and distributed web crawling
Open-source and optimized for large-scale scraping tasks
Provides built-in request throttling and rate limiting
Allows customizable scraping rules and data extraction patterns
Works with proxies for anonymous crawling
Supports headless browser automation with Puppeteer integration

Project Samples

Project Activity

See All Activity >

License

MIT License

Follow X-Crawl

X-Crawl Web Site

Other Useful Business Software

Our Free Plans just got better! | Auth0

With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.

You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.

Try free now

Rate This Project

User Reviews

Be the first to post a review of X-Crawl!

Additional Project Details

Programming Language

TypeScript

Related Categories

TypeScript Libraries

Registered

2025-02-14

Similar Business Software

DHTMLX

DHTMLX is a JavaScript UI library that provides a set of highly customizable and flexible components for building modern and responsive web applications. The library includes more than 30 UI components, such as Gantt, Scheduler, Kanban, diagrams, charts, grids, spreadsheets, calendars, trees,...

See Software
Webix

JavaScript UI library and framework for speeding up web development. JS Framework for cross-platform web Apps development 102 UI widgets and feature-rich CSS / HTML5 JavaScript controls. Save at least 3000+ development hours by using ready-made widgets and UI controls. Develop Web UI 30% faster....

See Software
Bryntum

Bryntum is a leading provider of high-performance scheduling solutions for the web. Our suite of JavaScript components—including Gantt, Scheduler, Task Board, and Calendar—enables developers to build modern project management applications with features like drag-and-drop scheduling, resource...

See Software
Kendo UI

Kendo UI is the ultimate collection of JavaScript UI components with libraries for jQuery, Angular, React, and Vue. Quickly build eye-catching, high-performance, responsive web applications—regardless of your JavaScript framework choice. Easily add advanced JavaScript components into your...

See Software
JsPlumb

JsPlumb is a Javascript diagramming library that allows you to rapidly create complex applications featuring visual connectivity without having to build any of the boring stuff: it provides pan/zoom, a minimap widget, automatic layouts, data binding, path finding, and much, much more. JsPlumb...

See Software
BullMQ

BullMQ is a Node.js library that implements a fast and robust queue system built on top of Redis, designed to address various challenges in modern microservices architectures. It is structured around four primary classes. BullMQ offers features like minimal CPU usage due to a polling-free...

See Software

Report inappropriate content

X-Crawl

Flexible Node.js AI-assisted crawler library

Get an email when there's a new version of X-Crawl

Features

Project Samples

Project Activity

Categories

License

Follow X-Crawl

User Reviews

Additional Project Details

Programming Language

Related Categories

Registered