Search Results for "web crawler delphi"
Sort By:
Showing 445 open source projects for "web crawler delphi"
View related business solutions-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
Go From Idea to Deployed AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
1
Spatie Crawler
An easy to use, powerful crawler implemented in PHP
Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites. -
2
EasySpider
A visual no-code/code-free web crawler/spider
A visual code-free/no-code web crawler/spider, supporting both Chinese and English. -
3
WebMagic
A scalable web crawler framework for Java
WebMagic is a scalable crawler framework. It covers the whole lifecycle of crawler, downloading, url management, content extraction and persistent. It can simplify the development of a specific crawler. WebMagic is a simple but scalable crawler framework. You can develop a crawler easily based on it. WebMagic has a simple core with high flexibility, a simple API for html extracting. It also provides annotation with POJO to customize a crawler, and no configuration is needed. Some other... -
4
Heritrix
Internet Archive's open-source, web-scale, web crawler project
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt. -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
5
crwlr
Library for Rapid (Web) Crawler and Scraper Development
...Before diving into the library, let's have a look at the terms crawling and scraping. For most real-world use cases, those two things go hand in hand, which is why this library helps with and combines both. A (web) crawler is a program that (down)loads documents and follows the links in it to load them as well. A crawler could just load actually all links it is finding (and is allowed to load according to the robots.txt file), then it would just load the whole internet (if the URL(s) it starts with are no dead end). Or it can be restricted to load only links matching certain criteria (on same domain/host, URL path starts with "/foo",...) or only to a certain depth. ... -
6
Horse
Fast, opinionated, minimalist web framework for Delphi
Horse is an Express-inspired web framework for Delphi and Lazarus. Designed to ease things up for fast development in a minimalist way and with high performance. Fast, opinionated, minimalist web framework for Delphi. Horse works with Delphi 11 Alexandria, Delphi 10.4 Sydney, Delphi 10.3 Rio, Delphi 10.2 Tokyo, Delphi 10.1 Berlin, Delphi 10 Seattle, Delphi XE8 and Delphi XE7. -
7
Web Spider, Web Crawler, Email Extractor
Free Extracts Emails, Phones and custom text from Web using JAVA Regex
In Files there is WebCrawlerMySQL.jar which supports MySql Connection Free Web Spider & Crawler. Extracts Information from Web by parsing millions of pages. Store data into Derby Database and data are not being lost after force closing the spider. - Free Web Spider , Parser, Extractor, Crawler - Extraction of Emails , Phones and Custom Text from Web - Export to Excel File - Data Saved into Derby and MySQL Database - Written in Java Cross Platform Also See Free email Sender : https://sourceforge.net/projects/gitst-free-email-ender/ Please install Microsoft OpenJDK to start the application https://www.microsoft.com/openjdk -
8
Crawl4AI
Open-source LLM Friendly Web Crawler & Scraper
Crawl4AI is a high-performance, AI‑ready web crawler tailored for LLM data ingestion and RAG pipelines. It supports adaptive crawling heuristics (stopping when enough info is gathered), structured markdown output, and high-speed parallel execution. Designed to operate at scale with optional Docker deployment and framework integrations. -
9
Snap Lens Web Crawler
Crawl and download Snap Lenses from lens.snapchat.com with ease.
Crawl and download Snap Lenses from lens.snapchat.com with ease. This crawler is a dependency of Snap Camera Server https://snap-camera-server.sourceforge.io -
Catch Bugs Before Your Customers DoMove from alert to fix before users notice. AppSignal monitors errors, performance bottlenecks, host health, and uptime—all from one dashboard. Instant notifications on deployments, anomaly triggers for memory spikes or error surges, and seamless log management. Works out of the box with Rails, Django, Express, Phoenix, Next.js, and dozens more. Starts at $23/month with no hidden fees.
-
10
Delphi Web Utils contain the uJson . uJson unit contain the class: JSONObject, JSONArray and JSONTokenezer .This classes handle json structures.
-
11
miniblink49
Lighter, faster browser kernel of blink to integrate HTML UI in apps
...Headless mode, which greatly saves resources and is suitable for crawlers (headless mode, be suitable for Web Crawler). -
12
X-Crawl
Flexible Node.js AI-assisted crawler library
A high-performance web crawling and scraping framework for Node.js, designed for large-scale data extraction. -
13
Text Editors
Sempare Template (scripting) Engine for Delphi
Sempare Template (scripting) Engine for Delphi allows for flexible dynamic text generation. It can be used for generating email, HTML, reports, source code, xml, configuration, etc. -
14
crawley
The unix-way web crawler
Crawls web pages and prints any link it can find. Fast HTML SAX-parser (powered by golang.org/x/net/html) Small (below 1500 SLOC), idiomatic, 100% test-covered codebase. Grabs most of useful resources URLs (pics, videos, audios, forms, etc...) Found URLs are streamed to stdout and guaranteed to be unique (with fragments omitted) Scan depth (limited by starting host and path, by default - 0) can be configured. Can crawl rules and sitemaps from robots.txt. Brute mode - scan HTML comments for... -
15
ngx_waf
Handy, High performance, ModSecurity compatible Nginx firewall module
Handy, High-performance Nginx firewall module. Such as black and white list of IPs or IP range, uri black and white list, and request body black list, etc. Directives and rules are easy to write and readable. The IP detection is a constant-time operation. Most of the remaining inspections use caching to improve performance. Compatible with ModSecurity's rules, you can use OWASP ModSecurity Core Rule Set. Supports verifying Google, Bing, Baidu and Yandex crawlers and allowing them... -
16
Python-Spider
Python3 web crawler practice
Python-Spider is a repository intended to teach or provide examples for writing web spiders / crawlers in Python — part of a broader learning and resource collection by its author. The code and documentation are oriented toward beginners or intermediate learners who want to learn how to fetch, parse, and extract data from websites programmatically. As part of the author’s public learning-path repositories, python-spider likely includes examples of HTTP requests, HTML parsing, maybe... -
17
Roach
The complete web scraping toolkit for PHP
Roach is a complete web scraping toolkit for PHP. It is a shameless clone heavily inspired by the popular Scrapy package for Python. Roach allows us to define spiders that crawl and scrape web documents. But wait, there’s more. Roach isn’t just a simple crawler, but includes an entire pipeline to clean, persist and otherwise process extracted data as well. -
18
SiteOne Crawler (desktop app)
A free, feature-rich web analyzer and exporter/cloner you will love!
A free in-depth website analyzer providing audits of security, performance, SEO, accessibility and other technical aspects. Available as a desktop application for Windows/macOS/Linux and as a CLI tool for advanced users and CI/CD processes. It also includes an offline web page exporter (website clone, mirror). -
19
-
20
Ascoos Web Server
Is a web server for all Web Developers and Web Designers
For PHP 5.6 - 8.4.X see: Ascoos Web Extended Studio (AWES) is here : https://sourceforge.net/projects/ascoos-web-extended-studio/ ASCOOS Web Server is a rich package designed as a versatile web server for development purposes. It incorporates third-party components such as PHP, MySQL, pgSQL, MongoDB and FileZilla and stands out through a compact setup and a well-built administrative panel. ASCOOS Web Server allows you to work with multiple versions of PHP and MySQL without having to... -
21
ahCrawler
A PHP search engine for your website and web analytics tool. GNU GPL3
ahCrawler is a set to implement your own search on your website and an analyzer for your web content. It can be used on a shared hosting. It consists of * crawler (spider) and indexer * search for your website(s) * search statistics * website analyzer (http header, short titles and keywords, linkchecker, ...) You need to install it on your own server. So all crawled data stay in your environment. You never know when an external webspider updated your content. ... -
22
Ascoos Web Extended Studio
Is a portable web server suite for windows 64Bit, for Web Development.
Ascoos Web Extended Studio (AWES) is a portable, free 64-bit web server environment for Windows, designed for professional web developers and designers who need flexibility, modularity, and multi-version testing capabilities. It provides a complete local development stack based on technologies such as Apache, PHP, Node.js, Python, MariaDB, MongoDB, FileZilla, and other essential tools. 🔧 Key Features: - Multi-version support for PHP and MariaDB - Modular and upgrade-friendly... -
23
CaIS System
50
M Technology developer FREE Software Library for Linux/Windows Programing languages: - M(umps) - Gambas (Linux Gamed BASIC) - Lazarus/Delphi (Windows) -
24
BA_PY
BA_PY: Optimize Your Workflow with Python!
mbapy is a Python package that includes a collection of useful Python scripts as sub-modules, and it's goal is Basic for All in Python. mbapy primarily focus on data works, including data-retrieval, data-management, data-visualization, data-analysis and data-computation. It is built for both python-users and command-line-users. -
25
