JSpider

A Java implementation of a flexible and extensible web spider engine.
Optional modules allow functionality to be added (searching dead links, testing the performance and scalability of a site, creating a sitemap, etc ..

Project Activity

See All Activity >

License

GNU Library or Lesser General Public License version 2.0 (LGPLv2)

Follow JSpider

JSpider Web Site

Other Useful Business Software

Orchestrate Your AI Agents with Zenflow

The multi-agent workflow engine for modern teams. Zenflow executes coding, testing, and verification with deep repo awareness

Zenflow orchestrates AI agents like a real engineering system. With parallel execution, spec-driven workflows, and deep multi-repo understanding, agents plan, implement, test, and verify end-to-end. Upgrade to AI workflows that work the way your team does.

Try free now

Rate This Project

User Ratings

3.0 out of 5 stars

★★★★★

★★★★

★★★

★★

★

ease 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

features 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

design 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

support 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

User Reviews

Filter Reviews:

All

kidanemariam Posted 2012-05-02

Hello, I am a MSc student in Addis Ababa University department of computer science, Ethiopia. I am working my MSc thesis on language specific search engine. So, I am using JSpider as a crawling tool. Since I am new for the tool, what configurations should I do on the tool so that it can be used for crawling pages from the web? Regards HB
yodaj Posted 2011-07-11

good job
mdtareque Posted 2010-06-16

No progress on this, I suppose... Crawler4j(2008) and Nutch(2009) are the on going ones and have stable releases also ...
kcleong Posted 2009-11-09

Seems outdated. The linkchecker doesn't work; results in exceptions.

Additional Project Details

Languages

English

Intended Audience

Developers, Information Technology, Science/Research, System Administrators, Telecommunications Industry

Programming Language

Java

Related Categories

Java Search Engines, Java Link Checking Software, Java Web Scrapers

Registered

2002-10-25

Similar Business Software

Oxylabs

Oxylabs is a market leader in web intelligence with enterprise-grade, ethical, and compliant solutions. Its proxy infrastructure spans one of the largest global networks, offering residential, ISP, mobile, datacenter, and dedicated datacenter proxies, along with Web Unblocker – an AI-driven...

See Software
NetNut

Get ready to experience unmatched control and insights with our user-friendly dashboard tailored to your needs. Monitor and adjust your proxies with just a few clicks. Track your usage and performance with detailed statistics. Our team is devoted to providing customers with proxy solutions...

See Software
PYPROXY

Market-leading proxy solution provides tens of millions of IP resources. Commercial residential and ISP proxy network includes 90M+ IPs around the world. Exclusive high-performance server requests access to real residential addresses. Abundant bandwidth support business demands. Real-time speed...

See Software
Price2Spy

Price2Spy makes automatic price adjustments easy to perform saving your most valuable resource - time, allowing your pricing team to focus on strategic planning and management. Since 2010, we have provided pricing intelligence for retailers and brands in 40+ countries, helping them smoothly...

See Software
APISCRAPY

APISCRAPY is an AI-driven web scraping and automation platform converting any web data into ready-to-use data API. Other Data Solutions from AIMLEAP: AI-Labeler: AI-augmented annotation & labeling tool AI-Data-Hub: On-demand data for building AI products & services PRICE-SCRAPY:...

See Software
Perplexity

Where knowledge begins. Perplexity is an AI search engine that gives you quick answers. Available for free at as a web app, desktop app, or on the go on iPhone or Android. Perplexity AI is an advanced search and question-answering tool that leverages large language models to provide...

See Software