I am now quite some time to crawl the Web. so I have a lot of sites in my database(~294,181). if the program checks for duplicate pages then it crashes. I use windows 7 64-bit.
You seem to have CSS turned off. Please don't fill out this field.
Log in to post a comment.
View and moderate all "bugs Discussion" comments posted by this user
Mark all as spam, and block user from posting to "Bugs"