Introduction - If you have any usage issues, please Google them yourself
WebCollector is a JAVA crawler framework (kernel) that does not need to be configured and easy to develop for two times. It provides a streamlined API and implements a powerful crawler with a small amount of code. WebCollector-Hadoop is the WebCollector version of Hadoop, which supports distributed crawling.