Description: Analog HTTP request: HttpClient 4.0
Target page structure analysis, HTTP request header information analysis: Firefox+ firebug/Chrome (F12 developer mode)
HTML parsing: Jsoup
To Search:
File list (Check if you may need any files):
YukiSpider\.classpath
..........\.project
..........\.settings\org.eclipse.core.resources.prefs
..........\.........\org.eclipse.jdt.core.prefs
..........\conf\log4j.properties
..........\....\spider.properties
..........\lib\apache-mime4j-0.6.jar
..........\...\commons-codec-1.4.jar
..........\...\commons-logging-1.1.1.jar
..........\...\httpclient-4.0.1.jar
..........\...\httpcore-4.0.1.jar
..........\...\httpmime-4.0.1.jar
..........\...\json.jar
..........\...\jsoup-1.5.2.jar
..........\...\log4j-1.2.17.jar
..........\...\mysql-connector-java-5.1.10-bin.jar
..........\src\cn\zju\yuki\spider\fetcher\PageFetcher.java
..........\...\..\...\....\......\handler\ContentHandler.java
..........\...\..\...\....\......\model\FetchedPage.java
..........\...\..\...\....\......\.....\SpiderParams.java
..........\...\..\...\....\......\parser\ContentParser.java
..........\...\..\...\....\......\queue\UrlQueue.java
..........\...\..\...\....\......\.....\VisitedUrlQueue.java
..........\...\..\...\....\......\SpiderStarter.java
..........\...\..\...\....\......\storage\DataStorage.java
..........\...\..\...\....\......\worker\SpiderWorker.java
..........\...\..\...\....\......\fetcher
..........\...\..\...\....\......\handler
..........\...\..\...\....\......\model
..........\...\..\...\....\......\parser
..........\...\..\...\....\......\queue
..........\...\..\...\....\......\storage
..........\...\..\...\....\......\worker
..........\...\..\...\....\spider
..........\...\..\...\yuki
..........\...\..\zju
..........\...\cn
..........\.settings
..........\bin
..........\conf
..........\lib
..........\src
YukiSpider