本文共 950 字，大约阅读时间需要 3 分钟。

scrapy常用指令

创建一个Scrapy工程

1.命令行输入该指令

import scrapyclass DemoSpider(scrapy.Spider):    name = "demo"    start_urls = ['http://python123.io/ws/demo.html']    def parse(self, response):        fname = response.url.split('/')[-1]        with open(fname, 'wb') as f:            f.write(response.body)        self.log('Save file %s.' % fname)

4.开始执行scrap crawl demo

优化

demo.py

import scrapyclass DemoSpider(scrapy.Spider):    name = "demo"    # 优化区别 start_urls = ['http://python123.io/ws/demo.html']    def start_requests(self): #新增        urls = ['http://python123.io/ws/demo.html']        for url in urls:            yield scrapy.Request(url=url, callback=self.parse)  #可以暂时挂起，下次进入    def parse(self, response):        fname = response.url.split('/')[-1]        with open(fname, 'wb') as f:            f.write(response.body)        self.log('Save file %s.' % fname)

转载地址：http://uyhpn.baihongyu.com/