首先要确定setting中ITEM_PIPELINES设置已添加:

# Configure item pipelines
# See https://doc.scrapy.org/en/latest/topics/item-pipeline.html
ITEM_PIPELINES = {
   'spider.spider.pipelines.SpiderPipeline': 300,
}

pipline生成文件中有如下提示:

# Don't forget to add your pipeline to the ITEM_PIPELINES setting

 

 

然后检查parse()函数中创建item后是否返回,即是否有yield item

即以下代码的spiderItem

        for item in employItems:
            spiderItem = SpiderItem()
            spiderItem['base'] = item.css('p::text').extract_first()[0:2]
            spiderItem['company'] = item.css('.company-text>.name>a::text').extract_first()
            spiderItem['position'] = item.css('.job-title::text').extract_first()
            spiderItem['minPayment'] = item.css('.red::text').extract_first()[0:2]
            spiderItem['maxPayment'] = item.css('.red::text').extract_first()[4:6]
            spiderItem['eduBg'] = item.css('p::text').extract()[-2:]
            spiderItem['updateTime'] = item.css('.info-publis>p::text').extract_first()[3:]
            yield spiderItem

 

Logo

北京人形旗下天工造物具身智能开源社区,聚焦具身天工与慧思开物两大平台

更多推荐