Learning Scrapy

Book Overview

Learning Scrapy is built on Scrapy 1.0 and systematically covers both the fundamentals of the Scrapy framework and its advanced use. Topics include Scrapy basics, HTML and XPath, data pipelines, deploying to Scrapinghub, configuration management, and distributed crawling.

Core Concepts

Based on my study notes, the core topics of this book include:

1. HTML/XPath
2. Crawling Basics
3. From Scrapy to Mobile Apps
4. Quick Scrapy Tips
5. Deploying to Scrapinghub
6. Configuration and Management
7. Scrapy Programming
8. Pipeline Recipes
9. Understanding Scrapy's Performance
10. Distributed Crawling with Scrapyd and Real-Time Analytics

Key Takeaways

I read this book at the end of July and took detailed chapter-by-chapter notes. It fits squarely into my data science goals — data collection is the first step of data science. Through this book, I gained the ability to pull data from the web efficiently and at scale with the Scrapy framework.