Life After Proxy
This article assumes you know the basics of Scrapy, so we will focus on how to scrape Instagram at scale without getting blocked. Scraper API so it can retrieve the HTML response. For this example spider, I will use Nike and Adidas as the two Instagram accounts I want to scrape. Scrapy is very easy to install and run. It eliminates the worry of blocking or having to design XPath…
Read more