Goal of the project was to extract information for damaged cars from a website that did not have proper API support.
Project Details
Scraper was receiving input urls from a BigQuery table and pushing into several other ones. So I have created appropriate pipeline files with Scrapy. Eventually, Scrapy bot was deployed on cloud to run 24/7.