README.md

CINERGIScrapy

About

This web scraper is based off of Scrapy. http://doc.scrapy.org/en/latest/intro/tutorial.html

First, metadataScraper will scrape the title and URL from an XML file. Then GoogleSearchScraper will google that title and collect any relevant information.

This project is still a work in progress.

How to install

Follow Scrapy's Installation Guide to install:

1. Python 2.7
2. pip and setuptools Python packages
3. lxml
4. OpenSSL
5. Scrapy