The Crolor library provides an extensible framework for cralwing websites and completing custom actions, extracting data, and generating reports. The primary use is to check for broken links. While many applications exist with this functionality, crolor provides a reusable process that may be triggered by automated jobs and integrated in to enterprise workflows. Future iterations aim to incorporate sessions to allow the crawling of sites that require login to access.
The Crolor project is currently in an experiental state, and not scheduled for release.
- pip installer
- requests 2.6.2
- beautifulsoup4 4.3.2
- openpyxl 2.2.2