courlan-0.9.0

• hardening of filters and URL parses (#14)
• normalize punicode to unicode
• methods added to UrlStore: get_crawl_delay(), print_unvisited_urls()
• UrlStore now triggers exit code 1 when interrupted
• argument added to extract_links(): no_filter
• code refactoring: simplifications

Full Changelog: v0.8.3...v0.9.0

courlan-0.8.3

• fixed bug in domain name extraction
• uniform logging parameters

Full Changelog: v0.8.2...v0.8.3

courlan-0.8.2

• full type hinting
• maintenance: code linted

Full Changelog: v0.8.1...v0.8.2

courlan-0.8.1

• add type annotations and check with mypy
• url_filter() function moved from Trafilatura
• code style: use black

courlan-0.8.0

• performance optimizations
• fast track for domain extraction (extract_domain(url, fast=True)), now taking subdomains into account

Full Changelog: v0.7.2...v0.8.0

courlan-0.7.2

• UrlStore: threading lock and convenience functions added

courlan-0.7.1

• bug in sampling fixed
• UrlStore: validation by default

Full Changelog: v0.7.0...v0.7.1

courlan-0.7.0

• UrlStore class added: data store containing URLs with relevant information
• code cleaning and maintenance (bugs, simplification)

Full Changelog: v0.6.0...v0.7.0

courlan-0.6.0

• reviewed code base: simplicity and execution speed
• dropped support for Python 3.5

courlan-0.5.0

• more complex language heuristics, use langcodes
• extended blacklists and whitelists
• more precise filters and more efficient code
• support for Python 3.10