Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This applies techniques from https://github.com/VeNoMouS/cloudscraper to more closely mimic requests from a browser, so there is a better chance of not being detected as a bot to have to deal with a captcha in the first place. In my testing, these changes created identical requests to cloudscraper, allowing me to access pages that were previously protected with a captcha.
Summary of changes:
For my purposes, I needed to be able to scrape https://bouqs.com/, but without these changes, I was blocked by a challenge. After these changes, requests go through with no challenge. I'm not terribly familiar with the Cloudflare firewall, but I know they compute a "Bot Threat Score" and I think these changes help to lower it enough that, at least in my case, it allows my requests through.