Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requests blocked due to security increase #64

Open
xavierartot opened this issue Jun 6, 2020 · 9 comments
Open

Requests blocked due to security increase #64

xavierartot opened this issue Jun 6, 2020 · 9 comments

Comments

@xavierartot
Copy link

This is still possible to use your library to read the data’s?

@tdurieux
Copy link
Owner

tdurieux commented Jun 7, 2020

It depends of your "Production" and how many requests you plan to do.

@tagazok
Copy link

tagazok commented Aug 31, 2020

Hi,

What do you mean by "Production" ?
I tried to use the api with the example in the readme and run it locally on my computer, I get a 403 error :/
Any way I can make this work ?

@tdurieux
Copy link
Owner

tdurieux commented Sep 1, 2020

I updated the user-agent used by the lib (v0.0.43), you should not get a 403 error now.

@nFonsat
Copy link

nFonsat commented Sep 2, 2020

Hello, I updated this package to 0.0.45 but I get a 403 error. Please help me.

[EDIT]
I just tried it with a vpn and it works. I think "datadome" or "leboncoin" blocked my IP.

@tdurieux
Copy link
Owner

tdurieux commented Sep 2, 2020

They drastically increase security yesterday. I think it is the end of the lib :(
The first request is generally not blocked but the second one is.

@nFonsat
Copy link

nFonsat commented Sep 2, 2020

I think so too :(. On their website, since yesterday, it is mandatory to validate a "captcha".
Can we retrieve in another way their content as other very well known sites do?

@tdurieux
Copy link
Owner

tdurieux commented Sep 2, 2020

i am not aware of other ways

@tdurieux tdurieux changed the title Production Requests blocked due to security increase Sep 2, 2020
@tdurieux tdurieux pinned this issue Sep 2, 2020
@ronycohen
Copy link

They drastically increase security yesterday. I think it is the end of the lib :(
The first request is generally not blocked but the second one is.

That was a great Journey :)

@victorbartel
Copy link

What do you think about to leverage the community strength and crawl their site with micro, silenum based scrapers from many spots(every community member could launch one for 30 min a day) in order to grab, package and expose the data through a clean API?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants