What's the best way to update session cookies on a scraper running on server after expiry?

[Aditya94A]

For me, new free accounts seem to get blocked after 50 profiles/day, sessions seem to randomly expire.

When the session cookie expires, as per the readme, we're supposed to use a new one. But we can't really keep doing this manual step every time when scraping at scale (think millions of accounts), so I'm wondering if there's a way to automate this.

Any ideas?

[jvandenaardweg]

Hmm that seems to be a major limitation then.

Might try LinkedIn's commercial options then? https://www.linkedin.com/help/linkedin/answer/52950

In other words: premium (paid) accounts seem to have higher limits.

For millions of accounts I can imagine you'll have to find other ways to do this. Seems logical LinkedIn tries to prevent that high volume of scraping.