Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Components] scrapeninja #15137

Open
pipedream-component-development opened this issue Jan 2, 2025 · 1 comment · May be fixed by #15753
Open

[Components] scrapeninja #15137

pipedream-component-development opened this issue Jan 2, 2025 · 1 comment · May be fixed by #15753
Assignees
@luancazarine
Labels
triaged For maintainers: This issue has been triaged by a Pipedream employee

Comments

@pipedream-component-development
Copy link
Collaborator

pipedream-component-development commented Jan 2, 2025 •
edited by luancazarine
Loading

scrapeninja

URLs

Actions

non-js-scraping

Prompt

Use high-performance web scraping endpoint with Chrome browser TLS fingerprint, but without JavaScript execution and real browser overhead. Required prop is 'url'. Optional props are: 'headers', 'retryNum', 'geo', 'proxy', 'followRedirects', 'timeout', 'textNotExpected', 'statusNotExpected' and 'extractor'.

URLs

scraping-with-js-rendering

Prompt

Launches real Chrome browser engine, use only when 'Non JS Scraping' action features are not enough. Required prop is 'url'. Optional props are: 'waitForSelector', 'postWaitTime', 'dumpIframe', 'waitForSelectorIframe', 'extractorTargetIframe', 'headers', 'retryNum', 'geo', 'proxy', 'timeout', 'textNotExpected', 'statusNotExpected', 'blockImages', 'blockMedia', 'screenshot', 'catchAjaxHeadersUrlMask', 'viewportWitdh', 'viewportHeight', 'viewportDeviceScaleFactor', 'viewportHasTouch', 'viewportIsMobile', 'viewportIsLandscape' and 'extractor'.

URLs
@vunguyenhung vunguyenhung added the triaged For maintainers: This issue has been triaged by a Pipedream employee label Jan 6, 2025
@luancazarine luancazarine self-assigned this Feb 26, 2025
@luancazarine luancazarine linked a pull request Feb 26, 2025 that will close this issue
Open
luancazarine added a commit that referenced this issue Feb 27, 2025
@luancazarine
[Components] scrapeninja #15137
3f31cb3 
Actions
 - Non JS Scraping
 - Scraping With JS Rendering
@luancazarine luancazarine moved this from Doing to Ready for PR Review in Component (Source and Action) Backlog Feb 27, 2025
@GTFalcao GTFalcao moved this from Ready for PR Review to Ready for QA in Component (Source and Action) Backlog Feb 27, 2025
@vunguyenhung vunguyenhung moved this from Ready for QA to In QA in Component (Source and Action) Backlog Feb 28, 2025
@vunguyenhung vunguyenhung moved this from In QA to Ready for Release in Component (Source and Action) Backlog Feb 28, 2025
@vunguyenhung
Copy link
Collaborator

Hi everyone, all test cases are passed! Ready for release!

Test report
https://vunguyenhung.notion.site/Components-scrapeninja-1a7bf548bb5e817a9fe2fce7a16f7ac1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@luancazarine luancazarine

Labels
triaged For maintainers: This issue has been triaged by a Pipedream employee
Projects
Component (Source and Action) Backlog
Status: Ready for Release
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

New Components - scrapeninja PipedreamHQ/pipedream
3 participants
@luancazarine @vunguyenhung @pipedream-component-development