Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

如何将自己编写的LoginHandler或者PageEvent集成到standaloneApplication程序里? #18

Open
ZhujingJava opened this issue Mar 18, 2024 · 2 comments

Comments

@ZhujingJava
Copy link

目前可以单独main方法跑
但是在springboot中找不到配置注册handler或者配置option的地方,有扩展点吗?

@annian101
Copy link

目前可以单独main方法跑 但是在springboot中找不到配置注册handler或者配置option的地方,有扩展点吗?

老哥,我有一些想法,可以加微信交流吗?

@platonai
Copy link
Owner

platonai commented Apr 5, 2024

The best practice is to create a hyperlink for each task and attach event handlers:

val options = session.options(args)
val event = options.event.browseEventHandlers // val event = options.event.browseEvent in older versions
event.onBrowserLaunched.addLast { page, driver ->
    // Warm up the browser to avoid being blocked by the website,
    // or choose global settings, such as your location.
    warmUpBrowser(page, driver)
}
event.onWillFetch.addLast { page, driver ->
    // Have to visit a referrer page before we can visit the desired page.
    waitForReferrer(page, driver)
    // Websites may prevent us from opening too many pages at a time, so we should open links one by one.
    waitForPreviousPage(page, driver)
}
event.onWillCheckDocumentState.addLast { page, driver ->
    // Wait for special fields to appear on the page.
    driver.waitForSelector("body h1[itemprop=name]")
    // Close the mask layer, which might be promotions, ads, or something else.
    driver.click(".mask-layer-close-button")
}
// Visit the URL and trigger events.
session.load(url, options)

Code: WalmartCrawler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants