Skip to content
This repository was archived by the owner on Jul 8, 2018. It is now read-only.

Project Roadmap #1

Open
8 of 12 tasks
liuzl opened this issue Jun 8, 2017 · 0 comments
Open
8 of 12 tasks

Project Roadmap #1

liuzl opened this issue Jun 8, 2017 · 0 comments

Comments

@liuzl
Copy link
Contributor

liuzl commented Jun 8, 2017 •
edited
Loading

  • Single Machine Demo
  • Downloader
    • http downloader
  • Parser
    • html parser, using xpath
    • json parser, mainly for restful api crawling
    • content parser, main content (title/content/time) extraction, mainly for news articles
  • Crawler
  • Controller
  • Crawler Conf Editor, web based json editor
  • Task Priority (Crawler level)
  • Dashboard
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@liuzl