Skip to content
/ web-crawler-worker-node-js Public

Worker part of the Web Crawler application. The Web Crawler app takes an input from the user such as a link maximum number of pages and depth. At the end it shows in real time a tree of all the links and pages that the crawler found in the provided URL.

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

almoggutin/web-crawler-worker-node-js

BranchesTags

Repository files navigation

Web Crawler Worker Side

Worker side of the Web Crawler Application

< Technologies /> 💻

Frontend Technologies:

  1. HTML5
  2. CSS3
  3. Javascript
  4. Handlebars.js

Backend Technologies:

  1. Node.js
  2. Express
  3. Cors
  4. AWS
  5. Redis
  6. Jest
  7. Node html parser

Scaling Services:

  1. Dcoker
  2. Kubernetes

< Repositories />

  1. Web Crawler Full App
  2. Client Side
  3. Server Side

About

Worker part of the Web Crawler application. The Web Crawler app takes an input from the user such as a link maximum number of pages and depth. At the end it shows in real time a tree of all the links and pages that the crawler found in the provided URL.

Topics

nodejs javascript docker kubernetes redis aws parser express jest rest-api web-crawler web-worker web-scraper backend-api

Resources

Readme
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages