Skip to content

Latest commit

 

History

History
44 lines (31 loc) · 1000 Bytes

README.md

File metadata and controls

44 lines (31 loc) · 1000 Bytes
Raw

Node.js Web Crawler Script

Description

This is a simple Node.js-based web crawler that extracts data(link) from websites. It can be customized to scrape specific content, such as text, images, or links, for various use cases like data collection or analysis.

Features

  • Crawl web pages
  • Extract specific content (e.g. links)

Installation

  1. Clone the repository: 
    git clone https://github.com/josephDev123/Nodejs-web-crawler-script.git
  2. Navigate to the project directory: 
    cd Nodejs-web-crawler-script
  3. Install dependencies: 
    npm install

Usage

  1. Modify the target URL and scraping logic in the src/index.ts file.
  2. Run the script: 
    npm run start baseUrl url
note("the baseUrl and url must be the same")

Configuration

  • Customize the crawler by editing the src/index.ts to target different websites or specific elements.

License

This project is licensed under the ----.