PriceWise

Dive into web scraping with a Next.js 13 eCommerce price tracker integrated with cheerio, cron jobs, sending emails, and more.

View Demo

Table of Contents

• Table of Contents
• 🌟 Introduction
  • Folder Structure
    • .vscode
    • app
    • components
    • lib
    • public
    • types
• 👾 Technologies Used
• 🧰 Getting Started
  • 🔑 Prerequisites, Environment Variables
  • ⚙️ Installation and Run Locally
  • Learn More
  • Deploy on Vercel
• 📷 Screenshots
• 👋 Contributing
• ⚠️ License
• 🤝 Contact

🌟 Introduction

PriceWise application is a comprehensive solution for tracking product prices on Amazon. This project is designed to scrape product details from Amazon, store the data in a MongoDB database, and send email notifications to users when there are changes in the product details. The project includes a web application with a user-friendly interface that allows users to search for products, view product details, and subscribe to product updates. The application is built with Next.js and Tailwind CSS, and it includes several components such as a home page, product details page, navbar, search bar, product card, price info card, and modal.

The server-side logic is implemented with serverless functions that handle various tasks such as connecting to the database, scraping product details, updating product information, and sending email notifications. The project also includes several utility functions for extracting information from web pages and formatting numbers.

Folder Structure

PriceWise code folder structure is as follows:

nextjs13-pricewise/
├── .vscode/
├   └── settings.json
├── app/
├   ├── api/
├   ├   └── cron/
├   ├   ├   └── route.ts
├   ├── products/
├   ├   └── page.tsx
├   ├── favicon.ico
├   ├── globals.css
├   ├── layout.tsx
├   └── page.tsx
├── components/
├   ├── HeroCarousel.tsx
├   ├── Modal.tsx
├   ├── Navbar.tsx
├   ├── PriceInfoCard.tsx
├   ├── ProductCard.tsx
├   └── Searchbar.tsx
├── lib/
├   ├── actions/
├   ├   └── index.ts
├   ├── models/
├   ├   └── Product.model.ts
├   ├── nodemailer/
├   ├   └── index.ts
├   ├── scraper/
├   ├   └── index.ts
├   └── mongoose.ts
├   └── utils.ts
├── public/
├   └── assets/
├   ├   ├── icons/
├   ├   ├   └── [[...]].svg
├   ├   └── images/
├   ├       └── [[...]].{png,svg}
├   ├── next.svg
├   └── vercel.svg
├── types/
├   └── index.d.ts
├── .eslintrc.json
├── next.config.js
├── package.json
├── postcss.config.js
├── tailwind.config.js
└── tsconfig.json

Now let's dive into the root folder and see what it contains.

.vscode

settings.json

This is a JSON file contains the settings used in Visual Studio Code.

app

favicon.ico - globals.css - layout.tsx - page.tsx - api/ - products/

The app directory contains the application's favicon, the global styles, the layout component and the page component which is the home page of the application, the api directory which contains the serverless functions, and the products directory which contains the product details page.

components

HeroCarousel.tsx - Modal.tsx - Navbar.tsx - PriceInfoCard.tsx - ProductCard.tsx - Searchbar.tsx

The components directory contains all the components used in the application.

lib

actions/ - models/ - nodemailer/ - scraper/ - mongoose.ts - utils.ts

The lib directory contains all the server-side logic used in the application. The actions directory contains all the functions used in the serverless functions, the models directory contains the Product model, the nodemailer directory contains the functions used to send emails, the scraper directory contains the functions used to scrape product details, the mongoose.ts file contains the database connection, and the utils.ts file contains the utility functions.

public

icons/ - images/ - next.svg - vercel.svg

The public directory contains the media used in the application. The assets folder contains all the images and icons used in the application.

types

index.d.ts

This is a TypeScript code contains all the types used in the application.

(back to top)

👾 Technologies Used

PriceWise web application is built using the following technologies:

• TypeScript: A statically typed superset of JavaScript. It is used for writing the code.
• Next.js: A React framework for building web applications. It is used for both the frontend and the backend of the application.
• Tailwind CSS: A utility-first CSS framework for rapidly building custom designs. It is used for styling the application.
• Mongoose: An Object Data Modeling (ODM) library for MongoDB and Node.js. It is used for defining the product schema and interacting with the MongoDB database.
• Nodemailer: A module for Node.js applications to allow easy email sending. It is used for sending email notifications to users.
• Axios: A promise-based HTTP client for the browser and Node.js. It is used for making HTTP requests to scrape product details from Amazon.
• Cheerio: A fast, flexible, and lean implementation of core jQuery designed specifically for the server. It is used for parsing the HTML response from the Amazon product page.
• React Responsive Carousel: A lightweight carousel component for React. It is used for displaying a carousel of images on the home page.
• Google Fonts: A library of free licensed font families. It is used for defining the font styles in the application.
• ESLint: ESLint is a static code analysis tool for identifying problematic patterns found in JavaScript code.
• Vercel: Vercel is a cloud platform for frontend developers, providing the frameworks, workflows, and infrastructure to build a faster, more personalized Web.

(back to top)

🧰 Getting Started

🔑 Prerequisites, Environment Variables

In order to run the project, you need to create a MongoDB database and get the connection URI. You can follow the instructions here to create a MongoDB database and get the connection URI, or you can use a MongoDB service such as MongoDB Atlas. then, you need to add the connection URI to the .env file in the root of your project:

MONGODB_URI=your_mongodb_uri

Also, you need to create a Bright Data account and establish Web Unlocker proxy

Automated website unlocking system that leverages the residential network and includes CAPTCHA solving, automatic retries, and fingerprint management.

then, you need to get the proxy username and password and add them to the .env file in the root of your project:

PROXY_USERNAME=your_proxy_username
PROXY_PASSWORD=your_proxy_password

⚙️ Installation and Run Locally

Follow these steps to install and run the project:

1. Clone the repository

   Open your terminal and run the following command to clone the repository:

   git clone https://github.com/ladunjexa/nextjs13-pricewise.git

2. Navigate to the project directory

   cd nextjs13-pricewise

3. Install Node.js

   The project requires Node.js to run. If you don't have it installed, you can download it from here.

4. Install the required packages

   The project requires several packages to be installed. Run the following command to install them:

   npm i

5. Start the server

   Run the following command to start the server:

   npm run start

Now, you should be able to access the project at http://localhost:3000.

Please note that the project requires the "m.media-amazon.com" domain to be accessible for image handling. If you are unable to access this domain, you may encounter issues with image loading.

This is a Next.js project bootstrapped with create-next-app.

Learn More

To learn more about Next.js, take a look at the following resources:

• Next.js Documentation - learn about Next.js features and API.
• Learn Next.js - an interactive Next.js tutorial.

You can check out the Next.js GitHub repository - your feedback and contributions are welcome!

Deploy on Vercel

The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.

Check out our Next.js deployment documentation for more details.

(back to top)

📷 Screenshots

(back to top)

👋 Contributing

Contributions are always welcome!

See contributing.md for ways to get started.

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

(back to top)

⚠️ License

PriceWise web application is open source and distributed under the MIT License.

(back to top)

🤝 Contact

If you want to contact me, you can reach me at @ladunjexa.

(back to top)