Skip to content

Latest commit

 

History

History
174 lines (145 loc) · 4.06 KB

README.md

File metadata and controls

174 lines (145 loc) · 4.06 KB
Raw

Social Crawler

This project utilizes Selenium, Helium, and HTTP GET requests to retrieve user information from Facebook, Instagram, TikTok, and Twitter. It is crucial to note that such actions are in violation of the terms of service of these platforms. Consequently, it is crucial to proceed with caution when deploying these scripts.

Additionally, the project incorporates a wrapper for the Sherlock Project.

Table of Contents

Installation

# clone the repo
$ git clone https://github.com/oscarmmv/social-crawler.git

# change the working directory to social crawler
$ cd social-crawler

Requirements

  • Project requires Google Chrome to be installed. (You can change to your perfered OS, please read WebDriver Documentation)
  • Supported Operating Systems:
    • win64
    • macx64
    • linuxx64
  driver_path = "path-to-drivers/chromedriver-<your-operating-system>/chromedriver"

Facebook Crawler

Import

  # Install the required dependencies
  # Import the scrape_facebook function into your Python script
  from facebook_scraper import scrape_facebook

Usage

  data = scrape_facebook(url)

Return Type The function returns a JSON object with the following structure:

data = {
  "Name": "user_name",
  "Status": "user_status",
  "Bio": "user_bio",
  "Followers": followers_count,
  "Likes": likes_count,
  "Email": "user_email",
  "Links": ["link_1", "link_2", ...]
}

Instagram Crawler

Import

  # Install the required dependencies
  # Import the scrape_instagram_user function into your Python script
  from instagram_scraper import scrape_instagram_user

Usage

  data = scrape_instagram_user(url)

Return Type The function returns a JSON object with the following structure:

data = {
  "Name": "user_name",
  "Followers": followers_count,
  "Following": following_count,
  "Posts": post_count
}

TikTok Crawler

Import
  # Install the required dependencies
  # Import the scrape_tiktok_profile function into your Python script
  from tiktok_scraper import scrape_tiktok_profile

Usage

  data = scrape_tiktok_profile(url)

Return Type The function returns a JSON object with the following structure:

data = {
  "Name": "name",
  "Username": "user_name",
  "Bio": "user_bio",
  "Link": link,
  "Followers": follower_count,
  "Following": following_count,
  "Like_Count": like_count
}

Twitter Crawler

Import

  # Install the required dependencies
  # Import the scrape_twitter_profile function into your Python script
  from twitter_scraper import scrape_twitter_profile

Usage

  # !!!Set username and password!!!
  data = scrape_twitter_profile(url)

Return Type The function returns a JSON object with the following structure:

data = {
  "Name": "name",
  "Bio": "user_bio",
  "Followers": follower_count,
  "Following": following_count,
  "Account Age": account_age,
  "Link": link,
}

Sherlock

This is a wrapper for the Sherlock Project

Usage

  # Install the required dependencies
  # run the tracker.py script
  python3 tracker.py

Input:"username" ---> returns all social media accounts associated to that username