Stop relying on overpriced APIs for popular domains! 🤚🏻
This repository contains scraping samples for the most scraped domains on the web, with full-length tutorials and functioning scrapers mainly built with Python and NodeJS (NodeJS files included in separate folders).
Scrapers in this repository are programmed to help you understand the basics of scraping these popular websites and serving as a base for you to customize and use in your own projects.
For Python, you'll be using:
- Python 3.8+,
- Requests for handling HTTP requests,
- Beautiful Soup for parsing information,
- and urllib.parse for URL encoding.
This repository also uses Scrape.do's web scraping API, to bypass Web Application Firewalls (WAFs) of the domains you're scraping and also interacting with the browser.
| Domain | Status | Scraped Data | Tutorial | Last Updated |
|---|---|---|---|---|
| Amazon | 🟢 Working | • Product data from single pages • Product variants with different sizes • Search results and categories |
Scrape Product Data from Amazon | Aug 13, 2025 |
| eBay | 🟢 Working | • Basic product listings • Product reviews with pagination • Search results with multiple layouts • Product variants and hidden data |
Scrape Products, Reviews, and Variants from eBay | Aug 1, 2025 |
| Google Search | 🟢 Working | • First page organic results • All paginated organic results • Paid search advertisements • Frequently asked questions • Related search terms |
Scrape Organic Results, Ads, and FAQs from Google | Jun 23, 2025 |
| 🟢 Working | • Profile information and bio • Top 12 recent posts • Individual post details • Post comments and replies |
Scrape Profiles, Posts, and Comments from Instagram | May 30, 2025 | |
| G2 | 🟢 Working | • Company and product data • Product reviews • Search results and categories |
Scrape Company Data and Reviews from G2 | Aug 5, 2025 |
| DoorDash | 🟢 Working | • Consumer address registration • Restaurant and store listings • Category-specific products • Full menu catalogs |
Scrape Restaurants, Products, and Menus from DoorDash | Jul 17, 2025 |
| Best Buy | 🟢 Working | • Product listings with Selenium • Product data with cloud rendering |
Scrape Products from Best Buy | Jul 9, 2025 |
| Uber Eats | 🟢 Working | • Store listings via API • Store listings via frontend • Restaurant menus • Chain store products |
Scrape Stores, Menus, and Products from Uber Eats | Aug 1, 2025 |
| Zillow | 🟢 Working | • Property listing details • Pricing and market data |
Scrape Property Data from Zillow | Aug 5, 2025 |
| Naver | 🟢 Working | • E-commerce product pages • Product data via API • Paid search advertisements • Organic search results • Image search results |
Scrape Products and Search Results from Naver | Jul 10, 2025 |
| Chewy | 🟢 Working | • Pet product information • Names and pricing |
Scrape Pet Products from Chewy | Aug 5, 2025 |
| Mouser | 🟢 Working | • Electronic component data • Product names and pricing |
Scrape Electronic Components from Mouser | Aug 5, 2025 |
| Fnac | 🟢 Working | • French retail product data • Names and pricing |
Scrape Products from Fnac | Aug 5, 2025 |
| Ubersuggest | 🟢 Working | • Bearer token generation • Keyword overview data • SERP results • Automated keyword analysis |
Scrape Keyword Data from Ubersuggest | Jun 23, 2025 |
| Zomato | 🟢 Working | • Restaurant information • Delivery menu data |
Scrape Restaurant Data from Zomato | Jul 2, 2025 |
| Allegro | 🟢 Working | • Polish e-commerce products • Names, pricing, and ratings |
Scrape Products from Allegro | Aug 5, 2025 |
| AUTODOC | 🟢 Working | • Automotive parts data • Product details and pricing |
Scrape Auto Parts from AUTODOC | Aug 5, 2025 |
| AutoScout24 | 🟢 Working | • Car listing information • Vehicle names and pricing |
Scrape Car Listings from AutoScout24 | Aug 5, 2025 |
| Idealista | 🟢 Working | • Spanish real estate listings • Property details, pricing, and location data • Search results with pagination |
Scrape Property Data from Idealista | Aug 6, 2025 |
| FastPeopleSearch | 🟢 Working | • Person information lookup • Names, ages, and addresses |
Scrape People Data from FastPeopleSearch | Aug 5, 2025 |
| HungerStation | 🟢 Working | • Restaurant listings • Store menu items |
Scrape Restaurants and Menus from HungerStation | Jul 17, 2025 |
| Imovelweb | 🟢 Working | • Brazilian real estate data • Property details and pricing |
Scrape Properties from Imovelweb | Aug 5, 2025 |
| Klium | 🟢 Working | • Tool and equipment data • Product names, pricing, and stock |
Scrape Tools from Klium | Aug 5, 2025 |
| MSC Direct | 🟢 Working | • Industrial supply products • Brand, names, and pricing |
Scrape Industrial Supplies from MSC Direct | Aug 5, 2025 |
| TicketMaster | 🟢 Working | • Event details and schedules • Dates, venues, and locations |
Scrape Events from TicketMaster | Aug 5, 2025 |
| TruePeopleSearch | 🟢 Working | • Person information lookup • Names, addresses, and contacts |
Scrape People Data from TruePeopleSearch | Aug 5, 2025 |
| Zoro | 🟢 Working | • Industrial business supplies • Product names and manufacturer data |
Scrape Business Supplies from Zoro | Aug 5, 2025 |
