Skip to content

HyperionGray/starbelly-crawlme

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 

Repository files navigation

Starbelly: Crawl Me

This repo contains a dummy website for testing a web crawler such as Starbelly. The dummy site is hosted by nginx and it contains a small number of pages and images.

You can run this inside a docker container as follows:

$ docker run \
    --rm --name crawlme -p 127.0.0.1:10080:80 \
    -v $PWD/www:/usr/share/nginx/html nginx:latest

To crawl this site with Starbelly, use the URL http://localhost:10800.

About

A small website for web crawler testing

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages