Search results

Support <code>&lt;base&gt;</code> field for relative urls

feature

html5HTML5HTML5

needs info

Status: Open.

#94 In postmodern/spidr;

· zealot128 opened on Aug 18, 2025

· 

0.8.0

Switch to <code>Addressable::URI</code> for URI parsing

feature

Status: Open.

#86 In postmodern/spidr;

· postmodern opened on Dec 22, 2022

Add Logging

feature

Status: Open.

#72 In postmodern/spidr;

· postmodern opened on Oct 25, 2019

· 

0.8.0

Add low-level HTTP request methods

feature

Status: Open.

#62 In postmodern/spidr;

· postmodern opened on May 25, 2017

Add ignore_paths and ignore_paths_like

feature

Status: Open.

#61 In postmodern/spidr;

· postmodern opened on May 25, 2017

· 

0.8.0

Multithreading

feature

Status: Open.

#26 In postmodern/spidr;

· ethicalhack3r opened on Jun 6, 2011

Automatically detect and parse sitemap.xml

feature

Status: Open.

#19 In postmodern/spidr;

· postmodern opened on Oct 16, 2010

Store history queue in Hash of host:port and paths.

feature

Status: Open.

#18 In postmodern/spidr;

· postmodern opened on Oct 16, 2010

Redirect fails silently - need debug messages

feature

Status: Open.

#13 In postmodern/spidr;

· perplexes opened on Aug 19, 2010

handle css @import spidering

feature

Status: Open.

#8 In postmodern/spidr;

· zapnap opened on Mar 17, 2010