Repositories list

• crawlab

  Public
  Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

  godockerplatform+ 10crawlerspiderweb-crawlerscrapywebcrawlerscrapyd-uiwebspider+ 3

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •1.8k•12k•116•1•Updated Feb 27, 2025Feb 27, 2025

• crawlab-python-sdk

  Public
  Python SDK for Crawlab

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1•1•0•0•Updated Feb 14, 2025Feb 14, 2025

• crawlab-node-sdk

  Public
  Node.js SDK for Crawlab

  TypeScript

  •

  BSD 3-Clause "New" or "Revised" License

  •0•0•0•0•Updated Jan 6, 2025Jan 6, 2025

• crawlab-go-sdk

  Public
  Crawlab Go SDK

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •9•8•0•0•Updated Jan 6, 2025Jan 6, 2025

• crawlab-java-sdk

  Public
  Java SDK for Crawlab

  Java

  •

  BSD 3-Clause "New" or "Revised" License

  •0•0•0•0•Updated Jan 3, 2025Jan 3, 2025

• e2e-tests

  Public
  TypeScript

  •0•1•0•0•Updated Dec 20, 2024Dec 20, 2024

• crawlab-test-report

  Public
  0•0•0•0•Updated Dec 19, 2024Dec 19, 2024

• crawlab-docs

  Public
  Documentation for Crawlab

  documentationgitbook-clicrawlab

  JavaScript

  •55•39•1•3•Updated Mar 1, 2025Mar 1, 2025

• crawlab-ai-sdk

  Public
  SDK for Crawlab AI

  Python

  •

  MIT License

  •3•8•2•0•Updated Jun 4, 2024Jun 4, 2024

• crawlab-sdk

  Public
  SDK for Crawlab, including SDK for different programming languages such as Python, Node.js and Java, and a CLI Tool written in Python.

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •51•56•2•5•Updated Jun 3, 2024Jun 3, 2024

• crawlab-fs

  Public
  Backend file system module for Crawlab

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •12•4•0•0•Updated Mar 29, 2024Mar 29, 2024

• scrapy-ai

  Public
  AI-powered scrapy plugin

  Python

  •

  MIT License

  •0•1•0•0•Updated Feb 7, 2024Feb 7, 2024

• crawlab-vcs

  Public
  Version Control System (VCS) for Crawlab

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •11•4•0•1•Updated Aug 8, 2023Aug 8, 2023

• crawlab-next

  Public
  C#

  •

  Apache License 2.0

  •1•0•0•0•Updated May 31, 2023May 31, 2023

• artipub

  Public
  Article publishing platform that automatically distributes your articles to various media channels

  nodejstypescriptarticle+ 6mediaarticle-publisherdockermongodbcsdnsegmentfault

  TypeScript

  •

  BSD 3-Clause "New" or "Revised" License

  •529•3.1k•29•11•Updated Mar 5, 2023Mar 5, 2023

• crawlab-lite

  Public
  Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

  platformcrawlerspider+ 8web-crawlerscrapyscrapydscrapy-uiscrapyd-uicrawling-taskscrawlab+ 1

  Vue

  •

  BSD 3-Clause "New" or "Revised" License

  •75•224•1•8•Updated Feb 9, 2023Feb 9, 2023

• crawlab-db

  Public
  Backend database module for Crawlab

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •16•4•1•0•Updated Dec 26, 2022Dec 26, 2022

• examples

  Public
  Python

  •

  MIT License

  •10•5•0•0•Updated Oct 19, 2022Oct 19, 2022

• tada

  Public
  BSD 3-Clause "New" or "Revised" License

  •0•0•0•0•Updated Oct 8, 2022Oct 8, 2022

• template-parser

  Public
  Go

  •

  MIT License

  •7•4•0•0•Updated Oct 6, 2022Oct 6, 2022

• public-plugins

  Public
  Public plugins repo for Crawlab

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •6•4•1•0•Updated Oct 6, 2022Oct 6, 2022

• crawlab-log

  Public
  Log module for Crawlab

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •9•1•0•0•Updated Oct 1, 2022Oct 1, 2022

• traffic

  Public
  BSD 3-Clause "New" or "Revised" License

  •0•0•0•0•Updated Jul 2, 2022Jul 2, 2022

• plugin-notification

  Public
  Plugin for notifications | 消息通知插件

  Go

  •

  MIT License

  •11•3•1•0•Updated Jun 3, 2022Jun 3, 2022

• plugin-spider-assistant

  Public
  Plugin for spider assistance | 爬虫助手插件

  Vue

  •

  MIT License

  •8•2•0•0•Updated Jun 3, 2022Jun 3, 2022

• plugin-dependency

  Public
  Plugin for managing dependencies | 依赖管理插件

  Go

  •

  MIT License

  •8•3•2•0•Updated Jun 3, 2022Jun 3, 2022

• crawlab-plugin

  Public
  Plugin framework for Crawlab

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •10•3•0•0•Updated Apr 17, 2022Apr 17, 2022

• tutorials

  Public
  Tutorials for Crawlab

  2•1•0•0•Updated Dec 24, 2021Dec 24, 2021

• go-trace

  Public
  Trace stack tools

  Go

  •

  BSD 3-Clause "New" or "Revised" License

  •10•3•0•0•Updated Dec 19, 2021Dec 19, 2021

• resources

  Public
  Resources served on Github

  3•0•0•0•Updated Dec 14, 2021Dec 14, 2021