Sealbook / Open source / Developer tools, CLI & editors

scrapy/dirbot

Scrapy project to scrape public web directories (educational) [DEPRECATED]

License & maintenance

No license stated: by default nobody may reuse the code. Ask the authors before using it in a product.

Last code push 8 years ago.

Similar projects

	Repository	Stars	Language	License	Activity
1	yt-dlp/yt-dlpA feature-rich command-line audio/video downloader	★ 199.1k	Python	Unlicense	Active
2	microsoft/vscodeVisual Studio Code	★ 193.8k	TypeScript	MIT	Active
3	github/gitignoreA collection of useful .gitignore templates	★ 176k	—	CC0-1.0	Active
4	jlevy/the-art-of-command-lineMaster the command line, in one page	★ 162.5k	—	—	Stale
5	ytdl-org/youtube-dlCommand-line program to download videos from YouTube.com and other video sites	★ 141.3k	Python	Unlicense	Slowing
6	rust-lang/rustEmpowering everyone to build reliable and efficient software.	★ 120.5k	Rust	Apache-2.0	Active
7	microsoft/terminalThe new Windows Terminal and the original Windows console host, all in the same place!	★ 105.2k	C++	MIT	Active
8	microsoft/playwrightPlaywright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.	★ 98k	TypeScript	Apache-2.0	Active