hakrawler
Fast, simple web crawler for gathering URLs and JS endpoints. Pipe-friendly.
Tags: crawler, urls, fast, hakluke
- Category
- crawl
- Maintenance signal
- maintained
Use this when: JavaScript. Collect URLs from a host that is in scope, then save them before you read them.
Install
go
go install github.com/hakluke/hakrawler@latestCommand templates
Crawl from stdin
echo {url} | hakrawler -d {depth}