gospider
Fast Go crawler that also extracts JS, forms, robots, sitemaps and subdomains. Good when katana is too quiet or you want form fields.
Tags: crawler, js, forms
- Category
- crawl
- Maintenance signal
- maintained
Use this when: JavaScript. Collect URLs from a host that is in scope, then save them before you read them.
Install
go
go install github.com/jaeles-project/gospider@latestCommand templates
Crawl + JS
gospider -s {url} -c 10 -d 3 -t 20 --js --robots --sitemap -o {output}