Fingerprinting / Crawling

Banner Grabbing

curl -I inlanefreight.com

Wafw00f

wafw00f inlanefreight.com

Nikto

nikto -h inlanefreight.com -Tuning b

The -Tuning b flag tells Nikto to only run the Software Identification modules.

Crawling / Scrapping

URLFinder

GitHub - projectdiscovery/urlfinder: A high-speed tool for passively gathering URLs, optimized for efficient and comprehensive web asset discovery without active scanning.GitHub

Katana

GitHub - projectdiscovery/katana: A next-generation crawling and spidering framework.GitHub

Gourlex

gourlex -t domain.com

GitHub - trap-bytes/gourlex: Gourlex is a simple tool that can be used to extract URLs and paths from web pages.GitHub

LazyEgg

GitHub - schooldropout1337/lazyeggGitHub

Reconspider

reconspider | Kali Linux ToolsKali Linux

GitHub - 0xSs0rZ/Recon: reconGitHub

pip3 install scrapy

$ wget https://academy.hackthebox.com/storage/modules/279/ReconSpider.zip
$ unzip ReconSpider.zip 

python3 ReconSpider.py http://inlanefreight.com

cat results.json