You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
16 lines
796 B
16 lines
796 B
2 years ago
|
Multi-UA web scraper geschreven in Perl voor educatieve doeleinden. Gebaseerd op het scannen van niet-prive IPv4 netwerken in volgorde van Class A naar Class E en het maken van rDNS requests als een web-gerelateerde port is open.
|
||
|
|
|
||
|
|
Dependencies: Selenium::Firefox, Net::DNS en Net::Curl::Easy.
|
||
|
|
|
||
|
|
|
||
|
|
[]
|
||
|
|
|
||
|
|
|
||
|
|
TODO:
|
||
|
|
|
||
|
|
- Add DB support for storing HTMLs, b64 screenshots, etc..
|
||
|
|
- Utilize the DB for randomizing the scanning of IPv4 blocks and supporting seemless continuation.
|
||
|
|
- Add RE-based querying system for scoped IPv4/domain scrapes.
|
||
|
|
- Reliably detect when being blocked by companies that own (too) much network space and delay requests appropriately (might not be necessary if various parts are sufficiently randomized).
|
||
|
|
- Windows support.
|