P5-gungho

Jul 20, 2023

Event Based High Performance Web Crawler Framework

Gungho is Yet Another Web Crawler Framework, aimed to be extensible and fast.

Gungho tries to build from clean structures, based upon principles from the likes of Catalyst and Plagger, so that you can easily extend it to your liking.

Features such as robot rules handling robots.txt and request throttling can be removed/added on the fly, just by specifying the components that you want to load. You can easily create additional functionality by writing your own component.



Checkout these related ports:
  • Zope213 - Object-based web application platform Version 2.13
  • Zola - Fast static site generator
  • Zgrab2 - Fast Go application scanner
  • Zerowait-httpd - Lightweight and fast http server
  • Zenphoto - Simpler web photo gallery
  • Zend-framework - Framework for developing PHP web applications
  • Yuicompressor - The Yahoo! JavaScript and CSS Compressor
  • Ytdl - YouTube downloader written in Go
  • Yt-dlp - Command-line program for downloading videos from various platforms
  • Youtube_dl - Program for downloading videos from various services
  • Yourls - Your Own URL Shortener
  • You-get - Dumb downloader that scrapes the web
  • Yaws - Web server for dynamic content written in Erlang
  • Yarr - Yet another rss reader
  • Yarn - Package manager for node, alternative to npm (meta port)