WWW::Google::SiteMap::Robot 1.09
WWW::Google::SiteMap::Robot is a Perl extension for creating Google SiteMaps by spidering a site. SYNOPSIS use WWW::Google::S
|
|||||||||||||||||||
WWW::Google::SiteMap::Robot is a Perl extension for creating Google SiteMaps by spidering a site.
SYNOPSIS
use WWW::Google::SiteMap::Robot;
my $robot = WWW::Google::SiteMap::Robot->new(
domain => 'www.jasonkohles.com',
restrict => qr{^http://www.jasonkohles.com/},
starting_url => ['/index.html','/google-me.html'],
delay => 1, # delay in minutes
sitemap_file => '/var/www/html/sitemap.gz',
sitemap_url => 'http://www.jasonkohles.com/sitemap.gz',
user_agent => 'MyOwnSpider/1.0',
);
$robot->run();
This is a simple robot class which subclasses LWP::RobotUA to create a web-crawling spider. By giving it the URL to your home page, it will crawl all the pages it can find and create a sitemap for them.
Requirements:
tags
Download WWW::Google::SiteMap::Robot 1.09
Authors software
|
|
|
|
|
|
|
|
|
|
Similar software
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Other software in this category
|
|
|
|
|
|
|
|
|
|
Featured Software
jEdit 4.3 pre8
jEdit is an Open Source text editor written in Java
Opera 9.02
Surf the Internet in a safer, faster, and easier way with Opera browser
GNU Aspell 0.60.4
GNU Aspell is a Free and Open Source spell checker designed to eventually replace Ispell
- Communications
- Database
- Desktop Environment
- Games
- Internet
- Multimedia
- Office
- Programming
- Science and Engineering
- System
- Text Editing&Processing
