WebGraph 1.6

WebGraph is a framework to study the web graph

Developer:   Sebastiano Vigna       more software by author → Price:  0.00 License:   GPL (GNU General Public License) File size:   178K Language:    OS:    Rating:   0 /5 (0 votes) Your vote:   enlarge screenshot

WebGraph is a framework to study the web graph. WebGraph provides simple ways to manage very large graphs, exploiting modern compression techniques. More precisely, it is currently made of:

1. A set of flat codes, called codes, which are particularly suitable for storing web graphs (or, in general, integers with power-law distribution in a certain exponent range). The fact that these codes work well can be easily tested empirically, but we also try to provide a detailed mathematical analysis.
2. Algorithms for compressing web graphs that exploit gap compression and referentiation (? la LINK), intervalisation and codes to provide a high compression ratio: for instance, the WebBase graph (2001 crawl) is compressed at 3.08 bits per link, and a snapshot of about 18,500,000 pages of the .uk domain gathered by UbiCrawler is compressed at 2.22 bits per link (the corresponding figures for the transposed graphs are 2.89 bits per link and 1.98 bits per link). The algorithms are controlled by several parameters, which provide different tradeoffs between access speed and compression ratio.
3. Algorithms for accessing a compressed graph without actually decompressing it, using lazy techniques that delay the decompression until it is actually necessary.
4. A complete, documented implementation of the algorithms above in Java, contained in the package it.unimi.dsi.webgraph. Besides a clearly defined API, the package contains several classes that allow to modify (e.g., transpose) or recompress a graph, so to experiment with various settings. The package relies on fastutil for a type-specific, high-performance collections framework, on MG4J for bit-level I/O, on the COLT distribution for ready-to-use, efficient algorithms and on GNU getopt for line-command parsing.
5. Data sets for very large graph (e.g., a billion of links). These are either gathered from public sources (such as WebBase), or produced by UbiCrawler.

In the end, with WebGraph you can access and analyse a very large web graph, even on a PC with as little as 256 Mbytes of RAM. Using WebGraph is as easy as installing a few jar files and downloading a data set. This makes studying phenomena such as PageRank, distribution of graph properties of the web graph, etc. very easy.

What's New in This Release:

From this release, Webgraph is Java 5 only.

The code has been updated to fastutil 5 and MG4J 1.1.

Load methods return more specific types.

A new transpose method uses an offline strategy to transpose arbitrarily large graphs using little memory.
tags per link  bits per  the package  very large  web graph  link the  the algorithms  link and  this release  compression ratio  large graphs  web graphs  algorithms for  

Download WebGraph 1.6

 http://webgraph.dsi.unimi.it/webgraph-1.6-bin.tar.gz
 http://webgraph.dsi.unimi.it/webgraph-1.6-src.tar.gz

Authors software

fax4CUPS 1.28 (by Sebastiano Vigna) fax4CUPS is a very simple shell script that acts as a CUPS backend for a serial fax modem

ne 1.42 (by Sebastiano Vigna) ne application is a text editor based on the POSIX standard that runs (we hope) on almost any UN*X machine. ne is easy to use for

ERW 1.0.11 (by Sebastiano Vigna) ERW is a set of specifications and tools that makes it easy to create, modify and maintain via web a database described by an entity-

METAGRAPH 0.2 (by Sebastiano Vigna) METAGRAPH is a small set of macros that help in drawing (un)directed graphs with METAPOST and the boxes package. METAGRAPH project

fastutil 5.0.5 (by Sebastiano Vigna) fastutil extends the Java Collections Framework by providing type-specific sets, maps, lists and queues with a small memory footprint

Similar software

WebGraph 1.6 (by Sebastiano Vigna) WebGraph is a framework to study the web graph

Magda 1.0 (by Michal) Magda project is a graph drawing application. Magda (Magda Aesthetic Graph Drawing Application) is an application developed for ma

graph-tool 0.9 (by Tiago de Paula Peixoto) graph-tool project is a program to help with statistical analysis of graphs. Here are some key features of "graph tool": · suppor

Text::Graph 0.23 (by G. Wade Johnson) Text::Graph is a Perl extension for generating text-based graphs. SYNOPSIS use Text::Graph; blah blah blah Some data is

Graph-includes 0.11 (by Yann Dirson) Graph-includes creates a graph of dependencies between source files and/or groups of source files, with an emphasis on getting readab

Big Faceless Graph Library 2.2.4 (by Big Faceless Organization) The Big Faceless Graph Library is a Java class library for creating graphs and charts

fastutil 5.0.5 (by Sebastiano Vigna) fastutil extends the Java Collections Framework by providing type-specific sets, maps, lists and queues with a small memory footprint

Kgraphviewer 1.0.3 (by Gael de Chalendar) KGraphViewer program is a Graphviz dot graph file viewer for KDE. I used, in this program, code from the profiling tool kcachegri

The Figaro 1.1.0 (by Daniel Sawitzki) The Figaro project is a framework for implicit graph algorithms and representations by OBDDs. The "Framework for Implicit Graph Al

Advanced Graph and Chart Collection 4.94 (by Sirius Computer Consultants Limited) Advanced Graph and Chart Collection allows you to add powerful reporting features to your Web or intranet pages and applications.

Other software in this category

SquirrelMail 1.5.1 (by The SquirrelMail Project Team) SquirrelMail is a standards-based Webmail package written in PHP4

Tiki CMS/Groupware 1.9.7 (by Luis Argerich)

Downloader for X 2.5.7 (by Chuchelo) Downloader for X is a tool for downloading files from the Internet via both HTT

Links 2.1pre26 (by Martin Pergel) Links is graphics and text mode WWW browser, similar to Lynx

Mozilla Firefox 1.5.0.8 (by Mozilla Project)