Toolkit

One of the main objectives of the IIPC has been to develop a set of high quality, easy-to-use and open source tools for setting up a web archiving chain. The working groups have identified the following needs and focused their work on:

Acquisition

An archiving quality crawler scaling up to domain level
A portable database extraction and migration tool for database-driven Deep Web sites

Focused selection and verification

Analytical and prioritization tools for automatically focusing the crawl
Curator tools to provide a non-technical interface for selecting, monitoring and verifying archived web sites

Collection storage and maintenance

Files manipulation tools

Access and finding aids

A tool providing a browsing interface for a web archive, which provides management of the linking environment, presentation of URI and navigation in time
An indexer able to provide full text indexing on large collections with basic functionality (boolean operators and proximity), including the time dimension of the archive
A query interface generator for archived databases