Working Groups

To achieve its mission, the consortium sets up dedicated committees consisting of members of some of the participating libraries working on specific topics and providing the consortium with various deliverables. A technical committee supervises and runs projects which have an impact on the overall Web archiving framework and technical architecture. It guarantees convergence and consistency of standards and practices in areas such as harvesting, access and preservation.

The chartered working groups are Standards, Harvesting, Access, and Preservation.

Standards

IIPC work on standards will depend on the directions and priorities to be proposed by the three other working groups. In the short term, the IIPC is focusing on the WARC standardization process. Future investigations may involve other standards, APIs, metadata, and metrics.

Harvesting

The Harvesting Working Group’s primary focus is the development of a smart crawler. Other areas of focus include:

Development and support of the WARC file format
Best practices
Feature requests for crawler
Harvesting the deep web
Harvesting video and streaming media

Access

The Access Working Group will focus on initiatives, procedures and tools required to provide immediate access and to preserve the future access to Internet material in a Web archive. Focus areas include:

Defining User Requirements to improve existing access tools (Wayback Machine, WERA)
Testing full text indexing using NutchWax
Defining requirements for user authentication/authorization/access controls
Access tools for the analysis of the content of the archived internet material
Access tools for the analysis of the structure of the archived internet material

Preservation

The IIPC Preservation Working Group is looking at policy, practices and resources in support of preserving the content and accessibility of Web archives. Over the past decade, there has been great attention paid to the processes of capturing online resources, as a necessary step in their preservation; however, work on maintaining accessibility for the long term remains reasonably undeveloped. At the same time, many approaches have been proposed and implemented for other kinds of digital collections. The Preservation Working Group aims to understand and report on how such approaches might be used with Web archives, as well as the special characteristics of Web archives that might require new approaches.

2003-2006

During 2003-2006, the following working groups were chartered: