The IIPC and the British Library co-sponsored a three-day technical training workshop, held on 14-16 January 2015 in London. The workshop provided the attendees with deeper understanding of Heritrix, OpenWayback and Solr. It was open to staff from all IIPC member institutions.
Invited speakers included:
- Andy Jackson, British Library
- Roger Coram, British Library
- Gil Hoggarth, British Library
- Kristinn Sigurðsson , National Library of Iceland
- Toke Eskildsen, NetArchive, Denmark
- Helen Hock-Yu. The British Library
- Gina Jones - Library of Congress
- understanding of advanced crawl configurations in Heritrix, including tuning of DecideRules to institutional needs.
- the ability to upgrade to and install OpenWayback 2.0.0.
- detailed overview of Solr and its application in web archiving, and practical experience in using Solr for web archives.
- the ability to apply learned knowledge to real-world data and issues.
- understanding of commonly used and relevant features of the tools
Outline of programme
The workshop consisted of taught knowledge, practical examples and hands-on learning.
Taught knowledge of Heritrix, OpenWayback and Solr will be reflected by a series of presentations on day-one, introducing specific aspects of each application. The presentation on Heritrix for example will provide an overview of the DecideRules and modules that can be customised. It will also include a walk-through of the important codes in Heritrix for which a thorough understanding enables better customisation.
Practical examples of local implementation of Heritrix, OpenWayback and Solr will be presented to show how things work in practice. The British Library for example will present in detail how Solr is integrated with the BL workflows and outline the key stages of deployment and indexing.
- Hands-on experience includes real-time installation of OpenWayback, the use of default Solr UI to explore data. Delegates will carry out hands-on exercise with real data, and compares and discuss local DecideRules in Heritrix.
The opening day of the workshop was recorded and all of the talks can be viewed at the IIPC YouTube channel.