Developing a Highly Automated Web Archiving System Based on IIPC Open Source Software
Abstract
In this paper, we describe our development of a highly automated web archiving system based on IIPC open source software at the National Science Library (NSL). We designed a web archiving platform which integrates with popular IIPC tools, as well as developing several modules to meet special requirements of the NSL. We have applied a cooperative mode of central management server and collecting client, which can complete the unified management of seeds and support the collaborative work of multiple crawlers. Some modules were developed to improve the automation of web archiving workflows and provide more services.Details
- Creators
- Zhenxin Wu; Jing Xie; Jiying Hu; Zhixiong Zhang
- Institutions
- Date
- 2015-09-01 00:00:00 +0000
- Keywords
- open source software; web archive; platform development process automation
- Publication Type
- paper
- License
- CC BY 4.0 International
- Download
- 813717 bytes