July 23, 2009 -- D-Lib Magazine has published an essay titled, 21st Century Shipping: Network Data Transfer to the Library of Congress (external link) in its July/August 2009 issue. Mike Ashenfelder, a Digital Media Project Coordinator in the National Digital Information Infrastructure and Preservation Program, writes about how the Library uses network technology to transfer digital collections over the internet.
Different Components of the Network can Create Data Bottlenecks and Impede Data Flow.
Network data transfer is quicker, more efficient and more cost-effective than shipping data on hardware. Transfers that may take weeks on hardware, including shipping time and human labor, may take only hours over the network. The Library is simplifying and standardizing the process so that any institution, sending any type or amount of data, can easily move their content over the network using open-source tools.
The Library, together with the California Digital Library (external link), created the BagIt specification (PDF, 98.9KB) to describe a standardized, simplified digital container - - a "bag" - - in which to ship data. The Library has developed a suite of software tools (external link) that support the creation and validation of bags, as well as tools to accelerate network transfers by splitting collections into multiple streams of data. Additionally, the Library has produced a video about Bagit and data transfer.
