January 2006: "What if NDIIPP knew what NDIIPP knows?"
The recent meeting of the preservation project partners for the National Digital Information Infrastructure and Preservation Program (NDIIPP) was convened to answer that very question.
Held in Berkeley, Calif., on Jan. 9-11, this third meeting of the NDIIPP project partners was designed to continue the work done during the two previous partner meetings: update the participants on the program and provide them with a forum to inform fellow participants on what they have learned so far and the common issues they face.
The question, which, said another way, expresses the idea that these meetings are a way for the NDIIPP partners to maximize their achievements by sharing their experiences with others, was posed by Clay Shirky, an adjunct professor in New York University’s graduate Interactive Telecommunications Program who has done consulting work for NDIIPP. Shirky was one of the featured speakers at the meeting.
About 100 people attended -- Library of Congress staff, members of the National Digital Strategy Advisory Board, experts in digital preservation and, of course, the preservation partners themselves – from eight consortia comprising 36 institutions. In September 2004, NDIIPP funded eight project consortia for a total of nearly $14 million. Details about the awards and the types of content to be collected and preserved are at http://www.digitalpreservation.gov/about/background.html.
Laura E. Campbell, associate librarian for Strategic Initiatives, who is leading the NDIIPP program for the Library, welcomed the participants the morning of Jan. 9. "We are glad you were able to join us as we continue to work to save the digital heritage of this nation. Your pioneering efforts will make it possible for other libraries, archives and institutions to base their digital preservation efforts on the excellent work you are doing."
William LeFurgy, an NDIIPP program manager who moderated the meeting, provided an update of the preservation program’s progress since the last meeting, in June 2005. "We held three workshops last spring with representatives from all the U.S. states and several territories to discuss what they are doing in the area of digital preservation. There is need for a national approach," he said. He noted that the meetings provided a forum for the state representatives to meet others facing the same issues – a meeting that would not otherwise have occurred.
He also spoke about the Library of Congress-National Science Foundation research awards which went to 10 teams to undertake pioneering research to support the long-term management of digital information. LeFurgy told the audience that the teams would report their results this summer, and that "there is the possibility of another round of such awards this year." The work of the research projects will be integrated with the larger NDIIPP activities. He also welcomed representatives from the projects, who were attending the partners meeting for the first time.
As part of its work on an international front, NDIIPP is planning to hold a joint workshop in Washington this year with the Joint Information Systems Committee of the United Kingdom "to compare our initiatives and look at future avenues for collaboration." The Joint Information Systems Committee provides "strategic guidance, advice and opportunities to support teaching, learning, research and administration" in association with higher education.
LeFurgy also reminded the audience that "we are halfway through NDIIPP," which began with a public law passed in December 2000. The Library will submit a comprehensive report to Congress in 2010 detailing NDIIPP’s achievements and a strategy for continuing and building upon the work done during the first decade of this century.
As part of his self-described "whirlwind tour" of what NDIIPP is doing currently, LeFurgy expressed the hope that the Section 108 Study Group would reach consensus on the recommendations it will make to Congress this summer on alterations to the copyright law for libraries and archives that handle digital media.
He also mentioned that during 2006, NDIIPP would be seeking more content from commercial owners.
What has the Library learned so far from its eight partners?
"There is a public concern about the proper care of digital information," LeFurgy said. Collecting institutions are also interested in the development of practical models for digital preservation, given the fact that their resources are limited. "This finding points to the need for the work you are doing, so that groups that haven’t worked together in the past can do so now.
"We also have learned that there are expectations for the Library to be a catalyst and coordinator for various digital preservation efforts, without asserting 'top-down' ownership," he continued.
If there is anything positive to the tragedy wrought by Hurricane Katrina it is that it has heightened the public’s awareness of preservation’s importance. In addition, stories in the media about identity theft and data loss, such as the vanishing collections of personal digital photo collections and the deterioration of compact discs and DVDs, have only added to growing worries among the public that digital information is neither secure nor safe from those who should not have access to it.
The issue of what "stewardship" means in a digital environment encompasses much of what NDIIPP is seeking to accomplish. According to Nancy Cline, the Roy E. Larsen librarian at Harvard University, whom LeFurgy quoted, "Stewardship can be represented with two faces: one looking back upon all that has been garnered over time and another that faces forward, anticipating, planning, preparing and thinking strategically."
Following LeFurgy’s discussion, someone in the audience wanted to know "What happens after 2010?"
Campbell answered that "nobody thought there would be an end in 2010. Who will pay for digital preservation is not clear. What we do know is that we all must share the workload and that we are not going to define one system for everyone. We have much to learn about how to share this responsibility, and it will require a transformation about how we do our work."
When it was time for Shirky to make his presentation he noted how "it’s incredible to see this much energy" focusing on digital preservation and the shared responsibility for it.
He suggested the larger, shared problem be addressed using the "shearing layers" model, as described in Stewart Brand’s book "How Buildings Learn." Brand’s six shearing layers for buildings are Site, Structure, Skin, Services, Space and Stuff, with each of the layers being handled by an expert for that layer. According to Shirky, the NDIIPP project can be viewed similarly: The problems of digital preservation must be approached in layers by various partners with various types of expertise. "There is no way the Library of Congress can solve this problem" alone, he said.
What is important for NDIIPP and its many partners is the quality of the "social networking" that the program catalyzes. "Which functions go together and the quality of the interfaces between the systems" will have a big impact on success.
Shirky pointed to NDIIPP’s Archive Ingest and Handling Test (AIHT) as an excellent example of how different approaches to the same problem can coexist. AIHT tested the ingest of a large archive into diverse systems. The digital archive was donated by George Mason University, and the Library conducted the test with Johns Hopkins, Harvard, Stanford and Old Dominion universities.
AIHT showed "how we can successfully share information among institutions without the same worldview. The notion is that within this room we can build value.
"Think about, What are your areas of expertise? What aren’t? We are looking for ways to 'hive' people into different groups," he continued. The four NDIIPP "affinity" groups (Collection and Selection, Economic Sustainability, Rights and Restrictions, and Technical Architecture), which convened during this and past NDIIPP partners meetings, were designed to bring people with expertise in various areas together to discuss their successes and problems that need attention. "Everyone is here because you do something well," Shirky concluded.
The Jan. 9 afternoon session began with a presentation on "Portico’s Experience in Building Partnerships," from Eileen Fenton, executive director of Portico, which is an NDIIPP partner. In October 2005, NDIIPP made a $3 million grant award to Portico for its development. Portico is a nonprofit electronic archiving service that is devising a technical infrastructure to support long-term preservation of e-journals and is also working to foster the development of new business models for digital preservation services.
Fenton promised to present "the good, the bad and the ugly." In other words, she would tell the audience "what we have learned." She spoke about the organization’s early days. "We rounded up 10 intrepid publishers representing the full gamut of publishing. Because e-journals are licensed, not owned, by libraries, collaboration is essential."
In many ways, Fenton pointed out, Portico is facing on a smaller scale some of the same issues as NDIIPP, for example, How to balance the rights and needs of the content owners and those who use it? and, How to pay for the content’s preservation?
Portico, like NDIIPP, has also learned that "you cannot mandate by fiat that the world will be what you wish it would be," that is, "you cannot mandate the formats" of the digital information you receive.
Initially, according to Fenton, publishers were leery of taking "any steps that would cause them to lose revenue." Portico has been able to successfully persuade journal publishers that their participation will not reduce their potential revenues and that partnering with Portico will reduce their internal archival work. For libraries, perpetual access to journals is an incentive to join Portico. Publishers annually pay from 5 to 10 percent total journal revenue for Portico’s services, and libraries pay $1,500 to $24,000, depending on a library’s total materials expenditure. View Fenton’s presentation (PDF) on the NDIIPP Web site.
Paul Courant, professor of economics and public policy and former provost of the University of Michigan, followed Fenton with a stimulating discussion of a topic that was of paramount concern to all present: how to ensure the economic sustainability of a digital preservation program.
"You’re expecting me to be not an economist but a magician who can solve all your problems," he began. The audience demonstrated its agreement with a round of laughter. "Your instincts that markets aren’t good at solving problems are tragically correct. But our cultural heritage is at stake."
Courant also reminded listeners that any arrangements made for digital preservation "are not simply about money. Value is hard to assess, and there is a conflict of interests for that money. Even before the digital age, libraries were facing acquisitions budget pressures."
The ease with which digital materials can be delivered makes access faster, while at the same time the ease with which digital materials can be produced -- and subsequently lost, altered or used in a manner that violates copyright law -- means that "access to our cultural heritage used to be less difficult."
A digital archive in which access is severely limited is the "easiest preservation solution," according to Courant. The building of such an archive also raises issues addressed by the NDIIPP affinity groups, for example, the Collections and Selection group. "We don’t want to preserve it all," he said. "We probably want all of Bruce Springsteen. 'American Idol'? Maybe some of it. 'Fear Factor'? Courant left the last questioned unanswered as the audience roared.
He left no doubt that the "economics of digital preservation are expensive," especially in the case of "storage where we can still find the stuff. The conflict for public libraries is that they have always provided free access, "purely for the public good. … So the right price to charge for access to digital materials is nothing – a poor business model, because the cost of producing and preserving a digital collection is not nothing," Courant admitted. "Thus, the markets are bad at producing" such collections.
He ended by asking rhetorically, "Who is going to be the Bono of digital preservation? Who will paint pictures of fear and bliss?"
Much of the second day of the meeting was devoted to various "breakout" sessions in which attendees engaged in discussions based on their areas of interest and expertise. An overview of Portico’s technical architecture by the company’s Evan Owens was also part of the Day 2 activities.
Clifford Lynch, executive director of the Coalition for Networked Information, was the only speaker on Day 3, as the meeting ended at 11 a.m. In "Making the Case for Digital Preservation," Lynch told the audience that "we must talk about the notion of acceptable losses" when collecting digital materials. "There is a need to admit that in an engineered system, on a large scale, there is no perfection, just as libraries misplace things occasionally. Does it make sense to spend too much for perfection?
"I probably have something to say to offend everyone," he joked.
Lynch then complimented the Portico business model, but noted that "Portico exists in a peculiar niche: scholarly journals. This model does not necessarily exist outside of journals." Lynch explained: "Everyone recognizes in the scholarly world that we need a credible means of electronic preservation." In other areas of digital media, the need for its preservation may not be as clear-cut, especially to producers of the material.
Lynch spoke of the transformations that will be necessary if libraries and archives are to remain viable and reliable sources of information in the future. "Saving our cultural and intellectual record is not an activity around the margins" of what cultural memory organizations are expected to do. They need to make "calls for basic digital preservation funding and reprioritize their operating budgets.
"It is easy to get focused on audiences that can write checks," an admittedly important aspect of digital preservation. But it is more important to get a conversation started about "the importance of the work cultural institutions do and connect it to broader social changes, connect it to broad populations."
Lynch also advocated for an "environmental scan" of the current state of digital preservation. "We can’t talk about digital preservation without quantitative facts. Just saying it’s too big is not acceptable."
[NDIIPP is currently undertaking just such a scan. It will look at digital information production and stewardship, and develop a high-level descriptive framework. The researchers will primarily review those sectors that produce large quantities of information of value to the nation. They will consult critical communities in both the public and private sectors. Definitions of "risk" and "value," as well as observations about stewardship arrangements, will also be a part of the scan’s report. The goal is to give a "big picture" context to the modeling and testing work of NDIIPP and its partners to help set the priorities for the national digital collection strategy requested by Congress when it passed the NDIIPP legislation in 2000.]
The Internet has helped to produce what Lynch called "a huge, troublesome new class of materials – people as their own publishers. How does that figure into the landscape?
"There is a sense of huge confusion for most people outside a very narrow community" regarding the importance of digital preservation. We need to make clear how the massive [and well publicized] digitization projects of organizations like Google and the Open Content Alliance interrelate" with a need for preserving these and other digital materials.
Lynch foresees "an enormous distributed train wreck approaching at the state and local level," with governments increasingly producing records digitally with no methodologies for ensuring their preservation and access in the long term. "Most state and local governments do not have the resources to deal with this. They very much need outreach and other help to get started with digital preservation efforts."
Ultimately, he concluded, "we need to know what information has been saved and what hasn’t been lost—yet." Details about digital content that has been lost are also critical to help make the case for digital preservation.
