Following is an interview with Nan Rubin, project director of the Preserving Public Television project. The interview was completed in August 2005. More details on Thirteen's project can be found at www.ptvdigitalarchive.org (external link).
When did producers such as Thirteen and WGBH begin producing programs in digital formats for distribution by PBS?
There are a few things to keep in mind when asking about digital formats of programs:
- A number of digital formats are used in program production, that is, the formats used for recording, acquiring and editing public television programs.
- Totally different digital formats are used for program distribution, which vary because of the different distribution channels used to get a program from PBS to each public television station.
- That's not all. There is yet another set of formats that are used solely for local on-air broadcasting, based on play-out servers, automation systems and other operational equipment.
Generally, distribution and broadcast formats are more compressed and of lower quality than production formats. One of our problems is to figure out how to maintain the highest quality video format while capturing the content of a program when it's "broadcast," since saving the program using the distribution or broadcast format isn't necessarily going to be the best for long-term preservation.
Digital videotape has been in use since the 1980s. The D1 and D2 videotape formats were developed to allow recording the video signal on tape in a digital form, allowing multigeneration recordings with minimal impairment. But they could only be used for recording, and the digital signals were all translated into an analog signal for distribution and broadcast.
Then, in the 1990s, advances in digital signal processing made it possible to build in "compression" that was useful for saving video using smaller files. One form of compression was standardized as JPEG (Joint Photographic Experts Group), which also came into common use for moving pictures as motion-JPEG, (M-JPEG). JPEG compression made the nonlinear editor and video server practical. This was followed by the development of the MPEG-1 (Motion Picture Experts Group) format for low-quality video, and later MPEG-2 for broadcast quality video and better.
Digital production, then, has been in place since the mid-1990s, but digital recording has been around a decade longer. All of us – Thirteen, WGBH and PBS -- have programs that were recorded on older digital formats like D2, and right now, we consider D2 to be a highly at-risk format.
Today, the broadcast chain from the production end is virtually all digital. Most of the broadcast end is all digital as well.
However, the distribution segment in between is not yet totally digital -- it still has a number of analog elements. If all goes according to plan, by the end of 2006, PBS will have completed the Next Generation Interconnection System, which will be a complete digital chain from end to end.
What digital-preservation activities were already under way at PBS prior to your partnership with NDIIPP and when did they begin?
There haven't been any organized preservation activities relating specifically to digital program files. Both Thirteen, which has had an archive for about five years, and WGBH, which has had an archive in place for more than 15, have been using digi-Beta tapes for archiving purposes and treating the tapes like analog materials. PBS is in a similar position, but they have a lot more programs to handle and can barely put tapes on shelves.
At the moment, the most extensive storage treatment of digital files is when programs are put onto servers for broadcast playout. What used to be a station's tape library is now a server that is generally referred to as the "archives," meaning programs are stored as digital files on some storage device until they are scheduled for air.
Since most programs have broadcast rights for three to four years, stations are now in the process of ingesting and storing these large program libraries for at least three to five years. After that, when the broadcast rights of any given program have expired, the automation system can be set to automatically purge the files so that the program is no longer available to be scheduled for broadcast.
Using servers instead of videotape machines as the source of playing back programs is relatively recent. They rely on a combination of different systems working together, including different program recording, storage and playback devices, all controlled by the station's automation system; any given station is likely to have a mix of both digital and analog devices that are slowly being replaced completely with a digital broadcast chain.
Getting all these computers to work well together is not easy. At WNET the system has been in place for barely two years, and many stations are just now in the process of acquiring the components they don't have so they can put a complete system in place.
Even so, none of these program storage systems are designed for long-term preservation. Our project is the first one anywhere within public television that is specifically aimed at designing a system for digital preservation.
Has there been any systematic conversion of analog programs to digital?
The stations with archives, Thirteen and WGBH, are able to do analog-to-digital remastering only on a sporadic basis because of the cost. Neither station has discretionary funds that can be used for this activity, so that, in general, such remastering is done "on-demand" – when another party has offered to pay the costs, or there is such a clear need or value to the materials that the station sees it as an important investment.
For example, Thirteen was able to digitize and remaster a collection of 75-plus programs from the '60s and '70s through a collaboration with the Broadway Theatre Archive, a commercial venture that covered all the costs plus made the major investment in clearing rights so the programs could be sold for home video. The station would never have taken on such a major digitizing project without dedicated funding.
At PBS, they have done no conversion of old materials, except that they are ingesting all their current programs now, as they are getting ready to start distributing digital files for broadcast.
There have been some smaller-scale local efforts, notably the Kentucky Educational Television Digital Archives project (external link), which had a goal of transferring its entire videotape program library to hard drive -- 6,549 tapes running 4,380 hours. It was paid for by an anonymous donation of $300,000 and matched by a grant from a local foundation.
More recently, Bill Moyers, as an independent producer, is starting to make plans to begin digitizing all of his own productions, spanning more than 20 years of materials.
You say in your project description that you will form an advisory committee "to develop criteria for selecting and cataloging materials for preservation." Has this been formed? If so, what has been accomplished so far?
We have not yet formed any formal advisory committees. However, we have collected a large number of names of interested folks, through a series of focus groups; from listservs and online discussions, and by making presentations at public television and other gatherings. During this second project year, I expect we will draw on our growing lists to organize one or more formal groups of advisers for the project.
Will the work of this committee replicate work done for analog programming? In other words, have you also had to make choices on which PBS analog programs should be preserved? If so, can you briefly describe the selection criteria and state how these criteria will apply to digital programs?
There is no comparable committee working with analog programming – the preservation policies that Thirteen and WGBH have in place are devised and administered by staff based solely on business, legal and other practical considerations. None of us have any committees involved with assessing the materials.
With no money to speak of for preservation, current collection practices at each entity rest largely on saving materials to which each owns the rights and thus has the potential to resell commercially. At PBS, copies of all nationally distributed programs are saved by contract agreement with the Library of Congress, and as the deposit copy for the Copyright Office.
Do you foresee/have you already identified any issues related to intellectual property rights that did not apply to your preservation of analog programming?
The biggest issue we face regarding intellectual property rights is that, among all the myriad rights "markets" that are available for program distribution, there is nothing explicit giving us rights for long-term preservation and access. The rights for analog materials are similar, and preservation rights can also be very muddy.
As the digital world has vastly expanded the possible distribution channels of any given work, the competition over rights and content control has become both fragmented and fierce. There is great uncertainty over our ability to acquire or exercise preservation rights to material that is not totally ours, and this is an area we need to pursue.
For PBS, which owns nothing, extending the rights from distribution only to preservation may not sound like that big a leap -- except that negotiating the rights for who can have access to the materials once they are preserved is a huge issue and one that PBS is not eager to take up.
Has PBS moved any of its programming to a tapeless environment, whereby programs are aired directly from a server?
At both WGBH and Thirteen, our broadcast chains are already tapeless. That transition at PBS is currently under way and is scheduled to be completed by the end of calendar year 2006. [See my discussion above about the differentiation of the stations from PBS.]
You have spoken of moving to a preservation system that preserves discrete elements of programs, such as opening credits, program segments, underwriter announcements and closing credits. Broadcasts would then be assembled by using the elements as needed. This would result in there no longer being a "definitive version" of a program. What are the advantages of moving to such a system? What are the implications of not having definitive versions of programs?
Dr. Howard Besser wrote in "The Moving Image," fall 2001, "…Preservationists need to … engage in some fundamental paradigm shifts in how they view the preservation process… [one] key shift will be … from saving finished works as a whole to an asset-management approach that deals both with component parts of works and with ancillary materials that relate to the work."
In the last few years, our entire business has changed in exactly that manner that Howard described – television production has become a process of asset management of many component parts, and the end products can be any number of variations depending on whom it is for.
In many ways, the system has been forced into this position whether it likes it or not. In theory, the advantages are that elements can be used over and over again in many different ways; updates and edits can be easily accomplished without having to edit the entire program; versions for different markets can be easily and efficiently produced, etc.
The implications, though, are that programs become a "moving target" with no single version that can be called "final." If all you have are program segments or elements, it becomes critical to keep documentation of the production accessible, so that it could be reconstructed or reassembled, or so that elements can be switch out, whenever necessary.
Could you briefly describe the Universal Preservation Format and any milestones that have been achieved in its creation so far?
I'm not the best person to explain this; it is really the area of expertise of Dave MacCarn, chief technologist at WGBH. Basically, the concept is that both the program "essence," that is, the digital file with the visual information, and the program "metadata" file with all the descriptive and technical information, are packaged together with a "wrapper" file that allows the information to be decoded, regardless of the system reading it, yet keeps the files tied together.
There are several technical schema working on developing such a solution. Dave started writing one himself a few years ago but didn't have the funding to complete it. Since then, the concept has become more accepted among technologists and fellow travelers, and we hope to update Dave's original work to finish the design taking into account recent improvements in the field.
Is there anything else you would like to discuss?
Our project is based, in part, on adopting an asset management system for storing and accessing our moving-image materials. This is turning out to be more difficult and complex than we had planned. At the same time, we were also unprepared for learning about the Library of Congress's National Audio-Visual Conservation Center and its implications for the future of television holdings at the Library.
We are now making adjustments to accommodate both these developments!
