Film-Tech Cinema Systems
Film-Tech Forum ARCHIVE


  
my profile | my password | search | faq & rules | forum home
  next oldest topic   next newest topic
» Film-Tech Forum ARCHIVE   » Community   » Film-Yak   » Digital Archiving Artical in NY Times (Page 1)

 
This topic comprises 2 pages: 1  2 
 
Author Topic: Digital Archiving Artical in NY Times
John Walsh
Film God

Posts: 2490
From: Connecticut, USA, Earth, Milky Way
Registered: Oct 1999


 - posted 12-23-2007 01:51 PM      Profile for John Walsh   Email John Walsh   Send New Private Message       Edit/Delete Post 
A not-to-technical, but interesting artical about digital archiving.

The New york Times
LOS ANGELES December 23, 2007
MICHAEL CIEPLY

The Afterlife Is Expensive for Digital Movies‏

TIME was, a movie studio could pack up a picture and all of its assorted bloopers, alternate takes and other odds and ends as soon as the production staff was done with them, and ship them off to the salt mine. Literally.
Having figured out that really big money comes from reselling old films — on broadcast television, then cable, videocassettes, DVDs, and so on — companies like Warner Brothers and Paramount Pictures for decades have been tucking their 35-millimeter film masters and associated source material into archives, some of which are housed in a Kansas salt mine, or in limestone mines in Kansas and Pennsylvania.
A picture could sit for many, many years, cool and comfortable, until some enterprising executive decided that the time was ripe for, say, a Wallace Beery special collection timed to a 25th-anniversary 3-D rerelease of “Barton Fink,” with a hitherto unseen, behind-the-scenes peek at the Coen brothers trying to explain a Hollywood in-joke to John Turturro.
It was a file-and-forget system that didn’t cost much, and made up for the self-destructive sins of an industry that discarded its earliest works or allowed films on old flammable stock to degrade. (Indeed, only half of the feature films shot before 1950 survive.)
But then came digital. And suddenly the film industry is wrestling again with the possibility that its most precious assets, the pictures, aren’t as durable as they used to be.
The problem became public, but just barely, last month, when the science and technology council of the Academy of Motion Picture Arts and Sciences released the results of a yearlong study of digital archiving in the movie business. Titled “The Digital Dilemma,” the council’s report surfaced just as Hollywood’s writers began their walkout. Busy walking, or dodging, the picket lines, industry types largely missed the report’s startling bottom line: To store a digital master record of a movie costs about $12,514 a year, versus the $1,059 it costs to keep a conventional film master.
Much worse, to keep the enormous swarm of data produced when a picture is “born digital” — that is, produced using all-electronic processes, rather than relying wholly or partially on film — pushes the cost of preservation to $208,569 a year, vastly higher than the $486 it costs to toss the equivalent camera negatives, audio recordings, on-set photographs and annotated scripts of an all-film production into the cold-storage vault.
All of this may seem counterintuitive. After all, digital magic is supposed to make information of all kinds more available, not less. But ubiquity, it turns out, is not the same as permanence.
In a telephone interview earlier this month, Milton Shefter, a longtime film preservationist who helped prepare the academy’s report, said the problems associated with digital movie storage, if not addressed, could point the industry “back to the early days, when they showed a picture for a week or two, and it was thrown away.”
Mr. Shefter and his associates do not contend that films are actually on the verge of becoming quite that ephemeral. But they do see difficulties and trends that could point many movies or the source material associated with them toward “digital extinction” over a relatively short span of years, unless something changes.
At present, a copy of virtually all studio movies — even those like “Click” or “Miami Vice” that are shot using digital processes — is being stored in film format, protecting the finished product for 100 years or more. For film aficionados, the current practice is already less than perfect. Regardless of how they are shot, most pictures are edited digitally, and then a digital master is transferred to film, which can result in an image of lower quality than a pure film process — and this is what becomes stored for the ages.
But over the next couple of decades, archivists reason, the conversion of theaters to digital projection will sharply reduce the overall demand for film, eventually making it a sunset market for the main manufacturers, Kodak, Fujifilm and Agfa. At that point, pure digital storage will become the norm, bringing with it a whole set of problems that never troubled film.
To begin with, the hardware and storage media — magnetic tapes, disks, whatever — on which a film is encoded are much less enduring than good old film. If not operated occasionally, a hard drive will freeze up in as little as two years. Similarly, DVDs tend to degrade: according to the report, only half of a collection of disks can be expected to last for 15 years, not a reassuring prospect to those who think about centuries. Digital audiotape, it was discovered, tends to hit a “brick wall” when it degrades. While conventional tape becomes scratchy, the digital variety becomes unreadable.
DIFFICULTIES of that sort are compounded by constant change in technology. As one generation of digital magic replaces the next, archived materials must be repeatedly “migrated” to the new format, or risk becoming unreadable. Thus, NASA scientists found in 1999 that they were unable to read digital data saved from a Viking space probe in 1975; the format had long been obsolete.
All of that makes digital archiving a dynamic rather than static process, and one that costs far more than studios have been accustomed to paying in the past — no small matter, given that movie companies rely on their libraries for about one-third of their $36 billion in annual revenue, according to a recent assessment by the research service Global Media Intelligence.
“It’s been in the air since we started talking about doing things digitally,” Chris Cookson, president of Warner’s technical operations and chief technology officer, said of the archiving quandary.
One of the most perplexing realities of a digital production like “Superman Returns” is that it sometimes generates more storable material than conventional film, creating new questions about what to save. Such pile-ups can occur, for instance, when a director or cinematographer who no longer has to husband film stock simply allows cameras to remain running for long stretches while working out scenes.
Much of the resulting data may be no more worth saving that the misspellings and awkward phrases deleted from a newspaper reporter’s word-processing screen. Then again, a telling exchange between star and filmmaker might yield gold as a “special feature” on some future home-viewing format — so who wants to be responsible for tossing it into the digital dustbin?
For now, studios are saving as much of this digital ephemera as possible, storing it on tapes or drives in vaults not unlike those that house traditional film. But how much of that material will be migrated when technology shifts in 7 or 10 years is anyone’s guess. (And archiving practices in the independent film world run the gamut, from studied preservation to complete inattention, noted Andrew Maltz, director of the academy’s science and technology council.)
According to Mr. Shefter, a universal standard for storage technology would go far toward reducing a problem that would otherwise grow every time the geniuses who create digital hardware come up with something a little better than their last bit of wizardry.
As the report put it, “If we allow technological obsolescence to repeat itself, we are tied either to continuously increasing costs — or worse — the failure to save important assets.”
In other words, we could be watching Wallace Beery long after more contemporary images are gone.

 |  IP: Logged

Frank Angel
Film God

Posts: 5305
From: Brooklyn NY USA
Registered: Dec 1999


 - posted 12-23-2007 03:51 PM      Profile for Frank Angel   Author's Homepage   Email Frank Angel   Send New Private Message       Edit/Delete Post 
Hey, we've been saying that for years here. All the AMPAS had to do was read this forum for the last few years and they could have saved the cost of a big investigation.

quote:
only half of a collection of (DVD) disks can be expected to last for 15 years, not a reassuring prospect
Yah, surely not a reassuring prospect for the consumer who will be have to repurchase their NTSC DVD collection with degrading Blu-Ray or HD-DVDs -- which, btw, they bought to replace their Laserdisc collection, which they bought to replace their rotting Laserdisc collection, which they bought to replace their VHS collection!

Then again, figuring in the "imperative to save factor" of how few movies made today anyone would really truly miss 15 years out, the studios might actually come out ahead of the game. [Razz]

And just to throw a little fire on the copy protection issue, this is a damn good reason why the studios should figure out a benign way to let consumers make protection copies of the discs they buy. Along with the ubiquitous, pain-in-the-ass FBI warning, that warning should also include a statement that informs the OWNER of the disc that the law ALLOWS them to make a single protection copy (wouldn't THAT bring tears to your eyes). In fact, one would expect if the studios were playing fair, they should give a warning explaining that the life of the disc is expected to be about 10-15 years and it would be wise for the OWNER to make a LEGAL back-up copy, the way the software companies used to do years ago when you bought a program on a floppy disk. In many cases they gave instructions right in the setup instruction as to how to make a backup copy. Same deal here, but even more so as the industry itself now states publicly that the DVDs have a finite life, unlike their original claim in the courts when they stopped the authors of DVDCopy from selling their copy program. The MPAA claimed there was no need to make backup copies because the DIGITAL medium would LAST FOREVER. Lying scum. [thumbsdown]

 |  IP: Logged

Mike Blakesley
Film God

Posts: 12767
From: Forsyth, Montana
Registered: Jun 99


 - posted 12-23-2007 05:12 PM      Profile for Mike Blakesley   Author's Homepage   Email Mike Blakesley   Send New Private Message       Edit/Delete Post 
Why would DVDs only last 15 years? I've got 25 year old CDs that are perfect. I know DVDs have a lot more data, but still.

What about DVD-Rs and such?

 |  IP: Logged

Bobby Henderson
"Ask me about Trajan."

Posts: 10973
From: Lawton, OK, USA
Registered: Apr 2001


 - posted 12-23-2007 05:24 PM      Profile for Bobby Henderson   Email Bobby Henderson   Send New Private Message       Edit/Delete Post 
"Rot" can be a problem. The glue that bonds together data layers and such can degrade over time, sometimes even a short time. However, 15 years of shelf life is a pretty dubious claim.

I, too, have music CDs that are over 20 years old and still play just fine. Hopefully those DVDs will last just as well.

 |  IP: Logged

John Walsh
Film God

Posts: 2490
From: Connecticut, USA, Earth, Milky Way
Registered: Oct 1999


 - posted 12-23-2007 07:39 PM      Profile for John Walsh   Email John Walsh   Send New Private Message       Edit/Delete Post 
Opps.. I meant to post this in a different forum ...

Is there any way to tell how much 'digital rot' a CD has? I was wondering if the error correction in the player might fool a person into thinking their DCs are OK. When is the point where a CD will no longer play?

 |  IP: Logged

Bobby Henderson
"Ask me about Trajan."

Posts: 10973
From: Lawton, OK, USA
Registered: Apr 2001


 - posted 12-23-2007 09:38 PM      Profile for Bobby Henderson   Email Bobby Henderson   Send New Private Message       Edit/Delete Post 
Usually when laser rot occurs you can see it on the disc surface. Sometimes glue separation will look like a slightly milky spill under the clear plastic.

You can get the other extreme too. I've had a couple of DVDs that didn't have a single visible blemish to the surface but just would flat out not play anyway.

 |  IP: Logged

Paul Linfesty
Phenomenal Film Handler

Posts: 1383
From: Bakersfield, CA, USA
Registered: Nov 1999


 - posted 12-24-2007 04:20 PM      Profile for Paul Linfesty   Email Paul Linfesty   Send New Private Message       Edit/Delete Post 
Its very possible that the DVDs that are being talked about in the context of the article are the use of DVD-ROMS for long-term data storage (and the author may not have made the proper distinction). This has turned out to be a major problem. While, as Bobby points out, laser-rot can appear in stamped out CD's and DVD's (although increasingly rare, due to higher level of purity of materials used), the burned-type of DVD's (ROMS) have turned out to have a poor track record for use in archival purposes. Several studios were using the DVD-Rs (or + or RAM) for storage and have discovered an almost 40 percent destruction rate already! (This rather bothered me, as I have archived a lot of my video productions, as well as recorded lots of shows off my cable box, on DVD-Rs). I was hoping to be able to save watching them in my old age! (Of course, I just turned 51, so that puts me in that category now!!!)

 |  IP: Logged

Wayne Keyser
Master Film Handler

Posts: 272
From: Arlington, Virginia, USA
Registered: May 2004


 - posted 12-24-2007 10:48 PM      Profile for Wayne Keyser   Author's Homepage   Email Wayne Keyser       Edit/Delete Post 
Maybe I'm stupid, but ... umpty thousand PER YEAR to archive a picture in digital form? The cost must be coming from someplace I don't see.

Of course, it may ... but hey, considering just the feature, ancillary materials aside, for the sake of blue-sky figgerin':

Let's say four copies on four hard drives in four locations ... throw in a dvd-rom or two with the same material in the same box with each hard drive, just in case. Don't trust to the wisdom of the major companies, make every movie deal include a pittance, say $40k, to be invested and the proceeds (maybe 5% annually) to fund an annual or biannual swapout to four new drives and four new sets of discs. The cost to store four shoebox-sized units, per year ... well, what does a bank safety-deposit box rent for? Administrative costs and annual swapout costs, modest, certainly well within the foreseeable profits of the "preservation cost" endowment, especially if that kind of money drove an enterprise solution like a digital archive service that just kept records and performed the yearly "refresh."

What have I missed?

 |  IP: Logged

Allison Parsons
Jedi Master Film Handler

Posts: 630
From: East Peoria, IL
Registered: Oct 2004


 - posted 12-25-2007 02:28 AM      Profile for Allison Parsons   Author's Homepage   Email Allison Parsons   Send New Private Message       Edit/Delete Post 
quote: Bobby Henderson
"Rot" can be a problem
Ask any Laserdisc owner!

 |  IP: Logged

Cameron Glendinning
Jedi Master Film Handler

Posts: 845
From: West Ryde, Sydney, NSW Australia
Registered: Dec 2005


 - posted 12-25-2007 03:44 AM      Profile for Cameron Glendinning   Email Cameron Glendinning   Send New Private Message       Edit/Delete Post 
quote: Wayne Keyser
What have I missed?


Good question and I am only guessing an answer, but labour would be a big factor, plus managment, plus higher rents due to the fact that people are now required. I guess that throwing it down an abandoned salt mine in the middle of know where is a really really cheap, on a long term basis for film but I doubt that they can do anything like that with digital.

Anyway as far as the high figures, it must involve hollywood style acountants doing the bookwork! Only they could possibly explain why some movies cost 200 million to make.

 |  IP: Logged

Frank Angel
Film God

Posts: 5305
From: Brooklyn NY USA
Registered: Dec 1999


 - posted 12-25-2007 01:34 PM      Profile for Frank Angel   Author's Homepage   Email Frank Angel   Send New Private Message       Edit/Delete Post 
The problem is that, unlike Laserdiscs which are analog and which, when disc rot began you could easily see the picture slowly deteriorating, CDs and DVDs are digital and the system is replete with error correction. This ec process will allow a disk which as already begun to degrade, to play quite flawlessly.

The owner gets no warning signs that he's about to have a catastrophic lose of data. When the error correction fails, the disk becomes unreadable or so badly distorted as to be useless. But the time those ON and OFF bits are no longer distinguishable, it's too late to worry about making that back up...the horse is already out of the barn. That's why I say that along with all those damn antipiracy warnings, the studios should also instruct the purchaser of their product, how to make a LEGAL protection copy so that in the even of a failure, either by reason of a degradation of the medium or by accidental damage, a backup copy will exist.

There is a precident for this by copyright owners of computer software, which is no less copyright protected than motion pictures or music, and which instructions for a single backup copy was always given with the software installation. The studios should get off their high horse (which they have already beaten half to death) and do the same. Consumer protection advocates should hound them till they do.

 |  IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99


 - posted 12-25-2007 06:31 PM      Profile for Mark Gulbrandsen   Email Mark Gulbrandsen   Send New Private Message       Edit/Delete Post 
Thais rediclous! They can just store B&W seps like they've always done.

Mark

 |  IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999


 - posted 12-25-2007 08:11 PM      Profile for Steve Guttag   Email Steve Guttag   Send New Private Message       Edit/Delete Post 
In fact, duping digital to film for archiving is indeed a booming industry.

However, one thing this article points out is something that most home-movie people have had to contend with. Back in the 8mm (or 16mm, depending on your era), you had a relatively short load with film and as such you always conserved your shots. Thus most of what you shot is something that you would at least consider to put in your final edited reel to show to the family. When Video taping came about...people just let that puppy roll and roll and roll and roll....heck there was at least 2-hours available!

With features, a similar thing has occurred. Studios are NOW accustomed to saving everything just in case one needs/wants to re-edit the directors final last ultimate cut (e.g. Blade Runner) or to use for other purposes. With film, you just store the whole lot. With video, there is apparantly MUCH more stuff and nobody that will decide what will stay and what will go. Do you dupe EVERYTHING to film on the possibility it MIGHT be used at a latter date. The cost of that grows geometrically. You have the initial cost of the dupe...then you have the upkeep cost of keeping the drives in good shape that are in storage and verifying that all is in good. Then on any technology change (we don't use 8" floppies anymore, or 5-1/4 or really 3-1/2...SCSCI drives are out....etc and then there are all of the file systems that could be in use or operating systems that can access the data...so there needs to be a planned migration of the digital media...that as more titles come about begets that many more drives that have to be kept up with. That is much worse than just throwing the lot in a cold-dry storage facility and deal with it when you want it. The archivists really seem to think they can keep film in good shape for many hundreds if not a 1000 years with current storage practices.

Steve

 |  IP: Logged

Wayne Keyser
Master Film Handler

Posts: 272
From: Arlington, Virginia, USA
Registered: May 2004


 - posted 12-25-2007 11:00 PM      Profile for Wayne Keyser   Author's Homepage   Email Wayne Keyser       Edit/Delete Post 
Were the industry (or a large part of it) to adopt a system such as I proposed above, the amount of money I proposed, with its annual earnings, would IMO be more than enough to support one or more businesses whose sole task is to do the following:

Retrieve the stored hard drives every year or two years, transfer the data to their computers (checking the transfer for errors and cross-checking with the other archive sites), and storing it again on new hard drives (of whatever advanced style comes along in the future) and new discs. SCSI drives are old-hat now, but I'm still running one on my second computer - just because new drives have come along doesn't mean the old style won't be usable on systems designed for it for a couple of years, then the data gets copied to new styles. Such businesses could be run "light on staff" with economies of volume.

But I feel the ring of truth behind the suggestion that "Hollywood style accountants" are behind the high figure they estimate. I once took a class on production and was surprised by the emphasis on the need to watch the accountants closely - the line between "above the line" and "below the line" expenses has made the differences between profit and loss for many films and many companies.

 |  IP: Logged

John Hawkinson
Film God

Posts: 2273
From: Cambridge, MA, USA
Registered: Feb 2002


 - posted 12-25-2007 11:30 PM      Profile for John Hawkinson   Email John Hawkinson   Send New Private Message       Edit/Delete Post 
Do people really consider 4 hard drives sitting in 4 closets to be reasonable? I don't think that I would. If I were doing this, I'd expect the data to be on multiple live (on-line) RAID systems in seperate geographical locations. And then the costs sound reasonable.

It'll be interesting to see the AMPAS report when it becomes public. It was released privately in November and there were discussions on the AMIA list where it was indicated it would be released publicly after some period of time, and there were also articles in Variety and other trade press.

--jhawk

 |  IP: Logged



All times are Central (GMT -6:00)
This topic comprises 2 pages: 1  2 
 
   Close Topic    Move Topic    Delete Topic    next oldest topic   next newest topic
 - Printer-friendly view of this topic
Hop To:



Powered by Infopop Corporation
UBB.classicTM 6.3.1.2

The Film-Tech Forums are designed for various members related to the cinema industry to express their opinions, viewpoints and testimonials on various products, services and events based upon speculation, personal knowledge and factual information through use, therefore all views represented here allow no liability upon the publishers of this web site and the owners of said views assume no liability for any ill will resulting from these postings. The posts made here are for educational as well as entertainment purposes and as such anyone viewing this portion of the website must accept these views as statements of the author of that opinion and agrees to release the authors from any and all liability.

© 1999-2020 Film-Tech Cinema Systems, LLC. All rights reserved.