Film-Tech Cinema Systems
Film-Tech Forum ARCHIVE


  
my profile | my password | search | faq & rules | forum home
  next oldest topic   next newest topic
» Film-Tech Forum ARCHIVE   » Operations   » Digital Cinema Forum   » Dolby DSS 200 Array Degraded and Interrupted Playback (Page 1)

 
This topic comprises 4 pages: 1  2  3  4 
 
Author Topic: Dolby DSS 200 Array Degraded and Interrupted Playback
Mohd Ridzuan
Film Handler

Posts: 25
From: K.L , Malaysia
Registered: Jul 2012


 - posted 12-10-2013 02:27 PM      Profile for Mohd Ridzuan   Email Mohd Ridzuan   Send New Private Message       Edit/Delete Post 
Hello guys, first of all thanks to FT because i learn so much thing in this forum. But still there were many other things i need to learn and all of your help is very appreciates.

I had a problem with my Dolby dss 200 + NEC 2500s. There were 2 warning error messages appear.

1. The array was degraded. I try to rebuild the hdd and successful. Delete the the content and reingest back. However after one show the hdd became degraged again. Ther hdd is new one. Just replace it this year about 6 months. Is it possiblem the hdd can be degraded within a short periods.

2. There was warning Interrupted playback. In the middle of the show the playback got stuck and freeze. Then we pull out the degraded hdd, we can run the show smoothly. But if we insert the hdd back the show get stucks.

Anyone can help me what cause this problem and what necessaries step to solved this problem.

Thanks

 |  IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007


 - posted 12-10-2013 02:44 PM      Profile for Marco Giustini   Email Marco Giustini   Send New Private Message       Edit/Delete Post 
Check how many reallocated sectors that HDD has (SYSTEM-THEATRE DEVICES). More than a few is bad, I bet you'll find a lot.

Anyway, an HDD can become faulty after a few months, if it's new Dolby will replace it for you. The ARRAY degrades (it's an IT term, it's not physically degrading!), that means that one HDD is possibly bad.

You shouldn't need to delete and re-ingest any content.

Bottom line, just replace the disk.

 |  IP: Logged

Mohd Ridzuan
Film Handler

Posts: 25
From: K.L , Malaysia
Registered: Jul 2012


 - posted 12-10-2013 06:59 PM      Profile for Mohd Ridzuan   Email Mohd Ridzuan   Send New Private Message       Edit/Delete Post 
Thanks Marco. What cause the HDD could be degraded? Is it cause by corrupted files? I had this problem before, but after doing rebuilding the RAID the problem solved. In this case its not. [Frown]

 |  IP: Logged

Manny Knowles
"What are these things and WHY are they BLUE???"

Posts: 4247
From: Bloomington, IN, USA
Registered: Feb 2002


 - posted 12-10-2013 07:36 PM      Profile for Manny Knowles   Email Manny Knowles   Send New Private Message       Edit/Delete Post 
A DSS200 is capable of functioning even if one hard drive goes bad. I know this because I have run shows with a degraded array, due to one faulty drive.

If your array is degraded to the point where you are having failures, then you might have more than one dead drive (or, possibly, some other problem that I haven't encountered).

My first degradation (and my only one, so far) occurred during the warranty period. So, yes, hard drives can die young.

When you replace the bad drive(s), do yourself a favor and order an extra one to keep on-hand as a spare. Steve Guttag made sure we were provided with a spare drive as part of our package, and I was glad to have it ready when it was needed. (Thanks, Steve!) The dead drive was replaced by Dolby, so that is now the spare drive.

 |  IP: Logged

Mohd Ridzuan
Film Handler

Posts: 25
From: K.L , Malaysia
Registered: Jul 2012


 - posted 12-10-2013 09:22 PM      Profile for Mohd Ridzuan   Email Mohd Ridzuan   Send New Private Message       Edit/Delete Post 
Just been inform by operator on site, from 4 HDD now 2 of them had degraded. Now they playing with only 2 HDD in the slot as they had pull out the 2 degraded HDD. Both degraded HDD had to pull out because if its was inside it will interrupt the show became stuck n freeze. My question is, is it possible to play with just 2 HDD? and how long it will stay?

p/s: waiting for the replacement HDD

 |  IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007


 - posted 12-11-2013 03:54 AM      Profile for Marco Giustini   Email Marco Giustini   Send New Private Message       Edit/Delete Post 
No, a DSS system won't play with only two drives, your operator must be mistaken.

Probably the system was badly handled when being shipped. An array degrades when a disk fails. It could degrade because the disk has a temporary glitch - always worth try to rebuild the array if the disk does not show any - or few - bad sectors before replacing the disk - but in general that means you have a bad disk. Check the reallocated sectors count as I suggested.

RAID stands for Redundant Array of Independent Disks. Basically if you have 4 HDD of 1TB each the system will write the information on three disks only; the fourth will be use as a parity disk. That is, say disk 1-2-3 are the data disks and disk 4 is the parity one, if disk 2 fails the system can extrapolate the missing information using the data contained in disk 4 (it does not work like that but just to give you an idea). Your storage capacity of the RAID will be only 3TB.

If ONE drive fails, the system is still perfectly capable of accessing all the information - one disk will also be redundant. If TWO drives fail, the information on your RAID will be lost permanently.

When that happen you'll have to replace the two drives and reinstall the OS on the system. You'll lose your content and some of your settings.

When a disk fails the RAID controller should be capable of assessing that and should disconnect the faulty disk before it causes trouble but that doesn't happen all the time and you may end up with an interrupted show before the controller takes the disk out of service.

Check the number of reallocated sectors on all your servers, replace anything with more than 10 bad sectors (this is what Dolby recommends, from my point of view a brand new drive should show ZERO bad sectors). If servers are brand new Dolby will pay for them.

When you replace a drive the system will realise that one drive does not contain any information and will start the "rebuild" process which is recreating the data on the new drive based on the data on the other three drives. The process will take several hours. Can be done during playback but it's not recommended, better to replace a drive early in the morning or last thing in the night.

 |  IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999


 - posted 12-11-2013 08:51 AM      Profile for Steve Guttag   Email Steve Guttag   Send New Private Message       Edit/Delete Post 
Close on the RAID description but not quite...All drives carry parity for the other drives as well as "their" data. Thus, drive one will have parity on drives 2, 3 and 4. There isn't a parity drive. The data is striped across all four drives (there are 3-drive RAID 5 but the same principles apply).

Perhaps you are thinking of a RAID array where a "hot spare" is in the frame ready to come on line if one drive craps out. Dolby does not have a hot-spare...all 4 drives work all the time. I believe what you were describing would more closely resemble a RAID 4...which isn't used in DCinema, as far as I know.

Other than that you are spot on. I'm a little more lax on how many reallocated sectors I'll allow...generally if I see less than 100, I'll let it fly. What I'm more interested in is how fast that number climbs. If it was 5 yesterday and 25 today...get it out of there FAST!

The other thing to check is in the logs and see how many errors, if any are showing as past events on the disks. I'm more strict on those because those are hidden unless you specifically look for them. You might have a drive that is having errors and then another drive degrades the Array and it can't rebuild because you have one with errors on it preventing the rebuild...I've had that happen.

Note, these principles apply to ALL of the servers, not just Dolby's It is a good idea to check the drives once or twice a year to see if anything is brewing. It is much easier to change drives out one at a time than to go through the whole re-initialization and loading content back on.

 |  IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007


 - posted 12-11-2013 12:32 PM      Profile for Marco Giustini   Email Marco Giustini   Send New Private Message       Edit/Delete Post 
Steve,

You're correct, but I also said
quote: Marco Giustini
it does not work like that but just to give you an idea
[Smile]
I just wanted to simplify!

I agree that you may end up with a disk with the same 50 bad sectors for years but if a drive is brand new I wouldn't accept anything more than zero. On a more aged one I may be a bit more relaxed as long as - as you say - the number does not quickly increase.
Dolby do not warn the customer of bad sectors and I've seen servers before ending with two bad HDDs in an endless rebuilding process. At that time it's usually too late and a full reinstall is required.

 |  IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999


 - posted 12-11-2013 01:37 PM      Profile for Steve Guttag   Email Steve Guttag   Send New Private Message       Edit/Delete Post 
On a new drive...I would agree that zero should be the case...though I don't mind seeing one.

But at what point do you stop declaring a drive new?

I have noted that those theatres that shut the servers down at night, the reallocated sector counts tend to be higher.

 |  IP: Logged

Gordon McLeod
Film God

Posts: 9532
From: Toronto Ontario Canada
Registered: Jun 99


 - posted 12-11-2013 01:53 PM      Profile for Gordon McLeod   Email Gordon McLeod   Send New Private Message       Edit/Delete Post 
We also use only enterprise grade drives
some of the consumer ones have issues especially the green ones as they go to sleep

 |  IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007


 - posted 12-11-2013 01:57 PM      Profile for Marco Giustini   Email Marco Giustini   Send New Private Message       Edit/Delete Post 
It's food for thought. I would say that I'd like to see zero bad sectors on a out-of-the-box machine and for at least a month or two. Then I would have the manufacturer to replace a drive which shows a number of bad sectors high enough to allow warranty claim. If Dolby say 10 (I believe it's on their knowledge base) I would replace a drive showing 11 bad sectors straight away while under warranty. After the warranty I'd do as you say, keep an eye on it.

But I'd rather replace a 3y old drive showing 40 bad sectors before the customer experiences troubles with it, even though that drive could potentially last for another 3 years.

 |  IP: Logged

Mohd Ridzuan
Film Handler

Posts: 25
From: K.L , Malaysia
Registered: Jul 2012


 - posted 12-11-2013 06:21 PM      Profile for Mohd Ridzuan   Email Mohd Ridzuan   Send New Private Message       Edit/Delete Post 
Thanks to Steve n Marco.

Steve you were right our server was shutdown every night after the session of the night. But we had another 3 halls that had no problem until now almost 2 years. Marco, this server was a new set, just been installed about 2 weeks back replace the old server which got mother board faulty. After 1 weeks using it this problem occurs.

My updates for this issue. Last night my operator said they now running without any hdd inside as we had a TMS system suggested by our installer. Is it possible? is there any incoming problem running the server without hdd? Our installer said try to reformat the hdd. If cannot then sent to Dolby for replacement.

Can anyone teach me how to reformat the hdd? If reformat failed, what is the step if i get a new replacement hdd. Just slot in the server and it will format n rebuild the RAID itself?

Thanks guys, i'm just a newbie...

 |  IP: Logged

Monte L Fullmer
Film God

Posts: 8367
From: Nampa, Idaho, USA
Registered: Nov 2004


 - posted 12-12-2013 04:18 AM      Profile for Monte L Fullmer   Email Monte L Fullmer   Send New Private Message       Edit/Delete Post 
Pull/disconnect the two power plugs from time to time to clear out stale memory.

 |  IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007


 - posted 12-12-2013 06:37 AM      Profile for Marco Giustini   Email Marco Giustini   Send New Private Message       Edit/Delete Post 
Mohd,

You probably misunderstood, a server cannot run without HDDs. If that was what your projectionist and installer actually suggested... get rid of them, quickly! [Smile]

I wouldn't bother for a second, if your disk is showing bad sectors - did you check that? - I would ask for a replacement straight away. Your server is 2 weeks old!

Turning the servers off at night won't hurt them in the short term. It's proved that on the long term your HDDs won't like it and will fail sooner than when the servers are on 24/7. But at this moment this is not the cause of your problem.

Anyway, if a disk is showing bad sectors you cannot cure them. You can rebuild the array if it says "degraded". It should happen automatically, or just go under SYSTEM/Server tools and click on REBUILD. To be on the safe side, do it overnight, after the last show.

If the array is NOT degraded, it does not need to be rebuilt.
If any of the HDDs is showing bad sectors, replace it.

 |  IP: Logged

Marcel Birgelen
Film God

Posts: 3357
From: Maastricht, Limburg, Netherlands
Registered: Feb 2012


 - posted 12-12-2013 02:59 PM      Profile for Marcel Birgelen   Email Marcel Birgelen   Send New Private Message       Edit/Delete Post 
quote: Monte L Fullmer
Pull/disconnect the two power plugs from time to time to clear out stale memory.
No modern computer system needs to be unplugged to "clear stale memory" and it also won't fix any degraded or even failed RAID.

I also totally agree with Steve on this one, the rate in which bad sectors are increasing is by far the most important number. Having a few bad sectors on a drive isn't an issue, as long as they're not increasing at a fast rate. If you replace an otherwise well performing drive with a new one, you never know what you get in return.

 |  IP: Logged



All times are Central (GMT -6:00)
This topic comprises 4 pages: 1  2  3  4 
 
   Close Topic    Move Topic    Delete Topic    next oldest topic   next newest topic
 - Printer-friendly view of this topic
Hop To:



Powered by Infopop Corporation
UBB.classicTM 6.3.1.2

The Film-Tech Forums are designed for various members related to the cinema industry to express their opinions, viewpoints and testimonials on various products, services and events based upon speculation, personal knowledge and factual information through use, therefore all views represented here allow no liability upon the publishers of this web site and the owners of said views assume no liability for any ill will resulting from these postings. The posts made here are for educational as well as entertainment purposes and as such anyone viewing this portion of the website must accept these views as statements of the author of that opinion and agrees to release the authors from any and all liability.

© 1999-2020 Film-Tech Cinema Systems, LLC. All rights reserved.