"Elvis" DSS220 ingestion bug?

[Leo Enticknap]

One of my service customers has reported to me that ingestion of Elvis stalled at 12% / 27 GB on multiple DSS220s (using a CRU boot hooked via eSATA). Their TMS is currently broken (motherboard failed; currently awaiting a new one), so they're having to ingest at the SMS.

I looked at the logs, and saw this at around the time the failed ingest was taking place:

bad_character_ingest_bug.jpg

I have a dim memory of a movie that came out around 2-3 years ago, in which an unusual character in the CPL name caused DSS servers to refuse to ingest it. Could something similar be happening here? I certainly don't see any unusual characters in the actual name, but I guess that the problem could be inside one of the DCP metadata files. Has anyone managed to ingest Elvis into a DSS server successfully (these are on 4.9.5.2)?

I've asked the theater to try ingesting into one of their non-DSS servers (the complex consists of nine DSS220/cat745s, an IMS2000, and an ICMP-X), and will post the result of that as soon as I have it. If that hangs in the same place, we're likely looking at a bad shipping drive, but if it ingests successfully, I'm wondering if there is a bad interaction between this CPL and a bug in the DSS software.

[Jason Sharp]

It ingested fine onto our DSL100 via FTP from the DCDC server. I did not test ingesting from a hard drive.

[Jim Cassedy]

I've just been asked by a co-worker to investigate why she can't seem to ingest ELVIS into one of our SONY XCT-10
digitalserverthingys. Content came by satellite, & into our Library Server OK- - and from there, it was ingested, via
TMS, into 3 other identical servers without any problems. But ingestion into one of our auditoriums is 'hung' and the
TMS is displaying this error message:

"CANT INGEST DCP USED BY SPL BECAUSE THE DCP IS NOT SUPPORTED AND NOT PLAYABLE
IN THE TARGET AUDITORIUM'

I'm in the middle of running a 35mm show, so I can't investigate at the moment, but it is odd that we're only
having the issue on one server. (and they're all provisioned identically and all using the exact same playlist
lineup of stingers & trailers)

ElvisIngestIssue.jpg




- -

[Marcel Birgelen]

Someone probably has used a 16-bit unicode character in a CPL in some kind of attribute where this is unexpected by the server's software. I guess it's also important to inform the distributor, because if sufficient people complain, they'll be more enticed to rectify the problem. Furthermore, this should also be reported to Dolby, even though the affected products may be eol.

[Leo Enticknap]

My customer reported that it wouldn't ingest into the IMS2000 or the Alchemy either, and has asked Deluxe for a replacement drive. Hopefully this is just a bad drive, because if there is a bug in this CPL that is preventing it from ingesting into three widely used models of server, the next couple of days are going to be interesting, to put it mildly.

Incidentally, I now remember the previous incident: it was Hobbs & Shaw, and the Dolby TMS didn't like the ampersand in the CPL name. Dolby patched that bug in the following (and, as it turned out, final) software update.

[Jim Cassedy]

Marcel said:

Someone probably has used a 16-bit unicode character in a CPL in
some kind of attribute where this is unexpected by the server's software.

But Marcel- - My point of confusion is why 3 other identical systems in the same
building are not having this issue. They're all using the same content files and
using the same show playlist.

[Carsten Kurz]

Jim - can you try a local ingest on that particular XCT-S10? Looks like a TMS issue to me.

[Jim Cassedy]

Thanks Carsten- I had thought of that last night. Our content arrived via satellite, and I had thought
of bypassing the TMS & either ingesting directly to the SONY from the library server, either through
the network or by dumping a copy from the library to a physical hard drive and ingesting locally from
that. But 1) I was running 35mm (change-overs) last nite, so my time to give this undivided
attention was limited, and I wouldn't have been able to start working on this till after midnite
2) There was a late show up in the auditorium with the ingestion issue, so even if I had been
able to start a local ingest, it wouldn't have finished till almost 2am ~ and I'd already worked
a full day and had to be somewhere early this morning.

So, in the end, I've documented everything and escalated it to the tech-team at the organization
I work for, and am waiting to see what they come up with. ELVIS doesn't play in that auditorium
until Friday afternoon, so there's still well more than 24hrs to get this figured out. (Or to re-arrange
those shows into other auditoriums that aren't having issues) But it's 'head scratcher' at this point.


Today's Fun Foto:
" ELFIS PRESLEY "
ElfisPresley.jpg

[Leo Enticknap]

But Marcel- - My point of confusion is why 3 other identical systems in the same building are not having this issue.

Is the problematic server on the same software/firmware version(s) as the ones that are OK?

[Harold Hallikainen]

The original log posted by Leo flagged the  HTML entity. That is an EOT (end of transmission) character that should not be in the XML. Different servers MIGHT just discard it, while others might throw an error. It would be interesting to know the software revisions of the same model that generates an error versus accepts the file. It's too bad the log does not show which file has the issue.

Good luck!

Harold

[Jim Cassedy]

Update: First of all, just for the record, YES, all the servers were identical & running
the same software version, except for the server one auditorium where we use dual
projectors to throw extra light on a very large screen, so that one has a slightly different
software version to do that. All the others are identical. I mentioned that several times.

HOWEVER: Our problem turned out to be something totally unrelated to the ELVIS DCP.

Although all the servers were running the same ELVIS content & show playlist, there was
ONE "Elvis" show, in one auditorium, that was part of a private event, which had a slightly
different trailer playlist associated with it.
> The person who built that playlist had inadvertently added a 3D clip in that SPL.
That auditorium is not equipped for 3D and so that's why the server rejected it.
(see the exact error message in my original post) Unfortunately, if a playlist is missing
an expected clip, or one is corrupted, the SONY TMS only gives an "Ingest DCP" error
message, but doesn't identify WHICH CPL is missing or corrupted. You've got to figure
that out for yourself. I didn't make that playlist, and couldn't take a look at the problem
until around 1am, after working a full day & also running a 35mm change-over show till
around midnite. So-- I wasn't quite awake and didn't catch the content error when I looked
before I went home. The person who did the initial programming was going to delete
everything & start over on that playlist in that auditorium when she came in on Thurs
morning- - and she spotted the errant 3D content problem.

So- that's why WE had an Elvis issue at my place, It would be interesting to know why
some other venues Leo mentioned were having problems.

[Leo Enticknap]

Thankfully, my case turned out to be simply a bad drive. I guess it was the memory of the Hobbs & Shaw incident that rang an alarm bell in the back of my mind.