QNAP NAS security vulnerabilities and the IMS3000

[Leo Enticknap]

Just stumbled across this article. This passage in particular caught my eye:

QNAP's firmware push was intended, in part, to cover recent security vulnerabilities in their devices. QNAP devices are a rich and frequent target of criminal hackers. A severe vulnerability from February 2023 allowed for remote SQL injections and potential administrative control of a device, affecting nearly 30,000 devices seen in network scans. It was a follow-on from attacks by DeadBolt, a ransomware gang that infected thousands of QNAP devices and cornered QNAP into automatically pushing emergency updates, even to customers with automatic updates turned off.

Security researchers at WatchTowr said they found 15 vulnerabilities in QNAP's operating systems and cloud services and informed the company of them. After QNAP failed to patch some of those vulnerabilities far beyond the typical 90-day window (and then some), WatchTowr went public with its findings, dubbed "QNAPping at the Wheel."

The NAS that Dolby offers for use with the IMS3000 for use cases where more local storage is needed than 3 x Dolby-approved drives in the internal RAID can provide is a QNAP model. Where this is relevant to the security concerns mentioned above is that the software customization Dolby adds to it means that these machines are stuck on a pretty old version of the base firmware: you can't upgrade it to current, because doing so will break the Dolby customization, and thus its ability to stream audio and video into the IMS3000.

So it appears that there is no way to avoid leaving this device in a vulnerable condition, especially if it has a gateway specified in the IPv4 settings, and access to the Internet. So the takeaway is that we need to flag up to customers for whom we install the IMS3000 with NAS option that they need either to leave it completely disconnected from the Internet, or have enterprise grade firewall and malware protection between it and the Internet.

[Frank Cox]

On the other hand there's this:

https://www.theregister.com/2024/11/...faulty_update/

Owners of QNAP network-attached storage (NAS) boxes are finding that a firmware update has left them unable to log into their device, and a reset doesn't seem to fix the issue.

The Taiwan-based storage biz specializes in NAS kit and offers a whole portfolio of models to address various needs. However, users are complaining of issues following a firmware release that went out to some products last week.

According to posts on the company's community forums, the update in question is QTS 5.2.2.2950, build 20241114. QTS is the firm's Linux-based operating system for its entry-level and mid-range products.

The firmware upgrade was removed for some models sometime after it was released, yet users are contiuing to gripe that QNAP has failed to disclose which models were affected by the errant update.

"I thought I had a problem with my QNAP, so I used three second reset and now I can't log in at all," one customer complained, who added they still see an error message saying: "Your login credentials are incorrect or account is no longer valid."

The user explained they have two identical TS-653D NAS servers. "Because I can no longer get to either machine, I have completely reset one by holding the reset button for 10 seconds and after two beeps released. This has been completely reset as I can see in Qfinder [QNAP's desktop tool], only I still cannot access it with 'admin' and The Cloud Key Password."

A seemingly more tech-savvy user revealed: "I have raised this with QNAP, but so far the devs/support are silent. Not even any guidance to any possible issues."

Another user said: "It is also available as an update for my TS-453D in AMIZ. But AMIZ is not able to apply it." AMIZcloud is a SaaS tool for deploying, managing, and monitoring QNAP devices.

In response to our queries, a QNAP spokesperson told us: "We recently released the QTS 5.2.2.2950 build 20241114 operating system update and received feedback from some users reporting issues with device functionality after installation.

"In response, QNAP promptly withdrew the operating system update, conducted a comprehensive investigation, and re-released a stable version of QTS 5.2.2.2950 build 20241114 within 24 hours."

I wonder if this is an issue where the box is hidden away behind a NAT. Unless there's some kind of automatic remote tunneling capability or I'm missing some other factor, bad actors shouldn't be able to contact it under those circumstances.

[Marco Giustini]

Cinema servers often run on very old Linux kernels, they're not supposed to be exposed to the internet without a firewall, you're not browsing the internet on them etc. I don't think those warning would apply - unless you ask your router to give them free, unprotected access to the internet.

[Joseph Boutros]

Over the past two years, multiple critical vulnerabilities have been identified and reported in various QNAP operating system versions.
These vulnerabilities potentially expose affected systems to security risks such as unauthorised access, data breaches, or system compromises.

https://www.qnap.com/en/security-advisory/qsa-24-09
https://www.qnap.com/en/security-advisory/qsa-24-32
https://www.qnap.com/en/security-advisory/qsa-23-01
https://www.qnap.com/en/security-advisory/qsa-23-57

However, we are pleased to confirm that the custom firmware utilised on QNAP NAS sold by Dolby with the IMS3000 remains unaffected by these vulnerabilities:

QTS 4.3.3.0408 (Build 20171217)
QTS 4.3.3.1994 (Build 20220421)

Our dedicated security team has thoroughly assessed these firmware versions and verified their integrity against the reported vulnerabilities.
Users of these specific firmware versions can continue to operate their systems with confidence, knowing that they are protected against the recently discovered operating system weaknesses.

If you have any further questions or concerns, please contact the Dolby Support Services team on CinemaSupport@dolby.com

Dolby remains committed to maintaining the highest standards of security for our products and will continue to monitor and respond to emerging threats proactively.

Dolby Support Services

[Steve Guttag]

Thanks Joseph!

Note, my media networks never have had direct internet access or gateways. They are connected to things that have internet access (TMS/LMS and content catch servers).

[Leo Enticknap]

Cinema servers often run on very old Linux kernels, they're not supposed to be exposed to the internet without a firewall, you're not browsing the internet on them etc. I don't think those warning would apply - unless you ask your router to give them free, unprotected access to the internet.

You'd be surprised how many screening venues want to do that - not so much chain sites or organizations with a dedicated IT management infrastructure (e.g. larger post houses and university campuses), but independent theaters and residence screening rooms are a high risk. Their people typically want access from home (especially Bel-Air Circuit screening rooms, content ingestion to which is usually now done over the Internet), and often expose the booth LAN to the Internet through an old and infrequently (if ever) updated consumer grade router/gateway.

Good to hear from Dolby that their QNAP software/firmware package is not affected by the vulnerabilities mentioned in that article. The IMS3000 itself (running version 3.5.20), however, does have vulnerabilities. I was told that the infected boot flash drive from that unit had been sent to the development team for analysis after I shipped it to Burbank, but no updates or hotfixes seem to have appeared as yet.

[Marco Giustini]

A router should be enough, as long as there is NAT.
What I meant is someone giving DIRECT access to the internet, such as opening the FTP port for direct remote access (that is, without using VPN or some other services). I'm guilty of doing that myself once, my up-to-date NAS was immediately compromised

[Joseph Boutros]

Good to hear from Dolby that their QNAP software/firmware package is not affected by the vulnerabilities mentioned in that article. The IMS3000 itself (running version 3.5.20), however, does have vulnerabilities. I was told that the infected boot flash drive from that unit had been sent to the development team for analysis after I shipped it to Burbank, but no updates or hotfixes seem to have appeared as yet.

We are pleased to inform you that our comprehensive investigation has concluded. We can confidently assert that the system's integrity remains intact, and no breach occurred. At most, the system was inadvertently utilized as temporary storage for potentially malicious files.

Our thorough examination revealed that an individual, operating from a local IP address, gained access to the IMS3000 using default root credentials. This user then proceeded to copy three Windows-based trojan files into a specific directory. It is crucial to emphasize that these files pose no threat to the IMS3000 and are incapable of execution on our platform.

Historically, we have maintained the same root password to streamline the installation process. However, we recognize the importance of robust security measures. Both our Web User Interface and manual explicitly advise users to modify the default passwords, particularly when deploying the system in non-controlled environments.

If you have any further questions or concerns, please contact the Dolby Support Services team on CinemaSupport@dolby.com

Dolby remains committed to maintaining the highest standards of security for our products and will continue to monitor and respond to emerging threats proactively.

Dolby Support Services

[Steve Guttag]

In fact, the UI will flag the system for still using default passwords.

[Leo Enticknap]

Our thorough examination revealed that an individual, operating from a local IP address, gained access to the IMS3000 using default root credentials. This user then proceeded to copy three Windows-based trojan files into a specific directory. It is crucial to emphasize that these files pose no threat to the IMS3000 and are incapable of execution on our platform.

That would explain why I found Windows malware executables in the backup package (assuming that the specific directory into which the Windows Trojans were copied is one that is captured in the backup package), but not the "out of memory" errors that showed up in the log (unless it meant "memory" as in space on the boot flash drive as distinct from RAM), the fact that the IT department of the university in which this IMS3000 was operating claimed that attempts to contact known bad actor IP addresses were originating from the IMS3000's address (which could not happen if the only malware involved in this incident could only run on Windows), and the fact that both symptoms went away after replacing the boot flash drive. Unless I'm missing something, those can only be explained by the IMS3000 itself running unwanted code.

The factory default credentials for the IMS3000 are the same as those for all other DolReMi products, going right back to the DCP2000. So they've been around for almost two decades, meaning that they have very likely been leaked onto the public Internet somewhere, and that therefore automated bots will be trying them. Factory default credentials are another growing issue when it comes to installing and maintaining digital cinema equipment. A tiny minority of customers I've dealt with (ironically, most of them in higher education) have a policy of changing them, but most actively don't want to do that. The residential theater circuit, post houses, and independent theaters all have people working for them who have known these credentials for ages, and for convenience's sake do not want them changed. I think I need to start a discussion within our company as to whether or not we should now actively advise that end users do change these passwords during an install, but I know that we will meet resistance if we do.

[Joseph Boutros]

That would explain why I found Windows malware executables in the backup package (assuming that the specific directory into which the Windows Trojans were copied is one that is captured in the backup package), but not the "out of memory" errors that showed up in the log (unless it meant "memory" as in space on the boot flash drive as distinct from RAM), the fact that the IT department of the university in which this IMS3000 was operating claimed that attempts to contact known bad actor IP addresses were originating from the IMS3000's address (which could not happen if the only malware involved in this incident could only run on Windows), and the fact that both symptoms went away after replacing the boot flash drive. Unless I'm missing something, those can only be explained by the IMS3000 itself running unwanted code.

The factory default credentials for the IMS3000 are the same as those for all other DolReMi products, going right back to the DCP2000. So they've been around for almost two decades, meaning that they have very likely been leaked onto the public Internet somewhere, and that therefore automated bots will be trying them. Factory default credentials are another growing issue when it comes to installing and maintaining digital cinema equipment. A tiny minority of customers I've dealt with (ironically, most of them in higher education) have a policy of changing them, but most actively don't want to do that. The residential theater circuit, post houses, and independent theaters all have people working for them who have known these credentials for ages, and for convenience's sake do not want them changed. I think I need to start a discussion within our company as to whether or not we should now actively advise that end users do change these passwords during an install, but I know that we will meet resistance if we do.

During our investigation, we encountered a similar issue. The files in question were located in the /doremi/etc/snmp directory, which is included in both the system backup and the detailed log report.

Regarding the other topics raised, I suggest we continue our discussion within our case management portal, as there are certain facts that I can not discuss on a public forum. In addition, this approach ensures that all relevant stakeholders have access to the information and allows for better tracking of our progress.

Concerning the use of default passwords, I acknowledge your point and your customer's preferences. However, it's crucial to emphasize that maintaining default passwords significantly compromises system security, especially in environments where access control is limited. This practice exposes the system to potential unauthorized access and breaches.

Furthermore, it's important to note that standard users should never require access to the root password. This aligns with the principle of least privilege, a fundamental concept in cybersecurity. By restricting root access, you can minimize the potential impact of security incidents and maintain better control over system modifications.

Dolby remains committed to maintaining the highest standards of security for our products and will continue to monitor and respond to emerging threats proactively.

Dolby Support Services