Some thoughts on the xz backdoor

While the exact details of this vulnerability are still investigated (see here if you want to catch up on the topic), I wanted to share some of the thoughts I had regarding to what this incident means for the wider open source ecosystem.

TL;DR: To summarize, these are the main points I found remarkable in this entire development:

• A backdoor was snuck relatively openly into an open source project
• It was done by a somewhat trusted maintainer
• The target was not even xz itself, but rather sshd through an obscure chain of dependencies
• Luckily, it was discovered within a few weeks before the backdoored version was widely adopted

Obviously, there are many examples of security vulnerabilities occurring in open source software. But these are usually due to oversights or mistakes of most likely well-meaning developers that end up enabling the possibility for critical exploits. In the case of the xz backdoor however, it was obviously constructed with malicious intent and high effort towards a precise target. Does anybody know of another vulnerability ending up in a high-profile open source project that is similar in that sense?

This was only possible because the malicious actor under the pseudonym Jia Tan had direct write access to the xz repository as a maintainer. I don't think it is too unreasonable that with enough time and effort, anyone can get maintenance access to openly developed projects like xz. That is part of the beauty of the democratic process in open source. But what this incident shows is that for projects that are as widely used as xz, even changes coming from seemingly trusted maintainers should be properly reviewed. I don't mean to say that the original maintainer Lasse Collin has any fault in this matter, or that he should have prevented it, this is too much of a burden to expect from a single person. Instead I think the large tech corporations should put more resources into vetting these kind of open source projects that much of their infrastructure so heavily relies on (in fact, this backdoor seems to mainly target servers).

Even just looking at the source code, the backdoor was very cleverly hidden in testing binaries for the compression algorithm. These things are always easy to say in hindsight, but I do believe that a closer review of the build system shenanigans used to install the backdoor would have at least raised some questions. There was just too much luck involved in the discovery of the backdoor with someone noticing ssh access taking 0.5 seconds longer than usual.

This isn't really news, but this incident again shows that just like a chain is only as strong as its weakest link, a program is only as strong as its weakest dependency. The fact that the backdoor just hooks into the dynamic library loading process and completely hijacks authorization functions of ssh from inside xz is pretty scary. Maybe this will encourage developers to be more careful and sparing with adding dependencies. However to be honest, up until recently I would have pretty blindly trusted xz to be a very safe dependency due to its popularity and relatively simple use-case.

By opening a backdoor into ssh servers, this is a very critical issue, and there was clearly a lot of time and effort put into making it seem innocuous and hard to detect. I'm very glad that it got found and patched by the time it did, but it does leave me wondering what else is out there. It would be illusionary to think that such attack vectors always get found out eventually.