• atzanteol@sh.itjust.worksEnglish
    150·
    8 个月前

    They noticed that some ssh sessions took 0.5 seconds too long under certain circumstances. 😲

    Holy hell that’s good QA.

      • krash@lemmy.ml
        25·
        8 个月前

        Don’t see why you’re being downvoted, the person in question who discovered this is a postgres maintainer employed by Microsoft.

        • ijhoo@lemmy.ml
          18·
          8 个月前

          Probably people think this is a troll or something.

          I wrote it because I was surprised, especially since I’m not a fan of microsoft and their policies. Lately, I have the feeling Microsoft is better than Google (relative terms) when it comes to oss.

          What is additionally surprising is the breaches of Microsoft services in the last year. There is one every few weeks or so… And then they pick up a backdoor because login took 0.5 instead of 0.1s.

          Anyway, his findings are amazing.

          • Wes_Dev@lemmy.ml
            92·
            8 个月前

            This isn’t the same thing, but I’m reminded of Minecraft.

            Minecraft is a massively popular game. Notch once said he planned to make it open source when its popularity died down. But now Microsoft owns it.

            Not only that, but Mojang accounts don’t work anymore. You have to have a Microsoft account to play it now. Even trying to download and play an older version of the game offline requires Microsoft to approve it. Microsoft is actively tightening the leash on the game because it makes them money. Open sourcing the game will likely never happen now. The best we can hope for it for versions to fall into public domain after 70-ish years.

            That’s how I see Microsoft. They only care about what its beneficial for them to drive profits. Working on open source projects, and open sourcing a few of their tools to get the benefits of community adoption and code review is great, sure. But they’d sooner try to incorporate Linux into Windows to keep people in their surveillance ecosystem, than to open source Windows.

            Remember when Windows 10 was the last version, until they changed their minds? Remember when they floated the idea of charging a recurring subscription to use Windows, before they silently dropped the idea? Remember when there was credible talk about the next version of Windows being cloud-based where they controlled all your data and you had no privacy? Hell, you have basically no privacy on Windows 10. Trying to reclaim some involves registry edits, special third party tools, and a constant battle with automatic updates reverting your changes.

            I’ll say it again. Microsoft doesn’t care about OSS. It’s just currently beneficial for them to pretend they do.

            Goggle seemed to care a lot about OSS, then started making everything in Android depend on their proprietary ecosystem to function. Now Google is using the dominant position they got by taking advantage of OSS adoption, and have been pushing privacy-invading standards and trying to get rid of ad blockers online, among many other things.

            For these huge companies, OSS is just a tool to get more control and power. The moment it’s no longer useful, they’ll find ways to work around the license and enshitify everything again.

            It keeps happening. I refuse to keep trusting bad actors every time they dangle a shiny trinket over our heads.

            I do appreciate the work this person did in finding the bug. It’s not all doom and gloom.

            • ijhoo@lemmy.ml
              3·
              8 个月前

              I agree with you sentiment here. That’s why I wrote ‘relative terms’ in my comment.

              Since Nadela took over, Microsoft did some open thing which benefited community. So, Microsoft opened somewhat.

              During the same time, under Pichai, google went the other way: they focus more on monetization and try to control stuff the apple way. Manifest v3? Google also didn’t do anything really worth mentioning in the last 10y in terms of products. Well, except ‘attention’ article. And even this they didn’t believe in and they cannot deliver a decent product. I just tried google advanced Gemini and it’s, to put it politely, shit. Google also had some positive actions like mainlining a lot of stuff in Linux Kernel to more easily upgrade android.

              So, while google is closing down and making mistakes, Microsoft is opening a bit up.

              If you look the state from the last year and the state now. Microsoft improved. Google went the other way.

              Microsoft doesn’t care about open source, they care about the money Cloud Services using open source bring them. I don’t think google cares as well. For reason read this: https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

    • 30p87@feddit.de
      18·
      8 个月前

      Well half a second delay is pretty noticeable when you ssh into a machine sitting right next to you. It should be instant. And if it isn’t something’s off.

  • capt_kafei@lemmy.caEnglish
    97·
    8 个月前

    Damn, it is actually scary that they managed to pull this off. The backdoor came from the second-largest contributor to xz too, not some random drive-by.

      • Alex@lemmy.ml
        611·
        8 个月前

        It’s looking more like a long game to compromise an upstream.

        • cjk@feddit.de
          18·
          8 个月前

          Either that or the attacker was very good at choosing their puppet…

          • Alex@lemmy.ml
            36·
            8 个月前

            Well the account is focused on one particular project which makes sense if you expect to get burned at some point and don’t want all your other exploits to be detected. It looks like there was a second sock puppet account involved in the original attack vector support code.

            We should certainly audit other projects for similar changes from other psudoanonymous accounts.

    • Alex@lemmy.ml
      34·
      8 个月前

      Time to audit all their contributions although it looks like they mostly contribute to xz. I guess we’ll have to wait for comments from the rest of the team or if the whole org needs to be considered comprimised.

      • sim642@lemm.ee
        2·
        8 个月前

        Assuming that it’s just that person, that it’s their actual name and that they’re in the US…

      • ugjka@lemmy.worldEnglish
        24·
        8 个月前

        there will be federal investigation just speculation if the culprit is a foreign actor

  • chameleon@kbin.social
    73·
    8 个月前

    This is a fun one we’re gonna be hearing about for a while…

    It’s fortunate it was discovered before any major releases of non-rolling-release distros were cut, but damn.

    • rolaulten@startrek.website
      7·
      8 个月前

      That’s the scary thing. It looks like this narrowly missed getting into Debian and RH. Downstream downstream that is… everything.

    • mumblerfish@lemmy.world
      13·
      8 个月前

      Gentoo just reverted back to the last tar signed by another author than the one seeming responsible for the backdoor. The person has been on the project for years, so one should keep up to date and possibly revert even further back than just from 5.6.*. Gentoo just reverted to 5.4.2.

    • flying_sheep@lemmy.ml
      912·
      8 个月前

      Backdoor only gets inserted when building RPM or DEB. So while updating frequently is a good idea, it won’t change anything for Arch users today.

        • flying_sheep@lemmy.ml
          13·
          8 个月前

          No, read the link you posted:

          Arch does not directly link openssh to liblzma, and thus this attack vector is not possible. You can confirm this by issuing the following command:

          ldd "$(command -v sshd)"

          However, out of an abundance of caution, we advise users to remove the malicious code from their system by upgrading either way.

        • progandy@feddit.de
          3·
          8 个月前

          Those getting the most recent software versions, so nothing that should be running in a server.

        • Laser@feddit.de
          2·
          8 个月前

          Fedora 41, Fedora Rawhide, Debian Sid are the currently known affected ones AFAIK.

        • flying_sheep@lemmy.ml
          1·
          8 个月前

          I think it needs to be

          • rolling release (because it was caught so quickly that it hasn’t made its way into any cadence based distro yet)
          • using the upstream Makefile task to build a RPM or DEB (because the compromised build script directly checks for that and therefore doesn’t trigger for a destdir build like Gentoo’s or Arch’s)
          • using the upstream provided tarball as opposed to the one GitHub provides, or a git clone (because only that contains the compromised Makefile, running autotools yourself is safe)

          Points 1 and 2 mean that only rolling release RPM and DEB distros like Debian Sid and Fedora are candidates. I didn’t check if they use the Makefile and the compromised tarballs.

    • chameleon@kbin.social
      581·
      8 个月前

      Won’t help here; this backdoor is entirely reproducible. That’s one of the scary parts.

      • OsrsNeedsF2P@lemmy.ml
        324·
        8 个月前

        The backdoor wasn’t in the source code, only in the distributed binary. So reproducible builds would have flagged the tar as not coming from what was in Git

        • chameleon@kbin.social
          41·
          8 个月前

          Reproducible builds generally work from the published source tarballs, as those tend to be easier to mirror and archive than a Git repository is. The GPG-signed source tarball includes all of the code to build the exploit.

          The Git repository does not include the code to build the backdoor (though it does include the actual backdoor itself, the binary “test file”, it’s simply disused).

          Verifying that the tarball and Git repository match would be neat, but is not a focus of any existing reproducible build project that I know of. It probably should be, but quite a number of projects have legitimate differences in their tarballs, often pre-compiling things like autotools-based configure scripts and man pages so that you can have a relaxed ./configure && make && make install build without having to hunt down all of the necessary generators.

          • flying_sheep@lemmy.ml
            13·
            8 个月前

            Time to change that tarball thing. Git repos come with built in checksums, that should be the way to go.

            • tal@lemmy.todayEnglish
              3·
              8 个月前

              Honestly, while the way they deployed the exploit helped hide it, I’m not sure that they couldn’t have figured out some similar way to hide it in autoconf stuff and commit it.

              Remember that the attacker had commit privileges to the repository, was a co-maintainer, and the primary maintainer was apparently away on a month-long vacation. How many parties other than the maintainer are going to go review a lot of complicated autoconf stuff?

              I’m not saying that your point’s invalid. Making sure that what comes out of the git repository is what goes to upstream is probably a good security practice. But I’m not sure that it really avoids this.

              Probably a lot of good lessons that could be learned.

              • It sounds like social engineering, including maybe use of sockpuppets, was used to target the maintainer, to get him to cede maintainer status.

              • Social engineering was used to pressure package maintainers to commit.

              • Apparently automated software testing software did trip on the changes, like some fuzz-tesing software at Google, but the attacker managed to get changes committed to avoid it. This was one point where a light really did get aimed at the changes. That being said, the attacker here was also a maintainer, and I don’t think that the fuzzer guys consider themselves responsible for identifying security holes. And while it did highlight the use of ifunc, it sounds like it was legitimately a bug. But, still, it might be possible to have some kind of security examination taking place when fuzzing software trips, especially if the fuzzing software isn’t under control of a project’s maintainer (as it was not, here).

              • The changes were apparently aimed at getting in shortly before Ubuntu freeze; the attacker was apparently recorded asking and ensuring that Ubuntu fed off Debian testing. Maybe there needs to be more-attention paid to things that go in shortly before freeze.

              • Part of the attack was hidden in autoconf scripts. Autoconf, especially with generated data going out the door, is hard to audit.

              • As you point out, using a chain that ensures that a backdoor that goes into downstream also goes into git would be a good idea.

              • Distros should probably be more careful about linking stuff to security-critical binaries like sshd. Apparently this was very much not necessary to achieve what they wanted to do in this case; it was possible to have a very small amount of code that performed the functionality that was actually needed.

              • Unless the systemd-notifier changes themselves were done by an attacker, it’s a good bet that the Jia Tan group and similar are monitoring software, looking for dependencies like the systemd-notifier introduction. Looking for similar problems that might affect similar remotely-accessible servers might be a good idea.

              • It might be a good idea to have servers run their auth component in an isolated module. I’d guess that it’d be possible to have a portion of sshd that accepts incoming connections (and is exposed to the outside, unauthenticated world) as an isolated process. That’d be kind of inetd-like functionality. The portion that performed authentication (and is also running exposed to the outside) as an isolated process, and the code that runs only after authentication succeeds run separately, with only the latter bringing in most libraries.

              • I’ve seen some arguments that systemd itself is large and complicated enough that it lends itself to attacks like this. I think that maybe there’s an argument that some sort of distinction should be made between more- or less-security-critical software, and different policies applied. Systemd alone is a pretty important piece of software to be able to compromise. Maybe there are ways to rearchitect things to be somewhat more-resilient and auditable.

              • I’m not familiar with the ifunc mechanism, but it sounds like attackers consider it to be a useful route to hide injected code. Maybe have some kind of auditing system to look for that.

              • The attacker modified the “in the event of an identified security hole” directions to discourage disclosure to anyone except the project for a 90-day embargo period, and made himself the contact point. That would have provided time to continue to use the exploit. In practice, perhaps software projects should not be the only contact point – perhaps it should be the norm to both notify software projects and a separate, unrelated-to-a-project security point. That increases the risk of the exploit leaking, but protects against compromise of the project maintainership.

              • flying_sheep@lemmy.ml
                1·
                8 个月前

                You’re right, there’s more parts to it, especially social engineering. Maybe there’s other ways to hide a payload, but there aren’t many avenues. You have to hide the payload in a binary artefact, which are pretty suspicious when you don’t do it in a (well scrutinized) cryptography lib, or a compression lib.

                Then that payload has to be executed for some reason, which means you need a really good reason to embed it (e.g. something like widevine), or have to modify the build script.

        • Virulent@reddthat.com
          2·
          8 个月前

          Not exactly - it was in the source tarbal available for download from the releases page but not the git source tree.

    • Daniel Quinn@lemmy.caEnglish
      3·
      8 个月前

      Why didn’t this become a thing? Surely in 2024, we should be able to build packages from source and sign releases with a private key.

      • Natanael@slrpnk.net
        5·
        8 个月前

        It’s becoming more of a thing but a lot of projects are so old that they haven’t been able to fix their entire build process yet

  • Doombot1@lemmy.one
    27·
    8 个月前

    ELI5 what does this mean for the average Linux user? I run a few Ubuntu 22.04 systems (yeah yeah, I know, canonical schmanonical) - but they aren’t bleeding edge, so they shouldn’t exhibit this vulnerability, right?

  • lemmyreader@lemmy.mlEnglish
    191·
    8 个月前

    t y for sharing.

    #showerthoughts The problem is in upstream and has only entered Debian Sid/unstable. Does this mean that for example bleeding edge Arch (btw) sshd users are compromised already ?

  • Wes_Dev@lemmy.ml
    12·
    8 个月前

    Damn fine work all around.

    I know this is an issue fraught with potential legal and political BS, and it’s impossible to check everything without automation these days, but is there an organization that trains and pays people to work as security researchers or QA for open source projects?

    Basically, a watchdog group that finds exploitable security vulnerabilities, and works with individuals or vendors to patch them? Maybe make it a publicly owned and operated group with mandatory reporting of some kind. An international project funded by multiple governments, where it’s harder for a single point of influence to hide exploits, abuse secrets, or interfere with the researchers? They don’t own or control any code, just find security issues and advise.

    I don’t know.

    Just thinking that modern security is getting pretty complicated, with so many moving parts and all.