• CaptainProton@lemmy.world
    link
    fedilink
    English
    arrow-up
    5
    ·
    10 months ago

    More memory means you can do real work with it, and enterprise AI training is a money printer that they’d be scavenging the shit out of with cards that are closer substitutes.

    • tal@lemmy.today
      link
      fedilink
      English
      arrow-up
      1
      ·
      10 months ago

      Honestly, the gap between the server parallel compute cards and the home video cards isn’t that large. 24GB on video cards, 80GB for a compute card.

      That’s not even two binary orders of magnitude. That’s a narrow window to try to make their money from. Plus, some tasks can be subdivided and run on multiple GPUs, and they can’t segment up the market for those.

      Like, in general, my bet is that when for most things that fit the above requirements of fitting in that window and having a task that can’t be subdivided, there’s probably enough room for algorithmic improvements to get two binary orders of magnitude of reduction in memory requirements.