I usually don’t even understand the lingo they use. “Open-weighted” is the most recent one, then it usually goes down to specific “models” that everybody is supposed to know about.

These are my thoughts (I will stick to the vague “it” for now, but of course therein lies another question: “and how does all this apply to various specialised AIs”):

  • Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?
  • If yes to the previous: the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?

If what I wrote above is true, what exactly are people arguing when they say it’s still possible to use LLMs ethically or true to FOSS philosophy, because … ???

edit

Thanks to all who answered.

I guess it’s my fault for asking several questions in one, but this thread has attracted exactly the type of people I’m writing about; several even used the term “open-weighted models” without explaining it.

Asking to get arguments explained, I got more arguments instead.

  • Franco Perdu@sh.itjust.works
    192·
    1 day ago

    I’ll take a crack at it.

    I run AI on an old Nvidia p40 purchased online for less than $200. The models I run on it came predominantly from Chinese companies and groups. On balance, they:

    • use much less fossil fuel in producing these LLMs than their Western counterparts

    • produce models that run much better on lower end consumer hardware

    As for training data, the inputs used to create these models, it varies greatly but the most popular line, Qwen, Came from the company’s own data from operating such huge networks and systems for so long

    I really fail to see how any of that is worse than playing a video game.

    None of this takes away from the very real issues around data center build out and Western companies using the systems to scare people and continue an economic bubble. That’s all true and bad. But there’s nuance. Not all AI is created equal

    • masterspace@lemmy.caEnglish
      102·
      1 day ago

      use much less fossil fuel in producing these LLMs than their Western counterparts

      Utterly false, given that all the decent Chinese models (especially Qwen) are distilled from Western frontier models.

      They literally could not exist without the enormously carbon emissive western models existing first to train them.

      • klankin@piefed.caEnglish
        3·
        1 day ago

        Also China has an insanely diversified grid, that utilises almost a global scale of fossil fuels.

        So its like green communism at best

      • A_norny_mousse@piefed.socialOPEnglish
        2·
        1 day ago

        distilled from Western frontier models.

        What does this mean please (remember the title of the post!) and how do you know?

        • masterspace@lemmy.caEnglish
          3·
          1 day ago

          Basically, it’s really really expensive (from a compute standpoint, and more compute = money and energy) to train a decent LLM just using books and conversations etc. Anthropic, OpenAI, etc spend a truly insane amount of money doing this is in data centers.

          Running an LLM like that is also extremely expensive from a compute standpoint, and right now the true frontier models can basically only be run in datacenters.

          However, once you have a really good LLM, you can use it to train another LLM. Because it’s basically already refined all its original training data (and is capable of further refining it’d output on the fly), training a new LLM from it is vastly easier and cheaper.

          Additionally, you can use a much smaller and more efficient model, so it can run on less powerful hardware.

          So all local LLMs that are any good that exist right now, were trained using the outputs of existing super expensive frontier models. They could not exist without the frontier models to train them. This is also how OpenAI and Anthropic create their cheaper models. Mythos/Fable is anthropics current frontier model and they distill it into (use it to train) their cheaper models like Sonnet and Opus.

            • masterspace@lemmy.caEnglish
              31·
              1 day ago

              They’re honestly not that rickety. If you’re getting your news from social media (including Lemmy) you may want to actually try them rather than just take the random incidents where things go wrong as broadly representative.

      • HubertManne@piefed.socialEnglish
        31·
        1 day ago

        yeah the comment was bad enough I looked at the user. been around for 2 years and no posts or comments till this one. made a note on the account.