but we can reasonably assume that Stable Diffusion can render the image on the right partly because it has stored visual elements from the image on the left.
No, you cannot reasonably assume that. It absolutely did not store the visual elements. What it did, was store some floating point values related to some keywords that the source image had pre-classified. When training, it will increase or decrease those floating point values a small amount when it encounters further images that use those same keywords.
What the examples demonstrate is a lack of diversity in the training set for those very specific keywords. There’s a reason why they chose Stable Diffusion 1.4 and not Stable Diffusion 2.0 (or later versions)… Because they drastically improved the model after that. These sorts of problems (with not-diverse-enough training data) are considered flaws by the very AI researchers creating the models. It’s exactly the type of thing they don’t want to happen!
The article seems to be implying that this is a common problem that happens constantly and that the companies creating these AI models just don’t give a fuck. This is false. It’s flaws like this that leave your model open to attack (and letting competitors figure out your weights; not that it matters with Stable Diffusion since that version is open source), not just copyright lawsuits!
Here’s the part I don’t get: Clearly nobody is distributing copyrighted images by asking AI to do its best to recreate them. When you do this, you end up with severely shitty hack images that nobody wants to look at. Basically, if no one is actually using these images except to say, “aha! My academic research uncovered this tiny flaw in your model that represents an obscure area of AI research!” why TF should anyone care?
They shouldn’t! The only reason why articles like this get any attention at all is because it’s rage bait for AI haters. People who severely hate generative AI will grasp at anything to justify their position. Why? I don’t get it. If you don’t like it, just say you don’t like it! Why do you need to point to absolutely, ridiculously obscure shit like finding a flaw in Stable Diffusion 1.4 (from years ago, before 99% of the world had even heard of generative image AI)?
Generative AI is just the latest way of giving instructions to computers. That’s it! That’s all it is.
Nobody gave a shit about this kind of thing when Star Trek was pretending to do generative AI in the Holodeck. Now that we’ve got he pre-alpha version of that very thing, a lot of extremely vocal haters are freaking TF out.
Do you want the cool shit from Star Trek’s imaginary future or not? This is literally what computer scientists have been dreaming of for decades. It’s here! Have some fun with it!
Generative AI uses up less power/water than streaming YouTube or Netflix (yes, it’s true). So if you’re about to say it’s bad for the environment, I expect you’re just as vocal about streaming video, yeah?
The article seems to be implying that this is a common problem that happens constantly and that the companies creating these AI models just don’t give a fuck.
Not only does the article not once state that this is a common problem, only explaining the technical details of how it works, and the possible legal ramifications of it, but they mention how, according to nearly any AI scholar/expert you can talk to, this is not some fixable problem. If you take data, and effectively do extremely lossy compression on it, there is still a way for that data to theoretically be recovered.
Advancing LLMs while claiming you’ll work on it doing this doesn’t change the fact that this is a problem inherent to LLMs. There are certainly ways to prevent it, reduce its likelihood, etc, but you can’t entirely remove the problem. The article is simply about how LLMs inherently memorize data, and while you can mask it with more varied training data, you still can’t avoid the fact that trained weights memorize inputs, and when combined together, can eventually reproduce those inputs.
To be very clear, again, I’m not saying it’s impossible to make this happen less, but it’s still an inherent part of how LLMs work, and isn’t some entirely fixable problem. Is it better now than it used to be? Sure. Is it fully fixable? Never.
Clearly nobody is distributing copyrighted images by asking AI to do its best to recreate them. When you do this, you end up with severely shitty hack images that nobody wants to look at
It’s actually a major problem for artists where people will pass their art through an AI model to reimagine it slightly differently so it can’t be copyright striked, but will still retain some of the more human choices, design elements, and overall composition.
Spend any amount of time on social platforms with artists and you’ll find many of them now don’t complain as much about people directly stealing their art and reposting it, but more people stealing their images and changing them a bit with AI, then reposting it so it’s just different enough they can feign innocence and tell their followers it’s all their work.
Basically, if no one is actually using these images except to say, “aha! My academic research uncovered this tiny flaw in your model that represents an obscure area of AI research!” why TF should anyone care?
The thing is, while these are isolated experiments meant to test for these behaviors as quickly as possible with a small set of researchers, when you look at the sheer scale of people using AI tools now, then statistically speaking, you will inevitably get people who put in a prompt that is similar enough to a work that was trained on, and it will output something almost identical to that work, without the prompter realizing.
Why do you need to point to absolutely, ridiculously obscure shit like finding a flaw in Stable Diffusion 1.4 (from years ago, before 99% of the world had even heard of generative image AI)?
Because they highlight the flaws that continue to plague existing models, but have been around for long enough that you can run long-term tests, run them more cheaply on current AI hardware at scale, and can repeat tests with the same conditions rather than starting over again every single time a new model is released.
Again, this memorization is inherent to how these AI models are trained, it gets better with new releases as more training data is used, and more alterations are made, but it cannot be removed, because removing the memorization removes all the training.
I’ll admit it’s less of a “smoking gun” against use of AI in itself than it used to be when the issue was more prevalent, but acting like it’s a non-issue isn’t right either.
Generative AI is just the latest way of giving instructions to computers. That’s it! That’s all it is.
It is not, unless you consider every single piece of software or code ever to be just “a way of giving instructions to computers” since code is just instructions for how a computer should operate, regardless of the actual tangible outcomes of those base-level instructions.
Generative AI is a type of computation that predicts the most likely sequence of text, or distribution of pixels in an image. That is all it is. It can be used to predict the most likely text, in a machine readable format, which can then control a computer, but that is not what it inherently is in its entirety.
It can also rip off artists and journalists, hallucinate plausible misinformation about current events, or delude you into believing you’re the smartest baby of 1996.
It’s like saying a kitchen knife is just a way to cut foods… when it can also be used to stab someone, make crafts, or open your packages. It can be “just a way of altering the size and quantity of pieces of food”, but it can also be a murder weapon or a letter opener.
Nobody gave a shit about this kind of thing when Star Trek was pretending to do generative AI in the Holodeck
That would be because it was a fictional series about a nonexistent future that didn’t affect anyone’s life today in a negative way if nonexistent job roles were replaced, and most people didn’t have to think about how it would affect them if it became reality today.
Do you want the cool shit from Star Trek’s imaginary future or not? This is literally what computer scientists have been dreaming of for decades. It’s here! Have some fun with it!
People also want flying cars without thinking of the noise pollution and traffic management. Fiction isn’t always what people think it could be.
Generative AI uses up less power/water than streaming YouTube or Netflix
But Generative AI is not replacing YouTube or Netflix, it’s primarily replacing web searches. So when someone goes to ChatGPT instead of Google, that uses anywhere from a few tens of times more energy to a couple hundreds more.
Yet they will still also use Netflix on top of that.
I expect you’re just as vocal about streaming video, yeah?
People generally aren’t, because streaming video tends to have a much more positive effect on their lives than AI.
Watching a new show or movie is fun and relaxing. If it isn’t, you just… stop watching. Nobody forces it down your throat.
Having LLMs pollute my search results with plausible sounding nonsense, and displace the jobs of artists I enjoy the art of, is not fun, nor relaxing. Talking with someone on social media just to find out they aren’t even a real human is annoying. Trying to troubleshoot an issue and finding made up solutions makes my problem even harder to solve.
We can’t necessarily all be focusing on every single possible thing that takes energy, but it’s easy to focus on the thing that most people have an overall negative association with the effects of.
No, you cannot reasonably assume that. It absolutely did not store the visual elements. What it did, was store some floating point values related to some keywords that the source image had pre-classified. When training, it will increase or decrease those floating point values a small amount when it encounters further images that use those same keywords.
What the examples demonstrate is a lack of diversity in the training set for those very specific keywords. There’s a reason why they chose Stable Diffusion 1.4 and not Stable Diffusion 2.0 (or later versions)… Because they drastically improved the model after that. These sorts of problems (with not-diverse-enough training data) are considered flaws by the very AI researchers creating the models. It’s exactly the type of thing they don’t want to happen!
The article seems to be implying that this is a common problem that happens constantly and that the companies creating these AI models just don’t give a fuck. This is false. It’s flaws like this that leave your model open to attack (and letting competitors figure out your weights; not that it matters with Stable Diffusion since that version is open source), not just copyright lawsuits!
Here’s the part I don’t get: Clearly nobody is distributing copyrighted images by asking AI to do its best to recreate them. When you do this, you end up with severely shitty hack images that nobody wants to look at. Basically, if no one is actually using these images except to say, “aha! My academic research uncovered this tiny flaw in your model that represents an obscure area of AI research!” why TF should anyone care?
They shouldn’t! The only reason why articles like this get any attention at all is because it’s rage bait for AI haters. People who severely hate generative AI will grasp at anything to justify their position. Why? I don’t get it. If you don’t like it, just say you don’t like it! Why do you need to point to absolutely, ridiculously obscure shit like finding a flaw in Stable Diffusion 1.4 (from years ago, before 99% of the world had even heard of generative image AI)?
Generative AI is just the latest way of giving instructions to computers. That’s it! That’s all it is.
Nobody gave a shit about this kind of thing when Star Trek was pretending to do generative AI in the Holodeck. Now that we’ve got he pre-alpha version of that very thing, a lot of extremely vocal haters are freaking TF out.
Do you want the cool shit from Star Trek’s imaginary future or not? This is literally what computer scientists have been dreaming of for decades. It’s here! Have some fun with it!
Generative AI uses up less power/water than streaming YouTube or Netflix (yes, it’s true). So if you’re about to say it’s bad for the environment, I expect you’re just as vocal about streaming video, yeah?
Not only does the article not once state that this is a common problem, only explaining the technical details of how it works, and the possible legal ramifications of it, but they mention how, according to nearly any AI scholar/expert you can talk to, this is not some fixable problem. If you take data, and effectively do extremely lossy compression on it, there is still a way for that data to theoretically be recovered.
Advancing LLMs while claiming you’ll work on it doing this doesn’t change the fact that this is a problem inherent to LLMs. There are certainly ways to prevent it, reduce its likelihood, etc, but you can’t entirely remove the problem. The article is simply about how LLMs inherently memorize data, and while you can mask it with more varied training data, you still can’t avoid the fact that trained weights memorize inputs, and when combined together, can eventually reproduce those inputs.
To be very clear, again, I’m not saying it’s impossible to make this happen less, but it’s still an inherent part of how LLMs work, and isn’t some entirely fixable problem. Is it better now than it used to be? Sure. Is it fully fixable? Never.
It’s actually a major problem for artists where people will pass their art through an AI model to reimagine it slightly differently so it can’t be copyright striked, but will still retain some of the more human choices, design elements, and overall composition.
Spend any amount of time on social platforms with artists and you’ll find many of them now don’t complain as much about people directly stealing their art and reposting it, but more people stealing their images and changing them a bit with AI, then reposting it so it’s just different enough they can feign innocence and tell their followers it’s all their work.
The thing is, while these are isolated experiments meant to test for these behaviors as quickly as possible with a small set of researchers, when you look at the sheer scale of people using AI tools now, then statistically speaking, you will inevitably get people who put in a prompt that is similar enough to a work that was trained on, and it will output something almost identical to that work, without the prompter realizing.
Because they highlight the flaws that continue to plague existing models, but have been around for long enough that you can run long-term tests, run them more cheaply on current AI hardware at scale, and can repeat tests with the same conditions rather than starting over again every single time a new model is released.
Again, this memorization is inherent to how these AI models are trained, it gets better with new releases as more training data is used, and more alterations are made, but it cannot be removed, because removing the memorization removes all the training.
I’ll admit it’s less of a “smoking gun” against use of AI in itself than it used to be when the issue was more prevalent, but acting like it’s a non-issue isn’t right either.
It is not, unless you consider every single piece of software or code ever to be just “a way of giving instructions to computers” since code is just instructions for how a computer should operate, regardless of the actual tangible outcomes of those base-level instructions.
Generative AI is a type of computation that predicts the most likely sequence of text, or distribution of pixels in an image. That is all it is. It can be used to predict the most likely text, in a machine readable format, which can then control a computer, but that is not what it inherently is in its entirety.
It can also rip off artists and journalists, hallucinate plausible misinformation about current events, or delude you into believing you’re the smartest baby of 1996.
It’s like saying a kitchen knife is just a way to cut foods… when it can also be used to stab someone, make crafts, or open your packages. It can be “just a way of altering the size and quantity of pieces of food”, but it can also be a murder weapon or a letter opener.
That would be because it was a fictional series about a nonexistent future that didn’t affect anyone’s life today in a negative way if nonexistent job roles were replaced, and most people didn’t have to think about how it would affect them if it became reality today.
People also want flying cars without thinking of the noise pollution and traffic management. Fiction isn’t always what people think it could be.
But Generative AI is not replacing YouTube or Netflix, it’s primarily replacing web searches. So when someone goes to ChatGPT instead of Google, that uses anywhere from a few tens of times more energy to a couple hundreds more.
Yet they will still also use Netflix on top of that.
People generally aren’t, because streaming video tends to have a much more positive effect on their lives than AI.
Watching a new show or movie is fun and relaxing. If it isn’t, you just… stop watching. Nobody forces it down your throat.
Having LLMs pollute my search results with plausible sounding nonsense, and displace the jobs of artists I enjoy the art of, is not fun, nor relaxing. Talking with someone on social media just to find out they aren’t even a real human is annoying. Trying to troubleshoot an issue and finding made up solutions makes my problem even harder to solve.
We can’t necessarily all be focusing on every single possible thing that takes energy, but it’s easy to focus on the thing that most people have an overall negative association with the effects of.
Two birds, one stone.