Conversations can drift to adjacent topics, yeah, but it’s not a “gotcha” when someone suddenly changes the topic to the inverse of what was being said, and then acts like they’re arguing against you because the thing that you said about the original topic doesn’t add up with the new topic.

If you change the topic, you need to at least give the other person an opportunity to respond to your new topic, not just assume that their same argument applies.

That makes sense. I see the problem with that, and I don’t have a good solution for it. It is a divergence of topic though, as we were discussing open-source programmers using LLMs which are potentially trained on closed-source code.

LLMs trained on open-source code is worth its own discussion, but I don’t see how it fits in this thread. The post isn’t about closed-source programmers using LLMs.

Besides, closed-source code developers could’ve been stealing open-source code all along. They don’t really need AI to do that.

Still, training LLMs on open-source code is a questionable practice for that reason, particularly when it comes to training commercial models on GPL code. But it’s probably hard to prove what code was used in their datasets, since it’s closed-source.

How would they launder it? Just declare it their own property because a few lines of code look similar? When there’s no established connection between the developers and anyone who has access to the closed-source code?

That makes no sense. Please tell me that wouldn’t hold up in court.

If it’s flagged as “assisted by <LLM>” then it’s easy to identify where that code came from. If a commercial LLM is trained on proprietary code, that’s on the AI company, not on the developer who used the LLM to write code. Unless they can somehow prove that the developer had access to said proprietary code and was able to personally exploit it.

If AI companies are claiming “fair use,” and it holds up in court, then there’s no way in hell open-source developers should be held accountable when closed-source snippets magically appear in AI-assisted code.

Granted, I am not a lawyer, and this is not legal advice. I think it’s better to avoid using AI-written code in general. At most use it to generate boilerplate, and maybe add a layer to security audits (not as a replacement for what’s already being done).

But if an LLM regurgitates closed-source code from its training data, I just can’t see any way how that would be the developer’s fault…

Training data isn’t stored in the model. The model processes that data and uses it to adjust the weights and measures of its parameters (usually several to a hundred billion or more for commercial models), which are divided among several layers, hidden sizes, and attention heads. These weights and the architecture are what are hard-coded into the model during training.

Inferencing is what happens when the model generates text from an input, and at this point the weights and measures are hard-coded so it doesn’t actually retain all that information it was trained on. The context window refers to how many tokens (words or phrases) it can store in its memory at a time.

For every token that it processes, it runs a series of calculations on embedded vectors passes them through several layers in which they’re considered against the context of all the other tokens in the context window. This involves matrix multiplication and is very compute-heavy. Think like 1-4GB of RAM for every billion parameters, plus several more GB of RAM for the context window. There’s just no way it would be able to hold its entire training dataset in RAM at a time.

You would need to integrate retrieval-augmented generation to fetch the relevant data into the context window before generating a response, but that’s not at all the same as containing all that knowledge in a stateful manner

In before tankies show up crying that Chinese cultural imperialism is okay because “it’s anti-west.”

Nevermind that they’re committing the same homogenizing colonization of thousand+ year-old cultures that tankies criticize the west for.

Yes, we know, the US sucks. So does the PRC.

You just haven’t noticed that it charges a fee every time

Where do you live where ATM rates are less than .5%?

By the way, using an ATM from the same bank as your card doesn’t count, because that’s usually free.

They should repossess it and place it in the hands of a responsible steward…

I don’t know, 0.5% isn’t a bad rate…

Combined with the news that they’re going to start requiring developer age verification even in the alternate app repositories…

200 hours was in reference to the main storyline.

There are also side quests, guild quests for dozens of different skills, and all sorts of other activities beyond questing. Plus several expansions. You really won’t ever run out of things to do in FFXIV…

Lets federate it (if it hasn’t already been done)

He must have believed the russian propaganda about them being imperialist aggressors or something…

So let’s recap…

You gave NATO and all your allies the middle finger by cutting their budget, tariffing them, insulting them, attempting to bully them into complying with your will, and threatening to invade some of them…

You seemingly forgot their contributions to America’s previous pointless wars in the Middle East by saying they “weren’t on the front lines”, when most of them had sent people who lost their lives…

You end US funding for Ukraine’s defense because you didn’t think it was your problem, and you told Europe that it’s their job to foot the bill…

You give the UN and NATO the middle finger with this farcical and extortionary “Board of Peace”…

And now you launch a pointless invasion of Iran and complain that NATO, a defensive organization is “rejecting” you by not participating in it?

Get fucked, old man!

Jensen Huang has all the GPUs, he can probably play games where each character has its own dedicated GPU and every atom and molecule of the environment is rendered in real time with a hyper-realistic physics engine, with built-in AI that plays for you so that even your idle pastimes are automated giving you more time to WORK AND PRODUCE VALUE FOR THE OWNER-CASTE.

“This game only needs two GPUs to run, what’s the problem?”

“The consumers don’t know what they want. I, the CEO, know what the consumers want. And the consumers want to give me money!”

I grew up with Windows 95 and later XP…

I remember watching the slow downfall ever since Office 2004 (or maybe that was the last one I liked, can’t remember if it was that or 2000) when they changed from primarily using toolbars with submenus to hiding everything in different tabs.

Like, I could find everything I needed to on the classic toolbars. The tabs, I never got used to…

I’ve hated enshittification since before there was even a word for it.

I strongly considered using Mate when I switched to Linux, but KDE is so sleek and I’m settling into it quite nicely. Plus, LibreOffice lets you choose your toolbar style, so I already have the classic version.

All my life I thought I hated computers, until I finally tried Linux. Then I realized I just hate Windows (and Mac, I tried Mac and hated it too).

I think I put off trying Linux for so long because I thought I was incompetent based on my experience with Windows, so I didn’t think I could learn Linux. I was wrong, and some of the beginner-friendly distros make it so easy.

I wonder if Windows does that on purpose, like an abuser who gaslights their victims by telling them they couldn’t make it on their own, that they need them, when they’re the ones holding them back…

I’m so glad I bought my laptop when I did recently, with plenty of RAM and storage. Just in time before that stuff becomes impossible to find or impossibly unaffordable.

This generation of hardware will probably be the plateau. The decline and fall is just around the bend. Maybe another generation of CPUs before those go for bust, too. They can rent out GPUs, RAM, and storage, but as far as I’m aware the CPU needs to be local on the device. At least, a CPU needs to be on the device…

I don’t think it’s a coincidence that this trend is occuring just as NPUs are starting to hit the market. Those chips are too powerful to let the plebeians own and use without restriction, in the corpo’s view.

Whether a better system replaces this one after it falls, or we enter an era where consumer hardware is a thing of the past, remains to be seen. Maybe when the AI bubble bursts, tech companies will have to liquidate their data centers and secondhand servers will become cheap. That would do wonders for the fediverse and the decentralized web…

Tell corporate to give you more RAM or let you choose a lighter operating system 🤷‍♀️