I would wager that the AI said inconvenient things, and then got lobotomized to have MAGA brain.
My personal local AI is able to infer consequences, and it is only a 180b parameters on three consumer GPUs. The DOW undoubtedly can use the most powerful hardware and have perfectly configured settings to do a much better job.
It depends on what backend you are using. UnSloth Desktop and KoboldCPP allow you to use multiple GPUs, alongside leveraging your RAM.
Unfortunately, Hermes Agent doesn’t allow you to control and leverage your GPUs as of yet.
If you are using ComfyUI, there is an extension called Distribution, if you are getting into picture generation. You can order up batches of pictures, and each card tackles a different one.
In the case you want to hook up your backend to another program for agentic purposes, I recommend UnSloth Desktop. I have been using it with Lingua Gacha to translate a RPGMaker game. KoboldCPP isn’t able to handle this kind of work yet, unfortunately.
I would wager that the AI said inconvenient things, and then got lobotomized to have MAGA brain.
My personal local AI is able to infer consequences, and it is only a 180b parameters on three consumer GPUs. The DOW undoubtedly can use the most powerful hardware and have perfectly configured settings to do a much better job.
They simply don’t want to.
How do you get the model split over 3 GPUs? I thought the entire model had to be loaded into memory on a card?
It depends on what backend you are using. UnSloth Desktop and KoboldCPP allow you to use multiple GPUs, alongside leveraging your RAM.
Unfortunately, Hermes Agent doesn’t allow you to control and leverage your GPUs as of yet.
If you are using ComfyUI, there is an extension called Distribution, if you are getting into picture generation. You can order up batches of pictures, and each card tackles a different one.
In the case you want to hook up your backend to another program for agentic purposes, I recommend UnSloth Desktop. I have been using it with Lingua Gacha to translate a RPGMaker game. KoboldCPP isn’t able to handle this kind of work yet, unfortunately.