SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 13 hours agoDo you host your own AI?message-squaremessage-square133linkfedilinkarrow-up192arrow-down125file-text
arrow-up167arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 13 hours agomessage-square133linkfedilinkfile-text
minus-squarejaschen306@sh.itjust.workslinkfedilinkEnglisharrow-up1·3 hours agoI’m running dwarfstar which is a 2 bit deepseek v4 flash. It’s quite capable even at 2 bit.
minus-squarezutto@lemmy.fedi.zutto.filinkfedilinkEnglisharrow-up1·3 hours agoThis dwarfstar looks interesting, can you elaborate on your setup and what kind of inference speeds you are getting?
I’m running dwarfstar which is a 2 bit deepseek v4 flash. It’s quite capable even at 2 bit.
This dwarfstar looks interesting, can you elaborate on your setup and what kind of inference speeds you are getting?