☆ Yσɠƚԋσʂ ☆@lemmygrad.ml to Technology@lemmygrad.mlEnglish · 2 months agoDeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAIventurebeat.comexternal-linkmessage-square14linkfedilinkarrow-up139arrow-down10cross-posted to: technology@beehaw.orgtechnology@lemmy.ml
arrow-up139arrow-down1external-linkDeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAIventurebeat.com☆ Yσɠƚԋσʂ ☆@lemmygrad.ml to Technology@lemmygrad.mlEnglish · 2 months agomessage-square14linkfedilinkcross-posted to: technology@beehaw.orgtechnology@lemmy.ml
minus-squareCriticalResist8@lemmygrad.mllinkfedilinkarrow-up4·2 months agoBut imagine what you’ll be able to run it on in four more months. But yeah, it’s stretching the definition of consumer hardware a bit.
minus-squarepinguinu [any]@lemmygrad.mllinkfedilinkarrow-up1·2 months agoYou can use the smaller models on (beefy) consumer hardware already. That’s something, right? 😅
minus-squareCriticalResist8@lemmygrad.mllinkfedilinkarrow-up3·2 months agoI want the full 1TB model running on my 10 year old linux laptop
minus-squarepinguinu [any]@lemmygrad.mllinkfedilinkarrow-up2·2 months agoJust put your persistent memory as swap. Easy
But imagine what you’ll be able to run it on in four more months. But yeah, it’s stretching the definition of consumer hardware a bit.
You can use the smaller models on (beefy) consumer hardware already. That’s something, right? 😅
I want the full 1TB model running on my 10 year old linux laptop
Just put your persistent memory as swap. Easy