alessandro@lemmy.ca to PC Gaming@lemmy.ca · 13 hours agoIntel CEO sees 'less need for discrete graphics' and now we're really worried about its upcoming Battlemage gaming GPU and the rest of Intel's graphics roadmapwww.pcgamer.comexternal-linkmessage-square15fedilinkarrow-up154arrow-down11
arrow-up153arrow-down1external-linkIntel CEO sees 'less need for discrete graphics' and now we're really worried about its upcoming Battlemage gaming GPU and the rest of Intel's graphics roadmapwww.pcgamer.comalessandro@lemmy.ca to PC Gaming@lemmy.ca · 13 hours agomessage-square15fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkarrow-up1·edit-26 hours agoOne can’t offload “usable” LLMs without tons of memory bandwidth and plenty of RAM. It’s just not physically possible. You can run small models like Phi pretty quick, but I don’t think people will be satisfied with that for copilot, even as basic autocomplete. About 2x faster than Intel’s current IGPs is the threshold where the offloading can happen, IMO. And that’s exactly what AMD/Apple are producing.
One can’t offload “usable” LLMs without tons of memory bandwidth and plenty of RAM. It’s just not physically possible.
You can run small models like Phi pretty quick, but I don’t think people will be satisfied with that for copilot, even as basic autocomplete.
About 2x faster than Intel’s current IGPs is the threshold where the offloading can happen, IMO. And that’s exactly what AMD/Apple are producing.