benni@lemmy.world to Lemmy Shitpost@lemmy.world · 2 months agoIn heatlemmy.worldimagemessage-square135linkfedilinkarrow-up11.24Karrow-down114
arrow-up11.23Karrow-down1imageIn heatlemmy.worldbenni@lemmy.world to Lemmy Shitpost@lemmy.world · 2 months agomessage-square135linkfedilink
minus-square_stranger_@lemmy.worldlinkfedilinkarrow-up4·2 months agoIt’s not even words, it “thinks” in “word parts” called tokens.
minus-squareSwedneck@discuss.tchncs.delinkfedilinkarrow-up1·1 month agoand those tokens? just numbers, indexes. LLMs have no concept of language or words or anything, it’s literally just a statistical calculator where the numbers encode some combination of letter(s)
It’s not even words, it “thinks” in “word parts” called tokens.
and those tokens? just numbers, indexes. LLMs have no concept of language or words or anything, it’s literally just a statistical calculator where the numbers encode some combination of letter(s)