ylai@lemmy.ml to LocalLLaMA@sh.itjust.worksEnglish · 6 months agoMozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizationswww.phoronix.comexternal-linkmessage-square3fedilinkarrow-up131arrow-down11
arrow-up130arrow-down1external-linkMozilla's Llamafile 0.8.2 Scores Big With New AVX2 Performance Optimizationswww.phoronix.comylai@lemmy.ml to LocalLLaMA@sh.itjust.worksEnglish · 6 months agomessage-square3fedilink
minus-squarexcjs@programming.devlinkfedilinkEnglisharrow-up1·edit-24 months agoI just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/
I just wanted to update this to mention that there are a lot of custom low level performance improvements for CPU based inferencing in Llamafile: https://justine.lol/matmul/