18 points | by albinstman 2 hours ago ago
4 comments
I don't know what basis these numbers are estimated on, but my M2 Ultra 128GB runs almost twice as fast as what's reported for the M3 Ultra 512GB.
I wished there was such a tool before picking my hardware, it's useful. But the estimates needs to be somewhat correct.
Thank you for the feedback. What model/quant did you test so I can look in it?
Shame they are not including old crap like my RX580 which is doing some impressive stuff with Qwen 9B Models.
Model list is a bit limited, and whole harness affects the speed, so the libs used in harness and their versions might be useful to show for each individual model setup.
I don't know what basis these numbers are estimated on, but my M2 Ultra 128GB runs almost twice as fast as what's reported for the M3 Ultra 512GB.
I wished there was such a tool before picking my hardware, it's useful. But the estimates needs to be somewhat correct.
Thank you for the feedback. What model/quant did you test so I can look in it?
Shame they are not including old crap like my RX580 which is doing some impressive stuff with Qwen 9B Models.
Model list is a bit limited, and whole harness affects the speed, so the libs used in harness and their versions might be useful to show for each individual model setup.