What is really impressive about this project and llama.cpp as well is that they keep finding new ways to speed up the code on the same hardware. Every time I upgrade (once a month or so) there is usually a 5 to 10% speed boost.
edit: Just to verify: I had planned to upgrade today, ran my usual benchmark prior and after and the runtime went from 4:22 to 3:59. That's a really long prompt (50K tokens) so prefill is a good chunk of that and the rest is spent on token generation.
Long time llama.cpp user (and user of other ggml based AI programs) running on my crappy AMD card that only supports vulkan and it's usable thanks to GGML and llama.cpp, great program great project.
tbh I much prefer this than another claude slop landing page. Not saying it couldn't be better but its kinda refreshing, the whole thing fits on my phone screen
> Claude, design a landing page with beige background color in a gradient, 3 fonts in UI, a blinking led at top right, and enough animation and javascript that makes laptops built before 2025 stutter in scrolling.
There's ample precedent for this. When you're building something to scratch an itch, the name is secondary. You cast about for one. Ah, you have a name. That'll do.
For example, DJ Delorie's djgpp project, built in 1989, was how I got into "real" programming on MS DOS.
Fun story time. I started programming on Turbo Pascal. I had read some C code from those demo scene guides, so felt I understood it fairly well by the time I got my hands on a copy of djgpp. I decided to port a relatively simple program I had written in Turbo Pascal, about 300 LOC, line-by-line. It was a small demo just doing a simple rotating, texture-mapped triangle.
I carefully converted it, and went over it several time one line at a time, fixing it until I had read over it three times and was happy.
I then ran the djgpp compiler...
Now, what I hadn't realized at the time was the difference between C and C++. The examples I had seen was C, and I didn't really know what C++ was.
The compiler churned on my code and spit out about 3500 errors and warnings (it counted for me).
I was shocked. How did my 300 line C program manage to produce over 10 errors and warnings per line, on average?!
I spent a long time trying to figure this one out. The first error was inside the first function, and the error made no sense to me.
After several days of carefully going over each line and banging my head against the table, I suddenly saw it. There, at the very start of the program, before the line the first error or warning pointed to.
In Turbo Pascal I had used const to define things like screen width and height. So I'd have
I removed the two semi-colons and the program compiled and ran just fine...
I ended up staying in Turbo Pascal, I preferred a compiler that gave me sensible errors and quickly. I did pick up C++ much later, and I guess ironically came to appreciate its const correctness.
Historically, Linus didn't name linux. Other people did it for him.
> During the start of his work on the system, he stored the files under the name "Freax" for about half of a year. Torvalds had already considered the name "Linux", but initially dismissed it as too egotistical.
What is really impressive about this project and llama.cpp as well is that they keep finding new ways to speed up the code on the same hardware. Every time I upgrade (once a month or so) there is usually a 5 to 10% speed boost.
edit: Just to verify: I had planned to upgrade today, ran my usual benchmark prior and after and the runtime went from 4:22 to 3:59. That's a really long prompt (50K tokens) so prefill is a good chunk of that and the rest is spent on token generation.
Awesome! ggml and llama.cpp are the Linux of the AI world. Driving local inference one bit at a time.
Wishing you unlimited power and may you never see a day of low battery. Respect and Love. Keep it up.
Long time llama.cpp user (and user of other ggml based AI programs) running on my crappy AMD card that only supports vulkan and it's usable thanks to GGML and llama.cpp, great program great project.
I prefer this entry point: https://github.com/ggml-org/ggml
In my opinion, this is a genuine engineering product. LLM.cpp, the savior of our difficult times...
Not just legacy hardware, but also ready for the immediate future, with RISC-V support.
Congrats for 1.0.0.
You are not serious with this landing page?
Please put a little more effort in for FFS, you have seed money?
Let your design show through; your product can still be professional, and you don't have over index on bland to over compensate.
https://ggml.ai/
I think the minimalism is the point. Compare with https://ssi.inc
> The ggml way
> Minimal
> We like simplicity and aim to keep the codebase as small and as simple as possible
I'm glad their philosophy extends to the website as well. Genuinely refreshing.
tbh I much prefer this than another claude slop landing page. Not saying it couldn't be better but its kinda refreshing, the whole thing fits on my phone screen
> Claude, design a landing page with beige background color in a gradient, 3 fonts in UI, a blinking led at top right, and enough animation and javascript that makes laptops built before 2025 stutter in scrolling.
And don't give any clues about what the project actually does, Just repeat how amazing and fast and modern it is, rephrased in 10 different ways.
Imagine naming a library after yourself.
> GGML is a C library for machine learning (ML) - the "GG" refers to the initials of its originator (Georgi Gerganov).
https://github.com/rustformers/llm/blob/main/crates/ggml/REA...
Every single .exe file begins with the two letters MZ because Mark Zbikowski needed a "magic number" to begin the PE format.
There's ample precedent for this. When you're building something to scratch an itch, the name is secondary. You cast about for one. Ah, you have a name. That'll do.
For example, DJ Delorie's djgpp project, built in 1989, was how I got into "real" programming on MS DOS.
> DJ Delorie's djgpp project
Fun story time. I started programming on Turbo Pascal. I had read some C code from those demo scene guides, so felt I understood it fairly well by the time I got my hands on a copy of djgpp. I decided to port a relatively simple program I had written in Turbo Pascal, about 300 LOC, line-by-line. It was a small demo just doing a simple rotating, texture-mapped triangle.
I carefully converted it, and went over it several time one line at a time, fixing it until I had read over it three times and was happy.
I then ran the djgpp compiler...
Now, what I hadn't realized at the time was the difference between C and C++. The examples I had seen was C, and I didn't really know what C++ was.
The compiler churned on my code and spit out about 3500 errors and warnings (it counted for me).
I was shocked. How did my 300 line C program manage to produce over 10 errors and warnings per line, on average?!
I spent a long time trying to figure this one out. The first error was inside the first function, and the error made no sense to me.
After several days of carefully going over each line and banging my head against the table, I suddenly saw it. There, at the very start of the program, before the line the first error or warning pointed to.
In Turbo Pascal I had used const to define things like screen width and height. So I'd have
Well, C doesn't have const, but I had picked up it had #define. However, my Turbo Pascal habits had kicked in, and so I had written I removed the two semi-colons and the program compiled and ran just fine...I ended up staying in Turbo Pascal, I preferred a compiler that gave me sensible errors and quickly. I did pick up C++ much later, and I guess ironically came to appreciate its const correctness.
For example, Debian. AWK. PKZIP. ARJ. Korn shell. Bourne shell. jemalloc, dlmalloc. djbdns.
I don't have to imagine.
Hey, it worked for Carnegie!
This is the winning response.
And we can only hope that our modern robber barons will use their ill-gotten gains to create something as beneficial as Carnegie libraries.
Par for course for being Linux of AI
Historically, Linus didn't name linux. Other people did it for him.
> During the start of his work on the system, he stored the files under the name "Freax" for about half of a year. Torvalds had already considered the name "Linux", but initially dismissed it as too egotistical.
https://en.wikipedia.org/wiki/History_of_Linux#Naming
He did name Git, tho
I'm biased (see: pianojacq.com)
Imagine trying to name a file format for yourself even if it wasn't yours to begin with...
They wrote a large portion (and all of the initial version I think), they can name it whatever tf they want.
What do you think Linux is named after?
Answered in another thread
jealous?