The review shows ARM64 software support is still painful vs x86. For $200 for the 16gb model, this is the price point where you could just get an Intel N150 mini PC in the same form factor. And those usually come with cases. They also tend to pull 5-8w at idle, while this is 15w. Cool if you really want ARM64, but at this end of the performance spectrum, why not stick with the x86 stack where everything just works a lot easier?
From the article: "[...] the Linux support for various parts of the boards, not being upstreamed and mainlined, is very likely to be stuck on an older version. This is usually what causes headaches down the road [...]".
The problem isn't support for the ARM architecture in general, it's the support for this particular board.
Other boards like the Raspberry Pi and many boards based on Rockchip SoCs have most of the necessary support mainlined, so the experience is quite painless. Many are starting to get support for UEFI as well.
That makes sense, as the Pi is as easy as x86 at this point. I almost never have to compile from scratch.
I'm not a compiler expert... But it seems each ARM64 board needs its own custom kernel support, but once that is done, it can support anything compiled to ARM64 as a general target? Or will we still need to have separate builds for RPi, for this board, etc?
This. The issue is the culture inside many of these HW companies that is oppositional to upstreaming changes and developing in the open in general.
Often an outright mediocre software development culture generally, that sees software as a pure cost centre, in fact. The "product" is seem to be the chip, the software "just" a side show (or worse, a channel by which their IP could leak).
The Rockchip stuff is better, but still has similar problems.
Agreed, at least for a likely "home use" case, such as a TV box, router, or general purpose file server or Docker host, I don't see how this board is better than something like a Beelink mini PC. The Orange Pi does not even come with a case, power supply or cooler. Contrast that with a Beelink that has a built-in power supply (no external brick) and of course a case and cooler.
With RAM it will be costing notably more, with 4 cores instead of 12. I'd expect this to run circles around an N150 for single-threaded perf too.
They are not in the same class, which is reflected in the power envelope.
BTW what's up with people pushing N150 and N300 in every single ARM SBC thread? Y'all Intel shareholders or something? I run both but not to the exclusion of everything else. There is nothing I've failed to run successfully on my ARM ones and the only thing I haven't tried is gaming.
1. Wow, never thought I'd need to do an investment disclosure for an HN comment. But sure thing: I'm sure Intel is somewhere in my 401K's index funds, but also probably Qualcomm. But I'm not a corporate shill, thank you very much for the good faith. Just a hobbyist looking to not get seduced by the lastest trend. If I were an ARM developer that'd be different, I get that.
2. The review says single core Geekbench performance is 1290, same as i5-10500 which is also similar to N150, which is 1235.
3. You can still get N150s with 16gb ram in a case for $200 all in.
Because most ARM SBCs are still limited to whatever linux distro they added support to. Intel SBCs might underperform but you can be sure it will run anything built for x86-64.
Are you sure you don't have single-threaded and multi-threaded backwards?
Why would the A720 at 2.8 GHz run circles around the N150 that boosts up to 3.6 GHz in single-threaded workloads, while the 12-core chip would wouldn't beat the 4-core chip in multithreaded workloads?
I can't speak to why other people bring up the N150 in ARM SBC threads any more than "AMD doesn't compete in the ~$200 SBC segment".
FWIW, as far as SBC/NUCs go, I've had a Pi 4, an RK3399 board, an RK3568 board, an N100 NUC from GMKTec, and a N150 NUC from Geekom, and the N150 has by far been my favorite out of those for real-world workloads rather than tinkering. The gap between the x86 software ecosystem and the ARM software ecosystem is no joke.
P.S. Stay away from GMKTec. Even if you don't get burned, your SODIMM cards will. There are stoves, ovens, and hot plates with better heat dissipation and thermals than GMKTec NUCs.
Yes x86 will win for convenience on about every metric (at least for now), but this SoC's CPU is much faster than a mere Intel N150 (especially for multicore use cases).
I've got two RK3588 boards here doing Linux-y things around my place (Jellyfin, Forgejo builders, Immich, etc) and ... I don't think I've run into pain? They're running various debian and just ... work? I can't think of a single package that I couldn't get for ARM64.
Likewise my VPS @ Hetzner is running Aarch64. No drama. Only pain is how brutal the Rust cross-compile is from my x86 machine.
(Those things are expensive, but I just ordered one [the ASUS variant] for myself.)
Meanwhile Apple is pushing the ARM64 architecture hard, and Windows is apparently actually quite viable now?
Personally... it's totally irrational, but I have always had a grudge against x86 since it "won" in the early 90s and I had to switch from 68k. I want diversity in ISAs. RISC-V would be nice, but I'll settle for ARM for now.
When something has an 30 TOPS NPU, what are the implications? Do NPUs like this have some common backend that ggml/llama.cpp targets? Is it proprietary and only works for some specific software? Does it have access to all the system RAM and at what bandwidth?
I know the concept has been around for a while but no idea if it actually means anything. I assume that people are targeting ones in common devices like Apple, but what about here?
The specific NPU doesn't seem to be mentioned in TFA, but my guess is that the blessed way to deal with it is the Neon SDK: https://www.arm.com/technologies/neon
I've not found Neon to be fun or easy to use, and I frequently see devices ignoring the NPU and inferring on CPU because it's easier. Maybe you get lucky and someone has made a backend for something specific you want, but it's not common.
Can't speak to this specific NPU but these kind of accelerators are really made more for more general ML things like machine vision etc. For example while people have made the (6 TOPS) NPU in the (similar board) RK3588 work with llama.cpp it isn't super useful because of the RAM constraints. I believe it has some sort of 32-bit memory addressing limit, so you can never give it more than 3 or 4 GB for example. So for LLMs, not all that useful.
It needs specific support, and for example llama.cpp would have support for some of them. But that comes with limitations in how much RAM they can allocate. But when they work, you see a flat CPU usage and the NPU does everything for inference.
The review shows ARM64 software support is still painful vs x86. For $200 for the 16gb model, this is the price point where you could just get an Intel N150 mini PC in the same form factor. And those usually come with cases. They also tend to pull 5-8w at idle, while this is 15w. Cool if you really want ARM64, but at this end of the performance spectrum, why not stick with the x86 stack where everything just works a lot easier?
From the article: "[...] the Linux support for various parts of the boards, not being upstreamed and mainlined, is very likely to be stuck on an older version. This is usually what causes headaches down the road [...]".
The problem isn't support for the ARM architecture in general, it's the support for this particular board.
Other boards like the Raspberry Pi and many boards based on Rockchip SoCs have most of the necessary support mainlined, so the experience is quite painless. Many are starting to get support for UEFI as well.
That makes sense, as the Pi is as easy as x86 at this point. I almost never have to compile from scratch.
I'm not a compiler expert... But it seems each ARM64 board needs its own custom kernel support, but once that is done, it can support anything compiled to ARM64 as a general target? Or will we still need to have separate builds for RPi, for this board, etc?
Apart from very rare cases, this will run any linux-arm64 binary.
This. The issue is the culture inside many of these HW companies that is oppositional to upstreaming changes and developing in the open in general.
Often an outright mediocre software development culture generally, that sees software as a pure cost centre, in fact. The "product" is seem to be the chip, the software "just" a side show (or worse, a channel by which their IP could leak).
The Rockchip stuff is better, but still has similar problems.
Agreed, at least for a likely "home use" case, such as a TV box, router, or general purpose file server or Docker host, I don't see how this board is better than something like a Beelink mini PC. The Orange Pi does not even come with a case, power supply or cooler. Contrast that with a Beelink that has a built-in power supply (no external brick) and of course a case and cooler.
This OrangePi 6 Plus board comes with cooling and a power supply (usb-c). No case, though.
With RAM it will be costing notably more, with 4 cores instead of 12. I'd expect this to run circles around an N150 for single-threaded perf too.
They are not in the same class, which is reflected in the power envelope.
BTW what's up with people pushing N150 and N300 in every single ARM SBC thread? Y'all Intel shareholders or something? I run both but not to the exclusion of everything else. There is nothing I've failed to run successfully on my ARM ones and the only thing I haven't tried is gaming.
1. Wow, never thought I'd need to do an investment disclosure for an HN comment. But sure thing: I'm sure Intel is somewhere in my 401K's index funds, but also probably Qualcomm. But I'm not a corporate shill, thank you very much for the good faith. Just a hobbyist looking to not get seduced by the lastest trend. If I were an ARM developer that'd be different, I get that.
2. The review says single core Geekbench performance is 1290, same as i5-10500 which is also similar to N150, which is 1235.
3. You can still get N150s with 16gb ram in a case for $200 all in.
> review says single core Geekbench performance is 1290, same as i5-10500 which is also similar to N150, which is 1235.
Single core, yes. Multi core score is much higher for this SBC vs the N150.
Because most ARM SBCs are still limited to whatever linux distro they added support to. Intel SBCs might underperform but you can be sure it will run anything built for x86-64.
No idea - the ryzen based ones are better!
Are you sure you don't have single-threaded and multi-threaded backwards?
Why would the A720 at 2.8 GHz run circles around the N150 that boosts up to 3.6 GHz in single-threaded workloads, while the 12-core chip would wouldn't beat the 4-core chip in multithreaded workloads?
Obviously, the Intel chip wins in single-threaded performance while losing in multi-threaded: https://www.cpubenchmark.net/compare/6304vs6617/Intel-N150-v...
I can't speak to why other people bring up the N150 in ARM SBC threads any more than "AMD doesn't compete in the ~$200 SBC segment".
FWIW, as far as SBC/NUCs go, I've had a Pi 4, an RK3399 board, an RK3568 board, an N100 NUC from GMKTec, and a N150 NUC from Geekom, and the N150 has by far been my favorite out of those for real-world workloads rather than tinkering. The gap between the x86 software ecosystem and the ARM software ecosystem is no joke.
P.S. Stay away from GMKTec. Even if you don't get burned, your SODIMM cards will. There are stoves, ovens, and hot plates with better heat dissipation and thermals than GMKTec NUCs.
Yes x86 will win for convenience on about every metric (at least for now), but this SoC's CPU is much faster than a mere Intel N150 (especially for multicore use cases).
I've got two RK3588 boards here doing Linux-y things around my place (Jellyfin, Forgejo builders, Immich, etc) and ... I don't think I've run into pain? They're running various debian and just ... work? I can't think of a single package that I couldn't get for ARM64.
Likewise my VPS @ Hetzner is running Aarch64. No drama. Only pain is how brutal the Rust cross-compile is from my x86 machine.
I mean, here's Geerling running a bunch of Steam games flawlessly on a Aarch64 NVIDIA GB10 machine: https://www.youtube.com/watch?v=FjRKvKC4ntw
(Those things are expensive, but I just ordered one [the ASUS variant] for myself.)
Meanwhile Apple is pushing the ARM64 architecture hard, and Windows is apparently actually quite viable now?
Personally... it's totally irrational, but I have always had a grudge against x86 since it "won" in the early 90s and I had to switch from 68k. I want diversity in ISAs. RISC-V would be nice, but I'll settle for ARM for now.
e-*ing-waste if you have to wait for manufacturer to provide supported images.
Upstream the drivers to the mainline kernel or go bankrupt. Nobody should buy these.
When something has an 30 TOPS NPU, what are the implications? Do NPUs like this have some common backend that ggml/llama.cpp targets? Is it proprietary and only works for some specific software? Does it have access to all the system RAM and at what bandwidth?
I know the concept has been around for a while but no idea if it actually means anything. I assume that people are targeting ones in common devices like Apple, but what about here?
The specific NPU doesn't seem to be mentioned in TFA, but my guess is that the blessed way to deal with it is the Neon SDK: https://www.arm.com/technologies/neon
I've not found Neon to be fun or easy to use, and I frequently see devices ignoring the NPU and inferring on CPU because it's easier. Maybe you get lucky and someone has made a backend for something specific you want, but it's not common.
Can't speak to this specific NPU but these kind of accelerators are really made more for more general ML things like machine vision etc. For example while people have made the (6 TOPS) NPU in the (similar board) RK3588 work with llama.cpp it isn't super useful because of the RAM constraints. I believe it has some sort of 32-bit memory addressing limit, so you can never give it more than 3 or 4 GB for example. So for LLMs, not all that useful.
It needs specific support, and for example llama.cpp would have support for some of them. But that comes with limitations in how much RAM they can allocate. But when they work, you see a flat CPU usage and the NPU does everything for inference.
Unfortunately, this board seems to be using the CIX CPU that has power management issues:
> 15W at idle, which is fairly high
Why bother with these obscure boards with spotty software support when you can get a better deal all around with an x86 mini PC with a N150 CPU?