This is an interesting review. I have a Chinese strix halo box that's isnt available in the west (favm faex1) I've been able to do some ok graphical gen, or some decent agentic tasks as a fallback for when some of the APIs are overloaded during business hours, but nothing amazing for sure, and also not both at the same time.
But here's the thing...it cost me 1800usd two months ago....and it's runs x86. I am struggling to see why people pay +2x more for the Arm Nvidia version, despite the slightly higher bandwidth it still does basically the same AI tasks and alot fewer high end general computing tasks...
I like my box but I wouldn't find it useful enough to pay more than I did for it or get more of them and cluster for instance.
Can anyone explain the allure of the Nvidia box, other than brand name?
I do appreciate how Nvidia tries to say close to vanilla with Linux and Android (nvidia shield). Instead of trying to build a shitty moat like Samsung with all their garbage software.
If nvidia ever releases Android smartphone, I'd probably stand in line to get one.
Neat blog! I was intrigued by this bullet point mentioned in passing:
>my four hard drive USB 3.2 ZFS raidz2 array with four 24 TB drives
Can you speak more about this? Which USB array did you choose? How well does it work? I've been slowly planning a transition away from my power-hungry surplus enterprise gear in the 19" rack towards a smaller, quieter, lower power setup ... but storage is the real kicker right now. I have a 12x18TB array in raidz2 built into a 1U NAS case, and I just can't quite figure out a better way to package something like that. I would need three USB arrays if I want to reuse the existing drives, which I think I do given how expensive storage is today.
It's an Orico 9948C3 with four Seagate Barracuda 24TB drives. They were on sale last year [1].
Unfortunately, the enclosure doesn't work super well on Linux.
There is a weird bug where the drives don't enumerate when I boot up my computer. This happens on both my x86_64 AMD machine running Linux, and on the DGX Spark. The solution is... simply power cycle the enclosure a couple of times by toggling the power button on it and then it works. Once all four drives show up in lsblk, I can `sudo zfs import ...` manually. This is really gross and annoying. Replacing the USB cable, flipping the USB-C cable 180 degrees, hot plugging it, etc, all didn't work, both on the DGX Spark and the other Linux machine. I've also read reports of it being unstable in UAS mode on Linux but I haven't found a big difference in stability between enabling UAS or falling back to usb-storage.
Once it starts up correctly though, the drives are fast. I store my huge amount of 100 megapixel photos on it.
The Seagate Barracudas are helium-filled HAMR/CMR drives and are apparently rebranded/binned Exos drives. They aren't rated for 24/7 use but then neither are the refurbished Exos drives.
I strongly considered it, but the one thing that scares me away from wanting to do the spark is you basically have to use nvidia's linux (from what I've read) and it doesn't appear the nvidia is interested in upstreaming their kernel changes.
I'm avoiding where possible buying electronics where support is controlled by the manufacturer and not me.
Many other distros almost work out of the box (as in they boot and run without any modifications). The custom kernel patches you mentioned address mainly non-critical bugs such as a bug where the Realtek r8127 stops working after a reboot (but it works if you turn it off and on again) [1] [2]. I'd consider it in a way better state than trying to run other Linux distros on certain device tree-based devices like, say, Qualcomm Snapdragon machines. The regular NVIDIA drivers with the open source kernel modules work just fine. Talos Linux supports DGX Spark since version 1.12 [3]. I also know of people using Fedora and nixOS successfully.
If I understand your comment correctly, I think he addresses that in the first few paragraphs:
> The DGX Spark runs “DGX OS” but it is in fact just plain old Ubuntu 24.04 with some additions. If you want, you can just install another Linux distribution easily (Fedora works well), although there may be a couple of weird bugs with the Realtek Ethernet driver so the NVIDIA version of the Linux kernel has a couple of patches. Unlike some other ARM devices, the DGX Spark is all ACPI rather than device tree based, so regular Linux builds for arm64 work just fine.
AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up.
The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.
It's interesting how many of these issues don't appear to be specific to the DGX Spark but to the standard "Nvidia GPUs suck on Linux" type of issues that afflict a lot of people.
I'm genuinely disappointed with my Spark. I don't know how anyone can claim it performs decently with LLMs or diffusion models. Back when I worked in VFX in the early 2000s, we had a saying: "Render time is coffee time" and if you try to run this thing with a usable context size, you'll be drinking a lot of coffee. Most of the optimizations it relies on for inference simply aren't available for training, so it crawls like a snail on almost every model. An RTX 6000 Blackwell would have been the better investment for an AI enthusiasts and for general computing there are cheaper offerings.
If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that.
If you just want local inference buy a Mac.
If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still the case, but you can buy a very nice Mac with $6k.
Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me.
I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.
The DGX Spark hits a sweet spot for me where it can simultaneously work for general computing and run local LLM inference fast enough for some hobbyist dabbling. The RTX 6000 Pro Blackwell is more than twice the price (it has increased quite a bit recently from $8000 to $11600), not to mention the "rest of the PC" needed to get it working, so it's not really a fair comparison. Compared to other 128 GB unified memory devices like the Mac Studio and the Strix Halo, the DGX Spark fairly priced in my opinion.
This is an interesting review. I have a Chinese strix halo box that's isnt available in the west (favm faex1) I've been able to do some ok graphical gen, or some decent agentic tasks as a fallback for when some of the APIs are overloaded during business hours, but nothing amazing for sure, and also not both at the same time. But here's the thing...it cost me 1800usd two months ago....and it's runs x86. I am struggling to see why people pay +2x more for the Arm Nvidia version, despite the slightly higher bandwidth it still does basically the same AI tasks and alot fewer high end general computing tasks... I like my box but I wouldn't find it useful enough to pay more than I did for it or get more of them and cluster for instance. Can anyone explain the allure of the Nvidia box, other than brand name?
I do appreciate how Nvidia tries to say close to vanilla with Linux and Android (nvidia shield). Instead of trying to build a shitty moat like Samsung with all their garbage software.
If nvidia ever releases Android smartphone, I'd probably stand in line to get one.
Neat blog! I was intrigued by this bullet point mentioned in passing:
>my four hard drive USB 3.2 ZFS raidz2 array with four 24 TB drives
Can you speak more about this? Which USB array did you choose? How well does it work? I've been slowly planning a transition away from my power-hungry surplus enterprise gear in the 19" rack towards a smaller, quieter, lower power setup ... but storage is the real kicker right now. I have a 12x18TB array in raidz2 built into a 1U NAS case, and I just can't quite figure out a better way to package something like that. I would need three USB arrays if I want to reuse the existing drives, which I think I do given how expensive storage is today.
It's an Orico 9948C3 with four Seagate Barracuda 24TB drives. They were on sale last year [1].
Unfortunately, the enclosure doesn't work super well on Linux. There is a weird bug where the drives don't enumerate when I boot up my computer. This happens on both my x86_64 AMD machine running Linux, and on the DGX Spark. The solution is... simply power cycle the enclosure a couple of times by toggling the power button on it and then it works. Once all four drives show up in lsblk, I can `sudo zfs import ...` manually. This is really gross and annoying. Replacing the USB cable, flipping the USB-C cable 180 degrees, hot plugging it, etc, all didn't work, both on the DGX Spark and the other Linux machine. I've also read reports of it being unstable in UAS mode on Linux but I haven't found a big difference in stability between enabling UAS or falling back to usb-storage.
Once it starts up correctly though, the drives are fast. I store my huge amount of 100 megapixel photos on it.
The Seagate Barracudas are helium-filled HAMR/CMR drives and are apparently rebranded/binned Exos drives. They aren't rated for 24/7 use but then neither are the refurbished Exos drives.
[1] https://www.reddit.com/r/buildapcsales/comments/1p29pm8/hdd_...
I strongly considered it, but the one thing that scares me away from wanting to do the spark is you basically have to use nvidia's linux (from what I've read) and it doesn't appear the nvidia is interested in upstreaming their kernel changes.
I'm avoiding where possible buying electronics where support is controlled by the manufacturer and not me.
Many other distros almost work out of the box (as in they boot and run without any modifications). The custom kernel patches you mentioned address mainly non-critical bugs such as a bug where the Realtek r8127 stops working after a reboot (but it works if you turn it off and on again) [1] [2]. I'd consider it in a way better state than trying to run other Linux distros on certain device tree-based devices like, say, Qualcomm Snapdragon machines. The regular NVIDIA drivers with the open source kernel modules work just fine. Talos Linux supports DGX Spark since version 1.12 [3]. I also know of people using Fedora and nixOS successfully.
[1] https://github.com/NVIDIA/NV-Kernels/compare/ea55925ab430f1e...
[2] https://forums.developer.nvidia.com/t/realtek-r8127-ethernet...
[3] https://github.com/siderolabs/talos/issues/12170
If I understand your comment correctly, I think he addresses that in the first few paragraphs:
> The DGX Spark runs “DGX OS” but it is in fact just plain old Ubuntu 24.04 with some additions. If you want, you can just install another Linux distribution easily (Fedora works well), although there may be a couple of weird bugs with the Realtek Ethernet driver so the NVIDIA version of the Linux kernel has a couple of patches. Unlike some other ARM devices, the DGX Spark is all ACPI rather than device tree based, so regular Linux builds for arm64 work just fine.
A good instinct. There are a lot of things a $500 AMD GPU can do in linux that a $5000 DGX cannot.
I know nothing about this topic but your comment piqued my curiosity. What would a $500 AMD GPU do better than DGX?
AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up.
The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.
It's interesting how many of these issues don't appear to be specific to the DGX Spark but to the standard "Nvidia GPUs suck on Linux" type of issues that afflict a lot of people.
it's actually very stable on x86_64 these days, even with optimus.
Super-powerful (if rather pricy) Linux desktop that happens to play games while doing everything else... that man is living my dream :'D
I'm genuinely disappointed with my Spark. I don't know how anyone can claim it performs decently with LLMs or diffusion models. Back when I worked in VFX in the early 2000s, we had a saying: "Render time is coffee time" and if you try to run this thing with a usable context size, you'll be drinking a lot of coffee. Most of the optimizations it relies on for inference simply aren't available for training, so it crawls like a snail on almost every model. An RTX 6000 Blackwell would have been the better investment for an AI enthusiasts and for general computing there are cheaper offerings.
If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that.
If you just want local inference buy a Mac.
If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still the case, but you can buy a very nice Mac with $6k.
Mhm... maybe read my full comment?
Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me.
I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.
The DGX Spark hits a sweet spot for me where it can simultaneously work for general computing and run local LLM inference fast enough for some hobbyist dabbling. The RTX 6000 Pro Blackwell is more than twice the price (it has increased quite a bit recently from $8000 to $11600), not to mention the "rest of the PC" needed to get it working, so it's not really a fair comparison. Compared to other 128 GB unified memory devices like the Mac Studio and the Strix Halo, the DGX Spark fairly priced in my opinion.
It's fairly priced but was not fairly advertised.
I would avoid the DGX Spark. For that price and its performance on running local models it is a complete scam. This tweet says it all [0]
[0] https://xcancel.com/petergostev/status/1978230978725507108
This is something I'd do if I've had the disposable income lol
>Non-Steam games have a lower chance of working
Wonder if it's true for GOG games because they are usually installed in a neatly packaged folder without any bloat.
Yes. Waiting for the Windows Version myseflf. RTX Spark Desktop.
I guess copilot needs all the help it can get?