Hey this is my work, although the exploits etc aren't my ideas by any means.
This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.
- you will need to run your card hotter than you might be comfortable with
- the real pay-off requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
This is about the 30' series in principle, I don't have a 4070 here to test with (I do have a couple of modded 4090's) but I suspect that at 240W you are not getting the maximum out of that card. At the same time: that is probably pretty close to optimal in terms of tokens/W.
> - you will need to run your card hotter than you might be comfortable with
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-)
There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
Hey this is my work, although the exploits etc aren't my ideas by any means. This is just a way to wrap the whole thing up in a much less annoying package than having to patch the nvidia driver. I originally got inspired by the CMP90/50 windows unlocks that run as standalone UEFI modules that chainloads into the OS.
Happy to answer questions!
This one is real, but there are some caveats:
- you will need to run your card hotter than you might be comfortable with
- the real pay-off requires increasing the maximum power to 450W
- This is well above what the stock cooler was designed for so a bit more cooling would help
What's nice is that you can stack this mod on top of (or rather, below of, this one loads first) the p2p mod that has been out for a while, they don't seem to interfere with each other.
These number claim 240W locked power draw on 4070:
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...
This is about the 30' series in principle, I don't have a 4070 here to test with (I do have a couple of modded 4090's) but I suspect that at 240W you are not getting the maximum out of that card. At the same time: that is probably pretty close to optimal in terms of tokens/W.
> - you will need to run your card hotter than you might be comfortable with
Can you explain this one? On my GPU I didn't have to change the power limit. Obviously the card might spend more time at max power usage, but it's still within the limit.
If you increase the power limit on the card you will be able to get much more mileage out of this modification. At 450W the difference is much, much larger but that is not something I'm comfortable with doing for longer runs. Note that the stock settings are already pretty extreme for that cooler, and that ambient temp and free airflow are going to be impacting the resulting GPU temp in a very direct way. I have my 3090's in an external box (with its own separate PCIe backplane) so they can talk p2p and have 'fresh air' rather than in the enclosure.
I've tested myself and the results are legit, this is incredible, even if a little risky (it relies on exploiting firmware): https://github.com/Zagitta/gpu-unlock/blob/main/docs/How%20A...
> Unfortunately for Nvidia the booter is vulnerable to a classic buffer overflow that writes straight into the stack which gives us the ability to execute code via Return Oriented Programming (ROP).
Repo for RTX 4090 and 4070 Ti Super: https://github.com/JustAResearcher/gpu-unlock-ada
Really interesting.
On the other hand, this only applies to a limited set of workloads (training is often memory bandwidth bound for example).
I also wonder how this affects power consumption.
Training is compute bound.
If its memory bandwidth bound you need to raise your batch sizes to leverage more parallelism.
Uh, no. This headline is wrong. It's FP32 unlocked from being halved, not 16 bit ops. And to be clear, over the last week I've looked many places and I've not seen any reports of this unlock being applied to any 3xxx card except the 3090/3090ti series. Even if it might apply more generally in theory.
It's the FP/BF16 FMA to FP32 instructions.
And also it does work on most 3000 series cards as they all run the same GA102 booter that's vulnerable :-) There are some reports of cards with weird vbios layouts having issues but i'll try to get that fixed
The repo has a lot of reports of people applying it to other 3xxx card, see https://github.com/Zagitta/gpu-unlock/issues
There's at least 3080 Ti, 3070 Ti, 3070, 3060
The repo has fp16 numbers.
Also this one for 4070s shows that its for fp8 as well:
https://github.com/JustAResearcher/gpu-unlock-ada/releases/t...