Using the same exploit that recently allowed unlocking features of Nvidia mining GPUs, an unlocker for RTX 3090 has been created. Unfortunately the card doesn’t have hidden VRAM to enable, but we can get some free floating point processing power.
The install instructions are a bit vague, but on my Fedora install I had to manually add the kernel command line arg (gpu_unlock.targets) and rebuild initramfs after running the build/install command.
I did some simple comparison using ComfyUI, and image generation became 33% faster when using fp8 and bf16 models. However, with an int8 convrot quantized model I only got around 8% speedup, which make sense since it would use the int8 tensor cores of the RTX 3090 which weren’t locked to begin with.
I didn’t write down my exact tokens/s in llama.cpp before installing the unlocker, but the numbers I got after were roughly the same as I remembered, which also makes sense since I use quantized models that afaik use integer tensor ops.
As with all hardware “hacking” you should be careful. This seemed relatively safe to me since it doesn’t permanently alter anything on the GPU so it should be easy to revert. And it shouldn’t mess with current limits or clock speeds, which could more easily damage the hardware.
The install procedure does involve changing some kernel boot parameters, so if you make a mistake it’s possible that you soft-brick your OS.
I didn’t write down my exact tokens/s in llama.cpp before installing the unlocker, but the numbers I got after were roughly the same as I remembered, which also makes sense since I use quantized models that afaik use integer tensor ops.
token generation is bottlenecked by memory speed not compute power. your prompt processing/prefill speeds might’ve gone up
Soft-locking hardware is a fucking crime. It’s anti-competitive dumping.
On the other hand, if it can be soft unlocked, that’s pretty nice.
It certainly beats AMD deliberately burning cores to maintain stratified pricing.
That example starts well enough - they were triaging failed four-core dies into three-core dies. But as yields predictably improved, they kept degrading some fraction of perfectly functional processors, because those were selling quite well, and they didn’t want to hurt the fat margins on the quad-core version. Even if they were all quad-core versions.
I’m split… A lot of the time it’s cheaper to make the low-end version of a product by locking down the high end version, due to economy of scale. If they were forced to use lower end hardware, some cheap products will end up costing more to manufacture.
I definitely think our right to tinker with stuff and unlock features should be protected, though. Along with the right to repair.
That’s what this bullshit is faking, to price out products which are genuinely cheaper.
They spent the full cost for the high-end product, but you don’t get to use it - because anchor framing lets them really gouge people for a marginally better product. Or the same goddamn product without those artificial restraints.
The reality of mass manufacturing means that, at least in some cases, it actually costs more to create a second factory line than it does to build the low end version with the the same internal hardware as the high end version. This is especially true with silicon, the vast majority of the cost is in design, test, verification, and building the fab. Once they’re manufacturing chips, each additional chip doesn’t cost all that much to produce.
Which is why… pretending you have a low-end and a high-end… is fraud.
If they had one price for that one product, other companies with other factories could offer difference price-for-value options. That competition drives prices down and value up. But it is actively prevented when the complex business strategy of lying lets them make things objectively worse on purpose, to squat across an arbitrary portion of the market, and make up whatever damn prices they want.
What do you think would happen if it was illegal? I’m only coming up with two scenarios:
- The cost of low end tech goes up because it costs more to manufacture
- Low end products are discontinued because they can’t make a profit
I just fucking explained how competition does the opposite of that.
If it costs more to manufacture a product than it can be sold for, it won’t be manufactured. Competition isn’t going to drive down the manufacturing cost, it only drives down sales price. If you drive the sales price too far down, or drive the manufacturing cost too far up (or a combination of the two), the product gets canceled.