When you think about graphics cards capable of efficiently running different kinds of local AI software from LLM hosting backends, through voice cloning apps, to Stable Diffusion WebUIs, you probably think of hardware you can’t really afford. What if I told you that there are some cheaper options out there which while they sometimes require a little bit more tinkering to set up, can be purchased without breaking the bank. Here are all of my data-backed propositions divided into two main categories – under $500, and under $1000.
This web portal is reader-supported, and is a part of the Aliexpress Partner Program, Amazon Services LLC Associates Program and the eBay Partner Network. When you buy using links on our site, we may earn an affiliate commission!
The Most Important Specs For AI (And The VRAM)
As I’ve already covered the most important graphics cards specs for AI in my main article containing all of the very best GPUs for local LLMs this year, I won’t be repeating much of that here. However, I’ll give you a very quick rundown of the most important GPU parameters that matter for local AI workflows.
You might be surprised, but the things that matter the most here are in that exact order: your chosen GPU’s support for standards which your selected AI software makes use of (like CUDA and ROCm), the amount of video memory (VRAM) on the card and the memory bandwidth, and only then, the max GPU clock speed.
So yes, high amount of VRAM is pretty much the most important thing to look out for when it comes to training and fine-tuning AI models locally as well as for local LLM inference, as long as you’re sure that you can actually get your chosen GPU to work with your project or particular piece of software, which can sometimes be somewhat challenging on non-NVIDIA GPUs (more on that later).
More VRAM in most AI workflows enables you to do computations much quicker as the data doesn’t need to wander between your GPU and the main system RAM. Also, in the LLM field, it lets you run larger, higher quality models.
While the larger amounts of video memory may not be that important for instance for AI image generation, in case of training or fine-tuning AI models locally as well as for instance hosting high quality large language models on your computer, it’s definitely beneficial to have as much fast VRAM as you can possibly get within your budget.
The max GPU clock speed, while important, in most cases doesn’t really matter that much on latest hardware. Most graphics cards released in the last three years or so, will almost always be fast enough to satisfy your needs in the case of most local AI workflow contexts.
Benchmarks And Different Use Cases
There exist quite a few useful AI benchmark resources online, for example ones showing local Stable Diffusion performance of different GPUs here. These can easily show you the general speed differences between different modern graphics cards.
Then there are some more, like for instance this one comparing the new Intel Arc B580 and the NVIDIA RTX 3090 in the context of local LLM training. When comparing any two GPUs together and looking for appropriate benchmarks, both Google and various online forums with user testimonies are your best friends.
Still, remember that the overall max speed differences between different cards will always matter to varying degrees, depending on the type of software you’re planning to use and your selected preferred workflows (working with diffusion models, LLMs, voice cloning, model training in any of these categories), as well as on the support for different technologies on your chosen GPU (such as CUDA on NVIDIA cards, ROCm for AMD GPUs, and so on).
Of course, things such as efficient cooling and overall reliability of your chosen GPU model are also important. That however is the case when getting a graphics card for pretty much any purpose, including gaming, and more complex productivity use cases such as 3D modelling and video editing.
With that said, after only briefly touching upon the software side of things, let’s quickly talk about the topic of software support when it comes to different GPU brands. This is arguably the thing that should be the most important for you when considering purchasing a non-NVIDIA GPU to save yourself some money in the process.
NVIDIA vs. AMD vs. Intel (Important Notes)
NVIDIA is the current leader when it comes to GPUs purchased for local AI related tasks, and that’s mainly because of their CUDA framework which all of the NVIDIA cards have native support for (and the AMD and Intel cards inherently lack), which is used by most if not all most popular AI software.
As the developers of local LLM software, Stable Diffusion WebUIs, and creators of most other kinds of programs like these are prioritizing the CUDA framework when designing their projects as it was (and still is) one of the most widespread standards for AI software, NVIDIA cards will generally be the ones which work best with such software out of the box.
So, NVIDIA graphics cards have native support for CUDA, which makes them pretty much a “plug-and-play” solution when it comes to local AI software, and both AMD and Intel GPUs don’t, as they use their own proprietary sets of instructions used for their GPUs to communicate with your system.
And so, AMD cards are compatible with their own ROCm software stack and have optional support for ZLUDA which in some cases allows the user to make use of CUDA-only programs with some performance drawbacks. Intel on the other hand, supports a traditional set of standard API’s (OpenGL, OpenCL, Vulkan, DirectX), which come with their own set of advantages and flaws.
What It Looks Like In Practice – Non-NVIDIA GPUs
The bottom line here is: most local AI software is developed with CUDA in mind first, and the support for other technologies like ROCm, DirectML, Vulkan Compute, SYCL or OpenCL is usually optional, and may or may not be implemented by the developer of a particular program.
When purchasing a non-NVIDIA GPU, be mindful of what software you plan to use with it, check its compatibility status, and in many cases be prepared for more tinkering during the software setup, less frequent updates and possible performance issues on the way.
Your experience can vary greatly depending on the programs and workflows you have in mind, as well as the specific GPU model you’ll decide to get.
In the end though, if you want to spare yourself some troubleshooting and compatibility issues, or you’re not yet sure what type of AI-related contexts you’ll be using your GPU in, it might be better to go for an NVIDIA graphics card regardless of their overall higher average market prices. YMMV.
These are all the things you should know before taking a look at the list below. Done reading? Let’s get to the main part.
Top 7 Affordable GPUs for LLMs and AI Software (Budget Picks Under $1000 & $500)
As promised, here is the full list of cards that, in my opinion, make the most sense to consider when looking for cheaper GPUs for running different types of local AI software on a limited budget. Enjoy!
Options Under $1000
1. NVIDIA RTX 4060 Ti 16GB
The NVIDIA RTX 4060 Ti 16GB is the first card on our list, and although it’s not the most efficient or the fastest one from the 4th generation of NVIDIA GPUs, it’s one of the most popular ones. While the RTX 4060 Ti has 2x lower memory bandwidth (288 GB/s) than the NVIDIA RTX 4070 (504 GB/s) which is coming up next, it does feature 4 bonus GB of VRAM on board. The choice between these two boils down to two things: the price, and how much you care about faster memory speeds in your specific use case.
Granted, for 2x the price you could have the RTX 3090 with 24GB of VRAM on board and memory bandwidth of 936 GB/s, however that would far exceed our self-imposed price limit of $500 here. Still, if you can a used RTX 3090 or a 3090 Ti would be a much better choice here in most cases.
2. NVIDIA RTX 4070 12GB
NVIDIA RTX 4070 12GB, as already mentioned, is another option you have when looking for a budget GPU for local AI tasks. The price of a new RTX 4070 is just a little bit higher than that of an RTX 4060 Ti, and while it features 4 GB of VRAM less, it has much higher memory bandwidth (almost 2x improvement), which in some contexts can make it much more efficient than the 4060.
In the end, there are many people who swear by the larger amount of slower video memory on the 4060 Ti (which can give you some advantages for instance for loading larger LLMs), and there are also many users that claim that the faster memory on the 4070 is a much better choice. This however, as already stated, will depend on your particular use contexts.
3. AMD RX 7800 XT 16GB
The AMD RX 7800 XT 16GB with its adequate amount of video memory for most basic AI-related tasks, and overall great benchmark scores is the second best card from the previous AMD Radeon GPU generation.
Provided that you’re sure you can use an AMD graphics card in your particular AI software setup, this is a really good choice performance-wise for instance for local LLMs, although it still can struggle a bit with local diffusion model-based image generation like all Radeon GPUs tend to do.
Remember that all of the quirks of both AMD and Intel Arc graphics cards in AI-related workflows that I’ve mentioned above still stand. Be mindful of them when purchasing a non-NVIDIA GPU for any kind of AI-related work.
Options Under $500
4. NVIDIA RTX 3060 12GB
While the NVIDIA RTX 3060 12GB is by no means a solution for a futureproof setup, it’s a card that’s arguably the most affordable NVIDIA GPU option for quickly getting into local AI, with sufficient VRAM for running smaller large language models locally, and with enough horsepower for relatively quick local image generation.
What makes this one even better is that you can often find it used for even lower prices (for example here, over on eBay). If you need an NVIDIA GPU, don’t want to break the bank, and are not bothered by not having the most powerful hardware available, you won’t be disappointed with this one.
5. Intel Arc B580 12GB
The Intel Arc B580 offers 12GB of video memory and the Intel’s latest GPU architecture. With very competitive pricing and promising benchmark scores, this one is among the newest and most commonly explored alternatives to both NVIDIA and AMD graphics cards both in context of productivity and gaming, and local AI software workflows. Here you can see it compared directly to the NVIDIA RTX 3090 in an LLM training task benchmark with the use of PyTorch.
This still relatively new GPU from the Intel “Battlemage” lineage is widely discussed online both in terms of its LLM inference capabilities, and performance in local image generation with Stable Diffusion and FLUX.
Both the solid performance and the very reasonable prices of the new Intel graphics cards make this one really worth considering. Once again, remember to check your software compatibility for the Intel Arc GPUs before deciding to get one of these.
Now let’s take a look at an older, but still reliable and adequate GPU from the “Alchemist” series with a little bit more video memory on board.
6. Intel Arc A770 16GB
Intel Arc A770 16GB, being the older and slightly less powerful option from the Intel GPU lineup with a larger video memory pool is also an excellent choice for a budget GPU for local AI use cases. Falling behind the B580 in terms of the overall performance (about 10-20% performance loss), it does feature an extra 4 GB of VRAM if you’re in need of that.
Paired with its price which is just as good as you might expect, it’s another strong contender when it comes to best cheap GPUs for local machine learning tasks. Still as always, be mindful of the limits imposed by the Intel’s ecosystem which might in some cases be incompatible with the software that you’re set on using.
7. AMD RX 6800 XT 16GB
The AMD RX 6800 XT 16GB is another great pick if you’re considering getting an AMD GPU and you want something that’s less expensive but still reliable in terms of processing speed and with a plenty of VRAM on board. Local LLM hosting apps like KoboldCpp have been proven to work with this one without any trouble and with reasonable speeds, and it also still does pretty well in terms of ROCm deep learning benchmarks.
The NVIDIA equivalent of this card when it comes to overall performance is the aforementioned NVIDIA GeForce RTX 4060 Ti 16GB, which in terms of gaming and productivity benchmark scores really does come rather close. If you want to save some money and your use case does make using an AMD card a viable strategy, it’s yet another one that you can seriously consider.
Bonus: A Used NVIDIA RTX 3090/Ti 24GB
I wouldn’t be myself if I didn’t include the NVIDIA RTX 3090 / Ti on this list, even as a bonus card at the very end. While the cost of this one doesn’t quite fit in our “budget” price brackets (you can see the current prices of the used units here over on eBay), this card is still among the very best GPU choices for local AI workflows this year.
24 GB of VRAM perfect for running larger LLMs, SLI compatibility, fast memory speeds and still great performance when it comes to both gaming and productivity tasks are only a few things that make this card worthwhile. If you can find it second-hand for a good price and in a good state, I wouldn’t think twice. It dominates the benchmarks both when it comes to local LLM inference, and in Stable Diffusion image generation tests, which position it right after the RTX 4070 Ti!
You can read much more about this one here! – NVIDIA GeForce 3090/Ti For AI Software – Still Worth It?