Artificial intelligence (AI) is the latest technological innovation to capture global interest. Many companies, including tech giants like Microsoft and Google, have been early adopters and investors in the technology’s future. Large corporations are more likely to have access to the latest and greatest hardware for AI, but consumers still have options.
In this article, we will establish definitions for AI, generative AI, and GPU. Then we will review the five top-performing consumer GPUs for generative AI workloads. The top performers are all Nvidia cards, so we also compare the two top performers from AMD, Intel, and Nvidia.
Note that we haven’t had the privilege of handling and benchmarking these graphics cards ourselves. We referenced and analyzed the graphics card comparison done by Tom’s Hardware, a dependable, decades’-old website that evaluates computer hardware. At the end of the article, you can look at our sources and a chart of the 45 GPUs that Tom’s Hardware benchmarked.
What is AI, exactly?
The hype for AI is massive, and media and rumor mills have spread fear of job loss and an AI takeover. First, let’s demystify the hype and fear and gain a better understanding of AI.
At its most basic, AI is just regular computing but at a massive scale. AI depends on billions and trillions of operations happening simultaneously to make predictions and generate content. And it bases these predictions on existing data.
What is generative AI?
You’ve probably heard about DALL-E and ChatGPT by now. These are both generative AI tools that have been trained to generate “unique” content from a prompt or description. DALL-E and ChatGPT are simply software built to run massive, complex mathematical equations that create text, images, videos, code, data, or 3D rendering from existing art and text mined from the internet.
What is a GPU?
The graphics processing unit (GPU) is a specialized piece of hardware designed to support computer graphics. A discrete GPU sits on the graphics card, which slots into a PCIe slot on the motherboard. Some motherboards, like those in laptops, have an integrated GPU.
Where a central processing unit (CPU) handles general compute processes within a computer tower or server rack, a GPU is responsible for what you see on your monitor. A GPU makes your video game textures, environments, lighting, and shadows look real. The GPU works with other PC components to support video game performance.
A CPU can only process information sequentially unless it has multiple cores or technology like hyperthreading that allows for simultaneous multiple processes. GPUs run tons of processes in parallel – where a CPU might have eight cores, a GPU can have thousands.
As mentioned, AI uses an inconceivable amount of parallel processing to solve and execute complex algorithms–and “algorithm” is just another way to say calculations (solving matrices, specifically). Most computers use one CPU and one GPU, but for folks who want better graphical performance or to support multiple monitors, two or more GPUs are required. Some uses of AI often require multiple GPUs, which means slotting two or more graphics cards into your motherboard.
You can likely use generative AI on your current computer, but if you want to train a machine learning model, you will need more parallel processing power. Training requires big datasets, and processing large amounts of data takes time – unless you can harness a lot of simultaneous parallel processing.
Current state of consumer GPUS for AI
The market for graphics cards doesn’t have a ton of variety. For a long time, the two major graphics cards players have been Nvidia and AMD. Nvidia GPUs are the most promising hardware for AI. Why? Well, Nvidia has been working on its AI-supportive technology a little longer than AMD, so AMD is still playing catch-up. Intel, probably known primarily for its CPUs, has entered the proverbial arms race to build hardware that supports AI. All three companies have designed their own GPUs and supporting software.
We had intended to compare the latest graphics cards built for AI from AMD, Intel, and Nvidia. While each companies’ GPUs enable the trillions of operations per second (TOPs) required to process AI-related tasks, their chip architecture and terminology cannot be compared one-to-one. AMD uses Compute Units and accelerators. Intel uses engines to support GPU cores. And Nvidia has specific cores for specific functions. Each company provides software and community support.
AMD architecture
AMD builds Compute Units into its GPUs. Each Compute Unit is a cluster of cores and accelerators. The accelerators support ray tracing and AI workloads.
Intel architecture
Each Intel GPU includes cores and engines, grouped by slice. The GPU architecture has ray tracing acceleration built-in. The cores, called Xe-cores, use engines to handle non-traditional and compute workloads. The XMX engine supports AI workloads.
Nvidia architecture
Nvidia has worked diligently to provide a solid foundation of software and hardware for its AI offerings. Its CUDA and cuDNN libraries have allowed Nvidia to lead in AI, as these software libraries are specifically designed to leverage GPU architecture for AI workloads. Nvidia GPUs include Tensor cores, CUDA cores, ray tracing cores, and shader cores.
CUDA cores are a parallel computing platform and programming model/API developed by Nvidia for general computing on its own GPU. Ray tracing cores process ray tracing workloads. Tensor cores process AI workloads and provide high performance computing (HPC) capabilities. Tensor cores help you train machine learning models faster and improve the speed and accuracy of the model’s predictions.
So how do you measure the performance of a GPU with generative AI?
Benchmarking process
Tom’s Hardware created a benchmark to measure how quickly 45 GPUs from Nvidia, AMD, and Intel can create images with the generative AI tool, Stable Diffusion. Tom’s Hardware installed each card into the same computer and performed configurations to unlock the full potential of the GPU architecture. For Intel and AMD, the configuration processes were similar. Nvidia graphics cards required more work to configure than Intel and AMD.
Then Tom’s Hardware counted how many 512 x 512 pixel images Stable Diffusion could generate per minute.
The 5 top performing GPUs for generative AI
The five top-performing GPUS are all made by Nvidia. The top performer is the Nvidia GeForce RTX 4090, with 75 images generated per minute. The rest of the 44 GPUs don’t even come close to this number.
The other four top performing GPUs generated between 41 and 51 images. The Nvidia GeForce RTX 4080 came in second with 51 images generated per minute. Next, the Nvidia GeForce RTX 3090 Ti generated 46 images per minute. The Nvidia GeForce RTX 3090 and Nvidia GeForce RTX 3080 Ti created 42 and 41 images per minute, respectively.
The top 5 performers
| Nvidia GeForce RTX 4090 | Nvidia GeForce RTX 4080 | Nvidia GeForce RTX 3090 Ti | Nvidia GeForce RTX 3090 | Nvidia GeForce RTX 3080 Ti | |
| Rank | 1 | 2 | 3 | 4 | 5 |
| Images generated per minute | 75 | 51 | 46 | 42 | 41 |
| Memory | 24 GB GDDR6X | 16 GB GDDR6X | 24 GB GDDR6X | 24 GB GDDR6X | 12 GB GDDR6X |
| Power consumption (watts) | 450 W | 320 W | 450 W | 350 W | 350 W |
| Required power (watts) | 850 W | 750 W | 850 W | 750 W | 750 W |
The 2 top performing GPUs from AMD, Intel, and Nvidia
The AMD Radeon RX 7900 XTX is the fastest AMD performer, in 13th place with 26 images per minute. In 16th place is the AMD Radeon RX 7900 XT, with 23 images generated per minute. Intel’s best performing card, the Intel Arc A770 16GB ranks 29th with 15 images generated per minute. Intel’s next best performing card generated 13 images and ranked 30th.
| Nvidia GeForce RTX 4090 | Nvidia GeForce RTX 4080 | AMD Radeon RX 7900 XTX | AMD Radeon RX 7900 XT | Intel Arc A770 16GB | Intel Arc A770 8GB | |
| Rank | 1 | 2 | 13 | 16 | 29 | 30 |
| Images generated per minute | 75 | 51 | 26 | 23 | 15 | 13 |
| Memory | 24 GB GDDR6X | 16 GB GDDR6X | 24 GB GDDR6 | 20 GB GDDR6 | 16 GB GDDR6 | 8 GB GDDR6 |
| Base clock speed (GHz) | 2.23 | 2.21 | 2.3* | 2.0* | 2.1* | 2.1* |
| Power consumption (watts) | 450 W | 320 W | 355 W | 315 W | 225 W | 225 W |
| Required power (watts) | 850 W | 750 W | 800 W | 750 W | n/a** | n/a** |
* AMD and Intel report the expected GPU clock when running gaming applications.
** Intel does not report a recommended PSU wattage and instead suggests you determine the required power based on the cumulative power requirements of all hardware components and usual tasks/workloads performed with the computer.
The best consumer graphics card for AI
How you use your computer determines how powerful you need your graphics card to be.For gaming, streaming, creating digital art, rendering 3D models, or using generative AI, the more memory and compute power, the better.
For generative AI, GPU architecture is also important. You can see this with how much Nvidia’s cards out-performed Intel and AMD. Nvidia’s GPUs have specific cores for AI workloads, where AMD and Intel GPUs add AI capabilities with engines and accelerators. Out of 45 cards, the Nvidia GeForce RTX 4090 graphics card generated the most images per minute, at a rate of about 1.25 images every second. This is our choice as the best graphics card for AI training and the best graphics card for AI overall.
However, the Nvidia GeForce RTX 4090 also consumes and requires the most power. If you want a GPU that doesn’t consume as much power, either the Intel Arc A770 graphics cards might be a good way to go – though it’s dramatically less powerful than the Nvidia GeForce RTX 4090 and the other top performing cards.
The best performing Nvidia GPUs use the new GDDR6X RAM. GDDR6X RAM slightly improves data loss prevention as the architecture can process two bits of data at once. GDDR6 only processes one bit at a time.
When building a PC for AI, consider your power and cooling systems in addition to architecture and memory. Training and frequently using AI creates more heat and consumes more electricity than other workloads. Your PSU must provide enough power and your cooling system must keep temperatures in a safe range.
AI is still evolving, and as long as the hype for it remains, big tech is going to keep putting money into it. These graphics cards may run you a couple thousand US dollars. Intel, Google, and other companies are exploring hardware alternatives to support AI workloads, so prices will fall over time as new hardware is released. As long as you’re not training AI models, generative AI should work on any graphics card made in the last decade.
45 GPUs ranked for generative AI
We rounded the images generated per minute to the whole number for simplicity. Check out the Tom’s Hardware article in the Sources below for more details.
| Rank | GPU | Images generated per minute |
| 1 | Nvidia GeForce RTX 4090 | 75 |
| 2 | Nvidia GeForce RTX 4080 | 51 |
| 3 | Nvidia GeForce RTX 3090 Ti | 46 |
| 4 | Nvidia GeForce RTX 3090 | 42 |
| 5 | Nvidia GeForce RTX 3080 Ti | 41 |
| 6 | Nvidia GeForce RTX 4070 Ti | 41 |
| 7 | Nvidia GeForce RTX 3080 12G | 37 |
| 8 | Nvidia GeForce RTX 3080 | 36 |
| 9 | Nvidia GeForce RTX 4070 | 32 |
| 10 | Nvidia Titan RTX | 31 |
| 11 | Nvidia GeForce RTX 2080 Ti | 29 |
| 12 | Nvidia GeForce RTX 3070 Ti | 28 |
| 13 | AMD Radeon RX 7900 XTX | 26 |
| 14 | Nvidia GeForce RTX 3070 | 26 |
| 15 | Nvidia GeForce RTX 2080 Super | 24 |
| 16 | AMD Radeon RX 7900 XT | 23 |
| 17 | Nvidia GeForce RTX 4060 Ti | 23 |
| 18 | Nvidia GeForce RTX 4060 Ti 16GB | 23 |
| 19 | Nvidia GeForce RTX 3060 Ti | 23 |
| 20 | Nvidia GeForce RTX 2080 | 22 |
| 21 | Nvidia GeForce RTX 2070 Super | 21 |
| 22 | Nvidia GeForce RTX 4060 | 19 |
| 23 | Nvidia GeForce RTX 2070 | 19 |
| 24 | Nvidia GeForce RTX 2060 Super | 18 |
| 25 | Nvidia GeForce RTX 3060 | 17 |
| 26 | AMD Radeon RX 7800 XT | 17 |
| 27 | AMD Radeon RX 7700 XT | 15 |
| 28 | Nvidia GeForce RTX 2060 | 15 |
| 29 | Intel Arc A770 16GB | 15 |
| 30 | Intel Arc A770 8GB | 13 |
| 31 | Intel Arc A750 | 13 |
| 32 | Nvidia GeForce RTX 3050 | 12 |
| 33 | Intel Arc A580 | 11 |
| 34 | AMD Radeon RX 7600 | 10 |
| 35 | AMD Radeon RX 6950 XT | 6 |
| 36 | AMD Radeon RX 6900 XT | 6 |
| 37 | AMD Radeon RX 6800 XT | 6 |
| 38 | AMD Radeon RX 6800 | 4 |
| 39 | Intel Arc A380 | 4 |
| 40 | AMD Radeon RX 6750 XT | 4 |
| 41 | AMD Radeon RX 6700 XT | 3 |
| 42 | AMD Radeon RX 6700 10G | 3 |
| 43 | AMD Radeon RX 6650 XT | 2 |
| 44 | AMD Radeon RX 6600 XT | 2 |
| 45 | AMD Radeon RX 6600 | 2 |
Sources
Anadiotis, George. “AI chips for big data and machine learning: GPUs, FPGAs, and hard choices in the cloud and on-premise.” ZDNET. https://www.zdnet.com/article/ai-chips-for-big-data-and-machine-learning-gpus-fpgas-and-hard-choices-in-the-cloud-and-on-premise/. Aug. 20, 2018.
Mujtaba, Hassan. “NVIDIA GeForce RTX 3090 Ti Graphics Card Launches at $1999 US: The Fastest Ampere GPU With Insane Custom Models Now Available.” Wccf tech. https://wccftech.com/nvidia-geforce-rtx-3090-ti-launch-1999-usd-price-fastest-ampere-gpu-insane-custom-models/. Mar. 29, 2022.
Walton, Jarred. “Stable Diffusion Benchmarks: 45 Nvidia, AMD, and Intel GPUs Compared.” Tom’s Hardware. https://www.tomshardware.com/pc-components/gpus/stable-diffusion-benchmarks. Dec. 15, 2023.
“A Deeper Look at Power Consumption: GeForce RTX 4080 Power Is About More Than TGP.” Nvidia. https://www.nvidia.com/en-us/geforce/news/geforce-rtx-40-series-ultra-efficient-beyond-fast/.
“AMD Radeon RX 7900 XT.” AMD. https://www.amd.com/en/products/graphics/amd-radeon-rx-7900xt.
“AMD Radeon RX 7900 XTX.” AMD. https://www.amd.com/en/products/graphics/amd-radeon-rx-7900xtx.
“Intel Arc A770 Graphics (8GB) – Product Specifications.” Intel. https://www.intel.com/content/www/us/en/products/sku/227955/intel-arc-a770-graphics-8gb/specifications.html.
“Intel Arc A770 Graphics (16GB) – Product Specifications.” Intel. https://www.intel.com/content/www/us/en/products/sku/229151/intel-arc-a770-graphics-16gb/specifications.html.
“Introduction to the Xe-HPG Architecture.” Intel. https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-the-xe-hpg-architecture.html.
“The Ultimate GeForce GPU Comparison.” Nvidia. https://www.nvidia.com/en-us/geforce/graphics-cards/compare/.
“The Ultimate Guide: Best GPU for Machine Learning.” Toolify. Mar. 2, 2024. https://www.toolify.ai/ai-news/the-ultimate-guide-best-gpu-for-machine-learning-2518167.
