GPU
What is it?
A GPU is a processor with thousands of small cores that does many calculations at once, making it ideal for graphics and AI.
Explain like I'm 5
Why was it created?
GPUs were built to render graphics, which needs lots of parallel math. That same parallelism turned out to be perfect for training and running AI models.
Where is it used?
- Training AI models
- Running model inference
- Graphics and gaming
- Scientific computing
Why should developers care?
Modern AI runs on GPUs, so their availability and cost directly shape what's possible in machine learning.
How does it work?
A GPU has many cores optimized to perform the same operation on lots of data simultaneously. AI math — especially the matrix operations in neural networks — fits this parallel style far better than a general-purpose CPU.
Real-world example
Training a large neural network on a CPU could take months; spread across many GPUs, it can finish in days.
Common use cases
- Deep learning training
- Large-scale inference
- Graphics rendering
- Parallel number-crunching
Advantages
- Massive parallelism
- Huge speedups for AI math
- Well-supported by ML frameworks
- Scales across many units
Disadvantages
- Expensive and power-hungry
- Limited memory per unit
- Can be scarce/high-demand
- Overkill for non-parallel tasks
When should you use it?
For training or running neural networks and other heavily parallel workloads.
When should you avoid it?
For ordinary, sequential application logic, where a CPU is cheaper and sufficient.
Alternatives
Related terms
Interview questions
Beginner
- What is a GPU?
- Why is it good for AI?
Intermediate
- How does a GPU differ from a CPU?
- What kind of math do neural networks need?
Senior
- Why is GPU memory often the bottleneck for large models?
- When would a specialized accelerator beat a GPU?
Common misconceptions
- "GPUs are only for gaming" — the same parallelism powers nearly all modern AI.
- "A faster CPU can replace a GPU for AI" — CPUs lack the massive parallelism deep learning needs.
Fun facts
- GPU stands for Graphics Processing Unit.
- Neural network math is largely matrix multiplication, exactly what GPUs excel at.
Timeline
- 2010s — GPUs become the workhorse of deep learning
Learning resources
Quick summary
A GPU performs thousands of calculations in parallel, making it the workhorse for training and running AI models as well as graphics.
Cheat sheet
- Many cores, massive parallelism
- Ideal for neural-network math
- Powers AI training + inference
- Pricey and power-hungry