Start with the machine
The model choice begins with RAM, VRAM, runtime, quantization, and whether the computer can stay usable while the model answers.
Last updated
2026-07-13
This page is maintained as a hardware-specific install path, not a static model catalog.
Latest maintenance note
Added a dedicated RTX 4070 guide instead of routing every GPU user to the RTX 4090 page.
Set the default picker to 32 GB RAM and 12 GB VRAM.
Clarified that this is a hardware-fit install path, not an official GPU vendor result.
Editorial checks
This guide orders local models by usable fit, not just model size.
Local LLM choices are easy to overstate. AI Jupyter treats a hardware page as a practical install order: what to try first, what to avoid, when to step down, and when the job should move to a stronger machine or hosted API.
Separate load from comfort
A model that loads once is not automatically a good daily choice. The page favors models that still feel practical with normal apps open.
Prefer realistic limits
Large context, repository-wide coding, and long-document tasks are treated as separate workload limits instead of being hidden inside a single score.
Link to real test records
Where AI Jupyter has a real machine test, the guide links to screenshots, raw notes, and test JSON so the model pick can be checked.
Hardware decision profile
How I would treat a RTX 4070 workstation before installing models
The most useful local LLM choice is rarely the biggest model in the list. This profile turns the hardware tier into a practical decision: what to try first, what it is good for, what not to force, and when to move up.
Start with
7B to 14B compact models
12GB VRAM is the hard constraint. Use the first installs to prove speed, context length, and driver/runtime compatibility before chasing larger models.
Use it for
GPU-offloaded local inference
Good for private chat, coding tests, local API experiments, and comparing open models against hosted models with lower latency.
Do not force
24B or 32B models as default installs
A model that loads once is not enough. The guide favors models that leave room for context, tools, and repeated prompts.
Upgrade when
VRAM becomes the wall
Move up when you need larger models comfortably, multiple models loaded, batch serving, shared access, or production-style uptime.
Why this page exists
A machine-first local LLM guide for RTX 4070 workstation
Targets 12 GB VRAM, the main constraint for RTX 4070 local LLM work.
Prioritizes 7B to 14B compact models models before treating larger models as stretch tests.
Uses VRAM fit, workload match, adoption signals, and install practicality rather than a single synthetic score.
Default query
- Device
- RTX 4070 workstation
- RAM
- 32 GB
- GPU memory
- 12 GB
- Updated
- 2026-07-13
Next decision
What to check after this hardware guide
Local model choice usually changes after one of three checks: measured hardware comfort, API fallback cost, or whether the task actually needs a stronger hosted model.
Compare the 8GB GPU test
Use the measured 8GB VRAM boundary before deciding how far 12GB VRAM can stretch.
Compare coding model quality
Use this when code quality, repository fixes, or agent behavior matters more than offline control.
Compare hosted API fallback
Estimate when a hosted text model is cheaper, faster, or more reliable than forcing the local machine.
Real-world fit
What a RTX 4070 is actually comfortable doing
Good fit
- Fast local chat, coding, and reasoning with 7B to 14B compact models models.
- GPU-offloaded inference while leaving memory for IDEs, browsers, vector stores, and tools.
- Testing local models against hosted APIs when privacy, latency, or offline control matters.
Be careful
- Treating 24B, 32B, vision, or long-context workloads as a daily default without checking quantization, context, and speed.
- Assuming a model that loads once will stay responsive during long chats or agent loops.
- Ignoring system RAM, driver versions, CUDA support, disk speed, and runtime-specific model support.
When to step up
Signals this hardware tier is no longer enough.
1
You need larger models to run comfortably instead of as compromise tests.
2
You want multiple models loaded, batch serving, shared access, or production uptime.
3
Long context, vision, or agent workloads push the GPU into memory pressure.
Setup notes
How to use this guide
1
Start with 7B to 14B compact models models before pushing context length.
2
Use CUDA-aware runtimes such as Ollama, LM Studio, llama.cpp, or vLLM when the model supports them.
3
Watch VRAM during longer prompts because context and vision inputs can exhaust memory after the base model loads.
Hardware FAQ
Practical answers before you install
What is the best local LLM for a RTX 4070?
The best choice depends on workload. This page defaults to 12 GB VRAM, then sorts models by whether they fit, how well they match the selected work, adoption signals, and remaining hardware headroom.
What model size is practical on 12 GB VRAM?
Start with 7B to 14B compact models models. Treat 24B, 32B, vision, or long-context workloads as a careful test until you have checked quantization, context length, runtime speed, and your real prompts.
Can a RTX 4070 run 70B local models?
Usually not comfortably as a daily default. It may be possible with aggressive quantization or CPU offload, but smaller models are normally faster and more reliable for interactive work.
Does system RAM still matter with 12 GB VRAM?
Yes. System RAM supports the runtime, browser, IDE, vector store, model cache, and any CPU-side work. A GPU can have enough VRAM while the desktop still feels constrained.
Is this GPU guide authoritative?
No. It is a hardware-fit guide, not an official NVIDIA, Ollama, or LM Studio result. Use it to choose what to test first, then validate on your own machine.
More local LLM hardware guides
Best Local LLMs for 8GB RAM in 2026: Ollama, LM Studio, and Model Picks
Practical local LLM picks for 8GB laptops and CPU-only machines.
Best Local LLMs for 16GB RAM in 2026: Ollama and LM Studio Picks
Balanced local LLMs for 16 GB laptops and MacBooks.
Best Local LLMs for 32GB RAM in 2026: 7B, 8B, and 14B Picks
Stronger local LLMs for 32 GB RAM systems.
Best Local LLMs for RTX 5090 in 2026: 32 GB VRAM Picks
Top-end single-GPU local LLM picks for 32 GB VRAM RTX 5090 builds.
Best Local LLMs for RTX 5080 in 2026: 16 GB VRAM Picks
Practical local LLM picks for 16 GB VRAM RTX 5080 systems.
Best Local LLMs for RTX 5070 Ti in 2026: 16 GB VRAM Picks
Balanced 16 GB VRAM local LLM picks for RTX 5070 Ti machines.
Best Local LLMs for RTX 5070 in 2026: 12 GB VRAM Picks
12 GB VRAM local LLM picks for RTX 5070 desktop builds.
Best Local LLMs for RTX 5060 Ti 16GB in 2026: 16 GB VRAM Picks
16 GB VRAM local LLM picks for RTX 5060 Ti 16GB systems.
Best Local LLMs for RTX 5060 Ti 8GB in 2026: 8 GB VRAM Picks
8 GB VRAM local LLM picks for RTX 5060 Ti 8GB systems.
Best Local LLMs for RTX 5060 in 2026: 8 GB VRAM Picks
8 GB VRAM local LLM picks for RTX 5060 systems.
Best Local LLMs for RTX 4090 in 2026: 24GB VRAM Picks
High-performance local LLMs for 24 GB VRAM RTX 4090 builds.
Best Local LLMs for RTX 4080 in 2026: 16 GB VRAM Picks
16 GB VRAM local LLM picks for RTX 4080 and RTX 4080 Super systems.
Best Local LLMs for RTX 4070 Ti Super in 2026: 16 GB VRAM Picks
16 GB VRAM local LLM picks for RTX 4070 Ti Super desktops.
Best Local LLMs for RTX 4070 Ti in 2026: 12 GB VRAM Picks
12 GB VRAM local LLM picks for RTX 4070 Ti builds.
Best Local LLMs for RTX 4060 Ti 16GB in 2026: 16 GB VRAM Picks
16 GB VRAM local LLM picks for RTX 4060 Ti 16GB systems.
Best Local LLMs for RTX 4060 Ti 8GB in 2026: 8 GB VRAM Picks
8 GB VRAM local LLM picks for RTX 4060 Ti 8GB systems.
Best Local LLMs for RTX 4060 in 2026: 8 GB VRAM Picks
8 GB VRAM local LLM picks for RTX 4060 systems.
Best Local LLMs for RTX 3090 in 2026: 24 GB VRAM Picks
24 GB VRAM local LLM picks for RTX 3090 workstations.
Best Local LLMs for RTX 3080 in 2026: 10 GB VRAM Picks
10 GB VRAM local LLM picks for RTX 3080 systems.
Best Local LLMs for RTX 3070 in 2026: 8 GB VRAM Picks
8 GB VRAM local LLM picks for RTX 3070 systems.
Best Local LLMs for RTX 3060 Ti in 2026: 8 GB VRAM Picks
8 GB VRAM local LLM picks for RTX 3060 Ti systems.
Best Local LLMs for RTX 3060 in 2026: 12 GB VRAM Picks
12 GB VRAM local LLM picks for RTX 3060 systems.
Best Local LLMs for MacBook in 2026: Apple Silicon Picks
MacBook-friendly local LLMs for Apple Silicon unified memory.