Running AI Models on a 4 GB Laptop GPU

I ran models on my own laptop with Ollama. It has an NVIDIA GPU with 4 GB of VRAM, which is the entry class, so I stayed on small models of about 3B to 4B parameters. They load and they answer. The answers were clearly worse than the hosted assistants I use, and the laptop got hot and filled its memory while I was doing nothing else on it. I use it to see whether a thing runs, not to get work done.

What I gave it	What happened
Writing and summarising	Readable, noticeably weaker than the hosted plans on the same text
Code questions	Not where I would spend the hour; I go back to the browser for these
Checking that it runs at all	This is the part it is genuinely good for
The machine, while any of the above runs	Hot, with memory filling up

The card decides the menu

Ollama's documentation says it supports Nvidia GPUs with compute capability 5.0 or newer, on driver 550 and newer. That covers a lot of older hardware, including the class of card in my laptop, so "will it run" is usually yes.

What the card decides is which weights you can hold. With 4 GB, the models that fit comfortably are the small ones, and small is where the quality gap with a hosted model is widest. Ollama's page publishes no VRAM figure per model size, so the way to find out is to pull one and watch your memory.

What it is actually for

I keep it installed for privacy and independence. A document I would not paste into a browser can go into a local prompt without leaving the disk. Nothing counts against a quota, which matters to me because the plan I pay for stops me on several days a week, as I set out in the comparison of the two plans I use.

On my machine the output does not match a hosted model, and the independence is still worth the disk space.

Before you buy hardware for this

If the plan is to spend money on a GPU so you can stop paying a subscription, work out the sum first. A plan at $20 a month is $240 a year. A card with enough VRAM to run the bigger weights costs several times that, and it draws power, and it sits in a laptop that will throttle. My 4 GB card was already in the machine, so trying Ollama cost me an evening and nothing else, which is the right way to find out whether local models suit your work.

Test it on the exact job you want to move off a hosted plan, with the real file.