Hook: "Run AI locally, it's free" is only true if you ignore your battery, your fans, and your laptop's lifespan — here's where the cost actually goes, and how to measure it on your own machine.
A quick honesty note before the findings: the numbers below are general, well-documented behavior of how LLM inference loads a CPU or GPU — not a single benchmarked run on one specific laptop, which varies a lot by chip, cooling design, and model choice. Treat this as the "what to expect and how to check it yourself" piece, then run the commands in the last section on your own machine to get your actual numbers.
What "running a local LLM" actually asks your hardware to do
When you run a model through Ollama or llama.cpp, every token you generate is a full forward pass through billions of parameters. That's sustained, heavy compute for the entire time text is streaming back to you — not a brief spike like opening an app or loading a webpage. On a laptop with no discrete GPU, that load lands on the CPU. On a laptop with a discrete GPU, llama.cpp and Ollama will offload layers to it if you've built with GPU support enabled, which shifts the load (and the heat) to the GPU instead.
Model size changes how much of your machine that load touches:
- 7B parameters, 4-bit quantized (Q4_K_M) — roughly 4-5 GB of memory. Fits comfortably in 16 GB of RAM alongside a browser and an IDE.
- 13B parameters, 4-bit quantized — roughly 8-9 GB. Workable on 16 GB, tight if you're also running other memory-hungry apps.
- 70B parameters, 4-bit quantized — 40 GB or more. This is not a "run it in the background while you work" model on consumer laptop hardware; it's a dedicated-session model, if it fits at all.
Where the battery drain actually comes from
A typical ultrabook CPU has a sustained TDP (thermal design power) somewhere in the 15-28W range; a laptop discrete GPU under sustained load can pull anywhere from 60-150W depending on the class of GPU. Inference pins whichever of those is doing the work at or near 100% utilization for as long as tokens are streaming. Compare that to typical "normal use" CPU utilization — mostly single-digit to low double-digit percentages, with brief spikes — and the gap in sustained draw is the entire story. A battery rated for "10 hours of typical use" was never rated against 100% sustained compute; that number assumes the bursty, mostly-idle pattern of normal laptop usage.
Why the fans spin up and stay up
Thermal throttling isn't a bug, it's the chip protecting itself: once die temperature crosses a vendor-set threshold (commonly in the 90-100°C range for both CPUs and GPUs), the chip reduces its own clock speed to shed heat, which is why a long generation can measurably slow down partway through on thinner laptops with less cooling headroom. The fan curve on most laptops is reactive — it ramps as temperature climbs, which means a long local-LLM session isn't just loud at the start, it gets progressively louder as the chassis soaks up heat over minutes, not seconds.
How to actually measure this on your own laptop
Don't take general ranges as your answer — here's how to pull your machine's real numbers for a specific model:
# macOS — live CPU/GPU power draw, updates every second
sudo powermetrics --samplers cpu_power,gpu_power -i 1000
# Linux (Intel) — package power and per-core frequency, watch for throttling
sudo turbostat --interval 1
# Linux (NVIDIA GPU) — live power draw, temperature, utilization
nvidia-smi --query-gpu=power.draw,temperature.gpu,utilization.gpu --format=csv -l 1
# Windows — generates an HTML battery usage report covering the last 3 days
powercfg /batteryreport /output battery-report.htmlRun one of these in a second terminal while you generate a long response from a 7B model, then again with a 13B model if your hardware supports it, and compare. The gap between those two numbers — not the general ranges above — is your real answer for your machine.
What this actually costs you
Three real costs, in order of how much they'll actually bother you day to day: battery life drops fastest during active generation, not idle model-loaded time, so a laptop quoted at "10 hours" might give you a small fraction of that while actively generating long responses back to back. Fan noise and heat are the most immediately noticeable cost on thin-and-light laptops without much thermal headroom. And over a long enough horizon, sustained high-temperature cycles are a real (if slow-moving and hard to quantify precisely) factor in battery degradation and overall component longevity — the same reason sustained gaming sessions get cited in the same conversation.
None of this means local LLMs aren't worth running — offline access, data privacy, and zero per-token cost are real, genuine advantages cloud APIs can't match. It means budget for it the way you'd budget for any other sustained-compute workload: know your model size, know your hardware's actual thermal headroom, and measure before you assume.
Follow for more.
Experienced DevOps Engineer proficient in Python, Docker, Kubernetes, CI/CD, and cloud technologies. Specialized in automating processes and building scalable infrastructure for seamless development and deployment.
