Running your own Large Language Models (LLMs) and image generation models on local hardware offers huge advantages in terms of privacy, control, and customizability. However, as soon as you initialize a model locally, your computer turns into an intensive computing cluster. CPUs and dedicated GPUs are pushed to their absolute limits, which is directly visible on your energy bill.
Many developers and AI enthusiasts wonder how the power consumption of local AI inference compares to the monthly costs of a cloud subscription (such as ChatGPT Plus or Claude Pro). In this article, we break down the actual energy consumption during peak load and inactivity (idle/sleep mode), explain how to measure this accurately with a power meter, and present concrete calculation examples per usage profile.
1. How AI hardware consumes power: The difference between loading and inference
To understand where the energy costs come from, we must distinguish between two phases: the idle phase (where the model is loaded into memory) and the active phase (inference, i.e., generating text or images).
When you start an LLM via, for example, Ollama or LM Studio, the model weights are loaded into your GPU's VRAM or your system's RAM. Once the model is loaded and waiting for a prompt, the processing load is minimal. The power consumption of the graphics card only rises slightly in this phase compared to normal desktop usage.
The actual power consumption occurs during inference. As soon as the AI starts processing your prompt (the context phase) and generating the response token-by-token (the generation phase), the load on the processing core spikes to 100%. When selecting suitable hardware for local LLMs, the Thermal Design Power (TDP) of your components is therefore the determining factor for the peak load.
Explicit assumption: In all calculation examples in this article, we use an average electricity price of € 0.30 per kWh. Your actual costs depend on your specific energy contract and any yield from solar panels.
2. Typical power consumption per hardware category
Power consumption varies greatly by hardware type. Below is an overview of commonly used systems and their moderate to maximum power consumption during inference.
| Hardware Type | System Idle (Sleep) | Inference Peak Consumption | Typical Models |
|---|---|---|---|
| Apple Silicon (M1/M2/M3 Pro/Max) | 5 - 15 Watts | 40 - 90 Watts | 7B - 32B quantized |
| Mid-range PC (RTX 3060 / 4060) | 40 - 60 Watts | 180 - 250 Watts | 7B - 13B quantized |
| High-end PC (RTX 4080 / 4090) | 60 - 90 Watts | 450 - 600 Watts | 14B - 70B quantized |
| Dual GPU Workstation (2x RTX 3090) | 100 - 150 Watts | 800 - 950 Watts | 70B FP16 / Q8 |
As the table shows, Apple Silicon chips are extremely efficient due to their integrated Unified Memory architecture. Dedicated NVIDIA graphics cards consume significantly more power, but generally deliver higher processing speeds (more tokens per second). When you are considering running an LLM locally, this trade-off between speed and efficiency determines your final electricity bill.
3. Measuring it yourself: How to map your consumption exactly?
Built-in software tools like nvidia-smi on Linux/Windows or powermetrics on macOS give a good indication of the power draw of the chip itself. However, they do not measure power supply loss (PSU efficiency) or the consumption of other components such as the motherboard, cooling, and storage drives.
Step-by-step plan for an accurate measurement
- Purchase a physical power meter: Use a smart plug (for example, a Zigbee/Wi-Fi plug with a power monitor) or a simple plug-in wattmeter between the wall outlet and your PC.
- Measure the baseline (Idle): Let the system boot up without any active AI software. Note the wattage.
- Measure the loaded state: Load your desired AI model into memory via Ollama or Docker. If needed, consult our guide on running LLMs in Docker for a clean installation. Check if the idle consumption changes.
- Run a stress test: Send a long prompt that generates a large amount of text (for example, a summary of a long document) and measure the constant wattage for 2 to 3 minutes.
# Software-based indication on Linux/Windows for NVIDIA GPUs:
nvidia-smi --query-gpu=power.draw,utilization.gpu --format=csv -l 1
# Expected output during inference:
# power.draw [W], utilization.gpu [%]
# 285.42 W, 99 %4. The idle consumption of an always-on home server
Many users set up a dedicated server or an old PC to offer AI services 24/7 within the local network. Here, the financial focus shifts from active inference consumption to continuous idle consumption.
Suppose you leave a powerful desktop PC (with an RTX 3090) running continuously to handle API requests via services like Open-WebUI or LocalAI.
Idle calculation example assumption:
- Idle consumption of the complete system: 80 Watts constant power draw.
- Duration: 24 hours a day, 365 days a year = 8,760 hours.
- Electricity price: € 0.30 per kWh.
The calculation for just keeping the server on:
(80 Watts * 8,760 hours) / 1000 = 700.8 kWh per year
700.8 kWh * € 0.30 = € 210.24 per year (€ 17.52 per month)
Just leaving a powerful PC running uselessly as a home server already costs you over 17 euros per month in electricity in this scenario, before a single token is even generated. It is therefore crucial to take energy-saving measures, such as setting up automatic sleep modes (Wake-on-LAN) or selecting energy-efficient hardware for your server.
5. Calculation examples per usage profile
To determine what running local AI costs in actual use, we look at three different user profiles. In all situations, we assume an electricity rate of € 0.30 / kWh.
Profile A: The Incidental Hobbyist
Runs a local model a few times a week on a laptop or existing PC for quick questions or experiments.
- Hardware: Mid-range PC (RTX 4060, inference consumption: 200 Watts total).
- Usage: 30 minutes of active inference per day. The PC is turned off after use.
- Daily consumption: 0.5 hours * 0.2 kW = 0.1 kWh.
- Monthly costs: 3 kWh * € 0.30 = € 0.90 per month.
Profile B: The Intensive Developer / Power User
Uses AI locally for writing code, processing documents (RAG), and daily support during work.
- Hardware: High-end Workstation (RTX 4090, inference consumption: 500 Watts total).
- Usage: The PC is on for 8 hours a day (of which 6 hours idle at 70W and 2 hours active inference consumption at 500W).
- Daily consumption: (6 hours * 0.07 kW) + (2 hours * 0.5 kW) = 0.42 kWh + 1.0 kWh = 1.42 kWh.
- Monthly costs (22 working days): 31.24 kWh * € 0.30 = € 9.37 per month.
Profile C: The Self-Hosted 24/7 AI Server
A home server that is continuously on for the family or a small office, including automated background tasks.
- Hardware: Dedicated server (RTX 3090, idle consumption 80W, inference consumption 400W).
- Usage: 21 hours per day idle, 3 hours per day active processing spread throughout the day.
- Daily consumption: (21 hours * 0.08 kW) + (3 hours * 0.4 kW) = 1.68 kWh + 1.2 kWh = 2.88 kWh.
- Monthly costs (30 days): 86.4 kWh * € 0.30 = € 25.92 per month.
6. Running locally versus a Cloud subscription
When we compare the power consumption of a local setup with commercial cloud services (such as ChatGPT Plus at $20 / approx. €19 per month or an API subscription), a few things stand out:
- Existing hardware: If you use your current PC only incidentally or for work (Profile A or B), the energy costs of running locally (€1 to €10 per month) are significantly lower than a cloud subscription.
- Hardware depreciation: Power is not the only cost factor. A graphics card costing €1,000 to €2,000 that is heavily loaded depreciates. Anyone purchasing hardware specifically to save money on cloud subscriptions often faces a long payback period.
- Privacy and Unlimited Use: With local use, you pay per kilowatt-hour and not per token. For processing gigabytes of sensitive documents via a local RAG pipeline, running locally is superior in terms of price-performance ratio and privacy, regardless of power consumption. You can read more about this in our external guide on RAG for beginners on leren.llmnet.nl.
Conclusion & Practical Tips for Energy Saving
Running local AI is less expensive in terms of power consumption than many people fear, provided you use the equipment wisely. Active inference consumption usually costs only a few cents a day, because a GPU only runs at full power during the actual generation of text.
Three tips to keep your energy costs low:
- Power Limiting: Scale back the maximum power of your GPU using tools like MSI Afterburner or
nvidia-smi -pl [wattage]. Lowering the power limit by 20% often costs only 5% in generation speed, but saves a lot in heat and power consumption. - Prevent unnecessary idle consumption: Turn off servers when they are not in use, or set up automatic standby modes.
- Choose the right quantization: Lighter, quantized models require less memory and shorter computing time per token, which directly results in a shorter peak load on your hardware.