Running your own Large Language Models (LLMs) and image generation models on local hardware offers massive benefits in terms of privacy, control, and customizability. However, the moment you initialize a model locally, your computer transforms into an intensive compute cluster. CPUs and dedicated GPUs are pushed to their absolute limits, which is directly noticeable on your energy bill.
Many developers and AI enthusiasts wonder how the power consumption of local AI inference compares to the monthly cost of a cloud subscription (such as ChatGPT Plus or Claude Pro). In this article, we break down the actual energy consumption during peak load and idle states, explain how to accurately measure this using a power meter, and present concrete calculation examples for different usage profiles.
1. How AI Hardware Consumes Power: The Difference Between Loading and Inference
To understand where energy costs come from, we need to distinguish between two phases: the idle phase (where the model is loaded into memory) and the active phase (inference, or generating text or images).
When you start an LLM via Ollama or LM Studio, for example, the model weights are loaded into your GPU's VRAM or your system's RAM. Once the model is loaded and waiting for a prompt, the processing load is minimal. In this phase, the graphics card's power consumption increases only slightly compared to standard desktop usage.
The actual power consumption occurs during inference. As soon as the AI starts processing your prompt (the context phase) and generating the token-by-token response (the generation phase), compute utilization shoots up to 100%. When selecting suitable hardware for local LLMs , the *Thermal Design Power* (TDP) of your components is therefore the decisive factor for peak load.
Explicit assumption: In all calculation examples in this article, we apply a deliberately conservative calculation assumption of 0.30 euros per kWh (reference date August 2026). At that time, the national average was around 0.27 euros per kWh, with a bandwidth of roughly 0.24 to 0.32 euros per kWh depending on contract type and supplier. Your actual costs depend on your specific energy contract and any yield from solar panels.
2. Typical power consumption by hardware category
Power consumption varies significantly by hardware type. Below is an overview of commonly used systems and their moderate to maximum power draw during inference.
| Hardware Type | System Idle (Sleep) | Inference Peak Consumption | Typical Models |
|---|---|---|---|
| Apple Silicon (M1/M2/M3 Pro/Max) | 5 - 15 Watts | 40 - 90 Watts | 7B - 32B quantized |
| Mid-range PC (RTX 3060 / 4060) | 40 - 60 Watts | 180 - 250 Watts | 7B - 13B quantized |
| High-end PC (RTX 4080 / 4090) | 60 - 90 Watts | 450 - 600 Watts | 14B - 70B quantized |
| Dual GPU Workstation (2x RTX 3090) | 100 - 150 Watts | 800 - 950 Watts | 70B FP16 / Q8 |
As shown in the table, Apple Silicon chips are extremely efficient due to their integrated Unified Memory architecture. Dedicated NVIDIA graphics cards consume significantly more power, but generally deliver higher processing speeds (more tokens per second). When you consider running an LLM locally, this trade-off between speed and efficiency will determine your final electricity bill.
3. Measuring it yourself: How to accurately track your consumption
Built-in software tools such as nvidia-smi on Linux/Windows or powermetrics on macOS provide a good indication of the power consumption of the chip itself. However, they do not measure power supply losses (PSU efficiency) or the consumption of other components such as the motherboard, cooling, and storage drives.
Step-by-step plan for accurate measurement
- Purchase a physical energy meter: Use a smart plug (for example, a Zigbee/Wi-Fi plug with power monitoring) or a simple plug-in wattmeter between the wall outlet and your PC.
- Measure the baseline (Idle): Boot the system without any AI software running. Note the wattage.
- Measure the loaded state: Load your desired AI model into memory via Ollama or Docker. If needed, refer to the guide on running LLMs in Docker for a clean setup. Check whether the idle power consumption changes.
- Perform a stress test: Send a long prompt that generates a large amount of text (for instance, a summary of a long document) and measure the sustained wattage over 2 to 3 minutes.
# Softwarematige indicatie op Linux/Windows voor NVIDIA GPU's:
nvidia-smi --query-gpu=power.draw,utilization.gpu --format=csv -l 1
# Verwachte output tijdens inferentie:
# power.draw [W], utilization.gpu [%]
# 285.42 W, 99 %4. Standby power consumption of an always-on home server
Many users set up a dedicated server or an old PC to serve AI workloads 24/7 on their local network. Here, the financial center of gravity shifts from active inference consumption to continuous standby power draw (idle consumption).
Suppose you leave a powerful desktop PC (equipped with an RTX 3090) running continuously to handle API requests via services like Open-WebUI or LocalAI.
Standby calculation example assumptions:
- Idle consumption of the complete system: 80 Watts constant power draw.
- Duration: 24 hours a day, 365 days a year = 8,760 hours.
- Electricity price: € 0.30 per kWh.
The calculation for simply keeping the server turned on:
(80 Watt * 8.760 uur) / 1000 = 700,8 kWh per jaar
700,8 kWh * € 0,30 = € 210,24 per jaar (€ 17,52 per maand)
In this scenario, simply leaving a powerful PC running idly as a home server already costs you over 17 euros per month in electricity, before generating a single token. It is therefore crucial to implement power-saving measures, such as configuring automatic sleep modes (Wake-on-LAN) or selecting energy-efficient hardware for your server.
5. Calculation examples by user profile
To determine the cost of running AI locally under actual usage, we examine three different user profiles. In all scenarios, we assume an electricity rate of € 0.30 / kWh.
Profile A: The Casual Hobbyist
Runs a local model a few times a week on a laptop or existing PC for quick questions or experiments.
- Hardware: Mid-range PC (RTX 4060, inference power consumption: 200 Watts total).
- Usage: 30 minutes of active inference per day. The PC is shut down after use.
- Daily consumption: 0.5 hours * 0.2 kW = 0.1 kWh.
- Monthly costs: 3 kWh * € 0.30 = € 0.90 per month.
Profile B: The Intensive Developer / Power User
Uses AI locally for code generation, document processing (RAG), and day-to-day workflow assistance.
- Hardware: High-end Workstation (RTX 4090, inference power consumption: 500 Watts total).
- Usage: The PC is on for 8 hours a day (including 6 hours idle at 70W and 2 hours of active inference power consumption at 500W).
- Daily consumption: (6 hours * 0.07 kW) + (2 hours * 0.5 kW) = 0.42 kWh + 1.0 kWh = 1.42 kWh.
- Monthly costs (22 working days): 31.24 kWh * € 0.30 = € 9.37 per month.
Profile C: The Self-Hosted 24/7 AI Server
A home server that runs continuously for the family or a small office, including automated background tasks.
- Hardware: Dedicated server (RTX 3090, idle power consumption 80W, inference power consumption 400W).
- Usage: 21 hours per day idle, 3 hours per day active processing spread throughout the day.
- Daily consumption: (21 hours * 0.08 kW) + (3 hours * 0.4 kW) = 1.68 kWh + 1.2 kWh = 2.88 kWh.
- Monthly costs (30 days): 86.4 kWh * € 0.30 = € 25.92 per month.
6. Running Locally versus a Cloud Subscription
When comparing the power consumption of a local setup to commercial cloud services (such as ChatGPT Plus at $ 20 / approx. € 19 per month or an API subscription), a few things stand out:
- Existing hardware: If you only use your current PC occasionally or for work (Profile A or B), the energy costs of running locally (€ 1 to € 10 per month) are significantly lower than a cloud subscription.
- Hardware depreciation: Electricity is not the only cost factor. A € 1,000 to € 2,000 graphics card under heavy load depreciates. Anyone purchasing dedicated hardware simply to save money on cloud subscriptions often faces a long payback period.
- Privacy and Unlimited Usage: With local usage, you pay per kilowatt-hour, not per token. For processing gigabytes of sensitive documents via a local RAG pipeline, running locally is superior in terms of price-to-performance ratio and privacy, regardless of power consumption. You can read more about this in the external guide on RAG for beginners at leren.llmnet.nl.
Conclusion & Practical Energy-Saving Tips
Running AI locally is less expensive in terms of power consumption than many people fear, provided you manage the hardware sensibly. Active inference consumption typically costs just a few dimes a day, as a GPU only runs at full power during actual text generation.
Three tips to keep your energy costs low:
- Power Limiting: Scale back the maximum power limit of your GPU using tools like MSI Afterburner or
nvidia-smi -pl [wattage]. Lowering the power limit by 20% often costs only 5% in generation speed, but significantly reduces heat and power consumption. - Prevent unnecessary idle consumption: Turn off servers when not in use, or set up automated sleep/standby modes.
- Choose the right quantization: Lighter, quantized models require less memory and shorter compute time per token, directly resulting in a shorter peak load on your hardware.
