# Nginx reverse proxy with authentication for your LLM

[Skip to content](#lm-inhoud)Network/NL[EN](/en/)[Hubhub.llmnet.nlCompare models by task, language, cost, and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns, and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs robust in software: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlImplementing AI in an organization, from pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, for your specific tasks.](https://benchmark.llmnet.nl/en/)[Jobsvacatures.llmnet.nlAI roles, salaries, and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, from beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRunning AI privately on your own Mac, PC, NAS, or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research, and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/)[](https://x.com/intent/post?url=https%3A%2F%2Fgids.llmnet.nl%2Fnginx-reverse-proxy-met-authenticatie-voor-je-lokale-llm&text=Nginx%20reverse%20proxy%20met%20authenticatie%20voor%20je%20LLM)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fgids.llmnet.nl%2Fnginx-reverse-proxy-met-authenticatie-voor-je-lokale-llm)[](https://www.reddit.com/submit?url=https%3A%2F%2Fgids.llmnet.nl%2Fnginx-reverse-proxy-met-authenticatie-voor-je-lokale-llm&title=Nginx%20reverse%20proxy%20met%20authenticatie%20voor%20je%20LLM)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fgids.llmnet.nl%2Fnginx-reverse-proxy-met-authenticatie-voor-je-lokale-llm&text=Nginx%20reverse%20proxy%20met%20authenticatie%20voor%20je%20LLM)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fgids.llmnet.nl%2Fnginx-reverse-proxy-met-authenticatie-voor-je-lokale-llm)[](https://www.reddit.com/submit?url=https%3A%2F%2Fgids.llmnet.nl%2Fnginx-reverse-proxy-met-authenticatie-voor-je-lokale-llm&title=Nginx%20reverse%20proxy%20met%20authenticatie%20voor%20je%20LLM)[](#)

 
# Nginx reverse proxy with authentication for your local LLM

 By Ivo Donker — compiled with AI assistance (Claude & Gemini)

 Within the journey of running AI locally, this article falls into the managing and servingphase, directly following the initial installation of a model runtime. When setting up your own AI infrastructure, you first select the required components using the overview on [hardware requirements for local LLMs](https://gids.llmnet.nl/en/hardware-voor-lokale-llm) and configure the underlying host using the step-by-step guide for [running local LLMs on Linux](https://gids.llmnet.nl/en/lokale-llm-op-linux). Once backends like Ollama, llama.cpp server, or vLLM are up and running, the question inevitably arises: how can this capacity be shared reliably with other workstations, applications, or mobile devices on the network? Out of the box, without an additional intermediary layer, these tools listen without any security on ports like 11434 or 8000.

 A well-designed reverse proxy configuration with Nginx serves as the central security shield for your GPU compute. It enables you to enforce strong TLS encryption, require valid API tokens or passwords before a request ever hits the inference engine, and protect GPU memory from overload via granular rate limiting. In this article, we walk step-by-step through the configuration, the specific streaming requirements for real-time token generation, and the accompanying network architecture.

 
 Hardware and software baseline for this configuration:
 
 
- Host operating system: Linux server distribution (such as Debian 12 or Ubuntu LTS)
 
- Web server: Nginx (stable distribution release with HTTP/2 and OpenSSL support)
 
- LLM Backends: Ollama (port 11434) or vLLM (port 8000) bound to loopback
 
- Hardware reference profile: Dedicated model host with 64 GB RAM and 24 GB VRAM
 
- Model configuration: Open instruction model (such as Qwen or Llama) in Q4 quantization. For background on weight compression and memory impact, see the guide on [quantization in local language models](https://gids.llmnet.nl/en/kwantisatie-uitgelegd).
 
 

 
## Why inference engines naturally lack network security

 Software projects for local inference are primarily built for maximum computing performance and easy local integration. When Ollama or llama-server is started, the process binds by default to the local loopback address (127.0.0.1). As soon as an administrator, via environment variables such as OLLAMA_HOST=0.0.0.0 opens the port to the local network, any internal access control mechanism is lacking. There are no built-in roles, no session validity periods, and no cryptographic checks on incoming IP packets.

 Exposing an unsecured engine directly to a local corporate or home network carries significant risks. Any client on the same subnet can submit arbitrary model requests, force context windows of tens of thousands of tokens, or unload models from memory and replace them with other variants. Because a language model can demand the full capacity of a GPU during token generation, a single uncontrolled script quickly leads to total host unavailability. Furthermore, unencrypted HTTP connections run the risk of being eavesdropped on by other network devices. For a broader overview of firewall zones and network segmentation, we refer to the article on [setting up local LLM servers securely in your network](https://gids.llmnet.nl/en/lokale-llm-netwerk-beveiliging).

 
## The network architecture: Loopback binding, SSL termination, and request flow

 The most secure setup strictly separates the network layer from the compute workload. The LLM backend listens exclusively on the internal loopback interface (127.0.0.1) and rejects direct traffic from external network interfaces. Nginx acts as the single public entry point on the machine, listening on TCP port 443 (HTTPS) and port 80 (solely for a permanent HTTP 301 redirect to HTTPS).

 The interaction between client and language model differs from standard web pages on two crucial points:

 
 
- Long processing time before the first byte: Ingesting a large system prompt and context history (the prompt processing phase) takes time before the GPU returns the first predicted token. Default web server timeouts of 30 or 60 seconds can terminate connections prematurely.
 
- Continuous streaming via Server-Sent Events (SSE): User interfaces display generated words incrementally. Nginx must pass incoming data chunks directly to the client without waiting for the internal transmission buffer to fill up.
 

 
## Installing Nginx and configuring it modularly

 Installation begins by installing Nginx and the associated password management utilities via the operating system's package manager:

 # Pakketbronnen bijwerken en Nginx met utility-tools installeren
sudo apt update
sudo apt install -y nginx apache2-utils

# Verifiëren dat de service correct is gestart
sudo systemctl status nginx

 To keep configurations organized, we place the proxy definitions in a separate file under /etc/nginx/sites-available/llm-proxy.conf and activate it via a symbolic link to /etc/nginx/sites-enabled/. This prevents cluttering the main nginx.conffile and allows for quick enabling and disabling.

 
## Authentication with static Bearer tokens via Nginx map tables

 Many client applications, SDKs (such as the OpenAI Python library), and programming frameworks communicate with language models via an HTTP Authorization: Bearer <TOKEN> header. Nginx provides, with the mapdirective, an extremely efficient method to verify incoming headers in memory without needing to query an external database or auth service.

 For in-depth background on structuring token formats and separating roles, consult the guide on [authentication and authorization for your own API](https://api.llmnet.nl/en/eigen-api-authenticatie-autorisatie). We define a mapping in the HTTP block that populates a variable $api_client with an identifiable client name as soon as a valid token is provided:

 # Plaats dit configuratieblok in /etc/nginx/conf.d/llm_auth_map.conf
map $http_authorization $api_client {
 default "";
 "Bearer sk-intern-automatisering-8812" "n8n_agent";
 "Bearer sk-ontwikkeling-werkstation-4421" "dev_machine";
 "Bearer sk-beheer-laptop-3190" "beheerder";
}

 In the server block, we then verify whether the variable $api_client is set. If not, Nginx immediately terminates the connection with an HTTP 401 Unauthorized response, before the request can reach the inference engine:

 # Controle binnen het location blok
if ($api_client = "") {
 default_type application/json;
 return 401 '{"error": "Ongeldige of ontbrekende Bearer-token"}';
}

 For instructions on securely generating, rotating, and storing these tokens in development environments, read the guide on [securely managing API keys for LLMs](https://api.llmnet.nl/en/api-sleutels-veilig-beheren).

 
## Security via HTTP Basic Auth for browser and dashboard clients

 Not every application supports Bearer authentication via headers. For web browsers, static dashboards, or simple automated webhooks, HTTP Basic Auth provides a solid and widely supported security layer. We generate a password file using bcrypt encryption:

 # Nieuw bestand genereren met sterke bcrypt-hashing (-B)
sudo htpasswd -B -c /etc/nginx/.htpasswd llm_beheerder

# Bestandsrechten strikt instellen voor de Nginx-proceseigenaar
sudo chown www-data:www-data /etc/nginx/.htpasswd
sudo chmod 600 /etc/nginx/.htpasswd

 Within a specific Nginx location block, we enable authentication using two simple directives:

 auth_basic "Lokale AI Toegang Beveiligd";
auth_basic_user_file /etc/nginx/.htpasswd;

 
## Real-time token streaming: SSE, buffer management, and increased timeouts

 The primary operational bottleneck when proxying LLM traffic is Nginx's default buffering strategy. Nginx typically buffers packets from the upstream server until an internal buffer is full before forwarding them to the client. With Server-Sent Events (SSE), this causes generated tokens to be held back; the user experiences a long pause, followed by a sudden burst of multiple lines of text at once.

 To prevent this, we explicitly disable buffering using proxy_buffering off; and increase response timeouts to at least 600 seconds. For heavy inference engines like vLLM, where concurrent batching can cause temporary queuing, generous timeouts are essential. For further engine-side optimizations, consult the guide on [configuring vLLM for high throughput on Linux](https://gids.llmnet.nl/en/vllm-server-configureren).

 
 
 
 
 Nginx Parameter | 
 Default Setting | 
 Recommended for LLM | 
 Description / Impact | 
 

 
 
 
 proxy_buffering | 
 on | 
 off | 
 Disables response caching; tokens stream directly to the client in real time. | 
 

 
 proxy_read_timeout | 
 60s | 
 600s | 
 Prevents premature HTTP 504 timeouts during heavy model inference. | 
 

 
 proxy_connect_timeout | 
 60s | 
 10s | 
 Quickly detects when the upstream LLM daemon is unresponsive. | 
 

 
 proxy_http_version | 
 1.0 | 
 1.1 | 
 Required for HTTP/1.1 chunked transfer encoding and persistent connections. | 
 

 
 client_max_body_size | 
 1m | 
 64m | 
 Allows uploading large RAG documents and image prompts. | 
 

 
 
 

 
## Rate limiting and concurrency: protecting the GPU against overload

 A model server can only process a limited number of concurrent token generation requests before memory bandwidth saturates or VRAM runs out. By configuring Nginx rate limiting, we protect the GPU against sudden traffic spikes.

 We define a rate-limiting zone in the HTTP block based on the client IP ($binary_remote_addr). This sets a limit of, for example, 10 requests per minute for generative endpoints:

 # Definieer de rate limit zone in het http-blok
limit_req_zone $binary_remote_addr zone=llm_generatie:10m rate=10r/m;

 In the proxy location block, we link this zone and add a burst buffer. This allows short traffic spikes to be handled temporarily without immediately triggering an HTTP 429 error message:

 # Binnen het location / blok
limit_req zone=llm_generatie burst=3 nodelay;
limit_req_status 429;

 
## The complete Nginx configuration file for production workloads

 Below is the complete configuration file that brings together TLS encryption, Bearer token authentication, rate limiting, and optimized streaming parameters into a single, coherent server definition. Save this configuration to /etc/nginx/sites-available/llm-proxy.conf.

 # HTTP: Automatische permanente omleiding naar HTTPS
server {
 listen 80;
 listen [::]:80;
 server_name ai-server.lokaal;

 return 301 https://$host$request_uri;
}

# HTTPS: Beveiligd proxy-eindpunt
server {
 listen 443 ssl;
 listen [::]:443 ssl;
 http2 on;
 server_name ai-server.lokaal;

 # SSL Certificaten (bijv. via een interne CA of Let's Encrypt)
 ssl_certificate /etc/ssl/certs/llm-proxy.crt;
 ssl_certificate_key /etc/ssl/private/llm-proxy.key;

 # Moderne encryptieprotocollen
 ssl_protocols TLSv1.2 TLSv1.3;
 ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
 ssl_prefer_server_ciphers off;
 ssl_session_cache shared:SSL:10m;
 ssl_session_timeout 1d;

 # Essentiële beveiligingsheaders
 add_header X-Content-Type-Options nosniff always;
 add_header X-Frame-Options DENY always;
 add_header Referrer-Policy no-referrer always;

 # Toegestane payloadgrootte voor omvangrijke prompts
 client_max_body_size 64M;

 # Hoofdlocatie voor AI-inferentie
 location / {
 # 1. Authenticatiecontrole via de geconfigureerde map-tabel
 if ($api_client = "") {
 default_type application/json;
 return 401 '{"error": "Toegang geweigerd: Geen geldige API-sleutel"}';
 }

 # 2. Rate limiting toepassen
 limit_req zone=llm_generatie burst=3 nodelay;

 # 3. Doorschakeling naar de lokale LLM backend (Ollama loopback)
 proxy_pass [http://127.0.0.1:11434](http://127.0.0.1:11434);

 # 4. Verbindings- en streamingheaders
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "upgrade";
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto $scheme;
 proxy_set_header X-Client-Name $api_client;

 # 5. Uitschakelen van response-buffering voor SSE
 proxy_buffering off;
 proxy_cache off;
 chunked_transfer_encoding on;

 # 6. Time-outs afgestemd op langdurige tokengeneratie
 proxy_connect_timeout 10s;
 proxy_send_timeout 600s;
 proxy_read_timeout 600s;
 }

 # Onbeveiligd health-check eindpunt voor uptime-monitoring
 location = /health {
 auth_basic off;
 access_log off;
 default_type application/json;
 return 200 '{"status": "online", "service": "llm-proxy"}';
 }
}

 Next, activate the configuration by creating a symbolic link and testing the syntax:

 # Configuratie koppelen aan actieve sites
sudo ln -sf /etc/nginx/sites-available/llm-proxy.conf /etc/nginx/sites-enabled/

# Syntaxis en paden controleren
sudo nginx -t

# Nginx herladen zonder actieve netwerkverbindingen te onderbreken
sudo systemctl reload nginx

 
## Verification, streaming tests, and error handling with cURL

 After reloading Nginx, we verify that everything works from a remote machine on the network using two targeted cURL requests.

 Test 1: Request without authentication (must fail immediately with HTTP 401):

 curl -k -s -o /dev/null -w "%{http_code}\n" [https://ai-server.lokaal/api/generate](https://ai-server.lokaal/api/generate) \
 -H "Content-Type: application/json" \
 -d '{"model": "qwen2.5:14b", "prompt": "Test"}'
# Verwachte statuscode: 401

 Test 2: Request with a valid Bearer token and streaming output:

 curl -k -N -X POST [https://ai-server.lokaal/api/generate](https://ai-server.lokaal/api/generate) \
 -H "Authorization: Bearer sk-intern-automatisering-8812" \
 -H "Content-Type: application/json" \
 -d '{
 "model": "qwen2.5:14b",
 "prompt": "Leg in twee korte zinnen uit waarom rate limiting belangrijk is.",
 "stream": true
 }'

 During Test 2, the JSON data streams should appear on the terminal line by line without delay, confirming that proxy_buffering off; is functioning properly.

 
## Testing with Dutch prompts and validating model behavior

 Once the proxy is running stably from a technical perspective, it is advisable to evaluate response quality and latency on Dutch-language instructions. Local models can sometimes exhibit inconsistent behavior when handling complex grammar or specific terminology. For practical guidance on optimizing prompt formulation, consult the recommendations on [getting your AI to perform better in Dutch](https://gids.llmnet.nl/en/beter-nederlands).

 By running test requests with typical Dutch sentence structures and summarization tasks, you can immediately determine whether the Time-to-First-Token remains acceptable under realistic workloads.

 
## Privacy, GDPR safeguards, and strict network isolation

 Securing the entry point to your local model server is essential for guaranteeing data privacy. Because all network packets are encrypted via TLS and access is restricted exclusively to authorized clients, confidential documents and prompt data remain entirely within your own managed environment. No data leaks to third-party cloud providers. For a more in-depth look at privacy frameworks within IT infrastructures, see the article on [privacy-friendly AI use](https://gids.llmnet.nl/en/privacyvriendelijk-ai).

 
## Power consumption and operational costs of an always-on server

 At idle, an Nginx process consumes virtually no measurable system resources (just a few megabytes of RAM and minimal CPU cycles). In contrast, the physical model host housing the graphics card does draw a continuous baseline amount of power when standing by 24/7 for network requests. A modern workstation with a high-end GPU draws an average of 35 to 50 watts at idle, rising to 350 to 450 watts at the wall outlet during intensive computation. To get an accurate estimate of these ongoing energy costs, consult the calculations in the guide on [what local AI power consumption costs](https://gids.llmnet.nl/en/stroomverbruik-lokale-ai).

 
## Limitations and trade-offs when deploying Nginx

 While Nginx provides an exceptionally robust and lightweight solution for access security, the system does have some clear functional limitations as the environment scales:

 
 
- Static key management: Tokens defined in an Nginx map require a configuration reload (systemctl reload nginx) upon every addition or revocation. For dynamic teams with hundreds of users, an advanced OAuth2 or OIDC proxy offers greater flexibility.
 
- No inspection of model capacity: Nginx routes at the HTTP level and has no insight into the available VRAM capacity or the current KV cache of the LLM backend. If multiple clients send heavy prompts simultaneously, the backend can still encounter out-of-memory issues unless strict engine-level queue management is active.
 
- No per-user token accounting: Nginx logs the number of bytes and HTTP requests, but cannot measure how many context and generation tokens a specific user consumes. Imposing hard token quotas per project or department requires a dedicated AI gateway.
 

 For home labs, internal development teams, and medium-sized organizations, however, a carefully configured Nginx reverse proxy provides an excellent balance between reliable security, minimal overhead, and maximum control over their own local AI capacity.