Tracking model updates locally: which version are you running

Within the local AI journey, tracking model updates belongs to the phase of managing and maintaining, directly following the selection, installation and operational setup of the software. Anyone starting out with local language models often begins with the fundamentals explained in the guide on hardware for local LLMs to determine how much computing power is needed. Next you work through a platform installation as described in the manual on installing Ollama on macOS to run your first prompts locally. Once this foundation is in place and applications or scripts start depending on model output, a new operational question arises: how do you know exactly which model weights are running, and how do you prevent an automatic update from disrupting your workflows?

Unlike commercial cloud services, where vendors push unannounced model changes behind a fixed API endpoint name, running locally gives you full control over the binary files. That control does come with the responsibility of maintaining strict version control yourself. A seemingly simple command such as ollama pull llama3.1 can overwrite the binary representation of the model in the background, breaking prompt templates, altering quantization artifacts or causing structured JSON output to fail.

The anatomy of a local model: tags, digests and GGUF files

To understand what happens during an update, we need to look at how local runtime environments store models. A local model consists of more than raw parameters alone; it includes a tensor layout, tokenizer vocabulary, chat templates and quantization metadata. Consult the overview on downloading and managing local models for insight into how model files are stored and organized on your disk.

Within Ollama, the storage system works with manifests and blobs, much like Docker images. When you download a model under a general tag such as qwen2.5:7b or llama3.1:latest, that tag points to a specific manifest file. This manifest contains SHA256 hashes (digests) for the individual layers: the model parameters, the template, the system parameters and the Modelfile. If the model authors release a revision — a bug fix in the chat template, say, or an alternative quantization layer — they modify the manifest in the central registry. The next time you run a pull command, the local tag reference overwrites the old digest without the tag name changing at all.

With llama.cpp and LM Studio you usually work directly with GGUF files. Here the filename is often misleading. A file named mistral-7b-instruct-v0.3.Q4_K_M.gguf tells you which quantization type was used, but not which specific git commit of the source weights underlies the conversion. If you want to understand the theory behind model compression, read the explanation in the article on quantization of LLMs, which covers the compression of floating-point variables into 4-bit or 8-bit integers in detail.

Why model regression occurs after local updates

In traditional software, a patch release (from 1.2.1 to 1.2.2) usually delivers bug fixes without breaking API contracts. With neural networks this works fundamentally differently. A small recalculation of the weights or a minor change to the stop tokens can have far-reaching consequences for the reliability of your local pipeline:

• Shifting prompt sensitivity: An instruction tune that scores better on general benchmarks may suddenly handle fixed system prompts or markdown formatting less strictly.
• Changed chat templates: If the separators between user input and the system message change (for example from <|im_start|> to [INST]), the model may forget its role or hallucinate text.
• Diverging tokenization: When special tokens for line breaks or code structures are interpreted differently, parser scripts that validate JSON output will break.
• Quantization differences: Switching from a traditional Q4_K_M quantization to a more optimal iMatrix quantization generally improves perplexity, but may diverge from earlier answers on specific domain tasks.

Anyone building autonomous AI systems professionally will find in the in-depth learning track on becoming an AI agent engineer in 2026 why determinism and version immutability are indispensable when designing reliable agent loops and tool integrations. Without pinned model versions, tracing faults in complex multi-step workflows is virtually impossible.

Inspecting the active model version in practice

To establish with certainty which weights your local server is currently executing, going by the tag name is not enough. We need to inspect the underlying hash.

Within Ollama you can request detailed layer information with the command ollama show. This shows not only the parameters, but also the exact Modelfile and the license details:

# Vraag de eigenschappen en template op van het actieve model
ollama show --modelfile llama3.1:8b

# Toon alle lokaal opgeslagen model-manifests inclusief unieke ID
ollama list

Under the hood you can inspect the specific JSON manifest on your file system. On Linux and macOS this lives by default in the home directory under ~/.ollama/models/manifests/registry.ollama.ai/library/. A manifest looks like this:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "digest": "sha256:87048abc452f357f83693e507b98d28e7e17424b918f6",
    "size": 486
  },
  "layers": [
    {
      "mediaType": "application/vnd.ollama.image.model",
      "digest": "sha256:6a0746a1ec1a7e3e30ab81682f24b20a0680cb18a",
      "size": 4920743936
    },
    {
      "mediaType": "application/vnd.ollama.image.template",
      "digest": "sha256:0ba8f0e314b4f123456789abcdef0123456789ab",
      "size": 743
    }
  ]
}

The digest of the model layer (sha256:6a0746a1...) is the cryptographic fingerprint of the GGUF weights. As long as this hash remains unchanged, you know with 100% certainty that the computational core of the model is identical, regardless of any updates to the external library.

Pinning versions with custom Modelfiles

The safest way to prevent unwanted overwrites is to decouple your production models from upstream tags. You do this by creating your own model name based on an explicit configuration. See the article on customizing the Ollama Modelfile for a complete manual on setting parameters and templates.

By copying the base model to an explicit release tag (using a date or version number, for example), you prevent a generic ollama pull from overwriting the model:

# 1. Maak een bevroren kopie aan onder een onveranderlijke tagnaam
ollama cp llama3.1:8b prod-llama-3.1-8b-20260815

# 2. Of bouw een expliciet Modelfile gebaseerd op een lokaal GGUF-bestand
cat << 'EOF' > Modelfile.production
FROM ./models/llama-3.1-8b-instruct-q4_k_m.gguf
PARAMETER temperature 0.2
PARAMETER top_p 0.9
PARAMETER stop "<|eot_id|>"
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
EOF

# 3. Compileer het model naar een strikt geïsoleerde tag
ollama create custom-llama3-frozen:v1 -f Modelfile.production

By having your applications, scripts or local agents communicate exclusively with custom-llama3-frozen:v1, you protect the stability of your production environment. Even if Ollama is updated to a newer application version in the background, the definition of this model remains intact.

Regression testing: validate before you accept

Before we admit a new model version into our daily workflow, we need to establish objectively whether the new version actually performs better and shows no regression on our specific tasks. Blindly trusting public leaderboards is risky, because they measure general knowledge and take no account of specific Dutch sentence constructions or customized extraction tasks.

For deeper conceptual background on structured testing, see the article on regression testing for prompts on the benchmark subdomain, which covers the design of test suites for model migrations. Locally, we run a compact validation suite against a fixed test file:

#!/usr/bin/env python3
"""
Eenvoudig lokaal regressietest-script voor modelvalidatie.
Draait een vaste set prompts tegen een lokaal endpoint en verifieert uitvoer.
"""
import urllib.request
import json
import time

ENDPOINT = "http://localhost:11434/api/generate"
CANDIDATE_MODEL = "llama3.1:8b"

TEST_CASES = [
    {
        "name": "JSON Structured Output Test",
        "prompt": "Geef de hoofdstad van Nederland en België in geldig JSON-formaat: {\"nl\": \"...\", \"be\": \"...\"}. Geef UITSLUITEND JSON.",
        "must_contain": ["Amsterdam", "Brussel", "nl", "be"]
    },
    {
        "name": "Nederlandse Grammatica & Instructie",
        "prompt": "Vertaal naar foutloos zakelijk Nederlands: 'The deployment failed due to an unhandled exception in the database layer.'",
        "must_contain": ["implementatie", "uitzondering", "databoselaag"] # flexibele verificatie
    }
]

def run_test(test):
    payload = json.dumps({
        "model": CANDIDATE_MODEL,
        "prompt": test["prompt"],
        "stream": False,
        "options": {"temperature": 0.0}
    }).encode("utf-8")
    
    req = urllib.request.Request(ENDPOINT, data=payload, headers={"Content-Type": "application/json"})
    t0 = time.time()
    try:
        with urllib.request.urlopen(req) as resp:
            data = json.loads(resp.read().decode("utf-8"))
            elapsed = time.time() - t0
            response_text = data.get("response", "")
            
            # Controleer inhoudelijke verwachtingen
            passed = all(item.lower() in response_text.lower() for item in test["must_contain"])
            return passed, elapsed, response_text
    except Exception as e:
        return False, 0.0, str(e)

print(f"Start validatietest voor kandidaat-model: {CANDIDATE_MODEL}\n" + "-"*60)
for t in TEST_CASES:
    success, duration, out = run_test(t)
    status = "GESLAAGD" if success else "GEFAALD"
    print(f"Test: {t['name']} -> [{status}] in {duration:.2f}s")
    if not success:
        print(f"  Foutieve uitvoer:\n{out[:200]}...")

When a model produces factual errors after an update, staying critical in your evaluation is essential; for this, see the article on fact-checking AI answers for proven methods to systematically expose hallucinations and factual mistakes.

Storage management and a safe rollback strategy

Local models demand considerable disk space. A 7B-parameter model at Q4_K_M quantization takes up roughly 4.5 to 5.0 GB, while a 70B model quickly claims 40 to 45 GB of fast NVMe storage. Download updates carelessly and within weeks your disk fills up with outdated blobs that are no longer in active use.

A well-considered storage and recovery policy is essential. In the manual on backup strategies for local models you can read how to separate and archive model weights, vector indexes and configuration files efficiently, without needless disk clutter.

To clean up unlinked layers within Ollama, use the prune mechanism. Note: never delete files manually from the blob directory, as this corrupts the internal manifest index.

# Verwijder een specifieke verouderde model-tag
ollama rm llama3:8b-instruct-q4-old

# Controleer de schijfruimte op Linux/macOS
du -sh ~/.ollama/models/

For a predictable rollback strategy we apply the N-1 principle: always keep at least one verified, older generation of the model under a unique tag name. Should the newest model show unexpected faults in production around token limits or latency, adjusting the configuration environment variable to the N-1 tag is enough to switch back within seconds and without downtime.

Privacy, network traffic and telemetry during updates

The foundation of running language models locally is data sovereignty and privacy. The moment you start an update command, however, the runtime contacts external registries (such as the Ollama Registry, Hugging Face Hub or GitHub Releases). Consult the document on using AI in a privacy-friendly way for an overview of network isolation and data security on local machines.

During a pull your machine sends metadata such as the IP address, the client version and the requested model tag to the central registry. In corporate or privacy-sensitive environments it is advisable to run updates exclusively through a shielded staging server, to scan the binary GGUF files locally for integrity using SHA256 checksums, and then to distribute them to the production machines over a local network path or an internal Docker Registry.

Careful management matters in a household setting too. The overview on using AI safely at home explains how families and home users can deploy local models without sharing private data or browsing behavior with external cloud platforms.

If you want to stay informed about broader changes in the open-source ecosystem, consult the overview page on tracking model updates and deprecations on the hub subdomain, where the release cycles of leading open architectures are analyzed systematically.

Checklist for controlled model update management

Before you roll out an update on a local system, working through a fixed checklist helps you avoid surprises:

1. Identify the current state: Note the active digest hash via ollama show or calculate the SHA256 hash of your GGUF file with sha256sum model.gguf.
2. Isolate the download: Pull the new version under an explicit candidate tag (for example kandidaat-model:v2) instead of overwriting the production tag.
3. Check the hardware requirements: Verify whether the context length and parameters have changed; a model update can quietly require more VRAM through a different context structure, causing offloading to CPU memory.
4. Run the regression test: Run an automated test suite with fixed prompts at zero temperature to compare deterministic answers.
5. Freeze the definition: Create a local Modelfile with fixed parameters and only assign the production tag once all tests have passed.
6. Keep the N-1 backup: Only remove the previous model version once the new one has proven itself trouble-free in your active workflows for at least two weeks.

By applying this systematic approach, you benefit directly from the rapid developments in open-source AI without giving up the stability and predictability of your local infrastructure.