Building local function calling with Ollama and Python

Within the learning cycle for local language models, we are now at the stage of applications and workflow integrations. Once a model has been set up successfully on your own machine, the need to have it perform actions follows immediately: retrieving data from a relational database, calling an internal API or inspecting files on the file system. If you still have the basic installation ahead of you, read the guide to choosing hardware for local LLMs first, and for a quick start on Apple hardware, see the step-by-step guide to installing Ollama on macOS.

Function calling turns a purely text-generating model into an operational decision engine. Instead of composing a free-form answer in prose, the language model generates a structured instruction with a function name and its parameters. In this guide we walk through the full chain: how the interaction loop works, how JSON schemas are drawn up with Pydantic, how error handling and self-correction are arranged for invalid model output, and which model choices are decisive for a reliable pipeline.

1. The anatomy of local function calling

In function calling (or tool use), the language model itself never executes program code or network requests directly. The model acts as the interpreting component that analyzes a natural-language request and translates it into a structured format the local Python application can validate and execute. To study the fundamental theory behind token segmentation and JSON representation, see the background on tool calling and structured output on the learning platform.

The interaction runs in a closed loop of four fixed steps:

1. Definition & call: The Python script sends the user message together with a list of available tools (defined through JSON schemas) to the local Ollama service.
2. Decision-making by the model: The model evaluates the context. If a tool is necessary to answer the question, it generates a JSON payload with the function name and the extracted arguments.
3. Local execution: The Python runtime parses the JSON output, validates the parameters against the data model, executes the actual local code (a database query or file scanner, for instance) and captures the result.
4. Synthesis: The function result is sent back to the model with the role tool. The model reads this context and formulates a final, natural-language answer for the user.

The great advantage of this local architecture is full control over data and privacy. Business-sensitive data, internal documents and API keys stay strictly within your own infrastructure. As described in the article on privacy-friendly AI use, not a single byte of interaction history leaves the local machine.

2. Defining JSON schemas with Python and Pydantic

Ollama expects function descriptions in a standardized JSON schema format. Hand-written dictionaries are error-prone: typos in field names or invalid JSON syntax can cause the model to generate invalid calls. Using Pydantic, data models are defined programmatically and converted automatically into flawless JSON schemas.

For quantization and model sizes, local environments generally work with 4-bit compression to balance memory usage and compute time. For a detailed explanation of how weight compression works, see the overview of quantization methods and formats.

Below is an implementation of two functions: a tool for looking up market prices and a tool for inspecting local files.

import json
from typing import List, Dict, Any
from pydantic import BaseModel, Field
import ollama

# 1. Definieer Pydantic-modellen voor parameter-validatie
class StockPriceArgs(BaseModel):
  ticker: str = Field(
    description="Het aandelensymbool, bijvoorbeeld AAPL, ASML of MSFT"
  )
  currency: str = Field(
    default="EUR",
    description="De gewenste valuta voor de prijs (EUR of USD)"
  )

class FileSearchArgs(BaseModel):
  directory: str = Field(
    description="Het absolute pad naar de te doorzoeken directory"
  )
  extension: str = Field(
    default=".log",
    description="De bestandsextensie waarop gefilterd moet worden"
  )

# 2. Converteer de definities naar het Ollama gereedschapsformaat
tools_schema = [
  {
    "type": "function",
    "function": {
      "name": "get_stock_price",
      "description": "Haal de meest recente marktprijs op voor een specifiek aandeel.",
      "parameters": StockPriceArgs.model_json_schema()
    }
  },
  {
    "type": "function",
    "function": {
      "name": "search_local_files",
      "description": "Zoek bestanden van een specifiek type binnen een lokale directory.",
      "parameters": FileSearchArgs.model_json_schema()
    }
  }
]

Keep the field descriptions (description) concise and unambiguous. Smaller open models perform considerably more stably when schemas are straight to the point and contain no contradictory terms.

3. Implementing the chat and execution loop

Interacting with the Ollama API requires an orchestration loop that checks whether the model message contains a tool_callsfield. When the model wants to call a function, the functions have to be looked up locally in a dispatch table, executed, and added to the conversation together with their output.

# Functie-implementaties
def get_stock_price(ticker: str, currency: str = "EUR") -> Dict[str, Any]:
  # Illustratieve implementatie van een externe databron
  mock_prices = {"ASML": 845.20, "AAPL": 215.50, "MSFT": 420.10}
  price = mock_prices.get(ticker.upper(), 100.0)
  return {"ticker": ticker.upper(), "price": price, "currency": currency}

def search_local_files(directory: str, extension: str = ".log") -> Dict[str, Any]:
  # Veilige mock voor bestandsinspectie
  return {
    "directory": directory,
    "found_files": [f"app_{extension.lstrip('.')}_01.log", f"app_{extension.lstrip('.')}_02.log"],
    "count": 2
  }

# Dispatch mapping tabel
TOOL_REGISTRY = {
  "get_stock_price": get_stock_price,
  "search_local_files": search_local_files
}

def run_agent_loop(user_prompt: str, model_name: str = "llama3.1:8b"):
  messages = [
    {
      "role": "system",
      "content": "Je bent een technische assistent. Gebruik de beschikbare tools wanneer dynamische of feitelijke gegevens vereist zijn."
    },
    {"role": "user", "content": user_prompt}
  ]

  client = ollama.Client()

  # Eerste aanroep naar Ollama met gereedschappen
  response = client.chat(
    model=model_name,
    messages=messages,
    tools=tools_schema,
    options={"temperature": 0.1}
  )

  messages.append(response["message"])

  # Controleer of het model gereedschap wil aanroepen
  tool_calls = response["message"].get("tool_calls", [])
  if not tool_calls:
    return response["message"]["content"]

  for tool in tool_calls:
    function_name = tool["function"]["name"]
    function_args = tool["function"]["arguments"]

    if function_name in TOOL_REGISTRY:
      # Voer de Python-functie lokaal uit
      tool_output = TOOL_REGISTRY[function_name](**function_args)

      # Voeg het resultaat toe als tool role
      messages.append({
        "role": "tool",
        "content": json.dumps(tool_output),
      })
    else:
      messages.append({
        "role": "tool",
        "content": json.dumps({"error": f"Functie {function_name} niet geregistreerd"})
      })

  # Tweede aanroep: laat het model het eindantwoord synthetiseren
  final_response = client.chat(
    model=model_name,
    messages=messages,
    options={"temperature": 0.2}
  )

  return final_response["message"]["content"]

If you are looking for an alternative that avoids writing your own Python scripts, you can automate the interaction through visual pipelines; see the article on local agentic workflows with n8n and Ollama.

4. Error handling and self-correction with local models

Unlike large commercial cloud APIs, compact local models deviate more often when generating strictly structured data. Common patterns are:

• Divergent parameter names: The model sends {"symbol": "ASML"} instead of the defined {"ticker": "ASML"}.
• Non-existent functions: The model invents a tool that does not appear in the list of definitions.
• JSON syntax errors: Missing closing braces or invalid characters in nested structures.

To keep the runtime from crashing, we build a recovery mechanism around execution. When Pydantic detects a validation error (ValidationError), we send the exact error message back to the model with a request to correct the parameters. This connects to the technique for enforcing structured JSON output.

from pydantic import ValidationError

def execute_with_recovery(model_name: str, messages: list, tool_call: dict, client: ollama.Client, max_retries: int = 2) -> dict:
  fn_name = tool_call["function"]["name"]
  raw_args = tool_call["function"]["arguments"]

  for attempt in range(max_retries):
    try:
      if fn_name == "get_stock_price":
        validated_args = StockPriceArgs(**raw_args)
        return get_stock_price(**validated_args.model_dump())
      elif fn_name == "search_local_files":
        validated_args = FileSearchArgs(**raw_args)
        return search_local_files(**validated_args.model_dump())
      else:
        raise ValueError(f"Onbekende functie '{fn_name}' aangeroepen.")

    except (ValidationError, ValueError) as err:
      if attempt == max_retries - 1:
        return {"error": f"Functieaanroep definitief mislukt na {max_retries} pogingen: {str(err)}"}

      # Voeg de validatiefout toe aan het gespreksverloop
      messages.append({
        "role": "tool",
        "content": json.dumps({
          "status": "validation_error",
          "error_details": str(err),
          "instruction": "Corrigeer de argumenten zodat ze exact voldoen aan het JSON-schema."
        })
      })

      correction_response = client.chat(
        model=model_name,
        messages=messages,
        tools=tools_schema,
        options={"temperature": 0.0}
      )
      
      new_calls = correction_response["message"].get("tool_calls", [])
      if new_calls:
        raw_args = new_calls[0]["function"]["arguments"]
      else:
        return {"error": "Model leverde geen gecorrigeerde parameters."}

5. A qualitative comparison of models for tool use

Not every open-source model has been trained with special tokens for tool use. Older architectures generated plain text that had to be parsed afterward with regular expressions. Contemporary models include specific instruction tuning for function calling.

When selecting a model, it is worth studying the benchmark methodology on testing function calling accuracy with complex schemas. The overview of choosing models for function calling also gives insight into architectural characteristics.

6. Performance with Dutch-language input

Local models are trained predominantly on English-language instruction sets. When a user asks a question in Dutch, the model performs a double translation: interpreting the Dutch intent and mapping it correctly onto English function names and JSON keys.

An illustrative practical example with a Dutch input sentence:

"Search the folder /var/log/audit for all files ending in .json and give me the statistics."

A well-tuned instruction model extracts the right structure from this:

{
  "name": "search_local_files",
  "arguments": {
    "directory": "/var/log/audit",
    "extension": ".json"
  }
}

With compact models, a parameter value may be translated unintentionally (such as .json-bestanden) or a path may not be copied literally. To optimize the reliability of Dutch interactions, apply the guidelines from the article on getting AI to perform better in Dutch. Adding clear examples within the field descriptions helps the model map Dutch terms onto the right parameters.

7. Security and the risks of local execution

Because function calling leads to actual code execution on the system, this architecture carries specific security risks. When external data — a downloaded document or web page, for instance — contains instructions that manipulate the model, the risk of so-called indirect prompt injection.

arises. Always apply the following security principles:

• No dynamic code evaluation: Avoid functions such as eval() or exec() and map function names exclusively through a fixed dispatch table.
• Strict path checking: With functions that accept file paths, always check that the target path falls within a safe directory, to prevent unauthorized access to system files.
• Separation of read and write permissions: For actions with lasting impact (deleting files or updating records, for instance), build in an explicit confirmation step before the code runs.
• Least privilege: Run the Python application under a user account with limited access rights on the host operating system.

Conclusion and next steps

Local function calling with Ollama and Python offers a solid, privacy-safe foundation for automated workflows. By validating JSON schemas with Pydantic and setting up a recovery mechanism for deviant model output, local models too can be deployed reliably in operational pipelines. For anyone looking to move on to more complex architectures with several cooperating agents, these building blocks form the foundation for safe local automation.