Ollama Out of Memory? Fix Model Loading and Long-Chat Crashes

Local AI troubleshooting · Official sources checked October 2, 2026

If Ollama reports out of memory, first identify what failed: model loading, a larger request, concurrent work, or the operating system itself. Download size is not a complete memory budget, and a generic “runner terminated” message is not proof that your GPU driver is broken. The useful fix is the smallest change that makes the same failing request work reliably.

This is a source-based diagnostic guide, not a HiseHub hardware benchmark. Commands are examples for your own authorized machine. Record your installed Ollama version; defaults, allocation behavior and backend support change between releases.

Match the failure pattern before changing settings

Ollama documents queue overload separately from memory allocation failures in its FAQ. “Connection refused” belongs to a different branch: use our Open WebUI connection guide instead of trying random memory flags.

1. Establish a baseline you can reproduce

Run the read-only checks below in the environment that actually talks to your Ollama server. In a remote setup, a client version alone does not establish the server version.

ollama --version
ollama list
ollama ps
ollama show --modelfile YOUR_EXACT_MODEL_TAG

Replace the placeholder; do not paste it as a real tag. Keep the tag and model ID from the listing, plus the quantization you downloaded. The CLI reference distinguishes downloaded models from running models; the Modelfile reference explains how to inspect a model’s configuration.

Write down: OS, GPU/backend, physical memory pool, memory currently available, context, input type, loaded models and number of active requests. If a UI adds documents or tool output, record that too. A ten-word question can become a large request after retrieval and chat history are attached.

Find the server log, not only the frontend’s error toast. Ollama’s troubleshooting documentation lists platform-specific locations: ~/.ollama/logs/server.log on macOS, the server.log file inside %LOCALAPPDATA%\Ollama on Windows, the service journal on systemd Linux, and container logs for Docker. Note the failing timestamp and nearby allocation/backend messages. Redact prompts, local paths and credentials before sharing logs.

2. Test an explicit context—not whatever the UI silently chooses

There is a reason not to prescribe “Ollama always defaults to X tokens.” Its current context-length page describes hardware-dependent defaults, while other reference pages show different default examples. The installed runtime and the actual loaded context are better evidence than a remembered number.

For a POSIX-style shell, this local API example creates a small controlled request. Replace the model value with an exact text model already installed. PowerShell users can send the same JSON with their normal HTTP client; the shell quoting below is not universal.

curl http://127.0.0.1:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "YOUR_EXACT_MODEL_TAG",
    "prompt": "Reply with one short sentence about a tree.",
    "stream": false,
    "options": {"num_ctx": 4096, "num_predict": 64},
    "keep_alive": "2m"
  }'

This is not read-only: it loads the model and performs inference. Run it when you can safely use the machine’s resources. The generate endpoint documents runtime options and keep-alive. After a successful response, inspect ollama ps while the model remains loaded; check the reported context and processor split. Do not launch a second server over an already running service or expose port 11434 publicly to perform this test.

If 4K works but your original request fails: compare 8K, then larger contexts only if the task requires them. Keep the same tag, prompt type and concurrency. If the frontend fails while the direct request works, inspect its model settings and assembled input before buying hardware.

If 4K still fails: lowering context further is unlikely to be a complete answer when weights or another process already exhaust the usable pool. Move to the next checks. A successful one-sentence answer proves only that this baseline runs—not that your original task fits.

3. Separate weight size from the memory the request needs

Use this planning relationship, not a promise of measured usage:

Required pool ≈ resident weights + context cache + runtime/workspace overhead + concurrent work.

A downloaded file estimates only part of the problem. Dedicated VRAM and host RAM are separate budgets; adding “12 GB GPU + 32 GB RAM” and calling it 44 GB of fast GPU memory is misleading. Apple unified memory is one shared pool, not RAM plus an extra GPU allowance.

Our VRAM calculator and model finder makes these pool boundaries explicit. It covers a small verified catalog, uses an FP16 KV-cache formula for supported architectures and includes a user-selected runtime buffer. It intentionally does not estimate unsupported hybrid-attention/vision entries.

Illustrative calculation, not a hardware test: the catalog’s qwen3:8b-q4_K_M snapshot uses about 4.87 GiB of model-layer data. Its supported FP16 KV formula estimates 0.56 GiB at 4K context and 4.50 GiB at 32K. With the calculator’s 0.50 GiB buffer, the planning totals are approximately 5.93 and 9.87 GiB. The weight file did not grow; the assumed context budget changed. Extra images, backend workspaces or parallel requests are not proved safe by these totals.

For a broader explanation of model size versus usable hardware, see our Ollama RAM and VRAM requirements guide. If Ollama unexpectedly runs on CPU, use the GPU detection guide; that can be a backend issue rather than insufficient capacity.

4. Remove competing work without deleting models or data

Check ollama ps for other resident models. A chat interface may use a separate embedding model or another background task even when you see only one chat window. Close or pause only the workloads you control.

ollama stop EXACT_UNUSED_MODEL_TAG

The stop command unloads a running model; it is not ollama rm. On a shared server, coordinate first: unloading or restarting can affect other users and trigger a later reload. Do not delete model directories, Docker volumes, chat databases or configuration files as an OOM fix.

Now repeat the original failing request once. If it succeeds only after competing work is removed, you have evidence of a capacity/scheduling conflict. “It works after reboot” alone does not identify which competing process caused it.

5. Test concurrency separately from context

Try the same requests one after another. If they work serially but fail together, reduce simultaneous work at the caller and check the server’s effective parallel configuration. Do not solve this by silently shrinking every user’s context.

Ollama’s current scheduler implementation combines effective context with parallel slots for its llama-server path; it also limits parallelism for some architectures. That implementation can change and is not a universal formula for every backend. Review OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS for your installed version; use ollama serve --help to check the available settings.

Apply server settings to the process that actually launches Ollama—not an unrelated terminal after the service has started. Save the previous values and schedule a safe restart if required. A useful result is “same model and context work with serialized requests,” not merely “the error went away after I changed five flags.”

Why a long chat can fail after short answers worked

Check whether the longer conversation changed the requested context, added retrieved text or images, or overlapped another request. Also check whether the model was reloaded with different options.

Do not assume that every additional turn grows GPU allocation in a perfectly straight line. A runner may reserve context memory during loading; different backends manage it differently. Logs showing the effective context and a reload are more useful than the number of visible messages.

A smaller context is a trade-off: older material may be truncated or no longer fit. For a document workflow, compare the answer against the required evidence after reducing context. A technically stable chat that omits the important document section is not a successful fix.

6. If the process was killed, inspect the host/container boundary

A container may have much less usable RAM than the host. Docker’s resource constraints guide explains memory limits and OOM behavior. Check the existing deployment’s limit and host pressure; do not disable the OOM killer or remove limits blindly on a shared machine.

For a Docker deployment, these commands inspect existing state. Replace the placeholder with the actual container name.

docker logs --tail 100 YOUR_OLLAMA_CONTAINER
docker inspect --type container \
  --format '{{.State.OOMKilled}} {{.State.ExitCode}} {{.HostConfig.Memory}}' \
  YOUR_OLLAMA_CONTAINER

The inspect reference covers formatting, and the Engine API reference documents OOMKilled. Read the result alongside logs and the applicable container/VM limits. Docker documents that exit 137 indicates SIGKILL and has several possible causes; that exit code alone does not prove an OOM kill.

7. Quantization is a fallback with two different meanings

Weight quantization changes the model artifact. KV-cache quantization changes the context cache. Downloading a Q4 weight file does not establish that the active KV cache is also Q4.

Ollama’s KV-cache documentation describes OLLAMA_KV_CACHE_TYPE, with f16, q8_0 and q4_0. Quantized KV requires Flash Attention; support depends on the backend/devices. The stated approximate half/quarter cache sizes are not half/quarter of total model memory. Precision and task quality can change, particularly with longer context.

Our practical order is: establish the low-context baseline, remove competing work, isolate concurrency, then consider cache quantization or a smaller/differently quantized weight artifact. Save the old cache setting. Validate the same long-context task for answer quality as well as stability before keeping the change. A smaller model that answers correctly is preferable to an unstable large model, but this guide does not rank model quality.

When is the problem actually fixed?

• The exact previously failing request completes—not just a tiny substitute.
• The loaded context and processor placement match your intended configuration.
• The required document or conversation evidence still survives context reduction.
• The intended concurrency works without repeated reloads, crashes or overload.
• You retained a before/after record and can restore the previous setting.

If it still fails with a small text request, one model and no competing work, collect a minimal issue report: runtime version, exact tag/ID, backend, available memory, request options and redacted loading logs. Check for an applicable version-specific fix before upgrading; preserve a rollback path. Do not report a calculator estimate as a measured peak or claim that a driver reinstall resolved an allocation failure unless the comparison actually demonstrates it.