Ollama Troubleshooting: Fixing Model Download, Memory and Connection Errors

Ollama troubleshooting for the errors everyone hits: downloads that stall, out-of-memory failures, GPU not detected, models unloading, and connection refused.

By Galactic News Editorial · Published 2026-11-07T15:04:05+00:00 · Updated 2026-11-07T15:04:05+00:00

Most Ollama errors come from five causes: the model does not fit in memory, the GPU is not detected, the download stalled, the server is not running, or the model was never pulled. This guide walks through each one with the exact commands to diagnose it and fix it. Work through the checks in order. Each step takes under a minute.

Ollama is a tool for running large language models on your own computer. It downloads models, loads them into memory, and serves them through a local API on port 11434. When something breaks, the error message is often vague. The fix is usually simple once you know where to look.

Step 1: Check whether the server is running

Many errors mean the server is not running at all. A connection refused error is the clearest sign. Before anything else, check the server.

1. Run ollama list. If it prints your models, the server is up.
2. If it fails, start the server with ollama serve and leave that terminal open.
3. On Linux, check the service with systemctl status ollama. Restart it with systemctl restart ollama.
4. Test the API directly: curl http://localhost:11434/api/tags. A JSON list of models means the server answers.

By default Ollama listens on 127.0.0.1:11434. That means only your own machine can reach it. If you connect from another device, set the OLLAMA_HOST environment variable to 0.0.0.0:11434 before starting the server. Do this only on a trusted network.

Step 2: Ollama troubleshooting common errors in model downloads

A failed or stuck download usually means a network problem, low disk space, or a partial file. Model files are large, often 4 to 40 GB, so a weak connection stalls them.

1. Check disk space with df -h ~/.ollama. Free space if the drive is nearly full.
2. Remove the partial download with ollama rm <model-name>, then pull again with ollama pull <model-name>.
3. Check your connection. A VPN or proxy can block the model registry. Try again with the VPN off.
4. On a slow link, pull a smaller model first to confirm downloads work at all, for example ollama pull llama3.2:3b.

The error 'model not found' means something different. It means the model was never downloaded to this machine. Run ollama list to see what is installed. Then pull the exact name from the Ollama model library. Model names are case sensitive, and tags matter: llama3.1:8b and llama3.1:70b are different models.

If downloads always fail at the same point, change where models are stored. Set OLLAMA_MODELS to a drive with more space, then restart the server. The default location is ~/.ollama/models.

Step 3: Fix out-of-memory errors

Out-of-memory errors happen when the model weights plus the context cache exceed available memory. The fix is to shrink one of the two.

1. Use a smaller quantization. Pull the Q2_K build instead of the default, for example ollama pull llama3.1:8b-instruct-q2_K. Smaller quants use less memory at some cost to quality.
2. Use a smaller model. A 3.8B model such as Phi-3.5 needs far less memory than an 8B model.
3. Shorten the context window. Set OLLAMA_CONTEXT_LENGTH to a smaller value, such as 2048, before starting the server. A long context grows the KV cache, which also lives in memory.
4. Reduce KV cache precision. Ollama documents lower-precision cache options when Flash Attention is enabled. Its q8_0 cache uses about half the memory of an f16 cache, and q4_0 uses about a quarter.

A model that fits at startup can still run out of memory later. Long conversations grow the KV cache until it exceeds what is free. If a chat works at first and then fails, shorten the context or start a fresh chat. NVIDIA documents that a 70B model at full 16-bit precision needs about 140 GB of weight memory alone. At 4-bit quantization the same model needs about 40 GB. Match the model to your hardware before you pull it.

Step 4: Fix GPU not detected and slow generation

This is the most common Ollama problem. The model loads, but it runs on the CPU at a few tokens per second instead of on the GPU. Check where the model is actually running.

1. Run ollama ps. The Processor column shows GPU, CPU, or a split between the two.
2. Check the GPU driver with nvidia-smi. If that command fails, the driver is missing or broken. Reinstall it.
3. Start the server with debug logging: OLLAMA_DEBUG=1 ollama serve. Look for GPU lines in the output. They show whether Ollama found the card.
4. Restart Ollama after any driver change. The server only detects GPUs at startup.

Ollama's documentation lists a minimum NVIDIA driver version of 550 for current builds. Older cards with compute capability 5.0 to 6.2 need driver 570 or newer. If your card is older than that, Ollama may fall back to the CPU even with a working driver.

On AMD cards, Ollama uses ROCm on Linux. If it falls back to CPU, check the GPU documentation for your card's support status. Some older AMD cards need the HSA_OVERRIDE_GFX_VERSION variable set to a supported version.

Slow generation with the GPU detected usually means the model is split. Part of it runs on the GPU and part on the CPU. That happens when the model barely fits. Use a smaller quantization or a shorter context so the whole model fits on the GPU.

Step 5: Fix models unloading too fast

By default, Ollama keeps a model in memory for 5 minutes after its last use, then unloads it. The next prompt then waits while the model reloads. That is normal behavior, not a bug. Change it when it annoys you.

1. Set OLLAMA_KEEP_ALIVE=30m before starting the server to keep models loaded for 30 minutes.
2. Set it to -1 to keep models loaded until you stop them yourself.
3. Set it to 0 to unload each model right after every response.
4. For one request only, send keep_alive in the API call. The API value overrides the environment variable.
5. Unload a model by hand with ollama stop <model-name>. This frees memory at once.

Keep-alive matters when two models share one GPU. The first model may still sit in memory when the second one loads. The second then appears to have too little VRAM, even though the first request finished long ago. Stop the idle model before you load the next one.

Step 6: Fix connection errors from your own app

Your code talks to Ollama through its API. When your app cannot connect, the cause is usually the address or the server state.

1. Confirm the server answers with curl http://localhost:11434/api/tags.
2. Check the address your app uses. The default is http://localhost:11434. A wrong port is the most common mistake.
3. If your app runs in Docker, localhost inside the container is not your host. Use the host address or the container network address instead.
4. Check for a firewall blocking port 11434 if you connect from another machine.
5. Send a test request with curl to confirm the API itself works before you debug your code.

A 404 from the API usually means a wrong endpoint path. The generate endpoint is /api/generate and the chat endpoint is /api/chat. Check the Ollama API reference for the exact paths and fields.

Quick reference: symptoms and first fixes

Bottom line

Start with ollama ps and ollama list. They tell you whether the server runs, whether the model is installed, and whether the GPU is doing the work. Most Ollama errors resolve to one of those three checks. When memory is the problem, shrink the model or the context. When speed is the problem, confirm the GPU is detected. When downloads fail, check disk space and the network first.