Ollama: why switching to streaming removes the 5-minute timeout

The problem: Ollama cuts the connection after 5 minutes on non-streamed requests.

This isn't a client-side timeout. It's the Ollama server (Go/GIN) closing the connection.

A longer client-side timeout (8 minutes, 10 minutes) changes nothing — Ollama has already hung up.

The cause

A non-streamed chat() request holds a silent connection open for the entire generation. On CPU with a 7B model and a medium-sized prompt, that can take several minutes.

The Ollama server treats this connection as dead and cuts it.

The fix

Switch to streaming. Tokens arrive as they're generated — the connection stays active the whole time.

// ❌ Non-streamed — risk of timeout after 5 min
const response = await ollama.chat({
  model,
  messages,
  options: { num_predict: 100 },
});
return response.message.content.trim();
 
// ✅ Streamed — connection active for the whole generation
const stream = await ollama.chat({
  model,
  messages,
  stream: true,
  keep_alive: -1,
  options: { num_predict: 100, num_ctx: 1024 },
});
 
let result = "";
for await (const chunk of stream) {
  result += chunk.message.content;
}
return result.trim();

The result is identical. The difference is in the transport: tokens arrive progressively instead of waiting for the end.

What keep_alive: -1 adds on top

Without keep_alive, Ollama unloads the model from RAM after 5 minutes of inactivity. The next request reloads the model from disk: 10-15 seconds of delay.

keep_alive: -1 keeps the model loaded indefinitely. For agents making regular requests, that's 10-15 seconds saved on every call.