Ollama

Ollama (local)

Ollama serves open-weight language models on the user's machine. This bundle reaches the HTTP API on localhost through MCPBundles Desktop — not a hosted inference cloud.

When to use local Ollama

Use local Ollama when the user wants private, on-device work the host model should delegate:

• Coding or reasoning passes on a specialist local model (smaller/faster than the host).
• Embeddings for RAG or similarity without sending text to a remote API.
• Vision on multimodal models when images must stay on the machine.
• Model management — list, pull, copy, create custom tags, or remove weights from chat.

Do not call local chat for every turn when the host model can answer directly. Prefer embeddings and specialist delegation over nested full conversations.

For hosted ollama.com inference, web search, or web fetch, use Ollama Cloud instead of this bundle.

Runtime requirements

• Ollama must be installed and running on the user's machine (menu bar app or ollama serve).
• Default API port is 11434. Match the port in the MCPBundles connection if the user changed it.
• MCPBundles Desktop must run on the same machine so the backend can forward HTTP to localhost.
• At least one model should be pulled locally before chat or embed calls (use list + pull workflow).

Typical workflows

1. Health check — confirm version and which models are currently loaded in memory.
2. Inventory — list downloaded models before choosing a name.
3. Pull — download a missing tag; large weights can take many minutes.
4. Copy or create — duplicate an existing tag or build a custom variant from a base model and system prompt.
5. Chat or generate — single-turn or multi-message chat with stream handled internally (one JSON response).
6. Embed — vectorize one string or a batch for search/RAG pipelines.

Gotchas

• Local hardware — speed depends on GPU/CPU RAM; first load after idle includes model load time.
• No built-in auth — default Ollama binds to localhost. Do not expose port 11434 to the public internet.
• Long pulls and generations — multi-GB pulls and large num_predict values can run many minutes; use explicit timeout parameters on slow operations.
• Model tags — names use model:tag form (for example llama3.2:latest); omitting the tag defaults to latest.
• Delete is permanent — removing a model frees disk space; the weight must be pulled again to restore.