Ollama

Ollama is an open-source platform for running large language models (LLMs) locally, without relying on cloud services.

Local LLM setup and management

Ollama is designed to download, run, and manage AI models on your own device, with support for NVIDIA and AMD GPUs.

• Run LLMs offline after installation
• Support for models like Llama 3.2, Mistral, and Phi-4
• Configure model parameters using Modelfile
• Developer API for integrating local models into apps
• Model version management
• Multimodal support (text and images)

In 2024, Ollama passed 130,000 downloads on GitHub.

Installation basics

Ollama works as a desktop app on macOS and Linux, with a preview version for Windows.

• Download the installer from https://ollama.com/
• Install Ollama on your computer
• Open a terminal
• Download a model with `ollama pull <model>`
• Start it with `ollama run`
• Enter prompts to get responses

Ollama is free to use, with no subscription or payment required. It supports English, and other language behavior can be adjusted via settings.