How to run LLM's locally with Ollama

Large language models (LLMs) have revolutionized the field of natural language processing (NLP), demonstrating exceptional capabilities in understanding and generating human-like text. These models, often based on transformer architectures, include pioneers like GPT-3 (Generative Pre-trained Transformer) available on platforms such as Hugging Face Transformers and OpenAI.

Among the pioneers in this domain are models like GPT-3 (Generative Pre-trained Transformer) and others available through platforms like Hugging Face Transformers and OpenAI. In this Answer, we’ll explore how to run LLMs locally with Ollama, providing you with practical insights into the process.

Running LLMs locally with Ollama

Ollama is a CLI-based tool that facilitates local LLM deployment for models like Llama, Mistral, Orca, Phi, and others. While primarily supported on Linux and macOS, Ollama can be used on Windows via WSL (Windows Subsystem for Linux).

To install Ollama on a Linux-based system, use the following command:

Otherwise, for macOS and Windows, one can download it from the official Ollama site. Once installed, initiate Ollama with any model you want by using the following command:

If the specified model hasn’t been downloaded yet, Ollama will automatically download all the necessary resources, including the model and config files, on the first run. Once the download is complete, you will see an empty line in the terminal followed by three arrows >>>, indicating that the model has been successfully downloaded and is now running.

Open-source LLMs available on Ollama

Here are some of the few open-source LLMs available on Ollama:

Note: Model compatibility depends on the hardware of your system. Larger models might not be able to run on every system due to their high resource requirements. Ensure your system meets the necessary specifications before attempting to run large models.

Let’s put our knowledge to test with some simple questions.