Ollama

Ollama is a command-line tool for running large language models on local computers. It allows users to download and run locally large language models such as Llama 2, Code Llama and other models, and supports customizing and creating their own models. This free and open-source project currently supports macOS and Linux operating systems and will support the Windows system in the future.

In addition, Ollama provides an official Docker image, making it much easier to deploy large language models using Docker containers. This ensures that all interactions with these models take place locally, without the need to send private data to third-party services. Ollama supports GPU acceleration on macOS and Linux and provides a simple command-line interface (CLI) as well as a REST API for interacting with applications.

This tool is particularly useful for developers or researchers who need to run and experiment with large language models on local machines without relying on external cloud services.