View on GitHub

docker-devbox

[DEV] Docker stacks to quickly setup a dev environment and test some tools.

ollama

Containers running Ollama

Usage with docker

Ensure that GPU support is enabled in docker (or adapt docker-compose.yaml) :

docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

Start : docker compose up -d
To use Ollama CLI :

# pull models from https://ollama.com/library
docker compose exec ollama ollama pull llama3
docker compose exec ollama ollama pull gemma2
# interactive model
docker compose exec ollama ollama run llama3.1

To use Ollama API :

# list models
curl -sS http://localhost:11434/api/tags | jq -r '.models[].name'

# pull model from https://ollama.com/library
curl http://localhost:11434/api/pull -d '{
  "name": "llama3"
}'

# use model
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why is the sky blue?"
}'

To create custom model from OLLAMA Modelfile, a sample models/geoassistant is available :

docker compose exec ollama /bin/bash
ollama create geoassistant -f /models/geoassistant/Modelfile
ollama run geoassistant
# Do you know the most visited museums in Paris?

Ressources

Clients :