Download the llama.cpp Docker image as a .tar

Get ghcr.io/ggml-org/llama.cpp:server, the llama.cpp inference server, as a docker-archive file from your browser. Then load it on any machine, even one with no internet access.

1 free export per email for images up to 500 MB. Pulled directly from ghcr.io, unmodified.

Load and run it on the target machine

Load the archive

docker load -i ghcr.io__ggml-org_llama.cpp__server.tar

With Podman use podman load -i. On a Kubernetes node use sudo ctr -n k8s.io images import.

Run it

docker run -d -p 8080:8080 -v /models:/models ghcr.io/ggml-org/llama.cpp:server -m /models/model.gguf

CUDA tags need the NVIDIA container runtime on the target machine. GGUF models are not included.

Popular tags

  • ghcr.io/ggml-org/llama.cpp:server
  • ghcr.io/ggml-org/llama.cpp:full
  • ghcr.io/ggml-org/llama.cpp:server-cuda

Any tag works: replace the part after the colon in the export field. Only linux/amd64 images are exported.

llama.cpp image questions

Is this the original image?

Yes. ContainerDL pulls ghcr.io/ggml-org/llama.cpp from ghcr.io and packages the layers as they are, in the same format as docker save.

Can I get an arm64 version?

Not yet. Exports are linux/amd64 only.

How long can I download the file?

Until the export expires; it is then deleted from our servers.