Download the llama.cpp Docker image as a .tar
Get ghcr.io/ggml-org/llama.cpp:server, the llama.cpp inference server, as a docker-archive file from your browser. Then load it on any machine, even one with no internet access.
Sign in to start your free export
Each email address gets 1 free export, for an image up to 500 MB. Enter yours and we'll send a sign-in link, no password. Your image stays filled in when you come back.
Your export is running. Keep this page open.
The registry is sending this image very slowly. You can cancel: your credit is refunded.
Download .tar
Your exports
1 free export per email for images up to 500 MB. Pulled directly from ghcr.io, unmodified.
Load and run it on the target machine
Load the archive
docker load -i ghcr.io__ggml-org_llama.cpp__server.tar
With Podman use podman load -i. On a Kubernetes node use sudo ctr -n k8s.io images import.
Run it
docker run -d -p 8080:8080 -v /models:/models ghcr.io/ggml-org/llama.cpp:server -m /models/model.gguf
CUDA tags need the NVIDIA container runtime on the target machine. GGUF models are not included.
Popular tags
Any tag works: replace the part after the colon in the export field. Only linux/amd64 images are exported.
llama.cpp image questions
Is this the original image?
Yes. ContainerDL pulls ghcr.io/ggml-org/llama.cpp from ghcr.io and packages the layers as they are, in the same format as docker save.
Can I get an arm64 version?
Not yet. Exports are linux/amd64 only.
How long can I download the file?
Until the export expires; it is then deleted from our servers.