llama-cpp

llama.cpp packaged as a snap. The main snap ships a CPU build that
works on any x8664 or arm64 machine. Optional snap components add
GPU backends:

sudo snap install llama-cpp # CPU
sudo snap install llama-cpp+hip # add AMD GPU
sudo snap install llama-cpp+cuda # add NVIDIA GPU
sudo snap install llama-cpp+vulkan # add Vulkan GPU (Intel, AMD, etc.)
sudo snap install llama-cpp+opencl # add OpenCL GPU (Adreno, Mali, etc.)
sudo snap install llama-cpp+sycl # add SYCL GPU (Intel)

Select a backend persistently with:

sudo snap set llama-cpp backend={cpu,hip,cuda,vulkan,opencl,sycl,auto} # auto = probe
snap get llama-cpp backend

or override it per invocation with the env var:

LLAMACPP_BACKEND=cpu llama-cpp cli ...

The llama.cpp web server can also be run using the llama-cpp.server app. This
is disabled by default but can be enabled by running:

snap start --enable llama-cpp.server

By default, no additional arguments are passed to the server, meaning it will
listen on the default port and address. Arguments can be specified by setting
the server-opts configuration value, e.g.:

snap set llama-cpp server-opts="--host 0.0.0.0"

Afterwards, the server must be restarted for the new arguments to take effect:

snap restart llama-cpp.server