Rozalia

Rozalia lends the language model hardware sitting on your desk to the
people you choose, behind one OpenAI-compatible endpoint.

The machines with the GPUs live at home, behind NAT, where nothing on the
internet can dial them. So they dial out instead: rozalia-proxy opens a
WebSocket to the server and holds it there, and the server pushes
requests down it, to whichever machine advertises the model asked for and
has the fewest requests in flight.

This snap ships the proxy. Lending a machine takes a one-time enrollment
code. If your account on the server may already lend hardware, that is
self-service: sign in, open your account page and create one under
Enrollment codes. Otherwise ask whoever runs the server for one. Then
enrol the machine, say what to offer and where, and turn it on:

 sudo rozalia.proxyctl enroll -server https://ai.example.org \

     -code rze_...

 sudo snap set rozalia \

     proxy-server=https://ai.example.org \

     proxy-backends=http://127.0.0.1:11434/v1 \

     proxy=true

proxy-backends takes a comma-separated list, for a machine running more
than one inference server. proxy-concurrency says how many requests to
run at once, and defaults to one, because a single GPU answering two
completions at once finishes neither sooner.

The cloud half is optional and ships as a separate component, so a
machine that only lends its GPU never downloads it:

 sudo snap install rozalia+server

 sudo snap set rozalia \

     base-url=https://ai.example.org \

     listen-addr=127.0.0.1:8080

 sudo snap set rozalia server=true

 sudo rozalia.serverctl account create -email you@example.org -admin

The server does not start on install: a fresh database has no accounts,
and the address it should listen on is not something a package can
guess.

The stable channel carries tagged releases. Edge is built from every
commit to the main branch, for trying a fix before it is released.