Inference Manager

Inference Manager provides a management interface and API gateway for local inference on Ubuntu.

Install:

snap install inference

Discover providers:

inference find

Install an optimized provider:

inference install gemma4

Explore models:

inference models

Inference Manager runs an API Gateway that routes inference requests (completions, embedding, systemone, models, etc) to the local providers. The API Gateway is server at http://localhost:8400/v1 by default.

Use snap get inference and snap set inference key-value to view and modify configurations.