Why use it

On-device LLM inference powered by X-Bit quantization for efficient, self-hosted deployment of language models.

Best for

developers · homelab

Deployment options

compose · docker

Resource requirements

moderate

Alternative to

No mapping yet

Common setup stack

Reverse proxy · HTTPS certs · auth gateway · backups