Host your Quack suite on your own
Create a `.env` file
.env
.Edit your environment variables
.env
and replace the following values:SUPERADMIN_GH_PAT
: your Github user Personal Access Token to authenticate you as the admin. Head over to your Developer settings on GitHub, and “Generate new token”,
pick a name and an expiration and confirm with “Generate token” (no need for extra permissions i.e. read-only)POSTGRES_PASSWORD
, SUPERADMIN_PWD
, GF_ADMIN_PWD
ACME_EMAIL
: the email linked to your certificate for HTTPSPOSTGRES_HOST
& POSTGRES_PORT
: the host and port of your remote PostgreSQL database service.BACKEND_HOST
: the subdomain where your users will access your API (e.g “api.mydomain.com”)Define a Docker orchestration
docker-compose.yml
.
You can comment the deploy
section of the ollama service if you wish to use your CPU to run the LLM instead.Setting the certificate access permission
Run the service
.env
, docker-compose.yml
) in your folder. Time to start the services:POSTGRES_HOST
& POSTGRES_PORT
in your .env
file.OLLAMA_MODEL
to use other models from the hub.
For a good performance/latency balance, we recommend you use one of the following models: dolphin-mistral:7b-v2.6-dpo-laser-q4_K_M
, deepseek-coder:6.7b-instruct-q4_K_M
.
Please don’t pick oversized hardware or models for your needs, to preserve both your hardware life expectancy and your energy bills 💚