kanto

NixOS and k3s configuration for shared inference and applications. Kanto runs Ollama, Lemonade, ComfyUI, Open WebUI, and Open Terminal on Vermilion.

See also Hoenn for personal machines, Johto for the homelab, and Sinnoh for production services.

Architecture

Host	Role
vermilion	k3s server, Ollama, Lemonade with Strata, ComfyUI, and backups

Open WebUI provides browser chat and an authenticated Chat Completions API. It connects to Ollama and Lemonade on Vermilion. Open Terminal runs code for WebUI in persistent workspaces. ComfyUI runs on the host with a shared workspace and job queue.

Separate Tailscale ingress nodes provide private HTTPS access to Open WebUI and ComfyUI. Vermilion uses local k3s networking.

Comin reconciles host configuration from master. Flux reconciles the Kubernetes resources in k8s/flux-system/ from the same branch.

Prometheus and Loki store metrics and logs on Vermilion for 30 days. Alloy collects host and Kubernetes telemetry. Use the Kanto data sources in the shared Grafana on Canalave.

See Use Kanto models for service access and client setup.

Repository layout

nix/
├── hosts/nixos/       Host configuration and hardware state
├── modules/           Host defaults and shared features
├── devShells.nix      Development tools
└── treefmt.nix        Formatting
k8s/                  Flux and application manifests
secrets/              SOPS-encrypted host secrets
keys/                 Public SOPS recipients
docs/                 Cluster operations and inference setup

flake.nix recursively imports the Nix files under nix/ and exposes nixosConfigurations.vermilion. Kubernetes secrets live in k8s/secrets/.

Development

Enter the pinned development shell with nix develop, or run direnv allow to load it automatically. Then run:

nix fmt
nix flake check
nix build .#nixosConfigurations.vermilion.config.system.build.toplevel

Shared validation and metrics helpers come from the pinned dvnpkgs input.

After Kubernetes changes, run just check-k8s. Run just to list the other maintenance commands.

Deployment

Commit and push validated changes, then let Comin and Flux reconcile master. Run just flux-status to inspect the cluster through the kanto context. Renovate merges routine dependency updates after the required checks pass.

See cluster operations for bootstrap, service sharing, backups, recovery, and updates. For a new NixOS installation, follow the bootstrap guide.

Secrets

Run just sops-bootstrap once per machine to derive your local age key from your SSH key. Then edit secrets with SOPS:

just sops-edit tailscale.yaml
just sops-edit k8s/secrets/tailscale-operator-values.sops.yaml

After changing keys/, run just sops-rekey. Commit .sops.yaml and all re-encrypted files together. Never commit decrypted secrets or private keys. Do not copy encrypted credentials or Flux identities from another cluster.