manifest-llm-gateway

Manifest is a self-hosted gateway in front of the LLM providers you already pay for. It gives you one OpenAI-compatible endpoint, a dashboard for keys, routing and fallbacks, and a record of what was actually sent and returned.

This chart follows the upstream Docker Compose deployment (docker/docker-compose.yml, docker/.env.example) closely enough that a Compose install maps onto it one setting at a time — see the environment variable mapping below. It carries over the same container hardening: a read-only root filesystem, all capabilities dropped, no privilege escalation, an in-memory /tmp, and the same 1 GiB memory ceiling.

Two things are deliberately different from Compose:

• The chart does not deploy PostgreSQL. manifest.database.url points at a database you run — an operator such as CloudNativePG, a managed instance, or a plain StatefulSet. A bundled single-replica database is a worse deal in Kubernetes than in Compose: it hides the backup and the major-version upgrade, which are the two things you actually need to own. Migrations still run by themselves when the application boots.
• The chart never generates secrets. See Secrets.

Not included, on purpose: no NetworkPolicy (a gateway needs egress to every provider on the internet, so the policy would be decorative), no PodDisruptionBudget and no HorizontalPodAutoscaler (the default is a single replica, and scaling out has real prerequisites — see High availability). The Compose file's pids_limit: 512 has no pod-level equivalent and is a node setting in Kubernetes.

Installation#

From the Helm repository:

helm repo add rgielen https://rgielen.github.io/charts
helm repo update
helm install my-manifest-llm-gateway rgielen/manifest-llm-gateway --version 2.11.0

Or directly from the OCI registry:

helm install my-manifest-llm-gateway oci://ghcr.io/rgielen/charts/manifest-llm-gateway --version 2.11.0

Source Code#

• https://github.com/mnfst/llm-gateway
• https://github.com/rgielen/charts

Values#

Workload#

Extensibility#

Networking#

Image#

Manifest: operations#

Manifest: core#

Manifest: LLM proxy#

Manifest: database#

Manifest: email and OAuth#

Manifest: request recordings#

Manifest: observability#

Persistence#

Quick start#

Generate two independent secrets and hand them to the chart out of band:

kubectl create namespace manifest

kubectl --namespace manifest create secret generic manifest-llm-gateway \
  --from-literal=BETTER_AUTH_SECRET="$(openssl rand -hex 32)" \
  --from-literal=MANIFEST_ENCRYPTION_KEY="$(openssl rand -hex 32)" \
  --from-literal=DATABASE_URL='postgresql://manifest:...@postgres.databases.svc:5432/manifest'

helm --namespace manifest install manifest rgielen/manifest-llm-gateway \
  --set manifest.existingSecret=manifest-llm-gateway \
  --set manifest.publicUrl=https://manifest.example.com \
  --set ingress.enabled=true \
  --set ingress.hosts[0].host=manifest.example.com \
  --set ingress.hosts[0].paths[0].path=/ \
  --set ingress.hosts[0].paths[0].pathType=Prefix

On first boot the setup wizard at /setup creates the admin account — there are no default credentials. Add a provider, copy the generated harness key (it starts with mnfst_), and point any OpenAI-compatible client at https://manifest.example.com/v1.

Secrets#

Sensitive settings can be given two ways:

• manifest.existingSecret — the name of a Secret whose keys are the upstream environment variable names (BETTER_AUTH_SECRET, MANIFEST_ENCRYPTION_KEY, DATABASE_URL, EMAIL_API_KEY, GOOGLE_CLIENT_SECRET, …). It is mounted with envFrom and wins over anything set in values. This is the right choice for GitOps: keep the Secret in sealed-secrets or external-secrets and leave the values empty.
• Plain values (manifest.auth.secret, manifest.database.url, …) — the chart renders them into a Secret of its own. Convenient for a quick helm install, but they end up in your values file.

Set at least one of them. Without a session secret or a database URL the chart refuses to render, with a message naming both ways out.

The chart never generates a secret for you, and that is not an oversight. MANIFEST_ENCRYPTION_KEY — or BETTER_AUTH_SECRET, which it falls back to — encrypts every stored provider API key and OAuth token at rest. ArgoCD renders charts with helm template and no cluster access, where Helm's lookup returns nothing: a generated value would come out different on every single sync, and every stored credential would become undecryptable. A key that changes on its own is still the failure case; a key you change deliberately is not, and has its own procedure below.

Set manifest.auth.encryptionKey to a second, independent value rather than letting it fall back. Otherwise one leaked session-signing secret also decrypts every provider credential you have stored.

Rotating the encryption key#

Changing manifest.auth.encryptionKey on its own leaves every stored credential undecryptable, and the read paths report that as "provider not connected" rather than as an error — the damage is silent. Rotate in three steps instead:

1. Set manifest.auth.previousEncryptionKey to the current key and manifest.auth.encryptionKey to the new one, then upgrade. Introducing a dedicated key on an install that had been falling back to the session secret is the same move, with manifest.auth.secret's value as the previous key.
2. After boot one pod rewrites every stored provider key, OAuth token, agent key and e-mail provider key onto the new key, in batches, while the old key keeps working. Wait for Nothing left under an older secret in the log. Anything it could not decrypt under either key it leaves untouched and names, rather than destroying it.
3. Remove manifest.auth.previousEncryptionKey and upgrade again. Leaving it set keeps the old key readable — and mounted in the Secret — indefinitely.

Recording bodies are the exception: they are encrypted with the same key but are not rewritten, so a rotation makes existing ones unreadable and retention eventually removes them. Metadata in the database is unaffected.

Reverse proxy and publicUrl#

manifest.publicUrl becomes BETTER_AUTH_URL and must match the URL the browser actually uses, or logins and OAuth callbacks fail in ways that look like unrelated bugs. When an Ingress is enabled and publicUrl is empty, the chart derives it from the first Ingress host, using https if that host appears in ingress.tls.

Prefer https://, but plain http on a LAN or tailnet host is supported again. Between appVersion 6.24.0 and 6.25.2 it was not: the upstream wired Better Auth's MCP plugin unconditionally and built the plugin's resource URL out of BETTER_AUTH_URL, the plugin rejected a non-HTTPS resource URL while the module loaded, and the process exited before it listened — a pod that never left CrashLoopBackOff (MCP resource URL must use HTTPS). This chart refused to render that configuration rather than let you find out that way.

Since 6.25.3 the upstream decides whether MCP can run before constructing the plugin, so an HTTP-only install boots and serves the dashboard and the gateway, without the MCP surface, naming the reason in its boot log. The chart's guard is therefore gone. Two consequences remain on plain http, and both are only that:

• No HSTS. The application logs a warning on every boot; manifest.disableHsts=true silences it where it is not wanted.
• No remote MCP endpoint. manifest.mcpEnabled=false acknowledges it and takes the boot line with it. A TLS-terminating proxy in front is what brings MCP back — a self-signed certificate is enough, only the scheme is inspected.

One caveat this removes the guard from: the crash belongs to the image, not to the chart. Pinning image.tag below the chart's appVersion on a plain-http host brings it back.

Streaming responses need a generous read timeout on the ingress controller. The annotation differs per controller — for ingress-nginx it is nginx.ingress.kubernetes.io/proxy-read-timeout, for Traefik it is a ServersTransport.

More than one host#

ingress.hosts takes a list, but publicUrl is one origin, and Better Auth pins both the OAuth callback and the session cookie to it. A login that starts on the second host is therefore redirected to a callback on publicUrl and leaves its cookie on an origin the second host's dashboard never sends back — the login simply does not stick, with nothing in the logs to say why.

Since appVersion 6.25.2 the upstream resolves the base URL from the request host instead, restricted to an allow-list, and manifest.authAllowedHosts fills it (BETTER_AUTH_ALLOWED_HOSTS). Entries are hostnames with an optional port, or full origins — the scheme is ignored, only the host is kept — and *.example.com wildcards are accepted. Better Auth derives its trusted origins from the same list, so the CSRF check follows along and nothing else needs configuring:

ingress:
  enabled: true
  hosts:
    - host: manifest.example.com
      paths: [{ path: /, pathType: Prefix }]
    - host: gateway.example.com
      paths: [{ path: /, pathType: Prefix }]
manifest:
  publicUrl: https://manifest.example.com
  authAllowedHosts:
    - gateway.example.com

Leave it empty for the usual single-host install. With only one host known the upstream keeps the static origin, which is exactly what it did before 6.25.2 — the chart does not derive the list from ingress.hosts for you, because trusting a host is a decision, not a side effect of serving it.

Two things the list does not move. The remote MCP endpoint stays bound to publicUrl: a self-hosted release advertises one resource URL, and MCP clients keep using that one. And an entry the upstream cannot parse is dropped without a word — values.schema.json rejects the obvious typos at install time instead.

Database#

manifest.database.url is a standard PostgreSQL connection string. Percent-encode special characters in the password (@ → %40, : → %3A, / → %2F). Migrations are applied by the chart — see Database migrations.

Back up the database, not the cluster: it holds accounts, provider credentials, harness keys and request metadata.

Request recordings#

Message metadata always lives in the database. Recorded message bodies are separate and optional, and go to one of two places:

• A volume, mounted at manifest.recordings.filesystemPath. Set persistence.enabled to back it with a PersistentVolumeClaim. Without it the mount is an emptyDir and the bodies are gone on every restart — the mount itself is always present because the root filesystem is read-only.
• S3-compatible storage, via manifest.recordings.s3. A complete S3 configuration takes precedence over the volume while manifest.recordings.storage is auto.

Retention defaults to 365 days upstream; override with manifest.recordings.retentionDays.

Bodies are encrypted at rest with manifest.auth.encryptionKey as of upstream 6.21.0 (recordings written before that stay readable). Unlike stored credentials they are not rewritten during a key rotation, so see Rotating the encryption key above before changing it.

Database migrations#

Migrations are applied by a pre-install/pre-upgrade hook Job that runs the upstream's own migration entry point, not by the application on boot. The Job runs exactly once per release, so the schema is in place before any pod starts, and a failed migration is a failed Job with readable logs instead of a pod in CrashLoopBackOff. Its ConfigMap and Secret are hook-scoped copies — a pre-install hook runs before the release's own resources exist, and on upgrade the release's copies still hold the previous values.

manifest.runMigrationsOnBoot is off by default because of this. Turning it on alongside more than one replica is refused outright, and that is not caution:

The boot path applies migrations without the advisory lock the migration entry point takes — no lock at all — so every replica runs the same pending migrations at the same time, and several of them are a CREATE INDEX CONCURRENTLY, which waits for every other session that can see the table. How badly concurrent index builds go wrong was observed before appVersion 6.26.0, even with the lock: three concurrent runners against an empty database hung indefinitely, the holder's index build waiting on the runners blocked on the lock, a cycle PostgreSQL neither recognises nor breaks. Upstream 6.26.0 fixed that for the migration entry point only — its waiters now poll for the lock instead of blocking — and the boot path still has nothing to wait on.

If manifest.database.url points at a transaction pooler such as PgBouncer, set manifest.database.migrationUrl to a direct connection. The advisory lock is session-scoped and transaction pooling does not preserve it.

Those concurrent index builds are not gated by manifest.mode — the dashboard and retention queries they serve run self-hosted too — so manifest.migrations.job.activeDeadlineSeconds governs them on every install. The 900 s default is ample for a small database, where they finish in seconds; on a database of millions of requests, upstream budgets minutes per index, and the Job's deadline has to cover the whole run.

One migration path the Job cannot cover. Besides the TypeORM migrations, the application creates Better Auth's own tables on module init, through a separate code path with no lock and no entry point of its own. On a first install with more than one replica the pods race on those CREATE TABLEs: one exits with a duplicate-object error and is restarted, then finds the tables in place and starts normally. The end state is correct, and rolling upgrades never hit it because only one new pod starts at a time. To avoid the restart entirely, install with replicaCount: 1 and scale up afterwards.

High availability#

Single-node is not a requirement, and never was — a multi-node cluster with one replica needs no special configuration. More than one replica is a different question.

What the chart handles:

• Rollout strategy. With no ReadWriteOnce volume in play it defaults to a surge-first RollingUpdate (maxUnavailable: 0), so an upgrade does not interrupt service. With persistence.enabled it falls back to Recreate, because a rolling update would wait forever for a claim the old pod still holds. Override with updateStrategy.
• Migrations. See above — the hook Job is what makes multiple replicas safe.
• Recordings. Configure manifest.recordings.s3; a ReadWriteOnce volume cannot be shared, and the chart refuses that combination rather than leaving pods Pending.
• Disruption. Set podDisruptionBudget.enabled: true. It is off by default because for a single replica it only blocks node drains.
• Placement. Nothing is spread across nodes unless you say so. This constraint is a no-op on a single-node cluster and spreads on a larger one:

yaml topologySpreadConstraints: - maxSkew: 1 topologyKey: kubernetes.io/hostname whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: app.kubernetes.io/name: manifest-llm-gateway

What the chart cannot fix, because it is upstream behaviour:

Recording retention cleanup is not on that list: it takes its own advisory lock upstream and is safe across replicas. Neither is the re-encryption pass that manifest.auth.previousEncryptionKey triggers — it takes a third lock, and takes it without waiting, so one pod does the rewrite and the others carry on serving.

If a strict global rate limit matters to you, enforce it at the ingress rather than relying on THROTTLE_LIMIT.

Upgrading#

Chart version and appVersion move independently, and both appear in every release. New upstream releases are picked up automatically: a scheduled workflow in this repository watches the image, bumps appVersion and the chart version together, and opens a pull request that merges itself once the chart still lints and installs.

Because a rollout restarts the single pod, an upgrade is a short outage on a single-node cluster. Migrations run on boot; take a database backup first for anything that changes the upstream major version.

Use from ArgoCD#

Always pin targetRevision to an explicit chart version:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: manifest
  namespace: argocd
spec:
  project: infrastructure
  source:
    repoURL: https://rgielen.github.io/charts
    chart: manifest-llm-gateway
    targetRevision: 2.11.0
    helm:
      valuesObject:
        manifest:
          existingSecret: manifest-llm-gateway
          publicUrl: https://manifest.example.com
        ingress:
          enabled: true
          hosts:
            - host: manifest.example.com
              paths:
                - path: /
                  pathType: Prefix
  destination:
    server: https://kubernetes.default.svc
    namespace: manifest

Environment variable mapping#

Every setting from the upstream .env.example, and where it lives here. Values marked secret belong in manifest.existingSecret under exactly the environment variable name in the left column.

Every variable is spelled out in full rather than abbreviated with a shared prefix, so .github/scripts/chart_audit.py can check this table for completeness instead of guessing.

Deliberately not modelled#

The upstream reads these and this chart does not expose them. They are decisions, not gaps — extraEnv reaches any of them if you disagree.

A value left empty is not passed to the container at all. That matters: the application reads several settings as Number(env ?? default), where an empty string becomes 0 rather than the documented default.

Maintainers#

Published versions#

Every version below is installable and immutable — a published version is never re-released with different content.

Upstream tracking#

This chart's appVersion follows a container image. A scheduled workflow watches that image, diffs the watched files between the two image commits, and opens a pull request that bumps appVersion and the chart version together. It merges unattended only when the bump is not a major one and the diff is clean.