Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/helm-template-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,33 @@ jobs:
--set 'gateway.parentRefs[0].namespace=gateway-system' \
--api-versions gateway.networking.k8s.io/v1/HTTPRoute \
--api-versions gateway.nginx.org/v1alpha1/ClientSettingsPolicy
# ListenerSet on a shared Gateway; the HTTPRoute attaches to it
helm template test ./charts/ai \
--set-file ctaiPropertiesFile=./charts/ai/ctai-configuration.properties.template \
--set service.fqdn=https://example.codetogether.io \
--set imageCredentials.username=test \
--set imageCredentials.password=test \
--set ingress.enabled=false \
--set gateway.enabled=true \
--set gateway.listenerSet.enabled=true \
--set gateway.listenerSet.parentRef.name=shared-gateway \
--set gateway.listenerSet.parentRef.namespace=gateway-system \
--api-versions gateway.networking.k8s.io/v1/HTTPRoute \
--api-versions gateway.networking.k8s.io/v1/ListenerSet \
> /tmp/listenerset.yaml
grep -q '^kind: ListenerSet$' /tmp/listenerset.yaml
grep -q 'kind: ListenerSet$' <(sed -n '/^kind: HTTPRoute$/,/^---/p' /tmp/listenerset.yaml)
# listenerSet without gateway.enabled must fail instead of rendering nothing
if helm template test ./charts/ai \
--set-file ctaiPropertiesFile=./charts/ai/ctai-configuration.properties.template \
--set service.fqdn=https://example.codetogether.io \
--set imageCredentials.username=test \
--set imageCredentials.password=test \
--set gateway.listenerSet.enabled=true \
--set gateway.listenerSet.parentRef.name=shared-gateway > /tmp/ls-no-gw.out 2>&1; then
echo "listenerSet without gateway.enabled rendered" >&2; exit 1
fi
grep -q 'requires gateway.enabled=true' /tmp/ls-no-gw.out
# helm upgrade --reuse-values from a chart without `gateway` drops the key
helm template test ./charts/ai \
--set-file ctaiPropertiesFile=./charts/ai/ctai-configuration.properties.template \
Expand Down
4 changes: 2 additions & 2 deletions charts/ai/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ name: codetogether-ai
description: CodeTogether AI on-premises Kubernetes deployment

type: application
version: 0.2.2
appVersion: "1.2.8165"
version: 0.2.3
appVersion: "1.2.8616"

icon: https://www.codetogether.com/wp-content/uploads/2020/02/codetogether-circle-128.png
home: https://www.codetogether.com
Expand Down
64 changes: 60 additions & 4 deletions charts/ai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,59 @@ The chart renders:
rendered only when the cluster serves that API. For another Gateway API
implementation, set the equivalent limit on your side.

The chart doesn't create the `Gateway`, its listeners or certificates. On the
Gateway side, check that:
#### ListenerSet: your own listener on a shared Gateway

Instead of adding a listener to the shared `Gateway` for this host, the chart
can create a `ListenerSet` (`gateway.networking.k8s.io/v1`, Gateway API v1.5+)
that adds the HTTPS listener for the `service_fqdn` host, with its certificate,
from the release namespace. The `HTTPRoute` then attaches to that ListenerSet, so leave
`gateway.parentRefs` empty:

```yaml
ingress:
enabled: false
gateway:
enabled: true
listenerSet:
enabled: true
parentRef:
name: gateway-ngf # the shared Gateway
namespace: kube-system
# The listener's hostname is the service_fqdn host. The certificate Secret
# defaults to <release fullname>-tls; set listenerSet.tlsSecretName to change it.
annotations:
cert-manager.io/cluster-issuer: cert-manager-clusterissuer
```

The shared Gateway has to accept ListenerSets from the release namespace
(`spec.allowedListeners.namespaces`). With a cert-manager version that supports
ListenerSets, the issuer annotation on the ListenerSet issues the certificate
into that Secret; otherwise create the Secret yourself. On cert-manager v1.21
that support is off by default; enable it in the cert-manager chart values:

```yaml
config:
apiVersion: controller.config.cert-manager.io/v1alpha1
kind: ControllerConfiguration
enableGatewayAPI: true
enableGatewayAPIListenerSet: true
featureGates: ListenerSets=true
```

Without it, cert-manager logs `skipping disabled controller ... listenerset` and
no certificate is issued.

If you already applied a ListenerSet with the same name by hand, Helm refuses to
take it over. Let Helm adopt it before the upgrade:

```bash
kubectl -n <namespace> label listenerset <release-fullname> app.kubernetes.io/managed-by=Helm
kubectl -n <namespace> annotate listenerset <release-fullname> \
meta.helm.sh/release-name=<release> meta.helm.sh/release-namespace=<namespace>
```

The chart doesn't create the `Gateway`, and without a ListenerSet it doesn't
create listeners or certificates either. On the Gateway side, check that:

- the listener allows routes from the release namespace
(`allowedRoutes.namespaces`);
Expand Down Expand Up @@ -244,10 +295,15 @@ The following table lists the primary configurable parameters and defaults.
| `ingress.annotations` | Ingress annotations | `nginx.ingress.kubernetes.io/proxy-body-size: "16m"` |
| `ingress.tls.secretName` | TLS Secret for Ingress | `codetogether-ai-tls` |
| `gateway.enabled` | Create a Gateway API `HTTPRoute` | `false` |
| `gateway.parentRefs` | Gateway(s) the `HTTPRoute` attaches to; required when enabled | `[]` |
| `gateway.annotations` / `gateway.labels` | Extra `HTTPRoute` metadata | `{}` |
| `gateway.parentRefs` | Gateway(s) the `HTTPRoute` attaches to; required when enabled unless `gateway.listenerSet.enabled` | `[]` |
| `gateway.annotations` / `gateway.labels` | Extra `HTTPRoute` metadata (`labels` also go on the `ListenerSet`) | `{}` |
| `gateway.nginxGatewayFabric.clientSettingsPolicy.enabled` | NGINX Gateway Fabric body-size policy: `auto`, `true` or `false` | `auto` |
| `gateway.nginxGatewayFabric.clientSettingsPolicy.bodyMaxSize` | Request body limit on the Gateway | `16m` |
| `gateway.listenerSet.enabled` | Create a `ListenerSet` with this release's HTTPS listener; the `HTTPRoute` attaches to it when `gateway.parentRefs` is empty. Requires `gateway.enabled=true` | `false` |
| `gateway.listenerSet.parentRef` | The shared Gateway (`name`, `namespace`); required when enabled | `{}` |
| `gateway.listenerSet.listenerName` / `port` | Listener name (the route's `sectionName`) and port | `https` / `443` |
| `gateway.listenerSet.tlsSecretName` | TLS Secret for the listener | `<release fullname>-tls` |
| `gateway.listenerSet.annotations` | `ListenerSet` annotations (e.g. the cert-manager issuer); labels come from `gateway.labels` | `{}` |
| `clientIp.trustedProxyHops` | Proxies in front of the pod that append to `X-Forwarded-For` | `2` |
| `serviceAccount.create` | Create a ServiceAccount | `true` |
| `serviceAccount.name` | ServiceAccount name | `codetogether-ai` |
Expand Down
8 changes: 8 additions & 0 deletions charts/ai/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ Gateway API:
{{- with .sectionName }}{{ $p = printf "%s listener %s" $p . }}{{ end }}
{{- $parents = append $parents $p }}
{{- end }}
{{- if eq (include "ctai.listenerSetEnabled" .) "true" }}
{{- $lsParent := $gw.listenerSet.parentRef }}
ListenerSet {{ include "ctai.fullname" . }} adds an HTTPS listener to Gateway {{ $lsParent.name }}{{ with $lsParent.namespace }} (namespace {{ . }}){{ end }}.
Check it was accepted: kubectl get listenerset {{ include "ctai.fullname" . }} -n {{ .Release.Namespace }} -o jsonpath='{.status.conditions}'
{{- if not $parents }}
{{- $parents = list (printf "ListenerSet %s listener %s" (include "ctai.fullname" .) ($gw.listenerSet.listenerName | default "https")) }}
{{- end }}
{{- end }}
HTTPRoute {{ include "ctai.fullname" . }} attaches to: {{ join ", " $parents }}
Check it was accepted: kubectl get httproute {{ include "ctai.fullname" . }} -n {{ .Release.Namespace }} -o jsonpath='{.status.parents[*].conditions}'
{{- if eq (include "ctai.renderClientSettingsPolicy" .) "true" }}
Expand Down
24 changes: 24 additions & 0 deletions charts/ai/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -297,3 +297,27 @@ from a chart without `gateway` keeps the old values and skips these defaults.
{{- define "ctai.clientSettingsBodyMaxSize" -}}
{{- (((.Values.gateway | default dict).nginxGatewayFabric | default dict).clientSettingsPolicy | default dict).bodyMaxSize | default "16m" -}}
{{- end }}

{{/*
ListenerSet settings, read defensively like the rest of `gateway`.
*/}}
{{- define "ctai.listenerSetEnabled" -}}
{{- $gw := .Values.gateway | default dict -}}
{{- if and $gw.enabled ($gw.listenerSet | default dict).enabled -}}true{{- end -}}
{{- end }}

{{/*
HTTPRoute parentRefs: gateway.parentRefs when set; otherwise the chart's
ListenerSet and its listener.
*/}}
{{- define "ctai.routeParentRefs" -}}
{{- $gw := .Values.gateway | default dict -}}
{{- if $gw.parentRefs -}}
{{- toYaml $gw.parentRefs -}}
{{- else if eq (include "ctai.listenerSetEnabled" .) "true" -}}
- group: gateway.networking.k8s.io
kind: ListenerSet
name: {{ include "ctai.fullname" . }}
sectionName: {{ ($gw.listenerSet | default dict).listenerName | default "https" }}
{{- end -}}
{{- end }}
6 changes: 3 additions & 3 deletions charts/ai/templates/httproute.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{- $gw := .Values.gateway | default dict }}
{{- if $gw.enabled }}
{{- if not $gw.parentRefs }}
{{- fail "gateway.enabled=true requires gateway.parentRefs: the Gateway (name, and namespace/sectionName if needed) the HTTPRoute attaches to." }}
{{- if not (include "ctai.routeParentRefs" .) }}
{{- fail "gateway.enabled=true requires gateway.parentRefs (the Gateway, and namespace/sectionName if needed, the HTTPRoute attaches to) or gateway.listenerSet.enabled=true." }}
{{- end }}
{{- if not (.Capabilities.APIVersions.Has "gateway.networking.k8s.io/v1/HTTPRoute") }}
{{- fail "gateway.enabled=true but the cluster does not serve gateway.networking.k8s.io/v1 HTTPRoute. Install the Gateway API CRDs (or pass --api-versions gateway.networking.k8s.io/v1/HTTPRoute to helm template)." }}
Expand All @@ -22,7 +22,7 @@ metadata:
{{- end }}
spec:
parentRefs:
{{- toYaml $gw.parentRefs | nindent 4 }}
{{- include "ctai.routeParentRefs" . | nindent 4 }}
hostnames:
- {{ include "ctai.routeHostname" . | quote }}
rules:
Expand Down
42 changes: 42 additions & 0 deletions charts/ai/templates/listenerset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{{- $gwValues := .Values.gateway | default dict }}
{{- if and ($gwValues.listenerSet | default dict).enabled (not $gwValues.enabled) }}
{{- fail "gateway.listenerSet.enabled=true requires gateway.enabled=true: the ListenerSet carries the HTTPRoute's listener, and without the HTTPRoute nothing is rendered." }}
{{- end }}
{{- if eq (include "ctai.listenerSetEnabled" .) "true" }}
{{- $ls := .Values.gateway.listenerSet }}
{{- $parent := $ls.parentRef | default dict }}
{{- if not $parent.name }}
{{- fail "gateway.listenerSet.enabled=true requires gateway.listenerSet.parentRef.name: the Gateway the ListenerSet adds a listener to." }}
{{- end }}
{{- if not (.Capabilities.APIVersions.Has "gateway.networking.k8s.io/v1/ListenerSet") }}
{{- fail "gateway.listenerSet.enabled=true but the cluster does not serve gateway.networking.k8s.io/v1 ListenerSet. Install Gateway API v1.5+ standard CRDs (or pass --api-versions gateway.networking.k8s.io/v1/ListenerSet to helm template)." }}
{{- end }}
# HTTPS listener for this release on the shared Gateway {{ $parent.name }}. With a
# cert-manager.io/(cluster-)issuer annotation, cert-manager issues the
# certificate into the Secret named below.
apiVersion: gateway.networking.k8s.io/v1
kind: ListenerSet
metadata:
name: {{ include "ctai.fullname" . }}
labels:
{{- include "ctai.labels" . | nindent 4 }}
{{- with (.Values.gateway.labels) }}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with $ls.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
parentRef:
{{- toYaml $parent | nindent 4 }}
listeners:
- name: {{ $ls.listenerName | default "https" }}
hostname: {{ include "ctai.routeHostname" . | quote }}
port: {{ $ls.port | default 443 }}
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: {{ $ls.tlsSecretName | default (printf "%s-tls" (include "ctai.fullname" .)) }}
{{- end }}
20 changes: 20 additions & 0 deletions charts/ai/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,26 @@ gateway:
# false: never render (another Gateway API implementation)
enabled: auto
bodyMaxSize: "16m"
# Optional ListenerSet (gateway.networking.k8s.io/v1): adds this release's
# HTTPS listener (the service_fqdn host and its certificate) to a shared Gateway, so the
# shared Gateway doesn't need a listener edited for each application. The
# Gateway must allow it (spec.allowedListeners). When enabled and parentRefs
# is empty, the HTTPRoute attaches to this ListenerSet.
listenerSet:
enabled: false
# The Gateway this ListenerSet adds a listener to. Required when enabled.
# Example:
# parentRef:
# name: gateway-ngf
# namespace: kube-system
parentRef: {}
listenerName: https
port: 443
# cert-manager reads these (cert-manager.io/cluster-issuer or
# cert-manager.io/issuer) and issues the certificate into tlsSecretName.
annotations: {}
# TLS Secret in the release namespace. Defaults to <release fullname>-tls.
tlsSecretName: ""

# Client address handling (X-Forwarded-For trust).
#
Expand Down
Loading