From 4ff71b699ac9282471bf662b1629b3dced571c37 Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Tue, 11 Aug 2026 14:32:10 +0530 Subject: [PATCH 1/8] docs: add Helm installation guide The chart has been in the repo since #163 but Helm is not mentioned anywhere in the book, so the install page only covered release manifests, kustomize and static pods. Adds a Helm section covering install from a checkout, since OCI chart releases are still WIP, which values turn on metrics, TLS and the validating webhook, and how to ship rules through the chart. Also documents two things that bite people and are specific to Helm. Helm only installs the CRD from crds/ on first install and never upgrades it, so a chart bump that changes the schema needs the CRD applied by hand. And rules declared in values are release resources, so helm uninstall removes them together with the controller and the taint finalizer has no controller left to run, which is the stuck-resource case already documented further down the page. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 86 +++++++++++++++++++++++- 1 file changed, 84 insertions(+), 2 deletions(-) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index 729386d4..89e906e9 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -62,7 +62,82 @@ REPO="registry.k8s.io/node-readiness-controller/node-readiness-controller" TAG=$(skopeo list-tags docker://$REPO | jq .'Tags[-1]' | tr -d '"') docker pull $REPO:$TAG ``` -### Option 2: Advanced Deployment (Kustomize) +### Option 2: Helm Chart + +The chart lives in the repository under `charts/nrr-controller`. Published chart releases via `registry.k8s.io` OCI are still work in progress, so install it from a checkout for now. + +```sh +git clone https://github.com/kubernetes-sigs/node-readiness-controller.git +cd node-readiness-controller + +helm install nrr-controller ./charts/nrr-controller \ + --namespace nrr-system --create-namespace +``` + +Requires Helm 3.x. This deploys the controller with the same defaults as the standard manifest: leader election on, metrics off, and the validating webhook off. + +#### Optional components + +Everything beyond the core controller is opt-in, matching the kustomize components. + +| Feature | Values | Prerequisites | +| :--- | :--- | :--- | +| Metrics endpoint | `metrics.enabled=true` | None for plain HTTP | +| Metrics over TLS | `metrics.enabled=true`, `metrics.secure=true`, `certManager.enabled=true` | `cert-manager` | +| Validating webhook | `webhook.enabled=true`, `validatingWebhook.enabled=true`, `certManager.enabled=true` | `cert-manager` | + +The webhook rejects rules whose taint key and effect collide with an existing rule over an overlapping node selector, so it is worth enabling in production. + +```sh +helm install nrr-controller ./charts/nrr-controller \ + --namespace nrr-system --create-namespace \ + --set certManager.enabled=true \ + --set webhook.enabled=true \ + --set validatingWebhook.enabled=true \ + --set metrics.enabled=true \ + --set metrics.secure=true +``` + +Both `webhook.enabled` and `validatingWebhook.enabled` are needed. The first runs the webhook server in the controller and mounts its certificate, the second registers the `ValidatingWebhookConfiguration` with the API server. + +#### Managing rules through the chart + +`NodeReadinessRule` objects can be shipped with the release through the `nodeReadinessRules` value: + +```yaml +nodeReadinessRules: + - name: kube-proxy-unhealthy-noschedule + enforcementMode: continuous + conditions: + - type: KubeProxyUnhealthy + requiredStatus: "False" + taint: + key: readiness.k8s.io/KubeProxyUnhealthy + value: "true" + effect: NoSchedule + nodeSelector: + matchLabels: + kubernetes.io/os: linux +``` + +`nodeSelector` is required on every entry. Set it explicitly, since an empty selector matches every node in the cluster. + +> [!NOTE] +> With the validating webhook enabled, apply rules only once the controller is serving admission requests. On a first install the webhook is not ready while the rules in the same release are being created, so install the controller first and add the rules in a follow-up `helm upgrade`. + +#### CRD upgrades + +Helm installs the CRD from the chart's `crds/` directory on first install only. It does not upgrade or remove it on `helm upgrade` or `helm uninstall`. + +Before moving to a chart version that changes the `NodeReadinessRule` schema, apply the CRD yourself: + +```sh +kubectl apply -f charts/nrr-controller/crds/nodereadinessrules.readiness.node.x-k8s.io.yaml +``` + +Skipping this leaves the old schema in place, and rules using newly added fields are rejected by the API server even though the controller supports them. + +### Option 3: Advanced Deployment (Kustomize) If you need deeper customization, you can use Kustomize directly from the source. @@ -76,7 +151,7 @@ kubectl apply -k config/default You can enable optional components (Metrics, TLS, Webhook) by creating a `kustomization.yaml` that includes the relevant components from the `config/` directory. For reference on how these components can be combined, see the `deploy-with-metrics`, `deploy-with-tls`, `deploy-with-webhook`, and `deploy-full` targets in the projects [`Makefile`](https://github.com/kubernetes-sigs/node-readiness-controller/blob/main/Makefile). -### Option 3: Deploy as a Static Pod (Control Plane) +### Option 4: Deploy as a Static Pod (Control Plane) Running the controller as a **Static Pod** on control-plane nodes is useful for self-managed clusters (e.g., `kubeadm`) where you want the controller to be available alongside core components like the API server. @@ -157,14 +232,21 @@ The controller uses a **finalizer** (`readiness.node.x-k8s.io/cleanup-taints`) o # OR if using Kustomize kubectl delete -k config/default + # OR if using Helm + helm uninstall nrr-controller --namespace nrr-system + # OR if using Static Pods # Remove the manifest from /etc/kubernetes/manifests/ on all control-plane nodes ``` + > [!CAUTION] + > Rules declared through the chart's `nodeReadinessRules` value are part of the release, so `helm uninstall` deletes them and the controller in one operation. Helm does not wait for the finalizer to run, which is exactly the situation described in [Recovering from Stuck Resources](#recovering-from-stuck-resources). Delete the rules and let them finish terminating before uninstalling the release. + 3. **Uninstall CRDs** (Optional): ```sh kubectl delete -k config/crd ``` + Helm does not remove the CRD it installed from the chart's `crds/` directory, so delete it explicitly if you used the chart. ### Recovering from Stuck Resources From c8442458f1d50a4705e910946dde8427333d1228 Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Thu, 13 Aug 2026 09:00:31 +0530 Subject: [PATCH 2/8] docs: add Helm upgrade and custom values to the install guide Review feedback on #388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 32 ++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index 89e906e9..a57a2a54 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -76,6 +76,16 @@ helm install nrr-controller ./charts/nrr-controller \ Requires Helm 3.x. This deploys the controller with the same defaults as the standard manifest: leader election on, metrics off, and the validating webhook off. +For anything beyond a couple of overrides, keep your settings in a file instead of a long `--set` list: + +```sh +helm show values ./charts/nrr-controller > custom-values.yaml + +helm install nrr-controller ./charts/nrr-controller \ + --namespace nrr-system --create-namespace \ + -f custom-values.yaml +``` + #### Optional components Everything beyond the core controller is opt-in, matching the kustomize components. @@ -125,6 +135,28 @@ nodeReadinessRules: > [!NOTE] > With the validating webhook enabled, apply rules only once the controller is serving admission requests. On a first install the webhook is not ready while the rules in the same release are being created, so install the controller first and add the rules in a follow-up `helm upgrade`. +#### Upgrading + +Pull the version of the chart you want and upgrade the release in place. Values you set at install time are carried over, so only pass the ones you are changing: + +```sh +git pull + +helm upgrade nrr-controller ./charts/nrr-controller \ + --namespace nrr-system \ + -f custom-values.yaml +``` + +`helm upgrade --install` works too if you want one command that handles both the first install and later upgrades. + +Check what changed before applying it to a live cluster: + +```sh +helm diff upgrade nrr-controller ./charts/nrr-controller --namespace nrr-system # needs the helm-diff plugin +``` + +Read the CRD note below first. Helm will not update the CRD for you, so a chart bump that changes the schema needs that step done by hand. + #### CRD upgrades Helm installs the CRD from the chart's `crds/` directory on first install only. It does not upgrade or remove it on `helm upgrade` or `helm uninstall`. From dddcd94442e213599b6dd40e54b3a6de98cd7d0c Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Fri, 14 Aug 2026 17:03:54 +0530 Subject: [PATCH 3/8] docs: cover the controller tuning values from #392 The chart gained a controller block with concurrency, QPS and pprof settings after this PR was opened, so document them alongside the rest of the install options. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index a57a2a54..39fa01fd 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -110,6 +110,21 @@ helm install nrr-controller ./charts/nrr-controller \ Both `webhook.enabled` and `validatingWebhook.enabled` are needed. The first runs the webhook server in the controller and mounts its certificate, the second registers the `ValidatingWebhookConfiguration` with the API server. +#### Tuning for larger clusters + +The values under `controller` map to the manager's own flags, and each one is only passed to the container when you move it off its default: + +| Value | Flag | Default | +| :--- | :--- | :--- | +| `controller.nodeConcurrentReconciles` | `--node-concurrent-reconciles` | `1` | +| `controller.ruleConcurrentReconciles` | `--rule-concurrent-reconciles` | `1` | +| `controller.kubeAPIQPS` | `--kube-api-qps` | `-1`, client-side throttling off | +| `controller.kubeAPIBurst` | `--kube-api-burst` | `-1`, client-side throttling off | +| `controller.enableNodeStateMetrics` | `--enable-node-state-metrics` | `false` | +| `controller.pprofBindAddress` | `--pprof-bind-address` | unset, disabled | + +Raising the two concurrency values is the usual response to readiness taints lagging behind node joins on a large cluster. `enableNodeStateMetrics` adds the per-rule `node_readiness_nodes_by_state` gauge at the cost of extra API reads on node updates, so turn it on when you want the fleet view and can afford the reads. + #### Managing rules through the chart `NodeReadinessRule` objects can be shipped with the release through the `nodeReadinessRules` value: From fd78bdcff412184baa688b220a606faa9209cda0 Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Fri, 14 Aug 2026 20:04:49 +0530 Subject: [PATCH 4/8] docs: follow the chart rename and fix the verification selector The chart directory and name became node-readiness-controller in #407, so the install, upgrade and uninstall commands in the Helm section needed updating along with the path to the bundled CRD. Also fixes the selector in Verification. It looked for component=node-readiness-controller, and nothing sets a component label, not the chart and not the kustomize manifests, so the command matched no pods on either path. Both set control-plane=controller-manager, so use that. A reader following the new Helm section lands on those commands straight after installing. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index 39fa01fd..67ac6b87 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -64,13 +64,13 @@ docker pull $REPO:$TAG ``` ### Option 2: Helm Chart -The chart lives in the repository under `charts/nrr-controller`. Published chart releases via `registry.k8s.io` OCI are still work in progress, so install it from a checkout for now. +The chart lives in the repository under `charts/node-readiness-controller`. Published chart releases via `registry.k8s.io` OCI are still work in progress, so install it from a checkout for now. ```sh git clone https://github.com/kubernetes-sigs/node-readiness-controller.git cd node-readiness-controller -helm install nrr-controller ./charts/nrr-controller \ +helm install node-readiness-controller ./charts/node-readiness-controller \ --namespace nrr-system --create-namespace ``` @@ -79,9 +79,9 @@ Requires Helm 3.x. This deploys the controller with the same defaults as the sta For anything beyond a couple of overrides, keep your settings in a file instead of a long `--set` list: ```sh -helm show values ./charts/nrr-controller > custom-values.yaml +helm show values ./charts/node-readiness-controller > custom-values.yaml -helm install nrr-controller ./charts/nrr-controller \ +helm install node-readiness-controller ./charts/node-readiness-controller \ --namespace nrr-system --create-namespace \ -f custom-values.yaml ``` @@ -99,7 +99,7 @@ Everything beyond the core controller is opt-in, matching the kustomize componen The webhook rejects rules whose taint key and effect collide with an existing rule over an overlapping node selector, so it is worth enabling in production. ```sh -helm install nrr-controller ./charts/nrr-controller \ +helm install node-readiness-controller ./charts/node-readiness-controller \ --namespace nrr-system --create-namespace \ --set certManager.enabled=true \ --set webhook.enabled=true \ @@ -157,7 +157,7 @@ Pull the version of the chart you want and upgrade the release in place. Values ```sh git pull -helm upgrade nrr-controller ./charts/nrr-controller \ +helm upgrade node-readiness-controller ./charts/node-readiness-controller \ --namespace nrr-system \ -f custom-values.yaml ``` @@ -167,7 +167,7 @@ helm upgrade nrr-controller ./charts/nrr-controller \ Check what changed before applying it to a live cluster: ```sh -helm diff upgrade nrr-controller ./charts/nrr-controller --namespace nrr-system # needs the helm-diff plugin +helm diff upgrade node-readiness-controller ./charts/node-readiness-controller --namespace nrr-system # needs the helm-diff plugin ``` Read the CRD note below first. Helm will not update the CRD for you, so a chart bump that changes the schema needs that step done by hand. @@ -179,7 +179,7 @@ Helm installs the CRD from the chart's `crds/` directory on first install only. Before moving to a chart version that changes the `NodeReadinessRule` schema, apply the CRD yourself: ```sh -kubectl apply -f charts/nrr-controller/crds/nodereadinessrules.readiness.node.x-k8s.io.yaml +kubectl apply -f charts/node-readiness-controller/crds/nodereadinessrules.readiness.node.x-k8s.io.yaml ``` Skipping this leaves the old schema in place, and rules using newly added fields are rejected by the API server even though the controller supports them. @@ -230,13 +230,13 @@ After installation, verify that the controller is running successfully. 1. **Check Pod Status**: ```sh - kubectl get pods -n ${NAMESPACE} -l component=node-readiness-controller + kubectl get pods -n ${NAMESPACE} -l control-plane=controller-manager ``` You should see the controller pods in `Running` status. 2. **Check Logs**: ```sh - kubectl logs -n ${NAMESPACE} -l component=node-readiness-controller + kubectl logs -n ${NAMESPACE} -l control-plane=controller-manager ``` Look for "Starting EventSource" or "Starting Controller" messages indicating the manager is active. @@ -280,7 +280,7 @@ The controller uses a **finalizer** (`readiness.node.x-k8s.io/cleanup-taints`) o kubectl delete -k config/default # OR if using Helm - helm uninstall nrr-controller --namespace nrr-system + helm uninstall node-readiness-controller --namespace nrr-system # OR if using Static Pods # Remove the manifest from /etc/kubernetes/manifests/ on all control-plane nodes From 1f075a5b3313ce715293abbe5e25e9b06802de63 Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Sat, 15 Aug 2026 05:52:34 +0530 Subject: [PATCH 5/8] Revert the verification selector change Backing this out. #401 was already filed and assigned for it, two days before I touched it, and the analysis there is better than mine. I claimed nothing sets a component label. That is wrong. examples/static-pod/node-readiness-controller.yaml sets it, and it is the only place in the repo that does. That pod carries component and tier and no control-plane label at all, so swapping the selector to control-plane=controller-manager would have broken verification for static pod installs, which is the one path where the documented command works today. Leaving the selector alone so #401 can fix it properly per install path. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index 67ac6b87..d81360fb 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -230,13 +230,13 @@ After installation, verify that the controller is running successfully. 1. **Check Pod Status**: ```sh - kubectl get pods -n ${NAMESPACE} -l control-plane=controller-manager + kubectl get pods -n ${NAMESPACE} -l component=node-readiness-controller ``` You should see the controller pods in `Running` status. 2. **Check Logs**: ```sh - kubectl logs -n ${NAMESPACE} -l control-plane=controller-manager + kubectl logs -n ${NAMESPACE} -l component=node-readiness-controller ``` Look for "Starting EventSource" or "Starting Controller" messages indicating the manager is active. From 24eb39d5347a131fbf45d786d279c778958e2f49 Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Tue, 25 Aug 2026 08:47:55 +0530 Subject: [PATCH 6/8] docs: update Helm install guide to use registry.k8s.io OCI chart Update Helm chart installation, values customization, and upgrade instructions to use the official registry.k8s.io OCI registry. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 44 ++++++++++++++++-------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index d81360fb..e1e778b1 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -64,24 +64,35 @@ docker pull $REPO:$TAG ``` ### Option 2: Helm Chart -The chart lives in the repository under `charts/node-readiness-controller`. Published chart releases via `registry.k8s.io` OCI are still work in progress, so install it from a checkout for now. +The official Helm chart is published to the OCI registry at `registry.k8s.io/node-readiness-controller/charts/node-readiness-controller`. ```sh -git clone https://github.com/kubernetes-sigs/node-readiness-controller.git -cd node-readiness-controller +# Replace with the desired version +VERSION=0.5.0 -helm install node-readiness-controller ./charts/node-readiness-controller \ +helm install node-readiness-controller \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${VERSION} \ --namespace nrr-system --create-namespace ``` -Requires Helm 3.x. This deploys the controller with the same defaults as the standard manifest: leader election on, metrics off, and the validating webhook off. +Requires Helm 3.8+ (native OCI support). This deploys the controller with the same defaults as the standard manifest: leader election on, metrics off, and the validating webhook off. + +> [!NOTE] +> You can also install the chart directly from a local repository checkout if developing locally: +> ```sh +> helm install node-readiness-controller ./charts/node-readiness-controller \ +> --namespace nrr-system --create-namespace +> ``` For anything beyond a couple of overrides, keep your settings in a file instead of a long `--set` list: ```sh -helm show values ./charts/node-readiness-controller > custom-values.yaml +helm show values oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller --version ${VERSION} > custom-values.yaml -helm install node-readiness-controller ./charts/node-readiness-controller \ +helm install node-readiness-controller \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${VERSION} \ --namespace nrr-system --create-namespace \ -f custom-values.yaml ``` @@ -99,7 +110,9 @@ Everything beyond the core controller is opt-in, matching the kustomize componen The webhook rejects rules whose taint key and effect collide with an existing rule over an overlapping node selector, so it is worth enabling in production. ```sh -helm install node-readiness-controller ./charts/node-readiness-controller \ +helm install node-readiness-controller \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${VERSION} \ --namespace nrr-system --create-namespace \ --set certManager.enabled=true \ --set webhook.enabled=true \ @@ -152,12 +165,12 @@ nodeReadinessRules: #### Upgrading -Pull the version of the chart you want and upgrade the release in place. Values you set at install time are carried over, so only pass the ones you are changing: +Upgrade the release in place using the OCI chart. Values you set at install time are carried over, so only pass the ones you are changing: ```sh -git pull - -helm upgrade node-readiness-controller ./charts/node-readiness-controller \ +helm upgrade node-readiness-controller \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${VERSION} \ --namespace nrr-system \ -f custom-values.yaml ``` @@ -167,7 +180,10 @@ helm upgrade node-readiness-controller ./charts/node-readiness-controller \ Check what changed before applying it to a live cluster: ```sh -helm diff upgrade node-readiness-controller ./charts/node-readiness-controller --namespace nrr-system # needs the helm-diff plugin +helm diff upgrade node-readiness-controller \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${VERSION} \ + --namespace nrr-system # needs the helm-diff plugin ``` Read the CRD note below first. Helm will not update the CRD for you, so a chart bump that changes the schema needs that step done by hand. @@ -179,7 +195,7 @@ Helm installs the CRD from the chart's `crds/` directory on first install only. Before moving to a chart version that changes the `NodeReadinessRule` schema, apply the CRD yourself: ```sh -kubectl apply -f charts/node-readiness-controller/crds/nodereadinessrules.readiness.node.x-k8s.io.yaml +kubectl apply -f https://github.com/kubernetes-sigs/node-readiness-controller/releases/download/${VERSION}/crds.yaml ``` Skipping this leaves the old schema in place, and rules using newly added fields are rejected by the API server even though the controller supports them. From 2bf74379aafb3c58b9574860c60ea45dd1c45dd7 Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Tue, 25 Aug 2026 09:13:22 +0530 Subject: [PATCH 7/8] docs: validate the Helm steps against the published 0.5.0 chart Verified the OCI chart resolves and pulls, appVersion v0.5.0, so the install commands are correct as written. Three fixes that came out of running them. The chart version had been written into a VERSION variable, which Option 1 already uses for the controller release tag. They are different numbers so reusing the name was asking for trouble. Renamed to CHART_VERSION. The CRD upgrade step reused that same VERSION, set far earlier in Option 1, so a reader arriving through the Helm path never had it. It now derives the release from the chart's own appVersion. The local checkout install needs image.tag set. make build-helm injects the real version and appVersion at package time, so Chart.yaml in the repository keeps a placeholder. Installing from a checkout without an override deploys v0.4.1 today rather than the release you checked out. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 35 ++++++++++++++++-------- 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index e1e778b1..35f7e37e 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -67,32 +67,39 @@ docker pull $REPO:$TAG The official Helm chart is published to the OCI registry at `registry.k8s.io/node-readiness-controller/charts/node-readiness-controller`. ```sh -# Replace with the desired version -VERSION=0.5.0 +# Chart version, which is independent of the controller release tag above. +CHART_VERSION=0.5.0 helm install node-readiness-controller \ oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ - --version ${VERSION} \ + --version ${CHART_VERSION} \ --namespace nrr-system --create-namespace ``` Requires Helm 3.8+ (native OCI support). This deploys the controller with the same defaults as the standard manifest: leader election on, metrics off, and the validating webhook off. > [!NOTE] -> You can also install the chart directly from a local repository checkout if developing locally: +> You can also install the chart from a local checkout when developing: > ```sh > helm install node-readiness-controller ./charts/node-readiness-controller \ -> --namespace nrr-system --create-namespace +> --namespace nrr-system --create-namespace \ +> --set image.tag=v0.5.0 > ``` +> +> Set `image.tag` explicitly here. The real chart and app versions are injected at +> package time by `make build-helm`, so `Chart.yaml` in the repository keeps a +> placeholder `appVersion`. Installing straight from a checkout without an override +> therefore deploys whatever that placeholder points at rather than the release you +> checked out. For anything beyond a couple of overrides, keep your settings in a file instead of a long `--set` list: ```sh -helm show values oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller --version ${VERSION} > custom-values.yaml +helm show values oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller --version ${CHART_VERSION} > custom-values.yaml helm install node-readiness-controller \ oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ - --version ${VERSION} \ + --version ${CHART_VERSION} \ --namespace nrr-system --create-namespace \ -f custom-values.yaml ``` @@ -112,7 +119,7 @@ The webhook rejects rules whose taint key and effect collide with an existing ru ```sh helm install node-readiness-controller \ oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ - --version ${VERSION} \ + --version ${CHART_VERSION} \ --namespace nrr-system --create-namespace \ --set certManager.enabled=true \ --set webhook.enabled=true \ @@ -170,7 +177,7 @@ Upgrade the release in place using the OCI chart. Values you set at install time ```sh helm upgrade node-readiness-controller \ oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ - --version ${VERSION} \ + --version ${CHART_VERSION} \ --namespace nrr-system \ -f custom-values.yaml ``` @@ -182,7 +189,7 @@ Check what changed before applying it to a live cluster: ```sh helm diff upgrade node-readiness-controller \ oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ - --version ${VERSION} \ + --version ${CHART_VERSION} \ --namespace nrr-system # needs the helm-diff plugin ``` @@ -192,10 +199,14 @@ Read the CRD note below first. Helm will not update the CRD for you, so a chart Helm installs the CRD from the chart's `crds/` directory on first install only. It does not upgrade or remove it on `helm upgrade` or `helm uninstall`. -Before moving to a chart version that changes the `NodeReadinessRule` schema, apply the CRD yourself: +Before moving to a chart version that changes the `NodeReadinessRule` schema, apply the CRD yourself. Use the controller release that the chart version ships, which `helm show chart` reports as its `appVersion`: ```sh -kubectl apply -f https://github.com/kubernetes-sigs/node-readiness-controller/releases/download/${VERSION}/crds.yaml +RELEASE=$(helm show chart \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${CHART_VERSION} | awk '/^appVersion:/ {print $2}') + +kubectl apply -f https://github.com/kubernetes-sigs/node-readiness-controller/releases/download/${RELEASE}/crds.yaml ``` Skipping this leaves the old schema in place, and rules using newly added fields are rejected by the API server even though the controller supports them. From e65dc4767c1a3104abfacf8e4a64ca7ac569aa4e Mon Sep 17 00:00:00 2001 From: tejassinghbhati Date: Tue, 1 Sep 2026 07:02:13 +0530 Subject: [PATCH 8/8] docs: trim the Helm section per review Four changes from ajaysundark's review. Dropped the local checkout install note, which also removes the make build-helm reference. That is a development workflow rather than something a user installing the controller needs. Reworked the values section so it is not opinionated about --set versus a values file. It now points at helm show values for the full list and shows both forms without preferring either. Removed the section on shipping rules through the nodeReadinessRules value, along with the uninstall caution that depended on it, since the UX there may get a second look. Signed-off-by: tejassinghbhati --- docs/book/src/user-guide/installation.md | 54 +++++------------------- 1 file changed, 11 insertions(+), 43 deletions(-) diff --git a/docs/book/src/user-guide/installation.md b/docs/book/src/user-guide/installation.md index 35f7e37e..04ff4ae3 100644 --- a/docs/book/src/user-guide/installation.md +++ b/docs/book/src/user-guide/installation.md @@ -78,24 +78,20 @@ helm install node-readiness-controller \ Requires Helm 3.8+ (native OCI support). This deploys the controller with the same defaults as the standard manifest: leader election on, metrics off, and the validating webhook off. -> [!NOTE] -> You can also install the chart from a local checkout when developing: -> ```sh -> helm install node-readiness-controller ./charts/node-readiness-controller \ -> --namespace nrr-system --create-namespace \ -> --set image.tag=v0.5.0 -> ``` -> -> Set `image.tag` explicitly here. The real chart and app versions are injected at -> package time by `make build-helm`, so `Chart.yaml` in the repository keeps a -> placeholder `appVersion`. Installing straight from a checkout without an override -> therefore deploys whatever that placeholder points at rather than the release you -> checked out. - -For anything beyond a couple of overrides, keep your settings in a file instead of a long `--set` list: +`helm show values` lists everything the chart exposes, and redirecting it gives you a starting point to edit: ```sh helm show values oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller --version ${CHART_VERSION} > custom-values.yaml +``` + +Override them with `--set`, or from a file with `-f`: + +```sh +helm install node-readiness-controller \ + oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ + --version ${CHART_VERSION} \ + --namespace nrr-system --create-namespace \ + --set metrics.enabled=true helm install node-readiness-controller \ oci://registry.k8s.io/node-readiness-controller/charts/node-readiness-controller \ @@ -145,31 +141,6 @@ The values under `controller` map to the manager's own flags, and each one is on Raising the two concurrency values is the usual response to readiness taints lagging behind node joins on a large cluster. `enableNodeStateMetrics` adds the per-rule `node_readiness_nodes_by_state` gauge at the cost of extra API reads on node updates, so turn it on when you want the fleet view and can afford the reads. -#### Managing rules through the chart - -`NodeReadinessRule` objects can be shipped with the release through the `nodeReadinessRules` value: - -```yaml -nodeReadinessRules: - - name: kube-proxy-unhealthy-noschedule - enforcementMode: continuous - conditions: - - type: KubeProxyUnhealthy - requiredStatus: "False" - taint: - key: readiness.k8s.io/KubeProxyUnhealthy - value: "true" - effect: NoSchedule - nodeSelector: - matchLabels: - kubernetes.io/os: linux -``` - -`nodeSelector` is required on every entry. Set it explicitly, since an empty selector matches every node in the cluster. - -> [!NOTE] -> With the validating webhook enabled, apply rules only once the controller is serving admission requests. On a first install the webhook is not ready while the rules in the same release are being created, so install the controller first and add the rules in a follow-up `helm upgrade`. - #### Upgrading Upgrade the release in place using the OCI chart. Values you set at install time are carried over, so only pass the ones you are changing: @@ -313,9 +284,6 @@ The controller uses a **finalizer** (`readiness.node.x-k8s.io/cleanup-taints`) o # Remove the manifest from /etc/kubernetes/manifests/ on all control-plane nodes ``` - > [!CAUTION] - > Rules declared through the chart's `nodeReadinessRules` value are part of the release, so `helm uninstall` deletes them and the controller in one operation. Helm does not wait for the finalizer to run, which is exactly the situation described in [Recovering from Stuck Resources](#recovering-from-stuck-resources). Delete the rules and let them finish terminating before uninstalling the release. - 3. **Uninstall CRDs** (Optional): ```sh kubectl delete -k config/crd