docs: add Helm installation guide - #388
Conversation
✅ Deploy Preview for node-readiness-controller canceled.
|
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: tejassinghbhati The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @tejassinghbhati. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Ayush4958
left a comment
There was a problem hiding this comment.
It's great documentation but i got some suggestion for it :-
- we should add helm upgrade section after initial installing it can be great for use
- mention that users can supply their own overrides file using
-f custom-values.yaml
Review feedback on kubernetes-sigs#388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
|
Good calls, added both in 9fde2a2. There is a values file section now with |
Review feedback on kubernetes-sigs#388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
9fde2a2 to
8b8bd68
Compare
|
Rebased on main and added a section for the controller tuning values that landed in #392, since they were not in the chart when I opened this. Checked all six flags exist in cmd/main.go and rendered them to confirm they only get passed when moved off their defaults. Chart tests are at 31 now, all green. |
Review feedback on kubernetes-sigs#388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
8b8bd68 to
70e9163
Compare
|
Rebased for the chart rename in #407, so the install, upgrade and uninstall commands and the CRD path all use node-readiness-controller now. Also added a section for the tuning values from #392 since those landed after I opened this. One extra thing in here that predates the PR, flagging it so it does not look random. The Verification section looked for pods with Happy to pull that into its own PR if you would rather keep this one to Helm. |
|
Backed out the verification selector change in f3057ce, and I got it wrong twice over. #401 was already filed and assigned for this two days before I touched it, so it was not mine to fix. And my analysis was wrong anyway. I said nothing sets a component label. examples/static-pod/node-readiness-controller.yaml sets it, and it is the only place in the repo that does. That pod has component and tier and no control-plane label, so switching the selector to control-plane=controller-manager would have broken verification for static pod installs, the one path where the command currently works. So this PR is back to just the Helm section plus the rename and tuning updates. Sorry for the noise. |
| ### Option 2: Advanced Deployment (Kustomize) | ||
| ### Option 2: Helm Chart | ||
|
|
||
| The chart lives in the repository under `charts/node-readiness-controller`. Published chart releases via `registry.k8s.io` OCI are still work in progress, so install it from a checkout for now. |
There was a problem hiding this comment.
This is now available for 0.5.0. Can you validate these steps and update the PR with changes?
There was a problem hiding this comment.
Updated in the latest commit to use the official
egistry.k8s.io OCI registry for chart install, values inspection, and upgrades (with a note kept for local checkout development).
There was a problem hiding this comment.
Validated against the published chart and updated in 0c54cbf.
It pulls fine, appVersion v0.5.0, so the install commands are right as written.
Three things came out of actually running them. The chart version was in a variable called VERSION, which Option 1 already uses for the release tag, and they are different numbers, so that is now CHART_VERSION. The CRD upgrade step reused that same VERSION set way back in Option 1, so anyone arriving via Helm never had it, it now derives the release from the chart appVersion.
The last one is worth a look beyond this PR. Installing from a local checkout deploys v0.4.1 right now, because build-helm injects version and app-version at package time and Chart.yaml keeps a placeholder. Same at the v0.5.0 tag. I have added --set image.tag to that snippet so the docs are not misleading, but the underlying thing might be worth fixing properly.
5141eff to
0c54cbf
Compare
Review feedback on kubernetes-sigs#388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
Review feedback on kubernetes-sigs#388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
0c54cbf to
fca2eef
Compare
| The official Helm chart is published to the OCI registry at `registry.k8s.io/node-readiness-controller/charts/node-readiness-controller`. | ||
|
|
||
| ```sh | ||
| # Chart version, which is independent of the controller release tag above. |
There was a problem hiding this comment.
@vitorfloriano is this because the chart cannot carry the "vX.Y.Z" pattern?
Could you help check if there's room for unifying / fixing this? Having it different would cause discovery issues as we have to keep documenting the mapping in our documentation somewhere.
There was a problem hiding this comment.
Not a Helm limitation, I checked. Helm takes v0.5.0 as a chart version fine, it packages, lints and renders, and helm show chart reads it back as v0.5.0.
The v gets dropped by one line in the Makefile:
CHART_VERSION ?= $(shell echo "$(RELEASE_VERSION)" | sed 's/^v//')That came in with #402, so the split is a choice rather than a constraint. Unifying is deleting that sed and setting Chart.yaml to v0.5.0. ct is not in the way either, check-version-increment is already false.
The case for leaving it stripped is that semver itself has no v and most published charts use the plain number, so anything parsing chart versions strictly expects 0.5.0. Helm is lenient, other consumers may not be.
Happy to do it if you want it. It touches the Makefile, Chart.yaml and the version check I added in #453, and it would remove the CHART_VERSION vs VERSION split from these docs, which is the discovery problem you are describing.
| > [!NOTE] | ||
| > You can also install the chart from a local checkout when developing: | ||
| > ```sh | ||
| > helm install node-readiness-controller ./charts/node-readiness-controller \ | ||
| > --namespace nrr-system --create-namespace \ | ||
| > --set image.tag=v0.5.0 | ||
| > ``` | ||
| > |
There was a problem hiding this comment.
We don't need to document this option. You can remove
| > --set image.tag=v0.5.0 | ||
| > ``` | ||
| > | ||
| > Set `image.tag` explicitly here. The real chart and app versions are injected at |
There was a problem hiding this comment.
I don't think the user installation documention need to capture the make commands used to create the charts
There was a problem hiding this comment.
Gone with the local checkout note in 9351d16.
| > therefore deploys whatever that placeholder points at rather than the release you | ||
| > checked out. | ||
|
|
||
| For anything beyond a couple of overrides, keep your settings in a file instead of a long `--set` list: |
There was a problem hiding this comment.
We should document the overrides possible, and show a sample how to use it in helm command. We don't need to be opinionated here about whether the user wants to use set or inject values via yaml
There was a problem hiding this comment.
Reworked in 9351d16. It now points at helm show values for the full list and shows both --set and -f without picking a side.
| `nodeSelector` is required on every entry. Set it explicitly, since an empty selector matches every node in the cluster. | ||
|
|
||
| > [!NOTE] | ||
| > With the validating webhook enabled, apply rules only once the controller is serving admission requests. On a first install the webhook is not ready while the rules in the same release are being created, so install the controller first and add the rules in a follow-up `helm upgrade`. |
There was a problem hiding this comment.
This feels like a hack. Are we better off not (documenting) supporting the rules array in helm chart for now. we could do a second look at the UX.
There was a problem hiding this comment.
Agreed, dropped the whole section in 9351d16, along with the uninstall caution that depended on it. The chart still supports the value, it is just not documented until the UX gets a second look.
The chart has been in the repo since kubernetes-sigs#163 but Helm is not mentioned anywhere in the book, so the install page only covered release manifests, kustomize and static pods. Adds a Helm section covering install from a checkout, since OCI chart releases are still WIP, which values turn on metrics, TLS and the validating webhook, and how to ship rules through the chart. Also documents two things that bite people and are specific to Helm. Helm only installs the CRD from crds/ on first install and never upgrades it, so a chart bump that changes the schema needs the CRD applied by hand. And rules declared in values are release resources, so helm uninstall removes them together with the controller and the taint finalizer has no controller left to run, which is the stuck-resource case already documented further down the page. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
Review feedback on kubernetes-sigs#388. Covers keeping settings in a values file rather than a long --set list, and upgrading an existing release, with a pointer to the CRD note since Helm will not update the CRD on upgrade. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
The chart gained a controller block with concurrency, QPS and pprof settings after this PR was opened, so document them alongside the rest of the install options. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
The chart directory and name became node-readiness-controller in kubernetes-sigs#407, so the install, upgrade and uninstall commands in the Helm section needed updating along with the path to the bundled CRD. Also fixes the selector in Verification. It looked for component=node-readiness-controller, and nothing sets a component label, not the chart and not the kustomize manifests, so the command matched no pods on either path. Both set control-plane=controller-manager, so use that. A reader following the new Helm section lands on those commands straight after installing. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
Backing this out. kubernetes-sigs#401 was already filed and assigned for it, two days before I touched it, and the analysis there is better than mine. I claimed nothing sets a component label. That is wrong. examples/static-pod/node-readiness-controller.yaml sets it, and it is the only place in the repo that does. That pod carries component and tier and no control-plane label at all, so swapping the selector to control-plane=controller-manager would have broken verification for static pod installs, which is the one path where the documented command works today. Leaving the selector alone so kubernetes-sigs#401 can fix it properly per install path. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
Update Helm chart installation, values customization, and upgrade instructions to use the official registry.k8s.io OCI registry. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
Verified the OCI chart resolves and pulls, appVersion v0.5.0, so the install commands are correct as written. Three fixes that came out of running them. The chart version had been written into a VERSION variable, which Option 1 already uses for the controller release tag. They are different numbers so reusing the name was asking for trouble. Renamed to CHART_VERSION. The CRD upgrade step reused that same VERSION, set far earlier in Option 1, so a reader arriving through the Helm path never had it. It now derives the release from the chart's own appVersion. The local checkout install needs image.tag set. make build-helm injects the real version and appVersion at package time, so Chart.yaml in the repository keeps a placeholder. Installing from a checkout without an override deploys v0.4.1 today rather than the release you checked out. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
Four changes from ajaysundark's review. Dropped the local checkout install note, which also removes the make build-helm reference. That is a development workflow rather than something a user installing the controller needs. Reworked the values section so it is not opinionated about --set versus a values file. It now points at helm show values for the full list and shows both forms without preferring either. Removed the section on shipping rules through the nodeReadinessRules value, along with the uninstall caution that depended on it, since the UX there may get a second look. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
9351d16 to
e65dc47
Compare
Description
The chart has been in the repo since #163 but Helm is not mentioned anywhere in the book. The install page only covered release manifests, kustomize and static pods, so there was no documented Helm path at all.
Adds a Helm section to the installation guide covering install from a checkout, since OCI chart releases are still WIP, which values turn on metrics, TLS and the validating webhook, and how to ship rules through
nodeReadinessRules.It also documents two things that are specific to Helm and easy to get caught by:
Helm installs the CRD from
crds/on first install only and never touches it again, so a chart bump that changes the schema needs the CRD applied by hand or rules using new fields get rejected while the controller supports them. This is in the chart README already but not where someone following the install guide would see it.Rules declared in
nodeReadinessRulesare release resources, sohelm uninstalldeletes them together with the controller. The taint finalizer then has no controller left to run it, which is the stuck-resource case already documented further down the same page.Related Issue
Fixes #387
Type of Change
/kind documentation
Testing
Docs only, no code change. I checked every value and default against the chart rather than writing from memory.
leaderElection.enabledtrue,metrics.enabledandmetrics.securefalse,webhook.enabledandvalidatingWebhook.enabledfalse,certManager.enabledfalse. TheValidatingWebhookConfigurationis gated onand .Values.webhook.enabled .Values.validatingWebhook.enabled, which is why the guide says both are needed.I also rendered the exact commands in the guide. The default install produces the controller, RBAC and service account only. The full one produces what the table promises:
Checklist
make testpassesmake lintpassesDoes this PR introduce a user-facing change?