feat: label the CRD with its managing installation and guard duplicat… - #440
feat: label the CRD with its managing installation and guard duplicat…#440yindia wants to merge 1 commit into
Conversation
✅ Deploy Preview for node-readiness-controller canceled.
|
|
Welcome @yindia! |
|
Hi @yindia. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
@ajaysundarkI'm not sure it's the best approach, it false-positives on reinstall since Helm keeps crds/ CRDs after uninstall. Open to a better ideas if anyone has one |
|
On the uninstall/reinstall false positive: The reason Two ways to resolve this without dropping the guard:
|
|
@DsThakurRawat Thanks for the detailed look. This finds the false positive well. Option 2 does not work here. Both the provider install and my install carry the default Option 1 fixes the uninstall and reinstall cycle. But it misses the main case. A provider can run the controller in a control-plane namespace that I cannot see. Then there is no Deployment in I think one version gets both. Keep the live workload as the signal, as you suggested. But look it up across all namespaces, not only the release namespace: Then match on Does this match the reinstall case that you saw? If yes, I will push it as a follow-up commit. cc: @ajaysundark |
|
yes, that matches the case i saw, and i checked it on a live cluster rather than reasoning about it. kind v1.31.2, helm 3.16.3, your branch at 55e6a23. the uninstall half holds up: while setting that up i hit something bigger than the reinstall case though, and i think it blocks the PR as it stands. the guard fires on a first install into an empty cluster. helm applies from a verified clean state, twice: that timestamp is from the install that then refused itself, and no release was recorded. two reasons this got past CI, and i think both are worth fixing next to it:
if you do go with the cluster-wide Deployment lookup, two things bit me while writing a probe chart for it.
the other one is that a Deployment carrying no labels at all is ordinary rather than an edge case. one of the two in a stock kind cluster is could be wrong about the ordering if you get a different result, but two clean runs said the same thing. can send the probe chart over if it saves you the setup. |
55e6a23 to
8103423
Compare
|
You're right on all counts, and thanks for testing it on a live cluster. I reproduced the same thing: a fresh Fixed by keying the guard on the controller Deployment instead of the CRD. The Deployment is applied after rendering, so one found during the check belongs to a different, already-running install. Both CI was passing because the verify step ran Verified on kind:
No need for the probe chart, but thanks for the offer. Pushed the fix. |
|
/ok-to-test |
|
/assign @ajaysundark |
| annotations: | ||
| controller-gen.kubebuilder.io/version: v0.19.0 | ||
| labels: | ||
| app.kubernetes.io/managed-by: node-readiness-controller |
There was a problem hiding this comment.
AFAIU, this is set as 'managed-by: kustomize', we can default to managed-by: helm' for charts. If there's way to expose this to be user-set value in helm, that'd be preferred.
I expect in managed-environments this would map to managed-by: <provider>, we should document as best-practice. so if there's conflict, it'd be properly bubble up as an error
8103423 to
da1e7ab
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: yindia The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
da1e7ab to
aad7ea1
Compare
Summary
Stamps the
NodeReadinessRuleCRD with amanaged-bylabel naming the installation that owns it, and refuses a freshhelm installwhen the CRD already exists.Design notes
Why the CRD carries the label. It's cluster scoped and singleton, so it's the one artifact every install flow contends over. It's set with a controller-gen marker, so
make manifestsregenerates it inconfig/crd/bases, andverify-chart-drift.shdiffs that against the chart'scrds/copy to catch the two drifting apart.Why a template guard, not a pre-install hook. The Helm guard uses
lookup+failin a template — same outcome without a ServiceAccount, ClusterRole or image, and it aborts at render time so the guidance lands in the terminal rather than Job logs.Why it's keyed on
.Release.IsInstallplus CRD existence, not the label value. Both installs may be carrying the default value, and that's precisely the accident being guarded against, so a value comparison would pass straight through it. Upgrades skip the check naturally.Two things worth a reviewer's attention
Label is CRD-only. Helm 3 forces
managed-by: Helmon resources it owns, so stamping the Deployment/RBAC/Services doesn't stick. The CRD works because Helm neither templates nor trackscrds/.Guard only covers
helm install.kubectl apply, kustomize and the static-pod path have none. A controller startup check would close the gap, but it's ruled out by alternative (1) in the issue — left out and documented. Happy to reconsider.Also included
Chart-rendered
NodeReadinessRuleCRs now get the standard chart labels, which they were missing. Unrelated tidy-up, happy to split it out.Related Issue
Fixes #438
Type of Change
/kind feature
Testing
helm installinto a clean cluster: CRD is created with app.kubernetes.io/managed-by=node-readiness-controller.helm installwith a second release name while the CRD exists: render aborts with the guidance message, nothing is applied.helm upgradeon the existing release: check is skipped, upgrade succeeds.helm templateoutput diffed againstdist/crds.yaml;verify-chart-drift.shpasses.Checklist
make testpassesmake lintpassesDoes this PR introduce a user-facing change?
Yes