Fix: add dry-run AzureCluster create to ensure CA bundle availability

[mboersma]

What type of PR is this?

/kind flake

What this PR does / why we need it:

Fixes a flaky e2e test failure where the kube-apiserver hasn't yet picked up updated webhook CA bundles from its informer cache, even though cert-manager's cainjector has already populated them on the webhook configurations: the well-known "x509 error."

After the existing check that waits for CA bundle injection into all ValidatingWebhookConfigurations and MutatingWebhookConfigurations, this adds a dry-run AzureCluster create to verify the CAPZ mutating webhook is actually reachable end-to-end with valid TLS. This closes the race window between the CA bundle being written and the apiserver serving requests through the webhook with the new certificate.

Which issue(s) this PR fixes:
Fixes #5690 (hopefully)
See also #6144, which apparently didn't work. :-(

Special notes for your reviewer:

TODOs:

• squashed commits
• includes documentation
• adds unit tests
• cherry-pick candidate

Release note:

NONE

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign fabriziopandini for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.74%. Comparing base (d359ee9) to head (6a44fd3).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6221      +/-   ##
==========================================
+ Coverage   43.66%   43.74%   +0.08%     
==========================================
  Files         289      289              
  Lines       25495    25475      -20     
==========================================
+ Hits        11132    11145      +13     
+ Misses      13561    13527      -34     
- Partials      802      803       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mboersma]

/label tide/merge-method-squash

[mboersma]

/test pull-cluster-api-provider-azure-e2e

This bug isn't deterministic, so we can't easily know if this fixes it. I'll run tests a few times and we can make a judgement call.

[mboersma]

/test pull-cluster-api-provider-azure-e2e

No failures yet... 🤞🏻