{AKS} Stabilize aks-preview live tests - #10184
Open
FumingZhang wants to merge 5 commits into
Open
Conversation
|
Hi FumingZhang, |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
microsoft-github-policy-service
Bot
requested review from
Julie Zhu (yanzhudd) and
Yong Zhang (yonzhan)
August 5, 2026 06:09
microsoft-github-policy-service
Bot
added
Auto-Assign
Member
|
/azp run |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
FumingZhang
force-pushed
the
fix/aks-preview-live-test-retries
branch
from
August 5, 2026 06:25
6aac9a5 to
37286d0
Compare
FumingZhang
changed the title
Member
|
/azp run |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
Member
|
/azp run |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
FumingZhang
force-pushed
the
fix/aks-preview-live-test-retries
branch
from
August 6, 2026 07:16
978e3da to
766460f
Compare
FumingZhang
changed the title
Member
|
/azp run |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
Port bounded AKS operation-conflict and provisioning-state retries to extension live tests while preventing retry-enabled runs from saving unreplayable recordings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Handle mapping-based SDK models in managed-system and kollect flows, preserve a valid IPv4 outbound count for dual-stack clusters, and stop pinning live tests to Kubernetes versions that have moved to LTS-only support.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Clear SDK discriminator fields when reducing ManagedSystem profiles and preserve dual-stack defaults without exceeding the load-balancer branch limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep the SDK field name correction after removing the now-unnecessary live-matrix reduction commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
FumingZhang
force-pushed
the
fix/aks-preview-live-test-retries
branch
from
August 6, 2026 08:41
766460f to
517f018
Compare
Trigger Azure DevOps PR validation after rebasing over the automated aks-preview index release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
|
/azp run |
|
Azure Pipelines: Successfully started running 2 pipeline(s). |
FumingZhang
marked this pull request as ready for review
August 7, 2026 06:43
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to stabilize aks-preview live tests by adding extension-scoped retry/poll/refetch behavior for transient AKS operation conflicts and by making several deterministic test adjustments (e.g., Kubernetes version selection and dual-stack LB defaults) so tests remain reliable as service defaults shift.
Changes:
- Add retry-aware
ScenarioTestcommand execution for live runs (gated byAZURE_CLI_TEST_RETRY_PROVISIONING_CHECK=true), including pollingresource showfor non-terminal provisioning states and refetching settled resources with nativeaks show/nodepool show. - Make test inputs more deterministic: prefer community-supported Kubernetes versions and default IPv4 managed outbound IP count when only IPv6 count is specified.
- Improve compatibility with mapping-based SDK shapes for certain flows (storage keys and agentpool reset behavior), with additional unit tests.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/aks-preview/azext_aks_preview/tests/latest/test_loadbalancer.py | Adds coverage for IPv6 outbound count defaulting behavior. |
| src/aks-preview/azext_aks_preview/tests/latest/test_aks_provisioning_retry.py | New unit tests validating the retry/poll/refetch adapter behavior. |
| src/aks-preview/azext_aks_preview/tests/latest/test_aks_diagnostics.py | Adds tests for handling mapping- vs attribute-style SDK responses. |
| src/aks-preview/azext_aks_preview/tests/latest/test_aks_commands.py | Implements retry-enabled live cmd() flow and updates Kubernetes version selection logic. |
| src/aks-preview/azext_aks_preview/tests/latest/test_agentpool_decorator.py | Updates assertions for ManagedSystem/Machines mode agentpool field resets. |
| src/aks-preview/azext_aks_preview/managed_cluster_decorator.py | Uses shared helper to reset ManagedSystem agentpool fields. |
| src/aks-preview/azext_aks_preview/aks_diagnostics.py | Adds mapping-aware _get_storage_account_key helper and uses it for SAS generation. |
| src/aks-preview/azext_aks_preview/agentpool_decorator.py | Uses shared helper to reset agentpool fields for special modes. |
| src/aks-preview/azext_aks_preview/_loadbalancer.py | Changes managed outbound IP count behavior when only IPv6 count is specified. |
| src/aks-preview/azext_aks_preview/_helpers.py | Introduces reset_agentpool_to_name_and_mode helper used by decorators. |
Suppressed comments (2)
src/aks-preview/azext_aks_preview/tests/latest/test_aks_commands.py:262
- _get_versions now filters to KubernetesOfficial patch versions, but it still assumes the filtered list is non-empty and that there are at least two different minors. If the query returns an empty list (or only one minor), this will raise IndexError/StopIteration and break many live tests.
This issue also appears on line 288 of the same file.
sorted_supported_versions = sorted(supported_versions, key=version_to_tuple, reverse=True)
upgrade_version = sorted_supported_versions[0]
# find the first version that doesn't start with the latest major.minor.
prefix = upgrade_version[:upgrade_version.rfind('.')]
create_version = next(x for x in sorted_supported_versions if not x.startswith(prefix))
src/aks-preview/azext_aks_preview/tests/latest/test_aks_commands.py:292
- _get_version_at_least uses
next(...)with no default; if a location doesn't have any KubernetesOfficial version >= the minimum (or the query returns empty), this will raise StopIteration and fail the test run instead of falling back to default version selection.
versions = sorted(versions, key=version_to_tuple, reverse=True)
minimum = version_to_tuple(min_version)
return next(
version for version in versions if version_to_tuple(version) >= minimum
)
Comment on lines
+175
to
+179
| profile.managed_outbound_i_ps.count = ( | ||
| managed_outbound_ip_count | ||
| if managed_outbound_ip_count is not None | ||
| else profile.managed_outbound_i_ps.count or 1 | ||
| ) |
Comment on lines
+49
to
+68
| def reset_agentpool_to_name_and_mode(agentpool, mode): | ||
| """Remove all agent pool fields except the resource name and pool mode.""" | ||
| name = agentpool.name | ||
| properties = getattr(agentpool, "properties", None) | ||
| if isinstance(properties, MutableMapping): | ||
| properties.clear() | ||
| properties["mode"] = mode | ||
| agentpool.clear() | ||
| agentpool["name"] = name | ||
| agentpool["properties"] = properties | ||
| elif isinstance(agentpool, MutableMapping): | ||
| agentpool.clear() | ||
| agentpool["name"] = name | ||
| agentpool["mode"] = mode | ||
| else: | ||
| agentpool.mode = mode | ||
| for attr in list(vars(agentpool)): | ||
| if attr not in ("name", "mode") and not attr.startswith("_") and hasattr(agentpool, attr): | ||
| setattr(agentpool, attr, None) | ||
| return agentpool |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 PR Validation — ️✔️ All clear
This checklist is used to make sure that common guidelines for a pull request are followed.
Related command
az aksDescription
Stabilize
aks-previewlive tests after the Azure CLI and CLI Runner fixes have landed.Extension-specific retry handling
Port the bounded AKS live-test retry adapter into the
aks-previewscenario base:properties.provisioningState;AZURE_CLI_TEST_RETRY_PROVISIONING_CHECK=true;The Azure CLI core retry fix does not automatically cover this extension because the extension scenario class derives directly from
ScenarioTest, not the core AKS scenario class. The merged CLI Runner switch now activates this adapter for extension runs.Deterministic fixes
The previous broad live-matrix reduction has been removed. The runner now permits 12 hours, uses 10 workers, and the prior untrimmed extension run completed in 7h49m, so retaining full live coverage is preferable.
Testing
VIRTUAL_ENV=/workspace/aenv /workspace/aenv/bin/azdev test aks-preview --no-exitfirstResult:
1174 passed, 117 skipped, 11 subtests passed.Also passed:
VIRTUAL_ENV=/workspace/aenv /workspace/aenv/bin/azdev style aks-previewpython scripts/ci/test_index.py -q(9 passed, 2 skipped)git diff --checkGeneral Guidelines
azdev style <YOUR_EXT>locally? (pip install azdevrequired)python scripts/ci/test_index.py -qlocally? (pip install azdevrequired)For new extensions:
About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update
src/index.jsonautomatically.You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify
src/index.json.Live validation
Subscription:
79a7390d-3a85-432d-9f6f-a11a703c8b83(AKS CLI PR Gate).AZURE_CLI_TEST_RETRY_PROVISIONING_CHECK=true azdev test --live --series \ azext_aks_preview.test_aks_nodepool_update_with_localdns_configResult:
1 passed in 625.97s (0:10:25).A ManagedSystem scenario was also attempted, but the service rejected it because this subscription is not whitelisted for the ManagedSystem preview. The test resource group entered deletion normally.