Add AKS Automatic custom virtual network quickstart samples - #490
Add AKS Automatic custom virtual network quickstart samples#490Allen Sudbring (asudbring) wants to merge 6 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Steven Ma (@stevenjma) Thanks! |
Steven (stemaMSFT)
left a comment
There was a problem hiding this comment.
The four-sample matrix is useful and the live apply/no-op-plan/destroy coverage is strong. I found several issues to address before merge:
- All four samples default to
eastus, but every successful deployment usedwestus2becauseeastusreturnedAKSCapacityHeavyUsagefor API Server VNet Integration. The literal copy/paste path therefore uses the known failing configuration. Please use the validated region as the default, or require location and put an explicit tested-region override beside the commands in every README. - Both AzAPI variants accept
cluster_sku_name = "Base", allowing these AKS Automatic quickstarts to create a different product mode. Please hard-codeAutomaticand remove the variable/README rows. - The private-cluster text says the API server is reachable only from inside the VNet. More precisely, it uses a private IP and requires private network connectivity and DNS; peering, VPN, or ExpressRoute can provide access from outside that VNet. Please also explain that enabling a public FQDN changes DNS naming, not endpoint routability.
Please also document subnet containment/non-overlap/sizing constraints, the blast radius of VNet-scoped Network Contributor when copied to a shared VNet, and that ignore_changes = [delegation] hides delegation drift. These are guardrails for a BYO-network sample and do not require redesigning the standalone examples.
Default to the validated westus2 region, hard-code the Automatic SKU in the AzAPI variants, clarify private cluster reachability, and document BYO-network guardrails. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks for the review Steven (@stemaMSFT) — all points are addressed in 1. Samples defaulted to All four samples now default to
The literal copy/paste path now uses the working configuration. 2. AzAPI variants accepted Hard-coded in both AzAPI samples. 3. Private-cluster reachability wording Rewritten in both private samples, in the README prose and the inline
4. BYO-network guardrails Added a Network guardrails section to all four READMEs, covering the four points you raised:
Kept as documentation, per your note that this doesn't require redesigning the four standalone examples. Validation after the changes
The corresponding article PR (azure-aks-docs-pr#3646) is updated so its embedded code still matches these samples byte-for-byte, and it carries the same private-cluster correction. |
Steven Ma (stevenjma)
left a comment
There was a problem hiding this comment.
a few small changes here and then we should be good to go
|
|
||
|
|
||
| > [!NOTE] | ||
| > The default location is `westus2` because that's the region these samples were validated in. At the time of testing, `eastus` returned `AKSCapacityHeavyUsage` for API Server VNet Integration. Set `resource_group_location` to deploy elsewhere. |
There was a problem hiding this comment.
This can be removed, it's just noise for the reader
| required_providers { | ||
| azurerm = { | ||
| source = "hashicorp/azurerm" | ||
| version = "~>5.0" |
There was a problem hiding this comment.
This sample uses 5.0 but the other providers.tf file used 4.0 as the pin. I'd recommend having both set to 5.0
|
|
||
|
|
||
| > [!NOTE] | ||
| > The default location is `westus2` because that's the region these samples were validated in. At the time of testing, `eastus` returned `AKSCapacityHeavyUsage` for API Server VNet Integration. Set `resource_group_location` to deploy elsewhere. |
There was a problem hiding this comment.
same as other file, remove for noise
| } | ||
| azurerm = { | ||
| source = "hashicorp/azurerm" | ||
| version = "~>4.0" |
There was a problem hiding this comment.
same as above, pin to a consistent version for these samples
|
|
||
| The virtual network contains a subnet delegated to the cluster API server, a subnet for the user node pools, and a subnet for the managed system node pool. The cluster uses a user-assigned managed identity that's granted the Network Contributor role on the virtual network, which is required when you bring your own network. Because the cluster is private, the API server endpoint is assigned a private IP address in the virtual network instead of a public one. Reaching it requires private network connectivity to that virtual network and DNS resolution of the private FQDN, so access isn't limited to clients inside the virtual network itself: virtual network peering, a VPN gateway, or Azure ExpressRoute can all provide a path from outside it. Enabling a public FQDN changes only how the cluster is named in DNS. It doesn't make the private endpoint routable from the internet. | ||
|
|
||
| The cluster is created with the AzureRM provider's [`azurerm_kubernetes_automatic_cluster`](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_automatic_cluster) resource, which is the recommended way to declare an AKS Automatic cluster. This resource requires AzureRM provider `5.0` or later. |
There was a problem hiding this comment.
| The cluster is created with the AzureRM provider's [`azurerm_kubernetes_automatic_cluster`](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_automatic_cluster) resource, which is the recommended way to declare an AKS Automatic cluster. This resource requires AzureRM provider `5.0` or later. | |
| The cluster is created with the AzureRM provider's [`azurerm_kubernetes_automatic_cluster`](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_automatic_cluster) resource, which is the recommended way to declare an AKS Automatic cluster. This resource requires AzureRM provider `v4.81` or later. |
Set the azurerm pin to ~>5.0 in the AzAPI samples to match the provider-native ones, correct the minimum provider version to v4.81, and remove the tested-region notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Thanks Steven Ma (@stevenjma) — all four inline comments addressed in Tested-region notes ( Removed from all four READMEs. The Provider pin consistency ( All four samples now pin
Applied, and to the other provider-native README as well. I verified before propagating: Validation After The article PR (azure-aks-docs-pr#3646) is updated to match: same version correction, notes removed, and the embedded code blocks resynced byte-for-byte against these samples. |
|
Allen Sudbring (@asudbring) my test account doesn't have permissions to live deploy this scenario and verify it works. can you paste proof that the apply and destroy is successful? then i will approve. |
Summary
Adds four Terraform quickstart samples that deploy AKS Automatic clusters into a
custom (bring-your-own) virtual network, covering both a public and a private
API server, in both the AzureRM and AzAPI providers.
quickstart/101-aks-automatic-custom-network~>5.0azurerm_kubernetes_automatic_clusterquickstart/101-aks-automatic-custom-network-azapi~>2.0azapi_resourcequickstart/101-aks-automatic-private-custom-network~>5.0azurerm_kubernetes_automatic_clusterquickstart/101-aks-automatic-private-custom-network-azapi~>2.0azapi_resourceBoth providers are covered per the review feedback on #487. The AzureRM samples
use the native
azurerm_kubernetes_automatic_clusterresource; the AzAPI samplesare for cases where you need direct control over the managed cluster API payload.
Each sample creates a virtual network with a subnet delegated to the cluster API
server, a user node subnet, and a managed system node subnet, plus a
user-assigned managed identity granted Network Contributor on the virtual
network. The readmes cross-link the four samples.
Testing
All four samples were deployed and torn down against a live Azure subscription
before submission.
terraform fmt -check -diffterraform validateterraform applysku.name = Automatic,identity = UserAssigned, API server / node / system node subnet IDs matched, one Network Contributor assignmentfalsefalsetruetrueterraform plan -detailed-exitcode(post-apply)terraform destroyRegion:
westus2. Note for anyone reproducing —eastuscurrently returnsAKSCapacityHeavyUsagefor API Server VNet Integration, so testing moved towestus2. The samples still default toeastus.The private AzAPI sample sets AzAPI
create/deletetimeouts of2hbecause adefault-timeout create returned before Azure finished provisioning, leaving a
successful deployment untracked in state.
TestRecord.mdis intentionally omitted for all four — the CI pipeline generates it.