diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 50bba2964..5628fcb1c 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 08/13/2026 +ms.date: 08/19/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit @@ -33,6 +33,7 @@ The following section lists features and enhancements that are currently in deve - Replaced redundant `tolower()` comparisons in hub KQL with case-insensitive operators (`has`, `=~`, `!~`) so the engine can use the term index instead of scanning every row ([#2213](https://github.com/microsoft/finops-toolkit/issues/2213)). - Replaced whole-term `contains` matches with `has` across hub KQL and the query catalog (resource ID paths, licensing phrases, SKU description terms) and added a per-row operator-equivalence regression harness with unit test coverage ([#2220](https://github.com/microsoft/finops-toolkit/pull/2220)). - **Fixed** + - Fixed private-network deployments that Azure Policy blocked when `defaultOutboundAccess` was omitted. Private mode subnets now set it to `false`, while an Azure Files private endpoint supports deployment-script storage and the NAT Gateway provides required container egress ([#2258](https://github.com/microsoft/finops-toolkit/issues/2258), [#2259](https://github.com/microsoft/finops-toolkit/pull/2259)). - Fixed the `ContractedCost` recompute guard to compare with a null-safe tolerance instead of exact float equality, eliminating millions of no-op rewrites that polluted the `x_SourceValues` audit trail while preserving the null-cost backfill and no longer overwriting an existing cost when the unit price is missing ([#2216](https://github.com/microsoft/finops-toolkit/issues/2216)). - Fixed the SQL VMs without Azure Hybrid Benefit recommendation query to join on the SQL VM `virtualMachineResourceId` instead of a case-sensitive VM name match that skipped VMs with uppercase names and dropped duplicate names, and made all Azure Resource Graph join kinds explicit so no query relies on the `innerunique` default ([#2225](https://github.com/microsoft/finops-toolkit/pull/2225)). - Switched dimension enrichment in the v1_0/v1_2 ingestion transforms (`PricingUnits`, `Regions`, `ResourceTypes`, `Services`) from `join` to the broadcast-optimized `lookup` operator and deduplicated the `Services` mapping per resource type to prevent cost row fan-out ([#2225](https://github.com/microsoft/finops-toolkit/pull/2225)). diff --git a/src/powershell/Public/Deploy-FinOpsHub.ps1 b/src/powershell/Public/Deploy-FinOpsHub.ps1 index 5f62f99ad..6140e0af8 100644 --- a/src/powershell/Public/Deploy-FinOpsHub.ps1 +++ b/src/powershell/Public/Deploy-FinOpsHub.ps1 @@ -85,7 +85,7 @@ Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13. .PARAMETER NetworkMode - Optional. Network mode for the hub: 'public' (default), 'vnet' (private endpoints, default outbound), or 'private' (private endpoints + NAT Gateway, subnets locked down with defaultOutboundAccess=false - required when the 'Subnets should be private' policy is enforced). + Optional. Network mode for the hub: 'public' (default), 'vnet' (private endpoints, default outbound), or 'private' (private endpoints + NAT Gateway for controlled outbound access - required when the 'Subnets should be private' policy is enforced). .PARAMETER DisablePublicAccess Optional. Deprecated. Use -NetworkMode 'vnet' or -NetworkMode 'private' instead. When set without -NetworkMode, behaves as -NetworkMode 'vnet'. Ignored when -NetworkMode is supplied. diff --git a/src/powershell/Tests/Unit/HubsPrivateNetworking.Tests.ps1 b/src/powershell/Tests/Unit/HubsPrivateNetworking.Tests.ps1 new file mode 100644 index 000000000..da202075d --- /dev/null +++ b/src/powershell/Tests/Unit/HubsPrivateNetworking.Tests.ps1 @@ -0,0 +1,42 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +Describe 'HubsPrivateNetworking' { + + BeforeAll { + $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path + $bicepPath = Join-Path $repoRoot 'src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep' + $templatePath = Join-Path ([System.IO.Path]::GetTempPath()) "finops-hub-infrastructure-$([guid]::NewGuid()).json" + } + + AfterAll { + Remove-Item -Path $templatePath -Force -ErrorAction SilentlyContinue + } + + It 'Should keep private subnets and use the supported deployment-script network path' { + if (-not (Get-Command 'bicep' -ErrorAction SilentlyContinue)) + { + Set-ItResult -Skipped -Because 'bicep CLI not found' + return + } + + bicep build $bicepPath --outfile $templatePath + if ($LASTEXITCODE -ne 0) + { + throw 'Failed to compile the FinOps hub infrastructure template.' + } + + $templateText = Get-Content -Path $templatePath -Raw + $template = $templateText | ConvertFrom-Json -Depth 100 + + ([regex]::Matches($template.variables.subnets, "'defaultOutboundAccess', not\(parameters\('hub'\)\.options\.natGateway\)")).Count | Should -Be 3 + ([regex]::Matches($template.variables.subnets, "'natGateway'")).Count | Should -Be 2 + + $template.resources.scriptStorageAccount.properties.publicNetworkAccess | Should -Be 'Disabled' + $template.resources.scriptStorageAccount.properties.networkAcls.PSObject.Properties.Name | Should -Not -Contain 'virtualNetworkRules' + + $template.resources.scriptEndpoint.properties.privateLinkServiceConnections[0].properties.groupIds | Should -Contain 'file' + $template.resources.'scriptEndpoint::scriptPrivateDnsZoneGroup'.properties.privateDnsZoneConfigs[0].properties.privateDnsZoneId | Should -Match "dnsZones\.file\.name" + $template.resources.filePrivateDnsZone.name | Should -Match "dnsZones\.file\.name" + } +} diff --git a/src/scripts/Deploy-Hub.ps1 b/src/scripts/Deploy-Hub.ps1 index 3cc008d9a..c4741aa3a 100644 --- a/src/scripts/Deploy-Hub.ps1 +++ b/src/scripts/Deploy-Hub.ps1 @@ -92,7 +92,7 @@ Optional. Deploy with private networking (VNet and private endpoints, default outbound access). Equivalent to -NetworkMode vnet. Kept for back-compat. Default: false. .PARAMETER NetworkMode - Optional. Network mode: 'public' (default), 'vnet' (private endpoints, default outbound), or 'private' (private endpoints + NAT Gateway, subnets locked down — required when the 'Subnets should be private' policy is enforced). + Optional. Network mode: 'public' (default), 'vnet' (private endpoints, default outbound), or 'private' (private endpoints + NAT Gateway for controlled outbound access — required when the 'Subnets should be private' policy is enforced). .PARAMETER VirtualNetworkAddressPrefix Optional. Virtual network address prefix for private networking. Requires a /26 CIDR block. When set, also sets -Private. Default: "10.20.30.0/26". diff --git a/src/templates/finops-hub/createUiDefinition.json b/src/templates/finops-hub/createUiDefinition.json index d7ea79097..52f5ae4e2 100644 --- a/src/templates/finops-hub/createUiDefinition.json +++ b/src/templates/finops-hub/createUiDefinition.json @@ -954,7 +954,7 @@ { "name": "enableNatGateway", "type": "Microsoft.Common.CheckBox", - "label": "Deploy NAT Gateway and disable default outbound access", + "label": "Deploy NAT Gateway for controlled outbound access", "toolTip": "Required by the September 2025 'Subnets should be private' policy. Adds a Standard NAT Gateway and Standard static Public IP attached to the script and Data Explorer subnets to provide controlled outbound internet access. Leave unchecked if you have your own egress path (UDR to firewall, peered hub with NAT, etc.).", "defaultValue": false, "visible": "[equals(steps('advanced').networking.enablePublicAccess, false)]" diff --git a/src/templates/finops-hub/main.bicep b/src/templates/finops-hub/main.bicep index 687c26174..d93fe98c4 100644 --- a/src/templates/finops-hub/main.bicep +++ b/src/templates/finops-hub/main.bicep @@ -159,7 +159,7 @@ param dataExplorerFinalRetentionInMonths int = 13 @description('Optional. Enable public access to FinOps hubs resources. Default: true.') param enablePublicAccess bool = true -@description('Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, subnets disable Azure default outbound access and route through the NAT Gateway. Ignored when enablePublicAccess is true. Default: false.') +@description('Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, the script and Data Explorer subnets route outbound traffic through the NAT Gateway. Ignored when enablePublicAccess is true. Default: false.') param enableNatGateway bool = false @description('Optional. Address space for the workload. Minimum /26 subnet size is required for the workload. Default: "10.20.30.0/26".') diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep index 85b5e9591..4c9934b94 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep @@ -25,6 +25,7 @@ var finopsHubSubnetName = 'private-endpoint-subnet' var scriptSubnetName = 'script-subnet' var dataExplorerSubnetName = 'dataExplorer-subnet' +// Azure Policy requires private mode subnets to set defaultOutboundAccess to false explicitly. var subnets = !hub.options.privateRouting ? [] : [ { name: finopsHubSubnetName @@ -62,11 +63,6 @@ var subnets = !hub.options.privateRouting ? [] : [ } } ] - serviceEndpoints: [ - { - service: 'Microsoft.Storage' - } - ] } } { @@ -211,9 +207,9 @@ resource vNet 'Microsoft.Network/virtualNetworks@2023-11-01' = if (hub.options.p } //------------------------------------------------------------------------------ -// NAT Gateway (provides outbound for script-subnet + dataExplorer-subnet when -// defaultOutboundAccess is disabled; required by the 'Subnets should be private' -// policy and the September 2025 implicit-outbound retirement) +// NAT Gateway (provides explicit outbound for script-subnet + dataExplorer-subnet; +// required by the 'Subnets should be private' policy and the September 2025 +// implicit-outbound retirement) //------------------------------------------------------------------------------ resource natGatewayPublicIp 'Microsoft.Network/publicIPAddresses@2023-11-01' = if (hub.options.natGateway) { @@ -296,6 +292,29 @@ resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if ( } } +// Required for deployment scripts +resource filePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting) { + name: string(hub.routing.dnsZones.file.name) + dependsOn: [ + vNet + ] + location: 'global' + tags: getHubTags(hub, 'Microsoft.Storage/privateDnsZones') + properties: {} + + resource filePrivateDnsZoneLink 'virtualNetworkLinks' = { + name: '${replace(filePrivateDnsZone.name, '.', '-')}-link' + location: 'global' + tags: getHubTags(hub, 'Microsoft.Network/privateDnsZones/virtualNetworkLinks') + properties: { + registrationEnabled: false + virtualNetwork: { + id: hub.routing.networkId + } + } + } +} + // Required for Azure Data Explorer resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting) { name: string(hub.routing.dnsZones.queue.name) @@ -363,22 +382,16 @@ resource scriptStorageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = i isHnsEnabled: false minimumTlsVersion: 'TLS1_2' allowBlobPublicAccess: false - publicNetworkAccess: 'Enabled' + publicNetworkAccess: 'Disabled' networkAcls: { bypass: 'AzureServices' defaultAction: 'Deny' - virtualNetworkRules: [ - { - id: hub.routing.subnets.scripts - action: 'Allow' - } - ] } } } resource scriptEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (hub.options.privateRouting) { - name: '${scriptStorageAccount.name}-blob-ep' + name: '${scriptStorageAccount.name}-file-ep' dependsOn: [ vNet::scriptSubnet ] @@ -393,20 +406,20 @@ resource scriptEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (hu name: 'scriptLink' properties: { privateLinkServiceId: scriptStorageAccount.id - groupIds: ['blob'] + groupIds: ['file'] } } ] } resource scriptPrivateDnsZoneGroup 'privateDnsZoneGroups' = { - name: 'blob-endpoint-zone' + name: 'file-endpoint-zone' properties: { privateDnsZoneConfigs: [ { - name: blobPrivateDnsZone.name + name: filePrivateDnsZone.name properties: { - privateDnsZoneId: blobPrivateDnsZone.id + privateDnsZoneId: filePrivateDnsZone.id } } ] diff --git a/src/templates/finops-hub/modules/fx/hub-types.bicep b/src/templates/finops-hub/modules/fx/hub-types.bicep index d9b197e75..f3727bf0b 100644 --- a/src/templates/finops-hub/modules/fx/hub-types.bicep +++ b/src/templates/finops-hub/modules/fx/hub-types.bicep @@ -28,6 +28,7 @@ type IdNameObject = { id: string, name: string } dnsZones: { blob: 'Resource ID and name for the blob storage DNS zone.' dfs: 'Resource ID and name for the DFS storage DNS zone.' + file: 'Resource ID and name for the file storage DNS zone.' queue: 'Resource ID and name for the queue storage DNS zone.' table: 'Resource ID and name for the table storage DNS zone.' } @@ -46,6 +47,7 @@ type HubRoutingProperties = { dnsZones: { blob: IdNameObject dfs: IdNameObject + file: IdNameObject queue: IdNameObject table: IdNameObject } @@ -72,7 +74,7 @@ type HubRoutingProperties = { keyVaultSku: 'KeyVault SKU. Allowed values: "standard", "premium".' keyVaultEnablePurgeProtection: 'Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.' networkAddressPrefix: 'Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.' - natGateway: 'Indicates whether a NAT Gateway should be deployed for controlled outbound internet access. When enabled, subnets disable Azure default outbound access and route through the NAT Gateway.' + natGateway: 'Indicates whether a NAT Gateway should be deployed for controlled outbound internet access. When enabled, the script and Data Explorer subnets route outbound traffic through the NAT Gateway.' privateRouting: 'Indicates whether private network routing is enabled.' publisherIsolation: 'Indicates whether FinOps hub resources should be separated by publisher for advanced security.' storageInfrastructureEncryption: 'Indicates whether infrastructure encryption is enabled for the storage account.' @@ -227,6 +229,7 @@ func newHubInternal( dnsZones: { blob: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('blob') dfs: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('dfs') + file: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('file') queue: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('queue') table: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('table') } diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index 0c9e6b9e5..0b18ee35e 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -170,7 +170,7 @@ param dataExplorerFinalRetentionInMonths int = 13 @description('Optional. Enable public access to the data lake. Default: true.') param enablePublicAccess bool = true -@description('Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, subnets disable Azure default outbound access and route through the NAT Gateway. Ignored when enablePublicAccess is true. Default: false.') +@description('Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, the script and Data Explorer subnets route outbound traffic through the NAT Gateway. Ignored when enablePublicAccess is true. Default: false.') param enableNatGateway bool = false @description('Optional. Address space for the workload. Minimum /26 subnet size is required for the workload. Default: "10.20.30.0/26".')