diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index c7c27e96c..f33354ae4 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 07/30/2026 +ms.date: 08/11/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit @@ -28,7 +28,10 @@ The following section lists features and enhancements that are currently in deve ### [FinOps hubs](hubs/finops-hubs-overview.md) - **Added** + - Added bring-your-own virtual network support for private networking with `virtualNetworkMode` (`new` or `existing`) and `existingVirtualNetworkResourceId` for existing virtual networks (#2163). + - Added configurable subnet names for private networking deployments (#2163). - Added VNet and private network modes, including opt-in NAT Gateway support for private mode; NAT Gateway incurs additional cost when enabled ([#2163](https://github.com/microsoft/finops-toolkit/pull/2163)). + - Added **createPrivateDnsZones** parameter (default: `true`) to skip private DNS zone and zone group creation for enterprise environments where a centralized DNS resolver manages private DNS registration. Private endpoints are still created when disabled. - **Changed** - Replaced redundant `tolower()` comparisons in hub KQL with case-insensitive operators (`has`, `=~`, `!~`) so the engine can use the term index instead of scanning every row ([#2213](https://github.com/microsoft/finops-toolkit/issues/2213)). - Replaced whole-term `contains` matches with `has` across hub KQL and the query catalog (resource ID paths, licensing phrases, SKU description terms) and added a per-row operator-equivalence regression harness with unit test coverage ([#2220](https://github.com/microsoft/finops-toolkit/pull/2220)). diff --git a/docs-mslearn/toolkit/hubs/private-networking.md b/docs-mslearn/toolkit/hubs/private-networking.md index d328c8d6e..698f04b08 100644 --- a/docs-mslearn/toolkit/hubs/private-networking.md +++ b/docs-mslearn/toolkit/hubs/private-networking.md @@ -3,7 +3,7 @@ title: Configure private networking in FinOps hubs description: Learn about data access options with FinOps hubs and how to configure secure access to your data with private endpoints. author: flanakin ms.author: micflan -ms.date: 04/01/2026 +ms.date: 08/12/2026 ms.topic: how-to ms.service: finops ms.reviewer: micflan @@ -88,7 +88,6 @@ If you need to reduce costs or simplify your FinOps hub deployment, you can remo ### Steps to remove private networking 1. **Plan the transition**: - - Identify all users and systems currently accessing the hub via private networking - Coordinate with your network administrators about the change - Schedule maintenance window as the hub will be temporarily inaccessible during the transition @@ -98,7 +97,6 @@ If you need to reduce costs or simplify your FinOps hub deployment, you can remo You have two options to redeploy your FinOps hub with public access: **Option 1: Redeploy from existing deployment** - - Navigate to your FinOps hub resource group in the Azure portal - Go to the **Deployments** tab on the resource group - Find and open the original FinOps hub deployment @@ -108,7 +106,6 @@ If you need to reduce costs or simplify your FinOps hub deployment, you can remo - Deploy the updated configuration **Option 2: Deploy latest toolkit version** - - Install the latest current version of the FinOps toolkit - Use the same resource group name, hub name, and Data Explorer cluster name as your existing deployment - These values can be obtained from the original deployment template or the config.json file in your hub storage account @@ -116,13 +113,11 @@ If you need to reduce costs or simplify your FinOps hub deployment, you can remo - Deploy with the same configuration to update your existing hub 3. **Verify the changes**: - - Confirm that storage accounts, Data Explorer, and Key Vault are accessible via public endpoints - Test data access from Power BI and other connected systems - Verify that Azure Data Factory pipelines continue to run successfully 4. **Clean up networking resources** (optional): - - Once you've confirmed the hub is working correctly with public access, you can delete the networking resources to stop incurring networking costs - Delete resources in the following order to avoid dependency conflicts: 1. Private endpoints @@ -155,19 +150,82 @@ If you need to reduce costs or simplify your FinOps hub deployment, you can remo ## FinOps hub virtual network -When private access is selected, your FinOps hub instance includes a virtual network to ensure communication between its various components remain private. +When private access is selected, choose your mode using **virtualNetworkMode**: + +- Set **virtualNetworkMode** to `new` to create a new virtual network and subnets. +- Set **virtualNetworkMode** to `existing` to use your existing virtual network. In this mode, set **existingVirtualNetworkResourceId**. + +### Virtual network mode and parameter usage + +The following table shows which network parameters are used in each mode when private access is enabled (**enablePublicAccess** = `false`). + +| Parameter | `new` mode | `existing` mode | Notes | +| ------------------------------------ | ---------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **virtualNetworkMode** | Used | Used | Selects whether FinOps hubs creates a VNet or reuses one. | +| **privateEndpointSubnetName** | Used | Used | Subnet name used for storage, Key Vault, and script storage private endpoints. | +| **scriptSubnetName** | Used | Used | Subnet name used for deployment script container groups. | +| **dataExplorerSubnetName** | Used | Used | Subnet name used for Azure Data Explorer private endpoint. | +| **createPrivateDnsZones** | Always on | Used | In `new` mode, FinOps hubs always creates private DNS zones and links them to the created virtual network. In `existing` mode, set to `false` to skip private DNS zones, zone groups, and virtual network links. | +| **virtualNetworkAddressPrefix** | Used | Ignored | Used only when creating a new VNet. | +| **enableNatGateway** | Used | Ignored | NAT Gateway is only created/attached when mode is `new`. | +| **existingVirtualNetworkResourceId** | Ignored | Used | Required in `existing` mode to target your VNet. | + +In `new` mode: + +- Set **virtualNetworkAddressPrefix** to define the VNet address space. +- FinOps hubs always creates the required private DNS zones and links them to the new virtual network. +- The virtual network can be any subnet size from **/8** to **/26**, with a minimum of **/26** (64 IP addresses) required. +- The default IP range is **10.20.30.0/26**. +- FinOps hubs creates three subnets from this range: **/28** for private endpoints, **/28** for deployment scripts, and **/27** reserved for Azure Data Explorer private endpoints. +- Choose a larger range (for example, **/24**) if you need more private address space for adjacent services like Power BI VNet Data Gateway. + +In `existing` mode: + +### Existing mode: bring your own virtual network subnet requirements and mapping -- The virtual network can be any subnet size from **/8** to **/26**, with a minimum of **/26** (64 IP addresses) required. The default is **/26** to conserve IP addresses while providing the minimum required subnet sizes for Container Services (used during deployments for running scripts) and Data Explorer. -- The IP range can be set at the time of deployment and defaults to **10.20.30.0/26**. Choose a larger subnet (like **/24** or smaller) if you need additional address space for services such as Power BI VNet Data Gateway. +If you're using an existing virtual network, configure these settings before deploying FinOps hubs: -If necessary, you can create the virtual network, subnets, and optionally peer it with your hub network before deploying FinOps hubs if you follow these requirements: +- Set **existingVirtualNetworkResourceId** to your VNet resource ID. +- Set **createPrivateDnsZones** to `true` to let FinOps hubs create the required private DNS zones, link them to your existing virtual network, and attach private endpoints to those zones. +- Set **createPrivateDnsZones** to `false` if your organization manages private DNS separately. In this case, FinOps hubs still creates private endpoints, but doesn't create private DNS zones, private DNS zone groups, or virtual network links. +- **virtualNetworkAddressPrefix** is ignored. +- FinOps hubs uses your existing VNet address space and the subnet names you provide. + +When **enablePublicAccess** is set to `false`, FinOps hubs uses the following requirements and subnet mapping: + +| Subnet parameter | Default subnet name | Minimum size | Required configuration | Contains | +| ----------------------------- | --------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| **privateEndpointSubnetName** | **private-endpoint-subnet** | **/28** | Service endpoint recommended: **Microsoft.Storage** | Azure Resource Manager private endpoints for hub storage (`blob`, `dfs`), Key Vault (`vault`), and deployment script storage (`blob`) | +| **scriptSubnetName** | **script-subnet** | **/28** | Subnet delegation required: **Microsoft.ContainerInstance/containerGroups**. Service endpoint recommended: **Microsoft.Storage**. Outbound routing required if default outbound is blocked. | Deployment script container groups (Azure Container Instances) | +| **dataExplorerSubnetName** | **dataExplorer-subnet** | **/27** | Outbound routing required if default outbound is blocked. | Azure Data Explorer cluster private endpoint (`cluster`) | + +Virtual network requirement: minimum **/26** address space. + +> [!NOTE] +> Data Factory managed private endpoints (for storage, Key Vault, and Data Explorer) are created in the Data Factory managed virtual network, not as Azure Resource Manager private endpoints in your bring-your-own virtual network subnets. + +> [!NOTE] +> Azure Data Explorer private endpoints commonly consume 8 private IPs in the subnet baseline (engine endpoint, data management endpoint, and transient storage endpoints). This baseline doesn't scale linearly with **dataExplorerSkuCapacity** (node count). Additional private IP usage is primarily tied to high-ingestion scenarios that scale transient storage endpoints. The **/27** minimum in bring-your-own virtual network mode is kept to provide operational headroom. + +### Bring your own virtual network NSG settings + +If your existing subnets have NSGs attached, configure rules that allow the same baseline traffic pattern FinOps hubs configures when it creates a new virtual network. + +| Direction | Priority | Rule | Source | Destination | Port/Protocol | Access | +| --------- | -------- | ----------------------------- | ----------------- | -------------- | ------------- | ------ | +| Inbound | 100 | AllowVnetInBound | VirtualNetwork | VirtualNetwork | Any / Any | Allow | +| Inbound | 200 | AllowAzureLoadBalancerInBound | AzureLoadBalancer | Any | Any / Any | Allow | +| Inbound | 4096 | DenyAllInBound | Any | Any | Any / Any | Deny | +| Outbound | 100 | AllowVnetOutBound | VirtualNetwork | VirtualNetwork | Any / Any | Allow | +| Outbound | 200 | AllowInternetOutBound | Any | Internet | Any / Any | Allow | +| Outbound | 4096 | DenyAllOutBound | Any | Any | Any / Any | Deny | + +Apply these rules to the NSGs used by the private endpoint, script, and Data Explorer subnets, or configure equivalent rules in your central firewall/segmentation architecture. + +> [!NOTE] +> FinOps hubs only creates/attaches NAT Gateway when it creates a new virtual network. In bring-your-own virtual network mode, configure outbound routing in your own network architecture. -- The virtual network should be a minimum of **/26** in size (64 IP addresses) but can be any size up to **/8** (16,777,216 IP addresses). -- The name should be `-vNet`. -- The virtual network must be divided into three subnets with the service delegations as specified: - - **private-endpoint-subnet** (**/28**) – no service delegations configured; hosts private endpoints for storage and key vault. - - **script-subnet** (**/28**) – delegated to container services for running scripts during deployment. - - **dataExplorer-subnet** (**/27**) – delegated to Azure Data Explorer. +You can override all subnet names at deployment time. FinOps hubs uses the configured names for both new and existing virtual network modes.
diff --git a/docs-mslearn/toolkit/hubs/template.md b/docs-mslearn/toolkit/hubs/template.md index 3073275b1..5b17ff63a 100644 --- a/docs-mslearn/toolkit/hubs/template.md +++ b/docs-mslearn/toolkit/hubs/template.md @@ -3,7 +3,7 @@ title: FinOps hub template description: Learn about what's included in the FinOps hub template including parameters, resources, and outputs. author: flanakin ms.author: micflan -ms.date: 06/03/2026 +ms.date: 08/12/2026 ms.topic: concept-article ms.service: finops ms.subservice: finops-toolkit @@ -102,7 +102,22 @@ Here are the parameters you can use to customize the deployment: | **enableAHBRecommendations** | Bool | Optional. Enable Azure Hybrid Benefit recommendations that flag VMs and SQL VMs without Azure Hybrid Benefit enabled. May generate noise if your organization does not have on-premises licenses. Requires enableRecommendations. | False | | **enableSpotRecommendations** | Bool | Optional. Enable non-Spot AKS cluster recommendations that flag AKS clusters with autoscaling but not using Spot VMs. May generate noise since Spot VMs are only appropriate for interruptible workloads. Requires enableRecommendations. | False | | **enablePublicAccess** | Bool | Optional. Disable public access to the data lake (storage firewall). | True | -| **virtualNetworkAddressPrefix** | String | Optional. IP Address range for the private virtual network used by FinOps hubs. Accepts any subnet size from `/8` to `/26` with a minimum of `/26` required. `/26` is recommended to avoid wasting IPs unless you need additional address space for services like Power BI VNet Data Gateway. Internally, the following subnets will be created: `/28` for private endpoints, another `/28` subnet for temporary deployment scripts (container instances), and `/27` for Azure Data Explorer, if enabled. | '10.20.30.0/26' | +| **privateEndpointSubnetName** | String | Optional. Subnet name for private endpoints used by storage, key vault, and Data Factory managed private endpoint routing when private routing is enabled. | 'private-endpoint-subnet' | +| **scriptSubnetName** | String | Optional. Subnet name used for deployment scripts when private routing is enabled. | 'script-subnet' | +| **dataExplorerSubnetName** | String | Optional. Subnet name used for Azure Data Explorer private endpoint routing when private routing is enabled. | 'dataExplorer-subnet' | +| **enableNatGateway** | Bool | Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, subnets disable Azure default outbound access and route through the NAT Gateway. Ignored when enablePublicAccess is true. | False | +| **createPrivateDnsZones** | Bool | Optional. Create private DNS zones for private endpoints when **virtualNetworkMode** is `existing`. In `new` mode, FinOps hubs always creates private DNS zones and links them to the created virtual network. In `existing` mode, set this value to `false` in enterprise environments where a centralized DNS resolver (such as Azure Private DNS Resolver or an on-premises DNS forwarder) manages private DNS registration. When `false`, private endpoints are still created but no private DNS zones, private DNS zone groups, or virtual network links are deployed. Ignored when enablePublicAccess is true. | True | +| **virtualNetworkMode** | String | Optional. Virtual network mode when private routing is enabled. Use `new` to create a virtual network and subnets. Use `existing` to bring your own virtual network. Ignored when enablePublicAccess is true. Allowed values: `new`, `existing`. | 'new' | +| **virtualNetworkAddressPrefix** | String | Optional. IP address range for the private virtual network used by FinOps hubs when private routing is enabled and **virtualNetworkMode** is `new`. Accepts any subnet size from `/8` to `/26` with a minimum of `/26` required. `/26` is recommended to avoid wasting IPs unless you need additional address space for services like Power BI VNet Data Gateway. In `new` mode, the deployment creates three subnets from this range: `/28` for private endpoints, `/28` for temporary deployment scripts (container instances), and `/27` reserved for Azure Data Explorer private endpoints. The Azure Data Explorer `/27` reservation is designed to provide headroom because private endpoint IP usage can increase for high-ingestion workloads and isn't linearly tied to cluster node count. Ignored when **virtualNetworkMode** is `existing` or **enablePublicAccess** is true. | '10.20.30.0/26' | +| **existingVirtualNetworkResourceId** | String | Optional. Resource ID of an existing virtual network to use when private routing is enabled and virtualNetworkMode is `existing`. Example: `/subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks/`. | '' | + +When **enablePublicAccess** is `false`, subnet usage is: + +- **privateEndpointSubnetName**: Azure Resource Manager private endpoints for hub storage (`blob`, `dfs`), Key Vault (`vault`), and deployment script storage (`blob`). +- **scriptSubnetName**: deployment script container groups. +- **dataExplorerSubnetName**: Azure Data Explorer cluster private endpoint (`cluster`). + +Data Factory managed private endpoints are created in the Data Factory managed virtual network, not in these BYO subnet definitions.
diff --git a/docs/deploy/finops-hub-latest.json b/docs/deploy/finops-hub-latest.json index 5d526c052..62c6e212a 100644 --- a/docs/deploy/finops-hub-latest.json +++ b/docs/deploy/finops-hub-latest.json @@ -2228,7 +2228,7 @@ ] }, "scriptEndpoint::scriptPrivateDnsZoneGroup": { - "condition": "[parameters('hub').options.privateRouting]", + "condition": "[and(parameters('hub').options.privateRouting, parameters('hub').options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('hub').routing.scriptStorage), 'blob-endpoint-zone')]", @@ -3034,7 +3034,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -3053,7 +3053,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -3089,7 +3089,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -9682,7 +9682,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -9701,7 +9701,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -9737,7 +9737,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -16158,7 +16158,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -16177,7 +16177,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -16213,7 +16213,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -18615,7 +18615,7 @@ ] }, "dataExplorerPrivateDnsZoneGroup": { - "condition": "[and(variables('useAzure'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('useAzure'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', replace(parameters('clusterName'), '_', '-')), 'dataExplorer-endpoint-zone')]", @@ -20452,7 +20452,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -20471,7 +20471,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -20507,7 +20507,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -25165,7 +25165,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -25184,7 +25184,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -25220,7 +25220,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -27650,7 +27650,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -27669,7 +27669,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -27705,7 +27705,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -29899,7 +29899,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -29918,7 +29918,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -29954,7 +29954,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", @@ -34480,7 +34480,7 @@ ] }, "blobEndpoint::blobPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-blob-ep', parameters('app').storage), 'storage-endpoint-zone')]", @@ -34499,7 +34499,7 @@ ] }, "dfsEndpoint::dfsPrivateDnsZoneGroup": { - "condition": "[and(variables('usesStorage'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesStorage'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-dfs-ep', parameters('app').storage), 'dfs-endpoint-zone')]", @@ -34535,7 +34535,7 @@ ] }, "keyVaultEndpoint::keyVaultPrivateDnsZoneGroup": { - "condition": "[and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting)]", + "condition": "[and(and(variables('usesKeyVault'), parameters('app').hub.options.privateRouting), parameters('app').hub.options.createPrivateDnsZones)]", "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups", "apiVersion": "2023-11-01", "name": "[format('{0}/{1}', format('{0}-ep', parameters('app').keyVault), 'keyvault-endpoint-zone')]", diff --git a/docs/hubs.md b/docs/hubs.md index c3ff69557..fcc7fc8c8 100644 --- a/docs/hubs.md +++ b/docs/hubs.md @@ -116,7 +116,7 @@ Create a new or update an existing FinOps hub instance. Most secure. Resources are only reachable from peered networks. Access is controlled via RBAC.

- Work with your network admin to configure peering and routing so the FinOps hubs isolated network is reachable from your network. + Work with your network admin to configure peering and routing so the FinOps hubs isolated network is reachable from your network. You can create a new virtual network during deployment or bring your own existing virtual network and subnet names. If using an existing virtual network, review the required subnet settings before deployment.

Plan for private networking diff --git a/src/templates/finops-hub/createUiDefinition.json b/src/templates/finops-hub/createUiDefinition.json index d7ea79097..90758adfb 100644 --- a/src/templates/finops-hub/createUiDefinition.json +++ b/src/templates/finops-hub/createUiDefinition.json @@ -936,16 +936,46 @@ } }, { - "name": "virtualNetworkAddressPrefix", + "name": "privateEndpointSubnetName", "type": "Microsoft.Common.TextBox", - "label": "Address prefix", - "toolTip": "Address space for the workload. Minimum /26 subnet size is required for the workload.", - "defaultValue": "10.20.30.0/26", + "label": "Private endpoint subnet name", + "toolTip": "Subnet name for storage, key vault, and Data Factory managed private endpoint routing.", + "defaultValue": "private-endpoint-subnet", + "visible": "[equals(steps('advanced').networking.enablePublicAccess, false)]" + }, + { + "name": "scriptSubnetName", + "type": "Microsoft.Common.TextBox", + "label": "Script subnet name", + "toolTip": "Subnet name used by deployment scripts.", + "defaultValue": "script-subnet", + "visible": "[equals(steps('advanced').networking.enablePublicAccess, false)]" + }, + { + "name": "dataExplorerSubnetName", + "type": "Microsoft.Common.TextBox", + "label": "Data Explorer subnet name", + "toolTip": "Subnet name used for Azure Data Explorer private endpoints.", + "defaultValue": "dataExplorer-subnet", + "visible": "[equals(steps('advanced').networking.enablePublicAccess, false)]" + }, + { + "name": "virtualNetworkMode", + "type": "Microsoft.Common.DropDown", + "label": "Virtual network mode", + "toolTip": "Choose whether FinOps hubs should create a new virtual network and subnets or use an existing virtual network.", + "defaultValue": "New", "constraints": { - "validations": [ + "allowedValues": [ { - "regex": "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\/([8-9]|1[0-9]|2[0-6])$", - "message": "Address prefix must be a valid IPv4 address in the format 'XXX.XXX.XXX.XXX/YY' where YY is between 8 and 26 (minimum /26 subnet size required)." + "label": "New", + "description": "Create a new virtual network and subnets.", + "value": "new" + }, + { + "label": "Existing", + "description": "Use an existing virtual network and existing subnets.", + "value": "existing" } ] }, @@ -957,7 +987,47 @@ "label": "Deploy NAT Gateway and disable default outbound access", "toolTip": "Required by the September 2025 'Subnets should be private' policy. Adds a Standard NAT Gateway and Standard static Public IP attached to the script and Data Explorer subnets to provide controlled outbound internet access. Leave unchecked if you have your own egress path (UDR to firewall, peered hub with NAT, etc.).", "defaultValue": false, - "visible": "[equals(steps('advanced').networking.enablePublicAccess, false)]" + "visible": "[and(equals(steps('advanced').networking.enablePublicAccess, false), equals(steps('advanced').networking.virtualNetworkMode, 'new'))]" + }, + { + "name": "createPrivateDnsZones", + "type": "Microsoft.Common.CheckBox", + "label": "Create private DNS zones", + "toolTip": "Used only when virtual network mode is Existing. Uncheck in enterprise environments where a centralized DNS resolver (such as Azure Private DNS Resolver or an on-premises forwarder) manages private DNS registration. When unchecked, private endpoints are still created but no private DNS zones, zone groups, or virtual network links are deployed.", + "defaultValue": true, + "visible": "[and(equals(steps('advanced').networking.enablePublicAccess, false), equals(steps('advanced').networking.virtualNetworkMode, 'existing'))]" + }, + { + "name": "existingVirtualNetworkResourceId", + "type": "Microsoft.Common.TextBox", + "label": "Existing virtual network resource ID", + "toolTip": "Resource ID of the existing virtual network to use when virtual network mode is Existing.", + "defaultValue": "", + "constraints": { + "validations": [ + { + "isValid": "[or(not(equals(steps('advanced').networking.virtualNetworkMode, 'existing')), and(startsWith(steps('advanced').networking.existingVirtualNetworkResourceId, '/subscriptions/'), contains(steps('advanced').networking.existingVirtualNetworkResourceId, '/providers/Microsoft.Network/virtualNetworks/')))]", + "message": "Enter a valid virtual network resource ID in the format /subscriptions/.../providers/Microsoft.Network/virtualNetworks/." + } + ] + }, + "visible": "[and(equals(steps('advanced').networking.enablePublicAccess, false), equals(steps('advanced').networking.virtualNetworkMode, 'existing'))]" + }, + { + "name": "virtualNetworkAddressPrefix", + "type": "Microsoft.Common.TextBox", + "label": "Address prefix", + "toolTip": "Address space for the workload. Minimum /26 subnet size is required.", + "defaultValue": "10.20.30.0/26", + "constraints": { + "validations": [ + { + "regex": "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\/([8-9]|1[0-9]|2[0-6])$", + "message": "Address prefix must be a valid IPv4 address in the format 'XXX.XXX.XXX.XXX/YY' where YY is between 8 and 26 (minimum /26 subnet size required)." + } + ] + }, + "visible": "[and(equals(steps('advanced').networking.enablePublicAccess, false), equals(steps('advanced').networking.virtualNetworkMode, 'new'))]" } ], "visible": true @@ -1003,8 +1073,14 @@ "enableAHBRecommendations": "[steps('recommendations').optional.enableAHBRecommendations]", "enableSpotRecommendations": "[steps('recommendations').optional.enableSpotRecommendations]", "enablePublicAccess": "[steps('advanced').networking.enablePublicAccess]", + "privateEndpointSubnetName": "[steps('advanced').networking.privateEndpointSubnetName]", + "scriptSubnetName": "[steps('advanced').networking.scriptSubnetName]", + "dataExplorerSubnetName": "[steps('advanced').networking.dataExplorerSubnetName]", "enableNatGateway": "[steps('advanced').networking.enableNatGateway]", + "createPrivateDnsZones": "[steps('advanced').networking.createPrivateDnsZones]", + "virtualNetworkMode": "[steps('advanced').networking.virtualNetworkMode]", "virtualNetworkAddressPrefix": "[steps('advanced').networking.virtualNetworkAddressPrefix]", + "existingVirtualNetworkResourceId": "[steps('advanced').networking.existingVirtualNetworkResourceId]", "dataExplorerSku": "[steps('pricing').dataExplorer.dataExplorerSku]", "exportRetentionInDays": "[steps('retention').storage.msexportsDays]", "ingestionRetentionInMonths": "[steps('retention').storage.ingestionMonths]", diff --git a/src/templates/finops-hub/main.bicep b/src/templates/finops-hub/main.bicep index 687c26174..e8b9dea2a 100644 --- a/src/templates/finops-hub/main.bicep +++ b/src/templates/finops-hub/main.bicep @@ -159,12 +159,34 @@ param dataExplorerFinalRetentionInMonths int = 13 @description('Optional. Enable public access to FinOps hubs resources. Default: true.') param enablePublicAccess bool = true +@description('Optional. Name of the subnet used for private endpoints (Storage, Key Vault, and Data Factory managed private endpoint routing). Used when private routing is enabled. Default: "private-endpoint-subnet".') +param privateEndpointSubnetName string = 'private-endpoint-subnet' + +@description('Optional. Name of the subnet used for deployment scripts. Used when private routing is enabled. Default: "script-subnet".') +param scriptSubnetName string = 'script-subnet' + +@description('Optional. Name of the subnet used for Azure Data Explorer private endpoints. Used when private routing is enabled. Default: "dataExplorer-subnet".') +param dataExplorerSubnetName string = 'dataExplorer-subnet' + @description('Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, subnets disable Azure default outbound access and route through the NAT Gateway. Ignored when enablePublicAccess is true. Default: false.') param enableNatGateway bool = false -@description('Optional. Address space for the workload. Minimum /26 subnet size is required for the workload. Default: "10.20.30.0/26".') +@description('Optional. Create private DNS zones for private endpoints when virtualNetworkMode is "existing". Always enabled when virtualNetworkMode is "new". Set to false in enterprise environments where a centralized DNS resolver manages private DNS registration. Ignored when enablePublicAccess is true. Default: true.') +param createPrivateDnsZones bool = true + +@description('Optional. Virtual network mode to use when private routing is enabled. Use "new" to create a virtual network and subnets. Use "existing" to bring your own virtual network. Ignored when enablePublicAccess is true. Allowed: "new", "existing". Default: "new".') +@allowed([ + 'new' + 'existing' +]) +param virtualNetworkMode string = 'new' + +@description('Optional. Address space for the workload when private routing is enabled and virtualNetworkMode is "new". Minimum /26 subnet size is required. Ignored when virtualNetworkMode is "existing" or enablePublicAccess is true. Default: "10.20.30.0/26".') param virtualNetworkAddressPrefix string = '10.20.30.0/26' +@description('Optional. Resource ID of an existing virtual network to use when private routing is enabled and virtualNetworkMode is "existing". Ignored otherwise. Example: /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks/. Default: "".') +param existingVirtualNetworkResourceId string = '' + //============================================================================== // Resources @@ -198,8 +220,14 @@ module hub 'modules/hub.bicep' = { remoteHubStorageUri: remoteHubStorageUri remoteHubStorageKey: remoteHubStorageKey enablePublicAccess: enablePublicAccess + privateEndpointSubnetName: privateEndpointSubnetName + scriptSubnetName: scriptSubnetName + dataExplorerSubnetName: dataExplorerSubnetName enableNatGateway: enableNatGateway + createPrivateDnsZones: createPrivateDnsZones + virtualNetworkMode: virtualNetworkMode virtualNetworkAddressPrefix: virtualNetworkAddressPrefix + existingVirtualNetworkResourceId: existingVirtualNetworkResourceId } } diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep index 84a29d95c..f3d9ba0c9 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep @@ -256,21 +256,21 @@ resource dataFactory 'Microsoft.DataFactory/factories@2018-06-01' existing = { ] } -resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { +resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (useAzure && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'privatelink.blob.${environment().suffixes.storage}' dependsOn: [ appRegistration ] } -resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { +resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (useAzure && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'privatelink.queue.${environment().suffixes.storage}' dependsOn: [ appRegistration ] } -resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = { +resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (useAzure && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'privatelink.table.${environment().suffixes.storage}' dependsOn: [ appRegistration @@ -459,7 +459,7 @@ resource clusterStorageAccess 'Microsoft.Authorization/roleAssignments@2022-04-0 } // DNS zone -resource dataExplorerPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (useAzure && app.hub.options.privateRouting) { +resource dataExplorerPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (useAzure && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: dataExplorerPrivateDnsZoneName location: 'global' tags: union(app.tags, app.hub.tagsByResource[?'Microsoft.Network/privateDnsZones'] ?? {}) @@ -467,7 +467,7 @@ resource dataExplorerPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-0 } // Link DNS zone to VNet -resource dataExplorerPrivateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2024-06-01' = if (useAzure && app.hub.options.privateRouting) { +resource dataExplorerPrivateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2024-06-01' = if (useAzure && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: '${replace(dataExplorerPrivateDnsZone.name, '.', '-')}-link' location: 'global' parent: dataExplorerPrivateDnsZone @@ -502,7 +502,7 @@ resource dataExplorerEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = } // DNS records for private endpoint -resource dataExplorerPrivateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (useAzure && app.hub.options.privateRouting) { +resource dataExplorerPrivateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (useAzure && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'dataExplorer-endpoint-zone' parent: dataExplorerEndpoint properties: { diff --git a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep index 85b5e9591..551a44b16 100644 --- a/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep +++ b/src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Core/infrastructure.bicep @@ -20,12 +20,14 @@ var nsgName = '${hub.routing.networkName}-nsg' var natGatewayName = '${hub.routing.networkName}-natgw' var natGatewayPipName = '${hub.routing.networkName}-natgw-pip' +var createNetwork = hub.options.privateRouting && hub.options.networkMode == 'new' + // Workaround https://github.com/Azure/bicep/issues/1853 -var finopsHubSubnetName = 'private-endpoint-subnet' -var scriptSubnetName = 'script-subnet' -var dataExplorerSubnetName = 'dataExplorer-subnet' +var finopsHubSubnetName = hub.routing.subnetNames.storage +var scriptSubnetName = hub.routing.subnetNames.scripts +var dataExplorerSubnetName = hub.routing.subnetNames.dataExplorer -var subnets = !hub.options.privateRouting ? [] : [ +var subnets = !createNetwork ? [] : [ { name: finopsHubSubnetName properties: { @@ -95,7 +97,7 @@ var subnets = !hub.options.privateRouting ? [] : [ // Network //------------------------------------------------------------------------------ -resource nsg 'Microsoft.Network/networkSecurityGroups@2023-11-01' = if (hub.options.privateRouting) { +resource nsg 'Microsoft.Network/networkSecurityGroups@2023-11-01' = if (createNetwork) { name: nsgName location: hub.location tags: getHubTags(hub, 'Microsoft.Storage/networkSecurityGroups') @@ -183,7 +185,7 @@ resource nsg 'Microsoft.Network/networkSecurityGroups@2023-11-01' = if (hub.opti } } -resource vNet 'Microsoft.Network/virtualNetworks@2023-11-01' = if (hub.options.privateRouting) { +resource vNet 'Microsoft.Network/virtualNetworks@2023-11-01' = if (createNetwork) { name: hub.routing.networkName location: hub.location tags: getHubTags(hub, 'Microsoft.Network/virtualNetworks') @@ -216,7 +218,7 @@ resource vNet 'Microsoft.Network/virtualNetworks@2023-11-01' = if (hub.options.p // policy and the September 2025 implicit-outbound retirement) //------------------------------------------------------------------------------ -resource natGatewayPublicIp 'Microsoft.Network/publicIPAddresses@2023-11-01' = if (hub.options.natGateway) { +resource natGatewayPublicIp 'Microsoft.Network/publicIPAddresses@2023-11-01' = if (createNetwork && hub.options.natGateway) { name: natGatewayPipName location: hub.location tags: getHubTags(hub, 'Microsoft.Network/publicIPAddresses') @@ -229,7 +231,7 @@ resource natGatewayPublicIp 'Microsoft.Network/publicIPAddresses@2023-11-01' = i } } -resource natGateway 'Microsoft.Network/natGateways@2023-11-01' = if (hub.options.natGateway) { +resource natGateway 'Microsoft.Network/natGateways@2023-11-01' = if (createNetwork && hub.options.natGateway) { name: natGatewayName location: hub.location tags: getHubTags(hub, 'Microsoft.Network/natGateways') @@ -251,11 +253,11 @@ resource natGateway 'Microsoft.Network/natGateways@2023-11-01' = if (hub.options //------------------------------------------------------------------------------ // Required for the Azure portal and Storage Explorer -resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting) { +resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting && hub.options.createPrivateDnsZones) { name: string(hub.routing.dnsZones.blob.name) - dependsOn: [ + dependsOn: createNetwork ? [ vNet - ] + ] : [] location: 'global' tags: getHubTags(hub, 'Microsoft.Storage/privateDnsZones') properties: {} @@ -274,11 +276,11 @@ resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if } // Required for Power BI -resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting) { +resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting && hub.options.createPrivateDnsZones) { name: string(hub.routing.dnsZones.dfs.name) - dependsOn: [ + dependsOn: createNetwork ? [ vNet - ] + ] : [] location: 'global' tags: getHubTags(hub, 'Microsoft.Storage/privateDnsZones') properties: {} @@ -297,11 +299,11 @@ resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if ( } // Required for Azure Data Explorer -resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting) { +resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting && hub.options.createPrivateDnsZones) { name: string(hub.routing.dnsZones.queue.name) - dependsOn: [ + dependsOn: createNetwork ? [ vNet - ] + ] : [] location: 'global' tags: getHubTags(hub, 'Microsoft.Storage/privateDnsZones') properties: {} @@ -320,11 +322,11 @@ resource queuePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if } // Required for Azure Data Explorer -resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting) { +resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (hub.options.privateRouting && hub.options.createPrivateDnsZones) { name: string(hub.routing.dnsZones.table.name) - dependsOn: [ + dependsOn: createNetwork ? [ vNet - ] + ] : [] location: 'global' tags: getHubTags(hub, 'Microsoft.Storage/privateDnsZones') properties: {} @@ -348,9 +350,9 @@ resource tablePrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if resource scriptStorageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = if (hub.options.privateRouting) { name: hub.routing.scriptStorage - dependsOn: [ + dependsOn: createNetwork ? [ vNet::scriptSubnet - ] + ] : [] location: hub.location sku: { name: 'Standard_LRS' @@ -379,9 +381,9 @@ resource scriptStorageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = i resource scriptEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (hub.options.privateRouting) { name: '${scriptStorageAccount.name}-blob-ep' - dependsOn: [ + dependsOn: createNetwork ? [ vNet::scriptSubnet - ] + ] : [] location: hub.location tags: getHubTags(hub, 'Microsoft.Network/privateEndpoints') properties: { @@ -399,7 +401,7 @@ resource scriptEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (hu ] } - resource scriptPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + resource scriptPrivateDnsZoneGroup 'privateDnsZoneGroups' = if (hub.options.privateRouting && hub.options.createPrivateDnsZones) { name: 'blob-endpoint-zone' properties: { privateDnsZoneConfigs: [ @@ -423,21 +425,21 @@ resource scriptEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (hu output config HubProperties = hub @description('Resource ID of the virtual network.') -output vNetId string = !hub.options.privateRouting ? '' : vNet.id +output vNetId string = !hub.options.privateRouting ? '' : hub.routing.networkId @description('Virtual network address prefixes.') #disable-next-line BCP318 // Null safety warning for conditional resource access -output vNetAddressSpace array = !hub.options.privateRouting ? [] : vNet.properties.addressSpace.addressPrefixes +output vNetAddressSpace array = !hub.options.privateRouting || !createNetwork ? [] : vNet.properties.addressSpace.addressPrefixes @description('Virtual network subnets.') #disable-next-line BCP318 // Null safety warning for conditional resource access -output vNetSubnets array = !hub.options.privateRouting ? [] : vNet.properties.subnets +output vNetSubnets array = !hub.options.privateRouting || !createNetwork ? [] : vNet.properties.subnets @description('Resource ID of the FinOps hub network subnet.') -output finopsHubSubnetId string = !hub.options.privateRouting ? '' : vNet::finopsHubSubnet.id +output finopsHubSubnetId string = !hub.options.privateRouting ? '' : hub.routing.subnets.storage @description('Resource ID of the script storage account network subnet.') -output scriptSubnetId string = !hub.options.privateRouting ? '' : vNet::scriptSubnet.id +output scriptSubnetId string = !hub.options.privateRouting ? '' : hub.routing.subnets.scripts @description('Resource ID of the Data Explorer network subnet.') -output dataExplorerSubnetId string = !hub.options.privateRouting ? '' : vNet::dataExplorerSubnet.id +output dataExplorerSubnetId string = !hub.options.privateRouting ? '' : hub.routing.subnets.dataExplorer diff --git a/src/templates/finops-hub/modules/fx/hub-app.bicep b/src/templates/finops-hub/modules/fx/hub-app.bicep index 6bdd0cf21..210b8bbc8 100644 --- a/src/templates/finops-hub/modules/fx/hub-app.bicep +++ b/src/templates/finops-hub/modules/fx/hub-app.bicep @@ -386,7 +386,7 @@ resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' = if (use } } -resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && app.hub.options.privateRouting) { +resource blobPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'privatelink.blob.${environment().suffixes.storage}' // cSpell:ignore privatelink } @@ -410,7 +410,7 @@ resource blobEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (uses ] } - resource blobPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + resource blobPrivateDnsZoneGroup 'privateDnsZoneGroups' = if (app.hub.options.createPrivateDnsZones) { name: 'storage-endpoint-zone' properties: { privateDnsZoneConfigs: [ @@ -425,7 +425,7 @@ resource blobEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (uses } } -resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && app.hub.options.privateRouting) { +resource dfsPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' existing = if (usesStorage && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'privatelink.dfs.${environment().suffixes.storage}' // cSpell:ignore privatelink } @@ -449,7 +449,7 @@ resource dfsEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if (usesS ] } - resource dfsPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + resource dfsPrivateDnsZoneGroup 'privateDnsZoneGroups' = if (app.hub.options.createPrivateDnsZones) { name: 'dfs-endpoint-zone' properties: { privateDnsZoneConfigs: [ @@ -495,7 +495,7 @@ resource keyVault 'Microsoft.KeyVault/vaults@2023-02-01' = if (usesKeyVault) { } } -resource keyVaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (usesKeyVault && app.hub.options.privateRouting) { +resource keyVaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2024-06-01' = if (usesKeyVault && app.hub.options.privateRouting && app.hub.options.createPrivateDnsZones) { name: 'privatelink${replace(environment().suffixes.keyvaultDns, 'vault', 'vaultcore')}' // cSpell:ignore privatelink, vaultcore location: 'global' tags: getAppPublisherTags(app, 'Microsoft.Network/privateDnsZones') @@ -534,7 +534,7 @@ resource keyVaultEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = if ( ] } - resource keyVaultPrivateDnsZoneGroup 'privateDnsZoneGroups' = { + resource keyVaultPrivateDnsZoneGroup 'privateDnsZoneGroups' = if (app.hub.options.createPrivateDnsZones) { name: 'keyvault-endpoint-zone' properties: { privateDnsZoneConfigs: [ diff --git a/src/templates/finops-hub/modules/fx/hub-types.bicep b/src/templates/finops-hub/modules/fx/hub-types.bicep index d9b197e75..c5ecb132c 100644 --- a/src/templates/finops-hub/modules/fx/hub-types.bicep +++ b/src/templates/finops-hub/modules/fx/hub-types.bicep @@ -31,6 +31,13 @@ type IdNameObject = { id: string, name: string } queue: 'Resource ID and name for the queue storage DNS zone.' table: 'Resource ID and name for the table storage DNS zone.' } + subnetNames: { + dataExplorer: 'Name of the subnet for the Data Explorer instance.' + dataFactory: 'Name of the subnet for Data Factory instances.' + keyVault: 'Name of the subnet for Key Vault instances.' + scripts: 'Name of the subnet for deployment script storage.' + storage: 'Name of the subnet for storage accounts.' + } subnets: { dataExplorer: 'Resource ID of the subnet for the Data Explorer instance.' dataFactory: 'Resource ID of the subnet for Data Factory instances.' @@ -49,6 +56,13 @@ type HubRoutingProperties = { queue: IdNameObject table: IdNameObject } + subnetNames: { + dataExplorer: string + dataFactory: string + keyVault: string + scripts: string + storage: string + } subnets: { dataExplorer: string dataFactory: string @@ -71,7 +85,8 @@ type HubRoutingProperties = { enableTelemetry: 'Indicates whether telemetry should be enabled for deployments.' keyVaultSku: 'KeyVault SKU. Allowed values: "standard", "premium".' keyVaultEnablePurgeProtection: 'Indicates whether purge protection is enabled for the Key Vault. When enabled, deleted Key Vault and its secrets cannot be permanently deleted until the retention period expires, which is required for compliance in some environments.' - networkAddressPrefix: 'Address prefix for the FinOps hub isolated virtual network, if private network routing is enabled.' + networkMode: 'Indicates whether private routing creates a new virtual network ("new") or uses an existing virtual network ("existing").' + networkAddressPrefix: 'Address prefix for the FinOps hub isolated virtual network when private network routing is enabled and networkMode is "new".' natGateway: 'Indicates whether a NAT Gateway should be deployed for controlled outbound internet access. When enabled, subnets disable Azure default outbound access and route through the NAT Gateway.' privateRouting: 'Indicates whether private network routing is enabled.' publisherIsolation: 'Indicates whether FinOps hub resources should be separated by publisher for advanced security.' @@ -96,9 +111,11 @@ type HubProperties = { enableTelemetry: bool keyVaultSku: string keyVaultEnablePurgeProtection: bool + networkMode: string networkAddressPrefix: string natGateway: bool privateRouting: bool + createPrivateDnsZones: bool publisherIsolation: bool storageInfrastructureEncryption: bool storageSku: string @@ -177,6 +194,8 @@ func idName(name string, resourceType string) IdNameObject => { // cSpell:ignore privatelink func dnsZoneIdName(type string) IdNameObject => idName('privatelink.${type}.${environment().suffixes.storage}', 'Microsoft.Network/privateDnsZones') +func resolvedNetworkId(enablePublicAccess bool, virtualNetworkMode string, existingVirtualNetworkResourceId string, networkName string) string => enablePublicAccess ? '' : (virtualNetworkMode == 'existing' ? existingVirtualNetworkResourceId : resourceId('Microsoft.Network/virtualNetworks', networkName)) + //------------------------------------------------------------------------------ // Hub config //------------------------------------------------------------------------------ @@ -194,16 +213,22 @@ func newHubInternal( keyVaultEnablePurgeProtection bool, enableInfrastructureEncryption bool, enablePublicAccess bool, + privateEndpointSubnetName string, + scriptSubnetName string, + dataExplorerSubnetName string, enableNatGateway bool, + createPrivateDnsZones bool, + virtualNetworkMode string, networkName string, networkAddressPrefix string, - isTelemetryEnabled bool, + existingVirtualNetworkResourceId string, + isTelemetryEnabled bool ) HubProperties => { id: id name: name location: location ?? resourceGroup().location tags: union(tags, { - 'cm-resource-parent': id // cm-resource-parent tag groups resources in Cost Management + 'cm-resource-parent': id // cm-resource-parent tag groups resources in Cost Management 'ftk-tool': 'FinOps hubs' 'ftk-version': finOpsToolkitVersion }) @@ -213,16 +238,18 @@ func newHubInternal( enableTelemetry: isTelemetryEnabled ?? true keyVaultSku: keyVaultSku keyVaultEnablePurgeProtection: keyVaultEnablePurgeProtection + networkMode: enablePublicAccess ? 'new' : virtualNetworkMode networkAddressPrefix: networkAddressPrefix - natGateway: !enablePublicAccess && enableNatGateway + natGateway: !enablePublicAccess && enableNatGateway && virtualNetworkMode == 'new' privateRouting: !enablePublicAccess + createPrivateDnsZones: !enablePublicAccess && (virtualNetworkMode == 'new' || createPrivateDnsZones) publisherIsolation: false // TODO: Expose publisher isolation option storageInfrastructureEncryption: enableInfrastructureEncryption storageSku: storageSku } routing: { - networkId: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks', networkName) - networkName: enablePublicAccess ? '' : networkName + networkId: resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName) + networkName: enablePublicAccess ? '' : last(split(resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName), '/')) scriptStorage: enablePublicAccess ? '' : '${take(safeStorageName(name), 16 - length(suffix))}script${suffix}' dnsZones: { blob: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('blob') @@ -230,12 +257,19 @@ func newHubInternal( queue: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('queue') table: enablePublicAccess ? { id:'', name:'' } : dnsZoneIdName('table') } + subnetNames: { + dataExplorer: enablePublicAccess ? '' : dataExplorerSubnetName + dataFactory: enablePublicAccess ? '' : privateEndpointSubnetName + keyVault: enablePublicAccess ? '' : privateEndpointSubnetName + scripts: enablePublicAccess ? '' : scriptSubnetName + storage: enablePublicAccess ? '' : privateEndpointSubnetName + } subnets: { - dataExplorer: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'dataExplorer-subnet')! - dataFactory: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! - keyVault: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! - scripts: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'script-subnet')! - storage: enablePublicAccess ? '' : resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, 'private-endpoint-subnet')! + dataExplorer: enablePublicAccess ? '' : '${resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName)}/subnets/${dataExplorerSubnetName}' + dataFactory: enablePublicAccess ? '' : '${resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName)}/subnets/${privateEndpointSubnetName}' + keyVault: enablePublicAccess ? '' : '${resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName)}/subnets/${privateEndpointSubnetName}' + scripts: enablePublicAccess ? '' : '${resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName)}/subnets/${scriptSubnetName}' + storage: enablePublicAccess ? '' : '${resolvedNetworkId(enablePublicAccess, virtualNetworkMode, existingVirtualNetworkResourceId, networkName)}/subnets/${privateEndpointSubnetName}' } } core: { @@ -255,26 +289,42 @@ func newHub( keyVaultEnablePurgeProtection bool, enableInfrastructureEncryption bool, enablePublicAccess bool, + privateEndpointSubnetName string, + scriptSubnetName string, + dataExplorerSubnetName string, enableNatGateway bool, + createPrivateDnsZones bool, + virtualNetworkMode string, networkAddressPrefix string, - isTelemetryEnabled bool, -) HubProperties => newHubInternal( - '${resourceGroup().id}/providers/Microsoft.Cloud/hubs/${name}', // id - name, - uniqueString(name, resourceGroup().id), // suffix - location, - tags, - tagsByResource, - storageSku, - keyVaultSku, - keyVaultEnablePurgeProtection, - enableInfrastructureEncryption, - enablePublicAccess, - enableNatGateway, - '${safeStorageName(name)}-vnet-${location}', // networkName, cSpell:ignore vnet - networkAddressPrefix, - isTelemetryEnabled ?? true -) + existingVirtualNetworkResourceId string, + isTelemetryEnabled bool +) HubProperties => + newHubInternal( + '${resourceGroup().id}/providers/Microsoft.Cloud/hubs/${name}', + // id + name, + uniqueString(name, resourceGroup().id), + // suffix + location, + tags, + tagsByResource, + storageSku, + keyVaultSku, + keyVaultEnablePurgeProtection, + enableInfrastructureEncryption, + enablePublicAccess, + privateEndpointSubnetName, + scriptSubnetName, + dataExplorerSubnetName, + enableNatGateway, + createPrivateDnsZones, + virtualNetworkMode, + '${safeStorageName(name)}-vnet-${location}', + // networkName, cSpell:ignore vnet + networkAddressPrefix, + existingVirtualNetworkResourceId, + isTelemetryEnabled ?? true + ) //------------------------------------------------------------------------------ // App config diff --git a/src/templates/finops-hub/modules/hub.bicep b/src/templates/finops-hub/modules/hub.bicep index 0c9e6b9e5..5824f3373 100644 --- a/src/templates/finops-hub/modules/hub.bicep +++ b/src/templates/finops-hub/modules/hub.bicep @@ -170,12 +170,34 @@ param dataExplorerFinalRetentionInMonths int = 13 @description('Optional. Enable public access to the data lake. Default: true.') param enablePublicAccess bool = true +@description('Optional. Name of the subnet used for private endpoints (Storage, Key Vault, and Data Factory managed private endpoint routing). Used when private routing is enabled. Default: "private-endpoint-subnet".') +param privateEndpointSubnetName string = 'private-endpoint-subnet' + +@description('Optional. Name of the subnet used for deployment scripts. Used when private routing is enabled. Default: "script-subnet".') +param scriptSubnetName string = 'script-subnet' + +@description('Optional. Name of the subnet used for Azure Data Explorer private endpoints. Used when private routing is enabled. Default: "dataExplorer-subnet".') +param dataExplorerSubnetName string = 'dataExplorer-subnet' + @description('Optional. Deploy a NAT Gateway for controlled outbound access when private routing is enabled. When true, subnets disable Azure default outbound access and route through the NAT Gateway. Ignored when enablePublicAccess is true. Default: false.') param enableNatGateway bool = false -@description('Optional. Address space for the workload. Minimum /26 subnet size is required for the workload. Default: "10.20.30.0/26".') +@description('Optional. Create private DNS zones for private endpoints when virtualNetworkMode is "existing". Always enabled when virtualNetworkMode is "new". Set to false in enterprise environments where a centralized DNS resolver manages private DNS registration. Ignored when enablePublicAccess is true. Default: true.') +param createPrivateDnsZones bool = true + +@description('Optional. Virtual network mode to use when private routing is enabled. Use "new" to create a virtual network and subnets. Use "existing" to bring your own virtual network. Ignored when enablePublicAccess is true. Allowed: "new", "existing". Default: "new".') +@allowed([ + 'new' + 'existing' +]) +param virtualNetworkMode string = 'new' + +@description('Optional. Address space for the workload when private routing is enabled and virtualNetworkMode is "new". Minimum /26 subnet size is required. Ignored when virtualNetworkMode is "existing" or enablePublicAccess is true. Default: "10.20.30.0/26".') param virtualNetworkAddressPrefix string = '10.20.30.0/26' +@description('Optional. Resource ID of an existing virtual network to use when private routing is enabled and virtualNetworkMode is "existing". Ignored otherwise. Example: /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks/. Default: "".') +param existingVirtualNetworkResourceId string = '' + @description('Optional. Enable telemetry to track anonymous module usage trends, monitor for bugs, and improve future releases.') param enableDefaultTelemetry bool = true @@ -197,8 +219,14 @@ var hub = newHub( enablePurgeProtection, enableInfrastructureEncryption, enablePublicAccess, + privateEndpointSubnetName, + scriptSubnetName, + dataExplorerSubnetName, enableNatGateway, + createPrivateDnsZones, + virtualNetworkMode, virtualNetworkAddressPrefix, + existingVirtualNetworkResourceId, enableDefaultTelemetry )