Skip to content

[Dataproc] Add attached_disk_config and missing disk_config fields to google_dataproc_workflow_template - #18945

Open
realdev12 wants to merge 4 commits into
GoogleCloudPlatform:mainfrom
realdev12:dataproc-workflow-template-disk-config
Open

[Dataproc] Add attached_disk_config and missing disk_config fields to google_dataproc_workflow_template#18945
realdev12 wants to merge 4 commits into
GoogleCloudPlatform:mainfrom
realdev12:dataproc-workflow-template-disk-config

Conversation

@realdev12

@realdev12 realdev12 commented Sep 10, 2026

Copy link
Copy Markdown

b/559176576

Description

Adds support for attached_disk_config and missing disk_config properties (boot_disk_provisioned_iops, boot_disk_provisioned_throughput, and local_ssd_interface) to google_dataproc_workflow_template across master_config, worker_config, secondary_worker_config, and instance_selection_list.

Key changes:

  • Added attached_disk_config block supporting Hyperdisks (hyperdisk-balanced, hyperdisk-extreme, hyperdisk-ml, hyperdisk-throughput) with type, disk_type (marked deprecated in favor of type), disk_size_gb, provisioned_iops, and provisioned_throughput.
  • Added bidirectional fallback between type and disk_type during expansion and flattening to ensure compatibility and prevent drift.
  • Corrected wire API serialization to attachedDiskConfigs (plural) according to protobuf and REST API specifications.
  • Added boot_disk_provisioned_iops, boot_disk_provisioned_throughput, and local_ssd_interface to disk_config.
  • Updated test coverage and documentation.

Note: Stacked on top of PR #<PR_1_NUMBER> (instance_flexibility_policy).

Release Note

dataproc: added `boot_disk_provisioned_iops`, `boot_disk_provisioned_throughput`, `local_ssd_interface`, and `attached_disk_config` fields to `google_dataproc_workflow_template`

@github-actions

github-actions Bot commented Sep 10, 2026

Copy link
Copy Markdown

Googlers: For automatic test runs see go/terraform-auto-test-runs.

@SirGitsalot, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

@modular-magician

modular-magician commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes for commit 642e4f2:

Diff report

Your PR generated the following diffs in downstream repositories:

Repository Diff Link Changes
google provider View Diff 6 files changed, 9452 insertions(+), 4125 deletions(-)
google-beta provider View Diff 6 files changed, 9576 insertions(+), 4249 deletions(-)

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_dataproc_workflow_template (5 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_dataproc_workflow_template" "primary" {
  placement {
    managed_cluster {
      config {
        master_config {
          disk_config {
            attached_disk_config {
              disk_type              = # value needed
              provisioned_iops       = # value needed
              provisioned_throughput = # value needed
            }
            boot_disk_provisioned_iops       = # value needed
            boot_disk_provisioned_throughput = # value needed
          }
          instance_flexibility_policy {
            instance_selection_list {
              disk_config {
                attached_disk_config {
                  disk_size_gb           = # value needed
                  disk_type              = # value needed
                  provisioned_iops       = # value needed
                  provisioned_throughput = # value needed
                  type                   = # value needed
                }
                boot_disk_provisioned_iops       = # value needed
                boot_disk_provisioned_throughput = # value needed
                local_ssd_interface              = # value needed
              }
            }
          }
        }
        secondary_worker_config {
          disk_config {
            attached_disk_config {
              disk_type              = # value needed
              provisioned_iops       = # value needed
              provisioned_throughput = # value needed
            }
            boot_disk_provisioned_iops       = # value needed
            boot_disk_provisioned_throughput = # value needed
          }
          instance_flexibility_policy {
            instance_selection_list {
              disk_config {
                attached_disk_config {
                  disk_size_gb           = # value needed
                  disk_type              = # value needed
                  provisioned_iops       = # value needed
                  provisioned_throughput = # value needed
                  type                   = # value needed
                }
                boot_disk_provisioned_iops       = # value needed
                boot_disk_provisioned_throughput = # value needed
                local_ssd_interface              = # value needed
              }
            }
          }
        }
        worker_config {
          disk_config {
            attached_disk_config {
              provisioned_iops       = # value needed
              provisioned_throughput = # value needed
              type                   = # value needed
            }
            boot_disk_provisioned_iops       = # value needed
            boot_disk_provisioned_throughput = # value needed
          }
          instance_flexibility_policy {
            instance_selection_list {
              disk_config {
                attached_disk_config {
                  disk_size_gb           = # value needed
                  disk_type              = # value needed
                  provisioned_iops       = # value needed
                  provisioned_throughput = # value needed
                  type                   = # value needed
                }
                boot_disk_provisioned_iops       = # value needed
                boot_disk_provisioned_throughput = # value needed
                local_ssd_interface              = # value needed
              }
            }
          }
        }
      }
    }
  }
}

Missing doc report (experimental)

The following resources have fields missing in documents.

  • google_dataproc_workflow_template
    • Expected Document Path: /website/docs/r/dataproc_workflow_template.html.markdown
    • Fields: [placement.managed_cluster.config.master_config.disk_config.attached_disk_config placement.managed_cluster.config.master_config.disk_config.attached_disk_config.disk_size_gb placement.managed_cluster.config.master_config.disk_config.attached_disk_config.disk_type placement.managed_cluster.config.master_config.disk_config.attached_disk_config.provisioned_iops placement.managed_cluster.config.master_config.disk_config.attached_disk_config.provisioned_throughput placement.managed_cluster.config.master_config.disk_config.attached_disk_config.type placement.managed_cluster.config.master_config.disk_config.boot_disk_provisioned_iops placement.managed_cluster.config.master_config.disk_config.boot_disk_provisioned_throughput placement.managed_cluster.config.master_config.disk_config.local_ssd_interface placement.managed_cluster.config.master_config.instance_flexibility_policy placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_machine_types placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.disk_size_gb placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.disk_type placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.provisioned_iops placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.provisioned_throughput placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.type placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_provisioned_iops placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_provisioned_throughput placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_size_gb placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_type placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.local_ssd_interface placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.disk_config.num_local_ssds placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.machine_types placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_list.rank placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_results placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_results.machine_type placement.managed_cluster.config.master_config.instance_flexibility_policy.instance_selection_results.vm_count placement.managed_cluster.config.secondary_worker_config.disk_config.attached_disk_config placement.managed_cluster.config.secondary_worker_config.disk_config.attached_disk_config.disk_size_gb placement.managed_cluster.config.secondary_worker_config.disk_config.attached_disk_config.disk_type placement.managed_cluster.config.secondary_worker_config.disk_config.attached_disk_config.provisioned_iops placement.managed_cluster.config.secondary_worker_config.disk_config.attached_disk_config.provisioned_throughput placement.managed_cluster.config.secondary_worker_config.disk_config.attached_disk_config.type placement.managed_cluster.config.secondary_worker_config.disk_config.boot_disk_provisioned_iops placement.managed_cluster.config.secondary_worker_config.disk_config.boot_disk_provisioned_throughput placement.managed_cluster.config.secondary_worker_config.disk_config.local_ssd_interface placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_machine_types placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.disk_size_gb placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.disk_type placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.provisioned_iops placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.provisioned_throughput placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.type placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_provisioned_iops placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_provisioned_throughput placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_size_gb placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_type placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.local_ssd_interface placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.disk_config.num_local_ssds placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.machine_types placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_list.rank placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_results placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_results.machine_type placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.instance_selection_results.vm_count placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.provisioning_model_mix placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.provisioning_model_mix.standard_capacity_base placement.managed_cluster.config.secondary_worker_config.instance_flexibility_policy.provisioning_model_mix.standard_capacity_percent_above_base placement.managed_cluster.config.worker_config.disk_config.attached_disk_config placement.managed_cluster.config.worker_config.disk_config.attached_disk_config.disk_size_gb placement.managed_cluster.config.worker_config.disk_config.attached_disk_config.disk_type placement.managed_cluster.config.worker_config.disk_config.attached_disk_config.provisioned_iops placement.managed_cluster.config.worker_config.disk_config.attached_disk_config.provisioned_throughput placement.managed_cluster.config.worker_config.disk_config.attached_disk_config.type placement.managed_cluster.config.worker_config.disk_config.boot_disk_provisioned_iops placement.managed_cluster.config.worker_config.disk_config.boot_disk_provisioned_throughput placement.managed_cluster.config.worker_config.disk_config.local_ssd_interface placement.managed_cluster.config.worker_config.instance_flexibility_policy placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_machine_types placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.disk_size_gb placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.disk_type placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.provisioned_iops placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.provisioned_throughput placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.attached_disk_config.type placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_provisioned_iops placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_provisioned_throughput placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_size_gb placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.boot_disk_type placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.local_ssd_interface placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.disk_config.num_local_ssds placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.machine_types placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_list.rank placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_results placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_results.machine_type placement.managed_cluster.config.worker_config.instance_flexibility_policy.instance_selection_results.vm_count]

Test report

Analytics

Total Tests Passed Skipped Affected
94 86 6 2
Affected Service Packages
  • dataproc

Learn how VCR tests work


Step 1: Replaying Mode

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit.

Click here to see the affected tests
  • TestAccDataprocWorkflowTemplate_diskConfig
  • TestAccDataprocWorkflowTemplate_instanceFlexibilityPolicy

View the replaying VCR build log


Step 2: Recording Mode

Recording Mode Replaying Rerun Test Name
❌ Error · Log - TestAccDataprocWorkflowTemplate_diskConfig
❌ Error · Log - TestAccDataprocWorkflowTemplate_instanceFlexibilityPolicy

Caution

Issues requiring attention before PR completion

🔴 Initial Recording Failed: Some tests failed during the recording step. See the table above for details.

Please address these issues to complete your PR. If you believe these detections are incorrect or unrelated to your change, please raise the concern with your reviewer.

View the recording VCR build log or the debug logs folder for detailed results.

@realdev12, @rileykarson VCR tests complete for 642e4f2!

@melinath

Copy link
Copy Markdown
Member

@modular-magician reassign-reviewer @SirGitsalot

@github-actions
github-actions Bot requested review from SirGitsalot and removed request for rileykarson September 10, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants