feat(rust/sedona-raster-gdal): add RS_MetaData by Kontinuation · Pull Request #833 · apache/sedona-db

Kontinuation · 2026-05-11T05:47:52Z

Summary

add RS_MetaData backed by the GDAL dataset provider
expose raster dimensions, transform, srid, band count, and GDAL block size metadata
keep the change limited to the incremental metadata surface on top of the current GDAL raster work

Dependency

Depends on feat(rust/sedona-raster-gdal): add RS_FromPath #831 (RS_FromPath)
This branch currently includes the parent RS_FromPath commit because GitHub upstream PRs cannot target a fork-only base branch.
Please review/merge this after feat(rust/sedona-raster-gdal): add RS_FromPath #831, or compare this PR commit-by-commit with that dependency in mind.

Testing

cargo test -p sedona-raster-gdal
cargo clippy -p sedona-raster-gdal -- -D warnings
cargo test -p sedona-gdal

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.

Commit changes made to Cargo.lock

paleolimbot

Thank you!

It would be great to get the example to be user runnable (whatever we do here will also help make realistic examples for other functions, too). Other than that these are just nits.

paleolimbot · 2026-06-10T14:33:53Z

+```sql
+SELECT RS_MetaData(
+  RS_FromPath('../../../submodules/sedona-testing/data/raster/test4.tiff')
+);
+```


Can we use the http:// url here so that this example is runnable by a user? Or enable RS_Example("fixture name")?

paleolimbot · 2026-06-10T14:35:32Z

+        Field::new("upperLeftX", DataType::Float64, true),
+        Field::new("upperLeftY", DataType::Float64, true),
+        Field::new("gridWidth", DataType::Int64, true),


I get that we probably have to do this for compatibility with Sedona Spark, but it's a bit of a bummer these aren't snake_case.

paleolimbot · 2026-06-10T14:36:32Z

+        let mut num_bands_builder = UInt32Builder::with_capacity(capacity);
+        let mut tile_width_builder = UInt64Builder::with_capacity(capacity);
+        let mut tile_height_builder = UInt64Builder::with_capacity(capacity);
+        let mut struct_validity = Vec::with_capacity(capacity);


Can you use a NullsBuffer here?

paleolimbot · 2026-06-10T14:38:05Z

+                        let srid = match raster.crs() {
+                            None => 0i32,
+                            Some(crs_str) => match deserialize_crs(crs_str) {
+                                Ok(Some(crs_ref)) => crs_ref
+                                    .srid()
+                                    .ok()
+                                    .flatten()
+                                    .and_then(|s| i32::try_from(s).ok())
+                                    .unwrap_or(0),
+                                _ => 0i32,
+                            },
+                        };
+                        srid_builder.append_value(srid);


I get how this is needed for compatibility, but we should arguably change the SedonaSpark version to return a string here so that we avoid this being lossy for un-sridable tiffs.

paleolimbot · 2026-06-10T14:44:13Z

+    fn invoke_array_result(raster_array: StructArray) -> StructArray {
+        metadata_udf_tester()
+            .invoke_array(Arc::new(raster_array))
+            .unwrap()
+            .as_struct()
+            .clone()
+    }
+
+    fn assert_scalar_result(result: ScalarValue) -> Arc<StructArray> {
+        match result {
+            ScalarValue::Struct(struct_array) => struct_array,
+            other => panic!("Expected struct scalar result, got {other:?}"),
+        }
+    }


For analyze agg we have a JSON based struct approach that could be copied or moved to sedona-testing and reused (but your tests here are quite readable and I'm not sure it's an improvement other than less helper code).

sedona-db/rust/sedona-functions/src/st_analyze_agg.rs

Lines 519 to 540 in f64ed45

fn actual_result_json(struct_array: Arc<StructArray>) -> Value {

// Create a schema that matches the struct array fields

let fields = struct_array.fields().to_vec();

let schema = Schema::new(fields);

// Create a record batch with just the struct array converted to a column

let columns: Vec<ArrayRef> = struct_array.columns().to_vec();

let record_batch = RecordBatch::try_new(Arc::new(schema), columns).unwrap();

// Serialize to JSON using Arrow's JSON writer

let buf = Vec::new();

let mut writer = ArrayWriter::new(buf);

writer.write_batches(&[&record_batch]).unwrap();

writer.finish().unwrap();

// Get the JSON string

let json_str = String::from_utf8(writer.into_inner()).unwrap();

let json_value: Value = serde_json::from_str(&json_str).unwrap();

// Get the first (and only) row from the JSON array

json_value[0].clone()

}

Kontinuation mentioned this pull request May 11, 2026

feat(rust/sedona-raster-gdal): add RS_MetaData Kontinuation/sedona-db#3

Closed

Kontinuation force-pushed the pr-g-rs-metadata branch 2 times, most recently from e6ea9a9 to 201f7f7 Compare June 6, 2026 13:52

Kontinuation requested a review from Copilot June 8, 2026 15:05

Copilot started reviewing on behalf of Kontinuation June 8, 2026 15:05 View session

Copilot AI reviewed Jun 8, 2026

Kontinuation requested a review from Copilot June 8, 2026 15:30

Copilot started reviewing on behalf of Kontinuation June 8, 2026 15:30 View session

Copilot AI reviewed Jun 8, 2026

View reviewed changes

Comment thread rust/sedona-raster-gdal/src/rs_metadata.rs

Comment thread rust/sedona-raster-gdal/src/rs_metadata.rs Outdated

Comment thread docs/reference/sql/rs_metadata.qmd Outdated

Comment thread rust/sedona-raster-gdal/src/rs_metadata.rs

Kontinuation marked this pull request as ready for review June 9, 2026 01:56

kontinuation added 5 commits June 10, 2026 21:05

feat(rust/sedona-raster-gdal): add RS_MetaData

fff6922

Commit changes made to Cargo.lock

test(docs/rust/sedona-raster-gdal): cover and document RS_MetaData

87e2e51

Change type of num bands field and don't normalize tile width/height

3ff35b3

fix(docs/rust/sedona-raster-gdal): address RS_MetaData review comments

120831a

fix(rust/sedona-raster-gdal): align RS_MetaData dimensions with i64

3168431

Kontinuation force-pushed the pr-g-rs-metadata branch from 77c404f to 3168431 Compare June 10, 2026 13:52

paleolimbot approved these changes Jun 10, 2026

View reviewed changes

fix(docs/rust/sedona-raster-gdal): make RS_MetaData examples runnable

a5bd403

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(rust/sedona-raster-gdal): add RS_MetaData#833

feat(rust/sedona-raster-gdal): add RS_MetaData#833
Kontinuation wants to merge 6 commits into
apache:mainfrom
Kontinuation:pr-g-rs-metadata

Kontinuation commented May 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paleolimbot left a comment

Uh oh!

paleolimbot Jun 10, 2026

Uh oh!

paleolimbot Jun 10, 2026

Uh oh!

paleolimbot Jun 10, 2026

Uh oh!

paleolimbot Jun 10, 2026

Uh oh!

paleolimbot Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	fn actual_result_json(struct_array: Arc<StructArray>) -> Value {
	// Create a schema that matches the struct array fields
	let fields = struct_array.fields().to_vec();
	let schema = Schema::new(fields);

	// Create a record batch with just the struct array converted to a column
	let columns: Vec<ArrayRef> = struct_array.columns().to_vec();
	let record_batch = RecordBatch::try_new(Arc::new(schema), columns).unwrap();

	// Serialize to JSON using Arrow's JSON writer
	let buf = Vec::new();
	let mut writer = ArrayWriter::new(buf);
	writer.write_batches(&[&record_batch]).unwrap();
	writer.finish().unwrap();

	// Get the JSON string
	let json_str = String::from_utf8(writer.into_inner()).unwrap();
	let json_value: Value = serde_json::from_str(&json_str).unwrap();

	// Get the first (and only) row from the JSON array
	json_value[0].clone()
	}

Conversation

Kontinuation commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Dependency

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paleolimbot left a comment

Choose a reason for hiding this comment

Uh oh!

paleolimbot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

paleolimbot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

paleolimbot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

paleolimbot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

paleolimbot Jun 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Kontinuation commented May 11, 2026 •

edited

Loading