Feat: Add SAHI (Slicing Aided Hyper Inference) Transform by DerrickUnleashed · Pull Request #329 · mlverse/torchvision

DerrickUnleashed · 2026-06-10T16:49:12Z

This PR Adds:

The implementation of transform_sahi_crop()
The implementation of target_transform_sahi_crop()
Adds test suites for the same
Adds export command to fix this error

✖ transforms-tensor.R:480: S3 method
  `get_image_size.torch_tensor` needs
  @export or @exportS3Method tag.

Updates nms.Rd Documentation to Roxygen2
Fixes Implement SAHI: (Slicing Aided Hyper Inference) as both transform_ and target_transform_ #324

DerrickUnleashed · 2026-06-11T20:03:10Z

@cregouby, I'm still a bit unclear about the usage of target_transform_sahi_crop() However, the transform_sahi_crop() seems to be working as expected

cregouby

praise I think that the cropping values logic is there.
todo design all transform_ functions shall return a object of the same type as the input (except transform to tensor) as they will be piped. Your only freedom is to change the shape of the tensor, and move the crops into a sequential batch, as transform_five_crop() do.
ex for x$shape [1, 3, 10, 10], transform_sahi_crop(x, c(4,4) , c(.2, .2)) shall output a tensor of shape [25, 3, 3, 3] (see

torchvision/tests/testthat/test-transforms-tensor.R

Lines 161 to 174 in ac32fa5

    
           test_that("five_crop", { 
        
             x <- torch_randn(3, 10, 12) 
        
             o <- transform_five_crop(x, c(3, 3)) 
        
             expect_length(o, 5) 
        
             expect_tensor_shape(o[[1]], c(3,3,3)) 
        
             ob <- transform_five_crop(x$unsqueeze(1), c(3, 3)) 
        
             expect_length(ob, 5) 
        
             expect_tensor_shape(ob[[1]], c(1,3,3,3)) 
        
           })

)
suggestion you should rely on the existing crop function to crop the tensor, saving you a lot of unit tests.
todo software design this transform shall be an S3 transform so shall have an entry in transform-generics.R for the dispatch, one in transforms-defaults.R, one in transforms-magick.R, one in transforms-tensor.R, as you never know what transform goes before, what transform is piped after.

todo code relocation Please keep (the badly named, my fault) transform-segmentation.R file for target_transforms
suggestion for clarity, you may rename the transform-segmentation.R into target-transform-segmentation.R (and the test file accordingly)

thought as SAHI intimately rely on input image size and coco bbox (so both x and y at the same time), and as there is no way to pass information from transform_sahi_crop to target_transform_sahi_crop, I think maybe a prepare_sahi_crop(dataset, size, overlap_size_ratio, ... ) function gathering all the needed context into a specific class object sahi_preparation and having both functions using it a the only input parameter transform_sahi_crop(x, sahi_preparation) and target_transform_sahi_crop(x, sahi_preparation) could be a nice API to SAHI.

DerrickUnleashed · 2026-06-13T17:56:37Z

todo software design this transform shall be an S3 transform so shall have an entry in transform-generics.R for the dispatch, one in transforms-defaults.R, one in transforms-magick.R, one in transforms-tensor.R, as you never know what transform goes before, what transform is piped after.

I didn't understand properly, Do I need to move the code of transform_sahi and target_transform_sahi to transform-generics ?

cregouby · 2026-06-15T05:23:18Z

No. Transform_sahi_crop must manage magick images, tensors and nothing else. The way to do it is S3 dispatch which required an entry in each of the mentioned R files. Le 13 juin 2026 19:56:59 GMT+02:00, Derrick Richard ***@***.***> a écrit :

…

DerrickUnleashed left a comment (mlverse/torchvision#329) > **todo** **software design** this transform shall be an S3 transform so shall have an entry in transform-generics.R for the dispatch, one in transforms-defaults.R, one in transforms-magick.R, one in transforms-tensor.R, as you never know what transform goes before, what transform is piped after. I didn't understand properly, Do I need to move the code of transform_sahi and target_transform_sahi to transform-generics ? -- Reply to this email directly or view it on GitHub: #329 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

cregouby

praise thanks for your modifications.
todo a small effort again for completness.

…-frame magick)

Co-authored-by: cregouby <cregouby@users.noreply.github.com>

…op-count test

…hed/torchvision into feat/sahiTransform

… expect_tensor_shape and expect_tensor_dtype

cregouby

praise thanks for those improvements.
todo I've add the proper output data model in here : #324 (comment). Sorry for doing that so late, but I think this is the only way to have target_transform_sahi_crop transparent to upstream target transform (like the young target_transform_resize) and eventually downstream.

Co-authored-by: cregouby <cregouby@users.noreply.github.com>

# Conflicts: # DESCRIPTION # NAMESPACE # R/transforms-segmentation.R

…hvision into feat/sahiTransform

…hed/torchvision into feat/sahiTransform

cregouby · 2026-07-05T17:18:17Z

@DerrickUnleashed ,
I've merge the latest addition in main so that you don't have to worry about file names and functions family.
You will now find target_transform_sahi_crop() into target_transform_detection.R
Hope it helps.

DerrickUnleashed · 2026-07-05T17:20:37Z

@DerrickUnleashed , I've merge the latest addition in main so that you don't have to worry about file names and functions family. You will now find target_transform_sahi_crop() into target_transform_detection.R Hope it helps.

Awesome thanks for this

cregouby · 2026-07-06T15:50:57Z

+#' @rdname transform_sahi_crop
+#' @family target_transforms_detection
+#' @export
+target_transform_sahi_crop <- function(y, sahi_split, min_area_ratio = 0.1) {
+
+  # Detect batch input: list of target lists
+  if (is.list(y) && !"boxes" %in% names(y)) {
+    if (is.list(sahi_split) && !inherits(sahi_split, "sahi_split")) {
+      return(Map(function(yi, sp) target_transform_sahi_crop(yi, sp, min_area_ratio),
+                 y, sahi_split))
+    }
+    return(lapply(y, function(yi) target_transform_sahi_crop(yi, sahi_split, min_area_ratio)))
+  }
+
+  boxes <- y$boxes
+  labels <- y$labels
+  labels_is_tensor <- inherits(labels, "torch_tensor")
+
+  n <- boxes$size(1)
+
+  crop_windows <- sahi_split$crop_windows
+
+  shape_y <- function(n_boxes, boxes_dtype, labels_val, labels_dtype, crop_h, crop_w) {
+    out <- y
+    out$boxes <- torch_zeros(c(n_boxes, 4), dtype = boxes_dtype)
+    if (labels_is_tensor)
+      out$labels <- torch_tensor(labels_val, dtype = labels_dtype)
+    else
+      out$labels <- labels_val
+    if (!is.null(y$area))
+      out$area <- torch_zeros(n_boxes, dtype = y$area$dtype)
+    if (!is.null(y$image_height))
+      out$image_height <- crop_h
+    if (!is.null(y$image_width))
+      out$image_width <- crop_w
+    out$iscrowd <- NULL
+    out
+  }
+
+  results <- lapply(crop_windows, function(cw) {
+
+    # Convert 1-based crop window coordinates to 0-based for box clipping
+    top <- cw$top - 1
+    left <- cw$left - 1
+    crop_h <- cw$height
+    crop_w <- cw$width
+
+    if (n == 0) {
+      labels_val <- if (labels_is_tensor) integer(0) else vector(typeof(labels), 0)
+      labels_dtype <- if (labels_is_tensor) labels$dtype else NULL
+      return(shape_y(0, boxes$dtype, labels_val, labels_dtype, crop_h, crop_w))
+    }
+
+    x1 <- boxes[, 1]
+    y1 <- boxes[, 2]
+    x2 <- boxes[, 3]
+    y2 <- boxes[, 4]
+
+    orig_area <- (x2 - x1) * (y2 - y1)
+
+    x1_clip <- torch_clamp(x1, min = left, max = left + crop_w)
+    y1_clip <- torch_clamp(y1, min = top, max = top + crop_h)
+    x2_clip <- torch_clamp(x2, min = left, max = left + crop_w)
+    y2_clip <- torch_clamp(y2, min = top, max = top + crop_h)
+
+    keep_w <- x2_clip - x1_clip
+    keep_h <- y2_clip - y1_clip
+    keep_area <- keep_w * keep_h
+
+    mask <- (keep_area > 0) & ((keep_area / orig_area) >= min_area_ratio)
+
+    mask_idx <- which(as.logical(mask))
+
+    if (length(mask_idx) == 0) {
+      labels_val <- if (labels_is_tensor) integer(0) else vector(typeof(labels), 0)
+      labels_dtype <- if (labels_is_tensor) labels$dtype else NULL
+      return(shape_y(0, boxes$dtype, labels_val, labels_dtype, crop_h, crop_w))
+    }
+
+    n_keep <- length(mask_idx)
+    new_boxes <- torch_zeros(n_keep, 4, dtype = boxes$dtype)
+    new_boxes[, 1] <- x1_clip[mask_idx] - left
+    new_boxes[, 2] <- y1_clip[mask_idx] - top
+    new_boxes[, 3] <- x2_clip[mask_idx] - left
+    new_boxes[, 4] <- y2_clip[mask_idx] - top
+
+    out_y <- y
+    out_y$boxes <- new_boxes
+
+    if (labels_is_tensor)
+      out_y$labels <- labels[mask_idx]
+    else
+      out_y$labels <- labels[mask_idx]
+
+    if (!is.null(y$area))
+      out_y$area <- keep_area[mask_idx]
+
+    if (!is.null(y$image_height))
+      out_y$image_height <- crop_h
+    if (!is.null(y$image_width))
+      out_y$image_width <- crop_w
+
+    out_y$iscrowd <- NULL
+
+    out_y
+  })
+
+  results
+}


todo code duplicate. Please remove

cregouby · 2026-07-06T15:53:07Z

+    if (!is.null(y$image_width))
+      out_y$image_width <- crop_w
+
+    out_y$iscrowd <- NULL


Suggested change

out_y$iscrowd <- NULL

out_y$iscrowd <- y$iscrowd[mask_idx]

cregouby · 2026-07-06T16:48:53Z

+#' @export
+prepare_sahi_split.dataset <- function(x, size = c(512L, 512L), overlap_size_ratio = c(0.2, 0.2)) {
+  meta <- x$image_metadata
+  if (!is.null(meta) && is.list(meta)) {
+    first <- meta[[1]]
+    image_height <- first$height
+    image_width <- first$width
+    if (!is.null(image_height) && !is.null(image_width))
+      return(compute_sahi_split(image_height, image_width, size, overlap_size_ratio))
+  }
+  item <- x$.getitem(1)
+  im <- item$x
+  if (inherits(im, "torch_tensor")) {
+    image_height <- im$size(-2)
+    image_width <- im$size(-1)
+  } else if (inherits(im, "array")) {
+    image_height <- dim(im)[1]
+    image_width <- dim(im)[2]
+  } else {


todo ordering logic dataset image metadata shall not be the first way to retrieve image shape, but the last resort one. This is because usual (coco) dataset have varying image size and will be first resized by end-user like via transform = transform_resize() or via target_transform = target_transform_resize(), or both if it is unclear witch is considered. The first one will change the item$x image size, the second will change the item$y image_wdth and image_heigh metadata values. Those are the primary value to consider for actual size.
Typical workflow with dataset :

ds <- coco_detection_dataset( train=FALSE, year = "2017", download = TRUE, transform = . %>% transform_to_tensor() %>% transform_resize(c(500, 400)), target_transform = . %>% target_transform_resize(c(500, 400)) ) item <- ds[1] sahi_prep <- prepare_sahi_split(ds, size = c(200L, 200L), overlap_size_ratio = c(0.2, 0.2)) ds_sahi <- coco_detection_dataset( train=FALSE, year = "2017", download = FALSE, transform = . %>% transform_to_tensor() %>% transform_resize(c(500, 400)) %>% transform_sahi_crop(sahi_prep), target_transform = . %>% target_transform_resize(c(500, 400)) %>% target_transform_sahi_crop(sahi_prep, min_area_ratio = 0.2) )

This currently fails with

Erreur : stack expects each tensor to be equal size, but got [3, 200, 200] at entry 0 and [3, 200, 80] at entry 2
Exception raised from check_stack_inputs at /pytorch/aten/src/ATen/native/TensorShape.cpp:3345 (most recent call first):

suggestion Add this workflow as a documentation exemple

cregouby · 2026-07-06T16:57:54Z

todo missing Are all the created bbox xyxy ?
suggestion use expect_bbox_is_xyxy(out[[1]]$boxes) extensively

DerrickUnleashed and others added 19 commits June 10, 2026 21:38

Chore: Update Documentation

ac2a0a8

Chore: Update Description & Namespace

9f1b9b0

Feat: Add SAHI transform

835cb2d

Chore: Add tests for SAHI

3b2555d

Fix: missing exporrt error

6f08cf5

Chore: Update the Documentation similar to other transforms

1a9be76

Merge branch 'main' into feat/sahiTransform

543fb9a

Chore: Update nms.Rd to Roxygen2

00dccd1

Init: Transform and Target Transform code

efbf0e7

Revert Documentation

0e407d6

Working versions of transforms

6be235f

Remove torch:: imports

f0a5c81

Fix lint

4374933

Chore: Add Documentation

d002a3d

Chore: Update NEWS.md

4afd862

Update doc for transform

720a06c

Update doc for transform

accf698

Chore Add tests for transforms

e01b8a1

Chore: Update docs for target_tranform

7ef61dd

DerrickUnleashed marked this pull request as ready for review June 11, 2026 20:02

DerrickUnleashed added 2 commits June 12, 2026 01:46

Add tests for target_transforms

732c11c

Using distince expect_ fucns for testing

d0633fa

cregouby requested changes Jun 13, 2026

View reviewed changes

Comment thread .vscode/settings.json Outdated

Comment thread R/transforms-tensor.R

Comment thread R/transforms-segmentation.R Outdated

Comment thread R/transforms-segmentation.R Outdated

DerrickUnleashed added 3 commits June 13, 2026 21:10

Remove unnecessary files

0c7f428

Remove family transforms

286b029

Add realistic example

5ba16b6

DerrickUnleashed force-pushed the feat/sahiTransform branch from a56d682 to 5ba16b6 Compare June 13, 2026 17:34

cregouby reviewed Jun 28, 2026

View reviewed changes

Comment thread tests/testthat/test-transforms-array.R Outdated

cregouby reviewed Jun 28, 2026

View reviewed changes

Comment thread tests/testthat/test-transforms-tensor.R

cregouby reviewed Jun 28, 2026

View reviewed changes

Comment thread tests/testthat/test-transforms-tensor.R Outdated

cregouby requested changes Jun 28, 2026

View reviewed changes

Comment thread tests/testthat/test-target-transforms-segmentation.R

DerrickUnleashed and others added 8 commits June 28, 2026 21:23

Updating tests

9111c5a

Add batch input tests for SAHI transforms (4D tensor, 4D array, multi…

9c550c8

…-frame magick)

Update tests/testthat/test-transforms-tensor.R

f8007cb

Co-authored-by: cregouby <cregouby@users.noreply.github.com>

Add batch input handling to target_transform_sahi_crop and overlap cr…

a27a4e5

…op-count test

Merge branch 'feat/sahiTransform' of https://github.com/DerrickUnleas…

d1f329e

…hed/torchvision into feat/sahiTransform

Remove convert-then-split test case

265a59c

Remove redundant array SAHI tests, strengthen batched array test with…

f16b75a

… expect_tensor_shape and expect_tensor_dtype

Fix iscrowd field name (COCO uses iscrowd not is_crowd)

5ecc45e

DerrickUnleashed requested a review from cregouby June 28, 2026 17:35

cregouby requested changes Jul 5, 2026

View reviewed changes

Comment thread tests/testthat/test-transforms-array.R Outdated

Comment thread tests/testthat/test-transforms-magick.R

Comment thread tests/testthat/test-transforms-tensor.R Outdated

DerrickUnleashed and others added 5 commits July 5, 2026 22:37

Update tests/testthat/test-transforms-tensor.R

07fada6

Co-authored-by: cregouby <cregouby@users.noreply.github.com>

Merge branch 'main' into feat/sahiTransform

9ef6a50

# Conflicts: # DESCRIPTION # NAMESPACE # R/transforms-segmentation.R

Merge branch 'feat/sahiTransform' of github.com:DerrickUnleashed/torc…

c294317

…hvision into feat/sahiTransform

Flatten crop and batch dims for 4D tensor input to preserve ndim

6b66334

Merge branch 'feat/sahiTransform' of https://github.com/DerrickUnleas…

000a8b8

…hed/torchvision into feat/sahiTransform

DerrickUnleashed added 4 commits July 5, 2026 22:53

Fix merge conflicts

6952a09

Retrigger workflows

6fd59d3

Retrigger workflows

657c12a

Retrigger Workflow

b7949ea

cregouby reviewed Jul 6, 2026

View reviewed changes

fix my double paste

7ac083d

cregouby reviewed Jul 6, 2026

View reviewed changes

	test_that("five_crop", {

	x <- torch_randn(3, 10, 12)
	o <- transform_five_crop(x, c(3, 3))

	expect_length(o, 5)
	expect_tensor_shape(o[[1]], c(3,3,3))

	ob <- transform_five_crop(x$unsqueeze(1), c(3, 3))

	expect_length(ob, 5)
	expect_tensor_shape(ob[[1]], c(1,3,3,3))

	})

Uh oh!

Conversation

DerrickUnleashed commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DerrickUnleashed commented Jun 11, 2026

Uh oh!

cregouby left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DerrickUnleashed commented Jun 13, 2026

Uh oh!

cregouby commented Jun 15, 2026 via email

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cregouby left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cregouby left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cregouby commented Jul 5, 2026

Uh oh!

DerrickUnleashed commented Jul 5, 2026

Uh oh!

cregouby Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

cregouby Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

cregouby Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cregouby Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

DerrickUnleashed commented Jun 10, 2026 •

edited

Loading

cregouby left a comment •

edited

Loading

cregouby Jul 6, 2026 •

edited

Loading