Skip to content

Feat: Add Model RF-DETR#333

Open
DerrickUnleashed wants to merge 22 commits into
mlverse:mainfrom
DerrickUnleashed:feat/modelRfdetr
Open

Feat: Add Model RF-DETR#333
DerrickUnleashed wants to merge 22 commits into
mlverse:mainfrom
DerrickUnleashed:feat/modelRfdetr

Conversation

@DerrickUnleashed

Copy link
Copy Markdown
Contributor

This PR adds :

  • Real-Time Detection Transformers Model (Object Detection Model)
  • Test Suite for the same

Closes #327

@DerrickUnleashed

Copy link
Copy Markdown
Contributor Author
=======================================
Running inference with: model_rfdetr_base
========================================
Model weights for <rfdetr_base> (~123 MB) will be downloaded and processed if
not already available.
Loaded pretrained weights for <rfdetr_base> (487/487 keys, 0 skipped).
Top detections:
  score=0.8337 label=17 box=[156,37,1440,1148]
  score=0.1323 label=18 box=[156,37,1440,1148]
  score=0.0865 label=23 box=[156,37,1440,1148]
  score=0.0846 label=20 box=[156,37,1440,1148]
  score=0.0698 label=18 box=[164,23,1440,1148]
Detections above 0.3: 1

========================================
Running inference with: model_rfdetr_base_2
========================================
Model weights for <rfdetr_base_2> (~123 MB) will be downloaded and processed
if not already available.
Loaded pretrained weights for <rfdetr_base_2> (487/487 keys, 0 skipped).
Top detections:
  score=0.6605 label=17 box=[156,38,1440,1152]
  score=0.2869 label=18 box=[156,38,1440,1152]
  score=0.1364 label=23 box=[156,38,1440,1152]
  score=0.0740 label=20 box=[156,38,1440,1152]
  score=0.0676 label=16 box=[156,38,1440,1152]
Detections above 0.3: 1

========================================
Running inference with: model_rfdetr_base_o365
========================================
Model weights for <rfdetr_base_o365> (~127 MB) will be downloaded and
processed if not already available.
Loaded pretrained weights for <rfdetr_base_o365> (487/487 keys, 0 skipped).
Top detections:
  score=0.5500 label=140 box=[158,34,1440,1144]
  score=0.4891 label=93 box=[158,34,1440,1144]
  score=0.1077 label=343 box=[158,34,1440,1144]
  score=0.0971 label=321 box=[158,34,1440,1144]
  score=0.0838 label=84 box=[158,34,1440,1144]
Detections above 0.3: 2

========================================
Running inference with: model_rfdetr_large
========================================
Model weights for <rfdetr_large> (~518 MB) will be downloaded and processed
if not already available.
Loaded pretrained weights for <rfdetr_large> (533/533 keys, 0 skipped).
Top detections:
  score=0.9274 label=18 box=[156,40,1444,1136]
  score=0.0538 label=18 box=[163,36,1440,1137]
  score=0.0460 label=19 box=[156,40,1444,1136]
  score=0.0403 label=17 box=[163,36,1440,1137]
  score=0.0336 label=11 box=[156,40,1444,1136]
Detections above 0.3: 1

========================================
Running inference with: model_rfdetr_medium
========================================
Model weights for <rfdetr_medium> (~116 MB) will be downloaded and processed
if not already available.
Loaded pretrained weights for <rfdetr_medium> (465/465 keys, 0 skipped).
Top detections:
  score=0.3381 label=17 box=[155,35,1460,1158]
  score=0.2565 label=18 box=[155,35,1460,1158]
  score=0.1775 label=23 box=[155,35,1460,1158]
  score=0.1350 label=23 box=[157,34,1461,1156]
  score=0.0925 label=17 box=[157,34,1461,1156]
Detections above 0.3: 1

========================================
Running inference with: model_rfdetr_nano
========================================
Model weights for <rfdetr_nano> (~116 MB) will be downloaded and processed if
not already available.
Loaded pretrained weights for <rfdetr_nano> (465/465 keys, 0 skipped).
Top detections:
  score=0.3381 label=17 box=[155,35,1460,1158]
  score=0.2565 label=18 box=[155,35,1460,1158]
  score=0.1775 label=23 box=[155,35,1460,1158]
  score=0.1350 label=23 box=[157,34,1461,1156]
  score=0.0925 label=17 box=[157,34,1461,1156]
Detections above 0.3: 1

========================================
Running inference with: model_rfdetr_small
========================================
Model weights for <rfdetr_small> (~116 MB) will be downloaded and processed
if not already available.
Loaded pretrained weights for <rfdetr_small> (465/465 keys, 0 skipped).
Top detections:
  score=0.5835 label=17 box=[154,34,1459,1155]
  score=0.4582 label=18 box=[154,34,1459,1155]
  score=0.3348 label=23 box=[154,34,1459,1155]
  score=0.0870 label=64 box=[154,34,1459,1155]
  score=0.0798 label=20 box=[154,34,1459,1155]
Detections above 0.3: 3
fileef4133e485e8

@DerrickUnleashed DerrickUnleashed changed the title Feat/model rfdetr Feat: Add Model RF-DETR Jun 16, 2026
@DerrickUnleashed

DerrickUnleashed commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Can we implement a vignette for this perhaps? @cregouby

@cregouby cregouby left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise This is massive, thanks for it
todo see inline.

Comment thread NEWS.md
Comment thread R/models-rfdetr_detection.R
Comment thread tests/testthat/test-models-rfdetr.R
Comment thread R/models-rfdetr_detection.R Outdated
Comment thread R/models-rfdetr_detection.R Outdated
Comment thread R/models-rfdetr_detection.R Outdated
Comment thread R/models-rfdetr_detection.R
Comment thread R/models-rfdetr_detection.R Outdated
DerrickUnleashed and others added 12 commits July 5, 2026 21:02
- Add representative usage example for model_rfdetr_large with image loading,
  inference, postprocessing, and bounding box visualization
- Add attribution comment referencing the original RF-DETR implementation
In eval mode, model forward now returns list(detections = ...) with each
detection containing boxes (xyxy), labels, and scores — matching the
convention used by LW-DETR, Faster R-CNN, and other models.
- Clamp leading corners to >= 0, enforce minimum 2px box dimensions
- Adjust test helper: check cat label anywhere in detections (not just top)
- Lower min_score default from 0.25 to 0.15 for smaller model variants
The model forward now accepts an optional target_sizes parameter to
scale boxes to original image dimensions (matching original behavior).
When omitted, boxes are scaled to the input tensor dimensions. Example
updated to pass target_sizes so boxes draw correctly on the original image.
… format

- Model now auto-resizes input images to its native resolution during
  eval, recording original dims for automatic box scaling via target_sizes
- Users no longer need to call nnf_interpolate or pass target_sizes manually
- Example simplified to match LW-DETR example style: load image, call model,
  draw top-5 detections with tensor_image_browse
@DerrickUnleashed

Copy link
Copy Markdown
Contributor Author
url <- "https://upload.wikimedia.org/wikipedia/commons/6/6f/Toy_Poodle_wearing_clothes_in_Tokyo.jpg"
img <- base_loader(url) %>%
  transform_to_tensor()

model <- model_rfdetr_large(pretrained = TRUE)
model$eval()

pred <- torch::with_no_grad(
  model(img$unsqueeze(1))
)$detections[[1]]

topk <- pred$scores$topk(k = 5L)[[2]]
boxes <- pred$boxes[topk, ]
labels <- coco_classes(as.integer(pred$labels[topk]))
canvas <- (img * 255)$to(dtype = torch_uint8())
boxed <- draw_bounding_boxes(canvas, boxes, labels = labels,
 colors = "black", width = 6)
tensor_image_browse(boxed)
test

@DerrickUnleashed DerrickUnleashed requested a review from cregouby July 6, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Object Detection Model] Please implement RF-DETR

2 participants