From ecbb4aab04855c1e78b7b4146ad9eb3a6e27c8f5 Mon Sep 17 00:00:00 2001 From: Dan Wahl Date: Wed, 8 Jul 2026 23:05:32 -0500 Subject: [PATCH 1/4] feat(space_satellite_sim_camera_cal): use CalibrateCameraPosePnp for scene camera cal Rewire the scene-camera calibration leaf to GetImage + GetCameraInfo + CalibrateCameraPosePnp (single joint PnP over all tag corners), replacing the DetectAprilTags + per-tag GetDetectionPose + CalibrateCameraPose path. Replace the decorative calibration_tag_right/left/bottom links with TF-only calibration_tag_0/1/2 frames that reproduce the MuJoCo-rendered AprilTag poses exactly (via a calibration_tags_root matching MuJoCo's calibration_tool body), so the behavior's per-id TF lookups resolve to what the camera actually sees. --- .../description/space_satellite_sim.xacro | 65 +++++++++---------- ...amera_pose_from_cal_tool_-_detect_tags.xml | 47 ++------------ 2 files changed, 36 insertions(+), 76 deletions(-) diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro index c306f2668..f24c7d8c1 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro @@ -559,45 +559,38 @@ - - - - - - - - - - + + + - - + + - - - - - - - - - - - - + + + + + - - - - - - - - - - - - + + + + + + + + + + + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml index 56f8e7b7d..a5e9b731b 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml @@ -7,7 +7,7 @@ - - - - Date: Thu, 9 Jul 2026 01:35:25 -0500 Subject: [PATCH 2/4] fix(space_satellite_sim_camera_cal): correct calibration tag frames for PnP Part B's tag frames added a spurious Rz(-90) parent rotation (calibration_tags_root) that displaced the tag positions from where they render in MuJoCo. CalibrateCameraPosePnp uses tag corners (orientation- and position-sensitive), so this made it drift to a different phantom camera pose at each waypoint, whereas the old center-only CalibrateCameraPose tolerated it. Parent calibration_tag_0/1/2 directly to calibration_tool_link at the proven tag-center offsets (same values the original centers-based calibration used); rpy="3.14159 0 0" keeps +Z out of the tag face for the corner PnP solve. Verified in sim: recovered optical pose is now consistent across all waypoints and the saved mount matches the URDF scene camera mount within noise. --- .../description/space_satellite_sim.xacro | 26 +++++++------------ 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro index f24c7d8c1..73e61661d 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro @@ -560,35 +560,29 @@ - - - - - - - + - + - + From 6f85559a897cfb58afe0b3dd1a966a46c9c49c56 Mon Sep 17 00:00:00 2001 From: Dan Wahl Date: Thu, 9 Jul 2026 10:48:28 -0500 Subject: [PATCH 3/4] feat(space_satellite_sim_camera_cal): add wrist-camera (eye-in-hand) PnP calibration Add a "Calibrate Wrist Camera" objective that calibrates the wrist-mounted camera against three static satellite AprilTags, reusing the same CalibrateCameraPosePnp subtree as the scene-camera (eye-to-hand) calibration. Only the frames, topics, tag prefix, and waypoints differ. - Scene/URDF: three tag36h11 markers (ids 3/4/5, distinct from the tool plate 0/1/2) as angled plates on the drum top, plus a wrist_camera_mount_link frame on the flange to receive the calibrated extrinsics. - Objectives: thread tag_frame_prefix, mounting_pose_parent_frame, and waypoint_N_name through the shared subtrees (defaults preserve the scene-camera path unchanged); new thin calibrate_wrist_camera.xml wires the wrist ports. - Per-waypoint fix: express each waypoint's optical pose in the mount-parent frame before averaging. For a moving (eye-in-hand) camera the optical pose in base_link differs per waypoint, so averaging it directly biased the result; the mount-parent frame is rigid to the camera, so the cross-waypoint average is unbiased. This is a no-op for the eye-to-hand case (parent == base_link). Verified in sim: recovers the declared wrist mount [0, -0.056, -0.058] Ry180 to sub-millimeter, with per-waypoint PnP reprojection RMS ~0.38 px. --- .../mujoco/assets/tag36_11_00003.png | 4 +- .../mujoco/assets/tag36_11_00004.png | 3 + .../mujoco/assets/tag36_11_00005.png | 3 + .../description/mujoco/gen3_7dof_assets.xml | 4 + .../description/mujoco/scene.xml | 43 ++++ .../description/space_satellite_sim.xacro | 35 ++++ .../calibrate_camera_-_detect_tags.xml | 21 +- .../calibrate_camera_at_waypoint_-_detect.xml | 17 +- .../objectives/calibrate_wrist_camera.xml | 33 ++++ ...amera_pose_from_cal_tool_-_detect_tags.xml | 3 +- .../waypoints/waypoints.yaml | 184 ++++++++++++++++++ 11 files changed, 341 insertions(+), 9 deletions(-) create mode 100644 src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00004.png create mode 100644 src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00005.png create mode 100644 src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_wrist_camera.xml diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00003.png b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00003.png index 994d7f42a..5fdf2ad30 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00003.png +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00003.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:6945a263eadff4ab15a426aad866c00dd9b8c756a10392d8b7e18c0dd0509443 -size 5480 +oid sha256:6c6ef757c14b3b961825df85f6cc196c1dea8bcc65ab4afbdb6d31b7855e20b7 +size 3259 diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00004.png b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00004.png new file mode 100644 index 000000000..a50d13f95 --- /dev/null +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00004.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a066e8a73f4c801c88f36409c95a6974ca61ecc4a20a755a79d4ada6a2ded355 +size 3265 diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00005.png b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00005.png new file mode 100644 index 000000000..8bcff332e --- /dev/null +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/assets/tag36_11_00005.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04c87a3eaf6310fc28b6470b9c2eee46373ac07c8abacee1a1d6e972d0dc8371 +size 3265 diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/gen3_7dof_assets.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/gen3_7dof_assets.xml index 5ce229ec8..c9c313906 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/gen3_7dof_assets.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/gen3_7dof_assets.xml @@ -204,10 +204,14 @@ + + + + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml index ad9574167..c7d839190 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml @@ -182,6 +182,49 @@ + + + + + + + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro index 73e61661d..374daa4a6 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/space_satellite_sim.xacro @@ -618,6 +618,41 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_-_detect_tags.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_-_detect_tags.xml index 2ebe3a0ec..fd08580a1 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_-_detect_tags.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_-_detect_tags.xml @@ -25,7 +25,9 @@ _collapsed="true" computed_poses_vector="{computed_poses_vector}" camera_optical_frame="{camera_optical_frame}" - waypoint_name="Camera Calibration 1" + tag_frame_prefix="{tag_frame_prefix}" + mounting_pose_parent_frame="{mounting_pose_parent_frame}" + waypoint_name="{waypoint_1_name}" camera_image_topic="{camera_image_topic}" camera_info_topic="{camera_info_topic}" /> @@ -34,7 +36,9 @@ _collapsed="true" computed_poses_vector="{computed_poses_vector}" camera_optical_frame="{camera_optical_frame}" - waypoint_name="Camera Calibration 2" + tag_frame_prefix="{tag_frame_prefix}" + mounting_pose_parent_frame="{mounting_pose_parent_frame}" + waypoint_name="{waypoint_2_name}" camera_image_topic="{camera_image_topic}" camera_info_topic="{camera_info_topic}" /> @@ -43,7 +47,9 @@ _collapsed="true" computed_poses_vector="{computed_poses_vector}" camera_optical_frame="{camera_optical_frame}" - waypoint_name="Camera Calibration 3" + tag_frame_prefix="{tag_frame_prefix}" + mounting_pose_parent_frame="{mounting_pose_parent_frame}" + waypoint_name="{waypoint_3_name}" camera_image_topic="{camera_image_topic}" camera_info_topic="{camera_info_topic}" /> @@ -52,7 +58,9 @@ _collapsed="true" computed_poses_vector="{computed_poses_vector}" camera_optical_frame="{camera_optical_frame}" - waypoint_name="Camera Calibration 4" + tag_frame_prefix="{tag_frame_prefix}" + mounting_pose_parent_frame="{mounting_pose_parent_frame}" + waypoint_name="{waypoint_4_name}" camera_image_topic="{camera_image_topic}" camera_info_topic="{camera_info_topic}" /> @@ -142,6 +150,11 @@ name="mounting_pose_save_file" default="calibrated_scene_camera_mounting_pose.txt" /> + + + + + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_at_waypoint_-_detect.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_at_waypoint_-_detect.xml index 423381e0d..2385e33a8 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_at_waypoint_-_detect.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/calibrate_camera_at_waypoint_-_detect.xml @@ -20,7 +20,7 @@ /> @@ -39,6 +39,7 @@ ID="Camera Pose from Cal Tool - Detect Tags" _collapsed="true" camera_optical_frame="{camera_optical_frame}" + tag_frame_prefix="{tag_frame_prefix}" computed_pose="{computed_pose}" camera_image_topic="{camera_image_topic}" camera_info_topic="{camera_info_topic}" @@ -68,10 +69,17 @@ marker_size="0.05" pose="{average_pose_stamped}" /> + @@ -88,6 +96,11 @@ name="camera_optical_frame" default="{camera_optical_frame}" /> + + + + + + + + + + + + + + + + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml index a5e9b731b..a8607df38 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/objectives/camera_pose_from_cal_tool_-_detect_tags.xml @@ -27,7 +27,7 @@ image="{image}" camera_info="{camera_info}" base_link="base_link" - tag_frame_prefix="calibration_tag_" + tag_frame_prefix="{tag_frame_prefix}" tag_size="0.051" tag_family="tag36h11" min_tags="3" @@ -55,6 +55,7 @@ name="camera_optical_frame" default="{camera_optical_frame}" /> + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml index 8e11bb5b3..de7a1bab1 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml @@ -306,3 +306,187 @@ twist: [] wrench: [] name: View Tag +- description: Wrist camera views the satellite tags (eye-in-hand cal) + favorite: false + joint_group_names: + - gripper + - manipulator + joint_state: + effort: [] + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + name: + - robotiq_85_left_knuckle_joint + - robotiq_85_left_finger_joint + - robotiq_85_right_finger_joint + - joint_1 + - joint_2 + - joint_3 + - joint_4 + - joint_5 + - joint_6 + - joint_7 + position: + - 0.0029407721601535414 + - 0.00014643648808031575 + - 0.0001464364880981395 + - -0.7466 + - 1.5857 + - -1.3846 + - -0.0286 + - -1.2893 + - -1.9375 + - 2.5092 + velocity: [] + multi_dof_joint_state: + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + joint_names: [] + transforms: [] + twist: [] + wrench: [] + name: Wrist Camera Calibration 1 +- description: Wrist camera views the satellite tags (eye-in-hand cal) + favorite: false + joint_group_names: + - gripper + - manipulator + joint_state: + effort: [] + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + name: + - robotiq_85_left_knuckle_joint + - robotiq_85_left_finger_joint + - robotiq_85_right_finger_joint + - joint_1 + - joint_2 + - joint_3 + - joint_4 + - joint_5 + - joint_6 + - joint_7 + position: + - 0.0029407721601535414 + - 0.00014643648808031575 + - 0.0001464364880981395 + - -0.9087 + - 1.5934 + - -1.4064 + - -0.527 + - -1.2613 + - -1.8057 + - 2.8234 + velocity: [] + multi_dof_joint_state: + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + joint_names: [] + transforms: [] + twist: [] + wrench: [] + name: Wrist Camera Calibration 2 +- description: Wrist camera views the satellite tags (eye-in-hand cal) + favorite: false + joint_group_names: + - gripper + - manipulator + joint_state: + effort: [] + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + name: + - robotiq_85_left_knuckle_joint + - robotiq_85_left_finger_joint + - robotiq_85_right_finger_joint + - joint_1 + - joint_2 + - joint_3 + - joint_4 + - joint_5 + - joint_6 + - joint_7 + position: + - 0.0029407721601535414 + - 0.00014643648808031575 + - 0.0001464364880981395 + - -0.7697 + - 1.5421 + - -1.3189 + - -0.0283 + - -1.3543 + - -1.9853 + - 2.5073 + velocity: [] + multi_dof_joint_state: + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + joint_names: [] + transforms: [] + twist: [] + wrench: [] + name: Wrist Camera Calibration 3 +- description: Wrist camera views the satellite tags (eye-in-hand cal) + favorite: false + joint_group_names: + - gripper + - manipulator + joint_state: + effort: [] + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + name: + - robotiq_85_left_knuckle_joint + - robotiq_85_left_finger_joint + - robotiq_85_right_finger_joint + - joint_1 + - joint_2 + - joint_3 + - joint_4 + - joint_5 + - joint_6 + - joint_7 + position: + - 0.0029407721601535414 + - 0.00014643648808031575 + - 0.0001464364880981395 + - -0.7479 + - 1.3555 + - -1.3291 + - -0.0284 + - -1.4385 + - -2.0564 + - 2.5514 + velocity: [] + multi_dof_joint_state: + header: + frame_id: '' + stamp: + nanosec: 0 + sec: 0 + joint_names: [] + transforms: [] + twist: [] + wrench: [] + name: Wrist Camera Calibration 4 From 6565293c4e51489ccb3c81197030d80b2d3e2125 Mon Sep 17 00:00:00 2001 From: Dan Wahl Date: Thu, 9 Jul 2026 13:25:40 -0500 Subject: [PATCH 4/4] refactor(space_satellite_sim_camera_cal): flat front-face wrist-cal tags + diverse waypoints Mount the three satellite AprilTags flat and coplanar on the servicer's front (+x) gold face, on the arm-base side of the docking mechanism, replacing the tilted drum-top placement (scene.xml geoms + matching sat_tag_* URDF frames; Ry90 so +Z exits the tag face). Regenerate the four wrist-camera waypoints for diverse, collision-free views that keep all three tags clear of the end effector. Sim-verified: every waypoint detects all 3 tags and the recovered wrist mount matches the URDF to ~0.6 mm / 0.02 deg. --- .../description/mujoco/scene.xml | 22 ++++---- .../description/space_satellite_sim.xacro | 15 ++--- .../waypoints/waypoints.yaml | 56 +++++++++---------- 3 files changed, 48 insertions(+), 45 deletions(-) diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml index c7d839190..21e98ca67 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/description/mujoco/scene.xml @@ -185,18 +185,20 @@ + ids 3/4/5, mounted flat and coplanar on the servicer's front (+x) gold face, on + the arm-base side of the central docking mechanism. rpy (Ry 90 deg) points +Z out + of the tag face (normal +x) for the corner-based PnP solve. Poses match the MuJoCo + satellite_tags geoms in description/mujoco/scene.xml. The "Calibrate Wrist Camera" + objective looks up frame 'sat_tag_'. --> - + - + - + diff --git a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml index de7a1bab1..5981fe3b3 100644 --- a/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml +++ b/src/moveit_pro_kinova_configs/space_satellite_sim_camera_cal/waypoints/waypoints.yaml @@ -333,13 +333,13 @@ - 0.0029407721601535414 - 0.00014643648808031575 - 0.0001464364880981395 - - -0.7466 - - 1.5857 - - -1.3846 - - -0.0286 - - -1.2893 - - -1.9375 - - 2.5092 + - -0.2415 + - 0.4937 + - -1.3862 + - -2.1321 + - -0.9097 + - -1.5143 + - -0.4093 velocity: [] multi_dof_joint_state: header: @@ -379,13 +379,13 @@ - 0.0029407721601535414 - 0.00014643648808031575 - 0.0001464364880981395 - - -0.9087 - - 1.5934 - - -1.4064 - - -0.527 - - -1.2613 - - -1.8057 - - 2.8234 + - -0.9033 + - 1.1219 + - -1.2598 + - -1.8335 + - -0.9293 + - -1.5142 + - -1.0942 velocity: [] multi_dof_joint_state: header: @@ -425,13 +425,13 @@ - 0.0029407721601535414 - 0.00014643648808031575 - 0.0001464364880981395 - - -0.7697 - - 1.5421 - - -1.3189 - - -0.0283 - - -1.3543 - - -1.9853 - - 2.5073 + - -1.2524 + - 1.0288 + - -0.8622 + - -2.4734 + - -1.1231 + - -1.2013 + - -0.5478 velocity: [] multi_dof_joint_state: header: @@ -471,13 +471,13 @@ - 0.0029407721601535414 - 0.00014643648808031575 - 0.0001464364880981395 - - -0.7479 - - 1.3555 - - -1.3291 - - -0.0284 - - -1.4385 - - -2.0564 - - 2.5514 + - -0.2284 + - 0.862 + - -1.5603 + - -1.5036 + - -0.93 + - -1.8527 + - -0.8653 velocity: [] multi_dof_joint_state: header: