first commit#516
Open
ZhangChuye wants to merge 57 commits into
Open
Conversation
…creating repo by time now
Lightweight alternative (~50K params) to ResNet18 (~11M) for cases where the image input carries little information — e.g. 64x64 black aria placeholders used while iterating on proprio-only training runs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
For 0418 rotatebox runs the image input is a 64x64 black placeholder, so the original ResNet18 + 16-block/d=256 trunk was heavily over-parameterized relative to the ~11k-frame dataset. Both vanilla and hierarchical-p1 april6d model configs now use: - TinyCNN encoder (output_dim=192) - trunk: embed_dim 256->192, num_blocks 16->8 - stems / cross_attn modality_embed_dim / head cond_dim: 256->192 - drop ColorJitter (no-op on a black image) Total params drop from ~27M to ~9.5M. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add --run-suffix flag so multiple training configs can run against the same converted LeRobot dataset (suffix only affects TRAIN_RUN_NAME). - Sanitize basenames before using them as dataset/job-name suffixes. - Quote name + folder_path Hydra overrides so paths with special chars don't break the submit script. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reduce default --black-image placeholder from 128x128 to 64x64 to match the resolution used by downstream image-free training runs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… scripts - submit_train_only.sbatch: train-only sbatch that takes an already-built LeRobot dataset (TRAIN_CONFIG / TRAIN_RUN_NAME / TRAIN_DATASET env vars). - submit_hdf5_to_train.sbatch: HDF5 -> LeRobot -> train pipeline. - submit_csv_sew_mink_train.sbatch: CSV -> HDF5 (SEW or Mink) -> LeRobot -> train. - sew_mink_batch_workflow.sh: dispatcher for the CSV-driven pipeline. - hierarchical_batch_workflow.sh: launches p1 + p2 hierarchical variants for each (task, solver) pair whose LeRobot dataset already exists. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- egomimic/scripts/sew_process/csv_to_hdf5_sew_custom.py: thin wrapper around the upstream SEW converter that lets us override solver __init__ kwargs at the call site instead of editing the upstream tree. - configs/sew_solver_mobile_base_user.yaml: mobile-base solver config consumed via RBY1SEWSolver(mobile_base_config_path=...). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- subset_robomimic_hdf5.py: copy a slice of demo groups out of a robomimic HDF5 to produce a smaller dataset. - RBY1_SEW_img_HDF5_config.json: robomimic_hd5.py config for SEW data that includes aria_image (vs. the lowdim variant). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ent configs - experiments/hierarchical/rotatebox_p1_april6d: hierarchical p1 variant for the 0418 rotatebox task using 6D april_tag (xyz + rotation). - experiments/rby_no_mobile/rotatebox_vanilla_april6d: vanilla variant of the same task; same obs/action schematic, different head. - experiments/hierarchical/realworld_0518_p1_masked_attn: hierarchical p1 for realworld 0518 (basket / push_cart, mink_eef / sew); dataset folder overridden per-job via CLI. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Skill that periodically checks a set of SLURM job IDs and alerts on terminal (non-requeue-able) failures. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a 3-D world-frame (x, y, theta) proprio derived by integrating the per-step base-frame action deltas (actions.joint_base_torso_head_arm_hand[..., 0:3]) forward from (0, 0, 0) at episode start. This gives the policy a dead-reckoning spatial anchor so april-tag-based mobile-manip tasks survive partial observation when the tag drops out, without paying for true obs history. - egoengine_lerobot_extract_base_pose.py: out-of-place post-processor in the style of extract_arm_hand. Idempotent (skips when output already has the new feature), symlinks videos by default to avoid duplicating disk. - visualize_base_pose.py: top-down + 3D triad plot with --overlay-recomputed sanity check; --all-episodes mode dumps one PNG per episode to a folder. - .gitignore: ignore the visualizer's --all-episodes output dir and shell session log noise. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…roprio Inherits experiments/hierarchical/rotatebox_p1_april6d (same april6d hierarchical decoder + data pipeline) and adds two things at the global package scope so they deep-merge into the parent: - data_schematic.schematic_dict.rby1.base_pose_integrated -> obs.base_pose_integrated - model.robomimic_model.stem_specs.rby1.state_base_pose_integrated: 3-D MLPPolicyStem with noise_std=0.10 + proprio_dropout_p=0.20 to guard against the train/inference distribution mismatch (GT base deltas at train, policy-predicted deltas at inference -> drift). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reuses an existing post-Step-2 LeRobot dataset and inserts Step 2b
(extract base_pose_integrated) before training; both Step 2b and the
training trainer point at <dataset>_basepose. Layout mirrors the
existing SEW hierarchical sbatch+workflow pair so the two coexist
without conflict.
- submit_sew_basepose_training.sbatch: idempotent Step 2b with mkdir
lock so concurrent variants share one extractor run; trains via
push_cart_p1_april6d_basepose with folder_path overridden to the
*_basepose dataset.
- sew_basepose_batch_workflow.sh: dispatcher across the two push_cart
datasets (0524_ye_push_cart_new_{ours,mink_eef_only}) with the same
variant filtering UX as the sew_hierarchical workflow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.