Gen-Simulation-cooperate

Imported from zz2ethankai/Gen-Simulation-cooperate (AGENTS.md). Install upstream with npx skills add zz2ethankai/Gen-Simulation-cooperate. Copyright stays with the author.

InterndataEngine Working Notes

Scene 4 Mobile Manipulation

• Scene-4 task positions are floor-center relative. Convert to world/layout XY with world_x = floor_center_x + x and world_y = floor_center_y + y; do not add another reference frame field.
• Keep generated scene-4 skill graphs short. The expected basic-task shape is usually five skills: nav_to_pick, pick_*, nav_to_place, place_*, home_*.
• Navigation uses the ROS-free local A* and waypoint controller; task YAML must not reintroduce external Nav2/ROS control.
• Do not fix base navigation by editing dummy/mobile_support nodes; they are not on the effective mobile-base control path.
• Use generated nav overlays and reports as design evidence only. A collision-free overlay does not prove 4WIS stability or arm reachability.

Validation Workflow

• Do not use a personal host Python path as a repository default. Use .venv/bin/python for host-only utilities, use the Isaac Docker image for simulator code, and pass any specialized interpreter explicitly to the helper that needs it.
• Start Isaac through scripts/docker/up_simbox_isaac.sh or the validation wrapper that calls it; avoid ad-hoc container startup.
• For real validation, prefer the Scene-4 validation wrapper when present, or scripts/docker/run_simbox_task.sh for one task. Judge success from the validation summary, per-task logs, and skill snapshots.
• A successful run needs Task is successful, mode=plan_with_render and no [LmdbLogger] Episode failed; a video or missing traceback is not enough.
• Keep emit_obs_on_failure disabled for strict validation. Placeholder observations can hide retry/reset behavior.
• Stop and inspect the first failure when using --stop-on-failure; use output/local_navigation/skills/* snapshots before changing logic.

Reset And Randomization

• Fixed rigid objects should normally use apply_randomization: false unless their reload/reset path has been verified.
• Retry reset should prefer restoring existing rigid-object pose, scale, visibility, and velocity when the USD path is unchanged. Repeated delete/recreate can race USD loading and produce invalid null prims.
• Fixed-object reset must run in the normal randomization() layout reset paths, not only in reset_after_failed_generation(), otherwise later retries can plan against stale object states.
• Randomized rigid-object USDs may use a different rigid-body child than the source asset. Validate the configured child and fall back to the first rigid body under the loaded object root.
• Grasp annotation paths should be resolved from the selected USD directory plus npy_name; do not derive them only by replacing Aligned_obj.usd.

Navigation Debugging

• Navigation points must balance obstacle clearance, 4WIS dynamic stability, and arm reachability. A point that is valid on the 2D map can still be too close to counters or force a bad lateral approach.
• If navigation reports bridge_aborted, compare world_xy, nav_xy, world_dist, nav_dist, yaw error, and the bridge command history before changing task points.
• If the base state becomes invalid, inspect roll/pitch, wheel/steering commands, and restore-after-navigation traces. Do not assume local A* path planning is the root cause.

Pick Skill Debugging

• Separate candidate generation from execution. Use pick_plan_snapshot.json to check candidates and pick_execution_trace.json to check whether the object actually moved.
• If later retries have success_found: true in pick_plan_snapshot.json, the old "no grasp candidates after first attempt" issue is not the active failure.
• To prove pick success, check that object z increases during close/post-grasp and that no pick_runtime_failure_snapshot.json is produced. Do not mislabel downstream place failure as pick failure.
• Command transitions in pick_execution_trace.json are the best timing evidence: pre-grasp/open should finish before close_gripper; attach_obj should occur after close.
• For apple-like top grasps, prefer physical ranking from actual candidate geometry and execution traces over loosening YAML filters blindly.
• post_grasp_offset_min and post_grasp_offset_max alone control planned post-grasp lift height. Do not add hidden caps that override these values.
• lift_th is only a pick success threshold. If absent, it defaults to 0.0 and disables the lift-height success check; it should not affect the post-grasp motion target.

Place Skill Debugging

• Place failures should be diagnosed from place_success_check_snapshot.json; inspect success_mode, target object, bbox limits, margin, and final object XY/Z.
• If a task name says "tray" but the place skill targets sink, trust the skill objects in YAML/runtime snapshots when diagnosing behavior.
• Debug artifact writing must never break an episode. Convert NumPy values and USD/Gf vector types such as Vec3d into JSON-safe scalars/lists before json.dump.
• When success_mode: xybbox fails, compare pick_xy against valid_xy_min/max; a small outside-bbox error is a placement target/settling issue, not a pick failure.

Scene-4 Task Skills and Positions Reference

• kitchen_apple_to_tray is the verified reference task; its pick/place parameters are treated as the scene-4 baseline. Do not modify this task unless a new validation run explicitly requires it.
• All scene-4 basic tasks use the 5-skill graph pattern (≤ 8 skills):
  1. base: navigate (id: nav_to_pick, depends_on: [])
  2. <arm>: pick (id: pick_<pick_object>, depends_on: [nav_to_pick])
  3. base: navigate (id: nav_to_place, depends_on: [pick_<pick_object>])
  4. <arm>: place (id: place_<pick_object>, depends_on: [nav_to_place])
  5. <arm>: heuristic__skill (id: home_<arm>, mode: home, depends_on: [place_<pick_object>])
• Navigation skills do not expose a heading-controller enable/disable option.
• Navigation tolerances: xy_goal_tolerance: 0.1, yaw_goal_tolerance: 0.1.
• Positions are floor-center relative (floor_center_layout_xy varies per room). Convert to world/layout XY with world_x = floor_center_x + x and world_y = floor_center_y + y.
• Positions and object-to-arm mappings for all 20 tasks are canonically stored in output/scene4_nav_skill_generation/scene4_nav_skill_generation_summary.json. When updating a task, read from that summary rather than recomputing from the obstacle map.

Task inventory (generated from nav-skill summary)

Pick/place parameter templates

• Left-arm pick (copied from kitchen_apple_to_tray):
  • filter_x_dir: [forward, 90], filter_z_dir: [downward, 140]
  • pre_grasp_offset: 0.12, post_grasp_offset_min: 0.26, post_grasp_offset_max: 0.28
  • lift_th: 0.02, gripper_change_steps: 20, t_eps: 0.025, o_eps: 1, process_valid: true
• Right-arm pick (room-level baseline with tuned offsets):
  • filter_y_dir: [forward, 60], filter_z_dir: [downward, 150]
  • pre_grasp_offset: 0.12, post_grasp_offset_min: 0.26, post_grasp_offset_max: 0.28
  • lift_th: 0.02, gripper_change_steps: 20, t_eps: 0.025, o_eps: 1, process_valid: true
• Place (generic, used for both arms):
  • position_constraint: object, success_mode: xybbox
  • filter_x_dir: [backward, 110], filter_y_dir: [downward, 120], filter_z_dir: [forward, 70]
  • x_ratio_range: [0.35, 0.65], y_ratio_range: [0.35, 0.65]
  • pre_place_z_offset: 0.1, place_z_offset: 0.1, gripper_change_steps: 20

Code And Git Hygiene

• Prefer existing SimBox helpers and local patterns over new abstractions.
• Keep fixes scoped: do not change YAML to mask a code bug, and do not change code when the user explicitly asks for YAML-only repair.
• Before risky rollback or checkpoint work, verify git status --short, branch, and git log -1 --oneline.
• When saving a successful state, create a clear checkpoint commit and verify the final status. Include generated assets only when the user explicitly asks.

Repository-wide Editing Contract

• Read .agents/notes/2026-08-25-environment-architecture-coding-standards.md before G1, locomotion, motion-planning, or data-generation work.
• Keep changes surgical. Every changed line must trace to the approved task; do not refactor adjacent modules or add speculative compatibility layers.
• Prefer the smallest existing extension point. Keep configuration and planning data immutable where practical; confine required simulator and controller mutation to explicit lifecycle methods.
• Validate external configuration, asset paths, robot joint maps, tensor/action shapes, and recorded episode schemas at their boundaries. Fail with actionable context; do not silently swallow runtime failures.
• Match repository Python style: Black and isort with line length 120, then flake8 and pylint. Comments should explain constraints or intent rather than restate code.
• Preserve runtime path classes: /workspace, /isaac-sim, and /opt/curobo are container contracts; asset/task paths should be repository-relative. Do not bulk-replace absolute paths in documentation or isolated legacy tools.
• Keep generated logs, plans, patches, and scratch data under the project-owned .agents/, output/, or tmp/<task>/ locations. Never commit credentials or machine-private values.
• Do not treat the existing Galaxea Genie1 integration as Unitree G1. Add Unitree G1 through a separate robot definition/controller boundary unless verified shared code is genuinely robot-agnostic.
• Keep humanoid gait generation separate from wheeled-base Navigate. Reuse orchestration and recording interfaces, but do not route walking through the 4WIS waypoint controller.
• For a new behavior, first add a focused regression/unit test where feasible, then run targeted static checks and the smallest Docker/GPU smoke that proves the changed boundary.
• A successful import, checkpoint load, command receipt, or rendered video is not task success. For generated episodes, require the workflow success marker and absence of LMDB episode failure.