DJI_Action4_poseetimation

Imported from yufeila/DJI_Action4_poseetimation (AGENTS.md). Install upstream with npx skills add yufeila/DJI_Action4_poseetimation. Copyright stays with the author.

AGENTS.md

Project mission

This project is an implementation-oriented assessment task. The goal is to reproduce a practical algorithm pipeline for instance-level object pose estimation on DJI Action 4 cameras appearing in RGB images/videos.

The expected high-level pipeline is:

1. Obtain or build a 3D model of the target object from images.
2. Use HCCEPose Blender scripts to render synthetic training data.
3. Implement an instance-level object pose estimation network.
4. Predict 8 corner points as heatmaps from a single RGB image.
5. Train on rendered data and test on target data.

The priority is not novelty at the beginning. The priority is:

• understand the pipeline correctly,
• build a minimal working baseline,
• verify each stage carefully,
• document assumptions and failures clearly.

Working style

This project follows a "copy and understand before innovate" philosophy.

Always prefer:

1. understanding the existing pipeline,
2. implementing a minimal reproducible version,
3. validating with small experiments,
4. then improving modules step by step.

Do not introduce large architectural changes before a baseline is working.

Communication expectations

The lab encourages frequent discussion. If something is ambiguous, blocked, or inconsistent, surface the issue clearly.

Weekly progress should be summarized every Monday with:

• what was finished,
• what failed,
• why it failed,
• next-step plan,
• help needed.

Coding rules

• All code comments must be written in English.
• For tensors, arrays, feature maps, and data flow, comments must include shape information whenever helpful.
• When adding or modifying a module, explain input shape, output shape, and key intermediate shapes.
• Prefer small, reviewable diffs.
• Avoid silent magic constants. Name them and explain them.
• Do not remove existing functionality unless necessary.
• Do not add dependencies unless clearly justified.

Experiment rules

Before launching large experiments:

• first run a tiny sanity check,
• verify dataset loading,
• verify annotation format,
• verify output tensor shape,
• verify loss decreases on a tiny subset,
• verify visualization of predicted heatmaps.

For every experiment, record:

• config,
• data split,
• backbone,
• input resolution,
• loss definition,
• training command,
• key metrics,
• qualitative failure cases.

Reporting rules

All reports intended for the user should be written in Chinese. Code comments remain in English.

A good report should include:

• task objective,
• current implementation status,
• experiment settings,
• observed results,
• current problems,
• next-step plan.

Planning requirements

For any non-trivial task, read PLANS.md first and produce a short execution plan before coding.

Non-trivial tasks include:

• adding a new model branch,
• changing label generation,
• changing loss functions,
• changing data pipeline,
• adding evaluation scripts,
• refactoring training code.

Domain-specific guidance

This task is about instance-level object pose estimation using 8 corner points.

Likely input:

• single-object or single-object image tensor, shape like [B, 3, H, W]

Likely output:

• 8-channel heatmap tensor, shape like [B, 8, H_out, W_out]

Typical implementation focus:

• data generation / annotation consistency,
• heatmap target construction,
• backbone + decoder design,
• training stability,
• corner extraction post-processing,
• qualitative visualization.

Always verify that the implemented target matches the actual task: predicting 8 projected box corners for the target object instance.

For full-image multi-instance prediction, HCCEPose should be treated as the primary technical reference for handling multiple target objects in the same frame. BoxDreamer is a secondary reference and should mainly be used for network-design inspiration where applicable.

Preferred workflow

1. Clarify the current sub-problem.
2. State input/output and tensor shapes.
3. Check existing code and reusable components.
4. Propose a minimal implementation.
5. Implement.
6. Run a sanity check.
7. Summarize results and open issues.

References

Potentially relevant resources may include:

• HCCEPose
• BoxDreamer
• project-specific paper reading notes
• AI coding workflow notes

Do not assume these references are implemented exactly as-is. Use them as guidance, and explicitly note any deviation.

Non-negotiable rules

• All code comments must be in English.
• For tensors and data flow, comments should include shape changes whenever helpful.
• All reports intended for review must be written in Chinese.
• Before implementing a new module, always specify input, output, loss, and validation method.
• Every experiment must include at least one qualitative visualization.