HoloPart – A Joint Open Source by HKU and VAST for Generating Fully Editable 3D Models of Complete Components

What is HoloPart?

HoloPart is a novel diffusion model introduced by the University of Hong Kong and the VAST team. It supports decomposing 3D objects into complete and editable semantic parts, even when some parts are occluded. HoloPart is based on a two-stage approach, utilizing local attention and global context attention mechanisms to ensure consistency in both the details of the parts and the overall shape. HoloPart significantly outperforms existing methods on the ABO and PartObjaverse-Tiny datasets, opening up new possibilities for downstream applications such as geometric editing, material editing, and animation production.

The main functions of HoloPart

• 3D Part Implicit Segmentation: Identify visible surface fragments, support the completion of occluded parts, and generate complete 3D parts.
• Geometric Super-resolution: Support super-resolution reconstruction of geometric details.
• Support for Downstream Applications: Support a variety of downstream applications, including geometric editing, material editing, animation production, and geometric processing.

The Technical Principle of HoloPart

• Two-stage method:
  ◦ Initial segmentation: Use existing 3D part segmentation techniques (e.g., SAMPart3D) to obtain initial, incomplete part fragments (surface fragments).
  ◦ Part completion: Use PartComp (a diffusion model-based network) to complete the fragments into a full 3D part.
• Diffusion Model: PartComp is a network based on the diffusion model. It captures the fine-grained geometric details of parts to ensure that the local features of parts are accurately restored. It utilizes the contextual information of the overall shape to ensure that the completed parts are geometrically and semantically consistent with the overall shape.
• Data Pre-training and Fine-tuning: Pre-train on large-scale complete 3D shape data using Variational Autoencoders (VAE) and diffusion models to learn a general representation of 3D shapes. Fine-tune the pre-trained model on limited part data to adapt to the part completion task, overcoming the challenge of data scarcity.

The project address of HoloPart

• Project Website: https://vast-ai-research.github.io/HoloPart/
• GitHub Repository: https://github.com/VAST-AI-Research/HoloPart
• Hugging Face Model Hub: https://huggingface.co/VAST-AI/HoloPart
• arXiv Technical Paper: https://arxiv.org/pdf/2504.07943
• Online Demo Experience: https://huggingface.co/spaces/VAST-AI/HoloPart

Application scenarios of HoloPart

• Geometric Editing: Modify the size, shape, and position of parts to meet design requirements.
• Material Assignment: Assign different materials to parts to enhance the visual effect.
• Animation Production: Enable parts to move independently, such as rotating wheels, to improve animation flexibility.
• Geometric Processing: Optimize the mesh division of parts to enhance the model quality.
• Data Generation: Generate high-quality part data for 3D model training to enrich creative materials.