PartCrafter – An AI-powered 3D generation model that supports joint generation of multiple components

What is PartCrafter?

PartCrafter is an advanced 3D generation model that can generate multiple semantically distinct and geometrically diverse 3D meshes from a single RGB image. It combines latent representations of individual 3D parts and uses a hierarchical attention mechanism to exchange information both within and across parts, ensuring global consistency in the generated 3D model. Built upon a pretrained 3D mesh diffusion transformer (DiT), PartCrafter inherits its weights, encoder, and decoder, further enhancing its generation capability. It supports end-to-end part-aware generation and can handle scenarios ranging from single objects to complex multi-object scenes. Experiments show that the model outperforms existing methods in generating decomposable 3D meshes and can accurately reconstruct parts that are not directly visible in the input image.

Key Features of PartCrafter

• Joint Multi-Part Generation:
  From a single image, PartCrafter can generate multiple objects and their parts, each with clear semantics and unique geometry.

• Part-Aware Generation:
  The model composes each 3D part through latent space representations and uses a hierarchical attention mechanism to facilitate structured information flow within and between parts.

• End-to-End Generation:
  PartCrafter supports end-to-end generation from a single image to complex scenes with multiple objects. With just one input image, it can generate an entire 3D scene, including distinct parts and objects.

• Part-Level Editing:
  The generated 3D models support fine-grained, part-level editing. Users can independently adjust the position, rotation, and scale of each part, enabling more flexible 3D design and creativity.

• Complex Scene Modeling:
  Suitable for modeling complex scenes such as mechanical assemblies and architectural structures, PartCrafter ensures that multiple parts are generated in coordination, creating more realistic and intricate 3D scenes.

• Support from Pretrained Models:
  Built upon a pretrained 3D mesh diffusion transformer (DiT), PartCrafter inherits its encoder, decoder, and weights, which boosts the model’s performance and generation efficiency.

Technical Principles of PartCrafter

• Compositional Latent Space:
  Each 3D part is represented by a set of disentangled latent tokens, allowing individual parts to evolve independently during generation while retaining part-level detail.

• Hierarchical Attention Mechanism:
  This mechanism enables structured information exchange both within single parts and among multiple parts, ensuring global consistency throughout the generation process.

• Pretrained 3D Mesh Diffusion Transformer (DiT):
  PartCrafter inherits the encoder, decoder, and pretrained weights from a 3D mesh DiT model, enhancing its capability to generate high-quality 3D shapes.

• End-to-End Part-Aware Generation:
  Given a single image, PartCrafter can simultaneously denoise and generate multiple 3D parts, achieving comprehensive part-aware generation across single objects and multi-object scenes.

Project Links for PartCrafter

• Official Website: https://wgsxm.github.io/projects/partcrafter/

• GitHub Repository: https://github.com/wgsxm/PartCrafter

• arXiv Technical Paper: https://arxiv.org/pdf/2506.05573

Application Scenarios for PartCrafter

• Game Development:
  PartCrafter can quickly generate high-quality 3D assets such as characters, props, and environments. Developers can use a single image or text description to rapidly produce large volumes of 3D models, speeding up the game development process.

• Architecture and Interior Design:
  In the fields of architecture and interior design, PartCrafter helps quickly produce 3D building and decor models. Designers can upload architectural drawings or concept sketches to generate 3D models for visual presentations, making it easier for clients to understand the design intent.

• Film Production:
  With its ultra-detailed geometric modeling and multi-view input support for PBR model generation, PartCrafter provides high-quality 3D assets for film production teams.

• Education:
  In education, PartCrafter can be used to visualize complex scientific concepts, such as molecular structures or human anatomy, enhancing students’ understanding and learning experience.

• Augmented Reality (AR) and Virtual Reality (VR):
  The realistic 3D models generated by PartCrafter are ideal for building immersive AR/VR experiences. Models generated from a single image can be directly integrated into virtual environments, enriching user interaction.