HybridSDF + PartSDF

Test the fusion of primitives from HybridSDF into PartSDF architecture. Forked from PartSDF.

See:
"HybridSDF: Combining Deep Implicit Shapes and Geometric Primitives for 3D Shape Representation and Manipulation". [Paper]
"PartSDF: Part-Based Implicit Neural Representation for Composite 3D Shape Parametrization and Optimization". [Paper]

Project status

Implemented:

• geometric + assisted parts (training, reconstruction, evaluation, meshing, visualization)
• ParamEncoder, L_geo / L_ga, Parts.PrimitiveSupport (where they apply), Parts.SplitScale (encode the aspect ratio only)
• inert without PrimitiveTypes → leads to original PartSDF

Working:

• reconstruction quality on par with all-generic, typed parts exact in the output
• neural parts adapt to the primitives through part_conv1d (e.g. wheel wells follow the wheels)

Not working: manipulation too entangled

• editing one part's pose deforms the others (bigger wheels → longer car body); not the case with all-generic
• ruled out: encoded scale in the latent (SplitScale)
• potentially ruled out: support of L_geo/L_ga (PrimitiveSupport)

Next step: training-time pose jitter (Parts.PoseJitter)

• jitter geometric/assisted poses during training, mask the GT-anchored recon terms on jittered rows
• design + verification plan: .claude/plans/pose-jitter.md

File structure

This repository is organized as follows:

PartSDF/
├── experiments/              <- Experimental config and results
│   └── template/
│       ├── specs.json        <- Template experimental config
│       └── specs_hybrid.json <- ... with geometric parts (see below)
├── notebooks/                <- Jupyter notebooks for visualizing results
├── scripts/                  <- Python scripts
└── src/                      <- Source code

See below for the data files structure.

How to use

Note: the scripts can be launched from the main directory with:

python3 scripts/script.py [--option [VALUE]]

Please, refer to the script itself or use the --help options for details regarding its options.

Training

To train a model, first create an experiment directory, e.g., under experiments, then copy there the template specifications experiments/template/specs.json, and adapt it as needed (such as data and part paths). Then, launch the training with:

python3 scripts/train.py <experiments/expdir>

The models, latents, and poses will be saved in <experiments/expdir>.

Reconstruction and evaluation

After training, reconstruct the test shapes with:

python3 scripts/reconstruct.py <experiments/expdir> --partsdf --parts --test

The reconstructions (meshes, parts, latents, and poses) will be saved in <experiments/expdir>/reconstruction/<epoch>_parts/.

Once that is done, you can evaluate them by running:

python3 scripts/evaluate.py <experiments/expdir> --parts --test

The metric values will be saved per-shape under <experiments/expdir>/evaluation/<epoch>_parts/.

Part manipulation

To check how manipulable a trained model is (cars only, with 5 or 6 parts), run:

python3 scripts/manipulate.py <experiments/expdir>

It takes one train and one test shape (--train-id / --test-id, default to the first of each split) and edits their wheels' poses: their radius, then their forward/backward position, then the track width, with the body parts' scale following it (the body, plus the back for the 6-part DrivAerNet++ layout). The latents are kept fixed, so the rest of the body's adaptation comes from the model itself. Each animation is saved as a GIF in <experiments/expdir>/manipulation/<epoch>_parts/. The test shape's latent is reused from its reconstruction if it exists, and optimized (with the fitted poses kept fixed) otherwise.

Geometric parts

By default, the primitive fitted to each part is only used as its coordinate frame, and every part is fully neural. Parts.PrimitiveTypes (a list of one string per part, in experiments/template/specs_hybrid.json) changes that, giving each part one of three types:

Type	In the output	Latent	Supervision
"generic"	the network's prediction	free	the part loss on its labelled points
"geometric:cuboid", "geometric:cylinder"	the analytic primitive	none (encoded parameters only)	L_geo, pulling its prediction onto the primitive
"assisted:cuboid", "assisted:cylinder"	the network's prediction	free + encoded parameters	the part loss and L_ga, a weaker L_geo

The related keys, all under Parts, are ParamEncoder (the encoder of the primitive parameters), GeometricLoss / GeometricLambda and AssistedLoss / AssistedLambda. Leaving PrimitiveTypes to null gives the original all-generic behavior.

The encoded parameters are the primitive's scale and type only, not its rotation and translation: an SDF is equivariant to rigid transforms and the query points are already inverse-transformed into the part's frame, so conditioning on the absolute pose would only make a part's geometry drift when that pose is edited. The scale is the exception, as it is divided out of the local coordinates while the network's output is in world units.

---

Based on the https://github.com/ntalabot/base-inr template.