DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

Fri, 01 Dec 2023 00:00:00 +0000

Pipeline

DiffAvatar generates simulation-ready avatar assets from inputs obtained through a multi-view capture. Our pipeline initially preprocesses the 3D scan to segment the target garment and establish the initial pose and shape of the parametric body model. We employ a differentiable simulation framework to align our simulated garment with the segmented garment by jointly optimizing the garment’s design and material parameters

Garment 2D Representation

Garments can take on a wide range of 3D shapes when draped onto a body, due to factors such as changing pose and dynamics or wearer manipulations. Despite this large variation in configurations, garments are compactly represented by their 2D patterns (Fig. 3), which consist of the individual pieces of fabric that are sewn together to create the 3D clothing. Therefore, we represent clothing in 2D pattern space, which ensures developable meshes and manufacturable clothing. Virtual garments are modeled as triangle meshes, with their rest shape encoded in these 2D patterns. The rest shape is crucial for modeling the in-plane stretching and shearing behavior of different fabrics.

Control Cage Garment Shape Optimization

We propose a regularized differentiable cage formulation to effectively and robustly optimize for the 2D patterns of garments such that the simulated and draped 3D representation of the garment closely aligns with the scan.

Control Cage Pattern Representation:

While it is possible to directly optimize for the 2D pattern vertices p directly, this approach is highly non-regularized and can produce ill-shaped or even non-physical inverted rest shape geometries that cause simulators to fail.

A high number of optimization variables can also cause the optimization to get stuck in a local minimum (See our ablation study in Sec. 5.4). Additionally, directly optimizing for the 2D coordinates does not respect design constraints that are better represented in a limited subspace of reasonable designs. Therefore, we further regulate the optimization problem by selecting and optimizing a set of 2D control vertices ζ on the boundaries of the individual panels of the 2D pattern that directly deform and manipulate the underlying 2D patterns through control cages instead.

Control Cage Handle Selection: We use the geometric information of the 2D garment patterns to automatically identify control cage points, see the inset figure above. Our algorithm first extracts the boundary loop of the underlying mesh for each connected component representing a garment panel in the 2D garment pattern, then processes the boundary loop and marks a vertex as a control point if it lies on the convex hull of the pattern or when its local curvature exceeds a threshold (10° in our implementation).

Demo

Citation

@inproceedings{li2023diffavatar,
title={{DiffAvatar}: Simulation-Ready Garment Optimization with Differentiable Simulation},
author={Li, Yifei and Chen, Hsiao-yu and Larionov, Egor and Sarafianos, Nikolaos and Matusik, Wojciech and Stuyck, Tuur},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
eprint={2311.12194},
year = {2024}
}

Keywords

graphics, physical simulation, digital avatar, differentiable simulation, cloth simulation

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact

Fri, 01 Apr 2022 00:00:00 +0000

Keywords

diffcloth, projective dynamics, jacobi solver, cloth simulation, differentiable simulation, physics simulation, inverse problems

Code 💻

Follow our Github repository Diffcloth

Demos

For details refer to the paper Sec 6. Applications.

Hat: Trajectory Optimization

Optimize manipulator end effector trajectories to move the hat onto the head.
1737 Dof, h=1/100s, 400 Timesteps, 18 Design Parameters

Sock: Trajectory Optimization

Optimize manipulator end effector trajectories to put on the sock.
3165 Dof, h=1/160s, 400 Timesteps, 36 Design Parameters

Hat-Controller: Closed-Loop Control

We train a generalizable closed-loop controller that can put on the hat from different initial positions.
1737 Dof, h=1/100s, 400 Timesteps, 117126 Design Parameters

Dress: Inverse Design

Optimize dress material parameters so that the spinning angle of the dress is 50 degrees.
10902 Dof, h=1/120s, 125 Timesteps, 2 Design Parameters

Sphere: System Identification

Optimize the fricitonal coefficient between the sphere and the cloth to match target trajectory.
1875 Dof, h=1/180s, 350 Timesteps, 1 Design Parameters

T-shirt: System Identification

Optimize wind model and cloth material parameters to match target trajectory.
4278 Dof, h=1/90s, 250 Timesteps, 6 Design Parameters

Demo

Presentation

Citation

@article{li2022diffcloth,
author = {Li, Yifei and Du, Tao and Wu, Kui and Xu, Jie and Matusik, Wojciech},
title = {DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact},
year = {2022},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3527660},
doi = {10.1145/3527660},
abstract = {Cloth simulation has wide applications in computer animation, garment design, and robot-assisted dressing. This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications. Our differentiable simulator extends a state-of-the-art cloth simulator based on Projective Dynamics (PD) and with dry frictional contact&nbsp;[Ly et&nbsp;al. 2020]},
note = {Just Accepted},
journal = {ACM Trans. Graph.},
month = {mar},
keywords = {cloth simulation, differentiable simulation, Projective Dynamics}
}

Acknowledgement

We thank Marco Renedo for his helpful discussions on the preconditioners, Junbang Liang for his help with running the baseline comparison code, and the anonymous reviewers for their helpful comments. This work was supported in part by the Defense Advanced Research Projects Agency (DARPA) under grant No. FA8750-20-C-0075.

cloth simulation | Yifei Li

DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

Pipeline

Garment 2D Representation

Control Cage Garment Shape Optimization

Demo

Citation

Keywords

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact

Keywords

Code 💻

Demos

Hat: Trajectory Optimization

Sock: Trajectory Optimization

Hat-Controller: Closed-Loop Control

Dress: Inverse Design

Sphere: System Identification

T-shirt: System Identification

Demo

Presentation

Citation

Acknowledgement