graphics | Yifei Li

NeuralFluid: Nueral Fluidic System Design and Control with Differentiable Simulation

Wed, 25 Sep 2024 00:00:00 +0000

Citation

@misc{li2024neuralfluidicdesigncontrol,
title={Neural Fluidic System Design and Control with Differentiable Simulation},
author={Yifei Li and Yuchen Sun and Pingchuan Ma and Eftychios Sifakis and Tao Du and Bo Zhu and Wojciech Matusik},
year={2024},
eprint={2405.14903},
archivePrefix={arXiv},
primaryClass={physics.flu-dyn},
url={https://arxiv.org/abs/2405.14903},

}

Keywords

fluid simulation, fluid control, navier-stokes, differentiable simulation, physical simulation, neural fluid, neural control, graphics

DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation

Fri, 01 Dec 2023 00:00:00 +0000

Pipeline

DiffAvatar generates simulation-ready avatar assets from inputs obtained through a multi-view capture. Our pipeline initially preprocesses the 3D scan to segment the target garment and establish the initial pose and shape of the parametric body model. We employ a differentiable simulation framework to align our simulated garment with the segmented garment by jointly optimizing the garment’s design and material parameters

Garment 2D Representation

Garments can take on a wide range of 3D shapes when draped onto a body, due to factors such as changing pose and dynamics or wearer manipulations. Despite this large variation in configurations, garments are compactly represented by their 2D patterns (Fig. 3), which consist of the individual pieces of fabric that are sewn together to create the 3D clothing. Therefore, we represent clothing in 2D pattern space, which ensures developable meshes and manufacturable clothing. Virtual garments are modeled as triangle meshes, with their rest shape encoded in these 2D patterns. The rest shape is crucial for modeling the in-plane stretching and shearing behavior of different fabrics.

Control Cage Garment Shape Optimization

We propose a regularized differentiable cage formulation to effectively and robustly optimize for the 2D patterns of garments such that the simulated and draped 3D representation of the garment closely aligns with the scan.

Control Cage Pattern Representation:

While it is possible to directly optimize for the 2D pattern vertices p directly, this approach is highly non-regularized and can produce ill-shaped or even non-physical inverted rest shape geometries that cause simulators to fail.

A high number of optimization variables can also cause the optimization to get stuck in a local minimum (See our ablation study in Sec. 5.4). Additionally, directly optimizing for the 2D coordinates does not respect design constraints that are better represented in a limited subspace of reasonable designs. Therefore, we further regulate the optimization problem by selecting and optimizing a set of 2D control vertices ζ on the boundaries of the individual panels of the 2D pattern that directly deform and manipulate the underlying 2D patterns through control cages instead.

Control Cage Handle Selection: We use the geometric information of the 2D garment patterns to automatically identify control cage points, see the inset figure above. Our algorithm first extracts the boundary loop of the underlying mesh for each connected component representing a garment panel in the 2D garment pattern, then processes the boundary loop and marks a vertex as a control point if it lies on the convex hull of the pattern or when its local curvature exceeds a threshold (10° in our implementation).

Demo

Citation

@inproceedings{li2023diffavatar,
title={{DiffAvatar}: Simulation-Ready Garment Optimization with Differentiable Simulation},
author={Li, Yifei and Chen, Hsiao-yu and Larionov, Egor and Sarafianos, Nikolaos and Matusik, Wojciech and Stuyck, Tuur},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
eprint={2311.12194},
year = {2024}
}

Keywords

graphics, physical simulation, digital avatar, differentiable simulation, cloth simulation

A Method for Automatically Animating Children’s Drawings of the Human Figure

Wed, 01 Feb 2023 00:00:00 +0000

Code 💻

Follow our Github repository Animated Drawings

Demo

Try animating your own drawing Here!

Technical Posts

Dataset of 180,000 annotated drawings: A new, unique AI dataset for animating amateur drawings
Meta AI Technical post: Using AI to bring children’s drawings to life
Meta AI interview: Meet the researcher using AI to build children’s animations

Dataset

We have open sourced our dataset of ~180,000 annotated drawings, which you can explore for your own research.

Website

Go to here

Video

Citation

@article{10.1145/3592788,
author = {Smith, Harrison Jesse and Zheng, Qingyuan and Li, Yifei and Jain, Somya and Hodgins, Jessica K.},
title = {A Method for Animating Children’s Drawings of the Human Figure},
year = {2023},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3592788},
doi = {10.1145/3592788},
abstract = {Children’s drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children’s drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.},
note = {Just Accepted},
journal = {ACM Trans. Graph.},
month = {apr}
}

Acknowledgement

We would like to thank FAIR Interfaces, FAIR X, and other members of Meta who helped in the building and release of the demo

Keywords

graphics, animation, image manipulation, machine learning, drawings, children drawings, non-photorealistic rendering, line drawing, computer graphics, dataset, image dataset

Fluidic Topology Optimization with an Anisotropic Mixture Model

Sun, 18 Sep 2022 00:00:00 +0000

Demos

For details refer to the paper Sec 7. Applications. Below we show three design problems: Twister, Tree Diffuser and Circuit. We visualize the final design and the design domain for each of the problem.

Twister

Domain Size: 100x100x100

Tree Diffuser

Domain Size: 80x80x80

Circuit

Domain Size: 80x80x80

Demo Video

Presentation

Acknowledgement

Yifei Li acknowledges the emotional support from 🐱🐈‍ Yihui Li . Wojciech Matusik acknowledges the funding support from NSF IIS-2106962 and the Defense Advanced Research Projects Agency (DARPA) under grant No. FA8750-20-C-0075. Bo Zhu acknowledges the funding supports from NSF IIS-2106733. Eftychios Sifakis acknowledges the funding supports from NSF IIS-2106768, IIS-2008584, IIS-1763638.

Citation

@article{li2022anisotropicStokes,
author = {Li, Yifei and Du, Tao and Grama Srinivasan, Sangeetha and Wu, Kui and Zhu, Bo and Sifakis, Eftychios and Matusik, Wojciech},
title = {Fluidic Topology Optimization with an Anisotropic Mixture Model},
year = {2022},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3550454.3555429},
doi = {10.1145/3550454.3555429},
abstract = {Fluidic devices are crucial components in many industrial applications involving fluid mechanics. Computational design of a high-performance fluidic system faces multifaceted challenges regarding its geometric representation and physical accuracy. We present a novel topology optimization method to design fluidic devices in a Stokes flow context. Our approach is featured by its capability in accommodating a broad spectrum of boundary conditions at the solid-fluid interface. Our key contribution is an anisotropic and differentiable constitutive model that unifies the representation of different phases and boundary conditions in a Stokes model, enabling a topology optimization method that can synthesize novel structures with accurate boundary conditions from a background grid discretization. We demonstrate the efficacy of our approach by conducting several fluidic system design tasks with over two million design parameters.},
journal = {ACM Trans. Graph.},
month = {nov},
articleno = {239},
numpages = {14},
keywords = {Topology optimization, Stokes flow, computational design, fluidic system design}
}

Keywords

fluidic devices, topology optimization, flow device optimization, differentiable simulation, physics simulation, computational fabrication, fluid topology optimization, Stokes flow optimization

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact

Fri, 01 Apr 2022 00:00:00 +0000

Keywords

diffcloth, projective dynamics, jacobi solver, cloth simulation, differentiable simulation, physics simulation, inverse problems

Code 💻

Follow our Github repository Diffcloth

Demos

For details refer to the paper Sec 6. Applications.

Hat: Trajectory Optimization

Optimize manipulator end effector trajectories to move the hat onto the head.
1737 Dof, h=1/100s, 400 Timesteps, 18 Design Parameters

Sock: Trajectory Optimization

Optimize manipulator end effector trajectories to put on the sock.
3165 Dof, h=1/160s, 400 Timesteps, 36 Design Parameters

Hat-Controller: Closed-Loop Control

We train a generalizable closed-loop controller that can put on the hat from different initial positions.
1737 Dof, h=1/100s, 400 Timesteps, 117126 Design Parameters

Dress: Inverse Design

Optimize dress material parameters so that the spinning angle of the dress is 50 degrees.
10902 Dof, h=1/120s, 125 Timesteps, 2 Design Parameters

Sphere: System Identification

Optimize the fricitonal coefficient between the sphere and the cloth to match target trajectory.
1875 Dof, h=1/180s, 350 Timesteps, 1 Design Parameters

T-shirt: System Identification

Optimize wind model and cloth material parameters to match target trajectory.
4278 Dof, h=1/90s, 250 Timesteps, 6 Design Parameters

Demo

Presentation

Citation

@article{li2022diffcloth,
author = {Li, Yifei and Du, Tao and Wu, Kui and Xu, Jie and Matusik, Wojciech},
title = {DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact},
year = {2022},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3527660},
doi = {10.1145/3527660},
abstract = {Cloth simulation has wide applications in computer animation, garment design, and robot-assisted dressing. This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications. Our differentiable simulator extends a state-of-the-art cloth simulator based on Projective Dynamics (PD) and with dry frictional contact&nbsp;[Ly et&nbsp;al. 2020]},
note = {Just Accepted},
journal = {ACM Trans. Graph.},
month = {mar},
keywords = {cloth simulation, differentiable simulation, Projective Dynamics}
}

Acknowledgement

We thank Marco Renedo for his helpful discussions on the preconditioners, Junbang Liang for his help with running the baseline comparison code, and the anonymous reviewers for their helpful comments. This work was supported in part by the Defense Advanced Research Projects Agency (DARPA) under grant No. FA8750-20-C-0075.

JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints

Tue, 01 Mar 2022 00:00:00 +0000

Code 💻

Follow our Github repository JoinABLe

Motivation

Physical products are often complex assemblies combining a multitude of 3D parts modeled in computer-aided design (CAD) software. CAD designers build up these assemblies by aligning individual parts to one another using constraints called joints. JoinABLe is a learning-based method that assembles parts together to form joints. JoinABLe uses the weak supervision available in standard parametric CAD files without the help of object class labels or human guidance. Our results show that by making network predictions over a graph representation of solid models we can outperform multiple baseline methods with an accuracy (79.53%) that approaches human performance (80%).

Task

Given a pair of parts, we aim to create a parametric joint between them, such that the two parts are constrained relative to one another with the same joint axis and pose as defined by the ground truth.

Representation

JoinABLe takes as input a pair of parts in the B-Rep format. We represent each part as a graph derived from the B-Rep topology. Graph vertices are either B-Rep faces or edges and graph edges are defined by adjacency.

Architecture

Our overall architecture is shown in the figure below and consists of an encoder module that outputs per-vertex embeddings for each B-Rep face and edge in our graph representation of the input parts. Using these embeddings we can predict a joint axis and then search for joint pose parameters.

Citation

 @inproceedings{willis2022joinable,
title = {JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints},
author = {Willis, Karl DD and Jayaraman, Pradeep Kumar and Chu, Hang and Tian, Yunsheng and Li, Yifei and Grandi, Daniele and Sanghi, Aditya and Tran, Linh and Lambourne, Joseph G and Solar-Lezama, Armando and Matusik, Wojciech},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2022}
}