A Method for Automatically Animating Children’s Drawings of the Human Figure

Wed, 01 Feb 2023 00:00:00 +0000

Code 💻

Follow our Github repository Animated Drawings

Demo

Try animating your own drawing Here!

Technical Posts

Dataset of 180,000 annotated drawings: A new, unique AI dataset for animating amateur drawings
Meta AI Technical post: Using AI to bring children’s drawings to life
Meta AI interview: Meet the researcher using AI to build children’s animations

Dataset

We have open sourced our dataset of ~180,000 annotated drawings, which you can explore for your own research.

Website

Go to here

Video

Citation

@article{10.1145/3592788,
author = {Smith, Harrison Jesse and Zheng, Qingyuan and Li, Yifei and Jain, Somya and Hodgins, Jessica K.},
title = {A Method for Animating Children’s Drawings of the Human Figure},
year = {2023},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3592788},
doi = {10.1145/3592788},
abstract = {Children’s drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children’s drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.},
note = {Just Accepted},
journal = {ACM Trans. Graph.},
month = {apr}
}

Acknowledgement

We would like to thank FAIR Interfaces, FAIR X, and other members of Meta who helped in the building and release of the demo

Keywords

graphics, animation, image manipulation, machine learning, drawings, children drawings, non-photorealistic rendering, line drawing, computer graphics, dataset, image dataset

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact

Fri, 01 Apr 2022 00:00:00 +0000

Keywords

diffcloth, projective dynamics, jacobi solver, cloth simulation, differentiable simulation, physics simulation, inverse problems

Code 💻

Follow our Github repository Diffcloth

Demos

For details refer to the paper Sec 6. Applications.

Hat: Trajectory Optimization

Optimize manipulator end effector trajectories to move the hat onto the head.
1737 Dof, h=1/100s, 400 Timesteps, 18 Design Parameters

Sock: Trajectory Optimization

Optimize manipulator end effector trajectories to put on the sock.
3165 Dof, h=1/160s, 400 Timesteps, 36 Design Parameters

Hat-Controller: Closed-Loop Control

We train a generalizable closed-loop controller that can put on the hat from different initial positions.
1737 Dof, h=1/100s, 400 Timesteps, 117126 Design Parameters

Dress: Inverse Design

Optimize dress material parameters so that the spinning angle of the dress is 50 degrees.
10902 Dof, h=1/120s, 125 Timesteps, 2 Design Parameters

Sphere: System Identification

Optimize the fricitonal coefficient between the sphere and the cloth to match target trajectory.
1875 Dof, h=1/180s, 350 Timesteps, 1 Design Parameters

T-shirt: System Identification

Optimize wind model and cloth material parameters to match target trajectory.
4278 Dof, h=1/90s, 250 Timesteps, 6 Design Parameters

Demo

Presentation

Citation

@article{li2022diffcloth,
author = {Li, Yifei and Du, Tao and Wu, Kui and Xu, Jie and Matusik, Wojciech},
title = {DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact},
year = {2022},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {0730-0301},
url = {https://doi.org/10.1145/3527660},
doi = {10.1145/3527660},
abstract = {Cloth simulation has wide applications in computer animation, garment design, and robot-assisted dressing. This work presents a differentiable cloth simulator whose additional gradient information facilitates cloth-related applications. Our differentiable simulator extends a state-of-the-art cloth simulator based on Projective Dynamics (PD) and with dry frictional contact&nbsp;[Ly et&nbsp;al. 2020]},
note = {Just Accepted},
journal = {ACM Trans. Graph.},
month = {mar},
keywords = {cloth simulation, differentiable simulation, Projective Dynamics}
}

Acknowledgement

We thank Marco Renedo for his helpful discussions on the preconditioners, Junbang Liang for his help with running the baseline comparison code, and the anonymous reviewers for their helpful comments. This work was supported in part by the Defense Advanced Research Projects Agency (DARPA) under grant No. FA8750-20-C-0075.

2 | Yifei Li

A Method for Automatically Animating Children’s Drawings of the Human Figure

Code 💻

Demo

Technical Posts

Dataset

Website

Video

Citation

Acknowledgement

Keywords

DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact

Keywords

Code 💻

Demos

Hat: Trajectory Optimization

Sock: Trajectory Optimization

Hat-Controller: Closed-Loop Control

Dress: Inverse Design

Sphere: System Identification

T-shirt: System Identification

Demo

Presentation

Citation

Acknowledgement