KO
|
EN
gitlite — search
Search
#javascript
#python
#hacktoberfest
#react
#ai
#typescript
#llm
#go
#golang
#android
#machine-learning
#rust
#deep-learning
#linux
dance_now
★ 8
Open GitHub ↗
Everybody Dance Now Anime Avatar version
Download README (.md)
Explore Similar Repositories
git-commands
:
Commandes pricincipales de git
extract_keyword_from_word2excel
:
1. 给定关键字；2. 在多个word中提取含有特定关键字的段落等信息；3. 汇总整理成excel。
Refrigerator_Magnet_Clock
:
No description available.
SAFpy
:
Python binding for Spatial Audio Framework (SAF)
analog-circuit-fault-diagnosis-based-on-deep-autoencoder-
:
analog circuit fault-diagnosis-based-on-deep-autoencoder-
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
dance_now
?
Download (.md)
# dance_now # Everybody Dance Now Anime Avatar version ### Context Following the "[Everybody Dance Now](https://github.com/topics/everybody-dance-now)" topic, we tried to make anime character move as we move. We provided 3 sets of different [Mixamo](https://www.mixamo.com/) characters for this image generation task. A generator-discriminator network was built for training anime image generation from key points for each character. The network was trained on each pair of the image and the corresponding keypoints o of the character's pose. Each keypoint set should be able to generate the image of the specific anime character with the same pose. Hope you enjoy teaching these characters to dance! <p align="center"> <img src="image/GAN AniReality.png", width="720"> </p> ### Dataset Data can be downloaded from [kaggle](https://www.kaggle.com/littlewayshuang/skeleton-to-anime-gan) The images were fetched from Mixamo and resized to 512x512 size. There are more than 4000 images for each character with different poses. The corresponding keypoints (facial features, joints, ...) were generated by [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose): 1. 70 facial keypoints 2. 25 body keypoints 3. 21 left-hand keypoints 4. 21 right-hand keypoints Specific keypoint assignment can be found at [openpose doc file](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/output.md) ### Usage #### Training: Enter directory: "dance_now/train_scripts/" and follow the order of the notebooks: * 0.make_real_head.py: Crop 64x64 head image from given anime image according to skeleton * 1.train_bodystick.py: train stick to body generation with multiple GPUs and save the model * 2.cut_fake_head.py: Crop 64x64 head image from generated anime image according to skeleton * 3_1.train_face_residue.py: Train the face residue enhancement process from raw generated face to the precise given anime face. #### Inference: In the root directory: The pre-trained NN models can be found in [Drive](http://) "inference_openpose.py" can be imported with NN models preloaded. "try_preload.py" was provided for testing inference. (Further Model discribed in https://hackmd.io/@usmile/Bkn9Vz-7P) ### Acknowledgements Thanks for the opportunity provided by [Taiwan AI Academy](https://en.aiacademy.tw/) for us to form a study group of GAN. Thanks to Eric for inviting all the members to the group. Thanks to all the members: [Cloud](https://github.com/cloud-yun),[Eric](https://github.com/eric07109), [SeanLin](https://github.com/SeanLinH), [ShuYu](https://github.com/ShuYuHuang), and TA of the GAN study group. ### Inspiration [Everybody Dance Now](https://github.com/topics/everybody-dance-now) is a topic in Github challenging for transferring ones' movement to another one. Inspired by this project, we proposed to automatically replace someone in the photo with the specific anime character with the same pose. ### Reference 1. Chan, C., Ginosar, S., Zhou, T., & Efros, A. A. (2019). Everybody dance now. In Proceedings of the IEEE International Conference on Computer Vision (pp. 5933-5942). 2. Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1125-1134). 3. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., … Chintala, S. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d extquotesingle Alch'e-Buc, E. Fox, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 32 (pp. 8024–8035). Curran Associates, Inc. 4. Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2018). OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. arXiv preprint arXiv:1812.08008. 5. Xu, N., Price, B., Cohen, S., & Huang, T. (2017). Deep image matting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2970-2979). 6. Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV) (pp. 801-818). 7. Wang, T. C., Liu, M. Y., Zhu, J. Y., Tao, A., Kautz, J., & Catanzaro, B. (2018). High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8798-8807).
